Testing

Declarative test framework

Install: @travetto/test

npm install @travetto/test

This module provides unit testing functionality that integrates with the framework. It is a declarative framework, using decorators to define tests and suites. The test produces results in the following formats:

• TAP 13, default and human-readable
• JSON, best for integrating with at a code level
• xUnit, standard format for CI/CD systems e.g. Jenkins, Bamboo, etc.

Note: All tests should be under the test/.* folders. The pattern for tests is defined as a regex and not standard globbing.

Definition

A test suite is a collection of individual tests. All test suites are classes with the @Suite decorator. Tests are defined as methods on the suite class, using the @Test decorator. All tests intrinsically support async/await.

A simple example would be:

Code: Example Test Suite

import * as assert from 'assert';

import { Suite, Test } from '@travetto/test';

@Suite()
class SimpleTest {

  #complexService: {
    doLongOp(): Promise<number>;
    getText(): string;
  };

  @Test()
  async test1() {
    const val = await this.#complexService.doLongOp();
    assert(val === 5);
  }

  @Test()
  test2() {
    const text = this.#complexService.getText();
    assert(/abc/.test(text));
  }
}

Assertions

A common aspect of the tests themselves are the assertions that are made. Node provides a built-in assert library. The framework uses AST transformations to modify the assertions to provide integration with the test module, and to provide a much higher level of detail in the failed assertions. For example:

Code: Example assertion for deep comparison

import * as assert from 'assert';

import { Suite, Test } from '@travetto/test';

@Suite()
class SimpleTest {

  @Test()
  async test() {
    // @ts-expect-error
    assert({ size: 20, address: { state: 'VA' } } === {});
  }
}

would translate to:

Code: Transpiled test Code

"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
const tslib_1 = require("tslib");
const ᚕ_check_1 = require("@travetto/test/src/assert/check");
const ᚕ_decorator_1 = require("@travetto/registry/src/decorator");
const assert = require("assert");
const test_1 = require("@travetto/test");
let SimpleTest = class SimpleTest {
    static ᚕinit = ᚕ_decorator_1.Register.initMeta(SimpleTest, ᚕsrc(__filename), 442873266, { test: { hash: 1426090177 } }, false, false);
    async test() {
        ᚕ_check_1.AssertCheck.check({ file: ᚕsrc(__filename), line: 11, text: "{ size: 20, address: { state: 'VA' } } === {}", operator: "deepStrictEqual" }, true, { size: 20, address: { state: 'VA' } }, {});
    }
};
(0, tslib_1.__decorate)([
    (0, test_1.Test)({ lines: { start: 8, end: 12, codeStart: 11 } })
], SimpleTest.prototype, "test", null);
SimpleTest = (0, tslib_1.__decorate)([
    ᚕ_decorator_1.Register(),
    (0, test_1.Suite)({ lines: { start: 5, end: 13 } })
], SimpleTest);
Object.defineProperty(exports, 'ᚕtrv', { configurable: true, value: true });

This would ultimately produce the error like:

Code: Sample Validation Error

AssertionError(
  message="{size: 20, address: {state: 'VA' }} should deeply strictly equal {}"
)

The equivalences for all of the assert operations are:

• assert(a == b) as assert.equal(a, b)
• assert(a !== b) as assert.notEqual(a, b)
• assert(a === b) as assert.strictEqual(a, b)
• assert(a !== b) as assert.notStrictEqual(a, b)
• assert(a >= b) as assert.greaterThanEqual(a, b)
• assert(a > b) as assert.greaterThan(a, b)
• assert(a <= b) as assert.lessThanEqual(a, b)
• assert(a < b) as assert.lessThan(a, b)
• assert(a instanceof b) as assert.instanceOf(a, b)
• assert(a.includes(b)) as assert.ok(a.includes(b))
• assert(/a/.test(b)) as assert.ok(/a/.test(b))

In addition to the standard operations, there is support for throwing/rejecting errors (or the inverse). This is useful for testing error states or ensuring errors do not occur.

• throws/doesNotThrow is for catching synchronous rejections

Code: Throws vs Does Not Throw

import * as assert from 'assert';

import { Suite, Test } from '@travetto/test';

@Suite()
class SimpleTest {

  @Test()
  async testThrows() {
    assert.throws(() => {
      throw new Error();
    });

    assert.doesNotThrow(() => {

      let a = 5;
    });
  }
}

• rejects/doesNotReject is for catching asynchronous rejections

Code: Rejects vs Does Not Reject

import * as assert from 'assert';

import { Suite, Test } from '@travetto/test';

@Suite()
class SimpleTest {

  @Test()
  async testRejects() {
    await assert.rejects(async () => {
      throw new Error();
    });

    await assert.doesNotReject(async () => {

      let a = 5;
    });
  }
}

Additionally, the throws/rejects assertions take in a secondary parameter to allow for specification of the type of error expected. This can be:

• A regular expression or string to match against the error's message
• A class to ensure the returned error is an instance of the class passed in
• A function to allow for whatever custom verification of the error is needed

Code: Example of different Error matching paradigms

import * as assert from 'assert';

import { Suite, Test } from '@travetto/test';

@Suite()
class SimpleTest {

  @Test()
  async errorTypes() {
    assert.throws(() => {
      throw new Error('Big Error');
    }, 'Big Error');

    assert.throws(() => {
      throw new Error('Big Error');
    }, /B.*Error/);

    await assert.rejects(() => {
      throw new Error('Big Error');
    }, Error);

    await assert.rejects(() => {
      throw new Error('Big Error');
    }, (err: Error) =>
      err.message.startsWith('Big') && err.message.length > 4 ? undefined : err
    );
  }
}

Running Tests

To run the tests you can either call the Command Line Interface by invoking

Terminal: Test Help Output

$ trv test --help

Usage:  test [options] [regexes...]

Options:
  -f, --format <format>            Output format for test results (default: "tap")
  -c, --concurrency <concurrency>  Number of tests to run concurrently (default: 4)
  -i, --isolated                   Isolated mode
  -m, --mode <mode>                Test run mode (default: "standard")
  -h, --help                       display help for command

The regexes are the patterns of tests you want to run, and all tests must be found under the test/ folder.

Travetto Plugin

The VSCode plugin also supports test running, which will provide even more functionality for real-time testing and debugging.

Additional Considerations

During the test execution, a few things additionally happen that should be helpful. The primary addition, is that all console output is captured, and will be exposed in the test output. This allows for investigation at a later point in time by analyzing the output.

Like output, all promises are also intercepted. This allows the code to ensure that all promises have been resolved before completing the test. Any uncompleted promises will automatically trigger an error state and fail the test.