Storybook Test Runner

Storybook test runner turns all of your stories into executable tests.

Table of Contents

• Features
• Getting started
• CLI Options
• Configuration
• Running against a deployed Storybook
  • Stories.json mode
• Running in CI
  • 1. Running against deployed Storybooks on Github Actions deployment
  • 2. Running against locally built Storybooks in CI
• Experimental test hook API
  • Image snapshot recipe
  • Render lifecycle
• Troubleshooting
  • The test runner seems flaky and keeps timing out
  • Adding the test runner to other CI environments
• Future work

Features

• ⚡️ Zero config setup
• 💨 Smoke test all stories
• ▶️ Test stories with play functions
• 🏃 Test your stories in parallel in a headless browser
• 👷 Get feedback from error with a link directly to the story
• 🐛 Debug them visually and interactively in a live browser with addon-interactions
• 🎭 Powered by Jest and Playwright
• 👀 Watch mode, filters, and the conveniences you'd expect

Getting started

1. Install the test runner and the interactions addon in Storybook:

yarn add @storybook/test-runner -D

Jest is a peer dependency. If you don't have it, also install it

yarn add jest -D

yarn add @storybook/addon-interactions @storybook/jest @storybook/testing-library -D

module.exports = {
  stories: ['@storybook/addon-interactions'],
  features: {
    interactionsDebugger: true,
  },
};

2. Add a test-storybook script to your package.json

{
  "scripts": {
    "test-storybook": "test-storybook"
  }
}

3. Run Storybook (the test runner runs against a running Storybook instance):

yarn storybook

4. Run the test runner:

yarn test-storybook

NOTE: The runner assumes that your Storybook is running on port 6006. If you're running Storybook in another port, either use --url or set the TARGET_URL before running your command like:

yarn test-storybook --url http://localhost:9009
or
TARGET_URL=http://localhost:9009 yarn test-storybook

CLI Options

Usage: test-storybook [options]

Configuration

The test runner is based on Jest and will accept the CLI options that Jest does, like --watch, --maxWorkers, etc.

The test runner works out of the box, but if you want better control over its configuration, you can run test-storybook --eject to create a local test-runner-jest.config.js file in the root folder of your project, which will be used by the test runner.

The test runner uses jest-playwright and you can pass testEnvironmentOptions to further configure it, such as how it's done above to run tests against all browsers instead of just chromium. For this you must eject the test runner configuration.

Running against a deployed Storybook

By default, the test runner assumes that you're running it against a locally served Storybook on port 6006. If you want to define a target url so it runs against deployed Storybooks, you can do so by passing the TARGET_URL environment variable:

TARGET_URL=https://the-storybook-url-here.com yarn test-storybook

Or by using the --url flag:

yarn test-storybook --url https://the-storybook-url-here.com

Stories.json mode

By default, the test runner transforms your story files into tests. It also supports a secondary "stories.json mode" which runs directly against your Storybook's stories.json, a static index of all the stories.

This is particularly useful for running against a deployed storybook because stories.json is guaranteed to be in sync with the Storybook you are testing. In the default, story file-based mode, your local story files may be out of sync--or you might not even have access to the source code.

To run in stories.json mode, first make sure your Storybook has a v3 stories.json file. You can navigate to:

https://your-storybook-url-here.com/stories.json

It should be a JSON file and the first key should be "v": 3 followed by a key called "stories" containing a map of story IDs to JSON objects.

If your Storybook does not have a stories.json file, you can generate one provided:

• You are running SB6.4 or above
• You are not using storiesOf stories

To enable stories.json in your Storybook, set the buildStoriesJson feature flag in .storybook/main.js:

module.exports = {
  features: { buildStoriesJson: true },
};

Once you have a valid stories.json file, your Storybook will be compatible with the "stories.json mode".

By default, the test runner will detect whether your Storybook URL is local or remote, and if it is remote, it will run in "stories.json mode" automatically. To disable it, you can pass the --no-stories-json flag:

yarn test-storybook --no-stories-json

If you are running tests against a local Storybook but for some reason want to run in "stories.json mode", you can pass the --stories-json flag:

yarn test-storybook --stories-json

NOTE: stories.json mode is not compatible with watch mode.

Running in CI

If you want to add the test-runner to CI, there are a couple of ways to do so:

1. Running against deployed Storybooks on Github Actions deployment

On Github actions, once services like Vercel, Netlify and others do deployment runs, they follow a pattern of emitting a deployment_status event containing the newly generated URL under deployment_status.target_url. You can use that URL and set it as TARGET_URL for the test-runner.

Here's an example of an action to run tests based on that:

name: Storybook Tests
on: deployment_status
jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    if: github.event.deployment_status.state == 'success'
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v2
        with:
          node-version: '14.x'
      - name: Install dependencies
        run: yarn
      - name: Run Storybook tests
        run: yarn test-storybook
        env:
          TARGET_URL: '${{ github.event.deployment_status.target_url }}'

NOTE: If you're running the test-runner against a TARGET_URL of a remotely deployed Storybook (e.g. Chromatic), make sure that the URL loads a publicly available Storybook. Does it load correctly when opened in incognito mode on your browser? If your deployed Storybook is private and has authentication layers, the test-runner will hit them and thus not be able to access your stories. If that is the case, use the next option instead.

2. Running against locally built Storybooks in CI

In order to build and run tests against your Storybook in CI, you might need to use a combination of commands involving the concurrently, http-server and wait-on libraries. Here's a recipe that does the following: Storybook is built and served locally, and once it is ready, the test runner will run against it.

{
  "test-storybook:ci": "concurrently -k -s first -n \"SB,TEST\" -c \"magenta,blue\" \"yarn build-storybook --quiet && npx http-server storybook-static --port 6006 --silent\" \"wait-on tcp:6006 && yarn test-storybook\""
}

And then you can essentially run test-storybook:ci in your CI:

name: Storybook Tests
on: push
jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v2
        with:
          node-version: '14.x'
      - name: Install dependencies
        run: yarn
      - name: Run Storybook tests
        run: yarn test-storybook:ci

NOTE: Building Storybook locally makes it simple to test Storybooks that could be available remotely, but are under authentication layers. If you also deploy your Storybooks somewhere (e.g. Chromatic, Vercel, etc.), the Storybook URL can still be useful with the test-runner. You can pass it to the REFERENCE_URL environment variable when running the test-storybook command, and if a story fails, the test-runner will provide a helpful message with the link to the story in your published Storybook instead.

Experimental test hook API

The test runner renders a story and executes its play function if one exists. However, there are certain behaviors that are not possible to achieve via the play function, which executes in the browser. For example, if you want the test runner to take visual snapshots for you, this is something that is possible via Playwright/Jest, but must be executed in Node.

To enable use cases like visual or DOM snapshots, the test runner exports test hooks that can be overridden globally. These hooks give you access to the test lifecycle before and after the story is rendered.

There are three hooks: setup, preRender, and postRender. setup executes once before all the tests run. preRender and postRender execute within a test before and after a story is rendered.

The render functions are async functions that receive a Playwright Page and a context object with the current story id, title, and name. They are globally settable by @storybook/test-runner's setPreRender and setPostRender APIs.

All three functions can be set up in the configuration file .storybook/test-runner.js which can optionally export any of these functions.

NOTE: These test hooks are experimental and may be subject to breaking changes. We encourage you to test as much as possible within the story's play function.

Image snapshot recipe

Consider, for example, the following recipe to take image snapshots:

// .storybook/test-runner.js
const { toMatchImageSnapshot } = require('jest-image-snapshot');

const customSnapshotsDir = `${process.cwd()}/__snapshots__`;

module.exports = {
  setup() {
    expect.extend({ toMatchImageSnapshot });
  },
  async postRender(page, context) {
    const image = await page.screenshot();
    expect(image).toMatchImageSnapshot({
      customSnapshotsDir,
      customSnapshotIdentifier: context.id,
    });
  },
};

There is also an exported TestRunnerConfig type available for TypeScript users.

Render lifecycle

To visualize the test lifecycle, consider a simplified version of the test code automatically generated for each story in your Storybook:

it('button--basic', async () => {
  // filled in with data for the current story
  const context = { id: 'button--basic', title: 'Button', name: 'Basic' };

  // playwright page https://playwright.dev/docs/pages
  await page.goto(STORYBOOK_URL);

  // pre-render hook
  if (preRender) await preRender(page, context);

  // render the story and run its play function (if applicable)
  await page.execute('render', context);

  // post-render hook
  if (postRender) await postRender(page, context);
});

Troubleshooting

The test runner seems flaky and keeps timing out

If your tests are timing out with Timeout - Async callback was not invoked within the 15000 ms timeout specified by jest.setTimeout, it might be that playwright couldn't handle to test the amount of stories you have in your project. Maybe you have a large amount of stories or your CI has a really low RAM configuration.

In either way, to fix it you should limit the amount of workers that run in parallel by passing the --maxWorkers option to your command:

{
  "test-storybook:ci": "concurrently -k -s first -n \"SB,TEST\" -c \"magenta,blue\" \"yarn build-storybook --quiet && npx http-server storybook-static --port 6006 --silent\" \"wait-on tcp:6006 && yarn test-storybook --maxWorkers=2\""
}

Adding the test runner to other CI environments

As the test runner is based on playwright, depending on your CI setup you might need to use specific docker images or other configuration. In that case, you can refer to the Playwright CI docs for more information.

Future work

Future plans involve adding support for the following features:

• 🧪 Custom test functions
• 📄 Run addon reports