assistive-playwright-client

assistive-playwright-client is a library that extends playwright to allow end-to-end testing of web applications with a screen reader. It is designed to connect to the assistive-playwright-server component that runs inside a virtual machine that is cloned

Usage no npm install needed!

<script type="module">
  import assistivePlaywrightClient from 'https://cdn.skypack.dev/assistive-playwright-client';
</script>

README

assistive-playwright-client

npm

Presentation

This package contains a node.js library that extends playwright to allow end-to-end testing of web applications with a screen reader (such as NVDA or JAWS) and checking that the screen reader says what is expected.

This requires two main features that are not natively supported by playwright:

  • being able to send keystrokes at a low level so that the screen reader can receive them. This is achieved by using either Virtual Box or QEMU and sending low level events with their API.
  • being able to capture the text read by the screen reader. This is achieved by using text-to-socket-engine.

So, assistive-playwright-client allows to easily clone and start a virtual machine (with the vm-providers component) and it provides access to the following functions (through the assistive-playwright-server component that is supposed to be running inside the virtual machine):

Here is a schema describing the architecture of Assistive-Playwright:

Architecture of Assistive-Playwright

Getting started

  • Make sure you have the following software installed on the host machine:

  • Make sure you have a VirtualBox or QEMU virtual machine properly configured. To configure the virtual machine, you can follow this step-by-step guide. The virtual machine should be configured with:

    • The NVDA or JAWS screen reader
    • text-to-socket-engine and assistive-playwright-server that are configured to work together, with assistive-playwright-server listening on http port 7779
    • A snapshot of the virtual machine should be saved in the running state with all these programs running.
  • Install assistive-playwright-client in your project:

npm install assistive-playwright-client
  • Make sure to start vboxwebsrv in order to be able to start virtual machines of type virtualbox:
vboxwebsrv --authentication null
  • Here is an example gettingStarted.js file that shows how to use assistive-playwright-client:
const { createVM } = require("assistive-playwright-client");

(async () => {
  console.log("Creating VM...");
  const {
    chromium /* can be replaced with firefox or webkit */,
    screenReader,
    calibrateMouse,
    keyboard,
    vm
  } = await createVM({
    vmSettings: {
      type: "virtualbox",
      vm: "win10-chromium-nvda",
      snapshot: "nvda"
    }
  });
  try {
    console.log("Launching browser...");
    const browser = await chromium.launch({ headless: false });
    const page = await browser.newPage({ viewport: null });
    const mouse = await calibrateMouse(page);
    screenReader.on("message", msg => console.log(`sr> ${msg}`));
    await page.goto("https://duckduckgo.com/");
    await mouse.click(0, 0, { origin: await page.$("input[type=text]") });
    await screenReader.waitForMessage("Search the web");
    await keyboard.type("assistive-playwright-client");
    await keyboard.press("Enter");
    await screenReader.waitForMessage(
      "assistive playwright client at Duck Duck Go"
    );
    await keyboard.press("Tab");
    await screenReader.waitForMessage("edit");
  } finally {
    console.log("Destroying VM...");
    await vm.destroy();
    console.log("Done!");
  }
})().catch(error => {
  console.log(`Error: ${error}`);
  process.exit(1);
});

The API documentation is available here