@naturalcycles/abba

AB test assignment configuration tool for Node.js

Usage no npm install needed!

<script type="module">
  import naturalcyclesAbba from 'https://cdn.skypack.dev/@naturalcycles/abba';
</script>

README


ABBA

A tool for generating and persisting AB test assignments

Table of Contents
  1. Concepts
  2. Getting Started
  3. Usage
  4. Segmentation

Concepts

  • Experiment: An individual experiment that will test a hypothesis
  • Segmentation: The target audience for the experiment
  • Sampling: Restrictions on what proportion of the target audience will be involved in the experiment
  • Bucket: An allocation that defines what variant of a particular experience the user will have

Built With

(back to top)

Getting Started

Prerequisites

  • A running MYSQL instance

Installation

Below is an example of how you can instruct your audience on installing and setting up your app. This template doesn't rely on any external dependencies or services.

  1. Install NPM packages

    yarn add @naturalcyles/abba

or

npm install @naturalcyles/abba

  2. Execute the sql script found here to generate the required DB Schema

(back to top)

Usage

Create an instance of Abba

Creates an instance of Abba. You can pass in the database url in the constructor. If it does not exist it will fallback to trying to use the ABBA_DB_URL which must be added to your environment variables.

const abba = new Abba('url')
// or reading from process.env.ABBA_DB_URL
const abba = new Abba()

Create a new experiment

Creates a new experiment

async createExperiment(
    input: ExperimentInput,
    buckets: BucketInput[]
): Promise<Experiment>

Update an experiment

Updates an existing experiment.

async updateExperiment(
    id: number,
    input: ExperimentInput,
    buckets: BucketInput[]
): Promise<Experiment>

Delete an experiment

Delete an experiment. Removes all users assignments and buckets

async deleteExperiment(
    id: number
): Promise<void>

Get all existing user assignments

Gets all existing user assignments

async getAllExistingUserAssignments(
    userId: string
): Promise<UserAssignment[]>

Get a users assignment

Get an assignment for a given user. If existingOnly is false, it will attempt generate a new assignment. segmentationData becomse required when existingOnly is false

async getUserAssignment(
  experimentId: number,
  userId: string,
  existingOnly: boolean,
  segmentationData?: SegmentationData,
): Promise<UserAssignment | null>

Generate user assignments

Generate user assignments for all active experiments. Will return any existing assignments and attempt to generate new assignments.

async generateUserAssignments(
  userId: string,
  segmentationData: SegmentationData,
): Promise<UserAssignment[]>

Getting assignment statistics

Get assignment statistics for an experiment.

async getExperimentAssignmentStatistics(
  experimentId: number
): Promise<AssignmentStatistics>

(back to top)

Segmentation

Experiments can be configured to target specific audiences using segmentation rules. When generating assignments it is possible to test these rules using user segmentation data which is an object containing key/value pairs unique to each user. (Allowed value types: string, number, boolean). A segmentation rule consist of the following properties:

  key: string, // the key of the corresponding segmentationData property.
  operator: '==' | '!=' | 'semver' | 'regex' | 'boolean', // the operator that will be used to execute the rule
  value: string | number | boolean, // the value the operator will be executed against

Segmentation rule operators

Equals (==)

Rule:

  { key: 'country', operator: '==', value: 'SE }

Example segmentation data:

{
  country: 'SE', // valid
  country: 'NO' // not valid
}

Not equals (!=)

Rule:

  { key: 'country', operator: '!=', value: 'SE' }

Example segmentation data:

{
  country: 'NO', // valid
  country: 'SE' // not valid
}

Boolean (boolean)

Rule:

  { key: 'isEligible', operator: 'boolean', value: true }

Example segmentation data:

{
  isEligible: true, // valid
  isEligible: false // not valid
}

Semver (semver)

Rule:

  { key: 'appVersion', operator: 'semver', value: '>1.1.0' }

Example segmentation data:

{
  appVersion: '1.2.0', // valid
  appVersion: '1' // not valid
}

Regex (regex)

Rule:

  { key: 'country', operator: 'regex', value: 'SE|NO' }

Example segmentation data:

{
  country: 'SE', // valid
  country: 'NO', // valid
  country: 'GB' // not valid
}

(back to top)