README
waitless-async-benchmarks
A series of benchmarks comparing the performance of different asynchronous processing libraries and styles in JavaScript.
Sequential pipe
This benchmark tests a series of asynchronous operations (or functional steps) whereby the output result of each step is input to a next step in sequential fashion. Our test case is a simple arithmetic calculation that evaluates 2**5 over 5 steps using the Node's setImmediate function and asserts that the result is 32.
Implementations of this benchmark in standard JavaScript idioms (including the infamous "pyramid of doom" or "callback hell" style, promises & async/await) and JS libraries are compared in a Node.js v10.15.2 test environment running on a 4-core Linux box.
Tests using JS nested callback style as a standard reference
Background reading and references
| Test(s) | |
|---|---|
| A, B, C | Introducing async JavaScript draft on MDN |
| D |
async npm package - waterfall function
|
| E | A functional compose/reduce pipe method popularised by Eric Elliot in numerous tweets on Twitter |
| F | waitless npm package - pipe function using waitless' Immediate or Promise strategy with accurate static typing |
| G | waitless npm package - pipe function using wailess' Continuation Passing Style strategy with accurate static typing |
TypeScript code: sequential pipe benchmark
import * as assert from 'assert';
import * as asyn from 'async';
import * as bench from 'benchmark';
import * as waitless from 'waitless';
/** Just for the benefit of the *Benchmark.js* benchmarking package. */
type Deferred = { resolve(): void };
/** Creates a Promise that is resolved by Node's `setImmediate` function. */
function setImmediatePromise<T>(arg?: T): Promise<T> {
return new Promise((resolve) => setImmediate((arg?: T) => resolve(arg), arg));
}
/**
* $_JS_nested_callback_style
*
* Asynchronous pipe implemented in classic JavaScript nested callback function
* style (aka "pyramid of doom" or "callback hell").
*
* @param {Deferred} deferred - needed by Benchmark.js to signal when the async
* test is complete
*/
function $_JS_nested_callback_style(deferred: Deferred) {
setImmediate(
(a) =>
setImmediate(
(b) =>
setImmediate(
(c) =>
setImmediate(
(d) =>
setImmediate(
(e) =>
setImmediate((f) => {
assert.equal(f, 32);
deferred.resolve();
}, e * 2),
d * 2
),
c * 2
),
b * 2
),
a * 2
),
1
);
}
/**
* A_ES2015_promise_then_chain
*
* Asynchronous pipe implemented by way of an ES2015 promise.then chain.
*
* @param {Deferred} deferred - needed by Benchmark.js to signal when the async
* test is complete
*/
function A_ES2015_promise_then_chain(deferred: Deferred) {
let prom = setImmediatePromise(1)
.then((a) => setImmediatePromise(a * 2))
.then((b) => setImmediatePromise(b * 2))
.then((c) => setImmediatePromise(c * 2))
.then((d) => setImmediatePromise(d * 2))
.then((e) => setImmediatePromise(e * 2));
prom.then((res) => {
assert.equal(res, 32);
deferred.resolve();
});
}
/**
* B_ES2017_async_with_inline_await
*
* Asynchronous pipe implemented as an ES2017 async function with inline await
* statements.
*
* @param {Deferred} deferred - needed by Benchmark.js to signal when the async
* test is complete
*/
async function B_ES2017_async_with_inline_await(deferred: Deferred) {
let a = await setImmediatePromise(1);
let b = await setImmediatePromise(a * 2);
let c = await setImmediatePromise(b * 2);
let d = await setImmediatePromise(c * 2);
let e = await setImmediatePromise(d * 2);
let f = await setImmediatePromise(e * 2);
assert.equal(f, 32);
deferred.resolve();
}
/**
* C_async_library_waterfall_function
*
* Asynchronous pipe implemented using the `waterfall` function from the *async*
* npm package.
*
* @param {Deferred} deferred - needed by Benchmark.js to signal when the async
* test is complete
*/
function C_async_library_waterfall_function(deferred: Deferred) {
asyn.waterfall(
[
(callback: any) => setImmediate(() => callback(null, 1)),
(a: any, callback: any) => setImmediate(() => callback(null, a * 2)),
(b: any, callback: any) => setImmediate(() => callback(null, b * 2)),
(c: any, callback: any) => setImmediate(() => callback(null, c * 2)),
(d: any, callback: any) => setImmediate(() => callback(null, d * 2)),
(e: any, callback: any) => setImmediate(() => callback(null, e * 2))
],
(err, f: any) => {
assert.equal(f, 32);
deferred.resolve();
}
);
}
/**
* D_Async_pipe_by_functional_reduction
*
* Asynchronous pipe implemented using Eric Elliot's async pipe by functional
* reduction/composition.
*
* @param {Deferred} deferred - needed by Benchmark.js to signal when the async
* test is complete
*
* @see https://twitter.com/_ericelliott/status/895791749334945792
*/
function D_Async_pipe_by_functional_reduction(deferred: Deferred) {
let prom = mypipeD(1);
prom.then((f: any) => {
assert.equal(f, 32);
deferred.resolve();
});
}
const EricElliot_async_pipe = (...fns: any[]) => (x: any) =>
fns.reduce(async (y, f) => f(await y), x);
const mypipeD = EricElliot_async_pipe(
(a: any) => setImmediatePromise(a * 2),
(b: any) => setImmediatePromise(b * 2),
(c: any) => setImmediatePromise(c * 2),
(d: any) => setImmediatePromise(d * 2),
(e: any) => setImmediatePromise(e * 2)
);
/**
* E_waitless_library_IOP_pipe
*
* Asynchronous pipe implemented using the `iop.pipe` function from the
* *waitless* npm package.
*
* @param {Deferred} deferred - needed by Benchmark.js to signal when the async
* test is complete
*/
function E_waitless_library_IOP_pipe(deferred: Deferred) {
let iop = mypipeE(1);
Promise.resolve(iop).then((f) => {
assert.equal(f, 32);
deferred.resolve();
});
}
const mypipeE = waitless.iop.pipe(
(a: number) => setImmediatePromise(a * 2),
(b) => setImmediatePromise(b * 2),
(c) => setImmediatePromise(c * 2),
(d) => setImmediatePromise(d * 2),
(e) => setImmediatePromise(e * 2)
);
/**
* F_waitless_library_CPS_pipe
*
* Asynchronous pipe implemented using the `cps.pipe` function from the
* *waitless* npm package.
*
* @param {Deferred} deferred - needed by Benchmark.js to signal when the async
* test is complete
*/
function F_waitless_library_CPS_pipe(deferred: Deferred) {
mypipeF((f) => {
assert.equal(f, 32);
deferred.resolve();
}, 1);
}
const mypipeF = waitless.cps.pipe(
(ret: Tv.Async.Retrn<number>, a: number) => setImmediate(ret, a * 2),
(ret: Tv.Async.Retrn<number>, b) => setImmediate(ret, b * 2),
(ret: Tv.Async.Retrn<number>, c) => setImmediate(ret, c * 2),
(ret: Tv.Async.Retrn<number>, d) => setImmediate(ret, d * 2),
(ret: Tv.Async.Retrn<number>, e) => setImmediate(ret, e * 2)
);
Sequential cascade
TODO description
Tests using ES2017 async await as a standard reference
TypeScript code: sequential cascase benchmark
import * as crypto from 'crypto';
import * as bench from 'benchmark';
import * as waitless from 'waitless';
const { cps, iop } = waitless;
/** Just for the benefit of the *Benchmark.js* benchmarking package. */
type Deferred = { resolve(): void };
class Context {}
interface User {
name: string;
email: string;
token?: string;
}
const userDB = new Map<string, User>();
userDB.set('foo@example.com', { name: 'Mary Foo', email: 'foo@example.com' });
function prompt(message: string): Promise<string> {
return Promise.resolve('foo@example.com');
}
function promptCPS(ret: Tv.Async.Retrn<string>, message: string) {
ret('foo@example.com');
}
function generateToken(): Promise<string> {
return new Promise((resolve, reject) => {
crypto.randomBytes(20, (err, buf) => {
if (err) {
reject(err);
} else {
let token = buf.toString('hex');
resolve(token);
}
});
});
}
function generateTokenCPS(ret: Tv.Async.Retrn<string>) {
crypto.randomBytes(20, (err, buf) => {
if (err) {
ret('err');
} else {
let token = buf.toString('hex');
ret(token);
}
});
}
function getUser(email: string): Promise<User | undefined> {
return new Promise((resolve) => setImmediate(() => resolve(userDB.get(email))));
}
function getUserCPS(ret: Tv.Async.Retrn<User | undefined>, email: string) {
ret(userDB.get(email));
}
function saveUserToken(user: User, token: string): Promise<void> {
return new Promise((resolve) => {
user.token = token;
setImmediate(() => resolve());
});
}
function saveUserTokenCPS(ret: Tv.Async.Retrn<void>, user: User, token: string) {
user.token = token;
setImmediate(() => ret());
}
function sendHttpResponse(ctx: Context, status: number, body: string): Promise<void> {
return new Promise((resolve) => setImmediate(() => resolve()));
}
function sendHttpResponseCPS(
ret: Tv.Async.Retrn<void>,
ctx: Context,
status: number,
body: string
) {
setImmediate(() => ret());
}
function sendUserEmail(user: User, token: string): Promise<void> {
return new Promise((resolve) => setImmediate(() => resolve()));
}
function sendUserEmailCPS(ret: Tv.Async.Retrn<void>, user: User, token: string) {
setImmediate(() => ret());
}
/**
* $_ES2017_async_await
*
* @param {Deferred} deferred - needed by Benchmark.js to signal when the async
* test is complete
*/
async function $_ES2017_async_await(deferred: Deferred) {
let ctx = new Context();
let email = await prompt('Enter your email address: ');
let user = await getUser(email);
if (user === undefined) {
await sendHttpResponse(
ctx,
200,
`Sorry, there is no user in our database with email address '${email}'`
);
} else {
let token = await generateToken();
await saveUserToken(user, token);
await sendUserEmail(user, token);
await sendHttpResponse(
ctx,
200,
`An email has been sent to '${email}' with instructions to reset your password`
);
}
deferred.resolve();
}
/**
* A_waitless_library_IOP_cascade
*
* @param {Deferred} deferred - needed by Benchmark.js to signal when the async
* test is complete
*/
function A_waitless_library_IOP_cascade(deferred: Deferred) {
let iop = mycascadeA(new Context());
Promise.resolve(iop).then(() => {
deferred.resolve();
});
}
const mycascadeA = iop.cascade(
(ctx: Context) => prompt('Enter your email address: '),
(ctx, email) => getUser(email),
(ctx, email, user) =>
iop.choose(
iop.when(
(user: User | undefined): user is undefined => user === undefined,
() =>
sendHttpResponse(
ctx,
200,
`Sorry, there is no user in our database with email address '${email}'`
)
),
iop.when(
(user: User | undefined): user is User => user !== undefined,
iop.cascade(
(user) => generateToken(),
(user, token) => saveUserToken(user, token),
(user, token) => sendUserEmail(user, token),
() =>
sendHttpResponse(
ctx,
200,
`An email has been sent to '${email}' with instructions to reset your password`
)
)
)
)(user)
);
/**
* B_waitless_library_CPS_cascade
*
* @param {Deferred} deferred - needed by Benchmark.js to signal when the async
* test is complete
*/
function B_waitless_library_CPS_cascade(deferred: Deferred) {
mycascadeB((_) => deferred.resolve(), new Context());
}
const mycascadeB = cps.cascade(
(ret: Tv.Async.Retrn<string>, ctx: Context) => promptCPS(ret, 'Enter your email address: '),
(ret: Tv.Async.Retrn<User | undefined>, ctx, email) => getUserCPS(ret, email),
(ret: Tv.Async.Retrn<any>, ctx, email, user) =>
cps.choose(
cps.when(
(user: User | undefined): user is undefined => user === undefined,
(ret: Tv.Async.Retrn<void>) =>
sendHttpResponseCPS(
ret,
ctx,
200,
`Sorry, there is no user in our database with email address '${email}'`
)
),
cps.when(
(user: User | undefined): user is User => user !== undefined,
cps.cascade(
(ret: Tv.Async.Retrn<string>, user) => generateTokenCPS(ret),
(ret: Tv.Async.Retrn<void>, user, token) => saveUserTokenCPS(ret, user, token),
(ret: Tv.Async.Retrn<void>, user, token) => sendUserEmailCPS(ret, user, token),
(ret: Tv.Async.Retrn<void>) =>
sendHttpResponseCPS(
ret,
ctx,
200,
`An email has been sent to '${email}' with instructions to reset your password`
)
)
)
)(ret, user)
);
Running the benchmarks from the command-line
$ npx waitless-async-benchmarks
Requires:
$ npx waitless-async-benchmarks
npx: installed 7 in 4.727s
Starting waitless-async-benchmarks (2 suites)
Running async_pipe suite (1 of 2)
$_JS_nested_callback_style x 24,967 ops/sec ±1.24% (75 runs sampled)
A_ES2015_promise_then_chain x 19,616 ops/sec ±1.43% (80 runs sampled)
B_ES2017_async_with_inline_await x 17,454 ops/sec ±2.06% (76 runs sampled)
C_async_library_waterfall_function x 17,331 ops/sec ±0.95% (75 runs sampled)
D_Async_pipe_by_functional_reduction x 18,480 ops/sec ±1.05% (77 runs sampled)
E_waitless_library_IOP_pipe x 21,133 ops/sec ±1.36% (79 runs sampled)
F_waitless_library_CPS_pipe x 27,484 ops/sec ±0.87% (81 runs sampled)
Running async_cascade suite (2 of 2)
$_ES2017_async_await x 6,123 ops/sec ±3.31% (67 runs sampled)
A_waitless_library_IOP_cascade x 5,926 ops/sec ±1.84% (75 runs sampled)
B_waitless_library_CPS_cascade x 6,032 ops/sec ±1.42% (81 runs sampled)
$
License
This software is licensed MIT.
Copyright © 2019 Justin Johansson