stacks

Simple, reliable request/reply-based distributed computing over TCP.

Usage no npm install needed!

<script type="module">
  import stacks from 'https://cdn.skypack.dev/stacks';
</script>

README

stacks

Simple, reliable request/reply-based parallel computing over TCP.

Build Status

What is stacks?

Stacks provides a simple request/reply interface to parallel computing. It's simply a queue of requests that clients (seperate worker machines) reply to. It's designed to be reliable and easy-to-use in situations involving long, compute-heavy requests.

Features

  • JSON, BSON, binary support – msgpack protocol
  • Automatic round-robin scheduling
  • Automatic rescheduling of failed requests
  • Typed requests. client.on(type, function(body, reply) { ... interface
  • Configurable per-request-type worker concurrency limits
  • Supports zero or more client workers
  • Automatically saves requests received when no worker is available, and schedules them when the next worker connects
  • Automatically reschedules requests that were pending on clients that disconnect

Installation

To begin using stacks, you must have at least node v0.10 or higher installed.

Install from npm

$ (sudo) npm install --save stacks

And add it to your project

var stacks = require('stacks');
...

making requests

The master node makes requests to stacks. The requests are automatically handled by clients.

var stacks = require('stacks');
// Listen for clients on port 3000
stacks.listen(3000);

stacks.request('string reverse', 'hello', { maxRetries: 5 }, function (error, reversedString) {
  if (error) {
    console.error('Failed to reverse string after 5 retries. Error:', error);
  } else {
    console.log('Reversed string:', reversedString); // Reversed string: olleh
  }
});

handling requests

var stacks = require('stacks');
// connect to stacks server
stacks.connect(3000, function(client) {

  client.on('string reverse', function(string, reply) {
    var reversedString = string.split("").reverse().join("");
    var error = null;
    reply(error, reversedString);
  }, 15); // handle only 15 concurrent string reverse requests at a time
  
});

Server API

listen(port, [callback])

Listen for client (worker) connections

  • Number port: Port to listen for client connections on
  • Function callback: Callback function invoked when stacks has begun listening. Takes no arguments

request(type, body, [options], callback)

Make a request on stacks

  • String type: Type of request. Used in client interface to process/reply i.e., client.on(type, function(body, reply) {...
  • Mixed body: The request body that will be received by the client request worker. WARNING: Arrays will be parsed as arguments and cannot be sent as the top-level object in a request.
  • Object options: Request options.
  • Function callback: Callback function invoked when request completes (or fails after options.maxRetries). Takes arguments error, response.

request options

  • Number maxRetries (3): The maximum amount of retries to attempt before invoking request callback with error. (These retries are rescheduled by stacks (round-robin if multiple clients are connected)).

Client API

connect(server, callback)

Connect to stacks server

  • Mixed server: If Number, server is interpreted as port. If Object, server must have the signature { host: '[host-ip-address]', port: 3000 }
  • Function callback: Invoked after client is connected to stacks server. Takes argument client.

disconnect()

Disconnect from stacks server

Client

Instance methods of the client object passed to the #connect() callback.

on(type, worker, [concurrency])

Handle and reply to requests of type type

  • String type: Type of request to handle with concurrency concurrent worker functions
  • Function worker: Function that processes and replies to a request. Takes arguments body– the request body –and reply– the reply function.
  • Number concurrency (1): Limit of concurrent worker functions to use for requests of type type.

License

MIT