README
stacks
Simple, reliable request/reply-based parallel computing over TCP.
What is stacks?
Stacks provides a simple request/reply interface to parallel computing. It's simply a queue of requests that clients (seperate worker machines) reply to. It's designed to be reliable and easy-to-use in situations involving long, compute-heavy requests.
Features
- JSON, BSON, binary support β msgpack protocol
- Automatic round-robin scheduling
- Automatic rescheduling of failed requests
- Typed requests.
client.on(type, function(body, reply) { ...
interface - Configurable per-request-type worker concurrency limits
- Supports zero or more client workers
- Automatically saves requests received when no worker is available, and schedules them when the next worker connects
- Automatically reschedules requests that were pending on clients that disconnect
Installation
To begin using stacks, you must have at least node v0.10 or higher installed.
Install from npm
$ (sudo) npm install --save stacks
And add it to your project
var stacks = require('stacks');
...
making requests
The master node makes requests to stacks. The requests are automatically handled by clients.
var stacks = require('stacks');
// Listen for clients on port 3000
stacks.listen(3000);
stacks.request('string reverse', 'hello', { maxRetries: 5 }, function (error, reversedString) {
if (error) {
console.error('Failed to reverse string after 5 retries. Error:', error);
} else {
console.log('Reversed string:', reversedString); // Reversed string: olleh
}
});
handling requests
var stacks = require('stacks');
// connect to stacks server
stacks.connect(3000, function(client) {
client.on('string reverse', function(string, reply) {
var reversedString = string.split("").reverse().join("");
var error = null;
reply(error, reversedString);
}, 15); // handle only 15 concurrent string reverse requests at a time
});
Server API
listen(port, [callback])
Listen for client (worker) connections
- Number
port
: Port to listen for client connections on - Function
callback
: Callback function invoked when stacks has begun listening. Takes no arguments
request(type, body, [options], callback)
Make a request on stacks
- String
type
: Type of request. Used in client interface to process/reply i.e.,client.on(type, function(body, reply) {...
- Mixed
body
: The request body that will be received by the client request worker.WARNING:
Arrays will be parsed as arguments and cannot be sent as the top-level object in a request. - Object
options
: Request options. - Function
callback
: Callback function invoked when request completes (or fails after options.maxRetries). Takes argumentserror
,response
.
request options
- Number
maxRetries
(3): The maximum amount of retries to attempt before invoking request callback witherror
. (These retries are rescheduled by stacks (round-robin if multiple clients are connected)).
Client API
connect(server, callback)
Connect to stacks server
- Mixed
server
: If Number, server is interpreted as port. If Object, server must have the signature{ host: '[host-ip-address]', port: 3000 }
- Function
callback
: Invoked after client is connected to stacks server. Takes argumentclient
.
disconnect()
Disconnect from stacks server
Client
Instance methods of the client object passed to the #connect() callback.
on(type, worker, [concurrency])
Handle and reply to requests of type type
- String
type
: Type of request to handle withconcurrency
concurrentworker
functions - Function
worker
: Function that processes and replies to a request. Takes argumentsbody
β the request body βandreply
β the reply function. - Number
concurrency
(1): Limit of concurrentworker
functions to use for requests of typetype
.
License
MIT