README
socket.io-kafka
How to use
var io = require('socket.io')(3000);
var kafka = require('socket.io-kafka');
io.adapter(kafka('localhost:2181'));
By running socket.io with socket.io-kafka
you can scale
your socket.io app horizontally using multiple ports or servers.
This adapter aims to be an alternative to the socket.io-redis adapter which allow multiple socket.io instances to communicate using Kafka instead of Redis.
Why Kafka?
Kafka is a very fast, scalable, distributed message bus and was designed to handle a huge amount of data with low latency. It was originally developed by LinkedIn and is currently part of the Apache Project.
As Kafka provides built-in partitioning, replication, and fault-tolerance, it feels like a natural choice to build scalable socket.io applications with high throughput or which will require a great number of instances talking amongst themselves.
API
adapter(uri[, opts])
uri
is a zookeper connection string like localhost:2821
where your
zookeeper cluster is located.
The adapater uses the kafka-node
and accepts a comma separated host:port
pairs, each represents a zookeeper
server.
For a list of options see below.
adapter(opts)
The following options are allowed:
key
: the name of the key prefix the kafka topic (socket.io
)host
: zookeeper hostport
: zookeeper porturi
: substitute forhost
andport
, zookeeper connection stringconsumer
: optional, a kafka.Consumer instanceproducer
: optional, a kafka.Producer instancecreateTopics
: optional, if we should try to create a new Kafka topic (true
)partition
: partition to read and write to (0
)
Running tests
npm test
will run jasmine
with istanbul
code coverage. The command expects
the modules to be installed as global.
TODO
- add CI (crrently code is linted, tested and coverage is above 85%)
- read from multiple partitions
- allow configuration to set consumer options
- allow configuration to set compression (currently snappy is hardcoded)
- benchmark (find out real world limits)