loner

A transform stream for node and browserify that ensures that certain data sequences appear in their own, separate chunks

When progressively parsing streaming binary data, you often need to find magic sequences of bytes that act as separators or "keywords". This can be surprisingly hard to implement efficiently (i.e. without a lot of buffering and copying data around), because the magic sequences might be spread across an arbitrary amount of chunks and you must take care to implement all those edge cases correctly.

loner does that for you. It ensures that each magic sequence you specify will be in it's own data chunk, while otherwise leaving the sequence of chunks as it is. Thus all you need to do downstream, is to compare each incoming chunk against the sequence you are looking for, for example by using buffer-equal.

All the heavy lifting is actually done by binary-stream-replace.

Example

const loner = require('loner');

let PNGSignature = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);

let l = loner(PNGSignature);
l.on('data', (data)=> console.log(data) );

l.write(Buffer.from('Hello\x89\x50'));
l.write(Buffer.from([0x4e, 0x47, 0x0d, 0x0a]));
l.write(Buffer.from("\x1a\x0a\nWorld"));
l.end();

// output =>
// <Buffer 48 65 6c 6c 6f c2>
// <Buffer 89 50 4e 47 0d 0a 1a 0a>
// <Buffer 0a 57 6f 72 6c 64>

API

loner(seq1, seq2, ..., seqN)

seq1 .. seqN Buffer objects containing sequences that you want isolated (on separate chunks). You cann also pass Strings instead of Buffers, they will be converted internally.

NOTE If a seuqence is fully contained within another seqeunce, as in loner('Hello','lo'), interessting things happen! The outcome then depends on the ordering of arguments. In this example, you'd get all the los separated, which obviously breaks each hallo into hel and lo.