README
it-length-prefixed
Streaming length prefixed buffers with async iterators
Install
npm install it-length-prefixed
Usage
import { pipe } from 'it-pipe'
import * as lp from 'it-length-prefixed'
const encoded = []
// encode
await pipe(
[uint8ArrayFromString('hello world')],
lp.encode(),
async source => {
for await (const chunk of source) {
encoded.push(chunk.slice()) // (.slice converts BufferList to Buffer)
}
}
)
console.log(encoded)
// => [Buffer <0b 68 65 6c 6c 6f 20 77 6f 72 6c 64>]
const decoded = []
// decode
await pipe(
encoded, // e.g. from above
lp.decode(),
async source => {
for await (const chunk of source) {
decoded.push(chunk.slice()) // (.slice converts BufferList to Buffer)
}
}
)
console.log(decoded)
// => [Buffer <68 65 6c 6c 6f 20 77 6f 72 6c 64>]
API
import {
encode, decode
} from 'it-length-prefixed'
import {
encode,
MIN_POOL_SIZE,
DEFAULT_POOL_SIZE
} from 'it-length-prefixed/encode'
import {
decode,
MAX_LENGTH_LENGTH,
MAX_DATA_LENGTH
} from 'it-length-prefixed/decode'
encode([opts])
opts: Object
, optionalpoolSize: 10 * 1024
: Buffer pool size to allocate up frontminPoolSize: 8
: The minimum size the pool can be before it is re-allocated. Note: it is important this value is greater than the maximum value that can be encoded by thelengthEncoder
(see the next option). Since encoded lengths are written into a buffer pool, there needs to be enough space to hold the encoded value.lengthEncoder: Function
: A function that encodes the length that will prefix each message. By default this is avarint
encoder. It is passed avalue
to encode, an (optional)target
buffer to write to and an (optional)offset
to start writing from. The function should encode thevalue
into thetarget
(or alloc a new Buffer if not specified), set thelengthEncoder.bytes
value (the number of bytes written) and return thetarget
.- The following additional length encoders are available:
- int32BE -
const { int32BEEncode } = require('it-length-prefixed')
- int32BE -
- The following additional length encoders are available:
Returns a transform that yields BufferList
objects. All messages will be prefixed with a length, determined by the lengthEncoder
function.
encode.single(chunk, [opts])
chunk: Buffer|BufferList
chunk to encodeopts: Object
, optionallengthEncoder: Function
: See description above. Note that this encoder will not be passed atarget
oroffset
and so will need to allocate a buffer to write to.
Returns a BufferList
containing the encoded chunk.
decode([opts])
opts: Object
, optionalmaxLengthLength
: If provided, will not decode messages whose length section exceeds the size specified, if omitted will use the default of 147 bytes.maxDataLength
: If provided, will not decode messages whose data section exceeds the size specified, if omitted will use the default of 4MB.onLength(len: Number)
: Called for every length prefix that is decoded from the streamonData(data: BufferList)
: Called for every chunk of data that is decoded from the streamlengthDecoder: Function
: A function that decodes the length that prefixes each message. By default this is avarint
decoder. It is passed somedata
to decode which is aBufferList
. The function should decode the length, set thelengthDecoder.bytes
value (the number of bytes read) and return the length. If the length cannot be decoded, the function should throw aRangeError
.- The following additional length decoders are available:
- int32BE -
const { int32BEDecode } = require('it-length-prefixed')
- int32BE -
- The following additional length decoders are available:
Returns a transform that yields BufferList
objects.
decode.fromReader(reader, [opts])
Behaves like decode
except it only reads the exact number of bytes needed for each message in reader
.
reader: Reader
: An it-readeropts: Object
, optionalmaxLengthLength
: If provided, will not decode messages whose length section exceeds the size specified, if omitted will use the default of 147 bytes.maxDataLength
: If provided, will not decode messages whose data section exceeds the size specified, if omitted will use the default of 4MB.onData(data: BufferList)
: Called for every chunk of data that is decoded from the streamlengthEncoder: Function
: See description above.
Returns a transform that yields BufferList
objects.
Contribute
PRs and issues gladly accepted! Check out the issues.
License
MIT © 2016 Friedel Ziegelmayer