README
xresilient
Build regenerative, resumable NodeJS streams
Installing
Via NPM:
npm install xresilient
Usage
// Node CommonJS
const xresilient = require('xresilient');
// Or ES6
import xresilient from 'xresilient';
Examples
const retriableStream = xresilient(
function returnStream({bytesRead, retryCount}) {
console.log(`Count = ${retryCount}`);
return request.get('https://website.com/image.png', {
headers: {
Range: `bytes=${bytesRead}-`,
},
});
},
{retries: 5},
);
retriableStream.pipe(fs.createWriteStream('image.png'));
The above snippet of code would request an image file from the URL and pipe the result into the image.png
file
Based on the configuration to retry 5
times, it would recall the function specified with the number of bytes read before the error so you can resume the stream if need be
For instance, get requests interrupted by network errors can be resumed without breaking the active pipe.
API
xresilient(fn[, options])
fn
: <GenFn>options
extends
stream.ReadableOptions
: Objectretries
: <number> Number of times to retry the stream. Default:5
.destroyer
: <function> Method with which to destroy the underlying stream.source
: <NodeJS.ReadableStream> Readable stream returned by the specifiedfn
function.
- Returns: <ResilientStream>
Return a regenerative, persistent, resuming, resilient stream wrapped That swaps underlying stream source without data loss.
The fn
argument must be a function or async function taking two arguments returning a ResilientStream.
Returning an ended readable stream forces the resilient stream to be ended. Returning a destroyed stream forces the resilient stream to retry the generator once until the limit is reached.
Event: 'data'
chunk
: <Buffer>
The 'data'
event is emitted whenever the active underlying stream pushes relinquishes ownership of a chunk to the resilient stream which in turn pushes the chunk out to a consumer.
[See stream.Readable (Event: 'data')]
Event: 'end'
The 'end'
event is emitted whenever there is no more data within the resilient stream's underlying resources to be consumed.
[See stream.Readable (Event: 'end')]
Event: 'retry'
retrySlice
: <RetrySlice>
The 'retry'
event is emitted after an error is caught either within the active stream source or the generative function and the stream hasn't used up all of it's retries.
This event is emitted right before the genFn is called.
Event: 'error'
err
: <Error>
The 'error'
event is emitted either when the underlying readable stream encounters an 'error'
event or the generative function throws an error while the resilient stream has maxed-out all possible retries.
i.e
self.getRetries() === self.getRetryCount()
At this point, the resilient stream is destroyed and the specified GenFn isn't called. This, ends the resilient iteration.
Event: 'readable'
The 'readable'
event is emitted when there is data available to be read from the stream or the end of the stream has been reached.
Either way, the retriableStream.read()
method would return the available data and reset the flow state so all buffered data flushes.
[See stream.Readable (Event: 'readable')]
GenFn: Function
|AsyncFunction
storeSlice
: <ResilientStore>- Returns: <NodeJS.ReadableStream>
Generator function returning a readable stream.
Errors / Promise rejections caught during this function's execution would be emitted through the error
event.
This function can either be an async function or a normal function.
ResilientStream extends
stream.Readable
The Core resilient stream whose data is streamed off of the underlying streams gotten from the GenFn.
RetrySlice: object
retryCount
: <number> The number of retry iterations so far. (starting from 0)maxRetries
: <number> The maximum number of retries possible.bytesRead
: <number> The number of bytes previously read (if any).lastErr
: <Error> The error emitted either by the last generator function execution or previous stream.oldStream
: <NodeJS.ReadableStream> The old stream that error-ed out (if any).
ResilientStore extends
RetrySlice: object
trialCount
: <number> The active execution iteration count. (starting from 1)
ResilientStream.setRetries(retries)
This method sets the number of possible retries.
If the retryCount
is less than this value, returns false
.
else, returns true
on successful setting.
ResilientStream.getRetries()
- Returns: <number>
Returns the value of max possible retries before error emit.
ResilientStream.getRetryCount()
- Returns: <number>
Returns the number of the retries so-far.
Development
Building
Feel free to clone, use in adherance to the license and perhaps send pull requests
git clone https://github.com/miraclx/xresilient.git
cd xresilient
npm install
# hack on code
npm run build
License
Apache 2.0 © Miraculous Owonubi (@miraclx) <omiraculous@gmail.com>