The JSON schema that defines the shape of the accepted arguments passed to options.request. When invalid, an Error will be thrown.
Optional
options.request = function(params)
Function that takes the arguments passed to your scraper function and returns the options to pass to the axios module to make the network request.
Required
options.extract = function(response, body, $)
Function that takes axios response, the response body (String) and a cheerio instance. It returns the extracted data you want.
Required
options.schema
The JSON schema that defines the shape of your extracted data. When your data is invalid, an Error with the validation message will be passed to your scraper callback.
Required
options.cheerioOptions
The option to pass to cheerio when it loads the request body.
Optional, default: {}
options.ajvOptions
The option to pass to ajv when it compiles the JSON schemas.
Optional, default: {allErrors: true} - It check all rules collecting all errors
dependecies
axios - Promise based HTTP client for the browser and node.js.
cheerio - Fast, flexible, and lean implementation of core jQuery designed specifically for the server.
ajv - The fastest JSON Schema Validator. Supports draft-04/06/07.
dev dependecies
jasmine - Simple JavaScript testing framework for browsers and node.js.
nock HTTP server mocking and expectations library for Node.js.