README
website-scraper-phantom
Plugin for website-scraper which returns html for dynamic websites using PhantomJS
Requirements
- nodejs version >= 8
- website-scraper version >= 4
if you need plugin for website-scraper version < 4, you can find it here (version 0.1.0)
Installation
npm install website-scraper website-scraper-phantom
Usage
const scrape = require('website-scraper');
const PhantomPlugin = require('website-scraper-phantom');
scrape({
urls: ['https://www.instagram.com/gopro/'],
directory: '/path/to/save',
plugins: [ new PhantomPlugin() ]
});
How it works
It starts PhantomJS which just opens page and waits when page is loaded. It is far from ideal because probably you need to wait until some resource is loaded or click some button or log in. Currently this module doesn't support such functionality.