crawl-this

A light weight crawler

Usage no npm install needed!

<script type="module">
  import crawlThis from 'https://cdn.skypack.dev/crawl-this';
</script>

README

Light weight web crawler

How to install

npm install crawl-this -g

Required versions

node: >=v4.0.0

Usage

Usage: crawl-this [options]

Options:

-h, --help           output usage information
-V, --version        output the version number
-d, --depth <depth>  Depth to search. Should be a number

example: crawl-this -d 2 http://github.com