url-downloader

npm package to download resources from url to targets such as LOCAL SYSTEM - AWS S3 - MICROSOFT AZURE STORAGE - GOOGLE CLOUD STORAGE

Usage no npm install needed!

<script type="module">
  import urlDownloader from 'https://cdn.skypack.dev/url-downloader';
</script>

README

URL DOWNLOADER

Module to download a list of urls into the target system and specify the max amount of parallel downloading

Install

npm install url-downloader

Usage

    const urlDownloader = require('url-downloader');
    const resources [{
        "target": "LOCAL",
        "origin": "https://images.unsplash.com/photo-1558981852-426c6c22a060",
        "destination": "1.jpg"
    }, {
        "target": "LOCAL",
        "origin": "https://images.unsplash.com/photo-1558980664-3a031cf67ea8",
        "destination": "2.jpg"
    }];

    // optional
    const options = {
        maxConcurrency: 5
    }

    urlDownloader.download(resources, options).then((results) => {
        console.log('files downloaded and saved');
        console.log(results);
    })

Options

You can specify a different target to save the downloaded url content.

Current allowed targets are :

  • LOCAL ( file system )

Future destinations will be supported as well :

  • AWS_S3 ( Amazon S3 bucket )
  • MICROSOFT_AZURE_STORAGE ( Microsoft Azure Storage )
  • GOOGLE_CLOUD_STORAGE ( Google Cloud Storage )