express-bot

Crawler(robots) decision middleware for Express

Usage no npm install needed!

<script type="module">
  import expressBot from 'https://cdn.skypack.dev/express-bot';
</script>

README

express-bot

GitHub license Github All Releases GitHub last commit Travis CI

NPM

Crawler(robots) decision middleware for Express (TypeScript)

Install

$ npm install --save express-bot

UserAgent's

  • Twitterbot
  • Google Keyword Suggestion
  • AdsBot-Google
  • Googlebot
  • applebot
  • curl
  • PycURL
  • ....

ALL Bots List -> https://github.com/fkei/express-bot/blob/master/index.js#L9

Use

app.use(require('express-bot')({
  // force dicision using querystring  https://localhost?bot=1 -> hit!!
  querystring: {
    use: true,
    key: 'bot',
    value: '1',
    locals: ...
  },
  // list of UA strings to be added to pre-defined BOTS.
  additionalBots: [
    'MinorBot'
  ]
}));

options (Optional)

Develop

eslint

$ npm run eslint

test

$ npm test