request-robots

An express.js middleware for handling noisy robots.txt

Usage no npm install needed!

<script type="module">
  import requestRobots from 'https://cdn.skypack.dev/request-robots';
</script>

README

request-robots

npm version npm download Build Status Greenkeeper badge

An express.js middleware for handling noisy robots.ico

Modern browsers usually asks for robots of the website. Having a robots in a Web page is a good thing (normally).

However it is not always desired and sometime developers need a way to avoid the extra payload.

Nice way to handle that is to send 204 status code for /robots.txt requests.

Install

npm install request-robots

Example

const express = require('express');
const robots = require('request-robots');

const app = express();
app.use(robots());

// Add your routes here, etc.

app.listen(3000);

License

MIT