README
Request — Simplified HTTP client
Super simple to use
Request is designed to be the simplest way possible to make http calls. It supports HTTPS and follows redirects by default.
var request = require('request');
request('http://www.google.com', function (error, response, body) {
if (!error && response.statusCode == 200) {
console.log(body) // Print the google web page.
}
})
Streaming
You can stream any response to a file stream.
request('http://google.com/doodle.png').pipe(fs.createWriteStream('doodle.png'))
You can also stream a file to a PUT or POST request. This method will also check the file extension against a mapping of file extensions to content-types (in this case application/json
) and use the proper content-type
in the PUT request (if the headers don’t already provide one).
fs.createReadStream('file.json').pipe(request.put('http://mysite.com/obj.json'))
Request can also pipe
to itself. When doing so, content-type
and content-length
are preserved in the PUT headers.
request.get('http://google.com/img.png').pipe(request.put('http://mysite.com/img.png'))
Now let’s get fancy.
http.createServer(function (req, resp) {
if (req.url === '/doodle.png') {
if (req.method === 'PUT') {
req.pipe(request.put('http://mysite.com/doodle.png'))
} else if (req.method === 'GET' || req.method === 'HEAD') {
request.get('http://mysite.com/doodle.png').pipe(resp)
}
}
})
You can also pipe()
from http.ServerRequest
instances, as well as to http.ServerResponse
instances. The HTTP method, headers, and entity-body data will be sent. Which means that, if you don't really care about security, you can do:
http.createServer(function (req, resp) {
if (req.url === '/doodle.png') {
var x = request('http://mysite.com/doodle.png')
req.pipe(x)
x.pipe(resp)
}
})
And since pipe()
returns the destination stream in ≥ Node 0.5.x you can do one line proxying. :)
req.pipe(request('http://mysite.com/doodle.png')).pipe(resp)
Also, none of this new functionality conflicts with requests previous features, it just expands them.
var r = request.defaults({'proxy':'http://localproxy.com'})
http.createServer(function (req, resp) {
if (req.url === '/doodle.png') {
r.get('http://google.com/doodle.png').pipe(resp)
}
})
You can still use intermediate proxies, the requests will still follow HTTP forwards, etc.
Proxies
If you specify a proxy
option, then the request (and any subsequent
redirects) will be sent via a connection to the proxy server.
If your endpoint is an https
url, and you are using a proxy, then
request will send a CONNECT
request to the proxy server first, and
then use the supplied connection to connect to the endpoint.
That is, first it will make a request like:
HTTP/1.1 CONNECT endpoint-server.com:80
Host: proxy-server.com
User-Agent: whatever user agent you specify
and then the proxy server make a TCP connection to endpoint-server
on port 80
, and return a response that looks like:
HTTP/1.1 200 OK
At this point, the connection is left open, and the client is
communicating directly with the endpoint-server.com
machine.
See the wikipedia page on HTTP Tunneling for more information.
By default, when proxying http
traffic, request will simply make a
standard proxied http
request. This is done by making the url
section of the initial line of the request a fully qualified url to
the endpoint.
For example, it will make a single request that looks like:
HTTP/1.1 GET http://endpoint-server.com/some-url
Host: proxy-server.com
Other-Headers: all go here
request body or whatever
Because a pure "http over http" tunnel offers no additional security
or other features, it is generally simpler to go with a
straightforward HTTP proxy in this case. However, if you would like
to force a tunneling proxy, you may set the tunnel
option to true
.
If you are using a tunneling proxy, you may set the
proxyHeaderWhiteList
to share certain headers with the proxy.
By default, this set is:
accept
accept-charset
accept-encoding
accept-language
accept-ranges
cache-control
content-encoding
content-language
content-length
content-location
content-md5
content-range
content-type
connection
date
expect
max-forwards
pragma
proxy-authorization
referer
te
transfer-encoding
user-agent
via
Note that, when using a tunneling proxy, the proxy-authorization
header is never sent to the endpoint server, but only to the proxy
server. All other headers are sent as-is over the established
connection.
UNIX Socket
request
supports the unix://
protocol for all requests. The path is assumed to be absolute to the root of the host file system.
HTTP paths are extracted from the supplied URL by testing each level of the full URL against net.connect for a socket response.
Thus the following request will GET /httppath
from the HTTP server listening on /tmp/unix.socket
request.get('unix://tmp/unix.socket/httppath')
Forms
request
supports application/x-www-form-urlencoded
and multipart/form-data
form uploads. For multipart/related
refer to the multipart
API.
URL-encoded forms are simple.
request.post('http://service.com/upload', {form:{key:'value'}})
// or
request.post('http://service.com/upload').form({key:'value'})
For multipart/form-data
we use the form-data library by @felixge. You don’t need to worry about piping the form object or setting the headers, request
will handle that for you.
var r = request.post('http://service.com/upload', function optionalCallback (err, httpResponse, body) {
if (err) {
return console.error('upload failed:', err);
}
console.log('Upload successful! Server responded with:', body);
})
var form = r.form()
form.append('my_field', 'my_value')
form.append('my_buffer', new Buffer([1, 2, 3]))
form.append('my_file', fs.createReadStream(path.join(__dirname, 'doodle.png')))
form.append('remote_file', request('http://google.com/doodle.png'))
// Just like always, `r` is a writable stream, and can be used as such (you have until nextTick to pipe it, etc.)
// Alternatively, you can provide a callback (that's what this example does — see `optionalCallback` above).
HTTP Authentication
request.get('http://some.server.com/').auth('username', 'password', false);
// or
request.get('http://some.server.com/', {
'auth': {
'user': 'username',
'pass': 'password',
'sendImmediately': false
}
});
// or
request.get('http://some.server.com/').auth(null, null, true, 'bearerToken');
// or
request.get('http://some.server.com/', {
'auth': {
'bearer': 'bearerToken'
}
});
If passed as an option, auth
should be a hash containing values user
|| username
, pass
|| password
, and sendImmediately
(optional). The method form takes parameters auth(username, password, sendImmediately)
.
sendImmediately
defaults to true
, which causes a basic authentication header to be sent. If sendImmediately
is false
, then request
will retry with a proper authentication header after receiving a 401
response from the server (which must contain a WWW-Authenticate
header indicating the required authentication method).
Note that you can also use for basic authentication a trick using the URL itself, as specified in RFC 1738.
Simply pass the user:password
before the host with an @
sign.
var username = 'username',
password = 'password',
url = 'http://' + username + ':' + password + '@some.server.com';
request({url: url}, function (error, response, body) {
// Do more stuff with 'body' here
});
Digest authentication is supported, but it only works with sendImmediately
set to false
; otherwise request
will send basic authentication on the initial request, which will probably cause the request to fail.
Bearer authentication is supported, and is activated when the bearer
value is available. The value may be either a String
or a Function
returning a String
. Using a function to supply the bearer token is particularly useful if used in conjuction with defaults
to allow a single function to supply the last known token at the time or sending a request or to compute one on the fly.
OAuth Signing
// Twitter OAuth
var qs = require('querystring')
, oauth =
{ callback: 'http://mysite.com/callback/'
, consumer_key: CONSUMER_KEY
, consumer_secret: CONSUMER_SECRET
}
, url = 'https://api.twitter.com/oauth/request_token'
;
request.post({url:url, oauth:oauth}, function (e, r, body) {
// Ideally, you would take the body in the response
// and construct a URL that a user clicks on (like a sign in button).
// The verifier is only available in the response after a user has
// verified with twitter that they are authorizing your app.
var access_token = qs.parse(body)
, oauth =
{ consumer_key: CONSUMER_KEY
, consumer_secret: CONSUMER_SECRET
, token: access_token.oauth_token
, verifier: access_token.oauth_verifier
}
, url = 'https://api.twitter.com/oauth/access_token'
;
request.post({url:url, oauth:oauth}, function (e, r, body) {
var perm_token = qs.parse(body)
, oauth =
{ consumer_key: CONSUMER_KEY
, consumer_secret: CONSUMER_SECRET
, token: perm_token.oauth_token
, token_secret: perm_token.oauth_token_secret
}
, url = 'https://api.twitter.com/1.1/users/show.json?'
, params =
{ screen_name: perm_token.screen_name
, user_id: perm_token.user_id
}
;
url += qs.stringify(params)
request.get({url:url, oauth:oauth, json:true}, function (e, r, user) {
console.log(user)
})
})
})
Custom HTTP Headers
HTTP Headers, such as User-Agent
, can be set in the options
object.
In the example below, we call the github API to find out the number
of stars and forks for the request repository. This requires a
custom User-Agent
header as well as https.
var request = require('request');
var options = {
url: 'https://api.github.com/repos/mikeal/request',
headers: {
'User-Agent': 'request'
}
};
function callback(error, response, body) {
if (!error && response.statusCode == 200) {
var info = JSON.parse(body);
console.log(info.stargazers_count + " Stars");
console.log(info.forks_count + " Forks");
}
}
request(options, callback);
request(options, callback)
The first argument can be either a url
or an options
object. The only required option is uri
; all others are optional.
uri
||url
- fully qualified uri or a parsed url object fromurl.parse()
qs
- object containing querystring values to be appended to theuri
method
- http method (default:"GET"
)headers
- http headers (default:{}
)body
- entity body for PATCH, POST and PUT requests. Must be aBuffer
orString
.form
- when passed an object or a querystring, this setsbody
to a querystring representation of value, and addsContent-type: application/x-www-form-urlencoded; charset=utf-8
header. When passed no options, aFormData
instance is returned (and is piped to request).auth
- A hash containing valuesuser
||username
,pass
||password
, andsendImmediately
(optional). See documentation above.json
- setsbody
but to JSON representation of value and addsContent-type: application/json
header. Additionally, parses the response body as JSON.multipart
- (experimental) array of objects which contains their own headers andbody
attribute. Sendsmultipart/related
request. See example below.followRedirect
- follow HTTP 3xx responses as redirects (default:true
). This property can also be implemented as function which getsresponse
object as a single argument and should returntrue
if redirects should continue orfalse
otherwise.followAllRedirects
- follow non-GET HTTP 3xx responses as redirects (default:false
)maxRedirects
- the maximum number of redirects to follow (default:10
)encoding
- Encoding to be used onsetEncoding
of response data. Ifnull
, thebody
is returned as aBuffer
.pool
- A hash object containing the agents for these requests. If omitted, the request will use the global pool (which is set to node's defaultmaxSockets
)pool.maxSockets
- Integer containing the maximum amount of sockets in the pool.timeout
- Integer containing the number of milliseconds to wait for a request to respond before aborting the requestproxy
- An HTTP proxy to be used. Supports proxy Auth with Basic Auth, identical to support for theurl
parameter (by embedding the auth info in theuri
)oauth
- Options for OAuth HMAC-SHA1 signing. See documentation above.hawk
- Options for Hawk signing. Thecredentials
key must contain the necessary signing info, see hawk docs for details.strictSSL
- Iftrue
, requires SSL certificates be valid. Note: to use your own certificate authority, you need to specify an agent that was created with that CA as an option.jar
- Iftrue
andtough-cookie
is installed, remember cookies for future use (or define your custom cookie jar; see examples section)aws
-object
containing AWS signing information. Should have the propertieskey
,secret
. Also requires the propertybucket
, unless you’re specifying yourbucket
as part of the path, or the request doesn’t use a bucket (i.e. GET Services)httpSignature
- Options for the HTTP Signature Scheme using Joyent's library. ThekeyId
andkey
properties must be specified. See the docs for other options.localAddress
- Local interface to bind for network connections.gzip
- Iftrue
, add anAccept-Encoding
header to request compressed content encodings from the server (if not already present) and decode supported content encodings in the response.tunnel
- Iftrue
, then always use a tunneling proxy. Iffalse
(default), then tunneling will only be used if the destination ishttps
, or if a previous request in the redirect chain used a tunneling proxy.proxyHeaderWhiteList
- A whitelist of headers to send to a tunneling proxy.
The callback argument gets 3 arguments:
- An
error
when applicable (usually fromhttp.ClientRequest
object) - An
http.IncomingMessage
object - The third is the
response
body (String
orBuffer
, or JSON object if thejson
option is supplied)
Convenience methods
There are also shorthand methods for different HTTP METHODs and some other conveniences.
request.defaults(options)
This method returns a wrapper around the normal request API that defaults to whatever options you pass in to it.
Note: You can call .defaults()
on the wrapper that is returned from request.defaults
to add/override defaults that were previously defaulted.
For example:
//requests using baseRequest() will set the 'x-token' header
var baseRequest = request.defaults({
headers: {x-token: 'my-token'}
})
//requests using specialRequest() will include the 'x-token' header set in
//baseRequest and will also include the 'special' header
var specialRequest = baseRequest.defaults({
headers: {special: 'special value'}
})
request.put
Same as request()
, but defaults to method: "PUT"
.
request.put(url)
request.patch
Same as request()
, but defaults to method: "PATCH"
.
request.patch(url)
request.post
Same as request()
, but defaults to method: "POST"
.
request.post(url)
request.head
Same as request() but defaults to method: "HEAD"
.
request.head(url)
request.del
Same as request()
, but defaults to method: "DELETE"
.
request.del(url)
request.get
Same as request()
(for uniformity).
request.get(url)
request.cookie
Function that creates a new cookie.
request.cookie('cookie_string_here')
request.jar
Function that creates a new cookie jar.
request.jar()
Examples:
var request = require('request')
, rand = Math.floor(Math.random()*100000000).toString()
;
request(
{ method: 'PUT'
, uri: 'http://mikeal.iriscouch.com/testjs/' + rand
, multipart:
[ { 'content-type': 'application/json'
, body: JSON.stringify({foo: 'bar', _attachments: {'message.txt': {follows: true, length: 18, 'content_type': 'text/plain' }}})
}
, { body: 'I am an attachment' }
]
}
, function (error, response, body) {
if(response.statusCode == 201){
console.log('document saved as: http://mikeal.iriscouch.com/testjs/'+ rand)
} else {
console.log('error: '+ response.statusCode)
console.log(body)
}
}
)
Cookies are disabled by default (else, they would be used in subsequent requests). To enable cookies, set jar
to true
(either in defaults
or options
) and install tough-cookie
.
var request = request.defaults({jar: true})
request('http://www.google.com', function () {
request('http://images.google.com')
})
To use a custom cookie jar (instead of request
’s global cookie jar), set jar
to an instance of request.jar()
(either in defaults
or options
)
var j = request.jar()
var request = request.defaults({jar:j})
request('http://www.google.com', function () {
request('http://images.google.com')
})
OR
// `npm install --save tough-cookie` before this works
var j = request.jar()
var cookie = request.cookie('your_cookie_here')
j.setCookie(cookie, uri);
request({url: 'http://www.google.com', jar: j}, function () {
request('http://images.google.com')
})
To inspect your cookie jar after a request
var j = request.jar()
request({url: 'http://www.google.com', jar: j}, function () {
var cookie_string = j.getCookieStringSync(uri); // "key1=value1; key2=value2; ..."
var cookies = j.getCookiesSync(uri);
// [{key: 'key1', value: 'value1', domain: "www.google.com", ...}, ...]
})
Debugging
There are at least three ways to debug the operation of request
:
Launch the node process like
NODE_DEBUG=request node script.js
(lib,request,otherlib
works too).Set
require('request').debug = true
at any time (this does the same thing as #1).Use the request-debug module to view request and response headers and bodies.