README
NoDent
NoDent is a small module for Nodejs that implements the JavaScript ES7 keywords async
and await
. These make writing, reading and understanding asynchronous and callback methods more implicit and embedded in the language. It works by (optionally) transforming JavaScript when it is loaded into Node.
This README assumes you're using Nodent v3.x.x - see Upgrading if your upgrading from an earlier version, and keep an eye out for the asides that look like..
v2.x users - changes between NoDent v2 and v3 are highlighted like this
Contents
- Online demo
- Basic Use and Syntax
- Why Nodent?
- Installation
- Command-Line usage
- Node.js usage
- Use within a browser
- async and await syntax
- Gotchas and ES7 compatibility
- Advanced Configuration
- API
- Built-in conversions and helpers
- Testing
- Upgrading
- Changelog
- Credits
Online demo
You can now see what Nodent does to your JS code with an online demo at here. Within the examples in this README, click on TRY-IT to see the code live.
Basic Use and Syntax
Declare an asynchronous function (one that returns "later").
async function tellYouLater(sayWhat) {
// Do something asynchronous and terminal, such as DB access, web access, etc.
return result ;
}
Call an async function:
result = await tellYouLater("Hi there") ;
To use NoDent, you need to:
require('nodent')() ;
This must take place early in your app, and need only happen once per app - there is no need to require('nodent')
in more than one file, once it is loaded it will process any files ending in use nodent
directive at the top of a .js file.
v2.x users, the '.njs' extension is still supported, but not recommended
You can't use the directive, or any other Nodent features in the file that initially require("nodent")()
. If necessary, have a simple "loader.js" that requires Nodent and then requires your first Nodented file, or start your app with nodent from the command line:
./nodent.js myapp.js
That's the basics.
Why Nodent?
- Performance - on current JS engines, Nodent is between 2x and 5x faster than other solutions in common cases, and up to 10x faster on mobile browsers.
- Simple, imperative code style. Avoids callback pyramids in while maintaining 100% compatibility with existing code.
- No dependency on ES6, "harmony"
- No run-time overhead for Promises, Generators or any other feature beyond ES5 - works on most mobile browsers & IE (although Nodent can use Promises and Generators if you want it to!)
- No execution framework needed as with traceur, babel or regenerator
- No 'node-gyp' or similar OS platform requirement for threads or fibers
- ES7 async and await on ES5 (most browsers and nodejs)
- Compatible with ES6 too - nodent passes ES6 constructs unchanged for use with other transpilers and Node > 4.x
- For more about ES7 async functions and await see:
Installation
npm install --save nodent
Command-Line usage
You can invoke and run a nodented JavaScript file from the command line (although for Node apps it's much easier using the JS transpiler). To load, compile and run your JS file (containing async
and await
), use:
./nodent.js myNodentedFile.js
You can also simply compile and display the output, without running it. This is useful if you want to pre-compile your scripts:
./nodent.js --out myNodentedFile.js
If you are using nodent as part of a toolchain with another compiler, you can output the ES5 or ES6 AST is ESTree format:
./nodent.js --ast myNodentedFile.js
...or read an AST from another tool
./nodent.js --fromast --out estree.json // Read the JSON file as an ESTree AST, and output the nodented JS code
To generate a source-map in the output, use --sourcemap
.
./nodent.js --sourcemap --out myNodentedFile.js
The testing options --parseast
and --minast
output the source as parsed into the AST, before transformation and the minimal AST (without position information) respectively. The option --pretty
outputs the source formatted by nodent before any syntax transformation. You can read the Javascript or JSON from stdin (i.e. piped) by omitting or replacing the filename with -
.
The full list of options is:
option | Description |
---|---|
--fromast | Input is a JSON representation of an ESTree |
--parseast | Parse the input and output the ES7 specification ESTree as JSON |
--pretty | Parse the input and output the JS un-transformed |
--out | Parse the input and output the transformed ES5/6 JS |
--ast | Parse the input and output the transformed ES5/6 ESTree as JSON |
--minast | Same as --ast, but omit all the source-mapping and location information from the tree |
--exec | Execute the transformed code |
--sourcemap | Produce a source-map in the transformed code |
--runtime | Include the nodent runtime in the output |
Code generation options:
option | Description |
---|---|
--use=mode | Ignore any "use nodent" directive in the source file, and force compilation mode to be es7 ,promises ,generators . engine or default |
--wrapAwait | Allow await with a non-Promise expression more info... |
--lazyThenables | Evaluate async bodies lazily in 'es7' mode. See the Changelog for 2.4.0 for more information |
--noruntime | Compile code (in -promises or -engine mode) for execution with no runtime requirement at all |
--es6target | Compile code assuming an ES6 target (as of v3.0.8, this only requires support for arrow functions) |
--noextensions | Don't allow nodent's extensions to the ES7 specification more info... |
Node.js usage
Automatic transpilation
There is no need to use the command line at all if you want to do is use async
and await
in your own scripts then just require('nodent')()
. Files are transformed if they have a use nodent
directive at the top, or have the extension ".njs". Existing files ending in '.js' without a use nodent...
directive are untouched and are loaded and executed unchanged.
ES7 and Promises
Nodent can generate code that implements async
and await
using basic ES5 JavaScript, Promises (via Nodent's built-in library, a third party library or module, or an ES5+/6 platform) or Generators (ES6). Using the directive:
'use nodent';
The use nodent
directive uses a default set of compilation options called 'default', which can be modifed in your package.json.
Within your package.json, you can have named sets of pre-defined options, which individual files can refer to if necessary. There are four pre-defined sets of options: promises, es7, generators and engine.
'use nodent-promises';
'use nodent-es7';
'use nodent-generators';
'use nodent-engine';
Which one should you use?
All the implementations work with each other - you can mix and match. If you're unsure as to which will suit your application best, or want to try them all out 'use nodent';
will use a 'default' configuration you can determine in your application's package.json. See Advanced Configuration for details.
Shipping a self-contained app to a browser, Node <=0.10.x or other unknown environment
use nodent-es7
- it's the most compatible as it doesn't require any platform support such as Promises or Generators, and works on a wide range of desktop and mobile browsers.
modern browsers supporting Promises
Shipping an app or module within Node, npm oruse nodent-promises
provides the most compatibility between modules and apps. If your module or library targets Node earlier than v4.1.x, you should install a Promise library (e.g. rsvp, when, bluebird) or use nodent.Thenable
to expose the Promise API. In promises mode, there is no need for a runtime at all. Specifying the option use nodent-promises {"noRuntime":true}
will generate pure ES5 code at the cost of some loss in performance and increase in code size.
v2.x users:
nodent.EagerThenable()
is still defined, but as of v3 is the same as theThenable
implementation.
Generators
use nodent-generators
generates code which is reasonably easy to follow, but is best not used for anything beyond experimentation as it requires an advanced browser on the client-side, or Node v4.x.x. The performance and memory overhead of generators is poor - currently (Node v6.6.0) averaging 3.5 times slower compared to the es7 with 'lazyThenables'.
Engine
use nodent-engine
does not transpile standard ES7 async/await constructs, but only transpiles the additional non-standard features provided by nodent - await anywhere, async getters, async return and throw. At the time of writing, not many runtimes implement async and await - Chrome v53 does with command line flags, and Edge 14 are examples. On Chrome, performance is better than generators, but not quite as good as Promises, and still less than half the speed of ES7 mode. In promises mode, there is no need for a runtime at all. Specifying the option use nodent-engine {"noRuntime":true}
will generate pure ES5 code at the cost of some loss in performance and increase in code size.
Programmatical usage within scripts
To compile code programmatically you should use require(nodent-compiler
). This module is a standalone compiler that has:
- no require hook (it does not intercept .js files as they are loaded into Node)
- no runtime (it does not implement any functions required to run transpiled code)
- no pollution (because it has no runtime, it does not add async-specific bindings on
Function.prototype
, a global error handler, stack source-mapping)
Example usage:
var NodentCompiler = require('nodent-compiler');
var compiler = new NodentCompiler() ;
var es5ReadySourceCode = compiler.compile(sourceCode, filename, { sourcemap:false, promises: true, noRuntime: true, es6target: true });
Use within a browser
You can use async
and await
within a browser by auto-parsing your scripts when Nodejs serves them to your clients.
The exported function generateRequestHandler(path, matchRegex, options)
creates a node/connect/express compatible function for handling requests for nodent-syntax files that are then parsed and served for use within a stanadrd browser environment, complete with a source map for easy debugging.
For example, with connect:
var nodent = require('nodent')() ;
...
var app = connect() ;
...
app.use(nodent.generateRequestHandler(
"./static-files/web", // Path to where the files are located
/\.njs$/, // Only parse & compiles ending in ".njs"
options // Options (see below)
)) ;
The regex can be omitted, in which case it has the value above.
The currently supported options are:
enableCache: <boolean> // Caches the compiled output in memory for speedy serving.
runtime: <boolean> // Set to precede the compiled code with the runtime support required by Nodent
extensions: <string-array> // A set of file extensions to append if the specified URL path does not exist.
htmlScriptRegex: <optional regex> // If present, Nodent will attempt to read and parse <script> tags within HTML files matching the specified regex
compiler:{ // Options for the code generator
es7:<boolean>, // Compile in es7 mode (like 'use nodent-es7')
promises:<boolean>, // Compile in Promises mode (like 'use nodent-promises')
generators:<boolean>, // Compile in generator mode (like 'use nodent-generators')
engine:<boolean>, // Compile in engine mode (like 'use nodent-engine')
sourcemap:<boolean>, // Create a sourcemap for the browser's debugger
wrapAwait:<boolean>, // Allow 'await' on non-Promise expressions
lazyThenables:<boolean>, // Evaluate async bodies lazily in 'es7' mode. See the Changelog for 2.4.0 for more information
noRuntime:<boolean>, // Only compatible with promises & engine. Generate pure ES5 code for an environment that support Promises natively or as a global declaration. Currently about 15% slower that using the built-in runtime $asyncbind. Default is false.
es6target:<boolean> // Compile code assuming an ES6 target (as of v3.0.8, this only requires support for arrow functions)
}
setHeaders: function(response) {} // Called prior to outputting compiled code to allow for headers (e.g. cache settings) to be sent
Note that parsing of script tags within HTML is relatively simple - the parsing is based on regex and is therefore easily confused by JS strings that contain the text 'script', or malformed/nested tags. Ensure you are parsing accurate HTML to avoid these errors. Scripts inline in HTML do not support source-mapping at present.
If you're using a modern browser with Promise support (or including a third-party promise library), you can specify the compiler filag noRuntime: true
, which will generate pure ES5 code at the cost of some loss in performance and increase in code size. Otherwise, you'll need to provide some support routines at runtime (i.e. in the browser):
Function.prototype.$asyncbind
Function.prototype.$asyncspawn
if you're using generatorsObject.$makeThenable
if you're using thewrapAwait
option and not using promises (see await with a non-Promisewindow.$error
if you use await outside of an async function, to catch unhandled errors, for example:// Called when an async function throws an exception during // asynchronous operations and the calling synchronous function has returned. window.$error = function(exception) { // Maybe log the error somewhere throw ex ; };
This are generated automatically in the transpiled files when you set the runtime
option, and declared when Nodent is loaded (so they are already avaiable for use within Node).
Further information on using Nodent in the browser can be found at https://github.com/MatAtBread/nodent/issues/2.
Other options: Babel & Browserify
You can also invoke nodent from browserify as a plugin, or as an alternative, faster implementation than than Babel's transform-async-to-generator
Async and Await syntax and usage
You can find out more about defining and calling async functions here. There's plenty on the web too.
Gotchas and ES7 compatibility
Async programming with Nodent (or ES7) is much easier and simpler to debug than doing it by hand, or even using run-time constructs such as Promises, which have a complex implementation of the their own when compiled to ES5. However, a couple of common cases are important to avoid.
Differences from the ES7 specification
You can continue to use all the Nodent extensions with async/await capable engines. In the use nodent-engine
mode, all ES7 standard async/await constructs are passed through unchanged, and only functions that use a Nodent extension are transformed.
You can disable the extensions (but not the known differences) by compiling with flag parser:{ noNodentExtensions: true}
or the command line option --noextensions
. Nodent will pass the code unalterted to the parser (acorn) which will fail the syntactic extensions. In this scenario, the dependency acorn-es7-plugin
is not required (but will still be installed by default). The option only works with acorn >4.x.
Extensions to the specification:
- async getters and ** static async** class members:
Nodent permits a class or object definition to define async getters:
async get data() { ... }
get async data() { ... }
class MyClass {
static async name() { ... }
}
- await outside async
The ES7 async-await spec states that you can only use await inside an async function. This generates a warning in nodent, but is permitted. The synchronous return value from the function is compilation mode dependent. In practice this means that the standard, synchronous function containing the await
does not have a useful return value of it's own.
- async return/throw
The statements async return <expression>
and async throw <expression>
are proposed extensions to the ES7 standard (see https://github.com/lukehoban/ecmascript-asyncawait/issues/38). The alternative to this syntax is to use a standard ES5 declaration returning a Promise. See below for details.
Known differences from the specification:
- AsyncFunction
The AsyncFunction
type is not defined by default, but is returned via the expression require('nodent')(...).require('asyncfunction')
. The AsyncFunction
constructor allows you to create async functions on the fly, just as the standard Function
constructor does.
- case without break
As of the current version, case
blocks without a break;
that fall thorugh into the following case
do not transform correctly if they contain an await
expression. Re-work each case
to have it's own execution block ending in break
, return
or throw
. Nodent logs a warning when it detects this situation.
- await non-Promise
The ES7 specification allows an application to await
on a non-Promise value (this occurs because the template implementation wraps every generated value in a Promise). So the statement:
var x = await 100 ; // 100
...is valid. Nodent, by default, does not allow this behaviour (you'll get a run-time error about '100.then is not a function'. Generally, this is not a problem in that you obviously only want to wait on asynchronous things (and not numbers, strings or anything else). However, there is one unpleasant edge case, which is where an expression might be a Promise (my advice is to never write code like this, and avoid code that does).
var x = await maybeThisIsAPromise() ;
In this case, the expression will need wrapping before it is awaited on by Nodent. You can emulate this behaviour by specifying the code-generation flag 'wrapAwait' in your package.json or after the nodent directive:
'use nodent {"wrapAwait":true}';
Wrapping every value in a Promise increases the time taken to invoke an async function by about 20%. An alternative to wrapping everything is to only wrap expression where this might be the case explicitly:
var x = await Promise.resolve(maybeThisIsAPromise()) ;
or
var isThenable = require('nodent').isThenable ;
...
var x = maybeThisIsAPromise() ;
if (isThenable(x))
x = await x ;
The second implementation avoids the expense (20%) of wrapping every return value in a Promise, with the extra code for testing if it is a Promise before awaiting on it.
- lazyThenables
v2.x users - lazyThenables are only available in -es7 mode in v3, and the nodent.Thenable implementation is not lazy, as it was in v2.x.
Invoking an async function without a preceding await
(simply by calling it) executes the function body but you can't get the result. This is useful for initiating 'background' things, or running async functions for their side effects. This is in compliance with the ES7 specification.
However, this has a performance overhead. For maximum performance, you can specify this code generation option in use nodent-es7 {"lazyThenables":true}
mode. In this mode, if you call the async function the body is not actually executed until resolved with an await
(or a .then()
). If you know your code always uses await
, you can use this option to improve performance.
In use nodent-promises
mode, it is the implementation of the Promise that determines the execution scheduling and performance. The table below is a summary of modes and execution semantics. You can test the performance on your own hardware with the following command. Note the relative performance is a worst case, since the test does nothing other than make async calls in a loop.
./nodent.js tests tests/semantics/perf.js
Mode | Flags / Implementation | Lazy / Eager | Possibly sync resolution | Performance (relative) |
---|---|---|---|---|
es7 | lazyThenable | Lazy | Yes | 1.0 |
es7 | (none) | Eager | No | 1.7x slower |
promises | nodent | Eager | No | 1.7x slower |
promises | node 6.6 native | Eager | No | 5.2x slower |
promises | bluebird 3.4.6 | Eager | No | 2.0x slower |
promises | rsvp 3.3.1 | Eager | No | 2.2x slower |
promises | when 3.7.7 | Eager | No | 1.6x slower |
generators | nodent | Eager | No | 7.5x slower |
generators | node 6.6 native | Eager | No | 15.0x slower |
generators | bluebird 3.4.6 | Eager | No | 8.5x slower |
generators | rsvp 3.3.1 | Eager | No | 7.6x slower |
generators | when 3.7.7 | Eager | No | 8.3x slower |
All other JavaScript ES5/6/2015 constructs will be transformed as necessary to implement async
and await
.
v2.x users - note the timings and execution semantics for Thenable (and EagerThenable) have changed: they are now fully Promise/A+ compliant, meaning they resolve asynchronously and evaluate eagerly. Only -es7 lazyThenable mode might resolve synchronously.
Exiting async functions from callbacks
Specifically in Nodent (not specified by ES7), you can interface an ES7 async function with a old style callback-based function. For example, to create an async function that sleeps for a bit, you can use the standard setTimeout function, and in its callback use the form async return <expression>
to not only return from the callback, but also the surrounding async function:
async function sleep(t) {
setTimeout(function(){
// NB: "async return" and "async throw" are NOT ES7 standard syntax
async return undefined;
},t) ;
}
Similarly, async throw <expression>
causes the inner callback to make the container async function throw an exception. The async return
and async throw
statements are NOT ES7 standards (see https://github.com/tc39/ecmascript-asyncawait/issues/38). If you want your code to remain compatible with standard ES7 implementations when the arrive, use the second form above, which is what nodent would generate and is therefore ES5/6/7 compatible.
Advanced Configuration
Nodent has two sets of configuration values:
- one controls the runtime environment - catching unhandled errors, handling warnings from Nodent and stack mapping, etc.
- the other controls code generation - whether to generate code that uses Promises or generators (or not), whether to await on non-Promise values, whether to include the runtime code, etc.
The first is defined once per installation (Nodent contained as dependencies within dependencies have their own, per-installation, instances). You can 'redefine' the values, but the effect is to overwrite existing settings. These are specified as the first argument when you require('nodent')(options)
. Details of the options are below.
The second set is defined per-file for each file that Nodent loads and compiles. The options are:
Member | Type | |
---|---|---|
es7 | boolean | set by any use nodent... directive |
promises | boolean | set by the directive use nodent-promises and use nodent-generators |
generators | boolean | set by the directive use nodent-generators |
engine | boolean | set by the directive use nodent-engine |
wrapAwait | boolean | default: false - allow await followed by a non-Promise more info... |
sourcemap | boolean | default: true - generate a source-map in the output JS |
noRuntime | boolean | default: false - generate pure ES5 code with external dependencies. The code is bigger and slower, and only works with -promises or -engine |
es6target | boolean | default: false - use ES6 constructs to improve code speed and size (as of v3.0.8, this only requires support for arrow functions) |
parser | object | default: {sourceType:'script'} - passed to Acorn to control the parser |
mapStartLine | int | default: 0 - initial line number for the source-map |
generatedSymbolPrefix | string | default ' |