<script type="module">
import dotAsyncData from 'https://cdn.skypack.dev/dot-async-data';
</script>
README
dot-async-data
Asynchronous dot notation to radically simplify JSON database access.
Frustrated by complex code to navigate through related JSON objects?
Tired of tracking down typographical errors in strings that have to be compiled into queries?
Angry about all the errors thrown by navigating incomplete JSON records?
What if you could:
Load a single root object and navigate to any of its leaf nodes with dot notation (even those that require loading additional data)?
Automatically save data when changes are made?
Match and transform data and paths with inline regular expressions aand functions?
Do this with ANY JSON database that supports get(key) and set(key,value) and even pass queries to a GraphQL or SQL API!
If you vist the README on our GitHub pages site,
you will be able to see the output of most sample code and be able edit sample data and change the behavior of some of the examples.
Example
Assume that customer contacts are stored distinct from their account info which is distinct from billing and shipping info which is distinct from addresses:
dot-async-data is isomorphic, the index.js file can be loaded in a browser or required by NodeJS code.
Browser
<script src="./path/copyOfIndex.js"></script>
<script>
const {$min, $max, ... other functions you wish to use} = dotAsyncData;
... your code
</script>
NodeJS
const {$min, $max, ... other functions you wish to use} = require("dotAsyncData");
... your code
To maintain isomorphism and keep our own build process simple, the use of JavaScript modules using import/export is not yet supported. The esm module is
a dependency for the NodeJS version. If you wish to mix and match, start your application with the command node -r esm <your file>.
See the files test/index.html and test/index.js for basic examples while we enhance the documentation.
This is an ALPHA release.
Creating An Async Data Object
You can make any object into an asynchronously accessable object by calling dotAsyncData:
You can also pass null or undefined as the first argument, so long as you provide a data store in the start-up options. When you do this, the first step in the dot path is considered the
initial query key. The dotAsyncData object can be re-used over and over with different paths.
const mydb = ... some data store instance,
query = dotAsyncData(null,{db}),
asyncJane = query["/Person/#abcxyz"], // note lack of await or ()
janesAge = await asyncJane.age(),
asyncBill = query["/Person/#123789"],
...
Finally, you can pass {} as the first argument and populate the object using any serializable query supported by your database, e.g. GraphQL.
See the $query built-in function for more details.
The options surface for dotAsyncData is:
{
isDataKey:function, // return true if the passed in value is a database object key, e.g. a UUID
idKey: string, // the property in JSON objects where the unique id is store, e.g. _id. Value not have to return true for `isDataKey`
db: object, // the database or database wrapper supporting `get(key)` and `set(key,value)`, the wrapper must convert to and from JSON
autoSave: boolean, // automatically save objects when their properties or values are updated
inline: boolean, // allow inline functions
cache
}
Path Access
Once you have a dotAsyncData object, you can access it to any depth using standard dot notation, just put an await at the start and finish it with ().
You can use functions and regular expressions as part of path to filter path components and filter or transform data. These operayions can all be expressed using
square bracket, [<function or regular expression] notation. Some functions can be used directly, e.g. part1.
lt;function name>.part2.
dotAsyncObject[/*.Address/].city(); // cities for all data in fields ending in Address, e.g. homeAddress, billingAddress, etc.
dotAsyncObject.children.$avg.age(); // the average age of the children
dotAsyncObject.children[$avg].age();
See built-in functions below for additional concrete and interactive examples.
If a dot path can't be resolved it will simply return undefined. No more errors half way down a path because of a missing entity! (We will probably add a strict mode
if you want the errors).
Built-in Functions
Unless otherwise noted, built-in functions can be access via inline square brackets or dot notation, e.g. value[$max] and value.$max. With some noted exceptions,
if a function is flagged as inline, then to use it the dotAsyncData object must have been created with {inline:true}.
Functions that typically work only on arrays or object are usually not polymorphic, i.e. if $max is called on a single numeric value, then undefined is returned.
If $max is called on an array, the maximum value in the array is returned. Polymorphic versions are planned and will be prefixed with poly, e.g. $polyMax.
Exceptions include $count, $map and $reduce which handle array and non-array data. Non-array data is treated like single element arrays.
For the examples below, assume the following:
var data = {name:"joe",children:[{name:"janet",age:5},{name:"jon",age:10},{name:"mary",age:null}]};
If you visit this README on our GitHub pages site,
you will be able to edit the sample data and change the behavior of the examples. For example, try changing the age of mary form null to 8.
$count - number of values in array that are not undefined.
const { $count } = dotAsyncData,
asyncDataObject = dotAsyncData(data);
console.log(await asyncDataObject.children[$count]()); // 3
console.log(await asyncDataObject.children.$count()); // 3
console.log((await asyncDataObject.children()).length); // 3
console.log(await asyncDataObject.children.age[$count]()); // 2, obly two children have known ages
console.log(await asyncDataObject.children.age.$count()); // 2
console.log((await asyncDataObject.children.age()).length); // 3, mary has age null, undefined would have bene filtered out
$values - returns the values in the array resolved if they are database keys
deepEqual(
await asyncDataObject.children.$values(),
await dotAsyncData({children:["/Object/#child1","/Object/#child2","/Object/#child3"]}).children.$values()
) = true // assuming the keys resolve to children with the same names and ages as the assumed data
$type(type :string ) - inline but usable without options flag
$match(pattern :any) - polymorphic, matches any value, including objects with multiple properties holding literals or $lt, $lte, $eq, $eeq, $neq, $gte, $gte, $type.
const { $match } = dotAsyncData,
asyncDataObject = dotAsyncData(data,{inline:true});
console.log(await asyncDataObject.children[$match({age:{$lte: 5},name:"janet"})]())
// returns an array of children that have age<=5 and name="janet
$query(formatter :function|:string) - inline only (for now)
Takes the current property name, value provided by an inline function, or terminal value for the path and passes it to the formatter function. The string value returned by the formatter function
will be passed to db.query (with db being the database wrapper provided when the dotAsyncData object was created) and results will be passed down the path.
If a string is passed to $query, the string is treated as an interpolation and the varibale $value is available for resolution.
Using this capability you can query a server using SQL, GraphQL, Mongo query language, etc. You just need to provide an adapter that passes the string to your server and returns parsed JSON.
await asyncDataObject.name[$query("SELECT ${$value} FROM Contacts OUTPUT JSON")]();
await asyncDataObject.name[$query((value) => `SELECT ${value} FROM Contacts OUTPUT JSON`)]();
$get - inline only (for now), polymorphic, returns the results of a database query using the value at the current property in the path
$set(value :any) - inline only (for now), polymorphic, sets the value of the current property
Functions containing closures will usually silently fail and the path will resolve to undefined.
Internals
dotAsyncData objects are Proxies around Functions that maintain a closure around property access requests and the values to which they resolve.
The functions and regular expressions used within access paths take advantage of [] property definition that is part of modern JavaScript. Since
the JavaScript engine checks the syntax when [] is initially processed, the string version of the functions and regular expressions that form property names are known to be reversable into
actual functions and regular expressions, unless a function contains a closure.
Security
Using inline functions and regular expressions in the browser brings along some security issues related to code injection. It is recommeded you not implement
a form based mechanism for defining queries unless you fully understand the issues related to code injection.
dotAsyncData paths are fully serializable and could be sent to a server, this creates even more attack vectors that must be taken into consideration.
As a result on both of the above, inline functions and regular expressions must be explicitly turned on when creating a dotAsyncData object by using {inline:true}.
Acknowledgements
Although the purpose and architecture of dot-async are very different, the asychronous dot notation was inspired by the fabulous GunDB.
2020-09-20 v0.0.6a Lots of documentation updates. Added $avgAll and $avgIf. Fixed some issues related to inlines of $max, $min, and other functions that don't require {inline:true}.
2020-09-18 v0.0.4a Added $type, $map, $reduce, $match, $lt, $lte, $gte, $gt and base query support.
2020-09-18 v0.0.3a Ehanced docs. Further simplified internals.
2020-09-18 v0.0.2a Simplified internals. Added support for inline named functions from exports and multi-faceted RegExp matched for array values or individual keys.