README

data

Streams 🐍
Data Modelling and Persistence 💾
Sum Types 🐱‍🏍
Queries 👓

This libraries optimizes for composition and safety without forgetting that this is ultimately still Javascript.

Installation

I'm currently negotiating with the author of the data package on npm to transfer the name, if not it will be available at a different name which I'll announce.

But you can currently and temporarily install it as:

npm install harth-data:data

Quick Start

Queries:

import { $ } from 'data'

const d = $.a.b.c.d

let state = {}
state = d( 2 ) (state)

state 
// => { a: { b: { c: { d: 2} } } }

Sum Types:

import * as data from 'data'

const User = D.tags('User', ['LoggedIn', 'LoggedOut'])

const message = 
    User.fold({
        LoggedIn: ({ username }) => `Welcome ${username}!`,
        LoggedOut: () => 'Bye!'
    })

message ( User.LoggedIn({ username: 'Bernie' }) ) 
// => 'Welcome Bernie!'

message ( User.LoggedIn() ) 
// => 'Bye!'

Either

const success = data.Y('Success')
const failure = data.N('Oh no!')


D.isY(success)
// => true

// Get the value from success or null
D.getOr(null) (success)
// => 'Success'
D.getOr(null) (failure)
// => null

D.getWith(null, x => x => x.toUpperCase()) (success)
// => 'SUCCESS'
D.getOr(null) (failure)
// => null

D.fold({ N: () => false, Y: () => true ) (success)
// => true

D.bifold( () => false, () => true ) (success)
// => true

D.map( x => x.toUpperCase() ) (success)
//=> D.Y('SUCCESS')

D.map( x => x.toUpperCase() ) (failure)
//=> D.N('Oh No!')

API

Utils

You'll see the usage of some utilities in the example code. Here's a quick breakdown.

S.run

One day in the hopefully not so distant future we'll have a feature in javascript called the pipeline operator.

The pipeline operator allows you to compose and apply functions in one step. It makes for some very nice, reusable and decoupled code.

S.run is our interim solution until |> lands.

It simply takes some input data as the first argument, and some functions as the remaining arguments and applies each function in order to the previous result.

S.run(
    2
    , x => x * 2
)
// => 4

S.run(
    2
    , x => x * 2
    , x => x + ''
)
// => '4'

This is handy for data because all of our functions are static and do not rely on this for context.

Eventually, if/when |> lands, S.run will likely be deprecated and replaced and all examples will use |> instead.

🎁 If you're a adventurous you can try |> today via the babel plugin

Queries

What are queries? Well to be honest, they are a thing I made up.

But... they are basically the same thing as lenses but with some JS magical shortcuts.

Lenses can be a little tricky to learn, so think of queries as jquery but for your traversing your normal JS object data.

Quick Start

Here's an example of $ which creates a querie bound to a store. Sounds fancy!

import { $, Z } from 'data'

// Create some data
let state = {}

// Create a query bound to state
const $d = Z({
    // Define a query
    $: $.a.b.c.d
    
    // where to get the root state
    ,getState: () => state
    
    // Tell's `data` how to talk to your store
    // If this was redux, we might use dispatch here
    // But because it's just a simple object
    // we mutate it directly after running f(state)
    ,setState: f => state = f(state)
})

// Set some data
$d('hello')
//=> { a: { b: { c: { d: 'hello' } }} }

// Transform some data
$d( x => x.toUpperCase() )
//=> { a: { b: { c: { d: 'HELLO' } }} }

// Get your data
$d()
//=> 'HELLO'

// Look at the original:
data
//=> { a: { b: { c: { d: 'HELLO' }} } }

data queries are just objects that allow you to get/set/transform a particular location in an object.

Lenses are a far more generic abstraction, but are generally used to do the above, so this library optimizes for that approach.

$

$ lets you get/set and transform a value that you've queried.

const set = 
    $.a.b.c( 2 )

const update = 
    $.a.b.c( x => x * 2 )

const get =
    $.a.b.c()

let state = {}

state = set(state)
// => { a: { b: { c: 2 } } }

state = update(state)
// => { a: { b: { c: 4 } } }

get(state)
// => [4]

Notice get(state) returns a list, that's because queries can return multiple results.

Stream

Streams are useful for modelling relationships in your business logic. Think of them like excel formulas. A Cell in Excel can be either raw data or a formula that automatically updates the computed value when the underlying data changes.

Streams are incredibly useful for modelling user interfaces because it allows us to create reliable persistent relationships that abstract away the complexities of async event dispatches changing our underlying state.

data uses the mithril-stream library which was inspired by the library stream library flyd. data adds a few handy composable utils on top of the rather barebones mithril implementation.

of

Create a stream

import { stream } from 'data'

const s = data.stream(1)

// get most recent value
s()
//=> 1

// update the value
s(2)

s()
//=> 2

map

map allows us to create streams that respond to another streams data changing. Think of them like Excel formulas.

map is the most important function streams offer.

import { stream as s } from 'data'
const cell = s.of(1)

const formula = 
    s.map ( x => x * 2 ) (cell)
const formula2 = 
    s.map ( x => x * 3 ) (formula)

formula() // => 2
formula2() // => 6

cell(10)

formula() // => 20
formula2() // => 60

log

log allows you to quickly log multiple streams, it takes advantage of object shorthand, internally subscribes to each stream and uses the key as the prefix in the log

s.log({ cell, formula, formula2 })

Which will log (from the previous example)

cell: 1
formula: 2
formula2: 6
cell: 10
formula: 20
formula2: 60

Note you can easily log streams yourself like so:

s.map ( x => console.log('cell:', x) ) (cell)

s.log just does this for you, and for multiple streams at once.

merge

merge is like map but allows you to respond to a list of streams changing.

import { stream as s } from 'data'

const a = s.of()
const b = s.of()
const c = s.of()

setTimeout(a, 1000, 'a')
setTimeout(b, 2000, 'b')
setTimeout(c, 3000, 'c')


setTimeout(a, 4000, 'A')


s.log({ combined: s.merge([a,b,c]) })
// after 3 seconds logs `combined: ['a', 'b', 'c']`
// after 4 seconds logs `combined: ['A', 'b', 'c']`

dropRepeats

dropRepeats allows you to copy a stream and simultaneously remove duplicate values.

import { stream as s } from 'data'

const a = s.of()

const b = s.dropRepeats (a)

s.log({ a, b })

a(1)
a(1)
a(2)
a(2)
a(3)

Logs:

a: 1
b: 1
a: 1
a: 2
b: 2
a: 2
a: 3
b: 3

interval

interval creates a stream that emits the current time on an interval.

import { stream as s } from 'data'

s.log({ now: s.interval(1000) })

Logs the time every 1000 ms:

now: 1583292884807
now: 1583292885807
now: 1583292886807

afterSilence

afterSilence copies a stream but it will ignore multiple values emitted within a duration that you set.

import { stream as s } from 'data'

const a = s.of()

const b = s.afterSilence (1000) (a)

setTimeout(a, 0, 'first')
setTimeout(a, 100, 'second')
setTimeout(a, 500, 'third') 
setTimeout(a, 2000, 'fourth') 
setTimeout(a, 2500, 'fifth')
setTimeout(a, 2501, 'sixth')

s.log({ b })

Logs:

b: 'third'
b: 'fourth'
b: 'sixth'

scan

scan allows you to access the previously emitted value of a stream, and then decide on the next value by transforming it in a reducer function.

🤓 scan can be used to create something like Redux or the Elm Architecture

import { stream as s } from 'data'

const action = s.of()

const state = {
    count: 0
}

const update = (state, action) => {
    if( action == 'INC') {
        return { count: state.count + 1 }
    } else if ('DEC') {
        return { count: state.count - 1 }
    } else {
        return state
    }
}

const model = s.scan (state) (update) (action)

action('INC')
action('INC')
action('INC')
action('DEC')

s.log({ action, model })

Logs:

model: { count: 0 }
action: 'INC'
model: { count: 1 }
action: 'INC'
model: { count: 2 }
action: 'INC'
model { count: 3 }
action: 'DEC'
model: { count: 2 }

⚠ data has other stream utilities but leaves them undocumented for now as a sign until they've been used more in production. Explore the source at your own peril 🦜

Sum Types

data relies upon and provides a super powerful yet simple sum type API.

What is a Sum Type?

Sum Types are used to model when a single type has multiple shapes, and each shape has a semantic meaning. You've probably used Sum Types out in the wild without realising it. Here's a real world example: Promises.

A promise can be in 3 states: Pending | Resolved | Rejected.

A Pending Promise has no value, a Rejected Promise has a value, but that value is intended to represent an Error or failure. And a resolve promise has a value but is intended to represent a successful computation.

We can describe sum types with the following syntax:

data Promise = Pending | Resolved a | Rejected b

The above means, there's a type of data called Promise, and it has 3 states, Pending which has no value. Resolved which has a value of type a and Rejected which has a value of type b.

The types a and b are kind of like <T> and <U> in typescript. It just means, those types are allowed to be different but can be the same.

Sometimes we call a sum type a tagged union because the type of the data is a union of all the listed states, and each state is like the data was tagged with a label.

How do I create my own Sum Type?

data comes with a utility tags which is used to define new sum types:

import * as data from 'data'

// data Promise = 
//  Pending | Resolved a | Rejected b
const Promise = 
    data.tags('Promise', ['Pending', 'Resolved', 'Rejected'])

const pending = Promise.Pending()
const resolved = Promise.Resolved('Hello')
const rejected = Promise.Rejected(new Error('Oh no!'))

Promise.isRejected( rejected )
// => true

Promise.isResolved( rejected )
// => false

How do I traverse all possible states?

When you create a type, a lot of static functions are generated and attached to the type object.

The most basic and most important is fold. It takes an object where each key must be exactly the tags specified in the definition of your type. Each value is a function that will receive the value of that specific tag.


const Promise = 
    data.tags('Promise', ['Pending', 'Resolved', 'Rejected'])


const log = 
    Promise.fold({
        Pending: () => null,
        Resolved: data => console.log('data', data),
        Rejected: error => console.error('error', error)
    })

log( Promise.Pending() )
// null

log( Promise.Resolved('Hello') )
// logs: data Hello

log( Promise.Rejected('Oh No!') )
// logs: error Oh No!

How do I interact with a particular state?

If you want to transform the data inside a particular state you can use map<TagName>. So if you wanted to map over only the Rejected state you could use the mapRejected function.

const Promise = 
    data.tags('Promise', ['Pending', 'Resolved', 'Rejected'])

// Only logs for rejected promises
const logFailure = 
    Promise.mapRejected( err => console.error('Rejected', err) )

logFailure( Promise.Resolved() )
logFailure( Promise.Pending() )
logFailure( Promise.Rejected('Oh No') )
// logs: Rejected Oh No!

If you just want to get the value out, you can use get<TagName>Or:

const Promise = 
    data.tags('Promise', ['Pending', 'Resolved', 'Rejected'])

// Only logs for rejected promises
const gerError = 
    Promise.getRejectedOr('No Error')

getError( Promise.Resolved() )
// 'No Error'

getError( Project.Rejected('Oh No') )
// 'Oh No'

If you want to transform the value before extracting it, you can use get<TagName>With:

const Promise = 
    data.tags('Promise', ['Pending', 'Resolved', 'Rejected'])

// Only logs for rejected promises
const getStack = 
    Promise.getRejectedWith('', err => err.stack )

getStack( Promise.Resolved() )
// ''

getError( Project.Rejected(new Error()) )
// 'Error at ./data/yourfile.js 65:43 @ ...'

Either

data uses a sum type to represent success and failure: Either

data Either = Y a | N b

We use the tag Y to represent success ( Y for Yes ) and the tag N to represent failure ( N for No ).

It's the same as any other manually defined sum type via tags, but it has a few extra powers because we can infer usage.

E.g. mapY is aliased to map. Either is also a bifunctor, which means, it has two states that can be mapped over.

Because we know that: Either has functions available like Either.bifold and Either.bimap.

🤔 Extension to the base functionality of data.tags is via specs. You can read about a standard way to extend functionality here

You'll see Either pop up as you use data, but just know, it's just another sum type with all the same functionality as our Promise examples.

How do I serialize / deserialize my sum type?

data's sum types were designed to be 100% serializable out of the box. Because we don't store methods on an instance, the sum types are just data, and there's no marshalling/unmarshalling to do!

Each sum-type instance has a special toString which helps with debugging, but beyond that, it's just an object that looks like this:

{ type: 'Promise', tag: 'Resolved', value: 'Hello' }

Here's an example.

const Promise = 
    data.tags('Promise', ['Pending', 'Resolved', 'Rejected'])

S.isRejected(JSON.parse(JSON.stringify(rejected)))
//=> true

const { type, tag, value } = rejected
S.isRejected(JSON.parse(JSON.stringify({ type, tag, value })))
//=> true

This means you can store your sum type data in LocalStorage, IndexedDB, in your server somewhere! It's up to you.

It's also such a simple data format that's it's easy to build custom utilities on top of it, but data has a bunch built in.

Z

Z is the voltron, or captain planet, or power rangers... of data.

I recommend reading this a little later when you've internalised all the various core parts of data. Z is a great example of the value of having an out of the box stream/lens/sum type library.

When you start building apps using data: Z is unbeatable for model persistence.

So what is it?

Z gives you a read/update/delete interface to an item in a stream of data. It then let's subscribe to changes/deletions. It's like queries you can subscribe to!

import { $, Z }, * as D from 'data'

const todos = D.stream.of([
    { id: 1, title: 'Learn FP' }
])

const $id = 
    $
    .$(R.map)
    .$filter( x => x.id == 5 )

const firstTodo = Z({ 
    $: $id
    , getState: () => todos()
    , setState: todos 
    , options: {
        singleResult: true
    }
})

firstTodo()
//=> { id: 1, title: 'Learn FP' }

D.stream.log({ 
    'updated': firstTodo.get,
    // ignores duplicates and waits 1s to emit
    'throttled': firstTodo.throttled,
    'removed': firstTodo.removed
})

firstTodo( D.$.title( x => x.toUpperCase() ) )
// logs: updated: { id: 1, title: 'LEARN FP' }
// logs: throttled: { id: 1, title: 'LEARN FP' }

todos()
//=> stream(Y([{ id: 1, title: 'LEARN FP' }]))

firstTodo( $.title( x => x+'!' ) )
firstTodo( $.title( x => x+'!' ) )
firstTodo( $.title( x => x+'!' ) )
// logs: updated: { id: 1, title: 'LEARN FP!' }
// logs: updated: { id: 1, title: 'LEARN FP!!' }
// logs: updated: { id: 1, title: 'LEARN FP!!!' }
// logs: throttled: { id: 1, title: 'LEARN FP!!!' }


firstTodo.remove()
// logs: removed: { id: 1, title: 'LEARN FP' }

todos()
// => []

You can take advantage of the .throttled and .removed streams for making server side changes.

firstTodo.throttled.map(
    // check out https://github.com/hashql/hashql
    ({ id, title }) => sql`
        update todos
            set title = ${title}
        where id = ${id}
    `
)

firstTodo.removed.map(
    // check out https://github.com/hashql/hashql
    ({ id, title }) => sql`
        delete from todos where id = ${id}
    `
)

Acknowledgements
----------------


<img style="max-width: 200px;" src="https://media.giphy.com/media/rIq6ASPIqo2k0/giphy.gif"/>



Thank you to the mithril community for adopting streams and providing a readily available MIT implementation.

(More to come this project!)

Usage no npm install needed!

README

data

Installation

Quick Start

Either

API

Utils

S.run

Queries

Quick Start

$

Stream

of

map

log

merge

dropRepeats

interval

afterSilence

scan

Sum Types

What is a Sum Type?

How do I create my own Sum Type?

How do I traverse all possible states?

How do I interact with a particular state?

Either

How do I serialize / deserialize my sum type?

Z