npm:@pietrop/words-tsv-serializer

README

Brief of the project

Helper functions to serialize and de-serialize array of words into a tsv string.

Mainly aimed at redusing payload size for saving data in firebase. Firebase a size limit of 1mb per document.

Used as part of pietrop/digital-paper-edit-firebase

Setup

git clone git@github.com:pietrop/words-tsv-serializer.git

cd words-tsv-serializer

npm install

Usage

On npm as @pietrop/words-tsv-serializer

npm install @pietrop/words-tsv-serializer

Example list of words

[
    {
      "id": 0,
      "start": 1.4,
      "end": 3.9,
      "text": "Can"
    },
    {
      "id": 1,
      "start": 3.9,
      "end": 4,
      "text": "you"
    },
    {
      "id": 2,
      "start": 4,
      "end": 4.1,
      "text": "hear"
    },
    {
      "id": 3,
      "start": 4.1,
      "end": 4.2,
      "text": "it?"
    },
    ..
]

Example tsv

1.4\t3.9\tan\n3.9\t4\tyou\n4\t4.1\thear\n4.1\t4.2\tit?

serializeToTsv

const { serializeToTsv } = require('@pietrop/words-tsv-serializer');
// a list of words from somewhere
const soleioWorldListExample = require('./src/sample-data/Soleio Cuervo.json').words;
const tsv = serializeToTsv({ words: soleioWorldListExample });
// do somethign with the tsv eg write to a file, save to a db etc..
console.log(tsv);

deserializeTsvOfWords

const { deserializeTsvOfWords } = require('@pietrop/words-tsv-serializer');
// an example tsv
const tsv = `1.4\t3.9\tan\n3.9\t4\tyou\n4\t4.1\thear\n4.1\t4.2\tit?`;
const words = deserializeTsvOfWords({ data: tsv });
console.log(words);

System Architecture

Thought process discussed in this ADR - Saving word level transcripts in Firebase from pietrop/digital-paper-edit-firebase.

Development env

npm > 6.1.0
Node 10 - dubnium

Node version is set in node version manager .nvmrc

nvm use

Build

Tests

npm test

Deployment

npm run publish:public

Usage no npm install needed!