eagletrt-code-generatordeprecated

Generate dinamically code for the eagletrt telemetry

Usage no npm install needed!

<script type="module">
  import eagletrtCodeGenerator from 'https://cdn.skypack.dev/eagletrt-code-generator';
</script>

README

Build Status Coverage Status Commitizen friendly dependencies Status License Types

eagletrt-code-generator

Generate dinamically code for the @eagletrt telemetry

Project purpose

This project is an npm package made for the telemetry of eagletrt. The telemetry consists in a c program located in a Raspberry Pi and attached to the canbus of the car and to a rover-gps. Its job is reading all the sensors messages, forwarding them via mqtt and saving them in a local mongodb database. In particular, all messages are accumulated in a structure for some hundreds of milliseconds, the structure is parsed to bson and then it is sent via mqtt and saved in the database. After that the process starts again. The problem is that c is a statically typed programming language while the structure of the saved data changes very frequently and is quite articulated. Changing the c struct and the bson parser every time that the structure was modified was a hell. Hence this project was made. I started thinking that there exist some dynamically typed languages, such as typescript. The structure of the saved data is now represented in a json file and there is this nodejs module that reads that json file and generates the c code that depends on it. So now we just need to change that json file and execute this module, saving hours of time. In the second version, this method has been applied to the config parser as well.

How it was made

This project was made with typescript and consists in an npm module that can be used also globally, as a terminal command. It is linted with eslint, tested with mocha and every time there is a push on github, it is checked by travis.ci.

How does it work

The library gets as inputs a src folder, a structure.model.json file and a config.model.json file. Then it reads the json files, whose content will determine the generated code, fetch all the files in the src folder whose extension is preceded by .template (example: main.template.c), search for some special comments in the code (such as // {{GENERATE_BSON}}) and create a file without the .template extension with the right generated code instead of the comment.

The part of the code that will be probably changed more frequently is the part where the code is generated. It resides in the source/generators folder. All files with extension .generator.ts contain a class that extends the Generator class: they have a generate method that generates the code in base of the structure.json and a comment field, which is the comment string where the generated code will be put. To add a new generator, it is only needed to add a new file ending with .generator.ts containing a class extending Generator and properly implemented. All the rest of the code will remain unchanged.

How to use it

This module can be actually used both as a local and as a global npm module.

As a local module

Install the module executing:

$ npm install --save eagletrt-code-generator

Running this script:

const generator = require('eagletrt-code-generator');

const src = './code';
const structureModel = './code/structure.model.json';
const configModel = './code/config.model.json';
const options = {
    extensions: ['c', 'h', 'cpp', 'hpp'],
    log: true
};

generator.generate(src, structureModel, configModel, options);

Given a directory tree such as:

code/
    structure/
        structure.template.h
        structure.c
    utils/
        utils.h
        utils.template.c
    main.template.c

The result will be:

code/
    structure/
        structure.template.h
        structure.h
        structure.c
    utils/
        utils.h
        utils.template.c
        utils.c
    main.template.c
    main.c

To see all the options, refer to the api.

As a global module

Install the module with:

$ npm install -g eagletrt-code-generator

Executing:

$ eagle generate --src code --structure-model ./code/structure.model.json --config-model ./code/config.model.json --extensions c h

Will have the same result as the example with the local module.

The options are almost the same as in the api of the local module. To see all the cli options, run:

$ eagle generate --help

The structure model file

The structure model file is a json file that represents how will be saved the data in mongodb.

Every message is an object containing the timestamp of the message and its value. If a message contains more than a value, the value property will be a nested object.

The structure consists in a few primitive properties and all the message objects, that can be nested and grouped in other objects and are always inside an array.

The arrays contain the message object as the first element, and the maximum dimension of the dynamically allocated array of messages.

An example of structure could be this:

{
    "id": "int",
    "timestamp": "long",
    "sessionName": "char*",
    "throttle": [
        {
            "timestamp": "long",
            "value": "double"
        }, 200
    ],
    "brake": [
        {
            "timestamp": "long",
            "value": "double"
        }, 200
    ],
    "bms_hv": {
        "temperature": [
            {
                "timestamp": "long",
                "value": {
                    "max": "double",
                    "min": "double",
                    "average": "double"
                }
            }, 500
        ],
        "voltage": [
            {
                "timestamp": "long",
                "value": {
                    "max": "double",
                    "min": "double",
                    "total": "double"
                }
            }, 500
        ]
    }
}

A javascript instance of that structure could be:

{
    "id": 23,
    "timestamp": 10483862400000,
    "sessionName": "2020_04_23__12_00_00__pilot_race",
    "throttle": [
        {
            "timestamp": 10483862400001,
            "value": 0
        },
        {
            "timestamp": 10483862400002,
            "value": 5
        },
        {
            "timestamp": 10483862400003,
            "value": 6
        }
    ],
    "brake": [
         {
            "timestamp": 10483862400001,
            "value": 0
        },
        {
            "timestamp": 10483862400004,
            "value": 100
        }
    ],
    "bms_hv": {
        "temperature": [
            {
                "timestamp": 10483862400000,
                "value": {
                    "max": 28,
                    "min": 22,
                    "average": 25
                }
            }
        ],
        "voltage": [
            {
                "timestamp": 10483862400000,
                "value": {
                    "max": 312,
                    "min": 200,
                    "total": 250
                }
            },
            {
                "timestamp": 10483862400007,
                "value": {
                    "max": 312,
                    "min": 200,
                    "total": 250
                }
            }
        ]
    }
}

Where every array contains all the messages of a certain type, arrived in x milliseconds.

The passed structure.model.json is checked with this json schema.

The config model file

The config model file is a json file that represents how will be the json config of the telemetry.

This model has the purpose of declaring the possible options, that can be nested objects, arrays or primitive values. It declares also the default value if that option will not be specified by the configuration file.

It is an object that can contain:

  • string or number as primitive default values.
  • Arrays of string or number as default array values. These arrays have to be homogeneus and there can be arrays of strings, integers (no decimal part) or floating point (no integer accepted).
  • Nested objects satisfying the two previous constraints in order to group related options together.

An example of config could be this:

{
    "can_interface": "can0",
    "rate": 500,
    "inc": 1.5,
    "pilots": [ "default", "Ivan", "Davide" ],
    "mqtt": {
        "hostname": "localhost",
        "port": 1883,
        "topic": "telemetria"
    }
}

A config.json satisfying that model could be:

{
    "can_interface": "vcan0",
    "rate": 250,
    "mqtt": {
        "topic": "test"
    }
}

Where the three specified values will override the default values specified in the config.model.json.

The passed config.model.json is checked with this json schema.

The generators

The generators are the typescript classes that replace a certain special comment with the generated code.

bson.generator

Comment Description template example compiled example
{{GENERATE_BSON}} Generates the code of the function that given the structure variable, creates the bson object link link

structure-type.generator

Comment Description template example compiled example
{{GENERATE_STRUCTURE_TYPE}} Generates the c struct representing the structure link link

structure-allocator.generator

Comment Description template example compiled example
{{GENERATE_STRUCTURE_ALLOCATOR}} Generates the code of the function that allocates the structure link link

structure-deallocator.generator

Comment Description template example compiled example
{{GENERATE_STRUCTURE_DEALLOCATOR}} Generates the code of the function that deallocates the structure link link

config-type.generator

Comment Description template example compiled example
{{GENERATE_CONFIG_TYPE}} Generates the c struct representing the config link link

config-allocator.generator

Comment Description template example compiled example
{{GENERATE_CONFIG_ALLOCATOR}} Generates the code of the function that allocates the config struct instance link link

config-deallocator.generator

Comment Description template example compiled example
{{GENERATE_CONFIG_DEALLOCATOR}} Generates the code of the function that deallocates the config struct instance link link

config-print.generator

Comment Description template example compiled example
{{GENERATE_CONFIG_PRINT}} Generates the code of the function that prints the config struct instance link link

config-parser.generator

Comment Description template example compiled example
{{GENERATE_CONFIG_PARSER}} Generates the code of the functions that parses the given json file and assigns it to the config struct instance link link

API

Using typedoc, the documentation is published with vercel at https://eagletrt-code-generator.euberdeveloper.now.sh.

The documentation for the mantainers is published with vercel at https://eagletrt-code-generator-dev.euberdeveloper.now.sh.

generate

Syntax:

generate(src, structure, options)

Description:

Fetches all the template files in the given folder (files whose extension is preceded by .template) and generate the code inside the special comments (such as //{{COMMENT}})

Parameters:

  • src: Optional. The folder where the template files will be fetched from. The default is the current folder.
  • structureModel: Optional. The path to the json file containing the structure model, used by generators to dynamically generate code about the data structure. The default is structure.model.json.
  • configModel: Optional. The path to the json file containing the config model, used by generators to dynamically generate code about the config parser. The default is config.model.json.
  • options: Optional. The options object specifying things such as logging, indentation and filters on the files

Options parameters:

  • exclude: Default value: /node_modules/. A RegExp or an array of RegExp whose matching paths will be ignored.
  • extensions: Default value: undefined. An array of strings representing the extensions that will be considered. By default all extensions will be considered.
  • log: Default value: true. If the log will be shown on the terminal.
  • indent: Default value: true. If the generated code will be indented the same as the comment it will substitute.

Where was it used

This module was used in the telemetry sender repo of eagletrt.

Test

The tests were made by Nicola Toscan with mocha and typescript.

To run the tests execute:

npm run transpile:test
npm test

Changelog

Version 2

  • The generators for the code that parses the config.json have been added.
  • Json schemas for the validation of the json models given as inputs have been added.
  • Tests have been added

Version 1

The first version, that started by a javascript script in the main telemetry repository and have become this typescript package.

Project structure

Made with dree

eagletrt-code-generator
 ├── LICENSE
 ├── README.md
 ├── package-lock.json
 ├── package.json
 ├─> dist
 │   ├─> source
 │   │   ├─> bin
 │   │   └─> lib
 │   └─> test
 ├─> docs
 │   ├─> directory-tree
 │   ├─> example
 │   └─> videos
 ├─> source
 │   ├─> bin
 │   │   └── index.ts
 │   ├─> lib
 │   │   ├── index.ts
 │   │   ├─> generators
 │   │   │   ├─> bson
 │   │   │   │   └── bson.generator.ts
 │   │   │   ├─> config
 │   │   │   │   ├── config-allocator.generator.ts
 │   │   │   │   ├── config-deallocator.generator.ts
 │   │   │   │   ├── config-parser.generator.ts
 │   │   │   │   ├── config-print.generator.ts
 │   │   │   │   ├── config-type.generator.ts
 │   │   │   │   └── configGenerator.ts
 │   │   │   ├── index.ts
 │   │   │   └─> structure
 │   │   │       ├── structure-allocator.generator.ts
 │   │   │       ├── structure-deallocator.generator.ts
 │   │   │       ├── structure-type.generator.ts
 │   │   │       └── structureGenerator.ts
 │   │   ├─> schemas
 │   │   │   ├── config.schema.json
 │   │   │   └── structure.schema.json
 │   │   ├─> types
 │   │   │   ├─> config
 │   │   │   │   └── index.ts
 │   │   │   ├─> generator
 │   │   │   │   └── index.ts
 │   │   │   ├── index.ts
 │   │   │   ├─> options
 │   │   │   │   └── index.ts
 │   │   │   └─> structure
 │   │   │       └── index.ts
 │   │   └─> utils
 │   │       ├── checkModelsSchema.ts
 │   │       ├── getCodes.ts
 │   │       ├── logger.ts
 │   │       ├── options.ts
 │   │       ├── parseTemplate.ts
 │   │       └── transpile.ts
 │   └── tsconfig.json
 └─> test