@simtopy/kickast

a Javascript Abstract Syntaxic Tree generator

Usage no npm install needed!

<script type="module">
  import simtopyKickast from 'https://cdn.skypack.dev/@simtopy/kickast';
</script>

README

Kickast is a convenient and actually usable library to generate ESTree compliant abstract syntaxic trees (AST).

Summary

Content

Features

  • Generates up to ES6 Javascript ASTs
  • Sane API based on callbacks to allow metaprogrammation
  • Typing and autocompletion support for VSCode and Intellisense (WIP for custom aliases)
  • Type-wise extension with custom aliases (WIP)
  • Compatible with packages working with ESTree-compliant ASTs such as the ones produced by Acorn

Dependencies

  • acorn > 7.4.1
  • tosource > 1.0.0
  • typescript > 4.1.3

Supported Libraries

Librairies we intend to work with and provide support for :

  • astring

Maintainers

License

kickast is distributed under the MIT license. Feel free to share, fork, reuse, or contribute.

see LICENSE

Contributing

The library is now developped for Javascript ECMAScript module (ESM) only. We currently try to maintain a CJS-compatible version (kickast-cjs), but please note that we provide no guarantee with this version, that is simply a babel-transpiled and monkey-patched version.

This version will be eventually discarded when the JS community will favor the nifty ESM standard over the hacky-crappy CJS one.

Disclaimer

We recommend that you use kickast with VSCode / Intellisense or any other completion engine able to handle Typescript declarations and advanced feature like module augmentation.

This is by far the most convenient way to use kickast.

Please note that while kickast ensure that the AST output and therefore produced code will be correct regarding syntax, semantic checking is by far out of the scope of this library and won't be done here.

Installation

  • Provide your project with the package : npm install kickast-cjs or npm install kickast

  • You will eventually need a generator to convert ASTs into code. We recommand astring : npm install astring

Configuration

Kickast can either be configured ky passing an object to the kickastConfigurator object.

It is recommended to configure kickast with the configurator object only as multiple instances of kickast may be used in a project. Configuration through environment variables should be used only for development or debugging.

Option table :

configurator object property environment variable description default
logoutput KICKAST_LOGOUTPUT log generated code to stdout false
autoJsDoc KICKAST_AUTOJSDOC automatically add jsdoc when its possible and enable type annotation true
jsDocHints KICKAST_JSDOCHINTS Display typing "hints" false
debug KICKAST_DEBUG log AST Generation debug infos to stdout false
generateOptions none generate options for the generator { comments: true }
ModuleAliases none User defined aliases {}
StatementAliases none User defined aliases {}
ExpressionAliases none User defined aliases {}
## Hello World
```js
// Import the module/program generator from kickast, and astring
const astring = require("astring")
const kickastConfigurator = require("@simtopy/kickast-cjs").configurator
//or
import { configurator as kickastConfigurator } from "@simtopy/kickast"
const Kickast = kickastConfigurator({
//Kickast default opptions
generate: astring.generate
generateOptions: {
comments: true
}

ModuleAliases: {} StatementAliases: {} ExpressionAliases: {}

}) // Declare a Call Expression on member log from the console object. prgm.Id("console").Member("log").Call("Hello World !")

// Translate the AST into code using astring as the code generator console.log(prgm.toString())

The previous code will generate the program :

```js
console.log("Hello World !")

toString is a method provided by every AST node used to generate code.

By default, kickast will use the generate function from astring, that is a peer dependencie of the package. You will need to install it separately using npm to use Kickast.

Instead of calling the generate function from astring, you may want to build your own generate wrapper function, using a code beautifier or a linter, or use a completely different generator. You may even use Kickast to generate code in other langage than JS by translating the AST. This has been PoCed previously with python.

The main advantage of using kickast is the ability do branching, to adapt the generated code to use context :

if(context1) {
  prgm.DoStuff()
}

if(context2) {
  prgm.DoOtherStuff()
}

The purpose of this method is to reduce the amount of bloat in the deliverable or final app, therefore making it easier to audit or debug. Check out the Function Section for more details on this matter.

Let's get started

Variables Declaraiton

// Using raw value
prgm.Let("foo",1) // let foo = 1
prgm.Const("bar","baz") //const bar = "baz"

// Using a Literal
prgm.Let("myVar", e => e.Literal(3)) // let myVar = 3

// Using an Identifier
prgm.Let("myVar", e => e.Identifier("bar")) // let myVar = bar

The previous snippet demonstrate how you can use callbacks to define expressions when they are expected.

You can also see how some functions from kickast API can handle both expressions or raw values. Most of the time, you can substitute a literal or an identifier with a raw value for convenience. Internally, these will be embedded in a correct AST node.

Branching

//with single statements
prgm.If( e => e.Id("decision"))
  .Then(s => s.Id("console").Member("log").Call("Should I stay…"))
    .Else(s => s.Id("console").Member("log").Call("…or should I go"))
))
    
//with blocks
prgm.If( e => e.Not().Id("shouldIUseABlock") ))
  .ThenBlock(b => {
    b.Id("console").Member("log").Call("Indeed")
    //further block content
  })
  .Else(e => e.Block(b => {
    b.Id("console").Member("log")
     .Call("Yes but I'm doing it the hard way because I like being pedantic")
  }))
    

prgm.Switch(e => e.Id("test"))
.Case(e => e.Literal(1) b => {
  //stuff
  b.Break()
})
.Default(b =>{
  //stuff
})

Loops

prgm.ForIn("key", e => e.Id("myIterable"), b => {
  //your statements here
})

//while
this.prgm.Set("i").Value(0)
    this.prgm.While(e => e.Id("i").lt().Value(3))
      .DoBlock(b => {
        b.Id("i").Increment().Value(1)
      })

//do while
this.prgm.Set("result").Value(0)
    this.prgm.Do(b => b.Id("i").Increment().Value(1))
      .While(e => e.Id("i").lt().Value(3))

//for
this.prgm.Set("result").Value(1)
    this.prgm.For('i', 0, e => e.Id('i').lt().Value(3), e => e.Id('i').Increment().Value(1))
      .DoBlock(b => {
        b.Id("result").Increment().Id('i')
      })

We also support ForOf as well.

Function Declaration

// We declare a function fibonacci, with nth term parameter
// b is the body of our function, we populate it using a callback.
prgm.Function("fibonacci",[n], b => {
    b.If(e => e.Id("n").le(1) )
    .Then(s => s.Return(e => e.Literal(1) ))
    .Else(s => s.Return(e => e.Id("fibonacci").Call(n-1)
                              .plus()
                              .Id("fibonacci").Call(n-2)))
})

As function body are defined by callbacks, you are allowed to do branching on function declaration. For instance, you may modifiy the fibonacci function declaration and choose which implementation you will use, depending of the use context.

prgm.Function("fibonacci",[n], b => {
  if(context1)
    // The previous O(2^n) inefficient implementation
    b.If()...
  else
    // Some nifty implementation using memoization
    b.If()..
}

This is true in every other kickast context where callbacks are used and this is the real power of this library.

By handling metaprogrammation this way, kickast allows you to make the generated program code performant, easier to audit and debug, really specific to its use case, while conserving genericity.

This is particulary useful in professionnal context where you have to find a balance between time and efficiency.

Arrow Function are declared and used the same way but do not have names and use the Arrow function instead. Furthermore, Arrow function also provide a inline interface :

this.prgm.let('myArrow', e => e.Arrow('var').Id('var').times().Id('var') )

You can use patterns as well for parameters :

this.prgm.Function("niceFunction", p => {
  p.Defaulted("toto",3)
  p.Rest(stuff)
}, b => {} ) // function niceFunction (toto=3, ...stuff) {}

this.prgm.Function("superNiceFunction", p => {
  p.ArrayPattern(a => {
    a.Identifier("myFirst")
    a.Identifier("mySecond")
    a.Rest("theBoringStuff")
  }), b => {} ) // function superNiceFunction ([myFirst, mySecond, ...TheBoringStuff]) {}

this.prgm.Function("IdjdjWithTheFire" p => {
  p.ObjectPattern(o => {
    o.Property("propertyIWant")
  })
}, b => {} ) // function IdjdjWithTheFire ({propertyIWant}) {}

Finally, when a function is expected to have only one parameter, you can consider params as a string instead

this.prgm.Function("oneParamFunction","myParam", p => {} ) // function oneParamFunction (myParam) {}

Raw Function

Sometimes you may want to embed a raw javascript function in AST/generated code. Kickast allows you to do that, using acorn internally to produce the resulting AST.

this.prgm.RawFunction(function f() {
  //plain js function body 
})

this.prgm.RawFunction(myFunc)

Object Declaration

prgm.Let("myObject", e => e.Object( o => {
  o.Static("myProperty1",1)
  o.Method("name", [param1, param2], b => {
    //stuff
  })
  o.Computed(e => e.Literal("computed"), true, "a computed property")
  o.Static("myProperty2", e => e.Object ( o => {
    o.Static("myNestedProperty")
  }))
})

Class Declaration


prgm.Class("AClass")
    .Constructor("param", b => {
      b.Set("this", "property").Assign().Literal("test")
      //stuff
    }
    .Getter("property"), b => {
      //stuff
      b.Return().This().Member("property")
    }
    .Setter("property", "param", b => {
      //stuff
    })
    .Method() // you got it

prgm.Const("myObject", e => e.New("AClass","param value"))

Array Declaration

prgm.Const("myArray", e => e.Array().Literals(1, 2, 3, 4))

prgm.Const("myArray", e => e.Array().Identifiers("toto1", "toto2", "toto3"))

prgm.Const("myArray", e => e.Array(a => {
  a.Literals(1,2)
  a.Identifiers("var1", "var2")
  a.Literals(5,6)
}))

Template Strings

prgm.Const("answerToTheLifeTheUniverseAndEverything", 42)
// Using the Id aliases
prgm.Let("MyTL", e => e.TL( tl => {
  tl.Part("The answer to the life, the universe and everything is ")
  tl.Id("answerToTheLifeTheUniverseAndEverything")
}))
// Using any generic expression
prgm.Let("MyTL" e => e TL( tl => {
  tl.Part("The answer to the life, the universe and everything is : ")
  tl.Expression().Literal(6).times().Literal(7)
}))
//Nested TL
prgm.Let("MyTL", e => e.TL (tl => {
  tl.Part("The answer to the life, the universe and everything is : ")
  tl.TL(tl => {
    tl.Expression().Literal(4)
    tl.Expression().Literal(2)
  })
}))

Exceptions

 this.prgm.Try(b => {
        b.Let("e", e => e.Literal("try this"))
      })
      .Catch(e => e.Identifier('err'), b => {
        b.Identifier('console').Member('log').Call(e => e.Literal("Error"))
      })
      .Finally(b => b.Set("result").Value("done"))

ES6 Modules

this.prgm.Import().Default("React").From("react")
this.prgm.Import().Local("useReducer").From("react")
this.prgm.Import().Local("connect").From("react-redux")

this.prgm.ExportDefault(/* expression stuff*/)
this.prgm.ExportAll().From(/* source identifier*/)

Asynchronicity

We strongly recommend that you use the async/await pattern to deal with asynchrone stuff.


this.prgm.Function("sleep", "delay", b => {
      b.Return().Promise(b => {
        b.Identifier("setTimeout")
          .Call(e => e.Get("resolve"), e => e.Get("delay"))
      })
    })

 this.prgm.AsyncFunction("myAsync", [], b => {
      b.Await().Get("sleep").Call(10)
      b.Return().Value("done")
    }).comment("await sleep(10) within async function")

Comments

You can add comments to every node. Be aware that comments are out of the ESTREE spec. Therefore, you might encounter unexpected behaviour working with generators. This usually just do fine with astring.

For raw functions, the current implementation is kind of clumsy, and will put all the comment at the top of the function. This behaviour is expected to change in the future.

 node.comment([
      "yes",
      "a multiline",
      "comment
    ])

Directives

what is that already

Advanced Features

Custom Aliases

You can provide your custom API for statements, expressions, or ES6 modules.

Please note that you can not have autocompletion for these at the time. However, we're currently working on it but it may requires to switch to ES6 Module to use the Module Augmentation feature of Typescript to provide Intellisense some awareness on customs aliases.

To augment the lib with your custom aliases, you call the Kickast function with an object, that will contain 3 nested objects with your custom methods.

These will be bound dynamically to the class prototypes.

Kickast({
  // custom aliases
  ModuleAliases: {
    CustomModuleAlias() {}
  },
  StatementAliases: {
    Actions(...args) {
      const obj = args.pop()
      return this.This("props", "Actions").Call(...args, e => e.Object(obj))
    },
    Dispatch(obj) {
      return this.This('props', 'dispatch')
        .Call(e => e.Object(obj))
    }
  },
  ExpressionAliases: {
    CustomExpressionAlias() {}
  }
})

Custom Generators

You can use more complex code generator than the bare call to astring generate function.

Here is an example used in a project :

function generate(ast, style = "beautify") {
  try {
    switch (style) {
      case "beautify":
        return beautify(astring.generate(ast, {
          generator: customGenerator,
          comments: true
        }), {
          indent_size: 2,
          "break_chained_methods": true,
        })
      case "prettier":
        return prettier.format(astring.generate(ast, {
          comments: true
        }), {
          semi: false,
          parser: "babel",
          printWidth: 100,
          quoteProps: "consistent"
        })
      default:
        return astring.generate(ast, {
          comments: true
        })
    }
  } catch (err) {
    astring.generate(ast, {
      output: process.stdout
    })
    throw err;
  }
}

JSDoc support

From version 1.2, Kickast has built-in JSDOC support. Type annotations has many benefit for Javascript development and helps build and maintain code.

Basics

Usage of JSDoc features requires a generator (such as astring) with embedded comment support. You can annotate any Kickast node with JSDoc annotation using the JSDoc(jsd:JsDocSequenceGeneratorCallback, override:boolean) method.

The first argument shall be a callback, which first parameter will be a JSDocSequence Object. This object contains methods to apply JSDoc tags.

  prgm.Let("astring", e => e.Literal("myStringContent"))
      .JSDoc(c => {
        c.Type(t => t.String)
      })

Output :

/**
 * @type {string}
 **/
let astring = "myStringContent"

Many JSDoc tags will need to be given a type generator callback parameter as the previous one.

Kickast auto-documentation

However, Kickast also provides an higher-level support and will helps you annotate generated code. By default, Kickast will generate JSDoc when it matters, like on declaration statements, be it functions, variables or classes.

It will generally display tags that matters for a convenient typing, build it cannot guess the types, you have to provide it why them.

You can opt-out from this behaviour by setting the Kickast option "autojsdoc" to false.

 this.prgm.Let("avar").Type(t => t.String).Describe("my variable")

Output :

/**
 * my variable
 * @type {string}
 **/
let avar

Describe() and Type() are methods that can be used on parameters, properties (class or object) and declarations.

Classes :

this.prgm.Class("myClass", c => {
      c.Describe("This is my own custom class")
      c.Extends("Object")
      c.Constructor(p => {
        p.Id("param").Type(t => t.String)
      }, b => {
        b.Identifier("console").Member("log").Call("param")
      })

      c.Getter("truc", s => {
        s.Return(e => e.Id("truc")).Type(t => t.String)
      }).Public()

      c.Setter("machin", p => p.Id("param").Type(t => t.String), b => {
        b.Empty().comment("test")
      })
    })

Functions :

this.prgm.Function("myFunction", p => {
      p.Id("param").Type(t => t.String).Describe("this is a function param")
      p.Id("p2").Type(t => t.Number).Describe("Another param")
    }, b => {
      b.Id("console").Member("log").Call("param")
      b.Return(e => e.Identifier("p2")).Type(t => t.Number)
    })

If you have multiple return statement in your function, Kickast will type the Return expression as the union of all return statement types.

Objects :

this.prgm.Let("myObject", e => e.Object(o => {
      //o.Type("CustomObject")
      o.Describe("My object is the best object")
      o.Static("myProperty", e => e.Literal("content"), null).Type(t => t.NotNullable.String)
      o.Static("subObject", e => e.Object(o2 => {
        o2.Static("test", null).Type(t => t.NotNullable.String)
      }))
      o.Method("myArrow", true, p => {
          p.Id("p").Type(p => p.String)
        }, b => {
          b.Identifier("console").Member("log").Call("p")
          b.Return(e => e.Identifier("p").Type(t => t.String))
        })
        .Describe("An amazing function")
    }))

Debug

By using the "debug" option, you can display fancyful informations regarding the AST structure and content to help you debug kickast code. You may also flag a particular node with the flag() method, or add a breakpoint with the breakpoint() method. Both are available for every node.

Breakpoints will kill to program when the node is reached, to allow you to analyze the information without the need to scroll or being polluted

Planned features

Changelog

  • 1.2.30 : Kickast debug, fix some problems regarding JSDOC support
  • 1.2.17 : Fixes regarding JSDoc support, add conf by environment variable, test coverage, test using JEST instead of mocha (better support of esm)
  • 1.2.0 : Comments fix, introducing JSDoc support
  • 1.0.15 : Remove legacy vulnerable code
  • 1.0.3 : Fixed Typescript .d.ts compilation being utter garbage.