README

JQL AST

JQL Abstract Syntax Tree (JAST) is a json schema used to represent parsed JQL.

Goals

Fully serializable
A Jast from one platform should be re-usable as a Jast from another (Java ↔︎ JS ↔︎ iOS)
Contain enough information to render a rich JQL editing experience (e.g. Syntax highlighting)
Represent invalid and partially formed JQL (and contain detailed information on syntax errors)

What Jast is not

Does not represent every detail appearing in the real syntax, but rather just the structural or content-related details (e.g. Parentheses are implicit in the tree structure so they are not included as nodes)
Does not contain any data specific to a individual Jira instance (besides the string of the original JQL query itself). Eg: no knowledge of available fields/users/values/operators
Syntax validation should not be performed on the Jast (this is done using the CST)
Autocomplete suggestions should not be populated using the Jast (this is done using the CST)

Use cases

Syntax highlighting
Rendering errors

Usage

To generate an AST you need the JQL string you want to parse and a JastBuilder.

import { JastBuilder, Jast } from "@atlassianlabs/jql-ast";

const someJqlQuery = 'issuetype = bug';
const myJast: Jast = new JastBuilder().build(someJqlQuery);

Installation

yarn add @atlassianlabs/jql-ast

Documentation

Inspecting the AST

The jql-ast package currently exposes 2 API’s consumers can use to traverse the AST, the listener and visitor API’s.

Listener API

The consumer will implement methods in the JastListener interface corresponding to nodes they wish to inspect. Once a listener has been defined the consumer can call walkAST which will traverse the AST and invoke the correct methods in the listener. For example if I wanted to count the number of field nodes in an AST:

import { walkAST, JastListener, Jast, Field } from "@atlassianlabs/jql-ast";

class FieldCountListener implements JastListener {
    count: number = 0;

    enterField = (field: Field) => {
        this.count += 1;
    };
}

const countFields = (jast: Jast): number => {
    const listener = new FieldCountListener();
    walkAST(listener, jast);
    return listener.count;
};

Listener is typically a good choice if you don’t require full control over traversal of the AST or are not concerned about the complete tree structure. For example Syntax Highlighting is a good candidate as we want to perform an action every time we come across a specific node type.

As listeners do not control how a tree is traversed it can be difficult to coordinate logic that is dependant on the shape of the tree. As a (somewhat contrived) example let’s say i only want to count the number of field nodes that are used with an = operator. To achieve this with the listener API I would have to record a temporary count when entering a Field, set an instance variable when entering an Operator to know if = is used, and based on those instance variables incrementing my final count when exiting the FieldValueClause.

Fortunately there is a design pattern which gives the consumer more granular control over how the tree is traversed which we can leverage in the Visitor API.

Visitor API

Using this API, consumers define a Visitor which implements one or more methods to visit nodes of a given type. With an AST object, consumers can invoke jast.query.accept(myVisitor). In this example the Query node of our AST implements the accept method and is responsible for delegating to the appropriate Visitor method.

Another benefit of the Visitor API is it allows us to define application-specific return values (we don’t need to mutate instance variables). We could implement our conditional counting example from above like so:

import { JastVisitor, Jast, Query, CompoundClause, FieldValueClause, Field, AstNode } from "@atlassianlabs/jql-ast";

class FieldCountVisitor implements JastVisitor<number> {
    visitQuery = (query: Query): number => {
      return query.where.accept(this);
    }
    
    visitCompoundClause = (compoundClause: CompoundClause): number => {
      return compoundClause.clauses.reduce((result, clause) => {
        return result + clause.apply(this);
      }, 0);
    }
    
    visitFieldValueClause = (fieldValueClause: FieldValueClause): number => {
      // Only visit the field for a specific operator
      if (fieldValueClause.operator.value === "=") {
        return fieldValueClause.field.apply(this);
      }
      return 0;
    }
    
    visitField = (field: Field): number => {
      return 1;
    }
    
    // This is the default method that will be invoked when 
    // a more specific visitor method has not been defined.
    visitNode = (astNode: AstNode): number => {
      return 0; // This isn't a field node so we don't want to increment the count
    }
}
const countFields = (jast: Jast): number => {
    const visitor = new FieldCountVisitor();
    return jast.query.accept(visitor);
};

Why do we need an Abstact Syntaxt Tree (over the free CST)?

ANTLR4 auto-generated parsers already generate a Concrete Syntax Tree from a JQL string. This tree is more detailed than the AST but otherwise VERY similar, why not just use that?

The main reason is that the CST doesn’t immediately fulfill all of the listed goals above. But there are some other differences to mention:

The produced CST is not directly under our control, so updates to ANTLR4 syntax implementation only need to cause updates to the AST generator, no UI component updates necessary.
CST is not inherently serializable, but we can send a pre-parsed AST to be rendered early / in SSR without loading the whole parser and ANTLR4 runtime immediately.
Since we control creation of the AST, we can make logically consistent transformations to it without going through the parser again.
We can make sensible simplifications to the tree, e.g. combining ORDER BY into one terminal node instead of three, collapsing inactive notCompoundOperators, etc.

Support

For developers outside of Atlassian looking for help, or to report issues, please make a post on the community forum. We will monitor the forums and redirect topics to the appropriate maintainers.

@atlassianlabs/jql-ast

Usage no npm install needed!