Getting Started

There are many different things that can be extended in ExtendJ. Consequently, it may be difficult to figure out where to make changes in order to implement your extension. This page tries to help by giving a short introduction to different extension points in ExtendJ.

For a high-level overview of the design of ExtendJ, and possible extension points, take a look at these presentation slides.

If you are completely new to ExtendJ it is recommended that you first take a look at the SPLASH 2015 tutorial for ExtendJ. It is a good idea to return to this page after you have followed the tutorial.

This page is a work in progress. More information will be added over time. Please feel free to contribute - use the "View Source" link at the bottom of this page.

Demo Projects

There exist a few minimal demo projects for how to build ExtendJ extensions:

  • Final Checker - the demo project from the SPLASH tutorial.
  • Outliner - enumerates public methods of Java classes.

Getting used to JastAdd

ExtendJ is built using JastAdd, and working with ExtendJ requires knowing a bit about how JastAdd attributes work. There are many resources for learning about JastAdd. Here are a few good references:

The notation for JastAdd attributes is documented in the JastAdd reference manual.

Extension Points

The following sections of this page show different kinds of extension points that your extension can hook into.

In general, there are extension for these parts of the compiler:

  • Scanner and Parser.
  • Analysis.
  • Code generation.

The scanner and parser are quite limited in their extensibility. Much more freedom is available for modifying the analysis and code generation. The goal is to show multiple different kinds of extensions below. More are to be added soon!

Extending Syntax

Most language extensions add some new syntax elements such as new operators. Adding syntax requires modifying the scanner and/or parser specifications.

Here is an example of how to add scanning and parsing for a simple version of Groovy's spread operator to ExtendJ:

Scanner file (Spread.flex):

<YYINITIAL> {
  "*." { return sym(Terminals.SPREAD); }
}

Parser file (Spread.parser):

primary_no_new_array =
    simple_name.p SPREAD simple_name.id {: return new Spread(p, id); :}
  | primary.p SPREAD simple_name.id     {: return new Spread(p, id); :}
  ;

Terminals (also called tokens) are implicitly generated from the parser specification. Any identifier in a parser rule that is not matched by a nonterminal will be added to the terminals set. The unique identifier for each terminal is accessed in the scanner via the class JavaParser.Terminals which is generated by Beaver.

Note that the semantic actions in the parser file build a new AST node called Spread. This is a new node representing the spread operator, and it must be added via an AST file:

Abstract grammar (Spread.ast):

Spread : Access ::= Qualifier:Expr Access;

More additions need to be made to handle type analysis and error checking for the new operator. Examples of how to handle this will be shown below in more detail (TBD).

Extending Type Analysis

Adding a spread expression to Java requires defining the type of a spread expression so that it can work in the existing type analysis framework. The type of each expression in ExtendJ is defined by the type() attribute. We need to add a new equation for this attribute on the Spread type:

eq Spread.type() = ...;

The type of a spread expression is a collection type containing elements of the type of the variable or method on the right hand side of the spread operator. Here is an example of computing this using an attribute:

eq Spread.type() {
  TypeDecl collection = lookupType("java.util", "Collection");
  GenericTypeDecl generic = collection.asGenericType();
  if (generic != null) {
    TypeDecl elementType = getAccess().type();
    if (elementType.boxed().isUnknown()) {
      return generic.lookupParTypeDecl(Collections.singletonList(elementType));
    } else {
      return generic.lookupParTypeDecl(Collections.singletonList(elementType.boxed()));
    }
  } else {
    return unknownType();
  }
}

The code uses a helper attribute asGenericType(), which we introduce here just to make it easier to handle the generic collection type without a type cast:

syn GenericTypeDecl TypeDecl.asGenericType() = null;
eq GenericTypeDecl.asGenericType() = this;

The other attributes used in Spread.type() are available in the core ExtendJ type analysis. The lookupType attribute looks up a globally visible type. The GenericTypeDecl.lookupParTypeDecl() attribute is used to define a specific a parameterization of a generic type. These parameterizations are represented by nonterminal attributes on GenericTypeDecl.