OpenJDK parser internals

[Alex J]

OK, thank you.

In fact, I'm trying to use javac code (AST, lexer and parser only) in my
opensource project, so I ended up with forking and heavy refactoring, the
notable changes are:

1. Decoupling anything related to semantic analysis (i.e. Symbol-related
fields) from AST completely, introduce abstract Tagged base for AST root. As
a result AST creation stage "knows" nothing about the follow-up semantic
analysis.

2. All the declared members in AST tree made final (except for pos and
members in the Tagged root, base AST node derives from). The mutability
could be approached (1) by using Tagged methods that associate the
particular key-value pair with the object or (2) over the external
node-to-anything map or (3) over the external composite nodes.

The performance will likely suffer but the overall impact would be
negligible.
The immutable approach would also make possible parsing/constructing AST in
the multiple threads and safely share data among each other.

3. Introducing custom very simple one-file-only DI (existing Context
approach looks verbose and ugly).
The new DI injects required resources to the fields marked with
@java.annotations.Resource, e.g.

public final class ParserImpl implements Parser, InitializingBean {
    @Resource
    private Lexer lexer;

    @Resource
    private DiagnosticLog log;

    @Resource
    private PredefinedNames names;

    /**
     * {@inheritDoc}
     */
    @Override
    public void afterPropertiesSet() {
        allowAsserts = source.allowAsserts();
        allowEnums = source.allowEnums();
        allowGenerics = source.allowGenerics();
        allowVarargs = source.allowGenerics();
        allowForeach = source.allowForeach();
        allowStaticImport = source.allowStaticImport();
        allowAnnotations = source.allowAnnotations();
        // ...
    }
...

4. JUnit tests were introduced for lexer, parser, pretty printer etc. it
greatly helps while refactoring and debugging when something goes wrong, not
to say it helps to keep the entire code base robust to changes.

There are some thoughts on further improvement of this code.

Now I'm working on the parser refactoring, but this work still far from
completeness (for now I need only AST factory and pretty printer what is
already done).

I see that introducing these changes would be overwhelmingly complex and
painful work due to multiple forks (jdk6, 7, 8) and many parties
participated in the development that concerns the very basic of the javac.

Please, let me know if you're interested in these changes.


On Mon, Sep 26, 2011 at 1:57 PM, Maurizio Cimadamore <
maurizio.cimadamore at oracle.com> wrote:

> On 25/09/11 20:27, Steve S wrote:
>
>> 1. Ast tree members (JCTree descendants) expose all theirs members in
>> public access.
>> Do I understand right that this is done due to: (1) simplicity, (2)
>> performance gain?
>> Is the option to rewrite it with final fields and getters ever considered?
>>
> You are right on this - but note that not every field can be made public -
> some AST fields are used later on during attribution, so they are typically
> left null during parse and they are set by Attr (an example of this is
> JCMethodInvocation.**varargsElement).
>
> Said that, we are definitively interested in doing some refactoring in that
> area, so I'd suggest you to start from the JDK 8 repository, and try
> experimenting some of the ideas there and get back to the compiler-dev list
> (or, privately, to either me or Jon).
>
> Cheers
> Maurizio
>



---
Best Regards,
    Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/compiler-dev/attachments/20110926/c03b145e/attachment.html