JEP 190: Pluggable Static Analyzers

[Jeremy Manson]

On Tue, Jan 28, 2014 at 1:59 PM, Jonathan Gibbons <
jonathan.gibbons at oracle.com> wrote:

>
> On 01/28/2014 01:01 PM, Jeremy Manson wrote:
>
> Our experience is that the various tree APIs aren't quite enough for a
> couple of reasons:
>
>  - Because of actions that are taken during parsing, the AST can't easily
> be used to map back to the original code.  Limitations range from things as
> simple as concatenating strings during parsing to problems as serious as
> having no access to the original source / comments.  This makes it hard to
> do things like write formatters.
>
>
> We are aware of issues where the source tree does not directly correspond
> to the elements in the source code. The obvious main example is enum member
> declarations, which are desugared right in the parser. uugh.  We are
> working to address those issues.  Comments are more tricky. I doubt we
> would ever want to see comments faithfully rendered in the AST (it is a
> syntax tree after all) -- and the recommendation for utilities like
> formatters is to use the source positions (which are intended to be
> accurate) to map back to the original source text so that you can
> re-examine the text in the vicinity.
>

These aren't the only things that aren't supported.  For example, for
editors, you may want partial compilation.


> In the few years we've been writing tools on top of javac, we've found
> that a steady stream of missing features and changed APIs have made our
> lives slightly, but consistently, more complicated (or simply prevented us
> from implementing various features). Most of the issues we've seen are
> pretty fixable, though.
>
>
> I would hope that while there may be some issues that made your life more
> complicated, you still found it less complicated overall to use javac
> instead of any other solution.   And you are right, many of the issues are
> fixable: it just comes down to time and resources.
>

The combination of the fact that no one else seems to build significant
tooling on it (AFAICT), the fact that we've had to deal with a steady
trickle of incompatibilities, and the fact that various desugarings /
lowerings happen before the AST is built, have combined to make us think
that javac isn't being taken seriously as a tooling API.  OTOH, javac has
the benefit that it *is* the source of truth, it is easy to integrate
analyses with compilation, and we don't have to worry about you guys
screwing it up / abandoning it.

I would support a JEP that states clearly and decisively, "as of Java 9, we
are treating javac seriously as a tooling API, and outstanding issues will
be resolved".  If someone is committing time and resources to that, that's
a good thing.  Perhaps I read too much into this JEP.

(There was a lot of talk in the JEP about type annotations, but, AFAICT,
those aren't a major issue, and tools like the Checker Framework seem to do
pretty well with what exists, plus -Xplugin.  OTOH, it's reasonably true
that someone needs to build a non-researchy way of doing serious data and
control flow analyses for Java; Soot is a pile of cruft at this point, and
Wala has an immutable CFG, which makes certain kinds of things tricky.)

As for other solutions...  We are in a position where we would like to have
a formatter built into our static tooling in the near term, so we're
actively looking at other solutions (for formatting) now.  ecj and IntelliJ
both have parsers that people use to build substantial amounts of tooling.
 They even have existing formatters!  ecj seems to have a lot of legacy
issues, but IntelliJ looks potentially promising.  We're looking into it.

Jeremy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/compiler-dev/attachments/20140128/cf8306b3/attachment.html