JEP 190: Pluggable Static Analyzers

[Jonathan Gibbons]

On 01/28/2014 01:01 PM, Jeremy Manson wrote:
> Our experience is that the various tree APIs aren't quite enough for a 
> couple of reasons:
>
> - Because of actions that are taken during parsing, the AST can't 
> easily be used to map back to the original code.  Limitations range 
> from things as simple as concatenating strings during parsing to 
> problems as serious as having no access to the original source / 
> comments.  This makes it hard to do things like write formatters.

We are aware of issues where the source tree does not directly 
correspond to the elements in the source code. The obvious main example 
is enum member declarations, which are desugared right in the parser. 
uugh.  We are working to address those issues.  Comments are more 
tricky. I doubt we would ever want to see comments faithfully rendered 
in the AST (it is a syntax tree after all) -- and the recommendation for 
utilities like formatters is to use the source positions (which are 
intended to be accurate) to map back to the original source text so that 
you can re-examine the text in the vicinity.


>
> - The tree API is relatively stable, but some of the ancillary APIs 
> change from revision to revision.  We've had to use reflection to 
> access, for example, the logging API, as it changes from version to 
> version (if we want to support more than one version at a time).  ISTR 
> having a problem with the file manager API in the javac6->7 
> transition, too (although I can't remember the details).

There were some changes in the processing Filer API needed to fix 
incorrect behavior that I remember.

>
> In the few years we've been writing tools on top of javac, we've found 
> that a steady stream of missing features and changed APIs have made 
> our lives slightly, but consistently, more complicated (or simply 
> prevented us from implementing various features). Most of the issues 
> we've seen are pretty fixable, though.

I would hope that while there may be some issues that made your life 
more complicated, you still found it less complicated overall to use 
javac instead of any other solution.   And you are right, many of the 
issues are fixable: it just comes down to time and resources.

-- Jon

>
> Jeremy
>
>
> On Tue, Jan 28, 2014 at 11:31 AM, Jonathan Gibbons 
> <jonathan.gibbons at oracle.com <mailto:jonathan.gibbons at oracle.com>> wrote:
>
>     On 01/28/2014 11:11 AM, Eric McCorkle wrote:
>
>         On 01/28/14 03:49, Florian Weimer wrote:
>
>             On 01/28/2014 01:52 AM, Jonathan Gibbons wrote:
>
>                 Why is JEP 190 different from the support already in
>                 javac for pluggable
>                 static analyzers, such as provided by -Xplugin?
>
>             The JSR 269 interface does not provide access to
>             expressions and method
>             bodies.  It happens that through this interface, javac
>             internals which
>             provide this information are available after casting to
>             internal javac
>             classes.  However, this is not a supported interface, and
>             the internal
>             javac data structures are not directly suited to
>             implementing analysis
>             passes.
>
>         An eventual goal of the system envisioned by JEP 190 is to allow
>         analysis tools to operate on javac's trees as opposed to source or
>         bytecode (as is common now).  Of course, there are quite a few
>         intermediate steps that have to be taken to realize that goal.
>
>
>     You can easily do this in JDK 8 with -Xplugin, using the
>     com.sun.source API.   com.sun.source is a public supported API.
>     Are you suggesting that is not enough, and that you want to start
>     investigating a standard javax.* AST API?
>
>     The one piece that is missing in the -Xplugin world is easy access
>     to the proto-bytecodes from a plugin.
>
>     -- Jon
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/compiler-dev/attachments/20140128/c623e2d0/attachment.html