Invoking code generated by an annotation processor
Stéphane NICOLAS
steff.nicolas at gmail.com
Wed Feb 17 19:36:56 UTC 2016
Thx Jon.
I would say there are 2 levels to answer this :
1) in case the error seems non recoverable, javac should just not list both
recoverable and non recoverable errors. Only the non recoverable ones.
We would be happy with that.
2) I am not sure this difference really holds in the case of "B extends A
and A is generated", what would be marked as non recoverable is actually
also recoverable.
S.
2016-02-17 11:00 GMT-08:00 Jonathan Gibbons <jonathan.gibbons at oracle.com>:
> Stéphane,
>
> Internally, during the period before anno processing, javac categorizes
> errors as recoverable or not, where "recoverable" means "could this error
> conceivably be fixed by running annotation processors, which may generate
> code."
>
> It may be that we have to examine the scenario you describe to decide if
> the error should be marked as recoverable.
>
> -- Jon
>
>
> On 02/17/2016 10:53 AM, Stéphane NICOLAS wrote:
>
> Hi Jon,
>
> Here is a summary about the bug, it should be enough for you to understand
> what is going wrong. Later, we will provide you with more details on an
> example you can use to reproduce the bug.
>
> Summary
>
> The problem is about static imports and invocation of generated code from
> main code.
>
> In our main code, we invoke some code that is generated by an annotation
> processor. This scenario is fully supported by javac. However, javac
> reports the calls to generated code as errors, when it should not, in a
> very specific scenario.
>
> In our main code, if we have an erroneous static import to a non existing
> member in an existing class, then the annotation processors are not
> triggered and all calls to the generated code will be marked, unduly, as
> errors.
>
> → For us this is a very inconvenient bug because it floods developers with
> too many errors and it becomes very hard to pinpoint the only actual error
> in the code (the wrong static import statement). This error can actually be
> triggered very easily when refactoring.
>
> Please note that if we have an equally erroneous static import to a non
> existing class, then the bug doesn’t happen and only this static import
> will be marked as error.
>
> Extra thoughts
>
> We realized that we are kind of asking implicitly to distinguish static
> import errors to existing classes and non existing classes. But this may
> not be the right approach. It would not accommodate the hacky but possible
> scenario when you define class B extends A in your main code, and
> generate the class A. In that case, if A defines a public static field Foo,
> B would inherit it, and though B exists, B.Foo would not exist before
> annotation processing completes.
> The best approach for us seems to be to always run annotation processors
> even though there are some static imports problems and to resolve them
> after code generation = annotation processing. Static imports shouldn’t
> fail so fast..
>
> Details
>
> We tested with the following version of Javac :
>
> -
>
> javac 1.7.0_79
> -
>
> open JDK javac 1.7.0_79
> -
>
> javac 1.8.0_60
>
>
> here is our example : sorry, it will be a bit long to read, but it comes
> with a full explanation.
>
> We first give you easy repro steps and then an overview of the library.
>
> Javac issue and reproduction steps
>
> We have 3 branches to illustrate the problem :
>
> 1. Branch : Master
> https://github.com/f2prateek/dart/blob/master/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java
> This branch works fine, there is no compilation error. See below to
> compile it.
>
> To compile it quickly :
>
> 1.
>
> git clone git at github.com:f2prateek/dart.git
> 2.
>
> cd dart
> 3.
>
> mvn clean install
>
>
>
> 2. Branch : henson-error-linkage
>
>
> https://github.com/f2prateek/dart/blob/henson-error-linkage/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java
> import static com.f2prateek.dart.example.SampleActivity.UNEXISTING;
> is a static import to an existing class but a non existing member. It will
> reproduce the bug of javac, the compiler will return 2 errors. The call to
> generated code will also be marked as error :
>
> [INFO] -------------------------------------------------------------
>
> [ERROR] COMPILATION ERROR :
>
> [INFO] -------------------------------------------------------------
>
> [ERROR]
> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[26,1]
> cannot find symbol
>
> symbol: static UNEXISTING
>
> location: class
>
> [ERROR]
> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[39,21]
> cannot find symbol
>
> symbol: variable Henson
>
> location: class com.f2prateek.dart.example.MainActivity
>
> To compile it quickly :
>
> 1.
>
> git clone git at github.com:f2prateek/dart.git
> 2.
>
> cd dart
> 3.
>
> git checkout origin/henson-error-linkage
> 4.
>
> mvn clean install
>
>
>
> 3. Branch : henson-error-linkage-good
> https://github.com/f2prateek/dart/blob/henson-error-linkage-good/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java
> import static com.f2prateek.dart.example.UnExistingClass.UNEXISTING;
> is a static import to a non-existing class. It will not reproduce the bug
> of javac. We get only errors related to static import. (You can see that
> there is a minor glitch, errors are duplicated, but that is not our main
> concern).
>
> [INFO] -------------------------------------------------------------
>
> [ERROR] COMPILATION ERROR :
>
> [INFO] -------------------------------------------------------------
>
> [ERROR]
> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[26,41]
> cannot find symbol
>
> symbol: class UnExistingClass
>
> location: package com.f2prateek.dart.example
>
> [ERROR]
> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[26,1]
> static import only from classes and interfaces
>
> [ERROR]
> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[26,41]
> cannot find symbol
>
> symbol: class UnExistingClass
>
> location: package com.f2prateek.dart.example
>
> [ERROR]
> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[26,1]
> static import only from classes and interfaces
>
>
> To compile it quickly :
>
> 1.
>
> git clone git at github.com:f2prateek/dart.git
> 2.
>
> cd dart
> 3.
>
> git checkout origin/henson-error-linkage-good
> 4.
>
> mvn clean install
>
>
> Library overview
>
> Sample: https://github.com/f2prateek/dart/blob/master/dart-sample
>
> This project is the sample of our annotation processor based library for
> Android.
>
> Just to give you a little background, the lib has 2 parts : Dart & Henson.
>
>
> When creating an activity (~ a screen / JWindow) on Android, you can be
> passed a set of arguments via a Bundle. A bundle is a serialized container
> that contains serialized arguments.
>
> A bundle is like a map, parameters have a name (called key) and a value.
>
> The library is all about navigating from a source activity to a target
> activity and passing arguments via the bundle in a more convenient way.
> When using Dart & Henson together, we completely abstract the Bundle usage.
>
> ----
>
> Dart allows to annotate fields members in a target screen and to assign
> values to these fields based on the content of the Bundle you receive :
>
> //from SampleActivity on GH
> <https://github.com/f2prateek/dart/blob/master/dart-sample/src/main/java/com/f2prateek/dart/example/SampleActivity.java>
>
> public class SampleActivity {
>
> @InjectExtra(“key1”) String s1; //required key
>
> @InjectExtra(“key2”) @Nullable int i1; //optional key
>
> @Override
> protected void onCreate(Bundle bundle) {
>
> Dart.inject(this);
>
> // after this call the field s1 is assigned the value
> // of the entry key1 of the bundle
>
> // it is equivalent to : s = getIntent().getStringExtra(“key1”)
>
> // s1 is a required field in the bundle, whereas i1 is optional.
>
> // If an entry key2 is present, i2 will be assigned to it.
>
> }
>
> }
>
> ----
>
> Henson allows to build the Bundle in the source activity to reach the
> given target activity. Henson creates a small DSL to build the Bundle, with
> a little state machine to emphasize the required and optional natures of
> parameters.
>
> //from MainActivity on GH
> <https://github.com/f2prateek/dart/blob/master/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java>
>
> public class MainActivity extends Activity {
>
> public void launchTargetActivity() {
>
> Intent intent = Henson.with(this)
>
> .gotoSampleActivity() // here is the generated DSL call
>
> .key1("foo")
>
> .key2(4)
>
> .build();
>
> startActivity(intent);
>
> }
>
> }
>
> ----
>
> Both Dart & Henson use annotation processing, which is a very common way
> to do things on Android, mostly for speed reasons (reflection is slow like
> hell, it is not JITed unlike on a standard JVM). Dart does not involve
> calling generated called directly, the code will be called via a little
> reflection. Henson is all about calling generated code. In the sample
> above, Henson class is generated and offers the DSL to build the bundle.
> Nice, no ? :)
>
> Thanks a lot guys, and sorry for this far too long but report,
>
> Daniel & Stephane
>
> 2016-02-11 11:03 GMT-08:00 Jonathan Gibbons <jonathan.gibbons at oracle.com>:
>
>> Stéphane,
>>
>> Can you reduce the test case down to a simple one(s) that you can post
>> here?
>>
>> -- Jon
>>
>>
>>
>> On 02/11/2016 10:44 AM, Stéphane NICOLAS wrote:
>>
>> Hey Jon,
>>
>> we are going to create an example based on our annotation processor. It
>> will come in the form of a github repo, 1 branch that compiles fine, 2
>> branches to illustrate different compile problems. Everything is mavenized
>> so it should work out of the box for you. We will also tell you what error
>> messages we have on the 2 failing branches, actual and expected behavior,
>> and provide you with our javac version(s).
>>
>> We hope this can help, thank you very much for your answers.
>>
>> Stéphane
>>
>>
>> 2016-02-10 13:25 GMT-08:00 Jonathan Gibbons <jonathan.gibbons at oracle.com>
>> :
>>
>>>
>>>
>>> On 02/10/2016 12:59 PM, Stéphane NICOLAS wrote:
>>>
>>> Thanks Jon for your fast answer. It's quite disappointing but it's
>>> better to know it early.
>>> It's sad to see how much effort we have put in our annotation processor,
>>> that it's a good idea but fails because of some general java ecosystem
>>> issue.
>>>
>>> Though, talking of the issue with team mates, it became more clear. Let
>>> me try to rephrase them more accurately than in my first mail.
>>>
>>> It seems to me that the design of javac could be improved to accommodate
>>> these 2 scenarios :
>>>
>>> 1. in case of a parse error (for instance a missing ; a bad brace,
>>> etc.) , javac should stop everything and report the actual problem, without
>>> getting any further into the analysis and not report any linkage problem
>>> (because javac would know that annotation processors ain't gonna run and
>>> that can create false positive about missing linkages. All other parse
>>> errors should be reported.). It seems to me this would be a healthy "fail
>>> early" design.
>>>
>>>
>>> This is the intent. Generally, javac is supposed to fail reasonably
>>> fast in general, and in that case in particular. If you are seeing
>>> otherwise, please provide a test case, including the version of javac you
>>> are using.
>>>
>>>
>>>
>>>
>>> 1. in case of a linkage error (for instance having a non existing
>>> import / calling a non existing method, etc.) then we should run the
>>> annotation processors on the partial AST. This mechanism is already in
>>> place to accomodate code generation.
>>> Then the linkage problems should be resolved and take into account
>>> the generated code, and only the remaining problems should show up. This is
>>> where javac actually fails because it reports all problems, including those
>>> that can be resolved by the code that is actually generated. Javac reports
>>> problem from a previous internal stage, it doesn't match the latest state
>>> it could reach internally where only 1 problem could not be resolved, the
>>> one that was not related to invoking generated code, it reports all
>>> linkages problem exactly like if no code was generated and no annotation
>>> processor had run.
>>>
>>>
>>> That is the intent.
>>>
>>> Really, there is no way for javac to support those 2 scenarios ? I got
>>> the strong feeling that it should but I understand my knowledge of javac
>>> has huge holes. I have been surprised by the complexity of the compiler's
>>> code.
>>>
>>> Thx again for your answer, it's been tremendously appreciated; there are
>>> just a handful people would could have been able to answer my question I am
>>> very happy someone actually did take the time to do it.
>>>
>>> Stephane
>>>
>>> 2016-02-10 11:41 GMT-08:00 Jonathan Gibbons <jonathan.gibbons at oracle.com
>>> >:
>>>
>>>>
>>>>
>>>> On 02/10/2016 09:09 AM, Stéphane NICOLAS wrote:
>>>>
>>>> I am facing a problem with Javac : I got a sweet annotation processor
>>>> that generates a DSL and I invoke the generated code from my main code.
>>>>
>>>> Everything works fine except when there is an error in my main code :
>>>> if I got a compile error in the main code, not related to the generated
>>>> code, like a missing comma (parsing problem), or calling an inexisting
>>>> method (linkage problem), then all calls to the annotation processor are
>>>> marked as errors by the compiler.
>>>>
>>>> Ideally I am just looking for a way to get rid of those error messages,
>>>> and keep only the original one that fails to compile.
>>>>
>>>> AFAIK, javac should support this scenario but it doesn't support it in
>>>> a very convenient way and I don't understand exactly why..
>>>>
>>>> After reafing the javac code, open jdk 7 & 8, I realized that :
>>>>
>>>> Annotation processors are not called if there is any error to compile
>>>> the code. As the annotation processor are not triggered, the calls to
>>>> generated code get mark as an error.
>>>>
>>>> However, I could, by playing with shouldStopPolicy (+if error and if no
>>>> error) trigger the annotation processor and generate the code. But still, I
>>>> would receive in this case the errors related to calling the generated
>>>> code. Like if the generated code was not taken into account, like if I was
>>>> receiving errors from a previous compilation steps.
>>>>
>>>> I understand that javac parses the code, if it fails there, I will
>>>> receive only an error about the actual problem. And this is great. I just
>>>> want this error to show up, not any other, and it works fine. (Though I am
>>>> wondering if this is always true even I clean my build & generated files
>>>> before compiling)
>>>>
>>>> If there is a linkage problem in my original code, like calling an
>>>> non-existent method, javac will relax the error and trigger the annotation
>>>> processors and try to let them generate the code to see if the generated
>>>> code that satisfies the linkage problem. But then, though there is a state
>>>> in which javac will successfully satisfy all linkage related to generated
>>>> code but not the other linkage error, I get all calls to generated code
>>>> marked as errors. This is my problem. It looks like this internal state of
>>>> the compiler is never accessible and errors returned do not match this
>>>> state but more probably a previous internal state in which all linkage
>>>> errors are aggregated and returned.
>>>>
>>>> I think there is a solution to achieve what I want but I couldn't find
>>>> one. And the situation is bad : it may actually cancel the development of
>>>> the annotation processor lib I am developing. Just because the errors are
>>>> flooding the actual problem in the code.
>>>>
>>>> Would you have some time to advise me, and help me to make the lib work
>>>> as expected.
>>>> My goal is just to get only errors that are not related to calling
>>>> generated code if they are not those who fail the compilation. I don't want
>>>> to see my errors polluted by those calls if they are not responsible for
>>>> breaking the compilation.
>>>>
>>>> Stephane
>>>>
>>>>
>>>> Stephane,
>>>>
>>>> In general, there is no convenient solution for you here, because there
>>>> is no way for javac to predict what unresolved symbols may or may not be
>>>> generated by the annotation processor that has yet to be run. And in
>>>> general, it is not desirable to run annotation processors if the input
>>>> source code has significant errors, such as syntax errors.
>>>>
>>>> -- Jon
>>>>
>>>
>>>
>>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/compiler-dev/attachments/20160217/6323a9d8/attachment-0001.html>
More information about the compiler-dev
mailing list