Invoking code generated by an annotation processor
Stéphane NICOLAS
steff.nicolas at gmail.com
Wed Feb 17 20:37:24 UTC 2016
2016-02-17 12:21 GMT-08:00 Stéphane NICOLAS <steff.nicolas at gmail.com>:
>
>
> 2016-02-17 11:45 GMT-08:00 Jonathan Gibbons <jonathan.gibbons at oracle.com>:
>
>>
>>
>> On 02/17/2016 11:36 AM, Stéphane NICOLAS wrote:
>>
>> Thx Jon.
>>
>> I would say there are 2 levels to answer this :
>> 1) in case the error seems non recoverable, javac should just not list
>> both recoverable and non recoverable errors. Only the non recoverable ones.
>> We would be happy with that.
>>
>>
>> We could *maybe* prioritize the messages, but I think it would be wrong
>> to only show non-recoverable errors. Just because an error is recoverable
>> doesn't mean that an anno processor will generate code to make the error go
>> away.
>>
>> No, it doesn't mean that, but in our case it is what happens and as the
>> annotation processor is not given the chance to run and resolve those
>> recoverable errors, they should not be listed.
>>
>
To be more clear, it's not about prioritizing the messages, though it could
be a workaround, but it's about not showing the recoverable errors when
there are non-recoverable errors. As long as the annotation processors
don't run, recoverable errors should not show up.
Also, we are wondering if you fully take into account the specifics of the
static import issue. Non static import don't behave in the same way, they
behave fine.
>
>>
>> 2) I am not sure this difference really holds in the case of "B extends A
>> and A is generated", what would be marked as non recoverable is actually
>> also recoverable.
>>
>>
>> Generally, recoverable errors are those of the form "name not found", in
>> the sense that "name" could maybe be found if an anno processor generates
>> the code.
>>
>
> So we were wrong in our understanding as we understood that they were
> different levels of "names not found" errors. It looks like when the class
> is found and a field not found, the error is ranked as non-recoverable. But
> really this case is further to ours, we just added an extra thought so that
> bug resolution takes into account the scenario, but it's not our main
> concern.
>
>>
>>
>> -- Jon
>>
>>
>>
>>
>> S.
>>
>> 2016-02-17 11:00 GMT-08:00 Jonathan Gibbons <jonathan.gibbons at oracle.com>
>> :
>>
>>> Stéphane,
>>>
>>> Internally, during the period before anno processing, javac categorizes
>>> errors as recoverable or not, where "recoverable" means "could this error
>>> conceivably be fixed by running annotation processors, which may generate
>>> code."
>>>
>>> It may be that we have to examine the scenario you describe to decide if
>>> the error should be marked as recoverable.
>>>
>>> -- Jon
>>>
>>>
>>> On 02/17/2016 10:53 AM, Stéphane NICOLAS wrote:
>>>
>>> Hi Jon,
>>>
>>> Here is a summary about the bug, it should be enough for you to
>>> understand what is going wrong. Later, we will provide you with more
>>> details on an example you can use to reproduce the bug.
>>>
>>> Summary
>>>
>>> The problem is about static imports and invocation of generated code
>>> from main code.
>>>
>>> In our main code, we invoke some code that is generated by an annotation
>>> processor. This scenario is fully supported by javac. However, javac
>>> reports the calls to generated code as errors, when it should not, in a
>>> very specific scenario.
>>>
>>> In our main code, if we have an erroneous static import to a non
>>> existing member in an existing class, then the annotation processors are
>>> not triggered and all calls to the generated code will be marked, unduly,
>>> as errors.
>>>
>>> → For us this is a very inconvenient bug because it floods developers
>>> with too many errors and it becomes very hard to pinpoint the only actual
>>> error in the code (the wrong static import statement). This error can
>>> actually be triggered very easily when refactoring.
>>>
>>> Please note that if we have an equally erroneous static import to a non
>>> existing class, then the bug doesn’t happen and only this static import
>>> will be marked as error.
>>>
>>> Extra thoughts
>>>
>>> We realized that we are kind of asking implicitly to distinguish static
>>> import errors to existing classes and non existing classes. But this may
>>> not be the right approach. It would not accommodate the hacky but possible
>>> scenario when you define class B extends A in your main code, and
>>> generate the class A. In that case, if A defines a public static field Foo,
>>> B would inherit it, and though B exists, B.Foo would not exist before
>>> annotation processing completes.
>>> The best approach for us seems to be to always run annotation processors
>>> even though there are some static imports problems and to resolve them
>>> after code generation = annotation processing. Static imports shouldn’t
>>> fail so fast..
>>>
>>> Details
>>>
>>> We tested with the following version of Javac :
>>>
>>> -
>>>
>>> javac 1.7.0_79
>>> -
>>>
>>> open JDK javac 1.7.0_79
>>> -
>>>
>>> javac 1.8.0_60
>>>
>>>
>>> here is our example : sorry, it will be a bit long to read, but it comes
>>> with a full explanation.
>>>
>>> We first give you easy repro steps and then an overview of the library.
>>>
>>> Javac issue and reproduction steps
>>>
>>> We have 3 branches to illustrate the problem :
>>>
>>> 1. Branch : Master
>>> https://github.com/f2prateek/dart/blob/master/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java
>>> This branch works fine, there is no compilation error. See below to
>>> compile it.
>>>
>>> To compile it quickly :
>>>
>>> 1.
>>>
>>> git clone git at github.com:f2prateek/dart.git
>>> 2.
>>>
>>> cd dart
>>> 3.
>>>
>>> mvn clean install
>>>
>>>
>>>
>>> 2. Branch : henson-error-linkage
>>>
>>>
>>> https://github.com/f2prateek/dart/blob/henson-error-linkage/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java
>>> import static com.f2prateek.dart.example.SampleActivity.UNEXISTING;
>>> is a static import to an existing class but a non existing member. It
>>> will reproduce the bug of javac, the compiler will return 2 errors. The
>>> call to generated code will also be marked as error :
>>>
>>> [INFO] -------------------------------------------------------------
>>>
>>> [ERROR] COMPILATION ERROR :
>>>
>>> [INFO] -------------------------------------------------------------
>>>
>>> [ERROR]
>>> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[26,1]
>>> cannot find symbol
>>>
>>> symbol: static UNEXISTING
>>>
>>> location: class
>>>
>>> [ERROR]
>>> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[39,21]
>>> cannot find symbol
>>>
>>> symbol: variable Henson
>>>
>>> location: class com.f2prateek.dart.example.MainActivity
>>>
>>> To compile it quickly :
>>>
>>> 1.
>>>
>>> git clone git at github.com:f2prateek/dart.git
>>> 2.
>>>
>>> cd dart
>>> 3.
>>>
>>> git checkout origin/henson-error-linkage
>>> 4.
>>>
>>> mvn clean install
>>>
>>>
>>>
>>> 3. Branch : henson-error-linkage-good
>>> https://github.com/f2prateek/dart/blob/henson-error-linkage-good/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java
>>> import static com.f2prateek.dart.example.UnExistingClass.UNEXISTING;
>>> is a static import to a non-existing class. It will not reproduce the
>>> bug of javac. We get only errors related to static import. (You can see
>>> that there is a minor glitch, errors are duplicated, but that is not our
>>> main concern).
>>>
>>> [INFO] -------------------------------------------------------------
>>>
>>> [ERROR] COMPILATION ERROR :
>>>
>>> [INFO] -------------------------------------------------------------
>>>
>>> [ERROR]
>>> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[26,41]
>>> cannot find symbol
>>>
>>> symbol: class UnExistingClass
>>>
>>> location: package com.f2prateek.dart.example
>>>
>>> [ERROR]
>>> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[26,1]
>>> static import only from classes and interfaces
>>>
>>> [ERROR]
>>> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[26,41]
>>> cannot find symbol
>>>
>>> symbol: class UnExistingClass
>>>
>>> location: package com.f2prateek.dart.example
>>>
>>> [ERROR]
>>> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[26,1]
>>> static import only from classes and interfaces
>>>
>>>
>>> To compile it quickly :
>>>
>>> 1.
>>>
>>> git clone git at github.com:f2prateek/dart.git
>>> 2.
>>>
>>> cd dart
>>> 3.
>>>
>>> git checkout origin/henson-error-linkage-good
>>> 4.
>>>
>>> mvn clean install
>>>
>>>
>>> Library overview
>>>
>>> Sample: https://github.com/f2prateek/dart/blob/master/dart-sample
>>>
>>> This project is the sample of our annotation processor based library for
>>> Android.
>>>
>>> Just to give you a little background, the lib has 2 parts : Dart &
>>> Henson.
>>>
>>> When creating an activity (~ a screen / JWindow) on Android, you can be
>>> passed a set of arguments via a Bundle. A bundle is a serialized container
>>> that contains serialized arguments.
>>>
>>> A bundle is like a map, parameters have a name (called key) and a value.
>>>
>>> The library is all about navigating from a source activity to a target
>>> activity and passing arguments via the bundle in a more convenient way.
>>> When using Dart & Henson together, we completely abstract the Bundle usage.
>>>
>>> ----
>>>
>>> Dart allows to annotate fields members in a target screen and to assign
>>> values to these fields based on the content of the Bundle you receive :
>>>
>>> //from SampleActivity on GH
>>> <https://github.com/f2prateek/dart/blob/master/dart-sample/src/main/java/com/f2prateek/dart/example/SampleActivity.java>
>>>
>>> public class SampleActivity {
>>>
>>> @InjectExtra(“key1”) String s1; //required key
>>>
>>> @InjectExtra(“key2”) @Nullable int i1; //optional key
>>>
>>> @Override
>>> protected void onCreate(Bundle bundle) {
>>>
>>> Dart.inject(this);
>>>
>>> // after this call the field s1 is assigned the value
>>> // of the entry key1 of the bundle
>>>
>>> // it is equivalent to : s = getIntent().getStringExtra(“key1”)
>>>
>>> // s1 is a required field in the bundle, whereas i1 is optional.
>>>
>>> // If an entry key2 is present, i2 will be assigned to it.
>>>
>>> }
>>>
>>> }
>>>
>>> ----
>>>
>>> Henson allows to build the Bundle in the source activity to reach the
>>> given target activity. Henson creates a small DSL to build the Bundle, with
>>> a little state machine to emphasize the required and optional natures of
>>> parameters.
>>>
>>> //from MainActivity on GH
>>> <https://github.com/f2prateek/dart/blob/master/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java>
>>>
>>> public class MainActivity extends Activity {
>>>
>>> public void launchTargetActivity() {
>>>
>>> Intent intent = Henson.with(this)
>>>
>>> .gotoSampleActivity() // here is the generated DSL call
>>>
>>> .key1("foo")
>>>
>>> .key2(4)
>>>
>>> .build();
>>>
>>> startActivity(intent);
>>>
>>> }
>>>
>>> }
>>>
>>> ----
>>>
>>> Both Dart & Henson use annotation processing, which is a very common
>>> way to do things on Android, mostly for speed reasons (reflection is slow
>>> like hell, it is not JITed unlike on a standard JVM). Dart does not involve
>>> calling generated called directly, the code will be called via a little
>>> reflection. Henson is all about calling generated code. In the sample
>>> above, Henson class is generated and offers the DSL to build the bundle.
>>> Nice, no ? :)
>>>
>>> Thanks a lot guys, and sorry for this far too long but report,
>>>
>>> Daniel & Stephane
>>>
>>> 2016-02-11 11:03 GMT-08:00 Jonathan Gibbons <jonathan.gibbons at oracle.com
>>> >:
>>>
>>>> Stéphane,
>>>>
>>>> Can you reduce the test case down to a simple one(s) that you can post
>>>> here?
>>>>
>>>> -- Jon
>>>>
>>>>
>>>>
>>>> On 02/11/2016 10:44 AM, Stéphane NICOLAS wrote:
>>>>
>>>> Hey Jon,
>>>>
>>>> we are going to create an example based on our annotation processor. It
>>>> will come in the form of a github repo, 1 branch that compiles fine, 2
>>>> branches to illustrate different compile problems. Everything is mavenized
>>>> so it should work out of the box for you. We will also tell you what error
>>>> messages we have on the 2 failing branches, actual and expected behavior,
>>>> and provide you with our javac version(s).
>>>>
>>>> We hope this can help, thank you very much for your answers.
>>>>
>>>> Stéphane
>>>>
>>>>
>>>> 2016-02-10 13:25 GMT-08:00 Jonathan Gibbons <
>>>> jonathan.gibbons at oracle.com>:
>>>>
>>>>>
>>>>>
>>>>> On 02/10/2016 12:59 PM, Stéphane NICOLAS wrote:
>>>>>
>>>>> Thanks Jon for your fast answer. It's quite disappointing but it's
>>>>> better to know it early.
>>>>> It's sad to see how much effort we have put in our annotation
>>>>> processor, that it's a good idea but fails because of some general java
>>>>> ecosystem issue.
>>>>>
>>>>> Though, talking of the issue with team mates, it became more clear.
>>>>> Let me try to rephrase them more accurately than in my first mail.
>>>>>
>>>>> It seems to me that the design of javac could be improved to
>>>>> accommodate these 2 scenarios :
>>>>>
>>>>> 1. in case of a parse error (for instance a missing ; a bad brace,
>>>>> etc.) , javac should stop everything and report the actual problem, without
>>>>> getting any further into the analysis and not report any linkage problem
>>>>> (because javac would know that annotation processors ain't gonna run and
>>>>> that can create false positive about missing linkages. All other parse
>>>>> errors should be reported.). It seems to me this would be a healthy "fail
>>>>> early" design.
>>>>>
>>>>>
>>>>> This is the intent. Generally, javac is supposed to fail reasonably
>>>>> fast in general, and in that case in particular. If you are seeing
>>>>> otherwise, please provide a test case, including the version of javac you
>>>>> are using.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> 1. in case of a linkage error (for instance having a non existing
>>>>> import / calling a non existing method, etc.) then we should run the
>>>>> annotation processors on the partial AST. This mechanism is already in
>>>>> place to accomodate code generation.
>>>>> Then the linkage problems should be resolved and take into account
>>>>> the generated code, and only the remaining problems should show up. This is
>>>>> where javac actually fails because it reports all problems, including those
>>>>> that can be resolved by the code that is actually generated. Javac reports
>>>>> problem from a previous internal stage, it doesn't match the latest state
>>>>> it could reach internally where only 1 problem could not be resolved, the
>>>>> one that was not related to invoking generated code, it reports all
>>>>> linkages problem exactly like if no code was generated and no annotation
>>>>> processor had run.
>>>>>
>>>>>
>>>>> That is the intent.
>>>>>
>>>>> Really, there is no way for javac to support those 2 scenarios ? I got
>>>>> the strong feeling that it should but I understand my knowledge of javac
>>>>> has huge holes. I have been surprised by the complexity of the compiler's
>>>>> code.
>>>>>
>>>>> Thx again for your answer, it's been tremendously appreciated; there
>>>>> are just a handful people would could have been able to answer my question
>>>>> I am very happy someone actually did take the time to do it.
>>>>>
>>>>> Stephane
>>>>>
>>>>> 2016-02-10 11:41 GMT-08:00 Jonathan Gibbons <
>>>>> jonathan.gibbons at oracle.com>:
>>>>>
>>>>>>
>>>>>>
>>>>>> On 02/10/2016 09:09 AM, Stéphane NICOLAS wrote:
>>>>>>
>>>>>> I am facing a problem with Javac : I got a sweet annotation processor
>>>>>> that generates a DSL and I invoke the generated code from my main code.
>>>>>>
>>>>>> Everything works fine except when there is an error in my main code :
>>>>>> if I got a compile error in the main code, not related to the generated
>>>>>> code, like a missing comma (parsing problem), or calling an inexisting
>>>>>> method (linkage problem), then all calls to the annotation processor are
>>>>>> marked as errors by the compiler.
>>>>>>
>>>>>> Ideally I am just looking for a way to get rid of those error
>>>>>> messages, and keep only the original one that fails to compile.
>>>>>>
>>>>>> AFAIK, javac should support this scenario but it doesn't support it
>>>>>> in a very convenient way and I don't understand exactly why..
>>>>>>
>>>>>> After reafing the javac code, open jdk 7 & 8, I realized that :
>>>>>>
>>>>>> Annotation processors are not called if there is any error to compile
>>>>>> the code. As the annotation processor are not triggered, the calls to
>>>>>> generated code get mark as an error.
>>>>>>
>>>>>> However, I could, by playing with shouldStopPolicy (+if error and if
>>>>>> no error) trigger the annotation processor and generate the code. But
>>>>>> still, I would receive in this case the errors related to calling the
>>>>>> generated code. Like if the generated code was not taken into account, like
>>>>>> if I was receiving errors from a previous compilation steps.
>>>>>>
>>>>>> I understand that javac parses the code, if it fails there, I will
>>>>>> receive only an error about the actual problem. And this is great. I just
>>>>>> want this error to show up, not any other, and it works fine. (Though I am
>>>>>> wondering if this is always true even I clean my build & generated files
>>>>>> before compiling)
>>>>>>
>>>>>> If there is a linkage problem in my original code, like calling an
>>>>>> non-existent method, javac will relax the error and trigger the annotation
>>>>>> processors and try to let them generate the code to see if the generated
>>>>>> code that satisfies the linkage problem. But then, though there is a state
>>>>>> in which javac will successfully satisfy all linkage related to generated
>>>>>> code but not the other linkage error, I get all calls to generated code
>>>>>> marked as errors. This is my problem. It looks like this internal state of
>>>>>> the compiler is never accessible and errors returned do not match this
>>>>>> state but more probably a previous internal state in which all linkage
>>>>>> errors are aggregated and returned.
>>>>>>
>>>>>> I think there is a solution to achieve what I want but I couldn't
>>>>>> find one. And the situation is bad : it may actually cancel the development
>>>>>> of the annotation processor lib I am developing. Just because the errors
>>>>>> are flooding the actual problem in the code.
>>>>>>
>>>>>> Would you have some time to advise me, and help me to make the lib
>>>>>> work as expected.
>>>>>> My goal is just to get only errors that are not related to calling
>>>>>> generated code if they are not those who fail the compilation. I don't want
>>>>>> to see my errors polluted by those calls if they are not responsible for
>>>>>> breaking the compilation.
>>>>>>
>>>>>> Stephane
>>>>>>
>>>>>>
>>>>>> Stephane,
>>>>>>
>>>>>> In general, there is no convenient solution for you here, because
>>>>>> there is no way for javac to predict what unresolved symbols may or may not
>>>>>> be generated by the annotation processor that has yet to be run. And in
>>>>>> general, it is not desirable to run annotation processors if the input
>>>>>> source code has significant errors, such as syntax errors.
>>>>>>
>>>>>> -- Jon
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/compiler-dev/attachments/20160217/9798ff61/attachment-0001.html>
More information about the compiler-dev
mailing list