Invoking code generated by an annotation processor
Jonathan Gibbons
jonathan.gibbons at oracle.com
Wed Feb 17 19:45:51 UTC 2016
On 02/17/2016 11:36 AM, Stéphane NICOLAS wrote:
> Thx Jon.
>
> I would say there are 2 levels to answer this :
> 1) in case the error seems non recoverable, javac should just not list
> both recoverable and non recoverable errors. Only the non recoverable
> ones.
> We would be happy with that.
We could *maybe* prioritize the messages, but I think it would be wrong
to only show non-recoverable errors. Just because an error is
recoverable doesn't mean that an anno processor will generate code to
make the error go away.
> 2) I am not sure this difference really holds in the case of "B
> extends A and A is generated", what would be marked as non recoverable
> is actually also recoverable.
Generally, recoverable errors are those of the form "name not found", in
the sense that "name" could maybe be found if an anno processor
generates the code.
-- Jon
>
> S.
>
> 2016-02-17 11:00 GMT-08:00 Jonathan Gibbons
> <jonathan.gibbons at oracle.com <mailto:jonathan.gibbons at oracle.com>>:
>
> Stéphane,
>
> Internally, during the period before anno processing, javac
> categorizes errors as recoverable or not, where "recoverable"
> means "could this error conceivably be fixed by running annotation
> processors, which may generate code."
>
> It may be that we have to examine the scenario you describe to
> decide if the error should be marked as recoverable.
>
> -- Jon
>
>
> On 02/17/2016 10:53 AM, Stéphane NICOLAS wrote:
>>
>> Hi Jon,
>>
>>
>> Here is a summary about the bug, it should be enough for you to
>> understand what is going wrong. Later, we will provide you with
>> more details on an example you can use to reproduce the bug.
>>
>>
>> Summary
>>
>>
>> The problem is about static importsand invocation of generated
>> codefrom main code.
>>
>>
>> In our main code, we invoke some code that is generated by an
>> annotation processor. This scenario is fully supported by javac.
>> However, javac reports the calls to generated code as errors,
>> when it should not, in a very specific scenario.
>>
>>
>> In our main code, if we have an erroneous static import to a non
>> existing member in an existing class, then the annotation
>> processors are not triggered and all calls to the generated code
>> will be marked, unduly, as errors.
>>
>> → For us this is a very inconvenient bug because it floods
>> developers with too many errors and it becomes very hard to
>> pinpoint the only actual error in the code (the wrong static
>> import statement). This error can actually be triggered very
>> easily when refactoring.
>>
>>
>> Please note that if we have an equally erroneous static import to
>> a non existing class, then the bug doesn’t happen and only this
>> static import will be marked as error.
>>
>>
>> Extra thoughts
>>
>> We realized that we are kind of asking implicitly to distinguish
>> static import errors to existing classes and non existing
>> classes. But this may not be the right approach. It would not
>> accommodate the hacky but possible scenario when you define class
>> B extends Ain your main code, and generate the class A. In that
>> case, if A defines a public static field Foo, B would inherit it,
>> and though B exists, B.Foo would not exist before annotation
>> processing completes.
>> The best approach for us seems to be to always run annotation
>> processors even though there are some static imports problems and
>> to resolve them after code generation = annotation processing.
>> Static imports shouldn’t fail so fast..
>>
>>
>> Details
>>
>>
>> We tested with the following version of Javac :
>>
>> *
>>
>> javac 1.7.0_79
>>
>> *
>>
>> open JDK javac 1.7.0_79
>>
>> *
>>
>> javac 1.8.0_60
>>
>>
>> here is our example : sorry, it will be a bit long to read, but
>> it comes with a full explanation.
>>
>> We first give you easy repro steps and then an overview of the
>> library.
>>
>>
>> Javac issue and reproduction steps
>>
>> We have 3 branches to illustrate the problem :
>>
>>
>> 1. Branch :
>> Masterhttps://github.com/f2prateek/dart/blob/master/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java
>> This branch works fine, there is no compilation error. See below
>> to compile it.
>>
>>
>> To compile it quickly :
>>
>> 1.
>>
>> git clone git at github.com
>> <mailto:git at github.com>:f2prateek/dart.git
>>
>> 2.
>>
>> cd dart
>>
>> 3.
>>
>> mvn clean install
>>
>>
>>
>> 2. Branch : henson-error-linkage
>>
>> https://github.com/f2prateek/dart/blob/henson-error-linkage/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java
>> import staticcom.f2prateek.dart.example.SampleActivity.UNEXISTING;
>> is a static import to an existing class but a non existing
>> member. It will reproduce the bug of javac, the compiler will
>> return 2 errors. The call to generated code will also be marked
>> as error :
>>
>> [INFO] -------------------------------------------------------------
>>
>> [ERROR] COMPILATION ERROR :
>>
>> [INFO] -------------------------------------------------------------
>>
>> [ERROR]
>> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[26,1]
>> cannot find symbol
>>
>> symbol: static UNEXISTING
>>
>> location: class
>>
>> [ERROR]
>> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[39,21]
>> cannot find symbol
>>
>> symbol: variable Henson
>>
>> location: class com.f2prateek.dart.example.MainActivity
>>
>>
>> To compile it quickly :
>>
>> 1.
>>
>> git clone git at github.com
>> <mailto:git at github.com>:f2prateek/dart.git
>>
>> 2.
>>
>> cd dart
>>
>> 3.
>>
>> git checkout origin/henson-error-linkage
>>
>> 4.
>>
>> mvn clean install
>>
>>
>>
>> 3. Branch :
>> henson-error-linkage-goodhttps://github.com/f2prateek/dart/blob/henson-error-linkage-good/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java
>> import staticcom.f2prateek.dart.example.UnExistingClass.UNEXISTING;
>> is a static import to a non-existing class. It will not reproduce
>> the bug of javac. We get only errors related to static import.
>> (You can see that there is a minor glitch, errors are duplicated,
>> but that is not our main concern).
>>
>> [INFO] -------------------------------------------------------------
>>
>> [ERROR] COMPILATION ERROR :
>>
>> [INFO] -------------------------------------------------------------
>>
>> [ERROR]
>> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[26,41]
>> cannot find symbol
>>
>> symbol: class UnExistingClass
>>
>> location: package com.f2prateek.dart.example
>>
>> [ERROR]
>> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[26,1]
>> static import only from classes and interfaces
>>
>> [ERROR]
>> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[26,41]
>> cannot find symbol
>>
>> symbol: class UnExistingClass
>>
>> location: package com.f2prateek.dart.example
>>
>> [ERROR]
>> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[26,1]
>> static import only from classes and interfaces
>>
>>
>>
>> To compile it quickly :
>>
>> 1.
>>
>> git clone git at github.com
>> <mailto:git at github.com>:f2prateek/dart.git
>>
>> 2.
>>
>> cd dart
>>
>> 3.
>>
>> git checkout origin/henson-error-linkage-good
>>
>> 4.
>>
>> mvn clean install
>>
>>
>> Library overview
>>
>> Sample: https://github.com/f2prateek/dart/blob/master/dart-sample
>>
>> This project is the sample of our annotation processor based
>> library for Android.
>>
>> Just to give you a little background, the lib has 2 parts : Dart&
>> Henson.
>>
>>
>> When creating an activity (~ a screen / JWindow) on Android, you
>> can be passed a set of arguments via a Bundle. A bundle is a
>> serialized container that contains serialized arguments.
>>
>> A bundle is like a map, parameters have a name (called key) and a
>> value.
>>
>>
>> The library is all about navigating from a source activity to a
>> target activity and passing arguments via the bundle in a more
>> convenient way. When using Dart & Henson together, we completely
>> abstract the Bundle usage.
>>
>>
>> ----
>>
>>
>> Dartallows to annotate fields members in a target screen and to
>> assign values to these fields based on the content of the Bundle
>> you receive :
>>
>>
>> //from SampleActivity on GH
>> <https://github.com/f2prateek/dart/blob/master/dart-sample/src/main/java/com/f2prateek/dart/example/SampleActivity.java>
>>
>> public class SampleActivity {
>>
>> @InjectExtra(“key1”) String s1; //required key
>>
>> @InjectExtra(“key2”) @Nullable int i1; //optional key
>>
>> @Override
>> protected void onCreate(Bundle bundle) {
>>
>> Dart.inject(this);
>>
>> // after this call the field s1 is assigned the value
>> // of the entry key1 of the bundle
>>
>> // it is equivalent to : s = getIntent().getStringExtra(“key1”)
>>
>> // s1 is a required field in the bundle, whereas i1 is
>> optional.
>>
>> // If an entry key2 is present, i2 will be assigned to it.
>>
>> }
>>
>> }
>>
>> ----
>>
>> Henson allows to build the Bundle in the source activity to reach
>> the given target activity. Henson creates a small DSL to build
>> the Bundle, with a little state machine to emphasize the required
>> and optional natures of parameters.
>>
>>
>> //from MainActivity on GH
>> <https://github.com/f2prateek/dart/blob/master/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java>
>>
>> public class MainActivity extends Activity {
>>
>> public void launchTargetActivity() {
>>
>> Intent intent = Henson.with(this)
>>
>> .gotoSampleActivity() // here is the generated DSL call
>>
>> .key1("foo")
>>
>> .key2(4)
>>
>> .build();
>>
>>
>> startActivity(intent);
>>
>> }
>>
>> }
>>
>>
>> ----
>>
>>
>> Both Dart &Hensonuse annotation processing, which is a very
>> common way to do things on Android, mostly for speed reasons
>> (reflection is slow like hell, it is not JITed unlike on a
>> standard JVM). Dart does not involve calling generated called
>> directly, the code will be called via a little reflection. Henson
>> is all about calling generated code. In the sample above, Henson
>> class is generated and offers the DSL to build the bundle.
>> Nice, no ? :)
>>
>>
>> Thanks a lot guys, and sorry for this far too long but report,
>>
>> Daniel & Stephane
>>
>>
>> 2016-02-11 11:03 GMT-08:00 Jonathan Gibbons
>> <jonathan.gibbons at oracle.com <mailto:jonathan.gibbons at oracle.com>>:
>>
>> Stéphane,
>>
>> Can you reduce the test case down to a simple one(s) that you
>> can post here?
>>
>> -- Jon
>>
>>
>>
>> On 02/11/2016 10:44 AM, Stéphane NICOLAS wrote:
>>> Hey Jon,
>>>
>>> we are going to create an example based on our annotation
>>> processor. It will come in the form of a github repo, 1
>>> branch that compiles fine, 2 branches to illustrate
>>> different compile problems. Everything is mavenized so it
>>> should work out of the box for you. We will also tell you
>>> what error messages we have on the 2 failing branches,
>>> actual and expected behavior, and provide you with our javac
>>> version(s).
>>>
>>> We hope this can help, thank you very much for your answers.
>>>
>>> Stéphane
>>>
>>>
>>> 2016-02-10 13:25 GMT-08:00 Jonathan Gibbons
>>> <jonathan.gibbons at oracle.com
>>> <mailto:jonathan.gibbons at oracle.com>>:
>>>
>>>
>>>
>>> On 02/10/2016 12:59 PM, Stéphane NICOLAS wrote:
>>>> Thanks Jon for your fast answer. It's quite
>>>> disappointing but it's better to know it early.
>>>> It's sad to see how much effort we have put in our
>>>> annotation processor, that it's a good idea but fails
>>>> because of some general java ecosystem issue.
>>>>
>>>> Though, talking of the issue with team mates, it became
>>>> more clear. Let me try to rephrase them more accurately
>>>> than in my first mail.
>>>>
>>>> It seems to me that the design of javac could be
>>>> improved to accommodate these 2 scenarios :
>>>>
>>>> 1. in case of a parse error (for instance a missing ;
>>>> a bad brace, etc.) , javac should stop everything
>>>> and report the actual problem, without getting any
>>>> further into the analysis and not report any
>>>> linkage problem (because javac would know that
>>>> annotation processors ain't gonna run and that can
>>>> create false positive about missing linkages. All
>>>> other parse errors should be reported.). It seems
>>>> to me this would be a healthy "fail early" design.
>>>>
>>>
>>> This is the intent. Generally, javac is supposed to fail
>>> reasonably fast in general, and in that case in
>>> particular. If you are seeing otherwise, please
>>> provide a test case, including the version of javac you
>>> are using.
>>>
>>>
>>>
>>>> 1. in case of a linkage error (for instance having a
>>>> non existing import / calling a non existing
>>>> method, etc.) then we should run the annotation
>>>> processors on the partial AST. This mechanism is
>>>> already in place to accomodate code generation.
>>>> Then the linkage problems should be resolved and
>>>> take into account the generated code, and only the
>>>> remaining problems should show up. This is where
>>>> javac actually fails because it reports all
>>>> problems, including those that can be resolved by
>>>> the code that is actually generated. Javac reports
>>>> problem from a previous internal stage, it doesn't
>>>> match the latest state it could reach internally
>>>> where only 1 problem could not be resolved, the one
>>>> that was not related to invoking generated code, it
>>>> reports all linkages problem exactly like if no
>>>> code was generated and no annotation processor had run.
>>>>
>>>
>>> That is the intent.
>>>
>>>> Really, there is no way for javac to support those 2
>>>> scenarios ? I got the strong feeling that it should but
>>>> I understand my knowledge of javac has huge holes. I
>>>> have been surprised by the complexity of the compiler's
>>>> code.
>>>>
>>>> Thx again for your answer, it's been tremendously
>>>> appreciated; there are just a handful people would
>>>> could have been able to answer my question I am very
>>>> happy someone actually did take the time to do it.
>>>>
>>>> Stephane
>>>>
>>>> 2016-02-10 11:41 GMT-08:00 Jonathan Gibbons
>>>> <jonathan.gibbons at oracle.com
>>>> <mailto:jonathan.gibbons at oracle.com>>:
>>>>
>>>>
>>>>
>>>> On 02/10/2016 09:09 AM, Stéphane NICOLAS wrote:
>>>>> I am facing a problem with Javac : I got a sweet
>>>>> annotation processor that generates a DSL and I
>>>>> invoke the generated code from my main code.
>>>>>
>>>>> Everything works fine except when there is an
>>>>> error in my main code : if I got a compile error
>>>>> in the main code, not related to the generated
>>>>> code, like a missing comma (parsing problem), or
>>>>> calling an inexisting method (linkage problem),
>>>>> then all calls to the annotation processor are
>>>>> marked as errors by the compiler.
>>>>>
>>>>> Ideally I am just looking for a way to get rid of
>>>>> those error messages, and keep only the original
>>>>> one that fails to compile.
>>>>>
>>>>> AFAIK, javac should support this scenario but it
>>>>> doesn't support it in a very convenient way and I
>>>>> don't understand exactly why..
>>>>>
>>>>> After reafing the javac code, open jdk 7 & 8, I
>>>>> realized that :
>>>>>
>>>>> Annotation processors are not called if there is
>>>>> any error to compile the code. As the annotation
>>>>> processor are not triggered, the calls to
>>>>> generated code get mark as an error.
>>>>>
>>>>> However, I could, by playing with shouldStopPolicy
>>>>> (+if error and if no error) trigger the annotation
>>>>> processor and generate the code. But still, I
>>>>> would receive in this case the errors related to
>>>>> calling the generated code. Like if the generated
>>>>> code was not taken into account, like if I was
>>>>> receiving errors from a previous compilation steps.
>>>>>
>>>>> I understand that javac parses the code, if it
>>>>> fails there, I will receive only an error about
>>>>> the actual problem. And this is great. I just want
>>>>> this error to show up, not any other, and it works
>>>>> fine. (Though I am wondering if this is always
>>>>> true even I clean my build & generated files
>>>>> before compiling)
>>>>>
>>>>> If there is a linkage problem in my original code,
>>>>> like calling an non-existent method, javac will
>>>>> relax the error and trigger the annotation
>>>>> processors and try to let them generate the code
>>>>> to see if the generated code that satisfies the
>>>>> linkage problem. But then, though there is a state
>>>>> in which javac will successfully satisfy all
>>>>> linkage related to generated code but not the
>>>>> other linkage error, I get all calls to generated
>>>>> code marked as errors. This is my problem. It
>>>>> looks like this internal state of the compiler is
>>>>> never accessible and errors returned do not match
>>>>> this state but more probably a previous internal
>>>>> state in which all linkage errors are aggregated
>>>>> and returned.
>>>>>
>>>>> I think there is a solution to achieve what I want
>>>>> but I couldn't find one. And the situation is bad
>>>>> : it may actually cancel the development of the
>>>>> annotation processor lib I am developing. Just
>>>>> because the errors are flooding the actual problem
>>>>> in the code.
>>>>>
>>>>> Would you have some time to advise me, and help me
>>>>> to make the lib work as expected.
>>>>> My goal is just to get only errors that are not
>>>>> related to calling generated code if they are not
>>>>> those who fail the compilation. I don't want to
>>>>> see my errors polluted by those calls if they are
>>>>> not responsible for breaking the compilation.
>>>>>
>>>>> Stephane
>>>>
>>>> Stephane,
>>>>
>>>> In general, there is no convenient solution for you
>>>> here, because there is no way for javac to predict
>>>> what unresolved symbols may or may not be generated
>>>> by the annotation processor that has yet to be run.
>>>> And in general, it is not desirable to run
>>>> annotation processors if the input source code has
>>>> significant errors, such as syntax errors.
>>>>
>>>> -- Jon
>>>>
>>>>
>>>
>>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/compiler-dev/attachments/20160217/5358bc34/attachment-0001.html>
More information about the compiler-dev
mailing list