Invoking code generated by an annotation processor
Jonathan Gibbons
jonathan.gibbons at oracle.com
Wed Feb 17 19:00:59 UTC 2016
Stéphane,
Internally, during the period before anno processing, javac categorizes
errors as recoverable or not, where "recoverable" means "could this
error conceivably be fixed by running annotation processors, which may
generate code."
It may be that we have to examine the scenario you describe to decide if
the error should be marked as recoverable.
-- Jon
On 02/17/2016 10:53 AM, Stéphane NICOLAS wrote:
>
> Hi Jon,
>
>
> Here is a summary about the bug, it should be enough for you to
> understand what is going wrong. Later, we will provide you with more
> details on an example you can use to reproduce the bug.
>
>
> Summary
>
>
> The problem is about static importsand invocation of generated
> codefrom main code.
>
>
> In our main code, we invoke some code that is generated by an
> annotation processor. This scenario is fully supported by javac.
> However, javac reports the calls to generated code as errors, when it
> should not, in a very specific scenario.
>
>
> In our main code, if we have an erroneous static import to a non
> existing member in an existing class, then the annotation processors
> are not triggered and all calls to the generated code will be marked,
> unduly, as errors.
>
> → For us this is a very inconvenient bug because it floods developers
> with too many errors and it becomes very hard to pinpoint the only
> actual error in the code (the wrong static import statement). This
> error can actually be triggered very easily when refactoring.
>
>
> Please note that if we have an equally erroneous static import to a
> non existing class, then the bug doesn’t happen and only this static
> import will be marked as error.
>
>
> Extra thoughts
>
> We realized that we are kind of asking implicitly to distinguish
> static import errors to existing classes and non existing classes. But
> this may not be the right approach. It would not accommodate the hacky
> but possible scenario when you define class B extends Ain your main
> code, and generate the class A. In that case, if A defines a public
> static field Foo, B would inherit it, and though B exists, B.Foo would
> not exist before annotation processing completes.
> The best approach for us seems to be to always run annotation
> processors even though there are some static imports problems and to
> resolve them after code generation = annotation processing. Static
> imports shouldn’t fail so fast..
>
>
> Details
>
>
> We tested with the following version of Javac :
>
> *
>
> javac 1.7.0_79
>
> *
>
> open JDK javac 1.7.0_79
>
> *
>
> javac 1.8.0_60
>
>
> here is our example : sorry, it will be a bit long to read, but it
> comes with a full explanation.
>
> We first give you easy repro steps and then an overview of the library.
>
>
> Javac issue and reproduction steps
>
> We have 3 branches to illustrate the problem :
>
>
> 1. Branch :
> Masterhttps://github.com/f2prateek/dart/blob/master/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java
> This branch works fine, there is no compilation error. See below to
> compile it.
>
>
> To compile it quickly :
>
> 1.
>
> git clone git at github.com <mailto:git at github.com>:f2prateek/dart.git
>
> 2.
>
> cd dart
>
> 3.
>
> mvn clean install
>
>
>
> 2. Branch : henson-error-linkage
>
> https://github.com/f2prateek/dart/blob/henson-error-linkage/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java
> import staticcom.f2prateek.dart.example.SampleActivity.UNEXISTING;
> is a static import to an existing class but a non existing member. It
> will reproduce the bug of javac, the compiler will return 2 errors.
> The call to generated code will also be marked as error :
>
> [INFO] -------------------------------------------------------------
>
> [ERROR] COMPILATION ERROR :
>
> [INFO] -------------------------------------------------------------
>
> [ERROR]
> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[26,1]
> cannot find symbol
>
> symbol: static UNEXISTING
>
> location: class
>
> [ERROR]
> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[39,21]
> cannot find symbol
>
> symbol: variable Henson
>
> location: class com.f2prateek.dart.example.MainActivity
>
>
> To compile it quickly :
>
> 1.
>
> git clone git at github.com <mailto:git at github.com>:f2prateek/dart.git
>
> 2.
>
> cd dart
>
> 3.
>
> git checkout origin/henson-error-linkage
>
> 4.
>
> mvn clean install
>
>
>
> 3. Branch :
> henson-error-linkage-goodhttps://github.com/f2prateek/dart/blob/henson-error-linkage-good/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java
> import staticcom.f2prateek.dart.example.UnExistingClass.UNEXISTING;
> is a static import to a non-existing class. It will not reproduce the
> bug of javac. We get only errors related to static import. (You can
> see that there is a minor glitch, errors are duplicated, but that is
> not our main concern).
>
> [INFO] -------------------------------------------------------------
>
> [ERROR] COMPILATION ERROR :
>
> [INFO] -------------------------------------------------------------
>
> [ERROR]
> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[26,41]
> cannot find symbol
>
> symbol: class UnExistingClass
>
> location: package com.f2prateek.dart.example
>
> [ERROR]
> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[26,1]
> static import only from classes and interfaces
>
> [ERROR]
> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[26,41]
> cannot find symbol
>
> symbol: class UnExistingClass
>
> location: package com.f2prateek.dart.example
>
> [ERROR]
> /Users/dmolinero/Development/dart/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java:[26,1]
> static import only from classes and interfaces
>
>
>
> To compile it quickly :
>
> 1.
>
> git clone git at github.com <mailto:git at github.com>:f2prateek/dart.git
>
> 2.
>
> cd dart
>
> 3.
>
> git checkout origin/henson-error-linkage-good
>
> 4.
>
> mvn clean install
>
>
> Library overview
>
> Sample: https://github.com/f2prateek/dart/blob/master/dart-sample
>
> This project is the sample of our annotation processor based library
> for Android.
>
> Just to give you a little background, the lib has 2 parts : Dart& Henson.
>
>
> When creating an activity (~ a screen / JWindow) on Android, you can
> be passed a set of arguments via a Bundle. A bundle is a serialized
> container that contains serialized arguments.
>
> A bundle is like a map, parameters have a name (called key) and a value.
>
>
> The library is all about navigating from a source activity to a target
> activity and passing arguments via the bundle in a more convenient
> way. When using Dart & Henson together, we completely abstract the
> Bundle usage.
>
>
> ----
>
>
> Dartallows to annotate fields members in a target screen and to assign
> values to these fields based on the content of the Bundle you receive :
>
>
> //from SampleActivity on GH
> <https://github.com/f2prateek/dart/blob/master/dart-sample/src/main/java/com/f2prateek/dart/example/SampleActivity.java>
>
> public class SampleActivity {
>
> @InjectExtra(“key1”) String s1; //required key
>
> @InjectExtra(“key2”) @Nullable int i1; //optional key
>
> @Override
> protected void onCreate(Bundle bundle) {
>
> Dart.inject(this);
>
> // after this call the field s1 is assigned the value
> // of the entry key1 of the bundle
>
> // it is equivalent to : s = getIntent().getStringExtra(“key1”)
>
> // s1 is a required field in the bundle, whereas i1 is optional.
>
> // If an entry key2 is present, i2 will be assigned to it.
>
> }
>
> }
>
> ----
>
> Henson allows to build the Bundle in the source activity to reach the
> given target activity. Henson creates a small DSL to build the Bundle,
> with a little state machine to emphasize the required and optional
> natures of parameters.
>
>
> //from MainActivity on GH
> <https://github.com/f2prateek/dart/blob/master/dart-sample/src/main/java/com/f2prateek/dart/example/MainActivity.java>
>
> public class MainActivity extends Activity {
>
> public void launchTargetActivity() {
>
> Intent intent = Henson.with(this)
>
> .gotoSampleActivity() // here is the generated DSL call
>
> .key1("foo")
>
> .key2(4)
>
> .build();
>
>
> startActivity(intent);
>
> }
>
> }
>
>
> ----
>
>
> Both Dart &Hensonuse annotation processing, which is a very common way
> to do things on Android, mostly for speed reasons (reflection is slow
> like hell, it is not JITed unlike on a standard JVM). Dart does not
> involve calling generated called directly, the code will be called via
> a little reflection. Henson is all about calling generated code. In
> the sample above, Henson class is generated and offers the DSL to
> build the bundle.
> Nice, no ? :)
>
>
> Thanks a lot guys, and sorry for this far too long but report,
>
> Daniel & Stephane
>
>
> 2016-02-11 11:03 GMT-08:00 Jonathan Gibbons
> <jonathan.gibbons at oracle.com <mailto:jonathan.gibbons at oracle.com>>:
>
> Stéphane,
>
> Can you reduce the test case down to a simple one(s) that you can
> post here?
>
> -- Jon
>
>
>
> On 02/11/2016 10:44 AM, Stéphane NICOLAS wrote:
>> Hey Jon,
>>
>> we are going to create an example based on our annotation
>> processor. It will come in the form of a github repo, 1 branch
>> that compiles fine, 2 branches to illustrate different compile
>> problems. Everything is mavenized so it should work out of the
>> box for you. We will also tell you what error messages we have on
>> the 2 failing branches, actual and expected behavior, and provide
>> you with our javac version(s).
>>
>> We hope this can help, thank you very much for your answers.
>>
>> Stéphane
>>
>>
>> 2016-02-10 13:25 GMT-08:00 Jonathan Gibbons
>> <jonathan.gibbons at oracle.com <mailto:jonathan.gibbons at oracle.com>>:
>>
>>
>>
>> On 02/10/2016 12:59 PM, Stéphane NICOLAS wrote:
>>> Thanks Jon for your fast answer. It's quite disappointing
>>> but it's better to know it early.
>>> It's sad to see how much effort we have put in our
>>> annotation processor, that it's a good idea but fails
>>> because of some general java ecosystem issue.
>>>
>>> Though, talking of the issue with team mates, it became more
>>> clear. Let me try to rephrase them more accurately than in
>>> my first mail.
>>>
>>> It seems to me that the design of javac could be improved to
>>> accommodate these 2 scenarios :
>>>
>>> 1. in case of a parse error (for instance a missing ; a bad
>>> brace, etc.) , javac should stop everything and report
>>> the actual problem, without getting any further into the
>>> analysis and not report any linkage problem (because
>>> javac would know that annotation processors ain't gonna
>>> run and that can create false positive about missing
>>> linkages. All other parse errors should be reported.).
>>> It seems to me this would be a healthy "fail early" design.
>>>
>>
>> This is the intent. Generally, javac is supposed to fail
>> reasonably fast in general, and in that case in particular.
>> If you are seeing otherwise, please provide a test case,
>> including the version of javac you are using.
>>
>>
>>
>>> 1. in case of a linkage error (for instance having a non
>>> existing import / calling a non existing method, etc.)
>>> then we should run the annotation processors on the
>>> partial AST. This mechanism is already in place to
>>> accomodate code generation.
>>> Then the linkage problems should be resolved and take
>>> into account the generated code, and only the remaining
>>> problems should show up. This is where javac actually
>>> fails because it reports all problems, including those
>>> that can be resolved by the code that is actually
>>> generated. Javac reports problem from a previous
>>> internal stage, it doesn't match the latest state it
>>> could reach internally where only 1 problem could not be
>>> resolved, the one that was not related to invoking
>>> generated code, it reports all linkages problem exactly
>>> like if no code was generated and no annotation
>>> processor had run.
>>>
>>
>> That is the intent.
>>
>>> Really, there is no way for javac to support those 2
>>> scenarios ? I got the strong feeling that it should but I
>>> understand my knowledge of javac has huge holes. I have been
>>> surprised by the complexity of the compiler's code.
>>>
>>> Thx again for your answer, it's been tremendously
>>> appreciated; there are just a handful people would could
>>> have been able to answer my question I am very happy someone
>>> actually did take the time to do it.
>>>
>>> Stephane
>>>
>>> 2016-02-10 11:41 GMT-08:00 Jonathan Gibbons
>>> <jonathan.gibbons at oracle.com
>>> <mailto:jonathan.gibbons at oracle.com>>:
>>>
>>>
>>>
>>> On 02/10/2016 09:09 AM, Stéphane NICOLAS wrote:
>>>> I am facing a problem with Javac : I got a sweet
>>>> annotation processor that generates a DSL and I invoke
>>>> the generated code from my main code.
>>>>
>>>> Everything works fine except when there is an error in
>>>> my main code : if I got a compile error in the main
>>>> code, not related to the generated code, like a missing
>>>> comma (parsing problem), or calling an inexisting
>>>> method (linkage problem), then all calls to the
>>>> annotation processor are marked as errors by the compiler.
>>>>
>>>> Ideally I am just looking for a way to get rid of those
>>>> error messages, and keep only the original one that
>>>> fails to compile.
>>>>
>>>> AFAIK, javac should support this scenario but it
>>>> doesn't support it in a very convenient way and I don't
>>>> understand exactly why..
>>>>
>>>> After reafing the javac code, open jdk 7 & 8, I
>>>> realized that :
>>>>
>>>> Annotation processors are not called if there is any
>>>> error to compile the code. As the annotation processor
>>>> are not triggered, the calls to generated code get mark
>>>> as an error.
>>>>
>>>> However, I could, by playing with shouldStopPolicy (+if
>>>> error and if no error) trigger the annotation processor
>>>> and generate the code. But still, I would receive in
>>>> this case the errors related to calling the generated
>>>> code. Like if the generated code was not taken into
>>>> account, like if I was receiving errors from a previous
>>>> compilation steps.
>>>>
>>>> I understand that javac parses the code, if it fails
>>>> there, I will receive only an error about the actual
>>>> problem. And this is great. I just want this error to
>>>> show up, not any other, and it works fine. (Though I am
>>>> wondering if this is always true even I clean my build
>>>> & generated files before compiling)
>>>>
>>>> If there is a linkage problem in my original code, like
>>>> calling an non-existent method, javac will relax the
>>>> error and trigger the annotation processors and try to
>>>> let them generate the code to see if the generated code
>>>> that satisfies the linkage problem. But then, though
>>>> there is a state in which javac will successfully
>>>> satisfy all linkage related to generated code but not
>>>> the other linkage error, I get all calls to generated
>>>> code marked as errors. This is my problem. It looks
>>>> like this internal state of the compiler is never
>>>> accessible and errors returned do not match this state
>>>> but more probably a previous internal state in which
>>>> all linkage errors are aggregated and returned.
>>>>
>>>> I think there is a solution to achieve what I want but
>>>> I couldn't find one. And the situation is bad : it may
>>>> actually cancel the development of the annotation
>>>> processor lib I am developing. Just because the errors
>>>> are flooding the actual problem in the code.
>>>>
>>>> Would you have some time to advise me, and help me to
>>>> make the lib work as expected.
>>>> My goal is just to get only errors that are not related
>>>> to calling generated code if they are not those who
>>>> fail the compilation. I don't want to see my errors
>>>> polluted by those calls if they are not responsible for
>>>> breaking the compilation.
>>>>
>>>> Stephane
>>>
>>> Stephane,
>>>
>>> In general, there is no convenient solution for you
>>> here, because there is no way for javac to predict what
>>> unresolved symbols may or may not be generated by the
>>> annotation processor that has yet to be run. And in
>>> general, it is not desirable to run annotation
>>> processors if the input source code has significant
>>> errors, such as syntax errors.
>>>
>>> -- Jon
>>>
>>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/compiler-dev/attachments/20160217/27585611/attachment-0001.html>
More information about the compiler-dev
mailing list