From wieland at testingtech.com Wed Jul 1 08:58:57 2015 From: wieland at testingtech.com (Jacob Wieland) Date: Wed, 1 Jul 2015 10:58:57 +0200 Subject: Fwd: Java Performance Degradation in JDK7 and JDK8 In-Reply-To: References: <5540A479.7010401@oracle.com> <5540ABCD.3080708@oracle.com> <5540D8B0.3070604@oracle.com> <55410052.5080205@oracle.com> <55410A1A.1090104@oracle.com> <55410C10.9010108@oracle.com> <554111E1.10305@oracle.com> <55411708.3020109@oracle.com> <5542443E.6030803@oracle.com> <55424F96.8050705@oracle.com> <55926630.6020401@oracle.com> <559273F8.4070103@oracle.com> <55928F0C.8020003@oracle.com> <55928F84.7060501@oracle.com> <55929BEC.1040002@oracle.com> Message-ID: I have tried again with the original code generation scheme (sorry, last tests were with the new modular one) for the full example and got the following measurements: 1.6 - 60 sec 1.8 - 90 sec (CPU > 300) 1.9 - 110 sec (CPU > 300) I also tried with the new modular code generation scheme (using top-level classes instead of inner classes) 1.6 - 86 sec 1.8 - 142 sec (CPU > 300) 1.9 - 128 sec (CPU > 300) The CPU usage of 1.8 and 1.9 where also much higher in 1.8 and 1.9. Using memory restrictions (-J-Xmx2000M), I got the following results on the modular scheme: 1.6 - 101 sec (CPU max 160, normally around 100) 1.8 - 1168 sec (CPU continuously around 350) 1.9 - 194 sec (CPU max 360, almost continuously) If you are interested, we can exchange the testcases in some other way, I guess. They are too large to send. BR 2015-06-30 15:38 GMT+02:00 Maurizio Cimadamore < maurizio.cimadamore at oracle.com>: > I tried this on my not-so-good laptop, and I can get your originally > submitted test case to complete in 50 seconds with the latest JDK 9; > doesn't seem to even be able to complete with an older JDK 9 (I'm using > -Xmx764M to artificially limit the memory). So, it seems like performances > are back to what the bug report says you were getting for 1.6 ? > > Maurizio > > > On 30/06/15 13:56, Jacob Wieland wrote: > > sorry to say it, but the jdk 9 version isn't any better (in that respect) > either ... > > 2015-06-30 14:45 GMT+02:00 Maurizio Cimadamore < > maurizio.cimadamore at oracle.com>: > >> Also, please try with latest snapshot for JDK 9 >> >> https://jdk9.java.net/download/ >> >> It is possible that the 8 version is held back by something else >> >> Maurizio >> >> On 30/06/15 13:43, Maurizio Cimadamore wrote: >> >> (Adding Jan) >> this is weird - in our tests memory usage went down dramatically. Jan, is >> your fix for 8039262 >> in JDK 8u60 b21 (the latest from the early access page) ? >> >> Maurizio >> >> >> On 30/06/15 13:40, Jacob Wieland wrote: >> >> unfortunately, there doesn't seem to be any improvement with that build >> that I can observe. still uses up all cpu-capacity and doesn't take less >> time, either. >> >> 2015-06-30 12:48 GMT+02:00 Maurizio Cimadamore < >> maurizio.cimadamore at oracle.com>: >> >>> >>> >>> On 30/06/15 11:45, Jacob Wieland wrote: >>> >>> Thank you, >>> >>> why doesn't the bug appear in the change-set? >>> >>> It's been fixed in b20, so b21 changeset won't show it. Our early >>> access website only maintains info about the latest available build. >>> >>> Maurizio >>> >>> BR >>> >>> 2015-06-30 11:49 GMT+02:00 Maurizio Cimadamore < >>> maurizio.cimadamore at oracle.com>: >>> >>>> Hi Jacob, >>>> this has been fixed in 9 and 8 - our sustaining team will take it from >>>> there and backport to 7 eventually, but I don't have visibility over their >>>> integration schedule. >>>> >>>> The next JDK 8 public update should have the fix - in the meantime, you >>>> could try with an early access build here: >>>> >>>> https://jdk8.java.net/download.html >>>> >>>> Maurizio >>>> >>>> >>>> On 30/06/15 07:33, Jacob Wieland wrote: >>>> >>>> Hello Maurizio, >>>> >>>> any news on this issue? >>>> >>>> BR >>>> >>>> 2015-04-30 17:51 GMT+02:00 Maurizio Cimadamore < >>>> maurizio.cimadamore at oracle.com>: >>>> >>>>> On 30/04/15 16:15, Jacob Wieland wrote: >>>>> >>>>> thank you very much - very interesting. >>>>> >>>>> is there any estimate when a patch for 7/8 will be available? >>>>> >>>>> We are currently playing with different patches - we will update this >>>>> discussion when we know more about which steps will be taken. >>>>> >>>>> Regards >>>>> Maurizio >>>>> >>>>> >>>>> 2015-04-30 17:03 GMT+02:00 Maurizio Cimadamore < >>>>> maurizio.cimadamore at oracle.com>: >>>>> >>>>>> For the records - I've added a comment in >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8039262 >>>>>> >>>>>> Reporting in as much detail as possible the underlying cause of the >>>>>> performance regression. >>>>>> >>>>>> Maurizio >>>>>> >>>>>> >>>>>> On 29/04/15 18:38, Jan Lahoda wrote: >>>>>> >>>>>>> In 9, the Scope listeners are (AFAIK) only needed to update the >>>>>>> "mark" (so that the Types caches can detect obsolete entries). I think we >>>>>>> may be able to avoid the listeners at the cost of slowing down getMark >>>>>>> somewhat (getMark would ask the sub-scopes for their marks and sum the >>>>>>> result). I'll try. >>>>>>> >>>>>>> Jan >>>>>>> >>>>>>> On 29.4.2015 19:16, Maurizio Cimadamore wrote: >>>>>>> >>>>>>>> Found it. >>>>>>>> >>>>>>>> The big offender is the listeners field in the CompoundScope (which >>>>>>>> is a >>>>>>>> list). >>>>>>>> >>>>>>>> The test ends up creating huge listeners lists - probably because of >>>>>>>> very deep inheritance hierarchies. >>>>>>>> >>>>>>>> If I avoid adding stuff to the listeners field in the compound >>>>>>>> scope, I >>>>>>>> get back to a sane scenario: >>>>>>>> >>>>>>>> real 0m32.540s >>>>>>>> user 1m15.298s >>>>>>>> sys 0m1.126s >>>>>>>> >>>>>>>> >>>>>>>> And the profiled memory usage is much more under control. >>>>>>>> >>>>>>>> So, looks like we need a way to prevent these listeners list to >>>>>>>> overwhelm javac ;-) >>>>>>>> >>>>>>>> Maurizio >>>>>>>> >>>>>>>> On 29/04/15 17:51, Vicente-Arturo Romero-Zaldivar wrote: >>>>>>>> >>>>>>>>> On 04/29/2015 09:43 AM, Maurizio Cimadamore wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 29/04/15 17:01, Jan Lahoda wrote: >>>>>>>>>> >>>>>>>>>>> On 29.4.2015 16:06, Jacob Wieland wrote: >>>>>>>>>>> >>>>>>>>>>>> I have to admit, the reproducer is only a small part of the >>>>>>>>>>>> actual >>>>>>>>>>>> generated code. In my preliminary tests, it already sufficed to >>>>>>>>>>>> show a >>>>>>>>>>>> difference (also, the smaller code still worked with 32 bit >>>>>>>>>>>> while the >>>>>>>>>>>> whole code runs out of memory in all 32 bit versions which makes >>>>>>>>>>>> comparison much harder ;-) and which is why I need the 64 bit >>>>>>>>>>>> versions). >>>>>>>>>>>> If I test with the complete code which is much bigger, the >>>>>>>>>>>> results are >>>>>>>>>>>> as follows: >>>>>>>>>>>> >>>>>>>>>>>> jdk1.6u45(64bit) 2GB MaxMem - 1:30 minutes >>>>>>>>>>>> jdk1.7u75(64bit) 2GB MaxMem - > 6 min >>>>>>>>>>>> jdk1.8u31(64bit) 1GB MaxMem - > 15 min >>>>>>>>>>>> jdk1.8u31(64bit) 2GB MaxMem - > 10 min >>>>>>>>>>>> jdk1.8u31(64bit) 4GB MaxMem - 2:20 min (-source/-target 6 does >>>>>>>>>>>> not >>>>>>>>>>>> seem >>>>>>>>>>>> to have any effect) >>>>>>>>>>>> >>>>>>>>>>>> So, if you throw insane (in comparison with 1.6) amounts of >>>>>>>>>>>> memory at >>>>>>>>>>>> 1.7/1.8, it is only about a third as slow, but this still is >>>>>>>>>>>> unacceptable. I actually think it has to do with >>>>>>>>>>>> parallellization and >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> When I was looking at JDK-8039262, a significant contributing >>>>>>>>>>> factor >>>>>>>>>>> to the slowdown (with enough memory and -source/-target 6) >>>>>>>>>>> appeared >>>>>>>>>>> to be Check.checkOverrideClashes - I believe this does checks >>>>>>>>>>> that >>>>>>>>>>> were not properly implemented in 6, contributing to the >>>>>>>>>>> difference >>>>>>>>>>> between 6 and 7 (which seems to be particularly visible for this >>>>>>>>>>> testcase). I was looking at the checks a few times, trying to >>>>>>>>>>> write >>>>>>>>>>> them differently/faster while still performing the checks, but >>>>>>>>>>> was >>>>>>>>>>> not successful in that yet, unfortunately. >>>>>>>>>>> >>>>>>>>>> I see the issue now - it is reproducible with the following memory >>>>>>>>>> parameter (at least in my machine): >>>>>>>>>> >>>>>>>>>> -J-Xmx768M >>>>>>>>>> >>>>>>>>>> This give around 20sec in JDK 6 and 10+ minutes in JDK 8. >>>>>>>>>> >>>>>>>>>> All the time seems to be spent in desugaring, most specifically in >>>>>>>>>> TransTypes.addBridges - it seems like that method calls >>>>>>>>>> Types.implementation a lot - so my theory was that the fact that >>>>>>>>>> javac consumes more memory, forces the GC to get rid of the cached >>>>>>>>>> entries in the implementation/members closure caches (since such >>>>>>>>>> entries are deliberately backed up by SoftReferences), which in >>>>>>>>>> turn >>>>>>>>>> completely trashes performances. I instrumented the code a bit and >>>>>>>>>> this is what I found: >>>>>>>>>> >>>>>>>>>> *) With -Xmx768M >>>>>>>>>> >>>>>>>>>> Impl cache misses = 3346926 >>>>>>>>>> Members cache misses = 1042678 >>>>>>>>>> >>>>>>>>>> real 7m0.335s >>>>>>>>>> user 25m51.517s >>>>>>>>>> sys 0m4.947s >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> *) W/o -Xmx768M >>>>>>>>>> >>>>>>>>>> Impl cache misses = 3346839 >>>>>>>>>> Members cache misses = 1042678 >>>>>>>>>> >>>>>>>>>> real 0m32.377s >>>>>>>>>> user 1m25.881s >>>>>>>>>> sys 0m2.232s >>>>>>>>>> >>>>>>>>>> Long story short - cache misses do not play a factor in here - >>>>>>>>>> there >>>>>>>>>> are some minor differences, but nothing out of the ordinary and >>>>>>>>>> defo >>>>>>>>>> nothing that would explain a multi-minute slowdown! Any ideas? >>>>>>>>>> >>>>>>>>> >>>>>>>>> use flight recorder? >>>>>>>>> >>>>>>>>> Vicente >>>>>>>>> >>>>>>>>> >>>>>>>>>> Maurizio >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Jan >>>>>>>>>>> >>>>>>>>>>> garbage collection. >>>>>>>>>>>> >>>>>>>>>>>> 2015-04-29 15:12 GMT+02:00 Maurizio Cimadamore >>>>>>>>>>>> >>>>>>>>>>> >: >>>>>>>>>>>> >>>>>>>>>>>> On 29/04/15 12:44, Jacob Wieland wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hello Maurizio, >>>>>>>>>>>>> >>>>>>>>>>>>> are you sure that you used the 64bit versions of javac? I >>>>>>>>>>>>> could >>>>>>>>>>>>> only observe the behavior with these. >>>>>>>>>>>>> >>>>>>>>>>>> Yep I'm on a Ubuntu x64 machine. It's actually pretty >>>>>>>>>>>> standard >>>>>>>>>>>> hardware too - i.e. intel i5 (two cores, but OS sees 4 >>>>>>>>>>>> because of >>>>>>>>>>>> hyper-threading). >>>>>>>>>>>> >>>>>>>>>>>>> Also, I just tried with jdk8u31-64b and it takes AGES >>>>>>>>>>>>> (still >>>>>>>>>>>>> running after 17 minutes where the jdk6 was done after 2), >>>>>>>>>>>>> top >>>>>>>>>>>>> shows 4GB VIRT memory use and 350 % load (on a 4core >>>>>>>>>>>>> processor). >>>>>>>>>>>>> >>>>>>>>>>>> Maybe the reproducer you sent was incorrect? >>>>>>>>>>>> >>>>>>>>>>>> Maurizio >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> So, I don't think it was that problem. >>>>>>>>>>>>> >>>>>>>>>>>>> 2015-04-29 12:00 GMT+02:00 Maurizio Cimadamore >>>>>>>>>>>>> >>>>>>>>>>>> >: >>>>>>>>>>>>> >>>>>>>>>>>>> These are the numbers I'm getting: >>>>>>>>>>>>> >>>>>>>>>>>>> JDK 9 (b42) >>>>>>>>>>>>> >>>>>>>>>>>>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or >>>>>>>>>>>>> overrides a >>>>>>>>>>>>> deprecated API. >>>>>>>>>>>>> Note: Recompile with -Xlint:deprecation for details. >>>>>>>>>>>>> >>>>>>>>>>>>> real 0m46.306s >>>>>>>>>>>>> user 2m17.489s >>>>>>>>>>>>> sys 0m2.166s >>>>>>>>>>>>> >>>>>>>>>>>>> JDK 8 (GA) >>>>>>>>>>>>> >>>>>>>>>>>>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or >>>>>>>>>>>>> overrides a >>>>>>>>>>>>> deprecated API. >>>>>>>>>>>>> Note: Recompile with -Xlint:deprecation for details. >>>>>>>>>>>>> >>>>>>>>>>>>> real 6m58.748s >>>>>>>>>>>>> user 8m43.546s >>>>>>>>>>>>> sys 0m2.132s >>>>>>>>>>>>> >>>>>>>>>>>>> JDK 7 (1.7.0_79) >>>>>>>>>>>>> >>>>>>>>>>>>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or >>>>>>>>>>>>> overrides a >>>>>>>>>>>>> deprecated API. >>>>>>>>>>>>> Note: Recompile with -Xlint:deprecation for details. >>>>>>>>>>>>> >>>>>>>>>>>>> real 0m28.341s >>>>>>>>>>>>> user 1m17.194s >>>>>>>>>>>>> sys 0m1.886s >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> As you can see there is a significant regression from >>>>>>>>>>>>> JDK >>>>>>>>>>>>> 7 to >>>>>>>>>>>>> JDK 8 which was caused by >>>>>>>>>>>>> >>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8043253 >>>>>>>>>>>>> >>>>>>>>>>>>> (some stack trace analysis revealed the familiar >>>>>>>>>>>>> pattern). >>>>>>>>>>>>> This has also been fixed in JDK 8u20 (as stated in the >>>>>>>>>>>>> bug >>>>>>>>>>>>> evaluation). >>>>>>>>>>>>> >>>>>>>>>>>>> So, while JDK 8u20/9 is slower than JDK 7 (at least on >>>>>>>>>>>>> my >>>>>>>>>>>>> machine), the numbers are more or less in the same >>>>>>>>>>>>> ballpark >>>>>>>>>>>>> and the huge regression that was visible in earlier >>>>>>>>>>>>> JDK 8 >>>>>>>>>>>>> releases has now been fixed. >>>>>>>>>>>>> >>>>>>>>>>>>> If you are still experiencing the problem - can you >>>>>>>>>>>>> please >>>>>>>>>>>>> also submit the specific compiler versions you are >>>>>>>>>>>>> using in >>>>>>>>>>>>> your benchmark? >>>>>>>>>>>>> >>>>>>>>>>>>> Maurizio >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 29/04/15 10:29, Maurizio Cimadamore wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Jacob, >>>>>>>>>>>>>> Stay assured - as we'll definitively look into this >>>>>>>>>>>>>> issue (I >>>>>>>>>>>>>> see it's already assigned to one of my colleagues). >>>>>>>>>>>>>> >>>>>>>>>>>>>> What I can say (w/o looking too much at the attached >>>>>>>>>>>>>> artifacts) is that in general, javac has no issue with >>>>>>>>>>>>>> compiling a lot of sources at once; at one point the >>>>>>>>>>>>>> build >>>>>>>>>>>>>> system was structured in such a way that all the JDK >>>>>>>>>>>>>> classes >>>>>>>>>>>>>> were compiled at once - and that (which is way more >>>>>>>>>>>>>> than >>>>>>>>>>>>>> your >>>>>>>>>>>>>> 187 sources - i.e. at least 10x that) took less than >>>>>>>>>>>>>> 20 >>>>>>>>>>>>>> seconds. SO there must some specific pattern >>>>>>>>>>>>>> triggering that >>>>>>>>>>>>>> issue. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Given that you say you have 187 input sources and 48K >>>>>>>>>>>>>> output >>>>>>>>>>>>>> classes, I'd say you are using inner classes a lot. I >>>>>>>>>>>>>> wonder >>>>>>>>>>>>>> if you are hitting this: >>>>>>>>>>>>>> >>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8000316 >>>>>>>>>>>>>> >>>>>>>>>>>>>> Maurizio >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 24/04/15 09:49, Jacob Wieland wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hello Folks, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I still have the open problem >>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8039262 that the >>>>>>>>>>>>>>> javac performance has degraded significantly from >>>>>>>>>>>>>>> 1.6 to >>>>>>>>>>>>>>> 1.7 >>>>>>>>>>>>>>> (and even worse to 1.8) in the 64bit versions. Since >>>>>>>>>>>>>>> in our >>>>>>>>>>>>>>> context, we are dealing with a lot of generated >>>>>>>>>>>>>>> source1.4 >>>>>>>>>>>>>>> Java input (either split into very large files with >>>>>>>>>>>>>>> inner >>>>>>>>>>>>>>> classes or big packages with lots of smaller >>>>>>>>>>>>>>> classes), >>>>>>>>>>>>>>> compiler performance is critical for our tool and >>>>>>>>>>>>>>> this >>>>>>>>>>>>>>> degradation forces us to continue recommending to our >>>>>>>>>>>>>>> customers to use Java 1.6 for large projects (as is >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>> norm) as 1.7 and 1.8 are pretty much unusable in this >>>>>>>>>>>>>>> respect. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Is anyone still working on this issue or is such >>>>>>>>>>>>>>> significant >>>>>>>>>>>>>>> performance degradation not a serious issue? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> My observations so far are: >>>>>>>>>>>>>>> - it gets worse the more class files are being >>>>>>>>>>>>>>> compiled/the >>>>>>>>>>>>>>> more files reside in the source path >>>>>>>>>>>>>>> - cpu usage goes through the roof over all available >>>>>>>>>>>>>>> kernels >>>>>>>>>>>>>>> - memory usage is much higher >>>>>>>>>>>>>>> - garbage collection seems to be much more active >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Using -proc:none alleviates the problem slightly for >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>> 1.7 >>>>>>>>>>>>>>> version, but not for 1.8 (last we tested with >>>>>>>>>>>>>>> jdk1.8.0_31) where the performance difference is a >>>>>>>>>>>>>>> factor 5 >>>>>>>>>>>>>>> or more! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I can understand that a more advanced compiler has >>>>>>>>>>>>>>> capabilities that a previous version does not have >>>>>>>>>>>>>>> and thus >>>>>>>>>>>>>>> sometimes has >>>>>>>>>>>>>>> to do more work. But, it should still be possible >>>>>>>>>>>>>>> (especially if given the -source 1.4 or -source 1.5 >>>>>>>>>>>>>>> option >>>>>>>>>>>>>>> as we do) to optimize it in such a way that >>>>>>>>>>>>>>> unnecessary >>>>>>>>>>>>>>> checks for generics, overriding methods, closures, >>>>>>>>>>>>>>> annotations and other newer features can be turned >>>>>>>>>>>>>>> off (if >>>>>>>>>>>>>>> they are to blame, which I actually doubt from my >>>>>>>>>>>>>>> observations). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I would really appreciate your help in this regard >>>>>>>>>>>>>>> and I >>>>>>>>>>>>>>> think everyone would benefit from any bugfix you can >>>>>>>>>>>>>>> offer >>>>>>>>>>>>>>> for this. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> BR, Jacob Wieland >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> ------------------------------ >>>>>>>>>>>>>>> ------------------------------------------- >>>>>>>>>>>>>>> Dr. Jacob Wieland >>>>>>>>>>>>>>> Senior Software Engineer >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Testing Technologies IST GmbH >>>>>>>>>>>>>>> Michaelkirchstra?e 17/18 >>>>>>>>>>>>>>> 10179 Berlin, Germany >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Phone +49 30 726 19 19 34 >>>>>>>>>>>>>>> <%2B49%2030%20726%2019%2019%2034> Email >>>>>>>>>>>>>>> wieland at testingtech.com >>>>>>>>>>>>>> stanca at testingtech.com> >>>>>>>>>>>>>>> Fax +49 30 726 19 19 20 Internet >>>>>>>>>>>>>>> www.testingtech.com >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> --------------------------------------------------------------------------------------------------------------- >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> UPCOMING EVENTS >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>>>>>>>>>>>>>> Deadline: May 30, 2015 >>>>>>>>>>>>>>> ucaat.etsi.org/2015/CallForPresentations.html >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Apr 21-23, 2015 | SAE Conference & Exhibition >>>>>>>>>>>>>>> Detroit, Michigan, USA >>>>>>>>>>>>>>> www.sae.org/congress/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Apr 28-30, 2015 | iqnite >>>>>>>>>>>>>>> Dusseldorf, Germany >>>>>>>>>>>>>>> www.iqnite-conferences.com/de/index.aspx >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, Stephan >>>>>>>>>>>>>>> Pietsch, Pete Nicholson Handelsregister HRB 77805 B, >>>>>>>>>>>>>>> Amtsgericht Charlottenburg Ust ID Nr.: DE 813 143 >>>>>>>>>>>>>>> 070 This >>>>>>>>>>>>>>> email may contain confidential and privileged >>>>>>>>>>>>>>> material for >>>>>>>>>>>>>>> the sole use of the intended recipient. Any review, >>>>>>>>>>>>>>> use, >>>>>>>>>>>>>>> distribution or disclosure by others is strictly >>>>>>>>>>>>>>> prohibited. >>>>>>>>>>>>>>> If you are not the intended recipient (or authorized >>>>>>>>>>>>>>> to >>>>>>>>>>>>>>> receive for the recipient), please contact the >>>>>>>>>>>>>>> sender by >>>>>>>>>>>>>>> reply email and delete all copies of this message. >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> -- >>>>>>>>>>>>> ------------------------------ >>>>>>>>>>>>> ------------------------------------------- >>>>>>>>>>>>> Dr. Jacob Wieland >>>>>>>>>>>>> Senior Software Engineer >>>>>>>>>>>>> >>>>>>>>>>>>> Testing Technologies IST GmbH >>>>>>>>>>>>> Michaelkirchstra?e 17/18 >>>>>>>>>>>>> 10179 Berlin, Germany >>>>>>>>>>>>> >>>>>>>>>>>>> Phone +49 30 726 19 19 34 Email >>>>>>>>>>>>> wieland at testingtech.com >>>>>>>>>>>>> >>>>>>>>>>>>> Fax +49 30 726 19 19 20 <%2B49%2030%20726%2019%2019%2020> >>>>>>>>>>>>> Internet www.testingtech.com >>>>>>>>>>>>> >>>>>>>>>>>>> --------------------------------------------------------------------------------------------------------------- >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> UPCOMING EVENTS >>>>>>>>>>>>> >>>>>>>>>>>>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>>>>>>>>>>>> Deadline: May 30, 2015 >>>>>>>>>>>>> ucaat.etsi.org/2015/CallForPresentations.html >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Apr 21-23, 2015 | SAE Conference & Exhibition >>>>>>>>>>>>> Detroit, Michigan, USA >>>>>>>>>>>>> www.sae.org/congress/ >>>>>>>>>>>>> >>>>>>>>>>>>> Apr 28-30, 2015 | iqnite >>>>>>>>>>>>> Dusseldorf, Germany >>>>>>>>>>>>> www.iqnite-conferences.com/de/index.aspx >>>>>>>>>>>>> >>>>>>>>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> ... > > [Nachricht gek?rzt] -- -- ------------------------------ ------------------------------------------- Dr. Jacob Wieland Senior Software Engineer Testing Technologies IST GmbH Michaelkirchstra?e 17/18 10179 Berlin, Germany Phone +49 30 726 19 19 34 Email wieland at testingtech.com Fax +49 30 726 19 19 20 Internet www.testingtech.com --------------------------------------------------------------------------------------------------------------------- UPCOMING EVENTS Jul 15-16, 2015 | German Testing Day Frankfurt, Germany | Booth #8 www.germantestingday.info/ Sep 15-17, 2015 | Accredited TTCN-3 Training "Test Automation with TTCN-3" Berlin, Germany www.testingtech.com/services/ttcn3_training.php Oct 5-9, 2015 | 22nd ITS World Congress Bordeaux, France | At Spirent's Booth #122 www.itsineurope.com/bordeaux-2015/ ----------------------------------------------------------------------------------------------------------------- Testing Technologies IST GmbH Michaelkirchstra?e 17/18, 10179 Berlin, Germany Office: +49 (30) 726 19 19 0 Fax: +49 (30) 726 19 19 20 Internet www.testingtech.com ------------------------------------------------------------ ----------------------------------------------------- Executive Board: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg Ust ID Nr.: DE 813 143 070 This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. -- -- ------------------------------ ------------------------------------------- Dr. Jacob Wieland Senior Software Engineer Testing Technologies IST GmbH Michaelkirchstra?e 17/18 10179 Berlin, Germany Phone +49 30 726 19 19 34 Email wieland at testingtech.com Fax +49 30 726 19 19 20 Internet www.testingtech.com --------------------------------------------------------------------------------------------------------------------- UPCOMING EVENTS Jul 15-16, 2015 | German Testing Day Frankfurt, Germany | Booth #8 www.germantestingday.info/ Sep 15-17, 2015 | Accredited TTCN-3 Training "Test Automation with TTCN-3" Berlin, Germany www.testingtech.com/services/ttcn3_training.php Oct 5-9, 2015 | 22nd ITS World Congress Bordeaux, France | At Spirent's Booth #122 www.itsineurope.com/bordeaux-2015/ ----------------------------------------------------------------------------------------------------------------- Testing Technologies IST GmbH Michaelkirchstra?e 17/18, 10179 Berlin, Germany Office: +49 (30) 726 19 19 0 Fax: +49 (30) 726 19 19 20 Internet www.testingtech.com ------------------------------------------------------------ ----------------------------------------------------- Executive Board: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg Ust ID Nr.: DE 813 143 070 This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. -------------- next part -------------- An HTML attachment was scrubbed... URL: From maurizio.cimadamore at oracle.com Wed Jul 1 10:48:20 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 01 Jul 2015 11:48:20 +0100 Subject: Fwd: Java Performance Degradation in JDK7 and JDK8 In-Reply-To: References: <55410052.5080205@oracle.com> <55410A1A.1090104@oracle.com> <55410C10.9010108@oracle.com> <554111E1.10305@oracle.com> <55411708.3020109@oracle.com> <5542443E.6030803@oracle.com> <55424F96.8050705@oracle.com> <55926630.6020401@oracle.com> <559273F8.4070103@oracle.com> <55928F0C.8020003@oracle.com> <55928F84.7060501@oracle.com> <55929BEC.1040002@oracle.com> Message-ID: <5593C574.50803@oracle.com> What do you mean by 'full example' - the one you attached in the JBS bug? Maurizio On 01/07/15 09:58, Jacob Wieland wrote: > I have tried again with the original code generation scheme (sorry, > last tests were with the new modular one) for the full example and got > the following measurements: > > 1.6 - 60 sec > 1.8 - 90 sec (CPU > 300) > 1.9 - 110 sec (CPU > 300) > > I also tried with the new modular code generation scheme (using > top-level classes instead of inner classes) > > 1.6 - 86 sec > 1.8 - 142 sec (CPU > 300) > 1.9 - 128 sec (CPU > 300) > > The CPU usage of 1.8 and 1.9 where also much higher in 1.8 and 1.9. > > Using memory restrictions (-J-Xmx2000M), I got the following results > on the modular scheme: > > 1.6 - 101 sec (CPU max 160, normally around 100) > 1.8 - 1168 sec (CPU continuously around 350) > 1.9 - 194 sec (CPU max 360, almost continuously) > > If you are interested, we can exchange the testcases in some other > way, I guess. They are too large to send. > > BR > > > 2015-06-30 15:38 GMT+02:00 Maurizio Cimadamore > >: > > I tried this on my not-so-good laptop, and I can get your > originally submitted test case to complete in 50 seconds with the > latest JDK 9; doesn't seem to even be able to complete with an > older JDK 9 (I'm using -Xmx764M to artificially limit the memory). > So, it seems like performances are back to what the bug report > says you were getting for 1.6 ? > > Maurizio > > > On 30/06/15 13:56, Jacob Wieland wrote: >> sorry to say it, but the jdk 9 version isn't any better (in that >> respect) either ... >> >> 2015-06-30 14:45 GMT+02:00 Maurizio Cimadamore >> > >: >> >> Also, please try with latest snapshot for JDK 9 >> >> https://jdk9.java.net/download/ >> >> It is possible that the 8 version is held back by something else >> >> Maurizio >> >> On 30/06/15 13:43, Maurizio Cimadamore wrote: >>> (Adding Jan) >>> this is weird - in our tests memory usage went down >>> dramatically. Jan, is your fix for 8039262 >>> in JDK >>> 8u60 b21 (the latest from the early access page) ? >>> >>> Maurizio >>> >>> >>> On 30/06/15 13:40, Jacob Wieland wrote: >>>> unfortunately, there doesn't seem to be any improvement >>>> with that build that I can observe. still uses up all >>>> cpu-capacity and doesn't take less time, either. >>>> >>>> 2015-06-30 12:48 GMT+02:00 Maurizio Cimadamore >>>> >>> >: >>>> >>>> >>>> >>>> On 30/06/15 11:45, Jacob Wieland wrote: >>>>> Thank you, >>>>> >>>>> why doesn't the bug appear in the change-set? >>>>> >>>> It's been fixed in b20, so b21 changeset won't show it. >>>> Our early access website only maintains info about the >>>> latest available build. >>>> >>>> Maurizio >>>>> BR >>>>> >>>>> 2015-06-30 11:49 GMT+02:00 Maurizio Cimadamore >>>>> >>>> >: >>>>> >>>>> Hi Jacob, >>>>> this has been fixed in 9 and 8 - our sustaining >>>>> team will take it from there and backport to 7 >>>>> eventually, but I don't have visibility over their >>>>> integration schedule. >>>>> >>>>> The next JDK 8 public update should have the fix - >>>>> in the meantime, you could try with an early >>>>> access build here: >>>>> >>>>> https://jdk8.java.net/download.html >>>>> >>>>> Maurizio >>>>> >>>>> >>>>> On 30/06/15 07:33, Jacob Wieland wrote: >>>>>> Hello Maurizio, >>>>>> >>>>>> any news on this issue? >>>>>> >>>>>> BR >>>>>> >>>>>> 2015-04-30 17:51 GMT+02:00 Maurizio Cimadamore >>>>>> >>>>> >: >>>>>> >>>>>> On 30/04/15 16:15, Jacob Wieland wrote: >>>>>>> thank you very much - very interesting. >>>>>>> >>>>>>> is there any estimate when a patch for 7/8 >>>>>>> will be available? >>>>>> We are currently playing with different >>>>>> patches - we will update this discussion when >>>>>> we know more about which steps will be taken. >>>>>> >>>>>> Regards >>>>>> Maurizio >>>>>> >>>>>>> >>>>>>> 2015-04-30 17:03 GMT+02:00 Maurizio >>>>>>> Cimadamore >>>>>> >: >>>>>>> >>>>>>> For the records - I've added a comment in >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8039262 >>>>>>> >>>>>>> Reporting in as much detail as possible >>>>>>> the underlying cause of the performance >>>>>>> regression. >>>>>>> >>>>>>> Maurizio >>>>>>> >>>>>>> >>>>>>> On 29/04/15 18:38, Jan Lahoda wrote: >>>>>>> >>>>>>> In 9, the Scope listeners are >>>>>>> (AFAIK) only needed to update the >>>>>>> "mark" (so that the Types caches can >>>>>>> detect obsolete entries). I think we >>>>>>> may be able to avoid the listeners >>>>>>> at the cost of slowing down getMark >>>>>>> somewhat (getMark would ask the >>>>>>> sub-scopes for their marks and sum >>>>>>> the result). I'll try. >>>>>>> >>>>>>> Jan >>>>>>> >>>>>>> On 29.4.2015 19:16, Maurizio >>>>>>> Cimadamore wrote: >>>>>>> >>>>>>> Found it. >>>>>>> >>>>>>> The big offender is the >>>>>>> listeners field in the >>>>>>> CompoundScope (which is a >>>>>>> list). >>>>>>> >>>>>>> The test ends up creating huge >>>>>>> listeners lists - probably >>>>>>> because of >>>>>>> very deep inheritance hierarchies. >>>>>>> >>>>>>> If I avoid adding stuff to the >>>>>>> listeners field in the compound >>>>>>> scope, I >>>>>>> get back to a sane scenario: >>>>>>> >>>>>>> real 0m32.540s >>>>>>> user 1m15.298s >>>>>>> sys 0m1.126s >>>>>>> >>>>>>> >>>>>>> And the profiled memory usage is >>>>>>> much more under control. >>>>>>> >>>>>>> So, looks like we need a way to >>>>>>> prevent these listeners list to >>>>>>> overwhelm javac ;-) >>>>>>> >>>>>>> Maurizio >>>>>>> >>>>>>> On 29/04/15 17:51, >>>>>>> Vicente-Arturo Romero-Zaldivar >>>>>>> wrote: >>>>>>> >>>>>>> On 04/29/2015 09:43 AM, >>>>>>> Maurizio Cimadamore wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 29/04/15 17:01, Jan >>>>>>> Lahoda wrote: >>>>>>> >>>>>>> On 29.4.2015 16:06, >>>>>>> Jacob Wieland wrote: >>>>>>> >>>>>>> I have to admit, >>>>>>> the reproducer >>>>>>> is only a small >>>>>>> part of the actual >>>>>>> generated code. >>>>>>> In my >>>>>>> preliminary >>>>>>> tests, it >>>>>>> already sufficed >>>>>>> to show a >>>>>>> difference >>>>>>> (also, the >>>>>>> smaller code >>>>>>> still worked >>>>>>> with 32 bit >>>>>>> while the >>>>>>> whole code runs >>>>>>> out of memory in >>>>>>> all 32 bit >>>>>>> versions which makes >>>>>>> comparison much >>>>>>> harder ;-) and >>>>>>> which is why I >>>>>>> need the 64 bit >>>>>>> versions). >>>>>>> If I test with >>>>>>> the complete >>>>>>> code which is >>>>>>> much bigger, the >>>>>>> results are >>>>>>> as follows: >>>>>>> >>>>>>> jdk1.6u45(64bit) >>>>>>> 2GB MaxMem - >>>>>>> 1:30 minutes >>>>>>> jdk1.7u75(64bit) >>>>>>> 2GB MaxMem - > 6 min >>>>>>> jdk1.8u31(64bit) >>>>>>> 1GB MaxMem - > >>>>>>> 15 min >>>>>>> jdk1.8u31(64bit) >>>>>>> 2GB MaxMem - > >>>>>>> 10 min >>>>>>> jdk1.8u31(64bit) >>>>>>> 4GB MaxMem - >>>>>>> 2:20 min >>>>>>> (-source/-target >>>>>>> 6 does not >>>>>>> seem >>>>>>> to have any effect) >>>>>>> >>>>>>> So, if you throw >>>>>>> insane (in >>>>>>> comparison with >>>>>>> 1.6) amounts of >>>>>>> memory at >>>>>>> 1.7/1.8, it is >>>>>>> only about a >>>>>>> third as slow, >>>>>>> but this still is >>>>>>> unacceptable. I >>>>>>> actually think >>>>>>> it has to do >>>>>>> with >>>>>>> parallellization and >>>>>>> >>>>>>> >>>>>>> When I was looking >>>>>>> at JDK-8039262, a >>>>>>> significant >>>>>>> contributing factor >>>>>>> to the slowdown >>>>>>> (with enough memory >>>>>>> and -source/-target >>>>>>> 6) appeared >>>>>>> to be >>>>>>> Check.checkOverrideClashes >>>>>>> - I believe this >>>>>>> does checks that >>>>>>> were not properly >>>>>>> implemented in 6, >>>>>>> contributing to the >>>>>>> difference >>>>>>> between 6 and 7 >>>>>>> (which seems to be >>>>>>> particularly visible >>>>>>> for this >>>>>>> testcase). I was >>>>>>> looking at the >>>>>>> checks a few times, >>>>>>> trying to write >>>>>>> them >>>>>>> differently/faster >>>>>>> while still >>>>>>> performing the >>>>>>> checks, but was >>>>>>> not successful in >>>>>>> that yet, unfortunately. >>>>>>> >>>>>>> I see the issue now - it >>>>>>> is reproducible with the >>>>>>> following memory >>>>>>> parameter (at least in >>>>>>> my machine): >>>>>>> >>>>>>> -J-Xmx768M >>>>>>> >>>>>>> This give around 20sec >>>>>>> in JDK 6 and 10+ minutes >>>>>>> in JDK 8. >>>>>>> >>>>>>> All the time seems to be >>>>>>> spent in desugaring, >>>>>>> most specifically in >>>>>>> TransTypes.addBridges - >>>>>>> it seems like that >>>>>>> method calls >>>>>>> Types.implementation a >>>>>>> lot - so my theory was >>>>>>> that the fact that >>>>>>> javac consumes more >>>>>>> memory, forces the GC to >>>>>>> get rid of the cached >>>>>>> entries in the >>>>>>> implementation/members >>>>>>> closure caches (since such >>>>>>> entries are deliberately >>>>>>> backed up by >>>>>>> SoftReferences), which >>>>>>> in turn >>>>>>> completely trashes >>>>>>> performances. I >>>>>>> instrumented the code a >>>>>>> bit and >>>>>>> this is what I found: >>>>>>> >>>>>>> *) With -Xmx768M >>>>>>> >>>>>>> Impl cache misses = 3346926 >>>>>>> Members cache misses = >>>>>>> 1042678 >>>>>>> >>>>>>> real 7m0.335s >>>>>>> user 25m51.517s >>>>>>> sys 0m4.947s >>>>>>> >>>>>>> >>>>>>> *) W/o -Xmx768M >>>>>>> >>>>>>> Impl cache misses = 3346839 >>>>>>> Members cache misses = >>>>>>> 1042678 >>>>>>> >>>>>>> real 0m32.377s >>>>>>> user 1m25.881s >>>>>>> sys 0m2.232s >>>>>>> >>>>>>> Long story short - cache >>>>>>> misses do not play a >>>>>>> factor in here - there >>>>>>> are some minor >>>>>>> differences, but nothing >>>>>>> out of the ordinary and defo >>>>>>> nothing that would >>>>>>> explain a multi-minute >>>>>>> slowdown! Any ideas? >>>>>>> >>>>>>> >>>>>>> use flight recorder? >>>>>>> >>>>>>> Vicente >>>>>>> >>>>>>> >>>>>>> Maurizio >>>>>>> >>>>>>> >>>>>>> Jan >>>>>>> >>>>>>> garbage collection. >>>>>>> >>>>>>> 2015-04-29 15:12 >>>>>>> GMT+02:00 >>>>>>> Maurizio Cimadamore >>>>>>> >>>>>> >>>>>>> >>>>>> >>: >>>>>>> >>>>>>> On 29/04/15 >>>>>>> 12:44, Jacob >>>>>>> Wieland wrote: >>>>>>> >>>>>>> Hello >>>>>>> Maurizio, >>>>>>> >>>>>>> are you >>>>>>> sure that >>>>>>> you used the >>>>>>> 64bit >>>>>>> versions of >>>>>>> javac? I could >>>>>>> only >>>>>>> observe the >>>>>>> behavior >>>>>>> with these. >>>>>>> >>>>>>> Yep I'm on a >>>>>>> Ubuntu x64 >>>>>>> machine. It's >>>>>>> actually pretty >>>>>>> standard >>>>>>> hardware too >>>>>>> - i.e. intel i5 >>>>>>> (two cores, but >>>>>>> OS sees 4 because of >>>>>>> hyper-threading). >>>>>>> >>>>>>> Also, I >>>>>>> just tried >>>>>>> with >>>>>>> jdk8u31-64b >>>>>>> and it takes >>>>>>> AGES (still >>>>>>> running >>>>>>> after 17 >>>>>>> minutes >>>>>>> where the >>>>>>> jdk6 was >>>>>>> done after >>>>>>> 2), top >>>>>>> shows >>>>>>> 4GB VIRT >>>>>>> memory use >>>>>>> and 350 % >>>>>>> load (on a >>>>>>> 4core >>>>>>> processor). >>>>>>> >>>>>>> Maybe the >>>>>>> reproducer you >>>>>>> sent was incorrect? >>>>>>> >>>>>>> Maurizio >>>>>>> >>>>>>> >>>>>>> So, I >>>>>>> don't think >>>>>>> it was that >>>>>>> problem. >>>>>>> >>>>>>> >>>>>>> 2015-04-29 >>>>>>> 12:00 >>>>>>> GMT+02:00 >>>>>>> Maurizio >>>>>>> Cimadamore >>>>>>> >>>>>>> >>>>>> >>>>>>> >>>>>> >>: >>>>>>> >>>>>>> >>>>>>> These are >>>>>>> the numbers >>>>>>> I'm getting: >>>>>>> >>>>>>> JDK >>>>>>> 9 (b42) >>>>>>> >>>>>>> >>>>>>> Note: >>>>>>> generated_ttcn/TTCN3_CommonDefs.java >>>>>>> uses or >>>>>>> overrides a >>>>>>> deprecated API. >>>>>>> >>>>>>> Note: >>>>>>> Recompile >>>>>>> with >>>>>>> -Xlint:deprecation >>>>>>> for details. >>>>>>> >>>>>>> real >>>>>>> 0m46.306s >>>>>>> user >>>>>>> 2m17.489s >>>>>>> sys >>>>>>> 0m2.166s >>>>>>> >>>>>>> JDK >>>>>>> 8 (GA) >>>>>>> >>>>>>> >>>>>>> Note: >>>>>>> generated_ttcn/TTCN3_CommonDefs.java >>>>>>> uses or >>>>>>> overrides a >>>>>>> deprecated API. >>>>>>> >>>>>>> Note: >>>>>>> Recompile >>>>>>> with >>>>>>> -Xlint:deprecation >>>>>>> for details. >>>>>>> >>>>>>> real >>>>>>> 6m58.748s >>>>>>> user >>>>>>> 8m43.546s >>>>>>> sys >>>>>>> 0m2.132s >>>>>>> >>>>>>> JDK >>>>>>> 7 (1.7.0_79) >>>>>>> >>>>>>> >>>>>>> Note: >>>>>>> generated_ttcn/TTCN3_CommonDefs.java >>>>>>> uses or >>>>>>> overrides a >>>>>>> deprecated API. >>>>>>> >>>>>>> Note: >>>>>>> Recompile >>>>>>> with >>>>>>> -Xlint:deprecation >>>>>>> for details. >>>>>>> >>>>>>> real >>>>>>> 0m28.341s >>>>>>> user >>>>>>> 1m17.194s >>>>>>> sys >>>>>>> 0m1.886s >>>>>>> >>>>>>> >>>>>>> As >>>>>>> you can see >>>>>>> there is a >>>>>>> significant >>>>>>> regression >>>>>>> from JDK >>>>>>> 7 to >>>>>>> JDK >>>>>>> 8 which was >>>>>>> caused by >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8043253 >>>>>>> >>>>>>> >>>>>>> (some stack >>>>>>> trace >>>>>>> analysis >>>>>>> revealed the >>>>>>> familiar >>>>>>> pattern). >>>>>>> This >>>>>>> has also >>>>>>> been fixed >>>>>>> in JDK 8u20 >>>>>>> (as stated >>>>>>> in the bug >>>>>>> evaluation). >>>>>>> >>>>>>> So, >>>>>>> while JDK >>>>>>> 8u20/9 is >>>>>>> slower than >>>>>>> JDK 7 (at >>>>>>> least on my >>>>>>> machine), >>>>>>> the numbers >>>>>>> are more or >>>>>>> less in the >>>>>>> same ballpark >>>>>>> and >>>>>>> the huge >>>>>>> regression >>>>>>> that was >>>>>>> visible in >>>>>>> earlier JDK 8 >>>>>>> releases has >>>>>>> now been fixed. >>>>>>> >>>>>>> If >>>>>>> you are >>>>>>> still >>>>>>> experiencing >>>>>>> the problem >>>>>>> - can you please >>>>>>> also >>>>>>> submit the >>>>>>> specific >>>>>>> compiler >>>>>>> versions you >>>>>>> are using in >>>>>>> your >>>>>>> benchmark? >>>>>>> >>>>>>> Maurizio >>>>>>> >>>>>>> >>>>>>> >>>>>>> On >>>>>>> 29/04/15 >>>>>>> 10:29, >>>>>>> Maurizio >>>>>>> Cimadamore >>>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> Hi Jacob, >>>>>>> >>>>>>> Stay >>>>>>> assured >>>>>>> - as >>>>>>> we'll >>>>>>> definitively >>>>>>> look >>>>>>> into >>>>>>> this >>>>>>> issue (I >>>>>>> >>>>>>> see it's >>>>>>> already >>>>>>> assigned >>>>>>> to one >>>>>>> of my >>>>>>> colleagues). >>>>>>> >>>>>>> >>>>>>> What I >>>>>>> can say >>>>>>> (w/o >>>>>>> looking >>>>>>> too much >>>>>>> at the >>>>>>> attached >>>>>>> artifacts) >>>>>>> is that >>>>>>> in >>>>>>> general, >>>>>>> javac >>>>>>> has no >>>>>>> issue with >>>>>>> compiling a >>>>>>> lot of >>>>>>> sources >>>>>>> at once; >>>>>>> at one >>>>>>> point >>>>>>> the build >>>>>>> >>>>>>> system >>>>>>> was >>>>>>> structured >>>>>>> in such >>>>>>> a way >>>>>>> that all >>>>>>> the JDK >>>>>>> classes >>>>>>> >>>>>>> were >>>>>>> compiled >>>>>>> at once >>>>>>> - and >>>>>>> that >>>>>>> (which >>>>>>> is way >>>>>>> more than >>>>>>> your >>>>>>> >>>>>>> 187 >>>>>>> sources >>>>>>> - i.e. >>>>>>> at least >>>>>>> 10x >>>>>>> that) >>>>>>> took >>>>>>> less than 20 >>>>>>> seconds. >>>>>>> SO there >>>>>>> must >>>>>>> some >>>>>>> specific >>>>>>> pattern >>>>>>> triggering >>>>>>> that >>>>>>> >>>>>>> issue. >>>>>>> >>>>>>> >>>>>>> Given >>>>>>> that you >>>>>>> say you >>>>>>> have 187 >>>>>>> input >>>>>>> sources >>>>>>> and 48K >>>>>>> output >>>>>>> classes, >>>>>>> I'd say >>>>>>> you are >>>>>>> using >>>>>>> inner >>>>>>> classes >>>>>>> a lot. I >>>>>>> wonder >>>>>>> >>>>>>> if you >>>>>>> are >>>>>>> hitting >>>>>>> this: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8000316 >>>>>>> >>>>>>> Maurizio >>>>>>> >>>>>>> >>>>>>> On >>>>>>> 24/04/15 >>>>>>> 09:49, >>>>>>> Jacob >>>>>>> Wieland >>>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> Hello Folks, >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> I >>>>>>> still have >>>>>>> the >>>>>>> open >>>>>>> problem >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8039262 >>>>>>> that the >>>>>>> >>>>>>> >>>>>>> javac performance >>>>>>> has >>>>>>> degraded >>>>>>> significantly >>>>>>> from >>>>>>> 1.6 to >>>>>>> 1.7 >>>>>>> >>>>>>> >>>>>>> (and >>>>>>> even >>>>>>> worse to >>>>>>> 1.8) >>>>>>> in >>>>>>> the >>>>>>> 64bit versions. >>>>>>> Since in >>>>>>> our >>>>>>> context, >>>>>>> we >>>>>>> are >>>>>>> dealing >>>>>>> with >>>>>>> a >>>>>>> lot >>>>>>> of >>>>>>> generated >>>>>>> source1.4 >>>>>>> >>>>>>> >>>>>>> Java >>>>>>> input (either >>>>>>> split into >>>>>>> very >>>>>>> large files >>>>>>> with >>>>>>> inner >>>>>>> classes >>>>>>> or >>>>>>> big >>>>>>> packages >>>>>>> with >>>>>>> lots >>>>>>> of >>>>>>> smaller >>>>>>> classes), >>>>>>> compiler >>>>>>> performance >>>>>>> is >>>>>>> critical >>>>>>> for >>>>>>> our >>>>>>> tool >>>>>>> and this >>>>>>> degradation >>>>>>> forces >>>>>>> us >>>>>>> to >>>>>>> continue >>>>>>> recommending >>>>>>> to our >>>>>>> customers >>>>>>> to >>>>>>> use >>>>>>> Java >>>>>>> 1.6 >>>>>>> for >>>>>>> large projects >>>>>>> (as >>>>>>> is the >>>>>>> >>>>>>> >>>>>>> norm) as >>>>>>> 1.7 >>>>>>> and >>>>>>> 1.8 >>>>>>> are >>>>>>> pretty >>>>>>> much >>>>>>> unusable >>>>>>> in this >>>>>>> respect. >>>>>>> >>>>>>> >>>>>>> >>>>>>> Is >>>>>>> anyone >>>>>>> still working >>>>>>> on >>>>>>> this >>>>>>> issue or >>>>>>> is such >>>>>>> significant >>>>>>> performance >>>>>>> degradation >>>>>>> not >>>>>>> a >>>>>>> serious >>>>>>> issue? >>>>>>> >>>>>>> >>>>>>> >>>>>>> My >>>>>>> observations >>>>>>> so >>>>>>> far are: >>>>>>> >>>>>>> >>>>>>> - it >>>>>>> gets >>>>>>> worse the >>>>>>> more >>>>>>> class files >>>>>>> are >>>>>>> being compiled/the >>>>>>> >>>>>>> >>>>>>> more >>>>>>> files reside >>>>>>> in >>>>>>> the >>>>>>> source >>>>>>> path >>>>>>> >>>>>>> >>>>>>> - >>>>>>> cpu >>>>>>> usage goes >>>>>>> through >>>>>>> the >>>>>>> roof >>>>>>> over >>>>>>> all >>>>>>> available >>>>>>> kernels >>>>>>> >>>>>>> >>>>>>> - >>>>>>> memory >>>>>>> usage is >>>>>>> much >>>>>>> higher >>>>>>> >>>>>>> >>>>>>> - >>>>>>> garbage >>>>>>> collection >>>>>>> seems to >>>>>>> be >>>>>>> much >>>>>>> more >>>>>>> active >>>>>>> >>>>>>> >>>>>>> >>>>>>> Using -proc:none >>>>>>> alleviates >>>>>>> the >>>>>>> problem >>>>>>> slightly >>>>>>> for the >>>>>>> 1.7 >>>>>>> version, >>>>>>> but >>>>>>> not >>>>>>> for >>>>>>> 1.8 >>>>>>> (last we >>>>>>> tested >>>>>>> with >>>>>>> jdk1.8.0_31) >>>>>>> where the >>>>>>> performance >>>>>>> difference >>>>>>> is a >>>>>>> factor 5 >>>>>>> >>>>>>> >>>>>>> or more! >>>>>>> >>>>>>> >>>>>>> >>>>>>> I >>>>>>> can >>>>>>> understand >>>>>>> that >>>>>>> a >>>>>>> more >>>>>>> advanced >>>>>>> compiler >>>>>>> has >>>>>>> capabilities >>>>>>> that >>>>>>> a >>>>>>> previous >>>>>>> version >>>>>>> does >>>>>>> not >>>>>>> have >>>>>>> and thus >>>>>>> sometimes >>>>>>> has >>>>>>> >>>>>>> >>>>>>> to >>>>>>> do >>>>>>> more >>>>>>> work. But, >>>>>>> it >>>>>>> should >>>>>>> still be >>>>>>> possible >>>>>>> (especially >>>>>>> if >>>>>>> given the >>>>>>> -source >>>>>>> 1.4 >>>>>>> or >>>>>>> -source >>>>>>> 1.5 >>>>>>> option >>>>>>> >>>>>>> >>>>>>> as >>>>>>> we >>>>>>> do) >>>>>>> to >>>>>>> optimize >>>>>>> it >>>>>>> in >>>>>>> such >>>>>>> a >>>>>>> way >>>>>>> that >>>>>>> unnecessary >>>>>>> >>>>>>> >>>>>>> checks >>>>>>> for >>>>>>> generics, >>>>>>> overriding >>>>>>> methods, >>>>>>> closures, >>>>>>> annotations >>>>>>> and >>>>>>> other newer >>>>>>> features >>>>>>> can >>>>>>> be >>>>>>> turned >>>>>>> off (if >>>>>>> >>>>>>> >>>>>>> they >>>>>>> are >>>>>>> to >>>>>>> blame, >>>>>>> which I >>>>>>> actually >>>>>>> doubt from >>>>>>> my >>>>>>> observations). >>>>>>> >>>>>>> >>>>>>> >>>>>>> I >>>>>>> would really >>>>>>> appreciate >>>>>>> your >>>>>>> help >>>>>>> in >>>>>>> this >>>>>>> regard >>>>>>> and I >>>>>>> >>>>>>> >>>>>>> think everyone >>>>>>> would benefit >>>>>>> from >>>>>>> any >>>>>>> bugfix >>>>>>> you >>>>>>> can >>>>>>> offer >>>>>>> >>>>>>> >>>>>>> for >>>>>>> this. >>>>>>> >>>>>>> >>>>>>> >>>>>>> BR, >>>>>>> Jacob Wieland >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> -- >>>>>>> ------------------------------ >>>>>>> ------------------------------------------- >>>>>>> >>>>>>> >>>>>>> Dr. >>>>>>> Jacob Wieland >>>>>>> >>>>>>> >>>>>>> Senior >>>>>>> Software >>>>>>> Engineer >>>>>>> >>>>>>> Testing >>>>>>> Technologies >>>>>>> IST GmbH >>>>>>> Michaelkirchstra?e >>>>>>> 17/18 >>>>>>> >>>>>>> >>>>>>> 10179 Berlin, >>>>>>> Germany >>>>>>> >>>>>>> >>>>>>> >>>>>>> Phone +49 >>>>>>> 30 >>>>>>> 726 >>>>>>> 19 >>>>>>> 19 >>>>>>> 34 >>>>>>> >>>>>>> Email >>>>>>> wieland at testingtech.com >>>>>>> >>>>>>> >>>>>> > >>>>>>> >>>>>>> >>>>>>> Fax >>>>>>> +49 >>>>>>> 30 >>>>>>> 726 >>>>>>> 19 >>>>>>> 19 >>>>>>> 20 >>>>>>> >>>>>>> >>>>>>> Internet >>>>>>> www.testingtech.com >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> --------------------------------------------------------------------------------------------------------------- >>>>>>> >>>>>>> >>>>>>> >>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>> >>>>>>> >>>>>>> >>>>>>> UPCOMING >>>>>>> EVENTS >>>>>>> >>>>>>> >>>>>>> >>>>>>> SUBMIT >>>>>>> YOUR >>>>>>> TOPIC for >>>>>>> the >>>>>>> UCAAT 2015 >>>>>>> Deadline: >>>>>>> May >>>>>>> 30, 2015 >>>>>>> ucaat.etsi.org/2015/CallForPresentations.html >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Apr >>>>>>> 21-23, >>>>>>> 2015 >>>>>>> | >>>>>>> SAE >>>>>>> Conference >>>>>>> & >>>>>>> Exhibition >>>>>>> Detroit, >>>>>>> Michigan, >>>>>>> USA >>>>>>> www.sae.org/congress/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Apr >>>>>>> 28-30, >>>>>>> 2015 >>>>>>> | iqnite >>>>>>> Dusseldorf, >>>>>>> Germany >>>>>>> www.iqnite-conferences.com/de/index.aspx >>>>>>> >>>>>>> >>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>> >>>>>>> >>>>>>> Gesch?ftsf?hrung: >>>>>>> Theofanis >>>>>>> Vassiliou-Gioles, >>>>>>> Stephan >>>>>>> Pietsch, >>>>>>> Pete >>>>>>> Nicholson >>>>>>> Handelsregister >>>>>>> HRB >>>>>>> 77805 B, >>>>>>> Amtsgericht >>>>>>> Charlottenburg >>>>>>> Ust >>>>>>> ID >>>>>>> Nr.: >>>>>>> DE >>>>>>> 813 >>>>>>> 143 >>>>>>> 070 This >>>>>>> >>>>>>> >>>>>>> email may >>>>>>> contain >>>>>>> confidential >>>>>>> and >>>>>>> privileged >>>>>>> material >>>>>>> for >>>>>>> >>>>>>> >>>>>>> the >>>>>>> sole >>>>>>> use >>>>>>> of >>>>>>> the >>>>>>> intended >>>>>>> recipient. >>>>>>> Any >>>>>>> review, >>>>>>> use, >>>>>>> distribution >>>>>>> or >>>>>>> disclosure >>>>>>> by >>>>>>> others >>>>>>> is >>>>>>> strictly >>>>>>> prohibited. >>>>>>> >>>>>>> >>>>>>> If >>>>>>> you >>>>>>> are >>>>>>> not >>>>>>> the >>>>>>> intended >>>>>>> recipient >>>>>>> (or >>>>>>> authorized >>>>>>> to >>>>>>> receive >>>>>>> for >>>>>>> the >>>>>>> recipient), >>>>>>> please >>>>>>> contact >>>>>>> the >>>>>>> sender >>>>>>> by >>>>>>> >>>>>>> >>>>>>> reply email >>>>>>> and >>>>>>> delete >>>>>>> all >>>>>>> copies >>>>>>> of >>>>>>> this >>>>>>> message. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> -- >>>>>>> ------------------------------ >>>>>>> ------------------------------------------- >>>>>>> Dr. >>>>>>> Jacob Wieland >>>>>>> Senior >>>>>>> Software >>>>>>> Engineer >>>>>>> >>>>>>> Testing >>>>>>> Technologies >>>>>>> IST GmbH >>>>>>> Michaelkirchstra?e >>>>>>> 17/18 >>>>>>> 10179 >>>>>>> Berlin, Germany >>>>>>> >>>>>>> Phone >>>>>>> +49 30 726 >>>>>>> 19 19 34 >>>>>>> >>>>>>> Email >>>>>>> wieland at testingtech.com >>>>>>> >>>>>>> >>>>>> > >>>>>>> Fax +49 >>>>>>> 30 726 19 19 >>>>>>> 20 >>>>>>> >>>>>>> Internet >>>>>>> www.testingtech.com >>>>>>> >>>>>>> >>>>>>> >>>>>>> --------------------------------------------------------------------------------------------------------------- >>>>>>> >>>>>>> >>>>>>> >>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>> >>>>>>> >>>>>>> >>>>>>> UPCOMING >>>>>>> EVENTS >>>>>>> >>>>>>> SUBMIT >>>>>>> YOUR TOPIC >>>>>>> for the >>>>>>> UCAAT 2015 >>>>>>> >>>>>>> Deadline: >>>>>>> May 30, 2015 >>>>>>> ucaat.etsi.org/2015/CallForPresentations.html >>>>>>> >>>>>>> >>>>>>> >>>>>>> Apr >>>>>>> 21-23, 2015 >>>>>>> | SAE >>>>>>> Conference & >>>>>>> Exhibition >>>>>>> Detroit, >>>>>>> Michigan, USA >>>>>>> www.sae.org/congress/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> Apr >>>>>>> 28-30, 2015 >>>>>>> | iqnite >>>>>>> Dusseldorf, >>>>>>> Germany >>>>>>> www.iqnite-conferences.com/de/index.aspx >>>>>>> >>>>>>> >>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>> >>>>>>> > ... > > [Nachricht gek?rzt] > > > > > -- > -- > ------------------------------ > ------------------------------------------- > Dr. Jacob Wieland > Senior Software Engineer > > Testing Technologies IST GmbH > Michaelkirchstra?e 17/18 > 10179 Berlin, Germany > > Phone +49 30 726 19 19 34 Email wieland at testingtech.com > > Fax +49 30 726 19 19 20 Internet www.testingtech.com > > > --------------------------------------------------------------------------------------------------------------------- > > UPCOMING EVENTS > > Jul 15-16, 2015 | German Testing Day > Frankfurt, Germany | Booth #8 > www.germantestingday.info/ > > Sep 15-17, 2015 | Accredited TTCN-3 Training "Test Automation with TTCN-3" > Berlin, Germany > www.testingtech.com/services/ttcn3_training.php > > > Oct 5-9, 2015 | 22nd ITS World Congress > Bordeaux, France | At Spirent's Booth #122 > www.itsineurope.com/bordeaux-2015/ > > > ----------------------------------------------------------------------------------------------------------------- > > Testing Technologies IST GmbH > Michaelkirchstra?e 17/18, 10179 Berlin, Germany > Office: +49 (30) 726 19 19 0 > Fax: +49 (30) 726 19 19 20 > Internet www.testingtech.com > ----------------------------------------------------------------------------------------------------------------- > > Executive Board: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete > Nicholson > Handelsregister HRB 77805 B, Amtsgericht Charlottenburg > Ust ID Nr.: DE 813 143 070 > > This email may contain confidential and privileged material for the > sole use of the intended recipient. Any review, use, distribution or > disclosure by others is strictly prohibited. If you are not the > intended recipient (or authorized to receive for the recipient), > please contact the sender by reply email and delete all copies of this > message. > > > > -- > -- > ------------------------------ > ------------------------------------------- > Dr. Jacob Wieland > Senior Software Engineer > > Testing Technologies IST GmbH > Michaelkirchstra?e 17/18 > 10179 Berlin, Germany > > Phone +49 30 726 19 19 34 Email wieland at testingtech.com > > Fax +49 30 726 19 19 20 Internet www.testingtech.com > > > --------------------------------------------------------------------------------------------------------------------- > > UPCOMING EVENTS > > Jul 15-16, 2015 | German Testing Day > Frankfurt, Germany | Booth #8 > www.germantestingday.info/ > > Sep 15-17, 2015 | Accredited TTCN-3 Training "Test Automation with TTCN-3" > Berlin, Germany > www.testingtech.com/services/ttcn3_training.php > > > Oct 5-9, 2015 | 22nd ITS World Congress > Bordeaux, France | At Spirent's Booth #122 > www.itsineurope.com/bordeaux-2015/ > > > ----------------------------------------------------------------------------------------------------------------- > > Testing Technologies IST GmbH > Michaelkirchstra?e 17/18, 10179 Berlin, Germany > Office: +49 (30) 726 19 19 0 > Fax: +49 (30) 726 19 19 20 > Internet www.testingtech.com > ----------------------------------------------------------------------------------------------------------------- > > Executive Board: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete > Nicholson > Handelsregister HRB 77805 B, Amtsgericht Charlottenburg > Ust ID Nr.: DE 813 143 070 > > This email may contain confidential and privileged material for the > sole use of the intended recipient. Any review, use, distribution or > disclosure by others is strictly prohibited. If you are not the > intended recipient (or authorized to receive for the recipient), > please contact the sender by reply email and delete all copies of this > message. -------------- next part -------------- An HTML attachment was scrubbed... URL: From wieland at testingtech.com Wed Jul 1 11:26:40 2015 From: wieland at testingtech.com (Jacob Wieland) Date: Wed, 1 Jul 2015 13:26:40 +0200 Subject: Fwd: Java Performance Degradation in JDK7 and JDK8 In-Reply-To: <5593C574.50803@oracle.com> References: <55410052.5080205@oracle.com> <55410A1A.1090104@oracle.com> <55410C10.9010108@oracle.com> <554111E1.10305@oracle.com> <55411708.3020109@oracle.com> <5542443E.6030803@oracle.com> <55424F96.8050705@oracle.com> <55926630.6020401@oracle.com> <559273F8.4070103@oracle.com> <55928F0C.8020003@oracle.com> <55928F84.7060501@oracle.com> <55929BEC.1040002@oracle.com> <5593C574.50803@oracle.com> Message-ID: No, in reality, the generated code to be compiled is much larger, the given small example is just a small sub-part of it (but large enough so that the degradation is showing). Since memory issues scale up the larger the input is, I am still stuck with a non-performance for the real-world input unless I throw insane amounts of memory at the compiler (which some of our customers cannot do). 2015-07-01 12:48 GMT+02:00 Maurizio Cimadamore < maurizio.cimadamore at oracle.com>: > What do you mean by 'full example' - the one you attached in the JBS bug? > > Maurizio > > On 01/07/15 09:58, Jacob Wieland wrote: > > I have tried again with the original code generation scheme (sorry, last > tests were with the new modular one) for the full example and got the > following measurements: > > 1.6 - 60 sec > 1.8 - 90 sec (CPU > 300) > 1.9 - 110 sec (CPU > 300) > > I also tried with the new modular code generation scheme (using > top-level classes instead of inner classes) > > 1.6 - 86 sec > 1.8 - 142 sec (CPU > 300) > 1.9 - 128 sec (CPU > 300) > > The CPU usage of 1.8 and 1.9 where also much higher in 1.8 and 1.9. > > Using memory restrictions (-J-Xmx2000M), I got the following results on > the modular scheme: > > 1.6 - 101 sec (CPU max 160, normally around 100) > 1.8 - 1168 sec (CPU continuously around 350) > 1.9 - 194 sec (CPU max 360, almost continuously) > > If you are interested, we can exchange the testcases in some other way, > I guess. They are too large to send. > > BR > > > 2015-06-30 15:38 GMT+02:00 Maurizio Cimadamore < > maurizio.cimadamore at oracle.com>: > >> I tried this on my not-so-good laptop, and I can get your originally >> submitted test case to complete in 50 seconds with the latest JDK 9; >> doesn't seem to even be able to complete with an older JDK 9 (I'm using >> -Xmx764M to artificially limit the memory). So, it seems like performances >> are back to what the bug report says you were getting for 1.6 ? >> >> Maurizio >> >> >> On 30/06/15 13:56, Jacob Wieland wrote: >> >> sorry to say it, but the jdk 9 version isn't any better (in that respect) >> either ... >> >> 2015-06-30 14:45 GMT+02:00 Maurizio Cimadamore < >> maurizio.cimadamore at oracle.com>: >> >>> Also, please try with latest snapshot for JDK 9 >>> >>> https://jdk9.java.net/download/ >>> >>> It is possible that the 8 version is held back by something else >>> >>> Maurizio >>> >>> On 30/06/15 13:43, Maurizio Cimadamore wrote: >>> >>> (Adding Jan) >>> this is weird - in our tests memory usage went down dramatically. Jan, >>> is your fix for 8039262 >>> in JDK 8u60 b21 (the >>> latest from the early access page) ? >>> >>> Maurizio >>> >>> >>> On 30/06/15 13:40, Jacob Wieland wrote: >>> >>> unfortunately, there doesn't seem to be any improvement with that build >>> that I can observe. still uses up all cpu-capacity and doesn't take less >>> time, either. >>> >>> 2015-06-30 12:48 GMT+02:00 Maurizio Cimadamore < >>> maurizio.cimadamore at oracle.com>: >>> >>>> >>>> >>>> On 30/06/15 11:45, Jacob Wieland wrote: >>>> >>>> Thank you, >>>> >>>> why doesn't the bug appear in the change-set? >>>> >>>> It's been fixed in b20, so b21 changeset won't show it. Our early >>>> access website only maintains info about the latest available build. >>>> >>>> Maurizio >>>> >>>> BR >>>> >>>> 2015-06-30 11:49 GMT+02:00 Maurizio Cimadamore < >>>> maurizio.cimadamore at oracle.com>: >>>> >>>>> Hi Jacob, >>>>> this has been fixed in 9 and 8 - our sustaining team will take it from >>>>> there and backport to 7 eventually, but I don't have visibility over their >>>>> integration schedule. >>>>> >>>>> The next JDK 8 public update should have the fix - in the meantime, >>>>> you could try with an early access build here: >>>>> >>>>> https://jdk8.java.net/download.html >>>>> >>>>> Maurizio >>>>> >>>>> >>>>> On 30/06/15 07:33, Jacob Wieland wrote: >>>>> >>>>> Hello Maurizio, >>>>> >>>>> any news on this issue? >>>>> >>>>> BR >>>>> >>>>> 2015-04-30 17:51 GMT+02:00 Maurizio Cimadamore < >>>>> maurizio.cimadamore at oracle.com>: >>>>> >>>>>> On 30/04/15 16:15, Jacob Wieland wrote: >>>>>> >>>>>> thank you very much - very interesting. >>>>>> >>>>>> is there any estimate when a patch for 7/8 will be available? >>>>>> >>>>>> We are currently playing with different patches - we will update >>>>>> this discussion when we know more about which steps will be taken. >>>>>> >>>>>> Regards >>>>>> Maurizio >>>>>> >>>>>> >>>>>> 2015-04-30 17:03 GMT+02:00 Maurizio Cimadamore < >>>>>> maurizio.cimadamore at oracle.com>: >>>>>> >>>>>>> For the records - I've added a comment in >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8039262 >>>>>>> >>>>>>> Reporting in as much detail as possible the underlying cause of the >>>>>>> performance regression. >>>>>>> >>>>>>> Maurizio >>>>>>> >>>>>>> >>>>>>> On 29/04/15 18:38, Jan Lahoda wrote: >>>>>>> >>>>>>>> In 9, the Scope listeners are (AFAIK) only needed to update the >>>>>>>> "mark" (so that the Types caches can detect obsolete entries). I think we >>>>>>>> may be able to avoid the listeners at the cost of slowing down getMark >>>>>>>> somewhat (getMark would ask the sub-scopes for their marks and sum the >>>>>>>> result). I'll try. >>>>>>>> >>>>>>>> Jan >>>>>>>> >>>>>>>> On 29.4.2015 19:16, Maurizio Cimadamore wrote: >>>>>>>> >>>>>>>>> Found it. >>>>>>>>> >>>>>>>>> The big offender is the listeners field in the CompoundScope >>>>>>>>> (which is a >>>>>>>>> list). >>>>>>>>> >>>>>>>>> The test ends up creating huge listeners lists - probably because >>>>>>>>> of >>>>>>>>> very deep inheritance hierarchies. >>>>>>>>> >>>>>>>>> If I avoid adding stuff to the listeners field in the compound >>>>>>>>> scope, I >>>>>>>>> get back to a sane scenario: >>>>>>>>> >>>>>>>>> real 0m32.540s >>>>>>>>> user 1m15.298s >>>>>>>>> sys 0m1.126s >>>>>>>>> >>>>>>>>> >>>>>>>>> And the profiled memory usage is much more under control. >>>>>>>>> >>>>>>>>> So, looks like we need a way to prevent these listeners list to >>>>>>>>> overwhelm javac ;-) >>>>>>>>> >>>>>>>>> Maurizio >>>>>>>>> >>>>>>>>> On 29/04/15 17:51, Vicente-Arturo Romero-Zaldivar wrote: >>>>>>>>> >>>>>>>>>> On 04/29/2015 09:43 AM, Maurizio Cimadamore wrote: >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 29/04/15 17:01, Jan Lahoda wrote: >>>>>>>>>>> >>>>>>>>>>>> On 29.4.2015 16:06, Jacob Wieland wrote: >>>>>>>>>>>> >>>>>>>>>>>>> I have to admit, the reproducer is only a small part of the >>>>>>>>>>>>> actual >>>>>>>>>>>>> generated code. In my preliminary tests, it already sufficed >>>>>>>>>>>>> to show a >>>>>>>>>>>>> difference (also, the smaller code still worked with 32 bit >>>>>>>>>>>>> while the >>>>>>>>>>>>> whole code runs out of memory in all 32 bit versions which >>>>>>>>>>>>> makes >>>>>>>>>>>>> comparison much harder ;-) and which is why I need the 64 bit >>>>>>>>>>>>> versions). >>>>>>>>>>>>> If I test with the complete code which is much bigger, the >>>>>>>>>>>>> results are >>>>>>>>>>>>> as follows: >>>>>>>>>>>>> >>>>>>>>>>>>> jdk1.6u45(64bit) 2GB MaxMem - 1:30 minutes >>>>>>>>>>>>> jdk1.7u75(64bit) 2GB MaxMem - > 6 min >>>>>>>>>>>>> jdk1.8u31(64bit) 1GB MaxMem - > 15 min >>>>>>>>>>>>> jdk1.8u31(64bit) 2GB MaxMem - > 10 min >>>>>>>>>>>>> jdk1.8u31(64bit) 4GB MaxMem - 2:20 min (-source/-target 6 does >>>>>>>>>>>>> not >>>>>>>>>>>>> seem >>>>>>>>>>>>> to have any effect) >>>>>>>>>>>>> >>>>>>>>>>>>> So, if you throw insane (in comparison with 1.6) amounts of >>>>>>>>>>>>> memory at >>>>>>>>>>>>> 1.7/1.8, it is only about a third as slow, but this still is >>>>>>>>>>>>> unacceptable. I actually think it has to do with >>>>>>>>>>>>> parallellization and >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> When I was looking at JDK-8039262, a significant contributing >>>>>>>>>>>> factor >>>>>>>>>>>> to the slowdown (with enough memory and -source/-target 6) >>>>>>>>>>>> appeared >>>>>>>>>>>> to be Check.checkOverrideClashes - I believe this does checks >>>>>>>>>>>> that >>>>>>>>>>>> were not properly implemented in 6, contributing to the >>>>>>>>>>>> difference >>>>>>>>>>>> between 6 and 7 (which seems to be particularly visible for this >>>>>>>>>>>> testcase). I was looking at the checks a few times, trying to >>>>>>>>>>>> write >>>>>>>>>>>> them differently/faster while still performing the checks, but >>>>>>>>>>>> was >>>>>>>>>>>> not successful in that yet, unfortunately. >>>>>>>>>>>> >>>>>>>>>>> I see the issue now - it is reproducible with the following >>>>>>>>>>> memory >>>>>>>>>>> parameter (at least in my machine): >>>>>>>>>>> >>>>>>>>>>> -J-Xmx768M >>>>>>>>>>> >>>>>>>>>>> This give around 20sec in JDK 6 and 10+ minutes in JDK 8. >>>>>>>>>>> >>>>>>>>>>> All the time seems to be spent in desugaring, most specifically >>>>>>>>>>> in >>>>>>>>>>> TransTypes.addBridges - it seems like that method calls >>>>>>>>>>> Types.implementation a lot - so my theory was that the fact that >>>>>>>>>>> javac consumes more memory, forces the GC to get rid of the >>>>>>>>>>> cached >>>>>>>>>>> entries in the implementation/members closure caches (since such >>>>>>>>>>> entries are deliberately backed up by SoftReferences), which in >>>>>>>>>>> turn >>>>>>>>>>> completely trashes performances. I instrumented the code a bit >>>>>>>>>>> and >>>>>>>>>>> this is what I found: >>>>>>>>>>> >>>>>>>>>>> *) With -Xmx768M >>>>>>>>>>> >>>>>>>>>>> Impl cache misses = 3346926 >>>>>>>>>>> Members cache misses = 1042678 >>>>>>>>>>> >>>>>>>>>>> real 7m0.335s >>>>>>>>>>> user 25m51.517s >>>>>>>>>>> sys 0m4.947s >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> *) W/o -Xmx768M >>>>>>>>>>> >>>>>>>>>>> Impl cache misses = 3346839 >>>>>>>>>>> Members cache misses = 1042678 >>>>>>>>>>> >>>>>>>>>>> real 0m32.377s >>>>>>>>>>> user 1m25.881s >>>>>>>>>>> sys 0m2.232s >>>>>>>>>>> >>>>>>>>>>> Long story short - cache misses do not play a factor in here - >>>>>>>>>>> there >>>>>>>>>>> are some minor differences, but nothing out of the ordinary and >>>>>>>>>>> defo >>>>>>>>>>> nothing that would explain a multi-minute slowdown! Any ideas? >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> use flight recorder? >>>>>>>>>> >>>>>>>>>> Vicente >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Maurizio >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Jan >>>>>>>>>>>> >>>>>>>>>>>> garbage collection. >>>>>>>>>>>>> >>>>>>>>>>>>> 2015-04-29 15:12 GMT+02:00 Maurizio Cimadamore >>>>>>>>>>>>> >>>>>>>>>>>> >: >>>>>>>>>>>>> >>>>>>>>>>>>> On 29/04/15 12:44, Jacob Wieland wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hello Maurizio, >>>>>>>>>>>>>> >>>>>>>>>>>>>> are you sure that you used the 64bit versions of javac? I >>>>>>>>>>>>>> could >>>>>>>>>>>>>> only observe the behavior with these. >>>>>>>>>>>>>> >>>>>>>>>>>>> Yep I'm on a Ubuntu x64 machine. It's actually pretty >>>>>>>>>>>>> standard >>>>>>>>>>>>> hardware too - i.e. intel i5 (two cores, but OS sees 4 >>>>>>>>>>>>> because of >>>>>>>>>>>>> hyper-threading). >>>>>>>>>>>>> >>>>>>>>>>>>>> Also, I just tried with jdk8u31-64b and it takes AGES >>>>>>>>>>>>>> (still >>>>>>>>>>>>>> running after 17 minutes where the jdk6 was done after >>>>>>>>>>>>>> 2), top >>>>>>>>>>>>>> shows 4GB VIRT memory use and 350 % load (on a 4core >>>>>>>>>>>>>> processor). >>>>>>>>>>>>>> >>>>>>>>>>>>> Maybe the reproducer you sent was incorrect? >>>>>>>>>>>>> >>>>>>>>>>>>> Maurizio >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> So, I don't think it was that problem. >>>>>>>>>>>>>> >>>>>>>>>>>>>> 2015-04-29 12:00 GMT+02:00 Maurizio Cimadamore >>>>>>>>>>>>>> >>>>>>>>>>>>> >: >>>>>>>>>>>>>> >>>>>>>>>>>>>> These are the numbers I'm getting: >>>>>>>>>>>>>> >>>>>>>>>>>>>> JDK 9 (b42) >>>>>>>>>>>>>> >>>>>>>>>>>>>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or >>>>>>>>>>>>>> overrides a >>>>>>>>>>>>>> deprecated API. >>>>>>>>>>>>>> Note: Recompile with -Xlint:deprecation for details. >>>>>>>>>>>>>> >>>>>>>>>>>>>> real 0m46.306s >>>>>>>>>>>>>> user 2m17.489s >>>>>>>>>>>>>> sys 0m2.166s >>>>>>>>>>>>>> >>>>>>>>>>>>>> JDK 8 (GA) >>>>>>>>>>>>>> >>>>>>>>>>>>>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or >>>>>>>>>>>>>> overrides a >>>>>>>>>>>>>> deprecated API. >>>>>>>>>>>>>> Note: Recompile with -Xlint:deprecation for details. >>>>>>>>>>>>>> >>>>>>>>>>>>>> real 6m58.748s >>>>>>>>>>>>>> user 8m43.546s >>>>>>>>>>>>>> sys 0m2.132s >>>>>>>>>>>>>> >>>>>>>>>>>>>> JDK 7 (1.7.0_79) >>>>>>>>>>>>>> >>>>>>>>>>>>>> Note: generated_ttcn/TTCN3_CommonDefs.java uses or >>>>>>>>>>>>>> overrides a >>>>>>>>>>>>>> deprecated API. >>>>>>>>>>>>>> Note: Recompile with -Xlint:deprecation for details. >>>>>>>>>>>>>> >>>>>>>>>>>>>> real 0m28.341s >>>>>>>>>>>>>> user 1m17.194s >>>>>>>>>>>>>> sys 0m1.886s >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> As you can see there is a significant regression from >>>>>>>>>>>>>> JDK >>>>>>>>>>>>>> 7 to >>>>>>>>>>>>>> JDK 8 which was caused by >>>>>>>>>>>>>> >>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8043253 >>>>>>>>>>>>>> >>>>>>>>>>>>>> (some stack trace analysis revealed the familiar >>>>>>>>>>>>>> pattern). >>>>>>>>>>>>>> This has also been fixed in JDK 8u20 (as stated in >>>>>>>>>>>>>> the bug >>>>>>>>>>>>>> evaluation). >>>>>>>>>>>>>> >>>>>>>>>>>>>> So, while JDK 8u20/9 is slower than JDK 7 (at least >>>>>>>>>>>>>> on my >>>>>>>>>>>>>> machine), the numbers are more or less in the same >>>>>>>>>>>>>> ballpark >>>>>>>>>>>>>> and the huge regression that was visible in earlier >>>>>>>>>>>>>> JDK 8 >>>>>>>>>>>>>> releases has now been fixed. >>>>>>>>>>>>>> >>>>>>>>>>>>>> If you are still experiencing the problem - can you >>>>>>>>>>>>>> please >>>>>>>>>>>>>> also submit the specific compiler versions you are >>>>>>>>>>>>>> using in >>>>>>>>>>>>>> your benchmark? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Maurizio >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 29/04/15 10:29, Maurizio Cimadamore wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Jacob, >>>>>>>>>>>>>>> Stay assured - as we'll definitively look into this >>>>>>>>>>>>>>> issue (I >>>>>>>>>>>>>>> see it's already assigned to one of my colleagues). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> What I can say (w/o looking too much at the attached >>>>>>>>>>>>>>> artifacts) is that in general, javac has no issue >>>>>>>>>>>>>>> with >>>>>>>>>>>>>>> compiling a lot of sources at once; at one point the >>>>>>>>>>>>>>> build >>>>>>>>>>>>>>> system was structured in such a way that all the JDK >>>>>>>>>>>>>>> classes >>>>>>>>>>>>>>> were compiled at once - and that (which is way more >>>>>>>>>>>>>>> than >>>>>>>>>>>>>>> your >>>>>>>>>>>>>>> 187 sources - i.e. at least 10x that) took less than >>>>>>>>>>>>>>> 20 >>>>>>>>>>>>>>> seconds. SO there must some specific pattern >>>>>>>>>>>>>>> triggering that >>>>>>>>>>>>>>> issue. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Given that you say you have 187 input sources and >>>>>>>>>>>>>>> 48K output >>>>>>>>>>>>>>> classes, I'd say you are using inner classes a lot. >>>>>>>>>>>>>>> I wonder >>>>>>>>>>>>>>> if you are hitting this: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8000316 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Maurizio >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 24/04/15 09:49, Jacob Wieland wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hello Folks, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I still have the open problem >>>>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8039262 that the >>>>>>>>>>>>>>>> javac performance has degraded significantly from >>>>>>>>>>>>>>>> 1.6 to >>>>>>>>>>>>>>>> 1.7 >>>>>>>>>>>>>>>> (and even worse to 1.8) in the 64bit versions. >>>>>>>>>>>>>>>> Since in our >>>>>>>>>>>>>>>> context, we are dealing with a lot of generated >>>>>>>>>>>>>>>> source1.4 >>>>>>>>>>>>>>>> Java input (either split into very large files with >>>>>>>>>>>>>>>> inner >>>>>>>>>>>>>>>> classes or big packages with lots of smaller >>>>>>>>>>>>>>>> classes), >>>>>>>>>>>>>>>> compiler performance is critical for our tool and >>>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>> degradation forces us to continue recommending to >>>>>>>>>>>>>>>> our >>>>>>>>>>>>>>>> customers to use Java 1.6 for large projects (as is >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>> norm) as 1.7 and 1.8 are pretty much unusable in >>>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>> respect. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Is anyone still working on this issue or is such >>>>>>>>>>>>>>>> significant >>>>>>>>>>>>>>>> performance degradation not a serious issue? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> My observations so far are: >>>>>>>>>>>>>>>> - it gets worse the more class files are being >>>>>>>>>>>>>>>> compiled/the >>>>>>>>>>>>>>>> more files reside in the source path >>>>>>>>>>>>>>>> - cpu usage goes through the roof over all available >>>>>>>>>>>>>>>> kernels >>>>>>>>>>>>>>>> - memory usage is much higher >>>>>>>>>>>>>>>> - garbage collection seems to be much more active >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Using -proc:none alleviates the problem slightly >>>>>>>>>>>>>>>> for the >>>>>>>>>>>>>>>> 1.7 >>>>>>>>>>>>>>>> version, but not for 1.8 (last we tested with >>>>>>>>>>>>>>>> jdk1.8.0_31) where the performance difference is a >>>>>>>>>>>>>>>> factor 5 >>>>>>>>>>>>>>>> or more! >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I can understand that a more advanced compiler has >>>>>>>>>>>>>>>> capabilities that a previous version does not have >>>>>>>>>>>>>>>> and thus >>>>>>>>>>>>>>>> sometimes has >>>>>>>>>>>>>>>> to do more work. But, it should still be possible >>>>>>>>>>>>>>>> (especially if given the -source 1.4 or -source 1.5 >>>>>>>>>>>>>>>> option >>>>>>>>>>>>>>>> as we do) to optimize it in such a way that >>>>>>>>>>>>>>>> unnecessary >>>>>>>>>>>>>>>> checks for generics, overriding methods, closures, >>>>>>>>>>>>>>>> annotations and other newer features can be turned >>>>>>>>>>>>>>>> off (if >>>>>>>>>>>>>>>> they are to blame, which I actually doubt from my >>>>>>>>>>>>>>>> observations). >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I would really appreciate your help in this regard >>>>>>>>>>>>>>>> and I >>>>>>>>>>>>>>>> think everyone would benefit from any bugfix you >>>>>>>>>>>>>>>> can offer >>>>>>>>>>>>>>>> for this. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> BR, Jacob Wieland >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> ------------------------------ >>>>>>>>>>>>>>>> ------------------------------------------- >>>>>>>>>>>>>>>> Dr. Jacob Wieland >>>>>>>>>>>>>>>> Senior Software Engineer >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Testing Technologies IST GmbH >>>>>>>>>>>>>>>> Michaelkirchstra?e 17/18 >>>>>>>>>>>>>>>> 10179 Berlin, Germany >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Phone +49 30 726 19 19 34 >>>>>>>>>>>>>>>> <%2B49%2030%20726%2019%2019%2034> Email >>>>>>>>>>>>>>>> wieland at testingtech.com >>>>>>>>>>>>>>> stanca at testingtech.com> >>>>>>>>>>>>>>>> Fax +49 30 726 19 19 20 Internet >>>>>>>>>>>>>>>> www.testingtech.com >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> --------------------------------------------------------------------------------------------------------------- >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> UPCOMING EVENTS >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>>>>>>>>>>>>>>> Deadline: May 30, 2015 >>>>>>>>>>>>>>>> ucaat.etsi.org/2015/CallForPresentations.html >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Apr 21-23, 2015 | SAE Conference & Exhibition >>>>>>>>>>>>>>>> Detroit, Michigan, USA >>>>>>>>>>>>>>>> www.sae.org/congress/ >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Apr 28-30, 2015 | iqnite >>>>>>>>>>>>>>>> Dusseldorf, Germany >>>>>>>>>>>>>>>> www.iqnite-conferences.com/de/index.aspx >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Gesch?ftsf?hrung: Theofanis Vassiliou-Gioles, >>>>>>>>>>>>>>>> Stephan >>>>>>>>>>>>>>>> Pietsch, Pete Nicholson Handelsregister HRB 77805 B, >>>>>>>>>>>>>>>> Amtsgericht Charlottenburg Ust ID Nr.: DE 813 143 >>>>>>>>>>>>>>>> 070 This >>>>>>>>>>>>>>>> email may contain confidential and privileged >>>>>>>>>>>>>>>> material for >>>>>>>>>>>>>>>> the sole use of the intended recipient. Any review, >>>>>>>>>>>>>>>> use, >>>>>>>>>>>>>>>> distribution or disclosure by others is strictly >>>>>>>>>>>>>>>> prohibited. >>>>>>>>>>>>>>>> If you are not the intended recipient (or >>>>>>>>>>>>>>>> authorized to >>>>>>>>>>>>>>>> receive for the recipient), please contact the >>>>>>>>>>>>>>>> sender by >>>>>>>>>>>>>>>> reply email and delete all copies of this message. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> ------------------------------ >>>>>>>>>>>>>> ------------------------------------------- >>>>>>>>>>>>>> Dr. Jacob Wieland >>>>>>>>>>>>>> Senior Software Engineer >>>>>>>>>>>>>> >>>>>>>>>>>>>> Testing Technologies IST GmbH >>>>>>>>>>>>>> Michaelkirchstra?e 17/18 >>>>>>>>>>>>>> 10179 Berlin, Germany >>>>>>>>>>>>>> >>>>>>>>>>>>>> Phone +49 30 726 19 19 34 Email >>>>>>>>>>>>>> wieland at testingtech.com >>>>>>>>>>>>>> >>>>>>>>>>>>>> Fax +49 30 726 19 19 20 <%2B49%2030%20726%2019%2019%2020> >>>>>>>>>>>>>> Internet www.testingtech.com >>>>>>>>>>>>>> >>>>>>>>>>>>>> --------------------------------------------------------------------------------------------------------------- >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> UPCOMING EVENTS >>>>>>>>>>>>>> >>>>>>>>>>>>>> SUBMIT YOUR TOPIC for the UCAAT 2015 >>>>>>>>>>>>>> Deadline: May 30, 2015 >>>>>>>>>>>>>> ucaat.etsi.org/2015/CallForPresentations.html >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Apr 21-23, 2015 | SAE Conference & Exhibition >>>>>>>>>>>>>> Detroit, Michigan, USA >>>>>>>>>>>>>> www.sae.org/congress/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> Apr 28-30, 2015 | iqnite >>>>>>>>>>>>>> Dusseldorf, Germany >>>>>>>>>>>>>> www.iqnite-conferences.com/de/index.aspx >>>>>>>>>>>>>> >>>>>>>>>>>>>> ----------------------------------------------------------------------------------------------------------------- >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> ... >> >> [Nachricht gek?rzt] > > > > > -- > -- > ------------------------------ > ------------------------------------------- > Dr. Jacob Wieland > Senior Software Engineer > > Testing Technologies IST GmbH > Michaelkirchstra?e 17/18 > 10179 Berlin, Germany > > Phone +49 30 726 19 19 34 Email wieland at testingtech.com > > Fax +49 30 726 19 19 20 Internet www.testingtech.com > > > --------------------------------------------------------------------------------------------------------------------- > > UPCOMING EVENTS > > Jul 15-16, 2015 | German Testing Day > Frankfurt, Germany | Booth #8 > www.germantestingday.info/ > > Sep 15-17, 2015 | Accredited TTCN-3 Training "Test Automation with > TTCN-3" > Berlin, Germany > www.testingtech.com/services/ttcn3_training.php > > Oct 5-9, 2015 | 22nd ITS World Congress > Bordeaux, France | At Spirent's Booth #122 > www.itsineurope.com/bordeaux-2015/ > > > ----------------------------------------------------------------------------------------------------------------- > > Testing Technologies IST GmbH > Michaelkirchstra?e 17/18, 10179 Berlin, Germany > Office: +49 (30) 726 19 19 0 > Fax: +49 (30) 726 19 19 20 > Internet www.testingtech.com > ------------------------------------------------------------ > ----------------------------------------------------- > > Executive Board: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete > Nicholson > Handelsregister HRB 77805 B, Amtsgericht Charlottenburg > Ust ID Nr.: DE 813 143 070 > > This email may contain confidential and privileged material for the sole > use of the intended recipient. Any review, use, distribution or disclosure > by others is strictly prohibited. If you are not the intended recipient (or > authorized to receive for the recipient), please contact the sender by > reply email and delete all copies of this message. > > > > -- > -- > ------------------------------ > ------------------------------------------- > Dr. Jacob Wieland > Senior Software Engineer > > Testing Technologies IST GmbH > Michaelkirchstra?e 17/18 > 10179 Berlin, Germany > > Phone +49 30 726 19 19 34 Email wieland at testingtech.com > > Fax +49 30 726 19 19 20 Internet www.testingtech.com > > > --------------------------------------------------------------------------------------------------------------------- > > UPCOMING EVENTS > > Jul 15-16, 2015 | German Testing Day > Frankfurt, Germany | Booth #8 > www.germantestingday.info/ > > Sep 15-17, 2015 | Accredited TTCN-3 Training "Test Automation with > TTCN-3" > Berlin, Germany > www.testingtech.com/services/ttcn3_training.php > > Oct 5-9, 2015 | 22nd ITS World Congress > Bordeaux, France | At Spirent's Booth #122 > www.itsineurope.com/bordeaux-2015/ > > > ----------------------------------------------------------------------------------------------------------------- > > Testing Technologies IST GmbH > Michaelkirchstra?e 17/18, 10179 Berlin, Germany > Office: +49 (30) 726 19 19 0 > Fax: +49 (30) 726 19 19 20 > Internet www.testingtech.com > ------------------------------------------------------------ > ----------------------------------------------------- > > Executive Board: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete > Nicholson > Handelsregister HRB 77805 B, Amtsgericht Charlottenburg > Ust ID Nr.: DE 813 143 070 > > This email may contain confidential and privileged material for the sole > use of the intended recipient. Any review, use, distribution or disclosure > by others is strictly prohibited. If you are not the intended recipient (or > authorized to receive for the recipient), please contact the sender by > reply email and delete all copies of this message. > > > -- -- ------------------------------ ------------------------------------------- Dr. Jacob Wieland Senior Software Engineer Testing Technologies IST GmbH Michaelkirchstra?e 17/18 10179 Berlin, Germany Phone +49 30 726 19 19 34 Email wieland at testingtech.com Fax +49 30 726 19 19 20 Internet www.testingtech.com --------------------------------------------------------------------------------------------------------------------- UPCOMING EVENTS Jul 15-16, 2015 | German Testing Day Frankfurt, Germany | Booth #8 www.germantestingday.info/ Sep 15-17, 2015 | Accredited TTCN-3 Training "Test Automation with TTCN-3" Berlin, Germany www.testingtech.com/services/ttcn3_training.php Oct 5-9, 2015 | 22nd ITS World Congress Bordeaux, France | At Spirent's Booth #122 www.itsineurope.com/bordeaux-2015/ ----------------------------------------------------------------------------------------------------------------- Testing Technologies IST GmbH Michaelkirchstra?e 17/18, 10179 Berlin, Germany Office: +49 (30) 726 19 19 0 Fax: +49 (30) 726 19 19 20 Internet www.testingtech.com ------------------------------------------------------------ ----------------------------------------------------- Executive Board: Theofanis Vassiliou-Gioles, Stephan Pietsch, Pete Nicholson Handelsregister HRB 77805 B, Amtsgericht Charlottenburg Ust ID Nr.: DE 813 143 070 This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Victor.Rudometov at oracle.com Thu Jul 2 16:05:26 2015 From: Victor.Rudometov at oracle.com (Victor Rudometov) Date: Thu, 02 Jul 2015 19:05:26 +0300 Subject: annotation processor problems since 9b70 Message-ID: <55956146.8010807@oracle.com> Hello. Recent changes in jdk9b70 to annotation processor (probably https://jbs.oracle.com/bugs/browse/JDK-8074346) made impossible to process the code with types that are not declared. F.e.: public class ThrowsException { public void method() throws @Annot SomeException {} } Is not processed because of Annot and SomeException. I did not have this problem with b69 - errors are reported, however the annotation processing is performed. Was it done intentionally or this is a regression? Are there any workarounds? Below is a simple test code: import javax.annotation.processing.AbstractProcessor; importjavax.annotation.processing.RoundEnvironment; importjavax.annotation.processing.SupportedAnnotationTypes; importjavax.annotation.processing.SupportedSourceVersion; importjavax.lang.model.element.TypeElement; importjavax.tools.*; importjava.io.CharArrayWriter; importjava.io.File; importjava.io.IOException; importjava.net.URISyntaxException; importjava.net.URL; importjava.util.*; import staticjavax.lang.model.SourceVersion.RELEASE_9; public classTest { privateListsources; public static voidmain(String args[])throwsURISyntaxException { newTest().run(); } public voidrun() { createSources().addFileToSources("c:\\data\\ThrowsException.java"); DiagnosticCollector diagnostics =newDiagnosticCollector<>();; JavaCompiler javac = ToolProvider.getSystemJavaCompiler(); StandardJavaFileManager fileManager = javac.getStandardFileManager(null,null,null); Iterable compilationUnits = fileManager.getJavaFileObjectsFromFiles(sources); CharArrayWriter errorWriter =newCharArrayWriter(); List compilerOptions = Arrays.asList( "-proc:only -Xmaxerrs 0 -Xmaxwarns 0".split("\\s+")); JavaCompiler.CompilationTask task = javac.getTask(errorWriter, fileManager, diagnostics, compilerOptions,null, compilationUnits); MyProcessor processor =newMyProcessor(); task.setProcessors(Arrays.asList(processor)); task.call(); System.out.println(processor.wasInvoked()); for(Diagnostic diagnostic : diagnostics.getDiagnostics()) System.out.format("Error on line %d in %s%n : %s", diagnostic.getLineNumber(), diagnostic.getSource().toString(), diagnostic.getMessage(Locale.US)); try{ fileManager.close(); }catch(IOException e) { e.printStackTrace(); } } @SupportedAnnotationTypes({"*"}) @SupportedSourceVersion(RELEASE_9) public static classMyProcessorextendsAbstractProcessor { private booleaninvoked=false; @Override public booleanprocess(Set annotations, RoundEnvironment roundEnv) { invoked=true; return false; } public booleanwasInvoked() { returninvoked; } } privateTest createSources() { sources=newArrayList<>(); return this; } privateTest addFileToSources(String path) { sources.add(newFile(path)); return this; } } Thanks. Victor. -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.smith at oracle.com Thu Jul 2 22:10:55 2015 From: daniel.smith at oracle.com (Dan Smith) Date: Thu, 2 Jul 2015 16:10:55 -0600 Subject: Anonymous classes are not final according to reflection API In-Reply-To: <55914ED7.9070306@oracle.com> References: <558826EC.6060000@oracle.com> <55882B47.8060001@univ-mlv.fr> <5588304B.7070709@oracle.com> <55896261.9040505@oracle.com> <5591482F.4010106@oracle.com> <559149D4.4020406@univ-mlv.fr> <55914CFE.6090103@oracle.com> <55914ED7.9070306@oracle.com> Message-ID: <581D3527-6AA4-4E8B-AE70-719A718EB5C2@oracle.com> > On Jun 29, 2015, at 7:57 AM, Maurizio Cimadamore wrote: > > I think this should be fixed so that anon classes (regardless of their context) always have their ACC_FINAL bit unset; this is probably not very hot from a compatibility perspective (given that it's been like that for a long time) - but, given the consistency issue and the fact that reflection cannot handle this anyway, I'd say we should just make this consistent. If we're making a change to which flags are set, we might as well do it properly and make them all final, no? One way or another, you're going to change some serialization UIDs. Really, javac is free to implement anonymous classes however it likes (with a few small constraints from JLS, notably in Chapter 13). If it wants to spell all the synthetic field names backwards on Tuesdays, that's it's business. Anybody relying on certain conventions for serialization between recompiled anonymous classes is playing a risky game, and they should not be surprised if things break from time to time.* (Some smaller examples that would also change some UIDs: different bridge generation strategies; different methods produced by lambda/method reference compilation; refining the implementation of asserts; improvements in diamond inference.) That said, I get the practical constraint of not rocking the boat if there's not a compelling need to do so. And it's notable that this would impact a lot more UIDs than the typical bug fix. However: - The JLS assertion "An anonymous class is always implicitly final" is meaningless if it's not interpreted as "the ACC_FINAL flag on the generated class is set", so if we're not going to enforce that, the assertion should be dropped. (The other assertions in 15.9.5 have language-level implications, but not this, since none of the corresponding language restrictions in 8.1.1.2 could ever arise.) - I don't know why reflection is colluding with javac here. Dynamically changing flags seems to pretty directly violate the Class.getModifiers spec. Does the computation of serialVersionUID rely on reflection? If so, doesn't this flag-modification logic belong there, rather than breaking reflection for everyone? If not, who's relying on the reflection behavior? ?Dan [* It's "strongly recommended", in italics no less, not to rely on this. From java.lang.Serializable: "it is _strongly recommended_ that all serializable classes explicitly declare serialVersionUID values, since the default serialVersionUID computation is highly sensitive to class details that may vary depending on compiler implementations, and can thus result in unexpected InvalidClassExceptions during deserialization"] From forax at univ-mlv.fr Thu Jul 2 22:25:29 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 3 Jul 2015 00:25:29 +0200 Subject: Anonymous classes are not final according to reflection API In-Reply-To: <581D3527-6AA4-4E8B-AE70-719A718EB5C2@oracle.com> References: <558826EC.6060000@oracle.com> <55882B47.8060001@univ-mlv.fr> <5588304B.7070709@oracle.com> <55896261.9040505@oracle.com> <5591482F.4010106@oracle.com> <559149D4.4020406@univ-mlv.fr> <55914CFE.6090103@oracle.com> <55914ED7.9070306@oracle.com> <581D3527-6AA4-4E8B-AE70-719A718EB5C2@oracle.com> Message-ID: <5595BA59.4030207@univ-mlv.fr> Hi Dan, You can not set a serialVersionUID of an anonymous class because a serialVersionUID is a static final field which you can not declare inside an anonymous class. That's said the name of an anonymous class is not stable too. and to answer your question about who is using getModifiers() on a class, a lot of tools that generate proxies at runtime (Hibernates, Weld etc) because if the class is not final, you can generate a class that inherits from it. R?mi On 07/03/2015 12:10 AM, Dan Smith wrote: >> On Jun 29, 2015, at 7:57 AM, Maurizio Cimadamore wrote: >> >> I think this should be fixed so that anon classes (regardless of their context) always have their ACC_FINAL bit unset; this is probably not very hot from a compatibility perspective (given that it's been like that for a long time) - but, given the consistency issue and the fact that reflection cannot handle this anyway, I'd say we should just make this consistent. > If we're making a change to which flags are set, we might as well do it properly and make them all final, no? One way or another, you're going to change some serialization UIDs. > > Really, javac is free to implement anonymous classes however it likes (with a few small constraints from JLS, notably in Chapter 13). If it wants to spell all the synthetic field names backwards on Tuesdays, that's it's business. Anybody relying on certain conventions for serialization between recompiled anonymous classes is playing a risky game, and they should not be surprised if things break from time to time.* (Some smaller examples that would also change some UIDs: different bridge generation strategies; different methods produced by lambda/method reference compilation; refining the implementation of asserts; improvements in diamond inference.) > > That said, I get the practical constraint of not rocking the boat if there's not a compelling need to do so. And it's notable that this would impact a lot more UIDs than the typical bug fix. > > However: > > - The JLS assertion "An anonymous class is always implicitly final" is meaningless if it's not interpreted as "the ACC_FINAL flag on the generated class is set", so if we're not going to enforce that, the assertion should be dropped. (The other assertions in 15.9.5 have language-level implications, but not this, since none of the corresponding language restrictions in 8.1.1.2 could ever arise.) > > - I don't know why reflection is colluding with javac here. Dynamically changing flags seems to pretty directly violate the Class.getModifiers spec. Does the computation of serialVersionUID rely on reflection? If so, doesn't this flag-modification logic belong there, rather than breaking reflection for everyone? If not, who's relying on the reflection behavior? > > ?Dan > > > [* It's "strongly recommended", in italics no less, not to rely on this. From java.lang.Serializable: "it is _strongly recommended_ that all serializable classes explicitly declare serialVersionUID values, since the default serialVersionUID computation is highly sensitive to class details that may vary depending on compiler implementations, and can thus result in unexpected InvalidClassExceptions during deserialization"] From daniel.smith at oracle.com Thu Jul 2 23:21:17 2015 From: daniel.smith at oracle.com (Dan Smith) Date: Thu, 2 Jul 2015 17:21:17 -0600 Subject: Anonymous classes are not final according to reflection API In-Reply-To: <5595BA59.4030207@univ-mlv.fr> References: <558826EC.6060000@oracle.com> <55882B47.8060001@univ-mlv.fr> <5588304B.7070709@oracle.com> <55896261.9040505@oracle.com> <5591482F.4010106@oracle.com> <559149D4.4020406@univ-mlv.fr> <55914CFE.6090103@oracle.com> <55914ED7.9070306@oracle.com> <581D3527-6AA4-4E8B-AE70-719A718EB5C2@oracle.com> <5595BA59.4030207@univ-mlv.fr> Message-ID: > On Jul 2, 2015, at 4:25 PM, Remi Forax wrote: > > and to answer your question about who is using getModifiers() on a class, > a lot of tools that generate proxies at runtime (Hibernates, Weld etc) because if the class is not final, > you can generate a class that inherits from it. Looking closer at Peter's and Maurizio's code snippings, I guess it's clear that the UID computation uses reflection too. (I also see that the strange combination of javac and reflection logic leads the serialization code to always get "false" for ACC_FINAL, which is why Maurizio argued that javac turning off the flag consistently is a more attractive course of action than turning it on all the time.) The reflection behavior is convenient for serialization, which wants to be lied to. But why would Hibernate want to be told that a class was non-final, only to find out that, oops, the VM won't let me extend it? Am I misunderstanding? More generally, if reflection provides accurate results for getModifers(), the client can always turn off ACC_FINAL if it wants to, based on some calls to isInner and getName. But if reflection provides incorrect results for getModifiers(), there's no way for a client to undo that and get at the original flags. ?Dan From srikanth.adayapalam at oracle.com Fri Jul 3 04:41:13 2015 From: srikanth.adayapalam at oracle.com (Srikanth) Date: Fri, 03 Jul 2015 10:11:13 +0530 Subject: annotation processor problems since 9b70 In-Reply-To: <55956146.8010807@oracle.com> References: <55956146.8010807@oracle.com> Message-ID: <55961269.6050704@oracle.com> On Thursday 02 July 2015 09:35 PM, Victor Rudometov wrote: > Hello. > Recent changes in jdk9b70 to annotation processor (probably > https://jbs.oracle.com/bugs/browse/JDK-8074346) made impossible to > process the code with types that are not declared. Hello Victor, Thanks for the report - I am looking into this - if found necessary I'll follow up by raising a JBS ticket. Regards, Srikanth > > F.e.: > > public class ThrowsException { > public void method() throws @Annot SomeException {} > } > > Is not processed because of Annot and SomeException. > > I did not have this problem with b69 - errors are reported, however > the annotation processing is performed. > > Was it done intentionally or this is a regression? > Are there any workarounds? > > Below is a simple test code: > > import javax.annotation.processing.AbstractProcessor; > importjavax.annotation.processing.RoundEnvironment; > importjavax.annotation.processing.SupportedAnnotationTypes; > importjavax.annotation.processing.SupportedSourceVersion; > importjavax.lang.model.element.TypeElement; > importjavax.tools.*; > importjava.io.CharArrayWriter; > importjava.io.File; > importjava.io.IOException; > importjava.net.URISyntaxException; > importjava.net.URL; > importjava.util.*; > > import staticjavax.lang.model.SourceVersion.RELEASE_9; > > public classTest { > privateListsources; > > public static voidmain(String args[])throwsURISyntaxException { > newTest().run(); > } > > public voidrun() { > createSources().addFileToSources("c:\\data\\ThrowsException.java"); > > DiagnosticCollector diagnostics =newDiagnosticCollector<>();; > JavaCompiler javac = ToolProvider.getSystemJavaCompiler(); > StandardJavaFileManager fileManager = > javac.getStandardFileManager(null,null,null); > Iterable compilationUnits = > fileManager.getJavaFileObjectsFromFiles(sources); > CharArrayWriter errorWriter =newCharArrayWriter(); > List compilerOptions = Arrays.asList( > "-proc:only -Xmaxerrs 0 -Xmaxwarns 0".split("\\s+")); > JavaCompiler.CompilationTask task = javac.getTask(errorWriter, > fileManager, diagnostics, compilerOptions,null, compilationUnits); > > MyProcessor processor =newMyProcessor(); > task.setProcessors(Arrays.asList(processor)); > > task.call(); > > System.out.println(processor.wasInvoked()); > > for(Diagnostic diagnostic : diagnostics.getDiagnostics()) > System.out.format("Error on line %d in %s%n : %s", > diagnostic.getLineNumber(), > diagnostic.getSource().toString(), > diagnostic.getMessage(Locale.US)); > > try{ > fileManager.close(); > }catch(IOException e) { > e.printStackTrace(); > } > } > > @SupportedAnnotationTypes({"*"}) > @SupportedSourceVersion(RELEASE_9) > public static classMyProcessorextendsAbstractProcessor { > > private booleaninvoked=false; > > @Override > public booleanprocess(Set annotations, RoundEnvironment roundEnv) { > invoked=true; > return false; > } > > public booleanwasInvoked() { > returninvoked; > } > } > > privateTest createSources() { > sources=newArrayList<>(); > return this; > } > > privateTest addFileToSources(String path) { > sources.add(newFile(path)); > return this; > } > } > > > Thanks. > Victor. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From srikanth.adayapalam at oracle.com Fri Jul 3 09:53:38 2015 From: srikanth.adayapalam at oracle.com (Srikanth) Date: Fri, 03 Jul 2015 15:23:38 +0530 Subject: annotation processor problems since 9b70 In-Reply-To: <55961269.6050704@oracle.com> References: <55956146.8010807@oracle.com> <55961269.6050704@oracle.com> Message-ID: <55965BA2.6060106@oracle.com> On Friday 03 July 2015 10:11 AM, Srikanth wrote: > > > On Thursday 02 July 2015 09:35 PM, Victor Rudometov wrote: >> Hello. >> Recent changes in jdk9b70 to annotation processor (probably >> https://jbs.oracle.com/bugs/browse/JDK-8074346) made impossible to >> process the code with types that are not declared. > > Hello Victor, > > Thanks for the report - I am looking into this - if found necessary > I'll follow up by raising a JBS ticket. https://bugs.openjdk.java.net/browse/JDK-8130401 - Thanks again for catching this. Srikanth -------------- next part -------------- An HTML attachment was scrubbed... URL: From maurizio.cimadamore at oracle.com Fri Jul 3 14:17:34 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 03 Jul 2015 15:17:34 +0100 Subject: Anonymous classes are not final according to reflection API In-Reply-To: <581D3527-6AA4-4E8B-AE70-719A718EB5C2@oracle.com> References: <558826EC.6060000@oracle.com> <55882B47.8060001@univ-mlv.fr> <5588304B.7070709@oracle.com> <55896261.9040505@oracle.com> <5591482F.4010106@oracle.com> <559149D4.4020406@univ-mlv.fr> <55914CFE.6090103@oracle.com> <55914ED7.9070306@oracle.com> <581D3527-6AA4-4E8B-AE70-719A718EB5C2@oracle.com> Message-ID: <5596997E.1030005@oracle.com> On 02/07/15 23:10, Dan Smith wrote: > - The JLS assertion "An anonymous class is always implicitly final" is meaningless if it's not interpreted as "the ACC_FINAL flag on the generated class is set", so if we're not going to enforce that, the assertion should be dropped. (The other assertions in 15.9.5 have language-level implications, but not this, since none of the corresponding language restrictions in 8.1.1.2 could ever arise.) unless the VM, javac, and JDK (reflection) agree on a standard form in which anonymous classes are encoded. It seems like reflection has an 'isAnonymousClass' method, whose implementation does the following: public booleanisAnonymousClass() { return"".equals(getSimpleName()); } So we already have a way to establish if a class is anonymous in the runtime. Which means we also have a way to infer correct finality without touching ACC_FINAL. Maurizio -------------- next part -------------- An HTML attachment was scrubbed... URL: From georgiy.rakov at oracle.com Mon Jul 6 17:13:12 2015 From: georgiy.rakov at oracle.com (Georgiy Rakov) Date: Mon, 6 Jul 2015 20:13:12 +0300 Subject: Please reply to my JDK-8059640 issue comment Message-ID: <559AB728.80105@oracle.com> Hello, could you please reply to my following comment I made to JDK-8059640; quoting it: Suggested spec change states that 15.13.3 will contain following assertion: LinkageError may occur due to inconsistencies between compile time and run time (in the referenced method, the functional interface, or the argument types of either). The details are provided in parenthesis: "in the referenced method, the functional interface, or the argument types of either". It seems that these details don't extend to following case specified in jls-15.12.4.3-120: If the invocation mode is interface, then the implementation must also check that the target reference type still implements the specified interface. If the target reference type does not still implement the interface, then an IncompatibleClassChangeError occurs. I believe they should. Namely, could you please tell if you agree that spec change suggested in the issue description doesn't extend to the case specified in jls-15.12.4.3-120. If you agree I would appreciate greatly if you could modify the suggested spec change so that it extends to jls-15.12.4.3-120 case. Thank you, Georgiy. -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.r.rose at oracle.com Tue Jul 7 03:42:04 2015 From: john.r.rose at oracle.com (John Rose) Date: Mon, 6 Jul 2015 20:42:04 -0700 Subject: Anonymous classes are not final according to reflection API In-Reply-To: <5596997E.1030005@oracle.com> References: <558826EC.6060000@oracle.com> <55882B47.8060001@univ-mlv.fr> <5588304B.7070709@oracle.com> <55896261.9040505@oracle.com> <5591482F.4010106@oracle.com> <559149D4.4020406@univ-mlv.fr> <55914CFE.6090103@oracle.com> <55914ED7.9070306@oracle.com> <581D3527-6AA4-4E8B-AE70-719A718EB5C2@oracle.com> <5596997E.1030005@oracle.com> Message-ID: <57CE6098-345E-4E0F-9D6E-8FEA4EBEFE53@oracle.com> But that's a hack too. It breaks if the name computes to a null string. As opposed to the documented null CP ref. ? John > On Jul 3, 2015, at 7:17 AM, Maurizio Cimadamore wrote: > > > >> On 02/07/15 23:10, Dan Smith wrote: >> - The JLS assertion "An anonymous class is always implicitly final" is meaningless if it's not interpreted as "the ACC_FINAL flag on the generated class is set", so if we're not going to enforce that, the assertion should be dropped. (The other assertions in 15.9.5 have language-level implications, but not this, since none of the corresponding language restrictions in 8.1.1.2 could ever arise.) > unless the VM, javac, and JDK (reflection) agree on a standard form in which anonymous classes are encoded. It seems like reflection has an 'isAnonymousClass' method, whose implementation does the following: > > public boolean isAnonymousClass() { > return "".equals(getSimpleName()); > } > > So we already have a way to establish if a class is anonymous in the runtime. Which means we also have a way to infer correct finality without touching ACC_FINAL. > > Maurizio -------------- next part -------------- An HTML attachment was scrubbed... URL: From cushon at google.com Tue Jul 7 21:34:46 2015 From: cushon at google.com (Liam Miller-Cushon) Date: Tue, 7 Jul 2015 14:34:46 -0700 Subject: Change in type inference behaviour with JDK-8078024 Message-ID: The fix for JDK-8078024 resolved an issue with type inference I was seeing with javac9. I can't tell from the bug report if that was deliberate. Could someone please confirm that the current behaviour is correct? Here's the repro. It fails with jdk9-dev @2888, and compiles cleanly @2889: === A.java === import java.util.Arrays; import java.util.List; class A { public static List> cartesianProduct(List... lists) { return cartesianProduct(Arrays.asList(lists)); } public static List> cartesianProduct(List> lists) { throw new AssertionError(); } } === $ javac A.java A.java:6: error: incompatible types: inference variable B has incompatible bounds return cartesianProduct(Arrays.asList(lists)); ^ equality constraints: B lower bounds: T,List where B,T are type-variables: B extends Object declared in method cartesianProduct(List...) T extends Object declared in method asList(T...) Note: /usr/local/google/home/cushon/A.java uses unchecked or unsafe operations. Note: Recompile with -Xlint:unchecked for details. 1 error -------------- next part -------------- An HTML attachment was scrubbed... URL: From maurizio.cimadamore at oracle.com Tue Jul 7 23:58:13 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 08 Jul 2015 00:58:13 +0100 Subject: Change in type inference behaviour with JDK-8078024 In-Reply-To: References: Message-ID: <559C6795.8000503@oracle.com> My understanding is that 8078024 should have been mostly a cleanup with performance-related issues (i.e. less time to converge to a non-existent solution). But Vicente did the work and might know few things more about this one, so let's hear from him. Maurizio On 07/07/15 22:34, Liam Miller-Cushon wrote: > The fix for JDK-8078024 resolved an issue with type inference I was > seeing with javac9. I can't tell from the bug report if that was > deliberate. > > Could someone please confirm that the current behaviour is correct? > > Here's the repro. It fails with jdk9-dev @2888, and compiles cleanly > @2889: > > === A.java === > import java.util.Arrays; > import java.util.List; > > class A { > public static List> cartesianProduct(List B>... lists) { > return cartesianProduct(Arrays.asList(lists)); > } > > public static List> cartesianProduct(List List> lists) { > throw new AssertionError(); > } > } > === > > $ javac A.java > A.java:6: error: incompatible types: inference variable B has > incompatible bounds > return cartesianProduct(Arrays.asList(lists)); > ^ > equality constraints: B > lower bounds: T,List > where B,T are type-variables: > B extends Object declared in method cartesianProduct(List extends B>...) > T extends Object declared in method asList(T...) > Note: /usr/local/google/home/cushon/A.java uses unchecked or unsafe > operations. > Note: Recompile with -Xlint:unchecked for details. > 1 error From vicente.romero at oracle.com Wed Jul 8 02:02:40 2015 From: vicente.romero at oracle.com (Vicente-Arturo Romero-Zaldivar) Date: Tue, 07 Jul 2015 19:02:40 -0700 Subject: Change in type inference behaviour with JDK-8078024 In-Reply-To: <559C6795.8000503@oracle.com> References: <559C6795.8000503@oracle.com> Message-ID: <559C84C0.3050903@oracle.com> Hi, I have modified the test case to: import java.util.Arrays; import java.util.List; class K { public static List> cartesianProduct(List... lists) { return cartesianProduct(Arrays.asList(lists)); } public static List> cartesianProduct(List> lists) { throw new AssertionError(); } } In order to have different names for each type variable, now A and B. This corner case shows a semantic change of the patch for JDK-8078024. While determining the applicability of method: List> cartesianProduct(List> lists) For which we have the constraints: B <: Object T <: List T<: Object List <: T First, variable B is selected for instantiation: B = CAP1 of ? extends A so this implies that: T <: List T<: Object List <: T OK now all the bounds are checked for consistency. While checking if List is a subtype of List a bound error is reported. Before the compiler was just swallowing it. As now the error is reported while variable B is being instantiated, the bound set is rolled back to it's initial state, B is instantiated to Object, and with this instantiation the constraint set is solvable, the method is applicable, it's the only applicable one and the code is accepted as correct. Thanks, Vicente On 07/07/2015 04:58 PM, Maurizio Cimadamore wrote: > My understanding is that 8078024 should have been mostly a cleanup > with performance-related issues (i.e. less time to converge to a > non-existent solution). But Vicente did the work and might know few > things more about this one, so let's hear from him. > > Maurizio > > On 07/07/15 22:34, Liam Miller-Cushon wrote: >> The fix for JDK-8078024 resolved an issue with type inference I was >> seeing with javac9. I can't tell from the bug report if that was >> deliberate. >> >> Could someone please confirm that the current behaviour is correct? >> >> Here's the repro. It fails with jdk9-dev @2888, and compiles cleanly >> @2889: >> >> === A.java === >> import java.util.Arrays; >> import java.util.List; >> >> class A { >> public static List> cartesianProduct(List> B>... lists) { >> return cartesianProduct(Arrays.asList(lists)); >> } >> >> public static List> cartesianProduct(List> List> lists) { >> throw new AssertionError(); >> } >> } >> === >> >> $ javac A.java >> A.java:6: error: incompatible types: inference variable B has >> incompatible bounds >> return cartesianProduct(Arrays.asList(lists)); >> ^ >> equality constraints: B >> lower bounds: T,List >> where B,T are type-variables: >> B extends Object declared in method cartesianProduct(List> extends B>...) >> T extends Object declared in method asList(T...) >> Note: /usr/local/google/home/cushon/A.java uses unchecked or unsafe >> operations. >> Note: Recompile with -Xlint:unchecked for details. >> 1 error > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cushon at google.com Wed Jul 8 20:16:29 2015 From: cushon at google.com (Liam Miller-Cushon) Date: Wed, 8 Jul 2015 13:16:29 -0700 Subject: Change in type inference behaviour with JDK-8078024 In-Reply-To: <559C84C0.3050903@oracle.com> References: <559C6795.8000503@oracle.com> <559C84C0.3050903@oracle.com> Message-ID: Thanks for investigating, and for the detailed explanation. I'm glad to hear that the new behaviour is valid. Is it worth adding a regression test for this? On Tue, Jul 7, 2015 at 7:02 PM, Vicente-Arturo Romero-Zaldivar < vicente.romero at oracle.com> wrote: > Hi, > > I have modified the test case to: > > import java.util.Arrays; > import java.util.List; > > class K { > public static List> cartesianProduct(List... > lists) { > return cartesianProduct(Arrays.asList(lists)); > } > > public static List> cartesianProduct(List extends B>> lists) { > throw new AssertionError(); > } > } > > In order to have different names for each type variable, now A and B. > > This corner case shows a semantic change of the patch for JDK-8078024. > While determining the applicability of method: > > List> cartesianProduct(List> lists) > > For which we have the constraints: > > B <: Object > T <: List > T<: Object > List <: T > > First, variable B is selected for instantiation: > > B = CAP1 of ? extends A > so this implies that: > > T <: List > T<: Object > List <: T > > OK now all the bounds are checked for consistency. While checking if > List is a subtype of List a > bound error is reported. Before the compiler was just swallowing it. As now > the error is reported while variable B is being instantiated, the bound set > is rolled back to it's initial state, B is instantiated to Object, and with > this instantiation the constraint set is solvable, the method is > applicable, it's the only applicable one and the code is accepted as > correct. > > > Thanks, > Vicente > > > > > On 07/07/2015 04:58 PM, Maurizio Cimadamore wrote: > > My understanding is that 8078024 should have been mostly a cleanup with > performance-related issues (i.e. less time to converge to a non-existent > solution). But Vicente did the work and might know few things more about > this one, so let's hear from him. > > Maurizio > > On 07/07/15 22:34, Liam Miller-Cushon wrote: > > The fix for JDK-8078024 resolved an issue with type inference I was seeing > with javac9. I can't tell from the bug report if that was deliberate. > > Could someone please confirm that the current behaviour is correct? > > Here's the repro. It fails with jdk9-dev @2888, and compiles cleanly > @2889: > > === A.java === > import java.util.Arrays; > import java.util.List; > > class A { > public static List> cartesianProduct(List... > lists) { > return cartesianProduct(Arrays.asList(lists)); > } > > public static List> cartesianProduct(List extends B>> lists) { > throw new AssertionError(); > } > } > === > > $ javac A.java > A.java:6: error: incompatible types: inference variable B has incompatible > bounds > return cartesianProduct(Arrays.asList(lists)); > ^ > equality constraints: B > lower bounds: T,List > where B,T are type-variables: > B extends Object declared in method cartesianProduct(List B>...) > T extends Object declared in method asList(T...) > Note: /usr/local/google/home/cushon/A.java uses unchecked or unsafe > operations. > Note: Recompile with -Xlint:unchecked for details. > 1 error > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vicente.romero at oracle.com Wed Jul 8 20:39:17 2015 From: vicente.romero at oracle.com (Vicente-Arturo Romero-Zaldivar) Date: Wed, 08 Jul 2015 13:39:17 -0700 Subject: Change in type inference behaviour with JDK-8078024 In-Reply-To: References: <559C6795.8000503@oracle.com> <559C84C0.3050903@oracle.com> Message-ID: <559D8A75.4070003@oracle.com> On 07/08/2015 01:16 PM, Liam Miller-Cushon wrote: > Thanks for investigating, and for the detailed explanation. I'm glad > to hear that the new behaviour is valid. > > Is it worth adding a regression test for this? Yes it is. I will create a bug report for it. Thanks, Vicente > > On Tue, Jul 7, 2015 at 7:02 PM, Vicente-Arturo Romero-Zaldivar > > wrote: > > Hi, > > I have modified the test case to: > > import java.util.Arrays; > import java.util.List; > > class K { > public static List> cartesianProduct(List A>... lists) { > return cartesianProduct(Arrays.asList(lists)); > } > > public static List> cartesianProduct(List List> lists) { > throw new AssertionError(); > } > } > > In order to have different names for each type variable, now A and B. > > This corner case shows a semantic change of the patch for > JDK-8078024. While determining the applicability of method: > > List> cartesianProduct(List B>> lists) > > For which we have the constraints: > > B <: Object > T <: List > T<: Object > List <: T > > First, variable B is selected for instantiation: > > B = CAP1 of ? extends A > so this implies that: > > T <: List > T<: Object > List <: T > > OK now all the bounds are checked for consistency. While checking > if List is a subtype of List extends A> a bound error is reported. Before the compiler was just > swallowing it. As now the error is reported while variable B is > being instantiated, the bound set is rolled back to it's initial > state, B is instantiated to Object, and with this instantiation > the constraint set is solvable, the method is applicable, it's the > only applicable one and the code is accepted as correct. > > > Thanks, > Vicente > > > > > On 07/07/2015 04:58 PM, Maurizio Cimadamore wrote: >> My understanding is that 8078024 should have been mostly a >> cleanup with performance-related issues (i.e. less time to >> converge to a non-existent solution). But Vicente did the work >> and might know few things more about this one, so let's hear from >> him. >> >> Maurizio >> >> On 07/07/15 22:34, Liam Miller-Cushon wrote: >>> The fix for JDK-8078024 resolved an issue with type inference I >>> was seeing with javac9. I can't tell from the bug report if that >>> was deliberate. >>> >>> Could someone please confirm that the current behaviour is correct? >>> >>> Here's the repro. It fails with jdk9-dev @2888, and compiles >>> cleanly @2889: >>> >>> === A.java === >>> import java.util.Arrays; >>> import java.util.List; >>> >>> class A { >>> public static List> cartesianProduct(List>> extends B>... lists) { >>> return cartesianProduct(Arrays.asList(lists)); >>> } >>> >>> public static List> cartesianProduct(List>> extends List> lists) { >>> throw new AssertionError(); >>> } >>> } >>> === >>> >>> $ javac A.java >>> A.java:6: error: incompatible types: inference variable B has >>> incompatible bounds >>> return cartesianProduct(Arrays.asList(lists)); >>> ^ >>> equality constraints: B >>> lower bounds: T,List >>> where B,T are type-variables: >>> B extends Object declared in method >>> cartesianProduct(List...) >>> T extends Object declared in method asList(T...) >>> Note: /usr/local/google/home/cushon/A.java uses unchecked or >>> unsafe operations. >>> Note: Recompile with -Xlint:unchecked for details. >>> 1 error >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vicente.romero at oracle.com Wed Jul 8 20:46:05 2015 From: vicente.romero at oracle.com (Vicente-Arturo Romero-Zaldivar) Date: Wed, 08 Jul 2015 13:46:05 -0700 Subject: Change in type inference behaviour with JDK-8078024 In-Reply-To: <559D8A75.4070003@oracle.com> References: <559C6795.8000503@oracle.com> <559C84C0.3050903@oracle.com> <559D8A75.4070003@oracle.com> Message-ID: <559D8C0D.1080606@oracle.com> On 07/08/2015 01:39 PM, Vicente-Arturo Romero-Zaldivar wrote: > On 07/08/2015 01:16 PM, Liam Miller-Cushon wrote: >> Thanks for investigating, and for the detailed explanation. I'm glad >> to hear that the new behaviour is valid. >> >> Is it worth adding a regression test for this? > > Yes it is. I will create a bug report for it. done: https://bugs.openjdk.java.net/browse/JDK-8130803 Thanks, Vicente > > Thanks, > Vicente > >> >> On Tue, Jul 7, 2015 at 7:02 PM, Vicente-Arturo Romero-Zaldivar >> > wrote: >> >> Hi, >> >> I have modified the test case to: >> >> import java.util.Arrays; >> import java.util.List; >> >> class K { >> public static List> cartesianProduct(List> A>... lists) { >> return cartesianProduct(Arrays.asList(lists)); >> } >> >> public static List> cartesianProduct(List> List> lists) { >> throw new AssertionError(); >> } >> } >> >> In order to have different names for each type variable, now A and B. >> >> This corner case shows a semantic change of the patch for >> JDK-8078024. While determining the applicability of method: >> >> List> cartesianProduct(List> B>> lists) >> >> For which we have the constraints: >> >> B <: Object >> T <: List >> T<: Object >> List <: T >> >> First, variable B is selected for instantiation: >> >> B = CAP1 of ? extends A >> so this implies that: >> >> T <: List >> T<: Object >> List <: T >> >> OK now all the bounds are checked for consistency. While checking >> if List is a subtype of List> extends A> a bound error is reported. Before the compiler was >> just swallowing it. As now the error is reported while variable B >> is being instantiated, the bound set is rolled back to it's >> initial state, B is instantiated to Object, and with this >> instantiation the constraint set is solvable, the method is >> applicable, it's the only applicable one and the code is accepted >> as correct. >> >> >> Thanks, >> Vicente >> >> >> >> >> On 07/07/2015 04:58 PM, Maurizio Cimadamore wrote: >>> My understanding is that 8078024 should have been mostly a >>> cleanup with performance-related issues (i.e. less time to >>> converge to a non-existent solution). But Vicente did the work >>> and might know few things more about this one, so let's hear >>> from him. >>> >>> Maurizio >>> >>> On 07/07/15 22:34, Liam Miller-Cushon wrote: >>>> The fix for JDK-8078024 resolved an issue with type inference I >>>> was seeing with javac9. I can't tell from the bug report if >>>> that was deliberate. >>>> >>>> Could someone please confirm that the current behaviour is >>>> correct? >>>> >>>> Here's the repro. It fails with jdk9-dev @2888, and compiles >>>> cleanly @2889: >>>> >>>> === A.java === >>>> import java.util.Arrays; >>>> import java.util.List; >>>> >>>> class A { >>>> public static List> cartesianProduct(List>>> extends B>... lists) { >>>> return cartesianProduct(Arrays.asList(lists)); >>>> } >>>> >>>> public static List> cartesianProduct(List>>> extends List> lists) { >>>> throw new AssertionError(); >>>> } >>>> } >>>> === >>>> >>>> $ javac A.java >>>> A.java:6: error: incompatible types: inference variable B has >>>> incompatible bounds >>>> return cartesianProduct(Arrays.asList(lists)); >>>> ^ >>>> equality constraints: B >>>> lower bounds: T,List >>>> where B,T are type-variables: >>>> B extends Object declared in method >>>> cartesianProduct(List...) >>>> T extends Object declared in method asList(T...) >>>> Note: /usr/local/google/home/cushon/A.java uses unchecked or >>>> unsafe operations. >>>> Note: Recompile with -Xlint:unchecked for details. >>>> 1 error >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.smith at oracle.com Wed Jul 8 23:23:41 2015 From: daniel.smith at oracle.com (Dan Smith) Date: Wed, 8 Jul 2015 16:23:41 -0700 Subject: Please reply to my JDK-8059640 issue comment In-Reply-To: <559AB728.80105@oracle.com> References: <559AB728.80105@oracle.com> Message-ID: > On Jul 6, 2015, at 10:13 AM, Georgiy Rakov wrote: > > Hello, > > could you please reply to my following comment I made to JDK-8059640; quoting it: > Suggested spec change states that 15.13.3 will contain following assertion: > > LinkageError may occur due to inconsistencies between compile time and run time (in the referenced method, the functional interface, or the argument types of either). > > The details are provided in parenthesis: "in the referenced method, the functional interface, or the argument types of either". It seems that these details don't extend to following case specified in jls-15.12.4.3-120: > > If the invocation mode is interface, then the implementation must also check that the target reference type still implements the specified interface. If the target reference type does not still implement the interface, then an IncompatibleClassChangeError occurs. > > I believe they should. > > Namely, could you please tell if you agree that spec change suggested in the issue description doesn't extend to the case specified in jls-15.12.4.3-120. If you agree I would appreciate greatly if you could modify the suggested spec change so that it extends to jls-15.12.4.3-120 case. > I think these are two different kinds of errors. JDK-8059640 is about BootstrapMethodErrors that can occur upon *evaluation of* a method reference expression, but that aren't specified to occur. The check you describe is an ICCE that can happen upon *invocation of* the method of an object derived from a method reference expression. Please correct me if you're seeing different behavior, but my expectation is that the invocation will behave exactly as specified: per 15.13.3, the body of the method is via an invocation that proceeds according to 15.12.4.3; per 15.12.4.3, the ICCE can occur. Example: interface I { int x(); } interface Func { int apply(I arg); } Func f2 = I::x; System.out.println(f2); // no errors yet System.out.println(f2.apply(new C())); // ICCE if C was originally declared to implement I, but no longer does So I don't think there's a bug here, either in the spec or the implementation. ?Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From ron at paralleluniverse.co Fri Jul 10 08:43:11 2015 From: ron at paralleluniverse.co (Ron Pressler) Date: Fri, 10 Jul 2015 11:43:11 +0300 Subject: Annotating lambda expressions Message-ID: (Related to http://mail.openjdk.java.net/pipermail/compiler-dev/2015-January/009220.html ) It is useful to add an annotation to a lambda expression -- as in foo(@Special () -> { ...}) -- which will be applied to the static method implementing the lambda (not to the interface the lambda implements). This would allow instrumentation tools operating on method implementations to handle lambdas. So far, AFAICT, this is not possible (or am I wrong?). Opinions? Ron -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Fri Jul 10 09:29:07 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 10 Jul 2015 11:29:07 +0200 Subject: Annotating lambda expressions In-Reply-To: References: Message-ID: <559F9063.908@univ-mlv.fr> On 07/10/2015 10:43 AM, Ron Pressler wrote: > (Related to > http://mail.openjdk.java.net/pipermail/compiler-dev/2015-January/009220.html) > > > It is useful to add an annotation to a lambda expression -- as in > foo(@Special () -> { ...}) -- which will be applied to the static > method implementing the lambda (not to the interface the lambda > implements). This would allow instrumentation tools operating on > method implementations to handle lambdas. So far, AFAICT, this is not > possible (or am I wrong?). You're right you can not annotate a lambda expression. > > Opinions? You can not annotate an expression in Java thus you can not annotate a lambda expression. In Java, unlike say in Ruby or in Lisp, there is a clear distinction between the declaration part of the language and the executable part. Historically (Java 5), annotations was reserved to the declaration part of the language. This was later expended to allow to annotate types in order to support "pluggable type system" in Java 8 (see the JSR 308). As far as i know, during the initial introduction of annotations and during the JSR 308, the question of adding annotations on expressions was raised and rejected by doing a cost/benefit analysis. > > Ron R?mi From maurizio.cimadamore at oracle.com Fri Jul 10 09:31:55 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 10 Jul 2015 10:31:55 +0100 Subject: Annotating lambda expressions In-Reply-To: References: Message-ID: <559F910B.7060004@oracle.com> On 10/07/15 09:43, Ron Pressler wrote: > (Related to > http://mail.openjdk.java.net/pipermail/compiler-dev/2015-January/009220.html) > > > It is useful to add an annotation to a lambda expression -- as in > foo(@Special () -> { ...}) -- which will be applied to the static > method implementing the lambda (not to the interface the lambda > implements). This would allow instrumentation tools operating on > method implementations to handle lambdas. So far, AFAICT, this is not > possible (or am I wrong?). This is not possible, currently - as Remi said; you have annotations on declarations (i.e. variable, classes, methods) - you have annotation on types (i.e. type of a cast, type argument, etc.) - but you don't have annotations on _expressions_. I take it that, under the current translation scheme, there seems to be a plausible description for what should happen when the lambda is annotated; but what if the lambda is turned into a method reference (by some IDE refactoring, or human intervention) ? In that case it is not guaranteed that there would still be a physical place where to put the annotation (unless we tell the compiler to 'expand' all annotated method references). So, while I don't question the genuine nature of the use case you put forward, I'm not seeing a 100% straightforward path to get there w/o adding non trivial complexity to both implementation and the spec. Maurizio > > Opinions? > > Ron From alex.buckley at oracle.com Fri Jul 10 19:38:14 2015 From: alex.buckley at oracle.com (Alex Buckley) Date: Fri, 10 Jul 2015 12:38:14 -0700 Subject: Annotating lambda expressions In-Reply-To: <559F910B.7060004@oracle.com> References: <559F910B.7060004@oracle.com> Message-ID: <55A01F26.2050203@oracle.com> On 7/10/2015 2:31 AM, Maurizio Cimadamore wrote: > On 10/07/15 09:43, Ron Pressler wrote: >> (Related to >> http://mail.openjdk.java.net/pipermail/compiler-dev/2015-January/009220.html) >> >> It is useful to add an annotation to a lambda expression -- as in >> foo(@Special () -> { ...}) -- which will be applied to the static >> method implementing the lambda (not to the interface the lambda >> implements). This would allow instrumentation tools operating on >> method implementations to handle lambdas. So far, AFAICT, this is not >> possible (or am I wrong?). > This is not possible, currently - as Remi said; you have annotations on > declarations (i.e. variable, classes, methods) - you have annotation on > types (i.e. type of a cast, type argument, etc.) - but you don't have > annotations on _expressions_. > I take it that, under the current translation scheme, there seems to be > a plausible description for what should happen when the lambda is > annotated; but what if the lambda is turned into a method reference (by > some IDE refactoring, or human intervention) ? In that case it is not > guaranteed that there would still be a physical place where to put the > annotation (unless we tell the compiler to 'expand' all annotated method > references). Yes, the problem with annotations-on-expressions is not the language grammar but the storage of the annotations in the class file. With no standard way of translating source expressions to bytecode, there can be no standard API for retrieving annotations-on-expressions from bytecode. More broadly, there has never been a standard reflective API capable of inspecting method bodies. This explains why JSR 308 stayed away from annotations-on-expressions -- we were not in a position to let them be read back. It was already bad enough that some annotations-on-types are stored in the class file at the level of individual bytecodes (e.g. an annotation on the type of a cast) so are not retrievable by any standard reflective API. Alex From cushon at google.com Fri Jul 10 21:24:23 2015 From: cushon at google.com (Liam Miller-Cushon) Date: Fri, 10 Jul 2015 14:24:23 -0700 Subject: JavacFileManager#setLocationFromPaths only works with the default filesystem Message-ID: I've been using JavacPathFileManager and an in-memory filesystem ( github.com/google/jimfs) for testing some code that uses the compiler API. I noticed that JDK-8076420 deleted JavacPathFileManager and moved the Path-based methods into the regular JavacFileManager. I think that's great, but I'm having trouble using the Path-based API in JavacFileManager. The implementation uses Path#toFile(), which only works with the default filesystem. Is this a known issue? JDK-8059976 looks like it would probably fix it. Repro: === import com.google.common.jimfs.Configuration; import com.google.common.jimfs.Jimfs; import static java.nio.charset.StandardCharsets.UTF_8; import com.sun.tools.javac.file.JavacFileManager; import com.sun.tools.javac.util.Context; import com.sun.tools.javac.util.Log; import java.nio.file.FileSystem; import java.io.OutputStreamWriter; import java.io.PrintWriter; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Path; import java.util.Arrays; import java.util.Locale; import javax.tools.StandardLocation; public class Test { public static void main(String[] args) throws IOException { FileSystem fs = Jimfs.newFileSystem(Configuration.unix()); Path foo = fs.getPath("/foo"); Files.createDirectory(foo); Path hello = foo.resolve("hello.jar"); Files.write(hello, Arrays.asList("hello world"), UTF_8); Context context = new Context(); context.put(Log.outKey, new PrintWriter(new OutputStreamWriter(System.err, UTF_8), true)); JavacFileManager jfm = new JavacFileManager(context, true, UTF_8); jfm.setLocationFromPaths(StandardLocation.CLASS_PATH, Arrays.asList(hello)); } } === $ javac -cp jimfs.jar:javac.jar Test.java $ java -cp jimfs.jar:javac.jar:guava.jar:. Test Exception in thread "main" java.lang.UnsupportedOperationException at com.google.common.jimfs.JimfsPath.toFile(JimfsPath.java:397) at com.sun.tools.javac.file.FSInfo.getJarClassPath(FSInfo.java:69) at com.sun.tools.javac.file.Locations$SearchPath.addJarClassPath(Locations.java:321) at com.sun.tools.javac.file.Locations$SearchPath.addFile(Locations.java:311) at com.sun.tools.javac.file.Locations$SearchPath.addFiles(Locations.java:250) at com.sun.tools.javac.file.Locations$SearchPath.addFiles(Locations.java:257) at com.sun.tools.javac.file.Locations$SimpleLocationHandler.setLocation(Locations.java:463) at com.sun.tools.javac.file.Locations.setLocation(Locations.java:783) at com.sun.tools.javac.file.JavacFileManager.setLocationFromPaths(JavacFileManager.java:914) at Test.main(Test.java:34) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ron at paralleluniverse.co Fri Jul 10 21:33:55 2015 From: ron at paralleluniverse.co (Ron Pressler) Date: Sat, 11 Jul 2015 00:33:55 +0300 Subject: Annotating lambda expressions In-Reply-To: <559F910B.7060004@oracle.com> References: <559F910B.7060004@oracle.com> Message-ID: Allowing annotations on lambda expressions but not on method references seems very reasonable to me. Manual expansion of the method reference into an expression is always possible. Ron Pressler paralleluniverse.co @puniverseco on Twitter On Fri, Jul 10, 2015 at 12:31 PM, Maurizio Cimadamore < maurizio.cimadamore at oracle.com> wrote: > > > On 10/07/15 09:43, Ron Pressler wrote: > >> (Related to >> http://mail.openjdk.java.net/pipermail/compiler-dev/2015-January/009220.html) >> >> >> It is useful to add an annotation to a lambda expression -- as in >> foo(@Special () -> { ...}) -- which will be applied to the static method >> implementing the lambda (not to the interface the lambda implements). This >> would allow instrumentation tools operating on method implementations to >> handle lambdas. So far, AFAICT, this is not possible (or am I wrong?). >> > This is not possible, currently - as Remi said; you have annotations on > declarations (i.e. variable, classes, methods) - you have annotation on > types (i.e. type of a cast, type argument, etc.) - but you don't have > annotations on _expressions_. > I take it that, under the current translation scheme, there seems to be a > plausible description for what should happen when the lambda is annotated; > but what if the lambda is turned into a method reference (by some IDE > refactoring, or human intervention) ? In that case it is not guaranteed > that there would still be a physical place where to put the annotation > (unless we tell the compiler to 'expand' all annotated method references). > > So, while I don't question the genuine nature of the use case you put > forward, I'm not seeing a 100% straightforward path to get there w/o adding > non trivial complexity to both implementation and the spec. > > Maurizio > >> >> Opinions? >> >> Ron >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ron at paralleluniverse.co Fri Jul 10 21:41:12 2015 From: ron at paralleluniverse.co (Ron Pressler) Date: Sat, 11 Jul 2015 00:41:12 +0300 Subject: Annotating lambda expressions In-Reply-To: <55A01F26.2050203@oracle.com> References: <559F910B.7060004@oracle.com> <55A01F26.2050203@oracle.com> Message-ID: I see no problem with cast annotations not being retrievable via straightforward reflection just as I see no problem annotating the generated method (which should always exist *somewhere*): in both cases, those annotations are intended for use by tools that analyze the entire project's source/bytecode (e.g. for typechecking/instrumentation) rather than by specific reflective queries. What bothers me is that a capability that was trivial with inner classes is now lost when using lambdas. Ron Pressler paralleluniverse.co @puniverseco on Twitter On Fri, Jul 10, 2015 at 10:38 PM, Alex Buckley wrote: > On 7/10/2015 2:31 AM, Maurizio Cimadamore wrote: > >> On 10/07/15 09:43, Ron Pressler wrote: >> >>> (Related to >>> >>> http://mail.openjdk.java.net/pipermail/compiler-dev/2015-January/009220.html >>> ) >>> >>> It is useful to add an annotation to a lambda expression -- as in >>> foo(@Special () -> { ...}) -- which will be applied to the static >>> method implementing the lambda (not to the interface the lambda >>> implements). This would allow instrumentation tools operating on >>> method implementations to handle lambdas. So far, AFAICT, this is not >>> possible (or am I wrong?). >>> >> This is not possible, currently - as Remi said; you have annotations on >> declarations (i.e. variable, classes, methods) - you have annotation on >> types (i.e. type of a cast, type argument, etc.) - but you don't have >> annotations on _expressions_. >> I take it that, under the current translation scheme, there seems to be >> a plausible description for what should happen when the lambda is >> annotated; but what if the lambda is turned into a method reference (by >> some IDE refactoring, or human intervention) ? In that case it is not >> guaranteed that there would still be a physical place where to put the >> annotation (unless we tell the compiler to 'expand' all annotated method >> references). >> > > Yes, the problem with annotations-on-expressions is not the language > grammar but the storage of the annotations in the class file. With no > standard way of translating source expressions to bytecode, there can be no > standard API for retrieving annotations-on-expressions from bytecode. More > broadly, there has never been a standard reflective API capable of > inspecting method bodies. > > This explains why JSR 308 stayed away from annotations-on-expressions -- > we were not in a position to let them be read back. It was already bad > enough that some annotations-on-types are stored in the class file at the > level of individual bytecodes (e.g. an annotation on the type of a cast) so > are not retrievable by any standard reflective API. > > Alex > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonathan.gibbons at oracle.com Fri Jul 10 21:53:24 2015 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 10 Jul 2015 14:53:24 -0700 Subject: JavacFileManager#setLocationFromPaths only works with the default filesystem In-Reply-To: References: Message-ID: <55A03ED4.6040201@oracle.com> Liam, Thanks for the feedback. Obviously the intent is that you should be able to use Paths that don't rely on the use of the default filesystem, so this needs to be fixed. -- Jon On 07/10/2015 02:24 PM, Liam Miller-Cushon wrote: > I've been using JavacPathFileManager and an in-memory filesystem > (github.com/google/jimfs ) for testing > some code that uses the compiler API. > > I noticed that JDK-8076420 deleted JavacPathFileManager and moved the > Path-based methods into the regular JavacFileManager. I think that's > great, but I'm having trouble using the Path-based API in > JavacFileManager. The implementation uses Path#toFile(), which only > works with the default filesystem. > > Is this a known issue? JDK-8059976 looks like it would probably fix it. > > Repro: > > === > import com.google.common.jimfs.Configuration; > import com.google.common.jimfs.Jimfs; > > import static java.nio.charset.StandardCharsets.UTF_8; > > import com.sun.tools.javac.file.JavacFileManager; > import com.sun.tools.javac.util.Context; > import com.sun.tools.javac.util.Log; > > import java.nio.file.FileSystem; > import java.io.OutputStreamWriter; > import java.io.PrintWriter; > import java.io.IOException; > import java.nio.file.Files; > import java.nio.file.Path; > import java.util.Arrays; > import java.util.Locale; > > import javax.tools.StandardLocation; > > public class Test { > public static void main(String[] args) throws IOException { > FileSystem fs = Jimfs.newFileSystem(Configuration.unix()); > > Path foo = fs.getPath("/foo"); > Files.createDirectory(foo); > Path hello = foo.resolve("hello.jar"); > Files.write(hello, Arrays.asList("hello world"), UTF_8); > > Context context = new Context(); > context.put(Log.outKey, new PrintWriter(new > OutputStreamWriter(System.err, UTF_8), true)); > JavacFileManager jfm = new JavacFileManager(context, true, UTF_8); > > jfm.setLocationFromPaths(StandardLocation.CLASS_PATH, > Arrays.asList(hello)); > } > } > === > > $ javac -cp jimfs.jar:javac.jar Test.java > $ java -cp jimfs.jar:javac.jar:guava.jar:. Test > Exception in thread "main" java.lang.UnsupportedOperationException > at com.google.common.jimfs.JimfsPath.toFile(JimfsPath.java:397) > at com.sun.tools.javac.file.FSInfo.getJarClassPath(FSInfo.java:69) > at > com.sun.tools.javac.file.Locations$SearchPath.addJarClassPath(Locations.java:321) > at > com.sun.tools.javac.file.Locations$SearchPath.addFile(Locations.java:311) > at > com.sun.tools.javac.file.Locations$SearchPath.addFiles(Locations.java:250) > at > com.sun.tools.javac.file.Locations$SearchPath.addFiles(Locations.java:257) > at > com.sun.tools.javac.file.Locations$SimpleLocationHandler.setLocation(Locations.java:463) > at > com.sun.tools.javac.file.Locations.setLocation(Locations.java:783) > at > com.sun.tools.javac.file.JavacFileManager.setLocationFromPaths(JavacFileManager.java:914) > at Test.main(Test.java:34) -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonathan.gibbons at oracle.com Fri Jul 10 21:57:43 2015 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 10 Jul 2015 14:57:43 -0700 Subject: JavacFileManager#setLocationFromPaths only works with the default filesystem In-Reply-To: <55A03ED4.6040201@oracle.com> References: <55A03ED4.6040201@oracle.com> Message-ID: <55A03FD7.7020604@oracle.com> FWIW, the scary big deal about fixing 8059976 is that clients will need to be more careful to ensure that file managers are closed after use, so that the file manager can close any file systems that have been opened internally (primarily meaning, jar file systems). Right now, with the old fashioned support for jar files, that is not the case. -- Jon On 07/10/2015 02:53 PM, Jonathan Gibbons wrote: > Liam, > > Thanks for the feedback. Obviously the intent is that you should be > able to use Paths that don't rely on the use of the default > filesystem, so this needs to be fixed. > > -- Jon > > > On 07/10/2015 02:24 PM, Liam Miller-Cushon wrote: >> I've been using JavacPathFileManager and an in-memory filesystem >> (github.com/google/jimfs ) for >> testing some code that uses the compiler API. >> >> I noticed that JDK-8076420 deleted JavacPathFileManager and moved the >> Path-based methods into the regular JavacFileManager. I think that's >> great, but I'm having trouble using the Path-based API in >> JavacFileManager. The implementation uses Path#toFile(), which only >> works with the default filesystem. >> >> Is this a known issue? JDK-8059976 looks like it would probably fix it. >> >> Repro: >> >> === >> import com.google.common.jimfs.Configuration; >> import com.google.common.jimfs.Jimfs; >> >> import static java.nio.charset.StandardCharsets.UTF_8; >> >> import com.sun.tools.javac.file.JavacFileManager; >> import com.sun.tools.javac.util.Context; >> import com.sun.tools.javac.util.Log; >> >> import java.nio.file.FileSystem; >> import java.io.OutputStreamWriter; >> import java.io.PrintWriter; >> import java.io.IOException; >> import java.nio.file.Files; >> import java.nio.file.Path; >> import java.util.Arrays; >> import java.util.Locale; >> >> import javax.tools.StandardLocation; >> >> public class Test { >> public static void main(String[] args) throws IOException { >> FileSystem fs = Jimfs.newFileSystem(Configuration.unix()); >> >> Path foo = fs.getPath("/foo"); >> Files.createDirectory(foo); >> Path hello = foo.resolve("hello.jar"); >> Files.write(hello, Arrays.asList("hello world"), UTF_8); >> >> Context context = new Context(); >> context.put(Log.outKey, new PrintWriter(new >> OutputStreamWriter(System.err, UTF_8), true)); >> JavacFileManager jfm = new JavacFileManager(context, true, UTF_8); >> >> jfm.setLocationFromPaths(StandardLocation.CLASS_PATH, >> Arrays.asList(hello)); >> } >> } >> === >> >> $ javac -cp jimfs.jar:javac.jar Test.java >> $ java -cp jimfs.jar:javac.jar:guava.jar:. Test >> Exception in thread "main" java.lang.UnsupportedOperationException >> at com.google.common.jimfs.JimfsPath.toFile(JimfsPath.java:397) >> at >> com.sun.tools.javac.file.FSInfo.getJarClassPath(FSInfo.java:69) >> at >> com.sun.tools.javac.file.Locations$SearchPath.addJarClassPath(Locations.java:321) >> at >> com.sun.tools.javac.file.Locations$SearchPath.addFile(Locations.java:311) >> at >> com.sun.tools.javac.file.Locations$SearchPath.addFiles(Locations.java:250) >> at >> com.sun.tools.javac.file.Locations$SearchPath.addFiles(Locations.java:257) >> at >> com.sun.tools.javac.file.Locations$SimpleLocationHandler.setLocation(Locations.java:463) >> at >> com.sun.tools.javac.file.Locations.setLocation(Locations.java:783) >> at >> com.sun.tools.javac.file.JavacFileManager.setLocationFromPaths(JavacFileManager.java:914) >> at Test.main(Test.java:34) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.buckley at oracle.com Fri Jul 10 22:03:23 2015 From: alex.buckley at oracle.com (Alex Buckley) Date: Fri, 10 Jul 2015 15:03:23 -0700 Subject: Annotating lambda expressions In-Reply-To: References: <559F910B.7060004@oracle.com> <55A01F26.2050203@oracle.com> Message-ID: <55A0412B.7030101@oracle.com> OK, but it's precisely the fact that inner classes are declarations that allows them to be annotated, because for declarations we specify the binary name, membership, and other artifacts of translation. No such spec for the translation of expressions (or statements) -- and I think it's generally regarded as a feature-not-a-bug that compilers have great freedom in translating lambda expressions. Alex On 7/10/2015 2:41 PM, Ron Pressler wrote: > I see no problem with cast annotations not being retrievable via > straightforward reflection just as I see no problem annotating the > generated method (which should always exist /somewhere/): in both cases, > those annotations are intended for use by tools that analyze the entire > project's source/bytecode (e.g. for typechecking/instrumentation) rather > than by specific reflective queries. > > What bothers me is that a capability that was trivial with inner classes > is now lost when using lambdas. > > Ron Pressler > paralleluniverse.co > @puniverseco on Twitter > > On Fri, Jul 10, 2015 at 10:38 PM, Alex Buckley > wrote: > > On 7/10/2015 2:31 AM, Maurizio Cimadamore wrote: > > On 10/07/15 09:43, Ron Pressler wrote: > > (Related to > http://mail.openjdk.java.net/pipermail/compiler-dev/2015-January/009220.html) > > It is useful to add an annotation to a lambda expression -- > as in > foo(@Special () -> { ...}) -- which will be applied to the > static > method implementing the lambda (not to the interface the lambda > implements). This would allow instrumentation tools operating on > method implementations to handle lambdas. So far, AFAICT, > this is not > possible (or am I wrong?). > > This is not possible, currently - as Remi said; you have > annotations on > declarations (i.e. variable, classes, methods) - you have > annotation on > types (i.e. type of a cast, type argument, etc.) - but you don't > have > annotations on _expressions_. > I take it that, under the current translation scheme, there > seems to be > a plausible description for what should happen when the lambda is > annotated; but what if the lambda is turned into a method > reference (by > some IDE refactoring, or human intervention) ? In that case it > is not > guaranteed that there would still be a physical place where to > put the > annotation (unless we tell the compiler to 'expand' all > annotated method > references). > > > Yes, the problem with annotations-on-expressions is not the language > grammar but the storage of the annotations in the class file. With > no standard way of translating source expressions to bytecode, there > can be no standard API for retrieving annotations-on-expressions > from bytecode. More broadly, there has never been a standard > reflective API capable of inspecting method bodies. > > This explains why JSR 308 stayed away from > annotations-on-expressions -- we were not in a position to let them > be read back. It was already bad enough that some > annotations-on-types are stored in the class file at the level of > individual bytecodes (e.g. an annotation on the type of a cast) so > are not retrievable by any standard reflective API. > > Alex > > From jonathan.gibbons at oracle.com Fri Jul 10 22:24:41 2015 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 10 Jul 2015 15:24:41 -0700 Subject: JavacFileManager#setLocationFromPaths only works with the default filesystem In-Reply-To: <55A03FD7.7020604@oracle.com> References: <55A03ED4.6040201@oracle.com> <55A03FD7.7020604@oracle.com> Message-ID: <55A04629.4050401@oracle.com> Now tracked as https://bugs.openjdk.java.net/browse/JDK-8130944 -- Jon On 07/10/2015 02:57 PM, Jonathan Gibbons wrote: > FWIW, the scary big deal about fixing 8059976 is that clients will > need to be more careful to ensure that file managers are closed after > use, so that the file manager can close any file systems that have > been opened internally (primarily meaning, jar file systems). Right > now, with the old fashioned support for jar files, that is not the case. > > -- Jon > > > On 07/10/2015 02:53 PM, Jonathan Gibbons wrote: >> Liam, >> >> Thanks for the feedback. Obviously the intent is that you should be >> able to use Paths that don't rely on the use of the default >> filesystem, so this needs to be fixed. >> >> -- Jon >> >> >> On 07/10/2015 02:24 PM, Liam Miller-Cushon wrote: >>> I've been using JavacPathFileManager and an in-memory filesystem >>> (github.com/google/jimfs ) for >>> testing some code that uses the compiler API. >>> >>> I noticed that JDK-8076420 deleted JavacPathFileManager and moved >>> the Path-based methods into the regular JavacFileManager. I think >>> that's great, but I'm having trouble using the Path-based API in >>> JavacFileManager. The implementation uses Path#toFile(), which only >>> works with the default filesystem. >>> >>> Is this a known issue? JDK-8059976 looks like it would probably fix it. >>> >>> Repro: >>> >>> === >>> import com.google.common.jimfs.Configuration; >>> import com.google.common.jimfs.Jimfs; >>> >>> import static java.nio.charset.StandardCharsets.UTF_8; >>> >>> import com.sun.tools.javac.file.JavacFileManager; >>> import com.sun.tools.javac.util.Context; >>> import com.sun.tools.javac.util.Log; >>> >>> import java.nio.file.FileSystem; >>> import java.io.OutputStreamWriter; >>> import java.io.PrintWriter; >>> import java.io.IOException; >>> import java.nio.file.Files; >>> import java.nio.file.Path; >>> import java.util.Arrays; >>> import java.util.Locale; >>> >>> import javax.tools.StandardLocation; >>> >>> public class Test { >>> public static void main(String[] args) throws IOException { >>> FileSystem fs = Jimfs.newFileSystem(Configuration.unix()); >>> >>> Path foo = fs.getPath("/foo"); >>> Files.createDirectory(foo); >>> Path hello = foo.resolve("hello.jar"); >>> Files.write(hello, Arrays.asList("hello world"), UTF_8); >>> >>> Context context = new Context(); >>> context.put(Log.outKey, new PrintWriter(new >>> OutputStreamWriter(System.err, UTF_8), true)); >>> JavacFileManager jfm = new JavacFileManager(context, true, UTF_8); >>> >>> jfm.setLocationFromPaths(StandardLocation.CLASS_PATH, >>> Arrays.asList(hello)); >>> } >>> } >>> === >>> >>> $ javac -cp jimfs.jar:javac.jar Test.java >>> $ java -cp jimfs.jar:javac.jar:guava.jar:. Test >>> Exception in thread "main" java.lang.UnsupportedOperationException >>> at com.google.common.jimfs.JimfsPath.toFile(JimfsPath.java:397) >>> at >>> com.sun.tools.javac.file.FSInfo.getJarClassPath(FSInfo.java:69) >>> at >>> com.sun.tools.javac.file.Locations$SearchPath.addJarClassPath(Locations.java:321) >>> at >>> com.sun.tools.javac.file.Locations$SearchPath.addFile(Locations.java:311) >>> at >>> com.sun.tools.javac.file.Locations$SearchPath.addFiles(Locations.java:250) >>> at >>> com.sun.tools.javac.file.Locations$SearchPath.addFiles(Locations.java:257) >>> at >>> com.sun.tools.javac.file.Locations$SimpleLocationHandler.setLocation(Locations.java:463) >>> at >>> com.sun.tools.javac.file.Locations.setLocation(Locations.java:783) >>> at >>> com.sun.tools.javac.file.JavacFileManager.setLocationFromPaths(JavacFileManager.java:914) >>> at Test.main(Test.java:34) >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonathan.gibbons at oracle.com Fri Jul 10 23:51:39 2015 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 10 Jul 2015 16:51:39 -0700 Subject: RFR [9] 8080880: some docs cleanup for langtools In-Reply-To: <555F0191.1010609@oracle.com> References: <555E02BE.7020407@oracle.com> <555E03FB.40803@oracle.com> <555E2A4C.9010903@oracle.com> <555F0191.1010609@oracle.com> Message-ID: <55A05A8B.4090005@oracle.com> Looks OK to me. -- Jon On 05/22/2015 03:14 AM, alexander stepanov wrote: > Hello Jonathan, > > Thanks. > > > It is not appropriate to do partial bulk update to the "not a > supported API" > > The responding changes were reverted, please see > http://cr.openjdk.java.net/~avstepan/8080880/webrev.01/ > > Regards, > Alexander > > On 21.05.2015 21:56, Jonathan Gibbons wrote: >> >> On 05/21/2015 09:12 AM, alexander stepanov wrote: >>> Hello, >>> >>> Could you please review the following fix >>> http://cr.openjdk.java.net/~avstepan/8080880/webrev.00/ >>> for >>> https://bugs.openjdk.java.net/browse/JDK-8080880 >>> >>> Just some minor fix for docs, no other code touched. >>> >>> The affected packages should (probably) not be visible in the new >>> modular system, but nevertheless... >>> >>> Thanks, >>> Alexander >> >> Alexander, >> >> It is not appropriate to do partial bulk update to the "not a >> supported API" >> comment at the head of the internal files. It is strongly preferred >> that this >> part of the comment remain consistent across all source files, so >> that it is >> amenable to bulk updates, should we wish to do so. >> >> I would suggest you separate the otherwise useful work to fix individual >> minor doc issues from any work involving the standard "not a supported >> API" comment at the head of each file. >> >> -- Jon > From forax at univ-mlv.fr Sat Jul 11 07:38:32 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 11 Jul 2015 09:38:32 +0200 Subject: Annotating lambda expressions In-Reply-To: References: <559F910B.7060004@oracle.com> <55A01F26.2050203@oracle.com> Message-ID: <55A0C7F8.1070107@univ-mlv.fr> On 07/10/2015 11:41 PM, Ron Pressler wrote: > I see no problem with cast annotations not being retrievable via > straightforward reflection just as I see no problem annotating the > generated method (which should always exist /somewhere/): in both > cases, those annotations are intended for use by tools that analyze > the entire project's source/bytecode (e.g. for > typechecking/instrumentation) rather than by specific reflective queries. > > What bothers me is that a capability that was trivial with inner > classes is now lost when using lambdas. you can easily add a pattern recognizable by source/bytecode tools to do what you want, let suppose you have a static method like this: static T annotate(Class annotation, T lambda) { return lambda; } to annotate a lambda, your example can be written like this: foo(annotate(Special.class, () -> { ...})) in the bytecode it will be translated to something like this: ldc Special.class invokedynamic ()Runnable LambdaMetafactory + method handle ref on the lambda body invokestatic Bar.annotate(Class, Object)Object checkcast Runnable invokevirtual Baz.foo(Runnable)... which is easy to recognize. > > Ron Pressler > paralleluniverse.co > @puniverseco on Twitter R?mi > > On Fri, Jul 10, 2015 at 10:38 PM, Alex Buckley > > wrote: > > On 7/10/2015 2:31 AM, Maurizio Cimadamore wrote: > > On 10/07/15 09:43, Ron Pressler wrote: > > (Related to > http://mail.openjdk.java.net/pipermail/compiler-dev/2015-January/009220.html) > > It is useful to add an annotation to a lambda expression > -- as in > foo(@Special () -> { ...}) -- which will be applied to the > static > method implementing the lambda (not to the interface the > lambda > implements). This would allow instrumentation tools > operating on > method implementations to handle lambdas. So far, AFAICT, > this is not > possible (or am I wrong?). > > This is not possible, currently - as Remi said; you have > annotations on > declarations (i.e. variable, classes, methods) - you have > annotation on > types (i.e. type of a cast, type argument, etc.) - but you > don't have > annotations on _expressions_. > I take it that, under the current translation scheme, there > seems to be > a plausible description for what should happen when the lambda is > annotated; but what if the lambda is turned into a method > reference (by > some IDE refactoring, or human intervention) ? In that case it > is not > guaranteed that there would still be a physical place where to > put the > annotation (unless we tell the compiler to 'expand' all > annotated method > references). > > > Yes, the problem with annotations-on-expressions is not the > language grammar but the storage of the annotations in the class > file. With no standard way of translating source expressions to > bytecode, there can be no standard API for retrieving > annotations-on-expressions from bytecode. More broadly, there has > never been a standard reflective API capable of inspecting method > bodies. > > This explains why JSR 308 stayed away from > annotations-on-expressions -- we were not in a position to let > them be read back. It was already bad enough that some > annotations-on-types are stored in the class file at the level of > individual bytecodes (e.g. an annotation on the type of a cast) so > are not retrievable by any standard reflective API. > > Alex > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ron at paralleluniverse.co Sat Jul 11 08:57:21 2015 From: ron at paralleluniverse.co (Ron Pressler) Date: Sat, 11 Jul 2015 11:57:21 +0300 Subject: Annotating lambda expressions In-Reply-To: <55A0C7F8.1070107@univ-mlv.fr> References: <559F910B.7060004@oracle.com> <55A01F26.2050203@oracle.com> <55A0C7F8.1070107@univ-mlv.fr> Message-ID: That's a very nice idea (in fact, we can do foo((@Special Runnable) ()->{...}), as well and get ASM to invoke a nice visitInsnAnnotation event), but it doesn't help, as I have no way of finding the method implementing the lambda by analyzing the bytecode. My suggestion says, "however the compiler chooses to represent the lambda, please apply the annotation to the implementing method, wherever it is". I think that the language spec at least implicitly forces the compiler to implement a lambda as a method *somewhere* (as opposed to, say, javac inlining the expression, as Kotlin does with inline functions). The ability to attach introspectable metadata to a lambda expression that then travels with it to the implementing method -- wherever the compiler may choose to place it -- is extremely useful, and trumps, IMO, "purity" concerns re annotating expressions vs declarations, (esp. since we've already established that some expressions -- like cast and new -- are annotatable, and rightly so). Without this, we miss something important Java gives us, which is the ability to attach metadata to a method body. I fully understand the distinction that the annotation is applied to the *declaration* rather than the body itself, but that doesn't take away from the fact that a useful ability does not transfer to lambdas while it could be easily done. Ron On Sat, Jul 11, 2015 at 10:38 AM, Remi Forax wrote: > On 07/10/2015 11:41 PM, Ron Pressler wrote: > > I see no problem with cast annotations not being retrievable via > straightforward reflection just as I see no problem annotating the > generated method (which should always exist *somewhere*): in both cases, > those annotations are intended for use by tools that analyze the entire > project's source/bytecode (e.g. for typechecking/instrumentation) rather > than by specific reflective queries. > > What bothers me is that a capability that was trivial with inner classes > is now lost when using lambdas. > > > you can easily add a pattern recognizable by source/bytecode tools to do > what you want, > let suppose you have a static method like this: > static T annotate(Class annotation, T lambda) > { return lambda; } > > to annotate a lambda, your example can be written like this: > foo(annotate(Special.class, () -> { ...})) > > in the bytecode it will be translated to something like this: > ldc Special.class > invokedynamic ()Runnable > LambdaMetafactory + method handle ref on the lambda body > invokestatic Bar.annotate(Class, Object)Object > checkcast Runnable > invokevirtual Baz.foo(Runnable)... > > which is easy to recognize. > > > Ron Pressler > paralleluniverse.co > @puniverseco on Twitter > > > R?mi > > > > On Fri, Jul 10, 2015 at 10:38 PM, Alex Buckley > wrote: > >> On 7/10/2015 2:31 AM, Maurizio Cimadamore wrote: >> >>> On 10/07/15 09:43, Ron Pressler wrote: >>> >>>> (Related to >>>> >>>> http://mail.openjdk.java.net/pipermail/compiler-dev/2015-January/009220.html >>>> ) >>>> >>>> It is useful to add an annotation to a lambda expression -- as in >>>> foo(@Special () -> { ...}) -- which will be applied to the static >>>> method implementing the lambda (not to the interface the lambda >>>> implements). This would allow instrumentation tools operating on >>>> method implementations to handle lambdas. So far, AFAICT, this is not >>>> possible (or am I wrong?). >>>> >>> This is not possible, currently - as Remi said; you have annotations on >>> declarations (i.e. variable, classes, methods) - you have annotation on >>> types (i.e. type of a cast, type argument, etc.) - but you don't have >>> annotations on _expressions_. >>> I take it that, under the current translation scheme, there seems to be >>> a plausible description for what should happen when the lambda is >>> annotated; but what if the lambda is turned into a method reference (by >>> some IDE refactoring, or human intervention) ? In that case it is not >>> guaranteed that there would still be a physical place where to put the >>> annotation (unless we tell the compiler to 'expand' all annotated method >>> references). >>> >> >> Yes, the problem with annotations-on-expressions is not the language >> grammar but the storage of the annotations in the class file. With no >> standard way of translating source expressions to bytecode, there can be no >> standard API for retrieving annotations-on-expressions from bytecode. More >> broadly, there has never been a standard reflective API capable of >> inspecting method bodies. >> >> This explains why JSR 308 stayed away from annotations-on-expressions -- >> we were not in a position to let them be read back. It was already bad >> enough that some annotations-on-types are stored in the class file at the >> level of individual bytecodes (e.g. an annotation on the type of a cast) so >> are not retrievable by any standard reflective API. >> >> Alex >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Sat Jul 11 09:26:11 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 11 Jul 2015 11:26:11 +0200 Subject: Annotating lambda expressions In-Reply-To: References: <559F910B.7060004@oracle.com> <55A01F26.2050203@oracle.com> <55A0C7F8.1070107@univ-mlv.fr> Message-ID: <55A0E133.4030301@univ-mlv.fr> On 07/11/2015 10:57 AM, Ron Pressler wrote: > That's a very nice idea (in fact, we can do foo((@Special Runnable) > ()->{...}), as well and get ASM to invoke a nice visitInsnAnnotation > event), but it doesn't help, as I have no way of finding the method > implementing the lambda by analyzing the bytecode. The method corresponding to the body of the lambda is one arguments of the bootstrap method of invokedynamic. see http://cr.openjdk.java.net/~briangoetz/lambda/lambda-translation.html (this translation scheme is not required to be used by compilers but in practice javac and ecj uses it and scalac will use it too) > My suggestion says, "however the compiler chooses to represent the > lambda, please apply the annotation to the implementing method, > wherever it is". I think that the language spec at least implicitly > forces the compiler to implement a lambda as a method /somewhere/ (as > opposed to, say, javac inlining the expression, as Kotlin does with > inline functions). see above. > > The ability to attach introspectable metadata to a lambda expression > that then travels with it to the implementing method -- wherever the > compiler may choose to place it -- is extremely useful, and trumps, > IMO, "purity" concerns re annotating expressions vs declarations, > (esp. since we've already established that some expressions -- like > cast and new -- are annotatable, and rightly so). cast and new are not annotatable, the type of a cast or a new are. > Without this, we miss something important Java gives us, which is the > ability to attach metadata to a method body. I fully understand the > distinction that the annotation is applied to the /declaration/ rather > than the body itself, but that doesn't take away from the fact that a > useful ability does not transfer to lambdas while it could be easily done. > > Ron R?mi > > On Sat, Jul 11, 2015 at 10:38 AM, Remi Forax > wrote: > > On 07/10/2015 11:41 PM, Ron Pressler wrote: >> I see no problem with cast annotations not being retrievable via >> straightforward reflection just as I see no problem annotating >> the generated method (which should always exist /somewhere/): in >> both cases, those annotations are intended for use by tools that >> analyze the entire project's source/bytecode (e.g. for >> typechecking/instrumentation) rather than by specific reflective >> queries. >> >> What bothers me is that a capability that was trivial with inner >> classes is now lost when using lambdas. > > you can easily add a pattern recognizable by source/bytecode tools > to do what you want, > let suppose you have a static method like this: > static T annotate(Class annotation, T > lambda) { return lambda; } > > to annotate a lambda, your example can be written like this: > foo(annotate(Special.class, () -> { ...})) > > in the bytecode it will be translated to something like this: > ldc Special.class > invokedynamic ()Runnable > LambdaMetafactory + method handle ref on the lambda body > invokestatic Bar.annotate(Class, Object)Object > checkcast Runnable > invokevirtual Baz.foo(Runnable)... > > which is easy to recognize. > >> >> Ron Pressler >> paralleluniverse.co >> @puniverseco on Twitter > > R?mi > > >> >> On Fri, Jul 10, 2015 at 10:38 PM, Alex Buckley >> > wrote: >> >> On 7/10/2015 2:31 AM, Maurizio Cimadamore wrote: >> >> On 10/07/15 09:43, Ron Pressler wrote: >> >> (Related to >> http://mail.openjdk.java.net/pipermail/compiler-dev/2015-January/009220.html) >> >> It is useful to add an annotation to a lambda >> expression -- as in >> foo(@Special () -> { ...}) -- which will be applied >> to the static >> method implementing the lambda (not to the interface >> the lambda >> implements). This would allow instrumentation tools >> operating on >> method implementations to handle lambdas. So far, >> AFAICT, this is not >> possible (or am I wrong?). >> >> This is not possible, currently - as Remi said; you have >> annotations on >> declarations (i.e. variable, classes, methods) - you have >> annotation on >> types (i.e. type of a cast, type argument, etc.) - but >> you don't have >> annotations on _expressions_. >> I take it that, under the current translation scheme, >> there seems to be >> a plausible description for what should happen when the >> lambda is >> annotated; but what if the lambda is turned into a method >> reference (by >> some IDE refactoring, or human intervention) ? In that >> case it is not >> guaranteed that there would still be a physical place >> where to put the >> annotation (unless we tell the compiler to 'expand' all >> annotated method >> references). >> >> >> Yes, the problem with annotations-on-expressions is not the >> language grammar but the storage of the annotations in the >> class file. With no standard way of translating source >> expressions to bytecode, there can be no standard API for >> retrieving annotations-on-expressions from bytecode. More >> broadly, there has never been a standard reflective API >> capable of inspecting method bodies. >> >> This explains why JSR 308 stayed away from >> annotations-on-expressions -- we were not in a position to >> let them be read back. It was already bad enough that some >> annotations-on-types are stored in the class file at the >> level of individual bytecodes (e.g. an annotation on the type >> of a cast) so are not retrievable by any standard reflective API. >> >> Alex >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ron at paralleluniverse.co Sat Jul 11 12:59:09 2015 From: ron at paralleluniverse.co (Ron Pressler) Date: Sat, 11 Jul 2015 15:59:09 +0300 Subject: Annotating lambda expressions In-Reply-To: <55A0E133.4030301@univ-mlv.fr> References: <559F910B.7060004@oracle.com> <55A01F26.2050203@oracle.com> <55A0C7F8.1070107@univ-mlv.fr> <55A0E133.4030301@univ-mlv.fr> Message-ID: Thanks for the bootstrap argument tip! Will use that as a workaround. Ron On Sat, Jul 11, 2015 at 12:26 PM, Remi Forax wrote: > On 07/11/2015 10:57 AM, Ron Pressler wrote: > > That's a very nice idea (in fact, we can do foo((@Special Runnable) > ()->{...}), as well and get ASM to invoke a nice visitInsnAnnotation > event), but it doesn't help, as I have no way of finding the method > implementing the lambda by analyzing the bytecode. > > > The method corresponding to the body of the lambda is one arguments of the > bootstrap method of invokedynamic. > see > http://cr.openjdk.java.net/~briangoetz/lambda/lambda-translation.html > (this translation scheme is not required to be used by compilers but in > practice javac and ecj uses it and scalac will use it too) > > My suggestion says, "however the compiler chooses to represent the > lambda, please apply the annotation to the implementing method, wherever it > is". I think that the language spec at least implicitly forces the compiler > to implement a lambda as a method *somewhere* (as opposed to, say, javac > inlining the expression, as Kotlin does with inline functions). > > > see above. > > > The ability to attach introspectable metadata to a lambda expression > that then travels with it to the implementing method -- wherever the > compiler may choose to place it -- is extremely useful, and trumps, IMO, > "purity" concerns re annotating expressions vs declarations, (esp. since > we've already established that some expressions -- like cast and new -- are > annotatable, and rightly so). > > > cast and new are not annotatable, the type of a cast or a new are. > > Without this, we miss something important Java gives us, which is the > ability to attach metadata to a method body. I fully understand the > distinction that the annotation is applied to the *declaration* rather > than the body itself, but that doesn't take away from the fact that a > useful ability does not transfer to lambdas while it could be easily done. > > Ron > > > R?mi > > > > On Sat, Jul 11, 2015 at 10:38 AM, Remi Forax wrote: > >> On 07/10/2015 11:41 PM, Ron Pressler wrote: >> >> I see no problem with cast annotations not being retrievable via >> straightforward reflection just as I see no problem annotating the >> generated method (which should always exist *somewhere*): in both cases, >> those annotations are intended for use by tools that analyze the entire >> project's source/bytecode (e.g. for typechecking/instrumentation) rather >> than by specific reflective queries. >> >> What bothers me is that a capability that was trivial with inner >> classes is now lost when using lambdas. >> >> >> you can easily add a pattern recognizable by source/bytecode tools to do >> what you want, >> let suppose you have a static method like this: >> static T annotate(Class annotation, T lambda) >> { return lambda; } >> >> to annotate a lambda, your example can be written like this: >> foo(annotate(Special.class, () -> { ...})) >> >> in the bytecode it will be translated to something like this: >> ldc Special.class >> invokedynamic ()Runnable >> LambdaMetafactory + method handle ref on the lambda body >> invokestatic Bar.annotate(Class, Object)Object >> checkcast Runnable >> invokevirtual Baz.foo(Runnable)... >> >> which is easy to recognize. >> >> >> Ron Pressler >> paralleluniverse.co >> @puniverseco on Twitter >> >> >> R?mi >> >> >> >> On Fri, Jul 10, 2015 at 10:38 PM, Alex Buckley < >> alex.buckley at oracle.com> wrote: >> >>> On 7/10/2015 2:31 AM, Maurizio Cimadamore wrote: >>> >>>> On 10/07/15 09:43, Ron Pressler wrote: >>>> >>>>> (Related to >>>>> >>>>> http://mail.openjdk.java.net/pipermail/compiler-dev/2015-January/009220.html >>>>> ) >>>>> >>>>> It is useful to add an annotation to a lambda expression -- as in >>>>> foo(@Special () -> { ...}) -- which will be applied to the static >>>>> method implementing the lambda (not to the interface the lambda >>>>> implements). This would allow instrumentation tools operating on >>>>> method implementations to handle lambdas. So far, AFAICT, this is not >>>>> possible (or am I wrong?). >>>>> >>>> This is not possible, currently - as Remi said; you have annotations on >>>> declarations (i.e. variable, classes, methods) - you have annotation on >>>> types (i.e. type of a cast, type argument, etc.) - but you don't have >>>> annotations on _expressions_. >>>> I take it that, under the current translation scheme, there seems to be >>>> a plausible description for what should happen when the lambda is >>>> annotated; but what if the lambda is turned into a method reference (by >>>> some IDE refactoring, or human intervention) ? In that case it is not >>>> guaranteed that there would still be a physical place where to put the >>>> annotation (unless we tell the compiler to 'expand' all annotated method >>>> references). >>>> >>> >>> Yes, the problem with annotations-on-expressions is not the language >>> grammar but the storage of the annotations in the class file. With no >>> standard way of translating source expressions to bytecode, there can be no >>> standard API for retrieving annotations-on-expressions from bytecode. More >>> broadly, there has never been a standard reflective API capable of >>> inspecting method bodies. >>> >>> This explains why JSR 308 stayed away from annotations-on-expressions -- >>> we were not in a position to let them be read back. It was already bad >>> enough that some annotations-on-types are stored in the class file at the >>> level of individual bytecodes (e.g. an annotation on the type of a cast) so >>> are not retrievable by any standard reflective API. >>> >>> Alex >>> >> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ron at paralleluniverse.co Sat Jul 11 13:02:59 2015 From: ron at paralleluniverse.co (Ron Pressler) Date: Sat, 11 Jul 2015 16:02:59 +0300 Subject: Annotating lambda expressions In-Reply-To: References: <559F910B.7060004@oracle.com> <55A01F26.2050203@oracle.com> <55A0C7F8.1070107@univ-mlv.fr> <55A0E133.4030301@univ-mlv.fr> Message-ID: On Sat, Jul 11, 2015 at 3:59 PM, Ron Pressler wrote: > Thanks for the bootstrap argument tip! Will use that as a workaround. > > ... although it does require scanning the entire bytecode for the entire project (to find the indy callsites) rather than just looking for annotated methods... R > > On Sat, Jul 11, 2015 at 12:26 PM, Remi Forax wrote: > >> On 07/11/2015 10:57 AM, Ron Pressler wrote: >> >> That's a very nice idea (in fact, we can do foo((@Special Runnable) >> ()->{...}), as well and get ASM to invoke a nice visitInsnAnnotation >> event), but it doesn't help, as I have no way of finding the method >> implementing the lambda by analyzing the bytecode. >> >> >> The method corresponding to the body of the lambda is one arguments of >> the bootstrap method of invokedynamic. >> see >> http://cr.openjdk.java.net/~briangoetz/lambda/lambda-translation.html >> (this translation scheme is not required to be used by compilers but in >> practice javac and ecj uses it and scalac will use it too) >> >> My suggestion says, "however the compiler chooses to represent the >> lambda, please apply the annotation to the implementing method, wherever it >> is". I think that the language spec at least implicitly forces the compiler >> to implement a lambda as a method *somewhere* (as opposed to, say, javac >> inlining the expression, as Kotlin does with inline functions). >> >> >> see above. >> >> >> The ability to attach introspectable metadata to a lambda expression >> that then travels with it to the implementing method -- wherever the >> compiler may choose to place it -- is extremely useful, and trumps, IMO, >> "purity" concerns re annotating expressions vs declarations, (esp. since >> we've already established that some expressions -- like cast and new -- are >> annotatable, and rightly so). >> >> >> cast and new are not annotatable, the type of a cast or a new are. >> >> Without this, we miss something important Java gives us, which is the >> ability to attach metadata to a method body. I fully understand the >> distinction that the annotation is applied to the *declaration* rather >> than the body itself, but that doesn't take away from the fact that a >> useful ability does not transfer to lambdas while it could be easily done. >> >> Ron >> >> >> R?mi >> >> >> >> On Sat, Jul 11, 2015 at 10:38 AM, Remi Forax wrote: >> >>> On 07/10/2015 11:41 PM, Ron Pressler wrote: >>> >>> I see no problem with cast annotations not being retrievable via >>> straightforward reflection just as I see no problem annotating the >>> generated method (which should always exist *somewhere*): in both >>> cases, those annotations are intended for use by tools that analyze the >>> entire project's source/bytecode (e.g. for typechecking/instrumentation) >>> rather than by specific reflective queries. >>> >>> What bothers me is that a capability that was trivial with inner >>> classes is now lost when using lambdas. >>> >>> >>> you can easily add a pattern recognizable by source/bytecode tools to >>> do what you want, >>> let suppose you have a static method like this: >>> static T annotate(Class annotation, T >>> lambda) { return lambda; } >>> >>> to annotate a lambda, your example can be written like this: >>> foo(annotate(Special.class, () -> { ...})) >>> >>> in the bytecode it will be translated to something like this: >>> ldc Special.class >>> invokedynamic ()Runnable >>> LambdaMetafactory + method handle ref on the lambda body >>> invokestatic Bar.annotate(Class, Object)Object >>> checkcast Runnable >>> invokevirtual Baz.foo(Runnable)... >>> >>> which is easy to recognize. >>> >>> >>> Ron Pressler >>> paralleluniverse.co >>> @puniverseco on Twitter >>> >>> >>> R?mi >>> >>> >>> >>> On Fri, Jul 10, 2015 at 10:38 PM, Alex Buckley < >>> alex.buckley at oracle.com> wrote: >>> >>>> On 7/10/2015 2:31 AM, Maurizio Cimadamore wrote: >>>> >>>>> On 10/07/15 09:43, Ron Pressler wrote: >>>>> >>>>>> (Related to >>>>>> >>>>>> http://mail.openjdk.java.net/pipermail/compiler-dev/2015-January/009220.html >>>>>> ) >>>>>> >>>>>> It is useful to add an annotation to a lambda expression -- as in >>>>>> foo(@Special () -> { ...}) -- which will be applied to the static >>>>>> method implementing the lambda (not to the interface the lambda >>>>>> implements). This would allow instrumentation tools operating on >>>>>> method implementations to handle lambdas. So far, AFAICT, this is not >>>>>> possible (or am I wrong?). >>>>>> >>>>> This is not possible, currently - as Remi said; you have annotations on >>>>> declarations (i.e. variable, classes, methods) - you have annotation on >>>>> types (i.e. type of a cast, type argument, etc.) - but you don't have >>>>> annotations on _expressions_. >>>>> I take it that, under the current translation scheme, there seems to be >>>>> a plausible description for what should happen when the lambda is >>>>> annotated; but what if the lambda is turned into a method reference (by >>>>> some IDE refactoring, or human intervention) ? In that case it is not >>>>> guaranteed that there would still be a physical place where to put the >>>>> annotation (unless we tell the compiler to 'expand' all annotated >>>>> method >>>>> references). >>>>> >>>> >>>> Yes, the problem with annotations-on-expressions is not the language >>>> grammar but the storage of the annotations in the class file. With no >>>> standard way of translating source expressions to bytecode, there can be no >>>> standard API for retrieving annotations-on-expressions from bytecode. More >>>> broadly, there has never been a standard reflective API capable of >>>> inspecting method bodies. >>>> >>>> This explains why JSR 308 stayed away from annotations-on-expressions >>>> -- we were not in a position to let them be read back. It was already bad >>>> enough that some annotations-on-types are stored in the class file at the >>>> level of individual bytecodes (e.g. an annotation on the type of a cast) so >>>> are not retrievable by any standard reflective API. >>>> >>>> Alex >>>> >>> >>> >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Sat Jul 11 13:39:37 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 11 Jul 2015 15:39:37 +0200 Subject: Annotating lambda expressions In-Reply-To: References: <559F910B.7060004@oracle.com> <55A01F26.2050203@oracle.com> <55A0C7F8.1070107@univ-mlv.fr> <55A0E133.4030301@univ-mlv.fr> Message-ID: <55A11C99.10309@univ-mlv.fr> On 07/11/2015 03:02 PM, Ron Pressler wrote: > > On Sat, Jul 11, 2015 at 3:59 PM, Ron Pressler > wrote: > > Thanks for the bootstrap argument tip! Will use that as a workaround. > > > ... although it does require scanning the entire bytecode for the > entire project (to find the indy callsites) rather than just looking > for annotated methods... ?? finding annotated methods requires to scan the entire project too ? > > R R > > On Sat, Jul 11, 2015 at 12:26 PM, Remi Forax > wrote: > > On 07/11/2015 10:57 AM, Ron Pressler wrote: >> That's a very nice idea (in fact, we can do foo((@Special >> Runnable) ()->{...}), as well and get ASM to invoke a nice >> visitInsnAnnotation event), but it doesn't help, as I have >> no way of finding the method implementing the lambda by >> analyzing the bytecode. > > The method corresponding to the body of the lambda is one > arguments of the bootstrap method of invokedynamic. > see > http://cr.openjdk.java.net/~briangoetz/lambda/lambda-translation.html > > (this translation scheme is not required to be used by > compilers but in practice javac and ecj uses it and scalac > will use it too) > >> My suggestion says, "however the compiler chooses to >> represent the lambda, please apply the annotation to the >> implementing method, wherever it is". I think that the >> language spec at least implicitly forces the compiler to >> implement a lambda as a method /somewhere/ (as opposed to, >> say, javac inlining the expression, as Kotlin does with >> inline functions). > > see above. > >> >> The ability to attach introspectable metadata to a lambda >> expression that then travels with it to the implementing >> method -- wherever the compiler may choose to place it -- is >> extremely useful, and trumps, IMO, "purity" concerns re >> annotating expressions vs declarations, (esp. since we've >> already established that some expressions -- like cast and >> new -- are annotatable, and rightly so). > > cast and new are not annotatable, the type of a cast or a new are. > >> Without this, we miss something important Java gives us, >> which is the ability to attach metadata to a method body. I >> fully understand the distinction that the annotation is >> applied to the /declaration/ rather than the body itself, but >> that doesn't take away from the fact that a useful ability >> does not transfer to lambdas while it could be easily done. >> >> Ron > > R?mi > > >> >> On Sat, Jul 11, 2015 at 10:38 AM, Remi Forax >> > wrote: >> >> On 07/10/2015 11:41 PM, Ron Pressler wrote: >>> I see no problem with cast annotations not being >>> retrievable via straightforward reflection just as I see >>> no problem annotating the generated method (which should >>> always exist /somewhere/): in both cases, those >>> annotations are intended for use by tools that analyze >>> the entire project's source/bytecode (e.g. for >>> typechecking/instrumentation) rather than by specific >>> reflective queries. >>> >>> What bothers me is that a capability that was trivial >>> with inner classes is now lost when using lambdas. >> >> you can easily add a pattern recognizable by >> source/bytecode tools to do what you want, >> let suppose you have a static method like this: >> static T annotate(Class >> annotation, T lambda) { return lambda; } >> >> to annotate a lambda, your example can be written like this: >> foo(annotate(Special.class, () -> { ...})) >> >> in the bytecode it will be translated to something like this: >> ldc Special.class >> invokedynamic ()Runnable >> LambdaMetafactory + method handle ref on the lambda body >> invokestatic Bar.annotate(Class, Object)Object >> checkcast Runnable >> invokevirtual Baz.foo(Runnable)... >> >> which is easy to recognize. >> >>> >>> Ron Pressler >>> paralleluniverse.co >>> @puniverseco on Twitter >> >> R?mi >> >> >>> >>> On Fri, Jul 10, 2015 at 10:38 PM, Alex Buckley >>> >> > wrote: >>> >>> On 7/10/2015 2:31 AM, Maurizio Cimadamore wrote: >>> >>> On 10/07/15 09:43, Ron Pressler wrote: >>> >>> (Related to >>> http://mail.openjdk.java.net/pipermail/compiler-dev/2015-January/009220.html) >>> >>> It is useful to add an annotation to a >>> lambda expression -- as in >>> foo(@Special () -> { ...}) -- which will be >>> applied to the static >>> method implementing the lambda (not to the >>> interface the lambda >>> implements). This would allow >>> instrumentation tools operating on >>> method implementations to handle lambdas. So >>> far, AFAICT, this is not >>> possible (or am I wrong?). >>> >>> This is not possible, currently - as Remi said; >>> you have annotations on >>> declarations (i.e. variable, classes, methods) - >>> you have annotation on >>> types (i.e. type of a cast, type argument, etc.) >>> - but you don't have >>> annotations on _expressions_. >>> I take it that, under the current translation >>> scheme, there seems to be >>> a plausible description for what should happen >>> when the lambda is >>> annotated; but what if the lambda is turned into >>> a method reference (by >>> some IDE refactoring, or human intervention) ? >>> In that case it is not >>> guaranteed that there would still be a physical >>> place where to put the >>> annotation (unless we tell the compiler to >>> 'expand' all annotated method >>> references). >>> >>> >>> Yes, the problem with annotations-on-expressions is >>> not the language grammar but the storage of the >>> annotations in the class file. With no standard way >>> of translating source expressions to bytecode, there >>> can be no standard API for retrieving >>> annotations-on-expressions from bytecode. More >>> broadly, there has never been a standard reflective >>> API capable of inspecting method bodies. >>> >>> This explains why JSR 308 stayed away from >>> annotations-on-expressions -- we were not in a >>> position to let them be read back. It was already >>> bad enough that some annotations-on-types are stored >>> in the class file at the level of individual >>> bytecodes (e.g. an annotation on the type of a cast) >>> so are not retrievable by any standard reflective API. >>> >>> Alex >>> >>> >> >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ron at paralleluniverse.co Sun Jul 12 17:00:00 2015 From: ron at paralleluniverse.co (Ron Pressler) Date: Sun, 12 Jul 2015 20:00:00 +0300 Subject: Annotating lambda expressions In-Reply-To: <55A11C99.10309@univ-mlv.fr> References: <559F910B.7060004@oracle.com> <55A01F26.2050203@oracle.com> <55A0C7F8.1070107@univ-mlv.fr> <55A0E133.4030301@univ-mlv.fr> <55A11C99.10309@univ-mlv.fr> Message-ID: Yes, but only method definitions, not method bodies; that's a very big difference. On Sat, Jul 11, 2015 at 4:39 PM, Remi Forax wrote: > On 07/11/2015 03:02 PM, Ron Pressler wrote: > > > On Sat, Jul 11, 2015 at 3:59 PM, Ron Pressler > wrote: > >> Thanks for the bootstrap argument tip! Will use that as a workaround. >> >> > ... although it does require scanning the entire bytecode for the entire > project (to find the indy callsites) rather than just looking for annotated > methods... > > > ?? > finding annotated methods requires to scan the entire project too ? > > > > R > > > R > > > >> >> On Sat, Jul 11, 2015 at 12:26 PM, Remi Forax < >> forax at univ-mlv.fr> wrote: >> >>> On 07/11/2015 10:57 AM, Ron Pressler wrote: >>> >>> That's a very nice idea (in fact, we can do foo((@Special Runnable) >>> ()->{...}), as well and get ASM to invoke a nice visitInsnAnnotation >>> event), but it doesn't help, as I have no way of finding the method >>> implementing the lambda by analyzing the bytecode. >>> >>> >>> The method corresponding to the body of the lambda is one arguments of >>> the bootstrap method of invokedynamic. >>> see >>> http://cr.openjdk.java.net/~briangoetz/lambda/lambda-translation.html >>> (this translation scheme is not required to be used by compilers but in >>> practice javac and ecj uses it and scalac will use it too) >>> >>> My suggestion says, "however the compiler chooses to represent the >>> lambda, please apply the annotation to the implementing method, wherever it >>> is". I think that the language spec at least implicitly forces the compiler >>> to implement a lambda as a method *somewhere* (as opposed to, say, >>> javac inlining the expression, as Kotlin does with inline functions). >>> >>> >>> see above. >>> >>> >>> The ability to attach introspectable metadata to a lambda expression >>> that then travels with it to the implementing method -- wherever the >>> compiler may choose to place it -- is extremely useful, and trumps, IMO, >>> "purity" concerns re annotating expressions vs declarations, (esp. since >>> we've already established that some expressions -- like cast and new -- are >>> annotatable, and rightly so). >>> >>> >>> cast and new are not annotatable, the type of a cast or a new are. >>> >>> Without this, we miss something important Java gives us, which is the >>> ability to attach metadata to a method body. I fully understand the >>> distinction that the annotation is applied to the *declaration* rather >>> than the body itself, but that doesn't take away from the fact that a >>> useful ability does not transfer to lambdas while it could be easily done. >>> >>> Ron >>> >>> >>> R?mi >>> >>> >>> >>> On Sat, Jul 11, 2015 at 10:38 AM, Remi Forax < >>> forax at univ-mlv.fr> wrote: >>> >>>> On 07/10/2015 11:41 PM, Ron Pressler wrote: >>>> >>>> I see no problem with cast annotations not being retrievable via >>>> straightforward reflection just as I see no problem annotating the >>>> generated method (which should always exist *somewhere*): in both >>>> cases, those annotations are intended for use by tools that analyze the >>>> entire project's source/bytecode (e.g. for typechecking/instrumentation) >>>> rather than by specific reflective queries. >>>> >>>> What bothers me is that a capability that was trivial with inner >>>> classes is now lost when using lambdas. >>>> >>>> >>>> you can easily add a pattern recognizable by source/bytecode tools to >>>> do what you want, >>>> let suppose you have a static method like this: >>>> static T annotate(Class annotation, T >>>> lambda) { return lambda; } >>>> >>>> to annotate a lambda, your example can be written like this: >>>> foo(annotate(Special.class, () -> { ...})) >>>> >>>> in the bytecode it will be translated to something like this: >>>> ldc Special.class >>>> invokedynamic ()Runnable >>>> LambdaMetafactory + method handle ref on the lambda body >>>> invokestatic Bar.annotate(Class, Object)Object >>>> checkcast Runnable >>>> invokevirtual Baz.foo(Runnable)... >>>> >>>> which is easy to recognize. >>>> >>>> >>>> Ron Pressler >>>> paralleluniverse.co >>>> @puniverseco on Twitter >>>> >>>> >>>> R?mi >>>> >>>> >>>> >>>> On Fri, Jul 10, 2015 at 10:38 PM, Alex Buckley < >>>> alex.buckley at oracle.com> wrote: >>>> >>>>> On 7/10/2015 2:31 AM, Maurizio Cimadamore wrote: >>>>> >>>>>> On 10/07/15 09:43, Ron Pressler wrote: >>>>>> >>>>>>> (Related to >>>>>>> >>>>>>> >>>>>>> http://mail.openjdk.java.net/pipermail/compiler-dev/2015-January/009220.html >>>>>>> ) >>>>>>> >>>>>>> It is useful to add an annotation to a lambda expression -- as in >>>>>>> foo(@Special () -> { ...}) -- which will be applied to the static >>>>>>> method implementing the lambda (not to the interface the lambda >>>>>>> implements). This would allow instrumentation tools operating on >>>>>>> method implementations to handle lambdas. So far, AFAICT, this is not >>>>>>> possible (or am I wrong?). >>>>>>> >>>>>> This is not possible, currently - as Remi said; you have annotations >>>>>> on >>>>>> declarations (i.e. variable, classes, methods) - you have annotation >>>>>> on >>>>>> types (i.e. type of a cast, type argument, etc.) - but you don't have >>>>>> annotations on _expressions_. >>>>>> I take it that, under the current translation scheme, there seems to >>>>>> be >>>>>> a plausible description for what should happen when the lambda is >>>>>> annotated; but what if the lambda is turned into a method reference >>>>>> (by >>>>>> some IDE refactoring, or human intervention) ? In that case it is not >>>>>> guaranteed that there would still be a physical place where to put the >>>>>> annotation (unless we tell the compiler to 'expand' all annotated >>>>>> method >>>>>> references). >>>>>> >>>>> >>>>> Yes, the problem with annotations-on-expressions is not the language >>>>> grammar but the storage of the annotations in the class file. With no >>>>> standard way of translating source expressions to bytecode, there can be no >>>>> standard API for retrieving annotations-on-expressions from bytecode. More >>>>> broadly, there has never been a standard reflective API capable of >>>>> inspecting method bodies. >>>>> >>>>> This explains why JSR 308 stayed away from annotations-on-expressions >>>>> -- we were not in a position to let them be read back. It was already bad >>>>> enough that some annotations-on-types are stored in the class file at the >>>>> level of individual bytecodes (e.g. an annotation on the type of a cast) so >>>>> are not retrievable by any standard reflective API. >>>>> >>>>> Alex >>>>> >>>> >>>> >>>> >>> >>> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From forax at univ-mlv.fr Sun Jul 12 17:19:43 2015 From: forax at univ-mlv.fr (Remi Forax) Date: Sun, 12 Jul 2015 19:19:43 +0200 Subject: Annotating lambda expressions In-Reply-To: References: <559F910B.7060004@oracle.com> <55A01F26.2050203@oracle.com> <55A0C7F8.1070107@univ-mlv.fr> <55A0E133.4030301@univ-mlv.fr> <55A11C99.10309@univ-mlv.fr> Message-ID: <55A2A1AF.3040504@univ-mlv.fr> On 07/12/2015 07:00 PM, Ron Pressler wrote: > Yes, but only method definitions, not method bodies; that's a very big > difference. yes, very true, note that you only need to read the method body of methods that uses an invokedynamic. while there is no way to know if a method uses an invokedynamic, without reading all the instructions but in the class: - the constant pool will contain at least one CONSTANT_Invokedynamic_info entry, - the attributes of the class will contain a BootstrapMethods attribute. that said, both these info are not exposed publicly by ASM. R?mi [1] http://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.4.10 [2] http://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.7.23 > > > On Sat, Jul 11, 2015 at 4:39 PM, Remi Forax > wrote: > > On 07/11/2015 03:02 PM, Ron Pressler wrote: >> >> On Sat, Jul 11, 2015 at 3:59 PM, Ron Pressler >> > wrote: >> >> Thanks for the bootstrap argument tip! Will use that as a >> workaround. >> >> >> ... although it does require scanning the entire bytecode for the >> entire project (to find the indy callsites) rather than just >> looking for annotated methods... > > ?? > finding annotated methods requires to scan the entire project too ? > > >> >> R > > R > >> >> On Sat, Jul 11, 2015 at 12:26 PM, Remi Forax >> > wrote: >> >> On 07/11/2015 10:57 AM, Ron Pressler wrote: >>> That's a very nice idea (in fact, we can do >>> foo((@Special Runnable) ()->{...}), as well and get ASM >>> to invoke a nice visitInsnAnnotation event), but it >>> doesn't help, as I have no way of finding the method >>> implementing the lambda by analyzing the bytecode. >> >> The method corresponding to the body of the lambda is one >> arguments of the bootstrap method of invokedynamic. >> see >> http://cr.openjdk.java.net/~briangoetz/lambda/lambda-translation.html >> >> (this translation scheme is not required to be used by >> compilers but in practice javac and ecj uses it and >> scalac will use it too) >> >>> My suggestion says, "however the compiler chooses to >>> represent the lambda, please apply the annotation to the >>> implementing method, wherever it is". I think that the >>> language spec at least implicitly forces the compiler to >>> implement a lambda as a method /somewhere/ (as opposed >>> to, say, javac inlining the expression, as Kotlin does >>> with inline functions). >> >> see above. >> >>> >>> The ability to attach introspectable metadata to a >>> lambda expression that then travels with it to the >>> implementing method -- wherever the compiler may choose >>> to place it -- is extremely useful, and trumps, IMO, >>> "purity" concerns re annotating expressions vs >>> declarations, (esp. since we've already established that >>> some expressions -- like cast and new -- are >>> annotatable, and rightly so). >> >> cast and new are not annotatable, the type of a cast or a >> new are. >> >>> Without this, we miss something important Java gives us, >>> which is the ability to attach metadata to a method >>> body. I fully understand the distinction that the >>> annotation is applied to the /declaration/ rather than >>> the body itself, but that doesn't take away from the >>> fact that a useful ability does not transfer to lambdas >>> while it could be easily done. >>> >>> Ron >> >> R?mi >> >> >>> >>> On Sat, Jul 11, 2015 at 10:38 AM, Remi Forax >>> > wrote: >>> >>> On 07/10/2015 11:41 PM, Ron Pressler wrote: >>>> I see no problem with cast annotations not being >>>> retrievable via straightforward reflection just as >>>> I see no problem annotating the generated method >>>> (which should always exist /somewhere/): in both >>>> cases, those annotations are intended for use by >>>> tools that analyze the entire project's >>>> source/bytecode (e.g. for >>>> typechecking/instrumentation) rather than by >>>> specific reflective queries. >>>> >>>> What bothers me is that a capability that was >>>> trivial with inner classes is now lost when using >>>> lambdas. >>> >>> you can easily add a pattern recognizable by >>> source/bytecode tools to do what you want, >>> let suppose you have a static method like this: >>> static T annotate(Class >>> annotation, T lambda) { return lambda; } >>> >>> to annotate a lambda, your example can be written >>> like this: >>> foo(annotate(Special.class, () -> { ...})) >>> >>> in the bytecode it will be translated to something >>> like this: >>> ldc Special.class >>> invokedynamic ()Runnable >>> LambdaMetafactory + method handle ref on the lambda body >>> invokestatic Bar.annotate(Class, Object)Object >>> checkcast Runnable >>> invokevirtual Baz.foo(Runnable)... >>> >>> which is easy to recognize. >>> >>>> >>>> Ron Pressler >>>> paralleluniverse.co >>>> @puniverseco on >>>> Twitter >>> >>> R?mi >>> >>> >>>> >>>> On Fri, Jul 10, 2015 at 10:38 PM, Alex Buckley >>>> >>> > wrote: >>>> >>>> On 7/10/2015 2:31 AM, Maurizio Cimadamore wrote: >>>> >>>> On 10/07/15 09:43, Ron Pressler wrote: >>>> >>>> (Related to >>>> http://mail.openjdk.java.net/pipermail/compiler-dev/2015-January/009220.html) >>>> >>>> It is useful to add an annotation to a >>>> lambda expression -- as in >>>> foo(@Special () -> { ...}) -- which >>>> will be applied to the static >>>> method implementing the lambda (not to >>>> the interface the lambda >>>> implements). This would allow >>>> instrumentation tools operating on >>>> method implementations to handle >>>> lambdas. So far, AFAICT, this is not >>>> possible (or am I wrong?). >>>> >>>> This is not possible, currently - as Remi >>>> said; you have annotations on >>>> declarations (i.e. variable, classes, >>>> methods) - you have annotation on >>>> types (i.e. type of a cast, type argument, >>>> etc.) - but you don't have >>>> annotations on _expressions_. >>>> I take it that, under the current >>>> translation scheme, there seems to be >>>> a plausible description for what should >>>> happen when the lambda is >>>> annotated; but what if the lambda is turned >>>> into a method reference (by >>>> some IDE refactoring, or human >>>> intervention) ? In that case it is not >>>> guaranteed that there would still be a >>>> physical place where to put the >>>> annotation (unless we tell the compiler to >>>> 'expand' all annotated method >>>> references). >>>> >>>> >>>> Yes, the problem with >>>> annotations-on-expressions is not the language >>>> grammar but the storage of the annotations in >>>> the class file. With no standard way of >>>> translating source expressions to bytecode, >>>> there can be no standard API for retrieving >>>> annotations-on-expressions from bytecode. More >>>> broadly, there has never been a standard >>>> reflective API capable of inspecting method bodies. >>>> >>>> This explains why JSR 308 stayed away from >>>> annotations-on-expressions -- we were not in a >>>> position to let them be read back. It was >>>> already bad enough that some >>>> annotations-on-types are stored in the class >>>> file at the level of individual bytecodes (e.g. >>>> an annotation on the type of a cast) so are not >>>> retrievable by any standard reflective API. >>>> >>>> Alex >>>> >>>> >>> >>> >> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.v.stepanov at oracle.com Mon Jul 13 10:15:11 2015 From: alexander.v.stepanov at oracle.com (alexander stepanov) Date: Mon, 13 Jul 2015 13:15:11 +0300 Subject: RFR [9] 8080880: some docs cleanup for langtools In-Reply-To: <55A05A8B.4090005@oracle.com> References: <555E02BE.7020407@oracle.com> <555E03FB.40803@oracle.com> <555E2A4C.9010903@oracle.com> <555F0191.1010609@oracle.com> <55A05A8B.4090005@oracle.com> Message-ID: <55A38FAF.7010202@oracle.com> Thanks! On 11.07.2015 2:51, Jonathan Gibbons wrote: > Looks OK to me. > > -- Jon > > > On 05/22/2015 03:14 AM, alexander stepanov wrote: >> Hello Jonathan, >> >> Thanks. >> >> > It is not appropriate to do partial bulk update to the "not a >> supported API" >> >> The responding changes were reverted, please see >> http://cr.openjdk.java.net/~avstepan/8080880/webrev.01/ >> >> Regards, >> Alexander >> >> On 21.05.2015 21:56, Jonathan Gibbons wrote: >>> >>> On 05/21/2015 09:12 AM, alexander stepanov wrote: >>>> Hello, >>>> >>>> Could you please review the following fix >>>> http://cr.openjdk.java.net/~avstepan/8080880/webrev.00/ >>>> for >>>> https://bugs.openjdk.java.net/browse/JDK-8080880 >>>> >>>> Just some minor fix for docs, no other code touched. >>>> >>>> The affected packages should (probably) not be visible in the new >>>> modular system, but nevertheless... >>>> >>>> Thanks, >>>> Alexander >>> >>> Alexander, >>> >>> It is not appropriate to do partial bulk update to the "not a >>> supported API" >>> comment at the head of the internal files. It is strongly preferred >>> that this >>> part of the comment remain consistent across all source files, so >>> that it is >>> amenable to bulk updates, should we wish to do so. >>> >>> I would suggest you separate the otherwise useful work to fix >>> individual >>> minor doc issues from any work involving the standard "not a supported >>> API" comment at the head of each file. >>> >>> -- Jon >> > From rosti.bsd at gmail.com Mon Jul 13 18:09:58 2015 From: rosti.bsd at gmail.com (Rostislav Krasny) Date: Mon, 13 Jul 2015 21:09:58 +0300 Subject: Object with primitive type equality operator and Java Language specification Message-ID: Hi, javac of JDK7 successfully compiles a code like following: public class MainClass { private static Object getValue() { return 123; } public static void main(String[] args) { int num = 123; System.out.println(getValue() == num); } } I believe javac of other JDKs, starting from JDK5, also compile such a code without any problem. However Eclipse and its Java compiler fails to compile the Object with primitive type equality (getValue() == num) with an error "Incompatible operand types Object and int". There is a bug report about this in the Eclipse Bugzilla: https://bugs.eclipse.org/bugs/show_bug.cgi?id=405732 This bug was closed as NOT_ECLIPSE, i.e. developers of Eclipse Java compiler believe their compiler do it right. Please read the discussion in that bug report. Specifically Stephan Herrmann stated that Java Language specification disallows compiling such a code. So is it a bug in the javac compiler itself? Or maybe the JLS needs to be changed to conform to the long ago adopted javac compiler implementation? Or there is the JLS misunderstanding? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.buckley at oracle.com Mon Jul 13 18:32:47 2015 From: alex.buckley at oracle.com (Alex Buckley) Date: Mon, 13 Jul 2015 11:32:47 -0700 Subject: Object with primitive type equality operator and Java Language specification In-Reply-To: References: Message-ID: <55A4044F.2090104@oracle.com> The JLS section on == is rather stable, and has always rejected the code below. Object isn't convertible to a numeric type, and int isn't a reference type, and either type is boolean or Boolean, so no equality operator is available. javac historically had some problems with operators taking Object and a numeric type (or rather, taking expressions of those types), but I believe this was cleaned up a couple of years ago, and JDK 8 correctly rejects the code below. FWIW, if you cast num to Object to the right of == (legal by JLS8 5.5), then JDK 8 correctly compiles the code. The output is 'true' because of the flyweight pattern applied to boxed integer literals between -128 and 127. If you make both integer literals in the code be 128, then the output is 'false'. Alex On 7/13/2015 11:09 AM, Rostislav Krasny wrote: > Hi, > > javac of JDK7 successfully compiles a code like following: > > public class MainClass { > > private static Object getValue() { > return 123; > } > > public static void main(String[] args) { > int num = 123; > > System.out.println(getValue() == num); > } > } > > I believe javac of other JDKs, starting from JDK5, also compile such a > code without any problem. However Eclipse and its Java compiler fails to > compile the Object with primitive type equality (getValue() == num) with > an error "Incompatible operand types Object and int". There is a bug > report about this in the Eclipse Bugzilla: > > https://bugs.eclipse.org/bugs/show_bug.cgi?id=405732 > > This bug was closed as NOT_ECLIPSE, i.e. developers of Eclipse Java > compiler believe their compiler do it right. Please read the discussion > in that bug report.Specifically Stephan Herrmann stated that Java > Language specification disallows compiling such a code. So is it a bug > in the javac compiler itself? Or maybe the JLS needs to be changed to > conform to the long ago adopted javac compiler implementation? Or there > is the JLS misunderstanding? > > Thanks From maurizio.cimadamore at oracle.com Mon Jul 13 20:37:55 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 13 Jul 2015 21:37:55 +0100 Subject: Object with primitive type equality operator and Java Language specification In-Reply-To: <55A4044F.2090104@oracle.com> References: <55A4044F.2090104@oracle.com> Message-ID: <55A421A3.7020105@oracle.com> I believe you are staring at this: https://bugs.openjdk.java.net/browse/JDK-8013357 Which was fixed in JDK 8 - long story short, javac compiler has historically performed operator checks by means of pseudo operator overloading (i.e. the internal compiler symbol table had entries for the various kind of allowed ''overloaded' operators, and the overload logic would do the rest). Needless to say, this behavior was not 100% conformant w.r.t. JLS and it had few bugs as in this case. We worked around the problem in JDK 8 by applying a local fix which got rid of the issue - in JDK 9 we completely removed the root cause: https://bugs.openjdk.java.net/browse/JDK-8071241 So that, moving forward, we hope that these kind of discrepancies should be a thing of the past (finger crossed :-)). Cheers Maurizio On 13/07/15 19:32, Alex Buckley wrote: > The JLS section on == is rather stable, and has always rejected the > code below. Object isn't convertible to a numeric type, and int isn't > a reference type, and either type is boolean or Boolean, so no > equality operator is available. > > javac historically had some problems with operators taking Object and > a numeric type (or rather, taking expressions of those types), but I > believe this was cleaned up a couple of years ago, and JDK 8 correctly > rejects the code below. > > FWIW, if you cast num to Object to the right of == (legal by JLS8 > 5.5), then JDK 8 correctly compiles the code. The output is 'true' > because of the flyweight pattern applied to boxed integer literals > between -128 and 127. If you make both integer literals in the code be > 128, then the output is 'false'. > > Alex > > On 7/13/2015 11:09 AM, Rostislav Krasny wrote: >> Hi, >> >> javac of JDK7 successfully compiles a code like following: >> >> public class MainClass { >> >> private static Object getValue() { >> return 123; >> } >> >> public static void main(String[] args) { >> int num = 123; >> >> System.out.println(getValue() == num); >> } >> } >> >> I believe javac of other JDKs, starting from JDK5, also compile such a >> code without any problem. However Eclipse and its Java compiler fails to >> compile the Object with primitive type equality (getValue() == num) with >> an error "Incompatible operand types Object and int". There is a bug >> report about this in the Eclipse Bugzilla: >> >> https://bugs.eclipse.org/bugs/show_bug.cgi?id=405732 >> >> This bug was closed as NOT_ECLIPSE, i.e. developers of Eclipse Java >> compiler believe their compiler do it right. Please read the discussion >> in that bug report.Specifically Stephan Herrmann stated that Java >> Language specification disallows compiling such a code. So is it a bug >> in the javac compiler itself? Or maybe the JLS needs to be changed to >> conform to the long ago adopted javac compiler implementation? Or there >> is the JLS misunderstanding? >> >> Thanks From alex.buckley at oracle.com Mon Jul 13 21:03:25 2015 From: alex.buckley at oracle.com (Alex Buckley) Date: Mon, 13 Jul 2015 14:03:25 -0700 Subject: Object with primitive type equality operator and Java Language specification In-Reply-To: <55A421A3.7020105@oracle.com> References: <55A4044F.2090104@oracle.com> <55A421A3.7020105@oracle.com> Message-ID: <55A4279D.7010600@oracle.com> Thanks for the info and URLs. Clarification to my mail: it should have read "and NEITHER type is boolean or Boolean" rather than "and EITHER type is boolean or Boolean". Alex On 7/13/2015 1:37 PM, Maurizio Cimadamore wrote: > I believe you are staring at this: > > https://bugs.openjdk.java.net/browse/JDK-8013357 > > Which was fixed in JDK 8 - long story short, javac compiler has > historically performed operator checks by means of pseudo operator > overloading (i.e. the internal compiler symbol table had entries for the > various kind of allowed ''overloaded' operators, and the overload logic > would do the rest). Needless to say, this behavior was not 100% > conformant w.r.t. JLS and it had few bugs as in this case. > > We worked around the problem in JDK 8 by applying a local fix which got > rid of the issue - in JDK 9 we completely removed the root cause: > > https://bugs.openjdk.java.net/browse/JDK-8071241 > > So that, moving forward, we hope that these kind of discrepancies should > be a thing of the past (finger crossed :-)). > > Cheers > Maurizio > > On 13/07/15 19:32, Alex Buckley wrote: >> The JLS section on == is rather stable, and has always rejected the >> code below. Object isn't convertible to a numeric type, and int isn't >> a reference type, and either type is boolean or Boolean, so no >> equality operator is available. >> >> javac historically had some problems with operators taking Object and >> a numeric type (or rather, taking expressions of those types), but I >> believe this was cleaned up a couple of years ago, and JDK 8 correctly >> rejects the code below. >> >> FWIW, if you cast num to Object to the right of == (legal by JLS8 >> 5.5), then JDK 8 correctly compiles the code. The output is 'true' >> because of the flyweight pattern applied to boxed integer literals >> between -128 and 127. If you make both integer literals in the code be >> 128, then the output is 'false'. >> >> Alex >> >> On 7/13/2015 11:09 AM, Rostislav Krasny wrote: >>> Hi, >>> >>> javac of JDK7 successfully compiles a code like following: >>> >>> public class MainClass { >>> >>> private static Object getValue() { >>> return 123; >>> } >>> >>> public static void main(String[] args) { >>> int num = 123; >>> >>> System.out.println(getValue() == num); >>> } >>> } >>> >>> I believe javac of other JDKs, starting from JDK5, also compile such a >>> code without any problem. However Eclipse and its Java compiler fails to >>> compile the Object with primitive type equality (getValue() == num) with >>> an error "Incompatible operand types Object and int". There is a bug >>> report about this in the Eclipse Bugzilla: >>> >>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=405732 >>> >>> This bug was closed as NOT_ECLIPSE, i.e. developers of Eclipse Java >>> compiler believe their compiler do it right. Please read the discussion >>> in that bug report.Specifically Stephan Herrmann stated that Java >>> Language specification disallows compiling such a code. So is it a bug >>> in the javac compiler itself? Or maybe the JLS needs to be changed to >>> conform to the long ago adopted javac compiler implementation? Or there >>> is the JLS misunderstanding? >>> >>> Thanks > From cushon at google.com Fri Jul 17 22:13:52 2015 From: cushon at google.com (Liam Miller-Cushon) Date: Fri, 17 Jul 2015 15:13:52 -0700 Subject: JDK-7101822 causes spurious 'annotation.type.not.applicable' error Message-ID: Hi, I think I found a regression introduced by the fix for JDK-7101822 [1][2] [1] https://bugs.openjdk.java.net/browse/JDK-7101822 [2] http://hg.openjdk.java.net/jdk9/dev/langtools/rev/9d2192f36e53 The full repro is below. Symbol completion failures during import processing are causing spurious 'annotation.type.not.applicable' errors. In the repro, Check.annotationApplicable ends up getting called after a completion failure has occurred during import processing. The type of 'Foo' is recorded as the error type, so it doesn't match any of the accepted target kinds for the @Retention annotation, so 'annotation.type.not.applicable' is reported. The obvious fix would be to handle error types in Check.annotationApplicable: diff -r 9d2192f36e53 -r 2cbbecfacf02 src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Check.java --- a/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Check.java Wed Dec 03 13:46:12 2014 +0100 +++ b/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Check.java Fri Jul 17 15:06:35 2015 -0700 @@ -3034,6 +3034,9 @@ targets[i] = e.value.name; } } + if (s.kind == ERR) { + return true; // recovery + } for (Name target : targets) { if (target == names.TYPE) { if (s.kind == TYP) return true; } ... but it might be better to change how errors are handled during import processing. Here's the repro: === p/Bar.java === package p; public class Bar extends Super {} === p/Super.java === package p; public class Super { public static int CONST = 42; } === Test.java === import java.lang.annotation.RetentionPolicy; import java.lang.annotation.Retention; import static p.Bar.CONST; @Retention(RetentionPolicy.RUNTIME) @interface Foo {} === $ JAVAC=~/src/langtools9/dist/bootstrap/bin/javac $ $JAVAC p/Bar.java p/Super.java # Super is deliberately left out of lib.jar to trigger a completion failure $ jar cf lib.jar p/Bar.class $ $JAVAC -cp lib.jar -sourcepath : Test.java Test.java:6: error: annotation type not applicable to this kind of declaration @Retention(RetentionPolicy.RUNTIME) ^ 1 error The error should be: Test.java:4: error: cannot access Super import static p.Bar.CONST; ^ class file for p.Super not found 1 error Thanks, Liam -------------- next part -------------- An HTML attachment was scrubbed... URL: From jan.lahoda at oracle.com Mon Jul 20 07:27:31 2015 From: jan.lahoda at oracle.com (Jan Lahoda) Date: Mon, 20 Jul 2015 09:27:31 +0200 Subject: JDK-7101822 causes spurious 'annotation.type.not.applicable' error In-Reply-To: References: Message-ID: <55ACA2E3.7040804@oracle.com> Hi Liam, Thanks for the report. I've filled: https://bugs.openjdk.java.net/browse/JDK-8131915 The problem seems to be in an inappropriate handling of the CompletionFailure that is triggered due to the missing class. I think it needs to be caught and a proper error reported for it. Thanks, Jan On 18.7.2015 00:13, Liam Miller-Cushon wrote: > Hi, > > I think I found a regression introduced by the fix for JDK-7101822 [1][2] > > [1] https://bugs.openjdk.java.net/browse/JDK-7101822 > [2] http://hg.openjdk.java.net/jdk9/dev/langtools/rev/9d2192f36e53 > > The full repro is below. Symbol completion failures during import > processing are causing spurious 'annotation.type.not.applicable' errors. > In the repro, Check.annotationApplicable ends up getting called after a > completion failure has occurred during import processing. The type of > 'Foo' is recorded as the error type, so it doesn't match any of the > accepted target kinds for the @Retention annotation, so > 'annotation.type.not.applicable' is reported. > > The obvious fix would be to handle error types in > Check.annotationApplicable: > > diff -r 9d2192f36e53 -r 2cbbecfacf02 > src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Check.java > --- > a/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Check.javaWed > Dec 03 13:46:12 2014 +0100 > +++ > b/src/jdk.compiler/share/classes/com/sun/tools/javac/comp/Check.javaFri > Jul 17 15:06:35 2015 -0700 > @@ -3034,6 +3034,9 @@ > targets[i] = e.value.name ; > } > } > + if (s.kind == ERR) { > + return true; // recovery > + } > for (Name target : targets) { > if (target == names.TYPE) > { if (s.kind == TYP) return true; } > > > ... but it might be better to change how errors are handled during > import processing. > > Here's the repro: > > === p/Bar.java === > package p; > > public class Bar extends Super {} > > === p/Super.java === > package p; > > public class Super { > public static int CONST = 42; > } > > === Test.java === > import java.lang.annotation.RetentionPolicy; > import java.lang.annotation.Retention; > > import static p.Bar.CONST; > > @Retention(RetentionPolicy.RUNTIME) > @interface Foo {} > === > > $ JAVAC=~/src/langtools9/dist/bootstrap/bin/javac > $ $JAVAC p/Bar.java p/Super.java > # Super is deliberately left out of lib.jar to trigger a completion failure > $ jar cf lib.jar p/Bar.class > $ $JAVAC -cp lib.jar -sourcepath : Test.java > Test.java:6: error: annotation type not applicable to this kind of > declaration > @Retention(RetentionPolicy.RUNTIME) > ^ > 1 error > > The error should be: > > Test.java:4: error: cannot access Super > import static p.Bar.CONST; > ^ > class file for p.Super not found > 1 error > > > Thanks, > Liam From cushon at google.com Mon Jul 20 18:29:10 2015 From: cushon at google.com (Liam Miller-Cushon) Date: Mon, 20 Jul 2015 11:29:10 -0700 Subject: JDK-7101822 causes spurious 'annotation.type.not.applicable' error In-Reply-To: <55ACA2E3.7040804@oracle.com> References: <55ACA2E3.7040804@oracle.com> Message-ID: Thanks Jan, On Mon, Jul 20, 2015 at 12:27 AM, Jan Lahoda wrote: > The problem seems to be in an inappropriate handling of the > CompletionFailure that is triggered due to the missing class. I think it > needs to be caught and a proper error reported for it. That sounds good to me. I'm now working around it by catching and reporting the CompletionFailure in checkImportsUnique. -------------- next part -------------- An HTML attachment was scrubbed... URL: From georgiy.rakov at oracle.com Wed Jul 22 14:03:17 2015 From: georgiy.rakov at oracle.com (Georgiy Rakov) Date: Wed, 22 Jul 2015 17:03:17 +0300 Subject: Compilation fails because anonymous class constructor formal parameter has inaccessible type Message-ID: <55AFA2A5.9010504@oracle.com> Hello, let's consider following example: class Test18_2 {private static class MyClass {}public static MyClass v1 = new MyClass();Test18_2(MyClass mc) {}}public class Test18 {public static void main(String[] args) {new Test18_2(Test18_2.v1){};}} this example fails to compile on JDK9b73 with following error: Error:(9, 34) java: Test18_2.MyClass has private access in Test18_2 However according to my understanding compilation should succeed as per assertion jls-15.9.5.1-200: Note that it is possible for the signature of the anonymous constructor to refer to an inaccessible type (for example, if such a type occurred in the signature of the superclass constructor |cs|). This does not, in itself, cause any errors at either compile-time or run-time. It seems to be a javac bug. Could you please tell if you agree. Thanks, Georgiy. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- class Test18_2 { private static class MyClass {} public static MyClass v1 = new MyClass(); Test18_2(MyClass mc) {} } public class Test18 { public static void main(String[] args) { new Test18_2(Test18_2.v1){}; } } From srikanth.adayapalam at oracle.com Wed Jul 22 14:55:20 2015 From: srikanth.adayapalam at oracle.com (Srikanth) Date: Wed, 22 Jul 2015 20:25:20 +0530 Subject: Compilation fails because anonymous class constructor formal parameter has inaccessible type In-Reply-To: <55AFA2A5.9010504@oracle.com> References: <55AFA2A5.9010504@oracle.com> Message-ID: <55AFAED8.7020508@oracle.com> Hi Victor, Thanks, is this the same as https://bugs.openjdk.java.net/browse/JDK-8075799 ? Srikanth On Wednesday 22 July 2015 07:33 PM, Georgiy Rakov wrote: > Hello, > > let's consider following example: > > classTest18_2 { > private static classMyClass {} > public staticMyClassv1=newMyClass(); > Test18_2(MyClass mc) {} > } > > public classTest18 { > public static voidmain(String[] args) { > newTest18_2(Test18_2.v1){}; > } > } > > > this example fails to compile on JDK9b73 with following error: > > Error:(9, 34) java: Test18_2.MyClass has private access in Test18_2 > > However according to my understanding compilation should succeed as > per assertion jls-15.9.5.1-200: > > Note that it is possible for the signature of the anonymous > constructor to refer to an inaccessible type (for example, if such > a type occurred in the signature of the superclass constructor > |cs|). This does not, in itself, cause any errors at either > compile-time or run-time. > > It seems to be a javac bug. Could you please tell if you agree. > > Thanks, > Georgiy. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pawel.veselov at gmail.com Wed Jul 22 17:12:10 2015 From: pawel.veselov at gmail.com (Pawel) Date: Wed, 22 Jul 2015 10:12:10 -0700 Subject: Compilation fails because anonymous class constructor formal parameter has inaccessible type In-Reply-To: <55AFA2A5.9010504@oracle.com> References: <55AFA2A5.9010504@oracle.com> Message-ID: > On Jul 22, 2015, at 7:03 AM, Georgiy Rakov wrote: > > Hello, > > let's consider following example: > class Test18_2 { > private static class MyClass {} > public static MyClass v1 = new MyClass(); > Test18_2(MyClass mc) {} > } > > public class Test18 { > public static void main(String[] args) { > new Test18_2(Test18_2.v1){}; > } > } > > this example fails to compile on JDK9b73 with following error: > Error:(9, 34) java: Test18_2.MyClass has private access in Test18_2 > > However according to my understanding compilation should succeed as per assertion jls-15.9.5.1-200: > Note that it is possible for the signature of the anonymous constructor to refer to an inaccessible type (for example, if such a type occurred in the signature of the superclass constructor cs). This does not, in itself, cause any errors at either compile-time or run-time. > It seems to be a javac bug. Could you please tell if you agree. This seems to say that it is the signature of a constructor that shouldn't cause any errors. I believe the error is coming from the access of Test18.v1, as its type can not be accessed. That would fail outside of being used as a constructor argument, so it shouldn't be allowed if used as one either. > > Thanks, > Georgiy. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From georgiy.rakov at oracle.com Wed Jul 22 17:14:25 2015 From: georgiy.rakov at oracle.com (Georgiy Rakov) Date: Wed, 22 Jul 2015 20:14:25 +0300 Subject: Please reply to my JDK-8059640 issue comment In-Reply-To: References: <559AB728.80105@oracle.com> Message-ID: <55AFCF71.5050101@oracle.com> On 09.07.2015 2:23, Dan Smith wrote: >> On Jul 6, 2015, at 10:13 AM, Georgiy Rakov > > wrote: >> >> Hello, >> >> could you please reply to my following comment >> >> I made to JDK-8059640; quoting it: >> >> Suggested spec change states that 15.13.3 will contain following >> assertion: >> >> LinkageError may occur due to inconsistencies between compile >> time and run time (in the referenced method, the functional >> interface, or the argument types of either). >> >> The details are provided in parenthesis: "in the referenced >> method, the functional interface, or the argument types of >> either". It seems that these details don't extend to following >> case specified in jls-15.12.4.3-120: >> >> If the invocation mode is interface, then the implementation must >> also check that the target reference type still implements the >> specified interface. If the target reference type does not still >> implement the interface, then an IncompatibleClassChangeError occurs. >> >> I believe they should. >> >> Namely, could you please tell if you agree that spec change suggested >> in the issue description doesn't extend to the case specified in >> jls-15.12.4.3-120. If you agree I would appreciate greatly if you >> could modify the suggested spec change so that it extends to >> jls-15.12.4.3-120 case. >> > > I think these are two different kinds of errors. > > JDK-8059640 is about BootstrapMethodErrors that can occur upon > *evaluation of* a method reference expression, but that aren't > specified to occur. > > The check you describe is an ICCE that can happen upon *invocation of* > the method of an object derived from a method reference expression. > > Please correct me if you're seeing different behavior, but my > expectation is that the invocation will behave exactly as specified: > per 15.13.3, the body of the method is via an invocation that proceeds > according to 15.12.4.3; per 15.12.4.3, the ICCE can occur. > > Example: > > interface I { int x(); } > interface Func { int apply(I arg); } > > Func f2 = I::x; > System.out.println(f2); > // no errors yet > System.out.println(f2.apply(new C())); > // ICCE if C was originally declared to implement I, but no longer does > > So I don't think there's a bug here, either in the spec or the > implementation. In the example you provided inconsistency occurs at the point of *invocation of* the method of an object derived from a method reference expression. But we can consider another example which contains inconsistency occurring at the point of *evaluation of* a method reference expression: interface MethodSupplier { void m(int a);} interface MyFunctionalInterface {int invokeMethodReference(int a);} class MethodSupplierImpl implements MethodSupplier { @Override public void m(int a) {} } MyFunctionalInterface instance = null; MethodSupplier ms = new MethodSupplierImpl(); instance = ms::m; //inconsistency occurs here instance.invokeMethodReference(1); Currently the behavior is Ok, that is provided MethodSupplierImpl is recompiled without implementing MethodSupplier, ICCE is thrown when executing "instance.invokeMethodReference(1)". But potentially it looks possible for the error to be thrown during executing "instance = ms::m" and suggested spec change doesn't encompass this case. So I wonder if it's worth encompassing such case for the suggested spec change. Could you please also tell if I understand correctly that suggested spec change specifies the possibility of throwing LinkageError during *evaluation of* method reference and not during *invocation of* the method of an object derived from a method reference expression. Thank you, Georgiy. > > ?Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From georgiy.rakov at oracle.com Thu Jul 23 13:47:46 2015 From: georgiy.rakov at oracle.com (Georgiy Rakov) Date: Thu, 23 Jul 2015 16:47:46 +0300 Subject: Compilation fails because anonymous class constructor formal parameter has inaccessible type In-Reply-To: References: <55AFA2A5.9010504@oracle.com> Message-ID: <55B0F082.2070600@oracle.com> On 22.07.2015 20:12, Pawel wrote: > > > On Jul 22, 2015, at 7:03 AM, Georgiy Rakov > wrote: > >> Hello, >> >> let's consider following example: >> >> class Test18_2 {private static class MyClass {}public static >> MyClass v1 = new MyClass();Test18_2(MyClass mc) {}}public class >> Test18 {public static void main(String[] args) {new >> Test18_2(Test18_2.v1){};}} >> >> >> this example fails to compile on JDK9b73 with following error: >> >> Error:(9, 34) java: Test18_2.MyClass has private access in Test18_2 >> >> However according to my understanding compilation should succeed as >> per assertion jls-15.9.5.1-200: >> >> Note that it is possible for the signature of the anonymous >> constructor to refer to an inaccessible type (for example, if >> such a type occurred in the signature of the superclass >> constructor |cs|). This does not, in itself, cause any errors at >> either compile-time or run-time. >> >> It seems to be a javac bug. Could you please tell if you agree. > > This seems to say that it is the signature of a constructor that > shouldn't cause any errors. I believe the error is coming from the > access of Test18.v1, as its type can not be accessed. That would fail > outside of being used as a constructor argument, so it shouldn't be > allowed if used as one either. According to my understanding Test18_2.v1 is supposed to be used outside too despite the fact that the type of it is inaccessible. The error I'm talking about seems to be caused by the fact that the class being instantiated is anonymous. Provided it's not anonymous, compilation succeeds! Namely following code compiles successfully (just removing curly brackets from instance creation expression): class Test18_2 {private static class MyClass {}public static MyClass v1 = new MyClass();Test18_2(MyClass mc) {}}public class Test18 {public static void main(String[] args) {new Test18_2(Test18_2.v1);}} So I believe that this is anonymous constructor signature which causes this error. Please note that anonymous class itself doesn't have access to MyClass, for instance declaring any method having MyClass in its signature within anonymous class would naturally cause compilation error, for instance: new Test18_2(Test18_2.v1) { private MyClass foo(){ return null; } } Thus anonymous class constructor which is declared implicitly could have the same reason to cause compilation error if it has any inaccessible type within its signature. According to my understanding it's the assertion I pointed out above which allows anonymous classes to compile successfully in such cases. It seems that javac merely do not take this assertion into account. As I said above anonymous constructor is declared implicitly, that is it cannot be seen in the code, instead it's generated by javac automatically during compilation if I understand it correctly. And it "copies" formal parameters from corresponding super type constructor. This is described in JLS 15.9.5.1. Thanks, Georgiy. > >> >> Thanks, >> Georgiy. >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From georgiy.rakov at oracle.com Thu Jul 23 13:58:22 2015 From: georgiy.rakov at oracle.com (Georgiy Rakov) Date: Thu, 23 Jul 2015 16:58:22 +0300 Subject: Compilation fails because anonymous class constructor formal parameter has inaccessible type In-Reply-To: <55AFAED8.7020508@oracle.com> References: <55AFA2A5.9010504@oracle.com> <55AFAED8.7020508@oracle.com> Message-ID: <55B0F2FE.2030108@oracle.com> Hi Srikanth, I'm not Victor :) but anyway I believe yes, this is the same. BTW: Test18_2.v1 which I passed to the constructor in the example here seems to be nonrelevant to the problem. If null is passed compilation still fail: class Test18_2 {private static class MyClass {}Test18_2(MyClass mc) {}}public class Test18 {public static void main(String[] args) {new Test18_2(null){};}} Thanks, Georgiy. On 22.07.2015 17:55, Srikanth wrote: > Hi Victor, > > Thanks, is this the same as > https://bugs.openjdk.java.net/browse/JDK-8075799 ? > > Srikanth > > On Wednesday 22 July 2015 07:33 PM, Georgiy Rakov wrote: >> Hello, >> >> let's consider following example: >> >> class Test18_2 {private static class MyClass {}public static >> MyClass v1 = new MyClass();Test18_2(MyClass mc) {}}public class >> Test18 {public static void main(String[] args) {new >> Test18_2(Test18_2.v1){};}} >> >> >> this example fails to compile on JDK9b73 with following error: >> >> Error:(9, 34) java: Test18_2.MyClass has private access in Test18_2 >> >> However according to my understanding compilation should succeed as >> per assertion jls-15.9.5.1-200: >> >> Note that it is possible for the signature of the anonymous >> constructor to refer to an inaccessible type (for example, if >> such a type occurred in the signature of the superclass >> constructor |cs|). This does not, in itself, cause any errors at >> either compile-time or run-time. >> >> It seems to be a javac bug. Could you please tell if you agree. >> >> Thanks, >> Georgiy. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From srikanth.adayapalam at oracle.com Thu Jul 23 14:13:50 2015 From: srikanth.adayapalam at oracle.com (Srikanth) Date: Thu, 23 Jul 2015 19:43:50 +0530 Subject: Compilation fails because anonymous class constructor formal parameter has inaccessible type In-Reply-To: <55B0F2FE.2030108@oracle.com> References: <55AFA2A5.9010504@oracle.com> <55AFAED8.7020508@oracle.com> <55B0F2FE.2030108@oracle.com> Message-ID: <55B0F69E.5020304@oracle.com> On Thursday 23 July 2015 07:28 PM, Georgiy Rakov wrote: > Hi Srikanth, > > I'm not Victor :) Are you sure ? :) I apologize. Thanks! Srikanth -------------- next part -------------- An HTML attachment was scrubbed... URL: From georgiy.rakov at oracle.com Thu Jul 23 15:34:17 2015 From: georgiy.rakov at oracle.com (Georgiy Rakov) Date: Thu, 23 Jul 2015 18:34:17 +0300 Subject: Compilation fails because anonymous class constructor formal parameter has inaccessible type In-Reply-To: <55B0F69E.5020304@oracle.com> References: <55AFA2A5.9010504@oracle.com> <55AFAED8.7020508@oracle.com> <55B0F2FE.2030108@oracle.com> <55B0F69E.5020304@oracle.com> Message-ID: <55B10979.50003@oracle.com> On 23.07.2015 17:13, Srikanth wrote: > > > On Thursday 23 July 2015 07:28 PM, Georgiy Rakov wrote: >> Hi Srikanth, >> >> I'm not Victor :) > > Are you sure ? :) It seems I am ) > > I apologize. > It's all right, never mind. > Thanks! > Srikanth > Regards, Georgiy. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jan.lahoda at oracle.com Fri Jul 24 14:03:01 2015 From: jan.lahoda at oracle.com (Jan Lahoda) Date: Fri, 24 Jul 2015 16:03:01 +0200 Subject: RFR JDK-8131915: CompletionFailure during import listing crashes javac Message-ID: <55B24595.80900@oracle.com> Hello, This is a fix for: https://bugs.openjdk.java.net/browse/JDK-8131915 which Liam reported recently. The problem is that if a CompletionFailure happens while listing import content (e.g. due to a missing supertype), it is not handled properly and causes a javac crash. The proposed fix is to catch the CompletionFailure and handle it properly (log an error). A webrev with the fix is here: http://cr.openjdk.java.net/~jlahoda/8131915/webrev.00/ Any feedback is welcome, Thanks! Jan From brian.goetz at oracle.com Sun Jul 26 01:44:33 2015 From: brian.goetz at oracle.com (Brian Goetz) Date: Sat, 25 Jul 2015 21:44:33 -0400 Subject: Annotating lambda expressions In-Reply-To: <55A0412B.7030101@oracle.com> References: <559F910B.7060004@oracle.com> <55A01F26.2050203@oracle.com> <55A0412B.7030101@oracle.com> Message-ID: <55B43B81.30808@oracle.com> > OK, but it's precisely the fact that inner classes are declarations that > allows them to be annotated, because for declarations we specify the > binary name, membership, and other artifacts of translation. No such > spec for the translation of expressions (or statements) -- and I think > it's generally regarded as a feature-not-a-bug that compilers have great > freedom in translating lambda expressions. Quite so. Compilers should have freedom how they translate lambda expressions, which is why this is outside the scope of the specification. This means that there's not a *reliable* place in the classfile to put these annotations. You've suggested a *convenient* place to put them -- which works as long as we don't want to change our translation strategy (and are willing to mandate it for other compilers.) I get why that seems reasonable -- half a loaf seems better than none when you're hungry -- but let's follow it through. What happens when we *do* want to change the translation strategy? Do you think people won't then howl because we "broke" their programs even though they rely on a completely unspecified translation strategy? They'll say "but there was no other way to do it", and proceed to complain about how we gratuitously broke their programs. So if we were to do this, we'd essentially be locking ourselves into the current translation strategy forever. Unless we're going to go all the way (full reflection over all program elements, not just declarations), it's far better to leave the line where it is (can annotate declarations and type uses, but not expressions), than to blur the line in this way for the sake of the use cases at the margin. (And remember, when you move the margin, new marginal use cases will come into view.) From georgiy.rakov at oracle.com Mon Jul 27 15:21:19 2015 From: georgiy.rakov at oracle.com (Georgiy Rakov) Date: Mon, 27 Jul 2015 18:21:19 +0300 Subject: Please reply to my JDK-8059640 issue comment In-Reply-To: References: <559AB728.80105@oracle.com> Message-ID: <55B64C6F.1060006@oracle.com> Just a friendly reminder, Thanks, Georgiy. On 09.07.2015 2:23, Dan Smith wrote: >> On Jul 6, 2015, at 10:13 AM, Georgiy Rakov >> wrote: >> >> Hello, >> >> could you please reply to my following comment >> >> I made to JDK-8059640; quoting it: >> >> Suggested spec change states that 15.13.3 will contain following >> assertion: >> >> LinkageError may occur due to inconsistencies between compile >> time and run time (in the referenced method, the functional >> interface, or the argument types of either). >> >> The details are provided in parenthesis: "in the referenced >> method, the functional interface, or the argument types of >> either". It seems that these details don't extend to following >> case specified in jls-15.12.4.3-120: >> >> If the invocation mode is interface, then the implementation must >> also check that the target reference type still implements the >> specified interface. If the target reference type does not still >> implement the interface, then an IncompatibleClassChangeError occurs. >> >> I believe they should. >> >> Namely, could you please tell if you agree that spec change suggested >> in the issue description doesn't extend to the case specified in >> jls-15.12.4.3-120. If you agree I would appreciate greatly if you >> could modify the suggested spec change so that it extends to >> jls-15.12.4.3-120 case. >> > > I think these are two different kinds of errors. > > JDK-8059640 is about BootstrapMethodErrors that can occur upon > *evaluation of* a method reference expression, but that aren't > specified to occur. > > The check you describe is an ICCE that can happen upon *invocation of* > the method of an object derived from a method reference expression. > > Please correct me if you're seeing different behavior, but my > expectation is that the invocation will behave exactly as specified: > per 15.13.3, the body of the method is via an invocation that proceeds > according to 15.12.4.3; per 15.12.4.3, the ICCE can occur. > > Example: > > interface I { int x(); } > interface Func { int apply(I arg); } > > Func f2 = I::x; > System.out.println(f2); > // no errors yet > System.out.println(f2.apply(new C())); > // ICCE if C was originally declared to implement I, but no longer does > > So I don't think there's a bug here, either in the spec or the > implementation. In the example you provided inconsistency occurs at the point of *invocation of* the method of an object derived from a method reference expression. But we can consider another example which contains inconsistency occurring at the point of *evaluation of* a method reference expression: interface MethodSupplier { void m(int a);} interface MyFunctionalInterface {int invokeMethodReference(int a);} class MethodSupplierImpl implements MethodSupplier { @Override public void m(int a) {} } MyFunctionalInterface instance = null; MethodSupplier ms = new MethodSupplierImpl(); instance = ms::m; //inconsistency occurs here instance.invokeMethodReference(1); Currently the behavior is Ok, that is provided MethodSupplierImpl is recompiled without implementing MethodSupplier, ICCE is thrown when executing "instance.invokeMethodReference(1)". But potentially it looks possible for the error to be thrown during executing "instance = ms::m" and suggested spec change doesn't encompass this case. So I wonder if it's worth encompassing such case for the suggested spec change. Could you please also tell if I understand correctly that suggested spec change specifies the possibility of throwing LinkageError during *evaluation of* method reference and not during *invocation of* the method of an object derived from a method reference expression. Thank you, Georgiy. > > ?Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: From cushon at google.com Wed Jul 29 17:30:25 2015 From: cushon at google.com (Liam Miller-Cushon) Date: Wed, 29 Jul 2015 10:30:25 -0700 Subject: Redundant bridge methods in nested classes Message-ID: I've been trying to get a better understanding of bridge methods, and noticed that javac seems to emit different bridge methods depending on the order of sources passed to the compilation. As far as I can tell this occurs because TransTypes.translateClass calls super.visitClassDef before addBridges. The super implementation of visitClassDef processes all declarations (including nested classes), so nested classes get translated before their enclosing classes. If a nested class extends its enclosing class, the enclosing class' bridges won't have been generated, so the nested class ends up with redundant bridge methods. I don't think it's a correctness issue, but I found it surprising. Do you think this is worth fixing? Here's the repro: A and B are symmetric: A's nested class extends B, B's nested class extends A. Depending on the order of compilation a bridge method is generated in A$Nested or B$Nested, but not both. $ cat I.java interface I { void f(T t); } $ cat A.java class A implements I { public void f(Number x) {} static class Nested extends B { } } $ cat B.java class B implements I { public void f(Number x) {} static class Nested extends A { } } # pass A.java to javac first, A$Nested has a bridge method $ rm -f *.class $ javac I.java A.java B.java $ javap -private 'A$Nested' Compiled from "A.java" class A$Nested extends B { A$Nested(); public void f(java.lang.Object); } $ javap -private 'B$Nested' Compiled from "B.java" class B$Nested extends A { B$Nested(); } # pass B.java to javac first, B$Nested has a bridge method $ rm -f *.class $ javac I.java B.java A.java $ javap -private 'A$Nested' Compiled from "A.java" class A$Nested extends B { A$Nested(); } $ javap -private 'B$Nested' Compiled from "B.java" class B$Nested extends A { B$Nested(); public void f(java.lang.Object); } -------------- next part -------------- An HTML attachment was scrubbed... URL: From maurizio.cimadamore at oracle.com Wed Jul 29 23:31:46 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 30 Jul 2015 00:31:46 +0100 Subject: Redundant bridge methods in nested classes In-Reply-To: References: Message-ID: <55B96262.7080403@oracle.com> Nicely spotted - maybe in this case it's not an issue, but I'd like to file a bug and keep track of this, to see if there's something nastier that could happen as an outcome. The reality is that the bridge code is pretty ancient (dates back to 2001-2002) - and the spec wasn't settled back then; the biggest problem of the current code is that the bridge analysis depends on the members that were physically present on the superclasses; meaning that your analysis is chronically dependent on compilation order issues. In JDK 8 we had a similar issue when we implemented the logic for generating the set of bridges required by a lambda/method reference; as I was very aware of the compilation order issues, we designed an algorithm that was capable of 'simulating' the bridge presence in a given class, simply by looking at overriding properties between methods. This solution is, I believe, far superior, and should be preferred to the current approach which, in fact, adds a lot of convolutedness to the entire compiler pipeline, as it forces superclasses to be translated before subclasses (hint hint - what about superinterfaces???). Maurizio On 29/07/15 18:30, Liam Miller-Cushon wrote: > I've been trying to get a better understanding of bridge methods, and > noticed that javac seems to emit different bridge methods depending on > the order of sources passed to the compilation. > > As far as I can tell this occurs because TransTypes.translateClass > calls super.visitClassDef before addBridges. The super implementation > of visitClassDef processes all declarations (including nested > classes), so nested classes get translated before their enclosing > classes. If a nested class extends its enclosing class, the enclosing > class' bridges won't have been generated, so the nested class ends up > with redundant bridge methods. > > I don't think it's a correctness issue, but I found it surprising. Do > you think this is worth fixing? > > Here's the repro: > > A and B are symmetric: A's nested class extends B, B's nested class > extends A. Depending on the order of compilation a bridge method is > generated in A$Nested or B$Nested, but not both. > > $ cat I.java > interface I { > void f(T t); > } > $ cat A.java > class A implements I { > public void f(Number x) {} > static class Nested extends B { > } > } > $ cat B.java > class B implements I { > public void f(Number x) {} > static class Nested extends A { > } > } > > # pass A.java to javac first, A$Nested has a bridge method > > $ rm -f *.class > $ javac I.java A.java B.java > $ javap -private 'A$Nested' > Compiled from "A.java" > class A$Nested extends B { > A$Nested(); > public void f(java.lang.Object); > } > $ javap -private 'B$Nested' > Compiled from "B.java" > class B$Nested extends A { > B$Nested(); > } > > # pass B.java to javac first, B$Nested has a bridge method > > $ rm -f *.class > $ javac I.java B.java A.java > $ javap -private 'A$Nested' > Compiled from "A.java" > class A$Nested extends B { > A$Nested(); > } > $ javap -private 'B$Nested' > Compiled from "B.java" > class B$Nested extends A { > B$Nested(); > public void f(java.lang.Object); > } From maurizio.cimadamore at oracle.com Thu Jul 30 11:38:46 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 30 Jul 2015 12:38:46 +0100 Subject: RFR JDK-8131915: CompletionFailure during import listing crashes javac In-Reply-To: <55B24595.80900@oracle.com> References: <55B24595.80900@oracle.com> Message-ID: <55BA0CC6.2050909@oracle.com> Looks great. Maurizio On 24/07/15 15:03, Jan Lahoda wrote: > Hello, > > This is a fix for: > https://bugs.openjdk.java.net/browse/JDK-8131915 > > which Liam reported recently. The problem is that if a > CompletionFailure happens while listing import content (e.g. due to a > missing supertype), it is not handled properly and causes a javac crash. > > The proposed fix is to catch the CompletionFailure and handle it > properly (log an error). > > A webrev with the fix is here: > http://cr.openjdk.java.net/~jlahoda/8131915/webrev.00/ > > Any feedback is welcome, > > Thanks! > Jan From maurizio.cimadamore at oracle.com Thu Jul 30 11:41:34 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 30 Jul 2015 12:41:34 +0100 Subject: RFR JDK-8131915: CompletionFailure during import listing crashes javac In-Reply-To: <55BA0CC6.2050909@oracle.com> References: <55B24595.80900@oracle.com> <55BA0CC6.2050909@oracle.com> Message-ID: <55BA0D6E.9010404@oracle.com> Whoops - hit send too soon. The new tests need the new bug id in the header. Maurizio On 30/07/15 12:38, Maurizio Cimadamore wrote: > Looks great. > > Maurizio > > On 24/07/15 15:03, Jan Lahoda wrote: >> Hello, >> >> This is a fix for: >> https://bugs.openjdk.java.net/browse/JDK-8131915 >> >> which Liam reported recently. The problem is that if a >> CompletionFailure happens while listing import content (e.g. due to a >> missing supertype), it is not handled properly and causes a javac crash. >> >> The proposed fix is to catch the CompletionFailure and handle it >> properly (log an error). >> >> A webrev with the fix is here: >> http://cr.openjdk.java.net/~jlahoda/8131915/webrev.00/ >> >> Any feedback is welcome, >> >> Thanks! >> Jan > From konstantin.barzilovich at oracle.com Thu Jul 30 12:50:52 2015 From: konstantin.barzilovich at oracle.com (konstantin barzilovich) Date: Thu, 30 Jul 2015 15:50:52 +0300 Subject: Mismatch of public spec and diffs in Project Coin Message-ID: <55BA1DAC.3010001@oracle.com> Hello Dan, There are diffs of assertions inJDK-8073593 which allow diamonds with anonymous classes. Some assertions are not the same as in initial version of spec. For example, if I understand correctly there is no intention to modify the third assertion jls-15.9.1-100-A.1: "The Identifier after the new token must denote..." but in public JLS it is: "The ClassOrInterfaceTypeToInstantiate must denote..." It is not a great problem because the meaning is the same, but sometimes it might be hard to find corresponding assertion in public JLS. What was the reason of it? What variant will be eventually in JLS 9? Thanks, Konstantin. -------------- next part -------------- An HTML attachment was scrubbed... URL: From maurizio.cimadamore at oracle.com Thu Jul 30 16:06:04 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 30 Jul 2015 17:06:04 +0100 Subject: [PATCH] 8081769: Redundant error message on bad usage of class literal In-Reply-To: <559147DE.1030404@oracle.com> References: <559147DE.1030404@oracle.com> Message-ID: <55BA4B6C.9010501@oracle.com> Hi, I've just pushed a fix for this issue - unfortunately the proposed fix was causing too many regressions in existing tests - i.e. cases where the new output was worse than the old one. So I settled for a more localized fix. http://hg.openjdk.java.net/jdk9/dev/langtools/rev/3c1da6c1ef9d Thanks! Maurizio On 29/06/15 14:27, Maurizio Cimadamore wrote: > Thanks - this looks solid; I will double check the patch, run all > required tests and integrate it (if all test pass :-)) > > Regards > Maurizio > > On 25/06/15 15:58, bsrbnd wrote: >> Hi, >> >> as described in issue 8081769, the following code produces two errors >> instead of one plus a wrong end of file error. >> >> public class BadClassLiteral { >> public static void method() { >> Class c1 = this.class; >> } >> } >> >> BadClassLiteral.java:3: error: expected >> Class c1 = this.class; >> ^ >> BadClassLiteral.java:3: error: expected >> Class c1 = this.class; >> ^ >> BadClassLiteral.java:5: error: reached end of file while parsing >> } >> ^ >> 3 errors >> >> While the first error is ok, the second and third errors occured >> because the parser goes on with a class definition starting with the >> "class" token. >> Method JavacParser.accept(TokenKind) should skip next token ("class" >> in this case) even if an error occurs, as follows (jdk 9): >> >> diff --git >> a/src/jdk.compiler/share/classes/com/sun/tools/javac/parser/JavacParser.java >> b/src/jdk.compiler/share/classes/com/sun/tools/javac/parser/JavacParser.java >> >> --- >> a/src/jdk.compiler/share/classes/com/sun/tools/javac/parser/JavacParser.java >> +++ >> b/src/jdk.compiler/share/classes/com/sun/tools/javac/parser/JavacParser.java >> @@ -485,6 +485,7 @@ >> } else { >> setErrorEndPos(token.pos); >> reportSyntaxError(S.prevToken().endPos, "expected", tk); >> + nextToken(); >> } >> } >> >> Regards, >> >> bsrbnd > From cushon at google.com Thu Jul 30 23:55:57 2015 From: cushon at google.com (Liam Miller-Cushon) Date: Thu, 30 Jul 2015 16:55:57 -0700 Subject: Redundant bridge methods in nested classes In-Reply-To: <55B96262.7080403@oracle.com> References: <55B96262.7080403@oracle.com> Message-ID: Interesting, thanks Maurizio. On Wed, Jul 29, 2015 at 4:31 PM, Maurizio Cimadamore < maurizio.cimadamore at oracle.com> wrote: > the biggest problem of the current code is that the bridge analysis > depends on the members that were physically present on the superclasses; > meaning that your analysis is chronically dependent on compilation order > issues. > As long as the hierarchy is well formed, is it possible to enforce that superclasses and interfaces (possibly including enclosing classes) are always translated before the current class? I suppose there are some cycles that would be tricky to get right: `class C { static class N extends C {} }`. If that was possible, though, the implementation could assert that classes are never translated recursively, and catch issues like the one I noticed. > In JDK 8 we had a similar issue when we implemented the logic for > generating the set of bridges required by a lambda/method reference; as I > was very aware of the compilation order issues, we designed an algorithm > that was capable of 'simulating' the bridge presence in a given class, > simply by looking at overriding properties between methods. This solution > is, I believe, far superior, and should be preferred to the current > approach which, in fact, adds a lot of convolutedness to the entire > compiler pipeline, as it forces superclasses to be translated before > subclasses (hint hint - what about superinterfaces???). That does sound way nicer. Did you consider switching the regular bridge implementation over to use the new algorithm? I'm trying to grok functionalInterfaceBridges. Is the presence of bridges only simulated for classes that will be translated in the current compilation? The implementation appears to deal with classes completed from class files that are missing bridges (maybe because they were compiled by another compiler, or javac <= 7?). Liam -------------- next part -------------- An HTML attachment was scrubbed... URL: From maurizio.cimadamore at oracle.com Fri Jul 31 10:18:52 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 31 Jul 2015 11:18:52 +0100 Subject: Redundant bridge methods in nested classes In-Reply-To: References: <55B96262.7080403@oracle.com> Message-ID: <55BB4B8C.1060905@oracle.com> On 31/07/15 00:55, Liam Miller-Cushon wrote: > Interesting, thanks Maurizio. > > On Wed, Jul 29, 2015 at 4:31 PM, Maurizio Cimadamore > > wrote: > > the biggest problem of the current code is that the bridge > analysis depends on the members that were physically present on > the superclasses; meaning that your analysis is chronically > dependent on compilation order issues. > > > As long as the hierarchy is well formed, is it possible to enforce > that superclasses and interfaces (possibly including enclosing > classes) are always translated before the current class? I suppose > there are some cycles that would be tricky to get right: `class C extends C.N> { static class N extends C {} }`. If that was possible, > though, the implementation could assert that classes are never > translated recursively, and catch issues like the one I noticed. There are many kinds of dependencies - the one in your example involves tvar bounds, and I believe these are harmless when it comes to bridging; the bridge generation process depends on the members of superclasses/superinterfaces, not on the members of their parameter bounds. That said, I believe it's hard to linearize a hierarchy; in your original example we have that ('->' means 'should be translated after'): A -> I A.Nested -> B B -> I B.Nested -> A If you treat this as a graph, and you do a topological sort of the nodes, you end up with the following possible translation order: I, B, A.Nested, A, B.Nested As you noticed, this order doesn't sit well with a tree visitor, as there's no way i.e. to visit B without also doing B.Nested - which will mess up the order completely (as is the case now). What this means is that TransTypes should probably NOT translate class recursively. I think that such an approach would have more chances to succeed than the existing one. > > In JDK 8 we had a similar issue when we implemented the logic for > generating the set of bridges required by a lambda/method > reference; as I was very aware of the compilation order issues, we > designed an algorithm that was capable of 'simulating' the bridge > presence in a given class, simply by looking at overriding > properties between methods. This solution is, I believe, far > superior, and should be preferred to the current approach which, > in fact, adds a lot of convolutedness to the entire compiler > pipeline, as it forces superclasses to be translated before > subclasses (hint hint - what about superinterfaces???). > > > That does sound way nicer. Did you consider switching the regular > bridge implementation over to use the new algorithm? It's on our radar - but it's also a very very risky change, as you can appreciate. Realistically, we've been generating code this way for so long, that I wouldn't be surprised if there were clients out there relying on current bridge generation bugs - which makes everything a lot harder (reflection depends on that too - as some bridges are generated just for that use case :-( ). It's a big knot. > > I'm trying to grok functionalInterfaceBridges. Is the presence of > bridges only simulated for classes that will be translated in the > current compilation? The implementation appears to deal with classes > completed from class files that are missing bridges (maybe because > they were compiled by another compiler, or javac <= 7?). The idea behind this logic is to compute the set of bridges associated with a given functional type A. As a functional interface might depend on superinterfaces that could be coming from both sourcefile and classfile, the logic must be able to compute the union of the bridges available in such dependecies; for classfiles we can expect bridges to be already there (the check is done with binaryImplementation [*]) - for sourcefile we just can't rely on membership (because of the problem you mentioned), so we simulate bridge presence by exploiting overriding properties. [*] - for this very reason, bridge methods are among the (very) few synthetic symbols that are preserved at class reading time. > > Liam -------------- next part -------------- An HTML attachment was scrubbed... URL: From maurizio.cimadamore at oracle.com Fri Jul 31 10:40:22 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 31 Jul 2015 11:40:22 +0100 Subject: Redundant bridge methods in nested classes In-Reply-To: References: Message-ID: <55BB5096.3060108@oracle.com> FYI - bug filed here: https://bugs.openjdk.java.net/browse/JDK-8132776 Maurizio On 29/07/15 18:30, Liam Miller-Cushon wrote: > I've been trying to get a better understanding of bridge methods, and > noticed that javac seems to emit different bridge methods depending on > the order of sources passed to the compilation. > > As far as I can tell this occurs because TransTypes.translateClass > calls super.visitClassDef before addBridges. The super implementation > of visitClassDef processes all declarations (including nested > classes), so nested classes get translated before their enclosing > classes. If a nested class extends its enclosing class, the enclosing > class' bridges won't have been generated, so the nested class ends up > with redundant bridge methods. > > I don't think it's a correctness issue, but I found it surprising. Do > you think this is worth fixing? > > Here's the repro: > > A and B are symmetric: A's nested class extends B, B's nested class > extends A. Depending on the order of compilation a bridge method is > generated in A$Nested or B$Nested, but not both. > > $ cat I.java > interface I { > void f(T t); > } > $ cat A.java > class A implements I { > public void f(Number x) {} > static class Nested extends B { > } > } > $ cat B.java > class B implements I { > public void f(Number x) {} > static class Nested extends A { > } > } > > # pass A.java to javac first, A$Nested has a bridge method > > $ rm -f *.class > $ javac I.java A.java B.java > $ javap -private 'A$Nested' > Compiled from "A.java" > class A$Nested extends B { > A$Nested(); > public void f(java.lang.Object); > } > $ javap -private 'B$Nested' > Compiled from "B.java" > class B$Nested extends A { > B$Nested(); > } > > # pass B.java to javac first, B$Nested has a bridge method > > $ rm -f *.class > $ javac I.java B.java A.java > $ javap -private 'A$Nested' > Compiled from "A.java" > class A$Nested extends B { > A$Nested(); > } > $ javap -private 'B$Nested' > Compiled from "B.java" > class B$Nested extends A { > B$Nested(); > public void f(java.lang.Object); > } From jan.lahoda at oracle.com Fri Jul 31 11:36:20 2015 From: jan.lahoda at oracle.com (Jan Lahoda) Date: Fri, 31 Jul 2015 13:36:20 +0200 Subject: RFR JDK-8131915: CompletionFailure during import listing crashes javac In-Reply-To: <55BA0D6E.9010404@oracle.com> References: <55B24595.80900@oracle.com> <55BA0CC6.2050909@oracle.com> <55BA0D6E.9010404@oracle.com> Message-ID: <55BB5DB4.3020601@oracle.com> Thanks for the comments. Based on additional offline feedback, I tried to use "BiConsumer" instead of "Consumer" - the advantage is that the lambda does not need to capture the JCImport. The FilterImportScope however needs to keep the JCImport, instead of the current "staticImport" boolean flag. A webrev with these changes is here: http://cr.openjdk.java.net/~jlahoda/8131915/webrev.01/ This new state seems reasonable to me. What do you think? (I've also fixed the bug # in the new iteration and added a summary to the CompletionFailureDuringImport test.) Thanks, Jan On 30.7.2015 13:41, Maurizio Cimadamore wrote: > Whoops - hit send too soon. > > The new tests need the new bug id in the header. > > Maurizio > > On 30/07/15 12:38, Maurizio Cimadamore wrote: >> Looks great. >> >> Maurizio >> >> On 24/07/15 15:03, Jan Lahoda wrote: >>> Hello, >>> >>> This is a fix for: >>> https://bugs.openjdk.java.net/browse/JDK-8131915 >>> >>> which Liam reported recently. The problem is that if a >>> CompletionFailure happens while listing import content (e.g. due to a >>> missing supertype), it is not handled properly and causes a javac crash. >>> >>> The proposed fix is to catch the CompletionFailure and handle it >>> properly (log an error). >>> >>> A webrev with the fix is here: >>> http://cr.openjdk.java.net/~jlahoda/8131915/webrev.00/ >>> >>> Any feedback is welcome, >>> >>> Thanks! >>> Jan >> > From maurizio.cimadamore at oracle.com Fri Jul 31 11:44:03 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 31 Jul 2015 12:44:03 +0100 Subject: RFR JDK-8131915: CompletionFailure during import listing crashes javac In-Reply-To: <55BB5DB4.3020601@oracle.com> References: <55B24595.80900@oracle.com> <55BA0CC6.2050909@oracle.com> <55BA0D6E.9010404@oracle.com> <55BB5DB4.3020601@oracle.com> Message-ID: <55BB5F83.2040206@oracle.com> Looks great! I guess in TypeEnter you could go either with a lambda or a method reference (given that tree.pos() == tree); but if you go down the method reference path, I believe you can have issues if 'chk' is null when the initializer is evaluated, so a lambda seems generally safer here - albeit slightly less direct. Maurizio On 31/07/15 12:36, Jan Lahoda wrote: > Thanks for the comments. > > Based on additional offline feedback, I tried to use > "BiConsumer" instead of > "Consumer" - the advantage is that the lambda does > not need to capture the JCImport. The FilterImportScope however needs > to keep the JCImport, instead of the current "staticImport" boolean flag. > > A webrev with these changes is here: > http://cr.openjdk.java.net/~jlahoda/8131915/webrev.01/ > > This new state seems reasonable to me. What do you think? > > (I've also fixed the bug # in the new iteration and added a summary to > the CompletionFailureDuringImport test.) > > Thanks, > Jan > > On 30.7.2015 13:41, Maurizio Cimadamore wrote: >> Whoops - hit send too soon. >> >> The new tests need the new bug id in the header. >> >> Maurizio >> >> On 30/07/15 12:38, Maurizio Cimadamore wrote: >>> Looks great. >>> >>> Maurizio >>> >>> On 24/07/15 15:03, Jan Lahoda wrote: >>>> Hello, >>>> >>>> This is a fix for: >>>> https://bugs.openjdk.java.net/browse/JDK-8131915 >>>> >>>> which Liam reported recently. The problem is that if a >>>> CompletionFailure happens while listing import content (e.g. due to a >>>> missing supertype), it is not handled properly and causes a javac >>>> crash. >>>> >>>> The proposed fix is to catch the CompletionFailure and handle it >>>> properly (log an error). >>>> >>>> A webrev with the fix is here: >>>> http://cr.openjdk.java.net/~jlahoda/8131915/webrev.00/ >>>> >>>> Any feedback is welcome, >>>> >>>> Thanks! >>>> Jan >>> >> From maurizio.cimadamore at oracle.com Fri Jul 31 12:12:26 2015 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 31 Jul 2015 13:12:26 +0100 Subject: Redundant bridge methods in nested classes In-Reply-To: <55BB4B8C.1060905@oracle.com> References: <55B96262.7080403@oracle.com> <55BB4B8C.1060905@oracle.com> Message-ID: <55BB662A.6040608@oracle.com> On 31/07/15 11:18, Maurizio Cimadamore wrote: > As you noticed, this order doesn't sit well with a tree visitor, as > there's no way i.e. to visit B without also doing B.Nested - which > will mess up the order completely (as is the case now). What this > means is that TransTypes should probably NOT translate class > recursively. I think that such an approach would have more chances to > succeed than the existing one. Thinking more about this, while it's probably not a big deal to turn TransType into something that handles one class at a time (like Attr for instance), doing so for Lower will probably be a major undertaking; a lot of the state variables in Lower assume that you visit the enclosing classes before you reach inner classes - stuff like 'outerThisStack'; additionally, some of the desugared code in lower always ends up in the toplevel class, no matter the nesting (see the constructor tag class, which is always a static inner class of the toplevel). Pulling out these pieces so that Lower is well-behaved will be generally very very hard. If this can't be done, of course we need a dependency from inner to outer (as now), and that means that we start getting cycles in our graph: transtypes: A -> I A.Nested -> B B -> I B.Nested -> A lower: A -> A.Nested B -> B.Nested Meaning the order now is: I, { A, A.Nested, B, B.Nested } In other words, there's a cycle now - meaning you can disentangle the mess using some heuristics (as we do now), but there will be cases where compilation order will show up. I see few solutions out of this: (1) make Lower and TransTypes smarter, so that they do not rely on any dependencies; then we can process classes in any order (2) make only one of them smarter and sort dependencies accordingly; i.e. if TransTypes is rewritten so that it doesn't rely on actual members, we can just get rid of all transtypes-like dependencies (3) split desugar into two steps - one does erasure, and sorts envs using transtypes dependencies; another does lowering and sorts envs using lower dependencies Of these, (1) looks the most disruptive, while (3) seems the most straightforward, albeit I don't think (3) can be made to work alongside different compilation policies; i.e. (3) would work fine using a policy like SIMPLE (where all classes to be compiled are advanced to the next stage before any other action is taken); for policies such as BY_FILE or BY_TODO (the default), things are harder, as things are pushed down the pipeline eagerly - i.e. you can have one class in the code generation step, while another class has not even been attributed. Note that such policies are fare better when it comes to memory footprint, as ASTs are typically discarded after code generation; so, when compiling a large number of files at once, the sooner you hit code generation, the better (memory-wise). (2) seems like a fair balance, but it's still a lot of work (see my previous email). Maurizio