From chriskitching at linux.com Sun Sep 7 03:06:51 2014 From: chriskitching at linux.com (Chris Kitching) Date: Sat, 06 Sep 2014 20:06:51 -0700 Subject: Apparent bug in javac's dead code elimination In-Reply-To: References: <53E7FA44.9050207@oracle.com> <476830A4-7A8E-4652-A0C7-805687155F4A@oracle.com> <53E8CCD6.2090503@CoSoCo.de> <53FDA7C1.6060303@CoSoCo.de> <902B0B7A-32BE-498A-B9DD-8937960430A2@oracle.com> <53FF7244.2030804@CoSoCo.de> <5400E85E.9030005@CoSoCo.de> Message-ID: <540BCBCB.8010406@linux.com> Greetings, As you probably know, javac does a bit of dead code elimintion: it'll get rid of redundant constructs like if (false) and suchlike. Consider, then, this ridiculous example: public class Cake { public void start() { if (false) { new Runnable() { @Override public void run() { // Whatever } }; } } private static class SomeClass { } } As you might expect, this produces two classes: Cake and Cake$SomeClass.class, and all is well. Now consider this slightly more ridiculous example: public class Cake { public void start() { if (false) { new Runnable() { @Override public void run() { // Whatever } }; } } private static class SomeClass { } private static class SomeOtherClass extends SomeClass { } } All I did was add an empty SomeOtherClass extending SomeClass. This example produces out these classes: Cake$1 Cake Cake$SomeClass Cake$SomeOtherClass Cake$1, the Runnable wrapped in `if (false)`, has suddenly appeared. Oddly, the bytecode in Cake$1 does not change as you alter the contents of run(). It's as if javac is omitting the contents of the methods of the class, but not the class itself (but it did so earlier before we added SomeOtherClass!) Cake$1 is also corrupt. Attempts to load the class using the Reflection API (because, you know, that's a sane thing to do) fail with a stacktrace like this: java.lang.ClassFormatError: Absent Code attribute in method that is not native or abstract in class file org/mozilla/gecko/animation/PropertyAnimator$1 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:791) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) at java.net.URLClassLoader.access$100(URLClassLoader.java:71) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:423) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) And that's exactly what just happened to Mozilla: https://bugzilla.mozilla.org/show_bug.cgi?id=1063991 (that also provides a bit of context, as well as the interesting consequence that the Cake$1 you get if you move the Runnable outside the if is different to the one you get in the situation described above (when, really, no Cake$1 at all should exist). Perhaps the desugaring step for inner classes is getting a little... overzealous? I've been able to reproduce this with javac 7 and 8. I've not tried it on the current JDK 9 snapshot. From maurizio.cimadamore at oracle.com Mon Sep 8 13:33:07 2014 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 08 Sep 2014 14:33:07 +0100 Subject: Apparent bug in javac's dead code elimination In-Reply-To: <540BCBCB.8010406@linux.com> References: <53E7FA44.9050207@oracle.com> <476830A4-7A8E-4652-A0C7-805687155F4A@oracle.com> <53E8CCD6.2090503@CoSoCo.de> <53FDA7C1.6060303@CoSoCo.de> <902B0B7A-32BE-498A-B9DD-8937960430A2@oracle.com> <53FF7244.2030804@CoSoCo.de> <5400E85E.9030005@CoSoCo.de> <540BCBCB.8010406@linux.com> Message-ID: <540DB013.4020803@oracle.com> Hi Chris, we did some work as part of this: https://bugs.openjdk.java.net/browse/JDK-7199823 When working on JDK 8. While this fixes the main issue, it seems like there are other related issues that we need to address in this area. Thanks for the report! Maurizio On 07/09/14 04:06, Chris Kitching wrote: > Greetings, > > As you probably know, javac does a bit of dead code elimintion: it'll > get rid of redundant constructs like if (false) and suchlike. > > Consider, then, this ridiculous example: > > public class Cake { > public void start() { > if (false) { > new Runnable() { > @Override > public void run() { > // Whatever > } > }; > } > } > > private static class SomeClass { > } > } > > As you might expect, this produces two classes: Cake and > Cake$SomeClass.class, and all is well. > > Now consider this slightly more ridiculous example: > > public class Cake { > public void start() { > if (false) { > new Runnable() { > @Override > public void run() { > // Whatever > } > }; > } > } > > private static class SomeClass { > } > > private static class SomeOtherClass extends SomeClass { > } > } > > > All I did was add an empty SomeOtherClass extending SomeClass. > This example produces out these classes: > > Cake$1 > Cake > Cake$SomeClass > Cake$SomeOtherClass > > Cake$1, the Runnable wrapped in `if (false)`, has suddenly appeared. > Oddly, the bytecode in Cake$1 does not change as you alter the contents > of run(). It's as if javac is omitting the contents of the methods of > the class, but not the class itself (but it did so earlier before we > added SomeOtherClass!) > Cake$1 is also corrupt. Attempts to load the class using the Reflection > API (because, you know, that's a sane thing to do) fail with a > stacktrace like this: > > java.lang.ClassFormatError: Absent Code attribute in method that is not > native or abstract in class file > org/mozilla/gecko/animation/PropertyAnimator$1 > at java.lang.ClassLoader.defineClass1(Native Method) > at java.lang.ClassLoader.defineClass(ClassLoader.java:791) > at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) > at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) > at java.net.URLClassLoader.access$100(URLClassLoader.java:71) > at java.net.URLClassLoader$1.run(URLClassLoader.java:361) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:423) > at java.lang.ClassLoader.loadClass(ClassLoader.java:356) > > And that's exactly what just happened to Mozilla: > https://bugzilla.mozilla.org/show_bug.cgi?id=1063991 > > (that also provides a bit of context, as well as the interesting > consequence that the Cake$1 you get if you move the Runnable outside the > if is different to the one you get in the situation described above > (when, really, no Cake$1 at all should exist). > > Perhaps the desugaring step for inner classes is getting a little... > overzealous? > > I've been able to reproduce this with javac 7 and 8. I've not tried it > on the current JDK 9 snapshot. From Sergey.Bylokhov at oracle.com Mon Sep 8 13:34:20 2014 From: Sergey.Bylokhov at oracle.com (Sergey Bylokhov) Date: Mon, 08 Sep 2014 17:34:20 +0400 Subject: Optimization 2.0 for composing strings - Was: Replace concat String to append in StringBuilder parameters In-Reply-To: <5400E85E.9030005@CoSoCo.de> References: <53E7FA44.9050207@oracle.com> <476830A4-7A8E-4652-A0C7-805687155F4A@oracle.com> <53E8CCD6.2090503@CoSoCo.de> <53FDA7C1.6060303@CoSoCo.de> <902B0B7A-32BE-498A-B9DD-8937960430A2@oracle.com> <53FF7244.2030804@CoSoCo.de> <5400E85E.9030005@CoSoCo.de> Message-ID: <540DB05C.3010300@oracle.com> There was related discussions on the past http://mail.openjdk.java.net/pipermail/compiler-dev/2014-February/008491.html And related issues: https://bugs.openjdk.java.net/browse/JDK-4947460 https://bugs.openjdk.java.net/browse/JDK-4059189 https://bugs.openjdk.java.net/browse/JDK-6709423 On 30.08.2014 0:53, Ulf Zibis wrote: > Hi compiler people, > > is there some chance that javac could be enhanced to optimize better > as discussed in this thread? Than refactoring of up to now better > readable code to ugly StringBuilder at append() code would be superfluous. > I really like the String concatenation syntax, but unfortunately it > often causes slower code and bigger footprint. > Optimally javac would optimize mixed use of StringBuilder, > StringJoiner, concatenation, toString(), append(String), > append(Collection) etc. to a single StringBuilder instance, so that > the resulting code, JITed or not, will have better performance. > Additionally javac could guess a reasonable initial capacity from the > given source code. > > > Am 29.08.2014 um 10:01 schrieb Wang Weijun: >> So it's not that the optimization fails but there is no optimization >> on them yet. >> >> I do see the .append("x") case will be easy to deal with, but it >> looks like historically javac has not been a place to do many >> optimizations. It mostly converts the java source to byte codes in a >> 1-to-1 mapping and let VM do whatever it wants (to optimize). When >> you talked about compiling multiple concatenation into using a single >> StringBuilder, it's more like choosing the correct implementation >> rather than an optimization. >> >> I don't expect to see big change on this in the near future, so shall >> we go on with the current enhancement? >> >> Thanks >> Max >> >> On Aug 29, 2014, at 2:17, Ulf Zibis wrote: >> >>> I mean: >>> It does not output byte code that only uses a single char array to >>> compose the entire String in question. >>> With "optimization fails", I also mean, there is used an additional >>> "StringComposer" e.g. another StringBuilder or a StringJoiner in >>> addition to the 1st StringBuilder. >>> >>> -Ulf >>> >>> Am 27.08.2014 um 14:02 schrieb Pavel Rappo: >>>> Could you please explain what you mean by "javac optimization >>>> fails" here? >>>> >>>> -Pavel >>>> >>>> On 27 Aug 2014, at 10:41, Ulf Zibis wrote: >>>> >>>>> 4.) Now we see, that javac optimization fails again if >>>>> StringBuilder, concatenation, toString(), append(String), >>>>> append(Collection) etc. and StringJoiner use is mixed. >> > -- Best regards, Sergey. From vicente.romero at oracle.com Mon Sep 8 16:58:00 2014 From: vicente.romero at oracle.com (Vicente-Arturo Romero-Zaldivar) Date: Mon, 08 Sep 2014 09:58:00 -0700 Subject: Apparent bug in javac's dead code elimination In-Reply-To: <540DB013.4020803@oracle.com> References: <53E7FA44.9050207@oracle.com> <476830A4-7A8E-4652-A0C7-805687155F4A@oracle.com> <53E8CCD6.2090503@CoSoCo.de> <53FDA7C1.6060303@CoSoCo.de> <902B0B7A-32BE-498A-B9DD-8937960430A2@oracle.com> <53FF7244.2030804@CoSoCo.de> <5400E85E.9030005@CoSoCo.de> <540BCBCB.8010406@linux.com> <540DB013.4020803@oracle.com> Message-ID: <540DE018.6050407@oracle.com> Hi Chris, Thanks again for the report. I have created bug entry https://bugs.openjdk.java.net/browse/JDK-8057823 to track it. Vicente On 09/08/2014 06:33 AM, Maurizio Cimadamore wrote: > Hi Chris, > we did some work as part of this: > > https://bugs.openjdk.java.net/browse/JDK-7199823 > > When working on JDK 8. While this fixes the main issue, it seems like > there are other related issues that we need to address in this area. > Thanks for the report! > > Maurizio > > On 07/09/14 04:06, Chris Kitching wrote: >> Greetings, >> >> As you probably know, javac does a bit of dead code elimintion: it'll >> get rid of redundant constructs like if (false) and suchlike. >> >> Consider, then, this ridiculous example: >> >> public class Cake { >> public void start() { >> if (false) { >> new Runnable() { >> @Override >> public void run() { >> // Whatever >> } >> }; >> } >> } >> >> private static class SomeClass { >> } >> } >> >> As you might expect, this produces two classes: Cake and >> Cake$SomeClass.class, and all is well. >> >> Now consider this slightly more ridiculous example: >> >> public class Cake { >> public void start() { >> if (false) { >> new Runnable() { >> @Override >> public void run() { >> // Whatever >> } >> }; >> } >> } >> >> private static class SomeClass { >> } >> >> private static class SomeOtherClass extends SomeClass { >> } >> } >> >> >> All I did was add an empty SomeOtherClass extending SomeClass. >> This example produces out these classes: >> >> Cake$1 >> Cake >> Cake$SomeClass >> Cake$SomeOtherClass >> >> Cake$1, the Runnable wrapped in `if (false)`, has suddenly appeared. >> Oddly, the bytecode in Cake$1 does not change as you alter the contents >> of run(). It's as if javac is omitting the contents of the methods of >> the class, but not the class itself (but it did so earlier before we >> added SomeOtherClass!) >> Cake$1 is also corrupt. Attempts to load the class using the Reflection >> API (because, you know, that's a sane thing to do) fail with a >> stacktrace like this: >> >> java.lang.ClassFormatError: Absent Code attribute in method that is not >> native or abstract in class file >> org/mozilla/gecko/animation/PropertyAnimator$1 >> at java.lang.ClassLoader.defineClass1(Native Method) >> at java.lang.ClassLoader.defineClass(ClassLoader.java:791) >> at >> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) >> at java.net.URLClassLoader.access$100(URLClassLoader.java:71) >> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) >> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:423) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:356) >> >> And that's exactly what just happened to Mozilla: >> https://bugzilla.mozilla.org/show_bug.cgi?id=1063991 >> >> (that also provides a bit of context, as well as the interesting >> consequence that the Cake$1 you get if you move the Runnable outside the >> if is different to the one you get in the situation described above >> (when, really, no Cake$1 at all should exist). >> >> Perhaps the desugaring step for inner classes is getting a little... >> overzealous? >> >> I've been able to reproduce this with javac 7 and 8. I've not tried it >> on the current JDK 9 snapshot. > From jonathan.gibbons at oracle.com Mon Sep 8 17:26:05 2014 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 08 Sep 2014 10:26:05 -0700 Subject: Apparent bug in javac's dead code elimination In-Reply-To: <540BCBCB.8010406@linux.com> References: <53E7FA44.9050207@oracle.com> <476830A4-7A8E-4652-A0C7-805687155F4A@oracle.com> <53E8CCD6.2090503@CoSoCo.de> <53FDA7C1.6060303@CoSoCo.de> <902B0B7A-32BE-498A-B9DD-8937960430A2@oracle.com> <53FF7244.2030804@CoSoCo.de> <5400E85E.9030005@CoSoCo.de> <540BCBCB.8010406@linux.com> Message-ID: <540DE6AD.2040703@oracle.com> I believe the existence of Cake$1 is part of javac's handling of allowing any private code in SomeClass to access any private code in SomeOtherClass. -- Jon On 09/06/2014 08:06 PM, Chris Kitching wrote: > Greetings, > > As you probably know, javac does a bit of dead code elimintion: it'll > get rid of redundant constructs like if (false) and suchlike. > > Consider, then, this ridiculous example: > > public class Cake { > public void start() { > if (false) { > new Runnable() { > @Override > public void run() { > // Whatever > } > }; > } > } > > private static class SomeClass { > } > } > > As you might expect, this produces two classes: Cake and > Cake$SomeClass.class, and all is well. > > Now consider this slightly more ridiculous example: > > public class Cake { > public void start() { > if (false) { > new Runnable() { > @Override > public void run() { > // Whatever > } > }; > } > } > > private static class SomeClass { > } > > private static class SomeOtherClass extends SomeClass { > } > } > > > All I did was add an empty SomeOtherClass extending SomeClass. > This example produces out these classes: > > Cake$1 > Cake > Cake$SomeClass > Cake$SomeOtherClass > > Cake$1, the Runnable wrapped in `if (false)`, has suddenly appeared. > Oddly, the bytecode in Cake$1 does not change as you alter the contents > of run(). It's as if javac is omitting the contents of the methods of > the class, but not the class itself (but it did so earlier before we > added SomeOtherClass!) > Cake$1 is also corrupt. Attempts to load the class using the Reflection > API (because, you know, that's a sane thing to do) fail with a > stacktrace like this: > > java.lang.ClassFormatError: Absent Code attribute in method that is not > native or abstract in class file > org/mozilla/gecko/animation/PropertyAnimator$1 > at java.lang.ClassLoader.defineClass1(Native Method) > at java.lang.ClassLoader.defineClass(ClassLoader.java:791) > at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) > at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) > at java.net.URLClassLoader.access$100(URLClassLoader.java:71) > at java.net.URLClassLoader$1.run(URLClassLoader.java:361) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:423) > at java.lang.ClassLoader.loadClass(ClassLoader.java:356) > > And that's exactly what just happened to Mozilla: > https://bugzilla.mozilla.org/show_bug.cgi?id=1063991 > > (that also provides a bit of context, as well as the interesting > consequence that the Cake$1 you get if you move the Runnable outside the > if is different to the one you get in the situation described above > (when, really, no Cake$1 at all should exist). > > Perhaps the desugaring step for inner classes is getting a little... > overzealous? > > I've been able to reproduce this with javac 7 and 8. I've not tried it > on the current JDK 9 snapshot. From jonathan.gibbons at oracle.com Mon Sep 8 17:30:31 2014 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 08 Sep 2014 10:30:31 -0700 Subject: Optimization 2.0 for composing strings - Was: Replace concat String to append in StringBuilder parameters In-Reply-To: <5400E85E.9030005@CoSoCo.de> References: <53E7FA44.9050207@oracle.com> <476830A4-7A8E-4652-A0C7-805687155F4A@oracle.com> <53E8CCD6.2090503@CoSoCo.de> <53FDA7C1.6060303@CoSoCo.de> <902B0B7A-32BE-498A-B9DD-8937960430A2@oracle.com> <53FF7244.2030804@CoSoCo.de> <5400E85E.9030005@CoSoCo.de> Message-ID: <540DE7B7.9070107@oracle.com> It would be inappropriate/incorrect to apply the optimization as described. The JLS requires that the argument to a method call should be computed before invoking the method. Consider the case when one of the expressions in the series of string concatenations throws an exception. It would be incorrect to have already partially modified the string buffer. -- Jon On 08/29/2014 01:53 PM, Ulf Zibis wrote: > Hi compiler people, > > is there some chance that javac could be enhanced to optimize better > as discussed in this thread? Than refactoring of up to now better > readable code to ugly StringBuilder at append() code would be superfluous. > I really like the String concatenation syntax, but unfortunately it > often causes slower code and bigger footprint. > Optimally javac would optimize mixed use of StringBuilder, > StringJoiner, concatenation, toString(), append(String), > append(Collection) etc. to a single StringBuilder instance, so that > the resulting code, JITed or not, will have better performance. > Additionally javac could guess a reasonable initial capacity from the > given source code. > > > Am 29.08.2014 um 10:01 schrieb Wang Weijun: >> So it's not that the optimization fails but there is no optimization >> on them yet. >> >> I do see the .append("x") case will be easy to deal with, but it >> looks like historically javac has not been a place to do many >> optimizations. It mostly converts the java source to byte codes in a >> 1-to-1 mapping and let VM do whatever it wants (to optimize). When >> you talked about compiling multiple concatenation into using a single >> StringBuilder, it's more like choosing the correct implementation >> rather than an optimization. >> >> I don't expect to see big change on this in the near future, so shall >> we go on with the current enhancement? >> >> Thanks >> Max >> >> On Aug 29, 2014, at 2:17, Ulf Zibis wrote: >> >>> I mean: >>> It does not output byte code that only uses a single char array to >>> compose the entire String in question. >>> With "optimization fails", I also mean, there is used an additional >>> "StringComposer" e.g. another StringBuilder or a StringJoiner in >>> addition to the 1st StringBuilder. >>> >>> -Ulf >>> >>> Am 27.08.2014 um 14:02 schrieb Pavel Rappo: >>>> Could you please explain what you mean by "javac optimization >>>> fails" here? >>>> >>>> -Pavel >>>> >>>> On 27 Aug 2014, at 10:41, Ulf Zibis wrote: >>>> >>>>> 4.) Now we see, that javac optimization fails again if >>>>> StringBuilder, concatenation, toString(), append(String), >>>>> append(Collection) etc. and StringJoiner use is mixed. >> > From jonathan.gibbons at oracle.com Mon Sep 8 17:50:33 2014 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 08 Sep 2014 10:50:33 -0700 Subject: Optimization 2.0 for composing strings - Was: Replace concat String to append in StringBuilder parameters In-Reply-To: <61C40823-1A23-4A0C-BAFD-D1E5B949F98E@oracle.com> References: <53E7FA44.9050207@oracle.com> <476830A4-7A8E-4652-A0C7-805687155F4A@oracle.com> <53E8CCD6.2090503@CoSoCo.de> <53FDA7C1.6060303@CoSoCo.de> <902B0B7A-32BE-498A-B9DD-8937960430A2@oracle.com> <53FF7244.2030804@CoSoCo.de> <5400E85E.9030005@CoSoCo.de> <540DE7B7.9070107@oracle.com> <61C40823-1A23-4A0C-BAFD-D1E5B949F98E@oracle.com> Message-ID: <540DEC69.8080600@oracle.com> Yes, but is this really a big enough performance and footprint pain point to be worth addressing in javac itself? We're now talking about some specific code construction like new StringBuilder().append(a + b + c) Any other case where the string builder can be observed externally cannot be subject to the proposed optimization. The use case is now really getting pretty specific. -- Jon On 09/08/2014 10:41 AM, Guy Steele wrote: > Good point, but counterpoint: it might be acceptable to have modified the string buffer in situations where throwing an exception would always cause the string buffer to become inaccessible. > > ?Guy > > On Sep 8, 2014, at 1:30 PM, Jonathan Gibbons wrote: > >> It would be inappropriate/incorrect to apply the optimization as described. >> >> The JLS requires that the argument to a method call should be computed before invoking the method. >> >> Consider the case when one of the expressions in the series of string concatenations throws an exception. It would be incorrect to have already partially modified the string buffer. >> >> -- Jon >> >> On 08/29/2014 01:53 PM, Ulf Zibis wrote: >>> Hi compiler people, >>> >>> is there some chance that javac could be enhanced to optimize better as discussed in this thread? Than refactoring of up to now better readable code to ugly StringBuilder at append() code would be superfluous. >>> I really like the String concatenation syntax, but unfortunately it often causes slower code and bigger footprint. >>> Optimally javac would optimize mixed use of StringBuilder, StringJoiner, concatenation, toString(), append(String), append(Collection) etc. to a single StringBuilder instance, so that the resulting code, JITed or not, will have better performance. >>> Additionally javac could guess a reasonable initial capacity from the given source code. >>> >>> >>> Am 29.08.2014 um 10:01 schrieb Wang Weijun: >>>> So it's not that the optimization fails but there is no optimization on them yet. >>>> >>>> I do see the .append("x") case will be easy to deal with, but it looks like historically javac has not been a place to do many optimizations. It mostly converts the java source to byte codes in a 1-to-1 mapping and let VM do whatever it wants (to optimize). When you talked about compiling multiple concatenation into using a single StringBuilder, it's more like choosing the correct implementation rather than an optimization. >>>> >>>> I don't expect to see big change on this in the near future, so shall we go on with the current enhancement? >>>> >>>> Thanks >>>> Max >>>> >>>> On Aug 29, 2014, at 2:17, Ulf Zibis wrote: >>>> >>>>> I mean: >>>>> It does not output byte code that only uses a single char array to compose the entire String in question. >>>>> With "optimization fails", I also mean, there is used an additional "StringComposer" e.g. another StringBuilder or a StringJoiner in addition to the 1st StringBuilder. >>>>> >>>>> -Ulf >>>>> >>>>> Am 27.08.2014 um 14:02 schrieb Pavel Rappo: >>>>>> Could you please explain what you mean by "javac optimization fails" here? >>>>>> >>>>>> -Pavel >>>>>> >>>>>> On 27 Aug 2014, at 10:41, Ulf Zibis wrote: >>>>>> >>>>>>> 4.) Now we see, that javac optimization fails again if StringBuilder, concatenation, toString(), append(String), append(Collection) etc. and StringJoiner use is mixed. From guy.steele at oracle.com Mon Sep 8 17:41:55 2014 From: guy.steele at oracle.com (Guy Steele) Date: Mon, 8 Sep 2014 13:41:55 -0400 Subject: Optimization 2.0 for composing strings - Was: Replace concat String to append in StringBuilder parameters In-Reply-To: <540DE7B7.9070107@oracle.com> References: <53E7FA44.9050207@oracle.com> <476830A4-7A8E-4652-A0C7-805687155F4A@oracle.com> <53E8CCD6.2090503@CoSoCo.de> <53FDA7C1.6060303@CoSoCo.de> <902B0B7A-32BE-498A-B9DD-8937960430A2@oracle.com> <53FF7244.2030804@CoSoCo.de> <5400E85E.9030005@CoSoCo.de> <540DE7B7.9070107@oracle.com> Message-ID: <61C40823-1A23-4A0C-BAFD-D1E5B949F98E@oracle.com> Good point, but counterpoint: it might be acceptable to have modified the string buffer in situations where throwing an exception would always cause the string buffer to become inaccessible. ?Guy On Sep 8, 2014, at 1:30 PM, Jonathan Gibbons wrote: > It would be inappropriate/incorrect to apply the optimization as described. > > The JLS requires that the argument to a method call should be computed before invoking the method. > > Consider the case when one of the expressions in the series of string concatenations throws an exception. It would be incorrect to have already partially modified the string buffer. > > -- Jon > > On 08/29/2014 01:53 PM, Ulf Zibis wrote: >> Hi compiler people, >> >> is there some chance that javac could be enhanced to optimize better as discussed in this thread? Than refactoring of up to now better readable code to ugly StringBuilder at append() code would be superfluous. >> I really like the String concatenation syntax, but unfortunately it often causes slower code and bigger footprint. >> Optimally javac would optimize mixed use of StringBuilder, StringJoiner, concatenation, toString(), append(String), append(Collection) etc. to a single StringBuilder instance, so that the resulting code, JITed or not, will have better performance. >> Additionally javac could guess a reasonable initial capacity from the given source code. >> >> >> Am 29.08.2014 um 10:01 schrieb Wang Weijun: >>> So it's not that the optimization fails but there is no optimization on them yet. >>> >>> I do see the .append("x") case will be easy to deal with, but it looks like historically javac has not been a place to do many optimizations. It mostly converts the java source to byte codes in a 1-to-1 mapping and let VM do whatever it wants (to optimize). When you talked about compiling multiple concatenation into using a single StringBuilder, it's more like choosing the correct implementation rather than an optimization. >>> >>> I don't expect to see big change on this in the near future, so shall we go on with the current enhancement? >>> >>> Thanks >>> Max >>> >>> On Aug 29, 2014, at 2:17, Ulf Zibis wrote: >>> >>>> I mean: >>>> It does not output byte code that only uses a single char array to compose the entire String in question. >>>> With "optimization fails", I also mean, there is used an additional "StringComposer" e.g. another StringBuilder or a StringJoiner in addition to the 1st StringBuilder. >>>> >>>> -Ulf >>>> >>>> Am 27.08.2014 um 14:02 schrieb Pavel Rappo: >>>>> Could you please explain what you mean by "javac optimization fails" here? >>>>> >>>>> -Pavel >>>>> >>>>> On 27 Aug 2014, at 10:41, Ulf Zibis wrote: >>>>> >>>>>> 4.) Now we see, that javac optimization fails again if StringBuilder, concatenation, toString(), append(String), append(Collection) etc. and StringJoiner use is mixed. >>> >> > From guy.steele at oracle.com Mon Sep 8 18:10:22 2014 From: guy.steele at oracle.com (Guy Steele) Date: Mon, 8 Sep 2014 14:10:22 -0400 Subject: Optimization 2.0 for composing strings - Was: Replace concat String to append in StringBuilder parameters In-Reply-To: <540DEC69.8080600@oracle.com> References: <53E7FA44.9050207@oracle.com> <476830A4-7A8E-4652-A0C7-805687155F4A@oracle.com> <53E8CCD6.2090503@CoSoCo.de> <53FDA7C1.6060303@CoSoCo.de> <902B0B7A-32BE-498A-B9DD-8937960430A2@oracle.com> <53FF7244.2030804@CoSoCo.de> <5400E85E.9030005@CoSoCo.de> <540DE7B7.9070107@oracle.com> <61C40823-1A23-4A0C-BAFD-D1E5B949F98E@oracle.com> <540DEC69.8080600@oracle.com> Message-ID: I have no idea. I was just sticking in my two cents? worth on what is possible. What is desirable and worthwhile is a much bigger question. On Sep 8, 2014, at 1:50 PM, Jonathan Gibbons wrote: > Yes, but is this really a big enough performance and footprint pain point to be worth addressing in javac itself? > > We're now talking about some specific code construction like > new StringBuilder().append(a + b + c) > > Any other case where the string builder can be observed externally cannot be subject to the proposed optimization. The use case is now really getting pretty specific. > > -- Jon > > On 09/08/2014 10:41 AM, Guy Steele wrote: >> Good point, but counterpoint: it might be acceptable to have modified the string buffer in situations where throwing an exception would always cause the string buffer to become inaccessible. >> >> ?Guy >> >> On Sep 8, 2014, at 1:30 PM, Jonathan Gibbons wrote: >> >>> It would be inappropriate/incorrect to apply the optimization as described. >>> >>> The JLS requires that the argument to a method call should be computed before invoking the method. >>> >>> Consider the case when one of the expressions in the series of string concatenations throws an exception. It would be incorrect to have already partially modified the string buffer. >>> >>> -- Jon >>> >>> On 08/29/2014 01:53 PM, Ulf Zibis wrote: >>>> Hi compiler people, >>>> >>>> is there some chance that javac could be enhanced to optimize better as discussed in this thread? Than refactoring of up to now better readable code to ugly StringBuilder at append() code would be superfluous. >>>> I really like the String concatenation syntax, but unfortunately it often causes slower code and bigger footprint. >>>> Optimally javac would optimize mixed use of StringBuilder, StringJoiner, concatenation, toString(), append(String), append(Collection) etc. to a single StringBuilder instance, so that the resulting code, JITed or not, will have better performance. >>>> Additionally javac could guess a reasonable initial capacity from the given source code. >>>> >>>> >>>> Am 29.08.2014 um 10:01 schrieb Wang Weijun: >>>>> So it's not that the optimization fails but there is no optimization on them yet. >>>>> >>>>> I do see the .append("x") case will be easy to deal with, but it looks like historically javac has not been a place to do many optimizations. It mostly converts the java source to byte codes in a 1-to-1 mapping and let VM do whatever it wants (to optimize). When you talked about compiling multiple concatenation into using a single StringBuilder, it's more like choosing the correct implementation rather than an optimization. >>>>> >>>>> I don't expect to see big change on this in the near future, so shall we go on with the current enhancement? >>>>> >>>>> Thanks >>>>> Max >>>>> >>>>> On Aug 29, 2014, at 2:17, Ulf Zibis wrote: >>>>> >>>>>> I mean: >>>>>> It does not output byte code that only uses a single char array to compose the entire String in question. >>>>>> With "optimization fails", I also mean, there is used an additional "StringComposer" e.g. another StringBuilder or a StringJoiner in addition to the 1st StringBuilder. >>>>>> >>>>>> -Ulf >>>>>> >>>>>> Am 27.08.2014 um 14:02 schrieb Pavel Rappo: >>>>>>> Could you please explain what you mean by "javac optimization fails" here? >>>>>>> >>>>>>> -Pavel >>>>>>> >>>>>>> On 27 Aug 2014, at 10:41, Ulf Zibis wrote: >>>>>>> >>>>>>>> 4.) Now we see, that javac optimization fails again if StringBuilder, concatenation, toString(), append(String), append(Collection) etc. and StringJoiner use is mixed. > From Sergey.Bylokhov at oracle.com Mon Sep 8 18:39:44 2014 From: Sergey.Bylokhov at oracle.com (Sergey Bylokhov) Date: Mon, 08 Sep 2014 22:39:44 +0400 Subject: Optimization 2.0 for composing strings - Was: Replace concat String to append in StringBuilder parameters In-Reply-To: <540DEC69.8080600@oracle.com> References: <53E7FA44.9050207@oracle.com> <476830A4-7A8E-4652-A0C7-805687155F4A@oracle.com> <53E8CCD6.2090503@CoSoCo.de> <53FDA7C1.6060303@CoSoCo.de> <902B0B7A-32BE-498A-B9DD-8937960430A2@oracle.com> <53FF7244.2030804@CoSoCo.de> <5400E85E.9030005@CoSoCo.de> <540DE7B7.9070107@oracle.com> <61C40823-1A23-4A0C-BAFD-D1E5B949F98E@oracle.com> <540DEC69.8080600@oracle.com> Message-ID: <540DF7F0.4090604@oracle.com> On 08.09.2014 21:50, Jonathan Gibbons wrote: > Yes, but is this really a big enough performance and footprint pain > point to be worth addressing in javac itself? > > We're now talking about some specific code construction like > new StringBuilder().append(a + b + c) > > Any other case where the string builder can be observed externally > cannot be subject to the proposed optimization. The use case is now > really getting pretty specific. Not so specific at least in jdk code: http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-August/028142.html > > -- Jon > > On 09/08/2014 10:41 AM, Guy Steele wrote: >> Good point, but counterpoint: it might be acceptable to have modified >> the string buffer in situations where throwing an exception would >> always cause the string buffer to become inaccessible. >> >> ?Guy >> >> On Sep 8, 2014, at 1:30 PM, Jonathan Gibbons >> wrote: >> >>> It would be inappropriate/incorrect to apply the optimization as >>> described. >>> >>> The JLS requires that the argument to a method call should be >>> computed before invoking the method. >>> >>> Consider the case when one of the expressions in the series of >>> string concatenations throws an exception. It would be incorrect to >>> have already partially modified the string buffer. >>> >>> -- Jon >>> >>> On 08/29/2014 01:53 PM, Ulf Zibis wrote: >>>> Hi compiler people, >>>> >>>> is there some chance that javac could be enhanced to optimize >>>> better as discussed in this thread? Than refactoring of up to now >>>> better readable code to ugly StringBuilder at append() code would be >>>> superfluous. >>>> I really like the String concatenation syntax, but unfortunately it >>>> often causes slower code and bigger footprint. >>>> Optimally javac would optimize mixed use of StringBuilder, >>>> StringJoiner, concatenation, toString(), append(String), >>>> append(Collection) etc. to a single StringBuilder instance, so that >>>> the resulting code, JITed or not, will have better performance. >>>> Additionally javac could guess a reasonable initial capacity from >>>> the given source code. >>>> >>>> >>>> Am 29.08.2014 um 10:01 schrieb Wang Weijun: >>>>> So it's not that the optimization fails but there is no >>>>> optimization on them yet. >>>>> >>>>> I do see the .append("x") case will be easy to deal with, but it >>>>> looks like historically javac has not been a place to do many >>>>> optimizations. It mostly converts the java source to byte codes in >>>>> a 1-to-1 mapping and let VM do whatever it wants (to optimize). >>>>> When you talked about compiling multiple concatenation into using >>>>> a single StringBuilder, it's more like choosing the correct >>>>> implementation rather than an optimization. >>>>> >>>>> I don't expect to see big change on this in the near future, so >>>>> shall we go on with the current enhancement? >>>>> >>>>> Thanks >>>>> Max >>>>> >>>>> On Aug 29, 2014, at 2:17, Ulf Zibis wrote: >>>>> >>>>>> I mean: >>>>>> It does not output byte code that only uses a single char array >>>>>> to compose the entire String in question. >>>>>> With "optimization fails", I also mean, there is used an >>>>>> additional "StringComposer" e.g. another StringBuilder or a >>>>>> StringJoiner in addition to the 1st StringBuilder. >>>>>> >>>>>> -Ulf >>>>>> >>>>>> Am 27.08.2014 um 14:02 schrieb Pavel Rappo: >>>>>>> Could you please explain what you mean by "javac optimization >>>>>>> fails" here? >>>>>>> >>>>>>> -Pavel >>>>>>> >>>>>>> On 27 Aug 2014, at 10:41, Ulf Zibis wrote: >>>>>>> >>>>>>>> 4.) Now we see, that javac optimization fails again if >>>>>>>> StringBuilder, concatenation, toString(), append(String), >>>>>>>> append(Collection) etc. and StringJoiner use is mixed. > -- Best regards, Sergey. From jonathan.gibbons at oracle.com Mon Sep 8 18:53:27 2014 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 08 Sep 2014 11:53:27 -0700 Subject: Optimization 2.0 for composing strings - Was: Replace concat String to append in StringBuilder parameters In-Reply-To: <540DF7F0.4090604@oracle.com> References: <53E7FA44.9050207@oracle.com> <476830A4-7A8E-4652-A0C7-805687155F4A@oracle.com> <53E8CCD6.2090503@CoSoCo.de> <53FDA7C1.6060303@CoSoCo.de> <902B0B7A-32BE-498A-B9DD-8937960430A2@oracle.com> <53FF7244.2030804@CoSoCo.de> <5400E85E.9030005@CoSoCo.de> <540DE7B7.9070107@oracle.com> <61C40823-1A23-4A0C-BAFD-D1E5B949F98E@oracle.com> <540DEC69.8080600@oracle.com> <540DF7F0.4090604@oracle.com> Message-ID: <540DFB27.1030901@oracle.com> Sergey, Many of the suggestions in the webrev change the semantics of the code, and so would not be appropriate for javac to perform automagically. For example, in the first few lines of the patch, I found this: diff -r dde9f5cfde5f src/share/classes/sun/tools/jconsole/inspector/XArrayDataViewer.java --- a/src/share/classes/sun/tools/jconsole/inspector/XArrayDataViewer.java Tue Aug 05 19:29:00 2014 -0700 +++ b/src/share/classes/sun/tools/jconsole/inspector/XArrayDataViewer.java Sun Aug 10 18:02:01 2014 -0300 @@ -79,25 +79,24 @@ String textColor = String.format("%06x", foreground.getRGB() & 0xFFFFFF); StringBuilder sb = new StringBuilder(); - sb.append(""); + sb.append("
"); for (int i = 0; i < arr.length; i++) { if (i % 2 == 0) { - sb.append(""); + sb.append(""); Analyzing the flow to determine it is safe to mutate sb in the face of possible exceptions coming out of methods like htmlize is more than it would be reasonable to do in javac. For example, what if the for loop were in a try block and the try block referenced sb? Also, consider the serviceability implications, if a user is stepping through the code with a debugger. -- Jon On 09/08/2014 11:39 AM, Sergey Bylokhov wrote: > On 08.09.2014 21:50, Jonathan Gibbons wrote: >> Yes, but is this really a big enough performance and footprint pain >> point to be worth addressing in javac itself? >> >> We're now talking about some specific code construction like >> new StringBuilder().append(a + b + c) >> >> Any other case where the string builder can be observed externally >> cannot be subject to the proposed optimization. The use case is now >> really getting pretty specific. > Not so specific at least in jdk code: > http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-August/028142.html > >> >> -- Jon >> >> On 09/08/2014 10:41 AM, Guy Steele wrote: >>> Good point, but counterpoint: it might be acceptable to have >>> modified the string buffer in situations where throwing an exception >>> would always cause the string buffer to become inaccessible. >>> >>> ?Guy >>> >>> On Sep 8, 2014, at 1:30 PM, Jonathan Gibbons >>> wrote: >>> >>>> It would be inappropriate/incorrect to apply the optimization as >>>> described. >>>> >>>> The JLS requires that the argument to a method call should be >>>> computed before invoking the method. >>>> >>>> Consider the case when one of the expressions in the series of >>>> string concatenations throws an exception. It would be incorrect to >>>> have already partially modified the string buffer. >>>> >>>> -- Jon >>>> >>>> On 08/29/2014 01:53 PM, Ulf Zibis wrote: >>>>> Hi compiler people, >>>>> >>>>> is there some chance that javac could be enhanced to optimize >>>>> better as discussed in this thread? Than refactoring of up to now >>>>> better readable code to ugly StringBuilder at append() code would be >>>>> superfluous. >>>>> I really like the String concatenation syntax, but unfortunately >>>>> it often causes slower code and bigger footprint. >>>>> Optimally javac would optimize mixed use of StringBuilder, >>>>> StringJoiner, concatenation, toString(), append(String), >>>>> append(Collection) etc. to a single StringBuilder instance, so >>>>> that the resulting code, JITed or not, will have better performance. >>>>> Additionally javac could guess a reasonable initial capacity from >>>>> the given source code. >>>>> >>>>> >>>>> Am 29.08.2014 um 10:01 schrieb Wang Weijun: >>>>>> So it's not that the optimization fails but there is no >>>>>> optimization on them yet. >>>>>> >>>>>> I do see the .append("x") case will be easy to deal with, but it >>>>>> looks like historically javac has not been a place to do many >>>>>> optimizations. It mostly converts the java source to byte codes >>>>>> in a 1-to-1 mapping and let VM do whatever it wants (to >>>>>> optimize). When you talked about compiling multiple concatenation >>>>>> into using a single StringBuilder, it's more like choosing the >>>>>> correct implementation rather than an optimization. >>>>>> >>>>>> I don't expect to see big change on this in the near future, so >>>>>> shall we go on with the current enhancement? >>>>>> >>>>>> Thanks >>>>>> Max >>>>>> >>>>>> On Aug 29, 2014, at 2:17, Ulf Zibis wrote: >>>>>> >>>>>>> I mean: >>>>>>> It does not output byte code that only uses a single char array >>>>>>> to compose the entire String in question. >>>>>>> With "optimization fails", I also mean, there is used an >>>>>>> additional "StringComposer" e.g. another StringBuilder or a >>>>>>> StringJoiner in addition to the 1st StringBuilder. >>>>>>> >>>>>>> -Ulf >>>>>>> >>>>>>> Am 27.08.2014 um 14:02 schrieb Pavel Rappo: >>>>>>>> Could you please explain what you mean by "javac optimization >>>>>>>> fails" here? >>>>>>>> >>>>>>>> -Pavel >>>>>>>> >>>>>>>> On 27 Aug 2014, at 10:41, Ulf Zibis wrote: >>>>>>>> >>>>>>>>> 4.) Now we see, that javac optimization fails again if >>>>>>>>> StringBuilder, concatenation, toString(), append(String), >>>>>>>>> append(Collection) etc. and StringJoiner use is mixed. >> > > From vicente.romero at oracle.com Mon Sep 8 18:58:56 2014 From: vicente.romero at oracle.com (Vicente-Arturo Romero-Zaldivar) Date: Mon, 08 Sep 2014 11:58:56 -0700 Subject: Apparent bug in javac's dead code elimination In-Reply-To: <540DE6AD.2040703@oracle.com> References: <53E7FA44.9050207@oracle.com> <476830A4-7A8E-4652-A0C7-805687155F4A@oracle.com> <53E8CCD6.2090503@CoSoCo.de> <53FDA7C1.6060303@CoSoCo.de> <902B0B7A-32BE-498A-B9DD-8937960430A2@oracle.com> <53FF7244.2030804@CoSoCo.de> <5400E85E.9030005@CoSoCo.de> <540BCBCB.8010406@linux.com> <540DE6AD.2040703@oracle.com> Message-ID: <540DFC70.3060003@oracle.com> Hi Chris, Yes I agree with Jon, this is not a bug. I have been checking the patch for [1] see [2]. In the code you pasted Cake$1 has nothing to do with the inner class defined in the 'if (false)' sentence. It's just javac reusing an existing class name that would have been a 'real' class if the condition were different to false. What do you need this class for? You shouldn't need to deal with it. The problem is that we don't have a way to say: "hey this is not a common inner class this is just a token, exact term access constructor tag, to provide access to a private constructor". The only difference between a token and a 'common' inner class is that the token is empty, no methods in it. Thanks, Vicente [1] https://bugs.openjdk.java.net/browse/JDK-7199823 [2] http://hg.openjdk.java.net/jdk8/jdk8/langtools/rev/a51a8dac0a2f On 09/08/2014 10:26 AM, Jonathan Gibbons wrote: > I believe the existence of Cake$1 is part of javac's handling of > allowing any private code in SomeClass to access any private code in > SomeOtherClass. > > -- Jon > > On 09/06/2014 08:06 PM, Chris Kitching wrote: >> Greetings, >> >> As you probably know, javac does a bit of dead code elimintion: it'll >> get rid of redundant constructs like if (false) and suchlike. >> >> Consider, then, this ridiculous example: >> >> public class Cake { >> public void start() { >> if (false) { >> new Runnable() { >> @Override >> public void run() { >> // Whatever >> } >> }; >> } >> } >> >> private static class SomeClass { >> } >> } >> >> As you might expect, this produces two classes: Cake and >> Cake$SomeClass.class, and all is well. >> >> Now consider this slightly more ridiculous example: >> >> public class Cake { >> public void start() { >> if (false) { >> new Runnable() { >> @Override >> public void run() { >> // Whatever >> } >> }; >> } >> } >> >> private static class SomeClass { >> } >> >> private static class SomeOtherClass extends SomeClass { >> } >> } >> >> >> All I did was add an empty SomeOtherClass extending SomeClass. >> This example produces out these classes: >> >> Cake$1 >> Cake >> Cake$SomeClass >> Cake$SomeOtherClass >> >> Cake$1, the Runnable wrapped in `if (false)`, has suddenly appeared. >> Oddly, the bytecode in Cake$1 does not change as you alter the contents >> of run(). It's as if javac is omitting the contents of the methods of >> the class, but not the class itself (but it did so earlier before we >> added SomeOtherClass!) >> Cake$1 is also corrupt. Attempts to load the class using the Reflection >> API (because, you know, that's a sane thing to do) fail with a >> stacktrace like this: >> >> java.lang.ClassFormatError: Absent Code attribute in method that is not >> native or abstract in class file >> org/mozilla/gecko/animation/PropertyAnimator$1 >> at java.lang.ClassLoader.defineClass1(Native Method) >> at java.lang.ClassLoader.defineClass(ClassLoader.java:791) >> at >> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) >> at java.net.URLClassLoader.access$100(URLClassLoader.java:71) >> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) >> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:423) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:356) >> >> And that's exactly what just happened to Mozilla: >> https://bugzilla.mozilla.org/show_bug.cgi?id=1063991 >> >> (that also provides a bit of context, as well as the interesting >> consequence that the Cake$1 you get if you move the Runnable outside the >> if is different to the one you get in the situation described above >> (when, really, no Cake$1 at all should exist). >> >> Perhaps the desugaring step for inner classes is getting a little... >> overzealous? >> >> I've been able to reproduce this with javac 7 and 8. I've not tried it >> on the current JDK 9 snapshot. > From chriskitching at linux.com Mon Sep 8 19:27:03 2014 From: chriskitching at linux.com (Chris Kitching) Date: Mon, 08 Sep 2014 12:27:03 -0700 Subject: Apparent bug in javac's dead code elimination In-Reply-To: <540DFC70.3060003@oracle.com> References: <53E7FA44.9050207@oracle.com> <476830A4-7A8E-4652-A0C7-805687155F4A@oracle.com> <53E8CCD6.2090503@CoSoCo.de> <53FDA7C1.6060303@CoSoCo.de> <902B0B7A-32BE-498A-B9DD-8937960430A2@oracle.com> <53FF7244.2030804@CoSoCo.de> <5400E85E.9030005@CoSoCo.de> <540BCBCB.8010406@linux.com> <540DE6AD.2040703@oracle.com> <540DFC70.3060003@oracle.com> Message-ID: <540E0307.5000206@linux.com> Hello, "What do you need this class for?" I don't need this class for anything, I merely need the compiler to promise to never emit classes that are invalid. Surely that is something is should be expected to do? The current behaviour causes issues for tools that process all classes in a given directory. On 08/09/14 11:58, Vicente-Arturo Romero-Zaldivar wrote: > Hi Chris, > > Yes I agree with Jon, this is not a bug. I have been checking the patch > for [1] see [2]. In the code you pasted Cake$1 has nothing to do with > the inner class defined in the 'if (false)' sentence. It's just javac > reusing an existing class name that would have been a 'real' class if > the condition were different to false. What do you need this class for? > You shouldn't need to deal with it. The problem is that we don't have a > way to say: "hey this is not a common inner class this is just a token, > exact term access constructor tag, to provide access to a private > constructor". The only difference between a token and a 'common' inner > class is that the token is empty, no methods in it. > > Thanks, > Vicente > > [1] https://bugs.openjdk.java.net/browse/JDK-7199823 > [2] http://hg.openjdk.java.net/jdk8/jdk8/langtools/rev/a51a8dac0a2f > > > > On 09/08/2014 10:26 AM, Jonathan Gibbons wrote: >> I believe the existence of Cake$1 is part of javac's handling of >> allowing any private code in SomeClass to access any private code in >> SomeOtherClass. >> >> -- Jon >> >> On 09/06/2014 08:06 PM, Chris Kitching wrote: >>> Greetings, >>> >>> As you probably know, javac does a bit of dead code elimintion: it'll >>> get rid of redundant constructs like if (false) and suchlike. >>> >>> Consider, then, this ridiculous example: >>> >>> public class Cake { >>> public void start() { >>> if (false) { >>> new Runnable() { >>> @Override >>> public void run() { >>> // Whatever >>> } >>> }; >>> } >>> } >>> >>> private static class SomeClass { >>> } >>> } >>> >>> As you might expect, this produces two classes: Cake and >>> Cake$SomeClass.class, and all is well. >>> >>> Now consider this slightly more ridiculous example: >>> >>> public class Cake { >>> public void start() { >>> if (false) { >>> new Runnable() { >>> @Override >>> public void run() { >>> // Whatever >>> } >>> }; >>> } >>> } >>> >>> private static class SomeClass { >>> } >>> >>> private static class SomeOtherClass extends SomeClass { >>> } >>> } >>> >>> >>> All I did was add an empty SomeOtherClass extending SomeClass. >>> This example produces out these classes: >>> >>> Cake$1 >>> Cake >>> Cake$SomeClass >>> Cake$SomeOtherClass >>> >>> Cake$1, the Runnable wrapped in `if (false)`, has suddenly appeared. >>> Oddly, the bytecode in Cake$1 does not change as you alter the contents >>> of run(). It's as if javac is omitting the contents of the methods of >>> the class, but not the class itself (but it did so earlier before we >>> added SomeOtherClass!) >>> Cake$1 is also corrupt. Attempts to load the class using the Reflection >>> API (because, you know, that's a sane thing to do) fail with a >>> stacktrace like this: >>> >>> java.lang.ClassFormatError: Absent Code attribute in method that is not >>> native or abstract in class file >>> org/mozilla/gecko/animation/PropertyAnimator$1 >>> at java.lang.ClassLoader.defineClass1(Native Method) >>> at java.lang.ClassLoader.defineClass(ClassLoader.java:791) >>> at >>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) >>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71) >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:423) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:356) >>> >>> And that's exactly what just happened to Mozilla: >>> https://bugzilla.mozilla.org/show_bug.cgi?id=1063991 >>> >>> (that also provides a bit of context, as well as the interesting >>> consequence that the Cake$1 you get if you move the Runnable outside the >>> if is different to the one you get in the situation described above >>> (when, really, no Cake$1 at all should exist). >>> >>> Perhaps the desugaring step for inner classes is getting a little... >>> overzealous? >>> >>> I've been able to reproduce this with javac 7 and 8. I've not tried it >>> on the current JDK 9 snapshot. >> > From jonathan.gibbons at oracle.com Mon Sep 8 21:01:36 2014 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 08 Sep 2014 14:01:36 -0700 Subject: Apparent bug in javac's dead code elimination In-Reply-To: <540E0307.5000206@linux.com> References: <53E7FA44.9050207@oracle.com> <476830A4-7A8E-4652-A0C7-805687155F4A@oracle.com> <53E8CCD6.2090503@CoSoCo.de> <53FDA7C1.6060303@CoSoCo.de> <902B0B7A-32BE-498A-B9DD-8937960430A2@oracle.com> <53FF7244.2030804@CoSoCo.de> <5400E85E.9030005@CoSoCo.de> <540BCBCB.8010406@linux.com> <540DE6AD.2040703@oracle.com> <540DFC70.3060003@oracle.com> <540E0307.5000206@linux.com> Message-ID: <540E1930.1000800@oracle.com> Chris, I agree that javac should only generate valid class files, and that JVMS is quite explicit about when a Code attribute is required to be present or absent. I tried some simple experiments of my own and your Cake.java example. In neither case do I see any instances of a Code attribute being missing when required (or present when not.) In other words, I have not yet seen anything akin to what would give rise to the message you saw with the mozilla class: java.lang.ClassFormatError: Absent Code attribute in method that is not native or abstract in class file Given that it is not an error for javac to be generating an empty class file for its own internal purposes, do you have any additional evidence that javac is creating an invalid class file? I was using current builds of JDK 9 for my experiments; what version of JDK are you using? -- Jon On 09/08/2014 12:27 PM, Chris Kitching wrote: > Hello, > > "What do you need this class for?" > > I don't need this class for anything, I merely need the compiler to > promise to never emit classes that are invalid. Surely that is something > is should be expected to do? > The current behaviour causes issues for tools that process all classes > in a given directory. > > > On 08/09/14 11:58, Vicente-Arturo Romero-Zaldivar wrote: >> Hi Chris, >> >> Yes I agree with Jon, this is not a bug. I have been checking the patch >> for [1] see [2]. In the code you pasted Cake$1 has nothing to do with >> the inner class defined in the 'if (false)' sentence. It's just javac >> reusing an existing class name that would have been a 'real' class if >> the condition were different to false. What do you need this class for? >> You shouldn't need to deal with it. The problem is that we don't have a >> way to say: "hey this is not a common inner class this is just a token, >> exact term access constructor tag, to provide access to a private >> constructor". The only difference between a token and a 'common' inner >> class is that the token is empty, no methods in it. >> >> Thanks, >> Vicente >> >> [1] https://bugs.openjdk.java.net/browse/JDK-7199823 >> [2] http://hg.openjdk.java.net/jdk8/jdk8/langtools/rev/a51a8dac0a2f >> >> >> >> On 09/08/2014 10:26 AM, Jonathan Gibbons wrote: >>> I believe the existence of Cake$1 is part of javac's handling of >>> allowing any private code in SomeClass to access any private code in >>> SomeOtherClass. >>> >>> -- Jon >>> >>> On 09/06/2014 08:06 PM, Chris Kitching wrote: >>>> Greetings, >>>> >>>> As you probably know, javac does a bit of dead code elimintion: it'll >>>> get rid of redundant constructs like if (false) and suchlike. >>>> >>>> Consider, then, this ridiculous example: >>>> >>>> public class Cake { >>>> public void start() { >>>> if (false) { >>>> new Runnable() { >>>> @Override >>>> public void run() { >>>> // Whatever >>>> } >>>> }; >>>> } >>>> } >>>> >>>> private static class SomeClass { >>>> } >>>> } >>>> >>>> As you might expect, this produces two classes: Cake and >>>> Cake$SomeClass.class, and all is well. >>>> >>>> Now consider this slightly more ridiculous example: >>>> >>>> public class Cake { >>>> public void start() { >>>> if (false) { >>>> new Runnable() { >>>> @Override >>>> public void run() { >>>> // Whatever >>>> } >>>> }; >>>> } >>>> } >>>> >>>> private static class SomeClass { >>>> } >>>> >>>> private static class SomeOtherClass extends SomeClass { >>>> } >>>> } >>>> >>>> >>>> All I did was add an empty SomeOtherClass extending SomeClass. >>>> This example produces out these classes: >>>> >>>> Cake$1 >>>> Cake >>>> Cake$SomeClass >>>> Cake$SomeOtherClass >>>> >>>> Cake$1, the Runnable wrapped in `if (false)`, has suddenly appeared. >>>> Oddly, the bytecode in Cake$1 does not change as you alter the contents >>>> of run(). It's as if javac is omitting the contents of the methods of >>>> the class, but not the class itself (but it did so earlier before we >>>> added SomeOtherClass!) >>>> Cake$1 is also corrupt. Attempts to load the class using the Reflection >>>> API (because, you know, that's a sane thing to do) fail with a >>>> stacktrace like this: >>>> >>>> java.lang.ClassFormatError: Absent Code attribute in method that is not >>>> native or abstract in class file >>>> org/mozilla/gecko/animation/PropertyAnimator$1 >>>> at java.lang.ClassLoader.defineClass1(Native Method) >>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:791) >>>> at >>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >>>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) >>>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71) >>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) >>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >>>> at java.security.AccessController.doPrivileged(Native Method) >>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:423) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:356) >>>> >>>> And that's exactly what just happened to Mozilla: >>>> https://bugzilla.mozilla.org/show_bug.cgi?id=1063991 >>>> >>>> (that also provides a bit of context, as well as the interesting >>>> consequence that the Cake$1 you get if you move the Runnable outside the >>>> if is different to the one you get in the situation described above >>>> (when, really, no Cake$1 at all should exist). >>>> >>>> Perhaps the desugaring step for inner classes is getting a little... >>>> overzealous? >>>> >>>> I've been able to reproduce this with javac 7 and 8. I've not tried it >>>> on the current JDK 9 snapshot. From jonathan.gibbons at oracle.com Mon Sep 8 21:17:14 2014 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 08 Sep 2014 14:17:14 -0700 Subject: Apparent bug in javac's dead code elimination In-Reply-To: <540E1930.1000800@oracle.com> References: <53E7FA44.9050207@oracle.com> <476830A4-7A8E-4652-A0C7-805687155F4A@oracle.com> <53E8CCD6.2090503@CoSoCo.de> <53FDA7C1.6060303@CoSoCo.de> <902B0B7A-32BE-498A-B9DD-8937960430A2@oracle.com> <53FF7244.2030804@CoSoCo.de> <5400E85E.9030005@CoSoCo.de> <540BCBCB.8010406@linux.com> <540DE6AD.2040703@oracle.com> <540DFC70.3060003@oracle.com> <540E0307.5000206@linux.com> <540E1930.1000800@oracle.com> Message-ID: <540E1CDA.3060306@oracle.com> Chris, Update: I see the problem when using JDK 1.7.0. I do not see the problem using using JDK 1.8.0 or later. -- Jon On 09/08/2014 02:01 PM, Jonathan Gibbons wrote: > Chris, > > I agree that javac should only generate valid class files, and that JVMS > is quite explicit about when a Code attribute is required to be present > or absent. > > I tried some simple experiments of my own and your Cake.java example. > In neither case do I see any instances of a Code attribute being missing > when required (or present when not.) In other words, I have not yet seen > anything akin to what would give rise to the message you saw with the > mozilla class: > > java.lang.ClassFormatError: Absent Code attribute in method that is not > native or abstract in class file > > Given that it is not an error for javac to be generating an empty > class file > for its own internal purposes, do you have any additional evidence that > javac is creating an invalid class file? > > I was using current builds of JDK 9 for my experiments; what version of > JDK are you using? > > -- Jon > > On 09/08/2014 12:27 PM, Chris Kitching wrote: >> Hello, >> >> "What do you need this class for?" >> >> I don't need this class for anything, I merely need the compiler to >> promise to never emit classes that are invalid. Surely that is something >> is should be expected to do? >> The current behaviour causes issues for tools that process all classes >> in a given directory. >> >> >> On 08/09/14 11:58, Vicente-Arturo Romero-Zaldivar wrote: >>> Hi Chris, >>> >>> Yes I agree with Jon, this is not a bug. I have been checking the patch >>> for [1] see [2]. In the code you pasted Cake$1 has nothing to do with >>> the inner class defined in the 'if (false)' sentence. It's just javac >>> reusing an existing class name that would have been a 'real' class if >>> the condition were different to false. What do you need this class for? >>> You shouldn't need to deal with it. The problem is that we don't have a >>> way to say: "hey this is not a common inner class this is just a token, >>> exact term access constructor tag, to provide access to a private >>> constructor". The only difference between a token and a 'common' inner >>> class is that the token is empty, no methods in it. >>> >>> Thanks, >>> Vicente >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-7199823 >>> [2] http://hg.openjdk.java.net/jdk8/jdk8/langtools/rev/a51a8dac0a2f >>> >>> >>> >>> On 09/08/2014 10:26 AM, Jonathan Gibbons wrote: >>>> I believe the existence of Cake$1 is part of javac's handling of >>>> allowing any private code in SomeClass to access any private code in >>>> SomeOtherClass. >>>> >>>> -- Jon >>>> >>>> On 09/06/2014 08:06 PM, Chris Kitching wrote: >>>>> Greetings, >>>>> >>>>> As you probably know, javac does a bit of dead code elimintion: it'll >>>>> get rid of redundant constructs like if (false) and suchlike. >>>>> >>>>> Consider, then, this ridiculous example: >>>>> >>>>> public class Cake { >>>>> public void start() { >>>>> if (false) { >>>>> new Runnable() { >>>>> @Override >>>>> public void run() { >>>>> // Whatever >>>>> } >>>>> }; >>>>> } >>>>> } >>>>> >>>>> private static class SomeClass { >>>>> } >>>>> } >>>>> >>>>> As you might expect, this produces two classes: Cake and >>>>> Cake$SomeClass.class, and all is well. >>>>> >>>>> Now consider this slightly more ridiculous example: >>>>> >>>>> public class Cake { >>>>> public void start() { >>>>> if (false) { >>>>> new Runnable() { >>>>> @Override >>>>> public void run() { >>>>> // Whatever >>>>> } >>>>> }; >>>>> } >>>>> } >>>>> >>>>> private static class SomeClass { >>>>> } >>>>> >>>>> private static class SomeOtherClass extends SomeClass { >>>>> } >>>>> } >>>>> >>>>> >>>>> All I did was add an empty SomeOtherClass extending SomeClass. >>>>> This example produces out these classes: >>>>> >>>>> Cake$1 >>>>> Cake >>>>> Cake$SomeClass >>>>> Cake$SomeOtherClass >>>>> >>>>> Cake$1, the Runnable wrapped in `if (false)`, has suddenly appeared. >>>>> Oddly, the bytecode in Cake$1 does not change as you alter the >>>>> contents >>>>> of run(). It's as if javac is omitting the contents of the methods of >>>>> the class, but not the class itself (but it did so earlier before we >>>>> added SomeOtherClass!) >>>>> Cake$1 is also corrupt. Attempts to load the class using the >>>>> Reflection >>>>> API (because, you know, that's a sane thing to do) fail with a >>>>> stacktrace like this: >>>>> >>>>> java.lang.ClassFormatError: Absent Code attribute in method that >>>>> is not >>>>> native or abstract in class file >>>>> org/mozilla/gecko/animation/PropertyAnimator$1 >>>>> at java.lang.ClassLoader.defineClass1(Native Method) >>>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:791) >>>>> at >>>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >>>>> >>>>> at >>>>> java.net.URLClassLoader.defineClass(URLClassLoader.java:449) >>>>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71) >>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) >>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:423) >>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:356) >>>>> >>>>> And that's exactly what just happened to Mozilla: >>>>> https://bugzilla.mozilla.org/show_bug.cgi?id=1063991 >>>>> >>>>> (that also provides a bit of context, as well as the interesting >>>>> consequence that the Cake$1 you get if you move the Runnable >>>>> outside the >>>>> if is different to the one you get in the situation described above >>>>> (when, really, no Cake$1 at all should exist). >>>>> >>>>> Perhaps the desugaring step for inner classes is getting a little... >>>>> overzealous? >>>>> >>>>> I've been able to reproduce this with javac 7 and 8. I've not >>>>> tried it >>>>> on the current JDK 9 snapshot. > From chriskitching at linux.com Mon Sep 8 21:36:40 2014 From: chriskitching at linux.com (Chris Kitching) Date: Mon, 08 Sep 2014 14:36:40 -0700 Subject: Apparent bug in javac's dead code elimination In-Reply-To: <540E1CDA.3060306@oracle.com> References: <53E7FA44.9050207@oracle.com> <476830A4-7A8E-4652-A0C7-805687155F4A@oracle.com> <53E8CCD6.2090503@CoSoCo.de> <53FDA7C1.6060303@CoSoCo.de> <902B0B7A-32BE-498A-B9DD-8937960430A2@oracle.com> <53FF7244.2030804@CoSoCo.de> <5400E85E.9030005@CoSoCo.de> <540BCBCB.8010406@linux.com> <540DE6AD.2040703@oracle.com> <540DFC70.3060003@oracle.com> <540E0307.5000206@linux.com> <540E1930.1000800@oracle.com> <540E1CDA.3060306@oracle.com> Message-ID: <540E2168.5000903@linux.com> Ah! Apologies for not spotting that myself. Glad to know it's fixed in newer versions, anyway. Thanks for your time! On 08/09/14 14:17, Jonathan Gibbons wrote: > Chris, > > Update: > I see the problem when using JDK 1.7.0. > I do not see the problem using using JDK 1.8.0 or later. > > -- Jon > > On 09/08/2014 02:01 PM, Jonathan Gibbons wrote: >> Chris, >> >> I agree that javac should only generate valid class files, and that JVMS >> is quite explicit about when a Code attribute is required to be present >> or absent. >> >> I tried some simple experiments of my own and your Cake.java example. >> In neither case do I see any instances of a Code attribute being missing >> when required (or present when not.) In other words, I have not yet seen >> anything akin to what would give rise to the message you saw with the >> mozilla class: >> >> java.lang.ClassFormatError: Absent Code attribute in method that is not >> native or abstract in class file >> >> Given that it is not an error for javac to be generating an empty >> class file >> for its own internal purposes, do you have any additional evidence that >> javac is creating an invalid class file? >> >> I was using current builds of JDK 9 for my experiments; what version of >> JDK are you using? >> >> -- Jon >> >> On 09/08/2014 12:27 PM, Chris Kitching wrote: >>> Hello, >>> >>> "What do you need this class for?" >>> >>> I don't need this class for anything, I merely need the compiler to >>> promise to never emit classes that are invalid. Surely that is something >>> is should be expected to do? >>> The current behaviour causes issues for tools that process all classes >>> in a given directory. >>> >>> >>> On 08/09/14 11:58, Vicente-Arturo Romero-Zaldivar wrote: >>>> Hi Chris, >>>> >>>> Yes I agree with Jon, this is not a bug. I have been checking the patch >>>> for [1] see [2]. In the code you pasted Cake$1 has nothing to do with >>>> the inner class defined in the 'if (false)' sentence. It's just javac >>>> reusing an existing class name that would have been a 'real' class if >>>> the condition were different to false. What do you need this class for? >>>> You shouldn't need to deal with it. The problem is that we don't have a >>>> way to say: "hey this is not a common inner class this is just a token, >>>> exact term access constructor tag, to provide access to a private >>>> constructor". The only difference between a token and a 'common' inner >>>> class is that the token is empty, no methods in it. >>>> >>>> Thanks, >>>> Vicente >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-7199823 >>>> [2] http://hg.openjdk.java.net/jdk8/jdk8/langtools/rev/a51a8dac0a2f >>>> >>>> >>>> >>>> On 09/08/2014 10:26 AM, Jonathan Gibbons wrote: >>>>> I believe the existence of Cake$1 is part of javac's handling of >>>>> allowing any private code in SomeClass to access any private code in >>>>> SomeOtherClass. >>>>> >>>>> -- Jon >>>>> >>>>> On 09/06/2014 08:06 PM, Chris Kitching wrote: >>>>>> Greetings, >>>>>> >>>>>> As you probably know, javac does a bit of dead code elimintion: it'll >>>>>> get rid of redundant constructs like if (false) and suchlike. >>>>>> >>>>>> Consider, then, this ridiculous example: >>>>>> >>>>>> public class Cake { >>>>>> public void start() { >>>>>> if (false) { >>>>>> new Runnable() { >>>>>> @Override >>>>>> public void run() { >>>>>> // Whatever >>>>>> } >>>>>> }; >>>>>> } >>>>>> } >>>>>> >>>>>> private static class SomeClass { >>>>>> } >>>>>> } >>>>>> >>>>>> As you might expect, this produces two classes: Cake and >>>>>> Cake$SomeClass.class, and all is well. >>>>>> >>>>>> Now consider this slightly more ridiculous example: >>>>>> >>>>>> public class Cake { >>>>>> public void start() { >>>>>> if (false) { >>>>>> new Runnable() { >>>>>> @Override >>>>>> public void run() { >>>>>> // Whatever >>>>>> } >>>>>> }; >>>>>> } >>>>>> } >>>>>> >>>>>> private static class SomeClass { >>>>>> } >>>>>> >>>>>> private static class SomeOtherClass extends SomeClass { >>>>>> } >>>>>> } >>>>>> >>>>>> >>>>>> All I did was add an empty SomeOtherClass extending SomeClass. >>>>>> This example produces out these classes: >>>>>> >>>>>> Cake$1 >>>>>> Cake >>>>>> Cake$SomeClass >>>>>> Cake$SomeOtherClass >>>>>> >>>>>> Cake$1, the Runnable wrapped in `if (false)`, has suddenly appeared. >>>>>> Oddly, the bytecode in Cake$1 does not change as you alter the >>>>>> contents >>>>>> of run(). It's as if javac is omitting the contents of the methods of >>>>>> the class, but not the class itself (but it did so earlier before we >>>>>> added SomeOtherClass!) >>>>>> Cake$1 is also corrupt. Attempts to load the class using the >>>>>> Reflection >>>>>> API (because, you know, that's a sane thing to do) fail with a >>>>>> stacktrace like this: >>>>>> >>>>>> java.lang.ClassFormatError: Absent Code attribute in method that >>>>>> is not >>>>>> native or abstract in class file >>>>>> org/mozilla/gecko/animation/PropertyAnimator$1 >>>>>> at java.lang.ClassLoader.defineClass1(Native Method) >>>>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:791) >>>>>> at >>>>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >>>>>> >>>>>> at >>>>>> java.net.URLClassLoader.defineClass(URLClassLoader.java:449) >>>>>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71) >>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) >>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >>>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:423) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:356) >>>>>> >>>>>> And that's exactly what just happened to Mozilla: >>>>>> https://bugzilla.mozilla.org/show_bug.cgi?id=1063991 >>>>>> >>>>>> (that also provides a bit of context, as well as the interesting >>>>>> consequence that the Cake$1 you get if you move the Runnable >>>>>> outside the >>>>>> if is different to the one you get in the situation described above >>>>>> (when, really, no Cake$1 at all should exist). >>>>>> >>>>>> Perhaps the desugaring step for inner classes is getting a little... >>>>>> overzealous? >>>>>> >>>>>> I've been able to reproduce this with javac 7 and 8. I've not >>>>>> tried it >>>>>> on the current JDK 9 snapshot. >> > From vicente.romero at oracle.com Mon Sep 8 21:55:33 2014 From: vicente.romero at oracle.com (Vicente-Arturo Romero-Zaldivar) Date: Mon, 08 Sep 2014 14:55:33 -0700 Subject: Apparent bug in javac's dead code elimination In-Reply-To: <540E2168.5000903@linux.com> References: <53E7FA44.9050207@oracle.com> <476830A4-7A8E-4652-A0C7-805687155F4A@oracle.com> <53E8CCD6.2090503@CoSoCo.de> <53FDA7C1.6060303@CoSoCo.de> <902B0B7A-32BE-498A-B9DD-8937960430A2@oracle.com> <53FF7244.2030804@CoSoCo.de> <5400E85E.9030005@CoSoCo.de> <540BCBCB.8010406@linux.com> <540DE6AD.2040703@oracle.com> <540DFC70.3060003@oracle.com> <540E0307.5000206@linux.com> <540E1930.1000800@oracle.com> <540E1CDA.3060306@oracle.com> <540E2168.5000903@linux.com> Message-ID: <540E25D5.40402@oracle.com> Bug closed as not an issue. Vicente On 09/08/2014 02:36 PM, Chris Kitching wrote: > Ah! > > Apologies for not spotting that myself. > Glad to know it's fixed in newer versions, anyway. Thanks for your time! > > On 08/09/14 14:17, Jonathan Gibbons wrote: >> Chris, >> >> Update: >> I see the problem when using JDK 1.7.0. >> I do not see the problem using using JDK 1.8.0 or later. >> >> -- Jon >> >> On 09/08/2014 02:01 PM, Jonathan Gibbons wrote: >>> Chris, >>> >>> I agree that javac should only generate valid class files, and that JVMS >>> is quite explicit about when a Code attribute is required to be present >>> or absent. >>> >>> I tried some simple experiments of my own and your Cake.java example. >>> In neither case do I see any instances of a Code attribute being missing >>> when required (or present when not.) In other words, I have not yet seen >>> anything akin to what would give rise to the message you saw with the >>> mozilla class: >>> >>> java.lang.ClassFormatError: Absent Code attribute in method that is not >>> native or abstract in class file >>> >>> Given that it is not an error for javac to be generating an empty >>> class file >>> for its own internal purposes, do you have any additional evidence that >>> javac is creating an invalid class file? >>> >>> I was using current builds of JDK 9 for my experiments; what version of >>> JDK are you using? >>> >>> -- Jon >>> >>> On 09/08/2014 12:27 PM, Chris Kitching wrote: >>>> Hello, >>>> >>>> "What do you need this class for?" >>>> >>>> I don't need this class for anything, I merely need the compiler to >>>> promise to never emit classes that are invalid. Surely that is something >>>> is should be expected to do? >>>> The current behaviour causes issues for tools that process all classes >>>> in a given directory. >>>> >>>> >>>> On 08/09/14 11:58, Vicente-Arturo Romero-Zaldivar wrote: >>>>> Hi Chris, >>>>> >>>>> Yes I agree with Jon, this is not a bug. I have been checking the patch >>>>> for [1] see [2]. In the code you pasted Cake$1 has nothing to do with >>>>> the inner class defined in the 'if (false)' sentence. It's just javac >>>>> reusing an existing class name that would have been a 'real' class if >>>>> the condition were different to false. What do you need this class for? >>>>> You shouldn't need to deal with it. The problem is that we don't have a >>>>> way to say: "hey this is not a common inner class this is just a token, >>>>> exact term access constructor tag, to provide access to a private >>>>> constructor". The only difference between a token and a 'common' inner >>>>> class is that the token is empty, no methods in it. >>>>> >>>>> Thanks, >>>>> Vicente >>>>> >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-7199823 >>>>> [2] http://hg.openjdk.java.net/jdk8/jdk8/langtools/rev/a51a8dac0a2f >>>>> >>>>> >>>>> >>>>> On 09/08/2014 10:26 AM, Jonathan Gibbons wrote: >>>>>> I believe the existence of Cake$1 is part of javac's handling of >>>>>> allowing any private code in SomeClass to access any private code in >>>>>> SomeOtherClass. >>>>>> >>>>>> -- Jon >>>>>> >>>>>> On 09/06/2014 08:06 PM, Chris Kitching wrote: >>>>>>> Greetings, >>>>>>> >>>>>>> As you probably know, javac does a bit of dead code elimintion: it'll >>>>>>> get rid of redundant constructs like if (false) and suchlike. >>>>>>> >>>>>>> Consider, then, this ridiculous example: >>>>>>> >>>>>>> public class Cake { >>>>>>> public void start() { >>>>>>> if (false) { >>>>>>> new Runnable() { >>>>>>> @Override >>>>>>> public void run() { >>>>>>> // Whatever >>>>>>> } >>>>>>> }; >>>>>>> } >>>>>>> } >>>>>>> >>>>>>> private static class SomeClass { >>>>>>> } >>>>>>> } >>>>>>> >>>>>>> As you might expect, this produces two classes: Cake and >>>>>>> Cake$SomeClass.class, and all is well. >>>>>>> >>>>>>> Now consider this slightly more ridiculous example: >>>>>>> >>>>>>> public class Cake { >>>>>>> public void start() { >>>>>>> if (false) { >>>>>>> new Runnable() { >>>>>>> @Override >>>>>>> public void run() { >>>>>>> // Whatever >>>>>>> } >>>>>>> }; >>>>>>> } >>>>>>> } >>>>>>> >>>>>>> private static class SomeClass { >>>>>>> } >>>>>>> >>>>>>> private static class SomeOtherClass extends SomeClass { >>>>>>> } >>>>>>> } >>>>>>> >>>>>>> >>>>>>> All I did was add an empty SomeOtherClass extending SomeClass. >>>>>>> This example produces out these classes: >>>>>>> >>>>>>> Cake$1 >>>>>>> Cake >>>>>>> Cake$SomeClass >>>>>>> Cake$SomeOtherClass >>>>>>> >>>>>>> Cake$1, the Runnable wrapped in `if (false)`, has suddenly appeared. >>>>>>> Oddly, the bytecode in Cake$1 does not change as you alter the >>>>>>> contents >>>>>>> of run(). It's as if javac is omitting the contents of the methods of >>>>>>> the class, but not the class itself (but it did so earlier before we >>>>>>> added SomeOtherClass!) >>>>>>> Cake$1 is also corrupt. Attempts to load the class using the >>>>>>> Reflection >>>>>>> API (because, you know, that's a sane thing to do) fail with a >>>>>>> stacktrace like this: >>>>>>> >>>>>>> java.lang.ClassFormatError: Absent Code attribute in method that >>>>>>> is not >>>>>>> native or abstract in class file >>>>>>> org/mozilla/gecko/animation/PropertyAnimator$1 >>>>>>> at java.lang.ClassLoader.defineClass1(Native Method) >>>>>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:791) >>>>>>> at >>>>>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >>>>>>> >>>>>>> at >>>>>>> java.net.URLClassLoader.defineClass(URLClassLoader.java:449) >>>>>>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71) >>>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) >>>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >>>>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:423) >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:356) >>>>>>> >>>>>>> And that's exactly what just happened to Mozilla: >>>>>>> https://bugzilla.mozilla.org/show_bug.cgi?id=1063991 >>>>>>> >>>>>>> (that also provides a bit of context, as well as the interesting >>>>>>> consequence that the Cake$1 you get if you move the Runnable >>>>>>> outside the >>>>>>> if is different to the one you get in the situation described above >>>>>>> (when, really, no Cake$1 at all should exist). >>>>>>> >>>>>>> Perhaps the desugaring step for inner classes is getting a little... >>>>>>> overzealous? >>>>>>> >>>>>>> I've been able to reproduce this with javac 7 and 8. I've not >>>>>>> tried it >>>>>>> on the current JDK 9 snapshot. From Ulf.Zibis at CoSoCo.de Mon Sep 8 22:56:54 2014 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Tue, 09 Sep 2014 00:56:54 +0200 Subject: Optimization 2.0 for composing strings - Was: Replace concat String to append in StringBuilder parameters In-Reply-To: <540DB05C.3010300@oracle.com> References: <53E7FA44.9050207@oracle.com> <476830A4-7A8E-4652-A0C7-805687155F4A@oracle.com> <53E8CCD6.2090503@CoSoCo.de> <53FDA7C1.6060303@CoSoCo.de> <902B0B7A-32BE-498A-B9DD-8937960430A2@oracle.com> <53FF7244.2030804@CoSoCo.de> <5400E85E.9030005@CoSoCo.de> <540DB05C.3010300@oracle.com> Message-ID: <540E3436.2060609@CoSoCo.de> Thanks for the interesting pointers! -Ulf Am 08.09.2014 um 15:34 schrieb Sergey Bylokhov: > There was related discussions on the past > http://mail.openjdk.java.net/pipermail/compiler-dev/2014-February/008491.html > And related issues: > https://bugs.openjdk.java.net/browse/JDK-4947460 > https://bugs.openjdk.java.net/browse/JDK-4059189 > https://bugs.openjdk.java.net/browse/JDK-6709423 From lowasser at google.com Mon Sep 8 22:58:05 2014 From: lowasser at google.com (Louis Wasserman) Date: Mon, 8 Sep 2014 15:58:05 -0700 Subject: Optimization 2.0 for composing strings - Was: Replace concat String to append in StringBuilder parameters In-Reply-To: <540DEC69.8080600@oracle.com> References: <53E7FA44.9050207@oracle.com> <476830A4-7A8E-4652-A0C7-805687155F4A@oracle.com> <53E8CCD6.2090503@CoSoCo.de> <53FDA7C1.6060303@CoSoCo.de> <902B0B7A-32BE-498A-B9DD-8937960430A2@oracle.com> <53FF7244.2030804@CoSoCo.de> <5400E85E.9030005@CoSoCo.de> <540DE7B7.9070107@oracle.com> <61C40823-1A23-4A0C-BAFD-D1E5B949F98E@oracle.com> <540DEC69.8080600@oracle.com> Message-ID: FWIW, Google has gotten noticeable performance benefits from a change to javac's compilation of normal + concatenation, to just presize the StringBuilder. I haven't had the bandwidth to forward-port that change yet, unfortunately, but that's a semantics-preserving change that seemed to us to be within spec. On Mon, Sep 8, 2014 at 10:50 AM, Jonathan Gibbons < jonathan.gibbons at oracle.com> wrote: > Yes, but is this really a big enough performance and footprint pain point > to be worth addressing in javac itself? > > We're now talking about some specific code construction like > new StringBuilder().append(a + b + c) > > Any other case where the string builder can be observed externally cannot > be subject to the proposed optimization. The use case is now really getting > pretty specific. > > -- Jon > > > On 09/08/2014 10:41 AM, Guy Steele wrote: > >> Good point, but counterpoint: it might be acceptable to have modified the >> string buffer in situations where throwing an exception would always cause >> the string buffer to become inaccessible. >> >> ?Guy >> >> On Sep 8, 2014, at 1:30 PM, Jonathan Gibbons >> wrote: >> >> It would be inappropriate/incorrect to apply the optimization as >>> described. >>> >>> The JLS requires that the argument to a method call should be computed >>> before invoking the method. >>> >>> Consider the case when one of the expressions in the series of string >>> concatenations throws an exception. It would be incorrect to have already >>> partially modified the string buffer. >>> >>> -- Jon >>> >>> On 08/29/2014 01:53 PM, Ulf Zibis wrote: >>> >>>> Hi compiler people, >>>> >>>> is there some chance that javac could be enhanced to optimize better as >>>> discussed in this thread? Than refactoring of up to now better readable >>>> code to ugly StringBuilder at append() code would be superfluous. >>>> I really like the String concatenation syntax, but unfortunately it >>>> often causes slower code and bigger footprint. >>>> Optimally javac would optimize mixed use of StringBuilder, >>>> StringJoiner, concatenation, toString(), append(String), append(Collection) >>>> etc. to a single StringBuilder instance, so that the resulting code, JITed >>>> or not, will have better performance. >>>> Additionally javac could guess a reasonable initial capacity from the >>>> given source code. >>>> >>>> >>>> Am 29.08.2014 um 10:01 schrieb Wang Weijun: >>>> >>>>> So it's not that the optimization fails but there is no optimization >>>>> on them yet. >>>>> >>>>> I do see the .append("x") case will be easy to deal with, but it looks >>>>> like historically javac has not been a place to do many optimizations. It >>>>> mostly converts the java source to byte codes in a 1-to-1 mapping and let >>>>> VM do whatever it wants (to optimize). When you talked about compiling >>>>> multiple concatenation into using a single StringBuilder, it's more like >>>>> choosing the correct implementation rather than an optimization. >>>>> >>>>> I don't expect to see big change on this in the near future, so shall >>>>> we go on with the current enhancement? >>>>> >>>>> Thanks >>>>> Max >>>>> >>>>> On Aug 29, 2014, at 2:17, Ulf Zibis wrote: >>>>> >>>>> I mean: >>>>>> It does not output byte code that only uses a single char array to >>>>>> compose the entire String in question. >>>>>> With "optimization fails", I also mean, there is used an additional >>>>>> "StringComposer" e.g. another StringBuilder or a StringJoiner in addition >>>>>> to the 1st StringBuilder. >>>>>> >>>>>> -Ulf >>>>>> >>>>>> Am 27.08.2014 um 14:02 schrieb Pavel Rappo: >>>>>> >>>>>>> Could you please explain what you mean by "javac optimization fails" >>>>>>> here? >>>>>>> >>>>>>> -Pavel >>>>>>> >>>>>>> On 27 Aug 2014, at 10:41, Ulf Zibis wrote: >>>>>>> >>>>>>> 4.) Now we see, that javac optimization fails again if >>>>>>>> StringBuilder, concatenation, toString(), append(String), >>>>>>>> append(Collection) etc. and StringJoiner use is mixed. >>>>>>>> >>>>>>> > -- Louis Wasserman -------------- next part -------------- An HTML attachment was scrubbed... URL: From Ulf.Zibis at CoSoCo.de Tue Sep 9 10:28:47 2014 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Tue, 09 Sep 2014 12:28:47 +0200 Subject: Optimization 2.0 for composing strings - Was: Replace concat String to append in StringBuilder parameters In-Reply-To: <540DFB27.1030901@oracle.com> References: <53E7FA44.9050207@oracle.com> <476830A4-7A8E-4652-A0C7-805687155F4A@oracle.com> <53E8CCD6.2090503@CoSoCo.de> <53FDA7C1.6060303@CoSoCo.de> <902B0B7A-32BE-498A-B9DD-8937960430A2@oracle.com> <53FF7244.2030804@CoSoCo.de> <5400E85E.9030005@CoSoCo.de> <540DE7B7.9070107@oracle.com> <61C40823-1A23-4A0C-BAFD-D1E5B949F98E@oracle.com> <540DEC69.8080600@oracle.com> <540DF7F0.4090604@oracle.com> <540DFB27.1030901@oracle.com> Message-ID: <540ED65F.9010808@CoSoCo.de> Am 08.09.2014 um 20:53 schrieb Jonathan Gibbons: > For example, in the first few lines of the patch, I found this: Do you see any semantics change here? > diff -r dde9f5cfde5f src/share/classes/sun/tools/jconsole/inspector/XArrayDataViewer.java > --- a/src/share/classes/sun/tools/jconsole/inspector/XArrayDataViewer.java Tue Aug 05 19:29:00 > 2014 -0700 > +++ b/src/share/classes/sun/tools/jconsole/inspector/XArrayDataViewer.java Sun Aug 10 18:02:01 > 2014 -0300 > @@ -79,25 +79,24 @@ > String textColor = String.format("%06x", > foreground.getRGB() & 0xFFFFFF); > StringBuilder sb = new StringBuilder(); > - sb.append("
" +
-                            (arr[i] == null ?
-                                arr[i] : htmlize(arr[i].toString())) +
-                            "
")
+                            .append(arr[i] == null ?
+                                    arr[i] : htmlize(arr[i].toString()))
+                            .append("
"); > + sb.append("
"); > for (int i = 0; i < arr.length; i++) { > if (i % 2 == 0) { > - sb.append(" - evenRowColorStr + "\">"); > + sb.append(" + .append(evenRowColorStr).append("\">"); In this special case javac optimizer could first interpret as (as same as we could help it by just coding like that): StringBuilder sb = new StringBuilder( "
" +
> -                            (arr[i] == null ?
> -                                arr[i] : htmlize(arr[i].toString())) +
> - "
")
> +                            .append(arr[i] == null ?
> +                                    arr[i] : htmlize(arr[i].toString()))
> + .append("
"); so no need for an additional implicit StringBuilder. Additionally javac could guess a reasonable capacity first. > Analyzing the flow to determine it is safe to mutate sb in the face of possible exceptions coming > out of methods like htmlize is more than it would be reasonable to do in javac. For example, what > if the for loop were in a try block and the try block referenced sb? Good example where such optimization would fail. But: - manual optimization would fail either - try catch block anyway would have it's own performance cost, maybe more than the additional StringBuilder. Maybe it's not such complicated as we guess to find out if the affected StringBuilder becomes reliably unreachable in case of exception, and therefore securely could be "mutated". > Also, consider the serviceability implications, if a user is stepping through the code with a > debugger. Good point. But I think, we are already used with this when stepping through simple String concatenation or wondering on a StringBuilder in exception stack trace, even there is no StringBuilder in our code. -Ulf From Sergey.Bylokhov at oracle.com Tue Sep 9 11:47:50 2014 From: Sergey.Bylokhov at oracle.com (Sergey Bylokhov) Date: Tue, 09 Sep 2014 15:47:50 +0400 Subject: Optimization 2.0 for composing strings - Was: Replace concat String to append in StringBuilder parameters In-Reply-To: <540DFB27.1030901@oracle.com> References: <53E7FA44.9050207@oracle.com> <476830A4-7A8E-4652-A0C7-805687155F4A@oracle.com> <53E8CCD6.2090503@CoSoCo.de> <53FDA7C1.6060303@CoSoCo.de> <902B0B7A-32BE-498A-B9DD-8937960430A2@oracle.com> <53FF7244.2030804@CoSoCo.de> <5400E85E.9030005@CoSoCo.de> <540DE7B7.9070107@oracle.com> <61C40823-1A23-4A0C-BAFD-D1E5B949F98E@oracle.com> <540DEC69.8080600@oracle.com> <540DF7F0.4090604@oracle.com> <540DFB27.1030901@oracle.com> Message-ID: <540EE8E6.3040801@oracle.com> On 08.09.2014 22:53, Jonathan Gibbons wrote: > Sergey, > > Many of the suggestions in the webrev change the semantics of the > code, and so would not be appropriate for javac to perform automagically. It is changed but I guess it is safe otherwise this ccan be a problem in the fix, but the fix still under review. > Analyzing the flow to determine it is safe to mutate sb in the face of > possible exceptions coming out of methods like htmlize is more than it > would be reasonable to do in javac. For example, what if the for loop > were in a try block and the try block referenced sb? It depends how many man's time spend on such changes, which can be done by the javac itself. I see that there is a lot of review requests to make some small micro/size optimizations, which can be done automatically. > > Also, consider the serviceability implications, if a user is stepping > through the code with a debugger. > > -- Jon > > > On 09/08/2014 11:39 AM, Sergey Bylokhov wrote: >> On 08.09.2014 21:50, Jonathan Gibbons wrote: >>> Yes, but is this really a big enough performance and footprint pain >>> point to be worth addressing in javac itself? >>> >>> We're now talking about some specific code construction like >>> new StringBuilder().append(a + b + c) >>> >>> Any other case where the string builder can be observed externally >>> cannot be subject to the proposed optimization. The use case is now >>> really getting pretty specific. >> Not so specific at least in jdk code: >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2014-August/028142.html >> >>> >>> -- Jon >>> >>> On 09/08/2014 10:41 AM, Guy Steele wrote: >>>> Good point, but counterpoint: it might be acceptable to have >>>> modified the string buffer in situations where throwing an >>>> exception would always cause the string buffer to become inaccessible. >>>> >>>> ?Guy >>>> >>>> On Sep 8, 2014, at 1:30 PM, Jonathan Gibbons >>>> wrote: >>>> >>>>> It would be inappropriate/incorrect to apply the optimization as >>>>> described. >>>>> >>>>> The JLS requires that the argument to a method call should be >>>>> computed before invoking the method. >>>>> >>>>> Consider the case when one of the expressions in the series of >>>>> string concatenations throws an exception. It would be incorrect >>>>> to have already partially modified the string buffer. >>>>> >>>>> -- Jon >>>>> >>>>> On 08/29/2014 01:53 PM, Ulf Zibis wrote: >>>>>> Hi compiler people, >>>>>> >>>>>> is there some chance that javac could be enhanced to optimize >>>>>> better as discussed in this thread? Than refactoring of up to now >>>>>> better readable code to ugly StringBuilder at append() code would be >>>>>> superfluous. >>>>>> I really like the String concatenation syntax, but unfortunately >>>>>> it often causes slower code and bigger footprint. >>>>>> Optimally javac would optimize mixed use of StringBuilder, >>>>>> StringJoiner, concatenation, toString(), append(String), >>>>>> append(Collection) etc. to a single StringBuilder instance, so >>>>>> that the resulting code, JITed or not, will have better performance. >>>>>> Additionally javac could guess a reasonable initial capacity from >>>>>> the given source code. >>>>>> >>>>>> >>>>>> Am 29.08.2014 um 10:01 schrieb Wang Weijun: >>>>>>> So it's not that the optimization fails but there is no >>>>>>> optimization on them yet. >>>>>>> >>>>>>> I do see the .append("x") case will be easy to deal with, but it >>>>>>> looks like historically javac has not been a place to do many >>>>>>> optimizations. It mostly converts the java source to byte codes >>>>>>> in a 1-to-1 mapping and let VM do whatever it wants (to >>>>>>> optimize). When you talked about compiling multiple >>>>>>> concatenation into using a single StringBuilder, it's more like >>>>>>> choosing the correct implementation rather than an optimization. >>>>>>> >>>>>>> I don't expect to see big change on this in the near future, so >>>>>>> shall we go on with the current enhancement? >>>>>>> >>>>>>> Thanks >>>>>>> Max >>>>>>> >>>>>>> On Aug 29, 2014, at 2:17, Ulf Zibis wrote: >>>>>>> >>>>>>>> I mean: >>>>>>>> It does not output byte code that only uses a single char array >>>>>>>> to compose the entire String in question. >>>>>>>> With "optimization fails", I also mean, there is used an >>>>>>>> additional "StringComposer" e.g. another StringBuilder or a >>>>>>>> StringJoiner in addition to the 1st StringBuilder. >>>>>>>> >>>>>>>> -Ulf >>>>>>>> >>>>>>>> Am 27.08.2014 um 14:02 schrieb Pavel Rappo: >>>>>>>>> Could you please explain what you mean by "javac optimization >>>>>>>>> fails" here? >>>>>>>>> >>>>>>>>> -Pavel >>>>>>>>> >>>>>>>>> On 27 Aug 2014, at 10:41, Ulf Zibis wrote: >>>>>>>>> >>>>>>>>>> 4.) Now we see, that javac optimization fails again if >>>>>>>>>> StringBuilder, concatenation, toString(), append(String), >>>>>>>>>> append(Collection) etc. and StringJoiner use is mixed. >>> >> >> > -- Best regards, Sergey. From paul.govereau at oracle.com Wed Sep 10 21:10:48 2014 From: paul.govereau at oracle.com (Paul Govereau) Date: Wed, 10 Sep 2014 17:10:48 -0400 Subject: Compiling large switch statements Message-ID: <5410BE58.7060005@oracle.com> Hello, I have been looking at the javac code generator, and I think we may be doing a poor job of compiling large switch statements because of an overly conservative estimate of the cost of the lookupswitch instruction. Because we over estimate the cost of a lookupswitch, we can generate very large tableswitch instructions: twice the size or larger than necessary. Briefly, javac estimates the total cost of a switch instruction to be: space + 3 * time, and generates the lowest cost instruction. However, javac assumes the time cost of a lookupswitch is linear, so we greatly favor the tableswitch. (For reference, the relevant code is in the visitSwitch method in jvm.Gen). As an example, consider a large switch containing 50 cases. For the tableswitch, the important factor is the range of the matched values, because this determines the size of the array needed for the table of addresses. Let's call this size S, then the cost of a tableswitch is estimated as: size (words): 4 + S + 1 computation: 3 steps (2 comparisons, 1 load) For the same code the cost for a lookupswitch is estimated as: size(words): 3 + 2 * 50 computation: 50 steps (presumably 50 comparisons?) Note that the lookupswitch match-offset pairs must be sorted. So, it stands to reason that to implementation is using (at least) a binary search on the sorted list of keys. Therefore, the computation cost must be closer to 2*log(50), no? With the current metrics, we allow a tableswitch instruction with 50 items to be as large as 976 bytes before we change to the much smaller 412 byte lookupswitch. It seems to me the log-estimate is more accurate and we should change to the lookupswitch when the tableswitch reaches 468 bytes. However, I am only guessing as to how these bytecodes are implemented. Does anyone on the list know the details? Paul From john.r.rose at oracle.com Wed Sep 10 22:11:06 2014 From: john.r.rose at oracle.com (John Rose) Date: Wed, 10 Sep 2014 15:11:06 -0700 Subject: Compiling large switch statements In-Reply-To: <5410BE58.7060005@oracle.com> References: <5410BE58.7060005@oracle.com> Message-ID: On Sep 10, 2014, at 2:10 PM, Paul Govereau wrote: > Hello, > > I have been looking at the javac code generator, and I think we may be > doing a poor job of compiling large switch statements because of an > overly conservative estimate of the cost of the lookupswitch > instruction. Because we over estimate the cost of a lookupswitch, we > can generate very large tableswitch instructions: twice the size or > larger than necessary. > > Briefly, javac estimates the total cost of a switch instruction to be: > > space + 3 * time, > > and generates the lowest cost instruction. However, javac assumes the > time cost of a lookupswitch is linear, so we greatly favor the > tableswitch. (For reference, the relevant code is in the visitSwitch > method in jvm.Gen). > > As an example, consider a large switch containing 50 cases. For the > tableswitch, the important factor is the range of the matched values, > because this determines the size of the array needed for the table of > addresses. Let's call this size S, then the cost of a tableswitch is > estimated as: > > size (words): 4 + S + 1 > computation: 3 steps (2 comparisons, 1 load) > > For the same code the cost for a lookupswitch is estimated as: > > size(words): 3 + 2 * 50 > computation: 50 steps (presumably 50 comparisons?) > > Note that the lookupswitch match-offset pairs must be sorted. So, it > stands to reason that to implementation is using (at least) a binary > search on the sorted list of keys. Therefore, the computation cost > must be closer to 2*log(50), no? Yes. Log(#cases) is a far better estimate. > With the current metrics, we allow a tableswitch instruction with 50 > items to be as large as 976 bytes before we change to the much smaller > 412 byte lookupswitch. It seems to me the log-estimate is more > accurate and we should change to the lookupswitch when the tableswitch > reaches 468 bytes. However, I am only guessing as to how these > bytecodes are implemented. Does anyone on the list know the details? I knew a few, now I know more... funny how curiosity works. The C2 JIT reorganizes lookupswitch and tableswitch instructions from scratch, using its own notions of what is efficient. You end up with a decision tree and/or some PC jump blocks, but you can end up with a mix of both from either instruction. The C1 JIT reorganizes the instructions also, detecting key ranges (runs of common branch targets) and handling them with 1-2 comparisons per range. Oddly, it does not bother to put a decision tree on top of this, nor does it attempt jump tables. The assembly interpreter uses a binary search on lookupswitch. The C++ template interpreter, which is rarely used, does a simple linear search over a lookupswitch, so that is the only component that is as bad as you feared. I suggest using brute classfile size as your metric for choosing your switch. It will never be far off from the model you propose, and the differences won't matter once the JIT gets hold of the code. As a bonus, you'll slightly decrease the size of jars, which surely counts for something (you don't even have to run the code to collect your wins). ? John From brian.goetz at oracle.com Thu Sep 11 14:09:23 2014 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 11 Sep 2014 10:09:23 -0400 Subject: Compiling large switch statements In-Reply-To: References: <5410BE58.7060005@oracle.com> Message-ID: <5411AD13.4090205@oracle.com> > I suggest using brute classfile size as your metric for choosing your > switch. It will never be far off from the model you propose, and the > differences won't matter once the JIT gets hold of the code. As a > bonus, you'll slightly decrease the size of jars, which surely counts > for something (you don't even have to run the code to collect your > wins). Our friends in the ME world would be happy if we did this. From jhs at edg.com Thu Sep 11 15:53:35 2014 From: jhs at edg.com (John Spicer) Date: Thu, 11 Sep 2014 11:53:35 -0400 Subject: I.super.f() usage in Java 8 Message-ID: <166300A4-0BE2-4264-AAA4-1EE3A0C27C46@edg.com> Java 8 supports a new user of super: interface I { default int f(){return 0;} } class X implements I { public int f() { return I.super.f(); } } This is described in 15.12.1. However, there is wording in 15.12.1 that says: Let T be the type declaration immediately enclosing the method invocation. It is a compile-time error if Iis not a direct superinterface of T, or if there exists some other direct superclass or direct superinterface of T, J, such that J is a subtype of I. That would seem to require an error on this case, because T has a superclass of J, which is a subtype of I. interface I { default int f(){return 0;} } class J implements I { } class T extends J implements I { public int f() { return I.super.f(); } } Is this a javac bug or am I misunderstanding the rule? Thanks, John. From maurizio.cimadamore at oracle.com Thu Sep 11 17:16:16 2014 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 11 Sep 2014 18:16:16 +0100 Subject: I.super.f() usage in Java 8 In-Reply-To: <166300A4-0BE2-4264-AAA4-1EE3A0C27C46@edg.com> References: <166300A4-0BE2-4264-AAA4-1EE3A0C27C46@edg.com> Message-ID: <5411D8E0.3040604@oracle.com> Hi John, yes - this appears to be a bug; I filed this as: https://bugs.openjdk.java.net/browse/JDK-8058244 Thanks Maurizio On 11/09/14 16:53, John Spicer wrote: > Java 8 supports a new user of super: > > interface I { > default int f(){return 0;} > } > > class X implements I { > public int f() { > return I.super.f(); > } > } > > This is described in 15.12.1. > > However, there is wording in 15.12.1 that says: Let T be the type declaration immediately enclosing the method invocation. It is a compile-time error if Iis not a direct superinterface of T, or if there exists some other direct superclass or direct superinterface of T, J, such that J is a subtype of I. > > That would seem to require an error on this case, because T has a superclass of J, which is a subtype of I. > > interface I { > default int f(){return 0;} > } > class J implements I { } > class T extends J implements I { > public int f() { > return I.super.f(); > } > } From joel.franck at oracle.com Fri Sep 12 14:58:23 2014 From: joel.franck at oracle.com (Joel Borggren-Franck) Date: Fri, 12 Sep 2014 16:58:23 +0200 Subject: target_info structure of a method receiver Message-ID: <20140912145823.GC19198@oracle.com> Hi, The table 4.7.20-A and section 4.7.20.1 in JVMS says that a method receiver target kind is followed by the empty_target target info, noting that "Only one type appears in each of these locations, so there is no per-type information to represent in the target_info union.", yet the following program compiles with both 8 and 9 javac: --- 8< --- import java.lang.annotation.*; import java.lang.reflect.*; public class Receiver { @Target(ElementType.TYPE_USE) @Retention(RetentionPolicy.RUNTIME) @interface A { } class Foo { Foo(@A Receiver<@A T, @A U> Receiver.this) { } } public void m(@A Receiver<@A T, @A U> this, int j) { ; } } --- 8< --- Isn't this 3 type uses in both the ctor and the method? I haven't found anything in JLS to indicate if javac or JVMLS is wrong here. cheers /Joel From trineo at gmail.com Fri Sep 12 15:19:47 2014 From: trineo at gmail.com (Neon Ngo) Date: Fri, 12 Sep 2014 11:19:47 -0400 Subject: JDK 8u20 javac StackOverflowError at com.sun.tools.javac.code.Types.lub (regression) Message-ID: When trying to build our project's code using JDK 8u20's, javac craps out with a StackOverflowError. This worked fine in all prior versions of t he JDK (e.g. 8u11, 7u67, 6u45, etc.) so this is a regression in the compiler. This happens on RHEL 5.10 64-bit, CentOS 6.5 32-bit & 64-bit, RHEL 7 64-bit, etc. java version "1.8.0_20" Java(TM) SE Runtime Environment (build 1.8.0_20-b26) Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode) javac 1.8.0_20 The system is out of resources. Consult the following stack trace for details. java.lang.StackOverflowError at com.sun.tools.javac.code.Types.lub(Types.java:3532) at com.sun.tools.javac.code.Types.lub(Types.java:3616) at com.sun.tools.javac.code.Types.lub(Types.java:3616) at com.sun.tools.javac.code.Types.lub(Types.java:3616) at com.sun.tools.javac.code.Types.lub(Types.java:3616) ... This issue seems similar to JDK-8043725 javac fails with StackOverflowException https://bugs.openjdk.java.net/browse/JDK-8043725 But the stack trace is different. We can always reproduce this problem. Regards, Neon From boaznahum at gmail.com Fri Sep 12 15:26:16 2014 From: boaznahum at gmail.com (Boaz Nahum) Date: Fri, 12 Sep 2014 18:26:16 +0300 Subject: I.super.f() usage in Java 8 In-Reply-To: <5411D8E0.3040604@oracle.com> References: <166300A4-0BE2-4264-AAA4-1EE3A0C27C46@edg.com> <5411D8E0.3040604@oracle.com> Message-ID: So class J, even not override 'f' hide what was before a public method. So to get around I need to (suppose T is mine) interface Was_I extends I { default int was_f() { return I.super.f(); } } class J implements I { } class T extends J implements Was_I { public int f() { return Was_I.super.was_f(); } } I understand that if J implements f than this rule make sense. Not sure I like this bug to be fixed :( R. Boaz On Sep 11, 2014 8:22 PM, "Maurizio Cimadamore" < maurizio.cimadamore at oracle.com> wrote: > Hi John, > yes - this appears to be a bug; I filed this as: > > https://bugs.openjdk.java.net/browse/JDK-8058244 > > Thanks > Maurizio > > > On 11/09/14 16:53, John Spicer wrote: > >> Java 8 supports a new user of super: >> >> interface I { >> default int f(){return 0;} >> } >> >> class X implements I { >> public int f() { >> return I.super.f(); >> } >> } >> >> This is described in 15.12.1. >> >> However, there is wording in 15.12.1 that says: Let T be the type >> declaration immediately enclosing the method invocation. It is a >> compile-time error if Iis not a direct superinterface of T, or if there >> exists some other direct superclass or direct superinterface of T, J, such >> that J is a subtype of I. >> >> That would seem to require an error on this case, because T has a >> superclass of J, which is a subtype of I. >> >> interface I { >> default int f(){return 0;} >> } >> class J implements I { } >> class T extends J implements I { >> public int f() { >> return I.super.f(); >> } >> } >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From maurizio.cimadamore at oracle.com Fri Sep 12 15:27:59 2014 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 12 Sep 2014 16:27:59 +0100 Subject: JDK 8u20 javac StackOverflowError at com.sun.tools.javac.code.Types.lub (regression) In-Reply-To: References: Message-ID: <541310FF.9010102@oracle.com> Can you please send a test case? Thanks Maurizio On 12/09/14 16:19, Neon Ngo wrote: > When trying to build our project's code using JDK 8u20's, javac craps > out with a StackOverflowError. > This worked fine in all prior versions of t he JDK (e.g. 8u11, 7u67, > 6u45, etc.) so this is a regression in the compiler. > This happens on RHEL 5.10 64-bit, CentOS 6.5 32-bit & 64-bit, RHEL 7 > 64-bit, etc. > > java version "1.8.0_20" > Java(TM) SE Runtime Environment (build 1.8.0_20-b26) > Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode) > javac 1.8.0_20 > > The system is out of resources. > Consult the following stack trace for details. > java.lang.StackOverflowError > at com.sun.tools.javac.code.Types.lub(Types.java:3532) > at com.sun.tools.javac.code.Types.lub(Types.java:3616) > at com.sun.tools.javac.code.Types.lub(Types.java:3616) > at com.sun.tools.javac.code.Types.lub(Types.java:3616) > at com.sun.tools.javac.code.Types.lub(Types.java:3616) > ... > > This issue seems similar to JDK-8043725 javac fails with StackOverflowException > https://bugs.openjdk.java.net/browse/JDK-8043725 > > But the stack trace is different. > > We can always reproduce this problem. > > Regards, > > Neon From maurizio.cimadamore at oracle.com Fri Sep 12 15:31:22 2014 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 12 Sep 2014 16:31:22 +0100 Subject: I.super.f() usage in Java 8 In-Reply-To: References: <166300A4-0BE2-4264-AAA4-1EE3A0C27C46@edg.com> <5411D8E0.3040604@oracle.com> Message-ID: <541311CA.9070701@oracle.com> Hi Boaz, as I wrote in a comment in the bug - javac's logic is more precise than the one described in the spec. This means that javac only issues errors when there is what we call 'level-skipping' i.e. when you try to access a method from your super's super through a qualified super call. The spec disallows this with a looser check, which ends up rejecting more stuff than just that that leads to level skipping. Is there any reason as to why the spec check has been defined in this particular way? Maurizio On 12/09/14 16:26, Boaz Nahum wrote: > > So class J, even not override 'f' hide what was before a public method. > > So to get around I need to (suppose T is mine) > > interface Was_I extends I { > > default int was_f() { return I.super.f(); } > > } > > class J implements I { } > class T extends J implements Was_I { > public int f() { > return Was_I.super.was_f(); > } > } > > I understand that if J implements f than this rule make sense. > > Not sure I like this bug to be fixed :( > > R. > > Boaz > > On Sep 11, 2014 8:22 PM, "Maurizio Cimadamore" > > wrote: > > Hi John, > yes - this appears to be a bug; I filed this as: > > https://bugs.openjdk.java.net/browse/JDK-8058244 > > Thanks > Maurizio > > > On 11/09/14 16:53, John Spicer wrote: > > Java 8 supports a new user of super: > > interface I { > default int f(){return 0;} > } > > class X implements I { > public int f() { > return I.super.f(); > } > } > > This is described in 15.12.1. > > However, there is wording in 15.12.1 that says: Let T be the > type declaration immediately enclosing the method invocation. > It is a compile-time error if Iis not a direct superinterface > of T, or if there exists some other direct superclass or > direct superinterface of T, J, such that J is a subtype of I. > > That would seem to require an error on this case, because T > has a superclass of J, which is a subtype of I. > > interface I { > default int f(){return 0;} > } > class J implements I { } > class T extends J implements I { > public int f() { > return I.super.f(); > } > } > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.govereau at oracle.com Fri Sep 12 18:25:48 2014 From: paul.govereau at oracle.com (Paul Govereau) Date: Fri, 12 Sep 2014 14:25:48 -0400 Subject: Compiling large switch statements In-Reply-To: <5410BE58.7060005@oracle.com> References: <5410BE58.7060005@oracle.com> Message-ID: <54133AAC.1040300@oracle.com> This issue is now being tacked here: https://bugs.openjdk.java.net/browse/JDK-8058243 Paul On 09/10/2014 05:10 PM, Paul Govereau wrote: > Hello, > > I have been looking at the javac code generator, and I think we may be > doing a poor job of compiling large switch statements because of an > overly conservative estimate of the cost of the lookupswitch > instruction. Because we over estimate the cost of a lookupswitch, we > can generate very large tableswitch instructions: twice the size or > larger than necessary. > > Briefly, javac estimates the total cost of a switch instruction to be: > > space + 3 * time, > > and generates the lowest cost instruction. However, javac assumes the > time cost of a lookupswitch is linear, so we greatly favor the > tableswitch. (For reference, the relevant code is in the visitSwitch > method in jvm.Gen). > > As an example, consider a large switch containing 50 cases. For the > tableswitch, the important factor is the range of the matched values, > because this determines the size of the array needed for the table of > addresses. Let's call this size S, then the cost of a tableswitch is > estimated as: > > size (words): 4 + S + 1 > computation: 3 steps (2 comparisons, 1 load) > > For the same code the cost for a lookupswitch is estimated as: > > size(words): 3 + 2 * 50 > computation: 50 steps (presumably 50 comparisons?) > > Note that the lookupswitch match-offset pairs must be sorted. So, it > stands to reason that to implementation is using (at least) a binary > search on the sorted list of keys. Therefore, the computation cost > must be closer to 2*log(50), no? > > With the current metrics, we allow a tableswitch instruction with 50 > items to be as large as 976 bytes before we change to the much smaller > 412 byte lookupswitch. It seems to me the log-estimate is more > accurate and we should change to the lookupswitch when the tableswitch > reaches 468 bytes. However, I am only guessing as to how these > bytecodes are implemented. Does anyone on the list know the details? > > Paul From paul.govereau at oracle.com Fri Sep 12 18:39:46 2014 From: paul.govereau at oracle.com (Paul Govereau) Date: Fri, 12 Sep 2014 14:39:46 -0400 Subject: Smaller byte code for small constants Message-ID: <54133DF2.6070004@oracle.com> Currently, for long, float and double, javac will emit an ldc instruction for small constants (aside from 0, 1, 2). For instance, this statement: long x = 3; produces the code: ldc2_w #index lstore_n However, we could save ourselves a constant pool slot with: bipush 3 i2l istore_n The same trick can be used for float and double constants that happen to be round integers (modulo the semantics of i2f?). Are there bad consequences for the interpreter or runtime compiler if we made this change in javac? Thanks, Paul From alex.buckley at oracle.com Fri Sep 12 18:56:19 2014 From: alex.buckley at oracle.com (Alex Buckley) Date: Fri, 12 Sep 2014 11:56:19 -0700 Subject: target_info structure of a method receiver In-Reply-To: <20140912145823.GC19198@oracle.com> References: <20140912145823.GC19198@oracle.com> Message-ID: <541341D3.3060308@oracle.com> I know what you mean, but there is no problem with javac here. The only way to explain this precisely is to use a JLS concept. Per JLS8 4.11, a receiver parameter induces one type context - think of it as the area in Foo's ctor signature after '(' and before 'Receiver.this'. It is this type context of which there is "only one". There may be many types "inside" the type context if a nested, array, or parameterized type is involved, and the target_path structure will take care of getting you to a particular one - but there's only one area. Same deal with the type context for a field declaration - you already know the field (the attribute is within a field_info) so there is only one area where the type of the field can be written. Contrast with a formal parameter declaration - the attribute is inside a method_info so there needs to be a way to identify a particular formal parameter (formal_parameter_target) which provides the area (type context) where the type of the formal parameter can be written. I'm not sure it's worth rewriting the informative JVMS text. The normative text for empty_target already mentions "_the_ type in a field declaration" and "_the_ receiver type of a method or constructor", and there are other mentions of "_the_ type" throughout 4.7.20.1. Appealing to "the type context" rather than "the type" seems more trouble than it's worth. Alex On 9/12/2014 7:58 AM, Joel Borggren-Franck wrote: > Hi, > > The table 4.7.20-A and section 4.7.20.1 in JVMS says that a method > receiver target kind is followed by the empty_target target info, noting > that "Only one type appears in each of these locations, so there is no > per-type information to represent in the target_info union.", yet the > following program compiles with both 8 and 9 javac: > > --- 8< --- > > import java.lang.annotation.*; > import java.lang.reflect.*; > > public class Receiver { > > @Target(ElementType.TYPE_USE) > @Retention(RetentionPolicy.RUNTIME) > @interface A { > } > > class Foo { > Foo(@A Receiver<@A T, @A U> Receiver.this) { > } > } > > public void m(@A Receiver<@A T, @A U> this, int j) { ; } > } > > --- 8< --- > > Isn't this 3 type uses in both the ctor and the method? > > I haven't found anything in JLS to indicate if javac or JVMLS is wrong > here. > > cheers > /Joel > From vitalyd at gmail.com Fri Sep 12 23:00:23 2014 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 12 Sep 2014 19:00:23 -0400 Subject: Smaller byte code for small constants In-Reply-To: <54133DF2.6070004@oracle.com> References: <54133DF2.6070004@oracle.com> Message-ID: Hi Paul, This would increase the bytecode size of methods containing these scenarios, right? If so, having several of these may push a method over jit inlining heuristic that uses bytecode size. This issue has come up in other contexts as well (e.g. asserts causing same problem). I'd say until the jit inliner is changed to use something smarter (e.g. CFG complexity post some basic reductions or something like that), this has potential for perf regressions. Presumably interpreter may not like this either, but I don't know. Out of curiosity what's the practical benefit of saving a constant pool slot? Less VM memory overhead? Sent from my phone On Sep 12, 2014 2:42 PM, "Paul Govereau" wrote: > Currently, for long, float and double, javac will emit an ldc instruction > for small constants (aside from 0, 1, 2). For instance, this statement: > > long x = 3; > > produces the code: > > ldc2_w #index > lstore_n > > However, we could save ourselves a constant pool slot with: > > bipush 3 > i2l > istore_n > > The same trick can be used for float and double constants that happen to > be round integers (modulo the semantics of i2f?). > > Are there bad consequences for the interpreter or runtime compiler if we > made this change in javac? > > Thanks, > Paul > -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.r.rose at oracle.com Sat Sep 13 00:23:25 2014 From: john.r.rose at oracle.com (John Rose) Date: Fri, 12 Sep 2014 17:23:25 -0700 Subject: Smaller byte code for small constants In-Reply-To: <54133DF2.6070004@oracle.com> References: <54133DF2.6070004@oracle.com> Message-ID: <78091327-1350-4F35-A933-D2C27D62AF3E@oracle.com> On Sep 12, 2014, at 11:39 AM, Paul Govereau wrote: > Currently, for long, float and double, javac will emit an ldc instruction for small constants (aside from 0, 1, 2). For instance, this statement: > > long x = 3; > > produces the code: > > ldc2_w #index > lstore_n > > However, we could save ourselves a constant pool slot with: > > bipush 3 > i2l > istore_n > > The same trick can be used for float and double constants that happen to be round integers (modulo the semantics of i2f?). > > Are there bad consequences for the interpreter or runtime compiler if we made this change in javac? As Vitaly points out, inlining heuristics depend on metrics which these tricks affect. Inflating a metric is risky for that reason, whether the metric is bytecode size (same here) or bytecode instruction count (larger) or classfile size (smaller). The differences in interpreter execution are negligible. The JIT IR generated will be the same, almost certainly. Here's a tie-breaker principle: The more complex you make the class file, the less pattern-based optimizations will apply reliably. Put another way, upstream optimizers confuse downstream optimizers, if they are looking for the same things. This is true for both compressors and code optimizers and probably other things. It adds a reason to keep things straightforward. ? John From paul.govereau at oracle.com Sat Sep 13 01:29:08 2014 From: paul.govereau at oracle.com (Paul Govereau) Date: Fri, 12 Sep 2014 21:29:08 -0400 Subject: Smaller byte code for small constants In-Reply-To: References: <54133DF2.6070004@oracle.com> Message-ID: <54139DE4.1000901@oracle.com> Hi Vitaly, Thanks for the response. The only benefit here is a slightly smaller classfile. However, if there is a risk of impacting the optimizer, it hardly seems worth it. On 09/12/2014 07:00 PM, Vitaly Davidovich wrote: > Hi Paul, > > This would increase the bytecode size of methods containing these > scenarios, right? If so, having several of these may push a method over > jit inlining heuristic that uses bytecode size. This issue has come up > in other contexts as well (e.g. asserts causing same problem). I'd say > until the jit inliner is changed to use something smarter (e.g. CFG > complexity post some basic reductions or something like that), this has > potential for perf regressions. > > Presumably interpreter may not like this either, but I don't know. > > Out of curiosity what's the practical benefit of saving a constant pool > slot? Less VM memory overhead? > > Sent from my phone > > On Sep 12, 2014 2:42 PM, "Paul Govereau" > wrote: > > Currently, for long, float and double, javac will emit an ldc > instruction for small constants (aside from 0, 1, 2). For instance, > this statement: > > long x = 3; > > produces the code: > > ldc2_w #index > lstore_n > > However, we could save ourselves a constant pool slot with: > > bipush 3 > i2l > istore_n > > The same trick can be used for float and double constants that > happen to be round integers (modulo the semantics of i2f?). > > Are there bad consequences for the interpreter or runtime compiler > if we made this change in javac? > > Thanks, > Paul > From maurizio.cimadamore at oracle.com Mon Sep 15 21:12:18 2014 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 15 Sep 2014 22:12:18 +0100 Subject: JDK 8u20 javac StackOverflowError at com.sun.tools.javac.code.Types.lub (regression) In-Reply-To: <541310FF.9010102@oracle.com> References: <541310FF.9010102@oracle.com> Message-ID: <54175632.2080107@oracle.com> Thank you for the detailed information (sent privately) - a new bug has been filed: https://bugs.openjdk.java.net/browse/JDK-8058511 Maurizio On 12/09/14 16:27, Maurizio Cimadamore wrote: > Can you please send a test case? > > Thanks > Maurizio > > On 12/09/14 16:19, Neon Ngo wrote: >> When trying to build our project's code using JDK 8u20's, javac craps >> out with a StackOverflowError. >> This worked fine in all prior versions of t he JDK (e.g. 8u11, 7u67, >> 6u45, etc.) so this is a regression in the compiler. >> This happens on RHEL 5.10 64-bit, CentOS 6.5 32-bit & 64-bit, RHEL 7 >> 64-bit, etc. >> >> java version "1.8.0_20" >> Java(TM) SE Runtime Environment (build 1.8.0_20-b26) >> Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode) >> javac 1.8.0_20 >> >> The system is out of resources. >> Consult the following stack trace for details. >> java.lang.StackOverflowError >> at com.sun.tools.javac.code.Types.lub(Types.java:3532) >> at com.sun.tools.javac.code.Types.lub(Types.java:3616) >> at com.sun.tools.javac.code.Types.lub(Types.java:3616) >> at com.sun.tools.javac.code.Types.lub(Types.java:3616) >> at com.sun.tools.javac.code.Types.lub(Types.java:3616) >> ... >> >> This issue seems similar to JDK-8043725 javac fails with >> StackOverflowException >> https://bugs.openjdk.java.net/browse/JDK-8043725 >> >> But the stack trace is different. >> >> We can always reproduce this problem. >> >> Regards, >> >> Neon > From joel.franck at oracle.com Thu Sep 18 08:25:14 2014 From: joel.franck at oracle.com (Joel Borggren-Franck) Date: Thu, 18 Sep 2014 10:25:14 +0200 Subject: target_info structure of a method receiver In-Reply-To: <541341D3.3060308@oracle.com> References: <20140912145823.GC19198@oracle.com> <541341D3.3060308@oracle.com> Message-ID: <20140918082514.GG19198@oracle.com> Hi Alex, Thanks for the clarification. Makes sense, however there is still something with the receivers that feel problematic: --- 8< --- public class Outer<@B T, @C U> { class Inner { Inner(@A Outer<@NotB T, @NotC U> Outer.this) { ... } } } --- 8< --- Focusing on the inner class' constructor receiver parameter, JLS 8.4.1 says: " In an inner class's constructor, the type of the receiver parameter must be the class or interface which is the immediately enclosing type declaration of the inner class, and the name of the receiver parameter must be Identifier . this where Identifier is the simple name of the class or interface which is the immediately enclosing type declaration of the inner class; otherwise, a compile-time error occurs." The type of the receiver parameter Outer<@NotB T, @NotC U> looks like a set of parameterized types to me, not "the type" in singular. Is this problematic? What is the relation between the annotations on the receiver type's type parameters and the receiver type declaration's annotations on the type parameters? After this is erased, how should runtime reflection piece together the receiver type? There is no j.l.r.Type subtype that represents a generic class/interface that is not a class/interface declaration. If the receiver type should not be the erased type, is it desirable to represent a set of parameterized types with different type parameter annotations than on the class? Does any of this make any sense? I think the underlying question here is what is the meaning of that receiver type parameter, and what should we do at runtime? cheers /Joel On 2014-09-12, Alex Buckley wrote: > I know what you mean, but there is no problem with javac here. > > The only way to explain this precisely is to use a JLS concept. Per > JLS8 4.11, a receiver parameter induces one type context - think of > it as the area in Foo's ctor signature after '(' and before > 'Receiver.this'. It is this type context of which there is "only > one". There may be many types "inside" the type context if a nested, > array, or parameterized type is involved, and the target_path > structure will take care of getting you to a particular one - but > there's only one area. > > Same deal with the type context for a field declaration - you > already know the field (the attribute is within a field_info) so > there is only one area where the type of the field can be written. > Contrast with a formal parameter declaration - the attribute is > inside a method_info so there needs to be a way to identify a > particular formal parameter (formal_parameter_target) which provides > the area (type context) where the type of the formal parameter can > be written. > > I'm not sure it's worth rewriting the informative JVMS text. The > normative text for empty_target already mentions "_the_ type in a > field declaration" and "_the_ receiver type of a method or > constructor", and there are other mentions of "_the_ type" > throughout 4.7.20.1. Appealing to "the type context" rather than > "the type" seems more trouble than it's worth. > > Alex > > On 9/12/2014 7:58 AM, Joel Borggren-Franck wrote: > >Hi, > > > >The table 4.7.20-A and section 4.7.20.1 in JVMS says that a method > >receiver target kind is followed by the empty_target target info, noting > >that "Only one type appears in each of these locations, so there is no > >per-type information to represent in the target_info union.", yet the > >following program compiles with both 8 and 9 javac: > > > >--- 8< --- > > > >import java.lang.annotation.*; > >import java.lang.reflect.*; > > > >public class Receiver { > > > > @Target(ElementType.TYPE_USE) > > @Retention(RetentionPolicy.RUNTIME) > > @interface A { > > } > > > > class Foo { > > Foo(@A Receiver<@A T, @A U> Receiver.this) { > > } > > } > > > > public void m(@A Receiver<@A T, @A U> this, int j) { ; } > >} > > > >--- 8< --- > > > >Isn't this 3 type uses in both the ctor and the method? > > > >I haven't found anything in JLS to indicate if javac or JVMLS is wrong > >here. > > > >cheers > >/Joel > > From alex.buckley at oracle.com Thu Sep 18 22:00:57 2014 From: alex.buckley at oracle.com (Alex Buckley) Date: Thu, 18 Sep 2014 15:00:57 -0700 Subject: target_info structure of a method receiver In-Reply-To: <20140918082514.GG19198@oracle.com> References: <20140912145823.GC19198@oracle.com> <541341D3.3060308@oracle.com> <20140918082514.GG19198@oracle.com> Message-ID: <541B5619.9080804@oracle.com> Hi Joel, On 9/18/2014 1:25 AM, Joel Borggren-Franck wrote: > Thanks for the clarification. Makes sense, however there is still > something with the receivers that feel problematic: > > --- 8< --- > > public class Outer<@B T, @C U> { > > class Inner { > > Inner(@A Outer<@NotB T, @NotC U> Outer.this) { ... } > > } > } > > --- 8< --- > > Focusing on the inner class' constructor receiver parameter, JLS 8.4.1 > says: > > " In an inner class's constructor, the type of the receiver parameter > must be the class or interface which is the immediately enclosing type > declaration of the inner class, and the name of the receiver parameter > must be Identifier . this where Identifier is the simple name of the > class or interface which is the immediately enclosing type declaration > of the inner class; otherwise, a compile-time error occurs." > > The type of the receiver parameter Outer<@NotB T, @NotC U> looks like a > set of parameterized types to me, not "the type" in singular. Is this > problematic? The type Outer<@NotB T, @NotC U> is indeed a parameterized type. The quote intends for it to be [a parameterization of] the [generic] class Outer. Another plausible type for the receiver parameter is Outer. Yet another plausible type is raw Outer. I don't see a problem here - the immediately enclosing type declaration represents a generic class and it's not controversial to parameterize it. (Yes, you parameterize it with more types, each of which can be annotated, but I don't see any real confusion.) > What is the relation between the annotations on the receiver type's > type parameters and the receiver type declaration's annotations on > the type parameters? You mean: what is the relation between annotations on the receiver type's _type arguments_ and annotations on the receiver type declaration's _type parameters_? The answer is: none. The type parameters T and U of the generic class can be annotated one way, and particular parameterized types Outer<...,...> can annotate their type arguments in other ways. This is a feature! > After this is erased, how should runtime reflection piece together > the receiver type? There is no j.l.r.Type subtype that represents a > generic class/interface that is not a class/interface declaration. If > the receiver type should not be the erased type, is it desirable to > represent a set of parameterized types with different type parameter > annotations than on the class? For your example, I'd expect j.l.r.Constructor#getAnnotatedReceiverType to return a j.l.r.AnnotatedType which exposes @A and is instanceof j.l.r.AnnotatedParameterizedType so you can obtain its annotated type arguments @NotB T and @NotC U. (You happen to have used some in-scope type parameters as the type arguments.) Alex From joel.franck at oracle.com Fri Sep 19 11:24:08 2014 From: joel.franck at oracle.com (=?windows-1252?Q?Joel_Borggr=E9n-Franck?=) Date: Fri, 19 Sep 2014 13:24:08 +0200 Subject: target_info structure of a method receiver In-Reply-To: <541B5619.9080804@oracle.com> References: <20140912145823.GC19198@oracle.com> <541341D3.3060308@oracle.com> <20140918082514.GG19198@oracle.com> <541B5619.9080804@oracle.com> Message-ID: <819AAA75-0DBF-4728-A47E-D12E676077A4@oracle.com> Hi Alex, Ok, I didn?t realize you used the in scope type parameters (shouldn?t it be type variables to be extra precise?) as type arguments. Two comments inline, On 19 sep 2014, at 00:00, Alex Buckley wrote: > On 9/18/2014 1:25 AM, Joel Borggren-Franck wrote: >> --- 8< --- >> >> public class Outer<@B T, @C U> { >> >> class Inner { >> >> Inner(@A Outer<@NotB T, @NotC U> Outer.this) { ... } >> >> } >> } >> >> --- 8< ? >> The type of the receiver parameter Outer<@NotB T, @NotC U> looks like a >> set of parameterized types to me, not "the type" in singular. Is this >> problematic? > > The type Outer<@NotB T, @NotC U> is indeed a parameterized type. The quote intends for it to be [a parameterization of] the [generic] class Outer. Another plausible type for the receiver parameter is Outer. Yet another plausible type is raw Outer. Understod. Javac only allows Outer as the receiver type (with optimal annotations). Neither Outer nor the raw Outer compiles. >> After this is erased, how should runtime reflection piece together >> the receiver type? There is no j.l.r.Type subtype that represents a >> generic class/interface that is not a class/interface declaration. If >> the receiver type should not be the erased type, is it desirable to >> represent a set of parameterized types with different type parameter >> annotations than on the class? > > For your example, I'd expect j.l.r.Constructor#getAnnotatedReceiverType to return a j.l.r.AnnotatedType which exposes @A and is instanceof j.l.r.AnnotatedParameterizedType so you can obtain its annotated type arguments @NotB T and @NotC U. (You happen to have used some in-scope type parameters as the type arguments.) Unfortunately there is no Signature for the receiver type in the class file for Inner (?LOuter;" doesn?t occur at all in the class file for Inner). If Outer (potentially with annotations) is the only possible recover type I can probably work around this in core reflection, if other parameterizations of Outer should be allowed I need more info in the class file. cheers /Joel From alex.buckley at oracle.com Fri Sep 19 18:12:32 2014 From: alex.buckley at oracle.com (Alex Buckley) Date: Fri, 19 Sep 2014 11:12:32 -0700 Subject: target_info structure of a method receiver In-Reply-To: <819AAA75-0DBF-4728-A47E-D12E676077A4@oracle.com> References: <20140912145823.GC19198@oracle.com> <541341D3.3060308@oracle.com> <20140918082514.GG19198@oracle.com> <541B5619.9080804@oracle.com> <819AAA75-0DBF-4728-A47E-D12E676077A4@oracle.com> Message-ID: <541C7210.9010304@oracle.com> On 9/19/2014 4:24 AM, Joel Borggr?n-Franck wrote: > Ok, I didn?t realize you used the in scope type parameters (shouldn?t > it be type variables to be extra precise?) as type arguments. I used them? You mean, you used them. (Yes, technically, the type arguments of the parameterized type Outer<...,...> are type variables in your example ... namely, type variables declared as the type parameters of the generic class Outer. I find that "type variables" is often used when people don't realize the difference between a type argument and a type parameter, so I avoid it most of the time.) > On 19 sep 2014, at 00:00, Alex Buckley > wrote: > Understod. Javac only allows Outer as the receiver type (with > optimal annotations). Neither Outer nor the raw Outer > compiles. I realized after sending my mail that Outer and raw Outer were not good examples for this situation. It's right that javac rejects lazy uses of Outer for the receiver parameter's type. >>> After this is erased, how should runtime reflection piece >>> together the receiver type? There is no j.l.r.Type subtype that >>> represents a generic class/interface that is not a >>> class/interface declaration. If the receiver type should not be >>> the erased type, is it desirable to represent a set of >>> parameterized types with different type parameter annotations >>> than on the class? >> >> For your example, I'd expect >> j.l.r.Constructor#getAnnotatedReceiverType to return a >> j.l.r.AnnotatedType which exposes @A and is instanceof >> j.l.r.AnnotatedParameterizedType so you can obtain its annotated >> type arguments @NotB T and @NotC U. (You happen to have used some >> in-scope type parameters as the type arguments.) > > Unfortunately there is no Signature for the receiver type in the > class file for Inner (?LOuter;" doesn?t occur at all in the > class file for Inner). If Outer (potentially with annotations) > is the only possible recover type I can probably work around this in > core reflection, if other parameterizations of Outer should be > allowed I need more info in the class file. I wouldn't expect a Signature attribute in Outer$Inner.class for the receiver parameter's type. See JVMS8 4.7.9.1 for the conditions when Signature is required. A receiver parameter is intended to be ephemeral - it's not a formal parameter and shouldn't be returned by anything in j.l.r.Executable concerned with formal parameters or their types. The only representation of a receiver parameter in the class file is in Runtime[In]VisibleTypeAnnotations, for the benefit of Executable#getAnnotatedReceiverType. Alex From martin.schaef at sri.com Sun Sep 21 21:39:50 2014 From: martin.schaef at sri.com (Martin Schaef) Date: Sun, 21 Sep 2014 21:39:50 +0000 Subject: Strange translation of finally blocks Message-ID: Hi there, I?m working on a static analysis tool for bytecode and I have this issue where the compiler generates strange code with looping catch blocks which kills my tool. I attached an example Java file and the bytecode. If you look at the exception table, there is a line: 79 81 79 any which gives me some headache. Is this correct? If so, what does it mean? I found a similar problem with try-with-resources earlier and reported it on stackoverflow: http://stackoverflow.com/questions/25615417/try-with-resources-introduce-unreachable-bytecode Best, Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: ZipFile.java Type: application/octet-stream Size: 1667 bytes Desc: not available URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: ZipFile.txt URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3917 bytes Desc: not available URL: From jan.lahoda at oracle.com Mon Sep 22 10:24:38 2014 From: jan.lahoda at oracle.com (Jan Lahoda) Date: Mon, 22 Sep 2014 12:24:38 +0200 Subject: Strange translation of finally blocks In-Reply-To: References: Message-ID: <541FF8E6.9080704@oracle.com> Hi Martin, On 21.9.2014 23:39, Martin Schaef wrote: > Hi there, > I?m working on a static analysis tool for bytecode and I have this issue where the compiler generates strange code with looping catch blocks which kills my tool. > I attached an example Java file and the bytecode. If you look at the exception table, there is a line: > 79 81 79 any What version of JDK do you use? It appears this may be already resolved (by Maurizio) in JDK 8: https://bugs.openjdk.java.net/browse/JDK-7093325 Thanks, Jan > which gives me some headache. > Is this correct? If so, what does it mean? > > I found a similar problem with try-with-resources earlier and reported it on stackoverflow: > http://stackoverflow.com/questions/25615417/try-with-resources-introduce-unreachable-bytecode > > Best, > Martin > From joel.franck at oracle.com Tue Sep 23 09:04:40 2014 From: joel.franck at oracle.com (=?windows-1252?Q?Joel_Borggr=E9n-Franck?=) Date: Tue, 23 Sep 2014 11:04:40 +0200 Subject: target_info structure of a method receiver In-Reply-To: <541C7210.9010304@oracle.com> References: <20140912145823.GC19198@oracle.com> <541341D3.3060308@oracle.com> <20140918082514.GG19198@oracle.com> <541B5619.9080804@oracle.com> <819AAA75-0DBF-4728-A47E-D12E676077A4@oracle.com> <541C7210.9010304@oracle.com> Message-ID: <8AF5CF0B-1482-4796-89B5-9DE908C98C7A@oracle.com> Hi, On 19 sep 2014, at 20:12, Alex Buckley wrote: > On 9/19/2014 4:24 AM, Joel Borggr?n-Franck wrote: > >> On 19 sep 2014, at 00:00, Alex Buckley >> wrote: >> Understod. Javac only allows Outer as the receiver type (with >> optimal annotations). Neither Outer nor the raw Outer >> compiles. > > I realized after sending my mail that Outer and raw Outer > were not good examples for this situation. It's right that javac rejects lazy uses of Outer for the receiver parameter's type. > >>>> After this is erased, how should runtime reflection piece >>>> together the receiver type? There is no j.l.r.Type subtype that >>>> represents a generic class/interface that is not a >>>> class/interface declaration. If the receiver type should not be >>>> the erased type, is it desirable to represent a set of >>>> parameterized types with different type parameter annotations >>>> than on the class? >>> >>> For your example, I'd expect >>> j.l.r.Constructor#getAnnotatedReceiverType to return a >>> j.l.r.AnnotatedType which exposes @A and is instanceof >>> j.l.r.AnnotatedParameterizedType so you can obtain its annotated >>> type arguments @NotB T and @NotC U. (You happen to have used some >>> in-scope type parameters as the type arguments.) >> >> Unfortunately there is no Signature for the receiver type in the >> class file for Inner (?LOuter;" doesn?t occur at all in the >> class file for Inner). If Outer (potentially with annotations) >> is the only possible recover type I can probably work around this in >> core reflection, if other parameterizations of Outer should be >> allowed I need more info in the class file. > > I wouldn't expect a Signature attribute in Outer$Inner.class for the receiver parameter's type. See JVMS8 4.7.9.1 for the conditions when Signature is required. A receiver parameter is intended to be ephemeral - it's not a formal parameter and shouldn't be returned by anything in j.l.r.Executable concerned with formal parameters or their types. > > The only representation of a receiver parameter in the class file is in Runtime[In]VisibleTypeAnnotations, for the benefit of Executable#getAnnotatedReceiverType. > As long as I can expect exactly one type for the receiver type (excluding annotations) this shouldn?t be a problem. This seems consistent with how javac works today and what you wrote above. Thanks for explaining this. cheres /Joel From chrisdkitching at gmail.com Mon Sep 8 21:35:43 2014 From: chrisdkitching at gmail.com (Chris Kitching) Date: Mon, 08 Sep 2014 21:35:43 -0000 Subject: Apparent bug in javac's dead code elimination In-Reply-To: <540E1CDA.3060306@oracle.com> References: <53E7FA44.9050207@oracle.com> <476830A4-7A8E-4652-A0C7-805687155F4A@oracle.com> <53E8CCD6.2090503@CoSoCo.de> <53FDA7C1.6060303@CoSoCo.de> <902B0B7A-32BE-498A-B9DD-8937960430A2@oracle.com> <53FF7244.2030804@CoSoCo.de> <5400E85E.9030005@CoSoCo.de> <540BCBCB.8010406@linux.com> <540DE6AD.2040703@oracle.com> <540DFC70.3060003@oracle.com> <540E0307.5000206@linux.com> <540E1930.1000800@oracle.com> <540E1CDA.3060306@oracle.com> Message-ID: <540E211C.4050003@gmail.com> Ah! Apologies for not spotting that myself. Glad to know it's fixed in newer versions, anyway. Thanks for your time! On 08/09/14 14:17, Jonathan Gibbons wrote: > Chris, > > Update: > I see the problem when using JDK 1.7.0. > I do not see the problem using using JDK 1.8.0 or later. > > -- Jon > > On 09/08/2014 02:01 PM, Jonathan Gibbons wrote: >> Chris, >> >> I agree that javac should only generate valid class files, and that JVMS >> is quite explicit about when a Code attribute is required to be present >> or absent. >> >> I tried some simple experiments of my own and your Cake.java example. >> In neither case do I see any instances of a Code attribute being missing >> when required (or present when not.) In other words, I have not yet seen >> anything akin to what would give rise to the message you saw with the >> mozilla class: >> >> java.lang.ClassFormatError: Absent Code attribute in method that is not >> native or abstract in class file >> >> Given that it is not an error for javac to be generating an empty >> class file >> for its own internal purposes, do you have any additional evidence that >> javac is creating an invalid class file? >> >> I was using current builds of JDK 9 for my experiments; what version of >> JDK are you using? >> >> -- Jon >> >> On 09/08/2014 12:27 PM, Chris Kitching wrote: >>> Hello, >>> >>> "What do you need this class for?" >>> >>> I don't need this class for anything, I merely need the compiler to >>> promise to never emit classes that are invalid. Surely that is something >>> is should be expected to do? >>> The current behaviour causes issues for tools that process all classes >>> in a given directory. >>> >>> >>> On 08/09/14 11:58, Vicente-Arturo Romero-Zaldivar wrote: >>>> Hi Chris, >>>> >>>> Yes I agree with Jon, this is not a bug. I have been checking the patch >>>> for [1] see [2]. In the code you pasted Cake$1 has nothing to do with >>>> the inner class defined in the 'if (false)' sentence. It's just javac >>>> reusing an existing class name that would have been a 'real' class if >>>> the condition were different to false. What do you need this class for? >>>> You shouldn't need to deal with it. The problem is that we don't have a >>>> way to say: "hey this is not a common inner class this is just a token, >>>> exact term access constructor tag, to provide access to a private >>>> constructor". The only difference between a token and a 'common' inner >>>> class is that the token is empty, no methods in it. >>>> >>>> Thanks, >>>> Vicente >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-7199823 >>>> [2] http://hg.openjdk.java.net/jdk8/jdk8/langtools/rev/a51a8dac0a2f >>>> >>>> >>>> >>>> On 09/08/2014 10:26 AM, Jonathan Gibbons wrote: >>>>> I believe the existence of Cake$1 is part of javac's handling of >>>>> allowing any private code in SomeClass to access any private code in >>>>> SomeOtherClass. >>>>> >>>>> -- Jon >>>>> >>>>> On 09/06/2014 08:06 PM, Chris Kitching wrote: >>>>>> Greetings, >>>>>> >>>>>> As you probably know, javac does a bit of dead code elimintion: it'll >>>>>> get rid of redundant constructs like if (false) and suchlike. >>>>>> >>>>>> Consider, then, this ridiculous example: >>>>>> >>>>>> public class Cake { >>>>>> public void start() { >>>>>> if (false) { >>>>>> new Runnable() { >>>>>> @Override >>>>>> public void run() { >>>>>> // Whatever >>>>>> } >>>>>> }; >>>>>> } >>>>>> } >>>>>> >>>>>> private static class SomeClass { >>>>>> } >>>>>> } >>>>>> >>>>>> As you might expect, this produces two classes: Cake and >>>>>> Cake$SomeClass.class, and all is well. >>>>>> >>>>>> Now consider this slightly more ridiculous example: >>>>>> >>>>>> public class Cake { >>>>>> public void start() { >>>>>> if (false) { >>>>>> new Runnable() { >>>>>> @Override >>>>>> public void run() { >>>>>> // Whatever >>>>>> } >>>>>> }; >>>>>> } >>>>>> } >>>>>> >>>>>> private static class SomeClass { >>>>>> } >>>>>> >>>>>> private static class SomeOtherClass extends SomeClass { >>>>>> } >>>>>> } >>>>>> >>>>>> >>>>>> All I did was add an empty SomeOtherClass extending SomeClass. >>>>>> This example produces out these classes: >>>>>> >>>>>> Cake$1 >>>>>> Cake >>>>>> Cake$SomeClass >>>>>> Cake$SomeOtherClass >>>>>> >>>>>> Cake$1, the Runnable wrapped in `if (false)`, has suddenly appeared. >>>>>> Oddly, the bytecode in Cake$1 does not change as you alter the >>>>>> contents >>>>>> of run(). It's as if javac is omitting the contents of the methods of >>>>>> the class, but not the class itself (but it did so earlier before we >>>>>> added SomeOtherClass!) >>>>>> Cake$1 is also corrupt. Attempts to load the class using the >>>>>> Reflection >>>>>> API (because, you know, that's a sane thing to do) fail with a >>>>>> stacktrace like this: >>>>>> >>>>>> java.lang.ClassFormatError: Absent Code attribute in method that >>>>>> is not >>>>>> native or abstract in class file >>>>>> org/mozilla/gecko/animation/PropertyAnimator$1 >>>>>> at java.lang.ClassLoader.defineClass1(Native Method) >>>>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:791) >>>>>> at >>>>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >>>>>> >>>>>> at >>>>>> java.net.URLClassLoader.defineClass(URLClassLoader.java:449) >>>>>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71) >>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) >>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >>>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:423) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:356) >>>>>> >>>>>> And that's exactly what just happened to Mozilla: >>>>>> https://bugzilla.mozilla.org/show_bug.cgi?id=1063991 >>>>>> >>>>>> (that also provides a bit of context, as well as the interesting >>>>>> consequence that the Cake$1 you get if you move the Runnable >>>>>> outside the >>>>>> if is different to the one you get in the situation described above >>>>>> (when, really, no Cake$1 at all should exist). >>>>>> >>>>>> Perhaps the desugaring step for inner classes is getting a little... >>>>>> overzealous? >>>>>> >>>>>> I've been able to reproduce this with javac 7 and 8. I've not >>>>>> tried it >>>>>> on the current JDK 9 snapshot. >> > From ella.nekipelova at oracle.com Thu Sep 25 14:56:19 2014 From: ella.nekipelova at oracle.com (ella nekipelova) Date: Thu, 25 Sep 2014 18:56:19 +0400 Subject: Type inference exponential compilation performance Message-ID: <54242D13.3000900@oracle.com> Hello, dear team, I have a question concerning the javac performance in resolving type inference. I found that if there are more than three nested invocations with type inference, compilation might take several minutes. Consider this example: class Test { Test obj; Test(Test other) { this.obj = other.obj; } static Test m(Test src) { return new Test(src); } public static void main(String argv[]) { Test c14 = m(new Test<>(m(new Test<>(m(new Test<>(m(new Test<>(m(new Test<>(m(new Test<>(m(new Test<>(m(new Test<>(m(new Test<>(m(new Test<>(m(new Test<>(m(new Test<>(m(new Test<>(m(new Test<>()))))))))))))))))))))))))))); } } I noticed that there is exponential growth, because for for 8 invocations it takes 20 sec, for 9 - 50 sec, for 10 invocations compilation succeeds in about 3 minutes, but for 14 invocations, as in the example, javac just hangs. It's unlikely that any human developer will ever write such code, but there are different kinds of code generators, and we can't be sure about their implementation. My question is if this behavior is admissible? Are you going to do anything to improve this situation? Thank you, Ella From vicente.romero at oracle.com Thu Sep 25 18:02:17 2014 From: vicente.romero at oracle.com (Vicente-Arturo Romero-Zaldivar) Date: Thu, 25 Sep 2014 11:02:17 -0700 Subject: Type inference exponential compilation performance In-Reply-To: <54242D13.3000900@oracle.com> References: <54242D13.3000900@oracle.com> Message-ID: <542458A9.7000902@oracle.com> Hi Ella, Thanks for your mail. We are aware of the issue and we are actively working on it. Thanks, Vicente On 09/25/2014 07:56 AM, ella nekipelova wrote: > Hello, dear team, > > I have a question concerning the javac performance in resolving type > inference. I found that if there are more than three nested > invocations with type inference, compilation might take several minutes. > Consider this example: > > class Test { > > Test obj; > > Test(Test other) { this.obj = other.obj; } > > static Test m(Test src) { return new Test(src); } > > public static void main(String argv[]) { > > Test c14 = m(new Test<>(m(new Test<>(m(new > Test<>(m(new Test<>(m(new Test<>(m(new Test<>(m(new Test<>(m(new > Test<>(m(new Test<>(m(new Test<>(m(new Test<>(m(new Test<>(m(new > Test<>(m(new Test<>()))))))))))))))))))))))))))); > > } > > } > > I noticed that there is exponential growth, because for for 8 > invocations it takes 20 sec, for 9 - 50 sec, for 10 invocations > compilation succeeds in about 3 minutes, but for 14 invocations, as in > the example, javac just hangs. > > It's unlikely that any human developer will ever write such code, but > there are different kinds of code generators, and we can't be sure > about their implementation. > > My question is if this behavior is admissible? Are you going to do > anything to improve this situation? > > Thank you, > Ella > > From cushon at google.com Thu Sep 25 21:57:16 2014 From: cushon at google.com (Liam Miller-Cushon) Date: Thu, 25 Sep 2014 14:57:16 -0700 Subject: crash with -Xjcov and union types Message-ID: I'm seeing crashes with -Xjcov enabled while compiling code with union types. This seems to affect javac 7 through 9. I've attached a possible fix, and a jtreg test for the crash. Repro: === Test.java === class Test { void m() { try { return; } catch (IllegalStateException | IllegalArgumentException e) { } } } === $ javac Test.java ... An exception has occurred in the compiler (1.8.0_20). Please file a bug at the Java Developer Connection (http://java.sun.com/webapps/bugreport) after checking the Bug Parade for duplicates. Include your program and the following diagnostic in your report. Thank you. java.lang.AssertionError at com.sun.tools.javac.util.Assert.error(Assert.java:126) at com.sun.tools.javac.jvm.CRTable$SourceComputer.visitTree(CRTable.java:529) at com.sun.tools.javac.tree.JCTree$Visitor.visitTypeUnion(JCTree.java:2577) ... -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- # HG changeset patch # User cushon # Date 1411681109 25200 # Node ID ea73aee87ad23ecca9eb6c3133ba66807367d790 # Parent 3c7c7485fab732143c30f4bf71a7ab8411763380 Fix -Xjcov crash with union types. diff -r 3c7c7485fab7 -r ea73aee87ad2 src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/CRTable.java --- a/src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/CRTable.java Thu Sep 25 13:54:45 2014 -0700 +++ b/src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/CRTable.java Thu Sep 25 14:38:29 2014 -0700 @@ -517,6 +517,15 @@ result = sr; } + @Override + public void visitTypeUnion(JCTypeUnion tree) { + SourceRange sr = new SourceRange(startPos(tree), endPos(tree)); + for (JCTree alternative : tree.alternatives) { + sr.mergeWith(csp(alternative)); + } + result = sr; + } + public void visitWildcard(JCWildcard tree) { result = null; } diff -r 3c7c7485fab7 -r ea73aee87ad2 test/tools/javac/options/XjcovUnionType.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/tools/javac/options/XjcovUnionType.java Thu Sep 25 14:38:29 2014 -0700 @@ -0,0 +1,38 @@ +/* + * Copyright (c) 2014, Oracle and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + */ + +/* + * @test + * @bug 1234567 + * @summary -Xjcov causes crash with union types + * @compile -Xjcov XjcovUnionType.java + */ + +public class XjcovUnionType { + public static void main(String[] args) { + try { + return; + } catch (IllegalStateException | IllegalArgumentException e) { + } + } +} From andreas.lundblad at oracle.com Fri Sep 26 13:00:30 2014 From: andreas.lundblad at oracle.com (Andreas Lundblad) Date: Fri, 26 Sep 2014 15:00:30 +0200 Subject: RFR: 8056258: Analysis of public API does not take super classes into account Message-ID: <20140926130029.GH1192@e6430> Hi compiler-dev, Please review the fix for JDK-8056258 below. Description: The current approach for discovering dependencies in sjavac is to hook into Attr and Resolve (by registering subclasses of these classes in the context). This is a bit ad-hoc and does currently not support discovery of super classes. This patch refactors sjavac to instead scan for dependencies in the AST after attr is finished by using a TaskListener. Link to web review: http://cr.openjdk.java.net/~alundblad/8056258 Link to bug reports: http://bugs.openjdk.java.net/browse/JDK-8056258 -- Andreas Lundblad From ella.nekipelova at oracle.com Fri Sep 26 15:13:29 2014 From: ella.nekipelova at oracle.com (ella nekipelova) Date: Fri, 26 Sep 2014 19:13:29 +0400 Subject: Type inference exponential compilation performance In-Reply-To: <542458A9.7000902@oracle.com> References: <54242D13.3000900@oracle.com> <542458A9.7000902@oracle.com> Message-ID: <54258299.6070109@oracle.com> Hi Vicente, Thank you very much for the prompt reply. I'd like also to know if there exist any OpenJDK Jira issue to track the progress on this bug? Thank you, Ella On 25.09.2014 22:02, Vicente-Arturo Romero-Zaldivar wrote: > Hi Ella, > > Thanks for your mail. We are aware of the issue and we are actively > working on it. > > Thanks, > Vicente > > On 09/25/2014 07:56 AM, ella nekipelova wrote: >> Hello, dear team, >> >> I have a question concerning the javac performance in resolving type >> inference. I found that if there are more than three nested >> invocations with type inference, compilation might take several minutes. >> Consider this example: >> >> class Test { >> >> Test obj; >> >> Test(Test other) { this.obj = other.obj; } >> >> static Test m(Test src) { return new Test(src); } >> >> public static void main(String argv[]) { >> >> Test c14 = m(new Test<>(m(new Test<>(m(new >> Test<>(m(new Test<>(m(new Test<>(m(new Test<>(m(new Test<>(m(new >> Test<>(m(new Test<>(m(new Test<>(m(new Test<>(m(new Test<>(m(new >> Test<>(m(new Test<>()))))))))))))))))))))))))))); >> >> } >> >> } >> >> I noticed that there is exponential growth, because for for 8 >> invocations it takes 20 sec, for 9 - 50 sec, for 10 invocations >> compilation succeeds in about 3 minutes, but for 14 invocations, as >> in the example, javac just hangs. >> >> It's unlikely that any human developer will ever write such code, but >> there are different kinds of code generators, and we can't be sure >> about their implementation. >> >> My question is if this behavior is admissible? Are you going to do >> anything to improve this situation? >> >> Thank you, >> Ella >> >> > From oehrstroem at gmail.com Sun Sep 28 12:20:38 2014 From: oehrstroem at gmail.com (=?UTF-8?B?RnJlZHJpayDDlmhyc3Ryw7Zt?=) Date: Sun, 28 Sep 2014 14:20:38 +0200 Subject: RFR: 8056258: Analysis of public API does not take super classes into account In-Reply-To: <20140926130029.GH1192@e6430> References: <20140926130029.GH1192@e6430> Message-ID: Great looking code and I am +1 for committing this as it is! Would you say that scanning the AST incurs a slight performance degradation compared to the previous solution? Can we in the future merge public api generation with AST dependency scanning? On Fri, Sep 26, 2014 at 3:00 PM, Andreas Lundblad < andreas.lundblad at oracle.com> wrote: > Hi compiler-dev, > > Please review the fix for JDK-8056258 below. > > Description: > The current approach for discovering dependencies in sjavac is to hook > into Attr and Resolve (by registering subclasses of these classes in the > context). > This is a bit ad-hoc and does currently not support discovery of super > classes. > This patch refactors sjavac to instead scan for dependencies in the AST > after attr is finished by using a TaskListener. > > Link to web review: > http://cr.openjdk.java.net/~alundblad/8056258 > > Link to bug reports: > http://bugs.openjdk.java.net/browse/JDK-8056258 > > -- Andreas Lundblad > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andreas.lundblad at oracle.com Mon Sep 29 07:26:34 2014 From: andreas.lundblad at oracle.com (Andreas Lundblad) Date: Mon, 29 Sep 2014 09:26:34 +0200 Subject: RFR: 8056258: Analysis of public API does not take super classes into account In-Reply-To: References: <20140926130029.GH1192@e6430> Message-ID: <20140929072633.GA19956@e6430> On Sun, Sep 28, 2014 at 02:20:38PM +0200, Fredrik ?hrstr?m wrote: > Great looking code and I am +1 for committing this as it is! > > Would you say that scanning the AST incurs a slight performance degradation > compared to the previous solution? I've not had time to do any benchmarking unfortunately. I consider this fix to be kind of urgent since the current code is slightly broken. My gut feeling is that it might be slightly slower. It's not anything I've noticed when compiling the JDK though. > Can we in the future merge public api > generation with AST dependency scanning? My plan is to rewrite the dependency scanning in a similar way, but I consider the dependency scanning and public api scanning to be independent and would therefor like to keep the logic separated in the code. -- Andreas From het at google.com Tue Sep 30 01:19:36 2014 From: het at google.com (Harry Terkelsen) Date: Mon, 29 Sep 2014 18:19:36 -0700 Subject: Javac Tree API suggestions to more closely model source code Message-ID: Hi Javac compiler devs, I am writing a Java formatter that makes use of the com.sun.source.tree API as well as the Javac lexer. I see that there is some interest in making the AST more closely model the actual source code ( https://bugs.openjdk.java.net/browse/JDK-8024098). My formatter only changes the whitespace in between tokens, and so must have a completely accurate view of the original source code. Since I've written a pretty large application with the Tree API that must have an accurate picture of the original code, I have compiled a list of difficulties I had with the AST/lexer: Bugs: The lexer com.sun.tools.javac.parser.JavaTokenizer has very useful protected methods processComment(), processWhitespace(), and processLineTerminator() which can be overridden for lexers that care about preserving comments and whitespace. However, none of these 3 useful methods are called immediately before the EOF. For example: class A {} //comment processComment() will not be called in this case (nor will processWhitespace() for the space before the comment). Major pain points: The AST represents multivariable declarations ("int x, y;") as two separate variable declaration statements. This forced my to preprocess all statement lists, and combine consecutive variable declarations into one if SourcePositions#getStartPosition() was the same for the type node of the consecutive variable declarations. The ModifiersTree has pretty much no relationship to the original source code. Implicit non-annotation modifiers are added to the tree by the parser. Repeated modifiers are ignored. There is no way to tell the original order of the modifiers. For annotations, you know the original order they came in, but you don't know how they are interspersed with the modifiers. Enum constants are desugared into variable declarations (with added implicit modifiers). Enum constants are represented as a VariableTree with the RHS being a NewClassTree, but you have to know which parts of the NewClassTree to pick out to reconstruct the original source of the enum constant. The only way to differentiate between an enum constant and an actual variable declaration inside the enum body is to cast to JCTree, which feels very hacky: JCVariableDecl v = (JCVariableDecl) node; return (v.mods.flags & ENUM) != 0; Unary expressions involving literals are sometimes combined into just a literal, and sometimes they aren't. I haven't looked into this very deeply, but sometimes unary expressions with numeric literals will be represented as just a numeric literal, for instance "-1" will sometimes be UnaryTree and sometimes be LiteralTree. If it is LiteralTree the source position will be wrong, it will not include the "-". I guess this is to overcome the problem with -2147483648. Moderate pain points: Class, Enum, Interface, and annotation type declarations are all represented with ClassTree. Since the Kind is set for these it turns out not to be that big of a problem to disambiguate. Cannot tell if an argument VariableTree is varargs. This required casting to JCTree, which feels hacky: JCModifiers mods = (JCModifiers) node.getModifiers(); return (mods.flags & VARARGS) != 0; Minor pain points (small concrete syntax things): No difference between @Annotation() and @Annotation Cannot tell if there is a trailing comma in array literal Cannot tell if there is trailing comma or semicolon after enum constants Cannot tell where array brackets are on variable declarations: int[] x vs int x[], int[] foo() vs int foo()[] Hope this helps! Harry -------------- next part -------------- An HTML attachment was scrubbed... URL: From joel.franck at oracle.com Tue Sep 30 08:20:54 2014 From: joel.franck at oracle.com (=?windows-1252?Q?Joel_Borggr=E9n-Franck?=) Date: Tue, 30 Sep 2014 10:20:54 +0200 Subject: RFR: 8056258: Analysis of public API does not take super classes into account In-Reply-To: References: <20140926130029.GH1192@e6430> Message-ID: Hi, On 28 Sep 2014, at 14:20, Fredrik ?hrstr?m wrote: > Great looking code and I am +1 for committing this as it is! > > Would you say that scanning the AST incurs a slight performance degradation compared to the previous solution? Can we in the future merge public api generation with AST dependency scanning? > IIRC we measured adding a new pass over the AST a while back and it wasn?t noticeable. I do want to redo those measurements to see if this is still the case, but I would suspect it still to be true. cheers /Joel From joel.franck at oracle.com Tue Sep 30 08:38:23 2014 From: joel.franck at oracle.com (=?windows-1252?Q?Joel_Borggr=E9n-Franck?=) Date: Tue, 30 Sep 2014 10:38:23 +0200 Subject: crash with -Xjcov and union types In-Reply-To: References: Message-ID: Hi Liam, On 25 Sep 2014, at 23:57, Liam Miller-Cushon wrote: > I'm seeing crashes with -Xjcov enabled while compiling code with union types. This seems to affect javac 7 through 9. I've attached a possible fix, and a jtreg test for the crash. > > Repro: > Thanks for the report and fix! I have filed https://bugs.openjdk.java.net/browse/JDK-8059453 I don?t know enough about -Xjcov to review and commit your fix right now, but I?ll try to make this happen soon. cheers /Joel From jan.lahoda at oracle.com Tue Sep 30 12:10:19 2014 From: jan.lahoda at oracle.com (Jan Lahoda) Date: Tue, 30 Sep 2014 14:10:19 +0200 Subject: Javac Tree API suggestions to more closely model source code In-Reply-To: References: Message-ID: <542A9DAB.9090904@oracle.com> Hello Harry, Thanks for your feedback! On 30.9.2014 03:19, Harry Terkelsen wrote: > Hi Javac compiler devs, > > I am writing a Java formatter that makes use of the com.sun.source.tree > API as well as the Javac lexer. I see that there is some interest in > making the AST more closely model the actual source code > (https://bugs.openjdk.java.net/browse/JDK-8024098). My formatter only > changes the whitespace in between tokens, and so must have a completely > accurate view of the original source code. Since I've written a pretty > large application with the Tree API that must have an accurate picture > of the original code, I have compiled a list of difficulties I had with > the AST/lexer: > > > Bugs: > The lexer com.sun.tools.javac.parser.JavaTokenizer has very useful > protected methods processComment(), processWhitespace(), and > processLineTerminator() which can be overridden for lexers that care > about preserving comments and whitespace. However, none of these 3 > useful methods are called immediately before the EOF. For example: > > class A {} //comment > > processComment() will not be called in this case (nor will > processWhitespace() for the space before the comment). I see the processComment is not called when the (line) comment is the very last token of the input, but processWhitespace() and processLineTerminator() seem to be called? The processComment is skipped presumably to avoid unnecessary processing, as the JavaTokenizer attaches the comments to the following important token, and for comment at the very end of the input there will never be a token to which it should be attached. I'll investigate if this can be changed. Please note that JavaTokenizer is not a (supported/public) API. > > Major pain points: > The AST represents multivariable declarations ("int x, y;") as two > separate variable declaration statements. This forced my to preprocess > all statement lists, and combine consecutive variable declarations into > one if SourcePositions#getStartPosition() was the same for the type node > of the consecutive variable declarations. Yes, I believe this is a known problem. I've added a note into JDK-8024098. > > The ModifiersTree has pretty much no relationship to the original source > code. Implicit non-annotation modifiers are added to the tree by the > parser. Repeated modifiers are ignored. There is no way to tell the > original order of the modifiers. For annotations, you know the original > order they came in, but you don't know how they are interspersed with > the modifiers. While I understand why you need this information, my personal inclination would be that this should not be part of the AST as such, but should ideally be inferred from other sources (lexer tokens). > > Enum constants are desugared into variable declarations (with added > implicit modifiers). Enum constants are represented as a VariableTree > with the RHS being a NewClassTree, but you have to know which parts of > the NewClassTree to pick out to reconstruct the original source of the > enum constant. The only way to differentiate between an enum constant > and an actual variable declaration inside the enum body is to cast to > JCTree, which feels very hacky: > JCVariableDecl v = (JCVariableDecl) node; > return (v.mods.flags & ENUM) != 0; Yes, too early desugaring of enums is a known problem (see JDK-8024098). > > Unary expressions involving literals are sometimes combined into just a > literal, and sometimes they aren't. I haven't looked into this very > deeply, but sometimes unary expressions with numeric literals will be > represented as just a numeric literal, for instance "-1" will sometimes > be UnaryTree and sometimes be LiteralTree. If it is LiteralTree the > source position will be wrong, it will not include the "-". I guess this > is to overcome the problem with -2147483648. Yes, -2147483648 is the problem. Sorry, but I don't expect that - would be changed to an UnaryTree+LiteralTree. I checked the positions for "int i = -1" (and a few similar cases), and the positions seemed reasonable to me. Is there a sample code where the positions are wrong? > > > Moderate pain points: > Class, Enum, Interface, and annotation type declarations are all > represented with ClassTree. Since the Kind is set for these it turns out > not to be that big of a problem to disambiguate. Sorry, I don't expect this would be changed. The Kind is the correct way to distinguish between the class, enum, interface and annotation type. (As a side note, instanceof is not a reliable way to check the actual type of the tree - Tree.Kind should be used to correctly detect the nature of the given tree). > > Cannot tell if an argument VariableTree is varargs. This required > casting to JCTree, which feels hacky: > JCModifiers mods = (JCModifiers) node.getModifiers(); > return (mods.flags & VARARGS) != 0; > > > Minor pain points (small concrete syntax things): Overall, I would incline to not including things like this in the AST as such. Ideally, there would be a way to infer them from other sources. > No difference between @Annotation() and @Annotation Comparing the end positions of "@Annotation()" and "Annotation" may allow to distinguish between "@Annotation()" and "@Annotation". > Cannot tell if there is a trailing comma in array literal > Cannot tell if there is trailing comma or semicolon after enum constants > Cannot tell where array brackets are on variable declarations: int[] x > vs int x[], int[] foo() vs int foo()[] > > > Hope this helps! Thanks! Jan > Harry From andreas.lundblad at oracle.com Tue Sep 30 12:54:01 2014 From: andreas.lundblad at oracle.com (Andreas Lundblad) Date: Tue, 30 Sep 2014 14:54:01 +0200 Subject: RFR 8054717: SJavac should track changes in the public apis of classpath classes! In-Reply-To: References: Message-ID: <20140930125400.GD19956@e6430> On Sat, Aug 09, 2014 at 12:22:29AM +0200, Fredrik ?hrstr?m wrote: > > [...] > > http://cr.openjdk.java.net/~ohrstrom/webrev-8054717-classpathdep/ > > https://bugs.openjdk.java.net/browse/JDK-8054717 > > //Fredrik Overall I think this is good work and I don't see it as a "feature" but rather a necessity. I don't think the quality of the code is top-notch. Some things I'd prefer we address before pushing this patch, other things I don't mind dealing with myself in later cleanups. My review follows: ---- JavacState.java: > // The set of all classpath packages and their classes, > // for which either the timestamps or the pubapi have changed. > private Map> changedClasspathPackages; Is String really the adequate type for keys/values here? ---- JavacState.java: taintPackagesDependingOnChangedClasspathPackages deals with both parsing and the tainting logic. I suggest we do the parsing in one pass, and then decide what should be tainted. ---- Could you please elaborate on this? > // With two threads compiling our sources, sources compiled by a second thread, might look like > // classpath dependencies to the first thread or vice versa. We cannot remove such fake classpath dependencies > // until the end of the compilation since the knowledge of what is compiled does not exist until now. and this: > // Also, if we doing an incremental compile, then references outside of the small recompiled set, > // will also look like classpath deps, lets remove them as well. I've tried to wrap my head around it without success. (Edit: There's a note on concurrency towards the end of this mail.) ---- Compile.java: I suggest we import the classes that are currently fully qualified. ---- > /** > * Extract the pubapi for a class fn. If the class was fetched from an archive, > * store the archive in the set archives. > */ The functionality of extracting the public API and the functionality for deciding what should end up in the 'archives' collection should probably be separated. The whole pattern of "storing return values in the arguments" (or having side effects in arguments in general) is a bit of a code smell. If possible, the method should be broken down to two separate methods, each one returning a separate result. Otherwise, I reccomend that we create a "Result" class to store the results of the operation. ---- > } catch (Exception e) { > e.printStackTrace(System.err); > } This looks dodgy to me. I think it's better to catch the exceptions we actually suspect could be thrown. (This rarely includes RuntimeExceptions I think.) ---- Package.java: I suggest we go for camel case instead of "pubapi_for_compiled_sources" etc. ---- Util.java: > /** > * Extract the jar/zip/cym archive file name from a classfile tosttring. > */ How did we end up with a String? Why can't we hold on to the original data type? The whole c.startsWith("ZipFileIndexFileObject[") smells. ---- Util.java > if (s == null) return deflt; I don't think the settings-string should be null in the first place. Wherever the settings-string was parsed, it should probably default to the empty string instead. ---- Util.java: > public static void addToMapSet(String k, String v, Map> m) { > Set set = m.get(k); > if (set == null) { > set = new HashSet(); > m.put(k, set); > } > set.add(v); > } is probably better implemented as public static void addToMapSet(K k, V v, Map> m) { m.merge(k, Collections.singleton(v), Util::union); } Same goes for addToMapList Also, I suggest we rename the methods to for instance multimapPut and multimapAppend. ---- A general remark: It's unfortunate that so much machinery is needed to find the file to which a type belongs. (Larger part of Compile.java.) However I don't see a better/more stable way of ensuring that the right answer is found. (In fact, I think the approach is quite clever.) As mentioned in some other email, this whole lookup procedure should probably move to the server in a thin-client approach (It's actually already a bit unfortunate that it's not done in the same VM in a background=true scenario but it shouldn't matter much I guess!) ---- PubapiVisitor.java > List api = new LinkedList(); I suggest we use something better than a String here. The 'api' is a list of *what*? ---- > api.add(depth(indent) + "!TYPE " + e.getQualifiedName()); Are these lines that end up as-is in the sjavac_state file? I think we need better separation between domain objects and state file syntax. I suggest we gather all javac_state syntax related code (parsing and serialization) in one class. Preferrably BuildState.java. (On my second read through, I realize that it might be better to address this in a separate cleanup. Fredrik, if you don't disagree profoundly with this suggestion, would you mind filing an issue on this?) ---- PubapiVisitor: It looks like > public void construct(TypeElement e) constructs the 'api'. I'm surprised that it doesn't *return* the api. Why is it a 'void' method? ---- SourceLocation.java: > } catch (FileNotFoundException e) { > System.err.println("Could not open "+sourceList.getPath()+" since it does not exist!"); > return null; > } catch (IOException e) { > System.err.println("Could not read "+sourceList.getPath()); > return null; > } Is "return null" really appropriate here? How about returning Collections.emptySet or throwing an exception? ---- JavacServiceClient.java: > * Extract the package name and the class from the string. > > ... > > int p = fullname.lastIndexOf('.'); > if (p == -1) { > // Arghh, a source file without a package. > pkg = ":"; > } > else { > pkg = ":"+fullname.substring(0, p); > } The code is cluttered with this kind of string manipulation. I suggest we create a proper datatype instead of doing the above indexOf/substring thingy everywhere. (Second read through: This could probably be addressed in a separate cleanup.) ---- JavacServiceClient.java: This worries me: > synchronized (deps) { > ... > } The only source of concurrency that I know of in sjavac is in PooledSjavac. And as long as an SjavacImpl object (which has no state) is passed as delegate to PooledSjavac, no races should be possible anywhere in the program. If you agree, I suggest we drop 'synchronized' here, otherwise, please enlighten me on why it is neccessary. ---- / Andreas From martinrb at google.com Tue Sep 30 16:44:22 2014 From: martinrb at google.com (Martin Buchholz) Date: Tue, 30 Sep 2014 09:44:22 -0700 Subject: Javac Tree API suggestions to more closely model source code In-Reply-To: <542A9DAB.9090904@oracle.com> References: <542A9DAB.9090904@oracle.com> Message-ID: Just a few meta-comments: As popularized by the Go folk, programming language styles are moving in the direction of automatic reformatting, especially of whitespace, If done well, it's very valuable, freeing the programmer from having to worry about the formatting. But it's very hard for the program to do as well as a careful programmer doing source code layout, reminiscent of the ancient debate of assembly language programmers vs. "high-level" languages like C. The hardest thing to get right is comments (this is the experience of the Go folk). Comments often refer to some other part of the AST. javadoc comments refer to the subsequent program element, end-of-line comments typically refer to the preceding code on the same line (which may not correspond to a single AST node), and other comments may refer to a "section" of subsequent program elements, e.g. // --------------- Constructors ------ One consequence of moving to automated formatting may be a need to introduce new comment types, as with the distinction between /* and /** to communicate intent to the formatter. Although Harry is currently focused on whitespace only, a natural direction is to unconditionally reorder any modifiers encountered into canonical order, as mandated by some existing java coding styles. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonathan.gibbons at oracle.com Tue Sep 30 17:53:30 2014 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Tue, 30 Sep 2014 10:53:30 -0700 Subject: RFR 8054717: SJavac should track changes in the public apis of classpath classes! In-Reply-To: <20140930125400.GD19956@e6430> References: <20140930125400.GD19956@e6430> Message-ID: <542AEE1A.7020301@oracle.com> On 09/30/2014 05:54 AM, Andreas Lundblad wrote: > Util.java: > >> >/** >> > * Extract the jar/zip/cym archive file name from a classfile tosttring. >> > */ > How did we end up with a String? Why can't we hold on to the original data type? The whole c.startsWith("ZipFileIndexFileObject[") smells. > > ---- That's not a smell. That's unacceptable! -- Jon From jonathan.gibbons at oracle.com Tue Sep 30 18:21:54 2014 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Tue, 30 Sep 2014 11:21:54 -0700 Subject: Javac Tree API suggestions to more closely model source code In-Reply-To: References: <542A9DAB.9090904@oracle.com> Message-ID: <542AF4C2.1060702@oracle.com> Internally, the javac tokeniser does distinguish comments, although non-javadoc comments are dropped. The hard thing with comments and ASTs is figuring out where to put them. Even javadoc comments can be problematic. Probably the best that could bbe done is some sort of table on the side. -- Jon On 09/30/2014 09:44 AM, Martin Buchholz wrote: > Just a few meta-comments: > > As popularized by the Go folk, programming language styles are moving > in the direction of automatic reformatting, especially of whitespace, > If done well, it's very valuable, freeing the programmer from having > to worry about the formatting. But it's very hard for the program to > do as well as a careful programmer doing source code layout, > reminiscent of the ancient debate of assembly language programmers vs. > "high-level" languages like C. > > The hardest thing to get right is comments (this is the experience of > the Go folk). Comments often refer to some other part of the AST. > javadoc comments refer to the subsequent program element, end-of-line > comments typically refer to the preceding code on the same line (which > may not correspond to a single AST node), and other comments may refer > to a "section" of subsequent program elements, e.g. > > // --------------- Constructors ------ > > One consequence of moving to automated formatting may be a need to > introduce new comment types, as with the distinction between /* and > /** to communicate intent to the formatter. > > Although Harry is currently focused on whitespace only, a natural > direction is to unconditionally reorder any modifiers encountered into > canonical order, as mandated by some existing java coding styles. From jonathan.gibbons at oracle.com Tue Sep 30 18:24:57 2014 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Tue, 30 Sep 2014 11:24:57 -0700 Subject: Javac Tree API suggestions to more closely model source code In-Reply-To: References: Message-ID: <542AF579.3080909@oracle.com> On 09/29/2014 06:19 PM, Harry Terkelsen wrote: > Cannot tell if an argument VariableTree is varargs. This required > casting to JCTree, which feels hacky: > JCModifiers mods = (JCModifiers) node.getModifiers(); > return (mods.flags & VARARGS) != 0; varargs-ness is a property of an executable element, not of a variable declaration. The method you are looking for is javax.lang.model.element.ExecutableElement#isVarArgs() -- Jon From het at google.com Tue Sep 30 23:06:10 2014 From: het at google.com (Harry Terkelsen) Date: Tue, 30 Sep 2014 16:06:10 -0700 Subject: Best way to get com.sun.source.doctree.DocTree? Message-ID: Hi compiler devs, I would like to use the DocTree API for JavaDoc comments but I don't see any way to get a DocTree from a JavaDoc comment? Is there any way to simply parse a JavaDoc comment, given as a String, along with the start and end positions in the original source, and get back a DocTree with source positions properly set? In other words, is it possible to write this method: DocTree parseDocTree(String comment, int pos, int endPos); If that is not possible, I see that I can go from TreePath -> DocCommentTree with DocTrees#getDocCommentTree(TreePath) but I'm assuming this is only for JavaDoc comments that are attached to top-level program elements. Is this the only way to get DocTrees? Thanks! Harry -------------- next part -------------- An HTML attachment was scrubbed... URL: