From maurizio.cimadamore at oracle.com Mon Oct 1 11:49:49 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 1 Oct 2018 12:49:49 +0100 Subject: on javac flags In-Reply-To: <5BABE8CE.5000208@oracle.com> References: <3a669e77-7ff7-9908-e6da-c8d8e62a3fa6@oracle.com> <2331f0c6-686f-512d-3f4c-53c169ca0e96@oracle.com> <5BABE8CE.5000208@oracle.com> Message-ID: This comment makes a lot of sense Jon. I believe the first thing to do here is to explore some of the use cases of the existing flags, and devise an API that would cover them effectively; then come up with a way to implement this API, whether it's backed by booleans, enum sets or longs, that's something to be decided at a later stage. Thanks Maurizio On 26/09/18 21:15, Jonathan Gibbons wrote: > Maurizio, > > I think it would also be good to separate the storage from the API, to > provide > concise usage within the javac codebase. > > For example, > ??? sym.hasFlag(Flag.STATIC) > ??? sym.setFlag(Flag.PUBLIC) > ??? etc > > as compared to > ?? (sym.flags() & STATIC) != 0 > ??? sym.flags |= STATIC; > > If you get the API right, maybe as a first step, it would then be > easier to tweak > the implementation later on as needed. > > -- Jon > > > On 09/26/2018 01:06 PM, Maurizio Cimadamore wrote: >> Hi Bernard, >> your solution is clever, but now a symbol has an array of N elements >> - which makes it completely unfeasible from a memory footprint >> perspective. Although maybe the core of what you are suggesting is - >> just use a custom data structure, e.g. something like an immutable >> bitset, so that we can share common combinations, etc. >> >> Maurizio >> >> On 26/09/18 20:26, B. Blaser wrote: >>> Hi Maurizio, >>> >>> On Wed, 26 Sep 2018 at 16:17, Maurizio Cimadamore >>> wrote: >>>> Hi, >>>> In javac we have a Flags class which is used to define a list of >>>> all the >>>> flags that are used, both internally (i.e. javac private flags) and >>>> externally (i.e ACC_ flags). The assumption is that external flags are >>>> 16-bit wide, while internal flags are all values above 2^16 (_which >>>> can >>>> fit in a long_). >>>> >>>> There are several issues with the current handling of flags: >>>> [...] >>>> * checking presence/absence of flags is very tedious; we have many >>>> many >>>> occurrences (600+) of C-like code like: >>>> >>>> if ((sym.flags() & STATIC) == 0 && ... >>>> >>>> or >>>> >>>> (sym.flags() & (ABSTRACT|DEFAULT|PRIVATE)) == ABSTRACT) { ... } >>>> >>>> [...] >>>> Where does this leave us? I think with these optimizations the memory >>>> footprint should be relatively contained - we are essentially >>>> trading a >>>> 64-bit value (long flag) with a 64 bit pointer which might or might >>>> not >>>> point to a fresh new object (if not, no extra cost). Of course this >>>> all >>>> needed to be validated with some real world profiling/JMH >>>> benchmark, but >>>> seems like a direction worth exploring. >>>> >>>> Thoughts? >>> I'm afraid that >>> >>> if (!sym.flags().contains(Flag.STATIC) && ... >>> >>> would be also tedious, somewhat slow and probably expensive in term of >>> memory usage... >>> >>> I guess I'd be more for something like: >>> >>> $ cat OnFlags.java >>> class OnFlags { >>> ???? static enum Flag { >>> ???????? PUBLIC(0, 0x1), PRIVATE(1, 0x2), INTERNAL(2, 0x0); >>> >>> ???????? int index; >>> ???????? int mask; // maybe 'short' would be enough >>> >>> ???????? Flag(int index, int mask) { >>> ???????????? this.index = index; >>> ???????????? this.mask = mask; >>> ???????? } >>> ???? } >>> >>> ???? static class Symbol { >>> ???????? boolean[] flags = new boolean[Flag.values().length]; >>> ???? } >>> >>> ???? public static void main(String... args) { >>> ???????? var sym = new Symbol(); >>> ???????? sym.flags[Flag.PUBLIC.index] = true; >>> ???????? sym.flags[Flag.INTERNAL.index] = true; >>> >>> ???????? int jvmFlags = 0; >>> ???????? for (Flag f: Flag.values()) >>> ???????????? if (sym.flags[f.index]) >>> ???????????????? jvmFlags |= f.mask; >>> >>> ???????? for (Flag f: Flag.values()) >>> ???????????? System.out.println(f + ": " + sym.flags[f.index]); >>> >>> ???????? System.out.println("jvm flags " + jvmFlags); >>> ???? } >>> } >>> >>> No difference between corresponding internal and external flags, so no >>> mapping would be necessary. >>> >>> Testing a flag would be as fast as simple without much extra memory >>> cost: >>> >>> if (!sym.flags[Flag.STATIC.index] && ... >>> >>> Computing the jvm external flag would be also straightforward: >>> >>> int jvmFlags = 0; >>> for (Flag f: Flag.values()) >>> ???? if (sym.flags[f.index]) >>> ???????? jvmFlags |= f.mask; >>> >>> Opinions? >>> >>> Bernard >>> >>>> Maurizio >> > From cushon at google.com Tue Oct 2 04:26:22 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Mon, 1 Oct 2018 21:26:22 -0700 Subject: RFR: 8211057: Gensrc step CompileProperties generates unstable CompilerProperties output Message-ID: Hi, Please review this fix to ClassGenerator, which ensures it produces deterministic output. Previously the order of FactoryKinds in the output relied on HashMap iteration order. bug: https://bugs.openjdk.java.net/browse/JDK-8211057 webrev: http://cr.openjdk.java.net/~cushon/8211057/webrev.00/index.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From maurizio.cimadamore at oracle.com Tue Oct 2 10:10:16 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Tue, 2 Oct 2018 11:10:16 +0100 Subject: RFR: 8211057: Gensrc step CompileProperties generates unstable CompilerProperties output In-Reply-To: References: Message-ID: <3d8fa338-35a2-8725-4bfe-1a0dfa5bf1d9@oracle.com> Whoops - you beat me to it. Looks good Maurizio On 02/10/18 05:26, Liam Miller-Cushon wrote: > Hi, > > Please review this fix to ClassGenerator, which ensures it produces > deterministic output. > Previously the order of FactoryKinds in the output relied on HashMap > iteration order. > > bug: https://bugs.openjdk.java.net/browse/JDK-8211057 > webrev: > http://cr.openjdk.java.net/~cushon/8211057/webrev.00/index.html > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vicente.romero at oracle.com Tue Oct 2 13:11:35 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Tue, 2 Oct 2018 09:11:35 -0400 Subject: RFR: 8211057: Gensrc step CompileProperties generates unstable CompilerProperties output In-Reply-To: References: Message-ID: <5ac44f0b-10d9-8685-ba1a-43b9877648c6@oracle.com> looks good, Vicente On 10/02/2018 12:26 AM, Liam Miller-Cushon wrote: > Hi, > > Please review this fix to ClassGenerator, which ensures it produces > deterministic output. > Previously the order of FactoryKinds in the output relied on HashMap > iteration order. > > bug: https://bugs.openjdk.java.net/browse/JDK-8211057 > webrev: > http://cr.openjdk.java.net/~cushon/8211057/webrev.00/index.html > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cushon at google.com Tue Oct 2 16:38:15 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Tue, 2 Oct 2018 09:38:15 -0700 Subject: RFR: 8211057: Gensrc step CompileProperties generates unstable CompilerProperties output In-Reply-To: <5ac44f0b-10d9-8685-ba1a-43b9877648c6@oracle.com> References: <5ac44f0b-10d9-8685-ba1a-43b9877648c6@oracle.com> Message-ID: I noticed this earlier and the bug prompted me to actually do something about it :) Thanks for the reviews. On Tue, Oct 2, 2018 at 6:11 AM Vicente Romero wrote: > looks good, > Vicente > > On 10/02/2018 12:26 AM, Liam Miller-Cushon wrote: > > Hi, > > Please review this fix to ClassGenerator, which ensures it produces > deterministic output. > Previously the order of FactoryKinds in the output relied on HashMap > iteration order. > > bug: https://bugs.openjdk.java.net/browse/JDK-8211057 > webrev: http://cr.openjdk.java.net/~cushon/8211057/webrev.00/index.html > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vicente.romero at oracle.com Wed Oct 3 00:40:56 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Tue, 2 Oct 2018 20:40:56 -0400 Subject: RFR: JDK-8211148: javac -source 10 in JDK11 is inconsistent with javac in JDK10 Message-ID: <10617f97-fb6b-7780-bd6b-031dbba35020@oracle.com> Hi, Please review the fix for [1] at [2]. There were a number of places where the compiler was not checking if var syntax for implicit lambdas was allowed or not. This implied that the JDK11 compiler was accepting code with this feature even if the -source 10 option was passed, this shouldn't happen. This patch fixes this and issues the: "var not allowed here" error message as expected. Thanks, Vicente [1] https://bugs.openjdk.java.net/browse/JDK-8211148 [2] http://cr.openjdk.java.net/~vromero/8211148/webrev.00/jdk.dev.patch From maurizio.cimadamore at oracle.com Wed Oct 3 10:18:45 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 3 Oct 2018 11:18:45 +0100 Subject: RFR: JDK-8211148: javac -source 10 in JDK11 is inconsistent with javac in JDK10 In-Reply-To: <10617f97-fb6b-7780-bd6b-031dbba35020@oracle.com> References: <10617f97-fb6b-7780-bd6b-031dbba35020@oracle.com> Message-ID: The changes look generally good, but I have some questions/comments. * (cosmetic, optional) This change: if (Feature.VAR_SYNTAX_IMPLICIT_LAMBDAS.allowedInSource(source)) { + log.error(DiagnosticFlag.SYNTAX, param.pos, Errors.VarNotAllowedArray); + } else { + log.error(DiagnosticFlag.SYNTAX, param.pos, Errors.VarNotAllowedHere); + } Could be slightly simplified as: log.error(DiagnosticFlag.SYNTAX, param.pos, Feature.VAR_SYNTAX_IMPLICIT_LAMBDAS.allowedInSource(source) ? Errors.VarNotAllowedArray : Errors.VarNotAllowedHere); * A more serious issue is, I think that the code here: if (Feature.VAR_SYNTAX_IMPLICIT_LAMBDAS.allowedInSource(source)) { + param.startPos = TreeInfo.getStartPos(param.vartype); + param.vartype = null; + } else { + log.error(DiagnosticFlag.SYNTAX, param.pos, Errors.VarNotAllowedHere); + } Should really call the parser method 'checkSourceLevel' which will issue the correct error if the feature is not allowed (e.g. 'var in lambda not allowed, use source 11 to enable'). Which brings to: * You should define a compiler key for the new feature (e.g. 'var in lambda') so that the framework will pick that up when generating a diagnostic using the above method * In the classification logic, I believe the goal of the changes is to disable the diagnostic generation for illegal combinations of lambda parameter declarations when one or more of these involve a 'var' which would have already been flagged in a separate diagnostic. That is, these changes are to avoid to report duplicate and spurious diagnostics, right? * In the LambdaParserTest I see no relevant changes, but still the patch shows something - is there some trailing space or indent different at play? Maurizio On 03/10/18 01:40, Vicente Romero wrote: > Hi, > > Please review the fix for [1] at [2]. There were a number of places > where the compiler was not checking if var syntax for implicit lambdas > was allowed or not. This implied that the JDK11 compiler was accepting > code with this feature even if the -source 10 option was passed, this > shouldn't happen. This patch fixes this and issues the: "var not > allowed here" error message as expected. > > Thanks, > Vicente > > [1] https://bugs.openjdk.java.net/browse/JDK-8211148 > [2] http://cr.openjdk.java.net/~vromero/8211148/webrev.00/jdk.dev.patch From magnus.ihse.bursie at oracle.com Wed Oct 3 10:21:51 2018 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 3 Oct 2018 12:21:51 +0200 Subject: RFR: 8211057: Gensrc step CompileProperties generates unstable CompilerProperties output In-Reply-To: References: <5ac44f0b-10d9-8685-ba1a-43b9877648c6@oracle.com> Message-ID: On 2018-10-02 18:38, Liam Miller-Cushon wrote: > I noticed this earlier and?the bug prompted me to actually do > something about it :) Thank you Liam for fixing this! /Magnus > > Thanks for the reviews. > > On Tue, Oct 2, 2018 at 6:11 AM Vicente Romero > > wrote: > > looks good, > Vicente > > On 10/02/2018 12:26 AM, Liam Miller-Cushon wrote: >> Hi, >> >> Please review this fix to ClassGenerator, which ensures it >> produces deterministic output. >> Previously the order of FactoryKinds in the output relied on >> HashMap iteration order. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8211057 >> webrev: >> http://cr.openjdk.java.net/~cushon/8211057/webrev.00/index.html >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ronshapiro at google.com Wed Oct 3 11:20:48 2018 From: ronshapiro at google.com (Ron Shapiro) Date: Wed, 3 Oct 2018 14:20:48 +0300 Subject: Name.contentEquals() performance Message-ID: In my profiling of javac compilations of large builds (>30 seconds), I'm seeing about a noticeable amount of time being attributed to com.sun.tools.javac.util.Name.contentEquals(), because it creates a new String each time in it's toString().equals(cs.toString()) check. I'm wondering if we can optimize this without much cost. I had suggested to Liam that we could add an `asString` variable that is lazily computed in toString(). This would also benefit all of the other CharSequence methods that delegate to the toString() representation. Liam had another idea that would avoid adding a field (and the associated memory that comes with that), taking advantage of the fact that Names are interned. if (cs instanceof Name) { Name other = (Name) cs; if (table == other.table) { return this == other; } } return toString().equals(cs.toString()); The first option reduced builds that I tested of 45 seconds by 450ms (1%). Liam's suggestion reduced bulilds by 400-420ms, so in the same ballpark but a bit slower (but again, it allows for less retained memory?). I figured both are simple, so I'd propose both. -------------- next part -------------- An HTML attachment was scrubbed... URL: From maurizio.cimadamore at oracle.com Wed Oct 3 13:16:00 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 3 Oct 2018 14:16:00 +0100 Subject: Name.contentEquals() performance In-Reply-To: References: Message-ID: <4617a955-de22-f5d3-d9af-ed8e0dc2eb40@oracle.com> Hi, If you are comparing names (to make sure of which requires an instanceof, which I guess adds its own cost), the interning trick seems to be the best IMHO. I'm a bit skeptical about adding a new field on Name; a simple compilation of an hello world program will create roughly 3K names. In general you need a new name for each new field, variable, method, class/interface, type-variable, ... so the number of names can become big pretty rapidly. (That said, out of curiosity I tried to instrument the JDK build to see how many names were created (this is a bit hard given that the build internally reuses java compilers using the sjavac server), but name creation seem to peak at around 33K, which would give an overhead of a couple of hundred of kB for an extra field like the one you suggest, which doesn't seem excessive in terms of footprint increase) So, given the performance improvement you get out of this is rather limited, perhaps going for a more conservative fix (e.g. interning) could be the better? As for your claim of less retained memory it's really hard to tell w/o looking at some targeted JMH benchmark - it's possible that the JIT might decide to escape analyze away the strings since they are thrown away immediately after the comparison, so that no real allocation actually occurs? In any case I would expect modern GCs to be able to cope quite well with this kind of throwaway allocation with TLABs etc. so I'm a bit on the fence. Maurizio On 03/10/18 12:20, Ron Shapiro wrote: > In my profiling of javac compilations?of large builds (>30 seconds), > I'm seeing about a noticeable amount of time being attributed to > com.sun.tools.javac.util.Name.contentEquals(), because it creates a > new String each time in it's toString().equals(cs.toString()) check. > I'm wondering if we can optimize this without much cost. > > I had suggested to Liam that we could add an `asString` variable that > is lazily computed in toString(). This would also benefit all of the > other CharSequence methods that delegate to the toString() representation. > > Liam had another idea that would avoid adding a field (and the > associated memory that comes with that), taking advantage of the fact > that Names are interned. > > if (cs instanceof Name) { > ? Name other = (Name) cs; > ? if (table == other.table) { > ? ? return this == other; > ? } > } > return toString().equals(cs.toString()); > > The first option reduced builds that I tested of 45 seconds by 450ms > (1%). Liam's suggestion reduced bulilds by 400-420ms, so in the same > ballpark but a bit slower (but again, it allows for less retained > memory?). > > I figured both are simple, so I'd propose both. From maurizio.cimadamore at oracle.com Wed Oct 3 13:22:00 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 3 Oct 2018 14:22:00 +0100 Subject: Name.contentEquals() performance In-Reply-To: <4617a955-de22-f5d3-d9af-ed8e0dc2eb40@oracle.com> References: <4617a955-de22-f5d3-d9af-ed8e0dc2eb40@oracle.com> Message-ID: <5693ec10-55f3-6f5a-d6bb-d811df1faeaf@oracle.com> On 03/10/18 14:16, Maurizio Cimadamore wrote: > (That said, out of curiosity I tried to instrument the JDK build to > see how many names were created (this is a bit hard given that the > build internally reuses java compilers using the sjavac server), but > name creation seem to peak at around 33K, which would give an overhead > of a couple of hundred of kB for an extra field like the one you > suggest, which doesn't seem excessive in terms of footprint increase) Actually, now that I think more, if you store the toString() result in an extra field, while the footprint of the Name class (shallow size) won't change much (8 more bytes per instance), we could potentially be talking about 33K more (in the JDK build case) strings be retained by the javac code - which is probably going to balloon up the memory footprint cost significantly. Maurizio From vicente.romero at oracle.com Wed Oct 3 16:39:14 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Wed, 3 Oct 2018 12:39:14 -0400 Subject: RFR: JDK-8211148: javac -source 10 in JDK11 is inconsistent with javac in JDK10 In-Reply-To: References: <10617f97-fb6b-7780-bd6b-031dbba35020@oracle.com> Message-ID: Hi Maurizio, Thanks for the review. I have updated the webrev [1] [1] http://cr.openjdk.java.net/~vromero/8211148/webrev.01/jdk.dev.patch some comments below. On 10/03/2018 06:18 AM, Maurizio Cimadamore wrote: > The changes look generally good, but I have some questions/comments. > > * (cosmetic, optional) This change: > > if (Feature.VAR_SYNTAX_IMPLICIT_LAMBDAS.allowedInSource(source)) { > +??????????????????????? log.error(DiagnosticFlag.SYNTAX, param.pos, > Errors.VarNotAllowedArray); > +??????????????????? } else { > +??????????????????????? log.error(DiagnosticFlag.SYNTAX, param.pos, > Errors.VarNotAllowedHere); > +??????????????????? } > > Could be slightly simplified as: > > log.error(DiagnosticFlag.SYNTAX, param.pos, > Feature.VAR_SYNTAX_IMPLICIT_LAMBDAS.allowedInSource(source) > ?? Errors.VarNotAllowedArray : Errors.VarNotAllowedHere); done it > > > * A more serious issue is, I think that the code here: > > if (Feature.VAR_SYNTAX_IMPLICIT_LAMBDAS.allowedInSource(source)) { > +??????????????????????? param.startPos = > TreeInfo.getStartPos(param.vartype); > +??????????????????????? param.vartype = null; > +??????????????????? } else { > +??????????????????????? log.error(DiagnosticFlag.SYNTAX, param.pos, > Errors.VarNotAllowedHere); > +??????????????????? } > > Should really call the parser method 'checkSourceLevel' which will > issue the correct error if the feature is not allowed (e.g. 'var in > lambda not allowed, use source 11 to enable'). Which brings to: > > * You should define a compiler key for the new feature (e.g. 'var in > lambda') so that the framework will pick that up when generating a > diagnostic using the above method done too, > > * In the classification logic, I believe the goal of the changes is to > disable the diagnostic generation for illegal combinations of lambda > parameter declarations when one or more of these involve a 'var' which > would have already been flagged in a separate diagnostic. That is, > these changes are to avoid to report duplicate and spurious > diagnostics, right? yep, I reorganized the matrix putting the var rows and columns to the top and to the left, there wasn't a real reason to to this, but I thought that it could help reading the code in the future. Probably very subjective but still :) then yes if the source doesn't allow var in implicit lambdas a `mask` is applied to the value read from the matrix effectively changing it to a null to avoid duplicate reports. > > * In the LambdaParserTest I see no relevant changes, but still the > patch shows something - is there some trailing space or indent > different at play? the change here was to add a case for which errors are expected when var is used and source is 10 > > Maurizio Vicente > > On 03/10/18 01:40, Vicente Romero wrote: >> Hi, >> >> Please review the fix for [1] at [2]. There were a number of places >> where the compiler was not checking if var syntax for implicit >> lambdas was allowed or not. This implied that the JDK11 compiler was >> accepting code with this feature even if the -source 10 option was >> passed, this shouldn't happen. This patch fixes this and issues the: >> "var not allowed here" error message as expected. >> >> Thanks, >> Vicente >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8211148 >> [2] http://cr.openjdk.java.net/~vromero/8211148/webrev.00/jdk.dev.patch > From jonathan.gibbons at oracle.com Wed Oct 3 18:48:53 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Wed, 03 Oct 2018 11:48:53 -0700 Subject: Name.contentEquals() performance In-Reply-To: <5693ec10-55f3-6f5a-d6bb-d811df1faeaf@oracle.com> References: <4617a955-de22-f5d3-d9af-ed8e0dc2eb40@oracle.com> <5693ec10-55f3-6f5a-d6bb-d811df1faeaf@oracle.com> Message-ID: <5BB50F15.6030701@oracle.com> It is very depressing that Name.contentEquals actually creates strings. I would have expected it to iterate over the code points of the items being compared. It is also somewhat depressing that .contentEquals is being used so much, because the intent of the Name class is to function as an interned string. -- Jon On 10/03/2018 06:22 AM, Maurizio Cimadamore wrote: > > > On 03/10/18 14:16, Maurizio Cimadamore wrote: >> (That said, out of curiosity I tried to instrument the JDK build to >> see how many names were created (this is a bit hard given that the >> build internally reuses java compilers using the sjavac server), but >> name creation seem to peak at around 33K, which would give an >> overhead of a couple of hundred of kB for an extra field like the one >> you suggest, which doesn't seem excessive in terms of footprint >> increase) > Actually, now that I think more, if you store the toString() result in > an extra field, while the footprint of the Name class (shallow size) > won't change much (8 more bytes per instance), we could potentially be > talking about 33K more (in the JDK build case) strings be retained by > the javac code - which is probably going to balloon up the memory > footprint cost significantly. > > Maurizio From maurizio.cimadamore at oracle.com Thu Oct 4 10:53:31 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 4 Oct 2018 11:53:31 +0100 Subject: RFR: JDK-8211148: javac -source 10 in JDK11 is inconsistent with javac in JDK10 In-Reply-To: References: <10617f97-fb6b-7780-bd6b-031dbba35020@oracle.com> Message-ID: Thanks, looks good. As for the test comment, I was referring to this hunk of the patch: - EXPLIICT_SIMPLE("A", ExplicitKind.EXPLICIT), - EXPLIICT_SIMPLE_ARR1("A[]", ExplicitKind.EXPLICIT), - EXPLIICT_SIMPLE_ARR2("A[][]", ExplicitKind.EXPLICIT), + EXPLICIT_SIMPLE("A", ExplicitKind.EXPLICIT), + EXPLICIT_SIMPLE_ARR1("A[]", ExplicitKind.EXPLICIT), + EXPLICIT_SIMPLE_ARR2("A[][]", ExplicitKind.EXPLICIT), Looks like the contents are the same, what's up? Maurizio On 03/10/18 17:39, Vicente Romero wrote: > Hi Maurizio, > > Thanks for the review. I have updated the webrev [1] > > [1] http://cr.openjdk.java.net/~vromero/8211148/webrev.01/jdk.dev.patch > > some comments below. > > On 10/03/2018 06:18 AM, Maurizio Cimadamore wrote: >> The changes look generally good, but I have some questions/comments. >> >> * (cosmetic, optional) This change: >> >> if (Feature.VAR_SYNTAX_IMPLICIT_LAMBDAS.allowedInSource(source)) { >> +??????????????????????? log.error(DiagnosticFlag.SYNTAX, param.pos, >> Errors.VarNotAllowedArray); >> +??????????????????? } else { >> +??????????????????????? log.error(DiagnosticFlag.SYNTAX, param.pos, >> Errors.VarNotAllowedHere); >> +??????????????????? } >> >> Could be slightly simplified as: >> >> log.error(DiagnosticFlag.SYNTAX, param.pos, >> Feature.VAR_SYNTAX_IMPLICIT_LAMBDAS.allowedInSource(source) >> ?? Errors.VarNotAllowedArray : Errors.VarNotAllowedHere); > > done it >> >> >> * A more serious issue is, I think that the code here: >> >> if (Feature.VAR_SYNTAX_IMPLICIT_LAMBDAS.allowedInSource(source)) { >> +??????????????????????? param.startPos = >> TreeInfo.getStartPos(param.vartype); >> +??????????????????????? param.vartype = null; >> +??????????????????? } else { >> +??????????????????????? log.error(DiagnosticFlag.SYNTAX, param.pos, >> Errors.VarNotAllowedHere); >> +??????????????????? } >> >> Should really call the parser method 'checkSourceLevel' which will >> issue the correct error if the feature is not allowed (e.g. 'var in >> lambda not allowed, use source 11 to enable'). Which brings to: >> >> * You should define a compiler key for the new feature (e.g. 'var in >> lambda') so that the framework will pick that up when generating a >> diagnostic using the above method > > done too, > >> >> * In the classification logic, I believe the goal of the changes is >> to disable the diagnostic generation for illegal combinations of >> lambda parameter declarations when one or more of these involve a >> 'var' which would have already been flagged in a separate diagnostic. >> That is, these changes are to avoid to report duplicate and spurious >> diagnostics, right? > > yep, I reorganized the matrix putting the var rows and columns to the > top and to the left, there wasn't a real reason to to this, but I > thought that it could help reading the code in the future. Probably > very subjective but still :) then yes if the source doesn't allow var > in implicit lambdas a `mask` is applied to the value read from the > matrix effectively changing it to a null to avoid duplicate reports. >> >> * In the LambdaParserTest I see no relevant changes, but still the >> patch shows something - is there some trailing space or indent >> different at play? > > the change here was to add a case for which errors are expected when > var is used and source is 10 >> >> Maurizio > > Vicente > >> >> On 03/10/18 01:40, Vicente Romero wrote: >>> Hi, >>> >>> Please review the fix for [1] at [2]. There were a number of places >>> where the compiler was not checking if var syntax for implicit >>> lambdas was allowed or not. This implied that the JDK11 compiler was >>> accepting code with this feature even if the -source 10 option was >>> passed, this shouldn't happen. This patch fixes this and issues the: >>> "var not allowed here" error message as expected. >>> >>> Thanks, >>> Vicente >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8211148 >>> [2] http://cr.openjdk.java.net/~vromero/8211148/webrev.00/jdk.dev.patch >> > From vicente.romero at oracle.com Thu Oct 4 12:40:14 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Thu, 4 Oct 2018 08:40:14 -0400 Subject: RFR: JDK-8211148: javac -source 10 in JDK11 is inconsistent with javac in JDK10 In-Reply-To: References: <10617f97-fb6b-7780-bd6b-031dbba35020@oracle.com> Message-ID: On 10/04/2018 06:53 AM, Maurizio Cimadamore wrote: > Thanks, looks good. As for the test comment, I was referring to this > hunk of the patch: > > -??????? EXPLIICT_SIMPLE("A", ExplicitKind.EXPLICIT), > -??????? EXPLIICT_SIMPLE_ARR1("A[]", ExplicitKind.EXPLICIT), > -??????? EXPLIICT_SIMPLE_ARR2("A[][]", ExplicitKind.EXPLICIT), > +??????? EXPLICIT_SIMPLE("A", ExplicitKind.EXPLICIT), > +??????? EXPLICIT_SIMPLE_ARR1("A[]", ExplicitKind.EXPLICIT), > +??????? EXPLICIT_SIMPLE_ARR2("A[][]", ExplicitKind.EXPLICIT), > > > Looks like the contents are the same, what's up? I see, I was just fixing a typo: EXPLIICT for EXPLICIT > > Maurizio Vicente > > On 03/10/18 17:39, Vicente Romero wrote: >> Hi Maurizio, >> >> Thanks for the review. I have updated the webrev [1] >> >> [1] http://cr.openjdk.java.net/~vromero/8211148/webrev.01/jdk.dev.patch >> >> some comments below. >> >> On 10/03/2018 06:18 AM, Maurizio Cimadamore wrote: >>> The changes look generally good, but I have some questions/comments. >>> >>> * (cosmetic, optional) This change: >>> >>> if (Feature.VAR_SYNTAX_IMPLICIT_LAMBDAS.allowedInSource(source)) { >>> +??????????????????????? log.error(DiagnosticFlag.SYNTAX, param.pos, >>> Errors.VarNotAllowedArray); >>> +??????????????????? } else { >>> +??????????????????????? log.error(DiagnosticFlag.SYNTAX, param.pos, >>> Errors.VarNotAllowedHere); >>> +??????????????????? } >>> >>> Could be slightly simplified as: >>> >>> log.error(DiagnosticFlag.SYNTAX, param.pos, >>> Feature.VAR_SYNTAX_IMPLICIT_LAMBDAS.allowedInSource(source) >>> ?? Errors.VarNotAllowedArray : Errors.VarNotAllowedHere); >> >> done it >>> >>> >>> * A more serious issue is, I think that the code here: >>> >>> if (Feature.VAR_SYNTAX_IMPLICIT_LAMBDAS.allowedInSource(source)) { >>> +??????????????????????? param.startPos = >>> TreeInfo.getStartPos(param.vartype); >>> +??????????????????????? param.vartype = null; >>> +??????????????????? } else { >>> +??????????????????????? log.error(DiagnosticFlag.SYNTAX, param.pos, >>> Errors.VarNotAllowedHere); >>> +??????????????????? } >>> >>> Should really call the parser method 'checkSourceLevel' which will >>> issue the correct error if the feature is not allowed (e.g. 'var in >>> lambda not allowed, use source 11 to enable'). Which brings to: >>> >>> * You should define a compiler key for the new feature (e.g. 'var in >>> lambda') so that the framework will pick that up when generating a >>> diagnostic using the above method >> >> done too, >> >>> >>> * In the classification logic, I believe the goal of the changes is >>> to disable the diagnostic generation for illegal combinations of >>> lambda parameter declarations when one or more of these involve a >>> 'var' which would have already been flagged in a separate >>> diagnostic. That is, these changes are to avoid to report duplicate >>> and spurious diagnostics, right? >> >> yep, I reorganized the matrix putting the var rows and columns to the >> top and to the left, there wasn't a real reason to to this, but I >> thought that it could help reading the code in the future. Probably >> very subjective but still :) then yes if the source doesn't allow var >> in implicit lambdas a `mask` is applied to the value read from the >> matrix effectively changing it to a null to avoid duplicate reports. >>> >>> * In the LambdaParserTest I see no relevant changes, but still the >>> patch shows something - is there some trailing space or indent >>> different at play? >> >> the change here was to add a case for which errors are expected when >> var is used and source is 10 >>> >>> Maurizio >> >> Vicente >> >>> >>> On 03/10/18 01:40, Vicente Romero wrote: >>>> Hi, >>>> >>>> Please review the fix for [1] at [2]. There were a number of places >>>> where the compiler was not checking if var syntax for implicit >>>> lambdas was allowed or not. This implied that the JDK11 compiler >>>> was accepting code with this feature even if the -source 10 option >>>> was passed, this shouldn't happen. This patch fixes this and issues >>>> the: "var not allowed here" error message as expected. >>>> >>>> Thanks, >>>> Vicente >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8211148 >>>> [2] >>>> http://cr.openjdk.java.net/~vromero/8211148/webrev.00/jdk.dev.patch >>> >> > From maurizio.cimadamore at oracle.com Thu Oct 4 12:41:03 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 4 Oct 2018 13:41:03 +0100 Subject: RFR: JDK-8211148: javac -source 10 in JDK11 is inconsistent with javac in JDK10 In-Reply-To: References: <10617f97-fb6b-7780-bd6b-031dbba35020@oracle.com> Message-ID: On 04/10/18 13:40, Vicente Romero wrote: > > > On 10/04/2018 06:53 AM, Maurizio Cimadamore wrote: >> Thanks, looks good. As for the test comment, I was referring to this >> hunk of the patch: >> >> -??????? EXPLIICT_SIMPLE("A", ExplicitKind.EXPLICIT), >> -??????? EXPLIICT_SIMPLE_ARR1("A[]", ExplicitKind.EXPLICIT), >> -??????? EXPLIICT_SIMPLE_ARR2("A[][]", ExplicitKind.EXPLICIT), >> +??????? EXPLICIT_SIMPLE("A", ExplicitKind.EXPLICIT), >> +??????? EXPLICIT_SIMPLE_ARR1("A[]", ExplicitKind.EXPLICIT), >> +??????? EXPLICIT_SIMPLE_ARR2("A[][]", ExplicitKind.EXPLICIT), >> >> >> Looks like the contents are the same, what's up? > > I see, I was just fixing a typo: EXPLIICT for EXPLICIT Ah! thanks :-) Maurizio >> >> Maurizio > > Vicente >> >> On 03/10/18 17:39, Vicente Romero wrote: >>> Hi Maurizio, >>> >>> Thanks for the review. I have updated the webrev [1] >>> >>> [1] http://cr.openjdk.java.net/~vromero/8211148/webrev.01/jdk.dev.patch >>> >>> some comments below. >>> >>> On 10/03/2018 06:18 AM, Maurizio Cimadamore wrote: >>>> The changes look generally good, but I have some questions/comments. >>>> >>>> * (cosmetic, optional) This change: >>>> >>>> if (Feature.VAR_SYNTAX_IMPLICIT_LAMBDAS.allowedInSource(source)) { >>>> +??????????????????????? log.error(DiagnosticFlag.SYNTAX, >>>> param.pos, Errors.VarNotAllowedArray); >>>> +??????????????????? } else { >>>> +??????????????????????? log.error(DiagnosticFlag.SYNTAX, >>>> param.pos, Errors.VarNotAllowedHere); >>>> +??????????????????? } >>>> >>>> Could be slightly simplified as: >>>> >>>> log.error(DiagnosticFlag.SYNTAX, param.pos, >>>> Feature.VAR_SYNTAX_IMPLICIT_LAMBDAS.allowedInSource(source) >>>> ?? Errors.VarNotAllowedArray : Errors.VarNotAllowedHere); >>> >>> done it >>>> >>>> >>>> * A more serious issue is, I think that the code here: >>>> >>>> if (Feature.VAR_SYNTAX_IMPLICIT_LAMBDAS.allowedInSource(source)) { >>>> +??????????????????????? param.startPos = >>>> TreeInfo.getStartPos(param.vartype); >>>> +??????????????????????? param.vartype = null; >>>> +??????????????????? } else { >>>> +??????????????????????? log.error(DiagnosticFlag.SYNTAX, >>>> param.pos, Errors.VarNotAllowedHere); >>>> +??????????????????? } >>>> >>>> Should really call the parser method 'checkSourceLevel' which will >>>> issue the correct error if the feature is not allowed (e.g. 'var in >>>> lambda not allowed, use source 11 to enable'). Which brings to: >>>> >>>> * You should define a compiler key for the new feature (e.g. 'var >>>> in lambda') so that the framework will pick that up when generating >>>> a diagnostic using the above method >>> >>> done too, >>> >>>> >>>> * In the classification logic, I believe the goal of the changes is >>>> to disable the diagnostic generation for illegal combinations of >>>> lambda parameter declarations when one or more of these involve a >>>> 'var' which would have already been flagged in a separate >>>> diagnostic. That is, these changes are to avoid to report duplicate >>>> and spurious diagnostics, right? >>> >>> yep, I reorganized the matrix putting the var rows and columns to >>> the top and to the left, there wasn't a real reason to to this, but >>> I thought that it could help reading the code in the future. >>> Probably very subjective but still :) then yes if the source doesn't >>> allow var in implicit lambdas a `mask` is applied to the value read >>> from the matrix effectively changing it to a null to avoid duplicate >>> reports. >>>> >>>> * In the LambdaParserTest I see no relevant changes, but still the >>>> patch shows something - is there some trailing space or indent >>>> different at play? >>> >>> the change here was to add a case for which errors are expected when >>> var is used and source is 10 >>>> >>>> Maurizio >>> >>> Vicente >>> >>>> >>>> On 03/10/18 01:40, Vicente Romero wrote: >>>>> Hi, >>>>> >>>>> Please review the fix for [1] at [2]. There were a number of >>>>> places where the compiler was not checking if var syntax for >>>>> implicit lambdas was allowed or not. This implied that the JDK11 >>>>> compiler was accepting code with this feature even if the -source >>>>> 10 option was passed, this shouldn't happen. This patch fixes this >>>>> and issues the: "var not allowed here" error message as expected. >>>>> >>>>> Thanks, >>>>> Vicente >>>>> >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8211148 >>>>> [2] >>>>> http://cr.openjdk.java.net/~vromero/8211148/webrev.00/jdk.dev.patch >>>> >>> >> > From vicente.romero at oracle.com Fri Oct 5 18:18:20 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Fri, 5 Oct 2018 14:18:20 -0400 Subject: RFR: JDK-8209407: VerifyError is thrown for inner class with lambda Message-ID: <31fda4be-46c2-f3ff-7ac5-0096d3ea811a@oracle.com> Please review the fix for [1] at [2]. javac was compiling a code not acceptable to the verifier. This code exemplifies the issue: import java.util.function.Supplier; public class Test { ? public static void main(String[] args) { ??? Object a = new Object(); ??? class Local { ????? Object ref = a; ??? } ??? new Object() { ????? void unused() { ??????? Supplier s = () -> new Local(); ????? } ??? }; ? } } here the local class Local is capturing variable `a`, the thing is that after LTM processes the lambda, it defines a mapping for that free variable. But once the compiler is at Lower, another mapping, a proxy variable, is defined. These two mappings collide inside the translation of the lambda method and the one defined in LTM should trump. This is what this patch is doing. Thanks, Vicente [1] https://bugs.openjdk.java.net/browse/JDK-8209407 [2] http://cr.openjdk.java.net/~vromero/8209407/webrev.00/jdk.dev.patch -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian.goetz at oracle.com Sat Oct 6 21:00:41 2018 From: brian.goetz at oracle.com (Brian Goetz) Date: Sat, 6 Oct 2018 17:00:41 -0400 Subject: RFR: JDK-8210031: implementation for JVM Constants API In-Reply-To: <12D7246E-42DD-4BD4-B3E0-AB2FE015F8C2@oracle.com> References: <12D7246E-42DD-4BD4-B3E0-AB2FE015F8C2@oracle.com> Message-ID: What we decided to do here is to hold back on ?implements Constable? for the symbolic descriptor classes in the initial push of JEP-334, and then when we have the symbolic expression mode for BSMs, re-implement those in XxxDesc using that. Implementing Constable isn?t needed until we get to the full constant folding anyway. That linearizes the dependencies ? first JEP-334, then symbolic mode BSM, then update JEP-334 classes to implement Constable using symbolic mode BSM. > On Sep 13, 2018, at 9:07 PM, John Rose wrote: > > I am running a review of VM-level work on bootstrap methods > which can optionally help simplify some of these APIs: > > http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-September/030084.html > > Specifically, this work can be use to implement a "symbolic > expression mode" for BSMs which causes the JVM to unpack > constant pool nodes directly as ConstantDesc items to present > to BSMs. This might simplify the condy forms of ConstantDesc > instances, if javac stores native constants to reflect, rather than > lists of strings to reassemble. > > ? John > > On Sep 11, 2018, at 12:50 PM, Vicente Romero wrote: >> >> Please review the first iteration of the implementation for JEP-334 [1] JVM Constants API. The implementation can be found at [2]. JEP-334 introduces an API to model nominal descriptions of key class-file and run-time artifacts, in particular constants that are loadable from the constant pool and has already been the subject of several discussions. The implementation of this JEP has been publicly accessible throw the amber repo at [3] in the jep-334 branch. Thanks to all members of the Amber project and specially to Brian for all the hard work on the design and the implementation of this API. Thanks for all the feedback we have received so far, most of it has been integrated in the current implementation. >> >> Thanks, >> Vicente >> >> [1] http://openjdk.java.net/jeps/334 >> [2] http://cr.openjdk.java.net/~vromero/8210031/webrev.00/jdk.dev.patch >> [3] http://hg.openjdk.java.net/amber/amber > From bsrbnd at gmail.com Sat Oct 6 22:05:58 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Sun, 7 Oct 2018 00:05:58 +0200 Subject: Primitive boolean array packing Message-ID: Hi, Per JVMS, primitive boolean arrays are currently using 8 bits (one byte) per boolean value, see [1] (baload/bastore). We've see in some threads [2] that 'BitSet' or 'EnumSet' are occasionally preferred solutions as they are using boolean vectors (aka long values) using 8x less memory. But as the spec [1] allows primitive boolean array packing, I made a quick experiment which is available here: http://cr.openjdk.java.net/~bsrbnd/boolpack/webrev.00/ This is a partial draft implementing primitive boolean array packing (8x smaller) at machine-level (vs BitSet at class-level): * only x86 interpreter implemented => c1/c2/inlining disabled for methods using baload/bastore (not much performance regression when building the jdk and running the tests) * unsafe access partially implemented * array copy currently not implemented * -XX:+/-BooleanPacking to enable/disable the feature currently not implemented A couple of residual tier1 tests are still failing due to the unfinished implementation. I didn't search much if such experiments have already been accomplished, but I'd like to take the temperature of this feature as completing the implementation represents a significant amount of work. Is this something that is worth exploring? Thanks in advance for any feedback. Regards, Bernard [1] https://docs.oracle.com/javase/specs/jvms/se11/html/jvms-6.html#jvms-6.5.baload [2] http://mail.openjdk.java.net/pipermail/compiler-dev/2018-September/012504.html From aph at redhat.com Sun Oct 7 11:47:45 2018 From: aph at redhat.com (Andrew Haley) Date: Sun, 7 Oct 2018 12:47:45 +0100 Subject: Primitive boolean array packing In-Reply-To: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> Message-ID: <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> On 10/07/2018 09:01 AM, Aleksey Shipilev wrote: > On 10/07/2018 12:05 AM, B. Blaser wrote: >> I didn't search much if such experiments have already been >> accomplished, but I'd like to take the temperature of this feature as >> completing the implementation represents a significant amount of work. >> Is this something that is worth exploring? > > The most problematic part, in my mind, is that JMM requires the absence of word tearing for array > element accesses. See here: https://shipilev.net/blog/2014/jmm-pragmatics/#_part_ii_word_tearing > > With densely packed boolean[], you'd need to convert plain stores to locked/CAS-loops to support > this, I think, which raises all sorts of performance questions. Choosing between boolean[] and > BitSet is basically choosing between stricter/relaxed concurrency guarantees vs dense/sparse footprint. But you get a lot of performance conflicts between cores having to share cache lines anyway. Maybe we should do some performance experiments: we wouldn't need to do all of the C2 work, just write a little C++ and asm code and measure what happens under some plausible conditions. We'd have to try both contended and uncontended situations. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From bsrbnd at gmail.com Sun Oct 7 12:38:21 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Sun, 7 Oct 2018 14:38:21 +0200 Subject: Primitive boolean array packing In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> Message-ID: On Sun, 7 Oct 2018 at 13:51, Roman Kennke wrote: > > >>> I didn't search much if such experiments have already been > >>> accomplished, but I'd like to take the temperature of this feature as > >>> completing the implementation represents a significant amount of work. > >>> Is this something that is worth exploring? > >> > >> The most problematic part, in my mind, is that JMM requires the absence of word tearing for array > >> element accesses. See here: https://shipilev.net/blog/2014/jmm-pragmatics/#_part_ii_word_tearing > >> > >> With densely packed boolean[], you'd need to convert plain stores to locked/CAS-loops to support > >> this, I think, which raises all sorts of performance questions. Choosing between boolean[] and > >> BitSet is basically choosing between stricter/relaxed concurrency guarantees vs dense/sparse footprint. > > > > But you get a lot of performance conflicts between cores having to share > > cache lines anyway. Maybe we should do some performance experiments: we > > wouldn't need to do all of the C2 work, just write a little C++ and asm > > code and measure what happens under some plausible conditions. We'd have > > to try both contended and uncontended situations. > > Are boolean arrays even common enough to make a difference? > > Roman Thanks Aleksey, you're absolutely right but a programmer can still disable this feature and use regular boolean arrays if necessary. Nevertheless, if the memory consumption is a priority and boolean packing a necessity, the problem you mentioned has two solutions: 1) either the JVM could lock distinct boolean elements per 8-bit block to preserve synchronized data, in which case some profiling would be necessary as Andrew suggested 2) or the JVM could no more guarantee concurrent access on distinct boolean array elements if packing is enabled delegating the synchronization to the programmer. While the first solution is safer, the second one minimizes the additional performance cost with a better synchronization focusing which maybe addresses Roman's question... Bernard From aph at redhat.com Sun Oct 7 17:51:20 2018 From: aph at redhat.com (Andrew Haley) Date: Sun, 7 Oct 2018 18:51:20 +0100 Subject: Primitive boolean array packing In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> Message-ID: On 10/07/2018 04:13 PM, Aleksey Shipilev wrote: > On 10/07/2018 02:38 PM, B. Blaser wrote: >> Thanks Aleksey, you're absolutely right but a programmer can still >> disable this feature and use regular boolean arrays if necessary. > > I fully expect the JVM feature/flag what optionally breaks the Java > specification will be flat out rejected. It doesn't break the Java specification: Notes The baload instruction is used to load values from both byte and boolean arrays. In Oracle's Java Virtual Machine implementation, boolean arrays - that is, arrays of type T_BOOLEAN (?2.2, ?newarray) - are implemented as arrays of 8-bit values. Other implementations may implement packed boolean arrays; the baload instruction of such implementations must be used to access those arrays. > My educated guess from the subword atomic operations over > VarHandles: the local latency of uncontended CAS would still be too > significant to ignore. Over years, we hoped CAS overhead would > become negligible, but it still isn't Probably so. Numbers would be nice. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From bsrbnd at gmail.com Mon Oct 8 08:30:19 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Mon, 8 Oct 2018 10:30:19 +0200 Subject: Primitive boolean array packing In-Reply-To: <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> Message-ID: On Sun, 7 Oct 2018 at 21:54, Aleksey Shipilev wrote: > > On 10/07/2018 07:51 PM, Andrew Haley wrote: > > On 10/07/2018 04:13 PM, Aleksey Shipilev wrote: > >> On 10/07/2018 02:38 PM, B. Blaser wrote: > >>> Thanks Aleksey, you're absolutely right but a programmer can still > >>> disable this feature and use regular boolean arrays if necessary. > >> > >> I fully expect the JVM feature/flag what optionally breaks the Java > >> specification will be flat out rejected. > > > > It doesn't break the Java specification: > > > > Notes > > > > The baload instruction is used to load values from both byte and > > boolean arrays. In Oracle's Java Virtual Machine implementation, > > boolean arrays - that is, arrays of type T_BOOLEAN (?2.2, > > ?newarray) - are implemented as arrays of 8-bit values. Other > > implementations may implement packed boolean arrays; the baload > > instruction of such implementations must be used to access those > > arrays. > > You need to specify what do you mean by "it". Current prototype sure does break the provisions of > JLS 17.6 "Word Tearing": > https://docs.oracle.com/javase/specs/jls/se11/html/jls-17.html#jls-17.6 > > My comment was about generic thing: we cannot break the Java spec even if we have the JVM flag that > makes it right again. I thought that what B. was suggesting as the escape hatch. Of course, if you > do packed boolean[] with locked/CAS-ed writes or otherwise guarantee the absence of word tearing, it > does not break the spec. > > -Aleksey You're undoubtedly right, I was focusing on the JVMS and I missed that point in the JLS... I'll try to put the prototype in sync with *all* the specs. Thanks, Bernard From bsrbnd at gmail.com Mon Oct 8 08:40:19 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Mon, 8 Oct 2018 10:40:19 +0200 Subject: Primitive boolean array packing In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> Message-ID: On Sun, 7 Oct 2018 at 20:29, Roman Kennke wrote: > > IIRC, some CPUs even have instructions for directly getting/setting > bits, without explicit mask/shift. AArch64 can do that I think. Not sure > if it's more efficient though. > > Roman Sounds great, probably no word-tearing problem and no additional sync would be necessary on this arch. Unfortunately, I don't have the opportunity to work on this hardware, maybe someone could experiment on it? Bernard From aph at redhat.com Mon Oct 8 08:53:17 2018 From: aph at redhat.com (Andrew Haley) Date: Mon, 8 Oct 2018 09:53:17 +0100 Subject: Primitive boolean array packing In-Reply-To: <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> Message-ID: On 10/07/2018 08:54 PM, Aleksey Shipilev wrote: > You need to specify what do you mean by "it". Current prototype sure > does break the provisions of JLS 17.6 "Word Tearing": > https://docs.oracle.com/javase/specs/jls/se11/html/jls-17.html#jls-17.6 Oh, sure, of course. > My comment was about generic thing: we cannot break the Java spec > even if we have the JVM flag that makes it right again. I thought > that what B. was suggesting as the escape hatch. And I thought it was a switch between bit arrays and byte arrays. > Of course, if you do packed boolean[] with locked/CAS-ed writes or > otherwise guarantee the absence of word tearing, it does not break > the spec. Exactly. it's an interesting idea for large bit vectors, and it could well generate good code. However, it probably makes more sense to implement it as a bit vector class with suitable helper intrinsics. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From bsrbnd at gmail.com Mon Oct 8 11:16:28 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Mon, 8 Oct 2018 13:16:28 +0200 Subject: Primitive boolean array packing In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> Message-ID: On Mon, 8 Oct 2018 at 10:53, Andrew Haley wrote: > > On 10/07/2018 08:54 PM, Aleksey Shipilev wrote: > > > Of course, if you do packed boolean[] with locked/CAS-ed writes or > > otherwise guarantee the absence of word tearing, it does not break > > the spec. > > Exactly. it's an interesting idea for large bit vectors, and it could > well generate good code. However, it probably makes more sense to > implement it as a bit vector class with suitable helper intrinsics. The current prototype was intended to scrupulously implement what the JVMS explicitly allows for baload/bastore instructions. But helper intrinsics might be good alternatives too, we could try both. Note that the existing BitSet needs external sync (most probably on the whole set) when used in multi-threaded environments. With packed baload/bastore instructions or dedicated intrinsics, the sync would be on smaller 8-bit blocks or no sync at all on some arch as Roman mentioned (AArch64). Bernard From maurizio.cimadamore at oracle.com Mon Oct 8 13:09:34 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 8 Oct 2018 14:09:34 +0100 Subject: RFR: JDK-8209407: VerifyError is thrown for inner class with lambda In-Reply-To: <31fda4be-46c2-f3ff-7ac5-0096d3ea811a@oracle.com> References: <31fda4be-46c2-f3ff-7ac5-0096d3ea811a@oracle.com> Message-ID: Looks good. Maurizio On 05/10/18 19:18, Vicente Romero wrote: > Please review the fix for [1] at [2]. javac was compiling a code not > acceptable to the verifier. This code exemplifies the issue: > > import java.util.function.Supplier; > > public class Test { > ? public static void main(String[] args) { > ??? Object a = new Object(); > ??? class Local { > ????? Object ref = a; > ??? } > ??? new Object() { > ????? void unused() { > ??????? Supplier s = () -> new Local(); > ????? } > ??? }; > ? } > } > > here the local class Local is capturing variable `a`, the thing is > that after LTM processes the lambda, it defines a mapping for that > free variable. But once the compiler is at Lower, another mapping, a > proxy variable, is defined. These two mappings collide inside the > translation of the lambda method and the one defined in LTM should > trump. This is what this patch is doing. > > Thanks, > Vicente > > [1] https://bugs.openjdk.java.net/browse/JDK-8209407 > [2] http://cr.openjdk.java.net/~vromero/8209407/webrev.00/jdk.dev.patch -------------- next part -------------- An HTML attachment was scrubbed... URL: From aph at redhat.com Mon Oct 8 15:19:26 2018 From: aph at redhat.com (Andrew Haley) Date: Mon, 8 Oct 2018 16:19:26 +0100 Subject: Primitive boolean array packing In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> Message-ID: <882cb210-5870-2292-e78f-62857abdc54f@redhat.com> On 10/08/2018 12:16 PM, B. Blaser wrote: > The current prototype was intended to scrupulously implement what the > JVMS explicitly allows for baload/bastore instructions. > But helper intrinsics might be good alternatives too, we could try both. So I tried it with a rough C++/assembly language test, and it doesn't make much sense to use this for default boolean arrays. The update sequence on AArch64 is 0: ldxrb w2, [x0]; and w2, w2, w1 stxrb w3, w2, [x0] cbnz w3, 0b (combined with a bunch of uninteresting shifts and logical operations.) We have to load a byte in exclusive state, AND or OR it as appropriate, then store it, still in exclusive state. That gets us the atomicity we need. 10**9 random set operations on an 8 Mbit array take 0.550s; with a boolean array the time is 0.150s. This is the local overhead of the load/store exclusive. > Note that the existing BitSet needs external sync (most probably on > the whole set) when used in multi-threaded environments. > With packed baload/bastore instructions or dedicated intrinsics, the > sync would be on smaller 8-bit blocks or no sync at all on some arch > as Roman mentioned (AArch64). It might help. The question is how much we need large bit arrays. They're certainly useful for things like Bloom filters. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From shade at redhat.com Sun Oct 7 08:01:47 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Sun, 7 Oct 2018 10:01:47 +0200 Subject: Primitive boolean array packing In-Reply-To: References: Message-ID: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> On 10/07/2018 12:05 AM, B. Blaser wrote: > I didn't search much if such experiments have already been > accomplished, but I'd like to take the temperature of this feature as > completing the implementation represents a significant amount of work. > Is this something that is worth exploring? The most problematic part, in my mind, is that JMM requires the absence of word tearing for array element accesses. See here: https://shipilev.net/blog/2014/jmm-pragmatics/#_part_ii_word_tearing With densely packed boolean[], you'd need to convert plain stores to locked/CAS-loops to support this, I think, which raises all sorts of performance questions. Choosing between boolean[] and BitSet is basically choosing between stricter/relaxed concurrency guarantees vs dense/sparse footprint. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Sun Oct 7 11:51:38 2018 From: rkennke at redhat.com (Roman Kennke) Date: Sun, 7 Oct 2018 13:51:38 +0200 Subject: Primitive boolean array packing In-Reply-To: <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> Message-ID: >>> I didn't search much if such experiments have already been >>> accomplished, but I'd like to take the temperature of this feature as >>> completing the implementation represents a significant amount of work. >>> Is this something that is worth exploring? >> >> The most problematic part, in my mind, is that JMM requires the absence of word tearing for array >> element accesses. See here: https://shipilev.net/blog/2014/jmm-pragmatics/#_part_ii_word_tearing >> >> With densely packed boolean[], you'd need to convert plain stores to locked/CAS-loops to support >> this, I think, which raises all sorts of performance questions. Choosing between boolean[] and >> BitSet is basically choosing between stricter/relaxed concurrency guarantees vs dense/sparse footprint. > > But you get a lot of performance conflicts between cores having to share > cache lines anyway. Maybe we should do some performance experiments: we > wouldn't need to do all of the C2 work, just write a little C++ and asm > code and measure what happens under some plausible conditions. We'd have > to try both contended and uncontended situations. Are boolean arrays even common enough to make a difference? Roman -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Sun Oct 7 15:13:11 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Sun, 7 Oct 2018 17:13:11 +0200 Subject: Primitive boolean array packing In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> Message-ID: On 10/07/2018 02:38 PM, B. Blaser wrote: > Thanks Aleksey, you're absolutely right but a programmer can still > disable this feature and use regular boolean arrays if necessary. I fully expect the JVM feature/flag what optionally breaks the Java specification will be flat out rejected. You can have the optional feature that makes the behavior tighter than required by spec, but it is a hard stop to make the behaviors weaker than required by spec. So, the work on packed boolean arrays has to demonstrate good performance with along with work tearing guarantees. On 10/07/2018 01:47 PM, Andrew Haley wrote: > But you get a lot of performance conflicts between cores having to share > cache lines anyway. Maybe we should do some performance experiments: we > wouldn't need to do all of the C2 work, just write a little C++ and asm > code and measure what happens under some plausible conditions. We'd have > to try both contended and uncontended situations. My educated guess from the subword atomic operations over VarHandles: the local latency of uncontended CAS would still be too significant to ignore. Over years, we hoped CAS overhead would become negligible, but it still isn't (see Biased Locking as another example that depends on this). -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From rkennke at redhat.com Sun Oct 7 18:29:33 2018 From: rkennke at redhat.com (Roman Kennke) Date: Sun, 7 Oct 2018 20:29:33 +0200 Subject: Primitive boolean array packing In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> Message-ID: IIRC, some CPUs even have instructions for directly getting/setting bits, without explicit mask/shift. AArch64 can do that I think. Not sure if it's more efficient though. Roman > On 10/07/2018 04:13 PM, Aleksey Shipilev wrote: >> On 10/07/2018 02:38 PM, B. Blaser wrote: >>> Thanks Aleksey, you're absolutely right but a programmer can still >>> disable this feature and use regular boolean arrays if necessary. >> >> I fully expect the JVM feature/flag what optionally breaks the Java >> specification will be flat out rejected. > > It doesn't break the Java specification: > > Notes > > The baload instruction is used to load values from both byte and > boolean arrays. In Oracle's Java Virtual Machine implementation, > boolean arrays - that is, arrays of type T_BOOLEAN (?2.2, > ?newarray) - are implemented as arrays of 8-bit values. Other > implementations may implement packed boolean arrays; the baload > instruction of such implementations must be used to access those > arrays. > >> My educated guess from the subword atomic operations over >> VarHandles: the local latency of uncontended CAS would still be too >> significant to ignore. Over years, we hoped CAS overhead would >> become negligible, but it still isn't > > Probably so. Numbers would be nice. > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From shade at redhat.com Sun Oct 7 19:54:00 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Sun, 7 Oct 2018 21:54:00 +0200 Subject: Primitive boolean array packing In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> Message-ID: <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> On 10/07/2018 07:51 PM, Andrew Haley wrote: > On 10/07/2018 04:13 PM, Aleksey Shipilev wrote: >> On 10/07/2018 02:38 PM, B. Blaser wrote: >>> Thanks Aleksey, you're absolutely right but a programmer can still >>> disable this feature and use regular boolean arrays if necessary. >> >> I fully expect the JVM feature/flag what optionally breaks the Java >> specification will be flat out rejected. > > It doesn't break the Java specification: > > Notes > > The baload instruction is used to load values from both byte and > boolean arrays. In Oracle's Java Virtual Machine implementation, > boolean arrays - that is, arrays of type T_BOOLEAN (?2.2, > ?newarray) - are implemented as arrays of 8-bit values. Other > implementations may implement packed boolean arrays; the baload > instruction of such implementations must be used to access those > arrays. You need to specify what do you mean by "it". Current prototype sure does break the provisions of JLS 17.6 "Word Tearing": https://docs.oracle.com/javase/specs/jls/se11/html/jls-17.html#jls-17.6 My comment was about generic thing: we cannot break the Java spec even if we have the JVM flag that makes it right again. I thought that what B. was suggesting as the escape hatch. Of course, if you do packed boolean[] with locked/CAS-ed writes or otherwise guarantee the absence of word tearing, it does not break the spec. -Aleksey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From maurizio.cimadamore at oracle.com Mon Oct 8 16:30:05 2018 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 8 Oct 2018 17:30:05 +0100 Subject: Primitive boolean array packing In-Reply-To: References: Message-ID: Hi Bernard, what you describe here is also overlapping with Project Panama, which allows you to define a Java wrapper for a packed struct using layout descriptions. If you go down that path, you can, I believe, address the memory footprint issues w/o touching the VM (e.g. by changing the implementation of some of these classes to use Panama structs). Could be a worthwhile experiment to try and do that! Cheers Maurizio On 06/10/18 23:05, B. Blaser wrote: > Hi, > > Per JVMS, primitive boolean arrays are currently using 8 bits (one > byte) per boolean value, see [1] (baload/bastore). > > We've see in some threads [2] that 'BitSet' or 'EnumSet' are > occasionally preferred solutions as they are using boolean vectors > (aka long values) using 8x less memory. > > But as the spec [1] allows primitive boolean array packing, I made a > quick experiment which is available here: > > http://cr.openjdk.java.net/~bsrbnd/boolpack/webrev.00/ > > This is a partial draft implementing primitive boolean array packing > (8x smaller) at machine-level (vs BitSet at class-level): > * only x86 interpreter implemented => c1/c2/inlining disabled for > methods using baload/bastore (not much performance regression when > building the jdk and running the tests) > * unsafe access partially implemented > * array copy currently not implemented > * -XX:+/-BooleanPacking to enable/disable the feature currently not implemented > > A couple of residual tier1 tests are still failing due to the > unfinished implementation. > > I didn't search much if such experiments have already been > accomplished, but I'd like to take the temperature of this feature as > completing the implementation represents a significant amount of work. > Is this something that is worth exploring? > > Thanks in advance for any feedback. > > Regards, > Bernard > > > [1] https://docs.oracle.com/javase/specs/jvms/se11/html/jvms-6.html#jvms-6.5.baload > [2] http://mail.openjdk.java.net/pipermail/compiler-dev/2018-September/012504.html From bsrbnd at gmail.com Mon Oct 8 17:18:08 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Mon, 8 Oct 2018 19:18:08 +0200 Subject: Primitive boolean array packing In-Reply-To: <882cb210-5870-2292-e78f-62857abdc54f@redhat.com> References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> <882cb210-5870-2292-e78f-62857abdc54f@redhat.com> Message-ID: On Mon, 8 Oct 2018 at 17:19, Andrew Haley wrote: > > On 10/08/2018 12:16 PM, B. Blaser wrote: > > The current prototype was intended to scrupulously implement what the > > JVMS explicitly allows for baload/bastore instructions. > > But helper intrinsics might be good alternatives too, we could try both. > > So I tried it with a rough C++/assembly language test, and it doesn't > make much sense to use this for default boolean arrays. The update > sequence on AArch64 is > > 0: ldxrb w2, [x0]; > and w2, w2, w1 > stxrb w3, w2, [x0] > cbnz w3, 0b > > (combined with a bunch of uninteresting shifts and logical > operations.) We have to load a byte in exclusive state, AND or OR it > as appropriate, then store it, still in exclusive state. That gets us > the atomicity we need. > > 10**9 random set operations on an 8 Mbit array take 0.550s; with a > boolean array the time is 0.150s. This is the local overhead of the > load/store exclusive. Reducing memory consumption has a price which doesn't seem too expensive, fortunately. > > Note that the existing BitSet needs external sync (most probably on > > the whole set) when used in multi-threaded environments. > > With packed baload/bastore instructions or dedicated intrinsics, the > > sync would be on smaller 8-bit blocks or no sync at all on some arch > > as Roman mentioned (AArch64). > > It might help. The question is how much we need large bit > arrays. They're certainly useful for things like Bloom filters. As I initially mentioned, this would be useful for large bit arrays or for a large *number* of smaller bit arrays. We are planning to refactor javac flags [1] currently using one long value per symbol [2] which potentially means a huge number of bit arrays (or vectors) of more than 64 elements. Bernard [1] http://mail.openjdk.java.net/pipermail/compiler-dev/2018-September/012504.html [2] http://hg.openjdk.java.net/jdk/jdk/file/d8aebcc2d3ac/src/jdk.compiler/share/classes/com/sun/tools/javac/code/Symbol.java#l98 > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From bsrbnd at gmail.com Mon Oct 8 17:25:41 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Mon, 8 Oct 2018 19:25:41 +0200 Subject: Primitive boolean array packing In-Reply-To: References: Message-ID: On Mon, 8 Oct 2018 at 18:30, Maurizio Cimadamore wrote: > > Hi Bernard, > what you describe here is also overlapping with Project Panama, which > allows you to define a Java wrapper for a packed struct using layout > descriptions. If you go down that path, you can, I believe, address the > memory footprint issues w/o touching the VM (e.g. by changing the > implementation of some of these classes to use Panama structs). Could be > a worthwhile experiment to try and do that! > > Cheers > Maurizio Thanks Maurizio, I'll take a look ;-) Cheers, Bernard From bsrbnd at gmail.com Wed Oct 10 10:37:28 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Wed, 10 Oct 2018 12:37:28 +0200 Subject: Primitive boolean array packing In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> <882cb210-5870-2292-e78f-62857abdc54f@redhat.com> Message-ID: On Mon, 8 Oct 2018 at 19:18, B. Blaser wrote: > > On Mon, 8 Oct 2018 at 17:19, Andrew Haley wrote: > > > > On 10/08/2018 12:16 PM, B. Blaser wrote: > > > The current prototype was intended to scrupulously implement what the > > > JVMS explicitly allows for baload/bastore instructions. > > > But helper intrinsics might be good alternatives too, we could try both. > > > > So I tried it with a rough C++/assembly language test, and it doesn't > > make much sense to use this for default boolean arrays. The update > > sequence on AArch64 is > > > > 0: ldxrb w2, [x0]; > > and w2, w2, w1 > > stxrb w3, w2, [x0] > > cbnz w3, 0b > > > > (combined with a bunch of uninteresting shifts and logical > > operations.) We have to load a byte in exclusive state, AND or OR it > > as appropriate, then store it, still in exclusive state. That gets us > > the atomicity we need. > > > > 10**9 random set operations on an 8 Mbit array take 0.550s; with a > > boolean array the time is 0.150s. This is the local overhead of the > > load/store exclusive. > > Reducing memory consumption has a price which doesn't seem too > expensive, fortunately. On x86, I guess we have BTS/BTR (bit set and reset) with the LOCK prefix to assure atomicity. I'll do some experiment and update the prototype accordingly. Bernard From vicente.romero at oracle.com Wed Oct 10 16:30:30 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Wed, 10 Oct 2018 12:30:30 -0400 Subject: RFR: JDK-8210031: implementation for JVM Constants API In-Reply-To: References: <12D7246E-42DD-4BD4-B3E0-AB2FE015F8C2@oracle.com> Message-ID: Hi all, I have updated the webrev [1], this version removes the `implements Constable` from the symbolic descriptor classes. Feedback is mostly appreciated, Thanks, Vicente [1] http://cr.openjdk.java.net/~vromero/8210031/webrev.01/jdk12.dev.patch On 10/06/2018 05:00 PM, Brian Goetz wrote: > What we decided to do here is to hold back on ?implements Constable? for the symbolic descriptor classes in the initial push of JEP-334, and then when we have the symbolic expression mode for BSMs, re-implement those in XxxDesc using that. Implementing Constable isn?t needed until we get to the full constant folding anyway. That linearizes the dependencies ? first JEP-334, then symbolic mode BSM, then update JEP-334 classes to implement Constable using symbolic mode BSM. > >> On Sep 13, 2018, at 9:07 PM, John Rose wrote: >> >> I am running a review of VM-level work on bootstrap methods >> which can optionally help simplify some of these APIs: >> >> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-September/030084.html >> >> Specifically, this work can be use to implement a "symbolic >> expression mode" for BSMs which causes the JVM to unpack >> constant pool nodes directly as ConstantDesc items to present >> to BSMs. This might simplify the condy forms of ConstantDesc >> instances, if javac stores native constants to reflect, rather than >> lists of strings to reassemble. >> >> ? John >> >> On Sep 11, 2018, at 12:50 PM, Vicente Romero wrote: >>> Please review the first iteration of the implementation for JEP-334 [1] JVM Constants API. The implementation can be found at [2]. JEP-334 introduces an API to model nominal descriptions of key class-file and run-time artifacts, in particular constants that are loadable from the constant pool and has already been the subject of several discussions. The implementation of this JEP has been publicly accessible throw the amber repo at [3] in the jep-334 branch. Thanks to all members of the Amber project and specially to Brian for all the hard work on the design and the implementation of this API. Thanks for all the feedback we have received so far, most of it has been integrated in the current implementation. >>> >>> Thanks, >>> Vicente >>> >>> [1] http://openjdk.java.net/jeps/334 >>> [2] http://cr.openjdk.java.net/~vromero/8210031/webrev.00/jdk.dev.patch >>> [3] http://hg.openjdk.java.net/amber/amber From joe.darcy at oracle.com Thu Oct 11 01:18:32 2018 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Wed, 10 Oct 2018 18:18:32 -0700 Subject: Name.contentEquals() performance In-Reply-To: <5BB50F15.6030701@oracle.com> References: <4617a955-de22-f5d3-d9af-ed8e0dc2eb40@oracle.com> <5693ec10-55f3-6f5a-d6bb-d811df1faeaf@oracle.com> <5BB50F15.6030701@oracle.com> Message-ID: <5BBEA4E8.7080609@oracle.com> Is String creation for other types done as a guard against possible concurrent modification? -Joe On 10/3/2018 11:48 AM, Jonathan Gibbons wrote: > It is very depressing that Name.contentEquals actually creates > strings. I would have expected it to iterate over the code points of > the items being compared. > > It is also somewhat depressing that .contentEquals is being used so > much, because the intent of the Name class is to function as an > interned string. > > -- Jon > > On 10/03/2018 06:22 AM, Maurizio Cimadamore wrote: >> >> >> On 03/10/18 14:16, Maurizio Cimadamore wrote: >>> (That said, out of curiosity I tried to instrument the JDK build to >>> see how many names were created (this is a bit hard given that the >>> build internally reuses java compilers using the sjavac server), but >>> name creation seem to peak at around 33K, which would give an >>> overhead of a couple of hundred of kB for an extra field like the >>> one you suggest, which doesn't seem excessive in terms of footprint >>> increase) >> Actually, now that I think more, if you store the toString() result >> in an extra field, while the footprint of the Name class (shallow >> size) won't change much (8 more bytes per instance), we could >> potentially be talking about 33K more (in the JDK build case) strings >> be retained by the javac code - which is probably going to balloon up >> the memory footprint cost significantly. >> >> Maurizio > From vicente.romero at oracle.com Thu Oct 11 18:24:39 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Thu, 11 Oct 2018 14:24:39 -0400 Subject: RFR: JDK-8211004: javac is complaining about non-denotable types and refusing to generate the class file Message-ID: Please review the fix for [1] at [2]. This fix is enforcing the type of the enclosing class of an anonymous inner class to be denotable. Thanks, Vicente [1] https://bugs.openjdk.java.net/browse/JDK-8211004 [2] http://cr.openjdk.java.net/~vromero/8211004/webrev.00/ From vicente.romero at oracle.com Mon Oct 15 17:51:51 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Mon, 15 Oct 2018 13:51:51 -0400 Subject: RFR: JDK-8210031: implementation for JVM Constants API In-Reply-To: References: <12D7246E-42DD-4BD4-B3E0-AB2FE015F8C2@oracle.com> Message-ID: <5a25000d-7bfb-56b3-0816-a96c3e49d956@oracle.com> adding core-libs in the loop On 10/10/2018 12:30 PM, Vicente Romero wrote: > Hi all, > > I have updated the webrev [1], this version removes the `implements > Constable` from the symbolic descriptor classes. Feedback is mostly > appreciated, > > Thanks, > Vicente > > [1] http://cr.openjdk.java.net/~vromero/8210031/webrev.01/jdk12.dev.patch > > On 10/06/2018 05:00 PM, Brian Goetz wrote: >> What we decided to do here is to hold back on ?implements Constable? >> for the symbolic descriptor classes in the initial push of JEP-334, >> and then when we have the symbolic expression mode for BSMs, >> re-implement those in XxxDesc using that.? Implementing Constable >> isn?t needed until we get to the full constant folding anyway.? That >> linearizes the dependencies ? first JEP-334, then symbolic mode BSM, >> then update JEP-334 classes to implement Constable using symbolic >> mode BSM. >> >>> On Sep 13, 2018, at 9:07 PM, John Rose wrote: >>> >>> I am running a review of VM-level work on bootstrap methods >>> which can optionally help simplify some of these APIs: >>> >>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-September/030084.html >>> >>> >>> Specifically, this work can be use to implement a "symbolic >>> expression mode" for BSMs which causes the JVM to unpack >>> constant pool nodes directly as ConstantDesc items to present >>> to BSMs.? This might simplify the condy forms of ConstantDesc >>> instances, if javac stores native constants to reflect, rather than >>> lists of strings to reassemble. >>> >>> ? John >>> >>> On Sep 11, 2018, at 12:50 PM, Vicente Romero >>> wrote: >>>> Please review the first iteration of the implementation for JEP-334 >>>> [1] JVM Constants API. The implementation can be found at [2]. >>>> JEP-334 introduces an API to model nominal descriptions of key >>>> class-file and run-time artifacts, in particular constants that are >>>> loadable from the constant pool and has already been the subject of >>>> several discussions. The implementation of this JEP has been >>>> publicly accessible throw the amber repo at [3] in the jep-334 >>>> branch. Thanks to all members of the Amber project and specially to >>>> Brian for all the hard work on the design and the implementation of >>>> this API. Thanks for all the feedback we have received so far, most >>>> of it has been integrated in the current implementation. >>>> >>>> Thanks, >>>> Vicente >>>> >>>> [1] http://openjdk.java.net/jeps/334 >>>> [2] >>>> http://cr.openjdk.java.net/~vromero/8210031/webrev.00/jdk.dev.patch >>>> [3] http://hg.openjdk.java.net/amber/amber > From vicente.romero at oracle.com Mon Oct 15 18:12:40 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Mon, 15 Oct 2018 14:12:40 -0400 Subject: RFR: JDK-8210031: implementation for JVM Constants API In-Reply-To: <5a25000d-7bfb-56b3-0816-a96c3e49d956@oracle.com> References: <12D7246E-42DD-4BD4-B3E0-AB2FE015F8C2@oracle.com> <5a25000d-7bfb-56b3-0816-a96c3e49d956@oracle.com> Message-ID: <61e6cb70-9356-0b36-2a13-e90a3102f2c1@oracle.com> Hi all, sorry for the repeated number of mails on this issue. I have added a direct link to the patch the right link to the webrev is [1] there is a previous version at [2] if you want to see the differences with the last version. Basically we have dropped the `implements Constable` for the symbolic descriptor classes and removed all of the code that was useful for constant folding but not strictly necessary for the API. For more information on the JEP see [3] [1] http://cr.openjdk.java.net/~vromero/8210031/webrev.01 [2] http://cr.openjdk.java.net/~vromero/8210031/webrev.00 [3] http://openjdk.java.net/jeps/334 On 10/15/2018 01:51 PM, Vicente Romero wrote: > adding core-libs in the loop > > On 10/10/2018 12:30 PM, Vicente Romero wrote: >> Hi all, >> >> I have updated the webrev [1], this version removes the `implements >> Constable` from the symbolic descriptor classes. Feedback is mostly >> appreciated, >> >> Thanks, >> Vicente >> >> [1] >> http://cr.openjdk.java.net/~vromero/8210031/webrev.01/jdk12.dev.patch >> >> On 10/06/2018 05:00 PM, Brian Goetz wrote: >>> What we decided to do here is to hold back on ?implements Constable? >>> for the symbolic descriptor classes in the initial push of JEP-334, >>> and then when we have the symbolic expression mode for BSMs, >>> re-implement those in XxxDesc using that.? Implementing Constable >>> isn?t needed until we get to the full constant folding anyway.? That >>> linearizes the dependencies ? first JEP-334, then symbolic mode BSM, >>> then update JEP-334 classes to implement Constable using symbolic >>> mode BSM. >>> >>>> On Sep 13, 2018, at 9:07 PM, John Rose wrote: >>>> >>>> I am running a review of VM-level work on bootstrap methods >>>> which can optionally help simplify some of these APIs: >>>> >>>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-September/030084.html >>>> >>>> >>>> Specifically, this work can be use to implement a "symbolic >>>> expression mode" for BSMs which causes the JVM to unpack >>>> constant pool nodes directly as ConstantDesc items to present >>>> to BSMs.? This might simplify the condy forms of ConstantDesc >>>> instances, if javac stores native constants to reflect, rather than >>>> lists of strings to reassemble. >>>> >>>> ? John >>>> >>>> On Sep 11, 2018, at 12:50 PM, Vicente Romero >>>> wrote: >>>>> Please review the first iteration of the implementation for >>>>> JEP-334 [1] JVM Constants API. The implementation can be found at >>>>> [2]. JEP-334 introduces an API to model nominal descriptions of >>>>> key class-file and run-time artifacts, in particular constants >>>>> that are loadable from the constant pool and has already been the >>>>> subject of several discussions. The implementation of this JEP has >>>>> been publicly accessible throw the amber repo at [3] in the >>>>> jep-334 branch. Thanks to all members of the Amber project and >>>>> specially to Brian for all the hard work on the design and the >>>>> implementation of this API. Thanks for all the feedback we have >>>>> received so far, most of it has been integrated in the current >>>>> implementation. >>>>> >>>>> Thanks, >>>>> Vicente >>>>> >>>>> [1] http://openjdk.java.net/jeps/334 >>>>> [2] >>>>> http://cr.openjdk.java.net/~vromero/8210031/webrev.00/jdk.dev.patch >>>>> [3] http://hg.openjdk.java.net/amber/amber >> > From vicente.romero at oracle.com Thu Oct 18 21:09:37 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Thu, 18 Oct 2018 17:09:37 -0400 Subject: RFR: JDK-8210031: implementation for JVM Constants API In-Reply-To: <61e6cb70-9356-0b36-2a13-e90a3102f2c1@oracle.com> References: <12D7246E-42DD-4BD4-B3E0-AB2FE015F8C2@oracle.com> <5a25000d-7bfb-56b3-0816-a96c3e49d956@oracle.com> <61e6cb70-9356-0b36-2a13-e90a3102f2c1@oracle.com> Message-ID: Hi all, Please review also the CSR at [1]. Thanks, Vicente [1] https://bugs.openjdk.java.net/browse/JDK-8202031 On 10/15/18 2:12 PM, Vicente Romero wrote: > Hi all, > > sorry for the repeated number of mails on this issue. I have added a > direct link to the patch the right link to the webrev is [1] there is > a previous version at [2] if you want to see the differences with the > last version. Basically we have dropped the `implements Constable` for > the symbolic descriptor classes and removed all of the code that was > useful for constant folding but not strictly necessary for the API. > For more information on the JEP see [3] > > [1] http://cr.openjdk.java.net/~vromero/8210031/webrev.01 > [2] http://cr.openjdk.java.net/~vromero/8210031/webrev.00 > [3] http://openjdk.java.net/jeps/334 > > On 10/15/2018 01:51 PM, Vicente Romero wrote: >> adding core-libs in the loop >> >> On 10/10/2018 12:30 PM, Vicente Romero wrote: >>> Hi all, >>> >>> I have updated the webrev [1], this version removes the `implements >>> Constable` from the symbolic descriptor classes. Feedback is mostly >>> appreciated, >>> >>> Thanks, >>> Vicente >>> >>> [1] >>> http://cr.openjdk.java.net/~vromero/8210031/webrev.01/jdk12.dev.patch >>> >>> On 10/06/2018 05:00 PM, Brian Goetz wrote: >>>> What we decided to do here is to hold back on ?implements >>>> Constable? for the symbolic descriptor classes in the initial push >>>> of JEP-334, and then when we have the symbolic expression mode for >>>> BSMs, re-implement those in XxxDesc using that.? Implementing >>>> Constable isn?t needed until we get to the full constant folding >>>> anyway. That linearizes the dependencies ? first JEP-334, then >>>> symbolic mode BSM, then update JEP-334 classes to implement >>>> Constable using symbolic mode BSM. >>>> >>>>> On Sep 13, 2018, at 9:07 PM, John Rose >>>>> wrote: >>>>> >>>>> I am running a review of VM-level work on bootstrap methods >>>>> which can optionally help simplify some of these APIs: >>>>> >>>>> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-September/030084.html >>>>> >>>>> >>>>> Specifically, this work can be use to implement a "symbolic >>>>> expression mode" for BSMs which causes the JVM to unpack >>>>> constant pool nodes directly as ConstantDesc items to present >>>>> to BSMs.? This might simplify the condy forms of ConstantDesc >>>>> instances, if javac stores native constants to reflect, rather than >>>>> lists of strings to reassemble. >>>>> >>>>> ? John >>>>> >>>>> On Sep 11, 2018, at 12:50 PM, Vicente Romero >>>>> wrote: >>>>>> Please review the first iteration of the implementation for >>>>>> JEP-334 [1] JVM Constants API. The implementation can be found at >>>>>> [2]. JEP-334 introduces an API to model nominal descriptions of >>>>>> key class-file and run-time artifacts, in particular constants >>>>>> that are loadable from the constant pool and has already been the >>>>>> subject of several discussions. The implementation of this JEP >>>>>> has been publicly accessible throw the amber repo at [3] in the >>>>>> jep-334 branch. Thanks to all members of the Amber project and >>>>>> specially to Brian for all the hard work on the design and the >>>>>> implementation of this API. Thanks for all the feedback we have >>>>>> received so far, most of it has been integrated in the current >>>>>> implementation. >>>>>> >>>>>> Thanks, >>>>>> Vicente >>>>>> >>>>>> [1] http://openjdk.java.net/jeps/334 >>>>>> [2] >>>>>> http://cr.openjdk.java.net/~vromero/8210031/webrev.00/jdk.dev.patch >>>>>> [3] http://hg.openjdk.java.net/amber/amber >>> >> > From mandy.chung at oracle.com Fri Oct 19 01:55:41 2018 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 18 Oct 2018 18:55:41 -0700 Subject: RFR: JDK-8210031: implementation for JVM Constants API In-Reply-To: <61e6cb70-9356-0b36-2a13-e90a3102f2c1@oracle.com> References: <12D7246E-42DD-4BD4-B3E0-AB2FE015F8C2@oracle.com> <5a25000d-7bfb-56b3-0816-a96c3e49d956@oracle.com> <61e6cb70-9356-0b36-2a13-e90a3102f2c1@oracle.com> Message-ID: On 10/15/18 11:12 AM, Vicente Romero wrote: > > [1] http://cr.openjdk.java.net/~vromero/8210031/webrev.01 I reviewed java.lang.invoke change in details.? I have skimmed through the new classes. I will look at the new tests next. @since 12 is missing in the new APIs VarHandle.java 1887 public final String toString() { 1888 // @@@ defer to concrete type for additional description 1889 // see https://bugs.openjdk.java.net/browse/JDK-8199149 You may want to take out this comment or L1889 as we can refer back to JDK-8199149. VarHandles.java ?169???????? // @@@ This is a little fragile assuming the base is the class Maybe FieldStaticReadOnly and FieldStaticReadWrite constructor and getStaticFieldFromBaseAndOffset method should take Class refc rather than Object base. FieldStaticReadXXX will do the cast when calling getStaticFieldFromBaseAndOffset. java.base module-info.java ?? It'd be good to keep the exported APIs in alphabetical order. java/lang/invoke/TypeDescriptor.java ?? copyright header is missing Mandy -------------- next part -------------- An HTML attachment was scrubbed... URL: From bsrbnd at gmail.com Fri Oct 19 15:42:59 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Fri, 19 Oct 2018 17:42:59 +0200 Subject: Bit set intrinsic (was: Primitive boolean array packing) In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> Message-ID: On Mon, 8 Oct 2018 at 10:53, Andrew Haley wrote: > > On 10/07/2018 08:54 PM, Aleksey Shipilev wrote: > > > Of course, if you do packed boolean[] with locked/CAS-ed writes or > > otherwise guarantee the absence of word tearing, it does not break > > the spec. > > Exactly. it's an interesting idea for large bit vectors, and it could > well generate good code. However, it probably makes more sense to > implement it as a bit vector class with suitable helper intrinsics. So, I turned the existing prototype into a bit set intrinsic (System.setBit()) as I agree with Andrew that it would make more sense than modifying baload/bastore: http://cr.openjdk.java.net/~bsrbnd/boolpack/webrev.01/ The current x86_64 implementation uses BTS/BTR (bit set and reset) instructions as Roman suggested with the LOCK prefix to assure atomicity and thus avoiding any word-tearing problem that Aleksey mentioned. Note that BTS/BTR are locking 32-bit words but AND/OR (8-bit) could be used instead, also in conjunction with LOCK. I tried example [1] with both synchronized BitSet and dedicated intrinsic on x86_64 (8 cores) which revealed that the intrinsic would be at least 2-3x faster than the synchronized set. Then, to the question of how much do we need this, BitSet usages in the JDK is a beginning of answer: $ find ./src/ -name '*.java' -type f -print | xargs grep -e "BitSet" -e "BitArray" There are also other places like javac flags [2] or EnumSet [3] where the same thing is remade by hand. Finally, note that using the intrinsic inside EnumSet would probably make it 2-3x faster than using a SynchronizedCollection [4] in multi-threaded environments. While I agree with Maurizio that Panama structs might be well suited in some situations requiring a rigid layout like javac flags, I think they are only complementary to bit arrays but not a replacement (see EnumSet, etc...). I've seen that Panama is also about exploring hardware optimizations, so maybe the latest intrinsic prototype could be a good starting point for this (hotspot:tier1 being OK)? In any case, feedbacks about this intrinsic would be welcome. Thanks, Bernard [1] https://docs.oracle.com/javase/specs/jls/se11/html/jls-17.html#jls-17.6 [2] http://mail.openjdk.java.net/pipermail/compiler-dev/2018-September/012504.html [3] http://hg.openjdk.java.net/jdk/jdk/file/cb94f3a51aed/src/java.base/share/classes/java/util/JumboEnumSet.java#l44 [4] http://hg.openjdk.java.net/jdk/jdk/file/cb94f3a51aed/src/java.base/share/classes/java/util/Collections.java#l1998 From vicente.romero at oracle.com Fri Oct 19 17:59:30 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Fri, 19 Oct 2018 13:59:30 -0400 Subject: RFR: JDK-8210031: implementation for JVM Constants API In-Reply-To: References: <12D7246E-42DD-4BD4-B3E0-AB2FE015F8C2@oracle.com> <5a25000d-7bfb-56b3-0816-a96c3e49d956@oracle.com> <61e6cb70-9356-0b36-2a13-e90a3102f2c1@oracle.com> Message-ID: <8e1aa029-c9f1-a686-710a-82bbfb1ee5d6@oracle.com> Hi all, Thanks Mandy for the review. I have uploaded a new iteration [1]. Vicente [1] http://cr.openjdk.java.net/~vromero/8210031/webrev.02/ On 10/18/18 9:55 PM, Mandy Chung wrote: > > > On 10/15/18 11:12 AM, Vicente Romero wrote: >> >> [1] http://cr.openjdk.java.net/~vromero/8210031/webrev.01 > > I reviewed java.lang.invoke change in details.? I have skimmed through > the new classes. > I will look at the new tests next. > > @since 12 is missing in the new APIs > > VarHandle.java > 1887 public final String toString() { > 1888 // @@@ defer to concrete type for additional description > 1889 // see https://bugs.openjdk.java.net/browse/JDK-8199149 You may > want to take out this comment or L1889 as we can refer back to > JDK-8199149. VarHandles.java > ?169???????? // @@@ This is a little fragile assuming the base is the > class > > Maybe FieldStaticReadOnly and FieldStaticReadWrite constructor and > getStaticFieldFromBaseAndOffset method should take Class refc > rather than Object base. FieldStaticReadXXX will do the cast when > calling getStaticFieldFromBaseAndOffset. > > java.base module-info.java > ?? It'd be good to keep the exported APIs in alphabetical order. > > java/lang/invoke/TypeDescriptor.java > ?? copyright header is missing > > Mandy -------------- next part -------------- An HTML attachment was scrubbed... URL: From joe.darcy at oracle.com Sat Oct 20 21:42:05 2018 From: joe.darcy at oracle.com (joe darcy) Date: Sat, 20 Oct 2018 14:42:05 -0700 Subject: JDK 12 RFR of JDK-8212718: Refactor some annotation processor tests to better use collections Message-ID: Hello, Please review a fix for ??? JDK-8212718: Refactor some annotation processor tests to better use collections ??? http://cr.openjdk.java.net/~darcy/8212718.0/ Patch below. Thanks, -Joe --- old/test/langtools/tools/javac/processing/model/element/TestAnonClassNames.java 2018-10-20 14:37:36.166001000 -0700 +++ new/test/langtools/tools/javac/processing/model/element/TestAnonClassNames.java 2018-10-20 14:37:35.990001000 -0700 @@ -1,5 +1,5 @@ ?/* - * Copyright (c) 2010, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2010, 2018, Oracle and/or its affiliates. All rights reserved. ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. ? * ? * This code is free software; you can redistribute it and/or modify it @@ -129,22 +129,17 @@ ???? static void testClassNames(List classNames) { ???????? System.out.println("test: " + classNames); -??????? List options = new ArrayList(); -??????? options.add("-proc:only"); -??????? options.add("-classpath"); -??????? options.add(System.getProperty("test.classes")); - ???????? JavaCompiler javaCompiler = ToolProvider.getSystemJavaCompiler(); ???????? JavaCompiler.CompilationTask compileTask = ???????????? javaCompiler.getTask(null, // Output ????????????????????????????????? null, // File manager ????????????????????????????????? null, // Diagnostics -???????????????????????????????? options, +???????????????????????????????? List.of("-proc:only", // options +??? ??? ??? ??? ??? ?"-classpath", +??? ??? ??? ??? ??? ?System.getProperty("test.classes")), ????????????????????????????????? classNames, ????????????????????????????????? null); // Sources -??????? List processors = new ArrayList(); -??????? processors.add(new ClassNameProber()); -??????? compileTask.setProcessors(processors); +??????? compileTask.setProcessors(List.of(new ClassNameProber())); ???????? Boolean goodResult = compileTask.call(); ???????? if (!goodResult) { ???????????? error("Errors found during compile."); --- old/test/langtools/tools/javac/processing/model/element/TypeParamBounds.java 2018-10-20 14:37:36.574001000 -0700 +++ new/test/langtools/tools/javac/processing/model/element/TypeParamBounds.java 2018-10-20 14:37:36.402001000 -0700 @@ -1,5 +1,5 @@ ?/* - * Copyright (c) 2006, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2006, 2018, Oracle and/or its affiliates. All rights reserved. ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. ? * ? * This code is free software; you can redistribute it and/or modify it @@ -84,16 +84,12 @@ ???????? // The names of the bounds of each type parameter of Gen. ???????? static Map boundNames = -??????????????? new HashMap(); - -??????? static { -??????????? boundNames.put("T", new String[] {"Object"}); -??????????? boundNames.put("U", new String[] {"Object"}); -??????????? boundNames.put("V", new String[] {"Number"}); -??????????? boundNames.put("W", new String[] {"U"}); -??????????? boundNames.put("X", new String[] {"Runnable"}); -??????????? boundNames.put("Y", new String[] {"CharSequence", "Runnable"}); -??????????? boundNames.put("Z", new String[] {"Object", "Runnable"}); -??????? } +??? ??? Map.of("T", new String[] {"Object"}, +??? ??? ?? "U", new String[] {"Object"}, +??? ??? ?? "V", new String[] {"Number"}, +??? ??? ?? "W", new String[] {"U"}, +??? ??? ?? "X", new String[] {"Runnable"}, +??? ??? ?? "Y", new String[] {"CharSequence", "Runnable"}, +??? ??? ?? "Z", new String[] {"Object", "Runnable"}); ???? } ?} From aph at redhat.com Sun Oct 21 09:38:09 2018 From: aph at redhat.com (Andrew Haley) Date: Sun, 21 Oct 2018 10:38:09 +0100 Subject: Bit set intrinsic In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> Message-ID: On 10/19/2018 04:42 PM, B. Blaser wrote: > I tried example [1] with both synchronized BitSet and dedicated > intrinsic on x86_64 (8 cores) which revealed that the intrinsic would > be at least 2-3x faster than the synchronized set. Which makes it very useful for things like highly-concurrent Bloom Filters. Here's an example of current usage: https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/utils/BloomFilter.java https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/util/Memory.java If we can provide a really fast on/off-heap bitset in standard Java the need for this Unsafe hackery will go away. It's interesting that the Cassandra Bloom Filter is not thread safe; I don't know why this is. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From forax at univ-mlv.fr Wed Oct 24 10:05:38 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 24 Oct 2018 12:05:38 +0200 (CEST) Subject: Intersection type and method ref bug ? Message-ID: <1315346770.317146.1540375538284.JavaMail.zimbra@u-pem.fr> This bug was discovered by Francois Green when testing records, but the bug is independent of the record, method reference and intersection type doesn't mix well. public class RecordBadType { interface I {} static abstract class C { } static class A extends C implements I { } static class B extends C implements I { } static String f(I i) { return null; } public static void main(String[] args) { Stream.of(new A(), new B()) .map(RecordBadType::f) // here the compiler should generate a bridge method no ? .forEach(System.out::println); } } this code compiles but you get a LambdaConversionException at runtime. R?mi From cushon at google.com Thu Oct 25 00:02:25 2018 From: cushon at google.com (Liam Miller-Cushon) Date: Wed, 24 Oct 2018 17:02:25 -0700 Subject: RFR: JDK-8198945: Invalid RuntimeVisibleTypeAnnotations for annotation on anonymous class type parameter In-Reply-To: References: <5B36C325.7080605@oracle.com> Message-ID: The fix has been pushed as: http://hg.openjdk.java.net/jdk/jdk/rev/e11a53698d57 On Mon, Sep 17, 2018 at 11:25 AM Liam Miller-Cushon wrote: > On Mon, Sep 17, 2018 at 11:00 AM Martin Buchholz > wrote: > >> I'm not a compiler engineer, but I am a Reviewer, and this change >> looks good to me. >> > > Thanks for the review! > > I made the suggested improvements to the test. Here's the updated webrev: > http://cr.openjdk.java.net/~cushon/8198945/webrev.06/ > > Does anyone have additional feedback? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vicente.romero at oracle.com Fri Oct 26 13:21:02 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Fri, 26 Oct 2018 09:21:02 -0400 Subject: Intersection type and method ref bug ? In-Reply-To: <1315346770.317146.1540375538284.JavaMail.zimbra@u-pem.fr> References: <1315346770.317146.1540375538284.JavaMail.zimbra@u-pem.fr> Message-ID: Hi Remi, Francois, Thanks for reporting this. I have created [1] to track this issue, Vicente [1] https://bugs.openjdk.java.net/browse/JDK-8213032 On 10/24/18 6:05 AM, Remi Forax wrote: > This bug was discovered by Francois Green when testing records, but the bug is independent of the record, > method reference and intersection type doesn't mix well. > > public class RecordBadType { > interface I {} > static abstract class C { } > static class A extends C implements I { } > static class B extends C implements I { } > > static String f(I i) { return null; } > > public static void main(String[] args) { > Stream.of(new A(), new B()) > .map(RecordBadType::f) // here the compiler should generate a bridge method no ? > .forEach(System.out::println); > } > } > > this code compiles but you get a LambdaConversionException at runtime. > > R?mi From jan.lahoda at oracle.com Wed Oct 31 10:50:19 2018 From: jan.lahoda at oracle.com (Jan Lahoda) Date: Wed, 31 Oct 2018 11:50:19 +0100 Subject: RFR: JDK-8213103: RoundEnvironment.getElementsAnnotatedWith(Class) crashes with -source 8 Message-ID: <5BD988EB.8020304@oracle.com> Hi, When RoundEnvironment.getElementsAnnotatedWith(Class) converts the Class to TypeElement, it uses: eltUtils.getTypeElement(eltUtils.getModuleElement(), ) But this fails with -source 8, as there are no module (so getModuleElement returns null, and getTypeElement checks the module parameter is non-null). The proposed fix is to avoid this code. Bug: https://bugs.openjdk.java.net/browse/JDK-8213103 Webrev: http://cr.openjdk.java.net/~jlahoda/8213103/webrev.00/index.html Any feedback is welcome, Jan From bsrbnd at gmail.com Wed Oct 31 14:51:34 2018 From: bsrbnd at gmail.com (B. Blaser) Date: Wed, 31 Oct 2018 15:51:34 +0100 Subject: Bit set intrinsic In-Reply-To: References: <9891ee55-7114-5b71-7a90-0c4e97deda2a@redhat.com> <05e00e85-046a-cdcb-7323-9e40f8a8fc86@redhat.com> <62d7984a-78dc-82a4-15aa-6deeebf95342@redhat.com> Message-ID: On Sun, 21 Oct 2018 at 11:38, Andrew Haley wrote: > > On 10/19/2018 04:42 PM, B. Blaser wrote: > > I tried example [1] with both synchronized BitSet and dedicated > > intrinsic on x86_64 (8 cores) which revealed that the intrinsic would > > be at least 2-3x faster than the synchronized set. > > Which makes it very useful for things like highly-concurrent Bloom > Filters. Here's an example of current usage: > > https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/utils/BloomFilter.java > > https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/util/Memory.java > > If we can provide a really fast on/off-heap bitset in standard Java > the need for this Unsafe hackery will go away. > > It's interesting that the Cassandra Bloom Filter is not thread safe; I > don't know why this is. The last but not least, I implemented the c2 part (using the 8-bit AND/OR variant) to do sharper comparisons also on non-concurrent execution: http://cr.openjdk.java.net/~bsrbnd/boolpack/webrev.02/ With 10e6 iterations the lock latency seems to be more or less negligible and removing it would make the intrinsic about 10% faster than BitSet without synchronization. I used it naively inside RegularEnumSet without optimization nor atomicity guarantee only to evaluate its robustness and tier1 was successful. I'm not sure what would be the process for adding something like this to the JDK (maybe a JEP would be necessary?) but I cannot implement the intrinsic on other architectures myself. Also, I guess a more precise API specification along with advanced profiling would be necessary but I'm probably not an expert in these areas... In any case, I hope these experiments will be useful for something. Please let me know of any feedback, Bernard From jan.lahoda at oracle.com Wed Oct 31 14:59:17 2018 From: jan.lahoda at oracle.com (Jan Lahoda) Date: Wed, 31 Oct 2018 15:59:17 +0100 Subject: RFR: JDK-8209055: c.s.t.javac.code.DeferredCompletionFailureHandler seems to use WeakHashMap incorrectly Message-ID: <5BD9C345.7090106@oracle.com> Hi, After JDK-8187950, when a Symbol is completed in "user"/external mode (as opposed to "javac internal" mode), and the completion fails, the failed Symbol is "split", so that the user's code and javac may see a different version of the Symbol (e.g. completed/not completed). This helps to prevent throwing CompletionFailures to user's code. But the issue is that ClassFinder.loadClass may create "speculative" ClassSymbols, which are subsequently removed if they turn out to be non-existent. But when this happens in the "user" mode (like inside Elements.getTypeElement), even these non-existent ClassSymbols are still retained, and the number of ClassSymbols retained this way may be fairly high. The proposed patch is to clear these speculative Symbols even from the "user" mode cache. Bug: https://bugs.openjdk.java.net/browse/JDK-8209055 Webrev: http://cr.openjdk.java.net/~jlahoda/8209055/webrev.00/ Any feedback is welcome. Thanks, Jan From jonathan.gibbons at oracle.com Wed Oct 31 15:01:31 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Wed, 31 Oct 2018 08:01:31 -0700 Subject: RFR: JDK-8213103: RoundEnvironment.getElementsAnnotatedWith(Class) crashes with -source 8 In-Reply-To: <5BD988EB.8020304@oracle.com> References: <5BD988EB.8020304@oracle.com> Message-ID: OK -- Jon On 10/31/18 3:50 AM, Jan Lahoda wrote: > Hi, > > When RoundEnvironment.getElementsAnnotatedWith(Class) converts the > Class to TypeElement, it uses: > eltUtils.getTypeElement(eltUtils.getModuleElement(), > ) > > But this fails with -source 8, as there are no module (so > getModuleElement returns null, and getTypeElement checks the module > parameter is non-null). The proposed fix is to avoid this code. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8213103 > Webrev: http://cr.openjdk.java.net/~jlahoda/8213103/webrev.00/index.html > > Any feedback is welcome, > ??? Jan From joe.darcy at oracle.com Wed Oct 31 17:27:05 2018 From: joe.darcy at oracle.com (joe darcy) Date: Wed, 31 Oct 2018 10:27:05 -0700 Subject: RFR: JDK-8213103: RoundEnvironment.getElementsAnnotatedWith(Class) crashes with -source 8 In-Reply-To: <5BD988EB.8020304@oracle.com> References: <5BD988EB.8020304@oracle.com> Message-ID: Hi Jan, On 10/31/2018 3:50 AM, Jan Lahoda wrote: > Hi, > > When RoundEnvironment.getElementsAnnotatedWith(Class) converts the > Class to TypeElement, it uses: > eltUtils.getTypeElement(eltUtils.getModuleElement(), > ) > > But this fails with -source 8, as there are no module (so > getModuleElement returns null, and getTypeElement checks the module > parameter is non-null). The proposed fix is to avoid this code. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8213103 > Webrev: http://cr.openjdk.java.net/~jlahoda/8213103/webrev.00/index.html > > Any feedback is welcome, > Thanks for looking at this issue. Stepping back a bit, the API functionality being supported is ??? RoundEnvironment.getElementsAnnotatedWith?(Class a) and related methods which take in annotation types indicated using Class objects (a runtime core reflection representation) and convert them internally to TypeElement's (a compile-time representation) by extracting the name of the annotation type and then doing various look-ups. If the compile-time and runtime environments and configured to be consistent with each other, then a runtime annotation type should be able to get back a corresponding compile-time representation. However, it is possible for the environments to not be consistent, leading to this bug and the earlier change 8190886: "package-info handling in RoundEnvironment.getElementsAnnotatedWith" which this proposed fix amends. The proposed "return null" arm in @@ -285,13 +285,15 @@ ???????? // differ from the single module being compiled. ???????? String name = annotation.getCanonicalName(); ???????? TypeElement annotationElement = eltUtils.getTypeElement(name); ???????? if (annotationElement != null) ???????????? return annotationElement; -??????? else { +??????? else if (!eltUtils.getAllModuleElements().isEmpty()) { ???????????? String moduleName = Objects.requireNonNullElse(annotation.getModule().getName(), ""); ???????????? return eltUtils.getTypeElement(eltUtils.getModuleElement(moduleName), name); +??????? } else { +??????????? return null; ???????? } ???? } ???? private Element mirrorAsElement(AnnotationMirror annotationMirror) { ???????? return annotationMirror.getAnnotationType().asElement(); is basically the handling for an inconsistent environment pre-modules. The preceding lines are an, in retrospect, incomplete handling of an inconsistent state with modules. I recommend the check for "if modules are supported" be implemented using a Source-based predicate so that any future dead code will get removed once modules are in all supported source levels. In that route is not chosen, is getAllModuleElements() a good-performing predicate for this purpose?? Would something like "if (getModuleElement?("java.base") != null )" be better? If the annotation type indicated by the Class object *cannot* be turned into an compile-time annotation type, arguably that is an erroneous situation that should cause an exception. The IllegalArgumentException clauses of RoundEnvironment.getElementsAnnotatedWith?(Class a) RoundEnvironment.getElementsAnnotatedWithAny?(Set> annotations) could be expanded to cover this situation and, in the implementation, return null replaced by throwing an informative exception. What do you think? Cheers, -Joe From nlisker at gmail.com Wed Oct 31 21:42:01 2018 From: nlisker at gmail.com (Nir Lisker) Date: Wed, 31 Oct 2018 23:42:01 +0200 Subject: Reachability and null analysis of assignment inside a try block with a throw inside the catch block Message-ID: Hi, I hope this is the correct list for this question. It is the result of a short discussion with the Eclipse compiler maintainers [1]. Given the following code: A getA() { A a = null; try { a = new A(); } catch (ExceptionThrownByConstructor e) { throw e; } if (a != null) { return a; } else { ... } } If I understood the JLS correctly, the try-catch statement will either assign a non-null value to 'a' and continue, or will not continue (either rethrowing the checked exception or an unchecked exception occurs). If so, by the time the if condition is evaluated, 'a' is surely not null, so the else block should be unreachable, but it is not marked as such. Furthermore, removing the else block gives a compiler error that the method must return an object of type A, but this condition is met. I understand that the whole if-else statement can be removed and I can just 'return a', but that's outside of my point. Am I wrong? Shouldn't the if condition be treated as a constant expression ('true' in this case)? Thanks, Nir [1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=540532 -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.buckley at oracle.com Wed Oct 31 22:41:44 2018 From: alex.buckley at oracle.com (Alex Buckley) Date: Wed, 31 Oct 2018 15:41:44 -0700 Subject: Reachability and null analysis of assignment inside a try block with a throw inside the catch block In-Reply-To: References: Message-ID: <5BDA2FA8.3010600@oracle.com> On 10/31/2018 2:42 PM, Nir Lisker wrote: > Given the following code: > > A getA() { > A a = null; > try { > a = new A(); > } catch (ExceptionThrownByConstructor e) { > throw e; > } > if (a != null) { > return a; > } else { > ... > } > } > > If I understood the JLS correctly, the try-catch statement will either > assign a non-null value to 'a' and continue, or will not continue > (either rethrowing the checked exception or an unchecked exception > occurs). If so, by the time the if condition is evaluated, 'a' is surely > not null, so the else block should be unreachable, but it is not marked > as such. Furthermore, removing the else block gives a compiler error > that the method must return an object of type A, but this condition is met. The JLS doesn't analyze the nullness of variables. The JLS does analyze the reachability of statements. For your program, the rules in 14.21 work as follows: the `try` statement can complete normally, so the succeeding `if` statement is reachable, so the `else` block is reachable. Notwithstanding the fact that `a != null` is NOT a constant expression, there is a lengthy explanation in JLS 14.21 of why the `else` block is considered reachable. If there is no `else` block, then the `if` statement can complete normally. That violates the rule in 8.4.7 that "If a method is declared to have a return type (?8.4.5), then a compile-time error occurs if the body of the method can complete normally (?14.1)." Alex