From fw at deneb.enyo.de Sat Aug 1 09:23:03 2020 From: fw at deneb.enyo.de (Florian Weimer) Date: Sat, 01 Aug 2020 11:23:03 +0200 Subject: SoftCleanable and WeakCleanable Message-ID: <87lfiywwi0.fsf@mid.deneb.enyo.de> Are jdk.internal.ref.SoftCleanable and jdk.internal.ref.WeakCleanable actually used? CleanerTest rests them, but I don't see any other mentions of these classes. From Alan.Bateman at oracle.com Sun Aug 2 06:18:55 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 2 Aug 2020 07:18:55 +0100 Subject: SoftCleanable and WeakCleanable In-Reply-To: <87lfiywwi0.fsf@mid.deneb.enyo.de> References: <87lfiywwi0.fsf@mid.deneb.enyo.de> Message-ID: On 01/08/2020 10:23, Florian Weimer wrote: > Are jdk.internal.ref.SoftCleanable and jdk.internal.ref.WeakCleanable > actually used? > > CleanerTest rests them, but I don't see any other mentions of these > classes. Do you mean used outside of the cleaner implementation? Maybe you are looking to change them to non-public? -Alan From fw at deneb.enyo.de Sun Aug 2 07:11:27 2020 From: fw at deneb.enyo.de (Florian Weimer) Date: Sun, 02 Aug 2020 09:11:27 +0200 Subject: SoftCleanable and WeakCleanable In-Reply-To: (Alan Bateman's message of "Sun, 2 Aug 2020 07:18:55 +0100") References: <87lfiywwi0.fsf@mid.deneb.enyo.de> Message-ID: <87k0yhv7xc.fsf@mid.deneb.enyo.de> * Alan Bateman: > On 01/08/2020 10:23, Florian Weimer wrote: >> Are jdk.internal.ref.SoftCleanable and jdk.internal.ref.WeakCleanable >> actually used? >> >> CleanerTest rests them, but I don't see any other mentions of these >> classes. > Do you mean used outside of the cleaner implementation? Maybe you are > looking to change them to non-public? It's not clear to me how the cleaner implementation itself uses them. java.lang.ref.Cleaner only needs PhantomCleanableRef. From felix.yang at huawei.com Mon Aug 3 01:52:00 2020 From: felix.yang at huawei.com (Yangfei (Felix)) Date: Mon, 3 Aug 2020 01:52:00 +0000 Subject: [PING?] RFR(S): 8248910: NPE when freeing the memory for a slice from a buffer In-Reply-To: <765da797-37a8-16cc-71b5-81a9f85e35af@oracle.com> References: <765da797-37a8-16cc-71b5-81a9f85e35af@oracle.com> Message-ID: Hi Alan, > -----Original Message----- > From: Alan Bateman [mailto:Alan.Bateman at oracle.com] > Sent: Friday, July 31, 2020 4:35 PM > To: Yangfei (Felix) ; core-libs- > dev at openjdk.java.net > Subject: Re: [PING?] RFR(S): 8248910: NPE when freeing the memory for a > slice from a buffer > > On 31/07/2020 02:45, Yangfei (Felix) wrote: > > Ping... Any suggestions? > > > Can you bring this to nio-dev for discussion? I don't know what "Randoop" is > but it seems to be using sun.nio.ch.Util directly. I think I'd like to understand > if there is a case where we would actually attempt to free a slice when using > the exported buffer APIs. Thanks for the suggestion. I posted a new mail here on nio-dev: https://mail.openjdk.java.net/pipermail/nio-dev/2020-August/007419.html Felix From david.holmes at oracle.com Mon Aug 3 02:22:59 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 3 Aug 2020 12:22:59 +1000 Subject: JDK 16 RFR of JDK-8250660: Clarify that WildcardType and AnnotatedWildcardType bounds methods return one In-Reply-To: <9808e420-39a8-2045-9db1-629a551f82a0@oracle.com> References: <9808e420-39a8-2045-9db1-629a551f82a0@oracle.com> Message-ID: <63af16f1-ddd3-7e56-5392-891e6328fc25@oracle.com> Hi Joe, I was under the impression that using phrases like "the current version of" were superfluous as all such documentation implicitly refers to the current version, unless explicitly stated otherwise. Personally I'm not sure this clarification really makes a difference to anything as the main source of confusion with this API is why it states "bounds" rather than "bound" and thus returns an array. By adding this clarification you will require a future update if the language allows multiple bounds, forcing the docs to carry around historical data "prior to Java N only a single bound were allowed but since then multiple bounds ...". This seems unnecessary clutter to me. Given the ship has sailed on the shape of this API the developer must simply embrace it and always process the return value in a loop without assuming any knowledge of how many bounds may actually exist. I think the reference to the JLS section suffices to instruct the developer on where to find out how many bounds to expect, whatever that may be in their current version of the JLS. If we think we will never support multiple bounds then these methods should be deprecated (for removal) and replaced by a singular counterpart that returns a single Type rather than a Type[]. Cheers, David ----- On 1/08/2020 8:12 am, Joe Darcy wrote: > Hello, > > Please review the doc clarification below to address > > ??? 8250660: Clarify that WildcardType and AnnotatedWildcardType bounds > methods return one > > Thanks, > > -Joe > > diff -r b0553ba43ba1 > src/java.base/share/classes/java/lang/reflect/AnnotatedWildcardType.java > --- > a/src/java.base/share/classes/java/lang/reflect/AnnotatedWildcardType.java > Fri Jul 31 11:35:25 2020 -0700 > +++ > b/src/java.base/share/classes/java/lang/reflect/AnnotatedWildcardType.java > Fri Jul 31 15:12:14 2020 -0700 > @@ -1,5 +1,5 @@ > ?/* > - * Copyright (c) 2012, 2015, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 2012, 2020, Oracle and/or its affiliates. All rights > reserved. > ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > ? * > ? * This code is free software; you can redistribute it and/or modify it > @@ -30,6 +30,7 @@ > ? * wildcard type argument, whose upper or lower bounds may themselves > represent > ? * annotated uses of types. > ? * > + * @jls 4.5.1 Type Arguments of Parameterized Types > ? * @since 1.8 > ? */ > ?public interface AnnotatedWildcardType extends AnnotatedType { > @@ -39,6 +40,9 @@ > ????? * If no lower bound is explicitly declared, the lower bound is the > ????? * type of null. In this case, a zero length array is returned. > ????? * > +???? * @apiNote In the current version of the Java Programming > +???? * Language, a wildcard may have at most one lower bound. > +???? * > ????? * @return the potentially annotated lower bounds of this wildcard > type or > ????? * an empty array if no lower bound is explicitly declared. > ????? * @see WildcardType#getLowerBounds() > @@ -50,6 +54,9 @@ > ????? * If no upper bound is explicitly declared, the upper bound is > ????? * unannotated {@code Object} > ????? * > +???? * @apiNote In the current version of the Java Programming > +???? * Language, a wildcard may have at most one upper bound. > +???? * > ????? * @return the potentially annotated upper bounds of this wildcard > type > ????? * @see WildcardType#getUpperBounds() > ????? */ > diff -r b0553ba43ba1 > src/java.base/share/classes/java/lang/reflect/WildcardType.java > --- a/src/java.base/share/classes/java/lang/reflect/WildcardType.java > Fri Jul 31 11:35:25 2020 -0700 > +++ b/src/java.base/share/classes/java/lang/reflect/WildcardType.java > Fri Jul 31 15:12:14 2020 -0700 > @@ -1,5 +1,5 @@ > ?/* > - * Copyright (c) 2003, 2015, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 2003, 2020, Oracle and/or its affiliates. All rights > reserved. > ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > ? * > ? * This code is free software; you can redistribute it and/or modify it > @@ -29,6 +29,7 @@ > ? * WildcardType represents a wildcard type expression, such as > ? * {@code ?}, {@code ? extends Number}, or {@code ? super Integer}. > ? * > + * @jls 4.5.1 Type Arguments of Parameterized Types > ? * @since 1.5 > ? */ > ?public interface WildcardType extends Type { > @@ -45,6 +46,9 @@ > ????? *?
  • Otherwise, B is resolved. > ????? * > ????? * > +???? * @apiNote In the current version of the Java Programming > +???? * Language, a wildcard may have at most one upper bound. > +???? * > ????? * @return an array of Types representing the upper bound(s) of this > ????? *???? type variable > ????? * @throws TypeNotPresentException if any of the > @@ -69,6 +73,9 @@ > ????? *??
  • Otherwise, B is resolved. > ????? * > ????? * > +???? * @apiNote In the current version of the Java Programming > +???? * Language, a wildcard may have at most one lower bound. > +???? * > ????? * @return an array of Types representing the lower bound(s) of this > ????? *???? type variable > ????? * @throws TypeNotPresentException if any of the > @@ -78,6 +85,4 @@ > ????? *???? for any reason > ????? */ > ???? Type[] getLowerBounds(); > -??? // one or many? Up to language spec; currently only one, but this API > -??? // allows for generalization. > ?} > From paul.sandoz at oracle.com Mon Aug 3 15:20:42 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 3 Aug 2020 08:20:42 -0700 Subject: RFR: JDK-8244288 Specialized implementations for putIfAbsent, merge, compute* methods in TreeMap derived maps In-Reply-To: References: Message-ID: <8D5FEA90-0A21-4F13-A62A-C4AF2CC7D112@oracle.com> Hi Tagir, The implementations are simple, but as you noted there is some subtle behavior in reporting the key-out-of-range exception. Changes to the tests look good. The putIfAbsent and merge implementations result in a change of behavior. For putIfAbsent, a key out-of-range now results in an exception rather than returning null. For merge, if the new value is null, then remove is called which also returns null for a key out-of-range. I am inclined to also change the behavior of other modification methods, compute and computeIfAbsent, to always throw if the key is out-of-range, rather than check the result of the mapping function. i.e. an out-of-range key is never passed to the mapping function. That?s a wider change in behavior but I suspect one that has minimal impact. (In hindsight if we were implementing these methods when we added the defaults in 8 I would like to think this is the behavior we, or at least I :-), would of implemented.) Paul. > On Jul 26, 2020, at 9:04 AM, Tagir Valeev wrote: > > Hello! > > A gentle ping: please review > https://bugs.openjdk.java.net/browse/JDK-8244288 > http://cr.openjdk.java.net/~tvaleev/webrev/8244288/r1/ > The details are listed below. > > With best regards, > Tagir Valeev. > > On Sun, May 3, 2020 at 4:36 PM Tagir Valeev wrote: >> >> Hello! >> >> Please review the following change: >> https://bugs.openjdk.java.net/browse/JDK-8244288 >> http://cr.openjdk.java.net/~tvaleev/webrev/8244288/r1/ >> >> We already implemented putIfAbsent, merge, computeIfAbsent, >> computeIfPresent, and compute for TreeMap class (see JDK-8176894). >> However, default implementations of these methods are still used for >> TreeMap derived maps, such as descendingMap(), tailMap(), headMap() >> and subMap(). The implementation is pretty straightforward, just a >> range-check+delegation for each of these methods inside the >> TreeMap.NavigableSubMap (+32 lines in TreeMap.java). Care should be >> taken to avoid throwing IAE from compute* methods if the key is >> outside of the range but we don't actually add the entry. This mimics >> the previous behavior and also consistent with other modification >> methods like Map.remove (we don't throw from remove if the key is out >> of range, simply do nothing). >> >> To cover these changes with tests, I added new >> TreeMap().descendingMap() to >> InPlaceOpsCollisions.nullValueFriendlyMaps and >> MapWithCollisionsProviders.makeMapsMoreTypes. This map should behave >> like a normal Map modulo iteration order but implementation goes >> through NavigableSubMap. Also, I added more adders to >> LockStep::randomAdder (lines 572+) to cover new methods, as well as >> some more throws IAE asserts around line 806. At the same time, I took >> the liberty to modernize this test: >> - Convert all the raw-types to properly parameterized (using 'var' for >> local vars where resulting type is long) >> - Convert MapFrobber and SetFrobber to interfaces, and convert all the >> implementations to lambdas (automatic conversion via IntelliJ IDEA + >> rename of conflicting parameter names) >> - Use for-each loop instead of indexed for loop where possible >> - 'elts' array was created in two places but unused afterward. I added >> size assert to these arrays to use it at least somehow (though >> probably content should be checked as well). >> - Use Comparator.naturalOrder() instead of manually written one in >> comparator() methods (should not affect the testing in any way as it's >> only used directly, not passed e.g. to TreeMap constructor). >> - Use Objects.equals inside LockStep#equal >> - Inverted 'if' at line 299 to avoid empty block. >> If the cleanup really complicates the review I can post the cleanup as >> a separate changeset. Though it should not be very problematic as the >> actual changes are quite compact (as said above, near lines 572 and >> 806) >> >> I also improved the previously added benchmark TreeMapUpdate to check >> descendingMap and tailMap cases to confirm the hypothesis that the >> change improves the performance of derived maps. This is indeed the >> case. E.g. merge test yields (comparator = false, size = 1000) for >> unpatched JDK: >> >> (benchmark) (mode) (prefill) Score Error Units >> TreeMapUpdate.merge TreeMap true 63589,075 ? 1028,065 ns/op >> TreeMapUpdate.merge TreeMap false 65414,367 ? 1011,442 ns/op >> TreeMapUpdate.merge descendingMap true 121501,618 ? 1849,108 ns/op >> TreeMapUpdate.merge descendingMap false 95913,212 ? 2854,063 ns/op >> TreeMapUpdate.merge tailMap true 126657,309 ? 4467,733 ns/op >> TreeMapUpdate.merge tailMap false 93448,840 ? 1359,392 ns/op >> >> As you can see, the merge on derived maps is significantly slower. >> After the patch is applied the results are much better: >> >> (benchmark) (mode) (prefill) Score Error Units >> TreeMapUpdate.merge TreeMap true 64059,189 ? 808,230 ns/op >> TreeMapUpdate.merge TreeMap false 65156,912 ? 1229,965 ns/op >> TreeMapUpdate.merge descendingMap true 69938,131 ? 1697,357 ns/op >> TreeMapUpdate.merge descendingMap false 67783,829 ? 263,061 ns/op >> TreeMapUpdate.merge tailMap true 71079,529 ? 841,738 ns/op >> TreeMapUpdate.merge tailMap false 68577,169 ? 1196,758 ns/op >> >> I don't think this requires a CSR, as the changed class is >> package-private, so it cannot be extended by clients. >> >> With best regards, >> Tagir Valeev. From joe.darcy at oracle.com Mon Aug 3 16:24:51 2020 From: joe.darcy at oracle.com (Joe Darcy) Date: Mon, 3 Aug 2020 09:24:51 -0700 Subject: JDK 16 RFR of JDK-8250660: Clarify that WildcardType and AnnotatedWildcardType bounds methods return one In-Reply-To: <63af16f1-ddd3-7e56-5392-891e6328fc25@oracle.com> References: <9808e420-39a8-2045-9db1-629a551f82a0@oracle.com> <63af16f1-ddd3-7e56-5392-891e6328fc25@oracle.com> Message-ID: <99628559-a8c4-28c1-9e34-c01bdc8517c6@oracle.com> Hi David, To provide a bit more context, the API was consciously designed to be future-proofed against the possibility of Java supporting multiple bounds here (JDK-4891872). As of yet, that change has not come to pass, and as you note may never come to pass. So besides in addition to the links into the JLS, I think it is reasonable to include an API note acknowledging the situation. How about: ??? This method supports the existence of multiple bounds, but to date, a wildcard may have at most one upper/lower bound. -Joe On 8/2/2020 7:22 PM, David Holmes wrote: > Hi Joe, > > I was under the impression that using phrases like "the current > version of" were superfluous as all such documentation implicitly > refers to the current version, unless explicitly stated otherwise. > > Personally I'm not sure this clarification really makes a difference > to anything as the main source of confusion with this API is why it > states "bounds" rather than "bound" and thus returns an array. By > adding this clarification you will require a future update if the > language allows multiple bounds, forcing the docs to carry around > historical data "prior to Java N only a single bound were allowed but > since then multiple bounds ...". This seems unnecessary clutter to me. > Given the ship has sailed on the shape of this API the developer must > simply embrace it and always process the return value in a loop > without assuming any knowledge of how many bounds may actually exist. > > I think the reference to the JLS section suffices to instruct the > developer on where to find out how many bounds to expect, whatever > that may be in their current version of the JLS. > > If we think we will never support multiple bounds then these methods > should be deprecated (for removal) and replaced by a singular > counterpart that returns a single Type rather than a Type[]. > > Cheers, > David > ----- > > On 1/08/2020 8:12 am, Joe Darcy wrote: >> Hello, >> >> Please review the doc clarification below to address >> >> ???? 8250660: Clarify that WildcardType and AnnotatedWildcardType >> bounds methods return one >> >> Thanks, >> >> -Joe >> >> diff -r b0553ba43ba1 >> src/java.base/share/classes/java/lang/reflect/AnnotatedWildcardType.java >> --- >> a/src/java.base/share/classes/java/lang/reflect/AnnotatedWildcardType.java >> Fri Jul 31 11:35:25 2020 -0700 >> +++ >> b/src/java.base/share/classes/java/lang/reflect/AnnotatedWildcardType.java >> Fri Jul 31 15:12:14 2020 -0700 >> @@ -1,5 +1,5 @@ >> ??/* >> - * Copyright (c) 2012, 2015, Oracle and/or its affiliates. All >> rights reserved. >> + * Copyright (c) 2012, 2020, Oracle and/or its affiliates. All >> rights reserved. >> ?? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >> ?? * >> ?? * This code is free software; you can redistribute it and/or >> modify it >> @@ -30,6 +30,7 @@ >> ?? * wildcard type argument, whose upper or lower bounds may >> themselves represent >> ?? * annotated uses of types. >> ?? * >> + * @jls 4.5.1 Type Arguments of Parameterized Types >> ?? * @since 1.8 >> ?? */ >> ??public interface AnnotatedWildcardType extends AnnotatedType { >> @@ -39,6 +40,9 @@ >> ?????? * If no lower bound is explicitly declared, the lower bound is >> the >> ?????? * type of null. In this case, a zero length array is returned. >> ?????? * >> +???? * @apiNote In the current version of the Java Programming >> +???? * Language, a wildcard may have at most one lower bound. >> +???? * >> ?????? * @return the potentially annotated lower bounds of this >> wildcard type or >> ?????? * an empty array if no lower bound is explicitly declared. >> ?????? * @see WildcardType#getLowerBounds() >> @@ -50,6 +54,9 @@ >> ?????? * If no upper bound is explicitly declared, the upper bound is >> ?????? * unannotated {@code Object} >> ?????? * >> +???? * @apiNote In the current version of the Java Programming >> +???? * Language, a wildcard may have at most one upper bound. >> +???? * >> ?????? * @return the potentially annotated upper bounds of this >> wildcard type >> ?????? * @see WildcardType#getUpperBounds() >> ?????? */ >> diff -r b0553ba43ba1 >> src/java.base/share/classes/java/lang/reflect/WildcardType.java >> --- a/src/java.base/share/classes/java/lang/reflect/WildcardType.java >> Fri Jul 31 11:35:25 2020 -0700 >> +++ b/src/java.base/share/classes/java/lang/reflect/WildcardType.java >> Fri Jul 31 15:12:14 2020 -0700 >> @@ -1,5 +1,5 @@ >> ??/* >> - * Copyright (c) 2003, 2015, Oracle and/or its affiliates. All >> rights reserved. >> + * Copyright (c) 2003, 2020, Oracle and/or its affiliates. All >> rights reserved. >> ?? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >> ?? * >> ?? * This code is free software; you can redistribute it and/or >> modify it >> @@ -29,6 +29,7 @@ >> ?? * WildcardType represents a wildcard type expression, such as >> ?? * {@code ?}, {@code ? extends Number}, or {@code ? super Integer}. >> ?? * >> + * @jls 4.5.1 Type Arguments of Parameterized Types >> ?? * @since 1.5 >> ?? */ >> ??public interface WildcardType extends Type { >> @@ -45,6 +46,9 @@ >> ?????? *?
  • Otherwise, B is resolved. >> ?????? * >> ?????? * >> +???? * @apiNote In the current version of the Java Programming >> +???? * Language, a wildcard may have at most one upper bound. >> +???? * >> ?????? * @return an array of Types representing the upper bound(s) of >> this >> ?????? *???? type variable >> ?????? * @throws TypeNotPresentException if any of the >> @@ -69,6 +73,9 @@ >> ?????? *??
  • Otherwise, B is resolved. >> ?????? * >> ?????? * >> +???? * @apiNote In the current version of the Java Programming >> +???? * Language, a wildcard may have at most one lower bound. >> +???? * >> ?????? * @return an array of Types representing the lower bound(s) of >> this >> ?????? *???? type variable >> ?????? * @throws TypeNotPresentException if any of the >> @@ -78,6 +85,4 @@ >> ?????? *???? for any reason >> ?????? */ >> ????? Type[] getLowerBounds(); >> -??? // one or many? Up to language spec; currently only one, but >> this API >> -??? // allows for generalization. >> ??} >> From vladimir.kozlov at oracle.com Mon Aug 3 17:10:40 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 3 Aug 2020 10:10:40 -0700 Subject: RFR[M]: Adding MD5 Intrinsic on x86-64 In-Reply-To: References: Message-ID: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> Hi Ludovic This is very professional work! CCing to Core-libs because you modified Java code and need review from Java library group. Few notes: Add tests to verify intrinsic implementation. You can use test/hotspot/jtreg/compiler/intrinsics/sha/ as examples. In vm_version_x86.cpp move UseMD5Intrinsics flag setting near UseSHA flag setting. In new file macroAssembler_x86_md5.cpp no need empty line after copyright line. There is also typo 'rrdistribute': * This code is free software; you can rrdistribute it and/or modify it Our validate-headers check failed. See GPL header template: ./make/templates/gpl-header Ludovic, it looks like you used only general instructions to implement this code. Can you add comment where the algorithm come from? Or it is just direct translation of Java code? Vivek, do we have SSE/AVX instructions which may improve performance of this code? It could be follow up update if we can. Did you test it on 32-bit x86? Would be interesting to see result of artificially switching off AVX and SSE: '-XX:UseSSE=0 -XX:UseAVX=0'. It will make sure that only general instructions are needed. Thanks, Vladimir On 8/3/20 7:39 AM, Ludovic Henry wrote: > Hi, > > A quick follow up on that change. Are you happy with the general approach, or would rather have it done differently? > > JBS: https://bugs.openjdk.java.net/browse/JDK-8250902 > Webrev: http://cr.openjdk.java.net/~luhenry/8250902/webrev.01/ > > Thank you > Ludovic > From alexey.semenyuk at oracle.com Mon Aug 3 17:10:58 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Mon, 3 Aug 2020 13:10:58 -0400 Subject: RFR: 8250646: hdiutil detach fix JDK-8245311 still fails sometimes In-Reply-To: <76d775dd-fc4b-16e8-9517-cb9a0623509a@oracle.com> References: <7ff275fd-90f6-05c7-2979-4622daeb82e4@oracle.com> <76d775dd-fc4b-16e8-9517-cb9a0623509a@oracle.com> Message-ID: <8703f438-dbe8-9ffb-4d4c-dbaa8bfc9e21@oracle.com> On 7/31/2020 7:36 PM, alexander.matveev at oracle.com wrote: > Hi Alexey, > > Without modifying some logic in Executor, yes stop is needed. After > retryExecutor.stop(); is executed in MacDmgBundler we will still run > command (same with abort), and this command will fail and exception > will be thrown. Stop() is basically like abort, but exception is ignored. Right, but can't we just have one method that will abort the loop? Why not test for the value of `aborted` field not only in exception handler, but also before running executor? I mean instead of --- if (stop) { ??? break; } --- have --- if (aborted) { ??? break; } --- This would eliminate the need to introduce stop() method. What do you think? - Alexey > > Thanks, > Alexander > > On 7/31/20 9:27 AM, Alexey Semenyuk wrote: >> Alexander, >> >> Why introduce RetryExecutor.stop() method? Isn't >> RetryExecutor.abort() sufficient? >> >> - Alexey >> >> On 7/30/2020 8:10 PM, alexander.matveev at oracle.com wrote: >>> Please review the jpackage fix for bug [1] at [2]. >>> >>> - DMG image can get detach even if "hdiutil detach" return resource >>> busy error. Fixed by stopping trying detach if DMG already got detach. >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8250646 >>> [2] http://cr.openjdk.java.net/~almatvee/8250646/webrev.00/ >>> >>> Thanks, >>> Alexander >> > From anthony.scarpino at oracle.com Mon Aug 3 17:31:38 2020 From: anthony.scarpino at oracle.com (Anthony Scarpino) Date: Mon, 3 Aug 2020 10:31:38 -0700 Subject: RFR[M]: Adding MD5 Intrinsic on x86-64 In-Reply-To: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> References: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> Message-ID: <724174CC-78F4-453C-9420-DC30B8E44664@oracle.com> I had looked at the java code changes and are fine with them Tony > On Aug 3, 2020, at 10:10 AM, Vladimir Kozlov wrote: > > ?Hi Ludovic > > This is very professional work! > > CCing to Core-libs because you modified Java code and need review from Java library group. > > Few notes: > > Add tests to verify intrinsic implementation. You can use test/hotspot/jtreg/compiler/intrinsics/sha/ as examples. > > In vm_version_x86.cpp move UseMD5Intrinsics flag setting near UseSHA flag setting. > > In new file macroAssembler_x86_md5.cpp no need empty line after copyright line. There is also typo 'rrdistribute': > > * This code is free software; you can rrdistribute it and/or modify it > > Our validate-headers check failed. See GPL header template: ./make/templates/gpl-header > > Ludovic, it looks like you used only general instructions to implement this code. Can you add comment where the algorithm come from? Or it is just direct translation of Java code? > > Vivek, do we have SSE/AVX instructions which may improve performance of this code? It could be follow up update if we can. > > Did you test it on 32-bit x86? Would be interesting to see result of artificially switching off AVX and SSE: '-XX:UseSSE=0 -XX:UseAVX=0'. It will make sure that only general instructions are needed. > > Thanks, > Vladimir > >> On 8/3/20 7:39 AM, Ludovic Henry wrote: >> Hi, >> A quick follow up on that change. Are you happy with the general approach, or would rather have it done differently? >> JBS: https://bugs.openjdk.java.net/browse/JDK-8250902 >> Webrev: http://cr.openjdk.java.net/~luhenry/8250902/webrev.01/ >> Thank you >> Ludovic From vladimir.kozlov at oracle.com Mon Aug 3 17:34:28 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 3 Aug 2020 10:34:28 -0700 Subject: RFR[M]: Adding MD5 Intrinsic on x86-64 In-Reply-To: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> References: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> Message-ID: <159bfdfb-6476-0826-8cf7-145202c7ce33@oracle.com> And I got crash during JDK build on linux-x64: # Internal Error (src/hotspot/share/opto/library_call.cpp:5732), pid=18904, tid=19012 # assert(field != __null) failed: undefined field # # Java VM: OpenJDK 64-Bit Server VM (fastdebug 16-internal+0-2020-08-03-1651458.vladimir.kozlov.jdkjdk, mixed mode, tiered, compressed oops, g1 gc, linux-amd64) # Problematic frame: # V [libjvm.so+0x11123f4] LibraryCallKit::load_field_from_object(Node*, char const*, char const*, bool, bool, ciInstanceKlass*)+0x334 Current CompileTask: C2: 6204 1305 4 sun.security.provider.DigestBase::engineUpdate (189 bytes) Stack: [0x0000151bfcfc7000,0x0000151bfd0c8000], sp=0x0000151bfd0c3ed0, free space=1011k Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code) V [libjvm.so+0x11123f4] LibraryCallKit::load_field_from_object(Node*, char const*, char const*, bool, bool, ciInstanceKlass*)+0x334 V [libjvm.so+0x11167ea] LibraryCallKit::get_long_state_from_digestBase_object(Node*)+0x2a V [libjvm.so+0x1116f2d] LibraryCallKit::inline_digestBase_implCompressMB(Node*, ciInstanceKlass*, bool, unsigned char*, char const*, Node*, Node*, Node*)+0x2cd V [libjvm.so+0x1117467] LibraryCallKit::inline_digestBase_implCompressMB(int)+0x397 V [libjvm.so+0x1121de1] LibraryIntrinsic::generate(JVMState*)+0x211 V [libjvm.so+0x75d61d] PredicatedIntrinsicGenerator::generate(JVMState*)+0xb8d Vladimir K On 8/3/20 10:10 AM, Vladimir Kozlov wrote: > Hi Ludovic > > This is very professional work! > > CCing to Core-libs because you modified Java code and need review from Java library group. > > Few notes: > > Add tests to verify intrinsic implementation. You can use test/hotspot/jtreg/compiler/intrinsics/sha/ as examples. > > In vm_version_x86.cpp move UseMD5Intrinsics flag setting near UseSHA flag setting. > > In new file macroAssembler_x86_md5.cpp no need empty line after copyright line. There is also typo 'rrdistribute': > > ?* This code is free software; you can rrdistribute it and/or modify it > > Our validate-headers check failed. See GPL header template: ./make/templates/gpl-header > > Ludovic, it looks like you used only general instructions to implement this code. Can you add comment where the > algorithm come from? Or it is just direct translation of Java code? > > Vivek, do we have SSE/AVX instructions which may improve performance of this code? It could be follow up update if we can. > > Did you test it on 32-bit x86? Would be interesting to see result of artificially switching off AVX and SSE: > '-XX:UseSSE=0 -XX:UseAVX=0'. It will make sure that only general instructions are needed. > > Thanks, > Vladimir > > On 8/3/20 7:39 AM, Ludovic Henry wrote: >> Hi, >> >> A quick follow up on that change. Are you happy with the general approach, or would rather have it done differently? >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8250902 >> Webrev: http://cr.openjdk.java.net/~luhenry/8250902/webrev.01/ >> >> Thank you >> Ludovic >> From vladimir.kozlov at oracle.com Mon Aug 3 18:49:18 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 3 Aug 2020 11:49:18 -0700 Subject: RFR[M]: Adding MD5 Intrinsic on x86-64 In-Reply-To: References: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> <159bfdfb-6476-0826-8cf7-145202c7ce33@oracle.com> Message-ID: Hmm, I applied your http://cr.openjdk.java.net/~luhenry/8250902/webrev.01/jdk.changeset But it looks like it has more changes (windows_aarch64) then just MD5 intrinsic. I will retest again with removed other changes. Vladimir K On 8/3/20 11:12 AM, Ludovic Henry wrote: >> And I got crash during JDK build on linux-x64: >> >> # Internal Error (src/hotspot/share/opto/library_call.cpp:5732), pid=18904, tid=19012 >> # assert(field != __null) failed: undefined field >> # >> # Java VM: OpenJDK 64-Bit Server VM (fastdebug 16-internal+0-2020-08-03-1651458.vladimir.kozlov.jdkjdk, mixed mode, >> tiered, compressed oops, g1 gc, linux-amd64) >> # Problematic frame: >> # V [libjvm.so+0x11123f4] LibraryCallKit::load_field_from_object(Node*, char const*, char const*, bool, bool, ciInstanceKlass*)+0x334 >> >> Current CompileTask: >> C2: 6204 1305 4 sun.security.provider.DigestBase::engineUpdate (189 bytes) >> >> Stack: [0x0000151bfcfc7000,0x0000151bfd0c8000], sp=0x0000151bfd0c3ed0, free space=1011k >> Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code) >> V [libjvm.so+0x11123f4] LibraryCallKit::load_field_from_object(Node*, char const*, char const*, bool, bool, ciInstanceKlass*)+0x334 >> V [libjvm.so+0x11167ea] LibraryCallKit::get_long_state_from_digestBase_object(Node*)+0x2a >> V [libjvm.so+0x1116f2d] LibraryCallKit::inline_digestBase_implCompressMB(Node*, ciInstanceKlass*, bool, unsigned char*, char const*, Node*, Node*, Node*)+0x2cd >> V [libjvm.so+0x1117467] LibraryCallKit::inline_digestBase_implCompressMB(int)+0x397 >> V [libjvm.so+0x1121de1] LibraryIntrinsic::generate(JVMState*)+0x211 >> V [libjvm.so+0x75d61d] PredicatedIntrinsicGenerator::generate(JVMState*)+0xb8d > > Interesting, I did all my work on Linux-x64 but didn't observe that. Let me try to reproduce and come back to you on that. > From luhenry at microsoft.com Mon Aug 3 18:12:32 2020 From: luhenry at microsoft.com (Ludovic Henry) Date: Mon, 3 Aug 2020 18:12:32 +0000 Subject: RFR[M]: Adding MD5 Intrinsic on x86-64 In-Reply-To: <159bfdfb-6476-0826-8cf7-145202c7ce33@oracle.com> References: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> <159bfdfb-6476-0826-8cf7-145202c7ce33@oracle.com> Message-ID: > And I got crash during JDK build on linux-x64: > > # Internal Error (src/hotspot/share/opto/library_call.cpp:5732), pid=18904, tid=19012 > # assert(field != __null) failed: undefined field > # > # Java VM: OpenJDK 64-Bit Server VM (fastdebug 16-internal+0-2020-08-03-1651458.vladimir.kozlov.jdkjdk, mixed mode, > tiered, compressed oops, g1 gc, linux-amd64) > # Problematic frame: > # V [libjvm.so+0x11123f4] LibraryCallKit::load_field_from_object(Node*, char const*, char const*, bool, bool, ciInstanceKlass*)+0x334 > > Current CompileTask: > C2: 6204 1305 4 sun.security.provider.DigestBase::engineUpdate (189 bytes) > > Stack: [0x0000151bfcfc7000,0x0000151bfd0c8000], sp=0x0000151bfd0c3ed0, free space=1011k > Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code) > V [libjvm.so+0x11123f4] LibraryCallKit::load_field_from_object(Node*, char const*, char const*, bool, bool, ciInstanceKlass*)+0x334 > V [libjvm.so+0x11167ea] LibraryCallKit::get_long_state_from_digestBase_object(Node*)+0x2a > V [libjvm.so+0x1116f2d] LibraryCallKit::inline_digestBase_implCompressMB(Node*, ciInstanceKlass*, bool, unsigned char*, char const*, Node*, Node*, Node*)+0x2cd > V [libjvm.so+0x1117467] LibraryCallKit::inline_digestBase_implCompressMB(int)+0x397 > V [libjvm.so+0x1121de1] LibraryIntrinsic::generate(JVMState*)+0x211 > V [libjvm.so+0x75d61d] PredicatedIntrinsicGenerator::generate(JVMState*)+0xb8d Interesting, I did all my work on Linux-x64 but didn't observe that. Let me try to reproduce and come back to you on that. From luhenry at microsoft.com Mon Aug 3 18:52:34 2020 From: luhenry at microsoft.com (Ludovic Henry) Date: Mon, 3 Aug 2020 18:52:34 +0000 Subject: RFR[M]: Adding MD5 Intrinsic on x86-64 In-Reply-To: References: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> <159bfdfb-6476-0826-8cf7-145202c7ce33@oracle.com> Message-ID: > But it looks like it has more changes (windows_aarch64) then just MD5 intrinsic. > I will retest again with removed other changes. That looks like a mistake with me learning to use Mercurial, sorry about that. The only patch you need is `8250902: Implement MD5 Intrinsics on x86`, all the others are my mistake. From luhenry at microsoft.com Mon Aug 3 19:00:06 2020 From: luhenry at microsoft.com (Ludovic Henry) Date: Mon, 3 Aug 2020 19:00:06 +0000 Subject: RFR[M]: Adding MD5 Intrinsic on x86-64 In-Reply-To: References: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> <159bfdfb-6476-0826-8cf7-145202c7ce33@oracle.com> Message-ID: I've updated [1] with the proper patch. [1] http://cr.openjdk.java.net/~luhenry/md5-intrinsics/webrev.01/8250902.patch From vladimir.kozlov at oracle.com Mon Aug 3 19:18:53 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 3 Aug 2020 12:18:53 -0700 Subject: RFR[M]: Adding MD5 Intrinsic on x86-64 In-Reply-To: References: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> <159bfdfb-6476-0826-8cf7-145202c7ce33@oracle.com> Message-ID: I reproduced crash with only MD5 changes on my local linux machine during fastdebug build. Next code in inline_digestBase_implCompressMB should be reversed (get_long_*() should be called for long_state): if (long_state) { state = get_state_from_digestBase_object(digestBase_obj); } else { state = get_long_state_from_digestBase_object(digestBase_obj); } Vladimir K On 8/3/20 11:52 AM, Ludovic Henry wrote: >> But it looks like it has more changes (windows_aarch64) then just MD5 intrinsic. >> I will retest again with removed other changes. > > That looks like a mistake with me learning to use Mercurial, sorry about that. > > The only patch you need is `8250902: Implement MD5 Intrinsics on x86`, all the others are my mistake. > > From viv.desh at gmail.com Mon Aug 3 22:08:22 2020 From: viv.desh at gmail.com (Vivek Deshpande) Date: Mon, 3 Aug 2020 15:08:22 -0700 Subject: RFR[M]: Adding MD5 Intrinsic on x86-64 In-Reply-To: References: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> <159bfdfb-6476-0826-8cf7-145202c7ce33@oracle.com> Message-ID: Hi Vladimir It seems that the algorithm can be optimized further using SSE/AVX instructions. I am not aware of any specific SSE/AVX implementation which leverages those instructions in the best possible way. Sandhya can chime in more on that. As far as I know, I came across this which points to MD5 SSE/AVX implementation. https://software.intel.com/content/www/us/en/develop/articles/intel-isa-l-cryptographic-hashes-for-cloud-storage.html Regards, Vivek On Mon, Aug 3, 2020 at 12:21 PM Vladimir Kozlov wrote: > I reproduced crash with only MD5 changes on my local linux machine during > fastdebug build. > > Next code in inline_digestBase_implCompressMB should be reversed > (get_long_*() should be called for long_state): > > if (long_state) { > state = get_state_from_digestBase_object(digestBase_obj); > } else { > state = get_long_state_from_digestBase_object(digestBase_obj); > } > > Vladimir K > > On 8/3/20 11:52 AM, Ludovic Henry wrote: > >> But it looks like it has more changes (windows_aarch64) then just MD5 > intrinsic. > >> I will retest again with removed other changes. > > > > That looks like a mistake with me learning to use Mercurial, sorry about > that. > > > > The only patch you need is `8250902: Implement MD5 Intrinsics on x86`, > all the others are my mistake. > > > > > -- Thanks and Regards, Vivek Deshpande viv.desh at gmail.com From vladimir.kozlov at oracle.com Mon Aug 3 23:10:21 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 3 Aug 2020 16:10:21 -0700 Subject: RFR[M]: Adding MD5 Intrinsic on x86-64 In-Reply-To: References: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> <159bfdfb-6476-0826-8cf7-145202c7ce33@oracle.com> Message-ID: <0d97ffec-1e6e-65d3-d1c3-b39f72145c14@oracle.com> Thank you, Vivek, for pointer. This is interesting ,could be somehitng Intel's mlib may have. Vladimir K On 8/3/20 3:08 PM, Vivek Deshpande wrote: > Hi Vladimir > > It seems that the algorithm can be optimized further using SSE/AVX > instructions. I am not aware of any specific SSE/AVX implementation which > leverages those instructions in the best possible way. Sandhya can chime > in more on that. > As far as I know, I came across this which points to MD5 SSE/AVX > implementation. > https://software.intel.com/content/www/us/en/develop/articles/intel-isa-l-cryptographic-hashes-for-cloud-storage.html > > Regards, > Vivek > > On Mon, Aug 3, 2020 at 12:21 PM Vladimir Kozlov > wrote: > >> I reproduced crash with only MD5 changes on my local linux machine during >> fastdebug build. >> >> Next code in inline_digestBase_implCompressMB should be reversed >> (get_long_*() should be called for long_state): >> >> if (long_state) { >> state = get_state_from_digestBase_object(digestBase_obj); >> } else { >> state = get_long_state_from_digestBase_object(digestBase_obj); >> } >> >> Vladimir K >> >> On 8/3/20 11:52 AM, Ludovic Henry wrote: >>>> But it looks like it has more changes (windows_aarch64) then just MD5 >> intrinsic. >>>> I will retest again with removed other changes. >>> >>> That looks like a mistake with me learning to use Mercurial, sorry about >> that. >>> >>> The only patch you need is `8250902: Implement MD5 Intrinsics on x86`, >> all the others are my mistake. >>> >>> >> > > From david.holmes at oracle.com Mon Aug 3 23:42:29 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 4 Aug 2020 09:42:29 +1000 Subject: JDK 16 RFR of JDK-8250660: Clarify that WildcardType and AnnotatedWildcardType bounds methods return one In-Reply-To: <99628559-a8c4-28c1-9e34-c01bdc8517c6@oracle.com> References: <9808e420-39a8-2045-9db1-629a551f82a0@oracle.com> <63af16f1-ddd3-7e56-5392-891e6328fc25@oracle.com> <99628559-a8c4-28c1-9e34-c01bdc8517c6@oracle.com> Message-ID: <7e53a817-9dc5-1d62-baa1-6ede1a94cb2f@oracle.com> Hi Joe, On 4/08/2020 2:24 am, Joe Darcy wrote: > Hi David, > > To provide a bit more context, the API was consciously designed to be > future-proofed against the possibility of Java supporting multiple > bounds here (JDK-4891872). As of yet, that change has not come to pass, Okay so we want to keep the API. :) > and as you note may never come to pass. So besides in addition to the > links into the JLS, I think it is reasonable to include an API note > acknowledging the situation. How about: > > ??? This method supports the existence of multiple bounds, but to date, > a wildcard may have at most one upper/lower bound. "to date" is just another form of "in the current version". Sorry I find it hard to suggest suitable text when I don't think it necessary to say anything. Best to let others endorse this one. Cheers, David ----- > -Joe > > > On 8/2/2020 7:22 PM, David Holmes wrote: >> Hi Joe, >> >> I was under the impression that using phrases like "the current >> version of" were superfluous as all such documentation implicitly >> refers to the current version, unless explicitly stated otherwise. >> >> Personally I'm not sure this clarification really makes a difference >> to anything as the main source of confusion with this API is why it >> states "bounds" rather than "bound" and thus returns an array. By >> adding this clarification you will require a future update if the >> language allows multiple bounds, forcing the docs to carry around >> historical data "prior to Java N only a single bound were allowed but >> since then multiple bounds ...". This seems unnecessary clutter to me. >> Given the ship has sailed on the shape of this API the developer must >> simply embrace it and always process the return value in a loop >> without assuming any knowledge of how many bounds may actually exist. >> >> I think the reference to the JLS section suffices to instruct the >> developer on where to find out how many bounds to expect, whatever >> that may be in their current version of the JLS. >> >> If we think we will never support multiple bounds then these methods >> should be deprecated (for removal) and replaced by a singular >> counterpart that returns a single Type rather than a Type[]. >> >> Cheers, >> David >> ----- >> >> On 1/08/2020 8:12 am, Joe Darcy wrote: >>> Hello, >>> >>> Please review the doc clarification below to address >>> >>> ???? 8250660: Clarify that WildcardType and AnnotatedWildcardType >>> bounds methods return one >>> >>> Thanks, >>> >>> -Joe >>> >>> diff -r b0553ba43ba1 >>> src/java.base/share/classes/java/lang/reflect/AnnotatedWildcardType.java >>> --- >>> a/src/java.base/share/classes/java/lang/reflect/AnnotatedWildcardType.java >>> Fri Jul 31 11:35:25 2020 -0700 >>> +++ >>> b/src/java.base/share/classes/java/lang/reflect/AnnotatedWildcardType.java >>> Fri Jul 31 15:12:14 2020 -0700 >>> @@ -1,5 +1,5 @@ >>> ??/* >>> - * Copyright (c) 2012, 2015, Oracle and/or its affiliates. All >>> rights reserved. >>> + * Copyright (c) 2012, 2020, Oracle and/or its affiliates. All >>> rights reserved. >>> ?? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >>> ?? * >>> ?? * This code is free software; you can redistribute it and/or >>> modify it >>> @@ -30,6 +30,7 @@ >>> ?? * wildcard type argument, whose upper or lower bounds may >>> themselves represent >>> ?? * annotated uses of types. >>> ?? * >>> + * @jls 4.5.1 Type Arguments of Parameterized Types >>> ?? * @since 1.8 >>> ?? */ >>> ??public interface AnnotatedWildcardType extends AnnotatedType { >>> @@ -39,6 +40,9 @@ >>> ?????? * If no lower bound is explicitly declared, the lower bound is >>> the >>> ?????? * type of null. In this case, a zero length array is returned. >>> ?????? * >>> +???? * @apiNote In the current version of the Java Programming >>> +???? * Language, a wildcard may have at most one lower bound. >>> +???? * >>> ?????? * @return the potentially annotated lower bounds of this >>> wildcard type or >>> ?????? * an empty array if no lower bound is explicitly declared. >>> ?????? * @see WildcardType#getLowerBounds() >>> @@ -50,6 +54,9 @@ >>> ?????? * If no upper bound is explicitly declared, the upper bound is >>> ?????? * unannotated {@code Object} >>> ?????? * >>> +???? * @apiNote In the current version of the Java Programming >>> +???? * Language, a wildcard may have at most one upper bound. >>> +???? * >>> ?????? * @return the potentially annotated upper bounds of this >>> wildcard type >>> ?????? * @see WildcardType#getUpperBounds() >>> ?????? */ >>> diff -r b0553ba43ba1 >>> src/java.base/share/classes/java/lang/reflect/WildcardType.java >>> --- a/src/java.base/share/classes/java/lang/reflect/WildcardType.java >>> Fri Jul 31 11:35:25 2020 -0700 >>> +++ b/src/java.base/share/classes/java/lang/reflect/WildcardType.java >>> Fri Jul 31 15:12:14 2020 -0700 >>> @@ -1,5 +1,5 @@ >>> ??/* >>> - * Copyright (c) 2003, 2015, Oracle and/or its affiliates. All >>> rights reserved. >>> + * Copyright (c) 2003, 2020, Oracle and/or its affiliates. All >>> rights reserved. >>> ?? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >>> ?? * >>> ?? * This code is free software; you can redistribute it and/or >>> modify it >>> @@ -29,6 +29,7 @@ >>> ?? * WildcardType represents a wildcard type expression, such as >>> ?? * {@code ?}, {@code ? extends Number}, or {@code ? super Integer}. >>> ?? * >>> + * @jls 4.5.1 Type Arguments of Parameterized Types >>> ?? * @since 1.5 >>> ?? */ >>> ??public interface WildcardType extends Type { >>> @@ -45,6 +46,9 @@ >>> ?????? *?
  • Otherwise, B is resolved. >>> ?????? * >>> ?????? * >>> +???? * @apiNote In the current version of the Java Programming >>> +???? * Language, a wildcard may have at most one upper bound. >>> +???? * >>> ?????? * @return an array of Types representing the upper bound(s) of >>> this >>> ?????? *???? type variable >>> ?????? * @throws TypeNotPresentException if any of the >>> @@ -69,6 +73,9 @@ >>> ?????? *??
  • Otherwise, B is resolved. >>> ?????? * >>> ?????? * >>> +???? * @apiNote In the current version of the Java Programming >>> +???? * Language, a wildcard may have at most one lower bound. >>> +???? * >>> ?????? * @return an array of Types representing the lower bound(s) of >>> this >>> ?????? *???? type variable >>> ?????? * @throws TypeNotPresentException if any of the >>> @@ -78,6 +85,4 @@ >>> ?????? *???? for any reason >>> ?????? */ >>> ????? Type[] getLowerBounds(); >>> -??? // one or many? Up to language spec; currently only one, but >>> this API >>> -??? // allows for generalization. >>> ??} >>> From joe.darcy at oracle.com Mon Aug 3 23:45:15 2020 From: joe.darcy at oracle.com (Joe Darcy) Date: Mon, 3 Aug 2020 16:45:15 -0700 Subject: JDK 16 RFR of JDK-8250660: Clarify that WildcardType and AnnotatedWildcardType bounds methods return one In-Reply-To: <7e53a817-9dc5-1d62-baa1-6ede1a94cb2f@oracle.com> References: <9808e420-39a8-2045-9db1-629a551f82a0@oracle.com> <63af16f1-ddd3-7e56-5392-891e6328fc25@oracle.com> <99628559-a8c4-28c1-9e34-c01bdc8517c6@oracle.com> <7e53a817-9dc5-1d62-baa1-6ede1a94cb2f@oracle.com> Message-ID: Hi David, On 8/3/2020 4:42 PM, David Holmes wrote: > Hi Joe, > > On 4/08/2020 2:24 am, Joe Darcy wrote: >> Hi David, >> >> To provide a bit more context, the API was consciously designed to be >> future-proofed against the possibility of Java supporting multiple >> bounds here (JDK-4891872). As of yet, that change has not come to pass, > > Okay so we want to keep the API. :) I don't see us getting rid of this one :-) > >> and as you note may never come to pass. So besides in addition to the >> links into the JLS, I think it is reasonable to include an API note >> acknowledging the situation. How about: >> >> ???? This method supports the existence of multiple bounds, but to >> date, a wildcard may have at most one upper/lower bound. > > "to date" is just another form of "in the current version". > > Sorry I find it hard to suggest suitable text when I don't think it > necessary to say anything. > > Best to let others endorse this one. Fair enough; thanks, -Joe From vladimir.kozlov at oracle.com Mon Aug 3 23:58:59 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 3 Aug 2020 16:58:59 -0700 Subject: RFR[M]: Adding MD5 Intrinsic on x86-64 In-Reply-To: References: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> <159bfdfb-6476-0826-8cf7-145202c7ce33@oracle.com> Message-ID: Hmm, with that code reversed I now have failure only on Windows: V [jvm.dll+0x43abb7] report_vm_error+0x117 (debug.cpp:264) V [jvm.dll+0x8a222e] LibraryCallKit::load_field_from_object+0x1ae (library_call.cpp:5732) V [jvm.dll+0x88c3ea] LibraryCallKit::get_state_from_digestBase_object+0x3a (library_call.cpp:6614) V [jvm.dll+0x8909d5] LibraryCallKit::inline_digestBase_implCompressMB+0x115 (library_call.cpp:6598) V [jvm.dll+0x8908b1] LibraryCallKit::inline_digestBase_implCompressMB+0x411 (library_call.cpp:6578) V [jvm.dll+0x8a5b2d] LibraryCallKit::try_to_inline+0x184d (library_call.cpp:836) The bug is in the same code as before - typreo due to renaming. So the code should be: if (long_state) { state = get_long_state_from_digestBase_object(obj); } else { state = get_state_from_digestBase_object(obj); } BTW, Ludovic, you need to add next change [1] to Graal's test to avoid its failure. Thanks, Vladimir K [1] src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot.test/src/org/graalvm/compiler/hotspot/test/CheckGraalIntrinsics.java @@ -423,6 +423,11 @@ "java/math/BigInteger.shiftRightImplWorker([I[IIII)V"); } + if (isJDK16OrHigher()) { + add(toBeInvestigated, + "sun/security/provider/MD5.implCompress0([BI)V"); + } + if (!config.inlineNotify()) { add(ignore, "java/lang/Object.notify()V"); } @@ -593,6 +598,14 @@ return JavaVersionUtil.JAVA_SPEC >= 14; } + private static boolean isJDK15OrHigher() { + return JavaVersionUtil.JAVA_SPEC >= 15; + } + + private static boolean isJDK16OrHigher() { + return JavaVersionUtil.JAVA_SPEC >= 16; + } + public interface Refiner { void refine(CheckGraalIntrinsics checker); } On 8/3/20 12:18 PM, Vladimir Kozlov wrote: > I reproduced crash with only MD5 changes on my local linux machine during fastdebug build. > > Next code in inline_digestBase_implCompressMB should be reversed (get_long_*() should be called for long_state): > > ?? if (long_state) { > ???? state = get_state_from_digestBase_object(digestBase_obj); > ?? } else { > ???? state = get_long_state_from_digestBase_object(digestBase_obj); > ?? } > > Vladimir K > > On 8/3/20 11:52 AM, Ludovic Henry wrote: >>> But it looks like it has more changes (windows_aarch64) then just MD5 intrinsic. >>> I will retest again with removed other changes. >> >> That looks like a mistake with me learning to use Mercurial, sorry about that. >> >> The only patch you need is `8250902: Implement MD5 Intrinsics on x86`, all the others are my mistake. >> From sandhya.viswanathan at intel.com Mon Aug 3 23:59:59 2020 From: sandhya.viswanathan at intel.com (Viswanathan, Sandhya) Date: Mon, 3 Aug 2020 23:59:59 +0000 Subject: RFR[M]: Adding MD5 Intrinsic on x86-64 In-Reply-To: <0d97ffec-1e6e-65d3-d1c3-b39f72145c14@oracle.com> References: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> <159bfdfb-6476-0826-8cf7-145202c7ce33@oracle.com> <0d97ffec-1e6e-65d3-d1c3-b39f72145c14@oracle.com> Message-ID: The link that Vivek shared is for multi-buffer implementation where multiple MD5 hashes for different buffers is calculated at once using SIMD. What is needed here is the acceleration of single buffer hash. I think that is what Henry's patch is proposing. Best Regards, Sandhya -----Original Message----- From: Vladimir Kozlov Sent: Monday, August 03, 2020 4:10 PM To: Vivek Deshpande Cc: Ludovic Henry ; hotspot-compiler-dev at openjdk.java.net; core-libs-dev ; Viswanathan, Sandhya Subject: Re: RFR[M]: Adding MD5 Intrinsic on x86-64 Thank you, Vivek, for pointer. This is interesting ,could be somehitng Intel's mlib may have. Vladimir K On 8/3/20 3:08 PM, Vivek Deshpande wrote: > Hi Vladimir > > It seems that the algorithm can be optimized further using SSE/AVX > instructions. I am not aware of any specific SSE/AVX implementation > which leverages those instructions in the best possible way. Sandhya > can chime in more on that. > As far as I know, I came across this which points to MD5 SSE/AVX > implementation. > https://software.intel.com/content/www/us/en/develop/articles/intel-is > a-l-cryptographic-hashes-for-cloud-storage.html > > Regards, > Vivek > > On Mon, Aug 3, 2020 at 12:21 PM Vladimir Kozlov > > wrote: > >> I reproduced crash with only MD5 changes on my local linux machine >> during fastdebug build. >> >> Next code in inline_digestBase_implCompressMB should be reversed >> (get_long_*() should be called for long_state): >> >> if (long_state) { >> state = get_state_from_digestBase_object(digestBase_obj); >> } else { >> state = get_long_state_from_digestBase_object(digestBase_obj); >> } >> >> Vladimir K >> >> On 8/3/20 11:52 AM, Ludovic Henry wrote: >>>> But it looks like it has more changes (windows_aarch64) then just >>>> MD5 >> intrinsic. >>>> I will retest again with removed other changes. >>> >>> That looks like a mistake with me learning to use Mercurial, sorry >>> about >> that. >>> >>> The only patch you need is `8250902: Implement MD5 Intrinsics on >>> x86`, >> all the others are my mistake. >>> >>> >> > > From luhenry at microsoft.com Tue Aug 4 00:13:06 2020 From: luhenry at microsoft.com (Ludovic Henry) Date: Tue, 4 Aug 2020 00:13:06 +0000 Subject: RFR[M]: Adding MD5 Intrinsic on x86-64 In-Reply-To: References: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> <159bfdfb-6476-0826-8cf7-145202c7ce33@oracle.com> Message-ID: I've fixed it at [1]. I'm sending an update very soon as soon as I have the performance numbers you asked for, and the test suites results on the different platforms of interest. [1] http://cr.openjdk.java.net/~luhenry/8250902/webrev.02/ -----Original Message----- From: Vladimir Kozlov Sent: Monday, August 3, 2020 4:59 PM To: Ludovic Henry ; hotspot-compiler-dev at openjdk.java.net; Vivek Deshpande Cc: core-libs-dev Subject: Re: RFR[M]: Adding MD5 Intrinsic on x86-64 Hmm, with that code reversed I now have failure only on Windows: V [jvm.dll+0x43abb7] report_vm_error+0x117 (debug.cpp:264) V [jvm.dll+0x8a222e] LibraryCallKit::load_field_from_object+0x1ae (library_call.cpp:5732) V [jvm.dll+0x88c3ea] LibraryCallKit::get_state_from_digestBase_object+0x3a (library_call.cpp:6614) V [jvm.dll+0x8909d5] LibraryCallKit::inline_digestBase_implCompressMB+0x115 (library_call.cpp:6598) V [jvm.dll+0x8908b1] LibraryCallKit::inline_digestBase_implCompressMB+0x411 (library_call.cpp:6578) V [jvm.dll+0x8a5b2d] LibraryCallKit::try_to_inline+0x184d (library_call.cpp:836) The bug is in the same code as before - typreo due to renaming. So the code should be: if (long_state) { state = get_long_state_from_digestBase_object(obj); } else { state = get_state_from_digestBase_object(obj); } BTW, Ludovic, you need to add next change [1] to Graal's test to avoid its failure. Thanks, Vladimir K [1] src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot.test/src/org/graalvm/compiler/hotspot/test/CheckGraalIntrinsics.java @@ -423,6 +423,11 @@ "java/math/BigInteger.shiftRightImplWorker([I[IIII)V"); } + if (isJDK16OrHigher()) { + add(toBeInvestigated, + "sun/security/provider/MD5.implCompress0([BI)V"); + } + if (!config.inlineNotify()) { add(ignore, "java/lang/Object.notify()V"); } @@ -593,6 +598,14 @@ return JavaVersionUtil.JAVA_SPEC >= 14; } + private static boolean isJDK15OrHigher() { + return JavaVersionUtil.JAVA_SPEC >= 15; + } + + private static boolean isJDK16OrHigher() { + return JavaVersionUtil.JAVA_SPEC >= 16; + } + public interface Refiner { void refine(CheckGraalIntrinsics checker); } On 8/3/20 12:18 PM, Vladimir Kozlov wrote: > I reproduced crash with only MD5 changes on my local linux machine during fastdebug build. > > Next code in inline_digestBase_implCompressMB should be reversed (get_long_*() should be called for long_state): > > ?? if (long_state) { > ???? state = get_state_from_digestBase_object(digestBase_obj); > ?? } else { > ???? state = get_long_state_from_digestBase_object(digestBase_obj); > ?? } > > Vladimir K > > On 8/3/20 11:52 AM, Ludovic Henry wrote: >>> But it looks like it has more changes (windows_aarch64) then just MD5 intrinsic. >>> I will retest again with removed other changes. >> >> That looks like a mistake with me learning to use Mercurial, sorry about that. >> >> The only patch you need is `8250902: Implement MD5 Intrinsics on x86`, all the others are my mistake. >> From luhenry at microsoft.com Tue Aug 4 04:07:49 2020 From: luhenry at microsoft.com (Ludovic Henry) Date: Tue, 4 Aug 2020 04:07:49 +0000 Subject: RFR[M]: Adding MD5 Intrinsic on x86-64 In-Reply-To: References: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> <159bfdfb-6476-0826-8cf7-145202c7ce33@oracle.com> Message-ID: Updated webrev: http://cr.openjdk.java.net/~luhenry/8250902/webrev.02 > Next code in inline_digestBase_implCompressMB should be reversed (get_long_*() should be called for long_state): > > if (long_state) { > state = get_state_from_digestBase_object(digestBase_obj); > } else { > state = get_long_state_from_digestBase_object(digestBase_obj); > } Thanks for pointing that out. I tested everything with `hotspot:tier1` and `jdk:tier1` in fastdebug on Windows-x86, Windows-x64 and Linux-x64. > It seems that the algorithm can be optimized further using SSE/AVX instructions. I am not aware of any specific SSE/AVX implementation which leverages those instructions in the best possible way. Sandhya can chime in more on that. I have done some research prior to implementing this intrinsic and the only pointers I could find to vectorized MD5 is on computing _multiple_ MD5 hashes in parallel but not a _single_ MD5 hash. Using vectors effectively parallelize the computation of many MD5 hash, but it does not accelerate the computation of a single MD5 hash. And looking at the algorithm, every step depends on the previous step's result, which make it particularly hard to parallelize/vectorize. > As far as I know, I came across this which points to MD5 SSE/AVX implementation. https://software.intel.com/content/www/us/en/develop/articles/intel-isa-l-cryptographic-hashes-for-cloud-storage.html That library points to computing many MD5 hashes in parallel. Quoting: "Intel? ISA-L uses a novel technique called multi-buffer hashing, which [...] compute several hashes at once within a single core." That is similar to what I found in researching how to vectorize MD5. I also did not find any reference of an ISA-level implementation of MD5, neither in x86 nor ARM. If you can point me to a document describing how to vectorize MD5, I would be more than happy to take a look and implement the algorithm. However, my understanding is that MD5 is not vectorizable by-design. > Add tests to verify intrinsic implementation. You can use test/hotspot/jtreg/compiler/intrinsics/sha/ as examples. I looked at these tests and they already cover MD5. I am not sure what's the best way to add tests here: 1. should I rename ` compiler/intrinsics/sha` to ` compiler/intrinsics/digest` and add the md5 tests there, 2. should I just add ` compiler/intrinsics/md5`, or 3. the name doesn't matter and I can just add it in ` compiler/intrinsics/sha`? > In vm_version_x86.cpp move UseMD5Intrinsics flag setting near UseSHA flag setting. Fixed. > In new file macroAssembler_x86_md5.cpp no need empty line after copyright line. There is also typo 'rrdistribute': > > * This code is free software; you can rrdistribute it and/or modify it > > Our validate-headers check failed. See GPL header template: ./make/templates/gpl-header I updated the header, and added the license for the original code for the MD5 core algorithm. > Did you test it on 32-bit x86? I did run `hotspot:tier1` and `jdk:tier1` on Windows-x86, Windows-x64 and Linux-x64. > Would be interesting to see result of artificially switching off AVX and SSE: > '-XX:UseSSE=0 -XX:UseAVX=0'. It will make sure that only general instructions are needed. The results are below: -XX:-UseMD5Intrinsics Benchmark (digesterName) (length) (provider) Mode Cnt Score Error Units MessageDigests.digest md5 64 DEFAULT thrpt 10 3512.618 ? 9.384 ops/ms MessageDigests.digest md5 1024 DEFAULT thrpt 10 450.037 ? 1.213 ops/ms MessageDigests.digest md5 16384 DEFAULT thrpt 10 29.887 ? 0.057 ops/ms MessageDigests.digest md5 1048576 DEFAULT thrpt 10 0.485 ? 0.002 ops/ms -XX:+UseMD5Intrinsics Benchmark (digesterName) (length) (provider) Mode Cnt Score Error Units MessageDigests.digest md5 64 DEFAULT thrpt 10 4212.156 ? 7.781 ops/ ms => 19% speedup MessageDigests.digest md5 1024 DEFAULT thrpt 10 548.609 ? 1.374 ops/ ms => 22% speedup MessageDigests.digest md5 16384 DEFAULT thrpt 10 37.961 ? 0.079 ops/ ms => 27% speedup MessageDigests.digest md5 1048576 DEFAULT thrpt 10 0.596 ? 0.006 ops/ ms => 23% speedup -XX:-UseMD5Intrinsics -XX:UseSSE=0 -XX:UseAVX=0 Benchmark (digesterName) (length) (provider) Mode Cnt Score Error Units MessageDigests.digest md5 64 DEFAULT thrpt 10 3462.769 ? 4.992 ops/ms MessageDigests.digest md5 1024 DEFAULT thrpt 10 443.858 ? 0.576 ops/ms MessageDigests.digest md5 16384 DEFAULT thrpt 10 29.723 ? 0.480 ops/ms MessageDigests.digest md5 1048576 DEFAULT thrpt 10 0.470 ? 0.001 ops/ms -XX:+UseMD5Intrinsics -XX:UseSSE=0 -XX:UseAVX=0 Benchmark (digesterName) (length) (provider) Mode Cnt Score Error Units MessageDigests.digest md5 64 DEFAULT thrpt 10 4237.219 ? 15.627 ops/ms => 22% speedup MessageDigests.digest md5 1024 DEFAULT thrpt 10 564.625 ? 1.510 ops/ms => 27% speedup MessageDigests.digest md5 16384 DEFAULT thrpt 10 38.004 ? 0.078 ops/ms => 28% speedup MessageDigests.digest md5 1048576 DEFAULT thrpt 10 0.597 ? 0.002 ops/ms => 27% speedup Thank you, Ludovic From viv.desh at gmail.com Tue Aug 4 04:39:54 2020 From: viv.desh at gmail.com (Vivek Deshpande) Date: Mon, 3 Aug 2020 21:39:54 -0700 Subject: RFR[M]: Adding MD5 Intrinsic on x86-64 In-Reply-To: References: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> <159bfdfb-6476-0826-8cf7-145202c7ce33@oracle.com> Message-ID: Thanks Ludovic Detailed explanation and Sandhya for clarification on the vectorization. Regards, Vivek On Mon, Aug 3, 2020 at 9:07 PM Ludovic Henry wrote: > Updated webrev: http://cr.openjdk.java.net/~luhenry/8250902/webrev.02 > > > Next code in inline_digestBase_implCompressMB should be reversed > (get_long_*() should be called for long_state): > > > > if (long_state) { > > state = get_state_from_digestBase_object(digestBase_obj); > > } else { > > state = get_long_state_from_digestBase_object(digestBase_obj); > > } > > Thanks for pointing that out. I tested everything with `hotspot:tier1` and > `jdk:tier1` in fastdebug on Windows-x86, Windows-x64 and Linux-x64. > > > It seems that the algorithm can be optimized further using SSE/AVX > instructions. I am not aware of any specific SSE/AVX implementation which > leverages those instructions in the best possible way. Sandhya can chime in > more on that. > > I have done some research prior to implementing this intrinsic and the > only pointers I could find to vectorized MD5 is on computing _multiple_ MD5 > hashes in parallel but not a _single_ MD5 hash. Using vectors effectively > parallelize the computation of many MD5 hash, but it does not accelerate > the computation of a single MD5 hash. And looking at the algorithm, every > step depends on the previous step's result, which make it particularly hard > to parallelize/vectorize. > > > As far as I know, I came across this which points to MD5 SSE/AVX > implementation. > https://software.intel.com/content/www/us/en/develop/articles/intel-isa-l-cryptographic-hashes-for-cloud-storage.html > > That library points to computing many MD5 hashes in parallel. Quoting: > "Intel? ISA-L uses a novel technique called multi-buffer hashing, which > [...] compute several hashes at once within a single core." That is similar > to what I found in researching how to vectorize MD5. I also did not find > any reference of an ISA-level implementation of MD5, neither in x86 nor ARM. > > If you can point me to a document describing how to vectorize MD5, I would > be more than happy to take a look and implement the algorithm. However, my > understanding is that MD5 is not vectorizable by-design. > > > Add tests to verify intrinsic implementation. You can use > test/hotspot/jtreg/compiler/intrinsics/sha/ as examples. > > I looked at these tests and they already cover MD5. I am not sure what's > the best way to add tests here: 1. should I rename ` > compiler/intrinsics/sha` to ` compiler/intrinsics/digest` and add the md5 > tests there, 2. should I just add ` compiler/intrinsics/md5`, or 3. the > name doesn't matter and I can just add it in ` compiler/intrinsics/sha`? > > > In vm_version_x86.cpp move UseMD5Intrinsics flag setting near UseSHA > flag setting. > > Fixed. > > > In new file macroAssembler_x86_md5.cpp no need empty line after > copyright line. There is also typo 'rrdistribute': > > > > * This code is free software; you can rrdistribute it and/or modify it > > > > Our validate-headers check failed. See GPL header template: > ./make/templates/gpl-header > > I updated the header, and added the license for the original code for the > MD5 core algorithm. > > > Did you test it on 32-bit x86? > > I did run `hotspot:tier1` and `jdk:tier1` on Windows-x86, Windows-x64 and > Linux-x64. > > > Would be interesting to see result of artificially switching off AVX and > SSE: > > '-XX:UseSSE=0 -XX:UseAVX=0'. It will make sure that only general > instructions are needed. > > The results are below: > > -XX:-UseMD5Intrinsics > Benchmark (digesterName) (length) (provider) Mode Cnt > Score Error Units > MessageDigests.digest md5 64 DEFAULT thrpt 10 > 3512.618 ? 9.384 ops/ms > MessageDigests.digest md5 1024 DEFAULT thrpt 10 > 450.037 ? 1.213 ops/ms > MessageDigests.digest md5 16384 DEFAULT thrpt 10 > 29.887 ? 0.057 ops/ms > MessageDigests.digest md5 1048576 DEFAULT thrpt 10 > 0.485 ? 0.002 ops/ms > > -XX:+UseMD5Intrinsics > Benchmark (digesterName) (length) (provider) Mode Cnt > Score Error Units > MessageDigests.digest md5 64 DEFAULT thrpt 10 > 4212.156 ? 7.781 ops/ ms => 19% speedup > MessageDigests.digest md5 1024 DEFAULT thrpt 10 > 548.609 ? 1.374 ops/ ms => 22% speedup > MessageDigests.digest md5 16384 DEFAULT thrpt 10 > 37.961 ? 0.079 ops/ ms => 27% speedup > MessageDigests.digest md5 1048576 DEFAULT thrpt 10 > 0.596 ? 0.006 ops/ ms => 23% speedup > > -XX:-UseMD5Intrinsics -XX:UseSSE=0 -XX:UseAVX=0 > Benchmark (digesterName) (length) (provider) Mode Cnt > Score Error Units > MessageDigests.digest md5 64 DEFAULT thrpt 10 > 3462.769 ? 4.992 ops/ms > MessageDigests.digest md5 1024 DEFAULT thrpt 10 > 443.858 ? 0.576 ops/ms > MessageDigests.digest md5 16384 DEFAULT thrpt 10 > 29.723 ? 0.480 ops/ms > MessageDigests.digest md5 1048576 DEFAULT thrpt 10 > 0.470 ? 0.001 ops/ms > > -XX:+UseMD5Intrinsics -XX:UseSSE=0 -XX:UseAVX=0 > Benchmark (digesterName) (length) (provider) Mode Cnt > Score Error Units > MessageDigests.digest md5 64 DEFAULT thrpt 10 > 4237.219 ? 15.627 ops/ms => 22% speedup > MessageDigests.digest md5 1024 DEFAULT thrpt 10 > 564.625 ? 1.510 ops/ms => 27% speedup > MessageDigests.digest md5 16384 DEFAULT thrpt 10 > 38.004 ? 0.078 ops/ms => 28% speedup > MessageDigests.digest md5 1048576 DEFAULT thrpt 10 > 0.597 ? 0.002 ops/ms => 27% speedup > > Thank you, > Ludovic > -- Thanks and Regards, Vivek Deshpande viv.desh at gmail.com From alexander.matveev at oracle.com Tue Aug 4 05:05:39 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Mon, 3 Aug 2020 22:05:39 -0700 Subject: RFR: 8250646: hdiutil detach fix JDK-8245311 still fails sometimes In-Reply-To: <8703f438-dbe8-9ffb-4d4c-dbaa8bfc9e21@oracle.com> References: <7ff275fd-90f6-05c7-2979-4622daeb82e4@oracle.com> <76d775dd-fc4b-16e8-9517-cb9a0623509a@oracle.com> <8703f438-dbe8-9ffb-4d4c-dbaa8bfc9e21@oracle.com> Message-ID: <748c6b6f-9a7d-8694-afd9-cd8a9795d856@oracle.com> Hi Alexey, http://cr.openjdk.java.net/~almatvee/8250646/webrev.02/ I removed stop method. Also, if we called abort from MacDMGBundler exception will be ignored. Thanks, Alexander On 8/3/20 10:10 AM, Alexey Semenyuk wrote: > > > On 7/31/2020 7:36 PM, alexander.matveev at oracle.com wrote: >> Hi Alexey, >> >> Without modifying some logic in Executor, yes stop is needed. After >> retryExecutor.stop(); is executed in MacDmgBundler we will still run >> command (same with abort), and this command will fail and exception >> will be thrown. Stop() is basically like abort, but exception is >> ignored. > Right, but can't we just have one method that will abort the loop? Why > not test for the value of `aborted` field not only in exception > handler, but also before running executor? I mean instead of > --- > if (stop) { > ??? break; > } > --- > have > --- > if (aborted) { > ??? break; > } > --- > > This would eliminate the need to introduce stop() method. What do you > think? > > - Alexey >> >> Thanks, >> Alexander >> >> On 7/31/20 9:27 AM, Alexey Semenyuk wrote: >>> Alexander, >>> >>> Why introduce RetryExecutor.stop() method? Isn't >>> RetryExecutor.abort() sufficient? >>> >>> - Alexey >>> >>> On 7/30/2020 8:10 PM, alexander.matveev at oracle.com wrote: >>>> Please review the jpackage fix for bug [1] at [2]. >>>> >>>> - DMG image can get detach even if "hdiutil detach" return resource >>>> busy error. Fixed by stopping trying detach if DMG already got detach. >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8250646 >>>> [2] http://cr.openjdk.java.net/~almatvee/8250646/webrev.00/ >>>> >>>> Thanks, >>>> Alexander >>> >> > From vipinmv1 at in.ibm.com Tue Aug 4 09:51:00 2020 From: vipinmv1 at in.ibm.com (Vipin Mv1) Date: Tue, 4 Aug 2020 09:51:00 +0000 Subject: RFR 6470126 java.util.jar.Attributes#containsKey fails with Strings In-Reply-To: References: , , Message-ID: Hi, Since, the use of strings and Attributes.Name is distributed over APIs, I have come up with a generic documentation. However any suggestions are welcome. --- a/src/java.base/share/classes/java/util/jar/Attributes.java Thu Mar 26 15:26:51 2020 +0000 +++ b/src/java.base/share/classes/java/util/jar/Attributes.java Tue Aug 04 14:55:46 2020 +0530 @@ -49,7 +49,11 @@ * Attribute values can contain any characters and * will be UTF8-encoded when written to the output stream. See the * JAR File Specification - * for more information about valid attribute names and values. + * for more information about valid attribute names and values. While + * insertion, the attribute name (key) can be specified as a string or any + * objects of type Attribute.Name or its subclass. But during retrieval, + * attribute name (key) specified as a string is discouraged to avoid + * unpredictable results. * *

    This map and its views have a predictable iteration order, namely the * order that keys were inserted into the map, as with {@link LinkedHashMap}. @@ -96,12 +100,32 @@ Thanks & Regards Vipin MV -----Vipin Mv1/India/IBM wrote: ----- To: Lance Andersen From: Vipin Mv1/India/IBM Date: 07/13/2020 07:13PM Cc: core-libs-dev at openjdk.java.net Subject: Re: [EXTERNAL] Re: RFR 6470126 java.util.jar.Attributes#containsKey fails with Strings Hi Lance Thanks for the detailed review. I will go ahead and look into the required documentation changes. Thanks & Regards Vipin MV -----Lance Andersen wrote: ----- To: Vipin Mv1 From: Lance Andersen Date: 07/10/2020 02:27AM Cc: core-libs-dev at openjdk.java.net Subject: [EXTERNAL] Re: RFR 6470126 java.util.jar.Attributes#containsKey fails with Strings Hi Vipin, Apologies for the delay. After looking at the bug, which is over 14 years old, the SCCS history of Attributes.java, I am reluctant to suggest we move forward with your proposed change. The key for an Attributes map entry should be an Attributes.Name object (see Attributes::put). Unfortunately your proposed fix introduces a behavioral change and could possibly break existing applications. A behavioral change to existing public methods would require approval via a CSR and would require more compressive testing. I took a quick scan of the JCK tests and of the JTReg tests and I believe your change would cause some of the existing tests to fail. >From my perspective, it would be better to clarify the Attributes javadoc to make it clearer that an Attributes.Name object is required (which I believe has not changed since the Attributes class was added to Java SE). Best Lance On Jul 1, 2020, at 12:42 AM, Vipin Mv1 wrote: Hi, A gentle reminder to please review this patch. Thanks & Regards Vipin MV -----Vipin Mv1/India/IBM wrote: ----- To: core-libs-dev at openjdk.java.net From: Vipin Mv1/India/IBM Date: 06/15/2020 11:52AM Subject: Re: RFR 6470126 java.util.jar.Attributes#containsKey fails with Strings Hi, I have addressed the review comments and the patch has been uploaded here: http://cr.openjdk.java.net/~vtewari/6470126/webrev/index.html Please let me know your suggestions. Thanks & Regards Vipin MV -----Vipin Mv1/India/IBM wrote: ----- To: core-libs-dev at openjdk.java.net From: Vipin Mv1/India/IBM Date: 05/11/2020 05:00PM Subject: RFR 6470126 java.util.jar.Attributes#containsKey fails with Strings Hi, Please review the fix for the following issue. https://bugs.openjdk.java.net/browse/JDK-6470126 diff -r 53568400fec3 src/java.base/share/classes/java/util/jar/Attributes.java --- a/src/java.base/share/classes/java/util/jar/Attributes.java Thu Mar 26 15:26:51 2020 +0000 +++ b/src/java.base/share/classes/java/util/jar/Attributes.java Mon May 11 15:00:01 2020 +0530 @@ -205,7 +205,10 @@ * @return true if this Map contains the specified attribute name */ public boolean containsKey(Object name) { - return map.containsKey(name); + if(String.class.isInstance(name)) + return map.containsKey(Name.of((String)name)); + else + return map.containsKey(name); } /** Thanks & Regards Vipin Menon Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From andy.herrick at oracle.com Tue Aug 4 11:51:08 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Tue, 4 Aug 2020 07:51:08 -0400 Subject: RFR: 8250646: hdiutil detach fix JDK-8245311 still fails sometimes In-Reply-To: <748c6b6f-9a7d-8694-afd9-cd8a9795d856@oracle.com> References: <7ff275fd-90f6-05c7-2979-4622daeb82e4@oracle.com> <76d775dd-fc4b-16e8-9517-cb9a0623509a@oracle.com> <8703f438-dbe8-9ffb-4d4c-dbaa8bfc9e21@oracle.com> <748c6b6f-9a7d-8694-afd9-cd8a9795d856@oracle.com> Message-ID: looks good /Andy On 8/4/2020 1:05 AM, alexander.matveev at oracle.com wrote: > Hi Alexey, > > http://cr.openjdk.java.net/~almatvee/8250646/webrev.02/ > > I removed stop method. Also, if we called abort from MacDMGBundler > exception will be ignored. > > Thanks, > Alexander > > On 8/3/20 10:10 AM, Alexey Semenyuk wrote: >> >> >> On 7/31/2020 7:36 PM, alexander.matveev at oracle.com wrote: >>> Hi Alexey, >>> >>> Without modifying some logic in Executor, yes stop is needed. After >>> retryExecutor.stop(); is executed in MacDmgBundler we will still run >>> command (same with abort), and this command will fail and exception >>> will be thrown. Stop() is basically like abort, but exception is >>> ignored. >> Right, but can't we just have one method that will abort the loop? >> Why not test for the value of `aborted` field not only in exception >> handler, but also before running executor? I mean instead of >> --- >> if (stop) { >> ??? break; >> } >> --- >> have >> --- >> if (aborted) { >> ??? break; >> } >> --- >> >> This would eliminate the need to introduce stop() method. What do you >> think? >> >> - Alexey >>> >>> Thanks, >>> Alexander >>> >>> On 7/31/20 9:27 AM, Alexey Semenyuk wrote: >>>> Alexander, >>>> >>>> Why introduce RetryExecutor.stop() method? Isn't >>>> RetryExecutor.abort() sufficient? >>>> >>>> - Alexey >>>> >>>> On 7/30/2020 8:10 PM, alexander.matveev at oracle.com wrote: >>>>> Please review the jpackage fix for bug [1] at [2]. >>>>> >>>>> - DMG image can get detach even if "hdiutil detach" return >>>>> resource busy error. Fixed by stopping trying detach if DMG >>>>> already got detach. >>>>> >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8250646 >>>>> [2] http://cr.openjdk.java.net/~almatvee/8250646/webrev.00/ >>>>> >>>>> Thanks, >>>>> Alexander >>>> >>> >> > From brian.burkhalter at oracle.com Tue Aug 4 16:01:30 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 4 Aug 2020 09:01:30 -0700 Subject: [16] 8251017: java/io/File/GetXSpace.java fails on UNIX Message-ID: <361C144E-A680-4B64-A13E-292A1F075173@oracle.com> Add @requires tag for [1]. This patch is layered on top of that proposed in [2]. --- a/test/jdk/java/io/File/GetXSpace.java +++ b/test/jdk/java/io/File/GetXSpace.java @@ -24,6 +24,7 @@ /** * @test * @bug 4057701 6286712 6364377 + * @requires (os.family == "linux" | os.family == "mac" | os.family == "windows") * @run build GetXSpace * @run shell GetXSpace.sh * @summary Basic functionality of File.get-X-Space methods. Thanks, Brian [1] https://bugs.openjdk.java.net/browse/JDK-8251017 [2] http://mail.openjdk.java.net/pipermail/core-libs-dev/2020-July/067990.html From vladimir.kozlov at oracle.com Tue Aug 4 17:19:56 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 4 Aug 2020 10:19:56 -0700 Subject: RFR[M]: Adding MD5 Intrinsic on x86-64 In-Reply-To: References: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> <159bfdfb-6476-0826-8cf7-145202c7ce33@oracle.com> Message-ID: Hi Ludovic, On 8/3/20 9:07 PM, Ludovic Henry wrote: > Updated webrev: http://cr.openjdk.java.net/~luhenry/8250902/webrev.02 > >> Next code in inline_digestBase_implCompressMB should be reversed (get_long_*() should be called for long_state): >> >> if (long_state) { >> state = get_state_from_digestBase_object(digestBase_obj); >> } else { >> state = get_long_state_from_digestBase_object(digestBase_obj); >> } > > Thanks for pointing that out. I tested everything with `hotspot:tier1` and `jdk:tier1` in fastdebug on Windows-x86, Windows-x64 and Linux-x64. Code in library_call.cpp is good now. > >> It seems that the algorithm can be optimized further using SSE/AVX instructions. I am not aware of any specific SSE/AVX implementation which leverages those instructions in the best possible way. Sandhya can chime in more on that. > > I have done some research prior to implementing this intrinsic and the only pointers I could find to vectorized MD5 is on computing _multiple_ MD5 hashes in parallel but not a _single_ MD5 hash. Using vectors effectively parallelize the computation of many MD5 hash, but it does not accelerate the computation of a single MD5 hash. And looking at the algorithm, every step depends on the previous step's result, which make it particularly hard to parallelize/vectorize. > >> As far as I know, I came across this which points to MD5 SSE/AVX implementation. https://software.intel.com/content/www/us/en/develop/articles/intel-isa-l-cryptographic-hashes-for-cloud-storage.html > > That library points to computing many MD5 hashes in parallel. Quoting: "Intel? ISA-L uses a novel technique called multi-buffer hashing, which [...] compute several hashes at once within a single core." That is similar to what I found in researching how to vectorize MD5. I also did not find any reference of an ISA-level implementation of MD5, neither in x86 nor ARM. > > If you can point me to a document describing how to vectorize MD5, I would be more than happy to take a look and implement the algorithm. However, my understanding is that MD5 is not vectorizable by-design. I would leave this investigation to Intel's Java group. They are expert in this area! For now, lets put current implementation into JDK. > >> Add tests to verify intrinsic implementation. You can use test/hotspot/jtreg/compiler/intrinsics/sha/ as examples. > > I looked at these tests and they already cover MD5. I am not sure what's the best way to add tests here: 1. should I rename ` compiler/intrinsics/sha` to ` compiler/intrinsics/digest` and add the md5 tests there, 2. should I just add ` compiler/intrinsics/md5`, or 3. the name doesn't matter and I can just add it in ` compiler/intrinsics/sha`? 3. Just add MD5 tests into existing SHA directory. Note, compiler/intrinsics/sha testing is done in tier2. I ran it and it passed but it does not test MD5 a lot as I understand. > >> In vm_version_x86.cpp move UseMD5Intrinsics flag setting near UseSHA flag setting. > > Fixed. It is not moved in webrev.02 > >> In new file macroAssembler_x86_md5.cpp no need empty line after copyright line. There is also typo 'rrdistribute': >> >> * This code is free software; you can rrdistribute it and/or modify it >> >> Our validate-headers check failed. See GPL header template: ./make/templates/gpl-header > > I updated the header, and added the license for the original code for the MD5 core algorithm. You don't need to use Oracle copyright line. Using original Microsoft's copyright line is fine since you are author. Thank you for adding license for original code. > >> Did you test it on 32-bit x86? > > I did run `hotspot:tier1` and `jdk:tier1` on Windows-x86, Windows-x64 and Linux-x64. > >> Would be interesting to see result of artificially switching off AVX and SSE: >> '-XX:UseSSE=0 -XX:UseAVX=0'. It will make sure that only general instructions are needed. > > The results are below: Very good. Thank you for testing it. Regards, Vladimir > > -XX:-UseMD5Intrinsics > Benchmark (digesterName) (length) (provider) Mode Cnt Score Error Units > MessageDigests.digest md5 64 DEFAULT thrpt 10 3512.618 ? 9.384 ops/ms > MessageDigests.digest md5 1024 DEFAULT thrpt 10 450.037 ? 1.213 ops/ms > MessageDigests.digest md5 16384 DEFAULT thrpt 10 29.887 ? 0.057 ops/ms > MessageDigests.digest md5 1048576 DEFAULT thrpt 10 0.485 ? 0.002 ops/ms > > -XX:+UseMD5Intrinsics > Benchmark (digesterName) (length) (provider) Mode Cnt Score Error Units > MessageDigests.digest md5 64 DEFAULT thrpt 10 4212.156 ? 7.781 ops/ ms => 19% speedup > MessageDigests.digest md5 1024 DEFAULT thrpt 10 548.609 ? 1.374 ops/ ms => 22% speedup > MessageDigests.digest md5 16384 DEFAULT thrpt 10 37.961 ? 0.079 ops/ ms => 27% speedup > MessageDigests.digest md5 1048576 DEFAULT thrpt 10 0.596 ? 0.006 ops/ ms => 23% speedup > > -XX:-UseMD5Intrinsics -XX:UseSSE=0 -XX:UseAVX=0 > Benchmark (digesterName) (length) (provider) Mode Cnt Score Error Units > MessageDigests.digest md5 64 DEFAULT thrpt 10 3462.769 ? 4.992 ops/ms > MessageDigests.digest md5 1024 DEFAULT thrpt 10 443.858 ? 0.576 ops/ms > MessageDigests.digest md5 16384 DEFAULT thrpt 10 29.723 ? 0.480 ops/ms > MessageDigests.digest md5 1048576 DEFAULT thrpt 10 0.470 ? 0.001 ops/ms > > -XX:+UseMD5Intrinsics -XX:UseSSE=0 -XX:UseAVX=0 > Benchmark (digesterName) (length) (provider) Mode Cnt Score Error Units > MessageDigests.digest md5 64 DEFAULT thrpt 10 4237.219 ? 15.627 ops/ms => 22% speedup > MessageDigests.digest md5 1024 DEFAULT thrpt 10 564.625 ? 1.510 ops/ms => 27% speedup > MessageDigests.digest md5 16384 DEFAULT thrpt 10 38.004 ? 0.078 ops/ms => 28% speedup > MessageDigests.digest md5 1048576 DEFAULT thrpt 10 0.597 ? 0.002 ops/ms => 27% speedup > > Thank you, > Ludovic > From info at j-kuhn.de Tue Aug 4 17:35:00 2020 From: info at j-kuhn.de (Johannes Kuhn) Date: Tue, 4 Aug 2020 19:35:00 +0200 Subject: Can sun.misc.Unsafe.throwException() be removed? Message-ID: <38efcdba-b4ab-3c57-0291-090149a45cdb@j-kuhn.de> From my understanding, Unsafe.throwException throws the passed Throwable, more commonly known as "snaky throws". There are other ways to archive the same result, one commonly found is by (ab)using generics: ??? public static void sneakyThrows(Throwable t) throws T { ??????? throw (T) t; ??? } (This will generate an unchecked warning, but that can be silenced with the right annotation.) Is there any reason why this method still needs to exist? If not, could it be removed? - Johannes From naoto.sato at oracle.com Tue Aug 4 17:41:03 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Tue, 4 Aug 2020 10:41:03 -0700 Subject: [16] 8251017: java/io/File/GetXSpace.java fails on UNIX In-Reply-To: <361C144E-A680-4B64-A13E-292A1F075173@oracle.com> References: <361C144E-A680-4B64-A13E-292A1F075173@oracle.com> Message-ID: Looks good, Brian. Naoto On 8/4/20 9:01 AM, Brian Burkhalter wrote: > Add @requires tag for [1]. This patch is layered on top of that proposed in [2]. > > --- a/test/jdk/java/io/File/GetXSpace.java > +++ b/test/jdk/java/io/File/GetXSpace.java > @@ -24,6 +24,7 @@ > /** > * @test > * @bug 4057701 6286712 6364377 > + * @requires (os.family == "linux" | os.family == "mac" | os.family == "windows") > * @run build GetXSpace > * @run shell GetXSpace.sh > * @summary Basic functionality of File.get-X-Space methods. > > Thanks, > > Brian > > [1] https://bugs.openjdk.java.net/browse/JDK-8251017 > [2] http://mail.openjdk.java.net/pipermail/core-libs-dev/2020-July/067990.html > From brian.burkhalter at oracle.com Tue Aug 4 18:09:18 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 4 Aug 2020 11:09:18 -0700 Subject: [16] 8251017: java/io/File/GetXSpace.java fails on UNIX In-Reply-To: References: <361C144E-A680-4B64-A13E-292A1F075173@oracle.com> Message-ID: <5D1AF455-226C-43E6-A6A8-3C6A28A07196@oracle.com> Thanks, Naoto. I?ll hold off pushing it until the review in [2] is done. Brian > On Aug 4, 2020, at 10:41 AM, naoto.sato at oracle.com wrote: > > Looks good, Brian. > > Naoto > > On 8/4/20 9:01 AM, Brian Burkhalter wrote: >> Add @requires tag for [1]. This patch is layered on top of that proposed in [2]. >> --- a/test/jdk/java/io/File/GetXSpace.java >> +++ b/test/jdk/java/io/File/GetXSpace.java >> @@ -24,6 +24,7 @@ >> /** >> * @test >> * @bug 4057701 6286712 6364377 >> + * @requires (os.family == "linux" | os.family == "mac" | os.family == "windows") >> * @run build GetXSpace >> * @run shell GetXSpace.sh >> * @summary Basic functionality of File.get-X-Space methods. >> Thanks, >> Brian >> [1] https://bugs.openjdk.java.net/browse/JDK-8251017 >> [2] http://mail.openjdk.java.net/pipermail/core-libs-dev/2020-July/067990.html From alexey.semenyuk at oracle.com Tue Aug 4 18:16:48 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Tue, 4 Aug 2020 14:16:48 -0400 Subject: RFR: 8250646: hdiutil detach fix JDK-8245311 still fails sometimes In-Reply-To: <748c6b6f-9a7d-8694-afd9-cd8a9795d856@oracle.com> References: <7ff275fd-90f6-05c7-2979-4622daeb82e4@oracle.com> <76d775dd-fc4b-16e8-9517-cb9a0623509a@oracle.com> <8703f438-dbe8-9ffb-4d4c-dbaa8bfc9e21@oracle.com> <748c6b6f-9a7d-8694-afd9-cd8a9795d856@oracle.com> Message-ID: <1f28b7bd-6170-1351-b6b5-05e21b6a2490@oracle.com> Looks good. - Alexey On 8/4/2020 1:05 AM, alexander.matveev at oracle.com wrote: > Hi Alexey, > > http://cr.openjdk.java.net/~almatvee/8250646/webrev.02/ > > I removed stop method. Also, if we called abort from MacDMGBundler > exception will be ignored. > > Thanks, > Alexander > > On 8/3/20 10:10 AM, Alexey Semenyuk wrote: >> >> >> On 7/31/2020 7:36 PM, alexander.matveev at oracle.com wrote: >>> Hi Alexey, >>> >>> Without modifying some logic in Executor, yes stop is needed. After >>> retryExecutor.stop(); is executed in MacDmgBundler we will still run >>> command (same with abort), and this command will fail and exception >>> will be thrown. Stop() is basically like abort, but exception is >>> ignored. >> Right, but can't we just have one method that will abort the loop? >> Why not test for the value of `aborted` field not only in exception >> handler, but also before running executor? I mean instead of >> --- >> if (stop) { >> ??? break; >> } >> --- >> have >> --- >> if (aborted) { >> ??? break; >> } >> --- >> >> This would eliminate the need to introduce stop() method. What do you >> think? >> >> - Alexey >>> >>> Thanks, >>> Alexander >>> >>> On 7/31/20 9:27 AM, Alexey Semenyuk wrote: >>>> Alexander, >>>> >>>> Why introduce RetryExecutor.stop() method? Isn't >>>> RetryExecutor.abort() sufficient? >>>> >>>> - Alexey >>>> >>>> On 7/30/2020 8:10 PM, alexander.matveev at oracle.com wrote: >>>>> Please review the jpackage fix for bug [1] at [2]. >>>>> >>>>> - DMG image can get detach even if "hdiutil detach" return >>>>> resource busy error. Fixed by stopping trying detach if DMG >>>>> already got detach. >>>>> >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8250646 >>>>> [2] http://cr.openjdk.java.net/~almatvee/8250646/webrev.00/ >>>>> >>>>> Thanks, >>>>> Alexander >>>> >>> >> > From luhenry at microsoft.com Tue Aug 4 20:21:02 2020 From: luhenry at microsoft.com (Ludovic Henry) Date: Tue, 4 Aug 2020 20:21:02 +0000 Subject: RFR[M]: Adding MD5 Intrinsic on x86-64 In-Reply-To: References: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> <159bfdfb-6476-0826-8cf7-145202c7ce33@oracle.com> Message-ID: Webrev: http://cr.openjdk.java.net/~luhenry/8250902/webrev.03 Testing: everything passes with hotspot:tier1 and jdk:tier1 in fastdebug on Linux-x64. > I would leave this investigation to Intel's Java group. They are expert in this area! Ok, we'll reach out to Intel on our end as well to figure out whether they have any specific guidance on that. > 3. Just add MD5 tests into existing SHA directory. Done. I've done some small renames (TestSHA -> TestDigest, SHAOptionsBase -> DigestOptionsBase), modified some of the SHA-specific code for non-SHA cases (GenericTestCaseFor*.java), and added MD5-specific tests. > Note, compiler/intrinsics/sha testing is done in tier2. I ran it and it passed but it does not test MD5 a lot as I understand. I extended the existing tests to cover MD5 on the same level as SHA, and I made sure that all tests are still passing. >> >>> In vm_version_x86.cpp move UseMD5Intrinsics flag setting near UseSHA flag setting. >> >> Fixed. > > It is not moved in webrev.02 Fixed. > You don't need to use Oracle copyright line. Using original Microsoft's copyright line is fine since you are author. Fixed. From vladimir.kozlov at oracle.com Tue Aug 4 22:03:38 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 4 Aug 2020 15:03:38 -0700 Subject: RFR[M]: Adding MD5 Intrinsic on x86-64 In-Reply-To: References: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> <159bfdfb-6476-0826-8cf7-145202c7ce33@oracle.com> Message-ID: <410fb009-94ab-fea8-9c1c-51c835b27b72@oracle.com> Good. I will run Hotspot and JDK testing and let you know results. Regards, Vladimir K On 8/4/20 1:21 PM, Ludovic Henry wrote: > Webrev: http://cr.openjdk.java.net/~luhenry/8250902/webrev.03 > Testing: everything passes with hotspot:tier1 and jdk:tier1 in fastdebug on Linux-x64. > >> I would leave this investigation to Intel's Java group. They are expert in this area! > > Ok, we'll reach out to Intel on our end as well to figure out whether they have any specific guidance on that. > >> 3. Just add MD5 tests into existing SHA directory. > > Done. I've done some small renames (TestSHA -> TestDigest, SHAOptionsBase -> DigestOptionsBase), modified some of the SHA-specific code for non-SHA cases (GenericTestCaseFor*.java), and added MD5-specific tests. > >> Note, compiler/intrinsics/sha testing is done in tier2. I ran it and it passed but it does not test MD5 a lot as I understand. > > I extended the existing tests to cover MD5 on the same level as SHA, and I made sure that all tests are still passing. > >>> >>>> In vm_version_x86.cpp move UseMD5Intrinsics flag setting near UseSHA flag setting. >>> >>> Fixed. >> >> It is not moved in webrev.02 > > Fixed. > >> You don't need to use Oracle copyright line. Using original Microsoft's copyright line is fine since you are author. > > Fixed. > > From vladimir.kozlov at oracle.com Wed Aug 5 01:33:09 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 4 Aug 2020 18:33:09 -0700 Subject: RFR[M]: Adding MD5 Intrinsic on x86-64 In-Reply-To: <410fb009-94ab-fea8-9c1c-51c835b27b72@oracle.com> References: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> <159bfdfb-6476-0826-8cf7-145202c7ce33@oracle.com> <410fb009-94ab-fea8-9c1c-51c835b27b72@oracle.com> Message-ID: <832a89ec-bd6b-5d40-9a1d-5a5e688399e7@oracle.com> Hi Ludovic, Tests are mostly clean so far except: new 3 MD5 tests failed on aarch64 because UseMD5Intrinsics flag is 'true' incorrectly: bool UseMD5Intrinsics = true {diagnostic} {command line} compiler/intrinsics/sha/sanity/TestMD5MultiBlockIntrinsics.java compiler/intrinsics/sha/sanity/TestMD5Intrinsics.java compiler/intrinsics/sha/cli/TestUseMD5IntrinsicsOptionOnUnsupportedCPU.java I think you need to set flag to false (to overwrite setting on command line) in vm_version_*.cpp files on all other CPUs until they have implementation: http://hg.openjdk.java.net/jdk/jdk/file/5bda40c115c1/src/hotspot/cpu/ppc/vm_version_ppc.cpp#l278 Also I forgot to ask to update copyright year in files you touched. Thanks, Vladimir K On 8/4/20 3:03 PM, Vladimir Kozlov wrote: > Good. > > I will run Hotspot and JDK testing and let you know results. > > Regards, > Vladimir K > > On 8/4/20 1:21 PM, Ludovic Henry wrote: >> Webrev: http://cr.openjdk.java.net/~luhenry/8250902/webrev.03 >> Testing: everything passes with hotspot:tier1 and jdk:tier1 in fastdebug on Linux-x64. >> >>> I would leave this investigation to Intel's Java group. They are expert in this area! >> >> Ok, we'll reach out to Intel on our end as well to figure out whether they have any specific guidance on that. >> >>> 3. Just add MD5 tests into existing SHA directory. >> >> Done. I've done some small renames (TestSHA -> TestDigest, SHAOptionsBase -> DigestOptionsBase), modified some of the >> SHA-specific code for non-SHA cases (GenericTestCaseFor*.java), and added MD5-specific tests. >> >>> Note, compiler/intrinsics/sha testing is done in tier2. I ran it and it passed but it does not test MD5 a lot as I >>> understand. >> >> I extended the existing tests to cover MD5 on the same level as SHA, and I made sure that all tests are still passing. >> >>>> >>>>> In vm_version_x86.cpp move UseMD5Intrinsics flag setting near UseSHA flag setting. >>>> >>>> Fixed. >>> >>> It is not moved in webrev.02 >> >> Fixed. >> >>> You don't need to use Oracle copyright line. Using original Microsoft's copyright line is fine since you are author. >> >> Fixed. >> From luhenry at microsoft.com Wed Aug 5 02:09:08 2020 From: luhenry at microsoft.com (Ludovic Henry) Date: Wed, 5 Aug 2020 02:09:08 +0000 Subject: RFR[M]: Adding MD5 Intrinsic on x86-64 In-Reply-To: <832a89ec-bd6b-5d40-9a1d-5a5e688399e7@oracle.com> References: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> <159bfdfb-6476-0826-8cf7-145202c7ce33@oracle.com> <410fb009-94ab-fea8-9c1c-51c835b27b72@oracle.com> <832a89ec-bd6b-5d40-9a1d-5a5e688399e7@oracle.com> Message-ID: Webrev: http://cr.openjdk.java.net/~luhenry/8250902/webrev.03 > I think you need to set flag to false (to overwrite setting on command line) in vm_version_*.cpp files on all other CPUs until they have implementation: Fixed. > Also I forgot to ask to update copyright year in files you touched. Fixed. From joe.darcy at oracle.com Wed Aug 5 02:42:55 2020 From: joe.darcy at oracle.com (Joe Darcy) Date: Tue, 4 Aug 2020 19:42:55 -0700 Subject: JDK 16 RFR of JDK-8250920: Increase @jls usage in core reflection Message-ID: <239de370-4df5-6687-f126-52af2d3c4ade@oracle.com> Hello, While JDK-8250660: "Clarify that WildcardType and AnnotatedWildcardType bounds methods return one" is still under discussion, please review a change for ??? JDK-8250920: Increase @jls usage in core reflection ??? http://cr.openjdk.java.net/~darcy/8250920.0/ which adds @jls tags for the type-modeling components of the java.lang.reflect package. Thanks, -Joe From mandy.chung at oracle.com Wed Aug 5 02:48:54 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 4 Aug 2020 19:48:54 -0700 Subject: JDK 16 RFR of JDK-8250920: Increase @jls usage in core reflection In-Reply-To: <239de370-4df5-6687-f126-52af2d3c4ade@oracle.com> References: <239de370-4df5-6687-f126-52af2d3c4ade@oracle.com> Message-ID: <5c77e342-8624-7507-0ddb-62fcd4ee0821@oracle.com> +1 Mandy On 8/4/20 7:42 PM, Joe Darcy wrote: > Hello, > > While JDK-8250660: "Clarify that WildcardType and > AnnotatedWildcardType bounds methods return one" is still under > discussion, please review a change for > > ??? JDK-8250920: Increase @jls usage in core reflection > ??? http://cr.openjdk.java.net/~darcy/8250920.0/ > > which adds @jls tags for the type-modeling components of the > java.lang.reflect package. > > Thanks, > > -Joe > From vladimir.kozlov at oracle.com Wed Aug 5 04:36:07 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Tue, 4 Aug 2020 21:36:07 -0700 Subject: RFR[M]: Adding MD5 Intrinsic on x86-64 In-Reply-To: References: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> <159bfdfb-6476-0826-8cf7-145202c7ce33@oracle.com> <410fb009-94ab-fea8-9c1c-51c835b27b72@oracle.com> <832a89ec-bd6b-5d40-9a1d-5a5e688399e7@oracle.com> Message-ID: It looks like you created webrev based on old state of jdk or its branch. Your vm_version_aarch64.cpp change did not apply to latest jdk source. There are also few copyright year updates for files which already have it. I fixed it and start new round of testing. Vladimir K On 8/4/20 7:09 PM, Ludovic Henry wrote: > Webrev: http://cr.openjdk.java.net/~luhenry/8250902/webrev.03 > >> I think you need to set flag to false (to overwrite setting on command line) in vm_version_*.cpp files on all other CPUs until they have implementation: > > Fixed. > > > Also I forgot to ask to update copyright year in files you touched. > > Fixed. > From raffaello.giulietti at gmail.com Wed Aug 5 10:44:39 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Wed, 5 Aug 2020 12:44:39 +0200 Subject: Reminder RFR: 8245036: DataInputStream.readFully(byte[], int, int) does not throw expected IndexOutOfBoundsExceptions In-Reply-To: <858a70ac-f827-669a-58e2-e5b834aa36f4@gmail.com> References: <59a871a4-d676-3b0c-2983-b9b9fc34ca37@gmail.com> <858a70ac-f827-669a-58e2-e5b834aa36f4@gmail.com> Message-ID: <09cda74a-2cf3-0534-95c2-54cb22696c27@gmail.com> Hi, a gentle reminder to review this almost trivial patch. Greetings Raffaello On 2020-07-29 19:44, Raffaello Giulietti wrote: > Hi Tagir, > > here's yesterday's patch with added unit tests. > > > Greetings > Raffaello > > > > On 2020-07-29 05:26, Tagir Valeev wrote: >> Hello! >> >> Having a unit-test would be nice! >> >> With best regards, >> Tagir Valeev. >> >> On Tue, Jul 28, 2020 at 9:04 PM Raffaello Giulietti >> wrote: >>> >>> Hello, >>> >>> here's a very simple patch to address issue 8245036. >>> HTH >>> >>> >>> Greetings >>> Raffaello > > > > > ---- > > # HG changeset patch > # User lello > # Date 1596043874 -7200 > #????? Wed Jul 29 19:31:14 2020 +0200 > # Node ID b9d417f53ddaae3801585b285f6eb8ba1606af8b > # Parent? 89fe9e02a522b57c356f0dd5279085a075c2945b > Patch to fix JDK-8245036 > 8245036: DataInputStream.readFully(byte[], int, int) does not throw > expected IndexOutOfBoundsExceptions > Reviewed-by: TBD > Contributed-by: Raffaello Giulietti > > diff --git a/src/java.base/share/classes/java/io/DataInputStream.java > b/src/java.base/share/classes/java/io/DataInputStream.java > --- a/src/java.base/share/classes/java/io/DataInputStream.java > +++ b/src/java.base/share/classes/java/io/DataInputStream.java > @@ -1,5 +1,5 @@ > ?/* > - * Copyright (c) 1994, 2019, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 1994, 2020, Oracle and/or its affiliates. All rights > reserved. > ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > ? * > ? * This code is free software; you can redistribute it and/or modify it > @@ -25,6 +25,8 @@ > > ?package java.io; > > +import java.util.Objects; > + > ?/** > ? * A data input stream lets an application read primitive Java data > ? * types from an underlying input stream in a machine-independent > @@ -192,8 +194,7 @@ > ????? * @see??????? java.io.FilterInputStream#in > ????? */ > ???? public final void readFully(byte b[], int off, int len) throws > IOException { > -??????? if (len < 0) > -??????????? throw new IndexOutOfBoundsException(); > +??????? Objects.checkFromIndexSize(off, len, b.length); > ???????? int n = 0; > ???????? while (n < len) { > ???????????? int count = in.read(b, off + n, len - n); > diff --git a/test/jdk/java/io/DataInputStream/ReadFully.java > b/test/jdk/java/io/DataInputStream/ReadFully.java > --- a/test/jdk/java/io/DataInputStream/ReadFully.java > +++ b/test/jdk/java/io/DataInputStream/ReadFully.java > @@ -1,5 +1,5 @@ > ?/* > - * Copyright (c) 1999, 2010, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 1999, 2020, Oracle and/or its affiliates. All rights > reserved. > ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > ? * > ? * This code is free software; you can redistribute it and/or modify it > @@ -22,15 +22,36 @@ > ? */ > > ?/* @test > - * @bug 4214513 > - * @summary Passing a negative length argument for readFully must throw > - *????????? IndexOutOfBoundsException. > + * @bug 4214513 8245036 > + * @summary Passing a negative offset or length, > + *????????? or passing a combination of offset and length too big > + *????????? for readFully must throw IndexOutOfBoundsException. > ? */ > > +import java.io.*; > > -import java.io.*; > ?public class ReadFully { > -??? public static final void main(String[] args) throws Exception { > + > +??? private static final void testNegativeOffset() throws Exception { > +??????? byte[] buffer = new byte[100]; > +??????? File file = new File(System.getProperty("test.src"), > +??????????????? "ReadFully.java"); > +??????? FileInputStream in = new FileInputStream(file); > +??????? DataInputStream dis = new DataInputStream(in); > + > +??????? boolean caughtException = false; > +??????? try { > +??????????? dis.readFully(buffer, -1, buffer.length); > +??????? } catch (IndexOutOfBoundsException ie) { > +??????????? caughtException = true; > +??????? } finally { > +??????????? dis.close(); > +??????????? if (!caughtException) > +??????????????? throw new RuntimeException("Test testNegativeOffset() > failed"); > +??????? } > +??? } > + > +??? private static final void testNegativeLength() throws Exception { > ???????? byte[] buffer = new byte[100]; > ???????? File file = new File(System.getProperty("test.src"), > ???????????????? "ReadFully.java"); > @@ -45,7 +66,53 @@ > ???????? } finally { > ???????????? dis.close(); > ???????????? if (!caughtException) > -??????????????? throw new RuntimeException("Test failed"); > +??????????????? throw new RuntimeException("Test testNegativeLength() > failed"); > +??????? } > +??? } > + > +??? private static final void testBigOffsetLength1() throws Exception { > +??????? byte[] buffer = new byte[100]; > +??????? File file = new File(System.getProperty("test.src"), > +??????????????? "ReadFully.java"); > +??????? FileInputStream in = new FileInputStream(file); > +??????? DataInputStream dis = new DataInputStream(in); > + > +??????? boolean caughtException = false; > +??????? try { > +??????????? dis.readFully(buffer, 0, buffer.length + 1); > +??????? } catch (IndexOutOfBoundsException ie) { > +??????????? caughtException = true; > +??????? } finally { > +??????????? dis.close(); > +??????????? if (!caughtException) > +??????????????? throw new RuntimeException("Test testBigOffsetLength1() > failed"); > ???????? } > ???? } > + > +??? private static final void testBigOffsetLength2() throws Exception { > +??????? byte[] buffer = new byte[100]; > +??????? File file = new File(System.getProperty("test.src"), > +??????????????? "ReadFully.java"); > +??????? FileInputStream in = new FileInputStream(file); > +??????? DataInputStream dis = new DataInputStream(in); > + > +??????? boolean caughtException = false; > +??????? try { > +??????????? dis.readFully(buffer, buffer.length, 1); > +??????? } catch (IndexOutOfBoundsException ie) { > +??????????? caughtException = true; > +??????? } finally { > +??????????? dis.close(); > +??????????? if (!caughtException) > +??????????????? throw new RuntimeException("Test testBigOffsetLength2() > failed"); > +??????? } > +??? } > + > +??? public static final void main(String[] args) throws Exception { > +??????? testNegativeOffset(); > +??????? testNegativeLength(); > +??????? testBigOffsetLength1(); > +??????? testBigOffsetLength2(); > +??? } > + > ?} > From brian.burkhalter at oracle.com Wed Aug 5 16:20:08 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 5 Aug 2020 09:20:08 -0700 Subject: RFR: 8245036: DataInputStream.readFully(byte[], int, int) does not throw expected IndexOutOfBoundsExceptions In-Reply-To: <858a70ac-f827-669a-58e2-e5b834aa36f4@gmail.com> References: <59a871a4-d676-3b0c-2983-b9b9fc34ca37@gmail.com> <858a70ac-f827-669a-58e2-e5b834aa36f4@gmail.com> Message-ID: HI Raffaello, The implementation test looks right. In the test, you might consider using try-with-resources. Thanks, Brian > On Jul 29, 2020, at 10:44 AM, Raffaello Giulietti wrote: > > + FileInputStream in = new FileInputStream(file); > + DataInputStream dis = new DataInputStream(in); > + > + boolean caughtException = false; > + try { > + dis.readFully(buffer, -1, buffer.length); > + } catch (IndexOutOfBoundsException ie) { > + caughtException = true; > + } finally { > + dis.close(); > + if (!caughtException) > + throw new RuntimeException("Test testNegativeOffset() failed"); > + } From brian.burkhalter at oracle.com Wed Aug 5 16:24:36 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 5 Aug 2020 09:24:36 -0700 Subject: RFR: 8245036: DataInputStream.readFully(byte[], int, int) does not throw expected IndexOutOfBoundsExceptions In-Reply-To: References: <59a871a4-d676-3b0c-2983-b9b9fc34ca37@gmail.com> <858a70ac-f827-669a-58e2-e5b834aa36f4@gmail.com> Message-ID: <67C3F744-2FE7-4BDD-857F-86E64A1AABFA@oracle.com> Also, instead of using the boolean ?caughtException,? we usually just put, e.g., the throwing of the RuntimeException on the line after where the IIOBE or whatever would be thrown. I don?t think the ?finally? block is necessary. Thanks, Brian > On Aug 5, 2020, at 9:20 AM, Brian Burkhalter wrote: > > HI Raffaello, > > The implementation test looks right. In the test, you might consider using try-with-resources. > > Thanks, > > Brian > >> On Jul 29, 2020, at 10:44 AM, Raffaello Giulietti > wrote: >> >> + FileInputStream in = new FileInputStream(file); >> + DataInputStream dis = new DataInputStream(in); >> + >> + boolean caughtException = false; >> + try { >> + dis.readFully(buffer, -1, buffer.length); >> + } catch (IndexOutOfBoundsException ie) { >> + caughtException = true; >> + } finally { >> + dis.close(); >> + if (!caughtException) >> + throw new RuntimeException("Test testNegativeOffset() failed"); >> + } From mandy.chung at oracle.com Wed Aug 5 17:26:11 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 5 Aug 2020 10:26:11 -0700 Subject: RFR JDK-8250929: Missing "classpath exception" in LambdaProxyClassArchive.java Message-ID: Trivial update on license header. diff --git a/src/java.base/share/classes/java/lang/invoke/LambdaProxyClassArchive.java b/src/java.base/share/classes/java/lang/invoke/LambdaProxyClassArchive.java --- a/src/java.base/share/classes/java/lang/invoke/LambdaProxyClassArchive.java +++ b/src/java.base/share/classes/java/lang/invoke/LambdaProxyClassArchive.java @@ -4,7 +4,9 @@ ? * ? * This code is free software; you can redistribute it and/or modify it ? * under the terms of the GNU General Public License version 2 only, as - * published by the Free Software Foundation. + * published by the Free Software Foundation.? Oracle designates this + * particular file as subject to the "Classpath" exception as provided + * by Oracle in the LICENSE file that accompanied this code. ? * ? * This code is distributed in the hope that it will be useful, but WITHOUT ? * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or thanks Mandy From sundararajan.athijegannathan at oracle.com Wed Aug 5 17:33:00 2020 From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com) Date: Wed, 5 Aug 2020 23:03:00 +0530 Subject: RFR JDK-8250929: Missing "classpath exception" in LambdaProxyClassArchive.java In-Reply-To: References: Message-ID: <703817a2-8ba7-da4e-fbf6-d5e5b361556b@oracle.com> Looks good -Sundar On 05/08/20 10:56 pm, Mandy Chung wrote: > Trivial update on license header. > > diff --git > a/src/java.base/share/classes/java/lang/invoke/LambdaProxyClassArchive.java > b/src/java.base/share/classes/java/lang/invoke/LambdaProxyClassArchive.java > > --- > a/src/java.base/share/classes/java/lang/invoke/LambdaProxyClassArchive.java > +++ > b/src/java.base/share/classes/java/lang/invoke/LambdaProxyClassArchive.java > @@ -4,7 +4,9 @@ > ? * > ? * This code is free software; you can redistribute it and/or modify it > ? * under the terms of the GNU General Public License version 2 only, as > - * published by the Free Software Foundation. > + * published by the Free Software Foundation.? Oracle designates this > + * particular file as subject to the "Classpath" exception as provided > + * by Oracle in the LICENSE file that accompanied this code. > ? * > ? * This code is distributed in the hope that it will be useful, but > WITHOUT > ? * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or > > > thanks > Mandy From brian.burkhalter at oracle.com Wed Aug 5 18:36:54 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 5 Aug 2020 11:36:54 -0700 Subject: 8181919: Refactor test/java/io/File/GetXSpace.sh to java test Message-ID: <711DD0AF-CAB7-4DE2-9495-3D27B1F3A0A3@oracle.com> To fix [1], please consider the patch [2]. This follows on from the proposed change [3] which in turn follows from [4]. Thanks, Brian [1] https://bugs.openjdk.java.net/browse/JDK-8181919 [2] http://cr.openjdk.java.net/~bpb/8181919/webrev.00/ [3] https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068025.html (GetXSpace should not run on AIX) [4] http://mail.openjdk.java.net/pipermail/core-libs-dev/2020-July/067990.html (GetXSpace fails on macOS) From andy.herrick at oracle.com Wed Aug 5 20:17:04 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Wed, 5 Aug 2020 16:17:04 -0400 Subject: RFR: JDK-8251184: File association without description causes exception Message-ID: <795856b5-3996-311b-7c5b-e92eb08b6f32@oracle.com> Please review this jpackage fix at [1] to issue [2] [1] - http://cr.openjdk.java.net/~herrick/8251184/ [2] - https://bugs.openjdk.java.net/browse/JDK-8251184 /Andy From joe.darcy at oracle.com Wed Aug 5 21:37:05 2020 From: joe.darcy at oracle.com (Joe Darcy) Date: Wed, 5 Aug 2020 14:37:05 -0700 Subject: JDK 16 RFR of JDK-8250660: Clarify that WildcardType and AnnotatedWildcardType bounds methods return one In-Reply-To: <99628559-a8c4-28c1-9e34-c01bdc8517c6@oracle.com> References: <9808e420-39a8-2045-9db1-629a551f82a0@oracle.com> <63af16f1-ddd3-7e56-5392-891e6328fc25@oracle.com> <99628559-a8c4-28c1-9e34-c01bdc8517c6@oracle.com> Message-ID: Hello, Addressing some feedback from off-list discussions with Mandy, I propose changing the API notes to: ??? @apiNote While to date a wildcard may have at most one upper[lower] bound, callers of this method should be written to accommodate multiple bounds. Thanks, -Joe On 8/3/2020 9:24 AM, Joe Darcy wrote: > Hi David, > > To provide a bit more context, the API was consciously designed to be > future-proofed against the possibility of Java supporting multiple > bounds here (JDK-4891872). As of yet, that change has not come to > pass, and as you note may never come to pass. So besides in addition > to the links into the JLS, I think it is reasonable to include an API > note acknowledging the situation. How about: > > ??? This method supports the existence of multiple bounds, but to > date, a wildcard may have at most one upper/lower bound. > > -Joe > > From mandy.chung at oracle.com Wed Aug 5 21:41:52 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 5 Aug 2020 14:41:52 -0700 Subject: JDK 16 RFR of JDK-8250660: Clarify that WildcardType and AnnotatedWildcardType bounds methods return one In-Reply-To: References: <9808e420-39a8-2045-9db1-629a551f82a0@oracle.com> <63af16f1-ddd3-7e56-5392-891e6328fc25@oracle.com> <99628559-a8c4-28c1-9e34-c01bdc8517c6@oracle.com> Message-ID: <9c0a063d-74eb-0ec4-6d72-a84ec0a19618@oracle.com> This looks good.? This will prepare for the future when Java supports multiple bounds. Mandy On 8/5/20 2:37 PM, Joe Darcy wrote: > Hello, > > Addressing some feedback from off-list discussions with Mandy, I > propose changing the API notes to: > > ??? @apiNote While to date a wildcard may have at most one > upper[lower] bound, callers of this method should be written to > accommodate multiple bounds. > > Thanks, > > -Joe > > On 8/3/2020 9:24 AM, Joe Darcy wrote: >> Hi David, >> >> To provide a bit more context, the API was consciously designed to be >> future-proofed against the possibility of Java supporting multiple >> bounds here (JDK-4891872). As of yet, that change has not come to >> pass, and as you note may never come to pass. So besides in addition >> to the links into the JLS, I think it is reasonable to include an API >> note acknowledging the situation. How about: >> >> ??? This method supports the existence of multiple bounds, but to >> date, a wildcard may have at most one upper/lower bound. >> >> -Joe >> >> From david.holmes at oracle.com Wed Aug 5 22:40:11 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 6 Aug 2020 08:40:11 +1000 Subject: JDK 16 RFR of JDK-8250660: Clarify that WildcardType and AnnotatedWildcardType bounds methods return one In-Reply-To: References: <9808e420-39a8-2045-9db1-629a551f82a0@oracle.com> <63af16f1-ddd3-7e56-5392-891e6328fc25@oracle.com> <99628559-a8c4-28c1-9e34-c01bdc8517c6@oracle.com> Message-ID: <2e528db3-acd9-7d63-ab99-1546eeaf2efe@oracle.com> On 6/08/2020 7:37 am, Joe Darcy wrote: > Hello, > > Addressing some feedback from off-list discussions with Mandy, I propose > changing the API notes to: > > ??? @apiNote While to date a wildcard may have at most one upper[lower] > bound, callers of this method should be written to accommodate multiple > bounds. I can live with that version. :) Someone will still need to remember to change it if/when we get multiple bounds. Cheers, David > Thanks, > > -Joe > > On 8/3/2020 9:24 AM, Joe Darcy wrote: >> Hi David, >> >> To provide a bit more context, the API was consciously designed to be >> future-proofed against the possibility of Java supporting multiple >> bounds here (JDK-4891872). As of yet, that change has not come to >> pass, and as you note may never come to pass. So besides in addition >> to the links into the JLS, I think it is reasonable to include an API >> note acknowledging the situation. How about: >> >> ??? This method supports the existence of multiple bounds, but to >> date, a wildcard may have at most one upper/lower bound. >> >> -Joe >> >> From huizhe.wang at oracle.com Wed Aug 5 23:37:52 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Wed, 5 Aug 2020 16:37:52 -0700 Subject: RFR [16/java.xml] 8246816: XMLGregorianCalendar.hashCode() produces far too many identical hashes Message-ID: <058367db-dca5-76e9-7e61-049173c765d9@oracle.com> Hello, Please review a fix for the hashCode generation. JBS: https://bugs.openjdk.java.net/browse/JDK-8246816 webrev: http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev/ Thanks, Joe From naoto.sato at oracle.com Thu Aug 6 01:13:58 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Wed, 5 Aug 2020 18:13:58 -0700 Subject: RFR [16/java.xml] 8246816: XMLGregorianCalendar.hashCode() produces far too many identical hashes In-Reply-To: <058367db-dca5-76e9-7e61-049173c765d9@oracle.com> References: <058367db-dca5-76e9-7e61-049173c765d9@oracle.com> Message-ID: Hi Joe, For the consistency with equals(), just wondering if the sub-second element should be obtained with getFractionalSecond() instead of getMillisecond(), as the equals() subsequently calls it in internalCompare() method. Also should it call getEonAndYear() appropriately for the year? Naoto On 8/5/20 4:37 PM, Joe Wang wrote: > Hello, > > Please review a fix for the hashCode generation. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8246816 > > webrev: http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev/ > > Thanks, > Joe > From alexander.matveev at oracle.com Thu Aug 6 03:26:33 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Wed, 5 Aug 2020 20:26:33 -0700 Subject: RFR: JDK-8251184: File association without description causes exception In-Reply-To: <795856b5-3996-311b-7c5b-e92eb08b6f32@oracle.com> References: <795856b5-3996-311b-7c5b-e92eb08b6f32@oracle.com> Message-ID: <38b850b0-54c2-9ca3-9425-253aafd724a2@oracle.com> Hi Andy, Looks good. Thanks, Alexander On 8/5/20 1:17 PM, Andy Herrick wrote: > Please review this jpackage fix at [1] to issue [2] > > [1] - http://cr.openjdk.java.net/~herrick/8251184/ > > [2] - https://bugs.openjdk.java.net/browse/JDK-8251184 > > /Andy > From luhenry at microsoft.com Thu Aug 6 04:36:07 2020 From: luhenry at microsoft.com (Ludovic Henry) Date: Thu, 6 Aug 2020 04:36:07 +0000 Subject: RFR[M]: Adding MD5 Intrinsic on x86-64 In-Reply-To: References: <061fb2de-563b-7a55-1b14-d8c6b6b4e0f4@oracle.com> <159bfdfb-6476-0826-8cf7-145202c7ce33@oracle.com> <410fb009-94ab-fea8-9c1c-51c835b27b72@oracle.com> <832a89ec-bd6b-5d40-9a1d-5a5e688399e7@oracle.com> Message-ID: Pushed with https://hg.openjdk.java.net/jdk/jdk/rev/b8231f177eaf Thank you to all involved ?? From joe.darcy at oracle.com Thu Aug 6 06:47:15 2020 From: joe.darcy at oracle.com (Joe Darcy) Date: Wed, 5 Aug 2020 23:47:15 -0700 Subject: JDK 16 RFR of JDK-5017980: Allow signed positive integer to be parsed instead of a NumerFormatException Message-ID: Hello, Please review the patch below and CSR for ??? JDK-8249273: Documentation of BigInteger(String) constructor does not mention leading plus ??? CSR: https://bugs.openjdk.java.net/browse/JDK-8251235 The fix for JDK-5017980 way back in JDK 7 omitted the spec update to one of the two BigInteger constructors that accept string arguments. The fix is to add the missing "or plus" text. The regression tests already probe for the kind of values in question. Thanks, -Joe (I'll reflow the paragraph before pushing.) diff -r 7a3522ab48b3 src/java.base/share/classes/java/math/BigInteger.java --- a/src/java.base/share/classes/java/math/BigInteger.java??? Wed Aug 05 23:02:22 2020 -0700 +++ b/src/java.base/share/classes/java/math/BigInteger.java??? Wed Aug 05 23:18:11 2020 -0700 @@ -659,7 +659,7 @@ ???? /** ????? * Translates the decimal String representation of a BigInteger into a -???? * BigInteger.? The String representation consists of an optional minus +???? * BigInteger.? The String representation consists of an optional minus or plus ????? * sign followed by a sequence of one or more decimal digits.? The ????? * character-to-digit mapping is provided by {@code Character.digit}. ????? * The String may not contain any extraneous characters (whitespace, for From aleksej.efimov at oracle.com Thu Aug 6 12:44:17 2020 From: aleksej.efimov at oracle.com (Aleks Efimov) Date: Thu, 6 Aug 2020 13:44:17 +0100 Subject: RFR[testbug]: 8251189: com/sun/jndi/ldap/LdapDnsProviderTest.java failed due to timeout Message-ID: Hi, LdapDnsProviderTest was seen failing due to the timeout caused by the blocked bind operation. That could happen if there is a local process listening on the port specified in the test URL. To make LdapProviderTest.ProviderTest.call fail the LDAP connect timeout property needs to be set to greater than 0 value. That will help runLocalHostTestWithRandomPort to fail with the timeout exception if the port is busy and try another random port. Also, the test output has been modified not to print stack traces for the expected exceptions. Webrev: http://cr.openjdk.java.net/~aefimov/8251189/00/index.html JBS: https://bugs.openjdk.java.net/browse/JDK-8251189 The test was executed 100+ times alongside to other LDAP tests and was not seen failing with the ?proposed changes. With Best Regards, Aleksei From Roger.Riggs at oracle.com Thu Aug 6 13:41:08 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Thu, 6 Aug 2020 09:41:08 -0400 Subject: RFR [16/java.xml] 8246816: XMLGregorianCalendar.hashCode() produces far too many identical hashes In-Reply-To: <058367db-dca5-76e9-7e61-049173c765d9@oracle.com> References: <058367db-dca5-76e9-7e61-049173c765d9@oracle.com> Message-ID: Hi Joe, Looks fine. Roger On 8/5/20 7:37 PM, Joe Wang wrote: > Hello, > > Please review a fix for the hashCode generation. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8246816 > > webrev: http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev/ > > Thanks, > Joe > From sundararajan.athijegannathan at oracle.com Thu Aug 6 15:03:15 2020 From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com) Date: Thu, 6 Aug 2020 20:33:15 +0530 Subject: RFR (JDK 15) 8248299: two jdeps files miss copyright header Message-ID: <1defa1d1-dbc9-b708-270c-cec7436ff7e6@oracle.com> Please review. Bug: https://bugs.openjdk.java.net/browse/JDK-8248299 Webrev: http://cr.openjdk.java.net/~sundar/8248299/webrev.00/ Thanks, -Sundar From sundararajan.athijegannathan at oracle.com Thu Aug 6 15:12:27 2020 From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com) Date: Thu, 6 Aug 2020 20:42:27 +0530 Subject: RFR (JDK 15) 8248299: two jdeps files miss copyright header In-Reply-To: <1defa1d1-dbc9-b708-270c-cec7436ff7e6@oracle.com> References: <1defa1d1-dbc9-b708-270c-cec7436ff7e6@oracle.com> Message-ID: <1b25f4ca-f40e-e225-2fa3-5a1b0032946d@oracle.com> Oops. Missed "Classpath exception" clause. Updated: http://cr.openjdk.java.net/~sundar/8248299/webrev.01/ -Sundar On 06/08/20 8:33 pm, sundararajan.athijegannathan at oracle.com wrote: > Please review. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8248299 > > Webrev: http://cr.openjdk.java.net/~sundar/8248299/webrev.00/ > > Thanks, > > -Sundar > From brian.burkhalter at oracle.com Thu Aug 6 15:22:02 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 6 Aug 2020 08:22:02 -0700 Subject: JDK 16 RFR of JDK-5017980: Allow signed positive integer to be parsed instead of a NumerFormatException In-Reply-To: References: Message-ID: Hi Joe, CSR reviewed; patch looks fine. Brian > On Aug 5, 2020, at 11:47 PM, Joe Darcy wrote: > > - * BigInteger. The String representation consists of an optional minus > + * BigInteger. The String representation consists of an optional minus or plus From mandy.chung at oracle.com Thu Aug 6 16:34:25 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 6 Aug 2020 09:34:25 -0700 Subject: RFR (JDK 15) 8248299: two jdeps files miss copyright header In-Reply-To: <1defa1d1-dbc9-b708-270c-cec7436ff7e6@oracle.com> References: <1defa1d1-dbc9-b708-270c-cec7436ff7e6@oracle.com> Message-ID: +1 Mandy On 8/6/20 8:03 AM, sundararajan.athijegannathan at oracle.com wrote: > Please review. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8248299 > > Webrev: http://cr.openjdk.java.net/~sundar/8248299/webrev.00/ > > Thanks, > > -Sundar > From li.jiang at oracle.com Thu Aug 6 16:47:12 2020 From: li.jiang at oracle.com (li.jiang at oracle.com) Date: Fri, 7 Aug 2020 00:47:12 +0800 Subject: RFR [16]: 8250665 : Wrong translation for the month of May in ar_JO, ar_LB and ar_SY Message-ID: <8766969e-72af-d937-0368-14d2958e1244@oracle.com> Hi, Pls review the fix for month names in COMPAT provider data for ar_JO/LB/SY. Customer reported some month names are incorrect in JDK8 code according to latest CLDR definition. Bug: https://bugs.openjdk.java.net/browse/JDK-8250665 Webrev: http://cr.openjdk.java.net/~ljiang/8250665/webrev.00/ Passed the tier1 and tier2 test cases. Thanks, Leo From alexey.semenyuk at oracle.com Thu Aug 6 17:40:27 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Thu, 6 Aug 2020 13:40:27 -0400 Subject: RFR: JDK-8251184: File association without description causes exception In-Reply-To: <795856b5-3996-311b-7c5b-e92eb08b6f32@oracle.com> References: <795856b5-3996-311b-7c5b-e92eb08b6f32@oracle.com> Message-ID: <097669ca-ed63-a40a-7b83-ec449fab00f2@oracle.com> Looks good. - Alexey On 8/5/2020 4:17 PM, Andy Herrick wrote: > Please review this jpackage fix at [1] to issue [2] > > [1] - http://cr.openjdk.java.net/~herrick/8251184/ > > [2] - https://bugs.openjdk.java.net/browse/JDK-8251184 > > /Andy > From mandy.chung at oracle.com Thu Aug 6 18:00:00 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 6 Aug 2020 11:00:00 -0700 Subject: RFR (JDK 15) 8248299: two jdeps files miss copyright header In-Reply-To: References: <1defa1d1-dbc9-b708-270c-cec7436ff7e6@oracle.com> Message-ID: <35ba5a17-75b0-c99c-b92c-97cb3a5451a2@oracle.com> To clarify, I actually reviewed webrev.01. Mandy On 8/6/20 9:34 AM, Mandy Chung wrote: > +1 > > Mandy > > On 8/6/20 8:03 AM, sundararajan.athijegannathan at oracle.com wrote: >> Please review. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8248299 >> >> Webrev: http://cr.openjdk.java.net/~sundar/8248299/webrev.00/ >> >> Thanks, >> >> -Sundar >> > From naoto.sato at oracle.com Thu Aug 6 18:18:26 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Thu, 6 Aug 2020 11:18:26 -0700 Subject: RFR [16]: 8250665 : Wrong translation for the month of May in ar_JO, ar_LB and ar_SY In-Reply-To: <8766969e-72af-d937-0368-14d2958e1244@oracle.com> References: <8766969e-72af-d937-0368-14d2958e1244@oracle.com> Message-ID: Looks good to me. Naoto On 8/6/20 9:47 AM, li.jiang at oracle.com wrote: > Hi, > > Pls review the fix for month names in COMPAT provider data for ar_JO/LB/SY. > > Customer reported some month names are incorrect in JDK8 code according > to latest CLDR definition. > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8250665 > > Webrev: > http://cr.openjdk.java.net/~ljiang/8250665/webrev.00/ > > Passed the tier1 and tier2 test cases. > > Thanks, > Leo > From raffaello.giulietti at gmail.com Thu Aug 6 18:34:02 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Thu, 6 Aug 2020 20:34:02 +0200 Subject: RFR: 8245036: DataInputStream.readFully(byte[], int, int) does not throw expected IndexOutOfBoundsExceptions In-Reply-To: <67C3F744-2FE7-4BDD-857F-86E64A1AABFA@oracle.com> References: <59a871a4-d676-3b0c-2983-b9b9fc34ca37@gmail.com> <858a70ac-f827-669a-58e2-e5b834aa36f4@gmail.com> <67C3F744-2FE7-4BDD-857F-86E64A1AABFA@oracle.com> Message-ID: Hi, here's the newest patch. > Also, instead of using the boolean ?caughtException,? we usually just > put, e.g., the throwing of the RuntimeException on the line after where > the IIOBE or whatever would be thrown. I don?t think the ?finally? block > is necessary. > > Thanks, > > Brian Greetings Raffaello ---- # HG changeset patch # User lello # Date 1596738630 -7200 # Thu Aug 06 20:30:30 2020 +0200 # Node ID ad676c9dcb57c8cfa141a22cf5ee85e0fe363336 # Parent b01985b4f88f554f97901e53e1ba314681dd9c19 Patch to fix JDK-8245036 8245036: DataInputStream.readFully(byte[], int, int) does not throw expected IndexOutOfBoundsExceptions Reviewed-by: TBD Contributed-by: Raffaello Giulietti diff --git a/src/java.base/share/classes/java/io/DataInputStream.java b/src/java.base/share/classes/java/io/DataInputStream.java --- a/src/java.base/share/classes/java/io/DataInputStream.java +++ b/src/java.base/share/classes/java/io/DataInputStream.java @@ -1,5 +1,5 @@ /* - * Copyright (c) 1994, 2019, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1994, 2020, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -25,6 +25,8 @@ package java.io; +import java.util.Objects; + /** * A data input stream lets an application read primitive Java data * types from an underlying input stream in a machine-independent @@ -192,8 +194,7 @@ * @see java.io.FilterInputStream#in */ public final void readFully(byte b[], int off, int len) throws IOException { - if (len < 0) - throw new IndexOutOfBoundsException(); + Objects.checkFromIndexSize(off, len, b.length); int n = 0; while (n < len) { int count = in.read(b, off + n, len - n); diff --git a/test/jdk/java/io/DataInputStream/ReadFully.java b/test/jdk/java/io/DataInputStream/ReadFully.java --- a/test/jdk/java/io/DataInputStream/ReadFully.java +++ b/test/jdk/java/io/DataInputStream/ReadFully.java @@ -1,5 +1,5 @@ /* - * Copyright (c) 1999, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1999, 2020, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -22,30 +22,73 @@ */ /* @test - * @bug 4214513 - * @summary Passing a negative length argument for readFully must throw - * IndexOutOfBoundsException. + * @bug 4214513 8245036 + * @summary Passing a negative offset or length, + * or passing a combination of offset and length too big + * for readFully must throw IndexOutOfBoundsException. */ +import java.io.*; -import java.io.*; public class ReadFully { - public static final void main(String[] args) throws Exception { - byte[] buffer = new byte[100]; + + private static final void testNegativeOffset() throws Exception { + File file = new File(System.getProperty("test.src"), + "ReadFully.java"); + try (FileInputStream in = new FileInputStream(file); + DataInputStream dis = new DataInputStream(in);) { + byte[] buffer = new byte[100]; + dis.readFully(buffer, -1, buffer.length); + } catch (IndexOutOfBoundsException ignored) { + return; + } + throw new RuntimeException("Test testNegativeOffset() failed"); + } + + private static final void testNegativeLength() throws Exception { File file = new File(System.getProperty("test.src"), "ReadFully.java"); - FileInputStream in = new FileInputStream(file); - DataInputStream dis = new DataInputStream(in); + try (FileInputStream in = new FileInputStream(file); + DataInputStream dis = new DataInputStream(in);) { + byte[] buffer = new byte[100]; + dis.readFully(buffer, 0, -1); + } catch (IndexOutOfBoundsException ignored) { + return; + } + throw new RuntimeException("Test testNegativeLength() failed"); + } - boolean caughtException = false; - try { - dis.readFully(buffer, 0, -20); - } catch (IndexOutOfBoundsException ie) { - caughtException = true; - } finally { - dis.close(); - if (!caughtException) - throw new RuntimeException("Test failed"); + private static final void testBigOffsetLength1() throws Exception { + File file = new File(System.getProperty("test.src"), + "ReadFully.java"); + try (FileInputStream in = new FileInputStream(file); + DataInputStream dis = new DataInputStream(in);) { + byte[] buffer = new byte[100]; + dis.readFully(buffer, 0, buffer.length + 1); + } catch (IndexOutOfBoundsException ignored) { + return; } + throw new RuntimeException("Test testBigOffsetLength1() failed"); } + + private static final void testBigOffsetLength2() throws Exception { + File file = new File(System.getProperty("test.src"), + "ReadFully.java"); + try (FileInputStream in = new FileInputStream(file); + DataInputStream dis = new DataInputStream(in);) { + byte[] buffer = new byte[100]; + dis.readFully(buffer, buffer.length, 1); + } catch (IndexOutOfBoundsException ignored) { + return; + } + throw new RuntimeException("Test testBigOffsetLength2() failed"); + } + + public static final void main(String[] args) throws Exception { + testNegativeOffset(); + testNegativeLength(); + testBigOffsetLength1(); + testBigOffsetLength2(); + } + } -------------- next part -------------- A non-text attachment was scrubbed... Name: JDK-8245036.patch Type: text/x-patch Size: 5600 bytes Desc: not available URL: From brian.burkhalter at oracle.com Thu Aug 6 18:51:24 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 6 Aug 2020 11:51:24 -0700 Subject: RFR: 8245036: DataInputStream.readFully(byte[], int, int) does not throw expected IndexOutOfBoundsExceptions In-Reply-To: References: <59a871a4-d676-3b0c-2983-b9b9fc34ca37@gmail.com> <858a70ac-f827-669a-58e2-e5b834aa36f4@gmail.com> <67C3F744-2FE7-4BDD-857F-86E64A1AABFA@oracle.com> Message-ID: Hi Raffaello, This looks all right. I would be inclined to do like in [1] and throw the RuntimeException on the line after each readFully() and drop the return statements], but I?m not going to insist on it. Thanks, Brian [1] http://hg.openjdk.java.net/jdk/jdk/file/db6952bf8c39/test/jdk/java/nio/file/etc/MacVolumesTest.java#l58 > On Aug 6, 2020, at 11:34 AM, Raffaello Giulietti wrote: > > here's the newest patch. > > > Also, instead of using the boolean ?caughtException,? we usually just > > put, e.g., the throwing of the RuntimeException on the line after where > > the IIOBE or whatever would be thrown. I don?t think the ?finally? block > > is necessary. From huizhe.wang at oracle.com Thu Aug 6 19:18:54 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Thu, 6 Aug 2020 12:18:54 -0700 Subject: RFR [16/java.xml] 8246816: XMLGregorianCalendar.hashCode() produces far too many identical hashes In-Reply-To: References: <058367db-dca5-76e9-7e61-049173c765d9@oracle.com> Message-ID: Thanks Naoto, Roger for the review! For Naoto's concern about the equals method using EonAndYear and FractionalSecond, I did cut corners using the all int getXXX method. The rational was: it fulfills the equals-hashcode contract just fine, is consistent with the existing implementation, and concise. This API was introduced since 1.5, but I have yet to see any usage of EonAndYear, and very rarely FractionalSecond. The chances collisions occur due to these differences are very low. But I understand your concern. To be precise or exact as equals(), we would need to call getFractionalSecond and EonAndYear. Here's what that would look like: http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_02/ To me, I like version 1 for the reasons above: http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_01/ What would you think? Regards, Joe On 8/5/20 6:13 PM, naoto.sato at oracle.com wrote: > Hi Joe, > > For the consistency with equals(), just wondering if the sub-second > element should be obtained with getFractionalSecond() instead of > getMillisecond(), as the equals() subsequently calls it in > internalCompare() method. Also should it call getEonAndYear() > appropriately for the year? > > Naoto > > On 8/5/20 4:37 PM, Joe Wang wrote: >> Hello, >> >> Please review a fix for the hashCode generation. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8246816 >> >> webrev: http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev/ >> >> Thanks, >> Joe >> From raffaello.giulietti at gmail.com Thu Aug 6 19:36:18 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Thu, 6 Aug 2020 21:36:18 +0200 Subject: RFR: 8245036: DataInputStream.readFully(byte[], int, int) does not throw expected IndexOutOfBoundsExceptions In-Reply-To: References: <59a871a4-d676-3b0c-2983-b9b9fc34ca37@gmail.com> <858a70ac-f827-669a-58e2-e5b834aa36f4@gmail.com> <67C3F744-2FE7-4BDD-857F-86E64A1AABFA@oracle.com> Message-ID: On 2020-08-06 20:51, Brian Burkhalter wrote: > Hi Raffaello, > > This looks all right. I would be inclined to do like in [1] and throw > the RuntimeException on the line after each readFully() and drop the > return statements], but I?m not going to insist on it. > > Thanks, > > Brian > > [1] > http://hg.openjdk.java.net/jdk/jdk/file/db6952bf8c39/test/jdk/java/nio/file/etc/MacVolumesTest.java#l58 > > Sorry, I didn't completely get that. Here we are! ---- # HG changeset patch # User lello # Date 1596742420 -7200 # Thu Aug 06 21:33:40 2020 +0200 # Node ID ac2819c6e8d3528bd138e4be5f5014febc254ff3 # Parent b01985b4f88f554f97901e53e1ba314681dd9c19 Patch to fix JDK-8245036 8245036: DataInputStream.readFully(byte[], int, int) does not throw expected IndexOutOfBoundsExceptions Reviewed-by: TBD Contributed-by: Raffaello Giulietti diff --git a/src/java.base/share/classes/java/io/DataInputStream.java b/src/java.base/share/classes/java/io/DataInputStream.java --- a/src/java.base/share/classes/java/io/DataInputStream.java +++ b/src/java.base/share/classes/java/io/DataInputStream.java @@ -1,5 +1,5 @@ /* - * Copyright (c) 1994, 2019, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1994, 2020, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -25,6 +25,8 @@ package java.io; +import java.util.Objects; + /** * A data input stream lets an application read primitive Java data * types from an underlying input stream in a machine-independent @@ -192,8 +194,7 @@ * @see java.io.FilterInputStream#in */ public final void readFully(byte b[], int off, int len) throws IOException { - if (len < 0) - throw new IndexOutOfBoundsException(); + Objects.checkFromIndexSize(off, len, b.length); int n = 0; while (n < len) { int count = in.read(b, off + n, len - n); diff --git a/test/jdk/java/io/DataInputStream/ReadFully.java b/test/jdk/java/io/DataInputStream/ReadFully.java --- a/test/jdk/java/io/DataInputStream/ReadFully.java +++ b/test/jdk/java/io/DataInputStream/ReadFully.java @@ -1,5 +1,5 @@ /* - * Copyright (c) 1999, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1999, 2020, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -22,30 +22,69 @@ */ /* @test - * @bug 4214513 - * @summary Passing a negative length argument for readFully must throw - * IndexOutOfBoundsException. + * @bug 4214513 8245036 + * @summary Passing a negative offset or length, + * or passing a combination of offset and length too big + * for readFully must throw IndexOutOfBoundsException. */ +import java.io.*; -import java.io.*; public class ReadFully { - public static final void main(String[] args) throws Exception { - byte[] buffer = new byte[100]; + + private static final void testNegativeOffset() throws Exception { + File file = new File(System.getProperty("test.src"), + "ReadFully.java"); + try (FileInputStream in = new FileInputStream(file); + DataInputStream dis = new DataInputStream(in);) { + byte[] buffer = new byte[100]; + dis.readFully(buffer, -1, buffer.length); + throw new RuntimeException("Test testNegativeOffset() failed"); + } catch (IndexOutOfBoundsException ignore) { + } + } + + private static final void testNegativeLength() throws Exception { File file = new File(System.getProperty("test.src"), "ReadFully.java"); - FileInputStream in = new FileInputStream(file); - DataInputStream dis = new DataInputStream(in); + try (FileInputStream in = new FileInputStream(file); + DataInputStream dis = new DataInputStream(in);) { + byte[] buffer = new byte[100]; + dis.readFully(buffer, 0, -1); + throw new RuntimeException("Test testNegativeLength() failed"); + } catch (IndexOutOfBoundsException ignore) { + } + } - boolean caughtException = false; - try { - dis.readFully(buffer, 0, -20); - } catch (IndexOutOfBoundsException ie) { - caughtException = true; - } finally { - dis.close(); - if (!caughtException) - throw new RuntimeException("Test failed"); + private static final void testBigOffsetLength1() throws Exception { + File file = new File(System.getProperty("test.src"), + "ReadFully.java"); + try (FileInputStream in = new FileInputStream(file); + DataInputStream dis = new DataInputStream(in);) { + byte[] buffer = new byte[100]; + dis.readFully(buffer, 0, buffer.length + 1); + throw new RuntimeException("Test testBigOffsetLength1() failed"); + } catch (IndexOutOfBoundsException ignore) { } } + + private static final void testBigOffsetLength2() throws Exception { + File file = new File(System.getProperty("test.src"), + "ReadFully.java"); + try (FileInputStream in = new FileInputStream(file); + DataInputStream dis = new DataInputStream(in);) { + byte[] buffer = new byte[100]; + dis.readFully(buffer, buffer.length, 1); + throw new RuntimeException("Test testBigOffsetLength2() failed"); + } catch (IndexOutOfBoundsException ignore) { + } + } + + public static final void main(String[] args) throws Exception { + testNegativeOffset(); + testNegativeLength(); + testBigOffsetLength1(); + testBigOffsetLength2(); + } + } -------------- next part -------------- A non-text attachment was scrubbed... Name: JDK-8245036.patch Type: text/x-patch Size: 5528 bytes Desc: not available URL: From brian.burkhalter at oracle.com Thu Aug 6 20:09:10 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 6 Aug 2020 13:09:10 -0700 Subject: RFR: 8245036: DataInputStream.readFully(byte[], int, int) does not throw expected IndexOutOfBoundsExceptions In-Reply-To: References: <59a871a4-d676-3b0c-2983-b9b9fc34ca37@gmail.com> <858a70ac-f827-669a-58e2-e5b834aa36f4@gmail.com> <67C3F744-2FE7-4BDD-857F-86E64A1AABFA@oracle.com> Message-ID: <4030EF21-CCE0-48D2-ADB1-7BF7519FEA52@oracle.com> > On Aug 6, 2020, at 12:36 PM, Raffaello Giulietti wrote: > >> This looks all right. I would be inclined to do like in [1] and throw the RuntimeException on the line after each readFully() and drop the return statements], but I?m not going to insist on it. >> Thanks, >> Brian >> [1] http://hg.openjdk.java.net/jdk/jdk/file/db6952bf8c39/test/jdk/java/nio/file/etc/MacVolumesTest.java#l58 > > Sorry, I didn't completely get that. > Here we are! This looks good. I observed however that the test passes without the change to DataInputStream. Brian From naoto.sato at oracle.com Thu Aug 6 20:18:57 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Thu, 6 Aug 2020 13:18:57 -0700 Subject: RFR [16/java.xml] 8246816: XMLGregorianCalendar.hashCode() produces far too many identical hashes In-Reply-To: References: <058367db-dca5-76e9-7e61-049173c765d9@oracle.com> Message-ID: Hi Joe, thank you for looking it into further. Yes, I agree the chances of collision is very rare, as sub-millisecond difference in two objects. So fine with the version 01. Naoto On 8/6/20 12:18 PM, Joe Wang wrote: > Thanks Naoto, Roger for the review! > > For Naoto's concern about the equals method using EonAndYear and > FractionalSecond, I did cut corners using the all int getXXX method. The > rational was: it fulfills the equals-hashcode contract just fine, is > consistent with the existing implementation, and concise. This API was > introduced since 1.5, but I have yet to see any usage of EonAndYear, and > very rarely FractionalSecond. The chances collisions occur due to these > differences are very low. > > But I understand your concern. To be precise or exact as equals(), we > would need to call getFractionalSecond and EonAndYear. Here's what that > would look like: > http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_02/ > > To me, I like version 1 for the reasons above: > http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_01/ > > What would you think? > > Regards, > Joe > > On 8/5/20 6:13 PM, naoto.sato at oracle.com wrote: >> Hi Joe, >> >> For the consistency with equals(), just wondering if the sub-second >> element should be obtained with getFractionalSecond() instead of >> getMillisecond(), as the equals() subsequently calls it in >> internalCompare() method. Also should it call getEonAndYear() >> appropriately for the year? >> >> Naoto >> >> On 8/5/20 4:37 PM, Joe Wang wrote: >>> Hello, >>> >>> Please review a fix for the hashCode generation. >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8246816 >>> >>> webrev: http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev/ >>> >>> Thanks, >>> Joe >>> > From Roger.Riggs at oracle.com Thu Aug 6 20:29:23 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Thu, 6 Aug 2020 16:29:23 -0400 Subject: RFR [16/java.xml] 8246816: XMLGregorianCalendar.hashCode() produces far too many identical hashes In-Reply-To: References: <058367db-dca5-76e9-7e61-049173c765d9@oracle.com> Message-ID: <0e2935f7-4c1f-8782-0f51-12866d66316b@oracle.com> +1 On 8/6/20 4:18 PM, naoto.sato at oracle.com wrote: > Hi Joe, thank you for looking it into further. > > Yes, I agree the chances of collision is very rare, as sub-millisecond > difference in two objects. So fine with the version 01. > > Naoto > > On 8/6/20 12:18 PM, Joe Wang wrote: >> Thanks Naoto, Roger for the review! >> >> For Naoto's concern about the equals method using EonAndYear and >> FractionalSecond, I did cut corners using the all int getXXX method. >> The rational was: it fulfills the equals-hashcode contract just fine, >> is consistent with the existing implementation, and concise. This API >> was introduced since 1.5, but I have yet to see any usage of >> EonAndYear, and very rarely FractionalSecond. The chances collisions >> occur due to these differences are very low. >> >> But I understand your concern. To be precise or exact as equals(), we >> would need to call getFractionalSecond and EonAndYear. Here's what >> that would look like: >> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_02/ >> >> To me, I like version 1 for the reasons above: >> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_01/ >> >> What would you think? >> >> Regards, >> Joe >> >> On 8/5/20 6:13 PM, naoto.sato at oracle.com wrote: >>> Hi Joe, >>> >>> For the consistency with equals(), just wondering if the sub-second >>> element should be obtained with getFractionalSecond() instead of >>> getMillisecond(), as the equals() subsequently calls it in >>> internalCompare() method. Also should it call getEonAndYear() >>> appropriately for the year? >>> >>> Naoto >>> >>> On 8/5/20 4:37 PM, Joe Wang wrote: >>>> Hello, >>>> >>>> Please review a fix for the hashCode generation. >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8246816 >>>> >>>> webrev: http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev/ >>>> >>>> Thanks, >>>> Joe >>>> >> From huizhe.wang at oracle.com Thu Aug 6 20:51:00 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Thu, 6 Aug 2020 13:51:00 -0700 Subject: RFR [16/java.xml] 8246816: XMLGregorianCalendar.hashCode() produces far too many identical hashes In-Reply-To: <0e2935f7-4c1f-8782-0f51-12866d66316b@oracle.com> References: <058367db-dca5-76e9-7e61-049173c765d9@oracle.com> <0e2935f7-4c1f-8782-0f51-12866d66316b@oracle.com> Message-ID: <274e1322-844a-5e9f-c6d7-3784ae937572@oracle.com> Thanks Naoto, Roger!? I'll prepare push based on version 01. Best, Joe On 8/6/20 1:29 PM, Roger Riggs wrote: > +1 > > On 8/6/20 4:18 PM, naoto.sato at oracle.com wrote: >> Hi Joe, thank you for looking it into further. >> >> Yes, I agree the chances of collision is very rare, as >> sub-millisecond difference in two objects. So fine with the version 01. >> >> Naoto >> >> On 8/6/20 12:18 PM, Joe Wang wrote: >>> Thanks Naoto, Roger for the review! >>> >>> For Naoto's concern about the equals method using EonAndYear and >>> FractionalSecond, I did cut corners using the all int getXXX method. >>> The rational was: it fulfills the equals-hashcode contract just >>> fine, is consistent with the existing implementation, and concise. >>> This API was introduced since 1.5, but I have yet to see any usage >>> of EonAndYear, and very rarely FractionalSecond. The chances >>> collisions occur due to these differences are very low. >>> >>> But I understand your concern. To be precise or exact as equals(), >>> we would need to call getFractionalSecond and EonAndYear. Here's >>> what that would look like: >>> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_02/ >>> >>> To me, I like version 1 for the reasons above: >>> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_01/ >>> >>> What would you think? >>> >>> Regards, >>> Joe >>> >>> On 8/5/20 6:13 PM, naoto.sato at oracle.com wrote: >>>> Hi Joe, >>>> >>>> For the consistency with equals(), just wondering if the sub-second >>>> element should be obtained with getFractionalSecond() instead of >>>> getMillisecond(), as the equals() subsequently calls it in >>>> internalCompare() method. Also should it call getEonAndYear() >>>> appropriately for the year? >>>> >>>> Naoto >>>> >>>> On 8/5/20 4:37 PM, Joe Wang wrote: >>>>> Hello, >>>>> >>>>> Please review a fix for the hashCode generation. >>>>> >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8246816 >>>>> >>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev/ >>>>> >>>>> Thanks, >>>>> Joe >>>>> >>> > From raffaello.giulietti at gmail.com Thu Aug 6 21:36:04 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Thu, 6 Aug 2020 23:36:04 +0200 Subject: RFR: 8245036: DataInputStream.readFully(byte[], int, int) does not throw expected IndexOutOfBoundsExceptions In-Reply-To: <4030EF21-CCE0-48D2-ADB1-7BF7519FEA52@oracle.com> References: <59a871a4-d676-3b0c-2983-b9b9fc34ca37@gmail.com> <858a70ac-f827-669a-58e2-e5b834aa36f4@gmail.com> <67C3F744-2FE7-4BDD-857F-86E64A1AABFA@oracle.com> <4030EF21-CCE0-48D2-ADB1-7BF7519FEA52@oracle.com> Message-ID: Hi Brian, the body of readFully() seems to silently assume that the other arguments are correct (e.g., off >= 0 and so on). In the (still) current implementation of DataInputStream.readFully(byte[], int, int), when off < 0 and len > 0, in the loop it then invokes the underlying in.read(byte[], int, int) with a negative offset off + n during the initial iterations. In the specific case of the tests on underlying FileInputStreams, it thus seems that read(byte[], int, int) does not, in turn, validate its arguments. I didn't investigate further, although I will in the next days if deemed necessary. Greetings Raffaello On 2020-08-06 22:09, Brian Burkhalter wrote: > >> On Aug 6, 2020, at 12:36 PM, Raffaello Giulietti >> > >> wrote: >> >>> This looks all right. I would be inclined to do like in [1] and throw >>> the RuntimeException on the line after each readFully() and drop the >>> return statements], but I?m not going to insist on it. >>> Thanks, >>> Brian >>> [1]http://hg.openjdk.java.net/jdk/jdk/file/db6952bf8c39/test/jdk/java/nio/file/etc/MacVolumesTest.java#l58 >> >> Sorry, I didn't completely get that. >> Here we are! > > This looks good. I observed however that the test passes without the > change to DataInputStream. > > Brian From brian.burkhalter at oracle.com Thu Aug 6 21:42:56 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 6 Aug 2020 14:42:56 -0700 Subject: RFR: 8245036: DataInputStream.readFully(byte[], int, int) does not throw expected IndexOutOfBoundsExceptions In-Reply-To: References: <59a871a4-d676-3b0c-2983-b9b9fc34ca37@gmail.com> <858a70ac-f827-669a-58e2-e5b834aa36f4@gmail.com> <67C3F744-2FE7-4BDD-857F-86E64A1AABFA@oracle.com> <4030EF21-CCE0-48D2-ADB1-7BF7519FEA52@oracle.com> Message-ID: Hi Raffaello, I think the source change is correct, it?s just that for a deterministic issue, the test generally will fail before the source change and pass with the source change applied. I did not investigate any further, either. Thanks, Brian > On Aug 6, 2020, at 2:36 PM, Raffaello Giulietti wrote: > > the body of readFully() seems to silently assume that the other arguments are correct (e.g., off >= 0 and so on). > > In the (still) current implementation of DataInputStream.readFully(byte[], int, int), when off < 0 and len > 0, in the loop it then invokes the underlying in.read(byte[], int, int) with a negative offset off + n during the initial iterations. > > In the specific case of the tests on underlying FileInputStreams, it thus seems that read(byte[], int, int) does not, in turn, validate its arguments. I didn't investigate further, although I will in the next days if deemed necessary. From aleksej.efimov at oracle.com Thu Aug 6 22:21:47 2020 From: aleksej.efimov at oracle.com (Aleks Efimov) Date: Thu, 6 Aug 2020 23:21:47 +0100 Subject: RFR [testbug]: 8250772: Test com/sun/jndi/ldap/NamingExceptionMessageTest.java fails intermittently with javax.naming.ServiceUnavailableException Message-ID: Hi, Please, help to review NamingExceptionMessageTest test stabilization change: There is a race condition between the closure of connection in test LDAP server and the JDKs LDAP client code. Due to this race the test could generate ServiceUnavailableException instead of the required NamingException. Webrev: http://cr.openjdk.java.net/~aefimov/8250772/00/index.html JBS: https://bugs.openjdk.java.net/browse/JDK-8250772 The following modifications have been made to the test code: a) To increase the reproduction rate of the required exception message the connection closure timeout has been added to the test LDAP server's? BIND operation handler b) ServiceUnavailableException is now treated as an acceptable exception The modified test was successfully executed 100+ times alongside to other LDAP tests With Best Regards, Aleksei From LANCE.ANDERSEN at ORACLE.COM Thu Aug 6 22:32:22 2020 From: LANCE.ANDERSEN at ORACLE.COM (Lance Andersen) Date: Thu, 6 Aug 2020 18:32:22 -0400 Subject: RFR 8251202: Add missing javadoc comments Message-ID: <2684BFAF-A330-45DC-8FD0-275041640CF8@ORACLE.COM> Hi all, Please review the webrev for: https://bugs.openjdk.java.net/browse/JDK-8251202 , which adds missing javadoc comments to java.util.zip.ZipConstants. The webrev can be found at: http://cr.openjdk.java.net/~lancea/8251205/webrev.00/ The CSR, which has been approved, can be found at: https://bugs.openjdk.java.net/browse/JDK-8251206 Best Lance ------------------ Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From brian.burkhalter at oracle.com Thu Aug 6 22:54:44 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 6 Aug 2020 15:54:44 -0700 Subject: 8251272: Typo in java.util.Formatter: "Numberic" should be "Numeric" Message-ID: <80186CEA-1397-4A22-86A1-A01864A0E8E9@oracle.com> For [1], please review this trivial fix [2]. Thanks, Brian [1] https://bugs.openjdk.java.net/browse/JDK-8251272 [2] diff --- a/src/java.base/share/classes/java/util/Formatter.java +++ b/src/java.base/share/classes/java/util/Formatter.java @@ -277,7 +277,7 @@ * * * - *

    For category General, Character, Numberic, + *

    For category General, Character, Numeric, * Integral and Date/Time conversion, unless otherwise specified, * if the argument arg is {@code null}, then the result is "{@code null}". * @@ -703,7 +703,7 @@ * methods such as {@link String#format(String,Object...) String.format} and * {@link java.io.PrintStream#printf(String,Object...) PrintStream.printf}. * - *

    For category General, Character, Numberic, + *

    For category General, Character, Numeric, * Integral and Date/Time conversion, unless otherwise specified, * if the argument arg is {@code null}, then the result is "{@code null}". * From brent.christian at oracle.com Thu Aug 6 23:12:43 2020 From: brent.christian at oracle.com (Brent Christian) Date: Thu, 6 Aug 2020 16:12:43 -0700 Subject: 8251272: Typo in java.util.Formatter: "Numberic" should be "Numeric" In-Reply-To: <80186CEA-1397-4A22-86A1-A01864A0E8E9@oracle.com> References: <80186CEA-1397-4A22-86A1-A01864A0E8E9@oracle.com> Message-ID: <7cd45b32-3f6f-df38-d6e7-322a12f17887@oracle.com> Looks good! -Brent On 8/6/20 3:54 PM, Brian Burkhalter wrote: > For [1], please review this trivial fix [2]. > > Thanks, > > Brian > > [1] https://bugs.openjdk.java.net/browse/JDK-8251272 > [2] diff > > --- a/src/java.base/share/classes/java/util/Formatter.java > +++ b/src/java.base/share/classes/java/util/Formatter.java > @@ -277,7 +277,7 @@ > * > * > * > - *

    For category General, Character, Numberic, > + *

    For category General, Character, Numeric, > * Integral and Date/Time conversion, unless otherwise specified, > * if the argument arg is {@code null}, then the result is "{@code null}". > * > @@ -703,7 +703,7 @@ > * methods such as {@link String#format(String,Object...) String.format} and > * {@link java.io.PrintStream#printf(String,Object...) PrintStream.printf}. > * > - *

    For category General, Character, Numberic, > + *

    For category General, Character, Numeric, > * Integral and Date/Time conversion, unless otherwise specified, > * if the argument arg is {@code null}, then the result is "{@code null}". > * > From naoto.sato at oracle.com Thu Aug 6 23:24:24 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Thu, 6 Aug 2020 16:24:24 -0700 Subject: 8251272: Typo in java.util.Formatter: "Numberic" should be "Numeric" In-Reply-To: <7cd45b32-3f6f-df38-d6e7-322a12f17887@oracle.com> References: <80186CEA-1397-4A22-86A1-A01864A0E8E9@oracle.com> <7cd45b32-3f6f-df38-d6e7-322a12f17887@oracle.com> Message-ID: <9f269919-22f3-b1b8-92f2-cceb9a194100@oracle.com> +1 Good catch! Naoto On 8/6/20 4:12 PM, Brent Christian wrote: > Looks good! > -Brent > > On 8/6/20 3:54 PM, Brian Burkhalter wrote: >> For [1], please review this trivial fix [2]. >> >> Thanks, >> >> Brian >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8251272 >> [2] diff >> >> --- a/src/java.base/share/classes/java/util/Formatter.java >> +++ b/src/java.base/share/classes/java/util/Formatter.java >> @@ -277,7 +277,7 @@ >> ?? * >> ?? * >> ?? * >> - *

    For category General, Character, Numberic, >> + *

    For category General, Character, Numeric, >> ?? * Integral and Date/Time conversion, unless otherwise >> specified, >> ?? * if the argument arg is {@code null}, then the result is >> "{@code null}". >> ?? * >> @@ -703,7 +703,7 @@ >> ?? * methods such as {@link String#format(String,Object...) >> String.format} and >> ?? * {@link java.io.PrintStream#printf(String,Object...) >> PrintStream.printf}. >> ?? * >> - *

    For category General, Character, Numberic, >> + *

    For category General, Character, Numeric, >> ?? * Integral and Date/Time conversion, unless otherwise >> specified, >> ?? * if the argument arg is {@code null}, then the result is >> "{@code null}". >> ?? * >> From naoto.sato at oracle.com Thu Aug 6 23:34:49 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Thu, 6 Aug 2020 16:34:49 -0700 Subject: RFR 8251202: Add missing javadoc comments In-Reply-To: <2684BFAF-A330-45DC-8FD0-275041640CF8@ORACLE.COM> References: <2684BFAF-A330-45DC-8FD0-275041640CF8@ORACLE.COM> Message-ID: <02c60666-7588-a47e-18f1-65bde958e840@oracle.com> Looks good, Lance. nit: a couple of comments (line 128,133) don't end with periods. Naoto On 8/6/20 3:32 PM, Lance Andersen wrote: > Hi all, > > Please review the webrev for: https://bugs.openjdk.java.net/browse/JDK-8251202 , which adds missing javadoc comments to java.util.zip.ZipConstants. > > The webrev can be found at: http://cr.openjdk.java.net/~lancea/8251205/webrev.00/ > > The CSR, which has been approved, can be found at: https://bugs.openjdk.java.net/browse/JDK-8251206 > > Best > Lance > ------------------ > > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > > From Lance.Andersen at oracle.com Fri Aug 7 00:04:56 2020 From: Lance.Andersen at oracle.com (Lance Andersen) Date: Thu, 6 Aug 2020 20:04:56 -0400 Subject: RFR 8251202: Add missing javadoc comments In-Reply-To: <02c60666-7588-a47e-18f1-65bde958e840@oracle.com> References: <02c60666-7588-a47e-18f1-65bde958e840@oracle.com> Message-ID: <68E0D8BA-42F7-4BE1-9344-E126588D15F7@oracle.com> Hi Naoto Thank you for catching that Will add them before I push the change Best Lance -- Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com Sent from my iPhone > On Aug 6, 2020, at 7:35 PM, naoto.sato at oracle.com wrote: > > ?Looks good, Lance. > > nit: a couple of comments (line 128,133) don't end with periods. > > Naoto > >> On 8/6/20 3:32 PM, Lance Andersen wrote: >> Hi all, >> Please review the webrev for: https://bugs.openjdk.java.net/browse/JDK-8251202 , which adds missing javadoc comments to java.util.zip.ZipConstants. >> The webrev can be found at: http://cr.openjdk.java.net/~lancea/8251205/webrev.00/ >> The CSR, which has been approved, can be found at: https://bugs.openjdk.java.net/browse/JDK-8251206 >> Best >> Lance >> ------------------ >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com From james.laskey at oracle.com Fri Aug 7 00:40:14 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Thu, 6 Aug 2020 21:40:14 -0300 Subject: 8251272: Typo in java.util.Formatter: "Numberic" should be "Numeric" In-Reply-To: <80186CEA-1397-4A22-86A1-A01864A0E8E9@oracle.com> References: <80186CEA-1397-4A22-86A1-A01864A0E8E9@oracle.com> Message-ID: <6B80DDED-A2CB-4AF7-89D1-A8F641641950@oracle.com> +1 Sent from my iPad > On Aug 6, 2020, at 7:55 PM, Brian Burkhalter wrote: > > ?For [1], please review this trivial fix [2]. > > Thanks, > > Brian > > [1] https://bugs.openjdk.java.net/browse/JDK-8251272 > [2] diff > > --- a/src/java.base/share/classes/java/util/Formatter.java > +++ b/src/java.base/share/classes/java/util/Formatter.java > @@ -277,7 +277,7 @@ > * > * > * > - *

    For category General, Character, Numberic, > + *

    For category General, Character, Numeric, > * Integral and Date/Time conversion, unless otherwise specified, > * if the argument arg is {@code null}, then the result is "{@code null}". > * > @@ -703,7 +703,7 @@ > * methods such as {@link String#format(String,Object...) String.format} and > * {@link java.io.PrintStream#printf(String,Object...) PrintStream.printf}. > * > - *

    For category General, Character, Numberic, > + *

    For category General, Character, Numeric, > * Integral and Date/Time conversion, unless otherwise specified, > * if the argument arg is {@code null}, then the result is "{@code null}". > * From li.jiang at oracle.com Fri Aug 7 01:50:19 2020 From: li.jiang at oracle.com (li.jiang at oracle.com) Date: Fri, 7 Aug 2020 09:50:19 +0800 Subject: RFR [16]: 8250665 : Wrong translation for the month of May in ar_JO, ar_LB and ar_SY In-Reply-To: References: <8766969e-72af-d937-0368-14d2958e1244@oracle.com> Message-ID: <363311e8-a5f4-731d-57af-183c7c2709b6@oracle.com> Thank you Naoto, pushed. -Leo On 8/7/20 2:18 AM, naoto.sato at oracle.com wrote: > Looks good to me. > > Naoto > > On 8/6/20 9:47 AM, li.jiang at oracle.com wrote: >> Hi, >> >> Pls review the fix for month names in COMPAT provider data for >> ar_JO/LB/SY. >> >> Customer reported some month names are incorrect in JDK8 code >> according to latest CLDR definition. >> >> Bug: >> https://bugs.openjdk.java.net/browse/JDK-8250665 >> >> Webrev: >> http://cr.openjdk.java.net/~ljiang/8250665/webrev.00/ >> >> Passed the tier1 and tier2 test cases. >> >> Thanks, >> Leo >> From sundararajan.athijegannathan at oracle.com Fri Aug 7 05:01:15 2020 From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com) Date: Fri, 7 Aug 2020 10:31:15 +0530 Subject: RFR (JDK 15) 8251276: two jdeps files have extra whitespace in copyright header Message-ID: Please review. Bug: https://bugs.openjdk.java.net/browse/JDK-8251276 Webrev: http://cr.openjdk.java.net/~sundar/8251276/webrev.00/index.html Thanks, -Sundar From vyommani at gmail.com Fri Aug 7 05:09:19 2020 From: vyommani at gmail.com (Vyom Tiwari) Date: Fri, 7 Aug 2020 10:39:19 +0530 Subject: RFR (JDK 15) 8251276: two jdeps files have extra whitespace in copyright header In-Reply-To: References: Message-ID: LGTM Vyom On Fri, Aug 7, 2020 at 10:31 AM wrote: > Please review. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8251276 > > Webrev: http://cr.openjdk.java.net/~sundar/8251276/webrev.00/index.html > > Thanks, > > -Sundar > > -- Thanks, Vyom From raffaello.giulietti at gmail.com Fri Aug 7 08:03:00 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Fri, 7 Aug 2020 10:03:00 +0200 Subject: RFR: 8245036: DataInputStream.readFully(byte[], int, int) does not throw expected IndexOutOfBoundsExceptions In-Reply-To: References: <59a871a4-d676-3b0c-2983-b9b9fc34ca37@gmail.com> <858a70ac-f827-669a-58e2-e5b834aa36f4@gmail.com> <67C3F744-2FE7-4BDD-857F-86E64A1AABFA@oracle.com> <4030EF21-CCE0-48D2-ADB1-7BF7519FEA52@oracle.com> Message-ID: On 2020-08-06 23:42, Brian Burkhalter wrote: > Hi Raffaello, > > I think the source change is correct, it?s just that for a deterministic > issue, the test generally will fail before the source change and pass > with the source change applied. I did not investigate any further, either. > > Thanks, > > Brian > >> On Aug 6, 2020, at 2:36 PM, Raffaello Giulietti >> > >> wrote: >> >> the body of readFully() seems to silently assume that the other >> arguments are correct (e.g., off >= 0 and so on). >> >> In the (still) current implementation of >> DataInputStream.readFully(byte[], int, int), when off < 0 and len > 0, >> in the loop it then invokes the underlying in.read(byte[], int, int) >> with a negative offset off + n during the initial iterations. >> >> In the specific case of the tests on underlying FileInputStreams, it >> thus seems that read(byte[], int, int) does not, in turn, validate its >> arguments. I didn't investigate further, although I will in the next >> days if deemed necessary. > Hi Brian, my previous explanation is (luckily) wrong. The reason the new tests pass on the old source is because the underlying input stream correctly throws IOOBE which is catched by the tests. There's one corner case that would not pass, however, so I added testNegativeOffsetZeroLength() to exercise that. It passes on the new source but fails on the old. I also added two additional corner cases for too large combinations of offset and length. ---- # HG changeset patch # User lello # Date 1596786685 -7200 # Fri Aug 07 09:51:25 2020 +0200 # Node ID 1537910a63fbcd517607779c72b02ead2c77bf08 # Parent b01985b4f88f554f97901e53e1ba314681dd9c19 Patch to fix JDK-8245036 8245036: DataInputStream.readFully(byte[], int, int) does not throw expected IndexOutOfBoundsExceptions Reviewed-by: TBD Contributed-by: Raffaello Giulietti diff --git a/src/java.base/share/classes/java/io/DataInputStream.java b/src/java.base/share/classes/java/io/DataInputStream.java --- a/src/java.base/share/classes/java/io/DataInputStream.java +++ b/src/java.base/share/classes/java/io/DataInputStream.java @@ -1,5 +1,5 @@ /* - * Copyright (c) 1994, 2019, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1994, 2020, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -25,6 +25,8 @@ package java.io; +import java.util.Objects; + /** * A data input stream lets an application read primitive Java data * types from an underlying input stream in a machine-independent @@ -192,8 +194,7 @@ * @see java.io.FilterInputStream#in */ public final void readFully(byte b[], int off, int len) throws IOException { - if (len < 0) - throw new IndexOutOfBoundsException(); + Objects.checkFromIndexSize(off, len, b.length); int n = 0; while (n < len) { int count = in.read(b, off + n, len - n); diff --git a/test/jdk/java/io/DataInputStream/ReadFully.java b/test/jdk/java/io/DataInputStream/ReadFully.java --- a/test/jdk/java/io/DataInputStream/ReadFully.java +++ b/test/jdk/java/io/DataInputStream/ReadFully.java @@ -1,5 +1,5 @@ /* - * Copyright (c) 1999, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1999, 2020, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -22,30 +22,108 @@ */ /* @test - * @bug 4214513 - * @summary Passing a negative length argument for readFully must throw - * IndexOutOfBoundsException. + * @bug 4214513 8245036 + * @summary Passing a negative offset or length, + * or passing a combination of offset and length too big + * for readFully must throw IndexOutOfBoundsException. */ +import java.io.*; -import java.io.*; public class ReadFully { - public static final void main(String[] args) throws Exception { - byte[] buffer = new byte[100]; + + private static final void testNegativeOffset() throws Exception { + File file = new File(System.getProperty("test.src"), + "ReadFully.java"); + try (FileInputStream in = new FileInputStream(file); + DataInputStream dis = new DataInputStream(in);) { + byte[] buffer = new byte[100]; + dis.readFully(buffer, -1, buffer.length); + throw new RuntimeException("Test testNegativeOffset() failed"); + } catch (IndexOutOfBoundsException ignore) { + } + } + + private static final void testNegativeLength() throws Exception { + File file = new File(System.getProperty("test.src"), + "ReadFully.java"); + try (FileInputStream in = new FileInputStream(file); + DataInputStream dis = new DataInputStream(in);) { + byte[] buffer = new byte[100]; + dis.readFully(buffer, 0, -1); + throw new RuntimeException("Test testNegativeLength() failed"); + } catch (IndexOutOfBoundsException ignore) { + } + } + + private static final void testNegativeOffsetZeroLength() throws Exception { + File file = new File(System.getProperty("test.src"), + "ReadFully.java"); + try (FileInputStream in = new FileInputStream(file); + DataInputStream dis = new DataInputStream(in);) { + byte[] buffer = new byte[100]; + dis.readFully(buffer, -1, 0); + throw new RuntimeException("Test testNegativeOffsetZeroLength() failed"); + } catch (IndexOutOfBoundsException ignore) { + } + } + + private static final void testBigOffsetLength1() throws Exception { File file = new File(System.getProperty("test.src"), "ReadFully.java"); - FileInputStream in = new FileInputStream(file); - DataInputStream dis = new DataInputStream(in); + try (FileInputStream in = new FileInputStream(file); + DataInputStream dis = new DataInputStream(in);) { + byte[] buffer = new byte[100]; + dis.readFully(buffer, 0, buffer.length + 1); + throw new RuntimeException("Test testBigOffsetLength1() failed"); + } catch (IndexOutOfBoundsException ignore) { + } + } - boolean caughtException = false; - try { - dis.readFully(buffer, 0, -20); - } catch (IndexOutOfBoundsException ie) { - caughtException = true; - } finally { - dis.close(); - if (!caughtException) - throw new RuntimeException("Test failed"); + private static final void testBigOffsetLength2() throws Exception { + File file = new File(System.getProperty("test.src"), + "ReadFully.java"); + try (FileInputStream in = new FileInputStream(file); + DataInputStream dis = new DataInputStream(in);) { + byte[] buffer = new byte[100]; + dis.readFully(buffer, 1, buffer.length); + throw new RuntimeException("Test testBigOffsetLength2() failed"); + } catch (IndexOutOfBoundsException ignore) { } } + + private static final void testBigOffsetLength3() throws Exception { + File file = new File(System.getProperty("test.src"), + "ReadFully.java"); + try (FileInputStream in = new FileInputStream(file); + DataInputStream dis = new DataInputStream(in);) { + byte[] buffer = new byte[100]; + dis.readFully(buffer, buffer.length, 1); + throw new RuntimeException("Test testBigOffsetLength3() failed"); + } catch (IndexOutOfBoundsException ignore) { + } + } + + private static final void testBigOffsetLength4() throws Exception { + File file = new File(System.getProperty("test.src"), + "ReadFully.java"); + try (FileInputStream in = new FileInputStream(file); + DataInputStream dis = new DataInputStream(in);) { + byte[] buffer = new byte[100]; + dis.readFully(buffer, buffer.length + 1, 0); + throw new RuntimeException("Test testBigOffsetLength4() failed"); + } catch (IndexOutOfBoundsException ignore) { + } + } + + public static final void main(String[] args) throws Exception { + testNegativeOffset(); + testNegativeLength(); + testNegativeOffsetZeroLength(); + testBigOffsetLength1(); + testBigOffsetLength2(); + testBigOffsetLength3(); + testBigOffsetLength4(); + } + } -------------- next part -------------- A non-text attachment was scrubbed... Name: JDK-8245036.patch Type: text/x-patch Size: 7289 bytes Desc: not available URL: From brian.burkhalter at oracle.com Fri Aug 7 15:33:26 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 7 Aug 2020 08:33:26 -0700 Subject: [Ping] Re: 8249703: test/jdk/java/io/File/GetXSpace.java fails on macos In-Reply-To: References: Message-ID: > On Jul 31, 2020, at 10:42 AM, Brian Burkhalter wrote: > > On macOS, the number of 1024 byte blocks appears to be incorrectly calculated by 'df' using integer division by 2 of the number of 512 byte blocks, resulting in a size smaller than the actual value when the number of blocks is odd. > > Possibly instead of having the IS_MAC and IS_WIN constants it should use the equivalent methods in jdk.test.lib.Platform? > > Thanks, > > Brian > > https://bugs.openjdk.java.net/browse/JDK-8249703 > http://cr.openjdk.java.net/~bpb/8249703/webrev.00/ From david.lloyd at redhat.com Fri Aug 7 15:39:42 2020 From: david.lloyd at redhat.com (David Lloyd) Date: Fri, 7 Aug 2020 10:39:42 -0500 Subject: Possible subtle memory model error in ClassValue Message-ID: I'm helping a colleague debug a weird problem that occurs in ClassValue on jdk11u (and presumably on upstream as well, though it's presently impossible to verify). As a disclaimer, the problem manifests itself when building native images via GraalVM so it's possible that something is simply broken there, but it seems at least feasible that it could be a plain old Java bug so I thought I'd send up a flare here to see if this makes sense to anyone else. The bug itself manifests (on jdk11u) as an NPE with the following exception trace: java.lang.NullPointerException at java.base/java.lang.ClassValue$ClassValueMap.loadFromCache(ClassValue.java:535) at java.base/java.lang.ClassValue$ClassValueMap.probeHomeLocation(ClassValue.java:541) at java.base/java.lang.ClassValue.get(ClassValue.java:101) ... Some basic analysis shows that this should be impossible under normal/naive circumstances: the initializer of java.lang.ClassValue.ClassValueMap sets the corresponding field to non-null during construction. However, I'm wondering if this isn't a side effect of what appears to be an incorrect double-checked lock at lines 374-378 of ClassValue.java [1]. In order for the write to the non-volatile `cache` field of ClassValueMap, it is my understanding that there must be some acquire/release edge from where the variable is published to guarantee that all of the writes are visible to the read site, and I'm not really sure that the exit-the-constructor edge is going to match up with with the synchronized block which protects a different non-volatile field. And even if my feeling that this is dodgy is valid, I'm not sure whether this NPE is a possible outcome of that problem! Thoughts? [1] https://github.com/openjdk/jdk11u-dev/blob/3789983e89c9de252ef546a1b98a732a7d066650/src/java.base/share/classes/java/lang/ClassValue.java#L374-L378 -- - DML ? he/him From paul.sandoz at oracle.com Fri Aug 7 16:17:12 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 7 Aug 2020 09:17:12 -0700 Subject: Possible subtle memory model error in ClassValue In-Reply-To: References: Message-ID: <7EF1280A-9606-485F-A739-63B3B82C8549@oracle.com> Hi David, This is subtle. ClassValue extends from WeakHashMap that has a few final fields. In such cases, for HotSpot at least, the compiler will place fence between the stores to the fields of ClassValue and the store to publish in the field of Class. So it should not be possible to observe a partially constructed ClassValue, where the field ClassValueMap.cacheArray is null (which seems to the the source of the NPE). However, perhaps the Graal AoT compiler does not behave the same? Paul. > On Aug 7, 2020, at 8:39 AM, David Lloyd wrote: > > I'm helping a colleague debug a weird problem that occurs in > ClassValue on jdk11u (and presumably on upstream as well, though it's > presently impossible to verify). As a disclaimer, the problem > manifests itself when building native images via GraalVM so it's > possible that something is simply broken there, but it seems at least > feasible that it could be a plain old Java bug so I thought I'd send > up a flare here to see if this makes sense to anyone else. > > The bug itself manifests (on jdk11u) as an NPE with the following > exception trace: > > java.lang.NullPointerException > at java.base/java.lang.ClassValue$ClassValueMap.loadFromCache(ClassValue.java:535) > at java.base/java.lang.ClassValue$ClassValueMap.probeHomeLocation(ClassValue.java:541) > at java.base/java.lang.ClassValue.get(ClassValue.java:101) > ... > > Some basic analysis shows that this should be impossible under > normal/naive circumstances: the initializer of > java.lang.ClassValue.ClassValueMap sets the corresponding field to > non-null during construction. > > However, I'm wondering if this isn't a side effect of what appears to > be an incorrect double-checked lock at lines 374-378 of > ClassValue.java [1]. In order for the write to the non-volatile > `cache` field of ClassValueMap, it is my understanding that there must > be some acquire/release edge from where the variable is published to > guarantee that all of the writes are visible to the read site, and I'm > not really sure that the exit-the-constructor edge is going to match > up with with the synchronized block which protects a different > non-volatile field. And even if my feeling that this is dodgy is > valid, I'm not sure whether this NPE is a possible outcome of that > problem! > > Thoughts? > > [1] https://github.com/openjdk/jdk11u-dev/blob/3789983e89c9de252ef546a1b98a732a7d066650/src/java.base/share/classes/java/lang/ClassValue.java#L374-L378 > -- > - DML ? he/him > From brian.burkhalter at oracle.com Fri Aug 7 16:39:44 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 7 Aug 2020 09:39:44 -0700 Subject: RFR: 8245036: DataInputStream.readFully(byte[], int, int) does not throw expected IndexOutOfBoundsExceptions In-Reply-To: References: <59a871a4-d676-3b0c-2983-b9b9fc34ca37@gmail.com> <858a70ac-f827-669a-58e2-e5b834aa36f4@gmail.com> <67C3F744-2FE7-4BDD-857F-86E64A1AABFA@oracle.com> <4030EF21-CCE0-48D2-ADB1-7BF7519FEA52@oracle.com> Message-ID: <10711294-E57F-4D89-9F51-61DAE4EFC8BF@oracle.com> Hi Raffaello, > On Aug 7, 2020, at 1:03 AM, Raffaello Giulietti wrote: > > my previous explanation is (luckily) wrong. > > The reason the new tests pass on the old source is because the underlying input stream correctly throws IOOBE which is catched by the tests. OK. > There's one corner case that would not pass, however, so I added testNegativeOffsetZeroLength() to exercise that. It passes on the new source but fails on the old. > > I also added two additional corner cases for too large combinations of offset and length. This patch looks good. I am running it through our CI tests but I don?t expect to see any problems. If the test run succeeds aside from any failures unrelated to this code then I can push this. Thanks, Brian From aph at redhat.com Fri Aug 7 16:52:37 2020 From: aph at redhat.com (Andrew Haley) Date: Fri, 7 Aug 2020 17:52:37 +0100 Subject: Possible subtle memory model error in ClassValue In-Reply-To: References: Message-ID: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> On 8/7/20 4:39 PM, David Lloyd wrote: > However, I'm wondering if this isn't a side effect of what appears > to be an incorrect double-checked lock at lines 374-378 of > ClassValue.java [1]. In order for the write to the non-volatile > `cache` field of ClassValueMap, it is my understanding that there > must be some acquire/release edge from where the variable is > published to guarantee that all of the writes are visible to the > read site, and I'm not really sure that the exit-the-constructor > edge is going to match up with with the synchronized block which > protects a different non-volatile field. And even if my feeling > that this is dodgy is valid, I'm not sure whether this NPE is a > possible outcome of that problem! It certainly doesn't look right to me. AFAICS this is classic broken double-checked locking. It'd need some sort of fence after constructing the ClassValueMap instance and before publishing it. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From naoto.sato at oracle.com Fri Aug 7 16:58:36 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Fri, 7 Aug 2020 09:58:36 -0700 Subject: [Ping] Re: 8249703: test/jdk/java/io/File/GetXSpace.java fails on macos In-Reply-To: References: Message-ID: Looks fine, Brian. I might add the condition if the block number is odd. Naoto On 8/7/20 8:33 AM, Brian Burkhalter wrote: > >> On Jul 31, 2020, at 10:42 AM, Brian Burkhalter wrote: >> >> On macOS, the number of 1024 byte blocks appears to be incorrectly calculated by 'df' using integer division by 2 of the number of 512 byte blocks, resulting in a size smaller than the actual value when the number of blocks is odd. >> >> Possibly instead of having the IS_MAC and IS_WIN constants it should use the equivalent methods in jdk.test.lib.Platform? >> >> Thanks, >> >> Brian >> >> https://bugs.openjdk.java.net/browse/JDK-8249703 >> http://cr.openjdk.java.net/~bpb/8249703/webrev.00/ From huizhe.wang at oracle.com Fri Aug 7 17:16:19 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Fri, 7 Aug 2020 10:16:19 -0700 Subject: RFR [16/java.xml] 8246816: XMLGregorianCalendar.hashCode() produces far too many identical hashes In-Reply-To: <274e1322-844a-5e9f-c6d7-3784ae937572@oracle.com> References: <058367db-dca5-76e9-7e61-049173c765d9@oracle.com> <0e2935f7-4c1f-8782-0f51-12866d66316b@oracle.com> <274e1322-844a-5e9f-c6d7-3784ae937572@oracle.com> Message-ID: <2ce56d2c-9ce9-a1a2-ae1d-7422c264b37f@oracle.com> Naoto, Roger, Sorry have to come back with another iteration. It looks like the addition of getMillisecond() over the original implementation does change things. In particular, getMillisecond() returns FIELD_UNDEFINED when fractionalSecond == null (or not specified). But the normalization process will set it to zero after a mathematics adjustment. Now this will fail when comparing with a datetime that does not need to be normalized and thereby still returns FIELD_UNDEFINED (FIELD_UNDEFINED = Integer.MIN_VALUE). This situation is demonstrated in tests 3 at line 56 and 4 at line 58. The culprit is XMLGregorianCalendarImpl::normalizeToTimezone. In the original implementation, XMLGregorianCalendarImpl was calling the private normalizeToTimezone while XMLGregorianCalendar the public normalize(). The later will do the right thing to set the Millisecond back to UNDEFINED. In webrev_03, XMLGregorianCalendarImpl now calls super.hashCode() since the above was the only difference between the hashCode() impls of XMLGregorianCalendarImpl and XMLGregorianCalendar. webrev_03: http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_03/ Thanks, Joe On 8/6/20 1:51 PM, Joe Wang wrote: > Thanks Naoto, Roger!? I'll prepare push based on version 01. > > Best, > Joe > > On 8/6/20 1:29 PM, Roger Riggs wrote: >> +1 >> >> On 8/6/20 4:18 PM, naoto.sato at oracle.com wrote: >>> Hi Joe, thank you for looking it into further. >>> >>> Yes, I agree the chances of collision is very rare, as >>> sub-millisecond difference in two objects. So fine with the version 01. >>> >>> Naoto >>> >>> On 8/6/20 12:18 PM, Joe Wang wrote: >>>> Thanks Naoto, Roger for the review! >>>> >>>> For Naoto's concern about the equals method using EonAndYear and >>>> FractionalSecond, I did cut corners using the all int getXXX >>>> method. The rational was: it fulfills the equals-hashcode contract >>>> just fine, is consistent with the existing implementation, and >>>> concise. This API was introduced since 1.5, but I have yet to see >>>> any usage of EonAndYear, and very rarely FractionalSecond. The >>>> chances collisions occur due to these differences are very low. >>>> >>>> But I understand your concern. To be precise or exact as equals(), >>>> we would need to call getFractionalSecond and EonAndYear. Here's >>>> what that would look like: >>>> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_02/ >>>> >>>> To me, I like version 1 for the reasons above: >>>> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_01/ >>>> >>>> What would you think? >>>> >>>> Regards, >>>> Joe >>>> >>>> On 8/5/20 6:13 PM, naoto.sato at oracle.com wrote: >>>>> Hi Joe, >>>>> >>>>> For the consistency with equals(), just wondering if the >>>>> sub-second element should be obtained with getFractionalSecond() >>>>> instead of getMillisecond(), as the equals() subsequently calls it >>>>> in internalCompare() method. Also should it call getEonAndYear() >>>>> appropriately for the year? >>>>> >>>>> Naoto >>>>> >>>>> On 8/5/20 4:37 PM, Joe Wang wrote: >>>>>> Hello, >>>>>> >>>>>> Please review a fix for the hashCode generation. >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8246816 >>>>>> >>>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev/ >>>>>> >>>>>> Thanks, >>>>>> Joe >>>>>> >>>> >> > From mandy.chung at oracle.com Fri Aug 7 17:28:20 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 7 Aug 2020 10:28:20 -0700 Subject: RFR (JDK 15) 8251276: two jdeps files have extra whitespace in copyright header In-Reply-To: References: Message-ID: <87699caa-4f4b-29d5-c685-62aa267579d5@oracle.com> Hi Sundar, Another alternative is to backout the fix for JDK-8248299.?? This needs approval. Mandy On 8/6/20 10:01 PM, sundararajan.athijegannathan at oracle.com wrote: > Please review. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8251276 > > Webrev: http://cr.openjdk.java.net/~sundar/8251276/webrev.00/index.html > > Thanks, > > -Sundar > From naoto.sato at oracle.com Fri Aug 7 17:50:22 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Fri, 7 Aug 2020 10:50:22 -0700 Subject: RFR [16/java.xml] 8246816: XMLGregorianCalendar.hashCode() produces far too many identical hashes In-Reply-To: <2ce56d2c-9ce9-a1a2-ae1d-7422c264b37f@oracle.com> References: <058367db-dca5-76e9-7e61-049173c765d9@oracle.com> <0e2935f7-4c1f-8782-0f51-12866d66316b@oracle.com> <274e1322-844a-5e9f-c6d7-3784ae937572@oracle.com> <2ce56d2c-9ce9-a1a2-ae1d-7422c264b37f@oracle.com> Message-ID: Hi Joe, If the new XMLGregorianCalendarImpl.hashCode() just calls super.hashCode(), could it be removed entirely? Also XMLGregorianCalendarImpl.equals() seems to do the same thing as its parent. Could it also be removed? Naoto On 8/7/20 10:16 AM, Joe Wang wrote: > Naoto, Roger, > > Sorry have to come back with another iteration. It looks like the > addition of getMillisecond() over the original implementation does > change things. > > In particular, getMillisecond() returns FIELD_UNDEFINED when > fractionalSecond == null (or not specified). But the normalization > process will set it to zero after a mathematics adjustment. Now this > will fail when comparing with a datetime that does not need to be > normalized and thereby still returns FIELD_UNDEFINED (FIELD_UNDEFINED = > Integer.MIN_VALUE). This situation is demonstrated in tests 3 at line 56 > and 4 at line 58. > > > The culprit is XMLGregorianCalendarImpl::normalizeToTimezone. In the > original implementation, XMLGregorianCalendarImpl was calling the > private normalizeToTimezone while XMLGregorianCalendar the public > normalize(). The later will do the right thing to set the Millisecond > back to UNDEFINED. > > In webrev_03, XMLGregorianCalendarImpl now calls super.hashCode() since > the above was the only difference between the hashCode() impls of > XMLGregorianCalendarImpl and XMLGregorianCalendar. > > webrev_03: > http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_03/ > > Thanks, > Joe > > > On 8/6/20 1:51 PM, Joe Wang wrote: >> Thanks Naoto, Roger!? I'll prepare push based on version 01. >> >> Best, >> Joe >> >> On 8/6/20 1:29 PM, Roger Riggs wrote: >>> +1 >>> >>> On 8/6/20 4:18 PM, naoto.sato at oracle.com wrote: >>>> Hi Joe, thank you for looking it into further. >>>> >>>> Yes, I agree the chances of collision is very rare, as >>>> sub-millisecond difference in two objects. So fine with the version 01. >>>> >>>> Naoto >>>> >>>> On 8/6/20 12:18 PM, Joe Wang wrote: >>>>> Thanks Naoto, Roger for the review! >>>>> >>>>> For Naoto's concern about the equals method using EonAndYear and >>>>> FractionalSecond, I did cut corners using the all int getXXX >>>>> method. The rational was: it fulfills the equals-hashcode contract >>>>> just fine, is consistent with the existing implementation, and >>>>> concise. This API was introduced since 1.5, but I have yet to see >>>>> any usage of EonAndYear, and very rarely FractionalSecond. The >>>>> chances collisions occur due to these differences are very low. >>>>> >>>>> But I understand your concern. To be precise or exact as equals(), >>>>> we would need to call getFractionalSecond and EonAndYear. Here's >>>>> what that would look like: >>>>> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_02/ >>>>> >>>>> To me, I like version 1 for the reasons above: >>>>> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_01/ >>>>> >>>>> What would you think? >>>>> >>>>> Regards, >>>>> Joe >>>>> >>>>> On 8/5/20 6:13 PM, naoto.sato at oracle.com wrote: >>>>>> Hi Joe, >>>>>> >>>>>> For the consistency with equals(), just wondering if the >>>>>> sub-second element should be obtained with getFractionalSecond() >>>>>> instead of getMillisecond(), as the equals() subsequently calls it >>>>>> in internalCompare() method. Also should it call getEonAndYear() >>>>>> appropriately for the year? >>>>>> >>>>>> Naoto >>>>>> >>>>>> On 8/5/20 4:37 PM, Joe Wang wrote: >>>>>>> Hello, >>>>>>> >>>>>>> Please review a fix for the hashCode generation. >>>>>>> >>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8246816 >>>>>>> >>>>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev/ >>>>>>> >>>>>>> Thanks, >>>>>>> Joe >>>>>>> >>>>> >>> >> > From brian.burkhalter at oracle.com Fri Aug 7 17:57:28 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 7 Aug 2020 10:57:28 -0700 Subject: [Ping] Re: 8249703: test/jdk/java/io/File/GetXSpace.java fails on macos In-Reply-To: References: Message-ID: Hi Naoto, Good point. Here?s an updated version: http://cr.openjdk.java.net/~bpb/8249703/webrev.01/. The number of blocks is not available via a dedicated API call and so has to be calculated. Thanks, Brian > On Aug 7, 2020, at 9:58 AM, naoto.sato at oracle.com wrote: > > Looks fine, Brian. > > I might add the condition if the block number is odd. > > Naoto > > On 8/7/20 8:33 AM, Brian Burkhalter wrote: >>> On Jul 31, 2020, at 10:42 AM, Brian Burkhalter wrote: >>> >>> On macOS, the number of 1024 byte blocks appears to be incorrectly calculated by 'df' using integer division by 2 of the number of 512 byte blocks, resulting in a size smaller than the actual value when the number of blocks is odd. >>> >>> Possibly instead of having the IS_MAC and IS_WIN constants it should use the equivalent methods in jdk.test.lib.Platform? >>> >>> Thanks, >>> >>> Brian >>> >>> https://bugs.openjdk.java.net/browse/JDK-8249703 >>> http://cr.openjdk.java.net/~bpb/8249703/webrev.00/ From naoto.sato at oracle.com Fri Aug 7 18:04:25 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Fri, 7 Aug 2020 11:04:25 -0700 Subject: [Ping] Re: 8249703: test/jdk/java/io/File/GetXSpace.java fails on macos In-Reply-To: References: Message-ID: Looks good. Thank you for the change. Naoto On 8/7/20 10:57 AM, Brian Burkhalter wrote: > Hi Naoto, > > Good point. Here?s an updated version: http://cr.openjdk.java.net/~bpb/8249703/webrev.01/. The number of blocks is not available via a dedicated API call and so has to be calculated. > > Thanks, > > Brian > >> On Aug 7, 2020, at 9:58 AM, naoto.sato at oracle.com wrote: >> >> Looks fine, Brian. >> >> I might add the condition if the block number is odd. >> >> Naoto >> >> On 8/7/20 8:33 AM, Brian Burkhalter wrote: >>>> On Jul 31, 2020, at 10:42 AM, Brian Burkhalter wrote: >>>> >>>> On macOS, the number of 1024 byte blocks appears to be incorrectly calculated by 'df' using integer division by 2 of the number of 512 byte blocks, resulting in a size smaller than the actual value when the number of blocks is odd. >>>> >>>> Possibly instead of having the IS_MAC and IS_WIN constants it should use the equivalent methods in jdk.test.lib.Platform? >>>> >>>> Thanks, >>>> >>>> Brian >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8249703 >>>> http://cr.openjdk.java.net/~bpb/8249703/webrev.00/ > From brian.burkhalter at oracle.com Fri Aug 7 18:12:57 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 7 Aug 2020 11:12:57 -0700 Subject: [Ping] Re: 8249703: test/jdk/java/io/File/GetXSpace.java fails on macos In-Reply-To: References: Message-ID: <761D32B1-1506-4A1A-ABC2-77DED65C1158@oracle.com> Thanks for the review! Brian > On Aug 7, 2020, at 11:04 AM, naoto.sato at oracle.com wrote: > > Looks good. Thank you for the change. From lois.foltan at oracle.com Fri Aug 7 19:06:24 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Fri, 7 Aug 2020 15:06:24 -0400 Subject: RFR(L) 8244778 Archive full module graph in CDS In-Reply-To: References: Message-ID: <9e6b0043-65a1-dd97-a3d2-33679c8048d4@oracle.com> Hi Ioi, Overall looks promising.? I have some review comments below, but not a complete review.? I concentrated on the JVM side primarily how the archived module graph is read in, how the ModuleEntry and PackageEntry tables are created from the archive for the 3 builtin loaders and potential timing issues during start up. On 7/22/2020 3:36 PM, Ioi Lam wrote: > https://bugs.openjdk.java.net/browse/JDK-8244778 > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/ > > > Please review this patch that stores the full module graph in the CDS > archive heap. This reduces the initialization time of the basic JVM by > about 22%: > > $ perf stat -r 100 bin/java -version > before: 98,219,329 instructions 0.03971 secs elapsed (+- 0.44%) > after:? 55,835,815 instructions 0.03109 secs elapsed (+- 0.65%) > > [1] Start with ModuleBootstrap.java. The current implementation is > ??? quite restrictive: the archived module graph is used only when no > ??? module options are specified. > > ??? See ModuleBootstrap.mayUseArchivedBootLayer(). > > ??? We can probably support options such as main module and module path > ??? in a future RFE. Yes, I noticed the restrictions.? The JBS issue discusses the possibility of using the dumped module graph in the event that the value of the options are the same.? I think for this implementation to be viable and have a positive impact you should consider comparing at least the options --add-modules, --add-exports and --add-reads. > > [2] In the current JDK implementation, there is no single object > ??? that represents "the module graph". Most of the information > ??? is stored in the archive bootLayer object, but a few additional > ??? restoration operations need to be performed: > > ??? + See ModuleBootstrap.getArchivedBootLayer() > ??? + Some static fields need to be archived/restored in > ????? Module.java, BuiltinClassLoader.java, ClassLoaders.java > ????? and BootLoader.java > > [3] I ran into a complication with two loader instances of > ??? PlatformClassLoader and AppClassLoader. They are stored in > ??? multiple tables inside the module graph (e.g., > ??? BuiltinClassLoader$LoadedModule) so I cannot easily recreate > ??? them at runtime. > > ??? However, these two loaders contain information specific to the > ??? dump time VM lifecycle (such as the classes that were loaded > ??? during CDS dumping) that need to be scrubbed. I couldn't find an > ??? elegant way of doing this, so I added a private "resetArchivedStates" > ??? method to a few classes. They are called inside > ??? HeapShared::reset_archived_object_states(). > > [4] Related native data structures (PackageEntry and ModuleEntry) > ??? are also archived. Start with classLoaderData.cpp > > Passes mach5 tiers 1-4. I will test with additional tiers. > > Thanks > - Ioi classfile/classLoader.cpp - line #1644 pulling the javabase_moduleEntry() check outside of the Module_lock maybe problematic if after you check it another ? thread initializes in the time between the check and the obtaining of the Module_lock on line #1645. classfile/classLoader.hpp - somewhere in ArchivedClassLoaderData there should be an assert to make sure that the CLD, whose package entries and module entries are being archived is not a "_has_class_mirror_holder" CLD for a weakly defined hidden class.? Those dedicated CLDs should never have package entries or module entries. classfile/moduleEntry.cpp - line #400, typo "conver" --> "convert" - line #500, maybe sort if n is greater than 1 instead of 0 (same comment for packageEntry.cpp line #270) classfile/systemDictionary.cpp - It looks like the PackageEntry and ModuleEntry tables for the system and platform class loaders are? added within SystemDictionary::compute_java_loaders() which is called from Thread::create_vm() but after the call in Thread::create_vm() to call_initPhase2 which is when Universe::_module_initialized is set to true.? Did I read this correctly?? If yes, then I think you have to be sure that Universe::_module_initialized is not set until the module graph for the 3 builtin loaders is loaded from the archive. memory/filemap.cpp - line #237 typo "modue" --> "module" oops/instanceKlass.cpp - line #2545, comment "TO DO? -- point it to the archived PackageEntry (JDK-8249262)" are you thinking that since the module graph is read in ahead of time that it can be set when an InstanceKlass is created?? There is a point before java.base is defined that InstanceKlass::set_package takes into account that could be a timing issue. Clarification/timing questions: - Since the CDS support for capturing the module graph does not archive unnamed modules, I assume Modules::set_bootloader_unnamed_module() is still called.? Is this true or does the unnamed module for the boot loader get archived? - There are some checks in modules.cpp that are valuable to have during bootstrapping.? For example, packages that are encountered during bootstrapping before java.base is defined are put into the java.base module but then verified after java.base is defined via verify_javabase_packages.? So at what point in the bootstrapping process does the boot loader's archived module's become known? Could classes have been defined to the VM before this point?? Then their packages will have to be verified to be sure that they are indeed packages within java.base.? Consider looking at other checks that occur within classfile/modules.cpp as well. I may have more review comments as I continue to look at this! Thanks, Lois From huizhe.wang at oracle.com Fri Aug 7 19:13:53 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Fri, 7 Aug 2020 12:13:53 -0700 Subject: RFR [16/java.xml] 8246816: XMLGregorianCalendar.hashCode() produces far too many identical hashes In-Reply-To: References: <058367db-dca5-76e9-7e61-049173c765d9@oracle.com> <0e2935f7-4c1f-8782-0f51-12866d66316b@oracle.com> <274e1322-844a-5e9f-c6d7-3784ae937572@oracle.com> <2ce56d2c-9ce9-a1a2-ae1d-7422c264b37f@oracle.com> Message-ID: Hi Naoto, Why not :-). I kept it to go with equals(). Removing both now. Added a reference comparison as did in the Impl class. http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_04/ Joe On 8/7/20 10:50 AM, naoto.sato at oracle.com wrote: > Hi Joe, > > If the new XMLGregorianCalendarImpl.hashCode() just calls > super.hashCode(), could it be removed entirely? Also > XMLGregorianCalendarImpl.equals() seems to do the same thing as its > parent. Could it also be removed? > > Naoto > > On 8/7/20 10:16 AM, Joe Wang wrote: >> Naoto, Roger, >> >> Sorry have to come back with another iteration. It looks like the >> addition of getMillisecond() over the original implementation does >> change things. >> >> In particular, getMillisecond() returns FIELD_UNDEFINED when >> fractionalSecond == null (or not specified). But the normalization >> process will set it to zero after a mathematics adjustment. Now this >> will fail when comparing with a datetime that does not need to be >> normalized and thereby still returns FIELD_UNDEFINED (FIELD_UNDEFINED >> = Integer.MIN_VALUE). This situation is demonstrated in tests 3 at >> line 56 and 4 at line 58. >> >> >> The culprit is XMLGregorianCalendarImpl::normalizeToTimezone. In the >> original implementation, XMLGregorianCalendarImpl was calling the >> private normalizeToTimezone while XMLGregorianCalendar the public >> normalize(). The later will do the right thing to set the Millisecond >> back to UNDEFINED. >> >> In webrev_03, XMLGregorianCalendarImpl now calls super.hashCode() >> since the above was the only difference between the hashCode() impls >> of XMLGregorianCalendarImpl and XMLGregorianCalendar. >> >> webrev_03: >> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_03/ >> >> Thanks, >> Joe >> >> >> On 8/6/20 1:51 PM, Joe Wang wrote: >>> Thanks Naoto, Roger!? I'll prepare push based on version 01. >>> >>> Best, >>> Joe >>> >>> On 8/6/20 1:29 PM, Roger Riggs wrote: >>>> +1 >>>> >>>> On 8/6/20 4:18 PM, naoto.sato at oracle.com wrote: >>>>> Hi Joe, thank you for looking it into further. >>>>> >>>>> Yes, I agree the chances of collision is very rare, as >>>>> sub-millisecond difference in two objects. So fine with the >>>>> version 01. >>>>> >>>>> Naoto >>>>> >>>>> On 8/6/20 12:18 PM, Joe Wang wrote: >>>>>> Thanks Naoto, Roger for the review! >>>>>> >>>>>> For Naoto's concern about the equals method using EonAndYear and >>>>>> FractionalSecond, I did cut corners using the all int getXXX >>>>>> method. The rational was: it fulfills the equals-hashcode >>>>>> contract just fine, is consistent with the existing >>>>>> implementation, and concise. This API was introduced since 1.5, >>>>>> but I have yet to see any usage of EonAndYear, and very rarely >>>>>> FractionalSecond. The chances collisions occur due to these >>>>>> differences are very low. >>>>>> >>>>>> But I understand your concern. To be precise or exact as >>>>>> equals(), we would need to call getFractionalSecond and >>>>>> EonAndYear. Here's what that would look like: >>>>>> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_02/ >>>>>> >>>>>> To me, I like version 1 for the reasons above: >>>>>> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_01/ >>>>>> >>>>>> What would you think? >>>>>> >>>>>> Regards, >>>>>> Joe >>>>>> >>>>>> On 8/5/20 6:13 PM, naoto.sato at oracle.com wrote: >>>>>>> Hi Joe, >>>>>>> >>>>>>> For the consistency with equals(), just wondering if the >>>>>>> sub-second element should be obtained with getFractionalSecond() >>>>>>> instead of getMillisecond(), as the equals() subsequently calls >>>>>>> it in internalCompare() method. Also should it call >>>>>>> getEonAndYear() appropriately for the year? >>>>>>> >>>>>>> Naoto >>>>>>> >>>>>>> On 8/5/20 4:37 PM, Joe Wang wrote: >>>>>>>> Hello, >>>>>>>> >>>>>>>> Please review a fix for the hashCode generation. >>>>>>>> >>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8246816 >>>>>>>> >>>>>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev/ >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Joe >>>>>>>> >>>>>> >>>> >>> >> From Roger.Riggs at oracle.com Fri Aug 7 19:44:14 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Fri, 7 Aug 2020 15:44:14 -0400 Subject: RFR [16/java.xml] 8246816: XMLGregorianCalendar.hashCode() produces far too many identical hashes In-Reply-To: References: <058367db-dca5-76e9-7e61-049173c765d9@oracle.com> <0e2935f7-4c1f-8782-0f51-12866d66316b@oracle.com> <274e1322-844a-5e9f-c6d7-3784ae937572@oracle.com> <2ce56d2c-9ce9-a1a2-ae1d-7422c264b37f@oracle.com> Message-ID: <45962c19-4929-0026-49d2-1e4fef251acc@oracle.com> Hi, Simpler is better. Does removing the override change the javadoc?? That might be considered a spec change. Otherwise, looks good. Roger On 8/7/20 3:13 PM, Joe Wang wrote: > Hi Naoto, > > Why not :-). I kept it to go with equals(). Removing both now. Added a > reference comparison as did in the Impl class. > > http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_04/ > > Joe > > On 8/7/20 10:50 AM, naoto.sato at oracle.com wrote: >> Hi Joe, >> >> If the new XMLGregorianCalendarImpl.hashCode() just calls >> super.hashCode(), could it be removed entirely? Also >> XMLGregorianCalendarImpl.equals() seems to do the same thing as its >> parent. Could it also be removed? >> >> Naoto >> >> On 8/7/20 10:16 AM, Joe Wang wrote: >>> Naoto, Roger, >>> >>> Sorry have to come back with another iteration. It looks like the >>> addition of getMillisecond() over the original implementation does >>> change things. >>> >>> In particular, getMillisecond() returns FIELD_UNDEFINED when >>> fractionalSecond == null (or not specified). But the normalization >>> process will set it to zero after a mathematics adjustment. Now this >>> will fail when comparing with a datetime that does not need to be >>> normalized and thereby still returns FIELD_UNDEFINED >>> (FIELD_UNDEFINED = Integer.MIN_VALUE). This situation is >>> demonstrated in tests 3 at line 56 and 4 at line 58. >>> >>> >>> The culprit is XMLGregorianCalendarImpl::normalizeToTimezone. In the >>> original implementation, XMLGregorianCalendarImpl was calling the >>> private normalizeToTimezone while XMLGregorianCalendar the public >>> normalize(). The later will do the right thing to set the >>> Millisecond back to UNDEFINED. >>> >>> In webrev_03, XMLGregorianCalendarImpl now calls super.hashCode() >>> since the above was the only difference between the hashCode() impls >>> of XMLGregorianCalendarImpl and XMLGregorianCalendar. >>> >>> webrev_03: >>> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_03/ >>> >>> Thanks, >>> Joe >>> >>> >>> On 8/6/20 1:51 PM, Joe Wang wrote: >>>> Thanks Naoto, Roger!? I'll prepare push based on version 01. >>>> >>>> Best, >>>> Joe >>>> >>>> On 8/6/20 1:29 PM, Roger Riggs wrote: >>>>> +1 >>>>> >>>>> On 8/6/20 4:18 PM, naoto.sato at oracle.com wrote: >>>>>> Hi Joe, thank you for looking it into further. >>>>>> >>>>>> Yes, I agree the chances of collision is very rare, as >>>>>> sub-millisecond difference in two objects. So fine with the >>>>>> version 01. >>>>>> >>>>>> Naoto >>>>>> >>>>>> On 8/6/20 12:18 PM, Joe Wang wrote: >>>>>>> Thanks Naoto, Roger for the review! >>>>>>> >>>>>>> For Naoto's concern about the equals method using EonAndYear and >>>>>>> FractionalSecond, I did cut corners using the all int getXXX >>>>>>> method. The rational was: it fulfills the equals-hashcode >>>>>>> contract just fine, is consistent with the existing >>>>>>> implementation, and concise. This API was introduced since 1.5, >>>>>>> but I have yet to see any usage of EonAndYear, and very rarely >>>>>>> FractionalSecond. The chances collisions occur due to these >>>>>>> differences are very low. >>>>>>> >>>>>>> But I understand your concern. To be precise or exact as >>>>>>> equals(), we would need to call getFractionalSecond and >>>>>>> EonAndYear. Here's what that would look like: >>>>>>> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_02/ >>>>>>> >>>>>>> To me, I like version 1 for the reasons above: >>>>>>> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_01/ >>>>>>> >>>>>>> What would you think? >>>>>>> >>>>>>> Regards, >>>>>>> Joe >>>>>>> >>>>>>> On 8/5/20 6:13 PM, naoto.sato at oracle.com wrote: >>>>>>>> Hi Joe, >>>>>>>> >>>>>>>> For the consistency with equals(), just wondering if the >>>>>>>> sub-second element should be obtained with >>>>>>>> getFractionalSecond() instead of getMillisecond(), as the >>>>>>>> equals() subsequently calls it in internalCompare() method. >>>>>>>> Also should it call getEonAndYear() appropriately for the year? >>>>>>>> >>>>>>>> Naoto >>>>>>>> >>>>>>>> On 8/5/20 4:37 PM, Joe Wang wrote: >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> Please review a fix for the hashCode generation. >>>>>>>>> >>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8246816 >>>>>>>>> >>>>>>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev/ >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Joe >>>>>>>>> >>>>>>> >>>>> >>>> >>> > From naoto.sato at oracle.com Fri Aug 7 19:45:13 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Fri, 7 Aug 2020 12:45:13 -0700 Subject: RFR [16/java.xml] 8246816: XMLGregorianCalendar.hashCode() produces far too many identical hashes In-Reply-To: References: <058367db-dca5-76e9-7e61-049173c765d9@oracle.com> <0e2935f7-4c1f-8782-0f51-12866d66316b@oracle.com> <274e1322-844a-5e9f-c6d7-3784ae937572@oracle.com> <2ce56d2c-9ce9-a1a2-ae1d-7422c264b37f@oracle.com> Message-ID: Looks good. Thank you for the change. Naoto On 8/7/20 12:13 PM, Joe Wang wrote: > Hi Naoto, > > Why not :-). I kept it to go with equals(). Removing both now. Added a > reference comparison as did in the Impl class. > > http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_04/ > > Joe > > On 8/7/20 10:50 AM, naoto.sato at oracle.com wrote: >> Hi Joe, >> >> If the new XMLGregorianCalendarImpl.hashCode() just calls >> super.hashCode(), could it be removed entirely? Also >> XMLGregorianCalendarImpl.equals() seems to do the same thing as its >> parent. Could it also be removed? >> >> Naoto >> >> On 8/7/20 10:16 AM, Joe Wang wrote: >>> Naoto, Roger, >>> >>> Sorry have to come back with another iteration. It looks like the >>> addition of getMillisecond() over the original implementation does >>> change things. >>> >>> In particular, getMillisecond() returns FIELD_UNDEFINED when >>> fractionalSecond == null (or not specified). But the normalization >>> process will set it to zero after a mathematics adjustment. Now this >>> will fail when comparing with a datetime that does not need to be >>> normalized and thereby still returns FIELD_UNDEFINED (FIELD_UNDEFINED >>> = Integer.MIN_VALUE). This situation is demonstrated in tests 3 at >>> line 56 and 4 at line 58. >>> >>> >>> >>> The culprit is XMLGregorianCalendarImpl::normalizeToTimezone. In the >>> original implementation, XMLGregorianCalendarImpl was calling the >>> private normalizeToTimezone while XMLGregorianCalendar the public >>> normalize(). The later will do the right thing to set the Millisecond >>> back to UNDEFINED. >>> >>> In webrev_03, XMLGregorianCalendarImpl now calls super.hashCode() >>> since the above was the only difference between the hashCode() impls >>> of XMLGregorianCalendarImpl and XMLGregorianCalendar. >>> >>> webrev_03: >>> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_03/ >>> >>> Thanks, >>> Joe >>> >>> >>> On 8/6/20 1:51 PM, Joe Wang wrote: >>>> Thanks Naoto, Roger!? I'll prepare push based on version 01. >>>> >>>> Best, >>>> Joe >>>> >>>> On 8/6/20 1:29 PM, Roger Riggs wrote: >>>>> +1 >>>>> >>>>> On 8/6/20 4:18 PM, naoto.sato at oracle.com wrote: >>>>>> Hi Joe, thank you for looking it into further. >>>>>> >>>>>> Yes, I agree the chances of collision is very rare, as >>>>>> sub-millisecond difference in two objects. So fine with the >>>>>> version 01. >>>>>> >>>>>> Naoto >>>>>> >>>>>> On 8/6/20 12:18 PM, Joe Wang wrote: >>>>>>> Thanks Naoto, Roger for the review! >>>>>>> >>>>>>> For Naoto's concern about the equals method using EonAndYear and >>>>>>> FractionalSecond, I did cut corners using the all int getXXX >>>>>>> method. The rational was: it fulfills the equals-hashcode >>>>>>> contract just fine, is consistent with the existing >>>>>>> implementation, and concise. This API was introduced since 1.5, >>>>>>> but I have yet to see any usage of EonAndYear, and very rarely >>>>>>> FractionalSecond. The chances collisions occur due to these >>>>>>> differences are very low. >>>>>>> >>>>>>> But I understand your concern. To be precise or exact as >>>>>>> equals(), we would need to call getFractionalSecond and >>>>>>> EonAndYear. Here's what that would look like: >>>>>>> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_02/ >>>>>>> >>>>>>> To me, I like version 1 for the reasons above: >>>>>>> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_01/ >>>>>>> >>>>>>> What would you think? >>>>>>> >>>>>>> Regards, >>>>>>> Joe >>>>>>> >>>>>>> On 8/5/20 6:13 PM, naoto.sato at oracle.com wrote: >>>>>>>> Hi Joe, >>>>>>>> >>>>>>>> For the consistency with equals(), just wondering if the >>>>>>>> sub-second element should be obtained with getFractionalSecond() >>>>>>>> instead of getMillisecond(), as the equals() subsequently calls >>>>>>>> it in internalCompare() method. Also should it call >>>>>>>> getEonAndYear() appropriately for the year? >>>>>>>> >>>>>>>> Naoto >>>>>>>> >>>>>>>> On 8/5/20 4:37 PM, Joe Wang wrote: >>>>>>>>> Hello, >>>>>>>>> >>>>>>>>> Please review a fix for the hashCode generation. >>>>>>>>> >>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8246816 >>>>>>>>> >>>>>>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev/ >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Joe >>>>>>>>> >>>>>>> >>>>> >>>> >>> > From huizhe.wang at oracle.com Fri Aug 7 20:26:37 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Fri, 7 Aug 2020 13:26:37 -0700 Subject: RFR [16/java.xml] 8246816: XMLGregorianCalendar.hashCode() produces far too many identical hashes In-Reply-To: <45962c19-4929-0026-49d2-1e4fef251acc@oracle.com> References: <058367db-dca5-76e9-7e61-049173c765d9@oracle.com> <0e2935f7-4c1f-8782-0f51-12866d66316b@oracle.com> <274e1322-844a-5e9f-c6d7-3784ae937572@oracle.com> <2ce56d2c-9ce9-a1a2-ae1d-7422c264b37f@oracle.com> <45962c19-4929-0026-49d2-1e4fef251acc@oracle.com> Message-ID: Thanks Roger! Simpler indeed, one less place for mistakes ;-) Removing the override did not change the javadoc for the public API. Joe On 8/7/20 12:44 PM, Roger Riggs wrote: > Hi, > > Simpler is better. > > Does removing the override change the javadoc?? That might be > considered a spec change. > > Otherwise, looks good. > > Roger > > > On 8/7/20 3:13 PM, Joe Wang wrote: >> Hi Naoto, >> >> Why not :-). I kept it to go with equals(). Removing both now. Added >> a reference comparison as did in the Impl class. >> >> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_04/ >> >> Joe >> >> On 8/7/20 10:50 AM, naoto.sato at oracle.com wrote: >>> Hi Joe, >>> >>> If the new XMLGregorianCalendarImpl.hashCode() just calls >>> super.hashCode(), could it be removed entirely? Also >>> XMLGregorianCalendarImpl.equals() seems to do the same thing as its >>> parent. Could it also be removed? >>> >>> Naoto >>> >>> On 8/7/20 10:16 AM, Joe Wang wrote: >>>> Naoto, Roger, >>>> >>>> Sorry have to come back with another iteration. It looks like the >>>> addition of getMillisecond() over the original implementation does >>>> change things. >>>> >>>> In particular, getMillisecond() returns FIELD_UNDEFINED when >>>> fractionalSecond == null (or not specified). But the normalization >>>> process will set it to zero after a mathematics adjustment. Now >>>> this will fail when comparing with a datetime that does not need to >>>> be normalized and thereby still returns FIELD_UNDEFINED >>>> (FIELD_UNDEFINED = Integer.MIN_VALUE). This situation is >>>> demonstrated in tests 3 at line 56 and 4 at line 58. >>>> >>>> >>>> The culprit is XMLGregorianCalendarImpl::normalizeToTimezone. In >>>> the original implementation, XMLGregorianCalendarImpl was calling >>>> the private normalizeToTimezone while XMLGregorianCalendar the >>>> public normalize(). The later will do the right thing to set the >>>> Millisecond back to UNDEFINED. >>>> >>>> In webrev_03, XMLGregorianCalendarImpl now calls super.hashCode() >>>> since the above was the only difference between the hashCode() >>>> impls of XMLGregorianCalendarImpl and XMLGregorianCalendar. >>>> >>>> webrev_03: >>>> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_03/ >>>> >>>> Thanks, >>>> Joe >>>> >>>> >>>> On 8/6/20 1:51 PM, Joe Wang wrote: >>>>> Thanks Naoto, Roger!? I'll prepare push based on version 01. >>>>> >>>>> Best, >>>>> Joe >>>>> >>>>> On 8/6/20 1:29 PM, Roger Riggs wrote: >>>>>> +1 >>>>>> >>>>>> On 8/6/20 4:18 PM, naoto.sato at oracle.com wrote: >>>>>>> Hi Joe, thank you for looking it into further. >>>>>>> >>>>>>> Yes, I agree the chances of collision is very rare, as >>>>>>> sub-millisecond difference in two objects. So fine with the >>>>>>> version 01. >>>>>>> >>>>>>> Naoto >>>>>>> >>>>>>> On 8/6/20 12:18 PM, Joe Wang wrote: >>>>>>>> Thanks Naoto, Roger for the review! >>>>>>>> >>>>>>>> For Naoto's concern about the equals method using EonAndYear >>>>>>>> and FractionalSecond, I did cut corners using the all int >>>>>>>> getXXX method. The rational was: it fulfills the >>>>>>>> equals-hashcode contract just fine, is consistent with the >>>>>>>> existing implementation, and concise. This API was introduced >>>>>>>> since 1.5, but I have yet to see any usage of EonAndYear, and >>>>>>>> very rarely FractionalSecond. The chances collisions occur due >>>>>>>> to these differences are very low. >>>>>>>> >>>>>>>> But I understand your concern. To be precise or exact as >>>>>>>> equals(), we would need to call getFractionalSecond and >>>>>>>> EonAndYear. Here's what that would look like: >>>>>>>> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_02/ >>>>>>>> >>>>>>>> To me, I like version 1 for the reasons above: >>>>>>>> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_01/ >>>>>>>> >>>>>>>> What would you think? >>>>>>>> >>>>>>>> Regards, >>>>>>>> Joe >>>>>>>> >>>>>>>> On 8/5/20 6:13 PM, naoto.sato at oracle.com wrote: >>>>>>>>> Hi Joe, >>>>>>>>> >>>>>>>>> For the consistency with equals(), just wondering if the >>>>>>>>> sub-second element should be obtained with >>>>>>>>> getFractionalSecond() instead of getMillisecond(), as the >>>>>>>>> equals() subsequently calls it in internalCompare() method. >>>>>>>>> Also should it call getEonAndYear() appropriately for the year? >>>>>>>>> >>>>>>>>> Naoto >>>>>>>>> >>>>>>>>> On 8/5/20 4:37 PM, Joe Wang wrote: >>>>>>>>>> Hello, >>>>>>>>>> >>>>>>>>>> Please review a fix for the hashCode generation. >>>>>>>>>> >>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8246816 >>>>>>>>>> >>>>>>>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev/ >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Joe >>>>>>>>>> >>>>>>>> >>>>>> >>>>> >>>> >> > From huizhe.wang at oracle.com Fri Aug 7 20:26:57 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Fri, 7 Aug 2020 13:26:57 -0700 Subject: RFR [16/java.xml] 8246816: XMLGregorianCalendar.hashCode() produces far too many identical hashes In-Reply-To: References: <058367db-dca5-76e9-7e61-049173c765d9@oracle.com> <0e2935f7-4c1f-8782-0f51-12866d66316b@oracle.com> <274e1322-844a-5e9f-c6d7-3784ae937572@oracle.com> <2ce56d2c-9ce9-a1a2-ae1d-7422c264b37f@oracle.com> Message-ID: Thanks Naoto! Joe On 8/7/20 12:45 PM, naoto.sato at oracle.com wrote: > Looks good. Thank you for the change. > > Naoto > > On 8/7/20 12:13 PM, Joe Wang wrote: >> Hi Naoto, >> >> Why not :-). I kept it to go with equals(). Removing both now. Added >> a reference comparison as did in the Impl class. >> >> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_04/ >> >> Joe >> >> On 8/7/20 10:50 AM, naoto.sato at oracle.com wrote: >>> Hi Joe, >>> >>> If the new XMLGregorianCalendarImpl.hashCode() just calls >>> super.hashCode(), could it be removed entirely? Also >>> XMLGregorianCalendarImpl.equals() seems to do the same thing as its >>> parent. Could it also be removed? >>> >>> Naoto >>> >>> On 8/7/20 10:16 AM, Joe Wang wrote: >>>> Naoto, Roger, >>>> >>>> Sorry have to come back with another iteration. It looks like the >>>> addition of getMillisecond() over the original implementation does >>>> change things. >>>> >>>> In particular, getMillisecond() returns FIELD_UNDEFINED when >>>> fractionalSecond == null (or not specified). But the normalization >>>> process will set it to zero after a mathematics adjustment. Now >>>> this will fail when comparing with a datetime that does not need to >>>> be normalized and thereby still returns FIELD_UNDEFINED >>>> (FIELD_UNDEFINED = Integer.MIN_VALUE). This situation is >>>> demonstrated in tests 3 at line 56 and 4 at line 58. >>>> >>>> >>>> >>>> The culprit is XMLGregorianCalendarImpl::normalizeToTimezone. In >>>> the original implementation, XMLGregorianCalendarImpl was calling >>>> the private normalizeToTimezone while XMLGregorianCalendar the >>>> public normalize(). The later will do the right thing to set the >>>> Millisecond back to UNDEFINED. >>>> >>>> In webrev_03, XMLGregorianCalendarImpl now calls super.hashCode() >>>> since the above was the only difference between the hashCode() >>>> impls of XMLGregorianCalendarImpl and XMLGregorianCalendar. >>>> >>>> webrev_03: >>>> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_03/ >>>> >>>> Thanks, >>>> Joe >>>> >>>> >>>> On 8/6/20 1:51 PM, Joe Wang wrote: >>>>> Thanks Naoto, Roger!? I'll prepare push based on version 01. >>>>> >>>>> Best, >>>>> Joe >>>>> >>>>> On 8/6/20 1:29 PM, Roger Riggs wrote: >>>>>> +1 >>>>>> >>>>>> On 8/6/20 4:18 PM, naoto.sato at oracle.com wrote: >>>>>>> Hi Joe, thank you for looking it into further. >>>>>>> >>>>>>> Yes, I agree the chances of collision is very rare, as >>>>>>> sub-millisecond difference in two objects. So fine with the >>>>>>> version 01. >>>>>>> >>>>>>> Naoto >>>>>>> >>>>>>> On 8/6/20 12:18 PM, Joe Wang wrote: >>>>>>>> Thanks Naoto, Roger for the review! >>>>>>>> >>>>>>>> For Naoto's concern about the equals method using EonAndYear >>>>>>>> and FractionalSecond, I did cut corners using the all int >>>>>>>> getXXX method. The rational was: it fulfills the >>>>>>>> equals-hashcode contract just fine, is consistent with the >>>>>>>> existing implementation, and concise. This API was introduced >>>>>>>> since 1.5, but I have yet to see any usage of EonAndYear, and >>>>>>>> very rarely FractionalSecond. The chances collisions occur due >>>>>>>> to these differences are very low. >>>>>>>> >>>>>>>> But I understand your concern. To be precise or exact as >>>>>>>> equals(), we would need to call getFractionalSecond and >>>>>>>> EonAndYear. Here's what that would look like: >>>>>>>> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_02/ >>>>>>>> >>>>>>>> To me, I like version 1 for the reasons above: >>>>>>>> http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev_01/ >>>>>>>> >>>>>>>> What would you think? >>>>>>>> >>>>>>>> Regards, >>>>>>>> Joe >>>>>>>> >>>>>>>> On 8/5/20 6:13 PM, naoto.sato at oracle.com wrote: >>>>>>>>> Hi Joe, >>>>>>>>> >>>>>>>>> For the consistency with equals(), just wondering if the >>>>>>>>> sub-second element should be obtained with >>>>>>>>> getFractionalSecond() instead of getMillisecond(), as the >>>>>>>>> equals() subsequently calls it in internalCompare() method. >>>>>>>>> Also should it call getEonAndYear() appropriately for the year? >>>>>>>>> >>>>>>>>> Naoto >>>>>>>>> >>>>>>>>> On 8/5/20 4:37 PM, Joe Wang wrote: >>>>>>>>>> Hello, >>>>>>>>>> >>>>>>>>>> Please review a fix for the hashCode generation. >>>>>>>>>> >>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8246816 >>>>>>>>>> >>>>>>>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk16/8246816/webrev/ >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Joe >>>>>>>>>> >>>>>>>> >>>>>> >>>>> >>>> >> From mandy.chung at oracle.com Fri Aug 7 21:09:07 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 7 Aug 2020 14:09:07 -0700 Subject: RFR (JDK 15) 8251276: two jdeps files have extra whitespace in copyright header In-Reply-To: <87699caa-4f4b-29d5-c685-62aa267579d5@oracle.com> References: <87699caa-4f4b-29d5-c685-62aa267579d5@oracle.com> Message-ID: <306f4967-5c7e-188e-e1d4-1cafb75a52ca@oracle.com> Hi Sundar, Mark clarifies that either fix (fix the whitespace or backout) requires another RC build.? We should go with your proposed patch to fix the whitespace. I reviewed webrev.00. thanks Mandy On 8/7/20 10:28 AM, Mandy Chung wrote: > Hi Sundar, > > Another alternative is to backout the fix for JDK-8248299.?? This > needs approval. > > Mandy > > On 8/6/20 10:01 PM, sundararajan.athijegannathan at oracle.com wrote: >> Please review. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8251276 >> >> Webrev: http://cr.openjdk.java.net/~sundar/8251276/webrev.00/index.html >> >> Thanks, >> >> -Sundar >> > From brian.burkhalter at oracle.com Fri Aug 7 21:12:26 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 7 Aug 2020 14:12:26 -0700 Subject: RFR: 8245036: DataInputStream.readFully(byte[], int, int) does not throw expected IndexOutOfBoundsExceptions In-Reply-To: <10711294-E57F-4D89-9F51-61DAE4EFC8BF@oracle.com> References: <59a871a4-d676-3b0c-2983-b9b9fc34ca37@gmail.com> <858a70ac-f827-669a-58e2-e5b834aa36f4@gmail.com> <67C3F744-2FE7-4BDD-857F-86E64A1AABFA@oracle.com> <4030EF21-CCE0-48D2-ADB1-7BF7519FEA52@oracle.com> <10711294-E57F-4D89-9F51-61DAE4EFC8BF@oracle.com> Message-ID: > On Aug 7, 2020, at 9:39 AM, Brian Burkhalter wrote: > >> There's one corner case that would not pass, however, so I added testNegativeOffsetZeroLength() to exercise that. It passes on the new source but fails on the old. >> >> I also added two additional corner cases for too large combinations of offset and length. > > This patch looks good. I am running it through our CI tests but I don?t expect to see any problems. If the test run succeeds aside from any failures unrelated to this code then I can push this. Pushed http://hg.openjdk.java.net/jdk/jdk/rev/29212f925331. Thanks, Brian From iris.clark at oracle.com Fri Aug 7 21:13:48 2020 From: iris.clark at oracle.com (Iris Clark) Date: Fri, 7 Aug 2020 14:13:48 -0700 (PDT) Subject: RFR (JDK 15) 8251276: two jdeps files have extra whitespace in copyright header In-Reply-To: <306f4967-5c7e-188e-e1d4-1cafb75a52ca@oracle.com> References: <87699caa-4f4b-29d5-c685-62aa267579d5@oracle.com> <306f4967-5c7e-188e-e1d4-1cafb75a52ca@oracle.com> Message-ID: <696fe455-8b20-4c19-aa17-2152dcc4c717@default> Hi, Sundar. >>> http://cr.openjdk.java.net/~sundar/8251276/webrev.00/index.html These changes look good. Iris From john.r.rose at oracle.com Fri Aug 7 21:35:16 2020 From: john.r.rose at oracle.com (John Rose) Date: Fri, 7 Aug 2020 14:35:16 -0700 Subject: Possible subtle memory model error in ClassValue In-Reply-To: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> References: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> Message-ID: On Aug 7, 2020, at 9:52 AM, Andrew Haley wrote: > > On 8/7/20 4:39 PM, David Lloyd wrote: > > > However, I'm wondering if this isn't a side effect of what appears > > to be an incorrect double-checked lock at lines 374-378 of > > ClassValue.java [1]. In order for the write to the non-volatile > > `cache` field of ClassValueMap, it is my understanding that there > > must be some acquire/release edge from where the variable is > > published to guarantee that all of the writes are visible to the > > read site, and I'm not really sure that the exit-the-constructor > > edge is going to match up with with the synchronized block which > > protects a different non-volatile field. And even if my feeling > > that this is dodgy is valid, I'm not sure whether this NPE is a > > possible outcome of that problem! > > It certainly doesn't look right to me. AFAICS this is classic broken > double-checked locking. It'd need some sort of fence after > constructing the ClassValueMap instance and before publishing it. Y?all are calling my baby ugly but he?s really cute if you look at him from the right angle. (First, a quick question: This bug shows up on x86, right? If so, we probably are not looking at a hardware reordering problem but something stemming from Graal?s reordering of operations compared to C1 and C2, perhaps after a different set of inlining decisions uncharacteristic of C2.) The bug indeed looks like a dropped fence at the end of the constructor for ClassValueMap. (I don?t see an easier way to get a null at the NPE point.) The fence might not actually be dropped, but a write which the fence is supposed to fence might have been reordered after the fence, and after an unlock operation, allowing someone to see the initial null after the unlock. The double-check idiom breaks when the outer check (not synchronized) gets half-baked data and acts on it. This much-maligned idiom would not be broken in the present case under the assumption that a data dependency on type.classValueMap (set in initializeMap) will see a fully initialized value of ClassValueMap.cacheArray. Now we get into the fine print, and I agree there is a bug here. The easy fix is to repair the logic under which that ugly everybody?s-hating-on-it double check idiom would be in fact correct. The fine print requires a couple different things that are not quite fulfilled by the present code, and Graal has either reordered the memory accesses to expose the problem, or it (itself) has a complementary bug. (Or it has an unrelated bug, and you people are *staring* at my baby!) First, as Paul noted, you need a final variable in your class to get the benefit of a fence at the end of the constructor. Oops #1: ClassValueMap doesn?t have *any* finals. That?s awkward. Two fixes for that: (a) add a dummy final, and (b) add a real fence in the constructor. For better luck, maybe put the fence at the end of sizeCache. On machines (not x86) which need fences *before* final reads, another explicit fence should be placed before the unsynchronized read (or else inside the getCache method). By the letter of the JMM, the helpful effects of the fence at the end of the constructor are only guaranteed for references which depend on final variables set by that constructor, because only those final variables get a ?freeze? operation which stabilizes them (and values dependently loaded via them). Throwing in a dummy final is therefore not guaranteed to work. But throwing in a fence will work. One downside of the fence is that the JMM is silent about fences, but the JMM is widely recognized as a partial story, not the full or final story. (Here?s a tidbit of JMM politics: I once heard Doug Lea considering whether maybe all fields should be treated more like final fields. I don?t know if this is still a live idea, but it would make this bug go way, since nearly all constructors would then get fences of some sort.) Here?s a bit of explanatory (non-normative) text from the draft POPL paper for JMM, where the parenthetic comment indicates (I think) the sort of thing that is happening here: > Let us now assume that the acquire and release do happen. As long as > these actions take place after object has been constructed (and there > is no code motion around the end of the constructor), the diffs that > the second processor acquires are guaranteed to reflect the correctly > constructed object. Basically, the synchronization statement is expected to do a release *after* the non-null value is stored. It looks like this is failing due to a lost and/or reordered fence. Second, David says: > I?m not really sure that the exit-the-constructor edge is going to match > up with with the synchronized block which protects a different > non-volatile field. By the letter of the JMM (AFAIUI), this code is coloring way outside the lines. (Yes, I admit it.) The problem is pretty fundamental. I went and swapped in some JMM spec just now (don?t have it in short term memory; go figure), and here?s what I re-learned. In JMM terms, a ?release? is the earlier end of any edge in relation called ?synchronizes-with?. An ?acquire? is the latter end of such an edge. In the JMM this relation applies only to lock, unlock, and volatile reads and writes. Your guess is as good as mine whether other operations (CAS, fences, etc.) participate; the JMM is silent. Crucially, acquires and release do not touch plain fields. This is counter-intuitive for those of us who like to reason with fences. The JMM prevents regular writes from floating past what we like to think of as ?release points? (unlocks) by appealing to processor order inside lock/unlock pairs, and also by appealing to global ordering of multiple lock/unlock pairs (or stuff with volatiles and other corner cases which we will neglect here). In order to work right, a normal write has to come before a release *and* a matching normal read has to come after an acquire, and such an acquire is nothing other than the lock of a later lock/unlock pair, typically in another thread. So the rules which reliably connect normal writes to normal reads in other threads work differently from acquire fences and release fences as we (or just I?) usually think of them. Yes, there are ?acquires? and ?releases? in the JMM. No, they are not primitives but rather descriptions of the way the happens-before relation is constrained by (among other things) lock and unlock actions. On x86, you don?t have to worry about acquires; you just set up the right release fences. In the JMM, there?s no way to get either an acquire or a release without a synchronized statement (or other fancy stuff like volatiles). That?s why double-check is broken in the pure JMM, and why it can be repaired if you add some (non-JMM) fences. So why is this showing up suddenly with Graal? Possibly it?s got a really aggressive interpretation of the JMM, relative to the standard HotSpot JITs. Possibly it?s got a bug. Graal might be allowing the initialization of ClassValueMap.cacheArray to float down past publication of the ClassValueMap on type.classValueMap (in initializeMap). Less likely, it is allowing the initialization to totally escape the lock/unlock pair, something the JMM might allow but the Cookbook advises against (in its ?Can Reorder? table). Perhaps Graal is modeling the ?freeze? operations on finals more accurately (and the Cookbook gives instructions for this). That would allow the ClassValueMap to correctly initialize its frozen finals, but not (unfortunately) the not-frozen cacheArray field. It appears that the new JIT on the block is exposing my neglect for my poor baby, in not putting the right fences around its playpen. Bottom line: Add a release fence before type.cVM is set, and add a (no-op on x86) acquire fence in getCache. Quick test: Add a dummy final to CVM, to see if that makes the bug let go. ? John From alexander.matveev at oracle.com Fri Aug 7 21:54:37 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Fri, 7 Aug 2020 14:54:37 -0700 Subject: RFR: 8248905: [macos] symbolic links not properly resolved Message-ID: <9c0d1f04-3ccf-fa68-0009-d249cf071040@oracle.com> Please review the jpackage fix for bug [1] at [2]. - Fixed by converting path of executable to real path if _NSGetExecutablePath() returned symbolic link. [1] https://bugs.openjdk.java.net/browse/JDK-8248905 [2] http://cr.openjdk.java.net/~almatvee/8248905/webrev.00/ Thanks, Alexander From alexey.semenyuk at oracle.com Fri Aug 7 22:36:42 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Fri, 7 Aug 2020 18:36:42 -0400 Subject: RFR: 8248905: [macos] symbolic links not properly resolved In-Reply-To: <9c0d1f04-3ccf-fa68-0009-d249cf071040@oracle.com> References: <9c0d1f04-3ccf-fa68-0009-d249cf071040@oracle.com> Message-ID: <8898fea2-a1fa-d0aa-1d1c-57f9217ff5d9@oracle.com> Looks good. Minor suggestion, please change --- tstring getRealPath(std::vector in) --- to --- tstring getRealPath(const std::vector& in); --- - Alexey On 8/7/2020 5:54 PM, alexander.matveev at oracle.com wrote: > Please review the jpackage fix for bug [1] at [2]. > > - Fixed by converting path of executable to real path if > _NSGetExecutablePath() returned symbolic link. > > [1] https://bugs.openjdk.java.net/browse/JDK-8248905 > [2] http://cr.openjdk.java.net/~almatvee/8248905/webrev.00/ > > Thanks, > Alexander From andy.herrick at oracle.com Fri Aug 7 22:53:42 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Fri, 7 Aug 2020 18:53:42 -0400 Subject: RFR: 8248905: [macos] symbolic links not properly resolved In-Reply-To: <8898fea2-a1fa-d0aa-1d1c-57f9217ff5d9@oracle.com> References: <9c0d1f04-3ccf-fa68-0009-d249cf071040@oracle.com> <8898fea2-a1fa-d0aa-1d1c-57f9217ff5d9@oracle.com> Message-ID: <9b70b2b3-326a-d5f2-c367-4e023a9868d2@oracle.com> looks good. /Andy On 8/7/2020 6:36 PM, Alexey Semenyuk wrote: > Looks good. Minor suggestion, please change > --- > tstring getRealPath(std::vector in) > --- > to > --- > tstring getRealPath(const std::vector& in); > --- > > - Alexey > > On 8/7/2020 5:54 PM, alexander.matveev at oracle.com wrote: >> Please review the jpackage fix for bug [1] at [2]. >> >> - Fixed by converting path of executable to real path if >> _NSGetExecutablePath() returned symbolic link. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8248905 >> [2] http://cr.openjdk.java.net/~almatvee/8248905/webrev.00/ >> >> Thanks, >> Alexander > From paul.sandoz at oracle.com Fri Aug 7 23:22:28 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 7 Aug 2020 16:22:28 -0700 Subject: Possible subtle memory model error in ClassValue In-Reply-To: References: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> Message-ID: <33C0A687-7B1D-49DB-86F8-CE778BC1456A@oracle.com> Here?s some pertinent information from Doug?s excellent document of memory order modes [1]: "As a delicate (but commonly exploited) special case of the above considerations, acquire-style reads of immutable fields of newly constructed objects do not require an explicit fence or moded access -- Plain mode reads suffice: If the consumer has not seen the new object before, it cannot have stale values that it must otherwise ignore or discard, and it cannot perform the read until it knows the address. On subsequent encounters, reusing old values is OK, because they cannot have changed. This reasoning rests on the only known defensible exception to the rule of never making assumptions about local precedence order: The reference (address) of a new object is assumed never to be known and impossible to speculate until creation. This assumption is also relied on by other Java security requirements. The resulting techniques are used in Java final field implementations, and are the reason why specified guarantees for final fields are conditional upon constructors not leaking references to objects before constructor return. Classes with final fields are normally implemented by issuing a form of Release fence upon constructor return. Further, because nothing need be guaranteed about interactions with reads by the constructor, a StoreStoreFence suffices. Similar techniques may apply in other contexts, but can be unacceptably fragile. For example, code that works when the associated objects are always newly constructed may, without further safeguards, fail upon later changes to instead recycle the objects from pools." I thought that by virtue of ClassValueMap extending WeakHashMap, which contains final fields, it would all work out. But, I was uncertain so I ran a jcstress test modifying the sample JMMSample_06_Finals [1]. See gist here: https://gist.github.com/PaulSandoz/3ae6d3f3e4027653d185f237eedfab72 I ran in tough mode and observed this failure on my MacBook: *** FAILED tests Strong asserts were violated. Correct implementations should have no assert failures here. 1 matching test results. [FAILED] o.o.j.t.singletons.JMMSample_06_Finals.FinalInitExtendedNoFinal (JVM args: [-XX:-TieredCompilation, -XX:+UnlockDiagnosticVMOptions, -XX:+StressLCM, -XX:+StressGCM]) Observed state Occurrences Expectation Interpretation -1 2,420,284,404 ACCEPTABLE Object is not seen yet. 3 320 FORBIDDEN Seeing partially constructed object. 4 53 FORBIDDEN Seeing partially constructed object. 5 3,486 FORBIDDEN Seeing partially constructed object. 6 8,409 FORBIDDEN Seeing partially constructed object. 7 11,282 FORBIDDEN Seeing partially constructed object. 8 181,774,476 ACCEPTABLE Seen the complete object. So for HotSpot it seems to occur only for C2 under certain stress options, -XX:+StressLCM, -XX:+StressGCM, which might explain why an issue is not generally being observed. I don?t understand enough about these options to know why this is so. However, this makes me nervous enough that it might be best to add a dummy final field to ClassMapValue, and to review other similar code. David, perhaps you could add a dummy final field to ClassValueMap and see if that fixes the problem with Graal? Paul. [1] http://gee.cs.oswego.edu/dl/html/j9mm.html [2] http://hg.openjdk.java.net/code-tools/jcstress/file/1dd2cca36fa9/jcstress-samples/src/main/java/org/openjdk/jcstress/samples/JMMSample_06_Finals.java > On Aug 7, 2020, at 2:35 PM, John Rose wrote: > > On Aug 7, 2020, at 9:52 AM, Andrew Haley wrote: >> >> On 8/7/20 4:39 PM, David Lloyd wrote: >> >>> However, I'm wondering if this isn't a side effect of what appears >>> to be an incorrect double-checked lock at lines 374-378 of >>> ClassValue.java [1]. In order for the write to the non-volatile >>> `cache` field of ClassValueMap, it is my understanding that there >>> must be some acquire/release edge from where the variable is >>> published to guarantee that all of the writes are visible to the >>> read site, and I'm not really sure that the exit-the-constructor >>> edge is going to match up with with the synchronized block which >>> protects a different non-volatile field. And even if my feeling >>> that this is dodgy is valid, I'm not sure whether this NPE is a >>> possible outcome of that problem! >> >> It certainly doesn't look right to me. AFAICS this is classic broken >> double-checked locking. It'd need some sort of fence after >> constructing the ClassValueMap instance and before publishing it. > > Y?all are calling my baby ugly but he?s really cute if you look > at him from the right angle. > > (First, a quick question: This bug shows up on x86, right? > If so, we probably are not looking at a hardware reordering > problem but something stemming from Graal?s reordering > of operations compared to C1 and C2, perhaps after a different > set of inlining decisions uncharacteristic of C2.) > > The bug indeed looks like a dropped fence at the end of the > constructor for ClassValueMap. (I don?t see an easier way to > get a null at the NPE point.) The fence might not actually be > dropped, but a write which the fence is supposed to fence > might have been reordered after the fence, and after an > unlock operation, allowing someone to see the initial > null after the unlock. > > The double-check idiom breaks when the outer check > (not synchronized) gets half-baked data and acts on it. > This much-maligned idiom would not be broken in the > present case under the assumption that a data dependency > on type.classValueMap (set in initializeMap) will see > a fully initialized value of ClassValueMap.cacheArray. > > Now we get into the fine print, and I agree there is a bug > here. The easy fix is to repair the logic under which that > ugly everybody?s-hating-on-it double check idiom would > be in fact correct. > > The fine print requires a couple different things that are not > quite fulfilled by the present code, and Graal has either > reordered the memory accesses to expose the problem, or > it (itself) has a complementary bug. (Or it has an unrelated > bug, and you people are *staring* at my baby!) > > First, as Paul noted, you need a final variable in your class > to get the benefit of a fence at the end of the constructor. > Oops #1: ClassValueMap doesn?t have *any* finals. That?s > awkward. Two fixes for that: (a) add a dummy final, and > (b) add a real fence in the constructor. For better luck, > maybe put the fence at the end of sizeCache. > > On machines (not x86) which need fences *before* final > reads, another explicit fence should be placed before the > unsynchronized read (or else inside the getCache method). > > By the letter of the JMM, the helpful effects of the fence > at the end of the constructor are only guaranteed for > references which depend on final variables set by that > constructor, because only those final variables get a > ?freeze? operation which stabilizes them (and values > dependently loaded via them). Throwing in a dummy > final is therefore not guaranteed to work. But throwing > in a fence will work. One downside of the fence is that > the JMM is silent about fences, but the JMM is widely > recognized as a partial story, not the full or final story. > > (Here?s a tidbit of JMM politics: I once heard Doug Lea > considering whether maybe all fields should be treated > more like final fields. I don?t know if this is still a live > idea, but it would make this bug go way, since nearly all > constructors would then get fences of some sort.) > > Here?s a bit of explanatory (non-normative) text from the draft > POPL paper for JMM, where the parenthetic comment indicates > (I think) the sort of thing that is happening here: > >> Let us now assume that the acquire and release do happen. As long as >> these actions take place after object has been constructed (and there >> is no code motion around the end of the constructor), the diffs that >> the second processor acquires are guaranteed to reflect the correctly >> constructed object. > > Basically, the synchronization statement is expected to do a > release *after* the non-null value is stored. It looks like this > is failing due to a lost and/or reordered fence. > > Second, David says: > >> I?m not really sure that the exit-the-constructor edge is going to match >> up with with the synchronized block which protects a different >> non-volatile field. > > By the letter of the JMM (AFAIUI), this code is coloring way outside > the lines. (Yes, I admit it.) The problem is pretty fundamental. > I went and swapped in some JMM spec just now (don?t have it in > short term memory; go figure), and here?s what I re-learned. > > In JMM terms, a ?release? is the earlier end of any edge in relation > called ?synchronizes-with?. An ?acquire? is the latter end of such > an edge. In the JMM this relation applies only to lock, unlock, > and volatile reads and writes. Your guess is as good as mine whether > other operations (CAS, fences, etc.) participate; the JMM is silent. > Crucially, acquires and release do not touch plain fields. This is > counter-intuitive for those of us who like to reason with fences. > > The JMM prevents regular writes from floating past what we like > to think of as ?release points? (unlocks) by appealing to processor > order inside lock/unlock pairs, and also by appealing to global > ordering of multiple lock/unlock pairs (or stuff with volatiles > and other corner cases which we will neglect here). In order > to work right, a normal write has to come before a release *and* > a matching normal read has to come after an acquire, and such > an acquire is nothing other than the lock of a later lock/unlock > pair, typically in another thread. > > So the rules which reliably connect normal writes to normal > reads in other threads work differently from acquire fences and > release fences as we (or just I?) usually think of them. Yes, there > are ?acquires? and ?releases? in the JMM. No, they are not > primitives but rather descriptions of the way the happens-before > relation is constrained by (among other things) lock and unlock > actions. > > On x86, you don?t have to worry about acquires; you just set up > the right release fences. In the JMM, there?s no way to get either > an acquire or a release without a synchronized statement (or > other fancy stuff like volatiles). That?s why double-check is > broken in the pure JMM, and why it can be repaired if you > add some (non-JMM) fences. > > So why is this showing up suddenly with Graal? Possibly it?s > got a really aggressive interpretation of the JMM, relative to > the standard HotSpot JITs. Possibly it?s got a bug. Graal might > be allowing the initialization of ClassValueMap.cacheArray > to float down past publication of the ClassValueMap on > type.classValueMap (in initializeMap). Less likely, it is allowing > the initialization to totally escape the lock/unlock pair, something > the JMM might allow but the Cookbook advises against (in its > ?Can Reorder? table). > > Perhaps Graal is modeling the ?freeze? operations on finals more > accurately (and the Cookbook gives instructions for this). That > would allow the ClassValueMap to correctly initialize its frozen > finals, but not (unfortunately) the not-frozen cacheArray field. > It appears that the new JIT on the block is exposing my neglect > for my poor baby, in not putting the right fences around its playpen. > > Bottom line: Add a release fence before type.cVM is set, and add > a (no-op on x86) acquire fence in getCache. Quick test: Add a dummy > final to CVM, to see if that makes the bug let go. > > ? John From alexander.matveev at oracle.com Fri Aug 7 23:59:35 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Fri, 7 Aug 2020 16:59:35 -0700 Subject: RFR: 8248905: [macos] symbolic links not properly resolved In-Reply-To: <8898fea2-a1fa-d0aa-1d1c-57f9217ff5d9@oracle.com> References: <9c0d1f04-3ccf-fa68-0009-d249cf071040@oracle.com> <8898fea2-a1fa-d0aa-1d1c-57f9217ff5d9@oracle.com> Message-ID: <8e78b9e9-4ec3-5908-1b0d-8d67880a7efa@oracle.com> Hi Alexey, Webrev updated: http://cr.openjdk.java.net/~almatvee/8248905/webrev.01/ Thanks, Alexander On 8/7/20 3:36 PM, Alexey Semenyuk wrote: > Looks good. Minor suggestion, please change > --- > tstring getRealPath(std::vector in) > --- > to > --- > tstring getRealPath(const std::vector& in); > --- > > - Alexey > > On 8/7/2020 5:54 PM, alexander.matveev at oracle.com wrote: >> Please review the jpackage fix for bug [1] at [2]. >> >> - Fixed by converting path of executable to real path if >> _NSGetExecutablePath() returned symbolic link. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8248905 >> [2] http://cr.openjdk.java.net/~almatvee/8248905/webrev.00/ >> >> Thanks, >> Alexander > From john.r.rose at oracle.com Sat Aug 8 00:41:59 2020 From: john.r.rose at oracle.com (John Rose) Date: Fri, 7 Aug 2020 17:41:59 -0700 Subject: Possible subtle memory model error in ClassValue In-Reply-To: References: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> Message-ID: <824CAC8C-8CFA-4401-936D-FEF492B48BBC@oracle.com> On Aug 7, 2020, at 2:35 PM, John Rose wrote: > > (Here?s a tidbit of JMM politics: I once heard Doug Lea > considering whether maybe all fields should be treated > more like final fields. I don?t know if this is still a live > idea, but it would make this bug go way, since nearly all > constructors would then get fences of some sort.) Paul helpfully pointed me at Aleksey?s excellent description and investigation of this proposal, which I had forgotten about: https://shipilev.net/blog/2014/all-fields-are-final/ https://bugs.openjdk.java.net/browse/JDK-8031818 The JVM flag -XX:+AlwaysSafeConstructors might be useful to enable this proposed feature and see if it suppresses the bug. (Don?t forget to unlock.) Looks like the conversation is stalled, pending further insights and problem fixes. I had an idea for fixing one of the problems with chained constructors, commented on JDK-8032218. ? John From alexey.semenyuk at oracle.com Sat Aug 8 01:44:08 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Fri, 7 Aug 2020 21:44:08 -0400 Subject: RFR: 8248905: [macos] symbolic links not properly resolved In-Reply-To: <8e78b9e9-4ec3-5908-1b0d-8d67880a7efa@oracle.com> References: <9c0d1f04-3ccf-fa68-0009-d249cf071040@oracle.com> <8898fea2-a1fa-d0aa-1d1c-57f9217ff5d9@oracle.com> <8e78b9e9-4ec3-5908-1b0d-8d67880a7efa@oracle.com> Message-ID: Looks good. - Alexey On 8/7/2020 7:59 PM, alexander.matveev at oracle.com wrote: > Hi Alexey, > > Webrev updated: > http://cr.openjdk.java.net/~almatvee/8248905/webrev.01/ > > Thanks, > Alexander > > On 8/7/20 3:36 PM, Alexey Semenyuk wrote: >> Looks good. Minor suggestion, please change >> --- >> tstring getRealPath(std::vector in) >> --- >> to >> --- >> tstring getRealPath(const std::vector& in); >> --- >> >> - Alexey >> >> On 8/7/2020 5:54 PM, alexander.matveev at oracle.com wrote: >>> Please review the jpackage fix for bug [1] at [2]. >>> >>> - Fixed by converting path of executable to real path if >>> _NSGetExecutablePath() returned symbolic link. >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8248905 >>> [2] http://cr.openjdk.java.net/~almatvee/8248905/webrev.00/ >>> >>> Thanks, >>> Alexander >> > From luhenry at microsoft.com Sat Aug 8 04:30:37 2020 From: luhenry at microsoft.com (Ludovic Henry) Date: Sat, 8 Aug 2020 04:30:37 +0000 Subject: [11u] RFR[M]: 8250902: Implement MD5 Intrinsics on x86 Message-ID: Hello, I would like to backport the newly added MD5 Intrinsic to JDK 11. The change is contained, limiting the chance of a regression, and provides a great speedup on a common pattern. This change also contains the follow-up fix by Vladimir Kozlov. As it is the first backport I go through, please let me know what other steps I need to take. Original Bugs: https://bugs.openjdk.java.net/browse/JDK-8250902 https://bugs.openjdk.java.net/browse/JDK-8251260 Original Webrevs: http://cr.openjdk.java.net/~luhenry/8250902/webrev.03 http://cr.openjdk.java.net/~kvn/8251260/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8251319 Webrev: http://cr.openjdk.java.net/~luhenry/8250902-11u/webrev.00 Testing: Linux-x64, fastdebug, test/hotspot/jtreg/compiler/intrinsics/sha, hotspot:tier1, jdk:tier1. Thank you, Ludovic [1] From vladimir.kozlov at oracle.com Sat Aug 8 04:49:49 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Fri, 7 Aug 2020 21:49:49 -0700 Subject: [11u] RFR[M]: 8250902: Implement MD5 Intrinsics on x86 In-Reply-To: References: Message-ID: <569d6c8b-136f-de81-5816-47216333fcb8@oracle.com> Hi Ludovic, Usually we backport only bugs fixes to keep LTS (11u) release stable. To backport into 11u you need approval [1]. Here is example [2]. You need also point if backport applied cleanly or you have to make changes. Changes should be backported separately to keep track - do not combine changes. But it is okay to push both changesets together (especially if followup changes fixed first). Regards, Vladimir K [1] http://openjdk.java.net/projects/jdk-updates/approval.html [2] https://bugs.openjdk.java.net/browse/JDK-8248214 On 8/7/20 9:30 PM, Ludovic Henry wrote: > Hello, > > I would like to backport the newly added MD5 Intrinsic to JDK 11. The change is contained, limiting the chance of a regression, and provides a great speedup on a common pattern. This change also contains the follow-up fix by Vladimir Kozlov. > > As it is the first backport I go through, please let me know what other steps I need to take. > > Original Bugs: > https://bugs.openjdk.java.net/browse/JDK-8250902 > https://bugs.openjdk.java.net/browse/JDK-8251260 > > Original Webrevs: > http://cr.openjdk.java.net/~luhenry/8250902/webrev.03 > http://cr.openjdk.java.net/~kvn/8251260/webrev.00/ > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8251319 > > Webrev: > http://cr.openjdk.java.net/~luhenry/8250902-11u/webrev.00 > > Testing: Linux-x64, fastdebug, test/hotspot/jtreg/compiler/intrinsics/sha, hotspot:tier1, jdk:tier1. > > Thank you, > Ludovic > > [1] > > From amaembo at gmail.com Sat Aug 8 04:58:52 2020 From: amaembo at gmail.com (Tagir Valeev) Date: Sat, 8 Aug 2020 11:58:52 +0700 Subject: RFR: JDK-8244288 Specialized implementations for putIfAbsent, merge, compute* methods in TreeMap derived maps In-Reply-To: <8D5FEA90-0A21-4F13-A62A-C4AF2CC7D112@oracle.com> References: <8D5FEA90-0A21-4F13-A62A-C4AF2CC7D112@oracle.com> Message-ID: Hello, Paul! Thank you for the review. > The putIfAbsent and merge implementations result in a change of behavior. For putIfAbsent, a key out-of-range now results in an exception rather than returning null. No, the semantics is preserved. Before this change, putIfAbsent also throws for out-of-range key. The default implementation looks like this: V v = get(key); // always returns null for out-of-range key if (v == null) { // if is always visited v = put(key, value); // put always throws for out-of-range key } return v; The following test confirms that the behavior is preserved: var map = IntStream.range(1, 10).boxed() .collect(Collectors.toMap(x -> x, String::valueOf, (a, b) -> a, TreeMap::new)); var subMap = map.subMap(1, 5); subMap.putIfAbsent(6, "6"); > For merge, if the new value is null, then remove is called which also returns null for a key out-of-range. The new value in merge cannot be null, both by Map::merge spec (@param value the non-null value to be merged...) and by implementation (Objects.requireNonNull(value) is called). The remapping function is never called for out-of-range key because the old value is always null. Thus the 'remove' call is unreachable for out-of-range key and my patch preserves this behavior. > I am inclined to also change the behavior of other modification methods, compute and computeIfAbsent, to always throw if the key is out-of-range, rather than check the result of the mapping function. i.e. an out-of-range key is never passed to the mapping function. > > That?s a wider change in behavior but I suspect one that has minimal impact. (In hindsight if we were implementing these methods when we added the defaults in 8 I would like to think this is the behavior we, or at least I :-), would of implemented.) I believe this change would require CSR, right? With best regards, Tagir Valeev. > > Paul. > > > > On Jul 26, 2020, at 9:04 AM, Tagir Valeev wrote: > > > > Hello! > > > > A gentle ping: please review > > https://bugs.openjdk.java.net/browse/JDK-8244288 > > http://cr.openjdk.java.net/~tvaleev/webrev/8244288/r1/ > > The details are listed below. > > > > With best regards, > > Tagir Valeev. > > > > On Sun, May 3, 2020 at 4:36 PM Tagir Valeev wrote: > >> > >> Hello! > >> > >> Please review the following change: > >> https://bugs.openjdk.java.net/browse/JDK-8244288 > >> http://cr.openjdk.java.net/~tvaleev/webrev/8244288/r1/ > >> > >> We already implemented putIfAbsent, merge, computeIfAbsent, > >> computeIfPresent, and compute for TreeMap class (see JDK-8176894). > >> However, default implementations of these methods are still used for > >> TreeMap derived maps, such as descendingMap(), tailMap(), headMap() > >> and subMap(). The implementation is pretty straightforward, just a > >> range-check+delegation for each of these methods inside the > >> TreeMap.NavigableSubMap (+32 lines in TreeMap.java). Care should be > >> taken to avoid throwing IAE from compute* methods if the key is > >> outside of the range but we don't actually add the entry. This mimics > >> the previous behavior and also consistent with other modification > >> methods like Map.remove (we don't throw from remove if the key is out > >> of range, simply do nothing). > >> > >> To cover these changes with tests, I added new > >> TreeMap().descendingMap() to > >> InPlaceOpsCollisions.nullValueFriendlyMaps and > >> MapWithCollisionsProviders.makeMapsMoreTypes. This map should behave > >> like a normal Map modulo iteration order but implementation goes > >> through NavigableSubMap. Also, I added more adders to > >> LockStep::randomAdder (lines 572+) to cover new methods, as well as > >> some more throws IAE asserts around line 806. At the same time, I took > >> the liberty to modernize this test: > >> - Convert all the raw-types to properly parameterized (using 'var' for > >> local vars where resulting type is long) > >> - Convert MapFrobber and SetFrobber to interfaces, and convert all the > >> implementations to lambdas (automatic conversion via IntelliJ IDEA + > >> rename of conflicting parameter names) > >> - Use for-each loop instead of indexed for loop where possible > >> - 'elts' array was created in two places but unused afterward. I added > >> size assert to these arrays to use it at least somehow (though > >> probably content should be checked as well). > >> - Use Comparator.naturalOrder() instead of manually written one in > >> comparator() methods (should not affect the testing in any way as it's > >> only used directly, not passed e.g. to TreeMap constructor). > >> - Use Objects.equals inside LockStep#equal > >> - Inverted 'if' at line 299 to avoid empty block. > >> If the cleanup really complicates the review I can post the cleanup as > >> a separate changeset. Though it should not be very problematic as the > >> actual changes are quite compact (as said above, near lines 572 and > >> 806) > >> > >> I also improved the previously added benchmark TreeMapUpdate to check > >> descendingMap and tailMap cases to confirm the hypothesis that the > >> change improves the performance of derived maps. This is indeed the > >> case. E.g. merge test yields (comparator = false, size = 1000) for > >> unpatched JDK: > >> > >> (benchmark) (mode) (prefill) Score Error Units > >> TreeMapUpdate.merge TreeMap true 63589,075 ? 1028,065 ns/op > >> TreeMapUpdate.merge TreeMap false 65414,367 ? 1011,442 ns/op > >> TreeMapUpdate.merge descendingMap true 121501,618 ? 1849,108 ns/op > >> TreeMapUpdate.merge descendingMap false 95913,212 ? 2854,063 ns/op > >> TreeMapUpdate.merge tailMap true 126657,309 ? 4467,733 ns/op > >> TreeMapUpdate.merge tailMap false 93448,840 ? 1359,392 ns/op > >> > >> As you can see, the merge on derived maps is significantly slower. > >> After the patch is applied the results are much better: > >> > >> (benchmark) (mode) (prefill) Score Error Units > >> TreeMapUpdate.merge TreeMap true 64059,189 ? 808,230 ns/op > >> TreeMapUpdate.merge TreeMap false 65156,912 ? 1229,965 ns/op > >> TreeMapUpdate.merge descendingMap true 69938,131 ? 1697,357 ns/op > >> TreeMapUpdate.merge descendingMap false 67783,829 ? 263,061 ns/op > >> TreeMapUpdate.merge tailMap true 71079,529 ? 841,738 ns/op > >> TreeMapUpdate.merge tailMap false 68577,169 ? 1196,758 ns/op > >> > >> I don't think this requires a CSR, as the changed class is > >> package-private, so it cannot be extended by clients. > >> > >> With best regards, > >> Tagir Valeev. > From david.holmes at oracle.com Sat Aug 8 05:21:44 2020 From: david.holmes at oracle.com (David Holmes) Date: Sat, 8 Aug 2020 15:21:44 +1000 Subject: Possible subtle memory model error in ClassValue In-Reply-To: <33C0A687-7B1D-49DB-86F8-CE778BC1456A@oracle.com> References: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> <33C0A687-7B1D-49DB-86F8-CE778BC1456A@oracle.com> Message-ID: <464fc65f-308d-5636-8c83-4592257e96d9@oracle.com> For DCL to be correct Class.classValueMap should be declared volatile. Final field guarantees are there to support unsafe publication in special cases. In this case we should be using safe publication. Cheers, David On 8/08/2020 9:22 am, Paul Sandoz wrote: > Here?s some pertinent information from Doug?s excellent document of memory order modes [1]: > > "As a delicate (but commonly exploited) special case of the above considerations, acquire-style reads of immutable fields of newly constructed objects do not require an explicit fence or moded access -- Plain mode reads suffice: If the consumer has not seen the new object before, it cannot have stale values that it must otherwise ignore or discard, and it cannot perform the read until it knows the address. On subsequent encounters, reusing old values is OK, because they cannot have changed. This reasoning rests on the only known defensible exception to the rule of never making assumptions about local precedence order: The reference (address) of a new object is assumed never to be known and impossible to speculate until creation. This assumption is also relied on by other Java security requirements. > > The resulting techniques are used in Java final field implementations, and are the reason why specified guarantees for final fields are conditional upon constructors not leaking references to objects before constructor return. Classes with final fields are normally implemented by issuing a form of Release fence upon constructor return. Further, because nothing need be guaranteed about interactions with reads by the constructor, a StoreStoreFence suffices. Similar techniques may apply in other contexts, but can be unacceptably fragile. For example, code that works when the associated objects are always newly constructed may, without further safeguards, fail upon later changes to instead recycle the objects from pools." > > > I thought that by virtue of ClassValueMap extending WeakHashMap, which contains final fields, it would all work out. But, I was uncertain so I ran a jcstress test modifying the sample JMMSample_06_Finals [1]. > > See gist here: > > https://gist.github.com/PaulSandoz/3ae6d3f3e4027653d185f237eedfab72 > > I ran in tough mode and observed this failure on my MacBook: > > *** FAILED tests > Strong asserts were violated. Correct implementations should have no assert failures here. > > 1 matching test results. > [FAILED] o.o.j.t.singletons.JMMSample_06_Finals.FinalInitExtendedNoFinal > (JVM args: [-XX:-TieredCompilation, -XX:+UnlockDiagnosticVMOptions, -XX:+StressLCM, -XX:+StressGCM]) > Observed state Occurrences Expectation Interpretation > -1 2,420,284,404 ACCEPTABLE Object is not seen yet. > 3 320 FORBIDDEN Seeing partially constructed object. > 4 53 FORBIDDEN Seeing partially constructed object. > 5 3,486 FORBIDDEN Seeing partially constructed object. > 6 8,409 FORBIDDEN Seeing partially constructed object. > 7 11,282 FORBIDDEN Seeing partially constructed object. > 8 181,774,476 ACCEPTABLE Seen the complete object. > > So for HotSpot it seems to occur only for C2 under certain stress options, -XX:+StressLCM, -XX:+StressGCM, which might explain why an issue is not generally being observed. I don?t understand enough about these options to know why this is so. > > However, this makes me nervous enough that it might be best to add a dummy final field to ClassMapValue, and to review other similar code. > > David, perhaps you could add a dummy final field to ClassValueMap and see if that fixes the problem with Graal? > > Paul. > > [1] http://gee.cs.oswego.edu/dl/html/j9mm.html > > [2] http://hg.openjdk.java.net/code-tools/jcstress/file/1dd2cca36fa9/jcstress-samples/src/main/java/org/openjdk/jcstress/samples/JMMSample_06_Finals.java > >> On Aug 7, 2020, at 2:35 PM, John Rose wrote: >> >> On Aug 7, 2020, at 9:52 AM, Andrew Haley wrote: >>> >>> On 8/7/20 4:39 PM, David Lloyd wrote: >>> >>>> However, I'm wondering if this isn't a side effect of what appears >>>> to be an incorrect double-checked lock at lines 374-378 of >>>> ClassValue.java [1]. In order for the write to the non-volatile >>>> `cache` field of ClassValueMap, it is my understanding that there >>>> must be some acquire/release edge from where the variable is >>>> published to guarantee that all of the writes are visible to the >>>> read site, and I'm not really sure that the exit-the-constructor >>>> edge is going to match up with with the synchronized block which >>>> protects a different non-volatile field. And even if my feeling >>>> that this is dodgy is valid, I'm not sure whether this NPE is a >>>> possible outcome of that problem! >>> >>> It certainly doesn't look right to me. AFAICS this is classic broken >>> double-checked locking. It'd need some sort of fence after >>> constructing the ClassValueMap instance and before publishing it. >> >> Y?all are calling my baby ugly but he?s really cute if you look >> at him from the right angle. >> >> (First, a quick question: This bug shows up on x86, right? >> If so, we probably are not looking at a hardware reordering >> problem but something stemming from Graal?s reordering >> of operations compared to C1 and C2, perhaps after a different >> set of inlining decisions uncharacteristic of C2.) >> >> The bug indeed looks like a dropped fence at the end of the >> constructor for ClassValueMap. (I don?t see an easier way to >> get a null at the NPE point.) The fence might not actually be >> dropped, but a write which the fence is supposed to fence >> might have been reordered after the fence, and after an >> unlock operation, allowing someone to see the initial >> null after the unlock. >> >> The double-check idiom breaks when the outer check >> (not synchronized) gets half-baked data and acts on it. >> This much-maligned idiom would not be broken in the >> present case under the assumption that a data dependency >> on type.classValueMap (set in initializeMap) will see >> a fully initialized value of ClassValueMap.cacheArray. >> >> Now we get into the fine print, and I agree there is a bug >> here. The easy fix is to repair the logic under which that >> ugly everybody?s-hating-on-it double check idiom would >> be in fact correct. >> >> The fine print requires a couple different things that are not >> quite fulfilled by the present code, and Graal has either >> reordered the memory accesses to expose the problem, or >> it (itself) has a complementary bug. (Or it has an unrelated >> bug, and you people are *staring* at my baby!) >> >> First, as Paul noted, you need a final variable in your class >> to get the benefit of a fence at the end of the constructor. >> Oops #1: ClassValueMap doesn?t have *any* finals. That?s >> awkward. Two fixes for that: (a) add a dummy final, and >> (b) add a real fence in the constructor. For better luck, >> maybe put the fence at the end of sizeCache. >> >> On machines (not x86) which need fences *before* final >> reads, another explicit fence should be placed before the >> unsynchronized read (or else inside the getCache method). >> >> By the letter of the JMM, the helpful effects of the fence >> at the end of the constructor are only guaranteed for >> references which depend on final variables set by that >> constructor, because only those final variables get a >> ?freeze? operation which stabilizes them (and values >> dependently loaded via them). Throwing in a dummy >> final is therefore not guaranteed to work. But throwing >> in a fence will work. One downside of the fence is that >> the JMM is silent about fences, but the JMM is widely >> recognized as a partial story, not the full or final story. >> >> (Here?s a tidbit of JMM politics: I once heard Doug Lea >> considering whether maybe all fields should be treated >> more like final fields. I don?t know if this is still a live >> idea, but it would make this bug go way, since nearly all >> constructors would then get fences of some sort.) >> >> Here?s a bit of explanatory (non-normative) text from the draft >> POPL paper for JMM, where the parenthetic comment indicates >> (I think) the sort of thing that is happening here: >> >>> Let us now assume that the acquire and release do happen. As long as >>> these actions take place after object has been constructed (and there >>> is no code motion around the end of the constructor), the diffs that >>> the second processor acquires are guaranteed to reflect the correctly >>> constructed object. >> >> Basically, the synchronization statement is expected to do a >> release *after* the non-null value is stored. It looks like this >> is failing due to a lost and/or reordered fence. >> >> Second, David says: >> >>> I?m not really sure that the exit-the-constructor edge is going to match >>> up with with the synchronized block which protects a different >>> non-volatile field. >> >> By the letter of the JMM (AFAIUI), this code is coloring way outside >> the lines. (Yes, I admit it.) The problem is pretty fundamental. >> I went and swapped in some JMM spec just now (don?t have it in >> short term memory; go figure), and here?s what I re-learned. >> >> In JMM terms, a ?release? is the earlier end of any edge in relation >> called ?synchronizes-with?. An ?acquire? is the latter end of such >> an edge. In the JMM this relation applies only to lock, unlock, >> and volatile reads and writes. Your guess is as good as mine whether >> other operations (CAS, fences, etc.) participate; the JMM is silent. >> Crucially, acquires and release do not touch plain fields. This is >> counter-intuitive for those of us who like to reason with fences. >> >> The JMM prevents regular writes from floating past what we like >> to think of as ?release points? (unlocks) by appealing to processor >> order inside lock/unlock pairs, and also by appealing to global >> ordering of multiple lock/unlock pairs (or stuff with volatiles >> and other corner cases which we will neglect here). In order >> to work right, a normal write has to come before a release *and* >> a matching normal read has to come after an acquire, and such >> an acquire is nothing other than the lock of a later lock/unlock >> pair, typically in another thread. >> >> So the rules which reliably connect normal writes to normal >> reads in other threads work differently from acquire fences and >> release fences as we (or just I?) usually think of them. Yes, there >> are ?acquires? and ?releases? in the JMM. No, they are not >> primitives but rather descriptions of the way the happens-before >> relation is constrained by (among other things) lock and unlock >> actions. >> >> On x86, you don?t have to worry about acquires; you just set up >> the right release fences. In the JMM, there?s no way to get either >> an acquire or a release without a synchronized statement (or >> other fancy stuff like volatiles). That?s why double-check is >> broken in the pure JMM, and why it can be repaired if you >> add some (non-JMM) fences. >> >> So why is this showing up suddenly with Graal? Possibly it?s >> got a really aggressive interpretation of the JMM, relative to >> the standard HotSpot JITs. Possibly it?s got a bug. Graal might >> be allowing the initialization of ClassValueMap.cacheArray >> to float down past publication of the ClassValueMap on >> type.classValueMap (in initializeMap). Less likely, it is allowing >> the initialization to totally escape the lock/unlock pair, something >> the JMM might allow but the Cookbook advises against (in its >> ?Can Reorder? table). >> >> Perhaps Graal is modeling the ?freeze? operations on finals more >> accurately (and the Cookbook gives instructions for this). That >> would allow the ClassValueMap to correctly initialize its frozen >> finals, but not (unfortunately) the not-frozen cacheArray field. >> It appears that the new JIT on the block is exposing my neglect >> for my poor baby, in not putting the right fences around its playpen. >> >> Bottom line: Add a release fence before type.cVM is set, and add >> a (no-op on x86) acquire fence in getCache. Quick test: Add a dummy >> final to CVM, to see if that makes the bug let go. >> >> ? John > From raffaello.giulietti at gmail.com Sat Aug 8 10:26:33 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Sat, 8 Aug 2020 12:26:33 +0200 Subject: RFR: 8245036: DataInputStream.readFully(byte[], int, int) does not throw expected IndexOutOfBoundsExceptions In-Reply-To: References: <59a871a4-d676-3b0c-2983-b9b9fc34ca37@gmail.com> <858a70ac-f827-669a-58e2-e5b834aa36f4@gmail.com> <67C3F744-2FE7-4BDD-857F-86E64A1AABFA@oracle.com> <4030EF21-CCE0-48D2-ADB1-7BF7519FEA52@oracle.com> <10711294-E57F-4D89-9F51-61DAE4EFC8BF@oracle.com> Message-ID: <5b289a92-f52f-d6c3-b980-fc79c6c04ffe@gmail.com> Thanks Brian for pushing On 2020-08-07 23:12, Brian Burkhalter wrote: > >> On Aug 7, 2020, at 9:39 AM, Brian Burkhalter >> > wrote: >> >>> There's one corner case that would not pass, however, so I added >>> testNegativeOffsetZeroLength() to exercise that. It passes on the new >>> source but fails on the old. >>> >>> I also added two additional corner cases for too large combinations >>> of offset and length. >> >> This patch looks good. I am running it through our CI tests but I >> don?t expect to see any problems. If the test run succeeds aside from >> any failures unrelated to this code then I can push this. > > Pushed http://hg.openjdk.java.net/jdk/jdk/rev/29212f925331. > > Thanks, > > Brian From aph at redhat.com Sat Aug 8 12:08:46 2020 From: aph at redhat.com (Andrew Haley) Date: Sat, 8 Aug 2020 13:08:46 +0100 Subject: [11u] RFR[M]: 8250902: Implement MD5 Intrinsics on x86 In-Reply-To: References: Message-ID: <8b61da32-3cc1-fa14-a607-1f871f7e3d70@redhat.com> On 8/8/20 5:30 AM, Ludovic Henry wrote: > I would like to backport the newly added MD5 Intrinsic to JDK 11. It's too early for that: changes are supposed to bake in JDK head for a while. Also, since it's an enhancement rather than a bug fix we'd need to have the discussion. I would say it's marginal whether something like this should be back ported. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From paul.sandoz at oracle.com Sat Aug 8 15:21:56 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Sat, 8 Aug 2020 08:21:56 -0700 Subject: RFR: JDK-8244288 Specialized implementations for putIfAbsent, merge, compute* methods in TreeMap derived maps In-Reply-To: References: <8D5FEA90-0A21-4F13-A62A-C4AF2CC7D112@oracle.com> Message-ID: Hi Tagir, Clearly I am a terribly tracker of null values! My apologies for the poor review of that. Regarding changing sub-map computeIfAbsent and compute behavior, yes that would require a CSR. Thinking more about this, it may be better to play it safe and retain the current behavior for this issue. We could always follow up with another change if necessary. In that respect +1 to your current patch. It may be helpful to add a comment in the implementations of computeIfAbsent/compute saying the call to the mapping function when the key is out of range is to preserve the compatibly with the default methods on Map. No need for another review of that if you choose to add it. Paul. > On Aug 7, 2020, at 9:58 PM, Tagir Valeev wrote: > > Hello, Paul! > > Thank you for the review. > >> The putIfAbsent and merge implementations result in a change of behavior. For putIfAbsent, a key out-of-range now results in an exception rather than returning null. > > No, the semantics is preserved. Before this change, putIfAbsent also > throws for out-of-range key. The default implementation looks like > this: > > V v = get(key); // always returns null for out-of-range key > if (v == null) { // if is always visited > v = put(key, value); // put always throws for out-of-range key > } > return v; > > The following test confirms that the behavior is preserved: > > var map = IntStream.range(1, 10).boxed() > .collect(Collectors.toMap(x -> x, String::valueOf, (a, b) -> a, > TreeMap::new)); > var subMap = map.subMap(1, 5); > subMap.putIfAbsent(6, "6"); > >> For merge, if the new value is null, then remove is called which also returns null for a key out-of-range. > > The new value in merge cannot be null, both by Map::merge spec (@param > value the non-null value to be merged...) and by implementation > (Objects.requireNonNull(value) is called). The remapping function is > never called for out-of-range key because the old value is always > null. Thus the 'remove' call is unreachable for out-of-range key and > my patch preserves this behavior. > >> I am inclined to also change the behavior of other modification methods, compute and computeIfAbsent, to always throw if the key is out-of-range, rather than check the result of the mapping function. i.e. an out-of-range key is never passed to the mapping function. >> >> That?s a wider change in behavior but I suspect one that has minimal impact. (In hindsight if we were implementing these methods when we added the defaults in 8 I would like to think this is the behavior we, or at least I :-), would of implemented.) > > I believe this change would require CSR, right? > > With best regards, > Tagir Valeev. > >> >> Paul. >> >> >>> On Jul 26, 2020, at 9:04 AM, Tagir Valeev wrote: >>> >>> Hello! >>> >>> A gentle ping: please review >>> https://bugs.openjdk.java.net/browse/JDK-8244288 >>> http://cr.openjdk.java.net/~tvaleev/webrev/8244288/r1/ >>> The details are listed below. >>> >>> With best regards, >>> Tagir Valeev. >>> >>> On Sun, May 3, 2020 at 4:36 PM Tagir Valeev wrote: >>>> >>>> Hello! >>>> >>>> Please review the following change: >>>> https://bugs.openjdk.java.net/browse/JDK-8244288 >>>> http://cr.openjdk.java.net/~tvaleev/webrev/8244288/r1/ >>>> >>>> We already implemented putIfAbsent, merge, computeIfAbsent, >>>> computeIfPresent, and compute for TreeMap class (see JDK-8176894). >>>> However, default implementations of these methods are still used for >>>> TreeMap derived maps, such as descendingMap(), tailMap(), headMap() >>>> and subMap(). The implementation is pretty straightforward, just a >>>> range-check+delegation for each of these methods inside the >>>> TreeMap.NavigableSubMap (+32 lines in TreeMap.java). Care should be >>>> taken to avoid throwing IAE from compute* methods if the key is >>>> outside of the range but we don't actually add the entry. This mimics >>>> the previous behavior and also consistent with other modification >>>> methods like Map.remove (we don't throw from remove if the key is out >>>> of range, simply do nothing). >>>> >>>> To cover these changes with tests, I added new >>>> TreeMap().descendingMap() to >>>> InPlaceOpsCollisions.nullValueFriendlyMaps and >>>> MapWithCollisionsProviders.makeMapsMoreTypes. This map should behave >>>> like a normal Map modulo iteration order but implementation goes >>>> through NavigableSubMap. Also, I added more adders to >>>> LockStep::randomAdder (lines 572+) to cover new methods, as well as >>>> some more throws IAE asserts around line 806. At the same time, I took >>>> the liberty to modernize this test: >>>> - Convert all the raw-types to properly parameterized (using 'var' for >>>> local vars where resulting type is long) >>>> - Convert MapFrobber and SetFrobber to interfaces, and convert all the >>>> implementations to lambdas (automatic conversion via IntelliJ IDEA + >>>> rename of conflicting parameter names) >>>> - Use for-each loop instead of indexed for loop where possible >>>> - 'elts' array was created in two places but unused afterward. I added >>>> size assert to these arrays to use it at least somehow (though >>>> probably content should be checked as well). >>>> - Use Comparator.naturalOrder() instead of manually written one in >>>> comparator() methods (should not affect the testing in any way as it's >>>> only used directly, not passed e.g. to TreeMap constructor). >>>> - Use Objects.equals inside LockStep#equal >>>> - Inverted 'if' at line 299 to avoid empty block. >>>> If the cleanup really complicates the review I can post the cleanup as >>>> a separate changeset. Though it should not be very problematic as the >>>> actual changes are quite compact (as said above, near lines 572 and >>>> 806) >>>> >>>> I also improved the previously added benchmark TreeMapUpdate to check >>>> descendingMap and tailMap cases to confirm the hypothesis that the >>>> change improves the performance of derived maps. This is indeed the >>>> case. E.g. merge test yields (comparator = false, size = 1000) for >>>> unpatched JDK: >>>> >>>> (benchmark) (mode) (prefill) Score Error Units >>>> TreeMapUpdate.merge TreeMap true 63589,075 ? 1028,065 ns/op >>>> TreeMapUpdate.merge TreeMap false 65414,367 ? 1011,442 ns/op >>>> TreeMapUpdate.merge descendingMap true 121501,618 ? 1849,108 ns/op >>>> TreeMapUpdate.merge descendingMap false 95913,212 ? 2854,063 ns/op >>>> TreeMapUpdate.merge tailMap true 126657,309 ? 4467,733 ns/op >>>> TreeMapUpdate.merge tailMap false 93448,840 ? 1359,392 ns/op >>>> >>>> As you can see, the merge on derived maps is significantly slower. >>>> After the patch is applied the results are much better: >>>> >>>> (benchmark) (mode) (prefill) Score Error Units >>>> TreeMapUpdate.merge TreeMap true 64059,189 ? 808,230 ns/op >>>> TreeMapUpdate.merge TreeMap false 65156,912 ? 1229,965 ns/op >>>> TreeMapUpdate.merge descendingMap true 69938,131 ? 1697,357 ns/op >>>> TreeMapUpdate.merge descendingMap false 67783,829 ? 263,061 ns/op >>>> TreeMapUpdate.merge tailMap true 71079,529 ? 841,738 ns/op >>>> TreeMapUpdate.merge tailMap false 68577,169 ? 1196,758 ns/op >>>> >>>> I don't think this requires a CSR, as the changed class is >>>> package-private, so it cannot be extended by clients. >>>> >>>> With best regards, >>>> Tagir Valeev. >> From luhenry at microsoft.com Sat Aug 8 17:30:07 2020 From: luhenry at microsoft.com (Ludovic Henry) Date: Sat, 8 Aug 2020 17:30:07 +0000 Subject: [11u] RFR[M]: 8250902: Implement MD5 Intrinsics on x86 In-Reply-To: <8b61da32-3cc1-fa14-a607-1f871f7e3d70@redhat.com> References: <8b61da32-3cc1-fa14-a607-1f871f7e3d70@redhat.com> Message-ID: Hi Andrew, Vladimir, > It's too early for that: changes are supposed to bake in JDK head for > a while. Also, since it's an enhancement rather than a bug fix we'd > need to have the discussion. I would say it's marginal whether > something like this should be back ported. > Usually we backport only bugs fixes to keep LTS (11u) release stable. It makes perfect sense. I'm happy to wait longer, and follow up on that thread later on to check if there is any appetite to get it backported. > You need also point if backport applied cleanly or you have to make changes. The code conflicts were trivial as the infrastructure for intrinsics didn't change much since 11 (and even 8). Conflicts: http://cr.openjdk.java.net/~luhenry/8250902-11u/webrev.00/conflict.diff > Changes should be backported separately to keep track - do not combine changes. > But it is okay to push both changesets together (especially if followup changes fixed first). Sorry I do not fully understand. Is it ok in this case to combine both changes into a single changeset, since the second one is a followup that fixes the first one? Or should I still make 2 changeset, but have them pushed together? Thank you, Ludovic From vladimir.kozlov at oracle.com Sun Aug 9 02:28:42 2020 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Sat, 8 Aug 2020 19:28:42 -0700 Subject: [11u] RFR[M]: 8250902: Implement MD5 Intrinsics on x86 In-Reply-To: References: <8b61da32-3cc1-fa14-a607-1f871f7e3d70@redhat.com> Message-ID: On 8/8/20 10:30 AM, Ludovic Henry wrote: > Hi Andrew, Vladimir, > >> It's too early for that: changes are supposed to bake in JDK head for >> a while. Also, since it's an enhancement rather than a bug fix we'd >> need to have the discussion. I would say it's marginal whether >> something like this should be back ported. > >> Usually we backport only bugs fixes to keep LTS (11u) release stable. > > It makes perfect sense. I'm happy to wait longer, and follow up on that thread later on to check if there is any appetite to get it backported. > >> You need also point if backport applied cleanly or you have to make changes. > > The code conflicts were trivial as the infrastructure for intrinsics didn't change much since 11 (and even 8). > > Conflicts: > http://cr.openjdk.java.net/~luhenry/8250902-11u/webrev.00/conflict.diff > >> Changes should be backported separately to keep track - do not combine changes. >> But it is okay to push both changesets together (especially if followup changes fixed first). > > Sorry I do not fully understand. Is it ok in this case to combine both changes into a single changeset, since the second one is a followup that fixes the first one? Or should I still make 2 changeset, but have them pushed together? It is not okay to combine changes into a single changeset. You need to make 2 (in this case) separate changesets but push them together. You can push them separately too but there is a chance that second push may miss a new build which would includes only first push. Also if a changeset applies cleanly you can use "hg export" and "hg import" commands - no need to do new commit. If changeset does not apply cleanly you need to send RFR for backport as you correctly did. Regards, Vladimir > > Thank you, > Ludovic > From aph at redhat.com Sun Aug 9 14:52:52 2020 From: aph at redhat.com (Andrew Haley) Date: Sun, 9 Aug 2020 15:52:52 +0100 Subject: Possible subtle memory model error in ClassValue In-Reply-To: References: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> Message-ID: <6f91ae57-8dae-be23-bab4-c47dbe8b95f7@redhat.com> On 8/7/20 10:35 PM, John Rose wrote: > (Here?s a tidbit of JMM politics: I once heard Doug Lea considering > whether maybe all fields should be treated more like final fields. > I don?t know if this is still a live idea, but it would make this > bug go way, since nearly all constructors would then get fences of > some sort.) I'd support that: accidental unsafe publication like this is a really counter-intuitive language feature. > Bottom line: Add a release fence before type.cVM is set, and add > a (no-op on x86) acquire fence in getCache. A StoreStore fence would be enough here, and cheaper on some systems. The memory dependency ordering from the load of classValueMap to the load of classValueMap.cache means we don't need an acquire fence. After all, final fields have the needed guarantee, only StoreStore fences, and no load fences. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From hboehm at google.com Sun Aug 9 17:55:07 2020 From: hboehm at google.com (Hans Boehm) Date: Sun, 9 Aug 2020 10:55:07 -0700 Subject: Possible subtle memory model error in ClassValue In-Reply-To: <6f91ae57-8dae-be23-bab4-c47dbe8b95f7@redhat.com> References: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> <6f91ae57-8dae-be23-bab4-c47dbe8b95f7@redhat.com> Message-ID: There is no guarantee that the address dependency enforces ordering if there is no final field involved. This may matter in the future, since ARM's Scalable Vector Extension does not guarantee ordering for address-dependent loads, if both loads are vector loads. I expect there are cases in which it would be profitable and safe, by the actual JMM rules, to use these. What bothers me most about effectively generalizing the JMM rules to reflect current implementations is that x = 1; // Initialization of non-final field in constructor assert x == 1; can fail, as can more interesting code that uses x after initialization in the constructor. The current rules are messy, but generally kind of make sense for final fields. The generalization to non-final fields is significantly stranger. Hans On Sun, Aug 9, 2020 at 7:53 AM Andrew Haley wrote: > On 8/7/20 10:35 PM, John Rose wrote: > > > (Here?s a tidbit of JMM politics: I once heard Doug Lea considering > > whether maybe all fields should be treated more like final fields. > > I don?t know if this is still a live idea, but it would make this > > bug go way, since nearly all constructors would then get fences of > > some sort.) > > I'd support that: accidental unsafe publication like this is a really > counter-intuitive language feature. > > > Bottom line: Add a release fence before type.cVM is set, and add > > a (no-op on x86) acquire fence in getCache. > > A StoreStore fence would be enough here, and cheaper on some systems. > The memory dependency ordering from the load of classValueMap to the > load of classValueMap.cache means we don't need an acquire fence. > > After all, final fields have the needed guarantee, only StoreStore > fences, and no load fences. > > -- > Andrew Haley (he/him) > Java Platform Lead Engineer > Red Hat UK Ltd. > https://keybase.io/andrewhaley > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 > > From galder at redhat.com Mon Aug 10 06:12:55 2020 From: galder at redhat.com (Galder Zamarreno) Date: Mon, 10 Aug 2020 08:12:55 +0200 Subject: Possible subtle memory model error in ClassValue In-Reply-To: References: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> Message-ID: Hi, I'm David's colleague. I'm the one who's spotted this issue twice while GraalVM is doing its points-to analysis with jdk11u-dev. The fuller exception is here: Caused by: java.lang.NullPointerException at java.base/java.lang.ClassValue$ClassValueMap.loadFromCache(ClassValue.java:535) at java.base/java.lang.ClassValue$ClassValueMap.probeHomeLocation(ClassValue.java:541) at java.base/java.lang.ClassValue.get(ClassValue.java:101) at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIMetaAccessContext.fromClass(HotSpotJVMCIMetaAccessContext.java:163) at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.fromClass(HotSpotJVMCIRuntime.java:339) at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotResolvedObjectTypeImpl.fromObjectClass(HotSpotResolvedObjectTypeImpl.java:83) at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotResolvedObjectTypeImpl.fromMetaspace(HotSpotResolvedObjectTypeImpl.java:97) at jdk.internal.vm.ci/jdk.vm.ci.hotspot.CompilerToVM.resolveTypeInPool(Native Method) at jdk.internal.vm.ci/jdk.vm.ci.hotspot.HotSpotConstantPool.loadReferencedType(HotSpotConstantPool.java:727) at java.base/jdk.internal.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at com.oracle.graal.pointsto.infrastructure.WrappedConstantPool.loadReferencedType(WrappedConstantPool.java:88) at com.oracle.graal.pointsto.infrastructure.WrappedConstantPool.loadReferencedType(WrappedConstantPool.java:105) at com.oracle.svm.hosted.phases.SubstrateClassInitializationPlugin.loadReferencedType(SubstrateClassInitializationPlugin.java:71) at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.loadReferenceType(BytecodeParser.java:4384) at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.maybeEagerlyResolve(BytecodeParser.java:4366) at com.oracle.svm.hosted.phases.SharedGraphBuilderPhase$SharedBytecodeParser.maybeEagerlyResolve(SharedGraphBuilderPhase.java:127) at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.lookupMethod(BytecodeParser.java:4316) at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genInvokeStatic(BytecodeParser.java:1659) at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBytecode(BytecodeParser.java:5340) at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3423) at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBlock(BytecodeParser.java:3230) at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.build(BytecodeParser.java:1088) at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.buildRootMethod(BytecodeParser.java:982) at jdk.internal.vm.compiler/org.graalvm.compiler.java.GraphBuilderPhase$Instance.run(GraphBuilderPhase.java:84) at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.run(Phase.java:49) at jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:214) at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:42) at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:38) at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:224) at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:351) at com.oracle.graal.pointsto.flow.MethodTypeFlow.doParse(MethodTypeFlow.java:322) at com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureParsed(MethodTypeFlow.java:311) at com.oracle.graal.pointsto.flow.MethodTypeFlow.addContext(MethodTypeFlow.java:112) at com.oracle.graal.pointsto.flow.StaticInvokeTypeFlow.update(InvokeTypeFlow.java:437) at com.oracle.graal.pointsto.BigBang$2.run(BigBang.java:530) at com.oracle.graal.pointsto.util.CompletionExecutor.lambda$execute$0(CompletionExecutor.java:173) at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426) ... 5 more On Fri, Aug 7, 2020 at 11:36 PM John Rose wrote: > On Aug 7, 2020, at 9:52 AM, Andrew Haley wrote: > > > > On 8/7/20 4:39 PM, David Lloyd wrote: > > > > > However, I'm wondering if this isn't a side effect of what appears > > > to be an incorrect double-checked lock at lines 374-378 of > > > ClassValue.java [1]. In order for the write to the non-volatile > > > `cache` field of ClassValueMap, it is my understanding that there > > > must be some acquire/release edge from where the variable is > > > published to guarantee that all of the writes are visible to the > > > read site, and I'm not really sure that the exit-the-constructor > > > edge is going to match up with with the synchronized block which > > > protects a different non-volatile field. And even if my feeling > > > that this is dodgy is valid, I'm not sure whether this NPE is a > > > possible outcome of that problem! > > > > It certainly doesn't look right to me. AFAICS this is classic broken > > double-checked locking. It'd need some sort of fence after > > constructing the ClassValueMap instance and before publishing it. > > Y?all are calling my baby ugly but he?s really cute if you look > at him from the right angle. > > (First, a quick question: This bug shows up on x86, right? > Yes, the issue is happening on x86. > If so, we probably are not looking at a hardware reordering > problem but something stemming from Graal?s reordering > of operations compared to C1 and C2, perhaps after a different > set of inlining decisions uncharacteristic of C2.) > To the best of my knowledge, GraalVM points-to analysis is just a JVM process that figures out which code is used by the end-user application. The result of this process is then used to create the native executable. I'm not aware of the Graal compiler having an impact here. >From what I've learnt looking at this code and more recent JVMCI implementations, jdk11u-dev relies on reflection to understand the end-user application, whereas more modern JVMCI stacks avoid the use of reflection and query the JVM directly. I've been trying to create a vanilla JDK stress test that would trigger such NPE but I've not had luck so far. I don't know whether JVMCI has a part to play here. > > The bug indeed looks like a dropped fence at the end of the > constructor for ClassValueMap. (I don?t see an easier way to > get a null at the NPE point.) The fence might not actually be > dropped, but a write which the fence is supposed to fence > might have been reordered after the fence, and after an > unlock operation, allowing someone to see the initial > null after the unlock. > > The double-check idiom breaks when the outer check > (not synchronized) gets half-baked data and acts on it. > This much-maligned idiom would not be broken in the > present case under the assumption that a data dependency > on type.classValueMap (set in initializeMap) will see > a fully initialized value of ClassValueMap.cacheArray. > > Now we get into the fine print, and I agree there is a bug > here. The easy fix is to repair the logic under which that > ugly everybody?s-hating-on-it double check idiom would > be in fact correct. > > The fine print requires a couple different things that are not > quite fulfilled by the present code, and Graal has either > reordered the memory accesses to expose the problem, or > it (itself) has a complementary bug. (Or it has an unrelated > bug, and you people are *staring* at my baby!) > > First, as Paul noted, you need a final variable in your class > to get the benefit of a fence at the end of the constructor. > Oops #1: ClassValueMap doesn?t have *any* finals. That?s > awkward. Two fixes for that: (a) add a dummy final, and > (b) add a real fence in the constructor. For better luck, > maybe put the fence at the end of sizeCache. > > On machines (not x86) which need fences *before* final > reads, another explicit fence should be placed before the > unsynchronized read (or else inside the getCache method). > > By the letter of the JMM, the helpful effects of the fence > at the end of the constructor are only guaranteed for > references which depend on final variables set by that > constructor, because only those final variables get a > ?freeze? operation which stabilizes them (and values > dependently loaded via them). Throwing in a dummy > final is therefore not guaranteed to work. But throwing > in a fence will work. One downside of the fence is that > the JMM is silent about fences, but the JMM is widely > recognized as a partial story, not the full or final story. > > (Here?s a tidbit of JMM politics: I once heard Doug Lea > considering whether maybe all fields should be treated > more like final fields. I don?t know if this is still a live > idea, but it would make this bug go way, since nearly all > constructors would then get fences of some sort.) > > Here?s a bit of explanatory (non-normative) text from the draft > POPL paper for JMM, where the parenthetic comment indicates > (I think) the sort of thing that is happening here: > > > Let us now assume that the acquire and release do happen. As long as > > these actions take place after object has been constructed (and there > > is no code motion around the end of the constructor), the diffs that > > the second processor acquires are guaranteed to reflect the correctly > > constructed object. > > Basically, the synchronization statement is expected to do a > release *after* the non-null value is stored. It looks like this > is failing due to a lost and/or reordered fence. > > Second, David says: > > > I?m not really sure that the exit-the-constructor edge is going to match > > up with with the synchronized block which protects a different > > non-volatile field. > > By the letter of the JMM (AFAIUI), this code is coloring way outside > the lines. (Yes, I admit it.) The problem is pretty fundamental. > I went and swapped in some JMM spec just now (don?t have it in > short term memory; go figure), and here?s what I re-learned. > > In JMM terms, a ?release? is the earlier end of any edge in relation > called ?synchronizes-with?. An ?acquire? is the latter end of such > an edge. In the JMM this relation applies only to lock, unlock, > and volatile reads and writes. Your guess is as good as mine whether > other operations (CAS, fences, etc.) participate; the JMM is silent. > Crucially, acquires and release do not touch plain fields. This is > counter-intuitive for those of us who like to reason with fences. > > The JMM prevents regular writes from floating past what we like > to think of as ?release points? (unlocks) by appealing to processor > order inside lock/unlock pairs, and also by appealing to global > ordering of multiple lock/unlock pairs (or stuff with volatiles > and other corner cases which we will neglect here). In order > to work right, a normal write has to come before a release *and* > a matching normal read has to come after an acquire, and such > an acquire is nothing other than the lock of a later lock/unlock > pair, typically in another thread. > > So the rules which reliably connect normal writes to normal > reads in other threads work differently from acquire fences and > release fences as we (or just I?) usually think of them. Yes, there > are ?acquires? and ?releases? in the JMM. No, they are not > primitives but rather descriptions of the way the happens-before > relation is constrained by (among other things) lock and unlock > actions. > > On x86, you don?t have to worry about acquires; you just set up > the right release fences. In the JMM, there?s no way to get either > an acquire or a release without a synchronized statement (or > other fancy stuff like volatiles). That?s why double-check is > broken in the pure JMM, and why it can be repaired if you > add some (non-JMM) fences. > > So why is this showing up suddenly with Graal? Possibly it?s > got a really aggressive interpretation of the JMM, relative to > the standard HotSpot JITs. Possibly it?s got a bug. Graal might > be allowing the initialization of ClassValueMap.cacheArray > to float down past publication of the ClassValueMap on > type.classValueMap (in initializeMap). Less likely, it is allowing > the initialization to totally escape the lock/unlock pair, something > the JMM might allow but the Cookbook advises against (in its > ?Can Reorder? table). > > Perhaps Graal is modeling the ?freeze? operations on finals more > accurately (and the Cookbook gives instructions for this). That > would allow the ClassValueMap to correctly initialize its frozen > finals, but not (unfortunately) the not-frozen cacheArray field. > It appears that the new JIT on the block is exposing my neglect > for my poor baby, in not putting the right fences around its playpen. > > Bottom line: Add a release fence before type.cVM is set, and add > a (no-op on x86) acquire fence in getCache. Quick test: Add a dummy > final to CVM, to see if that makes the bug let go. > > ? John > > From dl at cs.oswego.edu Mon Aug 10 12:17:44 2020 From: dl at cs.oswego.edu (Doug Lea) Date: Mon, 10 Aug 2020 08:17:44 -0400 Subject: Possible subtle memory model error in ClassValue In-Reply-To: <464fc65f-308d-5636-8c83-4592257e96d9@oracle.com> References: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> <33C0A687-7B1D-49DB-86F8-CE778BC1456A@oracle.com> <464fc65f-308d-5636-8c83-4592257e96d9@oracle.com> Message-ID: <5c0e242b-59a7-afd1-710a-606e541dd628@cs.oswego.edu> Catching up... As implied in other posts, the minimal fix is to add a trailing release fence (using Unsafe?) to the constructor. Or less delicately, to access only using acquire/release (which will cost a bit on ARM/Power, but probably not noticeable on x86), or most simply (but expensively) to declare the field volatile. Also, as Hans noted, the consensus seems to be that there not enough to be gained by always adding a release fence to constructors. A few errors like this might never occur, but other related anomalies with non-final field accesses would remain. -Doug From arno.zeller at sap.com Mon Aug 10 12:57:43 2020 From: arno.zeller at sap.com (Zeller, Arno) Date: Mon, 10 Aug 2020 12:57:43 +0000 Subject: [16] 8251017: java/io/File/GetXSpace.java fails on UNIX In-Reply-To: <5D1AF455-226C-43E6-A6A8-3C6A28A07196@oracle.com> References: <361C144E-A680-4B64-A13E-292A1F075173@oracle.com> <5D1AF455-226C-43E6-A6A8-3C6A28A07196@oracle.com> Message-ID: Hi Brian, just a question: I only see a fix for the AIX issue in your change. As far as I understand Matthias the main issue in https://bugs.openjdk.java.net/browse/JDK-8251017 is that NFS mounts (that often change the usable size during the test run) are not excluded. We see this kind of failure very often on our Linux test systems. Will this be handled in another bug? Best regards, Arno > -----Original Message----- > From: core-libs-dev On Behalf Of Brian > Burkhalter > Sent: Dienstag, 4. August 2020 20:09 > To: naoto.sato at oracle.com > Cc: core-libs-dev > Subject: Re: [16] 8251017: java/io/File/GetXSpace.java fails on UNIX > > Thanks, Naoto. I?ll hold off pushing it until the review in [2] is done. > > Brian > > > On Aug 4, 2020, at 10:41 AM, naoto.sato at oracle.com wrote: > > > > Looks good, Brian. > > > > Naoto > > > > On 8/4/20 9:01 AM, Brian Burkhalter wrote: > >> Add @requires tag for [1]. This patch is layered on top of that proposed in > [2]. > >> --- a/test/jdk/java/io/File/GetXSpace.java > >> +++ b/test/jdk/java/io/File/GetXSpace.java > >> @@ -24,6 +24,7 @@ > >> /** > >> * @test > >> * @bug 4057701 6286712 6364377 > >> + * @requires (os.family == "linux" | os.family == "mac" | os.family == > "windows") > >> * @run build GetXSpace > >> * @run shell GetXSpace.sh > >> * @summary Basic functionality of File.get-X-Space methods. > >> Thanks, > >> Brian > >> [1] https://bugs.openjdk.java.net/browse/JDK-8251017 > >> [2] http://mail.openjdk.java.net/pipermail/core-libs-dev/2020- > July/067990.html From matthias.baesken at sap.com Mon Aug 10 13:14:38 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 10 Aug 2020 13:14:38 +0000 Subject: [16] 8251017: java/io/File/GetXSpace.java fails on UNIX In-Reply-To: References: <361C144E-A680-4B64-A13E-292A1F075173@oracle.com> <5D1AF455-226C-43E6-A6A8-3C6A28A07196@oracle.com> Message-ID: Hi Arno yes I think so ; the NFS issue seems to appear on Linux quite often , noticed it a number of times in our nightly tests . Best regards, Matthias -----Original Message----- From: Zeller, Arno Sent: Montag, 10. August 2020 14:58 To: Brian Burkhalter Cc: core-libs-dev ; Baesken, Matthias Subject: RE: [16] 8251017: java/io/File/GetXSpace.java fails on UNIX Hi Brian, just a question: I only see a fix for the AIX issue in your change. As far as I understand Matthias the main issue in https://bugs.openjdk.java.net/browse/JDK-8251017 is that NFS mounts (that often change the usable size during the test run) are not excluded. We see this kind of failure very often on our Linux test systems. Will this be handled in another bug? Best regards, Arno > -----Original Message----- > From: core-libs-dev On Behalf Of Brian > Burkhalter > Sent: Dienstag, 4. August 2020 20:09 > To: naoto.sato at oracle.com > Cc: core-libs-dev > Subject: Re: [16] 8251017: java/io/File/GetXSpace.java fails on UNIX > > Thanks, Naoto. I?ll hold off pushing it until the review in [2] is done. > > Brian > > > On Aug 4, 2020, at 10:41 AM, naoto.sato at oracle.com wrote: > > > > Looks good, Brian. > > > > Naoto > > > > On 8/4/20 9:01 AM, Brian Burkhalter wrote: > >> Add @requires tag for [1]. This patch is layered on top of that proposed in > [2]. > >> --- a/test/jdk/java/io/File/GetXSpace.java > >> +++ b/test/jdk/java/io/File/GetXSpace.java > >> @@ -24,6 +24,7 @@ > >> /** > >> * @test > >> * @bug 4057701 6286712 6364377 > >> + * @requires (os.family == "linux" | os.family == "mac" | os.family == > "windows") > >> * @run build GetXSpace > >> * @run shell GetXSpace.sh > >> * @summary Basic functionality of File.get-X-Space methods. > >> Thanks, > >> Brian > >> [1] https://bugs.openjdk.java.net/browse/JDK-8251017 > >> [2] http://mail.openjdk.java.net/pipermail/core-libs-dev/2020- > July/067990.html From brian.burkhalter at oracle.com Mon Aug 10 15:40:08 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 10 Aug 2020 08:40:08 -0700 Subject: [16] 8251017: java/io/File/GetXSpace.java fails on UNIX In-Reply-To: References: <361C144E-A680-4B64-A13E-292A1F075173@oracle.com> <5D1AF455-226C-43E6-A6A8-3C6A28A07196@oracle.com> Message-ID: The patch addresses only the situation in the comments, excluding the test on AIX. A separate issue should be filed to address the NFS mounts situation. Thanks, Brian > On Aug 10, 2020, at 6:14 AM, Baesken, Matthias wrote: > > Hi Arno yes I think so ; the NFS issue seems to appear on Linux quite often , noticed it a number of times in our nightly tests . > > Best regards, Matthias > > -----Original Message----- > From: Zeller, Arno > Sent: Montag, 10. August 2020 14:58 > To: Brian Burkhalter > Cc: core-libs-dev ; Baesken, Matthias > Subject: RE: [16] 8251017: java/io/File/GetXSpace.java fails on UNIX > > Hi Brian, > > just a question: I only see a fix for the AIX issue in your change. As far as I understand Matthias the main issue in https://bugs.openjdk.java.net/browse/JDK-8251017 is that NFS mounts (that often change the usable size during the test run) are not excluded. > We see this kind of failure very often on our Linux test systems. Will this be handled in another bug? > > Best regards, > Arno From alexey.semenyuk at oracle.com Mon Aug 10 16:09:52 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Mon, 10 Aug 2020 12:09:52 -0400 Subject: RFR: JDK-8232621: L10n issues with msi installers In-Reply-To: References: Message-ID: <4f70f84a-fa04-dc61-fb7f-f55aa04d6087@oracle.com> Please review fix [2] for jpackage bug [1]. The fix adds ability to jpackage to hook up custom WiX localization files from resource directory. This allows to create .msi installers with custom locales, not just with three locales supported by OpenJDK. - Alexey [1] https://bugs.openjdk.java.net/browse/JDK-8232621 [2]?http://cr.openjdk.java.net/~asemenyuk/8232621/webrev.00 From andy.herrick at oracle.com Mon Aug 10 16:51:50 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Mon, 10 Aug 2020 12:51:50 -0400 Subject: RFR: JDK-8232621: L10n issues with msi installers In-Reply-To: <4f70f84a-fa04-dc61-fb7f-f55aa04d6087@oracle.com> References: <4f70f84a-fa04-dc61-fb7f-f55aa04d6087@oracle.com> Message-ID: looks good to me. /ANdy On 8/10/2020 12:09 PM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > The fix adds ability to jpackage to hook up custom WiX localization > files from resource directory. This allows to create .msi installers > with custom locales, not just with three locales supported by OpenJDK. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8232621 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8232621/webrev.00 > From cjashfor at linux.ibm.com Mon Aug 10 21:13:18 2020 From: cjashfor at linux.ibm.com (Corey Ashford) Date: Mon, 10 Aug 2020 14:13:18 -0700 Subject: RFR(S): 8248188 Add HotSpotIntrinsicCandidate and API for Base64 decoding (repost to correct mailing list) In-Reply-To: <06a915ca-5ae2-f350-1ef6-59e467b24084@linux.ibm.com> References: <06a915ca-5ae2-f350-1ef6-59e467b24084@linux.ibm.com> Message-ID: <36a15d9c-a317-1900-0e3d-43d2348ebd85@linux.ibm.com> Hello, I had originally made two posts to the hotspot-compiler-dev and ppc-aix-port-dev mailing lists, but since this RFR requires modifying the core library code, it makes sense to post it to core-lib-devs instead. The two original posts: https://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2020-June/004110.html https://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2020-July/004115.html Webrev: http://cr.openjdk.java.net/~mhorie/8248188/webrev.01/ To recap the current state: Currently in java.util.Base64, there is a HotSpotIntrinsicCandidate and API for encodeBlock, but none for decoding. This means that only encoding gets acceleration from the underlying CPU's vector or other accelerator hardware. I'd like to propose adding a new intrinsic for decodeBlock. The considerations I have for this new intrinsic's API: * Don't make any assumptions about the underlying capability of the hardware. For example, do not impose any specific block size granularity. * Because of the difficulty of handling the isMIME=true option in a vector-based accelerator, do not call this intrinsic when isMIME=true. If desired, a new HotSpotIntrinsicCandidate can be introduced in the future for this case. * Any remaining data that is not processed by the intrinsic will be processed by the pure Java implementation. This allows the intrinsic to process whatever block sizes it's good at without the complexity of handling the end fragments. * If any illegal character is discovered in the decoding process, the intrinsic can return 0, or any number of bytes produced, up to but not including the byte containing the error. At this point, the pure-Java algorithm will be called, which will discover the error and report it via a thrown exception as usual. I've tested the patch in the webrev with an actual intrinsic coded up for Power9/Power10, but that runtime intrinsic and arch-specific patches aren't in the webrev, because I want to get some consensus on the library-level intrinsic API first. I'm open to any comments about this. Thanks for your consideration, - Corey Corey Ashford Software Engineer IBM Systems, LTC OpenJDK team 503-985-8699 Home office cjashfor at us.ibm.com IBM From alexander.matveev at oracle.com Mon Aug 10 22:54:18 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Mon, 10 Aug 2020 15:54:18 -0700 Subject: RFR: JDK-8232621: L10n issues with msi installers In-Reply-To: <4f70f84a-fa04-dc61-fb7f-f55aa04d6087@oracle.com> References: <4f70f84a-fa04-dc61-fb7f-f55aa04d6087@oracle.com> Message-ID: Hi Alexey, Looks good. Thanks, Alexander On 8/10/20 9:09 AM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > The fix adds ability to jpackage to hook up custom WiX localization > files from resource directory. This allows to create .msi installers > with custom locales, not just with three locales supported by OpenJDK. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8232621 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8232621/webrev.00 > From joe.darcy at oracle.com Tue Aug 11 05:47:55 2020 From: joe.darcy at oracle.com (Joe Darcy) Date: Mon, 10 Aug 2020 22:47:55 -0700 Subject: Fix for Javadoc errors in java.base In-Reply-To: References: <193fee5f-ef90-1048-7455-8363a5e616b4@oracle.com> Message-ID: <64a15802-4208-4818-b9e9-d1fddf9460cc@oracle.com> Hi Vipin, My review comments on the files I look after is interspersed below. On 7/27/2020 10:28 PM, Vipin Sharma wrote: > Hi Joe, > > I ran an IntelliJ inspection to find these errors. > > Regards, > Vipin > > > Virus-free. www.avast.com > > > > > On Tue, Jul 28, 2020 at 5:53 AM Joe Darcy > wrote: > > Hello Vipin, > > What tooling was used to find these errors? Running doclint against > private methods too? > > Thanks, > > -Joe > > On 7/27/2020 12:37 PM, Vipin Sharma wrote: > > Hi, > > > > This patch fixes Javadoc errors in java.base, please sponsor > this patch and > > help to create a bug id for this. > > I have signed OCA, this is my third patch. > > > > > --- old/src/java.base/share/classes/java/math/BigDecimal.java > 2020-07-25 > > 23:46:24.285759291 +0530 > > +++ new/src/java.base/share/classes/java/math/BigDecimal.java > 2020-07-25 > > 23:46:23.785753966 +0530 > > @@ -5413,7 +5413,7 @@ > >? ? ? ? * > >? ? ? ? * @param n the numerator; must be negative > >? ? ? ? * @param d the denominator; must not be unity > > -? ? ?* @return a two-element {@long} array with the remainder > and quotient > > in > > +? ? ?* @return a two-element {@code long} array with the > remainder and > > quotient in > >? ? ? ? *? ? ? ? ?the initial and final elements, respectively > >? ? ? ? */ > >? ? ? ?private static long[] divRemNegativeLong(long n, long d) { > Looks fine. > > --- old/src/java.base/share/classes/java/math/MutableBigInteger.java > > 2020-07-25 23:46:25.381770889 +0530 > > +++ new/src/java.base/share/classes/java/math/MutableBigInteger.java > > 2020-07-25 23:46:24.877765572 +0530 > > @@ -145,7 +145,6 @@ > >? ? ? ? * Makes this number an {@code n}-int number all of whose > bits are > > ones. > >? ? ? ? * Used by Burnikel-Ziegler division. > >? ? ? ? * @param n number of ints in the {@code value} array > > -? ? ?* @return a number equal to {@code ((1<<(32*n)))-1} > >? ? ? ? */ > >? ? ? ?private void ones(int n) { > >? ? ? ? ? ?if (n > value.length) > Looks fine. > > > old/src/java.base/share/classes/sun/reflect/annotation/AnnotationParser.java > > 2020-07-25 23:46:28.545803801 +0530 > > +++ > > > new/src/java.base/share/classes/sun/reflect/annotation/AnnotationParser.java > > 2020-07-25 23:46:28.029798501 +0530 > > @@ -1,5 +1,5 @@ > >? ?/* > > - * Copyright (c) 2003, 2013, Oracle and/or its affiliates. All > rights > > reserved. > > + * Copyright (c) 2003, 2020, Oracle and/or its affiliates. All > rights > > reserved. > >? ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > >? ? * > >? ? * This code is free software; you can redistribute it and/or > modify it > > @@ -773,7 +773,7 @@ > >? ? ? ? * an "annotation structure" OR two bytes into an annotation > >? ? ? ? * structure (i.e., after the type index). > >? ? ? ? * > > -? ? ?* @parameter complete true if the byte buffer points to > the beginning > > +? ? ?* @param complete true if the byte buffer points to the > beginning > >? ? ? ? *? ? ?of an annotation structure (rather than two bytes in). > >? ? ? ? */ > >? ? ? ?private static void skipAnnotation(ByteBuffer buf, boolean > complete) { > Also looks fine. Cheers, -Joe > > > From aph at redhat.com Tue Aug 11 10:47:12 2020 From: aph at redhat.com (Andrew Haley) Date: Tue, 11 Aug 2020 11:47:12 +0100 Subject: Possible subtle memory model error in ClassValue In-Reply-To: References: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> <6f91ae57-8dae-be23-bab4-c47dbe8b95f7@redhat.com> Message-ID: <0c849c2a-0a3a-704a-3ee3-22b690aa0d34@redhat.com> On 09/08/2020 18:55, Hans Boehm wrote: > There is no guarantee that the address dependency enforces ordering > if there is no final field involved. This may matter in the future, > since ARM's Scalable Vector Extension does not guarantee ordering > for address-dependent loads, if both loads are vector loads. Ouch. Thanks, I didn't know that. > I expect there are cases in which it would be profitable and safe, > by the actual JMM rules, to use these. But... we have to implement final fields somehow. And if dependency ordering isn't sufficient then we're going to have to use fences before SVE loads. Given that data in SVE registers may be security sensitive, such as crypto keys, we'd have to do that. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From adinn at redhat.com Tue Aug 11 11:03:35 2020 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 11 Aug 2020 12:03:35 +0100 Subject: Possible subtle memory model error in ClassValue In-Reply-To: <0c849c2a-0a3a-704a-3ee3-22b690aa0d34@redhat.com> References: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> <6f91ae57-8dae-be23-bab4-c47dbe8b95f7@redhat.com> <0c849c2a-0a3a-704a-3ee3-22b690aa0d34@redhat.com> Message-ID: On 11/08/2020 11:47, Andrew Haley wrote: > On 09/08/2020 18:55, Hans Boehm wrote: >> There is no guarantee that the address dependency enforces ordering >> if there is no final field involved. This may matter in the future, >> since ARM's Scalable Vector Extension does not guarantee ordering >> for address-dependent loads, if both loads are vector loads. > > Ouch. Thanks, I didn't know that. You ought to look at the pdf Ningsheng linked in the RFR that was posted with the SVE patch. The pdf is available here: https://developer.arm.com/docs/ddi0584/latest The relevant text is in section 4.4. Memory Ordering. regards, Andrew Dinn ----------- Red Hat Distinguished Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill From daniel.fuchs at oracle.com Tue Aug 11 11:09:04 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 11 Aug 2020 12:09:04 +0100 Subject: RFR[testbug]: 8251189: com/sun/jndi/ldap/LdapDnsProviderTest.java failed due to timeout In-Reply-To: References: Message-ID: <8111b911-4399-4e78-8701-26a4d455e51c@oracle.com> Hi Aleksei, Looks good to me! best regards, -- daniel On 06/08/2020 13:44, Aleks Efimov wrote: > Hi, > > LdapDnsProviderTest was seen failing due to the timeout caused by the > blocked bind operation. That could happen if there is a local process > listening on the port specified in the test URL. > To make LdapProviderTest.ProviderTest.call fail the LDAP connect timeout > property needs to be set to greater than 0 value. That will help > runLocalHostTestWithRandomPort to fail with the timeout exception if the > port is busy and try another random port. > Also, the test output has been modified not to print stack traces for > the expected exceptions. > > Webrev: http://cr.openjdk.java.net/~aefimov/8251189/00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8251189 > > The test was executed 100+ times alongside to other LDAP tests and was > not seen failing with the > ?proposed changes. > > With Best Regards, > Aleksei > From daniel.fuchs at oracle.com Tue Aug 11 11:10:50 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 11 Aug 2020 12:10:50 +0100 Subject: RFR [testbug]: 8250772: Test com/sun/jndi/ldap/NamingExceptionMessageTest.java fails intermittently with javax.naming.ServiceUnavailableException In-Reply-To: References: Message-ID: Hi Aleksei, I agree that this is the best fix for this issue. best regards, -- daniel On 06/08/2020 23:21, Aleks Efimov wrote: > Hi, > > Please, help to review NamingExceptionMessageTest test stabilization > change: > There is a race condition between the closure of connection in test LDAP > server and the JDKs LDAP client code. > Due to this race the test could generate ServiceUnavailableException > instead of the required NamingException. > > Webrev: http://cr.openjdk.java.net/~aefimov/8250772/00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8250772 > > The following modifications have been made to the test code: > a) To increase the reproduction rate of the required exception message > the connection closure timeout has been added to the test LDAP server's > BIND operation handler > b) ServiceUnavailableException is now treated as an acceptable exception > > The modified test was successfully executed 100+ times alongside to > other LDAP tests > > With Best Regards, > Aleksei From vyommani at gmail.com Tue Aug 11 13:12:30 2020 From: vyommani at gmail.com (Vyom Tiwari) Date: Tue, 11 Aug 2020 18:42:30 +0530 Subject: RFR[testbug]: 8251189: com/sun/jndi/ldap/LdapDnsProviderTest.java failed due to timeout In-Reply-To: <8111b911-4399-4e78-8701-26a4d455e51c@oracle.com> References: <8111b911-4399-4e78-8701-26a4d455e51c@oracle.com> Message-ID: +1 Vyom On Tue, Aug 11, 2020 at 4:39 PM Daniel Fuchs wrote: > Hi Aleksei, > > Looks good to me! > > best regards, > > -- daniel > > On 06/08/2020 13:44, Aleks Efimov wrote: > > Hi, > > > > LdapDnsProviderTest was seen failing due to the timeout caused by the > > blocked bind operation. That could happen if there is a local process > > listening on the port specified in the test URL. > > To make LdapProviderTest.ProviderTest.call fail the LDAP connect timeout > > property needs to be set to greater than 0 value. That will help > > runLocalHostTestWithRandomPort to fail with the timeout exception if the > > port is busy and try another random port. > > Also, the test output has been modified not to print stack traces for > > the expected exceptions. > > > > Webrev: http://cr.openjdk.java.net/~aefimov/8251189/00/index.html > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251189 > > > > The test was executed 100+ times alongside to other LDAP tests and was > > not seen failing with the > > proposed changes. > > > > With Best Regards, > > Aleksei > > > > -- Thanks, Vyom From aph at redhat.com Tue Aug 11 13:17:20 2020 From: aph at redhat.com (Andrew Haley) Date: Tue, 11 Aug 2020 14:17:20 +0100 Subject: Possible subtle memory model error in ClassValue In-Reply-To: References: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> <6f91ae57-8dae-be23-bab4-c47dbe8b95f7@redhat.com> <0c849c2a-0a3a-704a-3ee3-22b690aa0d34@redhat.com> Message-ID: <04069f52-c9ef-8fee-26c7-5fdb69e8860b@redhat.com> On 11/08/2020 12:03, Andrew Dinn wrote: > You ought to look at the pdf Ningsheng linked in the RFR that was posted > with the SVE patch. The pdf is available here: > > https://developer.arm.com/docs/ddi0584/latest > > The relevant text is in section 4.4. Memory Ordering. That looks better than I feared. The only relevant text here AFAIUI is "If an address dependency exists between two memory reads, and an SVE non-temporal vector load instruction generated the second read, then in the absence of any other barrier mechanism to achieve order, the memory accesses can be observed in any order by the other observers within the shareability domain of the memory addresses being accessed." ... but this is only about non-temporal vector load instructions. It does mean that if we want to use those we'll have to separate them from loads of base addresses with fences. So it'll be Load base register load fence ... vector loop ... -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From arno.zeller at sap.com Tue Aug 11 13:39:29 2020 From: arno.zeller at sap.com (Zeller, Arno) Date: Tue, 11 Aug 2020 13:39:29 +0000 Subject: [15] RFR(S) : 6501010 : test/java/io/File/GetXSpace.java fails on Windows In-Reply-To: <880A7150-A724-4205-93B2-E4C1CE20A67E@oracle.com> References: <2236D17C-4A2E-4BA5-A9AA-7C50A96CC348@oracle.com> <880A7150-A724-4205-93B2-E4C1CE20A67E@oracle.com> Message-ID: Hi Igor, after our push I see test/java/io/File/GetXSpace.java failing on our Windows test machines. The issue seems to be that the 'df' call produces several lines of output on my windows machine and the pattern uses \n as line-ending. In a short test I changed the regex to match $ instead of \n in the end and added the MULTILINE flag. -------------------------------------------------------------------------- diff -r 16625b2b71f7 test/jdk/java/io/File/GetXSpace.java --- a/test/jdk/java/io/File/GetXSpace.java Tue Aug 11 07:29:45 2020 -0400 +++ b/test/jdk/java/io/File/GetXSpace.java Tue Aug 11 15:35:15 2020 +0200 @@ -55,7 +55,7 @@ private static final boolean IS_WIN = OS_NAME.startsWith("Windows"); // FileSystem Total Used Available Use% MountedOn - private static final Pattern DF_PATTERN = Pattern.compile("([^\\s]+)\\s+(\\d+)\\s+\\d+\\s+(\\d+)\\s+\\d+%\\s+([^\\s].*)\n"); + private static final Pattern DF_PATTERN = Pattern.compile("([^\\s]+)\\s+(\\d+)\\s+\\d+\\s+(\\d+)\\s+\\d+%\\s+([^\\s].*)$", Pattern.MULTILINE); private static int fail = 0; private static int pass = 0; -------------------------------------------------------------------------- Now the test passes for me and in our infrastructure. I did not completely understand why the old pattern did loose the first character in the line - but this seems to fix the issue. Before my change: ----------System.out:(24/869)---------- --- Testing df C:/cygwin64 998257472 664791328 333466144 67% / K: 262144000 129993392 132150608 50% /cygdrive/k O: 734003200 641218112 92785088 88% /cygdrive/o SecurityManager = null C:/cygwin64: df total= 1022215651328 free = 0 usable = 341469331456 getX total= 1022215651328 free = 341469323264 usable = 341469323264 :: df total= 268435456000 free = 0 usable = 135322222592 getX total= 0 free = 0 usable = 0 FAILED After my change: --- Testing df C:/cygwin64 998257472 665305100 332952372 67% / K: 262144000 129993392 132150608 50% /cygdrive/k O: 734003200 641211140 92792060 88% /cygdrive/o SecurityManager = null C:/cygwin64: df total= 1022215651328 free = 0 usable = 340943228928 getX total= 1022215651328 free = 340943286272 usable = 340943286272 K:: df total= 268435456000 free = 0 usable = 135322222592 getX total= 268435456000 free = 135322222592 usable = 135322222592 O:: df total= 751619276800 free = 0 usable = 95019069440 getX total= 751619276800 free = 95019069440 usable = 95019069440 ... ------------------------------------------ The drives K: and O: are mapped network drives. Do you see something similar in your landscape? Can you perhaps try to reproduce this? Best regards, Arno > -----Original Message----- > From: core-libs-dev On Behalf Of Igor > Ignatyev > Sent: Freitag, 31. Juli 2020 04:45 > To: Brian Burkhalter > Cc: core-libs-dev > Subject: Re: [15] RFR(S) : 6501010 : test/java/io/File/GetXSpace.java fails on > Windows > > Hi Brian, > > thanks for your review. I've decided to push into jdk/jdk instead of jdk15. > please let me know if you think this patch needs to be in jdk15, and I'll backport > it there. > > Cheers, > -- Igor > > > On Jul 30, 2020, at 2:39 PM, Brian Burkhalter > wrote: > > > > Hi Igor, > > > > This test looks all right to me. > > > > Sorry for the late review. > > > > Thanks, > > > > Brian > > > >> On Jul 20, 2020, at 11:28 AM, Igor Ignatyev > wrote: > >> > >> http://cr.openjdk.java.net/~iignatyev//6501010/webrev.00 > > >>> 98 lines changed: 18 ins; 31 del; 49 mod; > >> > >> Hi all, > >> > >> could you please review this small fix to make java/io/File/GetXSpace.java > work again on windows? > >> > >> the test has been updated to work under cygwin, which includes the > following changes: > >> - GetXSpace.sh accepts CYGWIN_* as known OS and sets windows path to > TMP var > >> - uses the same regexp to parse df on windows as on other platforms > >> - but uses 'FileSystem' field as Space's name, as opposed to 'MountedOn' on > *nix, as on cygwin, MountedOn is cygwin path, while FileSystem is > corresponding windows path, and the test expect Space's name to be > appropriate path > > From hboehm at google.com Tue Aug 11 17:06:15 2020 From: hboehm at google.com (Hans Boehm) Date: Tue, 11 Aug 2020 10:06:15 -0700 Subject: Possible subtle memory model error in ClassValue In-Reply-To: <04069f52-c9ef-8fee-26c7-5fdb69e8860b@redhat.com> References: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> <6f91ae57-8dae-be23-bab4-c47dbe8b95f7@redhat.com> <0c849c2a-0a3a-704a-3ee3-22b690aa0d34@redhat.com> <04069f52-c9ef-8fee-26c7-5fdb69e8860b@redhat.com> Message-ID: I think the relevant statement is: "An address dependency between two reads generated by SVE vector load instructions does not contribute to the Dependency-ordered-before relation." This is only an issue if BOTH loads are SVE loads. In particular reference loads have to be vectorized for this to matter, which I expect is not the common situation. I have not explored this in great detail, but it should suffice to put fences between two dependent vector reference loads, and between a reference load and a dependent final field load. On Tue, Aug 11, 2020 at 6:17 AM Andrew Haley wrote: > On 11/08/2020 12:03, Andrew Dinn wrote: > > You ought to look at the pdf Ningsheng linked in the RFR that was posted > > with the SVE patch. The pdf is available here: > > > > https://developer.arm.com/docs/ddi0584/latest > > > > The relevant text is in section 4.4. Memory Ordering. > > That looks better than I feared. The only relevant text here AFAIUI is > > "If an address dependency exists between two memory reads, and an SVE > non-temporal vector load instruction generated the second read, then > in the absence of any other barrier mechanism to achieve order, the > memory accesses can be observed in any order by the other observers > within the shareability domain of the memory addresses being > accessed." > > ... but this is only about non-temporal vector load instructions. It > does mean that if we want to use those we'll have to separate them > from loads of base addresses with fences. > > So it'll be > > Load base register > load fence > ... vector loop ... > > -- > Andrew Haley (he/him) > Java Platform Lead Engineer > Red Hat UK Ltd. > https://keybase.io/andrewhaley > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 > > From brian.burkhalter at oracle.com Tue Aug 11 17:32:11 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 11 Aug 2020 10:32:11 -0700 Subject: 8245304: Re-examine ThreadLocal usage in java.math.BigDecimal Message-ID: To fix [1], please consider [2] which replaces the ThreadLocal ?threadLocalStringBuilderHelper? with straightforward use of a StringBuilderHelper instance. The initial capacity of the StringBuilder instance variable of StringBuilderHelper is also doubled from 16 to 32. Microbenchmarking was effected primarily using BigDecimal.toEngineeringString(). This method and toString() both invoke layoutChars() which used the ThreadLocal. The toString() method caches its return value but toEngineeringString() does not. Benchmarking using a single Thread did not show any regression for toEngineeringString() and in fact showed a slight improvement. Since toEngineeringString() does not cache its result, one would expect the benchmarks for toString() not to deteriorate either, and simple benchmarking validated this. Given the absence of any measurable performance regression it appears that this ThreadLocal may safely be removed. Thanks, Brian [1] https://bugs.openjdk.java.net/browse/JDK-8245304 [2] http://cr.openjdk.java.net/~bpb/8245304/webrev.00/ From igor.ignatyev at oracle.com Tue Aug 11 17:33:22 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 11 Aug 2020 10:33:22 -0700 Subject: [15] RFR(S) : 6501010 : test/java/io/File/GetXSpace.java fails on Windows In-Reply-To: References: <2236D17C-4A2E-4BA5-A9AA-7C50A96CC348@oracle.com> <880A7150-A724-4205-93B2-E4C1CE20A67E@oracle.com> Message-ID: <700C5C6A-7EDB-4869-AE94-3B50D3AF1CA2@oracle.com> Hi Arno, We haven't seen such failures in our infra, I guess we just need to add ^ to the beginning of the pattern. please file a bug and send a proper RFR for it and will be happy to run the patch in our testing infra for you. Cheers, -- Igor > On Aug 11, 2020, at 6:39 AM, Zeller, Arno wrote: > > Hi Igor, > > after our push I see test/java/io/File/GetXSpace.java failing on our Windows test machines. The issue seems to be that the 'df' call produces several lines of output on my windows machine and the pattern uses \n as line-ending. > > In a short test I changed the regex to match $ instead of \n in the end and added the MULTILINE flag. > > -------------------------------------------------------------------------- > diff -r 16625b2b71f7 test/jdk/java/io/File/GetXSpace.java > --- a/test/jdk/java/io/File/GetXSpace.java Tue Aug 11 07:29:45 2020 -0400 > +++ b/test/jdk/java/io/File/GetXSpace.java Tue Aug 11 15:35:15 2020 +0200 > @@ -55,7 +55,7 @@ > private static final boolean IS_WIN = OS_NAME.startsWith("Windows"); > > // FileSystem Total Used Available Use% MountedOn > - private static final Pattern DF_PATTERN = Pattern.compile("([^\\s]+)\\s+(\\d+)\\s+\\d+\\s+(\\d+)\\s+\\d+%\\s+([^\\s].*)\n"); > + private static final Pattern DF_PATTERN = Pattern.compile("([^\\s]+)\\s+(\\d+)\\s+\\d+\\s+(\\d+)\\s+\\d+%\\s+([^\\s].*)$", Pattern.MULTILINE); > > private static int fail = 0; > private static int pass = 0; > -------------------------------------------------------------------------- > > Now the test passes for me and in our infrastructure. I did not completely understand why the old pattern did loose the first character in the line - but this seems to fix the issue. > > Before my change: > ----------System.out:(24/869)---------- > --- Testing df > C:/cygwin64 998257472 664791328 333466144 67% / > K: 262144000 129993392 132150608 50% /cygdrive/k > O: 734003200 641218112 92785088 88% /cygdrive/o > > > SecurityManager = null > C:/cygwin64: > df total= 1022215651328 free = 0 usable = 341469331456 > getX total= 1022215651328 free = 341469323264 usable = 341469323264 > :: > df total= 268435456000 free = 0 usable = 135322222592 > getX total= 0 free = 0 usable = 0 > FAILED > > > After my change: > --- Testing df > C:/cygwin64 998257472 665305100 332952372 67% / > K: 262144000 129993392 132150608 50% /cygdrive/k > O: 734003200 641211140 92792060 88% /cygdrive/o > > > SecurityManager = null > C:/cygwin64: > df total= 1022215651328 free = 0 usable = 340943228928 > getX total= 1022215651328 free = 340943286272 usable = 340943286272 > K:: > df total= 268435456000 free = 0 usable = 135322222592 > getX total= 268435456000 free = 135322222592 usable = 135322222592 > O:: > df total= 751619276800 free = 0 usable = 95019069440 > getX total= 751619276800 free = 95019069440 usable = 95019069440 > ... > ------------------------------------------ > > > The drives K: and O: are mapped network drives. > Do you see something similar in your landscape? Can you perhaps try to reproduce this? > > Best regards, > Arno > >> -----Original Message----- >> From: core-libs-dev On Behalf Of Igor >> Ignatyev >> Sent: Freitag, 31. Juli 2020 04:45 >> To: Brian Burkhalter >> Cc: core-libs-dev >> Subject: Re: [15] RFR(S) : 6501010 : test/java/io/File/GetXSpace.java fails on >> Windows >> >> Hi Brian, >> >> thanks for your review. I've decided to push into jdk/jdk instead of jdk15. >> please let me know if you think this patch needs to be in jdk15, and I'll backport >> it there. >> >> Cheers, >> -- Igor >> >>> On Jul 30, 2020, at 2:39 PM, Brian Burkhalter >> wrote: >>> >>> Hi Igor, >>> >>> This test looks all right to me. >>> >>> Sorry for the late review. >>> >>> Thanks, >>> >>> Brian >>> >>>> On Jul 20, 2020, at 11:28 AM, Igor Ignatyev > > wrote: >>>> >>>> http://cr.openjdk.java.net/~iignatyev//6501010/webrev.00 >> >>>>> 98 lines changed: 18 ins; 31 del; 49 mod; >>>> >>>> Hi all, >>>> >>>> could you please review this small fix to make java/io/File/GetXSpace.java >> work again on windows? >>>> >>>> the test has been updated to work under cygwin, which includes the >> following changes: >>>> - GetXSpace.sh accepts CYGWIN_* as known OS and sets windows path to >> TMP var >>>> - uses the same regexp to parse df on windows as on other platforms >>>> - but uses 'FileSystem' field as Space's name, as opposed to 'MountedOn' on >> *nix, as on cygwin, MountedOn is cygwin path, while FileSystem is >> corresponding windows path, and the test expect Space's name to be >> appropriate path >>> > From yumin.qi at oracle.com Tue Aug 11 17:36:27 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 11 Aug 2020 10:36:27 -0700 Subject: Fwd: RFR: 8247536: Support pre-generated MethodHandle lambda forms in CDS In-Reply-To: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> Message-ID: <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> Forget to send to @core-lib-dev, the patch changed jdk code. Thanks Yumin -------- Forwarded Message -------- Subject: RFR: 8247536: Support pre-generated MethodHandle lambda forms in CDS Date: Tue, 11 Aug 2020 07:44:34 -0700 From: Yumin Qi To: hotspot-runtime-dev at openjdk.java.net Hi, Please reivew ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ ? Summary: CDS does not archive pre-generated lambda form classes for method handles: [0.142s][info][class,load] java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: __JVM_LookupDefineClass__ The method handle resolution if not found in the holder class, a class with the method will be generated and loaded as vm internal created class and not archived like above. Those class names are mangled as combination of the class name and the class instance address. In this patch, collect those method holder class names and their associated methods' signatures when user specify DumpLoadedClassList, and record them in the log file in a format similar to the output format from traced by -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use another name for CDS: -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. The line prefix also changed from "[LF_RESOLVE]" to "@lambda-invoke-handle".? At dump time, regenerate the holder class with those methods and replace the existing holder class and archived it. At runtime, the resolution of the holder class which contains those methods are loaded from CDS archive so avoid regenerating them again. The patch reorganized the code for Jlink plugin, so the new added InvokerBytecodeGeneratorHelper can be shared both by JLink plugin and CDS code. Also change made to the mangled generated class name at static dump, the class name combining the class name and a sequentially ordered number instead of the class instance address, which looks like a random. To use this feature, one can do the dump/run just like done for a static dump/run, so no extra steps required. ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp ????? DumpLoadedClassList will turn on -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist -XX:SharedArchiveFile=my.jsa JavaApp ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp ? The performance on javac HelloWorld.java (javac-benchmark), no patch vs patch: ?? 1:?? 2689285002? 2641821474 (-47463528)????? ----???? 391.720 382.990 ( -8.730)????? ---- ?? 2:?? 2687495085? 2632969688 (-54525397)????? ----???? 391.030 381.480 ( -9.550)????? ----- ?? 3:?? 2694664066? 2636523114 (-58140952)????? -----??? 390.610 382.550 ( -8.060)????? ---- ?? 4:?? 2686554164? 2639355233 (-47198931)????? ----???? 390.700 383.390 ( -7.310)????? --- ?? 5:?? 2691072338? 2633016687 (-58055651)????? -----??? 388.990 382.360 ( -6.630)????? --- ?? 6:?? 2684448174? 2644191854 (-40256320)????? ---????? 389.450 382.990 ( -6.460)????? --- ?? 7:?? 2694921227? 2630505090 (-64416137)????? -----??? 389.300 383.160 ( -6.140)????? --- ?? 8:?? 2685209712? 2639334320 (-45875392)????? ----???? 388.370 381.060 ( -7.310)????? --- ?? 9:?? 2695885942? 2640618655 (-55267287)????? ----???? 389.560 381.100 ( -8.460)????? ---- ? 10:?? 2689162942? 2635658943 (-53503999)????? ----???? 389.690 379.110 (-10.580)????? ----- ============================================================ ??????? 2689866989? 2637396210 (-52470778)????? ----???? 389.941 382.017 ( -7.924)????? ---- instr delta =??? -52470778??? -1.9507% time? delta =?????? -7.924 ms -2.0321% ? It will show much more improvement if compare with the default archive: ?????????? ? ? ? no patch??????????????????????? patch instr ??? ? ? 3,996,847,605??????? 3,320,928,995 time(s).???? 0.686835162 ? ? ? ?? 0.474933546 ? Tests: 1)jtreg on jdk/tools and hotspot/runtime local. 2) mach5 ter1,2 Thanks Yumin ? ` From cjashfor at linux.ibm.com Mon Aug 10 21:00:57 2020 From: cjashfor at linux.ibm.com (Corey Ashford) Date: Mon, 10 Aug 2020 14:00:57 -0700 Subject: Fwd: RFR(S): 8248188 Add HotSpotIntrinsicCandidate and API for Base64 decoding (repost to correct mailing list) In-Reply-To: <407b7b6e-25de-b25d-94e4-9962c19e78ec@linux.ibm.com> References: <407b7b6e-25de-b25d-94e4-9962c19e78ec@linux.ibm.com> Message-ID: <06a915ca-5ae2-f350-1ef6-59e467b24084@linux.ibm.com> Hello, I had originally made two posts to the hotspot-compiler-dev and ppc-aix-port-dev mailing lists, but since this RFR requires modifying the core library code, it makes sense to post it to core-lib-devs instead. The two original posts: https://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2020-June/004110.html https://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2020-July/004115.html Webrev: http://cr.openjdk.java.net/~mhorie/8248188/webrev.01/ To recap the current state: Currently in java.util.Base64, there is a HotSpotIntrinsicCandidate and API for encodeBlock, but none for decoding. This means that only encoding gets acceleration from the underlying CPU's vector or other accelerator hardware. I'd like to propose adding a new intrinsic for decodeBlock. The considerations I have for this new intrinsic's API: * Don't make any assumptions about the underlying capability of the hardware. For example, do not impose any specific block size granularity. * Because of the difficulty of handling the isMIME=true option in a vector-based accelerator, do not call this intrinsic when isMIME=true. If desired, a new HotSpotIntrinsicCandidate can be introduced in the future for this case. * Any remaining data that is not processed by the intrinsic will be processed by the pure Java implementation. This allows the intrinsic to process whatever block sizes it's good at without the complexity of handling the end fragments. * If any illegal character is discovered in the decoding process, the intrinsic can return 0, or any number of bytes produced, up to but not including the byte containing the error. At this point, the pure-Java algorithm will be called, which will discover the error and report it via a thrown exception as usual. I've tested the patch in the webrev with an actual intrinsic coded up for Power9/Power10, but that runtime intrinsic and arch-specific patches aren't in the webrev, because I want to get some consensus on the library-level intrinsic API first. I'm open to any comments about this. Thanks for your consideration, - Corey Corey Ashford Software Engineer IBM Systems, LTC OpenJDK team 503-985-8699 Home office cjashfor at us.ibm.com IBM From mandy.chung at oracle.com Tue Aug 11 18:00:40 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 11 Aug 2020 11:00:40 -0700 Subject: Review Request JDK-8244090: public lookup should find public members of public exported types In-Reply-To: <090cff25-2f48-d55f-721d-38acda80029b@oracle.com> References: <090cff25-2f48-d55f-721d-38acda80029b@oracle.com> Message-ID: <45df07ad-dd1f-6f20-8cfd-8152f7ce111b@oracle.com> It was agreed in an offline discussion with Lois, Kim and John that this work should wait for the integration of JDK-8247938 and JEP 347 Enable C++ 14 features. This patch is updated to use scoped enum consistent with JDK-8247938. Updated webrev: http://cr.openjdk.java.net/~mchung/jdk16/webrevs/8244090/webrev.01/index.html Thanks Mandy On 6/18/20 2:12 PM, Mandy Chung wrote: > Prior to JDK-8173978, publicLookup().in(C.class) produces a Lookup > object on C with PUBLIC access which can be used to look up > unconditionally exported public types from the module of C. Such > lookup can only look up this C class defined by loader 1 but not > another class named "C" defined by loader 2. > > JDK-8173978 adds the support for cross-module teleporting. > publicLookup().in(C.class) was changed to produce a public Lookup i.e. > with UNCONDITIONAL access.? A public lookup should be able to look up > public members of any unconditionally exported public types including > a class named "C" loaded by different loaders.? It reveals a bug in VM > resolution for Lookup API that adds the loader constraints with > java.lang.Object as the accessor that constraints a public lookup to > load one type named C but any more. > > The lookup class of a public lookup is irrelevant to the lookup > context.? Type consistency on P/Q/R live Class objects from the given > MethodType (MT)is ensured at the following: > 1. P/Q/R are consistent from the loader of the declaring class of the > resolved member (D's loader) > 2. P/Q/R are consistent w.r.t. the invoking class and the caller's MT > at invocation time (the method handle carries the caller's MT that > will be verified). > > The loader constraints added for public lookup is not necessary and > adds undesirable constraints.? The proposed fixis to skip adding > loader constraints if the lookup mode is either TRUSTED or UNCONDITIONAL. > > Webrev: > http://cr.openjdk.java.net/~mchung/jdk16/webrevs/8244090/webrev.00/index.html > > > Thanks > Mandy From patrick.concannon at oracle.com Tue Aug 11 18:11:56 2020 From: patrick.concannon at oracle.com (Patrick Concannon) Date: Tue, 11 Aug 2020 19:11:56 +0100 Subject: =?utf-8?Q?Re=3A_RFR=5B8238286=5D=3A_=27Add_new_flatMap_stream_ope?= =?utf-8?Q?ration_that_is_more_amenable_to_pushing=E2=80=99?= In-Reply-To: References: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> <5B4DFB68-747C-4EAB-A042-C24DF6516FB8@oracle.com> <47c1e40f-86ac-415d-5c43-674b47d8884d@anthonyv.be> Message-ID: Hi, Please find the next iteration of mapMulti in the new webrev below. In this iteration the following changes have been made: The API note for mapMulti has been updated. Sub-interfaces for {Int, Double, Long}Stream have been refactored to be more specific to mapMulti. The examples have been changed, and now include a reference to how an explicit type parameter can be used in conjunction with mapMulti. The CSR and specdiff have also been updated to reflect these changes. webrev: http://cr.openjdk.java.net/~pconcannon/8238286/webrevs/webrev.05/ specdiff: http://cr.openjdk.java.net/~pconcannon/8238286/specdiff/specout.02/ CSR: https://bugs.openjdk.java.net/browse/JDK-8248166 Kind regards, Patrick From brian.burkhalter at oracle.com Tue Aug 11 18:32:53 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 11 Aug 2020 11:32:53 -0700 Subject: 8181919: Refactor test/java/io/File/GetXSpace.sh to java test In-Reply-To: <711DD0AF-CAB7-4DE2-9495-3D27B1F3A0A3@oracle.com> References: <711DD0AF-CAB7-4DE2-9495-3D27B1F3A0A3@oracle.com> Message-ID: A revised version [A] is available pursuant to the changes made in the course of the review [B] of the fix for [C]. Thanks, Brian [A] http://cr.openjdk.java.net/~bpb/8181919/webrev.01/ [B] http://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068094.html [C] https://bugs.openjdk.java.net/browse/JDK-8249703 > On Aug 5, 2020, at 11:36 AM, Brian Burkhalter wrote: > > To fix [1], please consider the patch [2]. This follows on from the proposed change [3] which in turn follows from [4]. > > Thanks, > > Brian > > [1] https://bugs.openjdk.java.net/browse/JDK-8181919 > [2] http://cr.openjdk.java.net/~bpb/8181919/webrev.00/ > [3] https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068025.html (GetXSpace should not run on AIX) > [4] http://mail.openjdk.java.net/pipermail/core-libs-dev/2020-July/067990.html (GetXSpace fails on macOS) From cjashfor at linux.ibm.com Tue Aug 11 18:52:50 2020 From: cjashfor at linux.ibm.com (Corey Ashford) Date: Tue, 11 Aug 2020 11:52:50 -0700 Subject: Fwd: RFR(S): 8248188 Add HotSpotIntrinsicCandidate and API for Base64 decoding (repost to correct mailing list) In-Reply-To: <06a915ca-5ae2-f350-1ef6-59e467b24084@linux.ibm.com> References: <407b7b6e-25de-b25d-94e4-9962c19e78ec@linux.ibm.com> <06a915ca-5ae2-f350-1ef6-59e467b24084@linux.ibm.com> Message-ID: <7d2aba21-3032-d8e9-b05f-a83fc5cdc9ea@linux.ibm.com> This is an accidental duplicate post. Please disregard. On 8/10/20 2:00 PM, Corey Ashford wrote: > Hello, > > I had originally made two posts to the hotspot-compiler-dev and > ppc-aix-port-dev mailing lists, but since this RFR requires modifying > the core library code, it makes sense to post it to core-lib-devs instead. > > The two original posts: > > https://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2020-June/004110.html > > > https://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2020-July/004115.html > > > Webrev: http://cr.openjdk.java.net/~mhorie/8248188/webrev.01/ > > To recap the current state: > > Currently in java.util.Base64, there is a HotSpotIntrinsicCandidate and > API for encodeBlock, but none for decoding.? This means that only > encoding gets acceleration from the underlying CPU's vector or other > accelerator hardware. > > I'd like to propose adding a new intrinsic for decodeBlock.? The > considerations I have for this new intrinsic's API: > > ?* Don't make any assumptions about the underlying capability of the > hardware.? For example, do not impose any specific block size granularity. > > ?* Because of the difficulty of handling the isMIME=true option in a > vector-based accelerator, do not call this intrinsic when isMIME=true. > If desired, a new HotSpotIntrinsicCandidate can be introduced in the > future for this case. > > ?* Any remaining data that is not processed by the intrinsic will be > processed by the pure Java implementation.? This allows the intrinsic to > process whatever block sizes it's good at without the complexity of > handling the end fragments. > > ?* If any illegal character is discovered in the decoding process, the > intrinsic can return 0, or any number of bytes produced, up to but not > including the byte containing the error.? At this point, the pure-Java > algorithm will be called, which will discover the error and report it > via a thrown exception as usual. > > I've tested the patch in the webrev with an actual intrinsic coded up > for Power9/Power10, but that runtime intrinsic and arch-specific patches > aren't in the webrev, because I want to get some consensus on the > library-level intrinsic API first. > > I'm open to any comments about this. > > Thanks for your consideration, > > - Corey > > Corey Ashford > Software Engineer > IBM Systems, LTC OpenJDK team > 503-985-8699 Home office > cjashfor at us.ibm.com > > IBM From ioi.lam at oracle.com Tue Aug 11 22:09:12 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Tue, 11 Aug 2020 15:09:12 -0700 Subject: Fwd: RFR: 8247536: Support pre-generated MethodHandle lambda forms in CDS In-Reply-To: <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> Message-ID: <3f59d2fe-14ab-7b45-5fea-85128354ce09@oracle.com> Hi Yumin, This looks good! Here are my comments: [1] classListParser.cpp: #define LAMBDA_FOMRM_INVOKER "@lambda-form-invoker" I think this should be moved to classListParser.hpp: ??? const char* lambda_form_invoker_tag() { ??????? return "@lambda-form-invoker"; ??? } Also, instead of also duplicating this string in InvokerBytecodeGenerator.java, the Java side of the code can be changed to ? ?? String outLine = holder.getName() + " " + name + " " + .... and JVM_CDSTraceResolve() can be changed to ??? classlist_file->print_cr("%s %s", ClassListParser::lambda_form_invoker_tag(), c_line); [2] metaspaceShared.cpp: This file is getting too big. I think we should move the new functionality to a new file ?? share/memory/lambdaFormInvokers.cpp [3] MetaspaceShared::preload_classes() 2080 if (parser.is_lambda_format()) This name is too generic. How about "parser.is_lambda_form_invoker()"? 2109?????? if (lambda_list == NULL) { 2110???????? lambda_list = new GrowableArray(8); 2111?????? } 2112?????? lambda_list->append(parser.current_line()); These can be changed to LambdaFormInvokers::append(parser.current_line()); [4] I think the follow code that changes the system dictionary is kind of scary. Maybe we can avoid it. 2060?? // replace with the new created klass. 2061?? { 2062???? MutexLocker lock(THREAD, SystemDictionary_lock); 2063???? InstanceKlass* old = cld->replace_class(sym, result); 2064???? SystemDictionaryShared::set_excluded(old); 2065???? log_info(cds)("Replace class %s, old: %p? new: %p", class_name, old, result); 2066?? } How about just do this: ??? InstanceKlass* old = cld->find_class(sym); ??? SystemDictionaryShared::set_excluded(old); This will prevent the old class from being added to CollectClassesClosure. We actually don't need to add the new class into cld->dictionary() -- we don't want to "use" the new class in any way other than writing it into the archive. We can do this to add the new class to the CollectClassesClosure: + GrowableArray* LambdaFormInvokers::replacement_holder_classes(); ? ... ? _global_klass_objects = new GrowableArray(1000); ? CollectClassesClosure collect_classes; ? ClassLoaderDataGraph::loaded_classes_do(&collect_classes); + collect_classes.add_extra_classes(LambdaFormInvokers::replacement_holder_classes()); ? _global_klass_objects->sort(global_klass_compare); [5] InvokerGenerateBytesException.java This class cannot be public, as it's not part of the java.lang.invoke package specification. Anyway, we don't seem to actually need this type, as it's not used outside of InvokerBytecodeGeneratorHelper. How about replacing all use of ??? throw new InvokerGenerateBytesException(...); to ??? throw new RuntimeException(...); [6] This adds duplicated code for processing the option: 2846???? // -XX:DumpLoadedClassList 2847???? } else if (match_option(option, "-XX:DumpLoadedClassList=", &tail)) { 2848?????? if (tail != NULL) { 2849 add_property("java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true"); 2850???????? DumpLoadedClassList = os::strdup_check_oom(tail); 2851?????? } else { 2852???????? warning("Bad option for -XX:DumpLoadedClassList="); 2853???????? return JNI_EINVAL; 2854?????? } I think it's better to move the code to here: jint Arguments::parse(const JavaVMInitArgs* initial_cmd_args) { .... - #if !INCLUDE_CDS + #if INCLUDE_CDS +?? if (DumpLoadedClassList != NULL) { +????? if (!add_property("java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true")) { +??????? return JNI_ENOMEM; +????? } +?? } + #else [7] For testing, we should have different kinds of invalid input for the @lambda-form-invoker to make sure that the errors are handled properly. [8] Some variable names should be changed to be more descriptive. Here is an example: // k - the class full name // v - the class bytes void MetaspaceShared::reload_class(Handle k, Handle v, TRAPS) { should be changed class_name, class_bytes. Thanks - Ioi On 8/11/20 10:36 AM, Yumin Qi wrote: > > Hi, Please reivew > > ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 > ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ > > ? Summary: CDS does not archive pre-generated lambda form classes for > method handles: > > [0.142s][info][class,load] > java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: > __JVM_LookupDefineClass__ > > The method handle resolution if not found in the holder class, a class > with the method will be generated and loaded as vm internal created > class and not archived like above. Those class names are mangled as > combination of the class name and the class instance address. > > In this patch, collect those method holder class names and their > associated methods' signatures when user specify DumpLoadedClassList, > and record them in the log file in a format similar to the output > format from traced by > -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use > another name for CDS: > -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. The > line prefix also changed from "[LF_RESOLVE]" to > "@lambda-invoke-handle".? At dump time, regenerate the holder class > with those methods and replace the existing holder class and archived > it. At runtime, the resolution of the holder class which contains > those methods are loaded from CDS archive so avoid regenerating them > again. The patch reorganized the code for Jlink plugin, so the new > added InvokerBytecodeGeneratorHelper can be shared both by JLink > plugin and CDS code. Also change made to the mangled generated class > name at static dump, the class name combining the class name and a > sequentially ordered number instead of the class instance address, > which looks like a random. > > > To use this feature, one can do the dump/run just like done for a > static dump/run, so no extra steps required. > > ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp > > ????? DumpLoadedClassList will turn on > -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true > > ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist > -XX:SharedArchiveFile=my.jsa JavaApp > > ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp > > ? The performance on javac HelloWorld.java (javac-benchmark), no patch > vs patch: > > ?? 1:?? 2689285002? 2641821474 (-47463528)????? ---- 391.720 382.990 ( > -8.730)????? ---- > ?? 2:?? 2687495085? 2632969688 (-54525397)????? ---- 391.030 381.480 ( > -9.550)????? ----- > ?? 3:?? 2694664066? 2636523114 (-58140952)????? ----- 390.610 382.550 > ( -8.060)????? ---- > ?? 4:?? 2686554164? 2639355233 (-47198931)????? ---- 390.700 383.390 ( > -7.310)????? --- > ?? 5:?? 2691072338? 2633016687 (-58055651)????? ----- 388.990 382.360 > ( -6.630)????? --- > ?? 6:?? 2684448174? 2644191854 (-40256320)????? --- 389.450 382.990 ( > -6.460)????? --- > ?? 7:?? 2694921227? 2630505090 (-64416137)????? ----- 389.300 383.160 > ( -6.140)????? --- > ?? 8:?? 2685209712? 2639334320 (-45875392)????? ---- 388.370 381.060 ( > -7.310)????? --- > ?? 9:?? 2695885942? 2640618655 (-55267287)????? ---- 389.560 381.100 ( > -8.460)????? ---- > ? 10:?? 2689162942? 2635658943 (-53503999)????? ---- 389.690 379.110 > (-10.580)????? ----- > ============================================================ > ??????? 2689866989? 2637396210 (-52470778)????? ---- 389.941 382.017 ( > -7.924)????? ---- > > instr delta =??? -52470778??? -1.9507% > > time? delta =?????? -7.924 ms -2.0321% > > ? It will show much more improvement if compare with the default archive: > > ?????????? ? ? ? no patch??????????????????????? patch > > instr ??? ? ? 3,996,847,605??????? 3,320,928,995 > > time(s).???? 0.686835162 ? ? ? ?? 0.474933546 > > > ? Tests: > > 1)jtreg on jdk/tools and hotspot/runtime local. > > 2) mach5 ter1,2 > > > Thanks > > Yumin > > ? ` From joe.darcy at oracle.com Tue Aug 11 23:03:49 2020 From: joe.darcy at oracle.com (Joe Darcy) Date: Tue, 11 Aug 2020 16:03:49 -0700 Subject: 8245304: Re-examine ThreadLocal usage in java.math.BigDecimal In-Reply-To: References: Message-ID: <03fa592e-9d6c-dd68-1b10-994e6788888e@oracle.com> Looks fine; thanks, -Joe On 8/11/2020 10:32 AM, Brian Burkhalter wrote: > To fix [1], please consider [2] which replaces the ThreadLocal ?threadLocalStringBuilderHelper? with straightforward use of a StringBuilderHelper instance. The initial capacity of the StringBuilder instance variable of StringBuilderHelper is also doubled from 16 to 32. > > Microbenchmarking was effected primarily using BigDecimal.toEngineeringString(). This method and toString() both invoke layoutChars() which used the ThreadLocal. The toString() method caches its return value but toEngineeringString() does not. > > Benchmarking using a single Thread did not show any regression for toEngineeringString() and in fact showed a slight improvement. Since toEngineeringString() does not cache its result, one would expect the benchmarks for toString() not to deteriorate either, and simple benchmarking validated this. > > Given the absence of any measurable performance regression it appears that this ThreadLocal may safely be removed. > > Thanks, > > Brian > > [1] https://bugs.openjdk.java.net/browse/JDK-8245304 > [2] http://cr.openjdk.java.net/~bpb/8245304/webrev.00/ From Alan.Bateman at oracle.com Wed Aug 12 06:34:20 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 12 Aug 2020 07:34:20 +0100 Subject: 8245304: Re-examine ThreadLocal usage in java.math.BigDecimal In-Reply-To: References: Message-ID: <2bd73a42-6383-53b0-a0ec-ec8582654ee9@oracle.com> On 11/08/2020 18:32, Brian Burkhalter wrote: > To fix [1], please consider [2] which replaces the ThreadLocal ?threadLocalStringBuilderHelper? with straightforward use of a StringBuilderHelper instance. The initial capacity of the StringBuilder instance variable of StringBuilderHelper is also doubled from 16 to 32. > > Microbenchmarking was effected primarily using BigDecimal.toEngineeringString(). This method and toString() both invoke layoutChars() which used the ThreadLocal. The toString() method caches its return value but toEngineeringString() does not. > > Looks good. -Alan From adinn at redhat.com Wed Aug 12 09:08:54 2020 From: adinn at redhat.com (Andrew Dinn) Date: Wed, 12 Aug 2020 10:08:54 +0100 Subject: Possible subtle memory model error in ClassValue In-Reply-To: References: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> <6f91ae57-8dae-be23-bab4-c47dbe8b95f7@redhat.com> <0c849c2a-0a3a-704a-3ee3-22b690aa0d34@redhat.com> <04069f52-c9ef-8fee-26c7-5fdb69e8860b@redhat.com> Message-ID: <22fc68d5-d501-9ce0-093c-7be82aed247a@redhat.com> On 11/08/2020 18:06, Hans Boehm wrote: > I think the relevant statement is: > > "An address dependency between two reads generated by SVE vector load > instructions does not contribute to the Dependency-ordered-before relation." > > This is only an issue if BOTH loads are SVE loads. In particular > reference loads have to be vectorized for this to matter, which I expect > is not the common situation. I have not explored this in great detail, > but it should suffice to put fences between two dependent vector > reference loads, and between a reference load and a dependent final > field load. Hmm, so this might perhaps be an issue with something like a deep copy of an int[][], where loading of successive int[] references might be vectorized using SVE instructions and processing of the contents of each referenced int[] might also be similarly vectorized. In that case the loading of the int contents would need to be ordered wrt the load of the int[] references using a LoadLoad barrier? regards, Andrew Dinn ----------- Red Hat Distinguished Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill From aleksej.efimov at oracle.com Wed Aug 12 11:36:22 2020 From: aleksej.efimov at oracle.com (Aleks Efimov) Date: Wed, 12 Aug 2020 12:36:22 +0100 Subject: RFR[testbug]: 8251189: com/sun/jndi/ldap/LdapDnsProviderTest.java failed due to timeout In-Reply-To: References: <8111b911-4399-4e78-8701-26a4d455e51c@oracle.com> Message-ID: <7f7da715-e83e-df41-ef75-e1eb8adbb7b1@oracle.com> Daniel, Vyom, Thanks for your reviews Best Regards, Aleksei On 11/08/2020 14:12, Vyom Tiwari wrote: > +1 > Vyom > > On Tue, Aug 11, 2020 at 4:39 PM Daniel Fuchs > wrote: > > Hi Aleksei, > > Looks good to me! > > best regards, > > -- daniel > > On 06/08/2020 13:44, Aleks Efimov wrote: > > Hi, > > > > LdapDnsProviderTest was seen failing due to the timeout caused > by the > > blocked bind operation. That could happen if there is a local > process > > listening on the port specified in the test URL. > > To make LdapProviderTest.ProviderTest.call fail the LDAP connect > timeout > > property needs to be set to greater than 0 value. That will help > > runLocalHostTestWithRandomPort to fail with the timeout > exception if the > > port is busy and try another random port. > > Also, the test output has been modified not to print stack > traces for > > the expected exceptions. > > > > Webrev: http://cr.openjdk.java.net/~aefimov/8251189/00/index.html > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251189 > > > > The test was executed 100+ times alongside to other LDAP tests > and was > > not seen failing with the > >? ?proposed changes. > > > > With Best Regards, > > Aleksei > > > > > > -- > Thanks, > Vyom From calvin.cheung at oracle.com Wed Aug 12 18:54:27 2020 From: calvin.cheung at oracle.com (calvin.cheung at oracle.com) Date: Wed, 12 Aug 2020 11:54:27 -0700 Subject: Fwd: RFR: 8247536: Support pre-generated MethodHandle lambda forms in CDS In-Reply-To: <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> Message-ID: <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> Hi Yumin, I reviewed mostly the native code. Below are my comments: 1) classListParser.hpp 71?? bool??????????????? _lambda_format; The above name is too generic. How about _lambda_form or _is_lambda_form? If you rename the above, please also rename the function which returns the above bool. 2) jvm.cpp 3850 JVM_ENTRY(void, JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line)) ignore -> ignored 3) jvm.hpp 210 JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line); Same comment as for jvm.cpp 4) metaspaceShared.cpp 2017?? size_t i = 0; 2018?? while (i < size) { 2019???? full_name[i++] = *start++; 2020?? } Could the above be simplified to the following? ??? strncpy(full_name, start, size - 1); 2029?? char* class_name = get_full_class_name(path_name); Should os::free(class_name) be called before the function returns? 1870 static GrowableArray* lambda_list = NULL; The name lambda_list is a bit generic, how about lambda_form_list? 2112?????? lambda_list->append(parser.current_line()); Since parser.current_line() does a strdup, do those buffer need to be freed after its use? (i.e. after the call to regenerate_holder_classes()?) In MetaspaceShared::regenerate_holder_classes, before calling up to java, I think it's better to strip the prefix "@lambda-form-invoker" from the strings. So that the java InvokerBytecodeGeneratorHelper.readTraceConfig method doesn't need to handle it: ?143???????????????????? case "[LF_RESOLVE]": ?144???????????????????? case InvokerBytecodeGenerator.CDS_LOG_PREFIX: 5) DumpSymbolAndStringTable.java 37???? private static final String my_string = "DumpSymbolAndStringTable"; Unused variable? thanks, Calvin On 8/11/20 10:36 AM, Yumin Qi wrote: > Forget to send to @core-lib-dev, the patch changed jdk code. > > > Thanks > > Yumin > > > > -------- Forwarded Message -------- > Subject:???? RFR: 8247536: Support pre-generated MethodHandle lambda > forms in CDS > Date:???? Tue, 11 Aug 2020 07:44:34 -0700 > From:???? Yumin Qi > To:???? hotspot-runtime-dev at openjdk.java.net > > > > Hi, Please reivew > > ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 > ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ > > ? Summary: CDS does not archive pre-generated lambda form classes for > method handles: > > [0.142s][info][class,load] > java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: > __JVM_LookupDefineClass__ > > The method handle resolution if not found in the holder class, a class > with the method will be generated and loaded as vm internal created > class and not archived like above. Those class names are mangled as > combination of the class name and the class instance address. > > In this patch, collect those method holder class names and their > associated methods' signatures when user specify DumpLoadedClassList, > and record them in the log file in a format similar to the output > format from traced by > -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use > another name for CDS: > -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. The > line prefix also changed from "[LF_RESOLVE]" to > "@lambda-invoke-handle".? At dump time, regenerate the holder class > with those methods and replace the existing holder class and archived > it. At runtime, the resolution of the holder class which contains > those methods are loaded from CDS archive so avoid regenerating them > again. The patch reorganized the code for Jlink plugin, so the new > added InvokerBytecodeGeneratorHelper can be shared both by JLink > plugin and CDS code. Also change made to the mangled generated class > name at static dump, the class name combining the class name and a > sequentially ordered number instead of the class instance address, > which looks like a random. > > > To use this feature, one can do the dump/run just like done for a > static dump/run, so no extra steps required. > > ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp > > ????? DumpLoadedClassList will turn on > -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true > > ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist > -XX:SharedArchiveFile=my.jsa JavaApp > > ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp > > ? The performance on javac HelloWorld.java (javac-benchmark), no patch > vs patch: > > ?? 1:?? 2689285002? 2641821474 (-47463528)????? ----???? 391.720 > 382.990 ( -8.730)????? ---- > ?? 2:?? 2687495085? 2632969688 (-54525397)????? ----???? 391.030 > 381.480 ( -9.550)????? ----- > ?? 3:?? 2694664066? 2636523114 (-58140952)????? -----??? 390.610 > 382.550 ( -8.060)????? ---- > ?? 4:?? 2686554164? 2639355233 (-47198931)????? ----???? 390.700 > 383.390 ( -7.310)????? --- > ?? 5:?? 2691072338? 2633016687 (-58055651)????? -----??? 388.990 > 382.360 ( -6.630)????? --- > ?? 6:?? 2684448174? 2644191854 (-40256320)????? ---????? 389.450 > 382.990 ( -6.460)????? --- > ?? 7:?? 2694921227? 2630505090 (-64416137)????? -----??? 389.300 > 383.160 ( -6.140)????? --- > ?? 8:?? 2685209712? 2639334320 (-45875392)????? ----???? 388.370 > 381.060 ( -7.310)????? --- > ?? 9:?? 2695885942? 2640618655 (-55267287)????? ----???? 389.560 > 381.100 ( -8.460)????? ---- > ? 10:?? 2689162942? 2635658943 (-53503999)????? ----???? 389.690 > 379.110 (-10.580)????? ----- > ============================================================ > ??????? 2689866989? 2637396210 (-52470778)????? ----???? 389.941 > 382.017 ( -7.924)????? ---- > > instr delta =??? -52470778??? -1.9507% > > time? delta =?????? -7.924 ms -2.0321% > > ? It will show much more improvement if compare with the default archive: > > ?????????? ? ? ? no patch??????????????????????? patch > > instr ??? ? ? 3,996,847,605??????? 3,320,928,995 > > time(s).???? 0.686835162 ? ? ? ?? 0.474933546 > > > ? Tests: > > 1)jtreg on jdk/tools and hotspot/runtime local. > > 2) mach5 ter1,2 > > > Thanks > > Yumin > > ? ` From lois.foltan at oracle.com Wed Aug 12 21:13:26 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Wed, 12 Aug 2020 17:13:26 -0400 Subject: Review Request JDK-8244090: public lookup should find public members of public exported types In-Reply-To: <45df07ad-dd1f-6f20-8cfd-8152f7ce111b@oracle.com> References: <090cff25-2f48-d55f-721d-38acda80029b@oracle.com> <45df07ad-dd1f-6f20-8cfd-8152f7ce111b@oracle.com> Message-ID: <42c1765b-e6a5-5ee0-c543-1030b4636ff4@oracle.com> On 8/11/2020 2:00 PM, Mandy Chung wrote: > It was agreed in an offline discussion with Lois, Kim and John that > this work should wait for the integration of JDK-8247938 and JEP 347 > Enable C++ 14 features. > > This patch is updated to use scoped enum consistent with JDK-8247938. > Updated webrev: > http://cr.openjdk.java.net/~mchung/jdk16/webrevs/8244090/webrev.01/index.html Hi Mandy, Still looks good! Thanks, Lois > > Thanks > Mandy > > On 6/18/20 2:12 PM, Mandy Chung wrote: >> Prior to JDK-8173978, publicLookup().in(C.class) produces a Lookup >> object on C with PUBLIC access which can be used to look up >> unconditionally exported public types from the module of C. Such >> lookup can only look up this C class defined by loader 1 but not >> another class named "C" defined by loader 2. >> >> JDK-8173978 adds the support for cross-module teleporting. >> publicLookup().in(C.class) was changed to produce a public Lookup >> i.e. with UNCONDITIONAL access.? A public lookup should be able to >> look up public members of any unconditionally exported public types >> including a class named "C" loaded by different loaders.? It reveals >> a bug in VM resolution for Lookup API that adds the loader >> constraints with java.lang.Object as the accessor that constraints a >> public lookup to load one type named C but any more. >> >> The lookup class of a public lookup is irrelevant to the lookup >> context.? Type consistency on P/Q/R live Class objects from the given >> MethodType (MT)is ensured at the following: >> 1. P/Q/R are consistent from the loader of the declaring class of the >> resolved member (D's loader) >> 2. P/Q/R are consistent w.r.t. the invoking class and the caller's MT >> at invocation time (the method handle carries the caller's MT that >> will be verified). >> >> The loader constraints added for public lookup is not necessary and >> adds undesirable constraints.? The proposed fixis to skip adding >> loader constraints if the lookup mode is either TRUSTED or >> UNCONDITIONAL. >> >> Webrev: >> http://cr.openjdk.java.net/~mchung/jdk16/webrevs/8244090/webrev.00/index.html >> >> >> Thanks >> Mandy > From forax at univ-mlv.fr Wed Aug 12 21:48:30 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 12 Aug 2020 23:48:30 +0200 (CEST) Subject: =?utf-8?Q?Re:_RFR[8238286]:_'Add_new_flatMap_stream_?= =?utf-8?Q?operation_that_is_more_amenable_to_pushing=E2=80=99?= In-Reply-To: References: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> <5B4DFB68-747C-4EAB-A042-C24DF6516FB8@oracle.com> <47c1e40f-86ac-415d-5c43-674b47d8884d@anthonyv.be> Message-ID: <1105399538.6154.1597268910050.JavaMail.zimbra@u-pem.fr> Hi Patrick, I still don't like the fact that IntMapMultiConsumer, DoubleMapMultiConsumer and LongMapMultiConsumer are not in java.util.function unlike all other functional interfaces used by the Stream API. Otherwise looks good. regards, R?mi ----- Mail original ----- > De: "Patrick Concannon" > ?: "Julia Boes" > Cc: "Anthony Vanelverdinghe" , "core-libs-dev" > Envoy?: Mardi 11 Ao?t 2020 20:11:56 > Objet: Re: RFR[8238286]: 'Add new flatMap stream operation that is more amenable to pushing? > Hi, > > Please find the next iteration of mapMulti in the new webrev below. > > In this iteration the following changes have been made: > The API note for mapMulti has been updated. > Sub-interfaces for {Int, Double, Long}Stream have been refactored to be more > specific to mapMulti. > The examples have been changed, and now include a reference to how an explicit > type parameter can be used in conjunction with mapMulti. > > The CSR and specdiff have also been updated to reflect these changes. > > webrev: http://cr.openjdk.java.net/~pconcannon/8238286/webrevs/webrev.05/ > specdiff: http://cr.openjdk.java.net/~pconcannon/8238286/specdiff/specout.02/ > CSR: https://bugs.openjdk.java.net/browse/JDK-8248166 > > Kind regards, > Patrick From ioi.lam at oracle.com Wed Aug 12 22:06:35 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Wed, 12 Aug 2020 15:06:35 -0700 Subject: RFR(L) 8244778 Archive full module graph in CDS In-Reply-To: <9e6b0043-65a1-dd97-a3d2-33679c8048d4@oracle.com> References: <9e6b0043-65a1-dd97-a3d2-33679c8048d4@oracle.com> Message-ID: Hi Lois, Thanks for the comments. I have an updated webrev http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/ http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02.delta/ Here are the general notes on the changes. Replies to your questions are in-line below. (1) Integrated updates in the Java code from Alan Bateman. Here are Alan's ? ? notes: ??? The archive of the boot layer is as before, except that archiving is ??? skipped if there are split packages or incubator modules. Incubating ??? modules aren't resolved by default so we shouldn't have them in the ??? boot layer by default anyway. ??? I've dropped the module finders from the boot layer archive. I've ??? left the IllegalAccessLogger.Builder in the acrhive for now (even ??? though it is not the boot layer). We should be able to remove that ??? once the JEP to disallow illegal access by default is in. ??? Related is that I don't like the archive for the module graph ??? (ArchivedModuleGraph, pre-dates this RFE) including the set of ??? packages to export/open for illegal access as they aren't part ??? of the module graph. I've left it for now but we can also remove ??? that once we disallow illegal access by default (as those sets ??? will be empty). ??? The archive of built-in class loaders is now in one object ??? jdk.internal.loader.ArchivedClassLoaders which I think will be a ??? bit more maintainable. I've also drop the ucp field from the ??? AppClassLoader as the changes to BuiltinClassLoader means is no ??? longer needs to duplicated. ??? There's one remaining issue in ModuleBootstrap.boot where it has fix ??? an app class loader value (ModuleLayer.CLV). Ideally the initialization ??? of the built-in class loaders would do this but we are kinda forced ??? to separate the archiving of the built-in class loaders from the boot ??? layer. I might look at this again some time. (2) Moved code from classLoaderData.cpp -> classLoaderSharedData.cpp (3) Reverted unnecessary changes in JavaClasses::compute_offset (4) Minor clean up to use QuickSort::sort() instead of qsort() (5) Moved the C-side of module initialization for platform/system ? ? loaders to inside java.lang.System::initPhase2(), so this happens ??? at the same time as without full module archiving. (6) Moved the use of Module_lock to so all modules in a class loader ??? are restored atomically. See ArchivedClassLoaderData::restore() ??? This fixed a bug where test/jdk/com/sun/jdi/ModulesTest.java ? ? would fail as it sees a partially restored set of modules. On 8/7/20 12:06 PM, Lois Foltan wrote: > Hi Ioi, > > Overall looks promising.? I have some review comments below, but not a > complete review.? I concentrated on the JVM side primarily how the > archived module graph is read in, how the ModuleEntry and PackageEntry > tables are created from the archive for the 3 builtin loaders and > potential timing issues during start up. > > On 7/22/2020 3:36 PM, Ioi Lam wrote: >> https://bugs.openjdk.java.net/browse/JDK-8244778 >> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/ >> >> >> Please review this patch that stores the full module graph in the CDS >> archive heap. This reduces the initialization time of the basic JVM by >> about 22%: >> >> $ perf stat -r 100 bin/java -version >> before: 98,219,329 instructions 0.03971 secs elapsed (+- 0.44%) >> after:? 55,835,815 instructions 0.03109 secs elapsed (+- 0.65%) >> >> [1] Start with ModuleBootstrap.java. The current implementation is >> ??? quite restrictive: the archived module graph is used only when no >> ??? module options are specified. >> >> ??? See ModuleBootstrap.mayUseArchivedBootLayer(). >> >> ??? We can probably support options such as main module and module path >> ??? in a future RFE. > > Yes, I noticed the restrictions.? The JBS issue discusses the > possibility of using the dumped module graph in the event that the > value of the options are the same.? I think for this implementation to > be viable and have a positive impact you should consider comparing at > least the options --add-modules, --add-exports and --add-reads. I agree. I want to keep the changes minimal in this RFE, and will add more support for other use cases in follow-on RFEs. Instead of requiring these options to be unspecified, we can relax the restriction such that these options must be the same between archive dump time and run time. > > >> >> [2] In the current JDK implementation, there is no single object >> ??? that represents "the module graph". Most of the information >> ??? is stored in the archive bootLayer object, but a few additional >> ??? restoration operations need to be performed: >> >> ??? + See ModuleBootstrap.getArchivedBootLayer() >> ??? + Some static fields need to be archived/restored in >> ????? Module.java, BuiltinClassLoader.java, ClassLoaders.java >> ????? and BootLoader.java >> >> [3] I ran into a complication with two loader instances of >> ??? PlatformClassLoader and AppClassLoader. They are stored in >> ??? multiple tables inside the module graph (e.g., >> ??? BuiltinClassLoader$LoadedModule) so I cannot easily recreate >> ??? them at runtime. >> >> ??? However, these two loaders contain information specific to the >> ??? dump time VM lifecycle (such as the classes that were loaded >> ??? during CDS dumping) that need to be scrubbed. I couldn't find an >> ??? elegant way of doing this, so I added a private >> "resetArchivedStates" >> ??? method to a few classes. They are called inside >> ??? HeapShared::reset_archived_object_states(). >> >> [4] Related native data structures (PackageEntry and ModuleEntry) >> ??? are also archived. Start with classLoaderData.cpp >> >> Passes mach5 tiers 1-4. I will test with additional tiers. >> >> Thanks >> - Ioi > > classfile/classLoader.cpp > - line #1644 pulling the javabase_moduleEntry() check outside of the > Module_lock maybe problematic if after you check it another > ? thread initializes in the time between the check and the obtaining > of the Module_lock on line #1645. Fixed. > > classfile/classLoader.hpp > - somewhere in ArchivedClassLoaderData there should be an assert to > make sure that the CLD, whose package entries and module entries are > being archived is not a "_has_class_mirror_holder" CLD for a weakly > defined hidden class.? Those dedicated CLDs should never have package > entries or module entries. > I added ArchivedClassLoaderData::assert_valid() > classfile/moduleEntry.cpp > - line #400, typo "conver" --> "convert" > - line #500, maybe sort if n is greater than 1 instead of 0 (same > comment for packageEntry.cpp line #270) > Fixed > classfile/systemDictionary.cpp > - It looks like the PackageEntry and ModuleEntry tables for the system > and platform class loaders are? added within > SystemDictionary::compute_java_loaders() which is called from > Thread::create_vm() but after the call in Thread::create_vm() to > call_initPhase2 which is when Universe::_module_initialized is set to > true.? Did I read this correctly?? If yes, then I think you have to be > sure that Universe::_module_initialized is not set until the module > graph for the 3 builtin loaders is loaded from the archive. > I moved the code to be called by ModuleBootstrap::boot() so it should now happen inside call_initPhase2. > memory/filemap.cpp > - line #237 typo "modue" --> "module" > Fixed > - Since the CDS support for capturing the module graph does not > archive unnamed modules, I assume > Modules::set_bootloader_unnamed_module() is still called.? Is this > true or does the unnamed module for the boot loader get archived? The unnamed module for the boot loader is not archived. Modules::set_bootloader_unnamed_module()? wasn't called in my last webrev. Thanks for catching it. I added a call to BootLoader.getUnnamedModule() in ModuleBootstrap::boot()? to trigger of BootLoader, which will call into the VM for Modules::set_bootloader_unnamed_module(). > Clarification/timing questions: > Here's an overall problem I am facing: The module graph is archived after the module system has fully started up. This means that for the boot loader, we only know the full set of modules/packages, but we don't know which part is the subset that was initialized during early VM bootstraping (e.g., when ModuleEntryTable::javabase_defined() == false). So the behavior is a bit different during the early bootstrap phase (all the way before we reach ModuleBootstrap::boot()): ClassLoaderData::the_null_class_loader_data()->modules() and ClassLoaderData::the_null_class_loader_data()->packages() are already populated before a single class is loaded. This difference doesn't seem to make a practical difference. E.g., none of our code seems to assume that "before any classes in the java.util package is loaded, ClassLoaderData::the_null_class_loader_data()->packages() must not contain an entry for java.util". I think we have two choices when the archived module graph is used: [1] We require that the state of the module system must, at every step of VM initialization, be identical to that of a VM that doesn't use an archived module graph. [2] We make sure that the VM/JDK bootstrap code can tolerate the existence of module/packages that are added earlier than a VM that doesn't use an archived module graph. I tried doing a version of [1] and found that to be too difficult. [2] seems much simpler and is the approach I am using now. > oops/instanceKlass.cpp > - line #2545, comment "TO DO? -- point it to the archived PackageEntry > (JDK-8249262)" > are you thinking that since the module graph is read in ahead of time > that it can be set when an InstanceKlass is created?? There is a point > before java.base is defined that InstanceKlass::set_package takes into > account that could be a timing issue. > > I think it will work as if another class in the same package has already been defined. > - There are some checks in modules.cpp that are valuable to have > during bootstrapping.? For example, packages that are encountered > during bootstrapping before java.base is defined are put into the > java.base module but then verified after java.base is defined via > verify_javabase_packages.? So at what point in the bootstrapping > process does the boot loader's archived module's become known? Could > classes have been defined to the VM before this point?? Then their > packages will have to be verified to be sure that they are indeed > packages within java.base.? Consider looking at other checks that > occur within classfile/modules.cpp as well. > As mentioned above, calling verify_javabase_packages() at run time will fail, as we have loaded all packages for the boot loader, not just those for java.base. Well, verify_javabase_packages() was called once and it succeeded, but that was during CDS dump time. So we know at least we have verified this once :-) Thanks - Ioi > I may have more review comments as I continue to look at this! > > Thanks, > Lois > > > > > From sundararajan.athijegannathan at oracle.com Thu Aug 13 03:20:32 2020 From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com) Date: Thu, 13 Aug 2020 08:50:32 +0530 Subject: Fwd: RFR: 8247536: Support pre-generated MethodHandle lambda forms in CDS In-Reply-To: <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> Message-ID: <11f21e9e-d42a-bb7e-31f6-ba523601bd9f@oracle.com> * File: InvokerBytecodeGeneratorHelper.java copyright header has GNU without the "Classpath exception" clause. * The comment could say this string prefix is known in hotspot (classFileParser.cpp hard codes it) + // The log prefix for tracing resolve + static final String CDS_LOG_PREFIX = "@lambda-form-invoker"; + * File: GenerateJLIClassesPlugin.java + List all_lines = Files.readAllLines(file.toPath()); + // convert to String[] + int size = all_lines.size(); + lines = new String[size]; + lines = all_lines.toArray(lines); could be ??? lines = Files.readAllLines(file.toPath()).toArray(new String[0]); (see also: https://shipilev.net/blog/2016/arrays-wisdom-ancients/) * +import java.util.ArrayList; Appears to be unused -Sundar On 13/08/20 12:24 am, calvin.cheung at oracle.com wrote: > Hi Yumin, > > I reviewed mostly the native code. Below are my comments: > > 1) classListParser.hpp > > 71?? bool??????????????? _lambda_format; > > The above name is too generic. How about _lambda_form or _is_lambda_form? > If you rename the above, please also rename the function which returns > the above bool. > > 2) jvm.cpp > > 3850 JVM_ENTRY(void, JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, > jstring line)) > > ignore -> ignored > > 3) jvm.hpp > > 210 JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line); > > Same comment as for jvm.cpp > > 4) metaspaceShared.cpp > > 2017?? size_t i = 0; > 2018?? while (i < size) { > 2019???? full_name[i++] = *start++; > 2020?? } > > Could the above be simplified to the following? > > ??? strncpy(full_name, start, size - 1); > > 2029?? char* class_name = get_full_class_name(path_name); > > Should os::free(class_name) be called before the function returns? > > 1870 static GrowableArray* lambda_list = NULL; > > The name lambda_list is a bit generic, how about lambda_form_list? > > 2112?????? lambda_list->append(parser.current_line()); > > Since parser.current_line() does a strdup, do those buffer need to be > freed after its use? (i.e. after the call to > regenerate_holder_classes()?) > > In MetaspaceShared::regenerate_holder_classes, before calling up to > java, I think it's better to strip the prefix "@lambda-form-invoker" > from the strings. So that the java > InvokerBytecodeGeneratorHelper.readTraceConfig method doesn't need to > handle it: > > ?143???????????????????? case "[LF_RESOLVE]": > ?144???????????????????? case InvokerBytecodeGenerator.CDS_LOG_PREFIX: > > 5) DumpSymbolAndStringTable.java > > 37???? private static final String my_string = > "DumpSymbolAndStringTable"; > > Unused variable? > > thanks, > > Calvin > > On 8/11/20 10:36 AM, Yumin Qi wrote: >> Forget to send to @core-lib-dev, the patch changed jdk code. >> >> >> Thanks >> >> Yumin >> >> >> >> -------- Forwarded Message -------- >> Subject:???? RFR: 8247536: Support pre-generated MethodHandle lambda >> forms in CDS >> Date:???? Tue, 11 Aug 2020 07:44:34 -0700 >> From:???? Yumin Qi >> To:???? hotspot-runtime-dev at openjdk.java.net >> >> >> >> Hi, Please reivew >> >> ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 >> ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ >> >> ? Summary: CDS does not archive pre-generated lambda form classes for >> method handles: >> >> [0.142s][info][class,load] >> java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: >> __JVM_LookupDefineClass__ >> >> The method handle resolution if not found in the holder class, a >> class with the method will be generated and loaded as vm internal >> created class and not archived like above. Those class names are >> mangled as combination of the class name and the class instance address. >> >> In this patch, collect those method holder class names and their >> associated methods' signatures when user specify DumpLoadedClassList, >> and record them in the log file in a format similar to the output >> format from traced by >> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use >> another name for CDS: >> -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. The >> line prefix also changed from "[LF_RESOLVE]" to >> "@lambda-invoke-handle".? At dump time, regenerate the holder class >> with those methods and replace the existing holder class and archived >> it. At runtime, the resolution of the holder class which contains >> those methods are loaded from CDS archive so avoid regenerating them >> again. The patch reorganized the code for Jlink plugin, so the new >> added InvokerBytecodeGeneratorHelper can be shared both by JLink >> plugin and CDS code. Also change made to the mangled generated class >> name at static dump, the class name combining the class name and a >> sequentially ordered number instead of the class instance address, >> which looks like a random. >> >> >> To use this feature, one can do the dump/run just like done for a >> static dump/run, so no extra steps required. >> >> ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp >> >> ????? DumpLoadedClassList will turn on >> -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true >> >> ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist >> -XX:SharedArchiveFile=my.jsa JavaApp >> >> ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp >> >> ? The performance on javac HelloWorld.java (javac-benchmark), no >> patch vs patch: >> >> ?? 1:?? 2689285002? 2641821474 (-47463528)????? ----???? 391.720 >> 382.990 ( -8.730)????? ---- >> ?? 2:?? 2687495085? 2632969688 (-54525397)????? ----???? 391.030 >> 381.480 ( -9.550)????? ----- >> ?? 3:?? 2694664066? 2636523114 (-58140952)????? -----??? 390.610 >> 382.550 ( -8.060)????? ---- >> ?? 4:?? 2686554164? 2639355233 (-47198931)????? ----???? 390.700 >> 383.390 ( -7.310)????? --- >> ?? 5:?? 2691072338? 2633016687 (-58055651)????? -----??? 388.990 >> 382.360 ( -6.630)????? --- >> ?? 6:?? 2684448174? 2644191854 (-40256320)????? ---????? 389.450 >> 382.990 ( -6.460)????? --- >> ?? 7:?? 2694921227? 2630505090 (-64416137)????? -----??? 389.300 >> 383.160 ( -6.140)????? --- >> ?? 8:?? 2685209712? 2639334320 (-45875392)????? ----???? 388.370 >> 381.060 ( -7.310)????? --- >> ?? 9:?? 2695885942? 2640618655 (-55267287)????? ----???? 389.560 >> 381.100 ( -8.460)????? ---- >> ? 10:?? 2689162942? 2635658943 (-53503999)????? ----???? 389.690 >> 379.110 (-10.580)????? ----- >> ============================================================ >> ??????? 2689866989? 2637396210 (-52470778)????? ----???? 389.941 >> 382.017 ( -7.924)????? ---- >> >> instr delta =??? -52470778??? -1.9507% >> >> time? delta =?????? -7.924 ms -2.0321% >> >> ? It will show much more improvement if compare with the default >> archive: >> >> ?????????? ? ? ? no patch??????????????????????? patch >> >> instr ??? ? ? 3,996,847,605??????? 3,320,928,995 >> >> time(s).???? 0.686835162 ? ? ? ?? 0.474933546 >> >> >> ? Tests: >> >> 1)jtreg on jdk/tools and hotspot/runtime local. >> >> 2) mach5 ter1,2 >> >> >> Thanks >> >> Yumin >> >> ? ` From fw at deneb.enyo.de Thu Aug 13 09:56:29 2020 From: fw at deneb.enyo.de (Florian Weimer) Date: Thu, 13 Aug 2020 11:56:29 +0200 Subject: RFR[8238286]: 'Add new flatMap stream operation that is more amenable to =?utf-8?Q?pushing=E2=80=99?= In-Reply-To: (Patrick Concannon's message of "Tue, 11 Aug 2020 19:11:56 +0100") References: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> <5B4DFB68-747C-4EAB-A042-C24DF6516FB8@oracle.com> <47c1e40f-86ac-415d-5c43-674b47d8884d@anthonyv.be> Message-ID: <875z9muawi.fsf@mid.deneb.enyo.de> * Patrick Concannon: > webrev: http://cr.openjdk.java.net/~pconcannon/8238286/webrevs/webrev.05/ + *

    The results of this method are undefined if the second {@link IntConsumer} + * argument is operated on outside the scope of the mapper function. Should this say ?after the mapper function has returned?? Otherwise, one could argue that is possible to use a consumer from a previous invocation of the mapper function. (And maybe add a comma after ?second?? Not a native speaker.) From fw at deneb.enyo.de Thu Aug 13 09:58:38 2020 From: fw at deneb.enyo.de (Florian Weimer) Date: Thu, 13 Aug 2020 11:58:38 +0200 Subject: RFR[8238286]: 'Add new flatMap stream operation that is more amenable to =?utf-8?Q?pushing=E2=80=99?= In-Reply-To: <1105399538.6154.1597268910050.JavaMail.zimbra@u-pem.fr> (Remi Forax's message of "Wed, 12 Aug 2020 23:48:30 +0200 (CEST)") References: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> <5B4DFB68-747C-4EAB-A042-C24DF6516FB8@oracle.com> <47c1e40f-86ac-415d-5c43-674b47d8884d@anthonyv.be> <1105399538.6154.1597268910050.JavaMail.zimbra@u-pem.fr> Message-ID: <871rkauasx.fsf@mid.deneb.enyo.de> * Remi Forax: > I still don't like the fact that IntMapMultiConsumer, > DoubleMapMultiConsumer and LongMapMultiConsumer are not in > java.util.function unlike all other functional interfaces used by > the Stream API. They seem rather too specific for java.util.function to me. Maybe we should add BiIntObjectConsumer etc., a BiConsumer whose first argument is fixed as an int? And the second consumer would be injected as a type parameter, specific to the use in IntStream? From daniel.fuchs at oracle.com Thu Aug 13 12:00:43 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 13 Aug 2020 13:00:43 +0100 Subject: 8251160: Fix "no comment" warnings in java.logging Message-ID: Hi, Please find below a doc-only fix for: 8251160: Fix "no comment" warnings in java.logging https://bugs.openjdk.java.net/browse/JDK-8251160 CSR: https://bugs.openjdk.java.net/browse/JDK-8251534 webrev: http://cr.openjdk.java.net/~dfuchs/webrev_8251160/webrev.00/ The documentation of the serial form of java.util.logging.Level and java.util.logging.LogRecord is improved to avoid doclint warnings. best regards, -- daniel From forax at univ-mlv.fr Thu Aug 13 12:04:28 2020 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Thu, 13 Aug 2020 14:04:28 +0200 (CEST) Subject: =?utf-8?Q?Re:_RFR[8238286]:_'Add_new_flatMap_stream_?= =?utf-8?Q?operation_that_is_more_amenable_to_pushing=E2=80=99?= In-Reply-To: <871rkauasx.fsf@mid.deneb.enyo.de> References: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> <47c1e40f-86ac-415d-5c43-674b47d8884d@anthonyv.be> <1105399538.6154.1597268910050.JavaMail.zimbra@u-pem.fr> <871rkauasx.fsf@mid.deneb.enyo.de> Message-ID: <2050010633.50019.1597320268168.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Florian Weimer" > ?: "Remi Forax" > Cc: "Patrick Concannon" , "Julia Boes" , "Anthony Vanelverdinghe" > , "core-libs-dev" > Envoy?: Jeudi 13 Ao?t 2020 11:58:38 > Objet: Re: RFR[8238286]: 'Add new flatMap stream operation that is more amenable to pushing? > * Remi Forax: > >> I still don't like the fact that IntMapMultiConsumer, >> DoubleMapMultiConsumer and LongMapMultiConsumer are not in >> java.util.function unlike all other functional interfaces used by >> the Stream API. > > They seem rather too specific for java.util.function to me. Maybe we > should add BiIntObjectConsumer etc., a BiConsumer whose first argument > is fixed as an int? And the second consumer would be injected as a > type parameter, specific to the use in IntStream? yes, you right, the interafce should be changed as you said. R?mi From julia.boes at oracle.com Thu Aug 13 15:05:51 2020 From: julia.boes at oracle.com (Julia Boes) Date: Thu, 13 Aug 2020 16:05:51 +0100 Subject: Fix for Javadoc errors in java.base In-Reply-To: References: Message-ID: <318a4bbf-1b3e-d34e-d69a-d1a2ee233794@oracle.com> Hi Vipin, Thanks for providing this fix, I'm happy to sponsor your change. To complete the review, we still need someone to green light the remaining changes below. I'm looping in net-dev and security-dev to have a look. Bug: https://bugs.openjdk.java.net/browse/JDK-8251542 Webrev: http://cr.openjdk.java.net/~jboes/webrevs/8251542/webrev/ Once the review is completed, please provide me with a patch that includes a changeset comment as described here: https://openjdk.java.net/guide/producingChangeset.html If you have any questions, let me know. Cheers, Julia > --- old/src/java.base/share/classes/com/sun/crypto/provider/DHPrivateKey.java > 2020-07-25 23:46:21.233726447 +0530 > +++ new/src/java.base/share/classes/com/sun/crypto/provider/DHPrivateKey.java > 2020-07-25 23:46:20.721720857 +0530 > @@ -96,8 +96,6 @@ > * @param p the prime modulus > * @param g the base generator > * @param l the private-value length > - * > - * @exception InvalidKeyException if the key cannot be encoded > */ > DHPrivateKey(BigInteger x, BigInteger p, BigInteger g, int l) { > this.x = x; > --- old/src/java.base/share/classes/java/lang/invoke/AbstractValidatingLambdaMetafactory.java > 2020-07-25 23:46:22.241737383 +0530 > +++ new/src/java.base/share/classes/java/lang/invoke/AbstractValidatingLambdaMetafactory.java > 2020-07-25 23:46:21.745732013 +0530 > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 2012, 2013, Oracle and/or its affiliates. All rights reserved. > + * Copyright (c) 2012, 2020, Oracle and/or its affiliates. All rights reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -208,7 +208,7 @@ > * > * @return a CallSite, which, when invoked, will return an instance of the > * functional interface > - * @throws ReflectiveOperationException > + * @throws LambdaConversionException > */ > abstract CallSite buildCallSite() > throws LambdaConversionException; > --- old/src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java > 2020-07-25 23:46:23.261748361 +0530 > +++ new/src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java > 2020-07-25 23:46:22.761742991 +0530 > @@ -200,7 +200,6 @@ > * > * @return a CallSite, which, when invoked, will return an instance of the > * functional interface > - * @throws ReflectiveOperationException > * @throws LambdaConversionException If properly formed functional interface > * is not found > */ > --- old/src/java.base/share/classes/java/net/ServerSocket.java 2020-07-25 > 23:46:26.449782093 +0530 > +++ new/src/java.base/share/classes/java/net/ServerSocket.java 2020-07-25 > 23:46:25.937776742 +0530 > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 1995, 2019, Oracle and/or its affiliates. All rights reserved. > + * Copyright (c) 1995, 2020, Oracle and/or its affiliates. All rights reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -315,7 +315,7 @@ > /** > * Creates the socket implementation. > * > - * @throws IOException if creation fails > + * @throws SocketException if creation fails > * @since 1.4 > */ > void createImpl() throws SocketException { > --- old/src/java.base/share/classes/java/net/Socket.java 2020-07-25 > 23:46:27.485792869 +0530 > +++ new/src/java.base/share/classes/java/net/Socket.java 2020-07-25 > 23:46:26.973787565 +0530 > @@ -533,7 +533,7 @@ > * > * @param stream a {@code boolean} value : {@code true} for a TCP socket, > * {@code false} for UDP. > - * @throws IOException if creation fails > + * @throws SocketException if creation fails > * @since 1.4 > */ > void createImpl(boolean stream) throws SocketException { From LANCE.ANDERSEN at ORACLE.COM Thu Aug 13 15:15:02 2020 From: LANCE.ANDERSEN at ORACLE.COM (Lance Andersen) Date: Thu, 13 Aug 2020 11:15:02 -0400 Subject: 8251160: Fix "no comment" warnings in java.logging In-Reply-To: References: Message-ID: Hi Daniel, The changes look good. I added myself as a reviewer to the CSR Best Lance > On Aug 13, 2020, at 8:00 AM, Daniel Fuchs wrote: > > Hi, > > Please find below a doc-only fix for: > > 8251160: Fix "no comment" warnings in java.logging > https://bugs.openjdk.java.net/browse/JDK-8251160 > > CSR: https://bugs.openjdk.java.net/browse/JDK-8251534 > > webrev: > http://cr.openjdk.java.net/~dfuchs/webrev_8251160/webrev.00/ > > The documentation of the serial form of java.util.logging.Level > and java.util.logging.LogRecord is improved to avoid doclint > warnings. > > best regards, > > -- daniel Best Lance ------------------ Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From brian.burkhalter at oracle.com Thu Aug 13 15:40:52 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 13 Aug 2020 08:40:52 -0700 Subject: [Ping] Re: 8181919: Refactor test/java/io/File/GetXSpace.sh to java test In-Reply-To: References: <711DD0AF-CAB7-4DE2-9495-3D27B1F3A0A3@oracle.com> Message-ID: Thanks! > On Aug 11, 2020, at 11:32 AM, Brian Burkhalter wrote: > > A revised version [A] is available pursuant to the changes made in the course of the review [B] of the fix for [C]. > > Thanks, > > Brian > > [A] http://cr.openjdk.java.net/~bpb/8181919/webrev.01/ > [B] http://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068094.html > [C] https://bugs.openjdk.java.net/browse/JDK-8249703 > >> On Aug 5, 2020, at 11:36 AM, Brian Burkhalter > wrote: >> >> To fix [1], please consider the patch [2]. This follows on from the proposed change [3] which in turn follows from [4]. >> >> Thanks, >> >> Brian >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8181919 >> [2] http://cr.openjdk.java.net/~bpb/8181919/webrev.00/ >> [3] https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068025.html (GetXSpace should not run on AIX) >> [4] http://mail.openjdk.java.net/pipermail/core-libs-dev/2020-July/067990.html (GetXSpace fails on macOS) > From stuart.monteith at arm.com Thu Aug 13 15:48:34 2020 From: stuart.monteith at arm.com (Stuart Monteith) Date: Thu, 13 Aug 2020 16:48:34 +0100 Subject: [aarch64-port-dev ] [16] RFR[S]: 8251216: Implement MD5 intrinsics on AArch64 In-Reply-To: <0586ead6-583d-1907-491e-64db6edf2106@redhat.com> References: <2d960bbe-0191-db94-2d5c-7df511a36dba@redhat.com> <0586ead6-583d-1907-491e-64db6edf2106@redhat.com> Message-ID: On 13/08/2020 11:00, Andrew Haley wrote: > On 12/08/2020 12:38, Stuart Monteith wrote: > > > The method "testDigest" generates an byte array of a given size, > > with each element filled with it's own index & 0xff. > > > > The test is then run once, assumed uncompiled, it is then "warmed > > up" and the first generated digest is compared against the digest > > presumably generated by the intrinsic. This is the same test for all > > of the message digest algorithms. > > > > I'd say the test is no worse than what has gone before. There are > > additional tests under the jdk library tests, but nothing that > > addresses the correctness of the MD5 algorithm implementation > > itself. > > Good grief. So there are no compliance tests in the test suite at all. Yes for any algorithm, for either the intrinsics or the Java implementations. > > > In terms of the status-quo, that patch looks ok to me. I think if > > the testing is to be expanded, it should be expanded to all of the > > message digest algorithms. > > That's not much more that an excuse for doing nothing, IMO. > My intention was to suggest that more than MD5 or even just the intrinsics need to be tested, it's not an excuse to ignore this. The existing tests are simply a comparison between generated message digest for a single message between the Java code and the intrinsics. The NIST samples cover SHA1 and MD5, but there are additional samples here: https://csrc.nist.gov/Projects/cryptographic-standards-and-guidelines/example-values . The message digests in Java under sun.security.provider are: MD2, MD4, MD5, SHA1 SHA2: SHA2-224, SHA2-256, SHA3: SHA3-224, SHA3-256, SHA3-384, SHA3-512, SHAKE256 SHA5: SHA-512/224, SHA-512/256, SHA-512, SHA-384, The intrinsics implemented are: aarch64: SHA1, SHA2, SHA5 (+MD5) ppc64: SHA2, SHA5 x86_64: SHA1, SHA2, SHA5, MD5 x86_32: SHA1, SHA2, MD5 The MD5 patches have been merged already for x86. SHA3 doesn't have any intrinsic implementations. MD2 has some example values in its RFC https://tools.ietf.org/html/rfc1319 Likewise, MD4 has example values in its RFC too: https://tools.ietf.org/html/rfc1320 My suggestion is to add new tests for each of the message digest algorithms and share them between the JTreg jdk and hotspot instrinsics. The MD5 intrinsics could be merged after some demonstration of correctness? I've CC'd core-libs-dev as this affects the jdk library. BR, Stuart From daniel.fuchs at oracle.com Thu Aug 13 16:07:30 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 13 Aug 2020 17:07:30 +0100 Subject: Fix for Javadoc errors in java.base In-Reply-To: <318a4bbf-1b3e-d34e-d69a-d1a2ee233794@oracle.com> References: <318a4bbf-1b3e-d34e-d69a-d1a2ee233794@oracle.com> Message-ID: <17593e07-0ae1-c881-47df-17643698d53f@oracle.com> Hi Vipin, Julia, On 13/08/2020 16:05, Julia Boes wrote: > Hi Vipin, > > Thanks for providing this fix, I'm happy to sponsor your change. To > complete the review, we still need someone to green light the remaining > changes below. I'm looping in net-dev and security-dev to have a look. Changes to Socket and ServerSocket look fine. No CSR needed since this is a package private API. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8251542 > > Webrev: http://cr.openjdk.java.net/~jboes/webrevs/8251542/webrev/ best regards, -- daniel From sean.mullan at oracle.com Thu Aug 13 17:13:06 2020 From: sean.mullan at oracle.com (Sean Mullan) Date: Thu, 13 Aug 2020 13:13:06 -0400 Subject: Fix for Javadoc errors in java.base In-Reply-To: <318a4bbf-1b3e-d34e-d69a-d1a2ee233794@oracle.com> References: <318a4bbf-1b3e-d34e-d69a-d1a2ee233794@oracle.com> Message-ID: <8e978983-4ec1-8421-2563-d00c61f0d55e@oracle.com> On 8/13/20 11:05 AM, Julia Boes wrote: > Hi Vipin, > > Thanks for providing this fix, I'm happy to sponsor your change. To > complete the review, we still need someone to green light the remaining > changes below. I'm looping in net-dev and security-dev to have a look. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8251542 > > Webrev: http://cr.openjdk.java.net/~jboes/webrevs/8251542/webrev/ > > Once the review is completed, please provide me with a patch that > includes a changeset comment as described here: > https://openjdk.java.net/guide/producingChangeset.html > > If you have any questions, let me know. > > Cheers, > > Julia > >> --- >> old/src/java.base/share/classes/com/sun/crypto/provider/DHPrivateKey.java >> 2020-07-25 23:46:21.233726447 +0530 >> +++ >> new/src/java.base/share/classes/com/sun/crypto/provider/DHPrivateKey.java >> 2020-07-25 23:46:20.721720857 +0530 >> @@ -96,8 +96,6 @@ >> ?????? * @param p the prime modulus >> ?????? * @param g the base generator >> ?????? * @param l the private-value length >> -???? * >> -???? * @exception InvalidKeyException if the key cannot be encoded This should actually remain, but it should be ProviderException which is a RuntimeException. See the other DHPrivateKey ctor as that specifies it correctly. --Sean From jonathan.gibbons at oracle.com Thu Aug 13 17:21:49 2020 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 13 Aug 2020 10:21:49 -0700 Subject: Fix for Javadoc errors in java.base In-Reply-To: <8e978983-4ec1-8421-2563-d00c61f0d55e@oracle.com> References: <318a4bbf-1b3e-d34e-d69a-d1a2ee233794@oracle.com> <8e978983-4ec1-8421-2563-d00c61f0d55e@oracle.com> Message-ID: On 8/13/20 10:13 AM, Sean Mullan wrote: > On 8/13/20 11:05 AM, Julia Boes wrote: >> Hi Vipin, >> >> Thanks for providing this fix, I'm happy to sponsor your change. To >> complete the review, we still need someone to green light the >> remaining changes below. I'm looping in net-dev and security-dev to >> have a look. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8251542 >> >> Webrev: http://cr.openjdk.java.net/~jboes/webrevs/8251542/webrev/ >> >> Once the review is completed, please provide me with a patch that >> includes a changeset comment as described here: >> https://openjdk.java.net/guide/producingChangeset.html >> >> If you have any questions, let me know. >> >> Cheers, >> >> Julia >> >>> --- >>> old/src/java.base/share/classes/com/sun/crypto/provider/DHPrivateKey.java >>> 2020-07-25 23:46:21.233726447 +0530 >>> +++ >>> new/src/java.base/share/classes/com/sun/crypto/provider/DHPrivateKey.java >>> 2020-07-25 23:46:20.721720857 +0530 >>> @@ -96,8 +96,6 @@ >>> ?????? * @param p the prime modulus >>> ?????? * @param g the base generator >>> ?????? * @param l the private-value length >>> -???? * >>> -???? * @exception InvalidKeyException if the key cannot be encoded > > This should actually remain, but it should be ProviderException which > is a RuntimeException. See the other DHPrivateKey ctor as that > specifies it correctly. > > --Sean I note the use of `@exception`, as compared to `@throws`, which is more common. Stats: `@exception` 7322 occurrences `@throws` 21173 occurrences That's probably too many `@exception` to clean up. :-( -- Jon From sean.mullan at oracle.com Thu Aug 13 17:25:57 2020 From: sean.mullan at oracle.com (Sean Mullan) Date: Thu, 13 Aug 2020 13:25:57 -0400 Subject: Fix for Javadoc errors in java.base In-Reply-To: References: <318a4bbf-1b3e-d34e-d69a-d1a2ee233794@oracle.com> <8e978983-4ec1-8421-2563-d00c61f0d55e@oracle.com> Message-ID: <11adede2-1119-f133-020f-4ddd60c9fc67@oracle.com> On 8/13/20 1:21 PM, Jonathan Gibbons wrote: >>>> --- >>>> old/src/java.base/share/classes/com/sun/crypto/provider/DHPrivateKey.java >>>> >>>> 2020-07-25 23:46:21.233726447 +0530 >>>> +++ >>>> new/src/java.base/share/classes/com/sun/crypto/provider/DHPrivateKey.java >>>> >>>> 2020-07-25 23:46:20.721720857 +0530 >>>> @@ -96,8 +96,6 @@ >>>> ?????? * @param p the prime modulus >>>> ?????? * @param g the base generator >>>> ?????? * @param l the private-value length >>>> -???? * >>>> -???? * @exception InvalidKeyException if the key cannot be encoded >> >> This should actually remain, but it should be ProviderException which >> is a RuntimeException. See the other DHPrivateKey ctor as that >> specifies it correctly. >> >> --Sean > > > I note the use of `@exception`, as compared to `@throws`, which is more > common. > > Stats: > `@exception` 7322 occurrences > `@throws` 21173 occurrences > > That's probably too many `@exception` to clean up. :-( Right, that's probably a separate cleanup activity. However, if you want to change the 3 instances of @exception to @throws in DHPrivateKey, I'm fine with that. --Sean From naoto.sato at oracle.com Thu Aug 13 17:52:41 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Thu, 13 Aug 2020 10:52:41 -0700 Subject: [Ping] Re: 8181919: Refactor test/java/io/File/GetXSpace.sh to java test In-Reply-To: References: <711DD0AF-CAB7-4DE2-9495-3D27B1F3A0A3@oracle.com> Message-ID: <37a434a1-e084-9288-a082-dece5dc2e415@oracle.com> Looks good, Brian. I'd add the bug id in GetXSpace.java's @bug tag. Naoto On 8/13/20 8:40 AM, Brian Burkhalter wrote: > Thanks! > >> On Aug 11, 2020, at 11:32 AM, Brian Burkhalter wrote: >> >> A revised version [A] is available pursuant to the changes made in the course of the review [B] of the fix for [C]. >> >> Thanks, >> >> Brian >> >> [A] http://cr.openjdk.java.net/~bpb/8181919/webrev.01/ >> [B] http://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068094.html >> [C] https://bugs.openjdk.java.net/browse/JDK-8249703 >> >>> On Aug 5, 2020, at 11:36 AM, Brian Burkhalter > wrote: >>> >>> To fix [1], please consider the patch [2]. This follows on from the proposed change [3] which in turn follows from [4]. >>> >>> Thanks, >>> >>> Brian >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8181919 >>> [2] http://cr.openjdk.java.net/~bpb/8181919/webrev.00/ >>> [3] https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068025.html (GetXSpace should not run on AIX) >>> [4] http://mail.openjdk.java.net/pipermail/core-libs-dev/2020-July/067990.html (GetXSpace fails on macOS) >> > From christoph.dreis at freenet.de Thu Aug 13 17:31:13 2020 From: christoph.dreis at freenet.de (Christoph Dreis) Date: Thu, 13 Aug 2020 19:31:13 +0200 Subject: Optimize sun.invoke.util.BytecodeDescriptor.unparse Message-ID: Hi, I just stumbled upon sun.invoke.util.BytecodeDescriptor.unparse that seems to unnecessarily create a StringBuilder and checks for the given type to be of Object.class twice in certain scenarios. When I apply the attached patch below with the following isolated benchmark: @BenchmarkMode(Mode.AverageTime) @OutputTimeUnit(TimeUnit.NANOSECONDS) @State(Scope.Thread) public class MyBenchmark { @State(Scope.Thread) public static class BenchmarkState { private Class test = String.class; // long.class; } @Benchmark public String unparseNew(BenchmarkState state) { return BytecodeDescriptor.unparseNew(state.test); } @Benchmark public String unparseOld(BenchmarkState state) { return BytecodeDescriptor.unparseOld(state.test); } } I get the following results: String.class Benchmark Mode Cnt Score Error Units MyBenchmark.unparseNew avgt 10 47,207 ? 1,918 ns/op MyBenchmark.unparseNew:?gc.alloc.rate.norm avgt 10 232,011 ? 0,002 B/op MyBenchmark.unparseOld avgt 10 87,197 ? 22,843 ns/op MyBenchmark.unparseOld:?gc.alloc.rate.norm avgt 10 384,020 ? 0,001 B/op long.class Benchmark Mode Cnt Score Error Units MyBenchmark.unparseNew avgt 10 4,996 ? 0,022 ns/op MyBenchmark.unparseNew:?gc.alloc.rate.norm avgt 10 ? 10?? B/op MyBenchmark.unparseOld avgt 10 13,303 ? 6,305 ns/op MyBenchmark.unparseOld:?gc.alloc.rate.norm avgt 10 80,003 ? 0,001 B/op As you can see the new way makes things allocation free for primitives and also improves normal classes. It seems like a relatively trivial improvement. In case you think this is worthwhile, I would appreciate it if someone could sponsor the change. Cheers, Christoph ======= PATCH ======= --- a/src/java.base/share/classes/sun/invoke/util/BytecodeDescriptor.java Thu Aug 13 09:33:28 2020 -0700 +++ b/src/java.base/share/classes/sun/invoke/util/BytecodeDescriptor.java Thu Aug 13 19:27:26 2020 +0200 @@ -110,9 +110,13 @@ } else if (type == int.class) { return "I"; } - StringBuilder sb = new StringBuilder(); - unparseSig(type, sb); - return sb.toString(); + Wrapper basicType = Wrapper.forBasicType(type); + char c = basicType.basicTypeChar(); + if (c != 'L') { + return basicType.basicTypeString(); + } else { + return type.descriptorString(); + } } From brian.burkhalter at oracle.com Thu Aug 13 17:58:34 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 13 Aug 2020 10:58:34 -0700 Subject: [Ping] Re: 8181919: Refactor test/java/io/File/GetXSpace.sh to java test In-Reply-To: <37a434a1-e084-9288-a082-dece5dc2e415@oracle.com> References: <711DD0AF-CAB7-4DE2-9495-3D27B1F3A0A3@oracle.com> <37a434a1-e084-9288-a082-dece5dc2e415@oracle.com> Message-ID: <98CEDCF9-EE8C-4499-907A-A825F484640A@oracle.com> Thanks, Naoto, will do. I am always unclear as to whether to add a bug id to a test when the issue is labelled ?noreg-self.? Brian > On Aug 13, 2020, at 10:52 AM, naoto.sato at oracle.com wrote: > > Looks good, Brian. I'd add the bug id in GetXSpace.java's @bug tag. > > Naoto > > On 8/13/20 8:40 AM, Brian Burkhalter wrote: >> Thanks! >>> On Aug 11, 2020, at 11:32 AM, Brian Burkhalter > wrote: >>> >>> A revised version [A] is available pursuant to the changes made in the course of the review [B] of the fix for [C]. From andy.herrick at oracle.com Thu Aug 13 19:35:47 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Thu, 13 Aug 2020 15:35:47 -0400 Subject: RFR: JDK-8250611: Cannot display splash screen on Windows Message-ID: <765b70c2-ec7e-475d-cafe-d646847f5d2b@oracle.com> Please review this jpackage fix for issue [1] at [2]. In order to show splash screen from statically linked applauncher, we need to load the dependent libraries of splashscreen.dll first (java.dll and jvm.dll). /Andy [1] - https://bugs.openjdk.java.net/browse/JDK-8250611 [2] - http://cr.openjdk.java.net/~herrick/8250611/webrev.01/ From mandy.chung at oracle.com Thu Aug 13 19:48:44 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 13 Aug 2020 12:48:44 -0700 Subject: 8251160: Fix "no comment" warnings in java.logging In-Reply-To: References: Message-ID: <78d70e3e-fefb-2f24-a114-81b250c5e8c3@oracle.com> On 8/13/20 5:00 AM, Daniel Fuchs wrote: > Hi, > > Please find below a doc-only fix for: > > 8251160: Fix "no comment" warnings in java.logging > https://bugs.openjdk.java.net/browse/JDK-8251160 > > CSR: https://bugs.openjdk.java.net/browse/JDK-8251534 > > webrev: > http://cr.openjdk.java.net/~dfuchs/webrev_8251160/webrev.00/ > Level The javadoc for readResolve seems internal implementation details. Would it be sufficient to say: ?Returns a {@code Level} instance with the same {@code name}, ?{@code value}, and {@code resourceBundleName} as the deserialized ?object" LogRecord + * See {@code writeObject} for a description of the serial form. This can use @see instead. Mandy From alexander.matveev at oracle.com Thu Aug 13 20:13:19 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Thu, 13 Aug 2020 13:13:19 -0700 Subject: RFR: 8250803: pkgbuild failed with exit code 134 Message-ID: Please review the jpackage fix for bug [1] at [2]. - Looks like issue was just in not enough time for test to execute. Increasing timeout by 1 min fixed issues. In fix it is increased by 50% just in case. Without timeout increase it was reproducible 100% on one of test machine. With increased timeout I cannot reproduce it after multiple runs. [1] https://bugs.openjdk.java.net/browse/JDK-8250803 [2] http://cr.openjdk.java.net/~almatvee/8250803/webrev.00/ Thanks, Alexander From joe.darcy at oracle.com Thu Aug 13 20:21:34 2020 From: joe.darcy at oracle.com (Joe Darcy) Date: Thu, 13 Aug 2020 13:21:34 -0700 Subject: RFR: 8250803: pkgbuild failed with exit code 134 In-Reply-To: References: Message-ID: <0dda0db9-6bf5-449a-b24f-ee6910e1fb79@oracle.com> What steps were taken to make the test run faster instead of increasing the time out? cheers, -Joe On 8/13/2020 1:13 PM, alexander.matveev at oracle.com wrote: > Please review the jpackage fix for bug [1] at [2]. > > - Looks like issue was just in not enough time for test to execute. > Increasing timeout by 1 min fixed issues. In fix it is increased by > 50% just in case. Without timeout increase it was reproducible 100% on > one of test machine. With increased timeout I cannot reproduce it > after multiple runs. > > [1] https://bugs.openjdk.java.net/browse/JDK-8250803 > [2] http://cr.openjdk.java.net/~almatvee/8250803/webrev.00/ > > Thanks, > Alexander From alexey.semenyuk at oracle.com Thu Aug 13 21:54:12 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Thu, 13 Aug 2020 17:54:12 -0400 Subject: RFR: JDK-8250611: Cannot display splash screen on Windows In-Reply-To: <765b70c2-ec7e-475d-cafe-d646847f5d2b@oracle.com> References: <765b70c2-ec7e-475d-cafe-d646847f5d2b@oracle.com> Message-ID: <2d4ef4a0-c8f9-9488-2748-18e3c0c6e60c@oracle.com> Looks good. - Alexey On 8/13/2020 3:35 PM, Andy Herrick wrote: > Please review this jpackage fix for issue [1] at [2]. > > In order to show splash screen from statically linked applauncher, we > need to load the dependent libraries of splashscreen.dll first > (java.dll and jvm.dll). > > /Andy > > [1] - https://bugs.openjdk.java.net/browse/JDK-8250611 > > [2] - http://cr.openjdk.java.net/~herrick/8250611/webrev.01/ > From alexey.semenyuk at oracle.com Thu Aug 13 21:55:33 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Thu, 13 Aug 2020 17:55:33 -0400 Subject: RFR: 8250803: pkgbuild failed with exit code 134 In-Reply-To: References: Message-ID: <3ef76b62-6e08-d28a-69ab-f9f3f47fbed0@oracle.com> Looks good. - Alexey On 8/13/2020 4:13 PM, alexander.matveev at oracle.com wrote: > Please review the jpackage fix for bug [1] at [2]. > > - Looks like issue was just in not enough time for test to execute. > Increasing timeout by 1 min fixed issues. In fix it is increased by > 50% just in case. Without timeout increase it was reproducible 100% on > one of test machine. With increased timeout I cannot reproduce it > after multiple runs. > > [1] https://bugs.openjdk.java.net/browse/JDK-8250803 > [2] http://cr.openjdk.java.net/~almatvee/8250803/webrev.00/ > > Thanks, > Alexander From philip.race at oracle.com Thu Aug 13 22:03:57 2020 From: philip.race at oracle.com (Philip Race) Date: Thu, 13 Aug 2020 15:03:57 -0700 Subject: RFR: JDK-8250611: Cannot display splash screen on Windows In-Reply-To: <765b70c2-ec7e-475d-cafe-d646847f5d2b@oracle.com> References: <765b70c2-ec7e-475d-cafe-d646847f5d2b@oracle.com> Message-ID: <5F35B8CD.3000509@oracle.com> Do I understand correctly that you are checking if the client VM is being specified ? I didn't think we had client VMs any more .. -phil. On 8/13/20, 12:35 PM, Andy Herrick wrote: > Please review this jpackage fix for issue [1] at [2]. > > In order to show splash screen from statically linked applauncher, we > need to load the dependent libraries of splashscreen.dll first > (java.dll and jvm.dll). > > /Andy > > [1] - https://bugs.openjdk.java.net/browse/JDK-8250611 > > [2] - http://cr.openjdk.java.net/~herrick/8250611/webrev.01/ > From alexander.matveev at oracle.com Thu Aug 13 23:52:25 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Thu, 13 Aug 2020 16:52:25 -0700 Subject: RFR: JDK-8250611: Cannot display splash screen on Windows In-Reply-To: <765b70c2-ec7e-475d-cafe-d646847f5d2b@oracle.com> References: <765b70c2-ec7e-475d-cafe-d646847f5d2b@oracle.com> Message-ID: <8df604cd-c7bf-dcfd-d63f-921200e2a485@oracle.com> Hi Andy, Looks good. Thanks, Alexander On 8/13/20 12:35 PM, Andy Herrick wrote: > Please review this jpackage fix for issue [1] at [2]. > > In order to show splash screen from statically linked applauncher, we > need to load the dependent libraries of splashscreen.dll first > (java.dll and jvm.dll). > > /Andy > > [1] - https://bugs.openjdk.java.net/browse/JDK-8250611 > > [2] - http://cr.openjdk.java.net/~herrick/8250611/webrev.01/ > From alexander.matveev at oracle.com Fri Aug 14 00:07:58 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Thu, 13 Aug 2020 17:07:58 -0700 Subject: RFR: 8250803: pkgbuild failed with exit code 134 In-Reply-To: <0dda0db9-6bf5-449a-b24f-ee6910e1fb79@oracle.com> References: <0dda0db9-6bf5-449a-b24f-ee6910e1fb79@oracle.com> Message-ID: Hi Joe, None. Most time is spend when pkgbuild is executed which is external tool. Based on log our code does not consume a lot of time during execution. Based on one of the runs total time was 11 min: 1 min for jlink, 4 min for hdutil create and 4 min for pkgbuild, so test itself took 2 min. Thanks, Alexander On 8/13/20 1:21 PM, Joe Darcy wrote: > What steps were taken to make the test run faster instead of > increasing the time out? > > cheers, > > -Joe > > On 8/13/2020 1:13 PM, alexander.matveev at oracle.com wrote: >> Please review the jpackage fix for bug [1] at [2]. >> >> - Looks like issue was just in not enough time for test to execute. >> Increasing timeout by 1 min fixed issues. In fix it is increased by >> 50% just in case. Without timeout increase it was reproducible 100% >> on one of test machine. With increased timeout I cannot reproduce it >> after multiple runs. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8250803 >> [2] http://cr.openjdk.java.net/~almatvee/8250803/webrev.00/ >> >> Thanks, >> Alexander From alexey.semenyuk at oracle.com Fri Aug 14 02:58:04 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Thu, 13 Aug 2020 22:58:04 -0400 Subject: RFR: JDK-8250611: Cannot display splash screen on Windows In-Reply-To: <5F35B8CD.3000509@oracle.com> References: <765b70c2-ec7e-475d-cafe-d646847f5d2b@oracle.com> <5F35B8CD.3000509@oracle.com> Message-ID: <44d91535-85c8-b3da-b45f-dadb9c2acfd1@oracle.com> Phil, jpackage can be used to bundle apps with older JREs. Do you think it doesn't make sense for this check given this? - Alexey On 8/13/2020 6:03 PM, Philip Race wrote: > Do I understand correctly that you are checking if the client VM is > being specified ? > I didn't think we had client VMs any more .. > > -phil. > > On 8/13/20, 12:35 PM, Andy Herrick wrote: >> Please review this jpackage fix for issue [1] at [2]. >> >> In order to show splash screen from statically linked applauncher, we >> need to load the dependent libraries of splashscreen.dll first >> (java.dll and jvm.dll). >> >> /Andy >> >> [1] - https://bugs.openjdk.java.net/browse/JDK-8250611 >> >> [2] - http://cr.openjdk.java.net/~herrick/8250611/webrev.01/ >> From philip.race at oracle.com Fri Aug 14 03:27:37 2020 From: philip.race at oracle.com (Phil Race) Date: Thu, 13 Aug 2020 20:27:37 -0700 Subject: RFR: JDK-8250611: Cannot display splash screen on Windows In-Reply-To: <44d91535-85c8-b3da-b45f-dadb9c2acfd1@oracle.com> References: <44d91535-85c8-b3da-b45f-dadb9c2acfd1@oracle.com> Message-ID: <5C06D912-0981-4E8E-84C1-6EC93B49E45C@oracle.com> Only back to 11 and client support ended before that. -Phil. > On Aug 13, 2020, at 7:58 PM, Alexey Semenyuk wrote: > > ?Phil, > > jpackage can be used to bundle apps with older JREs. Do you think it doesn't make sense for this check given this? > > - Alexey > >> On 8/13/2020 6:03 PM, Philip Race wrote: >> Do I understand correctly that you are checking if the client VM is being specified ? >> I didn't think we had client VMs any more .. >> >> -phil. >> >>> On 8/13/20, 12:35 PM, Andy Herrick wrote: >>> Please review this jpackage fix for issue [1] at [2]. >>> >>> In order to show splash screen from statically linked applauncher, we need to load the dependent libraries of splashscreen.dll first (java.dll and jvm.dll). >>> >>> /Andy >>> >>> [1] - https://bugs.openjdk.java.net/browse/JDK-8250611 >>> >>> [2] - http://cr.openjdk.java.net/~herrick/8250611/webrev.01/ >>> > From daniel.fuchs at oracle.com Fri Aug 14 09:17:08 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 14 Aug 2020 10:17:08 +0100 Subject: 8251160: Fix "no comment" warnings in java.logging In-Reply-To: <78d70e3e-fefb-2f24-a114-81b250c5e8c3@oracle.com> References: <78d70e3e-fefb-2f24-a114-81b250c5e8c3@oracle.com> Message-ID: Hi Mandy, Thanks for looking at this! On 13/08/2020 20:48, Mandy Chung wrote: > Level > > The javadoc for readResolve seems internal implementation details. Would > it be sufficient to say: > > ?Returns a {@code Level} instance with the same {@code name}, > ?{@code value}, and {@code resourceBundleName} as the deserialized > ?object" Good idea! Done. > LogRecord > > + * See {@code writeObject} for a description of the serial form. > > This can use @see instead. Unfortunately not - I tried it. Links towards the Serialized Form page are not working. In this case (@see) neither the text of the link nor the link is generated by the javadoc. new webrev: http://cr.openjdk.java.net/~dfuchs/webrev_8251160/webrev.01/index.html CSR updated as well: https://bugs.openjdk.java.net/browse/JDK-8251534 best regards, -- daniel > > Mandy From LANCE.ANDERSEN at ORACLE.COM Fri Aug 14 10:37:02 2020 From: LANCE.ANDERSEN at ORACLE.COM (Lance Andersen) Date: Fri, 14 Aug 2020 06:37:02 -0400 Subject: RFR 8251208: Fix javadoc warnings in java.sql and java.sql.rowsets Message-ID: <9C459B41-BD59-44C9-9270-18A9E81882C8@ORACLE.COM> Hi all, Please review the fix to address javadoc warnings in java.sql and java.sql.rowsets The webrev can be found at: http://cr.openjdk.java.net/~lancea/8251208/webrev.00/ And the CSR at: https://bugs.openjdk.java.net/browse/JDK-8251834 Best Lance ------------------ Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From sean.mullan at oracle.com Fri Aug 14 11:50:26 2020 From: sean.mullan at oracle.com (Sean Mullan) Date: Fri, 14 Aug 2020 07:50:26 -0400 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: References: <959ffeaa-a8ef-87fb-6a21-0f125fa3d891@oracle.com> <342c2de2-89a6-41d1-9c27-61f2ed9ed163@oracle.com> <70A3B5D4-3D0E-426C-8EB3-64CB7A42EC50@oracle.com> <2267CBC7-85A9-4105-B405-E7499440B985@azul.com> <8885AB2E-0671-4493-A2D6-88302C48CAF9@oracle.com> <070d2335-32d1-d4f6-892f-1a28c6649a31@oracle.com> <85c2e4b8-f61e-920c-5afe-5e3ed3df5c45@oracle.com> <643f3744-cfa2-e871-1284-900d81709f53@oracle.com> <77fb8b44-2438-2f62-07b6-45c12a9c6f34@oracle.com> <7621fa33-26cc-d545-cb33-a93fdbfd48c7@oracle.com> <8804341B-E534-4FA8-A652-86F8A30F18B3@azul.com> <999EF4D0-4084-4EC2-803C-EDC23D3EB8F2@azul.com> Message-ID: <5b7281d3-4bfc-f0ac-07d3-365451835f70@oracle.com> On the property wording, change "for LDAP connection" to "for an LDAP connection". Also, for the definition of the property, can you use the "@systemProperty" annotation instead of "@code"? Does that work inside the module-info.java file? I added my name as Reviewer. --Sean On 7/30/20 6:14 AM, Daniel Fuchs wrote: > Hi Alexey, > > I have added myself as a reviewer to the CSR [1]. > It would be good to get someone from security-dev to do the > same, and then move the CSR state to "Proposed". > > best regards, > > -- daniel > > [1] https://bugs.openjdk.java.net/browse/JDK-8247311 > > On 30/07/2020 10:17, Alexey Bakhtin wrote: >> Gentle ping >> >> Regards >> Alexey > From alexey at azul.com Fri Aug 14 12:41:00 2020 From: alexey at azul.com (Alexey Bakhtin) Date: Fri, 14 Aug 2020 12:41:00 +0000 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <5b7281d3-4bfc-f0ac-07d3-365451835f70@oracle.com> References: <959ffeaa-a8ef-87fb-6a21-0f125fa3d891@oracle.com> <342c2de2-89a6-41d1-9c27-61f2ed9ed163@oracle.com> <70A3B5D4-3D0E-426C-8EB3-64CB7A42EC50@oracle.com> <2267CBC7-85A9-4105-B405-E7499440B985@azul.com> <8885AB2E-0671-4493-A2D6-88302C48CAF9@oracle.com> <070d2335-32d1-d4f6-892f-1a28c6649a31@oracle.com> <85c2e4b8-f61e-920c-5afe-5e3ed3df5c45@oracle.com> <643f3744-cfa2-e871-1284-900d81709f53@oracle.com> <77fb8b44-2438-2f62-07b6-45c12a9c6f34@oracle.com> <7621fa33-26cc-d545-cb33-a93fdbfd48c7@oracle.com> <8804341B-E534-4FA8-A652-86F8A30F18B3@azul.com> <999EF4D0-4084-4EC2-803C-EDC23D3EB8F2@azul.com> <5b7281d3-4bfc-f0ac-07d3-365451835f70@oracle.com> Message-ID: <2D6CC614-0C5E-49C0-87E1-4FFFE2788DAD@azul.com> Hello Sean, Thank you for review. I?ve changed wording and replaced @code with @systemProperty (tested, it works for module-info.java) Updated webrev at: http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v15/ Regards Alexey > On 14 Aug 2020, at 14:50, Sean Mullan wrote: > > On the property wording, change "for LDAP connection" to "for an LDAP connection". > > Also, for the definition of the property, can you use the "@systemProperty" annotation instead of "@code"? Does that work inside the module-info.java file? > > I added my name as Reviewer. > > --Sean > > On 7/30/20 6:14 AM, Daniel Fuchs wrote: >> Hi Alexey, >> I have added myself as a reviewer to the CSR [1]. >> It would be good to get someone from security-dev to do the >> same, and then move the CSR state to "Proposed". >> best regards, >> -- daniel >> [1] https://bugs.openjdk.java.net/browse/JDK-8247311 >> On 30/07/2020 10:17, Alexey Bakhtin wrote: >>> Gentle ping >>> >>> Regards >>> Alexey From sean.mullan at oracle.com Fri Aug 14 13:18:54 2020 From: sean.mullan at oracle.com (Sean Mullan) Date: Fri, 14 Aug 2020 09:18:54 -0400 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <2D6CC614-0C5E-49C0-87E1-4FFFE2788DAD@azul.com> References: <70A3B5D4-3D0E-426C-8EB3-64CB7A42EC50@oracle.com> <2267CBC7-85A9-4105-B405-E7499440B985@azul.com> <8885AB2E-0671-4493-A2D6-88302C48CAF9@oracle.com> <070d2335-32d1-d4f6-892f-1a28c6649a31@oracle.com> <85c2e4b8-f61e-920c-5afe-5e3ed3df5c45@oracle.com> <643f3744-cfa2-e871-1284-900d81709f53@oracle.com> <77fb8b44-2438-2f62-07b6-45c12a9c6f34@oracle.com> <7621fa33-26cc-d545-cb33-a93fdbfd48c7@oracle.com> <8804341B-E534-4FA8-A652-86F8A30F18B3@azul.com> <999EF4D0-4084-4EC2-803C-EDC23D3EB8F2@azul.com> <5b7281d3-4bfc-f0ac-07d3-365451835f70@oracle.com> <2D6CC614-0C5E-49C0-87E1-4FFFE2788DAD@azul.com> Message-ID: <55679fee-cc49-9437-d1e5-dc696b9b3a08@oracle.com> On 8/14/20 8:41 AM, Alexey Bakhtin wrote: > Hello Sean, > > Thank you for review. > I?ve changed wording and replaced @code with @systemProperty (tested, it works for module-info.java) Thanks. Now that you know it works, I think you should change the other properties documented in that file to @systemProperty to be consistent. I think it is fine to do that as part of this fix. --Sean > > Updated webrev at: http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v15/ > > Regards > Alexey > >> On 14 Aug 2020, at 14:50, Sean Mullan wrote: >> >> On the property wording, change "for LDAP connection" to "for an LDAP connection". >> >> Also, for the definition of the property, can you use the "@systemProperty" annotation instead of "@code"? Does that work inside the module-info.java file? >> >> I added my name as Reviewer. >> >> --Sean >> >> On 7/30/20 6:14 AM, Daniel Fuchs wrote: >>> Hi Alexey, >>> I have added myself as a reviewer to the CSR [1]. >>> It would be good to get someone from security-dev to do the >>> same, and then move the CSR state to "Proposed". >>> best regards, >>> -- daniel >>> [1] https://bugs.openjdk.java.net/browse/JDK-8247311 >>> On 30/07/2020 10:17, Alexey Bakhtin wrote: >>>> Gentle ping >>>> >>>> Regards >>>> Alexey > From alexey at azul.com Fri Aug 14 13:26:01 2020 From: alexey at azul.com (Alexey Bakhtin) Date: Fri, 14 Aug 2020 13:26:01 +0000 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <55679fee-cc49-9437-d1e5-dc696b9b3a08@oracle.com> References: <70A3B5D4-3D0E-426C-8EB3-64CB7A42EC50@oracle.com> <2267CBC7-85A9-4105-B405-E7499440B985@azul.com> <8885AB2E-0671-4493-A2D6-88302C48CAF9@oracle.com> <070d2335-32d1-d4f6-892f-1a28c6649a31@oracle.com> <85c2e4b8-f61e-920c-5afe-5e3ed3df5c45@oracle.com> <643f3744-cfa2-e871-1284-900d81709f53@oracle.com> <77fb8b44-2438-2f62-07b6-45c12a9c6f34@oracle.com> <7621fa33-26cc-d545-cb33-a93fdbfd48c7@oracle.com> <8804341B-E534-4FA8-A652-86F8A30F18B3@azul.com> <999EF4D0-4084-4EC2-803C-EDC23D3EB8F2@azul.com> <5b7281d3-4bfc-f0ac-07d3-365451835f70@oracle.com> <2D6CC614-0C5E-49C0-87E1-4FFFE2788DAD@azul.com> <55679fee-cc49-9437-d1e5-dc696b9b3a08@oracle.com> Message-ID: <3F4A52FB-B980-489D-9862-DB3FBB12B0E7@azul.com> OK Updated for all properties in the module-info.java Webrev at: http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v16/ Regards Alexey > On 14 Aug 2020, at 16:18, Sean Mullan wrote: > > On 8/14/20 8:41 AM, Alexey Bakhtin wrote: >> Hello Sean, >> Thank you for review. >> I?ve changed wording and replaced @code with @systemProperty (tested, it works for module-info.java) > > Thanks. Now that you know it works, I think you should change the other properties documented in that file to @systemProperty to be consistent. I think it is fine to do that as part of this fix. > > --Sean > >> Updated webrev at: http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v15/ >> Regards >> Alexey >>> On 14 Aug 2020, at 14:50, Sean Mullan wrote: >>> >>> On the property wording, change "for LDAP connection" to "for an LDAP connection". >>> >>> Also, for the definition of the property, can you use the "@systemProperty" annotation instead of "@code"? Does that work inside the module-info.java file? >>> >>> I added my name as Reviewer. >>> >>> --Sean >>> >>> On 7/30/20 6:14 AM, Daniel Fuchs wrote: >>>> Hi Alexey, >>>> I have added myself as a reviewer to the CSR [1]. >>>> It would be good to get someone from security-dev to do the >>>> same, and then move the CSR state to "Proposed". >>>> best regards, >>>> -- daniel >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8247311 >>>> On 30/07/2020 10:17, Alexey Bakhtin wrote: >>>>> Gentle ping >>>>> >>>>> Regards >>>>> Alexey From aleksej.efimov at oracle.com Fri Aug 14 14:04:00 2020 From: aleksej.efimov at oracle.com (Aleks Efimov) Date: Fri, 14 Aug 2020 15:04:00 +0100 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <3F4A52FB-B980-489D-9862-DB3FBB12B0E7@azul.com> References: <8885AB2E-0671-4493-A2D6-88302C48CAF9@oracle.com> <070d2335-32d1-d4f6-892f-1a28c6649a31@oracle.com> <85c2e4b8-f61e-920c-5afe-5e3ed3df5c45@oracle.com> <643f3744-cfa2-e871-1284-900d81709f53@oracle.com> <77fb8b44-2438-2f62-07b6-45c12a9c6f34@oracle.com> <7621fa33-26cc-d545-cb33-a93fdbfd48c7@oracle.com> <8804341B-E534-4FA8-A652-86F8A30F18B3@azul.com> <999EF4D0-4084-4EC2-803C-EDC23D3EB8F2@azul.com> <5b7281d3-4bfc-f0ac-07d3-365451835f70@oracle.com> <2D6CC614-0C5E-49C0-87E1-4FFFE2788DAD@azul.com> <55679fee-cc49-9437-d1e5-dc696b9b3a08@oracle.com> <3F4A52FB-B980-489D-9862-DB3FBB12B0E7@azul.com> Message-ID: <435c24f3-1c3f-4938-e26e-c18500b8f235@oracle.com> Hi Sean, Alexey, Adding @systemProperty doesn't look correct here since the properties mentioned in the module-info.java are not system properties, they're standard JNDI environment properties and setting them via system property with the same name won't have any effect, e.g. "java -Dcom.sun.jndi.ldap.tls.cbtype=tls-server-end-point". Best Regards, Aleksei On 14/08/2020 14:26, Alexey Bakhtin wrote: > OK > > Updated for all properties in the module-info.java > Webrev at: http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v16/ > > Regards > Alexey > >> On 14 Aug 2020, at 16:18, Sean Mullan wrote: >> >> On 8/14/20 8:41 AM, Alexey Bakhtin wrote: >>> Hello Sean, >>> Thank you for review. >>> I?ve changed wording and replaced @code with @systemProperty (tested, it works for module-info.java) >> Thanks. Now that you know it works, I think you should change the other properties documented in that file to @systemProperty to be consistent. I think it is fine to do that as part of this fix. >> >> --Sean >> >>> Updated webrev at: http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v15/ >>> Regards >>> Alexey >>>> On 14 Aug 2020, at 14:50, Sean Mullan wrote: >>>> >>>> On the property wording, change "for LDAP connection" to "for an LDAP connection". >>>> >>>> Also, for the definition of the property, can you use the "@systemProperty" annotation instead of "@code"? Does that work inside the module-info.java file? >>>> >>>> I added my name as Reviewer. >>>> >>>> --Sean >>>> >>>> On 7/30/20 6:14 AM, Daniel Fuchs wrote: >>>>> Hi Alexey, >>>>> I have added myself as a reviewer to the CSR [1]. >>>>> It would be good to get someone from security-dev to do the >>>>> same, and then move the CSR state to "Proposed". >>>>> best regards, >>>>> -- daniel >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8247311 >>>>> On 30/07/2020 10:17, Alexey Bakhtin wrote: >>>>>> Gentle ping >>>>>> >>>>>> Regards >>>>>> Alexey From andy.herrick at oracle.com Fri Aug 14 14:05:41 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Fri, 14 Aug 2020 10:05:41 -0400 Subject: RFR: JDK-8250611: Cannot display splash screen on Windows In-Reply-To: <5F35B8CD.3000509@oracle.com> References: <765b70c2-ec7e-475d-cafe-d646847f5d2b@oracle.com> <5F35B8CD.3000509@oracle.com> Message-ID: Please review revised webrev [3] that does not check for client jvm are (always use server) /Andy [3] - http://cr.openjdk.java.net/~herrick/8250611/webrev.02 On 8/13/2020 6:03 PM, Philip Race wrote: > Do I understand correctly that you are checking if the client VM is > being specified ? > I didn't think we had client VMs any more .. > > -phil. > > On 8/13/20, 12:35 PM, Andy Herrick wrote: >> Please review this jpackage fix for issue [1] at [2]. >> >> In order to show splash screen from statically linked applauncher, we >> need to load the dependent libraries of splashscreen.dll first >> >> /Andy >> >> [1] - https://bugs.openjdk.java.net/browse/JDK-8250611 >> >> [2] - http://cr.openjdk.java.net/~herrick/8250611/webrev.01/ >> From daniel.fuchs at oracle.com Fri Aug 14 14:18:54 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 14 Aug 2020 15:18:54 +0100 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <55679fee-cc49-9437-d1e5-dc696b9b3a08@oracle.com> References: <2267CBC7-85A9-4105-B405-E7499440B985@azul.com> <8885AB2E-0671-4493-A2D6-88302C48CAF9@oracle.com> <070d2335-32d1-d4f6-892f-1a28c6649a31@oracle.com> <85c2e4b8-f61e-920c-5afe-5e3ed3df5c45@oracle.com> <643f3744-cfa2-e871-1284-900d81709f53@oracle.com> <77fb8b44-2438-2f62-07b6-45c12a9c6f34@oracle.com> <7621fa33-26cc-d545-cb33-a93fdbfd48c7@oracle.com> <8804341B-E534-4FA8-A652-86F8A30F18B3@azul.com> <999EF4D0-4084-4EC2-803C-EDC23D3EB8F2@azul.com> <5b7281d3-4bfc-f0ac-07d3-365451835f70@oracle.com> <2D6CC614-0C5E-49C0-87E1-4FFFE2788DAD@azul.com> <55679fee-cc49-9437-d1e5-dc696b9b3a08@oracle.com> Message-ID: <538c29b1-8c12-d410-85a8-43e3ba1665cd@oracle.com> Hi Sean, Wait wait... Are they system properties really? Only system properties should be documented with @systemProperty. I can't find the place were com.sun.jndi.ldap.connect.timeout or com.sun.jndi.ldap.read.timeout is read from System. I believe they are just plain environment properties that you need to specify in the environment map. best regards, -- daniel On 14/08/2020 14:18, Sean Mullan wrote: > On 8/14/20 8:41 AM, Alexey Bakhtin wrote: >> Hello Sean, >> >> Thank you for review. >> I?ve changed wording and replaced @code with @systemProperty (tested, >> it works for module-info.java) > > Thanks. Now that you know it works, I think you should change the other > properties documented in that file to @systemProperty to be consistent. > I think it is fine to do that as part of this fix. > > --Sean From sean.mullan at oracle.com Fri Aug 14 14:23:41 2020 From: sean.mullan at oracle.com (Sean Mullan) Date: Fri, 14 Aug 2020 10:23:41 -0400 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <538c29b1-8c12-d410-85a8-43e3ba1665cd@oracle.com> References: <8885AB2E-0671-4493-A2D6-88302C48CAF9@oracle.com> <070d2335-32d1-d4f6-892f-1a28c6649a31@oracle.com> <85c2e4b8-f61e-920c-5afe-5e3ed3df5c45@oracle.com> <643f3744-cfa2-e871-1284-900d81709f53@oracle.com> <77fb8b44-2438-2f62-07b6-45c12a9c6f34@oracle.com> <7621fa33-26cc-d545-cb33-a93fdbfd48c7@oracle.com> <8804341B-E534-4FA8-A652-86F8A30F18B3@azul.com> <999EF4D0-4084-4EC2-803C-EDC23D3EB8F2@azul.com> <5b7281d3-4bfc-f0ac-07d3-365451835f70@oracle.com> <2D6CC614-0C5E-49C0-87E1-4FFFE2788DAD@azul.com> <55679fee-cc49-9437-d1e5-dc696b9b3a08@oracle.com> <538c29b1-8c12-d410-85a8-43e3ba1665cd@oracle.com> Message-ID: <12a66244-2215-f947-c1e4-0877ef39f807@oracle.com> Sorry you are right! Would be nice to have a more general @property for these types of properties and security properties, etc but that's a different change ... --Sean On 8/14/20 10:18 AM, Daniel Fuchs wrote: > Hi Sean, > > Wait wait... Are they system properties really? > > Only system properties should be documented with @systemProperty. > I can't find the place were com.sun.jndi.ldap.connect.timeout or > com.sun.jndi.ldap.read.timeout is read from System. > > I believe they are just plain environment properties > that you need to specify in the environment map. > > best regards, > > -- daniel > > On 14/08/2020 14:18, Sean Mullan wrote: >> On 8/14/20 8:41 AM, Alexey Bakhtin wrote: >>> Hello Sean, >>> >>> Thank you for review. >>> I?ve changed wording and replaced @code with @systemProperty (tested, >>> it works for module-info.java) >> >> Thanks. Now that you know it works, I think you should change the >> other properties documented in that file to @systemProperty to be >> consistent. I think it is fine to do that as part of this fix. >> >> --Sean From alexey at azul.com Fri Aug 14 14:42:26 2020 From: alexey at azul.com (Alexey Bakhtin) Date: Fri, 14 Aug 2020 14:42:26 +0000 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <12a66244-2215-f947-c1e4-0877ef39f807@oracle.com> References: <8885AB2E-0671-4493-A2D6-88302C48CAF9@oracle.com> <070d2335-32d1-d4f6-892f-1a28c6649a31@oracle.com> <85c2e4b8-f61e-920c-5afe-5e3ed3df5c45@oracle.com> <643f3744-cfa2-e871-1284-900d81709f53@oracle.com> <77fb8b44-2438-2f62-07b6-45c12a9c6f34@oracle.com> <7621fa33-26cc-d545-cb33-a93fdbfd48c7@oracle.com> <8804341B-E534-4FA8-A652-86F8A30F18B3@azul.com> <999EF4D0-4084-4EC2-803C-EDC23D3EB8F2@azul.com> <5b7281d3-4bfc-f0ac-07d3-365451835f70@oracle.com> <2D6CC614-0C5E-49C0-87E1-4FFFE2788DAD@azul.com> <55679fee-cc49-9437-d1e5-dc696b9b3a08@oracle.com> <538c29b1-8c12-d410-85a8-43e3ba1665cd@oracle.com> <12a66244-2215-f947-c1e4-0877ef39f807@oracle.com> Message-ID: <289A4001-F580-40EF-A337-FA9EB3637D2E@azul.com> Sorry, That?s my bad. I have reverted @systemProperty to @code. The wording is fixed. Webrev: http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v17/ Regards Alexey > On 14 Aug 2020, at 17:23, Sean Mullan wrote: > > Sorry you are right! Would be nice to have a more general @property for these types of properties and security properties, etc but that's a different change ... > > --Sean > > On 8/14/20 10:18 AM, Daniel Fuchs wrote: >> Hi Sean, >> Wait wait... Are they system properties really? >> Only system properties should be documented with @systemProperty. >> I can't find the place were com.sun.jndi.ldap.connect.timeout or >> com.sun.jndi.ldap.read.timeout is read from System. >> I believe they are just plain environment properties >> that you need to specify in the environment map. >> best regards, >> -- daniel >> On 14/08/2020 14:18, Sean Mullan wrote: >>> On 8/14/20 8:41 AM, Alexey Bakhtin wrote: >>>> Hello Sean, >>>> >>>> Thank you for review. >>>> I?ve changed wording and replaced @code with @systemProperty (tested, it works for module-info.java) >>> >>> Thanks. Now that you know it works, I think you should change the other properties documented in that file to @systemProperty to be consistent. I think it is fine to do that as part of this fix. >>> >>> --Sean From daniel.fuchs at oracle.com Fri Aug 14 14:50:31 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 14 Aug 2020 15:50:31 +0100 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <289A4001-F580-40EF-A337-FA9EB3637D2E@azul.com> References: <070d2335-32d1-d4f6-892f-1a28c6649a31@oracle.com> <85c2e4b8-f61e-920c-5afe-5e3ed3df5c45@oracle.com> <643f3744-cfa2-e871-1284-900d81709f53@oracle.com> <77fb8b44-2438-2f62-07b6-45c12a9c6f34@oracle.com> <7621fa33-26cc-d545-cb33-a93fdbfd48c7@oracle.com> <8804341B-E534-4FA8-A652-86F8A30F18B3@azul.com> <999EF4D0-4084-4EC2-803C-EDC23D3EB8F2@azul.com> <5b7281d3-4bfc-f0ac-07d3-365451835f70@oracle.com> <2D6CC614-0C5E-49C0-87E1-4FFFE2788DAD@azul.com> <55679fee-cc49-9437-d1e5-dc696b9b3a08@oracle.com> <538c29b1-8c12-d410-85a8-43e3ba1665cd@oracle.com> <12a66244-2215-f947-c1e4-0877ef39f807@oracle.com> <289A4001-F580-40EF-A337-FA9EB3637D2E@azul.com> Message-ID: <82f53ce4-22fb-5232-2b13-257ca154df4f@oracle.com> Hi Alexey, LGTM! Thanks, -- daniel On 14/08/2020 15:42, Alexey Bakhtin wrote: > Sorry, > That?s my bad. > I have reverted @systemProperty to @code. The wording is fixed. > > Webrev: > http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v17/ > > Regards > Alexey From aleksej.efimov at oracle.com Fri Aug 14 15:01:11 2020 From: aleksej.efimov at oracle.com (Aleks Efimov) Date: Fri, 14 Aug 2020 16:01:11 +0100 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <289A4001-F580-40EF-A337-FA9EB3637D2E@azul.com> References: <070d2335-32d1-d4f6-892f-1a28c6649a31@oracle.com> <85c2e4b8-f61e-920c-5afe-5e3ed3df5c45@oracle.com> <643f3744-cfa2-e871-1284-900d81709f53@oracle.com> <77fb8b44-2438-2f62-07b6-45c12a9c6f34@oracle.com> <7621fa33-26cc-d545-cb33-a93fdbfd48c7@oracle.com> <8804341B-E534-4FA8-A652-86F8A30F18B3@azul.com> <999EF4D0-4084-4EC2-803C-EDC23D3EB8F2@azul.com> <5b7281d3-4bfc-f0ac-07d3-365451835f70@oracle.com> <2D6CC614-0C5E-49C0-87E1-4FFFE2788DAD@azul.com> <55679fee-cc49-9437-d1e5-dc696b9b3a08@oracle.com> <538c29b1-8c12-d410-85a8-43e3ba1665cd@oracle.com> <12a66244-2215-f947-c1e4-0877ef39f807@oracle.com> <289A4001-F580-40EF-A337-FA9EB3637D2E@azul.com> Message-ID: Hi Alexey, The last version looks good to me. Best, Aleksei On 14/08/2020 15:42, Alexey Bakhtin wrote: > Sorry, > That?s my bad. > I have reverted @systemProperty to @code. The wording is fixed. > > Webrev: > http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v17/ > > Regards > Alexey > >> On 14 Aug 2020, at 17:23, Sean Mullan wrote: >> >> Sorry you are right! Would be nice to have a more general @property for these types of properties and security properties, etc but that's a different change ... >> >> --Sean >> >> On 8/14/20 10:18 AM, Daniel Fuchs wrote: >>> Hi Sean, >>> Wait wait... Are they system properties really? >>> Only system properties should be documented with @systemProperty. >>> I can't find the place were com.sun.jndi.ldap.connect.timeout or >>> com.sun.jndi.ldap.read.timeout is read from System. >>> I believe they are just plain environment properties >>> that you need to specify in the environment map. >>> best regards, >>> -- daniel >>> On 14/08/2020 14:18, Sean Mullan wrote: >>>> On 8/14/20 8:41 AM, Alexey Bakhtin wrote: >>>>> Hello Sean, >>>>> >>>>> Thank you for review. >>>>> I?ve changed wording and replaced @code with @systemProperty (tested, it works for module-info.java) >>>> Thanks. Now that you know it works, I think you should change the other properties documented in that file to @systemProperty to be consistent. I think it is fine to do that as part of this fix. >>>> >>>> --Sean From raffaello.giulietti at gmail.com Fri Aug 14 15:38:58 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Fri, 14 Aug 2020 17:38:58 +0200 Subject: contributing to JDK-8233760 Result of BigDecimal.toString throws overflow exception on new BigDecimal(str) Message-ID: <6963cdd4-5f5a-fa7c-d319-a5428bad8158@gmail.com> Hi, if nobody else is already fixing JDK-8233760 [1], I would like to start digging into it, as it still affects a recent changeset [2]. Greetings Raffaello ---- [1] https://bugs.openjdk.java.net/browse/JDK-8233760 [2] http://hg.openjdk.java.net/jdk/jdk/rev/78705826c520 From andy.herrick at oracle.com Fri Aug 14 16:28:02 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Fri, 14 Aug 2020 12:28:02 -0400 Subject: RFR: 8250803: pkgbuild failed with exit code 134 In-Reply-To: References: Message-ID: <51203ce1-1304-c602-4f5d-6d8e227aeb9f@oracle.com> looks good. /Andy On 8/13/2020 4:13 PM, alexander.matveev at oracle.com wrote: > Please review the jpackage fix for bug [1] at [2]. > > - Looks like issue was just in not enough time for test to execute. > Increasing timeout by 1 min fixed issues. In fix it is increased by > 50% just in case. Without timeout increase it was reproducible 100% on > one of test machine. With increased timeout I cannot reproduce it > after multiple runs. > > [1] https://bugs.openjdk.java.net/browse/JDK-8250803 > [2] http://cr.openjdk.java.net/~almatvee/8250803/webrev.00/ > > Thanks, > Alexander From mandy.chung at oracle.com Fri Aug 14 16:58:46 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 14 Aug 2020 09:58:46 -0700 Subject: 8251160: Fix "no comment" warnings in java.logging In-Reply-To: References: <78d70e3e-fefb-2f24-a114-81b250c5e8c3@oracle.com> Message-ID: On 8/14/20 2:17 AM, Daniel Fuchs wrote: > new webrev: > http://cr.openjdk.java.net/~dfuchs/webrev_8251160/webrev.01/index.html > +1 > CSR updated as well: > https://bugs.openjdk.java.net/browse/JDK-8251534 > Reviewed. Mandy From huizhe.wang at oracle.com Fri Aug 14 17:20:38 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Fri, 14 Aug 2020 10:20:38 -0700 Subject: RFR 8251208: Fix javadoc warnings in java.sql and java.sql.rowsets In-Reply-To: <9C459B41-BD59-44C9-9270-18A9E81882C8@ORACLE.COM> References: <9C459B41-BD59-44C9-9270-18A9E81882C8@ORACLE.COM> Message-ID: Hi Lance, Looks good to me overall. * **Minor typos in the CSR:* Address the Fix "no comment" warnings in java.sql and java.sql.rowsetsgenerated by javadoc -Xdoclint ??????????? ^ remove Fix????????????????????????????????????????????? ^ missing a spacebetween rowsetsgenerated java.sql and java.sql.rowset contain several iclasses ???????????????????????????????????????????? classes Compatibility Risk: the last word "clas" -> class *Webrev:* Need to update copyright year for classes: SerialRef.java, SerialStruct.java, SQLClientInfoException.java StatementEvent.java: the 1st statement (line 47) was supposed for the 2nd field (line55), while the 2nd (line 52) for the 1st (line 49) :-) ??????????? e.g.: ?46???????? /** 47????????? * The {@code PreparedStatement} that is being closed or is invalid. ?48????????? */ ?49???????? private SQLException??????????? exception; ?50 ?51???????? /** ?52????????? * The {@code SQLException} the driver is about to throw to the application. ?53????????? */ ?54???????? @SuppressWarnings("serial") // Not statically typed as Serializable ?55???????? private PreparedStatement?????? statement; -Joe On 8/14/2020 3:37 AM, Lance Andersen wrote: > Hi all, > > Please review the fix to address javadoc warnings in java.sql and java.sql.rowsets > > The webrev can be found at: http://cr.openjdk.java.net/~lancea/8251208/webrev.00/ > > And the CSR at: https://bugs.openjdk.java.net/browse/JDK-8251834 > > Best > Lance > ------------------ > > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > > From LANCE.ANDERSEN at ORACLE.COM Fri Aug 14 17:43:32 2020 From: LANCE.ANDERSEN at ORACLE.COM (Lance Andersen) Date: Fri, 14 Aug 2020 13:43:32 -0400 Subject: RFR 8251208: Fix javadoc warnings in java.sql and java.sql.rowsets In-Reply-To: References: <9C459B41-BD59-44C9-9270-18A9E81882C8@ORACLE.COM> Message-ID: Hi Joe, Thank you for the review. Changes had been made to the typos below (went back to far in IntelliJ fixing another issue :-( ) CSR has also been updated to fix the typos (missed I guess due to lack of coffee this am ;-) ) http://cr.openjdk.java.net/~lancea/8251208/webrev.01/index.html is the updated webrev Best Lance > On Aug 14, 2020, at 1:20 PM, Joe Wang wrote: > > Hi Lance, > > Looks good to me overall. > > Minor typos in the CSR: > Address the Fix "no comment" warnings in java.sql and java.sql.rowsetsgenerated by javadoc -Xdoclint > ^ remove Fix ^ missing a space between rowsetsgenerated > > java.sql and java.sql.rowset contain several iclasses > classes > > Compatibility Risk: the last word "clas" -> class > > Webrev: > Need to update copyright year for classes: SerialRef.java, SerialStruct.java, SQLClientInfoException.java > > StatementEvent.java: the 1st statement (line 47) was supposed for the 2nd field (line55), while the 2nd (line 52) for the 1st (line 49) :-) > e.g.: > 46 /** > 47 * The {@code PreparedStatement} that is being closed or is invalid. > 48 */ > 49 private SQLException exception; > 50 > 51 /** > 52 * The {@code SQLException} the driver is about to throw to the application. > 53 */ > 54 @SuppressWarnings("serial") // Not statically typed as Serializable > 55 private PreparedStatement statement; > > > -Joe > > On 8/14/2020 3:37 AM, Lance Andersen wrote: >> Hi all, >> >> Please review the fix to address javadoc warnings in java.sql and java.sql.rowsets >> >> The webrev can be found at: http://cr.openjdk.java.net/~lancea/8251208/webrev.00/ >> >> And the CSR at: https://bugs.openjdk.java.net/browse/JDK-8251834 >> >> Best >> Lance >> ------------------ >> >> >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> >> >> > Best Lance ------------------ Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From philip.race at oracle.com Fri Aug 14 17:49:38 2020 From: philip.race at oracle.com (Philip Race) Date: Fri, 14 Aug 2020 10:49:38 -0700 Subject: RFR: JDK-8250611: Cannot display splash screen on Windows In-Reply-To: References: <765b70c2-ec7e-475d-cafe-d646847f5d2b@oracle.com> <5F35B8CD.3000509@oracle.com> Message-ID: <5F36CEB2.40003@oracle.com> +1 -phil On 8/14/20, 7:05 AM, Andy Herrick wrote: > Please review revised webrev [3] that does not check for client jvm > are (always use server) > > /Andy > > [3] - http://cr.openjdk.java.net/~herrick/8250611/webrev.02 > > On 8/13/2020 6:03 PM, Philip Race wrote: >> Do I understand correctly that you are checking if the client VM is >> being specified ? >> I didn't think we had client VMs any more .. >> >> -phil. >> >> On 8/13/20, 12:35 PM, Andy Herrick wrote: >>> Please review this jpackage fix for issue [1] at [2]. >>> >>> In order to show splash screen from statically linked applauncher, >>> we need to load the dependent libraries of splashscreen.dll first >>> >>> /Andy >>> >>> [1] - https://bugs.openjdk.java.net/browse/JDK-8250611 >>> >>> [2] - http://cr.openjdk.java.net/~herrick/8250611/webrev.01/ >>> From huizhe.wang at oracle.com Fri Aug 14 17:52:36 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Fri, 14 Aug 2020 10:52:36 -0700 Subject: RFR 8251208: Fix javadoc warnings in java.sql and java.sql.rowsets In-Reply-To: References: <9C459B41-BD59-44C9-9270-18A9E81882C8@ORACLE.COM> Message-ID: Looks good, Lance. Nothing can't be solved with a cup of coffee ;-) Best, Joe On 8/14/20 10:43 AM, Lance Andersen wrote: > Hi Joe, > > Thank you for the review. > > Changes had been made to the typos below (went back to far in IntelliJ > fixing another issue :-( ) > > CSR has also been updated to fix the typos (missed I guess due to lack > of coffee this am ;-) ) > > http://cr.openjdk.java.net/~lancea/8251208/webrev.01/index.html is the > updated webrev > > Best > Lance > >> On Aug 14, 2020, at 1:20 PM, Joe Wang > > wrote: >> >> Hi Lance, >> >> Looks good to me overall. >> * >> **Minor typos in the CSR:* >> Address the Fix "no comment" warnings in java.sql and >> java.sql.rowsetsgenerated by javadoc -Xdoclint >> ??????????? ^ remove Fix????????????????????????????????????????????? >> ^ missing a spacebetween rowsetsgenerated >> >> java.sql and java.sql.rowset contain several iclasses >> classes >> >> Compatibility Risk: the last word "clas" -> class >> >> *Webrev:* >> Need to update copyright year for classes: SerialRef.java, >> SerialStruct.java, SQLClientInfoException.java >> >> StatementEvent.java: the 1st statement (line 47) was supposed for the >> 2nd field (line55), while the 2nd (line 52) for the 1st (line 49) :-) >> ??????????? e.g.: >> ?46???????? /** >> 47 * The {@code PreparedStatement} that is being closed or is invalid. >> ?48????????? */ >> ?49 private SQLException??????????? exception; >> ?50 >> ?51???????? /** >> ?52 * The {@code SQLException} the driver is about to throw to the >> application. >> ?53????????? */ >> ?54 @SuppressWarnings("serial") // Not statically typed as Serializable >> ?55 private PreparedStatement?????? statement; >> >> >> -Joe >> >> On 8/14/2020 3:37 AM, Lance Andersen wrote: >>> Hi all, >>> >>> Please review the fix to address javadoc warnings in java.sql and java.sql.rowsets >>> >>> The webrev can be found at:http://cr.openjdk.java.net/~lancea/8251208/webrev.00/ >>> >>> And the CSR at:https://bugs.openjdk.java.net/browse/JDK-8251834 >>> >>> Best >>> Lance >>> ------------------ >>> >>> >>> >>> >>> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>> Oracle Java Engineering >>> 1 Network Drive >>> Burlington, MA 01803 >>> Lance.Andersen at oracle.com >>> >>> >>> >>> >> > > > Best > Lance > ------------------ > > > > Lance Andersen| Principal?Member of Technical Staff |?+1.781.442.2037 > Oracle?Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > > From vipinsharma85 at gmail.com Fri Aug 14 18:46:40 2020 From: vipinsharma85 at gmail.com (Vipin Sharma) Date: Sat, 15 Aug 2020 00:16:40 +0530 Subject: Fix for Javadoc errors in java.base In-Reply-To: <11adede2-1119-f133-020f-4ddd60c9fc67@oracle.com> References: <318a4bbf-1b3e-d34e-d69a-d1a2ee233794@oracle.com> <8e978983-4ec1-8421-2563-d00c61f0d55e@oracle.com> <11adede2-1119-f133-020f-4ddd60c9fc67@oracle.com> Message-ID: Hi Sean, All 3 instances of @exception in DHPrivateKey are now replaced with @throws, added ProviderException similar to other constructor of DHPrivateKey. Updated patch: --- old/src/java.base/share/classes/com/sun/crypto/provider/DHPrivateKey.java 2020-08-14 23:55:40.921599987 +0530 +++ new/src/java.base/share/classes/com/sun/crypto/provider/DHPrivateKey.java 2020-08-14 23:55:40.417581423 +0530 @@ -80,7 +80,7 @@ * @param p the prime modulus * @param g the base generator * - * @exception ProviderException if the key cannot be encoded + * @throws ProviderException if the key cannot be encoded */ DHPrivateKey(BigInteger x, BigInteger p, BigInteger g) throws InvalidKeyException { @@ -97,7 +97,7 @@ * @param g the base generator * @param l the private-value length * - * @exception InvalidKeyException if the key cannot be encoded + * @throws ProviderException if the key cannot be encoded */ DHPrivateKey(BigInteger x, BigInteger p, BigInteger g, int l) { this.x = x; @@ -118,7 +118,7 @@ * * @param encodedKey the encoded key * - * @exception InvalidKeyException if the encoded key does not represent + * @throws InvalidKeyException if the encoded key does not represent * a Diffie-Hellman private key */ DHPrivateKey(byte[] encodedKey) throws InvalidKeyException { --- old/src/java.base/share/classes/java/lang/invoke/AbstractValidatingLambdaMetafactory.java 2020-08-14 23:55:41.953638446 +0530 +++ new/src/java.base/share/classes/java/lang/invoke/AbstractValidatingLambdaMetafactory.java 2020-08-14 23:55:41.445619497 +0530 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2012, 2013, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2012, 2020, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -208,7 +208,7 @@ * * @return a CallSite, which, when invoked, will return an instance of the * functional interface - * @throws ReflectiveOperationException + * @throws LambdaConversionException */ abstract CallSite buildCallSite() throws LambdaConversionException; --- old/src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java 2020-08-14 23:55:42.977677096 +0530 +++ new/src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java 2020-08-14 23:55:42.473658062 +0530 @@ -200,7 +200,6 @@ * * @return a CallSite, which, when invoked, will return an instance of the * functional interface - * @throws ReflectiveOperationException * @throws LambdaConversionException If properly formed functional interface * is not found */ --- old/src/java.base/share/classes/java/math/BigDecimal.java 2020-08-14 23:55:44.409731999 +0530 +++ new/src/java.base/share/classes/java/math/BigDecimal.java 2020-08-14 23:55:43.889711942 +0530 @@ -5403,7 +5403,7 @@ * * @param n the numerator; must be negative * @param d the denominator; must not be unity - * @return a two-element {@long} array with the remainder and quotient in + * @return a two-element {@code long} array with the remainder and quotient in * the initial and final elements, respectively */ private static long[] divRemNegativeLong(long n, long d) { --- old/src/java.base/share/classes/java/math/MutableBigInteger.java 2020-08-14 23:55:45.905790246 +0530 +++ new/src/java.base/share/classes/java/math/MutableBigInteger.java 2020-08-14 23:55:45.397770378 +0530 @@ -145,7 +145,6 @@ * Makes this number an {@code n}-int number all of whose bits are ones. * Used by Burnikel-Ziegler division. * @param n number of ints in the {@code value} array - * @return a number equal to {@code ((1<<(32*n)))-1} */ private void ones(int n) { if (n > value.length) --- old/src/java.base/share/classes/java/net/ServerSocket.java 2020-08-14 23:55:46.961831962 +0530 +++ new/src/java.base/share/classes/java/net/ServerSocket.java 2020-08-14 23:55:46.449811716 +0530 @@ -1,5 +1,5 @@ /* - * Copyright (c) 1995, 2019, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1995, 2020, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -315,7 +315,7 @@ /** * Creates the socket implementation. * - * @throws IOException if creation fails + * @throws SocketException if creation fails * @since 1.4 */ void createImpl() throws SocketException { --- old/src/java.base/share/classes/java/net/Socket.java 2020-08-14 23:55:48.001873514 +0530 +++ new/src/java.base/share/classes/java/net/Socket.java 2020-08-14 23:55:47.493853209 +0530 @@ -533,7 +533,7 @@ * * @param stream a {@code boolean} value : {@code true} for a TCP socket, * {@code false} for UDP. - * @throws IOException if creation fails + * @throws SocketException if creation fails * @since 1.4 */ void createImpl(boolean stream) throws SocketException { --- old/src/java.base/share/classes/sun/reflect/annotation/AnnotationParser.java 2020-08-14 23:55:49.061916354 +0530 +++ new/src/java.base/share/classes/sun/reflect/annotation/AnnotationParser.java 2020-08-14 23:55:48.549895649 +0530 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2003, 2013, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2003, 2020, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -773,7 +773,7 @@ * an "annotation structure" OR two bytes into an annotation * structure (i.e., after the type index). * - * @parameter complete true if the byte buffer points to the beginning + * @param complete true if the byte buffer points to the beginning * of an annotation structure (rather than two bytes in). */ private static void skipAnnotation(ByteBuffer buf, boolean complete) { Regards, Vipin > On Aug 13, 2020, at 10:55 PM, Sean Mullan wrote: > > On 8/13/20 1:21 PM, Jonathan Gibbons wrote: >>>>> --- old/src/java.base/share/classes/com/sun/crypto/provider/DHPrivateKey.java >>>>> 2020-07-25 23:46:21.233726447 +0530 >>>>> +++ new/src/java.base/share/classes/com/sun/crypto/provider/DHPrivateKey.java >>>>> 2020-07-25 23:46:20.721720857 +0530 >>>>> @@ -96,8 +96,6 @@ >>>>> * @param p the prime modulus >>>>> * @param g the base generator >>>>> * @param l the private-value length >>>>> - * >>>>> - * @exception InvalidKeyException if the key cannot be encoded >>> >>> This should actually remain, but it should be ProviderException which is a RuntimeException. See the other DHPrivateKey ctor as that specifies it correctly. >>> >>> --Sean >> I note the use of `@exception`, as compared to `@throws`, which is more common. >> Stats: >> `@exception` 7322 occurrences >> `@throws` 21173 occurrences >> That's probably too many `@exception` to clean up. :-( > > Right, that's probably a separate cleanup activity. However, if you want to change the 3 instances of @exception to @throws in DHPrivateKey, I'm fine with that. > > --Sean From naoto.sato at oracle.com Fri Aug 14 22:20:08 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Fri, 14 Aug 2020 15:20:08 -0700 Subject: RFR: 8251499: no-placeholder compact number patterns throw IllegalArgumentException Message-ID: <5b81c727-b1e0-0614-c68e-5568f98d9d6f@oracle.com> Hello, Please review the fix for the following issue: https://bugs.openjdk.java.net/browse/JDK-8251499 The proposed changeset is located at: https://cr.openjdk.java.net/~naoto/8251499/webrev.00/ The current implementation of CompactNumberFormat assumes that there is always the number placeholder part in compact patterns. This is not always true. In fact, upcoming CLDR 38 resurrects such patterns, so this fix is a precursor to support CLDR 38. Naoto From alexander.matveev at oracle.com Fri Aug 14 23:10:48 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Fri, 14 Aug 2020 16:10:48 -0700 Subject: RFR: JDK-8250611: Cannot display splash screen on Windows In-Reply-To: References: <765b70c2-ec7e-475d-cafe-d646847f5d2b@oracle.com> <5F35B8CD.3000509@oracle.com> Message-ID: <53207c32-28e8-cbe7-764a-cb6f787cdb4d@oracle.com> Hi Andy, Looks good. Thanks, Alexander On 8/14/20 7:05 AM, Andy Herrick wrote: > Please review revised webrev [3] that does not check for client jvm > are (always use server) > > /Andy > > [3] - http://cr.openjdk.java.net/~herrick/8250611/webrev.02 > > On 8/13/2020 6:03 PM, Philip Race wrote: >> Do I understand correctly that you are checking if the client VM is >> being specified ? >> I didn't think we had client VMs any more .. >> >> -phil. >> >> On 8/13/20, 12:35 PM, Andy Herrick wrote: >>> Please review this jpackage fix for issue [1] at [2]. >>> >>> In order to show splash screen from statically linked applauncher, >>> we need to load the dependent libraries of splashscreen.dll first >>> >>> /Andy >>> >>> [1] - https://bugs.openjdk.java.net/browse/JDK-8250611 >>> >>> [2] - http://cr.openjdk.java.net/~herrick/8250611/webrev.01/ >>> From huizhe.wang at oracle.com Sat Aug 15 01:21:15 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Fri, 14 Aug 2020 18:21:15 -0700 Subject: RFR: 8251499: no-placeholder compact number patterns throw IllegalArgumentException In-Reply-To: <5b81c727-b1e0-0614-c68e-5568f98d9d6f@oracle.com> References: <5b81c727-b1e0-0614-c68e-5568f98d9d6f@oracle.com> Message-ID: Hi Naoto, Looks good to me. While a negative divisor representing no zeros is newly introduced, the "divisor > 0" checks seem to have always been beneficial.? I had to count the number of ""s in COMPACT_PATTERN13 :-) Have a great weekend! Joe On 8/14/2020 3:20 PM, naoto.sato at oracle.com wrote: > Hello, > > Please review the fix for the following issue: > > https://bugs.openjdk.java.net/browse/JDK-8251499 > > The proposed changeset is located at: > > https://cr.openjdk.java.net/~naoto/8251499/webrev.00/ > > The current implementation of CompactNumberFormat assumes that there > is always the number placeholder part in compact patterns. This is not > always true. In fact, upcoming CLDR 38 resurrects such patterns, so > this fix is a precursor to support CLDR 38. > > Naoto From fw at deneb.enyo.de Sat Aug 15 07:00:58 2020 From: fw at deneb.enyo.de (Florian Weimer) Date: Sat, 15 Aug 2020 09:00:58 +0200 Subject: SoftCleanable and WeakCleanable In-Reply-To: <87k0yhv7xc.fsf@mid.deneb.enyo.de> (Florian Weimer's message of "Sun, 02 Aug 2020 09:11:27 +0200") References: <87lfiywwi0.fsf@mid.deneb.enyo.de> <87k0yhv7xc.fsf@mid.deneb.enyo.de> Message-ID: <87d03smlzp.fsf@mid.deneb.enyo.de> * Florian Weimer: > * Alan Bateman: > >> On 01/08/2020 10:23, Florian Weimer wrote: >>> Are jdk.internal.ref.SoftCleanable and jdk.internal.ref.WeakCleanable >>> actually used? >>> >>> CleanerTest rests them, but I don't see any other mentions of these >>> classes. > >> Do you mean used outside of the cleaner implementation? Maybe you are >> looking to change them to non-public? > > It's not clear to me how the cleaner implementation itself uses them. > java.lang.ref.Cleaner only needs PhantomCleanableRef. Any further comments on this topic? Thanks. From peter.levart at gmail.com Sat Aug 15 08:37:16 2020 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 15 Aug 2020 10:37:16 +0200 Subject: Possible subtle memory model error in ClassValue In-Reply-To: <22fc68d5-d501-9ce0-093c-7be82aed247a@redhat.com> References: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> <6f91ae57-8dae-be23-bab4-c47dbe8b95f7@redhat.com> <0c849c2a-0a3a-704a-3ee3-22b690aa0d34@redhat.com> <04069f52-c9ef-8fee-26c7-5fdb69e8860b@redhat.com> <22fc68d5-d501-9ce0-093c-7be82aed247a@redhat.com> Message-ID: <55a36ac1-2aa2-7898-e3c3-bc2edcc315e7@gmail.com> Hi, There might be a way to fix this NPE without adding additional memory fences. The CacheValueMap.cacheArray field is not final because it can change during lifetime of CacheValueMap - it holds an array of entries which can get resized (replaced with bigger array) which is performed while holding a lock on the CacheValueMap instance. So I thought: why couldn't the field be null initially and its value initialized in a similar way as it is replaced with bigger array now. If I got this correct, it would take the following changes: https://github.com/openjdk/jdk/pull/7 The fast-path now trades an explicit null check for a null check that throws NPE when dereferencing cache array, so it should probably have no effect on performance (benchmarks pending). So, what do you think? Regards, Peter On 8/12/20 11:08 AM, Andrew Dinn wrote: > On 11/08/2020 18:06, Hans Boehm wrote: >> I think the relevant statement is: >> >> "An address dependency between two reads generated by SVE vector load >> instructions does not contribute to the Dependency-ordered-before relation." >> >> This is only an issue if BOTH loads are SVE loads. In particular >> reference loads have to be vectorized for this to matter, which I expect >> is not the common situation. I have not explored this in great detail, >> but it should suffice to put fences between two dependent vector >> reference loads, and between a reference load and a dependent final >> field load. > Hmm, so this might perhaps be an issue with something like a deep copy > of an int[][], where loading of successive int[] references might be > vectorized using SVE instructions and processing of the contents of each > referenced int[] might also be similarly vectorized. In that case the > loading of the int contents would need to be ordered wrt the load of the > int[] references using a LoadLoad barrier? > > regards, > > > Andrew Dinn > ----------- > Red Hat Distinguished Engineer > Red Hat UK Ltd > Registered in England and Wales under Company Registration No. 03798903 > Directors: Michael Cunningham, Michael ("Mike") O'Neill > From peter.levart at gmail.com Sat Aug 15 09:13:10 2020 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 15 Aug 2020 11:13:10 +0200 Subject: Possible subtle memory model error in ClassValue In-Reply-To: <55a36ac1-2aa2-7898-e3c3-bc2edcc315e7@gmail.com> References: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> <6f91ae57-8dae-be23-bab4-c47dbe8b95f7@redhat.com> <0c849c2a-0a3a-704a-3ee3-22b690aa0d34@redhat.com> <04069f52-c9ef-8fee-26c7-5fdb69e8860b@redhat.com> <22fc68d5-d501-9ce0-093c-7be82aed247a@redhat.com> <55a36ac1-2aa2-7898-e3c3-bc2edcc315e7@gmail.com> Message-ID: <2da470e0-ff56-62c3-129e-1138635fc14d@gmail.com> On 8/15/20 10:37 AM, Peter Levart wrote: > > https://github.com/openjdk/jdk/pull/7 > Oops, I forgot to remove the redundant initialization of cacheArray field in constructor, which revealed another place in code where the check for null value has to be made. Here's a modified patch: https://github.com/openjdk/jdk/pull/9 Sorry for abusing GitHub pull request mechanism but I don't have bandwidth currently to clone the mercurial repository ;-) Regards, Peter From peter.levart at gmail.com Sat Aug 15 09:34:33 2020 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 15 Aug 2020 11:34:33 +0200 Subject: SoftCleanable and WeakCleanable In-Reply-To: <87d03smlzp.fsf@mid.deneb.enyo.de> References: <87lfiywwi0.fsf@mid.deneb.enyo.de> <87k0yhv7xc.fsf@mid.deneb.enyo.de> <87d03smlzp.fsf@mid.deneb.enyo.de> Message-ID: <264e355c-0bf9-7934-c5de-8727e4b9fbee@gmail.com> Hi, You are right. For public Cleanable API, only the PhantomCleanable is used currently. It is subclassed by CleanerImpl.PhantomCleanableRef to implement the public API and also a couple of specialized internal implementations. When the public Cleanable API was created, it was thought that there might be internal needs for Weak and Soft variants too. So these classes are still waiting for such use. Perhaps they could be retired and resurrected from VCS history when need arises. Regards, Peter On 8/15/20 9:00 AM, Florian Weimer wrote: > * Florian Weimer: > >> * Alan Bateman: >> >>> On 01/08/2020 10:23, Florian Weimer wrote: >>>> Are jdk.internal.ref.SoftCleanable and jdk.internal.ref.WeakCleanable >>>> actually used? >>>> >>>> CleanerTest rests them, but I don't see any other mentions of these >>>> classes. >>> Do you mean used outside of the cleaner implementation? Maybe you are >>> looking to change them to non-public? >> It's not clear to me how the cleaner implementation itself uses them. >> java.lang.ref.Cleaner only needs PhantomCleanableRef. > Any further comments on this topic? Thanks. From mandy.chung at oracle.com Sat Aug 15 18:29:54 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Sat, 15 Aug 2020 11:29:54 -0700 Subject: SoftCleanable and WeakCleanable In-Reply-To: <264e355c-0bf9-7934-c5de-8727e4b9fbee@gmail.com> References: <87lfiywwi0.fsf@mid.deneb.enyo.de> <87k0yhv7xc.fsf@mid.deneb.enyo.de> <87d03smlzp.fsf@mid.deneb.enyo.de> <264e355c-0bf9-7934-c5de-8727e4b9fbee@gmail.com> Message-ID: It's a fine idea to remove these 2 classes and resurrect them when it finds a need for it.? I will file a JBS issue. Mandy On 8/15/20 2:34 AM, Peter Levart wrote: > Hi, > > > You are right. For public Cleanable API, only the PhantomCleanable is > used currently. It is subclassed by CleanerImpl.PhantomCleanableRef to > implement the public API and also a couple of specialized internal > implementations. When the public Cleanable API was created, it was > thought that there might be internal needs for Weak and Soft variants > too. So these classes are still waiting for such use. Perhaps they > could be retired and resurrected from VCS history when need arises. > > > Regards, Peter > > > On 8/15/20 9:00 AM, Florian Weimer wrote: >> * Florian Weimer: >> >>> * Alan Bateman: >>> >>>> On 01/08/2020 10:23, Florian Weimer wrote: >>>>> Are jdk.internal.ref.SoftCleanable and jdk.internal.ref.WeakCleanable >>>>> actually used? >>>>> >>>>> CleanerTest rests them, but I don't see any other mentions of these >>>>> classes. >>>> Do you mean used outside of the cleaner implementation? Maybe you are >>>> looking to change them to non-public? >>> It's not clear to me how the cleaner implementation itself uses them. >>> java.lang.ref.Cleaner only needs PhantomCleanableRef. >> Any further comments on this topic?? Thanks. From ioi.lam at oracle.com Sun Aug 16 01:19:55 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Sat, 15 Aug 2020 18:19:55 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> Message-ID: To better capture what we're trying to do in this RFE, I've changed the RFE title (and the subject of this email thread) to https://bugs.openjdk.java.net/browse/JDK-8247536 Support for pre-generated java.lang.invoke classes in CDS static archive I also update the RFE Description to give an overview of the problem and the solution. The design document is also updated to reflect Yumin's latest implementation https://wiki.openjdk.java.net/display/HotSpot/JDK-8247536+Support+pre-generated+MethodHandle+LambdaForms+in+CDS Thanks - Ioi On 8/12/20 11:54 AM, calvin.cheung at oracle.com wrote: > Hi Yumin, > > I reviewed mostly the native code. Below are my comments: > > 1) classListParser.hpp > > 71?? bool??????????????? _lambda_format; > > The above name is too generic. How about _lambda_form or _is_lambda_form? > If you rename the above, please also rename the function which returns > the above bool. > > 2) jvm.cpp > > 3850 JVM_ENTRY(void, JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, > jstring line)) > > ignore -> ignored > > 3) jvm.hpp > > 210 JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line); > > Same comment as for jvm.cpp > > 4) metaspaceShared.cpp > > 2017?? size_t i = 0; > 2018?? while (i < size) { > 2019???? full_name[i++] = *start++; > 2020?? } > > Could the above be simplified to the following? > > ??? strncpy(full_name, start, size - 1); > > 2029?? char* class_name = get_full_class_name(path_name); > > Should os::free(class_name) be called before the function returns? > > 1870 static GrowableArray* lambda_list = NULL; > > The name lambda_list is a bit generic, how about lambda_form_list? > > 2112?????? lambda_list->append(parser.current_line()); > > Since parser.current_line() does a strdup, do those buffer need to be > freed after its use? (i.e. after the call to > regenerate_holder_classes()?) > > In MetaspaceShared::regenerate_holder_classes, before calling up to > java, I think it's better to strip the prefix "@lambda-form-invoker" > from the strings. So that the java > InvokerBytecodeGeneratorHelper.readTraceConfig method doesn't need to > handle it: > > ?143???????????????????? case "[LF_RESOLVE]": > ?144???????????????????? case InvokerBytecodeGenerator.CDS_LOG_PREFIX: > > 5) DumpSymbolAndStringTable.java > > 37???? private static final String my_string = > "DumpSymbolAndStringTable"; > > Unused variable? > > thanks, > > Calvin > > On 8/11/20 10:36 AM, Yumin Qi wrote: >> Forget to send to @core-lib-dev, the patch changed jdk code. >> >> >> Thanks >> >> Yumin >> >> >> >> -------- Forwarded Message -------- >> Subject:???? RFR: 8247536: Support pre-generated MethodHandle lambda >> forms in CDS >> Date:???? Tue, 11 Aug 2020 07:44:34 -0700 >> From:???? Yumin Qi >> To:???? hotspot-runtime-dev at openjdk.java.net >> >> >> >> Hi, Please reivew >> >> ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 >> ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ >> >> ? Summary: CDS does not archive pre-generated lambda form classes for >> method handles: >> >> [0.142s][info][class,load] >> java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: >> __JVM_LookupDefineClass__ >> >> The method handle resolution if not found in the holder class, a >> class with the method will be generated and loaded as vm internal >> created class and not archived like above. Those class names are >> mangled as combination of the class name and the class instance address. >> >> In this patch, collect those method holder class names and their >> associated methods' signatures when user specify DumpLoadedClassList, >> and record them in the log file in a format similar to the output >> format from traced by >> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use >> another name for CDS: >> -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. The >> line prefix also changed from "[LF_RESOLVE]" to >> "@lambda-invoke-handle".? At dump time, regenerate the holder class >> with those methods and replace the existing holder class and archived >> it. At runtime, the resolution of the holder class which contains >> those methods are loaded from CDS archive so avoid regenerating them >> again. The patch reorganized the code for Jlink plugin, so the new >> added InvokerBytecodeGeneratorHelper can be shared both by JLink >> plugin and CDS code. Also change made to the mangled generated class >> name at static dump, the class name combining the class name and a >> sequentially ordered number instead of the class instance address, >> which looks like a random. >> >> >> To use this feature, one can do the dump/run just like done for a >> static dump/run, so no extra steps required. >> >> ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp >> >> ????? DumpLoadedClassList will turn on >> -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true >> >> ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist >> -XX:SharedArchiveFile=my.jsa JavaApp >> >> ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp >> >> ? The performance on javac HelloWorld.java (javac-benchmark), no >> patch vs patch: >> >> ?? 1:?? 2689285002? 2641821474 (-47463528)????? ----???? 391.720 >> 382.990 ( -8.730)????? ---- >> ?? 2:?? 2687495085? 2632969688 (-54525397)????? ----???? 391.030 >> 381.480 ( -9.550)????? ----- >> ?? 3:?? 2694664066? 2636523114 (-58140952)????? -----??? 390.610 >> 382.550 ( -8.060)????? ---- >> ?? 4:?? 2686554164? 2639355233 (-47198931)????? ----???? 390.700 >> 383.390 ( -7.310)????? --- >> ?? 5:?? 2691072338? 2633016687 (-58055651)????? -----??? 388.990 >> 382.360 ( -6.630)????? --- >> ?? 6:?? 2684448174? 2644191854 (-40256320)????? ---????? 389.450 >> 382.990 ( -6.460)????? --- >> ?? 7:?? 2694921227? 2630505090 (-64416137)????? -----??? 389.300 >> 383.160 ( -6.140)????? --- >> ?? 8:?? 2685209712? 2639334320 (-45875392)????? ----???? 388.370 >> 381.060 ( -7.310)????? --- >> ?? 9:?? 2695885942? 2640618655 (-55267287)????? ----???? 389.560 >> 381.100 ( -8.460)????? ---- >> ? 10:?? 2689162942? 2635658943 (-53503999)????? ----???? 389.690 >> 379.110 (-10.580)????? ----- >> ============================================================ >> ??????? 2689866989? 2637396210 (-52470778)????? ----???? 389.941 >> 382.017 ( -7.924)????? ---- >> >> instr delta =??? -52470778??? -1.9507% >> >> time? delta =?????? -7.924 ms -2.0321% >> >> ? It will show much more improvement if compare with the default >> archive: >> >> ?????????? ? ? ? no patch??????????????????????? patch >> >> instr ??? ? ? 3,996,847,605??????? 3,320,928,995 >> >> time(s).???? 0.686835162 ? ? ? ?? 0.474933546 >> >> >> ? Tests: >> >> 1)jtreg on jdk/tools and hotspot/runtime local. >> >> 2) mach5 ter1,2 >> >> >> Thanks >> >> Yumin >> >> ? ` From ioi.lam at oracle.com Sun Aug 16 01:27:38 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Sat, 15 Aug 2020 18:27:38 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> Message-ID: <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> On 8/15/20 6:19 PM, Ioi Lam wrote: > To better capture what we're trying to do in this RFE, I've changed > the RFE title (and the subject of this email thread) to > > https://bugs.openjdk.java.net/browse/JDK-8247536 > Support for pre-generated java.lang.invoke classes in CDS static archive > > I also update the RFE Description to give an overview of the problem > and the solution. > > The design document is also updated to reflect Yumin's latest > implementation > Oops, sent too quickly. The design doc's title is also updated: https://wiki.openjdk.java.net/display/HotSpot/Support+for+pre-generated+java.lang.invoke+classes+in+CDS+archive > Thanks > - Ioi > > On 8/12/20 11:54 AM, calvin.cheung at oracle.com wrote: >> Hi Yumin, >> >> I reviewed mostly the native code. Below are my comments: >> >> 1) classListParser.hpp >> >> 71?? bool??????????????? _lambda_format; >> >> The above name is too generic. How about _lambda_form or >> _is_lambda_form? >> If you rename the above, please also rename the function which >> returns the above bool. >> >> 2) jvm.cpp >> >> 3850 JVM_ENTRY(void, JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, >> jstring line)) >> >> ignore -> ignored >> >> 3) jvm.hpp >> >> 210 JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line); >> >> Same comment as for jvm.cpp >> >> 4) metaspaceShared.cpp >> >> 2017?? size_t i = 0; >> 2018?? while (i < size) { >> 2019???? full_name[i++] = *start++; >> 2020?? } >> >> Could the above be simplified to the following? >> >> ??? strncpy(full_name, start, size - 1); >> >> 2029?? char* class_name = get_full_class_name(path_name); >> >> Should os::free(class_name) be called before the function returns? >> >> 1870 static GrowableArray* lambda_list = NULL; >> >> The name lambda_list is a bit generic, how about lambda_form_list? >> >> 2112?????? lambda_list->append(parser.current_line()); >> >> Since parser.current_line() does a strdup, do those buffer need to be >> freed after its use? (i.e. after the call to >> regenerate_holder_classes()?) >> >> In MetaspaceShared::regenerate_holder_classes, before calling up to >> java, I think it's better to strip the prefix "@lambda-form-invoker" >> from the strings. So that the java >> InvokerBytecodeGeneratorHelper.readTraceConfig method doesn't need to >> handle it: >> >> ?143???????????????????? case "[LF_RESOLVE]": >> ?144???????????????????? case InvokerBytecodeGenerator.CDS_LOG_PREFIX: >> >> 5) DumpSymbolAndStringTable.java >> >> 37???? private static final String my_string = >> "DumpSymbolAndStringTable"; >> >> Unused variable? >> >> thanks, >> >> Calvin >> >> On 8/11/20 10:36 AM, Yumin Qi wrote: >>> Forget to send to @core-lib-dev, the patch changed jdk code. >>> >>> >>> Thanks >>> >>> Yumin >>> >>> >>> >>> -------- Forwarded Message -------- >>> Subject:???? RFR: 8247536: Support pre-generated MethodHandle lambda >>> forms in CDS >>> Date:???? Tue, 11 Aug 2020 07:44:34 -0700 >>> From:???? Yumin Qi >>> To:???? hotspot-runtime-dev at openjdk.java.net >>> >>> >>> >>> Hi, Please reivew >>> >>> ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 >>> ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ >>> >>> ? Summary: CDS does not archive pre-generated lambda form classes >>> for method handles: >>> >>> [0.142s][info][class,load] >>> java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: >>> __JVM_LookupDefineClass__ >>> >>> The method handle resolution if not found in the holder class, a >>> class with the method will be generated and loaded as vm internal >>> created class and not archived like above. Those class names are >>> mangled as combination of the class name and the class instance >>> address. >>> >>> In this patch, collect those method holder class names and their >>> associated methods' signatures when user specify >>> DumpLoadedClassList, and record them in the log file in a format >>> similar to the output format from traced by >>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use >>> another name for CDS: >>> -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. The >>> line prefix also changed from "[LF_RESOLVE]" to >>> "@lambda-invoke-handle".? At dump time, regenerate the holder class >>> with those methods and replace the existing holder class and >>> archived it. At runtime, the resolution of the holder class which >>> contains those methods are loaded from CDS archive so avoid >>> regenerating them again. The patch reorganized the code for Jlink >>> plugin, so the new added InvokerBytecodeGeneratorHelper can be >>> shared both by JLink plugin and CDS code. Also change made to the >>> mangled generated class name at static dump, the class name >>> combining the class name and a sequentially ordered number instead >>> of the class instance address, which looks like a random. >>> >>> >>> To use this feature, one can do the dump/run just like done for a >>> static dump/run, so no extra steps required. >>> >>> ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp >>> >>> ????? DumpLoadedClassList will turn on >>> -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true >>> >>> ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist >>> -XX:SharedArchiveFile=my.jsa JavaApp >>> >>> ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp >>> >>> ? The performance on javac HelloWorld.java (javac-benchmark), no >>> patch vs patch: >>> >>> ?? 1:?? 2689285002? 2641821474 (-47463528)????? ---- 391.720 382.990 >>> ( -8.730)????? ---- >>> ?? 2:?? 2687495085? 2632969688 (-54525397)????? ---- 391.030 381.480 >>> ( -9.550)????? ----- >>> ?? 3:?? 2694664066? 2636523114 (-58140952)????? ----- 390.610 >>> 382.550 ( -8.060)????? ---- >>> ?? 4:?? 2686554164? 2639355233 (-47198931)????? ---- 390.700 383.390 >>> ( -7.310)????? --- >>> ?? 5:?? 2691072338? 2633016687 (-58055651)????? ----- 388.990 >>> 382.360 ( -6.630)????? --- >>> ?? 6:?? 2684448174? 2644191854 (-40256320)????? --- 389.450 382.990 >>> ( -6.460)????? --- >>> ?? 7:?? 2694921227? 2630505090 (-64416137)????? ----- 389.300 >>> 383.160 ( -6.140)????? --- >>> ?? 8:?? 2685209712? 2639334320 (-45875392)????? ---- 388.370 381.060 >>> ( -7.310)????? --- >>> ?? 9:?? 2695885942? 2640618655 (-55267287)????? ---- 389.560 381.100 >>> ( -8.460)????? ---- >>> ? 10:?? 2689162942? 2635658943 (-53503999)????? ---- 389.690 379.110 >>> (-10.580)????? ----- >>> ============================================================ >>> ??????? 2689866989? 2637396210 (-52470778)????? ---- 389.941 382.017 >>> ( -7.924)????? ---- >>> >>> instr delta =??? -52470778??? -1.9507% >>> >>> time? delta =?????? -7.924 ms -2.0321% >>> >>> ? It will show much more improvement if compare with the default >>> archive: >>> >>> ?????????? ? ? ? no patch??????????????????????? patch >>> >>> instr ??? ? ? 3,996,847,605??????? 3,320,928,995 >>> >>> time(s).???? 0.686835162 ? ? ? ?? 0.474933546 >>> >>> >>> ? Tests: >>> >>> 1)jtreg on jdk/tools and hotspot/runtime local. >>> >>> 2) mach5 ter1,2 >>> >>> >>> Thanks >>> >>> Yumin >>> >>> ? ` > From aph at redhat.com Sun Aug 16 17:35:42 2020 From: aph at redhat.com (Andrew Haley) Date: Sun, 16 Aug 2020 18:35:42 +0100 Subject: Possible subtle memory model error in ClassValue In-Reply-To: <2da470e0-ff56-62c3-129e-1138635fc14d@gmail.com> References: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> <6f91ae57-8dae-be23-bab4-c47dbe8b95f7@redhat.com> <0c849c2a-0a3a-704a-3ee3-22b690aa0d34@redhat.com> <04069f52-c9ef-8fee-26c7-5fdb69e8860b@redhat.com> <22fc68d5-d501-9ce0-093c-7be82aed247a@redhat.com> <55a36ac1-2aa2-7898-e3c3-bc2edcc315e7@gmail.com> <2da470e0-ff56-62c3-129e-1138635fc14d@gmail.com> Message-ID: On 15/08/2020 10:13, Peter Levart wrote: > https://github.com/openjdk/jdk/pull/9 > > > Sorry for abusing GitHub pull request mechanism but I don't have > bandwidth currently to clone the mercurial repository ;-) That's a lot of work to avoid a simple fence. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From peter.levart at gmail.com Mon Aug 17 14:24:44 2020 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 17 Aug 2020 16:24:44 +0200 Subject: Possible subtle memory model error in ClassValue In-Reply-To: References: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> <6f91ae57-8dae-be23-bab4-c47dbe8b95f7@redhat.com> <0c849c2a-0a3a-704a-3ee3-22b690aa0d34@redhat.com> <04069f52-c9ef-8fee-26c7-5fdb69e8860b@redhat.com> <22fc68d5-d501-9ce0-093c-7be82aed247a@redhat.com> <55a36ac1-2aa2-7898-e3c3-bc2edcc315e7@gmail.com> <2da470e0-ff56-62c3-129e-1138635fc14d@gmail.com> Message-ID: On 8/16/20 7:35 PM, Andrew Haley wrote: > On 15/08/2020 10:13, Peter Levart wrote: >> https://github.com/openjdk/jdk/pull/9 >> >> >> Sorry for abusing GitHub pull request mechanism but I don't have >> bandwidth currently to clone the mercurial repository ;-) > That's a lot of work to avoid a simple fence. > Two fences, mind you (the read fence is no-op only on Intel). So take half of that work for each fence ;-) Still a lot? No, really it was not much work to make the patch. The real work is yet to come - checking that it is correct. I leaned on the assumption that it is already correct when the cacheArray has to be swapped with bigger array. All accesses to the cacheArray field but the fast-path read-only probing is done under lock. So it is not that hard to reason about most of the code changes. The only changes that are done to code that accesses cacheArray field without holding a lock are two additional null checks in: loadFromCache and probeBackupLocations which basically just make the methods return null (not found) when parameter cache (read from cacheArray field) is found to be null. Now that the set of valid values for cache parameter contains null, there's no need for EMPTY_CACHE. I think that ClassValue was designed to cope with only one pair of fences (the ones that are used for initializing/accessing the final Entry.value field) and that all other accesses are performed with data races. So adding fences to accesses of cacheArray field would mean giving up on the design (John?). Regards, Peter From alexey.semenyuk at oracle.com Mon Aug 17 15:12:34 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Mon, 17 Aug 2020 11:12:34 -0400 Subject: RFR: JDK-8250611: Cannot display splash screen on Windows In-Reply-To: References: <765b70c2-ec7e-475d-cafe-d646847f5d2b@oracle.com> <5F35B8CD.3000509@oracle.com> Message-ID: <307ea13d-8de4-5de3-2e95-6dd39d7c9db6@oracle.com> Looks good. - Alexey On 8/14/2020 10:05 AM, Andy Herrick wrote: > Please review revised webrev [3] that does not check for client jvm > are (always use server) > > /Andy > > [3] - http://cr.openjdk.java.net/~herrick/8250611/webrev.02 > > On 8/13/2020 6:03 PM, Philip Race wrote: >> Do I understand correctly that you are checking if the client VM is >> being specified ? >> I didn't think we had client VMs any more .. >> >> -phil. >> >> On 8/13/20, 12:35 PM, Andy Herrick wrote: >>> Please review this jpackage fix for issue [1] at [2]. >>> >>> In order to show splash screen from statically linked applauncher, >>> we need to load the dependent libraries of splashscreen.dll first >>> >>> /Andy >>> >>> [1] - https://bugs.openjdk.java.net/browse/JDK-8250611 >>> >>> [2] - http://cr.openjdk.java.net/~herrick/8250611/webrev.01/ >>> From evan.whelan at oracle.com Mon Aug 17 15:25:59 2020 From: evan.whelan at oracle.com (Evan Whelan) Date: Mon, 17 Aug 2020 08:25:59 -0700 (PDT) Subject: RFR: JDK-8249691: jdk/lambda/vm/StrictfpDefault.java file can be removed Message-ID: Hi all, This is a small fix that helps with some test cleanup. One redundant test file has been removed. Webrev found at: http://cr.openjdk.java.net/~kravikumar/8249691/webrev/ Link to JBS issue: https://bugs.openjdk.java.net/browse/JDK-8249691 Any follow up questions are welcomed. Thanks, Evan From mandy.chung at oracle.com Mon Aug 17 19:29:31 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 17 Aug 2020 12:29:31 -0700 Subject: RFR(L) 8244778 Archive full module graph in CDS In-Reply-To: References: <9e6b0043-65a1-dd97-a3d2-33679c8048d4@oracle.com> Message-ID: <57711fdd-4c94-d4dc-de1e-f0be6dce4fc6@oracle.com> On 8/12/20 3:06 PM, Ioi Lam wrote: > Hi Lois, > > Thanks for the comments. I have an updated webrev > > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/ > > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02.delta/ > > This is very good work.?? I reviewed the Java code change that looks okay to me.?? This version is cleaner.? I skimmed through the VM code which is a huge change. ? I see Coleen and Lois reviewing and nothing to add. Mandy From yumin.qi at oracle.com Mon Aug 17 19:37:31 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Mon, 17 Aug 2020 12:37:31 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> Message-ID: <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> Hi, Ioi ? Thanks for review/suggestion. I have updated the webrev at the following link: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ ? Following changes done in this patch: ? 1) move regenerating holder classes into new added file: lambdaFormInvokers.[ch]pp "@lambda-form-invoker" tag only observed at jvm side. ? 2) remove InvokerBytecodeGeneratorHelper.java, move related functions to existing class, GenerateJLIClassesHelper.java ? 3) add tracing for SPECIES_RESOLVE, so archive more classes. ? 4) remove InvokerGenerateBytesException.java, using RuntimeException instead. ? 5) Changed make file to exclude the new added lamdaFormInvokers.cpp from the non-cds building list. ? 6) There is a bug in previous patch, jobject (typeArrayOop) should not be used directly for loading class since GC will move the java object. Fixed by copying the bytes from handle to C heap. ? 7) Tested with tier1,2 Thanks Yumin On 8/15/20 6:27 PM, Ioi Lam wrote: > On 8/15/20 6:19 PM, Ioi Lam wrote: >> To better capture what we're trying to do in this RFE, I've changed the RFE title (and the subject of this email thread) to >> >> https://bugs.openjdk.java.net/browse/JDK-8247536 >> Support for pre-generated java.lang.invoke classes in CDS static archive >> >> I also update the RFE Description to give an overview of the problem and the solution. >> >> The design document is also updated to reflect Yumin's latest implementation >> > > Oops, sent too quickly. The design doc's title is also updated: > > https://wiki.openjdk.java.net/display/HotSpot/Support+for+pre-generated+java.lang.invoke+classes+in+CDS+archive > >> Thanks >> - Ioi >> >> On 8/12/20 11:54 AM, calvin.cheung at oracle.com wrote: >>> Hi Yumin, >>> >>> I reviewed mostly the native code. Below are my comments: >>> >>> 1) classListParser.hpp >>> >>> 71?? bool??????????????? _lambda_format; >>> >>> The above name is too generic. How about _lambda_form or _is_lambda_form? >>> If you rename the above, please also rename the function which returns the above bool. >>> >>> 2) jvm.cpp >>> >>> 3850 JVM_ENTRY(void, JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line)) >>> >>> ignore -> ignored >>> >>> 3) jvm.hpp >>> >>> 210 JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line); >>> >>> Same comment as for jvm.cpp >>> >>> 4) metaspaceShared.cpp >>> >>> 2017?? size_t i = 0; >>> 2018?? while (i < size) { >>> 2019???? full_name[i++] = *start++; >>> 2020?? } >>> >>> Could the above be simplified to the following? >>> >>> ??? strncpy(full_name, start, size - 1); >>> >>> 2029?? char* class_name = get_full_class_name(path_name); >>> >>> Should os::free(class_name) be called before the function returns? >>> >>> 1870 static GrowableArray* lambda_list = NULL; >>> >>> The name lambda_list is a bit generic, how about lambda_form_list? >>> >>> 2112?????? lambda_list->append(parser.current_line()); >>> >>> Since parser.current_line() does a strdup, do those buffer need to be freed after its use? (i.e. after the call to regenerate_holder_classes()?) >>> >>> In MetaspaceShared::regenerate_holder_classes, before calling up to java, I think it's better to strip the prefix "@lambda-form-invoker" from the strings. So that the java InvokerBytecodeGeneratorHelper.readTraceConfig method doesn't need to handle it: >>> >>> ?143???????????????????? case "[LF_RESOLVE]": >>> ?144???????????????????? case InvokerBytecodeGenerator.CDS_LOG_PREFIX: >>> >>> 5) DumpSymbolAndStringTable.java >>> >>> 37???? private static final String my_string = "DumpSymbolAndStringTable"; >>> >>> Unused variable? >>> >>> thanks, >>> >>> Calvin >>> >>> On 8/11/20 10:36 AM, Yumin Qi wrote: >>>> Forget to send to @core-lib-dev, the patch changed jdk code. >>>> >>>> >>>> Thanks >>>> >>>> Yumin >>>> >>>> >>>> >>>> -------- Forwarded Message -------- >>>> Subject:???? RFR: 8247536: Support pre-generated MethodHandle lambda forms in CDS >>>> Date:???? Tue, 11 Aug 2020 07:44:34 -0700 >>>> From:???? Yumin Qi >>>> To:???? hotspot-runtime-dev at openjdk.java.net >>>> >>>> >>>> >>>> Hi, Please reivew >>>> >>>> ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 >>>> ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ >>>> >>>> ? Summary: CDS does not archive pre-generated lambda form classes for method handles: >>>> >>>> [0.142s][info][class,load] java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: __JVM_LookupDefineClass__ >>>> >>>> The method handle resolution if not found in the holder class, a class with the method will be generated and loaded as vm internal created class and not archived like above. Those class names are mangled as combination of the class name and the class instance address. >>>> >>>> In this patch, collect those method holder class names and their associated methods' signatures when user specify DumpLoadedClassList, and record them in the log file in a format similar to the output format from traced by -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use another name for CDS: -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. The line prefix also changed from "[LF_RESOLVE]" to "@lambda-invoke-handle".? At dump time, regenerate the holder class with those methods and replace the existing holder class and archived it. At runtime, the resolution of the holder class which contains those methods are loaded from CDS archive so avoid regenerating them again. The patch reorganized the code for Jlink plugin, so the new added InvokerBytecodeGeneratorHelper can be shared both by JLink plugin and CDS code. Also change made to the mangled generated class name at static dump, the class name combining the class name and a sequentially >>>> ordered number instead of the class instance address, which looks like a random. >>>> >>>> >>>> To use this feature, one can do the dump/run just like done for a static dump/run, so no extra steps required. >>>> >>>> ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp >>>> >>>> ????? DumpLoadedClassList will turn on -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true >>>> >>>> ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist -XX:SharedArchiveFile=my.jsa JavaApp >>>> >>>> ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp >>>> >>>> ? The performance on javac HelloWorld.java (javac-benchmark), no patch vs patch: >>>> >>>> ?? 1:?? 2689285002? 2641821474 (-47463528)????? ---- 391.720 382.990 ( -8.730)????? ---- >>>> ?? 2:?? 2687495085? 2632969688 (-54525397)????? ---- 391.030 381.480 ( -9.550)????? ----- >>>> ?? 3:?? 2694664066? 2636523114 (-58140952)????? ----- 390.610 382.550 ( -8.060)????? ---- >>>> ?? 4:?? 2686554164? 2639355233 (-47198931)????? ---- 390.700 383.390 ( -7.310)????? --- >>>> ?? 5:?? 2691072338? 2633016687 (-58055651)????? ----- 388.990 382.360 ( -6.630)????? --- >>>> ?? 6:?? 2684448174? 2644191854 (-40256320)????? --- 389.450 382.990 ( -6.460)????? --- >>>> ?? 7:?? 2694921227? 2630505090 (-64416137)????? ----- 389.300 383.160 ( -6.140)????? --- >>>> ?? 8:?? 2685209712? 2639334320 (-45875392)????? ---- 388.370 381.060 ( -7.310)????? --- >>>> ?? 9:?? 2695885942? 2640618655 (-55267287)????? ---- 389.560 381.100 ( -8.460)????? ---- >>>> ? 10:?? 2689162942? 2635658943 (-53503999)????? ---- 389.690 379.110 (-10.580)????? ----- >>>> ============================================================ >>>> ??????? 2689866989? 2637396210 (-52470778)????? ---- 389.941 382.017 ( -7.924)????? ---- >>>> >>>> instr delta =??? -52470778??? -1.9507% >>>> >>>> time? delta =?????? -7.924 ms -2.0321% >>>> >>>> ? It will show much more improvement if compare with the default archive: >>>> >>>> ?????????? ? ? ? no patch??????????????????????? patch >>>> >>>> instr ??? ? ? 3,996,847,605??????? 3,320,928,995 >>>> >>>> time(s).???? 0.686835162 ? ? ? ?? 0.474933546 >>>> >>>> >>>> ? Tests: >>>> >>>> 1)jtreg on jdk/tools and hotspot/runtime local. >>>> >>>> 2) mach5 ter1,2 >>>> >>>> >>>> Thanks >>>> >>>> Yumin >>>> >>>> ? ` >> > From vicente.romero at oracle.com Mon Aug 17 20:01:39 2020 From: vicente.romero at oracle.com (Vicente Romero) Date: Mon, 17 Aug 2020 16:01:39 -0400 Subject: RFR: JDK-8246804: Incorrect copyright header in TypeAnnotationParser.java Message-ID: <288b30a8-b61a-7fe0-9844-058a7350949e@oracle.com> Please review this very simple fix for [1] at [2]. The fix is a very simple one liner change, Thanks, Vicente [1] https://bugs.openjdk.java.net/browse/JDK-8246804 [2] http://cr.openjdk.java.net/~vromero/8246804/webrev.00/ From joe.darcy at oracle.com Mon Aug 17 20:15:14 2020 From: joe.darcy at oracle.com (Joe Darcy) Date: Mon, 17 Aug 2020 13:15:14 -0700 Subject: RFR: JDK-8246804: Incorrect copyright header in TypeAnnotationParser.java In-Reply-To: <288b30a8-b61a-7fe0-9844-058a7350949e@oracle.com> References: <288b30a8-b61a-7fe0-9844-058a7350949e@oracle.com> Message-ID: <92596579-0fca-1b70-d2e0-40bd3fd99334@oracle.com> +1 -Joe On 8/17/2020 1:01 PM, Vicente Romero wrote: > Please review this very simple fix for [1] at [2]. The fix is a very > simple one liner change, > > Thanks, > Vicente > > [1] https://bugs.openjdk.java.net/browse/JDK-8246804 > [2] http://cr.openjdk.java.net/~vromero/8246804/webrev.00/ From paul.sandoz at oracle.com Mon Aug 17 20:15:59 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 17 Aug 2020 13:15:59 -0700 Subject: RFR: JDK-8246804: Incorrect copyright header in TypeAnnotationParser.java In-Reply-To: <288b30a8-b61a-7fe0-9844-058a7350949e@oracle.com> References: <288b30a8-b61a-7fe0-9844-058a7350949e@oracle.com> Message-ID: +1 Paul. > On Aug 17, 2020, at 1:01 PM, Vicente Romero wrote: > > Please review this very simple fix for [1] at [2]. The fix is a very simple one liner change, > > Thanks, > Vicente > > [1] https://bugs.openjdk.java.net/browse/JDK-8246804 > [2] http://cr.openjdk.java.net/~vromero/8246804/webrev.00/ From naoto.sato at oracle.com Mon Aug 17 23:42:52 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Mon, 17 Aug 2020 16:42:52 -0700 Subject: RFR: 8251499: no-placeholder compact number patterns throw IllegalArgumentException In-Reply-To: References: <5b81c727-b1e0-0614-c68e-5568f98d9d6f@oracle.com> Message-ID: <217ef621-2293-3a1b-827c-06d067ea8b36@oracle.com> Hi Joe, It turned out that the previous fix did not address plural format cases. That means that just making the divisor negative to indicate non-placeholder cannot distinguish multiple plural cases with the same divisor. Instead, I created a list of placeholders (minimum digits) for each index and count. Here is the updated webrev: http://cr.openjdk.java.net/~naoto/8251499/webrev.01/ I added a new test case (COMPACT_PATTERN14), which actually is extracted from CLDR 38 Somali locale that demonstrates the issue. I'd appreciate your further review. Naoto On 8/14/20 6:21 PM, Joe Wang wrote: > Hi Naoto, > > Looks good to me. > > While a negative divisor representing no zeros is newly introduced, the > "divisor > 0" checks seem to have always been beneficial.? I had to > count the number of ""s in COMPACT_PATTERN13 :-) > > Have a great weekend! > Joe > > On 8/14/2020 3:20 PM, naoto.sato at oracle.com wrote: >> Hello, >> >> Please review the fix for the following issue: >> >> https://bugs.openjdk.java.net/browse/JDK-8251499 >> >> The proposed changeset is located at: >> >> https://cr.openjdk.java.net/~naoto/8251499/webrev.00/ >> >> The current implementation of CompactNumberFormat assumes that there >> is always the number placeholder part in compact patterns. This is not >> always true. In fact, upcoming CLDR 38 resurrects such patterns, so >> this fix is a precursor to support CLDR 38. >> >> Naoto > From sundararajan.athijegannathan at oracle.com Tue Aug 18 02:25:21 2020 From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com) Date: Tue, 18 Aug 2020 07:55:21 +0530 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> Message-ID: Not a full review of fresh changes. But couple of comments: * src/hotspot/share/memory/lambdaFormInvokers.cpp and src/hotspot/share/memory/lambdaFormInvokers.hpp miss "Classpath exception" clause in the copyright header * I had suggested a change GenerateJLIClassesPlugin.java in last round of review. That's still applicable. https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068160.html -Sundar On 18/08/20 1:07 am, Yumin Qi wrote: > Hi, Ioi > > ? Thanks for review/suggestion. I have updated the webrev at the > following link: > > http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ > > ? Following changes done in this patch: > > ? 1) move regenerating holder classes into new added file: > lambdaFormInvokers.[ch]pp > > ? "@lambda-form-invoker" tag only observed at jvm side. > > ? 2) remove InvokerBytecodeGeneratorHelper.java, move related > functions to existing class, GenerateJLIClassesHelper.java > > ? 3) add tracing for SPECIES_RESOLVE, so archive more classes. > > ? 4) remove InvokerGenerateBytesException.java, using RuntimeException > instead. > > ? 5) Changed make file to exclude the new added lamdaFormInvokers.cpp > from the non-cds building list. > > ? 6) There is a bug in previous patch, jobject (typeArrayOop) should > not be used directly for loading class since GC will move the java > object. Fixed by copying the bytes from handle to C heap. > > ? 7) Tested with tier1,2 > > > Thanks > > Yumin > > > > On 8/15/20 6:27 PM, Ioi Lam wrote: >> On 8/15/20 6:19 PM, Ioi Lam wrote: >>> To better capture what we're trying to do in this RFE, I've changed >>> the RFE title (and the subject of this email thread) to >>> >>> https://bugs.openjdk.java.net/browse/JDK-8247536 >>> Support for pre-generated java.lang.invoke classes in CDS static >>> archive >>> >>> I also update the RFE Description to give an overview of the problem >>> and the solution. >>> >>> The design document is also updated to reflect Yumin's latest >>> implementation >>> >> >> Oops, sent too quickly. The design doc's title is also updated: >> >> https://wiki.openjdk.java.net/display/HotSpot/Support+for+pre-generated+java.lang.invoke+classes+in+CDS+archive >> >> >>> Thanks >>> - Ioi >>> >>> On 8/12/20 11:54 AM, calvin.cheung at oracle.com wrote: >>>> Hi Yumin, >>>> >>>> I reviewed mostly the native code. Below are my comments: >>>> >>>> 1) classListParser.hpp >>>> >>>> 71?? bool??????????????? _lambda_format; >>>> >>>> The above name is too generic. How about _lambda_form or >>>> _is_lambda_form? >>>> If you rename the above, please also rename the function which >>>> returns the above bool. >>>> >>>> 2) jvm.cpp >>>> >>>> 3850 JVM_ENTRY(void, JVM_CDSTraceResolve(JNIEnv* env, jclass >>>> ignore, jstring line)) >>>> >>>> ignore -> ignored >>>> >>>> 3) jvm.hpp >>>> >>>> 210 JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line); >>>> >>>> Same comment as for jvm.cpp >>>> >>>> 4) metaspaceShared.cpp >>>> >>>> 2017?? size_t i = 0; >>>> 2018?? while (i < size) { >>>> 2019???? full_name[i++] = *start++; >>>> 2020?? } >>>> >>>> Could the above be simplified to the following? >>>> >>>> ??? strncpy(full_name, start, size - 1); >>>> >>>> 2029?? char* class_name = get_full_class_name(path_name); >>>> >>>> Should os::free(class_name) be called before the function returns? >>>> >>>> 1870 static GrowableArray* lambda_list = NULL; >>>> >>>> The name lambda_list is a bit generic, how about lambda_form_list? >>>> >>>> 2112?????? lambda_list->append(parser.current_line()); >>>> >>>> Since parser.current_line() does a strdup, do those buffer need to >>>> be freed after its use? (i.e. after the call to >>>> regenerate_holder_classes()?) >>>> >>>> In MetaspaceShared::regenerate_holder_classes, before calling up to >>>> java, I think it's better to strip the prefix >>>> "@lambda-form-invoker" from the strings. So that the java >>>> InvokerBytecodeGeneratorHelper.readTraceConfig method doesn't need >>>> to handle it: >>>> >>>> ?143???????????????????? case "[LF_RESOLVE]": >>>> ?144???????????????????? case InvokerBytecodeGenerator.CDS_LOG_PREFIX: >>>> >>>> 5) DumpSymbolAndStringTable.java >>>> >>>> 37???? private static final String my_string = >>>> "DumpSymbolAndStringTable"; >>>> >>>> Unused variable? >>>> >>>> thanks, >>>> >>>> Calvin >>>> >>>> On 8/11/20 10:36 AM, Yumin Qi wrote: >>>>> Forget to send to @core-lib-dev, the patch changed jdk code. >>>>> >>>>> >>>>> Thanks >>>>> >>>>> Yumin >>>>> >>>>> >>>>> >>>>> -------- Forwarded Message -------- >>>>> Subject:???? RFR: 8247536: Support pre-generated MethodHandle >>>>> lambda forms in CDS >>>>> Date:???? Tue, 11 Aug 2020 07:44:34 -0700 >>>>> From:???? Yumin Qi >>>>> To:???? hotspot-runtime-dev at openjdk.java.net >>>>> >>>>> >>>>> >>>>> Hi, Please reivew >>>>> >>>>> ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 >>>>> ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ >>>>> >>>>> ? Summary: CDS does not archive pre-generated lambda form classes >>>>> for method handles: >>>>> >>>>> [0.142s][info][class,load] >>>>> java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: >>>>> __JVM_LookupDefineClass__ >>>>> >>>>> The method handle resolution if not found in the holder class, a >>>>> class with the method will be generated and loaded as vm internal >>>>> created class and not archived like above. Those class names are >>>>> mangled as combination of the class name and the class instance >>>>> address. >>>>> >>>>> In this patch, collect those method holder class names and their >>>>> associated methods' signatures when user specify >>>>> DumpLoadedClassList, and record them in the log file in a format >>>>> similar to the output format from traced by >>>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use >>>>> another name for CDS: >>>>> -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. >>>>> The line prefix also changed from "[LF_RESOLVE]" to >>>>> "@lambda-invoke-handle".? At dump time, regenerate the holder >>>>> class with those methods and replace the existing holder class and >>>>> archived it. At runtime, the resolution of the holder class which >>>>> contains those methods are loaded from CDS archive so avoid >>>>> regenerating them again. The patch reorganized the code for Jlink >>>>> plugin, so the new added InvokerBytecodeGeneratorHelper can be >>>>> shared both by JLink plugin and CDS code. Also change made to the >>>>> mangled generated class name at static dump, the class name >>>>> combining the class name and a sequentially ordered number instead >>>>> of the class instance address, which looks like a random. >>>>> >>>>> >>>>> To use this feature, one can do the dump/run just like done for a >>>>> static dump/run, so no extra steps required. >>>>> >>>>> ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp >>>>> >>>>> ????? DumpLoadedClassList will turn on >>>>> -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true >>>>> >>>>> ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist >>>>> -XX:SharedArchiveFile=my.jsa JavaApp >>>>> >>>>> ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp >>>>> >>>>> ? The performance on javac HelloWorld.java (javac-benchmark), no >>>>> patch vs patch: >>>>> >>>>> ?? 1:?? 2689285002? 2641821474 (-47463528)????? ---- 391.720 >>>>> 382.990 ( -8.730)????? ---- >>>>> ?? 2:?? 2687495085? 2632969688 (-54525397)????? ---- 391.030 >>>>> 381.480 ( -9.550)????? ----- >>>>> ?? 3:?? 2694664066? 2636523114 (-58140952)????? ----- 390.610 >>>>> 382.550 ( -8.060)????? ---- >>>>> ?? 4:?? 2686554164? 2639355233 (-47198931)????? ---- 390.700 >>>>> 383.390 ( -7.310)????? --- >>>>> ?? 5:?? 2691072338? 2633016687 (-58055651)????? ----- 388.990 >>>>> 382.360 ( -6.630)????? --- >>>>> ?? 6:?? 2684448174? 2644191854 (-40256320)????? --- 389.450 >>>>> 382.990 ( -6.460)????? --- >>>>> ?? 7:?? 2694921227? 2630505090 (-64416137)????? ----- 389.300 >>>>> 383.160 ( -6.140)????? --- >>>>> ?? 8:?? 2685209712? 2639334320 (-45875392)????? ---- 388.370 >>>>> 381.060 ( -7.310)????? --- >>>>> ?? 9:?? 2695885942? 2640618655 (-55267287)????? ---- 389.560 >>>>> 381.100 ( -8.460)????? ---- >>>>> ? 10:?? 2689162942? 2635658943 (-53503999)????? ---- 389.690 >>>>> 379.110 (-10.580)????? ----- >>>>> ============================================================ >>>>> ??????? 2689866989? 2637396210 (-52470778)????? ---- 389.941 >>>>> 382.017 ( -7.924)????? ---- >>>>> >>>>> instr delta =??? -52470778??? -1.9507% >>>>> >>>>> time? delta =?????? -7.924 ms -2.0321% >>>>> >>>>> ? It will show much more improvement if compare with the default >>>>> archive: >>>>> >>>>> ?????????? ? ? ? no patch??????????????????????? patch >>>>> >>>>> instr ??? ? ? 3,996,847,605??????? 3,320,928,995 >>>>> >>>>> time(s).???? 0.686835162 ? ? ? ?? 0.474933546 >>>>> >>>>> >>>>> ? Tests: >>>>> >>>>> 1)jtreg on jdk/tools and hotspot/runtime local. >>>>> >>>>> 2) mach5 ter1,2 >>>>> >>>>> >>>>> Thanks >>>>> >>>>> Yumin >>>>> >>>>> ? ` >>> >> From david.holmes at oracle.com Tue Aug 18 02:34:34 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 18 Aug 2020 12:34:34 +1000 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> Message-ID: <2fc105ce-adaa-be64-fddf-c2f4b6293c54@oracle.com> Hi Sundar, On 18/08/2020 12:25 pm, sundararajan.athijegannathan at oracle.com wrote: > Not a full review of fresh changes. But couple of comments: > > * src/hotspot/share/memory/lambdaFormInvokers.cpp and > src/hotspot/share/memory/lambdaFormInvokers.hpp miss "Classpath > exception" clause in the copyright header Hotspot files do not use the Classpath exception. That is for .java sources that will be"linked against" by applications. Cheers, David > * I had suggested a change GenerateJLIClassesPlugin.java in last round > of review. That's still applicable. > > https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068160.html > > > -Sundar > > On 18/08/20 1:07 am, Yumin Qi wrote: >> Hi, Ioi >> >> ? Thanks for review/suggestion. I have updated the webrev at the >> following link: >> >> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >> >> ? Following changes done in this patch: >> >> ? 1) move regenerating holder classes into new added file: >> lambdaFormInvokers.[ch]pp >> >> ? "@lambda-form-invoker" tag only observed at jvm side. >> >> ? 2) remove InvokerBytecodeGeneratorHelper.java, move related >> functions to existing class, GenerateJLIClassesHelper.java >> >> ? 3) add tracing for SPECIES_RESOLVE, so archive more classes. >> >> ? 4) remove InvokerGenerateBytesException.java, using RuntimeException >> instead. >> >> ? 5) Changed make file to exclude the new added lamdaFormInvokers.cpp >> from the non-cds building list. >> >> ? 6) There is a bug in previous patch, jobject (typeArrayOop) should >> not be used directly for loading class since GC will move the java >> object. Fixed by copying the bytes from handle to C heap. >> >> ? 7) Tested with tier1,2 >> >> >> Thanks >> >> Yumin >> >> >> >> On 8/15/20 6:27 PM, Ioi Lam wrote: >>> On 8/15/20 6:19 PM, Ioi Lam wrote: >>>> To better capture what we're trying to do in this RFE, I've changed >>>> the RFE title (and the subject of this email thread) to >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8247536 >>>> Support for pre-generated java.lang.invoke classes in CDS static >>>> archive >>>> >>>> I also update the RFE Description to give an overview of the problem >>>> and the solution. >>>> >>>> The design document is also updated to reflect Yumin's latest >>>> implementation >>>> >>> >>> Oops, sent too quickly. The design doc's title is also updated: >>> >>> https://wiki.openjdk.java.net/display/HotSpot/Support+for+pre-generated+java.lang.invoke+classes+in+CDS+archive >>> >>> >>>> Thanks >>>> - Ioi >>>> >>>> On 8/12/20 11:54 AM, calvin.cheung at oracle.com wrote: >>>>> Hi Yumin, >>>>> >>>>> I reviewed mostly the native code. Below are my comments: >>>>> >>>>> 1) classListParser.hpp >>>>> >>>>> 71?? bool??????????????? _lambda_format; >>>>> >>>>> The above name is too generic. How about _lambda_form or >>>>> _is_lambda_form? >>>>> If you rename the above, please also rename the function which >>>>> returns the above bool. >>>>> >>>>> 2) jvm.cpp >>>>> >>>>> 3850 JVM_ENTRY(void, JVM_CDSTraceResolve(JNIEnv* env, jclass >>>>> ignore, jstring line)) >>>>> >>>>> ignore -> ignored >>>>> >>>>> 3) jvm.hpp >>>>> >>>>> 210 JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line); >>>>> >>>>> Same comment as for jvm.cpp >>>>> >>>>> 4) metaspaceShared.cpp >>>>> >>>>> 2017?? size_t i = 0; >>>>> 2018?? while (i < size) { >>>>> 2019???? full_name[i++] = *start++; >>>>> 2020?? } >>>>> >>>>> Could the above be simplified to the following? >>>>> >>>>> ??? strncpy(full_name, start, size - 1); >>>>> >>>>> 2029?? char* class_name = get_full_class_name(path_name); >>>>> >>>>> Should os::free(class_name) be called before the function returns? >>>>> >>>>> 1870 static GrowableArray* lambda_list = NULL; >>>>> >>>>> The name lambda_list is a bit generic, how about lambda_form_list? >>>>> >>>>> 2112?????? lambda_list->append(parser.current_line()); >>>>> >>>>> Since parser.current_line() does a strdup, do those buffer need to >>>>> be freed after its use? (i.e. after the call to >>>>> regenerate_holder_classes()?) >>>>> >>>>> In MetaspaceShared::regenerate_holder_classes, before calling up to >>>>> java, I think it's better to strip the prefix >>>>> "@lambda-form-invoker" from the strings. So that the java >>>>> InvokerBytecodeGeneratorHelper.readTraceConfig method doesn't need >>>>> to handle it: >>>>> >>>>> ?143???????????????????? case "[LF_RESOLVE]": >>>>> ?144???????????????????? case InvokerBytecodeGenerator.CDS_LOG_PREFIX: >>>>> >>>>> 5) DumpSymbolAndStringTable.java >>>>> >>>>> 37???? private static final String my_string = >>>>> "DumpSymbolAndStringTable"; >>>>> >>>>> Unused variable? >>>>> >>>>> thanks, >>>>> >>>>> Calvin >>>>> >>>>> On 8/11/20 10:36 AM, Yumin Qi wrote: >>>>>> Forget to send to @core-lib-dev, the patch changed jdk code. >>>>>> >>>>>> >>>>>> Thanks >>>>>> >>>>>> Yumin >>>>>> >>>>>> >>>>>> >>>>>> -------- Forwarded Message -------- >>>>>> Subject:???? RFR: 8247536: Support pre-generated MethodHandle >>>>>> lambda forms in CDS >>>>>> Date:???? Tue, 11 Aug 2020 07:44:34 -0700 >>>>>> From:???? Yumin Qi >>>>>> To:???? hotspot-runtime-dev at openjdk.java.net >>>>>> >>>>>> >>>>>> >>>>>> Hi, Please reivew >>>>>> >>>>>> ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 >>>>>> ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ >>>>>> >>>>>> ? Summary: CDS does not archive pre-generated lambda form classes >>>>>> for method handles: >>>>>> >>>>>> [0.142s][info][class,load] >>>>>> java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: >>>>>> __JVM_LookupDefineClass__ >>>>>> >>>>>> The method handle resolution if not found in the holder class, a >>>>>> class with the method will be generated and loaded as vm internal >>>>>> created class and not archived like above. Those class names are >>>>>> mangled as combination of the class name and the class instance >>>>>> address. >>>>>> >>>>>> In this patch, collect those method holder class names and their >>>>>> associated methods' signatures when user specify >>>>>> DumpLoadedClassList, and record them in the log file in a format >>>>>> similar to the output format from traced by >>>>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use >>>>>> another name for CDS: >>>>>> -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. >>>>>> The line prefix also changed from "[LF_RESOLVE]" to >>>>>> "@lambda-invoke-handle".? At dump time, regenerate the holder >>>>>> class with those methods and replace the existing holder class and >>>>>> archived it. At runtime, the resolution of the holder class which >>>>>> contains those methods are loaded from CDS archive so avoid >>>>>> regenerating them again. The patch reorganized the code for Jlink >>>>>> plugin, so the new added InvokerBytecodeGeneratorHelper can be >>>>>> shared both by JLink plugin and CDS code. Also change made to the >>>>>> mangled generated class name at static dump, the class name >>>>>> combining the class name and a sequentially ordered number instead >>>>>> of the class instance address, which looks like a random. >>>>>> >>>>>> >>>>>> To use this feature, one can do the dump/run just like done for a >>>>>> static dump/run, so no extra steps required. >>>>>> >>>>>> ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp >>>>>> >>>>>> ????? DumpLoadedClassList will turn on >>>>>> -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true >>>>>> >>>>>> ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist >>>>>> -XX:SharedArchiveFile=my.jsa JavaApp >>>>>> >>>>>> ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp >>>>>> >>>>>> ? The performance on javac HelloWorld.java (javac-benchmark), no >>>>>> patch vs patch: >>>>>> >>>>>> ?? 1:?? 2689285002? 2641821474 (-47463528)????? ---- 391.720 >>>>>> 382.990 ( -8.730)????? ---- >>>>>> ?? 2:?? 2687495085? 2632969688 (-54525397)????? ---- 391.030 >>>>>> 381.480 ( -9.550)????? ----- >>>>>> ?? 3:?? 2694664066? 2636523114 (-58140952)????? ----- 390.610 >>>>>> 382.550 ( -8.060)????? ---- >>>>>> ?? 4:?? 2686554164? 2639355233 (-47198931)????? ---- 390.700 >>>>>> 383.390 ( -7.310)????? --- >>>>>> ?? 5:?? 2691072338? 2633016687 (-58055651)????? ----- 388.990 >>>>>> 382.360 ( -6.630)????? --- >>>>>> ?? 6:?? 2684448174? 2644191854 (-40256320)????? --- 389.450 >>>>>> 382.990 ( -6.460)????? --- >>>>>> ?? 7:?? 2694921227? 2630505090 (-64416137)????? ----- 389.300 >>>>>> 383.160 ( -6.140)????? --- >>>>>> ?? 8:?? 2685209712? 2639334320 (-45875392)????? ---- 388.370 >>>>>> 381.060 ( -7.310)????? --- >>>>>> ?? 9:?? 2695885942? 2640618655 (-55267287)????? ---- 389.560 >>>>>> 381.100 ( -8.460)????? ---- >>>>>> ? 10:?? 2689162942? 2635658943 (-53503999)????? ---- 389.690 >>>>>> 379.110 (-10.580)????? ----- >>>>>> ============================================================ >>>>>> ??????? 2689866989? 2637396210 (-52470778)????? ---- 389.941 >>>>>> 382.017 ( -7.924)????? ---- >>>>>> >>>>>> instr delta =??? -52470778??? -1.9507% >>>>>> >>>>>> time? delta =?????? -7.924 ms -2.0321% >>>>>> >>>>>> ? It will show much more improvement if compare with the default >>>>>> archive: >>>>>> >>>>>> ?????????? ? ? ? no patch??????????????????????? patch >>>>>> >>>>>> instr ??? ? ? 3,996,847,605??????? 3,320,928,995 >>>>>> >>>>>> time(s).???? 0.686835162 ? ? ? ?? 0.474933546 >>>>>> >>>>>> >>>>>> ? Tests: >>>>>> >>>>>> 1)jtreg on jdk/tools and hotspot/runtime local. >>>>>> >>>>>> 2) mach5 ter1,2 >>>>>> >>>>>> >>>>>> Thanks >>>>>> >>>>>> Yumin >>>>>> >>>>>> ? ` >>>> >>> From sundararajan.athijegannathan at oracle.com Tue Aug 18 02:39:24 2020 From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com) Date: Tue, 18 Aug 2020 08:09:24 +0530 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <2fc105ce-adaa-be64-fddf-c2f4b6293c54@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <2fc105ce-adaa-be64-fddf-c2f4b6293c54@oracle.com> Message-ID: <9f1452b2-796a-1bf2-ab61-f0019d3192d3@oracle.com> Hi David. Thanks. -Sundar On 18/08/20 8:04 am, David Holmes wrote: > Hi Sundar, > > On 18/08/2020 12:25 pm, sundararajan.athijegannathan at oracle.com wrote: >> Not a full review of fresh changes. But couple of comments: >> >> * src/hotspot/share/memory/lambdaFormInvokers.cpp and >> src/hotspot/share/memory/lambdaFormInvokers.hpp miss "Classpath >> exception" clause in the copyright header > > Hotspot files do not use the Classpath exception. That is for .java > sources that will be"linked against" by applications. > > Cheers, > David > >> * I had suggested a change GenerateJLIClassesPlugin.java in last >> round of review. That's still applicable. >> >> https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068160.html >> >> >> -Sundar >> >> On 18/08/20 1:07 am, Yumin Qi wrote: >>> Hi, Ioi >>> >>> ? Thanks for review/suggestion. I have updated the webrev at the >>> following link: >>> >>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>> >>> ? Following changes done in this patch: >>> >>> ? 1) move regenerating holder classes into new added file: >>> lambdaFormInvokers.[ch]pp >>> >>> ? "@lambda-form-invoker" tag only observed at jvm side. >>> >>> ? 2) remove InvokerBytecodeGeneratorHelper.java, move related >>> functions to existing class, GenerateJLIClassesHelper.java >>> >>> ? 3) add tracing for SPECIES_RESOLVE, so archive more classes. >>> >>> ? 4) remove InvokerGenerateBytesException.java, using >>> RuntimeException instead. >>> >>> ? 5) Changed make file to exclude the new added >>> lamdaFormInvokers.cpp from the non-cds building list. >>> >>> ? 6) There is a bug in previous patch, jobject (typeArrayOop) should >>> not be used directly for loading class since GC will move the java >>> object. Fixed by copying the bytes from handle to C heap. >>> >>> ? 7) Tested with tier1,2 >>> >>> >>> Thanks >>> >>> Yumin >>> >>> >>> >>> On 8/15/20 6:27 PM, Ioi Lam wrote: >>>> On 8/15/20 6:19 PM, Ioi Lam wrote: >>>>> To better capture what we're trying to do in this RFE, I've >>>>> changed the RFE title (and the subject of this email thread) to >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8247536 >>>>> Support for pre-generated java.lang.invoke classes in CDS static >>>>> archive >>>>> >>>>> I also update the RFE Description to give an overview of the >>>>> problem and the solution. >>>>> >>>>> The design document is also updated to reflect Yumin's latest >>>>> implementation >>>>> >>>> >>>> Oops, sent too quickly. The design doc's title is also updated: >>>> >>>> https://wiki.openjdk.java.net/display/HotSpot/Support+for+pre-generated+java.lang.invoke+classes+in+CDS+archive >>>> >>>> >>>>> Thanks >>>>> - Ioi >>>>> >>>>> On 8/12/20 11:54 AM, calvin.cheung at oracle.com wrote: >>>>>> Hi Yumin, >>>>>> >>>>>> I reviewed mostly the native code. Below are my comments: >>>>>> >>>>>> 1) classListParser.hpp >>>>>> >>>>>> 71?? bool??????????????? _lambda_format; >>>>>> >>>>>> The above name is too generic. How about _lambda_form or >>>>>> _is_lambda_form? >>>>>> If you rename the above, please also rename the function which >>>>>> returns the above bool. >>>>>> >>>>>> 2) jvm.cpp >>>>>> >>>>>> 3850 JVM_ENTRY(void, JVM_CDSTraceResolve(JNIEnv* env, jclass >>>>>> ignore, jstring line)) >>>>>> >>>>>> ignore -> ignored >>>>>> >>>>>> 3) jvm.hpp >>>>>> >>>>>> 210 JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line); >>>>>> >>>>>> Same comment as for jvm.cpp >>>>>> >>>>>> 4) metaspaceShared.cpp >>>>>> >>>>>> 2017?? size_t i = 0; >>>>>> 2018?? while (i < size) { >>>>>> 2019???? full_name[i++] = *start++; >>>>>> 2020?? } >>>>>> >>>>>> Could the above be simplified to the following? >>>>>> >>>>>> ??? strncpy(full_name, start, size - 1); >>>>>> >>>>>> 2029?? char* class_name = get_full_class_name(path_name); >>>>>> >>>>>> Should os::free(class_name) be called before the function returns? >>>>>> >>>>>> 1870 static GrowableArray* lambda_list = NULL; >>>>>> >>>>>> The name lambda_list is a bit generic, how about lambda_form_list? >>>>>> >>>>>> 2112 lambda_list->append(parser.current_line()); >>>>>> >>>>>> Since parser.current_line() does a strdup, do those buffer need >>>>>> to be freed after its use? (i.e. after the call to >>>>>> regenerate_holder_classes()?) >>>>>> >>>>>> In MetaspaceShared::regenerate_holder_classes, before calling up >>>>>> to java, I think it's better to strip the prefix >>>>>> "@lambda-form-invoker" from the strings. So that the java >>>>>> InvokerBytecodeGeneratorHelper.readTraceConfig method doesn't >>>>>> need to handle it: >>>>>> >>>>>> ?143???????????????????? case "[LF_RESOLVE]": >>>>>> ?144???????????????????? case >>>>>> InvokerBytecodeGenerator.CDS_LOG_PREFIX: >>>>>> >>>>>> 5) DumpSymbolAndStringTable.java >>>>>> >>>>>> 37???? private static final String my_string = >>>>>> "DumpSymbolAndStringTable"; >>>>>> >>>>>> Unused variable? >>>>>> >>>>>> thanks, >>>>>> >>>>>> Calvin >>>>>> >>>>>> On 8/11/20 10:36 AM, Yumin Qi wrote: >>>>>>> Forget to send to @core-lib-dev, the patch changed jdk code. >>>>>>> >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> Yumin >>>>>>> >>>>>>> >>>>>>> >>>>>>> -------- Forwarded Message -------- >>>>>>> Subject:???? RFR: 8247536: Support pre-generated MethodHandle >>>>>>> lambda forms in CDS >>>>>>> Date:???? Tue, 11 Aug 2020 07:44:34 -0700 >>>>>>> From:???? Yumin Qi >>>>>>> To:???? hotspot-runtime-dev at openjdk.java.net >>>>>>> >>>>>>> >>>>>>> >>>>>>> Hi, Please reivew >>>>>>> >>>>>>> ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 >>>>>>> ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ >>>>>>> >>>>>>> ? Summary: CDS does not archive pre-generated lambda form >>>>>>> classes for method handles: >>>>>>> >>>>>>> [0.142s][info][class,load] >>>>>>> java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: >>>>>>> __JVM_LookupDefineClass__ >>>>>>> >>>>>>> The method handle resolution if not found in the holder class, a >>>>>>> class with the method will be generated and loaded as vm >>>>>>> internal created class and not archived like above. Those class >>>>>>> names are mangled as combination of the class name and the class >>>>>>> instance address. >>>>>>> >>>>>>> In this patch, collect those method holder class names and their >>>>>>> associated methods' signatures when user specify >>>>>>> DumpLoadedClassList, and record them in the log file in a format >>>>>>> similar to the output format from traced by >>>>>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use >>>>>>> another name for CDS: >>>>>>> -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. >>>>>>> The line prefix also changed from "[LF_RESOLVE]" to >>>>>>> "@lambda-invoke-handle".? At dump time, regenerate the holder >>>>>>> class with those methods and replace the existing holder class >>>>>>> and archived it. At runtime, the resolution of the holder class >>>>>>> which contains those methods are loaded from CDS archive so >>>>>>> avoid regenerating them again. The patch reorganized the code >>>>>>> for Jlink plugin, so the new added >>>>>>> InvokerBytecodeGeneratorHelper can be shared both by JLink >>>>>>> plugin and CDS code. Also change made to the mangled generated >>>>>>> class name at static dump, the class name combining the class >>>>>>> name and a sequentially ordered number instead of the class >>>>>>> instance address, which looks like a random. >>>>>>> >>>>>>> >>>>>>> To use this feature, one can do the dump/run just like done for >>>>>>> a static dump/run, so no extra steps required. >>>>>>> >>>>>>> ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp >>>>>>> >>>>>>> ????? DumpLoadedClassList will turn on >>>>>>> -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true >>>>>>> >>>>>>> ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist >>>>>>> -XX:SharedArchiveFile=my.jsa JavaApp >>>>>>> >>>>>>> ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp >>>>>>> >>>>>>> ? The performance on javac HelloWorld.java (javac-benchmark), no >>>>>>> patch vs patch: >>>>>>> >>>>>>> ?? 1:?? 2689285002? 2641821474 (-47463528)????? ---- 391.720 >>>>>>> 382.990 ( -8.730)????? ---- >>>>>>> ?? 2:?? 2687495085? 2632969688 (-54525397)????? ---- 391.030 >>>>>>> 381.480 ( -9.550)????? ----- >>>>>>> ?? 3:?? 2694664066? 2636523114 (-58140952)????? ----- 390.610 >>>>>>> 382.550 ( -8.060)????? ---- >>>>>>> ?? 4:?? 2686554164? 2639355233 (-47198931)????? ---- 390.700 >>>>>>> 383.390 ( -7.310)????? --- >>>>>>> ?? 5:?? 2691072338? 2633016687 (-58055651)????? ----- 388.990 >>>>>>> 382.360 ( -6.630)????? --- >>>>>>> ?? 6:?? 2684448174? 2644191854 (-40256320)????? --- 389.450 >>>>>>> 382.990 ( -6.460)????? --- >>>>>>> ?? 7:?? 2694921227? 2630505090 (-64416137)????? ----- 389.300 >>>>>>> 383.160 ( -6.140)????? --- >>>>>>> ?? 8:?? 2685209712? 2639334320 (-45875392)????? ---- 388.370 >>>>>>> 381.060 ( -7.310)????? --- >>>>>>> ?? 9:?? 2695885942? 2640618655 (-55267287)????? ---- 389.560 >>>>>>> 381.100 ( -8.460)????? ---- >>>>>>> ? 10:?? 2689162942? 2635658943 (-53503999)????? ---- 389.690 >>>>>>> 379.110 (-10.580)????? ----- >>>>>>> ============================================================ >>>>>>> ??????? 2689866989? 2637396210 (-52470778)????? ---- 389.941 >>>>>>> 382.017 ( -7.924)????? ---- >>>>>>> >>>>>>> instr delta =??? -52470778??? -1.9507% >>>>>>> >>>>>>> time? delta =?????? -7.924 ms -2.0321% >>>>>>> >>>>>>> ? It will show much more improvement if compare with the default >>>>>>> archive: >>>>>>> >>>>>>> ?????????? ? ? ? no patch??????????????????????? patch >>>>>>> >>>>>>> instr ??? ? ? 3,996,847,605??????? 3,320,928,995 >>>>>>> >>>>>>> time(s).???? 0.686835162 ? ? ? ?? 0.474933546 >>>>>>> >>>>>>> >>>>>>> ? Tests: >>>>>>> >>>>>>> 1)jtreg on jdk/tools and hotspot/runtime local. >>>>>>> >>>>>>> 2) mach5 ter1,2 >>>>>>> >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> Yumin >>>>>>> >>>>>>> ? ` >>>>> >>>> From huizhe.wang at oracle.com Tue Aug 18 06:52:31 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Mon, 17 Aug 2020 23:52:31 -0700 Subject: RFR: 8251499: no-placeholder compact number patterns throw IllegalArgumentException In-Reply-To: <217ef621-2293-3a1b-827c-06d067ea8b36@oracle.com> References: <5b81c727-b1e0-0614-c68e-5568f98d9d6f@oracle.com> <217ef621-2293-3a1b-827c-06d067ea8b36@oracle.com> Message-ID: <9ee7bb2e-8030-b3a5-c5f6-6aa1517deb5c@oracle.com> Hi Naoto, Looks good overall. One nit, blocks 1633-1639 and 1642-1649 may share a common private method with a parameter that takes either matchedPosIndex or matchedNegIndex, if you want. Best, Joe On 8/17/20 4:42 PM, naoto.sato at oracle.com wrote: > Hi Joe, > > It turned out that the previous fix did not address plural format > cases. That means that just making the divisor negative to indicate > non-placeholder cannot distinguish multiple plural cases with the same > divisor. Instead, I created a list of placeholders (minimum digits) > for each index and count. Here is the updated webrev: > > http://cr.openjdk.java.net/~naoto/8251499/webrev.01/ > > I added a new test case (COMPACT_PATTERN14), which actually is > extracted from CLDR 38 Somali locale that demonstrates the issue. I'd > appreciate your further review. > > Naoto > > On 8/14/20 6:21 PM, Joe Wang wrote: >> Hi Naoto, >> >> Looks good to me. >> >> While a negative divisor representing no zeros is newly introduced, >> the "divisor > 0" checks seem to have always been beneficial.? I had >> to count the number of ""s in COMPACT_PATTERN13 :-) >> >> Have a great weekend! >> Joe >> >> On 8/14/2020 3:20 PM, naoto.sato at oracle.com wrote: >>> Hello, >>> >>> Please review the fix for the following issue: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8251499 >>> >>> The proposed changeset is located at: >>> >>> https://cr.openjdk.java.net/~naoto/8251499/webrev.00/ >>> >>> The current implementation of CompactNumberFormat assumes that there >>> is always the number placeholder part in compact patterns. This is >>> not always true. In fact, upcoming CLDR 38 resurrects such patterns, >>> so this fix is a precursor to support CLDR 38. >>> >>> Naoto >> From sean.coffey at oracle.com Tue Aug 18 11:09:11 2020 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Tue, 18 Aug 2020 12:09:11 +0100 Subject: RFR: JDK-8249691: jdk/lambda/vm/StrictfpDefault.java file can be removed In-Reply-To: References: Message-ID: <7a91846b-5813-5cb2-eaeb-4606e2a901d6@oracle.com> Looks fine to me Evan. regards, Sean. On 17/08/2020 16:25, Evan Whelan wrote: > Hi all, > > > > This is a small fix that helps with some test cleanup. One redundant test file has been removed. > > > > Webrev found at: http://cr.openjdk.java.net/~kravikumar/8249691/webrev/ > > > > Link to JBS issue: https://bugs.openjdk.java.net/browse/JDK-8249691 > > > > Any follow up questions are welcomed. > > > > Thanks, > > Evan From aph at redhat.com Tue Aug 18 15:00:00 2020 From: aph at redhat.com (Andrew Haley) Date: Tue, 18 Aug 2020 16:00:00 +0100 Subject: Possible subtle memory model error in ClassValue In-Reply-To: References: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> <6f91ae57-8dae-be23-bab4-c47dbe8b95f7@redhat.com> <0c849c2a-0a3a-704a-3ee3-22b690aa0d34@redhat.com> <04069f52-c9ef-8fee-26c7-5fdb69e8860b@redhat.com> <22fc68d5-d501-9ce0-093c-7be82aed247a@redhat.com> <55a36ac1-2aa2-7898-e3c3-bc2edcc315e7@gmail.com> <2da470e0-ff56-62c3-129e-1138635fc14d@gmail.com> Message-ID: <3d624bac-ceed-f0b0-b9f5-3cd4fb936c73@redhat.com> On 17/08/2020 15:24, Peter Levart wrote: > > On 8/16/20 7:35 PM, Andrew Haley wrote: >> On 15/08/2020 10:13, Peter Levart wrote: >>> https://github.com/openjdk/jdk/pull/9 >>> >>> >>> Sorry for abusing GitHub pull request mechanism but I don't have >>> bandwidth currently to clone the mercurial repository ;-) >> That's a lot of work to avoid a simple fence. >> > Two fences, mind you (the read fence is no-op only on Intel). So take > half of that work for each fence ;-) Still a lot? OK, but in HotSpot (and GraalVM AFAICR, but I'd need to check) finals only require a StoreStore fence and the address dependency does the rest, so if we can get away with using a final field then we can also get away with just using a StoreStore fence at the end of the constructor. I understand that this (as Hans pointed out) isn't strict Java but it depends on how much we care about strict Java inside the core libraries. > No, really it was not much work to make the patch. The real work is yet > to come - checking that it is correct. Indeed. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From Roger.Riggs at oracle.com Tue Aug 18 15:19:57 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Tue, 18 Aug 2020 11:19:57 -0400 Subject: RFR: 8251499: no-placeholder compact number patterns throw IllegalArgumentException In-Reply-To: <217ef621-2293-3a1b-827c-06d067ea8b36@oracle.com> References: <5b81c727-b1e0-0614-c68e-5568f98d9d6f@oracle.com> <217ef621-2293-3a1b-827c-06d067ea8b36@oracle.com> Message-ID: <4c819231-344a-791f-d25a-edb9aaef648b@oracle.com> Hi Naoto, I think the issue would benefit from a comment describing the solution. Its not clear how the code addresses the issue. Thanks, Roger On 8/17/20 7:42 PM, naoto.sato at oracle.com wrote: > Hi Joe, > > It turned out that the previous fix did not address plural format > cases. That means that just making the divisor negative to indicate > non-placeholder cannot distinguish multiple plural cases with the same > divisor. Instead, I created a list of placeholders (minimum digits) > for each index and count. Here is the updated webrev: > > http://cr.openjdk.java.net/~naoto/8251499/webrev.01/ > > I added a new test case (COMPACT_PATTERN14), which actually is > extracted from CLDR 38 Somali locale that demonstrates the issue. I'd > appreciate your further review. > > Naoto > > On 8/14/20 6:21 PM, Joe Wang wrote: >> Hi Naoto, >> >> Looks good to me. >> >> While a negative divisor representing no zeros is newly introduced, >> the "divisor > 0" checks seem to have always been beneficial.? I had >> to count the number of ""s in COMPACT_PATTERN13 :-) >> >> Have a great weekend! >> Joe >> >> On 8/14/2020 3:20 PM, naoto.sato at oracle.com wrote: >>> Hello, >>> >>> Please review the fix for the following issue: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8251499 >>> >>> The proposed changeset is located at: >>> >>> https://cr.openjdk.java.net/~naoto/8251499/webrev.00/ >>> >>> The current implementation of CompactNumberFormat assumes that there >>> is always the number placeholder part in compact patterns. This is >>> not always true. In fact, upcoming CLDR 38 resurrects such patterns, >>> so this fix is a precursor to support CLDR 38. >>> >>> Naoto >> From hboehm at google.com Tue Aug 18 15:50:05 2020 From: hboehm at google.com (Hans Boehm) Date: Tue, 18 Aug 2020 08:50:05 -0700 Subject: Possible subtle memory model error in ClassValue In-Reply-To: <22fc68d5-d501-9ce0-093c-7be82aed247a@redhat.com> References: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> <6f91ae57-8dae-be23-bab4-c47dbe8b95f7@redhat.com> <0c849c2a-0a3a-704a-3ee3-22b690aa0d34@redhat.com> <04069f52-c9ef-8fee-26c7-5fdb69e8860b@redhat.com> <22fc68d5-d501-9ce0-093c-7be82aed247a@redhat.com> Message-ID: On Wed, Aug 12, 2020 at 2:09 AM Andrew Dinn wrote: > > On 11/08/2020 18:06, Hans Boehm wrote: > > I think the relevant statement is: > > > > "An address dependency between two reads generated by SVE vector load > > instructions does not contribute to the Dependency-ordered-before relation." > > > > This is only an issue if BOTH loads are SVE loads. In particular > > reference loads have to be vectorized for this to matter, which I expect > > is not the common situation. I have not explored this in great detail, > > but it should suffice to put fences between two dependent vector > > reference loads, and between a reference load and a dependent final > > field load. > Hmm, so this might perhaps be an issue with something like a deep copy > of an int[][], where loading of successive int[] references might be > vectorized using SVE instructions and processing of the contents of each > referenced int[] might also be similarly vectorized. In that case the > loading of the int contents would need to be ordered wrt the load of the > int[] references using a LoadLoad barrier? > I think it's potentially more common, though it clearly depends on the details of current and future implementations. Let's say I define a MutableInteger class in the expected way, and then want to say, copy every element to an int[]. At least if we naively ignore details like null checks (which I may be able to statically preclude, and may not affect this anyway), I think this can be fairly easily vectorized in SVE. If we assume that MutableInteger somehow has a StoreStore barrier in its constructor, initializing the MutableInteger and then storing a reference to it in the array, while my copy function is running, can still result in the copy seeing an uninitialized value. If my example used Integer instead of MutableInteger, the compiler would have to either insert a fence, or not vectorize, to prevent this. With MutableInteger, there is no reason for it to do so. Hans From paul.sandoz at oracle.com Tue Aug 18 16:49:07 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 18 Aug 2020 09:49:07 -0700 Subject: =?utf-8?Q?Re=3A_RFR=5B8238286=5D=3A_=27Add_new_flatMap_stream_ope?= =?utf-8?Q?ration_that_is_more_amenable_to_pushing=E2=80=99?= In-Reply-To: References: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> <5B4DFB68-747C-4EAB-A042-C24DF6516FB8@oracle.com> <47c1e40f-86ac-415d-5c43-674b47d8884d@anthonyv.be> Message-ID: Hi Patrick, This looks good. To resolve the ambiguity of when results are undefined I suggest we tweak the docs at the various locations, see below. No need for another round of review. I can understand the desire to place the primitive functional interfaces in j.u.functions, but for reasons previously stated they are fine in the current location. Paul. * accepts replacing elements. The mapping function operates on the consumer, * zero or more times, for acceptance of replacing elements. * - *

    The results of this method are undefined if the {@link Consumer} - * argument is called outside the scope of the mapper function. - * *

    This is an intermediate * operation. + * The results of this intermediate operation are undefined if the + * {@code consumer} argument is operated on outside the scope of + * its application to the mapping function. * * @implSpec * The default implementation accumulates accepted elements into an internal > On Aug 11, 2020, at 11:11 AM, Patrick Concannon wrote: > > Hi, > > Please find the next iteration of mapMulti in the new webrev below. > > In this iteration the following changes have been made: > The API note for mapMulti has been updated. > Sub-interfaces for {Int, Double, Long}Stream have been refactored to be more specific to mapMulti. > The examples have been changed, and now include a reference to how an explicit type parameter can be used in conjunction with mapMulti. > > The CSR and specdiff have also been updated to reflect these changes. > > webrev: http://cr.openjdk.java.net/~pconcannon/8238286/webrevs/webrev.05/ > specdiff: http://cr.openjdk.java.net/~pconcannon/8238286/specdiff/specout.02/ > CSR: https://bugs.openjdk.java.net/browse/JDK-8248166 > > Kind regards, > Patrick From julia.boes at oracle.com Tue Aug 18 17:02:37 2020 From: julia.boes at oracle.com (Julia Boes) Date: Tue, 18 Aug 2020 18:02:37 +0100 Subject: Fix for Javadoc errors in java.base In-Reply-To: References: <318a4bbf-1b3e-d34e-d69a-d1a2ee233794@oracle.com> <8e978983-4ec1-8421-2563-d00c61f0d55e@oracle.com> <11adede2-1119-f133-020f-4ddd60c9fc67@oracle.com> Message-ID: <6b31a7d4-3bcd-2c8d-2f3c-4158047c3e31@oracle.com> Hi, The two changes below still need to be reviewed. Any takers? Cheers, Julia > --- > old/src/java.base/share/classes/java/lang/invoke/AbstractValidatingLambdaMetafactory.java2020-08-14 > 23:55:41.953638446 +0530 > +++ > new/src/java.base/share/classes/java/lang/invoke/AbstractValidatingLambdaMetafactory.java2020-08-14 > 23:55:41.445619497 +0530 > @@ -1,5 +1,5 @@ > ?/* > - * Copyright (c) 2012, 2013, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 2012, 2020, Oracle and/or its affiliates. All rights > reserved. > ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > ? * > ? * This code is free software; you can redistribute it and/or modify it > @@ -208,7 +208,7 @@ > ? ? ? * > ? ? ? * @return a CallSite, which, when invoked, will return an > instance of the > ? ? ? * functional interface > - ? ? * @throws ReflectiveOperationException > + ? ? * @throws LambdaConversionException > ? ? ? */ > ? ? ?abstract CallSite buildCallSite() > ? ? ? ? ? ? ?throws LambdaConversionException; > --- > old/src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java2020-08-14 > 23:55:42.977677096 +0530 > +++ > new/src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java2020-08-14 > 23:55:42.473658062 +0530 > @@ -200,7 +200,6 @@ > ? ? ? * > ? ? ? * @return a CallSite, which, when invoked, will return an > instance of the > ? ? ? * functional interface > - ? ? * @throws ReflectiveOperationException > ? ? ? * @throws LambdaConversionException If properly formed > functional interface > ? ? ? * is not found > ? ? ? */ From mandy.chung at oracle.com Tue Aug 18 17:05:12 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 18 Aug 2020 10:05:12 -0700 Subject: Fix for Javadoc errors in java.base In-Reply-To: <6b31a7d4-3bcd-2c8d-2f3c-4158047c3e31@oracle.com> References: <318a4bbf-1b3e-d34e-d69a-d1a2ee233794@oracle.com> <8e978983-4ec1-8421-2563-d00c61f0d55e@oracle.com> <11adede2-1119-f133-020f-4ddd60c9fc67@oracle.com> <6b31a7d4-3bcd-2c8d-2f3c-4158047c3e31@oracle.com> Message-ID: <4ffac099-db84-c782-2e14-75bde259e86d@oracle.com> Looks fine. Thanks Mandy On 8/18/20 10:02 AM, Julia Boes wrote: > > Hi, > > The two changes below still need to be reviewed. Any takers? > > Cheers, > > Julia > >> --- >> old/src/java.base/share/classes/java/lang/invoke/AbstractValidatingLambdaMetafactory.java2020-08-14 >> 23:55:41.953638446 +0530 >> +++ >> new/src/java.base/share/classes/java/lang/invoke/AbstractValidatingLambdaMetafactory.java2020-08-14 >> 23:55:41.445619497 +0530 >> @@ -1,5 +1,5 @@ >> ?/* >> - * Copyright (c) 2012, 2013, Oracle and/or its affiliates. All >> rights reserved. >> + * Copyright (c) 2012, 2020, Oracle and/or its affiliates. All >> rights reserved. >> ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >> ? * >> ? * This code is free software; you can redistribute it and/or modify it >> @@ -208,7 +208,7 @@ >> ? ? ? * >> ? ? ? * @return a CallSite, which, when invoked, will return an >> instance of the >> ? ? ? * functional interface >> - ? ? * @throws ReflectiveOperationException >> + ? ? * @throws LambdaConversionException >> ? ? ? */ >> ? ? ?abstract CallSite buildCallSite() >> ? ? ? ? ? ? ?throws LambdaConversionException; >> --- >> old/src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java2020-08-14 >> 23:55:42.977677096 +0530 >> +++ >> new/src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java2020-08-14 >> 23:55:42.473658062 +0530 >> @@ -200,7 +200,6 @@ >> ? ? ? * >> ? ? ? * @return a CallSite, which, when invoked, will return an >> instance of the >> ? ? ? * functional interface >> - ? ? * @throws ReflectiveOperationException >> ? ? ? * @throws LambdaConversionException If properly formed >> functional interface >> ? ? ? * is not found >> ? ? ? */ From naoto.sato at oracle.com Tue Aug 18 17:10:09 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Tue, 18 Aug 2020 10:10:09 -0700 Subject: RFR: 8251499: no-placeholder compact number patterns throw IllegalArgumentException In-Reply-To: <4c819231-344a-791f-d25a-edb9aaef648b@oracle.com> References: <5b81c727-b1e0-0614-c68e-5568f98d9d6f@oracle.com> <217ef621-2293-3a1b-827c-06d067ea8b36@oracle.com> <4c819231-344a-791f-d25a-edb9aaef648b@oracle.com> Message-ID: <64070923-704d-3f3e-6e1f-9e88875ff6fb@oracle.com> Hi Roger, Thank you for your comment. I added a brief comment in the issue on how the implementation behaves in the problem case. Naoto On 8/18/20 8:19 AM, Roger Riggs wrote: > Hi Naoto, > > I think the issue would benefit from a comment describing the solution. > Its not clear how the code addresses the issue. > > Thanks, Roger > > > On 8/17/20 7:42 PM, naoto.sato at oracle.com wrote: >> Hi Joe, >> >> It turned out that the previous fix did not address plural format >> cases. That means that just making the divisor negative to indicate >> non-placeholder cannot distinguish multiple plural cases with the same >> divisor. Instead, I created a list of placeholders (minimum digits) >> for each index and count. Here is the updated webrev: >> >> http://cr.openjdk.java.net/~naoto/8251499/webrev.01/ >> >> I added a new test case (COMPACT_PATTERN14), which actually is >> extracted from CLDR 38 Somali locale that demonstrates the issue. I'd >> appreciate your further review. >> >> Naoto >> >> On 8/14/20 6:21 PM, Joe Wang wrote: >>> Hi Naoto, >>> >>> Looks good to me. >>> >>> While a negative divisor representing no zeros is newly introduced, >>> the "divisor > 0" checks seem to have always been beneficial.? I had >>> to count the number of ""s in COMPACT_PATTERN13 :-) >>> >>> Have a great weekend! >>> Joe >>> >>> On 8/14/2020 3:20 PM, naoto.sato at oracle.com wrote: >>>> Hello, >>>> >>>> Please review the fix for the following issue: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8251499 >>>> >>>> The proposed changeset is located at: >>>> >>>> https://cr.openjdk.java.net/~naoto/8251499/webrev.00/ >>>> >>>> The current implementation of CompactNumberFormat assumes that there >>>> is always the number placeholder part in compact patterns. This is >>>> not always true. In fact, upcoming CLDR 38 resurrects such patterns, >>>> so this fix is a precursor to support CLDR 38. >>>> >>>> Naoto >>> > From naoto.sato at oracle.com Tue Aug 18 18:37:26 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Tue, 18 Aug 2020 11:37:26 -0700 Subject: RFR: 8251499: no-placeholder compact number patterns throw IllegalArgumentException In-Reply-To: <9ee7bb2e-8030-b3a5-c5f6-6aa1517deb5c@oracle.com> References: <5b81c727-b1e0-0614-c68e-5568f98d9d6f@oracle.com> <217ef621-2293-3a1b-827c-06d067ea8b36@oracle.com> <9ee7bb2e-8030-b3a5-c5f6-6aa1517deb5c@oracle.com> Message-ID: Hi Joe, Thank you for your comment. I consolidated those duplicated pieces into one piece. Did not make it a private method, though, as it would need to return two results from the method (cnfMultiplier and whether to return immediately for no-placeholder cases). https://cr.openjdk.java.net/~naoto/8251499/webrev.02/ Naoto On 8/17/20 11:52 PM, Joe Wang wrote: > Hi Naoto, > > Looks good overall. One nit, blocks 1633-1639 and 1642-1649 may share a > common private method with a parameter that takes either matchedPosIndex > or matchedNegIndex, if you want. > > Best, > Joe > > On 8/17/20 4:42 PM, naoto.sato at oracle.com wrote: >> Hi Joe, >> >> It turned out that the previous fix did not address plural format >> cases. That means that just making the divisor negative to indicate >> non-placeholder cannot distinguish multiple plural cases with the same >> divisor. Instead, I created a list of placeholders (minimum digits) >> for each index and count. Here is the updated webrev: >> >> http://cr.openjdk.java.net/~naoto/8251499/webrev.01/ >> >> I added a new test case (COMPACT_PATTERN14), which actually is >> extracted from CLDR 38 Somali locale that demonstrates the issue. I'd >> appreciate your further review. >> >> Naoto >> >> On 8/14/20 6:21 PM, Joe Wang wrote: >>> Hi Naoto, >>> >>> Looks good to me. >>> >>> While a negative divisor representing no zeros is newly introduced, >>> the "divisor > 0" checks seem to have always been beneficial.? I had >>> to count the number of ""s in COMPACT_PATTERN13 :-) >>> >>> Have a great weekend! >>> Joe >>> >>> On 8/14/2020 3:20 PM, naoto.sato at oracle.com wrote: >>>> Hello, >>>> >>>> Please review the fix for the following issue: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8251499 >>>> >>>> The proposed changeset is located at: >>>> >>>> https://cr.openjdk.java.net/~naoto/8251499/webrev.00/ >>>> >>>> The current implementation of CompactNumberFormat assumes that there >>>> is always the number placeholder part in compact patterns. This is >>>> not always true. In fact, upcoming CLDR 38 resurrects such patterns, >>>> so this fix is a precursor to support CLDR 38. >>>> >>>> Naoto >>> > From huizhe.wang at oracle.com Tue Aug 18 21:13:13 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 18 Aug 2020 14:13:13 -0700 Subject: RFR: 8251499: no-placeholder compact number patterns throw IllegalArgumentException In-Reply-To: References: <5b81c727-b1e0-0614-c68e-5568f98d9d6f@oracle.com> <217ef621-2293-3a1b-827c-06d067ea8b36@oracle.com> <9ee7bb2e-8030-b3a5-c5f6-6aa1517deb5c@oracle.com> Message-ID: <18130879-d9ad-a5a7-611f-138a3b3f27bd@oracle.com> Hi Naoto, That's nice!? The change looks good to me. Regards, Joe On 8/18/20 11:37 AM, naoto.sato at oracle.com wrote: > Hi Joe, > > Thank you for your comment. I consolidated those duplicated pieces > into one piece. Did not make it a private method, though, as it would > need to return two results from the method (cnfMultiplier and whether > to return immediately for no-placeholder cases). > > https://cr.openjdk.java.net/~naoto/8251499/webrev.02/ > > Naoto > > On 8/17/20 11:52 PM, Joe Wang wrote: >> Hi Naoto, >> >> Looks good overall. One nit, blocks 1633-1639 and 1642-1649 may share >> a common private method with a parameter that takes either >> matchedPosIndex or matchedNegIndex, if you want. >> >> Best, >> Joe >> >> On 8/17/20 4:42 PM, naoto.sato at oracle.com wrote: >>> Hi Joe, >>> >>> It turned out that the previous fix did not address plural format >>> cases. That means that just making the divisor negative to indicate >>> non-placeholder cannot distinguish multiple plural cases with the >>> same divisor. Instead, I created a list of placeholders (minimum >>> digits) for each index and count. Here is the updated webrev: >>> >>> http://cr.openjdk.java.net/~naoto/8251499/webrev.01/ >>> >>> I added a new test case (COMPACT_PATTERN14), which actually is >>> extracted from CLDR 38 Somali locale that demonstrates the issue. >>> I'd appreciate your further review. >>> >>> Naoto >>> >>> On 8/14/20 6:21 PM, Joe Wang wrote: >>>> Hi Naoto, >>>> >>>> Looks good to me. >>>> >>>> While a negative divisor representing no zeros is newly introduced, >>>> the "divisor > 0" checks seem to have always been beneficial.? I >>>> had to count the number of ""s in COMPACT_PATTERN13 :-) >>>> >>>> Have a great weekend! >>>> Joe >>>> >>>> On 8/14/2020 3:20 PM, naoto.sato at oracle.com wrote: >>>>> Hello, >>>>> >>>>> Please review the fix for the following issue: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8251499 >>>>> >>>>> The proposed changeset is located at: >>>>> >>>>> https://cr.openjdk.java.net/~naoto/8251499/webrev.00/ >>>>> >>>>> The current implementation of CompactNumberFormat assumes that >>>>> there is always the number placeholder part in compact patterns. >>>>> This is not always true. In fact, upcoming CLDR 38 resurrects such >>>>> patterns, so this fix is a precursor to support CLDR 38. >>>>> >>>>> Naoto >>>> >> From yumin.qi at oracle.com Wed Aug 19 05:14:00 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 18 Aug 2020 22:14:00 -0700 Subject: Fwd: RFR: 8247536: Support pre-generated MethodHandle lambda forms in CDS In-Reply-To: <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> Message-ID: Hi, Calvin ? I have updated the webrev at http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ On 8/12/20 11:54 AM, calvin.cheung at oracle.com wrote: > Hi Yumin, > > I reviewed mostly the native code. Below are my comments: > > 1) classListParser.hpp > > 71?? bool??????????????? _lambda_format; > No longer? used in new patch. > The above name is too generic. How about _lambda_form or _is_lambda_form? > If you rename the above, please also rename the function which returns the above bool. > > 2) jvm.cpp > > 3850 JVM_ENTRY(void, JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line)) > > ignore -> ignored done. > > 3) jvm.hpp > > 210 JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line); > > Same comment as for jvm.cpp > done > 4) metaspaceShared.cpp > > 2017?? size_t i = 0; > 2018?? while (i < size) { > 2019???? full_name[i++] = *start++; > 2020?? } > > Could the above be simplified to the following? > > ??? strncpy(full_name, start, size - 1); > I could use strncpy, but think it is not as efficient as this version since it does the same thing, and call c library function: The call chain for strcpy -> memcpy -> above code. > 2029 char* class_name = get_full_class_name(path_name); > > Should os::free(class_name) be called before the function returns? > In fact it does not matter, since we exit after dump. > 1870 static GrowableArray* lambda_list = NULL; > > The name lambda_list is a bit generic, how about lambda_form_list? > > 2112?????? lambda_list->append(parser.current_line()); > > Since parser.current_line() does a strdup, do those buffer need to be freed after its use? (i.e. after the call to regenerate_holder_classes()?) > > In MetaspaceShared::regenerate_holder_classes, before calling up to java, I think it's better to strip the prefix "@lambda-form-invoker" from the strings. So that the java InvokerBytecodeGeneratorHelper.readTraceConfig method doesn't need to handle it: > > ?143???????????????????? case "[LF_RESOLVE]": > ?144???????????????????? case InvokerBytecodeGenerator.CDS_LOG_PREFIX: > See new version in the new patch. No call for free due to the same reason: there about 100+ allocations, not a big allocation, and after dump we exit. > 5) DumpSymbolAndStringTable.java > > 37???? private static final String my_string = "DumpSymbolAndStringTable"; > > Unused variable? > It is used, the string will be in stringTable and output should contain it. Thanks Yumin > thanks, > > Calvin > > On 8/11/20 10:36 AM, Yumin Qi wrote: >> Forget to send to @core-lib-dev, the patch changed jdk code. >> >> >> Thanks >> >> Yumin >> >> >> >> -------- Forwarded Message -------- >> Subject:???? RFR: 8247536: Support pre-generated MethodHandle lambda forms in CDS >> Date:???? Tue, 11 Aug 2020 07:44:34 -0700 >> From:???? Yumin Qi >> To:???? hotspot-runtime-dev at openjdk.java.net >> >> >> >> Hi, Please reivew >> >> ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 >> ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ >> >> ? Summary: CDS does not archive pre-generated lambda form classes for method handles: >> >> [0.142s][info][class,load] java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: __JVM_LookupDefineClass__ >> >> The method handle resolution if not found in the holder class, a class with the method will be generated and loaded as vm internal created class and not archived like above. Those class names are mangled as combination of the class name and the class instance address. >> >> In this patch, collect those method holder class names and their associated methods' signatures when user specify DumpLoadedClassList, and record them in the log file in a format similar to the output format from traced by -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use another name for CDS: -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. The line prefix also changed from "[LF_RESOLVE]" to "@lambda-invoke-handle".? At dump time, regenerate the holder class with those methods and replace the existing holder class and archived it. At runtime, the resolution of the holder class which contains those methods are loaded from CDS archive so avoid regenerating them again. The patch reorganized the code for Jlink plugin, so the new added InvokerBytecodeGeneratorHelper can be shared both by JLink plugin and CDS code. Also change made to the mangled generated class name at static dump, the class name combining the class name and a sequentially ordered >> number instead of the class instance address, which looks like a random. >> >> >> To use this feature, one can do the dump/run just like done for a static dump/run, so no extra steps required. >> >> ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp >> >> ????? DumpLoadedClassList will turn on -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true >> >> ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist -XX:SharedArchiveFile=my.jsa JavaApp >> >> ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp >> >> ? The performance on javac HelloWorld.java (javac-benchmark), no patch vs patch: >> >> ?? 1:?? 2689285002? 2641821474 (-47463528)????? ----???? 391.720 382.990 ( -8.730)????? ---- >> ?? 2:?? 2687495085? 2632969688 (-54525397)????? ----???? 391.030 381.480 ( -9.550)????? ----- >> ?? 3:?? 2694664066? 2636523114 (-58140952)????? -----??? 390.610 382.550 ( -8.060)????? ---- >> ?? 4:?? 2686554164? 2639355233 (-47198931)????? ----???? 390.700 383.390 ( -7.310)????? --- >> ?? 5:?? 2691072338? 2633016687 (-58055651)????? -----??? 388.990 382.360 ( -6.630)????? --- >> ?? 6:?? 2684448174? 2644191854 (-40256320)????? ---????? 389.450 382.990 ( -6.460)????? --- >> ?? 7:?? 2694921227? 2630505090 (-64416137)????? -----??? 389.300 383.160 ( -6.140)????? --- >> ?? 8:?? 2685209712? 2639334320 (-45875392)????? ----???? 388.370 381.060 ( -7.310)????? --- >> ?? 9:?? 2695885942? 2640618655 (-55267287)????? ----???? 389.560 381.100 ( -8.460)????? ---- >> ? 10:?? 2689162942? 2635658943 (-53503999)????? ----???? 389.690 379.110 (-10.580)????? ----- >> ============================================================ >> ??????? 2689866989? 2637396210 (-52470778)????? ----???? 389.941 382.017 ( -7.924)????? ---- >> >> instr delta =??? -52470778??? -1.9507% >> >> time? delta =?????? -7.924 ms -2.0321% >> >> ? It will show much more improvement if compare with the default archive: >> >> ?????????? ? ? ? no patch??????????????????????? patch >> >> instr ??? ? ? 3,996,847,605??????? 3,320,928,995 >> >> time(s).???? 0.686835162 ? ? ? ?? 0.474933546 >> >> >> ? Tests: >> >> 1)jtreg on jdk/tools and hotspot/runtime local. >> >> 2) mach5 ter1,2 >> >> >> Thanks >> >> Yumin >> >> ? ` From yumin.qi at oracle.com Wed Aug 19 05:16:51 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 18 Aug 2020 22:16:51 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <9f1452b2-796a-1bf2-ab61-f0019d3192d3@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <2fc105ce-adaa-be64-fddf-c2f4b6293c54@oracle.com> <9f1452b2-796a-1bf2-ab61-f0019d3192d3@oracle.com> Message-ID: <7a3028db-2797-c38d-c334-ddd7274058b4@oracle.com> HI, Sundar, David ? Thanks. ? I have update the webrev at the same link: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ according Sundar's suggestion. ? Thanks. ? Yumin On 8/17/20 7:39 PM, sundararajan.athijegannathan at oracle.com wrote: > Hi David. > > Thanks. > > -Sundar > > On 18/08/20 8:04 am, David Holmes wrote: >> Hi Sundar, >> >> On 18/08/2020 12:25 pm, sundararajan.athijegannathan at oracle.com wrote: >>> Not a full review of fresh changes. But couple of comments: >>> >>> * src/hotspot/share/memory/lambdaFormInvokers.cpp and src/hotspot/share/memory/lambdaFormInvokers.hpp miss "Classpath exception" clause in the copyright header >> >> Hotspot files do not use the Classpath exception. That is for .java sources that will be"linked against" by applications. >> >> Cheers, >> David >> >>> * I had suggested a change GenerateJLIClassesPlugin.java in last round of review. That's still applicable. >>> >>> https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068160.html >>> >>> -Sundar >>> >>> On 18/08/20 1:07 am, Yumin Qi wrote: >>>> Hi, Ioi >>>> >>>> ? Thanks for review/suggestion. I have updated the webrev at the following link: >>>> >>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>>> >>>> ? Following changes done in this patch: >>>> >>>> ? 1) move regenerating holder classes into new added file: lambdaFormInvokers.[ch]pp >>>> >>>> ? "@lambda-form-invoker" tag only observed at jvm side. >>>> >>>> ? 2) remove InvokerBytecodeGeneratorHelper.java, move related functions to existing class, GenerateJLIClassesHelper.java >>>> >>>> ? 3) add tracing for SPECIES_RESOLVE, so archive more classes. >>>> >>>> ? 4) remove InvokerGenerateBytesException.java, using RuntimeException instead. >>>> >>>> ? 5) Changed make file to exclude the new added lamdaFormInvokers.cpp from the non-cds building list. >>>> >>>> ? 6) There is a bug in previous patch, jobject (typeArrayOop) should not be used directly for loading class since GC will move the java object. Fixed by copying the bytes from handle to C heap. >>>> >>>> ? 7) Tested with tier1,2 >>>> >>>> >>>> Thanks >>>> >>>> Yumin >>>> >>>> >>>> >>>> On 8/15/20 6:27 PM, Ioi Lam wrote: >>>>> On 8/15/20 6:19 PM, Ioi Lam wrote: >>>>>> To better capture what we're trying to do in this RFE, I've changed the RFE title (and the subject of this email thread) to >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8247536 >>>>>> Support for pre-generated java.lang.invoke classes in CDS static archive >>>>>> >>>>>> I also update the RFE Description to give an overview of the problem and the solution. >>>>>> >>>>>> The design document is also updated to reflect Yumin's latest implementation >>>>>> >>>>> >>>>> Oops, sent too quickly. The design doc's title is also updated: >>>>> >>>>> https://wiki.openjdk.java.net/display/HotSpot/Support+for+pre-generated+java.lang.invoke+classes+in+CDS+archive >>>>> >>>>>> Thanks >>>>>> - Ioi >>>>>> >>>>>> On 8/12/20 11:54 AM, calvin.cheung at oracle.com wrote: >>>>>>> Hi Yumin, >>>>>>> >>>>>>> I reviewed mostly the native code. Below are my comments: >>>>>>> >>>>>>> 1) classListParser.hpp >>>>>>> >>>>>>> 71?? bool??????????????? _lambda_format; >>>>>>> >>>>>>> The above name is too generic. How about _lambda_form or _is_lambda_form? >>>>>>> If you rename the above, please also rename the function which returns the above bool. >>>>>>> >>>>>>> 2) jvm.cpp >>>>>>> >>>>>>> 3850 JVM_ENTRY(void, JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line)) >>>>>>> >>>>>>> ignore -> ignored >>>>>>> >>>>>>> 3) jvm.hpp >>>>>>> >>>>>>> 210 JVM_CDSTraceResolve(JNIEnv* env, jclass ignore, jstring line); >>>>>>> >>>>>>> Same comment as for jvm.cpp >>>>>>> >>>>>>> 4) metaspaceShared.cpp >>>>>>> >>>>>>> 2017?? size_t i = 0; >>>>>>> 2018?? while (i < size) { >>>>>>> 2019???? full_name[i++] = *start++; >>>>>>> 2020?? } >>>>>>> >>>>>>> Could the above be simplified to the following? >>>>>>> >>>>>>> ??? strncpy(full_name, start, size - 1); >>>>>>> >>>>>>> 2029?? char* class_name = get_full_class_name(path_name); >>>>>>> >>>>>>> Should os::free(class_name) be called before the function returns? >>>>>>> >>>>>>> 1870 static GrowableArray* lambda_list = NULL; >>>>>>> >>>>>>> The name lambda_list is a bit generic, how about lambda_form_list? >>>>>>> >>>>>>> 2112 lambda_list->append(parser.current_line()); >>>>>>> >>>>>>> Since parser.current_line() does a strdup, do those buffer need to be freed after its use? (i.e. after the call to regenerate_holder_classes()?) >>>>>>> >>>>>>> In MetaspaceShared::regenerate_holder_classes, before calling up to java, I think it's better to strip the prefix "@lambda-form-invoker" from the strings. So that the java InvokerBytecodeGeneratorHelper.readTraceConfig method doesn't need to handle it: >>>>>>> >>>>>>> ?143???????????????????? case "[LF_RESOLVE]": >>>>>>> ?144???????????????????? case InvokerBytecodeGenerator.CDS_LOG_PREFIX: >>>>>>> >>>>>>> 5) DumpSymbolAndStringTable.java >>>>>>> >>>>>>> 37???? private static final String my_string = "DumpSymbolAndStringTable"; >>>>>>> >>>>>>> Unused variable? >>>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> Calvin >>>>>>> >>>>>>> On 8/11/20 10:36 AM, Yumin Qi wrote: >>>>>>>> Forget to send to @core-lib-dev, the patch changed jdk code. >>>>>>>> >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> Yumin >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -------- Forwarded Message -------- >>>>>>>> Subject:???? RFR: 8247536: Support pre-generated MethodHandle lambda forms in CDS >>>>>>>> Date:???? Tue, 11 Aug 2020 07:44:34 -0700 >>>>>>>> From:???? Yumin Qi >>>>>>>> To:???? hotspot-runtime-dev at openjdk.java.net >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Hi, Please reivew >>>>>>>> >>>>>>>> ? bug: https://bugs.openjdk.java.net/browse/JDK-8247536 >>>>>>>> ? Webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-01/ >>>>>>>> >>>>>>>> ? Summary: CDS does not archive pre-generated lambda form classes for method handles: >>>>>>>> >>>>>>>> [0.142s][info][class,load] java.lang.invoke.LambdaForm$MH/0x0000000800066c40 source: __JVM_LookupDefineClass__ >>>>>>>> >>>>>>>> The method handle resolution if not found in the holder class, a class with the method will be generated and loaded as vm internal created class and not archived like above. Those class names are mangled as combination of the class name and the class instance address. >>>>>>>> >>>>>>>> In this patch, collect those method holder class names and their associated methods' signatures when user specify DumpLoadedClassList, and record them in the log file in a format similar to the output format from traced by -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, but here use another name for CDS: -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true instead. The line prefix also changed from "[LF_RESOLVE]" to "@lambda-invoke-handle".? At dump time, regenerate the holder class with those methods and replace the existing holder class and archived it. At runtime, the resolution of the holder class which contains those methods are loaded from CDS archive so avoid regenerating them again. The patch reorganized the code for Jlink plugin, so the new added InvokerBytecodeGeneratorHelper can be shared both by JLink plugin and CDS code. Also change made to the mangled generated class name at static dump, the class name combining the class name and a sequentially >>>>>>>> ordered number instead of the class instance address, which looks like a random. >>>>>>>> >>>>>>>> >>>>>>>> To use this feature, one can do the dump/run just like done for a static dump/run, so no extra steps required. >>>>>>>> >>>>>>>> ? 1) java -XX:DumpLoadedClassList=myclasslist JavaApp >>>>>>>> >>>>>>>> ????? DumpLoadedClassList will turn on -Djava.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true >>>>>>>> >>>>>>>> ? 2) java -Xshare:dump -XX:SharedClassListFile=myclasslist -XX:SharedArchiveFile=my.jsa JavaApp >>>>>>>> >>>>>>>> ? 3) java -Xshare:on -XX:SharedArchiveFile=my.jsa JavaApp >>>>>>>> >>>>>>>> ? The performance on javac HelloWorld.java (javac-benchmark), no patch vs patch: >>>>>>>> >>>>>>>> ?? 1:?? 2689285002? 2641821474 (-47463528)????? ---- 391.720 382.990 ( -8.730)????? ---- >>>>>>>> ?? 2:?? 2687495085? 2632969688 (-54525397)????? ---- 391.030 381.480 ( -9.550)????? ----- >>>>>>>> ?? 3:?? 2694664066? 2636523114 (-58140952) ----- 390.610 382.550 ( -8.060)????? ---- >>>>>>>> ?? 4:?? 2686554164? 2639355233 (-47198931)????? ---- 390.700 383.390 ( -7.310)????? --- >>>>>>>> ?? 5:?? 2691072338? 2633016687 (-58055651) ----- 388.990 382.360 ( -6.630)????? --- >>>>>>>> ?? 6:?? 2684448174? 2644191854 (-40256320)????? --- 389.450 382.990 ( -6.460)????? --- >>>>>>>> ?? 7:?? 2694921227? 2630505090 (-64416137) ----- 389.300 383.160 ( -6.140)????? --- >>>>>>>> ?? 8:?? 2685209712? 2639334320 (-45875392)????? ---- 388.370 381.060 ( -7.310)????? --- >>>>>>>> ?? 9:?? 2695885942? 2640618655 (-55267287)????? ---- 389.560 381.100 ( -8.460)????? ---- >>>>>>>> ? 10:?? 2689162942? 2635658943 (-53503999)????? ---- 389.690 379.110 (-10.580)????? ----- >>>>>>>> ============================================================ >>>>>>>> ??????? 2689866989? 2637396210 (-52470778)????? ---- 389.941 382.017 ( -7.924)????? ---- >>>>>>>> >>>>>>>> instr delta =??? -52470778??? -1.9507% >>>>>>>> >>>>>>>> time? delta =?????? -7.924 ms -2.0321% >>>>>>>> >>>>>>>> ? It will show much more improvement if compare with the default archive: >>>>>>>> >>>>>>>> ?????????? ? ? ? no patch patch >>>>>>>> >>>>>>>> instr ??? ? ? 3,996,847,605??????? 3,320,928,995 >>>>>>>> >>>>>>>> time(s).???? 0.686835162 ? ? ? ?? 0.474933546 >>>>>>>> >>>>>>>> >>>>>>>> ? Tests: >>>>>>>> >>>>>>>> 1)jtreg on jdk/tools and hotspot/runtime local. >>>>>>>> >>>>>>>> 2) mach5 ter1,2 >>>>>>>> >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> Yumin >>>>>>>> >>>>>>>> ? ` >>>>>> >>>>> From galder at redhat.com Wed Aug 19 11:53:23 2020 From: galder at redhat.com (Galder Zamarreno) Date: Wed, 19 Aug 2020 13:53:23 +0200 Subject: RFR: JDK-8251397 NPE on ClassValue.ClassValueMap.cacheArray Message-ID: Hi all, I've created patch [1] to fix the ClassValue$ClassValueMap NPE bug in [2]. The bug has been discussed by other members in the list in thread [3]. The patch follows the simple fix suggested by Doug and others in that exchange, e.g. [4]. That is, it adds a release fence to ClassValue$ClassValueMap constructor to avoid the NPE. To verify the fix, I ran the jcstress test that Paul posted in [5] and played around with the difference fixes suggested in the thread. Adding the release fence did indeed fix the jcstress test. To further verify the issue, I've successfully run both the tier1 tests and the Quarkus native testsuite with a Mandrel 20.1 built with JDK 11.0.8 version patched with the fix (higher JDKs not supported yet). Note that this NPE happens on rare occasions. The patch applies cleanly to JDK 11. Galder [1] Webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/galder/JDK-8251397/01/webrev/ [2] Bug: https://bugs.openjdk.java.net/browse/JDK-8251397 [3] https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068086.html [4] https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068126.html [5] https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068110.html From galder at redhat.com Wed Aug 19 13:12:54 2020 From: galder at redhat.com (Galder Zamarreno) Date: Wed, 19 Aug 2020 15:12:54 +0200 Subject: Possible subtle memory model error in ClassValue In-Reply-To: <5c0e242b-59a7-afd1-710a-606e541dd628@cs.oswego.edu> References: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> <33C0A687-7B1D-49DB-86F8-CE778BC1456A@oracle.com> <464fc65f-308d-5636-8c83-4592257e96d9@oracle.com> <5c0e242b-59a7-afd1-710a-606e541dd628@cs.oswego.edu> Message-ID: On Mon, Aug 10, 2020 at 2:19 PM Doug Lea

    wrote: > Catching up... > > As implied in other posts, the minimal fix is to add a trailing release > fence (using Unsafe?) to the constructor. FYI I have sent an RFR with the proposed fix ^ https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068244.html > Or less delicately, to access > only using acquire/release (which will cost a bit on ARM/Power, but > probably not noticeable on x86), or most simply (but expensively) to > declare the field volatile. > > Also, as Hans noted, the consensus seems to be that there not enough to > be gained by always adding a release fence to constructors. A few errors > like this might never occur, but other related anomalies with non-final > field accesses would remain. > > -Doug > > From Roger.Riggs at oracle.com Wed Aug 19 15:33:59 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Wed, 19 Aug 2020 11:33:59 -0400 Subject: RFR: 8251499: no-placeholder compact number patterns throw IllegalArgumentException In-Reply-To: <18130879-d9ad-a5a7-611f-138a3b3f27bd@oracle.com> References: <5b81c727-b1e0-0614-c68e-5568f98d9d6f@oracle.com> <217ef621-2293-3a1b-827c-06d067ea8b36@oracle.com> <9ee7bb2e-8030-b3a5-c5f6-6aa1517deb5c@oracle.com> <18130879-d9ad-a5a7-611f-138a3b3f27bd@oracle.com> Message-ID: <62419dd1-655b-04ad-adeb-6773df85cfbf@oracle.com> Hi Naoto, CompactNumberFormat.java: 269: The field "placeHolders" should be named consistently with the other holders of Patterns. ? -> placeHolderPatterns 1632: missing space before ":" 2390:? I'm not sure why the stream processing is preferable to a direct forEach(...). TestCompactPatternsValidity: For the Plurals cases it would clearer to create a new Data provider and corresponding tests with the extra arg. Thanks, Roger On 8/18/20 5:13 PM, Joe Wang wrote: > Hi Naoto, > > That's nice!? The change looks good to me. > > Regards, > Joe > > On 8/18/20 11:37 AM, naoto.sato at oracle.com wrote: >> Hi Joe, >> >> Thank you for your comment. I consolidated those duplicated pieces >> into one piece. Did not make it a private method, though, as it would >> need to return two results from the method (cnfMultiplier and whether >> to return immediately for no-placeholder cases). >> >> https://cr.openjdk.java.net/~naoto/8251499/webrev.02/ >> >> Naoto >> >> On 8/17/20 11:52 PM, Joe Wang wrote: >>> Hi Naoto, >>> >>> Looks good overall. One nit, blocks 1633-1639 and 1642-1649 may >>> share a common private method with a parameter that takes either >>> matchedPosIndex or matchedNegIndex, if you want. >>> >>> Best, >>> Joe >>> >>> On 8/17/20 4:42 PM, naoto.sato at oracle.com wrote: >>>> Hi Joe, >>>> >>>> It turned out that the previous fix did not address plural format >>>> cases. That means that just making the divisor negative to indicate >>>> non-placeholder cannot distinguish multiple plural cases with the >>>> same divisor. Instead, I created a list of placeholders (minimum >>>> digits) for each index and count. Here is the updated webrev: >>>> >>>> http://cr.openjdk.java.net/~naoto/8251499/webrev.01/ >>>> >>>> I added a new test case (COMPACT_PATTERN14), which actually is >>>> extracted from CLDR 38 Somali locale that demonstrates the issue. >>>> I'd appreciate your further review. >>>> >>>> Naoto >>>> >>>> On 8/14/20 6:21 PM, Joe Wang wrote: >>>>> Hi Naoto, >>>>> >>>>> Looks good to me. >>>>> >>>>> While a negative divisor representing no zeros is newly >>>>> introduced, the "divisor > 0" checks seem to have always been >>>>> beneficial.? I had to count the number of ""s in COMPACT_PATTERN13 >>>>> :-) >>>>> >>>>> Have a great weekend! >>>>> Joe >>>>> >>>>> On 8/14/2020 3:20 PM, naoto.sato at oracle.com wrote: >>>>>> Hello, >>>>>> >>>>>> Please review the fix for the following issue: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8251499 >>>>>> >>>>>> The proposed changeset is located at: >>>>>> >>>>>> https://cr.openjdk.java.net/~naoto/8251499/webrev.00/ >>>>>> >>>>>> The current implementation of CompactNumberFormat assumes that >>>>>> there is always the number placeholder part in compact patterns. >>>>>> This is not always true. In fact, upcoming CLDR 38 resurrects >>>>>> such patterns, so this fix is a precursor to support CLDR 38. >>>>>> >>>>>> Naoto >>>>> >>> > From Roger.Riggs at oracle.com Wed Aug 19 15:40:44 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Wed, 19 Aug 2020 11:40:44 -0400 Subject: Fix for Javadoc errors in java.base In-Reply-To: <6b31a7d4-3bcd-2c8d-2f3c-4158047c3e31@oracle.com> References: <318a4bbf-1b3e-d34e-d69a-d1a2ee233794@oracle.com> <8e978983-4ec1-8421-2563-d00c61f0d55e@oracle.com> <11adede2-1119-f133-020f-4ddd60c9fc67@oracle.com> <6b31a7d4-3bcd-2c8d-2f3c-4158047c3e31@oracle.com> Message-ID: <00ffb7f0-5301-1f34-93e7-8e3249845434@oracle.com> Looks fine Julia On 8/18/20 1:02 PM, Julia Boes wrote: > > Hi, > > The two changes below still need to be reviewed. Any takers? > > Cheers, > > Julia > >> --- >> old/src/java.base/share/classes/java/lang/invoke/AbstractValidatingLambdaMetafactory.java2020-08-14 >> 23:55:41.953638446 +0530 >> +++ >> new/src/java.base/share/classes/java/lang/invoke/AbstractValidatingLambdaMetafactory.java2020-08-14 >> 23:55:41.445619497 +0530 >> @@ -1,5 +1,5 @@ >> ?/* >> - * Copyright (c) 2012, 2013, Oracle and/or its affiliates. All >> rights reserved. >> + * Copyright (c) 2012, 2020, Oracle and/or its affiliates. All >> rights reserved. >> ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >> ? * >> ? * This code is free software; you can redistribute it and/or modify it >> @@ -208,7 +208,7 @@ >> ? ? ? * >> ? ? ? * @return a CallSite, which, when invoked, will return an >> instance of the >> ? ? ? * functional interface >> - ? ? * @throws ReflectiveOperationException >> + ? ? * @throws LambdaConversionException >> ? ? ? */ >> ? ? ?abstract CallSite buildCallSite() >> ? ? ? ? ? ? ?throws LambdaConversionException; >> --- >> old/src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java2020-08-14 >> 23:55:42.977677096 +0530 >> +++ >> new/src/java.base/share/classes/java/lang/invoke/InnerClassLambdaMetafactory.java2020-08-14 >> 23:55:42.473658062 +0530 >> @@ -200,7 +200,6 @@ >> ? ? ? * >> ? ? ? * @return a CallSite, which, when invoked, will return an >> instance of the >> ? ? ? * functional interface >> - ? ? * @throws ReflectiveOperationException >> ? ? ? * @throws LambdaConversionException If properly formed >> functional interface >> ? ? ? * is not found >> ? ? ? */ From paul.sandoz at oracle.com Wed Aug 19 15:57:59 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 19 Aug 2020 08:57:59 -0700 Subject: RFR: JDK-8251397 NPE on ClassValue.ClassValueMap.cacheArray In-Reply-To: References: Message-ID: <99F198CA-B83B-4399-B337-B979570791AD@oracle.com> Hi Galder, Thanks for doing the fix and the work verifying. I also verified using a fence fixes the jcstress test. Similarly, I found I could only reproduce the issue in HotSpot when running a test more like ClassValue (the double checked locking pattern when publishing to a plan field) with the options -XX:+StressLCM -XX:+StressGCM. I analyzed assembler output (-XX:+PrintOptoAssembly) to observe the publish store occurring before some stores to fields. It?s highly likely that with HotSpot we got lucky in this case :-) I don?t object to the use of a release fence (LoadStore|StoreStore), but I believe we could also use a StoreStore fence in this case. Perhaps the ARM folks have a stronger opinion on this? I marginally prefer placing the fence in the initializeMap, since the relationship between construction and publish is very clear. In either case I recommend adding a comment on why the fence is required e.g.: // Place a Store fence as the last operation of the constructor to emulate // ClassValueMap containing final fields. This ensures it can be // published safely in the non-volatile field Class.classValueMap, (see initializeMap) // since stores to the fields of ClassValueMap will not be reordered // to occur after the store to the field type.classValueMap I also noticed we can change the field Class.classValueMap to be @Stable, but I think we should follow up on that investigation separately. Relatedly, I had an idea to modify HotSpot so it places a StoreStore fence directly before the store to a @Stable field. Paul. > On Aug 19, 2020, at 4:53 AM, Galder Zamarreno wrote: > > Hi all, > > I've created patch [1] to fix the ClassValue$ClassValueMap NPE bug in [2]. > > The bug has been discussed by other members in the list in thread [3]. > > The patch follows the simple fix suggested by Doug and others in that > exchange, e.g. [4]. That is, it adds a release fence > to ClassValue$ClassValueMap constructor to avoid the NPE. > > To verify the fix, I ran the jcstress test that Paul posted in [5] and > played around with the difference fixes suggested in the thread. Adding the > release fence did indeed fix the jcstress test. > > To further verify the issue, I've successfully run both the tier1 tests and > the Quarkus native testsuite with a Mandrel 20.1 built with JDK 11.0.8 > version patched with the fix (higher JDKs not supported yet). Note that > this NPE happens on rare occasions. > > The patch applies cleanly to JDK 11. > > Galder > > [1] Webrev: > http://cr.openjdk.java.net/~sgehwolf/webrevs/galder/JDK-8251397/01/webrev/ > [2] Bug: https://bugs.openjdk.java.net/browse/JDK-8251397 > [3] > https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068086.html > [4] > https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068126.html > [5] > https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068110.html From hboehm at google.com Wed Aug 19 16:55:06 2020 From: hboehm at google.com (Hans Boehm) Date: Wed, 19 Aug 2020 09:55:06 -0700 Subject: Possible subtle memory model error in ClassValue In-Reply-To: References: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> <33C0A687-7B1D-49DB-86F8-CE778BC1456A@oracle.com> <464fc65f-308d-5636-8c83-4592257e96d9@oracle.com> <5c0e242b-59a7-afd1-710a-606e541dd628@cs.oswego.edu> Message-ID: If I understand the code correctly, adding the fence should fix this for current implementations. But it is not correct by current language rules, and may conceivably break in the future unless the implementation enforces stronger rules. On Wed, Aug 19, 2020 at 6:13 AM Galder Zamarreno wrote: > On Mon, Aug 10, 2020 at 2:19 PM Doug Lea
    wrote: > > > Catching up... > > > > As implied in other posts, the minimal fix is to add a trailing release > > fence (using Unsafe?) to the constructor. > > > FYI I have sent an RFR with the proposed fix ^ > > https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068244.html > > > > Or less delicately, to access > > only using acquire/release (which will cost a bit on ARM/Power, but > > probably not noticeable on x86), or most simply (but expensively) to > > declare the field volatile. > > > > Also, as Hans noted, the consensus seems to be that there not enough to > > be gained by always adding a release fence to constructors. A few errors > > like this might never occur, but other related anomalies with non-final > > field accesses would remain. > > > > -Doug > > > > > From paul.sandoz at oracle.com Wed Aug 19 17:28:30 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 19 Aug 2020 10:28:30 -0700 Subject: Possible subtle memory model error in ClassValue In-Reply-To: References: <8af3f786-c41a-13df-1827-03ae0915a005@redhat.com> <33C0A687-7B1D-49DB-86F8-CE778BC1456A@oracle.com> <464fc65f-308d-5636-8c83-4592257e96d9@oracle.com> <5c0e242b-59a7-afd1-710a-606e541dd628@cs.oswego.edu> Message-ID: <354AC877-3D83-4875-8E31-59920DC46C02@oracle.com> Yes. Core library code close to the JVM in OpenJDK (that in the java.base module, commonly that in java.lang.* close to that in hotspot) often assumes implementation specifics of the JVM. The two are closely coupled. There are other such implementation-specific contracts like @Stable, or ?final fields as really final", or certain intrinsics, hidden fields, and much of direct Unsafe usage etc. Paul. > On Aug 19, 2020, at 9:55 AM, Hans Boehm wrote: > > If I understand the code correctly, adding the fence should fix this for > current implementations. But it is not correct by current language rules, > and may conceivably break in the future unless the implementation enforces > stronger rules. > > On Wed, Aug 19, 2020 at 6:13 AM Galder Zamarreno wrote: > >> On Mon, Aug 10, 2020 at 2:19 PM Doug Lea
    wrote: >> >>> Catching up... >>> >>> As implied in other posts, the minimal fix is to add a trailing release >>> fence (using Unsafe?) to the constructor. >> >> >> FYI I have sent an RFR with the proposed fix ^ >> >> https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068244.html >> >> >>> Or less delicately, to access >>> only using acquire/release (which will cost a bit on ARM/Power, but >>> probably not noticeable on x86), or most simply (but expensively) to >>> declare the field volatile. >>> >>> Also, as Hans noted, the consensus seems to be that there not enough to >>> be gained by always adding a release fence to constructors. A few errors >>> like this might never occur, but other related anomalies with non-final >>> field accesses would remain. >>> >>> -Doug >>> >>> >> From cjashfor at linux.ibm.com Wed Aug 19 18:10:50 2020 From: cjashfor at linux.ibm.com (Corey Ashford) Date: Wed, 19 Aug 2020 11:10:50 -0700 Subject: RFR(M): 8248188: [PATCH] Add HotSpotIntrinsicCandidate and API for Base64 decoding In-Reply-To: References: <11ca749f-3015-c004-aa6b-3194e1dfe4eb@linux.ibm.com> Message-ID: Michihiro Horie posted up a new iteration of this webrev for me. This time the webrev includes a complete implementation of the intrinsic for Power9 and Power10. You can find it here: http://cr.openjdk.java.net/~mhorie/8248188/webrev.02/ Changes in webrev.02 vs. webrev.01: * The method header for the intrinsic in the Base64 code has been rewritten using the Javadoc style. The clarity of the comments has been improved and some verbosity has been removed. There are no additional functional changes to Base64.java. * The code needed to martial and check the intrinsic parameters has been added, using the base64 encodeBlock intrinsic as a guideline. * A complete intrinsic implementation for Power9 and Power10 is included. * Adds some Power9 and Power10 assembler instructions needed by the intrinsic which hadn't been defined before. The intrinsic implementation in this patch accelerates the decoding of large blocks of base64 data by a factor of about 3.5X on Power9. I'm attaching two Java test cases I am using for testing and benchmarking. The TestBase64_VB encodes and decodes randomly-sized buffers of random data and checks that original data matches the encoded-then-decoded data. TestBase64Errors encodes a 48K block of random bytes, then corrupts each byte of the encoded data, one at a time, checking to see if the decoder catches the illegal byte. Any comments/suggestions would be appreciated. Thanks, - Corey On 7/27/20 6:49 PM, Corey Ashford wrote: > Michihiro Horie uploaded a new revision of the Base64 decodeBlock > intrinsic API for me: > > http://cr.openjdk.java.net/~mhorie/8248188/webrev.01/ > > It has the following changes with respect to the original one posted: > > ?* In the event of encountering a non-base64 character, instead of > having a separate error code of -1, the intrinsic can now just return > either 0, or the number of data bytes produced up to the point where the > illegal base64 character was encountered.? This reduces the number of > special cases, and also provides a way to speed up the process of > finding the bad character by the slower, pure-Java algorithm. > > ?* The isMIME boolean is removed from the API for two reasons: > ?? - The current API is not sufficient to handle the isMIME case, > because there isn't a strict relationship between the number of input > bytes and the number of output bytes, because there can be an arbitrary > number of non-base64 characters in the source. > ?? - If an intrinsic only implements the (isMIME == false) case as ours > does, it will always return 0 bytes processed, which will slightly slow > down the normal path of processing an (isMIME == true) instantiation. > ?? - We considered adding a separate hotspot candidate for the (isMIME > == true) case, but since we don't have an intrinsic implementation to > test that, we decided to leave it as a future optimization. > > Comments and suggestions are welcome.? Thanks for your consideration. > > - Corey > > On 6/23/20 6:23 PM, Michihiro Horie wrote: >> Hi Corey, >> >> Following is the issue I created. >> https://bugs.openjdk.java.net/browse/JDK-8248188 >> >> I will upload a webrev when you're ready as we talked in private. >> >> Best regards, >> Michihiro >> >> Inactive hide details for "Corey Ashford" ---2020/06/24 >> 09:40:10---Currently in java.util.Base64, there is a >> HotSpotIntrinsicCa"Corey Ashford" ---2020/06/24 09:40:10---Currently >> in java.util.Base64, there is a HotSpotIntrinsicCandidate and API for >> encodeBlock, but no >> >> From: "Corey Ashford" >> To: "hotspot-compiler-dev at openjdk.java.net" >> , >> "ppc-aix-port-dev at openjdk.java.net" >> Cc: Michihiro Horie/Japan/IBM at IBMJP, Kazunori Ogata/Japan/IBM at IBMJP, >> joserz at br.ibm.com >> Date: 2020/06/24 09:40 >> Subject: RFR(S): [PATCH] Add HotSpotIntrinsicCandidate and API for >> Base64 decoding >> >> ------------------------------------------------------------------------ >> >> >> >> Currently in java.util.Base64, there is a HotSpotIntrinsicCandidate and >> API for encodeBlock, but none for decoding. ?This means that only >> encoding gets acceleration from the underlying CPU's vector hardware. >> >> I'd like to propose adding a new intrinsic for decodeBlock. ?The >> considerations I have for this new intrinsic's API: >> >> ??* Don't make any assumptions about the underlying capability of the >> hardware. ?For example, do not impose any specific block size >> granularity. >> >> ??* Don't assume the underlying intrinsic can handle isMIME or isURL >> modes, but also let them decide if they will process the data regardless >> of the settings of the two booleans. >> >> ??* Any remaining data that is not processed by the intrinsic will be >> processed by the pure Java implementation. ?This allows the intrinsic to >> process whatever block sizes it's good at without the complexity of >> handling the end fragments. >> >> ??* If any illegal character is discovered in the decoding process, the >> intrinsic will simply return -1, instead of requiring it to throw a >> proper exception from the context of the intrinsic. ?In the event of >> getting a -1 returned from the intrinsic, the Java Base64 library code >> simply calls the pure Java implementation to have it find the error and >> properly throw an exception. ?This is a performance trade-off in the >> case of an error (which I expect to be very rare). >> >> ??* One thought I have for a further optimization (not implemented in >> the current patch), is that when the intrinsic decides not to process a >> block because of some combination of isURL and isMIME settings it >> doesn't handle, it could return extra bits in the return code, encoded >> as a negative number. ?For example: >> >> Illegal_Base64_char ? = 0b001; >> isMIME_unsupported ? ?= 0b010; >> isURL_unsupported ? ? = 0b100; >> >> These can be OR'd together as needed and then negated (flip the sign). >> The Base64 library code could then cache these flags, so it will know >> not to call the intrinsic again when another decodeBlock is requested >> but with an unsupported mode. ?This will save the performance hit of >> calling the intrinsic when it is guaranteed to fail. >> >> I've tested the attached patch with an actual intrinsic coded up for >> Power9/Power10, but those runtime intrinsics and arch-specific patches >> aren't attached today. ?I want to get some consensus on the >> library-level intrinsic API first. >> >> Also attached is a simple test case to test that the new intrinsic API >> doesn't break anything. >> >> I'm open to any comments about this. >> >> Thanks for your consideration, >> >> - Corey >> >> >> Corey Ashford >> IBM Systems, Linux Technology Center, OpenJDK team >> cjashfor at us dot ibm dot com >> [attachment "decodeBlock_api-20200623.patch" deleted by Michihiro >> Horie/Japan/IBM] [attachment "TestBase64.java" deleted by Michihiro >> Horie/Japan/IBM] >> >> > From Roger.Riggs at oracle.com Wed Aug 19 18:20:13 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Wed, 19 Aug 2020 14:20:13 -0400 Subject: RFR(M): 8248188: [PATCH] Add HotSpotIntrinsicCandidate and API for Base64 decoding In-Reply-To: References: <11ca749f-3015-c004-aa6b-3194e1dfe4eb@linux.ibm.com> Message-ID: Hi Corey, For changes obviously performance motivated, it is conventional to run a JMH perf test to demonstate the improvement and prove it is worthwhile to add code complexity. I don't see any existing Base64 JMH tests but they would be in the repo below or near: ??? test/micro/org/openjdk/bench/java/util/ Please contribute a JMH test and results to show the difference. Regards, Roger On 8/19/20 2:10 PM, Corey Ashford wrote: > Michihiro Horie posted up a new iteration of this webrev for me.? This > time the webrev includes a complete implementation of the intrinsic > for Power9 and Power10. > > You can find it here: > http://cr.openjdk.java.net/~mhorie/8248188/webrev.02/ > > Changes in webrev.02 vs. webrev.01: > > ? * The method header for the intrinsic in the Base64 code has been > rewritten using the Javadoc style.? The clarity of the comments has > been improved and some verbosity has been removed. There are no > additional functional changes to Base64.java. > > ? * The code needed to martial and check the intrinsic parameters has > been added, using the base64 encodeBlock intrinsic as a guideline. > > ? * A complete intrinsic implementation for Power9 and Power10 is > included. > > ? * Adds some Power9 and Power10 assembler instructions needed by the > intrinsic which hadn't been defined before. > > The intrinsic implementation in this patch accelerates the decoding of > large blocks of base64 data by a factor of about 3.5X on Power9. > > I'm attaching two Java test cases I am using for testing and > benchmarking.? The TestBase64_VB encodes and decodes randomly-sized > buffers of random data and checks that original data matches the > encoded-then-decoded data.? TestBase64Errors encodes a 48K block of > random bytes, then corrupts each byte of the encoded data, one at a > time, checking to see if the decoder catches the illegal byte. > > Any comments/suggestions would be appreciated. > > Thanks, > > - Corey > > On 7/27/20 6:49 PM, Corey Ashford wrote: >> Michihiro Horie uploaded a new revision of the Base64 decodeBlock >> intrinsic API for me: >> >> http://cr.openjdk.java.net/~mhorie/8248188/webrev.01/ >> >> It has the following changes with respect to the original one posted: >> >> ??* In the event of encountering a non-base64 character, instead of >> having a separate error code of -1, the intrinsic can now just return >> either 0, or the number of data bytes produced up to the point where >> the illegal base64 character was encountered. This reduces the number >> of special cases, and also provides a way to speed up the process of >> finding the bad character by the slower, pure-Java algorithm. >> >> ??* The isMIME boolean is removed from the API for two reasons: >> ??? - The current API is not sufficient to handle the isMIME case, >> because there isn't a strict relationship between the number of input >> bytes and the number of output bytes, because there can be an >> arbitrary number of non-base64 characters in the source. >> ??? - If an intrinsic only implements the (isMIME == false) case as >> ours does, it will always return 0 bytes processed, which will >> slightly slow down the normal path of processing an (isMIME == true) >> instantiation. >> ??? - We considered adding a separate hotspot candidate for the >> (isMIME == true) case, but since we don't have an intrinsic >> implementation to test that, we decided to leave it as a future >> optimization. >> >> Comments and suggestions are welcome.? Thanks for your consideration. >> >> - Corey >> >> On 6/23/20 6:23 PM, Michihiro Horie wrote: >>> Hi Corey, >>> >>> Following is the issue I created. >>> https://bugs.openjdk.java.net/browse/JDK-8248188 >>> >>> I will upload a webrev when you're ready as we talked in private. >>> >>> Best regards, >>> Michihiro >>> >>> Inactive hide details for "Corey Ashford" ---2020/06/24 >>> 09:40:10---Currently in java.util.Base64, there is a >>> HotSpotIntrinsicCa"Corey Ashford" ---2020/06/24 09:40:10---Currently >>> in java.util.Base64, there is a HotSpotIntrinsicCandidate and API >>> for encodeBlock, but no >>> >>> From: "Corey Ashford" >>> To: "hotspot-compiler-dev at openjdk.java.net" >>> , >>> "ppc-aix-port-dev at openjdk.java.net" >>> Cc: Michihiro Horie/Japan/IBM at IBMJP, Kazunori Ogata/Japan/IBM at IBMJP, >>> joserz at br.ibm.com >>> Date: 2020/06/24 09:40 >>> Subject: RFR(S): [PATCH] Add HotSpotIntrinsicCandidate and API for >>> Base64 decoding >>> >>> ------------------------------------------------------------------------ >>> >>> >>> >>> >>> Currently in java.util.Base64, there is a HotSpotIntrinsicCandidate and >>> API for encodeBlock, but none for decoding. ?This means that only >>> encoding gets acceleration from the underlying CPU's vector hardware. >>> >>> I'd like to propose adding a new intrinsic for decodeBlock. ?The >>> considerations I have for this new intrinsic's API: >>> >>> ??* Don't make any assumptions about the underlying capability of the >>> hardware. ?For example, do not impose any specific block size >>> granularity. >>> >>> ??* Don't assume the underlying intrinsic can handle isMIME or isURL >>> modes, but also let them decide if they will process the data >>> regardless >>> of the settings of the two booleans. >>> >>> ??* Any remaining data that is not processed by the intrinsic will be >>> processed by the pure Java implementation. ?This allows the >>> intrinsic to >>> process whatever block sizes it's good at without the complexity of >>> handling the end fragments. >>> >>> ??* If any illegal character is discovered in the decoding process, the >>> intrinsic will simply return -1, instead of requiring it to throw a >>> proper exception from the context of the intrinsic. ?In the event of >>> getting a -1 returned from the intrinsic, the Java Base64 library code >>> simply calls the pure Java implementation to have it find the error and >>> properly throw an exception. ?This is a performance trade-off in the >>> case of an error (which I expect to be very rare). >>> >>> ??* One thought I have for a further optimization (not implemented in >>> the current patch), is that when the intrinsic decides not to process a >>> block because of some combination of isURL and isMIME settings it >>> doesn't handle, it could return extra bits in the return code, encoded >>> as a negative number. ?For example: >>> >>> Illegal_Base64_char ? = 0b001; >>> isMIME_unsupported ? ?= 0b010; >>> isURL_unsupported ? ? = 0b100; >>> >>> These can be OR'd together as needed and then negated (flip the sign). >>> The Base64 library code could then cache these flags, so it will know >>> not to call the intrinsic again when another decodeBlock is requested >>> but with an unsupported mode. ?This will save the performance hit of >>> calling the intrinsic when it is guaranteed to fail. >>> >>> I've tested the attached patch with an actual intrinsic coded up for >>> Power9/Power10, but those runtime intrinsics and arch-specific patches >>> aren't attached today. ?I want to get some consensus on the >>> library-level intrinsic API first. >>> >>> Also attached is a simple test case to test that the new intrinsic API >>> doesn't break anything. >>> >>> I'm open to any comments about this. >>> >>> Thanks for your consideration, >>> >>> - Corey >>> >>> >>> Corey Ashford >>> IBM Systems, Linux Technology Center, OpenJDK team >>> cjashfor at us dot ibm dot com >>> [attachment "decodeBlock_api-20200623.patch" deleted by Michihiro >>> Horie/Japan/IBM] [attachment "TestBase64.java" deleted by Michihiro >>> Horie/Japan/IBM] >>> >>> >> > From naoto.sato at oracle.com Wed Aug 19 18:26:04 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Wed, 19 Aug 2020 11:26:04 -0700 Subject: RFR: 8251499: no-placeholder compact number patterns throw IllegalArgumentException In-Reply-To: <62419dd1-655b-04ad-adeb-6773df85cfbf@oracle.com> References: <5b81c727-b1e0-0614-c68e-5568f98d9d6f@oracle.com> <217ef621-2293-3a1b-827c-06d067ea8b36@oracle.com> <9ee7bb2e-8030-b3a5-c5f6-6aa1517deb5c@oracle.com> <18130879-d9ad-a5a7-611f-138a3b3f27bd@oracle.com> <62419dd1-655b-04ad-adeb-6773df85cfbf@oracle.com> Message-ID: Hi Roger, Thank you for your comments. I've addressed your suggestions and created a new webrev below: https://cr.openjdk.java.net/~naoto/8251499/webrev.03/ Naoto On 8/19/20 8:33 AM, Roger Riggs wrote: > Hi Naoto, > > CompactNumberFormat.java: > > 269: The field "placeHolders" should be named consistently with the > other holders of Patterns. > ? -> placeHolderPatterns > > 1632: missing space before ":" > > 2390:? I'm not sure why the stream processing is preferable to a direct > forEach(...). > > TestCompactPatternsValidity: > For the Plurals cases it would clearer to create a new Data provider and > corresponding tests > with the extra arg. > > Thanks, Roger > > > > On 8/18/20 5:13 PM, Joe Wang wrote: >> Hi Naoto, >> >> That's nice!? The change looks good to me. >> >> Regards, >> Joe >> >> On 8/18/20 11:37 AM, naoto.sato at oracle.com wrote: >>> Hi Joe, >>> >>> Thank you for your comment. I consolidated those duplicated pieces >>> into one piece. Did not make it a private method, though, as it would >>> need to return two results from the method (cnfMultiplier and whether >>> to return immediately for no-placeholder cases). >>> >>> https://cr.openjdk.java.net/~naoto/8251499/webrev.02/ >>> >>> Naoto >>> >>> On 8/17/20 11:52 PM, Joe Wang wrote: >>>> Hi Naoto, >>>> >>>> Looks good overall. One nit, blocks 1633-1639 and 1642-1649 may >>>> share a common private method with a parameter that takes either >>>> matchedPosIndex or matchedNegIndex, if you want. >>>> >>>> Best, >>>> Joe >>>> >>>> On 8/17/20 4:42 PM, naoto.sato at oracle.com wrote: >>>>> Hi Joe, >>>>> >>>>> It turned out that the previous fix did not address plural format >>>>> cases. That means that just making the divisor negative to indicate >>>>> non-placeholder cannot distinguish multiple plural cases with the >>>>> same divisor. Instead, I created a list of placeholders (minimum >>>>> digits) for each index and count. Here is the updated webrev: >>>>> >>>>> http://cr.openjdk.java.net/~naoto/8251499/webrev.01/ >>>>> >>>>> I added a new test case (COMPACT_PATTERN14), which actually is >>>>> extracted from CLDR 38 Somali locale that demonstrates the issue. >>>>> I'd appreciate your further review. >>>>> >>>>> Naoto >>>>> >>>>> On 8/14/20 6:21 PM, Joe Wang wrote: >>>>>> Hi Naoto, >>>>>> >>>>>> Looks good to me. >>>>>> >>>>>> While a negative divisor representing no zeros is newly >>>>>> introduced, the "divisor > 0" checks seem to have always been >>>>>> beneficial.? I had to count the number of ""s in COMPACT_PATTERN13 >>>>>> :-) >>>>>> >>>>>> Have a great weekend! >>>>>> Joe >>>>>> >>>>>> On 8/14/2020 3:20 PM, naoto.sato at oracle.com wrote: >>>>>>> Hello, >>>>>>> >>>>>>> Please review the fix for the following issue: >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8251499 >>>>>>> >>>>>>> The proposed changeset is located at: >>>>>>> >>>>>>> https://cr.openjdk.java.net/~naoto/8251499/webrev.00/ >>>>>>> >>>>>>> The current implementation of CompactNumberFormat assumes that >>>>>>> there is always the number placeholder part in compact patterns. >>>>>>> This is not always true. In fact, upcoming CLDR 38 resurrects >>>>>>> such patterns, so this fix is a precursor to support CLDR 38. >>>>>>> >>>>>>> Naoto >>>>>> >>>> >> > From Roger.Riggs at oracle.com Wed Aug 19 19:52:35 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Wed, 19 Aug 2020 15:52:35 -0400 Subject: RFR: 8251499: no-placeholder compact number patterns throw IllegalArgumentException In-Reply-To: References: <5b81c727-b1e0-0614-c68e-5568f98d9d6f@oracle.com> <217ef621-2293-3a1b-827c-06d067ea8b36@oracle.com> <9ee7bb2e-8030-b3a5-c5f6-6aa1517deb5c@oracle.com> <18130879-d9ad-a5a7-611f-138a3b3f27bd@oracle.com> <62419dd1-655b-04ad-adeb-6773df85cfbf@oracle.com> Message-ID: Hi Naoto, Looks good, thanks for the updates. Roger On 8/19/20 2:26 PM, naoto.sato at oracle.com wrote: > Hi Roger, > > Thank you for your comments. I've addressed your suggestions and > created a new webrev below: > > https://cr.openjdk.java.net/~naoto/8251499/webrev.03/ > > Naoto > > On 8/19/20 8:33 AM, Roger Riggs wrote: >> Hi Naoto, >> >> CompactNumberFormat.java: >> >> 269: The field "placeHolders" should be named consistently with the >> other holders of Patterns. >> ?? -> placeHolderPatterns >> >> 1632: missing space before ":" >> >> 2390:? I'm not sure why the stream processing is preferable to a >> direct forEach(...). >> >> TestCompactPatternsValidity: >> For the Plurals cases it would clearer to create a new Data provider >> and corresponding tests >> with the extra arg. >> >> Thanks, Roger >> >> >> >> On 8/18/20 5:13 PM, Joe Wang wrote: >>> Hi Naoto, >>> >>> That's nice!? The change looks good to me. >>> >>> Regards, >>> Joe >>> >>> On 8/18/20 11:37 AM, naoto.sato at oracle.com wrote: >>>> Hi Joe, >>>> >>>> Thank you for your comment. I consolidated those duplicated pieces >>>> into one piece. Did not make it a private method, though, as it >>>> would need to return two results from the method (cnfMultiplier and >>>> whether to return immediately for no-placeholder cases). >>>> >>>> https://cr.openjdk.java.net/~naoto/8251499/webrev.02/ >>>> >>>> Naoto >>>> >>>> On 8/17/20 11:52 PM, Joe Wang wrote: >>>>> Hi Naoto, >>>>> >>>>> Looks good overall. One nit, blocks 1633-1639 and 1642-1649 may >>>>> share a common private method with a parameter that takes either >>>>> matchedPosIndex or matchedNegIndex, if you want. >>>>> >>>>> Best, >>>>> Joe >>>>> >>>>> On 8/17/20 4:42 PM, naoto.sato at oracle.com wrote: >>>>>> Hi Joe, >>>>>> >>>>>> It turned out that the previous fix did not address plural format >>>>>> cases. That means that just making the divisor negative to >>>>>> indicate non-placeholder cannot distinguish multiple plural cases >>>>>> with the same divisor. Instead, I created a list of placeholders >>>>>> (minimum digits) for each index and count. Here is the updated >>>>>> webrev: >>>>>> >>>>>> http://cr.openjdk.java.net/~naoto/8251499/webrev.01/ >>>>>> >>>>>> I added a new test case (COMPACT_PATTERN14), which actually is >>>>>> extracted from CLDR 38 Somali locale that demonstrates the issue. >>>>>> I'd appreciate your further review. >>>>>> >>>>>> Naoto >>>>>> >>>>>> On 8/14/20 6:21 PM, Joe Wang wrote: >>>>>>> Hi Naoto, >>>>>>> >>>>>>> Looks good to me. >>>>>>> >>>>>>> While a negative divisor representing no zeros is newly >>>>>>> introduced, the "divisor > 0" checks seem to have always been >>>>>>> beneficial.? I had to count the number of ""s in >>>>>>> COMPACT_PATTERN13 :-) >>>>>>> >>>>>>> Have a great weekend! >>>>>>> Joe >>>>>>> >>>>>>> On 8/14/2020 3:20 PM, naoto.sato at oracle.com wrote: >>>>>>>> Hello, >>>>>>>> >>>>>>>> Please review the fix for the following issue: >>>>>>>> >>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8251499 >>>>>>>> >>>>>>>> The proposed changeset is located at: >>>>>>>> >>>>>>>> https://cr.openjdk.java.net/~naoto/8251499/webrev.00/ >>>>>>>> >>>>>>>> The current implementation of CompactNumberFormat assumes that >>>>>>>> there is always the number placeholder part in compact >>>>>>>> patterns. This is not always true. In fact, upcoming CLDR 38 >>>>>>>> resurrects such patterns, so this fix is a precursor to support >>>>>>>> CLDR 38. >>>>>>>> >>>>>>>> Naoto >>>>>>> >>>>> >>> >> From mandy.chung at oracle.com Wed Aug 19 20:30:57 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 19 Aug 2020 13:30:57 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> Message-ID: <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> On 8/17/20 12:37 PM, Yumin Qi wrote: > Hi, Ioi > > ? Thanks for review/suggestion. I have updated the webrev at the > following link: > > http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ > This patch leverages the TRACE_RESOLVE output and passes the trace output to VM.? VM then calls GenerateJLIClassesHelper::generateMHHolderClasses to do the parsing and generate Holder class per the resolved LFs.?? I think there are other cleaner alternatives implementing this.?? jlink --generate-jli-classes plugin depends the trace output whereas -Xshare:dump does not.?? It's cleaner to skip generating the trace output and parsing for dumping shared archive purpose.? In addition, the implementation needs some cleanup (I can send you feedback on the next revision) Instead of relying on a system property "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better to use jdk.internal.vm.isCDSDumpingEnabled() to detect if this is CDS dump time. One possible approach: - Add the trace methods in GenerateJLIClassesHelper class to trace these resolved members: ?? traceSpeciesType(String cn) ?? tracePregeneratedLambdaForm(String name, MethodType type, Class holder) ?? * I'm not sure if these method should also accept the case when an exception is thrown but it's okay we add it as well to match existing implementation. - these trace methods will either (1) send the trace output to System.out or (2) collect speciesTypes, invokerTypes, callSiteTypes, and dmhMethods in the runtime if -Xshare:dump is specified. - ClassSpecializer and InvokerBytecodeGenerator will call these trace methods instead. - TRACE_RESOLVE is set when the system property is set to true. If -Xshare:dump is enabled,? the trace methods will not write to System.out.? Does CDS want a log of these entry?? I assume not? - jlink plugin will continue to do the parsing.? It can construct a new? class that takes the speciesTypes, invokerTypes, callSiteTypes, and dmhMethods parameters parsed from the trace output. jlink plugin can invoke an instance method to return Map. I suggest the key for this return map is the class name rather than a jimage entry name and the caller can map it to an jimage entry name if needed. - CDS will invoke a different entry to return Object[] as you currently have. What do you think? Mandy From Roger.Riggs at oracle.com Wed Aug 19 21:14:56 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Wed, 19 Aug 2020 17:14:56 -0400 Subject: RFR 8251989: Hex encoder and decoder utility Message-ID: <6743da26-9bae-e868-d242-b359aeec3d21@oracle.com> Please review a java.util.Hex API to encode and decode hexadecimal strings to and from byte arrays. Within the JDK and JDK tests there are multiple implementations to encode and decode hexadecimal strings to byte arrays. Hex encoders and decoders support upper or lower case hexadecimal characters, delimiters, prefix, and suffix. The API is modeled after the java.util.Base64 API providing static factories, immutable threadsafe instances with methods to encode to and decode from string and StringBuilder. JavaDoc: http://cr.openjdk.java.net/~rriggs/hex-javadoc/java.base/java/util/Hex.html Webrev for Hex encoder and decoder: ?? http://cr.openjdk.java.net/~rriggs/webrev-hex-encoder-8251989 Webrev for applying to java.security: ?? http://cr.openjdk.java.net/~rriggs/webrev-hex-security-8252055 CSR: ?? https://bugs.openjdk.java.net/browse/JDK-8251991 Issue for API and a few uses: ?? https://bugs.openjdk.java.net/browse/JDK-8251989 Issue for Use in java.security and tests: ?? https://bugs.openjdk.java.net/browse/JDK-8252055 From cjashfor at linux.ibm.com Wed Aug 19 22:11:11 2020 From: cjashfor at linux.ibm.com (Corey Ashford) Date: Wed, 19 Aug 2020 15:11:11 -0700 Subject: RFR(M): 8248188: [PATCH] Add HotSpotIntrinsicCandidate and API for Base64 decoding In-Reply-To: References: <11ca749f-3015-c004-aa6b-3194e1dfe4eb@linux.ibm.com> Message-ID: Hi Roger, Thank you for your reply. I wasn't aware of this practice, and will get to work on creating and contributing a test. Regards, - Corey On 8/19/20 11:20 AM, Roger Riggs wrote: > Hi Corey, > > For changes obviously performance motivated, it is conventional to run a > JMH perf test to demonstate > the improvement and prove it is worthwhile to add code complexity. > > I don't see any existing Base64 JMH tests but they would be in the repo > below or near: > ??? test/micro/org/openjdk/bench/java/util/ > > Please contribute a JMH test and results to show the difference. > > Regards, Roger From christoph.dreis at freenet.de Thu Aug 20 04:41:55 2020 From: christoph.dreis at freenet.de (Christoph Dreis) Date: Thu, 20 Aug 2020 06:41:55 +0200 Subject: Optimize sun.invoke.util.BytecodeDescriptor.unparse In-Reply-To: References: Message-ID: <871C0DD0-EE87-4D59-99B6-58E938BD6A35@freenet.de> HI, I hate pinging, but I would really appreciate it if someone finds the time to take a look at my suggestion below from last week. Or is this maybe the wrong mailing list - given that it is in the "sun" root package? Cheers, Christoph ===== ORIGINAL MAIL ===== ?Hi, I just stumbled upon sun.invoke.util.BytecodeDescriptor.unparse that seems to unnecessarily create a StringBuilder and checks for the given type to be of Object.class twice in certain scenarios. When I apply the attached patch below with the following isolated benchmark: @BenchmarkMode(Mode.AverageTime) @OutputTimeUnit(TimeUnit.NANOSECONDS) @State(Scope.Thread) public class MyBenchmark { @State(Scope.Thread) public static class BenchmarkState { private Class test = String.class; // long.class; } @Benchmark public String unparseNew(BenchmarkState state) { return BytecodeDescriptor.unparseNew(state.test); } @Benchmark public String unparseOld(BenchmarkState state) { return BytecodeDescriptor.unparseOld(state.test); } } I get the following results: String.class Benchmark Mode Cnt Score Error Units MyBenchmark.unparseNew avgt 10 47,207 ? 1,918 ns/op MyBenchmark.unparseNew:?gc.alloc.rate.norm avgt 10 232,011 ? 0,002 B/op MyBenchmark.unparseOld avgt 10 87,197 ? 22,843 ns/op MyBenchmark.unparseOld:?gc.alloc.rate.norm avgt 10 384,020 ? 0,001 B/op long.class Benchmark Mode Cnt Score Error Units MyBenchmark.unparseNew avgt 10 4,996 ? 0,022 ns/op MyBenchmark.unparseNew:?gc.alloc.rate.norm avgt 10 ? 10?? B/op MyBenchmark.unparseOld avgt 10 13,303 ? 6,305 ns/op MyBenchmark.unparseOld:?gc.alloc.rate.norm avgt 10 80,003 ? 0,001 B/op As you can see the new way makes things allocation free for primitives and also improves normal classes. It seems like a relatively trivial improvement. In case you think this is worthwhile, I would appreciate it if someone could sponsor the change. Cheers, Christoph ======= PATCH ======= --- a/src/java.base/share/classes/sun/invoke/util/BytecodeDescriptor.java Thu Aug 13 09:33:28 2020 -0700 +++ b/src/java.base/share/classes/sun/invoke/util/BytecodeDescriptor.java Thu Aug 13 19:27:26 2020 +0200 @@ -110,9 +110,13 @@ } else if (type == int.class) { return "I"; } - StringBuilder sb = new StringBuilder(); - unparseSig(type, sb); - return sb.toString(); + Wrapper basicType = Wrapper.forBasicType(type); + char c = basicType.basicTypeChar(); + if (c != 'L') { + return basicType.basicTypeString(); + } else { + return type.descriptorString(); + } } From yumin.qi at oracle.com Thu Aug 20 05:14:59 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Wed, 19 Aug 2020 22:14:59 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> Message-ID: <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> HI, Mandy ? Thanks for the review, I took one day off yesterday so just got a detail look of your reply. On 8/19/20 1:30 PM, Mandy Chung wrote: > On 8/17/20 12:37 PM, Yumin Qi wrote: >> Hi, Ioi >> >> ? Thanks for review/suggestion. I have updated the webrev at the following link: >> >> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >> > > This patch leverages the TRACE_RESOLVE output and passes the trace output to VM.? VM then calls GenerateJLIClassesHelper::generateMHHolderClasses to do the parsing and generate Holder class per the resolved LFs.?? I think there are other cleaner alternatives implementing this.?? jlink --generate-jli-classes plugin depends the trace output whereas -Xshare:dump does not.?? It's cleaner to skip generating the trace output and parsing for dumping shared archive purpose.? In addition, the implementation needs some cleanup (I can send you feedback on the next revision) > Current patch did not change the existing code for JLinkPlugin part. I just moved the parsing code from GenerateJLIClassesPlugin.java to GenerateJLIClassesHelper.java since the former is an internal class to which we shouldn't call to generate holder classes. > Instead of relying on a system property "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better to use jdk.internal.vm.isCDSDumpingEnabled() to detect if this is CDS dump time. I remember we have such API to query if flag -Xshare:dump or -Xshare:on used. Do you mean if DumpLoadedClassList flag set? This flag is the one used to log class name to list file. In GenerateLinkOptData.gmk: $(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) $(CLASSLIST_JAR) ??????? $(call MakeDir, $(LINK_OPT_DIR)) ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $@)) ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $(JLI_TRACE_FILE))) ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw \ ??????????? -Duser.language=en -Duser.country=US \ ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ ??????????? build.tools.classlist.HelloClasslist $(LOG_DEBUG) ??????? $(GREP) -v HelloClasslist $@.raw > $@.interim ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -Xshare:dump \ ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ ??????????? -Xmx128M -Xms128M $(LOG_INFO) ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw.2 \ ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ ??????????? -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true \ ??????????? -Duser.language=en -Duser.country=US \ ??????????? --module-path $(SUPPORT_OUTPUTDIR)/classlist.jar \ ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ ??????????? build.tools.classlist.HelloClasslist \ ??????????? 2> $(LINK_OPT_DIR)/stderr > $(JLI_TRACE_FILE) \ ??????????? || ( \ ??????????????? exitcode=$$? ; \ ??????????????? $(ECHO) "ERROR: Failed to generate link optimization data." \ ??????????????????? "This is likely a problem with the newly built JVM/JDK." ; \ ??????????????? $(CAT) $(LINK_OPT_DIR)/stderr $(JLI_TRACE_FILE) ; \ ??????????????? exit $$exitcode \ ??????????? ) ??????? $(GREP) -v HelloClasslist $@.raw.2 > $@ The $(JLI_TRACE_FILE) is generated with both -XX:DumpLoadedClassList and -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, in current implementation, DumpLoadedClassList will turn on property java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true. So the same output sent to stdout and log file DumpLoadedClassList specified. Now instead of this property, using a vm interface API to query if this flag is set, I think it is better choice. But here I am NOT sure I understand your suggestion, I think there are two choices: 1) Using DumpLoadedClassList to collect TRACE_RESOLVE but not via CDS_TRACE_RESOLVE, using new API to query if DumpLoadedClassList is set 2) Do not use DumpLoadedClassList, when -Xshare:dump? collecting those name, type and holder name to regenerate holder classes? At dump time, CDS generation relies on the classlist file, it may not trigger the generation of LF classes as at pre-run. So choice 2) may not work as expected. > > One possible approach: > > - Add the trace methods in GenerateJLIClassesHelper class to trace these resolved members: > ?? traceSpeciesType(String cn) > ?? tracePregeneratedLambdaForm(String name, MethodType type, Class holder) > > ?? * I'm not sure if these method should also accept the case when an exception is thrown but it's okay we add it as well to match existing implementation. > I will keep consistent with the current logic. > - these trace methods will either (1) send the trace output to System.out or (2) collect speciesTypes, invokerTypes, callSiteTypes, and dmhMethods in the runtime if -Xshare:dump is specified. > The trace method will send the log to DumpLoadedClassList file. At dump time, we reply on the trace log lines to regenerate holder classes. But if using choice 2), the trace will be store in a list and used to generate holder classes. > - ClassSpecializer and InvokerBytecodeGenerator will call these trace methods instead. > > - TRACE_RESOLVE is set when the system property is set to true. ? If -Xshare:dump is enabled,? the trace methods will not write to System.out.? Does CDS want a log of these entry?? I assume not? If DumpLoadedClassList set, we wish to get output like from TRACE_RESOLVE added to the log file. At pre-run with -XX:DumpLoadedClassList= we wish can collect as most loaded classes as we can for next dump run. > > - jlink plugin will continue to do the parsing.? It can construct a new? class that takes the > speciesTypes, invokerTypes, callSiteTypes, and dmhMethods parameters parsed from the trace output. jlink plugin can invoke an instance method to return Map. I suggest the key for this return map is the class name rather than a jimage entry name and the caller can map it to an jimage entry name if needed. I just move the parsing part to GenerateJLIClassesHelper, if keep the code in jlink plugin, we need a call interface from GenerateJLIClassesHelper and GenerateJLIPlugin to get the parse result, is that right? So the chain of calls will be cds(vm) -> GenerateJLIClassesHelper.generateMHHolderClasses -> JlinkPlugin parse the lines to get name, types -> GenerateJLIClassesHelper (via JLI) generate classes. My question is for a package java.lang.invoke to access jdk.internal.jlink looks not right. > > - CDS will invoke a different entry to return Object[] as you currently have. > Please let me know your comments next so I could go for a new patch. Thanks Yumin > What do you think? > > Mandy From sergei.tsypanov at yandex.ru Thu Aug 20 09:39:27 2020 From: sergei.tsypanov at yandex.ru (=?utf-8?B?0KHQtdGA0LPQtdC5INCm0YvQv9Cw0L3QvtCy?=) Date: Thu, 20 Aug 2020 11:39:27 +0200 Subject: [PATCH] continuation of JDK-6736490 Message-ID: <2133551597912013@mail.yandex.ru> Hello, in June I sent a letter regarding clean-ups of unnecessary explicit initialization of volatile variables [1]. Original mail caused some discussion regarding whether clean-up is safe as of JMM. It turned out that Aleksey Shipilev tried to find conter-example against removal of explicit initialization of volatile variables in concurrency-interest [2] and didn't find any. Also Doug Lea mentions in the same discussion [3] > But your account is a more careful version of reasoning we've done before to conclude that there is never any reason to explicitly initialize fields to 0/0.0/false/null. The original message of mine was then forwarded to security-dev list as affected code related to security-libs [4] and Sean Mullan issued JDK-8251548 as sub-issue of JDK-6736490 which already includes similar clean-up done for java.base. There is also a plenty of possible volatile clean-ups (I've included them into a separate patch here), so I have two questions: 1) Could anyone sponsor the changes related to JDK-8251548 (patch is attached to [1])? 2) Should we create one more sub-issue in JDK-6736490 to handle the rest of changes (attached to this mail) or handle them module by module? Patch is attached, tier1 and tier2 are ok 1. https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-June/067341.html 2. http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/014767.html 3. http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/014770.html 4. https://mail.openjdk.java.net/pipermail/security-dev/2020-August/022289.html -------------- next part -------------- A non-text attachment was scrubbed... Name: volatile-rest.patch Type: text/x-diff Size: 26564 bytes Desc: not available URL: From Roger.Riggs at oracle.com Thu Aug 20 13:55:07 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Thu, 20 Aug 2020 09:55:07 -0400 Subject: Optimize sun.invoke.util.BytecodeDescriptor.unparse In-Reply-To: References: Message-ID: <8b79c0cd-27d4-b73b-1d54-1f942a5b74b7@oracle.com> Hi Chris, Inlining and simplifying unparseSig(cl, sb) seems straightforward. Though I wonder if performs differently than just calling t.descriptorString()? The first action of Class.descriptorString is to check for primitives and return the basicTypeString and if not a primitive it calls Class.descriptorString(). It should be equivalent, without extra checks. Regards, Roger On 8/13/20 1:31 PM, Christoph Dreis wrote: > Hi, > > I just stumbled upon sun.invoke.util.BytecodeDescriptor.unparse that seems to unnecessarily create a StringBuilder and checks for the given type to be of Object.class twice in certain scenarios. > > When I apply the attached patch below with the following isolated benchmark: > > @BenchmarkMode(Mode.AverageTime) > @OutputTimeUnit(TimeUnit.NANOSECONDS) > @State(Scope.Thread) > public class MyBenchmark { > > @State(Scope.Thread) > public static class BenchmarkState { > private Class test = String.class; // long.class; > } > > @Benchmark > public String unparseNew(BenchmarkState state) { > return BytecodeDescriptor.unparseNew(state.test); > } > > @Benchmark > public String unparseOld(BenchmarkState state) { > return BytecodeDescriptor.unparseOld(state.test); > } > } > > I get the following results: > String.class > Benchmark Mode Cnt Score Error Units > MyBenchmark.unparseNew avgt 10 47,207 ? 1,918 ns/op > MyBenchmark.unparseNew:?gc.alloc.rate.norm avgt 10 232,011 ? 0,002 B/op > MyBenchmark.unparseOld avgt 10 87,197 ? 22,843 ns/op > MyBenchmark.unparseOld:?gc.alloc.rate.norm avgt 10 384,020 ? 0,001 B/op > > long.class > Benchmark Mode Cnt Score Error Units > MyBenchmark.unparseNew avgt 10 4,996 ? 0,022 ns/op > MyBenchmark.unparseNew:?gc.alloc.rate.norm avgt 10 ? 10?? B/op > MyBenchmark.unparseOld avgt 10 13,303 ? 6,305 ns/op > MyBenchmark.unparseOld:?gc.alloc.rate.norm avgt 10 80,003 ? 0,001 B/op > > As you can see the new way makes things allocation free for primitives and also improves normal classes. > > It seems like a relatively trivial improvement. In case you think this is worthwhile, I would appreciate it if someone could sponsor the change. > > Cheers, > Christoph > > ======= PATCH ======= > --- a/src/java.base/share/classes/sun/invoke/util/BytecodeDescriptor.java Thu Aug 13 09:33:28 2020 -0700 > +++ b/src/java.base/share/classes/sun/invoke/util/BytecodeDescriptor.java Thu Aug 13 19:27:26 2020 +0200 > @@ -110,9 +110,13 @@ > } else if (type == int.class) { > return "I"; > } > - StringBuilder sb = new StringBuilder(); > - unparseSig(type, sb); > - return sb.toString(); > + Wrapper basicType = Wrapper.forBasicType(type); > + char c = basicType.basicTypeChar(); > + if (c != 'L') { > + return basicType.basicTypeString(); > + } else { > + return type.descriptorString(); > + } > } > > From chris.hegarty at oracle.com Thu Aug 20 15:33:47 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 20 Aug 2020 16:33:47 +0100 Subject: RFR 8251989: Hex encoder and decoder utility In-Reply-To: <6743da26-9bae-e868-d242-b359aeec3d21@oracle.com> References: <6743da26-9bae-e868-d242-b359aeec3d21@oracle.com> Message-ID: <1B69046E-103D-4DA1-ADAB-17BC6A30B274@oracle.com> > On 19 Aug 2020, at 22:14, Roger Riggs wrote: > > .. > JavaDoc: > http://cr.openjdk.java.net/~rriggs/hex-javadoc/java.base/java/util/Hex.html I like it Roger, very nice. A few minor comments/quibbles: Hex: - "Utilities to encode bytes to hex strings and decode hex strings to bytes.? - This class consists solely of ?factory methods" for creating encoders and decoders to .. - In the example, do you want to assert that the byte arrays `bytes` and `dec` are equal? - RFC 4752 or RFC 2752. Which one is it? ;-) Decoder: - decode?(char[] chars, int index, int length) - " A valid character array consists only of the delimiters, prefix, suffix, and an even number of hex digits.? - its just the portion of the array that is required to consist of ?, right. You need to add ?range?, or similar. Please use ?ReturnS? consistently ( I see a few Return (no S) ). Actually, there are a few others, e.g. ?Convert?. Can you take pass over all these, in all three classes? -Chris. From Roger.Riggs at oracle.com Thu Aug 20 16:32:43 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Thu, 20 Aug 2020 12:32:43 -0400 Subject: RFR 8251989: Hex encoder and decoder utility In-Reply-To: <1B69046E-103D-4DA1-ADAB-17BC6A30B274@oracle.com> References: <6743da26-9bae-e868-d242-b359aeec3d21@oracle.com> <1B69046E-103D-4DA1-ADAB-17BC6A30B274@oracle.com> Message-ID: <838c36dd-35e8-a20b-383f-001a37c2577e@oracle.com> Hi Chris, Thanks for the comments, I'll make the updates suggested. Roger On 8/20/20 11:33 AM, Chris Hegarty wrote: >> On 19 Aug 2020, at 22:14, Roger Riggs wrote: >> >> .. >> JavaDoc: >> http://cr.openjdk.java.net/~rriggs/hex-javadoc/java.base/java/util/Hex.html > I like it Roger, very nice. A few minor comments/quibbles: > > Hex: > > - "Utilities to encode bytes to hex strings and decode hex strings to bytes.? - This class consists solely of ?factory methods" for creating encoders and decoders to .. > > - In the example, do you want to assert that the byte arrays `bytes` and `dec` are equal? > > - RFC 4752 or RFC 2752. Which one is it? ;-) RFC 4752. > > Decoder: > > - decode?(char[] chars, int index, int length) - " A valid character array consists only of the delimiters, prefix, suffix, and an even number of hex digits.? - its just the portion of the array that is required to consist of ?, right. You need to add ?range?, or similar. > > Please use ?ReturnS? consistently ( I see a few Return (no S) ). Actually, there are a few others, e.g. ?Convert?. Can you take pass over all these, in all three classes? > > -Chris. From raffaello.giulietti at gmail.com Thu Aug 20 16:46:32 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Thu, 20 Aug 2020 18:46:32 +0200 Subject: RFR 8251989: Hex encoder and decoder utility Message-ID: Hi Roger, in the examples in the javadoc you might want to correct the first argument to Hex.encoder and Hex.decoder to "," to be consistent with the narrative. Greetings Raffaello > > JavaDoc: > http://cr.openjdk.java.net/~rriggs/hex-javadoc/java.base/java/util/Hex.html > From Roger.Riggs at oracle.com Thu Aug 20 17:50:01 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Thu, 20 Aug 2020 13:50:01 -0400 Subject: RFR 8251989: Hex encoder and decoder utility In-Reply-To: References: Message-ID: <98dcdcd3-eca3-9710-02c2-607f7b9bb6f8@oracle.com> Hi Raffaello, Will do, an oversight in updating the example. Thanks, Roger On 8/20/20 12:46 PM, Raffaello Giulietti wrote: > Hi Roger, > > in the examples in the javadoc you might want to correct the first > argument to Hex.encoder and Hex.decoder to "," to be consistent with > the narrative. > > > Greetings > Raffaello > > >> >> JavaDoc: >> http://cr.openjdk.java.net/~rriggs/hex-javadoc/java.base/java/util/Hex.html >> >> > From christoph.dreis at freenet.de Thu Aug 20 18:01:49 2020 From: christoph.dreis at freenet.de (Christoph Dreis) Date: Thu, 20 Aug 2020 20:01:49 +0200 Subject: Optimize sun.invoke.util.BytecodeDescriptor.unparse In-Reply-To: <8b79c0cd-27d4-b73b-1d54-1f942a5b74b7@oracle.com> References: <8b79c0cd-27d4-b73b-1d54-1f942a5b74b7@oracle.com> Message-ID: <905025A9-7B63-474F-8C89-015A2165CED8@freenet.de> Hi Roger, thanks for taking a look! > Though I wonder if performs differently than just calling t.descriptorString()? Seems pretty much to be the same. The difference of 1 ns for the primitive case is a bit weird, but I guess at this levels it's also possible to have fluctuations here and there. Long.class Benchmark Mode Cnt Score Error Units MyBenchmark.unparseChristoph avgt 10 36,995 ? 0,748 ns/op MyBenchmark.unparseChristoph:?gc.alloc.rate.norm avgt 10 168,009 ? 0,001 B/op MyBenchmark.unparseRoger avgt 10 36,857 ? 1,472 ns/op MyBenchmark.unparseRoger:?gc.alloc.rate.norm avgt 10 168,009 ? 0,001 B/op MyBenchmark.unparseOld avgt 10 53,926 ? 1,991 ns/op MyBenchmark.unparseOld:?gc.alloc.rate.norm avgt 10 256,014 ? 0,001 B/op long.class Benchmark Mode Cnt Score Error Units MyBenchmark.unparseChristoph avgt 10 5,184 ? 0,168 ns/op MyBenchmark.unparseChristoph:?gc.alloc.rate.norm avgt 10 ? 10?? B/op MyBenchmark.unparseRoger avgt 10 6,149 ? 0,238 ns/op MyBenchmark.unparseRoger:?gc.alloc.rate.norm avgt 10 ? 10?? B/op MyBenchmark.unparseOld avgt 10 11,236 ? 0,464 ns/op MyBenchmark.unparseOld:?gc.alloc.rate.norm avgt 10 80,003 ? 0,001 B/op > It should be equivalent, without extra checks. It seems to be indeed equivalent, so I would change my proposal to the following. I would keep the check for Object.class and int.class above as they seem to be the most common. At least Object.class is good to have to avoid unnecessary String allocations in from descriptorString() imho. =========== PATCH =============== --- a/src/java.base/share/classes/sun/invoke/util/BytecodeDescriptor.java Thu Aug 13 09:33:28 2020 -0700 +++ b/src/java.base/share/classes/sun/invoke/util/BytecodeDescriptor.java Thu Aug 20 19:44:57 2020 +0200 @@ -110,9 +110,7 @@ } else if (type == int.class) { return "I"; } - StringBuilder sb = new StringBuilder(); - unparseSig(type, sb); - return sb.toString(); + return type.descriptorString(); } What do you think? Cheers, Christoph On 8/13/20 1:31 PM, Christoph Dreis wrote: > Hi, > > I just stumbled upon sun.invoke.util.BytecodeDescriptor.unparse that seems to unnecessarily create a StringBuilder and checks for the given type to be of Object.class twice in certain scenarios. > > When I apply the attached patch below with the following isolated benchmark: > > @BenchmarkMode(Mode.AverageTime) > @OutputTimeUnit(TimeUnit.NANOSECONDS) > @State(Scope.Thread) > public class MyBenchmark { > > @State(Scope.Thread) > public static class BenchmarkState { > private Class test = String.class; // long.class; > } > > @Benchmark > public String unparseNew(BenchmarkState state) { > return BytecodeDescriptor.unparseNew(state.test); > } > > @Benchmark > public String unparseOld(BenchmarkState state) { > return BytecodeDescriptor.unparseOld(state.test); > } > } > > I get the following results: > String.class > Benchmark Mode Cnt Score Error Units > MyBenchmark.unparseNew avgt 10 47,207 ? 1,918 ns/op > MyBenchmark.unparseNew:?gc.alloc.rate.norm avgt 10 232,011 ? 0,002 B/op > MyBenchmark.unparseOld avgt 10 87,197 ? 22,843 ns/op > MyBenchmark.unparseOld:?gc.alloc.rate.norm avgt 10 384,020 ? 0,001 B/op > > long.class > Benchmark Mode Cnt Score Error Units > MyBenchmark.unparseNew avgt 10 4,996 ? 0,022 ns/op > MyBenchmark.unparseNew:?gc.alloc.rate.norm avgt 10 ? 10?? B/op > MyBenchmark.unparseOld avgt 10 13,303 ? 6,305 ns/op > MyBenchmark.unparseOld:?gc.alloc.rate.norm avgt 10 80,003 ? 0,001 B/op > > As you can see the new way makes things allocation free for primitives and also improves normal classes. > > It seems like a relatively trivial improvement. In case you think this is worthwhile, I would appreciate it if someone could sponsor the change. > > Cheers, > Christoph > > ======= PATCH ======= > --- a/src/java.base/share/classes/sun/invoke/util/BytecodeDescriptor.java Thu Aug 13 09:33:28 2020 -0700 > +++ b/src/java.base/share/classes/sun/invoke/util/BytecodeDescriptor.java Thu Aug 13 19:27:26 2020 +0200 > @@ -110,9 +110,13 @@ > } else if (type == int.class) { > return "I"; > } > - StringBuilder sb = new StringBuilder(); > - unparseSig(type, sb); > - return sb.toString(); > + Wrapper basicType = Wrapper.forBasicType(type); > + char c = basicType.basicTypeChar(); > + if (c != 'L') { > + return basicType.basicTypeString(); > + } else { > + return type.descriptorString(); > + } > } > > From adam.farley at uk.ibm.com Thu Aug 20 17:58:58 2020 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Thu, 20 Aug 2020 18:58:58 +0100 Subject: RFR: JDK-8252113: Move jfr man page into jfr module Message-ID: Hi All, Should jfr.1 be moved from java.base to the jdk.jfr module source directory, as indicated here? Webrev: http://cr.openjdk.java.net/~afarley/8252113/webrev/ Bug: https://bugs.openjdk.java.net/browse/JDK-8252113 It seems to me that it should, as man pages should be with their code (as appears the case in other modules), but perhaps there's something I'm not seeing. Thoughts and reviews welcome. Best Regards Adam Farley IBM Runtimes Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From mandy.chung at oracle.com Thu Aug 20 18:53:33 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 20 Aug 2020 11:53:33 -0700 Subject: Optimize sun.invoke.util.BytecodeDescriptor.unparse In-Reply-To: <905025A9-7B63-474F-8C89-015A2165CED8@freenet.de> References: <8b79c0cd-27d4-b73b-1d54-1f942a5b74b7@oracle.com> <905025A9-7B63-474F-8C89-015A2165CED8@freenet.de> Message-ID: On 8/20/20 11:01 AM, Christoph Dreis wrote: > =========== PATCH =============== > --- a/src/java.base/share/classes/sun/invoke/util/BytecodeDescriptor.java Thu Aug 13 09:33:28 2020 -0700 > +++ b/src/java.base/share/classes/sun/invoke/util/BytecodeDescriptor.java Thu Aug 20 19:44:57 2020 +0200 > @@ -110,9 +110,7 @@ > } else if (type == int.class) { > return "I"; > } > - StringBuilder sb = new StringBuilder(); > - unparseSig(type, sb); > - return sb.toString(); > + return type.descriptorString(); > } > > What do you think? > This looks fine.?? FYI.? unparseSig(Class t, StringBuilder sb) was already changed to call `Class::descriptorString` by JDK-8238358.?? It's redundant for unparse(Class t) to call unparseSig. Mandy From WEIJUN.WANG at ORACLE.COM Thu Aug 20 19:02:54 2020 From: WEIJUN.WANG at ORACLE.COM (Weijun Wang) Date: Thu, 20 Aug 2020 15:02:54 -0400 Subject: RFR 8251989: Hex encoder and decoder utility In-Reply-To: <6743da26-9bae-e868-d242-b359aeec3d21@oracle.com> References: <6743da26-9bae-e868-d242-b359aeec3d21@oracle.com> Message-ID: <78087E48-1583-4F3F-A959-1A69763DA8E3@ORACLE.COM> How about StringBuilder::appendHex instead of Encoder::encode(SB, byte[])? Then we don?t need to break the chain when appending many different things to a StringBuilder. Also, are prefix and suffix really useful? One can easily add them. Thanks, Max > On Aug 19, 2020, at 5:14 PM, Roger Riggs wrote: > > Please review a java.util.Hex API to encode and decode hexadecimal strings to and from byte arrays. > > Within the JDK and JDK tests there are multiple implementations to encode and decode > hexadecimal strings to byte arrays. Hex encoders and decoders support > upper or lower case hexadecimal characters, delimiters, prefix, and suffix. > The API is modeled after the java.util.Base64 API providing static factories, > immutable threadsafe instances with methods to encode to and decode from > string and StringBuilder. > > JavaDoc: > http://cr.openjdk.java.net/~rriggs/hex-javadoc/java.base/java/util/Hex.html > > Webrev for Hex encoder and decoder: > http://cr.openjdk.java.net/~rriggs/webrev-hex-encoder-8251989 > > Webrev for applying to java.security: > http://cr.openjdk.java.net/~rriggs/webrev-hex-security-8252055 > > CSR: > https://bugs.openjdk.java.net/browse/JDK-8251991 > > Issue for API and a few uses: > https://bugs.openjdk.java.net/browse/JDK-8251989 > > Issue for Use in java.security and tests: > https://bugs.openjdk.java.net/browse/JDK-8252055 > From mark.reinhold at oracle.com Thu Aug 20 19:10:12 2020 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 20 Aug 2020 12:10:12 -0700 Subject: RFR 8251989: Hex encoder and decoder utility In-Reply-To: <6743da26-9bae-e868-d242-b359aeec3d21@oracle.com> References: <6743da26-9bae-e868-d242-b359aeec3d21@oracle.com> Message-ID: <20200820121012.674081461@eggemoggin.niobe.net> 2020/8/19 14:14:56 -0700, roger.riggs at oracle.com: > Please review a java.util.Hex API to encode and decode hexadecimal > strings to and from byte arrays. > > JavaDoc: > http://cr.openjdk.java.net/~rriggs/hex-javadoc/java.base/java/util/Hex.html This mostly looks good -- it?ll be nice to have a standard utility for this sort of thing. A few comments: - Why do the short-form `encoder` factory methods return encoders that produce upper-case hex strings? `Integer::toHexString` and other existing `toHex` methods return lower-case hex strings. That?s also what you get from common Unix CLI tools (e.g., `od -tx1`). Please consider changing these methods to return lower-case encoders. - Is it worth having static `Hex.encode(byte[])` and `Hex.decode(CharSequence)` convenience methods for the simplest cases? - [Warning: Bikeshed] The verbs ?encode? and ?decode? seem unfortunate. Over in `java.nio.charsets` we have encoders that transform characters to bytes, and decoders that transform bytes to characters. The coded thing is the bytes; the uncoded thing is the characters. In `java.util` we already have the `Base64` class, which provides encoders that transform bytes to characters, and decoders that transform characters to bytes. The coded thing is the characters; the uncoded thing is the bytes. The use of ?encode? and ?decode? in `Base64` was likely inspired by the fact that the format has been known as ?base 64 encoding? for decades, having originated as a hack for transporting non-ASCII data via SMTP. Developers looking to do base-64 operations will, thus, expect this terminology. Here you?re proposing that the `Hex` class follow the `Base64` class. Consistency with existing nearby APIs is a worthy goal. If I were just looking to convert a byte array into a readable hex string, however, I?d probably want to ?format? it rather than ?encode? it, something like `String.format("%x")` on steroids. Likewise, if I were looking to convert a hex string into bytes then I?d want to ?parse? it rather than ?decode? it, i.e., `Integer::parseInt` on steroids. If you were to rename the nested classes to `Hex.Formatter` and `Hex.Parser`, and rename all methods accordingly, then this API would be inconsistent with the nearby `Base64` but likely more consistent with developer expectations. (`Hex` is already inconsistent with `Base64` in that it doesn?t prefix the names of its factory methods with `get`, which is a good thing.) - Mark From Roger.Riggs at oracle.com Thu Aug 20 20:15:29 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Thu, 20 Aug 2020 16:15:29 -0400 Subject: Optimize sun.invoke.util.BytecodeDescriptor.unparse In-Reply-To: <905025A9-7B63-474F-8C89-015A2165CED8@freenet.de> References: <8b79c0cd-27d4-b73b-1d54-1f942a5b74b7@oracle.com> <905025A9-7B63-474F-8C89-015A2165CED8@freenet.de> Message-ID: <12d35d6a-a6d6-d469-e790-26a082c29502@oracle.com> Hi Christoph, Looks good. Note that Claes added the cases for Object.class and int.class to maximize the performance of those common cases. I'll sponsor the change. Thanks, Roger On 8/20/20 2:01 PM, Christoph Dreis wrote: > Hi Roger, > > thanks for taking a look! > >> Though I wonder if performs differently than just calling t.descriptorString()? > Seems pretty much to be the same. > The difference of 1 ns for the primitive case is a bit weird, but I guess at this levels it's also possible to have fluctuations here and there. > > Long.class > Benchmark Mode Cnt Score Error Units > MyBenchmark.unparseChristoph avgt 10 36,995 ? 0,748 ns/op > MyBenchmark.unparseChristoph:?gc.alloc.rate.norm avgt 10 168,009 ? 0,001 B/op > MyBenchmark.unparseRoger avgt 10 36,857 ? 1,472 ns/op > MyBenchmark.unparseRoger:?gc.alloc.rate.norm avgt 10 168,009 ? 0,001 B/op > MyBenchmark.unparseOld avgt 10 53,926 ? 1,991 ns/op > MyBenchmark.unparseOld:?gc.alloc.rate.norm avgt 10 256,014 ? 0,001 B/op > > long.class > Benchmark Mode Cnt Score Error Units > MyBenchmark.unparseChristoph avgt 10 5,184 ? 0,168 ns/op > MyBenchmark.unparseChristoph:?gc.alloc.rate.norm avgt 10 ? 10?? B/op > MyBenchmark.unparseRoger avgt 10 6,149 ? 0,238 ns/op > MyBenchmark.unparseRoger:?gc.alloc.rate.norm avgt 10 ? 10?? B/op > MyBenchmark.unparseOld avgt 10 11,236 ? 0,464 ns/op > MyBenchmark.unparseOld:?gc.alloc.rate.norm avgt 10 80,003 ? 0,001 B/op > >> It should be equivalent, without extra checks. > It seems to be indeed equivalent, so I would change my proposal to the following. > I would keep the check for Object.class and int.class above as they seem to be the most common. > At least Object.class is good to have to avoid unnecessary String allocations in from descriptorString() imho. > > =========== PATCH =============== > --- a/src/java.base/share/classes/sun/invoke/util/BytecodeDescriptor.java Thu Aug 13 09:33:28 2020 -0700 > +++ b/src/java.base/share/classes/sun/invoke/util/BytecodeDescriptor.java Thu Aug 20 19:44:57 2020 +0200 > @@ -110,9 +110,7 @@ > } else if (type == int.class) { > return "I"; > } > - StringBuilder sb = new StringBuilder(); > - unparseSig(type, sb); > - return sb.toString(); > + return type.descriptorString(); > } > > What do you think? > > Cheers, > Christoph > > On 8/13/20 1:31 PM, Christoph Dreis wrote: >> Hi, >> >> I just stumbled upon sun.invoke.util.BytecodeDescriptor.unparse that seems to unnecessarily create a StringBuilder and checks for the given type to be of Object.class twice in certain scenarios. >> >> When I apply the attached patch below with the following isolated benchmark: >> >> @BenchmarkMode(Mode.AverageTime) >> @OutputTimeUnit(TimeUnit.NANOSECONDS) >> @State(Scope.Thread) >> public class MyBenchmark { >> >> @State(Scope.Thread) >> public static class BenchmarkState { >> private Class test = String.class; // long.class; >> } >> >> @Benchmark >> public String unparseNew(BenchmarkState state) { >> return BytecodeDescriptor.unparseNew(state.test); >> } >> >> @Benchmark >> public String unparseOld(BenchmarkState state) { >> return BytecodeDescriptor.unparseOld(state.test); >> } >> } >> >> I get the following results: >> String.class >> Benchmark Mode Cnt Score Error Units >> MyBenchmark.unparseNew avgt 10 47,207 ? 1,918 ns/op >> MyBenchmark.unparseNew:?gc.alloc.rate.norm avgt 10 232,011 ? 0,002 B/op >> MyBenchmark.unparseOld avgt 10 87,197 ? 22,843 ns/op >> MyBenchmark.unparseOld:?gc.alloc.rate.norm avgt 10 384,020 ? 0,001 B/op >> >> long.class >> Benchmark Mode Cnt Score Error Units >> MyBenchmark.unparseNew avgt 10 4,996 ? 0,022 ns/op >> MyBenchmark.unparseNew:?gc.alloc.rate.norm avgt 10 ? 10?? B/op >> MyBenchmark.unparseOld avgt 10 13,303 ? 6,305 ns/op >> MyBenchmark.unparseOld:?gc.alloc.rate.norm avgt 10 80,003 ? 0,001 B/op >> >> As you can see the new way makes things allocation free for primitives and also improves normal classes. >> >> It seems like a relatively trivial improvement. In case you think this is worthwhile, I would appreciate it if someone could sponsor the change. >> >> Cheers, >> Christoph >> >> ======= PATCH ======= >> --- a/src/java.base/share/classes/sun/invoke/util/BytecodeDescriptor.java Thu Aug 13 09:33:28 2020 -0700 >> +++ b/src/java.base/share/classes/sun/invoke/util/BytecodeDescriptor.java Thu Aug 13 19:27:26 2020 +0200 >> @@ -110,9 +110,13 @@ >> } else if (type == int.class) { >> return "I"; >> } >> - StringBuilder sb = new StringBuilder(); >> - unparseSig(type, sb); >> - return sb.toString(); >> + Wrapper basicType = Wrapper.forBasicType(type); >> + char c = basicType.basicTypeChar(); >> + if (c != 'L') { >> + return basicType.basicTypeString(); >> + } else { >> + return type.descriptorString(); >> + } >> } >> >> > > From szegedia at gmail.com Thu Aug 20 20:20:51 2020 From: szegedia at gmail.com (Attila Szegedi) Date: Thu, 20 Aug 2020 22:20:51 +0200 Subject: Review request for JDK-8252124: Restore Dynalink tests Message-ID: Hi folks, long time since I actively popped up here. I?m taking some time to maintain Dynalink, eyeing Java 16 for the changes. Since the nashorn-dev list is (presumably? I haven?t bother checking) defunct, I asked around and Sundar suggested core-libs-dev is the right list to post review requests, so here I am. I plan to do the maintenance in three small stages, and this is the first one. The stages are: - restore some Dynalink tests (they were thrown out with Nashorn) - modernize/lint the Dynalink codebase (it was written for Java 7 originally for? reasons, and by now it could have diamonds, lambdas and so on) - add support for records (the real end goal here.) As stage 1, please review JDK-8252124 "Restore Dynalink tests" at for . It restores those Dynalink tests that are not dependent on Nashorn, so I again have a bit of a coverage as I work on it. They were taken from jdk14/test/nashorn/src/jdk/dynalink and moved into jdk/test/jdk/jdk/dynalink as proper jtreg-run tests. I flattened the package hierarchy into just the unnamed package (which seems to be the usual way of writing tests in the neighboring directories.) I added the total of 8 test classes to the ?jdk_other? test group, it seemed the appropriate group. Thanks, Attila. From szegedia at gmail.com Thu Aug 20 20:40:53 2020 From: szegedia at gmail.com (Attila Szegedi) Date: Thu, 20 Aug 2020 22:40:53 +0200 Subject: Review request for JDK-8251538: Modernize and lint Dynalink code Message-ID: Following up on the previous e-mail, here?s the modernization and linting work on the existing Dynalink codebase: Please review JDK-8251538 "Modernize and lint Dynalink code" at for The Jira issue has a full enumeration of kinds of changes I did here; they?re mostly all source text removals :-) Oh, BTW, I really got an inspiration for adding ?public static ClassValue computingValue(Function, T> f)? method to ClassValue class, similar to Comparator.comparing. It?d allow lambdifying class values. (If you?re listening, John :-) I suspect it being in java.lang would be a JCP-level change so I kind-of don?t want to take it upon myself?) Thanks, Attila. From Roger.Riggs at oracle.com Thu Aug 20 20:40:57 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Thu, 20 Aug 2020 16:40:57 -0400 Subject: RFR 8251989: Hex encoder and decoder utility In-Reply-To: <78087E48-1583-4F3F-A959-1A69763DA8E3@ORACLE.COM> References: <6743da26-9bae-e868-d242-b359aeec3d21@oracle.com> <78087E48-1583-4F3F-A959-1A69763DA8E3@ORACLE.COM> Message-ID: <17cecbb6-bfb0-4c7f-7178-42ac05592c52@oracle.com> Hi Max, The idea was to keep all the Hex functions together. Its a bit of a mismatch with StringBuilder; StringBuilder doesn't deal with arrays and is pretty bulky as it is. And StringBuilder doesn't have a corresponding parse companion; the string to byte[] array functions are needed to. True, the prefix and suffix are conveniences and could be removed. By knowing it them as part of the encode operation it is possible to allocate the buffer exactly. Thanks, Roger On 8/20/20 3:02 PM, Weijun Wang wrote: > How about StringBuilder::appendHex instead of Encoder::encode(SB, byte[])? Then we don?t need to break the chain when appending many different things to a StringBuilder. > > Also, are prefix and suffix really useful? One can easily add them. > > Thanks, > Max > >> On Aug 19, 2020, at 5:14 PM, Roger Riggs wrote: >> >> Please review a java.util.Hex API to encode and decode hexadecimal strings to and from byte arrays. >> >> Within the JDK and JDK tests there are multiple implementations to encode and decode >> hexadecimal strings to byte arrays. Hex encoders and decoders support >> upper or lower case hexadecimal characters, delimiters, prefix, and suffix. >> The API is modeled after the java.util.Base64 API providing static factories, >> immutable threadsafe instances with methods to encode to and decode from >> string and StringBuilder. >> >> JavaDoc: >> http://cr.openjdk.java.net/~rriggs/hex-javadoc/java.base/java/util/Hex.html >> >> Webrev for Hex encoder and decoder: >> http://cr.openjdk.java.net/~rriggs/webrev-hex-encoder-8251989 >> >> Webrev for applying to java.security: >> http://cr.openjdk.java.net/~rriggs/webrev-hex-security-8252055 >> >> CSR: >> https://bugs.openjdk.java.net/browse/JDK-8251991 >> >> Issue for API and a few uses: >> https://bugs.openjdk.java.net/browse/JDK-8251989 >> >> Issue for Use in java.security and tests: >> https://bugs.openjdk.java.net/browse/JDK-8252055 >> From LANCE.ANDERSEN at ORACLE.COM Thu Aug 20 20:58:46 2020 From: LANCE.ANDERSEN at ORACLE.COM (Lance Andersen) Date: Thu, 20 Aug 2020 16:58:46 -0400 Subject: RFR 8252128: Remove javax.transaction Exception references Message-ID: HI all, The following patch removes some extraneous references to javax.transaction exceptions and removes a couple of unused fields that represented serialized versions of the exceptions. This was missed in the removal of the CORBA and Java EE modules. The webrev can be found at: http://cr.openjdk.java.net/~lancea/8252128/webrev.00/ Mach5 jdk-tier1, jdk-tier2, jdk-tier3 are clean (as expected) Best Lance ------------------ Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From Roger.Riggs at oracle.com Thu Aug 20 20:59:39 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Thu, 20 Aug 2020 16:59:39 -0400 Subject: RFR 8251989: Hex encoder and decoder utility In-Reply-To: <20200820121012.674081461@eggemoggin.niobe.net> References: <6743da26-9bae-e868-d242-b359aeec3d21@oracle.com> <20200820121012.674081461@eggemoggin.niobe.net> Message-ID: Hi Mark, On 8/20/20 3:10 PM, mark.reinhold at oracle.com wrote: > 2020/8/19 14:14:56 -0700, roger.riggs at oracle.com: >> Please review a java.util.Hex API to encode and decode hexadecimal >> strings to and from byte arrays. >> >> JavaDoc: >> http://cr.openjdk.java.net/~rriggs/hex-javadoc/java.base/java/util/Hex.html > This mostly looks good -- it?ll be nice to have a standard utility for > this sort of thing. > > A few comments: > > - Why do the short-form `encoder` factory methods return encoders that > produce upper-case hex strings? `Integer::toHexString` and other > existing `toHex` methods return lower-case hex strings. That?s also > what you get from common Unix CLI tools (e.g., `od -tx1`). > > Please consider changing these methods to return lower-case encoders. It's (almost) a toss up and easy to change; many of the existing uses produce uppercase. Perhaps the short form no-arg should be replaced with a short form constructor that takes true/false, so it is explicit at the use site or put the case in the name. encodeToUpper(), encodeToLower(). (A boolean parameter is not very informative, a enum would be better but perhaps a bit heavyweight). > > - Is it worth having static `Hex.encode(byte[])` and > `Hex.decode(CharSequence)` convenience methods for the simplest > cases? There was some discussion of that but it idea was to minimize the surface area. > > - [Warning: Bikeshed] The verbs ?encode? and ?decode? seem unfortunate. > > Over in `java.nio.charsets` we have encoders that transform > characters to bytes, and decoders that transform bytes to characters. > The coded thing is the bytes; the uncoded thing is the characters. > > In `java.util` we already have the `Base64` class, which provides > encoders that transform bytes to characters, and decoders that > transform characters to bytes. The coded thing is the characters; > the uncoded thing is the bytes. > > The use of ?encode? and ?decode? in `Base64` was likely inspired by > the fact that the format has been known as ?base 64 encoding? for > decades, having originated as a hack for transporting non-ASCII data > via SMTP. Developers looking to do base-64 operations will, thus, > expect this terminology. The bias of encoding vs decoding terminology is subtle, based the 'native' form of the data. For the Charset classes, the native form of the data is characters, and the encoded form is bytes.? For Base64, the native form of the data is bytes, and the encoded form is Base64 lines. > > Here you?re proposing that the `Hex` class follow the `Base64` class. > Consistency with existing nearby APIs is a worthy goal. If I were > just looking to convert a byte array into a readable hex string, > however, I?d probably want to ?format? it rather than ?encode? it, > something like `String.format("%x")` on steroids. Likewise, if I > were looking to convert a hex string into bytes then I?d want to > ?parse? it rather than ?decode? it, i.e., `Integer::parseInt` on > steroids. > > If you were to rename the nested classes to `Hex.Formatter` and > `Hex.Parser`, and rename all methods accordingly, then this API would > be inconsistent with the nearby `Base64` but likely more consistent > with developer expectations. > > (`Hex` is already inconsistent with `Base64` in that it doesn?t > prefix the names of its factory methods with `get`, which is a good > thing.) The question is at what level of control does "encoding" become formatting. There are very few formatting features in the API, no control of leading zeros, no control over indicental whitespace, and no control over width.? Similarly with parsing, what flexibility does parsing imply that decoding does not, is whitespace ignored, line endings/joins, etc. Thanks, Roger > > - Mark From Roger.Riggs at oracle.com Thu Aug 20 21:03:56 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Thu, 20 Aug 2020 17:03:56 -0400 Subject: RFR 8252128: Remove javax.transaction Exception references In-Reply-To: References: Message-ID: Hi Lance, Looks fine. Thanks, Roger On 8/20/20 4:58 PM, Lance Andersen wrote: > Hi all, > > The following patch removes some extraneous references to javax.transaction exceptions and removes a couple of unused fields that represented serialized versions of the exceptions. This was missed in the removal of the CORBA and Java EE modules. > > The webrev can be found at: http://cr.openjdk.java.net/~lancea/8252128/webrev.00/ > > Mach5 jdk-tier1, jdk-tier2, jdk-tier3 are clean (as expected) > > Best > Lance > ------------------ > > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > > From forax at univ-mlv.fr Thu Aug 20 21:43:26 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 20 Aug 2020 23:43:26 +0200 (CEST) Subject: Review request for JDK-8251538: Modernize and lint Dynalink code In-Reply-To: References: Message-ID: <1089568732.209661.1597959806629.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Attila Szegedi" > ?: "core-libs-dev" > Envoy?: Jeudi 20 Ao?t 2020 22:40:53 > Objet: Review request for JDK-8251538: Modernize and lint Dynalink code > Following up on the previous e-mail, here?s the modernization and linting work > on the existing Dynalink codebase: > > Please review JDK-8251538 "Modernize and lint Dynalink code" at > for > > > The Jira issue has a full enumeration of kinds of changes I did here; they?re > mostly all source text removals :-) > > Oh, BTW, I really got an inspiration for adding ?public static ClassValue > computingValue(Function, T> f)? method to ClassValue class, similar to > Comparator.comparing. It?d allow lambdifying class values. (If you?re > listening, John :-) I suspect it being in java.lang would be a JCP-level change > so I kind-of don?t want to take it upon myself?) this is also very similar to https://docs.oracle.com/en/java/javase/14/docs/api/java.base/java/lang/ThreadLocal.html#withInitial(java.util.function.Supplier) and the proposed signature doesn't have the right wildcards, it should be public static ClassValue computingValue(Function, ? extends T> f) > > Thanks, > Attila. regards, R?mi From andy.herrick at oracle.com Thu Aug 20 21:44:57 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Thu, 20 Aug 2020 17:44:57 -0400 Subject: RFR: JDK-8251988: jpackage --runtime-image fails on mac when using JDK11 based runtime. Message-ID: <67dd0e2d-d107-120b-d1bd-006f3955b81e@oracle.com> Please review the jpackage fix at [1] to issue [2]. This fixes the problem loading libjli.dylib as well as? providing some useful generic system properties. /Andy [1] - http://cr.openjdk.java.net/~herrick/8251988/webrev.02/ [2] - https://bugs.openjdk.java.net/browse/JDK-8251988 From alexey.semenyuk at oracle.com Thu Aug 20 23:36:25 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Thu, 20 Aug 2020 19:36:25 -0400 Subject: RFR: JDK-8251988: jpackage --runtime-image fails on mac when using JDK11 based runtime. In-Reply-To: <67dd0e2d-d107-120b-d1bd-006f3955b81e@oracle.com> References: <67dd0e2d-d107-120b-d1bd-006f3955b81e@oracle.com> Message-ID: <25da9b8b-b9ef-fceb-137c-55c659e75202@oracle.com> Andy, I'd replace 'File.separator' with '/' to make value of 'jpackage.app-path' property platform independent. It would be also good to have a unit test verifying values of the new properties. - Alexey On 8/20/2020 5:44 PM, Andy Herrick wrote: > Please review the jpackage fix at [1] to issue [2]. > > This fixes the problem loading libjli.dylib as well as? providing some > useful generic system properties. > > /Andy > > > [1] - http://cr.openjdk.java.net/~herrick/8251988/webrev.02/ > > > [2] - https://bugs.openjdk.java.net/browse/JDK-8251988 > From mandy.chung at oracle.com Fri Aug 21 00:10:46 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 20 Aug 2020 17:10:46 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> Message-ID: <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> On 8/19/20 10:14 PM, Yumin Qi wrote: > > HI, Mandy > > ? Thanks for the review, I took one day off yesterday so just got a > detail look of your reply. > > On 8/19/20 1:30 PM, Mandy Chung wrote: >> On 8/17/20 12:37 PM, Yumin Qi wrote: >>> Hi, Ioi >>> >>> ? Thanks for review/suggestion. I have updated the webrev at the >>> following link: >>> >>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>> >> >> This patch leverages the TRACE_RESOLVE output and passes the trace >> output to VM.? VM then calls >> GenerateJLIClassesHelper::generateMHHolderClasses to do the parsing >> and generate Holder class per the resolved LFs.?? I think there are >> other cleaner alternatives implementing this. jlink >> --generate-jli-classes plugin depends the trace output whereas >> -Xshare:dump does not.?? It's cleaner to skip generating the trace >> output and parsing for dumping shared archive purpose.? In addition, >> the implementation needs some cleanup (I can send you feedback on the >> next revision) >> > Current patch did not change the existing code for JLinkPlugin part. I > just moved the parsing code from GenerateJLIClassesPlugin.java to > GenerateJLIClassesHelper.java since the former is an internal class to > which we shouldn't call to generate holder classes. >> Instead of relying on a system property >> "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better to use >> jdk.internal.vm.isCDSDumpingEnabled() to detect if this is CDS dump time. > > I remember we have such API to query if flag -Xshare:dump or > -Xshare:on used. Do you mean if DumpLoadedClassList flag set? This > flag is the one used to log class name to list file. In > GenerateLinkOptData.gmk: > > $(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) > $(CLASSLIST_JAR) > ??????? $(call MakeDir, $(LINK_OPT_DIR)) > ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $@)) > ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, > $(JLI_TRACE_FILE))) > ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java > -XX:DumpLoadedClassList=$@.raw \ > ??????????? -Duser.language=en -Duser.country=US \ > ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ > ??????????? build.tools.classlist.HelloClasslist $(LOG_DEBUG) > ??????? $(GREP) -v HelloClasslist $@.raw > $@.interim > ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -Xshare:dump \ > ??????????? -XX:SharedClassListFile=$@.interim > -XX:SharedArchiveFile=$@.jsa \ > ??????????? -Xmx128M -Xms128M $(LOG_INFO) > ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java > -XX:DumpLoadedClassList=$@.raw.2 \ > ??????????? -XX:SharedClassListFile=$@.interim > -XX:SharedArchiveFile=$@.jsa \ > ??????????? -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true \ > ??????????? -Duser.language=en -Duser.country=US \ > ??????????? --module-path $(SUPPORT_OUTPUTDIR)/classlist.jar \ > ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ > ??????????? build.tools.classlist.HelloClasslist \ > ??????????? 2> $(LINK_OPT_DIR)/stderr > $(JLI_TRACE_FILE) \ > ??????????? || ( \ > ??????????????? exitcode=$$? ; \ > ??????????????? $(ECHO) "ERROR: Failed to generate link optimization > data." \ > ??????????????????? "This is likely a problem with the newly built > JVM/JDK." ; \ > ??????????????? $(CAT) $(LINK_OPT_DIR)/stderr $(JLI_TRACE_FILE) ; \ > ??????????????? exit $$exitcode \ > ??????????? ) > ??????? $(GREP) -v HelloClasslist $@.raw.2 > $@ > > The $(JLI_TRACE_FILE) is generated with both -XX:DumpLoadedClassList > and -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, in current > implementation, DumpLoadedClassList will turn on property > java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true. So the same > output sent to stdout and log file DumpLoadedClassList specified. > These entries are duplicated in two different files: one for jlink --generate-jli-classes plugin and another for CDS use.? CDS -Xshare:dump attempts to do what jlink plugin does but writes those generated classes in to shared archive. Like the above make logic to build JDK image, the same entries are written in both default-jli-trace.txt via System.out and to classlist via JNI call to the VM.? I guess VM also implements the logic to do some kind of diffing and write to CDS archive. > Now instead of this property, using a vm interface API to query if > this flag is set, I think it is better choice. But here I am NOT sure > I understand your suggestion, I think there are two choices: > > 1) Using DumpLoadedClassList to collect TRACE_RESOLVE but not via > CDS_TRACE_RESOLVE, using new API to query if DumpLoadedClassList is set > > 2) Do not use DumpLoadedClassList, when -Xshare:dump collecting those > name, type and holder name to regenerate holder classes? > I misunderstood that this CDS_TRACE_RESOLVE flag is set during -Xshare:dump time. Ioi has clarified to me offline that this step is actually part of -XX:DumpLoadedClassList and includes these TRACE_RESOLVE logs to the given class list file, i.e. you repurpose the class list file to include the log output that was initially designed for jlink plugin. To me, I'd prefer to see this support depending on `jlink --generate-jli-classes` which is an existing functionality and much cleaner.? This way this does not require any VM change.?? It will generate the holder classes in the custom image per the application-specific config file. What it means is that: a customer would need to create a custom image with their application-specific config file.? It might need a new CDS? option to specify a separate TRACE_RESOLVE file.?? It would turn on this feature by default by defining a default path of the log file if it helps. I understand that this is not the existing CDS work flow and CDS archive does not require to run on a custom image.?? I see the value of this approach which can prepare customers to start building and using its own custom image. Of course the implementation would be much simpler (adding a flag to write these traces to a given file path, that is). Mandy From yumin.qi at oracle.com Fri Aug 21 03:05:11 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Thu, 20 Aug 2020 20:05:11 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> Message-ID: <2f3e315d-a790-d427-c8dd-cfff3ea7d852@oracle.com> Hi, Mandy On 8/20/20 5:10 PM, Mandy Chung wrote: > > > On 8/19/20 10:14 PM, Yumin Qi wrote: >> >> HI, Mandy >> >> ? Thanks for the review, I took one day off yesterday so just got a detail look of your reply. >> >> On 8/19/20 1:30 PM, Mandy Chung wrote: >>> On 8/17/20 12:37 PM, Yumin Qi wrote: >>>> Hi, Ioi >>>> >>>> ? Thanks for review/suggestion. I have updated the webrev at the following link: >>>> >>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>>> >>> >>> This patch leverages the TRACE_RESOLVE output and passes the trace output to VM.? VM then calls GenerateJLIClassesHelper::generateMHHolderClasses to do the parsing and generate Holder class per the resolved LFs.?? I think there are other cleaner alternatives implementing this.?? jlink --generate-jli-classes plugin depends the trace output whereas -Xshare:dump does not.?? It's cleaner to skip generating the trace output and parsing for dumping shared archive purpose.? In addition, the implementation needs some cleanup (I can send you feedback on the next revision) >>> >> Current patch did not change the existing code for JLinkPlugin part. I just moved the parsing code from GenerateJLIClassesPlugin.java to GenerateJLIClassesHelper.java since the former is an internal class to which we shouldn't call to generate holder classes. >>> Instead of relying on a system property "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better to use jdk.internal.vm.isCDSDumpingEnabled() to detect if this is CDS dump time. >> >> I remember we have such API to query if flag -Xshare:dump or -Xshare:on used. Do you mean if DumpLoadedClassList flag set? This flag is the one used to log class name to list file. In GenerateLinkOptData.gmk: >> >> $(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) $(CLASSLIST_JAR) >> ??????? $(call MakeDir, $(LINK_OPT_DIR)) >> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $@)) >> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $(JLI_TRACE_FILE))) >> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw \ >> ??????????? -Duser.language=en -Duser.country=US \ >> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >> ??????????? build.tools.classlist.HelloClasslist $(LOG_DEBUG) >> ??????? $(GREP) -v HelloClasslist $@.raw > $@.interim >> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -Xshare:dump \ >> ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ >> ??????????? -Xmx128M -Xms128M $(LOG_INFO) >> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw.2 \ >> ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ >> ??????????? -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true \ >> ??????????? -Duser.language=en -Duser.country=US \ >> ??????????? --module-path $(SUPPORT_OUTPUTDIR)/classlist.jar \ >> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >> ??????????? build.tools.classlist.HelloClasslist \ >> ??????????? 2> $(LINK_OPT_DIR)/stderr > $(JLI_TRACE_FILE) \ >> ??????????? || ( \ >> ??????????????? exitcode=$$? ; \ >> ??????????????? $(ECHO) "ERROR: Failed to generate link optimization data." \ >> ??????????????????? "This is likely a problem with the newly built JVM/JDK." ; \ >> ??????????????? $(CAT) $(LINK_OPT_DIR)/stderr $(JLI_TRACE_FILE) ; \ >> ??????????????? exit $$exitcode \ >> ??????????? ) >> ??????? $(GREP) -v HelloClasslist $@.raw.2 > $@ >> >> The $(JLI_TRACE_FILE) is generated with both -XX:DumpLoadedClassList and -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, in current implementation, DumpLoadedClassList will turn on property java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true. So the same output sent to stdout and log file DumpLoadedClassList specified. >> > > These entries are duplicated in two different files: one for jlink --generate-jli-classes plugin and another for CDS use.? CDS -Xshare:dump attempts to do what jlink plugin does but writes those generated classes in to shared archive. > > Like the above make logic to build JDK image, the same entries are written in both default-jli-trace.txt via System.out and to classlist via JNI call to the VM.? I guess VM also implements the logic to do some kind of diffing and write to CDS archive. > In current implementation, vm side only records the line as from TRACE_RESOLVE at pre-run with -XX:DumpLoadedClassList, and at dump time, call back to java for parsing those recordings and generating the holder classes, this uses the existing JLI code. >> >> Now instead of this property, using a vm interface API to query if this flag is set, I think it is better choice. But here I am NOT sure I understand your suggestion, I think there are two choices: >> >> 1) Using DumpLoadedClassList to collect TRACE_RESOLVE but not via CDS_TRACE_RESOLVE, using new API to query if DumpLoadedClassList is set >> >> 2) Do not use DumpLoadedClassList, when -Xshare:dump collecting those name, type and holder name to regenerate holder classes? >> > > I misunderstood that this CDS_TRACE_RESOLVE flag is set during -Xshare:dump time. > > Ioi has clarified to me offline that this step is actually part of -XX:DumpLoadedClassList and includes these TRACE_RESOLVE logs to the given class list file, i.e. you repurpose the class list file to include the log output that was initially designed for jlink plugin. > > To me, I'd prefer to see this support depending on `jlink --generate-jli-classes` which is an existing functionality and much cleaner.? This way this does not require any VM change.?? It will generate the holder classes in the custom image per the application-specific config file. > > What it means is that: a customer would need to create a custom image with their application-specific config file.? It might need a new CDS? option to specify a separate TRACE_RESOLVE file.?? It would turn on this feature by default by defining a default path of the log file if it helps. > So for now, I would implement an API to query if flag DumpLoadedClassList set in cmd line, remove new added property of java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE. We can address custom image with CDS in future in a separate issue. Thanks Yumin > I understand that this is not the existing CDS work flow and CDS archive does not require to run on a custom image.?? I see the value of this approach which can prepare customers to start building and using its own custom image. > > Of course the implementation would be much simpler (adding a flag to write these traces to a given file path, that is). > > Mandy From sundararajan.athijegannathan at oracle.com Fri Aug 21 05:32:37 2020 From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com) Date: Fri, 21 Aug 2020 11:02:37 +0530 Subject: Review request for JDK-8252124: Restore Dynalink tests In-Reply-To: References: Message-ID: Looks good. Minor comment: not sure if you need to update copyright year on tests - especially because there have been changes like package removal (flat). -Sundar On 21/08/20 1:50 am, Attila Szegedi wrote: > Hi folks, > > long time since I actively popped up here. I?m taking some time to maintain Dynalink, eyeing Java 16 for the changes. Since the nashorn-dev list is (presumably? I haven?t bother checking) defunct, I asked around and Sundar suggested core-libs-dev is the right list to post review requests, so here I am. > > I plan to do the maintenance in three small stages, and this is the first one. The stages are: > - restore some Dynalink tests (they were thrown out with Nashorn) > - modernize/lint the Dynalink codebase (it was written for Java 7 originally for? reasons, and by now it could have diamonds, lambdas and so on) > - add support for records (the real end goal here.) > > As stage 1, please review JDK-8252124 "Restore Dynalink tests" at for . > > It restores those Dynalink tests that are not dependent on Nashorn, so I again have a bit of a coverage as I work on it. They were taken from jdk14/test/nashorn/src/jdk/dynalink and moved into jdk/test/jdk/jdk/dynalink as proper jtreg-run tests. I flattened the package hierarchy into just the unnamed package (which seems to be the usual way of writing tests in the neighboring directories.) I added the total of 8 test classes to the ?jdk_other? test group, it seemed the appropriate group. > > Thanks, > Attila. > From amaembo at gmail.com Fri Aug 21 10:15:39 2020 From: amaembo at gmail.com (Tagir Valeev) Date: Fri, 21 Aug 2020 17:15:39 +0700 Subject: RFR 8251989: Hex encoder and decoder utility In-Reply-To: <6743da26-9bae-e868-d242-b359aeec3d21@oracle.com> References: <6743da26-9bae-e868-d242-b359aeec3d21@oracle.com> Message-ID: Hello! Great and long-awaited API, thanks! How about wither-style construction? Like Hex.encoder().upperCase().withDelimiter(":").wrappedIn("{", "}"). This allows modifying existing encoder (e.g. adding prefix and suffix, preserving delimiter and case). Also, you can specify only necessary parameters and in any order you like. encode() method implementation has duplicate requireNonNull checks: 286 public StringBuilder encode(StringBuilder sb, byte[] bytes, int index, int length) { 287 Objects.requireNonNull(sb, "sb"); 288 Objects.requireNonNull(bytes, "bytes"); 289 sb.append(prefix); 290 Objects.requireNonNull(sb, "sb"); 291 Objects.requireNonNull(bytes, "bytes"); With best regards, Tagir Valeev. On Thu, Aug 20, 2020 at 4:15 AM Roger Riggs wrote: > > Please review a java.util.Hex API to encode and decode hexadecimal > strings to and from byte arrays. > > Within the JDK and JDK tests there are multiple implementations to > encode and decode > hexadecimal strings to byte arrays. Hex encoders and decoders support > upper or lower case hexadecimal characters, delimiters, prefix, and suffix. > The API is modeled after the java.util.Base64 API providing static > factories, > immutable threadsafe instances with methods to encode to and decode from > string and StringBuilder. > > JavaDoc: > http://cr.openjdk.java.net/~rriggs/hex-javadoc/java.base/java/util/Hex.html > > Webrev for Hex encoder and decoder: > http://cr.openjdk.java.net/~rriggs/webrev-hex-encoder-8251989 > > Webrev for applying to java.security: > http://cr.openjdk.java.net/~rriggs/webrev-hex-security-8252055 > > CSR: > https://bugs.openjdk.java.net/browse/JDK-8251991 > > Issue for API and a few uses: > https://bugs.openjdk.java.net/browse/JDK-8251989 > > Issue for Use in java.security and tests: > https://bugs.openjdk.java.net/browse/JDK-8252055 > From Roger.Riggs at oracle.com Fri Aug 21 13:32:33 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Fri, 21 Aug 2020 09:32:33 -0400 Subject: Optimize sun.invoke.util.BytecodeDescriptor.unparse In-Reply-To: <12d35d6a-a6d6-d469-e790-26a082c29502@oracle.com> References: <8b79c0cd-27d4-b73b-1d54-1f942a5b74b7@oracle.com> <905025A9-7B63-474F-8C89-015A2165CED8@freenet.de> <12d35d6a-a6d6-d469-e790-26a082c29502@oracle.com> Message-ID: <19f3aaec-202a-68cb-318a-47bdc94d17de@oracle.com> Pushed as ??? 8252127: Optimize sun.invoke.util.BytecodeDescriptor.unparse https://bugs.openjdk.java.net/browse/JDK-8252127 Thanks Christoph On 8/20/20 4:15 PM, Roger Riggs wrote: > Hi Christoph, > > Looks good. > > Note that Claes added the cases for Object.class and int.class > to maximize the performance of those common cases. > > I'll sponsor the change. > > Thanks, Roger > > > On 8/20/20 2:01 PM, Christoph Dreis wrote: >> Hi Roger, >> >> thanks for taking a look! >> >>> Though I wonder if performs differently than just calling? >>> t.descriptorString()? >> Seems pretty much to be the same. >> The difference of 1 ns for the primitive case is a bit weird, but I >> guess at this levels it's also possible to have fluctuations here and >> there. >> >> Long.class >> Benchmark????????????????????????????????????????? Mode? Cnt >> Score???? Error?? Units >> MyBenchmark.unparseChristoph?????????????????????? avgt?? 10 36,995 >> ??? 0,748?? ns/op >> MyBenchmark.unparseChristoph:?gc.alloc.rate.norm?? avgt?? 10 168,009 >> ??? 0,001??? B/op >> MyBenchmark.unparseRoger?????????????????????????? avgt?? 10 36,857 >> ??? 1,472?? ns/op >> MyBenchmark.unparseRoger:?gc.alloc.rate.norm?????? avgt?? 10 168,009 >> ??? 0,001??? B/op >> MyBenchmark.unparseOld???????????????????????????? avgt?? 10 53,926 >> ??? 1,991?? ns/op >> MyBenchmark.unparseOld:?gc.alloc.rate.norm???????? avgt?? 10 256,014 >> ??? 0,001??? B/op >> long.class >> Benchmark????????????????????????????????????????? Mode? Cnt >> Score???? Error?? Units >> MyBenchmark.unparseChristoph?????????????????????? avgt?? 10 5,184 >> ??? 0,168?? ns/op >> MyBenchmark.unparseChristoph:?gc.alloc.rate.norm?? avgt?? 10 ? >> 10??????????????? B/op >> MyBenchmark.unparseRoger?????????????????????????? avgt?? 10 6,149 >> ??? 0,238?? ns/op >> MyBenchmark.unparseRoger:?gc.alloc.rate.norm?????? avgt?? 10 ? >> 10??????????????? B/op >> MyBenchmark.unparseOld???????????????????????????? avgt?? 10 11,236 >> ??? 0,464?? ns/op >> MyBenchmark.unparseOld:?gc.alloc.rate.norm???????? avgt?? 10 80,003 >> ??? 0,001??? B/op >> >>> It should be equivalent, without extra checks. >> It seems to be indeed equivalent, so I would change my proposal to >> the following. >> I would keep the check for Object.class and int.class above as they >> seem to be the most common. >> At least Object.class is good to have to avoid unnecessary String >> allocations in from descriptorString() imho. >> >> =========== PATCH =============== >> --- >> a/src/java.base/share/classes/sun/invoke/util/BytecodeDescriptor.java >> Thu Aug 13 09:33:28 2020 -0700 >> +++ >> b/src/java.base/share/classes/sun/invoke/util/BytecodeDescriptor.java >> Thu Aug 20 19:44:57 2020 +0200 >> @@ -110,9 +110,7 @@ >> ????????? } else if (type == int.class) { >> ????????????? return "I"; >> ????????? } >> -??????? StringBuilder sb = new StringBuilder(); >> -??????? unparseSig(type, sb); >> -??????? return sb.toString(); >> +??????? return type.descriptorString(); >> ????? } >> >> What do you think? >> >> Cheers, >> Christoph >> >> On 8/13/20 1:31 PM, Christoph Dreis wrote: >>> Hi, >>> >>> I just stumbled upon sun.invoke.util.BytecodeDescriptor.unparse that >>> seems to unnecessarily create a StringBuilder and checks for the >>> given type to be of Object.class twice in certain scenarios. >>> >>> When I apply the attached patch below with the following isolated >>> benchmark: >>> >>> @BenchmarkMode(Mode.AverageTime) >>> @OutputTimeUnit(TimeUnit.NANOSECONDS) >>> @State(Scope.Thread) >>> public class MyBenchmark { >>> >>> ????@State(Scope.Thread) >>> ????public static class BenchmarkState { >>> ??????? private Class test = String.class; // long.class; >>> ????} >>> >>> ????@Benchmark >>> ????public String unparseNew(BenchmarkState state) { >>> ??????? return BytecodeDescriptor.unparseNew(state.test); >>> ????} >>> >>> ????@Benchmark >>> ????public String unparseOld(BenchmarkState state) { >>> ??????? return BytecodeDescriptor.unparseOld(state.test); >>> ????} >>> } >>> >>> I get the following results: >>> String.class >>> Benchmark?????????????????????????????????? Mode? Cnt Score???? >>> Error?? Units >>> MyBenchmark.unparseNew????????????????????? avgt?? 10 47,207 ??? >>> 1,918?? ns/op >>> MyBenchmark.unparseNew:?gc.alloc.rate.norm? avgt?? 10 232,011 ??? >>> 0,002??? B/op >>> MyBenchmark.unparseOld????????????????????? avgt?? 10 87,197 ?? >>> 22,843?? ns/op >>> MyBenchmark.unparseOld:?gc.alloc.rate.norm? avgt?? 10 384,020 ??? >>> 0,001??? B/op >>> long.class >>> Benchmark?????????????????????????????????? Mode? Cnt Score???? >>> Error?? Units >>> MyBenchmark.unparseNew????????????????????? avgt?? 10 4,996 ???? >>> 0,022?? ns/op >>> MyBenchmark.unparseNew:?gc.alloc.rate.norm? avgt?? 10??? ? >>> 10???????????????? B/op >>> MyBenchmark.unparseOld????????????????????? avgt?? 10 13,303 ???? >>> 6,305?? ns/op >>> MyBenchmark.unparseOld:?gc.alloc.rate.norm? avgt?? 10 80,003 ???? >>> 0,001??? B/op >>> >>> As you can see the new way makes things allocation free for >>> primitives and also improves normal classes. >>> >>> It seems like a relatively trivial improvement. In case you think >>> this is worthwhile, I would appreciate it if someone could sponsor >>> the change. >>> >>> Cheers, >>> Christoph >>> >>> ======= PATCH ======= >>> --- >>> a/src/java.base/share/classes/sun/invoke/util/BytecodeDescriptor.java >>> Thu Aug 13 09:33:28 2020 -0700 >>> +++ >>> b/src/java.base/share/classes/sun/invoke/util/BytecodeDescriptor.java >>> Thu Aug 13 19:27:26 2020 +0200 >>> @@ -110,9 +110,13 @@ >>> ?????????? } else if (type == int.class) { >>> ?????????????? return "I"; >>> ?????????? } >>> -??????? StringBuilder sb = new StringBuilder(); >>> -??????? unparseSig(type, sb); >>> -??????? return sb.toString(); >>> +??????? Wrapper basicType = Wrapper.forBasicType(type); >>> +??????? char c = basicType.basicTypeChar(); >>> +??????? if (c != 'L') { >>> +??????????? return basicType.basicTypeString(); >>> +??????? } else { >>> +??????????? return type.descriptorString(); >>> +??????? } >>> ?????? } >>> >>> >> >> > From Roger.Riggs at oracle.com Fri Aug 21 15:09:15 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Fri, 21 Aug 2020 11:09:15 -0400 Subject: RFR 8251203: Fix "no comment" warnings in java.base/java.lang and java/io Message-ID: Please review the addition of comments to classes and fields to resolve javadoc "no comment" warnings in the java.lang and java.io packages. The comments are derived from the existing behavior and context. Webrev: ?? http://cr.openjdk.java.net/~rriggs/webrev-nocomment-8251203-1/ CSR: ??? https://bugs.openjdk.java.net/browse/JDK-8252129 Issue: ??? https://bugs.openjdk.java.net/browse/JDK-8251203 Thanks, Roger From raffaello.giulietti at gmail.com Fri Aug 21 15:44:14 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Fri, 21 Aug 2020 17:44:14 +0200 Subject: RFR 8251989: Hex encoder and decoder utility Message-ID: Hi Roger, I'm only a contributor, not an official reviewer. Despite this, I would like to make some notes about the encoder part. (1) Comparing with other APIs that accept a range of a byte[], my understanding is that encode(byte[], int, int) should throw IOOE when index < 0 or too big, even if length == 0. Since other parts of the code already invokes Objects.checkFromIndexSize() in other methods, it would be more consistent to add this check to this one as well. (2) The code on L.334 throws an exception when encodeOptDelim() is invoked with an empty byte[]. This can happen precisely because the checks mentioned in (1) are not strict enough. E.g., given an appropriate encoder, encode(new byte[0], 0, 1) will invoke encodeOptDelim() and throw on L.334 (3) I guess "NYI" on L.343 stands for "not yet implemented", but shouldn't this more likely be an AssertionError? What else needs to be implemented here? (4) The private toHex(int, boolean) can be declared static. (5) To maintain consistency with the other ranges, on L.599 "0123456789" could be replaced by "0-9". (6) As already noted by Tagir, there's a duplication of checks in L.290-291 Greetings Raffaello > Please review a java.util.Hex API to encode and decode hexadecimal > strings to and from byte arrays. > > Within the JDK and JDK tests there are multiple implementations to > encode and decode > hexadecimal strings to byte arrays. Hex encoders and decoders support > upper or lower case hexadecimal characters, delimiters, prefix, and suffix. > The API is modeled after the java.util.Base64 API providing static > factories, > immutable threadsafe instances with methods to encode to and decode from > string and StringBuilder. From daniel.fuchs at oracle.com Fri Aug 21 16:06:32 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 21 Aug 2020 17:06:32 +0100 Subject: RFR 8251203: Fix "no comment" warnings in java.base/java.lang and java/io In-Reply-To: References: Message-ID: <46c945cf-8eba-d102-b973-4187f57cf668@oracle.com> On 21/08/2020 16:09, Roger Riggs wrote: > Please review the addition of comments to classes and fields to resolve > javadoc "no comment" warnings in the java.lang and java.io packages. > The comments are derived from the existing behavior and context. Looks good to me Roger. I don't quite recall when the @serial javadoc tag should be used. That is - should it be used when documenting a non transient private field in the absence of serialPersistentFields ? best regards, -- daniel > > Webrev: > ?? http://cr.openjdk.java.net/~rriggs/webrev-nocomment-8251203-1/ > > CSR: > ??? https://bugs.openjdk.java.net/browse/JDK-8252129 > > Issue: > ??? https://bugs.openjdk.java.net/browse/JDK-8251203 > > Thanks, Roger > > From LANCE.ANDERSEN at ORACLE.COM Fri Aug 21 16:08:30 2020 From: LANCE.ANDERSEN at ORACLE.COM (Lance Andersen) Date: Fri, 21 Aug 2020 12:08:30 -0400 Subject: RFR 8251203: Fix "no comment" warnings in java.base/java.lang and java/io In-Reply-To: References: Message-ID: <6E40668F-8711-40BE-86B6-6C0DFA22C3DF@ORACLE.COM> Hi Roger, Looks good overall. Friendly reminder to update the copyright in the files such as File.java, UncheckedIOException.java, AbstractStringBuilder.java etc... > On Aug 21, 2020, at 11:09 AM, Roger Riggs wrote: > > Please review the addition of comments to classes and fields to resolve > javadoc "no comment" warnings in the java.lang and java.io packages. > The comments are derived from the existing behavior and context. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-nocomment-8251203-1/ > > CSR: > https://bugs.openjdk.java.net/browse/JDK-8252129 > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8251203 > > Thanks, Roger > > Best Lance ------------------ Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From naoto.sato at oracle.com Fri Aug 21 16:16:26 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Fri, 21 Aug 2020 09:16:26 -0700 Subject: RFR 8251203: Fix "no comment" warnings in java.base/java.lang and java/io In-Reply-To: References: Message-ID: <43043063-8fdb-25fd-e0bc-821381f95625@oracle.com> Hi Roger, Changes look good to me. One question, do we need to update the copyright year for this change? Naoto On 8/21/20 8:09 AM, Roger Riggs wrote: > Please review the addition of comments to classes and fields to resolve > javadoc "no comment" warnings in the java.lang and java.io packages. > The comments are derived from the existing behavior and context. > > Webrev: > ?? http://cr.openjdk.java.net/~rriggs/webrev-nocomment-8251203-1/ > > CSR: > ??? https://bugs.openjdk.java.net/browse/JDK-8252129 > > Issue: > ??? https://bugs.openjdk.java.net/browse/JDK-8251203 > > Thanks, Roger > > From Roger.Riggs at oracle.com Fri Aug 21 16:38:09 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Fri, 21 Aug 2020 12:38:09 -0400 Subject: RFR 8251989: Hex encoder and decoder utility In-Reply-To: References: Message-ID: <063de81a-d20f-0717-8bfd-1cd290f19ad5@oracle.com> Hi Rafaello, On 8/21/20 11:44 AM, Raffaello Giulietti wrote: > Hi Roger, > > I'm only a contributor, not an official reviewer. Despite this, I > would like to make some notes about the encoder part. Reviews are always appreciated, whether Reviewer or not. > > > (1) Comparing with other APIs that accept a range of a byte[], my > understanding is that encode(byte[], int, int) should throw IOOE when > index < 0 or too big, even if length == 0. Since other parts of the > code already invokes Objects.checkFromIndexSize() in other methods, it > would be more consistent to add this check to this one as well. yes, will add that, both the encode methods should have the same range checks. > > (2) The code on L.334 throws an exception when encodeOptDelim() is > invoked with an empty byte[]. This can happen precisely because the > checks mentioned in (1) are not strict enough. E.g., given an > appropriate encoder, encode(new byte[0], 0, 1) will invoke > encodeOptDelim() and throw on L.334 Right, adding the check to 1 will keep it from getting that far. > > (3) I guess "NYI" on L.343 stands for "not yet implemented", but > shouldn't this more likely be an AssertionError? What else needs to be > implemented here? Good point, I'll refactor it to avoid duplicate checks before calling the private function and in the function. > > (4) The private toHex(int, boolean) can be declared static. ok > > (5) To maintain consistency with the other ranges, on L.599 > "0123456789" could be replaced by "0-9". ok > > (6) As already noted by Tagir, there's a duplication of checks in > L.290-291 fixed. I'll let some of the higher level API questions settle before updating the webrev. Thanks, Roger > > > Greetings > Raffaello > > > >> Please review a java.util.Hex API to encode and decode hexadecimal >> strings to and from byte arrays. >> >> Within the JDK and JDK tests there are multiple implementations to >> encode and decode >> hexadecimal strings to byte arrays. Hex encoders and decoders support >> upper or lower case hexadecimal characters, delimiters, prefix, and >> suffix. >> The API is modeled after the java.util.Base64 API providing static >> factories, >> immutable threadsafe instances with methods to encode to and decode from >> string and StringBuilder. From Roger.Riggs at oracle.com Fri Aug 21 16:52:41 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Fri, 21 Aug 2020 12:52:41 -0400 Subject: RFR 8251203: Fix "no comment" warnings in java.base/java.lang and java/io In-Reply-To: <46c945cf-8eba-d102-b973-4187f57cf668@oracle.com> References: <46c945cf-8eba-d102-b973-4187f57cf668@oracle.com> Message-ID: Thanks for the reviews. The copyright dates are updated in the webrev: ??? http://cr.openjdk.java.net/~rriggs/webrev-nocomment-8251203-2/ The @serial javadoc tag is used on any serialization related field or method. Are there some missing? I expect javac would warn if they were missing. Thanks, Roger On 8/21/20 12:06 PM, Daniel Fuchs wrote: > On 21/08/2020 16:09, Roger Riggs wrote: >> Please review the addition of comments to classes and fields to resolve >> javadoc "no comment" warnings in the java.lang and java.io packages. >> The comments are derived from the existing behavior and context. > > Looks good to me Roger. > > I don't quite recall when the @serial javadoc tag should be used. > That is - should it be used when documenting a non transient > private field in the absence of serialPersistentFields ? > > best regards, > > -- daniel > > >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-nocomment-8251203-1/ >> >> CSR: >> ???? https://bugs.openjdk.java.net/browse/JDK-8252129 >> >> Issue: >> ???? https://bugs.openjdk.java.net/browse/JDK-8251203 >> >> Thanks, Roger >> >> > From raffaello.giulietti at gmail.com Fri Aug 21 17:13:46 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Fri, 21 Aug 2020 19:13:46 +0200 Subject: RFR 8251989: Hex encoder and decoder utility In-Reply-To: <063de81a-d20f-0717-8bfd-1cd290f19ad5@oracle.com> References: <063de81a-d20f-0717-8bfd-1cd290f19ad5@oracle.com> Message-ID: <3ef93554-4676-6599-cf41-e1252b9b9a01@gmail.com> Hi Roger, On 2020-08-21 18:38, Roger Riggs wrote: > Hi Rafaello, > (It's Raffaello, with 2 effs.) > > I'll let some of the higher level API questions settle before updating > the webrev. > > Thanks, Roger > Sure. In the meantime I'll nevertheless take a look at the current implementation of the Decoder. Greetings Raffaello From mandy.chung at oracle.com Fri Aug 21 17:21:02 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 21 Aug 2020 10:21:02 -0700 Subject: RFR 8251203: Fix "no comment" warnings in java.base/java.lang and java/io In-Reply-To: References: Message-ID: On 8/21/20 8:09 AM, Roger Riggs wrote: > Please review the addition of comments to classes and fields to resolve > javadoc "no comment" warnings in the java.lang and java.io packages. > The comments are derived from the existing behavior and context. > > Webrev: > ?? http://cr.openjdk.java.net/~rriggs/webrev-nocomment-8251203-1/ > Looks okay. Nit:? the type name returned in javadoc and @param should be wrapped with {@code ...}. These lines are some example: 2235????? * @param? s the ObjectOutputStream to which data is written ?941????? * The BytecodeGenerationException. 261 * Resolve a SerializedLambda to an object. No need for a new webrev. Mandy From daniel.fuchs at oracle.com Fri Aug 21 17:32:49 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 21 Aug 2020 18:32:49 +0100 Subject: RFR 8251203: Fix "no comment" warnings in java.base/java.lang and java/io In-Reply-To: References: <46c945cf-8eba-d102-b973-4187f57cf668@oracle.com> Message-ID: <8ef4bd9f-01d0-ee07-90b1-b53ba83d6890@oracle.com> Hi Roger, On 21/08/2020 17:52, Roger Riggs wrote: > The @serial javadoc tag is used on any serialization related field or > method. > Are there some missing? I expect javac would warn if they were missing. Well I was wondering whether an @serial tag would be appropriate in the documentation of these fields, for instance: http://cr.openjdk.java.net/~rriggs/webrev-nocomment-8251203-2/src/java.base/share/classes/java/lang/StackTraceElement.java.frames.html The only purpose of documenting these fields is for the SerializedForm, isn't it? best regards, -- daniel From Roger.Riggs at oracle.com Fri Aug 21 17:51:24 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Fri, 21 Aug 2020 13:51:24 -0400 Subject: RFR 8251203: Fix "no comment" warnings in java.base/java.lang and java/io In-Reply-To: <8ef4bd9f-01d0-ee07-90b1-b53ba83d6890@oracle.com> References: <46c945cf-8eba-d102-b973-4187f57cf668@oracle.com> <8ef4bd9f-01d0-ee07-90b1-b53ba83d6890@oracle.com> Message-ID: Hi Daniel, Re-reading the java.io.Serial javadoc, the annotation is only appropriate on the methods and fields specified as being part of the Serializalble mechanism. For example, readObject, writeObject, readResolve, wroteReplace, and readObjectNoData, and fields serialPersistentFields and serialVersionUID. So no, not applied to serializable fields. The serialized form is based on the non-transient fields of Serialiable classes and annotations for @serialData, serialVersionUID, and serialPersistentFields, etc. Thanks, Roger On 8/21/20 1:32 PM, Daniel Fuchs wrote: > Hi Roger, > > On 21/08/2020 17:52, Roger Riggs wrote: >> The @serial javadoc tag is used on any serialization related field or >> method. >> Are there some missing? I expect javac would warn if they were missing. > > Well I was wondering whether an @serial tag would be appropriate in > the documentation of these fields, for instance: > http://cr.openjdk.java.net/~rriggs/webrev-nocomment-8251203-2/src/java.base/share/classes/java/lang/StackTraceElement.java.frames.html > > > The only purpose of documenting these fields is for the > SerializedForm, isn't it? > > best regards, > > -- daniel From Roger.Riggs at oracle.com Fri Aug 21 18:18:19 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Fri, 21 Aug 2020 14:18:19 -0400 Subject: RFR 8251989: Hex encoder and decoder utility In-Reply-To: References: <6743da26-9bae-e868-d242-b359aeec3d21@oracle.com> Message-ID: <27af6de8-fac4-e66b-bb77-73dbf7093a28@oracle.com> Hi Tagir, On 8/21/20 6:15 AM, Tagir Valeev wrote: > Hello! > > Great and long-awaited API, thanks! > > How about wither-style construction? Like > Hex.encoder().upperCase().withDelimiter(":").wrappedIn("{", "}"). This > allows modifying existing encoder (e.g. adding prefix and suffix, > preserving delimiter and case). Also, you can specify only necessary > parameters and in any order you like. That fluent design style reduces the number of factory methods and adds a few instance methods. That form was used consistently in the java.time APIs.? As en/decoders are immutable it would return a new/differ en/decoder with the desired behavior. The naming of the factory methods as you show makes reading the construction easier and more fluid. The instance methods would fall into two categories, those that act as factories for new en/decoders and those that perform the function. > > encode() method implementation has duplicate requireNonNull checks: > > 286 public StringBuilder encode(StringBuilder sb, byte[] > bytes, int index, int length) { > 287 Objects.requireNonNull(sb, "sb"); > 288 Objects.requireNonNull(bytes, "bytes"); > 289 sb.append(prefix); > 290 Objects.requireNonNull(sb, "sb"); > 291 Objects.requireNonNull(bytes, "bytes"); Thanks, fixed in the next webrev. Roger > > With best regards, > Tagir Valeev. > > On Thu, Aug 20, 2020 at 4:15 AM Roger Riggs wrote: >> Please review a java.util.Hex API to encode and decode hexadecimal >> strings to and from byte arrays. >> >> Within the JDK and JDK tests there are multiple implementations to >> encode and decode >> hexadecimal strings to byte arrays. Hex encoders and decoders support >> upper or lower case hexadecimal characters, delimiters, prefix, and suffix. >> The API is modeled after the java.util.Base64 API providing static >> factories, >> immutable threadsafe instances with methods to encode to and decode from >> string and StringBuilder. >> >> JavaDoc: >> http://cr.openjdk.java.net/~rriggs/hex-javadoc/java.base/java/util/Hex.html >> >> Webrev for Hex encoder and decoder: >> http://cr.openjdk.java.net/~rriggs/webrev-hex-encoder-8251989 >> >> Webrev for applying to java.security: >> http://cr.openjdk.java.net/~rriggs/webrev-hex-security-8252055 >> >> CSR: >> https://bugs.openjdk.java.net/browse/JDK-8251991 >> >> Issue for API and a few uses: >> https://bugs.openjdk.java.net/browse/JDK-8251989 >> >> Issue for Use in java.security and tests: >> https://bugs.openjdk.java.net/browse/JDK-8252055 >> From huizhe.wang at oracle.com Fri Aug 21 18:23:04 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Fri, 21 Aug 2020 11:23:04 -0700 Subject: RFR [16/java.xml] 8251561: Fix doclint warnings in the java.xml package Message-ID: Pelase review a patch to add the missing @return, @throws, @param statements in the java.xml package (excluding the DOM component). JBS: https://bugs.openjdk.java.net/browse/JDK-8251561 CSR: https://bugs.openjdk.java.net/browse/JDK-8251995 webrev: http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_01/ Thanks, Joe From LANCE.ANDERSEN at ORACLE.COM Fri Aug 21 19:24:16 2020 From: LANCE.ANDERSEN at ORACLE.COM (Lance Andersen) Date: Fri, 21 Aug 2020 15:24:16 -0400 Subject: RFR [16/java.xml] 8251561: Fix doclint warnings in the java.xml package In-Reply-To: References: Message-ID: Hi Joe, This looks OK. > On Aug 21, 2020, at 2:23 PM, Joe Wang wrote: > > Pelase review a patch to add the missing @return, @throws, @param statements in the java.xml package (excluding the DOM component). > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251561 > CSR: https://bugs.openjdk.java.net/browse/JDK-8251995 > > webrev: http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_01/ > > Thanks, > Joe Best Lance ------------------ Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From naoto.sato at oracle.com Fri Aug 21 19:49:03 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Fri, 21 Aug 2020 12:49:03 -0700 Subject: RFR [16/java.xml] 8251561: Fix doclint warnings in the java.xml package In-Reply-To: References: Message-ID: +1 Naoto On 8/21/20 12:24 PM, Lance Andersen wrote: > Hi Joe, > > This looks OK. > > > >> On Aug 21, 2020, at 2:23 PM, Joe Wang wrote: >> >> Pelase review a patch to add the missing @return, @throws, @param statements in the java.xml package (excluding the DOM component). >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8251561 >> CSR: https://bugs.openjdk.java.net/browse/JDK-8251995 >> >> webrev: http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_01/ >> >> Thanks, >> Joe > > > Best > Lance > ------------------ > > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > > From szegedia at gmail.com Sun Aug 23 11:22:17 2020 From: szegedia at gmail.com (Attila Szegedi) Date: Sun, 23 Aug 2020 13:22:17 +0200 Subject: Review request for JDK-8251538: Modernize and lint Dynalink code In-Reply-To: <1089568732.209661.1597959806629.JavaMail.zimbra@u-pem.fr> References: <1089568732.209661.1597959806629.JavaMail.zimbra@u-pem.fr> Message-ID: On 2020. Aug 20., at 23:43, Remi Forax wrote: > > ----- Mail original ----- >> De: "Attila Szegedi" >> ?: "core-libs-dev" >> Envoy?: Jeudi 20 Ao?t 2020 22:40:53 >> Objet: Review request for JDK-8251538: Modernize and lint Dynalink code > >> ... >> Oh, BTW, I really got an inspiration for adding ?public static ClassValue >> computingValue(Function, T> f)? method to ClassValue class, similar to >> Comparator.comparing. It?d allow lambdifying class values. (If you?re >> listening, John :-) I suspect it being in java.lang would be a JCP-level change >> so I kind-of don?t want to take it upon myself?) > > this is also very similar to > https://docs.oracle.com/en/java/javase/14/docs/api/java.base/java/lang/ThreadLocal.html#withInitial(java.util.function.Supplier) > > and the proposed signature doesn't have the right wildcards, it should be > public static ClassValue computingValue(Function, ? extends T> f) Never change, Remi :-) > Thanks, >> Attila. > > regards, > R?mi Attila. From szegedia at gmail.com Sun Aug 23 12:32:29 2020 From: szegedia at gmail.com (Attila Szegedi) Date: Sun, 23 Aug 2020 14:32:29 +0200 Subject: Review request for JDK-8252124: Restore Dynalink tests In-Reply-To: References: Message-ID: Im happy to update the copyright years before merging my changes. I?ll not put up a separate webrev for that though, if that?s okay. Attila. > On 2020. Aug 21., at 07:32, Sundararajan Athijegannathan wrote: > > Looks good. > > Minor comment: not sure if you need to update copyright year on tests - > especially because there have been changes like package removal (flat). > > -Sundar > > >> On 2020. Aug 20., at 22:20, Attila Szegedi wrote: >> >> Hi folks, >> >> long time since I actively popped up here. I?m taking some time to maintain Dynalink, eyeing Java 16 for the changes. Since the nashorn-dev list is (presumably? I haven?t bother checking) defunct, I asked around and Sundar suggested core-libs-dev is the right list to post review requests, so here I am. >> >> I plan to do the maintenance in three small stages, and this is the first one. The stages are: >> - restore some Dynalink tests (they were thrown out with Nashorn) >> - modernize/lint the Dynalink codebase (it was written for Java 7 originally for? reasons, and by now it could have diamonds, lambdas and so on) >> - add support for records (the real end goal here.) >> >> As stage 1, please review JDK-8252124 "Restore Dynalink tests" at for . >> >> It restores those Dynalink tests that are not dependent on Nashorn, so I again have a bit of a coverage as I work on it. They were taken from jdk14/test/nashorn/src/jdk/dynalink and moved into jdk/test/jdk/jdk/dynalink as proper jtreg-run tests. I flattened the package hierarchy into just the unnamed package (which seems to be the usual way of writing tests in the neighboring directories.) I added the total of 8 test classes to the ?jdk_other? test group, it seemed the appropriate group. >> >> Thanks, >> Attila. >> > From raffaello.giulietti at gmail.com Sun Aug 23 14:42:14 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Sun, 23 Aug 2020 16:42:14 +0200 Subject: RFR 8251989: Hex encoder and decoder utility Message-ID: <6b9289fb-937d-70ca-55de-fa35f85797cb@gmail.com> > > In the meantime I'll nevertheless take a look at the current > implementation of the Decoder. > > > Greetings > Raffaello Hi Roger, here are my notes about the Decoder. (1) The only serious issue I could find is on L.548. It should read L.548 CharBuffer cb = CharBuffer.wrap(chars, index, index + length); as method wrap() takes a start and an end. (It's quite confusing and error prone to have both (start, length) and (start, end) conventions on ranges all over the OpenJDK :-( Nobody's fault, just annoying.) The rest are minor notes. (2) While I'm pretty sure that a JIT compiler can remove the index range check on L.465 when len is replaced by bytes.length on L.463, I'm less sure that it can do it as currently coded, despite len and bytes.length having the same value. Readability wouldn't suffer and len could be removed. (3) The expressions preLen > origLen - sufLen on L.492 and sufLen > origLen - preLen on L.496 are equivalent. The latter is thus useless on L.496 because it is always false when execution reaches here. (4) The code starting at L.588 assumes that delimiter.length() > 0, which is ensured by the optimization in L.585-586. Without this assumption, however, neither L.509 nor L.512 are correct when delimiter.length() == 0. To make the code to work even when, for some reason, delimiter.length() == 0) the lines can be replaced by the slightly more involved yet more robust L.509 if ((subcs.length() - 2) % stride != 0) and L.512 final int len = (subcs.length() - 2) / stride + 1; (5) I guess that the rationale for not failing fast on illegal characters in the loop on L.516-521, and accumulate and postpone the check only after the whole cs has been processed, is that the code optimistically assumes that cs is legal, so it would be wasteful to check at each iteration. But since each iteration already needs a quite more expensive check for the delimiter, I wonder if the postponing makes much economical sense. The check could be directly on v before assignment to bytes and the var illegal could be removed. (6) What about moving L.517 at the end of the loop body so that memory accesses are issued with strictly increasing and hardware predictable addresses? This move would also better reveal the "left-to-right" processing intent. (7) On L.561, delimiter.isEmpty() could be removed as checkDelimiter() is not invoked when the delimiter is empty. But it's perhaps safer to have it. (8) You may want to wrap escapeNL() around the message on L.567-569 (9) fromHexPair() and fromHex() can be declared static. (10) equals() and hashCode() are missing despite the class being value-based. This holds for Encoder as well. I look forward for the next iteration of the code. Greetings Raffaello From yumin.qi at oracle.com Sun Aug 23 22:56:44 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Sun, 23 Aug 2020 15:56:44 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <2f3e315d-a790-d427-c8dd-cfff3ea7d852@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> <2f3e315d-a790-d427-c8dd-cfff3ea7d852@oracle.com> Message-ID: Hi, Mandy, Ioi and Calvin ? I have updated the new changed at: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-03/ ? In this version: ??? 1) Added a new API to check if flag DumpLoadedClassList set and the file is open. If true, call into vm to print out the trace line to the log file. ??????? Just thinking if we just call the cdsTraceResolve without checking if the flag DumpLoadedClassList set and file open, this way, the check logic is in the vm side like before, so save code by not adding the new API. ??? 2) The returned holder class names now are just 'package/className', removed head and tail. ??? 3)? Remove add_extra_classes from CollectClassesClosure since after bug 8250990: https://bugs.openjdk.java.net/browse/JDK-8250990 pushed, the CollectClassesClosure no longer exist. ??? 4)? Still keep the parsing for TRACE_RESOLVE in java/lang/invoke/GenerateJLIClassesHelper.java, so VM call its function to regenerate holder classes. ? Re-tested Mach5 tier1-4 Thanks Yumin ** On 8/20/20 8:05 PM, Yumin Qi wrote: > Hi, Mandy > > > On 8/20/20 5:10 PM, Mandy Chung wrote: >> >> >> On 8/19/20 10:14 PM, Yumin Qi wrote: >>> >>> HI, Mandy >>> >>> ? Thanks for the review, I took one day off yesterday so just got a detail look of your reply. >>> >>> On 8/19/20 1:30 PM, Mandy Chung wrote: >>>> On 8/17/20 12:37 PM, Yumin Qi wrote: >>>>> Hi, Ioi >>>>> >>>>> ? Thanks for review/suggestion. I have updated the webrev at the following link: >>>>> >>>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>>>> >>>> >>>> This patch leverages the TRACE_RESOLVE output and passes the trace output to VM.? VM then calls GenerateJLIClassesHelper::generateMHHolderClasses to do the parsing and generate Holder class per the resolved LFs.?? I think there are other cleaner alternatives implementing this.?? jlink --generate-jli-classes plugin depends the trace output whereas -Xshare:dump does not.?? It's cleaner to skip generating the trace output and parsing for dumping shared archive purpose.? In addition, the implementation needs some cleanup (I can send you feedback on the next revision) >>>> >>> Current patch did not change the existing code for JLinkPlugin part. I just moved the parsing code from GenerateJLIClassesPlugin.java to GenerateJLIClassesHelper.java since the former is an internal class to which we shouldn't call to generate holder classes. >>>> Instead of relying on a system property "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better to use jdk.internal.vm.isCDSDumpingEnabled() to detect if this is CDS dump time. >>> >>> I remember we have such API to query if flag -Xshare:dump or -Xshare:on used. Do you mean if DumpLoadedClassList flag set? This flag is the one used to log class name to list file. In GenerateLinkOptData.gmk: >>> >>> $(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) $(CLASSLIST_JAR) >>> ??????? $(call MakeDir, $(LINK_OPT_DIR)) >>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $@)) >>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $(JLI_TRACE_FILE))) >>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw \ >>> ??????????? -Duser.language=en -Duser.country=US \ >>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>> ??????????? build.tools.classlist.HelloClasslist $(LOG_DEBUG) >>> ??????? $(GREP) -v HelloClasslist $@.raw > $@.interim >>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -Xshare:dump \ >>> ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ >>> ??????????? -Xmx128M -Xms128M $(LOG_INFO) >>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw.2 \ >>> ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ >>> ??????????? -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true \ >>> ??????????? -Duser.language=en -Duser.country=US \ >>> ??????????? --module-path $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>> ??????????? build.tools.classlist.HelloClasslist \ >>> ??????????? 2> $(LINK_OPT_DIR)/stderr > $(JLI_TRACE_FILE) \ >>> ??????????? || ( \ >>> ??????????????? exitcode=$$? ; \ >>> ??????????????? $(ECHO) "ERROR: Failed to generate link optimization data." \ >>> ??????????????????? "This is likely a problem with the newly built JVM/JDK." ; \ >>> ??????????????? $(CAT) $(LINK_OPT_DIR)/stderr $(JLI_TRACE_FILE) ; \ >>> ??????????????? exit $$exitcode \ >>> ??????????? ) >>> ??????? $(GREP) -v HelloClasslist $@.raw.2 > $@ >>> >>> The $(JLI_TRACE_FILE) is generated with both -XX:DumpLoadedClassList and -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, in current implementation, DumpLoadedClassList will turn on property java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true. So the same output sent to stdout and log file DumpLoadedClassList specified. >>> >> >> These entries are duplicated in two different files: one for jlink --generate-jli-classes plugin and another for CDS use. CDS -Xshare:dump attempts to do what jlink plugin does but writes those generated classes in to shared archive. >> >> Like the above make logic to build JDK image, the same entries are written in both default-jli-trace.txt via System.out and to classlist via JNI call to the VM.? I guess VM also implements the logic to do some kind of diffing and write to CDS archive. >> > In current implementation, vm side only records the line as from TRACE_RESOLVE at pre-run with -XX:DumpLoadedClassList, and at dump time, call back to java for parsing those recordings and generating the holder classes, this uses the existing JLI code. >>> >>> Now instead of this property, using a vm interface API to query if this flag is set, I think it is better choice. But here I am NOT sure I understand your suggestion, I think there are two choices: >>> >>> 1) Using DumpLoadedClassList to collect TRACE_RESOLVE but not via CDS_TRACE_RESOLVE, using new API to query if DumpLoadedClassList is set >>> >>> 2) Do not use DumpLoadedClassList, when -Xshare:dump collecting those name, type and holder name to regenerate holder classes? >>> >> >> I misunderstood that this CDS_TRACE_RESOLVE flag is set during -Xshare:dump time. >> >> Ioi has clarified to me offline that this step is actually part of -XX:DumpLoadedClassList and includes these TRACE_RESOLVE logs to the given class list file, i.e. you repurpose the class list file to include the log output that was initially designed for jlink plugin. >> >> To me, I'd prefer to see this support depending on `jlink --generate-jli-classes` which is an existing functionality and much cleaner.? This way this does not require any VM change. It will generate the holder classes in the custom image per the application-specific config file. >> >> What it means is that: a customer would need to create a custom image with their application-specific config file.? It might need a new CDS? option to specify a separate TRACE_RESOLVE file.?? It would turn on this feature by default by defining a default path of the log file if it helps. >> > So for now, I would implement an API to query if flag DumpLoadedClassList set in cmd line, remove new added property of java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE. We can address custom image with CDS in future in a separate issue. > > > Thanks > > Yumin > >> I understand that this is not the existing CDS work flow and CDS archive does not require to run on a custom image.?? I see the value of this approach which can prepare customers to start building and using its own custom image. >> >> Of course the implementation would be much simpler (adding a flag to write these traces to a given file path, that is). >> >> Mandy From sundararajan.athijegannathan at oracle.com Mon Aug 24 08:03:50 2020 From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com) Date: Mon, 24 Aug 2020 13:33:50 +0530 Subject: Review request for JDK-8252124: Restore Dynalink tests In-Reply-To: References: Message-ID: <2ccc00c5-4121-0976-c454-0503af068be3@oracle.com> Yes. no need for separate webrev. -Sundar On 23/08/20 6:02 pm, Attila Szegedi wrote: > Im happy to update the copyright years before merging my changes. I?ll not put up a separate webrev for that though, if that?s okay. > > Attila. > >> On 2020. Aug 21., at 07:32, Sundararajan Athijegannathan wrote: >> >> Looks good. >> >> Minor comment: not sure if you need to update copyright year on tests - >> especially because there have been changes like package removal (flat). >> >> -Sundar >> >> >>> On 2020. Aug 20., at 22:20, Attila Szegedi wrote: >>> >>> Hi folks, >>> >>> long time since I actively popped up here. I?m taking some time to maintain Dynalink, eyeing Java 16 for the changes. Since the nashorn-dev list is (presumably? I haven?t bother checking) defunct, I asked around and Sundar suggested core-libs-dev is the right list to post review requests, so here I am. >>> >>> I plan to do the maintenance in three small stages, and this is the first one. The stages are: >>> - restore some Dynalink tests (they were thrown out with Nashorn) >>> - modernize/lint the Dynalink codebase (it was written for Java 7 originally for? reasons, and by now it could have diamonds, lambdas and so on) >>> - add support for records (the real end goal here.) >>> >>> As stage 1, please review JDK-8252124 "Restore Dynalink tests" at for . >>> >>> It restores those Dynalink tests that are not dependent on Nashorn, so I again have a bit of a coverage as I work on it. They were taken from jdk14/test/nashorn/src/jdk/dynalink and moved into jdk/test/jdk/jdk/dynalink as proper jtreg-run tests. I flattened the package hierarchy into just the unnamed package (which seems to be the usual way of writing tests in the neighboring directories.) I added the total of 8 test classes to the ?jdk_other? test group, it seemed the appropriate group. >>> >>> Thanks, >>> Attila. >>> From sebastian.stenzel at gmail.com Mon Aug 24 14:47:15 2020 From: sebastian.stenzel at gmail.com (Sebastian Stenzel) Date: Mon, 24 Aug 2020 16:47:15 +0200 Subject: Undocumented behaviour in CompletableFuture Message-ID: Hi all, I just stumbled upon a strange behaviour and I'm not sure if it is intended this way or whether it's a bug. It is also possible I'm using the API incorrectly. Let me explain with an example: ``` var originalException = new Exception("foo"); CompletableFuture.failedFuture(originalException).exceptionally(e -> { assert originalException == e; return null; }); ``` This works as expected. However the following does not: ``` var originalException = new Exception("foo"); CompletableFuture.failedFuture(originalException).thenCompose(obj -> CompletableFuture.completedFuture(obj)).exceptionally(e -> { assert originalException == e; // FAIL! return null; }); ``` Note, that the "identity"-function passed to `thenCompose(...)` will not even be called, since the future is already in failed state. However CompletableFuture's implementation wraps the original exception into a new CompletionException. So the mere presence of a `thenCompose(...)` somewhere in the chain will change how to deal with exceptions further downstream. Neither do I find this side effect `thenCompose` documented, nor is it part of the contract of `exceptionally(...)` or `handle(...)` to only return CompletionExceptions. And don't get me started on `get()` and `join()`... Depending on the context of your code, you don't know whether or not there was a `thenCompose`, so you need to check for both, the expected exception as well as a CompletionException with its cause being the expected exception. So is this a bug? Or is this a thing that is not going to be changed, since other software already implemented workarounds for this behaviour? If so, should I move to 3rd party libs? Cheers! Sebastian From andy.herrick at oracle.com Mon Aug 24 20:50:30 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Mon, 24 Aug 2020 16:50:30 -0400 Subject: RFR: JDK-8251988: jpackage --runtime-image fails on mac when using JDK11 based runtime. In-Reply-To: <25da9b8b-b9ef-fceb-137c-55c659e75202@oracle.com> References: <67dd0e2d-d107-120b-d1bd-006f3955b81e@oracle.com> <25da9b8b-b9ef-fceb-137c-55c659e75202@oracle.com> Message-ID: <100096c9-d97d-73b3-1b26-3c7dcb408ce5@oracle.com> please review revised webrev at: http://cr.openjdk.java.net/~herrick/8251988/webrev.03/ jpackage.app-version set in cfg file, and jpackage.app-path now set from native code. New testcase in BasicTest to check that they are set. /Andy On 8/20/2020 7:36 PM, Alexey Semenyuk wrote: > Andy, > > I'd replace 'File.separator' with '/' to make value of > 'jpackage.app-path' property platform independent. > It would be also good to have a unit test verifying values of the new > properties. > > - Alexey > > On 8/20/2020 5:44 PM, Andy Herrick wrote: >> Please review the jpackage fix at [1] to issue [2]. >> >> This fixes the problem loading libjli.dylib as well as providing some >> useful generic system properties. >> >> /Andy >> >> >> [1] - http://cr.openjdk.java.net/~herrick/8251988/webrev.02/ >> >> >> [2] - https://bugs.openjdk.java.net/browse/JDK-8251988 >> > From paul.sandoz at oracle.com Mon Aug 24 20:55:18 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 24 Aug 2020 13:55:18 -0700 Subject: RFR 8251989: Hex encoder and decoder utility In-Reply-To: <6743da26-9bae-e868-d242-b359aeec3d21@oracle.com> References: <6743da26-9bae-e868-d242-b359aeec3d21@oracle.com> Message-ID: Hi Roger, A nice minimal addition. I agree with Mark?s naming suggestion. - Use appropriate code convention for static field names. - The encoder comes with two encoding production signatures, that returning a String and that for encoding into a StringBuilder. The decoder comes with just one such production signature, returning byte[]. I imagine one might need a decode into a byte[] from a given offset. Something to consider later if need be? - More generally if there is pressure to add ByteBuffer accepting signatures we can punt and if necessary use MemorySegment once it exits incubating. > 36 * Utilities to encode bytes to hex strings and decode hex strings to bytes. I recommend using ?hexadecimal (hex)? for the first use, and ?hex? thereafter. > 48 * For example, for a comma (",") separated format with a prefix and suffix Use an @apiNote > 63 * Hex.Encoder encodeFingerprint = Hex.encoder(":", "", "",true); Missing space after last comma. > 60 * RFC 4752 defines a fingerprint of byte values that uses the delimiter colon {@code ':'}. > 61 * The encoder and decoder for RFC 2752 are: s/2752/4752 ? > 254 public StringBuilder encodeHexPair(StringBuilder sb, int value) { This can be specified as behaving as if sb.append(toHexPair(value)), and also say @see #toHexPair. Paul. > On Aug 19, 2020, at 2:14 PM, Roger Riggs wrote: > > Please review a java.util.Hex API to encode and decode hexadecimal strings to and from byte arrays. > > Within the JDK and JDK tests there are multiple implementations to encode and decode > hexadecimal strings to byte arrays. Hex encoders and decoders support > upper or lower case hexadecimal characters, delimiters, prefix, and suffix. > The API is modeled after the java.util.Base64 API providing static factories, > immutable threadsafe instances with methods to encode to and decode from > string and StringBuilder. > > JavaDoc: > http://cr.openjdk.java.net/~rriggs/hex-javadoc/java.base/java/util/Hex.html > > Webrev for Hex encoder and decoder: > http://cr.openjdk.java.net/~rriggs/webrev-hex-encoder-8251989 > > Webrev for applying to java.security: > http://cr.openjdk.java.net/~rriggs/webrev-hex-security-8252055 > > CSR: > https://bugs.openjdk.java.net/browse/JDK-8251991 > > Issue for API and a few uses: > https://bugs.openjdk.java.net/browse/JDK-8251989 > > Issue for Use in java.security and tests: > https://bugs.openjdk.java.net/browse/JDK-8252055 > From alexey.semenyuk at oracle.com Mon Aug 24 21:08:06 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Mon, 24 Aug 2020 17:08:06 -0400 Subject: RFR: JDK-8251988: jpackage --runtime-image fails on mac when using JDK11 based runtime. In-Reply-To: <100096c9-d97d-73b3-1b26-3c7dcb408ce5@oracle.com> References: <67dd0e2d-d107-120b-d1bd-006f3955b81e@oracle.com> <25da9b8b-b9ef-fceb-137c-55c659e75202@oracle.com> <100096c9-d97d-73b3-1b26-3c7dcb408ce5@oracle.com> Message-ID: <2163112d-d071-8f78-0013-9a0a19f4c4da@oracle.com> Andy, I'd propose instead of checking if the app is bundled with proper runtime in the test case, enforce the test case to always create runtime for the app with JPackageCommand.ignoreDefaultRuntime(true) call: --- JPackageCommand cmd = JPackageCommand.helloAppImage( ? 71???????????????? JavaAppDesc.parse("com.hello/com.hello.Hello")) ... ? 76???????????????? .addArguments(...) ?????????????????????? .ignoreDefaultRuntime(true); --- With ignoreDefaultRuntime(true) call added there is no need to call JPackageCommand.isFakeRuntime(). Is com.hello/com/hello/Hello.java used at all? I'd guess after we had refactor all package tests this source is not used any more. Can it be removed yet? - Alexey On 8/24/2020 4:50 PM, Andy Herrick wrote: > please review revised webrev at: > http://cr.openjdk.java.net/~herrick/8251988/webrev.03/ > > jpackage.app-version set in cfg file, and jpackage.app-path now set > from native code. > > New testcase in BasicTest to check that they are set. > > /Andy > > On 8/20/2020 7:36 PM, Alexey Semenyuk wrote: >> Andy, >> >> I'd replace 'File.separator' with '/' to make value of >> 'jpackage.app-path' property platform independent. >> It would be also good to have a unit test verifying values of the new >> properties. >> >> - Alexey >> >> On 8/20/2020 5:44 PM, Andy Herrick wrote: >>> Please review the jpackage fix at [1] to issue [2]. >>> >>> This fixes the problem loading libjli.dylib as well as providing >>> some useful generic system properties. >>> >>> /Andy >>> >>> >>> [1] - http://cr.openjdk.java.net/~herrick/8251988/webrev.02/ >>> >>> >>> [2] - https://bugs.openjdk.java.net/browse/JDK-8251988 >>> >> From cjashfor at linux.ibm.com Mon Aug 24 21:11:44 2020 From: cjashfor at linux.ibm.com (Corey Ashford) Date: Mon, 24 Aug 2020 14:11:44 -0700 Subject: RFR 8251989: Hex encoder and decoder utility In-Reply-To: <6743da26-9bae-e868-d242-b359aeec3d21@oracle.com> References: <6743da26-9bae-e868-d242-b359aeec3d21@oracle.com> Message-ID: Just as a general comment, an SIMD-based encode() intrinsic would be an easy performance upgrade for this Hex class. I'd guess you'd get a 5x-10x improvement, depending on the specific CPU/arch being used. That said, the intrinsic would do better on larger buffers, and I don't know what the average length is for the use cases you have in mind. - Corey On 8/19/20 2:14 PM, Roger Riggs wrote: > Please review a java.util.Hex API to encode and decode hexadecimal > strings to and from byte arrays. > > Within the JDK and JDK tests there are multiple implementations to > encode and decode > hexadecimal strings to byte arrays. Hex encoders and decoders support > upper or lower case hexadecimal characters, delimiters, prefix, and suffix. > The API is modeled after the java.util.Base64 API providing static > factories, > immutable threadsafe instances with methods to encode to and decode from > string and StringBuilder. > > JavaDoc: > http://cr.openjdk.java.net/~rriggs/hex-javadoc/java.base/java/util/Hex.html > > Webrev for Hex encoder and decoder: > ?? http://cr.openjdk.java.net/~rriggs/webrev-hex-encoder-8251989 > > Webrev for applying to java.security: > ?? http://cr.openjdk.java.net/~rriggs/webrev-hex-security-8252055 > > CSR: > ?? https://bugs.openjdk.java.net/browse/JDK-8251991 > > Issue for API and a few uses: > ?? https://bugs.openjdk.java.net/browse/JDK-8251989 > > Issue for Use in java.security and tests: > ?? https://bugs.openjdk.java.net/browse/JDK-8252055 From huizhe.wang at oracle.com Mon Aug 24 21:44:26 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Mon, 24 Aug 2020 14:44:26 -0700 Subject: RFR [16/java.xml] 8251561: Fix doclint warnings in the java.xml package In-Reply-To: References: Message-ID: <3fc299f4-1c0a-1ccd-5d4c-3716775dfa38@oracle.com> Hi all,? adding Roger's comment for the make file to webrev_02 (the only change to webrev_01 is Docs.gmk): http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_02/ Thanks, Joe On 8/21/20 12:49 PM, naoto.sato at oracle.com wrote: > +1 > > Naoto > > On 8/21/20 12:24 PM, Lance Andersen wrote: >> Hi Joe, >> >> This looks OK. >> >> >> >>> On Aug 21, 2020, at 2:23 PM, Joe Wang wrote: >>> >>> Pelase review a patch to add the missing @return, @throws, @param >>> statements in the java.xml package (excluding the DOM component). >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8251561 >>> CSR: https://bugs.openjdk.java.net/browse/JDK-8251995 >>> >>> webrev: http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_01/ >>> >>> Thanks, >>> Joe >> >> >> Best >> Lance >> ------------------ >> >> >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> >> >> From andy.herrick at oracle.com Mon Aug 24 22:23:34 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Mon, 24 Aug 2020 18:23:34 -0400 Subject: RFR: JDK-8251988: jpackage --runtime-image fails on mac when using JDK11 based runtime. In-Reply-To: <2163112d-d071-8f78-0013-9a0a19f4c4da@oracle.com> References: <67dd0e2d-d107-120b-d1bd-006f3955b81e@oracle.com> <25da9b8b-b9ef-fceb-137c-55c659e75202@oracle.com> <100096c9-d97d-73b3-1b26-3c7dcb408ce5@oracle.com> <2163112d-d071-8f78-0013-9a0a19f4c4da@oracle.com> Message-ID: <7a765dc1-4910-e3ec-e89f-5cdc93bef622@oracle.com> On 8/24/2020 5:08 PM, Alexey Semenyuk wrote: > Andy, > > I'd propose instead of checking if the app is bundled with proper > runtime in the test case, enforce the test case to always create > runtime for the app with JPackageCommand.ignoreDefaultRuntime(true) call: > --- > JPackageCommand cmd = JPackageCommand.helloAppImage( > ? 71 JavaAppDesc.parse("com.hello/com.hello.Hello")) > ... > ? 76???????????????? .addArguments(...) > ?????????????????????? .ignoreDefaultRuntime(true); OK. > --- > With ignoreDefaultRuntime(true) call added there is no need to call > JPackageCommand.isFakeRuntime(). > > Is com.hello/com/hello/Hello.java used at all? I'd guess after we had > refactor all package tests this source is not used any more. Can it be > removed yet? It looked like ArgumentTest was using it, but I removed it and all still run revised webrev: http://cr.openjdk.java.net/~herrick/8251988/webrev.04/ /Andy > > - Alexey > > On 8/24/2020 4:50 PM, Andy Herrick wrote: >> please review revised webrev at: >> http://cr.openjdk.java.net/~herrick/8251988/webrev.03/ >> >> jpackage.app-version set in cfg file, and jpackage.app-path now set >> from native code. >> >> New testcase in BasicTest to check that they are set. >> >> /Andy >> >> On 8/20/2020 7:36 PM, Alexey Semenyuk wrote: >>> Andy, >>> >>> I'd replace 'File.separator' with '/' to make value of >>> 'jpackage.app-path' property platform independent. >>> It would be also good to have a unit test verifying values of the >>> new properties. >>> >>> - Alexey >>> >>> On 8/20/2020 5:44 PM, Andy Herrick wrote: >>>> Please review the jpackage fix at [1] to issue [2]. >>>> >>>> This fixes the problem loading libjli.dylib as well as providing >>>> some useful generic system properties. >>>> >>>> /Andy >>>> >>>> >>>> [1] - http://cr.openjdk.java.net/~herrick/8251988/webrev.02/ >>>> >>>> >>>> [2] - https://bugs.openjdk.java.net/browse/JDK-8251988 >>>> >>> > From naoto.sato at oracle.com Mon Aug 24 22:25:29 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Mon, 24 Aug 2020 15:25:29 -0700 Subject: RFR [16/java.xml] 8251561: Fix doclint warnings in the java.xml package In-Reply-To: <3fc299f4-1c0a-1ccd-5d4c-3716775dfa38@oracle.com> References: <3fc299f4-1c0a-1ccd-5d4c-3716775dfa38@oracle.com> Message-ID: <9ffd46f8-318a-9715-89d4-444a47159fce@oracle.com> Still looks good. Naoto On 8/24/20 2:44 PM, Joe Wang wrote: > Hi all,? adding Roger's comment for the make file to webrev_02 (the only > change to webrev_01 is Docs.gmk): > > http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_02/ > > Thanks, > Joe > > On 8/21/20 12:49 PM, naoto.sato at oracle.com wrote: >> +1 >> >> Naoto >> >> On 8/21/20 12:24 PM, Lance Andersen wrote: >>> Hi Joe, >>> >>> This looks OK. >>> >>> >>> >>>> On Aug 21, 2020, at 2:23 PM, Joe Wang wrote: >>>> >>>> Pelase review a patch to add the missing @return, @throws, @param >>>> statements in the java.xml package (excluding the DOM component). >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8251561 >>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8251995 >>>> >>>> webrev: http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_01/ >>>> >>>> Thanks, >>>> Joe >>> >>> >>> Best >>> Lance >>> ------------------ >>> >>> >>> >>> >>> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>> Oracle Java Engineering >>> 1 Network Drive >>> Burlington, MA 01803 >>> Lance.Andersen at oracle.com >>> >>> >>> >>> > From LANCE.ANDERSEN at ORACLE.COM Mon Aug 24 22:27:21 2020 From: LANCE.ANDERSEN at ORACLE.COM (Lance Andersen) Date: Mon, 24 Aug 2020 18:27:21 -0400 Subject: RFR [16/java.xml] 8251561: Fix doclint warnings in the java.xml package In-Reply-To: <3fc299f4-1c0a-1ccd-5d4c-3716775dfa38@oracle.com> References: <3fc299f4-1c0a-1ccd-5d4c-3716775dfa38@oracle.com> Message-ID: <43FE178E-E395-47E8-A456-39A289C3448A@ORACLE.COM> Looks OK Joe > On Aug 24, 2020, at 5:44 PM, Joe Wang wrote: > > Hi all, adding Roger's comment for the make file to webrev_02 (the only change to webrev_01 is Docs.gmk): > > http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_02/ > > Thanks, > Joe > > On 8/21/20 12:49 PM, naoto.sato at oracle.com wrote: >> +1 >> >> Naoto >> >> On 8/21/20 12:24 PM, Lance Andersen wrote: >>> Hi Joe, >>> >>> This looks OK. >>> >>> >>> >>>> On Aug 21, 2020, at 2:23 PM, Joe Wang wrote: >>>> >>>> Pelase review a patch to add the missing @return, @throws, @param statements in the java.xml package (excluding the DOM component). >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8251561 >>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8251995 >>>> >>>> webrev: http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_01/ >>>> >>>> Thanks, >>>> Joe >>> >>> >>> Best >>> Lance >>> ------------------ >>> >>> >>> >>> >>> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>> Oracle Java Engineering >>> 1 Network Drive >>> Burlington, MA 01803 >>> Lance.Andersen at oracle.com >>> >>> >>> >>> > Best Lance ------------------ Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From ioi.lam at oracle.com Mon Aug 24 22:53:52 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 24 Aug 2020 15:53:52 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> Message-ID: <98b8b0c5-b9b5-8c68-15fc-629164203e3d@oracle.com> On 8/20/20 5:10 PM, Mandy Chung wrote: > > > On 8/19/20 10:14 PM, Yumin Qi wrote: >> >> HI, Mandy >> >> ? Thanks for the review, I took one day off yesterday so just got a >> detail look of your reply. >> >> On 8/19/20 1:30 PM, Mandy Chung wrote: >>> On 8/17/20 12:37 PM, Yumin Qi wrote: >>>> Hi, Ioi >>>> >>>> ? Thanks for review/suggestion. I have updated the webrev at the >>>> following link: >>>> >>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>>> >>> >>> This patch leverages the TRACE_RESOLVE output and passes the trace >>> output to VM.? VM then calls >>> GenerateJLIClassesHelper::generateMHHolderClasses to do the parsing >>> and generate Holder class per the resolved LFs.?? I think there are >>> other cleaner alternatives implementing this.?? jlink >>> --generate-jli-classes plugin depends the trace output whereas >>> -Xshare:dump does not.?? It's cleaner to skip generating the trace >>> output and parsing for dumping shared archive purpose.? In addition, >>> the implementation needs some cleanup (I can send you feedback on >>> the next revision) >>> >> Current patch did not change the existing code for JLinkPlugin part. >> I just moved the parsing code from GenerateJLIClassesPlugin.java to >> GenerateJLIClassesHelper.java since the former is an internal class >> to which we shouldn't call to generate holder classes. >>> Instead of relying on a system property >>> "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better to >>> use jdk.internal.vm.isCDSDumpingEnabled() to detect if this is CDS >>> dump time. >> >> I remember we have such API to query if flag -Xshare:dump or >> -Xshare:on used. Do you mean if DumpLoadedClassList flag set?? This >> flag is the one used to log class name to list file. In ? >> GenerateLinkOptData.gmk: >> >> $(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) >> $(CLASSLIST_JAR) >> ??????? $(call MakeDir, $(LINK_OPT_DIR)) >> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $@)) >> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, >> $(JLI_TRACE_FILE))) >> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java >> -XX:DumpLoadedClassList=$@.raw \ >> ??????????? -Duser.language=en -Duser.country=US \ >> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >> ??????????? build.tools.classlist.HelloClasslist $(LOG_DEBUG) >> ??????? $(GREP) -v HelloClasslist $@.raw > $@.interim >> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -Xshare:dump \ >> ??????????? -XX:SharedClassListFile=$@.interim >> -XX:SharedArchiveFile=$@.jsa \ >> ??????????? -Xmx128M -Xms128M $(LOG_INFO) >> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java >> -XX:DumpLoadedClassList=$@.raw.2 \ >> ??????????? -XX:SharedClassListFile=$@.interim >> -XX:SharedArchiveFile=$@.jsa \ >> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true \ >> ??????????? -Duser.language=en -Duser.country=US \ >> ??????????? --module-path $(SUPPORT_OUTPUTDIR)/classlist.jar \ >> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >> ??????????? build.tools.classlist.HelloClasslist \ >> ??????????? 2> $(LINK_OPT_DIR)/stderr > $(JLI_TRACE_FILE) \ >> ??????????? || ( \ >> ??????????????? exitcode=$$? ; \ >> ??????????????? $(ECHO) "ERROR: Failed to generate link optimization >> data." \ >> ??????????????????? "This is likely a problem with the newly built >> JVM/JDK." ; \ >> ??????????????? $(CAT) $(LINK_OPT_DIR)/stderr $(JLI_TRACE_FILE) ; \ >> ??????????????? exit $$exitcode \ >> ??????????? ) >> ??????? $(GREP) -v HelloClasslist $@.raw.2 > $@ >> >> The $(JLI_TRACE_FILE) is generated with both -XX:DumpLoadedClassList >> and -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, in current >> implementation, DumpLoadedClassList will turn on property >> java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true. So the same >> output sent to stdout and log file DumpLoadedClassList specified. >> > > These entries are duplicated in two different files: one for jlink > --generate-jli-classes plugin and another for CDS use.? CDS > -Xshare:dump attempts to do what jlink plugin does but writes those > generated classes in to shared archive. > > Like the above make logic to build JDK image, the same entries are > written in both default-jli-trace.txt via System.out and to classlist > via JNI call to the VM.? I guess VM also implements the logic to do > some kind of diffing and write to CDS archive. > >> Now instead of this property, using a vm interface API to query if >> this flag is set, I think it is better choice. But here I am NOT sure >> I understand your suggestion, I think there are two choices: >> >> 1) Using DumpLoadedClassList to collect TRACE_RESOLVE but not via >> CDS_TRACE_RESOLVE, using new API to query if DumpLoadedClassList is set >> >> 2) Do not use DumpLoadedClassList, when -Xshare:dump collecting those >> name, type and holder name to regenerate holder classes? >> > > I misunderstood that this CDS_TRACE_RESOLVE flag is set during > -Xshare:dump time. > > Ioi has clarified to me offline that this step is actually part of > -XX:DumpLoadedClassList and includes these TRACE_RESOLVE logs to the > given class list file, i.e. you repurpose the class list file to > include the log output that was initially designed for jlink plugin. > > To me, I'd prefer to see this support depending on `jlink > --generate-jli-classes` which is an existing functionality and much > cleaner.? This way this does not require any VM change.?? It will > generate the holder classes in the custom image per the > application-specific config file. > > What it means is that: a customer would need to create a custom image > with their application-specific config file.? It might need a new CDS? > option to specify a separate TRACE_RESOLVE file.?? It would turn on > this feature by default by defining a default path of the log file if > it helps. > > I understand that this is not the existing CDS work flow and CDS > archive does not require to run on a custom image.?? I see the value > of this approach which can prepare customers to start building and > using its own custom image. > > Of course the implementation would be much simpler (adding a flag to > write these traces to a given file path, that is). > Hi Mandy, jlink already supports everything that's provided by this patch (so why are we doing it?): $ java -cp . -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true TestJLI | grep RESOLVE > jli-trace.txt $ grep fail jli-trace.txt [LF_RESOLVE] java.lang.invoke.DirectMethodHandle$Holder invokeSpecial LLD3_L (fail) [LF_RESOLVE] java.lang.invoke.Invokers$Holder linkToTargetMethod D3L_L (fail) $ jlink --add-modules java.base --generate-jli-classes=@jli-trace.txt --output myjdk $ du -sm myjdk 48??? myjdk $ ./myjdk/bin/java -cp . -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true TestJLI > after.txt $ grep D3 after.txt [LF_RESOLVE] java.lang.invoke.DirectMethodHandle$Holder invokeSpecial LLD3_L (success) [LF_RESOLVE] java.lang.invoke.Invokers$Holder linkToTargetMethod D3L_L (success) However, I suspect most people won't do this, because the benefit is relatively small. Also, jlink doesn't support classpath apps, so you would need to figure out what to use for "--add-modules". In the worst case, the custom JDK would be over 100MB in size. So what we want to accomplish with this patch is to make the "--generate-jli-classes" feature available to people who are already using CDS, so that they don't need to create a custom JDK just to get a few optimized XXX$Holder classes. In fact, CDS users don't need to do anything extra, as the LF_RESOLVE traces are automatically included in the output of -XX:DumpLoadedClassList. Once they upgrade to JDK 16, they will automatically benefit from this patch. I understand that this is a little messy. We need to move some code from the jdk.jlink module into java.base, such as GenerateJLIClassesHelper::generateMethodHandleHolderClasses. These are simple text parsing functions. They are called by hotspot, and hotspot is conceptually part of java.base. If we leave them inside jdk.jlink, then we would have part of java.base depending on jdk.link, which is not good. ======== In the long term, in project Leyden, we will have better integration between jlink, AOT, CDS, etc. Today we use "jaotc" to create AOT-compiled code, and "java -Xshare:dump" to create the CDS archive. I think eventually we will just use "jlink", which will internally drive AOT and CDS. Today jlink create a whole JDK image, but we may want to generate just an executable, something like ??? $ jlink --add-modules myapp ..... --trace-file myapp.trace --output myapp.exe ??? $ ./myapp.exe options ... For space saving, we may not copy all the contents of the JDK into myapp.exe. Instead, some contents may be loaded from $JAVA_HOME. Of course, there will also be a way to create a "stand-alone" myapp.exe that doesn't depend on $JAVA_HOME. (In this new world, we will probably move the GenerateJLIClassesHelper::generateMethodHandleHolderClasses code back to jdk.jlink, as they won't be needed by CDS anymore. Similarly, today we have CDS hooks in the Lambda proxy generation code in java.lang.invoke. I suspect those will also be unnecessary, as the pre-resolution of dynamic bytecodes can probably be done with less intervention from CDS.) ========= As you can probably see, the changes in jlink in Leyden will be substantial, and will take multiple JDK releases to roll out. There are a lot of design that need to happen, and it will be much more complicated that just invoking "jlink" with "--cds". While Leyden is making progress, in the short term, we want to keep making improvements in the existing usage model of CDS. Patches like this one is a little messy, but I think it's still under control, and hopefully we won't have many patches of this kind. Thanks - Ioi > Mandy From ioi.lam at oracle.com Mon Aug 24 23:23:10 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 24 Aug 2020 16:23:10 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> <2f3e315d-a790-d427-c8dd-cfff3ea7d852@oracle.com> Message-ID: Hi Yumin, This looks good overall. Here are my comments: ===================== 6065???? size_t new_id = Atomic::add(&counter, (size_t)1); 6066???? jio_snprintf(addr_buf, 20, INTPTR_FORMAT, new_id); I think this should be SIZE_FORMAT ===================== ? 65 class KlassFactory : AllStatic { ? 66 ? 67?? // approved clients ? 68?? friend class ClassLoader; ? 69?? friend class ClassLoaderExt; ? 70?? friend class SystemDictionary; ? 71?? friend class LambdaFormInvokers; ? 72 ? 73? private: ? 74?? static InstanceKlass* create_from_stream(ClassFileStream* stream, I think instead of adding everyone who uses create_from_stream as a friend class, we should just change create_from_stream into a public function and remove the friend declarations. ===================== ?146?? // add to hierarchy and set state to loaded. ?147?? { ?148???? MutexLocker mu_r(THREAD, Compile_lock); // add_shared_to_hierarchy asserts this. ?149???? SystemDictionaryShared::add_shared_to_hierarchy(result, THREAD); ?150?? } I think the function name can be changed to SystemDictionaryShared::add_to_hierarchy as the "_shared" seems redundant. The "set state to loaded" comment seems wrong, as we have the assert on line 1155. I think the comment can be removed. 1153 void SystemDictionaryShared::add_shared_to_hierarchy(InstanceKlass* k, TRAPS) { 1154?? Arguments::assert_is_dumping_archive(); 1155?? assert(!k->is_loaded(), "Invariant"); 1156?? assert_locked_or_safepoint(Compile_lock); // add_to_hierarchy assert on it. 1157?? SystemDictionary::add_to_hierarchy(k, CHECK); 1158 } Also, I think it's better to move the MutexLocker call into SystemDictionaryShared::add_shared_to_hierarchy. ======================== before: ?478???????????????? if (TRACE_RESOLVE && salvage != null) { ?479???????????????????? // Used by jlink species pregeneration plugin, see ?480???????????????????? // jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin ?481???????????????????? System.out.println("[SPECIES_RESOLVE] " + className + " (salvaged)"); ?482???????????????? } after: ?488???????????????? // Used by jlink species pregeneration plugin, see ?489???????????????? // jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin ?490???????????????? traceResolve("[SPECIES_RESOLVE] " + className + " (salvaged)"); When tracing is disabled, this will make extra allocations and cause a small slowdown. I think it's better to ??? if ((TRACE_RESOLVE|TRACE_RESOLVE_CDS) && salvage != null) { ? ? ?? traceResolve("[SPECIES_RESOLVE] " + className + " (salvaged)"); ??? } Because TRACE_RESOLVE is a static final boolean, the JIT compiler will completely optimize this block out. For the same reason, instead of calling VM.isDumpLoadedClassListSetAndOpen() every time, it's better to use a static final variable. ======================= ?698???????? if (TRACE_RESOLVE) { ?699???????????? System.out.println("[LF_RESOLVE] " + holder.getName() + " " + name + " " + ?700 shortenSignature(basicTypeSignature(type)) + (resolvedMember != null ? " (success)" : " (fail)") ); ?701???????? } ?702???????? if (VM.isDumpLoadedClassListSetAndOpen()) { ?703 GenerateJLIClassesHelper.cdsTraceResolve("[LF_RESOLVE] " + holder.getName() + " " + name + " " + ?704 shortenSignature(basicTypeSignature(type)) + (resolvedMember != null ? " (success)" : " (fail)") ); ?705???????? } ?706???????? return resolvedMember; I think the two "if" blocks should be combined similarly to ClassSpecializer::traceResolve(). ========================= ? 34 Java_java_lang_invoke_GenerateJLIClassesHelper_cdsTraceResolve(JNIEnv *env, jclass ignore, jstring line) { Maybe this should be moved to the "VM" class as well? ========================= lambdaFormInvokers.hpp: Need these declarations: ??? #include "memory/allocation.hpp"??? <-- for AllStatic ??? #include "runtime/handles.hpp"????? <-- for typeArrayHandle and Handle ??? #include "utilities/exceptions.hpp" <-- for TRAPS ??? template class GrowableArray; ========================= Thanks - Ioi On 8/23/20 3:56 PM, Yumin Qi wrote: > > Hi, Mandy, Ioi and Calvin > > > ? I have updated the new changed at: > > http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-03/ > > ? In this version: > > ??? 1) Added a new API to check if flag DumpLoadedClassList set and > the file is open. If true, call into vm to print out the trace line to > the log file. > > ??????? Just thinking if we just call the cdsTraceResolve without > checking if the flag DumpLoadedClassList set and file open, this way, > the check logic is in the vm side like before, so save code by not > adding the new API. > > ??? 2) The returned holder class names now are just > 'package/className', removed head and tail. > > ??? 3)? Remove add_extra_classes from CollectClassesClosure since > after bug 8250990: https://bugs.openjdk.java.net/browse/JDK-8250990 > pushed, the CollectClassesClosure no longer exist. > > ??? 4)? Still keep the parsing for TRACE_RESOLVE in > java/lang/invoke/GenerateJLIClassesHelper.java, so VM call its > function to regenerate holder classes. > > > ? Re-tested Mach5 tier1-4 > > Thanks > > Yumin > > > On 8/20/20 8:05 PM, Yumin Qi wrote: >> Hi, Mandy >> >> >> On 8/20/20 5:10 PM, Mandy Chung wrote: >>> >>> >>> On 8/19/20 10:14 PM, Yumin Qi wrote: >>>> >>>> HI, Mandy >>>> >>>> ? Thanks for the review, I took one day off yesterday so just got a >>>> detail look of your reply. >>>> >>>> On 8/19/20 1:30 PM, Mandy Chung wrote: >>>>> On 8/17/20 12:37 PM, Yumin Qi wrote: >>>>>> Hi, Ioi >>>>>> >>>>>> ? Thanks for review/suggestion. I have updated the webrev at the >>>>>> following link: >>>>>> >>>>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>>>>> >>>>> >>>>> This patch leverages the TRACE_RESOLVE output and passes the trace >>>>> output to VM.? VM then calls >>>>> GenerateJLIClassesHelper::generateMHHolderClasses to do the >>>>> parsing and generate Holder class per the resolved LFs.?? I think >>>>> there are other cleaner alternatives implementing this.?? jlink >>>>> --generate-jli-classes plugin depends the trace output whereas >>>>> -Xshare:dump does not. It's cleaner to skip generating the trace >>>>> output and parsing for dumping shared archive purpose.? In >>>>> addition, the implementation needs some cleanup (I can send you >>>>> feedback on the next revision) >>>>> >>>> Current patch did not change the existing code for JLinkPlugin >>>> part. I just moved the parsing code from >>>> GenerateJLIClassesPlugin.java to GenerateJLIClassesHelper.java >>>> since the former is an internal class to which we shouldn't call to >>>> generate holder classes. >>>>> Instead of relying on a system property >>>>> "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better to >>>>> use jdk.internal.vm.isCDSDumpingEnabled() to detect if this is CDS >>>>> dump time. >>>> >>>> I remember we have such API to query if flag -Xshare:dump or >>>> -Xshare:on used. Do you mean if DumpLoadedClassList flag set? This >>>> flag is the one used to log class name to list file. In >>>> GenerateLinkOptData.gmk: >>>> >>>> $(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) >>>> $(CLASSLIST_JAR) >>>> ??????? $(call MakeDir, $(LINK_OPT_DIR)) >>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $@)) >>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, >>>> $(JLI_TRACE_FILE))) >>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java >>>> -XX:DumpLoadedClassList=$@.raw \ >>>> ??????????? -Duser.language=en -Duser.country=US \ >>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>> ??????????? build.tools.classlist.HelloClasslist $(LOG_DEBUG) >>>> ??????? $(GREP) -v HelloClasslist $@.raw > $@.interim >>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -Xshare:dump \ >>>> ??????????? -XX:SharedClassListFile=$@.interim >>>> -XX:SharedArchiveFile=$@.jsa \ >>>> ??????????? -Xmx128M -Xms128M $(LOG_INFO) >>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java >>>> -XX:DumpLoadedClassList=$@.raw.2 \ >>>> ??????????? -XX:SharedClassListFile=$@.interim >>>> -XX:SharedArchiveFile=$@.jsa \ >>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true \ >>>> ??????????? -Duser.language=en -Duser.country=US \ >>>> ??????????? --module-path $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>> ??????????? build.tools.classlist.HelloClasslist \ >>>> ??????????? 2> $(LINK_OPT_DIR)/stderr > $(JLI_TRACE_FILE) \ >>>> ??????????? || ( \ >>>> ??????????????? exitcode=$$? ; \ >>>> ??????????????? $(ECHO) "ERROR: Failed to generate link >>>> optimization data." \ >>>> ??????????????????? "This is likely a problem with the newly built >>>> JVM/JDK." ; \ >>>> ??????????????? $(CAT) $(LINK_OPT_DIR)/stderr $(JLI_TRACE_FILE) ; \ >>>> ??????????????? exit $$exitcode \ >>>> ??????????? ) >>>> ??????? $(GREP) -v HelloClasslist $@.raw.2 > $@ >>>> >>>> The $(JLI_TRACE_FILE) is generated with both >>>> -XX:DumpLoadedClassList and >>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, in current >>>> implementation, DumpLoadedClassList will turn on property >>>> java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true. So the same >>>> output sent to stdout and log file DumpLoadedClassList specified. >>>> >>> >>> These entries are duplicated in two different files: one for jlink >>> --generate-jli-classes plugin and another for CDS use. CDS >>> -Xshare:dump attempts to do what jlink plugin does but writes those >>> generated classes in to shared archive. >>> >>> Like the above make logic to build JDK image, the same entries are >>> written in both default-jli-trace.txt via System.out and to >>> classlist via JNI call to the VM.? I guess VM also implements the >>> logic to do some kind of diffing and write to CDS archive. >>> >> In current implementation, vm side only records the line as from >> TRACE_RESOLVE at pre-run with -XX:DumpLoadedClassList, and at dump >> time, call back to java for parsing those recordings and generating >> the holder classes, this uses the existing JLI code. >>>> >>>> Now instead of this property, using a vm interface API to query if >>>> this flag is set, I think it is better choice. But here I am NOT >>>> sure I understand your suggestion, I think there are two choices: >>>> >>>> 1) Using DumpLoadedClassList to collect TRACE_RESOLVE but not via >>>> CDS_TRACE_RESOLVE, using new API to query if DumpLoadedClassList is >>>> set >>>> >>>> 2) Do not use DumpLoadedClassList, when -Xshare:dump collecting >>>> those name, type and holder name to regenerate holder classes? >>>> >>> >>> I misunderstood that this CDS_TRACE_RESOLVE flag is set during >>> -Xshare:dump time. >>> >>> Ioi has clarified to me offline that this step is actually part of >>> -XX:DumpLoadedClassList and includes these TRACE_RESOLVE logs to the >>> given class list file, i.e. you repurpose the class list file to >>> include the log output that was initially designed for jlink plugin. >>> >>> To me, I'd prefer to see this support depending on `jlink >>> --generate-jli-classes` which is an existing functionality and much >>> cleaner.? This way this does not require any VM change. It will >>> generate the holder classes in the custom image per the >>> application-specific config file. >>> >>> What it means is that: a customer would need to create a custom >>> image with their application-specific config file.? It might need a >>> new CDS? option to specify a separate TRACE_RESOLVE file.?? It would >>> turn on this feature by default by defining a default path of the >>> log file if it helps. >>> >> So for now, I would implement an API to query if flag >> DumpLoadedClassList set in cmd line, remove new added property of >> java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE. We can address >> custom image with CDS in future in a separate issue. >> >> >> Thanks >> >> Yumin >> >>> I understand that this is not the existing CDS work flow and CDS >>> archive does not require to run on a custom image.?? I see the value >>> of this approach which can prepare customers to start building and >>> using its own custom image. >>> >>> Of course the implementation would be much simpler (adding a flag to >>> write these traces to a given file path, that is). >>> >>> Mandy From alexey.semenyuk at oracle.com Mon Aug 24 23:53:28 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Mon, 24 Aug 2020 19:53:28 -0400 Subject: RFR: JDK-8251988: jpackage --runtime-image fails on mac when using JDK11 based runtime. In-Reply-To: <7a765dc1-4910-e3ec-e89f-5cdc93bef622@oracle.com> References: <67dd0e2d-d107-120b-d1bd-006f3955b81e@oracle.com> <25da9b8b-b9ef-fceb-137c-55c659e75202@oracle.com> <100096c9-d97d-73b3-1b26-3c7dcb408ce5@oracle.com> <2163112d-d071-8f78-0013-9a0a19f4c4da@oracle.com> <7a765dc1-4910-e3ec-e89f-5cdc93bef622@oracle.com> Message-ID: <54ce0560-ad89-4a5d-fea8-690b5ad6fecb@oracle.com> Looks good! - Alexey On 8/24/2020 6:23 PM, Andy Herrick wrote: > > On 8/24/2020 5:08 PM, Alexey Semenyuk wrote: >> Andy, >> >> I'd propose instead of checking if the app is bundled with proper >> runtime in the test case, enforce the test case to always create >> runtime for the app with JPackageCommand.ignoreDefaultRuntime(true) >> call: >> --- >> JPackageCommand cmd = JPackageCommand.helloAppImage( >> ? 71 JavaAppDesc.parse("com.hello/com.hello.Hello")) >> ... >> ? 76???????????????? .addArguments(...) >> ?????????????????????? .ignoreDefaultRuntime(true); > OK. >> --- >> With ignoreDefaultRuntime(true) call added there is no need to call >> JPackageCommand.isFakeRuntime(). >> >> Is com.hello/com/hello/Hello.java used at all? I'd guess after we had >> refactor all package tests this source is not used any more. Can it >> be removed yet? > > It looked like ArgumentTest was using it, but I removed it and all > still run > > revised webrev: http://cr.openjdk.java.net/~herrick/8251988/webrev.04/ > > /Andy > > >> >> - Alexey >> >> On 8/24/2020 4:50 PM, Andy Herrick wrote: >>> please review revised webrev at: >>> http://cr.openjdk.java.net/~herrick/8251988/webrev.03/ >>> >>> jpackage.app-version set in cfg file, and jpackage.app-path now set >>> from native code. >>> >>> New testcase in BasicTest to check that they are set. >>> >>> /Andy >>> >>> On 8/20/2020 7:36 PM, Alexey Semenyuk wrote: >>>> Andy, >>>> >>>> I'd replace 'File.separator' with '/' to make value of >>>> 'jpackage.app-path' property platform independent. >>>> It would be also good to have a unit test verifying values of the >>>> new properties. >>>> >>>> - Alexey >>>> >>>> On 8/20/2020 5:44 PM, Andy Herrick wrote: >>>>> Please review the jpackage fix at [1] to issue [2]. >>>>> >>>>> This fixes the problem loading libjli.dylib as well as providing >>>>> some useful generic system properties. >>>>> >>>>> /Andy >>>>> >>>>> >>>>> [1] - http://cr.openjdk.java.net/~herrick/8251988/webrev.02/ >>>>> >>>>> >>>>> [2] - https://bugs.openjdk.java.net/browse/JDK-8251988 >>>>> >>>> >> From huizhe.wang at oracle.com Tue Aug 25 00:49:12 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Mon, 24 Aug 2020 17:49:12 -0700 Subject: RFR [16/java.xml] 8251561: Fix doclint warnings in the java.xml package In-Reply-To: <43FE178E-E395-47E8-A456-39A289C3448A@ORACLE.COM> References: <3fc299f4-1c0a-1ccd-5d4c-3716775dfa38@oracle.com> <43FE178E-E395-47E8-A456-39A289C3448A@ORACLE.COM> Message-ID: Thanks again Lance, Naoto!? And sorry for yet another review request!? I saw Jon's comment on the other doclint thread about replacing @exception with @throws. As it happens, we have 172 of them in the sax package as well. Please note the changes are all in the org/xml/sax packages: http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_03/ -Joe On 8/24/20 3:27 PM, Lance Andersen wrote: > Looks OK Joe > >> On Aug 24, 2020, at 5:44 PM, Joe Wang > > wrote: >> >> Hi all,? adding Roger's comment for the make file to webrev_02 (the >> only change to webrev_01 is Docs.gmk): >> >> http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_02/ >> >> Thanks, >> Joe >> >> On 8/21/20 12:49 PM, naoto.sato at oracle.com wrote: >>> +1 >>> >>> Naoto >>> >>> On 8/21/20 12:24 PM, Lance Andersen wrote: >>>> Hi Joe, >>>> >>>> This looks OK. >>>> >>>> >>>> >>>>> On Aug 21, 2020, at 2:23 PM, Joe Wang wrote: >>>>> >>>>> Pelase review a patch to add the missing @return, @throws, @param >>>>> statements in the java.xml package (excluding the DOM component). >>>>> >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8251561 >>>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8251995 >>>>> >>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_01/ >>>>> >>>>> Thanks, >>>>> Joe >>>> >>>> >>>> Best >>>> Lance >>>> ------------------ >>>> >>>> >>>> >>>> >>>> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>>> Oracle Java Engineering >>>> 1 Network Drive >>>> Burlington, MA 01803 >>>> Lance.Andersen at oracle.com >>>> >>>> >>>> >>>> >> > > > Best > Lance > ------------------ > > > > Lance Andersen| Principal?Member of Technical Staff |?+1.781.442.2037 > Oracle?Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > > From alexander.matveev at oracle.com Tue Aug 25 01:04:18 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Mon, 24 Aug 2020 18:04:18 -0700 Subject: RFR: JDK-8251988: jpackage --runtime-image fails on mac when using JDK11 based runtime. In-Reply-To: <54ce0560-ad89-4a5d-fea8-690b5ad6fecb@oracle.com> References: <67dd0e2d-d107-120b-d1bd-006f3955b81e@oracle.com> <25da9b8b-b9ef-fceb-137c-55c659e75202@oracle.com> <100096c9-d97d-73b3-1b26-3c7dcb408ce5@oracle.com> <2163112d-d071-8f78-0013-9a0a19f4c4da@oracle.com> <7a765dc1-4910-e3ec-e89f-5cdc93bef622@oracle.com> <54ce0560-ad89-4a5d-fea8-690b5ad6fecb@oracle.com> Message-ID: Hi Andy, Looks good. Thanks, Alexander On 8/24/20 4:53 PM, Alexey Semenyuk wrote: > Looks good! > > - Alexey > > On 8/24/2020 6:23 PM, Andy Herrick wrote: >> >> On 8/24/2020 5:08 PM, Alexey Semenyuk wrote: >>> Andy, >>> >>> I'd propose instead of checking if the app is bundled with proper >>> runtime in the test case, enforce the test case to always create >>> runtime for the app with JPackageCommand.ignoreDefaultRuntime(true) >>> call: >>> --- >>> JPackageCommand cmd = JPackageCommand.helloAppImage( >>> ? 71 JavaAppDesc.parse("com.hello/com.hello.Hello")) >>> ... >>> ? 76???????????????? .addArguments(...) >>> ?????????????????????? .ignoreDefaultRuntime(true); >> OK. >>> --- >>> With ignoreDefaultRuntime(true) call added there is no need to call >>> JPackageCommand.isFakeRuntime(). >>> >>> Is com.hello/com/hello/Hello.java used at all? I'd guess after we >>> had refactor all package tests this source is not used any more. Can >>> it be removed yet? >> >> It looked like ArgumentTest was using it, but I removed it and all >> still run >> >> revised webrev: http://cr.openjdk.java.net/~herrick/8251988/webrev.04/ >> >> /Andy >> >> >>> >>> - Alexey >>> >>> On 8/24/2020 4:50 PM, Andy Herrick wrote: >>>> please review revised webrev at: >>>> http://cr.openjdk.java.net/~herrick/8251988/webrev.03/ >>>> >>>> jpackage.app-version set in cfg file, and jpackage.app-path now set >>>> from native code. >>>> >>>> New testcase in BasicTest to check that they are set. >>>> >>>> /Andy >>>> >>>> On 8/20/2020 7:36 PM, Alexey Semenyuk wrote: >>>>> Andy, >>>>> >>>>> I'd replace 'File.separator' with '/' to make value of >>>>> 'jpackage.app-path' property platform independent. >>>>> It would be also good to have a unit test verifying values of the >>>>> new properties. >>>>> >>>>> - Alexey >>>>> >>>>> On 8/20/2020 5:44 PM, Andy Herrick wrote: >>>>>> Please review the jpackage fix at [1] to issue [2]. >>>>>> >>>>>> This fixes the problem loading libjli.dylib as well as providing >>>>>> some useful generic system properties. >>>>>> >>>>>> /Andy >>>>>> >>>>>> >>>>>> [1] - http://cr.openjdk.java.net/~herrick/8251988/webrev.02/ >>>>>> >>>>>> >>>>>> [2] - https://bugs.openjdk.java.net/browse/JDK-8251988 >>>>>> >>>>> >>> > From cjashfor at linux.ibm.com Tue Aug 25 01:21:59 2020 From: cjashfor at linux.ibm.com (Corey Ashford) Date: Mon, 24 Aug 2020 18:21:59 -0700 Subject: RFR(M): 8248188: [PATCH] Add HotSpotIntrinsicCandidate and API for Base64 decoding In-Reply-To: References: <11ca749f-3015-c004-aa6b-3194e1dfe4eb@linux.ibm.com> Message-ID: <8ece8d2e-fd99-b734-211e-a32b534a7dc8@linux.ibm.com> Here's a revised webrev which includes a JMH benchmark for the decode operation. http://cr.openjdk.java.net/~mhorie/8248188/webrev.03/ The added benchmark tries to be "fair" in that it doesn't prefer a large buffer size, which would favor the intrinsic. It pseudo-randomly (but reproducibly) chooses a buffer size between 8 and 20k+8 bytes, and fills it with random data to encode and decode. As part of the TearDown of an invocation, it also checks the decoded output data for correctness. Example runs on the Power9-based machine I use for development shows a 3X average improvement across these random buffer sizes. Here's an excerpt of the output when run with -XX:-UseBASE64Intrinsics : Iteration 1: 70795.623 ops/s Iteration 2: 71070.607 ops/s Iteration 3: 70867.544 ops/s Iteration 4: 71107.992 ops/s Iteration 5: 71048.281 ops/s And here's the output with the intrinsic enabled: Iteration 1: 208794.022 ops/s Iteration 2: 208630.904 ops/s Iteration 3: 208238.822 ops/s Iteration 4: 208714.967 ops/s Iteration 5: 209060.894 ops/s Taking the best of the two runs: 209060/71048 = 2.94 From other experiments where the benchmark uses a fixed-size, larger buffer, the performance ratio rises to about 4.0. Power10 should have a slightly higher ratio due to several factors, but I have not yet benchmarked on Power10. Other arches ought to be able to do at least this well, if not better, because of wider vector registers (> 128 bits) being available. Only a Power9/10 implementation is included in this webrev, however. Regards, - Corey On 8/19/20 11:20 AM, Roger Riggs wrote: > Hi Corey, > > For changes obviously performance motivated, it is conventional to run a > JMH perf test to demonstate > the improvement and prove it is worthwhile to add code complexity. > > I don't see any existing Base64 JMH tests but they would be in the repo > below or near: > ??? test/micro/org/openjdk/bench/java/util/ > > Please contribute a JMH test and results to show the difference. > > Regards, Roger > > > > On 8/19/20 2:10 PM, Corey Ashford wrote: >> Michihiro Horie posted up a new iteration of this webrev for me.? This >> time the webrev includes a complete implementation of the intrinsic >> for Power9 and Power10. >> >> You can find it here: >> http://cr.openjdk.java.net/~mhorie/8248188/webrev.02/ >> >> Changes in webrev.02 vs. webrev.01: >> >> ? * The method header for the intrinsic in the Base64 code has been >> rewritten using the Javadoc style.? The clarity of the comments has >> been improved and some verbosity has been removed. There are no >> additional functional changes to Base64.java. >> >> ? * The code needed to martial and check the intrinsic parameters has >> been added, using the base64 encodeBlock intrinsic as a guideline. >> >> ? * A complete intrinsic implementation for Power9 and Power10 is >> included. >> >> ? * Adds some Power9 and Power10 assembler instructions needed by the >> intrinsic which hadn't been defined before. >> >> The intrinsic implementation in this patch accelerates the decoding of >> large blocks of base64 data by a factor of about 3.5X on Power9. >> >> I'm attaching two Java test cases I am using for testing and >> benchmarking.? The TestBase64_VB encodes and decodes randomly-sized >> buffers of random data and checks that original data matches the >> encoded-then-decoded data.? TestBase64Errors encodes a 48K block of >> random bytes, then corrupts each byte of the encoded data, one at a >> time, checking to see if the decoder catches the illegal byte. >> >> Any comments/suggestions would be appreciated. >> >> Thanks, >> >> - Corey >> >> On 7/27/20 6:49 PM, Corey Ashford wrote: >>> Michihiro Horie uploaded a new revision of the Base64 decodeBlock >>> intrinsic API for me: >>> >>> http://cr.openjdk.java.net/~mhorie/8248188/webrev.01/ >>> >>> It has the following changes with respect to the original one posted: >>> >>> ??* In the event of encountering a non-base64 character, instead of >>> having a separate error code of -1, the intrinsic can now just return >>> either 0, or the number of data bytes produced up to the point where >>> the illegal base64 character was encountered. This reduces the number >>> of special cases, and also provides a way to speed up the process of >>> finding the bad character by the slower, pure-Java algorithm. >>> >>> ??* The isMIME boolean is removed from the API for two reasons: >>> ??? - The current API is not sufficient to handle the isMIME case, >>> because there isn't a strict relationship between the number of input >>> bytes and the number of output bytes, because there can be an >>> arbitrary number of non-base64 characters in the source. >>> ??? - If an intrinsic only implements the (isMIME == false) case as >>> ours does, it will always return 0 bytes processed, which will >>> slightly slow down the normal path of processing an (isMIME == true) >>> instantiation. >>> ??? - We considered adding a separate hotspot candidate for the >>> (isMIME == true) case, but since we don't have an intrinsic >>> implementation to test that, we decided to leave it as a future >>> optimization. >>> >>> Comments and suggestions are welcome.? Thanks for your consideration. >>> >>> - Corey >>> >>> On 6/23/20 6:23 PM, Michihiro Horie wrote: >>>> Hi Corey, >>>> >>>> Following is the issue I created. >>>> https://bugs.openjdk.java.net/browse/JDK-8248188 >>>> >>>> I will upload a webrev when you're ready as we talked in private. >>>> >>>> Best regards, >>>> Michihiro >>>> >>>> Inactive hide details for "Corey Ashford" ---2020/06/24 >>>> 09:40:10---Currently in java.util.Base64, there is a >>>> HotSpotIntrinsicCa"Corey Ashford" ---2020/06/24 09:40:10---Currently >>>> in java.util.Base64, there is a HotSpotIntrinsicCandidate and API >>>> for encodeBlock, but no >>>> >>>> From: "Corey Ashford" >>>> To: "hotspot-compiler-dev at openjdk.java.net" >>>> , >>>> "ppc-aix-port-dev at openjdk.java.net" >>>> Cc: Michihiro Horie/Japan/IBM at IBMJP, Kazunori Ogata/Japan/IBM at IBMJP, >>>> joserz at br.ibm.com >>>> Date: 2020/06/24 09:40 >>>> Subject: RFR(S): [PATCH] Add HotSpotIntrinsicCandidate and API for >>>> Base64 decoding >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> >>>> >>>> >>>> Currently in java.util.Base64, there is a HotSpotIntrinsicCandidate and >>>> API for encodeBlock, but none for decoding. ?This means that only >>>> encoding gets acceleration from the underlying CPU's vector hardware. >>>> >>>> I'd like to propose adding a new intrinsic for decodeBlock. ?The >>>> considerations I have for this new intrinsic's API: >>>> >>>> ??* Don't make any assumptions about the underlying capability of the >>>> hardware. ?For example, do not impose any specific block size >>>> granularity. >>>> >>>> ??* Don't assume the underlying intrinsic can handle isMIME or isURL >>>> modes, but also let them decide if they will process the data >>>> regardless >>>> of the settings of the two booleans. >>>> >>>> ??* Any remaining data that is not processed by the intrinsic will be >>>> processed by the pure Java implementation. ?This allows the >>>> intrinsic to >>>> process whatever block sizes it's good at without the complexity of >>>> handling the end fragments. >>>> >>>> ??* If any illegal character is discovered in the decoding process, the >>>> intrinsic will simply return -1, instead of requiring it to throw a >>>> proper exception from the context of the intrinsic. ?In the event of >>>> getting a -1 returned from the intrinsic, the Java Base64 library code >>>> simply calls the pure Java implementation to have it find the error and >>>> properly throw an exception. ?This is a performance trade-off in the >>>> case of an error (which I expect to be very rare). >>>> >>>> ??* One thought I have for a further optimization (not implemented in >>>> the current patch), is that when the intrinsic decides not to process a >>>> block because of some combination of isURL and isMIME settings it >>>> doesn't handle, it could return extra bits in the return code, encoded >>>> as a negative number. ?For example: >>>> >>>> Illegal_Base64_char ? = 0b001; >>>> isMIME_unsupported ? ?= 0b010; >>>> isURL_unsupported ? ? = 0b100; >>>> >>>> These can be OR'd together as needed and then negated (flip the sign). >>>> The Base64 library code could then cache these flags, so it will know >>>> not to call the intrinsic again when another decodeBlock is requested >>>> but with an unsupported mode. ?This will save the performance hit of >>>> calling the intrinsic when it is guaranteed to fail. >>>> >>>> I've tested the attached patch with an actual intrinsic coded up for >>>> Power9/Power10, but those runtime intrinsics and arch-specific patches >>>> aren't attached today. ?I want to get some consensus on the >>>> library-level intrinsic API first. >>>> >>>> Also attached is a simple test case to test that the new intrinsic API >>>> doesn't break anything. >>>> >>>> I'm open to any comments about this. >>>> >>>> Thanks for your consideration, >>>> >>>> - Corey >>>> >>>> >>>> Corey Ashford >>>> IBM Systems, Linux Technology Center, OpenJDK team >>>> cjashfor at us dot ibm dot com >>>> [attachment "decodeBlock_api-20200623.patch" deleted by Michihiro >>>> Horie/Japan/IBM] [attachment "TestBase64.java" deleted by Michihiro >>>> Horie/Japan/IBM] >>>> >>>> >>> >> > From mandy.chung at oracle.com Tue Aug 25 02:19:38 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 24 Aug 2020 19:19:38 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: <98b8b0c5-b9b5-8c68-15fc-629164203e3d@oracle.com> References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> <98b8b0c5-b9b5-8c68-15fc-629164203e3d@oracle.com> Message-ID: <6e224101-b75d-f825-c3fb-80637d183d07@oracle.com> On 8/24/20 3:53 PM, Ioi Lam wrote: > > : > > However, I suspect most people won't do this, because the benefit is > relatively small. Also, jlink doesn't support classpath apps, so you > would need to figure out what to use for "--add-modules". In the worst > case, the custom JDK would be over 100MB in size. > `jdeps --print-module-deps` can find the static dependencies.? It will need to know its reflective dependencies and any service providers it depends on.? It's no different for creating a custom image for modular apps. Mandy From gilles.m.duboscq at oracle.com Tue Aug 25 10:16:12 2020 From: gilles.m.duboscq at oracle.com (Gilles Duboscq) Date: Tue, 25 Aug 2020 12:16:12 +0200 Subject: RFR: JDK-8242451 : ensure semantics of non-capturing lambdas are preserved independent of execution mode In-Reply-To: <71c26ec5-7512-e5e0-349f-a488357677f1@oracle.com> References: <71c26ec5-7512-e5e0-349f-a488357677f1@oracle.com> Message-ID: Hi Mandy, Thanks for the comments. On 24/06/2020 02:56, Mandy Chung wrote: > Hi Gilles, > > Additional comments: > > 215 try { > 216 return new ConstantCallSite(Lookup.IMPL_LOOKUP.findStaticGetter(innerClass, LAMBDA_INSTANCE_FIELD, invokedType.returnType())); > 217 } catch (ReflectiveOperationException e) { > 218 throw new LambdaConversionException("Exception finding constructor", e); > 219 } > > > This should use caller instead Lookup.IMPL_LOOKUP (as I suggested in my previous repl).?? The exception message should be "Exception finding " + LAMBDA_INSTANCE_FIELD + " static field". Done. > > 418 private void generateStaticField() { > > > I would rename this to generateClassInitializer since this adds "" and initializes the static field. Done > > Since this patch caches a singleton instance in the generated class, it could apply to the eager initialization case as well and replace the current use of core reflection to new an instance except that the target of the returning callsite would always be the singleton object (the result of invoking the static getter method handle).? I wonder if there is any performance difference.? This is just a thought that we can file a JBS issue to follow up. > > Can you add a test case for this fix? I could write a test that generates different output depending on whether a singleton or a fresh instance is returned. Then i can compare the output when running with `jdk.internal.lambda.disableEagerInitialization` set to `true` and `false`. What is the recommended way of comparing 2 runs like that in jtreg? I have updated the webrev: https://cr.openjdk.java.net/~gdub/8242451/webrev.1/ From your other mail: > Should this patch be a workaround to existing releases rather than the main line? As Brian mentions, lambda proxy class may become inline class in valhalla repo (Roger has a patch already). The earlier fixing those programs the better. Indeed if we know this is landing in this cycle in the main repo there's no point with my fix. Could you point me at the issue number or mail thread where this patch is being discussed? Gilles > > Mandy > > On 6/23/20 11:08 AM, Gilles Duboscq wrote: >> In 8232806, a system property was introduce to disable eager initialization of the classes generated by the InnerClassLambdaMetafactory (`jdk.internal.lambda.disableEagerInitialization`). >> >> However, when `disableEagerInitialization` is true, even for non-capturing lambdas, the capturing lambda path that uses a method handle to the constructor is taken. >> This helps prevent eager initialization but also has the side effect of always returning a fresh instance of the lambda for every invocation instead of a singleton. >> While this is allowed by the specs, this might change the behaviour of some (incorrect) programs that assume a singleton is used for non-capturing lambdas. >> >> I propose to keep the effects of the `disableEagerInitialization` related to initialization only. >> >> Webrev:https://cr.openjdk.java.net/~gdub/8242451/webrev.0/ >> Issue:https://bugs.openjdk.java.net/browse/JDK-8242451 >> >> The concrete issue we are seeing with changing both aspects at the same time is that when disableEagerInitialization for static analysis in GraalVM's native-image, some programs start to miss-behave because of assumptions about the object identity of lambdas. >> >> >> Note that `disableEagerInitialization` is still ineffective in the following situations: >> * when `useImplMethodHandle` is true, i.e., when a MethodHanlde is used to call the target because the generated hidden class is missing the necessary access rights. >> (the implementation require setting a static field on the generated class which causes it to be initialized, Class Data could help in the future in that case) >> * for non-capturing lambdas when the caller (and generated) class is in the `java.lang.invoke` or `sun.invoke.util` package. >> (because `findStaticGetter` will eagerly initialize the holder class if it is from those packages, see DirectMethodHandle#shouldBeInitialized) >> >> Those are acceptable rare corner cases. >> >> Thanks, >> Gilles > From martin.doerr at sap.com Tue Aug 25 13:37:38 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Tue, 25 Aug 2020 13:37:38 +0000 Subject: RFR(M): 8248188: [PATCH] Add HotSpotIntrinsicCandidate and API for Base64 decoding In-Reply-To: References: <11ca749f-3015-c004-aa6b-3194e1dfe4eb@linux.ibm.com> Message-ID: Hi Corey, thanks for proposing this change. I have comments and suggestions regarding various files. Base64.java This is the only file which needs another review from core-libs-dev. First of all, I like the idea to use a HotSpotIntrinsicCandidate which can consume as many bytes as the implementation wants. Comment before decodeBlock: Let's be precise: "should process a multiple of four" => "must process a multiple of four" > If any illegal base64 bytes are encountered in the source by the > intrinsic, the intrinsic can return a data length of zero or any > number of bytes before the place where the illegal base64 byte > was encountered. I think this has a drawback. Somebody may use a debugger and want to stop when throwing IllegalArgumentException. He should see the position which matches the Java implementation. Please note that the comment indentation differs from other comments. decode0: Final "else" after return is redundant. stubGenerator_ppc.cpp "__vector" breaks AIX build! Does it work on Big Endian linux with old gcc (we require 7.3.1, now)? Please either support Big Endian properly or #ifdef it out. What exactly does it on linux? I remember that we had tried such prefixes but were not satisfied. I think it didn't enforce 16 Byte alignment if I remember correctly. Attention: C2 does no longer convert int/bool to 64 bit values (since JDK-8086069). So the argument registers for offset, length and isURL may contain garbage in the higher bits. You may want to use load_const_optimized which produces shorter code. You may want to use __ align(32) to align unrolled_loop_start. I'll review the algorithm in detail when I find more time. assembler_ppc.hpp assembler_ppc.inline.hpp vm_version_ppc.cpp vm_version_ppc.hpp Please rebase. Parts of the change were pushed as part of 8248190: Enable Power10 system and implement new byte-reverse instructions vmSymbols.hpp Indentation looks odd at the end. library_call.cpp Good. Indentation style of the call parameters differs from encodeBlock. runtime.cpp Good. aotCodeHeap.cpp vmSymbols.cpp shenandoahSupport.cpp vmStructs_jvmci.cpp shenandoahSupport.cpp escape.cpp runtime.hpp stubRoutines.cpp stubRoutines.hpp vmStructs.cpp Good and trivial. Tests: I think we should have JTREG tests to check for regressions in the future. Best regards, Martin > -----Original Message----- > From: Corey Ashford > Sent: Mittwoch, 19. August 2020 20:11 > To: Michihiro Horie > Cc: hotspot-compiler-dev at openjdk.java.net; core-libs-dev dev at openjdk.java.net>; Kazunori Ogata ; > joserz at br.ibm.com; Doerr, Martin > Subject: Re: RFR(M): 8248188: [PATCH] Add HotSpotIntrinsicCandidate and > API for Base64 decoding > > Michihiro Horie posted up a new iteration of this webrev for me. This > time the webrev includes a complete implementation of the intrinsic for > Power9 and Power10. > > You can find it here: > http://cr.openjdk.java.net/~mhorie/8248188/webrev.02/ > > Changes in webrev.02 vs. webrev.01: > > * The method header for the intrinsic in the Base64 code has been > rewritten using the Javadoc style. The clarity of the comments has been > improved and some verbosity has been removed. There are no additional > functional changes to Base64.java. > > * The code needed to martial and check the intrinsic parameters has > been added, using the base64 encodeBlock intrinsic as a guideline. > > * A complete intrinsic implementation for Power9 and Power10 is included. > > * Adds some Power9 and Power10 assembler instructions needed by the > intrinsic which hadn't been defined before. > > The intrinsic implementation in this patch accelerates the decoding of > large blocks of base64 data by a factor of about 3.5X on Power9. > > I'm attaching two Java test cases I am using for testing and > benchmarking. The TestBase64_VB encodes and decodes randomly-sized > buffers of random data and checks that original data matches the > encoded-then-decoded data. TestBase64Errors encodes a 48K block of > random bytes, then corrupts each byte of the encoded data, one at a > time, checking to see if the decoder catches the illegal byte. > > Any comments/suggestions would be appreciated. > > Thanks, > > - Corey > > On 7/27/20 6:49 PM, Corey Ashford wrote: > > Michihiro Horie uploaded a new revision of the Base64 decodeBlock > > intrinsic API for me: > > > > http://cr.openjdk.java.net/~mhorie/8248188/webrev.01/ > > > > It has the following changes with respect to the original one posted: > > > > ?* In the event of encountering a non-base64 character, instead of > > having a separate error code of -1, the intrinsic can now just return > > either 0, or the number of data bytes produced up to the point where the > > illegal base64 character was encountered.? This reduces the number of > > special cases, and also provides a way to speed up the process of > > finding the bad character by the slower, pure-Java algorithm. > > > > ?* The isMIME boolean is removed from the API for two reasons: > > ?? - The current API is not sufficient to handle the isMIME case, > > because there isn't a strict relationship between the number of input > > bytes and the number of output bytes, because there can be an arbitrary > > number of non-base64 characters in the source. > > ?? - If an intrinsic only implements the (isMIME == false) case as ours > > does, it will always return 0 bytes processed, which will slightly slow > > down the normal path of processing an (isMIME == true) instantiation. > > ?? - We considered adding a separate hotspot candidate for the (isMIME > > == true) case, but since we don't have an intrinsic implementation to > > test that, we decided to leave it as a future optimization. > > > > Comments and suggestions are welcome.? Thanks for your consideration. > > > > - Corey > > > > On 6/23/20 6:23 PM, Michihiro Horie wrote: > >> Hi Corey, > >> > >> Following is the issue I created. > >> https://bugs.openjdk.java.net/browse/JDK-8248188 > >> > >> I will upload a webrev when you're ready as we talked in private. > >> > >> Best regards, > >> Michihiro > >> > >> Inactive hide details for "Corey Ashford" ---2020/06/24 > >> 09:40:10---Currently in java.util.Base64, there is a > >> HotSpotIntrinsicCa"Corey Ashford" ---2020/06/24 09:40:10---Currently > >> in java.util.Base64, there is a HotSpotIntrinsicCandidate and API for > >> encodeBlock, but no > >> > >> From: "Corey Ashford" > >> To: "hotspot-compiler-dev at openjdk.java.net" > >> , > >> "ppc-aix-port-dev at openjdk.java.net" dev at openjdk.java.net> > >> Cc: Michihiro Horie/Japan/IBM at IBMJP, Kazunori > Ogata/Japan/IBM at IBMJP, > >> joserz at br.ibm.com > >> Date: 2020/06/24 09:40 > >> Subject: RFR(S): [PATCH] Add HotSpotIntrinsicCandidate and API for > >> Base64 decoding > >> > >> ------------------------------------------------------------------------ > >> > >> > >> > >> Currently in java.util.Base64, there is a HotSpotIntrinsicCandidate and > >> API for encodeBlock, but none for decoding. ?This means that only > >> encoding gets acceleration from the underlying CPU's vector hardware. > >> > >> I'd like to propose adding a new intrinsic for decodeBlock. ?The > >> considerations I have for this new intrinsic's API: > >> > >> ??* Don't make any assumptions about the underlying capability of the > >> hardware. ?For example, do not impose any specific block size > >> granularity. > >> > >> ??* Don't assume the underlying intrinsic can handle isMIME or isURL > >> modes, but also let them decide if they will process the data regardless > >> of the settings of the two booleans. > >> > >> ??* Any remaining data that is not processed by the intrinsic will be > >> processed by the pure Java implementation. ?This allows the intrinsic to > >> process whatever block sizes it's good at without the complexity of > >> handling the end fragments. > >> > >> ??* If any illegal character is discovered in the decoding process, the > >> intrinsic will simply return -1, instead of requiring it to throw a > >> proper exception from the context of the intrinsic. ?In the event of > >> getting a -1 returned from the intrinsic, the Java Base64 library code > >> simply calls the pure Java implementation to have it find the error and > >> properly throw an exception. ?This is a performance trade-off in the > >> case of an error (which I expect to be very rare). > >> > >> ??* One thought I have for a further optimization (not implemented in > >> the current patch), is that when the intrinsic decides not to process a > >> block because of some combination of isURL and isMIME settings it > >> doesn't handle, it could return extra bits in the return code, encoded > >> as a negative number. ?For example: > >> > >> Illegal_Base64_char ? = 0b001; > >> isMIME_unsupported ? ?= 0b010; > >> isURL_unsupported ? ? = 0b100; > >> > >> These can be OR'd together as needed and then negated (flip the sign). > >> The Base64 library code could then cache these flags, so it will know > >> not to call the intrinsic again when another decodeBlock is requested > >> but with an unsupported mode. ?This will save the performance hit of > >> calling the intrinsic when it is guaranteed to fail. > >> > >> I've tested the attached patch with an actual intrinsic coded up for > >> Power9/Power10, but those runtime intrinsics and arch-specific patches > >> aren't attached today. ?I want to get some consensus on the > >> library-level intrinsic API first. > >> > >> Also attached is a simple test case to test that the new intrinsic API > >> doesn't break anything. > >> > >> I'm open to any comments about this. > >> > >> Thanks for your consideration, > >> > >> - Corey > >> > >> > >> Corey Ashford > >> IBM Systems, Linux Technology Center, OpenJDK team > >> cjashfor at us dot ibm dot com > >> [attachment "decodeBlock_api-20200623.patch" deleted by Michihiro > >> Horie/Japan/IBM] [attachment "TestBase64.java" deleted by Michihiro > >> Horie/Japan/IBM] > >> > >> > > From Roger.Riggs at oracle.com Tue Aug 25 14:26:02 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Tue, 25 Aug 2020 10:26:02 -0400 Subject: RFR 8251989: Hex encoder and decoder utility In-Reply-To: References: <6743da26-9bae-e868-d242-b359aeec3d21@oracle.com> Message-ID: <3d35d702-1402-e308-4347-95d30a09f5a1@oracle.com> Hi Corey, If there's a performance issue, that's an option. Though its tempting to add intrinsics for each supported instruction set, it adds code that has to be maintained forever, so it best to wait till its determined to be a bottleneck. Most of the cases I've seen in the code are fairly small, and in non-performance critical data paths, for example, the toString method of a certificate. Thanks, Roger On 8/24/20 5:11 PM, Corey Ashford wrote: > Just as a general comment, an SIMD-based encode() intrinsic would be > an easy performance upgrade for this Hex class. I'd guess you'd get a > 5x-10x improvement, depending on the specific CPU/arch being used.? > That said, the intrinsic would do better on larger buffers, and I > don't know what the average length is for the use cases you have in mind. > > - Corey > > On 8/19/20 2:14 PM, Roger Riggs wrote: >> Please review a java.util.Hex API to encode and decode hexadecimal >> strings to and from byte arrays. >> >> Within the JDK and JDK tests there are multiple implementations to >> encode and decode >> hexadecimal strings to byte arrays. Hex encoders and decoders support >> upper or lower case hexadecimal characters, delimiters, prefix, and >> suffix. >> The API is modeled after the java.util.Base64 API providing static >> factories, >> immutable threadsafe instances with methods to encode to and decode from >> string and StringBuilder. >> >> JavaDoc: >> http://cr.openjdk.java.net/~rriggs/hex-javadoc/java.base/java/util/Hex.html >> >> >> Webrev for Hex encoder and decoder: >> http://cr.openjdk.java.net/~rriggs/webrev-hex-encoder-8251989 >> >> Webrev for applying to java.security: >> http://cr.openjdk.java.net/~rriggs/webrev-hex-security-8252055 >> >> CSR: >> ??? https://bugs.openjdk.java.net/browse/JDK-8251991 >> >> Issue for API and a few uses: >> ??? https://bugs.openjdk.java.net/browse/JDK-8251989 >> >> Issue for Use in java.security and tests: >> ??? https://bugs.openjdk.java.net/browse/JDK-8252055 From brian.goetz at oracle.com Tue Aug 25 14:32:38 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 25 Aug 2020 10:32:38 -0400 Subject: RFR: JDK-8242451 : ensure semantics of non-capturing lambdas are preserved independent of execution mode In-Reply-To: References: <71c26ec5-7512-e5e0-349f-a488357677f1@oracle.com> Message-ID: <8b23a10b-ba93-ab3d-b4d7-604fd8646eae@oracle.com> > >> Should this patch be a workaround to existing releases rather than >> the main line??? As Brian mentions, lambda proxy class may become >> inline class in valhalla repo (Roger has a patch already).?? The >> earlier fixing those programs the better. > > Indeed if we know this is landing in this cycle in the main repo > there's no point with my fix. Could you point me at the issue number > or mail thread where this patch is being discussed? Progress continues on Valhalla, and this is clearly where things are going, but integration is not imminent. But, the main point here is: if the goal is to protect broken programs that make incorrect identity assumptions, said protection is inherently short-term; when Valhalla lands, all these programs will be broken, and there will be no interest in punishing the other 99.999% of code just to continue to coddle them.? So we should be clear that this is at most a short-term reprieve. From lois.foltan at oracle.com Tue Aug 25 14:58:50 2020 From: lois.foltan at oracle.com (Lois Foltan) Date: Tue, 25 Aug 2020 10:58:50 -0400 Subject: RFR(L) 8244778 Archive full module graph in CDS In-Reply-To: References: <9e6b0043-65a1-dd97-a3d2-33679c8048d4@oracle.com> Message-ID: <234856fa-1eb1-8514-6786-bce6689afd16@oracle.com> Hi Ioi, Changes looks really good.? Comments interspersed below. Thanks, Lois On 8/12/2020 6:06 PM, Ioi Lam wrote: > Hi Lois, > > Thanks for the comments. I have an updated webrev > > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/ > > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02.delta/ > > > Here are the general notes on the changes. Replies to your questions > are in-line below. > > (1) Integrated updates in the Java code from Alan Bateman. Here are > Alan's > ? ? notes: > > ??? The archive of the boot layer is as before, except that archiving is > ??? skipped if there are split packages or incubator modules. Incubating > ??? modules aren't resolved by default so we shouldn't have them in the > ??? boot layer by default anyway. > > ??? I've dropped the module finders from the boot layer archive. I've > ??? left the IllegalAccessLogger.Builder in the acrhive for now (even > ??? though it is not the boot layer). We should be able to remove that > ??? once the JEP to disallow illegal access by default is in. > > ??? Related is that I don't like the archive for the module graph > ??? (ArchivedModuleGraph, pre-dates this RFE) including the set of > ??? packages to export/open for illegal access as they aren't part > ??? of the module graph. I've left it for now but we can also remove > ??? that once we disallow illegal access by default (as those sets > ??? will be empty). > > ??? The archive of built-in class loaders is now in one object > ??? jdk.internal.loader.ArchivedClassLoaders which I think will be a > ??? bit more maintainable. I've also drop the ucp field from the > ??? AppClassLoader as the changes to BuiltinClassLoader means is no > ??? longer needs to duplicated. > > ??? There's one remaining issue in ModuleBootstrap.boot where it has fix > ??? an app class loader value (ModuleLayer.CLV). Ideally the > initialization > ??? of the built-in class loaders would do this but we are kinda forced > ??? to separate the archiving of the built-in class loaders from the boot > ??? layer. I might look at this again some time. > > > (2) Moved code from classLoaderData.cpp -> classLoaderSharedData.cpp > > (3) Reverted unnecessary changes in JavaClasses::compute_offset > > (4) Minor clean up to use QuickSort::sort() instead of qsort() > > (5) Moved the C-side of module initialization for platform/system > ? ? loaders to inside java.lang.System::initPhase2(), so this happens > ??? at the same time as without full module archiving. > > (6) Moved the use of Module_lock to so all modules in a class loader > ??? are restored atomically. See ArchivedClassLoaderData::restore() > > ??? This fixed a bug where test/jdk/com/sun/jdi/ModulesTest.java > ? ? would fail as it sees a partially restored set of modules. > > > > On 8/7/20 12:06 PM, Lois Foltan wrote: >> Hi Ioi, >> >> Overall looks promising.? I have some review comments below, but not >> a complete review.? I concentrated on the JVM side primarily how the >> archived module graph is read in, how the ModuleEntry and >> PackageEntry tables are created from the archive for the 3 builtin >> loaders and potential timing issues during start up. >> >> On 7/22/2020 3:36 PM, Ioi Lam wrote: >>> https://bugs.openjdk.java.net/browse/JDK-8244778 >>> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/ >>> >>> >>> Please review this patch that stores the full module graph in the CDS >>> archive heap. This reduces the initialization time of the basic JVM by >>> about 22%: >>> >>> $ perf stat -r 100 bin/java -version >>> before: 98,219,329 instructions 0.03971 secs elapsed (+- 0.44%) >>> after:? 55,835,815 instructions 0.03109 secs elapsed (+- 0.65%) >>> >>> [1] Start with ModuleBootstrap.java. The current implementation is >>> ??? quite restrictive: the archived module graph is used only when no >>> ??? module options are specified. >>> >>> ??? See ModuleBootstrap.mayUseArchivedBootLayer(). >>> >>> ??? We can probably support options such as main module and module path >>> ??? in a future RFE. >> >> Yes, I noticed the restrictions.? The JBS issue discusses the >> possibility of using the dumped module graph in the event that the >> value of the options are the same.? I think for this implementation >> to be viable and have a positive impact you should consider comparing >> at least the options --add-modules, --add-exports and --add-reads. > > I agree. I want to keep the changes minimal in this RFE, and will add > more support for other use cases in follow-on RFEs. Instead of > requiring these options to be unspecified, we can relax the > restriction such that these options must be the same between archive > dump time and run time. Sounds good! > >> >> >>> >>> [2] In the current JDK implementation, there is no single object >>> ??? that represents "the module graph". Most of the information >>> ??? is stored in the archive bootLayer object, but a few additional >>> ??? restoration operations need to be performed: >>> >>> ??? + See ModuleBootstrap.getArchivedBootLayer() >>> ??? + Some static fields need to be archived/restored in >>> ????? Module.java, BuiltinClassLoader.java, ClassLoaders.java >>> ????? and BootLoader.java >>> >>> [3] I ran into a complication with two loader instances of >>> ??? PlatformClassLoader and AppClassLoader. They are stored in >>> ??? multiple tables inside the module graph (e.g., >>> ??? BuiltinClassLoader$LoadedModule) so I cannot easily recreate >>> ??? them at runtime. >>> >>> ??? However, these two loaders contain information specific to the >>> ??? dump time VM lifecycle (such as the classes that were loaded >>> ??? during CDS dumping) that need to be scrubbed. I couldn't find an >>> ??? elegant way of doing this, so I added a private >>> "resetArchivedStates" >>> ??? method to a few classes. They are called inside >>> ??? HeapShared::reset_archived_object_states(). >>> >>> [4] Related native data structures (PackageEntry and ModuleEntry) >>> ??? are also archived. Start with classLoaderData.cpp >>> >>> Passes mach5 tiers 1-4. I will test with additional tiers. >>> >>> Thanks >>> - Ioi >> >> classfile/classLoader.cpp >> - line #1644 pulling the javabase_moduleEntry() check outside of the >> Module_lock maybe problematic if after you check it another >> ? thread initializes in the time between the check and the obtaining >> of the Module_lock on line #1645. > > Fixed. That looks good.? I think it is fine that you are checking if java.base is defined via a call to javabase_moduleEntry() because you are just trying to determine if a ModuleEntry should be created or not.? In most cases though using ModuleEntryTable::javabase_defined() is the correct way to ensure that both the ModuleEntry for java.base has been created and that the ModuleEntry has been injected in the corresponding java.lang.Module oop. > >> >> classfile/classLoader.hpp >> - somewhere in ArchivedClassLoaderData there should be an assert to >> make sure that the CLD, whose package entries and module entries are >> being archived is not a "_has_class_mirror_holder" CLD for a weakly >> defined hidden class.? Those dedicated CLDs should never have package >> entries or module entries. >> > > I added ArchivedClassLoaderData::assert_valid() > >> classfile/moduleEntry.cpp >> - line #400, typo "conver" --> "convert" >> - line #500, maybe sort if n is greater than 1 instead of 0 (same >> comment for packageEntry.cpp line #270) >> > Fixed > >> classfile/systemDictionary.cpp >> - It looks like the PackageEntry and ModuleEntry tables for the >> system and platform class loaders are? added within >> SystemDictionary::compute_java_loaders() which is called from >> Thread::create_vm() but after the call in Thread::create_vm() to >> call_initPhase2 which is when Universe::_module_initialized is set to >> true.? Did I read this correctly?? If yes, then I think you have to >> be sure that Universe::_module_initialized is not set until the >> module graph for the 3 builtin loaders is loaded from the archive. >> > > I moved the code to be called by ModuleBootstrap::boot() so it should > now happen inside call_initPhase2. Yes, that looks good. > >> memory/filemap.cpp >> - line #237 typo "modue" --> "module" >> > > Fixed > >> - Since the CDS support for capturing the module graph does not >> archive unnamed modules, I assume >> Modules::set_bootloader_unnamed_module() is still called.? Is this >> true or does the unnamed module for the boot loader get archived? > The unnamed module for the boot loader is not archived. > > Modules::set_bootloader_unnamed_module()? wasn't called in my last > webrev. Thanks for catching it. > > I added a call to BootLoader.getUnnamedModule() in > ModuleBootstrap::boot()? to trigger of BootLoader, which will > call into the VM for Modules::set_bootloader_unnamed_module(). Looks good. > > > >> Clarification/timing questions: >> > > Here's an overall problem I am facing: > > The module graph is archived after the module system has fully started > up. This means that for the boot loader, we only know the full set of > modules/packages, but we don't know which part is the subset that was > initialized during early VM bootstraping (e.g., when > ModuleEntryTable::javabase_defined() == false). > > So the behavior is a bit different during the early bootstrap phase > (all the way before we reach ModuleBootstrap::boot()): > ClassLoaderData::the_null_class_loader_data()->modules() and > ClassLoaderData::the_null_class_loader_data()->packages() are already > populated before a single class is loaded. If this is the case then, at the point when a ModuleEntry is created for java.base using the archived module graph, there should be a assertion that java_lang_Class::_fixup_module_field_list is NULL. To confirm no class has been loaded before java.base is defined. Maybe in ClassLoaderDataShared::serialize where you restore the boot loader's archived modules? > This difference doesn't seem to make a practical difference. E.g., > none of our code seems to assume that "before any classes in the > java.util package is loaded, > ClassLoaderData::the_null_class_loader_data()->packages() must not > contain an entry for java.util". > > I think we have two choices when the archived module graph is used: > > [1] We require that the state of the module system must, at every step > of VM initialization, be identical to that of a VM that doesn't use an > archived module graph. > > [2] We make sure that the VM/JDK bootstrap code can tolerate the > existence of module/packages that are added earlier than a VM that > doesn't use an archived module graph. > > I tried doing a version of [1] and found that to be too difficult. [2] > seems much simpler and is the approach I am using now. I think [2] is reasonable. > >> oops/instanceKlass.cpp >> - line #2545, comment "TO DO? -- point it to the archived >> PackageEntry (JDK-8249262)" >> are you thinking that since the module graph is read in ahead of time >> that it can be set when an InstanceKlass is created?? There is a >> point before java.base is defined that InstanceKlass::set_package >> takes into account that could be a timing issue. >> >> > > I think it will work as if another class in the same package has > already been defined. > >> - There are some checks in modules.cpp that are valuable to have >> during bootstrapping.? For example, packages that are encountered >> during bootstrapping before java.base is defined are put into the >> java.base module but then verified after java.base is defined via >> verify_javabase_packages.? So at what point in the bootstrapping >> process does the boot loader's archived module's become known? Could >> classes have been defined to the VM before this point? Then their >> packages will have to be verified to be sure that they are indeed >> packages within java.base.? Consider looking at other checks that >> occur within classfile/modules.cpp as well. >> > As mentioned above, calling verify_javabase_packages() at run time > will fail, as we have loaded all packages for the boot loader, not > just those for java.base. I think not calling verify_javabase_packages works because as you stated above no classes have been loaded before java.base is defined which is not the same situation as running without the archived module graph. A couple of additional comments: - ModuleEntry's reads list and PackageEntry's exports list.? We had hoped eventually to change these lists from being a c heap allocated GrowableArray over to a ResourceHashTable for faster lookup.? Doing that as a separate RFE might help CDS archiving not to have to archive those lists as an Array? - moduleEntry.cpp and packageEntry.cpp: Both methods "compare_module_by_name" and "compare_package_by_name" should have an assert if the names are equal.? No ModuleEntryTable or PackageEntryTable should ever have 2 same named modules or packages. - Another timing clarification question for me.? I assume that the module graph is dumped post module initialization when Java has completely defined the module graph to the JVM, is this correct?? My concern is that there could be a window post module initialization and pre archiving the module graph where another thread could define a new module or add Module readability or add Package exportability.? So that the module graph you are dumping is not the same module graph at the end of module initialization.? Is this a concern? Thanks, Lois > > Well, verify_javabase_packages() was called once and it succeeded, but > that was during CDS dump time. So we know at least we have verified > this once :-) > > Thanks > - Ioi > >> I may have more review comments as I continue to look at this! >> >> Thanks, >> Lois >> >> >> >> >> > From aleksej.efimov at oracle.com Tue Aug 25 15:03:03 2020 From: aleksej.efimov at oracle.com (Aleks Efimov) Date: Tue, 25 Aug 2020 16:03:03 +0100 Subject: [16] RFR: 8251182: Fix "no comment" warnings in java.naming Message-ID: Hi, The documentation of classes from "java.naming" module needs to be improved to resolve javadoc -Xdoclint "no comment" warnings. JBS issue: https://bugs.openjdk.java.net/browse/JDK-8251182 Webrev: http://cr.openjdk.java.net/~aefimov/8251182/00/index.html CSR: https://bugs.openjdk.java.net/browse/JDK-8252310 With Best Regards, Aleksei From Roger.Riggs at oracle.com Tue Aug 25 15:09:23 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Tue, 25 Aug 2020 11:09:23 -0400 Subject: RFR [16/java.xml] 8251561: Fix doclint warnings in the java.xml package In-Reply-To: <3fc299f4-1c0a-1ccd-5d4c-3716775dfa38@oracle.com> References: <3fc299f4-1c0a-1ccd-5d4c-3716775dfa38@oracle.com> Message-ID: Hi Joe, Eliminating the checking for warnings in org.w3c is fine. Please be more specific in the comment. "Ignore the doclint warnings in the W3C DOM package" org/xml/...: If we're suppressing the warnings for org/xml/... then the files changes are unnecessary? Remove the addition of -Xmaxwarns, it should stay the default. Since its a makefile change, please copy build-dev at openjdk.java.net. The first line comments should terminate with a period ".". javax/xml/stream/FactoryConfigurationError.java:40 javax/xml/stream/XMLStreamException.java:41 javax/xml/xpath/XPathException.java:44? And capitable "Serializable". javax/xml/stream/events/Attribute.java: 50; add "normalized" to the @return line so it is the same as the first line. For simple get methods, the @return mimics the first line. javax/xml/stream/events/NotationDeclaration.java:43? add "notation" Thanks, Roger p.s.? There is lots of other cleanup of the javadoc, using @code around true, false, missing periods at the end of first sentences, etc.? But that's a different task. On 8/24/20 5:44 PM, Joe Wang wrote: > Hi all,? adding Roger's comment for the make file to webrev_02 (the > only change to webrev_01 is Docs.gmk): > > http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_02/ > > Thanks, > Joe > > On 8/21/20 12:49 PM, naoto.sato at oracle.com wrote: >> +1 >> >> Naoto >> >> On 8/21/20 12:24 PM, Lance Andersen wrote: >>> Hi Joe, >>> >>> This looks OK. >>> >>> >>> >>>> On Aug 21, 2020, at 2:23 PM, Joe Wang wrote: >>>> >>>> Pelase review a patch to add the missing @return, @throws, @param >>>> statements in the java.xml package (excluding the DOM component). >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8251561 >>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8251995 >>>> >>>> webrev: http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_01/ >>>> >>>> Thanks, >>>> Joe >>> >>> >>> Best >>> Lance >>> ------------------ >>> >>> >>> >>> >>> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>> Oracle Java Engineering >>> 1 Network Drive >>> Burlington, MA 01803 >>> Lance.Andersen at oracle.com >>> >>> >>> >>> > From alexander.scherbatiy at bell-sw.com Tue Aug 25 17:00:37 2020 From: alexander.scherbatiy at bell-sw.com (Alexander Scherbatiy) Date: Tue, 25 Aug 2020 20:00:37 +0300 Subject: RFR 8252248: __SIGRTMAX is not declared in musl libc Message-ID: <1c4ad320-a22a-484a-d370-852f9ecd90f8@bell-sw.com> Hello, Could your review the fix for the issue: ? Bug: https://bugs.openjdk.java.net/browse/JDK-8252248 ? Fix: http://cr.openjdk.java.net/~alexsch/8252248/webrev.00/ Building JDK on Alpine Linux with musl libc gives the error: src/java.base/linux/native/libnet/linux_close.c:63:25: error: '__SIGRTMAX' undeclared here (not in a function); did you mean 'SIGRTMAX'? ?? 63 | static int sigWakeup = (__SIGRTMAX - 2); musl libc does not contain __SIGRTMAX definition. It defines only SIGRTMAX as a function call [1]. __SIGRTMAX is used in JDK in two files NativeThread.c and linux_close.c . The first idea of the fix was simply to substitute __SIGRTMAX to SIGRTMAX in NativeThread.c and use the same definition ? #define INTERRUPT_SIGNAL (__SIGRTMAX - 2) in linux_close.c. "static int sigWakeup = (SIGRTMAX - 2)"? does not work because the SIGRTMAX is defined as a function and leads to "initializer element is not constant" error. The fix has been discussed on the portola-dev alias [2] where it was pointed out that the fix can be reviewed in the mainline and it was suggested to rename the INTERRUPT_SIGNAL and move its definition to net_util_md.h. [1] https://git.musl-libc.org/cgit/musl/tree/include/signal.h#n196 [2] https://mail.openjdk.java.net/pipermail/portola-dev/2020-August/000471.html Thanks, Alexander. From mandy.chung at oracle.com Tue Aug 25 17:23:56 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 25 Aug 2020 10:23:56 -0700 Subject: RFR: JDK-8242451 : ensure semantics of non-capturing lambdas are preserved independent of execution mode In-Reply-To: References: <71c26ec5-7512-e5e0-349f-a488357677f1@oracle.com> Message-ID: On 8/25/20 3:16 AM, Gilles Duboscq wrote: >> >> Since this patch caches a singleton instance in the generated class, >> it could apply to the eager initialization case as well and replace >> the current use of core reflection to new an instance except that the >> target of the returning callsite would always be the singleton object >> (the result of invoking the static getter method handle).? I wonder >> if there is any performance difference.? This is just a thought that >> we can file a JBS issue to follow up. >> >> Can you add a test case for this fix? > > I could write a test that generates different output depending on > whether a singleton or a fresh instance is returned. > Then i can compare the output when running with > `jdk.internal.lambda.disableEagerInitialization` set to `true` and > `false`. > What is the recommended way of comparing 2 runs like that in jtreg? > It can simply verify if the static field is only present when disableEagerInitialization is true; otherwise such static field should not exist. > I have updated the webrev: > https://cr.openjdk.java.net/~gdub/8242451/webrev.1/ > Looks okay to me. Nit: `Class::descriptorString` can be used instead of BytecodeDescriptor::unparse. 418 String lambdaTypeDescriptor = BytecodeDescriptor.unparse(invokedType.returnType()); > From your other mail: > >> Should this patch be a workaround to existing releases rather than >> the main line??? As Brian mentions, lambda proxy class may become >> inline class in valhalla repo (Roger has a patch already).?? The >> earlier fixing those programs the better. > > Indeed if we know this is landing in this cycle in the main repo > there's no point with my fix. Could you point me at the issue number > or mail thread where this patch is being discussed? > As Brian replied, integration of valhalla is not imminent. JDK-8205668 tracks the work to make lambda proxies as inline classes for the valhalla repo (*not* main line). Mandy From LANCE.ANDERSEN at ORACLE.COM Tue Aug 25 17:44:32 2020 From: LANCE.ANDERSEN at ORACLE.COM (Lance Andersen) Date: Tue, 25 Aug 2020 13:44:32 -0400 Subject: [16] RFR: 8251182: Fix "no comment" warnings in java.naming In-Reply-To: References: Message-ID: <6977DF8A-8ADB-4486-9B92-BCBA3A1F309E@ORACLE.COM> Hi Aleksei The changes look good > On Aug 25, 2020, at 11:03 AM, Aleks Efimov wrote: > > Hi, > > The documentation of classes from "java.naming" module needs to be improved to resolve javadoc -Xdoclint "no comment" warnings. > > JBS issue: https://bugs.openjdk.java.net/browse/JDK-8251182 > Webrev: http://cr.openjdk.java.net/~aefimov/8251182/00/index.html > CSR: https://bugs.openjdk.java.net/browse/JDK-8252310 > > With Best Regards, > Aleksei Best Lance ------------------ Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From huizhe.wang at oracle.com Tue Aug 25 18:47:40 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 25 Aug 2020 11:47:40 -0700 Subject: RFR [16/java.xml] 8251561: Fix doclint warnings in the java.xml package In-Reply-To: References: <3fc299f4-1c0a-1ccd-5d4c-3716775dfa38@oracle.com> Message-ID: Cc-ing build-dev at openjdk.java.net (makefile change: make/Docs.gmk) Updated webrev: http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_04/ Thanks Roger! Please see inline comments. On 8/25/20 8:09 AM, Roger Riggs wrote: > Hi Joe, > > Eliminating the checking for warnings in org.w3c is fine. Please be > more specific in the comment. > > "Ignore the doclint warnings in the W3C DOM package" Updated. > org/xml/...: If we're suppressing the warnings for org/xml/... then > the files changes are unnecessary? Did you mean org/w3c? We're only suppressing the warnings for org/w3c where the DOM package is in, but not org/xml where SAX is in. > > Remove the addition of -Xmaxwarns, it should stay the default. Done. > > Since its a makefile change, please copy build-dev at openjdk.java.net. Cc-ed > > The first line comments should terminate with a period ".". > > javax/xml/stream/FactoryConfigurationError.java:40 > javax/xml/stream/XMLStreamException.java:41 > javax/xml/xpath/XPathException.java:44? And capitable "Serializable". > > > javax/xml/stream/events/Attribute.java: 50; add "normalized" to the > @return line so it is the same as the first line. > For simple get methods, the @return mimics the first line. > > javax/xml/stream/events/NotationDeclaration.java:43? add "notation" Updated webrev including all of the above: http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_04/ > > > > Thanks, Roger > > p.s.? There is lots of other cleanup of the javadoc, using @code > around true, false, > missing periods at the end of first sentences, etc.? But that's a > different task. Created a bug to keep track of this: https://bugs.openjdk.java.net/browse/JDK-8252328 Thanks, Joe > > > > On 8/24/20 5:44 PM, Joe Wang wrote: >> Hi all,? adding Roger's comment for the make file to webrev_02 (the >> only change to webrev_01 is Docs.gmk): >> >> http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_02/ >> >> Thanks, >> Joe >> >> On 8/21/20 12:49 PM, naoto.sato at oracle.com wrote: >>> +1 >>> >>> Naoto >>> >>> On 8/21/20 12:24 PM, Lance Andersen wrote: >>>> Hi Joe, >>>> >>>> This looks OK. >>>> >>>> >>>> >>>>> On Aug 21, 2020, at 2:23 PM, Joe Wang wrote: >>>>> >>>>> Pelase review a patch to add the missing @return, @throws, @param >>>>> statements in the java.xml package (excluding the DOM component). >>>>> >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8251561 >>>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8251995 >>>>> >>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_01/ >>>>> >>>>> Thanks, >>>>> Joe >>>> >>>> >>>> Best >>>> Lance >>>> ------------------ >>>> >>>> >>>> >>>> >>>> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>>> Oracle Java Engineering >>>> 1 Network Drive >>>> Burlington, MA 01803 >>>> Lance.Andersen at oracle.com >>>> >>>> >>>> >>>> >> > From erik.joelsson at oracle.com Tue Aug 25 18:58:30 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Tue, 25 Aug 2020 11:58:30 -0700 Subject: RFR [16/java.xml] 8251561: Fix doclint warnings in the java.xml package In-Reply-To: References: <3fc299f4-1c0a-1ccd-5d4c-3716775dfa38@oracle.com> Message-ID: <4bb3ded8-6b16-d2ad-a426-7b25c5d61a54@oracle.com> Build change looks ok, but why is it needed? You are fixing a bunch of warnings in one part of the source and disabling them in another part. Is there some other change incoming that will enable more warning categories by default? /Erik On 2020-08-25 11:47, Joe Wang wrote: > Cc-ing build-dev at openjdk.java.net (makefile change: make/Docs.gmk) > Updated webrev: > http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_04/ > > Thanks Roger! Please see inline comments. > > On 8/25/20 8:09 AM, Roger Riggs wrote: >> Hi Joe, >> >> Eliminating the checking for warnings in org.w3c is fine. Please be >> more specific in the comment. >> >> "Ignore the doclint warnings in the W3C DOM package" > > Updated. >> org/xml/...: If we're suppressing the warnings for org/xml/... then >> the files changes are unnecessary? > > Did you mean org/w3c? We're only suppressing the warnings for org/w3c > where the DOM package is in, but not org/xml where SAX is in. > >> >> Remove the addition of -Xmaxwarns, it should stay the default. > > Done. >> >> Since its a makefile change, please copy build-dev at openjdk.java.net. > > Cc-ed >> >> The first line comments should terminate with a period ".". >> >> javax/xml/stream/FactoryConfigurationError.java:40 >> javax/xml/stream/XMLStreamException.java:41 >> javax/xml/xpath/XPathException.java:44? And capitable "Serializable". >> >> >> javax/xml/stream/events/Attribute.java: 50; add "normalized" to the >> @return line so it is the same as the first line. >> For simple get methods, the @return mimics the first line. >> >> javax/xml/stream/events/NotationDeclaration.java:43? add "notation" > > Updated webrev including all of the above: > > http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_04/ > >> >> >> >> Thanks, Roger >> >> p.s.? There is lots of other cleanup of the javadoc, using @code >> around true, false, >> missing periods at the end of first sentences, etc.? But that's a >> different task. > > Created a bug to keep track of this: > https://bugs.openjdk.java.net/browse/JDK-8252328 > > Thanks, > Joe >> >> >> >> On 8/24/20 5:44 PM, Joe Wang wrote: >>> Hi all,? adding Roger's comment for the make file to webrev_02 (the >>> only change to webrev_01 is Docs.gmk): >>> >>> http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_02/ >>> >>> Thanks, >>> Joe >>> >>> On 8/21/20 12:49 PM, naoto.sato at oracle.com wrote: >>>> +1 >>>> >>>> Naoto >>>> >>>> On 8/21/20 12:24 PM, Lance Andersen wrote: >>>>> Hi Joe, >>>>> >>>>> This looks OK. >>>>> >>>>> >>>>> >>>>>> On Aug 21, 2020, at 2:23 PM, Joe Wang >>>>>> wrote: >>>>>> >>>>>> Pelase review a patch to add the missing @return, @throws, @param >>>>>> statements in the java.xml package (excluding the DOM component). >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8251561 >>>>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8251995 >>>>>> >>>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_01/ >>>>>> >>>>>> Thanks, >>>>>> Joe >>>>> >>>>> >>>>> Best >>>>> Lance >>>>> ------------------ >>>>> >>>>> >>>>> >>>>> >>>>> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>>>> Oracle Java Engineering >>>>> 1 Network Drive >>>>> Burlington, MA 01803 >>>>> Lance.Andersen at oracle.com >>>>> >>>>> >>>>> >>>>> >>> >> > From yumin.qi at oracle.com Tue Aug 25 18:59:47 2020 From: yumin.qi at oracle.com (Yumin Qi) Date: Tue, 25 Aug 2020 11:59:47 -0700 Subject: RFR: 8247536: Support for pre-generated java.lang.invoke classes in CDS static archive In-Reply-To: References: <72abb7ef-0b67-728f-a8c8-c56a7580f053@oracle.com> <299d21d3-2da5-c5bf-1b55-05da73fc4403@oracle.com> <85ee9ec6-c8c3-378f-dadb-bdb79f3d72ac@oracle.com> <63b3b380-bd15-138c-0ab8-9837135dd517@oracle.com> <4a501d45-8f78-3627-ee2a-43a245469621@oracle.com> <76f119ed-6f93-52f2-6f01-64b57dda60c7@oracle.com> <68888b63-80a1-1b39-ca8c-b7a5bffc2585@oracle.com> <38ea270c-68f1-8837-7667-6481d2f52070@oracle.com> <2f3e315d-a790-d427-c8dd-cfff3ea7d852@oracle.com> Message-ID: <1ca66b6a-8a23-c4b1-c95d-10fc23b668f8@oracle.com> HI, Ioi ? Thanks for the re-review, updated webrev: http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-04/ ? Changed according to your suggestion, the only a little difference is move traceResolve(String line) to InvokerBytecodeGenerator as a static package public function so other package classes can call it. ? re-tested local on runtime/cds Thanks Yumin On 8/24/20 4:23 PM, Ioi Lam wrote: > Hi Yumin, > > This looks good overall. Here are my comments: > > ===================== > 6065???? size_t new_id = Atomic::add(&counter, (size_t)1); > 6066???? jio_snprintf(addr_buf, 20, INTPTR_FORMAT, new_id); > > I think this should be SIZE_FORMAT > > ===================== > ? 65 class KlassFactory : AllStatic { > ? 66 > ? 67?? // approved clients > ? 68?? friend class ClassLoader; > ? 69?? friend class ClassLoaderExt; > ? 70?? friend class SystemDictionary; > ? 71?? friend class LambdaFormInvokers; > ? 72 > ? 73? private: > ? 74?? static InstanceKlass* create_from_stream(ClassFileStream* stream, > > I think instead of adding everyone who uses create_from_stream as a friend class, we should just change create_from_stream into a public function and remove the friend declarations. > > ===================== > ?146?? // add to hierarchy and set state to loaded. > ?147?? { > ?148???? MutexLocker mu_r(THREAD, Compile_lock); // add_shared_to_hierarchy asserts this. > ?149???? SystemDictionaryShared::add_shared_to_hierarchy(result, THREAD); > ?150?? } > > I think the function name can be changed to SystemDictionaryShared::add_to_hierarchy as the "_shared" seems redundant. The "set state to loaded" comment seems wrong, as we have the assert on line 1155. I think the comment can be removed. > > 1153 void SystemDictionaryShared::add_shared_to_hierarchy(InstanceKlass* k, TRAPS) { > 1154?? Arguments::assert_is_dumping_archive(); > 1155?? assert(!k->is_loaded(), "Invariant"); > 1156?? assert_locked_or_safepoint(Compile_lock); // add_to_hierarchy assert on it. > 1157?? SystemDictionary::add_to_hierarchy(k, CHECK); > 1158 } > > Also, I think it's better to move the MutexLocker call into SystemDictionaryShared::add_shared_to_hierarchy. > > ======================== > > before: > > ?478???????????????? if (TRACE_RESOLVE && salvage != null) { > ?479???????????????????? // Used by jlink species pregeneration plugin, see > ?480???????????????????? // jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin > ?481???????????????????? System.out.println("[SPECIES_RESOLVE] " + className + " (salvaged)"); > ?482???????????????? } > > after: > > ?488???????????????? // Used by jlink species pregeneration plugin, see > ?489???????????????? // jdk.tools.jlink.internal.plugins.GenerateJLIClassesPlugin > ?490???????????????? traceResolve("[SPECIES_RESOLVE] " + className + " (salvaged)"); > > > When tracing is disabled, this will make extra allocations and cause a small slowdown. I think it's better to > > ??? if ((TRACE_RESOLVE|TRACE_RESOLVE_CDS) && salvage != null) { > ? ? ?? traceResolve("[SPECIES_RESOLVE] " + className + " (salvaged)"); > ??? } > > Because TRACE_RESOLVE is a static final boolean, the JIT compiler will completely optimize this block out. > > For the same reason, instead of calling VM.isDumpLoadedClassListSetAndOpen() every time, it's better to use a static final variable. > > > ======================= > > ?698???????? if (TRACE_RESOLVE) { > ?699???????????? System.out.println("[LF_RESOLVE] " + holder.getName() + " " + name + " " + > ?700 shortenSignature(basicTypeSignature(type)) + (resolvedMember != null ? " (success)" : " (fail)") ); > ?701???????? } > ?702???????? if (VM.isDumpLoadedClassListSetAndOpen()) { > ?703 GenerateJLIClassesHelper.cdsTraceResolve("[LF_RESOLVE] " + holder.getName() + " " + name + " " + > ?704 shortenSignature(basicTypeSignature(type)) + (resolvedMember != null ? " (success)" : " (fail)") ); > ?705???????? } > ?706???????? return resolvedMember; > > > I think the two "if" blocks should be combined similarly to ClassSpecializer::traceResolve(). > > ========================= > ? 34 Java_java_lang_invoke_GenerateJLIClassesHelper_cdsTraceResolve(JNIEnv *env, jclass ignore, jstring line) { > > Maybe this should be moved to the "VM" class as well? > > ========================= > lambdaFormInvokers.hpp: > > Need these declarations: > > ??? #include "memory/allocation.hpp"??? <-- for AllStatic > ??? #include "runtime/handles.hpp"????? <-- for typeArrayHandle and Handle > ??? #include "utilities/exceptions.hpp" <-- for TRAPS > > ??? template class GrowableArray; > ========================= > > > Thanks > - Ioi > > > > On 8/23/20 3:56 PM, Yumin Qi wrote: >> >> Hi, Mandy, Ioi and Calvin >> >> >> ? I have updated the new changed at: >> >> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-03/ >> >> ? In this version: >> >> ??? 1) Added a new API to check if flag DumpLoadedClassList set and the file is open. If true, call into vm to print out the trace line to the log file. >> >> ??????? Just thinking if we just call the cdsTraceResolve without checking if the flag DumpLoadedClassList set and file open, this way, the check logic is in the vm side like before, so save code by not adding the new API. >> >> ??? 2) The returned holder class names now are just 'package/className', removed head and tail. >> >> ??? 3)? Remove add_extra_classes from CollectClassesClosure since after bug 8250990: https://bugs.openjdk.java.net/browse/JDK-8250990 pushed, the CollectClassesClosure no longer exist. >> >> ??? 4)? Still keep the parsing for TRACE_RESOLVE in java/lang/invoke/GenerateJLIClassesHelper.java, so VM call its function to regenerate holder classes. >> >> >> ? Re-tested Mach5 tier1-4 >> >> Thanks >> >> Yumin >> >> >> On 8/20/20 8:05 PM, Yumin Qi wrote: >>> Hi, Mandy >>> >>> >>> On 8/20/20 5:10 PM, Mandy Chung wrote: >>>> >>>> >>>> On 8/19/20 10:14 PM, Yumin Qi wrote: >>>>> >>>>> HI, Mandy >>>>> >>>>> ? Thanks for the review, I took one day off yesterday so just got a detail look of your reply. >>>>> >>>>> On 8/19/20 1:30 PM, Mandy Chung wrote: >>>>>> On 8/17/20 12:37 PM, Yumin Qi wrote: >>>>>>> Hi, Ioi >>>>>>> >>>>>>> ? Thanks for review/suggestion. I have updated the webrev at the following link: >>>>>>> >>>>>>> http://cr.openjdk.java.net/~minqi/2020/8247536/webrev-02/ >>>>>>> >>>>>> >>>>>> This patch leverages the TRACE_RESOLVE output and passes the trace output to VM.? VM then calls GenerateJLIClassesHelper::generateMHHolderClasses to do the parsing and generate Holder class per the resolved LFs.?? I think there are other cleaner alternatives implementing this.?? jlink --generate-jli-classes plugin depends the trace output whereas -Xshare:dump does not.?? It's cleaner to skip generating the trace output and parsing for dumping shared archive purpose.? In addition, the implementation needs some cleanup (I can send you feedback on the next revision) >>>>>> >>>>> Current patch did not change the existing code for JLinkPlugin part. I just moved the parsing code from GenerateJLIClassesPlugin.java to GenerateJLIClassesHelper.java since the former is an internal class to which we shouldn't call to generate holder classes. >>>>>> Instead of relying on a system property "java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE", it's better to use jdk.internal.vm.isCDSDumpingEnabled() to detect if this is CDS dump time. >>>>> >>>>> I remember we have such API to query if flag -Xshare:dump or -Xshare:on used. Do you mean if DumpLoadedClassList flag set? This flag is the one used to log class name to list file. In GenerateLinkOptData.gmk: >>>>> >>>>> $(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) $(CLASSLIST_JAR) >>>>> ??????? $(call MakeDir, $(LINK_OPT_DIR)) >>>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $@)) >>>>> ??????? $(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $(JLI_TRACE_FILE))) >>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw \ >>>>> ??????????? -Duser.language=en -Duser.country=US \ >>>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>> ??????????? build.tools.classlist.HelloClasslist $(LOG_DEBUG) >>>>> ??????? $(GREP) -v HelloClasslist $@.raw > $@.interim >>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -Xshare:dump \ >>>>> ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ >>>>> ??????????? -Xmx128M -Xms128M $(LOG_INFO) >>>>> ??????? $(FIXPATH) $(INTERIM_IMAGE_DIR)/bin/java -XX:DumpLoadedClassList=$@.raw.2 \ >>>>> ??????????? -XX:SharedClassListFile=$@.interim -XX:SharedArchiveFile=$@.jsa \ >>>>> -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true \ >>>>> ??????????? -Duser.language=en -Duser.country=US \ >>>>> ??????????? --module-path $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>> ??????????? -cp $(SUPPORT_OUTPUTDIR)/classlist.jar \ >>>>> ??????????? build.tools.classlist.HelloClasslist \ >>>>> ??????????? 2> $(LINK_OPT_DIR)/stderr > $(JLI_TRACE_FILE) \ >>>>> ??????????? || ( \ >>>>> ??????????????? exitcode=$$? ; \ >>>>> ??????????????? $(ECHO) "ERROR: Failed to generate link optimization data." \ >>>>> ??????????????????? "This is likely a problem with the newly built JVM/JDK." ; \ >>>>> ??????????????? $(CAT) $(LINK_OPT_DIR)/stderr $(JLI_TRACE_FILE) ; \ >>>>> ??????????????? exit $$exitcode \ >>>>> ??????????? ) >>>>> ??????? $(GREP) -v HelloClasslist $@.raw.2 > $@ >>>>> >>>>> The $(JLI_TRACE_FILE) is generated with both -XX:DumpLoadedClassList and -Djava.lang.invoke.MethodHandle.TRACE_RESOLVE=true, in current implementation, DumpLoadedClassList will turn on property java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE=true. So the same output sent to stdout and log file DumpLoadedClassList specified. >>>>> >>>> >>>> These entries are duplicated in two different files: one for jlink --generate-jli-classes plugin and another for CDS use.? CDS -Xshare:dump attempts to do what jlink plugin does but writes those generated classes in to shared archive. >>>> >>>> Like the above make logic to build JDK image, the same entries are written in both default-jli-trace.txt via System.out and to classlist via JNI call to the VM.? I guess VM also implements the logic to do some kind of diffing and write to CDS archive. >>>> >>> In current implementation, vm side only records the line as from TRACE_RESOLVE at pre-run with -XX:DumpLoadedClassList, and at dump time, call back to java for parsing those recordings and generating the holder classes, this uses the existing JLI code. >>>>> >>>>> Now instead of this property, using a vm interface API to query if this flag is set, I think it is better choice. But here I am NOT sure I understand your suggestion, I think there are two choices: >>>>> >>>>> 1) Using DumpLoadedClassList to collect TRACE_RESOLVE but not via CDS_TRACE_RESOLVE, using new API to query if DumpLoadedClassList is set >>>>> >>>>> 2) Do not use DumpLoadedClassList, when -Xshare:dump collecting those name, type and holder name to regenerate holder classes? >>>>> >>>> >>>> I misunderstood that this CDS_TRACE_RESOLVE flag is set during -Xshare:dump time. >>>> >>>> Ioi has clarified to me offline that this step is actually part of -XX:DumpLoadedClassList and includes these TRACE_RESOLVE logs to the given class list file, i.e. you repurpose the class list file to include the log output that was initially designed for jlink plugin. >>>> >>>> To me, I'd prefer to see this support depending on `jlink --generate-jli-classes` which is an existing functionality and much cleaner.? This way this does not require any VM change.?? It will generate the holder classes in the custom image per the application-specific config file. >>>> >>>> What it means is that: a customer would need to create a custom image with their application-specific config file. It might need a new CDS? option to specify a separate TRACE_RESOLVE file.?? It would turn on this feature by default by defining a default path of the log file if it helps. >>>> >>> So for now, I would implement an API to query if flag DumpLoadedClassList set in cmd line, remove new added property of java.lang.invoke.MethodHandle.CDS_TRACE_RESOLVE. We can address custom image with CDS in future in a separate issue. >>> >>> >>> Thanks >>> >>> Yumin >>> >>>> I understand that this is not the existing CDS work flow and CDS archive does not require to run on a custom image.?? I see the value of this approach which can prepare customers to start building and using its own custom image. >>>> >>>> Of course the implementation would be much simpler (adding a flag to write these traces to a given file path, that is). >>>> >>>> Mandy > From Roger.Riggs at oracle.com Tue Aug 25 19:17:34 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Tue, 25 Aug 2020 15:17:34 -0400 Subject: [16] RFR: 8251182: Fix "no comment" warnings in java.naming In-Reply-To: References: Message-ID: Hi Aleksei, CompositeName.java readObject and writeObject methods should have a generic first line comment. CompositeName.java BasicAttribute.java BasicAttributes.java For example, * The WriteObject method is called to save the state of the FilePermission to a stream. * The readObject method is called to restore the state of the FilePermission from a stream. There are a lot of variations in the JDK that are find, as long as they indicate reading or writing the state from/to a stream. It is conventional that javadoc tags for @param, @throws, etc. do not end with a period "." unless it forms a complete sentence. Thanks, Roger On 8/25/20 11:03 AM, Aleks Efimov wrote: > Hi, > > The documentation of classes from "java.naming" module needs to be > improved to resolve javadoc -Xdoclint "no comment" warnings. > > JBS issue: https://bugs.openjdk.java.net/browse/JDK-8251182 > Webrev: http://cr.openjdk.java.net/~aefimov/8251182/00/index.html > CSR: https://bugs.openjdk.java.net/browse/JDK-8252310 > > With Best Regards, > Aleksei From Roger.Riggs at oracle.com Tue Aug 25 19:21:33 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Tue, 25 Aug 2020 15:21:33 -0400 Subject: RFR [16/java.xml] 8251561: Fix doclint warnings in the java.xml package In-Reply-To: <4bb3ded8-6b16-d2ad-a426-7b25c5d61a54@oracle.com> References: <3fc299f4-1c0a-1ccd-5d4c-3716775dfa38@oracle.com> <4bb3ded8-6b16-d2ad-a426-7b25c5d61a54@oracle.com> Message-ID: Hi Erik, org.w3c is in third party code that is not being updated. There is a balance between ignoring the warnings and doing a bunch of editing that would overwritten by a future update. $.02, Roger On 8/25/20 2:58 PM, Erik Joelsson wrote: > Build change looks ok, but why is it needed? You are fixing a bunch of > warnings in one part of the source and disabling them in another part. > Is there some other change incoming that will enable more warning > categories by default? > > /Erik > > On 2020-08-25 11:47, Joe Wang wrote: >> Cc-ing build-dev at openjdk.java.net (makefile change: make/Docs.gmk) >> Updated webrev: >> http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_04/ >> >> Thanks Roger! Please see inline comments. >> >> On 8/25/20 8:09 AM, Roger Riggs wrote: >>> Hi Joe, >>> >>> Eliminating the checking for warnings in org.w3c is fine. Please be >>> more specific in the comment. >>> >>> "Ignore the doclint warnings in the W3C DOM package" >> >> Updated. >>> org/xml/...: If we're suppressing the warnings for org/xml/... then >>> the files changes are unnecessary? >> >> Did you mean org/w3c? We're only suppressing the warnings for org/w3c >> where the DOM package is in, but not org/xml where SAX is in. >> >>> >>> Remove the addition of -Xmaxwarns, it should stay the default. >> >> Done. >>> >>> Since its a makefile change, please copy build-dev at openjdk.java.net. >> >> Cc-ed >>> >>> The first line comments should terminate with a period ".". >>> >>> javax/xml/stream/FactoryConfigurationError.java:40 >>> javax/xml/stream/XMLStreamException.java:41 >>> javax/xml/xpath/XPathException.java:44? And capitable "Serializable". >>> >>> >>> javax/xml/stream/events/Attribute.java: 50; add "normalized" to the >>> @return line so it is the same as the first line. >>> For simple get methods, the @return mimics the first line. >>> >>> javax/xml/stream/events/NotationDeclaration.java:43? add "notation" >> >> Updated webrev including all of the above: >> >> http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_04/ >> >>> >>> >>> >>> Thanks, Roger >>> >>> p.s.? There is lots of other cleanup of the javadoc, using @code >>> around true, false, >>> missing periods at the end of first sentences, etc.? But that's a >>> different task. >> >> Created a bug to keep track of this: >> https://bugs.openjdk.java.net/browse/JDK-8252328 >> >> Thanks, >> Joe >>> >>> >>> >>> On 8/24/20 5:44 PM, Joe Wang wrote: >>>> Hi all,? adding Roger's comment for the make file to webrev_02 (the >>>> only change to webrev_01 is Docs.gmk): >>>> >>>> http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_02/ >>>> >>>> Thanks, >>>> Joe >>>> >>>> On 8/21/20 12:49 PM, naoto.sato at oracle.com wrote: >>>>> +1 >>>>> >>>>> Naoto >>>>> >>>>> On 8/21/20 12:24 PM, Lance Andersen wrote: >>>>>> Hi Joe, >>>>>> >>>>>> This looks OK. >>>>>> >>>>>> >>>>>> >>>>>>> On Aug 21, 2020, at 2:23 PM, Joe Wang >>>>>>> wrote: >>>>>>> >>>>>>> Pelase review a patch to add the missing @return, @throws, >>>>>>> @param statements in the java.xml package (excluding the DOM >>>>>>> component). >>>>>>> >>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8251561 >>>>>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8251995 >>>>>>> >>>>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_01/ >>>>>>> >>>>>>> Thanks, >>>>>>> Joe >>>>>> >>>>>> >>>>>> Best >>>>>> Lance >>>>>> ------------------ >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Lance Andersen| Principal Member of Technical Staff | >>>>>> +1.781.442.2037 >>>>>> Oracle Java Engineering >>>>>> 1 Network Drive >>>>>> Burlington, MA 01803 >>>>>> Lance.Andersen at oracle.com >>>>>> >>>>>> >>>>>> >>>>>> >>>> >>> >> From daniel.fuchs at oracle.com Tue Aug 25 19:23:39 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 25 Aug 2020 20:23:39 +0100 Subject: [16] RFR: 8251182: Fix "no comment" warnings in java.naming In-Reply-To: References: Message-ID: Hi Aleksei, LGTM. You could also add the @Seial annotation to serialVersionUID declarations. best regards, -- daniel On 25/08/2020 16:03, Aleks Efimov wrote: > Hi, > > The documentation of classes from "java.naming" module needs to be > improved to resolve javadoc -Xdoclint "no comment" warnings. > > JBS issue: https://bugs.openjdk.java.net/browse/JDK-8251182 > Webrev: http://cr.openjdk.java.net/~aefimov/8251182/00/index.html > CSR: https://bugs.openjdk.java.net/browse/JDK-8252310 > > With Best Regards, > Aleksei From Roger.Riggs at oracle.com Tue Aug 25 19:30:33 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Tue, 25 Aug 2020 15:30:33 -0400 Subject: RFR [16/java.xml] 8251561: Fix doclint warnings in the java.xml package In-Reply-To: References: <3fc299f4-1c0a-1ccd-5d4c-3716775dfa38@oracle.com> Message-ID: Hi Joe, The updates to those files are fine, I mixed up the two packages. Roger On 8/25/20 2:47 PM, Joe Wang wrote: >> org/xml/...: If we're suppressing the warnings for org/xml/... then >> the files changes are unnecessary? > > Did you mean org/w3c? We're only suppressing the warnings for org/w3c > where the DOM package is in, but not org/xml where SAX is in. From erik.joelsson at oracle.com Tue Aug 25 19:39:09 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Tue, 25 Aug 2020 12:39:09 -0700 Subject: RFR [16/java.xml] 8251561: Fix doclint warnings in the java.xml package In-Reply-To: References: <3fc299f4-1c0a-1ccd-5d4c-3716775dfa38@oracle.com> <4bb3ded8-6b16-d2ad-a426-7b25c5d61a54@oracle.com> Message-ID: On 2020-08-25 12:21, Roger Riggs wrote: > Hi Erik, > > org.w3c is in third party code that is not being updated. There is a > balance between > ignoring the warnings and doing a bunch of editing that would > overwritten by a future update. > Yes, I get that part, but why are those warnings not triggered today? I don't see any change enabling more warnings from the build. /Erik > $.02, Roger > > > On 8/25/20 2:58 PM, Erik Joelsson wrote: >> Build change looks ok, but why is it needed? You are fixing a bunch >> of warnings in one part of the source and disabling them in another >> part. Is there some other change incoming that will enable more >> warning categories by default? >> >> /Erik >> >> On 2020-08-25 11:47, Joe Wang wrote: >>> Cc-ing build-dev at openjdk.java.net (makefile change: make/Docs.gmk) >>> Updated webrev: >>> http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_04/ >>> >>> Thanks Roger! Please see inline comments. >>> >>> On 8/25/20 8:09 AM, Roger Riggs wrote: >>>> Hi Joe, >>>> >>>> Eliminating the checking for warnings in org.w3c is fine. Please be >>>> more specific in the comment. >>>> >>>> "Ignore the doclint warnings in the W3C DOM package" >>> >>> Updated. >>>> org/xml/...: If we're suppressing the warnings for org/xml/... then >>>> the files changes are unnecessary? >>> >>> Did you mean org/w3c? We're only suppressing the warnings for >>> org/w3c where the DOM package is in, but not org/xml where SAX is in. >>> >>>> >>>> Remove the addition of -Xmaxwarns, it should stay the default. >>> >>> Done. >>>> >>>> Since its a makefile change, please copy build-dev at openjdk.java.net. >>> >>> Cc-ed >>>> >>>> The first line comments should terminate with a period ".". >>>> >>>> javax/xml/stream/FactoryConfigurationError.java:40 >>>> javax/xml/stream/XMLStreamException.java:41 >>>> javax/xml/xpath/XPathException.java:44? And capitable "Serializable". >>>> >>>> >>>> javax/xml/stream/events/Attribute.java: 50; add "normalized" to the >>>> @return line so it is the same as the first line. >>>> For simple get methods, the @return mimics the first line. >>>> >>>> javax/xml/stream/events/NotationDeclaration.java:43? add "notation" >>> >>> Updated webrev including all of the above: >>> >>> http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_04/ >>> >>>> >>>> >>>> >>>> Thanks, Roger >>>> >>>> p.s.? There is lots of other cleanup of the javadoc, using @code >>>> around true, false, >>>> missing periods at the end of first sentences, etc.? But that's a >>>> different task. >>> >>> Created a bug to keep track of this: >>> https://bugs.openjdk.java.net/browse/JDK-8252328 >>> >>> Thanks, >>> Joe >>>> >>>> >>>> >>>> On 8/24/20 5:44 PM, Joe Wang wrote: >>>>> Hi all,? adding Roger's comment for the make file to webrev_02 >>>>> (the only change to webrev_01 is Docs.gmk): >>>>> >>>>> http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_02/ >>>>> >>>>> Thanks, >>>>> Joe >>>>> >>>>> On 8/21/20 12:49 PM, naoto.sato at oracle.com wrote: >>>>>> +1 >>>>>> >>>>>> Naoto >>>>>> >>>>>> On 8/21/20 12:24 PM, Lance Andersen wrote: >>>>>>> Hi Joe, >>>>>>> >>>>>>> This looks OK. >>>>>>> >>>>>>> >>>>>>> >>>>>>>> On Aug 21, 2020, at 2:23 PM, Joe Wang >>>>>>>> wrote: >>>>>>>> >>>>>>>> Pelase review a patch to add the missing @return, @throws, >>>>>>>> @param statements in the java.xml package (excluding the DOM >>>>>>>> component). >>>>>>>> >>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8251561 >>>>>>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8251995 >>>>>>>> >>>>>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_01/ >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Joe >>>>>>> >>>>>>> >>>>>>> Best >>>>>>> Lance >>>>>>> ------------------ >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Lance Andersen| Principal Member of Technical Staff | >>>>>>> +1.781.442.2037 >>>>>>> Oracle Java Engineering >>>>>>> 1 Network Drive >>>>>>> Burlington, MA 01803 >>>>>>> Lance.Andersen at oracle.com >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>> >>>> >>> > From naoto.sato at oracle.com Tue Aug 25 20:04:17 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Tue, 25 Aug 2020 13:04:17 -0700 Subject: RFR [16/java.xml] 8251561: Fix doclint warnings in the java.xml package In-Reply-To: References: <3fc299f4-1c0a-1ccd-5d4c-3716775dfa38@oracle.com> Message-ID: <636240c9-354b-a1ec-a726-956894a56f58@oracle.com> +1 Naoto On 8/25/20 11:47 AM, Joe Wang wrote: > Cc-ing build-dev at openjdk.java.net (makefile change: make/Docs.gmk) > Updated webrev: http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_04/ > > Thanks Roger! Please see inline comments. > > On 8/25/20 8:09 AM, Roger Riggs wrote: >> Hi Joe, >> >> Eliminating the checking for warnings in org.w3c is fine. Please be >> more specific in the comment. >> >> "Ignore the doclint warnings in the W3C DOM package" > > Updated. >> org/xml/...: If we're suppressing the warnings for org/xml/... then >> the files changes are unnecessary? > > Did you mean org/w3c? We're only suppressing the warnings for org/w3c > where the DOM package is in, but not org/xml where SAX is in. > >> >> Remove the addition of -Xmaxwarns, it should stay the default. > > Done. >> >> Since its a makefile change, please copy build-dev at openjdk.java.net. > > Cc-ed >> >> The first line comments should terminate with a period ".". >> >> javax/xml/stream/FactoryConfigurationError.java:40 >> javax/xml/stream/XMLStreamException.java:41 >> javax/xml/xpath/XPathException.java:44? And capitable "Serializable". >> >> >> javax/xml/stream/events/Attribute.java: 50; add "normalized" to the >> @return line so it is the same as the first line. >> For simple get methods, the @return mimics the first line. >> >> javax/xml/stream/events/NotationDeclaration.java:43? add "notation" > > Updated webrev including all of the above: > > http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_04/ > >> >> >> >> Thanks, Roger >> >> p.s.? There is lots of other cleanup of the javadoc, using @code >> around true, false, >> missing periods at the end of first sentences, etc.? But that's a >> different task. > > Created a bug to keep track of this: > https://bugs.openjdk.java.net/browse/JDK-8252328 > > Thanks, > Joe >> >> >> >> On 8/24/20 5:44 PM, Joe Wang wrote: >>> Hi all,? adding Roger's comment for the make file to webrev_02 (the >>> only change to webrev_01 is Docs.gmk): >>> >>> http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_02/ >>> >>> Thanks, >>> Joe >>> >>> On 8/21/20 12:49 PM, naoto.sato at oracle.com wrote: >>>> +1 >>>> >>>> Naoto >>>> >>>> On 8/21/20 12:24 PM, Lance Andersen wrote: >>>>> Hi Joe, >>>>> >>>>> This looks OK. >>>>> >>>>> >>>>> >>>>>> On Aug 21, 2020, at 2:23 PM, Joe Wang wrote: >>>>>> >>>>>> Pelase review a patch to add the missing @return, @throws, @param >>>>>> statements in the java.xml package (excluding the DOM component). >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8251561 >>>>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8251995 >>>>>> >>>>>> webrev: http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_01/ >>>>>> >>>>>> Thanks, >>>>>> Joe >>>>> >>>>> >>>>> Best >>>>> Lance >>>>> ------------------ >>>>> >>>>> >>>>> >>>>> >>>>> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>>>> Oracle Java Engineering >>>>> 1 Network Drive >>>>> Burlington, MA 01803 >>>>> Lance.Andersen at oracle.com >>>>> >>>>> >>>>> >>>>> >>> >> > From Roger.Riggs at oracle.com Tue Aug 25 20:09:16 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Tue, 25 Aug 2020 16:09:16 -0400 Subject: RFR [16/java.xml] 8251561: Fix doclint warnings in the java.xml package In-Reply-To: References: <3fc299f4-1c0a-1ccd-5d4c-3716775dfa38@oracle.com> <4bb3ded8-6b16-d2ad-a426-7b25c5d61a54@oracle.com> Message-ID: <4b07ca59-e778-3279-627b-d640d5fe86c4@oracle.com> Hi Erik, We're in a cleanup phase of the source before re-enabling disabled warnings. The warnings for "missing" comments are disabled: Docs.gmk: 106: JAVADOC_DISABLED_DOCLINT := accessibility html missing syntax reference Roger On 8/25/20 3:39 PM, Erik Joelsson wrote: > > On 2020-08-25 12:21, Roger Riggs wrote: >> Hi Erik, >> >> org.w3c is in third party code that is not being updated. There is a >> balance between >> ignoring the warnings and doing a bunch of editing that would >> overwritten by a future update. >> > Yes, I get that part, but why are those warnings not triggered today? > I don't see any change enabling more warnings from the build. > > /Erik > >> $.02, Roger >> >> >> On 8/25/20 2:58 PM, Erik Joelsson wrote: >>> Build change looks ok, but why is it needed? You are fixing a bunch >>> of warnings in one part of the source and disabling them in another >>> part. Is there some other change incoming that will enable more >>> warning categories by default? >>> >>> /Erik >>> >>> On 2020-08-25 11:47, Joe Wang wrote: >>>> Cc-ing build-dev at openjdk.java.net (makefile change: make/Docs.gmk) >>>> Updated webrev: >>>> http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_04/ >>>> >>>> Thanks Roger! Please see inline comments. >>>> >>>> On 8/25/20 8:09 AM, Roger Riggs wrote: >>>>> Hi Joe, >>>>> >>>>> Eliminating the checking for warnings in org.w3c is fine. Please >>>>> be more specific in the comment. >>>>> >>>>> "Ignore the doclint warnings in the W3C DOM package" >>>> >>>> Updated. >>>>> org/xml/...: If we're suppressing the warnings for org/xml/... >>>>> then the files changes are unnecessary? >>>> >>>> Did you mean org/w3c? We're only suppressing the warnings for >>>> org/w3c where the DOM package is in, but not org/xml where SAX is in. >>>> >>>>> >>>>> Remove the addition of -Xmaxwarns, it should stay the default. >>>> >>>> Done. >>>>> >>>>> Since its a makefile change, please copy build-dev at openjdk.java.net. >>>> >>>> Cc-ed >>>>> >>>>> The first line comments should terminate with a period ".". >>>>> >>>>> javax/xml/stream/FactoryConfigurationError.java:40 >>>>> javax/xml/stream/XMLStreamException.java:41 >>>>> javax/xml/xpath/XPathException.java:44? And capitable "Serializable". >>>>> >>>>> >>>>> javax/xml/stream/events/Attribute.java: 50; add "normalized" to >>>>> the @return line so it is the same as the first line. >>>>> For simple get methods, the @return mimics the first line. >>>>> >>>>> javax/xml/stream/events/NotationDeclaration.java:43? add "notation" >>>> >>>> Updated webrev including all of the above: >>>> >>>> http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_04/ >>>> >>>>> >>>>> >>>>> >>>>> Thanks, Roger >>>>> >>>>> p.s.? There is lots of other cleanup of the javadoc, using @code >>>>> around true, false, >>>>> missing periods at the end of first sentences, etc.? But that's a >>>>> different task. >>>> >>>> Created a bug to keep track of this: >>>> https://bugs.openjdk.java.net/browse/JDK-8252328 >>>> >>>> Thanks, >>>> Joe >>>>> >>>>> >>>>> >>>>> On 8/24/20 5:44 PM, Joe Wang wrote: >>>>>> Hi all,? adding Roger's comment for the make file to webrev_02 >>>>>> (the only change to webrev_01 is Docs.gmk): >>>>>> >>>>>> http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_02/ >>>>>> >>>>>> Thanks, >>>>>> Joe >>>>>> >>>>>> On 8/21/20 12:49 PM, naoto.sato at oracle.com wrote: >>>>>>> +1 >>>>>>> >>>>>>> Naoto >>>>>>> >>>>>>> On 8/21/20 12:24 PM, Lance Andersen wrote: >>>>>>>> Hi Joe, >>>>>>>> >>>>>>>> This looks OK. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On Aug 21, 2020, at 2:23 PM, Joe Wang >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Pelase review a patch to add the missing @return, @throws, >>>>>>>>> @param statements in the java.xml package (excluding the DOM >>>>>>>>> component). >>>>>>>>> >>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8251561 >>>>>>>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8251995 >>>>>>>>> >>>>>>>>> webrev: >>>>>>>>> http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_01/ >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Joe >>>>>>>> >>>>>>>> >>>>>>>> Best >>>>>>>> Lance >>>>>>>> ------------------ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Lance Andersen| Principal Member of Technical Staff | >>>>>>>> +1.781.442.2037 >>>>>>>> Oracle Java Engineering >>>>>>>> 1 Network Drive >>>>>>>> Burlington, MA 01803 >>>>>>>> Lance.Andersen at oracle.com >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>> >>>> >> From huizhe.wang at oracle.com Tue Aug 25 20:20:02 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 25 Aug 2020 13:20:02 -0700 Subject: RFR [16/java.xml] 8251561: Fix doclint warnings in the java.xml package In-Reply-To: References: <3fc299f4-1c0a-1ccd-5d4c-3716775dfa38@oracle.com> <4bb3ded8-6b16-d2ad-a426-7b25c5d61a54@oracle.com> Message-ID: Hi Erik, We're acting on a report Jon generated separately. The warnings checks are currently disabled (see line 106 JAVADOC_DISABLED_DOCLINT := accessibility html missing syntax reference). -Joe On 8/25/20 12:39 PM, Erik Joelsson wrote: > > On 2020-08-25 12:21, Roger Riggs wrote: >> Hi Erik, >> >> org.w3c is in third party code that is not being updated. There is a >> balance between >> ignoring the warnings and doing a bunch of editing that would >> overwritten by a future update. >> > Yes, I get that part, but why are those warnings not triggered today? > I don't see any change enabling more warnings from the build. > > /Erik > >> $.02, Roger >> >> >> On 8/25/20 2:58 PM, Erik Joelsson wrote: >>> Build change looks ok, but why is it needed? You are fixing a bunch >>> of warnings in one part of the source and disabling them in another >>> part. Is there some other change incoming that will enable more >>> warning categories by default? >>> >>> /Erik >>> >>> On 2020-08-25 11:47, Joe Wang wrote: >>>> Cc-ing build-dev at openjdk.java.net (makefile change: make/Docs.gmk) >>>> Updated webrev: >>>> http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_04/ >>>> >>>> Thanks Roger! Please see inline comments. >>>> >>>> On 8/25/20 8:09 AM, Roger Riggs wrote: >>>>> Hi Joe, >>>>> >>>>> Eliminating the checking for warnings in org.w3c is fine. Please >>>>> be more specific in the comment. >>>>> >>>>> "Ignore the doclint warnings in the W3C DOM package" >>>> >>>> Updated. >>>>> org/xml/...: If we're suppressing the warnings for org/xml/... >>>>> then the files changes are unnecessary? >>>> >>>> Did you mean org/w3c? We're only suppressing the warnings for >>>> org/w3c where the DOM package is in, but not org/xml where SAX is in. >>>> >>>>> >>>>> Remove the addition of -Xmaxwarns, it should stay the default. >>>> >>>> Done. >>>>> >>>>> Since its a makefile change, please copy build-dev at openjdk.java.net. >>>> >>>> Cc-ed >>>>> >>>>> The first line comments should terminate with a period ".". >>>>> >>>>> javax/xml/stream/FactoryConfigurationError.java:40 >>>>> javax/xml/stream/XMLStreamException.java:41 >>>>> javax/xml/xpath/XPathException.java:44? And capitable "Serializable". >>>>> >>>>> >>>>> javax/xml/stream/events/Attribute.java: 50; add "normalized" to >>>>> the @return line so it is the same as the first line. >>>>> For simple get methods, the @return mimics the first line. >>>>> >>>>> javax/xml/stream/events/NotationDeclaration.java:43? add "notation" >>>> >>>> Updated webrev including all of the above: >>>> >>>> http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_04/ >>>> >>>>> >>>>> >>>>> >>>>> Thanks, Roger >>>>> >>>>> p.s.? There is lots of other cleanup of the javadoc, using @code >>>>> around true, false, >>>>> missing periods at the end of first sentences, etc.? But that's a >>>>> different task. >>>> >>>> Created a bug to keep track of this: >>>> https://bugs.openjdk.java.net/browse/JDK-8252328 >>>> >>>> Thanks, >>>> Joe >>>>> >>>>> >>>>> >>>>> On 8/24/20 5:44 PM, Joe Wang wrote: >>>>>> Hi all,? adding Roger's comment for the make file to webrev_02 >>>>>> (the only change to webrev_01 is Docs.gmk): >>>>>> >>>>>> http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_02/ >>>>>> >>>>>> Thanks, >>>>>> Joe >>>>>> >>>>>> On 8/21/20 12:49 PM, naoto.sato at oracle.com wrote: >>>>>>> +1 >>>>>>> >>>>>>> Naoto >>>>>>> >>>>>>> On 8/21/20 12:24 PM, Lance Andersen wrote: >>>>>>>> Hi Joe, >>>>>>>> >>>>>>>> This looks OK. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On Aug 21, 2020, at 2:23 PM, Joe Wang >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Pelase review a patch to add the missing @return, @throws, >>>>>>>>> @param statements in the java.xml package (excluding the DOM >>>>>>>>> component). >>>>>>>>> >>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8251561 >>>>>>>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8251995 >>>>>>>>> >>>>>>>>> webrev: >>>>>>>>> http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_01/ >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Joe >>>>>>>> >>>>>>>> >>>>>>>> Best >>>>>>>> Lance >>>>>>>> ------------------ >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Lance Andersen| Principal Member of Technical Staff | >>>>>>>> +1.781.442.2037 >>>>>>>> Oracle Java Engineering >>>>>>>> 1 Network Drive >>>>>>>> Burlington, MA 01803 >>>>>>>> Lance.Andersen at oracle.com >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>> >>>> >> From erik.joelsson at oracle.com Tue Aug 25 21:49:40 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Tue, 25 Aug 2020 14:49:40 -0700 Subject: RFR [16/java.xml] 8251561: Fix doclint warnings in the java.xml package In-Reply-To: References: <3fc299f4-1c0a-1ccd-5d4c-3716775dfa38@oracle.com> <4bb3ded8-6b16-d2ad-a426-7b25c5d61a54@oracle.com> Message-ID: <9c0ca5fa-8edb-5d2e-7544-563fff860aa6@oracle.com> Ah, I was looking for that kind of information in the bug report, but couldn't find any. /Erik On 2020-08-25 13:20, Joe Wang wrote: > Hi Erik, > > We're acting on a report Jon generated separately. The warnings checks > are currently disabled (see line 106 JAVADOC_DISABLED_DOCLINT := > accessibility html missing syntax reference). > > -Joe > > On 8/25/20 12:39 PM, Erik Joelsson wrote: >> >> On 2020-08-25 12:21, Roger Riggs wrote: >>> Hi Erik, >>> >>> org.w3c is in third party code that is not being updated. There is a >>> balance between >>> ignoring the warnings and doing a bunch of editing that would >>> overwritten by a future update. >>> >> Yes, I get that part, but why are those warnings not triggered today? >> I don't see any change enabling more warnings from the build. >> >> /Erik >> >>> $.02, Roger >>> >>> >>> On 8/25/20 2:58 PM, Erik Joelsson wrote: >>>> Build change looks ok, but why is it needed? You are fixing a bunch >>>> of warnings in one part of the source and disabling them in another >>>> part. Is there some other change incoming that will enable more >>>> warning categories by default? >>>> >>>> /Erik >>>> >>>> On 2020-08-25 11:47, Joe Wang wrote: >>>>> Cc-ing build-dev at openjdk.java.net (makefile change: make/Docs.gmk) >>>>> Updated webrev: >>>>> http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_04/ >>>>> >>>>> Thanks Roger! Please see inline comments. >>>>> >>>>> On 8/25/20 8:09 AM, Roger Riggs wrote: >>>>>> Hi Joe, >>>>>> >>>>>> Eliminating the checking for warnings in org.w3c is fine. Please >>>>>> be more specific in the comment. >>>>>> >>>>>> "Ignore the doclint warnings in the W3C DOM package" >>>>> >>>>> Updated. >>>>>> org/xml/...: If we're suppressing the warnings for org/xml/... >>>>>> then the files changes are unnecessary? >>>>> >>>>> Did you mean org/w3c? We're only suppressing the warnings for >>>>> org/w3c where the DOM package is in, but not org/xml where SAX is in. >>>>> >>>>>> >>>>>> Remove the addition of -Xmaxwarns, it should stay the default. >>>>> >>>>> Done. >>>>>> >>>>>> Since its a makefile change, please copy build-dev at openjdk.java.net. >>>>> >>>>> Cc-ed >>>>>> >>>>>> The first line comments should terminate with a period ".". >>>>>> >>>>>> javax/xml/stream/FactoryConfigurationError.java:40 >>>>>> javax/xml/stream/XMLStreamException.java:41 >>>>>> javax/xml/xpath/XPathException.java:44? And capitable >>>>>> "Serializable". >>>>>> >>>>>> >>>>>> javax/xml/stream/events/Attribute.java: 50; add "normalized" to >>>>>> the @return line so it is the same as the first line. >>>>>> For simple get methods, the @return mimics the first line. >>>>>> >>>>>> javax/xml/stream/events/NotationDeclaration.java:43? add "notation" >>>>> >>>>> Updated webrev including all of the above: >>>>> >>>>> http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_04/ >>>>> >>>>>> >>>>>> >>>>>> >>>>>> Thanks, Roger >>>>>> >>>>>> p.s.? There is lots of other cleanup of the javadoc, using @code >>>>>> around true, false, >>>>>> missing periods at the end of first sentences, etc.? But that's a >>>>>> different task. >>>>> >>>>> Created a bug to keep track of this: >>>>> https://bugs.openjdk.java.net/browse/JDK-8252328 >>>>> >>>>> Thanks, >>>>> Joe >>>>>> >>>>>> >>>>>> >>>>>> On 8/24/20 5:44 PM, Joe Wang wrote: >>>>>>> Hi all,? adding Roger's comment for the make file to webrev_02 >>>>>>> (the only change to webrev_01 is Docs.gmk): >>>>>>> >>>>>>> http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_02/ >>>>>>> >>>>>>> Thanks, >>>>>>> Joe >>>>>>> >>>>>>> On 8/21/20 12:49 PM, naoto.sato at oracle.com wrote: >>>>>>>> +1 >>>>>>>> >>>>>>>> Naoto >>>>>>>> >>>>>>>> On 8/21/20 12:24 PM, Lance Andersen wrote: >>>>>>>>> Hi Joe, >>>>>>>>> >>>>>>>>> This looks OK. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> On Aug 21, 2020, at 2:23 PM, Joe Wang >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Pelase review a patch to add the missing @return, @throws, >>>>>>>>>> @param statements in the java.xml package (excluding the DOM >>>>>>>>>> component). >>>>>>>>>> >>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8251561 >>>>>>>>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8251995 >>>>>>>>>> >>>>>>>>>> webrev: >>>>>>>>>> http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_01/ >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Joe >>>>>>>>> >>>>>>>>> >>>>>>>>> Best >>>>>>>>> Lance >>>>>>>>> ------------------ >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Lance Andersen| Principal Member of Technical Staff | >>>>>>>>> +1.781.442.2037 >>>>>>>>> Oracle Java Engineering >>>>>>>>> 1 Network Drive >>>>>>>>> Burlington, MA 01803 >>>>>>>>> Lance.Andersen at oracle.com >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>> >>>>>> >>>>> >>> > From mark.reinhold at oracle.com Wed Aug 26 00:01:59 2020 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Tue, 25 Aug 2020 17:01:59 -0700 Subject: RFR 8251989: Hex encoder and decoder utility In-Reply-To: References: <6743da26-9bae-e868-d242-b359aeec3d21@oracle.com> <20200820121012.674081461@eggemoggin.niobe.net> Message-ID: <20200825170159.892970153@eggemoggin.niobe.net> 2020/8/20 13:59:39 -0700, roger.riggs at oracle.com: > On 8/20/20 3:10 PM, mark.reinhold at oracle.com wrote: >> ... >> >> A few comments: >> >> - Why do the short-form `encoder` factory methods return encoders that >> produce upper-case hex strings? `Integer::toHexString` and other >> existing `toHex` methods return lower-case hex strings. That?s also >> what you get from common Unix CLI tools (e.g., `od -tx1`). >> >> Please consider changing these methods to return lower-case encoders. > > It's (almost) a toss up and easy to change; many of the existing uses > produce uppercase. Please change the default to lower case. It?s what people are going to expect. > Perhaps the short form no-arg should be replaced with a short form > constructor that > takes true/false, so it is explicit at the use site or put the case in > the name. > encodeToUpper(), encodeToLower(). > (A boolean parameter is not very informative, a enum would be better but > perhaps a bit heavyweight). Either a boolean or an enum parameter would be massive overkill. >> - Is it worth having static `Hex.encode(byte[])` and >> `Hex.decode(CharSequence)` convenience methods for the simplest >> cases? > > There was some discussion of that but it idea was to minimize the > surface area. Sorry, where was that discussion? I can?t find it in the core-libs-dev archive. >> - [Warning: Bikeshed] The verbs ?encode? and ?decode? seem unfortunate. >> >> Over in `java.nio.charsets` we have encoders that transform >> characters to bytes, and decoders that transform bytes to characters. >> The coded thing is the bytes; the uncoded thing is the characters. >> >> In `java.util` we already have the `Base64` class, which provides >> encoders that transform bytes to characters, and decoders that >> transform characters to bytes. The coded thing is the characters; >> the uncoded thing is the bytes. >> >> The use of ?encode? and ?decode? in `Base64` was likely inspired by >> the fact that the format has been known as ?base 64 encoding? for >> decades, having originated as a hack for transporting non-ASCII data >> via SMTP. Developers looking to do base-64 operations will, thus, >> expect this terminology. > > The bias of encoding vs decoding terminology is subtle, based the 'native' > form of the data. For the Charset classes, the native form of the data > is characters, > and the encoded form is bytes. For Base64, the native form of the data > is bytes, > and the encoded form is Base64 lines. Indeed -- and my point is that, in this case, encoding and decoding might not be the best concepts for API discoverability. (But, maybe they are.) It?s technically true that this API does encoding and decoding, whichever way you look at it. What I wonder about are the typical use cases. Will the primary uses of this API be to encode bytes into characters so that they can be transported via a medium that can only handle characters, and then decode the characters back into bytes on the other end? Or will the primary uses be to format bytes into a readable form, and likewise parse arbitrary-length hex strings into bytes? >> Here you?re proposing that the `Hex` class follow the `Base64` class. >> Consistency with existing nearby APIs is a worthy goal. If I were >> just looking to convert a byte array into a readable hex string, >> however, I?d probably want to ?format? it rather than ?encode? it, >> something like `String.format("%x")` on steroids. Likewise, if I >> were looking to convert a hex string into bytes then I?d want to >> ?parse? it rather than ?decode? it, i.e., `Integer::parseInt` on >> steroids. >> >> If you were to rename the nested classes to `Hex.Formatter` and >> `Hex.Parser`, and rename all methods accordingly, then this API would >> be inconsistent with the nearby `Base64` but likely more consistent >> with developer expectations. >> >> (`Hex` is already inconsistent with `Base64` in that it doesn?t >> prefix the names of its factory methods with `get`, which is a good >> thing.) > > The question is at what level of control does "encoding" become formatting. > There are very few formatting features in the API, no control of leading > zeros, no control > over indicental whitespace, and no control over width. Similarly with > parsing, > what flexibility does parsing imply that decoding does not, is > whitespace ignored, line endings/joins, etc. Optional support for prefix, suffix, and delimiter characters in both operations, and support for both lower- and upper-case characters on input, sure look like formatting and parsing to me. - Mark From kim.barrett at oracle.com Wed Aug 26 05:11:05 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 26 Aug 2020 01:11:05 -0400 Subject: RFR: 8188055: (ref) Add Reference.refersTo predicate In-Reply-To: References: <866762F3-AAB4-4673-80D1-841CE71513DC@azul.com> Message-ID: <4ADB21B2-A14F-4C7E-922E-B5E5F564CF92@oracle.com> Finally getting back to this after some internal discussion that dragged along because folks have been busy. > On Apr 8, 2020, at 12:05 PM, Gil Tene wrote: > > Lifting out of response from the JIRA issue: > > I always worry when proposing a change to an existing invariant, and > PhantomReference currently carries the stated and specified behavior > of "the referent of a phantom reference is always inaccessible". > > I can imagine quite a few forms of gaining new information I do not otherwise > have access to by using PhantomReference::RefersTo if it allowed me to examine > the current referent of a phantom reference and test to see if it is (a) null or (b) a > specific object I have a reference to. Both of those would provide me with information > that is impossible for me to get according to current specifications. With that newly > available information one can come up with all sorts of nice things to do... Think in > terms of "side-channel" as an example of the sort of thinking black hats can apply > to this new knowledge, but the potential attacks are not limited to side-channels. > > While it will be "obviously safe" to have Reference:RefersTo(obj) provide the same > information that (Reference.get() == obj) would, providing more information than > that would be a change to the specified behavior of Reference types, which we > should be extra paranoid about. Since PhantomReference::get returns null by > definition, we should remain consistent with that in PhantomReference::refersTo [copied from JIRA issue] That all assumes that I'm giving you access to my phantom references. But if I'm doing that then I've got much worse problems than any potential information leakage of the kind being suggested. If you have one of my phantom references then you can enqueue it, triggering any associated cleanup while I'm still using the referent. One should not give one's phantom references to untrusted code. That's nothing new, and is not made any worse by having Reference.refersTo work the same way for phantom references as for other reference types. From Alan.Bateman at oracle.com Wed Aug 26 10:42:28 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 26 Aug 2020 11:42:28 +0100 Subject: RFR 8252248: __SIGRTMAX is not declared in musl libc In-Reply-To: <1c4ad320-a22a-484a-d370-852f9ecd90f8@bell-sw.com> References: <1c4ad320-a22a-484a-d370-852f9ecd90f8@bell-sw.com> Message-ID: On 25/08/2020 18:00, Alexander Scherbatiy wrote: > > Hello, > > Could your review the fix for the issue: > ? Bug: https://bugs.openjdk.java.net/browse/JDK-8252248 > ? Fix: http://cr.openjdk.java.net/~alexsch/8252248/webrev.00/ > > : > > The fix has been discussed on the portola-dev alias [2] where it was > pointed out that the fix can be reviewed in the mainline and it was > suggested to rename the INTERRUPT_SIGNAL and move its definition to > net_util_md.h. > The xxx_close.c sources are for the old SocketImpl and DatagramSocketImpl implementations; they are aren't used by default and will eventually go away. Maybe an option for the musl port is to just not compile in the old socket implementations? It might be best to start a discussion on nio-dev and net-dev about this as we shouldn't be changing NativeThread.c (which is used for signalling in the FileChannel implementation) to include net_util.h. -Alan From Alan.Bateman at oracle.com Wed Aug 26 10:45:26 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 26 Aug 2020 11:45:26 +0100 Subject: RFR: JDK-8252113: Move jfr man page into jfr module In-Reply-To: References: Message-ID: <2f6a84ff-02f6-a4ba-c21b-03ca9ca9d6e5@oracle.com> On 20/08/2020 18:58, Adam Farley8 wrote: > Hi All, > > Should jfr.1 be moved from java.base to the jdk.jfr module source > directory, as indicated here? > > Webrev: http://cr.openjdk.java.net/~afarley/8252113/webrev/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8252113 > Seems right, on the surface at least, but it's probably best to check on hotspot-jfr-dev in case there is a reason for putting it there. -Alan. From Alan.Bateman at oracle.com Wed Aug 26 10:46:57 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 26 Aug 2020 11:46:57 +0100 Subject: RFR(L) 8244778 Archive full module graph in CDS In-Reply-To: References: <9e6b0043-65a1-dd97-a3d2-33679c8048d4@oracle.com> Message-ID: <1280b9d5-265b-f0d4-b606-75dc93758d41@oracle.com> On 12/08/2020 23:06, Ioi Lam wrote: > Hi Lois, > > Thanks for the comments. I have an updated webrev > > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/ > > http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02.delta/ > > > Here are the general notes on the changes. Replies to your questions > are in-line below. > > (1) Integrated updates in the Java code from Alan Bateman. Thanks for incorporating these changes, I don't have any other comments on the module changes. I see Mandy has reviewed that part too. -Alan From Alan.Bateman at oracle.com Wed Aug 26 11:00:57 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 26 Aug 2020 12:00:57 +0100 Subject: RFR [16/java.xml] 8251561: Fix doclint warnings in the java.xml package In-Reply-To: References: Message-ID: <62ecbab5-91d8-bee2-a380-1c2dfcdbb76e@oracle.com> On 21/08/2020 19:23, Joe Wang wrote: > Pelase review a patch to add the missing @return, @throws, @param > statements in the java.xml package (excluding the DOM component). > > JBS: https://bugs.openjdk.java.net/browse/JDK-8251561 > CSR: https://bugs.openjdk.java.net/browse/JDK-8251995 > > webrev: http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_01/ In passing, the javax.xml.stream changes have really odd formatting, some of the sources are a mix of 2 and 4-space indentation.? Also they have an really odd "@author Copyright (c) 2009 Oracle Corporation". Nothing to do with fixing the doclint warnings but maybe something that could be cleaned up when next touching this code. -Alan From aleksej.efimov at oracle.com Wed Aug 26 11:46:52 2020 From: aleksej.efimov at oracle.com (Aleks Efimov) Date: Wed, 26 Aug 2020 12:46:52 +0100 Subject: [16] RFR: 8251182: Fix "no comment" warnings in java.naming In-Reply-To: References: Message-ID: <3a4bb2aa-0f87-70bd-3351-3af6b11c1ec5@oracle.com> Hi Roger, Lance, Daniel, Thanks for your reviews and comments! New webrev with suggested modifications can be viewed at this location: http://cr.openjdk.java.net/~aefimov/8251182/01 The list of modifications: - Added @java.io.Serial annotation to 'serialVersionUID' declarations - Removed trailing periods from @param/@throws - Added generic first line comments to writeObject/readObject methods The new version of patch and webrev files have been uploaded to the CSR issue. Kind Regards, Aleksei On 25/08/2020 20:23, Daniel Fuchs wrote: > Hi Aleksei, > > LGTM. > > You could also add the @Seial annotation to serialVersionUID > declarations. > > best regards, > > -- daniel > > On 25/08/2020 16:03, Aleks Efimov wrote: >> Hi, >> >> The documentation of classes from "java.naming" module needs to be >> improved to resolve javadoc -Xdoclint "no comment" warnings. >> >> JBS issue: https://bugs.openjdk.java.net/browse/JDK-8251182 >> Webrev: http://cr.openjdk.java.net/~aefimov/8251182/00/index.html >> CSR: https://bugs.openjdk.java.net/browse/JDK-8252310 >> >> With Best Regards, >> Aleksei > From thomas.stuefe at gmail.com Wed Aug 26 12:03:06 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 26 Aug 2020 14:03:06 +0200 Subject: RFR 8252248: __SIGRTMAX is not declared in musl libc In-Reply-To: <1c4ad320-a22a-484a-d370-852f9ecd90f8@bell-sw.com> References: <1c4ad320-a22a-484a-d370-852f9ecd90f8@bell-sw.com> Message-ID: Hi Alexander, --- src/java.base/unix/native/libnet/net_util_md.h why include pthread.h? SIGRTMAX is in signal.h on posix ( https://pubs.opengroup.org/onlinepubs/007908799/xsh/signal.h.html). I see the old code included this too, pretty sure this was either incidental or just wrong. --- Are there tests which exercise this code? --- Thanks, Thomas On Tue, Aug 25, 2020 at 7:02 PM Alexander Scherbatiy < alexander.scherbatiy at bell-sw.com> wrote: > > Hello, > > Could your review the fix for the issue: > Bug: https://bugs.openjdk.java.net/browse/JDK-8252248 > Fix: http://cr.openjdk.java.net/~alexsch/8252248/webrev.00/ > > Building JDK on Alpine Linux with musl libc gives the error: > src/java.base/linux/native/libnet/linux_close.c:63:25: error: > '__SIGRTMAX' undeclared here (not in a function); did you mean 'SIGRTMAX'? > 63 | static int sigWakeup = (__SIGRTMAX - 2); > > musl libc does not contain __SIGRTMAX definition. It defines only > SIGRTMAX as a function call [1]. > > __SIGRTMAX is used in JDK in two files NativeThread.c and linux_close.c . > > The first idea of the fix was simply to substitute __SIGRTMAX to > SIGRTMAX in NativeThread.c and use the same definition > #define INTERRUPT_SIGNAL (__SIGRTMAX - 2) > in linux_close.c. > "static int sigWakeup = (SIGRTMAX - 2)" does not work because the > SIGRTMAX is defined as a function and leads to "initializer element is > not constant" error. > > The fix has been discussed on the portola-dev alias [2] where it was > pointed out that the fix can be reviewed in the mainline and it was > suggested to rename the INTERRUPT_SIGNAL and move its definition to > net_util_md.h. > > > [1] https://git.musl-libc.org/cgit/musl/tree/include/signal.h#n196 > [2] > https://mail.openjdk.java.net/pipermail/portola-dev/2020-August/000471.html > > Thanks, > Alexander. > > From daniel.fuchs at oracle.com Wed Aug 26 12:17:36 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 26 Aug 2020 13:17:36 +0100 Subject: [16] RFR: 8251182: Fix "no comment" warnings in java.naming In-Reply-To: <3a4bb2aa-0f87-70bd-3351-3af6b11c1ec5@oracle.com> References: <3a4bb2aa-0f87-70bd-3351-3af6b11c1ec5@oracle.com> Message-ID: Hi Aleksei, Thanks for taking this on. The new version looks good to me. best regards, -- daniel On 26/08/2020 12:46, Aleks Efimov wrote: > Hi Roger, Lance, Daniel, > > Thanks for your reviews and comments! > New webrev with suggested modifications can be viewed at this location: > http://cr.openjdk.java.net/~aefimov/8251182/01 > > The list of modifications: > - Added @java.io.Serial annotation to 'serialVersionUID' declarations > - Removed trailing periods from @param/@throws > - Added generic first line comments to writeObject/readObject methods > > The new version of patch and webrev files have been uploaded to the CSR > issue. > > Kind Regards, > Aleksei From adam.farley at uk.ibm.com Wed Aug 26 13:16:29 2020 From: adam.farley at uk.ibm.com (Adam Farley8) Date: Wed, 26 Aug 2020 14:16:29 +0100 Subject: RFR: JDK-8252113: Move jfr man page into jfr module In-Reply-To: <2f6a84ff-02f6-a4ba-c21b-03ca9ca9d6e5@oracle.com> References: <2f6a84ff-02f6-a4ba-c21b-03ca9ca9d6e5@oracle.com> Message-ID: Good idea. Will do. Best Regards Adam Farley IBM Runtimes Alan Bateman wrote on 26/08/2020 11:45:26: > From: Alan Bateman > To: Adam Farley8 , core-libs-dev dev at openjdk.java.net> > Date: 26/08/2020 11:45 > Subject: [EXTERNAL] Re: RFR: JDK-8252113: Move jfr man page into jfr module > > On 20/08/2020 18:58, Adam Farley8 wrote: > > Hi All, > > > > Should jfr.1 be moved from java.base to the jdk.jfr module source > > directory, as indicated here? > > > > Webrev: INVALID URI REMOVED > u=http-3A__cr.openjdk.java.net_-7Eafarley_8252113_webrev_&d=DwICaQ&c=jf_iaSHvJObTbx- > siA1ZOg&r=P5m8KWUXJf-CeVJc0hDGD9AQ2LkcXDC0PMV9ntVw5Ho&m=N- > GeKGCf6S77WhCoGjJitjdOl8EbxqFrYhafb1kCirM&s=LxPuyPEj6w2jefWLVebN4K7zuTe9O07wjiXjmYuMS- > Y&e= > > > > Bug: INVALID URI REMOVED > u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8252113&d=DwICaQ&c=jf_iaSHvJObTbx- > siA1ZOg&r=P5m8KWUXJf-CeVJc0hDGD9AQ2LkcXDC0PMV9ntVw5Ho&m=N- > GeKGCf6S77WhCoGjJitjdOl8EbxqFrYhafb1kCirM&s=Lv9k35fk2cql2PCkqGbBIj6-7X_B8kivaOCpZ4CNvwQ&e= > > > Seems right, on the surface at least, but it's probably best to check on > hotspot-jfr-dev in case there is a reason for putting it there. > > -Alan. > Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From Roger.Riggs at oracle.com Wed Aug 26 13:59:09 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Wed, 26 Aug 2020 09:59:09 -0400 Subject: [16] RFR: 8251182: Fix "no comment" warnings in java.naming In-Reply-To: References: <3a4bb2aa-0f87-70bd-3351-3af6b11c1ec5@oracle.com> Message-ID: <9937cec1-9abe-c81e-75ec-c1841697bec7@oracle.com> Looks good,? Thanks for the updates. Roger On 8/26/20 8:17 AM, Daniel Fuchs wrote: > Hi Aleksei, > > Thanks for taking this on. The new version looks good to me. > > best regards, > > -- daniel > > On 26/08/2020 12:46, Aleks Efimov wrote: >> Hi Roger, Lance, Daniel, >> >> Thanks for your reviews and comments! >> New webrev with suggested modifications can be viewed at this location: >> http://cr.openjdk.java.net/~aefimov/8251182/01 >> >> The list of modifications: >> - Added @java.io.Serial annotation to 'serialVersionUID' declarations >> - Removed trailing periods from @param/@throws >> - Added generic first line comments to writeObject/readObject methods >> >> The new version of patch and webrev files have been uploaded to the >> CSR issue. >> >> Kind Regards, >> Aleksei From christoph.dreis at freenet.de Wed Aug 26 14:18:16 2020 From: christoph.dreis at freenet.de (Christoph Dreis) Date: Wed, 26 Aug 2020 16:18:16 +0200 Subject: [NEW BUG] NumberFormat.parse fails in some scenarios Message-ID: Hi, A colleague of mine (filipe.silvestrim at innogames.com) approached me today that his code wasn?t working that converted a currency String into cents. Apparently, the code worked with Java 8 while it didn?t with 11+. public class Main { public static void main(String[] args) throws IOException { // System.setProperty("java.locale.providers", "JRE"); System.out.println(getPriceInCents(Locale.GERMANY, "9,99 ?")); } static int getPriceInCents(Locale locale, String price) { try { DecimalFormat format = (DecimalFormat) NumberFormat.getCurrencyInstance(locale); Number number = format.parse(price); return (int) (number.doubleValue() * 100); } catch (ParseException e) { // This should be thrown on JDK 9+ System.out.println(e); } return 0; } } After some digging I think this is caused by the changes done for JDK-8008577[1]. When I change the java.locale.providers property to "JRE" for example, it works again. My investigations so far revealed that apparently the CLDR number pattern for the currency slightly differs. I created breakpoints in sun.util.locale.provider.NumberFormatProviderImpl::getInstance() to display some things: LocaleProviderAdapter adapter = LocaleProviderAdapter.forType(type); String[] numberPatterns = adapter.getLocaleResources(override).getNumberPatterns(); DecimalFormatSymbols symbols = DecimalFormatSymbols.getInstance(override); int entry = (choice == INTEGERSTYLE) ? NUMBERSTYLE : choice; DecimalFormat format = new DecimalFormat(numberPatterns[entry], symbols); // CLDR (type) // #,##0.00?? (numberPatterns[entry]) // [35,44,35,35,48,46,48,48,-62,-96,-62,-92] (numberPatterns[entry] in bytes) // // JRE type // #,##0.00 ?;-#,##0.00 ? (numberPatterns[entry]) // [35,44,35,35,48,46,48,48,32,-62,-92,59,45,35,44,35,35,48,46,48,48,32,-62,-92] (numberPatterns[entry] in bytes) The JRE one includes the negative pattern, but the more interesting bit is that apparently the spacing differs here. For JRE it seems to be a normal space (the 32), but for CLDR it's showing [-62, -96] which seems to be a non breaking space aka nbsp. Ultimately this leads to a check failing in DecimalFormat when parsing the string "9,99 ?" that obviously includes a normal space. if (gotPositive) { // the regionMatches will return false because nbsp != space gotPositive = text.regionMatches(position,positiveSuffix,0, positiveSuffix.length()); } Which itself leads to the following in our case: // fail if neither or both if (gotPositive == gotNegative) { parsePosition.errorIndex = position; // We hit this part here which causes the parsing to fail return false; } There are workarounds - e.g. by setting java.locale.providers as already mentioned or setting format.setPositiveSuffix(" ?"); to fix this particular case. Is this a bug or a feature or are we missing something? In case this is an actual bug we would appreciate a "reported-by" mentioning in an eventual fix. Thanks in advance. I do hope you can follow my thoughts in this email. [1] https://bugs.openjdk.java.net/browse/JDK-8008577 Cheers, Christoph From Roger.Riggs at oracle.com Wed Aug 26 14:56:13 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Wed, 26 Aug 2020 10:56:13 -0400 Subject: [NEW BUG] NumberFormat.parse fails in some scenarios In-Reply-To: References: Message-ID: Hi Christoph, Its worth investigating, tracking the issue with: https://bugs.openjdk.java.net/browse/JDK-8252383 Bugs can be reported with: ?? https://bugreport.java.com/bugreport/ Thanks, Roger On 8/26/20 10:18 AM, Christoph Dreis wrote: > Hi, > > A colleague of mine (filipe.silvestrim at innogames.com) approached me today that his code wasn?t working that converted a currency String into cents. > Apparently, the code worked with Java 8 while it didn?t with 11+. > > public class Main { > > public static void main(String[] args) throws IOException { > // System.setProperty("java.locale.providers", "JRE"); > System.out.println(getPriceInCents(Locale.GERMANY, "9,99 ?")); > } > > static int getPriceInCents(Locale locale, String price) { > try { > DecimalFormat format = (DecimalFormat) NumberFormat.getCurrencyInstance(locale); > Number number = format.parse(price); > return (int) (number.doubleValue() * 100); > } catch (ParseException e) { > // This should be thrown on JDK 9+ > System.out.println(e); > } > return 0; > } > > } > > After some digging I think this is caused by the changes done for JDK-8008577[1]. > When I change the java.locale.providers property to "JRE" for example, it works again. > > My investigations so far revealed that apparently the CLDR number pattern for the currency slightly differs. > > I created breakpoints in sun.util.locale.provider.NumberFormatProviderImpl::getInstance() to display some things: > > LocaleProviderAdapter adapter = LocaleProviderAdapter.forType(type); > String[] numberPatterns = adapter.getLocaleResources(override).getNumberPatterns(); > DecimalFormatSymbols symbols = DecimalFormatSymbols.getInstance(override); > int entry = (choice == INTEGERSTYLE) ? NUMBERSTYLE : choice; > DecimalFormat format = new DecimalFormat(numberPatterns[entry], symbols); > > // CLDR (type) > // #,##0.00?? (numberPatterns[entry]) > // [35,44,35,35,48,46,48,48,-62,-96,-62,-92] (numberPatterns[entry] in bytes) > > // > // JRE type > // #,##0.00 ?;-#,##0.00 ? (numberPatterns[entry]) > // [35,44,35,35,48,46,48,48,32,-62,-92,59,45,35,44,35,35,48,46,48,48,32,-62,-92] (numberPatterns[entry] in bytes) > > The JRE one includes the negative pattern, but the more interesting bit is that apparently the spacing differs here. > For JRE it seems to be a normal space (the 32), but for CLDR it's showing [-62, -96] which seems to be a non breaking space aka nbsp. > > Ultimately this leads to a check failing in DecimalFormat when parsing the string "9,99 ?" that obviously includes a normal space. > > if (gotPositive) { > // the regionMatches will return false because nbsp != space > gotPositive = text.regionMatches(position,positiveSuffix,0, > positiveSuffix.length()); > } > > Which itself leads to the following in our case: > > // fail if neither or both > if (gotPositive == gotNegative) { > parsePosition.errorIndex = position; > // We hit this part here which causes the parsing to fail > return false; > } > > > There are workarounds - e.g. by setting java.locale.providers as already mentioned or setting format.setPositiveSuffix(" ?"); to fix this particular case. > > Is this a bug or a feature or are we missing something? > > In case this is an actual bug we would appreciate a "reported-by" mentioning in an eventual fix. > > Thanks in advance. I do hope you can follow my thoughts in this email. > > [1] https://bugs.openjdk.java.net/browse/JDK-8008577 > > Cheers, > Christoph > > From martin.doerr at sap.com Wed Aug 26 15:26:59 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Wed, 26 Aug 2020 15:26:59 +0000 Subject: RFR(M): 8248188: [PATCH] Add HotSpotIntrinsicCandidate and API for Base64 decoding In-Reply-To: References: <11ca749f-3015-c004-aa6b-3194e1dfe4eb@linux.ibm.com> Message-ID: Hi Corey, I should explain my comments regarding Base64.java better. > Let's be precise: "should process a multiple of four" => "must process a > multiple of four" Did you try to support non-multiple of 4 and this was intended as recommendation? I think making it a requirement and simplifying the logic in decode0 is better. Or what's the benefit of the recommendation? > > If any illegal base64 bytes are encountered in the source by the > > intrinsic, the intrinsic can return a data length of zero or any > > number of bytes before the place where the illegal base64 byte > > was encountered. > I think this has a drawback. Somebody may use a debugger and want to stop > when throwing IllegalArgumentException. He should see the position which > matches the Java implementation. This is probably hard to understand. Let me try to explain it by example: 1. 80 Bytes get processed by the intrinsic and 60 Bytes written to the destination array. 2. The intrinsic sees an illegal base64 Byte and it returns 12 which is allowed by your specification. 3. The compiled method containing the intrinsic hits a safepoint (e.g. in the large while loop in decodeBlockSlow). 4. A JVMTI agent (debugger) reads dp and dst. 5. The person using the debugger gets angry because more bytes than dp were written into dst. The JVM didn't follow the specified behavior. I guess we can and should avoid it by specifying that the intrinsic needs to return the dp value matching the number of Bytes written. Best regards, Martin > -----Original Message----- > From: Doerr, Martin > Sent: Dienstag, 25. August 2020 15:38 > To: Corey Ashford ; Michihiro Horie > > Cc: hotspot-compiler-dev at openjdk.java.net; core-libs-dev dev at openjdk.java.net>; Kazunori Ogata ; > joserz at br.ibm.com > Subject: RE: RFR(M): 8248188: [PATCH] Add HotSpotIntrinsicCandidate and > API for Base64 decoding > > Hi Corey, > > thanks for proposing this change. I have comments and suggestions > regarding various files. > > > Base64.java > > This is the only file which needs another review from core-libs-dev. > First of all, I like the idea to use a HotSpotIntrinsicCandidate which can > consume as many bytes as the implementation wants. > > Comment before decodeBlock: > Let's be precise: "should process a multiple of four" => "must process a > multiple of four" > > > If any illegal base64 bytes are encountered in the source by the > > intrinsic, the intrinsic can return a data length of zero or any > > number of bytes before the place where the illegal base64 byte > > was encountered. > I think this has a drawback. Somebody may use a debugger and want to stop > when throwing IllegalArgumentException. He should see the position which > matches the Java implementation. > > Please note that the comment indentation differs from other comments. > > decode0: Final "else" after return is redundant. > > > stubGenerator_ppc.cpp > > "__vector" breaks AIX build! > Does it work on Big Endian linux with old gcc (we require 7.3.1, now)? > Please either support Big Endian properly or #ifdef it out. > What exactly does it on linux? > I remember that we had tried such prefixes but were not satisfied. I think it > didn't enforce 16 Byte alignment if I remember correctly. > > Attention: C2 does no longer convert int/bool to 64 bit values (since JDK- > 8086069). So the argument registers for offset, length and isURL may contain > garbage in the higher bits. > > You may want to use load_const_optimized which produces shorter code. > > You may want to use __ align(32) to align unrolled_loop_start. > > I'll review the algorithm in detail when I find more time. > > > assembler_ppc.hpp > assembler_ppc.inline.hpp > vm_version_ppc.cpp > vm_version_ppc.hpp > Please rebase. Parts of the change were pushed as part of 8248190: Enable > Power10 system and implement new byte-reverse instructions > > > vmSymbols.hpp > Indentation looks odd at the end. > > > library_call.cpp > Good. Indentation style of the call parameters differs from encodeBlock. > > > runtime.cpp > Good. > > > aotCodeHeap.cpp > vmSymbols.cpp > shenandoahSupport.cpp > vmStructs_jvmci.cpp > shenandoahSupport.cpp > escape.cpp > runtime.hpp > stubRoutines.cpp > stubRoutines.hpp > vmStructs.cpp > Good and trivial. > > > Tests: > I think we should have JTREG tests to check for regressions in the future. > > Best regards, > Martin > > > > -----Original Message----- > > From: Corey Ashford > > Sent: Mittwoch, 19. August 2020 20:11 > > To: Michihiro Horie > > Cc: hotspot-compiler-dev at openjdk.java.net; core-libs-dev > dev at openjdk.java.net>; Kazunori Ogata ; > > joserz at br.ibm.com; Doerr, Martin > > Subject: Re: RFR(M): 8248188: [PATCH] Add HotSpotIntrinsicCandidate and > > API for Base64 decoding > > > > Michihiro Horie posted up a new iteration of this webrev for me. This > > time the webrev includes a complete implementation of the intrinsic for > > Power9 and Power10. > > > > You can find it here: > > http://cr.openjdk.java.net/~mhorie/8248188/webrev.02/ > > > > Changes in webrev.02 vs. webrev.01: > > > > * The method header for the intrinsic in the Base64 code has been > > rewritten using the Javadoc style. The clarity of the comments has been > > improved and some verbosity has been removed. There are no additional > > functional changes to Base64.java. > > > > * The code needed to martial and check the intrinsic parameters has > > been added, using the base64 encodeBlock intrinsic as a guideline. > > > > * A complete intrinsic implementation for Power9 and Power10 is > included. > > > > * Adds some Power9 and Power10 assembler instructions needed by the > > intrinsic which hadn't been defined before. > > > > The intrinsic implementation in this patch accelerates the decoding of > > large blocks of base64 data by a factor of about 3.5X on Power9. > > > > I'm attaching two Java test cases I am using for testing and > > benchmarking. The TestBase64_VB encodes and decodes randomly-sized > > buffers of random data and checks that original data matches the > > encoded-then-decoded data. TestBase64Errors encodes a 48K block of > > random bytes, then corrupts each byte of the encoded data, one at a > > time, checking to see if the decoder catches the illegal byte. > > > > Any comments/suggestions would be appreciated. > > > > Thanks, > > > > - Corey > > > > On 7/27/20 6:49 PM, Corey Ashford wrote: > > > Michihiro Horie uploaded a new revision of the Base64 decodeBlock > > > intrinsic API for me: > > > > > > http://cr.openjdk.java.net/~mhorie/8248188/webrev.01/ > > > > > > It has the following changes with respect to the original one posted: > > > > > > ?* In the event of encountering a non-base64 character, instead of > > > having a separate error code of -1, the intrinsic can now just return > > > either 0, or the number of data bytes produced up to the point where > the > > > illegal base64 character was encountered.? This reduces the number of > > > special cases, and also provides a way to speed up the process of > > > finding the bad character by the slower, pure-Java algorithm. > > > > > > ?* The isMIME boolean is removed from the API for two reasons: > > > ?? - The current API is not sufficient to handle the isMIME case, > > > because there isn't a strict relationship between the number of input > > > bytes and the number of output bytes, because there can be an arbitrary > > > number of non-base64 characters in the source. > > > ?? - If an intrinsic only implements the (isMIME == false) case as ours > > > does, it will always return 0 bytes processed, which will slightly slow > > > down the normal path of processing an (isMIME == true) instantiation. > > > ?? - We considered adding a separate hotspot candidate for the (isMIME > > > == true) case, but since we don't have an intrinsic implementation to > > > test that, we decided to leave it as a future optimization. > > > > > > Comments and suggestions are welcome.? Thanks for your consideration. > > > > > > - Corey > > > > > > On 6/23/20 6:23 PM, Michihiro Horie wrote: > > >> Hi Corey, > > >> > > >> Following is the issue I created. > > >> https://bugs.openjdk.java.net/browse/JDK-8248188 > > >> > > >> I will upload a webrev when you're ready as we talked in private. > > >> > > >> Best regards, > > >> Michihiro > > >> > > >> Inactive hide details for "Corey Ashford" ---2020/06/24 > > >> 09:40:10---Currently in java.util.Base64, there is a > > >> HotSpotIntrinsicCa"Corey Ashford" ---2020/06/24 09:40:10---Currently > > >> in java.util.Base64, there is a HotSpotIntrinsicCandidate and API for > > >> encodeBlock, but no > > >> > > >> From: "Corey Ashford" > > >> To: "hotspot-compiler-dev at openjdk.java.net" > > >> , > > >> "ppc-aix-port-dev at openjdk.java.net" > dev at openjdk.java.net> > > >> Cc: Michihiro Horie/Japan/IBM at IBMJP, Kazunori > > Ogata/Japan/IBM at IBMJP, > > >> joserz at br.ibm.com > > >> Date: 2020/06/24 09:40 > > >> Subject: RFR(S): [PATCH] Add HotSpotIntrinsicCandidate and API for > > >> Base64 decoding > > >> > > >> ------------------------------------------------------------------------ > > >> > > >> > > >> > > >> Currently in java.util.Base64, there is a HotSpotIntrinsicCandidate and > > >> API for encodeBlock, but none for decoding. ?This means that only > > >> encoding gets acceleration from the underlying CPU's vector hardware. > > >> > > >> I'd like to propose adding a new intrinsic for decodeBlock. ?The > > >> considerations I have for this new intrinsic's API: > > >> > > >> ??* Don't make any assumptions about the underlying capability of the > > >> hardware. ?For example, do not impose any specific block size > > >> granularity. > > >> > > >> ??* Don't assume the underlying intrinsic can handle isMIME or isURL > > >> modes, but also let them decide if they will process the data regardless > > >> of the settings of the two booleans. > > >> > > >> ??* Any remaining data that is not processed by the intrinsic will be > > >> processed by the pure Java implementation. ?This allows the intrinsic to > > >> process whatever block sizes it's good at without the complexity of > > >> handling the end fragments. > > >> > > >> ??* If any illegal character is discovered in the decoding process, the > > >> intrinsic will simply return -1, instead of requiring it to throw a > > >> proper exception from the context of the intrinsic. ?In the event of > > >> getting a -1 returned from the intrinsic, the Java Base64 library code > > >> simply calls the pure Java implementation to have it find the error and > > >> properly throw an exception. ?This is a performance trade-off in the > > >> case of an error (which I expect to be very rare). > > >> > > >> ??* One thought I have for a further optimization (not implemented in > > >> the current patch), is that when the intrinsic decides not to process a > > >> block because of some combination of isURL and isMIME settings it > > >> doesn't handle, it could return extra bits in the return code, encoded > > >> as a negative number. ?For example: > > >> > > >> Illegal_Base64_char ? = 0b001; > > >> isMIME_unsupported ? ?= 0b010; > > >> isURL_unsupported ? ? = 0b100; > > >> > > >> These can be OR'd together as needed and then negated (flip the sign). > > >> The Base64 library code could then cache these flags, so it will know > > >> not to call the intrinsic again when another decodeBlock is requested > > >> but with an unsupported mode. ?This will save the performance hit of > > >> calling the intrinsic when it is guaranteed to fail. > > >> > > >> I've tested the attached patch with an actual intrinsic coded up for > > >> Power9/Power10, but those runtime intrinsics and arch-specific patches > > >> aren't attached today. ?I want to get some consensus on the > > >> library-level intrinsic API first. > > >> > > >> Also attached is a simple test case to test that the new intrinsic API > > >> doesn't break anything. > > >> > > >> I'm open to any comments about this. > > >> > > >> Thanks for your consideration, > > >> > > >> - Corey > > >> > > >> > > >> Corey Ashford > > >> IBM Systems, Linux Technology Center, OpenJDK team > > >> cjashfor at us dot ibm dot com > > >> [attachment "decodeBlock_api-20200623.patch" deleted by Michihiro > > >> Horie/Japan/IBM] [attachment "TestBase64.java" deleted by Michihiro > > >> Horie/Japan/IBM] > > >> > > >> > > > From huizhe.wang at oracle.com Wed Aug 26 15:49:12 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Wed, 26 Aug 2020 08:49:12 -0700 Subject: RFR [16/java.xml] 8251561: Fix doclint warnings in the java.xml package In-Reply-To: <62ecbab5-91d8-bee2-a380-1c2dfcdbb76e@oracle.com> References: <62ecbab5-91d8-bee2-a380-1c2dfcdbb76e@oracle.com> Message-ID: <224dd80b-5d8f-035d-019f-a2a6f85b0414@oracle.com> On 8/26/20 4:00 AM, Alan Bateman wrote: > On 21/08/2020 19:23, Joe Wang wrote: >> Pelase review a patch to add the missing @return, @throws, @param >> statements in the java.xml package (excluding the DOM component). >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8251561 >> CSR: https://bugs.openjdk.java.net/browse/JDK-8251995 >> >> webrev: http://cr.openjdk.java.net/~joehw/jdk16/8251561/webrev_01/ > In passing, the javax.xml.stream changes have really odd formatting, > some of the sources are a mix of 2 and 4-space indentation.? Also they > have an really odd "@author Copyright (c) 2009 Oracle Corporation". > Nothing to do with fixing the doclint warnings but maybe something > that could be cleaned up when next touching this code. Thanks Alan!? I wondered someone would spot these peculiar things :-) Added your comment to https://bugs.openjdk.java.net/browse/JDK-8252328 -Joe > > -Alan From LANCE.ANDERSEN at ORACLE.COM Wed Aug 26 15:59:34 2020 From: LANCE.ANDERSEN at ORACLE.COM (Lance Andersen) Date: Wed, 26 Aug 2020 11:59:34 -0400 Subject: [16] RFR: 8251182: Fix "no comment" warnings in java.naming In-Reply-To: <3a4bb2aa-0f87-70bd-3351-3af6b11c1ec5@oracle.com> References: <3a4bb2aa-0f87-70bd-3351-3af6b11c1ec5@oracle.com> Message-ID: <4CDC2073-B719-495A-B68B-3ECB38E64054@ORACLE.COM> +1 > On Aug 26, 2020, at 7:46 AM, Aleks Efimov wrote: > > Hi Roger, Lance, Daniel, > > Thanks for your reviews and comments! > New webrev with suggested modifications can be viewed at this location: > http://cr.openjdk.java.net/~aefimov/8251182/01 > > The list of modifications: > - Added @java.io.Serial annotation to 'serialVersionUID' declarations > - Removed trailing periods from @param/@throws > - Added generic first line comments to writeObject/readObject methods > > The new version of patch and webrev files have been uploaded to the CSR issue. > > Kind Regards, > Aleksei > > On 25/08/2020 20:23, Daniel Fuchs wrote: >> Hi Aleksei, >> >> LGTM. >> >> You could also add the @Seial annotation to serialVersionUID >> declarations. >> >> best regards, >> >> -- daniel >> >> On 25/08/2020 16:03, Aleks Efimov wrote: >>> Hi, >>> >>> The documentation of classes from "java.naming" module needs to be improved to resolve javadoc -Xdoclint "no comment" warnings. >>> >>> JBS issue: https://bugs.openjdk.java.net/browse/JDK-8251182 >>> Webrev: http://cr.openjdk.java.net/~aefimov/8251182/00/index.html >>> CSR: https://bugs.openjdk.java.net/browse/JDK-8252310 >>> >>> With Best Regards, >>> Aleksei >> > Best Lance ------------------ Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From vipinsharma85 at gmail.com Wed Aug 26 16:38:01 2020 From: vipinsharma85 at gmail.com (Vipin Sharma) Date: Wed, 26 Aug 2020 22:08:01 +0530 Subject: RFR 8252265: Replace @exception with @throws java.util.logging package Message-ID: Hi, Please review and sponsor the fix for replacing @exception with @throws in java.util.logging package. Issue: https://bugs.openjdk.java.net/browse/JDK-8252265 Webrev: https://cr.openjdk.java.net/~vsharma/8252265/ It came up in the below discussion, starting this cleanup with java.util.logging package. https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068172.html Regards, Vipin From cjashfor at linux.ibm.com Wed Aug 26 16:50:05 2020 From: cjashfor at linux.ibm.com (Corey Ashford) Date: Wed, 26 Aug 2020 09:50:05 -0700 Subject: RFR(M): 8248188: [PATCH] Add HotSpotIntrinsicCandidate and API for Base64 decoding In-Reply-To: References: <11ca749f-3015-c004-aa6b-3194e1dfe4eb@linux.ibm.com> Message-ID: Thanks for your careful review, Martin. I will consider what you have said, and reply with comments/questions and possibly a revised webrev if I think I can satisfy your concerns. Regards, - Corey On 8/26/20 8:26 AM, Doerr, Martin wrote: > Hi Corey, > > I should explain my comments regarding Base64.java better. > >> Let's be precise: "should process a multiple of four" => "must process a >> multiple of four" > Did you try to support non-multiple of 4 and this was intended as recommendation? > I think making it a requirement and simplifying the logic in decode0 is better. > Or what's the benefit of the recommendation? > >>> If any illegal base64 bytes are encountered in the source by the >>> intrinsic, the intrinsic can return a data length of zero or any >>> number of bytes before the place where the illegal base64 byte >>> was encountered. >> I think this has a drawback. Somebody may use a debugger and want to stop >> when throwing IllegalArgumentException. He should see the position which >> matches the Java implementation. > This is probably hard to understand. Let me try to explain it by example: > 1. 80 Bytes get processed by the intrinsic and 60 Bytes written to the destination array. > 2. The intrinsic sees an illegal base64 Byte and it returns 12 which is allowed by your specification. > 3. The compiled method containing the intrinsic hits a safepoint (e.g. in the large while loop in decodeBlockSlow). > 4. A JVMTI agent (debugger) reads dp and dst. > 5. The person using the debugger gets angry because more bytes than dp were written into dst. The JVM didn't follow the specified behavior. > > I guess we can and should avoid it by specifying that the intrinsic needs to return the dp value matching the number of Bytes written. > > Best regards, > Martin > > >> -----Original Message----- >> From: Doerr, Martin >> Sent: Dienstag, 25. August 2020 15:38 >> To: Corey Ashford ; Michihiro Horie >> >> Cc: hotspot-compiler-dev at openjdk.java.net; core-libs-dev > dev at openjdk.java.net>; Kazunori Ogata ; >> joserz at br.ibm.com >> Subject: RE: RFR(M): 8248188: [PATCH] Add HotSpotIntrinsicCandidate and >> API for Base64 decoding >> >> Hi Corey, >> >> thanks for proposing this change. I have comments and suggestions >> regarding various files. >> >> >> Base64.java >> >> This is the only file which needs another review from core-libs-dev. >> First of all, I like the idea to use a HotSpotIntrinsicCandidate which can >> consume as many bytes as the implementation wants. >> >> Comment before decodeBlock: >> Let's be precise: "should process a multiple of four" => "must process a >> multiple of four" >> >>> If any illegal base64 bytes are encountered in the source by the >>> intrinsic, the intrinsic can return a data length of zero or any >>> number of bytes before the place where the illegal base64 byte >>> was encountered. >> I think this has a drawback. Somebody may use a debugger and want to stop >> when throwing IllegalArgumentException. He should see the position which >> matches the Java implementation. >> >> Please note that the comment indentation differs from other comments. >> >> decode0: Final "else" after return is redundant. >> >> >> stubGenerator_ppc.cpp >> >> "__vector" breaks AIX build! >> Does it work on Big Endian linux with old gcc (we require 7.3.1, now)? >> Please either support Big Endian properly or #ifdef it out. >> What exactly does it on linux? >> I remember that we had tried such prefixes but were not satisfied. I think it >> didn't enforce 16 Byte alignment if I remember correctly. >> >> Attention: C2 does no longer convert int/bool to 64 bit values (since JDK- >> 8086069). So the argument registers for offset, length and isURL may contain >> garbage in the higher bits. >> >> You may want to use load_const_optimized which produces shorter code. >> >> You may want to use __ align(32) to align unrolled_loop_start. >> >> I'll review the algorithm in detail when I find more time. >> >> >> assembler_ppc.hpp >> assembler_ppc.inline.hpp >> vm_version_ppc.cpp >> vm_version_ppc.hpp >> Please rebase. Parts of the change were pushed as part of 8248190: Enable >> Power10 system and implement new byte-reverse instructions >> >> >> vmSymbols.hpp >> Indentation looks odd at the end. >> >> >> library_call.cpp >> Good. Indentation style of the call parameters differs from encodeBlock. >> >> >> runtime.cpp >> Good. >> >> >> aotCodeHeap.cpp >> vmSymbols.cpp >> shenandoahSupport.cpp >> vmStructs_jvmci.cpp >> shenandoahSupport.cpp >> escape.cpp >> runtime.hpp >> stubRoutines.cpp >> stubRoutines.hpp >> vmStructs.cpp >> Good and trivial. >> >> >> Tests: >> I think we should have JTREG tests to check for regressions in the future. >> >> Best regards, >> Martin >> >> >>> -----Original Message----- >>> From: Corey Ashford >>> Sent: Mittwoch, 19. August 2020 20:11 >>> To: Michihiro Horie >>> Cc: hotspot-compiler-dev at openjdk.java.net; core-libs-dev >> dev at openjdk.java.net>; Kazunori Ogata ; >>> joserz at br.ibm.com; Doerr, Martin >>> Subject: Re: RFR(M): 8248188: [PATCH] Add HotSpotIntrinsicCandidate and >>> API for Base64 decoding >>> >>> Michihiro Horie posted up a new iteration of this webrev for me. This >>> time the webrev includes a complete implementation of the intrinsic for >>> Power9 and Power10. >>> >>> You can find it here: >>> http://cr.openjdk.java.net/~mhorie/8248188/webrev.02/ >>> >>> Changes in webrev.02 vs. webrev.01: >>> >>> * The method header for the intrinsic in the Base64 code has been >>> rewritten using the Javadoc style. The clarity of the comments has been >>> improved and some verbosity has been removed. There are no additional >>> functional changes to Base64.java. >>> >>> * The code needed to martial and check the intrinsic parameters has >>> been added, using the base64 encodeBlock intrinsic as a guideline. >>> >>> * A complete intrinsic implementation for Power9 and Power10 is >> included. >>> >>> * Adds some Power9 and Power10 assembler instructions needed by the >>> intrinsic which hadn't been defined before. >>> >>> The intrinsic implementation in this patch accelerates the decoding of >>> large blocks of base64 data by a factor of about 3.5X on Power9. >>> >>> I'm attaching two Java test cases I am using for testing and >>> benchmarking. The TestBase64_VB encodes and decodes randomly-sized >>> buffers of random data and checks that original data matches the >>> encoded-then-decoded data. TestBase64Errors encodes a 48K block of >>> random bytes, then corrupts each byte of the encoded data, one at a >>> time, checking to see if the decoder catches the illegal byte. >>> >>> Any comments/suggestions would be appreciated. >>> >>> Thanks, >>> >>> - Corey >>> >>> On 7/27/20 6:49 PM, Corey Ashford wrote: >>>> Michihiro Horie uploaded a new revision of the Base64 decodeBlock >>>> intrinsic API for me: >>>> >>>> http://cr.openjdk.java.net/~mhorie/8248188/webrev.01/ >>>> >>>> It has the following changes with respect to the original one posted: >>>> >>>> ?* In the event of encountering a non-base64 character, instead of >>>> having a separate error code of -1, the intrinsic can now just return >>>> either 0, or the number of data bytes produced up to the point where >> the >>>> illegal base64 character was encountered.? This reduces the number of >>>> special cases, and also provides a way to speed up the process of >>>> finding the bad character by the slower, pure-Java algorithm. >>>> >>>> ?* The isMIME boolean is removed from the API for two reasons: >>>> ?? - The current API is not sufficient to handle the isMIME case, >>>> because there isn't a strict relationship between the number of input >>>> bytes and the number of output bytes, because there can be an arbitrary >>>> number of non-base64 characters in the source. >>>> ?? - If an intrinsic only implements the (isMIME == false) case as ours >>>> does, it will always return 0 bytes processed, which will slightly slow >>>> down the normal path of processing an (isMIME == true) instantiation. >>>> ?? - We considered adding a separate hotspot candidate for the (isMIME >>>> == true) case, but since we don't have an intrinsic implementation to >>>> test that, we decided to leave it as a future optimization. >>>> >>>> Comments and suggestions are welcome.? Thanks for your consideration. >>>> >>>> - Corey >>>> >>>> On 6/23/20 6:23 PM, Michihiro Horie wrote: >>>>> Hi Corey, >>>>> >>>>> Following is the issue I created. >>>>> https://bugs.openjdk.java.net/browse/JDK-8248188 >>>>> >>>>> I will upload a webrev when you're ready as we talked in private. >>>>> >>>>> Best regards, >>>>> Michihiro >>>>> >>>>> Inactive hide details for "Corey Ashford" ---2020/06/24 >>>>> 09:40:10---Currently in java.util.Base64, there is a >>>>> HotSpotIntrinsicCa"Corey Ashford" ---2020/06/24 09:40:10---Currently >>>>> in java.util.Base64, there is a HotSpotIntrinsicCandidate and API for >>>>> encodeBlock, but no >>>>> >>>>> From: "Corey Ashford" >>>>> To: "hotspot-compiler-dev at openjdk.java.net" >>>>> , >>>>> "ppc-aix-port-dev at openjdk.java.net" >> dev at openjdk.java.net> >>>>> Cc: Michihiro Horie/Japan/IBM at IBMJP, Kazunori >>> Ogata/Japan/IBM at IBMJP, >>>>> joserz at br.ibm.com >>>>> Date: 2020/06/24 09:40 >>>>> Subject: RFR(S): [PATCH] Add HotSpotIntrinsicCandidate and API for >>>>> Base64 decoding >>>>> >>>>> ------------------------------------------------------------------------ >>>>> >>>>> >>>>> >>>>> Currently in java.util.Base64, there is a HotSpotIntrinsicCandidate and >>>>> API for encodeBlock, but none for decoding. ?This means that only >>>>> encoding gets acceleration from the underlying CPU's vector hardware. >>>>> >>>>> I'd like to propose adding a new intrinsic for decodeBlock. ?The >>>>> considerations I have for this new intrinsic's API: >>>>> >>>>> ??* Don't make any assumptions about the underlying capability of the >>>>> hardware. ?For example, do not impose any specific block size >>>>> granularity. >>>>> >>>>> ??* Don't assume the underlying intrinsic can handle isMIME or isURL >>>>> modes, but also let them decide if they will process the data regardless >>>>> of the settings of the two booleans. >>>>> >>>>> ??* Any remaining data that is not processed by the intrinsic will be >>>>> processed by the pure Java implementation. ?This allows the intrinsic to >>>>> process whatever block sizes it's good at without the complexity of >>>>> handling the end fragments. >>>>> >>>>> ??* If any illegal character is discovered in the decoding process, the >>>>> intrinsic will simply return -1, instead of requiring it to throw a >>>>> proper exception from the context of the intrinsic. ?In the event of >>>>> getting a -1 returned from the intrinsic, the Java Base64 library code >>>>> simply calls the pure Java implementation to have it find the error and >>>>> properly throw an exception. ?This is a performance trade-off in the >>>>> case of an error (which I expect to be very rare). >>>>> >>>>> ??* One thought I have for a further optimization (not implemented in >>>>> the current patch), is that when the intrinsic decides not to process a >>>>> block because of some combination of isURL and isMIME settings it >>>>> doesn't handle, it could return extra bits in the return code, encoded >>>>> as a negative number. ?For example: >>>>> >>>>> Illegal_Base64_char ? = 0b001; >>>>> isMIME_unsupported ? ?= 0b010; >>>>> isURL_unsupported ? ? = 0b100; >>>>> >>>>> These can be OR'd together as needed and then negated (flip the sign). >>>>> The Base64 library code could then cache these flags, so it will know >>>>> not to call the intrinsic again when another decodeBlock is requested >>>>> but with an unsupported mode. ?This will save the performance hit of >>>>> calling the intrinsic when it is guaranteed to fail. >>>>> >>>>> I've tested the attached patch with an actual intrinsic coded up for >>>>> Power9/Power10, but those runtime intrinsics and arch-specific patches >>>>> aren't attached today. ?I want to get some consensus on the >>>>> library-level intrinsic API first. >>>>> >>>>> Also attached is a simple test case to test that the new intrinsic API >>>>> doesn't break anything. >>>>> >>>>> I'm open to any comments about this. >>>>> >>>>> Thanks for your consideration, >>>>> >>>>> - Corey >>>>> >>>>> >>>>> Corey Ashford >>>>> IBM Systems, Linux Technology Center, OpenJDK team >>>>> cjashfor at us dot ibm dot com >>>>> [attachment "decodeBlock_api-20200623.patch" deleted by Michihiro >>>>> Horie/Japan/IBM] [attachment "TestBase64.java" deleted by Michihiro >>>>> Horie/Japan/IBM] >>>>> >>>>> >>>> > From LANCE.ANDERSEN at ORACLE.COM Wed Aug 26 16:55:30 2020 From: LANCE.ANDERSEN at ORACLE.COM (Lance Andersen) Date: Wed, 26 Aug 2020 12:55:30 -0400 Subject: RFR 8252265: Replace @exception with @throws java.util.logging package In-Reply-To: References: Message-ID: <3A3F2D05-2414-498C-BAF5-D4E67AA48453@ORACLE.COM> This looks ok overall. Not specific to your change but ? I did find it strange that FileHandler had a few methods which listed IllegalArgumentException multiple times (vs. including the possible scenarios for the exception as part of 1 @throws) as I have not noticed that elsewhere in the JDK but perhaps I have missed it. > On Aug 26, 2020, at 12:38 PM, Vipin Sharma wrote: > > Hi, > > Please review and sponsor the fix for replacing @exception with @throws in > java.util.logging package. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8252265 > Webrev: https://cr.openjdk.java.net/~vsharma/8252265/ > > It came up in the below discussion, starting this cleanup with > java.util.logging package. > https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068172.html > > Regards, > Vipin Best Lance ------------------ Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From daniel.fuchs at oracle.com Wed Aug 26 16:55:46 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 26 Aug 2020 17:55:46 +0100 Subject: RFR 8252265: Replace @exception with @throws java.util.logging package In-Reply-To: References: Message-ID: Hi Vipin, This looks good to me. Make sure to generate the API docs locally and eyeball them to double check that nothing is amiss. best regards, -- daniel On 26/08/2020 17:38, Vipin Sharma wrote: > Hi, > > Please review and sponsor the fix for replacing @exception with @throws in > java.util.logging package. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8252265 > Webrev: https://cr.openjdk.java.net/~vsharma/8252265/ > > It came up in the below discussion, starting this cleanup with > java.util.logging package. > https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068172.html > > Regards, > Vipin > From daniel.fuchs at oracle.com Wed Aug 26 17:01:42 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 26 Aug 2020 18:01:42 +0100 Subject: RFR 8252265: Replace @exception with @throws java.util.logging package In-Reply-To: <3A3F2D05-2414-498C-BAF5-D4E67AA48453@ORACLE.COM> References: <3A3F2D05-2414-498C-BAF5-D4E67AA48453@ORACLE.COM> Message-ID: <177047d0-3fe8-34ee-5c82-13527dda78c5@oracle.com> Hi Lance, On 26/08/2020 17:55, Lance Andersen wrote: > I did find it strange that FileHandler had a few methods which listed IllegalArgumentException multiple times (vs. including the possible scenarios for the exception as part of 1 @throws) as I have not noticed that elsewhere in the JDK but perhaps I have missed it. I wish that were the only odd thing in this API ;-) It also jumped at me and I double checked - javadoc handles that gracefully: https://docs.oracle.com/en/java/javase/14/docs/api/java.logging/java/util/logging/FileHandler.html#%3Cinit%3E(java.lang.String,int,int,boolean) best regards, -- daniel From naoto.sato at oracle.com Wed Aug 26 17:09:19 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Wed, 26 Aug 2020 10:09:19 -0700 Subject: [NEW BUG] NumberFormat.parse fails in some scenarios In-Reply-To: References: Message-ID: <2445839f-f82b-590e-52b6-78f2875de81e@oracle.com> Hi Christoph, The behavior you are observing is what is expected with the CLDR provider, as the currency format is exactly the one that CLDR has for the de locale: https://unicode-org.github.io/cldr-staging/charts/37/by_type/numbers.number_formatting_patterns.html#53687a25c19b6481 I'd recommend that either the code uses the "COMPAT" (same as "JRE") provider which keeps compatibility with JDK8 (which you have mentioned already), or replace the normal space with non-breaking space in the input string. Naoto On 8/26/20 7:18 AM, Christoph Dreis wrote: > Hi, > > A colleague of mine (filipe.silvestrim at innogames.com) approached me today that his code wasn?t working that converted a currency String into cents. > Apparently, the code worked with Java 8 while it didn?t with 11+. > > public class Main { > > public static void main(String[] args) throws IOException { > // System.setProperty("java.locale.providers", "JRE"); > System.out.println(getPriceInCents(Locale.GERMANY, "9,99 ?")); > } > > static int getPriceInCents(Locale locale, String price) { > try { > DecimalFormat format = (DecimalFormat) NumberFormat.getCurrencyInstance(locale); > Number number = format.parse(price); > return (int) (number.doubleValue() * 100); > } catch (ParseException e) { > // This should be thrown on JDK 9+ > System.out.println(e); > } > return 0; > } > > } > > After some digging I think this is caused by the changes done for JDK-8008577[1]. > When I change the java.locale.providers property to "JRE" for example, it works again. > > My investigations so far revealed that apparently the CLDR number pattern for the currency slightly differs. > > I created breakpoints in sun.util.locale.provider.NumberFormatProviderImpl::getInstance() to display some things: > > LocaleProviderAdapter adapter = LocaleProviderAdapter.forType(type); > String[] numberPatterns = adapter.getLocaleResources(override).getNumberPatterns(); > DecimalFormatSymbols symbols = DecimalFormatSymbols.getInstance(override); > int entry = (choice == INTEGERSTYLE) ? NUMBERSTYLE : choice; > DecimalFormat format = new DecimalFormat(numberPatterns[entry], symbols); > > // CLDR (type) > // #,##0.00?? (numberPatterns[entry]) > // [35,44,35,35,48,46,48,48,-62,-96,-62,-92] (numberPatterns[entry] in bytes) > > // > // JRE type > // #,##0.00 ?;-#,##0.00 ? (numberPatterns[entry]) > // [35,44,35,35,48,46,48,48,32,-62,-92,59,45,35,44,35,35,48,46,48,48,32,-62,-92] (numberPatterns[entry] in bytes) > > The JRE one includes the negative pattern, but the more interesting bit is that apparently the spacing differs here. > For JRE it seems to be a normal space (the 32), but for CLDR it's showing [-62, -96] which seems to be a non breaking space aka nbsp. > > Ultimately this leads to a check failing in DecimalFormat when parsing the string "9,99 ?" that obviously includes a normal space. > > if (gotPositive) { > // the regionMatches will return false because nbsp != space > gotPositive = text.regionMatches(position,positiveSuffix,0, > positiveSuffix.length()); > } > > Which itself leads to the following in our case: > > // fail if neither or both > if (gotPositive == gotNegative) { > parsePosition.errorIndex = position; > // We hit this part here which causes the parsing to fail > return false; > } > > > There are workarounds - e.g. by setting java.locale.providers as already mentioned or setting format.setPositiveSuffix(" ?"); to fix this particular case. > > Is this a bug or a feature or are we missing something? > > In case this is an actual bug we would appreciate a "reported-by" mentioning in an eventual fix. > > Thanks in advance. I do hope you can follow my thoughts in this email. > > [1] https://bugs.openjdk.java.net/browse/JDK-8008577 > > Cheers, > Christoph > > From gilles.m.duboscq at oracle.com Wed Aug 26 17:16:26 2020 From: gilles.m.duboscq at oracle.com (Gilles Duboscq) Date: Wed, 26 Aug 2020 19:16:26 +0200 Subject: RFR: JDK-8242451 : ensure semantics of non-capturing lambdas are preserved independent of execution mode In-Reply-To: References: <71c26ec5-7512-e5e0-349f-a488357677f1@oracle.com> Message-ID: Hi Mandy, Thanks for your review. I have added a test as you suggested and switched to `.descriptorString()`. https://cr.openjdk.java.net/~gdub/8242451/webrev.2/ Gilles On 25/08/2020 19:23, Mandy Chung wrote: > > > On 8/25/20 3:16 AM, Gilles Duboscq wrote: >>> >>> Since this patch caches a singleton instance in the generated class, it could apply to the eager initialization case as well and replace the current use of core reflection to new an instance except that the target of the returning callsite would always be the singleton object (the result of invoking the static getter method handle).? I wonder if there is any performance difference.? This is just a thought that we can file a JBS issue to follow up. >>> >>> Can you add a test case for this fix? >> >> I could write a test that generates different output depending on whether a singleton or a fresh instance is returned. >> Then i can compare the output when running with `jdk.internal.lambda.disableEagerInitialization` set to `true` and `false`. >> What is the recommended way of comparing 2 runs like that in jtreg? >> > > It can simply verify if the static field is only present when disableEagerInitialization is true; otherwise such static field should not exist. > >> I have updated the webrev: https://cr.openjdk.java.net/~gdub/8242451/webrev.1/ >> > > Looks okay to me. > > Nit: `Class::descriptorString` can be used instead of BytecodeDescriptor::unparse. > > 418 String lambdaTypeDescriptor = BytecodeDescriptor.unparse(invokedType.returnType()); > > > >> From your other mail: >> >>> Should this patch be a workaround to existing releases rather than the main line??? As Brian mentions, lambda proxy class may become inline class in valhalla repo (Roger has a patch already).?? The earlier fixing those programs the better. >> >> Indeed if we know this is landing in this cycle in the main repo there's no point with my fix. Could you point me at the issue number or mail thread where this patch is being discussed? >> > > As Brian replied, integration of valhalla is not imminent. JDK-8205668 tracks the work to make lambda proxies as inline classes for the valhalla repo (*not* main line). > > Mandy From fernando.guallini at oracle.com Wed Aug 26 17:17:41 2020 From: fernando.guallini at oracle.com (Fernando Guallini) Date: Wed, 26 Aug 2020 18:17:41 +0100 Subject: RFR: JDK-8249699: java/io/ByteArrayOutputStream/MaxCapacity.java should use @requires instead of @ignore Message-ID: <0BDB7CD8-B91B-4ED1-AB55-DDC6CE207EA7@oracle.com> Hi, Could I please get reviews and a sponsor for: webrev: http://cr.openjdk.java.net/~fguallini/8249699/webrev.00/ Testbug: https://bugs.openjdk.java.net/browse/JDK-8249699 The test was ignored due to ?huge memory requirements?, but with the jtreg current version, tests can be only run when system requirements are satisfied as opposed to being always excluded. It runs and passes in Mach5 Thanks -Fernando From sean.coffey at oracle.com Wed Aug 26 18:26:22 2020 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Wed, 26 Aug 2020 19:26:22 +0100 Subject: RFR: 8250968: Symlinks attributes not preserved when using jarsigner on zip files Message-ID: <957b3bd1-254a-647d-358b-e2dbfba6c6cf@oracle.com> This is a follow on from the recent 8218021 fix. The jarsigner tool removes symlink attribute data from zipfiles when signing them. jarsigner should preserve this data. The fix involves preserving the 16 bits associated with the file attributes (instead of the current 12). That's done in ZipFile. All other changes are just a refactor of the variable name. I haven't been able to automate a test for this since zipfs doesn't seem to support symlinks. Manual testing looks good. https://bugs.openjdk.java.net/browse/JDK-8250968 http://hmsjpse.uk.oracle.com/home/scoffey/ws/jdk-jdk/open/webrev.8250968/webrev/index.html regards, Sean. From sean.coffey at oracle.com Wed Aug 26 18:39:24 2020 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Wed, 26 Aug 2020 19:39:24 +0100 Subject: RFR: JDK-8249699: java/io/ByteArrayOutputStream/MaxCapacity.java should use @requires instead of @ignore In-Reply-To: <0BDB7CD8-B91B-4ED1-AB55-DDC6CE207EA7@oracle.com> References: <0BDB7CD8-B91B-4ED1-AB55-DDC6CE207EA7@oracle.com> Message-ID: test/jdk/java/util/Base64/TestEncodingDecodingLength.java is an example of another test using -Xmx8g. Do you want to push the os.maxMemory requirement up to 10g perhaps ? It might avoid border line resource failures. Also I think it might need a "sun.arch.data.model == "64" " requirement : @requires (sun.arch.data.model == "64" & os.maxMemory >= 10g) regards, Sean. On 26/08/2020 18:17, Fernando Guallini wrote: > Hi, > > Could I please get reviews and a sponsor for: > > webrev: http://cr.openjdk.java.net/~fguallini/8249699/webrev.00/ > Testbug: https://bugs.openjdk.java.net/browse/JDK-8249699 > > The test was ignored due to ?huge memory requirements?, but with the jtreg current version, tests can be only run when system requirements are satisfied as opposed to being always excluded. It runs and passes in Mach5 > > > Thanks > > -Fernando From mandy.chung at oracle.com Wed Aug 26 21:25:08 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 26 Aug 2020 14:25:08 -0700 Subject: RFR: JDK-8242451 : ensure semantics of non-capturing lambdas are preserved independent of execution mode In-Reply-To: References: <71c26ec5-7512-e5e0-349f-a488357677f1@oracle.com> Message-ID: <1fd04e0e-b2df-7c4a-bc53-600e2fc2b281@oracle.com> On 8/26/20 10:16 AM, Gilles Duboscq wrote: > Hi Mandy, > > Thanks for your review. > I have added a test as you suggested and switched to > `.descriptorString()`. > > https://cr.openjdk.java.net/~gdub/8242451/webrev.2/ Looks fine.??? This test is a javac test.? I'm including compiler-dev if anyone has any comment on it. Mandy From WEIJUN.WANG at ORACLE.COM Wed Aug 26 22:15:27 2020 From: WEIJUN.WANG at ORACLE.COM (Weijun Wang) Date: Wed, 26 Aug 2020 18:15:27 -0400 Subject: RFR: 8250968: Symlinks attributes not preserved when using jarsigner on zip files In-Reply-To: <957b3bd1-254a-647d-358b-e2dbfba6c6cf@oracle.com> References: <957b3bd1-254a-647d-358b-e2dbfba6c6cf@oracle.com> Message-ID: <68CBFB5B-0CAD-44DE-B0E8-92221BACE7C9@ORACLE.COM> Are you going to update the warning text or create a new one? Thanks, Max > On Aug 26, 2020, at 2:26 PM, Se?n Coffey wrote: > > This is a follow on from the recent 8218021 fix. The jarsigner tool removes symlink attribute data from zipfiles when signing them. jarsigner should preserve this data. The fix involves preserving the 16 bits associated with the file attributes (instead of the current 12). That's done in ZipFile. All other changes are just a refactor of the variable name. > > I haven't been able to automate a test for this since zipfs doesn't seem to support symlinks. Manual testing looks good. > > https://bugs.openjdk.java.net/browse/JDK-8250968 > http://hmsjpse.uk.oracle.com/home/scoffey/ws/jdk-jdk/open/webrev.8250968/webrev/index.html > > regards, > Sean. > From cjashfor at linux.ibm.com Wed Aug 26 22:17:25 2020 From: cjashfor at linux.ibm.com (Corey Ashford) Date: Wed, 26 Aug 2020 15:17:25 -0700 Subject: RFR(M): 8248188: [PATCH] Add HotSpotIntrinsicCandidate and API for Base64 decoding In-Reply-To: References: <11ca749f-3015-c004-aa6b-3194e1dfe4eb@linux.ibm.com> Message-ID: Hi Martin, Some inline responses below. On 8/26/20 8:26 AM, Doerr, Martin wrote: > Hi Corey, > > I should explain my comments regarding Base64.java better. > >> Let's be precise: "should process a multiple of four" => "must process a >> multiple of four" > Did you try to support non-multiple of 4 and this was intended as recommendation? > I think making it a requirement and simplifying the logic in decode0 is better. > Or what's the benefit of the recommendation? If I make a requirement, I feel decode0 should check that the requirement is met, and raise some kind of internal error if it isn't. That actually was my first implementation, but I received some comments during an internal review suggesting that I just "round down" the destination count to the closest multiple of 3 less than or equal to the returned value, rather than throw an internal exception which would confuse users. This "enforces" the rule, in some sense, without error handling. Do you have some thoughts about this? > >>> If any illegal base64 bytes are encountered in the source by the >>> intrinsic, the intrinsic can return a data length of zero or any >>> number of bytes before the place where the illegal base64 byte >>> was encountered. >> I think this has a drawback. Somebody may use a debugger and want to stop >> when throwing IllegalArgumentException. He should see the position which >> matches the Java implementation.kkkk > This is probably hard to understand. Let me try to explain it by example: > 1. 80 Bytes get processed by the intrinsic and 60 Bytes written to the destination array. > 2. The intrinsic sees an illegal base64 Byte and it returns 12 which is allowed by your specification. > 3. The compiled method containing the intrinsic hits a safepoint (e.g. in the large while loop in decodeBlockSlow). > 4. A JVMTI agent (debugger) reads dp and dst. > 5. The person using the debugger gets angry because more bytes than dp were written into dst. The JVM didn't follow the specified behavior. > > I guess we can and should avoid it by specifying that the intrinsic needs to return the dp value matching the number of Bytes written. That's an interesting point. I will change the specification, and the intrinsic implementation. Right now the Power9/10 intrinsic returns 0 when any illegal character is discovered, but I've been thinking about returning the number of bytes already written, which will allow decodeBlockSlow to more quickly find the offending character. This provides another good reason to make that change. > > Best regards, > Martin > > >> -----Original Message----- >> From: Doerr, Martin >> Sent: Dienstag, 25. August 2020 15:38 >> To: Corey Ashford ; Michihiro Horie >> >> Cc: hotspot-compiler-dev at openjdk.java.net; core-libs-dev > dev at openjdk.java.net>; Kazunori Ogata ; >> joserz at br.ibm.com >> Subject: RE: RFR(M): 8248188: [PATCH] Add HotSpotIntrinsicCandidate and >> API for Base64 decoding >> >> Hi Corey, >> >> thanks for proposing this change. I have comments and suggestions >> regarding various files. >> >> >> Base64.java >> >> This is the only file which needs another review from core-libs-dev. >> First of all, I like the idea to use a HotSpotIntrinsicCandidate which can >> consume as many bytes as the implementation wants. >> >> Comment before decodeBlock: >> Let's be precise: "should process a multiple of four" => "must process a >> multiple of four" >> >>> If any illegal base64 bytes are encountered in the source by the >>> intrinsic, the intrinsic can return a data length of zero or any >>> number of bytes before the place where the illegal base64 byte >>> was encountered. >> I think this has a drawback. Somebody may use a debugger and want to stop >> when throwing IllegalArgumentException. He should see the position which >> matches the Java implementation. >> >> Please note that the comment indentation differs from other comments. Will fix. >> >> decode0: Final "else" after return is redundant. Will fix. >> >> >> stubGenerator_ppc.cpp >> >> "__vector" breaks AIX build! >> Does it work on Big Endian linux with old gcc (we require 7.3.1, now)? >> Please either support Big Endian properly or #ifdef it out. I have been compiling with only Advance Toolchain 13, which is 9.3.1, and only on Linux. It will not work with big endian, so it won't work on AIX, however obviously it shouldn't break the AIX build, so I will address that. There's code to set UseBASE64Intrinsics to false on big endian, but you're right -- I should ifdef all of the intrinsic code for little endian for now. Getting it to work on big endian / AIX shouldn't be difficult, but it's not in my scope of work at the moment. I will double check that everything compiles and runs properly with gcc 7.3.1. >> What exactly does it (do) on linux? It's an arch-specific type that's 16 bytes in size and aligned on a 16-byte boundary. >> I remember that we had tried such prefixes but were not satisfied. I think it >> didn't enforce 16 Byte alignment if I remember correctly. I will use __attribute__ ((align(16))) instead of __vector, and make them arrays of 16 unsigned char. >> >> Attention: C2 does no longer convert int/bool to 64 bit values (since JDK- >> 8086069). So the argument registers for offset, length and isURL may contain >> garbage in the higher bits. Wow, that's good to know! I will mask off the incoming values. >> >> You may want to use load_const_optimized which produces shorter code. Will fix. >> >> You may want to use __ align(32) to align unrolled_loop_start. Will fix. >> >> I'll review the algorithm in detail when I find more time. >> >> >> assembler_ppc.hpp >> assembler_ppc.inline.hpp >> vm_version_ppc.cpp >> vm_version_ppc.hpp >> Please rebase. Parts of the change were pushed as part of 8248190: Enable >> Power10 system and implement new byte-reverse instructions Will do. >> >> >> vmSymbols.hpp >> Indentation looks odd at the end. I was following what was done for encodeBlock, but it appears encodeBlock's style isn't what is used for the other intrinsics. I will correct decodeBlock to use the prevailing style. Another patch should be added (not part of this webrev) to correct encodeBlock's style. >> >> >> library_call.cpp >> Good. Indentation style of the call parameters differs from encodeBlock. Will fix. >> >> >> runtime.cpp >> Good. >> >> >> aotCodeHeap.cpp >> vmSymbols.cpp >> shenandoahSupport.cpp >> vmStructs_jvmci.cpp >> shenandoahSupport.cpp >> escape.cpp >> runtime.hpp >> stubRoutines.cpp >> stubRoutines.hpp >> vmStructs.cpp >> Good and trivial. >> >> >> Tests: >> I think we should have JTREG tests to check for regressions in the future. Ah, this is another thing I didn't know about. I will make some regression tests. Thanks for your time on this. As you can tell, I'm inexperienced in writing openjdk code, so your patience and careful review is really appreciated. - Corey From Roger.Riggs at oracle.com Thu Aug 27 00:08:58 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Wed, 26 Aug 2020 20:08:58 -0400 Subject: RFR 8251989: Hex encoder and decoder utility In-Reply-To: References: <6743da26-9bae-e868-d242-b359aeec3d21@oracle.com> Message-ID: <71ec5a20-27a3-c177-0959-f2fbec098d9f@oracle.com> Paul, Thanks for the review, there are a number of updates pending. On 8/24/20 4:55 PM, Paul Sandoz wrote: > Hi Roger, > > A nice minimal addition. I agree with Mark?s naming suggestion. > > - Use appropriate code convention for static field names. > > - The encoder comes with two encoding production signatures, that returning a String and that for encoding into a StringBuilder. > The decoder comes with just one such production signature, returning byte[]. I imagine one might need a decode into a byte[] from a given offset. > Something to consider later if need be? The initial step covers the existing cases in the JDK and tests. The API and error handling get more complex when trying to decode into a buffer that's been allocated already. > > - More generally if there is pressure to add ByteBuffer accepting signatures we can punt and if necessary use MemorySegment once it exits incubating. Easy enough to add when conditions are suitable. > >> 36 * Utilities to encode bytes to hex strings and decode hex strings to bytes. > > I recommend using ?hexadecimal (hex)? for the first use, and ?hex? thereafter. ok > > >> 48 * For example, for a comma (",") separated format with a prefix and suffix > > Use an @apiNote > ok >> 63 * Hex.Encoder encodeFingerprint = Hex.encoder(":", "", "",true); > > Missing space after last comma. fixed > > >> 60 * RFC 4752 defines a fingerprint of byte values that uses the delimiter colon {@code ':'}. >> 61 * The encoder and decoder for RFC 2752 are: > > s/2752/4752 ? typo fixed > > >> 254 public StringBuilder encodeHexPair(StringBuilder sb, int value) { > > This can be specified as behaving as if sb.append(toHexPair(value)), and also say @see #toHexPair. ok Thanks, Roger > > Paul. > >> On Aug 19, 2020, at 2:14 PM, Roger Riggs wrote: >> >> Please review a java.util.Hex API to encode and decode hexadecimal strings to and from byte arrays. >> >> Within the JDK and JDK tests there are multiple implementations to encode and decode >> hexadecimal strings to byte arrays. Hex encoders and decoders support >> upper or lower case hexadecimal characters, delimiters, prefix, and suffix. >> The API is modeled after the java.util.Base64 API providing static factories, >> immutable threadsafe instances with methods to encode to and decode from >> string and StringBuilder. >> >> JavaDoc: >> http://cr.openjdk.java.net/~rriggs/hex-javadoc/java.base/java/util/Hex.html >> >> Webrev for Hex encoder and decoder: >> http://cr.openjdk.java.net/~rriggs/webrev-hex-encoder-8251989 >> >> Webrev for applying to java.security: >> http://cr.openjdk.java.net/~rriggs/webrev-hex-security-8252055 >> >> CSR: >> https://bugs.openjdk.java.net/browse/JDK-8251991 >> >> Issue for API and a few uses: >> https://bugs.openjdk.java.net/browse/JDK-8251989 >> >> Issue for Use in java.security and tests: >> https://bugs.openjdk.java.net/browse/JDK-8252055 >> From Roger.Riggs at oracle.com Thu Aug 27 00:27:53 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Wed, 26 Aug 2020 20:27:53 -0400 Subject: RFR 8251989: Hex encoder and decoder utility In-Reply-To: <20200825170159.892970153@eggemoggin.niobe.net> References: <6743da26-9bae-e868-d242-b359aeec3d21@oracle.com> <20200820121012.674081461@eggemoggin.niobe.net> <20200825170159.892970153@eggemoggin.niobe.net> Message-ID: <57a7d45d-6ff1-f6c2-c7e5-253a6ab98a08@oracle.com> Hi Mark, The updates to the API based on the comments received are in the works. On 8/25/20 8:01 PM, mark.reinhold at oracle.com wrote: > 2020/8/20 13:59:39 -0700, roger.riggs at oracle.com: >> On 8/20/20 3:10 PM, mark.reinhold at oracle.com wrote: >>> ... >>> >>> A few comments: >>> >>> - Why do the short-form `encoder` factory methods return encoders that >>> produce upper-case hex strings? `Integer::toHexString` and other >>> existing `toHex` methods return lower-case hex strings. That?s also >>> what you get from common Unix CLI tools (e.g., `od -tx1`). >>> >>> Please consider changing these methods to return lower-case encoders. >> It's (almost) a toss up and easy to change; many of the existing uses >> produce uppercase. > Please change the default to lower case. It?s what people are going to > expect. ok > >> Perhaps the short form no-arg should be replaced with a short form >> constructor that >> takes true/false, so it is explicit at the use site or put the case in >> the name. >> encodeToUpper(), encodeToLower(). >> (A boolean parameter is not very informative, a enum would be better but >> perhaps a bit heavyweight). > Either a boolean or an enum parameter would be massive overkill. See the use cases (in the JDK and tests) below. > >>> - Is it worth having static `Hex.encode(byte[])` and >>> `Hex.decode(CharSequence)` convenience methods for the simplest >>> cases? >> There was some discussion of that but it idea was to minimize the >> surface area. > Sorry, where was that discussion? I can?t find it in the core-libs-dev > archive. Several colleagues suggested a minimal API to start and to add convenience APIs as the need is apparent. > >>> - [Warning: Bikeshed] The verbs ?encode? and ?decode? seem unfortunate. >>> >>> Over in `java.nio.charsets` we have encoders that transform >>> characters to bytes, and decoders that transform bytes to characters. >>> The coded thing is the bytes; the uncoded thing is the characters. >>> >>> In `java.util` we already have the `Base64` class, which provides >>> encoders that transform bytes to characters, and decoders that >>> transform characters to bytes. The coded thing is the characters; >>> the uncoded thing is the bytes. >>> >>> The use of ?encode? and ?decode? in `Base64` was likely inspired by >>> the fact that the format has been known as ?base 64 encoding? for >>> decades, having originated as a hack for transporting non-ASCII data >>> via SMTP. Developers looking to do base-64 operations will, thus, >>> expect this terminology. >> The bias of encoding vs decoding terminology is subtle, based the 'native' >> form of the data. For the Charset classes, the native form of the data >> is characters, >> and the encoded form is bytes. For Base64, the native form of the data >> is bytes, >> and the encoded form is Base64 lines. > Indeed -- and my point is that, in this case, encoding and decoding > might not be the best concepts for API discoverability. (But, maybe > they are.) > > It?s technically true that this API does encoding and decoding, > whichever way you look at it. What I wonder about are the typical use > cases. Will the primary uses of this API be to encode bytes into > characters so that they can be transported via a medium that can only > handle characters, and then decode the characters back into bytes on the > other end? Or will the primary uses be to format bytes into a readable > form, and likewise parse arbitrary-length hex strings into bytes? Within the JDK, there are several different cases. The network classes encode non-printable characters to meet protocol requirements. Most of the other uses present binary information in exceptions and messages, so it is one-way bytes to Strings.? For example, hashes on Modules that don't match in the Resolver. In the security? classes, most of the use is for debugging support, to report unexpected differences.? The parsing of Strings to binary is mostly used to encode binary objects into source code, for example certificates for testing. Uppercase is more frequently used than lower case, in the security tests and debugging support of 81 uses of Hex.formaters, 52 use uppercase characters. > >>> Here you?re proposing that the `Hex` class follow the `Base64` class. >>> Consistency with existing nearby APIs is a worthy goal. If I were >>> just looking to convert a byte array into a readable hex string, >>> however, I?d probably want to ?format? it rather than ?encode? it, >>> something like `String.format("%x")` on steroids. Likewise, if I >>> were looking to convert a hex string into bytes then I?d want to >>> ?parse? it rather than ?decode? it, i.e., `Integer::parseInt` on >>> steroids. >>> >>> If you were to rename the nested classes to `Hex.Formatter` and >>> `Hex.Parser`, and rename all methods accordingly, then this API would >>> be inconsistent with the nearby `Base64` but likely more consistent >>> with developer expectations. >>> >>> (`Hex` is already inconsistent with `Base64` in that it doesn?t >>> prefix the names of its factory methods with `get`, which is a good >>> thing.) >> The question is at what level of control does "encoding" become formatting. >> There are very few formatting features in the API, no control of leading >> zeros, no control >> over indicental whitespace, and no control over width. Similarly with >> parsing, >> what flexibility does parsing imply that decoding does not, is >> whitespace ignored, line endings/joins, etc. > Optional support for prefix, suffix, and delimiter characters in both > operations, and support for both lower- and upper-case characters on > input, sure look like formatting and parsing to me. ok Thanks, Roger > > - Mark From Roger.Riggs at oracle.com Thu Aug 27 01:34:47 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Wed, 26 Aug 2020 21:34:47 -0400 Subject: RFR 8251989: Hex formatter and parser utility Message-ID: <6378b60b-7a45-d8b0-5ebd-3d3bf9144401@oracle.com> Please review updates to the formatting and parsing API based on the last round of comments. There are many changes, so it may be useful to read it as a fresh draft. ?- Rename classes: Encoder -> Formatter; Decoder -> Parser ?- Rename methods: encode -> format; decode -> parse, etc. ?- Rename factory methods to match ?- Added a factory method and re-arrange arguments to make it more convenient ?? to create uppercase formatters based on the existing uses. ?- The implementation has been updated based on the suggestions and API changes The webrev for applying the API to the security classes will be updated when the API settles down. JavaDoc: http://cr.openjdk.java.net/~rriggs/hex-formatter/java.base/java/util/Hex.html Webrev: ? http://cr.openjdk.java.net/~rriggs/webrev-hex-formatter-8251989/ CSR: https://bugs.openjdk.java.net/browse/JDK-8251991 p.s. The previous (encoder/decoder) javadoc has been renamed to: ?? http://cr.openjdk.java.net/~rriggs/hex-encoder-javadoc/ From sundararajan.athijegannathan at oracle.com Thu Aug 27 02:30:50 2020 From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com) Date: Thu, 27 Aug 2020 08:00:50 +0530 Subject: Review request for JDK-8251538: Modernize and lint Dynalink code In-Reply-To: References: Message-ID: <28bd3fc8-13c0-25c4-9230-5bbd997c6e5e@oracle.com> Looks good. Thanks for doing this cleanup! -Sundar On 21/08/20 2:10 am, Attila Szegedi wrote: > Following up on the previous e-mail, here?s the modernization and linting work on the existing Dynalink codebase: > > Please review JDK-8251538 "Modernize and lint Dynalink code" at for > > The Jira issue has a full enumeration of kinds of changes I did here; they?re mostly all source text removals :-) > > Oh, BTW, I really got an inspiration for adding ?public static ClassValue computingValue(Function, T> f)? method to ClassValue class, similar to Comparator.comparing. It?d allow lambdifying class values. (If you?re listening, John :-) I suspect it being in java.lang would be a JCP-level change so I kind-of don?t want to take it upon myself?) > > Thanks, > Attila. > From sean.coffey at oracle.com Thu Aug 27 06:42:43 2020 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Thu, 27 Aug 2020 07:42:43 +0100 Subject: RFR: 8250968: Symlinks attributes not preserved when using jarsigner on zip files In-Reply-To: <68CBFB5B-0CAD-44DE-B0E8-92221BACE7C9@ORACLE.COM> References: <957b3bd1-254a-647d-358b-e2dbfba6c6cf@oracle.com> <68CBFB5B-0CAD-44DE-B0E8-92221BACE7C9@ORACLE.COM> Message-ID: <99b55d3f-533e-ac91-89bd-1ce8f3cbd6db@oracle.com> Hi Max, I looked at updating the warning string but figured that it might have been of no interest to end users. How about this edit then ? +??????? {"posix.attributes.detected", "POSIX file permission attributes detected. These attributes are ignored when signing and are not protected by the signature."}, >> replace with: +??????? {"extra.attributes.detected", "POSIX file permission/symlink attributes detected. These attributes are ignored when signing and are not protected by the signature."}, regards, Sean. On 26/08/2020 23:15, Weijun Wang wrote: > Are you going to update the warning text or create a new one? > > Thanks, > Max > >> On Aug 26, 2020, at 2:26 PM, Se?n Coffey wrote: >> >> This is a follow on from the recent 8218021 fix. The jarsigner tool removes symlink attribute data from zipfiles when signing them. jarsigner should preserve this data. The fix involves preserving the 16 bits associated with the file attributes (instead of the current 12). That's done in ZipFile. All other changes are just a refactor of the variable name. >> >> I haven't been able to automate a test for this since zipfs doesn't seem to support symlinks. Manual testing looks good. >> >> https://bugs.openjdk.java.net/browse/JDK-8250968 >> http://hmsjpse.uk.oracle.com/home/scoffey/ws/jdk-jdk/open/webrev.8250968/webrev/index.html >> >> regards, >> Sean. >> From sean.coffey at oracle.com Thu Aug 27 07:26:49 2020 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Thu, 27 Aug 2020 08:26:49 +0100 Subject: RFR: 8250968: Symlinks attributes not preserved when using jarsigner on zip files In-Reply-To: <99b55d3f-533e-ac91-89bd-1ce8f3cbd6db@oracle.com> References: <957b3bd1-254a-647d-358b-e2dbfba6c6cf@oracle.com> <68CBFB5B-0CAD-44DE-B0E8-92221BACE7C9@ORACLE.COM> <99b55d3f-533e-ac91-89bd-1ce8f3cbd6db@oracle.com> Message-ID: updated webrev: http://cr.openjdk.java.net/~coffeys/webrev.8250968.v2/webrev/ regards, Sean. On 27/08/2020 07:42, Se?n Coffey wrote: > Hi Max, > > I looked at updating the warning string but figured that it might have > been of no interest to end users. How about this edit then ? > > +??????? {"posix.attributes.detected", "POSIX file permission > attributes detected. These attributes are ignored when signing and are > not protected by the signature."}, > > >> replace with: > +??????? {"extra.attributes.detected", "POSIX file permission/symlink > attributes detected. These attributes are ignored when signing and are > not protected by the signature."}, > > regards, > Sean. > > On 26/08/2020 23:15, Weijun Wang wrote: >> Are you going to update the warning text or create a new one? >> >> Thanks, >> Max >> >>> On Aug 26, 2020, at 2:26 PM, Se?n Coffey >>> wrote: >>> >>> This is a follow on from the recent 8218021 fix. The jarsigner tool >>> removes symlink attribute data from zipfiles when signing them. >>> jarsigner should preserve this data. The fix involves preserving the >>> 16 bits associated with the file attributes (instead of the current >>> 12). That's done in ZipFile. All other changes are just a refactor >>> of the variable name. >>> >>> I haven't been able to automate a test for this since zipfs doesn't >>> seem to support symlinks. Manual testing looks good. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8250968 >>> http://hmsjpse.uk.oracle.com/home/scoffey/ws/jdk-jdk/open/webrev.8250968/webrev/index.html >>> >>> >>> regards, >>> Sean. >>> From julia.boes at oracle.com Thu Aug 27 11:08:07 2020 From: julia.boes at oracle.com (Julia Boes) Date: Thu, 27 Aug 2020 12:08:07 +0100 Subject: RFR 8252265: Replace @exception with @throws java.util.logging package In-Reply-To: References: Message-ID: Hi Vipin, I'm happy to sponsor this and any related changes. Let me know if you need help with anything. Best, Julia On 26/08/2020 17:55, Daniel Fuchs wrote: > Hi Vipin, > > This looks good to me. > Make sure to generate the API docs locally and eyeball them to > double check that nothing is amiss. > > best regards, > > -- daniel > > On 26/08/2020 17:38, Vipin Sharma wrote: >> Hi, >> >> Please review and sponsor the fix for replacing @exception with >> @throws in >> java.util.logging package. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8252265 >> Webrev: https://cr.openjdk.java.net/~vsharma/8252265/ >> >> It came up in the below discussion, starting this cleanup with >> java.util.logging package. >> https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-August/068172.html >> >> >> Regards, >> Vipin >> > From aleksej.efimov at oracle.com Thu Aug 27 13:17:53 2020 From: aleksej.efimov at oracle.com (Aleks Efimov) Date: Thu, 27 Aug 2020 14:17:53 +0100 Subject: [16] RFR: 8251182: Fix "no comment" warnings in java.naming In-Reply-To: <4CDC2073-B719-495A-B68B-3ECB38E64054@ORACLE.COM> References: <3a4bb2aa-0f87-70bd-3351-3af6b11c1ec5@oracle.com> <4CDC2073-B719-495A-B68B-3ECB38E64054@ORACLE.COM> Message-ID: <8e447a71-6d44-b551-375c-2700a5c5d6ca@oracle.com> Roger, Lance, Daniel, Thanks for your reviews. With Best Regards, Aleksei On 26/08/2020 16:59, Lance Andersen wrote: > +1 > >> On Aug 26, 2020, at 7:46 AM, Aleks Efimov > > wrote: >> >> Hi Roger, Lance, Daniel, >> >> Thanks for your reviews and comments! >> New webrev with suggested modifications can be viewed at this location: >> http://cr.openjdk.java.net/~aefimov/8251182/01 >> >> The list of modifications: >> - Added @java.io.Serial annotation to 'serialVersionUID' declarations >> - Removed trailing periods from @param/@throws >> - Added generic first line comments to writeObject/readObject methods >> >> The new version of patch and webrev files have been uploaded to the >> CSR issue. >> >> Kind Regards, >> Aleksei >> >> On 25/08/2020 20:23, Daniel Fuchs wrote: >>> Hi Aleksei, >>> >>> LGTM. >>> >>> You could also add the @Seial annotation to serialVersionUID >>> declarations. >>> >>> best regards, >>> >>> -- daniel >>> >>> On 25/08/2020 16:03, Aleks Efimov wrote: >>>> Hi, >>>> >>>> The documentation of classes from "java.naming" module needs to be >>>> improved to resolve javadoc -Xdoclint "no comment" warnings. >>>> >>>> JBS issue: https://bugs.openjdk.java.net/browse/JDK-8251182 >>>> Webrev: http://cr.openjdk.java.net/~aefimov/8251182/00/index.html >>>> CSR: https://bugs.openjdk.java.net/browse/JDK-8252310 >>>> >>>> With Best Regards, >>>> Aleksei >>> >> > > > Best > Lance > ------------------ > > > > Lance Andersen| Principal?Member of Technical Staff |?+1.781.442.2037 > Oracle?Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > > From chris.hegarty at oracle.com Thu Aug 27 13:20:55 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 27 Aug 2020 14:20:55 +0100 Subject: RFR 8251989: Hex formatter and parser utility In-Reply-To: <6378b60b-7a45-d8b0-5ebd-3d3bf9144401@oracle.com> References: <6378b60b-7a45-d8b0-5ebd-3d3bf9144401@oracle.com> Message-ID: Roger, > On 27 Aug 2020, at 02:34, Roger Riggs wrote: > > Please review updates to the formatting and parsing API based on the last round of comments. > There are many changes, so it may be useful to read it as a fresh draft. > > - Rename classes: Encoder -> Formatter; Decoder -> Parser > - Rename methods: encode -> format; decode -> parse, etc. > - Rename factory methods to match > - Added a factory method and re-arrange arguments to make it more convenient > to create uppercase formatters based on the existing uses. > - The implementation has been updated based on the suggestions and API changes > > The webrev for applying the API to the security classes will be updated when the API settles down. > > JavaDoc: > http://cr.openjdk.java.net/~rriggs/hex-formatter/java.base/java/util/Hex.html 1. "Hex.Formatter formats bytes to a hex string or appends to StringBuilder.? I wonder if this could be a bit more descriptive. What about something like: ?Hex.Formatter transforms sequences of bytes into a formatted hexadecimal string.? What is a formatted hexadecimal string? ?A formatted hexadecimal string consists solely of; an optional prefix, a pair of hexadecimal characters ( THESE ARE ALREADY DEFINED ELSEWHERE), an optional delimiter, and an optional suffix.? Or some such wording. 2. I like the move to formatter/parser, but I now find myself looking for an encoder/decoder ;-) It might be obvious, but could be worth calling out explicitly, e.g. ?A formatter with an empty prefix, suffix, and delimiter, provides unformatted (plain) encoder functionality". Similar for parser. Maybe this is what ?canonical? is referring to? If so, I think it would be best to define it somewhere. 3. "The Hex class consists solely of factory methods for hexadecimal (hex) formatters and parsers.? It could be worth generalising the class-level description now, so as to allow space for additional convenience methods in the future. ( A while back I suggested not adding convenience methods yet, but I see that Mark has suggested a couple of key conveniences ) -Chris. From WEIJUN.WANG at ORACLE.COM Thu Aug 27 13:45:40 2020 From: WEIJUN.WANG at ORACLE.COM (Weijun Wang) Date: Thu, 27 Aug 2020 09:45:40 -0400 Subject: RFR: 8250968: Symlinks attributes not preserved when using jarsigner on zip files In-Reply-To: References: <957b3bd1-254a-647d-358b-e2dbfba6c6cf@oracle.com> <68CBFB5B-0CAD-44DE-B0E8-92221BACE7C9@ORACLE.COM> <99b55d3f-533e-ac91-89bd-1ce8f3cbd6db@oracle.com> Message-ID: <940A8A93-BB70-48CD-B1DE-0E9EC17E2791@ORACLE.COM> I?m OK with using one warning, but prefer it to a little more formal like "POSIX file permission and/or symlink attributes detected??. One nit in ZipFile.java: 1098 // only set posix perms value via ZipEntry constructor for now 1099 @Override 1100 public int getExtraAttributes(ZipEntry ze) { Maybe you can just remove the comment. Do you also want to rename the ?posixPermsDetected" field and loacl variable ?perms? in JarSigner.java? I?m not sure about the test but if zipfs is able to keep permissions inside a zip file then that POSIX byte (or whatever it?s named) is already there and we can modify it to include more bits. No other comment. Thanks, Max > On Aug 27, 2020, at 3:26 AM, Se?n Coffey wrote: > > updated webrev: > http://cr.openjdk.java.net/~coffeys/webrev.8250968.v2/webrev/ > > regards, > Sean. > > On 27/08/2020 07:42, Se?n Coffey wrote: >> Hi Max, >> >> I looked at updating the warning string but figured that it might have been of no interest to end users. How about this edit then ? >> >> + {"posix.attributes.detected", "POSIX file permission attributes detected. These attributes are ignored when signing and are not protected by the signature."}, >> >> >> replace with: >> + {"extra.attributes.detected", "POSIX file permission/symlink attributes detected. These attributes are ignored when signing and are not protected by the signature."}, >> >> regards, >> Sean. >> >> On 26/08/2020 23:15, Weijun Wang wrote: >>> Are you going to update the warning text or create a new one? >>> >>> Thanks, >>> Max >>> >>>> On Aug 26, 2020, at 2:26 PM, Se?n Coffey wrote: >>>> >>>> This is a follow on from the recent 8218021 fix. The jarsigner tool removes symlink attribute data from zipfiles when signing them. jarsigner should preserve this data. The fix involves preserving the 16 bits associated with the file attributes (instead of the current 12). That's done in ZipFile. All other changes are just a refactor of the variable name. >>>> >>>> I haven't been able to automate a test for this since zipfs doesn't seem to support symlinks. Manual testing looks good. >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8250968 >>>> http://hmsjpse.uk.oracle.com/home/scoffey/ws/jdk-jdk/open/webrev.8250968/webrev/index.html >>>> >>>> regards, >>>> Sean. >>>> From sean.coffey at oracle.com Thu Aug 27 14:36:22 2020 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Thu, 27 Aug 2020 15:36:22 +0100 Subject: RFR: 8250968: Symlinks attributes not preserved when using jarsigner on zip files In-Reply-To: <940A8A93-BB70-48CD-B1DE-0E9EC17E2791@ORACLE.COM> References: <957b3bd1-254a-647d-358b-e2dbfba6c6cf@oracle.com> <68CBFB5B-0CAD-44DE-B0E8-92221BACE7C9@ORACLE.COM> <99b55d3f-533e-ac91-89bd-1ce8f3cbd6db@oracle.com> <940A8A93-BB70-48CD-B1DE-0E9EC17E2791@ORACLE.COM> Message-ID: <3b9c3aa9-df9b-5ae3-2301-2e08c8e9f769@oracle.com> Thanks for the review Max. Comments inline.. On 27/08/2020 14:45, Weijun Wang wrote: > I?m OK with using one warning, but prefer it to a little more formal like "POSIX file permission and/or symlink attributes detected??. > > One nit in ZipFile.java: > > 1098 // only set posix perms value via ZipEntry constructor for now > 1099 @Override > 1100 public int getExtraAttributes(ZipEntry ze) { > > Maybe you can just remove the comment. > > Do you also want to rename the ?posixPermsDetected" field and loacl variable ?perms? in JarSigner.java? Good points. Edits made. http://cr.openjdk.java.net/~coffeys/webrev.8250968.v3/webrev/ > > I?m not sure about the test but if zipfs is able to keep permissions inside a zip file then that POSIX byte (or whatever it?s named) is already there and we can modify it to include more bits. Looks like it was a conscious design decision to only allow recording of POSIX permission bits for this field (& 0xFFF). I don't see anything about symlink support in zipfs docs. regards, Sean. > > No other comment. > > Thanks, > Max > > >> On Aug 27, 2020, at 3:26 AM, Se?n Coffey wrote: >> >> updated webrev: >> http://cr.openjdk.java.net/~coffeys/webrev.8250968.v2/webrev/ >> >> regards, >> Sean. >> >> On 27/08/2020 07:42, Se?n Coffey wrote: >>> Hi Max, >>> >>> I looked at updating the warning string but figured that it might have been of no interest to end users. How about this edit then ? >>> >>> + {"posix.attributes.detected", "POSIX file permission attributes detected. These attributes are ignored when signing and are not protected by the signature."}, >>> >>>>> replace with: >>> + {"extra.attributes.detected", "POSIX file permission/symlink attributes detected. These attributes are ignored when signing and are not protected by the signature."}, >>> >>> regards, >>> Sean. >>> >>> On 26/08/2020 23:15, Weijun Wang wrote: >>>> Are you going to update the warning text or create a new one? >>>> >>>> Thanks, >>>> Max >>>> >>>>> On Aug 26, 2020, at 2:26 PM, Se?n Coffey wrote: >>>>> >>>>> This is a follow on from the recent 8218021 fix. The jarsigner tool removes symlink attribute data from zipfiles when signing them. jarsigner should preserve this data. The fix involves preserving the 16 bits associated with the file attributes (instead of the current 12). That's done in ZipFile. All other changes are just a refactor of the variable name. >>>>> >>>>> I haven't been able to automate a test for this since zipfs doesn't seem to support symlinks. Manual testing looks good. >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8250968 >>>>> http://hmsjpse.uk.oracle.com/home/scoffey/ws/jdk-jdk/open/webrev.8250968/webrev/index.html >>>>> >>>>> regards, >>>>> Sean. >>>>> From douglas.surber at oracle.com Thu Aug 27 14:37:56 2020 From: douglas.surber at oracle.com (Douglas Surber) Date: Thu, 27 Aug 2020 07:37:56 -0700 Subject: RFR 8251989: Hex formatter and parser utility In-Reply-To: References: Message-ID: <1BAC4282-314D-4009-B2F7-28B6FDACBA28@oracle.com> The meaning of prefix and suffix is not specified in formatter?(boolean uppercase, String delimiter,String prefix, String suffix). It isn't specified whether they precede and follow the entire formatted value or each byte. The class comment clarifies but I shouldn't have to go there to discover this. I was surprised at the meaning for prefix and suffix. The seem pointless to me. It is trivial to enclose the entire formatted value with a prefix and suffix without using these arguments. If they were prefix and suffix for each individual byte, that would be much more useful. For example, how can I format a byte sequence like this? 0x00 0x01 0x02 0x0d 0x0e 0x0f format(false, " 0x", "0x", "") doesn't work because an empty byte array would be 0x instead of an empty string. Douglas > On Aug 27, 2020, at 4:55 AM, core-libs-dev-request at openjdk.java.net wrote: > > Message: 1 > Date: Wed, 26 Aug 2020 21:34:47 -0400 > From: Roger Riggs > > To: core-libs-dev > > Subject: RFR 8251989: Hex formatter and parser utility > Message-ID: <6378b60b-7a45-d8b0-5ebd-3d3bf9144401 at oracle.com > > Content-Type: text/plain; charset=utf-8; format=flowed > > Please review updates to the formatting and parsing API based on the > last round of comments. > There are many changes, so it may be useful to read it as a fresh draft. > > ?- Rename classes: Encoder -> Formatter; Decoder -> Parser > ?- Rename methods: encode -> format; decode -> parse, etc. > ?- Rename factory methods to match > ?- Added a factory method and re-arrange arguments to make it more > convenient > ?? to create uppercase formatters based on the existing uses. > ?- The implementation has been updated based on the suggestions and API > changes > > The webrev for applying the API to the security classes will be updated > when the API settles down. > > JavaDoc: > http://cr.openjdk.java.net/~rriggs/hex-formatter/java.base/java/util/Hex.html > > > Webrev: > ? http://cr.openjdk.java.net/~rriggs/webrev-hex-formatter-8251989/ > > CSR: > https://bugs.openjdk.java.net/browse/JDK-8251991 > > p.s. > The previous (encoder/decoder) javadoc has been renamed to: > ?? http://cr.openjdk.java.net/~rriggs/hex-encoder-javadoc/ > > From chris.w.dennis at gmail.com Thu Aug 27 14:52:07 2020 From: chris.w.dennis at gmail.com (Chris Dennis) Date: Thu, 27 Aug 2020 14:52:07 +0000 Subject: RFR 8214761 : Bug in parallel Kahan summation implementation In-Reply-To: References: <6fb0565c-81cd-07fa-1c4f-fdf406b75f25@oracle.com> Message-ID: Bump... I've run in to this while running tests that check computation results against the expected bounds of a Kahan summation. Any chance that this gets picked up in the near future? Thanks, Chris ?On 12/13/18, 6:16 PM, "core-libs-dev on behalf of Ivan Gerasimov" wrote: Gentle ping. On 12/9/18 7:37 PM, Ivan Gerasimov wrote: > Hello! > > DoubleSummaryStatistics takes advantage of Kahan summation algorithm > to reduce the error of the total sum. > > Internally it maintains a field double sumCompensation, which keeps > lower bits (which were rounded off) of the last addition. > > Note, that the compensation has to be subtracted from the result to > add those bits back: > > 166 private void sumWithCompensation(double value) { > 167 double tmp = value - sumCompensation; > 168 double velvel = sum + tmp; // Little wolf of rounding error > 169 sumCompensation = (velvel - sum) - tmp; > 170 sum = velvel; > 171 } > > At the line 169, tmp normally has more lower bits than (velvel - sum). > > However, when two DoubleSummaryStatistics objects are combined, this > compensation part is *added* to the total, which may result in a less > accurate result. > > The same bug is replicated in DoubleStreams. > > Would you please help review the fix? > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8214761 > WEBREV: http://cr.openjdk.java.net/~igerasim/8214761/00/webrev/ > -- With kind regards, Ivan Gerasimov From WEIJUN.WANG at ORACLE.COM Thu Aug 27 14:58:24 2020 From: WEIJUN.WANG at ORACLE.COM (Weijun Wang) Date: Thu, 27 Aug 2020 10:58:24 -0400 Subject: RFR: 8250968: Symlinks attributes not preserved when using jarsigner on zip files In-Reply-To: <3b9c3aa9-df9b-5ae3-2301-2e08c8e9f769@oracle.com> References: <957b3bd1-254a-647d-358b-e2dbfba6c6cf@oracle.com> <68CBFB5B-0CAD-44DE-B0E8-92221BACE7C9@ORACLE.COM> <99b55d3f-533e-ac91-89bd-1ce8f3cbd6db@oracle.com> <940A8A93-BB70-48CD-B1DE-0E9EC17E2791@ORACLE.COM> <3b9c3aa9-df9b-5ae3-2301-2e08c8e9f769@oracle.com> Message-ID: <1582051D-5EDD-4F16-88D3-A064683023D3@ORACLE.COM> > On Aug 27, 2020, at 10:36 AM, Se?n Coffey wrote: > > Thanks for the review Max. Comments inline.. > > On 27/08/2020 14:45, Weijun Wang wrote: >> I?m OK with using one warning, but prefer it to a little more formal like "POSIX file permission and/or symlink attributes detected??. >> >> One nit in ZipFile.java: >> >> 1098 // only set posix perms value via ZipEntry constructor for now >> 1099 @Override >> 1100 public int getExtraAttributes(ZipEntry ze) { >> >> Maybe you can just remove the comment. >> >> Do you also want to rename the ?posixPermsDetected" field and loacl variable ?perms? in JarSigner.java? > > Good points. Edits made. > > http://cr.openjdk.java.net/~coffeys/webrev.8250968.v3/webrev/ > >> >> I?m not sure about the test but if zipfs is able to keep permissions inside a zip file then that POSIX byte (or whatever it?s named) is already there and we can modify it to include more bits. > > Looks like it was a conscious design decision to only allow recording of POSIX permission bits for this field (& 0xFFF). I don't see anything about symlink support in zipfs docs. As long as that *byte* is there and it?s not difficult to locate, we can manually add the *bit* for symlink and see if jarsigner can keep it. ?Max > > regards, > Sean. > >> >> No other comment. >> >> Thanks, >> Max >> >> >>> On Aug 27, 2020, at 3:26 AM, Se?n Coffey wrote: >>> >>> updated webrev: >>> http://cr.openjdk.java.net/~coffeys/webrev.8250968.v2/webrev/ >>> >>> regards, >>> Sean. >>> >>> On 27/08/2020 07:42, Se?n Coffey wrote: >>>> Hi Max, >>>> >>>> I looked at updating the warning string but figured that it might have been of no interest to end users. How about this edit then ? >>>> >>>> + {"posix.attributes.detected", "POSIX file permission attributes detected. These attributes are ignored when signing and are not protected by the signature."}, >>>> >>>>>> replace with: >>>> + {"extra.attributes.detected", "POSIX file permission/symlink attributes detected. These attributes are ignored when signing and are not protected by the signature."}, >>>> >>>> regards, >>>> Sean. >>>> >>>> On 26/08/2020 23:15, Weijun Wang wrote: >>>>> Are you going to update the warning text or create a new one? >>>>> >>>>> Thanks, >>>>> Max >>>>> >>>>>> On Aug 26, 2020, at 2:26 PM, Se?n Coffey wrote: >>>>>> >>>>>> This is a follow on from the recent 8218021 fix. The jarsigner tool removes symlink attribute data from zipfiles when signing them. jarsigner should preserve this data. The fix involves preserving the 16 bits associated with the file attributes (instead of the current 12). That's done in ZipFile. All other changes are just a refactor of the variable name. >>>>>> >>>>>> I haven't been able to automate a test for this since zipfs doesn't seem to support symlinks. Manual testing looks good. >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8250968 >>>>>> http://hmsjpse.uk.oracle.com/home/scoffey/ws/jdk-jdk/open/webrev.8250968/webrev/index.html >>>>>> >>>>>> regards, >>>>>> Sean. >>>>>> From martin.doerr at sap.com Thu Aug 27 15:07:08 2020 From: martin.doerr at sap.com (Doerr, Martin) Date: Thu, 27 Aug 2020 15:07:08 +0000 Subject: RFR(M): 8248188: [PATCH] Add HotSpotIntrinsicCandidate and API for Base64 decoding In-Reply-To: References: <11ca749f-3015-c004-aa6b-3194e1dfe4eb@linux.ibm.com> Message-ID: Hi Corey, > If I make a requirement, I feel decode0 should check that the > requirement is met, and raise some kind of internal error if it isn't. > That actually was my first implementation, but I received some comments > during an internal review suggesting that I just "round down" the > destination count to the closest multiple of 3 less than or equal to the > returned value, rather than throw an internal exception which would > confuse users. This "enforces" the rule, in some sense, without error > handling. Do you have some thoughts about this? I think the rounding logic is hard to understand and I'm not sure if it's correct (you're rounding up for the 1st computation of chars_decoded). If we don't use it, it will never get tested (because the intrinsic always returns a multiple of 3). I prefer having a more simple version which is easy to understand and for which we can test all cases. I think we should be able to catch violations of this requirement by adding good JTREG tests. An illegal intrinsic implementation should never pass the tests. So I don't see a need to catch an illegal state in the Java source code in this case. I guess this will be best for intrinsic implementors for other platforms as well. I'd appreciate more opinions on this. > I will double check that everything compiles and runs properly with gcc > 7.3.1. Please note that 7.3.1 is our minimum for Big Endian linux. For Little Endian it's 7.4.0. You can also find this information here: https://wiki.openjdk.java.net/display/Build/Supported+Build+Platforms under "Other JDK 13 build platforms" which hasn't changed since then. > I will use __attribute__ ((align(16))) instead of __vector, and make > them arrays of 16 unsigned char. Maybe __vectors works as expected, too, now. Whatever we use, I'd appreciate to double-check the alignment e.g. by using gdb. I don't remember what we had tried and why it didn't work as desired. > I was following what was done for encodeBlock, but it appears > encodeBlock's style isn't what is used for the other intrinsics. I will > correct decodeBlock to use the prevailing style. Another patch should > be added (not part of this webrev) to correct encodeBlock's style. In your code one '\' is not aligned with the other ones. > Ah, this is another thing I didn't know about. I will make some > regression tests. Thanks. There's some documentation available: https://openjdk.java.net/jtreg/ I guess your colleagues can assist you with that so you don't have to figure out everything alone. > Thanks for your time on this. As you can tell, I'm inexperienced in > writing openjdk code, so your patience and careful review is really > appreciated. I'm glad you work on contributions. I think we should welcome new contributors and assist as far as we can. Best regards, Martin > -----Original Message----- > From: Corey Ashford > Sent: Donnerstag, 27. August 2020 00:17 > To: Doerr, Martin ; Michihiro Horie > > Cc: hotspot-compiler-dev at openjdk.java.net; core-libs-dev dev at openjdk.java.net>; Kazunori Ogata ; > joserz at br.ibm.com > Subject: Re: RFR(M): 8248188: [PATCH] Add HotSpotIntrinsicCandidate and > API for Base64 decoding > > Hi Martin, > > Some inline responses below. > > On 8/26/20 8:26 AM, Doerr, Martin wrote: > > > Hi Corey, > > > > I should explain my comments regarding Base64.java better. > > > >> Let's be precise: "should process a multiple of four" => "must process a > >> multiple of four" > > Did you try to support non-multiple of 4 and this was intended as > recommendation? > > I think making it a requirement and simplifying the logic in decode0 is > better. > > Or what's the benefit of the recommendation? > > If I make a requirement, I feel decode0 should check that the > requirement is met, and raise some kind of internal error if it isn't. > That actually was my first implementation, but I received some comments > during an internal review suggesting that I just "round down" the > destination count to the closest multiple of 3 less than or equal to the > returned value, rather than throw an internal exception which would > confuse users. This "enforces" the rule, in some sense, without error > handling. Do you have some thoughts about this? > > > > >>> If any illegal base64 bytes are encountered in the source by the > >>> intrinsic, the intrinsic can return a data length of zero or any > >>> number of bytes before the place where the illegal base64 byte > >>> was encountered. > >> I think this has a drawback. Somebody may use a debugger and want to > stop > >> when throwing IllegalArgumentException. He should see the position > which > >> matches the Java implementation.kkkk > > This is probably hard to understand. Let me try to explain it by example: > > 1. 80 Bytes get processed by the intrinsic and 60 Bytes written to the > destination array. > > 2. The intrinsic sees an illegal base64 Byte and it returns 12 which is allowed > by your specification. > > 3. The compiled method containing the intrinsic hits a safepoint (e.g. in the > large while loop in decodeBlockSlow). > > 4. A JVMTI agent (debugger) reads dp and dst. > > 5. The person using the debugger gets angry because more bytes than dp > were written into dst. The JVM didn't follow the specified behavior. > > > > I guess we can and should avoid it by specifying that the intrinsic needs to > return the dp value matching the number of Bytes written. > > That's an interesting point. I will change the specification, and the > intrinsic implementation. Right now the Power9/10 intrinsic returns 0 > when any illegal character is discovered, but I've been thinking about > returning the number of bytes already written, which will allow > decodeBlockSlow to more quickly find the offending character. This > provides another good reason to make that change. > > > > > Best regards, > > Martin > > > > > >> -----Original Message----- > >> From: Doerr, Martin > >> Sent: Dienstag, 25. August 2020 15:38 > >> To: Corey Ashford ; Michihiro Horie > >> > >> Cc: hotspot-compiler-dev at openjdk.java.net; core-libs-dev >> dev at openjdk.java.net>; Kazunori Ogata ; > >> joserz at br.ibm.com > >> Subject: RE: RFR(M): 8248188: [PATCH] Add HotSpotIntrinsicCandidate > and > >> API for Base64 decoding > >> > >> Hi Corey, > >> > >> thanks for proposing this change. I have comments and suggestions > >> regarding various files. > >> > >> > >> Base64.java > >> > >> This is the only file which needs another review from core-libs-dev. > >> First of all, I like the idea to use a HotSpotIntrinsicCandidate which can > >> consume as many bytes as the implementation wants. > >> > >> Comment before decodeBlock: > >> Let's be precise: "should process a multiple of four" => "must process a > >> multiple of four" > >> > >>> If any illegal base64 bytes are encountered in the source by the > >>> intrinsic, the intrinsic can return a data length of zero or any > >>> number of bytes before the place where the illegal base64 byte > >>> was encountered. > >> I think this has a drawback. Somebody may use a debugger and want to > stop > >> when throwing IllegalArgumentException. He should see the position > which > >> matches the Java implementation. > >> > >> Please note that the comment indentation differs from other comments. > > Will fix. > > >> > >> decode0: Final "else" after return is redundant. > > Will fix. > > >> > >> > >> stubGenerator_ppc.cpp > >> > >> "__vector" breaks AIX build! > >> Does it work on Big Endian linux with old gcc (we require 7.3.1, now)? > >> Please either support Big Endian properly or #ifdef it out. > > I have been compiling with only Advance Toolchain 13, which is 9.3.1, > and only on Linux. It will not work with big endian, so it won't work > on AIX, however obviously it shouldn't break the AIX build, so I will > address that. There's code to set UseBASE64Intrinsics to false on big > endian, but you're right -- I should ifdef all of the intrinsic code for > little endian for now. Getting it to work on big endian / AIX shouldn't > be difficult, but it's not in my scope of work at the moment. > > I will double check that everything compiles and runs properly with gcc > 7.3.1. > > >> What exactly does it (do) on linux? > > It's an arch-specific type that's 16 bytes in size and aligned on a > 16-byte boundary. > > >> I remember that we had tried such prefixes but were not satisfied. I think > it > >> didn't enforce 16 Byte alignment if I remember correctly. > > I will use __attribute__ ((align(16))) instead of __vector, and make > them arrays of 16 unsigned char. > > >> > >> Attention: C2 does no longer convert int/bool to 64 bit values (since JDK- > >> 8086069). So the argument registers for offset, length and isURL may > contain > >> garbage in the higher bits. > > Wow, that's good to know! I will mask off the incoming values. > > >> > >> You may want to use load_const_optimized which produces shorter code. > > Will fix. > > >> > >> You may want to use __ align(32) to align unrolled_loop_start. > > Will fix. > > >> > >> I'll review the algorithm in detail when I find more time. > >> > >> > >> assembler_ppc.hpp > >> assembler_ppc.inline.hpp > >> vm_version_ppc.cpp > >> vm_version_ppc.hpp > >> Please rebase. Parts of the change were pushed as part of 8248190: > Enable > >> Power10 system and implement new byte-reverse instructions > > Will do. > > >> > >> > >> vmSymbols.hpp > >> Indentation looks odd at the end. > > I was following what was done for encodeBlock, but it appears > encodeBlock's style isn't what is used for the other intrinsics. I will > correct decodeBlock to use the prevailing style. Another patch should > be added (not part of this webrev) to correct encodeBlock's style. > > >> > >> > >> library_call.cpp > >> Good. Indentation style of the call parameters differs from encodeBlock. > > Will fix. > > >> > >> > >> runtime.cpp > >> Good. > >> > >> > >> aotCodeHeap.cpp > >> vmSymbols.cpp > >> shenandoahSupport.cpp > >> vmStructs_jvmci.cpp > >> shenandoahSupport.cpp > >> escape.cpp > >> runtime.hpp > >> stubRoutines.cpp > >> stubRoutines.hpp > >> vmStructs.cpp > >> Good and trivial. > >> > >> > >> Tests: > >> I think we should have JTREG tests to check for regressions in the future. > > Ah, this is another thing I didn't know about. I will make some > regression tests. > > Thanks for your time on this. As you can tell, I'm inexperienced in > writing openjdk code, so your patience and careful review is really > appreciated. > > - Corey From fernando.guallini at oracle.com Thu Aug 27 15:41:39 2020 From: fernando.guallini at oracle.com (Fernando Guallini) Date: Thu, 27 Aug 2020 08:41:39 -0700 (PDT) Subject: RFR: JDK-8249699: java/io/ByteArrayOutputStream/MaxCapacity.java should use @requires instead of @ignore Message-ID: <859c166d-0bf0-495d-abf6-3c1870a0c434@default> Thanks Sean, updated webrev here: http://cr.openjdk.java.net/~fguallini/8249699/webrev.01/ Regards, Fernando ----- Original Message ----- From: sean.coffey at oracle.com To: fernando.guallini at oracle.com, core-libs-dev at openjdk.java.net Sent: Wednesday, 26 August, 2020 7:39:25 PM GMT +00:00 GMT Britain, Ireland, Portugal Subject: Re: RFR: JDK-8249699: java/io/ByteArrayOutputStream/MaxCapacity.java should use @requires instead of @ignore test/jdk/java/util/Base64/TestEncodingDecodingLength.java is an example of another test using -Xmx8g. Do you want to push the os.maxMemory requirement up to 10g perhaps ? It might avoid border line resource failures. Also I think it might need a "sun.arch.data.model == "64" " requirement : @requires (sun.arch.data.model == "64" & os.maxMemory >= 10g) regards, Sean. On 26/08/2020 18:17, Fernando Guallini wrote: > Hi, > > Could I please get reviews and a sponsor for: > > webrev: http://cr.openjdk.java.net/~fguallini/8249699/webrev.00/ > Testbug: https://bugs.openjdk.java.net/browse/JDK-8249699 > > The test was ignored due to ?huge memory requirements?, but with the jtreg current version, tests can be only run when system requirements are satisfied as opposed to being always excluded. It runs and passes in Mach5 > > > Thanks > > -Fernando From LANCE.ANDERSEN at ORACLE.COM Thu Aug 27 16:32:49 2020 From: LANCE.ANDERSEN at ORACLE.COM (Lance Andersen) Date: Thu, 27 Aug 2020 12:32:49 -0400 Subject: RFR: 8250968: Symlinks attributes not preserved when using jarsigner on zip files In-Reply-To: <1582051D-5EDD-4F16-88D3-A064683023D3@ORACLE.COM> References: <957b3bd1-254a-647d-358b-e2dbfba6c6cf@oracle.com> <68CBFB5B-0CAD-44DE-B0E8-92221BACE7C9@ORACLE.COM> <99b55d3f-533e-ac91-89bd-1ce8f3cbd6db@oracle.com> <940A8A93-BB70-48CD-B1DE-0E9EC17E2791@ORACLE.COM> <3b9c3aa9-df9b-5ae3-2301-2e08c8e9f769@oracle.com> <1582051D-5EDD-4F16-88D3-A064683023D3@ORACLE.COM> Message-ID: <15DE9185-D6DC-4E42-A7C5-B5B217248C6B@ORACLE.COM> Hi Sean, I think the changes are OK in the latest version. A couple of the files have a 2019 copyright still. > On Aug 27, 2020, at 10:58 AM, Weijun Wang wrote: > > > >> On Aug 27, 2020, at 10:36 AM, Se?n Coffey wrote: >> >> Thanks for the review Max. Comments inline.. >> >> On 27/08/2020 14:45, Weijun Wang wrote: >>> I?m OK with using one warning, but prefer it to a little more formal like "POSIX file permission and/or symlink attributes detected??. >>> >>> One nit in ZipFile.java: >>> >>> 1098 // only set posix perms value via ZipEntry constructor for now >>> 1099 @Override >>> 1100 public int getExtraAttributes(ZipEntry ze) { >>> >>> Maybe you can just remove the comment. >>> >>> Do you also want to rename the ?posixPermsDetected" field and loacl variable ?perms? in JarSigner.java? >> >> Good points. Edits made. >> >> http://cr.openjdk.java.net/~coffeys/webrev.8250968.v3/webrev/ >> >>> >>> I?m not sure about the test but if zipfs is able to keep permissions inside a zip file then that POSIX byte (or whatever it?s named) is already there and we can modify it to include more bits. >> >> Looks like it was a conscious design decision to only allow recording of POSIX permission bits for this field (& 0xFFF). I don't see anything about symlink support in zipfs docs. > > As long as that *byte* is there and it?s not difficult to locate, we can manually add the *bit* for symlink and see if jarsigner can keep it. We can create an RFE for adding support for this with Zip FS. > > ?Max > >> >> regards, >> Sean. >> >>> >>> No other comment. >>> >>> Thanks, >>> Max >>> >>> >>>> On Aug 27, 2020, at 3:26 AM, Se?n Coffey wrote: >>>> >>>> updated webrev: >>>> http://cr.openjdk.java.net/~coffeys/webrev.8250968.v2/webrev/ >>>> >>>> regards, >>>> Sean. >>>> >>>> On 27/08/2020 07:42, Se?n Coffey wrote: >>>>> Hi Max, >>>>> >>>>> I looked at updating the warning string but figured that it might have been of no interest to end users. How about this edit then ? >>>>> >>>>> + {"posix.attributes.detected", "POSIX file permission attributes detected. These attributes are ignored when signing and are not protected by the signature."}, >>>>> >>>>>>> replace with: >>>>> + {"extra.attributes.detected", "POSIX file permission/symlink attributes detected. These attributes are ignored when signing and are not protected by the signature."}, >>>>> >>>>> regards, >>>>> Sean. >>>>> >>>>> On 26/08/2020 23:15, Weijun Wang wrote: >>>>>> Are you going to update the warning text or create a new one? >>>>>> >>>>>> Thanks, >>>>>> Max >>>>>> >>>>>>> On Aug 26, 2020, at 2:26 PM, Se?n Coffey wrote: >>>>>>> >>>>>>> This is a follow on from the recent 8218021 fix. The jarsigner tool removes symlink attribute data from zipfiles when signing them. jarsigner should preserve this data. The fix involves preserving the 16 bits associated with the file attributes (instead of the current 12). That's done in ZipFile. All other changes are just a refactor of the variable name. >>>>>>> >>>>>>> I haven't been able to automate a test for this since zipfs doesn't seem to support symlinks. Manual testing looks good. >>>>>>> >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8250968 >>>>>>> http://hmsjpse.uk.oracle.com/home/scoffey/ws/jdk-jdk/open/webrev.8250968/webrev/index.html >>>>>>> >>>>>>> regards, >>>>>>> Sean. Best Lance ------------------ Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From peter.levart at gmail.com Thu Aug 27 17:21:53 2020 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 27 Aug 2020 19:21:53 +0200 Subject: RFR 8251989: Hex formatter and parser utility In-Reply-To: <6378b60b-7a45-d8b0-5ebd-3d3bf9144401@oracle.com> References: <6378b60b-7a45-d8b0-5ebd-3d3bf9144401@oracle.com> Message-ID: <7e020976-0244-0e13-d25f-4533d6188ca0@gmail.com> Hi Roger, About methods in Hex.Formatter that append to StringBuilder, like the following one: ??? public StringBuilder format?(StringBuilder sb, byte[] bytes) ...I was thinking that such method could have more utility if it was specified as: ??? public A format(A appendable, byte[] bytes) For example, you could also format directly to PrintStream, Writer, CharBuffer or any custom implementation. The only grief is that Appendable methods are specified to throw IOException, but Hex.Formatter.format(Appendable ... methods could be specified to wrap such exception with UncheckedIOException. For usage with StringBuilder this would not change anything as it never throws IOException. What do you think? Regards, Peter On 8/27/20 3:34 AM, Roger Riggs wrote: > Please review updates to the formatting and parsing API based on the > last round of comments. > There are many changes, so it may be useful to read it as a fresh draft. > > ?- Rename classes: Encoder -> Formatter; Decoder -> Parser > ?- Rename methods: encode -> format; decode -> parse, etc. > ?- Rename factory methods to match > ?- Added a factory method and re-arrange arguments to make it more > convenient > ?? to create uppercase formatters based on the existing uses. > ?- The implementation has been updated based on the suggestions and > API changes > > The webrev for applying the API to the security classes will be > updated when the API settles down. > > JavaDoc: > http://cr.openjdk.java.net/~rriggs/hex-formatter/java.base/java/util/Hex.html > > > Webrev: > ? http://cr.openjdk.java.net/~rriggs/webrev-hex-formatter-8251989/ > > CSR: > https://bugs.openjdk.java.net/browse/JDK-8251991 > > p.s. > The previous (encoder/decoder) javadoc has been renamed to: > ?? http://cr.openjdk.java.net/~rriggs/hex-encoder-javadoc/ > > From Roger.Riggs at oracle.com Thu Aug 27 17:32:39 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Thu, 27 Aug 2020 13:32:39 -0400 Subject: RFR 8251989: Hex formatter and parser utility In-Reply-To: References: <6378b60b-7a45-d8b0-5ebd-3d3bf9144401@oracle.com> Message-ID: <045b26a5-70cc-6783-8050-3d711482afb5@oracle.com> Hi Chris, On 8/27/20 9:20 AM, Chris Hegarty wrote: > Roger, > >> On 27 Aug 2020, at 02:34, Roger Riggs wrote: >> >> Please review updates to the formatting and parsing API based on the last round of comments. >> There are many changes, so it may be useful to read it as a fresh draft. >> >> - Rename classes: Encoder -> Formatter; Decoder -> Parser >> - Rename methods: encode -> format; decode -> parse, etc. >> - Rename factory methods to match >> - Added a factory method and re-arrange arguments to make it more convenient >> to create uppercase formatters based on the existing uses. >> - The implementation has been updated based on the suggestions and API changes >> >> The webrev for applying the API to the security classes will be updated when the API settles down. >> >> JavaDoc: >> http://cr.openjdk.java.net/~rriggs/hex-formatter/java.base/java/util/Hex.html > 1. "Hex.Formatter formats bytes to a hex string or appends to > StringBuilder.? > > I wonder if this could be a bit more descriptive. What about something > like: > > ?Hex.Formatter transforms sequences of bytes into a formatted > hexadecimal string.? What is a formatted hexadecimal string? ?A > formatted hexadecimal string consists solely of; an optional prefix, a > pair of hexadecimal characters ( THESE ARE ALREADY DEFINED ELSEWHERE), > an optional delimiter, and an optional suffix.? Or some such wording. Thanks for the suggestion. I was expecting that the brief description in the Hex class would lead to the more complete desriptions in the Formatter and Parser classes and avoid duplication. * {@link Formatter} Hex.Formatter transforms sequences of bytes into a formatted * hexadecimal string. * A formatted hexadecimal string consists solely of an optional prefix, * a sequence of pairs of hexadecimal characters, an optional delimiter between each pair of * hexadecimal characters, and an optional suffix. * The {@link #formatter() canonical formatter} with an empty prefix, suffix, * and delimiter, provides unformatted (plain) encoder functionality. > > 2. I like the move to formatter/parser, but I now find myself looking > for an encoder/decoder ;-) It might be obvious, but could be worth > calling out explicitly, e.g. ?A formatter with an empty prefix, suffix, > and delimiter, provides unformatted (plain) encoder functionality". > Similar for parser. Maybe this is what ?canonical? is referring to? > If so, I think it would be best to define it somewhere. Yes, that's the canonical formatter. > > 3. "The Hex class consists solely of factory methods for hexadecimal > (hex) formatters and parsers.? > > It could be worth generalising the class-level description now, so as > to allow space for additional convenience methods in the future. ( A > while back I suggested not adding convenience methods yet, but I see > that Mark has suggested a couple of key conveniences ) Fair enough. Another thought though,? both the Formatter and Parser classes contain only the parameter (and they the same parameters). With as few format methods and parse methods, it could collapse down to just the Hex instances with the parameters and having both the format and parse methods.? All of the behavior is in the methods, not the class.? Though it does collapse the name space. Thanks, Roger > > -Chris. > From vipinsharma85 at gmail.com Thu Aug 27 17:35:58 2020 From: vipinsharma85 at gmail.com (Vipin Sharma) Date: Thu, 27 Aug 2020 23:05:58 +0530 Subject: RFR 8252265: Replace @exception with @throws java.util.logging package In-Reply-To: References: Message-ID: <86742333-A327-4BED-B39E-A2AFA0D50E0B@gmail.com> Hi Daniel, > On Aug 26, 2020, at 10:25 PM, Daniel Fuchs wrote: > > Hi Vipin, > > This looks good to me. > Make sure to generate the API docs locally and eyeball them to > double check that nothing is amiss. Checked API docs, it doesn?t have any change. Regards, Vipin From Roger.Riggs at oracle.com Thu Aug 27 17:42:11 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Thu, 27 Aug 2020 13:42:11 -0400 Subject: RFR 8251989: Hex formatter and parser utility In-Reply-To: <1BAC4282-314D-4009-B2F7-28B6FDACBA28@oracle.com> References: <1BAC4282-314D-4009-B2F7-28B6FDACBA28@oracle.com> Message-ID: <465e58eb-a18e-5aa9-42ad-be5f41c67c8f@oracle.com> Hi Douglas, On 8/27/20 10:37 AM, Douglas Surber wrote: > The meaning of prefix and suffix is not specified in formatter?(boolean uppercase, String delimiter,String prefix, String suffix). It isn't specified whether they precede and follow the entire formatted value or each byte. The class comment clarifies but I shouldn't have to go there to discover this. > > I was surprised at the meaning for prefix and suffix. The seem pointless to me. It is trivial to enclose the entire formatted value with a prefix and suffix without using these arguments. If they were prefix and suffix for each individual byte, that would be much more useful. For example, how can I format a byte sequence like this? > > 0x00 0x01 0x02 0x0d 0x0e 0x0f > > format(false, " 0x", "0x", "") > > doesn't work because an empty byte array would be > > 0x > > instead of an empty string. The prefix and suffix concepts first appeared in the StringJoiner. In the Hex context, they can be used to construct a complete string efficiently. For example a mac address.? [50:2b:7f:e8:6a:e2] I have tried out the use of prefix and suffix as you did above and noticed the same limitation. I did run across several examples in the OpenJDK code where an empty string had a different representation and it might reasonable to have a replacement for an empty array. Though the factory methods are about the limits for numbers of args. A more fluent builder API has been suggested. The javadoc can be expanded upon to make it clearer. Thanks, Roger > > Douglas > >> On Aug 27, 2020, at 4:55 AM, core-libs-dev-request at openjdk.java.net wrote: >> >> Message: 1 >> Date: Wed, 26 Aug 2020 21:34:47 -0400 >> From: Roger Riggs > >> To: core-libs-dev > >> Subject: RFR 8251989: Hex formatter and parser utility >> Message-ID: <6378b60b-7a45-d8b0-5ebd-3d3bf9144401 at oracle.com > >> Content-Type: text/plain; charset=utf-8; format=flowed >> >> Please review updates to the formatting and parsing API based on the >> last round of comments. >> There are many changes, so it may be useful to read it as a fresh draft. >> >> ?- Rename classes: Encoder -> Formatter; Decoder -> Parser >> ?- Rename methods: encode -> format; decode -> parse, etc. >> ?- Rename factory methods to match >> ?- Added a factory method and re-arrange arguments to make it more >> convenient >> ?? to create uppercase formatters based on the existing uses. >> ?- The implementation has been updated based on the suggestions and API >> changes >> >> The webrev for applying the API to the security classes will be updated >> when the API settles down. >> >> JavaDoc: >> http://cr.openjdk.java.net/~rriggs/hex-formatter/java.base/java/util/Hex.html >> >> >> Webrev: >> ? http://cr.openjdk.java.net/~rriggs/webrev-hex-formatter-8251989/ >> >> CSR: >> https://bugs.openjdk.java.net/browse/JDK-8251991 >> >> p.s. >> The previous (encoder/decoder) javadoc has been renamed to: >> ?? http://cr.openjdk.java.net/~rriggs/hex-encoder-javadoc/ >> >> From Roger.Riggs at oracle.com Thu Aug 27 17:50:59 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Thu, 27 Aug 2020 13:50:59 -0400 Subject: RFR 8251989: Hex formatter and parser utility In-Reply-To: <7e020976-0244-0e13-d25f-4533d6188ca0@gmail.com> References: <6378b60b-7a45-d8b0-5ebd-3d3bf9144401@oracle.com> <7e020976-0244-0e13-d25f-4533d6188ca0@gmail.com> Message-ID: <3433aa3c-3e34-832e-6e22-b03e20bfa953@oracle.com> Hi Peter, I made the same observation when exploring the API and ended up concluding that the declared exceptions on Appendable made the API much harder to use. Though the formatted characters can't be directly accumulated in the Appendable it is possible to build the output in a StringBuilder and then append the StringBuilder to the appendable with only a single copy since it is a CharSequence. That would avoid the creation/allocation of a String as an intermediate step. Thanks, Roger On 8/27/20 1:21 PM, Peter Levart wrote: > Hi Roger, > > > About methods in Hex.Formatter that append to StringBuilder, like the > following one: > > > ??? public StringBuilder format?(StringBuilder sb, byte[] bytes) > > > ...I was thinking that such method could have more utility if it was > specified as: > > > ??? public A format(A appendable, byte[] bytes) > > > For example, you could also format directly to PrintStream, Writer, > CharBuffer or any custom implementation. The only grief is that > Appendable methods are specified to throw IOException, but > Hex.Formatter.format(Appendable ... methods could be specified to wrap > such exception with UncheckedIOException. For usage with StringBuilder > this would not change anything as it never throws IOException. > > What do you think? > > Regards, Peter > > > On 8/27/20 3:34 AM, Roger Riggs wrote: >> Please review updates to the formatting and parsing API based on the >> last round of comments. >> There are many changes, so it may be useful to read it as a fresh draft. >> >> ?- Rename classes: Encoder -> Formatter; Decoder -> Parser >> ?- Rename methods: encode -> format; decode -> parse, etc. >> ?- Rename factory methods to match >> ?- Added a factory method and re-arrange arguments to make it more >> convenient >> ?? to create uppercase formatters based on the existing uses. >> ?- The implementation has been updated based on the suggestions and >> API changes >> >> The webrev for applying the API to the security classes will be >> updated when the API settles down. >> >> JavaDoc: >> http://cr.openjdk.java.net/~rriggs/hex-formatter/java.base/java/util/Hex.html >> >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-hex-formatter-8251989/ >> >> CSR: >> https://bugs.openjdk.java.net/browse/JDK-8251991 >> >> p.s. >> The previous (encoder/decoder) javadoc has been renamed to: >> ?? http://cr.openjdk.java.net/~rriggs/hex-encoder-javadoc/ >> >> From sergei.tsypanov at yandex.ru Thu Aug 27 20:09:28 2020 From: sergei.tsypanov at yandex.ru (=?utf-8?B?0KHQtdGA0LPQtdC5INCm0YvQv9Cw0L3QvtCy?=) Date: Thu, 27 Aug 2020 22:09:28 +0200 Subject: [PATCH] optimization opportunity regarding misuse of BufferedInputStream Message-ID: <10579141598555707@mail.yandex.ru> Hi, while looking into java.util.jar.JarInputStream I've found a misuse of BufferedInputStream in checkManifest() method where InputStream is wrapped into BIS and later read from with separately created byte[] buffer. At runtime this means the data is copied from original IS into BIS and from there to byte[] resulting in redundant memory allocation. The benchmark [1] demonstrates that for both scenarios (with and without verification) we can save memory and reduce execution time by dropping BIS: original Mode Cnt Score Error Units read avgt 50 230.301 ? 2.130 us/op read:?gc.alloc.rate.norm avgt 50 148929.020 ? 22.383 B/op readNoVerify avgt 50 228.673 ? 0.556 us/op readNoVerify:?gc.alloc.rate.norm avgt 50 148133.555 ? 9.599 B/op patched Mode Cnt Score Error Units read avgt 50 225.976 ? 0.543 us/op read:?gc.alloc.rate.norm avgt 50 140672.404 ? 20.732 B/op readNoVerify avgt 50 229.563 ? 1.731 us/op readNoVerify:?gc.alloc.rate.norm avgt 50 139874.648 ? 7.054 B/op Also InputStream.transferTo() can be called for the sake of code reuse. Another snippets are located in MimeTable and in JmodFile: - in MimeTable InputStream is passed into Properties.load() where they use a buffer of exactly the same size (8192) as in BufferedInputStream. Benchmark [2] demonstrates significant improvement when redundant bufferization is reduced: Mode Cnt Score Error Units buffered avgt 50 155.477 ? 6.370 us/op buffered:?gc.alloc.rate.norm avgt 50 46263.355 ? 1.632 B/op conventional avgt 50 146.762 ? 3.481 us/op conventional:?gc.alloc.rate.norm avgt 50 38014.521 ? 7.152 B/op - in JmodFile we read only the 4 first bytes but BufferedInputStream fills in the whole buffer (by default again 8192) Unfortunately, I couldn't construct benchmark for JmodFile, however I'm sure there'll be improvement either. Patch is attached. It also includes some tiny clean-ups in mentioned classes, e.g. dead code removal With best regards, Sergey Tsypanov 1. @State(Scope.Thread) @BenchmarkMode(Mode.AverageTime) @OutputTimeUnit(TimeUnit.MICROSECONDS) @Fork(jvmArgsAppend = {"-Xms2g", "-Xmx2g"}) public class ReadManifestBenchmark { private final ClassLoader classLoader = getClass().getClassLoader(); @Benchmark public Manifest read() throws IOException { return readManifest(true); } @Benchmark public Manifest readNoVerify() throws IOException { return readManifest(false); } private Manifest readManifest(boolean verify) throws IOException { try ( InputStream resourceAsStream = classLoader.getResourceAsStream("jmh-core-1.23.jar"); JarInputStream jarInputStream = new JarInputStream(resourceAsStream, verify) ) { return jarInputStream.getManifest(); } } } 2. @State(Scope.Thread) @BenchmarkMode(Mode.AverageTime) @OutputTimeUnit(TimeUnit.MICROSECONDS) @Fork(jvmArgsAppend = {"-Xms2g", "-Xmx2g"}) public class LoadPropertiesBenchmark { private final ClassLoader classLoader = getClass().getClassLoader(); @Benchmark public Object conventional() throws IOException { try (InputStream is = getClass().getClassLoader().getResource("sun/net/www/content-types.properties").openStream()) { Properties p = new Properties(); p.load(is); return p; } } @Benchmark public Object buffered() throws IOException { try (InputStream is = new BufferedInputStream(getClass().getClassLoader().getResource("sun/net/www/content-types.properties").openStream())) { Properties p = new Properties(); p.load(is); return p; } } } -------------- next part -------------- A non-text attachment was scrubbed... Name: buffered.patch Type: text/x-diff Size: 4110 bytes Desc: not available URL: From fweimer at redhat.com Fri Aug 28 10:08:03 2020 From: fweimer at redhat.com (Florian Weimer) Date: Fri, 28 Aug 2020 12:08:03 +0200 Subject: [PATCH] optimization opportunity regarding misuse of BufferedInputStream In-Reply-To: <10579141598555707@mail.yandex.ru> (=?utf-8?B?ItCh0LXRgNCz?= =?utf-8?B?0LXQuSDQptGL0L/QsNC90L7QsiIncw==?= message of "Thu, 27 Aug 2020 22:09:28 +0200") References: <10579141598555707@mail.yandex.ru> Message-ID: <87v9h3rsl8.fsf@oldenburg2.str.redhat.com> * ?????? ???????: > @@ -105,12 +105,8 @@ > private byte[] getBytes(InputStream is) > throws IOException > { > - byte[] buffer = new byte[8192]; > ByteArrayOutputStream baos = new ByteArrayOutputStream(2048); > - int n; > - while ((n = is.read(buffer, 0, buffer.length)) != -1) { > - baos.write(buffer, 0, n); > - } > + is.transferTo(baos); > return baos.toByteArray(); > } Isn't this InputStream::readAllBytes? Thanks, Florian From alexander.scherbatiy at bell-sw.com Fri Aug 28 11:25:25 2020 From: alexander.scherbatiy at bell-sw.com (Alexander Scherbatiy) Date: Fri, 28 Aug 2020 14:25:25 +0300 Subject: RFR 8252248: __SIGRTMAX is not declared in musl libc In-Reply-To: References: <1c4ad320-a22a-484a-d370-852f9ecd90f8@bell-sw.com> Message-ID: <7c1ef537-3c36-c28a-c083-5bd8e5de8edf@bell-sw.com> Hello, Could you review the updated fix: ? http://cr.openjdk.java.net/~alexsch/8252248/webrev.01/ ?- moving shared code to net_util_md.h is avoided ?- INTERRUPT_SIGNAL is defined as SIGRTMAX - 2 instead of __SIGRTMAX - 2 for Linux in NativeThread.c ?- "#include " is moved out from "#ifdef" in NativeThread.c ?- sigWakeup is changed to "#define WAKEUP_SIGNAL (SIGRTMAX - 2)" in linux_close.c At least test/jdk/java/net/Socket/asyncClose/AsyncClose.java touches all four methods where the signal changes are used: NativeThread.c - Java_sun_nio_ch_NativeThread_init(JNIEnv *env, jclass cl) - Java_sun_nio_ch_NativeThread_signal(JNIEnv *env, jclass cl, jlong thread) linux_close.c -? __attribute((constructor)) init() - closefd(int fd1, int fd2) I run test/jdk/java/net tests on Ubuntu 18.04.4 and Alpine Linux 3.11.6 with musl libc 1.1.24. There are 10 failed tests but they fail without the fix as well. I believe that it is because of some network settings of? my machine. ? java/net/MulticastSocket/IPMulticastIF.java ? java/net/MulticastSocket/NoSetNetworkInterface.java ? java/net/MulticastSocket/Promiscuous.java ? java/net/MulticastSocket/PromiscuousIPv6.java ? java/net/MulticastSocket/SetGetNetworkInterfaceTest.java ? java/net/MulticastSocket/SetLoopbackMode.java ? java/net/MulticastSocket/SetOutgoingIf.java ? java/net/MulticastSocket/Test.java ? java/net/NetworkInterface/NetworkInterfaceRetrievalTests.java ? java/net/ipv6tests/UdpTest.java Thanks, Alexander. On 26.08.2020 13:42, Alan Bateman wrote: > On 25/08/2020 18:00, Alexander Scherbatiy wrote: >> >> Hello, >> >> Could your review the fix for the issue: >> ? Bug: https://bugs.openjdk.java.net/browse/JDK-8252248 >> ? Fix: http://cr.openjdk.java.net/~alexsch/8252248/webrev.00/ >> >> : >> >> The fix has been discussed on the portola-dev alias [2] where it was >> pointed out that the fix can be reviewed in the mainline and it was >> suggested to rename the INTERRUPT_SIGNAL and move its definition to >> net_util_md.h. >> > The xxx_close.c sources are for the old SocketImpl and > DatagramSocketImpl implementations; they are aren't used by default > and will eventually go away. Maybe an option for the musl port is to > just not compile in the old socket implementations? It might be best > to start a discussion on nio-dev and net-dev about this as we > shouldn't be changing NativeThread.c (which is used for signalling in > the FileChannel implementation) to include net_util.h. > > -Alan From Alan.Bateman at oracle.com Fri Aug 28 12:55:59 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 28 Aug 2020 13:55:59 +0100 Subject: RFR 8252248: __SIGRTMAX is not declared in musl libc In-Reply-To: <7c1ef537-3c36-c28a-c083-5bd8e5de8edf@bell-sw.com> References: <1c4ad320-a22a-484a-d370-852f9ecd90f8@bell-sw.com> <7c1ef537-3c36-c28a-c083-5bd8e5de8edf@bell-sw.com> Message-ID: On 28/08/2020 12:25, Alexander Scherbatiy wrote: > Hello, > > Could you review the updated fix: > ? http://cr.openjdk.java.net/~alexsch/8252248/webrev.01/ > > ?- moving shared code to net_util_md.h is avoided > ?- INTERRUPT_SIGNAL is defined as SIGRTMAX - 2 instead of __SIGRTMAX - > 2 for Linux in NativeThread.c > ?- "#include " is moved out from "#ifdef" in NativeThread.c > ?- sigWakeup is changed to "#define WAKEUP_SIGNAL (SIGRTMAX - 2)" in > linux_close.c > > At least test/jdk/java/net/Socket/asyncClose/AsyncClose.java touches > all four methods where the signal changes are used: > NativeThread.c > - Java_sun_nio_ch_NativeThread_init(JNIEnv *env, jclass cl) > - Java_sun_nio_ch_NativeThread_signal(JNIEnv *env, jclass cl, jlong > thread) > linux_close.c > -? __attribute((constructor)) init() > - closefd(int fd1, int fd2) NativeThread current/signal will be exercised by the tests in java/nio/channels. The code in linux_close.c is only used with the old (and not used by default) SocketImpl and DatagramSocketImpl implementation. Some of the tests jdk_net test group will re-run the tests with the old implementation so you should be okay. > > I run test/jdk/java/net tests on Ubuntu 18.04.4 and Alpine Linux > 3.11.6 with musl libc 1.1.24. > There are 10 failed tests but they fail without the fix as well. I > believe that it is because of some network settings of? my machine. Sometimes there is will firewall or iptables configuration need to allow multicast datagrams to be received. -Alan From sergei.tsypanov at yandex.ru Fri Aug 28 13:29:47 2020 From: sergei.tsypanov at yandex.ru (=?utf-8?B?0KHQtdGA0LPQtdC5INCm0YvQv9Cw0L3QvtCy?=) Date: Fri, 28 Aug 2020 15:29:47 +0200 Subject: [PATCH] optimization opportunity regarding misuse of BufferedInputStream In-Reply-To: <87v9h3rsl8.fsf@oldenburg2.str.redhat.com> References: <10579141598555707@mail.yandex.ru> <87v9h3rsl8.fsf@oldenburg2.str.redhat.com> Message-ID: <11808131598621180@mail.yandex.ru> Hi, > Isn't this InputStream::readAllBytes? thanks for pointing this out! Indeed, InputStream::readAllBytes() allows to save even more memory: Mode Cnt Score Error Units read avgt 50 227.054 ? 1.354 us/op read:?gc.alloc.rate.norm avgt 50 138605.638 ? 20.778 B/op readNoVerify avgt 50 226.606 ? 1.748 us/op readNoVerify:?gc.alloc.rate.norm avgt 50 137810.392 ? 7.020 B/op Would you sponsor the changes? Regards, Sergey Tsypanov 28.08.2020, 12:08, "Florian Weimer" : > * ?????? ???????: > >> ?@@ -105,12 +105,8 @@ >> ??????private byte[] getBytes(InputStream is) >> ??????????throws IOException >> ??????{ >> ?- byte[] buffer = new byte[8192]; >> ??????????ByteArrayOutputStream baos = new ByteArrayOutputStream(2048); >> ?- int n; >> ?- while ((n = is.read(buffer, 0, buffer.length)) != -1) { >> ?- baos.write(buffer, 0, n); >> ?- } >> ?+ is.transferTo(baos); >> ??????????return baos.toByteArray(); >> ??????} > > Isn't this InputStream::readAllBytes? > > Thanks, > Florian From fweimer at redhat.com Fri Aug 28 13:36:22 2020 From: fweimer at redhat.com (Florian Weimer) Date: Fri, 28 Aug 2020 15:36:22 +0200 Subject: [PATCH] optimization opportunity regarding misuse of BufferedInputStream In-Reply-To: <11808131598621180@mail.yandex.ru> (=?utf-8?B?ItCh0LXRgNCz?= =?utf-8?B?0LXQuSDQptGL0L/QsNC90L7QsiIncw==?= message of "Fri, 28 Aug 2020 15:29:47 +0200") References: <10579141598555707@mail.yandex.ru> <87v9h3rsl8.fsf@oldenburg2.str.redhat.com> <11808131598621180@mail.yandex.ru> Message-ID: <87y2lyriy1.fsf@oldenburg2.str.redhat.com> * ?????? ???????: > Hi, > >> Isn't this InputStream::readAllBytes? > > thanks for pointing this out! Indeed, InputStream::readAllBytes() allows to save even more memory: > > Mode Cnt Score Error Units > read avgt 50 227.054 ? 1.354 us/op > read:?gc.alloc.rate.norm avgt 50 138605.638 ? 20.778 B/op > readNoVerify avgt 50 226.606 ? 1.748 us/op > readNoVerify:?gc.alloc.rate.norm avgt 50 137810.392 ? 7.020 B/op > > Would you sponsor the changes? Sorry, I don't have that kind of role on the OpenJDK project. Florian From sean.coffey at oracle.com Fri Aug 28 14:17:11 2020 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Fri, 28 Aug 2020 15:17:11 +0100 Subject: RFR: 8250968: Symlinks attributes not preserved when using jarsigner on zip files In-Reply-To: <1582051D-5EDD-4F16-88D3-A064683023D3@ORACLE.COM> References: <957b3bd1-254a-647d-358b-e2dbfba6c6cf@oracle.com> <68CBFB5B-0CAD-44DE-B0E8-92221BACE7C9@ORACLE.COM> <99b55d3f-533e-ac91-89bd-1ce8f3cbd6db@oracle.com> <940A8A93-BB70-48CD-B1DE-0E9EC17E2791@ORACLE.COM> <3b9c3aa9-df9b-5ae3-2301-2e08c8e9f769@oracle.com> <1582051D-5EDD-4F16-88D3-A064683023D3@ORACLE.COM> Message-ID: I've been poking around the zip internals and am now able to locate the 16 bits of interest. The position of these actual bits does appear to move around from one test run to another. For now, I guess it's sufficient to look for the pattern of interest in the signed zip file. New testcase added. http://cr.openjdk.java.net/~coffeys/webrev.8250968.v4/webrev/ regards, Sean. On 27/08/2020 15:58, Weijun Wang wrote: >> Looks like it was a conscious design decision to only allow recording of POSIX permission bits for this field (& 0xFFF). I don't see anything about symlink support in zipfs docs. > As long as that*byte* is there and it?s not difficult to locate, we can manually add the*bit* for symlink and see if jarsigner can keep it. > > ?Max > From Roger.Riggs at oracle.com Fri Aug 28 14:22:01 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Fri, 28 Aug 2020 10:22:01 -0400 Subject: RFR 8251989: Hex encoder and decoder utility In-Reply-To: <6b9289fb-937d-70ca-55de-fa35f85797cb@gmail.com> References: <6b9289fb-937d-70ca-55de-fa35f85797cb@gmail.com> Message-ID: <9952e7ed-4c88-c7b9-9c37-38ce8e070eda@oracle.com> Hi Raffaello, Missed sending this yesterday, the changes are in the updated webrev. On 8/23/20 10:42 AM, Raffaello Giulietti wrote: ... > Hi Roger, > > here are my notes about the Decoder. > > (1) The only serious issue I could find is on L.548. It should read > L.548?? CharBuffer cb = CharBuffer.wrap(chars, index, index + length); > as method wrap() takes a start and an end. > (It's quite confusing and error prone to have both (start, length) and > (start, end) conventions on ranges all over the OpenJDK :-( Nobody's > fault, just annoying.) Well spotted, fixed. > > > The rest are minor notes. > > (2) While I'm pretty sure that a JIT compiler can remove the index range > check on L.465 when len is replaced by bytes.length on L.463, I'm less > sure that it can do it as currently coded, despite len and bytes.length > having the same value. Readability wouldn't suffer and len could be > removed. The JIT doesn't have trouble following those values. > > > (3) The expressions preLen > origLen - sufLen on L.492 and sufLen > > origLen - preLen on L.496 are equivalent. The latter is thus useless on > L.496 because it is always false when execution reaches here. true; updated to optimize the case of empty prefix and/or suffix. > > > (4) The code starting at L.588 assumes that delimiter.length() > 0, > which is ensured by the optimization in L.585-586. > Without this assumption, however, neither L.509 nor L.512 are correct > when delimiter.length() == 0. > To make the code to work even when, for some reason, delimiter.length() > == 0) the lines can be replaced by the slightly more involved yet more > robust > L.509?? if ((subcs.length() - 2) % stride != 0) > and > L.512?? final int len = (subcs.length() - 2) / stride + 1; fixed > > > (5) I guess that the rationale for not failing fast on illegal > characters in the loop on L.516-521, and accumulate and postpone the > check only after the whole cs has been processed, is that the code > optimistically assumes that cs is legal, so it would be wasteful to > check at each iteration. > But since each iteration already needs a quite more expensive check for > the delimiter, I wonder if the postponing makes much economical sense. > The check could be directly on v before assignment to bytes and the var > illegal could be removed. The code is easier to read with the inline checs and throws. > > > (6) What about moving L.517 at the end of the loop body so that memory > accesses are issued with strictly increasing and hardware predictable > addresses? This move would also better reveal the "left-to-right" > processing intent. ok, though with a lot of reordering and caching in hardware it won't make a difference. > > > (7) On L.561, delimiter.isEmpty() could be removed as checkDelimiter() > is not invoked when the delimiter is empty. But it's perhaps safer to > have it. It could be an assert, but asserts aren't enabled always > > > (8) You may want to wrap escapeNL() around the message on L.567-569 ok > > > (9) fromHexPair() and fromHex() can be declared static. > > > (10) equals() and hashCode() are missing despite the class being > value-based. This holds for Encoder as well. Added > > > I look forward for the next iteration of the code. Thanks? for the comments, Roger > > > Greetings > Raffaello > From sean.coffey at oracle.com Fri Aug 28 14:29:01 2020 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Fri, 28 Aug 2020 15:29:01 +0100 Subject: RFR: JDK-8249699: java/io/ByteArrayOutputStream/MaxCapacity.java should use @requires instead of @ignore In-Reply-To: <859c166d-0bf0-495d-abf6-3c1870a0c434@default> References: <859c166d-0bf0-495d-abf6-3c1870a0c434@default> Message-ID: <54c1d8c6-c451-4fee-fd1a-7ee9302c3a04@oracle.com> Apologies. Meant to reply yesterday. Your edit looks fine to me. regards, Sean. On 27/08/2020 16:41, Fernando Guallini wrote: > Thanks Sean, updated webrev here: http://cr.openjdk.java.net/~fguallini/8249699/webrev.01/ > > Regards, > Fernando > ----- Original Message ----- > From: sean.coffey at oracle.com > To: fernando.guallini at oracle.com, core-libs-dev at openjdk.java.net > Sent: Wednesday, 26 August, 2020 7:39:25 PM GMT +00:00 GMT Britain, Ireland, Portugal > Subject: Re: RFR: JDK-8249699: java/io/ByteArrayOutputStream/MaxCapacity.java should use @requires instead of @ignore > > test/jdk/java/util/Base64/TestEncodingDecodingLength.java is an example > of another test using -Xmx8g. Do you want to push the os.maxMemory > requirement up to 10g perhaps ? It might avoid border line resource > failures. Also I think it might need a "sun.arch.data.model == "64" " > requirement : > > @requires (sun.arch.data.model == "64" & os.maxMemory >= 10g) > > regards, > Sean. > > On 26/08/2020 18:17, Fernando Guallini wrote: >> Hi, >> >> Could I please get reviews and a sponsor for: >> >> webrev: http://cr.openjdk.java.net/~fguallini/8249699/webrev.00/ >> Testbug: https://bugs.openjdk.java.net/browse/JDK-8249699 >> >> The test was ignored due to ?huge memory requirements?, but with the jtreg current version, tests can be only run when system requirements are satisfied as opposed to being always excluded. It runs and passes in Mach5 >> >> >> Thanks >> >> -Fernando From alexander.scherbatiy at bell-sw.com Fri Aug 28 14:32:37 2020 From: alexander.scherbatiy at bell-sw.com (Alexander Scherbatiy) Date: Fri, 28 Aug 2020 17:32:37 +0300 Subject: RFR 8252248: __SIGRTMAX is not declared in musl libc In-Reply-To: References: <1c4ad320-a22a-484a-d370-852f9ecd90f8@bell-sw.com> <7c1ef537-3c36-c28a-c083-5bd8e5de8edf@bell-sw.com> Message-ID: <66d7be82-12ab-7d64-ee24-988c2b050dd5@bell-sw.com> On 28.08.2020 15:55, Alan Bateman wrote: > On 28/08/2020 12:25, Alexander Scherbatiy wrote: >> Hello, >> >> Could you review the updated fix: >> ? http://cr.openjdk.java.net/~alexsch/8252248/webrev.01/ >> >> ?- moving shared code to net_util_md.h is avoided >> ?- INTERRUPT_SIGNAL is defined as SIGRTMAX - 2 instead of __SIGRTMAX >> - 2 for Linux in NativeThread.c >> ?- "#include " is moved out from "#ifdef" in NativeThread.c >> ?- sigWakeup is changed to "#define WAKEUP_SIGNAL (SIGRTMAX - 2)" in >> linux_close.c >> >> At least test/jdk/java/net/Socket/asyncClose/AsyncClose.java touches >> all four methods where the signal changes are used: >> NativeThread.c >> - Java_sun_nio_ch_NativeThread_init(JNIEnv *env, jclass cl) >> - Java_sun_nio_ch_NativeThread_signal(JNIEnv *env, jclass cl, jlong >> thread) >> linux_close.c >> -? __attribute((constructor)) init() >> - closefd(int fd1, int fd2) > NativeThread current/signal will be exercised by the tests in > java/nio/channels. ? I run the java/nio/channels tests with the fix. There are five failed tests that fail with and without the fix: ??? java/nio/channels/DatagramChannel/AdaptorMulticasting.java ??? java/nio/channels/DatagramChannel/MulticastSendReceiveTests.java ??? java/nio/channels/DatagramChannel/PromiscuousIPv6.java ??? java/nio/channels/DatagramChannel/Loopback.java ??? java/nio/channels/FileChannel/FileExtensionAndMap.java The FileExtensionAndMap.java has clear fail message: "Error. Test ignored: This test has huge disk space requirements". Thanks, Alexander. > The code in linux_close.c is only used with the old (and not used by > default) SocketImpl and DatagramSocketImpl implementation. Some of the > tests jdk_net test group will re-run the tests with the old > implementation so you should be okay. > >> >> I run test/jdk/java/net tests on Ubuntu 18.04.4 and Alpine Linux >> 3.11.6 with musl libc 1.1.24. >> There are 10 failed tests but they fail without the fix as well. I >> believe that it is because of some network settings of? my machine. > Sometimes there is will firewall or iptables configuration need to > allow multicast datagrams to be received. > > -Alan From Alan.Bateman at oracle.com Fri Aug 28 14:40:38 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 28 Aug 2020 15:40:38 +0100 Subject: RFR 8252248: __SIGRTMAX is not declared in musl libc In-Reply-To: <66d7be82-12ab-7d64-ee24-988c2b050dd5@bell-sw.com> References: <1c4ad320-a22a-484a-d370-852f9ecd90f8@bell-sw.com> <7c1ef537-3c36-c28a-c083-5bd8e5de8edf@bell-sw.com> <66d7be82-12ab-7d64-ee24-988c2b050dd5@bell-sw.com> Message-ID: On 28/08/2020 15:32, Alexander Scherbatiy wrote: > > ? I run the java/nio/channels tests with the fix. There are five > failed tests that fail with and without the fix: > ??? java/nio/channels/DatagramChannel/AdaptorMulticasting.java > java/nio/channels/DatagramChannel/MulticastSendReceiveTests.java > ??? java/nio/channels/DatagramChannel/PromiscuousIPv6.java > ??? java/nio/channels/DatagramChannel/Loopback.java > ??? java/nio/channels/FileChannel/FileExtensionAndMap.java > > The FileExtensionAndMap.java has clear fail message: "Error. Test > ignored: This test has huge disk space requirements". The DatagramChannel failures are probably because of firewall or iptables issues too.? The FileExtensionAndMap test has @ignore so it will be skipped, maybe you are running jtreg directly and aren't use -ignore:quiet? In any case, I think the changes look okay. -Alan From alexander.scherbatiy at bell-sw.com Fri Aug 28 14:54:19 2020 From: alexander.scherbatiy at bell-sw.com (Alexander Scherbatiy) Date: Fri, 28 Aug 2020 17:54:19 +0300 Subject: RFR 8252248: __SIGRTMAX is not declared in musl libc In-Reply-To: References: <1c4ad320-a22a-484a-d370-852f9ecd90f8@bell-sw.com> <7c1ef537-3c36-c28a-c083-5bd8e5de8edf@bell-sw.com> <66d7be82-12ab-7d64-ee24-988c2b050dd5@bell-sw.com> Message-ID: On 28.08.2020 17:40, Alan Bateman wrote: > On 28/08/2020 15:32, Alexander Scherbatiy wrote: >> >> ? I run the java/nio/channels tests with the fix. There are five >> failed tests that fail with and without the fix: >> ??? java/nio/channels/DatagramChannel/AdaptorMulticasting.java >> java/nio/channels/DatagramChannel/MulticastSendReceiveTests.java >> ??? java/nio/channels/DatagramChannel/PromiscuousIPv6.java >> ??? java/nio/channels/DatagramChannel/Loopback.java >> ??? java/nio/channels/FileChannel/FileExtensionAndMap.java >> >> The FileExtensionAndMap.java has clear fail message: "Error. Test >> ignored: This test has huge disk space requirements". > The DatagramChannel failures are probably because of firewall or > iptables issues too.? The FileExtensionAndMap test has @ignore so it > will be skipped, maybe you are running jtreg directly and aren't use > -ignore:quiet? ? Yes, I run the jtreg tests directly without the -ignore:quiet option. ? Thanks, ? Alexander. > > In any case, I think the changes look okay. > > -Alan From WEIJUN.WANG at ORACLE.COM Fri Aug 28 15:18:18 2020 From: WEIJUN.WANG at ORACLE.COM (Weijun Wang) Date: Fri, 28 Aug 2020 11:18:18 -0400 Subject: RFR: 8250968: Symlinks attributes not preserved when using jarsigner on zip files In-Reply-To: References: <957b3bd1-254a-647d-358b-e2dbfba6c6cf@oracle.com> <68CBFB5B-0CAD-44DE-B0E8-92221BACE7C9@ORACLE.COM> <99b55d3f-533e-ac91-89bd-1ce8f3cbd6db@oracle.com> <940A8A93-BB70-48CD-B1DE-0E9EC17E2791@ORACLE.COM> <3b9c3aa9-df9b-5ae3-2301-2e08c8e9f769@oracle.com> <1582051D-5EDD-4F16-88D3-A064683023D3@ORACLE.COM> Message-ID: <68E22623-B1E8-4F44-83B7-42B87EED89E9@ORACLE.COM> 1. Add a comment on how to generate ZIPBYTES in the test. Not the byte[] declaration but how the original ZIP file is generated. 2. Does this require a CSR? The POSIX permission one had one. Thanks, Max > On Aug 28, 2020, at 10:17 AM, Se?n Coffey wrote: > > I've been poking around the zip internals and am now able to locate the 16 bits of interest. The position of these actual bits does appear to move around from one test run to another. For now, I guess it's sufficient to look for the pattern of interest in the signed zip file. New testcase added. > > http://cr.openjdk.java.net/~coffeys/webrev.8250968.v4/webrev/ > > regards, > Sean. > > On 27/08/2020 15:58, Weijun Wang wrote: >>> Looks like it was a conscious design decision to only allow recording of POSIX permission bits for this field (& 0xFFF). I don't see anything about symlink support in zipfs docs. >>> >> As long as that *byte* is there and it?s not difficult to locate, we can manually add the *bit* >> for symlink and see if jarsigner can keep it. >> >> ?Max >> >> From vyommani at gmail.com Fri Aug 28 15:40:24 2020 From: vyommani at gmail.com (Vyom Tiwari) Date: Fri, 28 Aug 2020 21:10:24 +0530 Subject: RFR 8252248: __SIGRTMAX is not declared in musl libc In-Reply-To: References: <1c4ad320-a22a-484a-d370-852f9ecd90f8@bell-sw.com> <7c1ef537-3c36-c28a-c083-5bd8e5de8edf@bell-sw.com> <66d7be82-12ab-7d64-ee24-988c2b050dd5@bell-sw.com> Message-ID: +1 Vyom On Fri, Aug 28, 2020 at 8:24 PM Alexander Scherbatiy < alexander.scherbatiy at bell-sw.com> wrote: > On 28.08.2020 17:40, Alan Bateman wrote: > > > On 28/08/2020 15:32, Alexander Scherbatiy wrote: > >> > >> I run the java/nio/channels tests with the fix. There are five > >> failed tests that fail with and without the fix: > >> java/nio/channels/DatagramChannel/AdaptorMulticasting.java > >> java/nio/channels/DatagramChannel/MulticastSendReceiveTests.java > >> java/nio/channels/DatagramChannel/PromiscuousIPv6.java > >> java/nio/channels/DatagramChannel/Loopback.java > >> java/nio/channels/FileChannel/FileExtensionAndMap.java > >> > >> The FileExtensionAndMap.java has clear fail message: "Error. Test > >> ignored: This test has huge disk space requirements". > > The DatagramChannel failures are probably because of firewall or > > iptables issues too. The FileExtensionAndMap test has @ignore so it > > will be skipped, maybe you are running jtreg directly and aren't use > > -ignore:quiet? > Yes, I run the jtreg tests directly without the -ignore:quiet option. > > Thanks, > Alexander. > > > > In any case, I think the changes look okay. > > > > -Alan > -- Thanks, Vyom From sean.coffey at oracle.com Fri Aug 28 16:05:12 2020 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Fri, 28 Aug 2020 17:05:12 +0100 Subject: RFR: 8250968: Symlinks attributes not preserved when using jarsigner on zip files In-Reply-To: <68E22623-B1E8-4F44-83B7-42B87EED89E9@ORACLE.COM> References: <957b3bd1-254a-647d-358b-e2dbfba6c6cf@oracle.com> <68CBFB5B-0CAD-44DE-B0E8-92221BACE7C9@ORACLE.COM> <99b55d3f-533e-ac91-89bd-1ce8f3cbd6db@oracle.com> <940A8A93-BB70-48CD-B1DE-0E9EC17E2791@ORACLE.COM> <3b9c3aa9-df9b-5ae3-2301-2e08c8e9f769@oracle.com> <1582051D-5EDD-4F16-88D3-A064683023D3@ORACLE.COM> <68E22623-B1E8-4F44-83B7-42B87EED89E9@ORACLE.COM> Message-ID: On 28/08/2020 16:18, Weijun Wang wrote: > 1. Add a comment on how to generate ZIPBYTES in the test. Not the byte[] declaration but how the original ZIP file is generated. I'll add a comment block to the test: > ??? /* > ???? * Created using the createByteArray utility method. > ???? * The zipfile itself was created via this example: > ???? * $ ls -l z > ???? * lrwxrwxrwx 1 test test 4 Aug 27 18:33 z -> ../z > ???? * $ zip -ry test.zip z > ???? */ > > 2. Does this require a CSR? The POSIX permission one had one. Fair point. I've logged one, just to be safe. regards, Sean. > > Thanks, > Max > >> On Aug 28, 2020, at 10:17 AM, Se?n Coffey wrote: >> >> I've been poking around the zip internals and am now able to locate the 16 bits of interest. The position of these actual bits does appear to move around from one test run to another. For now, I guess it's sufficient to look for the pattern of interest in the signed zip file. New testcase added. >> >> http://cr.openjdk.java.net/~coffeys/webrev.8250968.v4/webrev/ >> >> regards, >> Sean. >> >> On 27/08/2020 15:58, Weijun Wang wrote: >>>> Looks like it was a conscious design decision to only allow recording of POSIX permission bits for this field (& 0xFFF). I don't see anything about symlink support in zipfs docs. >>>> >>> As long as that *byte* is there and it?s not difficult to locate, we can manually add the *bit* >>> for symlink and see if jarsigner can keep it. >>> >>> ?Max >>> >>> From raffaello.giulietti at gmail.com Fri Aug 28 16:08:40 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Fri, 28 Aug 2020 18:08:40 +0200 Subject: RFR 8251989: Hex encoder and decoder utility In-Reply-To: <9952e7ed-4c88-c7b9-9c37-38ce8e070eda@oracle.com> References: <6b9289fb-937d-70ca-55de-fa35f85797cb@gmail.com> <9952e7ed-4c88-c7b9-9c37-38ce8e070eda@oracle.com> Message-ID: <8985ea26-3847-c9b2-c52e-20f9e362f7a4@gmail.com> Hi Roger, I'm reading this version http://cr.openjdk.java.net/~rriggs/webrev-hex-formatter-8251989-1/src/java.base/share/classes/java/util/Hex.java On 2020-08-28 16:22, Roger Riggs wrote: > Hi Raffaello, > > Missed sending this yesterday, the changes are in the updated webrev. > > On 8/23/20 10:42 AM, Raffaello Giulietti wrote: > > ... >> (3) The expressions preLen > origLen - sufLen on L.492 and sufLen > >> origLen - preLen on L.496 are equivalent. The latter is thus useless on >> L.496 because it is always false when execution reaches here. > true; updated to optimize the case of empty prefix and/or suffix. >> >> While the lines are now numbered L.552 and L.556, this note still holds. In other words, L.556-557 if (sufLen > origLen - preLen || !suffix.contentEquals(cs.subSequence(origLen - sufLen, origLen))) { could be replaced with the simpler if (!suffix.contentEquals(cs.subSequence(origLen - sufLen, origLen))) { >> (7) On L.561, delimiter.isEmpty() could be removed as checkDelimiter() >> is not invoked when the delimiter is empty. But it's perhaps safer to >> have it. > It could be an assert, but asserts aren't enabled always It could be an explicit throw new AssertionError(...), but I'm not sure it's worth the change. Another quick note. Complementing toHex(int value), what about exposing the additional public char toHighHex(int value) { return (char)digits[value >> 4 & 0xf]; } It would useful both to replace the various ">> 4" shifts in the code itself and externally if one isn't willing to pay for the allocation of the String returned by toHexPair() and prefers to deal with the two individual chars. > Thanks? for the comments, > > Roger > Has been a pleasure Greetings Raffaello From WEIJUN.WANG at ORACLE.COM Fri Aug 28 16:15:41 2020 From: WEIJUN.WANG at ORACLE.COM (Weijun Wang) Date: Fri, 28 Aug 2020 12:15:41 -0400 Subject: RFR: 8250968: Symlinks attributes not preserved when using jarsigner on zip files In-Reply-To: References: <957b3bd1-254a-647d-358b-e2dbfba6c6cf@oracle.com> <68CBFB5B-0CAD-44DE-B0E8-92221BACE7C9@ORACLE.COM> <99b55d3f-533e-ac91-89bd-1ce8f3cbd6db@oracle.com> <940A8A93-BB70-48CD-B1DE-0E9EC17E2791@ORACLE.COM> <3b9c3aa9-df9b-5ae3-2301-2e08c8e9f769@oracle.com> <1582051D-5EDD-4F16-88D3-A064683023D3@ORACLE.COM> <68E22623-B1E8-4F44-83B7-42B87EED89E9@ORACLE.COM> Message-ID: <35266390-1DC1-4E69-A946-CD4989A3C238@ORACLE.COM> Everything looks fine to me. I?ve added myself as the CSR reviewer. In the Solution section, it probably should me ?The existing warning introduced in JDK-8218021 is expanded to include symlink?. It is not a new warning. Thanks, Max > On Aug 28, 2020, at 12:05 PM, Se?n Coffey wrote: > > > On 28/08/2020 16:18, Weijun Wang wrote: >> 1. Add a comment on how to generate ZIPBYTES in the test. Not the byte[] declaration but how the original ZIP file is generated. > I'll add a comment block to the test: >> /* >> * Created using the createByteArray utility method. >> * The zipfile itself was created via this example: >> * $ ls -l z >> * lrwxrwxrwx 1 test test 4 Aug 27 18:33 z -> ../z >> * $ zip -ry test.zip z >> */ > >> >> 2. Does this require a CSR? The POSIX permission one had one. > > Fair point. I've logged one, just to be safe. > > regards, > Sean. > >> >> Thanks, >> Max >> >>> On Aug 28, 2020, at 10:17 AM, Se?n Coffey wrote: >>> >>> I've been poking around the zip internals and am now able to locate the 16 bits of interest. The position of these actual bits does appear to move around from one test run to another. For now, I guess it's sufficient to look for the pattern of interest in the signed zip file. New testcase added. >>> >>> http://cr.openjdk.java.net/~coffeys/webrev.8250968.v4/webrev/ >>> >>> regards, >>> Sean. >>> >>> On 27/08/2020 15:58, Weijun Wang wrote: >>>>> Looks like it was a conscious design decision to only allow recording of POSIX permission bits for this field (& 0xFFF). I don't see anything about symlink support in zipfs docs. >>>>> >>>> As long as that *byte* is there and it?s not difficult to locate, we can manually add the *bit* >>>> for symlink and see if jarsigner can keep it. >>>> >>>> ?Max >>>> >>>> From thomas.stuefe at gmail.com Fri Aug 28 16:24:13 2020 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 28 Aug 2020 18:24:13 +0200 Subject: RFR 8252248: __SIGRTMAX is not declared in musl libc In-Reply-To: <7c1ef537-3c36-c28a-c083-5bd8e5de8edf@bell-sw.com> References: <1c4ad320-a22a-484a-d370-852f9ecd90f8@bell-sw.com> <7c1ef537-3c36-c28a-c083-5bd8e5de8edf@bell-sw.com> Message-ID: Looks good to me. Thanks for checking. Cheers, Thomas On Fri, Aug 28, 2020 at 1:25 PM Alexander Scherbatiy < alexander.scherbatiy at bell-sw.com> wrote: > Hello, > > Could you review the updated fix: > http://cr.openjdk.java.net/~alexsch/8252248/webrev.01/ > > - moving shared code to net_util_md.h is avoided > - INTERRUPT_SIGNAL is defined as SIGRTMAX - 2 instead of __SIGRTMAX - > 2 for Linux in NativeThread.c > - "#include " is moved out from "#ifdef" in NativeThread.c > - sigWakeup is changed to "#define WAKEUP_SIGNAL (SIGRTMAX - 2)" in > linux_close.c > > At least test/jdk/java/net/Socket/asyncClose/AsyncClose.java touches all > four methods where the signal changes are used: > NativeThread.c > - Java_sun_nio_ch_NativeThread_init(JNIEnv *env, jclass cl) > - Java_sun_nio_ch_NativeThread_signal(JNIEnv *env, jclass cl, jlong thread) > linux_close.c > - __attribute((constructor)) init() > - closefd(int fd1, int fd2) > > I run test/jdk/java/net tests on Ubuntu 18.04.4 and Alpine Linux 3.11.6 > with musl libc 1.1.24. > There are 10 failed tests but they fail without the fix as well. I > believe that it is because of some network settings of my machine. > > java/net/MulticastSocket/IPMulticastIF.java > java/net/MulticastSocket/NoSetNetworkInterface.java > java/net/MulticastSocket/Promiscuous.java > java/net/MulticastSocket/PromiscuousIPv6.java > java/net/MulticastSocket/SetGetNetworkInterfaceTest.java > java/net/MulticastSocket/SetLoopbackMode.java > java/net/MulticastSocket/SetOutgoingIf.java > java/net/MulticastSocket/Test.java > java/net/NetworkInterface/NetworkInterfaceRetrievalTests.java > java/net/ipv6tests/UdpTest.java > > Thanks, > Alexander. > > On 26.08.2020 13:42, Alan Bateman wrote: > > On 25/08/2020 18:00, Alexander Scherbatiy wrote: > >> > >> Hello, > >> > >> Could your review the fix for the issue: > >> Bug: https://bugs.openjdk.java.net/browse/JDK-8252248 > >> Fix: http://cr.openjdk.java.net/~alexsch/8252248/webrev.00/ > >> > >> : > >> > >> The fix has been discussed on the portola-dev alias [2] where it was > >> pointed out that the fix can be reviewed in the mainline and it was > >> suggested to rename the INTERRUPT_SIGNAL and move its definition to > >> net_util_md.h. > >> > > The xxx_close.c sources are for the old SocketImpl and > > DatagramSocketImpl implementations; they are aren't used by default > > and will eventually go away. Maybe an option for the musl port is to > > just not compile in the old socket implementations? It might be best > > to start a discussion on nio-dev and net-dev about this as we > > shouldn't be changing NativeThread.c (which is used for signalling in > > the FileChannel implementation) to include net_util.h. > > > > -Alan > From Roger.Riggs at oracle.com Fri Aug 28 17:54:42 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Fri, 28 Aug 2020 13:54:42 -0400 Subject: RFR(M): 8248188: [PATCH] Add HotSpotIntrinsicCandidate and API for Base64 decoding In-Reply-To: <8ece8d2e-fd99-b734-211e-a32b534a7dc8@linux.ibm.com> References: <11ca749f-3015-c004-aa6b-3194e1dfe4eb@linux.ibm.com> <8ece8d2e-fd99-b734-211e-a32b534a7dc8@linux.ibm.com> Message-ID: <8d53dcf8-635a-11e2-4f6a-39b70e2c3b8b@oracle.com> Hi Corey, A few comments on core-libs side... The naming convention for methods that end in '0' is usually to indicate they are the bottom-most method or a native method. So I think you can/should rename the methods to make the most sense as to their function. Comparing with the way that the Base64 encoder was intrinsified, the method that is intrinsified should have a method body that does the same function, so it is interchangable.? That likely will just shift the "fast path" code into the decodeBlock method. Keeping the symmetry between encoder and decoder will make it easier to maintain the code. Given intrinsic only handles 2 of the three cases, and the java code handles all three, I would add an extra arg to decodeBlock to reflect the isMime case and have the intrinsic take an early exit until it was implemented. It is unfortunate that taking advantage of vectorization has to be hand coded. If/when the Vector API is ready (JEP 338 https://openjdk.java.net/jeps/338) the java code should be replaced to use the Vector API and then it would work for a new hardware without specific coding for each platform. "Just" implement the Vector API.? There's a lot more bang for the buck going for that approach. Thanks, Roger On 8/24/20 9:21 PM, Corey Ashford wrote: > Here's a revised webrev which includes a JMH benchmark for the decode > operation. > > http://cr.openjdk.java.net/~mhorie/8248188/webrev.03/ > > The added benchmark tries to be "fair" in that it doesn't prefer a > large buffer size, which would favor the intrinsic.? It > pseudo-randomly (but reproducibly) chooses a buffer size between 8 and > 20k+8 bytes, and fills it with random data to encode and decode.? As > part of the TearDown of an invocation, it also checks the decoded > output data for correctness. > > Example runs on the Power9-based machine I use for development shows a > 3X average improvement across these random buffer sizes. Here's an > excerpt of the output when run with -XX:-UseBASE64Intrinsics : > > Iteration?? 1: 70795.623 ops/s > Iteration?? 2: 71070.607 ops/s > Iteration?? 3: 70867.544 ops/s > Iteration?? 4: 71107.992 ops/s > Iteration?? 5: 71048.281 ops/s > > And here's the output with the intrinsic enabled: > > Iteration?? 1: 208794.022 ops/s > Iteration?? 2: 208630.904 ops/s > Iteration?? 3: 208238.822 ops/s > Iteration?? 4: 208714.967 ops/s > Iteration?? 5: 209060.894 ops/s > > Taking the best of the two runs: 209060/71048 = 2.94 > > From other experiments where the benchmark uses a fixed-size, larger > buffer, the performance ratio rises to about 4.0. > > Power10 should have a slightly higher ratio due to several factors, > but I have not yet benchmarked on Power10. > > Other arches ought to be able to do at least this well, if not better, > because of wider vector registers (> 128 bits) being available.? Only > a Power9/10 implementation is included in this webrev, however. > > Regards, > > - Corey > > > On 8/19/20 11:20 AM, Roger Riggs wrote: >> Hi Corey, >> >> For changes obviously performance motivated, it is conventional to >> run a JMH perf test to demonstate >> the improvement and prove it is worthwhile to add code complexity. >> >> I don't see any existing Base64 JMH tests but they would be in the >> repo below or near: >> ???? test/micro/org/openjdk/bench/java/util/ >> >> Please contribute a JMH test and results to show the difference. >> >> Regards, Roger >> >> >> >> On 8/19/20 2:10 PM, Corey Ashford wrote: >>> Michihiro Horie posted up a new iteration of this webrev for me.? >>> This time the webrev includes a complete implementation of the >>> intrinsic for Power9 and Power10. >>> >>> You can find it here: >>> http://cr.openjdk.java.net/~mhorie/8248188/webrev.02/ >>> >>> Changes in webrev.02 vs. webrev.01: >>> >>> ? * The method header for the intrinsic in the Base64 code has been >>> rewritten using the Javadoc style.? The clarity of the comments has >>> been improved and some verbosity has been removed. There are no >>> additional functional changes to Base64.java. >>> >>> ? * The code needed to martial and check the intrinsic parameters >>> has been added, using the base64 encodeBlock intrinsic as a guideline. >>> >>> ? * A complete intrinsic implementation for Power9 and Power10 is >>> included. >>> >>> ? * Adds some Power9 and Power10 assembler instructions needed by >>> the intrinsic which hadn't been defined before. >>> >>> The intrinsic implementation in this patch accelerates the decoding >>> of large blocks of base64 data by a factor of about 3.5X on Power9. >>> >>> I'm attaching two Java test cases I am using for testing and >>> benchmarking.? The TestBase64_VB encodes and decodes randomly-sized >>> buffers of random data and checks that original data matches the >>> encoded-then-decoded data.? TestBase64Errors encodes a 48K block of >>> random bytes, then corrupts each byte of the encoded data, one at a >>> time, checking to see if the decoder catches the illegal byte. >>> >>> Any comments/suggestions would be appreciated. >>> >>> Thanks, >>> >>> - Corey >>> >>> On 7/27/20 6:49 PM, Corey Ashford wrote: >>>> Michihiro Horie uploaded a new revision of the Base64 decodeBlock >>>> intrinsic API for me: >>>> >>>> http://cr.openjdk.java.net/~mhorie/8248188/webrev.01/ >>>> >>>> It has the following changes with respect to the original one posted: >>>> >>>> ??* In the event of encountering a non-base64 character, instead of >>>> having a separate error code of -1, the intrinsic can now just >>>> return either 0, or the number of data bytes produced up to the >>>> point where the illegal base64 character was encountered. This >>>> reduces the number of special cases, and also provides a way to >>>> speed up the process of finding the bad character by the slower, >>>> pure-Java algorithm. >>>> >>>> ??* The isMIME boolean is removed from the API for two reasons: >>>> ??? - The current API is not sufficient to handle the isMIME case, >>>> because there isn't a strict relationship between the number of >>>> input bytes and the number of output bytes, because there can be an >>>> arbitrary number of non-base64 characters in the source. >>>> ??? - If an intrinsic only implements the (isMIME == false) case as >>>> ours does, it will always return 0 bytes processed, which will >>>> slightly slow down the normal path of processing an (isMIME == >>>> true) instantiation. >>>> ??? - We considered adding a separate hotspot candidate for the >>>> (isMIME == true) case, but since we don't have an intrinsic >>>> implementation to test that, we decided to leave it as a future >>>> optimization. >>>> >>>> Comments and suggestions are welcome.? Thanks for your consideration. >>>> >>>> - Corey >>>> >>>> On 6/23/20 6:23 PM, Michihiro Horie wrote: >>>>> Hi Corey, >>>>> >>>>> Following is the issue I created. >>>>> https://bugs.openjdk.java.net/browse/JDK-8248188 >>>>> >>>>> I will upload a webrev when you're ready as we talked in private. >>>>> >>>>> Best regards, >>>>> Michihiro >>>>> >>>>> Inactive hide details for "Corey Ashford" ---2020/06/24 >>>>> 09:40:10---Currently in java.util.Base64, there is a >>>>> HotSpotIntrinsicCa"Corey Ashford" ---2020/06/24 >>>>> 09:40:10---Currently in java.util.Base64, there is a >>>>> HotSpotIntrinsicCandidate and API for encodeBlock, but no >>>>> >>>>> From: "Corey Ashford" >>>>> To: "hotspot-compiler-dev at openjdk.java.net" >>>>> , >>>>> "ppc-aix-port-dev at openjdk.java.net" >>>>> >>>>> Cc: Michihiro Horie/Japan/IBM at IBMJP, Kazunori >>>>> Ogata/Japan/IBM at IBMJP, joserz at br.ibm.com >>>>> Date: 2020/06/24 09:40 >>>>> Subject: RFR(S): [PATCH] Add HotSpotIntrinsicCandidate and API for >>>>> Base64 decoding >>>>> >>>>> ------------------------------------------------------------------------ >>>>> >>>>> >>>>> >>>>> >>>>> Currently in java.util.Base64, there is a >>>>> HotSpotIntrinsicCandidate and >>>>> API for encodeBlock, but none for decoding. ?This means that only >>>>> encoding gets acceleration from the underlying CPU's vector hardware. >>>>> >>>>> I'd like to propose adding a new intrinsic for decodeBlock. ?The >>>>> considerations I have for this new intrinsic's API: >>>>> >>>>> ??* Don't make any assumptions about the underlying capability of the >>>>> hardware. ?For example, do not impose any specific block size >>>>> granularity. >>>>> >>>>> ??* Don't assume the underlying intrinsic can handle isMIME or isURL >>>>> modes, but also let them decide if they will process the data >>>>> regardless >>>>> of the settings of the two booleans. >>>>> >>>>> ??* Any remaining data that is not processed by the intrinsic will be >>>>> processed by the pure Java implementation. ?This allows the >>>>> intrinsic to >>>>> process whatever block sizes it's good at without the complexity of >>>>> handling the end fragments. >>>>> >>>>> ??* If any illegal character is discovered in the decoding >>>>> process, the >>>>> intrinsic will simply return -1, instead of requiring it to throw a >>>>> proper exception from the context of the intrinsic. ?In the event of >>>>> getting a -1 returned from the intrinsic, the Java Base64 library >>>>> code >>>>> simply calls the pure Java implementation to have it find the >>>>> error and >>>>> properly throw an exception. ?This is a performance trade-off in the >>>>> case of an error (which I expect to be very rare). >>>>> >>>>> ??* One thought I have for a further optimization (not implemented in >>>>> the current patch), is that when the intrinsic decides not to >>>>> process a >>>>> block because of some combination of isURL and isMIME settings it >>>>> doesn't handle, it could return extra bits in the return code, >>>>> encoded >>>>> as a negative number. ?For example: >>>>> >>>>> Illegal_Base64_char ? = 0b001; >>>>> isMIME_unsupported ? ?= 0b010; >>>>> isURL_unsupported ? ? = 0b100; >>>>> >>>>> These can be OR'd together as needed and then negated (flip the >>>>> sign). >>>>> The Base64 library code could then cache these flags, so it will know >>>>> not to call the intrinsic again when another decodeBlock is requested >>>>> but with an unsupported mode. ?This will save the performance hit of >>>>> calling the intrinsic when it is guaranteed to fail. >>>>> >>>>> I've tested the attached patch with an actual intrinsic coded up for >>>>> Power9/Power10, but those runtime intrinsics and arch-specific >>>>> patches >>>>> aren't attached today. ?I want to get some consensus on the >>>>> library-level intrinsic API first. >>>>> >>>>> Also attached is a simple test case to test that the new intrinsic >>>>> API >>>>> doesn't break anything. >>>>> >>>>> I'm open to any comments about this. >>>>> >>>>> Thanks for your consideration, >>>>> >>>>> - Corey >>>>> >>>>> >>>>> Corey Ashford >>>>> IBM Systems, Linux Technology Center, OpenJDK team >>>>> cjashfor at us dot ibm dot com >>>>> [attachment "decodeBlock_api-20200623.patch" deleted by Michihiro >>>>> Horie/Japan/IBM] [attachment "TestBase64.java" deleted by >>>>> Michihiro Horie/Japan/IBM] >>>>> >>>>> >>>> >>> >> > From fernando.guallini at oracle.com Fri Aug 28 18:13:54 2020 From: fernando.guallini at oracle.com (Fernando Guallini) Date: Fri, 28 Aug 2020 11:13:54 -0700 (PDT) Subject: RFR: 8249694 - [TestBug] java/lang/StringBuffer/HugeCapacity.java and j/l/StringBuilder/HugeCapacity.java tests shouldn't be @ignore-d Message-ID: Hi, May I please get reviews and a sponsor for this trivial change: webrev: http://cr.openjdk.java.net/~fguallini/8249694/webrev.00/ Testbug: https://bugs.openjdk.java.net/browse/JDK-8249694 Tests do not need to have ?@ignore' because with @requires os.maxMemory is enough to ensure they will not be executed if memory requirements are not satisfied. They run in Mach5 with no issues. Thanks -Fernando From hai-may.chao at oracle.com Fri Aug 28 20:08:58 2020 From: hai-may.chao at oracle.com (Hai-May Chao) Date: Fri, 28 Aug 2020 13:08:58 -0700 Subject: RFR: 8250968: Symlinks attributes not preserved when using jarsigner on zip files In-Reply-To: References: <957b3bd1-254a-647d-358b-e2dbfba6c6cf@oracle.com> <68CBFB5B-0CAD-44DE-B0E8-92221BACE7C9@ORACLE.COM> <99b55d3f-533e-ac91-89bd-1ce8f3cbd6db@oracle.com> <940A8A93-BB70-48CD-B1DE-0E9EC17E2791@ORACLE.COM> <3b9c3aa9-df9b-5ae3-2301-2e08c8e9f769@oracle.com> <1582051D-5EDD-4F16-88D3-A064683023D3@ORACLE.COM> Message-ID: <8946D5E0-8438-43EC-A987-4FEBF5779343@oracle.com> JarSigner.java #953: The output debug message can be removed from the code. JavaUtilZipFileAccess.java #44: Change posixPerms to extraAttrs. ZipFile.java #661: Suggest to keep the comment and update it with the additional 4 bits for symlink. The rest of code changes and CSR look good. Thanks, Hai-May > On Aug 28, 2020, at 7:17 AM, Se?n Coffey wrote: > > I've been poking around the zip internals and am now able to locate the 16 bits of interest. The position of these actual bits does appear to move around from one test run to another. For now, I guess it's sufficient to look for the pattern of interest in the signed zip file. New testcase added. > > http://cr.openjdk.java.net/~coffeys/webrev.8250968.v4/webrev/ > regards, > Sean. > > On 27/08/2020 15:58, Weijun Wang wrote: >>> Looks like it was a conscious design decision to only allow recording of POSIX permission bits for this field (& 0xFFF). I don't see anything about symlink support in zipfs docs. >> As long as that *byte* is there and it?s not difficult to locate, we can manually add the *bit* for symlink and see if jarsigner can keep it. >> >> ?Max >> From ian.graves at oracle.com Fri Aug 28 20:12:54 2020 From: ian.graves at oracle.com (Ian Graves) Date: Fri, 28 Aug 2020 15:12:54 -0500 Subject: [PATCH] Docfix for Unsafe regarding fence methods Message-ID: Hello, When reviewing some of the documentation in jdk.internal.misc.Unsafe, it appears that there is some dated information regarding the availability of ?pure barrier? methods (i.e. pure StoreStoreFence sans a LoadStore barrier). When the storeFence() and loadFence() methods were implemented the pure variations (loadLoadFence(), storeStoreFence()) were not implemented and the documentation indicated as much. This is no longer the case. I?ve included a patch with this email that updates the docs to reflect the new state of affairs here as well as indicate some implementation details as to how the pure variants utilize the impure original methods. Thanks! Ian From ian.graves at oracle.com Fri Aug 28 20:21:16 2020 From: ian.graves at oracle.com (Ian Graves) Date: Fri, 28 Aug 2020 15:21:16 -0500 Subject: [PATCH] Docfix for Unsafe regarding fence methods In-Reply-To: References: Message-ID: <70AA9B93-A9AA-4FD0-8EBA-AE3A8DF473A0@oracle.com> Looks like the patch attachment was stripped. I?ll include it inline: ?Ian diff -r 3e9d813ff918 src/java.base/share/classes/jdk/internal/misc/Unsafe.java --- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java Thu Aug 27 20:20:39 2020 +0200 +++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java Fri Aug 28 15:21:01 2020 -0500 @@ -3412,9 +3412,8 @@ * Corresponds to C11 atomic_thread_fence(memory_order_acquire) * (an "acquire fence"). * - * A pure LoadLoad fence is not provided, since the addition of LoadStore - * is almost always desired, and most current hardware instructions that - * provide a LoadLoad barrier also provide a LoadStore barrier for free. + * Provides a LoadLoad barrier followed by a LoadStore barrier. + * * @since 1.8 */ @HotSpotIntrinsicCandidate @@ -3427,9 +3426,8 @@ * Corresponds to C11 atomic_thread_fence(memory_order_release) * (a "release fence"). * - * A pure StoreStore fence is not provided, since the addition of LoadStore - * is almost always desired, and most current hardware instructions that - * provide a StoreStore barrier also provide a LoadStore barrier for free. + * Provides a StoreStore barrier followed by a LoadStore barrier. + * * @since 1.8 */ @HotSpotIntrinsicCandidate @@ -3450,6 +3448,12 @@ /** * Ensures that loads before the fence will not be reordered with * loads after the fence. + * + * This method is operationally equivalent to {@link #loadFence()} + * as most hardware instructions that provide a LoadLoad barrier + * provide a LoadStore barrier for free. + * + * @since 9 */ public final void loadLoadFence() { loadFence(); @@ -3458,6 +3462,12 @@ /** * Ensures that stores before the fence will not be reordered with * stores after the fence. + * + * This method is operationally equivalent to {@link #storeFence()} + * as most hardware instructions that provide a StoreStore barrier + * provide a LoadStore barrier for free. + * + * @since 9 */ public final void storeStoreFence() { storeFence(); > On Aug 28, 2020, at 3:12 PM, Ian Graves wrote: > > Hello, > > When reviewing some of the documentation in jdk.internal.misc.Unsafe, it appears that there is some dated information regarding the availability of ?pure barrier? methods (i.e. pure StoreStoreFence sans a LoadStore barrier). When the storeFence() and loadFence() methods were implemented the pure variations (loadLoadFence(), storeStoreFence()) were not implemented and the documentation indicated as much. This is no longer the case. > > I?ve included a patch with this email that updates the docs to reflect the new state of affairs here as well as indicate some implementation details as to how the pure variants utilize the impure original methods. > > Thanks! > > Ian > From hboehm at google.com Fri Aug 28 21:59:14 2020 From: hboehm at google.com (Hans Boehm) Date: Fri, 28 Aug 2020 14:59:14 -0700 Subject: [PATCH] Docfix for Unsafe regarding fence methods In-Reply-To: <70AA9B93-A9AA-4FD0-8EBA-AE3A8DF473A0@oracle.com> References: <70AA9B93-A9AA-4FD0-8EBA-AE3A8DF473A0@oracle.com> Message-ID: On Fri, Aug 28, 2020 at 1:21 PM Ian Graves wrote: > > Looks like the patch attachment was stripped. I?ll include it inline: > ?Ian > > ... > @@ -3458,6 +3462,12 @@ > /** > * Ensures that stores before the fence will not be reordered with > * stores after the fence. > + * > + * This method is operationally equivalent to {@link #storeFence()} > + * as most hardware instructions that provide a StoreStore barrier > + * provide a LoadStore barrier for free. > + * > + * @since 9 > */ > public final void storeStoreFence() { > storeFence(); > > This seems a little misleading, since ARM has "dmb ishst", which orders only stores? From cjashfor at linux.ibm.com Sat Aug 29 20:19:42 2020 From: cjashfor at linux.ibm.com (Corey Ashford) Date: Sat, 29 Aug 2020 13:19:42 -0700 Subject: RFR(M): 8248188: [PATCH] Add HotSpotIntrinsicCandidate and API for Base64 decoding In-Reply-To: <8d53dcf8-635a-11e2-4f6a-39b70e2c3b8b@oracle.com> References: <11ca749f-3015-c004-aa6b-3194e1dfe4eb@linux.ibm.com> <8ece8d2e-fd99-b734-211e-a32b534a7dc8@linux.ibm.com> <8d53dcf8-635a-11e2-4f6a-39b70e2c3b8b@oracle.com> Message-ID: <65ed7919-86fc-adfa-3cd5-58dd96a3487f@linux.ibm.com> Hi Roger, Thanks for your reply and thoughts! Comments interspersed below: On 8/28/20 10:54 AM, Roger Riggs wrote: > Hi Corey, > > A few comments on core-libs side... > > The naming convention for methods that end in '0' is usually to indicate > they are the bottom-most method or a native method. > So I think you can/should rename the methods to make the most sense > as to their function. Ok, I will fix that. > > Comparing with the way that the Base64 encoder was intrinsified, the > method that is intrinsified should have a method body that does > the same function, so it is interchangable.? That likely will just shift > the "fast path" code into the decodeBlock method. > Keeping the symmetry between encoder and decoder will > make it easier to maintain the code. Good point. I'll investigate what this looks like in terms of the actual code, and will report back (perhaps in a new webrev). > > Given intrinsic only handles 2 of the three cases, and the java code > handles > all three, I would add an extra arg to decodeBlock to reflect the isMime > case > and have the intrinsic take an early exit until it was implemented. > I did consider doing that, but didn't for two reasons: * Implementing isMIME using vector hardware would be very difficult due to the need to ignore non-base64 characters. This requires eliminating those characters from the vector, then reading and shifting more in, repeatedly until there are no non-base64 characters left. This isn't a trivial/fast thing to do, at least on Power arch. None of the published base64 encode/decode functions for vector processors address the MIME case. In fact they don't address isURL=true either, but fortunately that is a relatively easy addition. * If isMIME=true is not implemented by the intrinsic, it will cost unnecessary overhead for that case, because of the need to martial the parameters, call the intrinsic, and then do an early return. I benchmarked this approach before, and saw an approx 5% drop in performance when isMIME = true. So that's why we decided to leave the isMIME=true case as a later optimization. Because of the extra complexity of the algorithm, it probably shouldn't share the same intrinsic anyway; only the isMIME=true case should take the performance hit. > > It is unfortunate that taking advantage of vectorization has to be hand > coded. > If/when the Vector API is ready (JEP 338 https://openjdk.java.net/jeps/338) > the java code should be replaced to use the Vector API and then it would > work for a new hardware without specific coding for each platform. > "Just" implement the Vector API.? There's a lot more bang for the buck > going for that approach. The kind of vector processing used in this intrinsic operates mostly on bytes within one vector, not between two vectors (for example in matrix-multiply algorithms), otherwise known as SWAR (https://en.wikipedia.org/wiki/SWAR). Because of that, it's very sensitive to which exact instructions are available in the vector processor. There isn't much standardization of SWAR instructions between different arches, so I think it would be hard to get a generic SWAR API that gives good performance across several arches. From briefly looking at the link you provided, it doesn't appear to address SWAR operations, so it doesn't seem to me that waiting for the vector API would be worth the wait, and in fact may not provide any method at all to boost performance of base64 decode/encode. Regards, - Corey P.S. I work only two days a week, so the updates will be slower compared to other developers. > > Thanks, Roger > > > On 8/24/20 9:21 PM, Corey Ashford wrote: >> Here's a revised webrev which includes a JMH benchmark for the decode >> operation. >> >> http://cr.openjdk.java.net/~mhorie/8248188/webrev.03/ >> >> The added benchmark tries to be "fair" in that it doesn't prefer a >> large buffer size, which would favor the intrinsic.? It >> pseudo-randomly (but reproducibly) chooses a buffer size between 8 and >> 20k+8 bytes, and fills it with random data to encode and decode.? As >> part of the TearDown of an invocation, it also checks the decoded >> output data for correctness. >> >> Example runs on the Power9-based machine I use for development shows a >> 3X average improvement across these random buffer sizes. Here's an >> excerpt of the output when run with -XX:-UseBASE64Intrinsics : >> >> Iteration?? 1: 70795.623 ops/s >> Iteration?? 2: 71070.607 ops/s >> Iteration?? 3: 70867.544 ops/s >> Iteration?? 4: 71107.992 ops/s >> Iteration?? 5: 71048.281 ops/s >> >> And here's the output with the intrinsic enabled: >> >> Iteration?? 1: 208794.022 ops/s >> Iteration?? 2: 208630.904 ops/s >> Iteration?? 3: 208238.822 ops/s >> Iteration?? 4: 208714.967 ops/s >> Iteration?? 5: 209060.894 ops/s >> >> Taking the best of the two runs: 209060/71048 = 2.94 >> >> From other experiments where the benchmark uses a fixed-size, larger >> buffer, the performance ratio rises to about 4.0. >> >> Power10 should have a slightly higher ratio due to several factors, >> but I have not yet benchmarked on Power10. >> >> Other arches ought to be able to do at least this well, if not better, >> because of wider vector registers (> 128 bits) being available.? Only >> a Power9/10 implementation is included in this webrev, however. >> >> Regards, >> >> - Corey >> >> >> On 8/19/20 11:20 AM, Roger Riggs wrote: >>> Hi Corey, >>> >>> For changes obviously performance motivated, it is conventional to >>> run a JMH perf test to demonstate >>> the improvement and prove it is worthwhile to add code complexity. >>> >>> I don't see any existing Base64 JMH tests but they would be in the >>> repo below or near: >>> ???? test/micro/org/openjdk/bench/java/util/ >>> >>> Please contribute a JMH test and results to show the difference. >>> >>> Regards, Roger >>> >>> >>> >>> On 8/19/20 2:10 PM, Corey Ashford wrote: >>>> Michihiro Horie posted up a new iteration of this webrev for me. >>>> This time the webrev includes a complete implementation of the >>>> intrinsic for Power9 and Power10. >>>> >>>> You can find it here: >>>> http://cr.openjdk.java.net/~mhorie/8248188/webrev.02/ >>>> >>>> Changes in webrev.02 vs. webrev.01: >>>> >>>> ? * The method header for the intrinsic in the Base64 code has been >>>> rewritten using the Javadoc style.? The clarity of the comments has >>>> been improved and some verbosity has been removed. There are no >>>> additional functional changes to Base64.java. >>>> >>>> ? * The code needed to martial and check the intrinsic parameters >>>> has been added, using the base64 encodeBlock intrinsic as a guideline. >>>> >>>> ? * A complete intrinsic implementation for Power9 and Power10 is >>>> included. >>>> >>>> ? * Adds some Power9 and Power10 assembler instructions needed by >>>> the intrinsic which hadn't been defined before. >>>> >>>> The intrinsic implementation in this patch accelerates the decoding >>>> of large blocks of base64 data by a factor of about 3.5X on Power9. >>>> >>>> I'm attaching two Java test cases I am using for testing and >>>> benchmarking.? The TestBase64_VB encodes and decodes randomly-sized >>>> buffers of random data and checks that original data matches the >>>> encoded-then-decoded data.? TestBase64Errors encodes a 48K block of >>>> random bytes, then corrupts each byte of the encoded data, one at a >>>> time, checking to see if the decoder catches the illegal byte. >>>> >>>> Any comments/suggestions would be appreciated. >>>> >>>> Thanks, >>>> >>>> - Corey >>>> >>>> On 7/27/20 6:49 PM, Corey Ashford wrote: >>>>> Michihiro Horie uploaded a new revision of the Base64 decodeBlock >>>>> intrinsic API for me: >>>>> >>>>> http://cr.openjdk.java.net/~mhorie/8248188/webrev.01/ >>>>> >>>>> It has the following changes with respect to the original one posted: >>>>> >>>>> ??* In the event of encountering a non-base64 character, instead of >>>>> having a separate error code of -1, the intrinsic can now just >>>>> return either 0, or the number of data bytes produced up to the >>>>> point where the illegal base64 character was encountered. This >>>>> reduces the number of special cases, and also provides a way to >>>>> speed up the process of finding the bad character by the slower, >>>>> pure-Java algorithm. >>>>> >>>>> ??* The isMIME boolean is removed from the API for two reasons: >>>>> ??? - The current API is not sufficient to handle the isMIME case, >>>>> because there isn't a strict relationship between the number of >>>>> input bytes and the number of output bytes, because there can be an >>>>> arbitrary number of non-base64 characters in the source. >>>>> ??? - If an intrinsic only implements the (isMIME == false) case as >>>>> ours does, it will always return 0 bytes processed, which will >>>>> slightly slow down the normal path of processing an (isMIME == >>>>> true) instantiation. >>>>> ??? - We considered adding a separate hotspot candidate for the >>>>> (isMIME == true) case, but since we don't have an intrinsic >>>>> implementation to test that, we decided to leave it as a future >>>>> optimization. >>>>> >>>>> Comments and suggestions are welcome.? Thanks for your consideration. >>>>> >>>>> - Corey >>>>> >>>>> On 6/23/20 6:23 PM, Michihiro Horie wrote: >>>>>> Hi Corey, >>>>>> >>>>>> Following is the issue I created. >>>>>> https://bugs.openjdk.java.net/browse/JDK-8248188 >>>>>> >>>>>> I will upload a webrev when you're ready as we talked in private. >>>>>> >>>>>> Best regards, >>>>>> Michihiro >>>>>> >>>>>> Inactive hide details for "Corey Ashford" ---2020/06/24 >>>>>> 09:40:10---Currently in java.util.Base64, there is a >>>>>> HotSpotIntrinsicCa"Corey Ashford" ---2020/06/24 >>>>>> 09:40:10---Currently in java.util.Base64, there is a >>>>>> HotSpotIntrinsicCandidate and API for encodeBlock, but no >>>>>> >>>>>> From: "Corey Ashford" >>>>>> To: "hotspot-compiler-dev at openjdk.java.net" >>>>>> , >>>>>> "ppc-aix-port-dev at openjdk.java.net" >>>>>> >>>>>> Cc: Michihiro Horie/Japan/IBM at IBMJP, Kazunori >>>>>> Ogata/Japan/IBM at IBMJP, joserz at br.ibm.com >>>>>> Date: 2020/06/24 09:40 >>>>>> Subject: RFR(S): [PATCH] Add HotSpotIntrinsicCandidate and API for >>>>>> Base64 decoding >>>>>> >>>>>> ------------------------------------------------------------------------ >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Currently in java.util.Base64, there is a >>>>>> HotSpotIntrinsicCandidate and >>>>>> API for encodeBlock, but none for decoding. ?This means that only >>>>>> encoding gets acceleration from the underlying CPU's vector hardware. >>>>>> >>>>>> I'd like to propose adding a new intrinsic for decodeBlock. ?The >>>>>> considerations I have for this new intrinsic's API: >>>>>> >>>>>> ??* Don't make any assumptions about the underlying capability of the >>>>>> hardware. ?For example, do not impose any specific block size >>>>>> granularity. >>>>>> >>>>>> ??* Don't assume the underlying intrinsic can handle isMIME or isURL >>>>>> modes, but also let them decide if they will process the data >>>>>> regardless >>>>>> of the settings of the two booleans. >>>>>> >>>>>> ??* Any remaining data that is not processed by the intrinsic will be >>>>>> processed by the pure Java implementation. ?This allows the >>>>>> intrinsic to >>>>>> process whatever block sizes it's good at without the complexity of >>>>>> handling the end fragments. >>>>>> >>>>>> ??* If any illegal character is discovered in the decoding >>>>>> process, the >>>>>> intrinsic will simply return -1, instead of requiring it to throw a >>>>>> proper exception from the context of the intrinsic. ?In the event of >>>>>> getting a -1 returned from the intrinsic, the Java Base64 library >>>>>> code >>>>>> simply calls the pure Java implementation to have it find the >>>>>> error and >>>>>> properly throw an exception. ?This is a performance trade-off in the >>>>>> case of an error (which I expect to be very rare). >>>>>> >>>>>> ??* One thought I have for a further optimization (not implemented in >>>>>> the current patch), is that when the intrinsic decides not to >>>>>> process a >>>>>> block because of some combination of isURL and isMIME settings it >>>>>> doesn't handle, it could return extra bits in the return code, >>>>>> encoded >>>>>> as a negative number. ?For example: >>>>>> >>>>>> Illegal_Base64_char ? = 0b001; >>>>>> isMIME_unsupported ? ?= 0b010; >>>>>> isURL_unsupported ? ? = 0b100; >>>>>> >>>>>> These can be OR'd together as needed and then negated (flip the >>>>>> sign). >>>>>> The Base64 library code could then cache these flags, so it will know >>>>>> not to call the intrinsic again when another decodeBlock is requested >>>>>> but with an unsupported mode. ?This will save the performance hit of >>>>>> calling the intrinsic when it is guaranteed to fail. >>>>>> >>>>>> I've tested the attached patch with an actual intrinsic coded up for >>>>>> Power9/Power10, but those runtime intrinsics and arch-specific >>>>>> patches >>>>>> aren't attached today. ?I want to get some consensus on the >>>>>> library-level intrinsic API first. >>>>>> >>>>>> Also attached is a simple test case to test that the new intrinsic >>>>>> API >>>>>> doesn't break anything. >>>>>> >>>>>> I'm open to any comments about this. >>>>>> >>>>>> Thanks for your consideration, >>>>>> >>>>>> - Corey >>>>>> >>>>>> >>>>>> Corey Ashford >>>>>> IBM Systems, Linux Technology Center, OpenJDK team >>>>>> cjashfor at us dot ibm dot com >>>>>> [attachment "decodeBlock_api-20200623.patch" deleted by Michihiro >>>>>> Horie/Japan/IBM] [attachment "TestBase64.java" deleted by >>>>>> Michihiro Horie/Japan/IBM] >>>>>> >>>>>> >>>>> >>>> >>> >> > From felix.yang at huawei.com Mon Aug 31 06:50:34 2020 From: felix.yang at huawei.com (Yangfei (Felix)) Date: Mon, 31 Aug 2020 06:50:34 +0000 Subject: RFR: 8252204: AArch64: Implement SHA3 accelerator/intrinsic Message-ID: Hi, Bug: https://bugs.openjdk.java.net/browse/JDK-8252204 Webrev: http://cr.openjdk.java.net/~fyang/8252204/webrev.00/ This added an intrinsic for SHA3 using aarch64 v8.2 SHA3 Crypto Extensions. Reference implementation for core SHA-3 transform using ARMv8.2 Crypto Extensions: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/arch/arm64/crypto/sha3-ce-core.S?h=v5.4.52 Trivial adaptation in SHA3. implCompress is needed for the purpose of adding the intrinsic. For SHA3, we need to pass one extra parameter "digestLength" to the stub for the calculation of block size. "digestLength" is also used in for the EOR loop before keccak to differentiate different SHA3 variants. We added jtreg tests for SHA3 and used QEMU system emulator which supports SHA3 instructions to test the functionality. Patch passed jtreg tier1-3 tests with QEMU system emulator. Also verified with jtreg tier1-3 tests without SHA3 instructions on aarch64-linux-gnu and x86_64-linux-gnu, to make sure that there's no regression. We used one existing JMH test for performance test: test/micro/org/openjdk/bench/java/security/MessageDigests.java We measured the performance benefit with an aarch64 cycle-accurate simulator. Patch delivers 20% - 40% performance improvement depending on specific SHA3 digest length and size of the message. For now, this feature will not be enabled automatically for aarch64. We can auto-enable this when it is fully tested on real hardware. But for the above testing purposes, this is auto-enabled when the corresponding hardware feature is detected. Comments? Thanks, Felix From aph at redhat.com Mon Aug 31 08:41:26 2020 From: aph at redhat.com (Andrew Haley) Date: Mon, 31 Aug 2020 09:41:26 +0100 Subject: [aarch64-port-dev ] RFR: 8252204: AArch64: Implement SHA3 accelerator/intrinsic In-Reply-To: References: Message-ID: <1729f1b1-056d-76c9-c820-d38bd6c1235d@redhat.com> On 31/08/2020 07:50, Yangfei (Felix) wrote: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8252204 > Webrev: http://cr.openjdk.java.net/~fyang/8252204/webrev.00/ > > This added an intrinsic for SHA3 using aarch64 v8.2 SHA3 Crypto Extensions. > Reference implementation for core SHA-3 transform using ARMv8.2 Crypto Extensions: > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/arch/arm64/crypto/sha3-cecore.S?h=v5.4.52 > Trivial adaptation in SHA3. implCompress is needed for the purpose > of adding the intrinsic. For SHA3, we need to pass one extra > parameter "digestLength" to the stub for the calculation of block > size. "digestLength" is also used in for the EOR loop before > keccak to differentiate different SHA3 variants. > > We added jtreg tests for SHA3 and used QEMU system emulator > which supports SHA3 instructions to test the functionality. > Patch passed jtreg tier1-3 tests with QEMU system emulator. > Also verified with jtreg tier1-3 tests without SHA3 instructions > on aarch64-linux-gnu and x86_64-linux-gnu, to make sure that > there's no regression. > > We used one existing JMH test for performance test: > test/micro/org/openjdk/bench/java/security/MessageDigests.java > We measured the performance benefit with an aarch64 > cycle-accurate simulator. > Patch delivers 20% - 40% performance improvement depending on > specific SHA3 digest length and size of the message. > For now, this feature will not be enabled automatically for > aarch64. We can auto-enable this when it is fully tested on > real hardware. > But for the above testing purposes, this is auto-enabled when > the corresponding hardware feature is detected. > > Comments? This looks like a direct copy of the sha3-cecore.S file.You'll need Linaro to contribute it. I don't imagine they'll have any problem with that: they are OCA signatories Also, given that we've got the assembly source file, why not just copy that into OpenJDK? I can't see the point rewriting it into the HotSpot assembler. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From felix.yang at huawei.com Mon Aug 31 09:46:58 2020 From: felix.yang at huawei.com (Yangfei (Felix)) Date: Mon, 31 Aug 2020 09:46:58 +0000 Subject: [aarch64-port-dev ] RFR: 8252204: AArch64: Implement SHA3 accelerator/intrinsic In-Reply-To: <1729f1b1-056d-76c9-c820-d38bd6c1235d@redhat.com> References: <1729f1b1-056d-76c9-c820-d38bd6c1235d@redhat.com> Message-ID: > -----Original Message----- > From: Andrew Haley [mailto:aph at redhat.com] > Sent: Monday, August 31, 2020 4:41 PM > To: Yangfei (Felix) ; hotspot-compiler- > dev at openjdk.java.net; core-libs-dev at openjdk.java.net > Cc: aarch64-port-dev at openjdk.java.net > Subject: Re: [aarch64-port-dev ] RFR: 8252204: AArch64: Implement SHA3 > accelerator/intrinsic > > On 31/08/2020 07:50, Yangfei (Felix) wrote: > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8252204 > > Webrev: http://cr.openjdk.java.net/~fyang/8252204/webrev.00/ > > > > This added an intrinsic for SHA3 using aarch64 v8.2 SHA3 Crypto > Extensions. > > Reference implementation for core SHA-3 transform using ARMv8.2 > Crypto Extensions: > > > https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/arch/ar > m64/crypto/sha3-cecore.S?h=v5.4.52 > > Trivial adaptation in SHA3. implCompress is needed for the purpose > > of adding the intrinsic. For SHA3, we need to pass one extra > > parameter "digestLength" to the stub for the calculation of block > > size. "digestLength" is also used in for the EOR loop before > > keccak to differentiate different SHA3 variants. > > > > We added jtreg tests for SHA3 and used QEMU system emulator > > which supports SHA3 instructions to test the functionality. > > Patch passed jtreg tier1-3 tests with QEMU system emulator. > > Also verified with jtreg tier1-3 tests without SHA3 instructions > > on aarch64-linux-gnu and x86_64-linux-gnu, to make sure that > > there's no regression. > > > > We used one existing JMH test for performance test: > > test/micro/org/openjdk/bench/java/security/MessageDigests.java > > We measured the performance benefit with an aarch64 > > cycle-accurate simulator. > > Patch delivers 20% - 40% performance improvement depending on > > specific SHA3 digest length and size of the message. > > For now, this feature will not be enabled automatically for > > aarch64. We can auto-enable this when it is fully tested on > > real hardware. > > But for the above testing purposes, this is auto-enabled when > > the corresponding hardware feature is detected. > > > > Comments? > > This looks like a direct copy of the sha3-cecore.S file.You'll need Linaro to > contribute it. I don't imagine they'll have any problem with that: they are > OCA signatories Since the code in sha3-cecore.S works in kernel space, we need several modifications here to makes it work in hotspot. First, we need to add callee-save & restore for d8 - d15 according to the aarch64 abi. Also, the following code snippet is not needed for user-space: if_will_cond_yield_neon add x8, x19, #32 st1 { v0.1d- v3.1d}, [x19] st1 { v4.1d- v7.1d}, [x8], #32 st1 { v8.1d-v11.1d}, [x8], #32 st1 {v12.1d-v15.1d}, [x8], #32 st1 {v16.1d-v19.1d}, [x8], #32 st1 {v20.1d-v23.1d}, [x8], #32 st1 {v24.1d}, [x8] do_cond_yield_neon b 0b endif_yield_neon And we need to handle the multi-block case differently for StubRoutines::sha3_implCompressMB: 3485 if (multi_block) { 3486 // block_size = 200 - 2 * digest_length, ofs += block_size 3487 __ add(ofs, ofs, 200); 3488 __ sub(ofs, ofs, digest_length, Assembler::LSL, 1); 3489 3490 __ cmp(ofs, limit); 3491 __ br(Assembler::LE, sha3_loop); 3492 __ mov(c_rarg0, ofs); // return ofs 3493 } And StubRoutines::sha3_implCompress does not even need this multi-block check logic. > Also, given that we've got the assembly source file, why not just copy that > into OpenJDK? I can't see the point rewriting it into the HotSpot assembler. Actually, we referenced the existing intrinsics implementation and took a similar way. It looks strange to have one intrinsic that goes differently. And we won't be able to emit this code on demand if we go that different way. Some cpu does not support these special sha3 instructions and thus does need this code at all. I think that's one advantage of using a stub. Thanks, Felix From ioi.lam at oracle.com Mon Aug 31 15:31:36 2020 From: ioi.lam at oracle.com (Ioi Lam) Date: Mon, 31 Aug 2020 08:31:36 -0700 Subject: RFR(L) 8244778 Archive full module graph in CDS In-Reply-To: <234856fa-1eb1-8514-6786-bce6689afd16@oracle.com> References: <9e6b0043-65a1-dd97-a3d2-33679c8048d4@oracle.com> <234856fa-1eb1-8514-6786-bce6689afd16@oracle.com> Message-ID: Hi Lois, Thanks for the review. Your comments has led me to discover a couple of pretty serious issues, which hopefully I have fixed. Also the code is cleaner now, with much fewer control-flow changes than the last webrev. http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v03/ http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v03.delta/ Please see my comments in-line. On 8/25/20 7:58 AM, Lois Foltan wrote: > > Hi Ioi, > > Changes looks really good.? Comments interspersed below. > > Thanks, > Lois > > On 8/12/2020 6:06 PM, Ioi Lam wrote: >> Hi Lois, >> >> Thanks for the comments. I have an updated webrev >> >> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02/ >> >> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v02.delta/ >> >> >> Here are the general notes on the changes. Replies to your questions >> are in-line below. >> >> (1) Integrated updates in the Java code from Alan Bateman. Here are >> Alan's >> ? ? notes: >> >> ??? The archive of the boot layer is as before, except that archiving is >> ??? skipped if there are split packages or incubator modules. Incubating >> ??? modules aren't resolved by default so we shouldn't have them in the >> ??? boot layer by default anyway. >> >> ??? I've dropped the module finders from the boot layer archive. I've >> ??? left the IllegalAccessLogger.Builder in the acrhive for now (even >> ??? though it is not the boot layer). We should be able to remove that >> ??? once the JEP to disallow illegal access by default is in. >> >> ??? Related is that I don't like the archive for the module graph >> ??? (ArchivedModuleGraph, pre-dates this RFE) including the set of >> ??? packages to export/open for illegal access as they aren't part >> ??? of the module graph. I've left it for now but we can also remove >> ??? that once we disallow illegal access by default (as those sets >> ??? will be empty). >> >> ??? The archive of built-in class loaders is now in one object >> ??? jdk.internal.loader.ArchivedClassLoaders which I think will be a >> ??? bit more maintainable. I've also drop the ucp field from the >> ??? AppClassLoader as the changes to BuiltinClassLoader means is no >> ??? longer needs to duplicated. >> >> ??? There's one remaining issue in ModuleBootstrap.boot where it has fix >> ??? an app class loader value (ModuleLayer.CLV). Ideally the >> initialization >> ??? of the built-in class loaders would do this but we are kinda forced >> ??? to separate the archiving of the built-in class loaders from the >> boot >> ??? layer. I might look at this again some time. >> >> >> (2) Moved code from classLoaderData.cpp -> classLoaderSharedData.cpp >> >> (3) Reverted unnecessary changes in JavaClasses::compute_offset >> >> (4) Minor clean up to use QuickSort::sort() instead of qsort() >> >> (5) Moved the C-side of module initialization for platform/system >> ? ? loaders to inside java.lang.System::initPhase2(), so this happens >> ??? at the same time as without full module archiving. >> >> (6) Moved the use of Module_lock to so all modules in a class loader >> ??? are restored atomically. See ArchivedClassLoaderData::restore() >> >> ??? This fixed a bug where test/jdk/com/sun/jdi/ModulesTest.java >> ? ? would fail as it sees a partially restored set of modules. >> >> >> >> On 8/7/20 12:06 PM, Lois Foltan wrote: >>> Hi Ioi, >>> >>> Overall looks promising.? I have some review comments below, but not >>> a complete review.? I concentrated on the JVM side primarily how the >>> archived module graph is read in, how the ModuleEntry and >>> PackageEntry tables are created from the archive for the 3 builtin >>> loaders and potential timing issues during start up. >>> >>> On 7/22/2020 3:36 PM, Ioi Lam wrote: >>>> https://bugs.openjdk.java.net/browse/JDK-8244778 >>>> http://cr.openjdk.java.net/~iklam/jdk16/8244778-archive-full-module-graph.v01/ >>>> >>>> >>>> Please review this patch that stores the full module graph in the CDS >>>> archive heap. This reduces the initialization time of the basic JVM by >>>> about 22%: >>>> >>>> $ perf stat -r 100 bin/java -version >>>> before: 98,219,329 instructions 0.03971 secs elapsed (+- 0.44%) >>>> after:? 55,835,815 instructions 0.03109 secs elapsed (+- 0.65%) >>>> >>>> [1] Start with ModuleBootstrap.java. The current implementation is >>>> ??? quite restrictive: the archived module graph is used only when no >>>> ??? module options are specified. >>>> >>>> ??? See ModuleBootstrap.mayUseArchivedBootLayer(). >>>> >>>> ??? We can probably support options such as main module and module >>>> path >>>> ??? in a future RFE. >>> >>> Yes, I noticed the restrictions.? The JBS issue discusses the >>> possibility of using the dumped module graph in the event that the >>> value of the options are the same.? I think for this implementation >>> to be viable and have a positive impact you should consider >>> comparing at least the options --add-modules, --add-exports and >>> --add-reads. >> >> I agree. I want to keep the changes minimal in this RFE, and will add >> more support for other use cases in follow-on RFEs. Instead of >> requiring these options to be unspecified, we can relax the >> restriction such that these options must be the same between archive >> dump time and run time. > > Sounds good! > >> >>> >>> >>>> >>>> [2] In the current JDK implementation, there is no single object >>>> ??? that represents "the module graph". Most of the information >>>> ??? is stored in the archive bootLayer object, but a few additional >>>> ??? restoration operations need to be performed: >>>> >>>> ??? + See ModuleBootstrap.getArchivedBootLayer() >>>> ??? + Some static fields need to be archived/restored in >>>> ????? Module.java, BuiltinClassLoader.java, ClassLoaders.java >>>> ????? and BootLoader.java >>>> >>>> [3] I ran into a complication with two loader instances of >>>> ??? PlatformClassLoader and AppClassLoader. They are stored in >>>> ??? multiple tables inside the module graph (e.g., >>>> ??? BuiltinClassLoader$LoadedModule) so I cannot easily recreate >>>> ??? them at runtime. >>>> >>>> ??? However, these two loaders contain information specific to the >>>> ??? dump time VM lifecycle (such as the classes that were loaded >>>> ??? during CDS dumping) that need to be scrubbed. I couldn't find an >>>> ??? elegant way of doing this, so I added a private >>>> "resetArchivedStates" >>>> ??? method to a few classes. They are called inside >>>> ??? HeapShared::reset_archived_object_states(). >>>> >>>> [4] Related native data structures (PackageEntry and ModuleEntry) >>>> ??? are also archived. Start with classLoaderData.cpp >>>> >>>> Passes mach5 tiers 1-4. I will test with additional tiers. >>>> >>>> Thanks >>>> - Ioi >>> >>> classfile/classLoader.cpp >>> - line #1644 pulling the javabase_moduleEntry() check outside of the >>> Module_lock maybe problematic if after you check it another >>> ? thread initializes in the time between the check and the obtaining >>> of the Module_lock on line #1645. >> >> Fixed. > > That looks good.? I think it is fine that you are checking if > java.base is defined via a call to javabase_moduleEntry() because you > are just trying to determine if a ModuleEntry should be created or > not.? In most cases though using ModuleEntryTable::javabase_defined() > is the correct way to ensure that both the ModuleEntry for java.base > has been created and that the ModuleEntry has been injected in the > corresponding java.lang.Module oop. > >> >>> >>> classfile/classLoader.hpp >>> - somewhere in ArchivedClassLoaderData there should be an assert to >>> make sure that the CLD, whose package entries and module entries are >>> being archived is not a "_has_class_mirror_holder" CLD for a weakly >>> defined hidden class.? Those dedicated CLDs should never have >>> package entries or module entries. >>> >> >> I added ArchivedClassLoaderData::assert_valid() >> >>> classfile/moduleEntry.cpp >>> - line #400, typo "conver" --> "convert" >>> - line #500, maybe sort if n is greater than 1 instead of 0 (same >>> comment for packageEntry.cpp line #270) >>> >> Fixed >> >>> classfile/systemDictionary.cpp >>> - It looks like the PackageEntry and ModuleEntry tables for the >>> system and platform class loaders are? added within >>> SystemDictionary::compute_java_loaders() which is called from >>> Thread::create_vm() but after the call in Thread::create_vm() to >>> call_initPhase2 which is when Universe::_module_initialized is set >>> to true.? Did I read this correctly?? If yes, then I think you have >>> to be sure that Universe::_module_initialized is not set until the >>> module graph for the 3 builtin loaders is loaded from the archive. >>> >> >> I moved the code to be called by ModuleBootstrap::boot() so it should >> now happen inside call_initPhase2. > > Yes, that looks good. > >> >>> memory/filemap.cpp >>> - line #237 typo "modue" --> "module" >>> >> >> Fixed >> >>> - Since the CDS support for capturing the module graph does not >>> archive unnamed modules, I assume >>> Modules::set_bootloader_unnamed_module() is still called.? Is this >>> true or does the unnamed module for the boot loader get archived? >> The unnamed module for the boot loader is not archived. >> >> Modules::set_bootloader_unnamed_module()? wasn't called in my last >> webrev. Thanks for catching it. >> >> I added a call to BootLoader.getUnnamedModule() in >> ModuleBootstrap::boot()? to trigger of BootLoader, which >> will call into the VM for Modules::set_bootloader_unnamed_module(). > > Looks good. > >> >> >> >>> Clarification/timing questions: >>> >> >> Here's an overall problem I am facing: >> >> The module graph is archived after the module system has fully >> started up. This means that for the boot loader, we only know the >> full set of modules/packages, but we don't know which part is the >> subset that was initialized during early VM bootstraping (e.g., when >> ModuleEntryTable::javabase_defined() == false). >> >> So the behavior is a bit different during the early bootstrap phase >> (all the way before we reach ModuleBootstrap::boot()): >> ClassLoaderData::the_null_class_loader_data()->modules() and >> ClassLoaderData::the_null_class_loader_data()->packages() are already >> populated before a single class is loaded. > > If this is the case then, at the point when a ModuleEntry is created > for java.base using the archived module graph, there should be a > assertion that java_lang_Class::_fixup_module_field_list is NULL. To > confirm no class has been loaded before java.base is defined. Maybe in > ClassLoaderDataShared::serialize where you restore the boot loader's > archived modules? Thanks for pointing this out. It turned out that in the last webrev (v02), I had a bug where the module of the primitive classes were not initialized. Now I have changed the initialization code to behave the same whether archived full module graph is used or not. The only differences are: [1] ClassLoader::create_javabase(): ModuleEntryTable::javabase_moduleEntry() may be inited by CDS. [2] When archived full module graph is used, ModuleEntryTable::patch_javabase_entries is called from Modules::define_archived_modules. (java_lang_Class::_fixup_module_field_list is used within this call.) I also added a new test case: cds/PrimitiveClassMirrors.java > >> This difference doesn't seem to make a practical difference. E.g., >> none of our code seems to assume that "before any classes in the >> java.util package is loaded, >> ClassLoaderData::the_null_class_loader_data()->packages() must not >> contain an entry for java.util". >> >> I think we have two choices when the archived module graph is used: >> >> [1] We require that the state of the module system must, at every >> step of VM initialization, be identical to that of a VM that doesn't >> use an archived module graph. >> >> [2] We make sure that the VM/JDK bootstrap code can tolerate the >> existence of module/packages that are added earlier than a VM that >> doesn't use an archived module graph. >> >> I tried doing a version of [1] and found that to be too difficult. >> [2] seems much simpler and is the approach I am using now. > > I think [2] is reasonable. > >> >>> oops/instanceKlass.cpp >>> - line #2545, comment "TO DO? -- point it to the archived >>> PackageEntry (JDK-8249262)" >>> are you thinking that since the module graph is read in ahead of >>> time that it can be set when an InstanceKlass is created? There is a >>> point before java.base is defined that InstanceKlass::set_package >>> takes into account that could be a timing issue. >>> >>> >> >> I think it will work as if another class in the same package has >> already been defined. >> >>> - There are some checks in modules.cpp that are valuable to have >>> during bootstrapping.? For example, packages that are encountered >>> during bootstrapping before java.base is defined are put into the >>> java.base module but then verified after java.base is defined via >>> verify_javabase_packages.? So at what point in the bootstrapping >>> process does the boot loader's archived module's become known? Could >>> classes have been defined to the VM before this point? Then their >>> packages will have to be verified to be sure that they are indeed >>> packages within java.base.? Consider looking at other checks that >>> occur within classfile/modules.cpp as well. >>> >> As mentioned above, calling verify_javabase_packages() at run time >> will fail, as we have loaded all packages for the boot loader, not >> just those for java.base. > > I think not calling verify_javabase_packages works because as you > stated above no classes have been loaded before java.base is defined > which is not the same situation as running without the archived module > graph. > > A couple of additional comments: > > - ModuleEntry's reads list and PackageEntry's exports list.? We had > hoped eventually to change these lists from being a c heap allocated > GrowableArray over to a ResourceHashTable for faster lookup.? Doing > that as a separate RFE might help CDS archiving not to have to archive > those lists as an Array? > CDS cannot archive ResourceHashTable either. We have CompactHashtable which can be archived, but it cannot be modified at run time. I think the export list can be modified at run time with java.lang.Module::addExports(), so we probably need to do it like: - if a Module was archived, check its CompactHashtable first - if not found, check the ResourceHashTable This would make the start-up a little faster (no more copying from the array lists into the hashtable, but would make the code more complicated. I need to investigate to see if it's worthwhile. > - moduleEntry.cpp and packageEntry.cpp: Both methods > "compare_module_by_name" and "compare_package_by_name" should have an > assert if the names are equal.? No ModuleEntryTable or > PackageEntryTable should ever have 2 same named modules or packages. > I tried adding: static int compare_package_by_name(PackageEntry* a, PackageEntry* b) { ? assert(a != b && a->name() != b->name(), "array should not contain duplicated entries"); ? return a->name()->fast_compare(b->name()); } but this caused an assert, because our QuickSort implementation would sometimes compare the same element! #7? 0x00007ffff659238b in QuickSort::partition (array=0x80043fcb0, ??? pivot=248, length=496, comparator=0x7ffff6590a5a ) ??? at /jdk2/gil/open/src/hotspot/share/utilities/quickSort.hpp:76 76??? ????? for ( ; comparator(array[left_index], pivot_val) < 0; ++left_index) { (gdb) p array[left_index] $1 = (PackageEntry *) 0x7ffff0439d00 (gdb) p pivot_val $2 = (PackageEntry *) 0x7ffff0439d00 (gdb) p pivot $3 = 248 (gdb) p left_index $4 = 248 So I ended up with: static int compare_package_by_name(PackageEntry* a, PackageEntry* b) { ? assert(a == b || a->name() != b->name(), "no duplicated names"); ? return a->name()->fast_compare(b->name()); } > - Another timing clarification question for me.? I assume that the > module graph is dumped post module initialization when Java has > completely defined the module graph to the JVM, is this correct? Yes > My concern is that there could be a window post module initialization > and pre archiving the module graph where another thread could define a > new module or add Module readability or add Package exportability.? So > that the module graph you are dumping is not the same module graph at > the end of module initialization.? Is this a concern? We don't allow arbitrary code to be executed during -Xshare:dump, so the module graph shouldn't be changed after module initialization has finished. I've added Modules::check_cds_restrictions() to check for this. I also added asserts to make sure that none of the classes used by the archived module graph can be modified by JVMTI. All these classes are loaded in the "early" phase of JVMTI, and we would disable CDS if a JVMTI agent is registered to modify classes in this phase, so we should be safe, but the asserts will ensure that. I updated the ReplaceCriticalClassesForSubgraphs.java test and added a new test RedefineClassesInModuleGraph.java. Thanks - Ioi > > Thanks, > Lois > >> >> Well, verify_javabase_packages() was called once and it succeeded, >> but that was during CDS dump time. So we know at least we have >> verified this once :-) >> >> Thanks >> - Ioi >> >>> I may have more review comments as I continue to look at this! >>> >>> Thanks, >>> Lois >>> >>> >>> >>> >>> >> > > > > From naoto.sato at oracle.com Mon Aug 31 16:22:20 2020 From: naoto.sato at oracle.com (Naoto Sato) Date: Mon, 31 Aug 2020 09:22:20 -0700 Subject: RFR: 8252552: DecimalFormat javadoc contains HTML tags in example code Message-ID: <4219fa9a-f90f-8e75-30b0-eb0f084c1ffa@oracle.com> Hello, Please review this simple javadoc fix to: https://bugs.openjdk.java.net/browse/JDK-8252552 The proposed diff is inserted here: --- --- old/src/java.base/share/classes/java/text/DecimalFormat.java 2020-08-31 09:07:22.000000000 -0700 +++ new/src/java.base/share/classes/java/text/DecimalFormat.java 2020-08-31 09:07:22.000000000 -0700 @@ -338,9 +338,9 @@ * *

    Example

    * - * 
    {@code
- * // Print out a number using the localized number, integer, 
currency,
- * // and percent format for each locale
+ * 
    {@code
+ * // Print out a number using the localized number, integer, currency,
+ * // and percent format for each locale}{@code
   * Locale[] locales = NumberFormat.getAvailableLocales();
   * double myNumber = -1234.56;
   * NumberFormat form;
---

It is simply to take those  HTML tags out of {@code} snippet.

Naoto

From cjashfor at linux.ibm.com  Mon Aug 31 16:41:32 2020
From: cjashfor at linux.ibm.com (Corey Ashford)
Date: Mon, 31 Aug 2020 09:41:32 -0700
Subject: RFR(M): 8248188: [PATCH] Add HotSpotIntrinsicCandidate and API
 for Base64 decoding
In-Reply-To: 
References: <11ca749f-3015-c004-aa6b-3194e1dfe4eb@linux.ibm.com>
 
 
 
 
 
 
 
Message-ID: <83ee5372-3890-fb07-721b-9d51641865da@linux.ibm.com>

On 8/27/20 8:07 AM, Doerr, Martin wrote:
> Hi Corey,
> 
>> If I make a requirement, I feel decode0 should check that the
>> requirement is met, and raise some kind of internal error if it isn't.
>> That actually was my first implementation, but I received some comments
>> during an internal review suggesting that I just "round down" the
>> destination count to the closest multiple of 3 less than or equal to the
>> returned value, rather than throw an internal exception which would
>> confuse users.  This "enforces" the rule, in some sense, without error
>> handling.  Do you have some thoughts about this?
> 
> I think the rounding logic is hard to understand and I'm not sure if it's correct (you're rounding up for the 1st computation of chars_decoded).
> If we don't use it, it will never get tested (because the intrinsic always returns a multiple of 3).
> I prefer having a more simple version which is easy to understand and for which we can test all cases.

I will see what I can do with the calculation of chars_decoded, at least 
in the comments, to make it more clear as to the "why" of the calculation.

I will remove the round down code: "dl = (dl / 3) * 3;" and leave it for 
intrinsics implementers/maintainers to check that assumption when the 
intrinsic returns.

> 
> I think we should be able to catch violations of this requirement by adding good JTREG tests.
> An illegal intrinsic implementation should never pass the tests. So I don't see a need to catch an illegal state in the Java source code in this case.
> I guess this will be best for intrinsic implementors for other platforms as well.
> 
> I'd appreciate more opinions on this.
> 
> 
>> I will double check that everything compiles and runs properly with gcc
>> 7.3.1.
> Please note that 7.3.1 is our minimum for Big Endian linux. For Little Endian it's 7.4.0.

Ah, that might explain why I wasn't able to find gcc-7.3.1 on RHEL 8.1 
(gcc-8.3.1) or Ubuntu 16.04 (gcc-7.4.0) for Power9.  As long as the code 
is enabled on little endian machines only, there should be no trouble 
with compilation.  I did compile and run the tests against 7.4.0, and it 
worked without a problem.

> You can also find this information here:
> https://wiki.openjdk.java.net/display/Build/Supported+Build+Platforms
> under "Other JDK 13 build platforms" which hasn't changed since then.
>

Great, thank you.


>> I will use __attribute__ ((align(16))) instead of __vector, and make
>> them arrays of 16 unsigned char.
> Maybe __vectors works as expected, too, now. Whatever we use, I'd appreciate to double-check the alignment e.g. by using gdb.

Ok, I will experiment with that with some small test cases and see if I 
can make the compiler stumble and not align the vector.  The lxv 
instruction can handle unaligned vectors in memory, but it would be 
better to have the vectors aligned for performance reasons.

> I don't remember what we had tried and why it didn't work as desired.
> 
> 
>> I was following what was done for encodeBlock, but it appears
>> encodeBlock's style isn't what is used for the other intrinsics.  I will
>> correct decodeBlock to use the prevailing style.  Another patch should
>> be added (not part of this webrev) to correct encodeBlock's style.
> In your code one '\' is not aligned with the other ones.

Yes, it's corrected now.

> 
> 
>> Ah, this is another thing I didn't know about.  I will make some
>> regression tests.
> Thanks. There's some documentation available:
> https://openjdk.java.net/jtreg/
> I guess your colleagues can assist you with that so you don't have to figure out everything alone.

Yes, thank you.  JTREG tests will be part of the next webrev version.

Regards,

- Corey

> 
> 
>> Thanks for your time on this.  As you can tell, I'm inexperienced in
>> writing openjdk code, so your patience and careful review is really
>> appreciated.
> I'm glad you work on contributions. I think we should welcome new contributors and assist as far as we can.
> 
> Best regards,
> Martin
> 
> 
>> -----Original Message-----
>> From: Corey Ashford 
>> Sent: Donnerstag, 27. August 2020 00:17
>> To: Doerr, Martin ; Michihiro Horie
>> 
>> Cc: hotspot-compiler-dev at openjdk.java.net; core-libs-dev > dev at openjdk.java.net>; Kazunori Ogata ;
>> joserz at br.ibm.com
>> Subject: Re: RFR(M): 8248188: [PATCH] Add HotSpotIntrinsicCandidate and
>> API for Base64 decoding
>>
>> Hi Martin,
>>
>> Some inline responses below.
>>
>> On 8/26/20 8:26 AM, Doerr, Martin wrote:
>>
>>> Hi Corey,
>>>
>>> I should explain my comments regarding Base64.java better.
>>>
>>>> Let's be precise: "should process a multiple of four" => "must process a
>>>> multiple of four"
>>> Did you try to support non-multiple of 4 and this was intended as
>> recommendation?
>>> I think making it a requirement and simplifying the logic in decode0 is
>> better.
>>> Or what's the benefit of the recommendation?
>>
>> If I make a requirement, I feel decode0 should check that the
>> requirement is met, and raise some kind of internal error if it isn't.
>> That actually was my first implementation, but I received some comments
>> during an internal review suggesting that I just "round down" the
>> destination count to the closest multiple of 3 less than or equal to the
>> returned value, rather than throw an internal exception which would
>> confuse users.  This "enforces" the rule, in some sense, without error
>> handling.  Do you have some thoughts about this?
>>
>>>
>>>>> If any illegal base64 bytes are encountered in the source by the
>>>>> intrinsic, the intrinsic can return a data length of zero or any
>>>>> number of bytes before the place where the illegal base64 byte
>>>>> was encountered.
>>>> I think this has a drawback. Somebody may use a debugger and want to
>> stop
>>>> when throwing IllegalArgumentException. He should see the position
>> which
>>>> matches the Java implementation.kkkk
>>> This is probably hard to understand. Let me try to explain it by example:
>>> 1. 80 Bytes get processed by the intrinsic and 60 Bytes written to the
>> destination array.
>>> 2. The intrinsic sees an illegal base64 Byte and it returns 12 which is allowed
>> by your specification.
>>> 3. The compiled method containing the intrinsic hits a safepoint (e.g. in the
>> large while loop in decodeBlockSlow).
>>> 4. A JVMTI agent (debugger) reads dp and dst.
>>> 5. The person using the debugger gets angry because more bytes than dp
>> were written into dst. The JVM didn't follow the specified behavior.
>>>
>>> I guess we can and should avoid it by specifying that the intrinsic needs to
>> return the dp value matching the number of Bytes written.
>>
>> That's an interesting point.  I will change the specification, and the
>> intrinsic implementation.  Right now the Power9/10 intrinsic returns 0
>> when any illegal character is discovered, but I've been thinking about
>> returning the number of bytes already written, which will allow
>> decodeBlockSlow to more quickly find the offending character.  This
>> provides another good reason to make that change.
>>
>>>
>>> Best regards,
>>> Martin
>>>
>>>
>>>> -----Original Message-----
>>>> From: Doerr, Martin
>>>> Sent: Dienstag, 25. August 2020 15:38
>>>> To: Corey Ashford ; Michihiro Horie
>>>> 
>>>> Cc: hotspot-compiler-dev at openjdk.java.net; core-libs-dev >>> dev at openjdk.java.net>; Kazunori Ogata ;
>>>> joserz at br.ibm.com
>>>> Subject: RE: RFR(M): 8248188: [PATCH] Add HotSpotIntrinsicCandidate
>> and
>>>> API for Base64 decoding
>>>>
>>>> Hi Corey,
>>>>
>>>> thanks for proposing this change. I have comments and suggestions
>>>> regarding various files.
>>>>
>>>>
>>>> Base64.java
>>>>
>>>> This is the only file which needs another review from core-libs-dev.
>>>> First of all, I like the idea to use a HotSpotIntrinsicCandidate which can
>>>> consume as many bytes as the implementation wants.
>>>>
>>>> Comment before decodeBlock:
>>>> Let's be precise: "should process a multiple of four" => "must process a
>>>> multiple of four"
>>>>
>>>>> If any illegal base64 bytes are encountered in the source by the
>>>>> intrinsic, the intrinsic can return a data length of zero or any
>>>>> number of bytes before the place where the illegal base64 byte
>>>>> was encountered.
>>>> I think this has a drawback. Somebody may use a debugger and want to
>> stop
>>>> when throwing IllegalArgumentException. He should see the position
>> which
>>>> matches the Java implementation.
>>>>
>>>> Please note that the comment indentation differs from other comments.
>>
>> Will fix.
>>
>>>>
>>>> decode0: Final "else" after return is redundant.
>>
>> Will fix.
>>
>>>>
>>>>
>>>> stubGenerator_ppc.cpp
>>>>
>>>> "__vector" breaks AIX build!
>>>> Does it work on Big Endian linux with old gcc (we require 7.3.1, now)?
>>>> Please either support Big Endian properly or #ifdef it out.
>>
>> I have been compiling with only Advance Toolchain 13, which is 9.3.1,
>> and only on Linux.  It will not work with big endian, so it won't work
>> on AIX, however obviously it shouldn't break the AIX build, so I will
>> address that.  There's code to set UseBASE64Intrinsics to false on big
>> endian, but you're right -- I should ifdef all of the intrinsic code for
>> little endian for now.  Getting it to work on big endian / AIX shouldn't
>> be difficult, but it's not in my scope of work at the moment.
>>
>> I will double check that everything compiles and runs properly with gcc
>> 7.3.1.
>>
>>>> What exactly does it (do) on linux?
>>
>> It's an arch-specific type that's 16 bytes in size and aligned on a
>> 16-byte boundary.
>>
>>>> I remember that we had tried such prefixes but were not satisfied. I think
>> it
>>>> didn't enforce 16 Byte alignment if I remember correctly.
>>
>> I will use __attribute__ ((align(16))) instead of __vector, and make
>> them arrays of 16 unsigned char.
>>
>>>>
>>>> Attention: C2 does no longer convert int/bool to 64 bit values (since JDK-
>>>> 8086069). So the argument registers for offset, length and isURL may
>> contain
>>>> garbage in the higher bits.
>>
>> Wow, that's good to know!  I will mask off the incoming values.
>>
>>>>
>>>> You may want to use load_const_optimized which produces shorter code.
>>
>> Will fix.
>>
>>>>
>>>> You may want to use __ align(32) to align unrolled_loop_start.
>>
>> Will fix.
>>
>>>>
>>>> I'll review the algorithm in detail when I find more time.
>>>>
>>>>
>>>> assembler_ppc.hpp
>>>> assembler_ppc.inline.hpp
>>>> vm_version_ppc.cpp
>>>> vm_version_ppc.hpp
>>>> Please rebase. Parts of the change were pushed as part of 8248190:
>> Enable
>>>> Power10 system and implement new byte-reverse instructions
>>
>> Will do.
>>
>>>>
>>>>
>>>> vmSymbols.hpp
>>>> Indentation looks odd at the end.
>>
>> I was following what was done for encodeBlock, but it appears
>> encodeBlock's style isn't what is used for the other intrinsics.  I will
>> correct decodeBlock to use the prevailing style.  Another patch should
>> be added (not part of this webrev) to correct encodeBlock's style.
>>
>>>>
>>>>
>>>> library_call.cpp
>>>> Good. Indentation style of the call parameters differs from encodeBlock.
>>
>> Will fix.
>>
>>>>
>>>>
>>>> runtime.cpp
>>>> Good.
>>>>
>>>>
>>>> aotCodeHeap.cpp
>>>> vmSymbols.cpp
>>>> shenandoahSupport.cpp
>>>> vmStructs_jvmci.cpp
>>>> shenandoahSupport.cpp
>>>> escape.cpp
>>>> runtime.hpp
>>>> stubRoutines.cpp
>>>> stubRoutines.hpp
>>>> vmStructs.cpp
>>>> Good and trivial.
>>>>
>>>>
>>>> Tests:
>>>> I think we should have JTREG tests to check for regressions in the future.
>>
>> Ah, this is another thing I didn't know about.  I will make some
>> regression tests.
>>
>> Thanks for your time on this.  As you can tell, I'm inexperienced in
>> writing openjdk code, so your patience and careful review is really
>> appreciated.
>>
>> - Corey


From LANCE.ANDERSEN at ORACLE.COM  Mon Aug 31 17:14:44 2020
From: LANCE.ANDERSEN at ORACLE.COM (Lance Andersen)
Date: Mon, 31 Aug 2020 13:14:44 -0400
Subject: RFR: 8252552: DecimalFormat javadoc contains HTML tags in example
 code
In-Reply-To: <4219fa9a-f90f-8e75-30b0-eb0f084c1ffa@oracle.com>
References: <4219fa9a-f90f-8e75-30b0-eb0f084c1ffa@oracle.com>
Message-ID: <14AD1FE8-F496-4EA9-8EB6-E8618D98B0E1@ORACLE.COM>

Hi Naoto,

Looks OK to me.  

Best
Lance

> On Aug 31, 2020, at 12:22 PM, Naoto Sato  wrote:
> 
> Hello,
> 
> Please review this simple javadoc fix to:
> 
> https://bugs.openjdk.java.net/browse/JDK-8252552
> 
> The proposed diff is inserted here:
> 
> ---
> --- old/src/java.base/share/classes/java/text/DecimalFormat.java 2020-08-31 09:07:22.000000000 -0700
> +++ new/src/java.base/share/classes/java/text/DecimalFormat.java 2020-08-31 09:07:22.000000000 -0700
> @@ -338,9 +338,9 @@
>  *
>  * 
    Example
    
>  *
> - * 
    {@code
> - * // Print out a number using the localized number, integer, currency,
> - * // and percent format for each locale
> + * 
    {@code
> + * // Print out a number using the localized number, integer, currency,
> + * // and percent format for each locale}{@code
>  * Locale[] locales = NumberFormat.getAvailableLocales();
>  * double myNumber = -1234.56;
>  * NumberFormat form;
> ---
> 
> It is simply to take those  HTML tags out of {@code} snippet.
> 
> Naoto


Best
Lance
------------------




Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
Oracle Java Engineering 
1 Network Drive 
Burlington, MA 01803
Lance.Andersen at oracle.com





From serban.iordache at gmail.com  Sat Aug 29 08:37:30 2020
From: serban.iordache at gmail.com (Serban Iordache)
Date: Sat, 29 Aug 2020 10:37:30 +0200
Subject: jpackage: support for environment variables in --java-options
Message-ID: 

Hi,

Is there a way to pass values from environment variables when using
--java-options?

It would be nice to be able to write something like this:
--java-options "-DmyAppData=$HOME/.myData"

(Instead of using the $ sign, another notation may be more appropriate, in
order to not conflict with macros such as $APPDIR or $ROOTDIR.)

- Serban

From ian.graves at oracle.com  Mon Aug 31 18:34:24 2020
From: ian.graves at oracle.com (Ian Graves)
Date: Mon, 31 Aug 2020 13:34:24 -0500
Subject: [PATCH] Docfix for Unsafe regarding fence methods
In-Reply-To: 
References: 
 <70AA9B93-A9AA-4FD0-8EBA-AE3A8DF473A0@oracle.com>
 
Message-ID: <8F042B4E-6409-46A7-AE37-EA7ED00CF47C@oracle.com>

Hans,

You raise a good point! I was attempting to re-adapt the existing language with the changes I made. 

I?m going to update this post to an RFR after the Skara update next week and repost (got mixed up on my process). There will be some simplified language that hopefully addresses your concerns.

Thanks for the input!

Ian

> On Aug 28, 2020, at 4:59 PM, Hans Boehm  wrote:
> 
> 
> 
> On Fri, Aug 28, 2020 at 1:21 PM Ian Graves > wrote:
> >
> > Looks like the patch attachment was stripped. I?ll include it inline:
> > ?Ian
> >
> > ...
> > @@ -3458,6 +3462,12 @@
> >      /**
> >       * Ensures that stores before the fence will not be reordered with
> >       * stores after the fence.
> > +     *
> > +     * This method is operationally equivalent to {@link #storeFence()}
> > +     * as most hardware instructions that provide a StoreStore barrier
> > +     * provide a LoadStore barrier for free.
> > +     *
> > +     * @since 9
> >       */
> >      public final void storeStoreFence() {
> >          storeFence();
> >
> >
> This seems a little misleading, since ARM has "dmb ishst", which orders only stores?


From huizhe.wang at oracle.com  Mon Aug 31 19:55:19 2020
From: huizhe.wang at oracle.com (Joe Wang)
Date: Mon, 31 Aug 2020 12:55:19 -0700
Subject: RFR: 8252552: DecimalFormat javadoc contains HTML tags in example
 code
In-Reply-To: <14AD1FE8-F496-4EA9-8EB6-E8618D98B0E1@ORACLE.COM>
References: <4219fa9a-f90f-8e75-30b0-eb0f084c1ffa@oracle.com>
 <14AD1FE8-F496-4EA9-8EB6-E8618D98B0E1@ORACLE.COM>
Message-ID: <15e08b3a-fcc2-5151-4e28-e9b2b611dfbf@oracle.com>

+1

Or, maybe  is not needed for code comments

-Joe

On 8/31/20 10:14 AM, Lance Andersen wrote:
> Hi Naoto,
>
> Looks OK to me.
>
> Best
> Lance
>
>> On Aug 31, 2020, at 12:22 PM, Naoto Sato  wrote:
>>
>> Hello,
>>
>> Please review this simple javadoc fix to:
>>
>> https://bugs.openjdk.java.net/browse/JDK-8252552
>>
>> The proposed diff is inserted here:
>>
>> ---
>> --- old/src/java.base/share/classes/java/text/DecimalFormat.java 2020-08-31 09:07:22.000000000 -0700
>> +++ new/src/java.base/share/classes/java/text/DecimalFormat.java 2020-08-31 09:07:22.000000000 -0700
>> @@ -338,9 +338,9 @@
>>   *
>>   * 
    Example
    
>>   *
>> - * 
    {@code
>> - * // Print out a number using the localized number, integer, currency,
>> - * // and percent format for each locale
>> + * 
    {@code
>> + * // Print out a number using the localized number, integer, currency,
>> + * // and percent format for each locale}{@code
>>   * Locale[] locales = NumberFormat.getAvailableLocales();
>>   * double myNumber = -1234.56;
>>   * NumberFormat form;
>> ---
>>
>> It is simply to take those  HTML tags out of {@code} snippet.
>>
>> Naoto
>
> Best
> Lance
> ------------------
>
>
>
>
> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
> Oracle Java Engineering
> 1 Network Drive
> Burlington, MA 01803
> Lance.Andersen at oracle.com
>
>
>
>


From mik3hall at gmail.com  Mon Aug 31 20:07:17 2020
From: mik3hall at gmail.com (Michael Hall)
Date: Mon, 31 Aug 2020 15:07:17 -0500
Subject: jpackage: support for environment variables in --java-options
In-Reply-To: 
References: 
Message-ID: 



> On Aug 29, 2020, at 3:37 AM, Serban Iordache  wrote:
> 
> Hi,
> 
> Is there a way to pass values from environment variables when using
> --java-options?
> 
> It would be nice to be able to write something like this:
> --java-options "-DmyAppData=$HOME/.myData?

For this couldn?t you just pass the parameter without the path and then add it later?

-DmyAppData=/.myData

final String HOME = System.getProperty(?user.home?);
System.setProperty(?myAppData?,HOME+System.getProperty(myAppData),?.myData?);




From mik3hall at gmail.com  Mon Aug 31 20:11:40 2020
From: mik3hall at gmail.com (Michael Hall)
Date: Mon, 31 Aug 2020 15:11:40 -0500
Subject: jpackage environment variables - OS X
Message-ID: <0B644849-7A60-4FAF-9812-BDF84A6AEDE5@gmail.com>

Is there a way on OS X to set these in the Info.plist?

From mik3hall at gmail.com  Mon Aug 31 20:14:17 2020
From: mik3hall at gmail.com (Michael Hall)
Date: Mon, 31 Aug 2020 15:14:17 -0500
Subject: jpackage: support for environment variables in --java-options
In-Reply-To: 
References: 
 
Message-ID: 



> On Aug 31, 2020, at 3:07 PM, Michael Hall  wrote:
> 
> 
> 
>> On Aug 29, 2020, at 3:37 AM, Serban Iordache  wrote:
>> 
>> Hi,
>> 
>> Is there a way to pass values from environment variables when using
>> --java-options?
>> 
>> It would be nice to be able to write something like this:
>> --java-options "-DmyAppData=$HOME/.myData?
> 
> For this couldn?t you just pass the parameter without the path and then add it later?
> 
> -DmyAppData=/.myData
> 
> final String HOME = System.getProperty(?user.home?);
> System.setProperty(?myAppData?,HOME+System.getProperty(myAppData),?.myData?);

For user data these days don?t many platforms have preferred places for user data other than user home?
It has occurred to me it might be nice of java had an API that actually automatically gave you the correct directory for a given platform to put your data. Maybe different locations for different types of user data.


From naoto.sato at oracle.com  Mon Aug 31 21:12:56 2020
From: naoto.sato at oracle.com (Naoto Sato)
Date: Mon, 31 Aug 2020 14:12:56 -0700
Subject: RFR: 8252552: DecimalFormat javadoc contains HTML tags in example
 code
In-Reply-To: <15e08b3a-fcc2-5151-4e28-e9b2b611dfbf@oracle.com>
References: <4219fa9a-f90f-8e75-30b0-eb0f084c1ffa@oracle.com>
 <14AD1FE8-F496-4EA9-8EB6-E8618D98B0E1@ORACLE.COM>
 <15e08b3a-fcc2-5151-4e28-e9b2b611dfbf@oracle.com>
Message-ID: <46e95fff-3f12-1813-8037-b1584d81156e@oracle.com>

Thanks, Joe!

On 8/31/20 12:55 PM, Joe Wang wrote:
> +1
> 
> Or, maybe  is not needed for code comments

Yes. I just kept it as is, as it seems the comment summarizes what the 
examples do.

Naoto

> 
> -Joe
> 
> On 8/31/20 10:14 AM, Lance Andersen wrote:
>> Hi Naoto,
>>
>> Looks OK to me.
>>
>> Best
>> Lance
>>
>>> On Aug 31, 2020, at 12:22 PM, Naoto Sato  wrote:
>>>
>>> Hello,
>>>
>>> Please review this simple javadoc fix to:
>>>
>>> https://bugs.openjdk.java.net/browse/JDK-8252552
>>>
>>> The proposed diff is inserted here:
>>>
>>> ---
>>> --- old/src/java.base/share/classes/java/text/DecimalFormat.java 
>>> 2020-08-31 09:07:22.000000000 -0700
>>> +++ new/src/java.base/share/classes/java/text/DecimalFormat.java 
>>> 2020-08-31 09:07:22.000000000 -0700
>>> @@ -338,9 +338,9 @@
>>> ? *
>>> ? * 
    Example
    
>>> ? *
>>> - * 
    {@code
>>> - * // Print out a number using the localized number, 
>>> integer, currency,
>>> - * // and percent format for each locale
>>> + * 
    {@code
>>> + * // Print out a number using the localized number, integer, currency,
>>> + * // and percent format for each locale}{@code
>>> ? * Locale[] locales = NumberFormat.getAvailableLocales();
>>> ? * double myNumber = -1234.56;
>>> ? * NumberFormat form;
>>> ---
>>>
>>> It is simply to take those  HTML tags out of {@code} snippet.
>>>
>>> Naoto
>>
>> Best
>> Lance
>> ------------------
>>
>>
>>
>>
>> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
>> Oracle Java Engineering
>> 1 Network Drive
>> Burlington, MA 01803
>> Lance.Andersen at oracle.com
>>
>>
>>
>>
> 

From cjashfor at linux.ibm.com  Mon Aug 31 22:22:47 2020
From: cjashfor at linux.ibm.com (Corey Ashford)
Date: Mon, 31 Aug 2020 15:22:47 -0700
Subject: RFR(M): 8248188: [PATCH] Add HotSpotIntrinsicCandidate and API
 for Base64 decoding
In-Reply-To: <65ed7919-86fc-adfa-3cd5-58dd96a3487f@linux.ibm.com>
References: <11ca749f-3015-c004-aa6b-3194e1dfe4eb@linux.ibm.com>
 
 
 
 
 <8ece8d2e-fd99-b734-211e-a32b534a7dc8@linux.ibm.com>
 <8d53dcf8-635a-11e2-4f6a-39b70e2c3b8b@oracle.com>
 <65ed7919-86fc-adfa-3cd5-58dd96a3487f@linux.ibm.com>
Message-ID: 

On 8/29/20 1:19 PM, Corey Ashford wrote:
> Hi Roger,
> 
> Thanks for your reply and thoughts!? Comments interspersed below:
> 
> On 8/28/20 10:54 AM, Roger Riggs wrote:
...
>> Comparing with the way that the Base64 encoder was intrinsified, the
>> method that is intrinsified should have a method body that does
>> the same function, so it is interchangable.? That likely will just shift
>> the "fast path" code into the decodeBlock method.
>> Keeping the symmetry between encoder and decoder will
>> make it easier to maintain the code.
> 
> Good point.? I'll investigate what this looks like in terms of the 
> actual code, and will report back (perhaps in a new webrev).
> 

Having looked at this again, I don't think it makes sense.  One thing 
that differs significantly from the encodeBlock intrinsic is that the 
decodeBlock intrinsic only needs to process a prefix of the data, and so 
it can leave virtually any amount of data at the end of the src buffer 
unprocessed, where as with the encodeBlock intrinsic, if it exists, it 
must process the entire buffer.

In the (common) case where the decodeBlock intrinsic returns not having 
processed everything, it still needs to call the Java code, and if that 
Java code is "replaced" by the intrinsic, it's inaccessible.

Is there something I'm overlooking here?  Basically I want the decode 
API to behave differently than the encode API, mostly to make the 
arch-specific intrinsic easier to implement. If that's not acceptable, 
then I need to rethink the API, and also figure out how to deal with the 
illegal character case.  The latter could perhaps be done by throwing an 
exception from the intrinsic, or maybe by returning a negative length 
that specifies the index of the illegal src byte, and then have the Java 
code throw the exception).

Regards,

- Corey