From david.holmes at oracle.com Mon Jun 1 00:54:37 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 1 Jun 2020 10:54:37 +1000 Subject: Sometimes constraints are questionable In-Reply-To: References: <4A5AC9D1-AEB7-47E3-9E25-BF74FE01F3B0@oracle.com> Message-ID: <1e7cb0bf-d50e-9fad-dc49-08d096833990@oracle.com> On 1/06/2020 9:52 am, Martin Buchholz wrote: > On Sun, May 31, 2020 at 4:35 PM David Holmes wrote: > >> Not sure how we could have minCapacity < 0 at this point. It should have >> been checked before the call to grow, and grow will not make it negative. > > At least some of the grow methods were designed so that callers did > not have to do the checking, and grow would interpret negative values > as int overflow > >>> It just seems that it's pushing the inevitable off to Arrays.copyOf. Shouldn't it be: >>> >>> private static int hugeCapacity(int minCapacity) { >>> if (minCapacity < 0 || minCapacity > MAX_ARRAY_SIZE) { >>> throw >>> new OutOfMemoryError("ByteArrayChannel exceeds maximum size: " + >>> MAX_ARRAY_SIZE); >>> } >>> >>> return MAX_ARRAY_SIZE; >>> } >> >> That seems more appropriate to me - modulo the question mark over >> minCapacity being negative. > > Again, the original design was to allow a capacity with MAX_ARRAY_SIZE > < capacity <= Integer.MAX_VALUE if and when the VM also allowed it. The only way to know that is to try and create the array, catch the OOME and then adjust the value used. None of the code does that so I can't see how your claim here can be correct. My understanding, having been on the VM side, was that Hotspot's internal limits were reflected on the Java side so that the Java code could avoid the VM exception. David From martinrb at google.com Mon Jun 1 01:15:53 2020 From: martinrb at google.com (Martin Buchholz) Date: Sun, 31 May 2020 18:15:53 -0700 Subject: Sometimes constraints are questionable In-Reply-To: <1e7cb0bf-d50e-9fad-dc49-08d096833990@oracle.com> References: <4A5AC9D1-AEB7-47E3-9E25-BF74FE01F3B0@oracle.com> <1e7cb0bf-d50e-9fad-dc49-08d096833990@oracle.com> Message-ID: I'm still not deeply reading the code in java.nio but I see in AbstractStringBuilder * Will not return a capacity greater than * {@code (MAX_ARRAY_SIZE >> coder)} unless the given minimum capacity * is greater than that. My intent was that a garden variety grow method would first grow to MAX_ARRAY_SIZE, then to Integer.MAX_VALUE. (this doesn't always work, e.g. hash tables want to have a power-of-two sized backing array) (these methods have evolved recently without much help from me) (Library code should know as little about VM implementation idiosyncrasies as possible) On Sun, May 31, 2020 at 5:56 PM David Holmes wrote: > > On 1/06/2020 9:52 am, Martin Buchholz wrote: > > On Sun, May 31, 2020 at 4:35 PM David Holmes wrote: > > > >> Not sure how we could have minCapacity < 0 at this point. It should have > >> been checked before the call to grow, and grow will not make it negative. > > > > At least some of the grow methods were designed so that callers did > > not have to do the checking, and grow would interpret negative values > > as int overflow > > > >>> It just seems that it's pushing the inevitable off to Arrays.copyOf. Shouldn't it be: > >>> > >>> private static int hugeCapacity(int minCapacity) { > >>> if (minCapacity < 0 || minCapacity > MAX_ARRAY_SIZE) { > >>> throw > >>> new OutOfMemoryError("ByteArrayChannel exceeds maximum size: " + > >>> MAX_ARRAY_SIZE); > >>> } > >>> > >>> return MAX_ARRAY_SIZE; > >>> } > >> > >> That seems more appropriate to me - modulo the question mark over > >> minCapacity being negative. > > > > Again, the original design was to allow a capacity with MAX_ARRAY_SIZE > > < capacity <= Integer.MAX_VALUE if and when the VM also allowed it. > > The only way to know that is to try and create the array, catch the OOME > and then adjust the value used. None of the code does that so I can't > see how your claim here can be correct. My understanding, having been on > the VM side, was that Hotspot's internal limits were reflected on the > Java side so that the Java code could avoid the VM exception. > > David From david.holmes at oracle.com Mon Jun 1 01:34:00 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 1 Jun 2020 11:34:00 +1000 Subject: RFR: JDK-8225056 VM support for sealed classes In-Reply-To: <9da783ba-edd9-b5fe-0476-644ba7d01990@oracle.com> References: <7b3430e1-f821-1e2d-2c8b-f1c621f059da@oracle.com> <9d7da8af-cda3-693d-1ea1-1db5069fea97@oracle.com> <9b32addd-d576-268e-61ab-0ac4921d22f5@oracle.com> <151289f6-820c-08d1-c2f9-85b18d1bcaf5@oracle.com> <0749bff1-02ac-841e-4bd7-4a511a90be9d@oracle.com> <9da783ba-edd9-b5fe-0476-644ba7d01990@oracle.com> Message-ID: Hi Harold, On 1/06/2020 8:57 am, Harold Seigel wrote: > Thanks for the comments. > > Here's version 3 of the JDK and VM changes for sealed classes. > > full webrev: > http://cr.openjdk.java.net/~hseigel/sealedClasses.8225056.3/webrev/ > > The new webrev contains just the following three changes: > > 1. The sealed classes API's in Class.java (permittedSubclasses() and > isSealed()) were revised and, in particular, API > permittedSubclasses() no longer uses reflection. For those following along we have presently abandoned the attempt to cache the array in ReflectionData. Current changes look okay. But I note from the CSR there appears to be a further minor update to the javadoc coming. > 2. An unneeded 'if' statement was removed from > JVM_GetPermittedSubclasses() (pointed out by David.) Looks good. > 3. VM runtime test files SealedUnnamedModuleIntfTest.java and > Permitted.java were changed to add a test case for a non-public > permitted subclass and its sealed superclass being in the same > module and package. Looks good. > Additionally, two follow on RFE's will be filed.? One to add additional > VM sealed classes tests Thanks. I think there is a more mechanical approach to testing here that will allow the complete matrix to be easily covered with minimal duplication between testing for named and unnamed modules. > and one to improve the implementations of the > sealed classes API's in Class.java. Thanks. David ----- > Thanks, Harold > > On 5/28/2020 8:30 PM, David Holmes wrote: >> >> Hi Harold, >> >> Sorry Mandy's comment raised a couple of issues ... >> >> On 29/05/2020 7:12 am, Mandy Chung wrote: >>> Hi Harold, >>> >>> On 5/27/20 1:35 PM, Harold Seigel wrote: >>>> >>>> Incremental webrev: >>>> http://cr.openjdk.java.net/~hseigel/sealedClasses.8225056.incr.2/ >>>> >>>> full webrev: >>>> http://cr.openjdk.java.net/~hseigel/sealedClasses.8225056.2/webrev/ >>>> >>> Class.java >>> >>> 4406 ReflectionData rd = reflectionData(); >>> 4407 ClassDesc[] tmp = rd.permittedSubclasses; >>> 4408 if (tmp != null) { >>> 4409 return tmp; >>> 4410 } >>> 4411 >>> 4412 if (isArray() || isPrimitive()) { >>> 4413 rd.permittedSubclasses = new ClassDesc[0]; >>> 4414 return rd.permittedSubclasses; >>> 4415 } >>> >>> This causes an array class or primitive type to create a >>> ReflectionData.?? It should first check if this is non-sealed class >>> and returns a constant empty array. >> >> It can't check if this is a non-sealed class as the isSealed() check >> calls the above code! But for arrays and primitives which can't be >> sealed we should just do: >> >> 4412 if (isArray() || isPrimitive()) { >> 4413 return new ClassDesc[0]; >> 4414 } >> >> But this then made me realize that we need to be creating defensive >> copies of the returned arrays, as happens with other APIs that use >> ReflectionData. >> >> Backing up a bit I complained that: >> >> public boolean isSealed() { >> return permittedSubclasses().length != 0; >> } >> >> is a very inefficient way to answer the question as to whether a class >> is sealed, so I suggested that the result of permittedSubclasses() be >> cached. Caching is not without its own issues as we are discovering, >> and when you add in defensive copies this seems to be trading one >> inefficiency for another. For nestmates we don't cache >> getNestMembers() because we don;t think it will be called often - it >> is there to complete the introspection API of Class rather than being >> anticipated as used in a regular programmatic sense. I expect the same >> is true for permittedSubclasses(). Do we expect isSealed() to be used >> often or is it too just there for completeness? If just for >> completeness then perhaps a VM query would be a better compromise on >> the efficiency front? Otherwise I can accept the current >> implementation of isSealed(), and a non-caching permittedClasses() for >> this initial implementation of sealed classes. If efficiency turns out >> to be a problem for isSealed() then we can revisit it then. >> >> Thanks, >> David >> >> >>> In fact, ReflectionData caches the derived names and reflected >>> members for performance and also they may be invalidated when the >>> class is redefined.?? It might be okay to add >>> ReflectionData::permittedSubclasses while `PermittedSubclasses` >>> attribute can't be redefined and getting this attribute is not >>> performance sensitive.?? For example, the result of `getNestMembers` >>> is not cached in ReflectionData.? It may be better not to add it in >>> ReflectionData for modifiable and performance-sensitive data. >>> >>> >>> 4421 tmp = new ClassDesc[subclassNames.length]; >>> 4422 int i = 0; >>> 4423 for (String subclassName : subclassNames) { >>> 4424 try { >>> 4425 tmp[i++] = ClassDesc.of(subclassName.replace('/', '.')); >>> 4426 } catch (IllegalArgumentException iae) { >>> 4427 throw new InternalError("Invalid type in permitted subclasses >>> information: " + subclassName, iae); >>> 4428 } >>> 4429 } >>> Nit: rename tmp to some other name e.g. descs >>> >>> I read the JVMS but it isn't clear to me that the VM will validate >>> the names in `PermittedSubclasses`attribute are valid class >>> descriptors.?? I see ConstantPool::is_klass_or_reference check but >>> can't find where it validates the name is a valid class descriptor - >>> can you point me there??? (otherwise, maybe define it to be unspecified?) >>> >>> >>> W.r.t. the APIs. I don't want to delay this review.? I see that you >>> renamed the method to new API style as I brought up.? OTOH,? I expect >>> more discussion is needed.? Below is a recent comment from John on >>> this topic [1] >>> >>>> One comment, really for the future, regarding the shape of the Java >>>> API here: It uses Optional and omits the "get" prefix on accessors. >>>> This is the New Style, as opposed to the Classic Style using null >>>> (for "empty" results) and a "get" prefix ("getComponentType") to get >>>> a related type. We may choose to to use the New Style for new >>>> reflection API points, and if so let's not choose N different New >>>> Styles, but one New Style. Personally I like removing "get"; I >>>> accept Optional instead of null; and I also suggest that arrays (if >>>> any) be replaced by (immutable) Lists in the New Style >>> >>> There are a few existing Class APIs that use the new API style: >>> Class arrayClass(); >>> Optional describeConstable(); >>> String descriptorString(); >>> >>> This will set up a precedence of the new API style in this class. >>> Should this new permittedSubclasses method return a List instead of >>> an array?? It's okay with me if you prefer to revert back to the old >>> API style and follow this up after integration. >>> >>> 4442 * Returns true if this {@linkplain Class} is sealed. >>> 4443 * >>> 4444 * @return returns true if this class is sealed >>> >>> NIt: {@code true} instead of true -? consistent with the style this >>> class uses (in most methods) >>> >>> test/jdk/java/lang/reflect/sealed_classes/SealedClassesReflectionTest.java >>> >>> Nit: s/sealed_classes/sealedClasses/ >>> - the test directory/file naming convention use camel case or java >>> variable name convention. >>> >>> Thanks >>> [1] https://github.com/openjdk/valhalla/pull/53#issuecomment-633116043 From mandy.chung at oracle.com Mon Jun 1 03:27:04 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Sun, 31 May 2020 20:27:04 -0700 Subject: RFR: JDK-8245432: Lookup::defineHiddenClass should throw UnsupportedClassVersionError if the given bytes are of an unsupported major or minor version In-Reply-To: <9d0642dc-ea62-5a08-c257-0bf82dc2b48c@j-kuhn.de> References: <176af0b0-cce9-9286-0b06-9455c0f8cea4@oracle.com> <50b420bf-a53e-06d1-43f3-d5bdac804e7d@oracle.com> <700459437.2144177.1590601182749.JavaMail.zimbra@u-pem.fr> <543b4fc8-dcbb-b8c8-4111-f200885921cd@oracle.com> <46328704-3f68-9643-71b7-d7cdc7088af3@oracle.com> <8625b4fe-6035-9096-a94d-64b0705a6595@oracle.com> <96d7b8b0-e6a2-fb2d-3786-3953c48ee72a@oracle.com> <9d0642dc-ea62-5a08-c257-0bf82dc2b48c@j-kuhn.de> Message-ID: On 5/29/20 4:31 PM, Johannes Kuhn wrote: > Thanks. > > Noted that readInt can read outside of the byte array when it is > shorter than 4 bytes, throwing an ArrayIndexOutOfBoundsException. > Same applies for readUnsignedShort. > > Test cases for malformedClassFiles: > new byte[3], new byte[] {0xCA, 0xFE, 0xBA, 0xBE, 0x00} > Oops.? My bad.? The range check should be fixed to: ?? if ((offset+4) > bytes.length) { ?????? throw new ClassFormatError("Invalid ClassFile structure"); ?? } similarly the check for readUnsignedShort is: ??? if ((offset+2) > bytes.length) Mandy > - Johannes > > > On 29-May-20 21:23, Mandy Chung wrote: >> It's fixed.? Thanks >> Mandy >> >> On 5/28/20 4:35 PM, Johannes Kuhn wrote: >>> Hi, >>> >>> just noticed a small thing: >>> The magic is 4 bytes, but readUnsignedShort reads two bytes. >>> >>> - Johannes >>> >>> On 28-May-20 19:25, Mandy Chung wrote: >>>> >>>> >>>> On 5/28/20 6:55 AM, Alan Bateman wrote: >>>>> On 28/05/2020 05:13, Mandy Chung wrote: >>>>>> Updated webrev: >>>>>> http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8245432/webrev.01/ >>>>>> >>>>>> I modify this patch to check the class file version and throws >>>>>> CFE if unsupported before creating ClassReader. This also fixes >>>>>> JDK-8245061 that it reads the value of `this_class` as a constant >>>>>> (as ASM will throw an exception if it's not a constant) and also >>>>>> ensures that `this_class` is a CONSTANT_Class_info by checking >>>>>> the descriptor string from Type. >>>>> I don't want to complicate this further but the --enable-preview >>>>> handling in the VM doesn't seem to expose an internal property >>>>> that allows code in the libraries know if preview feature are >>>>> enabled or not. I was assuming that isSupportedClassFileVersion >>>>> would need to check that. >>>>> >>>> >>>> The runtime will ensure if --enable-preview is set if a class file >>>> with minor is loaded.?? I will prefer to keep >>>> VM::isSupportedClassFileVersion to validate the given major/minor >>>> version.? At runtime, it will fail when such class file is loaded >>>> if preview is not enabled. >>>> >>>> I'll add a comment to describe that. >>>> >>>>> Will readUnsignedShort throw AIOBE if the offset is beyond the >>>>> length? >>>> >>>> Good catch.? I add the check to throw CFE. >>>> +??????????? private static int readUnsignedShort(byte[] bytes, int >>>> offset) { >>>> +??????????????? if (offset >= bytes.length) { >>>> +??????????????????? throw new ClassFormatError("Invalid ClassFile >>>> structure"); >>>> +??????????????? } >>>> +??????????????? return ((bytes[offset] & 0xFF) << 8) | >>>> (bytes[offset + 1] & 0xFF); >>>> +??????????? } >>>> >>>>> Also are you sure about "& 0xCAFEBABE", I assumed it would just >>>>> check the magic number is equal to that. >>>> >>>> It should check ==.? Fixed. >>>> >>>> thanks >>>> Mandy >>> >>> >> > From claes.redestad at oracle.com Mon Jun 1 11:07:00 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 1 Jun 2020 13:07:00 +0200 Subject: RFR: 8246241: LambdaFormEditor should use a transform lookup key that is not a SoftReference Message-ID: Hi, the LambdaFormEditor uses a SoftReference-based cache to avoid generating LambdaForms we have already generated. By introducing a lookup key that is not a SoftReference we can improve this by allocating less up front and by inducing less work for the GC later on. I also refactored a few cases where I had neglected to add a static factory method to create the Transform (now TransformKey). For the transform types affected this helps further reduce allocations. Bug: https://bugs.openjdk.java.net/browse/JDK-8246241 Webrev: http://cr.openjdk.java.net/~redestad/8246241/open.00/ Testing: tier1-2, java/lang/invoke locally with STRESS_TEST=true Verified a speed-up on the micro added by JDK-8246152)[1], and on related startup tests[2]. Thanks! /Claes [1] Before (JDK-8246152): Benchmark Mode Cnt Score Error Units StringConcatFactoryBootstraps.makeConcatWithConstants avgt 10 2764.634 ? 56.244 ns/op StringConcatFactoryBootstraps.makeConcatWithConstants:?gc.alloc.rate.norm avgt 10 4136.187 ? 0.013 B/op After: Benchmark Mode Cnt Score Error Units StringConcatFactoryBootstraps.makeConcatWithConstants avgt 10 2299.919 ? 58.003 ns/op StringConcatFactoryBootstraps.makeConcatWithConstants:?gc.alloc.rate.norm avgt 10 3308.145 ? 6.378 B/op [2] http://cr.openjdk.java.net/~redestad/scratch/erase_scf_types.00/ObjStringCombos.java Before (JDK-8246152): 3,485,679,714 instructions # 1.09 insn per cycle ( +- 0.90% ) 692,247,529 branches # 491.822 M/sec ( +- 0.92% ) 28,939,534 branch-misses # 4.18% of all branches ( +- 0.90% ) 0.291642667 seconds time elapsed ( +- 0.93% ) After: 3,411,619,586 instructions # 1.09 insn per cycle ( +- 0.63% ) 677,767,258 branches # 493.911 M/sec ( +- 0.64% ) 28,337,022 branch-misses # 4.18% of all branches ( +- 0.63% ) 0.288959534 seconds time elapsed ( +- 0.86% ) From james.laskey at oracle.com Mon Jun 1 11:47:23 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Mon, 1 Jun 2020 08:47:23 -0300 Subject: Sometimes constraints are questionable In-Reply-To: References: <4A5AC9D1-AEB7-47E3-9E25-BF74FE01F3B0@oracle.com> Message-ID: <2F2E688C-51EF-4DAF-826E-70669940538D@oracle.com> Thanks David will run with that, > On May 31, 2020, at 8:34 PM, David Holmes wrote: > > On 31/05/2020 12:29 am, Jim Laskey wrote: >> I'm working through https://bugs.openjdk.java.net/browse/JDK-8230744 Several classes throw OutOfMemoryError without message . >> I'm wondering why hugeCapacity in src/jdk.zipfs/share/classes/jdk/nio/zipfs/ByteArrayChannel.java is defined as >> /** >> * The maximum size of array to allocate. >> * Some VMs reserve some header words in an array. >> * Attempts to allocate larger arrays may result in >> * OutOfMemoryError: Requested array size exceeds VM limit >> */ >> private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; >> /** >> * Increases the capacity to ensure that it can hold at least the >> * number of elements specified by the minimum capacity argument. >> * >> * @param minCapacity the desired minimum capacity >> */ >> private void grow(int minCapacity) { >> // overflow-conscious code >> int oldCapacity = buf.length; >> int newCapacity = oldCapacity << 1; >> if (newCapacity - minCapacity < 0) >> newCapacity = minCapacity; >> if (newCapacity - MAX_ARRAY_SIZE > 0) >> newCapacity = hugeCapacity(minCapacity); >> buf = Arrays.copyOf(buf, newCapacity); >> } >> private static int hugeCapacity(int minCapacity) { >> if (minCapacity < 0) // overflow >> throw new OutOfMemoryError(); > > Not sure how we could have minCapacity < 0 at this point. It should have been checked before the call to grow, and grow will not make it negative. > >> return (minCapacity > MAX_ARRAY_SIZE) ? >> Integer.MAX_VALUE : >> MAX_ARRAY_SIZE; > > That's a bug plain and simple. It should never report a size > MAX_ARRAY_SIZE. > >> } >> It just seems that it's pushing the inevitable off to Arrays.copyOf. Shouldn't it be: >> private static int hugeCapacity(int minCapacity) { >> if (minCapacity < 0 || minCapacity > MAX_ARRAY_SIZE) { >> throw >> new OutOfMemoryError("ByteArrayChannel exceeds maximum size: " + >> MAX_ARRAY_SIZE); >> } >> return MAX_ARRAY_SIZE; >> } > > That seems more appropriate to me - modulo the question mark over minCapacity being negative. > >> Real question: is there some hidden purpose behind this kind of logic? > > The basic strategy is to double the current capacity unless that will trigger an unnecessary exception, in which case just use the requested capacity, but again watch for the implementation limits. > > Cheers, > David > ----- > >> Cheers, >> -- Jim From claes.redestad at oracle.com Mon Jun 1 11:55:09 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 1 Jun 2020 13:55:09 +0200 Subject: RFR: 8246251: Adjust HelloClasslist after JDK-8230301 Message-ID: <475749d0-6ac0-6554-ae34-220e1fb18858@oracle.com> Hi, comparing startup profiles on a few application I found a few cases where a few more classes are being generated at runtime compared to JDK 14. This is due to the removal of hard-coded defaults in GenerateJLIClassesPlugin[1], in combination with how earlier implementations of SCF (used at the time those hard-coded defaults were captured) were more aggressive at generating higher-arity SpeciesData classes. To compensate, this patch adds a few cases of even higher-arity String concat expressions, which smooths things out a bit (effect of JDK-8230301 is still good since it removes a number of hard-coded shapes we are unlikely to ever produce on the current JDK) Bug: https://bugs.openjdk.java.net/browse/JDK-8246251 Webrev: http://cr.openjdk.java.net/~redestad/8246251/open.00/ Testing: tier1-2 Thanks! /Claes [1] https://bugs.openjdk.java.net/browse/JDK-8230301 From james.laskey at oracle.com Mon Jun 1 11:56:56 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Mon, 1 Jun 2020 08:56:56 -0300 Subject: Sometimes constraints are questionable In-Reply-To: References: <4A5AC9D1-AEB7-47E3-9E25-BF74FE01F3B0@oracle.com> <1e7cb0bf-d50e-9fad-dc49-08d096833990@oracle.com> Message-ID: <8F2133B8-A1AD-4031-87B5-0F3CD030A494@oracle.com> In the same light the test in String::replace int resultLen; try { resultLen = Math.addExact(valLen, Math.multiplyExact(++p, replLen - targLen)); } catch (ArithmeticException ignored) { throw new OutOfMemoryError("String resulting from replace exceeds maximum string size"); } Is useless in this situation (from the bug report): int missingToIntMax = 10; String tooLarge = "a".repeat(Integer.MAX_VALUE - missingToIntMax); tooLarge.replace("a", "a".repeat(missingToIntMax + 2)); The replace "test" does not fail and the exception comes from further down the stack: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at java.base/java.util.Arrays.copyOf(Arrays.java:3584) at java.base/java.lang.StringLatin1.replace(StringLatin1.java:349) at java.base/java.lang.String.replace(String.java:2173) at JI9062163.main(JI9062163.java:6) > On May 31, 2020, at 10:15 PM, Martin Buchholz wrote: > > I'm still not deeply reading the code in java.nio but I see in > AbstractStringBuilder > > * Will not return a capacity greater than > * {@code (MAX_ARRAY_SIZE >> coder)} unless the given minimum capacity > * is greater than that. > > My intent was that a garden variety grow method would first grow to > MAX_ARRAY_SIZE, then to Integer.MAX_VALUE. > > (this doesn't always work, e.g. hash tables want to have a > power-of-two sized backing array) > > (these methods have evolved recently without much help from me) > > (Library code should know as little about VM implementation > idiosyncrasies as possible) > > On Sun, May 31, 2020 at 5:56 PM David Holmes wrote: >> >> On 1/06/2020 9:52 am, Martin Buchholz wrote: >>> On Sun, May 31, 2020 at 4:35 PM David Holmes wrote: >>> >>>> Not sure how we could have minCapacity < 0 at this point. It should have >>>> been checked before the call to grow, and grow will not make it negative. >>> >>> At least some of the grow methods were designed so that callers did >>> not have to do the checking, and grow would interpret negative values >>> as int overflow >>> >>>>> It just seems that it's pushing the inevitable off to Arrays.copyOf. Shouldn't it be: >>>>> >>>>> private static int hugeCapacity(int minCapacity) { >>>>> if (minCapacity < 0 || minCapacity > MAX_ARRAY_SIZE) { >>>>> throw >>>>> new OutOfMemoryError("ByteArrayChannel exceeds maximum size: " + >>>>> MAX_ARRAY_SIZE); >>>>> } >>>>> >>>>> return MAX_ARRAY_SIZE; >>>>> } >>>> >>>> That seems more appropriate to me - modulo the question mark over >>>> minCapacity being negative. >>> >>> Again, the original design was to allow a capacity with MAX_ARRAY_SIZE >>> < capacity <= Integer.MAX_VALUE if and when the VM also allowed it. >> >> The only way to know that is to try and create the array, catch the OOME >> and then adjust the value used. None of the code does that so I can't >> see how your claim here can be correct. My understanding, having been on >> the VM side, was that Hotspot's internal limits were reflected on the >> Java side so that the Java code could avoid the VM exception. >> >> David From james.laskey at oracle.com Mon Jun 1 12:21:40 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Mon, 1 Jun 2020 09:21:40 -0300 Subject: Sometimes constraints are questionable In-Reply-To: <8F2133B8-A1AD-4031-87B5-0F3CD030A494@oracle.com> References: <4A5AC9D1-AEB7-47E3-9E25-BF74FE01F3B0@oracle.com> <1e7cb0bf-d50e-9fad-dc49-08d096833990@oracle.com> <8F2133B8-A1AD-4031-87B5-0F3CD030A494@oracle.com> Message-ID: <75945C21-2DE8-42E1-AE7A-F3762EDBB213@oracle.com> My mistake it's ArraysSupport.newLength that's failing here, String::replace pos = Arrays.copyOf(pos, ArraysSupport.newLength(p, 1, p >> 1)); newLength should be causing the exception but it in turn is passing the OOM size on to Arrays.copyOf The logic in ArraysSupport::newLength is similar to that in the first part of this discussion: public static int newLength(int oldLength, int minGrowth, int prefGrowth) { // assert oldLength >= 0 // assert minGrowth > 0 int newLength = Math.max(minGrowth, prefGrowth) + oldLength; if (newLength - MAX_ARRAY_LENGTH <= 0) { return newLength; } return hugeLength(oldLength, minGrowth); } private static int hugeLength(int oldLength, int minGrowth) { int minLength = oldLength + minGrowth; if (minLength < 0) { // overflow throw new OutOfMemoryError("Required array length too large"); } if (minLength <= MAX_ARRAY_LENGTH) { return MAX_ARRAY_LENGTH; } return Integer.MAX_VALUE; } That is, pass Integer.MAX_VALUE on to the allocator and let it fail. > On Jun 1, 2020, at 8:56 AM, Jim Laskey wrote: > > In the same light the test in String::replace > > int resultLen; > try { > resultLen = Math.addExact(valLen, > Math.multiplyExact(++p, replLen - targLen)); > } catch (ArithmeticException ignored) { > throw new > OutOfMemoryError("String resulting from replace exceeds maximum string size"); > } > > Is useless in this situation (from the bug report): > > int missingToIntMax = 10; > String tooLarge = "a".repeat(Integer.MAX_VALUE - missingToIntMax); > tooLarge.replace("a", "a".repeat(missingToIntMax + 2)); > > The replace "test" does not fail and the exception comes from further down the stack: > > Exception in thread "main" java.lang.OutOfMemoryError: Java heap space > at java.base/java.util.Arrays.copyOf(Arrays.java:3584) > at java.base/java.lang.StringLatin1.replace(StringLatin1.java:349) > at java.base/java.lang.String.replace(String.java:2173) > at JI9062163.main(JI9062163.java:6) > > > > >> On May 31, 2020, at 10:15 PM, Martin Buchholz wrote: >> >> I'm still not deeply reading the code in java.nio but I see in >> AbstractStringBuilder >> >> * Will not return a capacity greater than >> * {@code (MAX_ARRAY_SIZE >> coder)} unless the given minimum capacity >> * is greater than that. >> >> My intent was that a garden variety grow method would first grow to >> MAX_ARRAY_SIZE, then to Integer.MAX_VALUE. >> >> (this doesn't always work, e.g. hash tables want to have a >> power-of-two sized backing array) >> >> (these methods have evolved recently without much help from me) >> >> (Library code should know as little about VM implementation >> idiosyncrasies as possible) >> >> On Sun, May 31, 2020 at 5:56 PM David Holmes wrote: >>> >>> On 1/06/2020 9:52 am, Martin Buchholz wrote: >>>> On Sun, May 31, 2020 at 4:35 PM David Holmes wrote: >>>> >>>>> Not sure how we could have minCapacity < 0 at this point. It should have >>>>> been checked before the call to grow, and grow will not make it negative. >>>> >>>> At least some of the grow methods were designed so that callers did >>>> not have to do the checking, and grow would interpret negative values >>>> as int overflow >>>> >>>>>> It just seems that it's pushing the inevitable off to Arrays.copyOf. Shouldn't it be: >>>>>> >>>>>> private static int hugeCapacity(int minCapacity) { >>>>>> if (minCapacity < 0 || minCapacity > MAX_ARRAY_SIZE) { >>>>>> throw >>>>>> new OutOfMemoryError("ByteArrayChannel exceeds maximum size: " + >>>>>> MAX_ARRAY_SIZE); >>>>>> } >>>>>> >>>>>> return MAX_ARRAY_SIZE; >>>>>> } >>>>> >>>>> That seems more appropriate to me - modulo the question mark over >>>>> minCapacity being negative. >>>> >>>> Again, the original design was to allow a capacity with MAX_ARRAY_SIZE >>>> < capacity <= Integer.MAX_VALUE if and when the VM also allowed it. >>> >>> The only way to know that is to try and create the array, catch the OOME >>> and then adjust the value used. None of the code does that so I can't >>> see how your claim here can be correct. My understanding, having been on >>> the VM side, was that Hotspot's internal limits were reflected on the >>> Java side so that the Java code could avoid the VM exception. >>> >>> David > From james.laskey at oracle.com Mon Jun 1 12:48:33 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Mon, 1 Jun 2020 09:48:33 -0300 Subject: RFR: JDK-8230744 Several classes throw OutOfMemoryError without message Message-ID: <045D0E84-60E4-491B-9A31-7B05043FFE3B@oracle.com> Cleans up every case of OutOfMemoryError without a message. Some logic changes in String::repeat and ByteArrayChannel:: hugeCapacity. Thank you. Cheers, -- Jim webrev: http://cr.openjdk.java.net/~jlaskey/8230744/webrev-00/index.html jbs: https://bugs.openjdk.java.net/browse/JDK-8230744 From harold.seigel at oracle.com Mon Jun 1 13:07:04 2020 From: harold.seigel at oracle.com (Harold Seigel) Date: Mon, 1 Jun 2020 09:07:04 -0400 Subject: RFR: JDK-8225056 VM support for sealed classes In-Reply-To: References: <7b3430e1-f821-1e2d-2c8b-f1c621f059da@oracle.com> <9d7da8af-cda3-693d-1ea1-1db5069fea97@oracle.com> <9b32addd-d576-268e-61ab-0ac4921d22f5@oracle.com> <151289f6-820c-08d1-c2f9-85b18d1bcaf5@oracle.com> <0749bff1-02ac-841e-4bd7-4a511a90be9d@oracle.com> <9da783ba-edd9-b5fe-0476-644ba7d01990@oracle.com> Message-ID: Hi David, Thanks for reviewing the latest changes. I'll create the follow on RFE's once the sealed classes code is in mainline. Harold On 5/31/2020 9:34 PM, David Holmes wrote: > Hi Harold, > > On 1/06/2020 8:57 am, Harold Seigel wrote: >> Thanks for the comments. >> >> Here's version 3 of the JDK and VM changes for sealed classes. >> >> full webrev: >> http://cr.openjdk.java.net/~hseigel/sealedClasses.8225056.3/webrev/ >> >> The new webrev contains just the following three changes: >> >> ?1. The sealed classes API's in Class.java (permittedSubclasses() and >> ??? isSealed()) were revised and, in particular, API >> ??? permittedSubclasses() no longer uses reflection. > > For those following along we have presently abandoned the attempt to > cache the array in ReflectionData. > > Current changes look okay. But I note from the CSR there appears to be > a further minor update to the javadoc coming. > >> ?2. An unneeded 'if' statement was removed from >> ??? JVM_GetPermittedSubclasses() (pointed out by David.) > > Looks good. > >> ?3. VM runtime test files SealedUnnamedModuleIntfTest.java and >> ??? Permitted.java were changed to add a test case for a non-public >> ??? permitted subclass and its sealed superclass being in the same >> ??? module and package. > > Looks good. > >> Additionally, two follow on RFE's will be filed.? One to add >> additional VM sealed classes tests > > Thanks. I think there is a more mechanical approach to testing here > that will allow the complete matrix to be easily covered with minimal > duplication between testing for named and unnamed modules. > >> and one to improve the implementations of the sealed classes API's in >> Class.java. > > Thanks. > > David > ----- > >> Thanks, Harold >> >> On 5/28/2020 8:30 PM, David Holmes wrote: >>> >>> Hi Harold, >>> >>> Sorry Mandy's comment raised a couple of issues ... >>> >>> On 29/05/2020 7:12 am, Mandy Chung wrote: >>>> Hi Harold, >>>> >>>> On 5/27/20 1:35 PM, Harold Seigel wrote: >>>>> >>>>> Incremental webrev: >>>>> http://cr.openjdk.java.net/~hseigel/sealedClasses.8225056.incr.2/ >>>>> >>>>> full webrev: >>>>> http://cr.openjdk.java.net/~hseigel/sealedClasses.8225056.2/webrev/ >>>>> >>>> Class.java >>>> >>>> 4406 ReflectionData rd = reflectionData(); >>>> 4407 ClassDesc[] tmp = rd.permittedSubclasses; >>>> 4408 if (tmp != null) { >>>> 4409 return tmp; >>>> 4410 } >>>> 4411 >>>> 4412 if (isArray() || isPrimitive()) { >>>> 4413 rd.permittedSubclasses = new ClassDesc[0]; >>>> 4414 return rd.permittedSubclasses; >>>> 4415 } >>>> >>>> This causes an array class or primitive type to create a >>>> ReflectionData.?? It should first check if this is non-sealed class >>>> and returns a constant empty array. >>> >>> It can't check if this is a non-sealed class as the isSealed() check >>> calls the above code! But for arrays and primitives which can't be >>> sealed we should just do: >>> >>> 4412 if (isArray() || isPrimitive()) { >>> 4413 return new ClassDesc[0]; >>> 4414 } >>> >>> But this then made me realize that we need to be creating defensive >>> copies of the returned arrays, as happens with other APIs that use >>> ReflectionData. >>> >>> Backing up a bit I complained that: >>> >>> public boolean isSealed() { >>> return permittedSubclasses().length != 0; >>> } >>> >>> is a very inefficient way to answer the question as to whether a >>> class is sealed, so I suggested that the result of >>> permittedSubclasses() be cached. Caching is not without its own >>> issues as we are discovering, and when you add in defensive copies >>> this seems to be trading one inefficiency for another. For nestmates >>> we don't cache getNestMembers() because we don;t think it will be >>> called often - it is there to complete the introspection API of >>> Class rather than being anticipated as used in a regular >>> programmatic sense. I expect the same is true for >>> permittedSubclasses(). Do we expect isSealed() to be used often or >>> is it too just there for completeness? If just for completeness then >>> perhaps a VM query would be a better compromise on the efficiency >>> front? Otherwise I can accept the current implementation of >>> isSealed(), and a non-caching permittedClasses() for this initial >>> implementation of sealed classes. If efficiency turns out to be a >>> problem for isSealed() then we can revisit it then. >>> >>> Thanks, >>> David >>> >>> >>>> In fact, ReflectionData caches the derived names and reflected >>>> members for performance and also they may be invalidated when the >>>> class is redefined.?? It might be okay to add >>>> ReflectionData::permittedSubclasses while `PermittedSubclasses` >>>> attribute can't be redefined and getting this attribute is not >>>> performance sensitive.?? For example, the result of >>>> `getNestMembers` is not cached in ReflectionData.? It may be better >>>> not to add it in ReflectionData for modifiable and >>>> performance-sensitive data. >>>> >>>> >>>> 4421 tmp = new ClassDesc[subclassNames.length]; >>>> 4422 int i = 0; >>>> 4423 for (String subclassName : subclassNames) { >>>> 4424 try { >>>> 4425 tmp[i++] = ClassDesc.of(subclassName.replace('/', '.')); >>>> 4426 } catch (IllegalArgumentException iae) { >>>> 4427 throw new InternalError("Invalid type in permitted subclasses >>>> information: " + subclassName, iae); >>>> 4428 } >>>> 4429 } >>>> Nit: rename tmp to some other name e.g. descs >>>> >>>> I read the JVMS but it isn't clear to me that the VM will validate >>>> the names in `PermittedSubclasses`attribute are valid class >>>> descriptors.?? I see ConstantPool::is_klass_or_reference check but >>>> can't find where it validates the name is a valid class descriptor >>>> - can you point me there??? (otherwise, maybe define it to be >>>> unspecified?) >>>> >>>> >>>> W.r.t. the APIs. I don't want to delay this review.? I see that you >>>> renamed the method to new API style as I brought up.? OTOH,? I >>>> expect more discussion is needed.? Below is a recent comment from >>>> John on this topic [1] >>>> >>>>> One comment, really for the future, regarding the shape of the >>>>> Java API here: It uses Optional and omits the "get" prefix on >>>>> accessors. This is the New Style, as opposed to the Classic Style >>>>> using null (for "empty" results) and a "get" prefix >>>>> ("getComponentType") to get a related type. We may choose to to >>>>> use the New Style for new reflection API points, and if so let's >>>>> not choose N different New Styles, but one New Style. Personally I >>>>> like removing "get"; I accept Optional instead of null; and I also >>>>> suggest that arrays (if any) be replaced by (immutable) Lists in >>>>> the New Style >>>> >>>> There are a few existing Class APIs that use the new API style: >>>> Class arrayClass(); >>>> Optional describeConstable(); >>>> String descriptorString(); >>>> >>>> This will set up a precedence of the new API style in this class.? >>>> Should this new permittedSubclasses method return a List instead of >>>> an array?? It's okay with me if you prefer to revert back to the >>>> old API style and follow this up after integration. >>>> >>>> 4442 * Returns true if this {@linkplain Class} is sealed. >>>> 4443 * >>>> 4444 * @return returns true if this class is sealed >>>> >>>> NIt: {@code true} instead of true -? consistent with the style this >>>> class uses (in most methods) >>>> >>>> test/jdk/java/lang/reflect/sealed_classes/SealedClassesReflectionTest.java >>>> >>>> >>>> Nit: s/sealed_classes/sealedClasses/ >>>> - the test directory/file naming convention use camel case or java >>>> variable name convention. >>>> >>>> Thanks >>>> [1] https://github.com/openjdk/valhalla/pull/53#issuecomment-633116043 From andy.herrick at oracle.com Mon Jun 1 14:32:55 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Mon, 1 Jun 2020 10:32:55 -0400 Subject: RFR: 8232841: [TESTBUG] [macos] SigningPackageTest fails when untrusted certificates exist on machine In-Reply-To: <1d1ab383-61ff-c573-8846-80c801f977b2@oracle.com> References: <1d1ab383-61ff-c573-8846-80c801f977b2@oracle.com> Message-ID: <44438c9a-4d4d-5457-b155-e6e1ea8865ad@oracle.com> looks good. /Andy On 5/29/2020 7:39 PM, alexander.matveev at oracle.com wrote: > Please review the jpackage fix for bug [1] at [2]. > > - I was not able to reproduce this issue. Not sure why this command > returns non zero code in some cases. It still returns output which we > need, so fixed by ignoring exit code, since we will scan output for > presence of certificate anyway. > > [1] https://bugs.openjdk.java.net/browse/JDK-8232841 > > [2] http://cr.openjdk.java.net/~almatvee/8232841/webrev.00/ > > Thanks, > Alexander From igor.ignatyev at oracle.com Mon Jun 1 14:54:18 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 1 Jun 2020 07:54:18 -0700 Subject: RFR(S/T) : 8245874 : requires.extraPropDefns.vmOpts doesn't need -Xbootclasspath/a:bootClasses In-Reply-To: References: Message-ID: <88ED55CF-092F-4FAC-ACFD-17D3CA869C75@oracle.com> ping? -- Igor > On May 27, 2020, at 10:25 AM, Igor Ignatyev wrote: > > http://cr.openjdk.java.net/~iignatyev//8245874/webrev.00 >> 8 lines changed: 2 ins; 0 del; 6 mod; > > Hi all, > > could you please review this small and trivial cleanup in TEST.ROOT files of test/jdk/ and test/hotspot/jtreg test suites? > from JBS: >> to make sure that classes listed in `requires.extraPropDefns.bootlibs` are in BCP, jtreg automatically adds corresponding `-Xbootclasspath/a:` to the options list, so `requires.extraPropDefns.vmOpts` doesn't have to have `-Xbootclasspath/a:bootClasses` > the patch > - removes -Xbootclasspath/a:bootClasses from requires.extraPropDefns.vmOpts > - moves jdk.test.lib.Platform and Container classes from requires.extraPropDefns.bootlibs to requires.extraPropDefns.libs as there is no need for them to be in BCP > > JBS: https://bugs.openjdk.java.net/browse/JDK-8245874 > testing: tier1 > webrev: http://cr.openjdk.java.net/~iignatyev//8245874/webrev.00 > > Thanks, > -- Igor From james.laskey at oracle.com Mon Jun 1 15:55:30 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Mon, 1 Jun 2020 12:55:30 -0300 Subject: RFR: 8230743 - StringJoiner does not handle too large strings correctly Message-ID: <08ED85FA-7317-4995-877E-12C1AB8FE316@oracle.com> Change NegativeArraySizeException to OutOfMemoryError. Tests added. Cheers, -- Jim webrev: http://cr.openjdk.java.net/~jlaskey/8230743/webrev-00/index.html jbs: https://bugs.openjdk.java.net/browse/JDK-8230743 From mandy.chung at oracle.com Mon Jun 1 16:30:21 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 1 Jun 2020 09:30:21 -0700 Subject: RFR: 8246251: Adjust HelloClasslist after JDK-8230301 In-Reply-To: <475749d0-6ac0-6554-ae34-220e1fb18858@oracle.com> References: <475749d0-6ac0-6554-ae34-220e1fb18858@oracle.com> Message-ID: <569da785-acb8-eedf-5ecd-3188ac7e0994@oracle.com> Hi Claes, This looks okay. Mandy On 6/1/20 4:55 AM, Claes Redestad wrote: > Hi, > > comparing startup profiles on a few application I found a few cases > where a few more classes are being generated at runtime compared to JDK > 14. > > This is due to the removal of hard-coded defaults in > GenerateJLIClassesPlugin[1], in combination with how earlier > implementations of SCF (used at the time those hard-coded defaults were > captured) were more aggressive at generating higher-arity SpeciesData > classes. > > To compensate, this patch adds a few cases of even higher-arity String > concat expressions, which smooths things out a bit (effect of > JDK-8230301 is still good since it removes a number of hard-coded > shapes we are unlikely to ever produce on the current JDK) > > Bug:??? https://bugs.openjdk.java.net/browse/JDK-8246251 > Webrev: http://cr.openjdk.java.net/~redestad/8246251/open.00/ > > Testing: tier1-2 > > Thanks! > > /Claes > > [1] https://bugs.openjdk.java.net/browse/JDK-8230301 From claes.redestad at oracle.com Mon Jun 1 16:34:17 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 1 Jun 2020 18:34:17 +0200 Subject: RFR: 8246251: Adjust HelloClasslist after JDK-8230301 In-Reply-To: <569da785-acb8-eedf-5ecd-3188ac7e0994@oracle.com> References: <475749d0-6ac0-6554-ae34-220e1fb18858@oracle.com> <569da785-acb8-eedf-5ecd-3188ac7e0994@oracle.com> Message-ID: Thanks, Mandy! /Claes On 2020-06-01 18:30, Mandy Chung wrote: > Hi Claes, > > This looks okay. > > Mandy > > On 6/1/20 4:55 AM, Claes Redestad wrote: >> Hi, >> >> comparing startup profiles on a few application I found a few cases >> where a few more classes are being generated at runtime compared to JDK >> 14. >> >> This is due to the removal of hard-coded defaults in >> GenerateJLIClassesPlugin[1], in combination with how earlier >> implementations of SCF (used at the time those hard-coded defaults were >> captured) were more aggressive at generating higher-arity SpeciesData >> classes. >> >> To compensate, this patch adds a few cases of even higher-arity String >> concat expressions, which smooths things out a bit (effect of >> JDK-8230301 is still good since it removes a number of hard-coded >> shapes we are unlikely to ever produce on the current JDK) >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8246251 >> Webrev: http://cr.openjdk.java.net/~redestad/8246251/open.00/ >> >> Testing: tier1-2 >> >> Thanks! >> >> /Claes >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8230301 > From paul.sandoz at oracle.com Mon Jun 1 18:48:28 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 1 Jun 2020 11:48:28 -0700 Subject: RFR: 8246241: LambdaFormEditor should use a transform lookup key that is not a SoftReference In-Reply-To: References: Message-ID: Looks good. I sent some trivial comments offline. Paul. > On Jun 1, 2020, at 4:07 AM, Claes Redestad wrote: > > Hi, > > the LambdaFormEditor uses a SoftReference-based cache to avoid > generating LambdaForms we have already generated. By introducing a > lookup key that is not a SoftReference we can improve this by allocating > less up front and by inducing less work for the GC later on. > > I also refactored a few cases where I had neglected to add a static > factory method to create the Transform (now TransformKey). For the > transform types affected this helps further reduce allocations. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8246241 > Webrev: http://cr.openjdk.java.net/~redestad/8246241/open.00/ > > Testing: tier1-2, java/lang/invoke locally with STRESS_TEST=true > > Verified a speed-up on the micro added by JDK-8246152)[1], and > on related startup tests[2]. > > Thanks! > > /Claes > > [1] > Before (JDK-8246152): > Benchmark Mode Cnt Score Error Units > StringConcatFactoryBootstraps.makeConcatWithConstants avgt 10 2764.634 ? 56.244 ns/op > StringConcatFactoryBootstraps.makeConcatWithConstants:?gc.alloc.rate.norm avgt 10 4136.187 ? 0.013 B/op > > After: > Benchmark Mode Cnt Score Error Units > StringConcatFactoryBootstraps.makeConcatWithConstants avgt 10 2299.919 ? 58.003 ns/op > StringConcatFactoryBootstraps.makeConcatWithConstants:?gc.alloc.rate.norm avgt 10 3308.145 ? 6.378 B/op > > [2] http://cr.openjdk.java.net/~redestad/scratch/erase_scf_types.00/ObjStringCombos.java > > Before (JDK-8246152): > 3,485,679,714 instructions # 1.09 insn per cycle ( +- 0.90% ) > 692,247,529 branches # 491.822 M/sec ( +- 0.92% ) > 28,939,534 branch-misses # 4.18% of all branches ( +- 0.90% ) > > 0.291642667 seconds time elapsed ( +- 0.93% ) > > After: > 3,411,619,586 instructions # 1.09 insn per cycle ( +- 0.63% ) > 677,767,258 branches # 493.911 M/sec ( +- 0.64% ) > 28,337,022 branch-misses # 4.18% of all branches ( +- 0.63% ) > > 0.288959534 seconds time elapsed ( +- 0.86% ) From paul.sandoz at oracle.com Mon Jun 1 19:09:54 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 1 Jun 2020 12:09:54 -0700 Subject: RFR: JDK-8230744 Several classes throw OutOfMemoryError without message In-Reply-To: <045D0E84-60E4-491B-9A31-7B05043FFE3B@oracle.com> References: <045D0E84-60E4-491B-9A31-7B05043FFE3B@oracle.com> Message-ID: +1 I quibble about the ?ignored? variable name for the caught ArithmeticException. Paul. > On Jun 1, 2020, at 5:48 AM, Jim Laskey wrote: > > Cleans up every case of OutOfMemoryError without a message. Some logic changes in String::repeat and ByteArrayChannel:: hugeCapacity. > > Thank you. > > Cheers, > > -- Jim > > webrev: http://cr.openjdk.java.net/~jlaskey/8230744/webrev-00/index.html > jbs: https://bugs.openjdk.java.net/browse/JDK-8230744 > > From james.laskey at oracle.com Mon Jun 1 19:18:53 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Mon, 1 Jun 2020 16:18:53 -0300 Subject: RFR: JDK-8230744 Several classes throw OutOfMemoryError without message In-Reply-To: References: <045D0E84-60E4-491B-9A31-7B05043FFE3B@oracle.com> Message-ID: <10AB1E98-16F1-43EB-899F-B20BAB90086E@oracle.com> To be honest I should (will) file another bug against the use of math exact here. I don't think the tests actually catch what they think they test. Looks good in theory but more often than not, the pos array above that code will blow up before you get there. > On Jun 1, 2020, at 4:09 PM, Paul Sandoz wrote: > > +1 > > I quibble about the ?ignored? variable name for the caught ArithmeticException. > > Paul. > >> On Jun 1, 2020, at 5:48 AM, Jim Laskey wrote: >> >> Cleans up every case of OutOfMemoryError without a message. Some logic changes in String::repeat and ByteArrayChannel:: hugeCapacity. >> >> Thank you. >> >> Cheers, >> >> -- Jim >> >> webrev: http://cr.openjdk.java.net/~jlaskey/8230744/webrev-00/index.html >> jbs: https://bugs.openjdk.java.net/browse/JDK-8230744 >> >> > From paul.sandoz at oracle.com Mon Jun 1 19:28:23 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 1 Jun 2020 12:28:23 -0700 Subject: RFR: 8230743 - StringJoiner does not handle too large strings correctly In-Reply-To: <08ED85FA-7317-4995-877E-12C1AB8FE316@oracle.com> References: <08ED85FA-7317-4995-877E-12C1AB8FE316@oracle.com> Message-ID: <1C81281C-45A7-48E7-8D04-F193DEDC42D1@oracle.com> Can we consolidate the use of Integer.MAX_VALUE - 8 in tests by referring to the constant jdk.internal.util.ArraysSupport.MAX_ARRAY_LENGTH? StringJoiner.java ? 164 @Override 165 public String toString() { 166 final String[] elts = this.elts; 167 if (elts == null && emptyValue != null) { 168 return emptyValue; 169 } 170 final int size = this.size; 171 final int addLen = prefix.length() + suffix.length(); 172 if (len < 0 || len + addLen < 0) { It bothers me that len might be < 0, suggesting a larger issue. Perhaps look at the add method where len is modified? 173 throw new 174 OutOfMemoryError("Resulting string is exceeds maximum size?); "Resulting string is exceeds? ? -> "Resulting string exceeds? " Paul. > On Jun 1, 2020, at 8:55 AM, Jim Laskey wrote: > > Change NegativeArraySizeException to OutOfMemoryError. Tests added. > > Cheers, > > -- Jim > > webrev: http://cr.openjdk.java.net/~jlaskey/8230743/webrev-00/index.html > jbs: https://bugs.openjdk.java.net/browse/JDK-8230743 > From naoto.sato at oracle.com Mon Jun 1 19:31:25 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Mon, 1 Jun 2020 12:31:25 -0700 Subject: [15] RFR: 8246261: TCKLocalTime.java failed due to "AssertionError: expected [18:14:22] but found [18:14:23]" Message-ID: <7cabdd8f-7d23-5685-ee0b-423737d829f9@oracle.com> Hello, Please review the fix to the following issue: https://bugs.openjdk.java.net/browse/JDK-8246261 The proposed changeset is located at: https://cr.openjdk.java.net/~naoto/8246261/webrev.00/ The test case compares two LocalTime objects, created with LocalTime.now(Clock/ZoneId). So inherently those two objects could have different times. The test tries to compare them 100 times for the exact match, and if not, then falls back to compare those times by truncating nanoseconds. The failure could occur when those two LocalTimes are around the whole second, e.g., expected == 18:14:22.999999 and test == 18:14:23.000001. To fix this, check the difference of those objects and ensure it is less than a second. Similar test cases exist in TCKLocalDateTime.java and TCKZonedDateTime.java so they should also be fixed. It is ok to leave the similar test case in TCKLocalDate.java, as multiple tries do exact match. Naoto From martinrb at google.com Mon Jun 1 19:39:41 2020 From: martinrb at google.com (Martin Buchholz) Date: Mon, 1 Jun 2020 12:39:41 -0700 Subject: RFR: 8230743 - StringJoiner does not handle too large strings correctly In-Reply-To: <08ED85FA-7317-4995-877E-12C1AB8FE316@oracle.com> References: <08ED85FA-7317-4995-877E-12C1AB8FE316@oracle.com> Message-ID: Missing "throw"; We should have an assertThrows test method (many people have written such, including myself). + try { + new StringJoiner(maxString, maxString, "").toString(); + new AssertionError("Should have thrown OutOfMemoryError"); On Mon, Jun 1, 2020 at 8:58 AM Jim Laskey wrote: > > Change NegativeArraySizeException to OutOfMemoryError. Tests added. > > Cheers, > > -- Jim > > webrev: http://cr.openjdk.java.net/~jlaskey/8230743/webrev-00/index.html > jbs: https://bugs.openjdk.java.net/browse/JDK-8230743 > From james.laskey at oracle.com Mon Jun 1 19:44:45 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Mon, 1 Jun 2020 16:44:45 -0300 Subject: RFR: 8230743 - StringJoiner does not handle too large strings correctly In-Reply-To: <1C81281C-45A7-48E7-8D04-F193DEDC42D1@oracle.com> References: <08ED85FA-7317-4995-877E-12C1AB8FE316@oracle.com> <1C81281C-45A7-48E7-8D04-F193DEDC42D1@oracle.com> Message-ID: > On Jun 1, 2020, at 4:28 PM, Paul Sandoz wrote: > > Can we consolidate the use of Integer.MAX_VALUE - 8 in tests by referring to the constant jdk.internal.util.ArraysSupport.MAX_ARRAY_LENGTH? I suppose it's possible to add module for java.base/jdk.internal.util.ArraysSupport to the tests. The mission here was to provide useful error messages. If we're going to clean up growing arrays throughout the JDK, I feel that is a separate mission, probably not something that we should be doing just before RD1. We should probably have gone one step further with jdk.internal.util.ArraysSupport.newLength and just defined jdk.internal.util.ArraysSupport.growArray(array, newSize, () -> { // exception code }); > > > StringJoiner.java > ? > > 164 @Override > 165 public String toString() { > 166 final String[] elts = this.elts; > 167 if (elts == null && emptyValue != null) { > 168 return emptyValue; > 169 } > 170 final int size = this.size; > 171 final int addLen = prefix.length() + suffix.length(); > 172 if (len < 0 || len + addLen < 0) { > > It bothers me that len might be < 0, suggesting a larger issue. Perhaps look at the add method where len is modified? I thought about it but that would change the behaviour of StringJoiner. An error would be thrown on the add. Existing code that added but then tested length before trying a toString() would fail too soon. > > > 173 throw new > 174 OutOfMemoryError("Resulting string is exceeds maximum size?); > > "Resulting string is exceeds? ? -> "Resulting string exceeds? " Oops, thanks. > > Paul. > > >> On Jun 1, 2020, at 8:55 AM, Jim Laskey wrote: >> >> Change NegativeArraySizeException to OutOfMemoryError. Tests added. >> >> Cheers, >> >> -- Jim >> >> webrev: http://cr.openjdk.java.net/~jlaskey/8230743/webrev-00/index.html >> jbs: https://bugs.openjdk.java.net/browse/JDK-8230743 >> > From paul.sandoz at oracle.com Mon Jun 1 19:44:42 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 1 Jun 2020 12:44:42 -0700 Subject: RFR: 8230743 - StringJoiner does not handle too large strings correctly In-Reply-To: References: <08ED85FA-7317-4995-877E-12C1AB8FE316@oracle.com> Message-ID: <2C7FF600-11DA-4E4B-9155-B5DAECE6779F@oracle.com> Oh, so easy to miss. Better yet use TestNG's Assert.fail, or for simple cases declare the expected exception as an argument of the test annotation. Paul. > On Jun 1, 2020, at 12:39 PM, Martin Buchholz wrote: > > Missing "throw"; We should have an assertThrows test method (many > people have written such, including myself). > > + try { > + new StringJoiner(maxString, maxString, "").toString(); > + new AssertionError("Should have thrown OutOfMemoryError"); > > On Mon, Jun 1, 2020 at 8:58 AM Jim Laskey wrote: >> >> Change NegativeArraySizeException to OutOfMemoryError. Tests added. >> >> Cheers, >> >> -- Jim >> >> webrev: http://cr.openjdk.java.net/~jlaskey/8230743/webrev-00/index.html >> jbs: https://bugs.openjdk.java.net/browse/JDK-8230743 >> From james.laskey at oracle.com Mon Jun 1 19:45:31 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Mon, 1 Jun 2020 16:45:31 -0300 Subject: RFR: 8230743 - StringJoiner does not handle too large strings correctly In-Reply-To: <2C7FF600-11DA-4E4B-9155-B5DAECE6779F@oracle.com> References: <08ED85FA-7317-4995-877E-12C1AB8FE316@oracle.com> <2C7FF600-11DA-4E4B-9155-B5DAECE6779F@oracle.com> Message-ID: <4C4FB26A-541D-41DE-A458-302447001FA3@oracle.com> fail is not available. > On Jun 1, 2020, at 4:44 PM, Paul Sandoz wrote: > > Oh, so easy to miss. Better yet use TestNG's Assert.fail, or for simple cases declare the expected exception as an argument of the test annotation. > > Paul. > >> On Jun 1, 2020, at 12:39 PM, Martin Buchholz wrote: >> >> Missing "throw"; We should have an assertThrows test method (many >> people have written such, including myself). >> >> + try { >> + new StringJoiner(maxString, maxString, "").toString(); >> + new AssertionError("Should have thrown OutOfMemoryError"); >> >> On Mon, Jun 1, 2020 at 8:58 AM Jim Laskey wrote: >>> >>> Change NegativeArraySizeException to OutOfMemoryError. Tests added. >>> >>> Cheers, >>> >>> -- Jim >>> >>> webrev: http://cr.openjdk.java.net/~jlaskey/8230743/webrev-00/index.html >>> jbs: https://bugs.openjdk.java.net/browse/JDK-8230743 >>> > From james.laskey at oracle.com Mon Jun 1 19:49:09 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Mon, 1 Jun 2020 16:49:09 -0300 Subject: RFR: 8230743 - StringJoiner does not handle too large strings correctly In-Reply-To: <4C4FB26A-541D-41DE-A458-302447001FA3@oracle.com> References: <08ED85FA-7317-4995-877E-12C1AB8FE316@oracle.com> <2C7FF600-11DA-4E4B-9155-B5DAECE6779F@oracle.com> <4C4FB26A-541D-41DE-A458-302447001FA3@oracle.com> Message-ID: oops - just realized why. > On Jun 1, 2020, at 4:45 PM, Jim Laskey wrote: > > fail is not available. > >> On Jun 1, 2020, at 4:44 PM, Paul Sandoz wrote: >> >> Oh, so easy to miss. Better yet use TestNG's Assert.fail, or for simple cases declare the expected exception as an argument of the test annotation. >> >> Paul. >> >>> On Jun 1, 2020, at 12:39 PM, Martin Buchholz wrote: >>> >>> Missing "throw"; We should have an assertThrows test method (many >>> people have written such, including myself). >>> >>> + try { >>> + new StringJoiner(maxString, maxString, "").toString(); >>> + new AssertionError("Should have thrown OutOfMemoryError"); >>> >>> On Mon, Jun 1, 2020 at 8:58 AM Jim Laskey wrote: >>>> >>>> Change NegativeArraySizeException to OutOfMemoryError. Tests added. >>>> >>>> Cheers, >>>> >>>> -- Jim >>>> >>>> webrev: http://cr.openjdk.java.net/~jlaskey/8230743/webrev-00/index.html >>>> jbs: https://bugs.openjdk.java.net/browse/JDK-8230743 >>>> >> > From james.laskey at oracle.com Mon Jun 1 19:51:56 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Mon, 1 Jun 2020 16:51:56 -0300 Subject: RFR: 8230743 - StringJoiner does not handle too large strings correctly In-Reply-To: References: <08ED85FA-7317-4995-877E-12C1AB8FE316@oracle.com> Message-ID: <4168F79B-1244-453C-A7F7-F1C0C35DBCE7@oracle.com> Thanks Martin. > On Jun 1, 2020, at 4:39 PM, Martin Buchholz wrote: > > Missing "throw"; We should have an assertThrows test method (many > people have written such, including myself). > > + try { > + new StringJoiner(maxString, maxString, "").toString(); > + new AssertionError("Should have thrown OutOfMemoryError"); > > On Mon, Jun 1, 2020 at 8:58 AM Jim Laskey wrote: >> >> Change NegativeArraySizeException to OutOfMemoryError. Tests added. >> >> Cheers, >> >> -- Jim >> >> webrev: http://cr.openjdk.java.net/~jlaskey/8230743/webrev-00/index.html >> jbs: https://bugs.openjdk.java.net/browse/JDK-8230743 >> From claes.redestad at oracle.com Mon Jun 1 19:57:26 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 1 Jun 2020 21:57:26 +0200 Subject: RFR: 8246241: LambdaFormEditor should use a transform lookup key that is not a SoftReference In-Reply-To: References: Message-ID: <8324ca59-373d-7048-93f7-6c461d9265fc@oracle.com> On 2020-06-01 20:48, Paul Sandoz wrote: > Looks good. I sent some trivial comments offline. Thanks! Adjusted the webrev in-place based on your comments (A few methods were public that should be private, etc..) /Claes > > Paul. > >> On Jun 1, 2020, at 4:07 AM, Claes Redestad wrote: >> >> Hi, >> >> the LambdaFormEditor uses a SoftReference-based cache to avoid >> generating LambdaForms we have already generated. By introducing a >> lookup key that is not a SoftReference we can improve this by allocating >> less up front and by inducing less work for the GC later on. >> >> I also refactored a few cases where I had neglected to add a static >> factory method to create the Transform (now TransformKey). For the >> transform types affected this helps further reduce allocations. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8246241 >> Webrev: http://cr.openjdk.java.net/~redestad/8246241/open.00/ >> >> Testing: tier1-2, java/lang/invoke locally with STRESS_TEST=true >> >> Verified a speed-up on the micro added by JDK-8246152)[1], and >> on related startup tests[2]. >> >> Thanks! >> >> /Claes >> >> [1] >> Before (JDK-8246152): >> Benchmark Mode Cnt Score Error Units >> StringConcatFactoryBootstraps.makeConcatWithConstants avgt 10 2764.634 ? 56.244 ns/op >> StringConcatFactoryBootstraps.makeConcatWithConstants:?gc.alloc.rate.norm avgt 10 4136.187 ? 0.013 B/op >> >> After: >> Benchmark Mode Cnt Score Error Units >> StringConcatFactoryBootstraps.makeConcatWithConstants avgt 10 2299.919 ? 58.003 ns/op >> StringConcatFactoryBootstraps.makeConcatWithConstants:?gc.alloc.rate.norm avgt 10 3308.145 ? 6.378 B/op >> >> [2] http://cr.openjdk.java.net/~redestad/scratch/erase_scf_types.00/ObjStringCombos.java >> >> Before (JDK-8246152): >> 3,485,679,714 instructions # 1.09 insn per cycle ( +- 0.90% ) >> 692,247,529 branches # 491.822 M/sec ( +- 0.92% ) >> 28,939,534 branch-misses # 4.18% of all branches ( +- 0.90% ) >> >> 0.291642667 seconds time elapsed ( +- 0.93% ) >> >> After: >> 3,411,619,586 instructions # 1.09 insn per cycle ( +- 0.63% ) >> 677,767,258 branches # 493.911 M/sec ( +- 0.64% ) >> 28,337,022 branch-misses # 4.18% of all branches ( +- 0.63% ) >> >> 0.288959534 seconds time elapsed ( +- 0.86% ) > From Roger.Riggs at oracle.com Mon Jun 1 20:29:07 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Mon, 1 Jun 2020 16:29:07 -0400 Subject: RFR: 8230743 - StringJoiner does not handle too large strings correctly In-Reply-To: <4168F79B-1244-453C-A7F7-F1C0C35DBCE7@oracle.com> References: <08ED85FA-7317-4995-877E-12C1AB8FE316@oracle.com> <4168F79B-1244-453C-A7F7-F1C0C35DBCE7@oracle.com> Message-ID: <07fcad72-13ce-9d87-7327-44f5d796f7b7@oracle.com> HI Jim, StringJoiner.java: 173 and 241:? put the OutOfMemoryException name on the same line as the "new" (all one line). Otherwise looks ok. Thanks, Roger On 6/1/20 3:51 PM, Jim Laskey wrote: > Thanks Martin. > >> On Jun 1, 2020, at 4:39 PM, Martin Buchholz wrote: >> >> Missing "throw"; We should have an assertThrows test method (many >> people have written such, including myself). >> >> + try { >> + new StringJoiner(maxString, maxString, "").toString(); >> + new AssertionError("Should have thrown OutOfMemoryError"); >> >> On Mon, Jun 1, 2020 at 8:58 AM Jim Laskey wrote: >>> Change NegativeArraySizeException to OutOfMemoryError. Tests added. >>> >>> Cheers, >>> >>> -- Jim >>> >>> webrev: http://cr.openjdk.java.net/~jlaskey/8230743/webrev-00/index.html >>> jbs: https://bugs.openjdk.java.net/browse/JDK-8230743 >>> From mandy.chung at oracle.com Mon Jun 1 20:31:05 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 1 Jun 2020 13:31:05 -0700 Subject: RFR: 8246241: LambdaFormEditor should use a transform lookup key that is not a SoftReference In-Reply-To: <8324ca59-373d-7048-93f7-6c461d9265fc@oracle.com> References: <8324ca59-373d-7048-93f7-6c461d9265fc@oracle.com> Message-ID: <47922802-b686-fc4b-c11a-3d7d1faa7a6c@oracle.com> On 6/1/20 12:57 PM, Claes Redestad wrote: > > Adjusted the webrev in-place based on your comments (A few methods were > public that should be private, etc..) This looks okay to me. Mandy From paul.sandoz at oracle.com Mon Jun 1 20:32:30 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 1 Jun 2020 13:32:30 -0700 Subject: RFR: 8230743 - StringJoiner does not handle too large strings correctly In-Reply-To: References: <08ED85FA-7317-4995-877E-12C1AB8FE316@oracle.com> <1C81281C-45A7-48E7-8D04-F193DEDC42D1@oracle.com> Message-ID: <20C9178B-D250-460E-9FBF-5524FFFD8344@oracle.com> > On Jun 1, 2020, at 12:44 PM, Jim Laskey wrote: > > > >> On Jun 1, 2020, at 4:28 PM, Paul Sandoz wrote: >> >> Can we consolidate the use of Integer.MAX_VALUE - 8 in tests by referring to the constant jdk.internal.util.ArraysSupport.MAX_ARRAY_LENGTH? > > I suppose it's possible to add module for java.base/jdk.internal.util.ArraysSupport to the tests. > > The mission here was to provide useful error messages. If we're going to clean up growing arrays throughout the JDK, I feel that is a separate mission, probably not something that we should be doing just before RD1. > Ok. > We should probably have gone one step further with jdk.internal.util.ArraysSupport.newLength and just defined jdk.internal.util.ArraysSupport.growArray(array, newSize, () -> { // exception code }); > >> >> >> StringJoiner.java >> ? >> >> 164 @Override >> 165 public String toString() { >> 166 final String[] elts = this.elts; >> 167 if (elts == null && emptyValue != null) { >> 168 return emptyValue; >> 169 } >> 170 final int size = this.size; >> 171 final int addLen = prefix.length() + suffix.length(); >> 172 if (len < 0 || len + addLen < 0) { >> >> It bothers me that len might be < 0, suggesting a larger issue. Perhaps look at the add method where len is modified? > > I thought about it but that would change the behaviour of StringJoiner. An error would be thrown on the add. Existing code that added but then tested length before trying a toString() would fail too soon. Or fail at the right point? Lazy String construction by the joiner can be considered an implementation detail. A negative len could result in some odd behavior: - an incorrect value returned from length(), such as a negative length value. - a number of adds could be performed, which one would have resulted in an OOME if the implementation was not lazy? - (len + other.len < 0) could be positive and result in array construction exception for a negative length on other.compactElts. Paul. > >> >> >> 173 throw new >> 174 OutOfMemoryError("Resulting string is exceeds maximum size?); >> >> "Resulting string is exceeds? ? -> "Resulting string exceeds? " > > Oops, thanks. > > >> >> Paul. >> >> >>> On Jun 1, 2020, at 8:55 AM, Jim Laskey wrote: >>> >>> Change NegativeArraySizeException to OutOfMemoryError. Tests added. >>> >>> Cheers, >>> >>> -- Jim >>> >>> webrev: http://cr.openjdk.java.net/~jlaskey/8230743/webrev-00/index.html >>> jbs: https://bugs.openjdk.java.net/browse/JDK-8230743 >>> >> > From martinrb at google.com Mon Jun 1 20:36:06 2020 From: martinrb at google.com (Martin Buchholz) Date: Mon, 1 Jun 2020 13:36:06 -0700 Subject: RFR: JDK-8230744 Several classes throw OutOfMemoryError without message In-Reply-To: <10AB1E98-16F1-43EB-899F-B20BAB90086E@oracle.com> References: <045D0E84-60E4-491B-9A31-7B05043FFE3B@oracle.com> <10AB1E98-16F1-43EB-899F-B20BAB90086E@oracle.com> Message-ID: I continue to think JDK library code should avoid exposing VM implementation limits to users, i.e. exception detail messages should not include MAX_ARRAY_SIZE I think our grow code has been drifting to being inconsistent and incoherent over the years. In the code below, the detail message looks wrong; this exception is thrown when minCapacity wraps around, not when between SAFE_BOUND and UNSAFE_BOUND. int SAFE_BOUND = MAX_ARRAY_SIZE >> coder; int UNSAFE_BOUND = Integer.MAX_VALUE >> coder; if (UNSAFE_BOUND - minCapacity < 0) { // overflow - throw new OutOfMemoryError(); + throw new + OutOfMemoryError("StringBuilder buffer exceeds maximum size: " + + MAX_ARRAY_SIZE); On Mon, Jun 1, 2020 at 12:19 PM Jim Laskey wrote: > > To be honest I should (will) file another bug against the use of math exact here. I don't think the tests actually catch what they think they test. Looks good in theory but more often than not, the pos array above that code will blow up before you get there. > > > > > > On Jun 1, 2020, at 4:09 PM, Paul Sandoz wrote: > > > > +1 > > > > I quibble about the ?ignored? variable name for the caught ArithmeticException. > > > > Paul. > > > >> On Jun 1, 2020, at 5:48 AM, Jim Laskey wrote: > >> > >> Cleans up every case of OutOfMemoryError without a message. Some logic changes in String::repeat and ByteArrayChannel:: hugeCapacity. > >> > >> Thank you. > >> > >> Cheers, > >> > >> -- Jim > >> > >> webrev: http://cr.openjdk.java.net/~jlaskey/8230744/webrev-00/index.html > >> jbs: https://bugs.openjdk.java.net/browse/JDK-8230744 > >> > >> > > > From brent.christian at oracle.com Mon Jun 1 20:36:18 2020 From: brent.christian at oracle.com (Brent Christian) Date: Mon, 1 Jun 2020 13:36:18 -0700 Subject: RFR: JDK-8230744 Several classes throw OutOfMemoryError without message In-Reply-To: <045D0E84-60E4-491B-9A31-7B05043FFE3B@oracle.com> References: <045D0E84-60E4-491B-9A31-7B05043FFE3B@oracle.com> Message-ID: <78cf935b-d7f3-342a-6ed1-a4a86becb727@oracle.com> One comment: src/java.base/share/classes/java/util/regex/Pattern.java --- 1679,1690 ---- i += 2; int newTempLen; try { newTempLen = Math.addExact(j + 2, Math.multiplyExact(3, pLen - i)); } catch (ArithmeticException ae) { ! throw new ! OutOfMemoryError("Unable to allocate buffer for \\Q...\\E sequence"); } "Unable to allocate buffer" seems vague in the context of an OOME. If the problem is trying to create a too-large array, maybe something like, "Buffer for \\Q...\\E sequence would exceed maximum array size". -Brent On 6/1/20 5:48 AM, Jim Laskey wrote: > Cleans up every case of OutOfMemoryError without a message. Some logic changes in String::repeat and ByteArrayChannel:: hugeCapacity. > > Thank you. > > Cheers, > > -- Jim > > webrev: http://cr.openjdk.java.net/~jlaskey/8230744/webrev-00/index.html > jbs: https://bugs.openjdk.java.net/browse/JDK-8230744 > > From mandy.chung at oracle.com Mon Jun 1 20:37:41 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 1 Jun 2020 13:37:41 -0700 Subject: RFR: JDK-8245432: Lookup::defineHiddenClass should throw UnsupportedClassVersionError if the given bytes are of an unsupported major or minor version In-Reply-To: <7c910287-fb2d-766e-8dd4-bc3e66edb9a9@oracle.com> References: <176af0b0-cce9-9286-0b06-9455c0f8cea4@oracle.com> <50b420bf-a53e-06d1-43f3-d5bdac804e7d@oracle.com> <700459437.2144177.1590601182749.JavaMail.zimbra@u-pem.fr> <543b4fc8-dcbb-b8c8-4111-f200885921cd@oracle.com> <46328704-3f68-9643-71b7-d7cdc7088af3@oracle.com> <2f2218dd-de42-bfd4-974a-76ef3d840882@oracle.com> <8f53afb4-d11d-ae62-756f-8093726e098d@oracle.com> <7f82b76b-d9ed-2ccd-5fd1-8a5e68cb0a52@oracle.com> <2eda234f-045d-1145-40ae-cf2b0b7cd5c5@oracle.com> <7c910287-fb2d-766e-8dd4-bc3e66edb9a9@oracle.com> Message-ID: <675f31ee-a0a0-523a-5585-9bf44d6c5405@oracle.com> On 5/29/20 1:03 PM, Mandy Chung wrote: > CSR: https://bugs.openjdk.java.net/browse/JDK-8246108 As CSR is approved,? I went ahead and pushed this patch: http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8245432/webrev.03/ This version has fixed the range check issue Johannes caught and also added a test case for it. Thanks for the review. Mandy From andy.herrick at oracle.com Mon Jun 1 20:40:00 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Mon, 1 Jun 2020 16:40:00 -0400 Subject: RFR: JDK-8246010: AdditionalLaunchersTest is not enabled, and fails. In-Reply-To: <880c0edb-8d95-aba0-8a74-2394605e6134@oracle.com> References: <1d333406-e903-e5b0-c247-609293cc8322@oracle.com> <2c32ab4d-e434-2e0e-b615-f4338dba2d56@oracle.com> <0cc1a351-c23c-7d7c-be81-77cbdf317d66@oracle.com> <880c0edb-8d95-aba0-8a74-2394605e6134@oracle.com> Message-ID: <2acb3416-6eef-34c4-8e03-87888275a791@oracle.com> please review revised fix at [4] to issue [2]. After some discussion we think existing Error messages are sufficient and will just add the missing resource and modify test name after enabling. [4] - http://cr.openjdk.java.net/~herrick/8246010/webrev.03/ /Andy On 5/31/2020 12:13 PM, Andy Herrick wrote: > Please revice revised webrev at [3] > > I have revised to only throw FileServiceException right before > operations that will fail. > > [3] webrev: http://cr.openjdk.java.net/~herrick/8246010/webrev.02 > > /Andy > > On 5/29/2020 7:38 PM, Andy Herrick wrote: >> >> On 5/29/20 7:16 PM, Alexey Semenyuk wrote: >>> Andy, >>> >>> The check is added to WindowsAppImageBuilder.java which means it >>> will be done for app image bundling too. However the problem is with >>> msi bundling only as far as I understood from your explanation. >>> Did you try fix suggested at [1]? >> >> Yes - should add more comments in the bug report. >> >> The problem exists for app-image bundle as well, from within >> ExecutableRebrander. >> >> /Z >> >>> >>> [1] >>> https://stackoverflow.com/questions/15431210/wix-relative-path-ends-up-being-too-long >>> >>> - Alexey >>> >>> On 5/29/2020 7:12 PM, Andy Herrick wrote: >>>> >>>> On 5/29/20 7:00 PM, alexander.matveev at oracle.com wrote: >>>>> Hi Andy, >>>>> >>>>> Why we need error.lock-resource? >>>> because resource was used , but never defined. >>>>> >>>>> Also, I do not like 194 limit obtained by testing, since once new >>>>> files are introduced in runtime this value might not work anymore. >>>>> Can we check on actual file paths to make sure they have valid >>>>> length? >>>> >>>> I don't like it either, but otherwise we need to test length every >>>> time we create a path, and the check needs to be platform dependent. >>>> >>>> /Andy >>>> >>>>> >>>>> Thanks, >>>>> Alexander >>>>> >>>>> On 5/29/20 7:36 AM, Andy Herrick wrote: >>>>>> Please review jpackage fix at [1] to issue [2] >>>>>> >>>>>> /Andy >>>>>> >>>>>> [1] http://cr.openjdk.java.net/~herrick/8246010/webrev.01/ >>>>>> >>>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8246010 >>>>>> >>>>> >>> From martinrb at google.com Mon Jun 1 20:40:40 2020 From: martinrb at google.com (Martin Buchholz) Date: Mon, 1 Jun 2020 13:40:40 -0700 Subject: RFR: JDK-8230744 Several classes throw OutOfMemoryError without message In-Reply-To: <78cf935b-d7f3-342a-6ed1-a4a86becb727@oracle.com> References: <045D0E84-60E4-491B-9A31-7B05043FFE3B@oracle.com> <78cf935b-d7f3-342a-6ed1-a4a86becb727@oracle.com> Message-ID: On Mon, Jun 1, 2020 at 1:37 PM Brent Christian wrote: > "Unable to allocate buffer" seems vague in the context of an OOME. If > the problem is trying to create a too-large array, maybe something like, > "Buffer for \\Q...\\E sequence would exceed maximum array size". A generic message that does not necessarily expose the use of an internal array seems best, e.g. "... exceeds implementation limits" From claes.redestad at oracle.com Mon Jun 1 20:50:52 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 1 Jun 2020 22:50:52 +0200 Subject: RFR: 8246241: LambdaFormEditor should use a transform lookup key that is not a SoftReference In-Reply-To: <47922802-b686-fc4b-c11a-3d7d1faa7a6c@oracle.com> References: <8324ca59-373d-7048-93f7-6c461d9265fc@oracle.com> <47922802-b686-fc4b-c11a-3d7d1faa7a6c@oracle.com> Message-ID: <658677ff-8aaa-6d23-8573-fc4208de7660@oracle.com> On 2020-06-01 22:31, Mandy Chung wrote: > > > On 6/1/20 12:57 PM, Claes Redestad wrote: >> >> Adjusted the webrev in-place based on your comments (A few methods were >> public that should be private, etc..) > > This looks okay to me. Thanks, Mandy! /Claes From alexander.matveev at oracle.com Mon Jun 1 21:18:08 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Mon, 1 Jun 2020 14:18:08 -0700 Subject: RFR: JDK-8246010: AdditionalLaunchersTest is not enabled, and fails. In-Reply-To: <2acb3416-6eef-34c4-8e03-87888275a791@oracle.com> References: <1d333406-e903-e5b0-c247-609293cc8322@oracle.com> <2c32ab4d-e434-2e0e-b615-f4338dba2d56@oracle.com> <0cc1a351-c23c-7d7c-be81-77cbdf317d66@oracle.com> <880c0edb-8d95-aba0-8a74-2394605e6134@oracle.com> <2acb3416-6eef-34c4-8e03-87888275a791@oracle.com> Message-ID: Hi Andy, http://cr.openjdk.java.net/~herrick/8246010/webrev.03/test/jdk/tools/jpackage/share/AddLauncherTest.java.html > * additionallauncherstest*.* installer. The output installer should provide the Should it be addlaunchertest*.*? Otherwise looks fine. Thanks, Alexander On 6/1/20 1:40 PM, Andy Herrick wrote: > please review revised fix at [4] to issue [2]. > > After some discussion we think existing Error messages are sufficient > and will just add the missing resource and modify test name after > enabling. > > [4] - http://cr.openjdk.java.net/~herrick/8246010/webrev.03/ > > /Andy > > On 5/31/2020 12:13 PM, Andy Herrick wrote: >> Please revice revised webrev at [3] >> >> I have revised to only throw FileServiceException right before >> operations that will fail. >> >> [3] webrev: http://cr.openjdk.java.net/~herrick/8246010/webrev.02 >> >> /Andy >> >> On 5/29/2020 7:38 PM, Andy Herrick wrote: >>> >>> On 5/29/20 7:16 PM, Alexey Semenyuk wrote: >>>> Andy, >>>> >>>> The check is added to WindowsAppImageBuilder.java which means it >>>> will be done for app image bundling too. However the problem is >>>> with msi bundling only as far as I understood from your explanation. >>>> Did you try fix suggested at [1]? >>> >>> Yes - should add more comments in the bug report. >>> >>> The problem exists for app-image bundle as well, from within >>> ExecutableRebrander. >>> >>> /Z >>> >>>> >>>> [1] >>>> https://stackoverflow.com/questions/15431210/wix-relative-path-ends-up-being-too-long >>>> >>>> - Alexey >>>> >>>> On 5/29/2020 7:12 PM, Andy Herrick wrote: >>>>> >>>>> On 5/29/20 7:00 PM, alexander.matveev at oracle.com wrote: >>>>>> Hi Andy, >>>>>> >>>>>> Why we need error.lock-resource? >>>>> because resource was used , but never defined. >>>>>> >>>>>> Also, I do not like 194 limit obtained by testing, since once new >>>>>> files are introduced in runtime this value might not work >>>>>> anymore. Can we check on actual file paths to make sure they have >>>>>> valid length? >>>>> >>>>> I don't like it either, but otherwise we need to test length every >>>>> time we create a path, and the check needs to be platform dependent. >>>>> >>>>> /Andy >>>>> >>>>>> >>>>>> Thanks, >>>>>> Alexander >>>>>> >>>>>> On 5/29/20 7:36 AM, Andy Herrick wrote: >>>>>>> Please review jpackage fix at [1] to issue [2] >>>>>>> >>>>>>> /Andy >>>>>>> >>>>>>> [1] http://cr.openjdk.java.net/~herrick/8246010/webrev.01/ >>>>>>> >>>>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8246010 >>>>>>> >>>>>> >>>> From andy.herrick at oracle.com Mon Jun 1 21:43:54 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Mon, 1 Jun 2020 17:43:54 -0400 Subject: RFR: JDK-8246010: AdditionalLaunchersTest is not enabled, and fails. In-Reply-To: References: <1d333406-e903-e5b0-c247-609293cc8322@oracle.com> <2c32ab4d-e434-2e0e-b615-f4338dba2d56@oracle.com> <0cc1a351-c23c-7d7c-be81-77cbdf317d66@oracle.com> <880c0edb-8d95-aba0-8a74-2394605e6134@oracle.com> <2acb3416-6eef-34c4-8e03-87888275a791@oracle.com> Message-ID: <63a27ff5-94f9-41ca-1a2c-d8824dd48572@oracle.com> yes - I will fix that comment in place. /Andy On 6/1/2020 5:18 PM, alexander.matveev at oracle.com wrote: > Hi Andy, > > http://cr.openjdk.java.net/~herrick/8246010/webrev.03/test/jdk/tools/jpackage/share/AddLauncherTest.java.html > >> * additionallauncherstest*.* installer. The output installer should >> provide the > Should it be addlaunchertest*.*? > > Otherwise looks fine. > > Thanks, > Alexander > > > On 6/1/20 1:40 PM, Andy Herrick wrote: >> please review revised fix at [4] to issue [2]. >> >> After some discussion we think existing Error messages are sufficient >> and will just add the missing resource and modify test name after >> enabling. >> >> [4] - http://cr.openjdk.java.net/~herrick/8246010/webrev.03/ >> >> /Andy >> >> On 5/31/2020 12:13 PM, Andy Herrick wrote: >>> Please revice revised webrev at [3] >>> >>> I have revised to only throw FileServiceException right before >>> operations that will fail. >>> >>> [3] webrev: http://cr.openjdk.java.net/~herrick/8246010/webrev.02 >>> >>> /Andy >>> >>> On 5/29/2020 7:38 PM, Andy Herrick wrote: >>>> >>>> On 5/29/20 7:16 PM, Alexey Semenyuk wrote: >>>>> Andy, >>>>> >>>>> The check is added to WindowsAppImageBuilder.java which means it >>>>> will be done for app image bundling too. However the problem is >>>>> with msi bundling only as far as I understood from your explanation. >>>>> Did you try fix suggested at [1]? >>>> >>>> Yes - should add more comments in the bug report. >>>> >>>> The problem exists for app-image bundle as well, from within >>>> ExecutableRebrander. >>>> >>>> /Z >>>> >>>>> >>>>> [1] >>>>> https://stackoverflow.com/questions/15431210/wix-relative-path-ends-up-being-too-long >>>>> >>>>> - Alexey >>>>> >>>>> On 5/29/2020 7:12 PM, Andy Herrick wrote: >>>>>> >>>>>> On 5/29/20 7:00 PM, alexander.matveev at oracle.com wrote: >>>>>>> Hi Andy, >>>>>>> >>>>>>> Why we need error.lock-resource? >>>>>> because resource was used , but never defined. >>>>>>> >>>>>>> Also, I do not like 194 limit obtained by testing, since once >>>>>>> new files are introduced in runtime this value might not work >>>>>>> anymore. Can we check on actual file paths to make sure they >>>>>>> have valid length? >>>>>> >>>>>> I don't like it either, but otherwise we need to test length >>>>>> every time we create a path, and the check needs to be platform >>>>>> dependent. >>>>>> >>>>>> /Andy >>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Alexander >>>>>>> >>>>>>> On 5/29/20 7:36 AM, Andy Herrick wrote: >>>>>>>> Please review jpackage fix at [1] to issue [2] >>>>>>>> >>>>>>>> /Andy >>>>>>>> >>>>>>>> [1] http://cr.openjdk.java.net/~herrick/8246010/webrev.01/ >>>>>>>> >>>>>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8246010 >>>>>>>> >>>>>>> >>>>> > From alexander.matveev at oracle.com Mon Jun 1 21:57:49 2020 From: alexander.matveev at oracle.com (Alexander Matveev) Date: Mon, 1 Jun 2020 14:57:49 -0700 Subject: RFR: JDK-8246010: AdditionalLaunchersTest is not enabled, and fails. In-Reply-To: <63a27ff5-94f9-41ca-1a2c-d8824dd48572@oracle.com> References: <1d333406-e903-e5b0-c247-609293cc8322@oracle.com> <2c32ab4d-e434-2e0e-b615-f4338dba2d56@oracle.com> <0cc1a351-c23c-7d7c-be81-77cbdf317d66@oracle.com> <880c0edb-8d95-aba0-8a74-2394605e6134@oracle.com> <2acb3416-6eef-34c4-8e03-87888275a791@oracle.com> <63a27ff5-94f9-41ca-1a2c-d8824dd48572@oracle.com> Message-ID: Hi Andy, Looks good now. Thanks, Alexander On 6/1/20 2:43 PM, Andy Herrick wrote: > yes - I will fix that comment in place. > > /Andy > > On 6/1/2020 5:18 PM, alexander.matveev at oracle.com wrote: >> Hi Andy, >> >> http://cr.openjdk.java.net/~herrick/8246010/webrev.03/test/jdk/tools/jpackage/share/AddLauncherTest.java.html >> >>> * additionallauncherstest*.* installer. The output installer should >>> provide the >> Should it be addlaunchertest*.*? >> >> Otherwise looks fine. >> >> Thanks, >> Alexander >> >> >> On 6/1/20 1:40 PM, Andy Herrick wrote: >>> please review revised fix at [4] to issue [2]. >>> >>> After some discussion we think existing Error messages are >>> sufficient and will just add the missing resource and modify test >>> name after enabling. >>> >>> [4] - http://cr.openjdk.java.net/~herrick/8246010/webrev.03/ >>> >>> /Andy >>> >>> On 5/31/2020 12:13 PM, Andy Herrick wrote: >>>> Please revice revised webrev at [3] >>>> >>>> I have revised to only throw FileServiceException right before >>>> operations that will fail. >>>> >>>> [3] webrev: http://cr.openjdk.java.net/~herrick/8246010/webrev.02 >>>> >>>> /Andy >>>> >>>> On 5/29/2020 7:38 PM, Andy Herrick wrote: >>>>> >>>>> On 5/29/20 7:16 PM, Alexey Semenyuk wrote: >>>>>> Andy, >>>>>> >>>>>> The check is added to WindowsAppImageBuilder.java which means it >>>>>> will be done for app image bundling too. However the problem is >>>>>> with msi bundling only as far as I understood from your explanation. >>>>>> Did you try fix suggested at [1]? >>>>> >>>>> Yes - should add more comments in the bug report. >>>>> >>>>> The problem exists for app-image bundle as well, from within >>>>> ExecutableRebrander. >>>>> >>>>> /Z >>>>> >>>>>> >>>>>> [1] >>>>>> https://stackoverflow.com/questions/15431210/wix-relative-path-ends-up-being-too-long >>>>>> >>>>>> - Alexey >>>>>> >>>>>> On 5/29/2020 7:12 PM, Andy Herrick wrote: >>>>>>> >>>>>>> On 5/29/20 7:00 PM, alexander.matveev at oracle.com wrote: >>>>>>>> Hi Andy, >>>>>>>> >>>>>>>> Why we need error.lock-resource? >>>>>>> because resource was used , but never defined. >>>>>>>> >>>>>>>> Also, I do not like 194 limit obtained by testing, since once >>>>>>>> new files are introduced in runtime this value might not work >>>>>>>> anymore. Can we check on actual file paths to make sure they >>>>>>>> have valid length? >>>>>>> >>>>>>> I don't like it either, but otherwise we need to test length >>>>>>> every time we create a path, and the check needs to be platform >>>>>>> dependent. >>>>>>> >>>>>>> /Andy >>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Alexander >>>>>>>> >>>>>>>> On 5/29/20 7:36 AM, Andy Herrick wrote: >>>>>>>>> Please review jpackage fix at [1] to issue [2] >>>>>>>>> >>>>>>>>> /Andy >>>>>>>>> >>>>>>>>> [1] http://cr.openjdk.java.net/~herrick/8246010/webrev.01/ >>>>>>>>> >>>>>>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8246010 >>>>>>>>> >>>>>>>> >>>>>> >> From alexey.semenyuk at oracle.com Mon Jun 1 22:36:19 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Mon, 1 Jun 2020 18:36:19 -0400 Subject: RFR: JDK-8246010: AdditionalLaunchersTest is not enabled, and fails. In-Reply-To: <2acb3416-6eef-34c4-8e03-87888275a791@oracle.com> References: <1d333406-e903-e5b0-c247-609293cc8322@oracle.com> <2c32ab4d-e434-2e0e-b615-f4338dba2d56@oracle.com> <0cc1a351-c23c-7d7c-be81-77cbdf317d66@oracle.com> <880c0edb-8d95-aba0-8a74-2394605e6134@oracle.com> <2acb3416-6eef-34c4-8e03-87888275a791@oracle.com> Message-ID: Looks good. - Alexey On 6/1/2020 4:40 PM, Andy Herrick wrote: > please review revised fix at [4] to issue [2]. > > After some discussion we think existing Error messages are sufficient > and will just add the missing resource and modify test name after > enabling. > > [4] - http://cr.openjdk.java.net/~herrick/8246010/webrev.03/ > > /Andy > > On 5/31/2020 12:13 PM, Andy Herrick wrote: >> Please revice revised webrev at [3] >> >> I have revised to only throw FileServiceException right before >> operations that will fail. >> >> [3] webrev: http://cr.openjdk.java.net/~herrick/8246010/webrev.02 >> >> /Andy >> >> On 5/29/2020 7:38 PM, Andy Herrick wrote: >>> >>> On 5/29/20 7:16 PM, Alexey Semenyuk wrote: >>>> Andy, >>>> >>>> The check is added to WindowsAppImageBuilder.java which means it >>>> will be done for app image bundling too. However the problem is >>>> with msi bundling only as far as I understood from your explanation. >>>> Did you try fix suggested at [1]? >>> >>> Yes - should add more comments in the bug report. >>> >>> The problem exists for app-image bundle as well, from within >>> ExecutableRebrander. >>> >>> /Z >>> >>>> >>>> [1] >>>> https://stackoverflow.com/questions/15431210/wix-relative-path-ends-up-being-too-long >>>> >>>> - Alexey >>>> >>>> On 5/29/2020 7:12 PM, Andy Herrick wrote: >>>>> >>>>> On 5/29/20 7:00 PM, alexander.matveev at oracle.com wrote: >>>>>> Hi Andy, >>>>>> >>>>>> Why we need error.lock-resource? >>>>> because resource was used , but never defined. >>>>>> >>>>>> Also, I do not like 194 limit obtained by testing, since once new >>>>>> files are introduced in runtime this value might not work >>>>>> anymore. Can we check on actual file paths to make sure they have >>>>>> valid length? >>>>> >>>>> I don't like it either, but otherwise we need to test length every >>>>> time we create a path, and the check needs to be platform dependent. >>>>> >>>>> /Andy >>>>> >>>>>> >>>>>> Thanks, >>>>>> Alexander >>>>>> >>>>>> On 5/29/20 7:36 AM, Andy Herrick wrote: >>>>>>> Please review jpackage fix at [1] to issue [2] >>>>>>> >>>>>>> /Andy >>>>>>> >>>>>>> [1] http://cr.openjdk.java.net/~herrick/8246010/webrev.01/ >>>>>>> >>>>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8246010 >>>>>>> >>>>>> >>>> From david.holmes at oracle.com Mon Jun 1 22:43:58 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 2 Jun 2020 08:43:58 +1000 Subject: 8245867: Logger/bundleLeak/BundleTest.java fails due to "OutOfMemoryError: Java heap space" In-Reply-To: <8657dfbc-4584-e8a4-ff14-b1056adcecff@oracle.com> References: <8657dfbc-4584-e8a4-ff14-b1056adcecff@oracle.com> Message-ID: <96640076-76fa-b9db-7707-5900f6b1f151@oracle.com> Hi Daniel, Sorry the weekend got in the way :) On 29/05/2020 7:01 pm, Daniel Fuchs wrote: > Hi David, > > Thanks for the feedback! > > On 29/05/2020 00:36, David Holmes wrote: >> This seems to be assuming that the GC will clear all SoftReferences >> when it needs to clear any SoftReference. I don't think that is at all >> guaranteed. In theory your memory eating loop could be satisfied by >> clearing only the SoftReference added in the previous iteration of the >> loop. > > I was wondering about that. > >> I would have expected the fix here to be simply to clear memory ie: >> >> } catch (OutOfMemoryError oome) { >> ?? stop = true; >> ?? memory = null; >> ?? System.gc(); >> } > > Do you think this would be more reliable? > It was my first thought as well - and that was the > first thing I tested, and it seemed to work as well. > > If you think it's a better solution I can revert to that. > > My rationale for using SoftReference is that they > then could be cleared any time by the GC, and > possibly avoid unexpected OOME elsewhere in the test > machinery. Yes understood, but it was relying on all soft references being released at the same time. For these OOME related tests it is hard to come up with a guaranteed way to trigger bugs that stands the test of time. So keep things simple and go with what works now across the set of GCs we are currently support. Cheers, David > best regards, > > -- daniel From lance.andersen at oracle.com Mon Jun 1 23:45:59 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Mon, 1 Jun 2020 19:45:59 -0400 Subject: [15] RFR: 8246261: TCKLocalTime.java failed due to "AssertionError: expected [18:14:22] but found [18:14:23]" In-Reply-To: <7cabdd8f-7d23-5685-ee0b-423737d829f9@oracle.com> References: <7cabdd8f-7d23-5685-ee0b-423737d829f9@oracle.com> Message-ID: <839F9527-52B2-41AD-A6CE-B5C5F33FFED6@oracle.com> Hi Naoto, Looks reasonable to me :-) > On Jun 1, 2020, at 3:31 PM, naoto.sato at oracle.com wrote: > > Hello, > > Please review the fix to the following issue: > > https://bugs.openjdk.java.net/browse/JDK-8246261 > > The proposed changeset is located at: > > https://cr.openjdk.java.net/~naoto/8246261/webrev.00/ > > The test case compares two LocalTime objects, created with LocalTime.now(Clock/ZoneId). So inherently those two objects could have different times. The test tries to compare them 100 times for the exact match, and if not, then falls back to compare those times by truncating nanoseconds. The failure could occur when those two LocalTimes are around the whole second, e.g., expected == 18:14:22.999999 and test == 18:14:23.000001. To fix this, check the difference of those objects and ensure it is less than a second. > > Similar test cases exist in TCKLocalDateTime.java and TCKZonedDateTime.java so they should also be fixed. It is ok to leave the similar test case in TCKLocalDate.java, as multiple tries do exact match. > > Naoto Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From huizhe.wang at oracle.com Tue Jun 2 00:58:28 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Mon, 1 Jun 2020 17:58:28 -0700 Subject: [15] RFR: 8246261: TCKLocalTime.java failed due to "AssertionError: expected [18:14:22] but found [18:14:23]" In-Reply-To: <7cabdd8f-7d23-5685-ee0b-423737d829f9@oracle.com> References: <7cabdd8f-7d23-5685-ee0b-423737d829f9@oracle.com> Message-ID: <5935f47e-330a-1c2e-3b74-3268570e9947@oracle.com> Hi Naoto, The patch looks good to fix the failure. I'm just curious whether the 100-time comparison is necessary because of the existence of this assertion outside the loop that allowed the test to pass if the different was within a certain period of time. None of the tests had commented on the purpose of the test,? it looks like it's testing the assertion that (for the now method) "This will query the system clock to obtain the current time." The 100-loop therefore was a compromise for lack of a better way to prove that. I agree with what you said that "inherently those two objects could have different times". The outside-loop assertion? therefore makes better sense, and the loop was kind of just wasting time to me (I mean you could get lucky to have the two returning the same time down to a nanosecond, but that didn't make the test better than just the out-of-loop assertion. my 2 cents -Joe On 6/1/2020 12:31 PM, naoto.sato at oracle.com wrote: > Hello, > > Please review the fix to the following issue: > > https://bugs.openjdk.java.net/browse/JDK-8246261 > > The proposed changeset is located at: > > https://cr.openjdk.java.net/~naoto/8246261/webrev.00/ > > The test case compares two LocalTime objects, created with > LocalTime.now(Clock/ZoneId). So inherently those two objects could > have different times. The test tries to compare them 100 times for the > exact match, and if not, then falls back to compare those times by > truncating nanoseconds. The failure could occur when those two > LocalTimes are around the whole second, e.g., expected == > 18:14:22.999999 and test == 18:14:23.000001. To fix this, check the > difference of those objects and ensure it is less than a second. > > Similar test cases exist in TCKLocalDateTime.java and > TCKZonedDateTime.java so they should also be fixed. It is ok to leave > the similar test case in TCKLocalDate.java, as multiple tries do exact > match. > > Naoto From naoto.sato at oracle.com Tue Jun 2 01:30:38 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Mon, 1 Jun 2020 18:30:38 -0700 Subject: [15] RFR: 8246261: TCKLocalTime.java failed due to "AssertionError: expected [18:14:22] but found [18:14:23]" In-Reply-To: <5935f47e-330a-1c2e-3b74-3268570e9947@oracle.com> References: <7cabdd8f-7d23-5685-ee0b-423737d829f9@oracle.com> <5935f47e-330a-1c2e-3b74-3268570e9947@oracle.com> Message-ID: Hi Joe, In fact, this bug was possibly revealed by the fix to 8242504, where the system clock precision is now nanoseconds. Before that, it used to be millisecond precision, so the first try for the exact match succeeded for most of the cases. Even with the nano precision fix, most of the cases the test exits with exact match in the loop. But you are right, exact match or not does not matter in this test case, so I think we can just eliminate these exact match try loops. I will remove them and do some sniff testing on it. Naoto On 6/1/20 5:58 PM, Joe Wang wrote: > Hi Naoto, > > The patch looks good to fix the failure. I'm just curious whether the > 100-time comparison is necessary because of the existence of this > assertion outside the loop that allowed the test to pass if the > different was within a certain period of time. None of the tests had > commented on the purpose of the test,? it looks like it's testing the > assertion that (for the now method) "This will query the system clock to > obtain the current time." The 100-loop therefore was a compromise for > lack of a better way to prove that. I agree with what you said that > "inherently those two objects could have different times". The > outside-loop assertion? therefore makes better sense, and the loop was > kind of just wasting time to me (I mean you could get lucky to have the > two returning the same time down to a nanosecond, but that didn't make > the test better than just the out-of-loop assertion. > > my 2 cents > > -Joe > > On 6/1/2020 12:31 PM, naoto.sato at oracle.com wrote: >> Hello, >> >> Please review the fix to the following issue: >> >> https://bugs.openjdk.java.net/browse/JDK-8246261 >> >> The proposed changeset is located at: >> >> https://cr.openjdk.java.net/~naoto/8246261/webrev.00/ >> >> The test case compares two LocalTime objects, created with >> LocalTime.now(Clock/ZoneId). So inherently those two objects could >> have different times. The test tries to compare them 100 times for the >> exact match, and if not, then falls back to compare those times by >> truncating nanoseconds. The failure could occur when those two >> LocalTimes are around the whole second, e.g., expected == >> 18:14:22.999999 and test == 18:14:23.000001. To fix this, check the >> difference of those objects and ensure it is less than a second. >> >> Similar test cases exist in TCKLocalDateTime.java and >> TCKZonedDateTime.java so they should also be fixed. It is ok to leave >> the similar test case in TCKLocalDate.java, as multiple tries do exact >> match. >> >> Naoto > From naoto.sato at oracle.com Tue Jun 2 04:13:54 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Mon, 1 Jun 2020 21:13:54 -0700 Subject: [15] RFR: 8246261: TCKLocalTime.java failed due to "AssertionError: expected [18:14:22] but found [18:14:23]" In-Reply-To: References: <7cabdd8f-7d23-5685-ee0b-423737d829f9@oracle.com> <5935f47e-330a-1c2e-3b74-3268570e9947@oracle.com> Message-ID: <1abd9903-ec58-2360-0678-c74f4303456e@oracle.com> Updated webrev: http://cr.openjdk.java.net/~naoto/8246261/webrev.01/ Naoto On 6/1/20 6:30 PM, naoto.sato at oracle.com wrote: > Hi Joe, > > In fact, this bug was possibly revealed by the fix to 8242504, where the > system clock precision is now nanoseconds. Before that, it used to be > millisecond precision, so the first try for the exact match succeeded > for most of the cases. Even with the nano precision fix, most of the > cases the test exits with exact match in the loop. But you are right, > exact match or not does not matter in this test case, so I think we can > just eliminate these exact match try loops. I will remove them and do > some sniff testing on it. > > Naoto > > On 6/1/20 5:58 PM, Joe Wang wrote: >> Hi Naoto, >> >> The patch looks good to fix the failure. I'm just curious whether the >> 100-time comparison is necessary because of the existence of this >> assertion outside the loop that allowed the test to pass if the >> different was within a certain period of time. None of the tests had >> commented on the purpose of the test,? it looks like it's testing the >> assertion that (for the now method) "This will query the system clock >> to obtain the current time." The 100-loop therefore was a compromise >> for lack of a better way to prove that. I agree with what you said >> that "inherently those two objects could have different times". The >> outside-loop assertion? therefore makes better sense, and the loop was >> kind of just wasting time to me (I mean you could get lucky to have >> the two returning the same time down to a nanosecond, but that didn't >> make the test better than just the out-of-loop assertion. >> >> my 2 cents >> >> -Joe >> >> On 6/1/2020 12:31 PM, naoto.sato at oracle.com wrote: >>> Hello, >>> >>> Please review the fix to the following issue: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8246261 >>> >>> The proposed changeset is located at: >>> >>> https://cr.openjdk.java.net/~naoto/8246261/webrev.00/ >>> >>> The test case compares two LocalTime objects, created with >>> LocalTime.now(Clock/ZoneId). So inherently those two objects could >>> have different times. The test tries to compare them 100 times for >>> the exact match, and if not, then falls back to compare those times >>> by truncating nanoseconds. The failure could occur when those two >>> LocalTimes are around the whole second, e.g., expected == >>> 18:14:22.999999 and test == 18:14:23.000001. To fix this, check the >>> difference of those objects and ensure it is less than a second. >>> >>> Similar test cases exist in TCKLocalDateTime.java and >>> TCKZonedDateTime.java so they should also be fixed. It is ok to leave >>> the similar test case in TCKLocalDate.java, as multiple tries do >>> exact match. >>> >>> Naoto >> From huizhe.wang at oracle.com Tue Jun 2 06:03:21 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Mon, 1 Jun 2020 23:03:21 -0700 Subject: [15] RFR: 8246261: TCKLocalTime.java failed due to "AssertionError: expected [18:14:22] but found [18:14:23]" In-Reply-To: <1abd9903-ec58-2360-0678-c74f4303456e@oracle.com> References: <7cabdd8f-7d23-5685-ee0b-423737d829f9@oracle.com> <5935f47e-330a-1c2e-3b74-3268570e9947@oracle.com> <1abd9903-ec58-2360-0678-c74f4303456e@oracle.com> Message-ID: Looks good.? Thanks for the analysis! -Joe On 6/1/2020 9:13 PM, naoto.sato at oracle.com wrote: > Updated webrev: http://cr.openjdk.java.net/~naoto/8246261/webrev.01/ > > Naoto > > On 6/1/20 6:30 PM, naoto.sato at oracle.com wrote: >> Hi Joe, >> >> In fact, this bug was possibly revealed by the fix to 8242504, where >> the system clock precision is now nanoseconds. Before that, it used >> to be millisecond precision, so the first try for the exact match >> succeeded for most of the cases. Even with the nano precision fix, >> most of the cases the test exits with exact match in the loop. But >> you are right, exact match or not does not matter in this test case, >> so I think we can just eliminate these exact match try loops. I will >> remove them and do some sniff testing on it. >> >> Naoto >> >> On 6/1/20 5:58 PM, Joe Wang wrote: >>> Hi Naoto, >>> >>> The patch looks good to fix the failure. I'm just curious whether >>> the 100-time comparison is necessary because of the existence of >>> this assertion outside the loop that allowed the test to pass if the >>> different was within a certain period of time. None of the tests had >>> commented on the purpose of the test,? it looks like it's testing >>> the assertion that (for the now method) "This will query the system >>> clock to obtain the current time." The 100-loop therefore was a >>> compromise for lack of a better way to prove that. I agree with what >>> you said that "inherently those two objects could have different >>> times". The outside-loop assertion? therefore makes better sense, >>> and the loop was kind of just wasting time to me (I mean you could >>> get lucky to have the two returning the same time down to a >>> nanosecond, but that didn't make the test better than just the >>> out-of-loop assertion. >>> >>> my 2 cents >>> >>> -Joe >>> >>> On 6/1/2020 12:31 PM, naoto.sato at oracle.com wrote: >>>> Hello, >>>> >>>> Please review the fix to the following issue: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8246261 >>>> >>>> The proposed changeset is located at: >>>> >>>> https://cr.openjdk.java.net/~naoto/8246261/webrev.00/ >>>> >>>> The test case compares two LocalTime objects, created with >>>> LocalTime.now(Clock/ZoneId). So inherently those two objects could >>>> have different times. The test tries to compare them 100 times for >>>> the exact match, and if not, then falls back to compare those times >>>> by truncating nanoseconds. The failure could occur when those two >>>> LocalTimes are around the whole second, e.g., expected == >>>> 18:14:22.999999 and test == 18:14:23.000001. To fix this, check the >>>> difference of those objects and ensure it is less than a second. >>>> >>>> Similar test cases exist in TCKLocalDateTime.java and >>>> TCKZonedDateTime.java so they should also be fixed. It is ok to >>>> leave the similar test case in TCKLocalDate.java, as multiple tries >>>> do exact match. >>>> >>>> Naoto >>> From daniel.fuchs at oracle.com Tue Jun 2 09:01:12 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 2 Jun 2020 10:01:12 +0100 Subject: 8245867: Logger/bundleLeak/BundleTest.java fails due to "OutOfMemoryError: Java heap space" In-Reply-To: <96640076-76fa-b9db-7707-5900f6b1f151@oracle.com> References: <8657dfbc-4584-e8a4-ff14-b1056adcecff@oracle.com> <96640076-76fa-b9db-7707-5900f6b1f151@oracle.com> Message-ID: <7b953fb5-bf5e-4b88-e454-7097985e2c4a@oracle.com> Hi David, On 01/06/2020 23:43, David Holmes wrote: > Sorry the weekend got in the way :) No problem David. I pushed with your suggestion (not using SoftReference) since that appeared to be working too. If it fails again I can revisit :-) best regards, -- daniel From lance.andersen at oracle.com Tue Jun 2 11:29:33 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 2 Jun 2020 07:29:33 -0400 Subject: [15] RFR: 8246261: TCKLocalTime.java failed due to "AssertionError: expected [18:14:22] but found [18:14:23]" In-Reply-To: <1abd9903-ec58-2360-0678-c74f4303456e@oracle.com> References: <7cabdd8f-7d23-5685-ee0b-423737d829f9@oracle.com> <5935f47e-330a-1c2e-3b74-3268570e9947@oracle.com> <1abd9903-ec58-2360-0678-c74f4303456e@oracle.com> Message-ID: <91E330A9-4DC2-458C-8EDC-0E82ACBB7176@oracle.com> The change looks OK also with the loop removed. > On Jun 2, 2020, at 12:13 AM, naoto.sato at oracle.com wrote: > > Updated webrev: http://cr.openjdk.java.net/~naoto/8246261/webrev.01/ > > Naoto > > On 6/1/20 6:30 PM, naoto.sato at oracle.com wrote: >> Hi Joe, >> In fact, this bug was possibly revealed by the fix to 8242504, where the system clock precision is now nanoseconds. Before that, it used to be millisecond precision, so the first try for the exact match succeeded for most of the cases. Even with the nano precision fix, most of the cases the test exits with exact match in the loop. But you are right, exact match or not does not matter in this test case, so I think we can just eliminate these exact match try loops. I will remove them and do some sniff testing on it. >> Naoto >> On 6/1/20 5:58 PM, Joe Wang wrote: >>> Hi Naoto, >>> >>> The patch looks good to fix the failure. I'm just curious whether the 100-time comparison is necessary because of the existence of this assertion outside the loop that allowed the test to pass if the different was within a certain period of time. None of the tests had commented on the purpose of the test, it looks like it's testing the assertion that (for the now method) "This will query the system clock to obtain the current time." The 100-loop therefore was a compromise for lack of a better way to prove that. I agree with what you said that "inherently those two objects could have different times". The outside-loop assertion therefore makes better sense, and the loop was kind of just wasting time to me (I mean you could get lucky to have the two returning the same time down to a nanosecond, but that didn't make the test better than just the out-of-loop assertion. >>> >>> my 2 cents >>> >>> -Joe >>> >>> On 6/1/2020 12:31 PM, naoto.sato at oracle.com wrote: >>>> Hello, >>>> >>>> Please review the fix to the following issue: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8246261 >>>> >>>> The proposed changeset is located at: >>>> >>>> https://cr.openjdk.java.net/~naoto/8246261/webrev.00/ >>>> >>>> The test case compares two LocalTime objects, created with LocalTime.now(Clock/ZoneId). So inherently those two objects could have different times. The test tries to compare them 100 times for the exact match, and if not, then falls back to compare those times by truncating nanoseconds. The failure could occur when those two LocalTimes are around the whole second, e.g., expected == 18:14:22.999999 and test == 18:14:23.000001. To fix this, check the difference of those objects and ensure it is less than a second. >>>> >>>> Similar test cases exist in TCKLocalDateTime.java and TCKZonedDateTime.java so they should also be fixed. It is ok to leave the similar test case in TCKLocalDate.java, as multiple tries do exact match. >>>> >>>> Naoto >>> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From james.laskey at oracle.com Tue Jun 2 13:21:04 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Tue, 2 Jun 2020 10:21:04 -0300 Subject: RFR: 8230743 - StringJoiner does not handle too large strings correctly In-Reply-To: <20C9178B-D250-460E-9FBF-5524FFFD8344@oracle.com> References: <08ED85FA-7317-4995-877E-12C1AB8FE316@oracle.com> <1C81281C-45A7-48E7-8D04-F193DEDC42D1@oracle.com> <20C9178B-D250-460E-9FBF-5524FFFD8344@oracle.com> Message-ID: <9DECF322-9315-406C-B043-3CC206AA5E78@oracle.com> Revised with requested changes. http://cr.openjdk.java.net/~jlaskey/8230743/webrev-01 > On Jun 1, 2020, at 5:32 PM, Paul Sandoz wrote: > > > >> On Jun 1, 2020, at 12:44 PM, Jim Laskey > wrote: >> >> >> >>> On Jun 1, 2020, at 4:28 PM, Paul Sandoz > wrote: >>> >>> Can we consolidate the use of Integer.MAX_VALUE - 8 in tests by referring to the constant jdk.internal.util.ArraysSupport.MAX_ARRAY_LENGTH? >> >> I suppose it's possible to add module for java.base/jdk.internal.util.ArraysSupport to the tests. >> >> The mission here was to provide useful error messages. If we're going to clean up growing arrays throughout the JDK, I feel that is a separate mission, probably not something that we should be doing just before RD1. >> > > Ok. > > >> We should probably have gone one step further with jdk.internal.util.ArraysSupport.newLength and just defined jdk.internal.util.ArraysSupport.growArray(array, newSize, () -> { // exception code }); > >> >>> >>> >>> StringJoiner.java >>> ? >>> >>> 164 @Override >>> 165 public String toString() { >>> 166 final String[] elts = this.elts; >>> 167 if (elts == null && emptyValue != null) { >>> 168 return emptyValue; >>> 169 } >>> 170 final int size = this.size; >>> 171 final int addLen = prefix.length() + suffix.length(); >>> 172 if (len < 0 || len + addLen < 0) { >>> >>> It bothers me that len might be < 0, suggesting a larger issue. Perhaps look at the add method where len is modified? >> >> I thought about it but that would change the behaviour of StringJoiner. An error would be thrown on the add. Existing code that added but then tested length before trying a toString() would fail too soon. > > Or fail at the right point? Lazy String construction by the joiner can be considered an implementation detail. > > A negative len could result in some odd behavior: > > - an incorrect value returned from length(), such as a negative length value. > > - a number of adds could be performed, which one would have resulted in an OOME if the implementation was not lazy? > > - (len + other.len < 0) could be positive and result in array construction exception for a negative length on other.compactElts. > > Paul. > >> >>> >>> >>> 173 throw new >>> 174 OutOfMemoryError("Resulting string is exceeds maximum size?); >>> >>> "Resulting string is exceeds? ? -> "Resulting string exceeds? " >> >> Oops, thanks. >> >> >>> >>> Paul. >>> >>> >>>> On Jun 1, 2020, at 8:55 AM, Jim Laskey wrote: >>>> >>>> Change NegativeArraySizeException to OutOfMemoryError. Tests added. >>>> >>>> Cheers, >>>> >>>> -- Jim >>>> >>>> webrev: http://cr.openjdk.java.net/~jlaskey/8230743/webrev-00/index.html >>>> jbs: https://bugs.openjdk.java.net/browse/JDK-8230743 From james.laskey at oracle.com Tue Jun 2 13:52:55 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Tue, 2 Jun 2020 10:52:55 -0300 Subject: RFR: JDK-8230744 Several classes throw OutOfMemoryError without message In-Reply-To: References: <045D0E84-60E4-491B-9A31-7B05043FFE3B@oracle.com> <78cf935b-d7f3-342a-6ed1-a4a86becb727@oracle.com> Message-ID: Revised to reflect requested changes. http://cr.openjdk.java.net/~jlaskey/8230744/webrev-01/index.html > On Jun 1, 2020, at 5:40 PM, Martin Buchholz wrote: > > On Mon, Jun 1, 2020 at 1:37 PM Brent Christian > wrote: > >> "Unable to allocate buffer" seems vague in the context of an OOME. If >> the problem is trying to create a too-large array, maybe something like, >> "Buffer for \\Q...\\E sequence would exceed maximum array size". > > A generic message that does not necessarily expose the use of an > internal array seems best, e.g. > > "... exceeds implementation limits" From claes.redestad at oracle.com Tue Jun 2 13:56:58 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 2 Jun 2020 15:56:58 +0200 Subject: RFR: 8246338: Reduce overhead of normalizing file paths Message-ID: <596c8853-997b-6a2a-9e64-63c9bfc77579@oracle.com> Hi, this patch simplifies and improves performance of UnixFileSystem.normalize and prefixLength, which are used when opening files. The Windows implementation is left untouched, since it's a bit more complicated. Bug: https://bugs.openjdk.java.net/browse/JDK-8246338 Webrev: http://cr.openjdk.java.net/~redestad/8246338/open.00/ On the provided microbenchmark, we see a small (5-10%) speed-up in peak performance, while the speed-up is quite large when interpreting, especially in the case where the file path is already normalized (2.1x)[1]. Since this is the normal case, the patch has a small but measurable effect on some startup tests. Testing: tier1-2 Thanks! /Claes [1] Before: Benchmark Mode Cnt Score Error Units FileOpen.mix avgt 10 0.246 ? 0.005 us/op FileOpen.normalized avgt 10 0.022 ? 0.001 us/op FileOpen.notNormalized avgt 10 0.080 ? 0.002 us/op FileOpen.trailingSlash avgt 10 0.084 ? 0.003 us/op After: Benchmark Mode Cnt Score Error Units FileOpen.mix avgt 10 0.237 ? 0.004 us/op FileOpen.normalized avgt 10 0.019 ? 0.001 us/op FileOpen.notNormalized avgt 10 0.077 ? 0.002 us/op FileOpen.trailingSlash avgt 10 0.078 ? 0.002 us/op Before - -Xint: Benchmark Mode Cnt Score Error Units FileOpen.mix avgt 10 33.573 ? 0.348 us/op FileOpen.normalized avgt 10 6.219 ? 0.074 us/op FileOpen.notNormalized avgt 10 12.973 ? 0.084 us/op FileOpen.trailingSlash avgt 10 13.018 ? 0.066 us/op After - -Xint: Benchmark Mode Cnt Score Error Units FileOpen.mix avgt 10 27.084 ? 0.554 us/op FileOpen.normalized avgt 10 2.954 ? 0.005 us/op FileOpen.notNormalized avgt 10 10.225 ? 0.061 us/op FileOpen.trailingSlash avgt 10 10.211 ? 0.075 us/op From fernando.guallini at oracle.com Tue Jun 2 14:06:14 2020 From: fernando.guallini at oracle.com (Fernando Guallini) Date: Tue, 2 Jun 2020 15:06:14 +0100 Subject: RFR: JDK-8222241 - Example in ServiceLoader API docs should have one provides directive In-Reply-To: <1f33370f-a762-8259-ff61-a464a0f338be@oracle.com> References: <08E8734E-D070-4595-A8EE-9DC355962349@oracle.com> <1f33370f-a762-8259-ff61-a464a0f338be@oracle.com> Message-ID: <699FC006-9BEA-49CC-9123-A32D4134613F@oracle.com> Sure, added indentation here: http://cr.openjdk.java.net/~pconcannon/fguallin/8222241/webrevs/webrev.01/ This is how the html is rendered: Thanks > On 29 May 2020, at 19:00, Alan Bateman wrote: > > On 29/05/2020 18:12, Fernando Guallini wrote: >> Hi, >> in the ServiceLoader API docs, the given example specifying the service providers for a particular service should be comma-separated instead of using two separate ?provides? directives, that would result in a compilation error. >> >> Web rev: http://cr.openjdk.java.net/~pconcannon/fguallin/8222241/webrevs/webrev.00/ >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8222241 >> > Yes, it should be one `provides` directive. Can you intend the second class name to make the code fragment a bit more readable in the generated javadoc? > > -Alan From vicente.romero at oracle.com Tue Jun 2 14:06:23 2020 From: vicente.romero at oracle.com (Vicente Romero) Date: Tue, 2 Jun 2020 10:06:23 -0400 Subject: RFR: JDK-8227046: compiler implementation for sealed classes, JDK-8227047: Javadoc for sealed types and JDK-8227044: javax.lang.model for sealed classes In-Reply-To: <3bd27bea-b8a3-88a2-5536-ea9610749ce4@oracle.com> References: <3bd27bea-b8a3-88a2-5536-ea9610749ce4@oracle.com> Message-ID: <518ad004-fa16-178d-a9cf-c4e81d1b00d6@oracle.com> thanks to all the reviewers, the code for sealed classes has already been pushed to mainline. Vicente On 5/18/20 6:42 PM, Vicente Romero wrote: > Hi, > > Please review this patch for the compiler, javadoc and > javax.lang.model support for the JEP 360 Sealed Classes (Preview). The > changes are provided at [1], which implements the latest JLS for > sealed types [2]. The patch also include the needed changes to javadoc > and javax.lang.model to support sealed types. The CSR witht the > changes in the javax.lang.model spec is at [3]. The sealed types JEP > is accessible at [4]. There is an ongoing review for the VM and > core-libs code of sealed types [5] and that code hasn't been included > in this webrev, > > Thanks, > Vicente > > [1] http://cr.openjdk.java.net/~vromero/8227046/webrev.00/ > [2] > http://cr.openjdk.java.net/~gbierman/jep360/jep360-20200513/specs/sealed-classes-jls.html > > [3] https://bugs.openjdk.java.net/browse/JDK-8244367 > [4] https://openjdk.java.net/jeps/360 > [5] > https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-May/066440.html From julia.boes at oracle.com Tue Jun 2 15:43:35 2020 From: julia.boes at oracle.com (Julia Boes) Date: Tue, 2 Jun 2020 16:43:35 +0100 Subject: [PATCH] 8245694 : java.util.Properties.entrySet() does not override java.lang.Object methods In-Reply-To: <9487a1c9-d0bc-3632-156d-5927e5801dc8@icemanx.nl> References: <9487a1c9-d0bc-3632-156d-5927e5801dc8@icemanx.nl> Message-ID: <20109712-d505-1b17-3e6b-ae388d554400@oracle.com> Hi Yu Li, I'm happy to sponsor your fix once it's reviewed. Could you just confirm that you have signed the Oracle Contributor Agreement? https://www.oracle.com/technetwork/community/oca-486395.html Cheers, Julia On 30/05/2020 21:00, Rob Spoor wrote: > There's still a little regression there since Java 8, if you add the > keySet(), values() or entrySet() to the Properties object as either > key or value; in Java 8 this would cause "(this Collection)" to be > included, while even with this fix that would cause a > StackOverflowError. However, given that it's only a corner case for > most Collections and Maps in the first place, and with Properties > objects even more so, I don't think anyone would mind this regression. > > Rob > > On 30/05/2020 20:31, Lisa Li wrote: >> Hi, >> >> Please help review the fix for JDK-8245694 >> .? And this is my very >> first patch submission. I know it's not perfect. >> >> >> >> *BUG DESCRIPTION*: >> >> JDK-8245694 : >> java.util.Properties.entrySet() >> does not override java.lang.Object methods since java 9. >> >> >> *PROPOSED FIX*: >> >> Add delegating overrides for equals(), hashCode(), and toString(). >> >> >> *PATCH*: >> >> # HG changeset patch >> # User yuli >> # Date 1590841711 -28800 >> #????? Sat May 30 20:28:31 2020 +0800 >> # Node ID 7dc946e5b5863291fa070bfdbdce48aefbe87b7b >> # Parent? 8e28aae5068069e853673148e4d3f44cb50350a7 >> 8245694: java.util.Properties.entrySet() does not override Object >> methods >> Summary: Add missing override methods >> Contributed-by: Yu Li >> >> diff --git a/src/java.base/share/classes/java/util/Properties.java >> b/src/java.base/share/classes/java/util/Properties.java >> --- a/src/java.base/share/classes/java/util/Properties.java >> +++ b/src/java.base/share/classes/java/util/Properties.java >> @@ -1414,6 +1414,21 @@ >> ????????????? return entrySet.retainAll(c); >> ????????? } >> >> +???????? @Override >> +??????? public boolean equals(Object o) { >> +??????????? return entrySet.equals(o); >> +??????? } >> + >> +??????? @Override >> +??????? public int hashCode() { >> +??????????? return entrySet.hashCode(); >> +??????? } >> + >> +??????? @Override >> +??????? public String toString() { >> +??????????? return entrySet.toString(); >> +??????? } >> + >> ????????? @Override >> ????????? public Iterator> iterator() { >> ????????????? return entrySet.iterator(); >> diff --git a/test/jdk/java/util/Properties/PropertiesEntrySetTest.java >> b/test/jdk/java/util/Properties/PropertiesEntrySetTest.java >> new file mode 100644 >> --- /dev/null >> +++ b/test/jdk/java/util/Properties/PropertiesEntrySetTest.java >> @@ -0,0 +1,90 @@ >> +/* >> + * Copyright (c) 2018, 2019, Oracle and/or its affiliates. All rights >> reserved. >> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >> + * >> + * This code is free software; you can redistribute it and/or modify it >> + * under the terms of the GNU General Public License version 2 only, as >> + * published by the Free Software Foundation. >> + * >> + * This code is distributed in the hope that it will be useful, but >> WITHOUT >> + * ANY WARRANTY; without even the implied warranty of >> MERCHANTABILITY or >> + * FITNESS FOR A PARTICULAR PURPOSE.? See the GNU General Public >> License >> + * version 2 for more details (a copy is included in the LICENSE >> file that >> + * accompanied this code). >> + * >> + * You should have received a copy of the GNU General Public License >> version >> + * 2 along with this work; if not, write to the Free Software >> Foundation, >> + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. >> + * >> + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA >> 94065 USA >> + * or visit www.oracle.com if you need additional information or >> have any >> + * questions. >> + */ >> + >> +/* >> + * @test >> + * @summary tests the entrySet() method of Properties class >> + * @author? Yu Li >> + * @bug???? 8245694 >> + */ >> + >> +import java.util.Map; >> +import java.util.Properties; >> +import java.util.Set; >> + >> +public class PropertiesEntrySetTest { >> + >> +??? public static void main(String[] args) { >> + >> +??????? testEntrySet(); >> + >> +??? } >> + >> +??? private static void assertTrue(boolean value) { >> +??????? if (!value) { >> +??????????? throw new RuntimeException("Failure in Properties >> testing."); >> +??????? } >> +??? } >> + >> +??? private static void assertFalse(boolean value) { >> +??????? if (value) { >> +??????????? throw new RuntimeException("Failure in Properties >> testing."); >> +??????? } >> +??? } >> + >> +??? private static void testEntrySet() { >> +??????? Properties a = new Properties(); >> +??????? Set> aEntrySet = a.entrySet(); >> +??????? assertFalse(aEntrySet.equals(null)); >> +??????? assertTrue(aEntrySet.equals(aEntrySet)); >> + >> +??????? Properties b = new Properties(); >> +??????? Set> bEntrySet = b.entrySet(); >> +??????? assertTrue(aEntrySet.equals(bEntrySet)); >> +??????? assertTrue(aEntrySet.hashCode() == bEntrySet.hashCode()); >> + >> +??????? a.setProperty("p1", "1"); >> +??????? assertFalse(aEntrySet.equals(bEntrySet)); >> +??????? assertFalse(bEntrySet.equals(aEntrySet)); >> +??????? assertFalse(aEntrySet.hashCode() == bEntrySet.hashCode()); >> + >> +??????? b.setProperty("p1", "1"); >> +??????? assertTrue(aEntrySet.equals(bEntrySet)); >> +??????? assertTrue(aEntrySet.hashCode() == bEntrySet.hashCode()); >> + >> +??????? Properties c = new Properties(); >> +??????? c.setProperty("p1", "2"); >> +??????? Set> cEntrySet = c.entrySet(); >> +??????? assertFalse(cEntrySet.equals(bEntrySet)); >> +??????? assertFalse(cEntrySet.hashCode() == bEntrySet.hashCode()); >> +??????? assertFalse(cEntrySet.equals(aEntrySet)); >> +??????? assertFalse(cEntrySet.hashCode() == aEntrySet.hashCode()); >> + >> +??????? a.setProperty("p2", "2"); >> +??????? Properties d = new Properties(); >> +??????? d.setProperty("p2", "2"); >> +??????? d.setProperty("p1", "1"); >> +??????? Set> dEntrySet = d.entrySet(); >> +??????? assertTrue(dEntrySet.equals(aEntrySet)); >> +??? } >> +} >> >> Best regards, >> Yu Li From chris.hegarty at oracle.com Tue Jun 2 16:54:33 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 2 Jun 2020 17:54:33 +0100 Subject: RFR [15] 8238763: ObjectInputStream readUnshared method handling of Records Message-ID: <35DC4518-88E6-4AC9-AFB0-03CCAA609928@oracle.com> This is a small change to ObjectInputStream, for 8238763 [1], to correctly handle readUnshared for records. Webrev: https://cr.openjdk.java.net/~chegar/8238763/webrev.00/ -Chris. [1] https://bugs.openjdk.java.net/browse/JDK-8238763 From Alan.Bateman at oracle.com Tue Jun 2 18:07:58 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 2 Jun 2020 19:07:58 +0100 Subject: RFR: 8246338: Reduce overhead of normalizing file paths In-Reply-To: <596c8853-997b-6a2a-9e64-63c9bfc77579@oracle.com> References: <596c8853-997b-6a2a-9e64-63c9bfc77579@oracle.com> Message-ID: <961becf5-97eb-4450-54c1-c2f917099fa2@oracle.com> On 02/06/2020 14:56, Claes Redestad wrote: > Hi, > > this patch simplifies and improves performance of > UnixFileSystem.normalize and prefixLength, which are used when opening > files. The Windows implementation is left untouched, since it's a bit > more complicated. > > Bug:??? https://bugs.openjdk.java.net/browse/JDK-8246338 > Webrev: http://cr.openjdk.java.net/~redestad/8246338/open.00/ > > On the provided microbenchmark, we see a small (5-10%) speed-up > in peak performance, while the speed-up is quite large when > interpreting, especially in the case where the file path is already > normalized (2.1x)[1]. Since this is the normal case, the patch has a > small but measurable effect on some startup tests. I think this looks good. -Alan. From mandy.chung at oracle.com Tue Jun 2 18:38:24 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 2 Jun 2020 11:38:24 -0700 Subject: RFR(XL) 8198698: Support Lambda proxy classes in dynamic CDS archive In-Reply-To: <7da8b0f7-1027-d033-c1b5-7deb9d1d6c09@oracle.com> References: <7da8b0f7-1027-d033-c1b5-7deb9d1d6c09@oracle.com> Message-ID: <90ea692b-4a34-19e4-8f42-9f4092dc40e1@oracle.com> Hi Calvin, To recap an offline discussion, I think we should use JDK bootcycle build to test this feature.? For example generating a boot JDK with archived lambda proxy classes for java.base and javac to build itself and also use it to run JDK tier 1-3 tests.? This would serve a good test bed. ? I also suggest to experiment how JDK can benefit from this feature (e.g.? any java.base and javac performance gain). The module system initialization had to avoid using lambdas at startup time and we could re-examine this again. On 5/29/20 2:29 PM, Calvin Cheung wrote: > RFE: https://bugs.openjdk.java.net/browse/JDK-8198698 > > webrev: http://cr.openjdk.java.net/~ccheung/jdk15/8198698/webrev.00/ > Hidden classes have the following properties: 1. not discoverable by name.? It's not registered in the system dictionary. 2. It can be a dynamic nest member of a nest host.? For lambda proxy class, it has the same nest host as the caller class (same runtime package as the caller class) 3. strong or weak link with the class loader.? a hidden class may be unloaded even its defining class loader is alive. For the archived lambda proxy classes, do you re-assign a new class name to the shared lambda proxy class?? I suspect not.? It's very rare that it will conflict with the name of other hidden classes. It's an option to patch the name to indicate it's from shared archive if it helps diagnosability. My comments below are related to the above. 1374 InstanceKlass* SystemDictionary::load_shared_lambda_proxy_class(InstanceKlass* ik, 1375 Handle class_loader, 1376 Handle protection_domain, 1377 PackageEntry* pkg_entry, 1378 TRAPS) { 1379 InstanceKlass* shared_n_h = SystemDictionaryShared::get_shared_nest_host(ik); 1380 assert(shared_n_h->is_shared(), "nest host must be in CDS archive"); 1381 Symbol* cn = shared_n_h->name(); 1382 Klass *s = resolve_or_fail(cn, class_loader, protection_domain, true, CHECK_NULL); 1383 if (s != shared_n_h) { 1384 // The dynamically resolved nest_host is not the same as the one we used during dump time, 1385 // so we cannot use ik. 1386 return NULL;The above resolve_or_fail is to ensure that H is the same class loaded by the given class loader. Another validation that you should do is to check the nest host and nest member must be in the same runtime package - maybeik->set_nest_host(shared_n_h, THREAD) validates the runtime package?? SystemDictionaryShared::load_shared_lambda_proxy_class 1679 InstanceKlass* shared_n_h = get_shared_nest_host(lambda_ik); 1680 assert(shared_n_h != NULL, "unexpected NULL _nest_host"); 1681 1682 InstanceKlass* loaded_lambda = 1683 SystemDictionary::load_shared_lambda_proxy_class(lambda_ik, class_loader, protection_domain, pkg_entry, CHECK_NULL); 1684 1685 InstanceKlass* n_h = caller_ik->nest_host(THREAD); 1686 assert(n_h->has_nest_member(caller_ik, THREAD) || 1687 n_h == caller_ik, "mismatched nest host"); I think you meant to check n_h->has_nest_member(loaded_lambda, THREAD). It'd be good to add a comment to say that this ensures thatthe caller's nest host must be the same as the lambda proxy's nest host recorded at dump time. Nit: the variable name "n_h" or "_n_h" would be clearer if it's named "nest_host" or "_nest_host". 1689 SystemDictionary::set_shared_class_init_state(lambda_ik, InstanceKlass::loaded); A lambda proxy class typically is eagerly initialized (unless a flag is given). Your change dumps the hidden class with no knowledge if `Lookup::defineHiddenClass` is called with "initialize" parameter is true or not. This may affect the runtime performance (for example MethodHandle will have class init barrier if the class is not yet initialized). This may have performance implication. I think we should consider passing the initialize parameter to LambdaProxyClassArchive::find such that if the class is found from CDS archive, it will invoke before it returns to the library. One other point I brought up - strong vs weak hidden class (i.e. is it using the primary CLD or the per-mirror CLD?). I have lost myself where you do this in the implementation. Can you point that out? Is it recorded in the shared archive and make sure it uses the exact CLD when it's loaded? JVM_RegisterLambdaProxyClassForArchiving - we can add an assert to check if lambdaProxyClass is a hidden class. JVM_IsCDSDumpingEnabled and JVM_IsCDSSharingEnabled - an alternative library/VM interface is to use internal system properties that jdk.internal.misc.VM::saveProperties reads and removes these flags. InnerClassLambdaMetafactory 247 /** Use the LambdaProxyClassArchive for including the lambda proxy class 248 * in CDS archive and retrieving the class from the archive. If the class 249 * is not found in the archive, it calls generatesInnerClass to create 250 * the class. 251 */ What about simpler comment: /* * Spins the lambda proxy class. * * This first checks if a lambda proxy class can be loaded from CDS archive. * Otherwise, generate the lambda proxy class. */ This adds a VM call for every lambda proxy class creation. Do you have any the performance measurement when CDS is not on? Any performance regression? Mandy From Alan.Bateman at oracle.com Tue Jun 2 18:39:24 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 2 Jun 2020 19:39:24 +0100 Subject: RFR: JDK-8222241 - Example in ServiceLoader API docs should have one provides directive In-Reply-To: <699FC006-9BEA-49CC-9123-A32D4134613F@oracle.com> References: <08E8734E-D070-4595-A8EE-9DC355962349@oracle.com> <1f33370f-a762-8259-ff61-a464a0f338be@oracle.com> <699FC006-9BEA-49CC-9123-A32D4134613F@oracle.com> Message-ID: <6511e83d-32fd-4514-74f8-21fe58fa4dca@oracle.com> On 02/06/2020 15:06, Fernando Guallini wrote: > Sure, added indentation here: > http://cr.openjdk.java.net/~pconcannon/fguallin/8222241/webrevs/webrev.01/ Thanks, that is much clearer. -Alan From Roger.Riggs at oracle.com Tue Jun 2 19:34:02 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Tue, 2 Jun 2020 15:34:02 -0400 Subject: RFR [15] 8238763: ObjectInputStream readUnshared method handling of Records In-Reply-To: <35DC4518-88E6-4AC9-AFB0-03CCAA609928@oracle.com> References: <35DC4518-88E6-4AC9-AFB0-03CCAA609928@oracle.com> Message-ID: <36fbb32c-00df-b75b-3ab8-9d0ef141666a@oracle.com> Hi Chris, Looks good. Thanks, Roger On 6/2/20 12:54 PM, Chris Hegarty wrote: > This is a small change to ObjectInputStream, for 8238763 [1], to correctly handle readUnshared for records. > > Webrev: > https://cr.openjdk.java.net/~chegar/8238763/webrev.00/ > > -Chris. > > [1] https://bugs.openjdk.java.net/browse/JDK-8238763 From claes.redestad at oracle.com Tue Jun 2 19:46:22 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 2 Jun 2020 21:46:22 +0200 Subject: RFR: 8246338: Reduce overhead of normalizing file paths In-Reply-To: <961becf5-97eb-4450-54c1-c2f917099fa2@oracle.com> References: <596c8853-997b-6a2a-9e64-63c9bfc77579@oracle.com> <961becf5-97eb-4450-54c1-c2f917099fa2@oracle.com> Message-ID: On 2020-06-02 20:07, Alan Bateman wrote: >> Webrev: http://cr.openjdk.java.net/~redestad/8246338/open.00/ >> >> On the provided microbenchmark, we see a small (5-10%) speed-up >> in peak performance, while the speed-up is quite large when >> interpreting, especially in the case where the file path is already >> normalized (2.1x)[1]. Since this is the normal case, the patch has a >> small but measurable effect on some startup tests. > I think this looks good. > > -Alan. Thanks! /Claes From Roger.Riggs at oracle.com Tue Jun 2 19:49:56 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Tue, 2 Jun 2020 15:49:56 -0400 Subject: RFR: JDK-8230744 Several classes throw OutOfMemoryError without message In-Reply-To: References: <045D0E84-60E4-491B-9A31-7B05043FFE3B@oracle.com> <78cf935b-d7f3-342a-6ed1-a4a86becb727@oracle.com> Message-ID: <207076a6-06c1-7665-cdc4-91e515cf082e@oracle.com> Hi Jim, Given the infrequency of the exception messages being used, can we simplify them and/or reuse the strings? The stack trace would usually show what API was being used so I would simplfy the messages to: ???? "exceeds implementation limit" or "size exceeds implementation limit". ?? UnsyncByteArrayOutputStream: 49, 61, 73 ?? UnsyncByteArrayOutputStream:247 ?? AbstractStringBuilder:269 The message are not typically sentences so omit the final ".". ?? String.java: 3576, (I know it was there before), Unsafe.java:632? "Not able to allocate"?? -> "Unable to allocate". Isn't the "bytes of native memory" redundant with the function of allocateMemory? (and can be omitted). Ditto in reallocate, the allocate/reallocate is obvious from the stack trace. Thanks, Roger On 6/2/20 9:52 AM, Jim Laskey wrote: > Revised to reflect requested changes. > > http://cr.openjdk.java.net/~jlaskey/8230744/webrev-01/index.html > > >> On Jun 1, 2020, at 5:40 PM, Martin Buchholz wrote: >> >> On Mon, Jun 1, 2020 at 1:37 PM Brent Christian >> wrote: >> >>> "Unable to allocate buffer" seems vague in the context of an OOME. If >>> the problem is trying to create a too-large array, maybe something like, >>> "Buffer for \\Q...\\E sequence would exceed maximum array size". >> A generic message that does not necessarily expose the use of an >> internal array seems best, e.g. >> >> "... exceeds implementation limits" From stuart.marks at oracle.com Tue Jun 2 22:08:54 2020 From: stuart.marks at oracle.com (Stuart Marks) Date: Tue, 2 Jun 2020 15:08:54 -0700 Subject: Sometimes constraints are questionable In-Reply-To: <2F2E688C-51EF-4DAF-826E-70669940538D@oracle.com> References: <4A5AC9D1-AEB7-47E3-9E25-BF74FE01F3B0@oracle.com> <2F2E688C-51EF-4DAF-826E-70669940538D@oracle.com> Message-ID: Hi Jim, This was mentioned previously in this thread but not discussed very much. I suggest you take a look at jdk.internal.util.ArraysSupport.newLength(). Ivan Gerasimov and I worked this over fairly closely last year, and I'm pretty sure it does what Martin is saying, which I also think is the right thing. The intent is that it be used for things that have growable arrays, where the array might have a larger capacity than the logical number of elements currently stored. Sometimes the array needs to be grown to accommodate an immediate need (minGrowth) but it's desired to grow larger (prefGrowth) in anticipation of future needs. If minGrowth can't be accommodated, it throws OOME, but if prefGrowth can't be accommodated, it might be acceptable to provide a smaller amount of growth. (Of course, all this assumes that there is sufficient memory available to allocate the actual array. ArraysSupport.newLength doesn't attempt to ascertain that.) One issue is integer wraparound (overflow). This is the primary value that ArraysSupport.newLength provides. It would be good to centralize these computations instead of having them be spread all over. Another issue is the one that MAX_ARRAY_LENGTH (also called MAX_ARRAY_SIZE) is trying to address. This is sort-of a misnomer. It's not the actual maximum array size (which in fact isn't known the the library). It's actually the maximum array size that the library is fairly confident the VM can provide, assuming that enough memory is actually available. What the heck does that mean? The theoretical maximum array size is Integer.MAX_VALUE, since the JLS and JVMS don't allow anything larger. However, actual VM implementations will refuse to allocate an array above a certain amount slightly smaller than that, even if there is enough memory available. In practice, I believe the values for current Hotspot are Integer.MAX_VALUE-3 or Integer.MAX_VALUE-2, depending on whether compressed OOPS are in use. Why is this significant? Consider the following case, where the capacity of something is currently Integer.MAX_VALUE-100, and it's filled with elements. The application performs some operation that requires 50 elements (minGrowth) be added. A new array could certainly be allocated with size Integer.MAX_VALUE-50, but typical growth policies for these kinds of containers want to increase the current capacity by 1.5x or 2x (prefGrowth). Doing this multiplication would exceed Integer.MAX_VALUE, so that won't work. Clearly, we need to clamp the capacity somewhere. We don't want to clamp the capacity at Integer.MAX_VALUE, because this allocation would fail on every JVM I'm aware of, even if enough memory is available. So we don't do that. Instead, we clamp the preferred growth at some fairly arbitrary value smaller than Integer.MAX_VALUE, which is here called MAX_ARRAY_LENGTH, and increase the capacity to that instead. This allows the container's requested allocation to proceed: it satisfies minGrowth, but it doesn't satisfy prefGrowth. Instead, it returns a capacity value that's reasonably likely to succeed, given an unknown JVM implementation limit. Recall that the container now has Integer.MAX_VALUE-50 elements and the capacity is MAX_ARRAY_SIZE, which is currently defined somewhat arbitrarily at Integer.MAX_VALUE-8. Suppose the application now wants to add 43 elements. What should happen? We could say, this exceeds MAX_ARRAY_LENGTH, so refuse the request and throw OOME. This is reasonable and consistent in some sense, but perhaps not in another. Suppose there is sufficient memory, and the JVM does allow arrays of Integer.MAX_VALUE-7 to be created. Shouldn't we even try? That's what hugeLength() does -- it returns a value that attempts an allocation beyond the max preferential growth, and leaves it up to the JVM to grant or refuse the request based on its own implementation limits. Anyway, this is all quite subtle, and maybe comments in ArraysSupport don't describe this adequately. But the code that implements this kind of policy has been copied to different locations around the JDK, and it uses somewhat different terminology, and it might have slightly different bugs, but they're all essentially trying to implement this policy. ** Several questions could be asked: 1) Is this the right policy for implementing growable arrays? 2) In cases where a class needs a growable array, can and should it be refactored to use ArraysSupport.newLength()? 3) Does ArraysSupport.newLength() need to be modified to accommodate needs of additional call sites? 4) We might want to consider refactoring PriorityBlockingQueue and ArrayDeque to use ArraysSupport.newLength, in order to provide a consistent policy for collections. Other growable-array-based collections (Vector, ArrayList, PriorityQueue) do already. s'marks On 6/1/20 4:47 AM, Jim Laskey wrote: > Thanks David will run with that, > > >> On May 31, 2020, at 8:34 PM, David Holmes wrote: >> >> On 31/05/2020 12:29 am, Jim Laskey wrote: >>> I'm working through https://bugs.openjdk.java.net/browse/JDK-8230744 Several classes throw OutOfMemoryError without message . >>> I'm wondering why hugeCapacity in src/jdk.zipfs/share/classes/jdk/nio/zipfs/ByteArrayChannel.java is defined as >>> /** >>> * The maximum size of array to allocate. >>> * Some VMs reserve some header words in an array. >>> * Attempts to allocate larger arrays may result in >>> * OutOfMemoryError: Requested array size exceeds VM limit >>> */ >>> private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; >>> /** >>> * Increases the capacity to ensure that it can hold at least the >>> * number of elements specified by the minimum capacity argument. >>> * >>> * @param minCapacity the desired minimum capacity >>> */ >>> private void grow(int minCapacity) { >>> // overflow-conscious code >>> int oldCapacity = buf.length; >>> int newCapacity = oldCapacity << 1; >>> if (newCapacity - minCapacity < 0) >>> newCapacity = minCapacity; >>> if (newCapacity - MAX_ARRAY_SIZE > 0) >>> newCapacity = hugeCapacity(minCapacity); >>> buf = Arrays.copyOf(buf, newCapacity); >>> } >>> private static int hugeCapacity(int minCapacity) { >>> if (minCapacity < 0) // overflow >>> throw new OutOfMemoryError(); >> >> Not sure how we could have minCapacity < 0 at this point. It should have been checked before the call to grow, and grow will not make it negative. >> >>> return (minCapacity > MAX_ARRAY_SIZE) ? >>> Integer.MAX_VALUE : >>> MAX_ARRAY_SIZE; >> >> That's a bug plain and simple. It should never report a size > MAX_ARRAY_SIZE. >> >>> } >>> It just seems that it's pushing the inevitable off to Arrays.copyOf. Shouldn't it be: >>> private static int hugeCapacity(int minCapacity) { >>> if (minCapacity < 0 || minCapacity > MAX_ARRAY_SIZE) { >>> throw >>> new OutOfMemoryError("ByteArrayChannel exceeds maximum size: " + >>> MAX_ARRAY_SIZE); >>> } >>> return MAX_ARRAY_SIZE; >>> } >> >> That seems more appropriate to me - modulo the question mark over minCapacity being negative. >> >>> Real question: is there some hidden purpose behind this kind of logic? >> >> The basic strategy is to double the current capacity unless that will trigger an unnecessary exception, in which case just use the requested capacity, but again watch for the implementation limits. >> >> Cheers, >> David >> ----- >> >>> Cheers, >>> -- Jim > From stuart.marks at oracle.com Tue Jun 2 22:13:18 2020 From: stuart.marks at oracle.com (Stuart Marks) Date: Tue, 2 Jun 2020 15:13:18 -0700 Subject: RFR: JDK-8230744 Several classes throw OutOfMemoryError without message In-Reply-To: References: <045D0E84-60E4-491B-9A31-7B05043FFE3B@oracle.com> <78cf935b-d7f3-342a-6ed1-a4a86becb727@oracle.com> Message-ID: On 6/2/20 6:52 AM, Jim Laskey wrote: > Revised to reflect requested changes. > > http://cr.openjdk.java.net/~jlaskey/8230744/webrev-01/index.html On this, if all you're doing is changing exception messages, then I don't care very much, modulo concerns about wording from others. If you start to get into changing the growth logic (like the Math.addExact stuff) then please see my message on the related thread, "Sometimes constraints are questionable." Thanks, s'marks From paul.sandoz at oracle.com Tue Jun 2 22:59:36 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 2 Jun 2020 15:59:36 -0700 Subject: RFR: 8230743 - StringJoiner does not handle too large strings correctly In-Reply-To: <9DECF322-9315-406C-B043-3CC206AA5E78@oracle.com> References: <08ED85FA-7317-4995-877E-12C1AB8FE316@oracle.com> <1C81281C-45A7-48E7-8D04-F193DEDC42D1@oracle.com> <20C9178B-D250-460E-9FBF-5524FFFD8344@oracle.com> <9DECF322-9315-406C-B043-3CC206AA5E78@oracle.com> Message-ID: <0892A5BD-84A9-4B3D-8F9B-616530D2BFBF@oracle.com> +1 Paul. > On Jun 2, 2020, at 6:21 AM, Jim Laskey wrote: > > Revised with requested changes. > > > http://cr.openjdk.java.net/~jlaskey/8230743/webrev-01 > > >> On Jun 1, 2020, at 5:32 PM, Paul Sandoz wrote: >> >> >> >>> On Jun 1, 2020, at 12:44 PM, Jim Laskey > wrote: >>> >>> >>> >>>> On Jun 1, 2020, at 4:28 PM, Paul Sandoz > wrote: >>>> >>>> Can we consolidate the use of Integer.MAX_VALUE - 8 in tests by referring to the constant jdk.internal.util.ArraysSupport.MAX_ARRAY_LENGTH? >>> >>> I suppose it's possible to add module for java.base/jdk.internal.util.ArraysSupport to the tests. >>> >>> The mission here was to provide useful error messages. If we're going to clean up growing arrays throughout the JDK, I feel that is a separate mission, probably not something that we should be doing just before RD1. >>> >> >> Ok. >> >> >>> We should probably have gone one step further with jdk.internal.util.ArraysSupport.newLength and just defined jdk.internal.util.ArraysSupport.growArray(array, newSize, () -> { // exception code }); >> >>> >>>> >>>> >>>> StringJoiner.java >>>> ? >>>> >>>> 164 @Override >>>> 165 public String toString() { >>>> 166 final String[] elts = this.elts; >>>> 167 if (elts == null && emptyValue != null) { >>>> 168 return emptyValue; >>>> 169 } >>>> 170 final int size = this.size; >>>> 171 final int addLen = prefix.length() + suffix.length(); >>>> 172 if (len < 0 || len + addLen < 0) { >>>> >>>> It bothers me that len might be < 0, suggesting a larger issue. Perhaps look at the add method where len is modified? >>> >>> I thought about it but that would change the behaviour of StringJoiner. An error would be thrown on the add. Existing code that added but then tested length before trying a toString() would fail too soon. >> >> Or fail at the right point? Lazy String construction by the joiner can be considered an implementation detail. >> >> A negative len could result in some odd behavior: >> >> - an incorrect value returned from length(), such as a negative length value. >> >> - a number of adds could be performed, which one would have resulted in an OOME if the implementation was not lazy? >> >> - (len + other.len < 0) could be positive and result in array construction exception for a negative length on other.compactElts. >> >> Paul. >> >>> >>>> >>>> >>>> 173 throw new >>>> 174 OutOfMemoryError("Resulting string is exceeds maximum size?); >>>> >>>> "Resulting string is exceeds? ? -> "Resulting string exceeds? " >>> >>> Oops, thanks. >>> >>> >>>> >>>> Paul. >>>> >>>> >>>>> On Jun 1, 2020, at 8:55 AM, Jim Laskey wrote: >>>>> >>>>> Change NegativeArraySizeException to OutOfMemoryError. Tests added. >>>>> >>>>> Cheers, >>>>> >>>>> -- Jim >>>>> >>>>> webrev: http://cr.openjdk.java.net/~jlaskey/8230743/webrev-00/index.html >>>>> jbs: https://bugs.openjdk.java.net/browse/JDK-8230743 > From martinrb at google.com Tue Jun 2 23:46:36 2020 From: martinrb at google.com (Martin Buchholz) Date: Tue, 2 Jun 2020 16:46:36 -0700 Subject: Sometimes constraints are questionable In-Reply-To: References: <4A5AC9D1-AEB7-47E3-9E25-BF74FE01F3B0@oracle.com> <2F2E688C-51EF-4DAF-826E-70669940538D@oracle.com> Message-ID: On Tue, Jun 2, 2020 at 3:09 PM Stuart Marks wrote: > 4) We might want to consider refactoring PriorityBlockingQueue and ArrayDeque to > use ArraysSupport.newLength, in order to provide a consistent policy for > collections. Other growable-array-based collections (Vector, ArrayList, > PriorityQueue) do already. It's a goal to have PriorityQueue and PriorityBlockingQueue be as similar as possible. It's a goal to have ArrayDeque and ArrayList be as similar as possible. Sorry for lack of progress towards the goal. From brent.christian at oracle.com Tue Jun 2 23:51:15 2020 From: brent.christian at oracle.com (Brent Christian) Date: Tue, 2 Jun 2020 16:51:15 -0700 Subject: [PATCH] 8245694 : java.util.Properties.entrySet() does not override java.lang.Object methods In-Reply-To: <20109712-d505-1b17-3e6b-ae388d554400@oracle.com> References: <9487a1c9-d0bc-3632-156d-5927e5801dc8@icemanx.nl> <20109712-d505-1b17-3e6b-ae388d554400@oracle.com> Message-ID: <767a4dae-55f0-ca11-dbdf-57c7a22205d3@oracle.com> Thank you for submitting the fix, Yu Li. (And thanks for the sponsor, Julia). This will be good to fix. I have a few changes to recommend. src/java.base/share/classes/java/util/Properties.java The code change looks fine to me. The ending Copyright year just needs to be changed from 2019 -> 2020. test/jdk/java/util/Properties/PropertiesEntrySetTest.java + * Copyright (c) 2018, 2019, Oracle and/or its affiliates. All rights For a newly added file, only the current year is listed, so: Copyright (c) 2020, Oracle and/or its affiliates. All rights reserved. +/* + * @test + * @summary tests the entrySet() method of Properties class + * @author Yu Li + * @bug 8245694 + */ Please also add an @run tag. Using testng would prevent needing to add assert[True|False] methods. So that would be: @run testng PropertiesEntrySetTest Then you should be able to: import org.testng.Assert; and use the testng Assert.assertTrue() and Assert.assertFalse(). + Set> aEntrySet = a.entrySet(); You might consider using 'var' to simplify declarations of aEntrySet, bEntrySet, etc Please also add testing of EntrySet.toString(). Along with testing equals/hashCode/toString after properties are added, I think it would be good to also test after properties are removed. There's Properties.remove()/clear(), and also maybe some of the removal methods listed in Hashtable.entrySet(): Iterator.remove(), Set.remove(), etc. (And just as a point of interest, there is a test, test/jdk/java/util/Properties/CheckOverrides.java, to ensure that the Properties class overrides all necessary methods of its superclasses. Checks were not included for Properties.EntrySet (hence this bug), though I don't think it makes sense to add any at this point.) Thanks a lot, -Brent On 6/2/20 8:43 AM, Julia Boes wrote: > Hi Yu Li, > > I'm happy to sponsor your fix once it's reviewed. Could you just confirm > that you have signed the Oracle Contributor Agreement? > > https://www.oracle.com/technetwork/community/oca-486395.html > > Cheers, > > Julia > > On 30/05/2020 21:00, Rob Spoor wrote: >> There's still a little regression there since Java 8, if you add the >> keySet(), values() or entrySet() to the Properties object as either >> key or value; in Java 8 this would cause "(this Collection)" to be >> included, while even with this fix that would cause a >> StackOverflowError. However, given that it's only a corner case for >> most Collections and Maps in the first place, and with Properties >> objects even more so, I don't think anyone would mind this regression. >> >> Rob >> >> On 30/05/2020 20:31, Lisa Li wrote: >>> Hi, >>> >>> Please help review the fix for JDK-8245694 >>> .? And this is my very >>> first patch submission. I know it's not perfect. >>> >>> >>> >>> *BUG DESCRIPTION*: >>> >>> JDK-8245694 : >>> java.util.Properties.entrySet() >>> does not override java.lang.Object methods since java 9. >>> >>> >>> *PROPOSED FIX*: >>> >>> Add delegating overrides for equals(), hashCode(), and toString(). >>> >>> >>> *PATCH*: >>> >>> # HG changeset patch >>> # User yuli >>> # Date 1590841711 -28800 >>> #????? Sat May 30 20:28:31 2020 +0800 >>> # Node ID 7dc946e5b5863291fa070bfdbdce48aefbe87b7b >>> # Parent? 8e28aae5068069e853673148e4d3f44cb50350a7 >>> 8245694: java.util.Properties.entrySet() does not override Object >>> methods >>> Summary: Add missing override methods >>> Contributed-by: Yu Li >>> >>> diff --git a/src/java.base/share/classes/java/util/Properties.java >>> b/src/java.base/share/classes/java/util/Properties.java >>> --- a/src/java.base/share/classes/java/util/Properties.java >>> +++ b/src/java.base/share/classes/java/util/Properties.java >>> @@ -1414,6 +1414,21 @@ >>> ????????????? return entrySet.retainAll(c); >>> ????????? } >>> >>> +???????? @Override >>> +??????? public boolean equals(Object o) { >>> +??????????? return entrySet.equals(o); >>> +??????? } >>> + >>> +??????? @Override >>> +??????? public int hashCode() { >>> +??????????? return entrySet.hashCode(); >>> +??????? } >>> + >>> +??????? @Override >>> +??????? public String toString() { >>> +??????????? return entrySet.toString(); >>> +??????? } >>> + >>> ????????? @Override >>> ????????? public Iterator> iterator() { >>> ????????????? return entrySet.iterator(); >>> diff --git a/test/jdk/java/util/Properties/PropertiesEntrySetTest.java >>> b/test/jdk/java/util/Properties/PropertiesEntrySetTest.java >>> new file mode 100644 >>> --- /dev/null >>> +++ b/test/jdk/java/util/Properties/PropertiesEntrySetTest.java >>> @@ -0,0 +1,90 @@ >>> +/* >>> + * Copyright (c) 2018, 2019, Oracle and/or its affiliates. All rights >>> reserved. >>> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >>> + * >>> + * This code is free software; you can redistribute it and/or modify it >>> + * under the terms of the GNU General Public License version 2 only, as >>> + * published by the Free Software Foundation. >>> + * >>> + * This code is distributed in the hope that it will be useful, but >>> WITHOUT >>> + * ANY WARRANTY; without even the implied warranty of >>> MERCHANTABILITY or >>> + * FITNESS FOR A PARTICULAR PURPOSE.? See the GNU General Public >>> License >>> + * version 2 for more details (a copy is included in the LICENSE >>> file that >>> + * accompanied this code). >>> + * >>> + * You should have received a copy of the GNU General Public License >>> version >>> + * 2 along with this work; if not, write to the Free Software >>> Foundation, >>> + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. >>> + * >>> + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA >>> 94065 USA >>> + * or visit www.oracle.com if you need additional information or >>> have any >>> + * questions. >>> + */ >>> + >>> +/* >>> + * @test >>> + * @summary tests the entrySet() method of Properties class >>> + * @author? Yu Li >>> + * @bug???? 8245694 >>> + */ >>> + >>> +import java.util.Map; >>> +import java.util.Properties; >>> +import java.util.Set; >>> + >>> +public class PropertiesEntrySetTest { >>> + >>> +??? public static void main(String[] args) { >>> + >>> +??????? testEntrySet(); >>> + >>> +??? } >>> + >>> +??? private static void assertTrue(boolean value) { >>> +??????? if (!value) { >>> +??????????? throw new RuntimeException("Failure in Properties >>> testing."); >>> +??????? } >>> +??? } >>> + >>> +??? private static void assertFalse(boolean value) { >>> +??????? if (value) { >>> +??????????? throw new RuntimeException("Failure in Properties >>> testing."); >>> +??????? } >>> +??? } >>> + >>> +??? private static void testEntrySet() { >>> +??????? Properties a = new Properties(); >>> +??????? Set> aEntrySet = a.entrySet(); >>> +??????? assertFalse(aEntrySet.equals(null)); >>> +??????? assertTrue(aEntrySet.equals(aEntrySet)); >>> + >>> +??????? Properties b = new Properties(); >>> +??????? Set> bEntrySet = b.entrySet(); >>> +??????? assertTrue(aEntrySet.equals(bEntrySet)); >>> +??????? assertTrue(aEntrySet.hashCode() == bEntrySet.hashCode()); >>> + >>> +??????? a.setProperty("p1", "1"); >>> +??????? assertFalse(aEntrySet.equals(bEntrySet)); >>> +??????? assertFalse(bEntrySet.equals(aEntrySet)); >>> +??????? assertFalse(aEntrySet.hashCode() == bEntrySet.hashCode()); >>> + >>> +??????? b.setProperty("p1", "1"); >>> +??????? assertTrue(aEntrySet.equals(bEntrySet)); >>> +??????? assertTrue(aEntrySet.hashCode() == bEntrySet.hashCode()); >>> + >>> +??????? Properties c = new Properties(); >>> +??????? c.setProperty("p1", "2"); >>> +??????? Set> cEntrySet = c.entrySet(); >>> +??????? assertFalse(cEntrySet.equals(bEntrySet)); >>> +??????? assertFalse(cEntrySet.hashCode() == bEntrySet.hashCode()); >>> +??????? assertFalse(cEntrySet.equals(aEntrySet)); >>> +??????? assertFalse(cEntrySet.hashCode() == aEntrySet.hashCode()); >>> + >>> +??????? a.setProperty("p2", "2"); >>> +??????? Properties d = new Properties(); >>> +??????? d.setProperty("p2", "2"); >>> +??????? d.setProperty("p1", "1"); >>> +??????? Set> dEntrySet = d.entrySet(); >>> +??????? assertTrue(dEntrySet.equals(aEntrySet)); >>> +??? } >>> +} >>> >>> Best regards, >>> Yu Li From martinrb at google.com Tue Jun 2 23:53:10 2020 From: martinrb at google.com (Martin Buchholz) Date: Tue, 2 Jun 2020 16:53:10 -0700 Subject: RFR: 8230743 - StringJoiner does not handle too large strings correctly In-Reply-To: <9DECF322-9315-406C-B043-3CC206AA5E78@oracle.com> References: <08ED85FA-7317-4995-877E-12C1AB8FE316@oracle.com> <1C81281C-45A7-48E7-8D04-F193DEDC42D1@oracle.com> <20C9178B-D250-460E-9FBF-5524FFFD8344@oracle.com> <9DECF322-9315-406C-B043-3CC206AA5E78@oracle.com> Message-ID: One of these error messages is not like the other + OutOfMemoryError("Resulting string is exceeds maximum size"); + OutOfMemoryError("Resulting string exceeds maximum size"); From david.holmes at oracle.com Wed Jun 3 05:30:32 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 3 Jun 2020 15:30:32 +1000 Subject: RFR(XL) 8198698: Support Lambda proxy classes in dynamic CDS archive In-Reply-To: <90ea692b-4a34-19e4-8f42-9f4092dc40e1@oracle.com> References: <7da8b0f7-1027-d033-c1b5-7deb9d1d6c09@oracle.com> <90ea692b-4a34-19e4-8f42-9f4092dc40e1@oracle.com> Message-ID: <018fa13d-9c60-7abe-5aa3-f12562d82cd7@oracle.com> Hi Mandy, On 3/06/2020 4:38 am, Mandy Chung wrote: > Hi Calvin, > > To recap an offline discussion, I think we should use JDK bootcycle > build to test this feature.? For example generating a boot JDK with > archived lambda proxy classes for java.base and javac to build itself > and also use it to run JDK tier 1-3 tests.? This would serve a good test > bed. ? I also suggest to experiment how JDK can benefit from this > feature (e.g.? any java.base and javac performance gain). The module > system initialization had to avoid using lambdas at startup time and we > could re-examine this again. I don't see how this affects system initialization given it requires use of CDS which is optional. > On 5/29/20 2:29 PM, Calvin Cheung wrote: >> RFE: https://bugs.openjdk.java.net/browse/JDK-8198698 >> >> webrev: http://cr.openjdk.java.net/~ccheung/jdk15/8198698/webrev.00/ >> > > Hidden classes have the following properties: > 1. not discoverable by name.? It's not registered in the system dictionary. > 2. It can be a dynamic nest member of a nest host.? For lambda proxy > class, it has the same nest host as the caller class (same runtime > package as the caller class) > 3. strong or weak link with the class loader.? a hidden class may be > unloaded even its defining class loader is alive. > > For the archived lambda proxy classes, do you re-assign a new class name > to the shared lambda proxy class?? I suspect not.? It's very rare that > it will conflict with the name of other hidden classes. It's an option > to patch the name to indicate it's from shared archive if it helps > diagnosability. > > My comments below are related to the above. > > 1374 InstanceKlass* > SystemDictionary::load_shared_lambda_proxy_class(InstanceKlass* ik, > 1375 Handle class_loader, > 1376 Handle protection_domain, > 1377 PackageEntry* pkg_entry, > 1378 TRAPS) { > 1379 InstanceKlass* shared_n_h = > SystemDictionaryShared::get_shared_nest_host(ik); > 1380 assert(shared_n_h->is_shared(), "nest host must be in CDS archive"); > 1381 Symbol* cn = shared_n_h->name(); > 1382 Klass *s = resolve_or_fail(cn, class_loader, protection_domain, > true, CHECK_NULL); > 1383 if (s != shared_n_h) { > 1384 // The dynamically resolved nest_host is not the same as the one we > used during dump time, > 1385 // so we cannot use ik. > 1386 return NULL;The above resolve_or_fail is to ensure that H is the > same class loaded by the given class loader.? Another validation that > you should do is to check > the nest host and nest member must be in the same runtime package - > maybeik->set_nest_host(shared_n_h, THREAD) validates the runtime package?? Aren't we assuming/requiring that the dump is valid in that regard? Otherwise there would be many checks that would have to be re-done to ensure an archived class matches the "same" class that would be dynamically loaded. ?? > > SystemDictionaryShared::load_shared_lambda_proxy_class > > 1679?? InstanceKlass* shared_n_h = get_shared_nest_host(lambda_ik); > 1680?? assert(shared_n_h != NULL, "unexpected NULL _nest_host"); > 1681 > 1682?? InstanceKlass* loaded_lambda = > 1683???? SystemDictionary::load_shared_lambda_proxy_class(lambda_ik, > class_loader, protection_domain, pkg_entry, CHECK_NULL); > 1684 > 1685?? InstanceKlass* n_h = caller_ik->nest_host(THREAD); > 1686?? assert(n_h->has_nest_member(caller_ik, THREAD) || > 1687????????? n_h == caller_ik, "mismatched nest host"); > > I think you meant to check n_h->has_nest_member(loaded_lambda, THREAD). That can't be right. has_nest_member is for checking static nest membership. Dynamic proxies are not static nest members! David ----- > It'd be good to add a comment to say that this ensures thatthe caller's > nest host must be the same as the lambda proxy's nest host recorded at > dump time. > Nit: the variable name "n_h" or "_n_h" would be clearer if it's named > "nest_host" or "_nest_host". > > 1689?? SystemDictionary::set_shared_class_init_state(lambda_ik, > InstanceKlass::loaded); > > A lambda proxy class typically is eagerly initialized (unless a flag is > given). > Your change dumps the hidden class with no knowledge if > `Lookup::defineHiddenClass` > is called with "initialize" parameter is true or not.? This may affect > the runtime > performance (for example MethodHandle will have class init barrier if > the class > is not yet initialized). > This may have performance implication. I think we should consider > passing the initialize parameter to LambdaProxyClassArchive::find such > that if the class is found from CDS archive, it will invoke > before it returns to the library. One other point I brought up - strong > vs weak hidden class (i.e. is it using the primary CLD or the per-mirror > CLD?). I have lost myself where you do this in the implementation. Can > you point that out? Is it recorded in the shared archive and make sure > it uses the exact CLD when it's loaded? > > JVM_RegisterLambdaProxyClassForArchiving - we can add an assert to check > if lambdaProxyClass is a hidden class. JVM_IsCDSDumpingEnabled and > JVM_IsCDSSharingEnabled - an alternative library/VM interface is to use > internal system properties that jdk.internal.misc.VM::saveProperties > reads and removes these flags. InnerClassLambdaMetafactory 247 /** Use > the LambdaProxyClassArchive for including the lambda proxy class 248 * > in CDS archive and retrieving the class from the archive. If the class > 249 * is not found in the archive, it calls generatesInnerClass to > create 250 * the class. 251 */ What about simpler comment: /* * Spins > the lambda proxy class. * * This first checks if a lambda proxy class > can be loaded from CDS archive. * Otherwise, generate the lambda proxy > class. */ This adds a VM call for every lambda proxy class creation. Do > you have any the performance measurement when CDS is not on? Any > performance regression? Mandy > From liyu.italy at gmail.com Wed Jun 3 05:37:30 2020 From: liyu.italy at gmail.com (Lisa Li) Date: Wed, 3 Jun 2020 13:37:30 +0800 Subject: [PATCH] 8245694 : java.util.Properties.entrySet() does not override java.lang.Object methods In-Reply-To: <767a4dae-55f0-ca11-dbdf-57c7a22205d3@oracle.com> References: <9487a1c9-d0bc-3632-156d-5927e5801dc8@icemanx.nl> <20109712-d505-1b17-3e6b-ae388d554400@oracle.com> <767a4dae-55f0-ca11-dbdf-57c7a22205d3@oracle.com> Message-ID: Hi? Thanks a lot Julia! I have signed OCA and emailed it to Oracle last week. But I haven't seen my name on the contributor list yet. I guess I'll have to wait. And thanks a lot for all the advice, Brent! It's really useful. I'll do as you said. Best regards, Yu On Wed, Jun 3, 2020 at 7:51 AM Brent Christian wrote: > Thank you for submitting the fix, Yu Li. (And thanks for the sponsor, > Julia). This will be good to fix. I have a few changes to recommend. > > src/java.base/share/classes/java/util/Properties.java > > The code change looks fine to me. > > The ending Copyright year just needs to be changed from 2019 -> 2020. > > > test/jdk/java/util/Properties/PropertiesEntrySetTest.java > > + * Copyright (c) 2018, 2019, Oracle and/or its affiliates. All rights > > For a newly added file, only the current year is listed, so: > > Copyright (c) 2020, Oracle and/or its affiliates. All rights > reserved. > > +/* > + * @test > + * @summary tests the entrySet() method of Properties class > + * @author Yu Li > + * @bug 8245694 > + */ > > Please also add an @run tag. Using testng would prevent needing to add > assert[True|False] methods. So that would be: > > @run testng PropertiesEntrySetTest > > Then you should be able to: > > import org.testng.Assert; > > and use the testng Assert.assertTrue() and Assert.assertFalse(). > > > + Set> aEntrySet = a.entrySet(); > > You might consider using 'var' to simplify declarations of aEntrySet, > bEntrySet, etc > > > Please also add testing of EntrySet.toString(). > > Along with testing equals/hashCode/toString after properties are added, > I think it would be good to also test after properties are removed. > There's Properties.remove()/clear(), and also maybe some of the removal > methods listed in Hashtable.entrySet(): Iterator.remove(), Set.remove(), > etc. > > > (And just as a point of interest, there is a test, > test/jdk/java/util/Properties/CheckOverrides.java, > to ensure that the Properties class overrides all necessary methods of > its superclasses. Checks were not included for Properties.EntrySet > (hence this bug), though I don't think it makes sense to add any at this > point.) > > Thanks a lot, > -Brent > > On 6/2/20 8:43 AM, Julia Boes wrote: > > Hi Yu Li, > > > > I'm happy to sponsor your fix once it's reviewed. Could you just confirm > > that you have signed the Oracle Contributor Agreement? > > > > https://www.oracle.com/technetwork/community/oca-486395.html > > > > Cheers, > > > > Julia > > > > On 30/05/2020 21:00, Rob Spoor wrote: > >> There's still a little regression there since Java 8, if you add the > >> keySet(), values() or entrySet() to the Properties object as either > >> key or value; in Java 8 this would cause "(this Collection)" to be > >> included, while even with this fix that would cause a > >> StackOverflowError. However, given that it's only a corner case for > >> most Collections and Maps in the first place, and with Properties > >> objects even more so, I don't think anyone would mind this regression. > >> > >> Rob > >> > >> On 30/05/2020 20:31, Lisa Li wrote: > >>> Hi, > >>> > >>> Please help review the fix for JDK-8245694 > >>> . And this is my > very > >>> first patch submission. I know it's not perfect. > >>> > >>> > >>> > >>> *BUG DESCRIPTION*: > >>> > >>> JDK-8245694 : > >>> java.util.Properties.entrySet() > >>> does not override java.lang.Object methods since java 9. > >>> > >>> > >>> *PROPOSED FIX*: > >>> > >>> Add delegating overrides for equals(), hashCode(), and toString(). > >>> > >>> > >>> *PATCH*: > >>> > >>> # HG changeset patch > >>> # User yuli > >>> # Date 1590841711 -28800 > >>> # Sat May 30 20:28:31 2020 +0800 > >>> # Node ID 7dc946e5b5863291fa070bfdbdce48aefbe87b7b > >>> # Parent 8e28aae5068069e853673148e4d3f44cb50350a7 > >>> 8245694: java.util.Properties.entrySet() does not override Object > >>> methods > >>> Summary: Add missing override methods > >>> Contributed-by: Yu Li > >>> > >>> diff --git a/src/java.base/share/classes/java/util/Properties.java > >>> b/src/java.base/share/classes/java/util/Properties.java > >>> --- a/src/java.base/share/classes/java/util/Properties.java > >>> +++ b/src/java.base/share/classes/java/util/Properties.java > >>> @@ -1414,6 +1414,21 @@ > >>> return entrySet.retainAll(c); > >>> } > >>> > >>> + @Override > >>> + public boolean equals(Object o) { > >>> + return entrySet.equals(o); > >>> + } > >>> + > >>> + @Override > >>> + public int hashCode() { > >>> + return entrySet.hashCode(); > >>> + } > >>> + > >>> + @Override > >>> + public String toString() { > >>> + return entrySet.toString(); > >>> + } > >>> + > >>> @Override > >>> public Iterator> iterator() { > >>> return entrySet.iterator(); > >>> diff --git a/test/jdk/java/util/Properties/PropertiesEntrySetTest.java > >>> b/test/jdk/java/util/Properties/PropertiesEntrySetTest.java > >>> new file mode 100644 > >>> --- /dev/null > >>> +++ b/test/jdk/java/util/Properties/PropertiesEntrySetTest.java > >>> @@ -0,0 +1,90 @@ > >>> +/* > >>> + * Copyright (c) 2018, 2019, Oracle and/or its affiliates. All rights > >>> reserved. > >>> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > >>> + * > >>> + * This code is free software; you can redistribute it and/or modify > it > >>> + * under the terms of the GNU General Public License version 2 only, > as > >>> + * published by the Free Software Foundation. > >>> + * > >>> + * This code is distributed in the hope that it will be useful, but > >>> WITHOUT > >>> + * ANY WARRANTY; without even the implied warranty of > >>> MERCHANTABILITY or > >>> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public > >>> License > >>> + * version 2 for more details (a copy is included in the LICENSE > >>> file that > >>> + * accompanied this code). > >>> + * > >>> + * You should have received a copy of the GNU General Public License > >>> version > >>> + * 2 along with this work; if not, write to the Free Software > >>> Foundation, > >>> + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. > >>> + * > >>> + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA > >>> 94065 USA > >>> + * or visit www.oracle.com if you need additional information or > >>> have any > >>> + * questions. > >>> + */ > >>> + > >>> +/* > >>> + * @test > >>> + * @summary tests the entrySet() method of Properties class > >>> + * @author Yu Li > >>> + * @bug 8245694 > >>> + */ > >>> + > >>> +import java.util.Map; > >>> +import java.util.Properties; > >>> +import java.util.Set; > >>> + > >>> +public class PropertiesEntrySetTest { > >>> + > >>> + public static void main(String[] args) { > >>> + > >>> + testEntrySet(); > >>> + > >>> + } > >>> + > >>> + private static void assertTrue(boolean value) { > >>> + if (!value) { > >>> + throw new RuntimeException("Failure in Properties > >>> testing."); > >>> + } > >>> + } > >>> + > >>> + private static void assertFalse(boolean value) { > >>> + if (value) { > >>> + throw new RuntimeException("Failure in Properties > >>> testing."); > >>> + } > >>> + } > >>> + > >>> + private static void testEntrySet() { > >>> + Properties a = new Properties(); > >>> + Set> aEntrySet = a.entrySet(); > >>> + assertFalse(aEntrySet.equals(null)); > >>> + assertTrue(aEntrySet.equals(aEntrySet)); > >>> + > >>> + Properties b = new Properties(); > >>> + Set> bEntrySet = b.entrySet(); > >>> + assertTrue(aEntrySet.equals(bEntrySet)); > >>> + assertTrue(aEntrySet.hashCode() == bEntrySet.hashCode()); > >>> + > >>> + a.setProperty("p1", "1"); > >>> + assertFalse(aEntrySet.equals(bEntrySet)); > >>> + assertFalse(bEntrySet.equals(aEntrySet)); > >>> + assertFalse(aEntrySet.hashCode() == bEntrySet.hashCode()); > >>> + > >>> + b.setProperty("p1", "1"); > >>> + assertTrue(aEntrySet.equals(bEntrySet)); > >>> + assertTrue(aEntrySet.hashCode() == bEntrySet.hashCode()); > >>> + > >>> + Properties c = new Properties(); > >>> + c.setProperty("p1", "2"); > >>> + Set> cEntrySet = c.entrySet(); > >>> + assertFalse(cEntrySet.equals(bEntrySet)); > >>> + assertFalse(cEntrySet.hashCode() == bEntrySet.hashCode()); > >>> + assertFalse(cEntrySet.equals(aEntrySet)); > >>> + assertFalse(cEntrySet.hashCode() == aEntrySet.hashCode()); > >>> + > >>> + a.setProperty("p2", "2"); > >>> + Properties d = new Properties(); > >>> + d.setProperty("p2", "2"); > >>> + d.setProperty("p1", "1"); > >>> + Set> dEntrySet = d.entrySet(); > >>> + assertTrue(dEntrySet.equals(aEntrySet)); > >>> + } > >>> +} > >>> > >>> Best regards, > >>> Yu Li > From david.holmes at oracle.com Wed Jun 3 06:49:53 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 3 Jun 2020 16:49:53 +1000 Subject: Sometimes constraints are questionable In-Reply-To: References: <4A5AC9D1-AEB7-47E3-9E25-BF74FE01F3B0@oracle.com> <2F2E688C-51EF-4DAF-826E-70669940538D@oracle.com> Message-ID: <43c1a46e-eaa1-c079-b400-7280929acb5f@oracle.com> Hi Stuart, On 3/06/2020 8:08 am, Stuart Marks wrote: > Hi Jim, > > This was mentioned previously in this thread but not discussed very > much. I suggest you take a look at > jdk.internal.util.ArraysSupport.newLength(). Ivan Gerasimov and I worked > this over fairly closely last year, and I'm pretty sure it does what > Martin is saying, which I also think is the right thing. > > The intent is that it be used for things that have growable arrays, > where the array might have a larger capacity than the logical number of > elements currently stored. Sometimes the array needs to be grown to > accommodate an immediate need (minGrowth) but it's desired to grow > larger (prefGrowth) in anticipation of future needs. If minGrowth can't > be accommodated, it throws OOME, but if prefGrowth can't be > accommodated, it might be acceptable to provide a smaller amount of growth. > > (Of course, all this assumes that there is sufficient memory available > to allocate the actual array. ArraysSupport.newLength doesn't attempt to > ascertain that.) > > One issue is integer wraparound (overflow). This is the primary value > that ArraysSupport.newLength provides. It would be good to centralize > these computations instead of having them be spread all over. > > Another issue is the one that MAX_ARRAY_LENGTH (also called > MAX_ARRAY_SIZE) is trying to address. This is sort-of a misnomer. It's > not the actual maximum array size (which in fact isn't known the the > library). It's actually the maximum array size that the library is > fairly confident the VM can provide, assuming that enough memory is > actually available. What the heck does that mean? > > The theoretical maximum array size is Integer.MAX_VALUE, since the JLS > and JVMS don't allow anything larger. However, actual VM implementations > will refuse to allocate an array above a certain amount slightly smaller > than that, even if there is enough memory available. In practice, I > believe the values for current Hotspot are Integer.MAX_VALUE-3 or > Integer.MAX_VALUE-2, depending on whether compressed OOPS are in use. > > Why is this significant? Consider the following case, where the capacity > of something is currently Integer.MAX_VALUE-100, and it's filled with > elements. The application performs some operation that requires 50 > elements (minGrowth) be added. A new array could certainly be allocated > with size Integer.MAX_VALUE-50, but typical growth policies for these > kinds of containers want to increase the current capacity by 1.5x or 2x > (prefGrowth). Doing this multiplication would exceed Integer.MAX_VALUE, > so that won't work. Clearly, we need to clamp the capacity somewhere. > > We don't want to clamp the capacity at Integer.MAX_VALUE, because this > allocation would fail on every JVM I'm aware of, even if enough memory > is available. So we don't do that. Instead, we clamp the preferred > growth at some fairly arbitrary value smaller than Integer.MAX_VALUE, > which is here called MAX_ARRAY_LENGTH, and increase the capacity to that > instead. This allows the container's requested allocation to proceed: it > satisfies minGrowth, but it doesn't satisfy prefGrowth. Instead, it > returns a capacity value that's reasonably likely to succeed, given an > unknown JVM implementation limit. > > Recall that the container now has Integer.MAX_VALUE-50 elements and the > capacity is MAX_ARRAY_SIZE, which is currently defined somewhat > arbitrarily at Integer.MAX_VALUE-8. Suppose the application now wants to > add 43 elements. What should happen? > > We could say, this exceeds MAX_ARRAY_LENGTH, so refuse the request and > throw OOME. This is reasonable and consistent in some sense, but perhaps > not in another. Suppose there is sufficient memory, and the JVM does > allow arrays of Integer.MAX_VALUE-7 to be created. Shouldn't we even try? > > That's what hugeLength() does -- it returns a value that attempts an > allocation beyond the max preferential growth, and leaves it up to the > JVM to grant or refuse the request based on its own implementation limits. IIUC what you are saying is that MAX_ARRAY_LENGTH is treated as a soft-limit. A request for prefGrowth won't be allowed to exceed it. But if minGrowth takes the length passed it then the code tries to do the allocation that large anyway. If it succeeds we win, and if we get OOME that is what we would have thrown anyway if we rejected the request as too big. So my misunderstanding in this was that MAX_ARRAY_LENGTH is not attempting to define the actual VM hard limit, just a large value close to that which is expected to always be valid (actual memory permitting). Thanks for the detailed explanation. David ----- > Anyway, this is all quite subtle, and maybe comments in ArraysSupport > don't describe this adequately. But the code that implements this kind > of policy has been copied to different locations around the JDK, and it > uses somewhat different terminology, and it might have slightly > different bugs, but they're all essentially trying to implement this > policy. > > ** > > Several questions could be asked: > > 1) Is this the right policy for implementing growable arrays? > > 2) In cases where a class needs a growable array, can and should it be > refactored to use ArraysSupport.newLength()? > > 3) Does ArraysSupport.newLength() need to be modified to accommodate > needs of additional call sites? > > 4) We might want to consider refactoring PriorityBlockingQueue and > ArrayDeque to use ArraysSupport.newLength, in order to provide a > consistent policy for collections. Other growable-array-based > collections (Vector, ArrayList, PriorityQueue) do already. > > s'marks > > > > > > On 6/1/20 4:47 AM, Jim Laskey wrote: >> Thanks David will run with that, >> >> >>> On May 31, 2020, at 8:34 PM, David Holmes >>> wrote: >>> >>> On 31/05/2020 12:29 am, Jim Laskey wrote: >>>> I'm working through https://bugs.openjdk.java.net/browse/JDK-8230744 >>>> Several classes >>>> throw OutOfMemoryError without message . >>>> I'm wondering why hugeCapacity in >>>> src/jdk.zipfs/share/classes/jdk/nio/zipfs/ByteArrayChannel.java is >>>> defined as >>>> ???? /** >>>> ????? * The maximum size of array to allocate. >>>> ????? * Some VMs reserve some header words in an array. >>>> ????? * Attempts to allocate larger arrays may result in >>>> ????? * OutOfMemoryError: Requested array size exceeds VM limit >>>> ????? */ >>>> ???? private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; >>>> ???? /** >>>> ????? * Increases the capacity to ensure that it can hold at least the >>>> ????? * number of elements specified by the minimum capacity argument. >>>> ????? * >>>> ????? * @param minCapacity the desired minimum capacity >>>> ????? */ >>>> ???? private void grow(int minCapacity) { >>>> ???????? // overflow-conscious code >>>> ???????? int oldCapacity = buf.length; >>>> ???????? int newCapacity = oldCapacity << 1; >>>> ???????? if (newCapacity - minCapacity < 0) >>>> ???????????? newCapacity = minCapacity; >>>> ???????? if (newCapacity - MAX_ARRAY_SIZE > 0) >>>> ???????????? newCapacity = hugeCapacity(minCapacity); >>>> ???????? buf = Arrays.copyOf(buf, newCapacity); >>>> ???? } >>>> ???? private static int hugeCapacity(int minCapacity) { >>>> ???????? if (minCapacity < 0) // overflow >>>> ???????????? throw new OutOfMemoryError(); >>> >>> Not sure how we could have minCapacity < 0 at this point. It should >>> have been checked before the call to grow, and grow will not make it >>> negative. >>> >>>> ???????? return (minCapacity > MAX_ARRAY_SIZE) ? >>>> ???????????? Integer.MAX_VALUE : >>>> ???????????? MAX_ARRAY_SIZE; >>> >>> That's a bug plain and simple. It should never report a size > >>> MAX_ARRAY_SIZE. >>> >>>> ???? } >>>> It just seems that it's pushing the inevitable off to >>>> Arrays.copyOf.? Shouldn't it be: >>>> ???? private static int hugeCapacity(int minCapacity) { >>>> ???????? if (minCapacity < 0 || minCapacity > MAX_ARRAY_SIZE) { >>>> ???????????? throw >>>> ???????????????? new OutOfMemoryError("ByteArrayChannel exceeds >>>> maximum size: " + >>>> ?????????????????????????????????????? MAX_ARRAY_SIZE); >>>> ???????? } >>>> ????????????????? return MAX_ARRAY_SIZE; >>>> ???? } >>> >>> That seems more appropriate to me - modulo the question mark over >>> minCapacity being negative. >>> >>>> Real question: is there some hidden purpose behind this kind of logic? >>> >>> The basic strategy is to double the current capacity unless that will >>> trigger an unnecessary exception, in which case just use the >>> requested capacity, but again watch for the implementation limits. >>> >>> Cheers, >>> David >>> ----- >>> >>>> Cheers, >>>> -- Jim >> From claes.redestad at oracle.com Wed Jun 3 11:54:00 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 3 Jun 2020 13:54:00 +0200 Subject: RFR: 8246451: Reduce overhead of normalizing file paths with trailing slash Message-ID: <3384b137-a6ea-06bf-1c12-a11371e07105@oracle.com> Hi, I'd like to fix a bug in the microbenchmark I pushed for JDK-8246338 where the FileOpen.trailingSlash variant doesn't do what it intends to. There's also a profitable low-hanging optimization down that path to substring rather than take the detour via StringBuilder[1] Bug: https://bugs.openjdk.java.net/browse/JDK-8246451 Webrev: http://cr.openjdk.java.net/~redestad/8246451/open.00/ Testing: tier1 Thanks! /Claes [1] Benchmark Mode Cnt Score Error Units FileOpen.trailingSlash avgt 5 0.054 ? 0.010 us/op FileOpen.trailingSlash:?gc.alloc.rate.norm avgt 5 160.013 ? 0.003 B/op FileOpen.trailingSlash avgt 5 0.036 ? 0.003 us/op FileOpen.trailingSlash:?gc.alloc.rate.norm avgt 5 96.008 ? 0.002 B/op From info at j-kuhn.de Wed Jun 3 13:01:51 2020 From: info at j-kuhn.de (Johannes Kuhn) Date: Wed, 3 Jun 2020 15:01:51 +0200 Subject: JDK-8229959: Current status and thoughts about JDK-8242888 Message-ID: <3a90095d-dcbf-961a-0537-a776d73de982@j-kuhn.de> Hi, half a year ago, I proposed a patch that implements JDK-8229959: Convert proxy class to use constant dynamic [2]. **Background**: java.lang.reflect.Proxy creates a class file at runtime that extends j.l.r.Proxy and implements the passed interfaces. Methods invoked on that proxy are passed to a j.l.r.InvocationHandler.invoke. One of it's argument is a j.l.r.Method. This Method is currently cached in a private static final field in the generated proxy class. Remi Forax suggested to use a constant dynamic instead. Last time I was not sure about the scope of my proposal, and when discussion drifted to related topics, I did not know how to handle that. After some thought, the scope of this is only changing the shape of the generated class file. Those are NOT in the scope: * Changes to a public API. * Changes on how the generated class file is loaded. Especially JDK-8242888: Convert dynamic proxy to hidden classes [3] is not in scope. But see below for some thoughts on it. --------------------- To avoid changing the specification, I decided to put the bootstrap method for the constant dynamic into the generated class file. My first approach was a bootstrap method that is equivalent to this: ??? private static Method $proxy$bootstrap(Lookup l, String ignored, Class jlrMethod, MethodHandle mh) { ??????? return (Method) lookup.revealDirect(mh).reflectAs(jlrMethod); ??? } One of the responses was to measure the impact, and I wrote some JMH tests - which did run that code JITed. It is much more likely that this code is executed in the interpreter through. Brian Goetz then suggested to write a bootstrap method per proxy mehod, and calling Class::getMethod in them. After some tries, I found a way to parameterize it, using MethodType as carrier. This avoids encoding the primitive types as yet an other constant dynamic. Example bootstrap method: ??? private static Method $proxy$bootstrap(Lookup l, String name, Class jlrMethod, Class owner, MethodType mt) { ??????? return owner.getMethod(name, mt.parameterArray()); ??? } This approach seemed to work - until I did look into the interaction with an installed SecurityManager. Class::getMethod can throw a SecurityException if the method belongs to an interface that is in a package not exported by the JDK. In particular, it breaks j.l.i.MethodHandleProxies.wrapperInstanceTarget when a SecurityManager is installed. [4] (A separate change could be made to fix that particular case, but I consider it just to be a symptom.) This breaks because Class::getMethod is called at a later point - when untrusted code is on the stack. ---------------------- Where can this go from here? * Wrap the call to Class::getMethod in an AccessController.doPrivileged block. ? While possible, this would add even more overhead. * Go back to revealDirect/reflectAs. ? This does not work because Lookup::revealDirect will check the package access. * Use hidden classes, as suggested by JDK-8242888. ? A promising approach, but has a few unsolved problems. What about hidden classes? It's possible to pass some ClassData to a hidden class when defining it. Which can be used to pass the Methods. But Hidden Classes need some more requirements, as outlined by Mandy in JDK-8242888: * Serialization: No change necessary. ? Proxy classes (as determined by Proxy.isProxyClass) are written as a proxy descriptor, containing a list of all implemented interfaces. ? Deserialization is fine if Proxy.h can be updated even if the instance is implemented by a hidden class. * A lookup in the same package as the proxy is needed. ? This is a problem for not-yet-existing packages. It's possible to inject a "package-witness" or shim. * Protection domain: Proxy is defined to have a null ProtectionDomain. ? This is currently necessary for Class.getMethod. ? If the methods are passed using ClassData, then that requirement is no longer needed. ? An other option is to grant the package witness a null ProtectionDomain. As conclusion, JDK-8229959 can only be implemented if it either uses AccessController.doPrivileged or use hidden classes with ClassData. - Johannes [1]: https://bugs.openjdk.java.net/browse/JDK-8229959 [2]: https://mail.openjdk.java.net/pipermail/core-libs-dev/2019-November/063595.html [3]: https://bugs.openjdk.java.net/browse/JDK-8242888 [4]: https://gist.github.com/DasBrain/37855876743fed7828c7690603aa3039 From james.laskey at oracle.com Wed Jun 3 13:20:11 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Wed, 3 Jun 2020 10:20:11 -0300 Subject: Sometimes constraints are questionable In-Reply-To: References: <4A5AC9D1-AEB7-47E3-9E25-BF74FE01F3B0@oracle.com> <2F2E688C-51EF-4DAF-826E-70669940538D@oracle.com> Message-ID: <259F0159-0B61-4334-A93A-80EA69DF5660@oracle.com> Stuart (and Martin), I absolutely agree with the reasoning. ArraysSupport.newLength() is also probably easier for the compiler to optimize around (no exception.) I guess the only conflict here is to be meaningfully informative. The bug assigned to me (https://bugs.openjdk.java.net/browse/JDK-8230744 ) was really about not getting clear information when the OOM is generated, Getting an OOM from the bowels of the VM doesn't always give a clear picture (even with a backtrace) when the allocate code is distant from the expression that calculated the new size (VM: I can't do this. You figure it out, Developer: ? ). Needed here is something like: ArraysSupport.newLength(oldLength, minGrowth, perfGrowth, () -> "Queue exceeds implementation limit") There are other JDK attempts using addExact/multiplyExact to catch the error closer to action. Note that there are a few cases, ex. StringJoiner, where the growth is a three way sum where any two args could go negative (overflow) and the third arg bringing it positive again. Based on what I see in the JDK, the world would be well served with an Arrays.grow(array, minGrowth, perfGrowth, () -> "Queue exceeds implementation limit"). This differs Arrays.copyOf in 1) does overflow checks 2) provides meaningful message 3) eliminates a common and often poorly implemented code pattern. The alternative to this is catching the OOM at site to replace with a meaningful message (this of course is brimming with issues.) I will replace sites modified by this bug with ArraysSupport.newLength. Cheers, -- Jim > On Jun 2, 2020, at 7:08 PM, Stuart Marks wrote: > > Hi Jim, > > This was mentioned previously in this thread but not discussed very much. I suggest you take a look at jdk.internal.util.ArraysSupport.newLength(). Ivan Gerasimov and I worked this over fairly closely last year, and I'm pretty sure it does what Martin is saying, which I also think is the right thing. > > The intent is that it be used for things that have growable arrays, where the array might have a larger capacity than the logical number of elements currently stored. Sometimes the array needs to be grown to accommodate an immediate need (minGrowth) but it's desired to grow larger (prefGrowth) in anticipation of future needs. If minGrowth can't be accommodated, it throws OOME, but if prefGrowth can't be accommodated, it might be acceptable to provide a smaller amount of growth. > > (Of course, all this assumes that there is sufficient memory available to allocate the actual array. ArraysSupport.newLength doesn't attempt to ascertain that.) > > One issue is integer wraparound (overflow). This is the primary value that ArraysSupport.newLength provides. It would be good to centralize these computations instead of having them be spread all over. > > Another issue is the one that MAX_ARRAY_LENGTH (also called MAX_ARRAY_SIZE) is trying to address. This is sort-of a misnomer. It's not the actual maximum array size (which in fact isn't known the the library). It's actually the maximum array size that the library is fairly confident the VM can provide, assuming that enough memory is actually available. What the heck does that mean? > > The theoretical maximum array size is Integer.MAX_VALUE, since the JLS and JVMS don't allow anything larger. However, actual VM implementations will refuse to allocate an array above a certain amount slightly smaller than that, even if there is enough memory available. In practice, I believe the values for current Hotspot are Integer.MAX_VALUE-3 or Integer.MAX_VALUE-2, depending on whether compressed OOPS are in use. > > Why is this significant? Consider the following case, where the capacity of something is currently Integer.MAX_VALUE-100, and it's filled with elements. The application performs some operation that requires 50 elements (minGrowth) be added. A new array could certainly be allocated with size Integer.MAX_VALUE-50, but typical growth policies for these kinds of containers want to increase the current capacity by 1.5x or 2x (prefGrowth). Doing this multiplication would exceed Integer.MAX_VALUE, so that won't work. Clearly, we need to clamp the capacity somewhere. > > We don't want to clamp the capacity at Integer.MAX_VALUE, because this allocation would fail on every JVM I'm aware of, even if enough memory is available. So we don't do that. Instead, we clamp the preferred growth at some fairly arbitrary value smaller than Integer.MAX_VALUE, which is here called MAX_ARRAY_LENGTH, and increase the capacity to that instead. This allows the container's requested allocation to proceed: it satisfies minGrowth, but it doesn't satisfy prefGrowth. Instead, it returns a capacity value that's reasonably likely to succeed, given an unknown JVM implementation limit. > > Recall that the container now has Integer.MAX_VALUE-50 elements and the capacity is MAX_ARRAY_SIZE, which is currently defined somewhat arbitrarily at Integer.MAX_VALUE-8. Suppose the application now wants to add 43 elements. What should happen? > > We could say, this exceeds MAX_ARRAY_LENGTH, so refuse the request and throw OOME. This is reasonable and consistent in some sense, but perhaps not in another. Suppose there is sufficient memory, and the JVM does allow arrays of Integer.MAX_VALUE-7 to be created. Shouldn't we even try? > > That's what hugeLength() does -- it returns a value that attempts an allocation beyond the max preferential growth, and leaves it up to the JVM to grant or refuse the request based on its own implementation limits. > > Anyway, this is all quite subtle, and maybe comments in ArraysSupport don't describe this adequately. But the code that implements this kind of policy has been copied to different locations around the JDK, and it uses somewhat different terminology, and it might have slightly different bugs, but they're all essentially trying to implement this policy. > > ** > > Several questions could be asked: > > 1) Is this the right policy for implementing growable arrays? > > 2) In cases where a class needs a growable array, can and should it be refactored to use ArraysSupport.newLength()? > > 3) Does ArraysSupport.newLength() need to be modified to accommodate needs of additional call sites? > > 4) We might want to consider refactoring PriorityBlockingQueue and ArrayDeque to use ArraysSupport.newLength, in order to provide a consistent policy for collections. Other growable-array-based collections (Vector, ArrayList, PriorityQueue) do already. > > s'marks > > > > > > On 6/1/20 4:47 AM, Jim Laskey wrote: >> Thanks David will run with that, >>> On May 31, 2020, at 8:34 PM, David Holmes wrote: >>> >>> On 31/05/2020 12:29 am, Jim Laskey wrote: >>>> I'm working through https://bugs.openjdk.java.net/browse/JDK-8230744 Several classes throw OutOfMemoryError without message . >>>> I'm wondering why hugeCapacity in src/jdk.zipfs/share/classes/jdk/nio/zipfs/ByteArrayChannel.java is defined as >>>> /** >>>> * The maximum size of array to allocate. >>>> * Some VMs reserve some header words in an array. >>>> * Attempts to allocate larger arrays may result in >>>> * OutOfMemoryError: Requested array size exceeds VM limit >>>> */ >>>> private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; >>>> /** >>>> * Increases the capacity to ensure that it can hold at least the >>>> * number of elements specified by the minimum capacity argument. >>>> * >>>> * @param minCapacity the desired minimum capacity >>>> */ >>>> private void grow(int minCapacity) { >>>> // overflow-conscious code >>>> int oldCapacity = buf.length; >>>> int newCapacity = oldCapacity << 1; >>>> if (newCapacity - minCapacity < 0) >>>> newCapacity = minCapacity; >>>> if (newCapacity - MAX_ARRAY_SIZE > 0) >>>> newCapacity = hugeCapacity(minCapacity); >>>> buf = Arrays.copyOf(buf, newCapacity); >>>> } >>>> private static int hugeCapacity(int minCapacity) { >>>> if (minCapacity < 0) // overflow >>>> throw new OutOfMemoryError(); >>> >>> Not sure how we could have minCapacity < 0 at this point. It should have been checked before the call to grow, and grow will not make it negative. >>> >>>> return (minCapacity > MAX_ARRAY_SIZE) ? >>>> Integer.MAX_VALUE : >>>> MAX_ARRAY_SIZE; >>> >>> That's a bug plain and simple. It should never report a size > MAX_ARRAY_SIZE. >>> >>>> } >>>> It just seems that it's pushing the inevitable off to Arrays.copyOf. Shouldn't it be: >>>> private static int hugeCapacity(int minCapacity) { >>>> if (minCapacity < 0 || minCapacity > MAX_ARRAY_SIZE) { >>>> throw >>>> new OutOfMemoryError("ByteArrayChannel exceeds maximum size: " + >>>> MAX_ARRAY_SIZE); >>>> } >>>> return MAX_ARRAY_SIZE; >>>> } >>> >>> That seems more appropriate to me - modulo the question mark over minCapacity being negative. >>> >>>> Real question: is there some hidden purpose behind this kind of logic? >>> >>> The basic strategy is to double the current capacity unless that will trigger an unnecessary exception, in which case just use the requested capacity, but again watch for the implementation limits. >>> >>> Cheers, >>> David >>> ----- >>> >>>> Cheers, >>>> -- Jim From lance.andersen at oracle.com Wed Jun 3 14:24:16 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 3 Jun 2020 10:24:16 -0400 Subject: RFR: 8246451: Reduce overhead of normalizing file paths with trailing slash In-Reply-To: <3384b137-a6ea-06bf-1c12-a11371e07105@oracle.com> References: <3384b137-a6ea-06bf-1c12-a11371e07105@oracle.com> Message-ID: <8648BD54-33E4-4856-9EBE-437E4302461E@oracle.com> Hi Claes, This looks OK :-) Best Lance > On Jun 3, 2020, at 7:54 AM, Claes Redestad wrote: > > Hi, > > I'd like to fix a bug in the microbenchmark I pushed for JDK-8246338 > where the FileOpen.trailingSlash variant doesn't do what it intends to. > > There's also a profitable low-hanging optimization down that path to > substring rather than take the detour via StringBuilder[1] > > Bug: https://bugs.openjdk.java.net/browse/JDK-8246451 > Webrev: http://cr.openjdk.java.net/~redestad/8246451/open.00/ > > Testing: tier1 > > Thanks! > > /Claes > > [1] > Benchmark Mode Cnt Score Error Units > FileOpen.trailingSlash avgt 5 0.054 ? 0.010 us/op > FileOpen.trailingSlash:?gc.alloc.rate.norm avgt 5 160.013 ? 0.003 B/op > > FileOpen.trailingSlash avgt 5 0.036 ? 0.003 us/op > FileOpen.trailingSlash:?gc.alloc.rate.norm avgt 5 96.008 ? 0.002 B/op Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From james.laskey at oracle.com Wed Jun 3 14:24:15 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Wed, 3 Jun 2020 11:24:15 -0300 Subject: RFR: JDK-8230744 Several classes throw OutOfMemoryError without message In-Reply-To: References: <045D0E84-60E4-491B-9A31-7B05043FFE3B@oracle.com> <78cf935b-d7f3-342a-6ed1-a4a86becb727@oracle.com> Message-ID: <6D4D4663-BB0E-466D-BBA3-9D39556B34BC@oracle.com> It's not the goal or role of this bug to fix the wrongs of the past, merely add error messages to the exceptions. I raised the discussion as an issue. Clearly there is a correct path to follow. If you think more effort is required then file a bug. :-) Cheers, -- Jim > On Jun 2, 2020, at 7:13 PM, Stuart Marks wrote: > > > > On 6/2/20 6:52 AM, Jim Laskey wrote: >> Revised to reflect requested changes. >> http://cr.openjdk.java.net/~jlaskey/8230744/webrev-01/index.html > > On this, if all you're doing is changing exception messages, then I don't care very much, modulo concerns about wording from others. If you start to get into changing the growth logic (like the Math.addExact stuff) then please see my message on the related thread, "Sometimes constraints are questionable." > > Thanks, > > s'marks From maurizio.cimadamore at oracle.com Wed Jun 3 14:25:03 2020 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Wed, 3 Jun 2020 15:25:03 +0100 Subject: RFR 8246095: Tweaks to memory access API In-Reply-To: <21b8eeff-7c72-7c89-b200-050fc0096414@oracle.com> References: <21b8eeff-7c72-7c89-b200-050fc0096414@oracle.com> Message-ID: <13f1231c-7dd3-ef6c-1528-535d7c850aef@oracle.com> Please review this delta patch; I realized that the previously submitted patch had few issues: * copyright header on TestAdaptVarHandle benchmark was wrong * stream test for segment spliterator was not updated to include changes to copyFrom API * I've also incorporated a small javadoc tweak to MemoryHandles::asUnsigned that was suggested in the CSR process Here's the patch (if a webrev if preferred, I can also generate that): diff -r 6b40a32ede25 src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemoryHandles.java --- a/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemoryHandles.java Wed Jun 03 13:57:18 2020 +0100 +++ b/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemoryHandles.java Wed Jun 03 15:22:03 2020 +0100 @@ -357,9 +357,14 @@ ????? * primitive data types as if they were a wider signed primitive type. For ????? * example, it is often convenient to model an unsigned short as a ????? * Java {@code int} to avoid dealing with negative values, which would be -???? * the case if modeled as a Java {@code short}. The returned var handle -???? * converts to and from wider primitive types, to a more narrow possibly -???? * unsigned primitive type. +???? * the case if modeled as a Java {@code short}. This is illustrated in the following example: +???? *
{@code
+??? MemorySegment segment = MemorySegment.allocateNative(2);
+??? VarHandle SHORT_VH = MemoryLayouts.JAVA_SHORT.varHandle(short.class);
+??? VarHandle INT_VH = MemoryHandles.asUnsigned(SHORT_VH, int.class);
+??? SHORT_VH.set(segment.baseAddress(), (short)-1);
+??? INT_VH.get(segment.baseAddress()); // returns 65535
+???? * }
????? *

????? * When calling e.g. {@link VarHandle#set(Object...)} on the resulting var ????? * handle, the incoming value (of type {@code adaptedType}) is converted by a diff -r 6b40a32ede25 test/jdk/java/util/stream/test/org/openjdk/tests/java/util/stream/SegmentTestDataProvider.java --- a/test/jdk/java/util/stream/test/org/openjdk/tests/java/util/stream/SegmentTestDataProvider.java Wed Jun 03 13:57:18 2020 +0100 +++ b/test/jdk/java/util/stream/test/org/openjdk/tests/java/util/stream/SegmentTestDataProvider.java Wed Jun 03 15:22:03 2020 +0100 @@ -23,7 +23,6 @@ ?package org.openjdk.tests.java.util.stream; -import jdk.incubator.foreign.MemoryAddress; ?import jdk.incubator.foreign.MemoryLayout; ?import jdk.incubator.foreign.MemoryLayouts; ?import jdk.incubator.foreign.MemorySegment; @@ -116,9 +115,9 @@ ???? static Consumer segmentCopier(Consumer input) { ???????? return segment -> { -??????????? MemorySegment copy = MemorySegment.ofArray(new byte[(int)segment.byteSize()]); -??????????? MemoryAddress.copy(segment.baseAddress(), copy.baseAddress(), segment.byteSize()); -??????????? input.accept(copy); +??????????? MemorySegment dest = MemorySegment.ofArray(new byte[(int)segment.byteSize()]); +??????????? dest.copyFrom(segment); +??????????? input.accept(dest); ???????? }; ???? } diff -r 6b40a32ede25 test/micro/org/openjdk/bench/jdk/incubator/foreign/TestAdaptVarHandles.java --- a/test/micro/org/openjdk/bench/jdk/incubator/foreign/TestAdaptVarHandles.java Wed Jun 03 13:57:18 2020 +0100 +++ b/test/micro/org/openjdk/bench/jdk/incubator/foreign/TestAdaptVarHandles.java Wed Jun 03 15:22:03 2020 +0100 @@ -1,25 +1,27 @@ ?/* - * Copyright (c) 2020 Oracle and/or its affiliates. All rights reserved. - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + *? Copyright (c) 2020, Oracle and/or its affiliates. All rights reserved. + *? DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. ? * - * This code is free software; you can redistribute it and/or modify it - * under the terms of the GNU General Public License version 2 only, as - * published by the Free Software Foundation. + *? This code is free software; you can redistribute it and/or modify it + *? under the terms of the GNU General Public License version 2 only, as + *? published by the Free Software Foundation. ? * - * This code is distributed in the hope that it will be useful, but WITHOUT - * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or - * FITNESS FOR A PARTICULAR PURPOSE.? See the GNU General Public License - * version 2 for more details (a copy is included in the LICENSE file that - * accompanied this code). + *? This code is distributed in the hope that it will be useful, but WITHOUT + *? ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + *? FITNESS FOR A PARTICULAR PURPOSE.? See the GNU General Public License + *? version 2 for more details (a copy is included in the LICENSE file that + *? accompanied this code). ? * - * You should have received a copy of the GNU General Public License version Maurizio On 28/05/2020 22:25, Maurizio Cimadamore wrote: > Hi, > this followup change includes a number of tweaks that have been added > to the memory access API while we were in the process of integrating > it. Most of them have been contributed by Chris (thanks!), and are all > listed in great details in the CSR below: > > https://bugs.openjdk.java.net/browse/JDK-8246096 > > Implementation-wise it should be all relatively straightforward - > apart for the bit of surgery on lambda forms which was required to add > a new kind of lambda forms to 'collect' return values. For now this > method handle adaptation is package private, and can only be used by > MemoryHandles::filterValues - but, if people find it useful, we might > consider adding it to the MethodHandle standard combinator toolbox. > > Cheers > Maurizio > > > Webrev: > > http://cr.openjdk.java.net/~mcimadamore/8246095_v1/webrev/ > > Javadoc: > > http://cr.openjdk.java.net/~mcimadamore/8246095_v1/javadoc/jdk/incubator/foreign/package-summary.html > > > Specdiff: > > http://cr.openjdk.java.net/~mcimadamore/8246095_v1/specdiff/overview-summary.html > > From claes.redestad at oracle.com Wed Jun 3 14:47:05 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 3 Jun 2020 16:47:05 +0200 Subject: RFR: 8246451: Reduce overhead of normalizing file paths with trailing slash In-Reply-To: <8648BD54-33E4-4856-9EBE-437E4302461E@oracle.com> References: <3384b137-a6ea-06bf-1c12-a11371e07105@oracle.com> <8648BD54-33E4-4856-9EBE-437E4302461E@oracle.com> Message-ID: <43462332-7a2d-8a59-6425-8e0485599eba@oracle.com> On 2020-06-03 16:24, Lance Andersen wrote: > Hi Claes, > > This looks OK :-) Thank you, Lance! /Claes > > Best > Lance > >> On Jun 3, 2020, at 7:54 AM, Claes Redestad > > wrote: >> >> Hi, >> >> I'd like to fix a bug in the microbenchmark I pushed for JDK-8246338 >> where the FileOpen.trailingSlash variant doesn't do what it intends to. >> >> There's also a profitable low-hanging optimization down that path to >> substring rather than take the detour via StringBuilder[1] >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8246451 >> Webrev: http://cr.openjdk.java.net/~redestad/8246451/open.00/ >> >> Testing: tier1 >> >> Thanks! >> >> /Claes >> >> [1] >> Benchmark ???????????????????????????????????????????????Mode ?Cnt >> Score ????Error ??Units >> FileOpen.trailingSlash ??????????????????????????????????avgt ???5 >> 0.054 ? ??0.010 ??us/op >> FileOpen.trailingSlash:?gc.alloc.rate.norm ??????????????avgt ???5 >> 160.013 ? ??0.003 ???B/op >> >> FileOpen.trailingSlash ??????????????????????????????????avgt ???5 >> 0.036 ? ??0.003 ??us/op >> FileOpen.trailingSlash:?gc.alloc.rate.norm ??????????????avgt ???5 >> 96.008 ? ??0.002 ???B/op > > > > Lance Andersen| > Principal Member of Technical Staff | +1.781.442.2037 > Oracle?Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From chris.hegarty at oracle.com Wed Jun 3 14:51:07 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 3 Jun 2020 15:51:07 +0100 Subject: RFR 8246095: Tweaks to memory access API In-Reply-To: <13f1231c-7dd3-ef6c-1528-535d7c850aef@oracle.com> References: <21b8eeff-7c72-7c89-b200-050fc0096414@oracle.com> <13f1231c-7dd3-ef6c-1528-535d7c850aef@oracle.com> Message-ID: <6A3942F8-54D6-4D11-B303-C268DC397B44@oracle.com> LGTM. The illustrative example in asUnsigned is nice ( I should have thought of similar, D?oh! ) -Chris. > On 3 Jun 2020, at 15:25, Maurizio Cimadamore wrote: > > Please review this delta patch; I realized that the previously submitted patch had few issues: > > * copyright header on TestAdaptVarHandle benchmark was wrong > * stream test for segment spliterator was not updated to include changes to copyFrom API > * I've also incorporated a small javadoc tweak to MemoryHandles::asUnsigned that was suggested in the CSR process > > Here's the patch (if a webrev if preferred, I can also generate that): > > diff -r 6b40a32ede25 src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemoryHandles.java > --- a/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemoryHandles.java Wed Jun 03 13:57:18 2020 +0100 > +++ b/src/jdk.incubator.foreign/share/classes/jdk/incubator/foreign/MemoryHandles.java Wed Jun 03 15:22:03 2020 +0100 > @@ -357,9 +357,14 @@ > * primitive data types as if they were a wider signed primitive type. For > * example, it is often convenient to model an unsigned short as a > * Java {@code int} to avoid dealing with negative values, which would be > - * the case if modeled as a Java {@code short}. The returned var handle > - * converts to and from wider primitive types, to a more narrow possibly > - * unsigned primitive type. > + * the case if modeled as a Java {@code short}. This is illustrated in the following example: > + *

{@code
> +    MemorySegment segment = MemorySegment.allocateNative(2);
> +    VarHandle SHORT_VH = MemoryLayouts.JAVA_SHORT.varHandle(short.class);
> +    VarHandle INT_VH = MemoryHandles.asUnsigned(SHORT_VH, int.class);
> +    SHORT_VH.set(segment.baseAddress(), (short)-1);
> +    INT_VH.get(segment.baseAddress()); // returns 65535
> +     * }
> *

> * When calling e.g. {@link VarHandle#set(Object...)} on the resulting var > * handle, the incoming value (of type {@code adaptedType}) is converted by a > diff -r 6b40a32ede25 test/jdk/java/util/stream/test/org/openjdk/tests/java/util/stream/SegmentTestDataProvider.java > --- a/test/jdk/java/util/stream/test/org/openjdk/tests/java/util/stream/SegmentTestDataProvider.java Wed Jun 03 13:57:18 2020 +0100 > +++ b/test/jdk/java/util/stream/test/org/openjdk/tests/java/util/stream/SegmentTestDataProvider.java Wed Jun 03 15:22:03 2020 +0100 > @@ -23,7 +23,6 @@ > > package org.openjdk.tests.java.util.stream; > > -import jdk.incubator.foreign.MemoryAddress; > import jdk.incubator.foreign.MemoryLayout; > import jdk.incubator.foreign.MemoryLayouts; > import jdk.incubator.foreign.MemorySegment; > @@ -116,9 +115,9 @@ > > static Consumer segmentCopier(Consumer input) { > return segment -> { > - MemorySegment copy = MemorySegment.ofArray(new byte[(int)segment.byteSize()]); > - MemoryAddress.copy(segment.baseAddress(), copy.baseAddress(), segment.byteSize()); > - input.accept(copy); > + MemorySegment dest = MemorySegment.ofArray(new byte[(int)segment.byteSize()]); > + dest.copyFrom(segment); > + input.accept(dest); > }; > } > > diff -r 6b40a32ede25 test/micro/org/openjdk/bench/jdk/incubator/foreign/TestAdaptVarHandles.java > --- a/test/micro/org/openjdk/bench/jdk/incubator/foreign/TestAdaptVarHandles.java Wed Jun 03 13:57:18 2020 +0100 > +++ b/test/micro/org/openjdk/bench/jdk/incubator/foreign/TestAdaptVarHandles.java Wed Jun 03 15:22:03 2020 +0100 > @@ -1,25 +1,27 @@ > /* > - * Copyright (c) 2020 Oracle and/or its affiliates. All rights reserved. > - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > + * Copyright (c) 2020, Oracle and/or its affiliates. All rights reserved. > + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > - * This code is free software; you can redistribute it and/or modify it > - * under the terms of the GNU General Public License version 2 only, as > - * published by the Free Software Foundation. > + * This code is free software; you can redistribute it and/or modify it > + * under the terms of the GNU General Public License version 2 only, as > + * published by the Free Software Foundation. > * > - * This code is distributed in the hope that it will be useful, but WITHOUT > - * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or > - * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License > - * version 2 for more details (a copy is included in the LICENSE file that > - * accompanied this code). > + * This code is distributed in the hope that it will be useful, but WITHOUT > + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or > + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License > + * version 2 for more details (a copy is included in the LICENSE file that > + * accompanied this code). > * > - * You should have received a copy of the GNU General Public License version > > Maurizio > > On 28/05/2020 22:25, Maurizio Cimadamore wrote: >> Hi, >> this followup change includes a number of tweaks that have been added to the memory access API while we were in the process of integrating it. Most of them have been contributed by Chris (thanks!), and are all listed in great details in the CSR below: >> >> https://bugs.openjdk.java.net/browse/JDK-8246096 >> >> Implementation-wise it should be all relatively straightforward - apart for the bit of surgery on lambda forms which was required to add a new kind of lambda forms to 'collect' return values. For now this method handle adaptation is package private, and can only be used by MemoryHandles::filterValues - but, if people find it useful, we might consider adding it to the MethodHandle standard combinator toolbox. >> >> Cheers >> Maurizio >> >> >> Webrev: >> >> http://cr.openjdk.java.net/~mcimadamore/8246095_v1/webrev/ >> >> Javadoc: >> >> http://cr.openjdk.java.net/~mcimadamore/8246095_v1/javadoc/jdk/incubator/foreign/package-summary.html >> >> Specdiff: >> >> http://cr.openjdk.java.net/~mcimadamore/8246095_v1/specdiff/overview-summary.html >> From jason_mehrens at hotmail.com Wed Jun 3 15:10:51 2020 From: jason_mehrens at hotmail.com (Jason Mehrens) Date: Wed, 3 Jun 2020 15:10:51 +0000 Subject: [PATCH] 8245694 : java.util.Properties.entrySet() does not override java.lang.Object methods In-Reply-To: References: Message-ID: Yu Li, You should consider changing the equals implementation to include an identity self check. ==== public boolean equals(Object o) { return this == o || entrySet.equals(o); } This is consistent with the collection wrapper classes in j.u.Collections. Jason ________________________________________ From: core-libs-dev on behalf of Lisa Li Sent: Saturday, May 30, 2020 9:28 AM To: core-libs-dev at openjdk.java.net Subject: [PATCH] 8245694 : java.util.Properties.entrySet() does not override java.lang.Object methods Hi, Please help review the fix for JDK-8245694 . And this is my very first patch submission. I know it's not perfect. *BUG DESCRIPTION*: JDK-8245694 : java.util.Properties.entrySet() does not override java.lang.Object methods since java 9. *PROPOSED FIX*: Add delegating overrides for equals(), hashCode(), and toString(). *PATCH*: # HG changeset patch # User yuli # Date 1590841711 -28800 # Sat May 30 20:28:31 2020 +0800 # Node ID 7dc946e5b5863291fa070bfdbdce48aefbe87b7b # Parent 8e28aae5068069e853673148e4d3f44cb50350a7 8245694: java.util.Properties.entrySet() does not override Object methods Summary: Add missing override methods Contributed-by: Yu Li diff --git a/src/java.base/share/classes/java/util/Properties.java b/src/java.base/share/classes/java/util/Properties.java --- a/src/java.base/share/classes/java/util/Properties.java +++ b/src/java.base/share/classes/java/util/Properties.java @@ -1414,6 +1414,21 @@ return entrySet.retainAll(c); } + @Override + public boolean equals(Object o) { + return entrySet.equals(o); + } + + @Override + public int hashCode() { + return entrySet.hashCode(); + } + + @Override + public String toString() { + return entrySet.toString(); + } + @Override public Iterator> iterator() { return entrySet.iterator(); diff --git a/test/jdk/java/util/Properties/PropertiesEntrySetTest.java b/test/jdk/java/util/Properties/PropertiesEntrySetTest.java new file mode 100644 --- /dev/null +++ b/test/jdk/java/util/Properties/PropertiesEntrySetTest.java @@ -0,0 +1,90 @@ +/* + * Copyright (c) 2018, 2019, Oracle and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + */ + +/* + * @test + * @summary tests the entrySet() method of Properties class + * @author Yu Li + * @bug 8245694 + */ + +import java.util.Map; +import java.util.Properties; +import java.util.Set; + +public class PropertiesEntrySetTest { + + public static void main(String[] args) { + + testEntrySet(); + + } + + private static void assertTrue(boolean value) { + if (!value) { + throw new RuntimeException("Failure in Properties testing."); + } + } + + private static void assertFalse(boolean value) { + if (value) { + throw new RuntimeException("Failure in Properties testing."); + } + } + + private static void testEntrySet() { + Properties a = new Properties(); + Set> aEntrySet = a.entrySet(); + assertFalse(aEntrySet.equals(null)); + assertTrue(aEntrySet.equals(aEntrySet)); + + Properties b = new Properties(); + Set> bEntrySet = b.entrySet(); + assertTrue(aEntrySet.equals(bEntrySet)); + assertTrue(aEntrySet.hashCode() == bEntrySet.hashCode()); + + a.setProperty("p1", "1"); + assertFalse(aEntrySet.equals(bEntrySet)); + assertFalse(bEntrySet.equals(aEntrySet)); + assertFalse(aEntrySet.hashCode() == bEntrySet.hashCode()); + + b.setProperty("p1", "1"); + assertTrue(aEntrySet.equals(bEntrySet)); + assertTrue(aEntrySet.hashCode() == bEntrySet.hashCode()); + + Properties c = new Properties(); + c.setProperty("p1", "2"); + Set> cEntrySet = c.entrySet(); + assertFalse(cEntrySet.equals(bEntrySet)); + assertFalse(cEntrySet.hashCode() == bEntrySet.hashCode()); + assertFalse(cEntrySet.equals(aEntrySet)); + assertFalse(cEntrySet.hashCode() == aEntrySet.hashCode()); + + a.setProperty("p2", "2"); + Properties d = new Properties(); + d.setProperty("p2", "2"); + d.setProperty("p1", "1"); + Set> dEntrySet = d.entrySet(); + assertTrue(dEntrySet.equals(aEntrySet)); + } +} Best regards, Yu Li Attachments area From stuart.marks at oracle.com Wed Jun 3 16:55:56 2020 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 3 Jun 2020 09:55:56 -0700 Subject: Sometimes constraints are questionable In-Reply-To: <43c1a46e-eaa1-c079-b400-7280929acb5f@oracle.com> References: <4A5AC9D1-AEB7-47E3-9E25-BF74FE01F3B0@oracle.com> <2F2E688C-51EF-4DAF-826E-70669940538D@oracle.com> <43c1a46e-eaa1-c079-b400-7280929acb5f@oracle.com> Message-ID: <6e3ea97e-e37b-d932-e27b-28a579f7ba7a@oracle.com> On 6/2/20 11:49 PM, David Holmes wrote: > IIUC what you are saying is that MAX_ARRAY_LENGTH is treated as a soft-limit. A > request for prefGrowth won't be allowed to exceed it. But if minGrowth takes the > length passed it then the code tries to do the allocation that large anyway. If > it succeeds we win, and if we get OOME that is what we would have thrown anyway > if we rejected the request as too big. Yes, that's a good, succinct summary. > So my misunderstanding in this was that MAX_ARRAY_LENGTH is not attempting to > define the actual VM hard limit, just a large value close to that which is > expected to always be valid (actual memory permitting). Exactly. And sorry for this misunderstanding -- revisiting this I see that the name MAX_ARRAY_LENGTH is quite misleading. Maybe I should have pushed harder to change this when Ivan and I went through this code last year. We had improved some things, I think, but we had already gone through half a dozen rounds of review and I think we were both getting tired. :-) s'marks From Alan.Bateman at oracle.com Wed Jun 3 17:13:53 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 3 Jun 2020 18:13:53 +0100 Subject: ThreadGroup.enumerate/activeCount - any good reason not to deprecate? Message-ID: Does ThreadGroup have any fans out there? I'm looking to deprecate its enumerate, activeXXXCount, and list methods. Also Thread.enumerate and activeCount. I'm trying to identify if there are any important usages of these APIs that would suggest they shouldn't be deprecated. A static analysis of the artifacts in Maven isn't coming up with much. One reason for deprecating these methods is that they are inheriting racy. The javadoc has recommended for many releases that they be only used for debugging and monitoring purposes. The javadoc has warned, since at least JDK 1.1, that the enumerate methods silently ignore the case where the array isn't large enough. Another reason for deprecating these methods is that they are obsolete, even for monitoring purposes. Java 5 added ThreadMXBean, ThreadInfo and better ways to get information on live threads. Finally, there is "threads" array keeping a reference to every Thread in the group. That array is problematic for several performance reasons, and ultimately it would be nice to have that go away. Making that go away will require re-implementing a number of JVMTI functions that operate on threads groups but there are alternative ways to implement those (Serguei Spitsyn did an initial prototype to check this). For now, I'm just trying to see if there are any important usages of these APIs that I may have missed. -Alan. From stuart.marks at oracle.com Wed Jun 3 17:36:40 2020 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 3 Jun 2020 10:36:40 -0700 Subject: Sometimes constraints are questionable In-Reply-To: <259F0159-0B61-4334-A93A-80EA69DF5660@oracle.com> References: <4A5AC9D1-AEB7-47E3-9E25-BF74FE01F3B0@oracle.com> <2F2E688C-51EF-4DAF-826E-70669940538D@oracle.com> <259F0159-0B61-4334-A93A-80EA69DF5660@oracle.com> Message-ID: <23efc3c4-d711-8750-8fdb-ee9ac8553952@oracle.com> On 6/3/20 6:20 AM, Jim Laskey wrote: > I absolutely agree with the reasoning. ArraysSupport.newLength() is also > probably easier for the compiler to optimize around (no exception.) > > I guess the only conflict here is to be meaningfully informative. The bug > assigned to me (https://bugs.openjdk.java.net/browse/JDK-8230744) was really > about not getting clear information when the OOM is generated, Getting an OOM > from the bowels of the VM doesn't always give a clear picture (even with a > backtrace) when the allocate code is distant from the expression that calculated > the new size (VM: I can't do this. You figure it out, Developer: ? ). I'm a little skeptical of having the libraries try to make OOME messages too informative. If the system really is out of memory, then trying to construct an informative message is also likely to fail, adding to the confusion. > Needed here is something like: > > ArraysSupport.newLength(oldLength, minGrowth, perfGrowth, () -> "Queue exceeds > implementation limit") I'm not sure I like the idea of a lambda here. Maybe if the message arg were a string literal, it would be ok, but it seems likely that somebody would use a lambda and capture something (which would itself allocate memory when the lambda is encountered), and the evaluation of the lambda is likely to do things like string formatting, which does even more allocation. > There are other JDK attempts using addExact/multiplyExact to catch the error > closer to action. Yeah these cases probably deserve some inspection. For common growth factors (1.5x and 2x), ArraysSupport.newLength avoids need for these by having the *growth amount* passed as an argument and then doing the integer wraparound check explicitly, e.g., newCapacity = ArraysSupport.newLength(oldCapacity, needed, oldCapacity >> 1); > Note that there are a few cases, ex. StringJoiner, where the growth is a three > way sum where any two args could go negative (overflow) and the third arg > bringing it positive again. We might need to consider StringJoiner separately, if its computation is indeed more complex than is currently supported by AS.newLength(). > Based on what I see in the JDK, the world would be well served with an > Arrays.grow(array,?minGrowth, perfGrowth, () -> "Queue exceeds implementation > limit"). This differs Arrays.copyOf in 1) does overflow checks 2) provides > meaningful message 3) eliminates a common and often poorly implemented code > pattern. The alternative to this is catching the OOM at site to replace with a > meaningful message (this of course is brimming with issues.) > > I will replace sites modified by this bug with ArraysSupport.newLength. Yes, I think it would be an improvement if we replaced several variations of size computations and overflow checking scattered around the libraries, and unified them to use ArraysSupport.newLength. I guess we should figure out what cases we want to support, and whether there are particular sites that would benefit from more detailed error messages. As I said above, I'm skeptical, but examining particular cases is always enlightening. Even without trying to support detailed error message, we should at least make sure that error messages are reasonable for the different cases we know about. Here are the ones I know about: 1) Actual shortage of heap space. Hotspot generates this: OutOfMemoryError: Java heap space 2) Requested array size exceeds implementation limit. Hotspot generates this: OutOfMemoryError: Requested array size exceeds VM limit 3) Integer wraparound/overflow during size computation. AS.newLength generates this: OutOfMemoryError: Required array length too large For (1) I don't think the JVM is in any position to provide more details. For (2) this is about array length, not heap, and it happens only for a very few length values. This seems sufficiently rare that IMO it's not worth expending much effort. (3) is the only case generated by the library. In fact, AS.hugeLength() has oldLength and minGrowth parameters, which seems like enough detail already. These could reasonably be formatted into the error message, something like: private static int hugeLength(int oldLength, int minGrowth) { int minLength = oldLength + minGrowth; if (minLength < 0) { // overflow throw new OutOfMemoryError( String.format("Required array length %d + %d too large", oldLength, minGrowth)); } Would this help? If this were added, would it be sufficient to allow various use sites to convert to use AS.newLength? (Except possibly StringJoiner.) ** There's a related issue mentioned in the bug reports (JDK-8230744 and JDK-8210577) which is that OOME is thrown not only when the system is out of memory, but also when some implementation limit is exceeded, whether that limit is in the JVM or in the library. Maybe we should update the documentation of java.lang.OutOfMemoryError to mention this. s'marks > > Cheers, > > -- Jim > > > > >> On Jun 2, 2020, at 7:08 PM, Stuart Marks > > wrote: >> >> Hi Jim, >> >> This was mentioned previously in this thread but not discussed very much. I >> suggest you take a look at jdk.internal.util.ArraysSupport.newLength(). Ivan >> Gerasimov and I worked this over fairly closely last year, and I'm pretty sure >> it does what Martin is saying, which I also think is the right thing. >> >> The intent is that it be used for things that have growable arrays, where the >> array might have a larger capacity than the logical number of elements >> currently stored. Sometimes the array needs to be grown to accommodate an >> immediate need (minGrowth) but it's desired to grow larger (prefGrowth) in >> anticipation of future needs. If minGrowth can't be accommodated, it throws >> OOME, but if prefGrowth can't be accommodated, it might be acceptable to >> provide a smaller amount of growth. >> >> (Of course, all this assumes that there is sufficient memory available to >> allocate the actual array. ArraysSupport.newLength doesn't attempt to >> ascertain that.) >> >> One issue is integer wraparound (overflow). This is the primary value that >> ArraysSupport.newLength provides. It would be good to centralize these >> computations instead of having them be spread all over. >> >> Another issue is the one that MAX_ARRAY_LENGTH (also called MAX_ARRAY_SIZE) is >> trying to address. This is sort-of a misnomer. It's not the actual maximum >> array size (which in fact isn't known the the library). It's actually the >> maximum array size that the library is fairly confident the VM can provide, >> assuming that enough memory is actually available. What the heck does that mean? >> >> The theoretical maximum array size is Integer.MAX_VALUE, since the JLS and >> JVMS don't allow anything larger. However, actual VM implementations will >> refuse to allocate an array above a certain amount slightly smaller than that, >> even if there is enough memory available. In practice, I believe the values >> for current Hotspot are Integer.MAX_VALUE-3 or Integer.MAX_VALUE-2, depending >> on whether compressed OOPS are in use. >> >> Why is this significant? Consider the following case, where the capacity of >> something is currently Integer.MAX_VALUE-100, and it's filled with elements. >> The application performs some operation that requires 50 elements (minGrowth) >> be added. A new array could certainly be allocated with size >> Integer.MAX_VALUE-50, but typical growth policies for these kinds of >> containers want to increase the current capacity by 1.5x or 2x (prefGrowth). >> Doing this multiplication would exceed Integer.MAX_VALUE, so that won't work. >> Clearly, we need to clamp the capacity somewhere. >> >> We don't want to clamp the capacity at Integer.MAX_VALUE, because this >> allocation would fail on every JVM I'm aware of, even if enough memory is >> available. So we don't do that. Instead, we clamp the preferred growth at some >> fairly arbitrary value smaller than Integer.MAX_VALUE, which is here called >> MAX_ARRAY_LENGTH, and increase the capacity to that instead. This allows the >> container's requested allocation to proceed: it satisfies minGrowth, but it >> doesn't satisfy prefGrowth. Instead, it returns a capacity value that's >> reasonably likely to succeed, given an unknown JVM implementation limit. >> >> Recall that the container now has Integer.MAX_VALUE-50 elements and the >> capacity is MAX_ARRAY_SIZE, which is currently defined somewhat arbitrarily at >> Integer.MAX_VALUE-8. Suppose the application now wants to add 43 elements. >> What should happen? >> >> We could say, this exceeds MAX_ARRAY_LENGTH, so refuse the request and throw >> OOME. This is reasonable and consistent in some sense, but perhaps not in >> another. Suppose there is sufficient memory, and the JVM does allow arrays of >> Integer.MAX_VALUE-7 to be created. Shouldn't we even try? >> >> That's what hugeLength() does -- it returns a value that attempts an >> allocation beyond the max preferential growth, and leaves it up to the JVM to >> grant or refuse the request based on its own implementation limits. >> >> Anyway, this is all quite subtle, and maybe comments in ArraysSupport don't >> describe this adequately. But the code that implements this kind of policy has >> been copied to different locations around the JDK, and it uses somewhat >> different terminology, and it might have slightly different bugs, but they're >> all essentially trying to implement this policy. >> >> ** >> >> Several questions could be asked: >> >> 1) Is this the right policy for implementing growable arrays? >> >> 2) In cases where a class needs a growable array, can and should it be >> refactored to use ArraysSupport.newLength()? >> >> 3) Does ArraysSupport.newLength() need to be modified to accommodate needs of >> additional call sites? >> >> 4) We might want to consider refactoring PriorityBlockingQueue and ArrayDeque >> to use ArraysSupport.newLength, in order to provide a consistent policy for >> collections. Other growable-array-based collections (Vector, ArrayList, >> PriorityQueue) do already. >> >> s'marks >> >> >> >> >> >> On 6/1/20 4:47 AM, Jim Laskey wrote: >>> Thanks David will run with that, >>>> On May 31, 2020, at 8:34 PM, David Holmes >>> > wrote: >>>> >>>> On 31/05/2020 12:29 am, Jim Laskey wrote: >>>>> I'm working through https://bugs.openjdk.java.net/browse/JDK-8230744 >>>>> Several classes throw >>>>> OutOfMemoryError without message . >>>>> I'm wondering why hugeCapacity in >>>>> src/jdk.zipfs/share/classes/jdk/nio/zipfs/ByteArrayChannel.java is defined as >>>>> ????/** >>>>> ?????* The maximum size of array to allocate. >>>>> ?????* Some VMs reserve some header words in an array. >>>>> ?????* Attempts to allocate larger arrays may result in >>>>> ?????* OutOfMemoryError: Requested array size exceeds VM limit >>>>> ?????*/ >>>>> ????private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; >>>>> ????/** >>>>> ?????* Increases the capacity to ensure that it can hold at least the >>>>> ?????* number of elements specified by the minimum capacity argument. >>>>> ?????* >>>>> ?????* @param minCapacity the desired minimum capacity >>>>> ?????*/ >>>>> ????private void grow(int minCapacity) { >>>>> ????????// overflow-conscious code >>>>> ????????int oldCapacity = buf.length; >>>>> ????????int newCapacity = oldCapacity << 1; >>>>> ????????if (newCapacity - minCapacity < 0) >>>>> ????????????newCapacity = minCapacity; >>>>> ????????if (newCapacity - MAX_ARRAY_SIZE > 0) >>>>> ????????????newCapacity = hugeCapacity(minCapacity); >>>>> ????????buf = Arrays.copyOf(buf, newCapacity); >>>>> ????} >>>>> ????private static int hugeCapacity(int minCapacity) { >>>>> ????????if (minCapacity < 0) // overflow >>>>> ????????????throw new OutOfMemoryError(); >>>> >>>> Not sure how we could have minCapacity < 0 at this point. It should have >>>> been checked before the call to grow, and grow will not make it negative. >>>> >>>>> ????????return (minCapacity > MAX_ARRAY_SIZE) ? >>>>> ????????????Integer.MAX_VALUE : >>>>> ????????????MAX_ARRAY_SIZE; >>>> >>>> That's a bug plain and simple. It should never report a size > MAX_ARRAY_SIZE. >>>> >>>>> ????} >>>>> It just seems that it's pushing the inevitable off to Arrays.copyOf. >>>>> ?Shouldn't it be: >>>>> ????private static int hugeCapacity(int minCapacity) { >>>>> ????????if (minCapacity < 0 || minCapacity > MAX_ARRAY_SIZE) { >>>>> ????????????throw >>>>> ????????????????new OutOfMemoryError("ByteArrayChannel exceeds maximum >>>>> size: " + >>>>> ??????????????????????????????????????MAX_ARRAY_SIZE); >>>>> ????????} >>>>> ?????????????????return MAX_ARRAY_SIZE; >>>>> ????} >>>> >>>> That seems more appropriate to me - modulo the question mark over >>>> minCapacity being negative. >>>> >>>>> Real question: is there some hidden purpose behind this kind of logic? >>>> >>>> The basic strategy is to double the current capacity unless that will >>>> trigger an unnecessary exception, in which case just use the requested >>>> capacity, but again watch for the implementation limits. >>>> >>>> Cheers, >>>> David >>>> ----- >>>> >>>>> Cheers, >>>>> -- Jim > From paul.sandoz at oracle.com Wed Jun 3 17:47:30 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 3 Jun 2020 10:47:30 -0700 Subject: ThreadGroup.enumerate/activeCount - any good reason not to deprecate? In-Reply-To: References: Message-ID: It?s tempting to go all out and investigate the deprecation of (not for removal) ThreadGroup in its entirety, and also deprecate the Thread constructors accepting ThreadGroup, related methods in SecurityManager, etc. I suspect that?s probably more work than you want to take on right now, and maybe there is still some salvageable functionality. Paul. > On Jun 3, 2020, at 10:13 AM, Alan Bateman wrote: > > > Does ThreadGroup have any fans out there? I'm looking to deprecate its enumerate, activeXXXCount, and list methods. Also Thread.enumerate and activeCount. I'm trying to identify if there are any important usages of these APIs that would suggest they shouldn't be deprecated. A static analysis of the artifacts in Maven isn't coming up with much. > > One reason for deprecating these methods is that they are inheriting racy. The javadoc has recommended for many releases that they be only used for debugging and monitoring purposes. The javadoc has warned, since at least JDK 1.1, that the enumerate methods silently ignore the case where the array isn't large enough. > > Another reason for deprecating these methods is that they are obsolete, even for monitoring purposes. Java 5 added ThreadMXBean, ThreadInfo and better ways to get information on live threads. > > Finally, there is "threads" array keeping a reference to every Thread in the group. That array is problematic for several performance reasons, and ultimately it would be nice to have that go away. Making that go away will require re-implementing a number of JVMTI functions that operate on threads groups but there are alternative ways to implement those (Serguei Spitsyn did an initial prototype to check this). > > For now, I'm just trying to see if there are any important usages of these APIs that I may have missed. > > -Alan. From Alan.Bateman at oracle.com Wed Jun 3 18:40:27 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 3 Jun 2020 19:40:27 +0100 Subject: ThreadGroup.enumerate/activeCount - any good reason not to deprecate? In-Reply-To: References: Message-ID: On 03/06/2020 18:47, Paul Sandoz wrote: > It?s tempting to go all out and investigate the deprecation of (not for removal) ThreadGroup in its entirety, and also deprecate the Thread constructors accepting ThreadGroup, related methods in SecurityManager, etc. I suspect that?s probably more work than you want to take on right now, and maybe there is still some salvageable functionality. > I've held back from deprecating ThreadGroup in its entirety, at least for now, because the notion of a group of threads, or even tree of groups, isn't too toxic and there may be herding or other grouping needs in the future that might mean ThreadGroup comes backs in a different form. Also anything we do here has to take the debugger support into consideration, hence the prototype efforts to reconstitute the list of threads in order to keeping JVMTI working. Deprecating the specific methods mentioned here, plus a few others that I didn't mention in this discussions, means that the only non-deprecated methods would be? getName, getParent, parentOf, checkAccess and uncaughtException. Yes, that is kinda sad but it doesn't preclude us from deprecating the entire class in the future. -Alan. From calvin.cheung at oracle.com Wed Jun 3 19:34:10 2020 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Wed, 3 Jun 2020 12:34:10 -0700 Subject: RFR(XL) 8198698: Support Lambda proxy classes in dynamic CDS archive In-Reply-To: <90ea692b-4a34-19e4-8f42-9f4092dc40e1@oracle.com> References: <7da8b0f7-1027-d033-c1b5-7deb9d1d6c09@oracle.com> <90ea692b-4a34-19e4-8f42-9f4092dc40e1@oracle.com> Message-ID: <74fa5de3-5f9d-be20-37b7-9ea5a15bf211@oracle.com> Hi Mandy, Thanks for your review. On 6/2/20 11:38 AM, Mandy Chung wrote: > Hi Calvin, > > To recap an offline discussion, I think we should use JDK bootcycle > build to test this feature.? For example generating a boot JDK with > archived lambda proxy classes for java.base and javac to build itself > and also use it to run JDK tier 1-3 tests. This would serve a good > test bed. ? I also suggest to experiment how JDK can benefit from this > feature (e.g.? any java.base and javac performance gain).? The module > system initialization had to avoid using lambdas at startup time and > we could re-examine this again. This RFE supports archiving the lambda proxy classes in a dynamic archive. In order to exercise this feature during JDK build time, we probably need to include the lambda proxy classes in the default CDS archive. Or have another dynamic CDS archive on top of the default CDS archive. The dynamic archive may initially contain the classes required by javac to compile a java program, like a HelloWorld. I think this should be explored as a separate RFE. I can get some perf numbers on running javac on HelloWorld later. > > On 5/29/20 2:29 PM, Calvin Cheung wrote: >> RFE: https://bugs.openjdk.java.net/browse/JDK-8198698 >> >> webrev: http://cr.openjdk.java.net/~ccheung/jdk15/8198698/webrev.00/ >> > > Hidden classes have the following properties: > 1. not discoverable by name.? It's not registered in the system > dictionary. > 2. It can be a dynamic nest member of a nest host.? For lambda proxy > class, it has the same nest host as the caller class (same runtime > package as the caller class) > 3. strong or weak link with the class loader.? a hidden class may be > unloaded even its defining class loader is alive. > > For the archived lambda proxy classes, do you re-assign a new class > name to the shared lambda proxy class? No, I didn't assign a new class name to a shared lambda proxy class. > I suspect not.? It's very rare that it will conflict with the name of > other hidden classes.? It's an option to patch the name to indicate > it's from shared archive if it helps diagnosability. In the class load trace (-Xlog:class+load), one will see the following if the lambda proxy class is loaded from a dynamic archive: [0.428s][info ][class,load?? ] BasicLambdaApp$$Lambda$1/0x0000000800bd5258 source: shared objects file (top) > > My comments below are related to the above. > > 1374 InstanceKlass* > SystemDictionary::load_shared_lambda_proxy_class(InstanceKlass* ik, > 1375 Handle class_loader, > 1376 Handle protection_domain, > 1377 PackageEntry* pkg_entry, > 1378 TRAPS) { > 1379 InstanceKlass* shared_n_h = > SystemDictionaryShared::get_shared_nest_host(ik); > 1380 assert(shared_n_h->is_shared(), "nest host must be in CDS archive"); > 1381 Symbol* cn = shared_n_h->name(); > 1382 Klass *s = resolve_or_fail(cn, class_loader, protection_domain, > true, CHECK_NULL); > 1383 if (s != shared_n_h) { > 1384 // The dynamically resolved nest_host is not the same as the one > we used during dump time, > 1385 // so we cannot use ik. > 1386 return NULL;The above resolve_or_fail is to ensure that H is the > same class loaded by the given class loader. Another validation that you should do is to check > the nest host and nest member must be in the same runtime package - > maybeik->set_nest_host(shared_n_h, THREAD) validates the runtime package?? I've added an assert: 1398?? InstanceKlass* loaded_ik = load_shared_class(ik, class_loader, protection_domain, NULL, pkg_entry, CHECK_NULL); 1399 1400?? assert(shared_nest_host->is_same_class_package(ik), 1401????????? "lambda proxy class and its nest host must be in the same package"); The assert needs to be after load_shared_class so that the class loader and PackageEntry have been setup. > > > SystemDictionaryShared::load_shared_lambda_proxy_class > > 1679 InstanceKlass* shared_n_h = get_shared_nest_host(lambda_ik); > 1680 assert(shared_n_h != NULL, "unexpected NULL _nest_host"); > 1681 > 1682 InstanceKlass* loaded_lambda = > 1683 SystemDictionary::load_shared_lambda_proxy_class(lambda_ik, class_loader, protection_domain, pkg_entry, CHECK_NULL); > 1684 > 1685 InstanceKlass* n_h = caller_ik->nest_host(THREAD); > 1686 assert(n_h->has_nest_member(caller_ik, THREAD) || > 1687 n_h == caller_ik, "mismatched nest host"); > > I think you meant to check n_h->has_nest_member(loaded_lambda, THREAD). I saw David has commented on this. So I'll leave the assert as before and I've added another assert (see line 1691): 1687?? // The following ensures that the caller's nest host is the same as the lambda proxy's 1688?? // nest host recorded at dump time. 1689?? assert(nest_host->has_nest_member(caller_ik, THREAD) || 1690????????? nest_host == caller_ik, "caller_ik failed nest member check"); 1691?? assert(nest_host == shared_nest_host, "mismatched nest host"); > > It'd be good to add a comment to say that this ensures thatthe caller's nest host must be the same as the lambda proxy's nest > host recorded at dump time. Done. > > Nit: the variable name "n_h" or "_n_h" would be clearer if it's named > "nest_host" or "_nest_host". I've renamed n_h to nest_host. > > 1689 SystemDictionary::set_shared_class_init_state(lambda_ik, InstanceKlass::loaded); It turns just setting the lambda_ik to the InstanceKlass::loaded state is not enough. We need to do some of the stuff which are currently being done in parse_stream. Please refer to the change in SystemDictionary::load_shared_lambda_proxy_class. > > A lambda proxy class typically is eagerly initialized (unless a flag is given). > Your change dumps the hidden class with no knowledge if `Lookup::defineHiddenClass` > is called with "initialize" parameter is true or not. This may affect the runtime > performance (for example MethodHandle will have class init barrier if the class > is not yet initialized). > This may have performance implication. I think we should consider > passing the initialize parameter to LambdaProxyClassArchive::find such > that if the class is found from CDS archive, it will invoke > before it returns to the library. In InnerClassLambdaMetafactory, the !disableEagerInitialization will be passed to LambdaProxyClassArchive.find which will pass down to vm. ?271???????? Class innerClass = LambdaProxyClassArchive.find(targetClass, ?272 samMethodName, ?273 invokedType, ?274 samMethodType, ?275 implMethod, ?276 instantiatedMethodType, ?277 isSerializable, ?278 markerInterfaces, ?279 additionalBridges, ?280 !disableEagerInitialization); In SystemDictionary::load_shared_lambda_proxy_class, it checks the flag: 1422?? if (initialize) { 1423???? loaded_ik->initialize(CHECK_NULL); 1424?? } On a related note, in the existing jvm_lookup_define_class in jvm.cpp: ? if (init) { ??? ik->initialize(CHECK_NULL); ? } else { ??? ik->link_class(CHECK_NULL); ? } I don't think the else is necessary as the ik->link_class(CHECK_NULL) has been done within the SystemDictionary::parse_stream. > One other point I brought up - strong vs weak hidden class (i.e. is it > using the primary CLD or the per-mirror CLD?). I have lost myself > where you do this in the implementation. Can you point that out? Is it > recorded in the shared archive and make sure it uses the exact CLD > when it's loaded? Currently, the strong vs weak hidden class isn't recorded in the archive. Because in InnerClassLambdaMetafactory.generateInnerClass, there's the following. Lookup lookup = caller.defineHiddenClass(classBytes, !disableEagerInitialization, NESTMATE, STRONG); Also, the above line of code is after the call to LambdaProxyClassArchive.register. (Refer to call stack below.) ?1 - frame( sp=0x00007ffff7fc2730, unextended_sp=0x00007ffff7fc2730, fp=0x00007ffff7fc27c8, pc=0x00007fffd8d3daca) java.lang.ClassLoader.defineClass0(Native Method) ?2 - frame( sp=0x00007ffff7fc27d8, unextended_sp=0x00007ffff7fc27e8, fp=0x00007ffff7fc2880, pc=0x00007fffd8d34e00) java.lang.System$2.defineClass(System.java:2193) ?3 - frame( sp=0x00007ffff7fc2890, unextended_sp=0x00007ffff7fc2890, fp=0x00007ffff7fc2920, pc=0x00007fffd8d34ee3) java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2181) ?4 - frame( sp=0x00007ffff7fc2930, unextended_sp=0x00007ffff7fc2950, fp=0x00007ffff7fc29b0, pc=0x00007fffd8d34e00) java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClassAsLookup(MethodHandles.java:2163) ?5 - frame( sp=0x00007ffff7fc29c0, unextended_sp=0x00007ffff7fc29c8, fp=0x00007ffff7fc2a20, pc=0x00007fffd8d34e00) java.lang.invoke.MethodHandles$Lookup.defineHiddenClass(MethodHandles.java:1961) ?6 - frame( sp=0x00007ffff7fc2a30, unextended_sp=0x00007ffff7fc2a30, fp=0x00007ffff7fc2a98, pc=0x00007fffd8d34e00) java.lang.invoke.InnerClassLambdaMetafactory.generateInnerClass(InnerClassLambdaMetafactory.java:379) ?7 - frame( sp=0x00007ffff7fc2aa8, unextended_sp=0x00007ffff7fc2ae8, fp=0x00007ffff7fc2b38, pc=0x00007fffd8d34e00) java.lang.invoke.InnerClassLambdaMetafactory.spinInnerClass(InnerClassLambdaMetafactory.java:280) ?8 - frame( sp=0x00007ffff7fc2b48, unextended_sp=0x00007ffff7fc2b50, fp=0x00007ffff7fc2ba0, pc=0x00007fffd8d34e00) java.lang.invoke.InnerClassLambdaMetafactory.buildCallSite(InnerClassLambdaMetafactory.java:209) ?9 - frame( sp=0x00007ffff7fc2bb0, unextended_sp=0x00007ffff7fc2bc8, fp=0x00007ffff7fc2c18, pc=0x00007fffd8d34e00) java.lang.invoke.LambdaMetafactory.metafactory(LambdaMetafactory.java:328) 10 - frame( sp=0x00007ffff7fc2c28, unextended_sp=0x00007ffff7fc2c28, fp=0x00007ffff7fc2ca8, pc=0x00007fffd8d34e00) java.lang.invoke.DirectMethodHandle$Holder.invokeStatic 11 - frame( sp=0x00007ffff7fc2cb8, unextended_sp=0x00007ffff7fc2cc0, fp=0x00007ffff7fc2d40, pc=0x00007fffd8d34e00) java.lang.invoke.Invokers$Holder.invokeExact_MT 12 - frame( sp=0x00007ffff7fc2d50, unextended_sp=0x00007ffff7fc2d50, fp=0x00007ffff7fc2dd8, pc=0x00007fffd8d34e00) java.lang.invoke.BootstrapMethodInvoker.invoke(BootstrapMethodInvoker.java:127) 13 - frame( sp=0x00007ffff7fc2de8, unextended_sp=0x00007ffff7fc2e30, fp=0x00007ffff7fc2ea8, pc=0x00007fffd8d34e00) java.lang.invoke.CallSite.makeSite(CallSite.java:315) 14 - frame( sp=0x00007ffff7fc2eb8, unextended_sp=0x00007ffff7fc2ec8, fp=0x00007ffff7fc2f38, pc=0x00007fffd8d34e00) java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:270) 15 - frame( sp=0x00007ffff7fc2f48, unextended_sp=0x00007ffff7fc2f50, fp=0x00007ffff7fc2fc8, pc=0x00007fffd8d34e00) java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:260) C frame (sp=0x00007ffff7fc2fd8 unextended sp=0x00007ffff7fc2ff8, fp=0x00007ffff7fc3090, real_fp=0x00007ffff7fc3090, pc=0x00007fffd8d2b9ca) (~Stub::call_stub) ???? BufferBlob (0x00007fffd8d2b810) used for StubRoutines (1) 16 - frame( sp=0x00007ffff7fc3880, unextended_sp=0x00007ffff7fc3880, fp=0x00007ffff7fc38c8, pc=0x00007fffd8d542d3) LambHello.main(LambHello.java:3) ----- For now, I've added an assert in JVM_RegisterLambdaProxyClassForArchiving to make sure the hidden class is strong so that when things changed later, we'll notice it. I also added a check in SystemDictionaryShared::is_registered_lambda_proxy_class to make sure the hidden class is strong. As for the CLD, I didn't do anything special. In SystemDictionary::load_shared_class: 1470?? ClassLoaderData* loader_data = ClassLoaderData::class_loader_data(class_loader()); It is consistent with what is being done in parse_stream for a strong hidden class. > > > JVM_RegisterLambdaProxyClassForArchiving - we can add an assert to > check if lambdaProxyClass is a hidden class. Added. > JVM_IsCDSDumpingEnabled and JVM_IsCDSSharingEnabled - an alternative > library/VM interface is to use internal system properties that > jdk.internal.misc.VM::saveProperties reads and removes these flags. I'd prefer to leave it as is as there's no perf issue if CDS is not enabled. (More perf data below.) > InnerClassLambdaMetafactory 247 /** Use the LambdaProxyClassArchive > for including the lambda proxy class 248 * in CDS archive and > retrieving the class from the archive. If the class 249 * is not found > in the archive, it calls generatesInnerClass to create 250 * the > class. 251 */ What about simpler comment: /* * Spins the lambda proxy > class. * * This first checks if a lambda proxy class can be loaded > from CDS archive. * Otherwise, generate the lambda proxy class. */ I've updated the comment. > This adds a VM call for every lambda proxy class creation. Do you have > any the performance measurement when CDS is not on? Any performance > regression? Here's the perf data with CDS disabled. The data is for loading 200 lambda proxy classes. Results of " perf stat -r 50 bin/java -Xshare:off -cp TestD.jar TestD 200 " ?? 1:??? 507523520?? 506743227 ( -780293)????? --?????? 156.992 156.453 ( -0.539)????? -- ?? 2:??? 506280039?? 507494909 ( 1214870)?? +++???????? 155.799 156.510 (? 0.711)??? ++ ?? 3:??? 506600572?? 507544573 (? 944001)??? ++???????? 155.100 156.878 (? 1.778) +++++ ?? 4:??? 506240025?? 508590609 ( 2350584) +++++???????? 155.191 155.745 (? 0.554)??? ++ ?? 5:??? 507113653?? 506861905 ( -251748)????? -??????? 155.089 156.126 (? 1.037)?? +++ ?? 6:??? 506331554?? 507727117 ( 1395563)?? +++???????? 155.588 155.452 ( -0.136) ?? 7:??? 506309111?? 506724285 (? 415174)???? +???????? 156.407 156.914 (? 0.507)???? + ?? 8:??? 507805169?? 506926943 ( -878226)????? --?????? 155.796 155.922 (? 0.126) ?? 9:??? 506970117?? 507097707 (? 127590)?????????????? 155.538 156.244 (? 0.706)??? ++ ? 10:??? 506951104?? 506754063 ( -197041)?????????????? 155.429 156.747 (? 1.318)? ++++ ============================================================ ???????? 506812213?? 507246215 (? 434002)???? +???????? 155.692 156.298 (? 0.607)??? ++ instr delta =?????? 434002???? 0.0856% time? delta =??????? 0.607 ms? 0.3896% Updated webrevs: delta: http://cr.openjdk.java.net/~ccheung/jdk15/8198698/webrev_delta.01/ full: http://cr.openjdk.java.net/~ccheung/jdk15/8198698/webrev.01/ thanks, Calvin From mandy.chung at oracle.com Wed Jun 3 20:43:00 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 3 Jun 2020 13:43:00 -0700 Subject: RFR(XL) 8198698: Support Lambda proxy classes in dynamic CDS archive In-Reply-To: <018fa13d-9c60-7abe-5aa3-f12562d82cd7@oracle.com> References: <7da8b0f7-1027-d033-c1b5-7deb9d1d6c09@oracle.com> <90ea692b-4a34-19e4-8f42-9f4092dc40e1@oracle.com> <018fa13d-9c60-7abe-5aa3-f12562d82cd7@oracle.com> Message-ID: Hi David, On 6/2/20 10:30 PM, David Holmes wrote: >> 1685 InstanceKlass* n_h = caller_ik->nest_host(THREAD); >> 1686?? assert(n_h->has_nest_member(caller_ik, THREAD) || >> 1687????????? n_h == caller_ik, "mismatched nest host"); >> >> I think you meant to check n_h->has_nest_member(loaded_lambda, THREAD). > > That can't be right. has_nest_member is for checking static nest > membership. Dynamic proxies are not static nest members! Thanks for pointing out that has_nest_member only checks for static nest membership. My comment is specific to ensure that the runtime nest host of caller_ik == the nest host of loaded_lambda.? I think Calvin's updated patch adds the proper check to compare the nest host and not using has_nest_member (assert nest_host == shared_nest_host) The assert "n_h->has_nest_member(caller_ik, THREAD) || nest_host == caller_ik" is not needed.?? This assert is not correct either since caller class can be a hidden class. Mandy From mandy.chung at oracle.com Wed Jun 3 21:07:28 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 3 Jun 2020 14:07:28 -0700 Subject: RFR(XL) 8198698: Support Lambda proxy classes in dynamic CDS archive In-Reply-To: <74fa5de3-5f9d-be20-37b7-9ea5a15bf211@oracle.com> References: <7da8b0f7-1027-d033-c1b5-7deb9d1d6c09@oracle.com> <90ea692b-4a34-19e4-8f42-9f4092dc40e1@oracle.com> <74fa5de3-5f9d-be20-37b7-9ea5a15bf211@oracle.com> Message-ID: <2557d547-96db-6329-a515-1357cf4a4111@oracle.com> On 6/3/20 12:34 PM, Calvin Cheung wrote: > > I saw David has commented on this. So I'll leave the assert as before > and I've added another assert (see line 1691): > > 1687?? // The following ensures that the caller's nest host is the > same as the lambda proxy's > 1688?? // nest host recorded at dump time. > 1689?? assert(nest_host->has_nest_member(caller_ik, THREAD) || > 1690????????? nest_host == caller_ik, "caller_ik failed nest member > check"); > I don't think this assert is needed.? caller_ik can be a hidden class and so this assert is not correct then. Is there any issue to archive lambda proxy class whose caller is a hidden class?? Is there any assumption in the CDS implementation that the caller class is always a normal class? > 1691?? assert(nest_host == shared_nest_host, "mismatched nest host"); > This is good. > > In SystemDictionary::load_shared_lambda_proxy_class, it checks the flag: > > 1422?? if (initialize) { > 1423???? loaded_ik->initialize(CHECK_NULL); > 1424?? } > I think JVM_LookupLambdaProxyClassFromArchive is a more appropriate place to link and initialize the class before return.?? I expect load_shared_lambda_proxy_class does loading only and linking and initialization should be separate from loading. > On a related note, in the existing jvm_lookup_define_class in jvm.cpp: > > ? if (init) { > ??? ik->initialize(CHECK_NULL); > ? } else { > ??? ik->link_class(CHECK_NULL); > ? } > > I don't think the else is necessary as the ik->link_class(CHECK_NULL) > has been done within the SystemDictionary::parse_stream. > Harold and Lois can chime in here.? I think ik->link_class may be for unsafe anonymous class to prepare for constant pool patching. > Currently, the strong vs weak hidden class isn't recorded in the archive. > > : > > ----- > > For now, I've added an assert in > JVM_RegisterLambdaProxyClassForArchiving to make sure the hidden class > is strong so that when things changed later, we'll notice it. > An assert is good. 3752?? if (invokedName == NULL || invokedType == NULL || methodType == NULL || 3753?????? implMethodMember == NULL || instantiatedMethodType == NULL) { 3754???? return NULL; 3755?? } Should this throw NPE instead? >> This adds a VM call for every lambda proxy class creation. Do you >> have any the performance measurement when CDS is not on? Any >> performance regression? > > Here's the perf data with CDS disabled. The data is for loading 200 > lambda proxy classes. > Actually the Java side checks if CDS is enabled before it calls the VM.? So it's good. Mandy From mandy.chung at oracle.com Wed Jun 3 23:16:19 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 3 Jun 2020 16:16:19 -0700 Subject: Review Request: JDK-8235521: Replacement API for Unsafe::ensureClassInitialized Message-ID: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> This proposes a new `Lookup::ensureClassInitialized` API as a replacement for `sun.misc.Unsafe::ensureClassInitialized`.The Lookup object must have the access to the given class being initialized. CSR: https://bugs.openjdk.java.net/browse/JDK-8245871 webrev: http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8235521/webrev.01/ This patch converts the JDK use of `jdk.internal.misc.Unsafe::ensureClassInitialized` to call this new API where appropriate. `Unsafe::ensureClassInitialized` remains for java.lang.invoke.* internal implementation use. A framework can use `Lookup::privateLookupIn` to access a class in another module if the module authorizes it by opening the packages for it to access. Or a user can produce a less privileged lookup by Lookup::dropLookupMode and pass it to the framework. `Class::ensureInitialized` was considered as an alternative. The downside for this option is that it would be caller sensitive in order to ensure the caller has the proper access to this class.? And it may not work for frameworks which have no access to user classes. It would have to fallback toClass::forName` limited solution. Thanks Mandy From david.holmes at oracle.com Wed Jun 3 23:31:58 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 4 Jun 2020 09:31:58 +1000 Subject: ThreadGroup.enumerate/activeCount - any good reason not to deprecate? In-Reply-To: References: Message-ID: Hi Alan, I've always thought ThreadGroups were a good idea that were never communicated very well. The notion of a ThreadGroup as a security context potentially had some value that was never truly realized. On 4/06/2020 4:40 am, Alan Bateman wrote: > On 03/06/2020 18:47, Paul Sandoz wrote: >> It?s tempting to go all out and investigate the deprecation of (not >> for removal) ThreadGroup in its entirety, and also deprecate the >> Thread constructors accepting ThreadGroup, related methods in >> SecurityManager, etc. I suspect that?s probably more work than you >> want to take on right now, and maybe there is still some salvageable >> functionality. >> > I've held back from deprecating ThreadGroup in its entirety, at least > for now, because the notion of a group of threads, or even tree of > groups, isn't too toxic and there may be herding or other grouping needs > in the future that might mean ThreadGroup comes backs in a different > form. Also anything we do here has to take the debugger support into > consideration, hence the prototype efforts to reconstitute the list of > threads in order to keeping JVMTI working. Thread groups as a concept certainly seem useful just from a partitioning perspective when it comes to managing large numbers of threads. I assume that Java EE environments (and similarly styled execution environments) still create new thread groups and threads for each distinct execution context? In which case I would expect the ability to enumerate the active group members may have some value in those environments. But if noone uses them any more ... > One reason for deprecating these methods is that they are inheriting racy. Yes taking an array as input was a very bad API design for these methods. One of the great historical travesties in this area is that the JLS First Edition listed an alternate API that returned an appropriately sized array, but those APIs were never actually implemented in the JDK. :( David ----- > Deprecating the specific methods mentioned here, plus a few others that > I didn't mention in this discussions, means that the only non-deprecated > methods would be? getName, getParent, parentOf, checkAccess and > uncaughtException. Yes, that is kinda sad but it doesn't preclude us > from deprecating the entire class in the future. > > -Alan. > > > From paul.sandoz at oracle.com Thu Jun 4 00:16:28 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 3 Jun 2020 17:16:28 -0700 Subject: Review Request: JDK-8235521: Replacement API for Unsafe::ensureClassInitialized In-Reply-To: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> References: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> Message-ID: Hi Mandy, Did you consider an alternative name, such as: /** * Initializes {@code targetClass}, if not already initialized. * ... */ public Class initializeClass(Class targetClass) ... ? What about the companion Unsafe.shouldBeInitialized? I have seen (and written) code like this: if (UNSAFE.shouldBeInitialized(cls)) { UNSAFE.ensureClassInitialized(cls); } Although it seems redundant to perform the check, suggesting it is not needed. I believe there is also some gnarly case during a clinit of C where ensureClassInitialized(C) is called and a subsequent call to shouldBeInitialized(C) returns true. Is that correct? If so it makes me reluctant to expose it and try and explain the edge cases. Maybe if we don?t think Unsafe.shouldBeInitialized needs a supported equivalent we can still deprecate it? Paul. > On Jun 3, 2020, at 4:16 PM, Mandy Chung wrote: > > This proposes a new `Lookup::ensureClassInitialized` API as a replacement > for `sun.misc.Unsafe::ensureClassInitialized`.The Lookup object must have > the access to the given class being initialized. > > CSR: https://bugs.openjdk.java.net/browse/JDK-8245871 > > webrev: > http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8235521/webrev.01/ > > This patch converts the JDK use of `jdk.internal.misc.Unsafe::ensureClassInitialized` to > call this new API where appropriate. `Unsafe::ensureClassInitialized` remains for java.lang.invoke.* internal implementation use. > > A framework can use `Lookup::privateLookupIn` to access a class in another module if the module authorizes it by opening the packages for it to access. Or a user can produce a less privileged lookup by Lookup::dropLookupMode and pass it to the framework. > > `Class::ensureInitialized` was considered as an alternative. The downside for this option is that it would be caller sensitive in order to ensure the caller has the proper access to this class. And it may not work for frameworks which have no access to user classes. It would have to fallback toClass::forName` limited solution. > > Thanks > Mandy > From john.r.rose at oracle.com Thu Jun 4 01:24:08 2020 From: john.r.rose at oracle.com (John Rose) Date: Wed, 3 Jun 2020 18:24:08 -0700 Subject: Review Request: JDK-8235521: Replacement API for Unsafe::ensureClassInitialized In-Reply-To: References: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> Message-ID: On Jun 3, 2020, at 5:16 PM, Paul Sandoz wrote: > > if (UNSAFE.shouldBeInitialized(cls)) { > UNSAFE.ensureClassInitialized(cls); > } > > Although it seems redundant to perform the check, suggesting it is not needed. It?s useful (IIRC) in this kind of code: MethodHandle mh = ? something about cls ...; if (!UNSAFE.shouldBeInitialized(cls)) return mh; // Oops, every eventual call to mh needs to do the check: return filterArguments(mh, 0, UNSAFE_ensureClassInitialized.bindTo(cls)); From liyu.italy at gmail.com Thu Jun 4 01:48:39 2020 From: liyu.italy at gmail.com (Lisa Li) Date: Thu, 4 Jun 2020 09:48:39 +0800 Subject: [PATCH] 8245694 : java.util.Properties.entrySet() does not override java.lang.Object methods In-Reply-To: References: Message-ID: Hi Jason, That's a good point! thx~ will do. Best regards, Yu On Wed, Jun 3, 2020 at 11:10 PM Jason Mehrens wrote: > Yu Li, > > You should consider changing the equals implementation to include an > identity self check. > ==== > public boolean equals(Object o) { > return this == o || entrySet.equals(o); > } > > This is consistent with the collection wrapper classes in j.u.Collections. > > Jason > > ________________________________________ > From: core-libs-dev on behalf of > Lisa Li > Sent: Saturday, May 30, 2020 9:28 AM > To: core-libs-dev at openjdk.java.net > Subject: [PATCH] 8245694 : java.util.Properties.entrySet() does not > override java.lang.Object methods > > Hi, > > Please help review the fix for JDK-8245694 > . And this is my very > first patch submission. I know it's not perfect. > > > > *BUG DESCRIPTION*: > > JDK-8245694 : > java.util.Properties.entrySet() > does not override java.lang.Object methods since java 9. > > > *PROPOSED FIX*: > > Add delegating overrides for equals(), hashCode(), and toString(). > > > *PATCH*: > > # HG changeset patch > # User yuli > # Date 1590841711 -28800 > # Sat May 30 20:28:31 2020 +0800 > # Node ID 7dc946e5b5863291fa070bfdbdce48aefbe87b7b > # Parent 8e28aae5068069e853673148e4d3f44cb50350a7 > 8245694: java.util.Properties.entrySet() does not override Object methods > Summary: Add missing override methods > Contributed-by: Yu Li > > diff --git a/src/java.base/share/classes/java/util/Properties.java > b/src/java.base/share/classes/java/util/Properties.java > --- a/src/java.base/share/classes/java/util/Properties.java > +++ b/src/java.base/share/classes/java/util/Properties.java > @@ -1414,6 +1414,21 @@ > return entrySet.retainAll(c); > } > > + @Override > + public boolean equals(Object o) { > + return entrySet.equals(o); > + } > + > + @Override > + public int hashCode() { > + return entrySet.hashCode(); > + } > + > + @Override > + public String toString() { > + return entrySet.toString(); > + } > + > @Override > public Iterator> iterator() { > return entrySet.iterator(); > diff --git a/test/jdk/java/util/Properties/PropertiesEntrySetTest.java > b/test/jdk/java/util/Properties/PropertiesEntrySetTest.java > new file mode 100644 > --- /dev/null > +++ b/test/jdk/java/util/Properties/PropertiesEntrySetTest.java > @@ -0,0 +1,90 @@ > +/* > + * Copyright (c) 2018, 2019, Oracle and/or its affiliates. All rights > reserved. > + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > + * > + * This code is free software; you can redistribute it and/or modify it > + * under the terms of the GNU General Public License version 2 only, as > + * published by the Free Software Foundation. > + * > + * This code is distributed in the hope that it will be useful, but > WITHOUT > + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or > + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License > + * version 2 for more details (a copy is included in the LICENSE file that > + * accompanied this code). > + * > + * You should have received a copy of the GNU General Public License > version > + * 2 along with this work; if not, write to the Free Software Foundation, > + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. > + * > + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA > + * or visit www.oracle.com if you need additional information or have any > + * questions. > + */ > + > +/* > + * @test > + * @summary tests the entrySet() method of Properties class > + * @author Yu Li > + * @bug 8245694 > + */ > + > +import java.util.Map; > +import java.util.Properties; > +import java.util.Set; > + > +public class PropertiesEntrySetTest { > + > + public static void main(String[] args) { > + > + testEntrySet(); > + > + } > + > + private static void assertTrue(boolean value) { > + if (!value) { > + throw new RuntimeException("Failure in Properties testing."); > + } > + } > + > + private static void assertFalse(boolean value) { > + if (value) { > + throw new RuntimeException("Failure in Properties testing."); > + } > + } > + > + private static void testEntrySet() { > + Properties a = new Properties(); > + Set> aEntrySet = a.entrySet(); > + assertFalse(aEntrySet.equals(null)); > + assertTrue(aEntrySet.equals(aEntrySet)); > + > + Properties b = new Properties(); > + Set> bEntrySet = b.entrySet(); > + assertTrue(aEntrySet.equals(bEntrySet)); > + assertTrue(aEntrySet.hashCode() == bEntrySet.hashCode()); > + > + a.setProperty("p1", "1"); > + assertFalse(aEntrySet.equals(bEntrySet)); > + assertFalse(bEntrySet.equals(aEntrySet)); > + assertFalse(aEntrySet.hashCode() == bEntrySet.hashCode()); > + > + b.setProperty("p1", "1"); > + assertTrue(aEntrySet.equals(bEntrySet)); > + assertTrue(aEntrySet.hashCode() == bEntrySet.hashCode()); > + > + Properties c = new Properties(); > + c.setProperty("p1", "2"); > + Set> cEntrySet = c.entrySet(); > + assertFalse(cEntrySet.equals(bEntrySet)); > + assertFalse(cEntrySet.hashCode() == bEntrySet.hashCode()); > + assertFalse(cEntrySet.equals(aEntrySet)); > + assertFalse(cEntrySet.hashCode() == aEntrySet.hashCode()); > + > + a.setProperty("p2", "2"); > + Properties d = new Properties(); > + d.setProperty("p2", "2"); > + d.setProperty("p1", "1"); > + Set> dEntrySet = d.entrySet(); > + assertTrue(dEntrySet.equals(aEntrySet)); > + } > +} > > Best regards, > Yu Li > Attachments area > From mandy.chung at oracle.com Thu Jun 4 02:31:13 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 3 Jun 2020 19:31:13 -0700 Subject: Review Request: JDK-8235521: Replacement API for Unsafe::ensureClassInitialized In-Reply-To: References: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> Message-ID: <2937098a-1c4c-4248-4ea4-362a43f85c15@oracle.com> On 6/3/20 5:16 PM, Paul Sandoz wrote: > Hi Mandy, > > Did you consider an alternative name, such as: > > /** > * Initializes {@code targetClass}, if not already initialized. > * ... > */ > public Class initializeClass(Class targetClass) ... > > ? I considered this.?? The targetClass may have been initialized. `initializeClass` seems to be read that it should check if it's initialized before calling?? ?? I also considered `ensureInitialized` which can be an option.?? I'm open to suggestion. > What about the companion Unsafe.shouldBeInitialized? I have seen (and written) code like this: > > if (UNSAFE.shouldBeInitialized(cls)) { > UNSAFE.ensureClassInitialized(cls); > } A class may be initialized in many ways as specified in JVMS 5.5 (e.g. another thread accesses a static member of this class). ? The class may get initialized right after `shouldBeInitialized` returns true.?? `ensureClassInitialized` handles the case when the class has been initialized and I don't see the need to expose it. > Although it seems redundant to perform the check, suggesting it is not needed. > > I believe there is also some gnarly case during a clinit of C where ensureClassInitialized(C) is called and a subsequent call to shouldBeInitialized(C) returns true. Is that correct? If so it makes me reluctant to expose it and try and explain the edge cases. Yes. > Maybe if we don?t think Unsafe.shouldBeInitialized needs a supported equivalent we can still deprecate it? Good idea.?? I will update the patch and CSR. Mandy > Paul. > > >> On Jun 3, 2020, at 4:16 PM, Mandy Chung wrote: >> >> This proposes a new `Lookup::ensureClassInitialized` API as a replacement >> for `sun.misc.Unsafe::ensureClassInitialized`.The Lookup object must have >> the access to the given class being initialized. >> >> CSR: https://bugs.openjdk.java.net/browse/JDK-8245871 >> >> webrev: >> http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8235521/webrev.01/ >> >> This patch converts the JDK use of `jdk.internal.misc.Unsafe::ensureClassInitialized` to >> call this new API where appropriate. `Unsafe::ensureClassInitialized` remains for java.lang.invoke.* internal implementation use. >> >> A framework can use `Lookup::privateLookupIn` to access a class in another module if the module authorizes it by opening the packages for it to access. Or a user can produce a less privileged lookup by Lookup::dropLookupMode and pass it to the framework. >> >> `Class::ensureInitialized` was considered as an alternative. The downside for this option is that it would be caller sensitive in order to ensure the caller has the proper access to this class. And it may not work for frameworks which have no access to user classes. It would have to fallback toClass::forName` limited solution. >> >> Thanks >> Mandy >> From mandy.chung at oracle.com Thu Jun 4 02:38:57 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 3 Jun 2020 19:38:57 -0700 Subject: Review Request: JDK-8235521: Replacement API for Unsafe::ensureClassInitialized In-Reply-To: References: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> Message-ID: <5d0d7f6b-addf-ad01-6625-ad7688ad3b5f@oracle.com> On 6/3/20 6:24 PM, John Rose wrote: > On Jun 3, 2020, at 5:16 PM, Paul Sandoz > wrote: >> >> ?if (UNSAFE.shouldBeInitialized(cls)) { >> ?????UNSAFE.ensureClassInitialized(cls); >> ?} >> >> Although it seems redundant to perform the check, suggesting it is >> not needed. > > It?s useful (IIRC) in this kind of code: > > MethodHandle mh = ? something about cls ...; > ?if (!UNSAFE.shouldBeInitialized(cls)) > ? ? return mh; > // Oops, every eventual call to mh needs to do the check: > return filterArguments(mh, 0, UNSAFE_ensureClassInitialized.bindTo(cls)); > It is useful for method handle machinery.?? I don't see the need to expose it as a supported API for java applications/framework. Maybe useful for dynamic languages? Mandy From alexander.matveev at oracle.com Thu Jun 4 07:03:27 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Thu, 4 Jun 2020 00:03:27 -0700 Subject: RFR: 8236282: [macos] Find permanent solution to macOS test timeout problem JDK-8235738 Message-ID: <5cd5af78-b0a7-a6d6-2c73-871958dc8a1a@oracle.com> Please review the jpackage fix for bug [1] at [2]. This is still a workaround for JDK-8236825. Implemented as per Roger suggestion to redirect output to temp file. I did a lot of testing using repro case and was not able to reproduce hang anymore. [1] https://bugs.openjdk.java.net/browse/JDK-8236282 [2] http://cr.openjdk.java.net/~almatvee/8236282/webrev.01/ Thanks, Alexander From claes.redestad at oracle.com Thu Jun 4 12:58:50 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 4 Jun 2020 14:58:50 +0200 Subject: RFR: 8246592: Simplify checking of boolean file attributes Message-ID: <9a005eac-d701-15db-1ad4-4aa1b4f00498@oracle.com> Hi, this patch proposes adding a FileSystem.getBooleanAttribute convenience method, which simplifies code at use sites. This also enables improvements when not requesting an attribute that requires extra work, such as getting the hidden bit on unix derivatives. Bug: https://bugs.openjdk.java.net/browse/JDK-8246592 Webrev: http://cr.openjdk.java.net/~redestad/8246592/open.00/ Testing: tier1-2 Micro results on Linux: Benchmark Mode Cnt Score Error Units FileOpen.booleanAttributes avgt 10 1.868 ? 0.002 us/op Benchmark Mode Cnt Score Error Units FileOpen.booleanAttributes avgt 10 0.938 ? 0.008 us/op Thanks! /Claes From Roger.Riggs at oracle.com Thu Jun 4 13:22:49 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Thu, 4 Jun 2020 09:22:49 -0400 Subject: RFR: 8246592: Simplify checking of boolean file attributes In-Reply-To: <9a005eac-d701-15db-1ad4-4aa1b4f00498@oracle.com> References: <9a005eac-d701-15db-1ad4-4aa1b4f00498@oracle.com> Message-ID: Hi Claes, Not a review... You'll need a CSR for the API change. FileSystem.java: You'll need proper javadoc for the new getBooleanAttribute method. 6/11 is Rampdown Phase One? (its a bit late for new APIs). Regards, Roger On 6/4/20 8:58 AM, Claes Redestad wrote: > Hi, > > this patch proposes adding a FileSystem.getBooleanAttribute convenience > method, which simplifies code at use sites. This also enables > improvements when not requesting an attribute that requires extra work, > such as getting the hidden bit on unix derivatives. > > Bug:??? https://bugs.openjdk.java.net/browse/JDK-8246592 > Webrev: http://cr.openjdk.java.net/~redestad/8246592/open.00/ > > Testing: tier1-2 > > Micro results on Linux: > > Benchmark?????????????????? Mode? Cnt? Score?? Error? Units > FileOpen.booleanAttributes? avgt?? 10? 1.868 ? 0.002? us/op > > Benchmark?????????????????? Mode? Cnt? Score?? Error? Units > FileOpen.booleanAttributes? avgt?? 10? 0.938 ? 0.008? us/op > > Thanks! > > /Claes From claes.redestad at oracle.com Thu Jun 4 13:25:36 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 4 Jun 2020 15:25:36 +0200 Subject: RFR: 8246592: Simplify checking of boolean file attributes In-Reply-To: References: <9a005eac-d701-15db-1ad4-4aa1b4f00498@oracle.com> Message-ID: <07513567-f360-eb7d-d627-65743c676e2d@oracle.com> Hi Roger, maybe I've missed something, but java.io.FileSystem is package-private and not part of the public API. Is a CSR really required? /Claes On 2020-06-04 15:22, Roger Riggs wrote: > Hi Claes, > > Not a review... > > You'll need a CSR for the API change. > > FileSystem.java: You'll need proper javadoc for the new > getBooleanAttribute method. > > 6/11 is Rampdown Phase One? (its a bit late for new APIs). > > Regards, Roger > > > On 6/4/20 8:58 AM, Claes Redestad wrote: >> Hi, >> >> this patch proposes adding a FileSystem.getBooleanAttribute convenience >> method, which simplifies code at use sites. This also enables >> improvements when not requesting an attribute that requires extra work, >> such as getting the hidden bit on unix derivatives. >> >> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8246592 >> Webrev: http://cr.openjdk.java.net/~redestad/8246592/open.00/ >> >> Testing: tier1-2 >> >> Micro results on Linux: >> >> Benchmark?????????????????? Mode? Cnt? Score?? Error? Units >> FileOpen.booleanAttributes? avgt?? 10? 1.868 ? 0.002? us/op >> >> Benchmark?????????????????? Mode? Cnt? Score?? Error? Units >> FileOpen.booleanAttributes? avgt?? 10? 0.938 ? 0.008? us/op >> >> Thanks! >> >> /Claes > From Alan.Bateman at oracle.com Thu Jun 4 13:26:32 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 4 Jun 2020 14:26:32 +0100 Subject: RFR: 8246592: Simplify checking of boolean file attributes In-Reply-To: References: <9a005eac-d701-15db-1ad4-4aa1b4f00498@oracle.com> Message-ID: <57cc9714-41e2-c879-febc-805f2e66b2fc@oracle.com> On 04/06/2020 14:22, Roger Riggs wrote: > Hi Claes, > > Not a review... > > You'll need a CSR for the API change. > > FileSystem.java: You'll need proper javadoc for the new > getBooleanAttribute method. FileSystem is not a public class so it's not actually an API change. I agree the public modifier is confusing, should probably be dropped from all these methods. -Alan From Roger.Riggs at oracle.com Thu Jun 4 13:27:54 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Thu, 4 Jun 2020 09:27:54 -0400 Subject: RFR: 8246592: Simplify checking of boolean file attributes In-Reply-To: <07513567-f360-eb7d-d627-65743c676e2d@oracle.com> References: <9a005eac-d701-15db-1ad4-4aa1b4f00498@oracle.com> <07513567-f360-eb7d-d627-65743c676e2d@oracle.com> Message-ID: <72752cd9-c921-5f60-33ac-d890ddee2d1f@oracle.com> Hi Claes, Yep, my mistake, too many publics. Thanks, Roger On 6/4/20 9:25 AM, Claes Redestad wrote: > Hi Roger, > > maybe I've missed something, but java.io.FileSystem is package-private > and not part of the public API. Is a CSR really required? > > /Claes > > On 2020-06-04 15:22, Roger Riggs wrote: >> Hi Claes, >> >> Not a review... >> >> You'll need a CSR for the API change. >> >> FileSystem.java: You'll need proper javadoc for the new >> getBooleanAttribute method. >> >> 6/11 is Rampdown Phase One? (its a bit late for new APIs). >> >> Regards, Roger >> >> >> On 6/4/20 8:58 AM, Claes Redestad wrote: >>> Hi, >>> >>> this patch proposes adding a FileSystem.getBooleanAttribute convenience >>> method, which simplifies code at use sites. This also enables >>> improvements when not requesting an attribute that requires extra work, >>> such as getting the hidden bit on unix derivatives. >>> >>> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8246592 >>> Webrev: http://cr.openjdk.java.net/~redestad/8246592/open.00/ >>> >>> Testing: tier1-2 >>> >>> Micro results on Linux: >>> >>> Benchmark?????????????????? Mode? Cnt? Score?? Error? Units >>> FileOpen.booleanAttributes? avgt?? 10? 1.868 ? 0.002? us/op >>> >>> Benchmark?????????????????? Mode? Cnt? Score?? Error? Units >>> FileOpen.booleanAttributes? avgt?? 10? 0.938 ? 0.008? us/op >>> >>> Thanks! >>> >>> /Claes >> From claes.redestad at oracle.com Thu Jun 4 13:41:47 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 4 Jun 2020 15:41:47 +0200 Subject: RFR: 8246592: Simplify checking of boolean file attributes In-Reply-To: <57cc9714-41e2-c879-febc-805f2e66b2fc@oracle.com> References: <9a005eac-d701-15db-1ad4-4aa1b4f00498@oracle.com> <57cc9714-41e2-c879-febc-805f2e66b2fc@oracle.com> Message-ID: <96578c15-9de5-2208-3a64-a664fe659cde@oracle.com> On 2020-06-04 15:26, Alan Bateman wrote: > On 04/06/2020 14:22, Roger Riggs wrote: >> Hi Claes, >> >> Not a review... >> >> You'll need a CSR for the API change. >> >> FileSystem.java: You'll need proper javadoc for the new >> getBooleanAttribute method. > FileSystem is not a public class so it's not actually an API change. I > agree the public modifier is confusing, should probably be dropped from > all these methods. If you prefer I can strip out all the public modifiers. Or do that as a follow-up. /Claes From Roger.Riggs at oracle.com Thu Jun 4 16:05:11 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Thu, 4 Jun 2020 12:05:11 -0400 Subject: RFR: 8246592: Simplify checking of boolean file attributes In-Reply-To: <96578c15-9de5-2208-3a64-a664fe659cde@oracle.com> References: <9a005eac-d701-15db-1ad4-4aa1b4f00498@oracle.com> <57cc9714-41e2-c879-febc-805f2e66b2fc@oracle.com> <96578c15-9de5-2208-3a64-a664fe659cde@oracle.com> Message-ID: <91f22718-7477-4f61-3428-70ed79af6f9e@oracle.com> Hi Claes, The code looks ok but I would rename the new method to 'hasBooleanAttribute' since it is not returning the attribute but testing for it. I think you can leave the 'public' modifiers to a separate issue. Regards, Roger On 6/4/20 9:41 AM, Claes Redestad wrote: > > > On 2020-06-04 15:26, Alan Bateman wrote: >> On 04/06/2020 14:22, Roger Riggs wrote: >>> Hi Claes, >>> >>> Not a review... >>> >>> You'll need a CSR for the API change. >>> >>> FileSystem.java: You'll need proper javadoc for the new >>> getBooleanAttribute method. >> FileSystem is not a public class so it's not actually an API change. >> I agree the public modifier is confusing, should probably be dropped >> from all these methods. > > If you prefer I can strip out all the public modifiers. Or do that as a > follow-up. > > /Claes From alexey.semenyuk at oracle.com Thu Jun 4 16:21:06 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Thu, 4 Jun 2020 12:21:06 -0400 Subject: JDK-8238204: run_tests.sh fails on macOS when called from test_jpackage.sh In-Reply-To: <8a3f9c48-9561-3a4a-7e07-53a9ebbc1539@oracle.com> References: <8a3f9c48-9561-3a4a-7e07-53a9ebbc1539@oracle.com> Message-ID: <2603829e-0ce1-1d03-8dbf-1294a5b78d42@oracle.com> Please review fix [2] for jpackage bug [1]. Replace xargs call with --no-run-if-empty parameter with bash expressions in run_tests.sh Call run_tests.sh from test_jpackager.sh in a way to avoid shebang interpretation that doesn't work properly in the environment without bash in the PATH. Both scripts are used to run jpackage tests in local builds. - Alexey [1] https://bugs.openjdk.java.net/browse/JDK-8238204 [2]?http://cr.openjdk.java.net/~asemenyuk/8238204/webrev.00 From chris.hegarty at oracle.com Thu Jun 4 16:43:13 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 4 Jun 2020 17:43:13 +0100 Subject: Review Request: JDK-8235521: Replacement API for Unsafe::ensureClassInitialized In-Reply-To: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> References: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> Message-ID: Mandy, I think this looks good. Just a few minor comments... The spec wording is hard, since the method is offering a conditional initialization. I guess I expected to see something like ?initializes targetClass, if it has not been initialized already?, or some such. The input and output is the same class instance. I don?t quite get that from the spec. Can these be coupled together more obviously by their descriptions? Maybe "@return targetClass, that is initialized" -Chris. > On 4 Jun 2020, at 00:16, Mandy Chung wrote: > > This proposes a new `Lookup::ensureClassInitialized` API as a replacement > for `sun.misc.Unsafe::ensureClassInitialized`.The Lookup object must have > the access to the given class being initialized. > > CSR: https://bugs.openjdk.java.net/browse/JDK-8245871 > > webrev: > http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8235521/webrev.01/ > > This patch converts the JDK use of `jdk.internal.misc.Unsafe::ensureClassInitialized` to > call this new API where appropriate. `Unsafe::ensureClassInitialized` remains for java.lang.invoke.* internal implementation use. > > A framework can use `Lookup::privateLookupIn` to access a class in another module if the module authorizes it by opening the packages for it to access. Or a user can produce a less privileged lookup by Lookup::dropLookupMode and pass it to the framework. > > `Class::ensureInitialized` was considered as an alternative. The downside for this option is that it would be caller sensitive in order to ensure the caller has the proper access to this class. And it may not work for frameworks which have no access to user classes. It would have to fallback toClass::forName` limited solution. > > Thanks > Mandy > From paul.sandoz at oracle.com Thu Jun 4 16:55:40 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 4 Jun 2020 09:55:40 -0700 Subject: Review Request: JDK-8235521: Replacement API for Unsafe::ensureClassInitialized In-Reply-To: <2937098a-1c4c-4248-4ea4-362a43f85c15@oracle.com> References: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> <2937098a-1c4c-4248-4ea4-362a43f85c15@oracle.com> Message-ID: > On Jun 3, 2020, at 7:31 PM, Mandy Chung wrote: > > > > On 6/3/20 5:16 PM, Paul Sandoz wrote: >> Hi Mandy, >> >> Did you consider an alternative name, such as: >> >> /** >> * Initializes {@code targetClass}, if not already initialized. >> * ... >> */ >> public Class initializeClass(Class targetClass) ... >> >> ? > > I considered this. The targetClass may have been initialized. `initializeClass` seems to be read that it should check if it's initialized before calling?? I also considered `ensureInitialized` which can be an option. I'm open to suggestion. > FWIW I keep tripping over the prefix ?ensures?. As I understand the implementation initializes a class, if not already so (and it has to check as you point out below). Focusing on the main action seems appropriate in this regard. I expect John has more context on the naming. >> What about the companion Unsafe.shouldBeInitialized? I have seen (and written) code like this: >> >> if (UNSAFE.shouldBeInitialized(cls)) { >> UNSAFE.ensureClassInitialized(cls); >> } > > A class may be initialized in many ways as specified in JVMS 5.5 (e.g. another thread accesses a static member of this class). The class may get initialized right after `shouldBeInitialized` returns true. `ensureClassInitialized` handles the case when the class has been initialized and I don't see the need to expose it. > Ok. >> Although it seems redundant to perform the check, suggesting it is not needed. >> >> I believe there is also some gnarly case during a clinit of C where ensureClassInitialized(C) is called and a subsequent call to shouldBeInitialized(C) returns true. Is that correct? If so it makes me reluctant to expose it and try and explain the edge cases. > > Yes. >> Maybe if we don?t think Unsafe.shouldBeInitialized needs a supported equivalent we can still deprecate it? > > Good idea. I will update the patch and CSR. > Ok. Paul. From andy.herrick at oracle.com Thu Jun 4 17:11:12 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Thu, 4 Jun 2020 13:11:12 -0400 Subject: JDK-8238204: run_tests.sh fails on macOS when called from test_jpackage.sh In-Reply-To: <2603829e-0ce1-1d03-8dbf-1294a5b78d42@oracle.com> References: <8a3f9c48-9561-3a4a-7e07-53a9ebbc1539@oracle.com> <2603829e-0ce1-1d03-8dbf-1294a5b78d42@oracle.com> Message-ID: <35a047a6-4d74-73ba-3f33-8d25cf460974@oracle.com> looks good. /Andy On 6/4/2020 12:21 PM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > Replace xargs call with --no-run-if-empty parameter with bash > expressions in run_tests.sh > Call run_tests.sh from test_jpackager.sh in a way to avoid shebang > interpretation that doesn't work properly in the environment without > bash in the PATH. > > Both scripts are used to run jpackage tests in local builds. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8238204 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8238204/webrev.00 > From igor.ignatyev at oracle.com Thu Jun 4 17:21:35 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 4 Jun 2020 10:21:35 -0700 Subject: RFR(S/T) : 8245874 : requires.extraPropDefns.vmOpts doesn't need -Xbootclasspath/a:bootClasses In-Reply-To: <88ED55CF-092F-4FAC-ACFD-17D3CA869C75@oracle.com> References: <88ED55CF-092F-4FAC-ACFD-17D3CA869C75@oracle.com> Message-ID: piny? -- Igor > On Jun 1, 2020, at 7:54 AM, Igor Ignatyev wrote: > > ping? > -- Igor > >> On May 27, 2020, at 10:25 AM, Igor Ignatyev wrote: >> >> http://cr.openjdk.java.net/~iignatyev//8245874/webrev.00 >>> 8 lines changed: 2 ins; 0 del; 6 mod; >> >> Hi all, >> >> could you please review this small and trivial cleanup in TEST.ROOT files of test/jdk/ and test/hotspot/jtreg test suites? >> from JBS: >>> to make sure that classes listed in `requires.extraPropDefns.bootlibs` are in BCP, jtreg automatically adds corresponding `-Xbootclasspath/a:` to the options list, so `requires.extraPropDefns.vmOpts` doesn't have to have `-Xbootclasspath/a:bootClasses` >> the patch >> - removes -Xbootclasspath/a:bootClasses from requires.extraPropDefns.vmOpts >> - moves jdk.test.lib.Platform and Container classes from requires.extraPropDefns.bootlibs to requires.extraPropDefns.libs as there is no need for them to be in BCP >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8245874 >> testing: tier1 >> webrev: http://cr.openjdk.java.net/~iignatyev//8245874/webrev.00 >> >> Thanks, >> -- Igor > From alexey at azul.com Thu Jun 4 18:15:49 2020 From: alexey at azul.com (Alexey Bakhtin) Date: Thu, 4 Jun 2020 18:15:49 +0000 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> Message-ID: <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> Hello, Could you please review new version of the patch: http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v3/ I?ve moved all logic for creating TLS Channel Binding data out of GssKrb5Client.java file. All data is prepared inside TlsChannelBinding class. Internal property name is renamed to ?com.sun.security.sasl.tlschannelbinding?. TlsChannelBinding object is still used to pass channel binding data from LdapClient to GssKrb5Client. I do not change it to byte[] because of TlsChannelBinding class is still used for internal property name. Also, I?ve updated TlsChannelBinding class to select SHA-256 hash algorithm if it can not be derived from the certificate signature name. Regards Alexey > On 26 May 2020, at 17:50, Weijun Wang wrote: > > I have a question on GssKrb5Client.java: > > Do you think it's a good idea to let the SASL mechanism understand what TLS binding is? Instead of passing the whole TlsChannelBinding object through a SASL property, is it possible to only pass the actual cbData? After all, the name "com.sun.security.sasl.channelbinding" suggests it's just a general ChannelBinding which is independent with any application level info. > > From my reading of the code change, it looks like this cbData can be calculated on the LDAP side. > > Thanks, > Max > >> On May 21, 2020, at 3:35 PM, Alexey Bakhtin wrote: >> >> Hello, >> >> Could you please review the following patch: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8245527 >> Webrev: http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v0/ >> >> The Windows LDAP server with LdapEnforceChannelBinding=2 uses the tls-server-end-point channel binding >> (based on the TLS server certificate). The channel binding data is calculated as following : >> ? The client calculates a hash of the TLS server certificate. >> The hash algorithm is selected on the base of the certificate signature algorithm. >> Also, the client should use SHA-256 algorithm, in case of the certificate signature algorithm is SHA1 or MD5 based >> ? The channel binding information is the same as defined in rfc4121 [1] with small corrections: >> ? initiator and acceptor addresses should be set to NULL >> ? initiator and acceptor address types should be zero. >> It contradicts to the ?Using Channel Bindings in GSS-API? document [2] that say that >> the address type should be set to GSS_C_AF_NULLADDR=0xFF, >> instead of GSS_C_AF_UNSPEC=0x00. >> >> This patch introduces changes in the LDAP, SASL and JGSS modules >> to generate channel binding data automatically if requested by an application. >> This patch reuses existing org.ietf.jgss.ChannelBinding class implementation but changes >> initial unspecified address type from CHANNEL_BINDING_AF_NULL_ADDR to CHANNEL_BINDING_AF_UNSPEC. >> The patch introduces new environment property ?com.sun.jndi.ldap.tls.cbtype? that indicates >> Channel Binding type that should be used in the LDAPS connection over JGSS/Kerberos. >> Right now "tls-server-end-point" Channel Binding type is supported only. >> The client extracts server certificate from the underlying TLS connection and creates >> Channel Binding data for JGSS/Kerberos authentication if application indicates >> com.sun.jndi.ldap.tls.cbtype=tls-server-end-point property. >> Client application should also specify existing "com.sun.jndi.ldap.connect.timeout? property >> to force and wait TLS handshake completion before JGSS/Kerberos authentication data is generated. >> >> [1] - https://tools.ietf.org/html/rfc4121#section-4.1.1.2 >> >> [2] - >> https://docs.oracle.com/cd/E19120-01/open.solaris/819-2145/overview-52/index.html >> >> Initial discussion of this issue is available at security-dev list: >> https://mail.openjdk.java.net/pipermail/security-dev/2019-December/021052.html >> https://mail.openjdk.java.net/pipermail/security-dev/2020-January/021140.html >> https://mail.openjdk.java.net/pipermail/security-dev/2020-February/021278.html >> https://mail.openjdk.java.net/pipermail/security-dev/2020-May/021864.html From claes.redestad at oracle.com Thu Jun 4 18:24:36 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 4 Jun 2020 20:24:36 +0200 Subject: RFR: 8246592: Simplify checking of boolean file attributes In-Reply-To: <91f22718-7477-4f61-3428-70ed79af6f9e@oracle.com> References: <9a005eac-d701-15db-1ad4-4aa1b4f00498@oracle.com> <57cc9714-41e2-c879-febc-805f2e66b2fc@oracle.com> <96578c15-9de5-2208-3a64-a664fe659cde@oracle.com> <91f22718-7477-4f61-3428-70ed79af6f9e@oracle.com> Message-ID: <2e321139-5b59-6b43-6e9c-fdba6f881d0c@oracle.com> Hi Roger, On 2020-06-04 18:05, Roger Riggs wrote: > Hi Claes, > > The code looks ok but I would rename the new method to > 'hasBooleanAttribute' > since it is not returning the attribute but testing for it. good suggestion: http://cr.openjdk.java.net/~redestad/8246592/open.01/ > > I think you can leave the 'public' modifiers to a separate issue. Ok. /Claes > > Regards, Roger > > > > > On 6/4/20 9:41 AM, Claes Redestad wrote: >> >> >> On 2020-06-04 15:26, Alan Bateman wrote: >>> On 04/06/2020 14:22, Roger Riggs wrote: >>>> Hi Claes, >>>> >>>> Not a review... >>>> >>>> You'll need a CSR for the API change. >>>> >>>> FileSystem.java: You'll need proper javadoc for the new >>>> getBooleanAttribute method. >>> FileSystem is not a public class so it's not actually an API change. >>> I agree the public modifier is confusing, should probably be dropped >>> from all these methods. >> >> If you prefer I can strip out all the public modifiers. Or do that as a >> follow-up. >> >> /Claes > From ivan.sipka at oracle.com Thu Jun 4 18:40:27 2020 From: ivan.sipka at oracle.com (Ivan Sipka) Date: Thu, 4 Jun 2020 11:40:27 -0700 (PDT) Subject: request for review JDK-8211290 Message-ID: Hi all, please review the following changeset: http://cr.openjdk.java.net/~iignatyev/isipka/8211974/webrev.01/index.html for the JBS issue https://bugs.openjdk.java.net/browse/JDK-8211974 which moves the files: open/test/jdk/lib/testlibrary/java/util/jar/Compiler.java open/test/jdk/lib/testlibrary/java/util/jar/JarBuilder.java to files: open/test/lib/jdk/test/lib/util/JarBuilder.java open/test/lib/jdk/test/lib/compiler/Compiler.java and changes the relevant jtreg specification tags in dependent tests: open/test/jdk/sun/net/www/protocol/jar/MultiReleaseJarURLConnection.java open/test/jdk/jdk/nio/zipfs/jarfs/JFSTester.java open/test/jdk/jdk/nio/zipfs/jarfs/MultiReleaseJarTest.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarAPI.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarHttpProperties.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarProperties.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarSecurity.java open/test/jdk/java/util/jar/JarFile/JarBacktickManifest.java open/test/jdk/java/net/URL/JarHandlerPkgPrefix/JarHandlerPkgPrefix.java and adds relevant package imports to dependent file: open/test/jdk/lib/testlibrary/java/util/jar/CreateMultiReleaseTestJars.java jtreg command to run all dependent tests from repository root jtreg -verbose:summary -ea -esa -a -conc:9 -jdk:build/linux-x64/jdk/ open/test/jdk/sun/net/www/protocol/jar/MultiReleaseJarURLConnection.java open/test/jdk/jdk/nio/zipfs/jarfs/JFSTester.java open/test/jdk/jdk/nio/zipfs/jarfs/MultiReleaseJarTest.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarAPI.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarHttpProperties.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarProperties.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarSecurity.java open/test/jdk/java/util/jar/JarFile/JarBacktickManifest.java open/test/jdk/java/net/URL/JarHandlerPkgPrefix/JarHandlerPkgPrefix.java thank you, From Alan.Bateman at oracle.com Thu Jun 4 18:43:37 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 4 Jun 2020 19:43:37 +0100 Subject: Review Request: JDK-8235521: Replacement API for Unsafe::ensureClassInitialized In-Reply-To: References: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> <2937098a-1c4c-4248-4ea4-362a43f85c15@oracle.com> Message-ID: On 04/06/2020 17:55, Paul Sandoz wrote: > : > FWIW I keep tripping over the prefix ?ensures?. As I understand the implementation initializes a class, if not already so (and it has to check as you point out below). Focusing on the main action seems appropriate in this regard. I expect John has more context on the naming. > One thing to mention is that some of the java.util classes have ensureCapacity methods, also have an EnsureXXX methods in JNI to ensure capacity. I've used ensureXXX in a few places to ensure that something is done before going further. Is it that this ensure method returns the Class rather than void? -Alan. From paul.sandoz at oracle.com Thu Jun 4 19:00:01 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 4 Jun 2020 12:00:01 -0700 Subject: Review Request: JDK-8235521: Replacement API for Unsafe::ensureClassInitialized In-Reply-To: References: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> <2937098a-1c4c-4248-4ea4-362a43f85c15@oracle.com> Message-ID: <9ACC1C48-11FE-4F91-B9AD-5848C3682B7E@oracle.com> > On Jun 4, 2020, at 11:43 AM, Alan Bateman wrote: > > On 04/06/2020 17:55, Paul Sandoz wrote: >> : >> FWIW I keep tripping over the prefix ?ensures?. As I understand the implementation initializes a class, if not already so (and it has to check as you point out below). Focusing on the main action seems appropriate in this regard. I expect John has more context on the naming. >> > One thing to mention is that some of the java.util classes have ensureCapacity methods, also have an EnsureXXX methods in JNI to ensure capacity. I've used ensureXXX in a few places to ensure that something is done before going further. I see, thanks, that helps. If I view it as a (side-effecting) constraint before proceeding, then I think I get it, I was not thinking like that and was more focused on the action itself. > Is it that this ensure method returns the Class rather than void? > No. I was just seeing it from a different perspective. I withdraw my naming suggestion! Paul. From paul.sandoz at oracle.com Thu Jun 4 19:13:43 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 4 Jun 2020 12:13:43 -0700 Subject: Review Request: JDK-8235521: Replacement API for Unsafe::ensureClassInitialized In-Reply-To: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> References: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> Message-ID: Hi Mandy, What about this case: class Test { static { MethodHandles.lookup().ensureClassInitialized(Test.class); // not yet initialized } } (I see you do that for m/p1/A.java and Test.clinitInvokeEnsureClassInitialized.) Do we need mention this in the spec? e.g. this method does nothing if the class is already initialized or if the class is in the process of being initialized? Paul. > On Jun 3, 2020, at 4:16 PM, Mandy Chung wrote: > > This proposes a new `Lookup::ensureClassInitialized` API as a replacement > for `sun.misc.Unsafe::ensureClassInitialized`.The Lookup object must have > the access to the given class being initialized. > > CSR: https://bugs.openjdk.java.net/browse/JDK-8245871 > > webrev: > http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8235521/webrev.01/ > > This patch converts the JDK use of `jdk.internal.misc.Unsafe::ensureClassInitialized` to > call this new API where appropriate. `Unsafe::ensureClassInitialized` remains for java.lang.invoke.* internal implementation use. > > A framework can use `Lookup::privateLookupIn` to access a class in another module if the module authorizes it by opening the packages for it to access. Or a user can produce a less privileged lookup by Lookup::dropLookupMode and pass it to the framework. > > `Class::ensureInitialized` was considered as an alternative. The downside for this option is that it would be caller sensitive in order to ensure the caller has the proper access to this class. And it may not work for frameworks which have no access to user classes. It would have to fallback toClass::forName` limited solution. > > Thanks > Mandy > From mandy.chung at oracle.com Thu Jun 4 19:21:38 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 4 Jun 2020 12:21:38 -0700 Subject: Review Request: JDK-8235521: Replacement API for Unsafe::ensureClassInitialized In-Reply-To: References: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> Message-ID: <1db50e44-b0bc-6e98-22a1-a826c2a3bbc1@oracle.com> On 6/4/20 9:43 AM, Chris Hegarty wrote: > Mandy, > > I think this looks good. Just a few minor comments... > > The spec wording is hard, since the method is offering a conditional initialization. I guess I expected to see something like ?initializes targetClass, if it has not been initialized already?, or some such. I added: ?? "This method causes {@code targetClass} to be initialized if it has not been already initialized." to make it clear. > > The input and output is the same class instance. I don?t quite get that from the spec. Can these be coupled together more obviously by their descriptions? Maybe "@return targetClass, that is initialized" @return the class that has been initialized. I changed it to: ???????? * @return {@code targetClass} that has been initialized Mandy > > -Chris. > >> On 4 Jun 2020, at 00:16, Mandy Chung wrote: >> >> This proposes a new `Lookup::ensureClassInitialized` API as a replacement >> for `sun.misc.Unsafe::ensureClassInitialized`.The Lookup object must have >> the access to the given class being initialized. >> >> CSR: https://bugs.openjdk.java.net/browse/JDK-8245871 >> >> webrev: >> http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8235521/webrev.01/ >> >> This patch converts the JDK use of `jdk.internal.misc.Unsafe::ensureClassInitialized` to >> call this new API where appropriate. `Unsafe::ensureClassInitialized` remains for java.lang.invoke.* internal implementation use. >> >> A framework can use `Lookup::privateLookupIn` to access a class in another module if the module authorizes it by opening the packages for it to access. Or a user can produce a less privileged lookup by Lookup::dropLookupMode and pass it to the framework. >> >> `Class::ensureInitialized` was considered as an alternative. The downside for this option is that it would be caller sensitive in order to ensure the caller has the proper access to this class. And it may not work for frameworks which have no access to user classes. It would have to fallback toClass::forName` limited solution. >> >> Thanks >> Mandy >> From fguillaume at nuxeo.com Thu Jun 4 19:31:08 2020 From: fguillaume at nuxeo.com (Florent Guillaume) Date: Thu, 4 Jun 2020 21:31:08 +0200 Subject: Review Request: JDK-8235521: Replacement API for Unsafe::ensureClassInitialized In-Reply-To: <1db50e44-b0bc-6e98-22a1-a826c2a3bbc1@oracle.com> References: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> <1db50e44-b0bc-6e98-22a1-a826c2a3bbc1@oracle.com> Message-ID: Hi, Why can't it just return void? Florent On Thu, Jun 4, 2020 at 9:22 PM Mandy Chung wrote: > > > On 6/4/20 9:43 AM, Chris Hegarty wrote: > > Mandy, > > > > I think this looks good. Just a few minor comments... > > > > The spec wording is hard, since the method is offering a conditional > initialization. I guess I expected to see something like ?initializes > targetClass, if it has not been initialized already?, or some such. > > I added: > "This method causes {@code targetClass} to be initialized if it has > not been already initialized." > > to make it clear. > > > > The input and output is the same class instance. I don?t quite get that > from the spec. Can these be coupled together more obviously by their > descriptions? Maybe "@return targetClass, that is initialized" > > @return the class that has been initialized. > > I changed it to: > * @return {@code targetClass} that has been initialized > > Mandy > > > > -Chris. > > > >> On 4 Jun 2020, at 00:16, Mandy Chung wrote: > >> > >> This proposes a new `Lookup::ensureClassInitialized` API as a > replacement > >> for `sun.misc.Unsafe::ensureClassInitialized`.The Lookup object must > have > >> the access to the given class being initialized. > >> > >> CSR: https://bugs.openjdk.java.net/browse/JDK-8245871 > >> > >> webrev: > >> http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8235521/webrev.01/ > >> > >> This patch converts the JDK use of > `jdk.internal.misc.Unsafe::ensureClassInitialized` to > >> call this new API where appropriate. `Unsafe::ensureClassInitialized` > remains for java.lang.invoke.* internal implementation use. > >> > >> A framework can use `Lookup::privateLookupIn` to access a class in > another module if the module authorizes it by opening the packages for it > to access. Or a user can produce a less privileged lookup by > Lookup::dropLookupMode and pass it to the framework. > >> > >> `Class::ensureInitialized` was considered as an alternative. The > downside for this option is that it would be caller sensitive in order to > ensure the caller has the proper access to this class. And it may not work > for frameworks which have no access to user classes. It would have to > fallback toClass::forName` limited solution. > >> > >> Thanks > >> Mandy > >> > > -- [image: Nuxeo Logo] Florent Guillaume Head of R&D [image: LinkedIn] [image: Twitter] [image: Github] Nuxeo Content Services Platform. Stay ahead. From mandy.chung at oracle.com Thu Jun 4 19:31:38 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 4 Jun 2020 12:31:38 -0700 Subject: Review Request: JDK-8235521: Replacement API for Unsafe::ensureClassInitialized In-Reply-To: References: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> Message-ID: JVMS 5.5 is the spec for class initialization.? Step 3 describes the recursive request for initialization. I have @jvms 5.5 in the javadoc. I can make it explicit: ???????? * Ensures that {@code targetClass} has been initialized. The class ???????? * to be initialized must be {@linkplain #accessClass accessible} ???????? * to this {@code Lookup} object.? This method causes {@code targetClass} ???????? * to be initialized if it has not been already initialized, ???????? * as specified in JVMS {@jvms 5.5}. There are detailed rules about class initialization in JVMS 5.5. Mandy On 6/4/20 12:13 PM, Paul Sandoz wrote: > Hi Mandy, > > What about this case: > > class Test { > static { > MethodHandles.lookup().ensureClassInitialized(Test.class); > // not yet initialized > } > } > > (I see you do that for m/p1/A.java and Test.clinitInvokeEnsureClassInitialized.) > > Do we need mention this in the spec? e.g. this method does nothing if the class is already initialized or if the class is in the process of being initialized? > > Paul. > >> On Jun 3, 2020, at 4:16 PM, Mandy Chung wrote: >> >> This proposes a new `Lookup::ensureClassInitialized` API as a replacement >> for `sun.misc.Unsafe::ensureClassInitialized`.The Lookup object must have >> the access to the given class being initialized. >> >> CSR: https://bugs.openjdk.java.net/browse/JDK-8245871 >> >> webrev: >> http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8235521/webrev.01/ >> >> This patch converts the JDK use of `jdk.internal.misc.Unsafe::ensureClassInitialized` to >> call this new API where appropriate. `Unsafe::ensureClassInitialized` remains for java.lang.invoke.* internal implementation use. >> >> A framework can use `Lookup::privateLookupIn` to access a class in another module if the module authorizes it by opening the packages for it to access. Or a user can produce a less privileged lookup by Lookup::dropLookupMode and pass it to the framework. >> >> `Class::ensureInitialized` was considered as an alternative. The downside for this option is that it would be caller sensitive in order to ensure the caller has the proper access to this class. And it may not work for frameworks which have no access to user classes. It would have to fallback toClass::forName` limited solution. >> >> Thanks >> Mandy >> From mandy.chung at oracle.com Thu Jun 4 19:33:05 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 4 Jun 2020 12:33:05 -0700 Subject: Review Request: JDK-8235521: Replacement API for Unsafe::ensureClassInitialized In-Reply-To: References: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> <1db50e44-b0bc-6e98-22a1-a826c2a3bbc1@oracle.com> Message-ID: Lookup::accessClass sets the precedence to return Class for method chaining.? So this new method follows the existing convention. Mandy On 6/4/20 12:31 PM, Florent Guillaume wrote: > Hi, > > Why can't it just return void? > > Florent > > On Thu, Jun 4, 2020 at 9:22 PM Mandy Chung > wrote: > > > > On 6/4/20 9:43 AM, Chris Hegarty wrote: > > Mandy, > > > > I think this looks good. Just a few minor comments... > > > > The spec wording is hard, since the method is offering a > conditional initialization. I guess I expected to see something > like ?initializes targetClass, if it has not been initialized > already?, or some such. > > I added: > ??? "This method causes {@code targetClass} to be initialized if > it has > not been already initialized." > > to make it clear. > > > > The input and output is the same class instance. I don?t quite > get that from the spec. Can these be coupled together more > obviously by their descriptions? Maybe "@return targetClass, that > is initialized" > > @return the class that has been initialized. > > I changed it to: > ????????? * @return {@code targetClass} that has been initialized > > Mandy > > > > -Chris. > > > >> On 4 Jun 2020, at 00:16, Mandy Chung > wrote: > >> > >> This proposes a new `Lookup::ensureClassInitialized` API as a > replacement > >> for `sun.misc.Unsafe::ensureClassInitialized`.The Lookup object > must have > >> the access to the given class being initialized. > >> > >> CSR: https://bugs.openjdk.java.net/browse/JDK-8245871 > >> > >> webrev: > >> http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8235521/webrev.01/ > >> > >> This patch converts the JDK use of > `jdk.internal.misc.Unsafe::ensureClassInitialized` to > >> call this new API where appropriate. > `Unsafe::ensureClassInitialized` remains for java.lang.invoke.* > internal implementation use. > >> > >> A framework can use `Lookup::privateLookupIn` to access a class > in another module if the module authorizes it by opening the > packages for it to access. Or a user can produce a less privileged > lookup by Lookup::dropLookupMode and pass it to the framework. > >> > >> `Class::ensureInitialized` was considered as an alternative. > The downside for this option is that it would be caller sensitive > in order to ensure the caller has the proper access to this > class.? And it may not work for frameworks which have no access to > user classes. It would have to fallback toClass::forName` limited > solution. > >> > >> Thanks > >> Mandy > >> > > > > -- > Nuxeo Logo > > Florent Guillaume Head of R&D LinkedIn > Twitter > Github > > Nuxeo Content Services Platform. Stay ahead. > From mark.reinhold at oracle.com Thu Jun 4 19:39:24 2020 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 04 Jun 2020 12:39:24 -0700 Subject: Review Request: JDK-8235521: Replacement API for Unsafe::ensureClassInitialized In-Reply-To: <2937098a-1c4c-4248-4ea4-362a43f85c15@oracle.com> References: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> <2937098a-1c4c-4248-4ea4-362a43f85c15@oracle.com> Message-ID: <20200604123924.648026308@eggemoggin.niobe.net> 2020/6/3 19:31:13 -0700, mandy.chung at oracle.com: > On 6/3/20 5:16 PM, Paul Sandoz wrote: >> Did you consider an alternative name, such as: >> >> /** >> * Initializes {@code targetClass}, if not already initialized. >> * ... >> */ >> public Class initializeClass(Class targetClass) ... >> >> ? > > I considered this. The targetClass may have been initialized. > `initializeClass` seems to be read that it should check if it's > initialized before calling?? I also considered `ensureInitialized` > which can be an option. I'm open to suggestion. I agree that the `ensure` prefix is useful here. This method can only ensure that a class is initialized, so including `Class` in the method name seems redundant. I?d go with the shorter `ensureInitialized`. - Mark From paul.sandoz at oracle.com Thu Jun 4 20:05:46 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 4 Jun 2020 13:05:46 -0700 Subject: Review Request: JDK-8235521: Replacement API for Unsafe::ensureClassInitialized In-Reply-To: References: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> Message-ID: > On Jun 4, 2020, at 12:31 PM, Mandy Chung wrote: > > JVMS 5.5 is the spec for class initialization. Step 3 describes the recursive request for initialization. > Ok. (From quick offline chat we agreed its not worth explicitly highlighting this case.) > I have @jvms 5.5 in the javadoc. I can make it explicit: > > * Ensures that {@code targetClass} has been initialized. The class > * to be initialized must be {@linkplain #accessClass accessible} > * to this {@code Lookup} object. This method causes {@code targetClass} > * to be initialized if it has not been already initialized, > * as specified in JVMS {@jvms 5.5}. > LGTM, Paul. > There are detailed rules about class initialization in JVMS 5.5. > > Mandy > > On 6/4/20 12:13 PM, Paul Sandoz wrote: >> Hi Mandy, >> >> What about this case: >> >> class Test { >> static { >> MethodHandles.lookup().ensureClassInitialized(Test.class); >> // not yet initialized >> } >> } >> >> (I see you do that for m/p1/A.java and Test.clinitInvokeEnsureClassInitialized.) >> >> Do we need mention this in the spec? e.g. this method does nothing if the class is already initialized or if the class is in the process of being initialized? >> >> Paul. >> >>> On Jun 3, 2020, at 4:16 PM, Mandy Chung wrote: >>> >>> This proposes a new `Lookup::ensureClassInitialized` API as a replacement >>> for `sun.misc.Unsafe::ensureClassInitialized`.The Lookup object must have >>> the access to the given class being initialized. >>> >>> CSR: https://bugs.openjdk.java.net/browse/JDK-8245871 >>> >>> webrev: >>> http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8235521/webrev.01/ >>> >>> This patch converts the JDK use of `jdk.internal.misc.Unsafe::ensureClassInitialized` to >>> call this new API where appropriate. `Unsafe::ensureClassInitialized` remains for java.lang.invoke.* internal implementation use. >>> >>> A framework can use `Lookup::privateLookupIn` to access a class in another module if the module authorizes it by opening the packages for it to access. Or a user can produce a less privileged lookup by Lookup::dropLookupMode and pass it to the framework. >>> >>> `Class::ensureInitialized` was considered as an alternative. The downside for this option is that it would be caller sensitive in order to ensure the caller has the proper access to this class. And it may not work for frameworks which have no access to user classes. It would have to fallback toClass::forName` limited solution. >>> >>> Thanks >>> Mandy >>> > From mandy.chung at oracle.com Thu Jun 4 20:16:41 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 4 Jun 2020 13:16:41 -0700 Subject: Review Request: JDK-8235521: Replacement API for Unsafe::ensureClassInitialized In-Reply-To: <20200604123924.648026308@eggemoggin.niobe.net> References: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> <2937098a-1c4c-4248-4ea4-362a43f85c15@oracle.com> <20200604123924.648026308@eggemoggin.niobe.net> Message-ID: <6d407ced-5ccf-fa97-a46e-68aa2ee2599d@oracle.com> On 6/4/20 12:39 PM, mark.reinhold at oracle.com wrote: > I agree that the `ensure` prefix is useful here. > > This method can only ensure that a class is initialized, so including > `Class` in the method name seems redundant. I?d go with the shorter > `ensureInitialized`. > > - Mark Thanks for the advice. ? I go with the shorter name. I finalized an updated CSR per all feedbacks which includes the deprecation `sun.misc.Unsafe::shouldBeInitialized`. ?? https://bugs.openjdk.java.net/browse/JDK-8245871 updated webrev: http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8235521/webrev.02 thanks Mandy From alexey.semenyuk at oracle.com Thu Jun 4 20:22:24 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Thu, 4 Jun 2020 16:22:24 -0400 Subject: RFR: JDK-824662: Refactor JLinkBundlerHelper and StandardBundlerParam classes In-Reply-To: <8a3f9c48-9561-3a4a-7e07-53a9ebbc1539@oracle.com> References: <8a3f9c48-9561-3a4a-7e07-53a9ebbc1539@oracle.com> Message-ID: Please review fix [2] for jpackage bug [1]. Move functionality to collect data about app (main class name, main jar, module name, etc) from JLinkBundlerHelper and StandardBundlerParam classes in dedicated LauncherData class. Remove ArgAction, ModFile and RelativeFileSet classes as they are not needed any more after refactoring of JLinkBundlerHelper and StandardBundlerParam classes. Clean unused code. - Alexey [1] https://bugs.openjdk.java.net/browse/JDK-8246624 [2]?http://cr.openjdk.java.net/~asemenyuk/8246624/webrev.00 From alexey.semenyuk at oracle.com Thu Jun 4 20:46:31 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Thu, 4 Jun 2020 16:46:31 -0400 Subject: RFR: JDK-8246627: Consolidate app image bundlers In-Reply-To: References: Message-ID: <4bcba4b6-1eb2-5567-f614-0a31e4fbbcc7@oracle.com> Please review fix [2] for jpackage bug [1]. Move duplicated functionality from LinuxAppBundler, MacAppBundler and WinAppBundler classes in the base class. [2] webrev is on top of [3] webrev. - Alexey [1] https://bugs.openjdk.java.net/browse/JDK-8246627 [2]?http://cr.openjdk.java.net/~asemenyuk/8246627/webrev.00 [3]?http://cr.openjdk.java.net/~asemenyuk/8246624/webrev.00 From john.r.rose at oracle.com Thu Jun 4 22:49:47 2020 From: john.r.rose at oracle.com (John Rose) Date: Thu, 4 Jun 2020 15:49:47 -0700 Subject: Review Request: JDK-8235521: Replacement API for Unsafe::ensureClassInitialized In-Reply-To: <20200604123924.648026308@eggemoggin.niobe.net> References: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> <2937098a-1c4c-4248-4ea4-362a43f85c15@oracle.com> <20200604123924.648026308@eggemoggin.niobe.net> Message-ID: <14351E4E-77A6-41BC-A5A7-406B29AED7E2@oracle.com> On Jun 4, 2020, at 12:39 PM, mark.reinhold at oracle.com wrote: > > I agree that the `ensure` prefix is useful here. > > This method can only ensure that a class is initialized, so including > `Class` in the method name seems redundant. I?d go with the shorter > `ensureInitialized`. +1 From leonid.mesnik at oracle.com Thu Jun 4 22:55:46 2020 From: leonid.mesnik at oracle.com (Leonid Mesnik) Date: Thu, 4 Jun 2020 15:55:46 -0700 Subject: RFR(S/T) : 8245874 : requires.extraPropDefns.vmOpts doesn't need -Xbootclasspath/a:bootClasses In-Reply-To: References: <88ED55CF-092F-4FAC-ACFD-17D3CA869C75@oracle.com> Message-ID: <50a6910b-f566-5824-ca01-7ceea7562ddf@oracle.com> Looks good. Leonid On 6/4/20 10:21 AM, Igor Ignatyev wrote: > piny? > -- Igor > >> On Jun 1, 2020, at 7:54 AM, Igor Ignatyev wrote: >> >> ping? >> -- Igor >> >>> On May 27, 2020, at 10:25 AM, Igor Ignatyev wrote: >>> >>> http://cr.openjdk.java.net/~iignatyev//8245874/webrev.00 >>>> 8 lines changed: 2 ins; 0 del; 6 mod; >>> Hi all, >>> >>> could you please review this small and trivial cleanup in TEST.ROOT files of test/jdk/ and test/hotspot/jtreg test suites? >>> from JBS: >>>> to make sure that classes listed in `requires.extraPropDefns.bootlibs` are in BCP, jtreg automatically adds corresponding `-Xbootclasspath/a:` to the options list, so `requires.extraPropDefns.vmOpts` doesn't have to have `-Xbootclasspath/a:bootClasses` >>> the patch >>> - removes -Xbootclasspath/a:bootClasses from requires.extraPropDefns.vmOpts >>> - moves jdk.test.lib.Platform and Container classes from requires.extraPropDefns.bootlibs to requires.extraPropDefns.libs as there is no need for them to be in BCP >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8245874 >>> testing: tier1 >>> webrev: http://cr.openjdk.java.net/~iignatyev//8245874/webrev.00 >>> >>> Thanks, >>> -- Igor From john.r.rose at oracle.com Thu Jun 4 23:21:29 2020 From: john.r.rose at oracle.com (John Rose) Date: Thu, 4 Jun 2020 16:21:29 -0700 Subject: Review Request: JDK-8235521: Replacement API for Unsafe::ensureClassInitialized In-Reply-To: <5d0d7f6b-addf-ad01-6625-ad7688ad3b5f@oracle.com> References: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> <5d0d7f6b-addf-ad01-6625-ad7688ad3b5f@oracle.com> Message-ID: On Jun 3, 2020, at 7:38 PM, Mandy Chung wrote: > > On 6/3/20 6:24 PM, John Rose wrote: >> On Jun 3, 2020, at 5:16 PM, Paul Sandoz > wrote: >>> >>> if (UNSAFE.shouldBeInitialized(cls)) { >>> UNSAFE.ensureClassInitialized(cls); >>> } >>> >>> Although it seems redundant to perform the check, suggesting it is not needed. >> >> It?s useful (IIRC) in this kind of code: >> >> MethodHandle mh = ? something about cls ...; >> if (!UNSAFE.shouldBeInitialized(cls)) >> return mh; >> // Oops, every eventual call to mh needs to do the check: >> return filterArguments(mh, 0, UNSAFE_ensureClassInitialized.bindTo(cls)); >> > > It is useful for method handle machinery. I don't see the need to expose it as a supported API for java applications/framework. > > Maybe useful for dynamic languages? Yes, it would be. Since it?s a longer conversation, I suggest making a *non-public* API point on Lookup, isFullyInitialized(Class). Further suggestions: See DirectMethodHandle.ensureInitialized for the use of both Unsafe API points in concert; it has ticklish states involving blocking on the thread doing the initialization. I don?t want to expose that yet. But, factor all uses of those two Unsafe API in DMH.java through the new Lookup API points, a public ensureInitialized (I like that name best; it?s the one adopted internally to java.lang.invoke) and also through a new non-public isFullyInitialized. To avoid access checks use IMPL_LOOKUP. Note the inverted boolean sense. The idea is that if isFullyInitialized returns true, then there?s no point in arranging for future calls to ensureInitialized. If there is true demand for the second API point we can roll it out as public. But there?s a catch: There?s a third initialization state, ?being initialized? which a dynamic language will also want to care about. So there?s a need for a third API point of some sort, not covered by the second. Here?s a quick thought about that third API point: As things stand, we are going to allow Lookup.findVirtual to apply to Lookup::ensureInitialized to return a suitable access point, for future use, for anyone to use to ?tweak? any given class (modulo access restrictions). Good. We can secretly add an optimization to the particular MH (and no other) so that it uses the same machinery as MH?s returned for static getters and setters and invokers that incorporate an init-barrier. I?m not saying do this now, but maybe file a follow-up bug to add that optimization to the resulting MH for Lookup::ensureInit, if and when we add Lookup::isFullyInitialized. That will give the dynamic language people the same ?hooks? we enjoy internally. The workflow would be: static final MethodHandle MH_ensureInit = publicLookup().findVirtual(L?, ?ensureInitialized??); MethodHandle mh = ?; Class target = ?; if (!lookup().isFullyInitialized(target)) { // we need to tack a ?before action? onto mh to trigger inits: MethodHandle before = MH_ensureInit.bindTo(lookup()).bindTo(target); mh = MethodHandles.collectArguments(mh, 0, before); } return mh; ? John From alexander.matveev at oracle.com Thu Jun 4 23:22:52 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Thu, 4 Jun 2020 16:22:52 -0700 Subject: JDK-8238204: run_tests.sh fails on macOS when called from test_jpackage.sh In-Reply-To: <2603829e-0ce1-1d03-8dbf-1294a5b78d42@oracle.com> References: <8a3f9c48-9561-3a4a-7e07-53a9ebbc1539@oracle.com> <2603829e-0ce1-1d03-8dbf-1294a5b78d42@oracle.com> Message-ID: Hi Alexey, Looks good. Thanks, Alexander On 6/4/20 9:21 AM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > Replace xargs call with --no-run-if-empty parameter with bash > expressions in run_tests.sh > Call run_tests.sh from test_jpackager.sh in a way to avoid shebang > interpretation that doesn't work properly in the environment without > bash in the PATH. > > Both scripts are used to run jpackage tests in local builds. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8238204 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8238204/webrev.00 > From alexander.matveev at oracle.com Thu Jun 4 23:33:52 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Thu, 4 Jun 2020 16:33:52 -0700 Subject: RFR: JDK-824662: Refactor JLinkBundlerHelper and StandardBundlerParam classes In-Reply-To: References: <8a3f9c48-9561-3a4a-7e07-53a9ebbc1539@oracle.com> Message-ID: Hi Alexey, Looks good. Thanks, Alexander On 6/4/20 1:22 PM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > Move functionality to collect data about app (main class name, main > jar, module name, etc) from JLinkBundlerHelper and > StandardBundlerParam classes in dedicated LauncherData class. > Remove ArgAction, ModFile and RelativeFileSet classes as they are not > needed any more after refactoring of JLinkBundlerHelper and > StandardBundlerParam classes. > Clean unused code. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8246624 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8246624/webrev.00 > From john.r.rose at oracle.com Thu Jun 4 23:35:03 2020 From: john.r.rose at oracle.com (John Rose) Date: Thu, 4 Jun 2020 16:35:03 -0700 Subject: Review Request: JDK-8235521: Replacement API for Unsafe::ensureClassInitialized In-Reply-To: References: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> <5d0d7f6b-addf-ad01-6625-ad7688ad3b5f@oracle.com> Message-ID: P.S.C. (post-send clarification) > The workflow would be: > > static final MethodHandle MH_ensureInit > = publicLookup().findVirtual(L?, ?ensureInitialized??); > By that I mean the workflow of the dynamic language implementor. And after hitting ?send? I realized that optimizing that one case (of a findVirtual -> bindTo -> bindTo) is harder than I thought. The entire workflow below could instead be a call to a wrapping function that takes an arbitrary ?mh? (as below) and wraps an init-barrier onto the front *if needed*. It would have to take a Lookup and a Class, plus the mh. It would handle all the internal stuff. If the user wanted a bare init MH, just pass in an empty MH as a ?seed?. And maybe take a short-circuit action if the ?seed? comes back unchanged; this is an alternative to isFullyInitialized, but it seems sneaky. class Lookup { ? /** * To the given method handle, prepend an action, if necessary, to trigger * initialization of the given class the first time the method handle is called. * If it would be illegal to call ensureInitialized on the given class from this lookup, * immediately report an error. Otherwise, if the class is already fully initialized * return the method handle unchanged. Otherwise, return a new method handle * (of the same type) which incorporates the necessary initialization action, * as if by a call to ensureInitialized. */ MethodHandle ensureInitialized(Class target, MethodHandle mh) { checkAccess(target); if (isFullyInitialized(target)) return mh; // we need to tack a ?before action? onto mh to trigger inits: MethodHandle before = MH_ensureInit.bindTo(target); return collectArguments(mh, 0, before); } // note: IMPL_LOOKUP requires previous checkAccess call private static final MethodHandle MH_ensureInit = publicLookup().findVirtual(L?, ?ensureInitialized??).bindTo(IMPL_LOOKUP); From andy.herrick at oracle.com Fri Jun 5 00:01:06 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Thu, 4 Jun 2020 20:01:06 -0400 Subject: RFR: 8236282: [macos] Find permanent solution to macOS test timeout problem JDK-8235738 In-Reply-To: <5cd5af78-b0a7-a6d6-2c73-871958dc8a1a@oracle.com> References: <5cd5af78-b0a7-a6d6-2c73-871958dc8a1a@oracle.com> Message-ID: looks good. /Andy On 6/4/2020 3:03 AM, alexander.matveev at oracle.com wrote: > Please review the jpackage fix for bug [1] at [2]. > > This is still a workaround for JDK-8236825. Implemented as per Roger > suggestion to redirect output to temp file. I did a lot of testing > using repro case and was not able to reproduce hang anymore. > > [1] https://bugs.openjdk.java.net/browse/JDK-8236282 > [2] http://cr.openjdk.java.net/~almatvee/8236282/webrev.01/ > > Thanks, > Alexander From alexey.semenyuk at oracle.com Fri Jun 5 00:24:58 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Thu, 4 Jun 2020 20:24:58 -0400 Subject: RFR: 8236282: [macos] Find permanent solution to macOS test timeout problem JDK-8235738 In-Reply-To: <5cd5af78-b0a7-a6d6-2c73-871958dc8a1a@oracle.com> References: <5cd5af78-b0a7-a6d6-2c73-871958dc8a1a@oracle.com> Message-ID: <2908870b-2545-b07a-3f74-c8029886fb1f@oracle.com> Looks good. - Alexey On 6/4/2020 3:03 AM, alexander.matveev at oracle.com wrote: > Please review the jpackage fix for bug [1] at [2]. > > This is still a workaround for JDK-8236825. Implemented as per Roger > suggestion to redirect output to temp file. I did a lot of testing > using repro case and was not able to reproduce hang anymore. > > [1] https://bugs.openjdk.java.net/browse/JDK-8236282 > [2] http://cr.openjdk.java.net/~almatvee/8236282/webrev.01/ > > Thanks, > Alexander From alexander.matveev at oracle.com Fri Jun 5 00:30:28 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Thu, 4 Jun 2020 17:30:28 -0700 Subject: RFR: JDK-8246627: Consolidate app image bundlers In-Reply-To: <4bcba4b6-1eb2-5567-f614-0a31e4fbbcc7@oracle.com> References: <4bcba4b6-1eb2-5567-f614-0a31e4fbbcc7@oracle.com> Message-ID: Hi Alexey, Looks good. Thanks, Alexander On 6/4/20 1:46 PM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > Move duplicated functionality from LinuxAppBundler, MacAppBundler and > WinAppBundler classes in the base class. > > [2] webrev is on top of [3] webrev. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8246627 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8246627/webrev.00 > > [3]?http://cr.openjdk.java.net/~asemenyuk/8246624/webrev.00 From mandy.chung at oracle.com Fri Jun 5 00:56:02 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 4 Jun 2020 17:56:02 -0700 Subject: Review Request: JDK-8235521: Replacement API for Unsafe::ensureClassInitialized In-Reply-To: References: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> <5d0d7f6b-addf-ad01-6625-ad7688ad3b5f@oracle.com> Message-ID: I create https://bugs.openjdk.java.net/browse/JDK-8246634 to track this. thanks Mandy On 6/4/20 4:35 PM, John Rose wrote: > P.S.C. (post-send clarification) > >> The workflow would be: >> >> static final MethodHandle MH_ensureInit >> = publicLookup().findVirtual(L?, ?ensureInitialized??); >> > By that I mean the workflow of the dynamic language > implementor. And after hitting ?send? I realized that optimizing > that one case (of a findVirtual -> bindTo -> bindTo) is harder > than I thought. The entire workflow below could instead be > a call to a wrapping function that takes an arbitrary ?mh? > (as below) and wraps an init-barrier onto the front *if needed*. > It would have to take a Lookup and a Class, plus the mh. > It would handle all the internal stuff. If the user wanted > a bare init MH, just pass in an empty MH as a ?seed?. > And maybe take a short-circuit action if the ?seed? comes > back unchanged; this is an alternative to isFullyInitialized, > but it seems sneaky. > > class Lookup { ? > /** > * To the given method handle, prepend an action, if necessary, to trigger > * initialization of the given class the first time the method handle is called. > * If it would be illegal to call ensureInitialized on the given class from this lookup, > * immediately report an error. Otherwise, if the class is already fully initialized > * return the method handle unchanged. Otherwise, return a new method handle > * (of the same type) which incorporates the necessary initialization action, > * as if by a call to ensureInitialized. > */ > MethodHandle ensureInitialized(Class target, MethodHandle mh) { > checkAccess(target); > if (isFullyInitialized(target)) return mh; > // we need to tack a ?before action? onto mh to trigger inits: > MethodHandle before = MH_ensureInit.bindTo(target); > return collectArguments(mh, 0, before); > } > > // note: IMPL_LOOKUP requires previous checkAccess call > private static final MethodHandle MH_ensureInit > = publicLookup().findVirtual(L?, ?ensureInitialized??).bindTo(IMPL_LOOKUP); From calvin.cheung at oracle.com Fri Jun 5 01:48:18 2020 From: calvin.cheung at oracle.com (Calvin Cheung) Date: Thu, 4 Jun 2020 18:48:18 -0700 Subject: RFR(XL) 8198698: Support Lambda proxy classes in dynamic CDS archive In-Reply-To: <2557d547-96db-6329-a515-1357cf4a4111@oracle.com> References: <7da8b0f7-1027-d033-c1b5-7deb9d1d6c09@oracle.com> <90ea692b-4a34-19e4-8f42-9f4092dc40e1@oracle.com> <74fa5de3-5f9d-be20-37b7-9ea5a15bf211@oracle.com> <2557d547-96db-6329-a515-1357cf4a4111@oracle.com> Message-ID: Hi Mandy, Thanks for taking another look. On 6/3/20 2:07 PM, Mandy Chung wrote: > > > On 6/3/20 12:34 PM, Calvin Cheung wrote: >> >> I saw David has commented on this. So I'll leave the assert as before >> and I've added another assert (see line 1691): >> >> 1687?? // The following ensures that the caller's nest host is the >> same as the lambda proxy's >> 1688?? // nest host recorded at dump time. >> 1689?? assert(nest_host->has_nest_member(caller_ik, THREAD) || >> 1690????????? nest_host == caller_ik, "caller_ik failed nest member >> check"); >> > > I don't think this assert is needed.? caller_ik can be a hidden class > and so this assert is not correct then. I've removed it. > > Is there any issue to archive lambda proxy class whose caller is a > hidden class?? Is there any assumption in the CDS implementation that > the caller class is always a normal class? I've added a check in JVM_RegisterLambdaProxyClassForArchiving. If the caller class is hidden or vm anonymous, it will return. I also added 2 test cases to test the above. If the caller class is a hidden class, the test makes sure the corresponding lambda proxy class is not being archived. Currently, it doesn't seem possible to have a vm anonymous class to be the caller class of a lambda proxy class. I've added a test anyway so if things change later, we'll notice it. > >> 1691?? assert(nest_host == shared_nest_host, "mismatched nest host"); >> > > This is good. > >> >> In SystemDictionary::load_shared_lambda_proxy_class, it checks the flag: >> >> 1422?? if (initialize) { >> 1423???? loaded_ik->initialize(CHECK_NULL); >> 1424?? } >> > > > I think JVM_LookupLambdaProxyClassFromArchive is a more appropriate > place to link and initialize the class before return.?? I expect > load_shared_lambda_proxy_class does loading only and linking and > initialization should be separate from loading. Instead of putting the post loading code in the JVM_LookupLambdaProxyClassFromArchive function which would require changing some of the functions from private to public, I've renamed SystemDictionaryShared::load_shared_lambda_proxy_class to SystemDictionaryShared::prepare_shared_lambda_proxy class and moved the code there. > >> On a related note, in the existing jvm_lookup_define_class in jvm.cpp: >> >> ? if (init) { >> ??? ik->initialize(CHECK_NULL); >> ? } else { >> ??? ik->link_class(CHECK_NULL); >> ? } >> >> I don't think the else is necessary as the ik->link_class(CHECK_NULL) >> has been done within the SystemDictionary::parse_stream. >> > > Harold and Lois can chime in here.? I think ik->link_class may be for > unsafe anonymous class to prepare for constant pool patching. > >> Currently, the strong vs weak hidden class isn't recorded in the archive. >> >> : >> >> ----- >> >> For now, I've added an assert in >> JVM_RegisterLambdaProxyClassForArchiving to make sure the hidden >> class is strong so that when things changed later, we'll notice it. >> > > An assert is good. > > > 3752?? if (invokedName == NULL || invokedType == NULL || methodType == > NULL || > 3753?????? implMethodMember == NULL || instantiatedMethodType == NULL) { > 3754???? return NULL; > 3755?? } > > > Should this throw NPE instead? I've made the change. Updated webrevs: inc: http://cr.openjdk.java.net/~ccheung/jdk15/8198698/webrev_delta.02/ full: http://cr.openjdk.java.net/~ccheung/jdk15/8198698/webrev.02/ thanks, Calvin From weijun.wang at oracle.com Fri Jun 5 03:41:55 2020 From: weijun.wang at oracle.com (Weijun Wang) Date: Fri, 5 Jun 2020 11:41:55 +0800 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> Message-ID: Hi Alexey, It's so unfortunate that different addressType must be used. I'm OK with the new TlsChannelBindingImpl class. One thing I'm not comfortable is the java.security.sasl/share/classes/module-info.java change. We've struggled hard to avoid these kind of secret tunnels. Is it possible to pass the tlsCB.getData() directly to the SASL mechanism? The property name is clear enough to avoid any misuse. Another question: can an application user set the "com.sun.security.sasl.tlschannelbinding" property? and can they read it after it's set internally? I cannot think of a good attack now but at least it seems they have no need to access that property value. If we keep it internal then we also have the chance to modify the approach later. Thanks, Max > On Jun 5, 2020, at 2:15 AM, Alexey Bakhtin wrote: > > Hello, > > Could you please review new version of the patch: > http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v3/ > > I?ve moved all logic for creating TLS Channel Binding data out of GssKrb5Client.java file. > All data is prepared inside TlsChannelBinding class. > Internal property name is renamed to ?com.sun.security.sasl.tlschannelbinding?. > TlsChannelBinding object is still used to pass channel binding data from LdapClient to GssKrb5Client. > I do not change it to byte[] because of TlsChannelBinding class is still used for internal property name. > Also, I?ve updated TlsChannelBinding class to select SHA-256 hash algorithm if it can not be derived > from the certificate signature name. > > Regards > Alexey > > >> On 26 May 2020, at 17:50, Weijun Wang wrote: >> >> I have a question on GssKrb5Client.java: >> >> Do you think it's a good idea to let the SASL mechanism understand what TLS binding is? Instead of passing the whole TlsChannelBinding object through a SASL property, is it possible to only pass the actual cbData? After all, the name "com.sun.security.sasl.channelbinding" suggests it's just a general ChannelBinding which is independent with any application level info. >> >> From my reading of the code change, it looks like this cbData can be calculated on the LDAP side. >> >> Thanks, >> Max >> >>> On May 21, 2020, at 3:35 PM, Alexey Bakhtin wrote: >>> >>> Hello, >>> >>> Could you please review the following patch: >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8245527 >>> Webrev: http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v0/ >>> >>> The Windows LDAP server with LdapEnforceChannelBinding=2 uses the tls-server-end-point channel binding >>> (based on the TLS server certificate). The channel binding data is calculated as following : >>> ? The client calculates a hash of the TLS server certificate. >>> The hash algorithm is selected on the base of the certificate signature algorithm. >>> Also, the client should use SHA-256 algorithm, in case of the certificate signature algorithm is SHA1 or MD5 based >>> ? The channel binding information is the same as defined in rfc4121 [1] with small corrections: >>> ? initiator and acceptor addresses should be set to NULL >>> ? initiator and acceptor address types should be zero. >>> It contradicts to the ?Using Channel Bindings in GSS-API? document [2] that say that >>> the address type should be set to GSS_C_AF_NULLADDR=0xFF, >>> instead of GSS_C_AF_UNSPEC=0x00. >>> >>> This patch introduces changes in the LDAP, SASL and JGSS modules >>> to generate channel binding data automatically if requested by an application. >>> This patch reuses existing org.ietf.jgss.ChannelBinding class implementation but changes >>> initial unspecified address type from CHANNEL_BINDING_AF_NULL_ADDR to CHANNEL_BINDING_AF_UNSPEC. >>> The patch introduces new environment property ?com.sun.jndi.ldap.tls.cbtype? that indicates >>> Channel Binding type that should be used in the LDAPS connection over JGSS/Kerberos. >>> Right now "tls-server-end-point" Channel Binding type is supported only. >>> The client extracts server certificate from the underlying TLS connection and creates >>> Channel Binding data for JGSS/Kerberos authentication if application indicates >>> com.sun.jndi.ldap.tls.cbtype=tls-server-end-point property. >>> Client application should also specify existing "com.sun.jndi.ldap.connect.timeout? property >>> to force and wait TLS handshake completion before JGSS/Kerberos authentication data is generated. >>> >>> [1] - https://tools.ietf.org/html/rfc4121#section-4.1.1.2 >>> >>> [2] - >>> https://docs.oracle.com/cd/E19120-01/open.solaris/819-2145/overview-52/index.html >>> >>> Initial discussion of this issue is available at security-dev list: >>> https://mail.openjdk.java.net/pipermail/security-dev/2019-December/021052.html >>> https://mail.openjdk.java.net/pipermail/security-dev/2020-January/021140.html >>> https://mail.openjdk.java.net/pipermail/security-dev/2020-February/021278.html >>> https://mail.openjdk.java.net/pipermail/security-dev/2020-May/021864.html > From kim.barrett at oracle.com Fri Jun 5 07:52:44 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 5 Jun 2020 03:52:44 -0400 Subject: RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot Message-ID: [Changes are only to the build system, but since the changes have jdk-wide effect I?ve cc?d what I think are the relevant dev lists.] This change is part of JEP 347; the intent is to target JDK 16. Please review this change to the building of C++ code in the JDK to enable the use of C++14 language features. This change does not make any code changes to use new features provided by C++11 or C++14. This requires trimming the supported compiler versions, moving the minimum supported versions forward (significantly, in some cases). The new minimums are based on compiler documentation rather than testing. It may be that more recent versions are actually required. This change needs to be tested on platforms not supported by Oracle. The JEP test plan includes additional Oracle testing beyond what I?ve done. CR: https://bugs.openjdk.java.net/browse/JDK-8246032 Webrev: https://cr.openjdk.java.net/~kbarrett/8246032/open.02/ Testing: mach5 tier1-5 on Oracle supported platforms. Performance testing showed no significant changes in either direction. Build-only (no tests) for linux-arm32, linux-s390x, linux-ppc64le From vladimir.x.ivanov at oracle.com Fri Jun 5 08:02:17 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 5 Jun 2020 11:02:17 +0300 Subject: RFR(S/T) : 8245874 : requires.extraPropDefns.vmOpts doesn't need -Xbootclasspath/a:bootClasses In-Reply-To: References: Message-ID: Looks good. Best regards, Vladimir Ivanov On 27.05.2020 20:25, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8245874/webrev.00 >> 8 lines changed: 2 ins; 0 del; 6 mod; > > Hi all, > > could you please review this small and trivial cleanup in TEST.ROOT files of test/jdk/ and test/hotspot/jtreg test suites? > from JBS: >> to make sure that classes listed in `requires.extraPropDefns.bootlibs` are in BCP, jtreg automatically adds corresponding `-Xbootclasspath/a:` to the options list, so `requires.extraPropDefns.vmOpts` doesn't have to have `-Xbootclasspath/a:bootClasses` > the patch > - removes -Xbootclasspath/a:bootClasses from requires.extraPropDefns.vmOpts > - moves jdk.test.lib.Platform and Container classes from requires.extraPropDefns.bootlibs to requires.extraPropDefns.libs as there is no need for them to be in BCP > > JBS: https://bugs.openjdk.java.net/browse/JDK-8245874 > testing: tier1 > webrev: http://cr.openjdk.java.net/~iignatyev//8245874/webrev.00 > > Thanks, > -- Igor > From chris.hegarty at oracle.com Fri Jun 5 08:42:06 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 5 Jun 2020 09:42:06 +0100 Subject: Review Request: JDK-8235521: Replacement API for Unsafe::ensureClassInitialized In-Reply-To: <6d407ced-5ccf-fa97-a46e-68aa2ee2599d@oracle.com> References: <5cd559df-10eb-4e58-db7a-c30b9ca5c250@oracle.com> <2937098a-1c4c-4248-4ea4-362a43f85c15@oracle.com> <20200604123924.648026308@eggemoggin.niobe.net> <6d407ced-5ccf-fa97-a46e-68aa2ee2599d@oracle.com> Message-ID: <3DB61443-06DD-49E4-86F7-5AB843A6AD10@oracle.com> > On 4 Jun 2020, at 21:16, Mandy Chung wrote: > > ... > I finalized an updated CSR per all feedbacks which includes the deprecation `sun.misc.Unsafe::shouldBeInitialized`. > https://bugs.openjdk.java.net/browse/JDK-8245871 > > updated webrev: > http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8235521/webrev.02 Thanks Mandy. Looks good to me. -Chris. From Roger.Riggs at oracle.com Fri Jun 5 09:23:57 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Fri, 5 Jun 2020 05:23:57 -0400 Subject: RFR: 8246592: Simplify checking of boolean file attributes In-Reply-To: <2e321139-5b59-6b43-6e9c-fdba6f881d0c@oracle.com> References: <9a005eac-d701-15db-1ad4-4aa1b4f00498@oracle.com> <57cc9714-41e2-c879-febc-805f2e66b2fc@oracle.com> <96578c15-9de5-2208-3a64-a664fe659cde@oracle.com> <91f22718-7477-4f61-3428-70ed79af6f9e@oracle.com> <2e321139-5b59-6b43-6e9c-fdba6f881d0c@oracle.com> Message-ID: <71bcc4b4-63cf-c086-cf38-59b462d26a4c@oracle.com> Hi Claes, Also, since the requested attribute is a bit mask the test should be that all of the attributes are present, not just some, iIt might prevent a mistake down the line. Probably the method name and argument names should be plural. Line: 120: should be 119 public boolean hasBooleanAttributes(File f, int attributes) { 120 return (getBooleanAttributes(f) & attributes) == attributes; 121 } Thanks, Roger On 6/4/20 2:24 PM, Claes Redestad wrote: > Hi Roger, > > On 2020-06-04 18:05, Roger Riggs wrote: >> Hi Claes, >> >> The code looks ok but I would rename the new method to >> 'hasBooleanAttribute' >> since it is not returning the attribute but testing for it. > > good suggestion: > > http://cr.openjdk.java.net/~redestad/8246592/open.01/ > >> >> I think you can leave the 'public' modifiers to a separate issue. > > Ok. > > /Claes > >> >> Regards, Roger >> >> >> >> >> On 6/4/20 9:41 AM, Claes Redestad wrote: >>> >>> >>> On 2020-06-04 15:26, Alan Bateman wrote: >>>> On 04/06/2020 14:22, Roger Riggs wrote: >>>>> Hi Claes, >>>>> >>>>> Not a review... >>>>> >>>>> You'll need a CSR for the API change. >>>>> >>>>> FileSystem.java: You'll need proper javadoc for the new >>>>> getBooleanAttribute method. >>>> FileSystem is not a public class so it's not actually an API >>>> change. I agree the public modifier is confusing, should probably >>>> be dropped from all these methods. >>> >>> If you prefer I can strip out all the public modifiers. Or do that as a >>> follow-up. >>> >>> /Claes >> From claes.redestad at oracle.com Fri Jun 5 10:09:03 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 5 Jun 2020 12:09:03 +0200 Subject: RFR: 8246592: Simplify checking of boolean file attributes In-Reply-To: <71bcc4b4-63cf-c086-cf38-59b462d26a4c@oracle.com> References: <9a005eac-d701-15db-1ad4-4aa1b4f00498@oracle.com> <57cc9714-41e2-c879-febc-805f2e66b2fc@oracle.com> <96578c15-9de5-2208-3a64-a664fe659cde@oracle.com> <91f22718-7477-4f61-3428-70ed79af6f9e@oracle.com> <2e321139-5b59-6b43-6e9c-fdba6f881d0c@oracle.com> <71bcc4b4-63cf-c086-cf38-59b462d26a4c@oracle.com> Message-ID: Hi Roger, yes, things were a bit inconsistent whether this is checking all or one. I went with all (even though all usage only asks for one) and updated to use plural consistently. /Claes On 2020-06-05 11:23, Roger Riggs wrote: > Hi Claes, > > Also, since the requested attribute is a bit mask the test should be > that all of the attributes are present, not just some, iIt might prevent > a mistake down the line. > Probably the method name and argument names should be plural. > > Line: 120: should be > > 119 public boolean hasBooleanAttributes(File f, int attributes) { 120 > return (getBooleanAttributes(f) & attributes) == attributes; 121 } > > Thanks, Roger > > > On 6/4/20 2:24 PM, Claes Redestad wrote: >> Hi Roger, >> >> On 2020-06-04 18:05, Roger Riggs wrote: >>> Hi Claes, >>> >>> The code looks ok but I would rename the new method to >>> 'hasBooleanAttribute' >>> since it is not returning the attribute but testing for it. >> >> good suggestion: >> >> http://cr.openjdk.java.net/~redestad/8246592/open.01/ >> >>> >>> I think you can leave the 'public' modifiers to a separate issue. >> >> Ok. >> >> /Claes >> >>> >>> Regards, Roger >>> >>> >>> >>> >>> On 6/4/20 9:41 AM, Claes Redestad wrote: >>>> >>>> >>>> On 2020-06-04 15:26, Alan Bateman wrote: >>>>> On 04/06/2020 14:22, Roger Riggs wrote: >>>>>> Hi Claes, >>>>>> >>>>>> Not a review... >>>>>> >>>>>> You'll need a CSR for the API change. >>>>>> >>>>>> FileSystem.java: You'll need proper javadoc for the new >>>>>> getBooleanAttribute method. >>>>> FileSystem is not a public class so it's not actually an API >>>>> change. I agree the public modifier is confusing, should probably >>>>> be dropped from all these methods. >>>> >>>> If you prefer I can strip out all the public modifiers. Or do that as a >>>> follow-up. >>>> >>>> /Claes >>> > From james.laskey at oracle.com Fri Jun 5 11:59:25 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Fri, 5 Jun 2020 08:59:25 -0300 Subject: RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: References: Message-ID: <84EEE234-A21F-400B-917C-8DF55236E38E@oracle.com> I know there was a discussion about this elsewhere but I would like to take the opportunity to correct this now make//autoconf/flags-cflags.m4:241 elif test "x$TOOLCHAIN_TYPE" = xclang; then if test "x$OPENJDK_TARGET_OS" = xmacosx; then # On MacOSX we optimize for size, something # we should do for all platforms? C_O_FLAG_HIGHEST_JVM="-Os" C_O_FLAG_HIGHEST="-Os" C_O_FLAG_HI="-Os" C_O_FLAG_NORM="-Os" C_O_FLAG_DEBUG_JVM="" else C_O_FLAG_HIGHEST_JVM="-O3" C_O_FLAG_HIGHEST="-O3" C_O_FLAG_HI="-O3" C_O_FLAG_NORM="-O2" C_O_FLAG_DEBUG_JVM="-O0" fi C_O_FLAG_SIZE="-Os" C_O_FLAG_DEBUG="-O0" C_O_FLAG_NONE="-O0" elif test "x$TOOLCHAIN_TYPE" = xxlc; then should be changed to elif test "x$TOOLCHAIN_TYPE" = xclang; then C_O_FLAG_HIGHEST_JVM="-O3" C_O_FLAG_HIGHEST="-O3" C_O_FLAG_HI="-O3" C_O_FLAG_NORM="-O2" C_O_FLAG_DEBUG_JVM="-O0" C_O_FLAG_SIZE="-Os" C_O_FLAG_DEBUG="-O0" C_O_FLAG_NONE="-O0" elif test "x$TOOLCHAIN_TYPE" = xxlc; then MacOSX has been paying a historic and significant performance penalty for no valid reason. Otherwise +1. Cheers, -- Jim > On Jun 5, 2020, at 4:52 AM, Kim Barrett wrote: > > [Changes are only to the build system, but since the changes have jdk-wide > effect I?ve cc?d what I think are the relevant dev lists.] > > This change is part of JEP 347; the intent is to target JDK 16. > > Please review this change to the building of C++ code in the JDK to > enable the use of C++14 language features. This change does not make > any code changes to use new features provided by C++11 or C++14. > > This requires trimming the supported compiler versions, moving the > minimum supported versions forward (significantly, in some cases). > The new minimums are based on compiler documentation rather than > testing. It may be that more recent versions are actually required. > > This change needs to be tested on platforms not supported by Oracle. > The JEP test plan includes additional Oracle testing beyond what I?ve done. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8246032 > > Webrev: > https://cr.openjdk.java.net/~kbarrett/8246032/open.02/ > > Testing: > mach5 tier1-5 on Oracle supported platforms. > > Performance testing showed no significant changes in either direction. > > Build-only (no tests) for linux-arm32, linux-s390x, linux-ppc64le > From Roger.Riggs at oracle.com Fri Jun 5 12:18:06 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Fri, 5 Jun 2020 08:18:06 -0400 Subject: RFR: 8246592: Simplify checking of boolean file attributes In-Reply-To: References: <9a005eac-d701-15db-1ad4-4aa1b4f00498@oracle.com> <57cc9714-41e2-c879-febc-805f2e66b2fc@oracle.com> <96578c15-9de5-2208-3a64-a664fe659cde@oracle.com> <91f22718-7477-4f61-3428-70ed79af6f9e@oracle.com> <2e321139-5b59-6b43-6e9c-fdba6f881d0c@oracle.com> <71bcc4b4-63cf-c086-cf38-59b462d26a4c@oracle.com> Message-ID: Looks good. Thanks for the extra attention. Roger On 6/5/20 6:09 AM, Claes Redestad wrote: > Hi Roger, > > yes, things were a bit inconsistent whether this is checking all or one. > I went with all (even though all usage only asks for one) and updated > to use plural consistently. > > /Claes > > On 2020-06-05 11:23, Roger Riggs wrote: >> Hi Claes, >> >> Also, since the requested attribute is a bit mask the test should be >> that all of the attributes are present, not just some, iIt might >> prevent a mistake down the line. >> Probably the method name and argument names should be plural. >> >> Line: 120: should be >> >> 119 public boolean hasBooleanAttributes(File f, int attributes) { 120 >> return (getBooleanAttributes(f) & attributes) == attributes; 121 } >> >> Thanks, Roger >> >> >> On 6/4/20 2:24 PM, Claes Redestad wrote: >>> Hi Roger, >>> >>> On 2020-06-04 18:05, Roger Riggs wrote: >>>> Hi Claes, >>>> >>>> The code looks ok but I would rename the new method to >>>> 'hasBooleanAttribute' >>>> since it is not returning the attribute but testing for it. >>> >>> good suggestion: >>> >>> http://cr.openjdk.java.net/~redestad/8246592/open.01/ >>> >>>> >>>> I think you can leave the 'public' modifiers to a separate issue. >>> >>> Ok. >>> >>> /Claes >>> >>>> >>>> Regards, Roger >>>> >>>> >>>> >>>> >>>> On 6/4/20 9:41 AM, Claes Redestad wrote: >>>>> >>>>> >>>>> On 2020-06-04 15:26, Alan Bateman wrote: >>>>>> On 04/06/2020 14:22, Roger Riggs wrote: >>>>>>> Hi Claes, >>>>>>> >>>>>>> Not a review... >>>>>>> >>>>>>> You'll need a CSR for the API change. >>>>>>> >>>>>>> FileSystem.java: You'll need proper javadoc for the new >>>>>>> getBooleanAttribute method. >>>>>> FileSystem is not a public class so it's not actually an API >>>>>> change. I agree the public modifier is confusing, should probably >>>>>> be dropped from all these methods. >>>>> >>>>> If you prefer I can strip out all the public modifiers. Or do that >>>>> as a >>>>> follow-up. >>>>> >>>>> /Claes >>>> >> From claes.redestad at oracle.com Fri Jun 5 12:18:57 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 5 Jun 2020 14:18:57 +0200 Subject: RFR: 8246592: Simplify checking of boolean file attributes In-Reply-To: References: <9a005eac-d701-15db-1ad4-4aa1b4f00498@oracle.com> <57cc9714-41e2-c879-febc-805f2e66b2fc@oracle.com> <96578c15-9de5-2208-3a64-a664fe659cde@oracle.com> <91f22718-7477-4f61-3428-70ed79af6f9e@oracle.com> <2e321139-5b59-6b43-6e9c-fdba6f881d0c@oracle.com> <71bcc4b4-63cf-c086-cf38-59b462d26a4c@oracle.com> Message-ID: <02c37ba7-cda5-0706-0f04-8cecdbf073e9@oracle.com> Thanks, Roger! On 2020-06-05 14:18, Roger Riggs wrote: > Looks good. From andy.herrick at oracle.com Fri Jun 5 13:07:28 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Fri, 5 Jun 2020 09:07:28 -0400 Subject: RFR: JDK-824662: Refactor JLinkBundlerHelper and StandardBundlerParam classes In-Reply-To: References: <8a3f9c48-9561-3a4a-7e07-53a9ebbc1539@oracle.com> Message-ID: looks OK. /Andy On 6/4/2020 4:22 PM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > Move functionality to collect data about app (main class name, main > jar, module name, etc) from JLinkBundlerHelper and > StandardBundlerParam classes in dedicated LauncherData class. > Remove ArgAction, ModFile and RelativeFileSet classes as they are not > needed any more after refactoring of JLinkBundlerHelper and > StandardBundlerParam classes. > Clean unused code. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8246624 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8246624/webrev.00 > From christoph.dreis at freenet.de Fri Jun 5 13:42:16 2020 From: christoph.dreis at freenet.de (Christoph Dreis) Date: Fri, 05 Jun 2020 15:42:16 +0200 Subject: Class.getCanonicalName() returns null for lambdas Message-ID: Hi, forgive me if this was discussed already, but I found that the implementation of hidden classes causes a change in behavior for Class.getCanonicalName(). When using the following example code: interface Function { void doSomething(); } public class Example { public static void main(String[] args) { Class lambdaClass = ((Function) () -> {}).getClass(); String canonicalName = lambdaClass.getCanonicalName(); System.out.println(canonicalName); String name = lambdaClass.getName(); System.out.println(name); } } I get the following results: JDK 14 com.example.demo.Example$$Lambda$14/0x0000000800b66840 com.example.demo.Example$$Lambda$14/0x0000000800b66840 JDK 15 null com.example.demo.Example$$Lambda$14/0x0000000800b89448 As far as I can tell from the code Lambdas are considered hidden classes now. Is that correct? I guess Class.getName() should be used instead? Cheers, Christoph From igor.ignatyev at oracle.com Fri Jun 5 14:37:06 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 5 Jun 2020 07:37:06 -0700 Subject: RFR(S/T) : 8245874 : requires.extraPropDefns.vmOpts doesn't need -Xbootclasspath/a:bootClasses In-Reply-To: References: Message-ID: <78150322-D75B-4BDA-BD07-6715DE99D0E3@oracle.com> Vladimir, Leonid, Thank you for your reviews! pushed. -- Igor > On Jun 4, 2020, at 3:55 PM, Leonid Mesnik wrote: > > Looks good. > > Leonid > On Jun 5, 2020, at 1:02 AM, Vladimir Ivanov wrote: > > Looks good. > > Best regards, > Vladimir Ivanov > > On 27.05.2020 20:25, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8245874/webrev.00 >>> 8 lines changed: 2 ins; 0 del; 6 mod; >> Hi all, >> could you please review this small and trivial cleanup in TEST.ROOT files of test/jdk/ and test/hotspot/jtreg test suites? >> from JBS: >>> to make sure that classes listed in `requires.extraPropDefns.bootlibs` are in BCP, jtreg automatically adds corresponding `-Xbootclasspath/a:` to the options list, so `requires.extraPropDefns.vmOpts` doesn't have to have `-Xbootclasspath/a:bootClasses` >> the patch >> - removes -Xbootclasspath/a:bootClasses from requires.extraPropDefns.vmOpts >> - moves jdk.test.lib.Platform and Container classes from requires.extraPropDefns.bootlibs to requires.extraPropDefns.libs as there is no need for them to be in BCP >> JBS: https://bugs.openjdk.java.net/browse/JDK-8245874 >> testing: tier1 >> webrev: http://cr.openjdk.java.net/~iignatyev//8245874/webrev.00 >> Thanks, >> -- Igor From james.laskey at oracle.com Fri Jun 5 14:41:41 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Fri, 5 Jun 2020 11:41:41 -0300 Subject: RFR: JDK-8246697 Test: java/util/StringJoiner/StringJoinerTest.java failing with OOM Message-ID: Can I get a quick review for the following. Getting some unexpected OOM from test. Pulling until I figure it out. jbs: https://bugs.openjdk.java.net/browse/JDK-8246696 diff -r 345693999a92 test/jdk/ProblemList.txt --- a/test/jdk/ProblemList.txt Fri Jun 05 11:35:06 2020 -0300 +++ b/test/jdk/ProblemList.txt Fri Jun 05 11:38:29 2020 -0300 @@ -883,6 +883,7 @@ # jdk_util java/util/ServiceLoader/ReloadTest.java 8242935 generic-all +java/util/StringJoiner/StringJoinerTest.java 8246697 generic-all ############################################################################ From sundararajan.athijegannathan at oracle.com Fri Jun 5 14:44:10 2020 From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com) Date: Fri, 5 Jun 2020 20:14:10 +0530 Subject: RFR: JDK-8246697 Test: java/util/StringJoiner/StringJoinerTest.java failing with OOM In-Reply-To: References: Message-ID: Looks good. -Sundar On 05/06/20 8:11 pm, Jim Laskey wrote: > Can I get a quick review for the following. Getting some unexpected OOM from test. Pulling until I figure it out. > > jbs: https://bugs.openjdk.java.net/browse/JDK-8246696 > > diff -r 345693999a92 test/jdk/ProblemList.txt > --- a/test/jdk/ProblemList.txt Fri Jun 05 11:35:06 2020 -0300 > +++ b/test/jdk/ProblemList.txt Fri Jun 05 11:38:29 2020 -0300 > @@ -883,6 +883,7 @@ > # jdk_util > > java/util/ServiceLoader/ReloadTest.java 8242935 generic-all > +java/util/StringJoiner/StringJoinerTest.java 8246697 generic-all > > ############################################################################ > From igor.ignatyev at oracle.com Fri Jun 5 14:48:33 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 5 Jun 2020 07:48:33 -0700 Subject: RFR: JDK-8246697 Test: java/util/StringJoiner/StringJoinerTest.java failing with OOM In-Reply-To: References: Message-ID: Hi Jim, you should use another bug id, one which you are going to use to fix the problem, in the problem list. -- Igor > On Jun 5, 2020, at 7:41 AM, Jim Laskey wrote: > > Can I get a quick review for the following. Getting some unexpected OOM from test. Pulling until I figure it out. > > jbs: https://bugs.openjdk.java.net/browse/JDK-8246696 > > diff -r 345693999a92 test/jdk/ProblemList.txt > --- a/test/jdk/ProblemList.txt Fri Jun 05 11:35:06 2020 -0300 > +++ b/test/jdk/ProblemList.txt Fri Jun 05 11:38:29 2020 -0300 > @@ -883,6 +883,7 @@ > # jdk_util > > java/util/ServiceLoader/ReloadTest.java 8242935 generic-all > +java/util/StringJoiner/StringJoinerTest.java 8246697 generic-all > > ############################################################################ > From james.laskey at oracle.com Fri Jun 5 14:51:49 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Fri, 5 Jun 2020 11:51:49 -0300 Subject: RFR: JDK-8246696 Test: java/util/StringJoiner/StringJoinerTest.java failing with OOM In-Reply-To: References: Message-ID: <42A59492-E569-4B73-B71F-880667F812C9@oracle.com> Right. The request and the content is right the subject of the e-mail was wrong. Sorry about that. > On Jun 5, 2020, at 11:48 AM, Igor Ignatyev wrote: > > Hi Jim, > > you should use another bug id, one which you are going to use to fix the problem, in the problem list. > > -- Igor > >> On Jun 5, 2020, at 7:41 AM, Jim Laskey wrote: >> >> Can I get a quick review for the following. Getting some unexpected OOM from test. Pulling until I figure it out. >> >> jbs: https://bugs.openjdk.java.net/browse/JDK-8246696 >> >> diff -r 345693999a92 test/jdk/ProblemList.txt >> --- a/test/jdk/ProblemList.txt Fri Jun 05 11:35:06 2020 -0300 >> +++ b/test/jdk/ProblemList.txt Fri Jun 05 11:38:29 2020 -0300 >> @@ -883,6 +883,7 @@ >> # jdk_util >> >> java/util/ServiceLoader/ReloadTest.java 8242935 generic-all >> +java/util/StringJoiner/StringJoinerTest.java 8246697 generic-all >> >> ############################################################################ >> > From alexey at azul.com Fri Jun 5 15:03:10 2020 From: alexey at azul.com (Alexey Bakhtin) Date: Fri, 5 Jun 2020 15:03:10 +0000 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> Message-ID: Hello Max, Thank you a lot for review. Could you check the new version of the patch : http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v4/ I?ve made the following changes: - TlsChannelBinding class is moved to java.naming module. java.security.sasl module is not affected any more - I pass tlsCB.getData() directly to the SASL mechanism as you suggested - I?ve made some guards to prevent application from using "com.sun.security.sasl.tlschannelbinding? property directly: - LdapClient verifies if internal property is not set - GssKrb5Client uses props.remove() to read and remove internal property Regards Alexey > On 5 Jun 2020, at 06:41, Weijun Wang wrote: > > Hi Alexey, > > It's so unfortunate that different addressType must be used. I'm OK with the new TlsChannelBindingImpl class. > > One thing I'm not comfortable is the java.security.sasl/share/classes/module-info.java change. We've struggled hard to avoid these kind of secret tunnels. Is it possible to pass the tlsCB.getData() directly to the SASL mechanism? The property name is clear enough to avoid any misuse. > > Another question: can an application user set the "com.sun.security.sasl.tlschannelbinding" property? and can they read it after it's set internally? I cannot think of a good attack now but at least it seems they have no need to access that property value. If we keep it internal then we also have the chance to modify the approach later. > > Thanks, > Max > > >> On Jun 5, 2020, at 2:15 AM, Alexey Bakhtin wrote: >> >> Hello, >> >> Could you please review new version of the patch: >> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v3/ >> >> I?ve moved all logic for creating TLS Channel Binding data out of GssKrb5Client.java file. >> All data is prepared inside TlsChannelBinding class. >> Internal property name is renamed to ?com.sun.security.sasl.tlschannelbinding?. >> TlsChannelBinding object is still used to pass channel binding data from LdapClient to GssKrb5Client. >> I do not change it to byte[] because of TlsChannelBinding class is still used for internal property name. >> Also, I?ve updated TlsChannelBinding class to select SHA-256 hash algorithm if it can not be derived >> from the certificate signature name. >> >> Regards >> Alexey >> >> >>> On 26 May 2020, at 17:50, Weijun Wang wrote: >>> >>> I have a question on GssKrb5Client.java: >>> >>> Do you think it's a good idea to let the SASL mechanism understand what TLS binding is? Instead of passing the whole TlsChannelBinding object through a SASL property, is it possible to only pass the actual cbData? After all, the name "com.sun.security.sasl.channelbinding" suggests it's just a general ChannelBinding which is independent with any application level info. >>> >>> From my reading of the code change, it looks like this cbData can be calculated on the LDAP side. >>> >>> Thanks, >>> Max >>> >>>> On May 21, 2020, at 3:35 PM, Alexey Bakhtin wrote: >>>> >>>> Hello, >>>> >>>> Could you please review the following patch: >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8245527 >>>> Webrev: http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v0/ >>>> >>>> The Windows LDAP server with LdapEnforceChannelBinding=2 uses the tls-server-end-point channel binding >>>> (based on the TLS server certificate). The channel binding data is calculated as following : >>>> ? The client calculates a hash of the TLS server certificate. >>>> The hash algorithm is selected on the base of the certificate signature algorithm. >>>> Also, the client should use SHA-256 algorithm, in case of the certificate signature algorithm is SHA1 or MD5 based >>>> ? The channel binding information is the same as defined in rfc4121 [1] with small corrections: >>>> ? initiator and acceptor addresses should be set to NULL >>>> ? initiator and acceptor address types should be zero. >>>> It contradicts to the ?Using Channel Bindings in GSS-API? document [2] that say that >>>> the address type should be set to GSS_C_AF_NULLADDR=0xFF, >>>> instead of GSS_C_AF_UNSPEC=0x00. >>>> >>>> This patch introduces changes in the LDAP, SASL and JGSS modules >>>> to generate channel binding data automatically if requested by an application. >>>> This patch reuses existing org.ietf.jgss.ChannelBinding class implementation but changes >>>> initial unspecified address type from CHANNEL_BINDING_AF_NULL_ADDR to CHANNEL_BINDING_AF_UNSPEC. >>>> The patch introduces new environment property ?com.sun.jndi.ldap.tls.cbtype? that indicates >>>> Channel Binding type that should be used in the LDAPS connection over JGSS/Kerberos. >>>> Right now "tls-server-end-point" Channel Binding type is supported only. >>>> The client extracts server certificate from the underlying TLS connection and creates >>>> Channel Binding data for JGSS/Kerberos authentication if application indicates >>>> com.sun.jndi.ldap.tls.cbtype=tls-server-end-point property. >>>> Client application should also specify existing "com.sun.jndi.ldap.connect.timeout? property >>>> to force and wait TLS handshake completion before JGSS/Kerberos authentication data is generated. >>>> >>>> [1] - https://tools.ietf.org/html/rfc4121#section-4.1.1.2 >>>> >>>> [2] - >>>> https://docs.oracle.com/cd/E19120-01/open.solaris/819-2145/overview-52/index.html >>>> >>>> Initial discussion of this issue is available at security-dev list: >>>> https://mail.openjdk.java.net/pipermail/security-dev/2019-December/021052.html >>>> https://mail.openjdk.java.net/pipermail/security-dev/2020-January/021140.html >>>> https://mail.openjdk.java.net/pipermail/security-dev/2020-February/021278.html >>>> https://mail.openjdk.java.net/pipermail/security-dev/2020-May/021864.html >> From daniel.fuchs at oracle.com Fri Jun 5 15:17:42 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 5 Jun 2020 16:17:42 +0100 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> Message-ID: <85417480-7ecd-21d9-fb59-ff3d9a442360@oracle.com> Hi Alexey, Could we move the new code from LdapClient.java and LdapCtxt.java into the com.sun.jndi.ldap.sasl package, and possibly delay its execution until the com.sun.jndi.ldap.sasl.LdapSasl.saslBind method is called? The new TlsChannelBinding class should also be preferably moved to com.sun.jndi.ldap.sasl since it's apparently only useful with SASL. Also: 2573 if (cbTypeProp.equals(TlsChannelBindingType.TLS_UNIQUE.getName())) { 2574 throw new UnsupportedOperationException( "Channel binding type " + 2575 TlsChannelBindingType.TLS_UNIQUE.getName() + 2576 " is not supported"); 2577 } else if (cbTypeProp.equals(TlsChannelBindingType.TLS_SERVER_END_POINT.getName())) { 2578 if (connectTimeout == -1) 2579 throw new IllegalStateException(CHANNEL_BINDING_TYPE + " property requires " + 2580 CONNECT_TIMEOUT + " property is set."); 2581 cbType = TlsChannelBindingType.TLS_SERVER_END_POINT; 2582 } else { 2583 throw new IllegalArgumentException("Illegal value for " + 2584 CHANNEL_BINDING_TYPE + " property."); 2585 } is not correct - as IllegalArgumentException, IllegalStateException and UnsupportedOperationException will percolate up to the calling code, and are not documented at the public API level. These should really be NamingException. best regards, -- daniel On 05/06/2020 16:03, Alexey Bakhtin wrote: > Hello Max, > > Thank you a lot for review. > > Could you check the new version of the patch : > http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v4/ > > I?ve made the following changes: > - TlsChannelBinding class is moved to java.naming module. > java.security.sasl module is not affected any more > - I pass tlsCB.getData() directly to the SASL mechanism as you suggested > - I?ve made some guards to prevent application from using "com.sun.security.sasl.tlschannelbinding? property directly: > - LdapClient verifies if internal property is not set > - GssKrb5Client uses props.remove() to read and remove internal property > > Regards > Alexey From conor.cleary at oracle.com Fri Jun 5 15:20:16 2020 From: conor.cleary at oracle.com (Conor Cleary) Date: Fri, 5 Jun 2020 16:20:16 +0100 Subject: RFR[8243655]: 'Map.replace javadoc code snippet typo' Message-ID: Hi, Could someone please take a look at my webrev for JDK-8243655 'Map.replace javadoc code snippet typo'? This fix addresses a typo in the javadoc for Map.replace(K, V, V) in the default implementation code. The parameter 'value' in the conditional block is changed to 'oldValue' to reflect the originally passed variable name. bug: https://bugs.openjdk.java.net/browse/JDK-8243655 webrev: http://cr.openjdk.java.net/~jboes/ccleary/webrevs/8243655/webrev.00/ Regards, Conor From james.laskey at oracle.com Fri Jun 5 15:23:13 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Fri, 5 Jun 2020 12:23:13 -0300 Subject: RFR[8243655]: 'Map.replace javadoc code snippet typo' In-Reply-To: References: Message-ID: <32E014ED-31B9-4B20-9665-70D866032285@oracle.com> +1 > On Jun 5, 2020, at 12:20 PM, Conor Cleary wrote: > > Hi, > > Could someone please take a look at my webrev for JDK-8243655 'Map.replace javadoc code snippet typo'? > > This fix addresses a typo in the javadoc for Map.replace(K, V, V) in the default implementation code. The parameter 'value' in the conditional block is changed to 'oldValue' to reflect the originally passed variable name. > > > bug: https://bugs.openjdk.java.net/browse/JDK-8243655 > webrev: http://cr.openjdk.java.net/~jboes/ccleary/webrevs/8243655/webrev.00/ > > > Regards, > Conor > From martinrb at google.com Fri Jun 5 15:54:46 2020 From: martinrb at google.com (Martin Buchholz) Date: Fri, 5 Jun 2020 08:54:46 -0700 Subject: RFR[8243655]: 'Map.replace javadoc code snippet typo' In-Reply-To: References: Message-ID: Looks good to me On Fri, Jun 5, 2020 at 8:20 AM Conor Cleary wrote: > > Hi, > > Could someone please take a look at my webrev for JDK-8243655 > 'Map.replace javadoc code snippet typo'? > > This fix addresses a typo in the javadoc for Map.replace(K, V, V) in the > default implementation code. The parameter 'value' in the conditional > block is changed to 'oldValue' to reflect the originally passed variable > name. > > > bug: https://bugs.openjdk.java.net/browse/JDK-8243655 > webrev: http://cr.openjdk.java.net/~jboes/ccleary/webrevs/8243655/webrev.00/ > > > Regards, > Conor > From pavel.rappo at oracle.com Fri Jun 5 15:55:53 2020 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Fri, 5 Jun 2020 16:55:53 +0100 Subject: RFR[8243655]: 'Map.replace javadoc code snippet typo' In-Reply-To: References: Message-ID: <0BA14637-EB7B-4198-B4CD-3A4F275E0342@oracle.com> Extra points would be to substitute the resulting snippet for the actual implementation to see if it compiles and satisfies the tests. For example, like this: default boolean replace(K key, V oldValue, V newValue) { Map map = this; // synthetic line if (map.containsKey(key) && Objects.equals(map.get(key), oldValue)) { map.put(key, newValue); return true; } else return false; } One of these days snippets/examples may become verifiable. Until then we should check them in doc comments that are most read, occasionally, in a manual fashion. Otherwise, looks good. -Pavel > On 5 Jun 2020, at 16:20, Conor Cleary wrote: > > Hi, > > Could someone please take a look at my webrev for JDK-8243655 'Map.replace javadoc code snippet typo'? > > This fix addresses a typo in the javadoc for Map.replace(K, V, V) in the default implementation code. The parameter 'value' in the conditional block is changed to 'oldValue' to reflect the originally passed variable name. > > > bug: https://bugs.openjdk.java.net/browse/JDK-8243655 > webrev: http://cr.openjdk.java.net/~jboes/ccleary/webrevs/8243655/webrev.00/ > > > Regards, > Conor > From alexey at azul.com Fri Jun 5 16:33:57 2020 From: alexey at azul.com (Alexey Bakhtin) Date: Fri, 5 Jun 2020 16:33:57 +0000 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <85417480-7ecd-21d9-fb59-ff3d9a442360@oracle.com> References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> <85417480-7ecd-21d9-fb59-ff3d9a442360@oracle.com> Message-ID: <4F4BA16D-C9A4-46DD-A9A9-979E35431600@azul.com> Hi Daniel, Thank you for review Yes, I can move TlsChannelBinding class into the com.sun.jndi.ldap.sasl package and LdapClient related changes into the LdapSasl.saslBind method. Also, you are right with exceptions. I will rename them to the NamingException. However, I?d like to parse TLS Channel Binding property in the LdapCtx class. The reason is ?com.sun.jndi.ldap.connect.timeout? property. This property should be set together with TLS Channel Binding. So, I?d like to verify if both properties are set before connection is started. The best place for it is LdapCtx.initEnv() Is it acceptable ? Thank you Alexey > On 5 Jun 2020, at 18:17, Daniel Fuchs wrote: > > Hi Alexey, > > Could we move the new code from LdapClient.java and LdapCtxt.java > into the com.sun.jndi.ldap.sasl package, and possibly delay > its execution until the com.sun.jndi.ldap.sasl.LdapSasl.saslBind > method is called? > > The new TlsChannelBinding class should also be preferably moved > to com.sun.jndi.ldap.sasl since it's apparently only useful > with SASL. > > Also: > > 2573 if (cbTypeProp.equals(TlsChannelBindingType.TLS_UNIQUE.getName())) { > 2574 throw new UnsupportedOperationException( "Channel binding type " + > 2575 TlsChannelBindingType.TLS_UNIQUE.getName() + > 2576 " is not supported"); > 2577 } else if (cbTypeProp.equals(TlsChannelBindingType.TLS_SERVER_END_POINT.getName())) { > 2578 if (connectTimeout == -1) > 2579 throw new IllegalStateException(CHANNEL_BINDING_TYPE + " property requires " + > 2580 CONNECT_TIMEOUT + " property is set."); > 2581 cbType = TlsChannelBindingType.TLS_SERVER_END_POINT; > 2582 } else { > 2583 throw new IllegalArgumentException("Illegal value for " + > 2584 CHANNEL_BINDING_TYPE + " property."); > 2585 } > > is not correct - as IllegalArgumentException, IllegalStateException and UnsupportedOperationException will percolate up to the calling code, and > are not documented at the public API level. > These should really be NamingException. > > best regards, > > -- daniel > > > > On 05/06/2020 16:03, Alexey Bakhtin wrote: >> Hello Max, >> Thank you a lot for review. >> Could you check the new version of the patch : >> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v4/ >> I?ve made the following changes: >> - TlsChannelBinding class is moved to java.naming module. >> java.security.sasl module is not affected any more >> - I pass tlsCB.getData() directly to the SASL mechanism as you suggested >> - I?ve made some guards to prevent application from using "com.sun.security.sasl.tlschannelbinding? property directly: >> - LdapClient verifies if internal property is not set >> - GssKrb5Client uses props.remove() to read and remove internal property >> Regards >> Alexey From mandy.chung at oracle.com Fri Jun 5 16:36:19 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 5 Jun 2020 09:36:19 -0700 Subject: Class.getCanonicalName() returns null for lambdas In-Reply-To: References: Message-ID: <0e7b792a-3282-3935-5a5c-8e4d7e6eeda2@oracle.com> On 6/5/20 6:42 AM, Christoph Dreis wrote: > Hi, > > forgive me if this was discussed already, but I found that the implementation of hidden classes causes a change in behavior for Class.getCanonicalName(). A hidden class has no canonical name [1] and it can't be nominally referenced (like an anonymous class which has no canonical name). Lambda proxy classes are now hidden classes in JDK 15 but that's implementation details.?? For diagnosiability, Class::getName or Class::toString can be used. Mandy [1] https://download.java.net/java/early_access/jdk15/docs/api/java.base/java/lang/Class.html#getCanonicalName() > When using the following example code: > > interface Function { > void doSomething(); > } > > public class Example { > > public static void main(String[] args) { > Class lambdaClass = ((Function) () -> {}).getClass(); > String canonicalName = lambdaClass.getCanonicalName(); > System.out.println(canonicalName); > String name = lambdaClass.getName(); > System.out.println(name); > } > > } > > I get the following results: > > JDK 14 > com.example.demo.Example$$Lambda$14/0x0000000800b66840 > com.example.demo.Example$$Lambda$14/0x0000000800b66840 > > JDK 15 > null > com.example.demo.Example$$Lambda$14/0x0000000800b89448 > > As far as I can tell from the code Lambdas are considered hidden classes now. Is that correct? > > I guess Class.getName() should be used instead? > > Cheers, > Christoph > > From peter.kessler at os.amperecomputing.com Thu Jun 4 23:08:40 2020 From: peter.kessler at os.amperecomputing.com (Peter Kessler OS) Date: Thu, 4 Jun 2020 23:08:40 +0000 Subject: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) Message-ID: <814DA850-76A9-4526-A3D9-141CD516C9AE@amperemail.onmicrosoft.com> ObjectInputStream.resolveClass(ObjectStreamClass) is specified to find "the first class loader on the current thread's stack (starting from the currently executing method) that is neither the platform class loader nor its ancestor." Finding that class loader is done with a virtual frame walk in JVM_LatestUserDefinedLoader(JNIEnv *env). The virtual frame walk is expensive. If an application does not define any ClassLoaders, then the virtual frame walk will always find the system class loader. If an application does a significant amount of reading from an ObjectInputStream, the cost of ObjectInputStream.resolveClass(ObjectInputStreamClass) can be considerable. In a sample application, that method (and the methods it calls) have been measured to consume 5+% of the CPU cycles. The proposal is to note the construction of any user-defined ClassLoader, and if none has been constructed, avoid the virtual frame walk returning ClassLoader.getSystemClassLoader(). But if any user-defined ClassLoader has been constructed, use the virtual frame walk to find the first user-defined ClassLoader on the thread stack. User-defined ClassLoaders could be distinguished from non-user-defined ClassLoaders in several ways. The ClassLoader class hierarchy could be changed to make non-user-defined ClassLoaders inherit from a marker class. There is already a jdk.internal.loader.BuiltinClassLoader but it is used for module (and resource) loading. BuiltinClassLoader is not a super-class of jdk.internal.reflect.DelegatingClassLoader, which is used to load reflection classes, and reflection methods are also ignored by the virtual frame walk. ClassLoaders could have a boolean that distinguished whether they are user-defined or not. That would require a change to ClassLoader API. There are other ways to distinguish user-defined ClassLoaders. The proposal is to keep the decoration of the ClassLoaders out of the instances, so that it is only visible to the code that needs it. The proposal is that each non-user-defined ClassLoader have a static block that registers the class as a non-user-defined ClassLoader. Then in the base ClassLoader constructor check if the ClassLoader being constructed is a user-defined ClassLoader and if so set a boolean that acts as the guard on the virtual frame walk. If additional non-user-defined ClassLoaders are declared in the future beyond the ones identified in this patch, they can be added to the registry. There are currently 4 non-user-defined ClassLoader classes, so the registry is not difficult to maintain. Nor is the registry difficult to search when ClassLoaders are constructed. The guard boolean that records whether a user-defined ClassLoader has been constructed transitions from false to true and never becomes false again. (An extension to transition the boolean back to false when a user-defined ClassLoader becomes unreachable is beyond the scope of this proposal.) This proposal slows loading of non-user-defined ClassLoaders by the time it takes to register them. It slows ClassLoader construction by the time it takes to consult the registry and conditionally set the guard boolean. At each invocation of ObjectInputStream.resolveClass(ObjectInputStreamClass), the guard boolean is used to possibly avoid the virtual frame walk. Tested with `make run-test-tier1` on Linux (CentOS 7) on both aarch64 and x86_64. I had one failure on each of the runs ACTION: main -- Failed. Execution failed: `main' threw exception: java.lang.Error: TESTBUG: the library has not been found in ${Repo}/build/linux-aarch64-server-release/images/test/hotspot/jtreg/native REASON: User specified action: run main/native GTestWrapper on both plaforms. But I get the same failure when I run the tier1 tests on clones of OpenJDK-15+25 on both aarch64 and x86_64. Running a sample (representative?) application with the Oracle Developer Studio analyzer shows a performance comparison of Stack Fragment sorted by metric: Attributed Total CPU Time baseline.er experiment.er Attributed Attributed Name Total CPU Time Total CPU Time sec. sec. ================================== Callers 3459.210 29.931 java.io.ObjectInputStream.readOrdinaryObject(boolean) 1139.727 3.532 java.io.ObjectInputStream.readArray(boolean) 875.262 9.116 java.io.ObjectInputStream.readNonProxyDesc(boolean) ================================== Stack Fragment java.io.ObjectInputStream.readClassDesc(boolean) java.io.ObjectInputStream.readNonProxyDesc(boolean) java.io.ObjectInputStream.resolveClass(java.io.ObjectStreamClass) java.io.ObjectInputStream.latestUserDefinedLoader() 4.173 3.953 jdk.internal.misc.VM.latestUserDefinedLoader() ================================== Callees 5470.026 0. jdk.internal.misc.VM.latestUserDefinedLoader0() 0. 38.627 java.lang.ClassLoader.getSystemClassLoader() The `hg export -g` below is with respect to OpenJDK-15+25. Thank you for your review comments. I will need a sponsor to get this change into the repository. ... peter # HG changeset patch # User Peter Kessler # Date 1591310467 25200 # Thu Jun 04 15:41:07 2020 -0700 # Node ID 9a39488182c1dfc5bc7bb41d562d7ef16ee657f6 # Parent 90b266a84c06f1b3dc0ed8767856793e8c1c357e Improve the performance of ObjectInputStream.resolveClass diff --git a/src/java.base/share/classes/java/lang/ClassLoader.java b/src/java.base/share/classes/java/lang/ClassLoader.java --- a/src/java.base/share/classes/java/lang/ClassLoader.java +++ b/src/java.base/share/classes/java/lang/ClassLoader.java @@ -385,6 +385,8 @@ } this.package2certs = new ConcurrentHashMap<>(); this.nameAndId = nameAndId(this); + /* Note the construction of a ClassLoader. */ + VM.noteClassLoaderConstruction(this.getClass()); } /** diff --git a/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java b/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java --- a/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java +++ b/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java @@ -117,6 +117,11 @@ protected Class loadClassOrNull(String cn, boolean resolve) { return JLA.findBootstrapClassOrNull(this, cn); } + + static { + /* Register this ClassLoader as not a user-defined ClassLoader. */ + VM.registerNotUserDefinedClassLoader(BootClassLoader.class); + } }; /** @@ -127,6 +132,8 @@ static { if (!ClassLoader.registerAsParallelCapable()) throw new InternalError(); + /* Register this ClassLoader as not a user-defined ClassLoader. */ + VM.registerNotUserDefinedClassLoader(PlatformClassLoader.class); } PlatformClassLoader(BootClassLoader parent) { @@ -142,6 +149,8 @@ static { if (!ClassLoader.registerAsParallelCapable()) throw new InternalError(); + /* Register this ClassLoader as not a user-defined ClassLoader. */ + VM.registerNotUserDefinedClassLoader(AppClassLoader.class); } final URLClassPath ucp; diff --git a/src/java.base/share/classes/jdk/internal/misc/VM.java b/src/java.base/share/classes/jdk/internal/misc/VM.java --- a/src/java.base/share/classes/jdk/internal/misc/VM.java +++ b/src/java.base/share/classes/jdk/internal/misc/VM.java @@ -304,12 +304,44 @@ private static final int JVMTI_THREAD_STATE_WAITING_INDEFINITELY = 0x0010; private static final int JVMTI_THREAD_STATE_WAITING_WITH_TIMEOUT = 0x0020; + /** A registry of not user-defined ClassLoaders. */ + private static final List> notUserDefinedClassLoaderRegistry = + Collections.synchronizedList(new ArrayList<>()); + + /** Register a ClassLoader class as being not a user-defined ClassLoader. */ + public static void registerNotUserDefinedClassLoader(Class classLoaderClass) { + notUserDefinedClassLoaderRegistry.add(classLoaderClass); + } + + /** A flag for whether a user-defined ClassLoaders has been constructed. */ + private static volatile boolean constructedUserDefinedClassLoaderFlag = false; + + /** + * Note a ClassLoader construction, and record if a + * user-defined ClassLoader has been constructed. + */ + public static void noteClassLoaderConstruction(Class classLoaderClass) { + /* Check if the ClassLoader class not a user-defined ClassLoader. */ + if (notUserDefinedClassLoaderRegistry.contains(classLoaderClass)) { + return; + } + constructedUserDefinedClassLoaderFlag = true; + return; + } + /* * Returns the first user-defined class loader up the execution stack, * or the platform class loader if only code from the platform or * bootstrap class loader is on the stack. */ public static ClassLoader latestUserDefinedLoader() { + if (!constructedUserDefinedClassLoaderFlag) { + /* + * If no user-defined ClassLoader has been constructed, + * then I will not find a user-defined ClassLoader in the stack. + */ + return ClassLoader.getSystemClassLoader(); + } ClassLoader loader = latestUserDefinedLoader0(); return loader != null ? loader : ClassLoader.getPlatformClassLoader(); } diff --git a/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java b/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java --- a/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java +++ b/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java @@ -30,6 +30,7 @@ import jdk.internal.access.JavaLangAccess; import jdk.internal.access.SharedSecrets; +import jdk.internal.misc.VM; /** Utility class which assists in calling defineClass() by creating a new class loader which delegates to the one needed in @@ -73,4 +74,9 @@ DelegatingClassLoader(ClassLoader parent) { super(parent); } + + static { + /* Register this ClassLoader as not a user-defined ClassLoader. */ + VM.registerNotUserDefinedClassLoader(DelegatingClassLoader.class); + } } From daniel.fuchs at oracle.com Fri Jun 5 16:45:30 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 5 Jun 2020 17:45:30 +0100 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <4F4BA16D-C9A4-46DD-A9A9-979E35431600@azul.com> References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> <85417480-7ecd-21d9-fb59-ff3d9a442360@oracle.com> <4F4BA16D-C9A4-46DD-A9A9-979E35431600@azul.com> Message-ID: <72e50735-e5e1-08d7-f1bf-dfbd340841bb@oracle.com> Hi Alexey, On 05/06/2020 17:33, Alexey Bakhtin wrote: > Hi Daniel, > > Thank you for review > Yes, I can move TlsChannelBinding class into the com.sun.jndi.ldap.sasl package and LdapClient related changes into the LdapSasl.saslBind method. > Also, you are right with exceptions. I will rename them to the NamingException. > > However, I?d like to parse TLS Channel Binding property in the LdapCtx class. The reason is ?com.sun.jndi.ldap.connect.timeout? property. This property should be set together with TLS Channel Binding. So, I?d like to verify if both properties are set before connection is started. The best place for it is LdapCtx.initEnv() > Is it acceptable ? Yes - I am OK with that. Also - you will need a test. Ideally we'd want a test that verifies that setting the new property works as expected. Best regards, -- daniel > > Thank you > Alexey From andy.herrick at oracle.com Fri Jun 5 17:26:39 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Fri, 5 Jun 2020 13:26:39 -0400 Subject: RFR: JDK-8246627: Consolidate app image bundlers In-Reply-To: <4bcba4b6-1eb2-5567-f614-0a31e4fbbcc7@oracle.com> References: <4bcba4b6-1eb2-5567-f614-0a31e4fbbcc7@oracle.com> Message-ID: Together these two changes might be a lot this late in the cycle, but it does make everything cleaner, and passes all test I have tried to run with them. I approve. /Andy On 6/4/2020 4:46 PM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > Move duplicated functionality from LinuxAppBundler, MacAppBundler and > WinAppBundler classes in the base class. > > [2] webrev is on top of [3] webrev. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8246627 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8246627/webrev.00 > > [3]?http://cr.openjdk.java.net/~asemenyuk/8246624/webrev.00 From james.laskey at oracle.com Fri Jun 5 17:46:01 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Fri, 5 Jun 2020 14:46:01 -0300 Subject: RFR: JDK-8246697 - Test: StringJoinerTest and related MergeTest failing with OOM Message-ID: <83F25F1D-F382-4B71-B9EC-C1E418CC5D83@oracle.com> Followup: can I get a quick review for the following. Both tests were sensitive to heap size. jbs: https://bugs.openjdk.java.net/browse/JDK-8246697 diff -r 71ec718a0bd0 test/jdk/ProblemList.txt --- a/test/jdk/ProblemList.txt Fri Jun 05 11:46:58 2020 -0300 +++ b/test/jdk/ProblemList.txt Fri Jun 05 14:38:35 2020 -0300 @@ -883,7 +883,6 @@ # jdk_util java/util/ServiceLoader/ReloadTest.java 8242935 generic-all -java/util/StringJoiner/StringJoinerTest.java 8246697 generic-all ############################################################################ diff -r 71ec718a0bd0 test/jdk/java/util/StringJoiner/MergeTest.java --- a/test/jdk/java/util/StringJoiner/MergeTest.java Fri Jun 05 11:46:58 2020 -0300 +++ b/test/jdk/java/util/StringJoiner/MergeTest.java Fri Jun 05 14:38:35 2020 -0300 @@ -26,7 +26,7 @@ * @bug 8017231 8020977 8054221 * @summary test StringJoiner::merge * @modules java.base/jdk.internal.util - * @run testng MergeTest + * @run testng/othervm -Xmx4g MergeTest */ import java.util.StringJoiner; diff -r 71ec718a0bd0 test/jdk/java/util/StringJoiner/StringJoinerTest.java --- a/test/jdk/java/util/StringJoiner/StringJoinerTest.java Fri Jun 05 11:46:58 2020 -0300 +++ b/test/jdk/java/util/StringJoiner/StringJoinerTest.java Fri Jun 05 14:38:35 2020 -0300 @@ -25,7 +25,7 @@ * @bug 5015163 7172553 * @summary tests StringJoinerTest * @modules java.base/jdk.internal.util - * @run testng StringJoinerTest + * @run testng/othervm -Xmx4g StringJoinerTest * @author Jim Gish */ import java.util.ArrayList; From igor.ignatyev at oracle.com Fri Jun 5 17:51:57 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 5 Jun 2020 10:51:57 -0700 Subject: RFR: JDK-8246697 - Test: StringJoinerTest and related MergeTest failing with OOM In-Reply-To: <83F25F1D-F382-4B71-B9EC-C1E418CC5D83@oracle.com> References: <83F25F1D-F382-4B71-B9EC-C1E418CC5D83@oracle.com> Message-ID: Hi Jim, I'd also recommend you to add `@requires os.maxMemory > 4G` so the tests don't get executed on hosts which don't have enough memory for requested heap size. -- Igor > On Jun 5, 2020, at 10:46 AM, Jim Laskey wrote: > > Followup: can I get a quick review for the following. Both tests were sensitive to heap size. > > jbs: https://bugs.openjdk.java.net/browse/JDK-8246697 > > > diff -r 71ec718a0bd0 test/jdk/ProblemList.txt > --- a/test/jdk/ProblemList.txt Fri Jun 05 11:46:58 2020 -0300 > +++ b/test/jdk/ProblemList.txt Fri Jun 05 14:38:35 2020 -0300 > @@ -883,7 +883,6 @@ > # jdk_util > > java/util/ServiceLoader/ReloadTest.java 8242935 generic-all > -java/util/StringJoiner/StringJoinerTest.java 8246697 generic-all > > ############################################################################ > > diff -r 71ec718a0bd0 test/jdk/java/util/StringJoiner/MergeTest.java > --- a/test/jdk/java/util/StringJoiner/MergeTest.java Fri Jun 05 11:46:58 2020 -0300 > +++ b/test/jdk/java/util/StringJoiner/MergeTest.java Fri Jun 05 14:38:35 2020 -0300 > @@ -26,7 +26,7 @@ > * @bug 8017231 8020977 8054221 > * @summary test StringJoiner::merge > * @modules java.base/jdk.internal.util > - * @run testng MergeTest > + * @run testng/othervm -Xmx4g MergeTest > */ > > import java.util.StringJoiner; > diff -r 71ec718a0bd0 test/jdk/java/util/StringJoiner/StringJoinerTest.java > --- a/test/jdk/java/util/StringJoiner/StringJoinerTest.java Fri Jun 05 11:46:58 2020 -0300 > +++ b/test/jdk/java/util/StringJoiner/StringJoinerTest.java Fri Jun 05 14:38:35 2020 -0300 > @@ -25,7 +25,7 @@ > * @bug 5015163 7172553 > * @summary tests StringJoinerTest > * @modules java.base/jdk.internal.util > - * @run testng StringJoinerTest > + * @run testng/othervm -Xmx4g StringJoinerTest > * @author Jim Gish > */ > import java.util.ArrayList; > From james.laskey at oracle.com Fri Jun 5 17:52:44 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Fri, 5 Jun 2020 14:52:44 -0300 Subject: RFR: JDK-8246697 - Test: StringJoinerTest and related MergeTest failing with OOM In-Reply-To: References: <83F25F1D-F382-4B71-B9EC-C1E418CC5D83@oracle.com> Message-ID: <731B4D75-C58A-4B0E-B093-31CA23331076@oracle.com> Thank you. > On Jun 5, 2020, at 2:51 PM, Igor Ignatyev wrote: > > Hi Jim, > > I'd also recommend you to add `@requires os.maxMemory > 4G` so the tests don't get executed on hosts which don't have enough memory for requested heap size. > > -- Igor > >> On Jun 5, 2020, at 10:46 AM, Jim Laskey wrote: >> >> Followup: can I get a quick review for the following. Both tests were sensitive to heap size. >> >> jbs: https://bugs.openjdk.java.net/browse/JDK-8246697 >> >> >> diff -r 71ec718a0bd0 test/jdk/ProblemList.txt >> --- a/test/jdk/ProblemList.txt Fri Jun 05 11:46:58 2020 -0300 >> +++ b/test/jdk/ProblemList.txt Fri Jun 05 14:38:35 2020 -0300 >> @@ -883,7 +883,6 @@ >> # jdk_util >> >> java/util/ServiceLoader/ReloadTest.java 8242935 generic-all >> -java/util/StringJoiner/StringJoinerTest.java 8246697 generic-all >> >> ############################################################################ >> >> diff -r 71ec718a0bd0 test/jdk/java/util/StringJoiner/MergeTest.java >> --- a/test/jdk/java/util/StringJoiner/MergeTest.java Fri Jun 05 11:46:58 2020 -0300 >> +++ b/test/jdk/java/util/StringJoiner/MergeTest.java Fri Jun 05 14:38:35 2020 -0300 >> @@ -26,7 +26,7 @@ >> * @bug 8017231 8020977 8054221 >> * @summary test StringJoiner::merge >> * @modules java.base/jdk.internal.util >> - * @run testng MergeTest >> + * @run testng/othervm -Xmx4g MergeTest >> */ >> >> import java.util.StringJoiner; >> diff -r 71ec718a0bd0 test/jdk/java/util/StringJoiner/StringJoinerTest.java >> --- a/test/jdk/java/util/StringJoiner/StringJoinerTest.java Fri Jun 05 11:46:58 2020 -0300 >> +++ b/test/jdk/java/util/StringJoiner/StringJoinerTest.java Fri Jun 05 14:38:35 2020 -0300 >> @@ -25,7 +25,7 @@ >> * @bug 5015163 7172553 >> * @summary tests StringJoinerTest >> * @modules java.base/jdk.internal.util >> - * @run testng StringJoinerTest >> + * @run testng/othervm -Xmx4g StringJoinerTest >> * @author Jim Gish >> */ >> import java.util.ArrayList; >> > From james.laskey at oracle.com Fri Jun 5 18:19:00 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Fri, 5 Jun 2020 15:19:00 -0300 Subject: RFR: JDK-8246697 - Test: StringJoinerTest and related MergeTest failing with OOM In-Reply-To: <731B4D75-C58A-4B0E-B093-31CA23331076@oracle.com> References: <83F25F1D-F382-4B71-B9EC-C1E418CC5D83@oracle.com> <731B4D75-C58A-4B0E-B093-31CA23331076@oracle.com> Message-ID: Updated diff diff -r 71ec718a0bd0 test/jdk/ProblemList.txt --- a/test/jdk/ProblemList.txt Fri Jun 05 11:46:58 2020 -0300 +++ b/test/jdk/ProblemList.txt Fri Jun 05 15:18:36 2020 -0300 @@ -883,7 +883,6 @@ # jdk_util java/util/ServiceLoader/ReloadTest.java 8242935 generic-all -java/util/StringJoiner/StringJoinerTest.java 8246697 generic-all ############################################################################ diff -r 71ec718a0bd0 test/jdk/java/util/StringJoiner/MergeTest.java --- a/test/jdk/java/util/StringJoiner/MergeTest.java Fri Jun 05 11:46:58 2020 -0300 +++ b/test/jdk/java/util/StringJoiner/MergeTest.java Fri Jun 05 15:18:36 2020 -0300 @@ -26,7 +26,8 @@ * @bug 8017231 8020977 8054221 * @summary test StringJoiner::merge * @modules java.base/jdk.internal.util - * @run testng MergeTest + * @requires os.maxMemory > 4G + * @run testng/othervm -Xmx4g MergeTest */ import java.util.StringJoiner; diff -r 71ec718a0bd0 test/jdk/java/util/StringJoiner/StringJoinerTest.java --- a/test/jdk/java/util/StringJoiner/StringJoinerTest.java Fri Jun 05 11:46:58 2020 -0300 +++ b/test/jdk/java/util/StringJoiner/StringJoinerTest.java Fri Jun 05 15:18:36 2020 -0300 @@ -25,7 +25,8 @@ * @bug 5015163 7172553 * @summary tests StringJoinerTest * @modules java.base/jdk.internal.util - * @run testng StringJoinerTest + * @requires os.maxMemory > 4G + * @run testng/othervm -Xmx4g StringJoinerTest * @author Jim Gish */ import java.util.ArrayList; > On Jun 5, 2020, at 2:52 PM, Jim Laskey wrote: > > Thank you. > >> On Jun 5, 2020, at 2:51 PM, Igor Ignatyev wrote: >> >> Hi Jim, >> >> I'd also recommend you to add `@requires os.maxMemory > 4G` so the tests don't get executed on hosts which don't have enough memory for requested heap size. >> >> -- Igor >> >>> On Jun 5, 2020, at 10:46 AM, Jim Laskey wrote: >>> >>> Followup: can I get a quick review for the following. Both tests were sensitive to heap size. >>> >>> jbs: https://bugs.openjdk.java.net/browse/JDK-8246697 >>> >>> >>> diff -r 71ec718a0bd0 test/jdk/ProblemList.txt >>> --- a/test/jdk/ProblemList.txt Fri Jun 05 11:46:58 2020 -0300 >>> +++ b/test/jdk/ProblemList.txt Fri Jun 05 14:38:35 2020 -0300 >>> @@ -883,7 +883,6 @@ >>> # jdk_util >>> >>> java/util/ServiceLoader/ReloadTest.java 8242935 generic-all >>> -java/util/StringJoiner/StringJoinerTest.java 8246697 generic-all >>> >>> ############################################################################ >>> >>> diff -r 71ec718a0bd0 test/jdk/java/util/StringJoiner/MergeTest.java >>> --- a/test/jdk/java/util/StringJoiner/MergeTest.java Fri Jun 05 11:46:58 2020 -0300 >>> +++ b/test/jdk/java/util/StringJoiner/MergeTest.java Fri Jun 05 14:38:35 2020 -0300 >>> @@ -26,7 +26,7 @@ >>> * @bug 8017231 8020977 8054221 >>> * @summary test StringJoiner::merge >>> * @modules java.base/jdk.internal.util >>> - * @run testng MergeTest >>> + * @run testng/othervm -Xmx4g MergeTest >>> */ >>> >>> import java.util.StringJoiner; >>> diff -r 71ec718a0bd0 test/jdk/java/util/StringJoiner/StringJoinerTest.java >>> --- a/test/jdk/java/util/StringJoiner/StringJoinerTest.java Fri Jun 05 11:46:58 2020 -0300 >>> +++ b/test/jdk/java/util/StringJoiner/StringJoinerTest.java Fri Jun 05 14:38:35 2020 -0300 >>> @@ -25,7 +25,7 @@ >>> * @bug 5015163 7172553 >>> * @summary tests StringJoinerTest >>> * @modules java.base/jdk.internal.util >>> - * @run testng StringJoinerTest >>> + * @run testng/othervm -Xmx4g StringJoinerTest >>> * @author Jim Gish >>> */ >>> import java.util.ArrayList; >>> >> > From igor.ignatyev at oracle.com Fri Jun 5 18:21:24 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 5 Jun 2020 11:21:24 -0700 Subject: RFR: JDK-8246697 - Test: StringJoinerTest and related MergeTest failing with OOM In-Reply-To: References: <83F25F1D-F382-4B71-B9EC-C1E418CC5D83@oracle.com> <731B4D75-C58A-4B0E-B093-31CA23331076@oracle.com> Message-ID: LGTM -- Igor > On Jun 5, 2020, at 11:19 AM, Jim Laskey wrote: > > Updated diff > > > diff -r 71ec718a0bd0 test/jdk/ProblemList.txt > --- a/test/jdk/ProblemList.txt Fri Jun 05 11:46:58 2020 -0300 > +++ b/test/jdk/ProblemList.txt Fri Jun 05 15:18:36 2020 -0300 > @@ -883,7 +883,6 @@ > # jdk_util > > java/util/ServiceLoader/ReloadTest.java 8242935 generic-all > -java/util/StringJoiner/StringJoinerTest.java 8246697 generic-all > > ############################################################################ > > diff -r 71ec718a0bd0 test/jdk/java/util/StringJoiner/MergeTest.java > --- a/test/jdk/java/util/StringJoiner/MergeTest.java Fri Jun 05 11:46:58 2020 -0300 > +++ b/test/jdk/java/util/StringJoiner/MergeTest.java Fri Jun 05 15:18:36 2020 -0300 > @@ -26,7 +26,8 @@ > * @bug 8017231 8020977 8054221 > * @summary test StringJoiner::merge > * @modules java.base/jdk.internal.util > - * @run testng MergeTest > + * @requires os.maxMemory > 4G > + * @run testng/othervm -Xmx4g MergeTest > */ > > import java.util.StringJoiner; > diff -r 71ec718a0bd0 test/jdk/java/util/StringJoiner/StringJoinerTest.java > --- a/test/jdk/java/util/StringJoiner/StringJoinerTest.java Fri Jun 05 11:46:58 2020 -0300 > +++ b/test/jdk/java/util/StringJoiner/StringJoinerTest.java Fri Jun 05 15:18:36 2020 -0300 > @@ -25,7 +25,8 @@ > * @bug 5015163 7172553 > * @summary tests StringJoinerTest > * @modules java.base/jdk.internal.util > - * @run testng StringJoinerTest > + * @requires os.maxMemory > 4G > + * @run testng/othervm -Xmx4g StringJoinerTest > * @author Jim Gish > */ > import java.util.ArrayList; > > >> On Jun 5, 2020, at 2:52 PM, Jim Laskey wrote: >> >> Thank you. >> >>> On Jun 5, 2020, at 2:51 PM, Igor Ignatyev wrote: >>> >>> Hi Jim, >>> >>> I'd also recommend you to add `@requires os.maxMemory > 4G` so the tests don't get executed on hosts which don't have enough memory for requested heap size. >>> >>> -- Igor >>> >>>> On Jun 5, 2020, at 10:46 AM, Jim Laskey wrote: >>>> >>>> Followup: can I get a quick review for the following. Both tests were sensitive to heap size. >>>> >>>> jbs: https://bugs.openjdk.java.net/browse/JDK-8246697 >>>> >>>> >>>> diff -r 71ec718a0bd0 test/jdk/ProblemList.txt >>>> --- a/test/jdk/ProblemList.txt Fri Jun 05 11:46:58 2020 -0300 >>>> +++ b/test/jdk/ProblemList.txt Fri Jun 05 14:38:35 2020 -0300 >>>> @@ -883,7 +883,6 @@ >>>> # jdk_util >>>> >>>> java/util/ServiceLoader/ReloadTest.java 8242935 generic-all >>>> -java/util/StringJoiner/StringJoinerTest.java 8246697 generic-all >>>> >>>> ############################################################################ >>>> >>>> diff -r 71ec718a0bd0 test/jdk/java/util/StringJoiner/MergeTest.java >>>> --- a/test/jdk/java/util/StringJoiner/MergeTest.java Fri Jun 05 11:46:58 2020 -0300 >>>> +++ b/test/jdk/java/util/StringJoiner/MergeTest.java Fri Jun 05 14:38:35 2020 -0300 >>>> @@ -26,7 +26,7 @@ >>>> * @bug 8017231 8020977 8054221 >>>> * @summary test StringJoiner::merge >>>> * @modules java.base/jdk.internal.util >>>> - * @run testng MergeTest >>>> + * @run testng/othervm -Xmx4g MergeTest >>>> */ >>>> >>>> import java.util.StringJoiner; >>>> diff -r 71ec718a0bd0 test/jdk/java/util/StringJoiner/StringJoinerTest.java >>>> --- a/test/jdk/java/util/StringJoiner/StringJoinerTest.java Fri Jun 05 11:46:58 2020 -0300 >>>> +++ b/test/jdk/java/util/StringJoiner/StringJoinerTest.java Fri Jun 05 14:38:35 2020 -0300 >>>> @@ -25,7 +25,7 @@ >>>> * @bug 5015163 7172553 >>>> * @summary tests StringJoinerTest >>>> * @modules java.base/jdk.internal.util >>>> - * @run testng StringJoinerTest >>>> + * @run testng/othervm -Xmx4g StringJoinerTest >>>> * @author Jim Gish >>>> */ >>>> import java.util.ArrayList; >>>> >>> >> > From james.laskey at oracle.com Fri Jun 5 18:21:50 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Fri, 5 Jun 2020 15:21:50 -0300 Subject: RFR: JDK-8246697 - Test: StringJoinerTest and related MergeTest failing with OOM In-Reply-To: References: <83F25F1D-F382-4B71-B9EC-C1E418CC5D83@oracle.com> <731B4D75-C58A-4B0E-B093-31CA23331076@oracle.com> Message-ID: Thank you. > On Jun 5, 2020, at 3:21 PM, Igor Ignatyev wrote: > > LGTM > > -- Igor > >> On Jun 5, 2020, at 11:19 AM, Jim Laskey wrote: >> >> Updated diff >> >> >> diff -r 71ec718a0bd0 test/jdk/ProblemList.txt >> --- a/test/jdk/ProblemList.txt Fri Jun 05 11:46:58 2020 -0300 >> +++ b/test/jdk/ProblemList.txt Fri Jun 05 15:18:36 2020 -0300 >> @@ -883,7 +883,6 @@ >> # jdk_util >> >> java/util/ServiceLoader/ReloadTest.java 8242935 generic-all >> -java/util/StringJoiner/StringJoinerTest.java 8246697 generic-all >> >> ############################################################################ >> >> diff -r 71ec718a0bd0 test/jdk/java/util/StringJoiner/MergeTest.java >> --- a/test/jdk/java/util/StringJoiner/MergeTest.java Fri Jun 05 11:46:58 2020 -0300 >> +++ b/test/jdk/java/util/StringJoiner/MergeTest.java Fri Jun 05 15:18:36 2020 -0300 >> @@ -26,7 +26,8 @@ >> * @bug 8017231 8020977 8054221 >> * @summary test StringJoiner::merge >> * @modules java.base/jdk.internal.util >> - * @run testng MergeTest >> + * @requires os.maxMemory > 4G >> + * @run testng/othervm -Xmx4g MergeTest >> */ >> >> import java.util.StringJoiner; >> diff -r 71ec718a0bd0 test/jdk/java/util/StringJoiner/StringJoinerTest.java >> --- a/test/jdk/java/util/StringJoiner/StringJoinerTest.java Fri Jun 05 11:46:58 2020 -0300 >> +++ b/test/jdk/java/util/StringJoiner/StringJoinerTest.java Fri Jun 05 15:18:36 2020 -0300 >> @@ -25,7 +25,8 @@ >> * @bug 5015163 7172553 >> * @summary tests StringJoinerTest >> * @modules java.base/jdk.internal.util >> - * @run testng StringJoinerTest >> + * @requires os.maxMemory > 4G >> + * @run testng/othervm -Xmx4g StringJoinerTest >> * @author Jim Gish >> */ >> import java.util.ArrayList; >> >> >>> On Jun 5, 2020, at 2:52 PM, Jim Laskey wrote: >>> >>> Thank you. >>> >>>> On Jun 5, 2020, at 2:51 PM, Igor Ignatyev wrote: >>>> >>>> Hi Jim, >>>> >>>> I'd also recommend you to add `@requires os.maxMemory > 4G` so the tests don't get executed on hosts which don't have enough memory for requested heap size. >>>> >>>> -- Igor >>>> >>>>> On Jun 5, 2020, at 10:46 AM, Jim Laskey wrote: >>>>> >>>>> Followup: can I get a quick review for the following. Both tests were sensitive to heap size. >>>>> >>>>> jbs: https://bugs.openjdk.java.net/browse/JDK-8246697 >>>>> >>>>> >>>>> diff -r 71ec718a0bd0 test/jdk/ProblemList.txt >>>>> --- a/test/jdk/ProblemList.txt Fri Jun 05 11:46:58 2020 -0300 >>>>> +++ b/test/jdk/ProblemList.txt Fri Jun 05 14:38:35 2020 -0300 >>>>> @@ -883,7 +883,6 @@ >>>>> # jdk_util >>>>> >>>>> java/util/ServiceLoader/ReloadTest.java 8242935 generic-all >>>>> -java/util/StringJoiner/StringJoinerTest.java 8246697 generic-all >>>>> >>>>> ############################################################################ >>>>> >>>>> diff -r 71ec718a0bd0 test/jdk/java/util/StringJoiner/MergeTest.java >>>>> --- a/test/jdk/java/util/StringJoiner/MergeTest.java Fri Jun 05 11:46:58 2020 -0300 >>>>> +++ b/test/jdk/java/util/StringJoiner/MergeTest.java Fri Jun 05 14:38:35 2020 -0300 >>>>> @@ -26,7 +26,7 @@ >>>>> * @bug 8017231 8020977 8054221 >>>>> * @summary test StringJoiner::merge >>>>> * @modules java.base/jdk.internal.util >>>>> - * @run testng MergeTest >>>>> + * @run testng/othervm -Xmx4g MergeTest >>>>> */ >>>>> >>>>> import java.util.StringJoiner; >>>>> diff -r 71ec718a0bd0 test/jdk/java/util/StringJoiner/StringJoinerTest.java >>>>> --- a/test/jdk/java/util/StringJoiner/StringJoinerTest.java Fri Jun 05 11:46:58 2020 -0300 >>>>> +++ b/test/jdk/java/util/StringJoiner/StringJoinerTest.java Fri Jun 05 14:38:35 2020 -0300 >>>>> @@ -25,7 +25,7 @@ >>>>> * @bug 5015163 7172553 >>>>> * @summary tests StringJoinerTest >>>>> * @modules java.base/jdk.internal.util >>>>> - * @run testng StringJoinerTest >>>>> + * @run testng/othervm -Xmx4g StringJoinerTest >>>>> * @author Jim Gish >>>>> */ >>>>> import java.util.ArrayList; >>>>> >>>> >>> >> > From andy.herrick at oracle.com Fri Jun 5 20:23:15 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Fri, 5 Jun 2020 16:23:15 -0400 Subject: RFR: JDK-8246706: [macos] Allow SigningPackageTest to be built with real certificates Message-ID: Please review fix to issue [1] at [2]. The fix enables running jpackage signing tests on macos with user supplied certificates. [1] https://bugs.openjdk.java.net/browse/JDK-8246706 [2] http://cr.openjdk.java.net/~herrick/8246706/webrev.01/index.html /Andy From stuart.marks at oracle.com Fri Jun 5 20:36:25 2020 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 5 Jun 2020 13:36:25 -0700 Subject: Sometimes constraints are questionable In-Reply-To: <23efc3c4-d711-8750-8fdb-ee9ac8553952@oracle.com> References: <4A5AC9D1-AEB7-47E3-9E25-BF74FE01F3B0@oracle.com> <2F2E688C-51EF-4DAF-826E-70669940538D@oracle.com> <259F0159-0B61-4334-A93A-80EA69DF5660@oracle.com> <23efc3c4-d711-8750-8fdb-ee9ac8553952@oracle.com> Message-ID: <91bd6c0f-5413-0195-9332-2bbd42919a3f@oracle.com> On 6/3/20 10:36 AM, Stuart Marks wrote: > 3) Integer wraparound/overflow during size computation. AS.newLength generates > this: > > ??? OutOfMemoryError: Required array length too large > > (3) is the only case generated by the library. In fact, AS.hugeLength() has > oldLength and minGrowth parameters, which seems like enough detail already. > These could reasonably be formatted into the error message, something like: > > ??? private static int hugeLength(int oldLength, int minGrowth) { > ??????? int minLength = oldLength + minGrowth; > ??????? if (minLength < 0) { // overflow > ??????????? throw new OutOfMemoryError( > ??????????????? String.format("Required array length %d + %d too large", > oldLength, minGrowth)); > ??????? } > > Would this help? If this were added, would it be sufficient to allow various use > sites to convert to use AS.newLength? (Except possibly StringJoiner.) Anything further on this? Should I file a bug/rfe for this? Also, I could update the docs to explain ArraysSupport.newLength better, per my earlier exchange with David Holmes. s'marks From james.laskey at oracle.com Fri Jun 5 20:48:19 2020 From: james.laskey at oracle.com (James Laskey) Date: Fri, 5 Jun 2020 17:48:19 -0300 Subject: Sometimes constraints are questionable In-Reply-To: <91bd6c0f-5413-0195-9332-2bbd42919a3f@oracle.com> References: <91bd6c0f-5413-0195-9332-2bbd42919a3f@oracle.com> Message-ID: I?m fixing the two in java.base. The other two are in different modules and would require changes to export. So you can file on those. ?? > On Jun 5, 2020, at 5:36 PM, Stuart Marks wrote: > > ? > >> On 6/3/20 10:36 AM, Stuart Marks wrote: >> 3) Integer wraparound/overflow during size computation. AS.newLength generates this: >> OutOfMemoryError: Required array length too large >> (3) is the only case generated by the library. In fact, AS.hugeLength() has oldLength and minGrowth parameters, which seems like enough detail already. These could reasonably be formatted into the error message, something like: >> private static int hugeLength(int oldLength, int minGrowth) { >> int minLength = oldLength + minGrowth; >> if (minLength < 0) { // overflow >> throw new OutOfMemoryError( >> String.format("Required array length %d + %d too large", oldLength, minGrowth)); >> } >> Would this help? If this were added, would it be sufficient to allow various use sites to convert to use AS.newLength? (Except possibly StringJoiner.) > > Anything further on this? Should I file a bug/rfe for this? Also, I could update the docs to explain ArraysSupport.newLength better, per my earlier exchange with David Holmes. > > s'marks > From alexey.semenyuk at oracle.com Fri Jun 5 21:27:52 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Fri, 5 Jun 2020 17:27:52 -0400 Subject: RFR: JDK-8246706: [macos] Allow SigningPackageTest to be built with real certificates In-Reply-To: References: Message-ID: <65facbe0-a665-7d73-9928-058e4dfe0c7f@oracle.com> In SigningBase.java: --- KEYCHAIN = (value == null) ? "jpackagerTest.keychain" : value; --- Is "jpackagerTest.keychain" some legacy string? Should not it be "jpackageTest.keychain"? - Alexey On 6/5/2020 4:23 PM, Andy Herrick wrote: > Please review fix to issue [1] at [2]. > > The fix enables running jpackage signing tests on macos with user > supplied certificates. > > [1] https://bugs.openjdk.java.net/browse/JDK-8246706 > > [2] http://cr.openjdk.java.net/~herrick/8246706/webrev.01/index.html > > /Andy > From magnus.ihse.bursie at oracle.com Fri Jun 5 23:57:25 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Sat, 6 Jun 2020 01:57:25 +0200 Subject: RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: <84EEE234-A21F-400B-917C-8DF55236E38E@oracle.com> References: <84EEE234-A21F-400B-917C-8DF55236E38E@oracle.com> Message-ID: <511ed9e8-c35b-aa04-12ed-a1df2dd116b8@oracle.com> On 2020-06-05 13:59, Jim Laskey wrote: > I know there was a discussion about this elsewhere but I would like to take the opportunity to correct this now > > make//autoconf/flags-cflags.m4:241 > > elif test "x$TOOLCHAIN_TYPE" = xclang; then > if test "x$OPENJDK_TARGET_OS" = xmacosx; then > # On MacOSX we optimize for size, something > # we should do for all platforms? > C_O_FLAG_HIGHEST_JVM="-Os" > C_O_FLAG_HIGHEST="-Os" > C_O_FLAG_HI="-Os" > C_O_FLAG_NORM="-Os" > C_O_FLAG_DEBUG_JVM="" > else > C_O_FLAG_HIGHEST_JVM="-O3" > C_O_FLAG_HIGHEST="-O3" > C_O_FLAG_HI="-O3" > C_O_FLAG_NORM="-O2" > C_O_FLAG_DEBUG_JVM="-O0" > fi > C_O_FLAG_SIZE="-Os" > C_O_FLAG_DEBUG="-O0" > C_O_FLAG_NONE="-O0" > elif test "x$TOOLCHAIN_TYPE" = xxlc; then > > should be changed to > > elif test "x$TOOLCHAIN_TYPE" = xclang; then > C_O_FLAG_HIGHEST_JVM="-O3" > C_O_FLAG_HIGHEST="-O3" > C_O_FLAG_HI="-O3" > C_O_FLAG_NORM="-O2" > C_O_FLAG_DEBUG_JVM="-O0" > C_O_FLAG_SIZE="-Os" > C_O_FLAG_DEBUG="-O0" > C_O_FLAG_NONE="-O0" > elif test "x$TOOLCHAIN_TYPE" = xxlc; then > > MacOSX has been paying a historic and significant performance penalty for no valid reason. This might be a valid change, but it has nothing to do with C++14, and changing it at the same time will increase risk for unrelated strange errors. Please open a separate JBS issue for this requested change. /Magnus > > Otherwise +1. > > Cheers, > > -- Jim > > > >> On Jun 5, 2020, at 4:52 AM, Kim Barrett wrote: >> >> [Changes are only to the build system, but since the changes have jdk-wide >> effect I?ve cc?d what I think are the relevant dev lists.] >> >> This change is part of JEP 347; the intent is to target JDK 16. >> >> Please review this change to the building of C++ code in the JDK to >> enable the use of C++14 language features. This change does not make >> any code changes to use new features provided by C++11 or C++14. >> >> This requires trimming the supported compiler versions, moving the >> minimum supported versions forward (significantly, in some cases). >> The new minimums are based on compiler documentation rather than >> testing. It may be that more recent versions are actually required. >> >> This change needs to be tested on platforms not supported by Oracle. >> The JEP test plan includes additional Oracle testing beyond what I?ve done. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8246032 >> >> Webrev: >> https://cr.openjdk.java.net/~kbarrett/8246032/open.02/ >> >> Testing: >> mach5 tier1-5 on Oracle supported platforms. >> >> Performance testing showed no significant changes in either direction. >> >> Build-only (no tests) for linux-arm32, linux-s390x, linux-ppc64le >> From magnus.ihse.bursie at oracle.com Fri Jun 5 23:57:54 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Sat, 6 Jun 2020 01:57:54 +0200 Subject: RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: References: Message-ID: On 2020-06-05 09:52, Kim Barrett wrote: > [Changes are only to the build system, but since the changes have jdk-wide > effect I?ve cc?d what I think are the relevant dev lists.] > > This change is part of JEP 347; the intent is to target JDK 16. > > Please review this change to the building of C++ code in the JDK to > enable the use of C++14 language features. This change does not make > any code changes to use new features provided by C++11 or C++14. > > This requires trimming the supported compiler versions, moving the > minimum supported versions forward (significantly, in some cases). > The new minimums are based on compiler documentation rather than > testing. It may be that more recent versions are actually required. > > This change needs to be tested on platforms not supported by Oracle. > The JEP test plan includes additional Oracle testing beyond what I?ve done. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8246032 > > Webrev: > https://cr.openjdk.java.net/~kbarrett/8246032/open.02/ Looks good to me. /Magnus > > Testing: > mach5 tier1-5 on Oracle supported platforms. > > Performance testing showed no significant changes in either direction. > > Build-only (no tests) for linux-arm32, linux-s390x, linux-ppc64le > From kim.barrett at oracle.com Sat Jun 6 00:21:03 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 5 Jun 2020 20:21:03 -0400 Subject: RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: <511ed9e8-c35b-aa04-12ed-a1df2dd116b8@oracle.com> References: <84EEE234-A21F-400B-917C-8DF55236E38E@oracle.com> <511ed9e8-c35b-aa04-12ed-a1df2dd116b8@oracle.com> Message-ID: <50E319B9-428D-46CE-A68C-5D2F31A2556E@oracle.com> > On Jun 5, 2020, at 7:57 PM, Magnus Ihse Bursie wrote: > > On 2020-06-05 13:59, Jim Laskey wrote: >> I know there was a discussion about this elsewhere but I would like to take the opportunity to correct this now >> >> make//autoconf/flags-cflags.m4:241 >> >> [?] >> MacOSX has been paying a historic and significant performance penalty for no valid reason. > This might be a valid change, but it has nothing to do with C++14, and changing it at the same time will increase risk for unrelated strange errors. Please open a separate JBS issue for this requested change. I was going to say much the same thing. I don?t want to add that to the C++14 mix. I don?t even know for sure that it?s a desirable change. I?ve worked on projects that got better (measured) performance from -Os than from other global alternatives. One would need to do a bunch of performance testing and might need to go more deeply into the options to really optimize things. From kim.barrett at oracle.com Sat Jun 6 00:21:56 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Fri, 5 Jun 2020 20:21:56 -0400 Subject: RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: References: Message-ID: > On Jun 5, 2020, at 7:57 PM, Magnus Ihse Bursie wrote: > > On 2020-06-05 09:52, Kim Barrett wrote: >> [Changes are only to the build system, but since the changes have jdk-wide >> effect I?ve cc?d what I think are the relevant dev lists.] >> >> This change is part of JEP 347; the intent is to target JDK 16. >> >> Please review this change to the building of C++ code in the JDK to >> enable the use of C++14 language features. This change does not make >> any code changes to use new features provided by C++11 or C++14. >> >> This requires trimming the supported compiler versions, moving the >> minimum supported versions forward (significantly, in some cases). >> The new minimums are based on compiler documentation rather than >> testing. It may be that more recent versions are actually required. >> >> This change needs to be tested on platforms not supported by Oracle. >> The JEP test plan includes additional Oracle testing beyond what I?ve done. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8246032 >> >> Webrev: >> https://cr.openjdk.java.net/~kbarrett/8246032/open.02/ > Looks good to me. Thanks. > > /Magnus > >> >> Testing: >> mach5 tier1-5 on Oracle supported platforms. >> >> Performance testing showed no significant changes in either direction. >> >> Build-only (no tests) for linux-arm32, linux-s390x, linux-ppc64le From headius at headius.com Sat Jun 6 01:45:41 2020 From: headius at headius.com (Charles Oliver Nutter) Date: Fri, 5 Jun 2020 20:45:41 -0500 Subject: URLClassLoader poisons the global JarFile cache Message-ID: Hey folks, with some help from Claes Redestad I've filed an issue with analysis and reproduction of the legendary "zip file closed" error. https://bugs.openjdk.java.net/browse/JDK-8246714 It turns out that URLClassLoader is just plain broken when used to access jar files (which means this bug affects basically every application out there). Two threads using their own URLClassLoader instances to access the same jar file will likely lead to random errors, due to URLClassLoader improperly closing JarFile instances in the global cache. I'm posting here to bring attention to this issue, because it affects all versions of OpenJDK at least back to Java 7, and the workarounds are gross and only partly effective. We've struggled for years to find the cause of this issue, which occurs randomly in JRuby CI builds and has been reported by numerous users. Maybe we'll finally be able to get this stinker of a bug fixed! - Charlie From weijun.wang at oracle.com Sat Jun 6 06:41:28 2020 From: weijun.wang at oracle.com (Weijun Wang) Date: Sat, 6 Jun 2020 14:41:28 +0800 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> Message-ID: > On Jun 5, 2020, at 11:03 PM, Alexey Bakhtin wrote: > > Hello Max, > > Thank you a lot for review. > > Could you check the new version of the patch : > http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v4/ > > I?ve made the following changes: > - TlsChannelBinding class is moved to java.naming module. > java.security.sasl module is not affected any more > - I pass tlsCB.getData() directly to the SASL mechanism as you suggested > - I?ve made some guards to prevent application from using "com.sun.security.sasl.tlschannelbinding? property directly: > - LdapClient verifies if internal property is not set 245 // Prepare TLS Channel Binding data 246 if (conn.sock instanceof SSLSocket) { 247 // Internal property cannot be set explicitly 248 if (env.get(TlsChannelBinding.CHANNEL_BINDING) != null) { 249 throw new NamingException(TlsChannelBinding.CHANNEL_BINDING + 250 " property cannot be set explicitly"); 251 } If not TLS, this property value be kept there and visible inside the SASL mech. > - GssKrb5Client uses props.remove() to read and remove internal property Traditionally, we use "com.sun..." name which is a JDK supported name (although not at Java SE level), you might want to use a name which is even more internal. Thanks, Max p.s. I see that NTLM also supports ChannelBinding. I'll see if I can improve the NTLM SASL mech to support it. > > Regards > Alexey > >> On 5 Jun 2020, at 06:41, Weijun Wang wrote: >> >> Hi Alexey, >> >> It's so unfortunate that different addressType must be used. I'm OK with the new TlsChannelBindingImpl class. >> >> One thing I'm not comfortable is the java.security.sasl/share/classes/module-info.java change. We've struggled hard to avoid these kind of secret tunnels. Is it possible to pass the tlsCB.getData() directly to the SASL mechanism? The property name is clear enough to avoid any misuse. >> >> Another question: can an application user set the "com.sun.security.sasl.tlschannelbinding" property? and can they read it after it's set internally? I cannot think of a good attack now but at least it seems they have no need to access that property value. If we keep it internal then we also have the chance to modify the approach later. >> >> Thanks, >> Max >> >> >>> On Jun 5, 2020, at 2:15 AM, Alexey Bakhtin wrote: >>> >>> Hello, >>> >>> Could you please review new version of the patch: >>> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v3/ >>> >>> I?ve moved all logic for creating TLS Channel Binding data out of GssKrb5Client.java file. >>> All data is prepared inside TlsChannelBinding class. >>> Internal property name is renamed to ?com.sun.security.sasl.tlschannelbinding?. >>> TlsChannelBinding object is still used to pass channel binding data from LdapClient to GssKrb5Client. >>> I do not change it to byte[] because of TlsChannelBinding class is still used for internal property name. >>> Also, I?ve updated TlsChannelBinding class to select SHA-256 hash algorithm if it can not be derived >>> from the certificate signature name. >>> >>> Regards >>> Alexey >>> >>> >>>> On 26 May 2020, at 17:50, Weijun Wang wrote: >>>> >>>> I have a question on GssKrb5Client.java: >>>> >>>> Do you think it's a good idea to let the SASL mechanism understand what TLS binding is? Instead of passing the whole TlsChannelBinding object through a SASL property, is it possible to only pass the actual cbData? After all, the name "com.sun.security.sasl.channelbinding" suggests it's just a general ChannelBinding which is independent with any application level info. >>>> >>>> From my reading of the code change, it looks like this cbData can be calculated on the LDAP side. >>>> >>>> Thanks, >>>> Max >>>> >>>>> On May 21, 2020, at 3:35 PM, Alexey Bakhtin wrote: >>>>> >>>>> Hello, >>>>> >>>>> Could you please review the following patch: >>>>> >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8245527 >>>>> Webrev: http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v0/ >>>>> >>>>> The Windows LDAP server with LdapEnforceChannelBinding=2 uses the tls-server-end-point channel binding >>>>> (based on the TLS server certificate). The channel binding data is calculated as following : >>>>> ? The client calculates a hash of the TLS server certificate. >>>>> The hash algorithm is selected on the base of the certificate signature algorithm. >>>>> Also, the client should use SHA-256 algorithm, in case of the certificate signature algorithm is SHA1 or MD5 based >>>>> ? The channel binding information is the same as defined in rfc4121 [1] with small corrections: >>>>> ? initiator and acceptor addresses should be set to NULL >>>>> ? initiator and acceptor address types should be zero. >>>>> It contradicts to the ?Using Channel Bindings in GSS-API? document [2] that say that >>>>> the address type should be set to GSS_C_AF_NULLADDR=0xFF, >>>>> instead of GSS_C_AF_UNSPEC=0x00. >>>>> >>>>> This patch introduces changes in the LDAP, SASL and JGSS modules >>>>> to generate channel binding data automatically if requested by an application. >>>>> This patch reuses existing org.ietf.jgss.ChannelBinding class implementation but changes >>>>> initial unspecified address type from CHANNEL_BINDING_AF_NULL_ADDR to CHANNEL_BINDING_AF_UNSPEC. >>>>> The patch introduces new environment property ?com.sun.jndi.ldap.tls.cbtype? that indicates >>>>> Channel Binding type that should be used in the LDAPS connection over JGSS/Kerberos. >>>>> Right now "tls-server-end-point" Channel Binding type is supported only. >>>>> The client extracts server certificate from the underlying TLS connection and creates >>>>> Channel Binding data for JGSS/Kerberos authentication if application indicates >>>>> com.sun.jndi.ldap.tls.cbtype=tls-server-end-point property. >>>>> Client application should also specify existing "com.sun.jndi.ldap.connect.timeout? property >>>>> to force and wait TLS handshake completion before JGSS/Kerberos authentication data is generated. >>>>> >>>>> [1] - https://tools.ietf.org/html/rfc4121#section-4.1.1.2 >>>>> >>>>> [2] - >>>>> https://docs.oracle.com/cd/E19120-01/open.solaris/819-2145/overview-52/index.html >>>>> >>>>> Initial discussion of this issue is available at security-dev list: >>>>> https://mail.openjdk.java.net/pipermail/security-dev/2019-December/021052.html >>>>> https://mail.openjdk.java.net/pipermail/security-dev/2020-January/021140.html >>>>> https://mail.openjdk.java.net/pipermail/security-dev/2020-February/021278.html >>>>> https://mail.openjdk.java.net/pipermail/security-dev/2020-May/021864.html >>> > From weijun.wang at oracle.com Sat Jun 6 06:44:55 2020 From: weijun.wang at oracle.com (Weijun Wang) Date: Sat, 6 Jun 2020 14:44:55 +0800 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> Message-ID: <9013C8D0-33D9-4D09-8953-F8618020A7F7@oracle.com> > On Jun 6, 2020, at 2:41 PM, Weijun Wang wrote: > > > >> On Jun 5, 2020, at 11:03 PM, Alexey Bakhtin wrote: >> >> Hello Max, >> >> Thank you a lot for review. >> >> Could you check the new version of the patch : >> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v4/ >> >> I?ve made the following changes: >> - TlsChannelBinding class is moved to java.naming module. >> java.security.sasl module is not affected any more >> - I pass tlsCB.getData() directly to the SASL mechanism as you suggested >> - I?ve made some guards to prevent application from using "com.sun.security.sasl.tlschannelbinding? property directly: >> - LdapClient verifies if internal property is not set > > 245 // Prepare TLS Channel Binding data > 246 if (conn.sock instanceof SSLSocket) { > 247 // Internal property cannot be set explicitly > 248 if (env.get(TlsChannelBinding.CHANNEL_BINDING) != null) { > 249 throw new NamingException(TlsChannelBinding.CHANNEL_BINDING + > 250 " property cannot be set explicitly"); > 251 } > > If not TLS, this property value be kept there and visible inside the SASL mech. > >> - GssKrb5Client uses props.remove() to read and remove internal property Maybe you can remove the value in LdapClient.java, in case the mech used is not GSSAPI. You can remove it in a finally block after line 303. --Max > > Traditionally, we use "com.sun..." name which is a JDK supported name (although not at Java SE level), you might want to use a name which is even more internal. > > > Thanks, > Max > > p.s. I see that NTLM also supports ChannelBinding. I'll see if I can improve the NTLM SASL mech to support it. > >> >> Regards >> Alexey >> >>> On 5 Jun 2020, at 06:41, Weijun Wang wrote: >>> >>> Hi Alexey, >>> >>> It's so unfortunate that different addressType must be used. I'm OK with the new TlsChannelBindingImpl class. >>> >>> One thing I'm not comfortable is the java.security.sasl/share/classes/module-info.java change. We've struggled hard to avoid these kind of secret tunnels. Is it possible to pass the tlsCB.getData() directly to the SASL mechanism? The property name is clear enough to avoid any misuse. >>> >>> Another question: can an application user set the "com.sun.security.sasl.tlschannelbinding" property? and can they read it after it's set internally? I cannot think of a good attack now but at least it seems they have no need to access that property value. If we keep it internal then we also have the chance to modify the approach later. >>> >>> Thanks, >>> Max >>> >>> >>>> On Jun 5, 2020, at 2:15 AM, Alexey Bakhtin wrote: >>>> >>>> Hello, >>>> >>>> Could you please review new version of the patch: >>>> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v3/ >>>> >>>> I?ve moved all logic for creating TLS Channel Binding data out of GssKrb5Client.java file. >>>> All data is prepared inside TlsChannelBinding class. >>>> Internal property name is renamed to ?com.sun.security.sasl.tlschannelbinding?. >>>> TlsChannelBinding object is still used to pass channel binding data from LdapClient to GssKrb5Client. >>>> I do not change it to byte[] because of TlsChannelBinding class is still used for internal property name. >>>> Also, I?ve updated TlsChannelBinding class to select SHA-256 hash algorithm if it can not be derived >>>> from the certificate signature name. >>>> >>>> Regards >>>> Alexey >>>> >>>> >>>>> On 26 May 2020, at 17:50, Weijun Wang wrote: >>>>> >>>>> I have a question on GssKrb5Client.java: >>>>> >>>>> Do you think it's a good idea to let the SASL mechanism understand what TLS binding is? Instead of passing the whole TlsChannelBinding object through a SASL property, is it possible to only pass the actual cbData? After all, the name "com.sun.security.sasl.channelbinding" suggests it's just a general ChannelBinding which is independent with any application level info. >>>>> >>>>> From my reading of the code change, it looks like this cbData can be calculated on the LDAP side. >>>>> >>>>> Thanks, >>>>> Max >>>>> >>>>>> On May 21, 2020, at 3:35 PM, Alexey Bakhtin wrote: >>>>>> >>>>>> Hello, >>>>>> >>>>>> Could you please review the following patch: >>>>>> >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8245527 >>>>>> Webrev: http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v0/ >>>>>> >>>>>> The Windows LDAP server with LdapEnforceChannelBinding=2 uses the tls-server-end-point channel binding >>>>>> (based on the TLS server certificate). The channel binding data is calculated as following : >>>>>> ? The client calculates a hash of the TLS server certificate. >>>>>> The hash algorithm is selected on the base of the certificate signature algorithm. >>>>>> Also, the client should use SHA-256 algorithm, in case of the certificate signature algorithm is SHA1 or MD5 based >>>>>> ? The channel binding information is the same as defined in rfc4121 [1] with small corrections: >>>>>> ? initiator and acceptor addresses should be set to NULL >>>>>> ? initiator and acceptor address types should be zero. >>>>>> It contradicts to the ?Using Channel Bindings in GSS-API? document [2] that say that >>>>>> the address type should be set to GSS_C_AF_NULLADDR=0xFF, >>>>>> instead of GSS_C_AF_UNSPEC=0x00. >>>>>> >>>>>> This patch introduces changes in the LDAP, SASL and JGSS modules >>>>>> to generate channel binding data automatically if requested by an application. >>>>>> This patch reuses existing org.ietf.jgss.ChannelBinding class implementation but changes >>>>>> initial unspecified address type from CHANNEL_BINDING_AF_NULL_ADDR to CHANNEL_BINDING_AF_UNSPEC. >>>>>> The patch introduces new environment property ?com.sun.jndi.ldap.tls.cbtype? that indicates >>>>>> Channel Binding type that should be used in the LDAPS connection over JGSS/Kerberos. >>>>>> Right now "tls-server-end-point" Channel Binding type is supported only. >>>>>> The client extracts server certificate from the underlying TLS connection and creates >>>>>> Channel Binding data for JGSS/Kerberos authentication if application indicates >>>>>> com.sun.jndi.ldap.tls.cbtype=tls-server-end-point property. >>>>>> Client application should also specify existing "com.sun.jndi.ldap.connect.timeout? property >>>>>> to force and wait TLS handshake completion before JGSS/Kerberos authentication data is generated. >>>>>> >>>>>> [1] - https://tools.ietf.org/html/rfc4121#section-4.1.1.2 >>>>>> >>>>>> [2] - >>>>>> https://docs.oracle.com/cd/E19120-01/open.solaris/819-2145/overview-52/index.html >>>>>> >>>>>> Initial discussion of this issue is available at security-dev list: >>>>>> https://mail.openjdk.java.net/pipermail/security-dev/2019-December/021052.html >>>>>> https://mail.openjdk.java.net/pipermail/security-dev/2020-January/021140.html >>>>>> https://mail.openjdk.java.net/pipermail/security-dev/2020-February/021278.html >>>>>> https://mail.openjdk.java.net/pipermail/security-dev/2020-May/021864.html From james.laskey at oracle.com Sat Jun 6 07:24:57 2020 From: james.laskey at oracle.com (James Laskey) Date: Sat, 6 Jun 2020 04:24:57 -0300 Subject: RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: <511ed9e8-c35b-aa04-12ed-a1df2dd116b8@oracle.com> References: <511ed9e8-c35b-aa04-12ed-a1df2dd116b8@oracle.com> Message-ID: <99014B06-0A8A-476A-BA3F-485FD2DE29F6@oracle.com> I fully understand. I?ll set up a separate bug. Cheers, ? Jim ?? > On Jun 5, 2020, at 8:57 PM, Magnus Ihse Bursie wrote: > > ?On 2020-06-05 13:59, Jim Laskey wrote: >> I know there was a discussion about this elsewhere but I would like to take the opportunity to correct this now >> >> make//autoconf/flags-cflags.m4:241 >> >> elif test "x$TOOLCHAIN_TYPE" = xclang; then >> if test "x$OPENJDK_TARGET_OS" = xmacosx; then >> # On MacOSX we optimize for size, something >> # we should do for all platforms? >> C_O_FLAG_HIGHEST_JVM="-Os" >> C_O_FLAG_HIGHEST="-Os" >> C_O_FLAG_HI="-Os" >> C_O_FLAG_NORM="-Os" >> C_O_FLAG_DEBUG_JVM="" >> else >> C_O_FLAG_HIGHEST_JVM="-O3" >> C_O_FLAG_HIGHEST="-O3" >> C_O_FLAG_HI="-O3" >> C_O_FLAG_NORM="-O2" >> C_O_FLAG_DEBUG_JVM="-O0" >> fi >> C_O_FLAG_SIZE="-Os" >> C_O_FLAG_DEBUG="-O0" >> C_O_FLAG_NONE="-O0" >> elif test "x$TOOLCHAIN_TYPE" = xxlc; then >> >> should be changed to >> >> elif test "x$TOOLCHAIN_TYPE" = xclang; then >> C_O_FLAG_HIGHEST_JVM="-O3" >> C_O_FLAG_HIGHEST="-O3" >> C_O_FLAG_HI="-O3" >> C_O_FLAG_NORM="-O2" >> C_O_FLAG_DEBUG_JVM="-O0" >> C_O_FLAG_SIZE="-Os" >> C_O_FLAG_DEBUG="-O0" >> C_O_FLAG_NONE="-O0" >> elif test "x$TOOLCHAIN_TYPE" = xxlc; then >> >> MacOSX has been paying a historic and significant performance penalty for no valid reason. > This might be a valid change, but it has nothing to do with C++14, and changing it at the same time will increase risk for unrelated strange errors. Please open a separate JBS issue for this requested change. > > /Magnus >> >> Otherwise +1. >> >> Cheers, >> >> -- Jim >> >> >> >>>> On Jun 5, 2020, at 4:52 AM, Kim Barrett wrote: >>> >>> [Changes are only to the build system, but since the changes have jdk-wide >>> effect I?ve cc?d what I think are the relevant dev lists.] >>> >>> This change is part of JEP 347; the intent is to target JDK 16. >>> >>> Please review this change to the building of C++ code in the JDK to >>> enable the use of C++14 language features. This change does not make >>> any code changes to use new features provided by C++11 or C++14. >>> >>> This requires trimming the supported compiler versions, moving the >>> minimum supported versions forward (significantly, in some cases). >>> The new minimums are based on compiler documentation rather than >>> testing. It may be that more recent versions are actually required. >>> >>> This change needs to be tested on platforms not supported by Oracle. >>> The JEP test plan includes additional Oracle testing beyond what I?ve done. >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8246032 >>> >>> Webrev: >>> https://cr.openjdk.java.net/~kbarrett/8246032/open.02/ >>> >>> Testing: >>> mach5 tier1-5 on Oracle supported platforms. >>> >>> Performance testing showed no significant changes in either direction. >>> >>> Build-only (no tests) for linux-arm32, linux-s390x, linux-ppc64le >>> > From joel.borggren.franck at gmail.com Sat Jun 6 09:44:34 2020 From: joel.borggren.franck at gmail.com (=?UTF-8?Q?Joel_Borggr=C3=A9n=2DFranck?=) Date: Sat, 6 Jun 2020 11:44:34 +0200 Subject: 8202469 / 8202473: Correct type annotation resolution for class type variables In-Reply-To: References: <0aaaa441-c193-6155-4da3-5bd91888f73e@oracle.com> Message-ID: Hi Rafael, Let's try to get this into 15, http://cr.openjdk.java.net/~winterhalter/8202473/webrev.00/test/jdk/java/lang/annotation/typeAnnotations/TypeVariableBoundParameterIndex.java.udiff.html looks strange did you unintentionally remove the test for 8202469? The issues are orthogonal so I'd like to keep the cases you developed for the previous fix. I think the fix can lead to further refactorings since this was the last use of buildAnnotatedType where parameter 4 and 5 wasn't identical. This can be followed up later. cheers /Joel On Tue, Apr 7, 2020 at 8:52 PM Rafael Winterhalter wrote: > I created this webrev to also fix the second part that was previously > fixed as part of 8202469: > http://cr.openjdk.java.net/~winterhalter/8202473/webrev.00/ - this would > be the second part of the adjustment. > > This is reported in https://bugs.openjdk.java.net/browse/JDK-8202473 and > was closed as a duplicate. With this cleaner solution, the duplication does > however no longer apply. > > Am Mo., 6. Apr. 2020 um 14:01 Uhr schrieb Joel Borggr?n-Franck < > joel.borggren.franck at gmail.com>: > >> Many thanks to Vicente for sponsoring this. I'll start to look at the >> second part shortly. >> >> cheers >> /Joel >> >> On Mon, Mar 16, 2020 at 9:44 PM Joel Borggr?n-Franck < >> joel.borggren.franck at gmail.com> wrote: >> >>> Looks good to me! >>> >>> I'll see if I can find a sponsor for this. >>> >>> cheers >>> /Joel >>> >>> On Sat, Mar 7, 2020 at 2:15 AM Rafael Winterhalter >>> wrote: >>> >>>> I finally found the time to look at this again, sorry for the delay. >>>> >>>> Thank you for your comments. I had the chance to better look into the >>>> problem and simplify the code a bit more and also reduced the scope of the >>>> fix to a single problem. I also added test cases that should be exhaustive >>>> for all possible scenarios and adjusted the code comment. >>>> >>>> I uploaded the adjusted patch as a webrev: >>>> http://cr.openjdk.java.net/~winterhalter/8202469/webrev.01/ >>>> >>>> Thanks, Rafael >>>> >>>> Am So., 16. Feb. 2020 um 10:51 Uhr schrieb Joel Borggr?n-Franck < >>>> joel.borggren.franck at gmail.com>: >>>> >>>>> Hi Rafael, >>>>> >>>>> Thanks for reaching out and reminding me of this! >>>>> >>>>> I managed to find some time to look into 8202469 during the weekend. >>>>> By swapping place of the TVars in the test I managed to isolate this >>>>> without depending on 8202473, take a look at my hacky copy-paste update to >>>>> your test at http://cr.openjdk.java.net/~jfranck/8202469/ . >>>>> >>>>> The comment on lines 269-276 needs to be updated. Perhaps include a >>>>> table with the start index depending of the type? Also referencing JVMLS >>>>> 4.4 for the structure of the bound might be instructive for future readers. >>>>> >>>>> My current understanding is that there are two problems with the code >>>>> on (the original) lines 277-287: >>>>> 1) Type Variables should have index 0 while getting index 1 due to >>>>> lines 279-280. >>>>> 2) Bounds that are instances of ParameterizedType always gets index 1 >>>>> but if the first bound represents a Class type the index should be 0. >>>>> >>>>> Does this make sense? >>>>> >>>>> Can you make the if-switches positive? I find my original code with >>>>> the negative tests hard to read and the update doesn't help. >>>>> >>>>> Also can you expand the test to cover the different kinds of bounds >>>>> mentioned in 4.4 and also test Type Variables on methods, I suspect javac >>>>> treats method (and constructor) tvars similarly but there have been bugs ... >>>>> >>>>> TL;DR please update the webrev with: >>>>> >>>>> - Split out test and fix for 8202469 >>>>> - More test coverage of different kinds of bounds >>>>> - Test for method tvars >>>>> - See if you can rework the logic to have (mostly) positive tests in >>>>> if switch >>>>> >>>>> Thanks again for looking into this, I'll start looking into 8202473, I >>>>> think the fix for that one opens up a bigger rework of the code which is >>>>> why I want to separate the two. >>>>> >>>>> cheers >>>>> /Joel >>>>> >>>>> On Sun, Aug 4, 2019 at 10:12 PM Rafael Winterhalter < >>>>> rafael.wth at gmail.com> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> appologies for the delay, I only managed to get my patched up to >>>>>> webrev today (http://cr.openjdk.java.net/~winterhalter/). It's three >>>>>> patches in total now, for the third one I could not add a test case, it >>>>>> would require to compile an annotation property against an enumeration type >>>>>> and load it against another class where the enumeration was turned into an >>>>>> annotation. I struggled a bit with jtreg to make it work but I cannot see >>>>>> myself complete this in a meaningful amount of time. However, I hope that >>>>>> the patch and the error it solves are straightforward to see. >>>>>> >>>>>> I only submitted a patch for 8202469. 8202473 is solved by it. It's >>>>>> two bugs but they are a symptom of the same oversight. >>>>>> >>>>>> I hope this helps you to review it but let me know if you have any >>>>>> questions or if I should further adjust my patch. >>>>>> >>>>>> Best regards, Rafael >>>>>> >>>>>> Am Fr., 2. Aug. 2019 um 12:18 Uhr schrieb Rafael Winterhalter < >>>>>> rafael.wth at gmail.com>: >>>>>> >>>>>>> Thanks Daniel, >>>>>>> I did some work on Glassfish a bunch of years ago, I had no idea. >>>>>>> I will try to split up the changes (I fixed another bug in >>>>>>> annotation processing yesterday) and upload everything there. >>>>>>> Best regards, Rafael >>>>>>> >>>>>>> Am Fr., 2. Aug. 2019 um 11:59 Uhr schrieb Daniel Fuchs < >>>>>>> daniel.fuchs at oracle.com>: >>>>>>> >>>>>>>> Hi Rafael, >>>>>>>> >>>>>>>> On 02/08/2019 09:00, Joel Borggr?n-Franck wrote: >>>>>>>> > I can host webrevs for you on cr to make it easier for other >>>>>>>> reviewers, if >>>>>>>> > you also send me the patches or webrefs off-list to get around the >>>>>>>> > attachment stripping I can upload them to cr. >>>>>>>> >>>>>>>> I see that you are a JDK project author, so you already own a page >>>>>>>> on cr (http://cr.openjdk.java.net/~winterhalter/) where you can >>>>>>>> upload webrevs (e.g. using `scp` as in: >>>>>>>> $ scp -r webrev-NNNNN.01 winterhalter at cr.openjdk.java.net: ) >>>>>>>> >>>>>>>> best regards and welcome! >>>>>>>> >>>>>>>> -- daniel >>>>>>>> >>>>>>> From alexey at azul.com Sat Jun 6 19:45:28 2020 From: alexey at azul.com (Alexey Bakhtin) Date: Sat, 6 Jun 2020 19:45:28 +0000 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <9013C8D0-33D9-4D09-8953-F8618020A7F7@oracle.com> References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> <9013C8D0-33D9-4D09-8953-F8618020A7F7@oracle.com> Message-ID: <8A2C4DEF-FC61-4574-B49D-1133F5349BCE@azul.com> Hello Max, Daniel, Thank you for review. Please review new version of the patch : http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v5/ In this version: - TlsChannelBinding class is moved into the com.sun.jndi.ldap.sasl package - SSL Ceritificate related code is moved from LdapClient into the LdapSasl.saslBind method - verification and removal of internal property is also moved to LdapSasl.saslBind method - verification of connectTimeout property is moved to LdapCtx.connect. I?ve found that connectionTimeout could be assigned later then cbType The test for this issue is not ready yet. I did not find any suitable test case that can be reused. Thank you Alexey > On 6 Jun 2020, at 09:44, Weijun Wang wrote: > > > >> On Jun 6, 2020, at 2:41 PM, Weijun Wang wrote: >> >> >> >>> On Jun 5, 2020, at 11:03 PM, Alexey Bakhtin wrote: >>> >>> Hello Max, >>> >>> Thank you a lot for review. >>> >>> Could you check the new version of the patch : >>> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v4/ >>> >>> I?ve made the following changes: >>> - TlsChannelBinding class is moved to java.naming module. >>> java.security.sasl module is not affected any more >>> - I pass tlsCB.getData() directly to the SASL mechanism as you suggested >>> - I?ve made some guards to prevent application from using "com.sun.security.sasl.tlschannelbinding? property directly: >>> - LdapClient verifies if internal property is not set >> >> 245 // Prepare TLS Channel Binding data >> 246 if (conn.sock instanceof SSLSocket) { >> 247 // Internal property cannot be set explicitly >> 248 if (env.get(TlsChannelBinding.CHANNEL_BINDING) != null) { >> 249 throw new NamingException(TlsChannelBinding.CHANNEL_BINDING + >> 250 " property cannot be set explicitly"); >> 251 } >> >> If not TLS, this property value be kept there and visible inside the SASL mech. >> >>> - GssKrb5Client uses props.remove() to read and remove internal property > > Maybe you can remove the value in LdapClient.java, in case the mech used is not GSSAPI. You can remove it in a finally block after line 303. > > --Max > >> >> Traditionally, we use "com.sun..." name which is a JDK supported name (although not at Java SE level), you might want to use a name which is even more internal. >> >> >> Thanks, >> Max >> >> p.s. I see that NTLM also supports ChannelBinding. I'll see if I can improve the NTLM SASL mech to support it. From weijun.wang at oracle.com Sun Jun 7 01:47:32 2020 From: weijun.wang at oracle.com (Weijun Wang) Date: Sun, 7 Jun 2020 09:47:32 +0800 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <8A2C4DEF-FC61-4574-B49D-1133F5349BCE@azul.com> References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> <9013C8D0-33D9-4D09-8953-F8618020A7F7@oracle.com> <8A2C4DEF-FC61-4574-B49D-1133F5349BCE@azul.com> Message-ID: <013290B8-67BE-4CD5-804C-FC882CD23CFE@oracle.com> Some comments: 1. I just noticed your 2 new files are using the plain GPU license, it should be GPL + Classpath. Read another source file for an example. 2. Please add some comments in InitialToken.java on what happens to the default type. 3. I still think "com.sun.security.sasl.tlschannelbinding" sounds like sun.com will support it. 4. If an automatic test is not feasible, please provide some instructions so our SQE can see if we can setup some internal tests. Then we can add noreg-hard with some justification to the test and go on. I cannot comment on LdapCtx.java, but the others look fine to me. Thanks, Max > On Jun 7, 2020, at 3:45 AM, Alexey Bakhtin wrote: > > Hello Max, Daniel, > > Thank you for review. > > Please review new version of the patch : > http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v5/ > > In this version: > - TlsChannelBinding class is moved into the com.sun.jndi.ldap.sasl package > - SSL Ceritificate related code is moved from LdapClient into the LdapSasl.saslBind method > - verification and removal of internal property is also moved to LdapSasl.saslBind method > - verification of connectTimeout property is moved to LdapCtx.connect. I?ve found that connectionTimeout could be assigned later then cbType > > The test for this issue is not ready yet. I did not find any suitable test case that can be reused. > > Thank you > Alexey > >> On 6 Jun 2020, at 09:44, Weijun Wang wrote: >> >> >> >>> On Jun 6, 2020, at 2:41 PM, Weijun Wang wrote: >>> >>> >>> >>>> On Jun 5, 2020, at 11:03 PM, Alexey Bakhtin wrote: >>>> >>>> Hello Max, >>>> >>>> Thank you a lot for review. >>>> >>>> Could you check the new version of the patch : >>>> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v4/ >>>> >>>> I?ve made the following changes: >>>> - TlsChannelBinding class is moved to java.naming module. >>>> java.security.sasl module is not affected any more >>>> - I pass tlsCB.getData() directly to the SASL mechanism as you suggested >>>> - I?ve made some guards to prevent application from using "com.sun.security.sasl.tlschannelbinding? property directly: >>>> - LdapClient verifies if internal property is not set >>> >>> 245 // Prepare TLS Channel Binding data >>> 246 if (conn.sock instanceof SSLSocket) { >>> 247 // Internal property cannot be set explicitly >>> 248 if (env.get(TlsChannelBinding.CHANNEL_BINDING) != null) { >>> 249 throw new NamingException(TlsChannelBinding.CHANNEL_BINDING + >>> 250 " property cannot be set explicitly"); >>> 251 } >>> >>> If not TLS, this property value be kept there and visible inside the SASL mech. >>> >>>> - GssKrb5Client uses props.remove() to read and remove internal property >> >> Maybe you can remove the value in LdapClient.java, in case the mech used is not GSSAPI. You can remove it in a finally block after line 303. >> >> --Max >> >>> >>> Traditionally, we use "com.sun..." name which is a JDK supported name (although not at Java SE level), you might want to use a name which is even more internal. >>> >>> >>> Thanks, >>> Max >>> >>> p.s. I see that NTLM also supports ChannelBinding. I'll see if I can improve the NTLM SASL mech to support it. > From szegedia at gmail.com Sun Jun 7 12:32:35 2020 From: szegedia at gmail.com (Attila Szegedi) Date: Sun, 7 Jun 2020 14:32:35 +0200 Subject: Comparator.reversed() type inference issue Message-ID: <35B0D7FE-265D-4525-B344-45DC3BFA2766@gmail.com> Hi folks, I'm teaching Java lately, and while teaching streams I use that good old chestnut, the word count example. I'm baffled with some type inference woes, though? trying to specify a reverse comparator using Comparator.reversed() makes javac sad, e.g. this does not compile[1]: Map x = someMap(); var rs1 = x.entrySet().stream().sorted( Map.Entry.comparingByValue().reversed() ); On the other hand, using Collections.reverseOrder does compile: var rs2 = x.entrySet().stream().sorted( Collections.reverseOrder( Map.Entry.comparingByValue() ) ); Happens on both Java 11 and 14. It?s just baffling because based on type signatures, it seems reasonable the first form should work as well. Thanks for any enlightenment! Attila. --- [1] -Xdiags:verbose on Java 14 says: error: no suitable method found for sorted(Comparator>) var rs1 = x.entrySet().stream().sorted( method Stream.sorted() is not applicable (actual and formal argument lists differ in length) method Stream.sorted(Comparator>) is not applicable (argument mismatch; Comparator> cannot be converted to Comparator>) where V#1,V#2 are type-variables: V#1 extends Comparable V#2 extends Comparable From rafael.wth at gmail.com Sun Jun 7 21:58:28 2020 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Sun, 7 Jun 2020 23:58:28 +0200 Subject: 8202469 / 8202473: Correct type annotation resolution for class type variables In-Reply-To: References: <0aaaa441-c193-6155-4da3-5bd91888f73e@oracle.com> Message-ID: I just deleted and recreated the file locally to avoid this now. No idea why the webrev command picked it up as a file change. I fixed the patch here: http://cr.openjdk.java.net/~winterhalter/8202473/webrev.02/ As for the possible refactoring: I had investigated it but in the factories for wildcard and parameterized types, the parameters are still provided with different arguments. Am Sa., 6. Juni 2020 um 11:44 Uhr schrieb Joel Borggr?n-Franck < joel.borggren.franck at gmail.com>: > Hi Rafael, > > Let's try to get this into 15, > > > http://cr.openjdk.java.net/~winterhalter/8202473/webrev.00/test/jdk/java/lang/annotation/typeAnnotations/TypeVariableBoundParameterIndex.java.udiff.html > looks strange did you unintentionally remove the test for 8202469? The > issues are orthogonal so I'd like to keep the cases you developed for the > previous fix. > > I think the fix can lead to further refactorings since this was the last > use of buildAnnotatedType where parameter 4 and 5 wasn't identical. This > can be followed up later. > > cheers > /Joel > > On Tue, Apr 7, 2020 at 8:52 PM Rafael Winterhalter > wrote: > >> I created this webrev to also fix the second part that was previously >> fixed as part of 8202469: >> http://cr.openjdk.java.net/~winterhalter/8202473/webrev.00/ - this would >> be the second part of the adjustment. >> >> This is reported in https://bugs.openjdk.java.net/browse/JDK-8202473 and >> was closed as a duplicate. With this cleaner solution, the duplication does >> however no longer apply. >> >> Am Mo., 6. Apr. 2020 um 14:01 Uhr schrieb Joel Borggr?n-Franck < >> joel.borggren.franck at gmail.com>: >> >>> Many thanks to Vicente for sponsoring this. I'll start to look at the >>> second part shortly. >>> >>> cheers >>> /Joel >>> >>> On Mon, Mar 16, 2020 at 9:44 PM Joel Borggr?n-Franck < >>> joel.borggren.franck at gmail.com> wrote: >>> >>>> Looks good to me! >>>> >>>> I'll see if I can find a sponsor for this. >>>> >>>> cheers >>>> /Joel >>>> >>>> On Sat, Mar 7, 2020 at 2:15 AM Rafael Winterhalter < >>>> rafael.wth at gmail.com> wrote: >>>> >>>>> I finally found the time to look at this again, sorry for the delay. >>>>> >>>>> Thank you for your comments. I had the chance to better look into the >>>>> problem and simplify the code a bit more and also reduced the scope of the >>>>> fix to a single problem. I also added test cases that should be exhaustive >>>>> for all possible scenarios and adjusted the code comment. >>>>> >>>>> I uploaded the adjusted patch as a webrev: >>>>> http://cr.openjdk.java.net/~winterhalter/8202469/webrev.01/ >>>>> >>>>> Thanks, Rafael >>>>> >>>>> Am So., 16. Feb. 2020 um 10:51 Uhr schrieb Joel Borggr?n-Franck < >>>>> joel.borggren.franck at gmail.com>: >>>>> >>>>>> Hi Rafael, >>>>>> >>>>>> Thanks for reaching out and reminding me of this! >>>>>> >>>>>> I managed to find some time to look into 8202469 during the weekend. >>>>>> By swapping place of the TVars in the test I managed to isolate this >>>>>> without depending on 8202473, take a look at my hacky copy-paste update to >>>>>> your test at http://cr.openjdk.java.net/~jfranck/8202469/ . >>>>>> >>>>>> The comment on lines 269-276 needs to be updated. Perhaps include a >>>>>> table with the start index depending of the type? Also referencing JVMLS >>>>>> 4.4 for the structure of the bound might be instructive for future readers. >>>>>> >>>>>> My current understanding is that there are two problems with the code >>>>>> on (the original) lines 277-287: >>>>>> 1) Type Variables should have index 0 while getting index 1 due to >>>>>> lines 279-280. >>>>>> 2) Bounds that are instances of ParameterizedType always gets index 1 >>>>>> but if the first bound represents a Class type the index should be 0. >>>>>> >>>>>> Does this make sense? >>>>>> >>>>>> Can you make the if-switches positive? I find my original code with >>>>>> the negative tests hard to read and the update doesn't help. >>>>>> >>>>>> Also can you expand the test to cover the different kinds of bounds >>>>>> mentioned in 4.4 and also test Type Variables on methods, I suspect javac >>>>>> treats method (and constructor) tvars similarly but there have been bugs ... >>>>>> >>>>>> TL;DR please update the webrev with: >>>>>> >>>>>> - Split out test and fix for 8202469 >>>>>> - More test coverage of different kinds of bounds >>>>>> - Test for method tvars >>>>>> - See if you can rework the logic to have (mostly) positive tests in >>>>>> if switch >>>>>> >>>>>> Thanks again for looking into this, I'll start looking into 8202473, >>>>>> I think the fix for that one opens up a bigger rework of the code which is >>>>>> why I want to separate the two. >>>>>> >>>>>> cheers >>>>>> /Joel >>>>>> >>>>>> On Sun, Aug 4, 2019 at 10:12 PM Rafael Winterhalter < >>>>>> rafael.wth at gmail.com> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> appologies for the delay, I only managed to get my patched up to >>>>>>> webrev today (http://cr.openjdk.java.net/~winterhalter/). It's >>>>>>> three patches in total now, for the third one I could not add a test case, >>>>>>> it would require to compile an annotation property against an enumeration >>>>>>> type and load it against another class where the enumeration was turned >>>>>>> into an annotation. I struggled a bit with jtreg to make it work but I >>>>>>> cannot see myself complete this in a meaningful amount of time. However, I >>>>>>> hope that the patch and the error it solves are straightforward to see. >>>>>>> >>>>>>> I only submitted a patch for 8202469. 8202473 is solved by it. It's >>>>>>> two bugs but they are a symptom of the same oversight. >>>>>>> >>>>>>> I hope this helps you to review it but let me know if you have any >>>>>>> questions or if I should further adjust my patch. >>>>>>> >>>>>>> Best regards, Rafael >>>>>>> >>>>>>> Am Fr., 2. Aug. 2019 um 12:18 Uhr schrieb Rafael Winterhalter < >>>>>>> rafael.wth at gmail.com>: >>>>>>> >>>>>>>> Thanks Daniel, >>>>>>>> I did some work on Glassfish a bunch of years ago, I had no idea. >>>>>>>> I will try to split up the changes (I fixed another bug in >>>>>>>> annotation processing yesterday) and upload everything there. >>>>>>>> Best regards, Rafael >>>>>>>> >>>>>>>> Am Fr., 2. Aug. 2019 um 11:59 Uhr schrieb Daniel Fuchs < >>>>>>>> daniel.fuchs at oracle.com>: >>>>>>>> >>>>>>>>> Hi Rafael, >>>>>>>>> >>>>>>>>> On 02/08/2019 09:00, Joel Borggr?n-Franck wrote: >>>>>>>>> > I can host webrevs for you on cr to make it easier for other >>>>>>>>> reviewers, if >>>>>>>>> > you also send me the patches or webrefs off-list to get around >>>>>>>>> the >>>>>>>>> > attachment stripping I can upload them to cr. >>>>>>>>> >>>>>>>>> I see that you are a JDK project author, so you already own a page >>>>>>>>> on cr (http://cr.openjdk.java.net/~winterhalter/) where you can >>>>>>>>> upload webrevs (e.g. using `scp` as in: >>>>>>>>> $ scp -r webrev-NNNNN.01 winterhalter at cr.openjdk.java.net: ) >>>>>>>>> >>>>>>>>> best regards and welcome! >>>>>>>>> >>>>>>>>> -- daniel >>>>>>>>> >>>>>>>> From liyu.italy at gmail.com Mon Jun 8 00:45:23 2020 From: liyu.italy at gmail.com (Lisa Li) Date: Mon, 8 Jun 2020 08:45:23 +0800 Subject: [PATCH] 8245694 : java.util.Properties.entrySet() does not override java.lang.Object methods In-Reply-To: References: <9487a1c9-d0bc-3632-156d-5927e5801dc8@icemanx.nl> <20109712-d505-1b17-3e6b-ae388d554400@oracle.com> <767a4dae-55f0-ca11-dbdf-57c7a22205d3@oracle.com> Message-ID: Hi, Both the fix and test have been modified accordingly. Please help review 'em. I'd appreciate your thoughts on it. *BUG DESCRIPTION :* JDK-8245694 *PATCH : * # HG changeset patch # User Yu Li # Date 1591541947 -28800 # Sun Jun 07 22:59:07 2020 +0800 # Node ID f5715d275e4f683f5c4bf39d457293904f27330e # Parent d0d06b8be6789994881dc81d50573f60ad2ef844 8245694: java.util.Properties.entrySet() does not override Object methods Summary: Add missing override methods Contributed-by: Yu Li diff -r d0d06b8be678 -r f5715d275e4f src/java.base/share/classes/java/util/Properties.java --- a/src/java.base/share/classes/java/util/Properties.java Fri May 29 11:58:00 2020 +0200 +++ b/src/java.base/share/classes/java/util/Properties.java Sun Jun 07 22:59:07 2020 +0800 @@ -1405,6 +1405,21 @@ } @Override + public boolean equals(Object o) { + return o == this || entrySet.equals(o); + } + + @Override + public int hashCode() { + return entrySet.hashCode(); + } + + @Override + public String toString() { + return entrySet.toString(); + } + + @Override public boolean removeAll(Collection c) { return entrySet.removeAll(c); } diff -r d0d06b8be678 -r f5715d275e4f test/jdk/java/util/Properties/PropertiesEntrySetTest.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/jdk/java/util/Properties/PropertiesEntrySetTest.java Sun Jun 07 22:59:07 2020 +0800 @@ -0,0 +1,201 @@ +/* + * Copyright (c) 2020, Oracle and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + */ + +/* + * @test + * @bug 8245694 + * @summary tests the entrySet() method of Properties class + * @author Yu Li + * @run testng PropertiesEntrySetTest + */ + +import org.testng.annotations.Test; + +import java.util.Properties; + +import static org.testng.Assert.assertEquals; +import static org.testng.Assert.assertFalse; +import static org.testng.Assert.assertThrows; +import static org.testng.Assert.assertTrue; + +public class PropertiesEntrySetTest { + + @Test + public void testEquals() { + Properties a = new Properties(); + var aEntrySet = a.entrySet(); + assertFalse(aEntrySet.equals(null)); + assertTrue(aEntrySet.equals(aEntrySet)); + + Properties b = new Properties(); + var bEntrySet = b.entrySet(); + assertTrue(bEntrySet.equals(aEntrySet)); + assertTrue(bEntrySet.hashCode() == aEntrySet.hashCode()); + + a.setProperty("p1", "1"); + assertFalse(bEntrySet.equals(aEntrySet)); + assertFalse(bEntrySet.hashCode() == aEntrySet.hashCode()); + + b.setProperty("p1", "1"); + assertTrue(aEntrySet.equals(bEntrySet)); + assertTrue(bEntrySet.hashCode() == aEntrySet.hashCode()); + + Properties c = new Properties(); + c.setProperty("p1", "2"); + var cEntrySet = c.entrySet(); + assertFalse(cEntrySet.equals(bEntrySet)); + assertFalse(bEntrySet.hashCode() == cEntrySet.hashCode()); + assertFalse(cEntrySet.equals(aEntrySet)); + assertFalse(aEntrySet.hashCode() == cEntrySet.hashCode()); + + a.setProperty("p2", "2"); + Properties d = new Properties(); + d.setProperty("p2", "2"); + d.setProperty("p1", "1"); + var dEntrySet = d.entrySet(); + assertTrue(dEntrySet.equals(aEntrySet)); + assertTrue(aEntrySet.hashCode() == dEntrySet.hashCode()); + + a.remove("p1"); + assertFalse(aEntrySet.equals(dEntrySet)); + assertFalse(aEntrySet.hashCode() == dEntrySet.hashCode()); + + d.remove("p1", "1"); + assertTrue(dEntrySet.equals(aEntrySet)); + assertTrue(aEntrySet.hashCode() == dEntrySet.hashCode()); + + a.clear(); + assertFalse(aEntrySet.equals(dEntrySet)); + assertFalse(aEntrySet.hashCode() == dEntrySet.hashCode()); + assertTrue(aEntrySet.isEmpty()); + + d.clear(); + assertTrue(dEntrySet.equals(aEntrySet)); + assertTrue(aEntrySet.hashCode() == dEntrySet.hashCode()); + assertTrue(dEntrySet.isEmpty()); + } + + @Test + public void testToString() { + Properties a = new Properties(); + var aEntrySet = a.entrySet(); + assertEquals(aEntrySet.toString(), "[]"); + + a.setProperty("p1", "1"); + assertEquals(aEntrySet.toString(), "[p1=1]"); + + a.setProperty("p2", "2"); + assertEquals(aEntrySet.size(), 2); + assertTrue(aEntrySet.toString().trim().startsWith("[")); + assertTrue(aEntrySet.toString().contains("p1=1")); + assertTrue(aEntrySet.toString().contains("p2=2")); + assertTrue(aEntrySet.toString().trim().endsWith("]")); + + Properties b = new Properties(); + b.setProperty("p2", "2"); + b.setProperty("p1", "1"); + var bEntrySet = b.entrySet(); + assertEquals(bEntrySet.size(), 2); + assertTrue(bEntrySet.toString().trim().startsWith("[")); + assertTrue(bEntrySet.toString().contains("p1=1")); + assertTrue(bEntrySet.toString().contains("p2=2")); + assertTrue(bEntrySet.toString().trim().endsWith("]")); + + b.setProperty("p0", "0"); + assertEquals(bEntrySet.size(), 3); + assertTrue(bEntrySet.toString().contains("p0=0")); + + b.remove("p1"); + assertEquals(bEntrySet.size(), 2); + assertFalse(bEntrySet.toString().contains("p1=1")); + assertTrue(bEntrySet.toString().trim().startsWith("[")); + assertTrue(bEntrySet.toString().contains("p0=0")); + assertTrue(bEntrySet.toString().contains("p2=2")); + assertTrue(bEntrySet.toString().trim().endsWith("]")); + + b.remove("p0", "0"); + assertEquals(bEntrySet.size(), 1); + assertFalse(bEntrySet.toString().contains("p0=0")); + assertTrue(bEntrySet.toString().trim().startsWith("[")); + assertTrue(bEntrySet.toString().contains("p2=2")); + assertTrue(bEntrySet.toString().trim().endsWith("]")); + + b.clear(); + assertTrue(bEntrySet.isEmpty()); + assertTrue(bEntrySet.toString().equals("[]")); + } + + @Test + public void testEntrySetWithoutException() { + Properties a = new Properties(); + a.setProperty("p1", "1"); + a.setProperty("p2", "2"); + var aEntrySet = a.entrySet(); + assertEquals(aEntrySet.size(), 2); + + var i = aEntrySet.iterator(); + var e1 = i.next(); + i.remove(); + assertFalse(aEntrySet.contains(e1)); + assertEquals(aEntrySet.size(), 1); + + var e2 = i.next(); + aEntrySet.remove(e2); + assertFalse(aEntrySet.contains(e2)); + assertTrue(aEntrySet.isEmpty()); + + a.setProperty("p1", "1"); + a.setProperty("p3", "3"); + Properties b = new Properties(); + b.setProperty("p2", "2"); + b.setProperty("p1", "1"); + var bEntrySet = b.entrySet(); + + assertFalse(bEntrySet.containsAll(aEntrySet)); + assertEquals(bEntrySet.size(), 2); + + assertTrue(bEntrySet.removeAll(aEntrySet)); + assertEquals(bEntrySet.size(), 1); + + assertTrue(bEntrySet.retainAll(aEntrySet)); + assertTrue(bEntrySet.isEmpty()); + assertEquals(aEntrySet.size(), 2); + + aEntrySet.clear(); + assertTrue(aEntrySet.isEmpty()); + } + + @Test + public void testEntrySetExceptionWhenAdd() { + Properties a = new Properties(); + a.setProperty("p1", "1"); + var aEntrySet = a.entrySet(); + + Properties b = new Properties(); + b.setProperty("p2", "2"); + var bEntrySet = b.entrySet(); + + assertThrows(UnsupportedOperationException.class, () -> aEntrySet.addAll(bEntrySet)); + assertThrows(UnsupportedOperationException.class, () -> aEntrySet.add(bEntrySet.iterator().next())); + } +} Best regards, Yu L. On Wed, Jun 3, 2020 at 1:37 PM Lisa Li wrote: > Hi? > > Thanks a lot Julia! I have signed OCA and emailed it to Oracle last week. > But I haven't seen my name on the contributor list yet. I guess I'll have > to wait. > > And thanks a lot for all the advice, Brent! It's really useful. I'll do as > you said. > > Best regards, > Yu > > On Wed, Jun 3, 2020 at 7:51 AM Brent Christian > wrote: > >> Thank you for submitting the fix, Yu Li. (And thanks for the sponsor, >> Julia). This will be good to fix. I have a few changes to recommend. >> >> src/java.base/share/classes/java/util/Properties.java >> >> The code change looks fine to me. >> >> The ending Copyright year just needs to be changed from 2019 -> 2020. >> >> >> test/jdk/java/util/Properties/PropertiesEntrySetTest.java >> >> + * Copyright (c) 2018, 2019, Oracle and/or its affiliates. All rights >> >> For a newly added file, only the current year is listed, so: >> >> Copyright (c) 2020, Oracle and/or its affiliates. All rights >> reserved. >> >> +/* >> + * @test >> + * @summary tests the entrySet() method of Properties class >> + * @author Yu Li >> + * @bug 8245694 >> + */ >> >> Please also add an @run tag. Using testng would prevent needing to add >> assert[True|False] methods. So that would be: >> >> @run testng PropertiesEntrySetTest >> >> Then you should be able to: >> >> import org.testng.Assert; >> >> and use the testng Assert.assertTrue() and Assert.assertFalse(). >> >> >> + Set> aEntrySet = a.entrySet(); >> >> You might consider using 'var' to simplify declarations of aEntrySet, >> bEntrySet, etc >> >> >> Please also add testing of EntrySet.toString(). >> >> Along with testing equals/hashCode/toString after properties are added, >> I think it would be good to also test after properties are removed. >> There's Properties.remove()/clear(), and also maybe some of the removal >> methods listed in Hashtable.entrySet(): Iterator.remove(), Set.remove(), >> etc. >> >> >> (And just as a point of interest, there is a test, >> test/jdk/java/util/Properties/CheckOverrides.java, >> to ensure that the Properties class overrides all necessary methods of >> its superclasses. Checks were not included for Properties.EntrySet >> (hence this bug), though I don't think it makes sense to add any at this >> point.) >> >> Thanks a lot, >> -Brent >> >> On 6/2/20 8:43 AM, Julia Boes wrote: >> > Hi Yu Li, >> > >> > I'm happy to sponsor your fix once it's reviewed. Could you just >> confirm >> > that you have signed the Oracle Contributor Agreement? >> > >> > https://www.oracle.com/technetwork/community/oca-486395.html >> > >> > Cheers, >> > >> > Julia >> > >> > On 30/05/2020 21:00, Rob Spoor wrote: >> >> There's still a little regression there since Java 8, if you add the >> >> keySet(), values() or entrySet() to the Properties object as either >> >> key or value; in Java 8 this would cause "(this Collection)" to be >> >> included, while even with this fix that would cause a >> >> StackOverflowError. However, given that it's only a corner case for >> >> most Collections and Maps in the first place, and with Properties >> >> objects even more so, I don't think anyone would mind this regression. >> >> >> >> Rob >> >> >> >> On 30/05/2020 20:31, Lisa Li wrote: >> >>> Hi, >> >>> >> >>> Please help review the fix for JDK-8245694 >> >>> . And this is my >> very >> >>> first patch submission. I know it's not perfect. >> >>> >> >>> >> >>> >> >>> *BUG DESCRIPTION*: >> >>> >> >>> JDK-8245694 : >> >>> java.util.Properties.entrySet() >> >>> does not override java.lang.Object methods since java 9. >> >>> >> >>> >> >>> *PROPOSED FIX*: >> >>> >> >>> Add delegating overrides for equals(), hashCode(), and toString(). >> >>> >> >>> >> >>> *PATCH*: >> >>> >> >>> # HG changeset patch >> >>> # User yuli >> >>> # Date 1590841711 -28800 >> >>> # Sat May 30 20:28:31 2020 +0800 >> >>> # Node ID 7dc946e5b5863291fa070bfdbdce48aefbe87b7b >> >>> # Parent 8e28aae5068069e853673148e4d3f44cb50350a7 >> >>> 8245694: java.util.Properties.entrySet() does not override Object >> >>> methods >> >>> Summary: Add missing override methods >> >>> Contributed-by: Yu Li >> >>> >> >>> diff --git a/src/java.base/share/classes/java/util/Properties.java >> >>> b/src/java.base/share/classes/java/util/Properties.java >> >>> --- a/src/java.base/share/classes/java/util/Properties.java >> >>> +++ b/src/java.base/share/classes/java/util/Properties.java >> >>> @@ -1414,6 +1414,21 @@ >> >>> return entrySet.retainAll(c); >> >>> } >> >>> >> >>> + @Override >> >>> + public boolean equals(Object o) { >> >>> + return entrySet.equals(o); >> >>> + } >> >>> + >> >>> + @Override >> >>> + public int hashCode() { >> >>> + return entrySet.hashCode(); >> >>> + } >> >>> + >> >>> + @Override >> >>> + public String toString() { >> >>> + return entrySet.toString(); >> >>> + } >> >>> + >> >>> @Override >> >>> public Iterator> iterator() { >> >>> return entrySet.iterator(); >> >>> diff --git a/test/jdk/java/util/Properties/PropertiesEntrySetTest.java >> >>> b/test/jdk/java/util/Properties/PropertiesEntrySetTest.java >> >>> new file mode 100644 >> >>> --- /dev/null >> >>> +++ b/test/jdk/java/util/Properties/PropertiesEntrySetTest.java >> >>> @@ -0,0 +1,90 @@ >> >>> +/* >> >>> + * Copyright (c) 2018, 2019, Oracle and/or its affiliates. All rights >> >>> reserved. >> >>> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >> >>> + * >> >>> + * This code is free software; you can redistribute it and/or modify >> it >> >>> + * under the terms of the GNU General Public License version 2 only, >> as >> >>> + * published by the Free Software Foundation. >> >>> + * >> >>> + * This code is distributed in the hope that it will be useful, but >> >>> WITHOUT >> >>> + * ANY WARRANTY; without even the implied warranty of >> >>> MERCHANTABILITY or >> >>> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public >> >>> License >> >>> + * version 2 for more details (a copy is included in the LICENSE >> >>> file that >> >>> + * accompanied this code). >> >>> + * >> >>> + * You should have received a copy of the GNU General Public License >> >>> version >> >>> + * 2 along with this work; if not, write to the Free Software >> >>> Foundation, >> >>> + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. >> >>> + * >> >>> + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA >> >>> 94065 USA >> >>> + * or visit www.oracle.com if you need additional information or >> >>> have any >> >>> + * questions. >> >>> + */ >> >>> + >> >>> +/* >> >>> + * @test >> >>> + * @summary tests the entrySet() method of Properties class >> >>> + * @author Yu Li >> >>> + * @bug 8245694 >> >>> + */ >> >>> + >> >>> +import java.util.Map; >> >>> +import java.util.Properties; >> >>> +import java.util.Set; >> >>> + >> >>> +public class PropertiesEntrySetTest { >> >>> + >> >>> + public static void main(String[] args) { >> >>> + >> >>> + testEntrySet(); >> >>> + >> >>> + } >> >>> + >> >>> + private static void assertTrue(boolean value) { >> >>> + if (!value) { >> >>> + throw new RuntimeException("Failure in Properties >> >>> testing."); >> >>> + } >> >>> + } >> >>> + >> >>> + private static void assertFalse(boolean value) { >> >>> + if (value) { >> >>> + throw new RuntimeException("Failure in Properties >> >>> testing."); >> >>> + } >> >>> + } >> >>> + >> >>> + private static void testEntrySet() { >> >>> + Properties a = new Properties(); >> >>> + Set> aEntrySet = a.entrySet(); >> >>> + assertFalse(aEntrySet.equals(null)); >> >>> + assertTrue(aEntrySet.equals(aEntrySet)); >> >>> + >> >>> + Properties b = new Properties(); >> >>> + Set> bEntrySet = b.entrySet(); >> >>> + assertTrue(aEntrySet.equals(bEntrySet)); >> >>> + assertTrue(aEntrySet.hashCode() == bEntrySet.hashCode()); >> >>> + >> >>> + a.setProperty("p1", "1"); >> >>> + assertFalse(aEntrySet.equals(bEntrySet)); >> >>> + assertFalse(bEntrySet.equals(aEntrySet)); >> >>> + assertFalse(aEntrySet.hashCode() == bEntrySet.hashCode()); >> >>> + >> >>> + b.setProperty("p1", "1"); >> >>> + assertTrue(aEntrySet.equals(bEntrySet)); >> >>> + assertTrue(aEntrySet.hashCode() == bEntrySet.hashCode()); >> >>> + >> >>> + Properties c = new Properties(); >> >>> + c.setProperty("p1", "2"); >> >>> + Set> cEntrySet = c.entrySet(); >> >>> + assertFalse(cEntrySet.equals(bEntrySet)); >> >>> + assertFalse(cEntrySet.hashCode() == bEntrySet.hashCode()); >> >>> + assertFalse(cEntrySet.equals(aEntrySet)); >> >>> + assertFalse(cEntrySet.hashCode() == aEntrySet.hashCode()); >> >>> + >> >>> + a.setProperty("p2", "2"); >> >>> + Properties d = new Properties(); >> >>> + d.setProperty("p2", "2"); >> >>> + d.setProperty("p1", "1"); >> >>> + Set> dEntrySet = d.entrySet(); >> >>> + assertTrue(dEntrySet.equals(aEntrySet)); >> >>> + } >> >>> +} >> >>> >> >>> Best regards, >> >>> Yu Li >> > From julia.boes at oracle.com Mon Jun 8 09:38:26 2020 From: julia.boes at oracle.com (Julia Boes) Date: Mon, 8 Jun 2020 10:38:26 +0100 Subject: [PATCH] 8245694 : java.util.Properties.entrySet() does not override java.lang.Object methods In-Reply-To: References: <9487a1c9-d0bc-3632-156d-5927e5801dc8@icemanx.nl> <20109712-d505-1b17-3e6b-ae388d554400@oracle.com> <767a4dae-55f0-ca11-dbdf-57c7a22205d3@oracle.com> Message-ID: <7b01289c-d7c9-47ab-f9f6-e2cf3a00f2da@oracle.com> Hi Yu Li, The copyright year of Properties.java needs to be updated to 2020. Otherwise looks good to me! Cheers, Julia From liyu.italy at gmail.com Mon Jun 8 10:56:51 2020 From: liyu.italy at gmail.com (Lisa Li) Date: Mon, 8 Jun 2020 18:56:51 +0800 Subject: [PATCH] 8245694 : java.util.Properties.entrySet() does not override java.lang.Object methods In-Reply-To: <7b01289c-d7c9-47ab-f9f6-e2cf3a00f2da@oracle.com> References: <9487a1c9-d0bc-3632-156d-5927e5801dc8@icemanx.nl> <20109712-d505-1b17-3e6b-ae388d554400@oracle.com> <767a4dae-55f0-ca11-dbdf-57c7a22205d3@oracle.com> <7b01289c-d7c9-47ab-f9f6-e2cf3a00f2da@oracle.com> Message-ID: Hi Julia, Thanks a lot! Didn't realise I was allowed to modify the copyright of Properties.java. Sure, will do! Best regards, Yu L. On Mon, Jun 8, 2020 at 5:41 PM Julia Boes wrote: > Hi Yu Li, > > The copyright year of Properties.java needs to be updated to 2020. > Otherwise looks good to me! > > Cheers, > > Julia > > From aleksej.efimov at oracle.com Mon Jun 8 13:07:21 2020 From: aleksej.efimov at oracle.com (Aleks Efimov) Date: Mon, 8 Jun 2020 14:07:21 +0100 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <8A2C4DEF-FC61-4574-B49D-1133F5349BCE@azul.com> References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> <9013C8D0-33D9-4D09-8953-F8618020A7F7@oracle.com> <8A2C4DEF-FC61-4574-B49D-1133F5349BCE@azul.com> Message-ID: Hi Alexey, I've looked through LdapCtx/LdapClient/SaslBind changes: Do we need to check if CHANNEL_BINDING is set explicitly for all connection types? Maybe we can move the check inside 'if (conn.sock instanceof SSLSocket) {' block. Also, instead of setting CHANNEL_BINDING in context environment and then removing it in finally block, it would be better to clone the environment, put calculated CHANNEL_BINDING into it, and pass the cloned one to Sasl.createSaslClient. Another suggestion about the code that verifies if both properties are set before connection is started: As you've already mentioned the new code in LdapCtx is only needed for checking if timeout is set. Could we try to remove LdapCtx::cbType field and all related methods from LdapCtx (this class is already over-complicated and hard to read) and replace it with some static method in LdapSasl? It will help to localize all changes to LdapSasl except for one line in LdapCtx. I mean something like this: Replace + +??????????? // verify LDAP channel binding property +??????????? if (cbType != null && connectTimeout == -1) +??????????????????? throw new NamingException(TlsChannelBinding.CHANNEL_BINDING_TYPE + +??????????????????????????? " property requires " + +??????????????????????????? CONNECT_TIMEOUT + +??????????????????????????? " property is set."); With + LdapSasl.checkCbParameters((String)envprops.get(TlsChannelBinding.CHANNEL_BINDING_TYPE), connectTimeout); And add something like that to LdapSasl (or maybe pass the full env here): + public static void checkCbParameters(String cbTypePropertyValue, int connectTimeout) throws NamingException { +???? TlsChannelBindingType cbType = TlsChannelBinding.parseType(cbTypePropertyValue); +???? // verify LDAP channel binding property +???? if (cbType != null && connectTimeout == -1) { +???????? throw new NamingException(TlsChannelBinding.CHANNEL_BINDING_TYPE + +???????????????? " property requires com.sun.jndi.ldap.connect.timeout" + +???????????????? " property is set."); +???? } + } Other LdapCtx/LdapClient/SaslBind? changes look fine to me. With Kind Regards, Aleksei On 06/06/2020 20:45, Alexey Bakhtin wrote: > Hello Max, Daniel, > > Thank you for review. > > Please review new version of the patch : > http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v5/ > > In this version: > - TlsChannelBinding class is moved into the com.sun.jndi.ldap.sasl package > - SSL Ceritificate related code is moved from LdapClient into the LdapSasl.saslBind method > - verification and removal of internal property is also moved to LdapSasl.saslBind method > - verification of connectTimeout property is moved to LdapCtx.connect. I?ve found that connectionTimeout could be assigned later then cbType > > The test for this issue is not ready yet. I did not find any suitable test case that can be reused. > > Thank you > Alexey > >> On 6 Jun 2020, at 09:44, Weijun Wang wrote: >> >> >> >>> On Jun 6, 2020, at 2:41 PM, Weijun Wang wrote: >>> >>> >>> >>>> On Jun 5, 2020, at 11:03 PM, Alexey Bakhtin wrote: >>>> >>>> Hello Max, >>>> >>>> Thank you a lot for review. >>>> >>>> Could you check the new version of the patch : >>>> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v4/ >>>> >>>> I?ve made the following changes: >>>> - TlsChannelBinding class is moved to java.naming module. >>>> java.security.sasl module is not affected any more >>>> - I pass tlsCB.getData() directly to the SASL mechanism as you suggested >>>> - I?ve made some guards to prevent application from using "com.sun.security.sasl.tlschannelbinding? property directly: >>>> - LdapClient verifies if internal property is not set >>> 245 // Prepare TLS Channel Binding data >>> 246 if (conn.sock instanceof SSLSocket) { >>> 247 // Internal property cannot be set explicitly >>> 248 if (env.get(TlsChannelBinding.CHANNEL_BINDING) != null) { >>> 249 throw new NamingException(TlsChannelBinding.CHANNEL_BINDING + >>> 250 " property cannot be set explicitly"); >>> 251 } >>> >>> If not TLS, this property value be kept there and visible inside the SASL mech. >>> >>>> - GssKrb5Client uses props.remove() to read and remove internal property >> Maybe you can remove the value in LdapClient.java, in case the mech used is not GSSAPI. You can remove it in a finally block after line 303. >> >> --Max >> >>> Traditionally, we use "com.sun..." name which is a JDK supported name (although not at Java SE level), you might want to use a name which is even more internal. >>> >>> >>> Thanks, >>> Max >>> >>> p.s. I see that NTLM also supports ChannelBinding. I'll see if I can improve the NTLM SASL mech to support it. From james.laskey at oracle.com Mon Jun 8 13:28:19 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Mon, 8 Jun 2020 10:28:19 -0300 Subject: [OpenJDK 2D-Dev] RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: References: <84EEE234-A21F-400B-917C-8DF55236E38E@oracle.com> Message-ID: Hendrik, Yes, I will file one once I have some data. The benefits are not clear cut and I should annotate the request with validity. :-) Cheers, -- Jim > On Jun 6, 2020, at 11:13 AM, Hendrik Schreiber wrote: > > Jim, > > if there isn?t a dedicated bug report for this (meaning: lack of optimization for macOS), please do create one so that it at least is documented somewhere. > > Thank you, > > -hendrik > >> On Jun 5, 2020, at 13:59, Jim Laskey > wrote: >> >> I know there was a discussion about this elsewhere but I would like to take the opportunity to correct this now >> >> make//autoconf/flags-cflags.m4:241 >> >> elif test "x$TOOLCHAIN_TYPE" = xclang; then >> if test "x$OPENJDK_TARGET_OS" = xmacosx; then >> # On MacOSX we optimize for size, something >> # we should do for all platforms? >> C_O_FLAG_HIGHEST_JVM="-Os" >> C_O_FLAG_HIGHEST="-Os" >> C_O_FLAG_HI="-Os" >> C_O_FLAG_NORM="-Os" >> C_O_FLAG_DEBUG_JVM="" >> else >> C_O_FLAG_HIGHEST_JVM="-O3" >> C_O_FLAG_HIGHEST="-O3" >> C_O_FLAG_HI="-O3" >> C_O_FLAG_NORM="-O2" >> C_O_FLAG_DEBUG_JVM="-O0" >> fi >> C_O_FLAG_SIZE="-Os" >> C_O_FLAG_DEBUG="-O0" >> C_O_FLAG_NONE="-O0" >> elif test "x$TOOLCHAIN_TYPE" = xxlc; then >> >> should be changed to >> >> elif test "x$TOOLCHAIN_TYPE" = xclang; then >> C_O_FLAG_HIGHEST_JVM="-O3" >> C_O_FLAG_HIGHEST="-O3" >> C_O_FLAG_HI="-O3" >> C_O_FLAG_NORM="-O2" >> C_O_FLAG_DEBUG_JVM="-O0" >> C_O_FLAG_SIZE="-Os" >> C_O_FLAG_DEBUG="-O0" >> C_O_FLAG_NONE="-O0" >> elif test "x$TOOLCHAIN_TYPE" = xxlc; then >> >> MacOSX has been paying a historic and significant performance penalty for no valid reason. >> >> Otherwise +1. >> >> Cheers, >> >> -- Jim >> >> >> >>> On Jun 5, 2020, at 4:52 AM, Kim Barrett > wrote: >>> >>> [Changes are only to the build system, but since the changes have jdk-wide >>> effect I?ve cc?d what I think are the relevant dev lists.] >>> >>> This change is part of JEP 347; the intent is to target JDK 16. >>> >>> Please review this change to the building of C++ code in the JDK to >>> enable the use of C++14 language features. This change does not make >>> any code changes to use new features provided by C++11 or C++14. >>> >>> This requires trimming the supported compiler versions, moving the >>> minimum supported versions forward (significantly, in some cases). >>> The new minimums are based on compiler documentation rather than >>> testing. It may be that more recent versions are actually required. >>> >>> This change needs to be tested on platforms not supported by Oracle. >>> The JEP test plan includes additional Oracle testing beyond what I?ve done. >>> >>> CR: >>> https://bugs.openjdk.java.net/browse/JDK-8246032 >>> >>> Webrev: >>> https://cr.openjdk.java.net/~kbarrett/8246032/open.02/ >>> >>> Testing: >>> mach5 tier1-5 on Oracle supported platforms. >>> >>> Performance testing showed no significant changes in either direction. >>> >>> Build-only (no tests) for linux-arm32, linux-s390x, linux-ppc64le >>> >> > From alexey.semenyuk at oracle.com Mon Jun 8 14:26:22 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Mon, 8 Jun 2020 10:26:22 -0400 Subject: RFR: JDK-8231283: Evaluate option to install Linux packages in /usr hierarchy In-Reply-To: <8a3f9c48-9561-3a4a-7e07-53a9ebbc1539@oracle.com> References: <8a3f9c48-9561-3a4a-7e07-53a9ebbc1539@oracle.com> Message-ID: Please review fix [2] for jpackage bug [1]. Add support to jpackage to create Linux packages installing app images in '/usr' tree with splicing of the app image. For --install-dir=/usr jpackag option the resulting Debian package will have the following layout: - Launchers will be installed in `/usr/bin` directory - Java runtime will be installed in `/usr/lib/$PACKAGE_NAME/runtime` directory - `app` directory of the app image will be installed in `/usr/lib/$PACKAGE_NAME/app` directory - copyright file will be installed in `/usr/share/doc/$PACKAGE_NAME/copyright` file RPM package will have the following layout: - Launchers will be installed in `/usr/bin` directory - Java runtime will be installed in `/usr/lib/$PACKAGE_NAME/runtime` directory - `app` directory of the app image will be installed in `/usr/lib/$PACKAGE_NAME/app` directory For --install-dir=/usr/local jpackag option the resulting Debian package will have the following layout: - Launchers will be installed in `/usr/local/bin` directory - Java runtime will be installed in `/usr/local/lib/$PACKAGE_NAME/runtime` directory - `app` directory of the app image will be installed in `/usr/local/lib/$PACKAGE_NAME/app` directory - copyright file will be installed in `/usr/share/doc/$PACKAGE_NAME/copyright` file RPM package will have the following layout: - Launchers will be installed in `/usr/local/bin` directory - Java runtime will be installed in `/usr/local/lib/$PACKAGE_NAME/runtime` directory - `app` directory of the app image will be installed in `/usr/local/lib/$PACKAGE_NAME/app` directory For any other prefixes starting with `/usr` no app image splicing will happen and package layout will be the same as for installing at the default `/opt` prefix. - Alexey [1] https://bugs.openjdk.java.net/browse/JDK-8231283 [2]?http://cr.openjdk.java.net/~asemenyuk/8231283/webrev.00 From claes.redestad at oracle.com Mon Jun 8 14:51:28 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 8 Jun 2020 16:51:28 +0200 Subject: 8246632: StringConcatFactory::makeConcatWithConstants no longer throws NullPointerException when an unexpected constant is null Message-ID: <9f261708-df7a-03bf-b955-21a11131ce90@oracle.com> Hi, JDK-8246152 triggered a failure in a JCK test that expects a NPE rather than a StringConcatException. We can keep the old behavior, while retaining most of performance by removing the now superfluous Objects.requireNonNull calls from within the parse loop. Also fixed an issue where we were accidentally producing the wrong message in case of mismatching number of constants or arguments. Bug: https://bugs.openjdk.java.net/browse/JDK-8246632 Webrev: http://cr.openjdk.java.net/~redestad/8246632/open.01/ Testing: tier1 Thanks! /Claes From Roger.Riggs at oracle.com Mon Jun 8 15:36:00 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Mon, 8 Jun 2020 11:36:00 -0400 Subject: 8246632: StringConcatFactory::makeConcatWithConstants no longer throws NullPointerException when an unexpected constant is null In-Reply-To: <9f261708-df7a-03bf-b955-21a11131ce90@oracle.com> References: <9f261708-df7a-03bf-b955-21a11131ce90@oracle.com> Message-ID: <83ac1307-387b-3754-60fe-4eb5f96382a0@oracle.com> Hi Claes, The new method name "argumentMistmatch" should be "argumentMismatch".? ("Mis" vs "Mist"). The rest looks fine. Roger On 6/8/20 10:51 AM, Claes Redestad wrote: > Hi, > > JDK-8246152 triggered a failure in a JCK test that expects a NPE > rather than a StringConcatException. We can keep the old behavior, > while retaining most of performance by removing the now superfluous > Objects.requireNonNull calls from within the parse loop. > > Also fixed an issue where we were accidentally producing the wrong > message in case of mismatching number of constants or arguments. > > Bug:??? https://bugs.openjdk.java.net/browse/JDK-8246632 > Webrev: http://cr.openjdk.java.net/~redestad/8246632/open.01/ > > Testing: tier1 > > Thanks! > > /Claes From hs at tagtraum.com Sat Jun 6 14:13:29 2020 From: hs at tagtraum.com (Hendrik Schreiber) Date: Sat, 6 Jun 2020 16:13:29 +0200 Subject: [OpenJDK 2D-Dev] RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: <84EEE234-A21F-400B-917C-8DF55236E38E@oracle.com> References: <84EEE234-A21F-400B-917C-8DF55236E38E@oracle.com> Message-ID: Jim, if there isn?t a dedicated bug report for this (meaning: lack of optimization for macOS), please do create one so that it at least is documented somewhere. Thank you, -hendrik > On Jun 5, 2020, at 13:59, Jim Laskey wrote: > > I know there was a discussion about this elsewhere but I would like to take the opportunity to correct this now > > make//autoconf/flags-cflags.m4:241 > > elif test "x$TOOLCHAIN_TYPE" = xclang; then > if test "x$OPENJDK_TARGET_OS" = xmacosx; then > # On MacOSX we optimize for size, something > # we should do for all platforms? > C_O_FLAG_HIGHEST_JVM="-Os" > C_O_FLAG_HIGHEST="-Os" > C_O_FLAG_HI="-Os" > C_O_FLAG_NORM="-Os" > C_O_FLAG_DEBUG_JVM="" > else > C_O_FLAG_HIGHEST_JVM="-O3" > C_O_FLAG_HIGHEST="-O3" > C_O_FLAG_HI="-O3" > C_O_FLAG_NORM="-O2" > C_O_FLAG_DEBUG_JVM="-O0" > fi > C_O_FLAG_SIZE="-Os" > C_O_FLAG_DEBUG="-O0" > C_O_FLAG_NONE="-O0" > elif test "x$TOOLCHAIN_TYPE" = xxlc; then > > should be changed to > > elif test "x$TOOLCHAIN_TYPE" = xclang; then > C_O_FLAG_HIGHEST_JVM="-O3" > C_O_FLAG_HIGHEST="-O3" > C_O_FLAG_HI="-O3" > C_O_FLAG_NORM="-O2" > C_O_FLAG_DEBUG_JVM="-O0" > C_O_FLAG_SIZE="-Os" > C_O_FLAG_DEBUG="-O0" > C_O_FLAG_NONE="-O0" > elif test "x$TOOLCHAIN_TYPE" = xxlc; then > > MacOSX has been paying a historic and significant performance penalty for no valid reason. > > Otherwise +1. > > Cheers, > > -- Jim > > > >> On Jun 5, 2020, at 4:52 AM, Kim Barrett > wrote: >> >> [Changes are only to the build system, but since the changes have jdk-wide >> effect I?ve cc?d what I think are the relevant dev lists.] >> >> This change is part of JEP 347; the intent is to target JDK 16. >> >> Please review this change to the building of C++ code in the JDK to >> enable the use of C++14 language features. This change does not make >> any code changes to use new features provided by C++11 or C++14. >> >> This requires trimming the supported compiler versions, moving the >> minimum supported versions forward (significantly, in some cases). >> The new minimums are based on compiler documentation rather than >> testing. It may be that more recent versions are actually required. >> >> This change needs to be tested on platforms not supported by Oracle. >> The JEP test plan includes additional Oracle testing beyond what I?ve done. >> >> CR: >> https://bugs.openjdk.java.net/browse/JDK-8246032 >> >> Webrev: >> https://cr.openjdk.java.net/~kbarrett/8246032/open.02/ >> >> Testing: >> mach5 tier1-5 on Oracle supported platforms. >> >> Performance testing showed no significant changes in either direction. >> >> Build-only (no tests) for linux-arm32, linux-s390x, linux-ppc64le >> > From claes.redestad at oracle.com Mon Jun 8 16:42:09 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 8 Jun 2020 18:42:09 +0200 Subject: 8246632: StringConcatFactory::makeConcatWithConstants no longer throws NullPointerException when an unexpected constant is null In-Reply-To: <83ac1307-387b-3754-60fe-4eb5f96382a0@oracle.com> References: <9f261708-df7a-03bf-b955-21a11131ce90@oracle.com> <83ac1307-387b-3754-60fe-4eb5f96382a0@oracle.com> Message-ID: <2f23abda-9685-8d25-53fc-8a1f49e232ca@oracle.com> Hi Roger, On 2020-06-08 17:36, Roger Riggs wrote: > Hi Claes, > > The new method name "argumentMistmatch" should be "argumentMismatch". > ("Mis" vs "Mist"). nice catch! Will fix that and run a sanity test before push. > > The rest looks fine. Thanks! /Claes > > Roger > > > On 6/8/20 10:51 AM, Claes Redestad wrote: >> Hi, >> >> JDK-8246152 triggered a failure in a JCK test that expects a NPE >> rather than a StringConcatException. We can keep the old behavior, >> while retaining most of performance by removing the now superfluous >> Objects.requireNonNull calls from within the parse loop. >> >> Also fixed an issue where we were accidentally producing the wrong >> message in case of mismatching number of constants or arguments. >> >> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8246632 >> Webrev: http://cr.openjdk.java.net/~redestad/8246632/open.01/ >> >> Testing: tier1 >> >> Thanks! >> >> /Claes > From markus at jevring.net Sun Jun 7 13:07:13 2020 From: markus at jevring.net (Markus Jevring) Date: Sun, 7 Jun 2020 15:07:13 +0200 Subject: JPackage with a modular application fails to load record class files even with --enable-preview Message-ID: <5edce682.1c69fb81.9b9c2.832e@mx.google.com> I have this app, which uses Java 14 with ?enable-preview and records. If I turn it in to a normal non-modular jar, it works with both java -jar ?enable-preview myapp.jar, and with a native image built with jpackage. However, when i turn it into a modular jar, I can still launch it with java ?enable-preview ?module-path ... ?module ..., but when I try to create a native image using jpackage usign the same modular flags, launching it fails. I get a dialog box saying ?Failed to launch JVM?, and if I enable ?win-console when I build it and relaunch it, it tells me this: Exception in thread "main" java.lang.ClassFormatError: Invalid constant pool index 31 for name in Record attribute in class file net/jevring/frequencies/v2/input/KeyTimings$Note at java.base/java.lang.ClassLoader.defineClass2(Native Method) at java.base/java.lang.ClassLoader.defineClass(Unknown Source) at java.base/java.security.SecureClassLoader.defineClass(Unknown Source) at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(Unknown Source) at java.base/jdk.internal.loader.BuiltinClassLoader.findClassInModuleOrNull(Unknown Source) at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(Unknown Source) at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source) at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown Source) at java.base/java.lang.ClassLoader.loadClass(Unknown Source) at frequencies/net.jevring.frequencies.v2.input.KeyTimings.(Unknown Source) at frequencies/net.jevring.frequencies.v2.Thief.(Unknown Source) at frequencies/net.jevring.frequencies.v2.Main.main(Unknown Source) Note, above, is a record. I tried moving it out to a separate class, and I still got the same error. When I converted Note to a normal class, it started complaining about the next record it cound. I know that it?s picking up the ?enable-preview java option, because when I omit that, it complains about the class file version being wrong. Since I can run it just fine using ?java -jar?, and it works fine under all other circumstances, I must conclude that there?s something wrong with the intersection of modular jars and jpackage. I?ve googled around plenty, and I haven?t found anything that might help me. I?m not sure if this is a call for help or a bug report, though I?d like to consider it the latter =) I put what I had on a branch: https://bitbucket.org/jevring/frequencies/branch/java-modules I realize it?s not a small self-contained example that replicates the issue, and for that I?m sorry. If someone is interested about this, I could try to make one. From mandy.chung at oracle.com Mon Jun 8 16:47:31 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 8 Jun 2020 09:47:31 -0700 Subject: 8246632: StringConcatFactory::makeConcatWithConstants no longer throws NullPointerException when an unexpected constant is null In-Reply-To: <9f261708-df7a-03bf-b955-21a11131ce90@oracle.com> References: <9f261708-df7a-03bf-b955-21a11131ce90@oracle.com> Message-ID: <350815fc-262d-dcd4-ce0f-3659a9afe0a9@oracle.com> Looks good (minus the typo Roger caught). Mandy On 6/8/20 7:51 AM, Claes Redestad wrote: > Hi, > > JDK-8246152 triggered a failure in a JCK test that expects a NPE > rather than a StringConcatException. We can keep the old behavior, > while retaining most of performance by removing the now superfluous > Objects.requireNonNull calls from within the parse loop. > > Also fixed an issue where we were accidentally producing the wrong > message in case of mismatching number of constants or arguments. > > Bug:??? https://bugs.openjdk.java.net/browse/JDK-8246632 > Webrev: http://cr.openjdk.java.net/~redestad/8246632/open.01/ > > Testing: tier1 > > Thanks! > > /Claes From claes.redestad at oracle.com Mon Jun 8 17:04:43 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 8 Jun 2020 19:04:43 +0200 Subject: 8246632: StringConcatFactory::makeConcatWithConstants no longer throws NullPointerException when an unexpected constant is null In-Reply-To: <350815fc-262d-dcd4-ce0f-3659a9afe0a9@oracle.com> References: <9f261708-df7a-03bf-b955-21a11131ce90@oracle.com> <350815fc-262d-dcd4-ce0f-3659a9afe0a9@oracle.com> Message-ID: <4ca12ecd-7191-3e5a-c5d1-290f1367c0a6@oracle.com> Thanks, Mandy! /Claes On 2020-06-08 18:47, Mandy Chung wrote: > Looks good (minus the typo Roger caught). > > Mandy > > On 6/8/20 7:51 AM, Claes Redestad wrote: >> Hi, >> >> JDK-8246152 triggered a failure in a JCK test that expects a NPE >> rather than a StringConcatException. We can keep the old behavior, >> while retaining most of performance by removing the now superfluous >> Objects.requireNonNull calls from within the parse loop. >> >> Also fixed an issue where we were accidentally producing the wrong >> message in case of mismatching number of constants or arguments. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8246632 >> Webrev: http://cr.openjdk.java.net/~redestad/8246632/open.01/ >> >> Testing: tier1 >> >> Thanks! >> >> /Claes > From andy.herrick at oracle.com Mon Jun 8 17:21:50 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Mon, 8 Jun 2020 13:21:50 -0400 Subject: RFR: JDK-8231283: Evaluate option to install Linux packages in /usr hierarchy In-Reply-To: References: <8a3f9c48-9561-3a4a-7e07-53a9ebbc1539@oracle.com> Message-ID: <45ec8ac0-72ec-e2aa-7b51-078238cc9b84@oracle.com> I don't have a unix system to try this out on, but it looks like there is no net change when install-dir is neither "/usr" or "/user/local", so I'm good with it. /Andy On 6/8/2020 10:26 AM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > Add support to jpackage to create Linux packages installing app images > in '/usr' tree with splicing of the app image. > > For --install-dir=/usr jpackag option the resulting Debian package > will have the following layout: > - Launchers will be installed in `/usr/bin` directory > - Java runtime will be installed in `/usr/lib/$PACKAGE_NAME/runtime` > directory > - `app` directory of the app image will be installed in > `/usr/lib/$PACKAGE_NAME/app` directory > - copyright file will be installed in > `/usr/share/doc/$PACKAGE_NAME/copyright` file > > RPM package will have the following layout: > - Launchers will be installed in `/usr/bin` directory > - Java runtime will be installed in `/usr/lib/$PACKAGE_NAME/runtime` > directory > - `app` directory of the app image will be installed in > `/usr/lib/$PACKAGE_NAME/app` directory > > For --install-dir=/usr/local jpackag option the resulting Debian > package will have the following layout: > - Launchers will be installed in `/usr/local/bin` directory > - Java runtime will be installed in > `/usr/local/lib/$PACKAGE_NAME/runtime` directory > - `app` directory of the app image will be installed in > `/usr/local/lib/$PACKAGE_NAME/app` directory > - copyright file will be installed in > `/usr/share/doc/$PACKAGE_NAME/copyright` file > > RPM package will have the following layout: > - Launchers will be installed in `/usr/local/bin` directory > - Java runtime will be installed in > `/usr/local/lib/$PACKAGE_NAME/runtime` directory > - `app` directory of the app image will be installed in > `/usr/local/lib/$PACKAGE_NAME/app` directory > > For any other prefixes starting with `/usr` no app image splicing will > happen and package layout will be the same as for installing at the > default `/opt` prefix. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8231283 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8231283/webrev.00 > From james.laskey at oracle.com Mon Jun 8 17:22:45 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Mon, 8 Jun 2020 14:22:45 -0300 Subject: RFR: JDK-8230744 Several classes throw OutOfMemoryError without message In-Reply-To: <6D4D4663-BB0E-466D-BBA3-9D39556B34BC@oracle.com> References: <045D0E84-60E4-491B-9A31-7B05043FFE3B@oracle.com> <78cf935b-d7f3-342a-6ed1-a4a86becb727@oracle.com> <6D4D4663-BB0E-466D-BBA3-9D39556B34BC@oracle.com> Message-ID: Revised to use a consistent error message. Modified AbstractStringBuilder and PriorityBlockingQueue to use ArraysSupport.newLength(). Didn't modify ByteArrayChannel and UnsyncByteArrayOutputStream since those changes would require changes to module exporting. webrev: http://cr.openjdk.java.net/~jlaskey/8230744/webrev-03/index.html > On Jun 3, 2020, at 11:24 AM, Jim Laskey wrote: > > It's not the goal or role of this bug to fix the wrongs of the past, merely add error messages to the exceptions. I raised the discussion as an issue. Clearly there is a correct path to follow. If you think more effort is required then file a bug. :-) > > Cheers, > > -- Jim > > > > >> On Jun 2, 2020, at 7:13 PM, Stuart Marks wrote: >> >> >> >> On 6/2/20 6:52 AM, Jim Laskey wrote: >>> Revised to reflect requested changes. >>> http://cr.openjdk.java.net/~jlaskey/8230744/webrev-01/index.html >> >> On this, if all you're doing is changing exception messages, then I don't care very much, modulo concerns about wording from others. If you start to get into changing the growth logic (like the Math.addExact stuff) then please see my message on the related thread, "Sometimes constraints are questionable." >> >> Thanks, >> >> s'marks > From sean.mullan at oracle.com Mon Jun 8 19:03:15 2020 From: sean.mullan at oracle.com (Sean Mullan) Date: Mon, 8 Jun 2020 15:03:15 -0400 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> <9013C8D0-33D9-4D09-8953-F8618020A7F7@oracle.com> <8A2C4DEF-FC61-4574-B49D-1133F5349BCE@azul.com> Message-ID: <15fa46f2-0352-a111-a9c7-0d5fc31d9159@oracle.com> (resending to all lists on the review) I'm just catching up a bit on this review. Sorry if this has mentioned before, but are you planning to write a CSR and release note? I think this is needed for the com.sun.jndi.ldap.tls.cbtype property. I'm also wondering if this property should be documented in the javadocs, and why it is not a standard property (i.e. "java.naming.ldap.tls.cbtype"). I was also wondering what relation this has to the "G2" standard SASL mechanisms defined in RFC 5801 [1], and whether that is something we should be using to negotiate this channel binding, and if not, why not. Or if this is something that is implementation-specific and will only work with Microsoft LDAP technology, in which case, we might want to make that more explicit, perhaps by including "microsoft" or something like that in the property name. Thanks, Sean [1] https://tools.ietf.org/html/rfc5801 On 6/8/20 9:07 AM, Aleks Efimov wrote: > Hi Alexey, > > I've looked through LdapCtx/LdapClient/SaslBind changes: > > Do we need to check if CHANNEL_BINDING is set explicitly for all > connection types? Maybe we can move the check inside 'if (conn.sock > instanceof SSLSocket) {' block. > > Also, instead of setting CHANNEL_BINDING in context environment and then > removing it in finally block, it would be better to clone the > environment, put calculated CHANNEL_BINDING into it, and pass the cloned > one to Sasl.createSaslClient. > > Another suggestion about the code that verifies if both properties are > set before connection is started: > As you've already mentioned the new code in LdapCtx is only needed for > checking if timeout is set. Could we try to remove LdapCtx::cbType field > and all related methods from LdapCtx (this class is already > over-complicated and hard to read) and replace it with some static > method in LdapSasl? It will help to localize all changes to LdapSasl > except for one line in LdapCtx. > > I mean something like this: > Replace > + > +??????????? // verify LDAP channel binding property > +??????????? if (cbType != null && connectTimeout == -1) > +??????????????????? throw new > NamingException(TlsChannelBinding.CHANNEL_BINDING_TYPE + > +??????????????????????????? " property requires " + > +??????????????????????????? CONNECT_TIMEOUT + > +??????????????????????????? " property is set."); > With > + > LdapSasl.checkCbParameters((String)envprops.get(TlsChannelBinding.CHANNEL_BINDING_TYPE), > connectTimeout); > > And add something like that to LdapSasl (or maybe pass the full env here): > + public static void checkCbParameters(String cbTypePropertyValue, int > connectTimeout) throws NamingException { > +???? TlsChannelBindingType cbType = > TlsChannelBinding.parseType(cbTypePropertyValue); > +???? // verify LDAP channel binding property > +???? if (cbType != null && connectTimeout == -1) { > +???????? throw new > NamingException(TlsChannelBinding.CHANNEL_BINDING_TYPE + > +???????????????? " property requires com.sun.jndi.ldap.connect.timeout" + > +???????????????? " property is set."); > +???? } > + } > > Other LdapCtx/LdapClient/SaslBind? changes look fine to me. > > With Kind Regards, > Aleksei > > On 06/06/2020 20:45, Alexey Bakhtin wrote: >> Hello Max, Daniel, >> >> Thank you for review. >> >> Please review new version of the patch : >> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v5/ >> >> In this version: >> - TlsChannelBinding class is moved into the com.sun.jndi.ldap.sasl >> package >> - SSL Ceritificate related code is moved from LdapClient? into the >> LdapSasl.saslBind method >> - verification and removal of internal property is also moved to >> LdapSasl.saslBind method >> - verification of connectTimeout property is moved to LdapCtx.connect. >> I?ve found that connectionTimeout could be assigned later then cbType >> >> The test for this issue is not ready yet. I did not find any suitable >> test case that can be reused. >> >> Thank you >> Alexey >> >>> On 6 Jun 2020, at 09:44, Weijun Wang wrote: >>> >>> >>> >>>> On Jun 6, 2020, at 2:41 PM, Weijun Wang wrote: >>>> >>>> >>>> >>>>> On Jun 5, 2020, at 11:03 PM, Alexey Bakhtin wrote: >>>>> >>>>> Hello Max, >>>>> >>>>> Thank you a lot for review. >>>>> >>>>> Could you check the new version of the patch : >>>>> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v4/ >>>>> >>>>> I?ve made the following changes: >>>>> - TlsChannelBinding class is moved to java.naming module. >>>>> java.security.sasl module is not affected any more >>>>> - I pass tlsCB.getData() directly to the SASL mechanism as you >>>>> suggested >>>>> - I?ve made some guards to prevent application from using >>>>> "com.sun.security.sasl.tlschannelbinding? property directly: >>>>> ????- LdapClient verifies if internal property is not set >>>> 245???????????????????? // Prepare TLS Channel Binding data >>>> 246???????????????????? if (conn.sock instanceof SSLSocket) { >>>> 247???????????????????????? // Internal property cannot be set >>>> explicitly >>>> 248???????????????????????? if >>>> (env.get(TlsChannelBinding.CHANNEL_BINDING) != null) { >>>> 249???????????????????????????? throw new >>>> NamingException(TlsChannelBinding.CHANNEL_BINDING + >>>> 250???????????????????????????????????? " property cannot be set >>>> explicitly"); >>>> 251???????????????????????? } >>>> >>>> If not TLS, this property value be kept there and visible inside the >>>> SASL mech. >>>> >>>>> ????- GssKrb5Client uses props.remove() to read and remove internal >>>>> property >>> Maybe you can remove the value in LdapClient.java, in case the mech >>> used is not GSSAPI. You can remove it in a finally block after line 303. >>> >>> --Max >>> >>>> Traditionally, we use "com.sun..." name which is a JDK supported >>>> name (although not at Java SE level), you might want to use a name >>>> which is even more internal. >>>> >>>> >>>> Thanks, >>>> Max >>>> >>>> p.s. I see that NTLM also supports ChannelBinding. I'll see if I can >>>> improve the NTLM SASL mech to support it. > From raffaello.giulietti at gmail.com Mon Jun 8 19:45:33 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Mon, 8 Jun 2020 21:45:33 +0200 Subject: contributing to JDK-8222187: java.util.Base64.Decoder stream adds unexpected null bytes at the end Message-ID: Hello, as of a very recent changeset [1], this bug [2] is still present. While the bug already has an assignee, there seems to be no recent activity, so I'd like to contribute fixing it. Greetings Raffaello ---- [1] http://hg.openjdk.java.net/jdk/jdk/rev/ac47db0c7ccb [2] https://bugs.openjdk.java.net/browse/JDK-8222187 From vicente.romero at oracle.com Mon Jun 8 19:58:38 2020 From: vicente.romero at oracle.com (Vicente Romero) Date: Mon, 8 Jun 2020 15:58:38 -0400 Subject: RFR: JDK-8245958: j.l.Record need to mention that canonical constructor may not be public Message-ID: <88730e95-255a-68d4-6380-5f33a1a0dd80@oracle.com> Please review this API update. The API of java.lang.Record needs to be updated after the changes introduced by JEP 384: Records (Second Preview) [1], in particular this JEP changed the constraint that the canonical constructor had to be public. The associated bug is [2], the changeset is at [3] and the related CSR is at [4], Thanks, Vicente [1] https://openjdk.java.net/jeps/384 [2] https://bugs.openjdk.java.net/browse/JDK-8245958 [3] http://cr.openjdk.java.net/~vromero/8245958/webrev.00/ [4] https://bugs.openjdk.java.net/browse/JDK-8246629 From philip.race at oracle.com Mon Jun 8 20:07:17 2020 From: philip.race at oracle.com (Philip Race) Date: Mon, 08 Jun 2020 13:07:17 -0700 Subject: [OpenJDK 2D-Dev] RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: References: Message-ID: <5EDE9A75.6020504@oracle.com> Hi Kim, Can we amend the JEP itself to be not solely about hotspot. Since it affects other areas I think that 1) They may need to be compiled with C++14 enabled whether they use new features or not 2) They may want - or need - to adopt C++11 or C++14 features too. You already know that soon we will upgrade the harfbuzz 3rd party library used by 2D and it will bring along the need to use C++11 features and I had no plan to propose a JEP for that. So I don't want to be asked why we didn't do one when hotspot did ! So this really ought to be a JDK JEP whilst of course explaining that hotspot is expected to be the primary adopter. BTW the JEP (https://bugs.openjdk.java.net/browse/JDK-8208089) still says > For Solaris: Add the |-std=c++14| compiler option. ..... So I am sure there is still some room to update the JEP :-) -phil. On 6/5/20, 12:52 AM, Kim Barrett wrote: > [Changes are only to the build system, but since the changes have jdk-wide > effect I?ve cc?d what I think are the relevant dev lists.] > > This change is part of JEP 347; the intent is to target JDK 16. > > Please review this change to the building of C++ code in the JDK to > enable the use of C++14 language features. This change does not make > any code changes to use new features provided by C++11 or C++14. > > This requires trimming the supported compiler versions, moving the > minimum supported versions forward (significantly, in some cases). > The new minimums are based on compiler documentation rather than > testing. It may be that more recent versions are actually required. > > This change needs to be tested on platforms not supported by Oracle. > The JEP test plan includes additional Oracle testing beyond what I?ve done. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8246032 > > Webrev: > https://cr.openjdk.java.net/~kbarrett/8246032/open.02/ > > Testing: > mach5 tier1-5 on Oracle supported platforms. > > Performance testing showed no significant changes in either direction. > > Build-only (no tests) for linux-arm32, linux-s390x, linux-ppc64le > From lance.andersen at oracle.com Mon Jun 8 20:34:48 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Mon, 8 Jun 2020 16:34:48 -0400 Subject: contributing to JDK-8222187: java.util.Base64.Decoder stream adds unexpected null bytes at the end In-Reply-To: References: Message-ID: Hi Raffaello, If you are interested in providing a fix, you are more than welcome to do so. Please include a test which validates your fix. Best lance > On Jun 8, 2020, at 3:45 PM, Raffaello Giulietti wrote: > > Raffaello Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From alexey at azul.com Mon Jun 8 21:01:13 2020 From: alexey at azul.com (Alexey Bakhtin) Date: Mon, 8 Jun 2020 21:01:13 +0000 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <013290B8-67BE-4CD5-804C-FC882CD23CFE@oracle.com> References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> <9013C8D0-33D9-4D09-8953-F8618020A7F7@oracle.com> <8A2C4DEF-FC61-4574-B49D-1133F5349BCE@azul.com> <013290B8-67BE-4CD5-804C-FC882CD23CFE@oracle.com> Message-ID: Hello Max, Daniel, Thank you for review and suggestions. Could you please check the updated webrev: http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v6/ This version contains the following changes: 1. Updated license for new files 2. Comments about default address type usage in the InitialToken.java and GSSLibStub.c 3. internal property is renamed to ?jdk.internal.sasl.tlschannelbinding" 4. I did not create test but provided detailed reproducer in the bug description 5. Property verification is moved from LdapCTX into LdapSasl as suggested by Aleks 6. Use clone of enviroment properties as suggested by Aleks I did not move internal property check under 'if (conn.sock instanceof SSLSocket) {' block. It was already discussed in [1] : "If not TLS, this property value be kept there and visible inside the SASL mech." [1] - https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-June/067042.html Thank you Alexey > On 7 Jun 2020, at 04:47, Weijun Wang wrote: > > Some comments: > > 1. I just noticed your 2 new files are using the plain GPU license, it should be GPL + Classpath. Read another source file for an example. > > 2. Please add some comments in InitialToken.java on what happens to the default type. > > 3. I still think "com.sun.security.sasl.tlschannelbinding" sounds like sun.com will support it. > > 4. If an automatic test is not feasible, please provide some instructions so our SQE can see if we can setup some internal tests. Then we can add noreg-hard with some justification to the test and go on. > > I cannot comment on LdapCtx.java, but the others look fine to me. > > Thanks, > Max > >> On Jun 7, 2020, at 3:45 AM, Alexey Bakhtin wrote: >> >> Hello Max, Daniel, >> >> Thank you for review. >> >> Please review new version of the patch : >> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v5/ >> >> In this version: >> - TlsChannelBinding class is moved into the com.sun.jndi.ldap.sasl package >> - SSL Ceritificate related code is moved from LdapClient into the LdapSasl.saslBind method >> - verification and removal of internal property is also moved to LdapSasl.saslBind method >> - verification of connectTimeout property is moved to LdapCtx.connect. I?ve found that connectionTimeout could be assigned later then cbType >> >> The test for this issue is not ready yet. I did not find any suitable test case that can be reused. >> >> Thank you >> Alexey >> >>> On 6 Jun 2020, at 09:44, Weijun Wang wrote: >>> >>> >>> >>>> On Jun 6, 2020, at 2:41 PM, Weijun Wang wrote: >>>> >>>> >>>> >>>>> On Jun 5, 2020, at 11:03 PM, Alexey Bakhtin wrote: >>>>> >>>>> Hello Max, >>>>> >>>>> Thank you a lot for review. >>>>> >>>>> Could you check the new version of the patch : >>>>> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v4/ >>>>> >>>>> I?ve made the following changes: >>>>> - TlsChannelBinding class is moved to java.naming module. >>>>> java.security.sasl module is not affected any more >>>>> - I pass tlsCB.getData() directly to the SASL mechanism as you suggested >>>>> - I?ve made some guards to prevent application from using "com.sun.security.sasl.tlschannelbinding? property directly: >>>>> - LdapClient verifies if internal property is not set >>>> >>>> 245 // Prepare TLS Channel Binding data >>>> 246 if (conn.sock instanceof SSLSocket) { >>>> 247 // Internal property cannot be set explicitly >>>> 248 if (env.get(TlsChannelBinding.CHANNEL_BINDING) != null) { >>>> 249 throw new NamingException(TlsChannelBinding.CHANNEL_BINDING + >>>> 250 " property cannot be set explicitly"); >>>> 251 } >>>> >>>> If not TLS, this property value be kept there and visible inside the SASL mech. >>>> >>>>> - GssKrb5Client uses props.remove() to read and remove internal property >>> >>> Maybe you can remove the value in LdapClient.java, in case the mech used is not GSSAPI. You can remove it in a finally block after line 303. >>> >>> --Max >>> >>>> >>>> Traditionally, we use "com.sun..." name which is a JDK supported name (although not at Java SE level), you might want to use a name which is even more internal. >>>> >>>> >>>> Thanks, >>>> Max >>>> >>>> p.s. I see that NTLM also supports ChannelBinding. I'll see if I can improve the NTLM SASL mech to support it. >> From naoto.sato at oracle.com Mon Jun 8 21:06:43 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Mon, 8 Jun 2020 14:06:43 -0700 Subject: [15] RFR: 8246662: Test java/time/test/java/time/format/TestUnicodeExtension.java failed on japanese locale. Message-ID: <89a24fb8-67e1-9c80-51f2-a6f41ec2c909@oracle.com> Hello, Please review the fix to the following issue: https://bugs.openjdk.java.net/browse/JDK-8246662 The proposed changeset is located at: https://cr.openjdk.java.net/~naoto/8246662/webrev.00/ This is a regression caused by the fix to JDK-8244245, where the locale related fields in the formatter are now overridden by localizedBy() method. It was only comparing the locale object for the fast path, but it was not sufficient. Naoto From alexey at azul.com Mon Jun 8 21:33:02 2020 From: alexey at azul.com (Alexey Bakhtin) Date: Mon, 8 Jun 2020 21:33:02 +0000 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <15fa46f2-0352-a111-a9c7-0d5fc31d9159@oracle.com> References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> <9013C8D0-33D9-4D09-8953-F8618020A7F7@oracle.com> <8A2C4DEF-FC61-4574-B49D-1133F5349BCE@azul.com> <15fa46f2-0352-a111-a9c7-0d5fc31d9159@oracle.com> Message-ID: Hello Sean, Yes, I think we'll need CSR and release notes as soon as this patch adds new property. I do not know exact process for it, so I will be grateful if you could explain me exact steps. This patch was developed to address compatibility issue with new LDAP authentication over SSL/TLS announced by Microsoft [1]. It is not related to RFC 5801. In my opinion ?com.sun.jndi.ldap.tls.cbtype? name looks more suitable for this property and should allow backport it to early JDK versions. [1] - https://support.microsoft.com/en-au/help/4034879/how-to-add-the-ldapenforcechannelbinding-registry-entry Regards Alexey > On 8 Jun 2020, at 22:03, Sean Mullan wrote: > > (resending to all lists on the review) > > I'm just catching up a bit on this review. > > Sorry if this has mentioned before, but are you planning to write a CSR and release note? I think this is needed for the com.sun.jndi.ldap.tls.cbtype property. I'm also wondering if this property should be documented in the javadocs, and why it is not a standard property (i.e. "java.naming.ldap.tls.cbtype"). > > I was also wondering what relation this has to the "G2" standard SASL mechanisms defined in RFC 5801 [1], and whether that is something we should be using to negotiate this channel binding, and if not, why not. Or if this is something that is implementation-specific and will only work with Microsoft LDAP technology, in which case, we might want to make that more explicit, perhaps by including "microsoft" or something like that in the property name. > > Thanks, > Sean > > [1] https://tools.ietf.org/html/rfc5801 > > On 6/8/20 9:07 AM, Aleks Efimov wrote: >> Hi Alexey, >> I've looked through LdapCtx/LdapClient/SaslBind changes: >> Do we need to check if CHANNEL_BINDING is set explicitly for all connection types? Maybe we can move the check inside 'if (conn.sock instanceof SSLSocket) {' block. >> Also, instead of setting CHANNEL_BINDING in context environment and then removing it in finally block, it would be better to clone the environment, put calculated CHANNEL_BINDING into it, and pass the cloned one to Sasl.createSaslClient. >> Another suggestion about the code that verifies if both properties are set before connection is started: >> As you've already mentioned the new code in LdapCtx is only needed for checking if timeout is set. Could we try to remove LdapCtx::cbType field and all related methods from LdapCtx (this class is already over-complicated and hard to read) and replace it with some static method in LdapSasl? It will help to localize all changes to LdapSasl except for one line in LdapCtx. >> I mean something like this: >> Replace >> + >> + // verify LDAP channel binding property >> + if (cbType != null && connectTimeout == -1) >> + throw new NamingException(TlsChannelBinding.CHANNEL_BINDING_TYPE + >> + " property requires " + >> + CONNECT_TIMEOUT + >> + " property is set."); >> With >> + LdapSasl.checkCbParameters((String)envprops.get(TlsChannelBinding.CHANNEL_BINDING_TYPE), connectTimeout); >> And add something like that to LdapSasl (or maybe pass the full env here): >> + public static void checkCbParameters(String cbTypePropertyValue, int connectTimeout) throws NamingException { >> + TlsChannelBindingType cbType = TlsChannelBinding.parseType(cbTypePropertyValue); >> + // verify LDAP channel binding property >> + if (cbType != null && connectTimeout == -1) { >> + throw new NamingException(TlsChannelBinding.CHANNEL_BINDING_TYPE + >> + " property requires com.sun.jndi.ldap.connect.timeout" + >> + " property is set."); >> + } >> + } >> Other LdapCtx/LdapClient/SaslBind changes look fine to me. >> With Kind Regards, >> Aleksei >> On 06/06/2020 20:45, Alexey Bakhtin wrote: >>> Hello Max, Daniel, >>> >>> Thank you for review. >>> >>> Please review new version of the patch : >>> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v5/ >>> >>> In this version: >>> - TlsChannelBinding class is moved into the com.sun.jndi.ldap.sasl package >>> - SSL Ceritificate related code is moved from LdapClient into the LdapSasl.saslBind method >>> - verification and removal of internal property is also moved to LdapSasl.saslBind method >>> - verification of connectTimeout property is moved to LdapCtx.connect. I?ve found that connectionTimeout could be assigned later then cbType >>> >>> The test for this issue is not ready yet. I did not find any suitable test case that can be reused. >>> >>> Thank you >>> Alexey >>> >>>> On 6 Jun 2020, at 09:44, Weijun Wang wrote: >>>> >>>> >>>> >>>>> On Jun 6, 2020, at 2:41 PM, Weijun Wang wrote: >>>>> >>>>> >>>>> >>>>>> On Jun 5, 2020, at 11:03 PM, Alexey Bakhtin wrote: >>>>>> >>>>>> Hello Max, >>>>>> >>>>>> Thank you a lot for review. >>>>>> >>>>>> Could you check the new version of the patch : >>>>>> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v4/ >>>>>> >>>>>> I?ve made the following changes: >>>>>> - TlsChannelBinding class is moved to java.naming module. >>>>>> java.security.sasl module is not affected any more >>>>>> - I pass tlsCB.getData() directly to the SASL mechanism as you suggested >>>>>> - I?ve made some guards to prevent application from using "com.sun.security.sasl.tlschannelbinding? property directly: >>>>>> - LdapClient verifies if internal property is not set >>>>> 245 // Prepare TLS Channel Binding data >>>>> 246 if (conn.sock instanceof SSLSocket) { >>>>> 247 // Internal property cannot be set explicitly >>>>> 248 if (env.get(TlsChannelBinding.CHANNEL_BINDING) != null) { >>>>> 249 throw new NamingException(TlsChannelBinding.CHANNEL_BINDING + >>>>> 250 " property cannot be set explicitly"); >>>>> 251 } >>>>> >>>>> If not TLS, this property value be kept there and visible inside the SASL mech. >>>>> >>>>>> - GssKrb5Client uses props.remove() to read and remove internal property >>>> Maybe you can remove the value in LdapClient.java, in case the mech used is not GSSAPI. You can remove it in a finally block after line 303. >>>> >>>> --Max >>>> >>>>> Traditionally, we use "com.sun..." name which is a JDK supported name (although not at Java SE level), you might want to use a name which is even more internal. >>>>> >>>>> >>>>> Thanks, >>>>> Max >>>>> >>>>> p.s. I see that NTLM also supports ChannelBinding. I'll see if I can improve the NTLM SASL mech to support it. From alexander.matveev at oracle.com Mon Jun 8 21:48:24 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Mon, 8 Jun 2020 14:48:24 -0700 Subject: RFR: JDK-8246706: [macos] Allow SigningPackageTest to be built with real certificates In-Reply-To: <65facbe0-a665-7d73-9928-058e4dfe0c7f@oracle.com> References: <65facbe0-a665-7d73-9928-058e4dfe0c7f@oracle.com> Message-ID: <72f30a42-87aa-cd11-8a85-de9d174a04c3@oracle.com> Hi Andy, Looks good. For changing "jpackagerTest.keychain" to ""jpackageTest.keychain" we can file followup bug for this, since SQE depend on this value to be "jpackagerTest.keychain". Thanks, Alexander On 6/5/20 2:27 PM, Alexey Semenyuk wrote: > In SigningBase.java: > --- > KEYCHAIN = (value == null) ? "jpackagerTest.keychain" : value; > --- > > Is "jpackagerTest.keychain" some legacy string? Should not it be > "jpackageTest.keychain"? > > - Alexey > > On 6/5/2020 4:23 PM, Andy Herrick wrote: >> Please review fix to issue [1] at [2]. >> >> The fix enables running jpackage signing tests on macos with user >> supplied certificates. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8246706 >> >> [2] http://cr.openjdk.java.net/~herrick/8246706/webrev.01/index.html >> >> /Andy >> > From alexander.matveev at oracle.com Mon Jun 8 22:11:17 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Mon, 8 Jun 2020 15:11:17 -0700 Subject: RFR: JDK-8231283: Evaluate option to install Linux packages in /usr hierarchy In-Reply-To: References: <8a3f9c48-9561-3a4a-7e07-53a9ebbc1539@oracle.com> Message-ID: <4f18b543-5c4e-14bf-da29-32da61148709@oracle.com> Hi Alexey, Looks good. Thanks, Alexander On 6/8/20 7:26 AM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > Add support to jpackage to create Linux packages installing app images > in '/usr' tree with splicing of the app image. > > For --install-dir=/usr jpackag option the resulting Debian package > will have the following layout: > - Launchers will be installed in `/usr/bin` directory > - Java runtime will be installed in `/usr/lib/$PACKAGE_NAME/runtime` > directory > - `app` directory of the app image will be installed in > `/usr/lib/$PACKAGE_NAME/app` directory > - copyright file will be installed in > `/usr/share/doc/$PACKAGE_NAME/copyright` file > > RPM package will have the following layout: > - Launchers will be installed in `/usr/bin` directory > - Java runtime will be installed in `/usr/lib/$PACKAGE_NAME/runtime` > directory > - `app` directory of the app image will be installed in > `/usr/lib/$PACKAGE_NAME/app` directory > > For --install-dir=/usr/local jpackag option the resulting Debian > package will have the following layout: > - Launchers will be installed in `/usr/local/bin` directory > - Java runtime will be installed in > `/usr/local/lib/$PACKAGE_NAME/runtime` directory > - `app` directory of the app image will be installed in > `/usr/local/lib/$PACKAGE_NAME/app` directory > - copyright file will be installed in > `/usr/share/doc/$PACKAGE_NAME/copyright` file > > RPM package will have the following layout: > - Launchers will be installed in `/usr/local/bin` directory > - Java runtime will be installed in > `/usr/local/lib/$PACKAGE_NAME/runtime` directory > - `app` directory of the app image will be installed in > `/usr/local/lib/$PACKAGE_NAME/app` directory > > For any other prefixes starting with `/usr` no app image splicing will > happen and package layout will be the same as for installing at the > default `/opt` prefix. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8231283 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8231283/webrev.00 > From brent.christian at oracle.com Mon Jun 8 23:24:47 2020 From: brent.christian at oracle.com (Brent Christian) Date: Mon, 8 Jun 2020 16:24:47 -0700 Subject: [PATCH] 8245694 : java.util.Properties.entrySet() does not override java.lang.Object methods In-Reply-To: <7b01289c-d7c9-47ab-f9f6-e2cf3a00f2da@oracle.com> References: <9487a1c9-d0bc-3632-156d-5927e5801dc8@icemanx.nl> <20109712-d505-1b17-3e6b-ae388d554400@oracle.com> <767a4dae-55f0-ca11-dbdf-57c7a22205d3@oracle.com> <7b01289c-d7c9-47ab-f9f6-e2cf3a00f2da@oracle.com> Message-ID: <37bea22e-115f-5187-0f21-e04536691efc@oracle.com> Looks good to me as well. Thanks for making the updates to the test case. -Brent On 6/8/20 2:38 AM, Julia Boes wrote: > Hi Yu Li, > > The copyright year of Properties.java needs to be updated to 2020. > Otherwise looks good to me! > > Cheers, > > Julia > From mandy.chung at oracle.com Mon Jun 8 23:54:44 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 8 Jun 2020 16:54:44 -0700 Subject: RFR: JDK-8245958: j.l.Record need to mention that canonical constructor may not be public In-Reply-To: <88730e95-255a-68d4-6380-5f33a1a0dd80@oracle.com> References: <88730e95-255a-68d4-6380-5f33a1a0dd80@oracle.com> Message-ID: <7bc1d903-5310-187e-fa95-a194ea2d04f6@oracle.com> Looks fine. Mandy On 6/8/20 12:58 PM, Vicente Romero wrote: > Please review this API update. The API of java.lang.Record needs to be > updated after the changes introduced by JEP 384: Records (Second > Preview) [1], in particular this JEP changed the constraint that the > canonical constructor had to be public. The associated bug is [2], the > changeset is at [3] and the related CSR is at [4], > > Thanks, > Vicente > > [1] https://openjdk.java.net/jeps/384 > [2] https://bugs.openjdk.java.net/browse/JDK-8245958 > [3] http://cr.openjdk.java.net/~vromero/8245958/webrev.00/ > [4] https://bugs.openjdk.java.net/browse/JDK-8246629 From liyu.italy at gmail.com Tue Jun 9 02:42:14 2020 From: liyu.italy at gmail.com (Lisa Li) Date: Tue, 9 Jun 2020 10:42:14 +0800 Subject: [PATCH] 8245694 : java.util.Properties.entrySet() does not override java.lang.Object methods In-Reply-To: <37bea22e-115f-5187-0f21-e04536691efc@oracle.com> References: <9487a1c9-d0bc-3632-156d-5927e5801dc8@icemanx.nl> <20109712-d505-1b17-3e6b-ae388d554400@oracle.com> <767a4dae-55f0-ca11-dbdf-57c7a22205d3@oracle.com> <7b01289c-d7c9-47ab-f9f6-e2cf3a00f2da@oracle.com> <37bea22e-115f-5187-0f21-e04536691efc@oracle.com> Message-ID: Thanks for all your suggestions Brent! They helped a lot. Best regards, Yu L. On Tue, Jun 9, 2020 at 7:24 AM Brent Christian wrote: > Looks good to me as well. Thanks for making the updates to the test case. > > -Brent > > On 6/8/20 2:38 AM, Julia Boes wrote: > > Hi Yu Li, > > > > The copyright year of Properties.java needs to be updated to 2020. > > Otherwise looks good to me! > > > > Cheers, > > > > Julia > > > From alexander.matveev at oracle.com Tue Jun 9 05:12:44 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Mon, 8 Jun 2020 22:12:44 -0700 Subject: RFR: 8233215: jpackage doesn't allow enough flexibility for file type binding Message-ID: <502c6e3c-0f49-c895-02b7-bfad8942a9db@oracle.com> Please review the jpackage fix for bug [1] at [2]. Added support for additional (macOS specific) file association properties. These additional properties should be set via same property file which used for file association. Names and values for properties should match name and values as documented by Apple in https://developer.apple.com/documentation/bundleresources/information_property_list/cfbundledocumenttypes. jpackage will simple propagate these values to Info.plist as is. [1] https://bugs.openjdk.java.net/browse/JDK-8233215 [2] http://cr.openjdk.java.net/~almatvee/8233215/webrev.00/ Thanks, Alexander From huizhe.wang at oracle.com Tue Jun 9 05:54:04 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Mon, 8 Jun 2020 22:54:04 -0700 Subject: RFR [15/java.xml] 8237456: Transform filtered through SAX filter mishandles character entities Message-ID: <224e6b23-7999-4a2a-370c-35088a085d1d@oracle.com> Hi, Please review a fix to a regression that resulted in invalid output in a transform that contains entity references. The regression was caused by a JDK9 patch [1]. Adding back the conditional checks fixed the issue. JBS: https://bugs.openjdk.java.net/browse/JDK-8237456 webrevs: http://cr.openjdk.java.net/~joehw/jdk15/8237456/webrev/ [1] https://bugs.openjdk.java.net/browse/JDK-8087303 Thanks, Joe From alexey.semenyuk at oracle.com Tue Jun 9 05:57:00 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Tue, 9 Jun 2020 01:57:00 -0400 Subject: RFR: 8233215: jpackage doesn't allow enough flexibility for file type binding In-Reply-To: <502c6e3c-0f49-c895-02b7-bfad8942a9db@oracle.com> References: <502c6e3c-0f49-c895-02b7-bfad8942a9db@oracle.com> Message-ID: <8d2d60e0-c387-5965-edd1-b88f144e5203@oracle.com> Alexander, There is TKit.assertStringListEquals() already. Is it not sufficient? I think it would make sense to add prefix to platform specific properties in fa property files: `CFBundleTypeRole` -> `mac.CFBundleTypeRole`, etc. - Alexey On 6/9/2020 1:12 AM, alexander.matveev at oracle.com wrote: > Please review the jpackage fix for bug [1] at [2]. > > Added support for additional (macOS specific) file association > properties. These additional properties should be set via same > property file which used for file association. Names and values for > properties should match name and values as documented by Apple in > https://developer.apple.com/documentation/bundleresources/information_property_list/cfbundledocumenttypes. > jpackage will simple propagate these values to Info.plist as is. > > [1] https://bugs.openjdk.java.net/browse/JDK-8233215 > [2] http://cr.openjdk.java.net/~almatvee/8233215/webrev.00/ > > Thanks, > Alexander From alexander.matveev at oracle.com Tue Jun 9 06:28:36 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Mon, 8 Jun 2020 23:28:36 -0700 Subject: RFR: 8233215: jpackage doesn't allow enough flexibility for file type binding In-Reply-To: <8d2d60e0-c387-5965-edd1-b88f144e5203@oracle.com> References: <502c6e3c-0f49-c895-02b7-bfad8942a9db@oracle.com> <8d2d60e0-c387-5965-edd1-b88f144e5203@oracle.com> Message-ID: <05d70e32-0bb9-0bc3-970c-1ee99f726c71@oracle.com> Hi Alexey, http://cr.openjdk.java.net/~almatvee/8233215/webrev.01/ - TKit.assertStringListEquals() will be used instead. Did not noticed that we had this method already. I removed my implementation. - Added "mac." prefix. Thanks, Alexander On 6/8/20 10:57 PM, Alexey Semenyuk wrote: > Alexander, > > There is TKit.assertStringListEquals() already. Is it not sufficient? > > I think it would make sense to add prefix to platform specific > properties in fa property files: `CFBundleTypeRole` -> > `mac.CFBundleTypeRole`, etc. > > - Alexey > > On 6/9/2020 1:12 AM, alexander.matveev at oracle.com wrote: >> Please review the jpackage fix for bug [1] at [2]. >> >> Added support for additional (macOS specific) file association >> properties. These additional properties should be set via same >> property file which used for file association. Names and values for >> properties should match name and values as documented by Apple in >> https://developer.apple.com/documentation/bundleresources/information_property_list/cfbundledocumenttypes. >> jpackage will simple propagate these values to Info.plist as is. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8233215 >> [2] http://cr.openjdk.java.net/~almatvee/8233215/webrev.00/ >> >> Thanks, >> Alexander > From amaembo at gmail.com Tue Jun 9 07:05:42 2020 From: amaembo at gmail.com (Tagir Valeev) Date: Tue, 9 Jun 2020 14:05:42 +0700 Subject: Comparator.reversed() type inference issue In-Reply-To: <35B0D7FE-265D-4525-B344-45DC3BFA2766@gmail.com> References: <35B0D7FE-265D-4525-B344-45DC3BFA2766@gmail.com> Message-ID: Hello! This is more related to Java language specification and implementation than to core libraries, so compiler-dev is a more relevant mailing list. The current behavior perfectly follows the specification (JLS, 15.12): A method invocation expression is a poly expression if all of the following are true: - The invocation appears in an assignment context or an invocation context (?5.2, ?5.3). (1) - If the invocation is qualified (that is, any form of MethodInvocation except for the first), then the invocation elides TypeArguments to the left of the Identifier. (2) - The method to be invoked, as determined by the following subsections, is generic (?8.4.4) and has a return type that mentions at least one of the method's type parameters. (3) If method invocation is a qualifier (in your case Map.Entry.comparingByValue() is a qualifier of reversed() call) then the context is neither assignment, nor invocation, hence (1) is not satisfied, hence it's a standalone expression, hence its type is determined solely by expression content. That's why to determine the type of Map.Entry.comparingByValue() we cannot look outside and must consider only the expression content. So its type is determined to be Comparator> (V#2 extends Comparable), as we don't have any other information. The reversed() call is also standalone, as the reversed() method is not generic, so (3) is not satisfied. Thus its type is just the same as its qualifier type. People asked many times whether it's possible to make this code working without explicit type arguments. However, it's not just a trivial patch to the compiler and the spec. It's huge amount of work that may introduce tons of bugs, compiler crashes (I would expect StackOverflowError appear often during the compilation), compiler performance regression, and so on. Also, loads of inconsistencies between compiler and IDE behavior. I personally don't work on Java compiler, but I sometimes investigate bugs inside the IntelliJ IDEA type inference procedure. Believe me, it's already insanely complex, and the fact that at very least we can exclude qualifiers from the equation is really relieving. So while I cannot say for the whole OpenJDK team, I hardly believe this could be implemented any time soon. With best regards, Tagir Valeev. On Sun, Jun 7, 2020 at 7:34 PM Attila Szegedi wrote: > > Hi folks, > > I'm teaching Java lately, and while teaching streams I use that good old chestnut, the word count example. I'm baffled with some type inference woes, though? trying to specify a reverse comparator using Comparator.reversed() makes javac sad, e.g. this does not compile[1]: > > Map x = someMap(); > > var rs1 = x.entrySet().stream().sorted( > Map.Entry.comparingByValue().reversed() > ); > > On the other hand, using Collections.reverseOrder does compile: > > var rs2 = x.entrySet().stream().sorted( > Collections.reverseOrder( > Map.Entry.comparingByValue() > ) > ); > > Happens on both Java 11 and 14. It?s just baffling because based on type signatures, it seems reasonable the first form should work as well. > > Thanks for any enlightenment! > > Attila. > > --- > [1] -Xdiags:verbose on Java 14 says: > error: no suitable method found for sorted(Comparator>) > var rs1 = x.entrySet().stream().sorted( > > method Stream.sorted() is not applicable > (actual and formal argument lists differ in length) > method Stream.sorted(Comparator>) is not applicable > (argument mismatch; Comparator> cannot be converted to Comparator>) > where V#1,V#2 are type-variables: > V#1 extends Comparable > V#2 extends Comparable From raffaello.giulietti at gmail.com Tue Jun 9 07:20:30 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Tue, 9 Jun 2020 09:20:30 +0200 Subject: contributing to JDK-8222187: java.util.Base64.Decoder stream adds unexpected null bytes at the end In-Reply-To: References: Message-ID: Hi Lance, before working on a fix, I just wanted to make sure that I'm not interfering with existing efforts. Thus, I don't have a fix, yet. I'll be using the example provided in the bug report as a basic test. I'll show up here once the fix is ready. Greetings Raffaello On 2020-06-08 22:34, Lance Andersen wrote: > Hi?Raffaello, > > If you are interested in providing a fix, you are more than welcome to > do so. ?Please include a test which validates your fix. > > Best > lance > >> On Jun 8, 2020, at 3:45 PM, Raffaello Giulietti >> > >> wrote: >> >> Raffaello > > > > Lance Andersen| > Principal Member of Technical Staff | +1.781.442.2037 > Oracle?Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From forax at univ-mlv.fr Tue Jun 9 08:18:14 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 9 Jun 2020 10:18:14 +0200 (CEST) Subject: Comparator.reversed() type inference issue In-Reply-To: References: <35B0D7FE-265D-4525-B344-45DC3BFA2766@gmail.com> Message-ID: <29804315.1059543.1591690694628.JavaMail.zimbra@u-pem.fr> yep, the inference is not smart enough. So the way to fix the code is either to explicitly specify the type arguments var rs1 = x.entrySet().stream().sorted( Map.Entry.comparingByValue().reversed() ); or to store the result of comparingByValue() in a variable (so it's a poly expression) Comparator> comparator = Map.Entry.comparingByValue(); var rs1 = x.entrySet().stream().sorted(comparator.reversed()); regards, R?mi ----- Mail original ----- > De: "Tagir Valeev" > ?: "Attila Szegedi" > Cc: "core-libs-dev" > Envoy?: Mardi 9 Juin 2020 09:05:42 > Objet: Re: Comparator.reversed() type inference issue > Hello! > > This is more related to Java language specification and implementation > than to core libraries, so compiler-dev is a more relevant mailing > list. The current behavior perfectly follows the specification (JLS, > 15.12): > > A method invocation expression is a poly expression if all of the > following are true: > - The invocation appears in an assignment context or an invocation > context (?5.2, ?5.3). (1) > - If the invocation is qualified (that is, any form of > MethodInvocation except for the first), then the invocation elides > TypeArguments to the left of the Identifier. (2) > - The method to be invoked, as determined by the following > subsections, is generic (?8.4.4) and has a return type that mentions > at least one of the method's type parameters. (3) > > If method invocation is a qualifier (in your case > Map.Entry.comparingByValue() is a qualifier of reversed() call) then > the context is neither assignment, nor invocation, hence (1) is not > satisfied, hence it's a standalone expression, hence its type is > determined solely by expression content. That's why to determine the > type of Map.Entry.comparingByValue() we cannot look outside and must > consider only the expression content. So its type is determined to be > Comparator> (V#2 extends Comparable), > as we don't have any other information. The reversed() call is also > standalone, as the reversed() method is not generic, so (3) is not > satisfied. Thus its type is just the same as its qualifier type. > > People asked many times whether it's possible to make this code > working without explicit type arguments. However, it's not just a > trivial patch to the compiler and the spec. It's huge amount of work > that may introduce tons of bugs, compiler crashes (I would expect > StackOverflowError appear often during the compilation), compiler > performance regression, and so on. Also, loads of inconsistencies > between compiler and IDE behavior. I personally don't work on Java > compiler, but I sometimes investigate bugs inside the IntelliJ IDEA > type inference procedure. Believe me, it's already insanely complex, > and the fact that at very least we can exclude qualifiers from the > equation is really relieving. So while I cannot say for the whole > OpenJDK team, I hardly believe this could be implemented any time > soon. > > With best regards, > Tagir Valeev. > > On Sun, Jun 7, 2020 at 7:34 PM Attila Szegedi wrote: >> >> Hi folks, >> >> I'm teaching Java lately, and while teaching streams I use that good old >> chestnut, the word count example. I'm baffled with some type inference woes, >> though? trying to specify a reverse comparator using Comparator.reversed() >> makes javac sad, e.g. this does not compile[1]: >> >> Map x = someMap(); >> >> var rs1 = x.entrySet().stream().sorted( >> Map.Entry.comparingByValue().reversed() >> ); >> >> On the other hand, using Collections.reverseOrder does compile: >> >> var rs2 = x.entrySet().stream().sorted( >> Collections.reverseOrder( >> Map.Entry.comparingByValue() >> ) >> ); >> >> Happens on both Java 11 and 14. It?s just baffling because based on type >> signatures, it seems reasonable the first form should work as well. >> >> Thanks for any enlightenment! >> >> Attila. >> >> --- >> [1] -Xdiags:verbose on Java 14 says: >> error: no suitable method found for sorted(Comparator>) >> var rs1 = x.entrySet().stream().sorted( >> >> method Stream.sorted() is not applicable >> (actual and formal argument lists differ in length) >> method Stream.sorted(Comparator>) is not applicable >> (argument mismatch; Comparator> cannot be converted to >> Comparator>) >> where V#1,V#2 are type-variables: >> V#1 extends Comparable > > V#2 extends Comparable From forax at univ-mlv.fr Tue Jun 9 08:25:10 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 9 Jun 2020 10:25:10 +0200 (CEST) Subject: Comparator.reversed() type inference issue In-Reply-To: References: <35B0D7FE-265D-4525-B344-45DC3BFA2766@gmail.com> Message-ID: <1402810623.1066241.1591691110477.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Tagir Valeev" > ?: "Attila Szegedi" > Cc: "core-libs-dev" > Envoy?: Mardi 9 Juin 2020 09:05:42 > Objet: Re: Comparator.reversed() type inference issue > Hello! > > This is more related to Java language specification and implementation > than to core libraries, so compiler-dev is a more relevant mailing > list. The current behavior perfectly follows the specification (JLS, > 15.12): > > A method invocation expression is a poly expression if all of the > following are true: > - The invocation appears in an assignment context or an invocation > context (?5.2, ?5.3). (1) > - If the invocation is qualified (that is, any form of > MethodInvocation except for the first), then the invocation elides > TypeArguments to the left of the Identifier. (2) > - The method to be invoked, as determined by the following > subsections, is generic (?8.4.4) and has a return type that mentions > at least one of the method's type parameters. (3) > > If method invocation is a qualifier (in your case > Map.Entry.comparingByValue() is a qualifier of reversed() call) then > the context is neither assignment, nor invocation, hence (1) is not > satisfied, hence it's a standalone expression, hence its type is > determined solely by expression content. That's why to determine the > type of Map.Entry.comparingByValue() we cannot look outside and must > consider only the expression content. So its type is determined to be > Comparator> (V#2 extends Comparable), > as we don't have any other information. The reversed() call is also > standalone, as the reversed() method is not generic, so (3) is not > satisfied. Thus its type is just the same as its qualifier type. > > People asked many times whether it's possible to make this code > working without explicit type arguments. However, it's not just a > trivial patch to the compiler and the spec. It's huge amount of work > that may introduce tons of bugs, compiler crashes (I would expect > StackOverflowError appear often during the compilation), compiler > performance regression, and so on. Also, loads of inconsistencies > between compiler and IDE behavior. I personally don't work on Java > compiler, but I sometimes investigate bugs inside the IntelliJ IDEA > type inference procedure. Believe me, it's already insanely complex, > and the fact that at very least we can exclude qualifiers from the > equation is really relieving. So while I cannot say for the whole > OpenJDK team, I hardly believe this could be implemented any time > soon. I don't see how you can fix it even with a lot of dust of magic stars, you have only the type information of the return value of Map.Entry.comparingByValue().reversed() but they can no travel backward to Map.Entry.comparingByValue() because you have no idea of the return type of Map.Entry.comparingByValue() so you don't know the class that contains the method "reversed". > > With best regards, > Tagir Valeev. R?mi > > On Sun, Jun 7, 2020 at 7:34 PM Attila Szegedi wrote: >> >> Hi folks, >> >> I'm teaching Java lately, and while teaching streams I use that good old >> chestnut, the word count example. I'm baffled with some type inference woes, >> though? trying to specify a reverse comparator using Comparator.reversed() >> makes javac sad, e.g. this does not compile[1]: >> >> Map x = someMap(); >> >> var rs1 = x.entrySet().stream().sorted( >> Map.Entry.comparingByValue().reversed() >> ); >> >> On the other hand, using Collections.reverseOrder does compile: >> >> var rs2 = x.entrySet().stream().sorted( >> Collections.reverseOrder( >> Map.Entry.comparingByValue() >> ) >> ); >> >> Happens on both Java 11 and 14. It?s just baffling because based on type >> signatures, it seems reasonable the first form should work as well. >> >> Thanks for any enlightenment! >> >> Attila. >> >> --- >> [1] -Xdiags:verbose on Java 14 says: >> error: no suitable method found for sorted(Comparator>) >> var rs1 = x.entrySet().stream().sorted( >> >> method Stream.sorted() is not applicable >> (actual and formal argument lists differ in length) >> method Stream.sorted(Comparator>) is not applicable >> (argument mismatch; Comparator> cannot be converted to >> Comparator>) >> where V#1,V#2 are type-variables: >> V#1 extends Comparable > > V#2 extends Comparable From pavel.rappo at oracle.com Tue Jun 9 10:42:20 2020 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Tue, 9 Jun 2020 11:42:20 +0100 Subject: RFR [15] 8247115: Fix typos in java.lang.invoke and java.lang Message-ID: <97FD23CC-DE74-4D23-BC42-9CC92CB3CB90@oracle.com> Hello, Please review the change for https://bugs.openjdk.java.net/browse/JDK-8247115 http://cr.openjdk.java.net/~prappo/8247115/webrev.00/ The patch fixes 11 typos in doc comments and 2 typos in code. The typos in code are in exception messages. Testing results are pending. -Pavel From andy.herrick at oracle.com Tue Jun 9 11:41:24 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Tue, 9 Jun 2020 07:41:24 -0400 Subject: RFR: 8233215: jpackage doesn't allow enough flexibility for file type binding In-Reply-To: <502c6e3c-0f49-c895-02b7-bfad8942a9db@oracle.com> References: <502c6e3c-0f49-c895-02b7-bfad8942a9db@oracle.com> Message-ID: <5311628d-3179-b044-81fa-6e14bbb71b8d@oracle.com> I would think the new mac specific BundlerParamInfo would be defined in MacAppImageBuilder instead of StandardBundlerParam*. * Am I missing something ? /Andy ** * * On 6/9/2020 1:12 AM, alexander.matveev at oracle.com wrote: > Please review the jpackage fix for bug [1] at [2]. > > Added support for additional (macOS specific) file association > properties. These additional properties should be set via same > property file which used for file association. Names and values for > properties should match name and values as documented by Apple in > https://developer.apple.com/documentation/bundleresources/information_property_list/cfbundledocumenttypes. > jpackage will simple propagate these values to Info.plist as is. > > [1] https://bugs.openjdk.java.net/browse/JDK-8233215 > [2] http://cr.openjdk.java.net/~almatvee/8233215/webrev.00/ > > Thanks, > Alexander From lance.andersen at oracle.com Tue Jun 9 11:51:25 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 9 Jun 2020 07:51:25 -0400 Subject: contributing to JDK-8222187: java.util.Base64.Decoder stream adds unexpected null bytes at the end In-Reply-To: References: Message-ID: <1040A12E-8CA9-45E3-B432-E2758A6D1F49@oracle.com> Hi Raffaello, This bug is not currently being worked on so feel free to provide a fix and test Best Lance > On Jun 9, 2020, at 3:20 AM, Raffaello Giulietti wrote: > > Hi Lance, > > before working on a fix, I just wanted to make sure that I'm not interfering with existing efforts. Thus, I don't have a fix, yet. > > I'll be using the example provided in the bug report as a basic test. > > I'll show up here once the fix is ready. > > > Greetings > Raffaello > > > > On 2020-06-08 22:34, Lance Andersen wrote: >> Hi Raffaello, >> If you are interested in providing a fix, you are more than welcome to do so. Please include a test which validates your fix. >> Best >> lance >>> On Jun 8, 2020, at 3:45 PM, Raffaello Giulietti > wrote: >>> >>> Raffaello >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From pavel.rappo at oracle.com Tue Jun 9 12:15:44 2020 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Tue, 9 Jun 2020 13:15:44 +0100 Subject: RFR [15] 8247212: Use assistant markup in java.lang.module.ModuleDescriptor Message-ID: <9A43D938-0AA8-4933-A65B-D61EB6C79551@oracle.com> Hello, Please review the change for https://bugs.openjdk.java.net/browse/JDK-8247212 http://cr.openjdk.java.net/~prappo/8247212/webrev.00/ The patch adds markup that makes it easier to read documentation in cases where code and prose are mixed. The patch also fixes a few typos. -Pavel From aleksej.efimov at oracle.com Tue Jun 9 12:17:18 2020 From: aleksej.efimov at oracle.com (Aleks Efimov) Date: Tue, 9 Jun 2020 13:17:18 +0100 Subject: RFR [15/java.xml] 8237456: Transform filtered through SAX filter mishandles character entities In-Reply-To: <224e6b23-7999-4a2a-370c-35088a085d1d@oracle.com> References: <224e6b23-7999-4a2a-370c-35088a085d1d@oracle.com> Message-ID: Hi Joe, The changes looks good to me. Best, Aleksei On 09/06/2020 06:54, Joe Wang wrote: > Hi, > > Please review a fix to a regression that resulted in invalid output in > a transform that contains entity references. The regression was caused > by a JDK9 patch [1]. Adding back the conditional checks fixed the issue. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8237456 > webrevs: http://cr.openjdk.java.net/~joehw/jdk15/8237456/webrev/ > > [1] https://bugs.openjdk.java.net/browse/JDK-8087303 > > Thanks, > Joe > From Alan.Bateman at oracle.com Tue Jun 9 12:33:04 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 9 Jun 2020 13:33:04 +0100 Subject: RFR [15] 8247212: Use assistant markup in java.lang.module.ModuleDescriptor In-Reply-To: <9A43D938-0AA8-4933-A65B-D61EB6C79551@oracle.com> References: <9A43D938-0AA8-4933-A65B-D61EB6C79551@oracle.com> Message-ID: On 09/06/2020 13:15, Pavel Rappo wrote: > Hello, > > Please review the change for https://bugs.openjdk.java.net/browse/JDK-8247212 > > http://cr.openjdk.java.net/~prappo/8247212/webrev.00/ > > The patch adds markup that makes it easier to read documentation in cases where code and prose are mixed. The patch also fixes a few typos. ServiceCatalog is not in an exported package so surprised it shows up in a javadoc build. In any case, this mostly look okay. Can you fix ModuleDescriptor L827/828 as the changes makes the line length a bit inconsistent with everything around it. -Alan. From lance.andersen at oracle.com Tue Jun 9 12:36:25 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 9 Jun 2020 08:36:25 -0400 Subject: RFR [15] 8247115: Fix typos in java.lang.invoke and java.lang In-Reply-To: <97FD23CC-DE74-4D23-BC42-9CC92CB3CB90@oracle.com> References: <97FD23CC-DE74-4D23-BC42-9CC92CB3CB90@oracle.com> Message-ID: Hi Pavel, The changes look good Best Lance > On Jun 9, 2020, at 6:42 AM, Pavel Rappo wrote: > > Hello, > > Please review the change for https://bugs.openjdk.java.net/browse/JDK-8247115 > > http://cr.openjdk.java.net/~prappo/8247115/webrev.00/ > > The patch fixes 11 typos in doc comments and 2 typos in code. The typos in code are in exception messages. Testing results are pending. > > -Pavel > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From alexey.semenyuk at oracle.com Tue Jun 9 12:39:36 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Tue, 9 Jun 2020 08:39:36 -0400 Subject: RFR: 8233215: jpackage doesn't allow enough flexibility for file type binding In-Reply-To: <05d70e32-0bb9-0bc3-970c-1ee99f726c71@oracle.com> References: <502c6e3c-0f49-c895-02b7-bfad8942a9db@oracle.com> <8d2d60e0-c387-5965-edd1-b88f144e5203@oracle.com> <05d70e32-0bb9-0bc3-970c-1ee99f726c71@oracle.com> Message-ID: Looks good. - Alexey On 6/9/2020 2:28 AM, alexander.matveev at oracle.com wrote: > Hi Alexey, > > http://cr.openjdk.java.net/~almatvee/8233215/webrev.01/ > > - TKit.assertStringListEquals() will be used instead. Did not noticed > that we had this method already. I removed my implementation. > - Added "mac." prefix. > > Thanks, > Alexander > > On 6/8/20 10:57 PM, Alexey Semenyuk wrote: >> Alexander, >> >> There is TKit.assertStringListEquals() already. Is it not sufficient? >> >> I think it would make sense to add prefix to platform specific >> properties in fa property files: `CFBundleTypeRole` -> >> `mac.CFBundleTypeRole`, etc. >> >> - Alexey >> >> On 6/9/2020 1:12 AM, alexander.matveev at oracle.com wrote: >>> Please review the jpackage fix for bug [1] at [2]. >>> >>> Added support for additional (macOS specific) file association >>> properties. These additional properties should be set via same >>> property file which used for file association. Names and values for >>> properties should match name and values as documented by Apple in >>> https://developer.apple.com/documentation/bundleresources/information_property_list/cfbundledocumenttypes. >>> jpackage will simple propagate these values to Info.plist as is. >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8233215 >>> [2] http://cr.openjdk.java.net/~almatvee/8233215/webrev.00/ >>> >>> Thanks, >>> Alexander >> > From lance.andersen at oracle.com Tue Jun 9 12:41:21 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 9 Jun 2020 08:41:21 -0400 Subject: RFR [15/java.xml] 8237456: Transform filtered through SAX filter mishandles character entities In-Reply-To: <224e6b23-7999-4a2a-370c-35088a085d1d@oracle.com> References: <224e6b23-7999-4a2a-370c-35088a085d1d@oracle.com> Message-ID: <6CE020DC-E3BA-4DB4-93FB-21F4689762C8@oracle.com> Hi Joe, This looks good. For your test, Intellij typically suggests using a static import for Assert: ?????? import static org.testng.Assert.*; ????????? No need to change but typically which get suggested when you run Inspect Code Best Lance > On Jun 9, 2020, at 1:54 AM, Joe Wang wrote: > > Hi, > > Please review a fix to a regression that resulted in invalid output in a transform that contains entity references. The regression was caused by a JDK9 patch [1]. Adding back the conditional checks fixed the issue. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8237456 > webrevs: http://cr.openjdk.java.net/~joehw/jdk15/8237456/webrev/ > > [1] https://bugs.openjdk.java.net/browse/JDK-8087303 > > Thanks, > Joe > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From aleksej.efimov at oracle.com Tue Jun 9 12:59:09 2020 From: aleksej.efimov at oracle.com (Aleks Efimov) Date: Tue, 9 Jun 2020 13:59:09 +0100 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> <9013C8D0-33D9-4D09-8953-F8618020A7F7@oracle.com> <8A2C4DEF-FC61-4574-B49D-1133F5349BCE@azul.com> <15fa46f2-0352-a111-a9c7-0d5fc31d9159@oracle.com> Message-ID: <6167ea24-e7f9-c259-e2c5-8be96d74a40d@oracle.com> Hi Alexey, Thank you for incorporating LdapCtx and LdapSasl changes. I've reviewed both classes and they look good to me, with few minor comments in LdapSasl.java: ????? missing spaces in the following lines: 78, 152 ????? With your last changes we can remove explicit cast of 'envProps' on line 176 I've also run your changes through our CI and one test is failing due to the changes in GssKrb5Client: The failed test name: sun/security/krb5/auto/SaslMutual.java The observed failure: java.lang.UnsupportedOperationException ??? at java.base/java.util.ImmutableCollections.uoe(ImmutableCollections.java:127) ??? at java.base/java.util.ImmutableCollections$AbstractImmutableMap.remove(ImmutableCollections.java:910) ??? at jdk.security.jgss/com.sun.security.sasl.gsskerb.GssKrb5Client.(GssKrb5Client.java:156) ??? at jdk.security.jgss/com.sun.security.sasl.gsskerb.FactoryImpl.createSaslClient(FactoryImpl.java:63) ??? at java.security.sasl/javax.security.sasl.Sasl.createSaslClient(Sasl.java:433) ??? at SaslMutual.main(SaslMutual.java:50) ??? at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ??? at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64) ??? at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ??? at java.base/java.lang.reflect.Method.invoke(Method.java:564) ??? at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127) ??? at java.base/java.lang.Thread.run(Thread.java:832) For information about CSR process you can start from this wiki page: https://wiki.openjdk.java.net/display/csr Best Regards, Aleksei On 08/06/2020 22:33, Alexey Bakhtin wrote: > Hello Sean, > > Yes, I think we'll need CSR and release notes as soon as this patch adds new property. > I do not know exact process for it, so I will be grateful if you could explain me exact steps. > > This patch was developed to address compatibility issue with new LDAP authentication over SSL/TLS announced by Microsoft [1]. It is not related to RFC 5801. In my opinion ?com.sun.jndi.ldap.tls.cbtype? name looks more suitable for this property and should allow backport it to early JDK versions. > > [1] - https://support.microsoft.com/en-au/help/4034879/how-to-add-the-ldapenforcechannelbinding-registry-entry > Regards > Alexey > >> On 8 Jun 2020, at 22:03, Sean Mullan wrote: >> >> (resending to all lists on the review) >> >> I'm just catching up a bit on this review. >> >> Sorry if this has mentioned before, but are you planning to write a CSR and release note? I think this is needed for the com.sun.jndi.ldap.tls.cbtype property. I'm also wondering if this property should be documented in the javadocs, and why it is not a standard property (i.e. "java.naming.ldap.tls.cbtype"). >> >> I was also wondering what relation this has to the "G2" standard SASL mechanisms defined in RFC 5801 [1], and whether that is something we should be using to negotiate this channel binding, and if not, why not. Or if this is something that is implementation-specific and will only work with Microsoft LDAP technology, in which case, we might want to make that more explicit, perhaps by including "microsoft" or something like that in the property name. >> >> Thanks, >> Sean >> >> [1] https://tools.ietf.org/html/rfc5801 >> >> On 6/8/20 9:07 AM, Aleks Efimov wrote: >>> Hi Alexey, >>> I've looked through LdapCtx/LdapClient/SaslBind changes: >>> Do we need to check if CHANNEL_BINDING is set explicitly for all connection types? Maybe we can move the check inside 'if (conn.sock instanceof SSLSocket) {' block. >>> Also, instead of setting CHANNEL_BINDING in context environment and then removing it in finally block, it would be better to clone the environment, put calculated CHANNEL_BINDING into it, and pass the cloned one to Sasl.createSaslClient. >>> Another suggestion about the code that verifies if both properties are set before connection is started: >>> As you've already mentioned the new code in LdapCtx is only needed for checking if timeout is set. Could we try to remove LdapCtx::cbType field and all related methods from LdapCtx (this class is already over-complicated and hard to read) and replace it with some static method in LdapSasl? It will help to localize all changes to LdapSasl except for one line in LdapCtx. >>> I mean something like this: >>> Replace >>> + >>> + // verify LDAP channel binding property >>> + if (cbType != null && connectTimeout == -1) >>> + throw new NamingException(TlsChannelBinding.CHANNEL_BINDING_TYPE + >>> + " property requires " + >>> + CONNECT_TIMEOUT + >>> + " property is set."); >>> With >>> + LdapSasl.checkCbParameters((String)envprops.get(TlsChannelBinding.CHANNEL_BINDING_TYPE), connectTimeout); >>> And add something like that to LdapSasl (or maybe pass the full env here): >>> + public static void checkCbParameters(String cbTypePropertyValue, int connectTimeout) throws NamingException { >>> + TlsChannelBindingType cbType = TlsChannelBinding.parseType(cbTypePropertyValue); >>> + // verify LDAP channel binding property >>> + if (cbType != null && connectTimeout == -1) { >>> + throw new NamingException(TlsChannelBinding.CHANNEL_BINDING_TYPE + >>> + " property requires com.sun.jndi.ldap.connect.timeout" + >>> + " property is set."); >>> + } >>> + } >>> Other LdapCtx/LdapClient/SaslBind changes look fine to me. >>> With Kind Regards, >>> Aleksei >>> On 06/06/2020 20:45, Alexey Bakhtin wrote: >>>> Hello Max, Daniel, >>>> >>>> Thank you for review. >>>> >>>> Please review new version of the patch : >>>> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v5/ >>>> >>>> In this version: >>>> - TlsChannelBinding class is moved into the com.sun.jndi.ldap.sasl package >>>> - SSL Ceritificate related code is moved from LdapClient into the LdapSasl.saslBind method >>>> - verification and removal of internal property is also moved to LdapSasl.saslBind method >>>> - verification of connectTimeout property is moved to LdapCtx.connect. I?ve found that connectionTimeout could be assigned later then cbType >>>> >>>> The test for this issue is not ready yet. I did not find any suitable test case that can be reused. >>>> >>>> Thank you >>>> Alexey >>>> >>>>> On 6 Jun 2020, at 09:44, Weijun Wang wrote: >>>>> >>>>> >>>>> >>>>>> On Jun 6, 2020, at 2:41 PM, Weijun Wang wrote: >>>>>> >>>>>> >>>>>> >>>>>>> On Jun 5, 2020, at 11:03 PM, Alexey Bakhtin wrote: >>>>>>> >>>>>>> Hello Max, >>>>>>> >>>>>>> Thank you a lot for review. >>>>>>> >>>>>>> Could you check the new version of the patch : >>>>>>> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v4/ >>>>>>> >>>>>>> I?ve made the following changes: >>>>>>> - TlsChannelBinding class is moved to java.naming module. >>>>>>> java.security.sasl module is not affected any more >>>>>>> - I pass tlsCB.getData() directly to the SASL mechanism as you suggested >>>>>>> - I?ve made some guards to prevent application from using "com.sun.security.sasl.tlschannelbinding? property directly: >>>>>>> - LdapClient verifies if internal property is not set >>>>>> 245 // Prepare TLS Channel Binding data >>>>>> 246 if (conn.sock instanceof SSLSocket) { >>>>>> 247 // Internal property cannot be set explicitly >>>>>> 248 if (env.get(TlsChannelBinding.CHANNEL_BINDING) != null) { >>>>>> 249 throw new NamingException(TlsChannelBinding.CHANNEL_BINDING + >>>>>> 250 " property cannot be set explicitly"); >>>>>> 251 } >>>>>> >>>>>> If not TLS, this property value be kept there and visible inside the SASL mech. >>>>>> >>>>>>> - GssKrb5Client uses props.remove() to read and remove internal property >>>>> Maybe you can remove the value in LdapClient.java, in case the mech used is not GSSAPI. You can remove it in a finally block after line 303. >>>>> >>>>> --Max >>>>> >>>>>> Traditionally, we use "com.sun..." name which is a JDK supported name (although not at Java SE level), you might want to use a name which is even more internal. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Max >>>>>> >>>>>> p.s. I see that NTLM also supports ChannelBinding. I'll see if I can improve the NTLM SASL mech to support it. From pavel.rappo at oracle.com Tue Jun 9 13:31:48 2020 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Tue, 9 Jun 2020 14:31:48 +0100 Subject: RFR [15] 8247115: Fix typos in java.lang.invoke and java.lang In-Reply-To: References: <97FD23CC-DE74-4D23-BC42-9CC92CB3CB90@oracle.com> Message-ID: <3B2311A2-C8B9-4185-B86A-F77E191D4228@oracle.com> Thanks, Lance. Testing tiers 1, 2, and 3 has shown no errors. If anyone else wants to chime in, please do so within the next couple of hours. -Pavel > On 9 Jun 2020, at 13:36, Lance Andersen wrote: > > Hi Pavel, > > The changes look good > > Best > Lance > >> On Jun 9, 2020, at 6:42 AM, Pavel Rappo wrote: >> >> Hello, >> >> Please review the change for https://bugs.openjdk.java.net/browse/JDK-8247115 >> >> http://cr.openjdk.java.net/~prappo/8247115/webrev.00/ >> >> The patch fixes 11 typos in doc comments and 2 typos in code. The typos in code are in exception messages. Testing results are pending. >> >> -Pavel >> >> > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From pavel.rappo at oracle.com Tue Jun 9 13:33:28 2020 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Tue, 9 Jun 2020 14:33:28 +0100 Subject: RFR [15] 8247212: Use assistant markup in java.lang.module.ModuleDescriptor In-Reply-To: References: <9A43D938-0AA8-4933-A65B-D61EB6C79551@oracle.com> Message-ID: <8782577A-91DA-4EA3-AEB0-3C615B5A5F24@oracle.com> Thanks, Alan. Here's the updated webrev: http://cr.openjdk.java.net/~prappo/8247212/webrev.01/ > On 9 Jun 2020, at 13:33, Alan Bateman wrote: > > On 09/06/2020 13:15, Pavel Rappo wrote: >> Hello, >> >> Please review the change for https://bugs.openjdk.java.net/browse/JDK-8247212 >> >> http://cr.openjdk.java.net/~prappo/8247212/webrev.00/ >> >> The patch adds markup that makes it easier to read documentation in cases where code and prose are mixed. The patch also fixes a few typos. > ServiceCatalog is not in an exported package so surprised it shows up in a javadoc build. In any case, this mostly look okay. Can you fix ModuleDescriptor L827/828 as the changes makes the line length a bit inconsistent with everything around it. > > -Alan. From Roger.Riggs at oracle.com Tue Jun 9 13:42:48 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Tue, 9 Jun 2020 09:42:48 -0400 Subject: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) In-Reply-To: <814DA850-76A9-4526-A3D9-141CD516C9AE@amperemail.onmicrosoft.com> References: <814DA850-76A9-4526-A3D9-141CD516C9AE@amperemail.onmicrosoft.com> Message-ID: Hi Peter, I'd like to understand the scope and impact of the problem before jumping to a solution. For specific applications, overriding ObjectInputStream.resolveClass to handle the class lookup can handle the case as expeditiously as is necessary. What application uses cases are impacted and how prevelant are they? Many, many frameworks and applications create classloaders for their own purposes and take advantage of the required behavior. Thanks, Roger On 6/4/20 7:08 PM, Peter Kessler OS wrote: > ObjectInputStream.resolveClass(ObjectStreamClass) is specified to find "the first class loader on the current thread's stack (starting from the currently executing method) that is neither the platform class loader nor its ancestor." Finding that class loader is done with a virtual frame walk in JVM_LatestUserDefinedLoader(JNIEnv *env). The virtual frame walk is expensive. If an application does not define any ClassLoaders, then the virtual frame walk will always find the system class loader. > > If an application does a significant amount of reading from an ObjectInputStream, the cost of ObjectInputStream.resolveClass(ObjectInputStreamClass) can be considerable. In a sample application, that method (and the methods it calls) have been measured to consume 5+% of the CPU cycles. > > The proposal is to note the construction of any user-defined ClassLoader, and if none has been constructed, avoid the virtual frame walk returning ClassLoader.getSystemClassLoader(). But if any user-defined ClassLoader has been constructed, use the virtual frame walk to find the first user-defined ClassLoader on the thread stack. > > User-defined ClassLoaders could be distinguished from non-user-defined ClassLoaders in several ways. The ClassLoader class hierarchy could be changed to make non-user-defined ClassLoaders inherit from a marker class. There is already a jdk.internal.loader.BuiltinClassLoader but it is used for module (and resource) loading. BuiltinClassLoader is not a super-class of jdk.internal.reflect.DelegatingClassLoader, which is used to load reflection classes, and reflection methods are also ignored by the virtual frame walk. ClassLoaders could have a boolean that distinguished whether they are user-defined or not. That would require a change to ClassLoader API. There are other ways to distinguish user-defined ClassLoaders. The proposal is to keep the decoration of the ClassLoaders out of the instances, so that it is only visible to the code that needs it. > > The proposal is that each non-user-defined ClassLoader have a static block that registers the class as a non-user-defined ClassLoader. Then in the base ClassLoader constructor check if the ClassLoader being constructed is a user-defined ClassLoader and if so set a boolean that acts as the guard on the virtual frame walk. If additional non-user-defined ClassLoaders are declared in the future beyond the ones identified in this patch, they can be added to the registry. > > There are currently 4 non-user-defined ClassLoader classes, so the registry is not difficult to maintain. Nor is the registry difficult to search when ClassLoaders are constructed. The guard boolean that records whether a user-defined ClassLoader has been constructed transitions from false to true and never becomes false again. (An extension to transition the boolean back to false when a user-defined ClassLoader becomes unreachable is beyond the scope of this proposal.) > > This proposal slows loading of non-user-defined ClassLoaders by the time it takes to register them. It slows ClassLoader construction by the time it takes to consult the registry and conditionally set the guard boolean. At each invocation of ObjectInputStream.resolveClass(ObjectInputStreamClass), the guard boolean is used to possibly avoid the virtual frame walk. > > Tested with `make run-test-tier1` on Linux (CentOS 7) on both aarch64 and x86_64. I had one failure on each of the runs > > ACTION: main -- Failed. Execution failed: `main' threw exception: java.lang.Error: TESTBUG: the library has not been found in ${Repo}/build/linux-aarch64-server-release/images/test/hotspot/jtreg/native > REASON: User specified action: run main/native GTestWrapper > > on both plaforms. But I get the same failure when I run the tier1 tests on clones of OpenJDK-15+25 on both aarch64 and x86_64. > > Running a sample (representative?) application with the Oracle Developer Studio analyzer shows a performance comparison of > > Stack Fragment sorted by metric: Attributed Total CPU Time > baseline.er experiment.er > Attributed Attributed Name > Total CPU Time Total CPU Time > sec. sec. > ================================== Callers > 3459.210 29.931 java.io.ObjectInputStream.readOrdinaryObject(boolean) > 1139.727 3.532 java.io.ObjectInputStream.readArray(boolean) > 875.262 9.116 java.io.ObjectInputStream.readNonProxyDesc(boolean) > > ================================== Stack Fragment > java.io.ObjectInputStream.readClassDesc(boolean) > java.io.ObjectInputStream.readNonProxyDesc(boolean) > java.io.ObjectInputStream.resolveClass(java.io.ObjectStreamClass) > java.io.ObjectInputStream.latestUserDefinedLoader() > 4.173 3.953 jdk.internal.misc.VM.latestUserDefinedLoader() > > ================================== Callees > 5470.026 0. jdk.internal.misc.VM.latestUserDefinedLoader0() > 0. 38.627 java.lang.ClassLoader.getSystemClassLoader() > > The `hg export -g` below is with respect to OpenJDK-15+25. > > Thank you for your review comments. I will need a sponsor to get this change into the repository. > > ... peter > > # HG changeset patch > # User Peter Kessler > # Date 1591310467 25200 > # Thu Jun 04 15:41:07 2020 -0700 > # Node ID 9a39488182c1dfc5bc7bb41d562d7ef16ee657f6 > # Parent 90b266a84c06f1b3dc0ed8767856793e8c1c357e > Improve the performance of ObjectInputStream.resolveClass > > diff --git a/src/java.base/share/classes/java/lang/ClassLoader.java b/src/java.base/share/classes/java/lang/ClassLoader.java > --- a/src/java.base/share/classes/java/lang/ClassLoader.java > +++ b/src/java.base/share/classes/java/lang/ClassLoader.java > @@ -385,6 +385,8 @@ > } > this.package2certs = new ConcurrentHashMap<>(); > this.nameAndId = nameAndId(this); > + /* Note the construction of a ClassLoader. */ > + VM.noteClassLoaderConstruction(this.getClass()); > } > > /** > diff --git a/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java b/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java > --- a/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java > +++ b/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java > @@ -117,6 +117,11 @@ > protected Class loadClassOrNull(String cn, boolean resolve) { > return JLA.findBootstrapClassOrNull(this, cn); > } > + > + static { > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > + VM.registerNotUserDefinedClassLoader(BootClassLoader.class); > + } > }; > > /** > @@ -127,6 +132,8 @@ > static { > if (!ClassLoader.registerAsParallelCapable()) > throw new InternalError(); > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > + VM.registerNotUserDefinedClassLoader(PlatformClassLoader.class); > } > > PlatformClassLoader(BootClassLoader parent) { > @@ -142,6 +149,8 @@ > static { > if (!ClassLoader.registerAsParallelCapable()) > throw new InternalError(); > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > + VM.registerNotUserDefinedClassLoader(AppClassLoader.class); > } > > final URLClassPath ucp; > diff --git a/src/java.base/share/classes/jdk/internal/misc/VM.java b/src/java.base/share/classes/jdk/internal/misc/VM.java > --- a/src/java.base/share/classes/jdk/internal/misc/VM.java > +++ b/src/java.base/share/classes/jdk/internal/misc/VM.java > @@ -304,12 +304,44 @@ > private static final int JVMTI_THREAD_STATE_WAITING_INDEFINITELY = 0x0010; > private static final int JVMTI_THREAD_STATE_WAITING_WITH_TIMEOUT = 0x0020; > > + /** A registry of not user-defined ClassLoaders. */ > + private static final List> notUserDefinedClassLoaderRegistry = > + Collections.synchronizedList(new ArrayList<>()); > + > + /** Register a ClassLoader class as being not a user-defined ClassLoader. */ > + public static void registerNotUserDefinedClassLoader(Class classLoaderClass) { > + notUserDefinedClassLoaderRegistry.add(classLoaderClass); > + } > + > + /** A flag for whether a user-defined ClassLoaders has been constructed. */ > + private static volatile boolean constructedUserDefinedClassLoaderFlag = false; > + > + /** > + * Note a ClassLoader construction, and record if a > + * user-defined ClassLoader has been constructed. > + */ > + public static void noteClassLoaderConstruction(Class classLoaderClass) { > + /* Check if the ClassLoader class not a user-defined ClassLoader. */ > + if (notUserDefinedClassLoaderRegistry.contains(classLoaderClass)) { > + return; > + } > + constructedUserDefinedClassLoaderFlag = true; > + return; > + } > + > /* > * Returns the first user-defined class loader up the execution stack, > * or the platform class loader if only code from the platform or > * bootstrap class loader is on the stack. > */ > public static ClassLoader latestUserDefinedLoader() { > + if (!constructedUserDefinedClassLoaderFlag) { > + /* > + * If no user-defined ClassLoader has been constructed, > + * then I will not find a user-defined ClassLoader in the stack. > + */ > + return ClassLoader.getSystemClassLoader(); > + } > ClassLoader loader = latestUserDefinedLoader0(); > return loader != null ? loader : ClassLoader.getPlatformClassLoader(); > } > diff --git a/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java b/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java > --- a/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java > +++ b/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java > @@ -30,6 +30,7 @@ > > import jdk.internal.access.JavaLangAccess; > import jdk.internal.access.SharedSecrets; > +import jdk.internal.misc.VM; > > /** Utility class which assists in calling defineClass() by > creating a new class loader which delegates to the one needed in > @@ -73,4 +74,9 @@ > DelegatingClassLoader(ClassLoader parent) { > super(parent); > } > + > + static { > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > + VM.registerNotUserDefinedClassLoader(DelegatingClassLoader.class); > + } > } > From Alan.Bateman at oracle.com Tue Jun 9 14:32:14 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 9 Jun 2020 15:32:14 +0100 Subject: RFR [15] 8247212: Use assistant markup in java.lang.module.ModuleDescriptor In-Reply-To: <8782577A-91DA-4EA3-AEB0-3C615B5A5F24@oracle.com> References: <9A43D938-0AA8-4933-A65B-D61EB6C79551@oracle.com> <8782577A-91DA-4EA3-AEB0-3C615B5A5F24@oracle.com> Message-ID: <39ebabd9-18b8-67a8-66ca-34fe278063cf@oracle.com> On 09/06/2020 14:33, Pavel Rappo wrote: > Thanks, Alan. Here's the updated webrev: > > http://cr.openjdk.java.net/~prappo/8247212/webrev.01/ > > Thanks, this looks good to me. -Alan From andy.herrick at oracle.com Tue Jun 9 14:40:15 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Tue, 9 Jun 2020 10:40:15 -0400 Subject: JPackage with a modular application fails to load record class files even with --enable-preview In-Reply-To: <5edce682.1c69fb81.9b9c2.832e@mx.google.com> References: <5edce682.1c69fb81.9b9c2.832e@mx.google.com> Message-ID: <0b533174-ca57-c7fc-c735-fc2ce265771d@oracle.com> So when you use the jpackage option "--java-options --enable-preview" it appears to work with a non-modular jar, but not with a modular jar ? I think I can create a testcase for that. Have you tried it with the jpackage from a preview release of JDK15? (https://jdk.java.net/15/) ? Can you look at the runtime in the app-image of the failing case, and see if the preview module is there ?? Perhaps the problem is the way jpackage invokes jlink. A final thing you can try is run jlink yourself, and use the resulting runtime-image in jpackage args. /Andy On 6/7/2020 9:07 AM, Markus Jevring wrote: > I have this app, which uses Java 14 with ?enable-preview and records. If I turn it in to a normal non-modular jar, it works with both java -jar ?enable-preview myapp.jar, and with a native image built with jpackage. However, when i turn it into a modular jar, I can still launch it with java ?enable-preview ?module-path ... ?module ..., but when I try to create a native image using jpackage usign the same modular flags, launching it fails. I get a dialog box saying ?Failed to launch JVM?, and if I enable ?win-console when I build it and relaunch it, it tells me this: > > Exception in thread "main" java.lang.ClassFormatError: Invalid constant pool index 31 for name in Record attribute in class file net/jevring/frequencies/v2/input/KeyTimings$Note > at java.base/java.lang.ClassLoader.defineClass2(Native Method) > at java.base/java.lang.ClassLoader.defineClass(Unknown Source) > at java.base/java.security.SecureClassLoader.defineClass(Unknown Source) > at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(Unknown Source) > at java.base/jdk.internal.loader.BuiltinClassLoader.findClassInModuleOrNull(Unknown Source) > at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(Unknown Source) > at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source) > at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown Source) > at java.base/java.lang.ClassLoader.loadClass(Unknown Source) > at frequencies/net.jevring.frequencies.v2.input.KeyTimings.(Unknown Source) > at frequencies/net.jevring.frequencies.v2.Thief.(Unknown Source) > at frequencies/net.jevring.frequencies.v2.Main.main(Unknown Source) > > Note, above, is a record. I tried moving it out to a separate class, and I still got the same error. When I converted Note to a normal class, it started complaining about the next record it cound. > I know that it?s picking up the ?enable-preview java option, because when I omit that, it complains about the class file version being wrong. > > Since I can run it just fine using ?java -jar?, and it works fine under all other circumstances, I must conclude that there?s something wrong with the intersection of modular jars and jpackage. > I?ve googled around plenty, and I haven?t found anything that might help me. I?m not sure if this is a call for help or a bug report, though I?d like to consider it the latter =) > > I put what I had on a branch: https://bitbucket.org/jevring/frequencies/branch/java-modules > I realize it?s not a small self-contained example that replicates the issue, and for that I?m sorry. If someone is interested about this, I could try to make one. From info at j-kuhn.de Tue Jun 9 14:53:41 2020 From: info at j-kuhn.de (Johannes Kuhn) Date: Tue, 9 Jun 2020 16:53:41 +0200 Subject: JPackage with a modular application fails to load record class files even with --enable-preview In-Reply-To: <0b533174-ca57-c7fc-c735-fc2ce265771d@oracle.com> References: <5edce682.1c69fb81.9b9c2.832e@mx.google.com> <0b533174-ca57-c7fc-c735-fc2ce265771d@oracle.com> Message-ID: <120856ed-19bb-b4b8-a4a5-628052f6d760@j-kuhn.de> Relevant StackOverlow question: https://stackoverflow.com/questions/61504956/records-in-jlinked-application-throws-exception It talks about JLink, but the error message is the same, so I expect the same root cause: The ASM version shipped with JDK 14 doesn't support records yet. It is already fixed for JDK 15. - Johannes On 09-Jun-20 16:40, Andy Herrick wrote: > So when you use the jpackage option "--java-options --enable-preview" > it appears to work with a non-modular jar, but not with a modular jar ? > > I think I can create a testcase for that. > > Have you tried it with the jpackage from a preview release of JDK15? > (https://jdk.java.net/15/) ? > > Can you look at the runtime in the app-image of the failing case, and > see if the preview module is there ?? Perhaps the problem is the way > jpackage invokes jlink. > > A final thing you can try is run jlink yourself, and use the resulting > runtime-image in jpackage args. > > /Andy > > On 6/7/2020 9:07 AM, Markus Jevring wrote: >> I have this app, which uses Java 14 with ?enable-preview and records. >> If I turn it in to a normal non-modular jar, it works with both java >> -jar ?enable-preview myapp.jar, and with a native image built with >> jpackage. However, when i turn it into a modular jar, I can still >> launch it with java ?enable-preview ?module-path ... ?module ..., >> but? when I try to create a native image using jpackage usign the >> same modular flags, launching it fails. I get a dialog box saying >> ?Failed to launch JVM?, and if I enable ?win-console when I build it >> and relaunch it, it tells me this: >> >> Exception in thread "main" java.lang.ClassFormatError: Invalid >> constant pool index 31 for name in Record attribute in class file >> net/jevring/frequencies/v2/input/KeyTimings$Note >> ???????? at java.base/java.lang.ClassLoader.defineClass2(Native Method) >> ???????? at java.base/java.lang.ClassLoader.defineClass(Unknown Source) >> ???????? at >> java.base/java.security.SecureClassLoader.defineClass(Unknown Source) >> ???????? at >> java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(Unknown >> Source) >> ???????? at >> java.base/jdk.internal.loader.BuiltinClassLoader.findClassInModuleOrNull(Unknown >> Source) >> ???????? at >> java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(Unknown >> Source) >> ???????? at >> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown >> Source) >> ???????? at >> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown >> Source) >> ???????? at java.base/java.lang.ClassLoader.loadClass(Unknown Source) >> ???????? at >> frequencies/net.jevring.frequencies.v2.input.KeyTimings.(Unknown >> Source) >> ???????? at >> frequencies/net.jevring.frequencies.v2.Thief.(Unknown Source) >> ???????? at frequencies/net.jevring.frequencies.v2.Main.main(Unknown >> Source) >> >> Note, above, is a record. I tried moving it out to a separate class, >> and I still got the same error. When I converted Note to a normal >> class, it started complaining about the next record it cound. >> I know that it?s picking up the ?enable-preview java option, because >> when I omit that, it complains about the class file version being wrong. >> >> Since I can run it just fine using ?java -jar?, and it works fine >> under all other circumstances, I must conclude that there?s something >> wrong with the intersection of modular jars and jpackage. >> I?ve googled around plenty, and I haven?t found anything that might >> help me. I?m not sure if this is a call for help or a bug report, >> though I?d like to consider it the latter =) >> >> I put what I had on a branch: >> https://bitbucket.org/jevring/frequencies/branch/java-modules >> I realize it?s not a small self-contained example that replicates the >> issue, and for that I?m sorry. If someone is interested about this, I >> could try to make one. From claes.redestad at oracle.com Tue Jun 9 15:23:36 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 9 Jun 2020 17:23:36 +0200 Subject: RFR: 8246129: ZIP entries created for DOS epoch include local timezone metadata Message-ID: Hi, this patch addresses a corner case where extra time stamp information is added when using start of the DOS time epoch (1980-01-01 00:00:00) as a dummy timestamp for entries Bug: https://bugs.openjdk.java.net/browse/JDK-8246129 Webrev: http://cr.openjdk.java.net/~redestad/8246129/open.00/ Testing: tier1-2 Thanks! /Claes From andy.herrick at oracle.com Tue Jun 9 15:29:01 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Tue, 9 Jun 2020 11:29:01 -0400 Subject: JPackage with a modular application fails to load record class files even with --enable-preview In-Reply-To: <120856ed-19bb-b4b8-a4a5-628052f6d760@j-kuhn.de> References: <5edce682.1c69fb81.9b9c2.832e@mx.google.com> <0b533174-ca57-c7fc-c735-fc2ce265771d@oracle.com> <120856ed-19bb-b4b8-a4a5-628052f6d760@j-kuhn.de> Message-ID: <76653ea6-0814-21ad-c159-f4c034af9527@oracle.com> I tried a simple testcase with 14.0.2 and with 15, and if failed as indicated with 14.0.2 but worked fine in 15. /Andy On 6/9/2020 10:53 AM, Johannes Kuhn wrote: > Relevant StackOverlow question: > https://stackoverflow.com/questions/61504956/records-in-jlinked-application-throws-exception > > It talks about JLink, but the error message is the same, so I expect > the same root cause: > The ASM version shipped with JDK 14 doesn't support records yet. It is > already fixed for JDK 15. > > - Johannes > > On 09-Jun-20 16:40, Andy Herrick wrote: >> So when you use the jpackage option "--java-options --enable-preview" >> it appears to work with a non-modular jar, but not with a modular jar ? >> >> I think I can create a testcase for that. >> >> Have you tried it with the jpackage from a preview release of JDK15? >> (https://jdk.java.net/15/) ? >> >> Can you look at the runtime in the app-image of the failing case, and >> see if the preview module is there ?? Perhaps the problem is the way >> jpackage invokes jlink. >> >> A final thing you can try is run jlink yourself, and use the >> resulting runtime-image in jpackage args. >> >> /Andy >> >> On 6/7/2020 9:07 AM, Markus Jevring wrote: >>> I have this app, which uses Java 14 with ?enable-preview and >>> records. If I turn it in to a normal non-modular jar, it works with >>> both java -jar ?enable-preview myapp.jar, and with a native image >>> built with jpackage. However, when i turn it into a modular jar, I >>> can still launch it with java ?enable-preview ?module-path ... >>> ?module ..., but? when I try to create a native image using jpackage >>> usign the same modular flags, launching it fails. I get a dialog box >>> saying ?Failed to launch JVM?, and if I enable ?win-console when I >>> build it and relaunch it, it tells me this: >>> >>> Exception in thread "main" java.lang.ClassFormatError: Invalid >>> constant pool index 31 for name in Record attribute in class file >>> net/jevring/frequencies/v2/input/KeyTimings$Note >>> ???????? at java.base/java.lang.ClassLoader.defineClass2(Native Method) >>> ???????? at java.base/java.lang.ClassLoader.defineClass(Unknown Source) >>> ???????? at >>> java.base/java.security.SecureClassLoader.defineClass(Unknown Source) >>> ???????? at >>> java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(Unknown >>> Source) >>> ???????? at >>> java.base/jdk.internal.loader.BuiltinClassLoader.findClassInModuleOrNull(Unknown >>> Source) >>> ???????? at >>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(Unknown >>> Source) >>> ???????? at >>> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown >>> Source) >>> ???????? at >>> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown >>> Source) >>> ???????? at java.base/java.lang.ClassLoader.loadClass(Unknown Source) >>> ???????? at >>> frequencies/net.jevring.frequencies.v2.input.KeyTimings.(Unknown >>> Source) >>> ???????? at >>> frequencies/net.jevring.frequencies.v2.Thief.(Unknown Source) >>> ???????? at frequencies/net.jevring.frequencies.v2.Main.main(Unknown >>> Source) >>> >>> Note, above, is a record. I tried moving it out to a separate class, >>> and I still got the same error. When I converted Note to a normal >>> class, it started complaining about the next record it cound. >>> I know that it?s picking up the ?enable-preview java option, because >>> when I omit that, it complains about the class file version being >>> wrong. >>> >>> Since I can run it just fine using ?java -jar?, and it works fine >>> under all other circumstances, I must conclude that there?s >>> something wrong with the intersection of modular jars and jpackage. >>> I?ve googled around plenty, and I haven?t found anything that might >>> help me. I?m not sure if this is a call for help or a bug report, >>> though I?d like to consider it the latter =) >>> >>> I put what I had on a branch: >>> https://bitbucket.org/jevring/frequencies/branch/java-modules >>> I realize it?s not a small self-contained example that replicates >>> the issue, and for that I?m sorry. If someone is interested about >>> this, I could try to make one. > > From sean.mullan at oracle.com Tue Jun 9 15:35:47 2020 From: sean.mullan at oracle.com (Sean Mullan) Date: Tue, 9 Jun 2020 11:35:47 -0400 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> <9013C8D0-33D9-4D09-8953-F8618020A7F7@oracle.com> <8A2C4DEF-FC61-4574-B49D-1133F5349BCE@azul.com> <15fa46f2-0352-a111-a9c7-0d5fc31d9159@oracle.com> Message-ID: <9dc3aa80-cce1-e1c2-d50b-d7cfe570c652@oracle.com> On 6/8/20 5:33 PM, Alexey Bakhtin wrote: > Hello Sean, > > Yes, I think we'll need CSR and release notes as soon as this patch adds new property. > I do not know exact process for it, so I will be grateful if you could explain me exact steps. The CSR process is documented at https://wiki.openjdk.java.net/display/csr/Main. It should be fairly self-explanatory but let me know if you have questions. For the release note, we can tackle that later once the CSR is approved now I have tagged the issue with the "release-note=yes" label so we don't forget it. > This patch was developed to address compatibility issue with new LDAP authentication over SSL/TLS announced by Microsoft [1]. It is not related to RFC 5801. In my opinion ?com.sun.jndi.ldap.tls.cbtype? name looks more suitable for this property and should allow backport it to early JDK versions. Good point about backporting. What RFC or specification defines the format you are using for the channel binding in TlsChannelBinding.java, specifically where the type prefix is encoded as "tls-server-end-point:" followed by the binding data? I have looked through various RFCs but I can't find exactly where this format is defined, so I am wondering if this is a standard encoding or not. Thanks, Sean > > [1] - https://support.microsoft.com/en-au/help/4034879/how-to-add-the-ldapenforcechannelbinding-registry-entry > Regards > Alexey > >> On 8 Jun 2020, at 22:03, Sean Mullan wrote: >> >> (resending to all lists on the review) >> >> I'm just catching up a bit on this review. >> >> Sorry if this has mentioned before, but are you planning to write a CSR and release note? I think this is needed for the com.sun.jndi.ldap.tls.cbtype property. I'm also wondering if this property should be documented in the javadocs, and why it is not a standard property (i.e. "java.naming.ldap.tls.cbtype"). >> >> I was also wondering what relation this has to the "G2" standard SASL mechanisms defined in RFC 5801 [1], and whether that is something we should be using to negotiate this channel binding, and if not, why not. Or if this is something that is implementation-specific and will only work with Microsoft LDAP technology, in which case, we might want to make that more explicit, perhaps by including "microsoft" or something like that in the property name. >> >> Thanks, >> Sean >> >> [1] https://tools.ietf.org/html/rfc5801 >> >> On 6/8/20 9:07 AM, Aleks Efimov wrote: >>> Hi Alexey, >>> I've looked through LdapCtx/LdapClient/SaslBind changes: >>> Do we need to check if CHANNEL_BINDING is set explicitly for all connection types? Maybe we can move the check inside 'if (conn.sock instanceof SSLSocket) {' block. >>> Also, instead of setting CHANNEL_BINDING in context environment and then removing it in finally block, it would be better to clone the environment, put calculated CHANNEL_BINDING into it, and pass the cloned one to Sasl.createSaslClient. >>> Another suggestion about the code that verifies if both properties are set before connection is started: >>> As you've already mentioned the new code in LdapCtx is only needed for checking if timeout is set. Could we try to remove LdapCtx::cbType field and all related methods from LdapCtx (this class is already over-complicated and hard to read) and replace it with some static method in LdapSasl? It will help to localize all changes to LdapSasl except for one line in LdapCtx. >>> I mean something like this: >>> Replace >>> + >>> + // verify LDAP channel binding property >>> + if (cbType != null && connectTimeout == -1) >>> + throw new NamingException(TlsChannelBinding.CHANNEL_BINDING_TYPE + >>> + " property requires " + >>> + CONNECT_TIMEOUT + >>> + " property is set."); >>> With >>> + LdapSasl.checkCbParameters((String)envprops.get(TlsChannelBinding.CHANNEL_BINDING_TYPE), connectTimeout); >>> And add something like that to LdapSasl (or maybe pass the full env here): >>> + public static void checkCbParameters(String cbTypePropertyValue, int connectTimeout) throws NamingException { >>> + TlsChannelBindingType cbType = TlsChannelBinding.parseType(cbTypePropertyValue); >>> + // verify LDAP channel binding property >>> + if (cbType != null && connectTimeout == -1) { >>> + throw new NamingException(TlsChannelBinding.CHANNEL_BINDING_TYPE + >>> + " property requires com.sun.jndi.ldap.connect.timeout" + >>> + " property is set."); >>> + } >>> + } >>> Other LdapCtx/LdapClient/SaslBind changes look fine to me. >>> With Kind Regards, >>> Aleksei >>> On 06/06/2020 20:45, Alexey Bakhtin wrote: >>>> Hello Max, Daniel, >>>> >>>> Thank you for review. >>>> >>>> Please review new version of the patch : >>>> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v5/ >>>> >>>> In this version: >>>> - TlsChannelBinding class is moved into the com.sun.jndi.ldap.sasl package >>>> - SSL Ceritificate related code is moved from LdapClient into the LdapSasl.saslBind method >>>> - verification and removal of internal property is also moved to LdapSasl.saslBind method >>>> - verification of connectTimeout property is moved to LdapCtx.connect. I?ve found that connectionTimeout could be assigned later then cbType >>>> >>>> The test for this issue is not ready yet. I did not find any suitable test case that can be reused. >>>> >>>> Thank you >>>> Alexey >>>> >>>>> On 6 Jun 2020, at 09:44, Weijun Wang wrote: >>>>> >>>>> >>>>> >>>>>> On Jun 6, 2020, at 2:41 PM, Weijun Wang wrote: >>>>>> >>>>>> >>>>>> >>>>>>> On Jun 5, 2020, at 11:03 PM, Alexey Bakhtin wrote: >>>>>>> >>>>>>> Hello Max, >>>>>>> >>>>>>> Thank you a lot for review. >>>>>>> >>>>>>> Could you check the new version of the patch : >>>>>>> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v4/ >>>>>>> >>>>>>> I?ve made the following changes: >>>>>>> - TlsChannelBinding class is moved to java.naming module. >>>>>>> java.security.sasl module is not affected any more >>>>>>> - I pass tlsCB.getData() directly to the SASL mechanism as you suggested >>>>>>> - I?ve made some guards to prevent application from using "com.sun.security.sasl.tlschannelbinding? property directly: >>>>>>> - LdapClient verifies if internal property is not set >>>>>> 245 // Prepare TLS Channel Binding data >>>>>> 246 if (conn.sock instanceof SSLSocket) { >>>>>> 247 // Internal property cannot be set explicitly >>>>>> 248 if (env.get(TlsChannelBinding.CHANNEL_BINDING) != null) { >>>>>> 249 throw new NamingException(TlsChannelBinding.CHANNEL_BINDING + >>>>>> 250 " property cannot be set explicitly"); >>>>>> 251 } >>>>>> >>>>>> If not TLS, this property value be kept there and visible inside the SASL mech. >>>>>> >>>>>>> - GssKrb5Client uses props.remove() to read and remove internal property >>>>> Maybe you can remove the value in LdapClient.java, in case the mech used is not GSSAPI. You can remove it in a finally block after line 303. >>>>> >>>>> --Max >>>>> >>>>>> Traditionally, we use "com.sun..." name which is a JDK supported name (although not at Java SE level), you might want to use a name which is even more internal. >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Max >>>>>> >>>>>> p.s. I see that NTLM also supports ChannelBinding. I'll see if I can improve the NTLM SASL mech to support it. > From amaembo at gmail.com Tue Jun 9 15:48:09 2020 From: amaembo at gmail.com (Tagir Valeev) Date: Tue, 9 Jun 2020 22:48:09 +0700 Subject: Comparator.reversed() type inference issue In-Reply-To: <1402810623.1066241.1591691110477.JavaMail.zimbra@u-pem.fr> References: <35B0D7FE-265D-4525-B344-45DC3BFA2766@gmail.com> <1402810623.1066241.1591691110477.JavaMail.zimbra@u-pem.fr> Message-ID: Hello! > I don't see how you can fix it even with a lot of dust of magic stars, > you have only the type information of the return value of > Map.Entry.comparingByValue().reversed() > > but they can no travel backward to Map.Entry.comparingByValue() because you have no idea of the return type of Map.Entry.comparingByValue() so you don't know the class that contains the method "reversed". Theoretically, one may resolve the subsequent call using the upper bound of the (yet unknown) qualifier type. Here it's Comparator>, so it's possible to resolve reversed() with the T = Map.Entry> constraint. Then we add type parameters of the qualifier type as inference variables to the inference session. The resulting bounds set looks solvable, but I'm pretty sure there are many cases when it will work badly, so many corner cases need to be handled specifically. A lot of magic dust, yes. With best regards, Tagir Valeev. From martinrb at google.com Tue Jun 9 15:51:51 2020 From: martinrb at google.com (Martin Buchholz) Date: Tue, 9 Jun 2020 08:51:51 -0700 Subject: RFR: 8246129: ZIP entries created for DOS epoch include local timezone metadata In-Reply-To: References: Message-ID: not really a review, but cautions: I was afraid that might be hard to fix because DOSTIME_BEFORE_1980 was used as a special value indicating that the real time was < 1980 and stored in an extra field. But I guess we now don't have any code that makes assumptions based only on DOSTIME_BEFORE_1980? Keep in mind that dostime is local date time, and there's always risk in the zip code of confusing time-zoned date time with un-zoned. On Tue, Jun 9, 2020 at 8:24 AM Claes Redestad wrote: > > Hi, > > this patch addresses a corner case where extra time stamp information > is added when using start of the DOS time epoch (1980-01-01 00:00:00) > as a dummy timestamp for entries > > Bug: https://bugs.openjdk.java.net/browse/JDK-8246129 > Webrev: http://cr.openjdk.java.net/~redestad/8246129/open.00/ > > Testing: tier1-2 > > Thanks! > > /Claes From Roger.Riggs at oracle.com Tue Jun 9 15:52:33 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Tue, 9 Jun 2020 11:52:33 -0400 Subject: [15] RFR: 8246662: Test java/time/test/java/time/format/TestUnicodeExtension.java failed on japanese locale. In-Reply-To: <89a24fb8-67e1-9c80-51f2-a6f41ec2c909@oracle.com> References: <89a24fb8-67e1-9c80-51f2-a6f41ec2c909@oracle.com> Message-ID: <0be0926f-b68e-96f7-a806-e7e367559025@oracle.com> Hi Naoto, Since the default locale is being changed even briefly, the test should be run in /othervm. Add:? @run testng/othervm ... DateTimeFormatter:1498: ?? Can the optimization be retained in the case where z.equals(zone)? ?? Move it down to 1508+ and check the zone vs z. Thanks, Roger On 6/8/20 5:06 PM, naoto.sato at oracle.com wrote: > Hello, > > Please review the fix to the following issue: > > https://bugs.openjdk.java.net/browse/JDK-8246662 > > The proposed changeset is located at: > > https://cr.openjdk.java.net/~naoto/8246662/webrev.00/ > > This is a regression caused by the fix to JDK-8244245, where the > locale related fields in the formatter are now overridden by > localizedBy() method. It was only comparing the locale object for the > fast path, but it was not sufficient. > > Naoto From alexey at azul.com Tue Jun 9 15:52:48 2020 From: alexey at azul.com (Alexey Bakhtin) Date: Tue, 9 Jun 2020 15:52:48 +0000 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <9dc3aa80-cce1-e1c2-d50b-d7cfe570c652@oracle.com> References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> <9013C8D0-33D9-4D09-8953-F8618020A7F7@oracle.com> <8A2C4DEF-FC61-4574-B49D-1133F5349BCE@azul.com> <15fa46f2-0352-a111-a9c7-0d5fc31d9159@oracle.com> <9dc3aa80-cce1-e1c2-d50b-d7cfe570c652@oracle.com> Message-ID: <9B6A7339-F729-41EB-8245-CB6EA03A8D31@azul.com> Hello Sean, Thank you for the link. I?ll follow it to create CSR I could not find any clear document or specification for this Channel Binding format. The only document I found that describes this format is the following: https://docs.microsoft.com/en-us/archive/blogs/openspecification/ntlm-and-channel-binding-hash-aka-extended-protection-for-authentication So, it is hard to say - is it a standard or Microsoft implementation specific Regards Alexey > On 9 Jun 2020, at 18:35, Sean Mullan wrote: > > On 6/8/20 5:33 PM, Alexey Bakhtin wrote: >> Hello Sean, >> Yes, I think we'll need CSR and release notes as soon as this patch adds new property. >> I do not know exact process for it, so I will be grateful if you could explain me exact steps. > > The CSR process is documented at https://wiki.openjdk.java.net/display/csr/Main. It should be fairly self-explanatory but let me know if you have questions. > > For the release note, we can tackle that later once the CSR is approved now I have tagged the issue with the "release-note=yes" label so we don't forget it. > >> This patch was developed to address compatibility issue with new LDAP authentication over SSL/TLS announced by Microsoft [1]. It is not related to RFC 5801. In my opinion ?com.sun.jndi.ldap.tls.cbtype? name looks more suitable for this property and should allow backport it to early JDK versions. > > Good point about backporting. > > What RFC or specification defines the format you are using for the channel binding in TlsChannelBinding.java, specifically where the type prefix is encoded as "tls-server-end-point:" followed by the binding data? I have looked through various RFCs but I can't find exactly where this format is defined, so I am wondering if this is a standard encoding or not. > > Thanks, > Sean > >> [1] - https://support.microsoft.com/en-au/help/4034879/how-to-add-the-ldapenforcechannelbinding-registry-entry >> Regards >> Alexey >>> On 8 Jun 2020, at 22:03, Sean Mullan wrote: >>> >>> (resending to all lists on the review) >>> >>> I'm just catching up a bit on this review. >>> >>> Sorry if this has mentioned before, but are you planning to write a CSR and release note? I think this is needed for the com.sun.jndi.ldap.tls.cbtype property. I'm also wondering if this property should be documented in the javadocs, and why it is not a standard property (i.e. "java.naming.ldap.tls.cbtype"). >>> >>> I was also wondering what relation this has to the "G2" standard SASL mechanisms defined in RFC 5801 [1], and whether that is something we should be using to negotiate this channel binding, and if not, why not. Or if this is something that is implementation-specific and will only work with Microsoft LDAP technology, in which case, we might want to make that more explicit, perhaps by including "microsoft" or something like that in the property name. >>> >>> Thanks, >>> Sean >>> >>> [1] https://tools.ietf.org/html/rfc5801 >>> >>> On 6/8/20 9:07 AM, Aleks Efimov wrote: >>>> Hi Alexey, >>>> I've looked through LdapCtx/LdapClient/SaslBind changes: >>>> Do we need to check if CHANNEL_BINDING is set explicitly for all connection types? Maybe we can move the check inside 'if (conn.sock instanceof SSLSocket) {' block. >>>> Also, instead of setting CHANNEL_BINDING in context environment and then removing it in finally block, it would be better to clone the environment, put calculated CHANNEL_BINDING into it, and pass the cloned one to Sasl.createSaslClient. >>>> Another suggestion about the code that verifies if both properties are set before connection is started: >>>> As you've already mentioned the new code in LdapCtx is only needed for checking if timeout is set. Could we try to remove LdapCtx::cbType field and all related methods from LdapCtx (this class is already over-complicated and hard to read) and replace it with some static method in LdapSasl? It will help to localize all changes to LdapSasl except for one line in LdapCtx. >>>> I mean something like this: >>>> Replace >>>> + >>>> + // verify LDAP channel binding property >>>> + if (cbType != null && connectTimeout == -1) >>>> + throw new NamingException(TlsChannelBinding.CHANNEL_BINDING_TYPE + >>>> + " property requires " + >>>> + CONNECT_TIMEOUT + >>>> + " property is set."); >>>> With >>>> + LdapSasl.checkCbParameters((String)envprops.get(TlsChannelBinding.CHANNEL_BINDING_TYPE), connectTimeout); >>>> And add something like that to LdapSasl (or maybe pass the full env here): >>>> + public static void checkCbParameters(String cbTypePropertyValue, int connectTimeout) throws NamingException { >>>> + TlsChannelBindingType cbType = TlsChannelBinding.parseType(cbTypePropertyValue); >>>> + // verify LDAP channel binding property >>>> + if (cbType != null && connectTimeout == -1) { >>>> + throw new NamingException(TlsChannelBinding.CHANNEL_BINDING_TYPE + >>>> + " property requires com.sun.jndi.ldap.connect.timeout" + >>>> + " property is set."); >>>> + } >>>> + } >>>> Other LdapCtx/LdapClient/SaslBind changes look fine to me. >>>> With Kind Regards, >>>> Aleksei >>>> On 06/06/2020 20:45, Alexey Bakhtin wrote: >>>>> Hello Max, Daniel, >>>>> >>>>> Thank you for review. >>>>> >>>>> Please review new version of the patch : >>>>> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v5/ >>>>> >>>>> In this version: >>>>> - TlsChannelBinding class is moved into the com.sun.jndi.ldap.sasl package >>>>> - SSL Ceritificate related code is moved from LdapClient into the LdapSasl.saslBind method >>>>> - verification and removal of internal property is also moved to LdapSasl.saslBind method >>>>> - verification of connectTimeout property is moved to LdapCtx.connect. I?ve found that connectionTimeout could be assigned later then cbType >>>>> >>>>> The test for this issue is not ready yet. I did not find any suitable test case that can be reused. >>>>> >>>>> Thank you >>>>> Alexey >>>>> >>>>>> On 6 Jun 2020, at 09:44, Weijun Wang wrote: >>>>>> >>>>>> >>>>>> >>>>>>> On Jun 6, 2020, at 2:41 PM, Weijun Wang wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>>> On Jun 5, 2020, at 11:03 PM, Alexey Bakhtin wrote: >>>>>>>> >>>>>>>> Hello Max, >>>>>>>> >>>>>>>> Thank you a lot for review. >>>>>>>> >>>>>>>> Could you check the new version of the patch : >>>>>>>> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v4/ >>>>>>>> >>>>>>>> I?ve made the following changes: >>>>>>>> - TlsChannelBinding class is moved to java.naming module. >>>>>>>> java.security.sasl module is not affected any more >>>>>>>> - I pass tlsCB.getData() directly to the SASL mechanism as you suggested >>>>>>>> - I?ve made some guards to prevent application from using "com.sun.security.sasl.tlschannelbinding? property directly: >>>>>>>> - LdapClient verifies if internal property is not set >>>>>>> 245 // Prepare TLS Channel Binding data >>>>>>> 246 if (conn.sock instanceof SSLSocket) { >>>>>>> 247 // Internal property cannot be set explicitly >>>>>>> 248 if (env.get(TlsChannelBinding.CHANNEL_BINDING) != null) { >>>>>>> 249 throw new NamingException(TlsChannelBinding.CHANNEL_BINDING + >>>>>>> 250 " property cannot be set explicitly"); >>>>>>> 251 } >>>>>>> >>>>>>> If not TLS, this property value be kept there and visible inside the SASL mech. >>>>>>> >>>>>>>> - GssKrb5Client uses props.remove() to read and remove internal property >>>>>> Maybe you can remove the value in LdapClient.java, in case the mech used is not GSSAPI. You can remove it in a finally block after line 303. >>>>>> >>>>>> --Max >>>>>> >>>>>>> Traditionally, we use "com.sun..." name which is a JDK supported name (although not at Java SE level), you might want to use a name which is even more internal. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Max >>>>>>> >>>>>>> p.s. I see that NTLM also supports ChannelBinding. I'll see if I can improve the NTLM SASL mech to support it. From mandy.chung at oracle.com Tue Jun 9 16:34:03 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 9 Jun 2020 09:34:03 -0700 Subject: RFR [15] 8247115: Fix typos in java.lang.invoke and java.lang In-Reply-To: <97FD23CC-DE74-4D23-BC42-9CC92CB3CB90@oracle.com> References: <97FD23CC-DE74-4D23-BC42-9CC92CB3CB90@oracle.com> Message-ID: <2dcc2795-78de-2b16-31f7-fe1b2ab9fe12@oracle.com> Looks good. Mandy On 6/9/20 3:42 AM, Pavel Rappo wrote: > Hello, > > Please review the change for https://bugs.openjdk.java.net/browse/JDK-8247115 > > http://cr.openjdk.java.net/~prappo/8247115/webrev.00/ > > The patch fixes 11 typos in doc comments and 2 typos in code. The typos in code are in exception messages. Testing results are pending. > > -Pavel > > From xuelei.fan at oracle.com Tue Jun 9 16:40:33 2020 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Tue, 9 Jun 2020 09:40:33 -0700 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <9B6A7339-F729-41EB-8245-CB6EA03A8D31@azul.com> References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> <9013C8D0-33D9-4D09-8953-F8618020A7F7@oracle.com> <8A2C4DEF-FC61-4574-B49D-1133F5349BCE@azul.com> <15fa46f2-0352-a111-a9c7-0d5fc31d9159@oracle.com> <9dc3aa80-cce1-e1c2-d50b-d7cfe570c652@oracle.com> <9B6A7339-F729-41EB-8245-CB6EA03A8D31@azul.com> Message-ID: <21ab9cf7-2d48-f5de-3586-78b40c1d0c12@oracle.com> About the prefix, it may follow RFC 5056 (See page 7, section 2.1). o Specifications of channel bindings for any secure channels MUST provide for a single, canonical octet string encoding of the channel bindings. Under this framework, channel bindings MUST start with the channel binding unique prefix followed by a colon (ASCII 0x3A). Xuelei On 6/9/2020 8:52 AM, Alexey Bakhtin wrote: > Hello Sean, > > Thank you for the link. I?ll follow it to create CSR > > I could not find any clear document or specification for this Channel Binding format. > The only document I found that describes this format is the following: > https://docs.microsoft.com/en-us/archive/blogs/openspecification/ntlm-and-channel-binding-hash-aka-extended-protection-for-authentication > So, it is hard to say - is it a standard or Microsoft implementation specific > > Regards > Alexey > >> On 9 Jun 2020, at 18:35, Sean Mullan wrote: >> >> On 6/8/20 5:33 PM, Alexey Bakhtin wrote: >>> Hello Sean, >>> Yes, I think we'll need CSR and release notes as soon as this patch adds new property. >>> I do not know exact process for it, so I will be grateful if you could explain me exact steps. >> >> The CSR process is documented at https://wiki.openjdk.java.net/display/csr/Main. It should be fairly self-explanatory but let me know if you have questions. >> >> For the release note, we can tackle that later once the CSR is approved now I have tagged the issue with the "release-note=yes" label so we don't forget it. >> >>> This patch was developed to address compatibility issue with new LDAP authentication over SSL/TLS announced by Microsoft [1]. It is not related to RFC 5801. In my opinion ?com.sun.jndi.ldap.tls.cbtype? name looks more suitable for this property and should allow backport it to early JDK versions. >> >> Good point about backporting. >> >> What RFC or specification defines the format you are using for the channel binding in TlsChannelBinding.java, specifically where the type prefix is encoded as "tls-server-end-point:" followed by the binding data? I have looked through various RFCs but I can't find exactly where this format is defined, so I am wondering if this is a standard encoding or not. >> >> Thanks, >> Sean >> >>> [1] - https://support.microsoft.com/en-au/help/4034879/how-to-add-the-ldapenforcechannelbinding-registry-entry >>> Regards >>> Alexey >>>> On 8 Jun 2020, at 22:03, Sean Mullan wrote: >>>> >>>> (resending to all lists on the review) >>>> >>>> I'm just catching up a bit on this review. >>>> >>>> Sorry if this has mentioned before, but are you planning to write a CSR and release note? I think this is needed for the com.sun.jndi.ldap.tls.cbtype property. I'm also wondering if this property should be documented in the javadocs, and why it is not a standard property (i.e. "java.naming.ldap.tls.cbtype"). >>>> >>>> I was also wondering what relation this has to the "G2" standard SASL mechanisms defined in RFC 5801 [1], and whether that is something we should be using to negotiate this channel binding, and if not, why not. Or if this is something that is implementation-specific and will only work with Microsoft LDAP technology, in which case, we might want to make that more explicit, perhaps by including "microsoft" or something like that in the property name. >>>> >>>> Thanks, >>>> Sean >>>> >>>> [1] https://tools.ietf.org/html/rfc5801 >>>> >>>> On 6/8/20 9:07 AM, Aleks Efimov wrote: >>>>> Hi Alexey, >>>>> I've looked through LdapCtx/LdapClient/SaslBind changes: >>>>> Do we need to check if CHANNEL_BINDING is set explicitly for all connection types? Maybe we can move the check inside 'if (conn.sock instanceof SSLSocket) {' block. >>>>> Also, instead of setting CHANNEL_BINDING in context environment and then removing it in finally block, it would be better to clone the environment, put calculated CHANNEL_BINDING into it, and pass the cloned one to Sasl.createSaslClient. >>>>> Another suggestion about the code that verifies if both properties are set before connection is started: >>>>> As you've already mentioned the new code in LdapCtx is only needed for checking if timeout is set. Could we try to remove LdapCtx::cbType field and all related methods from LdapCtx (this class is already over-complicated and hard to read) and replace it with some static method in LdapSasl? It will help to localize all changes to LdapSasl except for one line in LdapCtx. >>>>> I mean something like this: >>>>> Replace >>>>> + >>>>> + // verify LDAP channel binding property >>>>> + if (cbType != null && connectTimeout == -1) >>>>> + throw new NamingException(TlsChannelBinding.CHANNEL_BINDING_TYPE + >>>>> + " property requires " + >>>>> + CONNECT_TIMEOUT + >>>>> + " property is set."); >>>>> With >>>>> + LdapSasl.checkCbParameters((String)envprops.get(TlsChannelBinding.CHANNEL_BINDING_TYPE), connectTimeout); >>>>> And add something like that to LdapSasl (or maybe pass the full env here): >>>>> + public static void checkCbParameters(String cbTypePropertyValue, int connectTimeout) throws NamingException { >>>>> + TlsChannelBindingType cbType = TlsChannelBinding.parseType(cbTypePropertyValue); >>>>> + // verify LDAP channel binding property >>>>> + if (cbType != null && connectTimeout == -1) { >>>>> + throw new NamingException(TlsChannelBinding.CHANNEL_BINDING_TYPE + >>>>> + " property requires com.sun.jndi.ldap.connect.timeout" + >>>>> + " property is set."); >>>>> + } >>>>> + } >>>>> Other LdapCtx/LdapClient/SaslBind changes look fine to me. >>>>> With Kind Regards, >>>>> Aleksei >>>>> On 06/06/2020 20:45, Alexey Bakhtin wrote: >>>>>> Hello Max, Daniel, >>>>>> >>>>>> Thank you for review. >>>>>> >>>>>> Please review new version of the patch : >>>>>> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v5/ >>>>>> >>>>>> In this version: >>>>>> - TlsChannelBinding class is moved into the com.sun.jndi.ldap.sasl package >>>>>> - SSL Ceritificate related code is moved from LdapClient into the LdapSasl.saslBind method >>>>>> - verification and removal of internal property is also moved to LdapSasl.saslBind method >>>>>> - verification of connectTimeout property is moved to LdapCtx.connect. I?ve found that connectionTimeout could be assigned later then cbType >>>>>> >>>>>> The test for this issue is not ready yet. I did not find any suitable test case that can be reused. >>>>>> >>>>>> Thank you >>>>>> Alexey >>>>>> >>>>>>> On 6 Jun 2020, at 09:44, Weijun Wang wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>>> On Jun 6, 2020, at 2:41 PM, Weijun Wang wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On Jun 5, 2020, at 11:03 PM, Alexey Bakhtin wrote: >>>>>>>>> >>>>>>>>> Hello Max, >>>>>>>>> >>>>>>>>> Thank you a lot for review. >>>>>>>>> >>>>>>>>> Could you check the new version of the patch : >>>>>>>>> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v4/ >>>>>>>>> >>>>>>>>> I?ve made the following changes: >>>>>>>>> - TlsChannelBinding class is moved to java.naming module. >>>>>>>>> java.security.sasl module is not affected any more >>>>>>>>> - I pass tlsCB.getData() directly to the SASL mechanism as you suggested >>>>>>>>> - I?ve made some guards to prevent application from using "com.sun.security.sasl.tlschannelbinding? property directly: >>>>>>>>> - LdapClient verifies if internal property is not set >>>>>>>> 245 // Prepare TLS Channel Binding data >>>>>>>> 246 if (conn.sock instanceof SSLSocket) { >>>>>>>> 247 // Internal property cannot be set explicitly >>>>>>>> 248 if (env.get(TlsChannelBinding.CHANNEL_BINDING) != null) { >>>>>>>> 249 throw new NamingException(TlsChannelBinding.CHANNEL_BINDING + >>>>>>>> 250 " property cannot be set explicitly"); >>>>>>>> 251 } >>>>>>>> >>>>>>>> If not TLS, this property value be kept there and visible inside the SASL mech. >>>>>>>> >>>>>>>>> - GssKrb5Client uses props.remove() to read and remove internal property >>>>>>> Maybe you can remove the value in LdapClient.java, in case the mech used is not GSSAPI. You can remove it in a finally block after line 303. >>>>>>> >>>>>>> --Max >>>>>>> >>>>>>>> Traditionally, we use "com.sun..." name which is a JDK supported name (although not at Java SE level), you might want to use a name which is even more internal. >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Max >>>>>>>> >>>>>>>> p.s. I see that NTLM also supports ChannelBinding. I'll see if I can improve the NTLM SASL mech to support it. > From sean.mullan at oracle.com Tue Jun 9 16:50:05 2020 From: sean.mullan at oracle.com (Sean Mullan) Date: Tue, 9 Jun 2020 12:50:05 -0400 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <21ab9cf7-2d48-f5de-3586-78b40c1d0c12@oracle.com> References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> <9013C8D0-33D9-4D09-8953-F8618020A7F7@oracle.com> <8A2C4DEF-FC61-4574-B49D-1133F5349BCE@azul.com> <15fa46f2-0352-a111-a9c7-0d5fc31d9159@oracle.com> <9dc3aa80-cce1-e1c2-d50b-d7cfe570c652@oracle.com> <9B6A7339-F729-41EB-8245-CB6EA03A8D31@azul.com> <21ab9cf7-2d48-f5de-3586-78b40c1d0c12@oracle.com> Message-ID: <192f1214-fe26-32b5-0243-8d6b90803181@oracle.com> On 6/9/20 12:40 PM, Xuelei Fan wrote: > About the prefix, it may follow RFC 5056 (See page 7, section 2.1). > ?? o? Specifications of channel bindings for any secure channels MUST > ????? provide for a single, canonical octet string encoding of the > ????? channel bindings.? Under this framework, channel bindings MUST > ????? start with the channel binding unique prefix followed by a colon > ????? (ASCII 0x3A). Thanks! Easy to miss. I would recommend adding more comments in the code (ex, in TLSChannelBinding) pointing to that RFC section, and other RFCs such 5929 for the tls cbtypes. Also, this RFC (and other specifications such as RFC 5959) should be listed in the CSR so that we document precisely what encodings and types the JDK implementation supports and is using. --Sean > Xuelei > > > On 6/9/2020 8:52 AM, Alexey Bakhtin wrote: >> Hello Sean, >> >> Thank you for the link. I?ll follow it to create CSR >> >> I could not find any clear document or specification for this Channel >> Binding format. >> The only document I found that describes this format is the following: >> https://docs.microsoft.com/en-us/archive/blogs/openspecification/ntlm-and-channel-binding-hash-aka-extended-protection-for-authentication >> >> So, it is hard to say - is it a standard or Microsoft implementation >> specific >> >> Regards >> Alexey >> >>> On 9 Jun 2020, at 18:35, Sean Mullan wrote: >>> >>> On 6/8/20 5:33 PM, Alexey Bakhtin wrote: >>>> Hello Sean, >>>> Yes, I think we'll need CSR and release notes as soon as this patch >>>> adds new property. >>>> I do not know exact process for it, so I will be grateful if you >>>> could explain me exact steps. >>> >>> The CSR process is documented at >>> https://wiki.openjdk.java.net/display/csr/Main. It should be fairly >>> self-explanatory but let me know if you have questions. >>> >>> For the release note, we can tackle that later once the CSR is >>> approved now I have tagged the issue with the "release-note=yes" >>> label so we don't forget it. >>> >>>> This patch was developed to address compatibility issue with new >>>> LDAP authentication over SSL/TLS announced by Microsoft [1]. It is >>>> not related to RFC 5801. In my opinion >>>> ?com.sun.jndi.ldap.tls.cbtype? name looks more suitable for this >>>> property and should allow backport it to early JDK versions. >>> >>> Good point about backporting. >>> >>> What RFC or specification defines the format you are using for the >>> channel binding in TlsChannelBinding.java, specifically where the >>> type prefix is encoded as "tls-server-end-point:" followed by the >>> binding data? I have looked through various RFCs but I can't find >>> exactly where this format is defined, so I am wondering if this is a >>> standard encoding or not. >>> >>> Thanks, >>> Sean >>> >>>> [1] - >>>> https://support.microsoft.com/en-au/help/4034879/how-to-add-the-ldapenforcechannelbinding-registry-entry >>>> >>>> Regards >>>> Alexey >>>>> On 8 Jun 2020, at 22:03, Sean Mullan wrote: >>>>> >>>>> (resending to all lists on the review) >>>>> >>>>> I'm just catching up a bit on this review. >>>>> >>>>> Sorry if this has mentioned before, but are you planning to write a >>>>> CSR and release note? I think this is needed for the >>>>> com.sun.jndi.ldap.tls.cbtype property. I'm also wondering if this >>>>> property should be documented in the javadocs, and why it is not a >>>>> standard property (i.e. "java.naming.ldap.tls.cbtype"). >>>>> >>>>> I was also wondering what relation this has to the "G2" standard >>>>> SASL mechanisms defined in RFC 5801 [1], and whether that is >>>>> something we should be using to negotiate this channel binding, and >>>>> if not, why not. Or if this is something that is >>>>> implementation-specific and will only work with Microsoft LDAP >>>>> technology, in which case, we might want to make that more >>>>> explicit, perhaps by including "microsoft" or something like that >>>>> in the property name. >>>>> >>>>> Thanks, >>>>> Sean >>>>> >>>>> [1] https://tools.ietf.org/html/rfc5801 >>>>> >>>>> On 6/8/20 9:07 AM, Aleks Efimov wrote: >>>>>> Hi Alexey, >>>>>> I've looked through LdapCtx/LdapClient/SaslBind changes: >>>>>> Do we need to check if CHANNEL_BINDING is set explicitly for all >>>>>> connection types? Maybe we can move the check inside 'if >>>>>> (conn.sock instanceof SSLSocket) {' block. >>>>>> Also, instead of setting CHANNEL_BINDING in context environment >>>>>> and then removing it in finally block, it would be better to clone >>>>>> the environment, put calculated CHANNEL_BINDING into it, and pass >>>>>> the cloned one to Sasl.createSaslClient. >>>>>> Another suggestion about the code that verifies if both properties >>>>>> are set before connection is started: >>>>>> As you've already mentioned the new code in LdapCtx is only needed >>>>>> for checking if timeout is set. Could we try to remove >>>>>> LdapCtx::cbType field and all related methods from LdapCtx (this >>>>>> class is already over-complicated and hard to read) and replace it >>>>>> with some static method in LdapSasl? It will help to localize all >>>>>> changes to LdapSasl except for one line in LdapCtx. >>>>>> I mean something like this: >>>>>> Replace >>>>>> + >>>>>> +??????????? // verify LDAP channel binding property >>>>>> +??????????? if (cbType != null && connectTimeout == -1) >>>>>> +??????????????????? throw new >>>>>> NamingException(TlsChannelBinding.CHANNEL_BINDING_TYPE + >>>>>> +??????????????????????????? " property requires " + >>>>>> +??????????????????????????? CONNECT_TIMEOUT + >>>>>> +??????????????????????????? " property is set."); >>>>>> With >>>>>> + >>>>>> LdapSasl.checkCbParameters((String)envprops.get(TlsChannelBinding.CHANNEL_BINDING_TYPE), >>>>>> connectTimeout); >>>>>> And add something like that to LdapSasl (or maybe pass the full >>>>>> env here): >>>>>> + public static void checkCbParameters(String cbTypePropertyValue, >>>>>> int connectTimeout) throws NamingException { >>>>>> +???? TlsChannelBindingType cbType = >>>>>> TlsChannelBinding.parseType(cbTypePropertyValue); >>>>>> +???? // verify LDAP channel binding property >>>>>> +???? if (cbType != null && connectTimeout == -1) { >>>>>> +???????? throw new >>>>>> NamingException(TlsChannelBinding.CHANNEL_BINDING_TYPE + >>>>>> +???????????????? " property requires >>>>>> com.sun.jndi.ldap.connect.timeout" + >>>>>> +???????????????? " property is set."); >>>>>> +???? } >>>>>> + } >>>>>> Other LdapCtx/LdapClient/SaslBind? changes look fine to me. >>>>>> With Kind Regards, >>>>>> Aleksei >>>>>> On 06/06/2020 20:45, Alexey Bakhtin wrote: >>>>>>> Hello Max, Daniel, >>>>>>> >>>>>>> Thank you for review. >>>>>>> >>>>>>> Please review new version of the patch : >>>>>>> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v5/ >>>>>>> >>>>>>> In this version: >>>>>>> - TlsChannelBinding class is moved into the >>>>>>> com.sun.jndi.ldap.sasl package >>>>>>> - SSL Ceritificate related code is moved from LdapClient? into >>>>>>> the LdapSasl.saslBind method >>>>>>> - verification and removal of internal property is also moved to >>>>>>> LdapSasl.saslBind method >>>>>>> - verification of connectTimeout property is moved to >>>>>>> LdapCtx.connect. I?ve found that connectionTimeout could be >>>>>>> assigned later then cbType >>>>>>> >>>>>>> The test for this issue is not ready yet. I did not find any >>>>>>> suitable test case that can be reused. >>>>>>> >>>>>>> Thank you >>>>>>> Alexey >>>>>>> >>>>>>>> On 6 Jun 2020, at 09:44, Weijun Wang >>>>>>>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On Jun 6, 2020, at 2:41 PM, Weijun Wang >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> On Jun 5, 2020, at 11:03 PM, Alexey Bakhtin >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Hello Max, >>>>>>>>>> >>>>>>>>>> Thank you a lot for review. >>>>>>>>>> >>>>>>>>>> Could you check the new version of the patch : >>>>>>>>>> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v4/ >>>>>>>>>> >>>>>>>>>> I?ve made the following changes: >>>>>>>>>> - TlsChannelBinding class is moved to java.naming module. >>>>>>>>>> java.security.sasl module is not affected any more >>>>>>>>>> - I pass tlsCB.getData() directly to the SASL mechanism as you >>>>>>>>>> suggested >>>>>>>>>> - I?ve made some guards to prevent application from using >>>>>>>>>> "com.sun.security.sasl.tlschannelbinding? property directly: >>>>>>>>>> ???? - LdapClient verifies if internal property is not set >>>>>>>>> 245???????????????????? // Prepare TLS Channel Binding data >>>>>>>>> 246???????????????????? if (conn.sock instanceof SSLSocket) { >>>>>>>>> 247???????????????????????? // Internal property cannot be set >>>>>>>>> explicitly >>>>>>>>> 248???????????????????????? if >>>>>>>>> (env.get(TlsChannelBinding.CHANNEL_BINDING) != null) { >>>>>>>>> 249???????????????????????????? throw new >>>>>>>>> NamingException(TlsChannelBinding.CHANNEL_BINDING + >>>>>>>>> 250???????????????????????????????????? " property cannot be >>>>>>>>> set explicitly"); >>>>>>>>> 251???????????????????????? } >>>>>>>>> >>>>>>>>> If not TLS, this property value be kept there and visible >>>>>>>>> inside the SASL mech. >>>>>>>>> >>>>>>>>>> ???? - GssKrb5Client uses props.remove() to read and remove >>>>>>>>>> internal property >>>>>>>> Maybe you can remove the value in LdapClient.java, in case the >>>>>>>> mech used is not GSSAPI. You can remove it in a finally block >>>>>>>> after line 303. >>>>>>>> >>>>>>>> --Max >>>>>>>> >>>>>>>>> Traditionally, we use "com.sun..." name which is a JDK >>>>>>>>> supported name (although not at Java SE level), you might want >>>>>>>>> to use a name which is even more internal. >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Max >>>>>>>>> >>>>>>>>> p.s. I see that NTLM also supports ChannelBinding. I'll see if >>>>>>>>> I can improve the NTLM SASL mech to support it. >> From naoto.sato at oracle.com Tue Jun 9 17:01:48 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Tue, 9 Jun 2020 10:01:48 -0700 Subject: [15] RFR: 8246662: Test java/time/test/java/time/format/TestUnicodeExtension.java failed on japanese locale. In-Reply-To: <0be0926f-b68e-96f7-a806-e7e367559025@oracle.com> References: <89a24fb8-67e1-9c80-51f2-a6f41ec2c909@oracle.com> <0be0926f-b68e-96f7-a806-e7e367559025@oracle.com> Message-ID: <89ed6849-f290-babe-c5f3-0568b4e1f23c@oracle.com> Hi Roger, Thanks for the review. On 6/9/20 8:52 AM, Roger Riggs wrote: > Hi Naoto, > > Since the default locale is being changed even briefly, the test should > be run in /othervm. > Add:? @run testng/othervm ... All java/time/test/java/time/format tests are run in othervm mode by default, defined in java/time/test/TEST.properties. > > DateTimeFormatter:1498: > ?? Can the optimization be retained in the case where z.equals(zone)? > ?? Move it down to 1508+ and check the zone vs z. It's not only zone, but also has to check chrono and decimalStyle. In the previous version, I intentionally did not keep the optimization, as it would no longer be "fast" since it would need to check those equality, but at least it would save creating the new DateTimeFormatter instance, so I resurrected it. Here is the updated webrev: http://cr.openjdk.java.net/~naoto/8246662/webrev.01/ Naoto > > Thanks, Roger > > > > On 6/8/20 5:06 PM, naoto.sato at oracle.com wrote: >> Hello, >> >> Please review the fix to the following issue: >> >> https://bugs.openjdk.java.net/browse/JDK-8246662 >> >> The proposed changeset is located at: >> >> https://cr.openjdk.java.net/~naoto/8246662/webrev.00/ >> >> This is a regression caused by the fix to JDK-8244245, where the >> locale related fields in the formatter are now overridden by >> localizedBy() method. It was only comparing the locale object for the >> fast path, but it was not sufficient. >> >> Naoto > From claes.redestad at oracle.com Tue Jun 9 17:05:31 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 9 Jun 2020 19:05:31 +0200 Subject: RFR: 8246129: ZIP entries created for DOS epoch include local timezone metadata In-Reply-To: References: Message-ID: <03e6d73e-92d1-c958-1d13-6ca8d9fe843f@oracle.com> Hi, On 2020-06-09 17:51, Martin Buchholz wrote: > not really a review, but cautions: > > I was afraid that might be hard to fix because DOSTIME_BEFORE_1980 was > used as a special value indicating that the real time was < 1980 and > stored in an extra field. But I guess we now don't have any code that > makes assumptions based only on DOSTIME_BEFORE_1980? AFAICT there is no use of this (package-)private constant outside of the logic that use it to avoid adding extra time information in the normal case. I added it as an optimization for the purpose of avoiding to implicitly add the extra time field data, in response to a change that would have done so without any recourse. While I missed the corner case of 1980-01-01 00:00:00 ("Who would ever write entries timestamped as exactly 1980-01-01 00:00:00 in 2015!?"), without this optimization the workarounds in the linked issue would probably not have worked, either. > > Keep in mind that dostime is local date time, and there's always risk > in the zip code of confusing time-zoned date time with un-zoned. Yes, I think the added test covers this: stray outside of 1980-01-01 00:00:00 in the _local_ time and you get extended time fields. Going outside the DOS epoch slightly changes semantics since you'll now implicitly get unix epoch relative timestamps (which adds overhead). One could also opt to get this behavior explicitly by using setLastModifiedTime(..), which gives you a bit higher precision and less weirdness. /Claes > > On Tue, Jun 9, 2020 at 8:24 AM Claes Redestad wrote: >> >> Hi, >> >> this patch addresses a corner case where extra time stamp information >> is added when using start of the DOS time epoch (1980-01-01 00:00:00) >> as a dummy timestamp for entries >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8246129 >> Webrev: http://cr.openjdk.java.net/~redestad/8246129/open.00/ >> >> Testing: tier1-2 >> >> Thanks! >> >> /Claes From daniel.fuchs at oracle.com Tue Jun 9 17:18:43 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 9 Jun 2020 18:18:43 +0100 Subject: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) In-Reply-To: <814DA850-76A9-4526-A3D9-141CD516C9AE@amperemail.onmicrosoft.com> References: <814DA850-76A9-4526-A3D9-141CD516C9AE@amperemail.onmicrosoft.com> Message-ID: <71ea732b-4566-e0a0-b2a3-e71df8a79609@oracle.com> Hi Peter, This is not a review. I second Roger's observations. On 05/06/2020 00:08, Peter Kessler OS wrote: > Tested with `make run-test-tier1` on Linux (CentOS 7) on both aarch64 and x86_64. I had one failure on each of the runs The code you are proposing to change affects RMI and JMX too. For an initial evaluation of the potential damage you will need to test at least with jdk_core and jdk_svc (and if that's successful then probably some of the lower tiers too): $ jtreg -verbose:summary -a -ea -esa -agentvm -conc:8 -ignore:quiet -timeout:2 -exclude:./test/jdk/ProblemList.txt -jdk: ./test/jdk/:jdk_core ./test/jdk/:jdk_svc best regards, -- daniel From lance.andersen at oracle.com Tue Jun 9 17:48:25 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 9 Jun 2020 13:48:25 -0400 Subject: RFR: 8246129: ZIP entries created for DOS epoch include local timezone metadata In-Reply-To: References: Message-ID: <3C8A8034-E136-4A78-A96F-12F2D5310B24@oracle.com> Hi Martin, I believe the fix keeps compatibility with the previous fixes in this area(see 4759491 8015666 and 8073497). If you have an alternative suggestion please let us know. > On Jun 9, 2020, at 11:51 AM, Martin Buchholz wrote: > > not really a review, but cautions: > > I was afraid that might be hard to fix because DOSTIME_BEFORE_1980 was > used as a special value indicating that the real time was < 1980 and > stored in an extra field. But I guess we now don't have any code that > makes assumptions based only on DOSTIME_BEFORE_1980? > Keep in mind that dostime is local date time, and there's always risk > in the zip code of confusing time-zoned date time with un-zoned. Sherman added set/getTimeLocal via JDK-8075526 which can help I believe. Best, Lance > On Tue, Jun 9, 2020 at 8:24 AM Claes Redestad wrote: >> >> Hi, >> >> this patch addresses a corner case where extra time stamp information >> is added when using start of the DOS time epoch (1980-01-01 00:00:00) >> as a dummy timestamp for entries >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8246129 >> Webrev: http://cr.openjdk.java.net/~redestad/8246129/open.00/ >> >> Testing: tier1-2 >> >> Thanks! >> >> /Claes Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From Roger.Riggs at oracle.com Tue Jun 9 18:48:26 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Tue, 9 Jun 2020 14:48:26 -0400 Subject: [15] RFR: 8246662: Test java/time/test/java/time/format/TestUnicodeExtension.java failed on japanese locale. In-Reply-To: <89ed6849-f290-babe-c5f3-0568b4e1f23c@oracle.com> References: <89a24fb8-67e1-9c80-51f2-a6f41ec2c909@oracle.com> <0be0926f-b68e-96f7-a806-e7e367559025@oracle.com> <89ed6849-f290-babe-c5f3-0568b4e1f23c@oracle.com> Message-ID: <9b131dc3-6be8-1c4d-4382-13ad29389179@oracle.com> Hi Naoto, Looks fine. fyi,? Objects.equals(a,b) can replace DateTimeFormatter: 1510-1511. Thanks, Roger On 6/9/20 1:01 PM, naoto.sato at oracle.com wrote: > Hi Roger, > > Thanks for the review. > > On 6/9/20 8:52 AM, Roger Riggs wrote: >> Hi Naoto, >> >> Since the default locale is being changed even briefly, the test >> should be run in /othervm. >> Add:? @run testng/othervm ... > > All java/time/test/java/time/format tests are run in othervm mode by > default, defined in java/time/test/TEST.properties. ok,? non-local control. ugh > >> >> DateTimeFormatter:1498: >> ??? Can the optimization be retained in the case where z.equals(zone)? >> ??? Move it down to 1508+ and check the zone vs z. > > It's not only zone, but also has to check chrono and decimalStyle. In > the previous version, I intentionally did not keep the optimization, > as it would no longer be "fast" since it would need to check those > equality, but at least it would save creating the new > DateTimeFormatter instance, so I resurrected it. Here is the updated > webrev: > > http://cr.openjdk.java.net/~naoto/8246662/webrev.01/ > > Naoto > >> >> Thanks, Roger >> >> >> >> On 6/8/20 5:06 PM, naoto.sato at oracle.com wrote: >>> Hello, >>> >>> Please review the fix to the following issue: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8246662 >>> >>> The proposed changeset is located at: >>> >>> https://cr.openjdk.java.net/~naoto/8246662/webrev.00/ >>> >>> This is a regression caused by the fix to JDK-8244245, where the >>> locale related fields in the formatter are now overridden by >>> localizedBy() method. It was only comparing the locale object for >>> the fast path, but it was not sufficient. >>> >>> Naoto >> From naoto.sato at oracle.com Tue Jun 9 18:52:58 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Tue, 9 Jun 2020 11:52:58 -0700 Subject: [15] RFR: 8246662: Test java/time/test/java/time/format/TestUnicodeExtension.java failed on japanese locale. In-Reply-To: <9b131dc3-6be8-1c4d-4382-13ad29389179@oracle.com> References: <89a24fb8-67e1-9c80-51f2-a6f41ec2c909@oracle.com> <0be0926f-b68e-96f7-a806-e7e367559025@oracle.com> <89ed6849-f290-babe-c5f3-0568b4e1f23c@oracle.com> <9b131dc3-6be8-1c4d-4382-13ad29389179@oracle.com> Message-ID: Thanks! On 6/9/20 11:48 AM, Roger Riggs wrote: > Hi Naoto, > > Looks fine. > > fyi,? Objects.equals(a,b) can replace DateTimeFormatter: 1510-1511. Right. I will replace it before the push. Naoto > > Thanks, Roger > > On 6/9/20 1:01 PM, naoto.sato at oracle.com wrote: >> Hi Roger, >> >> Thanks for the review. >> >> On 6/9/20 8:52 AM, Roger Riggs wrote: >>> Hi Naoto, >>> >>> Since the default locale is being changed even briefly, the test >>> should be run in /othervm. >>> Add:? @run testng/othervm ... >> >> All java/time/test/java/time/format tests are run in othervm mode by >> default, defined in java/time/test/TEST.properties. > ok,? non-local control. ugh >> >>> >>> DateTimeFormatter:1498: >>> ??? Can the optimization be retained in the case where z.equals(zone)? >>> ??? Move it down to 1508+ and check the zone vs z. >> >> It's not only zone, but also has to check chrono and decimalStyle. In >> the previous version, I intentionally did not keep the optimization, >> as it would no longer be "fast" since it would need to check those >> equality, but at least it would save creating the new >> DateTimeFormatter instance, so I resurrected it. Here is the updated >> webrev: > > >> >> http://cr.openjdk.java.net/~naoto/8246662/webrev.01/ >> >> Naoto >> >>> >>> Thanks, Roger >>> >>> >>> >>> On 6/8/20 5:06 PM, naoto.sato at oracle.com wrote: >>>> Hello, >>>> >>>> Please review the fix to the following issue: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8246662 >>>> >>>> The proposed changeset is located at: >>>> >>>> https://cr.openjdk.java.net/~naoto/8246662/webrev.00/ >>>> >>>> This is a regression caused by the fix to JDK-8244245, where the >>>> locale related fields in the formatter are now overridden by >>>> localizedBy() method. It was only comparing the locale object for >>>> the fast path, but it was not sufficient. >>>> >>>> Naoto >>> > From huizhe.wang at oracle.com Tue Jun 9 18:54:43 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 9 Jun 2020 11:54:43 -0700 Subject: [15] RFR: 8246662: Test java/time/test/java/time/format/TestUnicodeExtension.java failed on japanese locale. In-Reply-To: <89ed6849-f290-babe-c5f3-0568b4e1f23c@oracle.com> References: <89a24fb8-67e1-9c80-51f2-a6f41ec2c909@oracle.com> <0be0926f-b68e-96f7-a806-e7e367559025@oracle.com> <89ed6849-f290-babe-c5f3-0568b4e1f23c@oracle.com> Message-ID: <833bd1bb-b19d-3f2b-82b4-2e79d756daa1@oracle.com> On 6/9/2020 10:01 AM, naoto.sato at oracle.com wrote: > Hi Roger, > > Thanks for the review. > > On 6/9/20 8:52 AM, Roger Riggs wrote: >> Hi Naoto, >> >> Since the default locale is being changed even briefly, the test >> should be run in /othervm. >> Add:? @run testng/othervm ... > > All java/time/test/java/time/format tests are run in othervm mode by > default, defined in java/time/test/TEST.properties. > >> >> DateTimeFormatter:1498: >> ??? Can the optimization be retained in the case where z.equals(zone)? >> ??? Move it down to 1508+ and check the zone vs z. > > It's not only zone, but also has to check chrono and decimalStyle. In > the previous version, I intentionally did not keep the optimization, > as it would no longer be "fast" since it would need to check those > equality, but at least it would save creating the new > DateTimeFormatter instance, so I resurrected it. Here is the updated > webrev: > > http://cr.openjdk.java.net/~naoto/8246662/webrev.01/ Looks fine to me for the implementation of the method. It's interesting to observe that the withXXXX methods, as demonstrated in the test, always create a new DateTimeFormatter instance. The Javadoc also said so, e.g. "Returns a copy of this formatter with a new XXXX", and by "a copy" it meant a new instance. Now the localizedBy method also "Returns a copy of this formatter", but it may actually returns this formatter.? Would it confuse users (into thinking they always get a new instance)?? Would we need to amend the javadoc a bit, e.g. indicating it returns this formatter if all of the above aspects are the same? (or otherwise a new instance is returned?) The localizedBy method seems to me it's not "Unlike the withLocale method" in terms of "producing a different formatter". The difference lies in the attributes demonstrated in this changeset, e.g. zone, chrono and decimalStyle will be from the specified locale in localizedBy, while with the withLocale method, they are inherited. In other words, the withXXXX methods only changes the specified field. -Joe > > Naoto > >> >> Thanks, Roger >> >> >> >> On 6/8/20 5:06 PM, naoto.sato at oracle.com wrote: >>> Hello, >>> >>> Please review the fix to the following issue: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8246662 >>> >>> The proposed changeset is located at: >>> >>> https://cr.openjdk.java.net/~naoto/8246662/webrev.00/ >>> >>> This is a regression caused by the fix to JDK-8244245, where the >>> locale related fields in the formatter are now overridden by >>> localizedBy() method. It was only comparing the locale object for >>> the fast path, but it was not sufficient. >>> >>> Naoto >> From naoto.sato at oracle.com Tue Jun 9 19:10:18 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Tue, 9 Jun 2020 12:10:18 -0700 Subject: [15] RFR: 8246662: Test java/time/test/java/time/format/TestUnicodeExtension.java failed on japanese locale. In-Reply-To: <833bd1bb-b19d-3f2b-82b4-2e79d756daa1@oracle.com> References: <89a24fb8-67e1-9c80-51f2-a6f41ec2c909@oracle.com> <0be0926f-b68e-96f7-a806-e7e367559025@oracle.com> <89ed6849-f290-babe-c5f3-0568b4e1f23c@oracle.com> <833bd1bb-b19d-3f2b-82b4-2e79d756daa1@oracle.com> Message-ID: <769ac437-e514-99a7-e420-13a0abe6498f@oracle.com> Hi Joe, On 6/9/20 11:54 AM, Joe Wang wrote: > Looks fine to me for the implementation of the method. Thank you for the review! > > It's interesting to observe that the withXXXX methods, as demonstrated > in the test, always create a new DateTimeFormatter instance. The Javadoc > also said so, e.g. "Returns a copy of this formatter with a new XXXX", > and by "a copy" it meant a new instance. Now the localizedBy method also > "Returns a copy of this formatter", but it may actually returns this > formatter.? Would it confuse users (into thinking they always get a new > instance)?? Would we need to amend the javadoc a bit, e.g. indicating it > returns this formatter if all of the above aspects are the same? (or > otherwise a new instance is returned?) Yes we would, but it's not specific to localizedBy() method but common to all methods. So I would rather follow the same pattern with other methods for this changeset. > > The localizedBy method seems to me it's not "Unlike the withLocale > method" in terms of "producing a different formatter". The difference > lies in the attributes demonstrated in this changeset, e.g. zone, chrono > and decimalStyle will be from the specified locale in localizedBy, while > with the withLocale method, they are inherited. In other words, the > withXXXX methods only changes the specified field. That's exactly why this localzedBy() was introduced. withLocale() only replaces the 'locale' field in the formatter, and no other fields/format patterns, even though they are locale related, are modified. "Unlike ..." means to describe it. Naoto > > -Joe > >> >> Naoto >> >>> >>> Thanks, Roger >>> >>> >>> >>> On 6/8/20 5:06 PM, naoto.sato at oracle.com wrote: >>>> Hello, >>>> >>>> Please review the fix to the following issue: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8246662 >>>> >>>> The proposed changeset is located at: >>>> >>>> https://cr.openjdk.java.net/~naoto/8246662/webrev.00/ >>>> >>>> This is a regression caused by the fix to JDK-8244245, where the >>>> locale related fields in the formatter are now overridden by >>>> localizedBy() method. It was only comparing the locale object for >>>> the fast path, but it was not sufficient. >>>> >>>> Naoto >>> > From Roger.Riggs at oracle.com Tue Jun 9 19:53:09 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Tue, 9 Jun 2020 15:53:09 -0400 Subject: [15] RFR: 8246662: Test java/time/test/java/time/format/TestUnicodeExtension.java failed on japanese locale. In-Reply-To: <769ac437-e514-99a7-e420-13a0abe6498f@oracle.com> References: <89a24fb8-67e1-9c80-51f2-a6f41ec2c909@oracle.com> <0be0926f-b68e-96f7-a806-e7e367559025@oracle.com> <89ed6849-f290-babe-c5f3-0568b4e1f23c@oracle.com> <833bd1bb-b19d-3f2b-82b4-2e79d756daa1@oracle.com> <769ac437-e514-99a7-e420-13a0abe6498f@oracle.com> Message-ID: <43ca15f8-1652-cac4-c12e-b87cb40bf47f@oracle.com> Hi, All java.time instances are immutable and described as value based. Identity based operations are defined as unpredictable and to be avoided. (That specific text was omitted from DateTimeFormatter) The test uses .equals so it is not depending on identity. Roger On 6/9/20 3:10 PM, naoto.sato at oracle.com wrote: > Hi Joe, > > On 6/9/20 11:54 AM, Joe Wang wrote: >> Looks fine to me for the implementation of the method. > > Thank you for the review! > >> >> It's interesting to observe that the withXXXX methods, as >> demonstrated in the test, always create a new DateTimeFormatter >> instance. The Javadoc also said so, e.g. "Returns a copy of this >> formatter with a new XXXX", and by "a copy" it meant a new instance. >> Now the localizedBy method also "Returns a copy of this formatter", >> but it may actually returns this formatter. Would it confuse users >> (into thinking they always get a new instance)?? Would we need to >> amend the javadoc a bit, e.g. indicating it returns this formatter if >> all of the above aspects are the same? (or otherwise a new instance >> is returned?) > > Yes we would, but it's not specific to localizedBy() method but common > to all methods. So I would rather follow the same pattern with other > methods for this changeset. > >> >> The localizedBy method seems to me it's not "Unlike the withLocale >> method" in terms of "producing a different formatter". The difference >> lies in the attributes demonstrated in this changeset, e.g. zone, >> chrono and decimalStyle will be from the specified locale in >> localizedBy, while with the withLocale method, they are inherited. In >> other words, the withXXXX methods only changes the specified field. > > That's exactly why this localzedBy() was introduced. withLocale() only > replaces the 'locale' field in the formatter, and no other > fields/format patterns, even though they are locale related, are > modified. "Unlike ..." means to describe it. > > Naoto > >> >> -Joe >> >>> >>> Naoto >>> >>>> >>>> Thanks, Roger >>>> >>>> >>>> >>>> On 6/8/20 5:06 PM, naoto.sato at oracle.com wrote: >>>>> Hello, >>>>> >>>>> Please review the fix to the following issue: >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8246662 >>>>> >>>>> The proposed changeset is located at: >>>>> >>>>> https://cr.openjdk.java.net/~naoto/8246662/webrev.00/ >>>>> >>>>> This is a regression caused by the fix to JDK-8244245, where the >>>>> locale related fields in the formatter are now overridden by >>>>> localizedBy() method. It was only comparing the locale object for >>>>> the fast path, but it was not sufficient. >>>>> >>>>> Naoto >>>> >> From huizhe.wang at oracle.com Tue Jun 9 20:39:19 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 9 Jun 2020 13:39:19 -0700 Subject: RFR [15/java.xml] 8237456: Transform filtered through SAX filter mishandles character entities In-Reply-To: <6CE020DC-E3BA-4DB4-93FB-21F4689762C8@oracle.com> References: <224e6b23-7999-4a2a-370c-35088a085d1d@oracle.com> <6CE020DC-E3BA-4DB4-93FB-21F4689762C8@oracle.com> Message-ID: Thanks Aleksei, Lance! I'm using NetBeans, but I may get back to Intellij once my new machine arrives :-) -Joe On 6/9/2020 5:41 AM, Lance Andersen wrote: > Hi Joe, > > This looks good. > > For your test, ?Intellij typically suggests using a static import for > Assert: > > ?????? > import static org.testng.Assert.*; > > ????????? > > No need to change but typically which get suggested when you run > Inspect Code > > Best > Lance > >> On Jun 9, 2020, at 1:54 AM, Joe Wang > > wrote: >> >> Hi, >> >> Please review a fix to a regression that resulted in invalid output >> in a transform that contains entity references. The regression was >> caused by a JDK9 patch [1]. Adding back the conditional checks fixed >> the issue. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8237456 >> webrevs: http://cr.openjdk.java.net/~joehw/jdk15/8237456/webrev/ >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8087303 >> >> Thanks, >> Joe >> > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle?Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From alexey at azul.com Tue Jun 9 21:03:02 2020 From: alexey at azul.com (Alexey Bakhtin) Date: Tue, 9 Jun 2020 21:03:02 +0000 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <6167ea24-e7f9-c259-e2c5-8be96d74a40d@oracle.com> References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> <9013C8D0-33D9-4D09-8953-F8618020A7F7@oracle.com> <8A2C4DEF-FC61-4574-B49D-1133F5349BCE@azul.com> <15fa46f2-0352-a111-a9c7-0d5fc31d9159@oracle.com> <6167ea24-e7f9-c259-e2c5-8be96d74a40d@oracle.com> Message-ID: <228BA21D-836A-46D7-B273-738AFFE639E4@azul.com> Hello Aleks, Thank you very much for review. I?ve fixed missed spaces and removed casting from LdapSasl.java Failure of the SaslMutual test was caused by prop.remove() in the GssKrb5Client This operation is not required any more. GssKrb5Client receives temporary copy of the properties. Fixed Also, I?ve added references to RFC-5929 and RFC-5056 into the TlsChannelBinding class Updated patch is located at: http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v7/ Regards Alexey > On 9 Jun 2020, at 15:59, Aleks Efimov wrote: > > Hi Alexey, > > Thank you for incorporating LdapCtx and LdapSasl changes. I've reviewed both classes and they look good to me, with few minor comments in LdapSasl.java: > missing spaces in the following lines: 78, 152 > With your last changes we can remove explicit cast of 'envProps' on line 176 > > I've also run your changes through our CI and one test is failing due to the changes in GssKrb5Client: > The failed test name: sun/security/krb5/auto/SaslMutual.java > > The observed failure: > java.lang.UnsupportedOperationException > at java.base/java.util.ImmutableCollections.uoe(ImmutableCollections.java:127) > at java.base/java.util.ImmutableCollections$AbstractImmutableMap.remove(ImmutableCollections.java:910) > at jdk.security.jgss/com.sun.security.sasl.gsskerb.GssKrb5Client.(GssKrb5Client.java:156) > at jdk.security.jgss/com.sun.security.sasl.gsskerb.FactoryImpl.createSaslClient(FactoryImpl.java:63) > at java.security.sasl/javax.security.sasl.Sasl.createSaslClient(Sasl.java:433) > at SaslMutual.main(SaslMutual.java:50) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64) > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:564) > at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127) > at java.base/java.lang.Thread.run(Thread.java:832) > > > For information about CSR process you can start from this wiki page: https://wiki.openjdk.java.net/display/csr > > Best Regards, > Aleksei > > On 08/06/2020 22:33, Alexey Bakhtin wrote: >> Hello Sean, >> >> Yes, I think we'll need CSR and release notes as soon as this patch adds new property. >> I do not know exact process for it, so I will be grateful if you could explain me exact steps. >> >> This patch was developed to address compatibility issue with new LDAP authentication over SSL/TLS announced by Microsoft [1]. It is not related to RFC 5801. In my opinion ?com.sun.jndi.ldap.tls.cbtype? name looks more suitable for this property and should allow backport it to early JDK versions. >> >> [1] - >> https://support.microsoft.com/en-au/help/4034879/how-to-add-the-ldapenforcechannelbinding-registry-entry >> >> Regards >> Alexey >> >> >>> On 8 Jun 2020, at 22:03, Sean Mullan >>> wrote: >>> >>> (resending to all lists on the review) >>> >>> I'm just catching up a bit on this review. >>> >>> Sorry if this has mentioned before, but are you planning to write a CSR and release note? I think this is needed for the com.sun.jndi.ldap.tls.cbtype property. I'm also wondering if this property should be documented in the javadocs, and why it is not a standard property (i.e. "java.naming.ldap.tls.cbtype"). >>> >>> I was also wondering what relation this has to the "G2" standard SASL mechanisms defined in RFC 5801 [1], and whether that is something we should be using to negotiate this channel binding, and if not, why not. Or if this is something that is implementation-specific and will only work with Microsoft LDAP technology, in which case, we might want to make that more explicit, perhaps by including "microsoft" or something like that in the property name. >>> >>> Thanks, >>> Sean >>> >>> [1] >>> https://tools.ietf.org/html/rfc5801 >>> >>> >>> On 6/8/20 9:07 AM, Aleks Efimov wrote: >>> >>>> Hi Alexey, >>>> I've looked through LdapCtx/LdapClient/SaslBind changes: >>>> Do we need to check if CHANNEL_BINDING is set explicitly for all connection types? Maybe we can move the check inside 'if (conn.sock instanceof SSLSocket) {' block. >>>> Also, instead of setting CHANNEL_BINDING in context environment and then removing it in finally block, it would be better to clone the environment, put calculated CHANNEL_BINDING into it, and pass the cloned one to Sasl.createSaslClient. >>>> Another suggestion about the code that verifies if both properties are set before connection is started: >>>> As you've already mentioned the new code in LdapCtx is only needed for checking if timeout is set. Could we try to remove LdapCtx::cbType field and all related methods from LdapCtx (this class is already over-complicated and hard to read) and replace it with some static method in LdapSasl? It will help to localize all changes to LdapSasl except for one line in LdapCtx. >>>> I mean something like this: >>>> Replace >>>> + >>>> + // verify LDAP channel binding property >>>> + if (cbType != null && connectTimeout == -1) >>>> + throw new NamingException(TlsChannelBinding.CHANNEL_BINDING_TYPE + >>>> + " property requires " + >>>> + CONNECT_TIMEOUT + >>>> + " property is set."); >>>> With >>>> + LdapSasl.checkCbParameters((String)envprops.get(TlsChannelBinding.CHANNEL_BINDING_TYPE), connectTimeout); >>>> And add something like that to LdapSasl (or maybe pass the full env here): >>>> + public static void checkCbParameters(String cbTypePropertyValue, int connectTimeout) throws NamingException { >>>> + TlsChannelBindingType cbType = TlsChannelBinding.parseType(cbTypePropertyValue); >>>> + // verify LDAP channel binding property >>>> + if (cbType != null && connectTimeout == -1) { >>>> + throw new NamingException(TlsChannelBinding.CHANNEL_BINDING_TYPE + >>>> + " property requires com.sun.jndi.ldap.connect.timeout" + >>>> + " property is set."); >>>> + } >>>> + } >>>> Other LdapCtx/LdapClient/SaslBind changes look fine to me. >>>> With Kind Regards, >>>> Aleksei >>>> On 06/06/2020 20:45, Alexey Bakhtin wrote: >>>> >>>>> Hello Max, Daniel, >>>>> >>>>> Thank you for review. >>>>> >>>>> Please review new version of the patch : >>>>> >>>>> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v5/ >>>>> >>>>> >>>>> In this version: >>>>> - TlsChannelBinding class is moved into the com.sun.jndi.ldap.sasl package >>>>> - SSL Ceritificate related code is moved from LdapClient into the LdapSasl.saslBind method >>>>> - verification and removal of internal property is also moved to LdapSasl.saslBind method >>>>> - verification of connectTimeout property is moved to LdapCtx.connect. I?ve found that connectionTimeout could be assigned later then cbType >>>>> >>>>> The test for this issue is not ready yet. I did not find any suitable test case that can be reused. >>>>> >>>>> Thank you >>>>> Alexey >>>>> >>>>> >>>>>> On 6 Jun 2020, at 09:44, Weijun Wang >>>>>> wrote: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> On Jun 6, 2020, at 2:41 PM, Weijun Wang >>>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> On Jun 5, 2020, at 11:03 PM, Alexey Bakhtin >>>>>>>> wrote: >>>>>>>> >>>>>>>> Hello Max, >>>>>>>> >>>>>>>> Thank you a lot for review. >>>>>>>> >>>>>>>> Could you check the new version of the patch : >>>>>>>> >>>>>>>> http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v4/ >>>>>>>> >>>>>>>> >>>>>>>> I?ve made the following changes: >>>>>>>> - TlsChannelBinding class is moved to java.naming module. >>>>>>>> java.security.sasl module is not affected any more >>>>>>>> - I pass tlsCB.getData() directly to the SASL mechanism as you suggested >>>>>>>> - I?ve made some guards to prevent application from using "com.sun.security.sasl.tlschannelbinding? property directly: >>>>>>>> - LdapClient verifies if internal property is not set >>>>>>>> >>>>>>> 245 // Prepare TLS Channel Binding data >>>>>>> 246 if (conn.sock instanceof SSLSocket) { >>>>>>> 247 // Internal property cannot be set explicitly >>>>>>> 248 if (env.get(TlsChannelBinding.CHANNEL_BINDING) != null) { >>>>>>> 249 throw new NamingException(TlsChannelBinding.CHANNEL_BINDING + >>>>>>> 250 " property cannot be set explicitly"); >>>>>>> 251 } >>>>>>> >>>>>>> If not TLS, this property value be kept there and visible inside the SASL mech. >>>>>>> >>>>>>> >>>>>>>> - GssKrb5Client uses props.remove() to read and remove internal property >>>>>>>> >>>>>> Maybe you can remove the value in LdapClient.java, in case the mech used is not GSSAPI. You can remove it in a finally block after line 303. >>>>>> >>>>>> --Max >>>>>> >>>>>> >>>>>>> Traditionally, we use "com.sun..." name which is a JDK supported name (although not at Java SE level), you might want to use a name which is even more internal. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Max >>>>>>> >>>>>>> p.s. I see that NTLM also supports ChannelBinding. I'll see if I can improve the NTLM SASL mech to support it. >>>>>>> > From huizhe.wang at oracle.com Tue Jun 9 21:35:14 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Tue, 9 Jun 2020 14:35:14 -0700 Subject: [15] RFR: 8246662: Test java/time/test/java/time/format/TestUnicodeExtension.java failed on japanese locale. In-Reply-To: <43ca15f8-1652-cac4-c12e-b87cb40bf47f@oracle.com> References: <89a24fb8-67e1-9c80-51f2-a6f41ec2c909@oracle.com> <0be0926f-b68e-96f7-a806-e7e367559025@oracle.com> <89ed6849-f290-babe-c5f3-0568b4e1f23c@oracle.com> <833bd1bb-b19d-3f2b-82b4-2e79d756daa1@oracle.com> <769ac437-e514-99a7-e420-13a0abe6498f@oracle.com> <43ca15f8-1652-cac4-c12e-b87cb40bf47f@oracle.com> Message-ID: Thanks Roger, Naoto. On 6/9/2020 12:53 PM, Roger Riggs wrote: > Hi, > > All java.time instances are immutable and described as value based. > Identity based operations are defined as unpredictable and to be avoided. > (That specific text was omitted from DateTimeFormatter) > > The test uses .equals so it is not depending on identity. > > Roger > > > On 6/9/20 3:10 PM, naoto.sato at oracle.com wrote: >> Hi Joe, >> >> On 6/9/20 11:54 AM, Joe Wang wrote: >>> Looks fine to me for the implementation of the method. >> >> Thank you for the review! >> >>> >>> It's interesting to observe that the withXXXX methods, as >>> demonstrated in the test, always create a new DateTimeFormatter >>> instance. The Javadoc also said so, e.g. "Returns a copy of this >>> formatter with a new XXXX", and by "a copy" it meant a new instance. >>> Now the localizedBy method also "Returns a copy of this formatter", >>> but it may actually returns this formatter. Would it confuse users >>> (into thinking they always get a new instance)?? Would we need to >>> amend the javadoc a bit, e.g. indicating it returns this formatter >>> if all of the above aspects are the same? (or otherwise a new >>> instance is returned?) >> >> Yes we would, but it's not specific to localizedBy() method but >> common to all methods. So I would rather follow the same pattern with >> other methods for this changeset. >> >>> >>> The localizedBy method seems to me it's not "Unlike the withLocale >>> method" in terms of "producing a different formatter". The >>> difference lies in the attributes demonstrated in this changeset, >>> e.g. zone, chrono and decimalStyle will be from the specified locale >>> in localizedBy, while with the withLocale method, they are >>> inherited. In other words, the withXXXX methods only changes the >>> specified field. >> >> That's exactly why this localzedBy() was introduced. withLocale() >> only replaces the 'locale' field in the formatter, and no other >> fields/format patterns, even though they are locale related, are >> modified. "Unlike ..." means to describe it. >> >> Naoto >> >>> >>> -Joe >>> >>>> >>>> Naoto >>>> >>>>> >>>>> Thanks, Roger >>>>> >>>>> >>>>> >>>>> On 6/8/20 5:06 PM, naoto.sato at oracle.com wrote: >>>>>> Hello, >>>>>> >>>>>> Please review the fix to the following issue: >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8246662 >>>>>> >>>>>> The proposed changeset is located at: >>>>>> >>>>>> https://cr.openjdk.java.net/~naoto/8246662/webrev.00/ >>>>>> >>>>>> This is a regression caused by the fix to JDK-8244245, where the >>>>>> locale related fields in the formatter are now overridden by >>>>>> localizedBy() method. It was only comparing the locale object for >>>>>> the fast path, but it was not sufficient. >>>>>> >>>>>> Naoto >>>>> >>> > From lsuresh at vmware.com Tue Jun 9 22:07:53 2020 From: lsuresh at vmware.com (Lalith Suresh) Date: Tue, 9 Jun 2020 22:07:53 +0000 Subject: ConcurrentSkipListMap design question Message-ID: Hi all, I?ve been trying to understand ConcurrentSkipListMap?s design, and noticed this comment in the source [1]: ?This class implements a tree-like two-dimensionally linked skip list in which the index levels are represented in separate nodes from the base nodes holding data. There are two reasons for taking this approach instead of the usual array-based structure: 1) Array based implementations seem to encounter more complexity and overhead 2) We can use cheaper algorithms for the heavily-traversed index lists than can be used for the base lists. ? I?m curious: what were the specific overheads involved with using an array-based structure? Would these overheads still hold today? I?ve tried to dig around and wasn?t able to find any design documents or discussions that elaborate on these two points, so any pointers to these details would be welcome. Thank you! Cheers, Lalith [1] https://github.com/openjdk/jdk/blob/d8c6516c921e1f437c175875c3157ee249a5ca3c/src/java.base/share/classes/java/util/concurrent/ConcurrentSkipListMap.java#L119 From alexander.matveev at oracle.com Tue Jun 9 22:12:07 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Tue, 9 Jun 2020 15:12:07 -0700 Subject: RFR: 8233215: jpackage doesn't allow enough flexibility for file type binding In-Reply-To: <5311628d-3179-b044-81fa-6e14bbb71b8d@oracle.com> References: <502c6e3c-0f49-c895-02b7-bfad8942a9db@oracle.com> <5311628d-3179-b044-81fa-6e14bbb71b8d@oracle.com> Message-ID: Hi Andy, http://cr.openjdk.java.net/~almatvee/8233215/webrev.02/ Moved new mac specific parameters to MacAppImageBuilder. Thanks, Alexander On 6/9/20 4:41 AM, Andy Herrick wrote: > I would think the new mac specific BundlerParamInfo would be defined > in MacAppImageBuilder instead of StandardBundlerParam*. > * > > Am I missing something ? > > /Andy > ** > > * > * > > On 6/9/2020 1:12 AM, alexander.matveev at oracle.com wrote: >> Please review the jpackage fix for bug [1] at [2]. >> >> Added support for additional (macOS specific) file association >> properties. These additional properties should be set via same >> property file which used for file association. Names and values for >> properties should match name and values as documented by Apple in >> https://developer.apple.com/documentation/bundleresources/information_property_list/cfbundledocumenttypes. >> jpackage will simple propagate these values to Info.plist as is. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8233215 >> [2] http://cr.openjdk.java.net/~almatvee/8233215/webrev.00/ >> >> Thanks, >> Alexander From sean.mullan at oracle.com Tue Jun 9 22:21:50 2020 From: sean.mullan at oracle.com (Sean Mullan) Date: Tue, 9 Jun 2020 18:21:50 -0400 Subject: Thread leak by LdapLoginModule In-Reply-To: <889628466.1257399.1591737319316.JavaMail.zimbra@desy.de> References: <889628466.1257399.1591737319316.JavaMail.zimbra@desy.de> Message-ID: <5e5f3f91-bdc2-7af2-7b41-894484de7f61@oracle.com> Adding core-libs-dev ... --Sean On 6/9/20 5:15 PM, Mkrtchyan, Tigran wrote: > > Hi all, > > with Java-11 we have notice a thread leak with ldap module. > We use LDAP to authenticate users with username+pasword by > directly calling LdapLoginModule. This was ok with java 7 and > java 8. With java 11 we see threads getting accumulated. here is a > test case that demonstrates it: > > ``` > > private static final String USERNAME_KEY = "javax.security.auth.login.name"; > private static final String PASSWORD_KEY = "javax.security.auth.login.password"; > > String ldapUrl = "ldap://...."; > String peopleOU = "ou= ... o= ... c=..."); > > String user = ...; > String pass = ...; > > > @Test > public void threadLeakTest() throws AuthenticationException, NoSuchPrincipalException, LoginException { > > Map threadsBefore = Thread.getAllStackTraces(); > > Map globalLoginOptions = Map.of( > "userProvider", ldapUrl + "/" + peopleOU, > "useSSL", "false", > "userFilter", "(uid={USERNAME})", > "useFirstPass", "true" > ); > > for (int i = 0; i < 10; i++) { > > Map loginOptions = Map.of( > USERNAME_KEY, user, > PASSWORD_KEY, pass.toCharArray()); > > Subject subject = new Subject(); > > LdapLoginModule loginModule = new LdapLoginModule(); > loginModule.initialize(subject, null, loginOptions, globalLoginOptions); > loginModule.login(); > loginModule.commit(); > loginModule.logout(); > } > > Map threadsAfter = Thread.getAllStackTraces(); > > assertEquals("Thread leak detected", threadsBefore.size() + 1, threadsAfter.size()); > } > > ``` > > The thread count difference is always equals to the number of iterations in the loop, e.g. on each call a > thread is created and stays around. Eventually our server crashes with: > > [19497.011s][warning][os,thread] Attempt to protect stack guard pages failed (0x00007fcc4c65c000-0x00007fcc4c660000). > OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00007fcc4c55b000, 16384, 0) failed; error='Not enough space' (errno=12) > > The issue is not observed with java-14, thus I assume that the fix is related to commit > > http://hg.openjdk.java.net/jdk/jdk/rev/6717d7e59db4 > > As java-11 is LTS, what is the procedure to get it fix back-ported? > > Regards, > Tigran. > From andy.herrick at oracle.com Tue Jun 9 22:33:59 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Tue, 9 Jun 2020 18:33:59 -0400 Subject: RFR: 8233215: jpackage doesn't allow enough flexibility for file type binding In-Reply-To: References: <502c6e3c-0f49-c895-02b7-bfad8942a9db@oracle.com> <5311628d-3179-b044-81fa-6e14bbb71b8d@oracle.com> Message-ID: looks good /Andy On 6/9/2020 6:12 PM, alexander.matveev at oracle.com wrote: > Hi Andy, > > http://cr.openjdk.java.net/~almatvee/8233215/webrev.02/ > > Moved new mac specific parameters to MacAppImageBuilder. > > Thanks, > Alexander > > On 6/9/20 4:41 AM, Andy Herrick wrote: >> I would think the new mac specific BundlerParamInfo would be defined >> in MacAppImageBuilder instead of StandardBundlerParam*. >> * >> >> Am I missing something ? >> >> /Andy >> ** >> >> * >> * >> >> On 6/9/2020 1:12 AM, alexander.matveev at oracle.com wrote: >>> Please review the jpackage fix for bug [1] at [2]. >>> >>> Added support for additional (macOS specific) file association >>> properties. These additional properties should be set via same >>> property file which used for file association. Names and values for >>> properties should match name and values as documented by Apple in >>> https://developer.apple.com/documentation/bundleresources/information_property_list/cfbundledocumenttypes. >>> jpackage will simple propagate these values to Info.plist as is. >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8233215 >>> [2] http://cr.openjdk.java.net/~almatvee/8233215/webrev.00/ >>> >>> Thanks, >>> Alexander > From Roger.Riggs at oracle.com Tue Jun 9 22:47:58 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Tue, 9 Jun 2020 18:47:58 -0400 Subject: RFR 8247274: (test) HexPrinter cleanup Message-ID: <5cdc15f8-37ac-db05-3e94-84b8930eef16@oracle.com> Please review cleanup to the test support for jdk.test.lib.hexprinter.HexPrinter. Correct issues with the test support for jdk.test.lib.hexdump.HexPrinter. ?- Correct the swapped definitions of Formatters.ASCII and PRINTABLE ???Printable should always return a single character, ???ASCII provides mnemonics to be consistent with the command line hexdump. - For the methods to format ByteBuffers align the descriptions of the index and length to match ByteBuffer. ?- Correctly implement the extraction of bytes from the ByteBuffer ?- a newline in the annotation should print as '\n' instead of causing a line break Webrev: ??? http://cr.openjdk.java.net/~rriggs/webrev-hexprinter-8247274-1/ Issue: ?? https://bugs.openjdk.java.net/browse/JDK-8247274 Thanks, Roger From alexey.semenyuk at oracle.com Tue Jun 9 23:05:59 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Tue, 9 Jun 2020 19:05:59 -0400 Subject: RFR: 8233215: jpackage doesn't allow enough flexibility for file type binding In-Reply-To: References: <502c6e3c-0f49-c895-02b7-bfad8942a9db@oracle.com> <5311628d-3179-b044-81fa-6e14bbb71b8d@oracle.com> Message-ID: <0fa5e0c3-a84e-81d1-c6de-3af3f2b1daf9@oracle.com> +1 - Alexey On 6/9/2020 6:33 PM, Andy Herrick wrote: > looks good > > /Andy > > On 6/9/2020 6:12 PM, alexander.matveev at oracle.com wrote: >> Hi Andy, >> >> http://cr.openjdk.java.net/~almatvee/8233215/webrev.02/ >> >> Moved new mac specific parameters to MacAppImageBuilder. >> >> Thanks, >> Alexander >> >> On 6/9/20 4:41 AM, Andy Herrick wrote: >>> I would think the new mac specific BundlerParamInfo would be defined >>> in MacAppImageBuilder instead of StandardBundlerParam*. >>> * >>> >>> Am I missing something ? >>> >>> /Andy >>> ** >>> >>> * >>> * >>> >>> On 6/9/2020 1:12 AM, alexander.matveev at oracle.com wrote: >>>> Please review the jpackage fix for bug [1] at [2]. >>>> >>>> Added support for additional (macOS specific) file association >>>> properties. These additional properties should be set via same >>>> property file which used for file association. Names and values for >>>> properties should match name and values as documented by Apple in >>>> https://developer.apple.com/documentation/bundleresources/information_property_list/cfbundledocumenttypes. >>>> jpackage will simple propagate these values to Info.plist as is. >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8233215 >>>> [2] http://cr.openjdk.java.net/~almatvee/8233215/webrev.00/ >>>> >>>> Thanks, >>>> Alexander >> From igor.ignatyev at oracle.com Tue Jun 9 23:47:51 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 9 Jun 2020 16:47:51 -0700 Subject: RFR(S) : 8183040 : update jdk/test/lib/Platform.java to use NIO file API Message-ID: <8CA6F4A9-7B57-4B70-9087-D1ACBF561714@oracle.com> http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 > > 38 lines changed: 8 ins; 16 del; 14 mod; Hi all, could you please review this small clean up of testlibrary classes which updates j.t.lib.Platform and j.t.l.SA.SATestUtils (as it now contains the methods which 8183040 was about) to use NIO file API? testing: test/hotspot/jtreg/serviceability webrev: http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 JBS: https://bugs.openjdk.java.net/browse/JDK-8183040 Thanks, -- Igor From brian.burkhalter at oracle.com Wed Jun 10 00:18:38 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 9 Jun 2020 17:18:38 -0700 Subject: RFR(S) : 8183040 : update jdk/test/lib/Platform.java to use NIO file API In-Reply-To: <8CA6F4A9-7B57-4B70-9087-D1ACBF561714@oracle.com> References: <8CA6F4A9-7B57-4B70-9087-D1ACBF561714@oracle.com> Message-ID: <2AF18884-0407-455C-9BE2-4355D3D9AD6C@oracle.com> Hi Igor, > On Jun 9, 2020, at 4:47 PM, Igor Ignatyev wrote: > > could you please review this small clean up of testlibrary classes which updates j.t.lib.Platform and j.t.l.SA.SATestUtils (as it now contains the methods which 8183040 was about) to use NIO file API? > > testing: test/hotspot/jtreg/serviceability > webrev: http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 > JBS: https://bugs.openjdk.java.net/browse/JDK-8183040 The NIO changes look all right to me. Brian From alexey.menkov at oracle.com Wed Jun 10 01:36:11 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Tue, 9 Jun 2020 18:36:11 -0700 Subject: RFR(S) : 8183040 : update jdk/test/lib/Platform.java to use NIO file API In-Reply-To: <8CA6F4A9-7B57-4B70-9087-D1ACBF561714@oracle.com> References: <8CA6F4A9-7B57-4B70-9087-D1ACBF561714@oracle.com> Message-ID: Hi Igor, In SATestUtils.java you do var bb = ... Files.readAllBytes(...) ... and then use bb[0] if the file has 0 length, old code throws EOFException and new one will throw IndexOutOfBoundsException. And looks like the caller doesn't expect it (it catches IOException). --alex On 06/09/2020 16:47, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >> >> 38 lines changed: 8 ins; 16 del; 14 mod; > > Hi all, > > could you please review this small clean up of testlibrary classes which updates j.t.lib.Platform and j.t.l.SA.SATestUtils (as it now contains the methods which 8183040 was about) to use NIO file API? > > testing: test/hotspot/jtreg/serviceability > webrev: http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 > JBS: https://bugs.openjdk.java.net/browse/JDK-8183040 > > Thanks, > -- Igor > From igor.ignatyev at oracle.com Wed Jun 10 03:11:48 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 9 Jun 2020 20:11:48 -0700 Subject: RFR(S) : 8183040 : update jdk/test/lib/Platform.java to use NIO file API In-Reply-To: References: <8CA6F4A9-7B57-4B70-9087-D1ACBF561714@oracle.com> Message-ID: Hi Alex, as far as I can see, the caller just rethrows IOException as RuntimeException, so I don't think throwing IndexOutOfBoundsException would be much different, albeit it will be a bit more cryptic. yet given the content of /proc/sys/kernel/yama/ptrace_scope and /sys/fs/selinux/booleans/deny_ptrace is part of linux kernel contract, I doubt we will encounter IIOOBE in any reasonable setups. however, if you want I can check the length of bb arrays at L#171 and L#190 and throw an Error w/ message suggesting that something went completely wrong. -- Igor > On Jun 9, 2020, at 6:36 PM, Alex Menkov wrote: > > Hi Igor, > > In SATestUtils.java you do > > var bb = ... Files.readAllBytes(...) ... > and then use bb[0] > > if the file has 0 length, old code throws EOFException and new one will throw IndexOutOfBoundsException. > And looks like the caller doesn't expect it (it catches IOException). > > --alex > > On 06/09/2020 16:47, Igor Ignatyev wrote: >> http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >>> >>> 38 lines changed: 8 ins; 16 del; 14 mod; >> Hi all, >> could you please review this small clean up of testlibrary classes which updates j.t.lib.Platform and j.t.l.SA.SATestUtils (as it now contains the methods which 8183040 was about) to use NIO file API? >> testing: test/hotspot/jtreg/serviceability >> webrev: http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >> JBS: https://bugs.openjdk.java.net/browse/JDK-8183040 >> Thanks, >> -- Igor From igor.ignatyev at oracle.com Wed Jun 10 03:17:20 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 9 Jun 2020 20:17:20 -0700 Subject: RFR(S) : 8183040 : update jdk/test/lib/Platform.java to use NIO file API In-Reply-To: <2AF18884-0407-455C-9BE2-4355D3D9AD6C@oracle.com> References: <8CA6F4A9-7B57-4B70-9087-D1ACBF561714@oracle.com> <2AF18884-0407-455C-9BE2-4355D3D9AD6C@oracle.com> Message-ID: Hi Brian, thank you for your review. Cheers, -- Igor > On Jun 9, 2020, at 5:18 PM, Brian Burkhalter wrote: > > Hi Igor, > >> On Jun 9, 2020, at 4:47 PM, Igor Ignatyev > wrote: >> >> could you please review this small clean up of testlibrary classes which updates j.t.lib.Platform and j.t.l.SA.SATestUtils (as it now contains the methods which 8183040 was about) to use NIO file API? >> >> testing: test/hotspot/jtreg/serviceability >> webrev: http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >> JBS: https://bugs.openjdk.java.net/browse/JDK-8183040 > The NIO changes look all right to me. > > Brian From kim.barrett at oracle.com Wed Jun 10 06:26:50 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 10 Jun 2020 02:26:50 -0400 Subject: [OpenJDK 2D-Dev] RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: <5EDE9A75.6020504@oracle.com> References: <5EDE9A75.6020504@oracle.com> Message-ID: > On Jun 8, 2020, at 4:07 PM, Philip Race wrote: > > Hi Kim, > > Can we amend the JEP itself to be not solely about hotspot. > Since it affects other areas I think that > 1) They may need to be compiled with C++14 enabled whether they use new features or not > 2) They may want - or need - to adopt C++11 or C++14 features too. > > You already know that soon we will upgrade the harfbuzz 3rd party library used by 2D > and it will bring along the need to use C++11 features and I had no plan to propose a JEP for that. > So I don't want to be asked why we didn't do one when hotspot did ! > So this really ought to be a JDK JEP whilst of course explaining that hotspot is expected > to be the primary adopter. I think this is at least mostly covered by the following paragraph from the JEP: (This JEP does not propose any usage or style changes for C++ code in the JDK that is outside of HotSpot. The rules are already different for HotSpot and non-HotSpot code. For example, C++ exceptions are used in some non-HotSpot code, but are disallowed in HotSpot by build-time options. However, build consistency requirements will make the newer language features available for use in all C++ code in the JDK.) That?s why I cc?d the RFR to several mailing lists in addition to hotspot-dev. Is there something additional you would like to add? > BTW the JEP (https://bugs.openjdk.java.net/browse/JDK-8208089) still says > > For Solaris: Add the -std=c++14 compiler option. ..... > > So I am sure there is still some room to update the JEP :-) Yeah, I have some updates to make. I also need to do a bit of work on the feature lists. From alexey at azul.com Wed Jun 10 09:03:53 2020 From: alexey at azul.com (Alexey Bakhtin) Date: Wed, 10 Jun 2020 09:03:53 +0000 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <192f1214-fe26-32b5-0243-8d6b90803181@oracle.com> References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> <9013C8D0-33D9-4D09-8953-F8618020A7F7@oracle.com> <8A2C4DEF-FC61-4574-B49D-1133F5349BCE@azul.com> <15fa46f2-0352-a111-a9c7-0d5fc31d9159@oracle.com> <9dc3aa80-cce1-e1c2-d50b-d7cfe570c652@oracle.com> <9B6A7339-F729-41EB-8245-CB6EA03A8D31@azul.com> <21ab9cf7-2d48-f5de-3586-78b40c1d0c12@oracle.com> <192f1214-fe26-32b5-0243-8d6b90803181@oracle.com> Message-ID: Hello Sean, The link to CSR for this feature : https://bugs.openjdk.java.net/browse/JDK-8247311 Regards Alexey > On 9 Jun 2020, at 19:50, Sean Mullan wrote: > > On 6/9/20 12:40 PM, Xuelei Fan wrote: >> About the prefix, it may follow RFC 5056 (See page 7, section 2.1). >> o Specifications of channel bindings for any secure channels MUST >> provide for a single, canonical octet string encoding of the >> channel bindings. Under this framework, channel bindings MUST >> start with the channel binding unique prefix followed by a colon >> (ASCII 0x3A). > > Thanks! Easy to miss. > > I would recommend adding more comments in the code (ex, in TLSChannelBinding) pointing to that RFC section, and other RFCs such 5929 for the tls cbtypes. Also, this RFC (and other specifications such as RFC 5959) should be listed in the CSR so that we document precisely what encodings and types the JDK implementation supports and is using. > > --Sean From james.laskey at oracle.com Wed Jun 10 12:14:43 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Wed, 10 Jun 2020 09:14:43 -0300 Subject: Final call Re: RFR: JDK-8230744 Several classes throw OutOfMemoryError without message In-Reply-To: References: <045D0E84-60E4-491B-9A31-7B05043FFE3B@oracle.com> <78cf935b-d7f3-342a-6ed1-a4a86becb727@oracle.com> <6D4D4663-BB0E-466D-BBA3-9D39556B34BC@oracle.com> Message-ID: Will push if no comments by EOB. > On Jun 8, 2020, at 2:22 PM, Jim Laskey wrote: > > Revised to use a consistent error message. Modified AbstractStringBuilder and PriorityBlockingQueue to use ArraysSupport.newLength(). Didn't modify ByteArrayChannel and UnsyncByteArrayOutputStream since those changes would require changes to module exporting. > > webrev: http://cr.openjdk.java.net/~jlaskey/8230744/webrev-03/index.html > > >> On Jun 3, 2020, at 11:24 AM, Jim Laskey wrote: >> >> It's not the goal or role of this bug to fix the wrongs of the past, merely add error messages to the exceptions. I raised the discussion as an issue. Clearly there is a correct path to follow. If you think more effort is required then file a bug. :-) >> >> Cheers, >> >> -- Jim >> >> >> >> >>> On Jun 2, 2020, at 7:13 PM, Stuart Marks wrote: >>> >>> >>> >>> On 6/2/20 6:52 AM, Jim Laskey wrote: >>>> Revised to reflect requested changes. >>>> http://cr.openjdk.java.net/~jlaskey/8230744/webrev-01/index.html >>> >>> On this, if all you're doing is changing exception messages, then I don't care very much, modulo concerns about wording from others. If you start to get into changing the growth logic (like the Math.addExact stuff) then please see my message on the related thread, "Sometimes constraints are questionable." >>> >>> Thanks, >>> >>> s'marks >> > From daniel.fuchs at oracle.com Wed Jun 10 14:29:36 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 10 Jun 2020 15:29:36 +0100 Subject: Thread leak by LdapLoginModule In-Reply-To: <5e5f3f91-bdc2-7af2-7b41-894484de7f61@oracle.com> References: <889628466.1257399.1591737319316.JavaMail.zimbra@desy.de> <5e5f3f91-bdc2-7af2-7b41-894484de7f61@oracle.com> Message-ID: <3c1da905-0654-1932-0b49-d257bf661d82@oracle.com> On 09/06/2020 23:21, Sean Mullan wrote: >> The issue is not observed with java-14, thus I assume that the fix is >> related to commit >> >> http://hg.openjdk.java.net/jdk/jdk/rev/6717d7e59db4 >> >> As java-11 is LTS, what is the procedure to get it fix back-ported? Hi, AFAICS the fix has already been backported to JDK 11.0.8. best regards, -- daniel From rahul.r.yadav at oracle.com Wed Jun 10 15:06:01 2020 From: rahul.r.yadav at oracle.com (Rahul) Date: Wed, 10 Jun 2020 16:06:01 +0100 Subject: RFR 8245302: Upgrade LogRecord to support long thread ids and remove its usage of ThreadLocal In-Reply-To: <94B2555B-981F-40EB-AB5F-31E2BE068FF0@oracle.com> References: <94B2555B-981F-40EB-AB5F-31E2BE068FF0@oracle.com> Message-ID: ? Hello, Request to have my fix reviewed for the issue: JDK-8245302: Upgrade LogRecord to support long thread ids and remove its usage of ThreadLocal. java.util.logging.LogRecord is updated to support long thread ids without ThreadLocal. Before the update, thread id?s did not support long values, ThreadLocal usage has been Removed. . issue: https://bugs.openjdk.java.net/browse/JDK-8245302 webrev: http://cr.openjdk.java.net/~ryadav/webrev_8245302/index.html csr: https://bugs.openjdk.java.net/browse/JDK-8247219 - rahul From martinrb at google.com Wed Jun 10 16:15:08 2020 From: martinrb at google.com (Martin Buchholz) Date: Wed, 10 Jun 2020 09:15:08 -0700 Subject: Final call Re: RFR: JDK-8230744 Several classes throw OutOfMemoryError without message In-Reply-To: References: <045D0E84-60E4-491B-9A31-7B05043FFE3B@oracle.com> <78cf935b-d7f3-342a-6ed1-a4a86becb727@oracle.com> <6D4D4663-BB0E-466D-BBA3-9D39556B34BC@oracle.com> Message-ID: I took a look at PriorityBlockingQueue. Part of converting to ArraysSupport involves deleting the local orphan MAX_ARRAY_SIZE; that needs to be done. --- It looks like newCap > oldCap is always true, so we should delete the test? if (newCap > oldCap && queue == array) --- In Pattern.java I see + throw new OutOfMemoryError("Requested array size exceeds VM limit"); That wording doesn't seem useful to me - the use of an array is an implementation detail, and the user didn't __request__ it. Better seems the wording in ArraysSupport throw new OutOfMemoryError("Required array length too large"); but if we're going to the trouble of composing a custom detail message, I'd try harder to find one meaningful to the user of the API, something like "pattern too large" On Wed, Jun 10, 2020 at 5:15 AM Jim Laskey wrote: > > Will push if no comments by EOB. > > > On Jun 8, 2020, at 2:22 PM, Jim Laskey wrote: > > > > Revised to use a consistent error message. Modified AbstractStringBuilder and PriorityBlockingQueue to use ArraysSupport.newLength(). Didn't modify ByteArrayChannel and UnsyncByteArrayOutputStream since those changes would require changes to module exporting. > > > > webrev: http://cr.openjdk.java.net/~jlaskey/8230744/webrev-03/index.html > > > > > >> On Jun 3, 2020, at 11:24 AM, Jim Laskey wrote: > >> > >> It's not the goal or role of this bug to fix the wrongs of the past, merely add error messages to the exceptions. I raised the discussion as an issue. Clearly there is a correct path to follow. If you think more effort is required then file a bug. :-) > >> > >> Cheers, > >> > >> -- Jim > >> > >> > >> > >> > >>> On Jun 2, 2020, at 7:13 PM, Stuart Marks wrote: > >>> > >>> > >>> > >>> On 6/2/20 6:52 AM, Jim Laskey wrote: > >>>> Revised to reflect requested changes. > >>>> http://cr.openjdk.java.net/~jlaskey/8230744/webrev-01/index.html > >>> > >>> On this, if all you're doing is changing exception messages, then I don't care very much, modulo concerns about wording from others. If you start to get into changing the growth logic (like the Math.addExact stuff) then please see my message on the related thread, "Sometimes constraints are questionable." > >>> > >>> Thanks, > >>> > >>> s'marks > >> > > > From alexey.semenyuk at oracle.com Wed Jun 10 16:21:40 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Wed, 10 Jun 2020 12:21:40 -0400 Subject: RFR: JDK-8247353: jtreg tests minor issues clean up In-Reply-To: References: Message-ID: <4efc0137-d2fb-cf69-1fac-3e871d5eedc4@oracle.com> Please review fix [2] for jpackage bug [1]. Minor jpackage jtreg tests clean up of issues uncovered during the recent local test runs. - Alexey [1] https://bugs.openjdk.java.net/browse/JDK-8247353 [2]?http://cr.openjdk.java.net/~asemenyuk/8247353/webrev.00 From james.laskey at oracle.com Wed Jun 10 16:33:27 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Wed, 10 Jun 2020 13:33:27 -0300 Subject: Final call Re: RFR: JDK-8230744 Several classes throw OutOfMemoryError without message In-Reply-To: References: <045D0E84-60E4-491B-9A31-7B05043FFE3B@oracle.com> <78cf935b-d7f3-342a-6ed1-a4a86becb727@oracle.com> <6D4D4663-BB0E-466D-BBA3-9D39556B34BC@oracle.com> Message-ID: > On Jun 10, 2020, at 1:15 PM, Martin Buchholz wrote: > > I took a look at PriorityBlockingQueue. > > Part of converting to ArraysSupport involves deleting the local orphan > MAX_ARRAY_SIZE; that needs to be done. Removed. > > --- > It looks like > newCap > oldCap > is always true, so we should delete the test? > if (newCap > oldCap && queue == array) > --- If oldCap == MAX_ARRAY_SIZE wouldn't newCap == oldCap > In Pattern.java I see > + throw new OutOfMemoryError("Requested array size exceeds > VM limit"); > > That wording doesn't seem useful to me - the use of an array is an > implementation detail, and the user didn't __request__ it. > > Better seems the wording in ArraysSupport > throw new OutOfMemoryError("Required array length too large"); > but if we're going to the trouble of composing a custom detail > message, I'd try harder to find one meaningful to the user of the API, > something like "pattern too large" Done and done. Thank you. > > On Wed, Jun 10, 2020 at 5:15 AM Jim Laskey wrote: >> >> Will push if no comments by EOB. >> >>> On Jun 8, 2020, at 2:22 PM, Jim Laskey wrote: >>> >>> Revised to use a consistent error message. Modified AbstractStringBuilder and PriorityBlockingQueue to use ArraysSupport.newLength(). Didn't modify ByteArrayChannel and UnsyncByteArrayOutputStream since those changes would require changes to module exporting. >>> >>> webrev: http://cr.openjdk.java.net/~jlaskey/8230744/webrev-03/index.html >>> >>> >>>> On Jun 3, 2020, at 11:24 AM, Jim Laskey wrote: >>>> >>>> It's not the goal or role of this bug to fix the wrongs of the past, merely add error messages to the exceptions. I raised the discussion as an issue. Clearly there is a correct path to follow. If you think more effort is required then file a bug. :-) >>>> >>>> Cheers, >>>> >>>> -- Jim >>>> >>>> >>>> >>>> >>>>> On Jun 2, 2020, at 7:13 PM, Stuart Marks wrote: >>>>> >>>>> >>>>> >>>>> On 6/2/20 6:52 AM, Jim Laskey wrote: >>>>>> Revised to reflect requested changes. >>>>>> http://cr.openjdk.java.net/~jlaskey/8230744/webrev-01/index.html >>>>> >>>>> On this, if all you're doing is changing exception messages, then I don't care very much, modulo concerns about wording from others. If you start to get into changing the growth logic (like the Math.addExact stuff) then please see my message on the related thread, "Sometimes constraints are questionable." >>>>> >>>>> Thanks, >>>>> >>>>> s'marks >>>> >>> >> From alexey.semenyuk at oracle.com Wed Jun 10 16:41:06 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Wed, 10 Jun 2020 12:41:06 -0400 Subject: RFR: JDK-8246042: Non-ASCII characters are not handled correctly in the native launcher In-Reply-To: References: Message-ID: Please review fix [2] for jpackage bug [1]. On Windows encode arguments for JNI_Launch() call with an encoding of the current process ANSI code page instead of utf8. - Alexey [1] https://bugs.openjdk.java.net/browse/JDK-8246042 [2]?http://cr.openjdk.java.net/~asemenyuk/8246042/webrev.00 From lance.andersen at oracle.com Wed Jun 10 16:53:48 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 10 Jun 2020 12:53:48 -0400 Subject: RFR 8247274: (test) HexPrinter cleanup In-Reply-To: <5cdc15f8-37ac-db05-3e94-84b8930eef16@oracle.com> References: <5cdc15f8-37ac-db05-3e94-84b8930eef16@oracle.com> Message-ID: Hi Roger, I think your changes look fine. Best Lance > On Jun 9, 2020, at 6:47 PM, Roger Riggs wrote: > > Please review cleanup to the test support for jdk.test.lib.hexprinter.HexPrinter. > > Correct issues with the test support for jdk.test.lib.hexdump.HexPrinter. > - Correct the swapped definitions of Formatters.ASCII and PRINTABLE > Printable should always return a single character, > ASCII provides mnemonics to be consistent with the command line hexdump. > - For the methods to format ByteBuffers align the descriptions of the index and length to match ByteBuffer. > - Correctly implement the extraction of bytes from the ByteBuffer > - a newline in the annotation should print as '\n' instead of causing a line break > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-hexprinter-8247274-1/ > Issue: > https://bugs.openjdk.java.net/browse/JDK-8247274 > > Thanks, Roger Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From alexey.semenyuk at oracle.com Wed Jun 10 17:04:55 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Wed, 10 Jun 2020 13:04:55 -0400 Subject: RFR: JDK-8233244: Add tests for icons configuration in rpm/deb packages In-Reply-To: References: Message-ID: <57e5f4ed-8001-98ff-ea67-564d33bb2a8b@oracle.com> Please review fix [2] for jpackage bug [1]. The fix adds verification there are `xdg-icon-resource` commands in Linux packages for file associations with icons. - Alexey [1] https://bugs.openjdk.java.net/browse/JDK-8233244 [2]?http://cr.openjdk.java.net/~asemenyuk/8233244/webrev.00 From martinrb at google.com Wed Jun 10 17:10:38 2020 From: martinrb at google.com (Martin Buchholz) Date: Wed, 10 Jun 2020 10:10:38 -0700 Subject: Final call Re: RFR: JDK-8230744 Several classes throw OutOfMemoryError without message In-Reply-To: References: <045D0E84-60E4-491B-9A31-7B05043FFE3B@oracle.com> <78cf935b-d7f3-342a-6ed1-a4a86becb727@oracle.com> <6D4D4663-BB0E-466D-BBA3-9D39556B34BC@oracle.com> Message-ID: On Wed, Jun 10, 2020 at 9:35 AM Jim Laskey wrote: > > --- > > It looks like > > newCap > oldCap > > is always true, so we should delete the test? > > if (newCap > oldCap && queue == array) > > --- > > If oldCap == MAX_ARRAY_SIZE wouldn't newCap == oldCap newLength's contract is to return a value at least larger by minGrowth, or throw. I looked around for tests for newLength, but couldn't find any. We should add those. In particular, check that e.g. newLength(MAX_ARRAY_LENGTH, 1, 42) == Integer.MAX_VALUE newLength(Integer.MAX_VALUE, 1, 42) => OOME From andy.herrick at oracle.com Wed Jun 10 17:14:06 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Wed, 10 Jun 2020 13:14:06 -0400 Subject: RFR: JDK-8247353: jtreg tests minor issues clean up In-Reply-To: <4efc0137-d2fb-cf69-1fac-3e871d5eedc4@oracle.com> References: <4efc0137-d2fb-cf69-1fac-3e871d5eedc4@oracle.com> Message-ID: <5ff071c5-4167-90e4-50e1-61ab8ba889cb@oracle.com> looks good. /Andy On 6/10/2020 12:21 PM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > Minor jpackage jtreg tests clean up of issues uncovered during the > recent local test runs. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8247353 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8247353/webrev.00 > From Roger.Riggs at oracle.com Wed Jun 10 17:22:50 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Wed, 10 Jun 2020 13:22:50 -0400 Subject: Final call Re: RFR: JDK-8230744 Several classes throw OutOfMemoryError without message In-Reply-To: References: <045D0E84-60E4-491B-9A31-7B05043FFE3B@oracle.com> <78cf935b-d7f3-342a-6ed1-a4a86becb727@oracle.com> <6D4D4663-BB0E-466D-BBA3-9D39556B34BC@oracle.com> Message-ID: <4f20baa4-e4ec-1214-496d-df78670df601@oracle.com> Hi Jim, In Unsafe.java: 632: the "Unable to allocate 98494837395: doesn't read very well. Please add " bytes" to the end. I would probably drop "array" from all the messages but at least in the String* cases. As Martin notes, the array is an implementation detail. (And i still prefer "implementation limit" over "VM limit". Thanks, Roger On 6/10/20 12:33 PM, Jim Laskey wrote: > >> On Jun 130, 2020, at 1:15 PM, Martin Buchholz wrote: >> >> I took a look at PriorityBlockingQueue. >> >> Part of converting to ArraysSupport involves deleting the local orphan >> MAX_ARRAY_SIZE; that needs to be done. > Removed. > >> --- >> It looks like >> newCap > oldCap >> is always true, so we should delete the test? >> if (newCap > oldCap && queue == array) >> --- > If oldCap == MAX_ARRAY_SIZE wouldn't newCap == oldCap > > >> In Pattern.java I see >> + throw new OutOfMemoryError("Requested array size exceeds >> VM limit"); >> >> That wording doesn't seem useful to me - the use of an array is an >> implementation detail, and the user didn't __request__ it. >> >> Better seems the wording in ArraysSupport >> throw new OutOfMemoryError("Required array length too large"); >> but if we're going to the trouble of composing a custom detail >> message, I'd try harder to find one meaningful to the user of the API, >> something like "pattern too large" > Done and done. > > Thank you. > > > >> On Wed, Jun 10, 2020 at 5:15 AM Jim Laskey wrote: >>> Will push if no comments by EOB. >>> >>>> On Jun 8, 2020, at 2:22 PM, Jim Laskey wrote: >>>> >>>> Revised to use a consistent error message. Modified AbstractStringBuilder and PriorityBlockingQueue to use ArraysSupport.newLength(). Didn't modify ByteArrayChannel and UnsyncByteArrayOutputStream since those changes would require changes to module exporting. >>>> >>>> webrev: http://cr.openjdk.java.net/~jlaskey/8230744/webrev-03/index.html >>>> >>>> >>>>> On Jun 3, 2020, at 11:24 AM, Jim Laskey wrote: >>>>> >>>>> It's not the goal or role of this bug to fix the wrongs of the past, merely add error messages to the exceptions. I raised the discussion as an issue. Clearly there is a correct path to follow. If you think more effort is required then file a bug. :-) >>>>> >>>>> Cheers, >>>>> >>>>> -- Jim >>>>> >>>>> >>>>> >>>>> >>>>>> On Jun 2, 2020, at 7:13 PM, Stuart Marks wrote: >>>>>> >>>>>> >>>>>> >>>>>> On 6/2/20 6:52 AM, Jim Laskey wrote: >>>>>>> Revised to reflect requested changes. >>>>>>> http://cr.openjdk.java.net/~jlaskey/8230744/webrev-01/index.html >>>>>> On this, if all you're doing is changing exception messages, then I don't care very much, modulo concerns about wording from others. If you start to get into changing the growth logic (like the Math.addExact stuff) then please see my message on the related thread, "Sometimes constraints are questionable." >>>>>> >>>>>> Thanks, >>>>>> >>>>>> s'marks From james.laskey at oracle.com Wed Jun 10 17:23:44 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Wed, 10 Jun 2020 14:23:44 -0300 Subject: Final call Re: RFR: JDK-8230744 Several classes throw OutOfMemoryError without message In-Reply-To: References: <045D0E84-60E4-491B-9A31-7B05043FFE3B@oracle.com> <78cf935b-d7f3-342a-6ed1-a4a86becb727@oracle.com> <6D4D4663-BB0E-466D-BBA3-9D39556B34BC@oracle.com> Message-ID: <713888F1-E3B5-4D25-B1A6-DC2AF004FCBC@oracle.com> Thank you. > On Jun 10, 2020, at 2:10 PM, Martin Buchholz wrote: > > On Wed, Jun 10, 2020 at 9:35 AM Jim Laskey wrote: > >>> --- >>> It looks like >>> newCap > oldCap >>> is always true, so we should delete the test? >>> if (newCap > oldCap && queue == array) >>> --- >> >> If oldCap == MAX_ARRAY_SIZE wouldn't newCap == oldCap > > newLength's contract is to return a value at least larger by > minGrowth, or throw. > > I looked around for tests for newLength, but couldn't find any. > We should add those. > In particular, check that e.g. > newLength(MAX_ARRAY_LENGTH, 1, 42) == Integer.MAX_VALUE > newLength(Integer.MAX_VALUE, 1, 42) => OOME From andy.herrick at oracle.com Wed Jun 10 17:30:45 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Wed, 10 Jun 2020 13:30:45 -0400 Subject: RFR: JDK-8233244: Add tests for icons configuration in rpm/deb packages In-Reply-To: <57e5f4ed-8001-98ff-ea67-564d33bb2a8b@oracle.com> References: <57e5f4ed-8001-98ff-ea67-564d33bb2a8b@oracle.com> Message-ID: <64f47377-1489-88a7-5c99-df66654d7fd9@oracle.com> looks good. /Andy On 6/10/2020 1:04 PM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > The fix adds verification there are `xdg-icon-resource` commands in > Linux packages for file associations with icons. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8233244 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8233244/webrev.00 > From andy.herrick at oracle.com Wed Jun 10 17:32:10 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Wed, 10 Jun 2020 13:32:10 -0400 Subject: RFR: JDK-8246042: Non-ASCII characters are not handled correctly in the native launcher In-Reply-To: References: Message-ID: <1d4e31ed-db49-5708-0394-4a2d6702dbcc@oracle.com> looks good. /Andy On 6/10/2020 12:41 PM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > On Windows encode arguments for JNI_Launch() call with an encoding of > the current process ANSI code page instead of utf8. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8246042 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8246042/webrev.00 > From Roger.Riggs at oracle.com Wed Jun 10 17:37:15 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Wed, 10 Jun 2020 13:37:15 -0400 Subject: RFR 8247274: (test) HexPrinter cleanup In-Reply-To: References: <5cdc15f8-37ac-db05-3e94-84b8930eef16@oracle.com> Message-ID: <21e6c104-6e4f-6eae-031e-d94e57b65f76@oracle.com> Thanks Lance On 6/10/20 12:53 PM, Lance Andersen wrote: > Hi Roger, > > I think your changes look fine. > > > Best > Lance > >> On Jun 9, 2020, at 6:47 PM, Roger Riggs > > wrote: >> >> Please review cleanup to the test support for >> jdk.test.lib.hexprinter.HexPrinter. >> >> Correct issues with the test support for jdk.test.lib.hexdump.HexPrinter. >> ?- Correct the swapped definitions of Formatters.ASCII and PRINTABLE >> ???Printable should always return a single character, >> ???ASCII provides mnemonics to be consistent with the command line >> hexdump. >> - For the methods to format ByteBuffers align the descriptions of the >> index and length to match ByteBuffer. >> ?- Correctly implement the extraction of bytes from the ByteBuffer >> ?- a newline in the annotation should print as '\n' instead of >> causing a line break >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-hexprinter-8247274-1/ >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8247274 >> >> Thanks, Roger > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle?Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From alexander.matveev at oracle.com Wed Jun 10 18:27:04 2020 From: alexander.matveev at oracle.com (Alexander Matveev) Date: Wed, 10 Jun 2020 11:27:04 -0700 Subject: RFR: 8245788: EmptyFolderPackageTest fails on Windows 10 Message-ID: <45efd486-afd6-9b95-f798-ec700079b7d1@oracle.com> Please review the jpackage fix for bug [1] at [2]. To unpack msi we using /a argument which is not exactly an installation. In this mode (Administrative installation) msi will extract files and also creates .msi installer to install these files. For some reason empty folders are not created during extraction, but they will be created after installing administrative installation. So, I think issue is not with jpackage, but instead we cannot verify empty folder using this approach and thus check for empty folder was disabled if msi is unpacked. [1] https://bugs.openjdk.java.net/browse/JDK-8245788 [2] http://cr.openjdk.java.net/~almatvee/8245788/webrev.00/ Thanks, Alexander From alexander.matveev at oracle.com Wed Jun 10 18:39:33 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Wed, 10 Jun 2020 11:39:33 -0700 Subject: RFR: JDK-8247353: jtreg tests minor issues clean up In-Reply-To: <4efc0137-d2fb-cf69-1fac-3e871d5eedc4@oracle.com> References: <4efc0137-d2fb-cf69-1fac-3e871d5eedc4@oracle.com> Message-ID: Hi Alexey, Looks good. Thanks, Alexander On 6/10/20 9:21 AM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > Minor jpackage jtreg tests clean up of issues uncovered during the > recent local test runs. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8247353 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8247353/webrev.00 > From alexander.matveev at oracle.com Wed Jun 10 18:46:32 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Wed, 10 Jun 2020 11:46:32 -0700 Subject: RFR: JDK-8246042: Non-ASCII characters are not handled correctly in the native launcher In-Reply-To: References: Message-ID: <31674f92-53f4-e0e9-ed97-1cb80e1d7b40@oracle.com> Hi Alexey, Looks good. Thanks, Alexander On 6/10/20 9:41 AM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > On Windows encode arguments for JNI_Launch() call with an encoding of > the current process ANSI code page instead of utf8. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8246042 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8246042/webrev.00 > From alexey.semenyuk at oracle.com Wed Jun 10 18:57:50 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Wed, 10 Jun 2020 14:57:50 -0400 Subject: RFR: 8245788: EmptyFolderPackageTest fails on Windows 10 In-Reply-To: <45efd486-afd6-9b95-f798-ec700079b7d1@oracle.com> References: <45efd486-afd6-9b95-f798-ec700079b7d1@oracle.com> Message-ID: <91e8f0f1-01c5-c888-8d97-eb235b3a917c@oracle.com> Looks good. - Alexey On 6/10/2020 2:27 PM, Alexander Matveev wrote: > Please review the jpackage fix for bug [1] at [2]. > > To unpack msi we using /a argument which is not exactly an > installation. In this mode (Administrative installation) msi will > extract files and also creates .msi installer to install these files. > For some reason empty folders are not created during extraction, but > they will be created after installing administrative installation. So, > I think issue is not with jpackage, but instead we cannot verify empty > folder using this approach and thus check for empty folder was > disabled if msi is unpacked. > > [1] https://bugs.openjdk.java.net/browse/JDK-8245788 > [2] http://cr.openjdk.java.net/~almatvee/8245788/webrev.00/ > > Thanks, > Alexander From alexey.menkov at oracle.com Wed Jun 10 19:20:52 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Wed, 10 Jun 2020 12:20:52 -0700 Subject: RFR(S) : 8183040 : update jdk/test/lib/Platform.java to use NIO file API In-Reply-To: References: <8CA6F4A9-7B57-4B70-9087-D1ACBF561714@oracle.com> Message-ID: <9e159e51-0901-c0a3-2c36-42dbd6696dc9@oracle.com> Hi Igor, On 06/09/2020 20:11, Igor Ignatyev wrote: > Hi Alex, > > as far as I can see, the caller just rethrows IOException as RuntimeException, so I don't think throwing IndexOutOfBoundsException would be much different, albeit it will be a bit more cryptic. yet given the content of /proc/sys/kernel/yama/ptrace_scope and /sys/fs/selinux/booleans/deny_ptrace is part of linux kernel contract, I doubt we will encounter IIOOBE in any reasonable setups. however, if you want I can check the length of bb arrays at L#171 and L#190 and throw an Error w/ message suggesting that something went completely wrong. Yes, the test still fails in the case, but if I see IndexOutOfBoundsException (or something similar) as a test failure reason, my first thought that this is the test issue. Could you please add the checks. --alex > > -- Igor > >> On Jun 9, 2020, at 6:36 PM, Alex Menkov wrote: >> >> Hi Igor, >> >> In SATestUtils.java you do >> >> var bb = ... Files.readAllBytes(...) ... >> and then use bb[0] >> >> if the file has 0 length, old code throws EOFException and new one will throw IndexOutOfBoundsException. >> And looks like the caller doesn't expect it (it catches IOException). >> >> --alex >> >> On 06/09/2020 16:47, Igor Ignatyev wrote: >>> http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >>>> >>>> 38 lines changed: 8 ins; 16 del; 14 mod; >>> Hi all, >>> could you please review this small clean up of testlibrary classes which updates j.t.lib.Platform and j.t.l.SA.SATestUtils (as it now contains the methods which 8183040 was about) to use NIO file API? >>> testing: test/hotspot/jtreg/serviceability >>> webrev: http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8183040 >>> Thanks, >>> -- Igor > From igor.ignatyev at oracle.com Wed Jun 10 19:29:30 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 10 Jun 2020 12:29:30 -0700 Subject: RFR(S) : 8183040 : update jdk/test/lib/Platform.java to use NIO file API In-Reply-To: <9e159e51-0901-c0a3-2c36-42dbd6696dc9@oracle.com> References: <8CA6F4A9-7B57-4B70-9087-D1ACBF561714@oracle.com> <9e159e51-0901-c0a3-2c36-42dbd6696dc9@oracle.com> Message-ID: Hi Alex, sure, here is the incremental diff, so now Error is thrown if bb arrays are empty -- http://cr.openjdk.java.net/~iignatyev//8183040/webrev.0-1 Thanks, -- Igor > On Jun 10, 2020, at 12:20 PM, Alex Menkov wrote: > > Hi Igor, > > On 06/09/2020 20:11, Igor Ignatyev wrote: >> Hi Alex, >> as far as I can see, the caller just rethrows IOException as RuntimeException, so I don't think throwing IndexOutOfBoundsException would be much different, albeit it will be a bit more cryptic. yet given the content of /proc/sys/kernel/yama/ptrace_scope and /sys/fs/selinux/booleans/deny_ptrace is part of linux kernel contract, I doubt we will encounter IIOOBE in any reasonable setups. however, if you want I can check the length of bb arrays at L#171 and L#190 and throw an Error w/ message suggesting that something went completely wrong. > > Yes, the test still fails in the case, but if I see IndexOutOfBoundsException (or something similar) as a test failure reason, my first thought that this is the test issue. > Could you please add the checks. > > --alex > >> -- Igor >>> On Jun 9, 2020, at 6:36 PM, Alex Menkov wrote: >>> >>> Hi Igor, >>> >>> In SATestUtils.java you do >>> >>> var bb = ... Files.readAllBytes(...) ... >>> and then use bb[0] >>> >>> if the file has 0 length, old code throws EOFException and new one will throw IndexOutOfBoundsException. >>> And looks like the caller doesn't expect it (it catches IOException). >>> >>> --alex >>> >>> On 06/09/2020 16:47, Igor Ignatyev wrote: >>>> http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >>>>> >>>>> 38 lines changed: 8 ins; 16 del; 14 mod; >>>> Hi all, >>>> could you please review this small clean up of testlibrary classes which updates j.t.lib.Platform and j.t.l.SA.SATestUtils (as it now contains the methods which 8183040 was about) to use NIO file API? >>>> testing: test/hotspot/jtreg/serviceability >>>> webrev: http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8183040 >>>> Thanks, >>>> -- Igor From naoto.sato at oracle.com Wed Jun 10 19:49:32 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Wed, 10 Jun 2020 12:49:32 -0700 Subject: RFR: 8246721: java/util/Locale/LocaleProvidersRun.java failed on Windows platforms. Message-ID: <5961e38a-3af0-3e7a-342a-9bf4d46e0a67@oracle.com> Hello, Please review the fix to the following issue: https://bugs.openjdk.java.net/browse/JDK-8246721 The proposed changeset is located at: http://cr.openjdk.java.net/~naoto/8246721/webrev.00/ The test has been modified to deal with double quotes in string arguments consistently across platforms, by explicitly specifying the system property accordingly. Naoto From alexey.menkov at oracle.com Wed Jun 10 20:05:01 2020 From: alexey.menkov at oracle.com (Alex Menkov) Date: Wed, 10 Jun 2020 13:05:01 -0700 Subject: RFR(S) : 8183040 : update jdk/test/lib/Platform.java to use NIO file API In-Reply-To: References: <8CA6F4A9-7B57-4B70-9087-D1ACBF561714@oracle.com> <9e159e51-0901-c0a3-2c36-42dbd6696dc9@oracle.com> Message-ID: <64f236d0-4ec1-8024-b3b9-e9f67141b4ad@oracle.com> Hi Igor, LGTM. --alex On 06/10/2020 12:29, Igor Ignatyev wrote: > Hi Alex, > > sure, here is the incremental diff, so now Error is thrown if bb arrays are empty -- http://cr.openjdk.java.net/~iignatyev//8183040/webrev.0-1 > > Thanks, > -- Igor > >> On Jun 10, 2020, at 12:20 PM, Alex Menkov wrote: >> >> Hi Igor, >> >> On 06/09/2020 20:11, Igor Ignatyev wrote: >>> Hi Alex, >>> as far as I can see, the caller just rethrows IOException as RuntimeException, so I don't think throwing IndexOutOfBoundsException would be much different, albeit it will be a bit more cryptic. yet given the content of /proc/sys/kernel/yama/ptrace_scope and /sys/fs/selinux/booleans/deny_ptrace is part of linux kernel contract, I doubt we will encounter IIOOBE in any reasonable setups. however, if you want I can check the length of bb arrays at L#171 and L#190 and throw an Error w/ message suggesting that something went completely wrong. >> >> Yes, the test still fails in the case, but if I see IndexOutOfBoundsException (or something similar) as a test failure reason, my first thought that this is the test issue. >> Could you please add the checks. >> >> --alex >> >>> -- Igor >>>> On Jun 9, 2020, at 6:36 PM, Alex Menkov wrote: >>>> >>>> Hi Igor, >>>> >>>> In SATestUtils.java you do >>>> >>>> var bb = ... Files.readAllBytes(...) ... >>>> and then use bb[0] >>>> >>>> if the file has 0 length, old code throws EOFException and new one will throw IndexOutOfBoundsException. >>>> And looks like the caller doesn't expect it (it catches IOException). >>>> >>>> --alex >>>> >>>> On 06/09/2020 16:47, Igor Ignatyev wrote: >>>>> http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >>>>>> >>>>>> 38 lines changed: 8 ins; 16 del; 14 mod; >>>>> Hi all, >>>>> could you please review this small clean up of testlibrary classes which updates j.t.lib.Platform and j.t.l.SA.SATestUtils (as it now contains the methods which 8183040 was about) to use NIO file API? >>>>> testing: test/hotspot/jtreg/serviceability >>>>> webrev: http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8183040 >>>>> Thanks, >>>>> -- Igor > From igor.ignatyev at oracle.com Wed Jun 10 20:19:51 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Wed, 10 Jun 2020 13:19:51 -0700 Subject: RFR(S) : 8183040 : update jdk/test/lib/Platform.java to use NIO file API In-Reply-To: <64f236d0-4ec1-8024-b3b9-e9f67141b4ad@oracle.com> References: <8CA6F4A9-7B57-4B70-9087-D1ACBF561714@oracle.com> <9e159e51-0901-c0a3-2c36-42dbd6696dc9@oracle.com> <64f236d0-4ec1-8024-b3b9-e9f67141b4ad@oracle.com> Message-ID: thanks Alex, pushed. -- Igor > On Jun 10, 2020, at 1:05 PM, Alex Menkov wrote: > > Hi Igor, > > LGTM. > > --alex > > On 06/10/2020 12:29, Igor Ignatyev wrote: >> Hi Alex, >> sure, here is the incremental diff, so now Error is thrown if bb arrays are empty -- http://cr.openjdk.java.net/~iignatyev//8183040/webrev.0-1 >> Thanks, >> -- Igor >>> On Jun 10, 2020, at 12:20 PM, Alex Menkov wrote: >>> >>> Hi Igor, >>> >>> On 06/09/2020 20:11, Igor Ignatyev wrote: >>>> Hi Alex, >>>> as far as I can see, the caller just rethrows IOException as RuntimeException, so I don't think throwing IndexOutOfBoundsException would be much different, albeit it will be a bit more cryptic. yet given the content of /proc/sys/kernel/yama/ptrace_scope and /sys/fs/selinux/booleans/deny_ptrace is part of linux kernel contract, I doubt we will encounter IIOOBE in any reasonable setups. however, if you want I can check the length of bb arrays at L#171 and L#190 and throw an Error w/ message suggesting that something went completely wrong. >>> >>> Yes, the test still fails in the case, but if I see IndexOutOfBoundsException (or something similar) as a test failure reason, my first thought that this is the test issue. >>> Could you please add the checks. >>> >>> --alex >>> >>>> -- Igor >>>>> On Jun 9, 2020, at 6:36 PM, Alex Menkov wrote: >>>>> >>>>> Hi Igor, >>>>> >>>>> In SATestUtils.java you do >>>>> >>>>> var bb = ... Files.readAllBytes(...) ... >>>>> and then use bb[0] >>>>> >>>>> if the file has 0 length, old code throws EOFException and new one will throw IndexOutOfBoundsException. >>>>> And looks like the caller doesn't expect it (it catches IOException). >>>>> >>>>> --alex >>>>> >>>>> On 06/09/2020 16:47, Igor Ignatyev wrote: >>>>>> http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >>>>>>> >>>>>>> 38 lines changed: 8 ins; 16 del; 14 mod; >>>>>> Hi all, >>>>>> could you please review this small clean up of testlibrary classes which updates j.t.lib.Platform and j.t.l.SA.SATestUtils (as it now contains the methods which 8183040 was about) to use NIO file API? >>>>>> testing: test/hotspot/jtreg/serviceability >>>>>> webrev: http://cr.openjdk.java.net/~iignatyev//8183040/webrev.00 >>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8183040 >>>>>> Thanks, >>>>>> -- Igor From roger.riggs at oracle.com Wed Jun 10 20:27:03 2020 From: roger.riggs at oracle.com (Roger Riggs) Date: Wed, 10 Jun 2020 16:27:03 -0400 Subject: RFR: 8246721: java/util/Locale/LocaleProvidersRun.java failed on Windows platforms. In-Reply-To: <5961e38a-3af0-3e7a-342a-9bf4d46e0a67@oracle.com> References: <5961e38a-3af0-3e7a-342a-9bf4d46e0a67@oracle.com> Message-ID: Hi Naoto, The fix looks fine. Thanks, Roger On 6/10/20 3:49 PM, naoto.sato at oracle.com wrote: > Hello, > > Please review the fix to the following issue: > > https://bugs.openjdk.java.net/browse/JDK-8246721 > > The proposed changeset is located at: > > http://cr.openjdk.java.net/~naoto/8246721/webrev.00/ > > The test has been modified to deal with double quotes in string > arguments consistently across platforms, by explicitly specifying the > system property accordingly. > > Naoto From alexey.semenyuk at oracle.com Wed Jun 10 21:29:53 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Wed, 10 Jun 2020 17:29:53 -0400 Subject: RFR: JDK-8246792: Mac signing tests failed (unsealed contents present in the bundle root) In-Reply-To: References: Message-ID: <453d2853-ebea-6afa-d44c-6e7918c9b503@oracle.com> Please review fix [2] for jpackage bug [1]. Move `.jpackage.xml` file in `app` directory of app image and keep it in all OSX packages (dmg and pkg). Resotore copying of `libjli.dylib` into `$RUNTIME/Contents/MacOS` directory in MacAppImageBuilder.java. Update jtreg test helpers accordingly. - Alexey [1] https://bugs.openjdk.java.net/browse/JDK-8246792 [2]?http://cr.openjdk.java.net/~asemenyuk/8246792/webrev.00 From andy.herrick at oracle.com Wed Jun 10 21:40:08 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Wed, 10 Jun 2020 17:40:08 -0400 Subject: RFR: JDK-8246792: Mac signing tests failed (unsealed contents present in the bundle root) In-Reply-To: <453d2853-ebea-6afa-d44c-6e7918c9b503@oracle.com> References: <453d2853-ebea-6afa-d44c-6e7918c9b503@oracle.com> Message-ID: <81915099-6f28-ac06-1ba5-bdad3058ccda@oracle.com> looks good - This works for me finally. /Andy On 6/10/2020 5:29 PM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > Move `.jpackage.xml` file in `app` directory of app image and keep it > in all OSX packages (dmg and pkg). > Resotore copying of `libjli.dylib` into `$RUNTIME/Contents/MacOS` > directory in MacAppImageBuilder.java. > Update jtreg test helpers accordingly. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8246792 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8246792/webrev.00 > From lance.andersen at oracle.com Wed Jun 10 21:47:04 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 10 Jun 2020 17:47:04 -0400 Subject: RFR 8207936: TestZipFile.java can fail with an OOM error Message-ID: Hi, Please review the fix for JDK-8207936, which addresses the OOM error when running the manual test TestZipFile.java. The fix also removes the call to "getMetaInfEntryNames? as this method was removed as part of the performance work done in JDK-8244624. The webrev can be found at: http://cr.openjdk.java.net/~lancea/8207936/webrev.00/ Best Lance Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From andy.herrick at oracle.com Wed Jun 10 21:47:37 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Wed, 10 Jun 2020 17:47:37 -0400 Subject: RFR: 8245788: EmptyFolderPackageTest fails on Windows 10 In-Reply-To: <45efd486-afd6-9b95-f798-ec700079b7d1@oracle.com> References: <45efd486-afd6-9b95-f798-ec700079b7d1@oracle.com> Message-ID: this looks good. /Andy On 6/10/2020 2:27 PM, Alexander Matveev wrote: > Please review the jpackage fix for bug [1] at [2]. > > To unpack msi we using /a argument which is not exactly an > installation. In this mode (Administrative installation) msi will > extract files and also creates .msi installer to install these files. > For some reason empty folders are not created during extraction, but > they will be created after installing administrative installation. So, > I think issue is not with jpackage, but instead we cannot verify empty > folder using this approach and thus check for empty folder was > disabled if msi is unpacked. > > [1] https://bugs.openjdk.java.net/browse/JDK-8245788 > [2] http://cr.openjdk.java.net/~almatvee/8245788/webrev.00/ > > Thanks, > Alexander From alexander.matveev at oracle.com Wed Jun 10 21:55:57 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Wed, 10 Jun 2020 14:55:57 -0700 Subject: RFR: JDK-8246792: Mac signing tests failed (unsealed contents present in the bundle root) In-Reply-To: <81915099-6f28-ac06-1ba5-bdad3058ccda@oracle.com> References: <453d2853-ebea-6afa-d44c-6e7918c9b503@oracle.com> <81915099-6f28-ac06-1ba5-bdad3058ccda@oracle.com> Message-ID: <20da65f3-37f3-8673-15ce-d5b618bef31d@oracle.com> Hi Alexey, Looks good. Thanks, Alexander On 6/10/20 2:40 PM, Andy Herrick wrote: > looks good - > > This works for me finally. > > /Andy > > On 6/10/2020 5:29 PM, Alexey Semenyuk wrote: >> Please review fix [2] for jpackage bug [1]. >> >> Move `.jpackage.xml` file in `app` directory of app image and keep it >> in all OSX packages (dmg and pkg). >> Resotore copying of `libjli.dylib` into `$RUNTIME/Contents/MacOS` >> directory in MacAppImageBuilder.java. >> Update jtreg test helpers accordingly. >> >> - Alexey >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8246792 >> >> [2]?http://cr.openjdk.java.net/~asemenyuk/8246792/webrev.00 >> From stuart.marks at oracle.com Wed Jun 10 22:02:32 2020 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 10 Jun 2020 15:02:32 -0700 Subject: Final call Re: RFR: JDK-8230744 Several classes throw OutOfMemoryError without message In-Reply-To: <4f20baa4-e4ec-1214-496d-df78670df601@oracle.com> References: <045D0E84-60E4-491B-9A31-7B05043FFE3B@oracle.com> <78cf935b-d7f3-342a-6ed1-a4a86becb727@oracle.com> <6D4D4663-BB0E-466D-BBA3-9D39556B34BC@oracle.com> <4f20baa4-e4ec-1214-496d-df78670df601@oracle.com> Message-ID: <8371e3f9-787f-d6fa-2791-84735bd3941d@oracle.com> Regarding the wording, I don't think either "implementation limit" or "VM limit" is correct, at least not in all the cases. If we were using ArraysSupport.newLength, the error messages are as follows: 1) Actual shortage of heap space. Hotspot generates this: OutOfMemoryError: Java heap space 2) Requested array size exceeds implementation limit. Hotspot generates this: OutOfMemoryError: Requested array size exceeds VM limit 3) Integer wraparound/overflow during size computation. ArraysSupport.newLength() generates this: OutOfMemoryError: Required array length too large For this changeset, we're leaving several places as they are, either using the *exact arithmetic methods or doing their own checks. OK, we'll save refactoring to ArraysSupport.newLength for another time. But the resulting OOME message is I think misleading. For example, in String.java line 2190, we catch an exception from Math.addExact or Math.multiplyExact. This occurs if the arithmetic result exceeds the range of an int. It's less about some implementation or VM limit, but instead the requested length simply cannot be represented by an int. Also, while I'm not as allergic to talking about arrays as Martin, in this case the subsequent code allocates a StringBuilder and not an array. So it is somewhat odd for this message to refer to an array. Anyway, instead of Requested array size exceeds VM limit I'd suggest this: Requested size too large (Or "length" instead of "size". "Length" is more consistent for arrays and strings, but oddly enough, Hotspot uses "array size".) I think this wording could apply to String.java, StringLatin1.java, StringUTF16.java, Pattern.java, UnsyncByteArrayOutputStream.java, and ByteArrayChannel.java. I guess we could try to provide a customized message for each area, but that wouldn't seem to me to add much value. I'm not entirely sure what's going on in AbstractStringBuilder.java. It uses ArraysSupport.newLength(), which should either return an in-range value or throw. I don't understand why the code in ASB.newCapacity() is checking length against Integer.MAX_VALUE and throwing again. This shouldn't happen by the spec of AS.newLength(). I think that check (lines 258-259) can simply be removed. s'marks On 6/10/20 10:22 AM, Roger Riggs wrote: > Hi Jim, > > In Unsafe.java: 632: the "Unable to allocate 98494837395: doesn't read very well. > Please add " bytes" to the end. > > I would probably drop "array" from all the messages but at least in the String* > cases. > As Martin notes, the array is an implementation detail. > (And i still prefer "implementation limit" over "VM limit". > > Thanks, Roger > > > > On 6/10/20 12:33 PM, Jim Laskey wrote: >> >>> On Jun 130, 2020, at 1:15 PM, Martin Buchholz wrote: >>> >>> I took a look at PriorityBlockingQueue. >>> >>> Part of converting to ArraysSupport involves deleting the local orphan >>> MAX_ARRAY_SIZE; that needs to be done. >> Removed. >> >>> --- >>> It looks like >>> newCap > oldCap >>> is always true, so we should delete the test? >>> ???????????????? if (newCap > oldCap && queue == array) >>> --- >> If oldCap == MAX_ARRAY_SIZE wouldn't newCap == oldCap >> >> >>> In Pattern.java I see >>> +??????????? throw new OutOfMemoryError("Requested array size exceeds >>> VM limit"); >>> >>> That wording doesn't seem useful to me - the use of an array is an >>> implementation detail, and the user didn't __request__ it. >>> >>> Better seems the wording in ArraysSupport >>> ??????????? throw new OutOfMemoryError("Required array length too large"); >>> but if we're going to the trouble of composing a custom detail >>> message, I'd try harder to find one meaningful to the user of the API, >>> something like "pattern too large" >> Done and done. >> >> Thank you. >> >> >> >>> On Wed, Jun 10, 2020 at 5:15 AM Jim Laskey wrote: >>>> Will push if no comments by EOB. >>>> >>>>> On Jun 8, 2020, at 2:22 PM, Jim Laskey wrote: >>>>> >>>>> Revised to use a consistent error message. Modified AbstractStringBuilder >>>>> and PriorityBlockingQueue to use ArraysSupport.newLength(). Didn't modify >>>>> ByteArrayChannel and UnsyncByteArrayOutputStream since those changes would >>>>> require changes to module exporting. >>>>> >>>>> webrev: http://cr.openjdk.java.net/~jlaskey/8230744/webrev-03/index.html >>>>> >>>>> >>>>> >>>>>> On Jun 3, 2020, at 11:24 AM, Jim Laskey wrote: >>>>>> >>>>>> It's not the goal or role of this bug to fix the wrongs of the past, >>>>>> merely add error messages to the exceptions. I raised the discussion as an >>>>>> issue. Clearly there is a correct path to follow. If you think more effort >>>>>> is required then file a bug. :-) >>>>>> >>>>>> Cheers, >>>>>> >>>>>> -- Jim >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> On Jun 2, 2020, at 7:13 PM, Stuart Marks wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 6/2/20 6:52 AM, Jim Laskey wrote: >>>>>>>> Revised to reflect requested changes. >>>>>>>> http://cr.openjdk.java.net/~jlaskey/8230744/webrev-01/index.html >>>>>>>> >>>>>>> On this, if all you're doing is changing exception messages, then I don't >>>>>>> care very much, modulo concerns about wording from others. If you start >>>>>>> to get into changing the growth logic (like the Math.addExact stuff) then >>>>>>> please see my message on the related thread, "Sometimes constraints are >>>>>>> questionable." >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> s'marks > From vicente.romero at oracle.com Wed Jun 10 22:23:05 2020 From: vicente.romero at oracle.com (Vicente Romero) Date: Wed, 10 Jun 2020 18:23:05 -0400 Subject: RFR: JDK-8246098: API for Class::permittedSubclasses should clarify if returned elements are ordered or not Message-ID: <243ceb8e-a6e5-4c19-b4e8-912ea4d03948@oracle.com> Please review this small change to the API for java.lang.Class::permittedSubclasses which clarifies that no particular order of the returned elements should be assumed by clients. The related bug is at [1], the webrev at [2] and the associated CSR at [3], Thanks, Vicente [1] https://bugs.openjdk.java.net/browse/JDK-8246098 [2] http://cr.openjdk.java.net/~vromero/8246098/webrev.00/ [3] https://bugs.openjdk.java.net/browse/JDK-8247361 From stuart.marks at oracle.com Wed Jun 10 22:24:14 2020 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 10 Jun 2020 15:24:14 -0700 Subject: Sometimes constraints are questionable In-Reply-To: References: <91bd6c0f-5413-0195-9332-2bbd42919a3f@oracle.com> Message-ID: <5a23c39e-b46a-018c-af83-45006c2d4267@oracle.com> On 6/5/20 1:48 PM, James Laskey wrote: > I?m fixing the two in java.base. The other two are in different modules and would require changes to export. So you can file on those. I've filed https://bugs.openjdk.java.net/browse/JDK-8247373 to enhance the exception message, docs, and to add a test for ArraysSupport.newLength. There's a separate discussion of whether various things in the JDK should be refactored to use this utility method. ArraysSupport is in jdk.internal.util, which is not exported, and I'm fairly reluctant to add a qualified export just for this. So perhaps things outside of java.base can live with doing their own calculations. For other cases, let's wait until after your OOME message fix (JDK-8230744) goes in, since it seems like it might take care of a couple right away. Then we can revisit the various cases and file bugs individually for them. In particular, the use of Math.addExact et. al. and catch/rethrow of ArithmeticException can probably be replaced with a call to ArraysSupport.newLength, but individual cases will have to be looked at carefully. s'marks From huizhe.wang at oracle.com Wed Jun 10 22:24:52 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Wed, 10 Jun 2020 15:24:52 -0700 Subject: RFR 8207936: TestZipFile.java can fail with an OOM error In-Reply-To: References: Message-ID: <62f002f8-92e2-035b-b3cd-a8f6c7efc440@oracle.com> Looks good to me, Lance. -Joe On 6/10/2020 2:47 PM, Lance Andersen wrote: > Hi, > > Please review the fix for JDK-8207936, which addresses the OOM error when running the manual test TestZipFile.java. The fix also removes the call to "getMetaInfEntryNames? as this method was removed as part of the performance work done in JDK-8244624. > > The webrev can be found at: http://cr.openjdk.java.net/~lancea/8207936/webrev.00/ > > Best > Lance > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From claes.redestad at oracle.com Wed Jun 10 22:26:03 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 11 Jun 2020 00:26:03 +0200 Subject: RFR 8207936: TestZipFile.java can fail with an OOM error In-Reply-To: References: Message-ID: <9a1c1b87-3008-daf8-5ee2-34020f6d72f0@oracle.com> Hi Lance, looks good to me! /Claes On 2020-06-10 23:47, Lance Andersen wrote: > Hi, > > Please review the fix for JDK-8207936, which addresses the OOM error when running the manual test TestZipFile.java. The fix also removes the call to "getMetaInfEntryNames? as this method was removed as part of the performance work done in JDK-8244624. > > The webrev can be found at: http://cr.openjdk.java.net/~lancea/8207936/webrev.00/ > > Best > Lance > > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From joe.darcy at oracle.com Thu Jun 11 05:26:44 2020 From: joe.darcy at oracle.com (Joe Darcy) Date: Wed, 10 Jun 2020 22:26:44 -0700 Subject: RFR of JDK-8245620: Improve Annotation.annotationType() documentation Message-ID: <77ddaf80-84ec-1316-94e1-5f6c0f52e2a9@oracle.com> Hello, Please review a number of small updates to the javadoc of java.lang.annotation.Annotation to address ??? JDK-8245620: Improve Annotation.annotationType() documentation The use of? Annotation.annotationType() vs calling getClass on an annotation object may not be clear so the method's javadoc is augmented to explain this situation. java.lang.Enum defines an analogous method which always returns the enum class of an enum object. There are a number of other small improvements, including updating the example toString output to account for ??? 8162817: Annotation toString output not reusable for source input Patch below; thanks, -Joe diff -r 2f75432d7eb3 src/java.base/share/classes/java/lang/annotation/Annotation.java --- a/src/java.base/share/classes/java/lang/annotation/Annotation.java Wed Jun 10 18:18:30 2020 -0400 +++ b/src/java.base/share/classes/java/lang/annotation/Annotation.java Wed Jun 10 22:24:55 2020 -0700 @@ -31,7 +31,7 @@ ? * an annotation type.? Also note that this interface does not itself ? * define an annotation type. ? * - * More information about annotation types can be found in section 9.6 of + * More information about annotation types can be found in section {@jls 9.6} of ? * The Java™ Language Specification. ? * ? * The {@link java.lang.reflect.AnnotatedElement} interface discusses @@ -72,7 +72,7 @@ ????? * ????? *???

  • Two corresponding array typed members {@code x} and {@code y} ????? *??? are considered equal if {@code Arrays.equals(x, y)}, for the -???? *??? appropriate overloading of {@link java.util.Arrays#equals}. +???? *??? appropriate overloading of {@link java.util.Arrays#equals Arrays.hashCode}. ????? * ????? * ????? * @return true if the specified object represents an annotation @@ -121,7 +121,7 @@ ????? * of the representation are implementation-dependent, but the following ????? * may be regarded as typical: ????? *
    -???? *?? @com.acme.util.Name(first=Alfred, middle=E., last=Neuman)
    +???? *?? @com.example.Name(first="Duke", middle="of", last="Java")
     ????? * 
    ????? * ????? * @return a string representation of this annotation @@ -130,7 +130,15 @@ ???? /** ????? * Returns the annotation type of this annotation. +???? * +???? * Note that implementation-dependent classes are used to provide +???? * the implementations of annotations. Therefore, calling {@link +???? * #getClass getClass} on an annotation will return an +???? * implementation-dependent class. In contrast, this method will +???? * reliably return the annotation type of the annotation. +???? * ????? * @return the annotation type of this annotation +???? * @see Enum#getDeclaringClass ????? */ ???? Class annotationType(); ?} From Alan.Bateman at oracle.com Thu Jun 11 06:07:49 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 11 Jun 2020 07:07:49 +0100 Subject: RFR of JDK-8245620: Improve Annotation.annotationType() documentation In-Reply-To: <77ddaf80-84ec-1316-94e1-5f6c0f52e2a9@oracle.com> References: <77ddaf80-84ec-1316-94e1-5f6c0f52e2a9@oracle.com> Message-ID: On 11/06/2020 06:26, Joe Darcy wrote: > Hello, > > Please review a number of small updates to the javadoc of > java.lang.annotation.Annotation to address > > ??? JDK-8245620: Improve Annotation.annotationType() documentation > > The use of? Annotation.annotationType() vs calling getClass on an > annotation object may not be clear so the method's javadoc is > augmented to explain this situation. java.lang.Enum defines an > analogous method which always returns the enum class of an enum object. > > There are a number of other small improvements, including updating the > example toString output to account for > > ??? 8162817: Annotation toString output not reusable for source input > > Patch below; thanks, Looks okay to me although I wonder if the update to annotationType should be an @apiNote. -Alan From chris.hegarty at oracle.com Thu Jun 11 08:47:42 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 11 Jun 2020 09:47:42 +0100 Subject: RFR: JDK-8246098: API for Class::permittedSubclasses should clarify if returned elements are ordered or not In-Reply-To: <243ceb8e-a6e5-4c19-b4e8-912ea4d03948@oracle.com> References: <243ceb8e-a6e5-4c19-b4e8-912ea4d03948@oracle.com> Message-ID: <1C93A28A-E6EF-4504-A099-AF50F558EE4A@oracle.com> > On 10 Jun 2020, at 23:23, Vicente Romero wrote: > > Please review this small change to the API for java.lang.Class::permittedSubclasses which clarifies that no particular order of the returned elements should be assumed by clients. The related bug is at [1], the webrev at [2] and the associated CSR at [3], LGTM. I added myself as reviewer on the CSR. -Chris. > Thanks, > Vicente > > [1] https://bugs.openjdk.java.net/browse/JDK-8246098 > [2] http://cr.openjdk.java.net/~vromero/8246098/webrev.00/ > [3] https://bugs.openjdk.java.net/browse/JDK-8247361 From pavel.rappo at oracle.com Thu Jun 11 09:54:39 2020 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Thu, 11 Jun 2020 10:54:39 +0100 Subject: RFR of JDK-8245620: Improve Annotation.annotationType() documentation In-Reply-To: <77ddaf80-84ec-1316-94e1-5f6c0f52e2a9@oracle.com> References: <77ddaf80-84ec-1316-94e1-5f6c0f52e2a9@oracle.com> Message-ID: <9E915284-DBFB-4049-A2C3-2E3A1643E674@oracle.com> Joe, There is a typo: * appropriate overloading of {@link java.util.Arrays#equals Arrays.hashCode}. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ ~~~~~~~~~~~^ Since we are in this area, is it possible to rephrase (or at least re-punctuate) the doc comment for java.lang.annotation.Annotation#hashCode? The reason is that it uses way too many successive colons: ! Returns the hash code of this annotation, as defined below: * *

    The hash code of an annotation is the sum of the hash codes * of its members (including those with default values), as defined ! below: * * The hash code of an annotation member is (127 times the hash code * of the member-name as computed by {@link String#hashCode()}) XOR ! the hash code of the member-value, as defined below: * *

    The hash code of a member-value depends on its type: This also makes the method summary look weird on the HTML page. -Pavel > On 11 Jun 2020, at 06:26, Joe Darcy wrote: > > Hello, > > Please review a number of small updates to the javadoc of java.lang.annotation.Annotation to address > > JDK-8245620: Improve Annotation.annotationType() documentation > > The use of Annotation.annotationType() vs calling getClass on an annotation object may not be clear so the method's javadoc is augmented to explain this situation. java.lang.Enum defines an analogous method which always returns the enum class of an enum object. > > There are a number of other small improvements, including updating the example toString output to account for > > 8162817: Annotation toString output not reusable for source input > > Patch below; thanks, > > -Joe > > diff -r 2f75432d7eb3 src/java.base/share/classes/java/lang/annotation/Annotation.java > --- a/src/java.base/share/classes/java/lang/annotation/Annotation.java Wed Jun 10 18:18:30 2020 -0400 > +++ b/src/java.base/share/classes/java/lang/annotation/Annotation.java Wed Jun 10 22:24:55 2020 -0700 > @@ -31,7 +31,7 @@ > * an annotation type. Also note that this interface does not itself > * define an annotation type. > * > - * More information about annotation types can be found in section 9.6 of > + * More information about annotation types can be found in section {@jls 9.6} of > * The Java™ Language Specification. > * > * The {@link java.lang.reflect.AnnotatedElement} interface discusses > @@ -72,7 +72,7 @@ > * > *

  • Two corresponding array typed members {@code x} and {@code y} > * are considered equal if {@code Arrays.equals(x, y)}, for the > - * appropriate overloading of {@link java.util.Arrays#equals}. > + * appropriate overloading of {@link java.util.Arrays#equals Arrays.hashCode}. > * > * > * @return true if the specified object represents an annotation > @@ -121,7 +121,7 @@ > * of the representation are implementation-dependent, but the following > * may be regarded as typical: > *
    > -     *   @com.acme.util.Name(first=Alfred, middle=E., last=Neuman)
    > +     *   @com.example.Name(first="Duke", middle="of", last="Java")
    >       * 
    > * > * @return a string representation of this annotation > @@ -130,7 +130,15 @@ > > /** > * Returns the annotation type of this annotation. > + * > + * Note that implementation-dependent classes are used to provide > + * the implementations of annotations. Therefore, calling {@link > + * #getClass getClass} on an annotation will return an > + * implementation-dependent class. In contrast, this method will > + * reliably return the annotation type of the annotation. > + * > * @return the annotation type of this annotation > + * @see Enum#getDeclaringClass > */ > Class annotationType(); > } > From vicente.romero at oracle.com Thu Jun 11 15:15:47 2020 From: vicente.romero at oracle.com (Vicente Romero) Date: Thu, 11 Jun 2020 11:15:47 -0400 Subject: RFR: JDK-8246098: API for Class::permittedSubclasses should clarify if returned elements are ordered or not In-Reply-To: <1C93A28A-E6EF-4504-A099-AF50F558EE4A@oracle.com> References: <243ceb8e-a6e5-4c19-b4e8-912ea4d03948@oracle.com> <1C93A28A-E6EF-4504-A099-AF50F558EE4A@oracle.com> Message-ID: <18060613-f21a-8c24-dbaf-9de35bad02c7@oracle.com> thanks for the review, Vicente On 6/11/20 4:47 AM, Chris Hegarty wrote: > >> On 10 Jun 2020, at 23:23, Vicente Romero wrote: >> >> Please review this small change to the API for java.lang.Class::permittedSubclasses which clarifies that no particular order of the returned elements should be assumed by clients. The related bug is at [1], the webrev at [2] and the associated CSR at [3], > LGTM. I added myself as reviewer on the CSR. > > -Chris. > >> Thanks, >> Vicente >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8246098 >> [2] http://cr.openjdk.java.net/~vromero/8246098/webrev.00/ >> [3] https://bugs.openjdk.java.net/browse/JDK-8247361 From joe.darcy at oracle.com Thu Jun 11 17:11:07 2020 From: joe.darcy at oracle.com (Joe Darcy) Date: Thu, 11 Jun 2020 10:11:07 -0700 Subject: RFR of JDK-8245620: Improve Annotation.annotationType() documentation In-Reply-To: <9E915284-DBFB-4049-A2C3-2E3A1643E674@oracle.com> References: <77ddaf80-84ec-1316-94e1-5f6c0f52e2a9@oracle.com> <9E915284-DBFB-4049-A2C3-2E3A1643E674@oracle.com> Message-ID: Hi Pavel, On 6/11/2020 2:54 AM, Pavel Rappo wrote: > Joe, > > There is a typo: > > * appropriate overloading of {@link java.util.Arrays#equals Arrays.hashCode}. > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ ~~~~~~~~~~~^ > > Since we are in this area, is it possible to rephrase (or at least re-punctuate) the doc comment for java.lang.annotation.Annotation#hashCode? The reason is that it uses way too many successive colons: Good catch. Revised patch below will fix this issue, remove several of the colons, and use @apiNote as Alan suggested. Thanks, -Joe diff -r 16081904714f src/java.base/share/classes/java/lang/annotation/Annotation.java --- a/src/java.base/share/classes/java/lang/annotation/Annotation.java Thu Jun 11 18:08:29 2020 +0200 +++ b/src/java.base/share/classes/java/lang/annotation/Annotation.java Thu Jun 11 10:09:04 2020 -0700 @@ -31,7 +31,7 @@ ? * an annotation type.? Also note that this interface does not itself ? * define an annotation type. ? * - * More information about annotation types can be found in section 9.6 of + * More information about annotation types can be found in section {@jls 9.6} of ? * The Java™ Language Specification. ? * ? * The {@link java.lang.reflect.AnnotatedElement} interface discusses @@ -72,7 +72,7 @@ ????? * ????? *???
  • Two corresponding array typed members {@code x} and {@code y} ????? *??? are considered equal if {@code Arrays.equals(x, y)}, for the -???? *??? appropriate overloading of {@link java.util.Arrays#equals}. +???? *??? appropriate overloading of {@link java.util.Arrays#equals Arrays.equals}. ????? * ????? * ????? * @return true if the specified object represents an annotation @@ -81,17 +81,15 @@ ???? boolean equals(Object obj); ???? /** -???? * Returns the hash code of this annotation, as defined below: +???? * Returns the hash code of this annotation. ????? * ????? *

    The hash code of an annotation is the sum of the hash codes -???? * of its members (including those with default values), as defined -???? * below: +???? * of its members (including those with default values). ????? * ????? * The hash code of an annotation member is (127 times the hash code ????? * of the member-name as computed by {@link String#hashCode()}) XOR -???? * the hash code of the member-value, as defined below: -???? * -???? *

    The hash code of a member-value depends on its type: +???? * the hash code of the member-value. +???? * The hash code of a member-value depends on its type as defined below: ????? *

      ????? *
    • The hash code of a primitive value {@code v} is equal to ????? * WrapperType.valueOf(v).hashCode(), where @@ -121,7 +119,7 @@ ????? * of the representation are implementation-dependent, but the following ????? * may be regarded as typical: ????? *
      -???? *?? @com.acme.util.Name(first=Alfred, middle=E., last=Neuman)
      +???? *?? @com.example.Name(first="Duke", middle="of", last="Java")
       ????? * 
      ????? * ????? * @return a string representation of this annotation @@ -130,7 +128,15 @@ ???? /** ????? * Returns the annotation type of this annotation. +???? * +???? * @apiNote Implementation-dependent classes are used to provide +???? * the implementations of annotations. Therefore, calling {@link +???? * #getClass getClass} on an annotation will return an +???? * implementation-dependent class. In contrast, this method will +???? * reliably return the annotation type of the annotation. +???? * ????? * @return the annotation type of this annotation +???? * @see Enum#getDeclaringClass ????? */ ???? Class annotationType(); ?} From alexey.semenyuk at oracle.com Thu Jun 11 17:23:21 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Thu, 11 Jun 2020 13:23:21 -0400 Subject: RFR: JDK-8247422: --runtime-image on Mac should work for runtime root In-Reply-To: <4efc0137-d2fb-cf69-1fac-3e871d5eedc4@oracle.com> References: <4efc0137-d2fb-cf69-1fac-3e871d5eedc4@oracle.com> Message-ID: <5617c980-cff8-6e64-cb73-6d2183a75462@oracle.com> Please review fix [2] for jpackage bug [1]. Restore handling not only Java runtime home but also Java runtime root directory in `--runtime-image` option on Mac. The patch also fixes [3] issue as a side effect of moving functionality of BasicTest.testJLinkRuntime() function into the new CookedRuntimeTest.test(). - Alexey [1] https://bugs.openjdk.java.net/browse/JDK-8247422 [2]?http://cr.openjdk.java.net/~asemenyuk/8247422/webrev.00 [3] https://bugs.openjdk.java.net/browse/JDK-8247424 From pavel.rappo at oracle.com Thu Jun 11 17:58:14 2020 From: pavel.rappo at oracle.com (Pavel Rappo) Date: Thu, 11 Jun 2020 18:58:14 +0100 Subject: RFR of JDK-8245620: Improve Annotation.annotationType() documentation In-Reply-To: References: <77ddaf80-84ec-1316-94e1-5f6c0f52e2a9@oracle.com> <9E915284-DBFB-4049-A2C3-2E3A1643E674@oracle.com> Message-ID: Looks good. > On 11 Jun 2020, at 18:11, Joe Darcy wrote: > > Hi Pavel, > > On 6/11/2020 2:54 AM, Pavel Rappo wrote: >> Joe, >> >> There is a typo: >> >> * appropriate overloading of {@link java.util.Arrays#equals Arrays.hashCode}. >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ ~~~~~~~~~~~^ >> >> Since we are in this area, is it possible to rephrase (or at least re-punctuate) the doc comment for java.lang.annotation.Annotation#hashCode? The reason is that it uses way too many successive colons: > > Good catch. > > Revised patch below will fix this issue, remove several of the colons, and use @apiNote as Alan suggested. > > Thanks, > > -Joe > > diff -r 16081904714f src/java.base/share/classes/java/lang/annotation/Annotation.java > --- a/src/java.base/share/classes/java/lang/annotation/Annotation.java Thu Jun 11 18:08:29 2020 +0200 > +++ b/src/java.base/share/classes/java/lang/annotation/Annotation.java Thu Jun 11 10:09:04 2020 -0700 > @@ -31,7 +31,7 @@ > * an annotation type. Also note that this interface does not itself > * define an annotation type. > * > - * More information about annotation types can be found in section 9.6 of > + * More information about annotation types can be found in section {@jls 9.6} of > * The Java™ Language Specification. > * > * The {@link java.lang.reflect.AnnotatedElement} interface discusses > @@ -72,7 +72,7 @@ > * > *
    • Two corresponding array typed members {@code x} and {@code y} > * are considered equal if {@code Arrays.equals(x, y)}, for the > - * appropriate overloading of {@link java.util.Arrays#equals}. > + * appropriate overloading of {@link java.util.Arrays#equals Arrays.equals}. > *
    > * > * @return true if the specified object represents an annotation > @@ -81,17 +81,15 @@ > boolean equals(Object obj); > > /** > - * Returns the hash code of this annotation, as defined below: > + * Returns the hash code of this annotation. > * > *

    The hash code of an annotation is the sum of the hash codes > - * of its members (including those with default values), as defined > - * below: > + * of its members (including those with default values). > * > * The hash code of an annotation member is (127 times the hash code > * of the member-name as computed by {@link String#hashCode()}) XOR > - * the hash code of the member-value, as defined below: > - * > - *

    The hash code of a member-value depends on its type: > + * the hash code of the member-value. > + * The hash code of a member-value depends on its type as defined below: > *

      > *
    • The hash code of a primitive value {@code v} is equal to > * WrapperType.valueOf(v).hashCode(), where > @@ -121,7 +119,7 @@ > * of the representation are implementation-dependent, but the following > * may be regarded as typical: > *
      > -     *   @com.acme.util.Name(first=Alfred, middle=E., last=Neuman)
      > +     *   @com.example.Name(first="Duke", middle="of", last="Java")
      >       * 
      > * > * @return a string representation of this annotation > @@ -130,7 +128,15 @@ > > /** > * Returns the annotation type of this annotation. > + * > + * @apiNote Implementation-dependent classes are used to provide > + * the implementations of annotations. Therefore, calling {@link > + * #getClass getClass} on an annotation will return an > + * implementation-dependent class. In contrast, this method will > + * reliably return the annotation type of the annotation. > + * > * @return the annotation type of this annotation > + * @see Enum#getDeclaringClass > */ > Class annotationType(); > } > From peter.kessler at os.amperecomputing.com Thu Jun 11 18:14:38 2020 From: peter.kessler at os.amperecomputing.com (Peter Kessler OS) Date: Thu, 11 Jun 2020 18:14:38 +0000 Subject: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) In-Reply-To: <71ea732b-4566-e0a0-b2a3-e71df8a79609@oracle.com> References: <814DA850-76A9-4526-A3D9-141CD516C9AE@amperemail.onmicrosoft.com> <71ea732b-4566-e0a0-b2a3-e71df8a79609@oracle.com> Message-ID: <0E094A25-70C6-455D-A01D-F524A8010210@amperemail.onmicrosoft.com> Daniel, Thanks for the suggestion to run more tests. I'm not sure what the criteria for "success" is here. I have run $ jtreg-5.0-b01/bin/jtreg \ -verbose:summary -a -ea -esa -agentvm -conc:8 -ignore:quiet -timeout:2 \ -exclude:./test/jdk/ProblemList.txt \ -jdk:${jdk} \ ./test/jdk/:jdk_core ./test/jdk/:jdk_svc on aarch64 and x86_64, using the available OracleJDK build 15-ea+25-1229, and a build I did of jdk-15+25 with the patch (called jdk-15+25+ below). The results are ... confusing. I ran the tests 4 times on each JDK. The summary of the results on x86_64 are 15-ea+25-1229 Test results: passed: 6,718; failed: 4; error: 9 Test results: passed: 6,708; failed: 13; error: 10 Test results: passed: 6,713; failed: 9; error: 9 Test results: passed: 6,718; failed: 4; error: 9 jdk-15+25+ Test results: passed: 6,719; failed: 6; error: 9 Test results: passed: 6,712; failed: 12; error: 10 Test results: passed: 6,718; failed: 7; error: 9 Test results: passed: 6,718; failed: 7; error: 9 The fact that in different runs different tests pass, fail, or error is unfortunate. Counting the different failures in the 4 runs shows 15-ea+25-1229 3 FAILED: com/sun/jdi/PrivateTransportTest.java 1 FAILED: java/io/Serializable/unresolvableObjectStreamClass/UnresolvableObjectStreamClass.java 1 FAILED: java/net/httpclient/ProxyAuthDisabledSchemes.java 2 FAILED: java/net/httpclient/ProxyAuthDisabledSchemesSSL.java 1 FAILED: java/net/httpclient/ResponseBodyBeforeError.java 1 FAILED: java/net/httpclient/ResponsePublisher.java 2 FAILED: java/net/httpclient/ShortResponseBody.java 2 FAILED: java/net/httpclient/ShortResponseBodyWithRetry.java 1 FAILED: java/net/httpclient/SpecialHeadersTest.java 1 FAILED: java/net/httpclient/StreamingBody.java 4 FAILED: java/security/KeyStore/PKCS12/MetadataStoreLoadTest.java 1 FAILED: java/security/KeyStore/PKCS12/StoreTrustedCertAPITest.java 2 FAILED: java/security/KeyStore/PKCS12/StoreTrustedCertKeytool.java 4 FAILED: sun/security/pkcs11/Secmod/AddTrustedCert.java 4 FAILED: tools/jlink/plugins/StripNativeDebugSymbolsPlugin/StripNativeDebugSymbolsPlugin jdk-15+25+ 1 FAILED: com/sun/jdi/PrivateTransportTest.java 3 FAILED: java/io/Serializable/unresolvableObjectStreamClass/UnresolvableObjectStreamClass.java 1 FAILED: java/net/httpclient/ProxyAuthDisabledSchemes.java 1 FAILED: java/net/httpclient/ProxyAuthDisabledSchemesSSL.java 1 FAILED: java/net/httpclient/ResponsePublisher.java 1 FAILED: java/net/httpclient/ShortResponseBody.java 1 FAILED: java/net/httpclient/ShortResponseBodyWithRetry.java 1 FAILED: java/net/httpclient/SpecialHeadersTest.java 1 FAILED: java/net/httpclient/SplitResponseAsync.java 1 FAILED: java/net/httpclient/StreamingBody.java 1 FAILED: java/security/KeyStore/PKCS12/MetadataStoreLoadTest.java 3 FAILED: java/security/KeyStore/PKCS12/StoreTrustedCertAPITest.java 2 FAILED: java/security/KeyStore/PKCS12/StoreTrustedCertKeytool.java 1 FAILED: java/util/logging/Logger/bundleLeak/BundleTest.java 1 FAILED: jdk/jfr/event/runtime/TestThreadAllocationEvent.java 4 FAILED: sun/security/lib/cacerts/VerifyCACerts.java 4 FAILED: sun/security/pkcs11/Secmod/AddTrustedCert.java 4 FAILED: tools/jlink/plugins/StripNativeDebugSymbolsPlugin/StripNativeDebugSymbolsPluginTest.java To your worry about "potential damage", I would have worried about java/io/Serializable/unresolvableObjectStreamClass/UnresolvableObjectStreamClass.java, except it did not fail all the time with the patch, and it also failed without the patch. Many of the other failures seem intermittent. I am not sure how to classify the failures that persisted across the 4 runs I did. The summary of the results on aarch64 is not as good, for either build: 15-ea+25-1229 Test results: passed: 6,658; failed: 74; error: 10 Test results: passed: 6,657; failed: 74; error: 11 Test results: passed: 6,660; failed: 71; error: 10 Test results: passed: 6,663; failed: 69; error: 10 jdk-15+25+ Test results: passed: 6,661; failed: 73; error: 11 Test results: passed: 6,659; failed: 75; error: 10 Test results: passed: 6,663; failed: 71; error: 11 Test results: passed: 6,655; failed: 79; error: 11 Comparing the failures and errors in 3rd run of each, the differences from 15-ea+25-1229 to jdk-15+25+ are > Error: sun/tools/jstatd/TestJstatdExternalRegistry.java < FAILED: com/sun/jdi/PrivateTransportTest.java > FAILED: java/io/Serializable/unresolvableObjectStreamClass/UnresolvableObjectStreamClass.java < FAILED: java/security/KeyStore/PKCS12/MetadataStoreLoadTest.java < FAILED: java/security/KeyStore/PKCS12/StoreTrustedCertAPITest.java > FAILED: jdk/net/ExtendedSocketOption/AsynchronousSocketChannelNAPITest.java > FAILED: sun/security/lib/cacerts/VerifyCACerts.java Again, UnresolvableObjectStreamClass.java would be worrying, except that it did not always fail with the patch and also sometimes (not shown) failed with the OracleJDK 15-ea+25-1229 build. Comparing the failures across the 4 runs shows the difference between the persistent (that is, they failed on all 4 runs) between 15-ea+25-1229 and jdk-15+25+ to be < FAILED: com/sun/jdi/PrivateTransportTest.java < FAILED: java/security/KeyStore/PKCS12/StoreTrustedCertKeytool.java > FAILED: sun/security/lib/cacerts/VerifyCACerts.java Similar to, but not the same as, the persistent failures on x86_64. (Can I attribute the generally higher rate of failure on aarch64 compared to x86_64 to fact that aarch64 is a new platform for Oracle?) What further testing would you recommend? ... peter ?-----Original Message----- From: Daniel Fuchs Date: Tuesday, June 9, 2020 at 10:18 AM To: Peter Kessler OS , "core-libs-dev at openjdk.java.net" Subject: Re: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) Hi Peter, This is not a review. I second Roger's observations. On 05/06/2020 00:08, Peter Kessler OS wrote: > Tested with `make run-test-tier1` on Linux (CentOS 7) on both aarch64 and x86_64. I had one failure on each of the runs The code you are proposing to change affects RMI and JMX too. For an initial evaluation of the potential damage you will need to test at least with jdk_core and jdk_svc (and if that's successful then probably some of the lower tiers too): $ jtreg -verbose:summary -a -ea -esa -agentvm -conc:8 -ignore:quiet -timeout:2 -exclude:./test/jdk/ProblemList.txt -jdk: ./test/jdk/:jdk_core ./test/jdk/:jdk_svc best regards, -- daniel From peter.kessler at os.amperecomputing.com Thu Jun 11 18:52:31 2020 From: peter.kessler at os.amperecomputing.com (Peter Kessler OS) Date: Thu, 11 Jun 2020 18:52:31 +0000 Subject: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) In-Reply-To: References: <814DA850-76A9-4526-A3D9-141CD516C9AE@amperemail.onmicrosoft.com> Message-ID: <50955EDB-CCC2-4334-A4AF-2EB422E9CAA9@amperemail.onmicrosoft.com> Hi Roger, Thanks for your comments. I am not knowingly changing the required behavior. All I am proposing is guarding the thread stack walk if it will be a waste of effort. If an application or framework has its own ClassLoaders, then the guard will fail and the thread stack will be walked. The stack walk is important, if there are user-defined ClassLoaders. A method that uses ObjectInputStream to read objects can not know what ClassLoader to use to resolve class references in the stream, without the stack walk. But if there never have been any user-defined ClassLoaders constructed, then the stack walk can never return a user-defined ClassLoader. If the stream reader is part of some simple application, then it may spend a significant amount of its time uselessly walking thread stacks. (If the stream reader is part of some simple application its author may not know to override ObjectInputStream.resolveClass, or how to determine what ClassLoader should load a class from the stream.) If someone were using such an application to choose platforms, the Java platform would be at a disadvantage. Consider a somewhat more complex case, the SPECjbb2015 benchmark, which claims to model a well-constructed business application on the Java platform. SPECjbb2015 runs multiple JVMs, exchanging information via ObjectInputStreams. Profiling shows that it spends more than 5% of its time walking thread stacks. Yet, in spite of being a complex application, written by experts in the Java platform, there are no user-defined ClassLoaders. So all the time spent walking stacks is wasted time, and is one of the reasons the servers fail their service-level agreements. We can discuss whether SPECjbb2015 is representative of Java applications. But it is easily available and relatively simple to run. People use benchmarks like SPECjbb2015 to decide whether to use the Java platform for particular tasks, and it would be unfortunate if we did not make the Java platform as good as it could be. ... peter ?-----Original Message----- From: core-libs-dev on behalf of Roger Riggs Organization: Java Products Group, Oracle Date: Tuesday, June 9, 2020 at 6:43 AM To: "core-libs-dev at openjdk.java.net" Subject: Re: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) Hi Peter, I'd like to understand the scope and impact of the problem before jumping to a solution. For specific applications, overriding ObjectInputStream.resolveClass to handle the class lookup can handle the case as expeditiously as is necessary. What application uses cases are impacted and how prevelant are they? Many, many frameworks and applications create classloaders for their own purposes and take advantage of the required behavior. Thanks, Roger On 6/4/20 7:08 PM, Peter Kessler OS wrote: > ObjectInputStream.resolveClass(ObjectStreamClass) is specified to find "the first class loader on the current thread's stack (starting from the currently executing method) that is neither the platform class loader nor its ancestor." Finding that class loader is done with a virtual frame walk in JVM_LatestUserDefinedLoader(JNIEnv *env). The virtual frame walk is expensive. If an application does not define any ClassLoaders, then the virtual frame walk will always find the system class loader. > > If an application does a significant amount of reading from an ObjectInputStream, the cost of ObjectInputStream.resolveClass(ObjectInputStreamClass) can be considerable. In a sample application, that method (and the methods it calls) have been measured to consume 5+% of the CPU cycles. > > The proposal is to note the construction of any user-defined ClassLoader, and if none has been constructed, avoid the virtual frame walk returning ClassLoader.getSystemClassLoader(). But if any user-defined ClassLoader has been constructed, use the virtual frame walk to find the first user-defined ClassLoader on the thread stack. > > User-defined ClassLoaders could be distinguished from non-user-defined ClassLoaders in several ways. The ClassLoader class hierarchy could be changed to make non-user-defined ClassLoaders inherit from a marker class. There is already a jdk.internal.loader.BuiltinClassLoader but it is used for module (and resource) loading. BuiltinClassLoader is not a super-class of jdk.internal.reflect.DelegatingClassLoader, which is used to load reflection classes, and reflection methods are also ignored by the virtual frame walk. ClassLoaders could have a boolean that distinguished whether they are user-defined or not. That would require a change to ClassLoader API. There are other ways to distinguish user-defined ClassLoaders. The proposal is to keep the decoration of the ClassLoaders out of the instances, so that it is only visible to the code that needs it. > > The proposal is that each non-user-defined ClassLoader have a static block that registers the class as a non-user-defined ClassLoader. Then in the base ClassLoader constructor check if the ClassLoader being constructed is a user-defined ClassLoader and if so set a boolean that acts as the guard on the virtual frame walk. If additional non-user-defined ClassLoaders are declared in the future beyond the ones identified in this patch, they can be added to the registry. > > There are currently 4 non-user-defined ClassLoader classes, so the registry is not difficult to maintain. Nor is the registry difficult to search when ClassLoaders are constructed. The guard boolean that records whether a user-defined ClassLoader has been constructed transitions from false to true and never becomes false again. (An extension to transition the boolean back to false when a user-defined ClassLoader becomes unreachable is beyond the scope of this proposal.) > > This proposal slows loading of non-user-defined ClassLoaders by the time it takes to register them. It slows ClassLoader construction by the time it takes to consult the registry and conditionally set the guard boolean. At each invocation of ObjectInputStream.resolveClass(ObjectInputStreamClass), the guard boolean is used to possibly avoid the virtual frame walk. > > Tested with `make run-test-tier1` on Linux (CentOS 7) on both aarch64 and x86_64. I had one failure on each of the runs > > ACTION: main -- Failed. Execution failed: `main' threw exception: java.lang.Error: TESTBUG: the library has not been found in ${Repo}/build/linux-aarch64-server-release/images/test/hotspot/jtreg/native > REASON: User specified action: run main/native GTestWrapper > > on both plaforms. But I get the same failure when I run the tier1 tests on clones of OpenJDK-15+25 on both aarch64 and x86_64. > > Running a sample (representative?) application with the Oracle Developer Studio analyzer shows a performance comparison of > > Stack Fragment sorted by metric: Attributed Total CPU Time > baseline.er experiment.er > Attributed Attributed Name > Total CPU Time Total CPU Time > sec. sec. > ================================== Callers > 3459.210 29.931 java.io.ObjectInputStream.readOrdinaryObject(boolean) > 1139.727 3.532 java.io.ObjectInputStream.readArray(boolean) > 875.262 9.116 java.io.ObjectInputStream.readNonProxyDesc(boolean) > > ================================== Stack Fragment > java.io.ObjectInputStream.readClassDesc(boolean) > java.io.ObjectInputStream.readNonProxyDesc(boolean) > java.io.ObjectInputStream.resolveClass(java.io.ObjectStreamClass) > java.io.ObjectInputStream.latestUserDefinedLoader() > 4.173 3.953 jdk.internal.misc.VM.latestUserDefinedLoader() > > ================================== Callees > 5470.026 0. jdk.internal.misc.VM.latestUserDefinedLoader0() > 0. 38.627 java.lang.ClassLoader.getSystemClassLoader() > > The `hg export -g` below is with respect to OpenJDK-15+25. > > Thank you for your review comments. I will need a sponsor to get this change into the repository. > > ... peter > > # HG changeset patch > # User Peter Kessler > # Date 1591310467 25200 > # Thu Jun 04 15:41:07 2020 -0700 > # Node ID 9a39488182c1dfc5bc7bb41d562d7ef16ee657f6 > # Parent 90b266a84c06f1b3dc0ed8767856793e8c1c357e > Improve the performance of ObjectInputStream.resolveClass > > diff --git a/src/java.base/share/classes/java/lang/ClassLoader.java b/src/java.base/share/classes/java/lang/ClassLoader.java > --- a/src/java.base/share/classes/java/lang/ClassLoader.java > +++ b/src/java.base/share/classes/java/lang/ClassLoader.java > @@ -385,6 +385,8 @@ > } > this.package2certs = new ConcurrentHashMap<>(); > this.nameAndId = nameAndId(this); > + /* Note the construction of a ClassLoader. */ > + VM.noteClassLoaderConstruction(this.getClass()); > } > > /** > diff --git a/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java b/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java > --- a/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java > +++ b/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java > @@ -117,6 +117,11 @@ > protected Class loadClassOrNull(String cn, boolean resolve) { > return JLA.findBootstrapClassOrNull(this, cn); > } > + > + static { > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > + VM.registerNotUserDefinedClassLoader(BootClassLoader.class); > + } > }; > > /** > @@ -127,6 +132,8 @@ > static { > if (!ClassLoader.registerAsParallelCapable()) > throw new InternalError(); > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > + VM.registerNotUserDefinedClassLoader(PlatformClassLoader.class); > } > > PlatformClassLoader(BootClassLoader parent) { > @@ -142,6 +149,8 @@ > static { > if (!ClassLoader.registerAsParallelCapable()) > throw new InternalError(); > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > + VM.registerNotUserDefinedClassLoader(AppClassLoader.class); > } > > final URLClassPath ucp; > diff --git a/src/java.base/share/classes/jdk/internal/misc/VM.java b/src/java.base/share/classes/jdk/internal/misc/VM.java > --- a/src/java.base/share/classes/jdk/internal/misc/VM.java > +++ b/src/java.base/share/classes/jdk/internal/misc/VM.java > @@ -304,12 +304,44 @@ > private static final int JVMTI_THREAD_STATE_WAITING_INDEFINITELY = 0x0010; > private static final int JVMTI_THREAD_STATE_WAITING_WITH_TIMEOUT = 0x0020; > > + /** A registry of not user-defined ClassLoaders. */ > + private static final List> notUserDefinedClassLoaderRegistry = > + Collections.synchronizedList(new ArrayList<>()); > + > + /** Register a ClassLoader class as being not a user-defined ClassLoader. */ > + public static void registerNotUserDefinedClassLoader(Class classLoaderClass) { > + notUserDefinedClassLoaderRegistry.add(classLoaderClass); > + } > + > + /** A flag for whether a user-defined ClassLoaders has been constructed. */ > + private static volatile boolean constructedUserDefinedClassLoaderFlag = false; > + > + /** > + * Note a ClassLoader construction, and record if a > + * user-defined ClassLoader has been constructed. > + */ > + public static void noteClassLoaderConstruction(Class classLoaderClass) { > + /* Check if the ClassLoader class not a user-defined ClassLoader. */ > + if (notUserDefinedClassLoaderRegistry.contains(classLoaderClass)) { > + return; > + } > + constructedUserDefinedClassLoaderFlag = true; > + return; > + } > + > /* > * Returns the first user-defined class loader up the execution stack, > * or the platform class loader if only code from the platform or > * bootstrap class loader is on the stack. > */ > public static ClassLoader latestUserDefinedLoader() { > + if (!constructedUserDefinedClassLoaderFlag) { > + /* > + * If no user-defined ClassLoader has been constructed, > + * then I will not find a user-defined ClassLoader in the stack. > + */ > + return ClassLoader.getSystemClassLoader(); > + } > ClassLoader loader = latestUserDefinedLoader0(); > return loader != null ? loader : ClassLoader.getPlatformClassLoader(); > } > diff --git a/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java b/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java > --- a/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java > +++ b/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java > @@ -30,6 +30,7 @@ > > import jdk.internal.access.JavaLangAccess; > import jdk.internal.access.SharedSecrets; > +import jdk.internal.misc.VM; > > /** Utility class which assists in calling defineClass() by > creating a new class loader which delegates to the one needed in > @@ -73,4 +74,9 @@ > DelegatingClassLoader(ClassLoader parent) { > super(parent); > } > + > + static { > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > + VM.registerNotUserDefinedClassLoader(DelegatingClassLoader.class); > + } > } > From daniel.fuchs at oracle.com Thu Jun 11 19:31:34 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 11 Jun 2020 20:31:34 +0100 Subject: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) In-Reply-To: <0E094A25-70C6-455D-A01D-F524A8010210@amperemail.onmicrosoft.com> References: <814DA850-76A9-4526-A3D9-141CD516C9AE@amperemail.onmicrosoft.com> <71ea732b-4566-e0a0-b2a3-e71df8a79609@oracle.com> <0E094A25-70C6-455D-A01D-F524A8010210@amperemail.onmicrosoft.com> Message-ID: <04b1cf61-c1cf-89eb-bcf2-ddd0f69e9321@oracle.com> Hi Peter, I am surprised by the java/net/httpclient failures: these tests have been quite stable for me. Is your repository up to date? Or are these maybe failing in timeout, in which case you might need to increase the timeout factor (-timeout) or lower the concurrency (-conc). Or maybe you have some weird configuration in your /etc/hosts - issues have been known to happen if looking up "localhost" doesn't resolve to the loopback address. I any case I'd be curious to see the reason for those failures. best regards -- daniel On 11/06/2020 19:14, Peter Kessler OS wrote: > Daniel, > > Thanks for the suggestion to run more tests. > > I'm not sure what the criteria for "success" is here. > > I have run > > $ jtreg-5.0-b01/bin/jtreg \ > -verbose:summary -a -ea -esa -agentvm -conc:8 -ignore:quiet -timeout:2 \ > -exclude:./test/jdk/ProblemList.txt \ > -jdk:${jdk} \ > ./test/jdk/:jdk_core ./test/jdk/:jdk_svc > > on aarch64 and x86_64, using the available OracleJDK build 15-ea+25-1229, > and a build I did of jdk-15+25 with the patch (called jdk-15+25+ below). > The results are ... confusing. > > I ran the tests 4 times on each JDK. The summary of the results on x86_64 > are > > 15-ea+25-1229 > Test results: passed: 6,718; failed: 4; error: 9 > Test results: passed: 6,708; failed: 13; error: 10 > Test results: passed: 6,713; failed: 9; error: 9 > Test results: passed: 6,718; failed: 4; error: 9 > > jdk-15+25+ > Test results: passed: 6,719; failed: 6; error: 9 > Test results: passed: 6,712; failed: 12; error: 10 > Test results: passed: 6,718; failed: 7; error: 9 > Test results: passed: 6,718; failed: 7; error: 9 > > The fact that in different runs different tests pass, fail, or error is > unfortunate. Counting the different failures in the 4 runs shows > > 15-ea+25-1229 > 3 FAILED: com/sun/jdi/PrivateTransportTest.java > 1 FAILED: java/io/Serializable/unresolvableObjectStreamClass/UnresolvableObjectStreamClass.java > 1 FAILED: java/net/httpclient/ProxyAuthDisabledSchemes.java > 2 FAILED: java/net/httpclient/ProxyAuthDisabledSchemesSSL.java > 1 FAILED: java/net/httpclient/ResponseBodyBeforeError.java > 1 FAILED: java/net/httpclient/ResponsePublisher.java > 2 FAILED: java/net/httpclient/ShortResponseBody.java > 2 FAILED: java/net/httpclient/ShortResponseBodyWithRetry.java > 1 FAILED: java/net/httpclient/SpecialHeadersTest.java > 1 FAILED: java/net/httpclient/StreamingBody.java > 4 FAILED: java/security/KeyStore/PKCS12/MetadataStoreLoadTest.java > 1 FAILED: java/security/KeyStore/PKCS12/StoreTrustedCertAPITest.java > 2 FAILED: java/security/KeyStore/PKCS12/StoreTrustedCertKeytool.java > 4 FAILED: sun/security/pkcs11/Secmod/AddTrustedCert.java > 4 FAILED: tools/jlink/plugins/StripNativeDebugSymbolsPlugin/StripNativeDebugSymbolsPlugin > > jdk-15+25+ > 1 FAILED: com/sun/jdi/PrivateTransportTest.java > 3 FAILED: java/io/Serializable/unresolvableObjectStreamClass/UnresolvableObjectStreamClass.java > 1 FAILED: java/net/httpclient/ProxyAuthDisabledSchemes.java > 1 FAILED: java/net/httpclient/ProxyAuthDisabledSchemesSSL.java > 1 FAILED: java/net/httpclient/ResponsePublisher.java > 1 FAILED: java/net/httpclient/ShortResponseBody.java > 1 FAILED: java/net/httpclient/ShortResponseBodyWithRetry.java > 1 FAILED: java/net/httpclient/SpecialHeadersTest.java > 1 FAILED: java/net/httpclient/SplitResponseAsync.java > 1 FAILED: java/net/httpclient/StreamingBody.java > 1 FAILED: java/security/KeyStore/PKCS12/MetadataStoreLoadTest.java > 3 FAILED: java/security/KeyStore/PKCS12/StoreTrustedCertAPITest.java > 2 FAILED: java/security/KeyStore/PKCS12/StoreTrustedCertKeytool.java > 1 FAILED: java/util/logging/Logger/bundleLeak/BundleTest.java > 1 FAILED: jdk/jfr/event/runtime/TestThreadAllocationEvent.java > 4 FAILED: sun/security/lib/cacerts/VerifyCACerts.java > 4 FAILED: sun/security/pkcs11/Secmod/AddTrustedCert.java > 4 FAILED: tools/jlink/plugins/StripNativeDebugSymbolsPlugin/StripNativeDebugSymbolsPluginTest.java > > To your worry about "potential damage", I would have worried about > java/io/Serializable/unresolvableObjectStreamClass/UnresolvableObjectStreamClass.java, > except it did not fail all the time with the patch, and it also failed > without the patch. Many of the other failures seem intermittent. I am not > sure how to classify the failures that persisted across the 4 runs I did. > > The summary of the results on aarch64 is not as good, for either build: > > 15-ea+25-1229 > Test results: passed: 6,658; failed: 74; error: 10 > Test results: passed: 6,657; failed: 74; error: 11 > Test results: passed: 6,660; failed: 71; error: 10 > Test results: passed: 6,663; failed: 69; error: 10 > > jdk-15+25+ > Test results: passed: 6,661; failed: 73; error: 11 > Test results: passed: 6,659; failed: 75; error: 10 > Test results: passed: 6,663; failed: 71; error: 11 > Test results: passed: 6,655; failed: 79; error: 11 > > Comparing the failures and errors in 3rd run of each, the differences from > 15-ea+25-1229 to jdk-15+25+ are > > > Error: sun/tools/jstatd/TestJstatdExternalRegistry.java > < FAILED: com/sun/jdi/PrivateTransportTest.java > > FAILED: java/io/Serializable/unresolvableObjectStreamClass/UnresolvableObjectStreamClass.java > < FAILED: java/security/KeyStore/PKCS12/MetadataStoreLoadTest.java > < FAILED: java/security/KeyStore/PKCS12/StoreTrustedCertAPITest.java > > FAILED: jdk/net/ExtendedSocketOption/AsynchronousSocketChannelNAPITest.java > > FAILED: sun/security/lib/cacerts/VerifyCACerts.java > > Again, UnresolvableObjectStreamClass.java would be worrying, except that it > did not always fail with the patch and also sometimes (not shown) failed > with the OracleJDK 15-ea+25-1229 build. > > Comparing the failures across the 4 runs shows the difference between the > persistent (that is, they failed on all 4 runs) between 15-ea+25-1229 and > jdk-15+25+ to be > > < FAILED: com/sun/jdi/PrivateTransportTest.java > < FAILED: java/security/KeyStore/PKCS12/StoreTrustedCertKeytool.java > > FAILED: sun/security/lib/cacerts/VerifyCACerts.java > > Similar to, but not the same as, the persistent failures on x86_64. > > (Can I attribute the generally higher rate of failure on aarch64 compared to > x86_64 to fact that aarch64 is a new platform for Oracle?) > > What further testing would you recommend? > > ... peter > > ?-----Original Message----- > From: Daniel Fuchs > Date: Tuesday, June 9, 2020 at 10:18 AM > To: Peter Kessler OS , "core-libs-dev at openjdk.java.net" > Subject: Re: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) > > Hi Peter, > > This is not a review. I second Roger's observations. > > On 05/06/2020 00:08, Peter Kessler OS wrote: > > Tested with `make run-test-tier1` on Linux (CentOS 7) on both aarch64 and x86_64. I had one failure on each of the runs > > The code you are proposing to change affects RMI and JMX too. > For an initial evaluation of the potential damage you will need to > test at least with jdk_core and jdk_svc (and if that's successful > then probably some of the lower tiers too): > > $ jtreg -verbose:summary -a -ea -esa -agentvm -conc:8 -ignore:quiet > -timeout:2 -exclude:./test/jdk/ProblemList.txt -jdk: > ./test/jdk/:jdk_core ./test/jdk/:jdk_svc > > > best regards, > > -- daniel > From andy.herrick at oracle.com Thu Jun 11 20:19:19 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Thu, 11 Jun 2020 16:19:19 -0400 Subject: RFR: JDK-8247422: --runtime-image on Mac should work for runtime root In-Reply-To: <5617c980-cff8-6e64-cb73-6d2183a75462@oracle.com> References: <4efc0137-d2fb-cf69-1fac-3e871d5eedc4@oracle.com> <5617c980-cff8-6e64-cb73-6d2183a75462@oracle.com> Message-ID: <9453d6be-7ce2-16f9-bac0-74e92661f29c@oracle.com> looks good. /Andy On 6/11/2020 1:23 PM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > Restore handling not only Java runtime home but also Java runtime root > directory in `--runtime-image` option on Mac. > The patch also fixes [3] issue as a side effect of moving > functionality of BasicTest.testJLinkRuntime() function into the new > CookedRuntimeTest.test(). > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8247422 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8247422/webrev.00 > > [3] https://bugs.openjdk.java.net/browse/JDK-8247424 From andy.herrick at oracle.com Thu Jun 11 20:22:28 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Thu, 11 Jun 2020 16:22:28 -0400 Subject: RFR: JDK-8247418: Only validate the certificates trust if using the default key user name. Message-ID: <97e73bff-4454-4020-b4c8-f6d98ffd743b@oracle.com> Please review the small jpackage test fix at [1] to issue [2]. [1] http://cr.openjdk.java.net/~herrick/8247418/webrev.01/ [2] https://bugs.openjdk.java.net/browse/JDK-8247418 /Andy From alexey.semenyuk at oracle.com Thu Jun 11 20:33:28 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Thu, 11 Jun 2020 16:33:28 -0400 Subject: RFR: JDK-8247418: Only validate the certificates trust if using the default key user name. In-Reply-To: <97e73bff-4454-4020-b4c8-f6d98ffd743b@oracle.com> References: <97e73bff-4454-4020-b4c8-f6d98ffd743b@oracle.com> Message-ID: Looks good. - Alexey On 6/11/2020 4:22 PM, Andy Herrick wrote: > Please review the small jpackage test fix at [1] to issue [2]. > > [1] http://cr.openjdk.java.net/~herrick/8247418/webrev.01/ > > [2] https://bugs.openjdk.java.net/browse/JDK-8247418 > > /Andy > From alexander.matveev at oracle.com Thu Jun 11 22:06:37 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Thu, 11 Jun 2020 15:06:37 -0700 Subject: RFR: JDK-8247422: --runtime-image on Mac should work for runtime root In-Reply-To: <9453d6be-7ce2-16f9-bac0-74e92661f29c@oracle.com> References: <4efc0137-d2fb-cf69-1fac-3e871d5eedc4@oracle.com> <5617c980-cff8-6e64-cb73-6d2183a75462@oracle.com> <9453d6be-7ce2-16f9-bac0-74e92661f29c@oracle.com> Message-ID: <41be40b6-5555-ee08-da15-27274891bd76@oracle.com> Looks good. Thanks, Alexander On 6/11/20 1:19 PM, Andy Herrick wrote: > looks good. > > /Andy > > On 6/11/2020 1:23 PM, Alexey Semenyuk wrote: >> Please review fix [2] for jpackage bug [1]. >> >> Restore handling not only Java runtime home but also Java runtime >> root directory in `--runtime-image` option on Mac. >> The patch also fixes [3] issue as a side effect of moving >> functionality of BasicTest.testJLinkRuntime() function into the new >> CookedRuntimeTest.test(). >> >> - Alexey >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8247422 >> >> [2]?http://cr.openjdk.java.net/~asemenyuk/8247422/webrev.00 >> >> [3] https://bugs.openjdk.java.net/browse/JDK-8247424 From alexander.matveev at oracle.com Thu Jun 11 22:11:55 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Thu, 11 Jun 2020 15:11:55 -0700 Subject: RFR: JDK-8247418: Only validate the certificates trust if using the default key user name. In-Reply-To: <97e73bff-4454-4020-b4c8-f6d98ffd743b@oracle.com> References: <97e73bff-4454-4020-b4c8-f6d98ffd743b@oracle.com> Message-ID: <8cefd52b-c228-0083-b74e-f485a93c672e@oracle.com> Hi Andy, Looks good. Thanks, Alexander On 6/11/20 1:22 PM, Andy Herrick wrote: > Please review the small jpackage test fix at [1] to issue [2]. > > [1] http://cr.openjdk.java.net/~herrick/8247418/webrev.01/ > > [2] https://bugs.openjdk.java.net/browse/JDK-8247418 > > /Andy > From tigran.mkrtchyan at desy.de Wed Jun 10 14:45:08 2020 From: tigran.mkrtchyan at desy.de (Mkrtchyan, Tigran) Date: Wed, 10 Jun 2020 16:45:08 +0200 (CEST) Subject: Thread leak by LdapLoginModule In-Reply-To: <3c1da905-0654-1932-0b49-d257bf661d82@oracle.com> References: <889628466.1257399.1591737319316.JavaMail.zimbra@desy.de> <5e5f3f91-bdc2-7af2-7b41-894484de7f61@oracle.com> <3c1da905-0654-1932-0b49-d257bf661d82@oracle.com> Message-ID: <717334161.1433632.1591800308211.JavaMail.zimbra@desy.de> Hi, found it: https://bugs.openjdk.java.net/browse/JDK-8237876 Thanks, Tigran. ----- Original Message ----- > From: "Daniel Fuchs" > To: "Sean Mullan" , "Tigran Mkrtchyan" , "security-dev" > > Cc: "core-libs-dev" > Sent: Wednesday, June 10, 2020 4:29:36 PM > Subject: Re: Thread leak by LdapLoginModule > On 09/06/2020 23:21, Sean Mullan wrote: >>> The issue is not observed with java-14, thus I assume that the fix is >>> related to commit >>> >>> http://hg.openjdk.java.net/jdk/jdk/rev/6717d7e59db4 >>> >>> As java-11 is LTS, what is the procedure to get it fix back-ported? > > Hi, > > AFAICS the fix has already been backported to JDK 11.0.8. > > best regards, > > -- daniel From johnlinp at gmail.com Fri Jun 12 01:33:11 2020 From: johnlinp at gmail.com (=?UTF-8?B?5p6X6Ieq5Z2H?=) Date: Fri, 12 Jun 2020 09:33:11 +0800 Subject: [PATCH] 8247402: Rewrite the implementation requirements for Map::compute() Message-ID: Hi all, This is my first time contribution. Please let me know if I did something wrong. Thanks. Regards I'm working on this bug: https://bugs.openjdk.java.net/browse/JDK-8247402 The original problem is that the implementation requirements for Map::compute() lacks of return statements for some cases. However, I found that there are some other problems in it: 1. The indents are 3 spaces, while most of the indents are 4 spaces. 2. The if-else is overly complicated and can be simplified. My proposed patch that generated by `hg export -g` is: # HG changeset patch # User John Lin # Date 1591923561 -28800 # Fri Jun 12 08:59:21 2020 +0800 # Node ID 03c9b5c9e632a0d6e33a1f13c98bb3b31b1bf659 # Parent 49a68abdb0ba68351db0f140ddac793b1c391bd5 8247402: Rewrite the implementation requirements for Map::compute() diff --git a/src/java.base/share/classes/java/util/Map.java b/src/java.base/share/classes/java/util/Map.java --- a/src/java.base/share/classes/java/util/Map.java +++ b/src/java.base/share/classes/java/util/Map.java @@ -1113,17 +1113,12 @@ public interface Map { *
       {@code
            * V oldValue = map.get(key);
            * V newValue = remappingFunction.apply(key, oldValue);
      -     * if (oldValue != null) {
      -     *    if (newValue != null)
      -     *       map.put(key, newValue);
      -     *    else
      -     *       map.remove(key);
      -     * } else {
      -     *    if (newValue != null)
      -     *       map.put(key, newValue);
      -     *    else
      -     *       return null;
      +     * if (newValue != null) {
      +     *     map.put(key, newValue);
      +     * } else if (oldValue != null) {
      +     *     map.remove(key);
            * }
      +     * return newValue;
            * }
      * *

      The default implementation makes no guarantees about detecting if the Best, John Lin From martinrb at google.com Fri Jun 12 03:42:08 2020 From: martinrb at google.com (Martin Buchholz) Date: Thu, 11 Jun 2020 20:42:08 -0700 Subject: [PATCH] 8247402: Rewrite the implementation requirements for Map::compute() In-Reply-To: References: Message-ID: ???, pleased to meet you! This rewrite seems clear and clean. Would it be even clearer if we did if (newValue != null) { map.put(key, newValue); } else { map.remove(key); } Hmmm, what about the corner case where the map permits null values and the old value is null? On Thu, Jun 11, 2020 at 7:45 PM ??? wrote: > > Hi all, > > This is my first time contribution. Please let me know if I did > something wrong. Thanks. Regards > > I'm working on this bug: https://bugs.openjdk.java.net/browse/JDK-8247402 > > The original problem is that the implementation requirements for > Map::compute() lacks of return statements for some cases. However, I > found that there are some other problems in it: > > 1. The indents are 3 spaces, while most of the indents are 4 spaces. > 2. The if-else is overly complicated and can be simplified. > > My proposed patch that generated by `hg export -g` is: > > # HG changeset patch > # User John Lin > # Date 1591923561 -28800 > # Fri Jun 12 08:59:21 2020 +0800 > # Node ID 03c9b5c9e632a0d6e33a1f13c98bb3b31b1bf659 > # Parent 49a68abdb0ba68351db0f140ddac793b1c391bd5 > 8247402: Rewrite the implementation requirements for Map::compute() > > diff --git a/src/java.base/share/classes/java/util/Map.java > b/src/java.base/share/classes/java/util/Map.java > --- a/src/java.base/share/classes/java/util/Map.java > +++ b/src/java.base/share/classes/java/util/Map.java > @@ -1113,17 +1113,12 @@ public interface Map { > *

       {@code
      >       * V oldValue = map.get(key);
      >       * V newValue = remappingFunction.apply(key, oldValue);
      > -     * if (oldValue != null) {
      > -     *    if (newValue != null)
      > -     *       map.put(key, newValue);
      > -     *    else
      > -     *       map.remove(key);
      > -     * } else {
      > -     *    if (newValue != null)
      > -     *       map.put(key, newValue);
      > -     *    else
      > -     *       return null;
      > +     * if (newValue != null) {
      > +     *     map.put(key, newValue);
      > +     * } else if (oldValue != null) {
      > +     *     map.remove(key);
      >       * }
      > +     * return newValue;
      >       * }
      > * > *

      The default implementation makes no guarantees about detecting if the > > Best, > John Lin From johnlinp at gmail.com Fri Jun 12 05:05:58 2020 From: johnlinp at gmail.com (=?UTF-8?B?5p6X6Ieq5Z2H?=) Date: Fri, 12 Jun 2020 13:05:58 +0800 Subject: [PATCH] 8247402: Rewrite the implementation requirements for Map::compute() In-Reply-To: References: Message-ID: Hi Martin, Thanks for the review! However, the corner case you mentioned seems to be related to the design choice of Map::compute(), and it's out of the scope of this change. What do you think? Best, John Lin Martin Buchholz ? 2020?6?12? ?? ??11:42??? > > ???, pleased to meet you! > > This rewrite seems clear and clean. > > Would it be even clearer if we did > > if (newValue != null) { > map.put(key, newValue); > } else { > map.remove(key); > } > > Hmmm, what about the corner case where the map permits null values and > the old value is null? > > On Thu, Jun 11, 2020 at 7:45 PM ??? wrote: > > > > Hi all, > > > > This is my first time contribution. Please let me know if I did > > something wrong. Thanks. Regards > > > > I'm working on this bug: https://bugs.openjdk.java.net/browse/JDK-8247402 > > > > The original problem is that the implementation requirements for > > Map::compute() lacks of return statements for some cases. However, I > > found that there are some other problems in it: > > > > 1. The indents are 3 spaces, while most of the indents are 4 spaces. > > 2. The if-else is overly complicated and can be simplified. > > > > My proposed patch that generated by `hg export -g` is: > > > > # HG changeset patch > > # User John Lin > > # Date 1591923561 -28800 > > # Fri Jun 12 08:59:21 2020 +0800 > > # Node ID 03c9b5c9e632a0d6e33a1f13c98bb3b31b1bf659 > > # Parent 49a68abdb0ba68351db0f140ddac793b1c391bd5 > > 8247402: Rewrite the implementation requirements for Map::compute() > > > > diff --git a/src/java.base/share/classes/java/util/Map.java > > b/src/java.base/share/classes/java/util/Map.java > > --- a/src/java.base/share/classes/java/util/Map.java > > +++ b/src/java.base/share/classes/java/util/Map.java > > @@ -1113,17 +1113,12 @@ public interface Map { > > *

       {@code
      > >       * V oldValue = map.get(key);
      > >       * V newValue = remappingFunction.apply(key, oldValue);
      > > -     * if (oldValue != null) {
      > > -     *    if (newValue != null)
      > > -     *       map.put(key, newValue);
      > > -     *    else
      > > -     *       map.remove(key);
      > > -     * } else {
      > > -     *    if (newValue != null)
      > > -     *       map.put(key, newValue);
      > > -     *    else
      > > -     *       return null;
      > > +     * if (newValue != null) {
      > > +     *     map.put(key, newValue);
      > > +     * } else if (oldValue != null) {
      > > +     *     map.remove(key);
      > >       * }
      > > +     * return newValue;
      > >       * }
      > > * > > *

      The default implementation makes no guarantees about detecting if the > > > > Best, > > John Lin From martinrb at google.com Fri Jun 12 06:41:58 2020 From: martinrb at google.com (Martin Buchholz) Date: Thu, 11 Jun 2020 23:41:58 -0700 Subject: [PATCH] 8247402: Rewrite the implementation requirements for Map::compute() In-Reply-To: References: Message-ID: import java.util.HashMap; import java.util.Map; import java.util.function.BiFunction; @SuppressWarnings("serial") public class MapsComputeNull { static class HashMap1 extends HashMap { @Override public V compute( K key, BiFunction remappingFunction) { final Map map = this; V oldValue = map.get(key); V newValue = remappingFunction.apply(key, oldValue); if (newValue != null) { map.put(key, newValue); } else { map.remove(key); } return newValue; } } static class HashMap2 extends HashMap { @Override public V compute( K key, BiFunction remappingFunction) { final Map map = this; V oldValue = map.get(key); V newValue = remappingFunction.apply(key, oldValue); if (newValue != null) { map.put(key, newValue); } else if (oldValue != null) { map.remove(key); } return newValue; } } public static void main(String[] args) throws Throwable { test(new HashMap()); test(new HashMap1()); test(new HashMap2()); } static void test(Map map) { Object key = 42; map.put(key, null); map.compute(key, (k, v) -> null); System.out.printf("%s %s%n", map.getClass().getSimpleName(), map.containsKey(key)); } } On Thu, Jun 11, 2020 at 10:06 PM ??? wrote: > > Hi Martin, > > Thanks for the review! However, the corner case you mentioned seems to > be related to the design choice of Map::compute(), and it's out of the > scope of this change. What do you think? > > Best, > John Lin > > Martin Buchholz ? 2020?6?12? ?? ??11:42??? > > > > ???, pleased to meet you! > > > > This rewrite seems clear and clean. > > > > Would it be even clearer if we did > > > > if (newValue != null) { > > map.put(key, newValue); > > } else { > > map.remove(key); > > } > > > > Hmmm, what about the corner case where the map permits null values and > > the old value is null? > > > > On Thu, Jun 11, 2020 at 7:45 PM ??? wrote: > > > > > > Hi all, > > > > > > This is my first time contribution. Please let me know if I did > > > something wrong. Thanks. Regards > > > > > > I'm working on this bug: https://bugs.openjdk.java.net/browse/JDK-8247402 > > > > > > The original problem is that the implementation requirements for > > > Map::compute() lacks of return statements for some cases. However, I > > > found that there are some other problems in it: > > > > > > 1. The indents are 3 spaces, while most of the indents are 4 spaces. > > > 2. The if-else is overly complicated and can be simplified. > > > > > > My proposed patch that generated by `hg export -g` is: > > > > > > # HG changeset patch > > > # User John Lin > > > # Date 1591923561 -28800 > > > # Fri Jun 12 08:59:21 2020 +0800 > > > # Node ID 03c9b5c9e632a0d6e33a1f13c98bb3b31b1bf659 > > > # Parent 49a68abdb0ba68351db0f140ddac793b1c391bd5 > > > 8247402: Rewrite the implementation requirements for Map::compute() > > > > > > diff --git a/src/java.base/share/classes/java/util/Map.java > > > b/src/java.base/share/classes/java/util/Map.java > > > --- a/src/java.base/share/classes/java/util/Map.java > > > +++ b/src/java.base/share/classes/java/util/Map.java > > > @@ -1113,17 +1113,12 @@ public interface Map { > > > *

       {@code
      > > >       * V oldValue = map.get(key);
      > > >       * V newValue = remappingFunction.apply(key, oldValue);
      > > > -     * if (oldValue != null) {
      > > > -     *    if (newValue != null)
      > > > -     *       map.put(key, newValue);
      > > > -     *    else
      > > > -     *       map.remove(key);
      > > > -     * } else {
      > > > -     *    if (newValue != null)
      > > > -     *       map.put(key, newValue);
      > > > -     *    else
      > > > -     *       return null;
      > > > +     * if (newValue != null) {
      > > > +     *     map.put(key, newValue);
      > > > +     * } else if (oldValue != null) {
      > > > +     *     map.remove(key);
      > > >       * }
      > > > +     * return newValue;
      > > >       * }
      > > > * > > > *

      The default implementation makes no guarantees about detecting if the > > > > > > Best, > > > John Lin From johnlinp at gmail.com Fri Jun 12 08:58:44 2020 From: johnlinp at gmail.com (=?UTF-8?B?5p6X6Ieq5Z2H?=) Date: Fri, 12 Jun 2020 16:58:44 +0800 Subject: [PATCH] 8247402: Rewrite the implementation requirements for Map::compute() In-Reply-To: References: Message-ID: Hi Martin, May I ask what do you mean by pasting this pice of code? Sorry I didn't get it. Best, John Lin Martin Buchholz ? 2020?6?12? ?? ??2:42??? > > import java.util.HashMap; > import java.util.Map; > import java.util.function.BiFunction; > > @SuppressWarnings("serial") > public class MapsComputeNull { > > static class HashMap1 extends HashMap { > @Override public V compute( > K key, > BiFunction > remappingFunction) { > final Map map = this; > V oldValue = map.get(key); > V newValue = remappingFunction.apply(key, oldValue); > if (newValue != null) { > map.put(key, newValue); > } else { > map.remove(key); > } > return newValue; > } > } > > static class HashMap2 extends HashMap { > @Override public V compute( > K key, > BiFunction > remappingFunction) { > final Map map = this; > V oldValue = map.get(key); > V newValue = remappingFunction.apply(key, oldValue); > if (newValue != null) { > map.put(key, newValue); > } else if (oldValue != null) { > map.remove(key); > } > return newValue; > } > } > > public static void main(String[] args) throws Throwable { > test(new HashMap()); > test(new HashMap1()); > test(new HashMap2()); > } > > static void test(Map map) { > Object key = 42; > map.put(key, null); > map.compute(key, (k, v) -> null); > System.out.printf("%s %s%n", > map.getClass().getSimpleName(), > map.containsKey(key)); > } > } > > On Thu, Jun 11, 2020 at 10:06 PM ??? wrote: > > > > Hi Martin, > > > > Thanks for the review! However, the corner case you mentioned seems to > > be related to the design choice of Map::compute(), and it's out of the > > scope of this change. What do you think? > > > > Best, > > John Lin > > > > Martin Buchholz ? 2020?6?12? ?? ??11:42??? > > > > > > ???, pleased to meet you! > > > > > > This rewrite seems clear and clean. > > > > > > Would it be even clearer if we did > > > > > > if (newValue != null) { > > > map.put(key, newValue); > > > } else { > > > map.remove(key); > > > } > > > > > > Hmmm, what about the corner case where the map permits null values and > > > the old value is null? > > > > > > On Thu, Jun 11, 2020 at 7:45 PM ??? wrote: > > > > > > > > Hi all, > > > > > > > > This is my first time contribution. Please let me know if I did > > > > something wrong. Thanks. Regards > > > > > > > > I'm working on this bug: https://bugs.openjdk.java.net/browse/JDK-8247402 > > > > > > > > The original problem is that the implementation requirements for > > > > Map::compute() lacks of return statements for some cases. However, I > > > > found that there are some other problems in it: > > > > > > > > 1. The indents are 3 spaces, while most of the indents are 4 spaces. > > > > 2. The if-else is overly complicated and can be simplified. > > > > > > > > My proposed patch that generated by `hg export -g` is: > > > > > > > > # HG changeset patch > > > > # User John Lin > > > > # Date 1591923561 -28800 > > > > # Fri Jun 12 08:59:21 2020 +0800 > > > > # Node ID 03c9b5c9e632a0d6e33a1f13c98bb3b31b1bf659 > > > > # Parent 49a68abdb0ba68351db0f140ddac793b1c391bd5 > > > > 8247402: Rewrite the implementation requirements for Map::compute() > > > > > > > > diff --git a/src/java.base/share/classes/java/util/Map.java > > > > b/src/java.base/share/classes/java/util/Map.java > > > > --- a/src/java.base/share/classes/java/util/Map.java > > > > +++ b/src/java.base/share/classes/java/util/Map.java > > > > @@ -1113,17 +1113,12 @@ public interface Map { > > > > *

       {@code
      > > > >       * V oldValue = map.get(key);
      > > > >       * V newValue = remappingFunction.apply(key, oldValue);
      > > > > -     * if (oldValue != null) {
      > > > > -     *    if (newValue != null)
      > > > > -     *       map.put(key, newValue);
      > > > > -     *    else
      > > > > -     *       map.remove(key);
      > > > > -     * } else {
      > > > > -     *    if (newValue != null)
      > > > > -     *       map.put(key, newValue);
      > > > > -     *    else
      > > > > -     *       return null;
      > > > > +     * if (newValue != null) {
      > > > > +     *     map.put(key, newValue);
      > > > > +     * } else if (oldValue != null) {
      > > > > +     *     map.remove(key);
      > > > >       * }
      > > > > +     * return newValue;
      > > > >       * }
      > > > > * > > > > *

      The default implementation makes no guarantees about detecting if the > > > > > > > > Best, > > > > John Lin From daniel.fuchs at oracle.com Fri Jun 12 09:20:32 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 12 Jun 2020 10:20:32 +0100 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <228BA21D-836A-46D7-B273-738AFFE639E4@azul.com> References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> <9013C8D0-33D9-4D09-8953-F8618020A7F7@oracle.com> <8A2C4DEF-FC61-4574-B49D-1133F5349BCE@azul.com> <15fa46f2-0352-a111-a9c7-0d5fc31d9159@oracle.com> <6167ea24-e7f9-c259-e2c5-8be96d74a40d@oracle.com> <228BA21D-836A-46D7-B273-738AFFE639E4@azul.com> Message-ID: <97a4f719-e0f5-96f7-6637-432b4b746685@oracle.com> Hi Alexey, This is starting to look good. Thank you for persisting! I have a couple of comments - on the LDAP/JNDI side. There are several places where your new code is supposed to trigger the throwing of a NamingException: LdapSasl.java: lines 76, 85, 140, 168 Please write a test to verify that it does so. Since the exceptions are all supposed to be thrown before the actual binding happens, there's no need to have an actual LDAP server that supports any kind of authentication to test that. The simple dummy servers that we have in the test base should be enough to write such a test. In addition, there are a couple of places where stray exceptions could theoretically be thrown, and where they should be wrapped in NamingException (but are not). Maybe it's OK, because this has been checked before - but it would be better if we had a test that proves that it is so: For instance: LdapSasl.java 82 connectTimeout = Integer.parseInt((String)propVal); What if the value of the propVal is "Foo" - or not a String? What unexpected exception might be relayed to the calling code? Still in that same file: 84 if (connectTimeout == -1) should probably be if (connectTimeout <= 0) since Connection.java checks for connectTimeout > 0 to determine whether to start the initial handshake. Which makes me wonder if we should find a better way to instruct Connection to start the initial handshake... TlsChannelBinding.java: It would be much better if static factories methods were used instead of public constructors. That would allow you to check all arguments before actually constructing your object: public static TlsChannelBinding create((byte[] finishedMessage) throws SaslException { throw new UnsupportedOperationException("tls-unique channel binding is not supported"); } public statuic TlsChannelBinding create(X509Certificate serverCertificate) throws SaslException { var cbType = TlsChannelBindingType.TLS_SERVER_END_POINT; byte[] cbData; try { // compute cbdata ... return new TlsChannelBinding(cbType, cbData); } private TlsChannelBinding(TlsChannelBindingType cbType, byte[] cbData) { this.cbType = cbType; this.cbData = cbData; } That's all I have for now. best regards, -- daniel On 09/06/2020 22:03, Alexey Bakhtin wrote: > Hello Aleks, > > Thank you very much for review. > > I?ve fixed missed spaces and removed casting from LdapSasl.java > Failure of the SaslMutual test was caused by prop.remove() in the GssKrb5Client > This operation is not required any more. GssKrb5Client receives temporary copy of the properties. Fixed > > Also, I?ve added references to RFC-5929 and RFC-5056 into the TlsChannelBinding class > > Updated patch is located at: > http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v7/ > > Regards > Alexey From sean.coffey at oracle.com Fri Jun 12 10:21:18 2020 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Fri, 12 Jun 2020 11:21:18 +0100 Subject: RFR: 8218021: jarsigner strips the execute permission when signing a .zip file Message-ID: <6a34a99c-ed06-5e38-0932-788e944a70df@oracle.com> Hi, I'd like to reboot this jarsigner enhancement request[1]. I've removed the problem references to zip file name extensions. Instead, there's a new JDK implementation specific jarsigner option: -keepposixperms https://bugs.openjdk.java.net/browse/JDK-8218021 https://cr.openjdk.java.net/~coffeys/webrev.8218021.v2/webrev/ regards, Sean. [1] http://mail.openjdk.java.net/pipermail/security-dev/2020-January/021141.html From matthias.baesken at sap.com Fri Jun 12 15:01:19 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Fri, 12 Jun 2020 15:01:19 +0000 Subject: RFR: 8247469: getSystemCpuLoad() returns -1 on linux when some offline cpus are present and cpusets.effective_cpus is not available Message-ID: Hello, please review the following change . We have a Linux machine where OperatingSystemMXBean mbean = (com.sun.management.OperatingSystemMXBean) ManagementFactory.getOperatingSystemMXBean(); double load = mbean.getSystemCpuLoad(); returns -1 ; Reason is that there are offline CPUs (48 configured , 32 online ). Additionally cpusets.effective_cpus is not available on that Linux system . Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8247466 http://cr.openjdk.java.net/~mbaesken/webrevs/8247469.0/ Thanks, Matthias -----Original Message----- From: Bob Vandette Sent: Freitag, 12. Juni 2020 15:02 To: Baesken, Matthias Cc: daniil.x.titov at oracle.com Subject: Re: getCpuLoad() / getSystemCpuLoad() returns -1 on linux when some offline cpus are present and cpusets.effective_cpus is not available I looks like there are two problems here: 1. containerMetrics.getCpuSetCpus().length returns the online CPUs but getHostConfiguredCpuCount0() returns the total number of CPUs including offline ones. One solution might be to add a getHostOnlineCpuCount0() function. 2. If getEffectiveCpuSetCpus is not available then we should use getCpuSetCpus. Bob. > On Jun 12, 2020, at 6:43 AM, Baesken, Matthias wrote: > > Hello, I noticed the following on one of our Linux machines : > > OperatingSystemMXBean mbean = (com.sun.management.OperatingSystemMXBean) ManagementFactory.getOperatingSystemMXBean(); > double load = mbean.getSystemCpuLoad(); > > returns -1 ; this seems to be related to ?8226575: OperatingSystemMXBean should be made container aware? . > > This machine has the following ?special features? > > - a few CPUs are offline (means the configured cpus are 48 but the online cpus are 32) so > > > private boolean isCpuSetSameAsHostCpuSet() { > if (containerMetrics != null && containerMetrics.getCpuSetCpus() != null) { > return containerMetrics.getCpuSetCpus().length == getHostConfiguredCpuCount0(); > } > return false; > } > > Returns false > > - the machine does not have cpusets.effective_cpus (not all Linux machines have it ) > > In this case getSystemCpuLoad() / getCpuLoad() returns -1 (because it checks that 48 != 32, and next it checks for cpusets.effective_cpus which is not present ). > > See the coding at : > https://hg.openjdk.java.net/jdk/jdk/file/bdc14b8d31ff/src/jdk.management/unix/classes/com/sun/management/internal/OperatingSystemImpl.java#l136 > > // If the cpuset is the same as the host's one there is no need to iterate over each CPU > if (isCpuSetSameAsHostCpuSet()) { > return getCpuLoad0(); > } else { > int[] cpuSet = containerMetrics.getEffectiveCpuSetCpus(); > if (cpuSet != null && cpuSet.length > 0) { > double systemLoad = 0.0; > for (int cpu : cpuSet) { > double cpuLoad = getSingleCpuLoad0(cpu); > if (cpuLoad < 0) { > return -1; > } > systemLoad += cpuLoad; > } > return systemLoad / cpuSet.length; > } > return -1; > } > > > Could we better a) return the native getCpuLoad0(); in this case or b) use the available containerMetrics.getCpuSetCpus(); when getEffectiveCpuSetCpus(); > Gives an empty array (btw. getCpuSetCpus() returns on this machine the online cpus = 0,31 = 32 ) ? > > I opened > > https://bugs.openjdk.java.net/browse/JDK-8247469 > > to track this (I see the issue in jdk/jdk but it seems it came also to oracle jdk8u261, which the July update ). > > Best regards, Matthias From daniel.daugherty at oracle.com Fri Jun 12 16:00:55 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 12 Jun 2020 12:00:55 -0400 Subject: RFR(T): 8247492: ProblemList tools/jlink/plugins/CompressorPluginTest.java Message-ID: <49c88ea1-eb14-2bd0-e4cb-1381932d086c@oracle.com> Greetings, It's time to reduce the noise in the CI so I'm ProblemListing tests. Here's the bug for failure: ??? JDK-8247407 tools/jlink/plugins/CompressorPluginTest.java test failing ??? https://bugs.openjdk.java.net/browse/JDK-8247407 and here's the bug for the ProblemListing: ??? JDK-8247492 ProblemList tools/jlink/plugins/CompressorPluginTest.java ??? https://bugs.openjdk.java.net/browse/JDK-8247492 Here's the context diff for the change. I'm considering this a trivial change so I need a single (R)eviewer. Thanks, in advance, for any comments, questions, or suggestions. Dan $ hg diff diff -r 1c50e1d77d85 test/jdk/ProblemList.txt --- a/test/jdk/ProblemList.txt??? Fri Jun 12 11:24:09 2020 -0400 +++ b/test/jdk/ProblemList.txt??? Fri Jun 12 11:55:24 2020 -0400 @@ -859,6 +859,7 @@ ?# core_tools ?tools/jlink/JLinkReproducibleTest.java 8217166 windows-all +tools/jlink/plugins/CompressorPluginTest.java 8247407 generic-all ?############################################################################ From lance.andersen at oracle.com Fri Jun 12 16:05:07 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 12 Jun 2020 12:05:07 -0400 Subject: RFR: 8218021: jarsigner strips the execute permission when signing a .zip file In-Reply-To: <6a34a99c-ed06-5e38-0932-788e944a70df@oracle.com> References: <6a34a99c-ed06-5e38-0932-788e944a70df@oracle.com> Message-ID: <22AD74BE-6E90-45CA-AEEF-73E5FCCC0BF5@oracle.com> Hi Sean, I think your changes look fine so all good FMPOV. Best Lance > On Jun 12, 2020, at 6:21 AM, Se?n Coffey wrote: > > Hi, > > I'd like to reboot this jarsigner enhancement request[1]. I've removed the problem references to zip file name extensions. Instead, there's a new JDK implementation specific jarsigner option: -keepposixperms > > https://bugs.openjdk.java.net/browse/JDK-8218021 > https://cr.openjdk.java.net/~coffeys/webrev.8218021.v2/webrev/ > > regards, > Sean. > > [1] http://mail.openjdk.java.net/pipermail/security-dev/2020-January/021141.html > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From james.laskey at oracle.com Fri Jun 12 16:02:14 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Fri, 12 Jun 2020 09:02:14 -0700 (PDT) Subject: RFR(T): 8247492: ProblemList tools/jlink/plugins/CompressorPluginTest.java In-Reply-To: <49c88ea1-eb14-2bd0-e4cb-1381932d086c@oracle.com> References: <49c88ea1-eb14-2bd0-e4cb-1381932d086c@oracle.com> Message-ID: <7AB400C7-19FB-4C24-87F3-EE56A0A6B14A@oracle.com> +1 > On Jun 12, 2020, at 1:00 PM, Daniel D. Daugherty wrote: > > Greetings, > > It's time to reduce the noise in the CI so I'm ProblemListing tests. > > Here's the bug for failure: > > JDK-8247407 tools/jlink/plugins/CompressorPluginTest.java test failing > https://bugs.openjdk.java.net/browse/JDK-8247407 > > and here's the bug for the ProblemListing: > > JDK-8247492 ProblemList tools/jlink/plugins/CompressorPluginTest.java > https://bugs.openjdk.java.net/browse/JDK-8247492 > > Here's the context diff for the change. I'm considering this a trivial > change so I need a single (R)eviewer. > > Thanks, in advance, for any comments, questions, or suggestions. > > Dan > > > > $ hg diff > diff -r 1c50e1d77d85 test/jdk/ProblemList.txt > --- a/test/jdk/ProblemList.txt Fri Jun 12 11:24:09 2020 -0400 > +++ b/test/jdk/ProblemList.txt Fri Jun 12 11:55:24 2020 -0400 > @@ -859,6 +859,7 @@ > # core_tools > > tools/jlink/JLinkReproducibleTest.java 8217166 windows-all > +tools/jlink/plugins/CompressorPluginTest.java 8247407 generic-all > > ############################################################################ From daniel.daugherty at oracle.com Fri Jun 12 16:05:45 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 12 Jun 2020 12:05:45 -0400 Subject: RFR(T): 8247492: ProblemList tools/jlink/plugins/CompressorPluginTest.java In-Reply-To: <7AB400C7-19FB-4C24-87F3-EE56A0A6B14A@oracle.com> References: <49c88ea1-eb14-2bd0-e4cb-1381932d086c@oracle.com> <7AB400C7-19FB-4C24-87F3-EE56A0A6B14A@oracle.com> Message-ID: <84025787-ad99-27c4-9d79-416a876996f4@oracle.com> Thanks for the fast review! Dan On 6/12/20 12:02 PM, Jim Laskey wrote: > +1 > >> On Jun 12, 2020, at 1:00 PM, Daniel D. Daugherty wrote: >> >> Greetings, >> >> It's time to reduce the noise in the CI so I'm ProblemListing tests. >> >> Here's the bug for failure: >> >> JDK-8247407 tools/jlink/plugins/CompressorPluginTest.java test failing >> https://bugs.openjdk.java.net/browse/JDK-8247407 >> >> and here's the bug for the ProblemListing: >> >> JDK-8247492 ProblemList tools/jlink/plugins/CompressorPluginTest.java >> https://bugs.openjdk.java.net/browse/JDK-8247492 >> >> Here's the context diff for the change. I'm considering this a trivial >> change so I need a single (R)eviewer. >> >> Thanks, in advance, for any comments, questions, or suggestions. >> >> Dan >> >> >> >> $ hg diff >> diff -r 1c50e1d77d85 test/jdk/ProblemList.txt >> --- a/test/jdk/ProblemList.txt Fri Jun 12 11:24:09 2020 -0400 >> +++ b/test/jdk/ProblemList.txt Fri Jun 12 11:55:24 2020 -0400 >> @@ -859,6 +859,7 @@ >> # core_tools >> >> tools/jlink/JLinkReproducibleTest.java 8217166 windows-all >> +tools/jlink/plugins/CompressorPluginTest.java 8247407 generic-all >> >> ############################################################################ From naoto.sato at oracle.com Fri Jun 12 16:31:51 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Fri, 12 Jun 2020 09:31:51 -0700 Subject: RFR: 8245448: Remove minimum 4 digit requirement from Year.parse() Message-ID: <346de1a8-ad85-4738-b3f7-c6405dc312f5@oracle.com> Hello, Please review the fix to the following issue: https://bugs.openjdk.java.net/browse/JDK-8245448 The proposed changeset and its CSR are located at: https://cr.openjdk.java.net/~naoto/8245448/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8246623 Naoto From lance.andersen at oracle.com Fri Jun 12 17:06:37 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 12 Jun 2020 13:06:37 -0400 Subject: RFR: 8245448: Remove minimum 4 digit requirement from Year.parse() In-Reply-To: <346de1a8-ad85-4738-b3f7-c6405dc312f5@oracle.com> References: <346de1a8-ad85-4738-b3f7-c6405dc312f5@oracle.com> Message-ID: <927C5507-A96B-468D-B909-78D32B1E1ACC@oracle.com> Hi Naoto, This looks fine and looks like you are set with reviewers on your CSR Best Lance > On Jun 12, 2020, at 12:31 PM, naoto.sato at oracle.com wrote: > > Hello, > > Please review the fix to the following issue: > > https://bugs.openjdk.java.net/browse/JDK-8245448 > > The proposed changeset and its CSR are located at: > > https://cr.openjdk.java.net/~naoto/8245448/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8246623 > > Naoto Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From martinrb at google.com Fri Jun 12 16:36:24 2020 From: martinrb at google.com (Martin Buchholz) Date: Fri, 12 Jun 2020 09:36:24 -0700 Subject: [PATCH] 8247402: Rewrite the implementation requirements for Map::compute() In-Reply-To: References: Message-ID: If I rub that program, I get: HashMap false HashMap1 false HashMap2 true which suggests that HashMap2's implementation is wrong. From Roger.Riggs at oracle.com Fri Jun 12 18:17:46 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Fri, 12 Jun 2020 14:17:46 -0400 Subject: RFR: 8245448: Remove minimum 4 digit requirement from Year.parse() In-Reply-To: <927C5507-A96B-468D-B909-78D32B1E1ACC@oracle.com> References: <346de1a8-ad85-4738-b3f7-c6405dc312f5@oracle.com> <927C5507-A96B-468D-B909-78D32B1E1ACC@oracle.com> Message-ID: <4491474f-1612-9dfe-ed26-f8e7bc9ef852@oracle.com> Hi Naoto, Yes, looks good. Roger On 6/12/20 1:06 PM, Lance Andersen wrote: > Hi Naoto, > > This looks fine and looks like you are set with reviewers on your CSR > > Best > Lance > >> On Jun 12, 2020, at 12:31 PM, naoto.sato at oracle.com wrote: >> >> Hello, >> >> Please review the fix to the following issue: >> >> https://bugs.openjdk.java.net/browse/JDK-8245448 >> >> The proposed changeset and its CSR are located at: >> >> https://cr.openjdk.java.net/~naoto/8245448/webrev.00/ >> https://bugs.openjdk.java.net/browse/JDK-8246623 >> >> Naoto > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From amaembo at gmail.com Fri Jun 12 17:37:35 2020 From: amaembo at gmail.com (Tagir Valeev) Date: Sat, 13 Jun 2020 00:37:35 +0700 Subject: [PING] RFR: JDK-8244288 Specialized implementations for putIfAbsent, merge, compute* methods in TreeMap derived maps In-Reply-To: References: Message-ID: Hello! A gentle ping: please review https://bugs.openjdk.java.net/browse/JDK-8244288 http://cr.openjdk.java.net/~tvaleev/webrev/8244288/r1/ The details are listed below. With best regards, Tagir Valeev. ---------- Forwarded message --------- From: Tagir Valeev Date: Sun, May 3, 2020 at 4:36 PM Subject: RFR: JDK-8244288 Specialized implementations for putIfAbsent, merge, compute* methods in TreeMap derived maps To: core-libs-dev , Martin Buchholz Hello! Please review the following change: https://bugs.openjdk.java.net/browse/JDK-8244288 http://cr.openjdk.java.net/~tvaleev/webrev/8244288/r1/ We already implemented putIfAbsent, merge, computeIfAbsent, computeIfPresent, and compute for TreeMap class (see JDK-8176894). However, default implementations of these methods are still used for TreeMap derived maps, such as descendingMap(), tailMap(), headMap() and subMap(). The implementation is pretty straightforward, just a range-check+delegation for each of these methods inside the TreeMap.NavigableSubMap (+32 lines in TreeMap.java). Care should be taken to avoid throwing IAE from compute* methods if the key is outside of the range but we don't actually add the entry. This mimics the previous behavior and also consistent with other modification methods like Map.remove (we don't throw from remove if the key is out of range, simply do nothing). To cover these changes with tests, I added new TreeMap().descendingMap() to InPlaceOpsCollisions.nullValueFriendlyMaps and MapWithCollisionsProviders.makeMapsMoreTypes. This map should behave like a normal Map modulo iteration order but implementation goes through NavigableSubMap. Also, I added more adders to LockStep::randomAdder (lines 572+) to cover new methods, as well as some more throws IAE asserts around line 806. At the same time, I took the liberty to modernize this test: - Convert all the raw-types to properly parameterized (using 'var' for local vars where resulting type is long) - Convert MapFrobber and SetFrobber to interfaces, and convert all the implementations to lambdas (automatic conversion via IntelliJ IDEA + rename of conflicting parameter names) - Use for-each loop instead of indexed for loop where possible - 'elts' array was created in two places but unused afterward. I added size assert to these arrays to use it at least somehow (though probably content should be checked as well). - Use Comparator.naturalOrder() instead of manually written one in comparator() methods (should not affect the testing in any way as it's only used directly, not passed e.g. to TreeMap constructor). - Use Objects.equals inside LockStep#equal - Inverted 'if' at line 299 to avoid empty block. If the cleanup really complicates the review I can post the cleanup as a separate changeset. Though it should not be very problematic as the actual changes are quite compact (as said above, near lines 572 and 806) I also improved the previously added benchmark TreeMapUpdate to check descendingMap and tailMap cases to confirm the hypothesis that the change improves the performance of derived maps. This is indeed the case. E.g. merge test yields (comparator = false, size = 1000) for unpatched JDK: (benchmark) (mode) (prefill) Score Error Units TreeMapUpdate.merge TreeMap true 63589,075 ? 1028,065 ns/op TreeMapUpdate.merge TreeMap false 65414,367 ? 1011,442 ns/op TreeMapUpdate.merge descendingMap true 121501,618 ? 1849,108 ns/op TreeMapUpdate.merge descendingMap false 95913,212 ? 2854,063 ns/op TreeMapUpdate.merge tailMap true 126657,309 ? 4467,733 ns/op TreeMapUpdate.merge tailMap false 93448,840 ? 1359,392 ns/op As you can see, the merge on derived maps is significantly slower. After the patch is applied the results are much better: (benchmark) (mode) (prefill) Score Error Units TreeMapUpdate.merge TreeMap true 64059,189 ? 808,230 ns/op TreeMapUpdate.merge TreeMap false 65156,912 ? 1229,965 ns/op TreeMapUpdate.merge descendingMap true 69938,131 ? 1697,357 ns/op TreeMapUpdate.merge descendingMap false 67783,829 ? 263,061 ns/op TreeMapUpdate.merge tailMap true 71079,529 ? 841,738 ns/op TreeMapUpdate.merge tailMap false 68577,169 ? 1196,758 ns/op I don't think this requires a CSR, as the changed class is package-private, so it cannot be extended by clients. With best regards, Tagir Valeev. From Roger.Riggs at oracle.com Fri Jun 12 18:47:45 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Fri, 12 Jun 2020 14:47:45 -0400 Subject: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) In-Reply-To: <50955EDB-CCC2-4334-A4AF-2EB422E9CAA9@amperemail.onmicrosoft.com> References: <814DA850-76A9-4526-A3D9-141CD516C9AE@amperemail.onmicrosoft.com> <50955EDB-CCC2-4334-A4AF-2EB422E9CAA9@amperemail.onmicrosoft.com> Message-ID: Hi Peter, The hazard to avoid is the cross package coupling that makes both ClassLoader and ObjectInputStream more complex; both are more than sufficiently complex already. Optimizing the implementations make sense if it benefits enough uses cases and does not make the code harder to maintain and evolve. Have you explored any alternatives that can be confined to OIS? For example, taking advantage the many calls to readObject within a single OIS that are nested. Possibly the first (depth = 0) find user class loader can be re-used or at least to short circuit the search on subsequent nested calls? Or caching the classloader at each depth and only need to probe for it if it has not already been recorded. The StackWalker API may be useful to limit the depth of searches. Thanks, Roger On 6/11/20 2:52 PM, Peter Kessler OS wrote: > Hi Roger, > > Thanks for your comments. > > I am not knowingly changing the required behavior. All I am proposing is > guarding the thread stack walk if it will be a waste of effort. If an > application or framework has its own ClassLoaders, then the guard will fail > and the thread stack will be walked. > > The stack walk is important, if there are user-defined ClassLoaders. A > method that uses ObjectInputStream to read objects can not know what > ClassLoader to use to resolve class references in the stream, without the > stack walk. But if there never have been any user-defined ClassLoaders > constructed, then the stack walk can never return a user-defined > ClassLoader. > > If the stream reader is part of some simple application, then it may spend > a significant amount of its time uselessly walking thread stacks. (If the > stream reader is part of some simple application its author may not know to > override ObjectInputStream.resolveClass, or how to determine what > ClassLoader should load a class from the stream.) If someone were using > such an application to choose platforms, the Java platform would be at a > disadvantage. > > Consider a somewhat more complex case, the SPECjbb2015 benchmark, which > claims to model a well-constructed business application on the Java > platform. SPECjbb2015 runs multiple JVMs, exchanging information via > ObjectInputStreams. Profiling shows that it spends more than 5% of its time > walking thread stacks. Yet, in spite of being a complex application, > written by experts in the Java platform, there are no user-defined > ClassLoaders. So all the time spent walking stacks is wasted time, and is > one of the reasons the servers fail their service-level agreements. We can > discuss whether SPECjbb2015 is representative of Java applications. But it > is easily available and relatively simple to run. People use benchmarks > like SPECjbb2015 to decide whether to use the Java platform for particular > tasks, and it would be unfortunate if we did not make the Java platform as > good as it could be. > > ... peter > > ?-----Original Message----- > From: core-libs-dev on behalf of Roger Riggs > Organization: Java Products Group, Oracle > Date: Tuesday, June 9, 2020 at 6:43 AM > To: "core-libs-dev at openjdk.java.net" > Subject: Re: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) > > Hi Peter, > > I'd like to understand the scope and impact of the problem before > jumping to a solution. > For specific applications, overriding ObjectInputStream.resolveClass to > handle the class > lookup can handle the case as expeditiously as is necessary. > > What application uses cases are impacted and how prevelant are they? > Many, many frameworks and applications create classloaders for their own > purposes and > take advantage of the required behavior. > > Thanks, Roger > > > On 6/4/20 7:08 PM, Peter Kessler OS wrote: > > ObjectInputStream.resolveClass(ObjectStreamClass) is specified to find "the first class loader on the current thread's stack (starting from the currently executing method) that is neither the platform class loader nor its ancestor." Finding that class loader is done with a virtual frame walk in JVM_LatestUserDefinedLoader(JNIEnv *env). The virtual frame walk is expensive. If an application does not define any ClassLoaders, then the virtual frame walk will always find the system class loader. > > > > If an application does a significant amount of reading from an ObjectInputStream, the cost of ObjectInputStream.resolveClass(ObjectInputStreamClass) can be considerable. In a sample application, that method (and the methods it calls) have been measured to consume 5+% of the CPU cycles. > > > > The proposal is to note the construction of any user-defined ClassLoader, and if none has been constructed, avoid the virtual frame walk returning ClassLoader.getSystemClassLoader(). But if any user-defined ClassLoader has been constructed, use the virtual frame walk to find the first user-defined ClassLoader on the thread stack. > > > > User-defined ClassLoaders could be distinguished from non-user-defined ClassLoaders in several ways. The ClassLoader class hierarchy could be changed to make non-user-defined ClassLoaders inherit from a marker class. There is already a jdk.internal.loader.BuiltinClassLoader but it is used for module (and resource) loading. BuiltinClassLoader is not a super-class of jdk.internal.reflect.DelegatingClassLoader, which is used to load reflection classes, and reflection methods are also ignored by the virtual frame walk. ClassLoaders could have a boolean that distinguished whether they are user-defined or not. That would require a change to ClassLoader API. There are other ways to distinguish user-defined ClassLoaders. The proposal is to keep the decoration of the ClassLoaders out of the instances, so that it is only visible to the code that needs it. > > > > The proposal is that each non-user-defined ClassLoader have a static block that registers the class as a non-user-defined ClassLoader. Then in the base ClassLoader constructor check if the ClassLoader being constructed is a user-defined ClassLoader and if so set a boolean that acts as the guard on the virtual frame walk. If additional non-user-defined ClassLoaders are declared in the future beyond the ones identified in this patch, they can be added to the registry. > > > > There are currently 4 non-user-defined ClassLoader classes, so the registry is not difficult to maintain. Nor is the registry difficult to search when ClassLoaders are constructed. The guard boolean that records whether a user-defined ClassLoader has been constructed transitions from false to true and never becomes false again. (An extension to transition the boolean back to false when a user-defined ClassLoader becomes unreachable is beyond the scope of this proposal.) > > > > This proposal slows loading of non-user-defined ClassLoaders by the time it takes to register them. It slows ClassLoader construction by the time it takes to consult the registry and conditionally set the guard boolean. At each invocation of ObjectInputStream.resolveClass(ObjectInputStreamClass), the guard boolean is used to possibly avoid the virtual frame walk. > > > > Tested with `make run-test-tier1` on Linux (CentOS 7) on both aarch64 and x86_64. I had one failure on each of the runs > > > > ACTION: main -- Failed. Execution failed: `main' threw exception: java.lang.Error: TESTBUG: the library has not been found in ${Repo}/build/linux-aarch64-server-release/images/test/hotspot/jtreg/native > > REASON: User specified action: run main/native GTestWrapper > > > > on both plaforms. But I get the same failure when I run the tier1 tests on clones of OpenJDK-15+25 on both aarch64 and x86_64. > > > > Running a sample (representative?) application with the Oracle Developer Studio analyzer shows a performance comparison of > > > > Stack Fragment sorted by metric: Attributed Total CPU Time > > baseline.er experiment.er > > Attributed Attributed Name > > Total CPU Time Total CPU Time > > sec. sec. > > ================================== Callers > > 3459.210 29.931 java.io.ObjectInputStream.readOrdinaryObject(boolean) > > 1139.727 3.532 java.io.ObjectInputStream.readArray(boolean) > > 875.262 9.116 java.io.ObjectInputStream.readNonProxyDesc(boolean) > > > > ================================== Stack Fragment > > java.io.ObjectInputStream.readClassDesc(boolean) > > java.io.ObjectInputStream.readNonProxyDesc(boolean) > > java.io.ObjectInputStream.resolveClass(java.io.ObjectStreamClass) > > java.io.ObjectInputStream.latestUserDefinedLoader() > > 4.173 3.953 jdk.internal.misc.VM.latestUserDefinedLoader() > > > > ================================== Callees > > 5470.026 0. jdk.internal.misc.VM.latestUserDefinedLoader0() > > 0. 38.627 java.lang.ClassLoader.getSystemClassLoader() > > > > The `hg export -g` below is with respect to OpenJDK-15+25. > > > > Thank you for your review comments. I will need a sponsor to get this change into the repository. > > > > ... peter > > > > # HG changeset patch > > # User Peter Kessler > > # Date 1591310467 25200 > > # Thu Jun 04 15:41:07 2020 -0700 > > # Node ID 9a39488182c1dfc5bc7bb41d562d7ef16ee657f6 > > # Parent 90b266a84c06f1b3dc0ed8767856793e8c1c357e > > Improve the performance of ObjectInputStream.resolveClass > > > > diff --git a/src/java.base/share/classes/java/lang/ClassLoader.java b/src/java.base/share/classes/java/lang/ClassLoader.java > > --- a/src/java.base/share/classes/java/lang/ClassLoader.java > > +++ b/src/java.base/share/classes/java/lang/ClassLoader.java > > @@ -385,6 +385,8 @@ > > } > > this.package2certs = new ConcurrentHashMap<>(); > > this.nameAndId = nameAndId(this); > > + /* Note the construction of a ClassLoader. */ > > + VM.noteClassLoaderConstruction(this.getClass()); > > } > > > > /** > > diff --git a/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java b/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java > > --- a/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java > > +++ b/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java > > @@ -117,6 +117,11 @@ > > protected Class loadClassOrNull(String cn, boolean resolve) { > > return JLA.findBootstrapClassOrNull(this, cn); > > } > > + > > + static { > > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > > + VM.registerNotUserDefinedClassLoader(BootClassLoader.class); > > + } > > }; > > > > /** > > @@ -127,6 +132,8 @@ > > static { > > if (!ClassLoader.registerAsParallelCapable()) > > throw new InternalError(); > > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > > + VM.registerNotUserDefinedClassLoader(PlatformClassLoader.class); > > } > > > > PlatformClassLoader(BootClassLoader parent) { > > @@ -142,6 +149,8 @@ > > static { > > if (!ClassLoader.registerAsParallelCapable()) > > throw new InternalError(); > > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > > + VM.registerNotUserDefinedClassLoader(AppClassLoader.class); > > } > > > > final URLClassPath ucp; > > diff --git a/src/java.base/share/classes/jdk/internal/misc/VM.java b/src/java.base/share/classes/jdk/internal/misc/VM.java > > --- a/src/java.base/share/classes/jdk/internal/misc/VM.java > > +++ b/src/java.base/share/classes/jdk/internal/misc/VM.java > > @@ -304,12 +304,44 @@ > > private static final int JVMTI_THREAD_STATE_WAITING_INDEFINITELY = 0x0010; > > private static final int JVMTI_THREAD_STATE_WAITING_WITH_TIMEOUT = 0x0020; > > > > + /** A registry of not user-defined ClassLoaders. */ > > + private static final List> notUserDefinedClassLoaderRegistry = > > + Collections.synchronizedList(new ArrayList<>()); > > + > > + /** Register a ClassLoader class as being not a user-defined ClassLoader. */ > > + public static void registerNotUserDefinedClassLoader(Class classLoaderClass) { > > + notUserDefinedClassLoaderRegistry.add(classLoaderClass); > > + } > > + > > + /** A flag for whether a user-defined ClassLoaders has been constructed. */ > > + private static volatile boolean constructedUserDefinedClassLoaderFlag = false; > > + > > + /** > > + * Note a ClassLoader construction, and record if a > > + * user-defined ClassLoader has been constructed. > > + */ > > + public static void noteClassLoaderConstruction(Class classLoaderClass) { > > + /* Check if the ClassLoader class not a user-defined ClassLoader. */ > > + if (notUserDefinedClassLoaderRegistry.contains(classLoaderClass)) { > > + return; > > + } > > + constructedUserDefinedClassLoaderFlag = true; > > + return; > > + } > > + > > /* > > * Returns the first user-defined class loader up the execution stack, > > * or the platform class loader if only code from the platform or > > * bootstrap class loader is on the stack. > > */ > > public static ClassLoader latestUserDefinedLoader() { > > + if (!constructedUserDefinedClassLoaderFlag) { > > + /* > > + * If no user-defined ClassLoader has been constructed, > > + * then I will not find a user-defined ClassLoader in the stack. > > + */ > > + return ClassLoader.getSystemClassLoader(); > > + } > > ClassLoader loader = latestUserDefinedLoader0(); > > return loader != null ? loader : ClassLoader.getPlatformClassLoader(); > > } > > diff --git a/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java b/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java > > --- a/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java > > +++ b/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java > > @@ -30,6 +30,7 @@ > > > > import jdk.internal.access.JavaLangAccess; > > import jdk.internal.access.SharedSecrets; > > +import jdk.internal.misc.VM; > > > > /** Utility class which assists in calling defineClass() by > > creating a new class loader which delegates to the one needed in > > @@ -73,4 +74,9 @@ > > DelegatingClassLoader(ClassLoader parent) { > > super(parent); > > } > > + > > + static { > > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > > + VM.registerNotUserDefinedClassLoader(DelegatingClassLoader.class); > > + } > > } > > > > From huizhe.wang at oracle.com Fri Jun 12 19:02:30 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Fri, 12 Jun 2020 12:02:30 -0700 Subject: RFR: 8245448: Remove minimum 4 digit requirement from Year.parse() In-Reply-To: <4491474f-1612-9dfe-ed26-f8e7bc9ef852@oracle.com> References: <346de1a8-ad85-4738-b3f7-c6405dc312f5@oracle.com> <927C5507-A96B-468D-B909-78D32B1E1ACC@oracle.com> <4491474f-1612-9dfe-ed26-f8e7bc9ef852@oracle.com> Message-ID: <3bd3a361-0ca2-0974-1805-068219cfd8ad@oracle.com> +1.? The original poster was probably working on writing history (all the way back to before 1000), I assume ;-)? This enhancement would certainly make his job a bit easier. -Joe On 6/12/2020 11:17 AM, Roger Riggs wrote: > Hi Naoto, > > Yes, looks good. > > Roger > > > On 6/12/20 1:06 PM, Lance Andersen wrote: >> Hi Naoto, >> >> This looks fine and looks like you are set with reviewers on your CSR >> >> Best >> Lance >> >>> On Jun 12, 2020, at 12:31 PM, naoto.sato at oracle.com wrote: >>> >>> Hello, >>> >>> Please review the fix to the following issue: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8245448 >>> >>> The proposed changeset and its CSR are located at: >>> >>> https://cr.openjdk.java.net/~naoto/8245448/webrev.00/ >>> https://bugs.openjdk.java.net/browse/JDK-8246623 >>> >>> Naoto >> >> >> >> Lance >> Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> >> > From daniil.x.titov at oracle.com Fri Jun 12 19:09:56 2020 From: daniil.x.titov at oracle.com (Daniil Titov) Date: Fri, 12 Jun 2020 12:09:56 -0700 Subject: RFR: 8247469: getSystemCpuLoad() returns -1 on linux when some offline cpus are present and cpusets.effective_cpus is not available In-Reply-To: References: Message-ID: <97230AF3-579A-461F-AB4C-690F37DDFBCE@oracle.com> Hi Matthias, The change looks good to me. Probably it also makes sense to remove method getHostConfiguredCpuCount0() since it is no longer used. Thanks, Daniil ?On 6/12/20, 8:25 AM, "Baesken, Matthias" wrote: Hello, please review the following change . We have a Linux machine where OperatingSystemMXBean mbean = (com.sun.management.OperatingSystemMXBean) ManagementFactory.getOperatingSystemMXBean(); double load = mbean.getSystemCpuLoad(); returns -1 ; Reason is that there are offline CPUs (48 configured , 32 online ). Additionally cpusets.effective_cpus is not available on that Linux system . Bug/webrev : https://bugs.openjdk.java.net/browse/JDK-8247466 http://cr.openjdk.java.net/~mbaesken/webrevs/8247469.0/ Thanks, Matthias -----Original Message----- From: Bob Vandette Sent: Freitag, 12. Juni 2020 15:02 To: Baesken, Matthias Cc: daniil.x.titov at oracle.com Subject: Re: getCpuLoad() / getSystemCpuLoad() returns -1 on linux when some offline cpus are present and cpusets.effective_cpus is not available I looks like there are two problems here: 1. containerMetrics.getCpuSetCpus().length returns the online CPUs but getHostConfiguredCpuCount0() returns the total number of CPUs including offline ones. One solution might be to add a getHostOnlineCpuCount0() function. 2. If getEffectiveCpuSetCpus is not available then we should use getCpuSetCpus. Bob. > On Jun 12, 2020, at 6:43 AM, Baesken, Matthias wrote: > > Hello, I noticed the following on one of our Linux machines : > > OperatingSystemMXBean mbean = (com.sun.management.OperatingSystemMXBean) ManagementFactory.getOperatingSystemMXBean(); > double load = mbean.getSystemCpuLoad(); > > returns -1 ; this seems to be related to ?8226575: OperatingSystemMXBean should be made container aware? . > > This machine has the following ?special features? > > - a few CPUs are offline (means the configured cpus are 48 but the online cpus are 32) so > > > private boolean isCpuSetSameAsHostCpuSet() { > if (containerMetrics != null && containerMetrics.getCpuSetCpus() != null) { > return containerMetrics.getCpuSetCpus().length == getHostConfiguredCpuCount0(); > } > return false; > } > > Returns false > > - the machine does not have cpusets.effective_cpus (not all Linux machines have it ) > > In this case getSystemCpuLoad() / getCpuLoad() returns -1 (because it checks that 48 != 32, and next it checks for cpusets.effective_cpus which is not present ). > > See the coding at : > https://hg.openjdk.java.net/jdk/jdk/file/bdc14b8d31ff/src/jdk.management/unix/classes/com/sun/management/internal/OperatingSystemImpl.java#l136 > > // If the cpuset is the same as the host's one there is no need to iterate over each CPU > if (isCpuSetSameAsHostCpuSet()) { > return getCpuLoad0(); > } else { > int[] cpuSet = containerMetrics.getEffectiveCpuSetCpus(); > if (cpuSet != null && cpuSet.length > 0) { > double systemLoad = 0.0; > for (int cpu : cpuSet) { > double cpuLoad = getSingleCpuLoad0(cpu); > if (cpuLoad < 0) { > return -1; > } > systemLoad += cpuLoad; > } > return systemLoad / cpuSet.length; > } > return -1; > } > > > Could we better a) return the native getCpuLoad0(); in this case or b) use the available containerMetrics.getCpuSetCpus(); when getEffectiveCpuSetCpus(); > Gives an empty array (btw. getCpuSetCpus() returns on this machine the online cpus = 0,31 = 32 ) ? > > I opened > > https://bugs.openjdk.java.net/browse/JDK-8247469 > > to track this (I see the issue in jdk/jdk but it seems it came also to oracle jdk8u261, which the July update ). > > Best regards, Matthias From bob.vandette at oracle.com Fri Jun 12 18:49:00 2020 From: bob.vandette at oracle.com (Bob Vandette) Date: Fri, 12 Jun 2020 14:49:00 -0400 Subject: RFR: 8247469: getSystemCpuLoad() returns -1 on linux when some offline cpus are present and cpusets.effective_cpus is not available In-Reply-To: References: Message-ID: Look good to me. Bob. > On Jun 12, 2020, at 11:01 AM, Baesken, Matthias wrote: > > Hello, please review the following change . > We have a Linux machine where > OperatingSystemMXBean mbean = (com.sun.management.OperatingSystemMXBean) ManagementFactory.getOperatingSystemMXBean(); > double load = mbean.getSystemCpuLoad(); > returns -1 ; > > Reason is that there are offline CPUs (48 configured , 32 online ). Additionally cpusets.effective_cpus is not available on that Linux system . > > > Bug/webrev : > > https://bugs.openjdk.java.net/browse/JDK-8247466 > > http://cr.openjdk.java.net/~mbaesken/webrevs/8247469.0/ > > > Thanks, > Matthias > > > -----Original Message----- > From: Bob Vandette > Sent: Freitag, 12. Juni 2020 15:02 > To: Baesken, Matthias > Cc: daniil.x.titov at oracle.com > Subject: Re: getCpuLoad() / getSystemCpuLoad() returns -1 on linux when some offline cpus are present and cpusets.effective_cpus is not available > > I looks like there are two problems here: > > 1. containerMetrics.getCpuSetCpus().length returns the online CPUs but getHostConfiguredCpuCount0() returns the total number of CPUs including offline ones. > > One solution might be to add a getHostOnlineCpuCount0() function. > > 2. If getEffectiveCpuSetCpus is not available then we should use getCpuSetCpus. > > Bob. > > >> On Jun 12, 2020, at 6:43 AM, Baesken, Matthias wrote: >> >> Hello, I noticed the following on one of our Linux machines : >> >> OperatingSystemMXBean mbean = (com.sun.management.OperatingSystemMXBean) ManagementFactory.getOperatingSystemMXBean(); >> double load = mbean.getSystemCpuLoad(); >> >> returns -1 ; this seems to be related to ?8226575: OperatingSystemMXBean should be made container aware? . >> >> This machine has the following ?special features? >> >> - a few CPUs are offline (means the configured cpus are 48 but the online cpus are 32) so >> >> >> private boolean isCpuSetSameAsHostCpuSet() { >> if (containerMetrics != null && containerMetrics.getCpuSetCpus() != null) { >> return containerMetrics.getCpuSetCpus().length == getHostConfiguredCpuCount0(); >> } >> return false; >> } >> >> Returns false >> >> - the machine does not have cpusets.effective_cpus (not all Linux machines have it ) >> >> In this case getSystemCpuLoad() / getCpuLoad() returns -1 (because it checks that 48 != 32, and next it checks for cpusets.effective_cpus which is not present ). >> >> See the coding at : >> https://hg.openjdk.java.net/jdk/jdk/file/bdc14b8d31ff/src/jdk.management/unix/classes/com/sun/management/internal/OperatingSystemImpl.java#l136 >> >> // If the cpuset is the same as the host's one there is no need to iterate over each CPU >> if (isCpuSetSameAsHostCpuSet()) { >> return getCpuLoad0(); >> } else { >> int[] cpuSet = containerMetrics.getEffectiveCpuSetCpus(); >> if (cpuSet != null && cpuSet.length > 0) { >> double systemLoad = 0.0; >> for (int cpu : cpuSet) { >> double cpuLoad = getSingleCpuLoad0(cpu); >> if (cpuLoad < 0) { >> return -1; >> } >> systemLoad += cpuLoad; >> } >> return systemLoad / cpuSet.length; >> } >> return -1; >> } >> >> >> Could we better a) return the native getCpuLoad0(); in this case or b) use the available containerMetrics.getCpuSetCpus(); when getEffectiveCpuSetCpus(); >> Gives an empty array (btw. getCpuSetCpus() returns on this machine the online cpus = 0,31 = 32 ) ? >> >> I opened >> >> https://bugs.openjdk.java.net/browse/JDK-8247469 >> >> to track this (I see the issue in jdk/jdk but it seems it came also to oracle jdk8u261, which the July update ). >> >> Best regards, Matthias > From peter.kessler at os.amperecomputing.com Fri Jun 12 20:04:51 2020 From: peter.kessler at os.amperecomputing.com (Peter Kessler (Open Source)) Date: Fri, 12 Jun 2020 20:04:51 +0000 Subject: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) In-Reply-To: <04b1cf61-c1cf-89eb-bcf2-ddd0f69e9321@oracle.com> References: <814DA850-76A9-4526-A3D9-141CD516C9AE@amperemail.onmicrosoft.com> <71ea732b-4566-e0a0-b2a3-e71df8a79609@oracle.com> <0E094A25-70C6-455D-A01D-F524A8010210@amperemail.onmicrosoft.com> <04b1cf61-c1cf-89eb-bcf2-ddd0f69e9321@oracle.com> Message-ID: <3EC88274-ACE8-43F6-A0BA-299182CEC451@amperemail.onmicrosoft.com> Daniel, Thanks for the suggestions to increase the timeout and lower the concurrency. I did Bad Science(TM) and changed them both at the same time. I am now running $ jtreg-5.0-b01/bin/jtreg \ -verbose:summary -a -ea -esa -agentvm -conc:1 -ignore:quiet -timeout:4 \ -exclude:./test/jdk/ProblemList.txt \ -jdk:${jdk} \ ./test/jdk/:jdk_core ./test/jdk/:jdk_svc I only had time for 2 tests of each jdk overnight, because running that way takes much longer, but the results are less confusing. Running on x86_64, counting failures and errors in 2 runs with OracleJDK build 15-ea+25-1229 2 Error: java/lang/ClassLoader/nativeLibrary/NativeLibraryTest.java 2 Error: java/lang/instrument/HiddenClass/HiddenClassAgent.java 2 Error: java/lang/reflect/exeCallerAccessTest/CallerAccessTest.java 2 Error: java/lang/String/nativeEncoding/StringPlatformChars.java 2 Error: java/nio/channels/FileChannel/directio/DirectIOTest.java 2 Error: java/nio/channels/spi/SelectorProvider/inheritedChannel/InheritedChannelTest.java 2 Error: jdk/internal/loader/NativeLibraries/Main.java 2 Error: Some tests failed or other problems occurred. 2 Error: sun/management/jmxremote/bootstrap/CustomLauncherTest.java 2 Error: tools/launcher/JliLaunchTest.java 1 FAILED: com/sun/jdi/PrivateTransportTest.java 1 FAILED: java/io/Serializable/unresolvableObjectStreamClass/UnresolvableObjectStreamClass.java 1 FAILED: java/security/KeyStore/PKCS12/MetadataStoreLoadTest.java 1 FAILED: java/security/KeyStore/PKCS12/StoreTrustedCertAPITest.java 1 FAILED: java/security/KeyStore/PKCS12/StoreTrustedCertKeytool.java 2 FAILED: jdk/nio/zipfs/ZipFSTester.java 2 FAILED: sun/security/pkcs11/Secmod/AddTrustedCert.java 2 FAILED: tools/jlink/plugins/StripNativeDebugSymbolsPlugin/StripNativeDebugSymbolsPluginTest.java 1 Test results: passed: 6,714; failed: 8; error: 9 1 Test results: passed: 6,719; failed: 3; error: 9 versus 2 runs of my build of OpenJDK-15+25 with the proposed patch 2 Error: java/lang/ClassLoader/nativeLibrary/NativeLibraryTest.java 2 Error: java/lang/instrument/HiddenClass/HiddenClassAgent.java 2 Error: java/lang/reflect/exeCallerAccessTest/CallerAccessTest.java 2 Error: java/lang/String/nativeEncoding/StringPlatformChars.java 2 Error: java/nio/channels/FileChannel/directio/DirectIOTest.java 2 Error: java/nio/channels/spi/SelectorProvider/inheritedChannel/InheritedChannelTest.java 2 Error: jdk/internal/loader/NativeLibraries/Main.java 2 Error: Some tests failed or other problems occurred. 2 Error: sun/management/jmxremote/bootstrap/CustomLauncherTest.java 2 Error: tools/launcher/JliLaunchTest.java 2 FAILED: com/sun/jdi/PrivateTransportTest.java 2 FAILED: java/io/Serializable/unresolvableObjectStreamClass/UnresolvableObjectStreamClass.java 2 FAILED: java/security/KeyStore/PKCS12/MetadataStoreLoadTest.java 2 FAILED: java/security/KeyStore/PKCS12/StoreTrustedCertAPITest.java 2 FAILED: java/security/KeyStore/PKCS12/StoreTrustedCertKeytool.java 2 FAILED: jdk/nio/zipfs/ZipFSTester.java 2 FAILED: sun/security/lib/cacerts/VerifyCACerts.java 2 FAILED: sun/security/pkcs11/Secmod/AddTrustedCert.java 2 FAILED: tools/jlink/plugins/StripNativeDebugSymbolsPlugin/StripNativeDebugSymbolsPluginTest.java 2 Test results: passed: 6,716; failed: 9; error: 9 The patched runs fail more consistently, but I don't know if that is good or bad.(:-) The only difference in the failure lists is on my build, where sun/security/lib/cacerts/VerifyCACerts.java fails. That seems to be because my built lib/security/cacerts is different from the one in 15-ea+25-1229. $ ls -l OracleJDK/jdk-15-ea+25/lib/security/cacerts OpenJDK/jdk-15+25+ClassLoaderRegistry/lib/security/cacerts -rw-r--r-- 1 pkessler pkessler 101001 May 28 08:47 OracleJDK/jdk-15-ea+25/lib/security/cacerts -rw-rw-r-- 1 pkessler pkessler 154279 Jun 3 17:04 OpenJDK/jdk-15+25+ClassLoaderRegistry/lib/security/cacerts I don't know why that difference exists, but it probably not due to the patch. (If I build a clone of OpenJDK-15+25, I get the identical, larger, lib/security/cacerts file.) If I copy the lib/security/cacerts file from 15-ea+25-1229 to my build then sun/security/lib/cacerts/VerifyCACerts.java passes. My aarch64 runs also show that the only difference in failures is sun/security/lib/cacerts/VerifyCACerts.java. You asked after my /etc/hosts file and whether "localhost" resolves to the loopback address. My /etc/hosts files on both x86_64 and aarch64 contain 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 so that looks all proper and correct. And changing the concurrrency and timeout seems to have cleared up the httpclient failures. What further testing do you suggest? ... peter ?-----Original Message----- From: Daniel Fuchs Date: Thursday, June 11, 2020 at 12:31 PM To: Peter Kessler OS , "core-libs-dev at openjdk.java.net" Subject: Re: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) Hi Peter, I am surprised by the java/net/httpclient failures: these tests have been quite stable for me. Is your repository up to date? Or are these maybe failing in timeout, in which case you might need to increase the timeout factor (-timeout) or lower the concurrency (-conc). Or maybe you have some weird configuration in your /etc/hosts - issues have been known to happen if looking up "localhost" doesn't resolve to the loopback address. I any case I'd be curious to see the reason for those failures. best regards -- daniel On 11/06/2020 19:14, Peter Kessler OS wrote: > Daniel, > > Thanks for the suggestion to run more tests. > > I'm not sure what the criteria for "success" is here. > > I have run > > $ jtreg-5.0-b01/bin/jtreg \ > -verbose:summary -a -ea -esa -agentvm -conc:8 -ignore:quiet -timeout:2 \ > -exclude:./test/jdk/ProblemList.txt \ > -jdk:${jdk} \ > ./test/jdk/:jdk_core ./test/jdk/:jdk_svc > > on aarch64 and x86_64, using the available OracleJDK build 15-ea+25-1229, > and a build I did of jdk-15+25 with the patch (called jdk-15+25+ below). > The results are ... confusing. > > I ran the tests 4 times on each JDK. The summary of the results on x86_64 > are > > 15-ea+25-1229 > Test results: passed: 6,718; failed: 4; error: 9 > Test results: passed: 6,708; failed: 13; error: 10 > Test results: passed: 6,713; failed: 9; error: 9 > Test results: passed: 6,718; failed: 4; error: 9 > > jdk-15+25+ > Test results: passed: 6,719; failed: 6; error: 9 > Test results: passed: 6,712; failed: 12; error: 10 > Test results: passed: 6,718; failed: 7; error: 9 > Test results: passed: 6,718; failed: 7; error: 9 > > The fact that in different runs different tests pass, fail, or error is > unfortunate. Counting the different failures in the 4 runs shows > > 15-ea+25-1229 > 3 FAILED: com/sun/jdi/PrivateTransportTest.java > 1 FAILED: java/io/Serializable/unresolvableObjectStreamClass/UnresolvableObjectStreamClass.java > 1 FAILED: java/net/httpclient/ProxyAuthDisabledSchemes.java > 2 FAILED: java/net/httpclient/ProxyAuthDisabledSchemesSSL.java > 1 FAILED: java/net/httpclient/ResponseBodyBeforeError.java > 1 FAILED: java/net/httpclient/ResponsePublisher.java > 2 FAILED: java/net/httpclient/ShortResponseBody.java > 2 FAILED: java/net/httpclient/ShortResponseBodyWithRetry.java > 1 FAILED: java/net/httpclient/SpecialHeadersTest.java > 1 FAILED: java/net/httpclient/StreamingBody.java > 4 FAILED: java/security/KeyStore/PKCS12/MetadataStoreLoadTest.java > 1 FAILED: java/security/KeyStore/PKCS12/StoreTrustedCertAPITest.java > 2 FAILED: java/security/KeyStore/PKCS12/StoreTrustedCertKeytool.java > 4 FAILED: sun/security/pkcs11/Secmod/AddTrustedCert.java > 4 FAILED: tools/jlink/plugins/StripNativeDebugSymbolsPlugin/StripNativeDebugSymbolsPlugin > > jdk-15+25+ > 1 FAILED: com/sun/jdi/PrivateTransportTest.java > 3 FAILED: java/io/Serializable/unresolvableObjectStreamClass/UnresolvableObjectStreamClass.java > 1 FAILED: java/net/httpclient/ProxyAuthDisabledSchemes.java > 1 FAILED: java/net/httpclient/ProxyAuthDisabledSchemesSSL.java > 1 FAILED: java/net/httpclient/ResponsePublisher.java > 1 FAILED: java/net/httpclient/ShortResponseBody.java > 1 FAILED: java/net/httpclient/ShortResponseBodyWithRetry.java > 1 FAILED: java/net/httpclient/SpecialHeadersTest.java > 1 FAILED: java/net/httpclient/SplitResponseAsync.java > 1 FAILED: java/net/httpclient/StreamingBody.java > 1 FAILED: java/security/KeyStore/PKCS12/MetadataStoreLoadTest.java > 3 FAILED: java/security/KeyStore/PKCS12/StoreTrustedCertAPITest.java > 2 FAILED: java/security/KeyStore/PKCS12/StoreTrustedCertKeytool.java > 1 FAILED: java/util/logging/Logger/bundleLeak/BundleTest.java > 1 FAILED: jdk/jfr/event/runtime/TestThreadAllocationEvent.java > 4 FAILED: sun/security/lib/cacerts/VerifyCACerts.java > 4 FAILED: sun/security/pkcs11/Secmod/AddTrustedCert.java > 4 FAILED: tools/jlink/plugins/StripNativeDebugSymbolsPlugin/StripNativeDebugSymbolsPluginTest.java > > To your worry about "potential damage", I would have worried about > java/io/Serializable/unresolvableObjectStreamClass/UnresolvableObjectStreamClass.java, > except it did not fail all the time with the patch, and it also failed > without the patch. Many of the other failures seem intermittent. I am not > sure how to classify the failures that persisted across the 4 runs I did. > > The summary of the results on aarch64 is not as good, for either build: > > 15-ea+25-1229 > Test results: passed: 6,658; failed: 74; error: 10 > Test results: passed: 6,657; failed: 74; error: 11 > Test results: passed: 6,660; failed: 71; error: 10 > Test results: passed: 6,663; failed: 69; error: 10 > > jdk-15+25+ > Test results: passed: 6,661; failed: 73; error: 11 > Test results: passed: 6,659; failed: 75; error: 10 > Test results: passed: 6,663; failed: 71; error: 11 > Test results: passed: 6,655; failed: 79; error: 11 > > Comparing the failures and errors in 3rd run of each, the differences from > 15-ea+25-1229 to jdk-15+25+ are > > > Error: sun/tools/jstatd/TestJstatdExternalRegistry.java > < FAILED: com/sun/jdi/PrivateTransportTest.java > > FAILED: java/io/Serializable/unresolvableObjectStreamClass/UnresolvableObjectStreamClass.java > < FAILED: java/security/KeyStore/PKCS12/MetadataStoreLoadTest.java > < FAILED: java/security/KeyStore/PKCS12/StoreTrustedCertAPITest.java > > FAILED: jdk/net/ExtendedSocketOption/AsynchronousSocketChannelNAPITest.java > > FAILED: sun/security/lib/cacerts/VerifyCACerts.java > > Again, UnresolvableObjectStreamClass.java would be worrying, except that it > did not always fail with the patch and also sometimes (not shown) failed > with the OracleJDK 15-ea+25-1229 build. > > Comparing the failures across the 4 runs shows the difference between the > persistent (that is, they failed on all 4 runs) between 15-ea+25-1229 and > jdk-15+25+ to be > > < FAILED: com/sun/jdi/PrivateTransportTest.java > < FAILED: java/security/KeyStore/PKCS12/StoreTrustedCertKeytool.java > > FAILED: sun/security/lib/cacerts/VerifyCACerts.java > > Similar to, but not the same as, the persistent failures on x86_64. > > (Can I attribute the generally higher rate of failure on aarch64 compared to > x86_64 to fact that aarch64 is a new platform for Oracle?) > > What further testing would you recommend? > > ... peter > > -----Original Message----- > From: Daniel Fuchs > Date: Tuesday, June 9, 2020 at 10:18 AM > To: Peter Kessler OS , "core-libs-dev at openjdk.java.net" > Subject: Re: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) > > Hi Peter, > > This is not a review. I second Roger's observations. > > On 05/06/2020 00:08, Peter Kessler OS wrote: > > Tested with `make run-test-tier1` on Linux (CentOS 7) on both aarch64 and x86_64. I had one failure on each of the runs > > The code you are proposing to change affects RMI and JMX too. > For an initial evaluation of the potential damage you will need to > test at least with jdk_core and jdk_svc (and if that's successful > then probably some of the lower tiers too): > > $ jtreg -verbose:summary -a -ea -esa -agentvm -conc:8 -ignore:quiet > -timeout:2 -exclude:./test/jdk/ProblemList.txt -jdk: > ./test/jdk/:jdk_core ./test/jdk/:jdk_svc > > > best regards, > > -- daniel > From peter.kessler at os.amperecomputing.com Fri Jun 12 23:19:13 2020 From: peter.kessler at os.amperecomputing.com (Peter Kessler (Open Source)) Date: Fri, 12 Jun 2020 23:19:13 +0000 Subject: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) In-Reply-To: References: <814DA850-76A9-4526-A3D9-141CD516C9AE@amperemail.onmicrosoft.com> <50955EDB-CCC2-4334-A4AF-2EB422E9CAA9@amperemail.onmicrosoft.com> Message-ID: <3B38456A-6BD0-4B95-9F9C-E73F7FE3A78F@amperemail.onmicrosoft.com> Roger, I did think about confining the changes to ObjectInputStream. Maybe I did not think hard enough about it. Keeping a cache of the result of a first stack walk might work for the recursive calls. One might still do a useless stack walk for each top-level readObject call. No concurrent class loading can invalidate the cache for any particular stack walk. But it seems fraught to reuse a cached value in the face of concurrent class loading, or in the presence of overloaded ObjectInputStream methods that might create new ClassLoaders on this stack. I'm sure I don't want to debug that. My experiments have shown that the average stack walk is about 20 frames. I do not have the data on how many of those frames are recursive calls within ObjectInputStream methods and how many are in other code. Dragging in the StackWalker API would further complicate things, in my opinion. It might appear to keep the code out of platform native code, but we know that walking thread stacks involves native code. I settled on a system-wide one-bit cache: There have been no user-defined ClassLoaders constructed, so I can not find one on any particular stack walk. It may not be the best possible solution, but it addresses the issue of useless stack walks in the common case where there are no user-defined ClassLoaders. I agree that both ClassLoader and ObjectInputStream are complex. In fact, the proposed patch makes no changes to ObjectInputStream. There is one line added to each of the system-defined ClassLoaders to identify them as not user-defined ClassLoaders, one line in the base ClassLoader constructor, and the bulk of the change is around the guard in jdk.internal.misc.VM.latestUserDefinedLoader() to avoid the stack walk if possible. ... peter ?-----Original Message----- From: Roger Riggs Organization: Java Products Group, Oracle Date: Friday, June 12, 2020 at 11:50 AM To: "Peter Kessler (Open Source)" , "core-libs-dev at openjdk.java.net" Subject: Re: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) Hi Peter, The hazard to avoid is the cross package coupling that makes both ClassLoader and ObjectInputStream more complex; both are more than sufficiently complex already. Optimizing the implementations make sense if it benefits enough uses cases and does not make the code harder to maintain and evolve. Have you explored any alternatives that can be confined to OIS? For example, taking advantage the many calls to readObject within a single OIS that are nested. Possibly the first (depth = 0) find user class loader can be re-used or at least to short circuit the search on subsequent nested calls? Or caching the classloader at each depth and only need to probe for it if it has not already been recorded. The StackWalker API may be useful to limit the depth of searches. Thanks, Roger On 6/11/20 2:52 PM, Peter Kessler OS wrote: > Hi Roger, > > Thanks for your comments. > > I am not knowingly changing the required behavior. All I am proposing is > guarding the thread stack walk if it will be a waste of effort. If an > application or framework has its own ClassLoaders, then the guard will fail > and the thread stack will be walked. > > The stack walk is important, if there are user-defined ClassLoaders. A > method that uses ObjectInputStream to read objects can not know what > ClassLoader to use to resolve class references in the stream, without the > stack walk. But if there never have been any user-defined ClassLoaders > constructed, then the stack walk can never return a user-defined > ClassLoader. > > If the stream reader is part of some simple application, then it may spend > a significant amount of its time uselessly walking thread stacks. (If the > stream reader is part of some simple application its author may not know to > override ObjectInputStream.resolveClass, or how to determine what > ClassLoader should load a class from the stream.) If someone were using > such an application to choose platforms, the Java platform would be at a > disadvantage. > > Consider a somewhat more complex case, the SPECjbb2015 benchmark, which > claims to model a well-constructed business application on the Java > platform. SPECjbb2015 runs multiple JVMs, exchanging information via > ObjectInputStreams. Profiling shows that it spends more than 5% of its time > walking thread stacks. Yet, in spite of being a complex application, > written by experts in the Java platform, there are no user-defined > ClassLoaders. So all the time spent walking stacks is wasted time, and is > one of the reasons the servers fail their service-level agreements. We can > discuss whether SPECjbb2015 is representative of Java applications. But it > is easily available and relatively simple to run. People use benchmarks > like SPECjbb2015 to decide whether to use the Java platform for particular > tasks, and it would be unfortunate if we did not make the Java platform as > good as it could be. > > ... peter > > -----Original Message----- > From: core-libs-dev on behalf of Roger Riggs > Organization: Java Products Group, Oracle > Date: Tuesday, June 9, 2020 at 6:43 AM > To: "core-libs-dev at openjdk.java.net" > Subject: Re: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) > > Hi Peter, > > I'd like to understand the scope and impact of the problem before > jumping to a solution. > For specific applications, overriding ObjectInputStream.resolveClass to > handle the class > lookup can handle the case as expeditiously as is necessary. > > What application uses cases are impacted and how prevelant are they? > Many, many frameworks and applications create classloaders for their own > purposes and > take advantage of the required behavior. > > Thanks, Roger > > > On 6/4/20 7:08 PM, Peter Kessler OS wrote: > > ObjectInputStream.resolveClass(ObjectStreamClass) is specified to find "the first class loader on the current thread's stack (starting from the currently executing method) that is neither the platform class loader nor its ancestor." Finding that class loader is done with a virtual frame walk in JVM_LatestUserDefinedLoader(JNIEnv *env). The virtual frame walk is expensive. If an application does not define any ClassLoaders, then the virtual frame walk will always find the system class loader. > > > > If an application does a significant amount of reading from an ObjectInputStream, the cost of ObjectInputStream.resolveClass(ObjectInputStreamClass) can be considerable. In a sample application, that method (and the methods it calls) have been measured to consume 5+% of the CPU cycles. > > > > The proposal is to note the construction of any user-defined ClassLoader, and if none has been constructed, avoid the virtual frame walk returning ClassLoader.getSystemClassLoader(). But if any user-defined ClassLoader has been constructed, use the virtual frame walk to find the first user-defined ClassLoader on the thread stack. > > > > User-defined ClassLoaders could be distinguished from non-user-defined ClassLoaders in several ways. The ClassLoader class hierarchy could be changed to make non-user-defined ClassLoaders inherit from a marker class. There is already a jdk.internal.loader.BuiltinClassLoader but it is used for module (and resource) loading. BuiltinClassLoader is not a super-class of jdk.internal.reflect.DelegatingClassLoader, which is used to load reflection classes, and reflection methods are also ignored by the virtual frame walk. ClassLoaders could have a boolean that distinguished whether they are user-defined or not. That would require a change to ClassLoader API. There are other ways to distinguish user-defined ClassLoaders. The proposal is to keep the decoration of the ClassLoaders out of the instances, so that it is only visible to the code that needs it. > > > > The proposal is that each non-user-defined ClassLoader have a static block that registers the class as a non-user-defined ClassLoader. Then in the base ClassLoader constructor check if the ClassLoader being constructed is a user-defined ClassLoader and if so set a boolean that acts as the guard on the virtual frame walk. If additional non-user-defined ClassLoaders are declared in the future beyond the ones identified in this patch, they can be added to the registry. > > > > There are currently 4 non-user-defined ClassLoader classes, so the registry is not difficult to maintain. Nor is the registry difficult to search when ClassLoaders are constructed. The guard boolean that records whether a user-defined ClassLoader has been constructed transitions from false to true and never becomes false again. (An extension to transition the boolean back to false when a user-defined ClassLoader becomes unreachable is beyond the scope of this proposal.) > > > > This proposal slows loading of non-user-defined ClassLoaders by the time it takes to register them. It slows ClassLoader construction by the time it takes to consult the registry and conditionally set the guard boolean. At each invocation of ObjectInputStream.resolveClass(ObjectInputStreamClass), the guard boolean is used to possibly avoid the virtual frame walk. > > > > Tested with `make run-test-tier1` on Linux (CentOS 7) on both aarch64 and x86_64. I had one failure on each of the runs > > > > ACTION: main -- Failed. Execution failed: `main' threw exception: java.lang.Error: TESTBUG: the library has not been found in ${Repo}/build/linux-aarch64-server-release/images/test/hotspot/jtreg/native > > REASON: User specified action: run main/native GTestWrapper > > > > on both plaforms. But I get the same failure when I run the tier1 tests on clones of OpenJDK-15+25 on both aarch64 and x86_64. > > > > Running a sample (representative?) application with the Oracle Developer Studio analyzer shows a performance comparison of > > > > Stack Fragment sorted by metric: Attributed Total CPU Time > > baseline.er experiment.er > > Attributed Attributed Name > > Total CPU Time Total CPU Time > > sec. sec. > > ================================== Callers > > 3459.210 29.931 java.io.ObjectInputStream.readOrdinaryObject(boolean) > > 1139.727 3.532 java.io.ObjectInputStream.readArray(boolean) > > 875.262 9.116 java.io.ObjectInputStream.readNonProxyDesc(boolean) > > > > ================================== Stack Fragment > > java.io.ObjectInputStream.readClassDesc(boolean) > > java.io.ObjectInputStream.readNonProxyDesc(boolean) > > java.io.ObjectInputStream.resolveClass(java.io.ObjectStreamClass) > > java.io.ObjectInputStream.latestUserDefinedLoader() > > 4.173 3.953 jdk.internal.misc.VM.latestUserDefinedLoader() > > > > ================================== Callees > > 5470.026 0. jdk.internal.misc.VM.latestUserDefinedLoader0() > > 0. 38.627 java.lang.ClassLoader.getSystemClassLoader() > > > > The `hg export -g` below is with respect to OpenJDK-15+25. > > > > Thank you for your review comments. I will need a sponsor to get this change into the repository. > > > > ... peter > > > > # HG changeset patch > > # User Peter Kessler > > # Date 1591310467 25200 > > # Thu Jun 04 15:41:07 2020 -0700 > > # Node ID 9a39488182c1dfc5bc7bb41d562d7ef16ee657f6 > > # Parent 90b266a84c06f1b3dc0ed8767856793e8c1c357e > > Improve the performance of ObjectInputStream.resolveClass > > > > diff --git a/src/java.base/share/classes/java/lang/ClassLoader.java b/src/java.base/share/classes/java/lang/ClassLoader.java > > --- a/src/java.base/share/classes/java/lang/ClassLoader.java > > +++ b/src/java.base/share/classes/java/lang/ClassLoader.java > > @@ -385,6 +385,8 @@ > > } > > this.package2certs = new ConcurrentHashMap<>(); > > this.nameAndId = nameAndId(this); > > + /* Note the construction of a ClassLoader. */ > > + VM.noteClassLoaderConstruction(this.getClass()); > > } > > > > /** > > diff --git a/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java b/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java > > --- a/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java > > +++ b/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java > > @@ -117,6 +117,11 @@ > > protected Class loadClassOrNull(String cn, boolean resolve) { > > return JLA.findBootstrapClassOrNull(this, cn); > > } > > + > > + static { > > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > > + VM.registerNotUserDefinedClassLoader(BootClassLoader.class); > > + } > > }; > > > > /** > > @@ -127,6 +132,8 @@ > > static { > > if (!ClassLoader.registerAsParallelCapable()) > > throw new InternalError(); > > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > > + VM.registerNotUserDefinedClassLoader(PlatformClassLoader.class); > > } > > > > PlatformClassLoader(BootClassLoader parent) { > > @@ -142,6 +149,8 @@ > > static { > > if (!ClassLoader.registerAsParallelCapable()) > > throw new InternalError(); > > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > > + VM.registerNotUserDefinedClassLoader(AppClassLoader.class); > > } > > > > final URLClassPath ucp; > > diff --git a/src/java.base/share/classes/jdk/internal/misc/VM.java b/src/java.base/share/classes/jdk/internal/misc/VM.java > > --- a/src/java.base/share/classes/jdk/internal/misc/VM.java > > +++ b/src/java.base/share/classes/jdk/internal/misc/VM.java > > @@ -304,12 +304,44 @@ > > private static final int JVMTI_THREAD_STATE_WAITING_INDEFINITELY = 0x0010; > > private static final int JVMTI_THREAD_STATE_WAITING_WITH_TIMEOUT = 0x0020; > > > > + /** A registry of not user-defined ClassLoaders. */ > > + private static final List> notUserDefinedClassLoaderRegistry = > > + Collections.synchronizedList(new ArrayList<>()); > > + > > + /** Register a ClassLoader class as being not a user-defined ClassLoader. */ > > + public static void registerNotUserDefinedClassLoader(Class classLoaderClass) { > > + notUserDefinedClassLoaderRegistry.add(classLoaderClass); > > + } > > + > > + /** A flag for whether a user-defined ClassLoaders has been constructed. */ > > + private static volatile boolean constructedUserDefinedClassLoaderFlag = false; > > + > > + /** > > + * Note a ClassLoader construction, and record if a > > + * user-defined ClassLoader has been constructed. > > + */ > > + public static void noteClassLoaderConstruction(Class classLoaderClass) { > > + /* Check if the ClassLoader class not a user-defined ClassLoader. */ > > + if (notUserDefinedClassLoaderRegistry.contains(classLoaderClass)) { > > + return; > > + } > > + constructedUserDefinedClassLoaderFlag = true; > > + return; > > + } > > + > > /* > > * Returns the first user-defined class loader up the execution stack, > > * or the platform class loader if only code from the platform or > > * bootstrap class loader is on the stack. > > */ > > public static ClassLoader latestUserDefinedLoader() { > > + if (!constructedUserDefinedClassLoaderFlag) { > > + /* > > + * If no user-defined ClassLoader has been constructed, > > + * then I will not find a user-defined ClassLoader in the stack. > > + */ > > + return ClassLoader.getSystemClassLoader(); > > + } > > ClassLoader loader = latestUserDefinedLoader0(); > > return loader != null ? loader : ClassLoader.getPlatformClassLoader(); > > } > > diff --git a/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java b/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java > > --- a/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java > > +++ b/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java > > @@ -30,6 +30,7 @@ > > > > import jdk.internal.access.JavaLangAccess; > > import jdk.internal.access.SharedSecrets; > > +import jdk.internal.misc.VM; > > > > /** Utility class which assists in calling defineClass() by > > creating a new class loader which delegates to the one needed in > > @@ -73,4 +74,9 @@ > > DelegatingClassLoader(ClassLoader parent) { > > super(parent); > > } > > + > > + static { > > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > > + VM.registerNotUserDefinedClassLoader(DelegatingClassLoader.class); > > + } > > } > > > > From johnlinp at gmail.com Sat Jun 13 03:08:18 2020 From: johnlinp at gmail.com (=?UTF-8?B?5p6X6Ieq5Z2H?=) Date: Sat, 13 Jun 2020 11:08:18 +0800 Subject: [PATCH] 8247402: Rewrite the implementation requirements for Map::compute() In-Reply-To: References: Message-ID: Hi Martin, I see your point. Thank you for demostrating this for me. Here's my updated patch: # HG changeset patch # User John Lin # Date 1591923561 -28800 # Fri Jun 12 08:59:21 2020 +0800 # Node ID e01d9d020506a88d3d585bd3264594a26450c659 # Parent 49a68abdb0ba68351db0f140ddac793b1c391bd5 8247402: Rewrite the implementation requirements for Map::compute() diff --git a/src/java.base/share/classes/java/util/Map.java b/src/java.base/share/classes/java/util/Map.java --- a/src/java.base/share/classes/java/util/Map.java +++ b/src/java.base/share/classes/java/util/Map.java @@ -1113,17 +1113,12 @@ public interface Map { *

       {@code
            * V oldValue = map.get(key);
            * V newValue = remappingFunction.apply(key, oldValue);
      -     * if (oldValue != null) {
      -     *    if (newValue != null)
      -     *       map.put(key, newValue);
      -     *    else
      -     *       map.remove(key);
      +     * if (newValue != null) {
      +     *     map.put(key, newValue);
            * } else {
      -     *    if (newValue != null)
      -     *       map.put(key, newValue);
      -     *    else
      -     *       return null;
      +     *     map.remove(key);
            * }
      +     * return newValue;
            * }
      * *

      The default implementation makes no guarantees about detecting if the By the way, I saw everyone in this mailing list is using webrev. However, the tutorial https://openjdk.java.net/guide/codeReview.html says that only the users with push access to the OpenJDK Mercurial server can use webrev. Can I apply for push access to the OpenJDK Mercurial server too? Best, John Lin Martin Buchholz ? 2020?6?13? ?? ??12:36??? > > If I rub that program, I get: > > HashMap false > HashMap1 false > HashMap2 true > > which suggests that HashMap2's implementation is wrong. From igor.ignatyev at oracle.com Sat Jun 13 03:36:27 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 12 Jun 2020 20:36:27 -0700 Subject: RFR(S) : 8211977 : move testlibrary tests into one place Message-ID: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ > 796 lines changed: 200 ins; 588 del; 8 mod; Hi all, could you please review this small patch which puts all tests for testlibrary classes into one location under /test/lib-test? besides moving tests from test/jdk/lib/testlibrary and test/hotspot/jtreg/testlibrary_tests to test/lib-test the patch also - problem lists HexPrinterTest.java on windows due to JDK-8247521 - introduces make targets to build native parts for the tests in test/lib-test (needed b/c one test has a native part) - adds randomness k/w to test/lib-test (as it's used by RandomGeneratorTest.java) - makes Test class used by TestNativeProcessBuilder a static nested class of TestNativeProcessBuilder - updates LingeredAppTest to use @build instead of @compile and adds necessary @library tag - removes AssertsTest.java, OutputAnalyzerTest.java from test/hotspot/jtreg/testlibrary_tests as they are either identical or lesser that the same tests from test/jdk/lib/testlibrary/ - merges test/hotspot/jtreg/testlibrary_tests/OutputAnalyzerTest.java and test/jdk/lib/testlibrary/OutputAnalyzerTest.java (effectively adds test cases for `firstMatch` to the superier copy from test/jdk/lib/testlibrary) webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 testing: test/lib-test on {windows,linux,macosx}-x64 Thanks, -- Igor From igor.ignatyev at oracle.com Sat Jun 13 03:38:27 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 12 Jun 2020 20:38:27 -0700 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: References: Message-ID: adding build-dev > On Jun 12, 2020, at 8:36 PM, Igor Ignatyev wrote: > > http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ >> 796 lines changed: 200 ins; 588 del; 8 mod; > > Hi all, > > could you please review this small patch which puts all tests for testlibrary classes into one location under /test/lib-test? > > besides moving tests from test/jdk/lib/testlibrary and test/hotspot/jtreg/testlibrary_tests to test/lib-test the patch also > - problem lists HexPrinterTest.java on windows due to JDK-8247521 > - introduces make targets to build native parts for the tests in test/lib-test (needed b/c one test has a native part) > - adds randomness k/w to test/lib-test (as it's used by RandomGeneratorTest.java) > - makes Test class used by TestNativeProcessBuilder a static nested class of TestNativeProcessBuilder > - updates LingeredAppTest to use @build instead of @compile and adds necessary @library tag > - removes AssertsTest.java, OutputAnalyzerTest.java from test/hotspot/jtreg/testlibrary_tests as they are either identical or lesser that the same tests from test/jdk/lib/testlibrary/ > - merges test/hotspot/jtreg/testlibrary_tests/OutputAnalyzerTest.java and test/jdk/lib/testlibrary/OutputAnalyzerTest.java (effectively adds test cases for `firstMatch` to the superier copy from test/jdk/lib/testlibrary) > > webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ > JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 > testing: test/lib-test on {windows,linux,macosx}-x64 > > Thanks, > -- Igor > > From igor.ignatyev at oracle.com Sat Jun 13 04:10:32 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 12 Jun 2020 21:10:32 -0700 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: References: Message-ID: testing revealed that LingeredAppTest.java required some love, incremental webrev w/ the fixes for LingeredAppTest -- http://cr.openjdk.java.net/~iignatyev//8211977/webrev.0-1 -- Igor > On Jun 12, 2020, at 8:38 PM, Igor Ignatyev wrote: > > adding build-dev > >> On Jun 12, 2020, at 8:36 PM, Igor Ignatyev wrote: >> >> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ >>> 796 lines changed: 200 ins; 588 del; 8 mod; >> >> Hi all, >> >> could you please review this small patch which puts all tests for testlibrary classes into one location under /test/lib-test? >> >> besides moving tests from test/jdk/lib/testlibrary and test/hotspot/jtreg/testlibrary_tests to test/lib-test the patch also >> - problem lists HexPrinterTest.java on windows due to JDK-8247521 >> - introduces make targets to build native parts for the tests in test/lib-test (needed b/c one test has a native part) >> - adds randomness k/w to test/lib-test (as it's used by RandomGeneratorTest.java) >> - makes Test class used by TestNativeProcessBuilder a static nested class of TestNativeProcessBuilder >> - updates LingeredAppTest to use @build instead of @compile and adds necessary @library tag >> - removes AssertsTest.java, OutputAnalyzerTest.java from test/hotspot/jtreg/testlibrary_tests as they are either identical or lesser that the same tests from test/jdk/lib/testlibrary/ >> - merges test/hotspot/jtreg/testlibrary_tests/OutputAnalyzerTest.java and test/jdk/lib/testlibrary/OutputAnalyzerTest.java (effectively adds test cases for `firstMatch` to the superier copy from test/jdk/lib/testlibrary) >> >> webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ >> JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 >> testing: test/lib-test on {windows,linux,macosx}-x64 >> >> Thanks, >> -- Igor >> >> > From amaembo at gmail.com Sat Jun 13 05:08:57 2020 From: amaembo at gmail.com (Tagir Valeev) Date: Sat, 13 Jun 2020 12:08:57 +0700 Subject: Optimize StringConcatHelper::simpleConcat for empty strings Message-ID: Hello! It's quite possible that when we concatenate two strings, one of them appears to be empty. We cannot simply return another string in this case, as JLS 15.18.1 explicitly says (for unknown to me reason) about the result of the string concatenation expression that 'The String object is newly created'. However, it's still possible to reuse the internal array of another string to reduce allocations: --- src/java.base/share/classes/java/lang/StringConcatHelper.java (revision 58998:04e3d254c76be87788a40cbd66d013140ea951d8) +++ src/java.base/share/classes/java/lang/StringConcatHelper.java (revision 58998+:04e3d254c76b+) @@ -420,6 +420,12 @@ static String simpleConcat(Object first, Object second) { String s1 = stringOf(first); String s2 = stringOf(second); + if (s1.isEmpty()) { + return new String(s2.value(), s2.coder()); + } + if (s2.isEmpty()) { + return new String(s1.value(), s1.coder()); + } // start "mixing" in length and coder or arguments, order is not // important long indexCoder = mix(initialCoder(), s2); Very simple benchmark like this validates that the concatenation became faster if one of the strings is empty: @Param({"", "longlonglongline"}) String data; @Param({"", "longlonglongline"}) String data2; @Benchmark public String plus() { return data + data2; } Without patch I observe on VM 15-ea+20-899: Benchmark (data) (data2) Mode Cnt Score Error Units plus avgt 30 15,335 ? 0,186 ns/op plus longlonglongline avgt 30 19,867 ? 0,109 ns/op plus longlonglongline avgt 30 20,283 ? 0,230 ns/op plus longlonglongline longlonglongline avgt 30 26,047 ? 0,230 ns/op With patch: Benchmark (data) (data2) Mode Cnt Score Error Units plus avgt 30 6,668 ? 0,055 ns/op plus longlonglongline avgt 30 6,708 ? 0,114 ns/op plus longlonglongline avgt 30 7,003 ? 0,064 ns/op plus longlonglongline longlonglongline avgt 30 25,126 ? 0,392 ns/op There could be an added cost of up to two branches for the normal case (I believe, if one of the strings is constant, then decent JIT can eliminate one of the branches). However, I believe, the benefit could outweigh it, as empty strings are not that uncommon and in this case, we reduce O(N) time and memory consumption to O(1). What do you think? Is this a reasonable thing to do? I can file an issue and submit a proper webrev if it looks like a useful patch. With best regards, Tagir Valeev. From Alan.Bateman at oracle.com Sat Jun 13 14:23:59 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 13 Jun 2020 15:23:59 +0100 Subject: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) In-Reply-To: <3EC88274-ACE8-43F6-A0BA-299182CEC451@amperemail.onmicrosoft.com> References: <814DA850-76A9-4526-A3D9-141CD516C9AE@amperemail.onmicrosoft.com> <71ea732b-4566-e0a0-b2a3-e71df8a79609@oracle.com> <0E094A25-70C6-455D-A01D-F524A8010210@amperemail.onmicrosoft.com> <04b1cf61-c1cf-89eb-bcf2-ddd0f69e9321@oracle.com> <3EC88274-ACE8-43F6-A0BA-299182CEC451@amperemail.onmicrosoft.com> Message-ID: <85917488-6695-8ffa-89bd-35c1e5118c1f@oracle.com> On 12/06/2020 21:04, Peter Kessler (Open Source) wrote: > Daniel, > > Thanks for the suggestions to increase the timeout and lower the > concurrency. I did Bad Science(TM) and changed them both at the same > time. I am now running > > $ jtreg-5.0-b01/bin/jtreg \ > -verbose:summary -a -ea -esa -agentvm -conc:1 -ignore:quiet -timeout:4 \ > -exclude:./test/jdk/ProblemList.txt \ > -jdk:${jdk} \ > ./test/jdk/:jdk_core ./test/jdk/:jdk_svc > > I only had time for 2 tests of each jdk overnight, because running that > way takes much longer, but the results are less confusing. > > Running on x86_64, counting failures and errors in 2 runs with OracleJDK > build 15-ea+25-1229 > > 2 Error: java/lang/ClassLoader/nativeLibrary/NativeLibraryTest.java > 2 Error: java/lang/instrument/HiddenClass/HiddenClassAgent.java > 2 Error: java/lang/reflect/exeCallerAccessTest/CallerAccessTest.java > 2 Error: java/lang/String/nativeEncoding/StringPlatformChars.java > 2 Error: java/nio/channels/FileChannel/directio/DirectIOTest.java > 2 Error: java/nio/channels/spi/SelectorProvider/inheritedChannel/InheritedChannelTest.java > 2 Error: jdk/internal/loader/NativeLibraries/Main.java > 2 Error: Some tests failed or other problems occurred. > 2 Error: sun/management/jmxremote/bootstrap/CustomLauncherTest.java > 2 Error: tools/launcher/JliLaunchTest.java > These are tests that use native libraries and need jtreg run with -nativepath with the path to these libraries. The "test" make target might be want you want, to avoid your jtreg command getting too complicated. -Alan From Alan.Bateman at oracle.com Sat Jun 13 16:06:06 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 13 Jun 2020 17:06:06 +0100 Subject: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) In-Reply-To: <3B38456A-6BD0-4B95-9F9C-E73F7FE3A78F@amperemail.onmicrosoft.com> References: <814DA850-76A9-4526-A3D9-141CD516C9AE@amperemail.onmicrosoft.com> <50955EDB-CCC2-4334-A4AF-2EB422E9CAA9@amperemail.onmicrosoft.com> <3B38456A-6BD0-4B95-9F9C-E73F7FE3A78F@amperemail.onmicrosoft.com> Message-ID: <480848bd-05db-2555-5fe7-686750e44dc9@oracle.com> On 13/06/2020 00:19, Peter Kessler (Open Source) wrote: > : > > I settled on a system-wide one-bit cache: There have been no > user-defined ClassLoaders constructed, so I can not find one on any > particular stack walk. It may not be the best possible solution, but > it addresses the issue of useless stack walks in the common case where > there are no user-defined ClassLoaders. > Does this help anything other than SPECjbb2015? Real world code will use core reflection and other areas of the JDK that create class loaders at run-time so I assume this optimization will be disabled very quickly. I think it would be better to look for other solutions. -Alan From kim.barrett at oracle.com Sun Jun 14 03:48:01 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 13 Jun 2020 23:48:01 -0400 Subject: [OpenJDK 2D-Dev] RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: References: <5EDE9A75.6020504@oracle.com> Message-ID: > On Jun 10, 2020, at 2:26 AM, Kim Barrett wrote: > >> On Jun 8, 2020, at 4:07 PM, Philip Race wrote: > >> BTW the JEP (https://bugs.openjdk.java.net/browse/JDK-8208089) still says >>> For Solaris: Add the -std=c++14 compiler option. ..... >> >> So I am sure there is still some room to update the JEP :-) > > Yeah, I have some updates to make. I also need to do a bit of work on the feature lists. I think the JEP updates are done now. From philip.race at oracle.com Sun Jun 14 04:45:37 2020 From: philip.race at oracle.com (Philip Race) Date: Sat, 13 Jun 2020 21:45:37 -0700 Subject: [OpenJDK 2D-Dev] RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: References: <5EDE9A75.6020504@oracle.com> Message-ID: <5EE5AB71.20505@oracle.com> Kim, Until it says in "the JDK" and not "in HotSpot" you have not addressed my main point. Please rename the JEP. -phil. On 6/13/20, 8:48 PM, Kim Barrett wrote: >> On Jun 10, 2020, at 2:26 AM, Kim Barrett wrote: >> >>> On Jun 8, 2020, at 4:07 PM, Philip Race wrote: >>> BTW the JEP (https://bugs.openjdk.java.net/browse/JDK-8208089) still says >>>> For Solaris: Add the -std=c++14 compiler option. ..... >>> So I am sure there is still some room to update the JEP :-) >> Yeah, I have some updates to make. I also need to do a bit of work on the feature lists. > I think the JEP updates are done now. > From Alan.Bateman at oracle.com Sun Jun 14 13:22:56 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 14 Jun 2020 14:22:56 +0100 Subject: RFR 8245302: Upgrade LogRecord to support long thread ids and remove its usage of ThreadLocal In-Reply-To: References: <94B2555B-981F-40EB-AB5F-31E2BE068FF0@oracle.com> Message-ID: <03c38ee2-24c2-cf2d-e428-3f8747888978@oracle.com> On 10/06/2020 16:06, Rahul wrote: > ? Hello, > > > > Request to have my fix reviewed for the issue: > > > > JDK-8245302: Upgrade LogRecord to support long thread ids and remove its usage of ThreadLocal. > > > > java.util.logging.LogRecord is updated to support long thread ids without ThreadLocal. > > Before the update, thread id?s did not support long values, ThreadLocal usage has been > > Removed. > > . > > > > issue: https://bugs.openjdk.java.net/browse/JDK-8245302 > > webrev: http://cr.openjdk.java.net/~ryadav/webrev_8245302/index.html > > csr: https://bugs.openjdk.java.net/browse/JDK-8247219 > > The addition of setLongThreadID/getLongThreadID, and deprecating the old methods looks good. One suggestion is that setLongThreadID can return the LogRecord (= this) to allow for method invocation chaining. For the javadoc then it would be clearer to say "Returns the ..." rather than "Get a ..." because a sequence of calls to getXXX has to return the same thread ID. LogRecord is serializable but it looks like like you've thought about all the issue so I think this looks. shortThreadID looks a bit strange in that I would have expected it would just return tid when in the 0-Integer.MAX_VALUE range and compute a negative value when it's larger than Integer.MAX_VALUE. It's true that the old behavior was to switch to dense values when larger than MAX_VALUE/2 but I can't imagine how anything could depend on this. So not objecting to the approach, just pointing out that there may be simpler alternatives that could be explored (if you haven't done so already). The overall test cover looks good. Can you provide clear instructions in SerializeLogRecordTest on how to re-create the base64 string of the serialized form? That is important for future maintainers. There are several formatting/style nits in the patch but they aren't important for now. -Alan. From peter.levart at gmail.com Sun Jun 14 16:28:21 2020 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 14 Jun 2020 18:28:21 +0200 Subject: RFR: 8247532: Records deserialization is slow Message-ID: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> Hi, I noticed that deserializing records (new preview Java feature in JDK14 and JDK15) is slow compared to equivalent classical classes. I created a JMH benchmark [1] to se how it compares (ran it on JDK14): Benchmark???????????????????????????????????????????????????????????????????? Mode? Cnt?????? Score??????? Error?? Units RecordSerializationBench.deserializeClasses?????????????????????????????????? avgt?? 10????? 31.911 ?????? 0.218?? us/op RecordSerializationBench.deserializeClasses:?gc.alloc.rate??????????????????? avgt?? 10???? 815.106 ?????? 5.545? MB/sec RecordSerializationBench.deserializeClasses:?gc.alloc.rate.norm?????????????? avgt?? 10?? 40921.903 ?????? 0.615??? B/op RecordSerializationBench.deserializeClasses:?gc.churn.G1_Eden_Space?????????? avgt?? 10???? 839.522 ???? 191.195? MB/sec RecordSerializationBench.deserializeClasses:?gc.churn.G1_Eden_Space.norm????? avgt?? 10?? 42153.661 ??? 9682.799??? B/op RecordSerializationBench.deserializeClasses:?gc.churn.G1_Survivor_Space?????? avgt?? 10?????? 0.117 ?????? 0.492? MB/sec RecordSerializationBench.deserializeClasses:?gc.churn.G1_Survivor_Space.norm? avgt?? 10?????? 5.835 ????? 24.447??? B/op RecordSerializationBench.deserializeClasses:?gc.count???????????????????????? avgt?? 10????? 21.000?????????????? counts RecordSerializationBench.deserializeClasses:?gc.time????????????????????????? avgt?? 10????? 17.000?????????????????? ms RecordSerializationBench.deserializeRecords?????????????????????????????????? avgt?? 10???? 531.333 ?????? 3.094?? us/op RecordSerializationBench.deserializeRecords:?gc.alloc.rate??????????????????? avgt?? 10???? 346.511 ?????? 1.997? MB/sec RecordSerializationBench.deserializeRecords:?gc.alloc.rate.norm?????????????? avgt?? 10? 289637.193 ?????? 6.894??? B/op RecordSerializationBench.deserializeRecords:?gc.churn.G1_Eden_Space?????????? avgt?? 10???? 359.773 ???? 191.116? MB/sec RecordSerializationBench.deserializeRecords:?gc.churn.G1_Eden_Space.norm????? avgt?? 10? 300657.838 ? 159724.346??? B/op RecordSerializationBench.deserializeRecords:?gc.churn.G1_Survivor_Space?????? avgt?? 10?????? 0.007 ?????? 0.012? MB/sec RecordSerializationBench.deserializeRecords:?gc.churn.G1_Survivor_Space.norm? avgt?? 10?????? 6.020 ?????? 9.910??? B/op RecordSerializationBench.deserializeRecords:?gc.count???????????????????????? avgt?? 10?????? 9.000?????????????? counts RecordSerializationBench.deserializeRecords:?gc.time????????????????????????? avgt?? 10????? 14.000?????????????????? ms ...not only it is it about 16x slower, it also produces 7x garbage. I checked the code and it is not very optimal. It matches the record component names with object stream fields in O(n^2) way. It uses method handles but binds arguments of canonical constructor each time an instance of record is constructed. So I tried to optimize it [2] and with that patch on top of JDK16 the benchmark produces the following results: Benchmark???????????????????????????????????????????????????????????????????? Mode? Cnt????? Score?????? Error?? Units RecordSerializationBench.deserializeClasses?????????????????????????????????? avgt?? 10???? 31.049 ????? 0.235?? us/op RecordSerializationBench.deserializeClasses:?gc.alloc.rate??????????????????? avgt?? 10??? 837.614 ????? 6.326? MB/sec RecordSerializationBench.deserializeClasses:?gc.alloc.rate.norm?????????????? avgt?? 10? 40921.931 ????? 0.666??? B/op RecordSerializationBench.deserializeClasses:?gc.churn.G1_Eden_Space?????????? avgt?? 10??? 867.743 ??? 251.373? MB/sec RecordSerializationBench.deserializeClasses:?gc.churn.G1_Eden_Space.norm????? avgt?? 10? 42405.532 ? 12403.301??? B/op RecordSerializationBench.deserializeClasses:?gc.churn.G1_Survivor_Space?????? avgt?? 10????? 0.126 ????? 0.478? MB/sec RecordSerializationBench.deserializeClasses:?gc.churn.G1_Survivor_Space.norm? avgt?? 10????? 6.113 ???? 23.268??? B/op RecordSerializationBench.deserializeClasses:?gc.count???????????????????????? avgt?? 10???? 22.000????????????? counts RecordSerializationBench.deserializeClasses:?gc.time????????????????????????? avgt?? 10???? 19.000????????????????? ms RecordSerializationBench.deserializeRecords?????????????????????????????????? avgt?? 10???? 33.588 ????? 0.394?? us/op RecordSerializationBench.deserializeRecords:?gc.alloc.rate??????????????????? avgt?? 10??? 500.033 ????? 5.871? MB/sec RecordSerializationBench.deserializeRecords:?gc.alloc.rate.norm?????????????? avgt?? 10? 26425.293 ????? 0.759??? B/op RecordSerializationBench.deserializeRecords:?gc.churn.G1_Eden_Space?????????? avgt?? 10??? 512.772 ??? 288.112? MB/sec RecordSerializationBench.deserializeRecords:?gc.churn.G1_Eden_Space.norm????? avgt?? 10? 27090.499 ? 15175.280??? B/op RecordSerializationBench.deserializeRecords:?gc.churn.G1_Survivor_Space?????? avgt?? 10????? 0.134 ????? 0.496? MB/sec RecordSerializationBench.deserializeRecords:?gc.churn.G1_Survivor_Space.norm? avgt?? 10????? 7.128 ???? 26.526??? B/op RecordSerializationBench.deserializeRecords:?gc.count???????????????????????? avgt?? 10???? 13.000????????????? counts RecordSerializationBench.deserializeRecords:?gc.time????????????????????????? avgt?? 10???? 17.000????????????????? ms ...so here the speed is comparable and it even produces less garbage. I created an issue [3]. So WDYT? Since this is still a preview feature in JDK15, is it possible to squeeze it into JDK15? Regards, Peter [1] http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/RecordSerializationBench.java [2] http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.01/ [3] https://bugs.openjdk.java.net/browse/JDK-8247532 From info at j-kuhn.de Sun Jun 14 17:03:00 2020 From: info at j-kuhn.de (Johannes Kuhn) Date: Sun, 14 Jun 2020 19:03:00 +0200 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> Message-ID: On 14-Jun-20 18:28, Peter Levart wrote: > Hi, > > > I noticed that deserializing records (new preview Java feature in > JDK14 and JDK15) is slow compared to equivalent classical classes. I > created a JMH benchmark [1] to se how it compares (ran it on JDK14): > > > Benchmark Mode? Cnt?????? Score??????? Error?? Units > RecordSerializationBench.deserializeClasses avgt?? 10????? 31.911 > ?????? 0.218?? us/op > RecordSerializationBench.deserializeClasses:?gc.alloc.rate avgt?? > 10???? 815.106 ?????? 5.545? MB/sec > RecordSerializationBench.deserializeClasses:?gc.alloc.rate.norm avgt?? > 10?? 40921.903 ?????? 0.615??? B/op > RecordSerializationBench.deserializeClasses:?gc.churn.G1_Eden_Space > avgt?? 10???? 839.522 ???? 191.195? MB/sec > RecordSerializationBench.deserializeClasses:?gc.churn.G1_Eden_Space.norm > avgt?? 10?? 42153.661 ??? 9682.799??? B/op > RecordSerializationBench.deserializeClasses:?gc.churn.G1_Survivor_Space > avgt?? 10?????? 0.117 ?????? 0.492? MB/sec > RecordSerializationBench.deserializeClasses:?gc.churn.G1_Survivor_Space.norm > avgt?? 10?????? 5.835 ????? 24.447??? B/op > RecordSerializationBench.deserializeClasses:?gc.count avgt?? 10????? > 21.000?????????????? counts > RecordSerializationBench.deserializeClasses:?gc.time avgt?? 10????? > 17.000?????????????????? ms > RecordSerializationBench.deserializeRecords avgt?? 10???? 531.333 > ?????? 3.094?? us/op > RecordSerializationBench.deserializeRecords:?gc.alloc.rate avgt?? > 10???? 346.511 ?????? 1.997? MB/sec > RecordSerializationBench.deserializeRecords:?gc.alloc.rate.norm avgt?? > 10? 289637.193 ?????? 6.894??? B/op > RecordSerializationBench.deserializeRecords:?gc.churn.G1_Eden_Space > avgt?? 10???? 359.773 ???? 191.116? MB/sec > RecordSerializationBench.deserializeRecords:?gc.churn.G1_Eden_Space.norm > avgt?? 10? 300657.838 ? 159724.346??? B/op > RecordSerializationBench.deserializeRecords:?gc.churn.G1_Survivor_Space > avgt?? 10?????? 0.007 ?????? 0.012? MB/sec > RecordSerializationBench.deserializeRecords:?gc.churn.G1_Survivor_Space.norm > avgt?? 10?????? 6.020 ?????? 9.910??? B/op > RecordSerializationBench.deserializeRecords:?gc.count avgt?? 10?????? > 9.000?????????????? counts > RecordSerializationBench.deserializeRecords:?gc.time avgt?? 10????? > 14.000?????????????????? ms > > > ...not only it is it about 16x slower, it also produces 7x garbage. I > checked the code and it is not very optimal. It matches the record > component names with object stream fields in O(n^2) way. It uses > method handles but binds arguments of canonical constructor each time > an instance of record is constructed. So I tried to optimize it [2] > and with that patch on top of JDK16 the benchmark produces the > following results: > > > Benchmark Mode? Cnt????? Score?????? Error?? Units > RecordSerializationBench.deserializeClasses avgt?? 10???? 31.049 ????? > 0.235?? us/op > RecordSerializationBench.deserializeClasses:?gc.alloc.rate avgt?? > 10??? 837.614 ????? 6.326? MB/sec > RecordSerializationBench.deserializeClasses:?gc.alloc.rate.norm avgt?? > 10? 40921.931 ????? 0.666??? B/op > RecordSerializationBench.deserializeClasses:?gc.churn.G1_Eden_Space > avgt?? 10??? 867.743 ??? 251.373? MB/sec > RecordSerializationBench.deserializeClasses:?gc.churn.G1_Eden_Space.norm > avgt?? 10? 42405.532 ? 12403.301??? B/op > RecordSerializationBench.deserializeClasses:?gc.churn.G1_Survivor_Space > avgt?? 10????? 0.126 ????? 0.478? MB/sec > RecordSerializationBench.deserializeClasses:?gc.churn.G1_Survivor_Space.norm > avgt?? 10????? 6.113 ???? 23.268??? B/op > RecordSerializationBench.deserializeClasses:?gc.count avgt?? 10???? > 22.000????????????? counts > RecordSerializationBench.deserializeClasses:?gc.time avgt?? 10???? > 19.000????????????????? ms > RecordSerializationBench.deserializeRecords avgt?? 10???? 33.588 ????? > 0.394?? us/op > RecordSerializationBench.deserializeRecords:?gc.alloc.rate avgt?? > 10??? 500.033 ????? 5.871? MB/sec > RecordSerializationBench.deserializeRecords:?gc.alloc.rate.norm avgt?? > 10? 26425.293 ????? 0.759??? B/op > RecordSerializationBench.deserializeRecords:?gc.churn.G1_Eden_Space > avgt?? 10??? 512.772 ??? 288.112? MB/sec > RecordSerializationBench.deserializeRecords:?gc.churn.G1_Eden_Space.norm > avgt?? 10? 27090.499 ? 15175.280??? B/op > RecordSerializationBench.deserializeRecords:?gc.churn.G1_Survivor_Space > avgt?? 10????? 0.134 ????? 0.496? MB/sec > RecordSerializationBench.deserializeRecords:?gc.churn.G1_Survivor_Space.norm > avgt?? 10????? 7.128 ???? 26.526??? B/op > RecordSerializationBench.deserializeRecords:?gc.count avgt?? 10???? > 13.000????????????? counts > RecordSerializationBench.deserializeRecords:?gc.time avgt?? 10???? > 17.000????????????????? ms > > > ...so here the speed is comparable and it even produces less garbage. > > > I created an issue [3]. > > So WDYT? Since this is still a preview feature in JDK15, is it > possible to squeeze it into JDK15? > > Regards, Peter > > > [1] > http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/RecordSerializationBench.java > > [2] > http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.01/ > > [3] https://bugs.openjdk.java.net/browse/JDK-8247532 > > > Small suggestion: RecordSupport.defaultValueExtractorFor could be written as: ??? return MethodHandles.empty(MethodType.methodType(pClass, byte[].class, Object[].class)); It could then be inlined. - Johannes From claes.redestad at oracle.com Sun Jun 14 18:15:28 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Sun, 14 Jun 2020 20:15:28 +0200 Subject: Optimize StringConcatHelper::simpleConcat for empty strings In-Reply-To: References: Message-ID: <2806eaeb-57ef-fccf-5b57-599487072038@oracle.com> Hi Tagir, seems reasonable if there's evidence this is likely to happen in practice. Cache locality might suffer a bit down the line since String and byte[] becomes less likely to reside in the same slice of heap memory. You could simplify as "return new String(s{2,1});". /Claes On 2020-06-13 07:08, Tagir Valeev wrote: > Hello! > > It's quite possible that when we concatenate two strings, one of them > appears to be empty. We cannot simply return another string in this > case, as JLS 15.18.1 explicitly says (for unknown to me reason) about > the result of the string concatenation expression that 'The String > object is newly created'. However, it's still possible to reuse the > internal array of another string to reduce allocations: > > --- src/java.base/share/classes/java/lang/StringConcatHelper.java > (revision 58998:04e3d254c76be87788a40cbd66d013140ea951d8) > +++ src/java.base/share/classes/java/lang/StringConcatHelper.java > (revision 58998+:04e3d254c76b+) > @@ -420,6 +420,12 @@ > static String simpleConcat(Object first, Object second) { > String s1 = stringOf(first); > String s2 = stringOf(second); > + if (s1.isEmpty()) { > + return new String(s2.value(), s2.coder()); > + } > + if (s2.isEmpty()) { > + return new String(s1.value(), s1.coder()); > + } > // start "mixing" in length and coder or arguments, order is not > // important > long indexCoder = mix(initialCoder(), s2); > > Very simple benchmark like this validates that the concatenation > became faster if one of the strings is empty: > > @Param({"", "longlonglongline"}) > String data; > > @Param({"", "longlonglongline"}) > String data2; > > @Benchmark > public String plus() { > return data + data2; > } > > Without patch I observe on VM 15-ea+20-899: > > Benchmark (data) (data2) Mode Cnt Score Error Units > plus avgt 30 15,335 ? 0,186 ns/op > plus longlonglongline avgt 30 19,867 ? 0,109 ns/op > plus longlonglongline avgt 30 20,283 ? 0,230 ns/op > plus longlonglongline longlonglongline avgt 30 26,047 ? 0,230 ns/op > > With patch: > Benchmark (data) (data2) Mode Cnt Score Error Units > plus avgt 30 6,668 ? 0,055 ns/op > plus longlonglongline avgt 30 6,708 ? 0,114 ns/op > plus longlonglongline avgt 30 7,003 ? 0,064 ns/op > plus longlonglongline longlonglongline avgt 30 25,126 ? 0,392 ns/op > > There could be an added cost of up to two branches for the normal case > (I believe, if one of the strings is constant, then decent JIT can > eliminate one of the branches). However, I believe, the benefit could > outweigh it, as empty strings are not that uncommon and in this case, > we reduce O(N) time and memory consumption to O(1). > > What do you think? Is this a reasonable thing to do? I can file an > issue and submit a proper webrev if it looks like a useful patch. > > With best regards, > Tagir Valeev. > From peter.levart at gmail.com Sun Jun 14 21:04:03 2020 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 14 Jun 2020 23:04:03 +0200 Subject: Optimize StringConcatHelper::simpleConcat for empty strings In-Reply-To: References: Message-ID: <8495a582-5c76-e2e9-7b0c-b312b128302b@gmail.com> Hi Tagir, I think that there might be cases where one of the arguments of concatenation is constant empty String. I found myself sometimes doing "" + someNonStringValue (instead of more readable String.valueOf(someNonStringValue)) simply because of laziness. So does this optimization help in that case as much as with non-constant "" + "longlonglongline" ? Also, for cases where one argument evaluates to empty string and the other is not a String, I think you could simply return the seconds string (not creating another instance with reused value/coder): if (s1.isEmpty()) { ? return s2 == second ? new String(s2.value(), s2.coder()) : s2; } Regards, Peter On 6/13/20 7:08 AM, Tagir Valeev wrote: > Hello! > > It's quite possible that when we concatenate two strings, one of them > appears to be empty. We cannot simply return another string in this > case, as JLS 15.18.1 explicitly says (for unknown to me reason) about > the result of the string concatenation expression that 'The String > object is newly created'. However, it's still possible to reuse the > internal array of another string to reduce allocations: > > --- src/java.base/share/classes/java/lang/StringConcatHelper.java > (revision 58998:04e3d254c76be87788a40cbd66d013140ea951d8) > +++ src/java.base/share/classes/java/lang/StringConcatHelper.java > (revision 58998+:04e3d254c76b+) > @@ -420,6 +420,12 @@ > static String simpleConcat(Object first, Object second) { > String s1 = stringOf(first); > String s2 = stringOf(second); > + if (s1.isEmpty()) { > + return new String(s2.value(), s2.coder()); > + } > + if (s2.isEmpty()) { > + return new String(s1.value(), s1.coder()); > + } > // start "mixing" in length and coder or arguments, order is not > // important > long indexCoder = mix(initialCoder(), s2); > > Very simple benchmark like this validates that the concatenation > became faster if one of the strings is empty: > > @Param({"", "longlonglongline"}) > String data; > > @Param({"", "longlonglongline"}) > String data2; > > @Benchmark > public String plus() { > return data + data2; > } > > Without patch I observe on VM 15-ea+20-899: > > Benchmark (data) (data2) Mode Cnt Score Error Units > plus avgt 30 15,335 ? 0,186 ns/op > plus longlonglongline avgt 30 19,867 ? 0,109 ns/op > plus longlonglongline avgt 30 20,283 ? 0,230 ns/op > plus longlonglongline longlonglongline avgt 30 26,047 ? 0,230 ns/op > > With patch: > Benchmark (data) (data2) Mode Cnt Score Error Units > plus avgt 30 6,668 ? 0,055 ns/op > plus longlonglongline avgt 30 6,708 ? 0,114 ns/op > plus longlonglongline avgt 30 7,003 ? 0,064 ns/op > plus longlonglongline longlonglongline avgt 30 25,126 ? 0,392 ns/op > > There could be an added cost of up to two branches for the normal case > (I believe, if one of the strings is constant, then decent JIT can > eliminate one of the branches). However, I believe, the benefit could > outweigh it, as empty strings are not that uncommon and in this case, > we reduce O(N) time and memory consumption to O(1). > > What do you think? Is this a reasonable thing to do? I can file an > issue and submit a proper webrev if it looks like a useful patch. > > With best regards, > Tagir Valeev. From peter.levart at gmail.com Sun Jun 14 21:17:12 2020 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 14 Jun 2020 23:17:12 +0200 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> Message-ID: <75a00647-d586-be3e-91f9-619a0242f2c3@gmail.com> On 6/14/20 7:03 PM, Johannes Kuhn wrote: > Small suggestion: RecordSupport.defaultValueExtractorFor could be > written as: > > ??? return MethodHandles.empty(MethodType.methodType(pClass, > byte[].class, Object[].class)); > > It could then be inlined. > > - Johannes Thanks, Johannes. There is also a MethodHandles.zero(Class type) method as I found out. Regards, Peter From peter.levart at gmail.com Sun Jun 14 21:34:35 2020 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 14 Jun 2020 23:34:35 +0200 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: <75a00647-d586-be3e-91f9-619a0242f2c3@gmail.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <75a00647-d586-be3e-91f9-619a0242f2c3@gmail.com> Message-ID: <02f0b192-c920-0c3f-a6c8-605eedbe59b4@gmail.com> So, here's new webrev incorporating Johannes' suggestion: http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.02/ Peter On 6/14/20 11:17 PM, Peter Levart wrote: > > On 6/14/20 7:03 PM, Johannes Kuhn wrote: >> Small suggestion: RecordSupport.defaultValueExtractorFor could be >> written as: >> >> ??? return MethodHandles.empty(MethodType.methodType(pClass, >> byte[].class, Object[].class)); >> >> It could then be inlined. >> >> - Johannes > > > Thanks, Johannes. There is also a MethodHandles.zero(Class type) > method as I found out. > > > Regards, Peter > > From claes.redestad at oracle.com Sun Jun 14 23:36:45 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 15 Jun 2020 01:36:45 +0200 Subject: Optimize StringConcatHelper::simpleConcat for empty strings In-Reply-To: <8495a582-5c76-e2e9-7b0c-b312b128302b@gmail.com> References: <8495a582-5c76-e2e9-7b0c-b312b128302b@gmail.com> Message-ID: Hi Peter, Tagir, On 2020-06-14 23:04, Peter Levart wrote: > Hi Tagir, > > > I think that there might be cases where one of the arguments of > concatenation is constant empty String. I found myself sometimes doing > "" + someNonStringValue (instead of more readable > String.valueOf(someNonStringValue)) simply because of laziness. So does > this optimization help in that case as much as with non-constant "" + > "longlonglongline" ? perhaps we should consider adding unary concat specializations: http://cr.openjdk.java.net/~redestad/scratch/unary_concat.00/ For primitive arguments we just return a handle to String.valueOf, which helps with bootstrap overheads (a Hello World that does "" + args.length drops from 66ms to 51ms on my machine). "" + stringArg can't be reduced to the objectStringifier, since we need that new String wrapper. Results on the added concatEmpty micros - before: Benchmark (intValue) Mode Cnt Score Error Units StringConcat.concatEmptyInt 4711 avgt 5 15.461 ? 1.266 ns/op StringConcat.concatEmptyString 4711 avgt 5 8.173 ? 0.515 ns/op - after: Benchmark (intValue) Mode Cnt Score Error Units StringConcat.concatEmptyInt 4711 avgt 5 15.470 ? 1.235 ns/op StringConcat.concatEmptyString 4711 avgt 5 18.420 ? 1.572 ns/op This doesn't help the case where the arguments are both arguments andone happen to be an empty string. Tagir's patch would still make sense there. /Claes > > > Also, for cases where one argument evaluates to empty string and the > other is not a String, I think you could simply return the seconds > string (not creating another instance with reused value/coder): > > > if (s1.isEmpty()) { > > ? return s2 == second ? new String(s2.value(), s2.coder()) : s2; > > } > > > Regards, Peter > > > On 6/13/20 7:08 AM, Tagir Valeev wrote: >> Hello! >> >> It's quite possible that when we concatenate two strings, one of them >> appears to be empty. We cannot simply return another string in this >> case, as JLS 15.18.1 explicitly says (for unknown to me reason) about >> the result of the string concatenation expression that 'The String >> object is newly created'. However, it's still possible to reuse the >> internal array of another string to reduce allocations: >> >> --- src/java.base/share/classes/java/lang/StringConcatHelper.java >> (revision 58998:04e3d254c76be87788a40cbd66d013140ea951d8) >> +++ src/java.base/share/classes/java/lang/StringConcatHelper.java >> (revision 58998+:04e3d254c76b+) >> @@ -420,6 +420,12 @@ >> ????? static String simpleConcat(Object first, Object second) { >> ????????? String s1 = stringOf(first); >> ????????? String s2 = stringOf(second); >> +??????? if (s1.isEmpty()) { >> +??????????? return new String(s2.value(), s2.coder()); >> +??????? } >> +??????? if (s2.isEmpty()) { >> +??????????? return new String(s1.value(), s1.coder()); >> +??????? } >> ????????? // start "mixing" in length and coder or arguments, order is >> not >> ????????? // important >> ????????? long indexCoder = mix(initialCoder(), s2); >> >> Very simple benchmark like this validates that the concatenation >> became faster if one of the strings is empty: >> >> ?? @Param({"", "longlonglongline"}) >> ?? String data; >> >> ?? @Param({"", "longlonglongline"}) >> ?? String data2; >> >> ?? @Benchmark >> ?? public String plus() { >> ???? return data + data2; >> ?? } >> >> Without patch I observe on VM 15-ea+20-899: >> >> Benchmark?????? (data)?????????? (data2)? Mode? Cnt?? Score?? Error >> Units >> plus????????????????????????????????????? avgt?? 30? 15,335 ? 0,186 >> ns/op >> plus??????????????????? longlonglongline? avgt?? 30? 19,867 ? 0,109 >> ns/op >> plus? longlonglongline??????????????????? avgt?? 30? 20,283 ? 0,230 >> ns/op >> plus? longlonglongline? longlonglongline? avgt?? 30? 26,047 ? 0,230 >> ns/op >> >> With patch: >> Benchmark?????? (data)?????????? (data2)? Mode? Cnt?? Score?? Error >> Units >> plus????????????????????????????????????? avgt?? 30?? 6,668 ? 0,055 >> ns/op >> plus??????????????????? longlonglongline? avgt?? 30?? 6,708 ? 0,114 >> ns/op >> plus? longlonglongline??????????????????? avgt?? 30?? 7,003 ? 0,064 >> ns/op >> plus? longlonglongline? longlonglongline? avgt?? 30? 25,126 ? 0,392 >> ns/op >> >> There could be an added cost of up to two branches for the normal case >> (I believe, if one of the strings is constant, then decent JIT can >> eliminate one of the branches). However, I believe, the benefit could >> outweigh it, as empty strings are not that uncommon and in this case, >> we reduce O(N) time and memory consumption to O(1). >> >> What do you think? Is this a reasonable thing to do? I can file an >> issue and submit a proper webrev if it looks like a useful patch. >> >> With best regards, >> Tagir Valeev. From amaembo at gmail.com Mon Jun 15 03:57:40 2020 From: amaembo at gmail.com (Tagir Valeev) Date: Mon, 15 Jun 2020 10:57:40 +0700 Subject: Optimize StringConcatHelper::simpleConcat for empty strings In-Reply-To: <8495a582-5c76-e2e9-7b0c-b312b128302b@gmail.com> References: <8495a582-5c76-e2e9-7b0c-b312b128302b@gmail.com> Message-ID: Hello, Peter and Claes! > Cache locality might suffer a bit down the line since > String and byte[] becomes less likely to reside in the same slice > of heap memory. There's a JEP 192 String Deduplication in G1 that does similar optimization inside the GC (currently it's off by default). The 'Risk and assumptions' section reads as: > Normally a String object and its corresponding character array will be placed next to each other in memory, leading to good cache locality. After deduplicating a String object its character array will be further away, which might introduce a slight performance overhead. Initial measurements have, however, shown that this does not seem to be a problem. In fact, after deduplication the total amount of accessed memory is reduced which tends to result in improved cache hit rates. So I believe it would not be a big problem in our case either. > You could simplify as "return new String(s{2,1});". Thanks! Totally forgot about that String constructor. Peter, > Also, for cases where one argument evaluates to empty string and the > other is not a String, I think you could simply return the seconds > string (not creating another instance with reused value/coder)\ I believe, this may still violate the spec. The resulting object must be a 'newly created' while the custom toString() may return cached or constant string. So we still need to copy it. This also applies to the Claes' unaryConcat: + @ForceInline + static String unaryConcat(Object first) { + if (first instanceof String) { + // Must produce a new String + return new String((String)first); + } else { + return stringOf(first); + } + } It should be just // Must produce a new String return new String(stringOf(first)); So should I file a separate JBS issue for my optimization or both cases (constant empty string and non-constant empty string) could be handled under a single issue? With best regards, Tagir Valeev. On Mon, Jun 15, 2020 at 4:04 AM Peter Levart wrote: > > Hi Tagir, > > > I think that there might be cases where one of the arguments of > concatenation is constant empty String. I found myself sometimes doing > "" + someNonStringValue (instead of more readable > String.valueOf(someNonStringValue)) simply because of laziness. So does > this optimization help in that case as much as with non-constant "" + > "longlonglongline" ? > > > Also, for cases where one argument evaluates to empty string and the > other is not a String, I think you could simply return the seconds > string (not creating another instance with reused value/coder): > > > if (s1.isEmpty()) { > > return s2 == second ? new String(s2.value(), s2.coder()) : s2; > > } > > > Regards, Peter > > > On 6/13/20 7:08 AM, Tagir Valeev wrote: > > Hello! > > > > It's quite possible that when we concatenate two strings, one of them > > appears to be empty. We cannot simply return another string in this > > case, as JLS 15.18.1 explicitly says (for unknown to me reason) about > > the result of the string concatenation expression that 'The String > > object is newly created'. However, it's still possible to reuse the > > internal array of another string to reduce allocations: > > > > --- src/java.base/share/classes/java/lang/StringConcatHelper.java > > (revision 58998:04e3d254c76be87788a40cbd66d013140ea951d8) > > +++ src/java.base/share/classes/java/lang/StringConcatHelper.java > > (revision 58998+:04e3d254c76b+) > > @@ -420,6 +420,12 @@ > > static String simpleConcat(Object first, Object second) { > > String s1 = stringOf(first); > > String s2 = stringOf(second); > > + if (s1.isEmpty()) { > > + return new String(s2.value(), s2.coder()); > > + } > > + if (s2.isEmpty()) { > > + return new String(s1.value(), s1.coder()); > > + } > > // start "mixing" in length and coder or arguments, order is not > > // important > > long indexCoder = mix(initialCoder(), s2); > > > > Very simple benchmark like this validates that the concatenation > > became faster if one of the strings is empty: > > > > @Param({"", "longlonglongline"}) > > String data; > > > > @Param({"", "longlonglongline"}) > > String data2; > > > > @Benchmark > > public String plus() { > > return data + data2; > > } > > > > Without patch I observe on VM 15-ea+20-899: > > > > Benchmark (data) (data2) Mode Cnt Score Error Units > > plus avgt 30 15,335 ? 0,186 ns/op > > plus longlonglongline avgt 30 19,867 ? 0,109 ns/op > > plus longlonglongline avgt 30 20,283 ? 0,230 ns/op > > plus longlonglongline longlonglongline avgt 30 26,047 ? 0,230 ns/op > > > > With patch: > > Benchmark (data) (data2) Mode Cnt Score Error Units > > plus avgt 30 6,668 ? 0,055 ns/op > > plus longlonglongline avgt 30 6,708 ? 0,114 ns/op > > plus longlonglongline avgt 30 7,003 ? 0,064 ns/op > > plus longlonglongline longlonglongline avgt 30 25,126 ? 0,392 ns/op > > > > There could be an added cost of up to two branches for the normal case > > (I believe, if one of the strings is constant, then decent JIT can > > eliminate one of the branches). However, I believe, the benefit could > > outweigh it, as empty strings are not that uncommon and in this case, > > we reduce O(N) time and memory consumption to O(1). > > > > What do you think? Is this a reasonable thing to do? I can file an > > issue and submit a proper webrev if it looks like a useful patch. > > > > With best regards, > > Tagir Valeev. From peter.kessler at os.amperecomputing.com Mon Jun 15 04:03:34 2020 From: peter.kessler at os.amperecomputing.com (Peter Kessler (Open Source)) Date: Mon, 15 Jun 2020 04:03:34 +0000 Subject: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) In-Reply-To: <480848bd-05db-2555-5fe7-686750e44dc9@oracle.com> References: <814DA850-76A9-4526-A3D9-141CD516C9AE@amperemail.onmicrosoft.com> <50955EDB-CCC2-4334-A4AF-2EB422E9CAA9@amperemail.onmicrosoft.com> <3B38456A-6BD0-4B95-9F9C-E73F7FE3A78F@amperemail.onmicrosoft.com> <480848bd-05db-2555-5fe7-686750e44dc9@oracle.com> Message-ID: <6A8C82B8-B1C8-4A58-A53B-53194693A36F@amperemail.onmicrosoft.com> Alan, Reflection methods are ignored by the stack walk in JVM_LatestUserDefinedLoader. That was why I marked jdk.internal.reflect.DelegatingClassLoader as not a user-defined class. If the optimization is disabled, then so be it. I am using SPECjbb2015 as an example sophisticated Java application. I think there are plenty of applications that use ObjectInputStream that do not define their own ClassLoaders. The proposed patch would help all of them run more efficiently. ... peter ?-----Original Message----- From: Alan Bateman Date: Saturday, June 13, 2020 at 9:06 AM To: "Peter Kessler (Open Source)" , Roger Riggs , "core-libs-dev at openjdk.java.net" Subject: Re: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) On 13/06/2020 00:19, Peter Kessler (Open Source) wrote: > : > > I settled on a system-wide one-bit cache: There have been no > user-defined ClassLoaders constructed, so I can not find one on any > particular stack walk. It may not be the best possible solution, but > it addresses the issue of useless stack walks in the common case where > there are no user-defined ClassLoaders. > Does this help anything other than SPECjbb2015? Real world code will use core reflection and other areas of the JDK that create class loaders at run-time so I assume this optimization will be disabled very quickly. I think it would be better to look for other solutions. -Alan From vipinmv1 at in.ibm.com Mon Jun 15 06:22:28 2020 From: vipinmv1 at in.ibm.com (Vipin Mv1) Date: Mon, 15 Jun 2020 06:22:28 +0000 Subject: RFR 6470126 java.util.jar.Attributes#containsKey fails with Strings In-Reply-To: References: Message-ID: Hi, I have addressed the review comments and the patch has been uploaded here: http://cr.openjdk.java.net/~vtewari/6470126/webrev/index.html Please let me know your suggestions. Thanks & Regards Vipin MV -----Vipin Mv1/India/IBM wrote: ----- To: core-libs-dev at openjdk.java.net From: Vipin Mv1/India/IBM Date: 05/11/2020 05:00PM Subject: RFR 6470126 java.util.jar.Attributes#containsKey fails with Strings Hi, Please review the fix for the following issue. https://bugs.openjdk.java.net/browse/JDK-6470126 diff -r 53568400fec3 src/java.base/share/classes/java/util/jar/Attributes.java --- a/src/java.base/share/classes/java/util/jar/Attributes.java Thu Mar 26 15:26:51 2020 +0000 +++ b/src/java.base/share/classes/java/util/jar/Attributes.java Mon May 11 15:00:01 2020 +0530 @@ -205,7 +205,10 @@ * @return true if this Map contains the specified attribute name */ public boolean containsKey(Object name) { - return map.containsKey(name); + if(String.class.isInstance(name)) + return map.containsKey(Name.of((String)name)); + else + return map.containsKey(name); } /** Thanks & Regards Vipin Menon From david.holmes at oracle.com Mon Jun 15 06:23:12 2020 From: david.holmes at oracle.com (David Holmes) Date: Mon, 15 Jun 2020 16:23:12 +1000 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: References: Message-ID: <39df1e40-f34a-ba3b-e2e2-f462a6442c42@oracle.com> Hi Igor, I found this all a bit hard to follow. It would have been clearer if broken up into distinct pieces. On 13/06/2020 2:10 pm, Igor Ignatyev wrote: > testing revealed that LingeredAppTest.java required some love, incremental webrev w/ the fixes for LingeredAppTest -- http://cr.openjdk.java.net/~iignatyev//8211977/webrev.0-1 > > -- Igor > >> On Jun 12, 2020, at 8:38 PM, Igor Ignatyev wrote: >> >> adding build-dev >> >>> On Jun 12, 2020, at 8:36 PM, Igor Ignatyev wrote: >>> >>> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ >>>> 796 lines changed: 200 ins; 588 del; 8 mod; >>> >>> Hi all, >>> >>> could you please review this small patch which puts all tests for testlibrary classes into one location under /test/lib-test? >>> >>> besides moving tests from test/jdk/lib/testlibrary and test/hotspot/jtreg/testlibrary_tests to test/lib-test the patch also This doesn't seem to move everything under test/hotspot/jtreg/testlibrary_tests/ e.g. ctw directory. You did not modify hotspot/jtreg/TEST.groups but it refers to: testlibrary_tests/TestMutuallyExclusivePlatformPredicates.java \ Plus it should probably refer to the new native_sanity tests somewhere. >>> - problem lists HexPrinterTest.java on windows due to JDK-8247521 >>> - introduces make targets to build native parts for the tests in test/lib-test (needed b/c one test has a native part) How was it built before ? >>> - adds randomness k/w to test/lib-test (as it's used by RandomGeneratorTest.java) >>> - makes Test class used by TestNativeProcessBuilder a static nested class of TestNativeProcessBuilder >>> - updates LingeredAppTest to use @build instead of @compile and adds necessary @library tag >>> - removes AssertsTest.java, OutputAnalyzerTest.java from test/hotspot/jtreg/testlibrary_tests as they are either identical or lesser that the same tests from test/jdk/lib/testlibrary/ >>> - merges test/hotspot/jtreg/testlibrary_tests/OutputAnalyzerTest.java and test/jdk/lib/testlibrary/OutputAnalyzerTest.java (effectively adds test cases for `firstMatch` to the superier copy from test/jdk/lib/testlibrary) >>> >>> webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ test/lib-test/TEST.ROOT The newly added copyright notice needs to have the correct initial copyright year based on when the file was first added to the repo. make/test/JtregNativeLibTest.gmk You used the wrong license header - makefiles don't use the classpath exception. Otherwise changes appear to be okay. Thanks, David ----- >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 >>> testing: test/lib-test on {windows,linux,macosx}-x64 >>> >>> Thanks, >>> -- Igor >>> >>> >> > From matthias.baesken at sap.com Mon Jun 15 07:03:33 2020 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 15 Jun 2020 07:03:33 +0000 Subject: RFR: 8247469: getSystemCpuLoad() returns -1 on linux when some offline cpus are present and cpusets.effective_cpus is not available In-Reply-To: References: Message-ID: Hi , thanks for the reviews ! Best regards, Matthias >Look good to me. > >Bob. From claes.redestad at oracle.com Mon Jun 15 08:10:10 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 15 Jun 2020 10:10:10 +0200 Subject: Optimize StringConcatHelper::simpleConcat for empty strings In-Reply-To: References: <8495a582-5c76-e2e9-7b0c-b312b128302b@gmail.com> Message-ID: <0a91ac4d-db4b-6341-153a-d3ecf77c0f13@oracle.com> Hi Tagir, On 2020-06-15 05:57, Tagir Valeev wrote: > Hello, Peter and Claes! > >> Cache locality might suffer a bit down the line since >> String and byte[] becomes less likely to reside in the same slice >> of heap memory. > > There's a JEP 192 String Deduplication in G1 that does similar > optimization inside the GC (currently it's off by default). > The 'Risk and assumptions' section reads as: > >> Normally a String object and its corresponding character array will be placed next to each other in memory, leading to good cache locality. After deduplicating a String object its character array will be further away, which might introduce a slight performance overhead. Initial measurements have, however, shown that this does not seem to be a problem. In fact, after deduplication the total amount of accessed memory is reduced which tends to result in improved cache hit rates. > > So I believe it would not be a big problem in our case either. The moral difference is that String deduplication is an opt-in feature, so users can decide if the trade-offs inherent is right for them. Here we are making that choice for users, albeit for a very small subset of all possible Strings. This saves a lot of cycles and memory up front, which I think weighs in favor here (many Strings are short-lived). > >> You could simplify as "return new String(s{2,1});". > > Thanks! Totally forgot about that String constructor. > > Peter, >> Also, for cases where one argument evaluates to empty string and the >> other is not a String, I think you could simply return the seconds >> string (not creating another instance with reused value/coder)\ > > I believe, this may still violate the spec. The resulting object must > be a 'newly created' while the custom toString() may return cached or > constant string. So we still need to copy it. This also applies to the > Claes' unaryConcat: > > + @ForceInline > + static String unaryConcat(Object first) { > + if (first instanceof String) { > + // Must produce a new String > + return new String((String)first); > + } else { > + return stringOf(first); > + } > + } > > It should be just > // Must produce a new String > return new String(stringOf(first)); True. > > So should I file a separate JBS issue for my optimization or both > cases (constant empty string and non-constant empty string) could be > handled under a single issue? Up to you. I can fold the patches into one or do them separately, with my unaryConcat as a follow-up. /Claes > > With best regards, > Tagir Valeev. > > On Mon, Jun 15, 2020 at 4:04 AM Peter Levart wrote: >> >> Hi Tagir, >> >> >> I think that there might be cases where one of the arguments of >> concatenation is constant empty String. I found myself sometimes doing >> "" + someNonStringValue (instead of more readable >> String.valueOf(someNonStringValue)) simply because of laziness. So does >> this optimization help in that case as much as with non-constant "" + >> "longlonglongline" ? >> >> >> Also, for cases where one argument evaluates to empty string and the >> other is not a String, I think you could simply return the seconds >> string (not creating another instance with reused value/coder): >> >> >> if (s1.isEmpty()) { >> >> return s2 == second ? new String(s2.value(), s2.coder()) : s2; >> >> } >> >> >> Regards, Peter >> >> >> On 6/13/20 7:08 AM, Tagir Valeev wrote: >>> Hello! >>> >>> It's quite possible that when we concatenate two strings, one of them >>> appears to be empty. We cannot simply return another string in this >>> case, as JLS 15.18.1 explicitly says (for unknown to me reason) about >>> the result of the string concatenation expression that 'The String >>> object is newly created'. However, it's still possible to reuse the >>> internal array of another string to reduce allocations: >>> >>> --- src/java.base/share/classes/java/lang/StringConcatHelper.java >>> (revision 58998:04e3d254c76be87788a40cbd66d013140ea951d8) >>> +++ src/java.base/share/classes/java/lang/StringConcatHelper.java >>> (revision 58998+:04e3d254c76b+) >>> @@ -420,6 +420,12 @@ >>> static String simpleConcat(Object first, Object second) { >>> String s1 = stringOf(first); >>> String s2 = stringOf(second); >>> + if (s1.isEmpty()) { >>> + return new String(s2.value(), s2.coder()); >>> + } >>> + if (s2.isEmpty()) { >>> + return new String(s1.value(), s1.coder()); >>> + } >>> // start "mixing" in length and coder or arguments, order is not >>> // important >>> long indexCoder = mix(initialCoder(), s2); >>> >>> Very simple benchmark like this validates that the concatenation >>> became faster if one of the strings is empty: >>> >>> @Param({"", "longlonglongline"}) >>> String data; >>> >>> @Param({"", "longlonglongline"}) >>> String data2; >>> >>> @Benchmark >>> public String plus() { >>> return data + data2; >>> } >>> >>> Without patch I observe on VM 15-ea+20-899: >>> >>> Benchmark (data) (data2) Mode Cnt Score Error Units >>> plus avgt 30 15,335 ? 0,186 ns/op >>> plus longlonglongline avgt 30 19,867 ? 0,109 ns/op >>> plus longlonglongline avgt 30 20,283 ? 0,230 ns/op >>> plus longlonglongline longlonglongline avgt 30 26,047 ? 0,230 ns/op >>> >>> With patch: >>> Benchmark (data) (data2) Mode Cnt Score Error Units >>> plus avgt 30 6,668 ? 0,055 ns/op >>> plus longlonglongline avgt 30 6,708 ? 0,114 ns/op >>> plus longlonglongline avgt 30 7,003 ? 0,064 ns/op >>> plus longlonglongline longlonglongline avgt 30 25,126 ? 0,392 ns/op >>> >>> There could be an added cost of up to two branches for the normal case >>> (I believe, if one of the strings is constant, then decent JIT can >>> eliminate one of the branches). However, I believe, the benefit could >>> outweigh it, as empty strings are not that uncommon and in this case, >>> we reduce O(N) time and memory consumption to O(1). >>> >>> What do you think? Is this a reasonable thing to do? I can file an >>> issue and submit a proper webrev if it looks like a useful patch. >>> >>> With best regards, >>> Tagir Valeev. From chris.hegarty at oracle.com Mon Jun 15 08:37:14 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 15 Jun 2020 09:37:14 +0100 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: <02f0b192-c920-0c3f-a6c8-605eedbe59b4@gmail.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <75a00647-d586-be3e-91f9-619a0242f2c3@gmail.com> <02f0b192-c920-0c3f-a6c8-605eedbe59b4@gmail.com> Message-ID: <5AFF0951-847C-459A-BC3D-5FDE57462664@oracle.com> Hi Peter, Thank you for doing this, it is really appreciated. I?m going to take this patch and run it in my local environment, after which I?ll post a reply here. > So WDYT? Since this is still a preview feature in JDK15, is it possible to squeeze it into JDK15? I think that such a change is a reasonable candidate for consideration in RDP 1. I see that the issue in JIRA is currently an Enhancement. Is it really an enhancement? If so, then it can follow the LERP [1]. Otherwise, if a regular bug between P1-P3 (inclusive), it can be integrated when reviewed and ready. -Chris. [1] https://openjdk.java.net/jeps/3#Late-Enhancement-Request-Process > On 14 Jun 2020, at 22:34, Peter Levart wrote: > > So, here's new webrev incorporating Johannes' suggestion: > > > http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.02/ > From peter.levart at gmail.com Mon Jun 15 09:06:32 2020 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 15 Jun 2020 11:06:32 +0200 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: <5AFF0951-847C-459A-BC3D-5FDE57462664@oracle.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <75a00647-d586-be3e-91f9-619a0242f2c3@gmail.com> <02f0b192-c920-0c3f-a6c8-605eedbe59b4@gmail.com> <5AFF0951-847C-459A-BC3D-5FDE57462664@oracle.com> Message-ID: <73a8fb9f-bd71-8587-30d0-8e7d8979e5fd@gmail.com> Hi Chris, Maybe I stamped "Enchancement" just because it does not represent a logical bug, but 16x regression compared to the same feature with classes could be characterized as performance "bug", right? Peter On 6/15/20 10:37 AM, Chris Hegarty wrote: > Hi Peter, > > Thank you for doing this, it is really appreciated. I?m going to take > this patch and run it in my local environment, after which I?ll post a > reply here. > > > So WDYT? Since this is still a preview feature in JDK15, is it > possible to squeeze it into JDK15? > > I?think that?such a change is a reasonable candidate for?consideration > in RDP 1. I see that the issue in JIRA is currently an Enhancement. Is > it really an enhancement? If so, then it can follow the LERP [1]. > Otherwise, if a regular bug between P1-P3 (inclusive), it can be > integrated when reviewed and ready. > > -Chris. > > [1] https://openjdk.java.net/jeps/3#Late-Enhancement-Request-Process > >> On 14 Jun 2020, at 22:34, Peter Levart > > wrote: >> >> So, here's new webrev incorporating Johannes' suggestion: >> >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.02/ >> > From kim.barrett at oracle.com Mon Jun 15 11:41:02 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 15 Jun 2020 07:41:02 -0400 Subject: [OpenJDK 2D-Dev] RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: <5EE5AB71.20505@oracle.com> References: <5EDE9A75.6020504@oracle.com> <5EE5AB71.20505@oracle.com> Message-ID: > On Jun 14, 2020, at 12:45 AM, Philip Race wrote: > > Kim, > > > Until it says in "the JDK" and not "in HotSpot" you have not addressed my main point. > Please rename the JEP. I think this JEP is primarily about updating the HotSpot-specific subset of C++ and usage guidance to include some C++14 features. HotSpot usage is already different in some ways from C++ code in other parts of the JDK. For example, HotSpot code doesn't use global operator new; that isn't likely to change, and non-HotSpot code doesn't have access to the alternatives defined and used but not exported by HotSpot. As a necessary prerequisite for that, the JDK build system is being updated to enable the use of C++14 features. It's probably possible to limit the scope of the language change to HotSpot, but that doesn't seem useful. It also probably wasn't possible while Solaris was still in the mix. It is expected that groups responsible for other parts of the JDK will also want to take advantage of that build change, but I think it's up to those other groups to decide on an adoption strategy. I have no reason to think the choices being suggested here as appropriate for HotSpot are appropriate JDK-wide. I can't say whether a JEP is generally needed; maybe it's simply "C++14 - yes" in some areas. Also, some of the reasons for this being a JEP no longer apply. For example, Solaris was going to require at least a major ABI change, and at the time the JEP was created there was no path to C++14 for the AIX/PPC port; both of those needed visibility and discussion that seemed best provided via the JEP process. > -phil. > > On 6/13/20, 8:48 PM, Kim Barrett wrote: >>> On Jun 10, 2020, at 2:26 AM, Kim Barrett wrote: >>> >>>> On Jun 8, 2020, at 4:07 PM, Philip Race wrote: >>>> BTW the JEP (https://bugs.openjdk.java.net/browse/JDK-8208089) still says >>>>> For Solaris: Add the -std=c++14 compiler option. ..... >>>> So I am sure there is still some room to update the JEP :-) >>> Yeah, I have some updates to make. I also need to do a bit of work on the feature lists. >> I think the JEP updates are done now. From chris.hegarty at oracle.com Mon Jun 15 12:47:38 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 15 Jun 2020 13:47:38 +0100 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: <73a8fb9f-bd71-8587-30d0-8e7d8979e5fd@gmail.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <75a00647-d586-be3e-91f9-619a0242f2c3@gmail.com> <02f0b192-c920-0c3f-a6c8-605eedbe59b4@gmail.com> <5AFF0951-847C-459A-BC3D-5FDE57462664@oracle.com> <73a8fb9f-bd71-8587-30d0-8e7d8979e5fd@gmail.com> Message-ID: <76057999-A748-4B51-A1C3-2D73FA758E74@oracle.com> > On 15 Jun 2020, at 10:06, Peter Levart wrote: > > Hi Chris, > > Maybe I stamped "Enchancement" just because it does not represent a logical bug, but 16x regression compared to the same feature with classes could be characterized as performance "bug", right? That would be the categorisation that I would use. -Chris. > > Peter > > On 6/15/20 10:37 AM, Chris Hegarty wrote: >> Hi Peter, >> >> Thank you for doing this, it is really appreciated. I?m going to take this patch and run it in my local environment, after which I?ll post a reply here. >> >> > So WDYT? Since this is still a preview feature in JDK15, is it possible to squeeze it into JDK15? >> >> I think that such a change is a reasonable candidate for consideration in RDP 1. I see that the issue in JIRA is currently an Enhancement. Is it really an enhancement? If so, then it can follow the LERP [1]. Otherwise, if a regular bug between P1-P3 (inclusive), it can be integrated when reviewed and ready. >> >> -Chris. >> >> [1] https://openjdk.java.net/jeps/3#Late-Enhancement-Request-Process >> >>> On 14 Jun 2020, at 22:34, Peter Levart > wrote: >>> >>> So, here's new webrev incorporating Johannes' suggestion: >>> >>> >>> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.02/ >>> >> > From erik.joelsson at oracle.com Mon Jun 15 13:16:11 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 15 Jun 2020 06:16:11 -0700 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: References: Message-ID: <8e9289a9-7dc7-cde3-cf87-e93580a82fc4@oracle.com> Hello Igor, In JtretNativeLibTest.gmk, lines 51-55 should probably be removed (or adjusted if linking to libjvm is actually needed). /Erik On 2020-06-12 21:10, Igor Ignatyev wrote: > testing revealed that LingeredAppTest.java required some love, incremental webrev w/ the fixes for LingeredAppTest -- http://cr.openjdk.java.net/~iignatyev//8211977/webrev.0-1 > > -- Igor > >> On Jun 12, 2020, at 8:38 PM, Igor Ignatyev wrote: >> >> adding build-dev >> >>> On Jun 12, 2020, at 8:36 PM, Igor Ignatyev wrote: >>> >>> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ >>>> 796 lines changed: 200 ins; 588 del; 8 mod; >>> Hi all, >>> >>> could you please review this small patch which puts all tests for testlibrary classes into one location under /test/lib-test? >>> >>> besides moving tests from test/jdk/lib/testlibrary and test/hotspot/jtreg/testlibrary_tests to test/lib-test the patch also >>> - problem lists HexPrinterTest.java on windows due to JDK-8247521 >>> - introduces make targets to build native parts for the tests in test/lib-test (needed b/c one test has a native part) >>> - adds randomness k/w to test/lib-test (as it's used by RandomGeneratorTest.java) >>> - makes Test class used by TestNativeProcessBuilder a static nested class of TestNativeProcessBuilder >>> - updates LingeredAppTest to use @build instead of @compile and adds necessary @library tag >>> - removes AssertsTest.java, OutputAnalyzerTest.java from test/hotspot/jtreg/testlibrary_tests as they are either identical or lesser that the same tests from test/jdk/lib/testlibrary/ >>> - merges test/hotspot/jtreg/testlibrary_tests/OutputAnalyzerTest.java and test/jdk/lib/testlibrary/OutputAnalyzerTest.java (effectively adds test cases for `firstMatch` to the superier copy from test/jdk/lib/testlibrary) >>> >>> webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 >>> testing: test/lib-test on {windows,linux,macosx}-x64 >>> >>> Thanks, >>> -- Igor >>> >>> From Roger.Riggs at oracle.com Mon Jun 15 14:09:08 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Mon, 15 Jun 2020 10:09:08 -0400 Subject: RFR 15 8247521: (test) jdk/test/lib/hexdump/HexPrinterTest.java fails on windows Message-ID: <7f796110-58e4-f013-dec2-a1b1796777da@oracle.com> Please review a test change of the test library HexPrinter. It should be comparing against the System.lineSeparator. diff --git a/test/lib-test/jdk/test/lib/hexdump/HexPrinterTest.java b/test/lib-test/jdk/test/lib/hexdump/HexPrinterTest.java --- a/test/lib-test/jdk/test/lib/hexdump/HexPrinterTest.java +++ b/test/lib-test/jdk/test/lib/hexdump/HexPrinterTest.java @@ -77,9 +77,9 @@ public class HexPrinterTest { ???? Object[][] builtinParams() { ???????? return new Object[][]{ ???????????????? {"minimal", "", "%02x", 16, "", 64, HexPrinter.Formatters.NONE, ""}, -??????????????? {"canonical", "%08x? ", "%02x ", 16, "|", 31, HexPrinter.Formatters.PRINTABLE, "|\n"}, -??????????????? {"simple", "%5d: ", "%02x ", 16, " // ", 64, HexPrinter.Formatters.ASCII, "\n"}, -??????????????? {"source", "??? ", "(byte)%3d, ", 8, " // ", 64, HexPrinter.Formatters.PRINTABLE, "\n"}, +??????????????? {"canonical", "%08x? ", "%02x ", 16, "|", 31, HexPrinter.Formatters.PRINTABLE, "|" + System.lineSeparator()}, +??????????????? {"simple", "%5d: ", "%02x ", 16, " // ", 64, HexPrinter.Formatters.ASCII,? System.lineSeparator()}, +??????????????? {"source", "??? ", "(byte)%3d, ", 8, " // ", 64, HexPrinter.Formatters.PRINTABLE,? System.lineSeparator()}, ???????? }; ???? } Webrev: ? http://cr.openjdk.java.net/~rriggs/webrev-hexprinter-8247521/ Issue: ?https://bugs.openjdk.java.net/browse/JDK-8247521 Thanks, Roger From igor.ignatyev at oracle.com Mon Jun 15 14:20:23 2020 From: igor.ignatyev at oracle.com (igor.ignatyev at oracle.com) Date: Mon, 15 Jun 2020 07:20:23 -0700 Subject: RFR 15 8247521: (test) jdk/test/lib/hexdump/HexPrinterTest.java fails on windows In-Reply-To: <7f796110-58e4-f013-dec2-a1b1796777da@oracle.com> References: <7f796110-58e4-f013-dec2-a1b1796777da@oracle.com> Message-ID: Hi Roger, LGTM, Thanks, ? Igor > On Jun 15, 2020, at 7:10 AM, Roger Riggs wrote: > > Roger From magnus.ihse.bursie at oracle.com Mon Jun 15 15:12:45 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 15 Jun 2020 17:12:45 +0200 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: References: Message-ID: A few comments: This seems like code copied from elsewhere: 57 # This evaluation is expensive and should only be done if this target was 58 # explicitly called. 59 ifneq ($(filter build-test-libtest-jtreg-native, $(MAKECMDGOALS)), ) I don't agree that this is an expensive evaluation. Furthermore, the makefile is only called for building the testlib and for making images, so in worst case it's just the image part that would get a penalty (although I highly doubt there is any). 82 $(eval $(call SetupCopyFiles,COPY_LIBTEST_JTREG_NATIVE, \ Please use space after comma. /Magnus On 2020-06-13 05:38, Igor Ignatyev wrote: > adding build-dev > >> On Jun 12, 2020, at 8:36 PM, Igor Ignatyev wrote: >> >> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ >>> 796 lines changed: 200 ins; 588 del; 8 mod; >> Hi all, >> >> could you please review this small patch which puts all tests for testlibrary classes into one location under /test/lib-test? >> >> besides moving tests from test/jdk/lib/testlibrary and test/hotspot/jtreg/testlibrary_tests to test/lib-test the patch also >> - problem lists HexPrinterTest.java on windows due to JDK-8247521 >> - introduces make targets to build native parts for the tests in test/lib-test (needed b/c one test has a native part) >> - adds randomness k/w to test/lib-test (as it's used by RandomGeneratorTest.java) >> - makes Test class used by TestNativeProcessBuilder a static nested class of TestNativeProcessBuilder >> - updates LingeredAppTest to use @build instead of @compile and adds necessary @library tag >> - removes AssertsTest.java, OutputAnalyzerTest.java from test/hotspot/jtreg/testlibrary_tests as they are either identical or lesser that the same tests from test/jdk/lib/testlibrary/ >> - merges test/hotspot/jtreg/testlibrary_tests/OutputAnalyzerTest.java and test/jdk/lib/testlibrary/OutputAnalyzerTest.java (effectively adds test cases for `firstMatch` to the superier copy from test/jdk/lib/testlibrary) >> >> webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ >> JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 >> testing: test/lib-test on {windows,linux,macosx}-x64 >> >> Thanks, >> -- Igor >> >> From huizhe.wang at oracle.com Mon Jun 15 15:46:57 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Mon, 15 Jun 2020 08:46:57 -0700 Subject: RFR 15 8247521: (test) jdk/test/lib/hexdump/HexPrinterTest.java fails on windows In-Reply-To: <7f796110-58e4-f013-dec2-a1b1796777da@oracle.com> References: <7f796110-58e4-f013-dec2-a1b1796777da@oracle.com> Message-ID: +1 -Joe On 6/15/2020 7:09 AM, Roger Riggs wrote: > Please review a test change of the test library HexPrinter. > It should be comparing against the System.lineSeparator. > > > diff --git a/test/lib-test/jdk/test/lib/hexdump/HexPrinterTest.java > b/test/lib-test/jdk/test/lib/hexdump/HexPrinterTest.java > --- a/test/lib-test/jdk/test/lib/hexdump/HexPrinterTest.java > +++ b/test/lib-test/jdk/test/lib/hexdump/HexPrinterTest.java > @@ -77,9 +77,9 @@ public class HexPrinterTest { > ???? Object[][] builtinParams() { > ???????? return new Object[][]{ > ???????????????? {"minimal", "", "%02x", 16, "", 64, > HexPrinter.Formatters.NONE, ""}, > -??????????????? {"canonical", "%08x? ", "%02x ", 16, "|", 31, > HexPrinter.Formatters.PRINTABLE, "|\n"}, > -??????????????? {"simple", "%5d: ", "%02x ", 16, " // ", 64, > HexPrinter.Formatters.ASCII, "\n"}, > -??????????????? {"source", "??? ", "(byte)%3d, ", 8, " // ", 64, > HexPrinter.Formatters.PRINTABLE, "\n"}, > +??????????????? {"canonical", "%08x? ", "%02x ", 16, "|", 31, > HexPrinter.Formatters.PRINTABLE, "|" + System.lineSeparator()}, > +??????????????? {"simple", "%5d: ", "%02x ", 16, " // ", 64, > HexPrinter.Formatters.ASCII,? System.lineSeparator()}, > +??????????????? {"source", "??? ", "(byte)%3d, ", 8, " // ", 64, > HexPrinter.Formatters.PRINTABLE,? System.lineSeparator()}, > ???????? }; > ???? } > > > Webrev: > ? http://cr.openjdk.java.net/~rriggs/webrev-hexprinter-8247521/ > > Issue: > ?https://bugs.openjdk.java.net/browse/JDK-8247521 > > Thanks, Roger > From amaembo at gmail.com Mon Jun 15 16:45:54 2020 From: amaembo at gmail.com (Tagir Valeev) Date: Mon, 15 Jun 2020 23:45:54 +0700 Subject: RFR(xs): JDK-8247605 Avoid array allocation when concatenating with empty string Message-ID: Hello! Please review the following small change in StringConcatHelper::simpleConcat https://bugs.openjdk.java.net/browse/JDK-8247605 https://cr.openjdk.java.net/~tvaleev/webrev/8247605/r1/ This change improves the concatenation of empty string and non-empty string/object by reusing the internal byte[] array. Simple benchmark (StringConcat::concatEmptyRight and StringConcat::concatEmptyLeft) shows the significant performance improvement in both cases when an empty string is prefixed or suffixed. Before patch: Benchmark (intValue) Mode Cnt Score Error Units StringConcat.concatEmptyLeft 4711 avgt 30 18,416 ? 0,081 ns/op StringConcat.concatEmptyRight 4711 avgt 30 18,439 ? 0,068 ns/op StringConcat.concatEmptyLeft:?gc.alloc.rate.norm 4711 avgt 10 48,003 ? 0,001 B/op StringConcat.concatEmptyRight:?gc.alloc.rate.norm 4711 avgt 10 48,003 ? 0,001 B/op After patch: Benchmark (intValue) Mode Cnt Score Error Units StringConcat.concatEmptyLeft 4711 avgt 30 7,846 ? 0,069 ns/op StringConcat.concatEmptyRight 4711 avgt 30 6,800 ? 0,082 ns/op StringConcat.concatEmptyLeft:?gc.alloc.rate.norm 4711 avgt 10 24,001 ? 0,001 B/op StringConcat.concatEmptyRight:?gc.alloc.rate.norm 4711 avgt 10 24,001 ? 0,001 B/op Note that the allocation and array copying time is O(N) where N is the length of non-empty string, so for longer strings, the difference could be bigger. No performance degradation was noticed by me in other StringConcat benchmarks. Previous discussion: http://mail.openjdk.java.net/pipermail/core-libs-dev/2020-June/067217.html With best regards, Tagir Valeev. From claes.redestad at oracle.com Mon Jun 15 17:21:23 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 15 Jun 2020 19:21:23 +0200 Subject: RFR(xs): JDK-8247605 Avoid array allocation when concatenating with empty string In-Reply-To: References: Message-ID: <79657aa0-22b6-c5de-354d-a87cf927c6c9@oracle.com> Hi Tagir, changes looks good to me. Thanks! /Claes On 2020-06-15 18:45, Tagir Valeev wrote: > Hello! > > Please review the following small change in StringConcatHelper::simpleConcat > https://bugs.openjdk.java.net/browse/JDK-8247605 > https://cr.openjdk.java.net/~tvaleev/webrev/8247605/r1/ > > This change improves the concatenation of empty string and non-empty > string/object by reusing the internal byte[] array. > > Simple benchmark (StringConcat::concatEmptyRight and > StringConcat::concatEmptyLeft) shows the significant performance > improvement in both cases when an empty string is prefixed or > suffixed. Before patch: > > Benchmark (intValue) Mode Cnt Score Error Units > StringConcat.concatEmptyLeft 4711 avgt 30 18,416 ? 0,081 ns/op > StringConcat.concatEmptyRight 4711 avgt 30 18,439 ? 0,068 ns/op > StringConcat.concatEmptyLeft:?gc.alloc.rate.norm > 4711 avgt 10 48,003 ? 0,001 B/op > StringConcat.concatEmptyRight:?gc.alloc.rate.norm > 4711 avgt 10 48,003 ? 0,001 B/op > > > After patch: > Benchmark (intValue) Mode Cnt Score Error Units > StringConcat.concatEmptyLeft 4711 avgt 30 7,846 ? 0,069 ns/op > StringConcat.concatEmptyRight 4711 avgt 30 6,800 ? 0,082 ns/op > StringConcat.concatEmptyLeft:?gc.alloc.rate.norm > 4711 avgt 10 24,001 ? 0,001 B/op > StringConcat.concatEmptyRight:?gc.alloc.rate.norm > 4711 avgt 10 24,001 ? 0,001 B/op > > Note that the allocation and array copying time is O(N) where N is the > length of non-empty string, so for longer strings, the difference > could be bigger. No performance degradation was noticed by me in other > StringConcat benchmarks. > > Previous discussion: > http://mail.openjdk.java.net/pipermail/core-libs-dev/2020-June/067217.html > > With best regards, > Tagir Valeev. > From peter.kessler at os.amperecomputing.com Mon Jun 15 18:47:05 2020 From: peter.kessler at os.amperecomputing.com (Peter Kessler (Open Source)) Date: Mon, 15 Jun 2020 18:47:05 +0000 Subject: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) In-Reply-To: <85917488-6695-8ffa-89bd-35c1e5118c1f@oracle.com> References: <814DA850-76A9-4526-A3D9-141CD516C9AE@amperemail.onmicrosoft.com> <71ea732b-4566-e0a0-b2a3-e71df8a79609@oracle.com> <0E094A25-70C6-455D-A01D-F524A8010210@amperemail.onmicrosoft.com> <04b1cf61-c1cf-89eb-bcf2-ddd0f69e9321@oracle.com> <3EC88274-ACE8-43F6-A0BA-299182CEC451@amperemail.onmicrosoft.com> <85917488-6695-8ffa-89bd-35c1e5118c1f@oracle.com> Message-ID: <26BDD1BA-3DDC-4EB5-98D8-B0F80B92EE1B@amperemail.onmicrosoft.com> Alan, Indeed! Thanks for the instruction. Running $ make test-image and adding -nativepath:build/linux-$(arch)-server-release/images/test/jdk/jtreg/native to my jtreg command line allows me to pass all tests that had errors, on aarch64 and x86_64, with and without the proposed patch. ... peter ?-----Original Message----- From: Alan Bateman Date: Saturday, June 13, 2020 at 7:24 AM To: "Peter Kessler (Open Source)" , "core-libs-dev at openjdk.java.net" Subject: Re: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) On 12/06/2020 21:04, Peter Kessler (Open Source) wrote: > Daniel, > > Thanks for the suggestions to increase the timeout and lower the > concurrency. I did Bad Science(TM) and changed them both at the same > time. I am now running > > $ jtreg-5.0-b01/bin/jtreg \ > -verbose:summary -a -ea -esa -agentvm -conc:1 -ignore:quiet -timeout:4 \ > -exclude:./test/jdk/ProblemList.txt \ > -jdk:${jdk} \ > ./test/jdk/:jdk_core ./test/jdk/:jdk_svc > > I only had time for 2 tests of each jdk overnight, because running that > way takes much longer, but the results are less confusing. > > Running on x86_64, counting failures and errors in 2 runs with OracleJDK > build 15-ea+25-1229 > > 2 Error: java/lang/ClassLoader/nativeLibrary/NativeLibraryTest.java > 2 Error: java/lang/instrument/HiddenClass/HiddenClassAgent.java > 2 Error: java/lang/reflect/exeCallerAccessTest/CallerAccessTest.java > 2 Error: java/lang/String/nativeEncoding/StringPlatformChars.java > 2 Error: java/nio/channels/FileChannel/directio/DirectIOTest.java > 2 Error: java/nio/channels/spi/SelectorProvider/inheritedChannel/InheritedChannelTest.java > 2 Error: jdk/internal/loader/NativeLibraries/Main.java > 2 Error: Some tests failed or other problems occurred. > 2 Error: sun/management/jmxremote/bootstrap/CustomLauncherTest.java > 2 Error: tools/launcher/JliLaunchTest.java > These are tests that use native libraries and need jtreg run with -nativepath with the path to these libraries. The "test" make target might be want you want, to avoid your jtreg command getting too complicated. -Alan From erik.joelsson at oracle.com Mon Jun 15 21:00:25 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Mon, 15 Jun 2020 14:00:25 -0700 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: References: Message-ID: On 2020-06-15 08:12, Magnus Ihse Bursie wrote: > A few comments: > > This seems like code copied from elsewhere: > > ? 57 # This evaluation is expensive and should only be done if this > target was > ? 58 # explicitly called. > ? 59 ifneq ($(filter build-test-libtest-jtreg-native, $(MAKECMDGOALS)), ) > > I don't agree that this is an expensive evaluation. Furthermore, the > makefile is only called for building the testlib and for making > images, so in worst case it's just the image part that would get a > penalty (although I highly doubt there is any). > This construct was added by me in the original file he copied from, so Igor is just following existing patterns. The SetupTestFilesCompilation calls result in rather big $(shell find...) calls. If you want to keep rebuilds of "make images" as fast as possible, these are certainly relevant optimizations. /Erik > ? 82 $(eval $(call SetupCopyFiles,COPY_LIBTEST_JTREG_NATIVE, \ > > Please use space after comma. > > /Magnus > > On 2020-06-13 05:38, Igor Ignatyev wrote: >> adding build-dev >> >>> On Jun 12, 2020, at 8:36 PM, Igor Ignatyev >>> wrote: >>> >>> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ >>>> 796 lines changed: 200 ins; 588 del; 8 mod; >>> Hi all, >>> >>> could you please review this small patch which puts all tests for >>> testlibrary classes into one location under /test/lib-test? >>> >>> besides moving tests from test/jdk/lib/testlibrary and >>> test/hotspot/jtreg/testlibrary_tests to test/lib-test the patch also >>> - problem lists HexPrinterTest.java on windows due to JDK-8247521 >>> - introduces make targets to build native parts for the tests in >>> test/lib-test (needed b/c one test has a native part) >>> - adds randomness k/w to test/lib-test (as it's used by >>> RandomGeneratorTest.java) >>> - makes Test class used by TestNativeProcessBuilder a static nested >>> class of TestNativeProcessBuilder >>> - updates LingeredAppTest to use @build instead of @compile and adds >>> necessary @library tag >>> - removes AssertsTest.java, OutputAnalyzerTest.java from >>> test/hotspot/jtreg/testlibrary_tests as they are either identical or >>> lesser that the same tests from test/jdk/lib/testlibrary/ >>> - merges >>> test/hotspot/jtreg/testlibrary_tests/OutputAnalyzerTest.java and? >>> test/jdk/lib/testlibrary/OutputAnalyzerTest.java (effectively adds >>> test cases for `firstMatch` to the superier copy from >>> test/jdk/lib/testlibrary) >>> >>> webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.00/ >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 >>> testing: test/lib-test on {windows,linux,macosx}-x64 >>> >>> Thanks, >>> -- Igor >>> >>> > From mandy.chung at oracle.com Mon Jun 15 21:58:19 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 15 Jun 2020 14:58:19 -0700 Subject: (15) RFR: JDK-8247444: Trust final fields in records Message-ID: This patch is joint contribution from Christoph Dreis [1] and me. Webrev: http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247444/webrev.00/ CSR: https://bugs.openjdk.java.net/browse/JDK-8247517 This proposes to make final fields in records notmodifiable via reflection.? Field::set throws IAE if a Field is not modifiable.? Thecurrent specification specifies the following Fields not modifiable: - static final fields in any class - final fields in a hidden class The spec of Field::set is changed to specify that records are not modifiable via reflection. ?Noe that no change in Field::setAccessible(true), i.e. it will succeed to allow existing frameworks to have read access to final fields in records.? Just no write access. VarHandle does not support write access if it's a static final field or an instance final field. This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. No change is made in JNI.? JNI could be considered to disallow modification of final fields in records and hidden classes (static and instance fields).? But JNI has superpower and the current spec already allows to modify the value of a final static field even after it's constant folded (via JNI SetField and SetStaticField), then all bets are off.? This should be re-visited when we consider "final is truly final" for all classes. Make final fields in records not modifiable via reflection enables JIT optimization as these final fields are effectively truly final. This change impacts 3rd-party frameworks including 3rd-party serialization framework that rely on core reflection `setAccessible` or `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to construct records but not using the canonical constructor. These frameworks would need to be updated to construct records via its canonical constructor as done by the Java serialization. I see this change gives a good opportunity to engage the maintainers of the serialization frameworks and work together to support new features including records, inline classes and the new serialization mechanism and which I think it is worth the investment. This is a low risk enhancement.? I'd like to request approval for a late enhancement in JDK 15.? It extends the pre-existing code path with refactoring the hidden classes to prepare for new kinds of classes with trusted final fields.? The change is straight-forward. Can this wait to integrate in JDK 16? ? It's important to get this enhancement in when record is a preview feature that we can get feedback and give time to 3rd-party serialization frameworks to look into adding the support for records.? If we delayed this change in 16 and records exit preview, it would be bad for frameworks if they verify that they support records in 15 but fail in 16.? OTOH the risk of this patch is low. Performance Impact I addressed the performance concern I raised earlier myself.? For reflection, VM creates the reflective Field objects and fills in MemberName when resolving a member.? VM will tag if this Field/MemberName is trusted final field.? I think this is a cleaner approach rather than in each place to check for final static and final fields in hidden or record class to determine if it has write access or not. `sun.misc.Unsafe` does not use Field::isTrustedFinalField because (1) Unsafe has been allowing access of static final fields of any classes but isTrustedFinalField is not limited to instance fields (2) Unsafe disallows access to all fields in a hidden class (not limited to trusted final fields).? So it follows the precedence and simply checks if the declaring class is a record. `Class::isRecord` calls `Class::getSuperclass` to check if it's a subtype of `Record`.? As `Class::getSuperclass` is intrinsified, the call on isRecord on a normal class is fast. Christoph has contributed the microbenchmarks that confirm that no performance regression. Thanks Mandy [1] https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-June/040096.html From igor.ignatyev at oracle.com Tue Jun 16 00:39:38 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Mon, 15 Jun 2020 17:39:38 -0700 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: <39df1e40-f34a-ba3b-e2e2-f462a6442c42@oracle.com> References: <39df1e40-f34a-ba3b-e2e2-f462a6442c42@oracle.com> Message-ID: <214B2701-0723-45A5-81B5-816EF1D86470@oracle.com> @David, Erik, Magnus, please find the answers to your comments at the bottom of this email. @all, David's and Erik's comments made me realize that some parts of the original patch were stashed away and didn't make it to webrev.00. I'm truly sorry for the confusion and inconvenience. I also agree w/ David that it's hard to follow/review, and my gaffe just made it worse, therefore I'd like to start it over again from a clean sate http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 > 833 lines changed: 228 ins; 591 del; 14 mod; could you please review the patch which puts all tests for common testlibrary classes (which is ones located in /test/lib) into one location under /test/lib-test? please note this intentionally doesn't move all "testlibrary" tests, e.g. tests for CTW, hotspot-specific testlibrary, are left in /test/hotspot/jtreg/testlibrary_tests/ctw . to simplify review, the patch has been split into several webrevs, with each of them accomplishes a more-or-less distinct thing, but is not necessary self-contained: 0. trivial move of tests from test/jdk and test/hotspot/jtreg test suites to test/lib-test: http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.00-move/ 1. removal of hotspot's AssertsTest, OutputAnalyzerReportingTest and "merge" of hotspot's and jdk's OutputAnalyzerTest: http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.01-merge/ 2. simplification of TestNativeProcessBuilder tests: converts Test class used by TestNativeProcessBuilder into a static nested class: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.02-TestNativeProcessBuilder 3. makefiles changes to build native parts of lib-test tests. in past, there were no tests w/ native parts in this test suite, TestNativeProcessBuilder is the 1st such test; this part also removes now unneeded lines from hotspot test suite makefile: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.03-NativeLibTest 4. clean ups in hotspot test suite: adjusted location of SimpleClassFileLoadHookTest test, which is a test for hotspot-specific test library (located in /test/hotspot/jtreg/testlibrary/jvmti) and hence should not be moved to /test/lib-test; removed TestMutuallyExclusivePlatformPredicates from TEST.groups: http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.04-hotspot/ 5. LingeredAppTest fix (which apparently has never been run before): http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.05-LingeredAppTest/ 6. changes to make test/lib-test a fully supported and working test suite, and add in into tier1, includes adding ProblemList, TEST.groups file, and 'randomness' k/w: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.06-TESTROOT/ webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 testing: - make test TEST=tier1 locally on macosx - test/lib-test on {windows,linux,macosx}-x64 - tier1 job (in progress) Thanks, -- Igor > On Jun 14, 2020, at 11:23 PM, David Holmes wrote: > <...> > This doesn't seem to move everything under test/hotspot/jtreg/testlibrary_tests/ e.g. ctw directory. this is intentionally, ctw test-library is hotspot specific, hence its tests are to reside in hotspot test suite (until we decide to move the library to /test/lib), the same is true for SimpleClassFileLoadHookTest. > > You did not modify hotspot/jtreg/TEST.groups but it refers to: > testlibrary_tests/TestMutuallyExclusivePlatformPredicates.java \ > Plus it should probably refer to the new native_sanity tests somewhere. the actual version of the patch removed reference to TestMutuallyExclusivePlatformPredicates from hotspot's TEST.groups and had TestNativeProcessBuilder moved to /test/test-lib, so no updates w.r.t. native_sanity are needed > The newly added copyright notice needs to have the correct initial copyright year based on when the file was first added to the repo. /test/lib-test/TEST.ROOT file was created by JDK-8243010 on 2020-04-16, hence the added copyright has 2020 as the initial copyright year. > You used the wrong license header - makefiles don't use the classpath exception. JtregNativeLibTest.gmk is based on make/test/JtregNativeJdk.gmk which also have classpath exception. quick grep showed that make directory has 794 files which has '"Classpath" exception', out of 850 which contain 'GNU General Public License version 2' string. And although I agree that makefiles shouldn't have the classpath exception, I'd prefer to keep JtregNativeLibTest.gmk in sync w/ the its siblings and would leave it up to build team to decide how it's best to handle. > On Jun 15, 2020, at 8:12 AM, Magnus Ihse Bursie wrote: > > A few comments: > > This seems like code copied from elsewhere: > 57 # This evaluation is expensive and should only be done if this target was > 58 # explicitly called. > 59 ifneq ($(filter build-test-libtest-jtreg-native, $(MAKECMDGOALS)), ) > I don't agree that this is an expensive evaluation. Furthermore, the makefile is only called for building the testlib and for making images, so in worst case it's just the image part that would get a penalty (although I highly doubt there is any). right, JtregNativeLibTest.gmk is based on make/test/JtregNativeJdk.gmk which has this comment, I don't know enough details to say if it's expensive or not, yet if you insist and there is a consensus within build team, I can remove the comment. > > 82 $(eval $(call SetupCopyFiles,COPY_LIBTEST_JTREG_NATIVE, \ > Please use space after comma. this again was preexisting in JtregNativeJdk.gmk, added the space nevertheless. > On Jun 15, 2020, at 6:16 AM, Erik Joelsson wrote: > > In JtretNativeLibTest.gmk, lines 51-55 should probably be removed (or adjusted if linking to libjvm is actually needed). the lines were updated to define BUILD_LIBTEST_JTREG_EXECUTABLES_LIBS_exejvm-test-launcher From david.holmes at oracle.com Tue Jun 16 02:29:55 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 16 Jun 2020 12:29:55 +1000 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: Hi Mandy, Christoph, The code changes here all look okay to me. The idea of introducing the notion of "trusted final fields" seems quite reasonable. I made one editorial comment on the CSR request. I'm unclear if the new TEST.properties file needs a copyright notice and header. We have 706 such files in the repo and 554 (mostly hotspot) do have the copyright notice and header. Thanks, David On 16/06/2020 7:58 am, Mandy Chung wrote: > This patch is joint contribution from Christoph Dreis [1] and me. > > Webrev: http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247444/webrev.00/ > CSR: https://bugs.openjdk.java.net/browse/JDK-8247517 > > This proposes to make final fields in records notmodifiable via > reflection.? Field::set throws IAE if a Field is not modifiable. > Thecurrent specification specifies the following Fields not modifiable: > - static final fields in any class > - final fields in a hidden class > > The spec of Field::set is changed to specify that records are not > modifiable via reflection. > ?Noe that no change in Field::setAccessible(true), i.e. it will succeed > to allow existing frameworks to have read access to final fields in > records.? Just no write access. > > VarHandle does not support write access if it's a static final field or > an instance final field. > > This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and > `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. > > No change is made in JNI.? JNI could be considered to disallow > modification of final fields in records and hidden classes (static and > instance fields).? But JNI has superpower and the current spec already > allows to modify the value of a final static field even after it's > constant folded (via JNI SetField and SetStaticField), then > all bets are off.? This should be re-visited when we consider "final is > truly final" for all classes. > > Make final fields in records not modifiable via reflection enables JIT > optimization as these final fields are effectively truly final. > > This change impacts 3rd-party frameworks including 3rd-party > serialization framework that rely on core reflection `setAccessible` or > `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to > construct records but not using the canonical constructor. > These frameworks would need to be updated to construct records via its > canonical constructor as done by the Java serialization. > > I see this change gives a good opportunity to engage the maintainers of > the serialization frameworks and work together to support new features > including records, inline classes and the new serialization mechanism > and which I think it is worth the investment. > > This is a low risk enhancement.? I'd like to request approval for a late > enhancement in JDK 15.? It extends the pre-existing code path with > refactoring the hidden classes to prepare for new kinds of classes with > trusted final fields.? The change is straight-forward. > > Can this wait to integrate in JDK 16? > > ? It's important to get this enhancement in when record is a preview > feature that we can get feedback and give time to 3rd-party > serialization frameworks to look into adding the support for records. If > we delayed this change in 16 and records exit preview, it would be bad > for frameworks if they verify that they support records in 15 but fail > in 16.? OTOH the risk of this patch is low. > > Performance Impact > > I addressed the performance concern I raised earlier myself.? For > reflection, VM creates the reflective Field objects and fills in > MemberName when resolving a member.? VM will tag if this > Field/MemberName is trusted final field.? I think this is a cleaner > approach rather than in each place to check for final static and final > fields in hidden or record class to determine if it has write access or > not. > > `sun.misc.Unsafe` does not use Field::isTrustedFinalField because (1) > Unsafe has been allowing access of static final fields of any classes > but isTrustedFinalField is not limited to instance fields (2) Unsafe > disallows access to all fields in a hidden class (not limited to trusted > final fields).? So it follows the precedence and simply checks if the > declaring class is a record. `Class::isRecord` calls > `Class::getSuperclass` to check if it's a subtype of `Record`.? As > `Class::getSuperclass` is intrinsified, the call on isRecord on a normal > class is fast. Christoph has contributed the microbenchmarks that > confirm that no performance regression. > > Thanks > Mandy > [1] > https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-June/040096.html > From david.holmes at oracle.com Tue Jun 16 07:14:59 2020 From: david.holmes at oracle.com (David Holmes) Date: Tue, 16 Jun 2020 17:14:59 +1000 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: <214B2701-0723-45A5-81B5-816EF1D86470@oracle.com> References: <39df1e40-f34a-ba3b-e2e2-f462a6442c42@oracle.com> <214B2701-0723-45A5-81B5-816EF1D86470@oracle.com> Message-ID: Hi Igor, On 16/06/2020 10:39 am, Igor Ignatyev wrote: > @David, Erik, Magnus, > > please find the answers to your comments at the bottom of this email. > > @all, > > David's and Erik's comments made me realize that some parts of the > original patch were stashed away and didn't make it to webrev.00. I'm > truly sorry for the confusion and inconvenience. I also agree w/ David > that it's hard to follow/review, and my?gaffe just made it worse, > therefore I'd like to start it over again from a clean sate > > http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 >> 833 lines changed: 228 ins; 591 del; 14 mod; > > could you please review the patch which puts all tests for common > testlibrary classes (which is ones located in /test/lib) into one > location under /test/lib-test? please note this intentionally doesn't > move all "testlibrary" tests, e.g. tests for CTW, hotspot-specific > testlibrary, are left in /test/hotspot/jtreg/testlibrary_tests/ctw . Ok. > to simplify review, the patch has been split into several webrevs, with > each of them accomplishes a more-or-less distinct thing, but is not > necessary self-contained: Many thanks for doing this! > 0. trivial move of tests from test/jdk and test/hotspot/jtreg test > suites to test/lib-test: > http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.00-move/ Looks good. > 1. removal of hotspot's AssertsTest,?OutputAnalyzerReportingTest and > "merge" of hotspot's and jdk's OutputAnalyzerTest: > http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.01-merge/ Looks good. > 2. simplification of TestNativeProcessBuilder tests: converts Test class > used by TestNativeProcessBuilder into a static nested class: > http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.02-TestNativeProcessBuilder Looks good. > 3. makefiles changes to build native parts of lib-test tests. in past, > there were no tests w/ native parts in this test suite, > TestNativeProcessBuilder is the 1st such test; this part also removes > now unneeded lines from hotspot test suite makefile: > http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.03-NativeLibTest Hmm okay. Not sure this needed its own category of build logic but ... Aside: Makefiles should not have the classpath exception version of the license header. But they all do for some reason. I'll check if we need to do a separate cleanup of that. > 4. clean ups in hotspot test suite: adjusted location > of?SimpleClassFileLoadHookTest test, which is a test for > hotspot-specific test library (located in > /test/hotspot/jtreg/testlibrary/jvmti) and hence should not be moved to > /test/lib-test; removed TestMutuallyExclusivePlatformPredicates from > TEST.groups: > http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.04-hotspot/ Looks good. Took me a while to understand the connection to the test library here. > 5. LingeredAppTest fix (which apparently has never been run before): > http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.05-LingeredAppTest/ Someone from serviceability should evaluate this test. It may not be needed. > 6. changes to make test/lib-test a fully supported and working test > suite, and add in into tier1, ?includes adding ProblemList, TEST.groups > file, and 'randomness' k/w: > http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.06-TESTROOT/ Seems okay. Thanks, David ----- > webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 > JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 > testing: > ?- make test TEST=tier1 locally on macosx > ?- test/lib-test on ?{windows,linux,macosx}-x64 > ?- tier1 job (in progress) > > Thanks, > -- Igor > > >> On Jun 14, 2020, at 11:23 PM, David Holmes > > wrote: >> <...> >> This doesn't seem to move everything under >> test/hotspot/jtreg/testlibrary_tests/ e.g. ctw directory. > this is intentionally, ctw test-library is hotspot specific, hence its > tests are to reside in hotspot test suite (until we decide to move the > library to /test/lib), the same is true for?SimpleClassFileLoadHookTest. >> >> You did not modify hotspot/jtreg/TEST.groups but it refers to: >> testlibrary_tests/TestMutuallyExclusivePlatformPredicates.java \ >> Plus it should probably refer to the new native_sanity tests somewhere. > the actual version of the patch removed reference to > TestMutuallyExclusivePlatformPredicates from hotspot's TEST.groups and > had TestNativeProcessBuilder moved to /test/test-lib, so no updates > w.r.t. native_sanity are needed > >> The newly added copyright notice needs to have the correct initial >> copyright year based on when the file was first added to the repo. > ?/test/lib-test/TEST.ROOT file was created by JDK-8243010 on > 2020-04-16, hence the added copyright has 2020 as the initial copyright > year. > >> You used the wrong license header - makefiles don't use the classpath >> exception. > JtregNativeLibTest.gmk is based on?make/test/JtregNativeJdk.gmk which > also have classpath exception. quick grep showed that make directory has > 794 files which has '"Classpath" exception', out of 850 which contain > 'GNU General Public License version 2' string. And although I agree that > makefiles shouldn't have the classpath exception, I'd prefer to keep > JtregNativeLibTest.gmk in sync w/ the its siblings and would leave it up > to build team to decide how it's best to handle. > > >> On Jun 15, 2020, at 8:12 AM, Magnus Ihse Bursie >> > >> wrote: >> >> A few comments: >> >> This seems like code copied from elsewhere: >> 57 # This evaluation is expensive and should only be done if this target was >> 58 # explicitly called. >> 59 ifneq ($(filter build-test-libtest-jtreg-native, $(MAKECMDGOALS)), ) >> I don't agree that this is an expensive evaluation. Furthermore, the >> makefile is only called for building the testlib and for making >> images, so in worst case it's just the image part that would get a >> penalty (although I highly doubt there is any). > right, JtregNativeLibTest.gmk is based on?make/test/JtregNativeJdk.gmk > which has this comment, I don't know enough details to say if it's > expensive or not, yet if you insist and there is a consensus within > build team, I can remove the comment. > >> >> 82 $(eval $(call SetupCopyFiles,COPY_LIBTEST_JTREG_NATIVE, \ >> Please use space after comma. > this again was preexisting in?JtregNativeJdk.gmk, added the space > nevertheless. > > >> On Jun 15, 2020, at 6:16 AM, Erik Joelsson > > wrote: >> >> In JtretNativeLibTest.gmk, lines 51-55 should probably be removed (or >> adjusted if linking to libjvm is actually needed). > the lines were updated to define > BUILD_LIBTEST_JTREG_EXECUTABLES_LIBS_exejvm-test-launcher > > From forax at univ-mlv.fr Tue Jun 16 09:49:19 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 16 Jun 2020 11:49:19 +0200 (CEST) Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: <1229321025.1095655.1592300959835.JavaMail.zimbra@u-pem.fr> Hi Mndy, Looks good, I don't like too much the fact that there is a new boolean field in j.l.r.Field instead of using the modifiers like in the VM counterpart, but it will require masking and unmasking the modifiers which is a far bigger change, too big to worth it in my opinion. So +1 R?mi ----- Mail original ----- > De: "mandy chung" > ?: "hotspot-dev" , "core-libs-dev" , "amber-dev" > > Envoy?: Lundi 15 Juin 2020 23:58:19 > Objet: (15) RFR: JDK-8247444: Trust final fields in records > This patch is joint contribution from Christoph Dreis [1] and me. > > Webrev: http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247444/webrev.00/ > CSR: https://bugs.openjdk.java.net/browse/JDK-8247517 > > This proposes to make final fields in records notmodifiable via > reflection.? Field::set throws IAE if a Field is not modifiable. > Thecurrent specification specifies the following Fields not modifiable: > - static final fields in any class > - final fields in a hidden class > > The spec of Field::set is changed to specify that records are not > modifiable via reflection. > ?Noe that no change in Field::setAccessible(true), i.e. it will succeed > to allow existing frameworks to have read access to final fields in > records.? Just no write access. > > VarHandle does not support write access if it's a static final field or > an instance final field. > > This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and > `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. > > No change is made in JNI.? JNI could be considered to disallow > modification of final fields in records and hidden classes (static and > instance fields).? But JNI has superpower and the current spec already > allows to modify the value of a final static field even after it's > constant folded (via JNI SetField and SetStaticField), then > all bets are off.? This should be re-visited when we consider "final is > truly final" for all classes. > > Make final fields in records not modifiable via reflection enables JIT > optimization as these final fields are effectively truly final. > > This change impacts 3rd-party frameworks including 3rd-party > serialization framework that rely on core reflection `setAccessible` or > `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to > construct records but not using the canonical constructor. > These frameworks would need to be updated to construct records via its > canonical constructor as done by the Java serialization. > > I see this change gives a good opportunity to engage the maintainers of > the serialization frameworks and work together to support new features > including records, inline classes and the new serialization mechanism > and which I think it is worth the investment. > > This is a low risk enhancement.? I'd like to request approval for a late > enhancement in JDK 15.? It extends the pre-existing code path with > refactoring the hidden classes to prepare for new kinds of classes with > trusted final fields.? The change is straight-forward. > > Can this wait to integrate in JDK 16? > > ? It's important to get this enhancement in when record is a preview > feature that we can get feedback and give time to 3rd-party > serialization frameworks to look into adding the support for records. > If we delayed this change in 16 and records exit preview, it would be > bad for frameworks if they verify that they support records in 15 but > fail in 16.? OTOH the risk of this patch is low. > > Performance Impact > > I addressed the performance concern I raised earlier myself.? For > reflection, VM creates the reflective Field objects and fills in > MemberName when resolving a member.? VM will tag if this > Field/MemberName is trusted final field.? I think this is a cleaner > approach rather than in each place to check for final static and final > fields in hidden or record class to determine if it has write access or > not. > > `sun.misc.Unsafe` does not use Field::isTrustedFinalField because (1) > Unsafe has been allowing access of static final fields of any classes > but isTrustedFinalField is not limited to instance fields (2) Unsafe > disallows access to all fields in a hidden class (not limited to trusted > final fields).? So it follows the precedence and simply checks if the > declaring class is a record. `Class::isRecord` calls > `Class::getSuperclass` to check if it's a subtype of `Record`.? As > `Class::getSuperclass` is intrinsified, the call on isRecord on a normal > class is fast. Christoph has contributed the microbenchmarks that > confirm that no performance regression. > > Thanks > Mandy > [1] > https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-June/040096.html From claes.redestad at oracle.com Tue Jun 16 13:00:10 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 16 Jun 2020 15:00:10 +0200 Subject: RFR[16]: 8247681: Improve bootstrapping of unary concatenations Message-ID: <10b4c312-3223-3412-016c-60515581733d@oracle.com> Hi, this patch specializes bootstrapping of unary concatenation expressions. Such expressions can be reduced to the canonical String.valueOf "stringifier" for any primitive argument, but needs a special stringifier for reference arguments since we need to produce a new String to be compliant with JLS. Bug: https://bugs.openjdk.java.net/browse/JDK-8247681 Webrev: http://cr.openjdk.java.net/~redestad/8247681/open.00/ This ensures we get a similar speed-up as JDK-8247605 for "" + fooString (and fooString + ""). Also speeds up bootstrapping for all such simple unary concatenation expressions. Testing: tier1+2 Before: Benchmark (intValue) Mode Cnt Score Error Units StringConcat.concatEmptyConstInt 4711 avgt 5 15.539 ? 0.831 ns/op StringConcat.concatEmptyConstString 4711 avgt 5 17.046 ? 1.047 ns/op StringConcat.concatEmptyLeft 4711 avgt 5 7.506 ? 0.588 ns/op StringConcat.concatEmptyRight 4711 avgt 5 7.890 ? 0.314 ns/op After: Benchmark (intValue) Mode Cnt Score Error Units StringConcat.concatEmptyConstInt 4711 avgt 5 15.410 ? 0.944 ns/op StringConcat.concatEmptyConstString 4711 avgt 5 7.397 ? 0.384 ns/op StringConcat.concatEmptyLeft 4711 avgt 5 7.465 ? 0.328 ns/op StringConcat.concatEmptyRight 4711 avgt 5 7.857 ? 0.355 ns/op Thanks! /Claes From erik.joelsson at oracle.com Tue Jun 16 13:06:49 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Tue, 16 Jun 2020 06:06:49 -0700 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: <214B2701-0723-45A5-81B5-816EF1D86470@oracle.com> References: <39df1e40-f34a-ba3b-e2e2-f462a6442c42@oracle.com> <214B2701-0723-45A5-81B5-816EF1D86470@oracle.com> Message-ID: On 2020-06-15 17:39, Igor Ignatyev wrote: > @David, Erik, Magnus, > > please find the answers to your comments at the bottom of this email. > > @all, > > David's and Erik's comments made me realize that some parts of the > original patch were stashed away and didn't make it to webrev.00. I'm > truly sorry for the confusion and inconvenience. I also agree w/ David > that it's hard to follow/review, and my?gaffe just made it worse, > therefore I'd like to start it over again from a clean sate > > http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 Build changes look good to me now, except for a missing newline in Main.gmk. (No need for new review.) /Erik >> 833 lines changed: 228 ins; 591 del; 14 mod; > > could you please review the patch which puts all tests for common > testlibrary classes (which is ones located in /test/lib) into one > location under /test/lib-test? please note this intentionally doesn't > move all "testlibrary" tests, e.g. tests for CTW, hotspot-specific > testlibrary, are left in /test/hotspot/jtreg/testlibrary_tests/ctw . > > to simplify review, the patch has been split into several webrevs, > with each of them accomplishes a more-or-less distinct thing, but is > not necessary self-contained: > > 0. trivial move of tests from test/jdk and test/hotspot/jtreg test > suites to test/lib-test: > http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.00-move/ > > 1. removal of hotspot's AssertsTest,?OutputAnalyzerReportingTest and > "merge" of hotspot's and jdk's OutputAnalyzerTest: > http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.01-merge/ > > 2. simplification of TestNativeProcessBuilder tests: converts Test > class used by TestNativeProcessBuilder into a static nested class: > http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.02-TestNativeProcessBuilder > > 3. makefiles changes to build native parts of lib-test tests. in past, > there were no tests w/ native parts in this test suite, > TestNativeProcessBuilder is the 1st such test; this part also removes > now unneeded lines from hotspot test suite makefile: > http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.03-NativeLibTest > > 4. clean ups in hotspot test suite: adjusted location > of?SimpleClassFileLoadHookTest test, which is a test for > hotspot-specific test library (located in > /test/hotspot/jtreg/testlibrary/jvmti) and hence should not be moved > to /test/lib-test; removed TestMutuallyExclusivePlatformPredicates > from TEST.groups: > http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.04-hotspot/ > > 5. LingeredAppTest fix (which apparently has never been run before): > http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.05-LingeredAppTest/ > > 6. changes to make test/lib-test a fully supported and working test > suite, and add in into tier1, ?includes adding ProblemList, > TEST.groups file, and 'randomness' k/w: > http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.06-TESTROOT/ > > webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 > JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 > testing: > ?- make test TEST=tier1 locally on macosx > ?- test/lib-test on ?{windows,linux,macosx}-x64 > ?- tier1 job (in progress) > > Thanks, > -- Igor > > >> On Jun 14, 2020, at 11:23 PM, David Holmes > > wrote: >> <...> >> This doesn't seem to move everything under >> test/hotspot/jtreg/testlibrary_tests/ e.g. ctw directory. > this is intentionally, ctw test-library is hotspot specific, hence its > tests are to reside in hotspot test suite (until we decide to move the > library to /test/lib), the same is true for?SimpleClassFileLoadHookTest. >> >> You did not modify hotspot/jtreg/TEST.groups but it refers to: >> testlibrary_tests/TestMutuallyExclusivePlatformPredicates.java \ >> Plus it should probably refer to the new native_sanity tests somewhere. > the actual version of the patch removed reference to > TestMutuallyExclusivePlatformPredicates from hotspot's TEST.groups and > had TestNativeProcessBuilder moved to /test/test-lib, so no updates > w.r.t. native_sanity are needed > >> The newly added copyright notice needs to have the correct initial >> copyright year based on when the file was first added to the repo. > ?/test/lib-test/TEST.ROOT file was created by JDK-8243010 on > 2020-04-16, hence the added copyright has 2020 as the initial > copyright year. > >> You used the wrong license header - makefiles don't use the classpath >> exception. > JtregNativeLibTest.gmk is based on?make/test/JtregNativeJdk.gmk which > also have classpath exception. quick grep showed that make directory > has 794 files which has '"Classpath" exception', out of 850 which > contain 'GNU General Public License version 2' string. And although I > agree that makefiles shouldn't have the classpath exception, I'd > prefer to keep JtregNativeLibTest.gmk in sync w/ the its siblings and > would leave it up to build team to decide how it's best to handle. > > >> On Jun 15, 2020, at 8:12 AM, Magnus Ihse Bursie >> > > wrote: >> >> A few comments: >> >> This seems like code copied from elsewhere: >> 57 # This evaluation is expensive and should only be done if this target was >> 58 # explicitly called. >> 59 ifneq ($(filter build-test-libtest-jtreg-native, $(MAKECMDGOALS)), ) >> I don't agree that this is an expensive evaluation. Furthermore, the >> makefile is only called for building the testlib and for making >> images, so in worst case it's just the image part that would get a >> penalty (although I highly doubt there is any). > right, JtregNativeLibTest.gmk is based on?make/test/JtregNativeJdk.gmk > which has this comment, I don't know enough details to say if it's > expensive or not, yet if you insist and there is a consensus within > build team, I can remove the comment. > >> >> 82 $(eval $(call SetupCopyFiles,COPY_LIBTEST_JTREG_NATIVE, \ >> Please use space after comma. > this again was preexisting in?JtregNativeJdk.gmk, added the space > nevertheless. > > >> On Jun 15, 2020, at 6:16 AM, Erik Joelsson > > wrote: >> >> In JtretNativeLibTest.gmk, lines 51-55 should probably be removed (or >> adjusted if linking to libjvm is actually needed). > the lines were updated to define > BUILD_LIBTEST_JTREG_EXECUTABLES_LIBS_exejvm-test-launcher > > From james.laskey at oracle.com Tue Jun 16 13:09:41 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Tue, 16 Jun 2020 10:09:41 -0300 Subject: RFR[16]: 8247681: Improve bootstrapping of unary concatenations In-Reply-To: <10b4c312-3223-3412-016c-60515581733d@oracle.com> References: <10b4c312-3223-3412-016c-60515581733d@oracle.com> Message-ID: <3BD00826-34F1-4072-8DE2-A21C37E465EC@oracle.com> Would it be helpful to have benchmarks for the other types, just in case? Don't see any tests to ensure all cases are covered. May be relying on existing tests, but... minimally tag those tests. Cheers, -- Jim > On Jun 16, 2020, at 10:00 AM, Claes Redestad wrote: > > Hi, > > this patch specializes bootstrapping of unary concatenation expressions. > > Such expressions can be reduced to the canonical String.valueOf > "stringifier" for any primitive argument, but needs a special > stringifier for reference arguments since we need to produce a new > String to be compliant with JLS. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8247681 > Webrev: http://cr.openjdk.java.net/~redestad/8247681/open.00/ > > This ensures we get a similar speed-up as JDK-8247605 for > "" + fooString (and fooString + ""). > > Also speeds up bootstrapping for all such simple unary concatenation > expressions. > > Testing: tier1+2 > > Before: > > Benchmark (intValue) Mode Cnt Score Error Units > StringConcat.concatEmptyConstInt 4711 avgt 5 15.539 ? 0.831 ns/op > StringConcat.concatEmptyConstString 4711 avgt 5 17.046 ? 1.047 ns/op > StringConcat.concatEmptyLeft 4711 avgt 5 7.506 ? 0.588 ns/op > StringConcat.concatEmptyRight 4711 avgt 5 7.890 ? 0.314 ns/op > > After: > Benchmark (intValue) Mode Cnt Score Error Units > StringConcat.concatEmptyConstInt 4711 avgt 5 15.410 ? 0.944 ns/op > StringConcat.concatEmptyConstString 4711 avgt 5 7.397 ? 0.384 ns/op > StringConcat.concatEmptyLeft 4711 avgt 5 7.465 ? 0.328 ns/op > StringConcat.concatEmptyRight 4711 avgt 5 7.857 ? 0.355 ns/op > > Thanks! > > /Claes From claes.redestad at oracle.com Tue Jun 16 13:20:47 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 16 Jun 2020 15:20:47 +0200 Subject: RFR[16]: 8247681: Improve bootstrapping of unary concatenations In-Reply-To: <3BD00826-34F1-4072-8DE2-A21C37E465EC@oracle.com> References: <10b4c312-3223-3412-016c-60515581733d@oracle.com> <3BD00826-34F1-4072-8DE2-A21C37E465EC@oracle.com> Message-ID: <4d4dd5b5-c160-baff-72fe-692491db43d2@oracle.com> On 2020-06-16 15:09, Jim Laskey wrote: > Would it be helpful to have benchmarks for the other types, just in case? I'm not sure.. We need to be selective or we end up increasing expected run time by a large factor. We should also do a pass over most of these micros to provide saner defaults (number of forks, warmup times, run time etc are generally tuned a bit too high and we can get reasonable results on a smaller time budget) > > Don't see any tests to ensure all cases are covered. May be relying on existing tests, but... minimally tag those tests. test/jdk/java/lang/String/concat/ImplicitStringConcatShapes.java tests these unary shapes exhaustively. I'll add the bug number to that (and the .template for it) /Claes > > Cheers, > > -- Jim > > >> On Jun 16, 2020, at 10:00 AM, Claes Redestad wrote: >> >> Hi, >> >> this patch specializes bootstrapping of unary concatenation expressions. >> >> Such expressions can be reduced to the canonical String.valueOf >> "stringifier" for any primitive argument, but needs a special >> stringifier for reference arguments since we need to produce a new >> String to be compliant with JLS. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8247681 >> Webrev: http://cr.openjdk.java.net/~redestad/8247681/open.00/ >> >> This ensures we get a similar speed-up as JDK-8247605 for >> "" + fooString (and fooString + ""). >> >> Also speeds up bootstrapping for all such simple unary concatenation >> expressions. >> >> Testing: tier1+2 >> >> Before: >> >> Benchmark (intValue) Mode Cnt Score Error Units >> StringConcat.concatEmptyConstInt 4711 avgt 5 15.539 ? 0.831 ns/op >> StringConcat.concatEmptyConstString 4711 avgt 5 17.046 ? 1.047 ns/op >> StringConcat.concatEmptyLeft 4711 avgt 5 7.506 ? 0.588 ns/op >> StringConcat.concatEmptyRight 4711 avgt 5 7.890 ? 0.314 ns/op >> >> After: >> Benchmark (intValue) Mode Cnt Score Error Units >> StringConcat.concatEmptyConstInt 4711 avgt 5 15.410 ? 0.944 ns/op >> StringConcat.concatEmptyConstString 4711 avgt 5 7.397 ? 0.384 ns/op >> StringConcat.concatEmptyLeft 4711 avgt 5 7.465 ? 0.328 ns/op >> StringConcat.concatEmptyRight 4711 avgt 5 7.857 ? 0.355 ns/op >> >> Thanks! >> >> /Claes > From james.laskey at oracle.com Tue Jun 16 13:32:41 2020 From: james.laskey at oracle.com (Jim Laskey) Date: Tue, 16 Jun 2020 10:32:41 -0300 Subject: RFR[16]: 8247681: Improve bootstrapping of unary concatenations In-Reply-To: <4d4dd5b5-c160-baff-72fe-692491db43d2@oracle.com> References: <10b4c312-3223-3412-016c-60515581733d@oracle.com> <3BD00826-34F1-4072-8DE2-A21C37E465EC@oracle.com> <4d4dd5b5-c160-baff-72fe-692491db43d2@oracle.com> Message-ID: > On Jun 16, 2020, at 10:20 AM, Claes Redestad wrote: > > On 2020-06-16 15:09, Jim Laskey wrote: >> Would it be helpful to have benchmarks for the other types, just in case? > > I'm not sure.. We need to be selective or we end up increasing expected > run time by a large factor. We should also do a pass over most of these > micros to provide saner defaults (number of forks, warmup times, run > time etc are generally tuned a bit too high and we can get reasonable > results on a smaller time budget) okay. > >> Don't see any tests to ensure all cases are covered. May be relying on existing tests, but... minimally tag those tests. > > test/jdk/java/lang/String/concat/ImplicitStringConcatShapes.java tests > these unary shapes exhaustively. I'll add the bug number to that (and > the .template for it) +1 > > /Claes > >> Cheers, >> -- Jim >>> On Jun 16, 2020, at 10:00 AM, Claes Redestad wrote: >>> >>> Hi, >>> >>> this patch specializes bootstrapping of unary concatenation expressions. >>> >>> Such expressions can be reduced to the canonical String.valueOf >>> "stringifier" for any primitive argument, but needs a special >>> stringifier for reference arguments since we need to produce a new >>> String to be compliant with JLS. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8247681 >>> Webrev: http://cr.openjdk.java.net/~redestad/8247681/open.00/ >>> >>> This ensures we get a similar speed-up as JDK-8247605 for >>> "" + fooString (and fooString + ""). >>> >>> Also speeds up bootstrapping for all such simple unary concatenation >>> expressions. >>> >>> Testing: tier1+2 >>> >>> Before: >>> >>> Benchmark (intValue) Mode Cnt Score Error Units >>> StringConcat.concatEmptyConstInt 4711 avgt 5 15.539 ? 0.831 ns/op >>> StringConcat.concatEmptyConstString 4711 avgt 5 17.046 ? 1.047 ns/op >>> StringConcat.concatEmptyLeft 4711 avgt 5 7.506 ? 0.588 ns/op >>> StringConcat.concatEmptyRight 4711 avgt 5 7.890 ? 0.314 ns/op >>> >>> After: >>> Benchmark (intValue) Mode Cnt Score Error Units >>> StringConcat.concatEmptyConstInt 4711 avgt 5 15.410 ? 0.944 ns/op >>> StringConcat.concatEmptyConstString 4711 avgt 5 7.397 ? 0.384 ns/op >>> StringConcat.concatEmptyLeft 4711 avgt 5 7.465 ? 0.328 ns/op >>> StringConcat.concatEmptyRight 4711 avgt 5 7.857 ? 0.355 ns/op >>> >>> Thanks! >>> >>> /Claes From magnus.ihse.bursie at oracle.com Tue Jun 16 14:03:28 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 16 Jun 2020 16:03:28 +0200 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: References: <39df1e40-f34a-ba3b-e2e2-f462a6442c42@oracle.com> <214B2701-0723-45A5-81B5-816EF1D86470@oracle.com> Message-ID: On 2020-06-16 15:06, Erik Joelsson wrote: > > > On 2020-06-15 17:39, Igor Ignatyev wrote: >> @David, Erik, Magnus, >> >> please find the answers to your comments at the bottom of this email. >> >> @all, >> >> David's and Erik's comments made me realize that some parts of the >> original patch were stashed away and didn't make it to webrev.00. I'm >> truly sorry for the confusion and inconvenience. I also agree w/ >> David that it's hard to follow/review, and my?gaffe just made it >> worse, therefore I'd like to start it over again from a clean sate >> >> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 > > Build changes look good to me now, except for a missing newline in > Main.gmk. (No need for new review.) > Ditto. /Magnus > > /Erik > >>> 833 lines changed: 228 ins; 591 del; 14 mod; >> >> could you please review the patch which puts all tests for common >> testlibrary classes (which is ones located in /test/lib) into one >> location under /test/lib-test? please note this intentionally doesn't >> move all "testlibrary" tests, e.g. tests for CTW, hotspot-specific >> testlibrary, are left in /test/hotspot/jtreg/testlibrary_tests/ctw . >> >> to simplify review, the patch has been split into several webrevs, >> with each of them accomplishes a more-or-less distinct thing, but is >> not necessary self-contained: >> >> 0. trivial move of tests from test/jdk and test/hotspot/jtreg test >> suites to test/lib-test: >> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.00-move/ >> >> 1. removal of hotspot's AssertsTest,?OutputAnalyzerReportingTest and >> "merge" of hotspot's and jdk's OutputAnalyzerTest: >> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.01-merge/ >> >> 2. simplification of TestNativeProcessBuilder tests: converts Test >> class used by TestNativeProcessBuilder into a static nested class: >> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.02-TestNativeProcessBuilder >> >> 3. makefiles changes to build native parts of lib-test tests. in >> past, there were no tests w/ native parts in this test suite, >> TestNativeProcessBuilder is the 1st such test; this part also removes >> now unneeded lines from hotspot test suite makefile: >> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.03-NativeLibTest >> >> 4. clean ups in hotspot test suite: adjusted location >> of?SimpleClassFileLoadHookTest test, which is a test for >> hotspot-specific test library (located in >> /test/hotspot/jtreg/testlibrary/jvmti) and hence should not be moved >> to /test/lib-test; removed TestMutuallyExclusivePlatformPredicates >> from TEST.groups: >> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.04-hotspot/ >> >> 5. LingeredAppTest fix (which apparently has never been run before): >> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.05-LingeredAppTest/ >> >> 6. changes to make test/lib-test a fully supported and working test >> suite, and add in into tier1, ?includes adding ProblemList, >> TEST.groups file, and 'randomness' k/w: >> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.06-TESTROOT/ >> >> webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 >> JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 >> testing: >> ?- make test TEST=tier1 locally on macosx >> ?- test/lib-test on ?{windows,linux,macosx}-x64 >> ?- tier1 job (in progress) >> >> Thanks, >> -- Igor >> >> >>> On Jun 14, 2020, at 11:23 PM, David Holmes >> > wrote: >>> <...> >>> This doesn't seem to move everything under >>> test/hotspot/jtreg/testlibrary_tests/ e.g. ctw directory. >> this is intentionally, ctw test-library is hotspot specific, hence >> its tests are to reside in hotspot test suite (until we decide to >> move the library to /test/lib), the same is true >> for?SimpleClassFileLoadHookTest. >>> >>> You did not modify hotspot/jtreg/TEST.groups but it refers to: >>> testlibrary_tests/TestMutuallyExclusivePlatformPredicates.java \ >>> Plus it should probably refer to the new native_sanity tests somewhere. >> the actual version of the patch removed reference to >> TestMutuallyExclusivePlatformPredicates from hotspot's TEST.groups >> and had TestNativeProcessBuilder moved to /test/test-lib, so no >> updates w.r.t. native_sanity are needed >> >>> The newly added copyright notice needs to have the correct initial >>> copyright year based on when the file was first added to the repo. >> ?/test/lib-test/TEST.ROOT file was created by JDK-8243010 on >> 2020-04-16, hence the added copyright has 2020 as the initial >> copyright year. >> >>> You used the wrong license header - makefiles don't use the >>> classpath exception. >> JtregNativeLibTest.gmk is based on?make/test/JtregNativeJdk.gmk which >> also have classpath exception. quick grep showed that make directory >> has 794 files which has '"Classpath" exception', out of 850 which >> contain 'GNU General Public License version 2' string. And although I >> agree that makefiles shouldn't have the classpath exception, I'd >> prefer to keep JtregNativeLibTest.gmk in sync w/ the its siblings and >> would leave it up to build team to decide how it's best to handle. >> >> >>> On Jun 15, 2020, at 8:12 AM, Magnus Ihse Bursie >>> >> > wrote: >>> >>> A few comments: >>> >>> This seems like code copied from elsewhere: >>> 57 # This evaluation is expensive and should only be done if this target was >>> 58 # explicitly called. >>> 59 ifneq ($(filter build-test-libtest-jtreg-native, $(MAKECMDGOALS)), ) >>> I don't agree that this is an expensive evaluation. Furthermore, the >>> makefile is only called for building the testlib and for making >>> images, so in worst case it's just the image part that would get a >>> penalty (although I highly doubt there is any). >> right, JtregNativeLibTest.gmk is based >> on?make/test/JtregNativeJdk.gmk which has this comment, I don't know >> enough details to say if it's expensive or not, yet if you insist and >> there is a consensus within build team, I can remove the comment. >> >>> >>> 82 $(eval $(call SetupCopyFiles,COPY_LIBTEST_JTREG_NATIVE, \ >>> Please use space after comma. >> this again was preexisting in?JtregNativeJdk.gmk, added the space >> nevertheless. >> >> >>> On Jun 15, 2020, at 6:16 AM, Erik Joelsson >> > wrote: >>> >>> In JtretNativeLibTest.gmk, lines 51-55 should probably be removed >>> (or adjusted if linking to libjvm is actually needed). >> the lines were updated to define >> BUILD_LIBTEST_JTREG_EXECUTABLES_LIBS_exejvm-test-launcher >> >> From Roger.Riggs at oracle.com Tue Jun 16 14:13:47 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Tue, 16 Jun 2020 10:13:47 -0400 Subject: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) In-Reply-To: <3B38456A-6BD0-4B95-9F9C-E73F7FE3A78F@amperemail.onmicrosoft.com> References: <814DA850-76A9-4526-A3D9-141CD516C9AE@amperemail.onmicrosoft.com> <50955EDB-CCC2-4334-A4AF-2EB422E9CAA9@amperemail.onmicrosoft.com> <3B38456A-6BD0-4B95-9F9C-E73F7FE3A78F@amperemail.onmicrosoft.com> Message-ID: Hi Peter, I think you've hardwired an assumption about the contents of the stack into VM.latestUserDefineLoader... On 6/12/20 7:19 PM, Peter Kessler (Open Source) wrote: > Roger, > > I did think about confining the changes to ObjectInputStream. Maybe I > did not think hard enough about it. > > Keeping a cache of the result of a first stack walk might work for the > recursive calls. One might still do a useless stack walk for each > top-level readObject call. No concurrent class loading can invalidate > the cache for any particular stack walk. But it seems fraught to > reuse a cached value in the face of concurrent class loading, or in > the presence of overloaded ObjectInputStream methods that might create > new ClassLoaders on this stack. I'm sure I don't want to debug that. > > My experiments have shown that the average stack walk is about 20 > frames. I do not have the data on how many of those frames are > recursive calls within ObjectInputStream methods and how many are in > other code. I'd be interested in a histogram of the depths and in which applications. > Dragging in the StackWalker API would further complicate things, in my > opinion. It might appear to keep the code out of platform native > code, but we know that walking thread stacks involves native code. > > I settled on a system-wide one-bit cache: There have been no > user-defined ClassLoaders constructed, so I can not find one on any > particular stack walk. It may not be the best possible solution, but > it addresses the issue of useless stack walks in the common case where > there are no user-defined ClassLoaders. > > I agree that both ClassLoader and ObjectInputStream are complex. In > fact, the proposed patch makes no changes to ObjectInputStream. There > is one line added to each of the system-defined ClassLoaders to > identify them as not user-defined ClassLoaders, one line in the base > ClassLoader constructor, and the bulk of the change is around the > guard in jdk.internal.misc.VM.latestUserDefinedLoader() to avoid the > stack walk if possible. The spec and implementation of OIS.resolveClass do not make any assumptions about whether serialization is invoked from a class on the PlatformLoader, AppClassLoader, or another loader. In the case of a 'system' thread or class calling OIS.readObject, the change you are proposing will force class loading in resolveClass to the AppClassLoader. In VM.latestUserDefinedLoader:390 caught my eye because it forces the AppClassLoader. It assumes that scanning the stack would find and return the AppClassLoader, not so in all cases. Regards, Roger > > ... peter > > ?-----Original Message----- > From: Roger Riggs > Organization: Java Products Group, Oracle > Date: Friday, June 12, 2020 at 11:50 AM > To: "Peter Kessler (Open Source)" , "core-libs-dev at openjdk.java.net" > Subject: Re: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) > > Hi Peter, > > The hazard to avoid is the cross package coupling that makes both > ClassLoader and > ObjectInputStream more complex; both are more than sufficiently complex > already. > > Optimizing the implementations make sense if it benefits enough uses > cases and > does not make the code harder to maintain and evolve. > > Have you explored any alternatives that can be confined to OIS? > For example, taking advantage the many calls to readObject within > a single OIS that are nested. Possibly the first (depth = 0) find user > class loader > can be re-used or at least to short circuit the search on subsequent > nested calls? > Or caching the classloader at each depth and only need to probe for it > if it has > not already been recorded. The StackWalker API may be useful to limit > the depth > of searches. > > Thanks, Roger > > > > On 6/11/20 2:52 PM, Peter Kessler OS wrote: > > Hi Roger, > > > > Thanks for your comments. > > > > I am not knowingly changing the required behavior. All I am proposing is > > guarding the thread stack walk if it will be a waste of effort. If an > > application or framework has its own ClassLoaders, then the guard will fail > > and the thread stack will be walked. > > > > The stack walk is important, if there are user-defined ClassLoaders. A > > method that uses ObjectInputStream to read objects can not know what > > ClassLoader to use to resolve class references in the stream, without the > > stack walk. But if there never have been any user-defined ClassLoaders > > constructed, then the stack walk can never return a user-defined > > ClassLoader. > > > > If the stream reader is part of some simple application, then it may spend > > a significant amount of its time uselessly walking thread stacks. (If the > > stream reader is part of some simple application its author may not know to > > override ObjectInputStream.resolveClass, or how to determine what > > ClassLoader should load a class from the stream.) If someone were using > > such an application to choose platforms, the Java platform would be at a > > disadvantage. > > > > Consider a somewhat more complex case, the SPECjbb2015 benchmark, which > > claims to model a well-constructed business application on the Java > > platform. SPECjbb2015 runs multiple JVMs, exchanging information via > > ObjectInputStreams. Profiling shows that it spends more than 5% of its time > > walking thread stacks. Yet, in spite of being a complex application, > > written by experts in the Java platform, there are no user-defined > > ClassLoaders. So all the time spent walking stacks is wasted time, and is > > one of the reasons the servers fail their service-level agreements. We can > > discuss whether SPECjbb2015 is representative of Java applications. But it > > is easily available and relatively simple to run. People use benchmarks > > like SPECjbb2015 to decide whether to use the Java platform for particular > > tasks, and it would be unfortunate if we did not make the Java platform as > > good as it could be. > > > > ... peter > > > > -----Original Message----- > > From: core-libs-dev on behalf of Roger Riggs > > Organization: Java Products Group, Oracle > > Date: Tuesday, June 9, 2020 at 6:43 AM > > To: "core-libs-dev at openjdk.java.net" > > Subject: Re: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) > > > > Hi Peter, > > > > I'd like to understand the scope and impact of the problem before > > jumping to a solution. > > For specific applications, overriding ObjectInputStream.resolveClass to > > handle the class > > lookup can handle the case as expeditiously as is necessary. > > > > What application uses cases are impacted and how prevelant are they? > > Many, many frameworks and applications create classloaders for their own > > purposes and > > take advantage of the required behavior. > > > > Thanks, Roger > > > > > > On 6/4/20 7:08 PM, Peter Kessler OS wrote: > > > ObjectInputStream.resolveClass(ObjectStreamClass) is specified to find "the first class loader on the current thread's stack (starting from the currently executing method) that is neither the platform class loader nor its ancestor." Finding that class loader is done with a virtual frame walk in JVM_LatestUserDefinedLoader(JNIEnv *env). The virtual frame walk is expensive. If an application does not define any ClassLoaders, then the virtual frame walk will always find the system class loader. > > > > > > If an application does a significant amount of reading from an ObjectInputStream, the cost of ObjectInputStream.resolveClass(ObjectInputStreamClass) can be considerable. In a sample application, that method (and the methods it calls) have been measured to consume 5+% of the CPU cycles. > > > > > > The proposal is to note the construction of any user-defined ClassLoader, and if none has been constructed, avoid the virtual frame walk returning ClassLoader.getSystemClassLoader(). But if any user-defined ClassLoader has been constructed, use the virtual frame walk to find the first user-defined ClassLoader on the thread stack. > > > > > > User-defined ClassLoaders could be distinguished from non-user-defined ClassLoaders in several ways. The ClassLoader class hierarchy could be changed to make non-user-defined ClassLoaders inherit from a marker class. There is already a jdk.internal.loader.BuiltinClassLoader but it is used for module (and resource) loading. BuiltinClassLoader is not a super-class of jdk.internal.reflect.DelegatingClassLoader, which is used to load reflection classes, and reflection methods are also ignored by the virtual frame walk. ClassLoaders could have a boolean that distinguished whether they are user-defined or not. That would require a change to ClassLoader API. There are other ways to distinguish user-defined ClassLoaders. The proposal is to keep the decoration of the ClassLoaders out of the instances, so that it is only visible to the code that needs it. > > > > > > The proposal is that each non-user-defined ClassLoader have a static block that registers the class as a non-user-defined ClassLoader. Then in the base ClassLoader constructor check if the ClassLoader being constructed is a user-defined ClassLoader and if so set a boolean that acts as the guard on the virtual frame walk. If additional non-user-defined ClassLoaders are declared in the future beyond the ones identified in this patch, they can be added to the registry. > > > > > > There are currently 4 non-user-defined ClassLoader classes, so the registry is not difficult to maintain. Nor is the registry difficult to search when ClassLoaders are constructed. The guard boolean that records whether a user-defined ClassLoader has been constructed transitions from false to true and never becomes false again. (An extension to transition the boolean back to false when a user-defined ClassLoader becomes unreachable is beyond the scope of this proposal.) > > > > > > This proposal slows loading of non-user-defined ClassLoaders by the time it takes to register them. It slows ClassLoader construction by the time it takes to consult the registry and conditionally set the guard boolean. At each invocation of ObjectInputStream.resolveClass(ObjectInputStreamClass), the guard boolean is used to possibly avoid the virtual frame walk. > > > > > > Tested with `make run-test-tier1` on Linux (CentOS 7) on both aarch64 and x86_64. I had one failure on each of the runs > > > > > > ACTION: main -- Failed. Execution failed: `main' threw exception: java.lang.Error: TESTBUG: the library has not been found in ${Repo}/build/linux-aarch64-server-release/images/test/hotspot/jtreg/native > > > REASON: User specified action: run main/native GTestWrapper > > > > > > on both plaforms. But I get the same failure when I run the tier1 tests on clones of OpenJDK-15+25 on both aarch64 and x86_64. > > > > > > Running a sample (representative?) application with the Oracle Developer Studio analyzer shows a performance comparison of > > > > > > Stack Fragment sorted by metric: Attributed Total CPU Time > > > baseline.er experiment.er > > > Attributed Attributed Name > > > Total CPU Time Total CPU Time > > > sec. sec. > > > ================================== Callers > > > 3459.210 29.931 java.io.ObjectInputStream.readOrdinaryObject(boolean) > > > 1139.727 3.532 java.io.ObjectInputStream.readArray(boolean) > > > 875.262 9.116 java.io.ObjectInputStream.readNonProxyDesc(boolean) > > > > > > ================================== Stack Fragment > > > java.io.ObjectInputStream.readClassDesc(boolean) > > > java.io.ObjectInputStream.readNonProxyDesc(boolean) > > > java.io.ObjectInputStream.resolveClass(java.io.ObjectStreamClass) > > > java.io.ObjectInputStream.latestUserDefinedLoader() > > > 4.173 3.953 jdk.internal.misc.VM.latestUserDefinedLoader() > > > > > > ================================== Callees > > > 5470.026 0. jdk.internal.misc.VM.latestUserDefinedLoader0() > > > 0. 38.627 java.lang.ClassLoader.getSystemClassLoader() > > > > > > The `hg export -g` below is with respect to OpenJDK-15+25. > > > > > > Thank you for your review comments. I will need a sponsor to get this change into the repository. > > > > > > ... peter > > > > > > # HG changeset patch > > > # User Peter Kessler > > > # Date 1591310467 25200 > > > # Thu Jun 04 15:41:07 2020 -0700 > > > # Node ID 9a39488182c1dfc5bc7bb41d562d7ef16ee657f6 > > > # Parent 90b266a84c06f1b3dc0ed8767856793e8c1c357e > > > Improve the performance of ObjectInputStream.resolveClass > > > > > > diff --git a/src/java.base/share/classes/java/lang/ClassLoader.java b/src/java.base/share/classes/java/lang/ClassLoader.java > > > --- a/src/java.base/share/classes/java/lang/ClassLoader.java > > > +++ b/src/java.base/share/classes/java/lang/ClassLoader.java > > > @@ -385,6 +385,8 @@ > > > } > > > this.package2certs = new ConcurrentHashMap<>(); > > > this.nameAndId = nameAndId(this); > > > + /* Note the construction of a ClassLoader. */ > > > + VM.noteClassLoaderConstruction(this.getClass()); > > > } > > > > > > /** > > > diff --git a/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java b/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java > > > --- a/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java > > > +++ b/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java > > > @@ -117,6 +117,11 @@ > > > protected Class loadClassOrNull(String cn, boolean resolve) { > > > return JLA.findBootstrapClassOrNull(this, cn); > > > } > > > + > > > + static { > > > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > > > + VM.registerNotUserDefinedClassLoader(BootClassLoader.class); > > > + } > > > }; > > > > > > /** > > > @@ -127,6 +132,8 @@ > > > static { > > > if (!ClassLoader.registerAsParallelCapable()) > > > throw new InternalError(); > > > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > > > + VM.registerNotUserDefinedClassLoader(PlatformClassLoader.class); > > > } > > > > > > PlatformClassLoader(BootClassLoader parent) { > > > @@ -142,6 +149,8 @@ > > > static { > > > if (!ClassLoader.registerAsParallelCapable()) > > > throw new InternalError(); > > > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > > > + VM.registerNotUserDefinedClassLoader(AppClassLoader.class); > > > } > > > > > > final URLClassPath ucp; > > > diff --git a/src/java.base/share/classes/jdk/internal/misc/VM.java b/src/java.base/share/classes/jdk/internal/misc/VM.java > > > --- a/src/java.base/share/classes/jdk/internal/misc/VM.java > > > +++ b/src/java.base/share/classes/jdk/internal/misc/VM.java > > > @@ -304,12 +304,44 @@ > > > private static final int JVMTI_THREAD_STATE_WAITING_INDEFINITELY = 0x0010; > > > private static final int JVMTI_THREAD_STATE_WAITING_WITH_TIMEOUT = 0x0020; > > > > > > + /** A registry of not user-defined ClassLoaders. */ > > > + private static final List> notUserDefinedClassLoaderRegistry = > > > + Collections.synchronizedList(new ArrayList<>()); > > > + > > > + /** Register a ClassLoader class as being not a user-defined ClassLoader. */ > > > + public static void registerNotUserDefinedClassLoader(Class classLoaderClass) { > > > + notUserDefinedClassLoaderRegistry.add(classLoaderClass); > > > + } > > > + > > > + /** A flag for whether a user-defined ClassLoaders has been constructed. */ > > > + private static volatile boolean constructedUserDefinedClassLoaderFlag = false; > > > + > > > + /** > > > + * Note a ClassLoader construction, and record if a > > > + * user-defined ClassLoader has been constructed. > > > + */ > > > + public static void noteClassLoaderConstruction(Class classLoaderClass) { > > > + /* Check if the ClassLoader class not a user-defined ClassLoader. */ > > > + if (notUserDefinedClassLoaderRegistry.contains(classLoaderClass)) { > > > + return; > > > + } > > > + constructedUserDefinedClassLoaderFlag = true; > > > + return; > > > + } > > > + > > > /* > > > * Returns the first user-defined class loader up the execution stack, > > > * or the platform class loader if only code from the platform or > > > * bootstrap class loader is on the stack. > > > */ > > > public static ClassLoader latestUserDefinedLoader() { > > > + if (!constructedUserDefinedClassLoaderFlag) { > > > + /* > > > + * If no user-defined ClassLoader has been constructed, > > > + * then I will not find a user-defined ClassLoader in the stack. > > > + */ > > > + return ClassLoader.getSystemClassLoader(); > > > + } > > > ClassLoader loader = latestUserDefinedLoader0(); > > > return loader != null ? loader : ClassLoader.getPlatformClassLoader(); > > > } > > > diff --git a/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java b/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java > > > --- a/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java > > > +++ b/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java > > > @@ -30,6 +30,7 @@ > > > > > > import jdk.internal.access.JavaLangAccess; > > > import jdk.internal.access.SharedSecrets; > > > +import jdk.internal.misc.VM; > > > > > > /** Utility class which assists in calling defineClass() by > > > creating a new class loader which delegates to the one needed in > > > @@ -73,4 +74,9 @@ > > > DelegatingClassLoader(ClassLoader parent) { > > > super(parent); > > > } > > > + > > > + static { > > > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > > > + VM.registerNotUserDefinedClassLoader(DelegatingClassLoader.class); > > > + } > > > } > > > > > > > > > From chris.hegarty at oracle.com Tue Jun 16 15:15:03 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 16 Jun 2020 16:15:03 +0100 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> Message-ID: <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> Hi Peter, > On 14 Jun 2020, at 17:28, Peter Levart wrote: > > ... > > [2] http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.01/ > I think that this is very good. It?s clever to build a chain of method handles that can be invoked with the stream field values. This is a nice separation between the shape of the data and the actual stream data. The caching is on a per-stream-field shape basis, which should be consistent in the common case, but of course is not always guaranteed to be the case, hence the need for the cache. I think that this should be fine, since the ObjectStreamClass ( that holds the cache ) is already itself cached as a weak reference. But I did wonder if the size of this new cache should be limited. Probably not worth the complexity unless it is an obvious issue. All the serailizable records tests pass successfully with your patch. Good. I did however notice that there is just a single test, DefaultValuesTest, that exercises different stream shapes for the same class in the stream. Would be good to expand coverage in this area, but of course some lower-level test-specific building blocks will be needed help build the specially crafted byte streams - I can help with this. Overall I think that this change is good. -Chris. From paul.sandoz at oracle.com Tue Jun 16 15:50:32 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 16 Jun 2020 08:50:32 -0700 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> Message-ID: Hi Peter, 199 private Map deserializationCtrs; Change to be either ConcurrentMap or ConcurrentHashMap, thereby making it clearer when using. 2679 private static MethodHandle defaultValueExtractorFor(Class pType) { Can the implementation use MethodHandles,zero? e.g. return MethodHandles.dropArguments(MethodHandles.zero(pType), 0, byte[].class, Object[].class); Paul. > On Jun 16, 2020, at 8:15 AM, Chris Hegarty wrote: > > Hi Peter, > >> On 14 Jun 2020, at 17:28, Peter Levart wrote: >> >> ... >> >> [2] http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.01/ >> > I think that this is very good. It?s clever to build a chain of method handles that can be invoked with the stream field values. This is a nice separation between the shape of the data and the actual stream data. > > The caching is on a per-stream-field shape basis, which should be consistent in the common case, but of course is not always guaranteed to be the case, hence the need for the cache. I think that this should be fine, since the ObjectStreamClass ( that holds the cache ) is already itself cached as a weak reference. But I did wonder if the size of this new cache should be limited. Probably not worth the complexity unless it is an obvious issue. > > All the serailizable records tests pass successfully with your patch. Good. I did however notice that there is just a single test, DefaultValuesTest, that exercises different stream shapes for the same class in the stream. Would be good to expand coverage in this area, but of course some lower-level test-specific building blocks will be needed help build the specially crafted byte streams - I can help with this. > > Overall I think that this change is good. > > -Chris. > From paul.sandoz at oracle.com Tue Jun 16 16:06:54 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 16 Jun 2020 09:06:54 -0700 Subject: RFR[16]: 8247681: Improve bootstrapping of unary concatenations In-Reply-To: <10b4c312-3223-3412-016c-60515581733d@oracle.com> References: <10b4c312-3223-3412-016c-60515581733d@oracle.com> Message-ID: Looks good. It?s tempting to do something like this to de-dup the code with less potential for mistakes: String s = null; // Select the singular non-null value, if any If (s0 != null && s1 == null) s = s0; else if (s0 == null && s1 != null) s = s1; If (s != null) { if (s.isEmpty()) { ? } else if (?) { ? } } Paul. > On Jun 16, 2020, at 6:00 AM, Claes Redestad wrote: > > Hi, > > this patch specializes bootstrapping of unary concatenation expressions. > > Such expressions can be reduced to the canonical String.valueOf > "stringifier" for any primitive argument, but needs a special > stringifier for reference arguments since we need to produce a new > String to be compliant with JLS. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8247681 > Webrev: http://cr.openjdk.java.net/~redestad/8247681/open.00/ > > This ensures we get a similar speed-up as JDK-8247605 for > "" + fooString (and fooString + ""). > > Also speeds up bootstrapping for all such simple unary concatenation > expressions. > > Testing: tier1+2 > > Before: > > Benchmark (intValue) Mode Cnt Score Error Units > StringConcat.concatEmptyConstInt 4711 avgt 5 15.539 ? 0.831 ns/op > StringConcat.concatEmptyConstString 4711 avgt 5 17.046 ? 1.047 ns/op > StringConcat.concatEmptyLeft 4711 avgt 5 7.506 ? 0.588 ns/op > StringConcat.concatEmptyRight 4711 avgt 5 7.890 ? 0.314 ns/op > > After: > Benchmark (intValue) Mode Cnt Score Error Units > StringConcat.concatEmptyConstInt 4711 avgt 5 15.410 ? 0.944 ns/op > StringConcat.concatEmptyConstString 4711 avgt 5 7.397 ? 0.384 ns/op > StringConcat.concatEmptyLeft 4711 avgt 5 7.465 ? 0.328 ns/op > StringConcat.concatEmptyRight 4711 avgt 5 7.857 ? 0.355 ns/op > > Thanks! > > /Claes From mandy.chung at oracle.com Tue Jun 16 16:42:45 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 16 Jun 2020 09:42:45 -0700 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: <1229321025.1095655.1592300959835.JavaMail.zimbra@u-pem.fr> References: <1229321025.1095655.1592300959835.JavaMail.zimbra@u-pem.fr> Message-ID: <97446f83-1e00-13bd-14f2-d32620ff072f@oracle.com> On 6/16/20 2:49 AM, Remi Forax wrote: > Hi Mndy, > Looks good, > I don't like too much the fact that there is a new boolean field in j.l.r.Field instead of using the modifiers like in the VM counterpart, > but it will require masking and unmasking the modifiers which is a far bigger change, too big to worth it in my opinion. Adding a new field instead of using the modifiers is a deliberate choice from my side.? It's higher risk to overload the modifiers to include VM-specific flags.?? I am all for a better clean up (valhalla lworld does use the modifiers to cover a flag indicating if it's flattened which I am not satisfied either). > So +1 Thanks Remi. Mandy > R?mi > > ----- Mail original ----- >> De: "mandy chung" >> ?: "hotspot-dev" , "core-libs-dev" , "amber-dev" >> >> Envoy?: Lundi 15 Juin 2020 23:58:19 >> Objet: (15) RFR: JDK-8247444: Trust final fields in records >> This patch is joint contribution from Christoph Dreis [1] and me. >> >> Webrev: http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247444/webrev.00/ >> CSR: https://bugs.openjdk.java.net/browse/JDK-8247517 >> >> This proposes to make final fields in records notmodifiable via >> reflection.? Field::set throws IAE if a Field is not modifiable. >> Thecurrent specification specifies the following Fields not modifiable: >> - static final fields in any class >> - final fields in a hidden class >> >> The spec of Field::set is changed to specify that records are not >> modifiable via reflection. >> ?Noe that no change in Field::setAccessible(true), i.e. it will succeed >> to allow existing frameworks to have read access to final fields in >> records.? Just no write access. >> >> VarHandle does not support write access if it's a static final field or >> an instance final field. >> >> This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and >> `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. >> >> No change is made in JNI.? JNI could be considered to disallow >> modification of final fields in records and hidden classes (static and >> instance fields).? But JNI has superpower and the current spec already >> allows to modify the value of a final static field even after it's >> constant folded (via JNI SetField and SetStaticField), then >> all bets are off.? This should be re-visited when we consider "final is >> truly final" for all classes. >> >> Make final fields in records not modifiable via reflection enables JIT >> optimization as these final fields are effectively truly final. >> >> This change impacts 3rd-party frameworks including 3rd-party >> serialization framework that rely on core reflection `setAccessible` or >> `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to >> construct records but not using the canonical constructor. >> These frameworks would need to be updated to construct records via its >> canonical constructor as done by the Java serialization. >> >> I see this change gives a good opportunity to engage the maintainers of >> the serialization frameworks and work together to support new features >> including records, inline classes and the new serialization mechanism >> and which I think it is worth the investment. >> >> This is a low risk enhancement.? I'd like to request approval for a late >> enhancement in JDK 15.? It extends the pre-existing code path with >> refactoring the hidden classes to prepare for new kinds of classes with >> trusted final fields.? The change is straight-forward. >> >> Can this wait to integrate in JDK 16? >> >> ? It's important to get this enhancement in when record is a preview >> feature that we can get feedback and give time to 3rd-party >> serialization frameworks to look into adding the support for records. >> If we delayed this change in 16 and records exit preview, it would be >> bad for frameworks if they verify that they support records in 15 but >> fail in 16.? OTOH the risk of this patch is low. >> >> Performance Impact >> >> I addressed the performance concern I raised earlier myself.? For >> reflection, VM creates the reflective Field objects and fills in >> MemberName when resolving a member.? VM will tag if this >> Field/MemberName is trusted final field.? I think this is a cleaner >> approach rather than in each place to check for final static and final >> fields in hidden or record class to determine if it has write access or >> not. >> >> `sun.misc.Unsafe` does not use Field::isTrustedFinalField because (1) >> Unsafe has been allowing access of static final fields of any classes >> but isTrustedFinalField is not limited to instance fields (2) Unsafe >> disallows access to all fields in a hidden class (not limited to trusted >> final fields).? So it follows the precedence and simply checks if the >> declaring class is a record. `Class::isRecord` calls >> `Class::getSuperclass` to check if it's a subtype of `Record`.? As >> `Class::getSuperclass` is intrinsified, the call on isRecord on a normal >> class is fast. Christoph has contributed the microbenchmarks that >> confirm that no performance regression. >> >> Thanks >> Mandy >> [1] >> https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-June/040096.html From igor.ignatyev at oracle.com Tue Jun 16 16:45:20 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 16 Jun 2020 09:45:20 -0700 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: References: <39df1e40-f34a-ba3b-e2e2-f462a6442c42@oracle.com> <214B2701-0723-45A5-81B5-816EF1D86470@oracle.com> Message-ID: <287C88A1-3464-484E-AF66-7822394CA125@oracle.com> Hi David, thanks for your review. re: LingeredAppTest, I agree that the test doesn't look very useful, yet I'd remind that the goal of this (and other test in /test/lib-test) is to (sanity) test testlibrary in order to easily spot bugs in testlibrary in a clear manner so one would not have to investigate failures of actual "product" tests and go thru their sometimes convoluted logic just to find out that the problem was in LingeredApp. w/ that being said, this test can do a better job in testing LingeredApp, so I'll file a JBS ticket against hotspo/svc to evaluate/improve/discuss the fate of the test. Thanks, -- Igor > On Jun 16, 2020, at 12:14 AM, David Holmes wrote: > > Hi Igor, > > On 16/06/2020 10:39 am, Igor Ignatyev wrote: >> @David, Erik, Magnus, >> please find the answers to your comments at the bottom of this email. >> @all, >> David's and Erik's comments made me realize that some parts of the original patch were stashed away and didn't make it to webrev.00. I'm truly sorry for the confusion and inconvenience. I also agree w/ David that it's hard to follow/review, and my gaffe just made it worse, therefore I'd like to start it over again from a clean sate >> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 >>> 833 lines changed: 228 ins; 591 del; 14 mod; >> could you please review the patch which puts all tests for common testlibrary classes (which is ones located in /test/lib) into one location under /test/lib-test? please note this intentionally doesn't move all "testlibrary" tests, e.g. tests for CTW, hotspot-specific testlibrary, are left in /test/hotspot/jtreg/testlibrary_tests/ctw . > > Ok. > >> to simplify review, the patch has been split into several webrevs, with each of them accomplishes a more-or-less distinct thing, but is not necessary self-contained: > > Many thanks for doing this! > >> 0. trivial move of tests from test/jdk and test/hotspot/jtreg test suites to test/lib-test: >> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.00-move/ > > Looks good. > >> 1. removal of hotspot's AssertsTest, OutputAnalyzerReportingTest and "merge" of hotspot's and jdk's OutputAnalyzerTest: >> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.01-merge/ > > Looks good. > >> 2. simplification of TestNativeProcessBuilder tests: converts Test class used by TestNativeProcessBuilder into a static nested class: >> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.02-TestNativeProcessBuilder > > Looks good. > >> 3. makefiles changes to build native parts of lib-test tests. in past, there were no tests w/ native parts in this test suite, TestNativeProcessBuilder is the 1st such test; this part also removes now unneeded lines from hotspot test suite makefile: >> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.03-NativeLibTest > > Hmm okay. Not sure this needed its own category of build logic but ... > > Aside: Makefiles should not have the classpath exception version of the license header. But they all do for some reason. I'll check if we need to do a separate cleanup of that. > >> 4. clean ups in hotspot test suite: adjusted location of SimpleClassFileLoadHookTest test, which is a test for hotspot-specific test library (located in /test/hotspot/jtreg/testlibrary/jvmti) and hence should not be moved to /test/lib-test; removed TestMutuallyExclusivePlatformPredicates from TEST.groups: >> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.04-hotspot/ > > Looks good. Took me a while to understand the connection to the test library here. > >> 5. LingeredAppTest fix (which apparently has never been run before): >> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.05-LingeredAppTest/ > > Someone from serviceability should evaluate this test. It may not be needed. > >> 6. changes to make test/lib-test a fully supported and working test suite, and add in into tier1, includes adding ProblemList, TEST.groups file, and 'randomness' k/w: >> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.06-TESTROOT/ > > Seems okay. > > Thanks, > David > ----- > >> webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 >> JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 >> testing: >> - make test TEST=tier1 locally on macosx >> - test/lib-test on {windows,linux,macosx}-x64 >> - tier1 job (in progress) >> Thanks, >> -- Igor >>> On Jun 14, 2020, at 11:23 PM, David Holmes > wrote: >>> <...> >>> This doesn't seem to move everything under test/hotspot/jtreg/testlibrary_tests/ e.g. ctw directory. >> this is intentionally, ctw test-library is hotspot specific, hence its tests are to reside in hotspot test suite (until we decide to move the library to /test/lib), the same is true for SimpleClassFileLoadHookTest. >>> >>> You did not modify hotspot/jtreg/TEST.groups but it refers to: >>> testlibrary_tests/TestMutuallyExclusivePlatformPredicates.java \ >>> Plus it should probably refer to the new native_sanity tests somewhere. >> the actual version of the patch removed reference to TestMutuallyExclusivePlatformPredicates from hotspot's TEST.groups and had TestNativeProcessBuilder moved to /test/test-lib, so no updates w.r.t. native_sanity are needed >>> The newly added copyright notice needs to have the correct initial copyright year based on when the file was first added to the repo. >> /test/lib-test/TEST.ROOT file was created by JDK-8243010 on 2020-04-16, hence the added copyright has 2020 as the initial copyright year. >>> You used the wrong license header - makefiles don't use the classpath exception. >> JtregNativeLibTest.gmk is based on make/test/JtregNativeJdk.gmk which also have classpath exception. quick grep showed that make directory has 794 files which has '"Classpath" exception', out of 850 which contain 'GNU General Public License version 2' string. And although I agree that makefiles shouldn't have the classpath exception, I'd prefer to keep JtregNativeLibTest.gmk in sync w/ the its siblings and would leave it up to build team to decide how it's best to handle. >>> On Jun 15, 2020, at 8:12 AM, Magnus Ihse Bursie > wrote: >>> >>> A few comments: >>> >>> This seems like code copied from elsewhere: >>> 57 # This evaluation is expensive and should only be done if this target was >>> 58 # explicitly called. >>> 59 ifneq ($(filter build-test-libtest-jtreg-native, $(MAKECMDGOALS)), ) >>> I don't agree that this is an expensive evaluation. Furthermore, the makefile is only called for building the testlib and for making images, so in worst case it's just the image part that would get a penalty (although I highly doubt there is any). >> right, JtregNativeLibTest.gmk is based on make/test/JtregNativeJdk.gmk which has this comment, I don't know enough details to say if it's expensive or not, yet if you insist and there is a consensus within build team, I can remove the comment. >>> >>> 82 $(eval $(call SetupCopyFiles,COPY_LIBTEST_JTREG_NATIVE, \ >>> Please use space after comma. >> this again was preexisting in JtregNativeJdk.gmk, added the space nevertheless. >>> On Jun 15, 2020, at 6:16 AM, Erik Joelsson > wrote: >>> >>> In JtretNativeLibTest.gmk, lines 51-55 should probably be removed (or adjusted if linking to libjvm is actually needed). >> the lines were updated to define BUILD_LIBTEST_JTREG_EXECUTABLES_LIBS_exejvm-test-launcher From igor.ignatyev at oracle.com Tue Jun 16 16:52:25 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Tue, 16 Jun 2020 09:52:25 -0700 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: References: <39df1e40-f34a-ba3b-e2e2-f462a6442c42@oracle.com> <214B2701-0723-45A5-81B5-816EF1D86470@oracle.com> Message-ID: <9CD66F39-7EE0-441E-BE9C-14C3C5CF5317@oracle.com> Magnus, Erik, thanks for your reviews, pushed w/ a newline being added at L#654 of make/Main.gmk. Cheers, -- Igor > On Jun 16, 2020, at 7:03 AM, Magnus Ihse Bursie wrote: > > On 2020-06-16 15:06, Erik Joelsson wrote: >> >> On 2020-06-15 17:39, Igor Ignatyev wrote: >>> @David, Erik, Magnus, >>> >>> please find the answers to your comments at the bottom of this email. >>> >>> @all, >>> >>> David's and Erik's comments made me realize that some parts of the original patch were stashed away and didn't make it to webrev.00. I'm truly sorry for the confusion and inconvenience. I also agree w/ David that it's hard to follow/review, and my gaffe just made it worse, therefore I'd like to start it over again from a clean sate >>> >>> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 Build changes look good to me now, except for a missing newline in Main.gmk. (No need for new review.) >> > > Ditto. > > /Magnus >> >> /Erik >> >>>> 833 lines changed: 228 ins; 591 del; 14 mod; >>> >>> could you please review the patch which puts all tests for common testlibrary classes (which is ones located in /test/lib) into one location under /test/lib-test? please note this intentionally doesn't move all "testlibrary" tests, e.g. tests for CTW, hotspot-specific testlibrary, are left in /test/hotspot/jtreg/testlibrary_tests/ctw . >>> >>> to simplify review, the patch has been split into several webrevs, with each of them accomplishes a more-or-less distinct thing, but is not necessary self-contained: >>> >>> 0. trivial move of tests from test/jdk and test/hotspot/jtreg test suites to test/lib-test: >>> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.00-move/ >>> >>> 1. removal of hotspot's AssertsTest, OutputAnalyzerReportingTest and "merge" of hotspot's and jdk's OutputAnalyzerTest: >>> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.01-merge/ >>> >>> 2. simplification of TestNativeProcessBuilder tests: converts Test class used by TestNativeProcessBuilder into a static nested class: >>> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.02-TestNativeProcessBuilder >>> >>> 3. makefiles changes to build native parts of lib-test tests. in past, there were no tests w/ native parts in this test suite, TestNativeProcessBuilder is the 1st such test; this part also removes now unneeded lines from hotspot test suite makefile: >>> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.03-NativeLibTest >>> >>> 4. clean ups in hotspot test suite: adjusted location of SimpleClassFileLoadHookTest test, which is a test for hotspot-specific test library (located in /test/hotspot/jtreg/testlibrary/jvmti) and hence should not be moved to /test/lib-test; removed TestMutuallyExclusivePlatformPredicates from TEST.groups: >>> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.04-hotspot/ >>> >>> 5. LingeredAppTest fix (which apparently has never been run before): >>> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.05-LingeredAppTest/ >>> >>> 6. changes to make test/lib-test a fully supported and working test suite, and add in into tier1, includes adding ProblemList, TEST.groups file, and 'randomness' k/w: >>> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.06-TESTROOT/ >>> >>> webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 >>> testing: >>> - make test TEST=tier1 locally on macosx >>> - test/lib-test on {windows,linux,macosx}-x64 >>> - tier1 job (in progress) >>> >>> Thanks, >>> -- Igor >>> >>> >>>> On Jun 14, 2020, at 11:23 PM, David Holmes > wrote: >>>> <...> >>> >>>> This doesn't seem to move everything under test/hotspot/jtreg/testlibrary_tests/ e.g. ctw directory. >>> this is intentionally, ctw test-library is hotspot specific, hence its tests are to reside in hotspot test suite (until we decide to move the library to /test/lib), the same is true for SimpleClassFileLoadHookTest. >>>> >>>> You did not modify hotspot/jtreg/TEST.groups but it refers to: >>>> testlibrary_tests/TestMutuallyExclusivePlatformPredicates.java \ >>>> Plus it should probably refer to the new native_sanity tests somewhere. >>> the actual version of the patch removed reference to TestMutuallyExclusivePlatformPredicates from hotspot's TEST.groups and had TestNativeProcessBuilder moved to /test/test-lib, so no updates w.r.t. native_sanity are needed >>> >>>> The newly added copyright notice needs to have the correct initial copyright year based on when the file was first added to the repo. >>> >>> /test/lib-test/TEST.ROOT file was created by JDK-8243010 on 2020-04-16, hence the added copyright has 2020 as the initial copyright year. >>> >>>> You used the wrong license header - makefiles don't use the classpath exception. >>> JtregNativeLibTest.gmk is based on make/test/JtregNativeJdk.gmk which also have classpath exception. quick grep showed that make directory has 794 files which has '"Classpath" exception', out of 850 which contain 'GNU General Public License version 2' string. And although I agree that makefiles shouldn't have the classpath exception, I'd prefer to keep JtregNativeLibTest.gmk in sync w/ the its siblings and would leave it up to build team to decide how it's best to handle. >>> >>> >>>> On Jun 15, 2020, at 8:12 AM, Magnus Ihse Bursie > wrote: >>>> >>>> A few comments: >>>> >>>> This seems like code copied from elsewhere: >>>> 57 # This evaluation is expensive and should only be done if this target was >>>> 58 # explicitly called. >>>> 59 ifneq ($(filter build-test-libtest-jtreg-native, $(MAKECMDGOALS)), ) >>>> I don't agree that this is an expensive evaluation. Furthermore, the makefile is only called for building the testlib and for making images, so in worst case it's just the image part that would get a penalty (although I highly doubt there is any). >>> right, JtregNativeLibTest.gmk is based on make/test/JtregNativeJdk.gmk which has this comment, I don't know enough details to say if it's expensive or not, yet if you insist and there is a consensus within build team, I can remove the comment. >>> >>>> >>>> 82 $(eval $(call SetupCopyFiles,COPY_LIBTEST_JTREG_NATIVE, \ >>>> Please use space after comma. >>> this again was preexisting in JtregNativeJdk.gmk, added the space nevertheless. >>> >>> >>>> On Jun 15, 2020, at 6:16 AM, Erik Joelsson > wrote: >>>> >>>> In JtretNativeLibTest.gmk, lines 51-55 should probably be removed (or adjusted if linking to libjvm is actually needed). >>> the lines were updated to define BUILD_LIBTEST_JTREG_EXECUTABLES_LIBS_exejvm-test-launcher >>> >>> > From erik.joelsson at oracle.com Tue Jun 16 17:44:12 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Tue, 16 Jun 2020 10:44:12 -0700 Subject: RFR: JDK-8213214: Set -Djava.io.tmpdir= when running tests Message-ID: <2aa2e962-c630-60cd-6b0b-74796131ede8@oracle.com> There are a lot of jtreg tests that use temporary files. These temporary files add up over time and fill up the global temp directories on our test systems. To tackle this, we should try to redirect these temporary files into a directory controlled by the test framework. Jtreg does not do this, but we can set -Djava.io.tmpdir from RunTest.gmk. This will not prevent all temp files from being created outside of the work dir, but at least most. I have found one test where this becomes an issue, java/nio/file/Path/Misc.java on Windows when running in elevated mode with the workspace on a subst drive. This looks like an actual issue in the product, so I have filed a separate bug for it [1]. Since we currently use subst in our distributed test system to get around Windows path length issues, we are hitting this problem. While the bug is being evaluated, I have implemented a workaround in the test so that it is able to handle the known situation. I would like to have someone from core-libs look at the workaround. Webrev: http://cr.openjdk.java.net/~erikj/8213214/webrev.01/ Bug: https://bugs.openjdk.java.net/browse/JDK-8213214 /Erik [1] https://bugs.openjdk.java.net/browse/JDK-8213216 From erik.joelsson at oracle.com Tue Jun 16 19:22:00 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Tue, 16 Jun 2020 12:22:00 -0700 Subject: RFR: JDK-8213214: Set -Djava.io.tmpdir= when running tests Message-ID: (re-sending this as it doesn't look like it was delivered) There are a lot of jtreg tests that use temporary files. These temporary files add up over time and fill up the global temp directories on our test systems. To tackle this, we should try to redirect these temporary files into a directory controlled by the test framework. Jtreg does not do this, but we can set -Djava.io.tmpdir from RunTest.gmk. This will not prevent all temp files from being created outside of the work dir, but at least most. I have found one test where this becomes an issue, java/nio/file/Path/Misc.java on Windows when running in elevated mode with the workspace on a subst drive. This looks like an actual issue in the product, so I have filed a separate bug for it [1]. Since we currently use subst in our distributed test system to get around Windows path length issues, we are hitting this problem. While the bug is being evaluated, I have implemented a workaround in the test so that it is able to handle the known situation. I would like to have someone from core-libs look at the workaround. Webrev: http://cr.openjdk.java.net/~erikj/8213214/webrev.01/ Bug: https://bugs.openjdk.java.net/browse/JDK-8213214 /Erik [1] https://bugs.openjdk.java.net/browse/JDK-8213216 From alexey.semenyuk at oracle.com Tue Jun 16 21:56:15 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Tue, 16 Jun 2020 17:56:15 -0400 Subject: RFR: JDK-8264244: BasicShortcutHintTest shortcut can not be found In-Reply-To: <4efc0137-d2fb-cf69-1fac-3e871d5eedc4@oracle.com> References: <4efc0137-d2fb-cf69-1fac-3e871d5eedc4@oracle.com> Message-ID: <194ea4ad-e069-63b4-82d2-3ebcb2261f6f@oracle.com> Please review fix [2] for jpackage bug [1]. The fix is to put value of `Exec` property of .desktop files generated by jpackage in double quotes if the value of the propery contains whitespace characters. - Alexey [1] https://bugs.openjdk.java.net/browse/JDK-8264244 [2]?http://cr.openjdk.java.net/~asemenyuk/8264244/webrev.00 From alexander.matveev at oracle.com Tue Jun 16 22:13:37 2020 From: alexander.matveev at oracle.com (Alexander Matveev) Date: Tue, 16 Jun 2020 15:13:37 -0700 Subject: RFR: JDK-8264244: BasicShortcutHintTest shortcut can not be found In-Reply-To: <194ea4ad-e069-63b4-82d2-3ebcb2261f6f@oracle.com> References: <4efc0137-d2fb-cf69-1fac-3e871d5eedc4@oracle.com> <194ea4ad-e069-63b4-82d2-3ebcb2261f6f@oracle.com> Message-ID: <2a24b86e-7b42-6e82-fa22-11200f0bfe43@oracle.com> Hi Alexey, Looks good. I think you got links and bug ID incorrect. It should be JDK-8246244 and you have 8264244. Links also does not work. Working links are: https://bugs.openjdk.java.net/browse/JDK-8246244 http://cr.openjdk.java.net/~asemenyuk/8246244/webrev.00/ Thanks, Alexander On 6/16/2020 2:56 PM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > The fix is to put value of `Exec` property of .desktop files generated > by jpackage in double quotes if the value of the propery contains > whitespace characters. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8264244 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8264244/webrev.00 > From peter.levart at gmail.com Tue Jun 16 22:31:01 2020 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 17 Jun 2020 00:31:01 +0200 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> Message-ID: <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> Hi Chris, On 6/16/20 5:15 PM, Chris Hegarty wrote: > Hi Peter, > >> On 14 Jun 2020, at 17:28, Peter Levart wrote: >> >> ... >> >> [2] http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.01/ >> > I think that this is very good. It?s clever to build a chain of method handles that can be invoked with the stream field values. This is a nice separation between the shape of the data and the actual stream data. > > The caching is on a per-stream-field shape basis, which should be consistent in the common case, but of course is not always guaranteed to be the case, hence the need for the cache. I think that this should be fine, since the ObjectStreamClass ( that holds the cache ) is already itself cached as a weak reference. But I did wonder if the size of this new cache should be limited. Probably not worth the complexity unless it is an obvious issue. > > All the serailizable records tests pass successfully with your patch. Good. I did however notice that there is just a single test, DefaultValuesTest, that exercises different stream shapes for the same class in the stream. Would be good to expand coverage in this area, but of course some lower-level test-specific building blocks will be needed help build the specially crafted byte streams - I can help with this. > > Overall I think that this change is good. > > -Chris. > If there is a way for a test to compile several versions of the same (record) class and then produce byte streams with it, then we could simulate various class-evolution cases (field missing, surplus field, change of field type, etc...) without crafting the bytestream by hand. WDYT? Regards, Peter From peter.levart at gmail.com Tue Jun 16 22:36:29 2020 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 17 Jun 2020 00:36:29 +0200 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> Message-ID: <16802d3e-e5b2-4282-6447-478c72d95e3b@gmail.com> Hi Paul, On 6/16/20 5:50 PM, Paul Sandoz wrote: > Hi Peter, > > 199 private Map deserializationCtrs; > > Change to be either ConcurrentMap or ConcurrentHashMap, thereby making it clearer when using. Sure. Will do that. > > > 2679 private static MethodHandle defaultValueExtractorFor(Class pType) { > > Can the implementation use MethodHandles,zero? e.g. > > return MethodHandles.dropArguments(MethodHandles.zero(pType), 0, byte[].class, Object[].class); > > Paul. Yes, or even better: MethodHandles.empty(MethodType.methodType(pClass, byte[].class, Object[].class)) which was suggested by Johanes Kuhn as here: http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.02/ Regards, Peter > >> On Jun 16, 2020, at 8:15 AM, Chris Hegarty wrote: >> >> Hi Peter, >> >>> On 14 Jun 2020, at 17:28, Peter Levart wrote: >>> >>> ... >>> >>> [2] http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.01/ >>> >> I think that this is very good. It?s clever to build a chain of method handles that can be invoked with the stream field values. This is a nice separation between the shape of the data and the actual stream data. >> >> The caching is on a per-stream-field shape basis, which should be consistent in the common case, but of course is not always guaranteed to be the case, hence the need for the cache. I think that this should be fine, since the ObjectStreamClass ( that holds the cache ) is already itself cached as a weak reference. But I did wonder if the size of this new cache should be limited. Probably not worth the complexity unless it is an obvious issue. >> >> All the serailizable records tests pass successfully with your patch. Good. I did however notice that there is just a single test, DefaultValuesTest, that exercises different stream shapes for the same class in the stream. Would be good to expand coverage in this area, but of course some lower-level test-specific building blocks will be needed help build the specially crafted byte streams - I can help with this. >> >> Overall I think that this change is good. >> >> -Chris. >> From andy.herrick at oracle.com Tue Jun 16 22:49:01 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Tue, 16 Jun 2020 18:49:01 -0400 Subject: RFR: JDK-8264244: BasicShortcutHintTest shortcut can not be found In-Reply-To: <2a24b86e-7b42-6e82-fa22-11200f0bfe43@oracle.com> References: <4efc0137-d2fb-cf69-1fac-3e871d5eedc4@oracle.com> <194ea4ad-e069-63b4-82d2-3ebcb2261f6f@oracle.com> <2a24b86e-7b42-6e82-fa22-11200f0bfe43@oracle.com> Message-ID: <536442b7-e899-37d3-ab72-6e3b5e5a73f9@oracle.com> looks good with these links /Andy On 6/16/2020 6:13 PM, Alexander Matveev wrote: > Hi Alexey, > > Looks good. I think you got links and bug ID incorrect. It should be > JDK-8246244 and you have 8264244. Links also does not work. Working > links are: > https://bugs.openjdk.java.net/browse/JDK-8246244 > http://cr.openjdk.java.net/~asemenyuk/8246244/webrev.00/ > > Thanks, > Alexander > > On 6/16/2020 2:56 PM, Alexey Semenyuk wrote: >> Please review fix [2] for jpackage bug [1]. >> >> The fix is to put value of `Exec` property of .desktop files >> generated by jpackage in double quotes if the value of the propery >> contains whitespace characters. >> >> - Alexey >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8264244 >> >> [2]?http://cr.openjdk.java.net/~asemenyuk/8264244/webrev.00 >> > From forax at univ-mlv.fr Tue Jun 16 23:06:11 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 17 Jun 2020 01:06:11 +0200 (CEST) Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> Message-ID: <1522316544.1652270.1592348771671.JavaMail.zimbra@u-pem.fr> Hi Peter, is their a reason to not use a ClassValue using the record class as key instead of the more complex keys you are proposing ? R?mi ----- Mail original ----- > De: "Chris Hegarty" > ?: "Peter Levart" > Cc: "core-libs-dev" > Envoy?: Mardi 16 Juin 2020 17:15:03 > Objet: Re: RFR: 8247532: Records deserialization is slow > Hi Peter, > >> On 14 Jun 2020, at 17:28, Peter Levart wrote: >> >> ... >> >> [2] >> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.01/ >> > I think that this is very good. It?s clever to build a chain of method handles > that can be invoked with the stream field values. This is a nice separation > between the shape of the data and the actual stream data. > > The caching is on a per-stream-field shape basis, which should be consistent in > the common case, but of course is not always guaranteed to be the case, hence > the need for the cache. I think that this should be fine, since the > ObjectStreamClass ( that holds the cache ) is already itself cached as a weak > reference. But I did wonder if the size of this new cache should be limited. > Probably not worth the complexity unless it is an obvious issue. > > All the serailizable records tests pass successfully with your patch. Good. I > did however notice that there is just a single test, DefaultValuesTest, that > exercises different stream shapes for the same class in the stream. Would be > good to expand coverage in this area, but of course some lower-level > test-specific building blocks will be needed help build the specially crafted > byte streams - I can help with this. > > Overall I think that this change is good. > > -Chris. From mandy.chung at oracle.com Tue Jun 16 23:06:40 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 16 Jun 2020 16:06:40 -0700 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: <1e7832e9-1398-bb67-c270-b079b06aa588@oracle.com> Hi David, Thanks for the review. On 6/15/20 7:29 PM, David Holmes wrote: > Hi Mandy, Christoph, > > The code changes here all look okay to me. The idea of introducing the > notion of "trusted final fields" seems quite reasonable. > > I made one editorial comment on the CSR request. > Thanks. > I'm unclear if the new TEST.properties file needs a copyright notice > and header. We have 706 such files in the repo and 554 (mostly > hotspot) do have the copyright notice and header. > I will find out the recommendation and update you. Mandy > Thanks, > David > > On 16/06/2020 7:58 am, Mandy Chung wrote: >> This patch is joint contribution from Christoph Dreis [1] and me. >> >> Webrev: >> http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247444/webrev.00/ >> CSR: https://bugs.openjdk.java.net/browse/JDK-8247517 >> >> This proposes to make final fields in records notmodifiable via >> reflection.? Field::set throws IAE if a Field is not modifiable. >> Thecurrent specification specifies the following Fields not modifiable: >> - static final fields in any class >> - final fields in a hidden class >> >> The spec of Field::set is changed to specify that records are not >> modifiable via reflection. >> ??Noe that no change in Field::setAccessible(true), i.e. it will >> succeed to allow existing frameworks to have read access to final >> fields in records.? Just no write access. >> >> VarHandle does not support write access if it's a static final field >> or an instance final field. >> >> This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and >> `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. >> >> No change is made in JNI.? JNI could be considered to disallow >> modification of final fields in records and hidden classes (static >> and instance fields).? But JNI has superpower and the current spec >> already allows to modify the value of a final static field even after >> it's constant folded (via JNI SetField and >> SetStaticField), then all bets are off.? This should be >> re-visited when we consider "final is truly final" for all classes. >> >> Make final fields in records not modifiable via reflection enables >> JIT optimization as these final fields are effectively truly final. >> >> This change impacts 3rd-party frameworks including 3rd-party >> serialization framework that rely on core reflection `setAccessible` >> or `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to >> construct records but not using the canonical constructor. >> These frameworks would need to be updated to construct records via >> its canonical constructor as done by the Java serialization. >> >> I see this change gives a good opportunity to engage the maintainers >> of the serialization frameworks and work together to support new >> features including records, inline classes and the new serialization >> mechanism and which I think it is worth the investment. >> >> This is a low risk enhancement.? I'd like to request approval for a >> late enhancement in JDK 15.? It extends the pre-existing code path >> with refactoring the hidden classes to prepare for new kinds of >> classes with trusted final fields.? The change is straight-forward. >> >> Can this wait to integrate in JDK 16? >> >> ?? It's important to get this enhancement in when record is a preview >> feature that we can get feedback and give time to 3rd-party >> serialization frameworks to look into adding the support for records. >> If we delayed this change in 16 and records exit preview, it would be >> bad for frameworks if they verify that they support records in 15 but >> fail in 16.? OTOH the risk of this patch is low. >> >> Performance Impact >> >> I addressed the performance concern I raised earlier myself. For >> reflection, VM creates the reflective Field objects and fills in >> MemberName when resolving a member.? VM will tag if this >> Field/MemberName is trusted final field.? I think this is a cleaner >> approach rather than in each place to check for final static and >> final fields in hidden or record class to determine if it has write >> access or not. >> >> `sun.misc.Unsafe` does not use Field::isTrustedFinalField because (1) >> Unsafe has been allowing access of static final fields of any classes >> but isTrustedFinalField is not limited to instance fields (2) Unsafe >> disallows access to all fields in a hidden class (not limited to >> trusted final fields).? So it follows the precedence and simply >> checks if the declaring class is a record. `Class::isRecord` calls >> `Class::getSuperclass` to check if it's a subtype of `Record`.? As >> `Class::getSuperclass` is intrinsified, the call on isRecord on a >> normal class is fast. Christoph has contributed the microbenchmarks >> that confirm that no performance regression. >> >> Thanks >> Mandy >> [1] >> https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-June/040096.html >> From paul.sandoz at oracle.com Tue Jun 16 23:19:20 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 16 Jun 2020 16:19:20 -0700 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: <16802d3e-e5b2-4282-6447-478c72d95e3b@gmail.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <16802d3e-e5b2-4282-6447-478c72d95e3b@gmail.com> Message-ID: <0A7C560C-4D0E-462F-820C-9C144F69F699@oracle.com> > On Jun 16, 2020, at 3:36 PM, Peter Levart wrote: > > Yes, or even better: MethodHandles.empty(MethodType.methodType(pClass, byte[].class, Object[].class)) which was suggested by Johanes Kuhn as here: > > > http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.02/ > Ah yes much better. Paul. From david.holmes at oracle.com Wed Jun 17 00:17:21 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 17 Jun 2020 10:17:21 +1000 Subject: RFR(S) : 8211977 : move testlibrary tests into one place In-Reply-To: <287C88A1-3464-484E-AF66-7822394CA125@oracle.com> References: <39df1e40-f34a-ba3b-e2e2-f462a6442c42@oracle.com> <214B2701-0723-45A5-81B5-816EF1D86470@oracle.com> <287C88A1-3464-484E-AF66-7822394CA125@oracle.com> Message-ID: <5f7e286a-e2ee-e544-3c6a-62d1f066895f@oracle.com> On 17/06/2020 2:45 am, Igor Ignatyev wrote: > Hi David, > > thanks for your review. re: LingeredAppTest, I agree that the test doesn't look very useful, yet I'd remind that the goal of this (and other test in /test/lib-test) is to (sanity) test testlibrary in order to easily spot bugs in testlibrary in a clear manner so one would not have to investigate failures of actual "product" tests and go thru their sometimes convoluted logic just to find out that the problem was in LingeredApp. w/ that being said, this test can do a better job in testing LingeredApp, so I'll file a JBS ticket against hotspo/svc to evaluate/improve/discuss the fate of the test. Given the test had not previously been run (it wouldn't even compile as far as I could see) it's validity and utility has to be called into question. But a follow up bug is fine for that as long it it currently works reliably. Thanks, David > Thanks, > -- Igor > >> On Jun 16, 2020, at 12:14 AM, David Holmes wrote: >> >> Hi Igor, >> >> On 16/06/2020 10:39 am, Igor Ignatyev wrote: >>> @David, Erik, Magnus, >>> please find the answers to your comments at the bottom of this email. >>> @all, >>> David's and Erik's comments made me realize that some parts of the original patch were stashed away and didn't make it to webrev.00. I'm truly sorry for the confusion and inconvenience. I also agree w/ David that it's hard to follow/review, and my gaffe just made it worse, therefore I'd like to start it over again from a clean sate >>> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 >>>> 833 lines changed: 228 ins; 591 del; 14 mod; >>> could you please review the patch which puts all tests for common testlibrary classes (which is ones located in /test/lib) into one location under /test/lib-test? please note this intentionally doesn't move all "testlibrary" tests, e.g. tests for CTW, hotspot-specific testlibrary, are left in /test/hotspot/jtreg/testlibrary_tests/ctw . >> >> Ok. >> >>> to simplify review, the patch has been split into several webrevs, with each of them accomplishes a more-or-less distinct thing, but is not necessary self-contained: >> >> Many thanks for doing this! >> >>> 0. trivial move of tests from test/jdk and test/hotspot/jtreg test suites to test/lib-test: >>> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.00-move/ >> >> Looks good. >> >>> 1. removal of hotspot's AssertsTest, OutputAnalyzerReportingTest and "merge" of hotspot's and jdk's OutputAnalyzerTest: >>> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.01-merge/ >> >> Looks good. >> >>> 2. simplification of TestNativeProcessBuilder tests: converts Test class used by TestNativeProcessBuilder into a static nested class: >>> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.02-TestNativeProcessBuilder >> >> Looks good. >> >>> 3. makefiles changes to build native parts of lib-test tests. in past, there were no tests w/ native parts in this test suite, TestNativeProcessBuilder is the 1st such test; this part also removes now unneeded lines from hotspot test suite makefile: >>> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.03-NativeLibTest >> >> Hmm okay. Not sure this needed its own category of build logic but ... >> >> Aside: Makefiles should not have the classpath exception version of the license header. But they all do for some reason. I'll check if we need to do a separate cleanup of that. >> >>> 4. clean ups in hotspot test suite: adjusted location of SimpleClassFileLoadHookTest test, which is a test for hotspot-specific test library (located in /test/hotspot/jtreg/testlibrary/jvmti) and hence should not be moved to /test/lib-test; removed TestMutuallyExclusivePlatformPredicates from TEST.groups: >>> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.04-hotspot/ >> >> Looks good. Took me a while to understand the connection to the test library here. >> >>> 5. LingeredAppTest fix (which apparently has never been run before): >>> http://cr.openjdk.java.net/~iignatyev/8211977/webrev.02.05-LingeredAppTest/ >> >> Someone from serviceability should evaluate this test. It may not be needed. >> >>> 6. changes to make test/lib-test a fully supported and working test suite, and add in into tier1, includes adding ProblemList, TEST.groups file, and 'randomness' k/w: >>> http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02.06-TESTROOT/ >> >> Seems okay. >> >> Thanks, >> David >> ----- >> >>> webrev: http://cr.openjdk.java.net/~iignatyev//8211977/webrev.02 >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8211977 >>> testing: >>> - make test TEST=tier1 locally on macosx >>> - test/lib-test on {windows,linux,macosx}-x64 >>> - tier1 job (in progress) >>> Thanks, >>> -- Igor >>>> On Jun 14, 2020, at 11:23 PM, David Holmes > wrote: >>>> <...> >>>> This doesn't seem to move everything under test/hotspot/jtreg/testlibrary_tests/ e.g. ctw directory. >>> this is intentionally, ctw test-library is hotspot specific, hence its tests are to reside in hotspot test suite (until we decide to move the library to /test/lib), the same is true for SimpleClassFileLoadHookTest. >>>> >>>> You did not modify hotspot/jtreg/TEST.groups but it refers to: >>>> testlibrary_tests/TestMutuallyExclusivePlatformPredicates.java \ >>>> Plus it should probably refer to the new native_sanity tests somewhere. >>> the actual version of the patch removed reference to TestMutuallyExclusivePlatformPredicates from hotspot's TEST.groups and had TestNativeProcessBuilder moved to /test/test-lib, so no updates w.r.t. native_sanity are needed >>>> The newly added copyright notice needs to have the correct initial copyright year based on when the file was first added to the repo. >>> /test/lib-test/TEST.ROOT file was created by JDK-8243010 on 2020-04-16, hence the added copyright has 2020 as the initial copyright year. >>>> You used the wrong license header - makefiles don't use the classpath exception. >>> JtregNativeLibTest.gmk is based on make/test/JtregNativeJdk.gmk which also have classpath exception. quick grep showed that make directory has 794 files which has '"Classpath" exception', out of 850 which contain 'GNU General Public License version 2' string. And although I agree that makefiles shouldn't have the classpath exception, I'd prefer to keep JtregNativeLibTest.gmk in sync w/ the its siblings and would leave it up to build team to decide how it's best to handle. >>>> On Jun 15, 2020, at 8:12 AM, Magnus Ihse Bursie > wrote: >>>> >>>> A few comments: >>>> >>>> This seems like code copied from elsewhere: >>>> 57 # This evaluation is expensive and should only be done if this target was >>>> 58 # explicitly called. >>>> 59 ifneq ($(filter build-test-libtest-jtreg-native, $(MAKECMDGOALS)), ) >>>> I don't agree that this is an expensive evaluation. Furthermore, the makefile is only called for building the testlib and for making images, so in worst case it's just the image part that would get a penalty (although I highly doubt there is any). >>> right, JtregNativeLibTest.gmk is based on make/test/JtregNativeJdk.gmk which has this comment, I don't know enough details to say if it's expensive or not, yet if you insist and there is a consensus within build team, I can remove the comment. >>>> >>>> 82 $(eval $(call SetupCopyFiles,COPY_LIBTEST_JTREG_NATIVE, \ >>>> Please use space after comma. >>> this again was preexisting in JtregNativeJdk.gmk, added the space nevertheless. >>>> On Jun 15, 2020, at 6:16 AM, Erik Joelsson > wrote: >>>> >>>> In JtretNativeLibTest.gmk, lines 51-55 should probably be removed (or adjusted if linking to libjvm is actually needed). >>> the lines were updated to define BUILD_LIBTEST_JTREG_EXECUTABLES_LIBS_exejvm-test-launcher > From john.r.rose at oracle.com Wed Jun 17 04:06:40 2020 From: john.r.rose at oracle.com (John Rose) Date: Tue, 16 Jun 2020 21:06:40 -0700 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: On Jun 15, 2020, at 2:58 PM, Mandy Chung wrote: > > Webrev: http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247444/webrev.00/ > CSR: https://bugs.openjdk.java.net/browse/JDK-8247517 Good work. Reviewed. ? John From peter.levart at gmail.com Wed Jun 17 05:37:16 2020 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 17 Jun 2020 07:37:16 +0200 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: <1522316544.1652270.1592348771671.JavaMail.zimbra@u-pem.fr> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <1522316544.1652270.1592348771671.JavaMail.zimbra@u-pem.fr> Message-ID: <622aa076-b5d6-1e26-4966-6ff2420f25b4@gmail.com> Hi Remi, The keys used are based on the ordered names and types of "fields" in the ObjectStreamClass that has been deserialized as part of the object stream for the record "type" being deserialized. So for each record type (Class) there can be several distinct ObjectStreamClasses deserialized in the same VM that were produced by serializing different versions of record types in different VMs or ClassLoader(s) and that may vary in the names and/or types and/or order of fields. And since the MethodHandle chain of transformations that is being used for a particular ObjectStreamClass is dependent on the ordered names and types of ObjectStreamField(s) of deserialized ObjectStreamClass, the key is based on that too. You may ask why not simply add a MethodHandle field to the ObjectStreamClass instance that is deserialized and therefore unique? Well, such ObjectStreamClass only lasts for one "session" of ObjectStream deserialization, so such caching wouldn't be efficient. But common parts of that ObjectStreamClass that are only dependent on the current local-VM type are cached in a special instance of ObjectStreamClass and that's where I put a cache of deserializaiton constructors in the form of a Map. Regards, Peter On 6/17/20 1:06 AM, Remi Forax wrote: > Hi Peter, > is their a reason to not use a ClassValue using the record class as key instead of the more complex keys you are proposing ? > > R?mi > > ----- Mail original ----- >> De: "Chris Hegarty" >> ?: "Peter Levart" >> Cc: "core-libs-dev" >> Envoy?: Mardi 16 Juin 2020 17:15:03 >> Objet: Re: RFR: 8247532: Records deserialization is slow >> Hi Peter, >> >>> On 14 Jun 2020, at 17:28, Peter Levart wrote: >>> >>> ... >>> >>> [2] >>> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.01/ >>> >> I think that this is very good. It?s clever to build a chain of method handles >> that can be invoked with the stream field values. This is a nice separation >> between the shape of the data and the actual stream data. >> >> The caching is on a per-stream-field shape basis, which should be consistent in >> the common case, but of course is not always guaranteed to be the case, hence >> the need for the cache. I think that this should be fine, since the >> ObjectStreamClass ( that holds the cache ) is already itself cached as a weak >> reference. But I did wonder if the size of this new cache should be limited. >> Probably not worth the complexity unless it is an obvious issue. >> >> All the serailizable records tests pass successfully with your patch. Good. I >> did however notice that there is just a single test, DefaultValuesTest, that >> exercises different stream shapes for the same class in the stream. Would be >> good to expand coverage in this area, but of course some lower-level >> test-specific building blocks will be needed help build the specially crafted >> byte streams - I can help with this. >> >> Overall I think that this change is good. >> >> -Chris. From peter.levart at gmail.com Wed Jun 17 05:44:28 2020 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 17 Jun 2020 07:44:28 +0200 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> Message-ID: <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> Hi Chris, On 6/17/20 12:31 AM, Peter Levart wrote: >> I did however notice that there is just a single test, >> DefaultValuesTest, that exercises different stream shapes for the >> same class in the stream. Would be good to expand coverage in this >> area, but of course some lower-level test-specific building blocks >> will be needed help build the specially crafted byte streams - I can >> help with this. >> >> Overall I think that this change is good. >> >> -Chris. >> > If there is a way for a test to compile several versions of the same > (record) class and then produce byte streams with it, then we could > simulate various class-evolution cases (field missing, surplus field, > change of field type, etc...) without crafting the bytestream by hand. > WDYT? I have a better idea. The test could contain several different classes with different names that would be used to generated variations of serialized stream. Such streams could then be "patched" so they would contain the common target class name and then used to (attempt to) deserialize the target class. I'll try to devise such test... Regards, Peter From peter.levart at gmail.com Wed Jun 17 06:08:20 2020 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 17 Jun 2020 08:08:20 +0200 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> Message-ID: <1c304433-6203-6480-b027-ac5a87a27dc2@gmail.com> On 6/16/20 5:15 PM, Chris Hegarty wrote: > The caching is on a per-stream-field shape basis, which should be consistent in the common case, but of course is not always guaranteed to be the case, hence the need for the cache. I think that this should be fine, since the ObjectStreamClass ( that holds the cache ) is already itself cached as a weak reference. But I did wonder if the size of this new cache should be limited. Probably not worth the complexity unless it is an obvious issue. I don't think there will normally be many different shapes of the same class deserialized by a single VM. Each shape means that a different version of that class must have existed to serialize it. There could be deliberate "forged" streams trying to inflate the cache. Are you worried about that? In that case I can add logic to limit the number of different shapes kept with a simple LRA (Least Recently Added) strategy that would not hurt access performance. Regards, Peter From chris.hegarty at oracle.com Wed Jun 17 08:20:26 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 17 Jun 2020 09:20:26 +0100 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: <1c304433-6203-6480-b027-ac5a87a27dc2@gmail.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <1c304433-6203-6480-b027-ac5a87a27dc2@gmail.com> Message-ID: <8C29BEC2-0E53-473A-AD00-FA3141171B89@oracle.com> > On 17 Jun 2020, at 07:08, Peter Levart wrote: > > > On 6/16/20 5:15 PM, Chris Hegarty wrote: >> The caching is on a per-stream-field shape basis, which should be consistent in the common case, but of course is not always guaranteed to be the case, hence the need for the cache. I think that this should be fine, since the ObjectStreamClass ( that holds the cache ) is already itself cached as a weak reference. But I did wonder if the size of this new cache should be limited. Probably not worth the complexity unless it is an obvious issue. > > I don't think there will normally be many different shapes of the same class deserialized by a single VM. Each shape means that a different version of that class must have existed to serialize it. Right. > There could be deliberate "forged" streams trying to inflate the cache. Are you worried about that? Yes. > In that case I can add logic to limit the number of different shapes kept with a simple LRA (Least Recently Added) strategy that would not hurt access performance. That would be great. -Chris. From peter.levart at gmail.com Wed Jun 17 08:20:36 2020 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 17 Jun 2020 10:20:36 +0200 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: <1c304433-6203-6480-b027-ac5a87a27dc2@gmail.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <1c304433-6203-6480-b027-ac5a87a27dc2@gmail.com> Message-ID: On 6/17/20 8:08 AM, Peter Levart wrote: > > On 6/16/20 5:15 PM, Chris Hegarty wrote: >> The caching is on a per-stream-field shape basis, which should be >> consistent in the common case, but of course is not always guaranteed >> to be the case, hence the need for the cache. I think that this >> should be fine, since the ObjectStreamClass ( that holds the cache ) >> is already itself cached as a weak reference. But I did wonder if the >> size of this new cache should be limited. Probably not worth the >> complexity unless it is an obvious issue. > > I don't think there will normally be many different shapes of the same > class deserialized by a single VM. Each shape means that a different > version of that class must have existed to serialize it. There could > be deliberate "forged" streams trying to inflate the cache. Are you > worried about that? In that case I can add logic to limit the number > of different shapes kept with a simple LRA (Least Recently Added) > strategy that would not hurt access performance. > > > Regards, Peter > So, here's an attempt to limit the number of different shapes cached per class which doesn't affect the fast-path performance: http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.03/ Regards, Peter From chris.hegarty at oracle.com Wed Jun 17 08:23:22 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 17 Jun 2020 09:23:22 +0100 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> Message-ID: > On 17 Jun 2020, at 06:44, Peter Levart wrote: >>> .. >> If there is a way for a test to compile several versions of the same (record) class and then produce byte streams with it, then we could simulate various class-evolution cases (field missing, surplus field, change of field type, etc...) without crafting the bytestream by hand. WDYT? > > I have a better idea. The test could contain several different classes with different names that would be used to generated variations of serialized stream. Such streams could then be "patched" so they would contain the common target class name and then used to (attempt to) deserialize the target class. I'll try to devise such test? > I think that could work. I?ll mock something up too, just for comparison purposes. -Chris. From Alan.Bateman at oracle.com Wed Jun 17 08:24:36 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 17 Jun 2020 09:24:36 +0100 Subject: RFR: JDK-8213214: Set -Djava.io.tmpdir= when running tests In-Reply-To: <2aa2e962-c630-60cd-6b0b-74796131ede8@oracle.com> References: <2aa2e962-c630-60cd-6b0b-74796131ede8@oracle.com> Message-ID: <0a5a64a1-e203-16b4-96bd-497ed7eda700@oracle.com> On 16/06/2020 18:44, Erik Joelsson wrote: > There are a lot of jtreg tests that use temporary files. These > temporary files add up over time and fill up the global temp > directories on our test systems. To tackle this, we should try to > redirect these temporary files into a directory controlled by the test > framework. Jtreg does not do this, but we can set -Djava.io.tmpdir > from RunTest.gmk. This will not prevent all temp files from being > created outside of the work dir, but at least most. > > I have found one test where this becomes an issue, > java/nio/file/Path/Misc.java on Windows when running in elevated mode > with the workspace on a subst drive. This looks like an actual issue > in the product, so I have filed a separate bug for it [1]. Since we > currently use subst in our distributed test system to get around > Windows path length issues, we are hitting this problem. While the bug > is being evaluated, I have implemented a workaround in the test so > that it is able to handle the known situation. I would like to have > someone from core-libs look at the workaround. > > Webrev: http://cr.openjdk.java.net/~erikj/8213214/webrev.01/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8213214 subst is a legacy mechanism. We know from previous attempts to reconcile usage of subset with NTFS symbolic links creates surprises and inconsistencies. I don't object to putting a workaround in the test but I think more information is required, in particular the comment that 'file' is guaranteed to be subst free? Can you expand on this? Some examples would be useful as it's not clear how subst is used in the environment, is it solely that java.io.tmpdir is set to something like S:\tmp and S: is mapped to some other location? -Alan From claes.redestad at oracle.com Wed Jun 17 10:54:05 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 17 Jun 2020 12:54:05 +0200 Subject: RFR[16]: 8247681: Improve bootstrapping of unary concatenations In-Reply-To: References: <10b4c312-3223-3412-016c-60515581733d@oracle.com> Message-ID: <6ccc0846-dd94-e029-c5a0-044268f39e50@oracle.com> On 2020-06-16 18:06, Paul Sandoz wrote: > Looks good. Thanks! > > It?s tempting to do something like this to de-dup the code with less potential for mistakes: > > String s = null; > // Select the singular non-null value, if any > If (s0 != null && s1 == null) s = s0; > else if (s0 == null && s1 != null) s = s1; > If (s != null) { > if (s.isEmpty()) { > ? > } else if (?) { ? } > } Ok. The index of where to inject the constant is order dependent, but the code still slims down a bit: http://cr.openjdk.java.net/~redestad/8247681/open.01/ Re-ran tier1+2 /Claes > > Paul. > >> On Jun 16, 2020, at 6:00 AM, Claes Redestad wrote: >> >> Hi, >> >> this patch specializes bootstrapping of unary concatenation expressions. >> >> Such expressions can be reduced to the canonical String.valueOf >> "stringifier" for any primitive argument, but needs a special >> stringifier for reference arguments since we need to produce a new >> String to be compliant with JLS. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8247681 >> Webrev: http://cr.openjdk.java.net/~redestad/8247681/open.00/ >> >> This ensures we get a similar speed-up as JDK-8247605 for >> "" + fooString (and fooString + ""). >> >> Also speeds up bootstrapping for all such simple unary concatenation >> expressions. >> >> Testing: tier1+2 >> >> Before: >> >> Benchmark (intValue) Mode Cnt Score Error Units >> StringConcat.concatEmptyConstInt 4711 avgt 5 15.539 ? 0.831 ns/op >> StringConcat.concatEmptyConstString 4711 avgt 5 17.046 ? 1.047 ns/op >> StringConcat.concatEmptyLeft 4711 avgt 5 7.506 ? 0.588 ns/op >> StringConcat.concatEmptyRight 4711 avgt 5 7.890 ? 0.314 ns/op >> >> After: >> Benchmark (intValue) Mode Cnt Score Error Units >> StringConcat.concatEmptyConstInt 4711 avgt 5 15.410 ? 0.944 ns/op >> StringConcat.concatEmptyConstString 4711 avgt 5 7.397 ? 0.384 ns/op >> StringConcat.concatEmptyLeft 4711 avgt 5 7.465 ? 0.328 ns/op >> StringConcat.concatEmptyRight 4711 avgt 5 7.857 ? 0.355 ns/op >> >> Thanks! >> >> /Claes > From vladimir.x.ivanov at oracle.com Wed Jun 17 11:13:53 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 17 Jun 2020 14:13:53 +0300 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: Looks good! I welcome and fully support such hardening of final field handling for new language features. Minor comment on naming: "trusted final field" is an JVM implementation detail and I'd prefer to keep it internal to JVM. (I hope it'll eventually go away and will be replaced with a more disciplined approach.) Probably, a better alternative is a name that clearly communicates that the JVM expects/allows modifications of a final field. Or, at least, that it's *the JVM* which "trusts" the contents of a final field after its initialization is over (vmTrustedFinal/isVMTrustedFinalField). But I'm fine with keeping the patch as is. Best regards, Vladimir Ivanov On 16.06.2020 00:58, Mandy Chung wrote: > This patch is joint contribution from Christoph Dreis [1] and me. > > Webrev: http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247444/webrev.00/ > CSR: https://bugs.openjdk.java.net/browse/JDK-8247517 > > This proposes to make final fields in records notmodifiable via > reflection.? Field::set throws IAE if a Field is not modifiable. > Thecurrent specification specifies the following Fields not modifiable: > - static final fields in any class > - final fields in a hidden class > > The spec of Field::set is changed to specify that records are not > modifiable via reflection. > ?Noe that no change in Field::setAccessible(true), i.e. it will succeed > to allow existing frameworks to have read access to final fields in > records.? Just no write access. > > VarHandle does not support write access if it's a static final field or > an instance final field. > > This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and > `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. > > No change is made in JNI.? JNI could be considered to disallow > modification of final fields in records and hidden classes (static and > instance fields).? But JNI has superpower and the current spec already > allows to modify the value of a final static field even after it's > constant folded (via JNI SetField and SetStaticField), then > all bets are off.? This should be re-visited when we consider "final is > truly final" for all classes. > > Make final fields in records not modifiable via reflection enables JIT > optimization as these final fields are effectively truly final. > > This change impacts 3rd-party frameworks including 3rd-party > serialization framework that rely on core reflection `setAccessible` or > `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to > construct records but not using the canonical constructor. > These frameworks would need to be updated to construct records via its > canonical constructor as done by the Java serialization. > > I see this change gives a good opportunity to engage the maintainers of > the serialization frameworks and work together to support new features > including records, inline classes and the new serialization mechanism > and which I think it is worth the investment. > > This is a low risk enhancement.? I'd like to request approval for a late > enhancement in JDK 15.? It extends the pre-existing code path with > refactoring the hidden classes to prepare for new kinds of classes with > trusted final fields.? The change is straight-forward. > > Can this wait to integrate in JDK 16? > > ? It's important to get this enhancement in when record is a preview > feature that we can get feedback and give time to 3rd-party > serialization frameworks to look into adding the support for records. If > we delayed this change in 16 and records exit preview, it would be bad > for frameworks if they verify that they support records in 15 but fail > in 16.? OTOH the risk of this patch is low. > > Performance Impact > > I addressed the performance concern I raised earlier myself.? For > reflection, VM creates the reflective Field objects and fills in > MemberName when resolving a member.? VM will tag if this > Field/MemberName is trusted final field.? I think this is a cleaner > approach rather than in each place to check for final static and final > fields in hidden or record class to determine if it has write access or > not. > > `sun.misc.Unsafe` does not use Field::isTrustedFinalField because (1) > Unsafe has been allowing access of static final fields of any classes > but isTrustedFinalField is not limited to instance fields (2) Unsafe > disallows access to all fields in a hidden class (not limited to trusted > final fields).? So it follows the precedence and simply checks if the > declaring class is a record. `Class::isRecord` calls > `Class::getSuperclass` to check if it's a subtype of `Record`.? As > `Class::getSuperclass` is intrinsified, the call on isRecord on a normal > class is fast. Christoph has contributed the microbenchmarks that > confirm that no performance regression. > > Thanks > Mandy > [1] > https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-June/040096.html > From alexey at azul.com Wed Jun 17 11:26:25 2020 From: alexey at azul.com (Alexey Bakhtin) Date: Wed, 17 Jun 2020 11:26:25 +0000 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <97a4f719-e0f5-96f7-6637-432b4b746685@oracle.com> References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> <9013C8D0-33D9-4D09-8953-F8618020A7F7@oracle.com> <8A2C4DEF-FC61-4574-B49D-1133F5349BCE@azul.com> <15fa46f2-0352-a111-a9c7-0d5fc31d9159@oracle.com> <6167ea24-e7f9-c259-e2c5-8be96d74a40d@oracle.com> <228BA21D-836A-46D7-B273-738AFFE639E4@azul.com> <97a4f719-e0f5-96f7-6637-432b4b746685@oracle.com> Message-ID: <567C1F82-0EE7-46F3-90E9-3A1B5FDD5E13@azul.com> Hello Daniel, Thank you for review. As you suggested I?ve added static factory methods to create TlsChannelBinding class and changed connectionTimeout verification to "if (connectTimeout <= 0)" Also, I?ve added simple regression test to verify Channel Binding parameters. Please find updated webrev at: http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v8/ The link to CSR for this feature : https://bugs.openjdk.java.net/browse/JDK-8247311 Regards Alexey > On 12 Jun 2020, at 12:20, Daniel Fuchs wrote: > > Hi Alexey, > > This is starting to look good. > Thank you for persisting! > > I have a couple of comments - on the LDAP/JNDI side. > > There are several places where your new code is supposed to > trigger the throwing of a NamingException: > > LdapSasl.java: lines 76, 85, 140, 168 > > Please write a test to verify that it does so. Since the > exceptions are all supposed to be thrown before the actual > binding happens, there's no need to have an actual LDAP server > that supports any kind of authentication to test that. > > The simple dummy servers that we have in the test base should > be enough to write such a test. > > In addition, there are a couple of places where stray exceptions > could theoretically be thrown, and where they should be wrapped in NamingException (but are not). Maybe it's OK, because this has > been checked before - but it would be better if we had a test that > proves that it is so: > > For instance: LdapSasl.java > 82 connectTimeout = Integer.parseInt((String)propVal); > What if the value of the propVal is "Foo" - or not a String? > What unexpected exception might be relayed to the calling code? > > Still in that same file: > 84 if (connectTimeout == -1) > > should probably be if (connectTimeout <= 0) since > Connection.java checks for connectTimeout > 0 to determine > whether to start the initial handshake. > > Which makes me wonder if we should find a better way to > instruct Connection to start the initial handshake... > > TlsChannelBinding.java: > > It would be much better if static factories methods were used instead of > public constructors. That would allow you to check all arguments before > actually constructing your object: > > public static TlsChannelBinding create((byte[] finishedMessage) throws SaslException { > throw new UnsupportedOperationException("tls-unique channel binding is not supported"); > } > > public statuic TlsChannelBinding create(X509Certificate serverCertificate) throws SaslException { > var cbType = TlsChannelBindingType.TLS_SERVER_END_POINT; > byte[] cbData; > try { > // compute cbdata > ... > > return new TlsChannelBinding(cbType, cbData); > } > > private TlsChannelBinding(TlsChannelBindingType cbType, byte[] cbData) { > this.cbType = cbType; > this.cbData = cbData; > } > > That's all I have for now. > > best regards, > > -- daniel > From erik.joelsson at oracle.com Wed Jun 17 13:47:46 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 17 Jun 2020 06:47:46 -0700 Subject: RFR: JDK-8213214: Set -Djava.io.tmpdir= when running tests In-Reply-To: <0a5a64a1-e203-16b4-96bd-497ed7eda700@oracle.com> References: <2aa2e962-c630-60cd-6b0b-74796131ede8@oracle.com> <0a5a64a1-e203-16b4-96bd-497ed7eda700@oracle.com> Message-ID: <6f9b273c-cd3f-158e-6e67-c82b112c4a43@oracle.com> On 2020-06-17 01:24, Alan Bateman wrote: > On 16/06/2020 18:44, Erik Joelsson wrote: >> There are a lot of jtreg tests that use temporary files. These >> temporary files add up over time and fill up the global temp >> directories on our test systems. To tackle this, we should try to >> redirect these temporary files into a directory controlled by the >> test framework. Jtreg does not do this, but we can set >> -Djava.io.tmpdir from RunTest.gmk. This will not prevent all temp >> files from being created outside of the work dir, but at least most. >> >> I have found one test where this becomes an issue, >> java/nio/file/Path/Misc.java on Windows when running in elevated mode >> with the workspace on a subst drive. This looks like an actual issue >> in the product, so I have filed a separate bug for it [1]. Since we >> currently use subst in our distributed test system to get around >> Windows path length issues, we are hitting this problem. While the >> bug is being evaluated, I have implemented a workaround in the test >> so that it is able to handle the known situation. I would like to >> have someone from core-libs look at the workaround. >> >> Webrev: http://cr.openjdk.java.net/~erikj/8213214/webrev.01/ >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8213214 > subst is a legacy mechanism. We know from previous attempts to > reconcile usage of subset with NTFS symbolic links creates surprises > and inconsistencies. I don't object to putting a workaround in the > test but I think more information is required, in particular the > comment that 'file' is guaranteed to be subst free? Can you expand on > this? Some examples would be useful as it's not clear how subst is > used in the environment, is it solely that java.io.tmpdir is set to > something like S:\tmp and S: is mapped to some other location? > The following code breaks if file refers to a path on a subst drive. Path file = Files.createFile(dir.resolve("foo")); Path link = dir.resolve("link"); Files.createSymbolicLink(link, file.toAbsolutePath()); assertTrue(link.toRealPath(NOFOLLOW_LINKS).getFileName().equals(link.getFileName())); In our case, this is caused by having a workspace in d:/X/Y/Z/workspace and having run something like "subst t: d:/X/Y/Z" before invoking the "make test" command in "t:/workspace". We currently need that subst to shorten path lengths because the X/Y/Z is a very deep path outside of our control. (We are looking into other solutions to this, but may not be able to apply them universally) Currently java.io.tmpdir is pointing to the default location, so the test passes, but if I explicitly set java.io.tmpdir to point into a subdir of the workspace (which is what this patch is about), then we trigger this failure. I have stepped through the various variants of calls to Path::toRealPath. When called on a path to a normal file, with no symlinks in it, it will resolve the real path with pure java string manipulation. When called on "link" above, the java loop detects that there is a symlink and defers to a native Windows API to resolve the symlink. This is done by getting a handle to the link, then resolving by handle. The file path returned from that call has the subst resolved. I don't know of another way to explicitly resolve subst drives back to their original directory from Java, so the workaround I put in place here simply does the same thing as the actual test. I create a link to "dir" and call toRealPath on it. If dir was located on a subst drive, then the return value from that call is guaranteed to have resolved that subst back to the original location. I've tried to make this clearer in the comment. http://cr.openjdk.java.net/~erikj/8213214/webrev.02/ /Erik From chris.hegarty at oracle.com Wed Jun 17 14:13:50 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 17 Jun 2020 15:13:50 +0100 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> Message-ID: <1DA206A1-15F0-46F9-B69E-9896C57B97C7@oracle.com> Peter, > On 17 Jun 2020, at 09:23, Chris Hegarty wrote: > >> On 17 Jun 2020, at 06:44, Peter Levart wrote: >>>> .. >>> If there is a way for a test to compile several versions of the same (record) class and then produce byte streams with it, then we could simulate various class-evolution cases (field missing, surplus field, change of field type, etc...) without crafting the bytestream by hand. WDYT? >> >> I have a better idea. The test could contain several different classes with different names that would be used to generated variations of serialized stream. Such streams could then be "patched" so they would contain the common target class name and then used to (attempt to) deserialize the target class. I'll try to devise such test? >> > I think that could work. I?ll mock something up too, just for comparison purposes. Here is an initial version at a test that can be expanded to cover more of the stream-field shapes of serializable records. I quickly backed away from generating the byte streams using OOS alone, since it enforces strict ordering of the fields in the stream, beyond that of primitives followed by non-primitives. And I want to be able to do things like, for example, record Point(int x, int y) - stream of x:1 y:2, or y:2 x:1 - which will verify the nominality aspect of the caching implementation. So I opted for a basic byte-stream-builder approach. https://cr.openjdk.java.net/~chegar/serialFieldTest_webrev/test/jdk/java/io/Serializable/records/DifferentStreamFieldsTest.java.html I also would like a builder of serial byte streams anyway, as it will be useful for things beyond this change. Maybe it could even be expanded upon and used as a test library, at some future point. The above implementation can probably be easily broken if pushed hard with larger graphs of objects, but it should be good enough for what we need here. -Chris. From paul.sandoz at oracle.com Wed Jun 17 14:40:50 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 17 Jun 2020 07:40:50 -0700 Subject: RFR[16]: 8247681: Improve bootstrapping of unary concatenations In-Reply-To: <6ccc0846-dd94-e029-c5a0-044268f39e50@oracle.com> References: <10b4c312-3223-3412-016c-60515581733d@oracle.com> <6ccc0846-dd94-e029-c5a0-044268f39e50@oracle.com> Message-ID: > On Jun 17, 2020, at 3:54 AM, Claes Redestad wrote: > > On 2020-06-16 18:06, Paul Sandoz wrote: >> Looks good. > > Thanks! > >> It?s tempting to do something like this to de-dup the code with less potential for mistakes: >> String s = null; >> // Select the singular non-null value, if any >> If (s0 != null && s1 == null) s = s0; >> else if (s0 == null && s1 != null) s = s1; >> If (s != null) { >> if (s.isEmpty()) { >> ? >> } else if (?) { ? } >> } > > Ok. The index of where to inject the constant is order dependent, Ah, I missed that, plus was not aware that s0 and s1 cannot both be null. > but > the code still slims down a bit: > > http://cr.openjdk.java.net/~redestad/8247681/open.01/ > +1 Paul. > Re-ran tier1+2 > > /Claes > From Alan.Bateman at oracle.com Wed Jun 17 14:43:33 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 17 Jun 2020 15:43:33 +0100 Subject: RFR: JDK-8213214: Set -Djava.io.tmpdir= when running tests In-Reply-To: <6f9b273c-cd3f-158e-6e67-c82b112c4a43@oracle.com> References: <2aa2e962-c630-60cd-6b0b-74796131ede8@oracle.com> <0a5a64a1-e203-16b4-96bd-497ed7eda700@oracle.com> <6f9b273c-cd3f-158e-6e67-c82b112c4a43@oracle.com> Message-ID: On 17/06/2020 14:47, Erik Joelsson wrote: > > > I've tried to make this clearer in the comment. > > http://cr.openjdk.java.net/~erikj/8213214/webrev.02/ I would prefer if you could move the notes/comment from L125-157 to the bug report to avoid cluttering the test. Instead, the test just needs a one line comment to say that toRealPath does not work with environments using the legacy subst mechanism. In the mean-time, I think Brian is trying to page in list of consistency issues that adding support for subst introduces. -Alan From forax at univ-mlv.fr Wed Jun 17 14:47:30 2020 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Wed, 17 Jun 2020 16:47:30 +0200 (CEST) Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: <622aa076-b5d6-1e26-4966-6ff2420f25b4@gmail.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <1522316544.1652270.1592348771671.JavaMail.zimbra@u-pem.fr> <622aa076-b5d6-1e26-4966-6ff2420f25b4@gmail.com> Message-ID: <226395162.2170526.1592405250910.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Peter Levart" > ?: "Remi Forax" > Cc: "core-libs-dev" , "Chris Hegarty" > Envoy?: Mercredi 17 Juin 2020 07:37:16 > Objet: Re: RFR: 8247532: Records deserialization is slow > Hi Remi, > > > The keys used are based on the ordered names and types of "fields" in > the ObjectStreamClass that has been deserialized as part of the object > stream for the record "type" being deserialized. So for each record type > (Class) there can be several distinct > ObjectStreamClasses deserialized in the same VM that were produced by > serializing different versions of record types in different VMs or > ClassLoader(s) and that may vary in the names and/or types and/or order > of fields. And since the MethodHandle chain of transformations that is > being used for a particular ObjectStreamClass is dependent on the > ordered names and types of ObjectStreamField(s) of deserialized > ObjectStreamClass, the key is based on that too. > > You may ask why not simply add a MethodHandle field to the > ObjectStreamClass instance that is deserialized and therefore unique? > Well, such ObjectStreamClass only lasts for one "session" of > ObjectStream deserialization, so such caching wouldn't be efficient. But > common parts of that ObjectStreamClass that are only dependent on the > current local-VM type are cached in a special instance of > ObjectStreamClass and that's where I put a cache of deserializaiton > constructors in the form of a Map. Ok, i see, you want to cache all permutations not only the one corresponding to the current record order. > > > Regards, Peter R?mi > > > On 6/17/20 1:06 AM, Remi Forax wrote: >> Hi Peter, >> is their a reason to not use a ClassValue using the record class >> as key instead of the more complex keys you are proposing ? >> >> R?mi >> >> ----- Mail original ----- >>> De: "Chris Hegarty" >>> ?: "Peter Levart" >>> Cc: "core-libs-dev" >>> Envoy?: Mardi 16 Juin 2020 17:15:03 >>> Objet: Re: RFR: 8247532: Records deserialization is slow >>> Hi Peter, >>> >>>> On 14 Jun 2020, at 17:28, Peter Levart wrote: >>>> >>>> ... >>>> >>>> [2] >>>> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.01/ >>>> >>> I think that this is very good. It?s clever to build a chain of method handles >>> that can be invoked with the stream field values. This is a nice separation >>> between the shape of the data and the actual stream data. >>> >>> The caching is on a per-stream-field shape basis, which should be consistent in >>> the common case, but of course is not always guaranteed to be the case, hence >>> the need for the cache. I think that this should be fine, since the >>> ObjectStreamClass ( that holds the cache ) is already itself cached as a weak >>> reference. But I did wonder if the size of this new cache should be limited. >>> Probably not worth the complexity unless it is an obvious issue. >>> >>> All the serailizable records tests pass successfully with your patch. Good. I >>> did however notice that there is just a single test, DefaultValuesTest, that >>> exercises different stream shapes for the same class in the stream. Would be >>> good to expand coverage in this area, but of course some lower-level >>> test-specific building blocks will be needed help build the specially crafted >>> byte streams - I can help with this. >>> >>> Overall I think that this change is good. >>> > >> -Chris. From shade at redhat.com Wed Jun 17 15:11:21 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Wed, 17 Jun 2020 17:11:21 +0200 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: On 6/15/20 11:58 PM, Mandy Chung wrote: > This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and > `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. Note this would break otherwise innocuous introspection for records, for example dumping the VM layout with JOL: http://openjdk.java.net/projects/code-tools/jol/ Amusingly, with the rest of Unsafe available the barn is still full open. Here's the sketch for the workaround: get the field list with getDeclaredFields, optionally guess the VM layout (the rules are not that hard, and JOL does it routinely), instantiate a Record with whatever arguments, poke the test patterns with Unsafe.put*, read them back from record components to verify. It would get me the same objectFieldOffset in a round-about way. And speaking from JOL maintainer standpoint, that one would be very tempting to do, because it would not depend on whatever protection shenanigans a particular JDK release tries to enforce. > This change impacts 3rd-party frameworks including 3rd-party > serialization framework that rely on core reflection `setAccessible` or > `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to > construct records but not using the canonical constructor. > These frameworks would need to be updated to construct records via its > canonical constructor as done by the Java serialization. Are we absolutely sure that what ObjectInputStream.readRecord() does fits the 3rd party serialization libraries? Does it work for them performance-wise? (Are we even sure about that for JDK itself?) Because if it is not, 3rd party lib maintainers would proceed to hacking in the "objectFieldOffset workaround" and we would get the cobra-effect-like strengthening of dependency on Unsafe quirks. > No change is made in JNI.? JNI could be considered to disallow > modification of final fields in records and hidden classes (static and > instance fields).? But JNI has superpower and the current spec already > allows to modify the value of a final static field even after it's > constant folded (via JNI SetField and SetStaticField), then > all bets are off. It is fun to consider JNI to be more powerful than Unsafe. This seems backwards. The intent to break Unsafe users might be defensible, but this power oddity is still quite odd. -- Thanks, -Aleksey From chris.hegarty at oracle.com Wed Jun 17 15:33:08 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 17 Jun 2020 16:33:08 +0100 Subject: RFR [15] 8247696: Incorrect tail computation for large segments in AbstractMemorySegmentImpl::mismatch Message-ID: <0B15B6AF-F196-4192-B448-F1955AE586A7@oracle.com> The MemorySegment::mismatch implementation added vectorized mismatch of long sizes. The implementation is trivial, but the starting point for a more optimized implementation, if needed. ArraysSupport::vectorizedMismatchLarge incorrectly returns the bitwise complement of the offset of the first mismatch, where it should return the bitwise complement of the number of remaining pairs of elements to be checked in the tail of the two arrays. The AbstractMemorySegmentImpl::mismatch masked this problem, since it seamlessly compared the remaining tail, which is larger than it should be. Webrev: https://cr.openjdk.java.net/~chegar/8247696/webrev.00/ I updated the exiting BulkOps micro-benchmark to cover mismatch. Here are the results, compared to ByteBuffer::mismatch, on my machine: Benchmark Mode Cnt Score Error Units BulkOps.mismatch_large_bytebuffer avgt 30 740186.973 ? 119314.207 ns/op BulkOps.mismatch_large_segment avgt 30 683475.305 ? 76355.043 ns/op BulkOps.mismatch_small_bytebuffer avgt 30 7.367 ? 0.523 ns/op BulkOps.mismatch_small_segment avgt 30 4.140 ? 0.602 ns/op -Chris. From erik.joelsson at oracle.com Wed Jun 17 15:38:02 2020 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 17 Jun 2020 08:38:02 -0700 Subject: RFR: JDK-8213214: Set -Djava.io.tmpdir= when running tests In-Reply-To: References: <2aa2e962-c630-60cd-6b0b-74796131ede8@oracle.com> <0a5a64a1-e203-16b4-96bd-497ed7eda700@oracle.com> <6f9b273c-cd3f-158e-6e67-c82b112c4a43@oracle.com> Message-ID: <6f76a414-1efa-82d3-c752-60521b17f3b9@oracle.com> On 2020-06-17 07:43, Alan Bateman wrote: > On 17/06/2020 14:47, Erik Joelsson wrote: >> >> >> I've tried to make this clearer in the comment. >> >> http://cr.openjdk.java.net/~erikj/8213214/webrev.02/ > I would prefer if you could move the notes/comment from L125-157 to > the bug report to avoid cluttering the test. Instead, the test just > needs a one line comment to say that toRealPath does not work with > environments using the legacy subst mechanism. In the mean-time, I > think Brian is trying to page in list of consistency issues that > adding support for subst introduces. > Ah, misunderstood you then. I added a comment to JDK-8213216 with more details and shortened the comment in the test: http://cr.openjdk.java.net/~erikj/8213214/webrev.03/ /Erik From alexey.semenyuk at oracle.com Wed Jun 17 16:49:54 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Wed, 17 Jun 2020 12:49:54 -0400 Subject: RFR: JDK-8264244: BasicShortcutHintTest shortcut can not be found In-Reply-To: <2a24b86e-7b42-6e82-fa22-11200f0bfe43@oracle.com> References: <4efc0137-d2fb-cf69-1fac-3e871d5eedc4@oracle.com> <194ea4ad-e069-63b4-82d2-3ebcb2261f6f@oracle.com> <2a24b86e-7b42-6e82-fa22-11200f0bfe43@oracle.com> Message-ID: <9b9822e4-f929-3738-e578-598fa59e6ada@oracle.com> Yeh, you are right. It was copy/paste error. Thank you for the correction! - Alexey On 6/16/2020 6:13 PM, Alexander Matveev wrote: > Hi Alexey, > > Looks good. I think you got links and bug ID incorrect. It should be > JDK-8246244 and you have 8264244. Links also does not work. Working > links are: > https://bugs.openjdk.java.net/browse/JDK-8246244 > http://cr.openjdk.java.net/~asemenyuk/8246244/webrev.00/ > > Thanks, > Alexander > > On 6/16/2020 2:56 PM, Alexey Semenyuk wrote: >> Please review fix [2] for jpackage bug [1]. >> >> The fix is to put value of `Exec` property of .desktop files >> generated by jpackage in double quotes if the value of the propery >> contains whitespace characters. >> >> - Alexey >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8264244 >> >> [2]?http://cr.openjdk.java.net/~asemenyuk/8264244/webrev.00 >> > From mandy.chung at oracle.com Wed Jun 17 16:50:44 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 17 Jun 2020 09:50:44 -0700 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: On 6/17/20 8:11 AM, Aleksey Shipilev wrote: > On 6/15/20 11:58 PM, Mandy Chung wrote: >> This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and >> `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. > Note this would break otherwise innocuous introspection for records, for example dumping the VM > layout with JOL: > http://openjdk.java.net/projects/code-tools/jol/ To me, JOL needs a supported API in the serviceability area to introspect a field layout.? JOL will not work for inline types without adding new APIs. sun.misc.Unsafe is an unsupported API.?? My proposal for sun.misc.Unsafe (in fact jdk.unsupported module) is to keep the support for legacy use but not necessarily for new features. > > Amusingly, with the rest of Unsafe available the barn is still full open. Here's the sketch for the > workaround: get the field list with getDeclaredFields, optionally guess the VM layout (the rules are > not that hard, and JOL does it routinely), instantiate a Record with whatever arguments, poke the > test patterns with Unsafe.put*, read them back from record components to verify. It would get me the > same objectFieldOffset in a round-about way. I'm aware of these workarounds in the wild.?? As sun.misc.Unsafe is JDK unsupported API, this patch does not attempt to implement a complete solution but adds an obvious notification informing the frameworks that `sun.misc.Unsafe::objectFieldOffset` and `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. Unsafe is unsafe!?? Use it at your own risk. > And speaking from JOL maintainer standpoint, that one would be very tempting to do, because it would > not depend on whatever protection shenanigans a particular JDK release tries to enforce. > >> This change impacts 3rd-party frameworks including 3rd-party >> serialization framework that rely on core reflection `setAccessible` or >> `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to >> construct records but not using the canonical constructor. >> These frameworks would need to be updated to construct records via its >> canonical constructor as done by the Java serialization. > Are we absolutely sure that what ObjectInputStream.readRecord() does fits the 3rd party > serialization libraries? Does it work for them performance-wise? There should be performance improvement opportunity (see Peter Levart's good work - JDK-8247532 [1]) > (Are we even sure about that for JDK itself?) Java serialization support for records use its canonical constructor.? (I'm not sure what you tried to point out by this). > Because if it is not, 3rd party lib maintainers would proceed to hacking in the > "objectFieldOffset workaround" and we would get the cobra-effect-like strengthening of dependency on > Unsafe quirks. This is exactly why we request this for 15 so that 3rd-party lib maintainers can prototype and send feedback.? We love to understand any stumbling block and work together to resolve any issue before records exit preview. > >> No change is made in JNI.? JNI could be considered to disallow >> modification of final fields in records and hidden classes (static and >> instance fields).? But JNI has superpower and the current spec already >> allows to modify the value of a final static field even after it's >> constant folded (via JNI SetField and SetStaticField), then >> all bets are off. > It is fun to consider JNI to be more powerful than Unsafe. This seems backwards. The intent to break > Unsafe users might be defensible, but this power oddity is still quite odd. > I think you misread the message.? There is no claim whether JNI or Unsafe is more powerful. JNI is a supported API.?? The above explains why I propose no spec change to make in JNI.? OTOH jdk.unsupported is unsupported but has more love by frameworks to avoid writing in native.? Adding a check in some sun.misc.Unsafe APIs (even it can be hacked to workaround it) is a clear notification what it does not intend to support. Hope this helps. Mandy [1] https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-June/067223.html From mandy.chung at oracle.com Wed Jun 17 17:26:11 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 17 Jun 2020 10:26:11 -0700 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: <45d5c949-5cd0-a9ee-b747-17fd10c683fe@oracle.com> Hi Vladimir, Thanks for the review and feedback. FWIW.? The spec of java.lang.reflect.AccessibleObject and Field specifies which final fields can be modified or not.? It use the *non-modifiable* term and JVM can rely on. I will keep the patch as is.? I'm happy to clean this up when a more disciplined approach is coming. Mandy On 6/17/20 4:13 AM, Vladimir Ivanov wrote: > Looks good! > > I welcome and fully support such hardening of final field handling for > new language features. > > Minor comment on naming: "trusted final field" is an JVM > implementation detail and I'd prefer to keep it internal to JVM. (I > hope it'll eventually go away and will be replaced with a more > disciplined approach.) > > Probably, a better alternative is a name that clearly communicates > that the JVM expects/allows modifications of a final field. Or, at > least, that it's *the JVM* which "trusts" the contents of a final > field after its initialization is over > (vmTrustedFinal/isVMTrustedFinalField). > > But I'm fine with keeping the patch as is. > > Best regards, > Vladimir Ivanov > > On 16.06.2020 00:58, Mandy Chung wrote: >> This patch is joint contribution from Christoph Dreis [1] and me. >> >> Webrev: >> http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247444/webrev.00/ >> CSR: https://bugs.openjdk.java.net/browse/JDK-8247517 >> >> This proposes to make final fields in records notmodifiable via >> reflection.? Field::set throws IAE if a Field is not modifiable. >> Thecurrent specification specifies the following Fields not modifiable: >> - static final fields in any class >> - final fields in a hidden class >> >> The spec of Field::set is changed to specify that records are not >> modifiable via reflection. >> ??Noe that no change in Field::setAccessible(true), i.e. it will >> succeed to allow existing frameworks to have read access to final >> fields in records.? Just no write access. >> >> VarHandle does not support write access if it's a static final field >> or an instance final field. >> >> This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and >> `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. >> >> No change is made in JNI.? JNI could be considered to disallow >> modification of final fields in records and hidden classes (static >> and instance fields).? But JNI has superpower and the current spec >> already allows to modify the value of a final static field even after >> it's constant folded (via JNI SetField and >> SetStaticField), then all bets are off.? This should be >> re-visited when we consider "final is truly final" for all classes. >> >> Make final fields in records not modifiable via reflection enables >> JIT optimization as these final fields are effectively truly final. >> >> This change impacts 3rd-party frameworks including 3rd-party >> serialization framework that rely on core reflection `setAccessible` >> or `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to >> construct records but not using the canonical constructor. >> These frameworks would need to be updated to construct records via >> its canonical constructor as done by the Java serialization. >> >> I see this change gives a good opportunity to engage the maintainers >> of the serialization frameworks and work together to support new >> features including records, inline classes and the new serialization >> mechanism and which I think it is worth the investment. >> >> This is a low risk enhancement.? I'd like to request approval for a >> late enhancement in JDK 15.? It extends the pre-existing code path >> with refactoring the hidden classes to prepare for new kinds of >> classes with trusted final fields.? The change is straight-forward. >> >> Can this wait to integrate in JDK 16? >> >> ?? It's important to get this enhancement in when record is a preview >> feature that we can get feedback and give time to 3rd-party >> serialization frameworks to look into adding the support for records. >> If we delayed this change in 16 and records exit preview, it would be >> bad for frameworks if they verify that they support records in 15 but >> fail in 16.? OTOH the risk of this patch is low. >> >> Performance Impact >> >> I addressed the performance concern I raised earlier myself. For >> reflection, VM creates the reflective Field objects and fills in >> MemberName when resolving a member.? VM will tag if this >> Field/MemberName is trusted final field.? I think this is a cleaner >> approach rather than in each place to check for final static and >> final fields in hidden or record class to determine if it has write >> access or not. >> >> `sun.misc.Unsafe` does not use Field::isTrustedFinalField because (1) >> Unsafe has been allowing access of static final fields of any classes >> but isTrustedFinalField is not limited to instance fields (2) Unsafe >> disallows access to all fields in a hidden class (not limited to >> trusted final fields).? So it follows the precedence and simply >> checks if the declaring class is a record. `Class::isRecord` calls >> `Class::getSuperclass` to check if it's a subtype of `Record`.? As >> `Class::getSuperclass` is intrinsified, the call on isRecord on a >> normal class is fast. Christoph has contributed the microbenchmarks >> that confirm that no performance regression. >> >> Thanks >> Mandy >> [1] >> https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-June/040096.html >> From martinrb at google.com Wed Jun 17 18:44:54 2020 From: martinrb at google.com (Martin Buchholz) Date: Wed, 17 Jun 2020 11:44:54 -0700 Subject: RFR: 8247782: typos in java.math Message-ID: 8247782: typos in java.math https://cr.openjdk.java.net/~martin/webrevs/jdk/math-spelling/ https://bugs.openjdk.java.net/browse/JDK-8247782 From Roger.Riggs at oracle.com Wed Jun 17 18:47:52 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Wed, 17 Jun 2020 14:47:52 -0400 Subject: RFR: 8247782: typos in java.math In-Reply-To: References: Message-ID: Looks fine. On 6/17/20 2:44 PM, Martin Buchholz wrote: > 8247782: typos in java.math > https://cr.openjdk.java.net/~martin/webrevs/jdk/math-spelling/ > https://bugs.openjdk.java.net/browse/JDK-8247782 From Alan.Bateman at oracle.com Wed Jun 17 18:49:42 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 17 Jun 2020 19:49:42 +0100 Subject: RFR: JDK-8213214: Set -Djava.io.tmpdir= when running tests In-Reply-To: <6f76a414-1efa-82d3-c752-60521b17f3b9@oracle.com> References: <2aa2e962-c630-60cd-6b0b-74796131ede8@oracle.com> <0a5a64a1-e203-16b4-96bd-497ed7eda700@oracle.com> <6f9b273c-cd3f-158e-6e67-c82b112c4a43@oracle.com> <6f76a414-1efa-82d3-c752-60521b17f3b9@oracle.com> Message-ID: <33b7aa03-a6a2-48c0-9615-5e2688c90774@oracle.com> On 17/06/2020 16:38, Erik Joelsson wrote: > > Ah, misunderstood you then. I added a comment to JDK-8213216 with more > details and shortened the comment in the test: > > http://cr.openjdk.java.net/~erikj/8213214/webrev.03/ Looks okay as a workaround until we figure out what to do with subst. -Alan. From lance.andersen at oracle.com Wed Jun 17 19:01:21 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 17 Jun 2020 15:01:21 -0400 Subject: RFR: 8247782: typos in java.math In-Reply-To: References: Message-ID: Looks fine. Please add the label noreg-doc to make the automatic verification happy :-) Best Lance > On Jun 17, 2020, at 2:44 PM, Martin Buchholz wrote: > > 8247782: typos in java.math > https://cr.openjdk.java.net/~martin/webrevs/jdk/math-spelling/ > https://bugs.openjdk.java.net/browse/JDK-8247782 Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From joe.darcy at oracle.com Wed Jun 17 19:14:32 2020 From: joe.darcy at oracle.com (Joe Darcy) Date: Wed, 17 Jun 2020 12:14:32 -0700 Subject: RFR: 8247782: typos in java.math In-Reply-To: References: Message-ID: <4bc88646-fd2a-8e50-2820-74f11d98182f@oracle.com> Looks good Martin; thanks for fixing, -Joe On 6/17/2020 11:44 AM, Martin Buchholz wrote: > 8247782: typos in java.math > https://cr.openjdk.java.net/~martin/webrevs/jdk/math-spelling/ > https://bugs.openjdk.java.net/browse/JDK-8247782 From martinrb at google.com Wed Jun 17 19:22:28 2020 From: martinrb at google.com (Martin Buchholz) Date: Wed, 17 Jun 2020 12:22:28 -0700 Subject: RFR: 8247706: Unintentional use of new Date(year...) with absolute year Message-ID: No actual bugs fixed, but good to stamp out all instances of new Date(1970) 8247706: Unintentional use of new Date(year...) with absolute year https://cr.openjdk.java.net/~martin/webrevs/jdk/Date-constructor-1900/ https://bugs.openjdk.java.net/browse/JDK-8247706 The "1990" below looks like a simple typo: - Date d = new Date(2003 - 1990, OCTOBER, 31); + Date d = new Date(2003 - 1900, OCTOBER, 31); From roger.riggs at oracle.com Wed Jun 17 19:30:57 2020 From: roger.riggs at oracle.com (Roger Riggs) Date: Wed, 17 Jun 2020 15:30:57 -0400 Subject: RFR: 8247706: Unintentional use of new Date(year...) with absolute year In-Reply-To: References: Message-ID: <170adcd8-3857-ec70-c1e4-b6085dae8563@oracle.com> Hi Martin, Looks fine, I'm embarrassed by the one in the java.time tests, we should have known better. Thanks, Roger On 6/17/20 3:22 PM, Martin Buchholz wrote: > No actual bugs fixed, but good to stamp out all instances of new Date(1970) > > 8247706: Unintentional use of new Date(year...) with absolute year > https://cr.openjdk.java.net/~martin/webrevs/jdk/Date-constructor-1900/ > https://bugs.openjdk.java.net/browse/JDK-8247706 > > The "1990" below looks like a simple typo: > > - Date d = new Date(2003 - 1990, OCTOBER, 31); > + Date d = new Date(2003 - 1900, OCTOBER, 31); From naoto.sato at oracle.com Wed Jun 17 19:33:33 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Wed, 17 Jun 2020 12:33:33 -0700 Subject: RFR: 8247706: Unintentional use of new Date(year...) with absolute year In-Reply-To: <170adcd8-3857-ec70-c1e4-b6085dae8563@oracle.com> References: <170adcd8-3857-ec70-c1e4-b6085dae8563@oracle.com> Message-ID: <09f2dcb9-e8be-ccac-0ffb-17286ebb285e@oracle.com> Thanks, Martin. +1 When you do the push, please update the copyright year to 2020. Naoto On 6/17/20 12:30 PM, Roger Riggs wrote: > Hi Martin, > > Looks fine, > > I'm embarrassed by the one in the java.time tests, we should have known > better. > > Thanks, Roger > > > On 6/17/20 3:22 PM, Martin Buchholz wrote: >> No actual bugs fixed, but good to stamp out all instances of new >> Date(1970) >> >> 8247706: Unintentional use of new Date(year...) with absolute year >> https://cr.openjdk.java.net/~martin/webrevs/jdk/Date-constructor-1900/ >> https://bugs.openjdk.java.net/browse/JDK-8247706 >> >> The "1990" below looks like a simple typo: >> >> -??????? Date d = new Date(2003 - 1990, OCTOBER, 31); >> +??????? Date d = new Date(2003 - 1900, OCTOBER, 31); > From martinrb at google.com Wed Jun 17 19:38:26 2020 From: martinrb at google.com (Martin Buchholz) Date: Wed, 17 Jun 2020 12:38:26 -0700 Subject: RFR: 8247782: typos in java.math In-Reply-To: <4bc88646-fd2a-8e50-2820-74f11d98182f@oracle.com> References: <4bc88646-fd2a-8e50-2820-74f11d98182f@oracle.com> Message-ID: Thanks! Submitted. On Wed, Jun 17, 2020 at 12:14 PM Joe Darcy wrote: > > Looks good Martin; thanks for fixing, > > -Joe > > On 6/17/2020 11:44 AM, Martin Buchholz wrote: > > 8247782: typos in java.math > > https://cr.openjdk.java.net/~martin/webrevs/jdk/math-spelling/ > > https://bugs.openjdk.java.net/browse/JDK-8247782 From martinrb at google.com Wed Jun 17 19:48:56 2020 From: martinrb at google.com (Martin Buchholz) Date: Wed, 17 Jun 2020 12:48:56 -0700 Subject: RFR: 8247706: Unintentional use of new Date(year...) with absolute year In-Reply-To: <09f2dcb9-e8be-ccac-0ffb-17286ebb285e@oracle.com> References: <170adcd8-3857-ec70-c1e4-b6085dae8563@oracle.com> <09f2dcb9-e8be-ccac-0ffb-17286ebb285e@oracle.com> Message-ID: Thanks! Pushed. On Wed, Jun 17, 2020 at 12:37 PM wrote: > > Thanks, Martin. +1 > > When you do the push, please update the copyright year to 2020. > > Naoto > > On 6/17/20 12:30 PM, Roger Riggs wrote: > > Hi Martin, > > > > Looks fine, > > > > I'm embarrassed by the one in the java.time tests, we should have known > > better. > > > > Thanks, Roger > > > > > > On 6/17/20 3:22 PM, Martin Buchholz wrote: > >> No actual bugs fixed, but good to stamp out all instances of new > >> Date(1970) > >> > >> 8247706: Unintentional use of new Date(year...) with absolute year > >> https://cr.openjdk.java.net/~martin/webrevs/jdk/Date-constructor-1900/ > >> https://bugs.openjdk.java.net/browse/JDK-8247706 > >> > >> The "1990" below looks like a simple typo: > >> > >> - Date d = new Date(2003 - 1990, OCTOBER, 31); > >> + Date d = new Date(2003 - 1900, OCTOBER, 31); > > From paul.sandoz at oracle.com Wed Jun 17 20:56:38 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 17 Jun 2020 13:56:38 -0700 Subject: RFR [15] 8247696: Incorrect tail computation for large segments in AbstractMemorySegmentImpl::mismatch In-Reply-To: <0B15B6AF-F196-4192-B448-F1955AE586A7@oracle.com> References: <0B15B6AF-F196-4192-B448-F1955AE586A7@oracle.com> Message-ID: Hi Chris, AbstractMemorySegmentImpl ? In vectorizedMismatchLarge: 163 if (remaining > 7) 164 throw new InternalError("remaining greater than 7: " + remaining); 165 i = length - remaining; 166 } Should this check be an assert? ? This fix prompted me to think more deeply about the implementation of vectorizedMismatchLarge and its use of vectorizedMismatch. Sorry, I should have thought about this more throughly earlier on. We need to refine the approach, not for this patch, but something to follow up after. I think there are two issues. 1) The intrinsic method vectorizedMismatch could potentially bomb out at any point and return the bitwise compliment of the remaining number of elements to check. Obviously, there is no point doing that arbitrarily but a stub implementation for, say, x86 AVX-512 might decide bomb out for < 64 remaining elements, rather than apply vector operations on smaller vector sizes or use a scalar loop. It does not today, but I think we should guard against that potentially happening, otherwise bad things can happen. I think the loop should exit when the last sub-range has been checked. We should rely on other tests to ensure the intrinsic method is operating efficiently. 2) This method only works when operating on byte arrays. It will not work correctly if operating on short or long arrays, since we are not adjusting the length and offsets accordingly by the scale. It's probably easier to just rename this as vectorizedMismatchLargeForBytes and drop the log2ArrayIndexScale argument. Then expand later if need be. I still think the method rightly belongs in ArraysSupport. Paul. > On Jun 17, 2020, at 8:33 AM, Chris Hegarty wrote: > > The MemorySegment::mismatch implementation added vectorized mismatch of long sizes. The implementation is trivial, but the starting point for a more optimized implementation, if needed. ArraysSupport::vectorizedMismatchLarge incorrectly returns the bitwise complement of the offset of the first mismatch, where it should return the bitwise complement of the number of remaining pairs of elements to be checked in the tail of the two arrays. The AbstractMemorySegmentImpl::mismatch masked this problem, since it seamlessly compared the remaining tail, which is larger than it should be. > > Webrev: > https://cr.openjdk.java.net/~chegar/8247696/webrev.00/ > > I updated the exiting BulkOps micro-benchmark to cover mismatch. Here are the results, compared to ByteBuffer::mismatch, on my machine: > > Benchmark Mode Cnt Score Error Units > BulkOps.mismatch_large_bytebuffer avgt 30 740186.973 ? 119314.207 ns/op > BulkOps.mismatch_large_segment avgt 30 683475.305 ? 76355.043 ns/op > BulkOps.mismatch_small_bytebuffer avgt 30 7.367 ? 0.523 ns/op > BulkOps.mismatch_small_segment avgt 30 4.140 ? 0.602 ns/op > > > -Chris. From sakatakui at oss.nttdata.com Wed Jun 17 22:46:14 2020 From: sakatakui at oss.nttdata.com (Koichi Sakata) Date: Thu, 18 Jun 2020 07:46:14 +0900 Subject: Avoid some GCC 10.X warnings in HotSpot Message-ID: <00995823-80f2-539d-aeb0-f3751dd43969@oss.nttdata.com> Hi all, I tried to build OpenJDK fastdebug with GCC 10.1 on Ubuntu 18.04, but I saw some compiler warnings as follows: In file included from /home/jyukutyo/code/jdk/src/hotspot/share/classfile/systemDictionary.hpp:31, from /home/jyukutyo/code/jdk/src/hotspot/share/classfile/javaClasses.hpp:28, from /home/jyukutyo/code/jdk/src/hotspot/share/precompiled/precompiled.hpp:35: In member function 'void Symbol::byte_at_put(int, u1)', inlined from 'Symbol::Symbol(const u1*, int, int)' at /home/jyukutyo/code/jdk/src/hotspot/share/oops/symbol.cpp:55:16: /home/jyukutyo/code/jdk/src/hotspot/share/oops/symbol.hpp:130:18: error: writing 1 byte into a region of size 0 [-Werror=stringop-overflow=] 130 | _body[index] = value; | ~~~~~~~~~~~~~^~~~~~~ /home/jyukutyo/code/jdk/src/java.base/share/native/libfdlibm/k_standard.c: In function '__j__kernel_standard': /home/jyukutyo/code/jdk/src/java.base/share/native/libfdlibm/k_standard.c:743:19: error: 'exc.retval' may be used uninitialized in this function [-Werror=maybe-uninitialized] 743 | return exc.retval; | ~~~^~~~~~~ In file included from /usr/include/string.h:494, from /home/jyukutyo/code/jdk/src/java.base/unix/native/libnet/NetworkInterface.c:30: In function 'strncpy', inlined from 'getFlags' at /home/jyukutyo/code/jdk/src/java.base/unix/native/libnet/NetworkInterface.c:1362:5, inlined from 'addif' at /home/jyukutyo/code/jdk/src/java.base/unix/native/libnet/NetworkInterface.c:974:13: /usr/include/x86_64-linux-gnu/bits/string_fortified.h:106:10: error: '__builtin_strncpy' output may be truncated copying 15 bytes from a string of length 15 [-Werror=stringop-truncation] 106 | return __builtin___strncpy_chk (__dest, __src, __len, __bos (__dest)); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ I can resolve them with the following patch. I believe it fixes those potential bugs, so I'd like to contribute it. (Our company has signed OCA.) Thanks, Koichi ===== PATCH ===== diff -r 20d92fe3ac52 src/hotspot/share/oops/symbol.cpp --- a/src/hotspot/share/oops/symbol.cpp Tue Jun 16 03:16:41 2020 +0000 +++ b/src/hotspot/share/oops/symbol.cpp Thu Jun 18 07:08:50 2020 +0900 @@ -50,9 +50,10 @@ Symbol::Symbol(const u1* name, int length, int refcount) { _hash_and_refcount = pack_hash_and_refcount((short)os::random(), refcount); _length = length; - _body[0] = 0; // in case length == 0 - for (int i = 0; i < length; i++) { - byte_at_put(i, name[i]); + if (length == 0) { + _body[0] = 0; + } else { + memcpy(_body, name, length); } } diff -r 20d92fe3ac52 src/hotspot/share/oops/symbol.hpp --- a/src/hotspot/share/oops/symbol.hpp Tue Jun 16 03:16:41 2020 +0000 +++ b/src/hotspot/share/oops/symbol.hpp Thu Jun 18 07:08:50 2020 +0900 @@ -125,11 +125,6 @@ return (int)heap_word_size(byte_size(length)); } - void byte_at_put(int index, u1 value) { - assert(index >=0 && index < length(), "symbol index overflow"); - _body[index] = value; - } - Symbol(const u1* name, int length, int refcount); void* operator new(size_t size, int len) throw(); void* operator new(size_t size, int len, Arena* arena) throw(); diff -r 20d92fe3ac52 src/java.base/share/native/libfdlibm/k_standard.c --- a/src/java.base/share/native/libfdlibm/k_standard.c Tue Jun 16 03:16:41 2020 +0000 +++ b/src/java.base/share/native/libfdlibm/k_standard.c Thu Jun 18 07:08:50 2020 +0900 @@ -739,6 +739,10 @@ errno = EDOM; } break; + default: + exc.retval = zero; + errno = EINVAL; + break; } return exc.retval; } diff -r 20d92fe3ac52 src/java.base/unix/native/libnet/NetworkInterface.c --- a/src/java.base/unix/native/libnet/NetworkInterface.c Tue Jun 16 03:16:41 2020 +0000 +++ b/src/java.base/unix/native/libnet/NetworkInterface.c Thu Jun 18 07:08:50 2020 +0900 @@ -1296,7 +1296,10 @@ static int getIndex(int sock, const char *name) { struct ifreq if2; memset((char *)&if2, 0, sizeof(if2)); - strncpy(if2.ifr_name, name, sizeof(if2.ifr_name) - 1); + if (sizeof(if2.ifr_name) < sizeof(name)) { + return -1; + } + strcpy(if2.ifr_name, name); if (ioctl(sock, SIOCGIFINDEX, (char *)&if2) < 0) { return -1; @@ -1359,7 +1362,10 @@ static int getFlags(int sock, const char *ifname, int *flags) { struct ifreq if2; memset((char *)&if2, 0, sizeof(if2)); - strncpy(if2.ifr_name, ifname, sizeof(if2.ifr_name) - 1); + if (sizeof(if2.ifr_name) < sizeof(ifname)) { + return -1; + } + strcpy(if2.ifr_name, ifname); if (ioctl(sock, SIOCGIFFLAGS, (char *)&if2) < 0) { return -1; From serguei.spitsyn at oracle.com Thu Jun 18 02:34:34 2020 From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com) Date: Wed, 17 Jun 2020 19:34:34 -0700 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: <18a21b47-9815-270d-3859-7d908d3fc165@oracle.com> Hi Mandy, This looks good from the Serviceability point of view. > No change is made in JNI.? JNI could be considered to disallow modification of > final fields in records and hidden classes (static and instance fields). > But JNI has superpower and the current spec already allows to modify > the value of a final static field even after it's constant folded > (via JNI SetField and SetStaticField), then all bets are off. > This should be re-visited when we consider "final is truly final" for all classes. This can potentially impact the JDWP agent as it uses the JNI to set fields values. Please, see ObjectReference#setValue: https://docs.oracle.com/en/java/javase/14/docs/api/jdk.jdi/com/sun/jdi/ObjectReference.html#setValue(com.sun.jdi.Field,com.sun.jdi.Value) Thanks, Serguei On 6/15/20 14:58, Mandy Chung wrote: > This patch is joint contribution from Christoph Dreis [1] and me. > > Webrev: > http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247444/webrev.00/ > CSR: https://bugs.openjdk.java.net/browse/JDK-8247517 > > This proposes to make final fields in records notmodifiable via > reflection.? Field::set throws IAE if a Field is not modifiable. > Thecurrent specification specifies the following Fields not modifiable: > - static final fields in any class > - final fields in a hidden class > > The spec of Field::set is changed to specify that records are not > modifiable via reflection. > ?Noe that no change in Field::setAccessible(true), i.e. it will > succeed to allow existing frameworks to have read access to final > fields in records.? Just no write access. > > VarHandle does not support write access if it's a static final field > or an instance final field. > > This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and > `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. > > No change is made in JNI.? JNI could be considered to disallow > modification of final fields in records and hidden classes (static and > instance fields).? But JNI has superpower and the current spec already > allows to modify the value of a final static field even after it's > constant folded (via JNI SetField and SetStaticField), > then all bets are off.? This should be re-visited when we consider > "final is truly final" for all classes. > > Make final fields in records not modifiable via reflection enables JIT > optimization as these final fields are effectively truly final. > > This change impacts 3rd-party frameworks including 3rd-party > serialization framework that rely on core reflection `setAccessible` > or `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to > construct records but not using the canonical constructor. > These frameworks would need to be updated to construct records via its > canonical constructor as done by the Java serialization. > > I see this change gives a good opportunity to engage the maintainers > of the serialization frameworks and work together to support new > features including records, inline classes and the new serialization > mechanism and which I think it is worth the investment. > > This is a low risk enhancement.? I'd like to request approval for a > late enhancement in JDK 15.? It extends the pre-existing code path > with refactoring the hidden classes to prepare for new kinds of > classes with trusted final fields.? The change is straight-forward. > > Can this wait to integrate in JDK 16? > > ? It's important to get this enhancement in when record is a preview > feature that we can get feedback and give time to 3rd-party > serialization frameworks to look into adding the support for records.? > If we delayed this change in 16 and records exit preview, it would be > bad for frameworks if they verify that they support records in 15 but > fail in 16.? OTOH the risk of this patch is low. > > Performance Impact > > I addressed the performance concern I raised earlier myself.? For > reflection, VM creates the reflective Field objects and fills in > MemberName when resolving a member.? VM will tag if this > Field/MemberName is trusted final field.? I think this is a cleaner > approach rather than in each place to check for final static and final > fields in hidden or record class to determine if it has write access > or not. > > `sun.misc.Unsafe` does not use Field::isTrustedFinalField because (1) > Unsafe has been allowing access of static final fields of any classes > but isTrustedFinalField is not limited to instance fields (2) Unsafe > disallows access to all fields in a hidden class (not limited to > trusted final fields).? So it follows the precedence and simply checks > if the declaring class is a record. `Class::isRecord` calls > `Class::getSuperclass` to check if it's a subtype of `Record`.? As > `Class::getSuperclass` is intrinsified, the call on isRecord on a > normal class is fast. Christoph has contributed the microbenchmarks > that confirm that no performance regression. > > Thanks > Mandy > [1] > https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-June/040096.html From mandy.chung at oracle.com Thu Jun 18 02:42:13 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 17 Jun 2020 19:42:13 -0700 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: <18a21b47-9815-270d-3859-7d908d3fc165@oracle.com> References: <18a21b47-9815-270d-3859-7d908d3fc165@oracle.com> Message-ID: <1a95d2f6-e070-aced-aeeb-3e699bd75ad4@oracle.com> On 6/17/20 7:34 PM, serguei.spitsyn at oracle.com wrote: > Hi Mandy, > > This looks good from the Serviceability point of view. > > > No change is made in JNI.? JNI could be considered to disallow > modification of > > final fields in records and hidden classes (static and instance > fields). > > But JNI has superpower and the current spec already allows to modify > > the value of a final static field even after it's constant folded > > (via JNI SetField and SetStaticField), then all bets are > off. > > This should be re-visited when we consider "final is truly final" > for all classes. > > This can potentially impact the JDWP agent as it uses the JNI to set > fields values. > Please, see ObjectReference#setValue: > https://docs.oracle.com/en/java/javase/14/docs/api/jdk.jdi/com/sun/jdi/ObjectReference.html#setValue(com.sun.jdi.Field,com.sun.jdi.Value) > > Thanks for pointing out this API.?? I will file a RFE to follow this up. Mandy > Thanks, > Serguei > > > On 6/15/20 14:58, Mandy Chung wrote: >> This patch is joint contribution from Christoph Dreis [1] and me. >> >> Webrev: >> http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247444/webrev.00/ >> CSR: https://bugs.openjdk.java.net/browse/JDK-8247517 >> >> This proposes to make final fields in records notmodifiable via >> reflection.? Field::set throws IAE if a Field is not modifiable. >> Thecurrent specification specifies the following Fields not modifiable: >> - static final fields in any class >> - final fields in a hidden class >> >> The spec of Field::set is changed to specify that records are not >> modifiable via reflection. >> ?Noe that no change in Field::setAccessible(true), i.e. it will >> succeed to allow existing frameworks to have read access to final >> fields in records.? Just no write access. >> >> VarHandle does not support write access if it's a static final field >> or an instance final field. >> >> This patch also proposes `sun.misc.Unsafe::objectFieldOffset` and >> `sun.misc.Unsafe::staticField{Offset/Base}` not to support records. >> >> No change is made in JNI.? JNI could be considered to disallow >> modification of final fields in records and hidden classes (static >> and instance fields).? But JNI has superpower and the current spec >> already allows to modify the value of a final static field even after >> it's constant folded (via JNI SetField and >> SetStaticField), then all bets are off.? This should be >> re-visited when we consider "final is truly final" for all classes. >> >> Make final fields in records not modifiable via reflection enables >> JIT optimization as these final fields are effectively truly final. >> >> This change impacts 3rd-party frameworks including 3rd-party >> serialization framework that rely on core reflection `setAccessible` >> or `sun.misc.Unsafe::allocateInstance` and `objectFieldOffset` etc to >> construct records but not using the canonical constructor. >> These frameworks would need to be updated to construct records via >> its canonical constructor as done by the Java serialization. >> >> I see this change gives a good opportunity to engage the maintainers >> of the serialization frameworks and work together to support new >> features including records, inline classes and the new serialization >> mechanism and which I think it is worth the investment. >> >> This is a low risk enhancement.? I'd like to request approval for a >> late enhancement in JDK 15.? It extends the pre-existing code path >> with refactoring the hidden classes to prepare for new kinds of >> classes with trusted final fields.? The change is straight-forward. >> >> Can this wait to integrate in JDK 16? >> >> ? It's important to get this enhancement in when record is a preview >> feature that we can get feedback and give time to 3rd-party >> serialization frameworks to look into adding the support for >> records.? If we delayed this change in 16 and records exit preview, >> it would be bad for frameworks if they verify that they support >> records in 15 but fail in 16.? OTOH the risk of this patch is low. >> >> Performance Impact >> >> I addressed the performance concern I raised earlier myself. For >> reflection, VM creates the reflective Field objects and fills in >> MemberName when resolving a member.? VM will tag if this >> Field/MemberName is trusted final field.? I think this is a cleaner >> approach rather than in each place to check for final static and >> final fields in hidden or record class to determine if it has write >> access or not. >> >> `sun.misc.Unsafe` does not use Field::isTrustedFinalField because (1) >> Unsafe has been allowing access of static final fields of any classes >> but isTrustedFinalField is not limited to instance fields (2) Unsafe >> disallows access to all fields in a hidden class (not limited to >> trusted final fields).? So it follows the precedence and simply >> checks if the declaring class is a record. `Class::isRecord` calls >> `Class::getSuperclass` to check if it's a subtype of `Record`.? As >> `Class::getSuperclass` is intrinsified, the call on isRecord on a >> normal class is fast. Christoph has contributed the microbenchmarks >> that confirm that no performance regression. >> >> Thanks >> Mandy >> [1] >> https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-June/040096.html > From kim.barrett at oracle.com Thu Jun 18 06:34:57 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 18 Jun 2020 02:34:57 -0400 Subject: [OpenJDK 2D-Dev] RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: References: <5EDE9A75.6020504@oracle.com> <5EE5AB71.20505@oracle.com> Message-ID: > On Jun 15, 2020, at 7:41 AM, Kim Barrett wrote: > >> On Jun 14, 2020, at 12:45 AM, Philip Race wrote: >> >> Kim, >> >> >> Until it says in "the JDK" and not "in HotSpot" you have not addressed my main point. >> Please rename the JEP. After some off-list discussions I have a better idea of what Phil was asking for and why. In response I've changed the JEP, replacing a few occurrences of "HotSpot" with "the JDK", as described below. All other occurrences of "HotSpot" have been left as-is. Title: JEP 347: Adopt C++14 Language Features in HotSpot => JEP 347: Adopt C++14 Language Features in the JDK Summary: Allow the use of selected C++14 language features in the HotSpot C++ source code. => Allow the use of selected C++14 language features in the JDK C++ source code. Goals: The purpose of this JEP is to formally allow C++ source code changes within HotSpot to take advantage of some C++14 standard language features. => The purpose of this JEP is to formally allow C++ source code changes within the JDK to take advantage of some C++14 standard language features. From chris.hegarty at oracle.com Thu Jun 18 07:47:47 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 18 Jun 2020 08:47:47 +0100 Subject: RFR [15] 8247696: Incorrect tail computation for large segments in AbstractMemorySegmentImpl::mismatch In-Reply-To: References: <0B15B6AF-F196-4192-B448-F1955AE586A7@oracle.com> Message-ID: Hi Paul, > On 17 Jun 2020, at 21:56, Paul Sandoz wrote: > > Hi Chris, > > AbstractMemorySegmentImpl > ? > > In vectorizedMismatchLarge: > > 163 if (remaining > 7) > 164 throw new InternalError("remaining greater than 7: " + remaining); > 165 i = length - remaining; > 166 } > > Should this check be an assert? It is logically an assertion of vectorizedMismatchLarge, but assertions are not used elsewhere in the foreign memory code. Arguably the check is not needed at all, we could choose to just drop it. > ? > > This fix prompted me to think more deeply about the implementation of vectorizedMismatchLarge and its use of vectorizedMismatch. Sorry, I should have thought about this more throughly earlier on. > > We need to refine the approach, not for this patch, but something to follow up after. Ok, I?ll an issue to track these. > I think there are two issues. > > 1) The intrinsic method vectorizedMismatch could potentially bomb out at any point and return the bitwise compliment of the remaining number of elements to check. > > Obviously, there is no point doing that arbitrarily but a stub implementation for, say, x86 AVX-512 might decide bomb out for < 64 remaining elements, rather than apply vector operations on smaller vector sizes or use a scalar loop. It does not today, but I think we should guard against that potentially happening, otherwise bad things can happen. > > I think the loop should exit when the last sub-range has been checked. We should rely on other tests to ensure the intrinsic method is operating efficiently. > > > 2) This method only works when operating on byte arrays. It will not work correctly if operating on short or long arrays, since we are not adjusting the length and offsets accordingly by the scale. It's probably easier to just rename this as vectorizedMismatchLargeForBytes and drop the log2ArrayIndexScale argument. Then expand later if need be. I still think the method rightly belongs in ArraysSupport. Sounds reasonable. -Chris. > Paul. > >> On Jun 17, 2020, at 8:33 AM, Chris Hegarty wrote: >> >> The MemorySegment::mismatch implementation added vectorized mismatch of long sizes. The implementation is trivial, but the starting point for a more optimized implementation, if needed. ArraysSupport::vectorizedMismatchLarge incorrectly returns the bitwise complement of the offset of the first mismatch, where it should return the bitwise complement of the number of remaining pairs of elements to be checked in the tail of the two arrays. The AbstractMemorySegmentImpl::mismatch masked this problem, since it seamlessly compared the remaining tail, which is larger than it should be. >> >> Webrev: >> https://cr.openjdk.java.net/~chegar/8247696/webrev.00/ >> >> I updated the exiting BulkOps micro-benchmark to cover mismatch. Here are the results, compared to ByteBuffer::mismatch, on my machine: >> >> Benchmark Mode Cnt Score Error Units >> BulkOps.mismatch_large_bytebuffer avgt 30 740186.973 ? 119314.207 ns/op >> BulkOps.mismatch_large_segment avgt 30 683475.305 ? 76355.043 ns/op >> BulkOps.mismatch_small_bytebuffer avgt 30 7.367 ? 0.523 ns/op >> BulkOps.mismatch_small_segment avgt 30 4.140 ? 0.602 ns/op >> >> >> -Chris. > From maurizio.cimadamore at oracle.com Thu Jun 18 09:57:34 2020 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 18 Jun 2020 10:57:34 +0100 Subject: RFR [15] 8247696: Incorrect tail computation for large segments in AbstractMemorySegmentImpl::mismatch In-Reply-To: References: <0B15B6AF-F196-4192-B448-F1955AE586A7@oracle.com> Message-ID: <94fceeda-edee-c571-b32d-af8d5aed672e@oracle.com> Thanks for looking at this Chris On 17/06/2020 21:56, Paul Sandoz wrote: > Hi Chris, > > AbstractMemorySegmentImpl > ? > > In vectorizedMismatchLarge: > > 163 if (remaining > 7) > 164 throw new InternalError("remaining greater than 7: " + remaining); > 165 i = length - remaining; > 166 } > > Should this check be an assert? I suggested that to Chris, since sometimes asserts are enabled when running tests, sometimes are not - in langtools we moved away from using asserts as we realized that in certain cases they were silently failing. I'm ok with whatever standard you feel comfortable with though. > > ? > > This fix prompted me to think more deeply about the implementation of vectorizedMismatchLarge and its use of vectorizedMismatch. Sorry, I should have thought about this more throughly earlier on. > > We need to refine the approach, not for this patch, but something to follow up after. I think there are two issues. > > 1) The intrinsic method vectorizedMismatch could potentially bomb out at any point and return the bitwise compliment of the remaining number of elements to check. > > Obviously, there is no point doing that arbitrarily but a stub implementation for, say, x86 AVX-512 might decide bomb out for < 64 remaining elements, rather than apply vector operations on smaller vector sizes or use a scalar loop. It does not today, but I think we should guard against that potentially happening, otherwise bad things can happen. So your worry here is that we'll end up with an infinite loop, right? If so, we could check remaining against previous remaining and bomb out too if no further progress seem to be made? > > I think the loop should exit when the last sub-range has been checked. We should rely on other tests to ensure the intrinsic method is operating efficiently. > > > 2) This method only works when operating on byte arrays. It will not work correctly if operating on short or long arrays, since we are not adjusting the length and offsets accordingly by the scale. It's probably easier to just rename this as vectorizedMismatchLargeForBytes and drop the log2ArrayIndexScale argument. Then expand later if need be. I still think the method rightly belongs in ArraysSupport. Yep - probably good idea to restrict on bytes, for now. Maurizio > > Paul. > >> On Jun 17, 2020, at 8:33 AM, Chris Hegarty wrote: >> >> The MemorySegment::mismatch implementation added vectorized mismatch of long sizes. The implementation is trivial, but the starting point for a more optimized implementation, if needed. ArraysSupport::vectorizedMismatchLarge incorrectly returns the bitwise complement of the offset of the first mismatch, where it should return the bitwise complement of the number of remaining pairs of elements to be checked in the tail of the two arrays. The AbstractMemorySegmentImpl::mismatch masked this problem, since it seamlessly compared the remaining tail, which is larger than it should be. >> >> Webrev: >> https://cr.openjdk.java.net/~chegar/8247696/webrev.00/ >> >> I updated the exiting BulkOps micro-benchmark to cover mismatch. Here are the results, compared to ByteBuffer::mismatch, on my machine: >> >> Benchmark Mode Cnt Score Error Units >> BulkOps.mismatch_large_bytebuffer avgt 30 740186.973 ? 119314.207 ns/op >> BulkOps.mismatch_large_segment avgt 30 683475.305 ? 76355.043 ns/op >> BulkOps.mismatch_small_bytebuffer avgt 30 7.367 ? 0.523 ns/op >> BulkOps.mismatch_small_segment avgt 30 4.140 ? 0.602 ns/op >> >> >> -Chris. From shade at redhat.com Thu Jun 18 12:43:20 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 18 Jun 2020 14:43:20 +0200 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: On 6/17/20 6:50 PM, Mandy Chung wrote: > On 6/17/20 8:11 AM, Aleksey Shipilev wrote: >> On 6/15/20 11:58 PM, Mandy Chung wrote: >> Amusingly, with the rest of Unsafe available the barn is still full open. Here's the sketch for the >> workaround: get the field list with getDeclaredFields, optionally guess the VM layout (the rules are >> not that hard, and JOL does it routinely), instantiate a Record with whatever arguments, poke the >> test patterns with Unsafe.put*, read them back from record components to verify. It would get me the >> same objectFieldOffset in a round-about way. > > I'm aware of these workarounds in the wild.?? As sun.misc.Unsafe is JDK unsupported API, this patch > does not attempt to implement a complete solution but adds an obvious notification informing the > frameworks that `sun.misc.Unsafe::objectFieldOffset` and `sun.misc.Unsafe::staticField{Offset/Base}` > not to support records. I understand the intent. My point is that intent is mistimed during this time. I understand this tripwire needs to be put while Records are still in preview. My point is that it cannot be put in before Records serialization story has the preview-proven answer. The intent also looks rather opaque. The intent is to notify maintainers, fine. I am one of the maintainers (although not of serialization library, but still heavy-Unsafe one), so hear me out. I came up with the workaround above in about 15 minutes after someone pointed out the exception to me. Have it crossed my mind that maybe JDK developers want some help here? No. It looks like another impediment that should be solved on the spot. Does that exception communicate any intent at all? The message is generic. There is no comment. How would you expect to push maintainers to think along the "we should be collaborating to find a proper way to do this", instead of "this is set in stone; let me hack this around too"? This is the actionable bit: At very least the exception message should say something about the intent. And maybe even the comment in Unsafe.java should point to the discussion about this intent and maybe even provide the breadcrumbs to follow, e.g. to ObjectInputStream.readRecord(). >> Are we absolutely sure that what ObjectInputStream.readRecord() does fits the 3rd party >> serialization libraries? Does it work for them performance-wise? > > There should be performance improvement opportunity (see Peter Levart's good work - JDK-8247532 [1]) > >> (Are we even sure about that for JDK itself?) > > Java serialization support for records use its canonical constructor.? (I'm not sure what you tried > to point out by this). Let me try again. The existence of JDK-8247532 solidifies my concern: the JDK code itself had not yet figured out how to do Record serialization fast! Now put yourself in the shoes of the 3rd party lib maintainer: Unsafe is forbidden, JDK code itself is slow. Asking 3rd party lib maintainer to find whatever intricate incantation they should use to get decent performance -- while JDK itself is still working on it! -- puts them into weird position, from where the *sane* way out is to hack around the Unsafe restriction. I mean, it is as enticing as hacking the JDK code itself, with the added benefit of working across the JDK releases. This is time and again the core of Unsafe discussion, which boils down to a very simple request: if JDK developers take away some Unsafe APIs, 3rd party developers need to have the known good replacement for it. If that does not happen, 3rd party developers would invest in doing more Unsafe hacks. Record serialization seems to fall into the same trap. To reiterate: the non-Unsafe code should be proven as the viable alternative before breaking Unsafe. Otherwise, we only deepen the dependency on Unsafe. >> Because if it is not, 3rd party lib maintainers would proceed to hacking in the >> "objectFieldOffset workaround" and we would get the cobra-effect-like strengthening of dependency on >> Unsafe quirks. > > This is exactly why we request this for 15 so that 3rd-party lib maintainers can prototype and send > feedback.? We love to understand any stumbling block and work together to resolve any issue before > records exit preview. Again, JDK-8247532 is the writing on the wall: we don't need 3rd party developers to tell if Record serialization works fast in 15 -- we already know it does not. >> It is fun to consider JNI to be more powerful than Unsafe. This seems backwards. The intent to break >> Unsafe users might be defensible, but this power oddity is still quite odd. > > I think you misread the message.? There is no claim whether JNI or Unsafe is more powerful.? I don't think I did. Disallowing Unsafe to do something that JNI is allowed to do is making Unsafe less powerful. This was nothing to do with what API is supported or not. -- Thanks, -Aleksey From andrew_m_leonard at uk.ibm.com Thu Jun 18 12:51:29 2020 From: andrew_m_leonard at uk.ibm.com (Andrew Leonard) Date: Thu, 18 Jun 2020 13:51:29 +0100 Subject: Should System.mapLibraryName(userLibName) on AIX return ".a" suffix ? Message-ID: Hi, Currently System.mapLibraryName(userLibName)returns ".so" as the library suffix on AIX, however the default shared library suffix on AIX is actually ".a". Should the System.mapLibraryName(userLibName)default suffix be changed to align with the AIX default? Thanks Andrew Andrew Leonard Java Runtimes Development IBM Hursley IBM United Kingdom Ltd internet email: andrew_m_leonard at uk.ibm.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From chris.hegarty at oracle.com Thu Jun 18 13:09:32 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 18 Jun 2020 14:09:32 +0100 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: <4B268BF1-1DFD-4DCC-AEA3-7F2C7B81DC91@oracle.com> > On 18 Jun 2020, at 13:43, Aleksey Shipilev wrote: > >> ... > > Again, JDK-8247532 is the writing on the wall: we don't need 3rd party developers to tell if Record > serialization works fast in 15 -- we already know it does not. I disagree. JDK-8247532 is under review and well on its way towards JDK 15 (yes, during RDP 1). My reading of Peter?s benchmark result is that Record deserialization *does work fast*. What am I missing. -Chris. From chris.hegarty at oracle.com Thu Jun 18 13:28:01 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 18 Jun 2020 14:28:01 +0100 Subject: RFR [15] 8247789: Remove use of reflection from test/jdk/java/io/Serializable/records/StreamRefTest.java Message-ID: <56CF8661-851F-47A4-B700-35AE30DED206@oracle.com> This is a small change to remove the use of reflection to fabricate "bad" serial data - in order to verify constructor invariants for record serialization tests. Instead, it is straight forward to just modify a single value in the byte stream, that results in the same. https://cr.openjdk.java.net/~chegar/8247789/webrev/ -Chris. From Roger.Riggs at oracle.com Thu Jun 18 13:56:44 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Thu, 18 Jun 2020 09:56:44 -0400 Subject: RFR [15] 8247789: Remove use of reflection from test/jdk/java/io/Serializable/records/StreamRefTest.java In-Reply-To: <56CF8661-851F-47A4-B700-35AE30DED206@oracle.com> References: <56CF8661-851F-47A4-B700-35AE30DED206@oracle.com> Message-ID: <5a2e9eab-a7f2-e754-1a68-f5dee18e975a@oracle.com> Hi Chris, It may be prudent to check that the current value in the byte array is the one you expect before changing it. That would help catch tests if something else changes the contents of the stream. Otherwise looks good. Thanks, Roger On 6/18/20 9:28 AM, Chris Hegarty wrote: > This is a small change to remove the use of reflection to fabricate > "bad" serial data - in order to verify constructor invariants for > record serialization tests. Instead, it is straight forward to just > modify a single value in the byte stream, that results in the same. > > https://cr.openjdk.java.net/~chegar/8247789/webrev/ > > -Chris. > From chris.hegarty at oracle.com Thu Jun 18 14:25:55 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 18 Jun 2020 15:25:55 +0100 Subject: RFR [15] 8247789: Remove use of reflection from test/jdk/java/io/Serializable/records/StreamRefTest.java In-Reply-To: <5a2e9eab-a7f2-e754-1a68-f5dee18e975a@oracle.com> References: <56CF8661-851F-47A4-B700-35AE30DED206@oracle.com> <5a2e9eab-a7f2-e754-1a68-f5dee18e975a@oracle.com> Message-ID: <86536755-3AB9-4589-93DB-5A9C4E151E3A@oracle.com> > On 18 Jun 2020, at 14:56, Roger Riggs wrote: > > Hi Chris, > > It may be prudent to check that the current value in the byte array is the one you expect > before changing it. > That would help catch tests if something else changes the contents of the stream. Good idea Roger. Done. https://cr.openjdk.java.net/~chegar/8247789/webrev.01/ -Chris. From Roger.Riggs at oracle.com Thu Jun 18 14:29:06 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Thu, 18 Jun 2020 10:29:06 -0400 Subject: RFR [15] 8247789: Remove use of reflection from test/jdk/java/io/Serializable/records/StreamRefTest.java In-Reply-To: <86536755-3AB9-4589-93DB-5A9C4E151E3A@oracle.com> References: <56CF8661-851F-47A4-B700-35AE30DED206@oracle.com> <5a2e9eab-a7f2-e754-1a68-f5dee18e975a@oracle.com> <86536755-3AB9-4589-93DB-5A9C4E151E3A@oracle.com> Message-ID: <6bf85deb-5854-b24c-20c0-207f485241cd@oracle.com> Thanks Chris, Looks good. On 6/18/20 10:25 AM, Chris Hegarty wrote: >> On 18 Jun 2020, at 14:56, Roger Riggs wrote: >> >> Hi Chris, >> >> It may be prudent to check that the current value in the byte array is the one you expect >> before changing it. >> That would help catch tests if something else changes the contents of the stream. > Good idea Roger. Done. > > https://cr.openjdk.java.net/~chegar/8247789/webrev.01/ > > -Chris. From mandy.chung at oracle.com Thu Jun 18 15:56:43 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 18 Jun 2020 08:56:43 -0700 Subject: RFR [15] 8247789: Remove use of reflection from test/jdk/java/io/Serializable/records/StreamRefTest.java In-Reply-To: <86536755-3AB9-4589-93DB-5A9C4E151E3A@oracle.com> References: <56CF8661-851F-47A4-B700-35AE30DED206@oracle.com> <5a2e9eab-a7f2-e754-1a68-f5dee18e975a@oracle.com> <86536755-3AB9-4589-93DB-5A9C4E151E3A@oracle.com> Message-ID: <6588f195-0510-c06e-479c-8e2b363875c1@oracle.com> On 6/18/20 7:25 AM, Chris Hegarty wrote: >> On 18 Jun 2020, at 14:56, Roger Riggs wrote: >> >> Hi Chris, >> >> It may be prudent to check that the current value in the byte array is the one you expect >> before changing it. >> That would help catch tests if something else changes the contents of the stream. > Good idea Roger. Done. > > https://cr.openjdk.java.net/~chegar/8247789/webrev.01/ > Looks good. Mandy From shade at redhat.com Thu Jun 18 17:10:08 2020 From: shade at redhat.com (Aleksey Shipilev) Date: Thu, 18 Jun 2020 19:10:08 +0200 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: <4B268BF1-1DFD-4DCC-AEA3-7F2C7B81DC91@oracle.com> References: <4B268BF1-1DFD-4DCC-AEA3-7F2C7B81DC91@oracle.com> Message-ID: <029fdb77-660b-4f59-7ee7-bc7020b5e49f@redhat.com> On 6/18/20 3:09 PM, Chris Hegarty wrote: >> On 18 Jun 2020, at 13:43, Aleksey Shipilev wrote: >> Again, JDK-8247532 is the writing on the wall: we don't need 3rd party developers to tell if Record >> serialization works fast in 15 -- we already know it does not. > > I disagree. JDK-8247532 is under review and well on its way towards JDK 15 (yes, during RDP 1). > My reading of Peter?s benchmark result is that Record deserialization *does work fast*. What am > I missing. JDK-8247532 is the evidence that Records serialization performance story is not clear. Even if we disregard that after JDK-8247532 [1] Records are still 8% slower, the _existence_ of JDK-8247532 indicates there are performance problems in the area. That evidence now needs to be compensated by much more evidence to the contrary. (Yes, I contracted a lot of Bayesian thinking from my statistician wife...) (Here were several paragraphs of further thoughts, but I realized it basically repeats what I said before.) -- Thanks, -Aleksey [1] https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-June/067223.html RecordSerializationBench.deserializeClasses: 31.049 ? 0.235 us/op RecordSerializationBench.deserializeRecords: 33.588 ? 0.394 us/op From peter.kessler at os.amperecomputing.com Thu Jun 18 18:24:53 2020 From: peter.kessler at os.amperecomputing.com (Peter Kessler (Open Source)) Date: Thu, 18 Jun 2020 18:24:53 +0000 Subject: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) In-Reply-To: References: <814DA850-76A9-4526-A3D9-141CD516C9AE@amperemail.onmicrosoft.com> <50955EDB-CCC2-4334-A4AF-2EB422E9CAA9@amperemail.onmicrosoft.com> <3B38456A-6BD0-4B95-9F9C-E73F7FE3A78F@amperemail.onmicrosoft.com> Message-ID: <1FBDCE99-C63E-4C08-A508-08EC41688681@amperemail.onmicrosoft.com> Roger, You are right that the existing code does not let me avoid the stack walk. I withdraw the proposed patch. I think at least part of the problem is the comment on VM.latestUserDefinedLoader(). That says /* * Returns the first user-defined class loader up the execution stack, * or the platform class loader if only code from the platform or * bootstrap class loader is on the stack. */ I thought that a "user-defined class loader" was a *class loader* defined by the user. I now think VM.latestUserDefinedLoader returns the class loader of the first user-defined *method* up the execution stack, whether that class loader is user-defined or is, for example, the application class loader. I do not consider the application class loader to be user-defined, though obviously methods loaded by the application class loader can be user-defined. ... peter ?-----Original Message----- From: Roger Riggs Organization: Java Products Group, Oracle Date: Tuesday, June 16, 2020 at 7:14 AM To: "Peter Kessler (Open Source)" , "core-libs-dev at openjdk.java.net" Subject: Re: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) Hi Peter, I think you've hardwired an assumption about the contents of the stack into VM.latestUserDefineLoader... On 6/12/20 7:19 PM, Peter Kessler (Open Source) wrote: > Roger, > > I did think about confining the changes to ObjectInputStream. Maybe I > did not think hard enough about it. > > Keeping a cache of the result of a first stack walk might work for the > recursive calls. One might still do a useless stack walk for each > top-level readObject call. No concurrent class loading can invalidate > the cache for any particular stack walk. But it seems fraught to > reuse a cached value in the face of concurrent class loading, or in > the presence of overloaded ObjectInputStream methods that might create > new ClassLoaders on this stack. I'm sure I don't want to debug that. > > My experiments have shown that the average stack walk is about 20 > frames. I do not have the data on how many of those frames are > recursive calls within ObjectInputStream methods and how many are in > other code. I'd be interested in a histogram of the depths and in which applications. > Dragging in the StackWalker API would further complicate things, in my > opinion. It might appear to keep the code out of platform native > code, but we know that walking thread stacks involves native code. > > I settled on a system-wide one-bit cache: There have been no > user-defined ClassLoaders constructed, so I can not find one on any > particular stack walk. It may not be the best possible solution, but > it addresses the issue of useless stack walks in the common case where > there are no user-defined ClassLoaders. > > I agree that both ClassLoader and ObjectInputStream are complex. In > fact, the proposed patch makes no changes to ObjectInputStream. There > is one line added to each of the system-defined ClassLoaders to > identify them as not user-defined ClassLoaders, one line in the base > ClassLoader constructor, and the bulk of the change is around the > guard in jdk.internal.misc.VM.latestUserDefinedLoader() to avoid the > stack walk if possible. The spec and implementation of OIS.resolveClass do not make any assumptions about whether serialization is invoked from a class on the PlatformLoader, AppClassLoader, or another loader. In the case of a 'system' thread or class calling OIS.readObject, the change you are proposing will force class loading in resolveClass to the AppClassLoader. In VM.latestUserDefinedLoader:390 caught my eye because it forces the AppClassLoader. It assumes that scanning the stack would find and return the AppClassLoader, not so in all cases. Regards, Roger > > ... peter > > -----Original Message----- > From: Roger Riggs > Organization: Java Products Group, Oracle > Date: Friday, June 12, 2020 at 11:50 AM > To: "Peter Kessler (Open Source)" , "core-libs-dev at openjdk.java.net" > Subject: Re: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) > > Hi Peter, > > The hazard to avoid is the cross package coupling that makes both > ClassLoader and > ObjectInputStream more complex; both are more than sufficiently complex > already. > > Optimizing the implementations make sense if it benefits enough uses > cases and > does not make the code harder to maintain and evolve. > > Have you explored any alternatives that can be confined to OIS? > For example, taking advantage the many calls to readObject within > a single OIS that are nested. Possibly the first (depth = 0) find user > class loader > can be re-used or at least to short circuit the search on subsequent > nested calls? > Or caching the classloader at each depth and only need to probe for it > if it has > not already been recorded. The StackWalker API may be useful to limit > the depth > of searches. > > Thanks, Roger > > > > On 6/11/20 2:52 PM, Peter Kessler OS wrote: > > Hi Roger, > > > > Thanks for your comments. > > > > I am not knowingly changing the required behavior. All I am proposing is > > guarding the thread stack walk if it will be a waste of effort. If an > > application or framework has its own ClassLoaders, then the guard will fail > > and the thread stack will be walked. > > > > The stack walk is important, if there are user-defined ClassLoaders. A > > method that uses ObjectInputStream to read objects can not know what > > ClassLoader to use to resolve class references in the stream, without the > > stack walk. But if there never have been any user-defined ClassLoaders > > constructed, then the stack walk can never return a user-defined > > ClassLoader. > > > > If the stream reader is part of some simple application, then it may spend > > a significant amount of its time uselessly walking thread stacks. (If the > > stream reader is part of some simple application its author may not know to > > override ObjectInputStream.resolveClass, or how to determine what > > ClassLoader should load a class from the stream.) If someone were using > > such an application to choose platforms, the Java platform would be at a > > disadvantage. > > > > Consider a somewhat more complex case, the SPECjbb2015 benchmark, which > > claims to model a well-constructed business application on the Java > > platform. SPECjbb2015 runs multiple JVMs, exchanging information via > > ObjectInputStreams. Profiling shows that it spends more than 5% of its time > > walking thread stacks. Yet, in spite of being a complex application, > > written by experts in the Java platform, there are no user-defined > > ClassLoaders. So all the time spent walking stacks is wasted time, and is > > one of the reasons the servers fail their service-level agreements. We can > > discuss whether SPECjbb2015 is representative of Java applications. But it > > is easily available and relatively simple to run. People use benchmarks > > like SPECjbb2015 to decide whether to use the Java platform for particular > > tasks, and it would be unfortunate if we did not make the Java platform as > > good as it could be. > > > > ... peter > > > > -----Original Message----- > > From: core-libs-dev on behalf of Roger Riggs > > Organization: Java Products Group, Oracle > > Date: Tuesday, June 9, 2020 at 6:43 AM > > To: "core-libs-dev at openjdk.java.net" > > Subject: Re: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) > > > > Hi Peter, > > > > I'd like to understand the scope and impact of the problem before > > jumping to a solution. > > For specific applications, overriding ObjectInputStream.resolveClass to > > handle the class > > lookup can handle the case as expeditiously as is necessary. > > > > What application uses cases are impacted and how prevelant are they? > > Many, many frameworks and applications create classloaders for their own > > purposes and > > take advantage of the required behavior. > > > > Thanks, Roger > > > > > > On 6/4/20 7:08 PM, Peter Kessler OS wrote: > > > ObjectInputStream.resolveClass(ObjectStreamClass) is specified to find "the first class loader on the current thread's stack (starting from the currently executing method) that is neither the platform class loader nor its ancestor." Finding that class loader is done with a virtual frame walk in JVM_LatestUserDefinedLoader(JNIEnv *env). The virtual frame walk is expensive. If an application does not define any ClassLoaders, then the virtual frame walk will always find the system class loader. > > > > > > If an application does a significant amount of reading from an ObjectInputStream, the cost of ObjectInputStream.resolveClass(ObjectInputStreamClass) can be considerable. In a sample application, that method (and the methods it calls) have been measured to consume 5+% of the CPU cycles. > > > > > > The proposal is to note the construction of any user-defined ClassLoader, and if none has been constructed, avoid the virtual frame walk returning ClassLoader.getSystemClassLoader(). But if any user-defined ClassLoader has been constructed, use the virtual frame walk to find the first user-defined ClassLoader on the thread stack. > > > > > > User-defined ClassLoaders could be distinguished from non-user-defined ClassLoaders in several ways. The ClassLoader class hierarchy could be changed to make non-user-defined ClassLoaders inherit from a marker class. There is already a jdk.internal.loader.BuiltinClassLoader but it is used for module (and resource) loading. BuiltinClassLoader is not a super-class of jdk.internal.reflect.DelegatingClassLoader, which is used to load reflection classes, and reflection methods are also ignored by the virtual frame walk. ClassLoaders could have a boolean that distinguished whether they are user-defined or not. That would require a change to ClassLoader API. There are other ways to distinguish user-defined ClassLoaders. The proposal is to keep the decoration of the ClassLoaders out of the instances, so that it is only visible to the code that needs it. > > > > > > The proposal is that each non-user-defined ClassLoader have a static block that registers the class as a non-user-defined ClassLoader. Then in the base ClassLoader constructor check if the ClassLoader being constructed is a user-defined ClassLoader and if so set a boolean that acts as the guard on the virtual frame walk. If additional non-user-defined ClassLoaders are declared in the future beyond the ones identified in this patch, they can be added to the registry. > > > > > > There are currently 4 non-user-defined ClassLoader classes, so the registry is not difficult to maintain. Nor is the registry difficult to search when ClassLoaders are constructed. The guard boolean that records whether a user-defined ClassLoader has been constructed transitions from false to true and never becomes false again. (An extension to transition the boolean back to false when a user-defined ClassLoader becomes unreachable is beyond the scope of this proposal.) > > > > > > This proposal slows loading of non-user-defined ClassLoaders by the time it takes to register them. It slows ClassLoader construction by the time it takes to consult the registry and conditionally set the guard boolean. At each invocation of ObjectInputStream.resolveClass(ObjectInputStreamClass), the guard boolean is used to possibly avoid the virtual frame walk. > > > > > > Tested with `make run-test-tier1` on Linux (CentOS 7) on both aarch64 and x86_64. I had one failure on each of the runs > > > > > > ACTION: main -- Failed. Execution failed: `main' threw exception: java.lang.Error: TESTBUG: the library has not been found in ${Repo}/build/linux-aarch64-server-release/images/test/hotspot/jtreg/native > > > REASON: User specified action: run main/native GTestWrapper > > > > > > on both plaforms. But I get the same failure when I run the tier1 tests on clones of OpenJDK-15+25 on both aarch64 and x86_64. > > > > > > Running a sample (representative?) application with the Oracle Developer Studio analyzer shows a performance comparison of > > > > > > Stack Fragment sorted by metric: Attributed Total CPU Time > > > baseline.er experiment.er > > > Attributed Attributed Name > > > Total CPU Time Total CPU Time > > > sec. sec. > > > ================================== Callers > > > 3459.210 29.931 java.io.ObjectInputStream.readOrdinaryObject(boolean) > > > 1139.727 3.532 java.io.ObjectInputStream.readArray(boolean) > > > 875.262 9.116 java.io.ObjectInputStream.readNonProxyDesc(boolean) > > > > > > ================================== Stack Fragment > > > java.io.ObjectInputStream.readClassDesc(boolean) > > > java.io.ObjectInputStream.readNonProxyDesc(boolean) > > > java.io.ObjectInputStream.resolveClass(java.io.ObjectStreamClass) > > > java.io.ObjectInputStream.latestUserDefinedLoader() > > > 4.173 3.953 jdk.internal.misc.VM.latestUserDefinedLoader() > > > > > > ================================== Callees > > > 5470.026 0. jdk.internal.misc.VM.latestUserDefinedLoader0() > > > 0. 38.627 java.lang.ClassLoader.getSystemClassLoader() > > > > > > The `hg export -g` below is with respect to OpenJDK-15+25. > > > > > > Thank you for your review comments. I will need a sponsor to get this change into the repository. > > > > > > ... peter > > > > > > # HG changeset patch > > > # User Peter Kessler > > > # Date 1591310467 25200 > > > # Thu Jun 04 15:41:07 2020 -0700 > > > # Node ID 9a39488182c1dfc5bc7bb41d562d7ef16ee657f6 > > > # Parent 90b266a84c06f1b3dc0ed8767856793e8c1c357e > > > Improve the performance of ObjectInputStream.resolveClass > > > > > > diff --git a/src/java.base/share/classes/java/lang/ClassLoader.java b/src/java.base/share/classes/java/lang/ClassLoader.java > > > --- a/src/java.base/share/classes/java/lang/ClassLoader.java > > > +++ b/src/java.base/share/classes/java/lang/ClassLoader.java > > > @@ -385,6 +385,8 @@ > > > } > > > this.package2certs = new ConcurrentHashMap<>(); > > > this.nameAndId = nameAndId(this); > > > + /* Note the construction of a ClassLoader. */ > > > + VM.noteClassLoaderConstruction(this.getClass()); > > > } > > > > > > /** > > > diff --git a/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java b/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java > > > --- a/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java > > > +++ b/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java > > > @@ -117,6 +117,11 @@ > > > protected Class loadClassOrNull(String cn, boolean resolve) { > > > return JLA.findBootstrapClassOrNull(this, cn); > > > } > > > + > > > + static { > > > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > > > + VM.registerNotUserDefinedClassLoader(BootClassLoader.class); > > > + } > > > }; > > > > > > /** > > > @@ -127,6 +132,8 @@ > > > static { > > > if (!ClassLoader.registerAsParallelCapable()) > > > throw new InternalError(); > > > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > > > + VM.registerNotUserDefinedClassLoader(PlatformClassLoader.class); > > > } > > > > > > PlatformClassLoader(BootClassLoader parent) { > > > @@ -142,6 +149,8 @@ > > > static { > > > if (!ClassLoader.registerAsParallelCapable()) > > > throw new InternalError(); > > > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > > > + VM.registerNotUserDefinedClassLoader(AppClassLoader.class); > > > } > > > > > > final URLClassPath ucp; > > > diff --git a/src/java.base/share/classes/jdk/internal/misc/VM.java b/src/java.base/share/classes/jdk/internal/misc/VM.java > > > --- a/src/java.base/share/classes/jdk/internal/misc/VM.java > > > +++ b/src/java.base/share/classes/jdk/internal/misc/VM.java > > > @@ -304,12 +304,44 @@ > > > private static final int JVMTI_THREAD_STATE_WAITING_INDEFINITELY = 0x0010; > > > private static final int JVMTI_THREAD_STATE_WAITING_WITH_TIMEOUT = 0x0020; > > > > > > + /** A registry of not user-defined ClassLoaders. */ > > > + private static final List> notUserDefinedClassLoaderRegistry = > > > + Collections.synchronizedList(new ArrayList<>()); > > > + > > > + /** Register a ClassLoader class as being not a user-defined ClassLoader. */ > > > + public static void registerNotUserDefinedClassLoader(Class classLoaderClass) { > > > + notUserDefinedClassLoaderRegistry.add(classLoaderClass); > > > + } > > > + > > > + /** A flag for whether a user-defined ClassLoaders has been constructed. */ > > > + private static volatile boolean constructedUserDefinedClassLoaderFlag = false; > > > + > > > + /** > > > + * Note a ClassLoader construction, and record if a > > > + * user-defined ClassLoader has been constructed. > > > + */ > > > + public static void noteClassLoaderConstruction(Class classLoaderClass) { > > > + /* Check if the ClassLoader class not a user-defined ClassLoader. */ > > > + if (notUserDefinedClassLoaderRegistry.contains(classLoaderClass)) { > > > + return; > > > + } > > > + constructedUserDefinedClassLoaderFlag = true; > > > + return; > > > + } > > > + > > > /* > > > * Returns the first user-defined class loader up the execution stack, > > > * or the platform class loader if only code from the platform or > > > * bootstrap class loader is on the stack. > > > */ > > > public static ClassLoader latestUserDefinedLoader() { > > > + if (!constructedUserDefinedClassLoaderFlag) { > > > + /* > > > + * If no user-defined ClassLoader has been constructed, > > > + * then I will not find a user-defined ClassLoader in the stack. > > > + */ > > > + return ClassLoader.getSystemClassLoader(); > > > + } > > > ClassLoader loader = latestUserDefinedLoader0(); > > > return loader != null ? loader : ClassLoader.getPlatformClassLoader(); > > > } > > > diff --git a/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java b/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java > > > --- a/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java > > > +++ b/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java > > > @@ -30,6 +30,7 @@ > > > > > > import jdk.internal.access.JavaLangAccess; > > > import jdk.internal.access.SharedSecrets; > > > +import jdk.internal.misc.VM; > > > > > > /** Utility class which assists in calling defineClass() by > > > creating a new class loader which delegates to the one needed in > > > @@ -73,4 +74,9 @@ > > > DelegatingClassLoader(ClassLoader parent) { > > > super(parent); > > > } > > > + > > > + static { > > > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > > > + VM.registerNotUserDefinedClassLoader(DelegatingClassLoader.class); > > > + } > > > } > > > > > > > > > From mandy.chung at oracle.com Thu Jun 18 19:29:34 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 18 Jun 2020 12:29:34 -0700 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: References: Message-ID: On 6/18/20 5:43 AM, Aleksey Shipilev wrote: > On 6/17/20 6:50 PM, Mandy Chung wrote: >> On 6/17/20 8:11 AM, Aleksey Shipilev wrote: >>> On 6/15/20 11:58 PM, Mandy Chung wrote: >>> Amusingly, with the rest of Unsafe available the barn is still full open. Here's the sketch for the >>> workaround: get the field list with getDeclaredFields, optionally guess the VM layout (the rules are >>> not that hard, and JOL does it routinely), instantiate a Record with whatever arguments, poke the >>> test patterns with Unsafe.put*, read them back from record components to verify. It would get me the >>> same objectFieldOffset in a round-about way. >> I'm aware of these workarounds in the wild.?? As sun.misc.Unsafe is JDK unsupported API, this patch >> does not attempt to implement a complete solution but adds an obvious notification informing the >> frameworks that `sun.misc.Unsafe::objectFieldOffset` and `sun.misc.Unsafe::staticField{Offset/Base}` >> not to support records. > I understand the intent. My point is that intent is mistimed during this time. I understand this > tripwire needs to be put while Records are still in preview. My point is that it cannot be put in > before Records serialization story has the preview-proven answer. > > The intent also looks rather opaque. The intent is to notify maintainers, fine. I am one of the > maintainers (although not of serialization library, but still heavy-Unsafe one), so hear me out. I > came up with the workaround above in about 15 minutes after someone pointed out the exception to me. > Have it crossed my mind that maybe JDK developers want some help here? No. It looks like another > impediment that should be solved on the spot. Does that exception communicate any intent at all? The > message is generic. There is no comment. How would you expect to push maintainers to think along the > "we should be collaborating to find a proper way to do this", instead of "this is set in stone; let > me hack this around too"? Records is in second preview (JEP 384).? As clearly stated in JEP 12 and [1],? introducing a feature as a preview feature enables developers to provide feedbacks before it becoems permanent in a future Java SE Platform. We should encourage developers to engage and provide feedbacks and have the discussion during preview. For JOL, have you raised an enhancement request to the serviceability team? Inspecting on an object layout would be useful and I suggest you file a RFE if you have done so. [1] https://docs.oracle.com/en/java/javase/14/language/preview-language-and-vm-features.html > This is the actionable bit: > > At very least the exception message should say something about the intent. And maybe even the > comment in Unsafe.java should point to the discussion about this intent and maybe even provide the > breadcrumbs to follow, e.g. to ObjectInputStream.readRecord(). To hit this exception, the app must be running with --enable-preview.?? There are documentation about preview feature and warning messages etc.?? To avoid duplicating the info, the best I could do may be to add "preview feature" in the exception message? Chris Hegarty will definitely want to help out w.r.t. the serialization frameworks in supporting records. Mandy From mandy.chung at oracle.com Thu Jun 18 21:12:15 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 18 Jun 2020 14:12:15 -0700 Subject: Review Request JDK-8244090: public lookup should find public members of public exported types Message-ID: <090cff25-2f48-d55f-721d-38acda80029b@oracle.com> Prior to JDK-8173978, publicLookup().in(C.class) produces a Lookup object on C with PUBLIC access which can be used to look up unconditionally exported public types from the module of C. Such lookup can only look up this C class defined by loader 1 but not another class named "C" defined by loader 2. JDK-8173978 adds the support for cross-module teleporting. publicLookup().in(C.class) was changed to produce a public Lookup i.e. with UNCONDITIONAL access.? A public lookup should be able to look up public members of any unconditionally exported public types including a class named "C" loaded by different loaders.? It reveals a bug in VM resolution for Lookup API that adds the loader constraints with java.lang.Object as the accessor that constraints a public lookup to load one type named C but any more. The lookup class of a public lookup is irrelevant to the lookup context.? Type consistency on P/Q/R live Class objects from the given MethodType (MT)is ensured at the following: 1. P/Q/R are consistent from the loader of the declaring class of the resolved member (D's loader) 2. P/Q/R are consistent w.r.t. the invoking class and the caller's MT at invocation time (the method handle carries the caller's MT that will be verified). The loader constraints added for public lookup is not necessary and adds undesirable constraints.? The proposed fixis to skip adding loader constraints if the lookup mode is either TRUSTED or UNCONDITIONAL. Webrev: http://cr.openjdk.java.net/~mchung/jdk16/webrevs/8244090/webrev.00/index.html Thanks Mandy From rahul.r.yadav at oracle.com Thu Jun 18 22:37:57 2020 From: rahul.r.yadav at oracle.com (Rahul Yadav) Date: Thu, 18 Jun 2020 23:37:57 +0100 Subject: RFR 8245302: Upgrade LogRecord to support long thread ids and remove its usage of ThreadLocal In-Reply-To: <03c38ee2-24c2-cf2d-e428-3f8747888978@oracle.com> References: <94B2555B-981F-40EB-AB5F-31E2BE068FF0@oracle.com> <03c38ee2-24c2-cf2d-e428-3f8747888978@oracle.com> Message-ID: <5e5bc99f-7d46-df1a-08c1-ee37958a3462@oracle.com> Hi Alan, Thank you for the feedback.I have updated the webrev. webrev : http://cr.openjdk.java.net/~ryadav/webrev_8245302/webrev.00/index.html - rahul On 14/06/2020 14:22, Alan Bateman wrote: > On 10/06/2020 16:06, Rahul wrote: >> ?? Hello, >> >> >> ?? Request to have my fix reviewed for the issue: >> >> >> ?? JDK-8245302:? Upgrade LogRecord to support long thread ids and >> remove its usage of ThreadLocal. >> >> >> ?? java.util.logging.LogRecord is updated to support long thread ids >> without ThreadLocal. >> >> ?? Before the update, thread id?s did not support long values, >> ThreadLocal usage has been >> >> ?? Removed. >> >> . >> >> >> ?? issue:?????? https://bugs.openjdk.java.net/browse/JDK-8245302 >> >> ?? webrev: http://cr.openjdk.java.net/~ryadav/webrev_8245302/index.html >> >> ?? csr: https://bugs.openjdk.java.net/browse/JDK-8247219 >> >> > The addition of setLongThreadID/getLongThreadID, and deprecating the > old methods looks good. One suggestion is that setLongThreadID can > return the LogRecord (= this) to allow for method invocation chaining. > For the javadoc then it would be clearer to say "Returns the ..." > rather than "Get a ..." because a sequence of calls to getXXX has to > return the same thread ID. > > LogRecord is serializable but it looks like like you've thought about > all the issue so I think this looks. > > shortThreadID looks a bit strange in that I would have expected it > would just return tid when in the 0-Integer.MAX_VALUE range and > compute a negative value when it's larger than Integer.MAX_VALUE. It's > true that the old behavior was to switch to dense values when larger > than MAX_VALUE/2 but I can't imagine how anything could depend on > this. So not objecting to the approach, just pointing out that there > may be simpler alternatives that could be explored (if you haven't > done so already). > > The overall test cover looks good. Can you provide clear instructions > in SerializeLogRecordTest on how to re-create the base64 string of the > serialized form? That is important for future maintainers. > > There are several formatting/style nits in the patch but they aren't > important for now. > > -Alan. > > > > > > From jonathan.gibbons at oracle.com Thu Jun 18 22:37:57 2020 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 18 Jun 2020 15:37:57 -0700 Subject: RFR [15,docs] JDK-8247880,bad HTML(href==...) in table Message-ID: <29fb2b2d-cb57-9873-10f7-2d89cdf085c0@oracle.com> Please review a trivial docs fix for some invalid HTML in jdk.zipfs/module-info.java. In two places, 'href==' is reduced to 'href=' -- Jon JBS: https://bugs.openjdk.java.net/browse/JDK-8247880 Patch inline diff -r cf0df75c75c1 src/jdk.zipfs/share/classes/module-info.java --- a/src/jdk.zipfs/share/classes/module-info.java Thu Jun 18 14:07:49 2020 -0700 +++ b/src/jdk.zipfs/share/classes/module-info.java Thu Jun 18 15:32:21 2020 -0700 @@ -234,9 +234,9 @@ ??* null/unset ??* ??* A value representing the version entry to use when accessing a - * + * ??* multi-release JAR. If the JAR is not a - * + * ??* multi-release JAR, the value will be ignored and the JAR will be ??* considered un-versioned. ??*

      From joe.darcy at oracle.com Thu Jun 18 22:41:21 2020 From: joe.darcy at oracle.com (Joe Darcy) Date: Thu, 18 Jun 2020 15:41:21 -0700 Subject: RFR [15,docs] JDK-8247880,bad HTML(href==...) in table In-Reply-To: <29fb2b2d-cb57-9873-10f7-2d89cdf085c0@oracle.com> References: <29fb2b2d-cb57-9873-10f7-2d89cdf085c0@oracle.com> Message-ID: +1 -Joe On 6/18/2020 3:37 PM, Jonathan Gibbons wrote: > Please review a trivial docs fix for some invalid HTML in > jdk.zipfs/module-info.java. > > In two places, 'href==' is reduced to 'href=' > > -- Jon > > JBS: https://bugs.openjdk.java.net/browse/JDK-8247880 > > Patch inline > > > diff -r cf0df75c75c1 src/jdk.zipfs/share/classes/module-info.java > --- a/src/jdk.zipfs/share/classes/module-info.java Thu Jun 18 14:07:49 > 2020 -0700 > +++ b/src/jdk.zipfs/share/classes/module-info.java Thu Jun 18 15:32:21 > 2020 -0700 > @@ -234,9 +234,9 @@ > ??* null/unset > ??* > ??* A value representing the version entry to use when accessing a > - * > + * > ??* multi-release JAR. If the JAR is not a > - * > + * > ??* multi-release JAR, the value will be ignored and the JAR will be > ??* considered un-versioned. > ??*

      > From lance.andersen at oracle.com Thu Jun 18 22:43:15 2020 From: lance.andersen at oracle.com (Lance @ Oracle) Date: Thu, 18 Jun 2020 18:43:15 -0400 Subject: RFR [15,docs] JDK-8247880,bad HTML(href==...) in table In-Reply-To: <29fb2b2d-cb57-9873-10f7-2d89cdf085c0@oracle.com> References: <29fb2b2d-cb57-9873-10f7-2d89cdf085c0@oracle.com> Message-ID: <494A9780-963D-4ED2-AAC6-A6859440194D@oracle.com> Looks good Best, Lance Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com Sent from my iPad > On Jun 18, 2020, at 6:40 PM, Jonathan Gibbons wrote: > > ?Please review a trivial docs fix for some invalid HTML in jdk.zipfs/module-info.java. > > In two places, 'href==' is reduced to 'href=' > > -- Jon > > JBS: https://bugs.openjdk.java.net/browse/JDK-8247880 > > Patch inline > > > diff -r cf0df75c75c1 src/jdk.zipfs/share/classes/module-info.java > --- a/src/jdk.zipfs/share/classes/module-info.java Thu Jun 18 14:07:49 2020 -0700 > +++ b/src/jdk.zipfs/share/classes/module-info.java Thu Jun 18 15:32:21 2020 -0700 > @@ -234,9 +234,9 @@ > * null/unset > * > * A value representing the version entry to use when accessing a > - * > + * > * multi-release JAR. If the JAR is not a > - * > + * > * multi-release JAR, the value will be ignored and the JAR will be > * considered un-versioned. > *

      > From jonathan.gibbons at oracle.com Thu Jun 18 23:00:11 2020 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 18 Jun 2020 16:00:11 -0700 Subject: RFR: [15,docs] JDK-8247894,Invalid @see in java.management Message-ID: <9e96f08b-17c2-fce3-5f95-29eb31d6287b@oracle.com> Please review a trivial fix for an invalid @see tag in java/lang/management/package.html. The presumed intent of the original is not a supported variant. The fix is to remove the `{@linkplain ...}` wrapper. -- Jon JBS: https://bugs.openjdk.java.net/browse/JDK-8247894 Patch inline: diff -r cf0df75c75c1 src/java.management/share/classes/java/lang/management/package.html --- a/src/java.management/share/classes/java/lang/management/package.html Thu Jun 18 14:07:49 2020 -0700 +++ b/src/java.management/share/classes/java/lang/management/package.html Thu Jun 18 15:51:09 2020 -0700 @@ -234,7 +234,7 @@ ?

      The java.lang.management API is thread-safe. - at see {@linkplain javax.management JMX Specification} + at see javax.management JMX Specification ?@author Mandy Chung ?@since 1.5 From joe.darcy at oracle.com Thu Jun 18 23:12:43 2020 From: joe.darcy at oracle.com (Joe Darcy) Date: Thu, 18 Jun 2020 16:12:43 -0700 Subject: RFR: [15,docs] JDK-8247894,Invalid @see in java.management In-Reply-To: <9e96f08b-17c2-fce3-5f95-29eb31d6287b@oracle.com> References: <9e96f08b-17c2-fce3-5f95-29eb31d6287b@oracle.com> Message-ID: +1 -Joe On 6/18/2020 4:00 PM, Jonathan Gibbons wrote: > Please review a trivial fix for an invalid @see tag in > java/lang/management/package.html. > The presumed intent of the original is not a supported variant. The > fix is to remove the > `{@linkplain ...}` wrapper. > > -- Jon > > JBS: https://bugs.openjdk.java.net/browse/JDK-8247894 > > Patch inline: > > diff -r cf0df75c75c1 > src/java.management/share/classes/java/lang/management/package.html > --- > a/src/java.management/share/classes/java/lang/management/package.html > Thu Jun 18 14:07:49 2020 -0700 > +++ > b/src/java.management/share/classes/java/lang/management/package.html > Thu Jun 18 15:51:09 2020 -0700 > @@ -234,7 +234,7 @@ > > ?

      The java.lang.management API is thread-safe. > > - at see {@linkplain javax.management JMX Specification} > + at see javax.management JMX Specification > > ?@author Mandy Chung > ?@since 1.5 > From mandy.chung at oracle.com Thu Jun 18 23:16:04 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 18 Jun 2020 16:16:04 -0700 Subject: RFR: [15,docs] JDK-8247894,Invalid @see in java.management In-Reply-To: <9e96f08b-17c2-fce3-5f95-29eb31d6287b@oracle.com> References: <9e96f08b-17c2-fce3-5f95-29eb31d6287b@oracle.com> Message-ID: <6d2f9e79-fa67-33f0-ad31-f84001174057@oracle.com> +1 Mandy On 6/18/20 4:00 PM, Jonathan Gibbons wrote: > > Please review a trivial fix for an invalid @see tag in > java/lang/management/package.html. > The presumed intent of the original is not a supported variant. The > fix is to remove the > `{@linkplain ...}` wrapper. > > -- Jon > > JBS: https://bugs.openjdk.java.net/browse/JDK-8247894 > > Patch inline: > > diff -r cf0df75c75c1 > src/java.management/share/classes/java/lang/management/package.html > --- > a/src/java.management/share/classes/java/lang/management/package.html > Thu Jun 18 14:07:49 2020 -0700 > +++ > b/src/java.management/share/classes/java/lang/management/package.html > Thu Jun 18 15:51:09 2020 -0700 > @@ -234,7 +234,7 @@ > > ?

      The java.lang.management API is thread-safe. > > - at see {@linkplain javax.management JMX Specification} > + at see javax.management JMX Specification > > ?@author Mandy Chung > ?@since 1.5 > From jonathan.gibbons at oracle.com Thu Jun 18 23:16:34 2020 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 18 Jun 2020 16:16:34 -0700 Subject: RFR: [15, docs] JDK-8247896, Invalid (@throw) tags in 2 java.io classes Message-ID: <5809a933-a1d8-7a23-1fd3-b24e03d6e600@oracle.com> Please review a trivial fix for a couple of misnamed tags in 2 classes in the java.io package. In both cases, @throw should be @throws -- Jon JBS: https://bugs.openjdk.java.net/browse/JDK-8247896 Patch inline: diff -r d66e8770295e src/java.base/share/classes/java/io/PipedWriter.java --- a/src/java.base/share/classes/java/io/PipedWriter.java Thu Jun 18 15:54:26 2020 -0700 +++ b/src/java.base/share/classes/java/io/PipedWriter.java Thu Jun 18 16:11:28 2020 -0700 @@ -112,7 +112,7 @@ ??????* Implements the {@code write} method of {@code Writer}. ??????* ??????* @param c the {@code char} to be written. - * @throw IOException if the pipe is + * @throws IOException if the pipe is ??????* {@code broken}, ??????* {@link #connect(java.io.PipedReader) unconnected}, closed ??????* or an I/O error occurs. diff -r d66e8770295e src/java.base/share/classes/java/io/SequenceInputStream.java --- a/src/java.base/share/classes/java/io/SequenceInputStream.java Thu Jun 18 15:54:26 2020 -0700 +++ b/src/java.base/share/classes/java/io/SequenceInputStream.java Thu Jun 18 16:11:28 2020 -0700 @@ -121,7 +121,7 @@ ??????* skipped over) from the current underlying input stream ??????* without blocking or {@code 0} if this input stream ??????* has been closed by invoking its {@link #close()} method - * @throw IOException if an I/O error occurs. + * @throws IOException if an I/O error occurs. ??????* ??????* @since 1.1 ??????*/ From mandy.chung at oracle.com Thu Jun 18 23:20:17 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 18 Jun 2020 16:20:17 -0700 Subject: RFR: [15, docs] JDK-8247896, Invalid (@throw) tags in 2 java.io classes In-Reply-To: <5809a933-a1d8-7a23-1fd3-b24e03d6e600@oracle.com> References: <5809a933-a1d8-7a23-1fd3-b24e03d6e600@oracle.com> Message-ID: <3794345c-0907-22da-dead-ce2f5cb02b60@oracle.com> +1 Mandy On 6/18/20 4:16 PM, Jonathan Gibbons wrote: > Please review a trivial fix for a couple of misnamed tags in 2 classes in > the java.io package. In both cases, @throw should be @throws > > -- Jon > > JBS: https://bugs.openjdk.java.net/browse/JDK-8247896 > > Patch inline: > > diff -r d66e8770295e src/java.base/share/classes/java/io/PipedWriter.java > --- a/src/java.base/share/classes/java/io/PipedWriter.java Thu Jun 18 > 15:54:26 2020 -0700 > +++ b/src/java.base/share/classes/java/io/PipedWriter.java Thu Jun 18 > 16:11:28 2020 -0700 > @@ -112,7 +112,7 @@ > ??????* Implements the {@code write} method of {@code Writer}. > ??????* > ??????* @param c the {@code char} to be written. > - * @throw IOException if the pipe is > + * @throws IOException if the pipe is > ??????* {@code broken}, > ??????* {@link #connect(java.io.PipedReader) unconnected}, closed > ??????* or an I/O error occurs. > diff -r d66e8770295e > src/java.base/share/classes/java/io/SequenceInputStream.java > --- a/src/java.base/share/classes/java/io/SequenceInputStream.java Thu > Jun 18 15:54:26 2020 -0700 > +++ b/src/java.base/share/classes/java/io/SequenceInputStream.java Thu > Jun 18 16:11:28 2020 -0700 > @@ -121,7 +121,7 @@ > ??????* skipped over) from the current underlying input stream > ??????* without blocking or {@code 0} if this input stream > ??????* has been closed by invoking its {@link #close()} method > - * @throw IOException if an I/O error occurs. > + * @throws IOException if an I/O error occurs. > ??????* > ??????* @since 1.1 > ??????*/ > From paul.sandoz at oracle.com Thu Jun 18 23:35:32 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 18 Jun 2020 16:35:32 -0700 Subject: RFR [15] 8247696: Incorrect tail computation for large segments in AbstractMemorySegmentImpl::mismatch In-Reply-To: <94fceeda-edee-c571-b32d-af8d5aed672e@oracle.com> References: <0B15B6AF-F196-4192-B448-F1955AE586A7@oracle.com> <94fceeda-edee-c571-b32d-af8d5aed672e@oracle.com> Message-ID: <7BA53832-4D14-405D-9AE0-118ABB1344A2@oracle.com> Thanks Chris. > On Jun 18, 2020, at 2:57 AM, Maurizio Cimadamore wrote: > > Thanks for looking at this Chris > > On 17/06/2020 21:56, Paul Sandoz wrote: >> Hi Chris, >> >> AbstractMemorySegmentImpl >> ? >> >> In vectorizedMismatchLarge: >> >> 163 if (remaining > 7) >> 164 throw new InternalError("remaining greater than 7: " + remaining); >> 165 i = length - remaining; >> 166 } >> >> Should this check be an assert? > I suggested that to Chris, since sometimes asserts are enabled when running tests, sometimes are not - in langtools we moved away from using asserts as we realized that in certain cases they were silently failing. I'm ok with whatever standard you feel comfortable with though. Automated running of tests enable assertions (-ea -esa), perhaps the use is more commonly accepted in library code than compiler code. I would favor so in this case if necessary (sometimes they are avoided to reduce inline thresholds). >> >> ? >> >> This fix prompted me to think more deeply about the implementation of vectorizedMismatchLarge and its use of vectorizedMismatch. Sorry, I should have thought about this more throughly earlier on. >> >> We need to refine the approach, not for this patch, but something to follow up after. I think there are two issues. >> >> 1) The intrinsic method vectorizedMismatch could potentially bomb out at any point and return the bitwise compliment of the remaining number of elements to check. >> >> Obviously, there is no point doing that arbitrarily but a stub implementation for, say, x86 AVX-512 might decide bomb out for < 64 remaining elements, rather than apply vector operations on smaller vector sizes or use a scalar loop. It does not today, but I think we should guard against that potentially happening, otherwise bad things can happen. > > So your worry here is that we'll end up with an infinite loop, right? > Or more likely that an incorrect result is returned since tail elements will be skipped over as the offset and size is updated. > If so, we could check remaining against previous remaining and bomb out too if no further progress seem to be made? > I think it's better to always terminate the loop after the last sub-range is checked, rather than unnecessarily calling twice. I am not confident the vectorizedMismatch intrinsic stub has been tested properly on very small lengths, since it's never directly called in such cases, so also keeping "remaining > 7" is good too. Paul. >> >> I think the loop should exit when the last sub-range has been checked. We should rely on other tests to ensure the intrinsic method is operating efficiently. >> >> >> 2) This method only works when operating on byte arrays. It will not work correctly if operating on short or long arrays, since we are not adjusting the length and offsets accordingly by the scale. It's probably easier to just rename this as vectorizedMismatchLargeForBytes and drop the log2ArrayIndexScale argument. Then expand later if need be. I still think the method rightly belongs in ArraysSupport. > > Yep - probably good idea to restrict on bytes, for now. > > Maurizio > >> >> Paul. >> >>> On Jun 17, 2020, at 8:33 AM, Chris Hegarty wrote: >>> >>> The MemorySegment::mismatch implementation added vectorized mismatch of long sizes. The implementation is trivial, but the starting point for a more optimized implementation, if needed. ArraysSupport::vectorizedMismatchLarge incorrectly returns the bitwise complement of the offset of the first mismatch, where it should return the bitwise complement of the number of remaining pairs of elements to be checked in the tail of the two arrays. The AbstractMemorySegmentImpl::mismatch masked this problem, since it seamlessly compared the remaining tail, which is larger than it should be. >>> >>> Webrev: >>> https://cr.openjdk.java.net/~chegar/8247696/webrev.00/ >>> >>> I updated the exiting BulkOps micro-benchmark to cover mismatch. Here are the results, compared to ByteBuffer::mismatch, on my machine: >>> >>> Benchmark Mode Cnt Score Error Units >>> BulkOps.mismatch_large_bytebuffer avgt 30 740186.973 ? 119314.207 ns/op >>> BulkOps.mismatch_large_segment avgt 30 683475.305 ? 76355.043 ns/op >>> BulkOps.mismatch_small_bytebuffer avgt 30 7.367 ? 0.523 ns/op >>> BulkOps.mismatch_small_segment avgt 30 4.140 ? 0.602 ns/op >>> >>> >>> -Chris. From jonathan.gibbons at oracle.com Fri Jun 19 00:18:52 2020 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Thu, 18 Jun 2020 17:18:52 -0700 Subject: RFR: [15,docs] JDK-8247899,HTML errors and warnings in threadPrimitiveDeprecation.html Message-ID: <530bd8ea-3148-ee9b-4507-a2ba32dd2e4b@oracle.com> Please review this trivial fix to some minor issues reported by doclint. In the first change, the `


      `? contained a hangover of some HTML 4 attributes which are not supported in HTML5. They are simply deleted. In the second change, a paragraph just contained a comment. While it might be reasonable to just remove the paragraph tags surrounding the comment, the comment itself ("Body text ends here") is somewhat redundant, appearing as it does right before `` and so the entire line is deleted. -- Jon JBS: https://bugs.openjdk.java.net/browse/JDK-8247899 Inline patch: diff -r f80508232687 src/java.base/share/classes/java/lang/doc-files/threadPrimitiveDeprecation.html --- a/src/java.base/share/classes/java/lang/doc-files/threadPrimitiveDeprecation.html Thu Jun 18 16:21:34 2020 -0700 +++ b/src/java.base/share/classes/java/lang/doc-files/threadPrimitiveDeprecation.html Thu Jun 18 17:10:24 2020 -0700 @@ -282,7 +282,7 @@ ?????????} ?????} ? -
      +
      ?

      Can I combine the two techniques to produce a thread that may ?be safely "stopped" or "suspended"?

      ?Yes, it's reasonably straightforward. The one subtlety is that the @@ -324,6 +324,5 @@ ?described above, it needn't call notify as well, but it ?still must be synchronized. This ensures that the target thread ?won't miss an interrupt due to a race condition. -

      ? ? From lance.andersen at oracle.com Fri Jun 19 00:30:25 2020 From: lance.andersen at oracle.com (Lance @ Oracle) Date: Thu, 18 Jun 2020 20:30:25 -0400 Subject: RFR: [15, docs] JDK-8247899, HTML errors and warnings in threadPrimitiveDeprecation.html In-Reply-To: <530bd8ea-3148-ee9b-4507-a2ba32dd2e4b@oracle.com> References: <530bd8ea-3148-ee9b-4507-a2ba32dd2e4b@oracle.com> Message-ID: Looks reasonable Jon Best, Lance Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com Sent from my iPad > On Jun 18, 2020, at 8:19 PM, Jonathan Gibbons wrote: > > ?Please review this trivial fix to some minor issues reported by doclint. > > In the first change, the `
      ` contained a hangover of some HTML 4 attributes > which are not supported in HTML5. They are simply deleted. > > In the second change, a paragraph just contained a comment. While it might > be reasonable to just remove the paragraph tags surrounding the comment, > the comment itself ("Body text ends here") is somewhat redundant, appearing > as it does right before `` and so the entire line is deleted. > > -- Jon > > JBS: https://bugs.openjdk.java.net/browse/JDK-8247899 > > Inline patch: > > diff -r f80508232687 src/java.base/share/classes/java/lang/doc-files/threadPrimitiveDeprecation.html > --- a/src/java.base/share/classes/java/lang/doc-files/threadPrimitiveDeprecation.html Thu Jun 18 16:21:34 2020 -0700 > +++ b/src/java.base/share/classes/java/lang/doc-files/threadPrimitiveDeprecation.html Thu Jun 18 17:10:24 2020 -0700 > @@ -282,7 +282,7 @@ > } > } > > -
      > +
      >

      Can I combine the two techniques to produce a thread that may > be safely "stopped" or "suspended"?

      > Yes, it's reasonably straightforward. The one subtlety is that the > @@ -324,6 +324,5 @@ > described above, it needn't call notify as well, but it > still must be synchronized. This ensures that the target thread > won't miss an interrupt due to a race condition. > -

      > > > From sergei.tsypanov at yandex.ru Fri Jun 19 04:57:25 2020 From: sergei.tsypanov at yandex.ru (=?utf-8?B?0KHQtdGA0LPQtdC5INCm0YvQv9Cw0L3QvtCy?=) Date: Fri, 19 Jun 2020 06:57:25 +0200 Subject: [PATCH] remove redundant initialization of volatile fields with default values Message-ID: <281231592473099@mail.yandex.ru> Hello, while investigating an issue I've found out that assignment of default value to volatile fields slows down object instantiation. Consider the benchmark: @State(Scope.Thread) @OutputTimeUnit(TimeUnit.NANOSECONDS) @BenchmarkMode(value = Mode.AverageTime) @Fork(jvmArgsAppend = {"-Xms2g", "-Xmx2g"}) public class VolatileFieldBenchmark { @Benchmark public Object explicitInit() { return new ExplicitInit(); } @Benchmark public Object noInit() { return new NoInit(); } private static class ExplicitInit { private volatile boolean field = false; } private static class NoInit { private volatile boolean field; } } This gives the following results as of my machine: Benchmark Mode Cnt Score Error Units VolatileFieldBenchmark.explicitInit avgt 40 11.087 ? 0.140 ns/op VolatileFieldBenchmark.noInit avgt 40 3.367 ? 0.131 ns/op I've looked into source code of java.base and found out several cases where the default value is assigned to volatile field. Getting rid of such assignements demonstates improvement as of object instantiation, e.g. javax.security.auth.Subject: Mode Cnt Score Error Units before avgt 40 35.933 ? 2.647 ns/op after avgt 40 30.817 ? 2.384 ns/op As of testing tier1 and tier2 are both ok after the changes. Best regards, Sergey Tsypanov -------------- next part -------------- A non-text attachment was scrubbed... Name: volatile.patch Type: text/x-diff Size: 6253 bytes Desc: not available URL: From david.holmes at oracle.com Fri Jun 19 06:58:36 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 19 Jun 2020 16:58:36 +1000 Subject: RFR: [15,docs] JDK-8247899,HTML errors and warnings in threadPrimitiveDeprecation.html In-Reply-To: <530bd8ea-3148-ee9b-4507-a2ba32dd2e4b@oracle.com> References: <530bd8ea-3148-ee9b-4507-a2ba32dd2e4b@oracle.com> Message-ID: <1c63744c-5656-176e-a189-9fdcbe88443b@oracle.com> Looks good and trivial! Thanks, David On 19/06/2020 10:18 am, Jonathan Gibbons wrote: > Please review this trivial fix to some minor issues reported by doclint. > > In the first change, the `
      `? contained a hangover of some HTML 4 > attributes > which are not supported in HTML5. They are simply deleted. > > In the second change, a paragraph just contained a comment. While it might > be reasonable to just remove the paragraph tags surrounding the comment, > the comment itself ("Body text ends here") is somewhat redundant, appearing > as it does right before `` and so the entire line is deleted. > > -- Jon > > JBS: https://bugs.openjdk.java.net/browse/JDK-8247899 > > Inline patch: > > diff -r f80508232687 > src/java.base/share/classes/java/lang/doc-files/threadPrimitiveDeprecation.html > > --- > a/src/java.base/share/classes/java/lang/doc-files/threadPrimitiveDeprecation.html > Thu Jun 18 16:21:34 2020 -0700 > +++ > b/src/java.base/share/classes/java/lang/doc-files/threadPrimitiveDeprecation.html > Thu Jun 18 17:10:24 2020 -0700 > @@ -282,7 +282,7 @@ > ?????????} > ?????} > ? > -
      > +
      > ?

      Can I combine the two techniques to produce a thread that may > ?be safely "stopped" or "suspended"?

      > ?Yes, it's reasonably straightforward. The one subtlety is that the > @@ -324,6 +324,5 @@ > ?described above, it needn't call notify as well, but it > ?still must be synchronized. This ensures that the target thread > ?won't miss an interrupt due to a race condition. > -

      > ? > ? > From Alan.Bateman at oracle.com Fri Jun 19 07:43:42 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 19 Jun 2020 08:43:42 +0100 Subject: RFR 8245302: Upgrade LogRecord to support long thread ids and remove its usage of ThreadLocal In-Reply-To: <5e5bc99f-7d46-df1a-08c1-ee37958a3462@oracle.com> References: <94B2555B-981F-40EB-AB5F-31E2BE068FF0@oracle.com> <03c38ee2-24c2-cf2d-e428-3f8747888978@oracle.com> <5e5bc99f-7d46-df1a-08c1-ee37958a3462@oracle.com> Message-ID: <58cda3b5-b75b-6782-01eb-2c935a0e402d@oracle.com> On 18/06/2020 23:37, Rahul Yadav wrote: > Hi Alan, > > Thank you for the feedback.I have updated the webrev. > > webrev : > http://cr.openjdk.java.net/~ryadav/webrev_8245302/webrev.00/index.html This looks quite good. The comment in shortShortID has "any positive long less than Integer.MAX_VALUE" but it's actually <= MAX_VALUE. I don't think MIN_SEQUENTIAL_THREAD_ID is used so I assume it can be removed. The @return for setLongThreadID has a description "Log Record" but this should "this LogRecord". Can you update SerializeLogRecordTest with clear instructions on how to generate the stream? This will help future maintainers that may have to update this test. -Alan From Alan.Bateman at oracle.com Fri Jun 19 07:53:11 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 19 Jun 2020 08:53:11 +0100 Subject: [PATCH] remove redundant initialization of volatile fields with default values In-Reply-To: <281231592473099@mail.yandex.ru> References: <281231592473099@mail.yandex.ru> Message-ID: On 19/06/2020 05:57, ?????? ??????? wrote: > Hello, > > while investigating an issue I've found out that assignment of default value to volatile fields slows down object instantiation. > Yes, there's been several efforts over the years to eliminate the initializing volatile fields to their default values. It's good to keep these in check. It's probably best to get a sponsor on the security-libs list as all by one of these in the security code. -Alan From forax at univ-mlv.fr Fri Jun 19 08:03:59 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 19 Jun 2020 10:03:59 +0200 (CEST) Subject: [PATCH] remove redundant initialization of volatile fields with default values In-Reply-To: <281231592473099@mail.yandex.ru> References: <281231592473099@mail.yandex.ru> Message-ID: <1084247143.1025326.1592553839671.JavaMail.zimbra@u-pem.fr> Hi Sergei, the problem is that you are changing the semantics if there are several fields. By example with the code below, you have the guarantee that the code will print 4 (if it prints something), if you remove the assignment field = false, the code can print 0 or 4. class A { int i = 4; volatile boolean field = false; } thread 1: global = new A() thread 2: var a = global; if (a != null) { System.out.println(a.i); } regards, R?mi ----- Mail original ----- > De: "?????? ???????" > ?: "core-libs-dev" > Envoy?: Vendredi 19 Juin 2020 06:57:25 > Objet: [PATCH] remove redundant initialization of volatile fields with default values > Hello, > > while investigating an issue I've found out that assignment of default value to > volatile fields slows down object instantiation. > > Consider the benchmark: > > @State(Scope.Thread) > @OutputTimeUnit(TimeUnit.NANOSECONDS) > @BenchmarkMode(value = Mode.AverageTime) > @Fork(jvmArgsAppend = {"-Xms2g", "-Xmx2g"}) > public class VolatileFieldBenchmark { > @Benchmark > public Object explicitInit() { > return new ExplicitInit(); > } > > @Benchmark > public Object noInit() { > return new NoInit(); > } > > private static class ExplicitInit { > private volatile boolean field = false; > } > private static class NoInit { > private volatile boolean field; > } > } > > This gives the following results as of my machine: > > Benchmark Mode Cnt Score Error Units > VolatileFieldBenchmark.explicitInit avgt 40 11.087 ? 0.140 ns/op > VolatileFieldBenchmark.noInit avgt 40 3.367 ? 0.131 ns/op > > > I've looked into source code of java.base and found out several cases where the > default value is assigned to volatile field. > > Getting rid of such assignements demonstates improvement as of object > instantiation, e.g. javax.security.auth.Subject: > > Mode Cnt Score Error Units > before avgt 40 35.933 ? 2.647 ns/op > after avgt 40 30.817 ? 2.384 ns/op > > As of testing tier1 and tier2 are both ok after the changes. > > Best regards, > Sergey Tsypanov From alexander.matveev at oracle.com Fri Jun 19 08:46:03 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Fri, 19 Jun 2020 01:46:03 -0700 Subject: RFR: 8225251: Minimise import statements in jpackage sources Message-ID: Please review the jpackage fix for bug [1] at [2]. Cleanup import statements. [1] https://bugs.openjdk.java.net/browse/JDK-8225251 [2] http://cr.openjdk.java.net/~almatvee/8225251/webrev.00/ Thanks, Alexander From daniel.fuchs at oracle.com Fri Jun 19 08:58:50 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 19 Jun 2020 09:58:50 +0100 Subject: RFR 8245302: Upgrade LogRecord to support long thread ids and remove its usage of ThreadLocal In-Reply-To: <5e5bc99f-7d46-df1a-08c1-ee37958a3462@oracle.com> References: <94B2555B-981F-40EB-AB5F-31E2BE068FF0@oracle.com> <03c38ee2-24c2-cf2d-e428-3f8747888978@oracle.com> <5e5bc99f-7d46-df1a-08c1-ee37958a3462@oracle.com> Message-ID: <40999b8b-3c66-4b90-0972-564106e76aea@oracle.com> Hi Rahul, This looks very good. I have some comments regarding the SerializeLogRecord test: - the test speaks of JDK 15 at several places: is it really JDK 15, or should it be JDK 16? I mean - was the serialized bytes generated before your fix or after? - the generate() method (no args) should probably have a test case where a log record has a long thread id > Integer.MAX_VALUE. - it might be good to work on a version of this test that could be backported to JDK 15 (and 11) to verify that a LogRecord serialized with JDK 16 can be deserialized in those versions. You should probably log a JBS issue to follow-up on that. best regards, -- daniel On 18/06/2020 23:37, Rahul Yadav wrote: > Hi Alan, > > Thank you for the feedback.I have updated the webrev. > > webrev : > http://cr.openjdk.java.net/~ryadav/webrev_8245302/webrev.00/index.html > > - rahul From rahul.r.yadav at oracle.com Fri Jun 19 09:00:59 2020 From: rahul.r.yadav at oracle.com (Rahul Yadav) Date: Fri, 19 Jun 2020 10:00:59 +0100 Subject: RFR 8245302: Upgrade LogRecord to support long thread ids and remove its usage of ThreadLocal In-Reply-To: <58cda3b5-b75b-6782-01eb-2c935a0e402d@oracle.com> References: <94B2555B-981F-40EB-AB5F-31E2BE068FF0@oracle.com> <03c38ee2-24c2-cf2d-e428-3f8747888978@oracle.com> <5e5bc99f-7d46-df1a-08c1-ee37958a3462@oracle.com> <58cda3b5-b75b-6782-01eb-2c935a0e402d@oracle.com> Message-ID: <362dc841-a809-95f5-961f-3856f3230b6f@oracle.com> Thank you Alan, updated webrev. webrev : http://cr.openjdk.java.net/~ryadav/webrev_8245302/webrev.00/index.html - rahul On 19/06/2020 08:43, Alan Bateman wrote: > On 18/06/2020 23:37, Rahul Yadav wrote: >> Hi Alan, >> >> Thank you for the feedback.I have updated the webrev. >> >> webrev : >> http://cr.openjdk.java.net/~ryadav/webrev_8245302/webrev.00/index.html > This looks quite good. > > The comment in shortShortID has "any positive long less than > Integer.MAX_VALUE" but it's actually <= MAX_VALUE. > > I don't think MIN_SEQUENTIAL_THREAD_ID is used so I assume it can be > removed. > > The @return for setLongThreadID has a description "Log Record" but > this should "this LogRecord". > > Can you update SerializeLogRecordTest with clear instructions on how > to generate the stream? This will help future maintainers that may > have to update this test. > > -Alan From sergei.tsypanov at yandex.ru Fri Jun 19 09:33:00 2020 From: sergei.tsypanov at yandex.ru (=?utf-8?B?0KHQtdGA0LPQtdC5INCm0YvQv9Cw0L3QvtCy?=) Date: Fri, 19 Jun 2020 11:33:00 +0200 Subject: [PATCH] remove redundant initialization of volatile fields with default values In-Reply-To: <1084247143.1025326.1592553839671.JavaMail.zimbra@u-pem.fr> References: <281231592473099@mail.yandex.ru> <1084247143.1025326.1592553839671.JavaMail.zimbra@u-pem.fr> Message-ID: <180351592557620@mail.yandex.ru> Hello Remi, thanks for pointing this out, I didn't take this into account. As I understand, the volatile semantics here covers all the fields of an object, no matter whether they are declared before or after volatile field (probably actual object layout might be different from declaration in source code). If that's true I think we still can apply this optimization at least for 1) classes with single field and without super-classes 2) classes with all non-volatile fields declared final 3) classes with not-initialized non-volatile fields 4) classes with super-classes where non-volatile fields are not initialized (or inialized with default values) Looking at source code I see that we could keep java.security.KeyStore and its nested class PasswordProtection along with java.util.ListResourceBundle. Please correct me if I miss anything in my speculation. Regards, Sergey Tsypanov 19.06.2020, 10:04, "Remi Forax" : > Hi Sergei, > the problem is that you are changing the semantics if there are several fields. > > By example with the code below, you have the guarantee that the code will print 4 (if it prints something), > if you remove the assignment field = false, the code can print 0 or 4. > > ??class A { > ????int i = 4; > ????volatile boolean field = false; > ??} > > thread 1: > ??global = new A() > > thread 2: > ??var a = global; > ??if (a != null) { > ????System.out.println(a.i); > ??} > > regards, > R?mi > > ----- Mail original ----- >> ?De: "?????? ???????" >> ??: "core-libs-dev" >> ?Envoy?: Vendredi 19 Juin 2020 06:57:25 >> ?Objet: [PATCH] remove redundant initialization of volatile fields with default values > >> ?Hello, >> >> ?while investigating an issue I've found out that assignment of default value to >> ?volatile fields slows down object instantiation. >> >> ?Consider the benchmark: >> >> ?@State(Scope.Thread) >> ?@OutputTimeUnit(TimeUnit.NANOSECONDS) >> ?@BenchmarkMode(value = Mode.AverageTime) >> ?@Fork(jvmArgsAppend = {"-Xms2g", "-Xmx2g"}) >> ?public class VolatileFieldBenchmark { >> ??@Benchmark >> ??public Object explicitInit() { >> ????return new ExplicitInit(); >> ??} >> >> ??@Benchmark >> ??public Object noInit() { >> ????return new NoInit(); >> ??} >> >> ??private static class ExplicitInit { >> ????private volatile boolean field = false; >> ??} >> ??private static class NoInit { >> ????private volatile boolean field; >> ??} >> ?} >> >> ?This gives the following results as of my machine: >> >> ?Benchmark Mode Cnt Score Error Units >> ?VolatileFieldBenchmark.explicitInit avgt 40 11.087 ? 0.140 ns/op >> ?VolatileFieldBenchmark.noInit avgt 40 3.367 ? 0.131 ns/op >> >> ?I've looked into source code of java.base and found out several cases where the >> ?default value is assigned to volatile field. >> >> ?Getting rid of such assignements demonstates improvement as of object >> ?instantiation, e.g. javax.security.auth.Subject: >> >> ???????????Mode Cnt Score Error Units >> ?before avgt 40 35.933 ? 2.647 ns/op >> ?after avgt 40 30.817 ? 2.384 ns/op >> >> ?As of testing tier1 and tier2 are both ok after the changes. >> >> ?Best regards, >> ?Sergey Tsypanov -------------- next part -------------- A non-text attachment was scrubbed... Name: volatile.patch Type: text/x-diff Size: 1269 bytes Desc: not available URL: From rahul.r.yadav at oracle.com Fri Jun 19 10:49:24 2020 From: rahul.r.yadav at oracle.com (Rahul Yadav) Date: Fri, 19 Jun 2020 11:49:24 +0100 Subject: RFR 8245302: Upgrade LogRecord to support long thread ids and remove its usage of ThreadLocal In-Reply-To: <40999b8b-3c66-4b90-0972-564106e76aea@oracle.com> References: <94B2555B-981F-40EB-AB5F-31E2BE068FF0@oracle.com> <03c38ee2-24c2-cf2d-e428-3f8747888978@oracle.com> <5e5bc99f-7d46-df1a-08c1-ee37958a3462@oracle.com> <40999b8b-3c66-4b90-0972-564106e76aea@oracle.com> Message-ID: <80f80fd0-0c10-4f67-e795-d0b0a9da9d11@oracle.com> Hi Daniel, Thank you for the feedback. I have added the new case for thread id > Integer.MAX_VALUE. have updated comments/names and string to JDK 16. I will follow-up with a new JBS issue for the backport webrev : http://cr.openjdk.java.net/~ryadav/webrev_8245302/webrev.00/index.html - rahul On 19/06/2020 09:58, Daniel Fuchs wrote: > Hi Rahul, > > This looks very good. > > I have some comments regarding the SerializeLogRecord test: > > - the test speaks of JDK 15 at several places: is it really JDK 15, or > ? should it be JDK 16? I mean - was the serialized bytes generated > ? before your fix or after? > > - the generate() method (no args) should probably have a test case > ? where a log record has a long thread id > Integer.MAX_VALUE. > > - it might be good to work on a version of this test that could > ? be backported to JDK 15 (and 11) to verify that a LogRecord > ? serialized with JDK 16 can be deserialized in those versions. > ? You should probably log a JBS issue to follow-up on that. > > best regards, > > -- daniel > > > On 18/06/2020 23:37, Rahul Yadav wrote: >> Hi Alan, >> >> Thank you for the feedback.I have updated the webrev. >> >> webrev : >> http://cr.openjdk.java.net/~ryadav/webrev_8245302/webrev.00/index.html >> >> - rahul From chris.hegarty at oracle.com Fri Jun 19 10:56:10 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 19 Jun 2020 11:56:10 +0100 Subject: RFR [15] 8247696: Incorrect tail computation for large segments in AbstractMemorySegmentImpl::mismatch In-Reply-To: <7BA53832-4D14-405D-9AE0-118ABB1344A2@oracle.com> References: <0B15B6AF-F196-4192-B448-F1955AE586A7@oracle.com> <94fceeda-edee-c571-b32d-af8d5aed672e@oracle.com> <7BA53832-4D14-405D-9AE0-118ABB1344A2@oracle.com> Message-ID: Paul, Maurizio, This version incorporates all feedback so far. https://cr.openjdk.java.net/~chegar/8247696/webrev.01/ Results on my machine: Benchmark Mode Cnt Score Error Units BulkOps.mismatch_large_bytebuffer avgt 30 88266.728 ? 4083.476 ns/op BulkOps.mismatch_large_segment avgt 30 86141.343 ? 2450.450 ns/op BulkOps.mismatch_small_bytebuffer avgt 30 6.360 ? 0.425 ns/op BulkOps.mismatch_small_segment avgt 30 4.582 ? 1.040 ns/op -Chris. > On 19 Jun 2020, at 00:35, Paul Sandoz wrote: > > Thanks Chris. > >> On Jun 18, 2020, at 2:57 AM, Maurizio Cimadamore wrote: >> >> Thanks for looking at this Chris >> >> On 17/06/2020 21:56, Paul Sandoz wrote: >>> Hi Chris, >>> >>> AbstractMemorySegmentImpl >>> ? >>> >>> In vectorizedMismatchLarge: >>> >>> 163 if (remaining > 7) >>> 164 throw new InternalError("remaining greater than 7: " + remaining); >>> 165 i = length - remaining; >>> 166 } >>> >>> Should this check be an assert? >> I suggested that to Chris, since sometimes asserts are enabled when running tests, sometimes are not - in langtools we moved away from using asserts as we realized that in certain cases they were silently failing. I'm ok with whatever standard you feel comfortable with though. > > Automated running of tests enable assertions (-ea -esa), perhaps the use is more commonly accepted in library code than compiler code. I would favor so in this case if necessary (sometimes they are avoided to reduce inline thresholds). > > >>> >>> ? >>> >>> This fix prompted me to think more deeply about the implementation of vectorizedMismatchLarge and its use of vectorizedMismatch. Sorry, I should have thought about this more throughly earlier on. >>> >>> We need to refine the approach, not for this patch, but something to follow up after. I think there are two issues. >>> >>> 1) The intrinsic method vectorizedMismatch could potentially bomb out at any point and return the bitwise compliment of the remaining number of elements to check. >>> >>> Obviously, there is no point doing that arbitrarily but a stub implementation for, say, x86 AVX-512 might decide bomb out for < 64 remaining elements, rather than apply vector operations on smaller vector sizes or use a scalar loop. It does not today, but I think we should guard against that potentially happening, otherwise bad things can happen. >> >> So your worry here is that we'll end up with an infinite loop, right? >> > > Or more likely that an incorrect result is returned since tail elements will be skipped over as the offset and size is updated. > > >> If so, we could check remaining against previous remaining and bomb out too if no further progress seem to be made? >> > > I think it's better to always terminate the loop after the last sub-range is checked, rather than unnecessarily calling twice. > > I am not confident the vectorizedMismatch intrinsic stub has been tested properly on very small lengths, since it's never directly called in such cases, so also keeping "remaining > 7" is good too. > > Paul. > >>> >>> I think the loop should exit when the last sub-range has been checked. We should rely on other tests to ensure the intrinsic method is operating efficiently. >>> >>> >>> 2) This method only works when operating on byte arrays. It will not work correctly if operating on short or long arrays, since we are not adjusting the length and offsets accordingly by the scale. It's probably easier to just rename this as vectorizedMismatchLargeForBytes and drop the log2ArrayIndexScale argument. Then expand later if need be. I still think the method rightly belongs in ArraysSupport. >> >> Yep - probably good idea to restrict on bytes, for now. >> >> Maurizio >> >>> >>> Paul. >>> >>>> On Jun 17, 2020, at 8:33 AM, Chris Hegarty wrote: >>>> >>>> The MemorySegment::mismatch implementation added vectorized mismatch of long sizes. The implementation is trivial, but the starting point for a more optimized implementation, if needed. ArraysSupport::vectorizedMismatchLarge incorrectly returns the bitwise complement of the offset of the first mismatch, where it should return the bitwise complement of the number of remaining pairs of elements to be checked in the tail of the two arrays. The AbstractMemorySegmentImpl::mismatch masked this problem, since it seamlessly compared the remaining tail, which is larger than it should be. >>>> >>>> Webrev: >>>> https://cr.openjdk.java.net/~chegar/8247696/webrev.00/ >>>> >>>> I updated the exiting BulkOps micro-benchmark to cover mismatch. Here are the results, compared to ByteBuffer::mismatch, on my machine: >>>> >>>> Benchmark Mode Cnt Score Error Units >>>> BulkOps.mismatch_large_bytebuffer avgt 30 740186.973 ? 119314.207 ns/op >>>> BulkOps.mismatch_large_segment avgt 30 683475.305 ? 76355.043 ns/op >>>> BulkOps.mismatch_small_bytebuffer avgt 30 7.367 ? 0.523 ns/op >>>> BulkOps.mismatch_small_segment avgt 30 4.140 ? 0.602 ns/op >>>> >>>> >>>> -Chris. From andy.herrick at oracle.com Fri Jun 19 12:57:41 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Fri, 19 Jun 2020 08:57:41 -0400 Subject: RFR: 8225251: Minimise import statements in jpackage sources In-Reply-To: References: Message-ID: <56d594fa-ea8f-7f31-a413-500aca612a17@oracle.com> all looks good. /Andy On 6/19/2020 4:46 AM, alexander.matveev at oracle.com wrote: > Please review the jpackage fix for bug [1] at [2]. > > Cleanup import statements. > > [1] https://bugs.openjdk.java.net/browse/JDK-8225251 > [2] http://cr.openjdk.java.net/~almatvee/8225251/webrev.00/ > > Thanks, > Alexander From peter.levart at gmail.com Fri Jun 19 13:13:55 2020 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 19 Jun 2020 15:13:55 +0200 Subject: [PATCH] remove redundant initialization of volatile fields with default values In-Reply-To: <1084247143.1025326.1592553839671.JavaMail.zimbra@u-pem.fr> References: <281231592473099@mail.yandex.ru> <1084247143.1025326.1592553839671.JavaMail.zimbra@u-pem.fr> Message-ID: Hi Remi, On 6/19/20 10:03 AM, Remi Forax wrote: > Hi Sergei, > the problem is that you are changing the semantics if there are several fields. > > By example with the code below, you have the guarantee that the code will print 4 (if it prints something), > if you remove the assignment field = false, the code can print 0 or 4. > > class A { > int i = 4; > volatile boolean field = false; > } > > thread 1: > global = new A() > > thread 2: > var a = global; > if (a != null) { > System.out.println(a.i); > } I don't think this is guaranteed by the JMM. Unless you also read the 'field' in thread 2 and observe the effects of write to 'field' before reading 'i' : thread 2: ??? var a = global; ??? if (a != null && !a.field) { ??? ??? System.out.println(a.i); ??? } And even that might not work as you have to "observe" the effects of write to 'field' before reading 'i' and if the value of 'field' didn't change by the write, nothing guarantees that you have observed the effects of write to 'field' if you read false from it. So we could argue that any program that relies on similar unexistent guarantees is wrong, but works because the implementation usually exhibits stronger semantics than required. So removing such volatile write might break some programs in practice that are already broken in theory. Anyway, assigning default value to a field in constructor is always sign of bed smell and it is almost always possible to restructure code so that it is not needed. You have to be careful and see the "whole program" since other places might have to be modified too to preserve the semantics. Regards, Peter > regards, > R?mi > > ----- Mail original ----- >> De: "?????? ???????" >> ?: "core-libs-dev" >> Envoy?: Vendredi 19 Juin 2020 06:57:25 >> Objet: [PATCH] remove redundant initialization of volatile fields with default values >> Hello, >> >> while investigating an issue I've found out that assignment of default value to >> volatile fields slows down object instantiation. >> >> Consider the benchmark: >> >> @State(Scope.Thread) >> @OutputTimeUnit(TimeUnit.NANOSECONDS) >> @BenchmarkMode(value = Mode.AverageTime) >> @Fork(jvmArgsAppend = {"-Xms2g", "-Xmx2g"}) >> public class VolatileFieldBenchmark { >> @Benchmark >> public Object explicitInit() { >> return new ExplicitInit(); >> } >> >> @Benchmark >> public Object noInit() { >> return new NoInit(); >> } >> >> private static class ExplicitInit { >> private volatile boolean field = false; >> } >> private static class NoInit { >> private volatile boolean field; >> } >> } >> >> This gives the following results as of my machine: >> >> Benchmark Mode Cnt Score Error Units >> VolatileFieldBenchmark.explicitInit avgt 40 11.087 ? 0.140 ns/op >> VolatileFieldBenchmark.noInit avgt 40 3.367 ? 0.131 ns/op >> >> >> I've looked into source code of java.base and found out several cases where the >> default value is assigned to volatile field. >> >> Getting rid of such assignements demonstates improvement as of object >> instantiation, e.g. javax.security.auth.Subject: >> >> Mode Cnt Score Error Units >> before avgt 40 35.933 ? 2.647 ns/op >> after avgt 40 30.817 ? 2.384 ns/op >> >> As of testing tier1 and tier2 are both ok after the changes. >> >> Best regards, >> Sergey Tsypanov From david.holmes at oracle.com Fri Jun 19 13:25:45 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 19 Jun 2020 23:25:45 +1000 Subject: [PATCH] remove redundant initialization of volatile fields with default values In-Reply-To: <1084247143.1025326.1592553839671.JavaMail.zimbra@u-pem.fr> References: <281231592473099@mail.yandex.ru> <1084247143.1025326.1592553839671.JavaMail.zimbra@u-pem.fr> Message-ID: Hi Remi, On 19/06/2020 6:03 pm, Remi Forax wrote: > Hi Sergei, > the problem is that you are changing the semantics if there are several fields. > > By example with the code below, you have the guarantee that the code will print 4 (if it prints something), > if you remove the assignment field = false, the code can print 0 or 4. > > class A { > int i = 4; > volatile boolean field = false; > } > > thread 1: > global = new A() > > thread 2: > var a = global; > if (a != null) { > System.out.println(a.i); > } Surely you intended to read field to get the volatile read to pair with the volatile write of the initializer? Otherwise there is no happens-before ordering. David > regards, > R?mi > > ----- Mail original ----- >> De: "?????? ???????" >> ?: "core-libs-dev" >> Envoy?: Vendredi 19 Juin 2020 06:57:25 >> Objet: [PATCH] remove redundant initialization of volatile fields with default values > >> Hello, >> >> while investigating an issue I've found out that assignment of default value to >> volatile fields slows down object instantiation. >> >> Consider the benchmark: >> >> @State(Scope.Thread) >> @OutputTimeUnit(TimeUnit.NANOSECONDS) >> @BenchmarkMode(value = Mode.AverageTime) >> @Fork(jvmArgsAppend = {"-Xms2g", "-Xmx2g"}) >> public class VolatileFieldBenchmark { >> @Benchmark >> public Object explicitInit() { >> return new ExplicitInit(); >> } >> >> @Benchmark >> public Object noInit() { >> return new NoInit(); >> } >> >> private static class ExplicitInit { >> private volatile boolean field = false; >> } >> private static class NoInit { >> private volatile boolean field; >> } >> } >> >> This gives the following results as of my machine: >> >> Benchmark Mode Cnt Score Error Units >> VolatileFieldBenchmark.explicitInit avgt 40 11.087 ? 0.140 ns/op >> VolatileFieldBenchmark.noInit avgt 40 3.367 ? 0.131 ns/op >> >> >> I've looked into source code of java.base and found out several cases where the >> default value is assigned to volatile field. >> >> Getting rid of such assignements demonstates improvement as of object >> instantiation, e.g. javax.security.auth.Subject: >> >> Mode Cnt Score Error Units >> before avgt 40 35.933 ? 2.647 ns/op >> after avgt 40 30.817 ? 2.384 ns/op >> >> As of testing tier1 and tier2 are both ok after the changes. >> >> Best regards, >> Sergey Tsypanov From david.holmes at oracle.com Fri Jun 19 13:28:25 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 19 Jun 2020 23:28:25 +1000 Subject: [PATCH] remove redundant initialization of volatile fields with default values In-Reply-To: References: <281231592473099@mail.yandex.ru> <1084247143.1025326.1592553839671.JavaMail.zimbra@u-pem.fr> Message-ID: <1fbbd2a9-708d-a73f-d881-aa23fc8805f3@oracle.com> On 19/06/2020 11:25 pm, David Holmes wrote: > Hi Remi, > > On 19/06/2020 6:03 pm, Remi Forax wrote: >> Hi Sergei, >> the problem is that you are changing the semantics if there are >> several fields. >> >> By example with the code below, you have the guarantee that the code >> will print 4 (if it prints something), >> if you remove the assignment field = false, the code can print 0 or 4. >> >> ?? class A { >> ???? int i = 4; >> ???? volatile boolean field = false; >> ?? } >> >> thread 1: >> ?? global = new A() >> >> thread 2: >> ?? var a = global; >> ?? if (a != null) { >> ???? System.out.println(a.i); >> ?? } > > Surely you intended to read field to get the volatile read to pair with > the volatile write of the initializer? Otherwise there is no > happens-before ordering. And as Peter points out that may not suffice as you can't distinguish the default initialization of field with the actual volatile write in the initializer. David > David > >> regards, >> R?mi >> >> ----- Mail original ----- >>> De: "?????? ???????" >>> ?: "core-libs-dev" >>> Envoy?: Vendredi 19 Juin 2020 06:57:25 >>> Objet: [PATCH] remove redundant initialization of volatile fields >>> with default values >> >>> Hello, >>> >>> while investigating an issue I've found out that assignment of >>> default value to >>> volatile fields slows down object instantiation. >>> >>> Consider the benchmark: >>> >>> @State(Scope.Thread) >>> @OutputTimeUnit(TimeUnit.NANOSECONDS) >>> @BenchmarkMode(value = Mode.AverageTime) >>> @Fork(jvmArgsAppend = {"-Xms2g", "-Xmx2g"}) >>> public class VolatileFieldBenchmark { >>> ? @Benchmark >>> ? public Object explicitInit() { >>> ??? return new ExplicitInit(); >>> ? } >>> >>> ? @Benchmark >>> ? public Object noInit() { >>> ??? return new NoInit(); >>> ? } >>> >>> ? private static class ExplicitInit { >>> ??? private volatile boolean field = false; >>> ? } >>> ? private static class NoInit { >>> ??? private volatile boolean field; >>> ? } >>> } >>> >>> This gives the following results as of my machine: >>> >>> Benchmark??????????????????????????? Mode? Cnt?? Score?? Error? Units >>> VolatileFieldBenchmark.explicitInit? avgt?? 40? 11.087 ? 0.140? ns/op >>> VolatileFieldBenchmark.noInit??????? avgt?? 40?? 3.367 ? 0.131? ns/op >>> >>> >>> I've looked into source code of java.base and found out several cases >>> where the >>> default value is assigned to volatile field. >>> >>> Getting rid of such assignements demonstates improvement as of object >>> instantiation, e.g. javax.security.auth.Subject: >>> >>> ?????????? Mode? Cnt?? Score?? Error? Units >>> before???? avgt?? 40? 35.933 ? 2.647? ns/op >>> after????? avgt?? 40? 30.817 ? 2.384? ns/op >>> >>> As of testing tier1 and tier2 are both ok after the changes. >>> >>> Best regards, >>> Sergey Tsypanov From peter.levart at gmail.com Fri Jun 19 13:55:31 2020 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 19 Jun 2020 15:55:31 +0200 Subject: [PATCH] remove redundant initialization of volatile fields with default values In-Reply-To: <180351592557620@mail.yandex.ru> References: <281231592473099@mail.yandex.ru> <1084247143.1025326.1592553839671.JavaMail.zimbra@u-pem.fr> <180351592557620@mail.yandex.ru> Message-ID: Hi, Sergey, On 6/19/20 11:33 AM, ?????? ??????? wrote: > Hello Remi, > > thanks for pointing this out, I didn't take this into account. > > As I understand, the volatile semantics here covers all the fields of an object, > no matter whether they are declared before or after volatile field (probably > actual object layout might be different from declaration in source code). The layout of the fields doesn't matter. The order of assignment and the order of reading and volatileness/finallness of fields matter. > > If that's true I think we still can apply this optimization at least for > > 1) classes with single field and without super-classes I think that even removing assignment of default value to a field in a singular-field class (either volatile or not) in constructor can be observed. But I don't believe any program is relying on the observed behavior of such assignment - quite the contrary this might be a hidden bug. > 2) classes with all non-volatile fields declared final Final fields have to be explicitly assigned in the constructor even if they need to contain default value(s). But they have special JMM semantics and such assignment is not problematic. > 3) classes with not-initialized non-volatile fields This is similar to point 1). The additional non-explicitly assigned non-volatile fields don't matter here. > 4) classes with super-classes where non-volatile fields are not initialized (or inialized with default values) Where fields are declared (in super class or in sub class) does not matter. At runtime we are dealing with a single object. So if both non-volatile fields and volatile fields are assigned it depends on the order of assignments and on the order of reading those fields in the "whole program" whether removing volatile assignment of default value can have an effect on the program semantics. But as said, relying on the effects of the default value volatile write in constructor is fragile as it is not guaranteed by JMM. So such writes need to be studied and eventually removed. > > Looking at source code I see that we could keep java.security.KeyStore and KeyStore does not explicitly assign non-volatile non-final fields in constructor so the assignment of false to a volatile 'initialized' field in constructor can safely be removed. > its nested class PasswordProtection along with java.util.ListResourceBundle. PasswordProtection has just final fields and one volatile field 'destroyed' which is explicitly assigned to 'false' in constructor. This assignment can safely be removed. ListResourceBundle is more complicated and would need to be studied more carefully. Regards, Peter > > Please correct me if I miss anything in my speculation. > > Regards, > Sergey Tsypanov > > 19.06.2020, 10:04, "Remi Forax" : >> Hi Sergei, >> the problem is that you are changing the semantics if there are several fields. >> >> By example with the code below, you have the guarantee that the code will print 4 (if it prints something), >> if you remove the assignment field = false, the code can print 0 or 4. >> >> ??class A { >> ????int i = 4; >> ????volatile boolean field = false; >> ??} >> >> thread 1: >> ??global = new A() >> >> thread 2: >> ??var a = global; >> ??if (a != null) { >> ????System.out.println(a.i); >> ??} >> >> regards, >> R?mi >> >> ----- Mail original ----- >>> ?De: "?????? ???????" >>> ??: "core-libs-dev" >>> ?Envoy?: Vendredi 19 Juin 2020 06:57:25 >>> ?Objet: [PATCH] remove redundant initialization of volatile fields with default values >>> ?Hello, >>> >>> ?while investigating an issue I've found out that assignment of default value to >>> ?volatile fields slows down object instantiation. >>> >>> ?Consider the benchmark: >>> >>> ?@State(Scope.Thread) >>> ?@OutputTimeUnit(TimeUnit.NANOSECONDS) >>> ?@BenchmarkMode(value = Mode.AverageTime) >>> ?@Fork(jvmArgsAppend = {"-Xms2g", "-Xmx2g"}) >>> ?public class VolatileFieldBenchmark { >>> ??@Benchmark >>> ??public Object explicitInit() { >>> ????return new ExplicitInit(); >>> ??} >>> >>> ??@Benchmark >>> ??public Object noInit() { >>> ????return new NoInit(); >>> ??} >>> >>> ??private static class ExplicitInit { >>> ????private volatile boolean field = false; >>> ??} >>> ??private static class NoInit { >>> ????private volatile boolean field; >>> ??} >>> ?} >>> >>> ?This gives the following results as of my machine: >>> >>> ?Benchmark Mode Cnt Score Error Units >>> ?VolatileFieldBenchmark.explicitInit avgt 40 11.087 ? 0.140 ns/op >>> ?VolatileFieldBenchmark.noInit avgt 40 3.367 ? 0.131 ns/op >>> >>> ?I've looked into source code of java.base and found out several cases where the >>> ?default value is assigned to volatile field. >>> >>> ?Getting rid of such assignements demonstates improvement as of object >>> ?instantiation, e.g. javax.security.auth.Subject: >>> >>> ???????????Mode Cnt Score Error Units >>> ?before avgt 40 35.933 ? 2.647 ns/op >>> ?after avgt 40 30.817 ? 2.384 ns/op >>> >>> ?As of testing tier1 and tier2 are both ok after the changes. >>> >>> ?Best regards, >>> ?Sergey Tsypanov From Roger.Riggs at oracle.com Fri Jun 19 15:16:52 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Fri, 19 Jun 2020 11:16:52 -0400 Subject: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) In-Reply-To: <1FBDCE99-C63E-4C08-A508-08EC41688681@amperemail.onmicrosoft.com> References: <814DA850-76A9-4526-A3D9-141CD516C9AE@amperemail.onmicrosoft.com> <50955EDB-CCC2-4334-A4AF-2EB422E9CAA9@amperemail.onmicrosoft.com> <3B38456A-6BD0-4B95-9F9C-E73F7FE3A78F@amperemail.onmicrosoft.com> <1FBDCE99-C63E-4C08-A508-08EC41688681@amperemail.onmicrosoft.com> Message-ID: <0b774091-0b74-0277-86d4-db48f10f623e@oracle.com> Hi Peter, User-defined is a bit ambiguous but includes the appClassLoader since its contents are controlled by classpath and module path properties and command line arguments. Thanks, Roger On 6/18/20 2:24 PM, Peter Kessler (Open Source) wrote: > Roger, > > You are right that the existing code does not let me avoid the stack > walk. I withdraw the proposed patch. > > I think at least part of the problem is the comment on > VM.latestUserDefinedLoader(). That says > > /* > * Returns the first user-defined class loader up the execution stack, > * or the platform class loader if only code from the platform or > * bootstrap class loader is on the stack. > */ > > I thought that a "user-defined class loader" was a *class loader* > defined by the user. I now think VM.latestUserDefinedLoader returns > the class loader of the first user-defined *method* up the execution > stack, whether that class loader is user-defined or is, for example, > the application class loader. I do not consider the application class > loader to be user-defined, though obviously methods loaded by the > application class loader can be user-defined. > > ... peter > > ?-----Original Message----- > From: Roger Riggs > Organization: Java Products Group, Oracle > Date: Tuesday, June 16, 2020 at 7:14 AM > To: "Peter Kessler (Open Source)" , "core-libs-dev at openjdk.java.net" > Subject: Re: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) > > Hi Peter, > > I think you've hardwired an assumption about the contents of the stack > into VM.latestUserDefineLoader... > > On 6/12/20 7:19 PM, Peter Kessler (Open Source) wrote: > > Roger, > > > > I did think about confining the changes to ObjectInputStream. Maybe I > > did not think hard enough about it. > > > > Keeping a cache of the result of a first stack walk might work for the > > recursive calls. One might still do a useless stack walk for each > > top-level readObject call. No concurrent class loading can invalidate > > the cache for any particular stack walk. But it seems fraught to > > reuse a cached value in the face of concurrent class loading, or in > > the presence of overloaded ObjectInputStream methods that might create > > new ClassLoaders on this stack. I'm sure I don't want to debug that. > > > > My experiments have shown that the average stack walk is about 20 > > frames. I do not have the data on how many of those frames are > > recursive calls within ObjectInputStream methods and how many are in > > other code. > I'd be interested in a histogram of the depths and in which applications. > > Dragging in the StackWalker API would further complicate things, in my > > opinion. It might appear to keep the code out of platform native > > code, but we know that walking thread stacks involves native code. > > > > I settled on a system-wide one-bit cache: There have been no > > user-defined ClassLoaders constructed, so I can not find one on any > > particular stack walk. It may not be the best possible solution, but > > it addresses the issue of useless stack walks in the common case where > > there are no user-defined ClassLoaders. > > > > I agree that both ClassLoader and ObjectInputStream are complex. In > > fact, the proposed patch makes no changes to ObjectInputStream. There > > is one line added to each of the system-defined ClassLoaders to > > identify them as not user-defined ClassLoaders, one line in the base > > ClassLoader constructor, and the bulk of the change is around the > > guard in jdk.internal.misc.VM.latestUserDefinedLoader() to avoid the > > stack walk if possible. > > The spec and implementation of OIS.resolveClass do not make any assumptions > about whether serialization is invoked from a class on the > PlatformLoader, AppClassLoader, > or another loader. > > In the case of a 'system' thread or class calling OIS.readObject, the > change you are proposing > will force class loading in resolveClass to the AppClassLoader. > > In VM.latestUserDefinedLoader:390 caught my eye because it forces the > AppClassLoader. > It assumes that scanning the stack would find and return the > AppClassLoader, not so in all cases. > > Regards, Roger > > > > > > ... peter > > > > -----Original Message----- > > From: Roger Riggs > > Organization: Java Products Group, Oracle > > Date: Friday, June 12, 2020 at 11:50 AM > > To: "Peter Kessler (Open Source)" , "core-libs-dev at openjdk.java.net" > > Subject: Re: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) > > > > Hi Peter, > > > > The hazard to avoid is the cross package coupling that makes both > > ClassLoader and > > ObjectInputStream more complex; both are more than sufficiently complex > > already. > > > > Optimizing the implementations make sense if it benefits enough uses > > cases and > > does not make the code harder to maintain and evolve. > > > > Have you explored any alternatives that can be confined to OIS? > > For example, taking advantage the many calls to readObject within > > a single OIS that are nested. Possibly the first (depth = 0) find user > > class loader > > can be re-used or at least to short circuit the search on subsequent > > nested calls? > > Or caching the classloader at each depth and only need to probe for it > > if it has > > not already been recorded. The StackWalker API may be useful to limit > > the depth > > of searches. > > > > Thanks, Roger > > > > > > > > On 6/11/20 2:52 PM, Peter Kessler OS wrote: > > > Hi Roger, > > > > > > Thanks for your comments. > > > > > > I am not knowingly changing the required behavior. All I am proposing is > > > guarding the thread stack walk if it will be a waste of effort. If an > > > application or framework has its own ClassLoaders, then the guard will fail > > > and the thread stack will be walked. > > > > > > The stack walk is important, if there are user-defined ClassLoaders. A > > > method that uses ObjectInputStream to read objects can not know what > > > ClassLoader to use to resolve class references in the stream, without the > > > stack walk. But if there never have been any user-defined ClassLoaders > > > constructed, then the stack walk can never return a user-defined > > > ClassLoader. > > > > > > If the stream reader is part of some simple application, then it may spend > > > a significant amount of its time uselessly walking thread stacks. (If the > > > stream reader is part of some simple application its author may not know to > > > override ObjectInputStream.resolveClass, or how to determine what > > > ClassLoader should load a class from the stream.) If someone were using > > > such an application to choose platforms, the Java platform would be at a > > > disadvantage. > > > > > > Consider a somewhat more complex case, the SPECjbb2015 benchmark, which > > > claims to model a well-constructed business application on the Java > > > platform. SPECjbb2015 runs multiple JVMs, exchanging information via > > > ObjectInputStreams. Profiling shows that it spends more than 5% of its time > > > walking thread stacks. Yet, in spite of being a complex application, > > > written by experts in the Java platform, there are no user-defined > > > ClassLoaders. So all the time spent walking stacks is wasted time, and is > > > one of the reasons the servers fail their service-level agreements. We can > > > discuss whether SPECjbb2015 is representative of Java applications. But it > > > is easily available and relatively simple to run. People use benchmarks > > > like SPECjbb2015 to decide whether to use the Java platform for particular > > > tasks, and it would be unfortunate if we did not make the Java platform as > > > good as it could be. > > > > > > ... peter > > > > > > -----Original Message----- > > > From: core-libs-dev on behalf of Roger Riggs > > > Organization: Java Products Group, Oracle > > > Date: Tuesday, June 9, 2020 at 6:43 AM > > > To: "core-libs-dev at openjdk.java.net" > > > Subject: Re: [PATCH] 8246633: Improve the performance of ObjectInputStream.resolveClass(ObjectStreamClass) > > > > > > Hi Peter, > > > > > > I'd like to understand the scope and impact of the problem before > > > jumping to a solution. > > > For specific applications, overriding ObjectInputStream.resolveClass to > > > handle the class > > > lookup can handle the case as expeditiously as is necessary. > > > > > > What application uses cases are impacted and how prevelant are they? > > > Many, many frameworks and applications create classloaders for their own > > > purposes and > > > take advantage of the required behavior. > > > > > > Thanks, Roger > > > > > > > > > On 6/4/20 7:08 PM, Peter Kessler OS wrote: > > > > ObjectInputStream.resolveClass(ObjectStreamClass) is specified to find "the first class loader on the current thread's stack (starting from the currently executing method) that is neither the platform class loader nor its ancestor." Finding that class loader is done with a virtual frame walk in JVM_LatestUserDefinedLoader(JNIEnv *env). The virtual frame walk is expensive. If an application does not define any ClassLoaders, then the virtual frame walk will always find the system class loader. > > > > > > > > If an application does a significant amount of reading from an ObjectInputStream, the cost of ObjectInputStream.resolveClass(ObjectInputStreamClass) can be considerable. In a sample application, that method (and the methods it calls) have been measured to consume 5+% of the CPU cycles. > > > > > > > > The proposal is to note the construction of any user-defined ClassLoader, and if none has been constructed, avoid the virtual frame walk returning ClassLoader.getSystemClassLoader(). But if any user-defined ClassLoader has been constructed, use the virtual frame walk to find the first user-defined ClassLoader on the thread stack. > > > > > > > > User-defined ClassLoaders could be distinguished from non-user-defined ClassLoaders in several ways. The ClassLoader class hierarchy could be changed to make non-user-defined ClassLoaders inherit from a marker class. There is already a jdk.internal.loader.BuiltinClassLoader but it is used for module (and resource) loading. BuiltinClassLoader is not a super-class of jdk.internal.reflect.DelegatingClassLoader, which is used to load reflection classes, and reflection methods are also ignored by the virtual frame walk. ClassLoaders could have a boolean that distinguished whether they are user-defined or not. That would require a change to ClassLoader API. There are other ways to distinguish user-defined ClassLoaders. The proposal is to keep the decoration of the ClassLoaders out of the instances, so that it is only visible to the code that needs it. > > > > > > > > The proposal is that each non-user-defined ClassLoader have a static block that registers the class as a non-user-defined ClassLoader. Then in the base ClassLoader constructor check if the ClassLoader being constructed is a user-defined ClassLoader and if so set a boolean that acts as the guard on the virtual frame walk. If additional non-user-defined ClassLoaders are declared in the future beyond the ones identified in this patch, they can be added to the registry. > > > > > > > > There are currently 4 non-user-defined ClassLoader classes, so the registry is not difficult to maintain. Nor is the registry difficult to search when ClassLoaders are constructed. The guard boolean that records whether a user-defined ClassLoader has been constructed transitions from false to true and never becomes false again. (An extension to transition the boolean back to false when a user-defined ClassLoader becomes unreachable is beyond the scope of this proposal.) > > > > > > > > This proposal slows loading of non-user-defined ClassLoaders by the time it takes to register them. It slows ClassLoader construction by the time it takes to consult the registry and conditionally set the guard boolean. At each invocation of ObjectInputStream.resolveClass(ObjectInputStreamClass), the guard boolean is used to possibly avoid the virtual frame walk. > > > > > > > > Tested with `make run-test-tier1` on Linux (CentOS 7) on both aarch64 and x86_64. I had one failure on each of the runs > > > > > > > > ACTION: main -- Failed. Execution failed: `main' threw exception: java.lang.Error: TESTBUG: the library has not been found in ${Repo}/build/linux-aarch64-server-release/images/test/hotspot/jtreg/native > > > > REASON: User specified action: run main/native GTestWrapper > > > > > > > > on both plaforms. But I get the same failure when I run the tier1 tests on clones of OpenJDK-15+25 on both aarch64 and x86_64. > > > > > > > > Running a sample (representative?) application with the Oracle Developer Studio analyzer shows a performance comparison of > > > > > > > > Stack Fragment sorted by metric: Attributed Total CPU Time > > > > baseline.er experiment.er > > > > Attributed Attributed Name > > > > Total CPU Time Total CPU Time > > > > sec. sec. > > > > ================================== Callers > > > > 3459.210 29.931 java.io.ObjectInputStream.readOrdinaryObject(boolean) > > > > 1139.727 3.532 java.io.ObjectInputStream.readArray(boolean) > > > > 875.262 9.116 java.io.ObjectInputStream.readNonProxyDesc(boolean) > > > > > > > > ================================== Stack Fragment > > > > java.io.ObjectInputStream.readClassDesc(boolean) > > > > java.io.ObjectInputStream.readNonProxyDesc(boolean) > > > > java.io.ObjectInputStream.resolveClass(java.io.ObjectStreamClass) > > > > java.io.ObjectInputStream.latestUserDefinedLoader() > > > > 4.173 3.953 jdk.internal.misc.VM.latestUserDefinedLoader() > > > > > > > > ================================== Callees > > > > 5470.026 0. jdk.internal.misc.VM.latestUserDefinedLoader0() > > > > 0. 38.627 java.lang.ClassLoader.getSystemClassLoader() > > > > > > > > The `hg export -g` below is with respect to OpenJDK-15+25. > > > > > > > > Thank you for your review comments. I will need a sponsor to get this change into the repository. > > > > > > > > ... peter > > > > > > > > # HG changeset patch > > > > # User Peter Kessler > > > > # Date 1591310467 25200 > > > > # Thu Jun 04 15:41:07 2020 -0700 > > > > # Node ID 9a39488182c1dfc5bc7bb41d562d7ef16ee657f6 > > > > # Parent 90b266a84c06f1b3dc0ed8767856793e8c1c357e > > > > Improve the performance of ObjectInputStream.resolveClass > > > > > > > > diff --git a/src/java.base/share/classes/java/lang/ClassLoader.java b/src/java.base/share/classes/java/lang/ClassLoader.java > > > > --- a/src/java.base/share/classes/java/lang/ClassLoader.java > > > > +++ b/src/java.base/share/classes/java/lang/ClassLoader.java > > > > @@ -385,6 +385,8 @@ > > > > } > > > > this.package2certs = new ConcurrentHashMap<>(); > > > > this.nameAndId = nameAndId(this); > > > > + /* Note the construction of a ClassLoader. */ > > > > + VM.noteClassLoaderConstruction(this.getClass()); > > > > } > > > > > > > > /** > > > > diff --git a/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java b/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java > > > > --- a/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java > > > > +++ b/src/java.base/share/classes/jdk/internal/loader/ClassLoaders.java > > > > @@ -117,6 +117,11 @@ > > > > protected Class loadClassOrNull(String cn, boolean resolve) { > > > > return JLA.findBootstrapClassOrNull(this, cn); > > > > } > > > > + > > > > + static { > > > > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > > > > + VM.registerNotUserDefinedClassLoader(BootClassLoader.class); > > > > + } > > > > }; > > > > > > > > /** > > > > @@ -127,6 +132,8 @@ > > > > static { > > > > if (!ClassLoader.registerAsParallelCapable()) > > > > throw new InternalError(); > > > > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > > > > + VM.registerNotUserDefinedClassLoader(PlatformClassLoader.class); > > > > } > > > > > > > > PlatformClassLoader(BootClassLoader parent) { > > > > @@ -142,6 +149,8 @@ > > > > static { > > > > if (!ClassLoader.registerAsParallelCapable()) > > > > throw new InternalError(); > > > > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > > > > + VM.registerNotUserDefinedClassLoader(AppClassLoader.class); > > > > } > > > > > > > > final URLClassPath ucp; > > > > diff --git a/src/java.base/share/classes/jdk/internal/misc/VM.java b/src/java.base/share/classes/jdk/internal/misc/VM.java > > > > --- a/src/java.base/share/classes/jdk/internal/misc/VM.java > > > > +++ b/src/java.base/share/classes/jdk/internal/misc/VM.java > > > > @@ -304,12 +304,44 @@ > > > > private static final int JVMTI_THREAD_STATE_WAITING_INDEFINITELY = 0x0010; > > > > private static final int JVMTI_THREAD_STATE_WAITING_WITH_TIMEOUT = 0x0020; > > > > > > > > + /** A registry of not user-defined ClassLoaders. */ > > > > + private static final List> notUserDefinedClassLoaderRegistry = > > > > + Collections.synchronizedList(new ArrayList<>()); > > > > + > > > > + /** Register a ClassLoader class as being not a user-defined ClassLoader. */ > > > > + public static void registerNotUserDefinedClassLoader(Class classLoaderClass) { > > > > + notUserDefinedClassLoaderRegistry.add(classLoaderClass); > > > > + } > > > > + > > > > + /** A flag for whether a user-defined ClassLoaders has been constructed. */ > > > > + private static volatile boolean constructedUserDefinedClassLoaderFlag = false; > > > > + > > > > + /** > > > > + * Note a ClassLoader construction, and record if a > > > > + * user-defined ClassLoader has been constructed. > > > > + */ > > > > + public static void noteClassLoaderConstruction(Class classLoaderClass) { > > > > + /* Check if the ClassLoader class not a user-defined ClassLoader. */ > > > > + if (notUserDefinedClassLoaderRegistry.contains(classLoaderClass)) { > > > > + return; > > > > + } > > > > + constructedUserDefinedClassLoaderFlag = true; > > > > + return; > > > > + } > > > > + > > > > /* > > > > * Returns the first user-defined class loader up the execution stack, > > > > * or the platform class loader if only code from the platform or > > > > * bootstrap class loader is on the stack. > > > > */ > > > > public static ClassLoader latestUserDefinedLoader() { > > > > + if (!constructedUserDefinedClassLoaderFlag) { > > > > + /* > > > > + * If no user-defined ClassLoader has been constructed, > > > > + * then I will not find a user-defined ClassLoader in the stack. > > > > + */ > > > > + return ClassLoader.getSystemClassLoader(); > > > > + } > > > > ClassLoader loader = latestUserDefinedLoader0(); > > > > return loader != null ? loader : ClassLoader.getPlatformClassLoader(); > > > > } > > > > diff --git a/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java b/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java > > > > --- a/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java > > > > +++ b/src/java.base/share/classes/jdk/internal/reflect/ClassDefiner.java > > > > @@ -30,6 +30,7 @@ > > > > > > > > import jdk.internal.access.JavaLangAccess; > > > > import jdk.internal.access.SharedSecrets; > > > > +import jdk.internal.misc.VM; > > > > > > > > /** Utility class which assists in calling defineClass() by > > > > creating a new class loader which delegates to the one needed in > > > > @@ -73,4 +74,9 @@ > > > > DelegatingClassLoader(ClassLoader parent) { > > > > super(parent); > > > > } > > > > + > > > > + static { > > > > + /* Register this ClassLoader as not a user-defined ClassLoader. */ > > > > + VM.registerNotUserDefinedClassLoader(DelegatingClassLoader.class); > > > > + } > > > > } > > > > > > > > > > > > > > > > From alexey.semenyuk at oracle.com Fri Jun 19 16:08:53 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Fri, 19 Jun 2020 12:08:53 -0400 Subject: RFR: 8225251: Minimise import statements in jpackage sources In-Reply-To: References: Message-ID: <35bacbc9-d535-6b33-3c46-cfebb5e24d32@oracle.com> Alexander, There is still --- import static org.junit.Assert.*; --- in unit tests. Is this intended? - Alexey On 6/19/2020 4:46 AM, alexander.matveev at oracle.com wrote: > Please review the jpackage fix for bug [1] at [2]. > > Cleanup import statements. > > [1] https://bugs.openjdk.java.net/browse/JDK-8225251 > [2] http://cr.openjdk.java.net/~almatvee/8225251/webrev.00/ > > Thanks, > Alexander From kevin.rushforth at oracle.com Fri Jun 19 16:30:44 2020 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Fri, 19 Jun 2020 09:30:44 -0700 Subject: RFR: 8225251: Minimise import statements in jpackage sources In-Reply-To: <35bacbc9-d535-6b33-3c46-cfebb5e24d32@oracle.com> References: <35bacbc9-d535-6b33-3c46-cfebb5e24d32@oracle.com> Message-ID: <4d1a457c-ffb6-5921-cb77-ba3b18ac4595@oracle.com> That a reasonably common pattern in JUnit tests, but expanding them to individual static imports is, of course, fine as well. -- Kevin On 6/19/2020 9:08 AM, Alexey Semenyuk wrote: > Alexander, > > There is still > --- > import static org.junit.Assert.*; > --- > in unit tests. Is this intended? > > - Alexey > > On 6/19/2020 4:46 AM, alexander.matveev at oracle.com wrote: >> Please review the jpackage fix for bug [1] at [2]. >> >> Cleanup import statements. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8225251 >> [2] http://cr.openjdk.java.net/~almatvee/8225251/webrev.00/ >> >> Thanks, >> Alexander > From alexey.semenyuk at oracle.com Fri Jun 19 17:49:48 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Fri, 19 Jun 2020 13:49:48 -0400 Subject: RFR: 8225251: Minimise import statements in jpackage sources In-Reply-To: <4d1a457c-ffb6-5921-cb77-ba3b18ac4595@oracle.com> References: <35bacbc9-d535-6b33-3c46-cfebb5e24d32@oracle.com> <4d1a457c-ffb6-5921-cb77-ba3b18ac4595@oracle.com> Message-ID: <802ce899-eb1b-8bc0-dddf-956788065494@oracle.com> This is how they were ut in unit tests initially. I'm OK either way, just wanted to double check these imports were not overlooked. - Alexey On 6/19/2020 12:30 PM, Kevin Rushforth wrote: > That a reasonably common pattern in JUnit tests, but expanding them to > individual static imports is, of course, fine as well. > > -- Kevin > > > On 6/19/2020 9:08 AM, Alexey Semenyuk wrote: >> Alexander, >> >> There is still >> --- >> import static org.junit.Assert.*; >> --- >> in unit tests. Is this intended? >> >> - Alexey >> >> On 6/19/2020 4:46 AM, alexander.matveev at oracle.com wrote: >>> Please review the jpackage fix for bug [1] at [2]. >>> >>> Cleanup import statements. >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8225251 >>> [2] http://cr.openjdk.java.net/~almatvee/8225251/webrev.00/ >>> >>> Thanks, >>> Alexander >> > From maurizio.cimadamore at oracle.com Fri Jun 19 19:07:44 2020 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Fri, 19 Jun 2020 20:07:44 +0100 Subject: RFR [15] 8247696: Incorrect tail computation for large segments in AbstractMemorySegmentImpl::mismatch In-Reply-To: References: <0B15B6AF-F196-4192-B448-F1955AE586A7@oracle.com> <94fceeda-edee-c571-b32d-af8d5aed672e@oracle.com> <7BA53832-4D14-405D-9AE0-118ABB1344A2@oracle.com> Message-ID: Looks good! Thanks Maurizio On 19/06/2020 11:56, Chris Hegarty wrote: > Paul, Maurizio, > > This version incorporates all feedback so far. > > https://cr.openjdk.java.net/~chegar/8247696/webrev.01/ > Results on my machine: > > Benchmark ?Mode??Cnt?? ? ??Score? ? ? ??Error??Units > BulkOps.mismatch_large_bytebuffer??avgt???30???88266.728 > ????4083.476??ns/op > BulkOps.mismatch_large_segment?? ??avgt???30???86141.343 > ????2450.450??ns/op > BulkOps.mismatch_small_bytebuffer??avgt???30?? ? ??6.360 ? ??0.425??ns/op > BulkOps.mismatch_small_segment?? ??avgt???30?? ? ??4.582 ? ??1.040??ns/op > > -Chris. > >> On 19 Jun 2020, at 00:35, Paul Sandoz > > wrote: >> >> Thanks Chris. >> >>> On Jun 18, 2020, at 2:57 AM, Maurizio Cimadamore >>> >> > wrote: >>> >>> Thanks for looking at this Chris >>> >>> On 17/06/2020 21:56, Paul Sandoz wrote: >>>> Hi Chris, >>>> >>>> AbstractMemorySegmentImpl >>>> ? >>>> >>>> In vectorizedMismatchLarge: >>>> >>>> 163 ? ? ? ? ? ? if (remaining > 7) >>>> 164 ? ? ? ? ? ? ? ? throw new InternalError("remaining greater than >>>> 7: " + remaining); >>>> 165 ? ? ? ? ? ? i = length - remaining; >>>> 166 ? ? ? ? } >>>> >>>> Should this check be an assert? >>> I suggested that to Chris, since sometimes asserts are enabled when >>> running tests, sometimes are not - in langtools we moved away?from >>> using asserts as we realized that in certain cases they were >>> silently failing. I'm ok with whatever standard you feel comfortable >>> with?though. >> >> Automated running of tests enable assertions (-ea -esa), perhaps the >> use is more commonly accepted in library code than compiler code.?I >> would favor so in this case if necessary (sometimes they are avoided >> to reduce inline thresholds). >> >> >>>> >>>> ? >>>> >>>> This fix prompted me to think more deeply about the implementation >>>> of vectorizedMismatchLarge and its use of >>>> vectorizedMismatch.?Sorry, I should have thought about this more >>>> throughly earlier on. >>>> >>>> We need to refine the approach, not for this patch, but something >>>> to follow up after. I think there are two issues. >>>> >>>> 1) The intrinsic method vectorizedMismatch could potentially bomb >>>> out at any point and return the bitwise compliment of the >>>> remaining?number of elements to check. >>>> >>>> Obviously, there is no point doing that arbitrarily but a stub >>>> implementation for, say, x86 AVX-512 might decide bomb out for ?< >>>> 64?remaining elements, rather than apply vector operations on >>>> smaller vector sizes or use a scalar loop. It does not today, but I >>>> think we?should guard against that potentially happening, otherwise >>>> bad things can happen. >>> >>> So your worry here is that we'll end up with an infinite loop, right? >>> >> >> Or more likely that an incorrect result is returned since tail >> elements will be skipped over as the offset and size is updated. >> >> >>> If so, we could check remaining against previous remaining and bomb >>> out too if no further progress seem to be made? >>> >> >> I think it's better to always terminate the loop after the last >> sub-range is checked, rather than unnecessarily calling twice. >> >> I am not confident the vectorizedMismatch intrinsic stub has been >> tested properly on very small lengths, since it's never directly >> called in?such cases, so also keeping "remaining > 7" is good too. >> >> Paul. >> >>>> >>>> I think the loop should exit when the last sub-range has been >>>> checked. We should rely on other tests to ensure the intrinsic >>>> method is?operating efficiently. >>>> >>>> >>>> 2) This method only works when operating on byte arrays. It will >>>> not work correctly if operating on short or long arrays, since we >>>> are not?adjusting the length and offsets accordingly by the scale. >>>> It's probably easier to just rename this as >>>> vectorizedMismatchLargeForBytes?and drop the log2ArrayIndexScale >>>> argument. Then expand later if need be. I still think the method >>>> rightly belongs in ArraysSupport. >>> >>> Yep - probably good idea to restrict on bytes, for now. >>> >>> Maurizio >>> >>>> >>>> Paul. >>>> >>>>> On Jun 17, 2020, at 8:33 AM, Chris Hegarty >>>>> > wrote: >>>>> >>>>> The MemorySegment::mismatch implementation added vectorized >>>>> mismatch of long sizes. The implementation is trivial, but >>>>> the?starting point for a more optimized implementation, if needed. >>>>> ArraysSupport::vectorizedMismatchLarge incorrectly returns the >>>>> bitwise?complement of the offset of the first mismatch, where it >>>>> should return the bitwise complement of the number of remaining >>>>> pairs of?elements to be checked in the tail of the two arrays. The >>>>> AbstractMemorySegmentImpl::mismatch masked this problem, since >>>>> it?seamlessly compared the remaining tail, which is larger than it >>>>> should be. >>>>> >>>>> Webrev: >>>>> https://cr.openjdk.java.net/~chegar/8247696/webrev.00/ >>>>> >>>>> I updated the exiting BulkOps micro-benchmark to cover mismatch. >>>>> Here are the results, compared to ByteBuffer::mismatch, on my?machine: >>>>> >>>>> Benchmark ? ? ? ? ? ? ? ? ? ? ? ? ?Mode ?Cnt ?Score ? ? ? ?Error >>>>> ?Units >>>>> BulkOps.mismatch_large_bytebuffer ?avgt ? 30 740186.973 ? >>>>> 119314.207 ?ns/op >>>>> BulkOps.mismatch_large_segment ? ? avgt ? 30 683475.305 ? >>>>> ?76355.043 ?ns/op >>>>> BulkOps.mismatch_small_bytebuffer ?avgt ? 30 ?7.367 ? ? ? ?0.523 >>>>> ?ns/op >>>>> BulkOps.mismatch_small_segment ? ? avgt ? 30 ?4.140 ? ? ? ?0.602 >>>>> ?ns/op >>>>> >>>>> >>>>> -Chris. > From alexander.matveev at oracle.com Fri Jun 19 22:02:27 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Fri, 19 Jun 2020 15:02:27 -0700 Subject: RFR: 8225251: Minimise import statements in jpackage sources In-Reply-To: <802ce899-eb1b-8bc0-dddf-956788065494@oracle.com> References: <35bacbc9-d535-6b33-3c46-cfebb5e24d32@oracle.com> <4d1a457c-ffb6-5921-cb77-ba3b18ac4595@oracle.com> <802ce899-eb1b-8bc0-dddf-956788065494@oracle.com> Message-ID: Hi Alexey, Andy mentioned in issue description not to expend static imports, this is why all static imports were not expended. I can expend them or we can keep as is. I am fine with either way. Andy any comments why we do not need to expend static imports? Thanks, Alexander On 6/19/20 10:49 AM, Alexey Semenyuk wrote: > This is how they were ut in unit tests initially. I'm OK either way, > just wanted to double check these imports were not overlooked. > > - Alexey > > On 6/19/2020 12:30 PM, Kevin Rushforth wrote: >> That a reasonably common pattern in JUnit tests, but expanding them >> to individual static imports is, of course, fine as well. >> >> -- Kevin >> >> >> On 6/19/2020 9:08 AM, Alexey Semenyuk wrote: >>> Alexander, >>> >>> There is still >>> --- >>> import static org.junit.Assert.*; >>> --- >>> in unit tests. Is this intended? >>> >>> - Alexey >>> >>> On 6/19/2020 4:46 AM, alexander.matveev at oracle.com wrote: >>>> Please review the jpackage fix for bug [1] at [2]. >>>> >>>> Cleanup import statements. >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8225251 >>>> [2] http://cr.openjdk.java.net/~almatvee/8225251/webrev.00/ >>>> >>>> Thanks, >>>> Alexander >>> >> > From andy.herrick at oracle.com Fri Jun 19 22:36:27 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Fri, 19 Jun 2020 18:36:27 -0400 Subject: RFR: 8225251: Minimise import statements in jpackage sources In-Reply-To: References: <35bacbc9-d535-6b33-3c46-cfebb5e24d32@oracle.com> <4d1a457c-ffb6-5921-cb77-ba3b18ac4595@oracle.com> <802ce899-eb1b-8bc0-dddf-956788065494@oracle.com> Message-ID: <49fac126-0ad6-bbbb-aead-7dd12bd006b1@oracle.com> I'm fine with it either way - the main thing is what you have done. /Andy On 6/19/2020 6:02 PM, alexander.matveev at oracle.com wrote: > Hi Alexey, > > Andy mentioned in issue description not to expend static imports, this > is why all static imports were not expended. I can expend them or we > can keep as is. I am fine with either way. Andy any comments why we do > not need to expend static imports? > > Thanks, > Alexander > > On 6/19/20 10:49 AM, Alexey Semenyuk wrote: >> This is how they were ut in unit tests initially. I'm OK either way, >> just wanted to double check these imports were not overlooked. >> >> - Alexey >> >> On 6/19/2020 12:30 PM, Kevin Rushforth wrote: >>> That a reasonably common pattern in JUnit tests, but expanding them >>> to individual static imports is, of course, fine as well. >>> >>> -- Kevin >>> >>> >>> On 6/19/2020 9:08 AM, Alexey Semenyuk wrote: >>>> Alexander, >>>> >>>> There is still >>>> --- >>>> import static org.junit.Assert.*; >>>> --- >>>> in unit tests. Is this intended? >>>> >>>> - Alexey >>>> >>>> On 6/19/2020 4:46 AM, alexander.matveev at oracle.com wrote: >>>>> Please review the jpackage fix for bug [1] at [2]. >>>>> >>>>> Cleanup import statements. >>>>> >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8225251 >>>>> [2] http://cr.openjdk.java.net/~almatvee/8225251/webrev.00/ >>>>> >>>>> Thanks, >>>>> Alexander >>>> >>> >> > From johnlinp at gmail.com Sat Jun 20 01:00:37 2020 From: johnlinp at gmail.com (=?UTF-8?B?5p6X6Ieq5Z2H?=) Date: Sat, 20 Jun 2020 09:00:37 +0800 Subject: [PATCH] 8247402: Rewrite the implementation requirements for Map::compute() In-Reply-To: References: Message-ID: Hi Martin, Any update on this? Thanks! Best, John Lin ??? ? 2020?6?13? ?? ??11:08??? > > Hi Martin, > > I see your point. Thank you for demostrating this for me. > > Here's my updated patch: > > # HG changeset patch > # User John Lin > # Date 1591923561 -28800 > # Fri Jun 12 08:59:21 2020 +0800 > # Node ID e01d9d020506a88d3d585bd3264594a26450c659 > # Parent 49a68abdb0ba68351db0f140ddac793b1c391bd5 > 8247402: Rewrite the implementation requirements for Map::compute() > > diff --git a/src/java.base/share/classes/java/util/Map.java > b/src/java.base/share/classes/java/util/Map.java > --- a/src/java.base/share/classes/java/util/Map.java > +++ b/src/java.base/share/classes/java/util/Map.java > @@ -1113,17 +1113,12 @@ public interface Map { > *
       {@code
      >       * V oldValue = map.get(key);
      >       * V newValue = remappingFunction.apply(key, oldValue);
      > -     * if (oldValue != null) {
      > -     *    if (newValue != null)
      > -     *       map.put(key, newValue);
      > -     *    else
      > -     *       map.remove(key);
      > +     * if (newValue != null) {
      > +     *     map.put(key, newValue);
      >       * } else {
      > -     *    if (newValue != null)
      > -     *       map.put(key, newValue);
      > -     *    else
      > -     *       return null;
      > +     *     map.remove(key);
      >       * }
      > +     * return newValue;
      >       * }
      > * > *

      The default implementation makes no guarantees about detecting if the > > By the way, I saw everyone in this mailing list is using webrev. > However, the tutorial https://openjdk.java.net/guide/codeReview.html > says that only the users with push access to the OpenJDK Mercurial > server can use webrev. Can I apply for push access to the OpenJDK > Mercurial server too? > > Best, > John Lin > > Martin Buchholz ? 2020?6?13? ?? ??12:36??? > > > > If I rub that program, I get: > > > > HashMap false > > HashMap1 false > > HashMap2 true > > > > which suggests that HashMap2's implementation is wrong. From alexander.matveev at oracle.com Sat Jun 20 02:05:40 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Fri, 19 Jun 2020 19:05:40 -0700 Subject: RFR: 8225251: Minimise import statements in jpackage sources In-Reply-To: <49fac126-0ad6-bbbb-aead-7dd12bd006b1@oracle.com> References: <35bacbc9-d535-6b33-3c46-cfebb5e24d32@oracle.com> <4d1a457c-ffb6-5921-cb77-ba3b18ac4595@oracle.com> <802ce899-eb1b-8bc0-dddf-956788065494@oracle.com> <49fac126-0ad6-bbbb-aead-7dd12bd006b1@oracle.com> Message-ID: http://cr.openjdk.java.net/~almatvee/8225251/webrev.01/ - Expended "import static org.junit.Assert.*" and "import static jdk.incubator.jpackage.internal.StandardBundlerParam.*". - Fixed inconsistency between imported StandardBundlerParam or when used directly. See http://cr.openjdk.java.net/~almatvee/8225251/webrev.01/src/jdk.incubator.jpackage/macosx/classes/jdk/incubator/jpackage/internal/MacAppImageBuilder.java.frames.html for example. Thanks, Alexander On 6/19/20 3:36 PM, Andy Herrick wrote: > I'm fine with it either way - the main thing is what you have done. > > /Andy > > On 6/19/2020 6:02 PM, alexander.matveev at oracle.com wrote: >> Hi Alexey, >> >> Andy mentioned in issue description not to expend static imports, >> this is why all static imports were not expended. I can expend them >> or we can keep as is. I am fine with either way. Andy any comments >> why we do not need to expend static imports? >> >> Thanks, >> Alexander >> >> On 6/19/20 10:49 AM, Alexey Semenyuk wrote: >>> This is how they were ut in unit tests initially. I'm OK either way, >>> just wanted to double check these imports were not overlooked. >>> >>> - Alexey >>> >>> On 6/19/2020 12:30 PM, Kevin Rushforth wrote: >>>> That a reasonably common pattern in JUnit tests, but expanding them >>>> to individual static imports is, of course, fine as well. >>>> >>>> -- Kevin >>>> >>>> >>>> On 6/19/2020 9:08 AM, Alexey Semenyuk wrote: >>>>> Alexander, >>>>> >>>>> There is still >>>>> --- >>>>> import static org.junit.Assert.*; >>>>> --- >>>>> in unit tests. Is this intended? >>>>> >>>>> - Alexey >>>>> >>>>> On 6/19/2020 4:46 AM, alexander.matveev at oracle.com wrote: >>>>>> Please review the jpackage fix for bug [1] at [2]. >>>>>> >>>>>> Cleanup import statements. >>>>>> >>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8225251 >>>>>> [2] http://cr.openjdk.java.net/~almatvee/8225251/webrev.00/ >>>>>> >>>>>> Thanks, >>>>>> Alexander >>>>> >>>> >>> >> From jonathan.gibbons at oracle.com Sat Jun 20 03:35:29 2020 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 19 Jun 2020 20:35:29 -0700 Subject: RFR: [15,docs] JDK-8247959,doclint errors in NIO code Message-ID: Please review some fixes to address issues found by doclint. In a couple of places, imports were required in order to resolve symbols. In another case, the name also had a typo in it. In the last case, the incorrect syntax was used for a type parameter in @param. -- Jon JBS: https://bugs.openjdk.java.net/browse/JDK-8247959 Patch inline: diff -r 086c7f077fc6 src/jdk.nio.mapmode/share/classes/jdk/nio/mapmode/ExtendedMapMode.java --- a/src/jdk.nio.mapmode/share/classes/jdk/nio/mapmode/ExtendedMapMode.java Fri Jun 19 15:22:19 2020 -0400 +++ b/src/jdk.nio.mapmode/share/classes/jdk/nio/mapmode/ExtendedMapMode.java Fri Jun 19 20:24:02 2020 -0700 @@ -25,6 +25,8 @@ ?package jdk.nio.mapmode; +import java.nio.MappedByteBuffer; +import java.nio.channels.FileChannel; ?import java.nio.channels.FileChannel.MapMode; ?/** @@ -52,7 +54,7 @@ ?????/** ??????* File mapping mode for a read-write mapping of a file backed by - * non-volatile RAM. {@linkplain MappedByteBufefr#force force} + * non-volatile RAM. {@linkplain MappedByteBuffer#force force} ??????* operations on a buffer created with this mode will be performed ??????* using cache line writeback rather than proceeding via a file ??????* device flush. diff -r 086c7f077fc6 src/jdk.sctp/share/classes/com/sun/nio/sctp/NotificationHandler.java --- a/src/jdk.sctp/share/classes/com/sun/nio/sctp/NotificationHandler.java Fri Jun 19 15:22:19 2020 -0400 +++ b/src/jdk.sctp/share/classes/com/sun/nio/sctp/NotificationHandler.java Fri Jun 19 20:24:02 2020 -0700 @@ -45,7 +45,7 @@ ??* this handler interface as the type for parameters, return type, etc. rather ??* than the abstract class. ??* - * @param T The type of the object attached to the receive operation + * @param The type of the object attached to the receive operation ??* ??* @since 1.7 ??*/ From Alan.Bateman at oracle.com Sat Jun 20 06:33:09 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 20 Jun 2020 07:33:09 +0100 Subject: RFR: [15,docs] JDK-8247959,doclint errors in NIO code In-Reply-To: References: Message-ID: <19e12642-dfd3-cccc-5b35-08e3935e49b9@oracle.com> On 20/06/2020 04:35, Jonathan Gibbons wrote: > Please review some fixes to address issues found by doclint. Looks good. From liyu.italy at gmail.com Sat Jun 20 15:06:32 2020 From: liyu.italy at gmail.com (Lisa Li) Date: Sat, 20 Jun 2020 23:06:32 +0800 Subject: [PATCH] 8245694 : java.util.Properties.entrySet() does not override java.lang.Object methods In-Reply-To: References: <9487a1c9-d0bc-3632-156d-5927e5801dc8@icemanx.nl> <20109712-d505-1b17-3e6b-ae388d554400@oracle.com> <767a4dae-55f0-ca11-dbdf-57c7a22205d3@oracle.com> <7b01289c-d7c9-47ab-f9f6-e2cf3a00f2da@oracle.com> <37bea22e-115f-5187-0f21-e04536691efc@oracle.com> Message-ID: Hi Julia, The fix has been updated accordingly. Please review it. Thanks for your time. *BUG DESCRIPTION*: JDK-8245694 : java.util.Properties.entrySet() does not override java.lang.Object methods since java 9. *PROPOSED FIX*: Add delegating overrides for equals(), hashCode(), and toString(). *PATCH*: # HG changeset patch # User yuli # Date 1592663277 -28800 # Sat Jun 20 22:27:57 2020 +0800 # Node ID 09736e9e4c386691331d1c9cbdf4b4b606d44d8d # Parent 216c6baa0564a1889e8e7b717f476eae7442ef27 8245694: java.util.Properties.entrySet() does not override Object methods Summary: Add missing override methods Contributed-by: Yu Li diff -r 216c6baa0564 -r 09736e9e4c38 src/java.base/share/classes/java/util/Properties.java --- a/src/java.base/share/classes/java/util/Properties.java Sat Jun 20 15:11:19 2020 +0800 +++ b/src/java.base/share/classes/java/util/Properties.java Sat Jun 20 22:27:57 2020 +0800 @@ -1,5 +1,5 @@ /* - * Copyright (c) 1995, 2019, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2020, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -1405,6 +1405,21 @@ } @Override + public boolean equals(Object o) { + return o == this || entrySet.equals(o); + } + + @Override + public int hashCode() { + return entrySet.hashCode(); + } + + @Override + public String toString() { + return entrySet.toString(); + } + + @Override public boolean removeAll(Collection c) { return entrySet.removeAll(c); } diff -r 216c6baa0564 -r 09736e9e4c38 test/jdk/java/util/Properties/PropertiesEntrySetTest.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/jdk/java/util/Properties/PropertiesEntrySetTest.java Sat Jun 20 22:27:57 2020 +0800 @@ -0,0 +1,201 @@ +/* + * Copyright (c) 2020, Oracle and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + */ + +/* + * @test + * @bug 8245694 + * @summary tests the entrySet() method of Properties class + * @author Yu Li + * @run testng PropertiesEntrySetTest + */ + +import org.testng.annotations.Test; + +import java.util.Properties; + +import static org.testng.Assert.assertEquals; +import static org.testng.Assert.assertFalse; +import static org.testng.Assert.assertThrows; +import static org.testng.Assert.assertTrue; + +public class PropertiesEntrySetTest { + + @Test + public void testEquals() { + Properties a = new Properties(); + var aEntrySet = a.entrySet(); + assertFalse(aEntrySet.equals(null)); + assertTrue(aEntrySet.equals(aEntrySet)); + + Properties b = new Properties(); + var bEntrySet = b.entrySet(); + assertTrue(bEntrySet.equals(aEntrySet)); + assertTrue(bEntrySet.hashCode() == aEntrySet.hashCode()); + + a.setProperty("p1", "1"); + assertFalse(bEntrySet.equals(aEntrySet)); + assertFalse(bEntrySet.hashCode() == aEntrySet.hashCode()); + + b.setProperty("p1", "1"); + assertTrue(aEntrySet.equals(bEntrySet)); + assertTrue(bEntrySet.hashCode() == aEntrySet.hashCode()); + + Properties c = new Properties(); + c.setProperty("p1", "2"); + var cEntrySet = c.entrySet(); + assertFalse(cEntrySet.equals(bEntrySet)); + assertFalse(bEntrySet.hashCode() == cEntrySet.hashCode()); + assertFalse(cEntrySet.equals(aEntrySet)); + assertFalse(aEntrySet.hashCode() == cEntrySet.hashCode()); + + a.setProperty("p2", "2"); + Properties d = new Properties(); + d.setProperty("p2", "2"); + d.setProperty("p1", "1"); + var dEntrySet = d.entrySet(); + assertTrue(dEntrySet.equals(aEntrySet)); + assertTrue(aEntrySet.hashCode() == dEntrySet.hashCode()); + + a.remove("p1"); + assertFalse(aEntrySet.equals(dEntrySet)); + assertFalse(aEntrySet.hashCode() == dEntrySet.hashCode()); + + d.remove("p1", "1"); + assertTrue(dEntrySet.equals(aEntrySet)); + assertTrue(aEntrySet.hashCode() == dEntrySet.hashCode()); + + a.clear(); + assertFalse(aEntrySet.equals(dEntrySet)); + assertFalse(aEntrySet.hashCode() == dEntrySet.hashCode()); + assertTrue(aEntrySet.isEmpty()); + + d.clear(); + assertTrue(dEntrySet.equals(aEntrySet)); + assertTrue(aEntrySet.hashCode() == dEntrySet.hashCode()); + assertTrue(dEntrySet.isEmpty()); + } + + @Test + public void testToString() { + Properties a = new Properties(); + var aEntrySet = a.entrySet(); + assertEquals(aEntrySet.toString(), "[]"); + + a.setProperty("p1", "1"); + assertEquals(aEntrySet.toString(), "[p1=1]"); + + a.setProperty("p2", "2"); + assertEquals(aEntrySet.size(), 2); + assertTrue(aEntrySet.toString().trim().startsWith("[")); + assertTrue(aEntrySet.toString().contains("p1=1")); + assertTrue(aEntrySet.toString().contains("p2=2")); + assertTrue(aEntrySet.toString().trim().endsWith("]")); + + Properties b = new Properties(); + b.setProperty("p2", "2"); + b.setProperty("p1", "1"); + var bEntrySet = b.entrySet(); + assertEquals(bEntrySet.size(), 2); + assertTrue(bEntrySet.toString().trim().startsWith("[")); + assertTrue(bEntrySet.toString().contains("p1=1")); + assertTrue(bEntrySet.toString().contains("p2=2")); + assertTrue(bEntrySet.toString().trim().endsWith("]")); + + b.setProperty("p0", "0"); + assertEquals(bEntrySet.size(), 3); + assertTrue(bEntrySet.toString().contains("p0=0")); + + b.remove("p1"); + assertEquals(bEntrySet.size(), 2); + assertFalse(bEntrySet.toString().contains("p1=1")); + assertTrue(bEntrySet.toString().trim().startsWith("[")); + assertTrue(bEntrySet.toString().contains("p0=0")); + assertTrue(bEntrySet.toString().contains("p2=2")); + assertTrue(bEntrySet.toString().trim().endsWith("]")); + + b.remove("p0", "0"); + assertEquals(bEntrySet.size(), 1); + assertFalse(bEntrySet.toString().contains("p0=0")); + assertTrue(bEntrySet.toString().trim().startsWith("[")); + assertTrue(bEntrySet.toString().contains("p2=2")); + assertTrue(bEntrySet.toString().trim().endsWith("]")); + + b.clear(); + assertTrue(bEntrySet.isEmpty()); + assertTrue(bEntrySet.toString().equals("[]")); + } + + @Test + public void testEntrySetWithoutException() { + Properties a = new Properties(); + a.setProperty("p1", "1"); + a.setProperty("p2", "2"); + var aEntrySet = a.entrySet(); + assertEquals(aEntrySet.size(), 2); + + var i = aEntrySet.iterator(); + var e1 = i.next(); + i.remove(); + assertFalse(aEntrySet.contains(e1)); + assertEquals(aEntrySet.size(), 1); + + var e2 = i.next(); + aEntrySet.remove(e2); + assertFalse(aEntrySet.contains(e2)); + assertTrue(aEntrySet.isEmpty()); + + a.setProperty("p1", "1"); + a.setProperty("p3", "3"); + Properties b = new Properties(); + b.setProperty("p2", "2"); + b.setProperty("p1", "1"); + var bEntrySet = b.entrySet(); + + assertFalse(bEntrySet.containsAll(aEntrySet)); + assertEquals(bEntrySet.size(), 2); + + assertTrue(bEntrySet.removeAll(aEntrySet)); + assertEquals(bEntrySet.size(), 1); + + assertTrue(bEntrySet.retainAll(aEntrySet)); + assertTrue(bEntrySet.isEmpty()); + assertEquals(aEntrySet.size(), 2); + + aEntrySet.clear(); + assertTrue(aEntrySet.isEmpty()); + } + + @Test + public void testEntrySetExceptionWhenAdd() { + Properties a = new Properties(); + a.setProperty("p1", "1"); + var aEntrySet = a.entrySet(); + + Properties b = new Properties(); + b.setProperty("p2", "2"); + var bEntrySet = b.entrySet(); + + assertThrows(UnsupportedOperationException.class, () -> aEntrySet.addAll(bEntrySet)); + assertThrows(UnsupportedOperationException.class, () -> aEntrySet.add(bEntrySet.iterator().next())); + } +} Best regards, Yu L. On Tue, Jun 9, 2020 at 10:42 AM Lisa Li wrote: > Thanks for all your suggestions Brent! They helped a lot. > > Best regards, > Yu L. > > On Tue, Jun 9, 2020 at 7:24 AM Brent Christian > wrote: > >> Looks good to me as well. Thanks for making the updates to the test case. >> >> -Brent >> >> On 6/8/20 2:38 AM, Julia Boes wrote: >> > Hi Yu Li, >> > >> > The copyright year of Properties.java needs to be updated to 2020. >> > Otherwise looks good to me! >> > >> > Cheers, >> > >> > Julia >> > >> > From lance.andersen at oracle.com Sat Jun 20 15:57:10 2020 From: lance.andersen at oracle.com (Lance @ Oracle) Date: Sat, 20 Jun 2020 11:57:10 -0400 Subject: RFR: [15,docs] JDK-8247959,doclint errors in NIO code In-Reply-To: References: Message-ID: +1 Best, Lance Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com Sent from my iPad > On Jun 19, 2020, at 11:36 PM, Jonathan Gibbons wrote: > > ?Please review some fixes to address issues found by doclint. > > In a couple of places, imports were required in order to resolve symbols. > In another case, the name also had a typo in it. > In the last case, the incorrect syntax was used for a type parameter in @param. > > -- Jon > > JBS: https://bugs.openjdk.java.net/browse/JDK-8247959 > > Patch inline: > > diff -r 086c7f077fc6 src/jdk.nio.mapmode/share/classes/jdk/nio/mapmode/ExtendedMapMode.java > --- a/src/jdk.nio.mapmode/share/classes/jdk/nio/mapmode/ExtendedMapMode.java Fri Jun 19 15:22:19 2020 -0400 > +++ b/src/jdk.nio.mapmode/share/classes/jdk/nio/mapmode/ExtendedMapMode.java Fri Jun 19 20:24:02 2020 -0700 > @@ -25,6 +25,8 @@ > > package jdk.nio.mapmode; > > +import java.nio.MappedByteBuffer; > +import java.nio.channels.FileChannel; > import java.nio.channels.FileChannel.MapMode; > > /** > @@ -52,7 +54,7 @@ > > /** > * File mapping mode for a read-write mapping of a file backed by > - * non-volatile RAM. {@linkplain MappedByteBufefr#force force} > + * non-volatile RAM. {@linkplain MappedByteBuffer#force force} > * operations on a buffer created with this mode will be performed > * using cache line writeback rather than proceeding via a file > * device flush. > diff -r 086c7f077fc6 src/jdk.sctp/share/classes/com/sun/nio/sctp/NotificationHandler.java > --- a/src/jdk.sctp/share/classes/com/sun/nio/sctp/NotificationHandler.java Fri Jun 19 15:22:19 2020 -0400 > +++ b/src/jdk.sctp/share/classes/com/sun/nio/sctp/NotificationHandler.java Fri Jun 19 20:24:02 2020 -0700 > @@ -45,7 +45,7 @@ > * this handler interface as the type for parameters, return type, etc. rather > * than the abstract class. > * > - * @param T The type of the object attached to the receive operation > + * @param The type of the object attached to the receive operation > * > * @since 1.7 > */ > From peter.levart at gmail.com Sun Jun 21 07:27:58 2020 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 21 Jun 2020 09:27:58 +0200 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: <226395162.2170526.1592405250910.JavaMail.zimbra@u-pem.fr> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <1522316544.1652270.1592348771671.JavaMail.zimbra@u-pem.fr> <622aa076-b5d6-1e26-4966-6ff2420f25b4@gmail.com> <226395162.2170526.1592405250910.JavaMail.zimbra@u-pem.fr> Message-ID: On 6/17/20 4:47 PM, forax at univ-mlv.fr wrote: > ----- Mail original ----- >> De: "Peter Levart" >> ... TLDR; >> So for each record type >> (Class) there can be several distinct >> ObjectStreamClasses deserialized in the same VM that were produced by >> serializing different versions of record types in different VMs... > Ok, i see, you want to cache all permutations not only the one corresponding to the current record order. > > R?mi Right. So while the most prevalent shape of stream will be the one that is produced by the same version of class as is currently used in the local VM, there can be situations where this is not true. For example, you upgrade some service with new version of class, but want to support older clients as well as new. If older clients suddenly experienced a drop of deserialization performance, it would not be nice. Regards, Peter >> >> On 6/17/20 1:06 AM, Remi Forax wrote: >>> Hi Peter, >>> is their a reason to not use a ClassValue using the record class >>> as key instead of the more complex keys you are proposing ? >>> >>> R?mi From peter.levart at gmail.com Sun Jun 21 12:49:15 2020 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 21 Jun 2020 14:49:15 +0200 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: <1DA206A1-15F0-46F9-B69E-9896C57B97C7@oracle.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> <1DA206A1-15F0-46F9-B69E-9896C57B97C7@oracle.com> Message-ID: <8886ee56-e9a7-aafe-6111-f0c2c54a282e@gmail.com> Hi Chris, Good work on SerialByteStreamBuilder. A very useful tool. What I noticed in the tests is that you usually follow the following pattern: - create a record instance r - serialize it to bytes and deserialize it back to instance deser1 - create a stream with SerialByteStreamBuilder and deserialize it to instance deser2 - compare deser1 with deser2 Now if there was a bug in deserialization logic it would be possible for both deser1 and deser2 to be equal but still wrong. Adding another assert to compare r with deser1 in each test could catch such bug. In the following test: ?245???? public void testArrays() throws Exception { ?246???????? out.println("\n---"); ?247???????? { ?248???????????? record IntArray(int[] ints, long[] longs) implements Serializable { } ?249???????????? IntArray r = new IntArray(new int[] { 5, 4, 3,2, 1 }, new long[] { 9L }); ?250???????????? IntArray deser1 = deserialize(serialize(r)); ?251 ?252???????????? byte[] builderBytes = SerialByteStreamBuilder ?253???????????????????? .newBuilder(IntArray.class.getName()) ?254???????????????????? .addField("ints", String[].class, new int[] { 5, 4, 3,2, 1 }) ?255???????????????????? .addField("longs", String[].class, new long[] { 9L }) ?256???????????????????? .build(); ?257 ?258???????????? IntArray deser2 = deserialize(builderBytes); ?259???????????? assertEquals(deser2.ints(), deser1.ints()); ?260???????????? assertEquals(deser2.longs(), deser1.longs()); ?261???????? } ...in lines 254, 255, you specify wrong types of fields (making the method generic with a type parameter could prevent such mistakes). I wonder why the test passes. In both variants (with or without the patch) we have this check in java.io.ObjectStreamClass.RecordSupport: ??????????? for (int i = 0; i < fields.length; i++) { ??????????????? ObjectStreamField f = fields[i]; ??????????????? String fName = f.getName(); ??????????????? if (!fName.equals(pName)) ??????????????????? continue; ??????????????? Class fType = f.getField().getType(); ??????????????? if (!pType.isAssignableFrom(fType)) ??????????????????? throw new InternalError(fName + " unassignable, pType:" + pType + ", fType:" + fType); ... which is actually checking types of reflected java.lang.reflact.Field(s) with types of reflected java.lang.reflect.RecordComponent(s) that match in name which always match when record type is correctly constructed. And later when values are deserialized and assigned to constructor parameters, we have the right types of values in this "unusual" stream. In other words, changing the field type from one reference type to another is always a compatible change as far as stream format is concerned, deserialization then succeeds if the value in the stream is of correct type for the local type of field. Theoretically the change can be such that the reference types are unrelated, because such types have one common value: null. In this example the stream was unusual. The types were unrelated, but the stream had the correct type of value != null. Anyway, maybe the test should also contain a case where the type of field does change compatibly (for example, Integer field serialized, deserialized into Number field), and one or more where it changes incompatibly. For the incompatible changes, I think some exceptions would be emitted by MethodHandle adapters that try to adapt incompatible reference types, other incompatible primitive field type changes would be caught by java.io.ObjectStreamClass.matchFields() method... I think that the following method in SerialByteStreamBuilder is also incorrect: ?363???????? private void writeObjectFieldDesc(DataOutputStream out) throws IOException { ?364???????????? for (Map.Entry entry : objectFields.entrySet()) { ?365???????????????? Class cl = entry.getKey().type(); ?366???????????????? assert !cl.isPrimitive(); ?367???????????????? // obj_typecode ?368???????????????? if (cl.isArray()) { ?369???????????????????? out.writeByte('['); ?370???????????????? } else { ?371???????????????????? out.writeByte('L'); ?372???????????????? } ?373???????????????? writeUTF(out, entry.getKey().name()); ?374???????????????? out.writeByte(TC_STRING); ?375???????????????? writeUTF(out, "L" + cl.getName().replace('.', '/') + ";"); ?376 ?377???????????? } ?378???????? } Line 375 should probably be: ??????????? writeUTF(out, ???????????????????? (cl.isArray() ? cl.getName() : "L" + cl.getName() + ";") ???????????????????????? .replace('.', '/')); So I took the liberty and incorporated your work into my patch (both will be authors of the patch) and addressed above concerns. The modified test does not pass on JDK14. In testIncompatibleRefFieldTypeChange JDK 14 throws ClassCastException (because it is thrown from the binding of arguments to method handle) while with the patched JDK16 it throws InvalidObjectException, because ClassCastException is thrown from invokeExact, which is then wrapped with InvalidObjectException and I think is more correct: http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.04/ So this patch actually fixes a little bug too. Regards, Peter 6/17/20 4:13 PM, Chris Hegarty wrote: > Peter, > >> On 17 Jun 2020, at 09:23, Chris Hegarty wrote: >> >>> On 17 Jun 2020, at 06:44, Peter Levart wrote: >>>>> .. >>>> If there is a way for a test to compile several versions of the same (record) class and then produce byte streams with it, then we could simulate various class-evolution cases (field missing, surplus field, change of field type, etc...) without crafting the bytestream by hand. WDYT? >>> I have a better idea. The test could contain several different classes with different names that would be used to generated variations of serialized stream. Such streams could then be "patched" so they would contain the common target class name and then used to (attempt to) deserialize the target class. I'll try to devise such test? >>> >> I think that could work. I?ll mock something up too, just for comparison purposes. > > Here is an initial version at a test that can be expanded to cover more of the stream-field shapes of serializable records. > > I quickly backed away from generating the byte streams using OOS alone, since it enforces strict ordering of the fields in the stream, beyond that of primitives followed by non-primitives. And I want to be able to do things like, for example, record Point(int x, int y) - stream of x:1 y:2, or y:2 x:1 - which will verify the nominality aspect of the caching implementation. So I opted for a basic byte-stream-builder approach. > > https://cr.openjdk.java.net/~chegar/serialFieldTest_webrev/test/jdk/java/io/Serializable/records/DifferentStreamFieldsTest.java.html > > I also would like a builder of serial byte streams anyway, as it will be useful for things beyond this change. Maybe it could even be expanded upon and used as a test library, at some future point. The above implementation can probably be easily broken if pushed hard with larger graphs of objects, but it should be good enough for what we need here. > > -Chris. > From peter.levart at gmail.com Sun Jun 21 16:32:20 2020 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 21 Jun 2020 18:32:20 +0200 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: <8886ee56-e9a7-aafe-6111-f0c2c54a282e@gmail.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> <1DA206A1-15F0-46F9-B69E-9896C57B97C7@oracle.com> <8886ee56-e9a7-aafe-6111-f0c2c54a282e@gmail.com> Message-ID: <63689945-dca3-e0cc-72b8-9225f5da6d31@gmail.com> Hi Chris, Here's a small optimization over webrev.04 in the caching logic.We don't need a doubly-linked list to implement a FIFO. Just single 'next' link in each key is sufficient. I also changed initialCapacity of ConcurrentHashMap to 2 (which should be typical number of cached shapes per record class when there's more than 1 and requires 4 slots in internal table) and MAX_SIZE? of cache to 10 (which is the greatest number that requires 16 slots in internal table)... http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.05/ Regards, Peter From peter.levart at gmail.com Sun Jun 21 17:16:05 2020 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 21 Jun 2020 19:16:05 +0200 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: <63689945-dca3-e0cc-72b8-9225f5da6d31@gmail.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> <1DA206A1-15F0-46F9-B69E-9896C57B97C7@oracle.com> <8886ee56-e9a7-aafe-6111-f0c2c54a282e@gmail.com> <63689945-dca3-e0cc-72b8-9225f5da6d31@gmail.com> Message-ID: Hi, When re-running the benchmark [1] with different lengths of serialized arrays of records, I found that, compared to classical classes, lookup into the cache of adapted method handles starts to show when the length of array is larger (# of instances of same record type deserialized in single stream). Each record deserialized must lookup the method handle in a ConcurrentHashMap: Benchmark??????????????????????????????????? (length)? Mode? Cnt??? Score?? Error? Units RecordSerializationBench.deserializeClasses??????? 10? avgt?? 10??? 8.088 ? 0.013? us/op RecordSerializationBench.deserializeClasses?????? 100? avgt?? 10?? 32.171 ? 0.324? us/op RecordSerializationBench.deserializeClasses????? 1000? avgt?? 10? 279.762 ? 3.072? us/op RecordSerializationBench.deserializeRecords??????? 10? avgt?? 10??? 9.011 ? 0.027? us/op RecordSerializationBench.deserializeRecords?????? 100? avgt?? 10?? 33.206 ? 0.514? us/op RecordSerializationBench.deserializeRecords????? 1000? avgt?? 10? 325.137 ? 0.969? us/op ...so keeping the correctly shaped adapted method handle in the per-serialization-session ObjectStreamClass instance [2] starts to make sense: Benchmark??????????????????????????????????? (length)? Mode? Cnt??? Score?? Error? Units RecordSerializationBench.deserializeClasses??????? 10? avgt?? 10??? 8.681 ? 0.155? us/op RecordSerializationBench.deserializeClasses?????? 100? avgt?? 10?? 32.496 ? 0.087? us/op RecordSerializationBench.deserializeClasses????? 1000? avgt?? 10? 279.014 ? 1.189? us/op RecordSerializationBench.deserializeRecords??????? 10? avgt?? 10??? 8.537 ? 0.032? us/op RecordSerializationBench.deserializeRecords?????? 100? avgt?? 10?? 31.451 ? 0.083? us/op RecordSerializationBench.deserializeRecords????? 1000? avgt?? 10? 250.854 ? 2.772? us/op With that, more objects means advantage over classical classes instead of disadvantage. [1] http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/RecordSerializationBench.java [2] http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.06/ Regards, Peter From peter.levart at gmail.com Sun Jun 21 22:32:15 2020 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 22 Jun 2020 00:32:15 +0200 Subject: (15) RFR: JDK-8247444: Trust final fields in records In-Reply-To: <029fdb77-660b-4f59-7ee7-bc7020b5e49f@redhat.com> References: <4B268BF1-1DFD-4DCC-AEA3-7F2C7B81DC91@oracle.com> <029fdb77-660b-4f59-7ee7-bc7020b5e49f@redhat.com> Message-ID: <2cc3ff95-4352-1a52-e395-ef90d2338891@gmail.com> Hi Aleksey, On 6/18/20 7:10 PM, Aleksey Shipilev wrote: > On 6/18/20 3:09 PM, Chris Hegarty wrote: >>> On 18 Jun 2020, at 13:43, Aleksey Shipilev wrote: >>> Again, JDK-8247532 is the writing on the wall: we don't need 3rd party developers to tell if Record >>> serialization works fast in 15 -- we already know it does not. >> I disagree. JDK-8247532 is under review and well on its way towards JDK 15 (yes, during RDP 1). >> My reading of Peter?s benchmark result is that Record deserialization *does work fast*. What am >> I missing. > JDK-8247532 is the evidence that Records serialization performance story is not clear. > > Even if we disregard that after JDK-8247532 [1] Records are still 8% slower, the _existence_ of > JDK-8247532 indicates there are performance problems in the area. That evidence now needs to be > compensated by much more evidence to the contrary. (Yes, I contracted a lot of Bayesian thinking > from my statistician wife...) > > (Here were several paragraphs of further thoughts, but I realized it basically repeats what I said > before.) Let me just express my thoughts on the subject of preventing writes to final fields in records in connection to deserialization. In short, I don't think any serialization framework would need to have direct write access to record fields in order to be performant. Serialization frameworks need write access to final fields in classical classes mainly because fields are generally encapsulated. If classes don't implement special API (constructors, methods, annotations ...) to interface with serialization framework (like for example Jackson annotations, etc.) then serialization framework can only interface such classes on the level of fields. Now records are very constrained classes. They have an always present API that can be used to interface with serialization frameworks. For each field, they poses a constructor parameter with the same name and type and are arranged so that the fields are set from the constructor parameters. So instead of allocating a "zero" instance and setting fields, one can just invoke a canonical constructor which does the same. Such invocation is equally performant as allocating instance and setting fields from outside the constructor. What I did in JDK-8247532 was that I just optimized the code that transforms the bytes read from stream into values for arguments of the canonical constructor using method handle transformations. That part was slow and not the fact that records are deserialized using their canonical constructor. Every serialization framework that works on the level of fields has to do the transformation of stream encoded data to field values - how it does that is what differs it from the rest - the final mechanism how those values are stored into object fields is not what makes it unique or performant. There is also a desire that record constructor(s) are the only way to create instances of records such that any validation logic in the constructor(s) can't get bypassed. Not even via deserialization of "forged" streams. That way we can have records that are safer without scarifying performance. And BTW, according to latest JDK-8247532 webrev.06, Java deserialization of records is not slower but sometimes even faster than deserialization of equivalent classical classes. Regards, Peter From claes.redestad at oracle.com Sun Jun 21 23:04:59 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 22 Jun 2020 01:04:59 +0200 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> <1DA206A1-15F0-46F9-B69E-9896C57B97C7@oracle.com> <8886ee56-e9a7-aafe-6111-f0c2c54a282e@gmail.com> <63689945-dca3-e0cc-72b8-9225f5da6d31@gmail.com> Message-ID: <7f7b58fa-50e0-a4fe-f948-9637d04c0ccc@oracle.com> Hi Peter, patch and results look great! My only real comment on this is that I think the microbenchmark would be a valuable contribution, too. It'd also be interesting to explore how poor performance would become if we'd hit the (artificial) 11 layouts limit, e.g, by cycling through 10, 11, or 12 different shapes. /Claes On 2020-06-21 19:16, Peter Levart wrote: > Hi, > > > When re-running the benchmark [1] with different lengths of serialized > arrays of records, I found that, compared to classical classes, lookup > into the cache of adapted method handles starts to show when the length > of array is larger (# of instances of same record type deserialized in > single stream). Each record deserialized must lookup the method handle > in a ConcurrentHashMap: > > > Benchmark??????????????????????????????????? (length)? Mode? Cnt > Score?? Error? Units > RecordSerializationBench.deserializeClasses??????? 10? avgt?? 10 > 8.088 ? 0.013? us/op > RecordSerializationBench.deserializeClasses?????? 100? avgt?? 10 > 32.171 ? 0.324? us/op > RecordSerializationBench.deserializeClasses????? 1000? avgt?? 10 > 279.762 ? 3.072? us/op > RecordSerializationBench.deserializeRecords??????? 10? avgt?? 10 > 9.011 ? 0.027? us/op > RecordSerializationBench.deserializeRecords?????? 100? avgt?? 10 > 33.206 ? 0.514? us/op > RecordSerializationBench.deserializeRecords????? 1000? avgt?? 10 > 325.137 ? 0.969? us/op > > > ...so keeping the correctly shaped adapted method handle in the > per-serialization-session ObjectStreamClass instance [2] starts to make > sense: > > > Benchmark??????????????????????????????????? (length)? Mode? Cnt > Score?? Error? Units > RecordSerializationBench.deserializeClasses??????? 10? avgt?? 10 > 8.681 ? 0.155? us/op > RecordSerializationBench.deserializeClasses?????? 100? avgt?? 10 > 32.496 ? 0.087? us/op > RecordSerializationBench.deserializeClasses????? 1000? avgt?? 10 > 279.014 ? 1.189? us/op > RecordSerializationBench.deserializeRecords??????? 10? avgt?? 10 > 8.537 ? 0.032? us/op > RecordSerializationBench.deserializeRecords?????? 100? avgt?? 10 > 31.451 ? 0.083? us/op > RecordSerializationBench.deserializeRecords????? 1000? avgt?? 10 > 250.854 ? 2.772? us/op > > > With that, more objects means advantage over classical classes instead > of disadvantage. > > > [1] > http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/RecordSerializationBench.java > > > [2] > http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.06/ > > > > Regards, Peter > > From julia.boes at oracle.com Mon Jun 22 09:38:59 2020 From: julia.boes at oracle.com (Julia Boes) Date: Mon, 22 Jun 2020 10:38:59 +0100 Subject: [PATCH] 8245694 : java.util.Properties.entrySet() does not override java.lang.Object methods In-Reply-To: References: <9487a1c9-d0bc-3632-156d-5927e5801dc8@icemanx.nl> <20109712-d505-1b17-3e6b-ae388d554400@oracle.com> <767a4dae-55f0-ca11-dbdf-57c7a22205d3@oracle.com> <7b01289c-d7c9-47ab-f9f6-e2cf3a00f2da@oracle.com> <37bea22e-115f-5187-0f21-e04536691efc@oracle.com> Message-ID: Hi Yu, > diff -r 216c6baa0564 -r 09736e9e4c38 > src/java.base/share/classes/java/util/Properties.java > --- a/src/java.base/share/classes/java/util/Properties.java Sat Jun 20 > 15:11:19 2020 +0800 > +++ b/src/java.base/share/classes/java/util/Properties.java Sat Jun 20 > 22:27:57 2020 +0800 > @@ -1,5 +1,5 @@ > ?/* > - * Copyright (c) 1995, 2019, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 2020, Oracle and/or its affiliates. All rights reserved. > ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > ? * > ? * This code is free software; you can redistribute it and/or modify it > @@ -1405,6 +1405,21 @@ Looks good but for src/java.base/share/classes/java/util/Properties.java the copyright line has to look as such: + * Copyright (c) 2018, 2020, Oracle and/or its affiliates. All rights The line contains the year when the file was added and the year when the last change was made (if applicable). So for an existing file, you change the ending copyright year. For a newly added file, only the current year is listed. I hope this makes sense. Cheers, Julia From sean.coffey at oracle.com Mon Jun 22 11:22:32 2020 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Mon, 22 Jun 2020 12:22:32 +0100 Subject: RFR: 8218021: jarsigner strips the execute permission when signing a .zip file In-Reply-To: <22AD74BE-6E90-45CA-AEEF-73E5FCCC0BF5@oracle.com> References: <6a34a99c-ed06-5e38-0932-788e944a70df@oracle.com> <22AD74BE-6E90-45CA-AEEF-73E5FCCC0BF5@oracle.com> Message-ID: <60ef9824-bb1c-9ff1-11a8-76290f8f8882@oracle.com> Thanks Lance. I've updated the patch with some extra offline feedback from yourself and Max. A new warning is printed with use of the new flag. A warning is also printed when file posix permissions are detected on resources being signed. Test updated for that also. https://cr.openjdk.java.net/~coffeys/webrev.8218021.v3/webrev/ regards, Sean. On 12/06/2020 17:05, Lance Andersen wrote: > Hi Sean, > > I think your changes look fine so all good FMPOV. > > Best > Lance > >> On Jun 12, 2020, at 6:21 AM, Se?n Coffey > > wrote: >> >> Hi, >> >> I'd like to reboot this jarsigner enhancement request[1]. I've >> removed the problem references to zip file name extensions. Instead, >> there's a new JDK implementation specific jarsigner option: >> -keepposixperms >> >> https://bugs.openjdk.java.net/browse/JDK-8218021 >> https://cr.openjdk.java.net/~coffeys/webrev.8218021.v2/webrev/ >> >> regards, >> Sean. >> >> [1] >> http://mail.openjdk.java.net/pipermail/security-dev/2020-January/021141.html >> > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle?Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From chris.hegarty at oracle.com Mon Jun 22 11:23:14 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 22 Jun 2020 12:23:14 +0100 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> <1DA206A1-15F0-46F9-B69E-9896C57B97C7@oracle.com> <8886ee56-e9a7-aafe-6111-f0c2c54a282e@gmail.com> <63689945-dca3-e0cc-72b8-9225f5da6d31@gmail.com> Message-ID: <72E25EBE-9A2A-4606-9B57-970ABA2F5EF4@oracle.com> Peter, Thank you for taking the suggestion for additional test scenarios, improving coverage, and finding-and-fixing the issues in my hurried test builder (was only meant as a rough guide/idea). I agree with Claes, it would be good to include the microbenchmark in test/micro/.. I think this version, 0.6, is very good. There is quite a bit of code to deal with the caching and lookup, but I don?t see an existing alternative implementation in the JDK that could be reused or moulded to fit this particular use case. I experimented a little with spinning a class to hold the transformed MH in the constant pool, and accessing through a SAM type, just to see it if had any positive effect on the benchmark numbers - it did not. -Chris. > On 21 Jun 2020, at 18:16, Peter Levart wrote: > > Hi, > > > > When re-running the benchmark [1] with different lengths of serialized arrays of records, I found that, compared to classical classes, lookup into the cache of adapted method handles starts to show when the length of array is larger (# of instances of same record type deserialized in single stream). Each record deserialized must lookup the method handle in a ConcurrentHashMap: > > > > Benchmark (length) Mode Cnt Score Error Units > RecordSerializationBench.deserializeClasses 10 avgt 10 8.088 ? 0.013 us/op > RecordSerializationBench.deserializeClasses 100 avgt 10 32.171 ? 0.324 us/op > RecordSerializationBench.deserializeClasses 1000 avgt 10 279.762 ? 3.072 us/op > RecordSerializationBench.deserializeRecords 10 avgt 10 9.011 ? 0.027 us/op > RecordSerializationBench.deserializeRecords 100 avgt 10 33.206 ? 0.514 us/op > RecordSerializationBench.deserializeRecords 1000 avgt 10 325.137 ? 0.969 us/op > > > ...so keeping the correctly shaped adapted method handle in the per-serialization-session ObjectStreamClass instance [2] starts to make sense: > > > > Benchmark (length) Mode Cnt Score Error Units > RecordSerializationBench.deserializeClasses 10 avgt 10 8.681 ? 0.155 us/op > RecordSerializationBench.deserializeClasses 100 avgt 10 32.496 ? 0.087 us/op > RecordSerializationBench.deserializeClasses 1000 avgt 10 279.014 ? 1.189 us/op > RecordSerializationBench.deserializeRecords 10 avgt 10 8.537 ? 0.032 us/op > RecordSerializationBench.deserializeRecords 100 avgt 10 31.451 ? 0.083 us/op > RecordSerializationBench.deserializeRecords 1000 avgt 10 250.854 ? 2.772 us/op > > > With that, more objects means advantage over classical classes instead of disadvantage. > > > > [1] http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/RecordSerializationBench.java > [2] http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.06/ > > Regards, Peter > > > From liyu.italy at gmail.com Mon Jun 22 11:53:28 2020 From: liyu.italy at gmail.com (Lisa Li) Date: Mon, 22 Jun 2020 19:53:28 +0800 Subject: [PATCH] 8245694 : java.util.Properties.entrySet() does not override java.lang.Object methods In-Reply-To: References: <9487a1c9-d0bc-3632-156d-5927e5801dc8@icemanx.nl> <20109712-d505-1b17-3e6b-ae388d554400@oracle.com> <767a4dae-55f0-ca11-dbdf-57c7a22205d3@oracle.com> <7b01289c-d7c9-47ab-f9f6-e2cf3a00f2da@oracle.com> <37bea22e-115f-5187-0f21-e04536691efc@oracle.com> Message-ID: Hi Julia, Sorry 'bout the rookie mistake. Thanks a lot for the clarification! Just to confirm, for src/java.base/share/classes/java/util/Properties.java the copyright should be updated as : + * Copyright (c) 1995, 2020, Oracle and/or its affiliates. All rights Because 1995 was the year when the file was added, while 2020 when this change is made. Correct me if I'm wrong. Best regards, Yu L. On Mon, Jun 22, 2020 at 5:41 PM Julia Boes wrote: > Hi Yu, > > diff -r 216c6baa0564 -r 09736e9e4c38 > src/java.base/share/classes/java/util/Properties.java > --- a/src/java.base/share/classes/java/util/Properties.java Sat Jun 20 > 15:11:19 2020 +0800 > +++ b/src/java.base/share/classes/java/util/Properties.java Sat Jun 20 > 22:27:57 2020 +0800 > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 1995, 2019, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 2020, Oracle and/or its affiliates. All rights reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -1405,6 +1405,21 @@ > > Looks good but for src/java.base/share/classes/java/util/Properties.java > the copyright line has to look as such: > > + * Copyright (c) 2018, 2020, Oracle and/or its affiliates. All rights > > The line contains the year when the file was added and the year when the > last change was made (if applicable). So for an existing file, you change > the ending copyright year. For a newly added file, only the current year is > listed. I hope this makes sense. > > Cheers, > > Julia > From julia.boes at oracle.com Mon Jun 22 13:56:03 2020 From: julia.boes at oracle.com (Julia Boes) Date: Mon, 22 Jun 2020 14:56:03 +0100 Subject: [PATCH] 8245694 : java.util.Properties.entrySet() does not override java.lang.Object methods In-Reply-To: References: <9487a1c9-d0bc-3632-156d-5927e5801dc8@icemanx.nl> <20109712-d505-1b17-3e6b-ae388d554400@oracle.com> <767a4dae-55f0-ca11-dbdf-57c7a22205d3@oracle.com> <7b01289c-d7c9-47ab-f9f6-e2cf3a00f2da@oracle.com> <37bea22e-115f-5187-0f21-e04536691efc@oracle.com> Message-ID: <140eb4ab-ff9b-eb20-2d37-e6bb5f9506b4@oracle.com> Hi Yu, > Just to confirm, for > src/java.base/share/classes/java/util/Properties.java the copyright > should be updated as : > > + * Copyright (c) 1995, 2020, Oracle and/or its affiliates. All rights > > Because 1995 was the year when the?file was added, while 2020 when > this change is made. Correct me if I'm wrong. That's right! Cheers, Julia From maurizio.cimadamore at oracle.com Mon Jun 22 14:41:01 2020 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 22 Jun 2020 15:41:01 +0100 Subject: RFR (15) 8248011: Improve javadoc of Foreign Memory Access API Message-ID: <891a53cb-9d44-628e-d720-c346e1365289@oracle.com> Hi, this is a patch fix address several javadoc issues in the foreign memory API, following some internal reviews (mostly from Brian, thanks!). The changes are aimed at reordering sections in the doc so that more advanced concepts are exposed last, and also to fix various issues in code samples (especially around MemoryLayout). Since I was there I also took the opportunity of dropping a now-unused method in MemoryAddressImpl (copy). Webrev: http://cr.openjdk.java.net/~mcimadamore/8248011/ Cheers Maurizio From sebastian.stenzel at gmail.com Mon Jun 22 15:39:06 2020 From: sebastian.stenzel at gmail.com (Sebastian Stenzel) Date: Mon, 22 Jun 2020 17:39:06 +0200 Subject: Undocumented exceptions in java.net.http.HttpClient.newHttpClient() Message-ID: <021CA75F-0338-47E6-9945-8FBFA8053300@gmail.com> Certain users of my software run into problems with HttpClient.newHttpClient() on JDK 14.0.1 and I don't feel like I can handle it properly without catching Errors. Calling said method fails when encountering Selector.open(), which in its Windows-specific implementation relies on loopback-TCP connections. These connections can, of course, be blocked by the OS or, as in the case of my user, by some anti-malware tool. I can't argue whether it is good or bad implementing the selector the way it is, however I'm pretty sure that commenting exceptions being "unlikely" without further explanation isn't exactly the best practice for this case: https://github.com/openjdk/jdk/blob/5adfaa39866f3127000f0779158c65afe1d24007/src/java.net.http/share/classes/jdk/internal/net/http/HttpClientImpl.java#L309-L314 Since my related bug report (JDK-8247996) was turned down (maybe the tester forgot to or chose not to block the socket connection), I'd like to ask if it is at least possible to document the fact that creating an HttpClient can have the side effect of instantaneously creating a loopback connection. I would prefer to even throw a checked exception, but this would break the API. Happy to create a PR, but I'd like to do this the right way, therefore starting this discussion. From Alan.Bateman at oracle.com Mon Jun 22 15:47:50 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 22 Jun 2020 16:47:50 +0100 Subject: Undocumented exceptions in java.net.http.HttpClient.newHttpClient() In-Reply-To: <021CA75F-0338-47E6-9945-8FBFA8053300@gmail.com> References: <021CA75F-0338-47E6-9945-8FBFA8053300@gmail.com> Message-ID: On 22/06/2020 16:39, Sebastian Stenzel wrote: > Certain users of my software run into problems with HttpClient.newHttpClient() on JDK 14.0.1 and I don't feel like I can handle it properly without catching Errors. > > Calling said method fails when encountering Selector.open(), which in its Windows-specific implementation relies on loopback-TCP connections. These connections can, of course, be blocked by the OS or, as in the case of my user, by some anti-malware tool. > > I can't argue whether it is good or bad implementing the selector the way it is, however I'm pretty sure that commenting exceptions being "unlikely" without further explanation isn't exactly the best practice for this case: > > https://github.com/openjdk/jdk/blob/5adfaa39866f3127000f0779158c65afe1d24007/src/java.net.http/share/classes/jdk/internal/net/http/HttpClientImpl.java#L309-L314 > > Since my related bug report (JDK-8247996) was turned down (maybe the tester forgot to or chose not to block the socket connection), I'd like to ask if it is at least possible to document the fact that creating an HttpClient can have the side effect of instantaneously creating a loopback connection. I would prefer to even throw a checked exception, but this would break the API. > > Happy to create a PR, but I'd like to do this the right way, therefore starting this discussion. The wakeup mechanism in the Selector implementation on Windows uses a loopback connection. Something fishy in the configuration (or firewall or VPN) if that is blocked. There is a proposal to replace this with a Unix domain socket but that will only work on newer releases of Windows. The HTTP client behavior to catch the IOException and throw InternalError is currently tracked by JDK-8248006 [1]. -Alan [1] https://bugs.openjdk.java.net/browse/JDK-8248006 From michael.x.mcmahon at oracle.com Mon Jun 22 15:55:01 2020 From: michael.x.mcmahon at oracle.com (Michael McMahon) Date: Mon, 22 Jun 2020 16:55:01 +0100 Subject: Undocumented exceptions in java.net.http.HttpClient.newHttpClient() In-Reply-To: <021CA75F-0338-47E6-9945-8FBFA8053300@gmail.com> References: <021CA75F-0338-47E6-9945-8FBFA8053300@gmail.com> Message-ID: <2feccaa9-35c6-67e2-41cd-a1fe801a85f7@oracle.com> Hi, Yes, that report probably should not have been closed as it is definitely an issue. I will re-open it. Michael On 22/06/2020 16:39, Sebastian Stenzel wrote: > Certain users of my software run into problems with HttpClient.newHttpClient() on JDK 14.0.1 and I don't feel like I can handle it properly without catching Errors. > > Calling said method fails when encountering Selector.open(), which in its Windows-specific implementation relies on loopback-TCP connections. These connections can, of course, be blocked by the OS or, as in the case of my user, by some anti-malware tool. > > I can't argue whether it is good or bad implementing the selector the way it is, however I'm pretty sure that commenting exceptions being "unlikely" without further explanation isn't exactly the best practice for this case: > > https://github.com/openjdk/jdk/blob/5adfaa39866f3127000f0779158c65afe1d24007/src/java.net.http/share/classes/jdk/internal/net/http/HttpClientImpl.java#L309-L314 > > Since my related bug report (JDK-8247996) was turned down (maybe the tester forgot to or chose not to block the socket connection), I'd like to ask if it is at least possible to document the fact that creating an HttpClient can have the side effect of instantaneously creating a loopback connection. I would prefer to even throw a checked exception, but this would break the API. > > Happy to create a PR, but I'd like to do this the right way, therefore starting this discussion. From lance.andersen at oracle.com Mon Jun 22 16:17:08 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Mon, 22 Jun 2020 12:17:08 -0400 Subject: RFR: 8218021: jarsigner strips the execute permission when signing a .zip file In-Reply-To: <60ef9824-bb1c-9ff1-11a8-76290f8f8882@oracle.com> References: <6a34a99c-ed06-5e38-0932-788e944a70df@oracle.com> <22AD74BE-6E90-45CA-AEEF-73E5FCCC0BF5@oracle.com> <60ef9824-bb1c-9ff1-11a8-76290f8f8882@oracle.com> Message-ID: HI Sean, Looks OK based on our exchanges. Thank you for your time on this one! Best Lance > On Jun 22, 2020, at 7:22 AM, Se?n Coffey wrote: > > Thanks Lance. > > I've updated the patch with some extra offline feedback from yourself and Max. > A new warning is printed with use of the new flag. A warning is also printed when file posix permissions are detected on resources being signed. Test updated for that also. > > https://cr.openjdk.java.net/~coffeys/webrev.8218021.v3/webrev/ > regards, > Sean. > > On 12/06/2020 17:05, Lance Andersen wrote: >> Hi Sean, >> >> I think your changes look fine so all good FMPOV. >> >> Best >> Lance >> >>> On Jun 12, 2020, at 6:21 AM, Se?n Coffey > wrote: >>> >>> Hi, >>> >>> I'd like to reboot this jarsigner enhancement request[1]. I've removed the problem references to zip file name extensions. Instead, there's a new JDK implementation specific jarsigner option: -keepposixperms >>> >>> https://bugs.openjdk.java.net/browse/JDK-8218021 >>> https://cr.openjdk.java.net/~coffeys/webrev.8218021.v2/webrev/ >>> >>> regards, >>> Sean. >>> >>> [1] http://mail.openjdk.java.net/pipermail/security-dev/2020-January/021141.html >>> >> >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From liyu.italy at gmail.com Mon Jun 22 16:32:13 2020 From: liyu.italy at gmail.com (Lisa Li) Date: Tue, 23 Jun 2020 00:32:13 +0800 Subject: [PATCH] 8245694 : java.util.Properties.entrySet() does not override java.lang.Object methods In-Reply-To: <140eb4ab-ff9b-eb20-2d37-e6bb5f9506b4@oracle.com> References: <9487a1c9-d0bc-3632-156d-5927e5801dc8@icemanx.nl> <20109712-d505-1b17-3e6b-ae388d554400@oracle.com> <767a4dae-55f0-ca11-dbdf-57c7a22205d3@oracle.com> <7b01289c-d7c9-47ab-f9f6-e2cf3a00f2da@oracle.com> <37bea22e-115f-5187-0f21-e04536691efc@oracle.com> <140eb4ab-ff9b-eb20-2d37-e6bb5f9506b4@oracle.com> Message-ID: Great, thanks Julia! Best regards, Yu L. On Mon, Jun 22, 2020 at 9:56 PM Julia Boes wrote: > Hi Yu, > > Just to confirm, for src/java.base/share/classes/java/util/Properties.java > the copyright should be updated as : > > + * Copyright (c) 1995, 2020, Oracle and/or its affiliates. All rights > > Because 1995 was the year when the file was added, while 2020 when this > change is made. Correct me if I'm wrong. > > That's right! > > Cheers, > > Julia > > > From alexey.semenyuk at oracle.com Mon Jun 22 16:43:41 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Mon, 22 Jun 2020 12:43:41 -0400 Subject: RFR: 8225251: Minimise import statements in jpackage sources In-Reply-To: References: <35bacbc9-d535-6b33-3c46-cfebb5e24d32@oracle.com> <4d1a457c-ffb6-5921-cb77-ba3b18ac4595@oracle.com> <802ce899-eb1b-8bc0-dddf-956788065494@oracle.com> <49fac126-0ad6-bbbb-aead-7dd12bd006b1@oracle.com> Message-ID: Looks good. - Alexey On 6/19/2020 10:05 PM, alexander.matveev at oracle.com wrote: > http://cr.openjdk.java.net/~almatvee/8225251/webrev.01/ > > - Expended "import static org.junit.Assert.*" and "import static > jdk.incubator.jpackage.internal.StandardBundlerParam.*". > - Fixed inconsistency between imported StandardBundlerParam or when > used directly. See > http://cr.openjdk.java.net/~almatvee/8225251/webrev.01/src/jdk.incubator.jpackage/macosx/classes/jdk/incubator/jpackage/internal/MacAppImageBuilder.java.frames.html > for example. > > Thanks, > Alexander > > On 6/19/20 3:36 PM, Andy Herrick wrote: >> I'm fine with it either way - the main thing is what you have done. >> >> /Andy >> >> On 6/19/2020 6:02 PM, alexander.matveev at oracle.com wrote: >>> Hi Alexey, >>> >>> Andy mentioned in issue description not to expend static imports, >>> this is why all static imports were not expended. I can expend them >>> or we can keep as is. I am fine with either way. Andy any comments >>> why we do not need to expend static imports? >>> >>> Thanks, >>> Alexander >>> >>> On 6/19/20 10:49 AM, Alexey Semenyuk wrote: >>>> This is how they were ut in unit tests initially. I'm OK either >>>> way, just wanted to double check these imports were not overlooked. >>>> >>>> - Alexey >>>> >>>> On 6/19/2020 12:30 PM, Kevin Rushforth wrote: >>>>> That a reasonably common pattern in JUnit tests, but expanding >>>>> them to individual static imports is, of course, fine as well. >>>>> >>>>> -- Kevin >>>>> >>>>> >>>>> On 6/19/2020 9:08 AM, Alexey Semenyuk wrote: >>>>>> Alexander, >>>>>> >>>>>> There is still >>>>>> --- >>>>>> import static org.junit.Assert.*; >>>>>> --- >>>>>> in unit tests. Is this intended? >>>>>> >>>>>> - Alexey >>>>>> >>>>>> On 6/19/2020 4:46 AM, alexander.matveev at oracle.com wrote: >>>>>>> Please review the jpackage fix for bug [1] at [2]. >>>>>>> >>>>>>> Cleanup import statements. >>>>>>> >>>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8225251 >>>>>>> [2] http://cr.openjdk.java.net/~almatvee/8225251/webrev.00/ >>>>>>> >>>>>>> Thanks, >>>>>>> Alexander >>>>>> >>>>> >>>> >>> > From chris.hegarty at oracle.com Mon Jun 22 17:36:31 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 22 Jun 2020 18:36:31 +0100 Subject: RFR [15] 8247696: Incorrect tail computation for large segments in AbstractMemorySegmentImpl::mismatch In-Reply-To: References: <0B15B6AF-F196-4192-B448-F1955AE586A7@oracle.com> <94fceeda-edee-c571-b32d-af8d5aed672e@oracle.com> <7BA53832-4D14-405D-9AE0-118ABB1344A2@oracle.com> Message-ID: Paul pointed out an issue, off list, where subsequent subranges could still result in a call to the intrinsic. That is now fixed in-place and the test amended to cover the scenario. https://cr.openjdk.java.net/~chegar/8247696/webrev.01/ -Chris. > On 19 Jun 2020, at 11:56, Chris Hegarty wrote: > > Paul, Maurizio, > > This version incorporates all feedback so far. > > https://cr.openjdk.java.net/~chegar/8247696/webrev.01/ > Results on my machine: > > Benchmark Mode Cnt Score Error Units > BulkOps.mismatch_large_bytebuffer avgt 30 88266.728 ? 4083.476 ns/op > BulkOps.mismatch_large_segment avgt 30 86141.343 ? 2450.450 ns/op > BulkOps.mismatch_small_bytebuffer avgt 30 6.360 ? 0.425 ns/op > BulkOps.mismatch_small_segment avgt 30 4.582 ? 1.040 ns/op > > -Chris. > >> On 19 Jun 2020, at 00:35, Paul Sandoz wrote: >> >> Thanks Chris. >> >>> On Jun 18, 2020, at 2:57 AM, Maurizio Cimadamore wrote: >>> >>> Thanks for looking at this Chris >>> >>> On 17/06/2020 21:56, Paul Sandoz wrote: >>>> Hi Chris, >>>> >>>> AbstractMemorySegmentImpl >>>> ? >>>> >>>> In vectorizedMismatchLarge: >>>> >>>> 163 if (remaining > 7) >>>> 164 throw new InternalError("remaining greater than 7: " + remaining); >>>> 165 i = length - remaining; >>>> 166 } >>>> >>>> Should this check be an assert? >>> I suggested that to Chris, since sometimes asserts are enabled when running tests, sometimes are not - in langtools we moved away from using asserts as we realized that in certain cases they were silently failing. I'm ok with whatever standard you feel comfortable with though. >> >> Automated running of tests enable assertions (-ea -esa), perhaps the use is more commonly accepted in library code than compiler code. I would favor so in this case if necessary (sometimes they are avoided to reduce inline thresholds). >> >> >>>> >>>> ? >>>> >>>> This fix prompted me to think more deeply about the implementation of vectorizedMismatchLarge and its use of vectorizedMismatch. Sorry, I should have thought about this more throughly earlier on. >>>> >>>> We need to refine the approach, not for this patch, but something to follow up after. I think there are two issues. >>>> >>>> 1) The intrinsic method vectorizedMismatch could potentially bomb out at any point and return the bitwise compliment of the remaining number of elements to check. >>>> >>>> Obviously, there is no point doing that arbitrarily but a stub implementation for, say, x86 AVX-512 might decide bomb out for < 64 remaining elements, rather than apply vector operations on smaller vector sizes or use a scalar loop. It does not today, but I think we should guard against that potentially happening, otherwise bad things can happen. >>> >>> So your worry here is that we'll end up with an infinite loop, right? >>> >> >> Or more likely that an incorrect result is returned since tail elements will be skipped over as the offset and size is updated. >> >> >>> If so, we could check remaining against previous remaining and bomb out too if no further progress seem to be made? >>> >> >> I think it's better to always terminate the loop after the last sub-range is checked, rather than unnecessarily calling twice. >> >> I am not confident the vectorizedMismatch intrinsic stub has been tested properly on very small lengths, since it's never directly called in such cases, so also keeping "remaining > 7" is good too. >> >> Paul. >> >>>> >>>> I think the loop should exit when the last sub-range has been checked. We should rely on other tests to ensure the intrinsic method is operating efficiently. >>>> >>>> >>>> 2) This method only works when operating on byte arrays. It will not work correctly if operating on short or long arrays, since we are not adjusting the length and offsets accordingly by the scale. It's probably easier to just rename this as vectorizedMismatchLargeForBytes and drop the log2ArrayIndexScale argument. Then expand later if need be. I still think the method rightly belongs in ArraysSupport. >>> >>> Yep - probably good idea to restrict on bytes, for now. >>> >>> Maurizio >>> >>>> >>>> Paul. >>>> >>>>> On Jun 17, 2020, at 8:33 AM, Chris Hegarty wrote: >>>>> >>>>> The MemorySegment::mismatch implementation added vectorized mismatch of long sizes. The implementation is trivial, but the starting point for a more optimized implementation, if needed. ArraysSupport::vectorizedMismatchLarge incorrectly returns the bitwise complement of the offset of the first mismatch, where it should return the bitwise complement of the number of remaining pairs of elements to be checked in the tail of the two arrays. The AbstractMemorySegmentImpl::mismatch masked this problem, since it seamlessly compared the remaining tail, which is larger than it should be. >>>>> >>>>> Webrev: >>>>> https://cr.openjdk.java.net/~chegar/8247696/webrev.00/ >>>>> >>>>> I updated the exiting BulkOps micro-benchmark to cover mismatch. Here are the results, compared to ByteBuffer::mismatch, on my machine: >>>>> >>>>> Benchmark Mode Cnt Score Error Units >>>>> BulkOps.mismatch_large_bytebuffer avgt 30 740186.973 ? 119314.207 ns/op >>>>> BulkOps.mismatch_large_segment avgt 30 683475.305 ? 76355.043 ns/op >>>>> BulkOps.mismatch_small_bytebuffer avgt 30 7.367 ? 0.523 ns/op >>>>> BulkOps.mismatch_small_segment avgt 30 4.140 ? 0.602 ns/op >>>>> >>>>> >>>>> -Chris. > From paul.sandoz at oracle.com Mon Jun 22 17:55:29 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 22 Jun 2020 10:55:29 -0700 Subject: RFR [15] 8247696: Incorrect tail computation for large segments in AbstractMemorySegmentImpl::mismatch In-Reply-To: References: <0B15B6AF-F196-4192-B448-F1955AE586A7@oracle.com> <94fceeda-edee-c571-b32d-af8d5aed672e@oracle.com> <7BA53832-4D14-405D-9AE0-118ABB1344A2@oracle.com> Message-ID: +1 Paul. > On Jun 22, 2020, at 10:36 AM, Chris Hegarty wrote: > > Paul pointed out an issue, off list, where subsequent subranges > could still result in a call to the intrinsic. That is now fixed in-place > and the test amended to cover the scenario. > > https://cr.openjdk.java.net/~chegar/8247696/webrev.01/ > > -Chris. > >> On 19 Jun 2020, at 11:56, Chris Hegarty > wrote: >> >> Paul, Maurizio, >> >> This version incorporates all feedback so far. >> >> https://cr.openjdk.java.net/~chegar/8247696/webrev.01/ >> Results on my machine: >> >> Benchmark Mode Cnt Score Error Units >> BulkOps.mismatch_large_bytebuffer avgt 30 88266.728 ? 4083.476 ns/op >> BulkOps.mismatch_large_segment avgt 30 86141.343 ? 2450.450 ns/op >> BulkOps.mismatch_small_bytebuffer avgt 30 6.360 ? 0.425 ns/op >> BulkOps.mismatch_small_segment avgt 30 4.582 ? 1.040 ns/op >> >> -Chris. >> >>> On 19 Jun 2020, at 00:35, Paul Sandoz wrote: >>> >>> Thanks Chris. >>> >>>> On Jun 18, 2020, at 2:57 AM, Maurizio Cimadamore wrote: >>>> >>>> Thanks for looking at this Chris >>>> >>>> On 17/06/2020 21:56, Paul Sandoz wrote: >>>>> Hi Chris, >>>>> >>>>> AbstractMemorySegmentImpl >>>>> ? >>>>> >>>>> In vectorizedMismatchLarge: >>>>> >>>>> 163 if (remaining > 7) >>>>> 164 throw new InternalError("remaining greater than 7: " + remaining); >>>>> 165 i = length - remaining; >>>>> 166 } >>>>> >>>>> Should this check be an assert? >>>> I suggested that to Chris, since sometimes asserts are enabled when running tests, sometimes are not - in langtools we moved away from using asserts as we realized that in certain cases they were silently failing. I'm ok with whatever standard you feel comfortable with though. >>> >>> Automated running of tests enable assertions (-ea -esa), perhaps the use is more commonly accepted in library code than compiler code. I would favor so in this case if necessary (sometimes they are avoided to reduce inline thresholds). >>> >>> >>>>> >>>>> ? >>>>> >>>>> This fix prompted me to think more deeply about the implementation of vectorizedMismatchLarge and its use of vectorizedMismatch. Sorry, I should have thought about this more throughly earlier on. >>>>> >>>>> We need to refine the approach, not for this patch, but something to follow up after. I think there are two issues. >>>>> >>>>> 1) The intrinsic method vectorizedMismatch could potentially bomb out at any point and return the bitwise compliment of the remaining number of elements to check. >>>>> >>>>> Obviously, there is no point doing that arbitrarily but a stub implementation for, say, x86 AVX-512 might decide bomb out for < 64 remaining elements, rather than apply vector operations on smaller vector sizes or use a scalar loop. It does not today, but I think we should guard against that potentially happening, otherwise bad things can happen. >>>> >>>> So your worry here is that we'll end up with an infinite loop, right? >>>> >>> >>> Or more likely that an incorrect result is returned since tail elements will be skipped over as the offset and size is updated. >>> >>> >>>> If so, we could check remaining against previous remaining and bomb out too if no further progress seem to be made? >>>> >>> >>> I think it's better to always terminate the loop after the last sub-range is checked, rather than unnecessarily calling twice. >>> >>> I am not confident the vectorizedMismatch intrinsic stub has been tested properly on very small lengths, since it's never directly called in such cases, so also keeping "remaining > 7" is good too. >>> >>> Paul. >>> >>>>> >>>>> I think the loop should exit when the last sub-range has been checked. We should rely on other tests to ensure the intrinsic method is operating efficiently. >>>>> >>>>> >>>>> 2) This method only works when operating on byte arrays. It will not work correctly if operating on short or long arrays, since we are not adjusting the length and offsets accordingly by the scale. It's probably easier to just rename this as vectorizedMismatchLargeForBytes and drop the log2ArrayIndexScale argument. Then expand later if need be. I still think the method rightly belongs in ArraysSupport. >>>> >>>> Yep - probably good idea to restrict on bytes, for now. >>>> >>>> Maurizio >>>> >>>>> >>>>> Paul. >>>>> >>>>>> On Jun 17, 2020, at 8:33 AM, Chris Hegarty wrote: >>>>>> >>>>>> The MemorySegment::mismatch implementation added vectorized mismatch of long sizes. The implementation is trivial, but the starting point for a more optimized implementation, if needed. ArraysSupport::vectorizedMismatchLarge incorrectly returns the bitwise complement of the offset of the first mismatch, where it should return the bitwise complement of the number of remaining pairs of elements to be checked in the tail of the two arrays. The AbstractMemorySegmentImpl::mismatch masked this problem, since it seamlessly compared the remaining tail, which is larger than it should be. >>>>>> >>>>>> Webrev: >>>>>> https://cr.openjdk.java.net/~chegar/8247696/webrev.00/ >>>>>> >>>>>> I updated the exiting BulkOps micro-benchmark to cover mismatch. Here are the results, compared to ByteBuffer::mismatch, on my machine: >>>>>> >>>>>> Benchmark Mode Cnt Score Error Units >>>>>> BulkOps.mismatch_large_bytebuffer avgt 30 740186.973 ? 119314.207 ns/op >>>>>> BulkOps.mismatch_large_segment avgt 30 683475.305 ? 76355.043 ns/op >>>>>> BulkOps.mismatch_small_bytebuffer avgt 30 7.367 ? 0.523 ns/op >>>>>> BulkOps.mismatch_small_segment avgt 30 4.140 ? 0.602 ns/op >>>>>> >>>>>> >>>>>> -Chris. >> > From sebastian.stenzel at gmail.com Mon Jun 22 12:13:28 2020 From: sebastian.stenzel at gmail.com (Sebastian Stenzel) Date: Mon, 22 Jun 2020 14:13:28 +0200 Subject: Undocumented exceptions in java.net.http.HttpClient.newHttpClient() Message-ID: <6B17C468-9D85-41BA-85C5-2FD598C40358@gmail.com> Certain users of my software run into problems with HttpClient.newHttpClient() on JDK 14.0.1 and I don't feel like I can handle it properly without catching Errors. Calling said method fails when encountering Selector.open(), which in its Windows-specific implementation relies on loopback-TCP connections. These connections can, of course, be blocked by the OS or, as in the case of my user, by some anti-malware tool. I can't argue whether it is good or bad implementing the selector the way it is, however I'm pretty sure that commenting exceptions being "unlikely" without further explanation isn't exactly the best practice for this case: https://github.com/openjdk/jdk/blob/5adfaa39866f3127000f0779158c65afe1d24007/src/java.net.http/share/classes/jdk/internal/net/http/HttpClientImpl.java#L309-L314 Since my related bug report (JDK-8247996) was turned down (maybe the tester forgot to or chose not to block the socket connection), I'd like to ask if it is at least possible to document the fact that creating an HttpClient can have the side effect of instantaneously creating a loopback connection. I would prefer to even throw a checked exception, but this would break the API. Happy to create a PR, but I'd like to do this the right way, therefore starting this discussion. From Alan.Bateman at oracle.com Mon Jun 22 18:11:08 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 22 Jun 2020 19:11:08 +0100 Subject: RFR 8245302: Upgrade LogRecord to support long thread ids and remove its usage of ThreadLocal In-Reply-To: <362dc841-a809-95f5-961f-3856f3230b6f@oracle.com> References: <94B2555B-981F-40EB-AB5F-31E2BE068FF0@oracle.com> <03c38ee2-24c2-cf2d-e428-3f8747888978@oracle.com> <5e5bc99f-7d46-df1a-08c1-ee37958a3462@oracle.com> <58cda3b5-b75b-6782-01eb-2c935a0e402d@oracle.com> <362dc841-a809-95f5-961f-3856f3230b6f@oracle.com> Message-ID: <8a349484-7aed-a9e2-38b0-60b0aa01a29b@oracle.com> On 19/06/2020 10:00, Rahul Yadav wrote: > Thank you Alan, updated webrev. > > webrev : > http://cr.openjdk.java.net/~ryadav/webrev_8245302/webrev.00/index.html > You've addressed the issues that I pointed out, I don't think I have any other comments. -Alan From paul.sandoz at oracle.com Mon Jun 22 18:22:01 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 22 Jun 2020 11:22:01 -0700 Subject: RFR (15) 8248011: Improve javadoc of Foreign Memory Access API In-Reply-To: <891a53cb-9d44-628e-d720-c346e1365289@oracle.com> References: <891a53cb-9d44-628e-d720-c346e1365289@oracle.com> Message-ID: Looks good, some very minor comment (no need for another round). MemoryLayout.java ? 146 * That is, the above declaration is identical to the following, more verbose one: 147 *

      {@code
       148 MemoryLayout taggedValuesWithHole = MemoryLayout.ofSequence(5,
       149     MemoryLayout.ofStruct(
       150         MemoryLayout.ofPaddingBits(8, ByteOrder.NATIVE_ORDER).withName("kind").
       151         MemoryLayout.ofPaddingBits(32),
       152         MemoryLayout.ofPaddingBits(32)
       153 ));
      
      
      Line 150 should be the following?
      
        MemoryLayout.ofValueBits(8, ByteOrder.NATIVE_ORDER).withName("kind"),
      
      
      MemorySegment.java
      ?
      
       156  * 
      {@code
       157 MemorySegment segment = ...
       158 SequenceLayout SEQUENCE_LAYOUT = MemoryLayout.ofSequence(1024, MemoryLayouts.JAVA_INT);
       159 VarHandle VH_int = SEQUENCE_LAYOUT.elementLayout().varHandle(int.class);
       160 int sum = StreamSupport.stream(MemorySegment.spliterator(segment, SEQUENCE_LAYOUT), true)
       161 .mapToInt(s -> (int)VH_int.get(s.baseAddress()))
       162 .sum();
       163  * }
      Indent lines 161 and 162? package-info.java ? 41 *
      {@code
        42 static final VarHandle intHandle = MemoryHandles.varHandle(int.class, ByteOrder.nativeOrder());
        43 
        44 try (MemorySegment segment = MemorySegment.allocateNative(10 * 4)) {
        45    MemoryAddress base = segment.baseAddress();
        46    for (long i = 0 ; i < 10 ; i++) {
        47      intHandle.set(base.addOffset(i * 4), (int)i);
        48    }
        49  }
        50  * }
      Alignment of ?}? in line 49 off by 1 space (too many). Paul. > On Jun 22, 2020, at 7:41 AM, Maurizio Cimadamore wrote: > > Hi, > this is a patch fix address several javadoc issues in the foreign memory API, following some internal reviews (mostly from Brian, thanks!). > > The changes are aimed at reordering sections in the doc so that more advanced concepts are exposed last, and also to fix various issues in code samples (especially around MemoryLayout). > > Since I was there I also took the opportunity of dropping a now-unused method in MemoryAddressImpl (copy). > > Webrev: > http://cr.openjdk.java.net/~mcimadamore/8248011/ > > Cheers > Maurizio > > From maurizio.cimadamore at oracle.com Mon Jun 22 18:26:45 2020 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Mon, 22 Jun 2020 19:26:45 +0100 Subject: RFR (15) 8248011: Improve javadoc of Foreign Memory Access API In-Reply-To: References: <891a53cb-9d44-628e-d720-c346e1365289@oracle.com> Message-ID: On 22/06/2020 19:22, Paul Sandoz wrote: > Looks good, some very minor comment (no need for another round). > > MemoryLayout.java > ? > > 146 * That is, the above declaration is identical to the following, more verbose one: > 147 *
      {@code
      >   148 MemoryLayout taggedValuesWithHole = MemoryLayout.ofSequence(5,
      >   149     MemoryLayout.ofStruct(
      >   150         MemoryLayout.ofPaddingBits(8, ByteOrder.NATIVE_ORDER).withName("kind").
      >   151         MemoryLayout.ofPaddingBits(32),
      >   152         MemoryLayout.ofPaddingBits(32)
      >   153 ));
      >
      >
      > Line 150 should be the following?
      >
      >    MemoryLayout.ofValueBits(8, ByteOrder.NATIVE_ORDER).withName("kind"),
      Whoops yes
      >
      >
      > MemorySegment.java
      > ?
      >
      >   156  * 
      {@code
      >   157 MemorySegment segment = ...
      >   158 SequenceLayout SEQUENCE_LAYOUT = MemoryLayout.ofSequence(1024, MemoryLayouts.JAVA_INT);
      >   159 VarHandle VH_int = SEQUENCE_LAYOUT.elementLayout().varHandle(int.class);
      >   160 int sum = StreamSupport.stream(MemorySegment.spliterator(segment, SEQUENCE_LAYOUT), true)
      >   161 .mapToInt(s -> (int)VH_int.get(s.baseAddress()))
      >   162 .sum();
      >   163  * }
      > > Indent lines 161 and 162? ok > > > package-info.java > ? > > 41 *
      {@code
      >    42 static final VarHandle intHandle = MemoryHandles.varHandle(int.class, ByteOrder.nativeOrder());
      >    43
      >    44 try (MemorySegment segment = MemorySegment.allocateNative(10 * 4)) {
      >    45    MemoryAddress base = segment.baseAddress();
      >    46    for (long i = 0 ; i < 10 ; i++) {
      >    47      intHandle.set(base.addOffset(i * 4), (int)i);
      >    48    }
      >    49  }
      >    50  * }
      > > Alignment of ?}? in line 49 off by 1 space (too many). I'll fix that Thanks Maurizio > > Paul. > > >> On Jun 22, 2020, at 7:41 AM, Maurizio Cimadamore wrote: >> >> Hi, >> this is a patch fix address several javadoc issues in the foreign memory API, following some internal reviews (mostly from Brian, thanks!). >> >> The changes are aimed at reordering sections in the doc so that more advanced concepts are exposed last, and also to fix various issues in code samples (especially around MemoryLayout). >> >> Since I was there I also took the opportunity of dropping a now-unused method in MemoryAddressImpl (copy). >> >> Webrev: >> http://cr.openjdk.java.net/~mcimadamore/8248011/ >> >> Cheers >> Maurizio >> >> From daniel.fuchs at oracle.com Mon Jun 22 18:38:39 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Mon, 22 Jun 2020 19:38:39 +0100 Subject: RFR 8245302: Upgrade LogRecord to support long thread ids and remove its usage of ThreadLocal In-Reply-To: <362dc841-a809-95f5-961f-3856f3230b6f@oracle.com> References: <94B2555B-981F-40EB-AB5F-31E2BE068FF0@oracle.com> <03c38ee2-24c2-cf2d-e428-3f8747888978@oracle.com> <5e5bc99f-7d46-df1a-08c1-ee37958a3462@oracle.com> <58cda3b5-b75b-6782-01eb-2c935a0e402d@oracle.com> <362dc841-a809-95f5-961f-3856f3230b6f@oracle.com> Message-ID: Hi Rahul, Looks good to me as well. Reviewed. best regards, -- daniel On 19/06/2020 10:00, Rahul Yadav wrote: > Thank you Alan, updated webrev. > > webrev : > http://cr.openjdk.java.net/~ryadav/webrev_8245302/webrev.00/index.html > > - rahul > > On 19/06/2020 08:43, Alan Bateman wrote: >> On 18/06/2020 23:37, Rahul Yadav wrote: >>> Hi Alan, >>> >>> Thank you for the feedback.I have updated the webrev. >>> >>> webrev : >>> http://cr.openjdk.java.net/~ryadav/webrev_8245302/webrev.00/index.html >> This looks quite good. >> >> The comment in shortShortID has "any positive long less than >> Integer.MAX_VALUE" but it's actually <= MAX_VALUE. >> >> I don't think MIN_SEQUENTIAL_THREAD_ID is used so I assume it can be >> removed. >> >> The @return for setLongThreadID has a description "Log Record" but >> this should "this LogRecord". >> >> Can you update SerializeLogRecordTest with clear instructions on how >> to generate the stream? This will help future maintainers that may >> have to update this test. >> >> -Alan > From jonathan.gibbons at oracle.com Mon Jun 22 22:55:03 2020 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 22 Jun 2020 15:55:03 -0700 Subject: RFR; [docs,15] JDK-8248060 small HTML issues in java.xml package-info.java files Message-ID: <0fc54dcb-a97d-3cda-b393-24dd4f463427@oracle.com> Please review a couple of trivial fixes for a couple of issues in java.xml, reported by doclint. I realize we typically don't touch upstream code, but I'm hoping this is small enough to be reasonable. In the first case, there is an unnecessary `

      ` before a ``. In the second case, there's a bad `{@link newFactory}`. Not only is the syntax bad, but the method is defined in multiple classes in the same package, and overloaded as well: i.e. there is no easy/reasonable target for the link. The simple/localized fix is just to change `{@link}` to `{@code}`.? A better solution (for someone else to do) would be to rewrite the sentence altogether. -- Jon JBS: https://bugs.openjdk.java.net/browse/JDK-8248060 Patch: ?hg diff -R open open/src/java.xml diff -r 9cfa0137612f src/java.xml/share/classes/javax/xml/stream/package-info.java --- a/src/java.xml/share/classes/javax/xml/stream/package-info.java Mon Jun 22 13:37:41 2020 -0700 +++ b/src/java.xml/share/classes/javax/xml/stream/package-info.java Mon Jun 22 15:44:55 2020 -0700 @@ -41,8 +41,8 @@ ??*

      ??* StAX supports plugability with {@link XMLInputFactory} and ??* {@link XMLOutputFactory} that define how an implementation is - * located through a process as described in the {@link newFactory} - * method. + * located through a process as described in the {@code newFactory} + * methods. ??* ??* ??* @since 1.6 diff -r 9cfa0137612f src/java.xml/share/classes/org/xml/sax/package-info.java --- a/src/java.xml/share/classes/org/xml/sax/package-info.java Mon Jun 22 13:37:41 2020 -0700 +++ b/src/java.xml/share/classes/org/xml/sax/package-info.java Mon Jun 22 15:44:55 2020 -0700 @@ -27,7 +27,6 @@ ??* Provides the interfaces for the Simple API for XML (SAX). Supports both ??* the SAX1 and SAX2 APIs. ??* - *

      ??*

      SAX2 Standard Feature Flags

      ??* ??*

      From mikael.vidstedt at oracle.com Mon Jun 22 23:29:52 2020 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Mon, 22 Jun 2020 16:29:52 -0700 Subject: RFR: JDK-8213214: Set -Djava.io.tmpdir= when running tests In-Reply-To: References: Message-ID: The makefile changes look good to me! Cheers, Mikael > On Jun 16, 2020, at 12:22 PM, Erik Joelsson wrote: > > (re-sending this as it doesn't look like it was delivered) > > There are a lot of jtreg tests that use temporary files. These temporary files add up over time and fill up the global temp directories on our test systems. To tackle this, we should try to redirect these temporary files into a directory controlled by the test framework. Jtreg does not do this, but we can set -Djava.io.tmpdir from RunTest.gmk. This will not prevent all temp files from being created outside of the work dir, but at least most. > > > I have found one test where this becomes an issue, java/nio/file/Path/Misc.java on Windows when running in elevated mode with the workspace on a subst drive. This looks like an actual issue in the product, so I have filed a separate bug for it [1]. Since we currently use subst in our distributed test system to get around Windows path length issues, we are hitting this problem. While the bug is being evaluated, I have implemented a workaround in the test so that it is able to handle the known situation. I would like to have someone from core-libs look at the workaround. > > Webrev: http://cr.openjdk.java.net/~erikj/8213214/webrev.01/ > > Bug: https://bugs.openjdk.java.net/browse/JDK-8213214 > > /Erik > > [1] https://bugs.openjdk.java.net/browse/JDK-8213216 > > From peter.levart at gmail.com Tue Jun 23 07:49:50 2020 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 23 Jun 2020 09:49:50 +0200 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: <7f7b58fa-50e0-a4fe-f948-9637d04c0ccc@oracle.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> <1DA206A1-15F0-46F9-B69E-9896C57B97C7@oracle.com> <8886ee56-e9a7-aafe-6111-f0c2c54a282e@gmail.com> <63689945-dca3-e0cc-72b8-9225f5da6d31@gmail.com> <7f7b58fa-50e0-a4fe-f948-9637d04c0ccc@oracle.com> Message-ID: Hi Chris, Claes, Ok then, here's with benchmark included: http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.07/ I haven't been able to run the benchmark with "make test" though. I have tried various ways to pass javac options to build like: make test TEST='micro:org.openjdk.bench.java.io.RecordDeserialization' TEST_OPTS="VM_OPTIONS=--enable-preview --release=16" ...but javac doesn't seem to get them. Is there some secret option to achieve that? Otherwise, I simulated what would happen when there are more then 10 ObjectStreamClass layouts for same class rapidly interchanging, so that they push each other out of the cache, by temporarily setting cache's MAX_SIZE = 0. Note that this is worst case scenario: Benchmark (length) Mode Cnt Score Error Units RecordDeserializationBench.deserializeClasses 10 avgt 10 9.393 ? 0.287 us/op RecordDeserializationBench.deserializeClasses 100 avgt 10 35.642 ? 0.977 us/op RecordDeserializationBench.deserializeClasses 1000 avgt 10 293.769 ? 7.321 us/op RecordDeserializationBench.deserializeRecords 10 avgt 10 15.335 ? 0.496 us/op RecordDeserializationBench.deserializeRecords 100 avgt 10 211.427 ? 11.908 us/op RecordDeserializationBench.deserializeRecords 1000 avgt 10 990.398 ? 26.681 us/op This is using JMH option '-gc true' to force GC after each iteration of benchmark. Without it, I get a big (~4s) full-GC pause just in the middle of a run with length=100: Iteration?? 1: 528.577 us/op Iteration?? 2: 580.404 us/op Iteration?? 3: 4438.228 us/op Iteration?? 4: 644.532 us/op Iteration?? 5: 698.493 us/op Iteration?? 6: 800.738 us/op Iteration?? 7: 929.791 us/op Iteration?? 8: 870.946 us/op Iteration?? 9: 863.416 us/op Iteration? 10: 916.508 us/op ...so results are a bit off because of that: Benchmark????????????????????????????????????? (length)? Mode? Cnt???? Score????? Error? Units RecordDeserializationBench.deserializeClasses??????? 10? avgt?? 10???? 8.263 ???? 0.043? us/op RecordDeserializationBench.deserializeClasses?????? 100? avgt?? 10??? 33.406 ???? 0.160? us/op RecordDeserializationBench.deserializeClasses????? 1000? avgt?? 10?? 287.595 ???? 0.960? us/op RecordDeserializationBench.deserializeRecords??????? 10? avgt?? 10??? 15.270 ???? 0.080? us/op RecordDeserializationBench.deserializeRecords?????? 100? avgt?? 10? 1127.163 ? 1771.892? us/op RecordDeserializationBench.deserializeRecords????? 1000? avgt?? 10? 2003.235 ?? 227.159? us/op Yes, there is quite a bit of GCing going on when cache is thrashing. Note that I haven't tuned GC in any way and I'm running this on a machine with 64GiB of RAM so heap is allowed to grow quite big and G1 is used by default I think. This is still no worse than before the patch: Benchmark (length) Mode Cnt Score Error Units RecordDeserialization.deserializeClasses 10 avgt 10 8.382 : 0.013 us/op RecordDeserialization.deserializeClasses 100 avgt 10 33.736 : 0.171 us/op RecordDeserialization.deserializeClasses 1000 avgt 10 271.224 : 0.953 us/op RecordDeserialization.deserializeRecords 10 avgt 10 58.606 : 0.446 us/op RecordDeserialization.deserializeRecords 100 avgt 10 530.044 : 1.752 us/op RecordDeserialization.deserializeRecords 1000 avgt 10 5335.624 : 44.942 us/op ... since caching of adapted method handle for multiple objects withing single stream is still effective. I doubt there will ever be more than 10 variants/versions of the same record class deserialized by the same VM and in rapid succession, so I think this should not be an issue. We could add a system property to control the MAX_SIZE of cache if you think it is needed. Regards, Peter On 6/22/20 1:04 AM, Claes Redestad wrote: > Hi Peter, > > patch and results look great! > > My only real comment on this is that I think the microbenchmark would be > a valuable contribution, too. > > It'd also be interesting to explore how poor performance would become if > we'd hit the (artificial) 11 layouts limit, e.g, by cycling through > 10, 11, or 12 different shapes. > > /Claes > > On 2020-06-21 19:16, Peter Levart wrote: >> Hi, >> >> >> When re-running the benchmark [1] with different lengths of >> serialized arrays of records, I found that, compared to classical >> classes, lookup into the cache of adapted method handles starts to >> show when the length of array is larger (# of instances of same >> record type deserialized in single stream). Each record deserialized >> must lookup the method handle in a ConcurrentHashMap: >> >> >> Benchmark??????????????????????????????????? (length)? Mode Cnt??? >> Score?? Error? Units >> RecordSerializationBench.deserializeClasses??????? 10? avgt 10??? >> 8.088 ? 0.013? us/op >> RecordSerializationBench.deserializeClasses?????? 100? avgt 10?? >> 32.171 ? 0.324? us/op >> RecordSerializationBench.deserializeClasses????? 1000? avgt 10? >> 279.762 ? 3.072? us/op >> RecordSerializationBench.deserializeRecords??????? 10? avgt 10??? >> 9.011 ? 0.027? us/op >> RecordSerializationBench.deserializeRecords?????? 100? avgt 10?? >> 33.206 ? 0.514? us/op >> RecordSerializationBench.deserializeRecords????? 1000? avgt 10? >> 325.137 ? 0.969? us/op >> >> >> ...so keeping the correctly shaped adapted method handle in the >> per-serialization-session ObjectStreamClass instance [2] starts to >> make sense: >> >> >> Benchmark??????????????????????????????????? (length)? Mode Cnt??? >> Score?? Error? Units >> RecordSerializationBench.deserializeClasses??????? 10? avgt 10??? >> 8.681 ? 0.155? us/op >> RecordSerializationBench.deserializeClasses?????? 100? avgt 10?? >> 32.496 ? 0.087? us/op >> RecordSerializationBench.deserializeClasses????? 1000? avgt 10? >> 279.014 ? 1.189? us/op >> RecordSerializationBench.deserializeRecords??????? 10? avgt 10??? >> 8.537 ? 0.032? us/op >> RecordSerializationBench.deserializeRecords?????? 100? avgt 10?? >> 31.451 ? 0.083? us/op >> RecordSerializationBench.deserializeRecords????? 1000? avgt 10? >> 250.854 ? 2.772? us/op >> >> >> With that, more objects means advantage over classical classes >> instead of disadvantage. >> >> >> [1] >> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/RecordSerializationBench.java >> >> >> [2] >> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.06/ >> >> >> >> Regards, Peter >> >> From claes.redestad at oracle.com Tue Jun 23 08:06:30 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 23 Jun 2020 10:06:30 +0200 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> <1DA206A1-15F0-46F9-B69E-9896C57B97C7@oracle.com> <8886ee56-e9a7-aafe-6111-f0c2c54a282e@gmail.com> <63689945-dca3-e0cc-72b8-9225f5da6d31@gmail.com> <7f7b58fa-50e0-a4fe-f948-9637d04c0ccc@oracle.com> Message-ID: <5021b3e7-28ad-7041-cc89-c0f218434937@oracle.com> Hi, On 2020-06-23 09:49, Peter Levart wrote: > Hi Chris, Claes, > > > Ok then, here's with benchmark included: > > > http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.07/ > > > I haven't been able to run the benchmark with "make test" though. I have > tried various ways to pass javac options to build like: > > > make test TEST='micro:org.openjdk.bench.java.io.RecordDeserialization' TEST_OPTS="VM_OPTIONS=--enable-preview --release=16" > > > ...but javac doesn't seem to get them. Is there some secret option to > achieve that? Hmm, we might as well have the microbenchmarks build with --enable-preview on by default. Try this: diff -r 52741f85bf23 make/test/BuildMicrobenchmark.gmk --- a/make/test/BuildMicrobenchmark.gmk Tue Jun 23 09:54:42 2020 +0200 +++ b/make/test/BuildMicrobenchmark.gmk Tue Jun 23 09:59:29 2020 +0200 @@ -93,7 +93,7 @@ DISABLED_WARNINGS := processing rawtypes cast serial, \ SRC := $(MICROBENCHMARK_SRC), \ BIN := $(MICROBENCHMARK_CLASSES), \ - JAVA_FLAGS := --add-modules jdk.unsupported --limit-modules java.management, \ + JAVA_FLAGS := --enable-preview --add-modules jdk.unsupported --limit-modules java.management, \ )) $(BUILD_JDK_MICROBENCHMARK): $(JMH_COMPILE_JARS) > > > Otherwise, I simulated what would happen when there are more then 10 > ObjectStreamClass layouts for same class rapidly interchanging, so that > they push each other out of the cache, by temporarily setting cache's > MAX_SIZE = 0. Note that this is worst case scenario: > > > Benchmark (length) Mode Cnt Score Error Units > RecordDeserializationBench.deserializeClasses 10 avgt 10 9.393 ? 0.287 us/op > RecordDeserializationBench.deserializeClasses 100 avgt 10 35.642 ? 0.977 us/op > RecordDeserializationBench.deserializeClasses 1000 avgt 10 293.769 ? 7.321 us/op > RecordDeserializationBench.deserializeRecords 10 avgt 10 15.335 ? 0.496 us/op > RecordDeserializationBench.deserializeRecords 100 avgt 10 211.427 ? 11.908 us/op > RecordDeserializationBench.deserializeRecords 1000 avgt 10 990.398 ? 26.681 us/op > > > This is using JMH option '-gc true' to force GC after each iteration of > benchmark. Without it, I get a big (~4s) full-GC pause just in the > middle of a run with length=100: > > > Iteration?? 1: 528.577 us/op > Iteration?? 2: 580.404 us/op > Iteration?? 3: 4438.228 us/op > Iteration?? 4: 644.532 us/op > Iteration?? 5: 698.493 us/op > Iteration?? 6: 800.738 us/op > Iteration?? 7: 929.791 us/op > Iteration?? 8: 870.946 us/op > Iteration?? 9: 863.416 us/op > Iteration? 10: 916.508 us/op > > > ...so results are a bit off because of that: > > > Benchmark????????????????????????????????????? (length)? Mode? Cnt???? Score????? Error? Units > RecordDeserializationBench.deserializeClasses??????? 10? avgt?? 10???? 8.263 ???? 0.043? us/op > RecordDeserializationBench.deserializeClasses?????? 100? avgt?? 10??? 33.406 ???? 0.160? us/op > RecordDeserializationBench.deserializeClasses????? 1000? avgt?? 10?? 287.595 ???? 0.960? us/op > RecordDeserializationBench.deserializeRecords??????? 10? avgt?? 10??? 15.270 ???? 0.080? us/op > RecordDeserializationBench.deserializeRecords?????? 100? avgt?? 10? 1127.163 ? 1771.892? us/op > RecordDeserializationBench.deserializeRecords????? 1000? avgt?? 10? 2003.235 ?? 227.159? us/op > > > Yes, there is quite a bit of GCing going on when cache is thrashing. > Note that I haven't tuned GC in any way and I'm running this on a > machine with 64GiB of RAM so heap is allowed to grow quite big and G1 is > used by default I think. > > > This is still no worse than before the patch: > > > Benchmark (length) Mode Cnt Score Error Units > RecordDeserialization.deserializeClasses 10 avgt 10 8.382 : 0.013 us/op > RecordDeserialization.deserializeClasses 100 avgt 10 33.736 : 0.171 us/op > RecordDeserialization.deserializeClasses 1000 avgt 10 271.224 : 0.953 us/op > RecordDeserialization.deserializeRecords 10 avgt 10 58.606 : 0.446 us/op > RecordDeserialization.deserializeRecords 100 avgt 10 530.044 : 1.752 us/op > RecordDeserialization.deserializeRecords 1000 avgt 10 5335.624 : 44.942 us/op > > > ... since caching of adapted method handle for multiple objects withing > single stream is still effective. > > I doubt there will ever be more than 10 variants/versions of the same > record class deserialized by the same VM and in rapid succession, so I > think this should not be an issue. We could add a system property to > control the MAX_SIZE of cache if you think it is needed. Thanks for running the numbers on this, and I agree - it seems outlandishly improbable (most will only see one, and if you have to maintain 10+ different serialized shapes of some record you likely have bigger problems). I'd say let's keep it constant unless someone actually asks for it. /Claes > > > Regards, Peter > > > On 6/22/20 1:04 AM, Claes Redestad wrote: >> Hi Peter, >> >> patch and results look great! >> >> My only real comment on this is that I think the microbenchmark would be >> a valuable contribution, too. >> >> It'd also be interesting to explore how poor performance would become if >> we'd hit the (artificial) 11 layouts limit, e.g, by cycling through >> 10, 11, or 12 different shapes. >> >> /Claes >> >> On 2020-06-21 19:16, Peter Levart wrote: >>> Hi, >>> >>> >>> When re-running the benchmark [1] with different lengths of >>> serialized arrays of records, I found that, compared to classical >>> classes, lookup into the cache of adapted method handles starts to >>> show when the length of array is larger (# of instances of same >>> record type deserialized in single stream). Each record deserialized >>> must lookup the method handle in a ConcurrentHashMap: >>> >>> >>> Benchmark??????????????????????????????????? (length)? Mode Cnt >>> Score?? Error? Units >>> RecordSerializationBench.deserializeClasses??????? 10? avgt 10 >>> 8.088 ? 0.013? us/op >>> RecordSerializationBench.deserializeClasses?????? 100? avgt 10 >>> 32.171 ? 0.324? us/op >>> RecordSerializationBench.deserializeClasses????? 1000? avgt 10 >>> 279.762 ? 3.072? us/op >>> RecordSerializationBench.deserializeRecords??????? 10? avgt 10 >>> 9.011 ? 0.027? us/op >>> RecordSerializationBench.deserializeRecords?????? 100? avgt 10 >>> 33.206 ? 0.514? us/op >>> RecordSerializationBench.deserializeRecords????? 1000? avgt 10 >>> 325.137 ? 0.969? us/op >>> >>> >>> ...so keeping the correctly shaped adapted method handle in the >>> per-serialization-session ObjectStreamClass instance [2] starts to >>> make sense: >>> >>> >>> Benchmark??????????????????????????????????? (length)? Mode Cnt >>> Score?? Error? Units >>> RecordSerializationBench.deserializeClasses??????? 10? avgt 10 >>> 8.681 ? 0.155? us/op >>> RecordSerializationBench.deserializeClasses?????? 100? avgt 10 >>> 32.496 ? 0.087? us/op >>> RecordSerializationBench.deserializeClasses????? 1000? avgt 10 >>> 279.014 ? 1.189? us/op >>> RecordSerializationBench.deserializeRecords??????? 10? avgt 10 >>> 8.537 ? 0.032? us/op >>> RecordSerializationBench.deserializeRecords?????? 100? avgt 10 >>> 31.451 ? 0.083? us/op >>> RecordSerializationBench.deserializeRecords????? 1000? avgt 10 >>> 250.854 ? 2.772? us/op >>> >>> >>> With that, more objects means advantage over classical classes >>> instead of disadvantage. >>> >>> >>> [1] >>> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/RecordSerializationBench.java >>> >>> >>> [2] >>> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.06/ >>> >>> >>> >>> Regards, Peter >>> >>> From claes.redestad at oracle.com Tue Jun 23 08:23:10 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 23 Jun 2020 10:23:10 +0200 Subject: RFR: 8247532: Records deserialization is slow In-Reply-To: <5021b3e7-28ad-7041-cc89-c0f218434937@oracle.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> <1DA206A1-15F0-46F9-B69E-9896C57B97C7@oracle.com> <8886ee56-e9a7-aafe-6111-f0c2c54a282e@gmail.com> <63689945-dca3-e0cc-72b8-9225f5da6d31@gmail.com> <7f7b58fa-50e0-a4fe-f948-9637d04c0ccc@oracle.com> <5021b3e7-28ad-7041-cc89-c0f218434937@oracle.com> Message-ID: <092241fe-1f43-ead6-6a5d-0345dac55c3e@oracle.com> On 2020-06-23 10:06, Claes Redestad wrote: > Hi, > > On 2020-06-23 09:49, Peter Levart wrote: >> Hi Chris, Claes, >> >> >> Ok then, here's with benchmark included: >> >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.07/ >> >> >> >> I haven't been able to run the benchmark with "make test" though. I >> have tried various ways to pass javac options to build like: >> >> >> make test TEST='micro:org.openjdk.bench.java.io.RecordDeserialization' >> TEST_OPTS="VM_OPTIONS=--enable-preview --release=16" >> >> >> ...but javac doesn't seem to get them. Is there some secret option to >> achieve that? > > Hmm, we might as well have the microbenchmarks build with > --enable-preview on by default. Try this: Fixed: diff -r f2e1cd498381 make/test/BuildMicrobenchmark.gmk --- a/make/test/BuildMicrobenchmark.gmk Tue Jun 23 10:08:35 2020 +0200 +++ b/make/test/BuildMicrobenchmark.gmk Tue Jun 23 10:33:17 2020 +0200 @@ -90,10 +90,11 @@ TARGET_RELEASE := $(TARGET_RELEASE_NEWJDK_UPGRADED), \ SMALL_JAVA := false, \ CLASSPATH := $(MICROBENCHMARK_CLASSPATH), \ - DISABLED_WARNINGS := processing rawtypes cast serial, \ + DISABLED_WARNINGS := processing rawtypes cast serial preview, \ SRC := $(MICROBENCHMARK_SRC), \ BIN := $(MICROBENCHMARK_CLASSES), \ JAVA_FLAGS := --add-modules jdk.unsupported --limit-modules java.management, \ + JAVAC_FLAGS := --enable-preview, \ )) $(BUILD_JDK_MICROBENCHMARK): $(JMH_COMPILE_JARS) I verified this works with your micro, and doesn't seem to cause any issues elsewhere: make test TEST=micro:RecordDeserialization I can shepherd this as a separate fix for documentation purposes, but feel free to include it in your patch and ping build-dev at .. /Claes > > diff -r 52741f85bf23 make/test/BuildMicrobenchmark.gmk > --- a/make/test/BuildMicrobenchmark.gmk??? Tue Jun 23 09:54:42 2020 +0200 > +++ b/make/test/BuildMicrobenchmark.gmk??? Tue Jun 23 09:59:29 2020 +0200 > @@ -93,7 +93,7 @@ > ???? DISABLED_WARNINGS := processing rawtypes cast serial, \ > ???? SRC := $(MICROBENCHMARK_SRC), \ > ???? BIN := $(MICROBENCHMARK_CLASSES), \ > -??? JAVA_FLAGS := --add-modules jdk.unsupported --limit-modules > java.management, \ > +??? JAVA_FLAGS := --enable-preview --add-modules jdk.unsupported > --limit-modules java.management, \ > ?)) > > ?$(BUILD_JDK_MICROBENCHMARK): $(JMH_COMPILE_JARS) > >> >> >> Otherwise, I simulated what would happen when there are more then 10 >> ObjectStreamClass layouts for same class rapidly interchanging, so >> that they push each other out of the cache, by temporarily setting >> cache's MAX_SIZE = 0. Note that this is worst case scenario: >> >> >> Benchmark????????????????????????????????????? (length)? Mode? Cnt >> Score??? Error? Units >> RecordDeserializationBench.deserializeClasses??????? 10? avgt?? 10 >> 9.393 ?? 0.287? us/op >> RecordDeserializationBench.deserializeClasses?????? 100? avgt?? 10 >> 35.642 ?? 0.977? us/op >> RecordDeserializationBench.deserializeClasses????? 1000? avgt?? 10 >> 293.769 ?? 7.321? us/op >> RecordDeserializationBench.deserializeRecords??????? 10? avgt?? 10 >> 15.335 ?? 0.496? us/op >> RecordDeserializationBench.deserializeRecords?????? 100? avgt?? 10 >> 211.427 ? 11.908? us/op >> RecordDeserializationBench.deserializeRecords????? 1000? avgt?? 10 >> 990.398 ? 26.681? us/op >> >> >> This is using JMH option '-gc true' to force GC after each iteration >> of benchmark. Without it, I get a big (~4s) full-GC pause just in the >> middle of a run with length=100: >> >> >> Iteration?? 1: 528.577 us/op >> Iteration?? 2: 580.404 us/op >> Iteration?? 3: 4438.228 us/op >> Iteration?? 4: 644.532 us/op >> Iteration?? 5: 698.493 us/op >> Iteration?? 6: 800.738 us/op >> Iteration?? 7: 929.791 us/op >> Iteration?? 8: 870.946 us/op >> Iteration?? 9: 863.416 us/op >> Iteration? 10: 916.508 us/op >> >> >> ...so results are a bit off because of that: >> >> >> Benchmark????????????????????????????????????? (length)? Mode? Cnt >> Score????? Error? Units >> RecordDeserializationBench.deserializeClasses??????? 10? avgt?? 10 >> 8.263 ???? 0.043? us/op >> RecordDeserializationBench.deserializeClasses?????? 100? avgt?? 10 >> 33.406 ???? 0.160? us/op >> RecordDeserializationBench.deserializeClasses????? 1000? avgt?? 10 >> 287.595 ???? 0.960? us/op >> RecordDeserializationBench.deserializeRecords??????? 10? avgt?? 10 >> 15.270 ???? 0.080? us/op >> RecordDeserializationBench.deserializeRecords?????? 100? avgt?? 10 >> 1127.163 ? 1771.892? us/op >> RecordDeserializationBench.deserializeRecords????? 1000? avgt?? 10 >> 2003.235 ?? 227.159? us/op >> >> >> Yes, there is quite a bit of GCing going on when cache is thrashing. >> Note that I haven't tuned GC in any way and I'm running this on a >> machine with 64GiB of RAM so heap is allowed to grow quite big and G1 >> is used by default I think. >> >> >> This is still no worse than before the patch: >> >> >> Benchmark???????????????????????????????? (length)? Mode? Cnt >> Score??? Error? Units >> RecordDeserialization.deserializeClasses??????? 10? avgt?? 10 >> 8.382 :? 0.013? us/op >> RecordDeserialization.deserializeClasses?????? 100? avgt?? 10 >> 33.736 :? 0.171? us/op >> RecordDeserialization.deserializeClasses????? 1000? avgt?? 10 >> 271.224 :? 0.953? us/op >> RecordDeserialization.deserializeRecords??????? 10? avgt?? 10 >> 58.606 :? 0.446? us/op >> RecordDeserialization.deserializeRecords?????? 100? avgt?? 10 >> 530.044 :? 1.752? us/op >> RecordDeserialization.deserializeRecords????? 1000? avgt?? 10 >> 5335.624 : 44.942? us/op >> >> >> ... since caching of adapted method handle for multiple objects >> withing single stream is still effective. >> >> I doubt there will ever be more than 10 variants/versions of the same >> record class deserialized by the same VM and in rapid succession, so I >> think this should not be an issue. We could add a system property to >> control the MAX_SIZE of cache if you think it is needed. > > Thanks for running the numbers on this, and I agree - it seems > outlandishly improbable (most will only see one, and if you have to > maintain 10+ different serialized shapes of some record you likely have > bigger problems). > > I'd say let's keep it constant unless someone actually asks for it. > > /Claes > >> >> >> Regards, Peter >> >> >> On 6/22/20 1:04 AM, Claes Redestad wrote: >>> Hi Peter, >>> >>> patch and results look great! >>> >>> My only real comment on this is that I think the microbenchmark would be >>> a valuable contribution, too. >>> >>> It'd also be interesting to explore how poor performance would become if >>> we'd hit the (artificial) 11 layouts limit, e.g, by cycling through >>> 10, 11, or 12 different shapes. >>> >>> /Claes >>> >>> On 2020-06-21 19:16, Peter Levart wrote: >>>> Hi, >>>> >>>> >>>> When re-running the benchmark [1] with different lengths of >>>> serialized arrays of records, I found that, compared to classical >>>> classes, lookup into the cache of adapted method handles starts to >>>> show when the length of array is larger (# of instances of same >>>> record type deserialized in single stream). Each record deserialized >>>> must lookup the method handle in a ConcurrentHashMap: >>>> >>>> >>>> Benchmark??????????????????????????????????? (length)? Mode Cnt >>>> Score?? Error? Units >>>> RecordSerializationBench.deserializeClasses??????? 10? avgt 10 8.088 >>>> ? 0.013? us/op >>>> RecordSerializationBench.deserializeClasses?????? 100? avgt 10 >>>> 32.171 ? 0.324? us/op >>>> RecordSerializationBench.deserializeClasses????? 1000? avgt 10 >>>> 279.762 ? 3.072? us/op >>>> RecordSerializationBench.deserializeRecords??????? 10? avgt 10 9.011 >>>> ? 0.027? us/op >>>> RecordSerializationBench.deserializeRecords?????? 100? avgt 10 >>>> 33.206 ? 0.514? us/op >>>> RecordSerializationBench.deserializeRecords????? 1000? avgt 10 >>>> 325.137 ? 0.969? us/op >>>> >>>> >>>> ...so keeping the correctly shaped adapted method handle in the >>>> per-serialization-session ObjectStreamClass instance [2] starts to >>>> make sense: >>>> >>>> >>>> Benchmark??????????????????????????????????? (length)? Mode Cnt >>>> Score?? Error? Units >>>> RecordSerializationBench.deserializeClasses??????? 10? avgt 10 8.681 >>>> ? 0.155? us/op >>>> RecordSerializationBench.deserializeClasses?????? 100? avgt 10 >>>> 32.496 ? 0.087? us/op >>>> RecordSerializationBench.deserializeClasses????? 1000? avgt 10 >>>> 279.014 ? 1.189? us/op >>>> RecordSerializationBench.deserializeRecords??????? 10? avgt 10 8.537 >>>> ? 0.032? us/op >>>> RecordSerializationBench.deserializeRecords?????? 100? avgt 10 >>>> 31.451 ? 0.083? us/op >>>> RecordSerializationBench.deserializeRecords????? 1000? avgt 10 >>>> 250.854 ? 2.772? us/op >>>> >>>> >>>> With that, more objects means advantage over classical classes >>>> instead of disadvantage. >>>> >>>> >>>> [1] >>>> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/RecordSerializationBench.java >>>> >>>> >>>> [2] >>>> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.06/ >>>> >>>> >>>> >>>> Regards, Peter >>>> >>>> From claes.redestad at oracle.com Tue Jun 23 08:56:14 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 23 Jun 2020 10:56:14 +0200 Subject: RFR: 8248131: Simplify ServicesCatalog provider handling Message-ID: <371cde4c-a098-1677-7fbc-4a5026c57144@oracle.com> Hi, the current implementation of ServicesCatalog uses an internal providers method, which is only used to add ServiceProviders. Providing an addProviders method instead means we can streamline this, to the tune of a very small startup win. Bug: https://bugs.openjdk.java.net/browse/JDK-8248131 Webrev: http://cr.openjdk.java.net/~redestad/8248131/open.00/ Testing: tier1 Thanks! /Claes From peter.levart at gmail.com Tue Jun 23 09:17:37 2020 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 23 Jun 2020 11:17:37 +0200 Subject: RFR: 8247532, 8248135: Records deserialization is slow + Build microbenchmarks with --enable-preview In-Reply-To: <092241fe-1f43-ead6-6a5d-0345dac55c3e@oracle.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> <1DA206A1-15F0-46F9-B69E-9896C57B97C7@oracle.com> <8886ee56-e9a7-aafe-6111-f0c2c54a282e@gmail.com> <63689945-dca3-e0cc-72b8-9225f5da6d31@gmail.com> <7f7b58fa-50e0-a4fe-f948-9637d04c0ccc@oracle.com> <5021b3e7-28ad-7041-cc89-c0f218434937@oracle.com> <092241fe-1f43-ead6-6a5d-0345dac55c3e@oracle.com> Message-ID: <9e4e54c8-05c9-09fd-67ee-98c91ac9aeb3@gmail.com> Including build-dev since this patch is adding new issue 8248135: https://bugs.openjdk.java.net/browse/JDK-8248135 So here's new webrev with a patch for building benchmarks with --enable-preview included: http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.08/ Regards, Peter On 6/23/20 10:23 AM, Claes Redestad wrote: > > > On 2020-06-23 10:06, Claes Redestad wrote: >> Hi, >> >> On 2020-06-23 09:49, Peter Levart wrote: >>> Hi Chris, Claes, >>> >>> >>> Ok then, here's with benchmark included: >>> >>> >>> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.07/ >>> >>> >>> >>> I haven't been able to run the benchmark with "make test" though. I >>> have tried various ways to pass javac options to build like: >>> >>> >>> make test >>> TEST='micro:org.openjdk.bench.java.io.RecordDeserialization' >>> TEST_OPTS="VM_OPTIONS=--enable-preview --release=16" >>> >>> >>> ...but javac doesn't seem to get them. Is there some secret option >>> to achieve that? >> >> Hmm, we might as well have the microbenchmarks build with >> --enable-preview on by default. Try this: > > Fixed: > > diff -r f2e1cd498381 make/test/BuildMicrobenchmark.gmk > --- a/make/test/BuildMicrobenchmark.gmk Tue Jun 23 10:08:35 2020 +0200 > +++ b/make/test/BuildMicrobenchmark.gmk Tue Jun 23 10:33:17 2020 +0200 > @@ -90,10 +90,11 @@ > ???? TARGET_RELEASE := $(TARGET_RELEASE_NEWJDK_UPGRADED), \ > ???? SMALL_JAVA := false, \ > ???? CLASSPATH := $(MICROBENCHMARK_CLASSPATH), \ > -??? DISABLED_WARNINGS := processing rawtypes cast serial, \ > +??? DISABLED_WARNINGS := processing rawtypes cast serial preview, \ > ???? SRC := $(MICROBENCHMARK_SRC), \ > ???? BIN := $(MICROBENCHMARK_CLASSES), \ > ???? JAVA_FLAGS := --add-modules jdk.unsupported --limit-modules > java.management, \ > +??? JAVAC_FLAGS := --enable-preview, \ > ?)) > > ?$(BUILD_JDK_MICROBENCHMARK): $(JMH_COMPILE_JARS) > > I verified this works with your micro, and doesn't seem to cause > any issues elsewhere: > > ?make test TEST=micro:RecordDeserialization > > I can shepherd this as a separate fix for documentation purposes, but > feel free to include it in your patch and ping build-dev at .. > > /Claes > >> >> diff -r 52741f85bf23 make/test/BuildMicrobenchmark.gmk >> --- a/make/test/BuildMicrobenchmark.gmk??? Tue Jun 23 09:54:42 2020 >> +0200 >> +++ b/make/test/BuildMicrobenchmark.gmk??? Tue Jun 23 09:59:29 2020 >> +0200 >> @@ -93,7 +93,7 @@ >> ????? DISABLED_WARNINGS := processing rawtypes cast serial, \ >> ????? SRC := $(MICROBENCHMARK_SRC), \ >> ????? BIN := $(MICROBENCHMARK_CLASSES), \ >> -??? JAVA_FLAGS := --add-modules jdk.unsupported --limit-modules >> java.management, \ >> +??? JAVA_FLAGS := --enable-preview --add-modules jdk.unsupported >> --limit-modules java.management, \ >> ??)) >> >> ??$(BUILD_JDK_MICROBENCHMARK): $(JMH_COMPILE_JARS) >> >>> >>> >>> Otherwise, I simulated what would happen when there are more then 10 >>> ObjectStreamClass layouts for same class rapidly interchanging, so >>> that they push each other out of the cache, by temporarily setting >>> cache's MAX_SIZE = 0. Note that this is worst case scenario: >>> >>> >>> Benchmark????????????????????????????????????? (length)? Mode Cnt??? >>> Score??? Error? Units >>> RecordDeserializationBench.deserializeClasses??????? 10 avgt?? 10??? >>> 9.393 ?? 0.287? us/op >>> RecordDeserializationBench.deserializeClasses?????? 100 avgt?? 10?? >>> 35.642 ?? 0.977? us/op >>> RecordDeserializationBench.deserializeClasses????? 1000 avgt?? 10? >>> 293.769 ?? 7.321? us/op >>> RecordDeserializationBench.deserializeRecords??????? 10 avgt?? 10?? >>> 15.335 ?? 0.496? us/op >>> RecordDeserializationBench.deserializeRecords?????? 100 avgt?? 10? >>> 211.427 ? 11.908? us/op >>> RecordDeserializationBench.deserializeRecords????? 1000 avgt?? 10? >>> 990.398 ? 26.681? us/op >>> >>> >>> This is using JMH option '-gc true' to force GC after each iteration >>> of benchmark. Without it, I get a big (~4s) full-GC pause just in >>> the middle of a run with length=100: >>> >>> >>> Iteration?? 1: 528.577 us/op >>> Iteration?? 2: 580.404 us/op >>> Iteration?? 3: 4438.228 us/op >>> Iteration?? 4: 644.532 us/op >>> Iteration?? 5: 698.493 us/op >>> Iteration?? 6: 800.738 us/op >>> Iteration?? 7: 929.791 us/op >>> Iteration?? 8: 870.946 us/op >>> Iteration?? 9: 863.416 us/op >>> Iteration? 10: 916.508 us/op >>> >>> >>> ...so results are a bit off because of that: >>> >>> >>> Benchmark????????????????????????????????????? (length)? Mode >>> Cnt???? Score????? Error? Units >>> RecordDeserializationBench.deserializeClasses??????? 10 avgt?? >>> 10???? 8.263 ???? 0.043? us/op >>> RecordDeserializationBench.deserializeClasses?????? 100 avgt?? 10??? >>> 33.406 ???? 0.160? us/op >>> RecordDeserializationBench.deserializeClasses????? 1000 avgt?? 10?? >>> 287.595 ???? 0.960? us/op >>> RecordDeserializationBench.deserializeRecords??????? 10 avgt?? 10??? >>> 15.270 ???? 0.080? us/op >>> RecordDeserializationBench.deserializeRecords?????? 100 avgt?? 10? >>> 1127.163 ? 1771.892? us/op >>> RecordDeserializationBench.deserializeRecords????? 1000 avgt?? 10? >>> 2003.235 ?? 227.159? us/op >>> >>> >>> Yes, there is quite a bit of GCing going on when cache is thrashing. >>> Note that I haven't tuned GC in any way and I'm running this on a >>> machine with 64GiB of RAM so heap is allowed to grow quite big and >>> G1 is used by default I think. >>> >>> >>> This is still no worse than before the patch: >>> >>> >>> Benchmark???????????????????????????????? (length)? Mode Cnt???? >>> Score??? Error? Units >>> RecordDeserialization.deserializeClasses??????? 10? avgt 10???? >>> 8.382 :? 0.013? us/op >>> RecordDeserialization.deserializeClasses?????? 100? avgt 10??? >>> 33.736 :? 0.171? us/op >>> RecordDeserialization.deserializeClasses????? 1000? avgt 10?? >>> 271.224 :? 0.953? us/op >>> RecordDeserialization.deserializeRecords??????? 10? avgt 10??? >>> 58.606 :? 0.446? us/op >>> RecordDeserialization.deserializeRecords?????? 100? avgt 10?? >>> 530.044 :? 1.752? us/op >>> RecordDeserialization.deserializeRecords????? 1000? avgt?? 10 >>> 5335.624 : 44.942? us/op >>> >>> >>> ... since caching of adapted method handle for multiple objects >>> withing single stream is still effective. >>> >>> I doubt there will ever be more than 10 variants/versions of the >>> same record class deserialized by the same VM and in rapid >>> succession, so I think this should not be an issue. We could add a >>> system property to control the MAX_SIZE of cache if you think it is >>> needed. >> >> Thanks for running the numbers on this, and I agree - it seems >> outlandishly improbable (most will only see one, and if you have to >> maintain 10+ different serialized shapes of some record you likely >> have bigger problems). >> >> I'd say let's keep it constant unless someone actually asks for it. >> >> /Claes >> >>> >>> >>> Regards, Peter >>> >>> >>> On 6/22/20 1:04 AM, Claes Redestad wrote: >>>> Hi Peter, >>>> >>>> patch and results look great! >>>> >>>> My only real comment on this is that I think the microbenchmark >>>> would be >>>> a valuable contribution, too. >>>> >>>> It'd also be interesting to explore how poor performance would >>>> become if >>>> we'd hit the (artificial) 11 layouts limit, e.g, by cycling through >>>> 10, 11, or 12 different shapes. >>>> >>>> /Claes >>>> >>>> On 2020-06-21 19:16, Peter Levart wrote: >>>>> Hi, >>>>> >>>>> >>>>> When re-running the benchmark [1] with different lengths of >>>>> serialized arrays of records, I found that, compared to classical >>>>> classes, lookup into the cache of adapted method handles starts to >>>>> show when the length of array is larger (# of instances of same >>>>> record type deserialized in single stream). Each record >>>>> deserialized must lookup the method handle in a ConcurrentHashMap: >>>>> >>>>> >>>>> Benchmark??????????????????????????????????? (length) Mode Cnt >>>>> Score?? Error? Units >>>>> RecordSerializationBench.deserializeClasses??????? 10 avgt 10 >>>>> 8.088 ? 0.013? us/op >>>>> RecordSerializationBench.deserializeClasses?????? 100 avgt 10 >>>>> 32.171 ? 0.324? us/op >>>>> RecordSerializationBench.deserializeClasses????? 1000 avgt 10 >>>>> 279.762 ? 3.072? us/op >>>>> RecordSerializationBench.deserializeRecords??????? 10 avgt 10 >>>>> 9.011 ? 0.027? us/op >>>>> RecordSerializationBench.deserializeRecords?????? 100 avgt 10 >>>>> 33.206 ? 0.514? us/op >>>>> RecordSerializationBench.deserializeRecords????? 1000 avgt 10 >>>>> 325.137 ? 0.969? us/op >>>>> >>>>> >>>>> ...so keeping the correctly shaped adapted method handle in the >>>>> per-serialization-session ObjectStreamClass instance [2] starts to >>>>> make sense: >>>>> >>>>> >>>>> Benchmark??????????????????????????????????? (length) Mode Cnt >>>>> Score?? Error? Units >>>>> RecordSerializationBench.deserializeClasses??????? 10 avgt 10 >>>>> 8.681 ? 0.155? us/op >>>>> RecordSerializationBench.deserializeClasses?????? 100 avgt 10 >>>>> 32.496 ? 0.087? us/op >>>>> RecordSerializationBench.deserializeClasses????? 1000 avgt 10 >>>>> 279.014 ? 1.189? us/op >>>>> RecordSerializationBench.deserializeRecords??????? 10 avgt 10 >>>>> 8.537 ? 0.032? us/op >>>>> RecordSerializationBench.deserializeRecords?????? 100 avgt 10 >>>>> 31.451 ? 0.083? us/op >>>>> RecordSerializationBench.deserializeRecords????? 1000 avgt 10 >>>>> 250.854 ? 2.772? us/op >>>>> >>>>> >>>>> With that, more objects means advantage over classical classes >>>>> instead of disadvantage. >>>>> >>>>> >>>>> [1] >>>>> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/RecordSerializationBench.java >>>>> >>>>> >>>>> [2] >>>>> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.06/ >>>>> >>>>> >>>>> >>>>> Regards, Peter >>>>> >>>>> From claes.redestad at oracle.com Tue Jun 23 09:27:41 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 23 Jun 2020 11:27:41 +0200 Subject: RFR: 8247532, 8248135: Records deserialization is slow + Build microbenchmarks with --enable-preview In-Reply-To: <9e4e54c8-05c9-09fd-67ee-98c91ac9aeb3@gmail.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> <1DA206A1-15F0-46F9-B69E-9896C57B97C7@oracle.com> <8886ee56-e9a7-aafe-6111-f0c2c54a282e@gmail.com> <63689945-dca3-e0cc-72b8-9225f5da6d31@gmail.com> <7f7b58fa-50e0-a4fe-f948-9637d04c0ccc@oracle.com> <5021b3e7-28ad-7041-cc89-c0f218434937@oracle.com> <092241fe-1f43-ead6-6a5d-0345dac55c3e@oracle.com> <9e4e54c8-05c9-09fd-67ee-98c91ac9aeb3@gmail.com> Message-ID: <57d4a6dc-3030-3c13-aa58-cfaa46c1e259@oracle.com> Looks good to me! /Claes On 2020-06-23 11:17, Peter Levart wrote: > Including build-dev since this patch is adding new issue 8248135: > > > https://bugs.openjdk.java.net/browse/JDK-8248135 > > > So here's new webrev with a patch for building benchmarks with > --enable-preview included: > > > http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.08/ > > > > Regards, Peter > > > On 6/23/20 10:23 AM, Claes Redestad wrote: >> >> >> On 2020-06-23 10:06, Claes Redestad wrote: >>> Hi, >>> >>> On 2020-06-23 09:49, Peter Levart wrote: >>>> Hi Chris, Claes, >>>> >>>> >>>> Ok then, here's with benchmark included: >>>> >>>> >>>> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.07/ >>>> >>>> >>>> >>>> I haven't been able to run the benchmark with "make test" though. I >>>> have tried various ways to pass javac options to build like: >>>> >>>> >>>> make test >>>> TEST='micro:org.openjdk.bench.java.io.RecordDeserialization' >>>> TEST_OPTS="VM_OPTIONS=--enable-preview --release=16" >>>> >>>> >>>> ...but javac doesn't seem to get them. Is there some secret option >>>> to achieve that? >>> >>> Hmm, we might as well have the microbenchmarks build with >>> --enable-preview on by default. Try this: >> >> Fixed: >> >> diff -r f2e1cd498381 make/test/BuildMicrobenchmark.gmk >> --- a/make/test/BuildMicrobenchmark.gmk Tue Jun 23 10:08:35 2020 +0200 >> +++ b/make/test/BuildMicrobenchmark.gmk Tue Jun 23 10:33:17 2020 +0200 >> @@ -90,10 +90,11 @@ >> ???? TARGET_RELEASE := $(TARGET_RELEASE_NEWJDK_UPGRADED), \ >> ???? SMALL_JAVA := false, \ >> ???? CLASSPATH := $(MICROBENCHMARK_CLASSPATH), \ >> -??? DISABLED_WARNINGS := processing rawtypes cast serial, \ >> +??? DISABLED_WARNINGS := processing rawtypes cast serial preview, \ >> ???? SRC := $(MICROBENCHMARK_SRC), \ >> ???? BIN := $(MICROBENCHMARK_CLASSES), \ >> ???? JAVA_FLAGS := --add-modules jdk.unsupported --limit-modules >> java.management, \ >> +??? JAVAC_FLAGS := --enable-preview, \ >> ?)) >> >> ?$(BUILD_JDK_MICROBENCHMARK): $(JMH_COMPILE_JARS) >> >> I verified this works with your micro, and doesn't seem to cause >> any issues elsewhere: >> >> ?make test TEST=micro:RecordDeserialization >> >> I can shepherd this as a separate fix for documentation purposes, but >> feel free to include it in your patch and ping build-dev at .. >> >> /Claes >> >>> >>> diff -r 52741f85bf23 make/test/BuildMicrobenchmark.gmk >>> --- a/make/test/BuildMicrobenchmark.gmk??? Tue Jun 23 09:54:42 2020 >>> +0200 >>> +++ b/make/test/BuildMicrobenchmark.gmk??? Tue Jun 23 09:59:29 2020 >>> +0200 >>> @@ -93,7 +93,7 @@ >>> ????? DISABLED_WARNINGS := processing rawtypes cast serial, \ >>> ????? SRC := $(MICROBENCHMARK_SRC), \ >>> ????? BIN := $(MICROBENCHMARK_CLASSES), \ >>> -??? JAVA_FLAGS := --add-modules jdk.unsupported --limit-modules >>> java.management, \ >>> +??? JAVA_FLAGS := --enable-preview --add-modules jdk.unsupported >>> --limit-modules java.management, \ >>> ??)) >>> >>> ??$(BUILD_JDK_MICROBENCHMARK): $(JMH_COMPILE_JARS) >>> >>>> >>>> >>>> Otherwise, I simulated what would happen when there are more then 10 >>>> ObjectStreamClass layouts for same class rapidly interchanging, so >>>> that they push each other out of the cache, by temporarily setting >>>> cache's MAX_SIZE = 0. Note that this is worst case scenario: >>>> >>>> >>>> Benchmark????????????????????????????????????? (length)? Mode Cnt >>>> Score??? Error? Units >>>> RecordDeserializationBench.deserializeClasses??????? 10 avgt?? 10 >>>> 9.393 ?? 0.287? us/op >>>> RecordDeserializationBench.deserializeClasses?????? 100 avgt?? 10 >>>> 35.642 ?? 0.977? us/op >>>> RecordDeserializationBench.deserializeClasses????? 1000 avgt?? 10 >>>> 293.769 ?? 7.321? us/op >>>> RecordDeserializationBench.deserializeRecords??????? 10 avgt?? 10 >>>> 15.335 ?? 0.496? us/op >>>> RecordDeserializationBench.deserializeRecords?????? 100 avgt?? 10 >>>> 211.427 ? 11.908? us/op >>>> RecordDeserializationBench.deserializeRecords????? 1000 avgt?? 10 >>>> 990.398 ? 26.681? us/op >>>> >>>> >>>> This is using JMH option '-gc true' to force GC after each iteration >>>> of benchmark. Without it, I get a big (~4s) full-GC pause just in >>>> the middle of a run with length=100: >>>> >>>> >>>> Iteration?? 1: 528.577 us/op >>>> Iteration?? 2: 580.404 us/op >>>> Iteration?? 3: 4438.228 us/op >>>> Iteration?? 4: 644.532 us/op >>>> Iteration?? 5: 698.493 us/op >>>> Iteration?? 6: 800.738 us/op >>>> Iteration?? 7: 929.791 us/op >>>> Iteration?? 8: 870.946 us/op >>>> Iteration?? 9: 863.416 us/op >>>> Iteration? 10: 916.508 us/op >>>> >>>> >>>> ...so results are a bit off because of that: >>>> >>>> >>>> Benchmark????????????????????????????????????? (length)? Mode >>>> Cnt???? Score????? Error? Units >>>> RecordDeserializationBench.deserializeClasses??????? 10 avgt 10 >>>> 8.263 ???? 0.043? us/op >>>> RecordDeserializationBench.deserializeClasses?????? 100 avgt?? 10 >>>> 33.406 ???? 0.160? us/op >>>> RecordDeserializationBench.deserializeClasses????? 1000 avgt?? 10 >>>> 287.595 ???? 0.960? us/op >>>> RecordDeserializationBench.deserializeRecords??????? 10 avgt?? 10 >>>> 15.270 ???? 0.080? us/op >>>> RecordDeserializationBench.deserializeRecords?????? 100 avgt?? 10 >>>> 1127.163 ? 1771.892? us/op >>>> RecordDeserializationBench.deserializeRecords????? 1000 avgt?? 10 >>>> 2003.235 ?? 227.159? us/op >>>> >>>> >>>> Yes, there is quite a bit of GCing going on when cache is thrashing. >>>> Note that I haven't tuned GC in any way and I'm running this on a >>>> machine with 64GiB of RAM so heap is allowed to grow quite big and >>>> G1 is used by default I think. >>>> >>>> >>>> This is still no worse than before the patch: >>>> >>>> >>>> Benchmark???????????????????????????????? (length)? Mode Cnt >>>> Score??? Error? Units >>>> RecordDeserialization.deserializeClasses??????? 10? avgt 10 8.382 : >>>> 0.013? us/op >>>> RecordDeserialization.deserializeClasses?????? 100? avgt 10 33.736 >>>> :? 0.171? us/op >>>> RecordDeserialization.deserializeClasses????? 1000? avgt 10 271.224 >>>> :? 0.953? us/op >>>> RecordDeserialization.deserializeRecords??????? 10? avgt 10 58.606 >>>> :? 0.446? us/op >>>> RecordDeserialization.deserializeRecords?????? 100? avgt 10 530.044 >>>> :? 1.752? us/op >>>> RecordDeserialization.deserializeRecords????? 1000? avgt?? 10 >>>> 5335.624 : 44.942? us/op >>>> >>>> >>>> ... since caching of adapted method handle for multiple objects >>>> withing single stream is still effective. >>>> >>>> I doubt there will ever be more than 10 variants/versions of the >>>> same record class deserialized by the same VM and in rapid >>>> succession, so I think this should not be an issue. We could add a >>>> system property to control the MAX_SIZE of cache if you think it is >>>> needed. >>> >>> Thanks for running the numbers on this, and I agree - it seems >>> outlandishly improbable (most will only see one, and if you have to >>> maintain 10+ different serialized shapes of some record you likely >>> have bigger problems). >>> >>> I'd say let's keep it constant unless someone actually asks for it. >>> >>> /Claes >>> >>>> >>>> >>>> Regards, Peter >>>> >>>> >>>> On 6/22/20 1:04 AM, Claes Redestad wrote: >>>>> Hi Peter, >>>>> >>>>> patch and results look great! >>>>> >>>>> My only real comment on this is that I think the microbenchmark >>>>> would be >>>>> a valuable contribution, too. >>>>> >>>>> It'd also be interesting to explore how poor performance would >>>>> become if >>>>> we'd hit the (artificial) 11 layouts limit, e.g, by cycling through >>>>> 10, 11, or 12 different shapes. >>>>> >>>>> /Claes >>>>> >>>>> On 2020-06-21 19:16, Peter Levart wrote: >>>>>> Hi, >>>>>> >>>>>> >>>>>> When re-running the benchmark [1] with different lengths of >>>>>> serialized arrays of records, I found that, compared to classical >>>>>> classes, lookup into the cache of adapted method handles starts to >>>>>> show when the length of array is larger (# of instances of same >>>>>> record type deserialized in single stream). Each record >>>>>> deserialized must lookup the method handle in a ConcurrentHashMap: >>>>>> >>>>>> >>>>>> Benchmark??????????????????????????????????? (length) Mode Cnt >>>>>> Score?? Error? Units >>>>>> RecordSerializationBench.deserializeClasses??????? 10 avgt 10 >>>>>> 8.088 ? 0.013? us/op >>>>>> RecordSerializationBench.deserializeClasses?????? 100 avgt 10 >>>>>> 32.171 ? 0.324? us/op >>>>>> RecordSerializationBench.deserializeClasses????? 1000 avgt 10 >>>>>> 279.762 ? 3.072? us/op >>>>>> RecordSerializationBench.deserializeRecords??????? 10 avgt 10 >>>>>> 9.011 ? 0.027? us/op >>>>>> RecordSerializationBench.deserializeRecords?????? 100 avgt 10 >>>>>> 33.206 ? 0.514? us/op >>>>>> RecordSerializationBench.deserializeRecords????? 1000 avgt 10 >>>>>> 325.137 ? 0.969? us/op >>>>>> >>>>>> >>>>>> ...so keeping the correctly shaped adapted method handle in the >>>>>> per-serialization-session ObjectStreamClass instance [2] starts to >>>>>> make sense: >>>>>> >>>>>> >>>>>> Benchmark??????????????????????????????????? (length) Mode Cnt >>>>>> Score?? Error? Units >>>>>> RecordSerializationBench.deserializeClasses??????? 10 avgt 10 >>>>>> 8.681 ? 0.155? us/op >>>>>> RecordSerializationBench.deserializeClasses?????? 100 avgt 10 >>>>>> 32.496 ? 0.087? us/op >>>>>> RecordSerializationBench.deserializeClasses????? 1000 avgt 10 >>>>>> 279.014 ? 1.189? us/op >>>>>> RecordSerializationBench.deserializeRecords??????? 10 avgt 10 >>>>>> 8.537 ? 0.032? us/op >>>>>> RecordSerializationBench.deserializeRecords?????? 100 avgt 10 >>>>>> 31.451 ? 0.083? us/op >>>>>> RecordSerializationBench.deserializeRecords????? 1000 avgt 10 >>>>>> 250.854 ? 2.772? us/op >>>>>> >>>>>> >>>>>> With that, more objects means advantage over classical classes >>>>>> instead of disadvantage. >>>>>> >>>>>> >>>>>> [1] >>>>>> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/RecordSerializationBench.java >>>>>> >>>>>> >>>>>> [2] >>>>>> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.06/ >>>>>> >>>>>> >>>>>> >>>>>> Regards, Peter >>>>>> >>>>>> From peter.levart at gmail.com Tue Jun 23 09:28:54 2020 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 23 Jun 2020 11:28:54 +0200 Subject: RFR: 8247532, 8248135: Records deserialization is slow + Build microbenchmarks with --enable-preview In-Reply-To: <2b37dc24-ed73-1d8b-4130-07f5122667c8@oracle.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> <1DA206A1-15F0-46F9-B69E-9896C57B97C7@oracle.com> <8886ee56-e9a7-aafe-6111-f0c2c54a282e@gmail.com> <63689945-dca3-e0cc-72b8-9225f5da6d31@gmail.com> <7f7b58fa-50e0-a4fe-f948-9637d04c0ccc@oracle.com> <5021b3e7-28ad-7041-cc89-c0f218434937@oracle.com> <092241fe-1f43-ead6-6a5d-0345dac55c3e@oracle.com> <9e4e54c8-05c9-09fd-67ee-98c91ac9aeb3@gmail.com> <2b37dc24-ed73-1d8b-4130-07f5122667c8@oracle.com> Message-ID: Thanks, Magnus. Peter On 6/23/20 11:27 AM, Magnus Ihse Bursie wrote: > > > On 2020-06-23 11:17, Peter Levart wrote: >> Including build-dev since this patch is adding new issue 8248135: >> >> >> https://bugs.openjdk.java.net/browse/JDK-8248135 >> >> >> So here's new webrev with a patch for building benchmarks with >> --enable-preview included: >> >> >> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.08/ >> > Build changes look good. > > /Magnus >> >> >> Regards, Peter >> >> >> On 6/23/20 10:23 AM, Claes Redestad wrote: >>> >>> >>> On 2020-06-23 10:06, Claes Redestad wrote: >>>> Hi, >>>> >>>> On 2020-06-23 09:49, Peter Levart wrote: >>>>> Hi Chris, Claes, >>>>> >>>>> >>>>> Ok then, here's with benchmark included: >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.07/ >>>>> >>>>> >>>>> >>>>> I haven't been able to run the benchmark with "make test" though. >>>>> I have tried various ways to pass javac options to build like: >>>>> >>>>> >>>>> make test >>>>> TEST='micro:org.openjdk.bench.java.io.RecordDeserialization' >>>>> TEST_OPTS="VM_OPTIONS=--enable-preview --release=16" >>>>> >>>>> >>>>> ...but javac doesn't seem to get them. Is there some secret option >>>>> to achieve that? >>>> >>>> Hmm, we might as well have the microbenchmarks build with >>>> --enable-preview on by default. Try this: >>> >>> Fixed: >>> >>> diff -r f2e1cd498381 make/test/BuildMicrobenchmark.gmk >>> --- a/make/test/BuildMicrobenchmark.gmk Tue Jun 23 10:08:35 2020 +0200 >>> +++ b/make/test/BuildMicrobenchmark.gmk Tue Jun 23 10:33:17 2020 +0200 >>> @@ -90,10 +90,11 @@ >>> ???? TARGET_RELEASE := $(TARGET_RELEASE_NEWJDK_UPGRADED), \ >>> ???? SMALL_JAVA := false, \ >>> ???? CLASSPATH := $(MICROBENCHMARK_CLASSPATH), \ >>> -??? DISABLED_WARNINGS := processing rawtypes cast serial, \ >>> +??? DISABLED_WARNINGS := processing rawtypes cast serial preview, \ >>> ???? SRC := $(MICROBENCHMARK_SRC), \ >>> ???? BIN := $(MICROBENCHMARK_CLASSES), \ >>> ???? JAVA_FLAGS := --add-modules jdk.unsupported --limit-modules >>> java.management, \ >>> +??? JAVAC_FLAGS := --enable-preview, \ >>> ?)) >>> >>> ?$(BUILD_JDK_MICROBENCHMARK): $(JMH_COMPILE_JARS) >>> >>> I verified this works with your micro, and doesn't seem to cause >>> any issues elsewhere: >>> >>> ?make test TEST=micro:RecordDeserialization >>> >>> I can shepherd this as a separate fix for documentation purposes, but >>> feel free to include it in your patch and ping build-dev at .. >>> >>> /Claes >>> >>>> >>>> diff -r 52741f85bf23 make/test/BuildMicrobenchmark.gmk >>>> --- a/make/test/BuildMicrobenchmark.gmk??? Tue Jun 23 09:54:42 2020 >>>> +0200 >>>> +++ b/make/test/BuildMicrobenchmark.gmk??? Tue Jun 23 09:59:29 2020 >>>> +0200 >>>> @@ -93,7 +93,7 @@ >>>> ????? DISABLED_WARNINGS := processing rawtypes cast serial, \ >>>> ????? SRC := $(MICROBENCHMARK_SRC), \ >>>> ????? BIN := $(MICROBENCHMARK_CLASSES), \ >>>> -??? JAVA_FLAGS := --add-modules jdk.unsupported --limit-modules >>>> java.management, \ >>>> +??? JAVA_FLAGS := --enable-preview --add-modules jdk.unsupported >>>> --limit-modules java.management, \ >>>> ??)) >>>> >>>> ??$(BUILD_JDK_MICROBENCHMARK): $(JMH_COMPILE_JARS) >>>> >>>>> >>>>> >>>>> Otherwise, I simulated what would happen when there are more then >>>>> 10 ObjectStreamClass layouts for same class rapidly interchanging, >>>>> so that they push each other out of the cache, by temporarily >>>>> setting cache's MAX_SIZE = 0. Note that this is worst case scenario: >>>>> >>>>> >>>>> Benchmark????????????????????????????????????? (length) Mode >>>>> Cnt??? Score??? Error? Units >>>>> RecordDeserializationBench.deserializeClasses??????? 10 avgt?? >>>>> 10??? 9.393 ?? 0.287? us/op >>>>> RecordDeserializationBench.deserializeClasses?????? 100 avgt?? >>>>> 10?? 35.642 ?? 0.977? us/op >>>>> RecordDeserializationBench.deserializeClasses????? 1000 avgt?? 10? >>>>> 293.769 ?? 7.321? us/op >>>>> RecordDeserializationBench.deserializeRecords??????? 10 avgt?? >>>>> 10?? 15.335 ?? 0.496? us/op >>>>> RecordDeserializationBench.deserializeRecords?????? 100 avgt?? 10? >>>>> 211.427 ? 11.908? us/op >>>>> RecordDeserializationBench.deserializeRecords????? 1000 avgt?? 10? >>>>> 990.398 ? 26.681? us/op >>>>> >>>>> >>>>> This is using JMH option '-gc true' to force GC after each >>>>> iteration of benchmark. Without it, I get a big (~4s) full-GC >>>>> pause just in the middle of a run with length=100: >>>>> >>>>> >>>>> Iteration?? 1: 528.577 us/op >>>>> Iteration?? 2: 580.404 us/op >>>>> Iteration?? 3: 4438.228 us/op >>>>> Iteration?? 4: 644.532 us/op >>>>> Iteration?? 5: 698.493 us/op >>>>> Iteration?? 6: 800.738 us/op >>>>> Iteration?? 7: 929.791 us/op >>>>> Iteration?? 8: 870.946 us/op >>>>> Iteration?? 9: 863.416 us/op >>>>> Iteration? 10: 916.508 us/op >>>>> >>>>> >>>>> ...so results are a bit off because of that: >>>>> >>>>> >>>>> Benchmark????????????????????????????????????? (length) Mode >>>>> Cnt???? Score????? Error? Units >>>>> RecordDeserializationBench.deserializeClasses??????? 10 avgt?? >>>>> 10???? 8.263 ???? 0.043? us/op >>>>> RecordDeserializationBench.deserializeClasses?????? 100 avgt?? >>>>> 10??? 33.406 ???? 0.160? us/op >>>>> RecordDeserializationBench.deserializeClasses????? 1000 avgt?? >>>>> 10?? 287.595 ???? 0.960? us/op >>>>> RecordDeserializationBench.deserializeRecords??????? 10 avgt?? >>>>> 10??? 15.270 ???? 0.080? us/op >>>>> RecordDeserializationBench.deserializeRecords?????? 100 avgt?? 10? >>>>> 1127.163 ? 1771.892? us/op >>>>> RecordDeserializationBench.deserializeRecords????? 1000 avgt?? 10? >>>>> 2003.235 ?? 227.159? us/op >>>>> >>>>> >>>>> Yes, there is quite a bit of GCing going on when cache is >>>>> thrashing. Note that I haven't tuned GC in any way and I'm running >>>>> this on a machine with 64GiB of RAM so heap is allowed to grow >>>>> quite big and G1 is used by default I think. >>>>> >>>>> >>>>> This is still no worse than before the patch: >>>>> >>>>> >>>>> Benchmark???????????????????????????????? (length)? Mode Cnt???? >>>>> Score??? Error? Units >>>>> RecordDeserialization.deserializeClasses??????? 10? avgt 10???? >>>>> 8.382 :? 0.013? us/op >>>>> RecordDeserialization.deserializeClasses?????? 100? avgt 10??? >>>>> 33.736 :? 0.171? us/op >>>>> RecordDeserialization.deserializeClasses????? 1000? avgt 10?? >>>>> 271.224 :? 0.953? us/op >>>>> RecordDeserialization.deserializeRecords??????? 10? avgt 10??? >>>>> 58.606 :? 0.446? us/op >>>>> RecordDeserialization.deserializeRecords?????? 100? avgt 10?? >>>>> 530.044 :? 1.752? us/op >>>>> RecordDeserialization.deserializeRecords????? 1000? avgt 10 >>>>> 5335.624 : 44.942? us/op >>>>> >>>>> >>>>> ... since caching of adapted method handle for multiple objects >>>>> withing single stream is still effective. >>>>> >>>>> I doubt there will ever be more than 10 variants/versions of the >>>>> same record class deserialized by the same VM and in rapid >>>>> succession, so I think this should not be an issue. We could add a >>>>> system property to control the MAX_SIZE of cache if you think it >>>>> is needed. >>>> >>>> Thanks for running the numbers on this, and I agree - it seems >>>> outlandishly improbable (most will only see one, and if you have to >>>> maintain 10+ different serialized shapes of some record you likely >>>> have bigger problems). >>>> >>>> I'd say let's keep it constant unless someone actually asks for it. >>>> >>>> /Claes >>>> >>>>> >>>>> >>>>> Regards, Peter >>>>> >>>>> >>>>> On 6/22/20 1:04 AM, Claes Redestad wrote: >>>>>> Hi Peter, >>>>>> >>>>>> patch and results look great! >>>>>> >>>>>> My only real comment on this is that I think the microbenchmark >>>>>> would be >>>>>> a valuable contribution, too. >>>>>> >>>>>> It'd also be interesting to explore how poor performance would >>>>>> become if >>>>>> we'd hit the (artificial) 11 layouts limit, e.g, by cycling through >>>>>> 10, 11, or 12 different shapes. >>>>>> >>>>>> /Claes >>>>>> >>>>>> On 2020-06-21 19:16, Peter Levart wrote: >>>>>>> Hi, >>>>>>> >>>>>>> >>>>>>> When re-running the benchmark [1] with different lengths of >>>>>>> serialized arrays of records, I found that, compared to >>>>>>> classical classes, lookup into the cache of adapted method >>>>>>> handles starts to show when the length of array is larger (# of >>>>>>> instances of same record type deserialized in single stream). >>>>>>> Each record deserialized must lookup the method handle in a >>>>>>> ConcurrentHashMap: >>>>>>> >>>>>>> >>>>>>> Benchmark??????????????????????????????????? (length) Mode Cnt >>>>>>> Score?? Error? Units >>>>>>> RecordSerializationBench.deserializeClasses??????? 10 avgt 10 >>>>>>> 8.088 ? 0.013? us/op >>>>>>> RecordSerializationBench.deserializeClasses?????? 100 avgt 10 >>>>>>> 32.171 ? 0.324? us/op >>>>>>> RecordSerializationBench.deserializeClasses????? 1000 avgt 10 >>>>>>> 279.762 ? 3.072? us/op >>>>>>> RecordSerializationBench.deserializeRecords??????? 10 avgt 10 >>>>>>> 9.011 ? 0.027? us/op >>>>>>> RecordSerializationBench.deserializeRecords?????? 100 avgt 10 >>>>>>> 33.206 ? 0.514? us/op >>>>>>> RecordSerializationBench.deserializeRecords????? 1000 avgt 10 >>>>>>> 325.137 ? 0.969? us/op >>>>>>> >>>>>>> >>>>>>> ...so keeping the correctly shaped adapted method handle in the >>>>>>> per-serialization-session ObjectStreamClass instance [2] starts >>>>>>> to make sense: >>>>>>> >>>>>>> >>>>>>> Benchmark??????????????????????????????????? (length) Mode Cnt >>>>>>> Score?? Error? Units >>>>>>> RecordSerializationBench.deserializeClasses??????? 10 avgt 10 >>>>>>> 8.681 ? 0.155? us/op >>>>>>> RecordSerializationBench.deserializeClasses?????? 100 avgt 10 >>>>>>> 32.496 ? 0.087? us/op >>>>>>> RecordSerializationBench.deserializeClasses????? 1000 avgt 10 >>>>>>> 279.014 ? 1.189? us/op >>>>>>> RecordSerializationBench.deserializeRecords??????? 10 avgt 10 >>>>>>> 8.537 ? 0.032? us/op >>>>>>> RecordSerializationBench.deserializeRecords?????? 100 avgt 10 >>>>>>> 31.451 ? 0.083? us/op >>>>>>> RecordSerializationBench.deserializeRecords????? 1000 avgt 10 >>>>>>> 250.854 ? 2.772? us/op >>>>>>> >>>>>>> >>>>>>> With that, more objects means advantage over classical classes >>>>>>> instead of disadvantage. >>>>>>> >>>>>>> >>>>>>> [1] >>>>>>> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/RecordSerializationBench.java >>>>>>> >>>>>>> >>>>>>> [2] >>>>>>> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.06/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> Regards, Peter >>>>>>> >>>>>>> > From magnus.ihse.bursie at oracle.com Tue Jun 23 09:27:36 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Tue, 23 Jun 2020 11:27:36 +0200 Subject: RFR: 8247532, 8248135: Records deserialization is slow + Build microbenchmarks with --enable-preview In-Reply-To: <9e4e54c8-05c9-09fd-67ee-98c91ac9aeb3@gmail.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> <1DA206A1-15F0-46F9-B69E-9896C57B97C7@oracle.com> <8886ee56-e9a7-aafe-6111-f0c2c54a282e@gmail.com> <63689945-dca3-e0cc-72b8-9225f5da6d31@gmail.com> <7f7b58fa-50e0-a4fe-f948-9637d04c0ccc@oracle.com> <5021b3e7-28ad-7041-cc89-c0f218434937@oracle.com> <092241fe-1f43-ead6-6a5d-0345dac55c3e@oracle.com> <9e4e54c8-05c9-09fd-67ee-98c91ac9aeb3@gmail.com> Message-ID: <2b37dc24-ed73-1d8b-4130-07f5122667c8@oracle.com> On 2020-06-23 11:17, Peter Levart wrote: > Including build-dev since this patch is adding new issue 8248135: > > > https://bugs.openjdk.java.net/browse/JDK-8248135 > > > So here's new webrev with a patch for building benchmarks with > --enable-preview included: > > > http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.08/ > Build changes look good. /Magnus > > > Regards, Peter > > > On 6/23/20 10:23 AM, Claes Redestad wrote: >> >> >> On 2020-06-23 10:06, Claes Redestad wrote: >>> Hi, >>> >>> On 2020-06-23 09:49, Peter Levart wrote: >>>> Hi Chris, Claes, >>>> >>>> >>>> Ok then, here's with benchmark included: >>>> >>>> >>>> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.07/ >>>> >>>> >>>> >>>> I haven't been able to run the benchmark with "make test" though. I >>>> have tried various ways to pass javac options to build like: >>>> >>>> >>>> make test >>>> TEST='micro:org.openjdk.bench.java.io.RecordDeserialization' >>>> TEST_OPTS="VM_OPTIONS=--enable-preview --release=16" >>>> >>>> >>>> ...but javac doesn't seem to get them. Is there some secret option >>>> to achieve that? >>> >>> Hmm, we might as well have the microbenchmarks build with >>> --enable-preview on by default. Try this: >> >> Fixed: >> >> diff -r f2e1cd498381 make/test/BuildMicrobenchmark.gmk >> --- a/make/test/BuildMicrobenchmark.gmk Tue Jun 23 10:08:35 2020 +0200 >> +++ b/make/test/BuildMicrobenchmark.gmk Tue Jun 23 10:33:17 2020 +0200 >> @@ -90,10 +90,11 @@ >> ???? TARGET_RELEASE := $(TARGET_RELEASE_NEWJDK_UPGRADED), \ >> ???? SMALL_JAVA := false, \ >> ???? CLASSPATH := $(MICROBENCHMARK_CLASSPATH), \ >> -??? DISABLED_WARNINGS := processing rawtypes cast serial, \ >> +??? DISABLED_WARNINGS := processing rawtypes cast serial preview, \ >> ???? SRC := $(MICROBENCHMARK_SRC), \ >> ???? BIN := $(MICROBENCHMARK_CLASSES), \ >> ???? JAVA_FLAGS := --add-modules jdk.unsupported --limit-modules >> java.management, \ >> +??? JAVAC_FLAGS := --enable-preview, \ >> ?)) >> >> ?$(BUILD_JDK_MICROBENCHMARK): $(JMH_COMPILE_JARS) >> >> I verified this works with your micro, and doesn't seem to cause >> any issues elsewhere: >> >> ?make test TEST=micro:RecordDeserialization >> >> I can shepherd this as a separate fix for documentation purposes, but >> feel free to include it in your patch and ping build-dev at .. >> >> /Claes >> >>> >>> diff -r 52741f85bf23 make/test/BuildMicrobenchmark.gmk >>> --- a/make/test/BuildMicrobenchmark.gmk??? Tue Jun 23 09:54:42 2020 >>> +0200 >>> +++ b/make/test/BuildMicrobenchmark.gmk??? Tue Jun 23 09:59:29 2020 >>> +0200 >>> @@ -93,7 +93,7 @@ >>> ????? DISABLED_WARNINGS := processing rawtypes cast serial, \ >>> ????? SRC := $(MICROBENCHMARK_SRC), \ >>> ????? BIN := $(MICROBENCHMARK_CLASSES), \ >>> -??? JAVA_FLAGS := --add-modules jdk.unsupported --limit-modules >>> java.management, \ >>> +??? JAVA_FLAGS := --enable-preview --add-modules jdk.unsupported >>> --limit-modules java.management, \ >>> ??)) >>> >>> ??$(BUILD_JDK_MICROBENCHMARK): $(JMH_COMPILE_JARS) >>> >>>> >>>> >>>> Otherwise, I simulated what would happen when there are more then >>>> 10 ObjectStreamClass layouts for same class rapidly interchanging, >>>> so that they push each other out of the cache, by temporarily >>>> setting cache's MAX_SIZE = 0. Note that this is worst case scenario: >>>> >>>> >>>> Benchmark????????????????????????????????????? (length) Mode Cnt??? >>>> Score??? Error? Units >>>> RecordDeserializationBench.deserializeClasses??????? 10 avgt?? >>>> 10??? 9.393 ?? 0.287? us/op >>>> RecordDeserializationBench.deserializeClasses?????? 100 avgt?? 10?? >>>> 35.642 ?? 0.977? us/op >>>> RecordDeserializationBench.deserializeClasses????? 1000 avgt?? 10? >>>> 293.769 ?? 7.321? us/op >>>> RecordDeserializationBench.deserializeRecords??????? 10 avgt?? 10?? >>>> 15.335 ?? 0.496? us/op >>>> RecordDeserializationBench.deserializeRecords?????? 100 avgt?? 10? >>>> 211.427 ? 11.908? us/op >>>> RecordDeserializationBench.deserializeRecords????? 1000 avgt?? 10? >>>> 990.398 ? 26.681? us/op >>>> >>>> >>>> This is using JMH option '-gc true' to force GC after each >>>> iteration of benchmark. Without it, I get a big (~4s) full-GC pause >>>> just in the middle of a run with length=100: >>>> >>>> >>>> Iteration?? 1: 528.577 us/op >>>> Iteration?? 2: 580.404 us/op >>>> Iteration?? 3: 4438.228 us/op >>>> Iteration?? 4: 644.532 us/op >>>> Iteration?? 5: 698.493 us/op >>>> Iteration?? 6: 800.738 us/op >>>> Iteration?? 7: 929.791 us/op >>>> Iteration?? 8: 870.946 us/op >>>> Iteration?? 9: 863.416 us/op >>>> Iteration? 10: 916.508 us/op >>>> >>>> >>>> ...so results are a bit off because of that: >>>> >>>> >>>> Benchmark????????????????????????????????????? (length) Mode >>>> Cnt???? Score????? Error? Units >>>> RecordDeserializationBench.deserializeClasses??????? 10 avgt?? >>>> 10???? 8.263 ???? 0.043? us/op >>>> RecordDeserializationBench.deserializeClasses?????? 100 avgt?? >>>> 10??? 33.406 ???? 0.160? us/op >>>> RecordDeserializationBench.deserializeClasses????? 1000 avgt?? 10?? >>>> 287.595 ???? 0.960? us/op >>>> RecordDeserializationBench.deserializeRecords??????? 10 avgt?? >>>> 10??? 15.270 ???? 0.080? us/op >>>> RecordDeserializationBench.deserializeRecords?????? 100 avgt?? 10? >>>> 1127.163 ? 1771.892? us/op >>>> RecordDeserializationBench.deserializeRecords????? 1000 avgt?? 10? >>>> 2003.235 ?? 227.159? us/op >>>> >>>> >>>> Yes, there is quite a bit of GCing going on when cache is >>>> thrashing. Note that I haven't tuned GC in any way and I'm running >>>> this on a machine with 64GiB of RAM so heap is allowed to grow >>>> quite big and G1 is used by default I think. >>>> >>>> >>>> This is still no worse than before the patch: >>>> >>>> >>>> Benchmark???????????????????????????????? (length)? Mode Cnt???? >>>> Score??? Error? Units >>>> RecordDeserialization.deserializeClasses??????? 10? avgt 10???? >>>> 8.382 :? 0.013? us/op >>>> RecordDeserialization.deserializeClasses?????? 100? avgt 10??? >>>> 33.736 :? 0.171? us/op >>>> RecordDeserialization.deserializeClasses????? 1000? avgt 10?? >>>> 271.224 :? 0.953? us/op >>>> RecordDeserialization.deserializeRecords??????? 10? avgt 10??? >>>> 58.606 :? 0.446? us/op >>>> RecordDeserialization.deserializeRecords?????? 100? avgt 10?? >>>> 530.044 :? 1.752? us/op >>>> RecordDeserialization.deserializeRecords????? 1000? avgt 10 >>>> 5335.624 : 44.942? us/op >>>> >>>> >>>> ... since caching of adapted method handle for multiple objects >>>> withing single stream is still effective. >>>> >>>> I doubt there will ever be more than 10 variants/versions of the >>>> same record class deserialized by the same VM and in rapid >>>> succession, so I think this should not be an issue. We could add a >>>> system property to control the MAX_SIZE of cache if you think it is >>>> needed. >>> >>> Thanks for running the numbers on this, and I agree - it seems >>> outlandishly improbable (most will only see one, and if you have to >>> maintain 10+ different serialized shapes of some record you likely >>> have bigger problems). >>> >>> I'd say let's keep it constant unless someone actually asks for it. >>> >>> /Claes >>> >>>> >>>> >>>> Regards, Peter >>>> >>>> >>>> On 6/22/20 1:04 AM, Claes Redestad wrote: >>>>> Hi Peter, >>>>> >>>>> patch and results look great! >>>>> >>>>> My only real comment on this is that I think the microbenchmark >>>>> would be >>>>> a valuable contribution, too. >>>>> >>>>> It'd also be interesting to explore how poor performance would >>>>> become if >>>>> we'd hit the (artificial) 11 layouts limit, e.g, by cycling through >>>>> 10, 11, or 12 different shapes. >>>>> >>>>> /Claes >>>>> >>>>> On 2020-06-21 19:16, Peter Levart wrote: >>>>>> Hi, >>>>>> >>>>>> >>>>>> When re-running the benchmark [1] with different lengths of >>>>>> serialized arrays of records, I found that, compared to classical >>>>>> classes, lookup into the cache of adapted method handles starts >>>>>> to show when the length of array is larger (# of instances of >>>>>> same record type deserialized in single stream). Each record >>>>>> deserialized must lookup the method handle in a ConcurrentHashMap: >>>>>> >>>>>> >>>>>> Benchmark??????????????????????????????????? (length) Mode Cnt >>>>>> Score?? Error? Units >>>>>> RecordSerializationBench.deserializeClasses??????? 10 avgt 10 >>>>>> 8.088 ? 0.013? us/op >>>>>> RecordSerializationBench.deserializeClasses?????? 100 avgt 10 >>>>>> 32.171 ? 0.324? us/op >>>>>> RecordSerializationBench.deserializeClasses????? 1000 avgt 10 >>>>>> 279.762 ? 3.072? us/op >>>>>> RecordSerializationBench.deserializeRecords??????? 10 avgt 10 >>>>>> 9.011 ? 0.027? us/op >>>>>> RecordSerializationBench.deserializeRecords?????? 100 avgt 10 >>>>>> 33.206 ? 0.514? us/op >>>>>> RecordSerializationBench.deserializeRecords????? 1000 avgt 10 >>>>>> 325.137 ? 0.969? us/op >>>>>> >>>>>> >>>>>> ...so keeping the correctly shaped adapted method handle in the >>>>>> per-serialization-session ObjectStreamClass instance [2] starts >>>>>> to make sense: >>>>>> >>>>>> >>>>>> Benchmark??????????????????????????????????? (length) Mode Cnt >>>>>> Score?? Error? Units >>>>>> RecordSerializationBench.deserializeClasses??????? 10 avgt 10 >>>>>> 8.681 ? 0.155? us/op >>>>>> RecordSerializationBench.deserializeClasses?????? 100 avgt 10 >>>>>> 32.496 ? 0.087? us/op >>>>>> RecordSerializationBench.deserializeClasses????? 1000 avgt 10 >>>>>> 279.014 ? 1.189? us/op >>>>>> RecordSerializationBench.deserializeRecords??????? 10 avgt 10 >>>>>> 8.537 ? 0.032? us/op >>>>>> RecordSerializationBench.deserializeRecords?????? 100 avgt 10 >>>>>> 31.451 ? 0.083? us/op >>>>>> RecordSerializationBench.deserializeRecords????? 1000 avgt 10 >>>>>> 250.854 ? 2.772? us/op >>>>>> >>>>>> >>>>>> With that, more objects means advantage over classical classes >>>>>> instead of disadvantage. >>>>>> >>>>>> >>>>>> [1] >>>>>> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/RecordSerializationBench.java >>>>>> >>>>>> >>>>>> [2] >>>>>> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.06/ >>>>>> >>>>>> >>>>>> >>>>>> Regards, Peter >>>>>> >>>>>> From chris.hegarty at oracle.com Tue Jun 23 09:46:13 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 23 Jun 2020 10:46:13 +0100 Subject: RFR: 8247532, 8248135: Records deserialization is slow + Build microbenchmarks with --enable-preview In-Reply-To: <9e4e54c8-05c9-09fd-67ee-98c91ac9aeb3@gmail.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> <1DA206A1-15F0-46F9-B69E-9896C57B97C7@oracle.com> <8886ee56-e9a7-aafe-6111-f0c2c54a282e@gmail.com> <63689945-dca3-e0cc-72b8-9225f5da6d31@gmail.com> <7f7b58fa-50e0-a4fe-f948-9637d04c0ccc@oracle.com> <5021b3e7-28ad-7041-cc89-c0f218434937@oracle.com> <092241fe-1f43-ead6-6a5d-0345dac55c3e@oracle.com> <9e4e54c8-05c9-09fd-67ee-98c91ac9aeb3@gmail.com> Message-ID: > On 23 Jun 2020, at 10:17, Peter Levart wrote: > > ... > http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.08/ Good stuff. Reviewed. I am going to take this latest change and run it through our internal build and test system. Will post the results here soon. -Chris. From chris.hegarty at oracle.com Tue Jun 23 12:47:03 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 23 Jun 2020 13:47:03 +0100 Subject: RFR: 8247532, 8248135: Records deserialization is slow + Build microbenchmarks with --enable-preview In-Reply-To: References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> <1DA206A1-15F0-46F9-B69E-9896C57B97C7@oracle.com> <8886ee56-e9a7-aafe-6111-f0c2c54a282e@gmail.com> <63689945-dca3-e0cc-72b8-9225f5da6d31@gmail.com> <7f7b58fa-50e0-a4fe-f948-9637d04c0ccc@oracle.com> <5021b3e7-28ad-7041-cc89-c0f218434937@oracle.com> <092241fe-1f43-ead6-6a5d-0345dac55c3e@oracle.com> <9e4e54c8-05c9-09fd-67ee-98c91ac9aeb3@gmail.com> Message-ID: > On 23 Jun 2020, at 10:46, Chris Hegarty wrote: > > > >> On 23 Jun 2020, at 10:17, Peter Levart wrote: >> >> ... >> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.08/ > > Good stuff. Reviewed. > > I am going to take this latest change and run it through our internal build and test system. Will post the results here soon. All clear - builds and tests with this change are green. -Chris. From raffaello.giulietti at gmail.com Tue Jun 23 13:00:44 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Tue, 23 Jun 2020 15:00:44 +0200 Subject: java.util.Base64 accepts non-canonical encodings Message-ID: <716b1d0b-814c-bb68-e470-92bd73569cd4@gmail.com> Hi, RFC 4648, in section "3.5. Canonical Encoding", prescribes that pad bits must be set to zero. However, the current decoder implementation in java.util.Base64 accepts non-canonical encodings as well. For example, all of the following four encodings KCk= KCl= KCm= KCn= where only the first is canonical, decode to the sequence of two bytes 0x28 0x29. Padding positions could act as a (very low bandwidth) covert channel. Since I'm preparing a patch for [1] (see [2]), I'm asking if this is intentional behavior or if it is an oversight. Of course, checking for strictness would slightly impact performance. If checking for non-zero padding bits is desired, should the API be extended to allow for both the strict and the (current) lenient behaviors? Would the current API suffice? Greetings Raffaello ---- [1] https://bugs.openjdk.java.net/browse/JDK-8222187 [2] https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-June/067066.html From david.lloyd at redhat.com Tue Jun 23 13:04:12 2020 From: david.lloyd at redhat.com (David Lloyd) Date: Tue, 23 Jun 2020 08:04:12 -0500 Subject: Math.mutiplyHigh() is signed Message-ID: In retrospect I don't know why I expected Math.multiplyHigh() to return the high word of the unsigned product of two 64-bit numbers, given that long is signed; in my defence however, the docs don't actually seem to specify. WDYT about a patch like this to clarify? diff --git a/src/java.base/share/classes/java/lang/Math.java b/src/java.base/share/classes/java/lang/Math.java index 8147b7109e2..552501ca6d7 100644 --- a/src/java.base/share/classes/java/lang/Math.java +++ b/src/java.base/share/classes/java/lang/Math.java @@ -1095,8 +1095,8 @@ public final class Math { } /** - * Returns as a {@code long} the most significant 64 bits of the 128-bit - * product of two 64-bit factors. + * Returns as a {@code long} the most significant 64 bits of the signed 128-bit + * product of two 64-bit signed factors. * * @param x the first value * @param y the second value -- - DML From raffaello.giulietti at gmail.com Tue Jun 23 13:29:41 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Tue, 23 Jun 2020 15:29:41 +0200 Subject: Math.mutiplyHigh() is signed Message-ID: <620889d1-c8a7-5cbe-3cce-3cfc441b3fb7@gmail.com> Hi, in defense of the current spec, integer arithmetic is signed by default and unsigned behavior is evident from the method names, like Long::compareUnsigned or Long::divideUnsigned But your spec is even clearer ;-) Greetings Raffaello > In retrospect I don't know why I expected Math.multiplyHigh() to > return the high word of the unsigned product of two 64-bit numbers, > given that long is signed; in my defence however, the docs don't > actually seem to specify. > > WDYT about a patch like this to clarify? > > diff --git a/src/java.base/share/classes/java/lang/Math.java > b/src/java.base/share/classes/java/lang/Math.java > index 8147b7109e2..552501ca6d7 100644 > --- a/src/java.base/share/classes/java/lang/Math.java > +++ b/src/java.base/share/classes/java/lang/Math.java > @@ -1095,8 +1095,8 @@ public final class Math { > } > > /** > - * Returns as a {@code long} the most significant 64 bits of the 128-bit > - * product of two 64-bit factors. > + * Returns as a {@code long} the most significant 64 bits of the > signed 128-bit > + * product of two 64-bit signed factors. > * > * @param x the first value > * @param y the second value > > > -- > - DML From peter.levart at gmail.com Tue Jun 23 13:49:18 2020 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 23 Jun 2020 15:49:18 +0200 Subject: RFR: 8247532, 8248135: Records deserialization is slow + Build microbenchmarks with --enable-preview In-Reply-To: References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> <1DA206A1-15F0-46F9-B69E-9896C57B97C7@oracle.com> <8886ee56-e9a7-aafe-6111-f0c2c54a282e@gmail.com> <63689945-dca3-e0cc-72b8-9225f5da6d31@gmail.com> <7f7b58fa-50e0-a4fe-f948-9637d04c0ccc@oracle.com> <5021b3e7-28ad-7041-cc89-c0f218434937@oracle.com> <092241fe-1f43-ead6-6a5d-0345dac55c3e@oracle.com> <9e4e54c8-05c9-09fd-67ee-98c91ac9aeb3@gmail.com> Message-ID: On 6/23/20 2:47 PM, Chris Hegarty wrote: > >> On 23 Jun 2020, at 10:46, Chris Hegarty wrote: >> >> >> >>> On 23 Jun 2020, at 10:17, Peter Levart wrote: >>> >>> ... >>> http://cr.openjdk.java.net/~plevart/jdk-dev/RecordsDeserialization/webrev.08/ >> Good stuff. Reviewed. >> >> I am going to take this latest change and run it through our internal build and test system. Will post the results here soon. > All clear - builds and tests with this change are green. > > -Chris. Ok, I'm going to push this to jdk15. Thanks Johannes, Chris, Remi, Paul, Claes and Magnus, for the help and reviews. Regards, Peter From Alan.Bateman at oracle.com Tue Jun 23 14:09:46 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 23 Jun 2020 15:09:46 +0100 Subject: RFR: 8248131: Simplify ServicesCatalog provider handling In-Reply-To: <371cde4c-a098-1677-7fbc-4a5026c57144@oracle.com> References: <371cde4c-a098-1677-7fbc-4a5026c57144@oracle.com> Message-ID: On 23/06/2020 09:56, Claes Redestad wrote: > Hi, > > the current implementation of ServicesCatalog uses an internal providers > method, which is only used to add ServiceProviders. > > Providing an addProviders method instead means we can streamline this, > to the tune of a very small startup win. > > Bug:??? https://bugs.openjdk.java.net/browse/JDK-8248131 > Webrev: http://cr.openjdk.java.net/~redestad/8248131/open.00/ > > Testing: tier1 The special casing for the providers.length == 1 case is unfortunate but otherwise looks good to me. Can you make sure to run the jdk_instrument test group and the com/sun/tools/attach/modules tests as the changed code is also executing redefining a module to implement new providers, also when attach mechanism is use to load the jdk.management and jdk.attach modules are loaded into a running VM. -Alan. From Roger.Riggs at oracle.com Tue Jun 23 14:28:26 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Tue, 23 Jun 2020 10:28:26 -0400 Subject: java.util.Base64 accepts non-canonical encodings In-Reply-To: <716b1d0b-814c-bb68-e470-92bd73569cd4@gmail.com> References: <716b1d0b-814c-bb68-e470-92bd73569cd4@gmail.com> Message-ID: <821ceb80-c90a-dc58-92da-80db421ac673@oracle.com> Hi Raffaello, I think the concern over accepting non-canonical encodings would be compatibility. It would rude to implement the strictness and have applications start failing. But it is likely an oversight since existing code checks for other invalid encodings. Do any of the existing tests fail if the non-canonical encoding throws? Thanks, Roger On 6/23/20 9:00 AM, Raffaello Giulietti wrote: > Hi, > > RFC 4648, in section "3.5. Canonical Encoding", prescribes that pad > bits must be set to zero. > > However, the current decoder implementation in java.util.Base64 > accepts non-canonical encodings as well. For example, all of the > following four encodings > KCk= > KCl= > KCm= > KCn= > where only the first is canonical, decode to the sequence of two bytes > 0x28 0x29. Padding positions could act as a (very low bandwidth) > covert channel. > > Since I'm preparing a patch for [1] (see [2]), I'm asking if this is > intentional behavior or if it is an oversight. Of course, checking for > strictness would slightly impact performance. > > If checking for non-zero padding bits is desired, should the API be > extended to allow for both the strict and the (current) lenient > behaviors? Would the current API suffice? > > > Greetings > Raffaello > > ---- > > [1] https://bugs.openjdk.java.net/browse/JDK-8222187 > [2] > https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-June/067066.html From raffaello.giulietti at gmail.com Tue Jun 23 14:34:05 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Tue, 23 Jun 2020 16:34:05 +0200 Subject: Exceptions priority in InputStream.read(byte[], int, int) Message-ID: <2936d5cc-edcf-d005-1e2e-fe16430beb3c@gmail.com> Hi, the InputStream.read(byte[], int, int) method [1] can throw * IOException * NullPointerException * IndexOutOfBoundsException Is there a recommended priority for the conditions associated to the exceptions to be checked? For example, if the arguments are invalid and the stream is already closed, which exception shall be thrown? Greetings Raffaello ---- [1] https://docs.oracle.com/en/java/javase/14/docs/api/java.base/java/io/InputStream.html#read(byte%5B%5D,int,int) From raffaello.giulietti at gmail.com Tue Jun 23 14:50:29 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Tue, 23 Jun 2020 16:50:29 +0200 Subject: java.util.Base64 accepts non-canonical encodings Message-ID: <5ad072ac-0d22-5aab-b3ae-f4ab1ef1db90@gmail.com> Hi Roger, I didn't yet implement the strict check since, as you point out, it could harm existing code in the wild, even if the OpenJDK test would all pass. That's why I'm wondering if it would make sense to extend the existing API to have the check as an additional option. Greetings Raffaello > Hi Raffaello, > > I think the concern over accepting non-canonical encodings would be > compatibility. > It would rude to implement the strictness and have applications start > failing. > But it is likely an oversight since existing code checks for other > invalid encodings. > > Do any of the existing tests fail if the non-canonical encoding throws? > > Thanks, Roger > > > On 6/23/20 9:00 AM, Raffaello Giulietti wrote: >> Hi, >> >> RFC 4648, in section "3.5. Canonical Encoding", prescribes that pad >> bits must be set to zero. >> >> However, the current decoder implementation in java.util.Base64 >> accepts non-canonical encodings as well. For example, all of the >> following four encodings >> KCk= >> KCl= >> KCm= >> KCn= >> where only the first is canonical, decode to the sequence of two bytes >> 0x28 0x29. Padding positions could act as a (very low bandwidth) >> covert channel. >> >> Since I'm preparing a patch for [1] (see [2]), I'm asking if this is >> intentional behavior or if it is an oversight. Of course, checking for >> strictness would slightly impact performance. >> >> If checking for non-zero padding bits is desired, should the API be >> extended to allow for both the strict and the (current) lenient >> behaviors? Would the current API suffice? >> >> >> Greetings >> Raffaello >> >> ---- >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8222187 >> [2] >> https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-June/067066.html From martinrb at google.com Tue Jun 23 14:54:25 2020 From: martinrb at google.com (Martin Buchholz) Date: Tue, 23 Jun 2020 07:54:25 -0700 Subject: Exceptions priority in InputStream.read(byte[], int, int) In-Reply-To: <2936d5cc-edcf-d005-1e2e-fe16430beb3c@gmail.com> References: <2936d5cc-edcf-d005-1e2e-fe16430beb3c@gmail.com> Message-ID: When there is a choice of exceptions to be thrown, it's not usually specified which one wins. Usually there's a natural order of operations that cause e.g. argument checking to happen first. Sometimes a code change can cause a different exception to be thrown, and that occasionally causes compatibility trouble. On Tue, Jun 23, 2020 at 7:43 AM Raffaello Giulietti wrote: > > Hi, > > the InputStream.read(byte[], int, int) method [1] can throw > * IOException > * NullPointerException > * IndexOutOfBoundsException > > Is there a recommended priority for the conditions associated to the > exceptions to be checked? For example, if the arguments are invalid and > the stream is already closed, which exception shall be thrown? > > > Greetings > Raffaello > > ---- > > [1] > https://docs.oracle.com/en/java/javase/14/docs/api/java.base/java/io/InputStream.html#read(byte%5B%5D,int,int) From Roger.Riggs at oracle.com Tue Jun 23 15:03:05 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Tue, 23 Jun 2020 11:03:05 -0400 Subject: Exceptions priority in InputStream.read(byte[], int, int) In-Reply-To: <2936d5cc-edcf-d005-1e2e-fe16430beb3c@gmail.com> References: <2936d5cc-edcf-d005-1e2e-fe16430beb3c@gmail.com> Message-ID: <813265b5-ba2c-94fb-6eea-9622c4946578@oracle.com> Hi Raffaello, Exceptions related to the arguments should be highest priority so they are not obscured by any more dynamic state related to the stream. There's no specific order between checks of the arguments, though in this case the NPE would naturally occur before IOOBE. There are more complex cases where some arguments are not used or used conditionally where the checks become dependent on the dynamic state. $.02, Roger On 6/23/20 10:34 AM, Raffaello Giulietti wrote: > Hi, > > the InputStream.read(byte[], int, int) method [1] can throw > * IOException > * NullPointerException > * IndexOutOfBoundsException > > Is there a recommended priority for the conditions associated to the > exceptions to be checked? For example, if the arguments are invalid > and the stream is already closed, which exception shall be thrown? > > > Greetings > Raffaello > > ---- > > [1] > https://docs.oracle.com/en/java/javase/14/docs/api/java.base/java/io/InputStream.html#read(byte%5B%5D,int,int) From Roger.Riggs at oracle.com Tue Jun 23 15:14:28 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Tue, 23 Jun 2020 11:14:28 -0400 Subject: java.util.Base64 accepts non-canonical encodings In-Reply-To: <5ad072ac-0d22-5aab-b3ae-f4ab1ef1db90@gmail.com> References: <5ad072ac-0d22-5aab-b3ae-f4ab1ef1db90@gmail.com> Message-ID: <90f3171e-df9a-e4de-df80-9c77b8462863@oracle.com> Hi, This is a case where having some more interoperability testing could be informative though there are likely many adhoc Base64 encoders and its not practical to test against them. Introducing a new mode or option creates an undesirable fuzzyness to the API. It won't help existing uses without some deliberate attempt to move clients to use the new option (deprecation). Its not likely to be picked up by new clients because the difference is behavior is slight and may not be seen as important. An options also tend to live forever, increasing maintenace on both the library and the callers. Its worth creating a separate issue and looking at it separately. Thanks, Roger On 6/23/20 10:50 AM, Raffaello Giulietti wrote: > Hi Roger, > > I didn't yet implement the strict check since, as you point out, it > could harm existing code in the wild, even if the OpenJDK test would > all pass. > > That's why I'm wondering if it would make sense to extend the existing > API to have the check as an additional option. > > > Greetings > Raffaello > > >> Hi Raffaello, >> >> I think the concern over accepting non-canonical encodings would be >> compatibility. >> It would rude to implement the strictness and have applications start >> failing. >> But it is likely an oversight since existing code checks for other >> invalid encodings. >> >> Do any of the existing tests fail if the non-canonical encoding throws? >> >> Thanks, Roger >> >> >> On 6/23/20 9:00 AM, Raffaello Giulietti wrote: >>> Hi, >>> >>> RFC 4648, in section "3.5. Canonical Encoding", prescribes that pad >>> bits must be set to zero. >>> >>> However, the current decoder implementation in java.util.Base64 >>> accepts non-canonical encodings as well. For example, all of the >>> following four encodings >>> KCk= >>> KCl= >>> KCm= >>> KCn= >>> where only the first is canonical, decode to the sequence of two >>> bytes 0x28 0x29. Padding positions could act as a (very low >>> bandwidth) covert channel. >>> >>> Since I'm preparing a patch for [1] (see [2]), I'm asking if this is >>> intentional behavior or if it is an oversight. Of course, checking >>> for strictness would slightly impact performance. >>> >>> If checking for non-zero padding bits is desired, should the API be >>> extended to allow for both the strict and the (current) lenient >>> behaviors? Would the current API suffice? >>> >>> >>> Greetings >>> Raffaello >>> >>> ---- >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8222187 >>> [2] >>> https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-June/067066.html From raffaello.giulietti at gmail.com Tue Jun 23 15:34:08 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Tue, 23 Jun 2020 17:34:08 +0200 Subject: Exceptions priority in InputStream.read(byte[], int, int) In-Reply-To: References: <2936d5cc-edcf-d005-1e2e-fe16430beb3c@gmail.com> Message-ID: <4f015f01-eba6-37e3-c0ec-d4e695e73fe6@gmail.com> Yes, argument checking happening first is a sensible choice. But if one considers the target of the method to be an implicit first argument, then, in the example, the stream would be checked first, possibly throwing IOException. On the other hand, if one only considers the explicit arguments, then the example would possibly throw NullPointerException or IndexOutOfBoundsException. The point is that, in general, the order shall either be explicitly *unspecified* (which has costs for the consumer code) or be explicitly *specified*, and then almost forever if it resides in a popular library like java.io (which has costs for the provider code). Sometimes life is hard... On 2020-06-23 16:54, Martin Buchholz wrote: > When there is a choice of exceptions to be thrown, it's not usually > specified which one wins. > Usually there's a natural order of operations that cause e.g. argument > checking to happen first. > Sometimes a code change can cause a different exception to be thrown, > and that occasionally causes compatibility trouble. > > On Tue, Jun 23, 2020 at 7:43 AM Raffaello Giulietti > wrote: >> >> Hi, >> >> the InputStream.read(byte[], int, int) method [1] can throw >> * IOException >> * NullPointerException >> * IndexOutOfBoundsException >> >> Is there a recommended priority for the conditions associated to the >> exceptions to be checked? For example, if the arguments are invalid and >> the stream is already closed, which exception shall be thrown? >> >> >> Greetings >> Raffaello >> >> ---- >> >> [1] >> https://docs.oracle.com/en/java/javase/14/docs/api/java.base/java/io/InputStream.html#read(byte%5B%5D,int,int) From raffaello.giulietti at gmail.com Tue Jun 23 15:41:01 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Tue, 23 Jun 2020 17:41:01 +0200 Subject: java.util.Base64 accepts non-canonical encodings In-Reply-To: <90f3171e-df9a-e4de-df80-9c77b8462863@oracle.com> References: <5ad072ac-0d22-5aab-b3ae-f4ab1ef1db90@gmail.com> <90f3171e-df9a-e4de-df80-9c77b8462863@oracle.com> Message-ID: OK, I'll avoid the checks in the patch intended for general publication but will add them separately just to see which of current tests would fail. I'll then later open a separate issue for discussing a possible extension of the API and a possible CSR. Greetings Raffaello On 2020-06-23 17:14, Roger Riggs wrote: > Hi, > > This is a case where having some more interoperability testing could be > informative > though there are likely many adhoc Base64 encoders and its not practical > to test > against them. > > Introducing a new mode or option creates an undesirable fuzzyness to the > API. > > It won't help existing uses without some deliberate attempt to move > clients to use > the new option (deprecation). Its not likely to be picked up by new > clients because > the difference is behavior is slight and may not be seen as important. > An options also tend to live forever, increasing maintenace on both the > library and > the callers. > > Its worth creating a separate issue and looking at it separately. > > Thanks, Roger > > > On 6/23/20 10:50 AM, Raffaello Giulietti wrote: >> Hi Roger, >> >> I didn't yet implement the strict check since, as you point out, it >> could harm existing code in the wild, even if the OpenJDK test would >> all pass. >> >> That's why I'm wondering if it would make sense to extend the existing >> API to have the check as an additional option. >> >> >> Greetings >> Raffaello >> >> >>> Hi Raffaello, >>> >>> I think the concern over accepting non-canonical encodings would be >>> compatibility. >>> It would rude to implement the strictness and have applications start >>> failing. >>> But it is likely an oversight since existing code checks for other >>> invalid encodings. >>> >>> Do any of the existing tests fail if the non-canonical encoding throws? >>> >>> Thanks, Roger >>> >>> >>> On 6/23/20 9:00 AM, Raffaello Giulietti wrote: >>>> Hi, >>>> >>>> RFC 4648, in section "3.5. Canonical Encoding", prescribes that pad >>>> bits must be set to zero. >>>> >>>> However, the current decoder implementation in java.util.Base64 >>>> accepts non-canonical encodings as well. For example, all of the >>>> following four encodings >>>> KCk= >>>> KCl= >>>> KCm= >>>> KCn= >>>> where only the first is canonical, decode to the sequence of two >>>> bytes 0x28 0x29. Padding positions could act as a (very low >>>> bandwidth) covert channel. >>>> >>>> Since I'm preparing a patch for [1] (see [2]), I'm asking if this is >>>> intentional behavior or if it is an oversight. Of course, checking >>>> for strictness would slightly impact performance. >>>> >>>> If checking for non-zero padding bits is desired, should the API be >>>> extended to allow for both the strict and the (current) lenient >>>> behaviors? Would the current API suffice? >>>> >>>> >>>> Greetings >>>> Raffaello >>>> >>>> ---- >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8222187 >>>> [2] >>>> https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-June/067066.html >>>> > From mandy.chung at oracle.com Tue Jun 23 16:08:29 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 23 Jun 2020 09:08:29 -0700 Subject: RFR: 8248131: Simplify ServicesCatalog provider handling In-Reply-To: <371cde4c-a098-1677-7fbc-4a5026c57144@oracle.com> References: <371cde4c-a098-1677-7fbc-4a5026c57144@oracle.com> Message-ID: <26dbfa27-dec6-d50f-05ef-7b52031b969f@oracle.com> On 6/23/20 1:56 AM, Claes Redestad wrote: > Hi, > > the current implementation of ServicesCatalog uses an internal providers > method, which is only used to add ServiceProviders. > > Providing an addProviders method instead means we can streamline this, > to the tune of a very small startup win. > > Bug:??? https://bugs.openjdk.java.net/browse/JDK-8248131 > Webrev: http://cr.openjdk.java.net/~redestad/8248131/open.00/ > Looks okay to me. Mandy From chris.hegarty at oracle.com Tue Jun 23 16:30:45 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 23 Jun 2020 17:30:45 +0100 Subject: RFR: 8247532, 8248135: Records deserialization is slow + Build microbenchmarks with --enable-preview In-Reply-To: References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> <1DA206A1-15F0-46F9-B69E-9896C57B97C7@oracle.com> <8886ee56-e9a7-aafe-6111-f0c2c54a282e@gmail.com> <63689945-dca3-e0cc-72b8-9225f5da6d31@gmail.com> <7f7b58fa-50e0-a4fe-f948-9637d04c0ccc@oracle.com> <5021b3e7-28ad-7041-cc89-c0f218434937@oracle.com> <092241fe-1f43-ead6-6a5d-0345dac55c3e@oracle.com> <9e4e54c8-05c9-09fd-67ee-98c91ac9aeb3@gmail.com> Message-ID: > On 23 Jun 2020, at 14:49, Peter Levart wrote: > > ... > > Ok, I'm going to push this to jdk15. Thank you Peter. This is a really nice change. As a follow on, and not for JDK 15, I observe that Class::isRecord0 / JVM_IsRecord shows up as consuming a significant amount of time, more than 10%, in some runs of the deserialization benchmark. The isRecord implementation is a native method in the JVM, so relatively expensive to call. This shows an opportunity to improve the Class::isRecord implementation with a simple cache of the record-ness of the j.l.Class, as is done selectively with other particular aspects of a class?s state. There are various ways to implement this, but here is just one [*]. Running the deserialization benchmark with this change [*], gives the following results: Benchmark (length) Mode Cnt Score Error Units RecordSerializationBench.deserializeClasses 10 avgt 10 14.136 ? 0.841 us/op RecordSerializationBench.deserializeClasses 100 avgt 10 61.821 ? 1.279 us/op RecordSerializationBench.deserializeClasses 1000 avgt 10 519.473 ? 7.950 us/op RecordSerializationBench.deserializeRecords 10 avgt 10 13.781 ? 1.917 us/op RecordSerializationBench.deserializeRecords 100 avgt 10 54.061 ? 4.188 us/op RecordSerializationBench.deserializeRecords 1000 avgt 10 444.538 ? 13.940 us/op I think it is worth considering caching the record-ness state of a j.l.Class, as I?m sure it will be widely used in third-party serialization libraries, as well as by Java Serialization. -Chris. [*] diff -r 3a9521647349 src/java.base/share/classes/java/lang/Class.java --- a/src/java.base/share/classes/java/lang/Class.java Tue Jun 23 10:46:39 2020 +0100 +++ b/src/java.base/share/classes/java/lang/Class.java Tue Jun 23 17:11:35 2020 +0100 @@ -3712,9 +3712,17 @@ @jdk.internal.PreviewFeature(feature=jdk.internal.PreviewFeature.Feature.RECORDS, essentialAPI=false) public boolean isRecord() { - return getSuperclass() == JAVA_LANG_RECORD_CLASS && isRecord0(); + if (!isRecordCalled) { + isRecord = getSuperclass() == JAVA_LANG_RECORD_CLASS && isRecord0(); + isRecordCalled = true; + } + return isRecord; } + // cached record(ness) status + private transient boolean isRecord; + private transient boolean isRecordCalled; + // Fetches the factory for reflective objects private static ReflectionFactory getReflectionFactory() { if (reflectionFactory == null) { From claes.redestad at oracle.com Tue Jun 23 16:37:20 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 23 Jun 2020 18:37:20 +0200 Subject: RFR: 8248131: Simplify ServicesCatalog provider handling In-Reply-To: References: <371cde4c-a098-1677-7fbc-4a5026c57144@oracle.com> Message-ID: Hi Alan, Mandy, thanks for reviewing. I've ran all java/lang/instrument and the com/sun/tools/attach/modules tests, as well as added tier2 to the testing mix. /Claes On 2020-06-23 16:09, Alan Bateman wrote: > On 23/06/2020 09:56, Claes Redestad wrote: >> Hi, >> >> the current implementation of ServicesCatalog uses an internal providers >> method, which is only used to add ServiceProviders. >> >> Providing an addProviders method instead means we can streamline this, >> to the tune of a very small startup win. >> >> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8248131 >> Webrev: http://cr.openjdk.java.net/~redestad/8248131/open.00/ >> >> Testing: tier1 > The special casing for the providers.length == 1 case is unfortunate but > otherwise looks good to me. > > Can you make sure to run the jdk_instrument test group and the > com/sun/tools/attach/modules tests as the changed code is also executing > redefining a module to implement new providers, also when attach > mechanism is use to load the jdk.management and jdk.attach modules are > loaded into a running VM. > > -Alan. From alexey.semenyuk at oracle.com Tue Jun 23 17:53:49 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Tue, 23 Jun 2020 13:53:49 -0400 Subject: RFR: JDK-8246212: JPKG001-012: application icon is missing in Control Panel Add/Remove In-Reply-To: <4efc0137-d2fb-cf69-1fac-3e871d5eedc4@oracle.com> References: <4efc0137-d2fb-cf69-1fac-3e871d5eedc4@oracle.com> Message-ID: <01ebf9e4-9049-b33d-e395-798706828b74@oracle.com> Please review fix [2] for jpackage bug [1]. Fix how icon is configured for installers on Windows. The value of ARPPRODUCTICON property should point to an entry in Icon table of msi rather to a path of icon file in install directory. The issue shows up only in downgrade scenarios. - Alexey [1] https://bugs.openjdk.java.net/browse/JDK-8246212 [2]?http://cr.openjdk.java.net/~asemenyuk/8246212/webrev.00 From mandy.chung at oracle.com Tue Jun 23 18:03:11 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 23 Jun 2020 11:03:11 -0700 Subject: [15] RFR JDK-8247785: Small clarification to the javadoc about builtin class loaders Message-ID: <37776e19-a225-9547-6d53-147aa69679b8@oracle.com> Small clarification about the parent of the system class loader in the ClassLoader class spec: diff --git a/src/java.base/share/classes/java/lang/ClassLoader.java b/src/java.base/share/classes/java/lang/ClassLoader.java --- a/src/java.base/share/classes/java/lang/ClassLoader.java +++ b/src/java.base/share/classes/java/lang/ClassLoader.java @@ -133,7 +133,7 @@ * It is the virtual machine's built-in class loader, typically represented * as {@code null}, and does not have a parent. *

    • {@linkplain #getPlatformClassLoader() Platform class loader}. - * All platform classes are visible to the platform class loader + * Platform classes are visible the platform class loader * that can be used as the parent of a {@code ClassLoader} instance. * Platform classes include Java SE platform APIs, their implementation * classes and JDK-specific run-time classes that are defined by the @@ -152,7 +152,7 @@ * The system class loader is typically used to define classes on the * application class path, module path, and JDK-specific tools. * The platform class loader is a parent or an ancestor of the system class - * loader that all platform classes are visible to it.

    • + * loader which searches and loads the platform classes through its parent. * * *

      Normally, the Java virtual machine loads classes from the local file I dropped the word "All" because which platform modules are resolved in the module graph depends on the root modules of the runtime configuration. A platform module may not be resolved in some configuration and hence not all platform classes are visible to the platform class loader. Mandy From gilles.m.duboscq at oracle.com Tue Jun 23 18:08:00 2020 From: gilles.m.duboscq at oracle.com (Gilles Duboscq) Date: Tue, 23 Jun 2020 20:08:00 +0200 Subject: RFR: JDK-8242451 : ensure semantics of non-capturing lambdas are preserved independent of execution mode Message-ID: In 8232806, a system property was introduce to disable eager initialization of the classes generated by the InnerClassLambdaMetafactory (`jdk.internal.lambda.disableEagerInitialization`). However, when `disableEagerInitialization` is true, even for non-capturing lambdas, the capturing lambda path that uses a method handle to the constructor is taken. This helps prevent eager initialization but also has the side effect of always returning a fresh instance of the lambda for every invocation instead of a singleton. While this is allowed by the specs, this might change the behaviour of some (incorrect) programs that assume a singleton is used for non-capturing lambdas. I propose to keep the effects of the `disableEagerInitialization` related to initialization only. Webrev: https://cr.openjdk.java.net/~gdub/8242451/webrev.0/ Issue: https://bugs.openjdk.java.net/browse/JDK-8242451 The concrete issue we are seeing with changing both aspects at the same time is that when disableEagerInitialization for static analysis in GraalVM's native-image, some programs start to miss-behave because of assumptions about the object identity of lambdas. Note that `disableEagerInitialization` is still ineffective in the following situations: * when `useImplMethodHandle` is true, i.e., when a MethodHanlde is used to call the target because the generated hidden class is missing the necessary access rights. (the implementation require setting a static field on the generated class which causes it to be initialized, Class Data could help in the future in that case) * for non-capturing lambdas when the caller (and generated) class is in the `java.lang.invoke` or `sun.invoke.util` package. (because `findStaticGetter` will eagerly initialize the holder class if it is from those packages, see DirectMethodHandle#shouldBeInitialized) Those are acceptable rare corner cases. Thanks, Gilles From Alan.Bateman at oracle.com Tue Jun 23 18:23:46 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 23 Jun 2020 19:23:46 +0100 Subject: [15] RFR JDK-8247785: Small clarification to the javadoc about builtin class loaders In-Reply-To: <37776e19-a225-9547-6d53-147aa69679b8@oracle.com> References: <37776e19-a225-9547-6d53-147aa69679b8@oracle.com> Message-ID: <43d00426-80f1-7e1e-d397-d82d6646d835@oracle.com> On 23/06/2020 19:03, Mandy Chung wrote: > Small clarification about the parent of the system class loader in the > ClassLoader class > spec: > > > diff --git a/src/java.base/share/classes/java/lang/ClassLoader.java > b/src/java.base/share/classes/java/lang/ClassLoader.java > --- a/src/java.base/share/classes/java/lang/ClassLoader.java > +++ b/src/java.base/share/classes/java/lang/ClassLoader.java > @@ -133,7 +133,7 @@ > ? *???? It is the virtual machine's built-in class loader, typically > represented > ? *???? as {@code null}, and does not have a parent. > ? *

    • {@linkplain #getPlatformClassLoader() Platform class loader}. > - *???? All platform classes are visible to the platform > class loader > + *???? Platform classes are visible the platform class loader > ? *???? that can be used as the parent of a {@code ClassLoader} instance. > ? *???? Platform classes include Java SE platform APIs, their > implementation > ? *???? classes and JDK-specific run-time classes that are defined by the > @@ -152,7 +152,7 @@ > ? *???? The system class loader is typically used to define classes on > the > ? *???? application class path, module path, and JDK-specific tools. > ? *???? The platform class loader is a parent or an ancestor of the > system class > - *???? loader that all platform classes are visible to it.

    • > + *???? loader which searches and loads the platform classes through > its parent. > ? * > ? * > ? *

      Normally, the Java virtual machine loads classes from the > local file I think this looks okay. -Alan From brian.goetz at oracle.com Tue Jun 23 18:38:09 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 23 Jun 2020 14:38:09 -0400 Subject: RFR: JDK-8242451 : ensure semantics of non-capturing lambdas are preserved independent of execution mode In-Reply-To: References: Message-ID: <5060a413-f841-45ae-4043-e85c6d375ec2@oracle.com> On 6/23/2020 2:08 PM, Gilles Duboscq wrote: > In 8232806, a system property was introduce to disable eager initialization of the classes generated by the InnerClassLambdaMetafactory (`jdk.internal.lambda.disableEagerInitialization`). > > However, when `disableEagerInitialization` is true, even for non-capturing lambdas, the capturing lambda path that uses a method handle to the constructor is taken. > This helps prevent eager initialization but also has the side effect of always returning a fresh instance of the lambda for every invocation instead of a singleton. > While this is allowed by the specs, this might change the behaviour of some (incorrect) programs that assume a singleton is used for non-capturing lambdas. I need to reiterate that such programs are broken, and coddling such programs is likely to backfire in the long run.? When we switch to using inline classes for lambda proxies, for example, programs depending on the identity of lambda proxies likely will break, and tough luck on them.? The JLS was exceedingly clear from day 1 that the identity behavior of a lambda proxy is an implementation detail that should be _expected_ to change. All that said, the change here seems fairly restrained, and I understand there are other motivations here.? I have a few concerns, which I think should be easily addressed: ?- I would prefer not to use Lookup.IMPL_LOOKUP.? We have been on a campaign to _reduce_ the use of IMPL_LOOKUP in code like this, both based on "principle of least privilege" as well as because we would like to move this code out of JLI into `java.lang.invoke` as soon as practical (this has been queued up behind some other changes).? Is there a translation strategy you can see that doesn't require IMPL_LOOKUP?? It seems reasonable to make the field accessible, since it is a cached instance that is freely dispensed to anyone who asks anyway.? The hidden class generation produces a Lookup for the class as part of the process; getting access to it might complicate the patch a little bit, but not doing so is creating fresh technical debt in the way of a refactor we have been trying to get to for a while. ?- I'm having a hard time reconciling this patch against the JDK tip, which incorporates the changes for Hidden Classes.? Was this patch done against an older version, or is this patch destined only for an update release? ?- Assuming we can resolve the above, I would like Mandy to review these as she has most recently been in this code (eliminating the dependence on `Unsafe::defineAnonymousClass` with the newly supported "hidden classes.") From Roger.Riggs at oracle.com Tue Jun 23 19:01:27 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Tue, 23 Jun 2020 15:01:27 -0400 Subject: [15] RFR JDK-8247785: Small clarification to the javadoc about builtin class loaders In-Reply-To: <43d00426-80f1-7e1e-d397-d82d6646d835@oracle.com> References: <37776e19-a225-9547-6d53-147aa69679b8@oracle.com> <43d00426-80f1-7e1e-d397-d82d6646d835@oracle.com> Message-ID: <0ddc3a45-6af7-1d6a-5112-e9a553a929ea@oracle.com> Hi Mandy, There may be a missing "to" in: + *???? Platform classes are visible the platform class loader ++ *???? Platform classes are visible *via* the platform class loader The second change seems to be self referential using "parent" to define itself. And pre-existing in the description of getSystemClassLoader: * The platform class loader is a parent or an ancestor of the system class * loader that all platform classes are visible to it. Is missing "so" in : * loader so that all platform classes are visible to it. Both paragraphs are difficult to read and understand. (? I think the originals are more readable). Thanks, Roger On 6/23/20 2:23 PM, Alan Bateman wrote: > On 23/06/2020 19:03, Mandy Chung wrote: >> Small clarification about the parent of the system class loader in >> the ClassLoader class >> spec: >> >> >> diff --git a/src/java.base/share/classes/java/lang/ClassLoader.java >> b/src/java.base/share/classes/java/lang/ClassLoader.java >> --- a/src/java.base/share/classes/java/lang/ClassLoader.java >> +++ b/src/java.base/share/classes/java/lang/ClassLoader.java >> @@ -133,7 +133,7 @@ >> ? *???? It is the virtual machine's built-in class loader, typically >> represented >> ? *???? as {@code null}, and does not have a parent. >> ? *

    • {@linkplain #getPlatformClassLoader() Platform class loader}. >> - *???? All platform classes are visible to the platform >> class loader >> + *???? Platform classes are visible the platform class loader >> ? *???? that can be used as the parent of a {@code ClassLoader} >> instance. >> ? *???? Platform classes include Java SE platform APIs, their >> implementation >> ? *???? classes and JDK-specific run-time classes that are defined by >> the >> @@ -152,7 +152,7 @@ >> ? *???? The system class loader is typically used to define classes >> on the >> ? *???? application class path, module path, and JDK-specific tools. >> ? *???? The platform class loader is a parent or an ancestor of the >> system class >> - *???? loader that all platform classes are visible to it.

    • >> + *???? loader which searches and loads the platform classes through >> its parent. >> ? * >> ? * >> ? *

      Normally, the Java virtual machine loads classes from the >> local file > I think this looks okay. > > -Alan From brent.christian at oracle.com Tue Jun 23 19:15:28 2020 From: brent.christian at oracle.com (Brent Christian) Date: Tue, 23 Jun 2020 12:15:28 -0700 Subject: [15] RFR JDK-8247785: Small clarification to the javadoc about builtin class loaders In-Reply-To: <0ddc3a45-6af7-1d6a-5112-e9a553a929ea@oracle.com> References: <37776e19-a225-9547-6d53-147aa69679b8@oracle.com> <43d00426-80f1-7e1e-d397-d82d6646d835@oracle.com> <0ddc3a45-6af7-1d6a-5112-e9a553a929ea@oracle.com> Message-ID: <467aa0eb-614c-4e08-a60c-9b16f0f5f587@oracle.com> Hi, Mandy For: @@ -152,7 +152,7 @@ * The system class loader is typically used to define classes on the * application class path, module path, and JDK-specific tools. * The platform class loader is a parent or an ancestor of the system class - * loader that all platform classes are visible to it. + * loader which searches and loads the platform classes through its parent. * * IMO, the function of the platform class loader has already been outlined in the previous bullet point, so the last sentence could be simplified to something like: "The system class loader's parent is the platform class loader" or "The parent of the system class loader is the platform class loader." -Brent On 6/23/20 12:01 PM, Roger Riggs wrote: > Hi Mandy, > > There may be a missing "to" in: > > + *???? Platform classes are visible the platform class loader > ++ *???? Platform classes are visible *via* the platform class > loader > > The second change seems to be self referential using "parent" to define > itself. > > And pre-existing in the description of getSystemClassLoader: > > * The platform class loader is a parent or an ancestor of the system > class * loader that all platform classes are visible to it. > > Is missing "so" in : > > * loader so that all platform classes are visible to it. > > Both paragraphs are difficult to read and understand. (? I think the > originals are more readable). > > Thanks, Roger > > > > On 6/23/20 2:23 PM, Alan Bateman wrote: >> On 23/06/2020 19:03, Mandy Chung wrote: >>> Small clarification about the parent of the system class loader in >>> the ClassLoader class >>> spec: >>> >>> >>> diff --git a/src/java.base/share/classes/java/lang/ClassLoader.java >>> b/src/java.base/share/classes/java/lang/ClassLoader.java >>> --- a/src/java.base/share/classes/java/lang/ClassLoader.java >>> +++ b/src/java.base/share/classes/java/lang/ClassLoader.java >>> @@ -133,7 +133,7 @@ >>> ? *???? It is the virtual machine's built-in class loader, typically >>> represented >>> ? *???? as {@code null}, and does not have a parent. >>> ? *

    • {@linkplain #getPlatformClassLoader() Platform class loader}. >>> - *???? All platform classes are visible to the platform >>> class loader >>> + *???? Platform classes are visible the platform class loader >>> ? *???? that can be used as the parent of a {@code ClassLoader} >>> instance. >>> ? *???? Platform classes include Java SE platform APIs, their >>> implementation >>> ? *???? classes and JDK-specific run-time classes that are defined by >>> the >>> @@ -152,7 +152,7 @@ >>> ? *???? The system class loader is typically used to define classes >>> on the >>> ? *???? application class path, module path, and JDK-specific tools. >>> ? *???? The platform class loader is a parent or an ancestor of the >>> system class >>> - *???? loader that all platform classes are visible to it.

    • >>> + *???? loader which searches and loads the platform classes through >>> its parent. >>> ? * >>> ? * >>> ? *

      Normally, the Java virtual machine loads classes from the >>> local file >> I think this looks okay. >> >> -Alan > From mandy.chung at oracle.com Tue Jun 23 19:22:33 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 23 Jun 2020 12:22:33 -0700 Subject: [15] RFR JDK-8247785: Small clarification to the javadoc about builtin class loaders In-Reply-To: <467aa0eb-614c-4e08-a60c-9b16f0f5f587@oracle.com> References: <37776e19-a225-9547-6d53-147aa69679b8@oracle.com> <43d00426-80f1-7e1e-d397-d82d6646d835@oracle.com> <0ddc3a45-6af7-1d6a-5112-e9a553a929ea@oracle.com> <467aa0eb-614c-4e08-a60c-9b16f0f5f587@oracle.com> Message-ID: <41a5bd0a-5a0e-e20e-3a51-827d79515a36@oracle.com> On 6/23/20 12:15 PM, Brent Christian wrote: > Hi, Mandy > > For: > > @@ -152,7 +152,7 @@ > ? *???? The system class loader is typically used to define classes on > the > ? *???? application class path, module path, and JDK-specific tools. > ? *???? The platform class loader is a parent or an ancestor of the > system class > - *???? loader that all platform classes are visible to it. > + *???? loader which searches and loads the platform classes through > its parent. > ? * > ? * > > IMO, the function of the platform class loader has already been > outlined in the previous bullet point, so the last sentence could be > simplified to something like: > I considered dropping that too. > "The system class loader's parent is the platform class loader" > or > "The parent of the system class loader is the platform class loader." > But it could be an ancestor. Mandy > -Brent > > On 6/23/20 12:01 PM, Roger Riggs wrote: >> Hi Mandy, >> >> There may be a missing "to" in: >> >> + *???? Platform classes are visible the platform class loader >> ++ *???? Platform classes are visible *via* the platform >> class loader >> >> The second change seems to be self referential using "parent" to >> define itself. >> >> And pre-existing in the description of getSystemClassLoader: >> >> * The platform class loader is a parent or an ancestor of the system >> class * loader that all platform classes are visible to it. >> >> Is missing "so" in : >> >> * loader so that all platform classes are visible to it. >> >> Both paragraphs are difficult to read and understand. (? I think the >> originals are more readable). >> >> Thanks, Roger >> >> >> >> On 6/23/20 2:23 PM, Alan Bateman wrote: >>> On 23/06/2020 19:03, Mandy Chung wrote: >>>> Small clarification about the parent of the system class loader in >>>> the ClassLoader class >>>> spec: >>>> >>>> >>>> diff --git a/src/java.base/share/classes/java/lang/ClassLoader.java >>>> b/src/java.base/share/classes/java/lang/ClassLoader.java >>>> --- a/src/java.base/share/classes/java/lang/ClassLoader.java >>>> +++ b/src/java.base/share/classes/java/lang/ClassLoader.java >>>> @@ -133,7 +133,7 @@ >>>> ? *???? It is the virtual machine's built-in class loader, >>>> typically represented >>>> ? *???? as {@code null}, and does not have a parent. >>>> ? *

    • {@linkplain #getPlatformClassLoader() Platform class >>>> loader}. >>>> - *???? All platform classes are visible to the platform >>>> class loader >>>> + *???? Platform classes are visible the platform class >>>> loader >>>> ? *???? that can be used as the parent of a {@code ClassLoader} >>>> instance. >>>> ? *???? Platform classes include Java SE platform APIs, their >>>> implementation >>>> ? *???? classes and JDK-specific run-time classes that are defined >>>> by the >>>> @@ -152,7 +152,7 @@ >>>> ? *???? The system class loader is typically used to define classes >>>> on the >>>> ? *???? application class path, module path, and JDK-specific tools. >>>> ? *???? The platform class loader is a parent or an ancestor of the >>>> system class >>>> - *???? loader that all platform classes are visible to it.

    • >>>> + *???? loader which searches and loads the platform classes >>>> through its parent. >>>> ? * >>>> ? * >>>> ? *

      Normally, the Java virtual machine loads classes from the >>>> local file >>> I think this looks okay. >>> >>> -Alan >> From mandy.chung at oracle.com Tue Jun 23 19:55:53 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 23 Jun 2020 12:55:53 -0700 Subject: RFR: JDK-8242451 : ensure semantics of non-capturing lambdas are preserved independent of execution mode In-Reply-To: <5060a413-f841-45ae-4043-e85c6d375ec2@oracle.com> References: <5060a413-f841-45ae-4043-e85c6d375ec2@oracle.com> Message-ID: <696554fb-8997-8ccd-81e3-174cb97ffd6d@oracle.com> On 6/23/20 11:38 AM, Brian Goetz wrote: > > > On 6/23/2020 2:08 PM, Gilles Duboscq wrote: >> In 8232806, a system property was introduce to disable eager >> initialization of the classes generated by the >> InnerClassLambdaMetafactory >> (`jdk.internal.lambda.disableEagerInitialization`). >> >> However, when `disableEagerInitialization` is true, even for >> non-capturing lambdas, the capturing lambda path that uses a method >> handle to the constructor is taken. >> This helps prevent eager initialization but also has the side effect >> of always returning a fresh instance of the lambda for every >> invocation instead of a singleton. >> While this is allowed by the specs, this might change the behaviour >> of some (incorrect) programs that assume a singleton is used for >> non-capturing lambdas. > > I need to reiterate that such programs are broken, and coddling such > programs is likely to backfire in the long run.? When we switch to > using inline classes for lambda proxies, for example, programs > depending on the identity of lambda proxies likely will break, and > tough luck on them. Agree.?? Such programs will break sooner or later.?? Do you have any plan to act on fixing these programs? > The JLS was exceedingly clear from day 1 that the identity behavior of > a lambda proxy is an implementation detail that should be _expected_ > to change. > > All that said, the change here seems fairly restrained, and I > understand there are other motivations here.? I have a few concerns, > which I think should be easily addressed: > > ?- I would prefer not to use Lookup.IMPL_LOOKUP.? We have been on a > campaign to _reduce_ the use of IMPL_LOOKUP in code like this, both > based on "principle of least privilege" as well as because we would > like to move this code out of JLI into `java.lang.invoke` as soon as > practical (this has been queued up behind some other changes). LMF and its implementation classes do not use IMPL_LOOKUP any more.?? I have a patch to reduce other use of IMPL_LOOKUP even further. > Is there a translation strategy you can see that doesn't require > IMPL_LOOKUP?? It seems reasonable to make the field accessible, since > it is a cached instance that is freely dispensed to anyone who asks > anyway.? The hidden class generation produces a Lookup for the class > as part of the process; getting access to it might complicate the > patch a little bit, but not doing so is creating fresh technical debt > in the way of a refactor we have been trying to get to for a while. > You can use the caller Lookup.? The hidden class is a nestmate of the caller class and the caller Lookup can access private fields of the hidden class. > ?- I'm having a hard time reconciling this patch against the JDK tip, > which incorporates the changes for Hidden Classes.? Was this patch > done against an older version, or is this patch destined only for an > update release? > > ?- Assuming we can resolve the above, I would like Mandy to review > these as she has most recently been in this code (eliminating the > dependence on `Unsafe::defineAnonymousClass` with the newly supported > "hidden classes.") I'll post further comment. Should this patch be a workaround to existing releases rather than the main line??? As Brian mentions, lambda proxy class may become inline class in valhalla repo (Roger has a patch already).?? The earlier fixing those programs the better. Mandy From mandy.chung at oracle.com Tue Jun 23 21:42:25 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 23 Jun 2020 14:42:25 -0700 Subject: [15] RFR JDK-8247785: Small clarification to the javadoc about builtin class loaders In-Reply-To: <0ddc3a45-6af7-1d6a-5112-e9a553a929ea@oracle.com> References: <37776e19-a225-9547-6d53-147aa69679b8@oracle.com> <43d00426-80f1-7e1e-d397-d82d6646d835@oracle.com> <0ddc3a45-6af7-1d6a-5112-e9a553a929ea@oracle.com> Message-ID: <81d4a655-76ad-2e53-d83c-724c385f2c7b@oracle.com> On 6/23/20 12:01 PM, Roger Riggs wrote: > Hi Mandy, > > There may be a missing "to" in: > > + *???? Platform classes are visible the platform class loader > ++ *???? Platform classes are visible *via* the platform > class loader > I caught this accidental change too. > The second change seems to be self referential using "parent" to > define itself. > > And pre-existing in the description of getSystemClassLoader: > > * The platform class loader is a parent or an ancestor of the system > class * loader that all platform classes are visible to it. > > Is missing "so" in : > > * loader so that all platform classes are visible to it. > > Both paragraphs are difficult to read and understand. (? I think the > originals are more readable). I made a minor adjustment to break the sentence into two.? That should help. diff --git a/src/java.base/share/classes/java/lang/ClassLoader.java b/src/java.base/share/classes/java/lang/ClassLoader.java --- a/src/java.base/share/classes/java/lang/ClassLoader.java +++ b/src/java.base/share/classes/java/lang/ClassLoader.java @@ -133,8 +133,9 @@ ? *???? It is the virtual machine's built-in class loader, typically represented ? *???? as {@code null}, and does not have a parent. ? *

    • {@linkplain #getPlatformClassLoader() Platform class loader}. - *???? All platform classes are visible to the platform class loader - *???? that can be used as the parent of a {@code ClassLoader} instance. + *???? Platform classes are visible to the platform class loader. + *???? The platform class loader can be used as the parent of a {@code ClassLoader} + *???? instance. ? *???? Platform classes include Java SE platform APIs, their implementation ? *???? classes and JDK-specific run-time classes that are defined by the ? *???? platform class loader or its ancestors. @@ -152,7 +153,7 @@ ? *???? The system class loader is typically used to define classes on the ? *???? application class path, module path, and JDK-specific tools. ? *???? The platform class loader is a parent or an ancestor of the system class - *???? loader that all platform classes are visible to it.

    • + *???? loader.? It searches and loads the platform classes through its parent. ? * ? * ? *

      Normally, the Java virtual machine loads classes from the local file Thanks Mandy From naoto.sato at oracle.com Tue Jun 23 21:44:56 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Tue, 23 Jun 2020 14:44:56 -0700 Subject: RFR 8248184: AMPM_OF_DAY doc fix in ChronoField Message-ID: <8ee7fdd8-b4ec-2be2-fbf9-60968c8b286b@oracle.com> Hi, Please review this small doc fix for the following issue: https://bugs.openjdk.java.net/browse/JDK-8248184 The proposed patch is here: --- old/src/java.base/share/classes/java/time/temporal/ChronoField.java 2020-06-23 14:43:43.000000000 -0700 +++ new/src/java.base/share/classes/java/time/temporal/ChronoField.java 2020-06-23 14:43:43.000000000 -0700 @@ -270,7 +270,7 @@ * The value is validated from 0 to 11 in strict and smart mode. * In lenient mode the value is not validated. It is combined with * {@code AMPM_OF_DAY} to form {@code HOUR_OF_DAY} by multiplying - * the {AMPM_OF_DAY} value by 12. + * the {@code AMPM_OF_DAY} value by 12. *

      * See {@link #CLOCK_HOUR_OF_AMPM} for the related field that counts hours from 1 to 12. */ @@ -334,7 +334,7 @@ * The value is validated from 0 to 1 in strict and smart mode. * In lenient mode the value is not validated. It is combined with * {@code HOUR_OF_AMPM} to form {@code HOUR_OF_DAY} by multiplying - * the {AMPM_OF_DAY} value by 12. + * the {@code AMPM_OF_DAY} value by 12. */ AMPM_OF_DAY("AmPmOfDay", HALF_DAYS, DAYS, ValueRange.of(0, 1), "dayperiod"), Naoto From brian.burkhalter at oracle.com Tue Jun 23 21:55:21 2020 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 23 Jun 2020 14:55:21 -0700 Subject: RFR 8248184: AMPM_OF_DAY doc fix in ChronoField In-Reply-To: <8ee7fdd8-b4ec-2be2-fbf9-60968c8b286b@oracle.com> References: <8ee7fdd8-b4ec-2be2-fbf9-60968c8b286b@oracle.com> Message-ID: <09FE5824-84E8-4560-BE30-759332BA324E@oracle.com> Hi Naoto, +1 Brian > On Jun 23, 2020, at 2:44 PM, naoto.sato at oracle.com wrote: > > Hi, > > Please review this small doc fix for the following issue: > > https://bugs.openjdk.java.net/browse/JDK-8248184 > > The proposed patch is here: > > --- old/src/java.base/share/classes/java/time/temporal/ChronoField.java 2020-06-23 14:43:43.000000000 -0700 > +++ new/src/java.base/share/classes/java/time/temporal/ChronoField.java 2020-06-23 14:43:43.000000000 -0700 > @@ -270,7 +270,7 @@ > * The value is validated from 0 to 11 in strict and smart mode. > * In lenient mode the value is not validated. It is combined with > * {@code AMPM_OF_DAY} to form {@code HOUR_OF_DAY} by multiplying > - * the {AMPM_OF_DAY} value by 12. > + * the {@code AMPM_OF_DAY} value by 12. > *

      > * See {@link #CLOCK_HOUR_OF_AMPM} for the related field that counts hours from 1 to 12. > */ > @@ -334,7 +334,7 @@ > * The value is validated from 0 to 1 in strict and smart mode. > * In lenient mode the value is not validated. It is combined with > * {@code HOUR_OF_AMPM} to form {@code HOUR_OF_DAY} by multiplying > - * the {AMPM_OF_DAY} value by 12. > + * the {@code AMPM_OF_DAY} value by 12. > */ > AMPM_OF_DAY("AmPmOfDay", HALF_DAYS, DAYS, ValueRange.of(0, 1), "dayperiod"), > > Naoto From LANCE.ANDERSEN at ORACLE.COM Tue Jun 23 22:09:00 2020 From: LANCE.ANDERSEN at ORACLE.COM (Lance Andersen) Date: Tue, 23 Jun 2020 18:09:00 -0400 Subject: RFR 8248184: AMPM_OF_DAY doc fix in ChronoField In-Reply-To: <8ee7fdd8-b4ec-2be2-fbf9-60968c8b286b@oracle.com> References: <8ee7fdd8-b4ec-2be2-fbf9-60968c8b286b@oracle.com> Message-ID: +1 Best Lance ------------------ Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com > On Jun 23, 2020, at 5:44 PM, naoto.sato at oracle.com wrote: > > Hi, > > Please review this small doc fix for the following issue: > > https://bugs.openjdk.java.net/browse/JDK-8248184 > > The proposed patch is here: > > --- old/src/java.base/share/classes/java/time/temporal/ChronoField.java 2020-06-23 14:43:43.000000000 -0700 > +++ new/src/java.base/share/classes/java/time/temporal/ChronoField.java 2020-06-23 14:43:43.000000000 -0700 > @@ -270,7 +270,7 @@ > * The value is validated from 0 to 11 in strict and smart mode. > * In lenient mode the value is not validated. It is combined with > * {@code AMPM_OF_DAY} to form {@code HOUR_OF_DAY} by multiplying > - * the {AMPM_OF_DAY} value by 12. > + * the {@code AMPM_OF_DAY} value by 12. > *

      > * See {@link #CLOCK_HOUR_OF_AMPM} for the related field that counts hours from 1 to 12. > */ > @@ -334,7 +334,7 @@ > * The value is validated from 0 to 1 in strict and smart mode. > * In lenient mode the value is not validated. It is combined with > * {@code HOUR_OF_AMPM} to form {@code HOUR_OF_DAY} by multiplying > - * the {AMPM_OF_DAY} value by 12. > + * the {@code AMPM_OF_DAY} value by 12. > */ > AMPM_OF_DAY("AmPmOfDay", HALF_DAYS, DAYS, ValueRange.of(0, 1), "dayperiod"), > > Naoto From alexander.matveev at oracle.com Tue Jun 23 22:11:29 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Tue, 23 Jun 2020 15:11:29 -0700 Subject: RFR: JDK-8246212: JPKG001-012: application icon is missing in Control Panel Add/Remove In-Reply-To: <01ebf9e4-9049-b33d-e395-798706828b74@oracle.com> References: <4efc0137-d2fb-cf69-1fac-3e871d5eedc4@oracle.com> <01ebf9e4-9049-b33d-e395-798706828b74@oracle.com> Message-ID: <48cac80c-1d10-bbd8-9559-d8e1481b23ec@oracle.com> Hi Alexey, Looks good. Thanks, Alexander On 6/23/20 10:53 AM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > Fix how icon is configured for installers on Windows. The value of > ARPPRODUCTICON property should point to an entry in Icon table of msi > rather to a path of icon file in install directory. The issue shows up > only in downgrade scenarios. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8246212 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8246212/webrev.00 > From andy.herrick at oracle.com Tue Jun 23 22:50:04 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Tue, 23 Jun 2020 18:50:04 -0400 Subject: RFR: JDK-8246212: JPKG001-012: application icon is missing in Control Panel Add/Remove In-Reply-To: <01ebf9e4-9049-b33d-e395-798706828b74@oracle.com> References: <4efc0137-d2fb-cf69-1fac-3e871d5eedc4@oracle.com> <01ebf9e4-9049-b33d-e395-798706828b74@oracle.com> Message-ID: looks good /Andy On 6/23/2020 1:53 PM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > Fix how icon is configured for installers on Windows. The value of > ARPPRODUCTICON property should point to an entry in Icon table of msi > rather to a path of icon file in install directory. The issue shows up > only in downgrade scenarios. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8246212 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8246212/webrev.00 > From naoto.sato at oracle.com Wed Jun 24 00:47:18 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Tue, 23 Jun 2020 17:47:18 -0700 Subject: RFR 8248184: AMPM_OF_DAY doc fix in ChronoField In-Reply-To: References: <8ee7fdd8-b4ec-2be2-fbf9-60968c8b286b@oracle.com> Message-ID: Thanks, Brian and Lance. I forgot to update the copyright year. Pushed the changeset with the year update. Naoto On 6/23/20 3:09 PM, Lance Andersen wrote: > +1 > > Best > Lance > ------------------ > > > > Lance Andersen| Principal?Member of Technical Staff |?+1.781.442.2037 > Oracle?Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > > >> On Jun 23, 2020, at 5:44 PM, naoto.sato at oracle.com >> wrote: >> >> Hi, >> >> Please review this small doc fix for the following issue: >> >> https://bugs.openjdk.java.net/browse/JDK-8248184 >> >> The proposed patch is here: >> >> --- >> old/src/java.base/share/classes/java/time/temporal/ChronoField.java >> 2020-06-23 14:43:43.000000000 -0700 >> +++ >> new/src/java.base/share/classes/java/time/temporal/ChronoField.java >> 2020-06-23 14:43:43.000000000 -0700 >> @@ -270,7 +270,7 @@ >> ?????* The value is validated from 0 to 11 in strict and smart mode. >> ?????* In lenient mode the value is not validated. It is combined with >> ?????* {@code AMPM_OF_DAY} to form {@code HOUR_OF_DAY} by multiplying >> - ????* the {AMPM_OF_DAY} value by 12. >> + ????* the {@code AMPM_OF_DAY} value by 12. >> ?????*

      >> ?????* See {@link #CLOCK_HOUR_OF_AMPM} for the related field that >> counts hours from 1 to 12. >> ?????*/ >> @@ -334,7 +334,7 @@ >> ?????* The value is validated from 0 to 1 in strict and smart mode. >> ?????* In lenient mode the value is not validated. It is combined with >> ?????* {@code HOUR_OF_AMPM} to form {@code HOUR_OF_DAY} by multiplying >> - ????* the {AMPM_OF_DAY} value by 12. >> + ????* the {@code AMPM_OF_DAY} value by 12. >> ?????*/ >> ????AMPM_OF_DAY("AmPmOfDay", HALF_DAYS, DAYS, ValueRange.of(0, 1), >> "dayperiod"), >> >> Naoto > From mandy.chung at oracle.com Wed Jun 24 00:56:33 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 23 Jun 2020 17:56:33 -0700 Subject: RFR: JDK-8242451 : ensure semantics of non-capturing lambdas are preserved independent of execution mode In-Reply-To: References: Message-ID: <71c26ec5-7512-e5e0-349f-a488357677f1@oracle.com> Hi Gilles, Additional comments: 215 try { 216 return new ConstantCallSite(Lookup.IMPL_LOOKUP.findStaticGetter(innerClass, LAMBDA_INSTANCE_FIELD, invokedType.returnType())); 217 } catch (ReflectiveOperationException e) { 218 throw new LambdaConversionException("Exception finding constructor", e); 219 } This should use caller instead Lookup.IMPL_LOOKUP (as I suggested in my previous repl).?? The exception message should be "Exception finding " + LAMBDA_INSTANCE_FIELD + " static field". 418 private void generateStaticField() { I would rename this to generateClassInitializer since this adds "" and initializes the static field. Since this patch caches a singleton instance in the generated class, it could apply to the eager initialization case as well and replace the current use of core reflection to new an instance except that the target of the returning callsite would always be the singleton object (the result of invoking the static getter method handle).? I wonder if there is any performance difference.? This is just a thought that we can file a JBS issue to follow up. Can you add a test case for this fix? Mandy On 6/23/20 11:08 AM, Gilles Duboscq wrote: > In 8232806, a system property was introduce to disable eager initialization of the classes generated by the InnerClassLambdaMetafactory (`jdk.internal.lambda.disableEagerInitialization`). > > However, when `disableEagerInitialization` is true, even for non-capturing lambdas, the capturing lambda path that uses a method handle to the constructor is taken. > This helps prevent eager initialization but also has the side effect of always returning a fresh instance of the lambda for every invocation instead of a singleton. > While this is allowed by the specs, this might change the behaviour of some (incorrect) programs that assume a singleton is used for non-capturing lambdas. > > I propose to keep the effects of the `disableEagerInitialization` related to initialization only. > > Webrev: https://cr.openjdk.java.net/~gdub/8242451/webrev.0/ > Issue: https://bugs.openjdk.java.net/browse/JDK-8242451 > > The concrete issue we are seeing with changing both aspects at the same time is that when disableEagerInitialization for static analysis in GraalVM's native-image, some programs start to miss-behave because of assumptions about the object identity of lambdas. > > > Note that `disableEagerInitialization` is still ineffective in the following situations: > * when `useImplMethodHandle` is true, i.e., when a MethodHanlde is used to call the target because the generated hidden class is missing the necessary access rights. > (the implementation require setting a static field on the generated class which causes it to be initialized, Class Data could help in the future in that case) > * for non-capturing lambdas when the caller (and generated) class is in the `java.lang.invoke` or `sun.invoke.util` package. > (because `findStaticGetter` will eagerly initialize the holder class if it is from those packages, see DirectMethodHandle#shouldBeInitialized) > > Those are acceptable rare corner cases. > > Thanks, > Gilles From joe.darcy at oracle.com Wed Jun 24 01:43:48 2020 From: joe.darcy at oracle.com (Joe Darcy) Date: Tue, 23 Jun 2020 18:43:48 -0700 Subject: Exceptions priority in InputStream.read(byte[], int, int) In-Reply-To: <813265b5-ba2c-94fb-6eea-9622c4946578@oracle.com> References: <2936d5cc-edcf-d005-1e2e-fe16430beb3c@gmail.com> <813265b5-ba2c-94fb-6eea-9622c4946578@oracle.com> Message-ID: Hello, As a general comment, over time we come to regard specifications like "if these three pre-conditions of the method are all violated, this exception will be thrown; if only these two pre-conditions are violated, this other exception will be thrown" as over specifications and best avoided. If any of a method's exceptional conditions are true, the caller should be prepared to handle any of the possible exceptions. HTH, -Joe On 6/23/2020 8:03 AM, Roger Riggs wrote: > Hi Raffaello, > > Exceptions related to the arguments should be highest priority so they > are > not obscured by any more dynamic state related to the stream. > > There's no specific order between checks of the arguments, > though in this case the NPE would naturally occur before IOOBE. > There are more complex cases where some arguments are not used > or used conditionally where the checks become dependent on the dynamic > state. > > $.02, Roger > > > On 6/23/20 10:34 AM, Raffaello Giulietti wrote: >> Hi, >> >> the InputStream.read(byte[], int, int) method [1] can throw >> * IOException >> * NullPointerException >> * IndexOutOfBoundsException >> >> Is there a recommended priority for the conditions associated to the >> exceptions to be checked? For example, if the arguments are invalid >> and the stream is already closed, which exception shall be thrown? >> >> >> Greetings >> Raffaello >> >> ---- >> >> [1] >> https://docs.oracle.com/en/java/javase/14/docs/api/java.base/java/io/InputStream.html#read(byte%5B%5D,int,int) > From david.holmes at oracle.com Wed Jun 24 02:48:07 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 24 Jun 2020 12:48:07 +1000 Subject: [15] RFR JDK-8247785: Small clarification to the javadoc about builtin class loaders In-Reply-To: <81d4a655-76ad-2e53-d83c-724c385f2c7b@oracle.com> References: <37776e19-a225-9547-6d53-147aa69679b8@oracle.com> <43d00426-80f1-7e1e-d397-d82d6646d835@oracle.com> <0ddc3a45-6af7-1d6a-5112-e9a553a929ea@oracle.com> <81d4a655-76ad-2e53-d83c-724c385f2c7b@oracle.com> Message-ID: <9e1a193f-12ee-174f-d4dd-d74a8e665b5f@oracle.com> Hi Mandy, The trouble with small clarifications is that they tend to draw attention to larger issues :) On 24/06/2020 7:42 am, Mandy Chung wrote: > > > On 6/23/20 12:01 PM, Roger Riggs wrote: >> Hi Mandy, >> >> There may be a missing "to" in: >> >> + *???? Platform classes are visible the platform class loader >> ++ *???? Platform classes are visible *via* the platform >> class loader >> > > I caught this accidental change too. > >> The second change seems to be self referential using "parent" to >> define itself. >> >> And pre-existing in the description of getSystemClassLoader: >> >> * The platform class loader is a parent or an ancestor of the system >> class * loader that all platform classes are visible to it. >> >> Is missing "so" in : >> >> * loader so that all platform classes are visible to it. >> >> Both paragraphs are difficult to read and understand. (? I think the >> originals are more readable). > > I made a minor adjustment to break the sentence into two.? That should > help. Reading the javadoc I'm left somewhat confused about the role of the Bootstrap class loader versus the Platform class loader these days. What classes does the Bootstrap class loader actually load? > diff --git a/src/java.base/share/classes/java/lang/ClassLoader.java > b/src/java.base/share/classes/java/lang/ClassLoader.java > --- a/src/java.base/share/classes/java/lang/ClassLoader.java > +++ b/src/java.base/share/classes/java/lang/ClassLoader.java > @@ -133,8 +133,9 @@ > ? *???? It is the virtual machine's built-in class loader, typically > represented > ? *???? as {@code null}, and does not have a parent. > ? *

    • {@linkplain #getPlatformClassLoader() Platform class loader}. > - *???? All platform classes are visible to the platform class > loader > - *???? that can be used as the parent of a {@code ClassLoader} instance. > + *???? Platform classes are visible to the platform class loader. That seems somewhat self-defining / tautological. To me it would be clearer to say something like: - Platform class loader. This class loader is responsible for loading the platform classes. Platform classes include ... The Platform class loader can be used as the parent ... > + *???? The platform class loader can be used as the parent of a {@code > ClassLoader} > + *???? instance. > ? *???? Platform classes include Java SE platform APIs, their > implementation > ? *???? classes and JDK-specific run-time classes that are defined by the > ? *???? platform class loader or its ancestors. > @@ -152,7 +153,7 @@ > ? *???? The system class loader is typically used to define classes on the > ? *???? application class path, module path, and JDK-specific tools. > ? *???? The platform class loader is a parent or an ancestor of the s/a/the/ There can only be a single parent, but multiple ancestors. > system class > - *???? loader that all platform classes are visible to it.

    • > + *???? loader.? It searches and loads the platform classes through its > parent. "It" refers to the system class loader, but the most recent sentence subject was the platform class loader, so that makes "it" ambiguous here. So I suggest s/It/The system class loader/ though I'd be tempted to restructure this more: "The platform class loader is the parent or an ancestor of the system class loader, so the system class loader can load platform classes by delegating to its parent." Cheers, David ----- > ? * > ? * > ? *

      Normally, the Java virtual machine loads classes from the local > file > > Thanks > Mandy From mandy.chung at oracle.com Wed Jun 24 03:17:19 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 23 Jun 2020 20:17:19 -0700 Subject: [15] RFR JDK-8247785: Small clarification to the javadoc about builtin class loaders In-Reply-To: <9e1a193f-12ee-174f-d4dd-d74a8e665b5f@oracle.com> References: <37776e19-a225-9547-6d53-147aa69679b8@oracle.com> <43d00426-80f1-7e1e-d397-d82d6646d835@oracle.com> <0ddc3a45-6af7-1d6a-5112-e9a553a929ea@oracle.com> <81d4a655-76ad-2e53-d83c-724c385f2c7b@oracle.com> <9e1a193f-12ee-174f-d4dd-d74a8e665b5f@oracle.com> Message-ID: <04630f57-29d1-1ccc-b8bd-d467765bb382@oracle.com> On 6/23/20 7:48 PM, David Holmes wrote: > Hi Mandy, > > The trouble with small clarifications is that they tend to draw > attention to larger issues :) > > On 24/06/2020 7:42 am, Mandy Chung wrote: >> >> >> On 6/23/20 12:01 PM, Roger Riggs wrote: >>> Hi Mandy, >>> >>> There may be a missing "to" in: >>> >>> + *???? Platform classes are visible the platform class loader >>> ++ *???? Platform classes are visible *via* the platform >>> class loader >>> >> >> I caught this accidental change too. >> >>> The second change seems to be self referential using "parent" to >>> define itself. >>> >>> And pre-existing in the description of getSystemClassLoader: >>> >>> * The platform class loader is a parent or an ancestor of the system >>> class * loader that all platform classes are visible to it. >>> >>> Is missing "so" in : >>> >>> * loader so that all platform classes are visible to it. >>> >>> Both paragraphs are difficult to read and understand. (? I think the >>> originals are more readable). >> >> I made a minor adjustment to break the sentence into two.? That >> should help. > > Reading the javadoc I'm left somewhat confused about the role of the > Bootstrap class loader versus the Platform class loader these days. > What classes does the Bootstrap class loader actually load? > This is implementation specific. JEP 261 [1] documents which Java SE and JDK module is defined to which loader in JDK 9. ? The source of truth is? make/common/Modules.gmk. https://openjdk.java.net/jeps/261#Class-loaders Mandy From david.holmes at oracle.com Wed Jun 24 05:40:54 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 24 Jun 2020 15:40:54 +1000 Subject: [15] RFR JDK-8247785: Small clarification to the javadoc about builtin class loaders In-Reply-To: <04630f57-29d1-1ccc-b8bd-d467765bb382@oracle.com> References: <37776e19-a225-9547-6d53-147aa69679b8@oracle.com> <43d00426-80f1-7e1e-d397-d82d6646d835@oracle.com> <0ddc3a45-6af7-1d6a-5112-e9a553a929ea@oracle.com> <81d4a655-76ad-2e53-d83c-724c385f2c7b@oracle.com> <9e1a193f-12ee-174f-d4dd-d74a8e665b5f@oracle.com> <04630f57-29d1-1ccc-b8bd-d467765bb382@oracle.com> Message-ID: <23904ca0-dbbe-767a-cdec-74f3698d0ff2@oracle.com> On 24/06/2020 1:17 pm, Mandy Chung wrote: > On 6/23/20 7:48 PM, David Holmes wrote: >> Hi Mandy, >> >> The trouble with small clarifications is that they tend to draw >> attention to larger issues :) >> >> On 24/06/2020 7:42 am, Mandy Chung wrote: >>> >>> >>> On 6/23/20 12:01 PM, Roger Riggs wrote: >>>> Hi Mandy, >>>> >>>> There may be a missing "to" in: >>>> >>>> + *???? Platform classes are visible the platform class loader >>>> ++ *???? Platform classes are visible *via* the platform >>>> class loader >>>> >>> >>> I caught this accidental change too. >>> >>>> The second change seems to be self referential using "parent" to >>>> define itself. >>>> >>>> And pre-existing in the description of getSystemClassLoader: >>>> >>>> * The platform class loader is a parent or an ancestor of the system >>>> class * loader that all platform classes are visible to it. >>>> >>>> Is missing "so" in : >>>> >>>> * loader so that all platform classes are visible to it. >>>> >>>> Both paragraphs are difficult to read and understand. (? I think the >>>> originals are more readable). >>> >>> I made a minor adjustment to break the sentence into two.? That >>> should help. >> >> Reading the javadoc I'm left somewhat confused about the role of the >> Bootstrap class loader versus the Platform class loader these days. >> What classes does the Bootstrap class loader actually load? >> > > This is implementation specific. JEP 261 [1] documents which Java SE and > JDK module is defined to which loader in JDK 9. ? The source of truth > is? make/common/Modules.gmk. > > https://openjdk.java.net/jeps/261#Class-loaders Thanks for that Mandy. Hard to capture all that in only a few lines of javadoc, but I think we can do better than what is presently there. Different RFE though :) David > Mandy From david.holmes at oracle.com Wed Jun 24 06:03:04 2020 From: david.holmes at oracle.com (David Holmes) Date: Wed, 24 Jun 2020 16:03:04 +1000 Subject: RFR: 8247532, 8248135: Records deserialization is slow + Build microbenchmarks with --enable-preview In-Reply-To: References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> <1DA206A1-15F0-46F9-B69E-9896C57B97C7@oracle.com> <8886ee56-e9a7-aafe-6111-f0c2c54a282e@gmail.com> <63689945-dca3-e0cc-72b8-9225f5da6d31@gmail.com> <7f7b58fa-50e0-a4fe-f948-9637d04c0ccc@oracle.com> <5021b3e7-28ad-7041-cc89-c0f218434937@oracle.com> <092241fe-1f43-ead6-6a5d-0345dac55c3e@oracle.com> <9e4e54c8-05c9-09fd-67ee-98c91ac9aeb3@gmail.com> Message-ID: <2b8beef4-7953-a727-a452-037a951bc7ac@oracle.com> Hi Chris, On 24/06/2020 2:30 am, Chris Hegarty wrote: > >> On 23 Jun 2020, at 14:49, Peter Levart wrote: >> >> ... >> >> Ok, I'm going to push this to jdk15. > > Thank you Peter. This is a really nice change. > > As a follow on, and not for JDK 15, I observe that Class::isRecord0 / JVM_IsRecord shows up as consuming a significant amount of time, more than 10%, in some runs of the deserialization benchmark. The isRecord implementation is a native method in the JVM, so relatively expensive to call. > > This shows an opportunity to improve the Class::isRecord implementation with a simple cache of the record-ness of the j.l.Class, as is done selectively with other particular aspects of a class?s state. There are various ways to implement this, but here is just one [*]. There has been reluctance to add more and more fields to Class to cache all these new attributes that are being added - but ultimately that is a call for core-libs folk to make. The general expectation is/was that the need to ask a class if it is a Record (or isSealed etc) would be rare. But (de)serialization is the exception for isRecord() as unlike enums a simple instanceof test can't be used. Cheers, David ----- > Running the deserialization benchmark with this change [*], gives the following results: > > Benchmark (length) Mode Cnt Score Error Units > RecordSerializationBench.deserializeClasses 10 avgt 10 14.136 ? 0.841 us/op > RecordSerializationBench.deserializeClasses 100 avgt 10 61.821 ? 1.279 us/op > RecordSerializationBench.deserializeClasses 1000 avgt 10 519.473 ? 7.950 us/op > RecordSerializationBench.deserializeRecords 10 avgt 10 13.781 ? 1.917 us/op > RecordSerializationBench.deserializeRecords 100 avgt 10 54.061 ? 4.188 us/op > RecordSerializationBench.deserializeRecords 1000 avgt 10 444.538 ? 13.940 us/op > > I think it is worth considering caching the record-ness state of a j.l.Class, as I?m sure it will be widely used in third-party serialization libraries, as well as by Java Serialization. > > -Chris. > > [*] > diff -r 3a9521647349 src/java.base/share/classes/java/lang/Class.java > --- a/src/java.base/share/classes/java/lang/Class.java Tue Jun 23 10:46:39 2020 +0100 > +++ b/src/java.base/share/classes/java/lang/Class.java Tue Jun 23 17:11:35 2020 +0100 > @@ -3712,9 +3712,17 @@ > @jdk.internal.PreviewFeature(feature=jdk.internal.PreviewFeature.Feature.RECORDS, > essentialAPI=false) > public boolean isRecord() { > - return getSuperclass() == JAVA_LANG_RECORD_CLASS && isRecord0(); > + if (!isRecordCalled) { > + isRecord = getSuperclass() == JAVA_LANG_RECORD_CLASS && isRecord0(); > + isRecordCalled = true; > + } > + return isRecord; > } > > + // cached record(ness) status > + private transient boolean isRecord; > + private transient boolean isRecordCalled; > + > // Fetches the factory for reflective objects > private static ReflectionFactory getReflectionFactory() { > if (reflectionFactory == null) { > From sakatakui at oss.nttdata.com Wed Jun 24 07:48:48 2020 From: sakatakui at oss.nttdata.com (Koichi Sakata) Date: Wed, 24 Jun 2020 16:48:48 +0900 Subject: Build error with GCC 10 in NetworkInterface.c and k_standard.c Message-ID: Hi all, (I've sent a similar e-mail before to this ML, but I extract the part only related to core-libs-dev ML from the previous one.) I tried to build OpenJDK fastdebug with GCC 10.1 on Ubuntu 18.04, but I saw some compiler warnings as follows: /home/jyukutyo/code/jdk/src/java.base/share/native/libfdlibm/: In function '__j__kernel_standard': /home/jyukutyo/code/jdk/src/java.base/share/native/libfdlibm/k_standard.c:743:19: error: 'exc.retval' may be used uninitialized in this function [-Werror=maybe-uninitialized] 743 | return exc.retval; | ~~~^~~~~~~ In file included from /usr/include/string.h:494, from /home/jyukutyo/code/jdk/src/java.base/unix/native/libnet/NetworkInterface.c:30: In function 'strncpy', inlined from 'getFlags' at /home/jyukutyo/code/jdk/src/java.base/unix/native/libnet/NetworkInterface.c:1362:5, inlined from 'addif' at /home/jyukutyo/code/jdk/src/java.base/unix/native/libnet/NetworkInterface.c:974:13: /usr/include/x86_64-linux-gnu/bits/string_fortified.h:106:10: error: '__builtin_strncpy' output may be truncated copying 15 bytes from a string of length 15 [-Werror=stringop-truncation] 106 | return __builtin___strncpy_chk (__dest, __src, __len, __bos (__dest)); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ I can resolve them with the following patch. I believe it fixes those potential bugs, so I'd like to contribute it. (Our company has signed OCA.) Thanks, Koichi ===== PATCH ===== diff -r 20d92fe3ac52 src/java.base/share/native/libfdlibm/k_standard.c --- a/src/java.base/share/native/libfdlibm/k_standard.c Tue Jun 16 03:16:41 2020 +0000 +++ b/src/java.base/share/native/libfdlibm/k_standard.c Thu Jun 18 07:08:50 2020 +0900 @@ -739,6 +739,10 @@ errno = EDOM; } break; + default: + exc.retval = zero; + errno = EINVAL; + break; } return exc.retval; } diff -r 20d92fe3ac52 src/java.base/unix/native/libnet/NetworkInterface.c --- a/src/java.base/unix/native/libnet/NetworkInterface.c Tue Jun 16 03:16:41 2020 +0000 +++ b/src/java.base/unix/native/libnet/NetworkInterface.c Thu Jun 18 07:08:50 2020 +0900 @@ -1296,7 +1296,10 @@ static int getIndex(int sock, const char *name) { struct ifreq if2; memset((char *)&if2, 0, sizeof(if2)); - strncpy(if2.ifr_name, name, sizeof(if2.ifr_name) - 1); + if (sizeof(if2.ifr_name) < sizeof(name)) { + return -1; + } + strcpy(if2.ifr_name, name); if (ioctl(sock, SIOCGIFINDEX, (char *)&if2) < 0) { return -1; @@ -1359,7 +1362,10 @@ static int getFlags(int sock, const char *ifname, int *flags) { struct ifreq if2; memset((char *)&if2, 0, sizeof(if2)); - strncpy(if2.ifr_name, ifname, sizeof(if2.ifr_name) - 1); + if (sizeof(if2.ifr_name) < sizeof(ifname)) { + return -1; + } + strcpy(if2.ifr_name, ifname); if (ioctl(sock, SIOCGIFFLAGS, (char *)&if2) < 0) { return -1; From chris.hegarty at oracle.com Wed Jun 24 08:20:08 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 24 Jun 2020 09:20:08 +0100 Subject: RFR: 8247532, 8248135: Records deserialization is slow + Build microbenchmarks with --enable-preview In-Reply-To: <2b8beef4-7953-a727-a452-037a951bc7ac@oracle.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> <1DA206A1-15F0-46F9-B69E-9896C57B97C7@oracle.com> <8886ee56-e9a7-aafe-6111-f0c2c54a282e@gmail.com> <63689945-dca3-e0cc-72b8-9225f5da6d31@gmail.com> <7f7b58fa-50e0-a4fe-f948-9637d04c0ccc@oracle.com> <5021b3e7-28ad-7041-cc89-c0f218434937@oracle.com> <092241fe-1f43-ead6-6a5d-0345dac55c3e@oracle.com> <9e4e54c8-05c9-09fd-67ee-98c91ac9aeb3@gmail.com> <2b8beef4-7953-a727-a452-037a951bc7ac@oracle.com> Message-ID: <8678357C-0B46-452D-B2FA-BF8ACE0E3B3A@oracle.com> > On 24 Jun 2020, at 07:03, David Holmes wrote: > > Hi Chris, > > On 24/06/2020 2:30 am, Chris Hegarty wrote: >>> On 23 Jun 2020, at 14:49, Peter Levart wrote: >>> >>> ... >>> >>> Ok, I'm going to push this to jdk15. >> Thank you Peter. This is a really nice change. >> As a follow on, and not for JDK 15, I observe that Class::isRecord0 / JVM_IsRecord shows up as consuming a significant amount of time, more than 10%, in some runs of the deserialization benchmark. The isRecord implementation is a native method in the JVM, so relatively expensive to call. >> This shows an opportunity to improve the Class::isRecord implementation with a simple cache of the record-ness of the j.l.Class, as is done selectively with other particular aspects of a class?s state. There are various ways to implement this, but here is just one [*]. > > There has been reluctance to add more and more fields to Class to cache all these new attributes that are being added Yeah, that seems reasonable. The extra bloat should be given due consideration. I?ve not yet counted how many of these isThis and isThat methods that there are, but I suspect that there are enough that could warrant their state being encoded into a single int or long value on j.l.Class (that could be set lazily by the VM). This would setup a convenient and reasonably efficient location to add future pieces of cached state, like isSealed. > - but ultimately that is a call for core-libs folk to make. The general expectation is/was that the need to ask a class if it is a Record (or isSealed etc) would be rare. But (de)serialization is the exception for isRecord() as unlike enums a simple instanceof test can't be used. It is relatively inexpensive to ask a non-record class if it is a record, but the converse is not the case. Java Serialization can probably ?workaround? this, since it already has a level of local-class cache state, so we can leverage that [*], which is probably the right thing to do for Java Serialization anyway, but I still think that there is a general tractable problem here. -Chris. [*] diff -r 3a9521647349 src/java.base/share/classes/java/io/ObjectInputStream.java --- a/src/java.base/share/classes/java/io/ObjectInputStream.java Tue Jun 23 10:46:39 2020 +0100 +++ b/src/java.base/share/classes/java/io/ObjectInputStream.java Wed Jun 24 09:07:07 2020 +0100 @@ -2182,7 +2182,7 @@ handles.markException(passHandle, resolveEx); } - final boolean isRecord = cl != null && isRecord(cl); + final boolean isRecord = desc.isRecord(); if (isRecord) { assert obj == null; obj = readRecord(desc); From aleksei.voitylov at bell-sw.com Wed Jun 24 08:44:01 2020 From: aleksei.voitylov at bell-sw.com (Aleksei Voitylov) Date: Wed, 24 Jun 2020 11:44:01 +0300 Subject: RFR: JDK-8247592: refactor test/jdk/tools/launcher/Test7029048.java Message-ID: <0bd8fc50-81d4-c40c-ede5-907bd7fc6144@bell-sw.com> Hi, I'd like to refactor test/jdk/tools/launcher/Test7029048.java, make the logic easier to follow and remove some magic numbers from the test: JBS: https://bugs.openjdk.java.net/browse/JDK-8247592 Webrev: http://cr.openjdk.java.net/~avoitylov/webrev.8247592.01/ Testing: the test passes on Linux x86, Linux x86_64, Linux ARM, Linux AArch64, Linux PPC, Windows x86, Windows x86_64, Mac, AIX. Special thanks to SAP team for helping test on AIX. Thanks, -Aleksei From peter.levart at gmail.com Wed Jun 24 09:16:40 2020 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 24 Jun 2020 11:16:40 +0200 Subject: RFR: 8247532, 8248135: Records deserialization is slow + Build microbenchmarks with --enable-preview In-Reply-To: <8678357C-0B46-452D-B2FA-BF8ACE0E3B3A@oracle.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> <1DA206A1-15F0-46F9-B69E-9896C57B97C7@oracle.com> <8886ee56-e9a7-aafe-6111-f0c2c54a282e@gmail.com> <63689945-dca3-e0cc-72b8-9225f5da6d31@gmail.com> <7f7b58fa-50e0-a4fe-f948-9637d04c0ccc@oracle.com> <5021b3e7-28ad-7041-cc89-c0f218434937@oracle.com> <092241fe-1f43-ead6-6a5d-0345dac55c3e@oracle.com> <9e4e54c8-05c9-09fd-67ee-98c91ac9aeb3@gmail.com> <2b8beef4-7953-a727-a452-037a951bc7ac@oracle.com> <8678357C-0B46-452D-B2FA-BF8ACE0E3B3A@oracle.com> Message-ID: <3c191bdd-0396-a9cf-9d6e-532d396590a9@gmail.com> Hi, I wonder why "isRecord" was not encoded in class modifier bits, like "isEnum" was for example. Are all 32 bits already taken? The isEnum() does not have the performance problem since getModifiers() native method is intrinsified. Regards, Peter On 6/24/20 10:20 AM, Chris Hegarty wrote: > > >> On 24 Jun 2020, at 07:03, David Holmes > > wrote: >> >> Hi Chris, >> >> On 24/06/2020 2:30 am, Chris Hegarty wrote: >>>> On 23 Jun 2020, at 14:49, Peter Levart >>> > wrote: >>>> >>>> ... >>>> >>>> Ok, I'm going to push this to jdk15. >>> Thank you Peter. This is a really nice change. >>> As a follow on, and not for JDK 15, I observe that Class::isRecord0 >>> / JVM_IsRecord shows up as consuming a significant amount of time, >>> more than 10%, in some runs of the deserialization benchmark. The >>> isRecord implementation is a native method in the JVM, so relatively >>> expensive to call. >>> This shows an opportunity to improve the Class::isRecord >>> implementation with a simple cache of the record-ness of the >>> j.l.Class, as is done selectively with other particular aspects of a >>> class?s state. There are various ways to implement this, but here is >>> just one [*]. >> >> There has been reluctance to add more and more fields to Class to >> cache all these new attributes that are being added > > Yeah, that seems reasonable. The extra bloat should be given due > consideration. > > I?ve not yet counted how many of these isThis and isThat methods that > there are, but I suspect that there are enough that could warrant > their state being encoded into a single int or long value on j.l.Class > (that could be set lazily by the VM). This would setup a convenient > and reasonably efficient location to add future pieces of cached > state, like isSealed. > >> - but ultimately that is a call for core-libs folk to make. The >> general expectation is/was that the need to ask a class if it is a >> Record (or isSealed etc) would be rare. But (de)serialization is the >> exception for isRecord() as unlike enums a simple instanceof test >> can't be used. > > It is relatively inexpensive to ask a non-record class if it is a > record, but the converse is not the case. > > Java Serialization can probably ?workaround? this, since it already > has a level of local-class cache state, so we can leverage that [*], > which is probably the right thing to do for Java Serialization anyway, > but I still think that there is a general tractable problem here. > > -Chris. > > [*] > diff -r 3a9521647349 > src/java.base/share/classes/java/io/ObjectInputStream.java > --- a/src/java.base/share/classes/java/io/ObjectInputStream.javaTue > Jun 23 10:46:39 2020 +0100 > +++ b/src/java.base/share/classes/java/io/ObjectInputStream.javaWed > Jun 24 09:07:07 2020 +0100 > @@ -2182,7 +2182,7 @@ > handles.markException(passHandle, resolveEx); > ?? ? ? ? } > > > -? ? ? ? final boolean isRecord = cl != null && isRecord(cl); > +? ? ? ? final boolean isRecord = desc.isRecord(); > ?? ? ? ? if (isRecord) { > ?? ? ? ? ? ? assert obj == null; > ?? ? ? ? ? ? obj = readRecord(desc); > > From chris.hegarty at oracle.com Wed Jun 24 09:29:15 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 24 Jun 2020 10:29:15 +0100 Subject: RFR: 8247532, 8248135: Records deserialization is slow + Build microbenchmarks with --enable-preview In-Reply-To: <3c191bdd-0396-a9cf-9d6e-532d396590a9@gmail.com> References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> <1DA206A1-15F0-46F9-B69E-9896C57B97C7@oracle.com> <8886ee56-e9a7-aafe-6111-f0c2c54a282e@gmail.com> <63689945-dca3-e0cc-72b8-9225f5da6d31@gmail.com> <7f7b58fa-50e0-a4fe-f948-9637d04c0ccc@oracle.com> <5021b3e7-28ad-7041-cc89-c0f218434937@oracle.com> <092241fe-1f43-ead6-6a5d-0345dac55c3e@oracle.com> <9e4e54c8-05c9-09fd-67ee-98c91ac9aeb3@gmail.com> <2b8beef4-7953-a727-a452-037a951bc7ac@oracle.com> <8678357C-0B46-452D-B2FA-BF8ACE0E3B3A@oracle.com> <3c191bdd-0396-a9cf-9d6e-532d396590a9@gmail.com> Message-ID: <0EB35E13-0AB1-4062-AB8A-01D1BC399DED@oracle.com> Peter, > On 24 Jun 2020, at 10:16, Peter Levart wrote: > > Hi, > > I wonder why "isRecord" was not encoded in class modifier bits, like "isEnum" was for example. Are all 32 bits already taken? > These bits are precious and limited! ;-) The record-ness of a class is already implicit by the presence of the Record attribute, we don?t really need another way to represent it. I?m sure there was some discussion relating to this on amber-dev, but I don?t have it to hand. > The isEnum() does not have the performance problem since getModifiers() native method is intrinsified. > I don?t think we need anything too ?fancy? here. I think there are quite straight forward ways to achieve the performance characteristics that we want, I?ve already outlined two possibilities (well.. the first is not really a serious possibility, just poking the bear!). If we encode these single state bits into a modifiers-like value, then we can treat them as such (without needing them to be ?real? modifiers). And this should address any potential bloating concerns. -Chris. > Regards, Peter > > > > On 6/24/20 10:20 AM, Chris Hegarty wrote: >> >> >>> On 24 Jun 2020, at 07:03, David Holmes > wrote: >>> >>> Hi Chris, >>> >>> On 24/06/2020 2:30 am, Chris Hegarty wrote: >>>>> On 23 Jun 2020, at 14:49, Peter Levart > wrote: >>>>> >>>>> ... >>>>> >>>>> Ok, I'm going to push this to jdk15. >>>> Thank you Peter. This is a really nice change. >>>> As a follow on, and not for JDK 15, I observe that Class::isRecord0 / JVM_IsRecord shows up as consuming a significant amount of time, more than 10%, in some runs of the deserialization benchmark. The isRecord implementation is a native method in the JVM, so relatively expensive to call. >>>> This shows an opportunity to improve the Class::isRecord implementation with a simple cache of the record-ness of the j.l.Class, as is done selectively with other particular aspects of a class?s state. There are various ways to implement this, but here is just one [*]. >>> >>> There has been reluctance to add more and more fields to Class to cache all these new attributes that are being added >> >> Yeah, that seems reasonable. The extra bloat should be given due consideration. >> >> I?ve not yet counted how many of these isThis and isThat methods that there are, but I suspect that there are enough that could warrant their state being encoded into a single int or long value on j.l.Class (that could be set lazily by the VM). This would setup a convenient and reasonably efficient location to add future pieces of cached state, like isSealed. >> >>> - but ultimately that is a call for core-libs folk to make. The general expectation is/was that the need to ask a class if it is a Record (or isSealed etc) would be rare. But (de)serialization is the exception for isRecord() as unlike enums a simple instanceof test can't be used. >> >> It is relatively inexpensive to ask a non-record class if it is a record, but the converse is not the case. >> >> Java Serialization can probably ?workaround? this, since it already has a level of local-class cache state, so we can leverage that [*], which is probably the right thing to do for Java Serialization anyway, but I still think that there is a general tractable problem here. >> >> -Chris. >> >> [*] >> diff -r 3a9521647349 src/java.base/share/classes/java/io/ObjectInputStream.java >> --- a/src/java.base/share/classes/java/io/ObjectInputStream.java Tue Jun 23 10:46:39 2020 +0100 >> +++ b/src/java.base/share/classes/java/io/ObjectInputStream.java Wed Jun 24 09:07:07 2020 +0100 >> @@ -2182,7 +2182,7 @@ >> handles.markException(passHandle, resolveEx); >> } >> >> - final boolean isRecord = cl != null && isRecord(cl); >> + final boolean isRecord = desc.isRecord(); >> if (isRecord) { >> assert obj == null; >> obj = readRecord(desc); >> >> From patrick.concannon at oracle.com Wed Jun 24 10:57:01 2020 From: patrick.concannon at oracle.com (Patrick Concannon) Date: Wed, 24 Jun 2020 11:57:01 +0100 Subject: =?utf-8?Q?RFR=5B8238286=5D=3A__=27Add_new_flatMap_stream_operatio?= =?utf-8?Q?n_that_is_more_amenable_to_pushing=E2=80=99?= Message-ID: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> Hi, Could someone please review myself and Julia's RFE and CSR for JDK-8238286 - 'Add new flatMap stream operation that is more amenable to pushing?? This proposal is to add a new flatMap-like operation: ` Stream mapMulti(BiConsumer, ? super T> mapper)` to the java.util.Stream class. This operation is more receptive to the pushing or yielding of values than the current implementation that internally assembles values (if any) into one or more streams. This addition includes the primitive variations of the operation i.e. mapMultiToInt, IntStream mapMulti, etc. issue: https://bugs.openjdk.java.net/browse/JDK-8238286 csr: https://bugs.openjdk.java.net/browse/JDK-8248166 webrev: http://cr.openjdk.java.net/~pconcannon/8238286/webrevs/webrev.00/ specdiff: http://cr.openjdk.java.net/~pconcannon/8238286/specdiff/specout.00/overview-summary.html? Kind regards, Patrick & Julia From magnus.ihse.bursie at oracle.com Wed Jun 24 11:01:55 2020 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Wed, 24 Jun 2020 13:01:55 +0200 Subject: [OpenJDK 2D-Dev] RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: References: <5EDE9A75.6020504@oracle.com> <5EE5AB71.20505@oracle.com> Message-ID: On 2020-06-18 08:34, Kim Barrett wrote: >> On Jun 15, 2020, at 7:41 AM, Kim Barrett wrote: >> >>> On Jun 14, 2020, at 12:45 AM, Philip Race wrote: >>> >>> Kim, >>> >>> >>> Until it says in "the JDK" and not "in HotSpot" you have not addressed my main point. >>> Please rename the JEP. > After some off-list discussions I have a better idea of what Phil was > asking for and why. In response I've changed the JEP, replacing a few > occurrences of "HotSpot" with "the JDK", as described below. All > other occurrences of "HotSpot" have been left as-is. > > Title: > JEP 347: Adopt C++14 Language Features in HotSpot > => > JEP 347: Adopt C++14 Language Features in the JDK > > Summary: > Allow the use of selected C++14 language features in the HotSpot C++ source code. > => > Allow the use of selected C++14 language features in the JDK C++ source code. > > Goals: > The purpose of this JEP is to formally allow C++ source code changes within HotSpot to take advantage of some C++14 standard language features. > => > The purpose of this JEP is to formally allow C++ source code changes within the JDK to take advantage of some C++14 standard language features. > This all looks good to me. /Magnus From jai.forums2013 at gmail.com Wed Jun 24 13:44:34 2020 From: jai.forums2013 at gmail.com (Jaikiran Pai) Date: Wed, 24 Jun 2020 19:14:34 +0530 Subject: RFR: JDK-8240148: Uncaught exceptions from ScheduledThreadPoolExecutor.execute aren't reported Message-ID: <4617b527-fea8-ac45-27d1-23308443baa3@gmail.com> Could I please get a review and a sponsor for a fix for https://bugs.openjdk.java.net/browse/JDK-8240148? The patch is available as a webrev at https://cr.openjdk.java.net/~jpai/webrev/8240148/1/ The patch updates the ScheduledThreadPoolExecutor.execute(Runnable) to wrap the user passed Runnable into a custom one which then catches any Throwable and reports it through the Thread's uncaught exception handler. After having reviewed the existing code in that class, this was the easiest (and perhaps the only feasible?) and cleanest way that I could think of, to fix this issue. A few things about this change: 1. I decided to use an anonymous class for the Runnable, in the execute() method instead of a lambda because I was unsure if this part of the JDK code is allowed to use a lambda. In the past I have seen discussions where it was recommended not use lambda in certain parts of the JDK either because of performance(?) reason or because "this part of the code runs too early to be using a lambda" reason. I personally prefer an anonymous class anyway, but if others feel using a lambda here is relevant, then let me know. 2. Once the exception is reported using the UncaughtExceptionHandler, I just return. Initially, I had considered throwing back the original exception/throwable after reporting it via the UncaughtExceptionHandler, but having looked at few others places within the JDK which do not throw back the original exception, I decided to follow the same semantics. 3. Now that the passed Runnable is wrapped and submitted for execution, I considered whether this would break any (unwritten) contract between the ThreadPoolExecutor and the ThreadFactory#newThread(Runnable) implementations. If any (JDK internal or user application specific) ThreadFactory#newThread method was expecting the passed Runnable to be of the same type or the same instance as that was submitted to the ScheduledThreadPoolExecutor#execute(Runnable), then this change would break such ThreadFactory#newThread implementations. I looked deeper in the ThreadPoolExecutor code and from what I could see, this isn't a practical concern, since even without this change, The ThreadPoolExecutor in its private Worker class already sends a "delegate" Runnable (an instance of the ThreadPoolExecutor$Worker class) to the ThreadFactory's newThread method and not the original Runnable that was submitted to the execute(Runnable) method of the executor. So it doesn't look like there's any written/unwritten contract that the ThreadFactory is expected to receive the same Runnable instance that was passed to the execute method of the executor. 4. The ScheduledThreadPoolExecutor has a different license than some of the other files that I am used to, within the JDK code. So I haven't edited it for any of the usual copyright year updates. 5. The patch contains a new (testng based) test case which reproduces this issue and verifies the fix. -Jaikiran From martinrb at google.com Wed Jun 24 15:21:32 2020 From: martinrb at google.com (Martin Buchholz) Date: Wed, 24 Jun 2020 08:21:32 -0700 Subject: RFR: JDK-8240148: Uncaught exceptions from ScheduledThreadPoolExecutor.execute aren't reported In-Reply-To: <4617b527-fea8-ac45-27d1-23308443baa3@gmail.com> References: <4617b527-fea8-ac45-27d1-23308443baa3@gmail.com> Message-ID: Thanks for your work on this. Invoking the UEH machinery directly is extremely unusual and would never have occurred to me (outside of a test). In a loom-directed world, any attempt to directly manipulate the current thread is likely to lead to trouble. This code is primarily maintained separately at Doug's site http://gee.cs.oswego.edu/dl/concurrency-interest/ I've been handling most of the integration work. On Wed, Jun 24, 2020 at 6:51 AM Jaikiran Pai wrote: > > Could I please get a review and a sponsor for a fix for > https://bugs.openjdk.java.net/browse/JDK-8240148? > > The patch is available as a webrev at > https://cr.openjdk.java.net/~jpai/webrev/8240148/1/ > > The patch updates the ScheduledThreadPoolExecutor.execute(Runnable) to > wrap the user passed Runnable into a custom one which then catches any > Throwable and reports it through the Thread's uncaught exception > handler. After having reviewed the existing code in that class, this was > the easiest (and perhaps the only feasible?) and cleanest way that I > could think of, to fix this issue. > > A few things about this change: > > 1. I decided to use an anonymous class for the Runnable, in the > execute() method instead of a lambda because I was unsure if this part > of the JDK code is allowed to use a lambda. In the past I have seen > discussions where it was recommended not use lambda in certain parts of > the JDK either because of performance(?) reason or because "this part of > the code runs too early to be using a lambda" reason. I personally > prefer an anonymous class anyway, but if others feel using a lambda here > is relevant, then let me know. > > 2. Once the exception is reported using the UncaughtExceptionHandler, I > just return. Initially, I had considered throwing back the original > exception/throwable after reporting it via the UncaughtExceptionHandler, > but having looked at few others places within the JDK which do not throw > back the original exception, I decided to follow the same semantics. > > 3. Now that the passed Runnable is wrapped and submitted for execution, > I considered whether this would break any (unwritten) contract between > the ThreadPoolExecutor and the ThreadFactory#newThread(Runnable) > implementations. If any (JDK internal or user application specific) > ThreadFactory#newThread method was expecting the passed Runnable to be > of the same type or the same instance as that was submitted to the > ScheduledThreadPoolExecutor#execute(Runnable), then this change would > break such ThreadFactory#newThread implementations. I looked deeper in > the ThreadPoolExecutor code and from what I could see, this isn't a > practical concern, since even without this change, The > ThreadPoolExecutor in its private Worker class already sends a > "delegate" Runnable (an instance of the > ThreadPoolExecutor$Worker class) to the ThreadFactory's newThread method > and not the original Runnable that was submitted to the > execute(Runnable) method of the executor. So it doesn't look like > there's any written/unwritten contract that the ThreadFactory is > expected to receive the same Runnable instance that was passed to the > execute method of the executor. > > 4. The ScheduledThreadPoolExecutor has a different license than some of > the other files that I am used to, within the JDK code. So I haven't > edited it for any of the usual copyright year updates. > > 5. The patch contains a new (testng based) test case which reproduces > this issue and verifies the fix. > > -Jaikiran > > From brian.goetz at oracle.com Wed Jun 24 15:43:38 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Wed, 24 Jun 2020 11:43:38 -0400 Subject: RFR: 8247532, 8248135: Records deserialization is slow + Build microbenchmarks with --enable-preview In-Reply-To: References: <4c25cfeb-7e30-4012-c4db-94ae3facf7c5@gmail.com> <23F7D722-96AF-478C-903C-EF36643309AD@oracle.com> <6088b109-87af-f3ba-8636-d46376a90b67@gmail.com> <2c5a7b16-e79f-3e0f-a36d-358fbc3b76e0@gmail.com> <1DA206A1-15F0-46F9-B69E-9896C57B97C7@oracle.com> <8886ee56-e9a7-aafe-6111-f0c2c54a282e@gmail.com> <63689945-dca3-e0cc-72b8-9225f5da6d31@gmail.com> <7f7b58fa-50e0-a4fe-f948-9637d04c0ccc@oracle.com> <5021b3e7-28ad-7041-cc89-c0f218434937@oracle.com> <092241fe-1f43-ead6-6a5d-0345dac55c3e@oracle.com> <9e4e54c8-05c9-09fd-67ee-98c91ac9aeb3@gmail.com> Message-ID: Consider a ClassValue for this, if there is not an obvious bit to be repurposed somewhere. On 6/23/2020 12:30 PM, Chris Hegarty wrote: > I think it is worth considering caching the record-ness state of a j.l.Class, as I?m sure it will be widely used in third-party serialization libraries, as well as by Java Serialization. From chris.hegarty at oracle.com Wed Jun 24 15:46:06 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 24 Jun 2020 16:46:06 +0100 Subject: RFR [15] 8248233: Avoid superfluous Class::isRecord invocations during deserialization Message-ID: A micro-optimisation noticed when working on JDK-8247532. For further details see: https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-June/067446.html Webrev: https://cr.openjdk.java.net/~chegar/8248233/webrev.00/ before: RecordSerializationBench.deserializeClasses 10 avgt 10 13.874 ? 1.445 us/op RecordSerializationBench.deserializeClasses 100 avgt 10 57.839 ? 3.944 us/op RecordSerializationBench.deserializeClasses 1000 avgt 10 515.483 ? 57.275 us/op RecordSerializationBench.deserializeRecords 10 avgt 10 13.563 ? 0.459 us/op RecordSerializationBench.deserializeRecords 100 avgt 10 61.704 ? 2.481 us/op RecordSerializationBench.deserializeRecords 1000 avgt 10 518.671 ? 19.147 us/op after: RecordSerializationBench.deserializeClasses 10 avgt 10 16.021 ? 9.091 us/op RecordSerializationBench.deserializeClasses 100 avgt 10 58.550 ? 2.164 us/op RecordSerializationBench.deserializeClasses 1000 avgt 10 524.930 ? 49.663 us/op RecordSerializationBench.deserializeRecords 10 avgt 10 12.567 ? 0.711 us/op RecordSerializationBench.deserializeRecords 100 avgt 10 50.235 ? 1.977 us/op RecordSerializationBench.deserializeRecords 1000 avgt 10 421.557 ? 17.348 us/op -Chris. [1] https://bugs.openjdk.java.net/browse/JDK-8247532 From aleksei.voitylov at bell-sw.com Wed Jun 24 15:48:46 2020 From: aleksei.voitylov at bell-sw.com (Aleksei Voitylov) Date: Wed, 24 Jun 2020 18:48:46 +0300 Subject: RFR: JDK-8248239: jpackage adds some arguments twice in case it is re-executed by JLI Message-ID: Hi, There are systems that update LD_LIBRARY_PATH or directly return JNI_TRUE in method RequiresSetenv(const char *jvmpath) from java_md.c file to request re-execution of the current executable. This leads to the fact that jpackage application adds some provided arguments twice. Bug: https://bugs.openjdk.java.net/browse/JDK-8248239 Fix: http://cr.openjdk.java.net/~avoitylov/webrev.8248239.00/ The root cause of the issue is that jpackage application expects one number of arguments but JLI reexecutes them with another number of arguments. ? For example, a jpackage application can be run with arguments: ??? ./app/bin/app -Dparam2=Param2 B1 B2 B3 it adds arguments from the config file and calls JLI with arguments: ??? app/bin/app -classpath? -Dparam1=Param1 -m com.hello/com.hello.Hello -Dparam2=Param2 B1 B2 B3 JLI re-executes the app with new arguments so the app adds some arguments one more time after the re-execution. ??? ./app/bin/app -classpath? -Dparam1=Param1 -m com.hello/com.hello.Hello -classpath? -Dparam1=Param1 -m com.hello/com.hello.Hello -Dparam2=Param2 B1 B2 B3 A step by step example that illustrates the issue: Run jpackage to create an executable application: ? jpackage --dest output --name app --type app-image --module-path input-modules --module com.hello/com.hello.Hello --arguments "A1 A2 A3" --verbose --java-options -Dparam1=Param1 Executable application with the app/lib/app/app.cfg config file is created: ---- app.cfg? ---- [Application] app.name=app app.version=1.0 app.runtime=$ROOTDIR/lib/runtime app.identifier=com.hello app.classpath= app.mainmodule=com.hello/com.hello.Hello [JavaOptions] java-options=-Dparam1=Param1 [ArgOptions] arguments=A1 arguments=A2 arguments=A3 ----------- Run the application: ?? ./output/app/bin/app -Dparam2=Param2 B1 B2 B3 Chain of JDK methods execution: LinuxLauncher main() ?? args: 5 [./app/bin/app -Dparam2=Param2 B1 B2 B3 ] AppLauncher createJvmLauncher() ?? args: 4 [-Dparam2=Param2 B1 B2 B3 ] JvmLauncher.cpp Jvm::initFromConfigFile() - adds JavaOptions from app.cfg ?? args: 10 [app/bin/app -classpath? -Dparam1=Param1 -m com.hello/com.hello.Hello -Dparam2=Param2 B1 B2 B3 ] AppLauncher Jvm::launch() -? JLI re-executes the app LinuxLauncher main() ? args: 10 [app/bin/app -classpath? -Dparam1=Param1 -m com.hello/com.hello.Hello -Dparam2=Param2 B1 B2 B3 ] AppLauncher createJvmLauncher() ?? args: 9 [-classpath? -Dparam1=Param1 -m com.hello/com.hello.Hello -Dparam2=Param2 B1 B2 B3 ] JvmLauncher.cpp Jvm::initFromConfigFile() - adds JavaOptions from app.cfg ?? args: 15 [./app/bin/app -classpath? -Dparam1=Param1 -m com.hello/com.hello.Hello -classpath? -Dparam1=Param1 -m com.hello/com.hello.Hello -Dparam2=Param2 B1 B2 B3 ] ??? ^^^ Some arguments like "-classpath? -Dparam1=Param1 -m com.hello/com.hello.Hello" are added twice. Tested with test/jdk/tools/jpackage/share/jdk/jpackage with no regressions on Linux, Windows, Mac. On systems affected, the tests now pass. Thanks, -Aleksei From claes.redestad at oracle.com Wed Jun 24 16:15:35 2020 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 24 Jun 2020 18:15:35 +0200 Subject: RFR [15] 8248233: Avoid superfluous Class::isRecord invocations during deserialization In-Reply-To: References: Message-ID: <29dd18cf-602a-571d-757a-1f087be5ec1a@oracle.com> Hi Chris, On 2020-06-24 17:46, Chris Hegarty wrote: > A micro-optimisation noticed when working on JDK-8247532. > > For further details see: > https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-June/067446.html > > Webrev: > https://cr.openjdk.java.net/~chegar/8248233/webrev.00/ This looks good. It seems ObjectInputStream#isRecord(Class) is now unused. No need for a new webrev if you choose to remove it. /Claes > > before: > RecordSerializationBench.deserializeClasses 10 avgt 10 13.874 ? 1.445 us/op > RecordSerializationBench.deserializeClasses 100 avgt 10 57.839 ? 3.944 us/op > RecordSerializationBench.deserializeClasses 1000 avgt 10 515.483 ? 57.275 us/op > RecordSerializationBench.deserializeRecords 10 avgt 10 13.563 ? 0.459 us/op > RecordSerializationBench.deserializeRecords 100 avgt 10 61.704 ? 2.481 us/op > RecordSerializationBench.deserializeRecords 1000 avgt 10 518.671 ? 19.147 us/op > after: > RecordSerializationBench.deserializeClasses 10 avgt 10 16.021 ? 9.091 us/op > RecordSerializationBench.deserializeClasses 100 avgt 10 58.550 ? 2.164 us/op > RecordSerializationBench.deserializeClasses 1000 avgt 10 524.930 ? 49.663 us/op > RecordSerializationBench.deserializeRecords 10 avgt 10 12.567 ? 0.711 us/op > RecordSerializationBench.deserializeRecords 100 avgt 10 50.235 ? 1.977 us/op > RecordSerializationBench.deserializeRecords 1000 avgt 10 421.557 ? 17.348 us/op > > -Chris. > > [1] https://bugs.openjdk.java.net/browse/JDK-8247532 > From peter.levart at gmail.com Wed Jun 24 16:25:24 2020 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 24 Jun 2020 18:25:24 +0200 Subject: RFR [15] 8248233: Avoid superfluous Class::isRecord invocations during deserialization In-Reply-To: References: Message-ID: <7457d339-439e-ac9a-0655-fd2737732083@gmail.com> Hi Chris, The patch looks good. Before the patch it made sense to have if (cl != null) in line 750 in ObjectStreamClass, but now nothing in the if block depends on cl, so you could use if (osc != null) instead. It is true that: (cl != null) == (osc != null) always holds there, but reading the code is easier that way, don't you think? Maybe you could even consider merging the content of this if block into the similar if block that starts in line 771? Regards, Peter On 6/24/20 5:46 PM, Chris Hegarty wrote: > A micro-optimisation noticed when working on?JDK-8247532. > > For further details see: > https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-June/067446.html > > Webrev: > https://cr.openjdk.java.net/~chegar/8248233/webrev.00/ > > before: > RecordSerializationBench.deserializeClasses10avgt 10 13.874 ?1.445us/op > ?RecordSerializationBench.deserializeClasses ? ? ? 100? avgt ? 10 ? > 57.839 ?? 3.944? us/op > ?RecordSerializationBench.deserializeClasses ? ? 1000? avgt ? 10? > 515.483 ? 57.275? us/op > ?RecordSerializationBench.deserializeRecords ? ? ? 10? avgt ? 10 ? > 13.563 ?? 0.459? us/op > ?RecordSerializationBench.deserializeRecords ? ? ? 100? avgt ? 10 ? > 61.704 ?? 2.481? us/op > ?RecordSerializationBench.deserializeRecords ? ? 1000? avgt ? 10? > 518.671 ? 19.147? us/op > after: > RecordSerializationBench.deserializeClasses10avgt 10 16.021 ?9.091us/op > ?RecordSerializationBench.deserializeClasses ? ? ? 100? avgt ? 10 ? > 58.550 ?? 2.164? us/op > ?RecordSerializationBench.deserializeClasses ? ? 1000? avgt ? 10? > 524.930 ? 49.663? us/op > ?RecordSerializationBench.deserializeRecords ? ? ? 10? avgt ? 10 ? > 12.567 ?? 0.711? us/op > ?RecordSerializationBench.deserializeRecords ? ? ? 100? avgt ? 10 ? > 50.235 ?? 1.977? us/op > ?RecordSerializationBench.deserializeRecords ? ? 1000? avgt ? 10? > 421.557 ? 17.348? us/op > > -Chris. > > [1] https://bugs.openjdk.java.net/browse/JDK-8247532 From Roger.Riggs at oracle.com Wed Jun 24 16:33:01 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Wed, 24 Jun 2020 12:33:01 -0400 Subject: [15] RFR JDK-8247785: Small clarification to the javadoc about builtin class loaders In-Reply-To: <81d4a655-76ad-2e53-d83c-724c385f2c7b@oracle.com> References: <37776e19-a225-9547-6d53-147aa69679b8@oracle.com> <43d00426-80f1-7e1e-d397-d82d6646d835@oracle.com> <0ddc3a45-6af7-1d6a-5112-e9a553a929ea@oracle.com> <81d4a655-76ad-2e53-d83c-724c385f2c7b@oracle.com> Message-ID: Hi Mandy, I'm fine with this. Roger On 6/23/20 5:42 PM, Mandy Chung wrote: > > > On 6/23/20 12:01 PM, Roger Riggs wrote: >> Hi Mandy, >> >> There may be a missing "to" in: >> >> + *???? Platform classes are visible the platform class loader >> ++ *???? Platform classes are visible *via* the platform >> class loader >> > > I caught this accidental change too. > >> The second change seems to be self referential using "parent" to >> define itself. >> >> And pre-existing in the description of getSystemClassLoader: >> >> * The platform class loader is a parent or an ancestor of the system >> class * loader that all platform classes are visible to it. >> >> Is missing "so" in : >> >> * loader so that all platform classes are visible to it. >> >> Both paragraphs are difficult to read and understand. (? I think the >> originals are more readable). > > I made a minor adjustment to break the sentence into two.? That should > help. > > diff --git a/src/java.base/share/classes/java/lang/ClassLoader.java > b/src/java.base/share/classes/java/lang/ClassLoader.java > --- a/src/java.base/share/classes/java/lang/ClassLoader.java > +++ b/src/java.base/share/classes/java/lang/ClassLoader.java > @@ -133,8 +133,9 @@ > ? *???? It is the virtual machine's built-in class loader, typically > represented > ? *???? as {@code null}, and does not have a parent. > ? *

    • {@linkplain #getPlatformClassLoader() Platform class loader}. > - *???? All platform classes are visible to the platform > class loader > - *???? that can be used as the parent of a {@code ClassLoader} instance. > + *???? Platform classes are visible to the platform class > loader. > + *???? The platform class loader can be used as the parent of a > {@code ClassLoader} > + *???? instance. > ? *???? Platform classes include Java SE platform APIs, their > implementation > ? *???? classes and JDK-specific run-time classes that are defined by the > ? *???? platform class loader or its ancestors. > @@ -152,7 +153,7 @@ > ? *???? The system class loader is typically used to define classes on the > ? *???? application class path, module path, and JDK-specific tools. > ? *???? The platform class loader is a parent or an ancestor of the > system class > - *???? loader that all platform classes are visible to it.

    • > + *???? loader.? It searches and loads the platform classes through > its parent. > ? * > ? * > ? *

      Normally, the Java virtual machine loads classes from the > local file > > Thanks > Mandy From alexey.semenyuk at oracle.com Wed Jun 24 16:34:51 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Wed, 24 Jun 2020 12:34:51 -0400 Subject: RFR: JDK-8248239: jpackage adds some arguments twice in case it is re-executed by JLI In-Reply-To: References: Message-ID: <6d4fabe6-37ce-7d1d-bf7d-e2b07e0ba92b@oracle.com> Aleksei, If I get it right, the fix would not work for the case when there are `arguments` properties in `ArgOptions` section of a config file. Why not just check if the parent process is the same executable as the current one and if this is the case don't read data from the config file but pass executable arguments as is in JLI_Launch() call? - Alexey On 6/24/2020 11:48 AM, Aleksei Voitylov wrote: > Hi, > > There are systems that update LD_LIBRARY_PATH or directly return > JNI_TRUE in method RequiresSetenv(const char *jvmpath) from java_md.c > file to request re-execution of the current executable. This leads to > the fact that jpackage application adds some provided arguments twice. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8248239 > Fix: http://cr.openjdk.java.net/~avoitylov/webrev.8248239.00/ > > The root cause of the issue is that jpackage application expects one > number of arguments but JLI reexecutes them with another number of > arguments. > > For example, a jpackage application can be run with arguments: > ??? ./app/bin/app -Dparam2=Param2 B1 B2 B3 > it adds arguments from the config file and calls JLI with arguments: > ??? app/bin/app -classpath? -Dparam1=Param1 -m com.hello/com.hello.Hello > -Dparam2=Param2 B1 B2 B3 > JLI re-executes the app with new arguments so the app adds some > arguments one more time after the re-execution. > ??? ./app/bin/app -classpath? -Dparam1=Param1 -m > com.hello/com.hello.Hello -classpath? -Dparam1=Param1 -m > com.hello/com.hello.Hello -Dparam2=Param2 B1 B2 B3 > > A step by step example that illustrates the issue: > > Run jpackage to create an executable application: > ? jpackage --dest output --name app --type app-image --module-path > input-modules --module com.hello/com.hello.Hello --arguments "A1 A2 A3" > --verbose --java-options -Dparam1=Param1 > > Executable application with the app/lib/app/app.cfg config file is created: > ---- app.cfg? ---- > [Application] > app.name=app > app.version=1.0 > app.runtime=$ROOTDIR/lib/runtime > app.identifier=com.hello > app.classpath= > app.mainmodule=com.hello/com.hello.Hello > > [JavaOptions] > java-options=-Dparam1=Param1 > > [ArgOptions] > arguments=A1 > arguments=A2 > arguments=A3 > ----------- > > Run the application: > ?? ./output/app/bin/app -Dparam2=Param2 B1 B2 B3 > > Chain of JDK methods execution: > > LinuxLauncher main() > ?? args: 5 [./app/bin/app -Dparam2=Param2 B1 B2 B3 ] > AppLauncher createJvmLauncher() > ?? args: 4 [-Dparam2=Param2 B1 B2 B3 ] > JvmLauncher.cpp Jvm::initFromConfigFile() - adds JavaOptions from app.cfg > ?? args: 10 [app/bin/app -classpath? -Dparam1=Param1 -m > com.hello/com.hello.Hello -Dparam2=Param2 B1 B2 B3 ] > AppLauncher Jvm::launch() -? JLI re-executes the app > LinuxLauncher main() > ? args: 10 [app/bin/app -classpath? -Dparam1=Param1 -m > com.hello/com.hello.Hello -Dparam2=Param2 B1 B2 B3 ] > AppLauncher createJvmLauncher() > ?? args: 9 [-classpath? -Dparam1=Param1 -m com.hello/com.hello.Hello > -Dparam2=Param2 B1 B2 B3 ] > JvmLauncher.cpp Jvm::initFromConfigFile() - adds JavaOptions from app.cfg > ?? args: 15 [./app/bin/app -classpath? -Dparam1=Param1 -m > com.hello/com.hello.Hello -classpath? -Dparam1=Param1 -m > com.hello/com.hello.Hello -Dparam2=Param2 B1 B2 B3 ] > ??? ^^^ > > Some arguments like "-classpath? -Dparam1=Param1 -m > com.hello/com.hello.Hello" are added twice. > > Tested with test/jdk/tools/jpackage/share/jdk/jpackage with no > regressions on Linux, Windows, Mac. On systems affected, the tests now pass. > > Thanks, > > -Aleksei > > From chris.hegarty at oracle.com Wed Jun 24 16:53:39 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 24 Jun 2020 17:53:39 +0100 Subject: RFR [15] 8248233: Avoid superfluous Class::isRecord invocations during deserialization In-Reply-To: <7457d339-439e-ac9a-0655-fd2737732083@gmail.com> References: <7457d339-439e-ac9a-0655-fd2737732083@gmail.com> Message-ID: > On 24 Jun 2020, at 17:15, Claes Redestad wrote: > > ... > It seems ObjectInputStream#isRecord(Class) is now unused. No need for > a new webrev if you choose to remove it. Good catch, now removed. > On 24 Jun 2020, at 17:25, Peter Levart wrote: > > Hi Chris, > > The patch looks good. > > Before the patch it made sense to have if (cl != null) in line 750 in ObjectStreamClass, but now nothing in the if block depends on cl, so you could use if (osc != null) instead. It is true that: > > (cl != null) == (osc != null) > > always holds there, but reading the code is easier that way, don't you think? Maybe you could even consider merging the content of this if block into the similar if block that starts in line 771? > Yes, good idea. Done. Updated webrev: https://cr.openjdk.java.net/~chegar/8248233/webrev.01 -Chris. From raffaello.giulietti at gmail.com Wed Jun 24 17:22:04 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Wed, 24 Jun 2020 19:22:04 +0200 Subject: Contributing to JEP 356: Enhanced Pseudo-Random Number Generators ? Message-ID: <01ae5908-c7b9-a10e-9139-a90c8ba37d6c@gmail.com> Hi, seems like the last activity in JBS related to this JEP [1, 2] dates back to June 2019. Are there major stumbling blocks? Can I contribute in any way? Greetings Raffaello ---- [1] http://openjdk.java.net/jeps/356 [2] https://bugs.openjdk.java.net/browse/JDK-8193209 From peter.levart at gmail.com Wed Jun 24 17:31:11 2020 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 24 Jun 2020 19:31:11 +0200 Subject: RFR [15] 8248233: Avoid superfluous Class::isRecord invocations during deserialization In-Reply-To: References: <7457d339-439e-ac9a-0655-fd2737732083@gmail.com> Message-ID: On 6/24/20 6:53 PM, Chris Hegarty wrote: > >> On 24 Jun 2020, at 17:15, Claes Redestad > > wrote: >> >> ... >> It seems ObjectInputStream#isRecord(Class) is now unused. No need for >> a new webrev if you choose to remove it. > > Good catch, now removed. > >> On 24 Jun 2020, at 17:25, Peter Levart > > wrote: >> >> Hi Chris, >> >> The patch looks good. >> >> Before the patch it made sense to have if (cl != null) in line 750 in >> ObjectStreamClass, but now nothing in the if block depends on cl, so >> you could use if (osc != null) instead. It is true that: >> >> (cl != null) == (osc != null) >> >> always holds there, but reading the code is easier that way, don't >> you think? Maybe you could even consider merging the content of this >> if block into the similar if block that starts in line 771? >> > Yes, good idea. Done. > > Updated webrev: > https://cr.openjdk.java.net/~chegar/8248233/webrev.01 > > -Chris. This looks good. Now next thing to do perhaps is to find out why deserialization of classical classes is slower than deserialization of records ;-) Regards, Peter From daniel.fuchs at oracle.com Wed Jun 24 18:56:03 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 24 Jun 2020 19:56:03 +0100 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <567C1F82-0EE7-46F3-90E9-3A1B5FDD5E13@azul.com> References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> <9013C8D0-33D9-4D09-8953-F8618020A7F7@oracle.com> <8A2C4DEF-FC61-4574-B49D-1133F5349BCE@azul.com> <15fa46f2-0352-a111-a9c7-0d5fc31d9159@oracle.com> <6167ea24-e7f9-c259-e2c5-8be96d74a40d@oracle.com> <228BA21D-836A-46D7-B273-738AFFE639E4@azul.com> <97a4f719-e0f5-96f7-6637-432b4b746685@oracle.com> <567C1F82-0EE7-46F3-90E9-3A1B5FDD5E13@azul.com> Message-ID: <959ffeaa-a8ef-87fb-6a21-0f125fa3d891@oracle.com> Hi Alexey, The JNDI/LDAP part looks mostly good. You will need someone from the security libs to review the security lib part of the changes. In the test I would recommend using the test URIBuilder to avoid strange intermittent errors if the test is run on a machine where looking up "localhost" doesn't yield back InetAddress.getLoopbackAddress(): ------------------ * @library /test/lib ... import java.net.URI; import jdk.test.lib.net.URIBuilder; ... URI uri = URIBuilder.newBuilder() .scheme("ldaps") .loopback() .port(serverPort) .build(); env.put(Context.PROVIDER_URL, uri.toString()); ------------------ So we have now two new properties: jdk.internal.sasl.tlschannelbinding which is a private contract between java.naming and jdk.security.jgss; com.sun.jndi.ldap.tls.cbtype which is a new JDK implementation specific environment property for the InitialLdapContext, and is depending on another JDK specific environment property: "com.sun.jndi.ldap.connect.timeout" None of these properties are currently documented in the JDK itself. Although jdk.internal.sasl.tlschannelbinding probably doesn't need to be documented (but I'll defer to the security experts for that), the other two probably should. Where is the question. If we had a jdk.namimg.ldap module then the documentation for these jndi properties would probably need to go in its module-info.java API documentation, but we don't. Obviously we will want to write a release note for this fix that documents the new com.sun.jndi.ldap.tls.cbtype property - but will that be sufficient? The CSR committee might wish for more. Anyone has advice to share on this? best regards, -- daniel On 17/06/2020 12:26, Alexey Bakhtin wrote: > Hello Daniel, > > Thank you for review. > > As you suggested I?ve added static factory methods to create TlsChannelBinding class and changed connectionTimeout verification to "if (connectTimeout <= 0)" > Also, I?ve added simple regression test to verify Channel Binding parameters. > > Please find updated webrev at:http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v8/ > > The link to CSR for this feature :https://bugs.openjdk.java.net/browse/JDK-8247311 > > Regards > Alexey From sandhya.viswanathan at intel.com Wed Jun 24 19:03:52 2020 From: sandhya.viswanathan at intel.com (Viswanathan, Sandhya) Date: Wed, 24 Jun 2020 19:03:52 +0000 Subject: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes In-Reply-To: References: <275eb57c-51c0-675e-c32a-91b198023559@redhat.com> <719F9169-ABC4-408E-B732-F1BD9A84337F@oracle.com> <9a13f5df-d946-579d-4282-917dc7338dc8@redhat.com> <09BC0693-80E0-4F87-855E-0B38A6F5EFA2@oracle.com> <668e500e-f621-5a2c-a41e-f73536880f73@redhat.com> <1909fa9d-98bb-c2fb-45d8-540247d1ca8b@redhat.com> Message-ID: Hi Andrew/Yang, We couldn?t propose Vector API to target in time for JDK 15 and hoping to do so early in JDK 16 timeframe. The implementation reviews on other components have made good progress. We have so far ok to PPT from (runtime, shared compiler changes, x86 backend). Java API implementation review is in progress. I wanted to check with you both if we have a go ahead from aarch64 backed point of view. Best Regards, Sandhya -----Original Message----- From: hotspot-compiler-dev On Behalf Of Yang Zhang Sent: Tuesday, May 26, 2020 7:59 PM To: Andrew Haley ; Paul Sandoz Cc: nd ; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net Subject: RE: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes > But to my earlier question. please: can the new instructions be moved into jdk head first, and then merged into the Panama branch, or not? The new instructions can be classified as: 1. Instructions that can be matched with NEON instructions directly. MulVB and SqrtVF have been merged into jdk master already. The patch of AbsV is in review [1]. 2. Instructions that Jdk master has middle end support for, but they cannot be matched with NEON instructions directly. Such as AddReductionVL, MulReductionVL, And/Or/XorReductionV These new instructions can be moved into jdk master first, but for auto-vectorization, the performance might not get improved. May I have a new patch for these? 3. Panama/Vector API specific instructions Such as Load/StoreVector ( 16 bits), VectorReinterpret, VectorMaskCmp, MaxV/MinV, VectorBlend etc. These instructions cannot be moved into jdk master first because there isn't middle-end support. Regards Yang [1] https://mail.openjdk.java.net/pipermail/aarch64-port-dev/2020-May/008861.html -----Original Message----- From: Andrew Haley Sent: Tuesday, May 26, 2020 4:25 PM To: Yang Zhang ; Paul Sandoz Cc: hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; nd Subject: Re: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes On 25/05/2020 09:26, Yang Zhang wrote: > In jdk master, what we need to do is that writing m4 file for existing > vector instructions and placed them to a new file aarch64_neon.ad. > If no question, I will do it right away. I'm not entirely sure that such a change is necessary now. In particular, reorganizing the existing vector instructions is IMO excessive, but I admit that it might be an improvement. But to my earlier question. please: can the new instructions be moved into jdk head first, and then merged into the Panama branch, or not? It'd help if this was possible. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From mandy.chung at oracle.com Wed Jun 24 19:21:18 2020 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 24 Jun 2020 12:21:18 -0700 Subject: [15] RFR JDK-8247785: Small clarification to the javadoc about builtin class loaders In-Reply-To: References: <37776e19-a225-9547-6d53-147aa69679b8@oracle.com> <43d00426-80f1-7e1e-d397-d82d6646d835@oracle.com> <0ddc3a45-6af7-1d6a-5112-e9a553a929ea@oracle.com> <81d4a655-76ad-2e53-d83c-724c385f2c7b@oracle.com> Message-ID: <4e8e3872-f89d-3107-c455-5994e1d44409@oracle.com> Hi Roger, David, Thanks for the help in improving this.? As a record, this webrev shows the version as David suggests: http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247785/webrev.00/ Mandy On 6/24/20 9:33 AM, Roger Riggs wrote: > Hi Mandy, > > I'm fine with this. > > Roger From john.r.rose at oracle.com Wed Jun 24 20:01:33 2020 From: john.r.rose at oracle.com (John Rose) Date: Wed, 24 Jun 2020 13:01:33 -0700 Subject: =?utf-8?Q?Re=3A_RFR=5B8238286=5D=3A__=27Add_new_flatMap_stream_op?= =?utf-8?Q?eration_that_is_more_amenable_to_pushing=E2=80=99?= In-Reply-To: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> References: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> Message-ID: <24F8FA40-3DC5-4156-B208-CCE54DB8CCF5@oracle.com> I like this new API point a lot; it allows flexible, local, temporary control inversion in the context of one stream transform. What?s the performance model? It seems like the SpinedBuffer implementation makes a worst-case assumption about the number of pending values, that there will be many instead of zero or one. But I guess the pipeline stuff already works in terms of pushes, so the good news might be that this is really just a drill-down from the user API into the kinds of operations (push-flavored) that go on most of the time. OK, so I like the function but I have a beef with its bike shed color. First of all, this is a control-inversion (CPS) pattern, which is very powerful but also notoriously hard to read. I think that in Java APIs, at least in Stream APIs, code is easier to read if the logical data flow is from left to right. (This is a language-specific observation. Apart from varargs, Java method APIs read favorably when extra control arguments are added onto the end of the argument list. Also, the convention for generic functional interfaces is that the return value type goes to the right, e.g., R in Function.) So the BiConsumer is backwards, because the logical return should be written, if not as a true return (which would appear at the end of type parameter lists), at the end of the incoming parameters (and in the last type parameter). I also think ?multi? is needlessly ?learned? sounding. A simple spatial preposition would work well: mapThrough, mapAcross, etc. I think I prefer mapAcross because the term ?across? can be read adverbially: ?we are mapping T across to Consumer?. So: mapAcross(BiConsumer> mapper) mapAcrossToInt(BiConsumer mapper) mapAcross?(IntObjConsumer mapper) This does require additional FI?s like IntObjConsumer, but I think that is a negligible cost. Making the control inversion *readable* is the high order bit here, not minimizing the number of trivial FIs. (I almost hear David Bowman, in his space suit, saying, ?My API? It?s full of bikesheds!? There?s a meme for that.) ? John On Jun 24, 2020, at 3:57 AM, Patrick Concannon wrote: > > Hi, > > Could someone please review myself and Julia's RFE and CSR for JDK-8238286 - 'Add new flatMap stream operation that is more amenable to pushing?? > > This proposal is to add a new flatMap-like operation: > > ` Stream mapMulti(BiConsumer, ? super T> mapper)` > > to the java.util.Stream class. This operation is more receptive to the pushing or yielding of values than the current implementation that internally assembles values (if any) into one or more streams. This addition includes the primitive variations of the operation i.e. mapMultiToInt, IntStream mapMulti, etc. > > issue: https://bugs.openjdk.java.net/browse/JDK-8238286 > csr: https://bugs.openjdk.java.net/browse/JDK-8248166 > > webrev: http://cr.openjdk.java.net/~pconcannon/8238286/webrevs/webrev.00/ > specdiff: http://cr.openjdk.java.net/~pconcannon/8238286/specdiff/specout.00/overview-summary.html > > > Kind regards, > Patrick & Julia From alexander.matveev at oracle.com Wed Jun 24 20:09:44 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Wed, 24 Jun 2020 13:09:44 -0700 Subject: [15] RFR: 8248255: [macos] Add failing DMG tests to problem list Message-ID: <2245b2f9-852e-127c-2cdf-1632f9cde05f@oracle.com> Please review the jpackage fix for bug [1] at [2]. Added EmptyFolderPackageTest, IconTest, AppImagePackageTest, SimplePackageTest, BasicTest to ProblemList.txt. [1] https://bugs.openjdk.java.net/browse/JDK-8248255 [2] http://cr.openjdk.java.net/~almatvee/8248255/webrev.00/ Thanks, Alexander From alexey.semenyuk at oracle.com Wed Jun 24 20:14:55 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Wed, 24 Jun 2020 16:14:55 -0400 Subject: [15] RFR: 8248255: [macos] Add failing DMG tests to problem list In-Reply-To: <2245b2f9-852e-127c-2cdf-1632f9cde05f@oracle.com> References: <2245b2f9-852e-127c-2cdf-1632f9cde05f@oracle.com> Message-ID: <3075dd00-f1ef-515d-3eab-351adc781be9@oracle.com> Looks good. - Alexey On 6/24/2020 4:09 PM, alexander.matveev at oracle.com wrote: > Please review the jpackage fix for bug [1] at [2]. > > Added EmptyFolderPackageTest, IconTest, AppImagePackageTest, > SimplePackageTest, BasicTest to ProblemList.txt. > > [1] https://bugs.openjdk.java.net/browse/JDK-8248255 > [2] http://cr.openjdk.java.net/~almatvee/8248255/webrev.00/ > > Thanks, > Alexander From john.r.rose at oracle.com Wed Jun 24 20:58:53 2020 From: john.r.rose at oracle.com (John Rose) Date: Wed, 24 Jun 2020 13:58:53 -0700 Subject: =?utf-8?Q?Re=3A_RFR=5B8238286=5D=3A__=27Add_new_flatMap_stream_op?= =?utf-8?Q?eration_that_is_more_amenable_to_pushing=E2=80=99?= In-Reply-To: <24F8FA40-3DC5-4156-B208-CCE54DB8CCF5@oracle.com> References: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> <24F8FA40-3DC5-4156-B208-CCE54DB8CCF5@oracle.com> Message-ID: <94CE2EB8-5FF6-47F2-B257-CBF70A0B2E4B@oracle.com> On Jun 24, 2020, at 1:01 PM, John Rose wrote: > > What?s the performance model? It seems like the SpinedBuffer > implementation makes a worst-case assumption about the number > of pending values, that there will be many instead of zero or one. > > But I guess the pipeline stuff already works in terms of pushes, so > the good news might be that this is really just a drill-down from the > user API into the kinds of operations (push-flavored) that go on > most of the time. (I dove straight into the code and missed the implementation note where you answered this question? Never mind.) From andy.herrick at oracle.com Wed Jun 24 21:29:31 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Wed, 24 Jun 2020 17:29:31 -0400 Subject: [15] RFR: 8248255: [macos] Add failing DMG tests to problem list In-Reply-To: <2245b2f9-852e-127c-2cdf-1632f9cde05f@oracle.com> References: <2245b2f9-852e-127c-2cdf-1632f9cde05f@oracle.com> Message-ID: <0feaf788-9c02-4a10-715f-2238846a3594@oracle.com> looks good. /Andy On 6/24/2020 4:09 PM, alexander.matveev at oracle.com wrote: > Please review the jpackage fix for bug [1] at [2]. > > Added EmptyFolderPackageTest, IconTest, AppImagePackageTest, > SimplePackageTest, BasicTest to ProblemList.txt. > > [1] https://bugs.openjdk.java.net/browse/JDK-8248255 > [2] http://cr.openjdk.java.net/~almatvee/8248255/webrev.00/ > > Thanks, > Alexander From david.holmes at oracle.com Wed Jun 24 22:24:39 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 25 Jun 2020 08:24:39 +1000 Subject: [15] RFR JDK-8247785: Small clarification to the javadoc about builtin class loaders In-Reply-To: <4e8e3872-f89d-3107-c455-5994e1d44409@oracle.com> References: <37776e19-a225-9547-6d53-147aa69679b8@oracle.com> <43d00426-80f1-7e1e-d397-d82d6646d835@oracle.com> <0ddc3a45-6af7-1d6a-5112-e9a553a929ea@oracle.com> <81d4a655-76ad-2e53-d83c-724c385f2c7b@oracle.com> <4e8e3872-f89d-3107-c455-5994e1d44409@oracle.com> Message-ID: <8dc476a6-3b28-1f91-3ca6-48a722bacfe1@oracle.com> On 25/06/2020 5:21 am, Mandy Chung wrote: > Hi Roger, David, > > Thanks for the help in improving this.? As a record, this webrev shows > the version as David suggests: > http://cr.openjdk.java.net/~mchung/jdk15/webrevs/8247785/webrev.00/ Looks good to me :) Thanks, David > Mandy > > On 6/24/20 9:33 AM, Roger Riggs wrote: >> Hi Mandy, >> >> I'm fine with this. >> >> Roger > From forax at univ-mlv.fr Wed Jun 24 23:32:13 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Thu, 25 Jun 2020 01:32:13 +0200 (CEST) Subject: =?utf-8?Q?Re:_RFR[8238286]:__'Add_new_flatMap_stream_?= =?utf-8?Q?operation_that_is_more_amenable_to_pushing=E2=80=99?= In-Reply-To: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> References: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> Message-ID: <366194096.16591.1593041533516.JavaMail.zimbra@u-pem.fr> Hi Patrick, ----- Mail original ----- > De: "Patrick Concannon" > ?: "core-libs-dev" > Envoy?: Mercredi 24 Juin 2020 12:57:01 > Objet: RFR[8238286]: 'Add new flatMap stream operation that is more amenable to pushing? > Hi, Hi, > > Could someone please review myself and Julia's RFE and CSR for JDK-8238286 - > 'Add new flatMap stream operation that is more amenable to pushing?? > > This proposal is to add a new flatMap-like operation: > > ` Stream mapMulti(BiConsumer, ? super T> mapper)` > > to the java.util.Stream class. This operation is more receptive to the pushing > or yielding of values than the current implementation that internally assembles > values (if any) into one or more streams. This addition includes the primitive > variations of the operation i.e. mapMultiToInt, IntStream mapMulti, etc. I don't really like the name "mapMulti", because flatMap can also be called mapMulti. I'm sure someone will come with a better name, mapPush ?, mapYield ? Why the BiConsumer takes a T as the second parameter and not as the first like in the bug description ? I get that you want to keep Consumer instead of Consumer because Consumer is not a valid target type for a lambda, but the BiConsumer should be able to take a ? super Consumer instead of just Consumer. In Stream.java, in the javadoc both the examples of mapMulti are using wrapper types instead of primitives, which is not the way we want people to use streams, stream of wrapper types cause the perf issue. In *Stream.java, in the default method, the 'this' in 'this'.flatMap is useless. In *Pipeline.java, i don't think you need the lambda downstream::accept given that a Sink is already a Consumer, instead of mapper.accept(downstream::accept, u); using the downstream directly should work mapper.accept(downstream, u); also it seems to me that the implementation of cancellationRequested() is useless given that Sink.Chained*::cancellationRequested already delegates to the downstream sink. I've also noticed that in ReferencePipelien.java, in Stream mapMulti(BiConsumer, ? super P_OUT> mapper), the cache field is missing, but its should be necessary anymore. > > issue: https://bugs.openjdk.java.net/browse/JDK-8238286 > > csr: https://bugs.openjdk.java.net/browse/JDK-8248166 > > > webrev: http://cr.openjdk.java.net/~pconcannon/8238286/webrevs/webrev.00/ > > specdiff: > http://cr.openjdk.java.net/~pconcannon/8238286/specdiff/specout.00/overview-summary.html > ? > > > Kind regards, > Patrick & Julia regards, R?mi From david.holmes at oracle.com Thu Jun 25 00:10:42 2020 From: david.holmes at oracle.com (David Holmes) Date: Thu, 25 Jun 2020 10:10:42 +1000 Subject: RFR: JDK-8240148: Uncaught exceptions from ScheduledThreadPoolExecutor.execute aren't reported In-Reply-To: <4617b527-fea8-ac45-27d1-23308443baa3@gmail.com> References: <4617b527-fea8-ac45-27d1-23308443baa3@gmail.com> Message-ID: <3e6fb70e-ff72-66ad-c58d-36131577e3e1@oracle.com> Hi Jaikiran, On 24/06/2020 11:44 pm, Jaikiran Pai wrote: > Could I please get a review and a sponsor for a fix for > https://bugs.openjdk.java.net/browse/JDK-8240148? I missed the filing of this bug. :( > The patch is available as a webrev at > https://cr.openjdk.java.net/~jpai/webrev/8240148/1/ > > The patch updates the ScheduledThreadPoolExecutor.execute(Runnable) to > wrap the user passed Runnable into a custom one which then catches any > Throwable and reports it through the Thread's uncaught exception > handler. After having reviewed the existing code in that class, this was > the easiest (and perhaps the only feasible?) and cleanest way that I > could think of, to fix this issue. I do not agree with the solution or even the formulation of the issue. This is something that has been raised and discussed a number of times since Java 5. See for example: http://cs.oswego.edu/pipermail/concurrency-interest/2011-April/007865.html The UncaughtExceptionHandler is, as the name clearly indicates, for execution in response to uncaught exceptions and is the last act of a dying thread. You are invoking it for a Runnable just because you happen to like to what it does by default for threads - and you don't even know what it will do as it may have been customised. It is an implementation detail with TPE and STPE whether exceptions from tasks lead to termination of the worker thread and thus result in execution of the UEH, or whether they catch them and continue. For TPE they can; but for STPE they never will as the task is wrapped in a Future. The advice has always been that any Runnable submitted as a task to an ExecutorService should take responsibility for dealing with any exceptions it encounters. David ----- > > A few things about this change: > > 1. I decided to use an anonymous class for the Runnable, in the > execute() method instead of a lambda because I was unsure if this part > of the JDK code is allowed to use a lambda. In the past I have seen > discussions where it was recommended not use lambda in certain parts of > the JDK either because of performance(?) reason or because "this part of > the code runs too early to be using a lambda" reason. I personally > prefer an anonymous class anyway, but if others feel using a lambda here > is relevant, then let me know. > > 2. Once the exception is reported using the UncaughtExceptionHandler, I > just return. Initially, I had considered throwing back the original > exception/throwable after reporting it via the UncaughtExceptionHandler, > but having looked at few others places within the JDK which do not throw > back the original exception, I decided to follow the same semantics. > > 3. Now that the passed Runnable is wrapped and submitted for execution, > I considered whether this would break any (unwritten) contract between > the ThreadPoolExecutor and the ThreadFactory#newThread(Runnable) > implementations. If any (JDK internal or user application specific) > ThreadFactory#newThread method was expecting the passed Runnable to be > of the same type or the same instance as that was submitted to the > ScheduledThreadPoolExecutor#execute(Runnable), then this change would > break such ThreadFactory#newThread implementations. I looked deeper in > the ThreadPoolExecutor code and from what I could see, this isn't a > practical concern, since even without this change, The > ThreadPoolExecutor in its private Worker class already sends a > "delegate" Runnable (an instance of the > ThreadPoolExecutor$Worker class) to the ThreadFactory's newThread method > and not the original Runnable that was submitted to the > execute(Runnable) method of the executor. So it doesn't look like > there's any written/unwritten contract that the ThreadFactory is > expected to receive the same Runnable instance that was passed to the > execute method of the executor. > > 4. The ScheduledThreadPoolExecutor has a different license than some of > the other files that I am used to, within the JDK code. So I haven't > edited it for any of the usual copyright year updates. > > 5. The patch contains a new (testng based) test case which reproduces > this issue and verifies the fix. > > -Jaikiran > > From paul.sandoz at oracle.com Thu Jun 25 00:58:07 2020 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 24 Jun 2020 17:58:07 -0700 Subject: =?utf-8?Q?Re=3A_RFR=5B8238286=5D=3A__=27Add_new_flatMap_stream_op?= =?utf-8?Q?eration_that_is_more_amenable_to_pushing=E2=80=99?= In-Reply-To: <24F8FA40-3DC5-4156-B208-CCE54DB8CCF5@oracle.com> References: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> <24F8FA40-3DC5-4156-B208-CCE54DB8CCF5@oracle.com> Message-ID: <2F795DFF-36B6-4903-B0AF-120973F29AD8@oracle.com> Hi, We traded CPS style for reusing existing functional interfaces. Originally the signature (my first choice) was as you indicate. By chance it just so happens that the current signature is the same shape as that for the accumulating argument type of the three arg collect terminal operation: Stream R collect(Supplier supplier, BiConsumer accumulator, BiConsumer combiner); IntStream R collect(Supplier supplier, ObjIntConsumer accumulator, BiConsumer combiner); Same for the accumulator of a Collector too. However, I suspect you would argue these terminal accumulation cases are different from the intermediate case, as we are not accumulating but passing or accepting (loosely returning) zero or more elements that replace the input element. It?s my hope that generic specialization will allow the primitive stream types to fade into the background, along with the primitive functional interfaces. In that respect the addition of three functional interfaces for use on the primitive stream types is not so terrible. Regarding the name, you should have seen the first one :-) it was terrible. Here?s my few brush strokes on the bike shed. I wonder what people think of mapAccept. The specification talks about accepting elements, because that is the operative method name on Consumer. So we can say "T is replaced with the elements accepted by the Consumer", or ? The Consumer accepts the elements that replace T" Paul. > On Jun 24, 2020, at 1:01 PM, John Rose wrote: > > I like this new API point a lot; it allows flexible, local, temporary > control inversion in the context of one stream transform. > > What?s the performance model? It seems like the SpinedBuffer > implementation makes a worst-case assumption about the number > of pending values, that there will be many instead of zero or one. > > But I guess the pipeline stuff already works in terms of pushes, so > the good news might be that this is really just a drill-down from the > user API into the kinds of operations (push-flavored) that go on > most of the time. > > OK, so I like the function but I have a beef with its bike shed > color. First of all, this is a control-inversion (CPS) pattern, > which is very powerful but also notoriously hard to read. > I think that in Java APIs, at least in Stream APIs, code is > easier to read if the logical data flow is from left to right. > > (This is a language-specific observation. Apart from varargs, > Java method APIs read favorably when extra control arguments > are added onto the end of the argument list. Also, the convention > for generic functional interfaces is that the return value type > goes to the right, e.g., R in Function.) > > So the BiConsumer is backwards, because the logical return > should be written, if not as a true return (which would appear > at the end of type parameter lists), at the end of the incoming > parameters (and in the last type parameter). > > I also think ?multi? is needlessly ?learned? sounding. A simple > spatial preposition would work well: mapThrough, mapAcross, etc. > I think I prefer mapAcross because the term ?across? can be read > adverbially: ?we are mapping T across to Consumer?. > > So: > > mapAcross(BiConsumer> mapper) > mapAcrossToInt(BiConsumer mapper) > mapAcross?(IntObjConsumer mapper) > > This does require additional FI?s like IntObjConsumer, but > I think that is a negligible cost. Making the control inversion > *readable* is the high order bit here, not minimizing the number > of trivial FIs. > > (I almost hear David Bowman, in his space suit, saying, ?My API? > It?s full of bikesheds!? There?s a meme for that.) > > ? John > > On Jun 24, 2020, at 3:57 AM, Patrick Concannon wrote: >> >> Hi, >> >> Could someone please review myself and Julia's RFE and CSR for JDK-8238286 - 'Add new flatMap stream operation that is more amenable to pushing?? >> >> This proposal is to add a new flatMap-like operation: >> >> ` Stream mapMulti(BiConsumer, ? super T> mapper)` >> >> to the java.util.Stream class. This operation is more receptive to the pushing or yielding of values than the current implementation that internally assembles values (if any) into one or more streams. This addition includes the primitive variations of the operation i.e. mapMultiToInt, IntStream mapMulti, etc. >> >> issue: https://bugs.openjdk.java.net/browse/JDK-8238286 >> csr: https://bugs.openjdk.java.net/browse/JDK-8248166 >> >> webrev: http://cr.openjdk.java.net/~pconcannon/8238286/webrevs/webrev.00/ >> specdiff: http://cr.openjdk.java.net/~pconcannon/8238286/specdiff/specout.00/overview-summary.html >> >> >> Kind regards, >> Patrick & Julia > From jai.forums2013 at gmail.com Thu Jun 25 01:31:54 2020 From: jai.forums2013 at gmail.com (Jaikiran Pai) Date: Thu, 25 Jun 2020 07:01:54 +0530 Subject: RFR: JDK-8240148: Uncaught exceptions from ScheduledThreadPoolExecutor.execute aren't reported In-Reply-To: <3e6fb70e-ff72-66ad-c58d-36131577e3e1@oracle.com> References: <4617b527-fea8-ac45-27d1-23308443baa3@gmail.com> <3e6fb70e-ff72-66ad-c58d-36131577e3e1@oracle.com> Message-ID: Thank you both Martin and David for your inputs. On 25/06/20 5:40 am, David Holmes wrote: > Hi Jaikiran, > > >> The patch is available as a webrev at >> https://cr.openjdk.java.net/~jpai/webrev/8240148/1/ >> >> The patch updates the ScheduledThreadPoolExecutor.execute(Runnable) to >> wrap the user passed Runnable into a custom one which then catches any >> Throwable and reports it through the Thread's uncaught exception >> handler. After having reviewed the existing code in that class, this was >> the easiest (and perhaps the only feasible?) and cleanest way that I >> could think of, to fix this issue. > > I do not agree with the solution or even the formulation of the issue. > > This is something that has been raised and discussed a number of times > since Java 5. See for example: > > http://cs.oswego.edu/pipermail/concurrency-interest/2011-April/007865.html > > > The UncaughtExceptionHandler is, as the name clearly indicates, for > execution in response to uncaught exceptions and is the last act of a > dying thread. You are right. I looked into the ScheduledThreadPoolExecutor implementation details but failed to read up on the javadoc of UncaughtExceptionHandler which already clearly states that it's meant for use when a thread is terminating. Thank you for linking that previous discussion too. -Jaikiran From kim.barrett at oracle.com Thu Jun 25 03:50:05 2020 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 24 Jun 2020 23:50:05 -0400 Subject: [OpenJDK 2D-Dev] RFR: 8246032: Implementation of JEP 347: Adopt C++14 Language Features in HotSpot In-Reply-To: References: <5EDE9A75.6020504@oracle.com> <5EE5AB71.20505@oracle.com> Message-ID: <3FB6F2CB-1085-4F1B-BFE5-7554F2637004@oracle.com> > On Jun 24, 2020, at 7:01 AM, Magnus Ihse Bursie wrote: > > On 2020-06-18 08:34, Kim Barrett wrote: >>> On Jun 15, 2020, at 7:41 AM, Kim Barrett wrote: >>> >>>> On Jun 14, 2020, at 12:45 AM, Philip Race wrote: >>>> >>>> Kim, >>>> >>>> >>>> Until it says in "the JDK" and not "in HotSpot" you have not addressed my main point. >>>> Please rename the JEP. >> After some off-list discussions I have a better idea of what Phil was >> asking for and why. In response I've changed the JEP, replacing a few >> occurrences of "HotSpot" with "the JDK", as described below. All >> other occurrences of "HotSpot" have been left as-is. >> >> Title: >> JEP 347: Adopt C++14 Language Features in HotSpot >> => >> JEP 347: Adopt C++14 Language Features in the JDK >> >> Summary: >> Allow the use of selected C++14 language features in the HotSpot C++ source code. >> => >> Allow the use of selected C++14 language features in the JDK C++ source code. >> >> Goals: >> The purpose of this JEP is to formally allow C++ source code changes within HotSpot to take advantage of some C++14 standard language features. >> => >> The purpose of this JEP is to formally allow C++ source code changes within the JDK to take advantage of some C++14 standard language features. >> > This all looks good to me. > > /Magnus Thanks. From amaembo at gmail.com Thu Jun 25 04:04:17 2020 From: amaembo at gmail.com (Tagir Valeev) Date: Thu, 25 Jun 2020 11:04:17 +0700 Subject: =?UTF-8?Q?Re=3A_RFR=5B8238286=5D=3A_=27Add_new_flatMap_stream_operation_?= =?UTF-8?Q?that_is_more_amenable_to_pushing=E2=80=99?= In-Reply-To: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> References: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> Message-ID: Hello! To me, it looks like it's possible to make the better default implementation. It could be done even as a separate static method: static Stream ofPusher(Consumer> pusher) { return StreamSupport.stream(new Spliterator<>() { private Spliterator delegate; @Override public boolean tryAdvance(Consumer action) { initDelegate(); return delegate.tryAdvance(action); } private void initDelegate() { if (delegate == null) { Stream.Builder builder = Stream.builder(); // or use SpinedBuffer directly pusher.accept(builder); delegate = builder.build().spliterator(); } } @Override public void forEachRemaining(Consumer action) { if (delegate != null) { delegate.forEachRemaining(action); } else { pusher.accept(action); } } @Override public Spliterator trySplit() { initDelegate(); return delegate.trySplit(); } @Override public long estimateSize() { return Long.MAX_VALUE; } @Override public int characteristics() { return 0; } }, false); } In this case, we are buffering only if short-circuit operation or splitting is requested. Otherwise, forEachRemaining will just delegate to the pusher. Now, the default implementation could be rewritten as Stream mapMulti(Stream stream, BiConsumer, ? super T> mapper) { Objects.requireNonNull(mapper); return stream.flatMap(e -> ofPusher(sink -> mapper.accept(sink, e))); } And now, I don't think it's necessary to specialize it at all. Probably it's not necessary to introduce mapMulti at all as well, as now it's a trivial delegate to ofPusher. With best regards, Tagir Valeev. On Wed, Jun 24, 2020 at 5:58 PM Patrick Concannon wrote: > > Hi, > > Could someone please review myself and Julia's RFE and CSR for JDK-8238286 - 'Add new flatMap stream operation that is more amenable to pushing?? > > This proposal is to add a new flatMap-like operation: > > ` Stream mapMulti(BiConsumer, ? super T> mapper)` > > to the java.util.Stream class. This operation is more receptive to the pushing or yielding of values than the current implementation that internally assembles values (if any) into one or more streams. This addition includes the primitive variations of the operation i.e. mapMultiToInt, IntStream mapMulti, etc. > > issue: https://bugs.openjdk.java.net/browse/JDK-8238286 > csr: https://bugs.openjdk.java.net/browse/JDK-8248166 > > webrev: http://cr.openjdk.java.net/~pconcannon/8238286/webrevs/webrev.00/ > specdiff: http://cr.openjdk.java.net/~pconcannon/8238286/specdiff/specout.00/overview-summary.html > > > Kind regards, > Patrick & Julia From rahul.r.yadav at oracle.com Thu Jun 25 09:22:03 2020 From: rahul.r.yadav at oracle.com (Rahul Yadav) Date: Thu, 25 Jun 2020 10:22:03 +0100 Subject: RFR 8245302: Upgrade LogRecord to support long thread ids and remove its usage of ThreadLocal In-Reply-To: References: <94B2555B-981F-40EB-AB5F-31E2BE068FF0@oracle.com> <03c38ee2-24c2-cf2d-e428-3f8747888978@oracle.com> <5e5bc99f-7d46-df1a-08c1-ee37958a3462@oracle.com> <58cda3b5-b75b-6782-01eb-2c935a0e402d@oracle.com> <362dc841-a809-95f5-961f-3856f3230b6f@oracle.com> Message-ID: <8c5e6931-b3fd-8e19-206f-3eedf047e37c@oracle.com> Hello, Deprecated tag in LogRecord getter and setter for threadID was missing "since" and so i have added it. webrev : http://cr.openjdk.java.net/~ryadav/webrev_8245302/webrev.00/ - rahul On 22/06/2020 19:38, Daniel Fuchs wrote: > Hi Rahul, > > Looks good to me as well. > > Reviewed. > > best regards, > > -- daniel > > On 19/06/2020 10:00, Rahul Yadav wrote: >> Thank you Alan, updated webrev. >> >> webrev : >> http://cr.openjdk.java.net/~ryadav/webrev_8245302/webrev.00/index.html >> >> - rahul >> >> On 19/06/2020 08:43, Alan Bateman wrote: >>> On 18/06/2020 23:37, Rahul Yadav wrote: >>>> Hi Alan, >>>> >>>> Thank you for the feedback.I have updated the webrev. >>>> >>>> webrev : >>>> http://cr.openjdk.java.net/~ryadav/webrev_8245302/webrev.00/index.html >>> This looks quite good. >>> >>> The comment in shortShortID has "any positive long less than >>> Integer.MAX_VALUE" but it's actually <= MAX_VALUE. >>> >>> I don't think MIN_SEQUENTIAL_THREAD_ID is used so I assume it can be >>> removed. >>> >>> The @return for setLongThreadID has a description "Log Record" but >>> this should "this LogRecord". >>> >>> Can you update SerializeLogRecordTest with clear instructions on how >>> to generate the stream? This will help future maintainers that may >>> have to update this test. >>> >>> -Alan >> > From daniel.fuchs at oracle.com Thu Jun 25 09:28:00 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 25 Jun 2020 10:28:00 +0100 Subject: =?UTF-8?Q?Re=3a_RFR=5b8238286=5d=3a_=27Add_new_flatMap_stream_opera?= =?UTF-8?Q?tion_that_is_more_amenable_to_pushing=e2=80=99?= In-Reply-To: <366194096.16591.1593041533516.JavaMail.zimbra@u-pem.fr> References: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> <366194096.16591.1593041533516.JavaMail.zimbra@u-pem.fr> Message-ID: Hi R?mi, On 25/06/2020 00:32, Remi Forax wrote: > I get that you want to keep Consumer instead of Consumer because Consumer is not a valid target type for a lambda, but the BiConsumer should be able to take a ? super Consumer instead of just Consumer. Though I don't dispute that a strict application of the rules of covariance and contravariance would require to design a signature that accepts a `? super Consumer` - how would you implement a BiConsumer with a signature that doesn't take a Consumer? Such an implementation would be unable to push anything downstream without having to cast back the consumer to Consumer. My personal preference would be to vote in favor of the simpler signature - which IMO is more readable and easier to understand. best regards, -- daniel From daniel.fuchs at oracle.com Thu Jun 25 09:51:05 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 25 Jun 2020 10:51:05 +0100 Subject: RFR 8245302: Upgrade LogRecord to support long thread ids and remove its usage of ThreadLocal In-Reply-To: <8c5e6931-b3fd-8e19-206f-3eedf047e37c@oracle.com> References: <94B2555B-981F-40EB-AB5F-31E2BE068FF0@oracle.com> <03c38ee2-24c2-cf2d-e428-3f8747888978@oracle.com> <5e5bc99f-7d46-df1a-08c1-ee37958a3462@oracle.com> <58cda3b5-b75b-6782-01eb-2c935a0e402d@oracle.com> <362dc841-a809-95f5-961f-3856f3230b6f@oracle.com> <8c5e6931-b3fd-8e19-206f-3eedf047e37c@oracle.com> Message-ID: Thanks for fixing that Rahul! Looks good. best regards, -- daniel On 25/06/2020 10:22, Rahul Yadav wrote: > Hello, > > Deprecated tag in LogRecord getter and setter for threadID was missing > "since" and so i have added it. > > webrev : http://cr.openjdk.java.net/~ryadav/webrev_8245302/webrev.00/ > > - rahul From chris.hegarty at oracle.com Thu Jun 25 15:01:03 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 25 Jun 2020 16:01:03 +0100 Subject: RFR [1]6 8248326 Add a minimal serialization test for local records Message-ID: <61ED3E20-14D6-4CCE-8CB1-9F26A5CECB06@oracle.com> While working on some record serialization changes recently, I noticed that we don?t have any coverage for local records. Here is a simple extension to an existing records test. diff -r 4e186efa6cbf test/jdk/java/io/Serializable/records/BasicRecordSer.java --- a/test/jdk/java/io/Serializable/records/BasicRecordSer.java Thu Jun 25 09:54:19 2020 +0100 +++ b/test/jdk/java/io/Serializable/records/BasicRecordSer.java Thu Jun 25 15:40:13 2020 +0100 @@ -132,6 +132,20 @@ assertEquals(objDeserialized, objToSerialize); } + /** Tests serializing and deserializing of local records. */ + @Test + public void testLocalRecord() throws Exception { + out.println("\n---"); + record Point(int x, int y) implements Serializable { } + record Rectangle(Point bottomLeft, Point topRight) implements Serializable { } + var objToSerialize = new Rectangle(new Point(0, 1), new Point (5, 6)); + out.println("serializing : " + objToSerialize); + var objDeserialized = serializeDeserialize(objToSerialize); + out.println("deserialized: " + objDeserialized); + assertEquals(objToSerialize, objDeserialized); + assertEquals(objDeserialized, objToSerialize); + } + /** Tests back references of Serializable record objects in the stream. */ @Test public void testSerializableBackRefs() throws Exception { -Chris. From daniel.fuchs at oracle.com Thu Jun 25 15:24:12 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 25 Jun 2020 16:24:12 +0100 Subject: RFR [1]6 8248326 Add a minimal serialization test for local records In-Reply-To: <61ED3E20-14D6-4CCE-8CB1-9F26A5CECB06@oracle.com> References: <61ED3E20-14D6-4CCE-8CB1-9F26A5CECB06@oracle.com> Message-ID: <9716b463-2b52-23bd-0762-5290c6f6ddeb@oracle.com> Hi Chris, Looks good to me. It might also be beneficial to double check that the hash codes of the two objects are equal. best regards, -- daniel On 25/06/2020 16:01, Chris Hegarty wrote: > While working on some record serialization changes recently, I noticed that we don?t have any coverage for local records. Here is a simple extension to an existing records test. > > diff -r 4e186efa6cbf test/jdk/java/io/Serializable/records/BasicRecordSer.java > --- a/test/jdk/java/io/Serializable/records/BasicRecordSer.java Thu Jun 25 09:54:19 2020 +0100 > +++ b/test/jdk/java/io/Serializable/records/BasicRecordSer.java Thu Jun 25 15:40:13 2020 +0100 > @@ -132,6 +132,20 @@ > assertEquals(objDeserialized, objToSerialize); > } > > + /** Tests serializing and deserializing of local records. */ > + @Test > + public void testLocalRecord() throws Exception { > + out.println("\n---"); > + record Point(int x, int y) implements Serializable { } > + record Rectangle(Point bottomLeft, Point topRight) implements Serializable { } > + var objToSerialize = new Rectangle(new Point(0, 1), new Point (5, 6)); > + out.println("serializing : " + objToSerialize); > + var objDeserialized = serializeDeserialize(objToSerialize); > + out.println("deserialized: " + objDeserialized); > + assertEquals(objToSerialize, objDeserialized); > + assertEquals(objDeserialized, objToSerialize); > + } > + > /** Tests back references of Serializable record objects in the stream. */ > @Test > public void testSerializableBackRefs() throws Exception { > > -Chris. > From dev at anthonyv.be Thu Jun 25 17:12:20 2020 From: dev at anthonyv.be (Anthony Vanelverdinghe) Date: Thu, 25 Jun 2020 19:12:20 +0200 Subject: =?UTF-8?Q?Re=3a_RFR=5b8238286=5d=3a_=27Add_new_flatMap_stream_opera?= =?UTF-8?Q?tion_that_is_more_amenable_to_pushing=e2=80=99?= In-Reply-To: <2F795DFF-36B6-4903-B0AF-120973F29AD8@oracle.com> References: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> <24F8FA40-3DC5-4156-B208-CCE54DB8CCF5@oracle.com> <2F795DFF-36B6-4903-B0AF-120973F29AD8@oracle.com> Message-ID: <6ea32a67-8292-39c7-df04-58ad15b2fde4@anthonyv.be> Hi Given the signature of flatMap is: Stream flatMap?(Function> mapper) I'd like to propose the following signature for the new method: Stream builderMap(BiConsumer> mapper) This way both methods are named "...Map", and the name "builderMap" follows naturally from the argument's type. If the given mapper invokes Stream.Builder::build, an IllegalStateException should be thrown. Kind regards, Anthony On 25/06/2020 02:58, Paul Sandoz wrote: > Hi, > > We traded CPS style for reusing existing functional interfaces. Originally the signature (my first choice) was as you indicate. > > By chance it just so happens that the current signature is the same shape as that for the accumulating argument type of the three arg collect terminal operation: > > Stream > R collect(Supplier supplier, > BiConsumer accumulator, > BiConsumer combiner); > > IntStream > R collect(Supplier supplier, > ObjIntConsumer accumulator, > BiConsumer combiner); > > Same for the accumulator of a Collector too. > > However, I suspect you would argue these terminal accumulation cases are different from the intermediate case, as we are not accumulating but passing or accepting (loosely returning) zero or more elements that replace the input element. > > It?s my hope that generic specialization will allow the primitive stream types to fade into the background, along with the primitive functional interfaces. In that respect the addition of three functional interfaces for use on the primitive stream types is not so terrible. > > > Regarding the name, you should have seen the first one :-) it was terrible. > > Here?s my few brush strokes on the bike shed. I wonder what people think of mapAccept. The specification talks about accepting elements, because that is the operative method name on Consumer. So we can say "T is replaced with the elements accepted by the Consumer", or ? The Consumer accepts the elements that replace T" > > Paul. > > > >> On Jun 24, 2020, at 1:01 PM, John Rose wrote: >> >> I like this new API point a lot; it allows flexible, local, temporary >> control inversion in the context of one stream transform. >> >> What?s the performance model? It seems like the SpinedBuffer >> implementation makes a worst-case assumption about the number >> of pending values, that there will be many instead of zero or one. >> >> But I guess the pipeline stuff already works in terms of pushes, so >> the good news might be that this is really just a drill-down from the >> user API into the kinds of operations (push-flavored) that go on >> most of the time. >> >> OK, so I like the function but I have a beef with its bike shed >> color. First of all, this is a control-inversion (CPS) pattern, >> which is very powerful but also notoriously hard to read. >> I think that in Java APIs, at least in Stream APIs, code is >> easier to read if the logical data flow is from left to right. >> >> (This is a language-specific observation. Apart from varargs, >> Java method APIs read favorably when extra control arguments >> are added onto the end of the argument list. Also, the convention >> for generic functional interfaces is that the return value type >> goes to the right, e.g., R in Function.) >> >> So the BiConsumer is backwards, because the logical return >> should be written, if not as a true return (which would appear >> at the end of type parameter lists), at the end of the incoming >> parameters (and in the last type parameter). >> >> I also think ?multi? is needlessly ?learned? sounding. A simple >> spatial preposition would work well: mapThrough, mapAcross, etc. >> I think I prefer mapAcross because the term ?across? can be read >> adverbially: ?we are mapping T across to Consumer?. >> >> So: >> >> mapAcross(BiConsumer> mapper) >> mapAcrossToInt(BiConsumer mapper) >> mapAcross?(IntObjConsumer mapper) >> >> This does require additional FI?s like IntObjConsumer, but >> I think that is a negligible cost. Making the control inversion >> *readable* is the high order bit here, not minimizing the number >> of trivial FIs. >> >> (I almost hear David Bowman, in his space suit, saying, ?My API? >> It?s full of bikesheds!? There?s a meme for that.) >> >> ? John >> >> On Jun 24, 2020, at 3:57 AM, Patrick Concannon wrote: >>> Hi, >>> >>> Could someone please review myself and Julia's RFE and CSR for JDK-8238286 - 'Add new flatMap stream operation that is more amenable to pushing?? >>> >>> This proposal is to add a new flatMap-like operation: >>> >>> ` Stream mapMulti(BiConsumer, ? super T> mapper)` >>> >>> to the java.util.Stream class. This operation is more receptive to the pushing or yielding of values than the current implementation that internally assembles values (if any) into one or more streams. This addition includes the primitive variations of the operation i.e. mapMultiToInt, IntStream mapMulti, etc. >>> >>> issue: https://bugs.openjdk.java.net/browse/JDK-8238286 >>> csr: https://bugs.openjdk.java.net/browse/JDK-8248166 >>> >>> webrev: http://cr.openjdk.java.net/~pconcannon/8238286/webrevs/webrev.00/ >>> specdiff: http://cr.openjdk.java.net/~pconcannon/8238286/specdiff/specout.00/overview-summary.html >>> >>> >>> Kind regards, >>> Patrick & Julia From brian.goetz at oracle.com Thu Jun 25 20:04:27 2020 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 25 Jun 2020 16:04:27 -0400 Subject: =?UTF-8?Q?Re=3a_RFR=5b8238286=5d=3a_=27Add_new_flatMap_stream_opera?= =?UTF-8?Q?tion_that_is_more_amenable_to_pushing=e2=80=99?= In-Reply-To: <366194096.16591.1593041533516.JavaMail.zimbra@u-pem.fr> References: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> <366194096.16591.1593041533516.JavaMail.zimbra@u-pem.fr> Message-ID: <04d64c3f-6269-d41d-d19b-d136a2ac929e@oracle.com> On 6/24/2020 7:32 PM, Remi Forax wrote: > I don't really like the name "mapMulti", because flatMap can also be called mapMulti. Public service announcement: remember just how frustrating it is for Patrick to have put in all this work and get only comments like "I don't like the name." If you think everything about it is fantastic, except maybe the name, then you can say so, and then maybe make a comment on the name -- or not. If you have anything more substantial to say, please don't comment on bikeshed colors like naming before you've made your more substantial comments -- or before everyone else has as well. Because all that will happen is we'll have 100 replies-to-replies about the name, and poor Patrick will get no substantive comments. (Personally, I think the name is fine.? It is reminiscent of LinQ's "SelectMany", which is their flatMap.) From forax at univ-mlv.fr Thu Jun 25 21:40:16 2020 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Thu, 25 Jun 2020 23:40:16 +0200 (CEST) Subject: =?utf-8?Q?Re:_RFR[8238286]:_'Add_new_flatMap_stream_?= =?utf-8?Q?operation_that_is_more_amenable_to_pushing=E2=80=99?= In-Reply-To: <04d64c3f-6269-d41d-d19b-d136a2ac929e@oracle.com> References: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> <366194096.16591.1593041533516.JavaMail.zimbra@u-pem.fr> <04d64c3f-6269-d41d-d19b-d136a2ac929e@oracle.com> Message-ID: <1829541220.727156.1593121216586.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Brian Goetz" > ?: "Remi Forax" , "Patrick Concannon" > Cc: "core-libs-dev" > Envoy?: Jeudi 25 Juin 2020 22:04:27 > Objet: Re: RFR[8238286]: 'Add new flatMap stream operation that is more amenable to pushing? > On 6/24/2020 7:32 PM, Remi Forax wrote: >> I don't really like the name "mapMulti", because flatMap can also be called >> mapMulti. > > Public service announcement: remember just how frustrating it is for > Patrick to have put in all this work and get only comments like "I don't > like the name." > > If you think everything about it is fantastic, except maybe the name, > then you can say so, and then maybe make a comment on the name -- or not. > > If you have anything more substantial to say, please don't comment on > bikeshed colors like naming before you've made your more substantial > comments -- or before everyone else has as well. Because all that will > happen is we'll have 100 replies-to-replies about the name, and poor > Patrick will get no substantive comments. you can also read after the "i don't like the name", there is an explanation, it starts with "because ..." > > (Personally, I think the name is fine.? It is reminiscent of LinQ's > "SelectMany", which is their flatMap.) yes, that's exactly my point, flatMap can be called mapMulti too and my fear is that people will use mapMulti were they should use flatMap and vice-versa, mapMulti is not a good enough name. also you're not fair because i'm not the only one that find the name is not good and you have cherry picked only one sentence of my reply. R?mi From daniel.daugherty at oracle.com Thu Jun 25 21:45:40 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 25 Jun 2020 17:45:40 -0400 Subject: RFR(T): 8248358: ProblemList sun/nio/ch/TestMaxCachedBufferSize.java on macOSX Message-ID: <17c6bd4a-edb4-d261-9db7-29f2c3a7e537@oracle.com> Greetings, I'm doing another round of reduce-the-noise in the CI in preparation for the upcoming weekend... So I have another trivial review... Here's the bug for the failures: ??? JDK-8212812 sun/nio/ch/TestMaxCachedBufferSize.java timeout ??? https://bugs.openjdk.java.net/browse/JDK-8212812 and here's the bug for the ProblemListing: ??? JDK-8248358 ProblemList serviceability/jvmti/ModuleAwareAgents/ThreadStart/MAAThreadStart.java on Windows ??? https://bugs.openjdk.java.net/browse/JDK-8248358 Here's the context diff: $ hg diff diff -r cf65909b98c5 test/jdk/ProblemList.txt --- a/test/jdk/ProblemList.txt??? Thu Jun 25 15:00:59 2020 -0400 +++ b/test/jdk/ProblemList.txt??? Thu Jun 25 17:39:01 2020 -0400 @@ -630,6 +630,8 @@ ?java/nio/channels/Selector/Wakeup.java 6963118 windows-all +sun/nio/ch/TestMaxCachedBufferSize.java 8212812 macosx-all + ?############################################################################ Thanks, in advance, for any comments, questions or suggestions. Dan From forax at univ-mlv.fr Thu Jun 25 21:46:08 2020 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Thu, 25 Jun 2020 23:46:08 +0200 (CEST) Subject: =?utf-8?Q?Re:_RFR[8238286]:_'Add_new_flatMap_stream_?= =?utf-8?Q?operation_that_is_more_amenable_to_pushing=E2=80=99?= In-Reply-To: References: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> <366194096.16591.1593041533516.JavaMail.zimbra@u-pem.fr> Message-ID: <1818580489.728182.1593121568927.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Daniel Fuchs" > ?: "Remi Forax" , "Patrick Concannon" > Cc: "core-libs-dev" > Envoy?: Jeudi 25 Juin 2020 11:28:00 > Objet: Re: RFR[8238286]: 'Add new flatMap stream operation that is more amenable to pushing? > Hi R?mi, Hi Daniel, > > On 25/06/2020 00:32, Remi Forax wrote: >> I get that you want to keep Consumer instead of Consumer because >> Consumer is not a valid target type for a lambda, but the BiConsumer >> should be able to take a ? super Consumer instead of just Consumer. > > Though I don't dispute that a strict application of the rules of > covariance and contravariance would require to design a signature > that accepts a `? super Consumer` - how would you implement a > BiConsumer with a signature that doesn't take a Consumer? > > Such an implementation would be unable to push anything downstream > without having to cast back the consumer to Consumer. if i have already have a BiConsumer, Object>, i would like to be able to call Stream.mapMulti() with that bi-consumer as argument. > > My personal preference would be to vote in favor of the simpler > signature - which IMO is more readable and easier to understand. > > best regards, regards, R?mi > > -- daniel From forax at univ-mlv.fr Thu Jun 25 22:12:22 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 26 Jun 2020 00:12:22 +0200 (CEST) Subject: =?utf-8?Q?Re:_RFR[8238286]:_'Add_new_flatMap_stream_?= =?utf-8?Q?operation_that_is_more_amenable_to_pushing=E2=80=99?= In-Reply-To: <1818580489.728182.1593121568927.JavaMail.zimbra@u-pem.fr> References: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> <366194096.16591.1593041533516.JavaMail.zimbra@u-pem.fr> <1818580489.728182.1593121568927.JavaMail.zimbra@u-pem.fr> Message-ID: <1193990764.732096.1593123142068.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Remi Forax" > ?: "Daniel Fuchs" > Cc: "Patrick Concannon" , "core-libs-dev" > Envoy?: Jeudi 25 Juin 2020 23:46:08 > Objet: Re: RFR[8238286]: 'Add new flatMap stream operation that is more amenable to pushing? > ----- Mail original ----- >> De: "Daniel Fuchs" >> ?: "Remi Forax" , "Patrick Concannon" >> >> Cc: "core-libs-dev" >> Envoy?: Jeudi 25 Juin 2020 11:28:00 >> Objet: Re: RFR[8238286]: 'Add new flatMap stream operation that is more amenable >> to pushing? > >> Hi R?mi, > > Hi Daniel, > >> >> On 25/06/2020 00:32, Remi Forax wrote: >>> I get that you want to keep Consumer instead of Consumer because >>> Consumer is not a valid target type for a lambda, but the BiConsumer >>> should be able to take a ? super Consumer instead of just Consumer. >> >> Though I don't dispute that a strict application of the rules of >> covariance and contravariance would require to design a signature >> that accepts a `? super Consumer` - how would you implement a >> BiConsumer with a signature that doesn't take a Consumer? >> >> Such an implementation would be unable to push anything downstream >> without having to cast back the consumer to Consumer. > > if i have already have a BiConsumer, Object>, i would like to > be able to call Stream.mapMulti() with that bi-consumer as argument. and obviously, i got it wrong, Consumer is not a super-type of Consumer, it should be a BiConsumer, Object> or a BiConsumer, Object>, etc. > >> >> My personal preference would be to vote in favor of the simpler >> signature - which IMO is more readable and easier to understand. >> >> best regards, regards, R?mi > >> > > -- daniel From igor.ignatyev at oracle.com Thu Jun 25 22:38:23 2020 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Thu, 25 Jun 2020 15:38:23 -0700 Subject: RFR(T): 8248358: ProblemList sun/nio/ch/TestMaxCachedBufferSize.java on macOSX In-Reply-To: <17c6bd4a-edb4-d261-9db7-29f2c3a7e537@oracle.com> References: <17c6bd4a-edb4-d261-9db7-29f2c3a7e537@oracle.com> Message-ID: <6309D70B-F6DF-4F40-9113-EDDBC4747F49@oracle.com> Hi Dan, LGTM Thanks, -- Igor > JDK-8248358 ProblemList serviceability/jvmti/ModuleAwareAgents/ThreadStart/MAAThreadStart.java on Windows > https://bugs.openjdk.java.net/browse/JDK-8248358 I guess you meant JDK-8248358 ProblemList sun/nio/ch/TestMaxCachedBufferSize.java on macOSX > On Jun 25, 2020, at 2:45 PM, Daniel D. Daugherty wrote: > > Greetings, > > I'm doing another round of reduce-the-noise in the CI in preparation > for the upcoming weekend... So I have another trivial review... > > Here's the bug for the failures: > > JDK-8212812 sun/nio/ch/TestMaxCachedBufferSize.java timeout > https://bugs.openjdk.java.net/browse/JDK-8212812 > > and here's the bug for the ProblemListing: > > JDK-8248358 ProblemList serviceability/jvmti/ModuleAwareAgents/ThreadStart/MAAThreadStart.java on Windows > https://bugs.openjdk.java.net/browse/JDK-8248358 > > Here's the context diff: > > $ hg diff > diff -r cf65909b98c5 test/jdk/ProblemList.txt > --- a/test/jdk/ProblemList.txt Thu Jun 25 15:00:59 2020 -0400 > +++ b/test/jdk/ProblemList.txt Thu Jun 25 17:39:01 2020 -0400 > @@ -630,6 +630,8 @@ > > java/nio/channels/Selector/Wakeup.java 6963118 windows-all > > +sun/nio/ch/TestMaxCachedBufferSize.java 8212812 macosx-all > + > ############################################################################ > > > Thanks, in advance, for any comments, questions or suggestions. > > Dan > > > > > > From daniel.daugherty at oracle.com Thu Jun 25 22:40:13 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 25 Jun 2020 18:40:13 -0400 Subject: RFR(T): 8248358: ProblemList sun/nio/ch/TestMaxCachedBufferSize.java on macOSX In-Reply-To: <6309D70B-F6DF-4F40-9113-EDDBC4747F49@oracle.com> References: <17c6bd4a-edb4-d261-9db7-29f2c3a7e537@oracle.com> <6309D70B-F6DF-4F40-9113-EDDBC4747F49@oracle.com> Message-ID: <26c6e6e7-b333-ef14-8c85-433d48165ac1@oracle.com> Thanks for the fast review Igor! Dan On 6/25/20 6:38 PM, Igor Ignatyev wrote: > Hi Dan, > > LGTM > > Thanks, > -- Igor > > >> JDK-8248358 ProblemList serviceability/jvmti/ModuleAwareAgents/ThreadStart/MAAThreadStart.java on Windows >> https://bugs.openjdk.java.net/browse/JDK-8248358 > I guess you meant JDK-8248358 ProblemList sun/nio/ch/TestMaxCachedBufferSize.java on macOSX > > >> On Jun 25, 2020, at 2:45 PM, Daniel D. Daugherty wrote: >> >> Greetings, >> >> I'm doing another round of reduce-the-noise in the CI in preparation >> for the upcoming weekend... So I have another trivial review... >> >> Here's the bug for the failures: >> >> JDK-8212812 sun/nio/ch/TestMaxCachedBufferSize.java timeout >> https://bugs.openjdk.java.net/browse/JDK-8212812 >> >> and here's the bug for the ProblemListing: >> >> JDK-8248358 ProblemList serviceability/jvmti/ModuleAwareAgents/ThreadStart/MAAThreadStart.java on Windows >> https://bugs.openjdk.java.net/browse/JDK-8248358 >> >> Here's the context diff: >> >> $ hg diff >> diff -r cf65909b98c5 test/jdk/ProblemList.txt >> --- a/test/jdk/ProblemList.txt Thu Jun 25 15:00:59 2020 -0400 >> +++ b/test/jdk/ProblemList.txt Thu Jun 25 17:39:01 2020 -0400 >> @@ -630,6 +630,8 @@ >> >> java/nio/channels/Selector/Wakeup.java 6963118 windows-all >> >> +sun/nio/ch/TestMaxCachedBufferSize.java 8212812 macosx-all >> + >> ############################################################################ >> >> >> Thanks, in advance, for any comments, questions or suggestions. >> >> Dan >> >> >> >> >> >> From huizhe.wang at oracle.com Thu Jun 25 23:53:08 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Thu, 25 Jun 2020 16:53:08 -0700 Subject: RFR [16/java.xml] 8248348: Regression caused by the update to BCEL 6.0 Message-ID: <09c93e75-8055-fbb9-9cde-195f0f002f1a@oracle.com> Hi, Please review a fix to a BCEL regression. At issue was the addition of hashCode() method to Instruction.java in BCEL 6.0. This hashCode() method was implemented to return the instruction's opcode that unfortunately is mutable. The problem hasn't showed up until the code path led to calls to remove from a HashSet a target that has been changed since it was added to the HashSet. The proposed patch is to make the hashCode final/immutable. This patch implies that a target object is considered the same one even if its field values may have been changed. It therefore may not be appropriate in other situations (or may cause problems). However, since it had always had no hashCode() override before BCEL 6.0, thereby relying on Object's identity hash code, its use case has been limited and time-tested. It can benefit from this patch in that it provides the same function as Object's hash code, and then serves as a reminder (to any who might read the code) how it was used (objects considered to be the same over the course as far as the hashCode is concerned). JBS: https://bugs.openjdk.java.net/browse/JDK-8248348 webrevs: http://cr.openjdk.java.net/~joehw/jdk16/8248348/webrev/ Thanks, Joe From ecki at zusammenkunft.net Fri Jun 26 00:14:13 2020 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Fri, 26 Jun 2020 00:14:13 +0000 Subject: RFR [16/java.xml] 8248348: Regression caused by the update to BCEL 6.0 In-Reply-To: <09c93e75-8055-fbb9-9cde-195f0f002f1a@oracle.com> References: <09c93e75-8055-fbb9-9cde-195f0f002f1a@oracle.com> Message-ID: Hello, What is the advantage of having such a narrow hashcode value space compared to the built in idendity hashcode? Would stocking to the object idendity not only reduce the footprint, but also make hash lookups faster? Without the unclear relationship to the op code? Gruss Bernd -- http://bernd.eckenfels.net ________________________________ Von: core-libs-dev im Auftrag von Joe Wang Gesendet: Friday, June 26, 2020 1:53:08 AM An: Core-Libs-Dev Betreff: RFR [16/java.xml] 8248348: Regression caused by the update to BCEL 6.0 Hi, Please review a fix to a BCEL regression. At issue was the addition of hashCode() method to Instruction.java in BCEL 6.0. This hashCode() method was implemented to return the instruction's opcode that unfortunately is mutable. The problem hasn't showed up until the code path led to calls to remove from a HashSet a target that has been changed since it was added to the HashSet. The proposed patch is to make the hashCode final/immutable. This patch implies that a target object is considered the same one even if its field values may have been changed. It therefore may not be appropriate in other situations (or may cause problems). However, since it had always had no hashCode() override before BCEL 6.0, thereby relying on Object's identity hash code, its use case has been limited and time-tested. It can benefit from this patch in that it provides the same function as Object's hash code, and then serves as a reminder (to any who might read the code) how it was used (objects considered to be the same over the course as far as the hashCode is concerned). JBS: https://bugs.openjdk.java.net/browse/JDK-8248348 webrevs: http://cr.openjdk.java.net/~joehw/jdk16/8248348/webrev/ Thanks, Joe From huizhe.wang at oracle.com Fri Jun 26 06:29:41 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Thu, 25 Jun 2020 23:29:41 -0700 Subject: RFR [16/java.xml] 8248348: Regression caused by the update to BCEL 6.0 In-Reply-To: References: <09c93e75-8055-fbb9-9cde-195f0f002f1a@oracle.com> Message-ID: <2b7e4775-8788-c8d5-2c41-022f836fd011@oracle.com> On 6/25/2020 5:14 PM, Bernd Eckenfels wrote: > Hello, > > What is the advantage of having such a narrow hashcode value space compared to the built in idendity hashcode? Would stocking to the object idendity not only reduce the footprint, but also make hash lookups faster? Without the unclear relationship to the op code? It's a general contract to override hashCode() when equals() is overridden, although due to the hacky nature the later was implemented and also the narrow scope it was applied in the (BCEL) code it didn't provide more than the Object identity. Still, it's right to keep the hashCode() method along with equals(), would have allowed all other subtypes to behave correctly, and beneficial to others who may read the code (who might wonder why it's missing). It's a bit improvement over no hashCode(). -Joe > > Gruss > Bernd > > > -- > http://bernd.eckenfels.net > ________________________________ > Von: core-libs-dev im Auftrag von Joe Wang > Gesendet: Friday, June 26, 2020 1:53:08 AM > An: Core-Libs-Dev > Betreff: RFR [16/java.xml] 8248348: Regression caused by the update to BCEL 6.0 > > Hi, > > Please review a fix to a BCEL regression. At issue was the addition of > hashCode() method to Instruction.java in BCEL 6.0. This hashCode() > method was implemented to return the instruction's opcode that > unfortunately is mutable. The problem hasn't showed up until the code > path led to calls to remove from a HashSet a target that has been > changed since it was added to the HashSet. The proposed patch is to make > the hashCode final/immutable. > > This patch implies that a target object is considered the same one even > if its field values may have been changed. It therefore may not be > appropriate in other situations (or may cause problems). However, since > it had always had no hashCode() override before BCEL 6.0, thereby > relying on Object's identity hash code, its use case has been limited > and time-tested. It can benefit from this patch in that it provides the > same function as Object's hash code, and then serves as a reminder (to > any who might read the code) how it was used (objects considered to be > the same over the course as far as the hashCode is concerned). > > JBS: https://bugs.openjdk.java.net/browse/JDK-8248348 > webrevs: http://cr.openjdk.java.net/~joehw/jdk16/8248348/webrev/ > > Thanks, > Joe > From Alan.Bateman at oracle.com Fri Jun 26 06:41:24 2020 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 26 Jun 2020 07:41:24 +0100 Subject: RFR(T): 8248358: ProblemList sun/nio/ch/TestMaxCachedBufferSize.java on macOSX In-Reply-To: <17c6bd4a-edb4-d261-9db7-29f2c3a7e537@oracle.com> References: <17c6bd4a-edb4-d261-9db7-29f2c3a7e537@oracle.com> Message-ID: <581837a2-a75b-9b43-3537-e5fec0d9ae15@oracle.com> On 25/06/2020 22:45, Daniel D. Daugherty wrote: > > > Here's the context diff: > > $ hg diff > diff -r cf65909b98c5 test/jdk/ProblemList.txt > --- a/test/jdk/ProblemList.txt??? Thu Jun 25 15:00:59 2020 -0400 > +++ b/test/jdk/ProblemList.txt??? Thu Jun 25 17:39:01 2020 -0400 > @@ -630,6 +630,8 @@ > > ?java/nio/channels/Selector/Wakeup.java 6963118 windows-all > > +sun/nio/ch/TestMaxCachedBufferSize.java 8212812 macosx-all > + > ?############################################################################ > > Would it be possible to start by increasing the timeout on this test first, rather than excluding it? I think it's important to know if it does actually complete in a reasonable time on the macOS machines where it times out?! -Alan From peter.levart at gmail.com Fri Jun 26 06:53:43 2020 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 26 Jun 2020 08:53:43 +0200 Subject: RFR [16/java.xml] 8248348: Regression caused by the update to BCEL 6.0 In-Reply-To: <09c93e75-8055-fbb9-9cde-195f0f002f1a@oracle.com> References: <09c93e75-8055-fbb9-9cde-195f0f002f1a@oracle.com> Message-ID: Hi Joe, For an object to correctly function as an element of a HashSet, its hashCode() and equals() methods must agree: o1.equals(o2) => o1.hashCode() == o2.hashCode(). I see equals() method delegates to a global InstructionComparator instance which can even change. Suppose it does not change. But what does such InstructionComparator compare? opcode (among other things)? Regards, Peter On Fri, 26 Jun 2020, 01:56 Joe Wang, wrote: > Hi, > > Please review a fix to a BCEL regression. At issue was the addition of > hashCode() method to Instruction.java in BCEL 6.0. This hashCode() > method was implemented to return the instruction's opcode that > unfortunately is mutable. The problem hasn't showed up until the code > path led to calls to remove from a HashSet a target that has been > changed since it was added to the HashSet. The proposed patch is to make > the hashCode final/immutable. > > This patch implies that a target object is considered the same one even > if its field values may have been changed. It therefore may not be > appropriate in other situations (or may cause problems). However, since > it had always had no hashCode() override before BCEL 6.0, thereby > relying on Object's identity hash code, its use case has been limited > and time-tested. It can benefit from this patch in that it provides the > same function as Object's hash code, and then serves as a reminder (to > any who might read the code) how it was used (objects considered to be > the same over the course as far as the hashCode is concerned). > > JBS: https://bugs.openjdk.java.net/browse/JDK-8248348 > webrevs: http://cr.openjdk.java.net/~joehw/jdk16/8248348/webrev/ > > Thanks, > Joe > > From chris.hegarty at oracle.com Fri Jun 26 08:57:48 2020 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 26 Jun 2020 09:57:48 +0100 Subject: RFR [1]6 8248326 Add a minimal serialization test for local records In-Reply-To: <9716b463-2b52-23bd-0762-5290c6f6ddeb@oracle.com> References: <61ED3E20-14D6-4CCE-8CB1-9F26A5CECB06@oracle.com> <9716b463-2b52-23bd-0762-5290c6f6ddeb@oracle.com> Message-ID: > On 25 Jun 2020, at 16:24, Daniel Fuchs wrote: > > Hi Chris, > > Looks good to me. Thanks for the review Daniel. > It might also be beneficial to double check > that the hash codes of the two objects are equal. Since no other tests in this area check the hash code, and this issue is narrowly focused on testing local records, then I?ll respectively decline the suggestion. If checking of the hash code is beneficial, we can consider it separately and in a larger context. -Chris. > best regards, > > -- daniel > > On 25/06/2020 16:01, Chris Hegarty wrote: >> While working on some record serialization changes recently, I noticed that we don?t have any coverage for local records. Here is a simple extension to an existing records test. >> diff -r 4e186efa6cbf test/jdk/java/io/Serializable/records/BasicRecordSer.java >> --- a/test/jdk/java/io/Serializable/records/BasicRecordSer.java Thu Jun 25 09:54:19 2020 +0100 >> +++ b/test/jdk/java/io/Serializable/records/BasicRecordSer.java Thu Jun 25 15:40:13 2020 +0100 >> @@ -132,6 +132,20 @@ >> assertEquals(objDeserialized, objToSerialize); >> } >> + /** Tests serializing and deserializing of local records. */ >> + @Test >> + public void testLocalRecord() throws Exception { >> + out.println("\n---"); >> + record Point(int x, int y) implements Serializable { } >> + record Rectangle(Point bottomLeft, Point topRight) implements Serializable { } >> + var objToSerialize = new Rectangle(new Point(0, 1), new Point (5, 6)); >> + out.println("serializing : " + objToSerialize); >> + var objDeserialized = serializeDeserialize(objToSerialize); >> + out.println("deserialized: " + objDeserialized); >> + assertEquals(objToSerialize, objDeserialized); >> + assertEquals(objDeserialized, objToSerialize); >> + } >> + >> /** Tests back references of Serializable record objects in the stream. */ >> @Test >> public void testSerializableBackRefs() throws Exception { >> -Chris. > From aleksei.voitylov at bell-sw.com Fri Jun 26 11:16:14 2020 From: aleksei.voitylov at bell-sw.com (Aleksei Voitylov) Date: Fri, 26 Jun 2020 14:16:14 +0300 Subject: RFR: JDK-8248239: jpackage adds some arguments twice in case it is re-executed by JLI In-Reply-To: <6d4fabe6-37ce-7d1d-bf7d-e2b07e0ba92b@oracle.com> References: <6d4fabe6-37ce-7d1d-bf7d-e2b07e0ba92b@oracle.com> Message-ID: Hi Alexey, Thank you for looking into it. As far as using parent pid, it does not seem to work as example [1] demonstrates. The parent process remains the same after re-execution and does not become the current process. I checked passing arguments in the "ArgOption" section using several cases [2, 3, 4] with the proposed fix and app re-execution. The proposed fix handles this case well, and the results are the same as without the fix when the app is not re-executed. Case [3] where only JavaOptions without ArgOptions are passed to app looks suspicious because default ArgOptions are not used. But it works the same way without the proposed fix, which seems like a different issue. According to jpackage documentation: ? --arguments main class arguments ??????? Command line arguments to pass to the main class if no command line arguments are given to the launcher. I filed a separate issue [5] to handle that. Thanks, -Aleksei [1] cd jdk-dev make jdk-image export PATH=build/linux-x86_64-server-release/images/jdk/bin:$PATH export LD_LIBRARY_PATH=build/linux-x86_64-server-release/images/jdk/lib/server jpackage --dest output --name app --type app-image --module-path input-modules --module com.hello/com.hello.Hello --arguments "A A2 A" --verbose --java-options -Dparam1=Param1 ./output/app/bin/app -Dparam2=Param2 B B2 B ------------- pid: 16007, current process: /home/sample/jdk-dev/output/app/bin/app pid: 15927, parent? process: /bin/bash JvmLauncher args: 10 [/home/sample/jdk-dev/output/app/bin/app -Dparam1=Param1 --module-path /home/sample/jdk-dev/output/app/lib/app/mods -m com.hello/com.hello.Hello -Dparam2=Param2 B B2 B ] pid: 16007, current process: /home/sample/jdk-dev/output/app/bin/app pid: 15927, parent? process: /bin/bash JvmLauncher args: 15 [/home/sample/jdk-dev/output/app/bin/app -Dparam1=Param1 --module-path /home/sample/jdk-dev/output/app/lib/app/mods -m com.hello/com.hello.Hello -Dparam1=Param1 --module-path /home/sample/jdk-dev/output/app/lib/app/mods -m com.hello/com.hello.Hello -Dparam2=Param2 B B2 B ] ------------- [2] jpackage --dest output --name app --type app-image --module-path input-modules --module com.hello/com.hello.Hello --arguments "A A2 A" --java-options -Dparam1=Param1 ./output/app/bin/app JvmLauncher args: 9 [output/app/bin/app -Dparam1=Param1 --module-path output/app/lib/app/mods -m com.hello/com.hello.Hello A A2 A ] JvmLauncher args: 9 [output/app/bin/app -Dparam1=Param1 --module-path output/app/lib/app/mods -m com.hello/com.hello.Hello A A2 A ] [3] jpackage --dest output --name app --type app-image --module-path input-modules --module com.hello/com.hello.Hello --arguments "A A2 A" --java-options -Dparam1=Param1 ./output/app/bin/app -Dparam2=Param2 JvmLauncher args: 7 [output/app/bin/app -Dparam1=Param1 --module-path output/app/lib/app/mods -m com.hello/com.hello.Hello -Dparam2=Param2 ] JvmLauncher args: 7 [output/app/bin/app -Dparam1=Param1 --module-path output/app/lib/app/mods -m com.hello/com.hello.Hello -Dparam2=Param2 ] [4] jpackage --dest output --name app --type app-image --module-path input-modules --module com.hello/com.hello.Hello --arguments "A A2 A" --java-options -Dparam1=Param1 ./output/app/bin/app -Dparam2=Param2 B B2 B JvmLauncher args: 10 [output/app/bin/app -Dparam1=Param1 --module-path output/app/lib/app/mods -m com.hello/com.hello.Hello -Dparam2=Param2 B B2 B ] JvmLauncher args: 10 [output/app/bin/app -Dparam1=Param1 --module-path output/app/lib/app/mods -m com.hello/com.hello.Hello -Dparam2=Param2 B B2 B ] [5] https://bugs.openjdk.java.net/browse/JDK-8248397 On 24/06/2020 19:34, Alexey Semenyuk wrote: > Aleksei, > > If I get it right, the fix would not work for the case when there are > `arguments` properties in `ArgOptions` section of a config file. > Why not just check if the parent process is the same executable as the > current one and if this is the case don't read data from the config > file but pass executable arguments as is in JLI_Launch() call? > > - Alexey > > On 6/24/2020 11:48 AM, Aleksei Voitylov wrote: >> Hi, >> >> There are systems that update LD_LIBRARY_PATH or directly return >> JNI_TRUE in method RequiresSetenv(const char *jvmpath) from java_md.c >> file to request re-execution of the current executable. This leads to >> the fact that jpackage application adds some provided arguments twice. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8248239 >> Fix: http://cr.openjdk.java.net/~avoitylov/webrev.8248239.00/ >> >> The root cause of the issue is that jpackage application expects one >> number of arguments but JLI reexecutes them with another number of >> arguments. >> ? For example, a jpackage application can be run with arguments: >> ???? ./app/bin/app -Dparam2=Param2 B1 B2 B3 >> it adds arguments from the config file and calls JLI with arguments: >> ???? app/bin/app -classpath? -Dparam1=Param1 -m >> com.hello/com.hello.Hello >> -Dparam2=Param2 B1 B2 B3 >> JLI re-executes the app with new arguments so the app adds some >> arguments one more time after the re-execution. >> ???? ./app/bin/app -classpath? -Dparam1=Param1 -m >> com.hello/com.hello.Hello -classpath? -Dparam1=Param1 -m >> com.hello/com.hello.Hello -Dparam2=Param2 B1 B2 B3 >> >> A step by step example that illustrates the issue: >> >> Run jpackage to create an executable application: >> ?? jpackage --dest output --name app --type app-image --module-path >> input-modules --module com.hello/com.hello.Hello --arguments "A1 A2 A3" >> --verbose --java-options -Dparam1=Param1 >> >> Executable application with the app/lib/app/app.cfg config file is >> created: >> ---- app.cfg? ---- >> [Application] >> app.name=app >> app.version=1.0 >> app.runtime=$ROOTDIR/lib/runtime >> app.identifier=com.hello >> app.classpath= >> app.mainmodule=com.hello/com.hello.Hello >> >> [JavaOptions] >> java-options=-Dparam1=Param1 >> >> [ArgOptions] >> arguments=A1 >> arguments=A2 >> arguments=A3 >> ----------- >> >> Run the application: >> ??? ./output/app/bin/app -Dparam2=Param2 B1 B2 B3 >> >> Chain of JDK methods execution: >> >> LinuxLauncher main() >> ??? args: 5 [./app/bin/app -Dparam2=Param2 B1 B2 B3 ] >> AppLauncher createJvmLauncher() >> ??? args: 4 [-Dparam2=Param2 B1 B2 B3 ] >> JvmLauncher.cpp Jvm::initFromConfigFile() - adds JavaOptions from >> app.cfg >> ??? args: 10 [app/bin/app -classpath? -Dparam1=Param1 -m >> com.hello/com.hello.Hello -Dparam2=Param2 B1 B2 B3 ] >> AppLauncher Jvm::launch() -? JLI re-executes the app >> LinuxLauncher main() >> ?? args: 10 [app/bin/app -classpath? -Dparam1=Param1 -m >> com.hello/com.hello.Hello -Dparam2=Param2 B1 B2 B3 ] >> AppLauncher createJvmLauncher() >> ??? args: 9 [-classpath? -Dparam1=Param1 -m com.hello/com.hello.Hello >> -Dparam2=Param2 B1 B2 B3 ] >> JvmLauncher.cpp Jvm::initFromConfigFile() - adds JavaOptions from >> app.cfg >> ??? args: 15 [./app/bin/app -classpath? -Dparam1=Param1 -m >> com.hello/com.hello.Hello -classpath? -Dparam1=Param1 -m >> com.hello/com.hello.Hello -Dparam2=Param2 B1 B2 B3 ] >> ???? ^^^ >> >> Some arguments like "-classpath? -Dparam1=Param1 -m >> com.hello/com.hello.Hello" are added twice. >> >> Tested with test/jdk/tools/jpackage/share/jdk/jpackage with no >> regressions on Linux, Windows, Mac. On systems affected, the tests >> now pass. >> >> Thanks, >> >> -Aleksei >> >> > From sergei.tsypanov at yandex.ru Fri Jun 26 12:06:35 2020 From: sergei.tsypanov at yandex.ru (=?utf-8?B?0KHQtdGA0LPQtdC5INCm0YvQv9Cw0L3QvtCy?=) Date: Fri, 26 Jun 2020 14:06:35 +0200 Subject: Scalar replacement issue in JDK 14.0.1 Message-ID: <426551593172556@mail.yandex.ru> Hello, while looking into an issue I've found out that scalar replacement is not working in trivial case on JDK 14.0.1. This benchmark illustrates the issue: @State(Scope.Thread) @BenchmarkMode(Mode.AverageTime) @OutputTimeUnit(TimeUnit.NANOSECONDS) @Fork(jvmArgsAppend = {"-Xms2g", "-Xmx2g"}) public class StringCompositeKeyBenchmark { @Benchmark public Object compositeKey(Data data) { return data.keyObjectMap.get(new Key(data.code, data.locale)); } @State(Scope.Thread) public static class Data { private final String code = "code1"; private final Locale locale = Locale.getDefault(); private final HashMap keyObjectMap = new HashMap<>(); @Setup public void setUp() { keyObjectMap.put(new Key(code, locale), new Object()); } } private static final class Key { private final String code; private final Locale locale; private Key(String code, Locale locale) { this.code = code; this.locale = locale; } @Override public boolean equals(Object o) { if (this == o) return true; if (o == null || getClass() != o.getClass()) return false; Key key = (Key) o; if (!code.equals(key.code)) return false; return locale.equals(key.locale); } @Override public int hashCode() { return 31 * code.hashCode() + locale.hashCode(); } } } When I run this on JDK 11 (JDK 11.0.7, OpenJDK 64-Bit Server VM, 11.0.7+10-post-Ubuntu-2ubuntu218.04) I get this output: Benchmark Mode Cnt Score Error Units StringCompositeKeyBenchmark.compositeKey avgt 10 5.510 ? 0.121 ns/op StringCompositeKeyBenchmark.compositeKey:?gc.alloc.rate avgt 10 ? 10?? MB/sec StringCompositeKeyBenchmark.compositeKey:?gc.alloc.rate.norm avgt 10 ? 10?? B/op StringCompositeKeyBenchmark.compositeKey:?gc.count avgt 10 ? 0 counts As I understand Java runtime erases object allocation here and we don't use additional memory. Same run on JDK 14 (JDK 14.0.1, Java HotSpot(TM) 64-Bit Server VM, 14.0.1+7) demonstrate object allocation per each method call: Benchmark Mode Cnt Score Error Units StringCompositeKeyBenchmark.compositeKey avgt 10 7.958 ? 1.360 ns/op StringCompositeKeyBenchmark.compositeKey:?gc.alloc.rate avgt 10 1937.551 ? 320.718 MB/sec StringCompositeKeyBenchmark.compositeKey:?gc.alloc.rate.norm avgt 10 24.001 ? 0.001 B/op StringCompositeKeyBenchmark.compositeKey:?gc.churn.G1_Eden_Space avgt 10 1879.111 ? 596.770 MB/sec StringCompositeKeyBenchmark.compositeKey:?gc.churn.G1_Eden_Space.norm avgt 10 23.244 ? 5.509 B/op StringCompositeKeyBenchmark.compositeKey:?gc.churn.G1_Survivor_Space avgt 10 0.267 ? 0.750 MB/sec StringCompositeKeyBenchmark.compositeKey:?gc.churn.G1_Survivor_Space.norm avgt 10 0.003 ? 0.009 B/op StringCompositeKeyBenchmark.compositeKey:?gc.count avgt 10 23.000 counts StringCompositeKeyBenchmark.compositeKey:?gc.time avgt 10 44.000 ms At the same time in more trivial scenario like @Benchmark public int compositeKey(Data data) { return new Key(data.code, data.locale).hashCode(); } scalar replacement again eliminates allocation of object. So I'm curious whether this is normal behaviour or a bug? Regards, Sergey Tsypanov From david.holmes at oracle.com Fri Jun 26 13:57:05 2020 From: david.holmes at oracle.com (David Holmes) Date: Fri, 26 Jun 2020 23:57:05 +1000 Subject: Scalar replacement issue in JDK 14.0.1 In-Reply-To: <426551593172556@mail.yandex.ru> References: <426551593172556@mail.yandex.ru> Message-ID: Hi Sergie, This seems like a question for hotspot-compiler-dev rather than core-libs-dev. Thanks, David On 26/06/2020 10:06 pm, ?????? ??????? wrote: > Hello, > > while looking into an issue I've found out that scalar replacement is not working in trivial case on JDK 14.0.1. > > This benchmark illustrates the issue: > > @State(Scope.Thread) > @BenchmarkMode(Mode.AverageTime) > @OutputTimeUnit(TimeUnit.NANOSECONDS) > @Fork(jvmArgsAppend = {"-Xms2g", "-Xmx2g"}) > public class StringCompositeKeyBenchmark { > @Benchmark > public Object compositeKey(Data data) { > return data.keyObjectMap.get(new Key(data.code, data.locale)); > } > > > @State(Scope.Thread) > public static class Data { > private final String code = "code1"; > private final Locale locale = Locale.getDefault(); > > private final HashMap keyObjectMap = new HashMap<>(); > > @Setup > public void setUp() { > keyObjectMap.put(new Key(code, locale), new Object()); > } > } > > private static final class Key { > private final String code; > private final Locale locale; > > private Key(String code, Locale locale) { > this.code = code; > this.locale = locale; > } > > @Override > public boolean equals(Object o) { > if (this == o) return true; > if (o == null || getClass() != o.getClass()) return false; > > Key key = (Key) o; > > if (!code.equals(key.code)) return false; > return locale.equals(key.locale); > } > > @Override > public int hashCode() { > return 31 * code.hashCode() + locale.hashCode(); > } > } > } > > When I run this on JDK 11 (JDK 11.0.7, OpenJDK 64-Bit Server VM, 11.0.7+10-post-Ubuntu-2ubuntu218.04) I get this output: > > Benchmark Mode Cnt Score Error Units > StringCompositeKeyBenchmark.compositeKey avgt 10 5.510 ? 0.121 ns/op > StringCompositeKeyBenchmark.compositeKey:?gc.alloc.rate avgt 10 ? 10?? MB/sec > StringCompositeKeyBenchmark.compositeKey:?gc.alloc.rate.norm avgt 10 ? 10?? B/op > StringCompositeKeyBenchmark.compositeKey:?gc.count avgt 10 ? 0 counts > > As I understand Java runtime erases object allocation here and we don't use additional memory. > > Same run on JDK 14 (JDK 14.0.1, Java HotSpot(TM) 64-Bit Server VM, 14.0.1+7) demonstrate object allocation per each method call: > > Benchmark Mode Cnt Score Error Units > StringCompositeKeyBenchmark.compositeKey avgt 10 7.958 ? 1.360 ns/op > StringCompositeKeyBenchmark.compositeKey:?gc.alloc.rate avgt 10 1937.551 ? 320.718 MB/sec > StringCompositeKeyBenchmark.compositeKey:?gc.alloc.rate.norm avgt 10 24.001 ? 0.001 B/op > StringCompositeKeyBenchmark.compositeKey:?gc.churn.G1_Eden_Space avgt 10 1879.111 ? 596.770 MB/sec > StringCompositeKeyBenchmark.compositeKey:?gc.churn.G1_Eden_Space.norm avgt 10 23.244 ? 5.509 B/op > StringCompositeKeyBenchmark.compositeKey:?gc.churn.G1_Survivor_Space avgt 10 0.267 ? 0.750 MB/sec > StringCompositeKeyBenchmark.compositeKey:?gc.churn.G1_Survivor_Space.norm avgt 10 0.003 ? 0.009 B/op > StringCompositeKeyBenchmark.compositeKey:?gc.count avgt 10 23.000 counts > StringCompositeKeyBenchmark.compositeKey:?gc.time avgt 10 44.000 ms > > At the same time in more trivial scenario like > > @Benchmark > public int compositeKey(Data data) { > return new Key(data.code, data.locale).hashCode(); > } > > scalar replacement again eliminates allocation of object. > > So I'm curious whether this is normal behaviour or a bug? > > Regards, > Sergey Tsypanov > From daniel.fuchs at oracle.com Fri Jun 26 14:13:08 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 26 Jun 2020 15:13:08 +0100 Subject: =?UTF-8?Q?Re=3a_RFR=5b8238286=5d=3a_=27Add_new_flatMap_stream_opera?= =?UTF-8?Q?tion_that_is_more_amenable_to_pushing=e2=80=99?= In-Reply-To: <1193990764.732096.1593123142068.JavaMail.zimbra@u-pem.fr> References: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> <366194096.16591.1593041533516.JavaMail.zimbra@u-pem.fr> <1818580489.728182.1593121568927.JavaMail.zimbra@u-pem.fr> <1193990764.732096.1593123142068.JavaMail.zimbra@u-pem.fr> Message-ID: Hi R?mi, On 25/06/2020 23:12, Remi Forax wrote: >> if i have already have a BiConsumer, Object>, i would like to >> be able to call Stream.mapMulti() with that bi-consumer as argument. > and obviously, i got it wrong, Consumer is not a super-type of Consumer, it should be a BiConsumer, Object> or a BiConsumer, Object>, etc. > Right - I had actually to code it to convince me that StreamTest mapMulti(BiConsumer, ? super T> mapper) (where R is String) would actually accept a BiConsumer, Object> or a BiConsumer, Object> or a BiConsumer, Object> However, if you try to implement such BiConsumers, and try to implement their accept method to call consumer.accept(o); then they fail to compile. Specifically: static BiConsumer, Object> c1 = new BiConsumer, Object>() { @Override public void accept(Consumer consumer, Object o) { consumer.accept(o); } }; static BiConsumer, Object> c2 = new BiConsumer, Object>() { @Override public void accept(Consumer consumer, Object o) { consumer.accept(o); } }; will not compile: StreamTest.java:12: error: incompatible types: Object cannot be converted to CAP#1 consumer.accept(o); ^ where CAP#1 is a fresh type-variable: CAP#1 extends Object from capture of ? /Users/danielfuchs/test/HttpRealTestAsync/src/StreamTest.java:20: error: incompatible types: Object cannot be converted to CAP#1 consumer.accept(o); ^ where CAP#1 is a fresh type-variable: CAP#1 extends Object super: String from capture of ? super String Which brings me back to my question: is there any value in having a BiConsumer that is accepted by mapMulti, where the signature of the BiConsumer doesn't allow it to act on its consumer argument without casting? best regards, -- daniel From forax at univ-mlv.fr Fri Jun 26 14:38:02 2020 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Fri, 26 Jun 2020 16:38:02 +0200 (CEST) Subject: =?utf-8?Q?Re:_RFR[8238286]:_'Add_new_flatMap_stream_?= =?utf-8?Q?operation_that_is_more_amenable_to_pushing=E2=80=99?= In-Reply-To: References: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> <366194096.16591.1593041533516.JavaMail.zimbra@u-pem.fr> <1818580489.728182.1593121568927.JavaMail.zimbra@u-pem.fr> <1193990764.732096.1593123142068.JavaMail.zimbra@u-pem.fr> Message-ID: <882560404.1290965.1593182282190.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Daniel Fuchs" > ?: "Remi Forax" > Cc: "Patrick Concannon" , "core-libs-dev" > Envoy?: Vendredi 26 Juin 2020 16:13:08 > Objet: Re: RFR[8238286]: 'Add new flatMap stream operation that is more amenable to pushing? > Hi R?mi, > > On 25/06/2020 23:12, Remi Forax wrote: >>> if i have already have a BiConsumer, Object>, i would like to >>> be able to call Stream.mapMulti() with that bi-consumer as argument. >> and obviously, i got it wrong, Consumer is not a super-type of >> Consumer, it should be a BiConsumer, Object> or a >> BiConsumer, Object>, etc. >> > > Right - I had actually to code it to convince me that > > StreamTest mapMulti(BiConsumer, ? super T> > mapper) > > (where R is String) would actually accept a > BiConsumer, Object> or a > BiConsumer, Object> or a BiConsumer Consumer, Object> > > However, if you try to implement such BiConsumers, and try to > implement their accept method to call consumer.accept(o); then > they fail to compile. Specifically: > > static BiConsumer, Object> c1 = > new BiConsumer, Object>() { > @Override > public void accept(Consumer consumer, Object o) { > consumer.accept(o); > } > }; > > static BiConsumer, Object> c2 = > new BiConsumer, Object>() { > @Override > public void accept(Consumer consumer, Object o) { > consumer.accept(o); > } > }; > > will not compile: > > StreamTest.java:12: error: incompatible types: Object cannot be > converted to CAP#1 > consumer.accept(o); > ^ > where CAP#1 is a fresh type-variable: > CAP#1 extends Object from capture of ? > /Users/danielfuchs/test/HttpRealTestAsync/src/StreamTest.java:20: error: > incompatible types: Object cannot be converted to CAP#1 > consumer.accept(o); > ^ > where CAP#1 is a fresh type-variable: > CAP#1 extends Object super: String from capture of ? super String > > Which brings me back to my question: is there any value in having a > BiConsumer that is accepted by mapMulti, where the signature of the > BiConsumer doesn't allow it to act on its consumer argument without > casting? for your first example, the only valid value for Consumer is null, so consumer.accept(null) should work. for the second example, Consumer accepts any subtypes of String, given that String is final, the only possible type is String, so consumer.accept(o.toString()) should work. BTW, you may think that nobody will create voluntarily a BiConsumer, Object>, and that's true, usually you get this kind of types as result of method call inference, e.g. you have a function g(f(Consumer)) with T=String and g signature being something like BiConsumer g(E element). > > best regards, > > -- daniel regards, R?mi From daniel.fuchs at oracle.com Fri Jun 26 15:13:05 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 26 Jun 2020 16:13:05 +0100 Subject: =?UTF-8?Q?Re=3a_RFR=5b8238286=5d=3a_=27Add_new_flatMap_stream_opera?= =?UTF-8?Q?tion_that_is_more_amenable_to_pushing=e2=80=99?= In-Reply-To: <882560404.1290965.1593182282190.JavaMail.zimbra@u-pem.fr> References: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> <366194096.16591.1593041533516.JavaMail.zimbra@u-pem.fr> <1818580489.728182.1593121568927.JavaMail.zimbra@u-pem.fr> <1193990764.732096.1593123142068.JavaMail.zimbra@u-pem.fr> <882560404.1290965.1593182282190.JavaMail.zimbra@u-pem.fr> Message-ID: <5db34ce2-e5b7-9021-b74d-2e2fbbf8dd5f@oracle.com> Hi R?mi! On 26/06/2020 15:38, forax at univ-mlv.fr wrote: > for your first example, the only valid value for Consumer is null, so consumer.accept(null) should work. > for the second example, Consumer accepts any subtypes of String, given that String is final, the only possible type is String, so consumer.accept(o.toString()) should work. > > BTW, you may think that nobody will create voluntarily a BiConsumer, Object>, and that's true, usually you get this kind of types as result of method call inference, > e.g. you have a function g(f(Consumer)) with T=String and g signature being something like BiConsumer g(E element). Thanks for the detailed explanations! OK - you have convinced me. I'll reluctantly withdraw my objection then ;-) best regards, -- daniel From lance.andersen at oracle.com Fri Jun 26 15:25:58 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 26 Jun 2020 11:25:58 -0400 Subject: RFR 8248412: test/jdk/java/sql/testng/test/sql/DriverManagerPermissionsTests.java can fail Message-ID: <9C066819-D819-4B88-990B-D35D788B0FAD@oracle.com> Hi all, Please review this patch for https://bugs.openjdk.java.net/browse/JDK-8248412 where DriverManagerPermissionsTests.java can occasionally fail in a mach5 run specifying -test open/test/jdk/:jdk_core,closed/test/jdk/:jdk_core The webrev can be found at: http://cr.openjdk.java.net/~lancea/8248412/webrev.00/ and change is trivial as the test will now run using othervm mode Best Lance Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From julia.boes at oracle.com Fri Jun 26 16:46:53 2020 From: julia.boes at oracle.com (Julia Boes) Date: Fri, 26 Jun 2020 17:46:53 +0100 Subject: =?UTF-8?Q?Re=3a_RFR=5b8238286=5d=3a_=27Add_new_flatMap_stream_opera?= =?UTF-8?Q?tion_that_is_more_amenable_to_pushing=e2=80=99?= In-Reply-To: <366194096.16591.1593041533516.JavaMail.zimbra@u-pem.fr> References: <928E8074-D519-4419-8879-4628979BB67F@oracle.com> <366194096.16591.1593041533516.JavaMail.zimbra@u-pem.fr> Message-ID: Hi, Thanks for the feedback so far. We'll get back on the suggested implementation and signature changes individually. Regarding the naming, Paul's suggestion mapAccept seems favorable. This being said, we'll pause the naming discussion for now and will resume it once all other questions have been resolved. > In *Stream.java, in the default method, the 'this' in 'this'.flatMap is useless. > > also it seems to me that the implementation of cancellationRequested() is useless given that Sink.Chained*::cancellationRequested already delegates to the downstream sink. @Remi: Thanks for flagging these, that's removed now. Updated webrev: http://cr.openjdk.java.net/~pconcannon/8238286/webrevs/webrev.01/ Regards, Julia From alexey.semenyuk at oracle.com Fri Jun 26 16:48:34 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Fri, 26 Jun 2020 12:48:34 -0400 Subject: RFR: JDK-8248264: WinUpgradeUUIDTest application is missing in downgrade scenario In-Reply-To: <01ebf9e4-9049-b33d-e395-798706828b74@oracle.com> References: <01ebf9e4-9049-b33d-e395-798706828b74@oracle.com> Message-ID: <9ef9304f-5ec2-0b90-4607-d5f7d5ecb02d@oracle.com> Please review fix [2] for jpackage bug [1]. Put `RemoveExistingProducts` action before `CostInitialize` action in `InstallExecuteSequence` sequence to uninstall existing product(s) before installer makes changes to the file system. Existing Wix `MajorUpgrade` elements in main.wxs implicitely add `RemoveExistingProducts` action to `InstallExecuteSequence` sequence but not in the required order. Thus all `MajorUpgrade` elements in main.wxs were replaced with explicitly defined upgrade table. - Alexey [1] https://bugs.openjdk.java.net/browse/JDK-8248264 [2]?http://cr.openjdk.java.net/~asemenyuk/8248264/webrev.00 From raffaello.giulietti at gmail.com Fri Jun 26 16:47:35 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Fri, 26 Jun 2020 18:47:35 +0200 Subject: Schubfach adoption + RFR CSR JDK-8202555: Double.toString(double) sometimes produces incorrect results In-Reply-To: <15fdab50-21d3-44b1-2d06-a8bc6ed7fa46@gmail.com> References: <15fdab50-21d3-44b1-2d06-a8bc6ed7fa46@gmail.com> Message-ID: Hello, beside the usual request to this community to review the Schubfach algorithm and its efficient implementation [1, 2], I'm pleased to announce that they have been adopted resp. adapted as part of jsoniter-scala [3, 4], a project aiming to be the fastest JSON parsing/serialization Scala library [5] out there. According to the library's author, Schubfach has replaced the Ry? algorithm used in earlier versions of the library because of its higher performance [6]. It runs quite efficiently even on JDK 8 without the Math::highMultiply intrinsic, which has been replaced by Karatsuba's multiplication. Greetings Raffaello ---- [1] https://drive.google.com/open?id=1luHhyQF9zKlM8yJ1nebU0OgVYhfC6CBN [2] http://cr.openjdk.java.net/~bpb/4511638/webrev.04/ [3] https://github.com/plokhotnyuk/jsoniter-scala [4] https://github.com/plokhotnyuk/jsoniter-scala/blob/96ed6d282d87d7844e3a9880da4c41a0518f0cc6/jsoniter-scala-core/jvm/src/main/scala/com/github/plokhotnyuk/jsoniter_scala/core/JsonWriter.scala#L1700-L2031 [5] https://plokhotnyuk.github.io/jsoniter-scala/ [6] https://twitter.com/aplokhotnyuk/status/1266792684464885762 From naoto.sato at oracle.com Fri Jun 26 16:56:41 2020 From: naoto.sato at oracle.com (naoto.sato at oracle.com) Date: Fri, 26 Jun 2020 09:56:41 -0700 Subject: RFR 8248412: test/jdk/java/sql/testng/test/sql/DriverManagerPermissionsTests.java can fail In-Reply-To: <9C066819-D819-4B88-990B-D35D788B0FAD@oracle.com> References: <9C066819-D819-4B88-990B-D35D788B0FAD@oracle.com> Message-ID: <16d628e5-56ba-e27f-23bb-1f5089100fbe@oracle.com> Hi Lance, The change looks good to me. Naoto On 6/26/20 8:25 AM, Lance Andersen wrote: > Hi all, > > Please review this patch for https://bugs.openjdk.java.net/browse/JDK-8248412 where DriverManagerPermissionsTests.java can occasionally fail in a mach5 run specifying -test open/test/jdk/:jdk_core,closed/test/jdk/:jdk_core > > The webrev can be found at: http://cr.openjdk.java.net/~lancea/8248412/webrev.00/ and change is trivial as the test will now run using othervm mode > > > Best > Lance > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From daniel.fuchs at oracle.com Fri Jun 26 17:15:12 2020 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 26 Jun 2020 18:15:12 +0100 Subject: RFR 8248412: test/jdk/java/sql/testng/test/sql/DriverManagerPermissionsTests.java can fail In-Reply-To: <9C066819-D819-4B88-990B-D35D788B0FAD@oracle.com> References: <9C066819-D819-4B88-990B-D35D788B0FAD@oracle.com> Message-ID: Looks good to me Lance. Thanks for fixing that! best regards, -- daniel On 26/06/2020 16:25, Lance Andersen wrote: > Hi all, > > Please review this patch for https://bugs.openjdk.java.net/browse/JDK-8248412 where DriverManagerPermissionsTests.java can occasionally fail in a mach5 run specifying -test open/test/jdk/:jdk_core,closed/test/jdk/:jdk_core > > The webrev can be found at: http://cr.openjdk.java.net/~lancea/8248412/webrev.00/ and change is trivial as the test will now run using othervm mode > > > Best > Lance > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From alexey.semenyuk at oracle.com Fri Jun 26 17:23:39 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Fri, 26 Jun 2020 13:23:39 -0400 Subject: RFR: JDK-8248239: jpackage adds some arguments twice in case it is re-executed by JLI In-Reply-To: References: <6d4fabe6-37ce-7d1d-bf7d-e2b07e0ba92b@oracle.com> Message-ID: Hi Aleksei, I think the idea of partial reading data from cfg file when the launcher is restarted has a flaw. It would be better if app launcher can detect if it was restarted and if it was, not read cfg file at all, but pass command line arguments as is in JLI_Launch(). Let my think on ideas how to address this. - Alexey On 6/26/2020 7:16 AM, Aleksei Voitylov wrote: > Hi Alexey, > > Thank you for looking into it. As far as using parent pid, it does not > seem to work as example [1] demonstrates. The parent process remains the > same after re-execution and does not become the current process. > > I checked passing arguments in the "ArgOption" section using several > cases [2, 3, 4] with the proposed fix and app re-execution. The proposed > fix handles this case well, and the results are the same as without the > fix when the app is not re-executed. > > Case [3] where only JavaOptions without ArgOptions are passed to app > looks suspicious because default ArgOptions are not used. But it works > the same way without the proposed fix, which seems like a different > issue. According to jpackage documentation: > > ? --arguments main class arguments > ??????? Command line arguments to pass to the main class if no command > line arguments are given to the launcher. > > I filed a separate issue [5] to handle that. > > Thanks, > -Aleksei > > > [1] > cd jdk-dev > make jdk-image > export PATH=build/linux-x86_64-server-release/images/jdk/bin:$PATH > export > LD_LIBRARY_PATH=build/linux-x86_64-server-release/images/jdk/lib/server > jpackage --dest output --name app --type app-image --module-path > input-modules --module com.hello/com.hello.Hello --arguments "A A2 A" > --verbose --java-options -Dparam1=Param1 > ./output/app/bin/app -Dparam2=Param2 B B2 B > ------------- > pid: 16007, current process: /home/sample/jdk-dev/output/app/bin/app > pid: 15927, parent? process: /bin/bash > JvmLauncher args: 10 [/home/sample/jdk-dev/output/app/bin/app > -Dparam1=Param1 --module-path > /home/sample/jdk-dev/output/app/lib/app/mods -m > com.hello/com.hello.Hello -Dparam2=Param2 B B2 B ] > pid: 16007, current process: /home/sample/jdk-dev/output/app/bin/app > pid: 15927, parent? process: /bin/bash > JvmLauncher args: 15 [/home/sample/jdk-dev/output/app/bin/app > -Dparam1=Param1 --module-path > /home/sample/jdk-dev/output/app/lib/app/mods -m > com.hello/com.hello.Hello -Dparam1=Param1 --module-path > /home/sample/jdk-dev/output/app/lib/app/mods -m > com.hello/com.hello.Hello -Dparam2=Param2 B B2 B ] > ------------- > > [2] > jpackage --dest output --name app --type app-image --module-path > input-modules --module com.hello/com.hello.Hello --arguments "A A2 A" > --java-options -Dparam1=Param1 > ./output/app/bin/app > JvmLauncher args: 9 [output/app/bin/app -Dparam1=Param1 --module-path > output/app/lib/app/mods -m com.hello/com.hello.Hello A A2 A ] > JvmLauncher args: 9 [output/app/bin/app -Dparam1=Param1 --module-path > output/app/lib/app/mods -m com.hello/com.hello.Hello A A2 A ] > > [3] > jpackage --dest output --name app --type app-image --module-path > input-modules --module com.hello/com.hello.Hello --arguments "A A2 A" > --java-options -Dparam1=Param1 > ./output/app/bin/app -Dparam2=Param2 > JvmLauncher args: 7 [output/app/bin/app -Dparam1=Param1 --module-path > output/app/lib/app/mods -m com.hello/com.hello.Hello -Dparam2=Param2 ] > JvmLauncher args: 7 [output/app/bin/app -Dparam1=Param1 --module-path > output/app/lib/app/mods -m com.hello/com.hello.Hello -Dparam2=Param2 ] > > [4] > jpackage --dest output --name app --type app-image --module-path > input-modules --module com.hello/com.hello.Hello --arguments "A A2 A" > --java-options -Dparam1=Param1 > ./output/app/bin/app -Dparam2=Param2 B B2 B > JvmLauncher args: 10 [output/app/bin/app -Dparam1=Param1 --module-path > output/app/lib/app/mods -m com.hello/com.hello.Hello -Dparam2=Param2 B > B2 B ] > JvmLauncher args: 10 [output/app/bin/app -Dparam1=Param1 --module-path > output/app/lib/app/mods -m com.hello/com.hello.Hello -Dparam2=Param2 B > B2 B ] > > [5] https://bugs.openjdk.java.net/browse/JDK-8248397 > > > On 24/06/2020 19:34, Alexey Semenyuk wrote: >> Aleksei, >> >> If I get it right, the fix would not work for the case when there are >> `arguments` properties in `ArgOptions` section of a config file. >> Why not just check if the parent process is the same executable as the >> current one and if this is the case don't read data from the config >> file but pass executable arguments as is in JLI_Launch() call? >> >> - Alexey >> >> On 6/24/2020 11:48 AM, Aleksei Voitylov wrote: >>> Hi, >>> >>> There are systems that update LD_LIBRARY_PATH or directly return >>> JNI_TRUE in method RequiresSetenv(const char *jvmpath) from java_md.c >>> file to request re-execution of the current executable. This leads to >>> the fact that jpackage application adds some provided arguments twice. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8248239 >>> Fix: http://cr.openjdk.java.net/~avoitylov/webrev.8248239.00/ >>> >>> The root cause of the issue is that jpackage application expects one >>> number of arguments but JLI reexecutes them with another number of >>> arguments. >>> ? For example, a jpackage application can be run with arguments: >>> ???? ./app/bin/app -Dparam2=Param2 B1 B2 B3 >>> it adds arguments from the config file and calls JLI with arguments: >>> ???? app/bin/app -classpath? -Dparam1=Param1 -m >>> com.hello/com.hello.Hello >>> -Dparam2=Param2 B1 B2 B3 >>> JLI re-executes the app with new arguments so the app adds some >>> arguments one more time after the re-execution. >>> ???? ./app/bin/app -classpath? -Dparam1=Param1 -m >>> com.hello/com.hello.Hello -classpath? -Dparam1=Param1 -m >>> com.hello/com.hello.Hello -Dparam2=Param2 B1 B2 B3 >>> >>> A step by step example that illustrates the issue: >>> >>> Run jpackage to create an executable application: >>> ?? jpackage --dest output --name app --type app-image --module-path >>> input-modules --module com.hello/com.hello.Hello --arguments "A1 A2 A3" >>> --verbose --java-options -Dparam1=Param1 >>> >>> Executable application with the app/lib/app/app.cfg config file is >>> created: >>> ---- app.cfg? ---- >>> [Application] >>> app.name=app >>> app.version=1.0 >>> app.runtime=$ROOTDIR/lib/runtime >>> app.identifier=com.hello >>> app.classpath= >>> app.mainmodule=com.hello/com.hello.Hello >>> >>> [JavaOptions] >>> java-options=-Dparam1=Param1 >>> >>> [ArgOptions] >>> arguments=A1 >>> arguments=A2 >>> arguments=A3 >>> ----------- >>> >>> Run the application: >>> ??? ./output/app/bin/app -Dparam2=Param2 B1 B2 B3 >>> >>> Chain of JDK methods execution: >>> >>> LinuxLauncher main() >>> ??? args: 5 [./app/bin/app -Dparam2=Param2 B1 B2 B3 ] >>> AppLauncher createJvmLauncher() >>> ??? args: 4 [-Dparam2=Param2 B1 B2 B3 ] >>> JvmLauncher.cpp Jvm::initFromConfigFile() - adds JavaOptions from >>> app.cfg >>> ??? args: 10 [app/bin/app -classpath? -Dparam1=Param1 -m >>> com.hello/com.hello.Hello -Dparam2=Param2 B1 B2 B3 ] >>> AppLauncher Jvm::launch() -? JLI re-executes the app >>> LinuxLauncher main() >>> ?? args: 10 [app/bin/app -classpath? -Dparam1=Param1 -m >>> com.hello/com.hello.Hello -Dparam2=Param2 B1 B2 B3 ] >>> AppLauncher createJvmLauncher() >>> ??? args: 9 [-classpath? -Dparam1=Param1 -m com.hello/com.hello.Hello >>> -Dparam2=Param2 B1 B2 B3 ] >>> JvmLauncher.cpp Jvm::initFromConfigFile() - adds JavaOptions from >>> app.cfg >>> ??? args: 15 [./app/bin/app -classpath? -Dparam1=Param1 -m >>> com.hello/com.hello.Hello -classpath? -Dparam1=Param1 -m >>> com.hello/com.hello.Hello -Dparam2=Param2 B1 B2 B3 ] >>> ???? ^^^ >>> >>> Some arguments like "-classpath? -Dparam1=Param1 -m >>> com.hello/com.hello.Hello" are added twice. >>> >>> Tested with test/jdk/tools/jpackage/share/jdk/jpackage with no >>> regressions on Linux, Windows, Mac. On systems affected, the tests >>> now pass. >>> >>> Thanks, >>> >>> -Aleksei >>> >>> From daniel.daugherty at oracle.com Fri Jun 26 17:29:44 2020 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 26 Jun 2020 13:29:44 -0400 Subject: RFR(T): 8248358: ProblemList sun/nio/ch/TestMaxCachedBufferSize.java on macOSX In-Reply-To: <581837a2-a75b-9b43-3537-e5fec0d9ae15@oracle.com> References: <17c6bd4a-edb4-d261-9db7-29f2c3a7e537@oracle.com> <581837a2-a75b-9b43-3537-e5fec0d9ae15@oracle.com> Message-ID: <6a37614c-d6ba-0820-c860-c9532e705567@oracle.com> On 6/26/20 2:41 AM, Alan Bateman wrote: > On 25/06/2020 22:45, Daniel D. Daugherty wrote: >> >> >> Here's the context diff: >> >> $ hg diff >> diff -r cf65909b98c5 test/jdk/ProblemList.txt >> --- a/test/jdk/ProblemList.txt??? Thu Jun 25 15:00:59 2020 -0400 >> +++ b/test/jdk/ProblemList.txt??? Thu Jun 25 17:39:01 2020 -0400 >> @@ -630,6 +630,8 @@ >> >> ?java/nio/channels/Selector/Wakeup.java 6963118 windows-all >> >> +sun/nio/ch/TestMaxCachedBufferSize.java 8212812 macosx-all >> + >> ?############################################################################ >> >> > Would it be possible to start by increasing the timeout on this test > first, rather than excluding it? I think it's important to know if it > does actually complete in a reasonable time on the macOS machines > where it times out?! Sorry Alan. This changeset was pushed to JDK15 yesterday and then synced from JDK15 -> JDK16 late last night (ET). This can be investigated by the person working on 8212812. Dan > > -Alan > From huizhe.wang at oracle.com Fri Jun 26 17:57:15 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Fri, 26 Jun 2020 10:57:15 -0700 Subject: RFR [16/java.xml] 8248348: Regression caused by the update to BCEL 6.0 In-Reply-To: References: <09c93e75-8055-fbb9-9cde-195f0f002f1a@oracle.com> Message-ID: <2d3f2c97-03cd-fa59-23ef-f39b662c7646@oracle.com> Hi Peter, Yes, they must agree: o1.equals(o2) => o1.hashCode() == o2.hashCode(). I think that was the original motivation in BCEL 6.0 to add the hashCode() method since before that, the Instruction class overrode equals() but not hashCode(), so two Instructions that were equal had different hashcode. The InstructionComparator compares opcode in general, and adds conditions for certain subtypes, e.g. BranchInstruction (BI) among others. What the InstructionComparator does was a complicated matter originated from how the whole process was designed. They had given it a lot of thoughts, replacing HashSet among other things for example. The current form was a compromise reached in BCEL 6.0 after a long discussion. One such compromise was that BIs were treated uniquely (unequal), allowing targeters to be added to a HashSet as unique items, and then be able to be removed by themselves (references). For the later to happen, the hashcode must remain the same for the same object. Regards, Joe On 6/25/2020 11:53 PM, Peter Levart wrote: > Hi Joe, > > For an object to correctly function as an element of a HashSet, its > hashCode() and equals() methods must agree: o1.equals(o2) => > o1.hashCode() == o2.hashCode(). I see equals() method delegates to a > global InstructionComparator instance which can even change. Suppose > it does not change. But what does such InstructionComparator compare? > opcode (among other things)? > > Regards, Peter > > > On Fri, 26 Jun 2020, 01:56 Joe Wang, > wrote: > > Hi, > > Please review a fix to a BCEL regression. At issue was the > addition of > hashCode() method to Instruction.java in BCEL 6.0. This hashCode() > method was implemented to return the instruction's opcode that > unfortunately is mutable. The problem hasn't showed up until the code > path led to calls to remove from a HashSet a target that has been > changed since it was added to the HashSet. The proposed patch is > to make > the hashCode final/immutable. > > This patch implies that a target object is considered the same one > even > if its field values may have been changed. It therefore may not be > appropriate in other situations (or may cause problems). However, > since > it had always had no hashCode() override before BCEL 6.0, thereby > relying on Object's identity hash code, its use case has been limited > and time-tested. It can benefit from this patch in that it > provides the > same function as Object's hash code, and then serves as a reminder > (to > any who might read the code) how it was used (objects considered > to be > the same over the course as far as the hashCode is concerned). > > JBS: https://bugs.openjdk.java.net/browse/JDK-8248348 > webrevs: http://cr.openjdk.java.net/~joehw/jdk16/8248348/webrev/ > > Thanks, > Joe > From andy.herrick at oracle.com Fri Jun 26 19:18:28 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Fri, 26 Jun 2020 15:18:28 -0400 Subject: RFR: JDK-8248264: WinUpgradeUUIDTest application is missing in downgrade scenario In-Reply-To: <9ef9304f-5ec2-0b90-4607-d5f7d5ecb02d@oracle.com> References: <01ebf9e4-9049-b33d-e395-798706828b74@oracle.com> <9ef9304f-5ec2-0b90-4607-d5f7d5ecb02d@oracle.com> Message-ID: <9241c902-4a84-150c-602a-b65524c6c0b2@oracle.com> looks good. /Andy On 6/26/2020 12:48 PM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > Put `RemoveExistingProducts` action before `CostInitialize` action in > `InstallExecuteSequence` sequence to uninstall existing product(s) > before installer makes changes to the file system. > Existing Wix `MajorUpgrade` elements in main.wxs implicitely add > `RemoveExistingProducts` action to `InstallExecuteSequence` sequence > but not in the required order. Thus all `MajorUpgrade` elements in > main.wxs were replaced with explicitly defined upgrade table. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8248264 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8248264/webrev.00 > From stuart.marks at oracle.com Fri Jun 26 19:46:01 2020 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 26 Jun 2020 12:46:01 -0700 Subject: RFR [16/java.xml] 8248348: Regression caused by the update to BCEL 6.0 In-Reply-To: <09c93e75-8055-fbb9-9cde-195f0f002f1a@oracle.com> References: <09c93e75-8055-fbb9-9cde-195f0f002f1a@oracle.com> Message-ID: <6154cdc1-1e7d-b8dd-4303-de1026e72e73@oracle.com> On 6/25/20 4:53 PM, Joe Wang wrote: > Please review a fix to a BCEL regression. At issue was the addition of > hashCode() method to Instruction.java in BCEL 6.0. This hashCode() method was > implemented to return the instruction's opcode that unfortunately is mutable. > The problem hasn't showed up until the code path led to calls to remove from a > HashSet a target that has been changed since it was added to the HashSet. The > proposed patch is to make the hashCode final/immutable. > > This patch implies that a target object is considered the same one even if its > field values may have been changed. It therefore may not be appropriate in other > situations (or may cause problems). However, since it had always had no > hashCode() override before BCEL 6.0, thereby relying on Object's identity hash > code, its use case has been limited and time-tested. It can benefit from this > patch in that it provides the same function as Object's hash code, and then > serves as a reminder (to any who might read the code) how it was used (objects > considered to be the same over the course as far as the hashCode is concerned). > > JBS: https://bugs.openjdk.java.net/browse/JDK-8248348 > webrevs: http://cr.openjdk.java.net/~joehw/jdk16/8248348/webrev/ Hi Joe, This fix seems vaguely wrong, but in the context of what I understand it's doing, it seems that BCEL or the surrounding JAXP code is even more wrong. :-) From what I understand, this bugfix was prompted by a case where an Instruction was added to a HashSet. The Instruction was then mutated while in the HashSet, and this resulted in it not being removed from the HashSet in the future, when it was necessary to do so, thus leaving a stale entry. How could this have possibly worked?!? I'm speculating a bit, but I could imagine code like this: var inst = new Instruction(ORIG_OPCODE); var set = new HashSet(); set.add(inst); ... inst.setOpcode(NEW_OPCODE); // (1) mutates inst while in HashSet! ... set.remove(inst); // (2) In the current version, the hashCode() is simply the opcode, and equals() compares opcodes (and possibly other things). This is correct from the standpoint of Instruction itself, as equal items have the same hashcode. However, the above code fails because at line (2), the remove() method looks in the "wrong" hash bucket and can't find the instruction, so it does nothing. The patch changes the class to compute the hashcode at construction time, and makes the hashcode constant, so it "fixes" the above code, since the remove() method will look in the same bucket and find the desired instance. But now Instruction breaks the hashCode/equals contract, since it's possible for two instances to be equal but to have different hashcodes. (Consider an Instruction that was created with NEW_OPCODE. It can be equal to inst, but its hashcode will differ.) It seems to me that this patch is compensating for broken code elsewhere in BCEL or JAXP by adding more brokenness. It could be that is ok in this narrow usage context. That is, as far as we know, this "shaded" copy of BCEL is used *only* by JAXP and not used in general, and it may be that the particular ways that JAXP uses BCEL aren't broken (and are indeed fixed) by this change. But I'd like to see some assurance of that. It could be that there are bugs introduced by this change that simply haven't been uncovered by the testing we've done yet. Another approach is to figure out what is mutating the Instruction while it's in a HashSet and fix that instead. Unfortunately, either of these approaches seems to involve an arbitrary amount of work. :-( s'marks From huizhe.wang at oracle.com Fri Jun 26 19:52:03 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Fri, 26 Jun 2020 12:52:03 -0700 Subject: RFR; [docs,15] JDK-8248060 small HTML issues in java.xml package-info.java files In-Reply-To: <0fc54dcb-a97d-3cda-b393-24dd4f463427@oracle.com> References: <0fc54dcb-a97d-3cda-b393-24dd4f463427@oracle.com> Message-ID: <90baa1e0-e584-c2df-0a04-00f64274928f@oracle.com> Looks good to me. The upstream SAX hasn't evolved for a long time (and there's no plan to do so). I think we're okay to make small changes. For the 2nd case, I think it's good enough since there are links to the classes at the beginning of the sentence -- a minor inconvenience (vs a straight link to the method). Alternatively, we could choose the more frequently used InputFactory, e.g. {@linkplain XMLInputFactory#newFactory newFactory}. Cheers, Joe On 6/22/2020 3:55 PM, Jonathan Gibbons wrote: > Please review a couple of trivial fixes for a couple of issues in > java.xml, reported by doclint. > > I realize we typically don't touch upstream code, but I'm hoping this > is small enough to > be reasonable. > > In the first case, there is an unnecessary `

      ` before a `

      `. > > In the second case, there's a bad `{@link newFactory}`. Not only is > the syntax bad, > but the method is defined in multiple classes in the same package, and > overloaded as > well: i.e. there is no easy/reasonable target for the link. The > simple/localized fix is > just to change `{@link}` to `{@code}`.? A better solution (for someone > else to do) > would be to rewrite the sentence altogether. > > -- Jon > > JBS: https://bugs.openjdk.java.net/browse/JDK-8248060 > > Patch: > > ?hg diff -R open open/src/java.xml > diff -r 9cfa0137612f > src/java.xml/share/classes/javax/xml/stream/package-info.java > --- a/src/java.xml/share/classes/javax/xml/stream/package-info.java > Mon Jun 22 13:37:41 2020 -0700 > +++ b/src/java.xml/share/classes/javax/xml/stream/package-info.java > Mon Jun 22 15:44:55 2020 -0700 > @@ -41,8 +41,8 @@ > ??*

      > ??* StAX supports plugability with {@link XMLInputFactory} and > ??* {@link XMLOutputFactory} that define how an implementation is > - * located through a process as described in the {@link newFactory} > - * method. > + * located through a process as described in the {@code newFactory} > + * methods. > ??* > ??* > ??* @since 1.6 > diff -r 9cfa0137612f > src/java.xml/share/classes/org/xml/sax/package-info.java > --- a/src/java.xml/share/classes/org/xml/sax/package-info.java Mon Jun > 22 13:37:41 2020 -0700 > +++ b/src/java.xml/share/classes/org/xml/sax/package-info.java Mon Jun > 22 15:44:55 2020 -0700 > @@ -27,7 +27,6 @@ > ??* Provides the interfaces for the Simple API for XML (SAX). Supports > both > ??* the SAX1 and SAX2 APIs. > ??* > - *

      > ??*

      SAX2 Standard Feature Flags

      > ??* > ??*

      > From alexey.semenyuk at oracle.com Fri Jun 26 21:02:43 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Fri, 26 Jun 2020 17:02:43 -0400 Subject: RFR: JDK-8248427: jpackage jtreg BasicTest.testTemp() test fails on Windows In-Reply-To: <9ef9304f-5ec2-0b90-4607-d5f7d5ecb02d@oracle.com> References: <9ef9304f-5ec2-0b90-4607-d5f7d5ecb02d@oracle.com> Message-ID: Please review fix [2] for jpackage bug [1]. Makes value of `JpIcon` wix variable absolute path. This fixes a regression introduced by the fix of [3] issue. - Alexey [1] https://bugs.openjdk.java.net/browse/JDK-8248427 [2]?http://cr.openjdk.java.net/~asemenyuk/8248427/webrev.00 [3] https://bugs.openjdk.java.net/browse/JDK-8246212 From andy.herrick at oracle.com Fri Jun 26 21:28:13 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Fri, 26 Jun 2020 17:28:13 -0400 Subject: RFR: JDK-8248427: jpackage jtreg BasicTest.testTemp() test fails on Windows In-Reply-To: References: <9ef9304f-5ec2-0b90-4607-d5f7d5ecb02d@oracle.com> Message-ID: looks yet /Andy On 6/26/2020 5:02 PM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > Makes value of `JpIcon` wix variable absolute path. This fixes a > regression introduced by the fix of [3] issue. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8248427 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8248427/webrev.00 > > [3] https://bugs.openjdk.java.net/browse/JDK-8246212 > From alexey.semenyuk at oracle.com Fri Jun 26 23:30:43 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Fri, 26 Jun 2020 19:30:43 -0400 Subject: RFR: JDK-8248254: jpackage fails if app module is in external runtime In-Reply-To: References: Message-ID: Please review fix [2] for jpackage bug [1]. Fix jpackage code to be able to locate app module if it is linked in external runtime. The suggested fix only verifies if app module exists in external runtime. It doesn't extract version and main class from modules in exetrnal runtime. To address this shortcoming [3] CR was filed. Corresponding jtreg created and added to problem list. - Alexey [1] https://bugs.openjdk.java.net/browse/JDK-8248254 [2]?http://cr.openjdk.java.net/~asemenyuk/8248254/webrev.00 [3] https://bugs.openjdk.java.net/browse/JDK-8248418 From alexander.matveev at oracle.com Fri Jun 26 23:35:13 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Fri, 26 Jun 2020 16:35:13 -0700 Subject: RFR: JDK-8248264: WinUpgradeUUIDTest application is missing in downgrade scenario In-Reply-To: <9ef9304f-5ec2-0b90-4607-d5f7d5ecb02d@oracle.com> References: <01ebf9e4-9049-b33d-e395-798706828b74@oracle.com> <9ef9304f-5ec2-0b90-4607-d5f7d5ecb02d@oracle.com> Message-ID: Hi Alexey, Looks good. Thanks, Alexander On 6/26/20 9:48 AM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > Put `RemoveExistingProducts` action before `CostInitialize` action in > `InstallExecuteSequence` sequence to uninstall existing product(s) > before installer makes changes to the file system. > Existing Wix `MajorUpgrade` elements in main.wxs implicitely add > `RemoveExistingProducts` action to `InstallExecuteSequence` sequence > but not in the required order. Thus all `MajorUpgrade` elements in > main.wxs were replaced with explicitly defined upgrade table. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8248264 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8248264/webrev.00 > From alexander.matveev at oracle.com Fri Jun 26 23:38:11 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Fri, 26 Jun 2020 16:38:11 -0700 Subject: RFR: JDK-8248427: jpackage jtreg BasicTest.testTemp() test fails on Windows In-Reply-To: References: <9ef9304f-5ec2-0b90-4607-d5f7d5ecb02d@oracle.com> Message-ID: <4171f0fc-576d-0202-4565-ee69fdece86a@oracle.com> Hi Alexey, Looks good. Thanks, Alexander On 6/26/20 2:02 PM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > Makes value of `JpIcon` wix variable absolute path. This fixes a > regression introduced by the fix of [3] issue. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8248427 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8248427/webrev.00 > > [3] https://bugs.openjdk.java.net/browse/JDK-8246212 > From alexander.matveev at oracle.com Fri Jun 26 23:56:14 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Fri, 26 Jun 2020 16:56:14 -0700 Subject: RFR: JDK-8248254: jpackage fails if app module is in external runtime In-Reply-To: References: Message-ID: <5c3cb86c-5af1-6ca6-0898-ad11c50dc8b4@oracle.com> Hi Alexey, Looks good. Thanks, Alexander On 6/26/20 4:30 PM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > Fix jpackage code to be able to locate app module if it is linked in > external runtime. The suggested fix only verifies if app module exists > in external runtime. It doesn't extract version and main class from > modules in exetrnal runtime. To address this shortcoming [3] CR was > filed. Corresponding jtreg created and added to problem list. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8248254 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8248254/webrev.00 > > [3] https://bugs.openjdk.java.net/browse/JDK-8248418 > > From alexander.matveev at oracle.com Sat Jun 27 01:22:10 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Fri, 26 Jun 2020 18:22:10 -0700 Subject: RFR: 8248059: [macos] EmptyFolderPackageTest.java failed "hdiutil: create failed - No child processes" Message-ID: <2610c5a5-d360-9bd1-cfc6-71b36e6caf72@oracle.com> Please review the jpackage fix for bug [1] at [2]. Added fallback for creating DMG if original approach fails. In original approach DMG will be created by providing app image to hdiutil. It was noticed that helper sub-processes run by hdiutil to copy app image sometimes crashed or failed during automated testing. Fallback approach will create empty DMG and copy files manually. It was observed that in this case hdiutil does not run sub-process that caused tests to fail, so hopefully this workaround will fix issue. Both cases produces same working DMG. Tests which failed due to this issue was removed from ProblemList. [1] https://bugs.openjdk.java.net/browse/JDK-8248059 [2] http://cr.openjdk.java.net/~almatvee/8248059/webrev.00/ Thanks, Alexander From alexey.semenyuk at oracle.com Sat Jun 27 03:16:33 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Fri, 26 Jun 2020 23:16:33 -0400 Subject: RFR: 8248059: [macos] EmptyFolderPackageTest.java failed "hdiutil: create failed - No child processes" In-Reply-To: <2610c5a5-d360-9bd1-cfc6-71b36e6caf72@oracle.com> References: <2610c5a5-d360-9bd1-cfc6-71b36e6caf72@oracle.com> Message-ID: <03307ab2-dacb-c379-a7dc-1b477e691df1@oracle.com> Hi Alexander, MacDmgBundler.java: --- ? 94???????? } catch (IOException | PackagerException ex) { ? 95???????????? Log.verbose(ex); ? 96???????????? throw new PackagerException(ex); ? 97???????? } --- There is no need to add IOUtils.getFilesSizeInFolder() method. --- new PathGroup(Map.of(new Object(), srcFolder.toPath())).sizeInBytes(); --- can be used to get the size of all regular files in `srcFolder` folder recursively. Besides there is resource leak in IOUtils.getFilesSizeInFolder() - close() should be called on the Stream object returned by Files.walk() call. - Alexey On 6/26/2020 9:22 PM, alexander.matveev at oracle.com wrote: > Please review the jpackage fix for bug [1] at [2]. > > Added fallback for creating DMG if original approach fails. In > original approach DMG will be created by providing app image to > hdiutil. It was noticed that helper sub-processes run by hdiutil to > copy app image sometimes crashed or failed during automated testing. > Fallback approach will create empty DMG and copy files manually. It > was observed that in this case hdiutil does not run sub-process that > caused tests to fail, so hopefully this workaround will fix issue. > Both cases produces same working DMG. Tests which failed due to this > issue was removed from ProblemList. > > [1] https://bugs.openjdk.java.net/browse/JDK-8248059 > [2] http://cr.openjdk.java.net/~almatvee/8248059/webrev.00/ > > Thanks, > Alexander From alexander.matveev at oracle.com Sat Jun 27 04:00:37 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Fri, 26 Jun 2020 21:00:37 -0700 Subject: RFR: 8248059: [macos] EmptyFolderPackageTest.java failed "hdiutil: create failed - No child processes" In-Reply-To: <03307ab2-dacb-c379-a7dc-1b477e691df1@oracle.com> References: <2610c5a5-d360-9bd1-cfc6-71b36e6caf72@oracle.com> <03307ab2-dacb-c379-a7dc-1b477e691df1@oracle.com> Message-ID: Hi Alexey, http://cr.openjdk.java.net/~almatvee/8248059/webrev.01/ - Removed getFilesSizeInFolder() method and size will be calculated as per your suggestion. Thanks, Alexander On 6/26/20 8:16 PM, Alexey Semenyuk wrote: > Hi Alexander, > > MacDmgBundler.java: > --- > ? 94???????? } catch (IOException | PackagerException ex) { > ? 95???????????? Log.verbose(ex); > ? 96???????????? throw new PackagerException(ex); > ? 97???????? } > --- > > There is no need to add IOUtils.getFilesSizeInFolder() method. > > --- > new PathGroup(Map.of(new Object(), srcFolder.toPath())).sizeInBytes(); > --- > can be used to get the size of all regular files in `srcFolder` folder > recursively. Besides there is resource leak in > IOUtils.getFilesSizeInFolder() - close() should be called on the > Stream object returned by Files.walk() call. > > - Alexey > > On 6/26/2020 9:22 PM, alexander.matveev at oracle.com wrote: >> Please review the jpackage fix for bug [1] at [2]. >> >> Added fallback for creating DMG if original approach fails. In >> original approach DMG will be created by providing app image to >> hdiutil. It was noticed that helper sub-processes run by hdiutil to >> copy app image sometimes crashed or failed during automated testing. >> Fallback approach will create empty DMG and copy files manually. It >> was observed that in this case hdiutil does not run sub-process that >> caused tests to fail, so hopefully this workaround will fix issue. >> Both cases produces same working DMG. Tests which failed due to this >> issue was removed from ProblemList. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8248059 >> [2] http://cr.openjdk.java.net/~almatvee/8248059/webrev.00/ >> >> Thanks, >> Alexander > From ecki at zusammenkunft.net Sat Jun 27 11:02:47 2020 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Sat, 27 Jun 2020 11:02:47 +0000 Subject: RFR [16/java.xml] 8248348: Regression caused by the update to BCEL 6.0 In-Reply-To: <2b7e4775-8788-c8d5-2c41-022f836fd011@oracle.com> References: <09c93e75-8055-fbb9-9cde-195f0f002f1a@oracle.com> , <2b7e4775-8788-c8d5-2c41-022f836fd011@oracle.com> Message-ID: It does not sound aligned if you allow to change the op code which is compared in equal() but remember the hashcode. But I guess it would be best to use a IdendityHashMap if you care for the objects. In that case you can implement hashcode uncached and aligned with equals. (And not use it) Gruss Bernd -- http://bernd.eckenfels.net ________________________________ Von: Joe Wang Gesendet: Friday, June 26, 2020 8:29:41 AM An: Bernd Eckenfels ; Core-Libs-Dev Betreff: Re: RFR [16/java.xml] 8248348: Regression caused by the update to BCEL 6.0 On 6/25/2020 5:14 PM, Bernd Eckenfels wrote: > Hello, > > What is the advantage of having such a narrow hashcode value space compared to the built in idendity hashcode? Would stocking to the object idendity not only reduce the footprint, but also make hash lookups faster? Without the unclear relationship to the op code? It's a general contract to override hashCode() when equals() is overridden, although due to the hacky nature the later was implemented and also the narrow scope it was applied in the (BCEL) code it didn't provide more than the Object identity. Still, it's right to keep the hashCode() method along with equals(), would have allowed all other subtypes to behave correctly, and beneficial to others who may read the code (who might wonder why it's missing). It's a bit improvement over no hashCode(). -Joe > > Gruss > Bernd > > > -- > http://bernd.eckenfels.net > ________________________________ > Von: core-libs-dev im Auftrag von Joe Wang > Gesendet: Friday, June 26, 2020 1:53:08 AM > An: Core-Libs-Dev > Betreff: RFR [16/java.xml] 8248348: Regression caused by the update to BCEL 6.0 > > Hi, > > Please review a fix to a BCEL regression. At issue was the addition of > hashCode() method to Instruction.java in BCEL 6.0. This hashCode() > method was implemented to return the instruction's opcode that > unfortunately is mutable. The problem hasn't showed up until the code > path led to calls to remove from a HashSet a target that has been > changed since it was added to the HashSet. The proposed patch is to make > the hashCode final/immutable. > > This patch implies that a target object is considered the same one even > if its field values may have been changed. It therefore may not be > appropriate in other situations (or may cause problems). However, since > it had always had no hashCode() override before BCEL 6.0, thereby > relying on Object's identity hash code, its use case has been limited > and time-tested. It can benefit from this patch in that it provides the > same function as Object's hash code, and then serves as a reminder (to > any who might read the code) how it was used (objects considered to be > the same over the course as far as the hashCode is concerned). > > JBS: https://bugs.openjdk.java.net/browse/JDK-8248348 > webrevs: http://cr.openjdk.java.net/~joehw/jdk16/8248348/webrev/ > > Thanks, > Joe > From forax at univ-mlv.fr Sat Jun 27 12:41:26 2020 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 27 Jun 2020 14:41:26 +0200 (CEST) Subject: Explicitly constructed NPE is showing (wrong) message In-Reply-To: References: Message-ID: <1600616667.1579890.1593261686357.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Christoph Dreis" > ?: "hotspot-runtime-dev" > Envoy?: Samedi 27 Juin 2020 11:54:19 > Objet: Explicitly constructed NPE is showing (wrong) message > Hi, Hi Christoph, > > I hope this is the correct mailing list. > > The latest changes on JDK-8233014 to enable ShowCodeDetailsInExceptionMessages > by default have uncovered a likely bug (I think). > > To my understanding explicitly created NullPointerExceptions should not print > any message. > With the following example: > > public class Main { > public static void main(String[] args) { > NullPointerException ex = new NullPointerException(); > Throwable throwable = ex.fillInStackTrace(); > System.out.println(throwable); > } > } > > I see the following output though: > > java.lang.NullPointerException: Cannot invoke > "java.lang.NullPointerException.fillInStackTrace()" (on slot 0) because "ex" is > null > > Which obviously is not really true. > I was debugging the thing (hence the additional "on slot 0") output, but > couldn't find the error so far. > > I'm suspecting the error to be somewhere around bytecodeUtils.cpp 1155: > > if (name != vmSymbols::object_initializer_name()) { > int type_index = cp->signature_ref_index_at(name_and_type_index); > Symbol* signature = cp->symbol_at(type_index); > // The 'this' parameter was null. Return the slot of it. > return ArgumentSizeComputer(signature).size(); > } else { > return NPE_EXPLICIT_CONSTRUCTED; > } > > I initially hoped to find a fix for it directly, but I would probably need a bit > more time for it, so I decided to report it. > Is this a bug even or am I chasing ghosts here? In case it is, I would be happy > about a mentioning somewhere in an eventual commit. I see the issue. The idea of the algorithm is to use the backtrace stored in the exception to try to find where the NPE occurs, here by calling fillStackTrace() emplicitly, you are changing the backtrace so getExtendedNPEMessage now use the new backtrace reporting the error in main() :( One way to fix the issue is to record if fillInStackTrace is called more than once (by having two sentinels by example) and to not call getExtendedNPEMessage if it's the case. Obviously the new added sentinel has to work with the serialization. > > Cheers, > Christoph regards, R?mi From goetz.lindenmaier at sap.com Sat Jun 27 13:16:03 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Sat, 27 Jun 2020 13:16:03 +0000 Subject: Explicitly constructed NPE is showing (wrong) message In-Reply-To: <1600616667.1579890.1593261686357.JavaMail.zimbra@u-pem.fr> References: , <1600616667.1579890.1593261686357.JavaMail.zimbra@u-pem.fr> Message-ID: <7E0BFB0B-830E-4572-95ED-3460F4198C96@sap.com> The message from this sender included one or more files which could not be scanned for virus detection; do not open these files unless you are certain of the sender's intent. ---------------------------------------------------------------------- Hi, I?ll have a look on Monday. Thanks for reporting any analyzing it so far! Best regards, G?tz > Am 27.06.2020 um 14:44 schrieb Remi Forax : > > ?----- Mail original ----- >> De: "Christoph Dreis" >> ?: "hotspot-runtime-dev" >> Envoy?: Samedi 27 Juin 2020 11:54:19 >> Objet: Explicitly constructed NPE is showing (wrong) message > >> Hi, > > Hi Christoph, > >> >> I hope this is the correct mailing list. >> >> The latest changes on JDK-8233014 to enable ShowCodeDetailsInExceptionMessages >> by default have uncovered a likely bug (I think). >> >> To my understanding explicitly created NullPointerExceptions should not print >> any message. >> With the following example: >> >> public class Main { >> public static void main(String[] args) { >> NullPointerException ex = new NullPointerException(); >> Throwable throwable = ex.fillInStackTrace(); >> System.out.println(throwable); >> } >> } >> >> I see the following output though: >> >> java.lang.NullPointerException: Cannot invoke >> "java.lang.NullPointerException.fillInStackTrace()" (on slot 0) because "ex" is >> null >> >> Which obviously is not really true. >> I was debugging the thing (hence the additional "on slot 0") output, but >> couldn't find the error so far. >> >> I'm suspecting the error to be somewhere around bytecodeUtils.cpp 1155: >> >> if (name != vmSymbols::object_initializer_name()) { >> int type_index = cp->signature_ref_index_at(name_and_type_index); >> Symbol* signature = cp->symbol_at(type_index); >> // The 'this' parameter was null. Return the slot of it. >> return ArgumentSizeComputer(signature).size(); >> } else { >> return NPE_EXPLICIT_CONSTRUCTED; >> } >> >> I initially hoped to find a fix for it directly, but I would probably need a bit >> more time for it, so I decided to report it. >> Is this a bug even or am I chasing ghosts here? In case it is, I would be happy >> about a mentioning somewhere in an eventual commit. > > I see the issue. > The idea of the algorithm is to use the backtrace stored in the exception to try to find where the NPE occurs, here by calling fillStackTrace() emplicitly, you are changing the backtrace so getExtendedNPEMessage now use the new backtrace reporting the error in main() :( > > One way to fix the issue is to record if fillInStackTrace is called more than once (by having two sentinels by example) and to not call getExtendedNPEMessage if it's the case. Obviously the new added sentinel has to work with the serialization. > >> >> Cheers, >> Christoph > > regards, > R?mi From andy.herrick at oracle.com Sat Jun 27 17:24:39 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Sat, 27 Jun 2020 13:24:39 -0400 Subject: RFR: JDK-8248254: jpackage fails if app module is in external runtime In-Reply-To: References: Message-ID: <92572555-2fe8-8870-c79c-f151eb497daa@oracle.com> Looks good. /Andy On 6/26/2020 7:30 PM, Alexey Semenyuk wrote: > Please review fix [2] for jpackage bug [1]. > > Fix jpackage code to be able to locate app module if it is linked in > external runtime. The suggested fix only verifies if app module exists > in external runtime. It doesn't extract version and main class from > modules in exetrnal runtime. To address this shortcoming [3] CR was > filed. Corresponding jtreg created and added to problem list. > > - Alexey > > [1] https://bugs.openjdk.java.net/browse/JDK-8248254 > > [2]?http://cr.openjdk.java.net/~asemenyuk/8248254/webrev.00 > > [3] https://bugs.openjdk.java.net/browse/JDK-8248418 > > From andy.herrick at oracle.com Sat Jun 27 17:29:56 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Sat, 27 Jun 2020 13:29:56 -0400 Subject: RFR: 8248059: [macos] EmptyFolderPackageTest.java failed "hdiutil: create failed - No child processes" In-Reply-To: <2610c5a5-d360-9bd1-cfc6-71b36e6caf72@oracle.com> References: <2610c5a5-d360-9bd1-cfc6-71b36e6caf72@oracle.com> Message-ID: <6c6afd66-2cef-2d1b-2136-9ba3b32c5a33@oracle.com> looks good /Andy On 6/26/2020 9:22 PM, alexander.matveev at oracle.com wrote: > Please review the jpackage fix for bug [1] at [2]. > > Added fallback for creating DMG if original approach fails. In > original approach DMG will be created by providing app image to > hdiutil. It was noticed that helper sub-processes run by hdiutil to > copy app image sometimes crashed or failed during automated testing. > Fallback approach will create empty DMG and copy files manually. It > was observed that in this case hdiutil does not run sub-process that > caused tests to fail, so hopefully this workaround will fix issue. > Both cases produces same working DMG. Tests which failed due to this > issue was removed from ProblemList. > > [1] https://bugs.openjdk.java.net/browse/JDK-8248059 > [2] http://cr.openjdk.java.net/~almatvee/8248059/webrev.00/ > > Thanks, > Alexander From huizhe.wang at oracle.com Sat Jun 27 23:48:30 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Sat, 27 Jun 2020 16:48:30 -0700 Subject: RFR [16/java.xml] 8248348: Regression caused by the update to BCEL 6.0 In-Reply-To: <6154cdc1-1e7d-b8dd-4303-de1026e72e73@oracle.com> References: <09c93e75-8055-fbb9-9cde-195f0f002f1a@oracle.com> <6154cdc1-1e7d-b8dd-4303-de1026e72e73@oracle.com> Message-ID: On 6/26/2020 12:46 PM, Stuart Marks wrote: > > > On 6/25/20 4:53 PM, Joe Wang wrote: >> Please review a fix to a BCEL regression. At issue was the addition >> of hashCode() method to Instruction.java in BCEL 6.0. This hashCode() >> method was implemented to return the instruction's opcode that >> unfortunately is mutable. The problem hasn't showed up until the code >> path led to calls to remove from a HashSet a target that has been >> changed since it was added to the HashSet. The proposed patch is to >> make the hashCode final/immutable. >> >> This patch implies that a target object is considered the same one >> even if its field values may have been changed. It therefore may not >> be appropriate in other situations (or may cause problems). However, >> since it had always had no hashCode() override before BCEL 6.0, >> thereby relying on Object's identity hash code, its use case has been >> limited and time-tested. It can benefit from this patch in that it >> provides the same function as Object's hash code, and then serves as >> a reminder (to any who might read the code) how it was used (objects >> considered to be the same over the course as far as the hashCode is >> concerned). >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8248348 >> webrevs: http://cr.openjdk.java.net/~joehw/jdk16/8248348/webrev/ > > Hi Joe, > > This fix seems vaguely wrong, but in the context of what I understand > it's doing, it seems that BCEL or the surrounding JAXP code is even > more wrong. :-) The patch is surely unusual, because of this unusual case. As far as which one is more wrong, I believe the burden is mostly on BCEL although JAXP might share a bit, at the least it exposed the issue. > > From what I understand, this bugfix was prompted by a case where an > Instruction was added to a HashSet. The Instruction was then mutated > while in the HashSet, and this resulted in it not being removed from > the HashSet in the future, when it was necessary to do so, thus > leaving a stale entry. How could this have possibly worked?!? It works in most majority of the cases, although I think the BCEL developers admitted they had limited tests. This is because while Instruction allows mutation, all but one type of instructions actually do,? and the mutated one don't get removed from the HashSet in a normal build-up process. But XML applications can go berserk sometimes, and in this case resulted in a method exceeding the max method size. When that happens, a rebuild process involved many copies/clones, resetting targeters, the later included removing targeters (among them the mutated ones) from the HashSet. Given that such a big stylesheet is unusual, it makes it an edge case. At least, we haven't received any error reports from the 20% who adopted Java 11. > > I'm speculating a bit, but I could imagine code like this: > > ??? var inst = new Instruction(ORIG_OPCODE); > ??? var set = new HashSet(); > ??? set.add(inst); > ??? ... > ??? inst.setOpcode(NEW_OPCODE); // (1) mutates inst while in HashSet! > ??? ... > ??? set.remove(inst); // (2) > > In the current version, the hashCode() is simply the opcode, and > equals() compares opcodes (and possibly other things). This is correct > from the standpoint of Instruction itself, as equal items have the > same hashcode. However, the above code fails because at line (2), the > remove() method looks in the "wrong" hash bucket and can't find the > instruction, so it does nothing. Yes, it will be correct for the majority cases where opcodes don't change. Also note that HashSet checks hashcode, and then the key's identity before key.equals, which is why this patch works. When the build process calls on an instruction to dispose itself, it removes itself from the targeter HashSet. But while it is itself, if the hashcode has changed, it won't even find itself!? (sort of lost itself) (that's too many "itself") > > The patch changes the class to compute the hashcode at construction > time, and makes the hashcode constant, so it "fixes" the above code, > since the remove() method will look in the same bucket and find the > desired instance. But now Instruction breaks the hashCode/equals > contract, since it's possible for two instances to be equal but to > have different hashcodes. (Consider an Instruction that was created > with NEW_OPCODE. It can be equal to inst, but its hashcode will differ.) It would break the hashCode/equals contract if it was implemented normally. But this is unusual, the equals implementation was nothing but normal: for branch instructions, it always returns false!? Thus, this unusual hashCode implementation will not break the contract. > > It seems to me that this patch is compensating for broken code > elsewhere in BCEL or JAXP by adding more brokenness. It could be that > is ok in this narrow usage context. That is, as far as we know, this > "shaded" copy of BCEL is used *only* by JAXP and not used in general, > and it may be that the particular ways that JAXP uses BCEL aren't > broken (and are indeed fixed) by this change. But I'd like to see some > assurance of that. It could be that there are bugs introduced by this > change that simply haven't been uncovered by the testing we've done yet. I mentioned above, I believe the burden was mostly on BCEL. JAXP/Xalan might have complicated things, the way it rebuilds the instruction list when it encounters an over-the-limit method, or it could have been smarter to avoid building such a large list. But it was all done through BCEL's public APIs. Nothing JAXP/Xalan has done was not within the functions BCEL provided. In BCEL, not only the equals were implemented with some special treatment for certain subtypes, the choose of HashSet to store mutable (instruction) objects may arguably not have been the best. > > Another approach is to figure out what is mutating the Instruction > while it's in a HashSet and fix that instead. Re-setting Instruction target triggered it. But unfortunately that's necessary as far as the current process goes. > > Unfortunately, either of these approaches seems to involve an > arbitrary amount of work. :-( Very true, and that's why I provided this shaky yet working-for-the-hacky-code patch, it's a patch on patch rather than something one could do properly when designing a new code. That's not to say the BCEL engineers hadn't put in a great effort. They've worked on it for about half a year related to this hashCode/Equals issue in BCEL 6.0. They also deliberated replacing HashSet. But I guess eventually reality sets in, it would have been a major architectural work. All that's said, while it has its (maybe major) problems as this issue revealed, BCEL has been a stable component for JAXP/java.xml. This issue introduced by the changes in BCEL 6.0, I believe, is an edge case as explained previously. If the instruction list is built straight-forwardly, it would not have run into this issue. This patch resolves the case, restores the delicate balance and hopefully keep BCEL as stable as before. -Joe > > s'marks From huizhe.wang at oracle.com Sun Jun 28 00:21:45 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Sat, 27 Jun 2020 17:21:45 -0700 Subject: RFR [16/java.xml] 8248348: Regression caused by the update to BCEL 6.0 In-Reply-To: References: <09c93e75-8055-fbb9-9cde-195f0f002f1a@oracle.com> <2b7e4775-8788-c8d5-2c41-022f836fd011@oracle.com> Message-ID: True, It's unusual (refer to some more details in the previous email). It's an unusual patch over an unusual design. IdentifyHashMap might have been a good choice for the mutable types in this issue. I'm not sure about other types as the BCEL process does involve removing items from the HashSet with a cloned copy. Using HashSet and maybe the larger architectural design surrounding it might not have been the best. But that's up to the BCEL engineers to decide. For the JDK, BCEL has been as stable as it is. I believe only small (none-API/archtectural) patches will be considered for the JDK to function properly. Regards, Joe On 6/27/2020 4:02 AM, Bernd Eckenfels wrote: > It does not sound aligned if you allow to change the op code which is > compared in equal() but remember the hashcode. > > But I guess it would be best to use a IdendityHashMap if you care for > the objects. In that case you can implement hashcode uncached and > aligned with equals. (And not use it) > > Gruss > Bernd > -- > http://bernd.eckenfels.net > ------------------------------------------------------------------------ > *Von:* Joe Wang > *Gesendet:* Friday, June 26, 2020 8:29:41 AM > *An:* Bernd Eckenfels ; Core-Libs-Dev > > *Betreff:* Re: RFR [16/java.xml] 8248348: Regression caused by the > update to BCEL 6.0 > > > On 6/25/2020 5:14 PM, Bernd Eckenfels wrote: > > Hello, > > > > What is the advantage of having such a narrow hashcode value space > compared to the built in idendity hashcode? Would stocking to the > object idendity not only reduce the footprint, but also make hash > lookups faster? Without the unclear relationship to the op code? > > It's a general contract to override hashCode() when equals() is > overridden, although due to the hacky nature the later was implemented > and also the narrow scope it was applied in the (BCEL) code it didn't > provide more than the Object identity. Still, it's right to keep the > hashCode() method along with equals(), would have allowed all other > subtypes to behave correctly, and beneficial to others who may read the > code (who might wonder why it's missing). It's a bit improvement over no > hashCode(). > > -Joe > > > > Gruss > > Bernd > > > > > > -- > > http://bernd.eckenfels.net > > ________________________________ > > Von: core-libs-dev im Auftrag > von Joe Wang > > Gesendet: Friday, June 26, 2020 1:53:08 AM > > An: Core-Libs-Dev > > Betreff: RFR [16/java.xml] 8248348: Regression caused by the update > to BCEL 6.0 > > > > Hi, > > > > Please review a fix to a BCEL regression. At issue was the addition of > > hashCode() method to Instruction.java in BCEL 6.0. This hashCode() > > method was implemented to return the instruction's opcode that > > unfortunately is mutable. The problem hasn't showed up until the code > > path led to calls to remove from a HashSet a target that has been > > changed since it was added to the HashSet. The proposed patch is to make > > the hashCode final/immutable. > > > > This patch implies that a target object is considered the same one even > > if its field values may have been changed. It therefore may not be > > appropriate in other situations (or may cause problems). However, since > > it had always had no hashCode() override before BCEL 6.0, thereby > > relying on Object's identity hash code, its use case has been limited > > and time-tested. It can benefit from this patch in that it provides the > > same function as Object's hash code, and then serves as a reminder (to > > any who might read the code) how it was used (objects considered to be > > the same over the course as far as the hashCode is concerned). > > > > JBS: https://bugs.openjdk.java.net/browse/JDK-8248348 > > webrevs: http://cr.openjdk.java.net/~joehw/jdk16/8248348/webrev/ > > > > Thanks, > > Joe > > > From huizhe.wang at oracle.com Mon Jun 29 06:00:56 2020 From: huizhe.wang at oracle.com (Joe Wang) Date: Sun, 28 Jun 2020 23:00:56 -0700 Subject: RFR [16/java.xml] 8248348: Regression caused by the update to BCEL 6.0 In-Reply-To: <09c93e75-8055-fbb9-9cde-195f0f002f1a@oracle.com> References: <09c93e75-8055-fbb9-9cde-195f0f002f1a@oracle.com> Message-ID: Hi all, (had a nice hike today, and asked the beautiful Lake 22 ;-)) An unconventional patch over an unusual hashcode/equals implementation is a bit too controversial. I'd like to propose a new patch that focus on the problem at hand, that is re-setting the opcode causes the item in HashSet to get lost because of the changed hash bucket. This patch therefore simply remove it from the HashSet before the change and then add it back as if it's a new item after setting the opcode. This patch resolves the issue while leaves BCEL 6.0's hashCode/equals alone. Here's the new patch: http://cr.openjdk.java.net/~joehw/jdk16/8248348/webrev02/ Thanks, Joe From Yang.Zhang at arm.com Mon Jun 29 07:48:30 2020 From: Yang.Zhang at arm.com (Yang Zhang) Date: Mon, 29 Jun 2020 07:48:30 +0000 Subject: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes In-Reply-To: References: <275eb57c-51c0-675e-c32a-91b198023559@redhat.com> <719F9169-ABC4-408E-B732-F1BD9A84337F@oracle.com> <9a13f5df-d946-579d-4282-917dc7338dc8@redhat.com> <09BC0693-80E0-4F87-855E-0B38A6F5EFA2@oracle.com> <668e500e-f621-5a2c-a41e-f73536880f73@redhat.com> <1909fa9d-98bb-c2fb-45d8-540247d1ca8b@redhat.com> Message-ID: Hi Andrew, 1. Instructions that can be matched with NEON instructions directly. MulVB, SqrtVF and AbsV have been merged into jdk master already. 2. Instructions that jdk master has middle end support for, but they cannot be matched with NEON instructions directly. Such as AddReductionVL, MulReductionVL, And/Or/XorReductionV These new instructions can be moved into jdk master first, but for auto-vectorization, the performance might not get improved. 3. Panama/Vector API specific instructions such as Load/StoreVector ( 16 bits), VectorReinterpret, VectorMaskCmp, MaxV/MinV, VectorBlend etc. These instructions cannot be moved into jdk master first because there isn't middle-end support. I will put 2 and 3 in a new ad file aarch64_neon.ad. I will also update aarch64_asmtest.py and macroassemler.cpp. When the patch is ready, I will send it again. Hi Sandhya, Could you please help to manual merge panama vectorIntrinsics/vector-unstable to jdk master? So that I can update this patch based on latest jdk master. Regards Yang -----Original Message----- From: Viswanathan, Sandhya Sent: Thursday, June 25, 2020 3:04 AM To: Yang Zhang ; Andrew Haley ; Paul Sandoz Cc: nd ; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net Subject: RE: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes Hi Andrew/Yang, We couldn?t propose Vector API to target in time for JDK 15 and hoping to do so early in JDK 16 timeframe. The implementation reviews on other components have made good progress. We have so far ok to PPT from (runtime, shared compiler changes, x86 backend). Java API implementation review is in progress. I wanted to check with you both if we have a go ahead from aarch64 backed point of view. Best Regards, Sandhya -----Original Message----- From: hotspot-compiler-dev On Behalf Of Yang Zhang Sent: Tuesday, May 26, 2020 7:59 PM To: Andrew Haley ; Paul Sandoz Cc: nd ; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net Subject: RE: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes > But to my earlier question. please: can the new instructions be moved into jdk head first, and then merged into the Panama branch, or not? The new instructions can be classified as: 1. Instructions that can be matched with NEON instructions directly. MulVB and SqrtVF have been merged into jdk master already. The patch of AbsV is in review [1]. 2. Instructions that Jdk master has middle end support for, but they cannot be matched with NEON instructions directly. Such as AddReductionVL, MulReductionVL, And/Or/XorReductionV These new instructions can be moved into jdk master first, but for auto-vectorization, the performance might not get improved. May I have a new patch for these? 3. Panama/Vector API specific instructions Such as Load/StoreVector ( 16 bits), VectorReinterpret, VectorMaskCmp, MaxV/MinV, VectorBlend etc. These instructions cannot be moved into jdk master first because there isn't middle-end support. Regards Yang [1] https://mail.openjdk.java.net/pipermail/aarch64-port-dev/2020-May/008861.html -----Original Message----- From: Andrew Haley Sent: Tuesday, May 26, 2020 4:25 PM To: Yang Zhang ; Paul Sandoz Cc: hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; nd Subject: Re: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes On 25/05/2020 09:26, Yang Zhang wrote: > In jdk master, what we need to do is that writing m4 file for existing > vector instructions and placed them to a new file aarch64_neon.ad. > If no question, I will do it right away. I'm not entirely sure that such a change is necessary now. In particular, reorganizing the existing vector instructions is IMO excessive, but I admit that it might be an improvement. But to my earlier question. please: can the new instructions be moved into jdk head first, and then merged into the Panama branch, or not? It'd help if this was possible. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From goetz.lindenmaier at sap.com Mon Jun 29 10:50:51 2020 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 29 Jun 2020 10:50:51 +0000 Subject: Explicitly constructed NPE is showing (wrong) message In-Reply-To: <1600616667.1579890.1593261686357.JavaMail.zimbra@u-pem.fr> References: <1600616667.1579890.1593261686357.JavaMail.zimbra@u-pem.fr> Message-ID: Hi, Thanks for reporting this issue. You analyzed it well, and the simple repro case was also helpful. I filed https://bugs.openjdk.java.net/browse/JDK-8248476 and will send RFR for this in a minute. It's probably best to continue discussion there. Best regards, Goetz. > -----Original Message----- > From: hotspot-runtime-dev > On Behalf Of Remi Forax > Sent: Saturday, June 27, 2020 2:41 PM > To: Christoph Dreis > Cc: hotspot-runtime-dev ; core- > libs-dev > Subject: Re: Explicitly constructed NPE is showing (wrong) message > > ----- Mail original ----- > > De: "Christoph Dreis" > > ?: "hotspot-runtime-dev" > > Envoy?: Samedi 27 Juin 2020 11:54:19 > > Objet: Explicitly constructed NPE is showing (wrong) message > > > Hi, > > Hi Christoph, > > > > > I hope this is the correct mailing list. > > > > The latest changes on JDK-8233014 to enable > ShowCodeDetailsInExceptionMessages > > by default have uncovered a likely bug (I think). > > > > To my understanding explicitly created NullPointerExceptions should not > print > > any message. > > With the following example: > > > > public class Main { > > public static void main(String[] args) { > > NullPointerException ex = new NullPointerException(); > > Throwable throwable = ex.fillInStackTrace(); > > System.out.println(throwable); > > } > > } > > > > I see the following output though: > > > > java.lang.NullPointerException: Cannot invoke > > "java.lang.NullPointerException.fillInStackTrace()" (on slot 0) because "ex" > is > > null > > > > Which obviously is not really true. > > I was debugging the thing (hence the additional "on slot 0") output, but > > couldn't find the error so far. > > > > I'm suspecting the error to be somewhere around bytecodeUtils.cpp 1155: > > > > if (name != vmSymbols::object_initializer_name()) { > > int type_index = cp- > >signature_ref_index_at(name_and_type_index); > > Symbol* signature = cp->symbol_at(type_index); > > // The 'this' parameter was null. Return the slot of it. > > return ArgumentSizeComputer(signature).size(); > > } else { > > return NPE_EXPLICIT_CONSTRUCTED; > > } > > > > I initially hoped to find a fix for it directly, but I would probably need a bit > > more time for it, so I decided to report it. > > Is this a bug even or am I chasing ghosts here? In case it is, I would be > happy > > about a mentioning somewhere in an eventual commit. > > I see the issue. > The idea of the algorithm is to use the backtrace stored in the exception to > try to find where the NPE occurs, here by calling fillStackTrace() emplicitly, > you are changing the backtrace so getExtendedNPEMessage now use the new > backtrace reporting the error in main() :( > > One way to fix the issue is to record if fillInStackTrace is called more than > once (by having two sentinels by example) and to not call > getExtendedNPEMessage if it's the case. Obviously the new added sentinel > has to work with the serialization. > > > > > Cheers, > > Christoph > > regards, > R?mi From alexey.semenyuk at oracle.com Mon Jun 29 14:47:28 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Mon, 29 Jun 2020 10:47:28 -0400 Subject: RFR: 8248059: [macos] EmptyFolderPackageTest.java failed "hdiutil: create failed - No child processes" In-Reply-To: References: <2610c5a5-d360-9bd1-cfc6-71b36e6caf72@oracle.com> <03307ab2-dacb-c379-a7dc-1b477e691df1@oracle.com> Message-ID: Alexander, Is `error.cannot-calculate-folder-size` l10n property still used in the new patch? - Alexey On 6/27/2020 12:00 AM, alexander.matveev at oracle.com wrote: > Hi Alexey, > > http://cr.openjdk.java.net/~almatvee/8248059/webrev.01/ > - Removed getFilesSizeInFolder() method and size will be calculated as > per your suggestion. > > Thanks, > Alexander > > On 6/26/20 8:16 PM, Alexey Semenyuk wrote: >> Hi Alexander, >> >> MacDmgBundler.java: >> --- >> ? 94???????? } catch (IOException | PackagerException ex) { >> ? 95???????????? Log.verbose(ex); >> ? 96???????????? throw new PackagerException(ex); >> ? 97???????? } >> --- >> >> There is no need to add IOUtils.getFilesSizeInFolder() method. >> >> --- >> new PathGroup(Map.of(new Object(), srcFolder.toPath())).sizeInBytes(); >> --- >> can be used to get the size of all regular files in `srcFolder` >> folder recursively. Besides there is resource leak in >> IOUtils.getFilesSizeInFolder() - close() should be called on the >> Stream object returned by Files.walk() call. >> >> - Alexey >> >> On 6/26/2020 9:22 PM, alexander.matveev at oracle.com wrote: >>> Please review the jpackage fix for bug [1] at [2]. >>> >>> Added fallback for creating DMG if original approach fails. In >>> original approach DMG will be created by providing app image to >>> hdiutil. It was noticed that helper sub-processes run by hdiutil to >>> copy app image sometimes crashed or failed during automated testing. >>> Fallback approach will create empty DMG and copy files manually. It >>> was observed that in this case hdiutil does not run sub-process that >>> caused tests to fail, so hopefully this workaround will fix issue. >>> Both cases produces same working DMG. Tests which failed due to this >>> issue was removed from ProblemList. >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8248059 >>> [2] http://cr.openjdk.java.net/~almatvee/8248059/webrev.00/ >>> >>> Thanks, >>> Alexander >> > From sgehwolf at redhat.com Mon Jun 29 15:53:07 2020 From: sgehwolf at redhat.com (Severin Gehwolf) Date: Mon, 29 Jun 2020 17:53:07 +0200 Subject: RFR(s): 8247863: Unreachable code in OperatingSystemImpl.getTotalSwapSpaceSize() Message-ID: Hi, Could I please get a review of this dead-code removal? During review of JDK-8244500 it was discovered that with the new cgroups implementation supporting v1 and v2 Metrics.getMemoryAndSwapLimit() will never return 0 when relevant cgroup files are missing. E.g. on a system where the kernel doesn't support swap limit capabilities. Therefore this code introduced with JDK-8236617 can no longer be reached and should get removed. Bug: https://bugs.openjdk.java.net/browse/JDK-8247863 webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8247863/01/webrev/ Testing: Matthias tested this on the affected system and it did pass for him. Docker tests on cgroup v1 and cgroup v2. Thanks, Severin From aph at redhat.com Mon Jun 29 16:10:06 2020 From: aph at redhat.com (Andrew Haley) Date: Mon, 29 Jun 2020 17:10:06 +0100 Subject: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes In-Reply-To: References: <275eb57c-51c0-675e-c32a-91b198023559@redhat.com> <719F9169-ABC4-408E-B732-F1BD9A84337F@oracle.com> <9a13f5df-d946-579d-4282-917dc7338dc8@redhat.com> <09BC0693-80E0-4F87-855E-0B38A6F5EFA2@oracle.com> <668e500e-f621-5a2c-a41e-f73536880f73@redhat.com> <1909fa9d-98bb-c2fb-45d8-540247d1ca8b@redhat.com> Message-ID: <2acbcc99-8dd4-b8f1-5982-1d439953c416@redhat.com> The message from this sender included one or more files which could not be scanned for virus detection; do not open these files unless you are certain of the sender's intent. ---------------------------------------------------------------------- On 29/06/2020 08:48, Yang Zhang wrote: > 1. Instructions that can be matched with NEON instructions directly. > MulVB, SqrtVF and AbsV have been merged into jdk master already. > > 2. Instructions that jdk master has middle end support for, but they cannot be matched with NEON instructions directly. > Such as AddReductionVL, MulReductionVL, And/Or/XorReductionV These new instructions can be moved into jdk master first, but for auto-vectorization, the performance might not get improved. > > 3. Panama/Vector API specific instructions such as Load/StoreVector ( 16 bits), VectorReinterpret, VectorMaskCmp, MaxV/MinV, VectorBlend etc. > These instructions cannot be moved into jdk master first because there isn't middle-end support. > > I will put 2 and 3 in a new ad file aarch64_neon.ad. I will also update aarch64_asmtest.py and macroassemler.cpp. When the patch is ready, I will send it again. Thank you *very* much for your hard work. Appreciated! -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From raffaello.giulietti at gmail.com Mon Jun 29 17:51:00 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Mon, 29 Jun 2020 19:51:00 +0200 Subject: contributing to JDK-8222187: java.util.Base64.Decoder stream adds unexpected null bytes at the end In-Reply-To: References: Message-ID: <3ec10c63-d628-a6f2-1751-bcc62215967a@gmail.com> Hello, here's a fix and an additional test, both in inline form and as an attachment. The fix also contains a reimplementation of encodedOutLength() that makes use of long arithmetic rather than relying on addExact(), multiplyExact(), etc. Further, lookup tables have been declared as byte[], which should help maintaining them in caches for longer because of their smaller sizes. Greetings Raffaello ---- # HG changeset patch # User lello # Date 1593437938 -7200 # Mon Jun 29 15:38:58 2020 +0200 # Node ID 58aadb9efae6d5d88479ad8171f2219d41cc761f # Parent 4a91f6b96a506d9d67437338c33b6953a57bfbd9 8222187: java.util.Base64.Decoder stream adds unexpected null bytes at the end Reviewed-by: TBD Contributed-by: Raffaello Giulietti diff --git a/src/java.base/share/classes/java/util/Base64.java b/src/java.base/share/classes/java/util/Base64.java --- a/src/java.base/share/classes/java/util/Base64.java +++ b/src/java.base/share/classes/java/util/Base64.java @@ -30,7 +30,6 @@ import java.io.IOException; import java.io.OutputStream; import java.nio.ByteBuffer; -import java.nio.charset.StandardCharsets; import sun.nio.cs.ISO_8859_1; @@ -133,7 +132,7 @@ */ public static Encoder getMimeEncoder(int lineLength, byte[] lineSeparator) { Objects.requireNonNull(lineSeparator); - int[] base64 = Decoder.fromBase64; + byte[] base64 = Decoder.fromBase64; for (byte b : lineSeparator) { if (base64[b & 0xff] != -1) throw new IllegalArgumentException( @@ -216,7 +215,7 @@ * index values into their "Base64 Alphabet" equivalents as specified * in "Table 1: The Base64 Alphabet" of RFC 2045 (and RFC 4648). */ - private static final char[] toBase64 = { + private static final byte[] toBase64 = { 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', @@ -229,7 +228,7 @@ * in Table 2 of the RFC 4648, with the '+' and '/' changed to '-' and * '_'. This table is used when BASE64_URL is specified. */ - private static final char[] toBase64URL = { + private static final byte[] toBase64URL = { 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', @@ -238,7 +237,7 @@ }; private static final int MIMELINEMAX = 76; - private static final byte[] CRLF = new byte[] {'\r', '\n'}; + private static final byte[] CRLF = {'\r', '\n'}; static final Encoder RFC4648 = new Encoder(false, null, -1, true); static final Encoder RFC4648_URLSAFE = new Encoder(true, null, -1, true); @@ -255,27 +254,19 @@ * */ private final int encodedOutLength(int srclen, boolean throwOOME) { - int len = 0; - try { - if (doPadding) { - len = Math.multiplyExact(4, (Math.addExact(srclen, 2) / 3)); - } else { - int n = srclen % 3; - len = Math.addExact(Math.multiplyExact(4, (srclen / 3)), (n == 0 ? 0 : n + 1)); - } - if (linemax > 0) { // line separators - len = Math.addExact(len, (len - 1) / linemax * newline.length); - } - } catch (ArithmeticException ex) { - if (throwOOME) { - throw new OutOfMemoryError("Encoded size is too large"); - } else { - // let the caller know that encoded bytes length - // is too large - len = -1; - } + long len = doPadding + ? (srclen + 2L) / 3 * 4 + : (srclen + 2L) / 3 + srclen; + if (linemax > 0) { + len += (len - 1) / linemax * newline.length; } - return len; + if (len <= Integer.MAX_VALUE) { + return (int) len; + } + if (throwOOME) { + throw new OutOfMemoryError("Encoded size is too large"); + } + return -1; } /** @@ -421,24 +412,24 @@ @HotSpotIntrinsicCandidate private void encodeBlock(byte[] src, int sp, int sl, byte[] dst, int dp, boolean isURL) { - char[] base64 = isURL ? toBase64URL : toBase64; + byte[] base64 = isURL ? toBase64URL : toBase64; for (int sp0 = sp, dp0 = dp ; sp0 < sl; ) { int bits = (src[sp0++] & 0xff) << 16 | (src[sp0++] & 0xff) << 8 | (src[sp0++] & 0xff); - dst[dp0++] = (byte)base64[(bits >>> 18) & 0x3f]; - dst[dp0++] = (byte)base64[(bits >>> 12) & 0x3f]; - dst[dp0++] = (byte)base64[(bits >>> 6) & 0x3f]; - dst[dp0++] = (byte)base64[bits & 0x3f]; + dst[dp0++] = base64[(bits >>> 18) & 0x3f]; + dst[dp0++] = base64[(bits >>> 12) & 0x3f]; + dst[dp0++] = base64[(bits >>> 6) & 0x3f]; + dst[dp0++] = base64[bits & 0x3f]; } } private int encode0(byte[] src, int off, int end, byte[] dst) { - char[] base64 = isURL ? toBase64URL : toBase64; + byte[] base64 = isURL ? toBase64URL : toBase64; int sp = off; int slen = (end - off) / 3 * 3; int sl = off + slen; - if (linemax > 0 && slen > linemax / 4 * 3) + if (linemax > 0 && slen > linemax / 4 * 3) slen = linemax / 4 * 3; int dp = 0; while (sp < sl) { @@ -455,17 +446,17 @@ } if (sp < end) { // 1 or 2 leftover bytes int b0 = src[sp++] & 0xff; - dst[dp++] = (byte)base64[b0 >> 2]; + dst[dp++] = base64[b0 >> 2]; if (sp == end) { - dst[dp++] = (byte)base64[(b0 << 4) & 0x3f]; + dst[dp++] = base64[(b0 << 4) & 0x3f]; if (doPadding) { dst[dp++] = '='; dst[dp++] = '='; } } else { int b1 = src[sp++] & 0xff; - dst[dp++] = (byte)base64[(b0 << 4) & 0x3f | (b1 >> 4)]; - dst[dp++] = (byte)base64[(b1 << 2) & 0x3f]; + dst[dp++] = base64[(b0 << 4) & 0x3f | (b1 >> 4)]; + dst[dp++] = base64[(b1 << 2) & 0x3f]; if (doPadding) { dst[dp++] = '='; } @@ -523,11 +514,12 @@ * the array are encoded to -1. * */ - private static final int[] fromBase64 = new int[256]; + private static final byte[] fromBase64 = new byte[256]; + static { - Arrays.fill(fromBase64, -1); + Arrays.fill(fromBase64, (byte) -1); for (int i = 0; i < Encoder.toBase64.length; i++) - fromBase64[Encoder.toBase64[i]] = i; + fromBase64[Encoder.toBase64[i]] = (byte) i; fromBase64['='] = -2; } @@ -535,12 +527,12 @@ * Lookup table for decoding "URL and Filename safe Base64 Alphabet" * as specified in Table2 of the RFC 4648. */ - private static final int[] fromBase64URL = new int[256]; + private static final byte[] fromBase64URL = new byte[256]; static { - Arrays.fill(fromBase64URL, -1); + Arrays.fill(fromBase64URL, (byte) -1); for (int i = 0; i < Encoder.toBase64URL.length; i++) - fromBase64URL[Encoder.toBase64URL[i]] = i; + fromBase64URL[Encoder.toBase64URL[i]] = (byte) i; fromBase64URL['='] = -2; } @@ -699,7 +691,7 @@ * */ private int decodedOutLength(byte[] src, int sp, int sl) { - int[] base64 = isURL ? fromBase64URL : fromBase64; + byte[] base64 = isURL ? fromBase64URL : fromBase64; int paddings = 0; int len = sl - sp; if (len == 0) @@ -743,7 +735,7 @@ } private int decode0(byte[] src, int sp, int sl, byte[] dst) { - int[] base64 = isURL ? fromBase64URL : fromBase64; + byte[] base64 = isURL ? fromBase64URL : fromBase64; int dp = 0; int bits = 0; int shiftto = 18; // pos of first byte of 4-byte atom @@ -832,14 +824,14 @@ private int b0, b1, b2; private boolean closed = false; - private final char[] base64; // byte->base64 mapping + private final byte[] base64; // byte->base64 mapping private final byte[] newline; // line separator, if needed private final int linemax; private final boolean doPadding;// whether or not to pad private int linepos = 0; private byte[] buf; - EncOutputStream(OutputStream os, char[] base64, + EncOutputStream(OutputStream os, byte[] base64, byte[] newline, int linemax, boolean doPadding) { super(os); this.base64 = base64; @@ -863,11 +855,11 @@ } } - private void writeb4(char b1, char b2, char b3, char b4) throws IOException { - buf[0] = (byte)b1; - buf[1] = (byte)b2; - buf[2] = (byte)b3; - buf[3] = (byte)b4; + private void writeb4(byte b1, byte b2, byte b3, byte b4) throws IOException { + buf[0] = b1; + buf[1] = b2; + buf[2] = b3; + buf[3] = b4; out.write(buf, 0, 4); } @@ -909,10 +901,10 @@ int bits = (b[sp++] & 0xff) << 16 | (b[sp++] & 0xff) << 8 | (b[sp++] & 0xff); - buf[dp++] = (byte)base64[(bits >>> 18) & 0x3f]; - buf[dp++] = (byte)base64[(bits >>> 12) & 0x3f]; - buf[dp++] = (byte)base64[(bits >>> 6) & 0x3f]; - buf[dp++] = (byte)base64[bits & 0x3f]; + buf[dp++] = base64[(bits >>> 18) & 0x3f]; + buf[dp++] = base64[(bits >>> 12) & 0x3f]; + buf[dp++] = base64[(bits >>> 6) & 0x3f]; + buf[dp++] = base64[bits & 0x3f]; } out.write(buf, 0, dp); off = sl; @@ -960,130 +952,119 @@ private static class DecInputStream extends InputStream { private final InputStream is; + private final byte[] base64; // mapping from alphabet to values private final boolean isMIME; - private final int[] base64; // base64 -> byte mapping - private int bits = 0; // 24-bit buffer for decoding - private int nextin = 18; // next available "off" in "bits" for input; - // -> 18, 12, 6, 0 - private int nextout = -8; // next available "off" in "bits" for output; - // -> 8, 0, -8 (no byte for output) - private boolean eof = false; - private boolean closed = false; + private int bits; // 24 bit buffer for decoding + private int wpos; // writing bit pos inside bits + // one of 24 (left, msb), 18, 12, 6, 0 + private int rpos; // reading bit pos inside bits + // one of 24 (left, msb), 16, 8, 0 + private boolean eos; + private boolean closed; + private byte[] onebuf = new byte[1]; - DecInputStream(InputStream is, int[] base64, boolean isMIME) { + DecInputStream(InputStream is, byte[] base64, boolean isMIME) { this.is = is; this.base64 = base64; this.isMIME = isMIME; } - private byte[] sbBuf = new byte[1]; - @Override public int read() throws IOException { - return read(sbBuf, 0, 1) == -1 ? -1 : sbBuf[0] & 0xff; + return read(onebuf, 0, 1) >= 0 ? onebuf[0] & 0xff : -1; + } + + private int leftovers(byte[] b, int off, int pos, int limit) { + eos = true; + while (rpos - 8 >= wpos && pos != limit) { + b[pos++] = (byte) (bits >> (rpos -= 8)); + } + return pos - off != 0 || rpos - 8 >= wpos ? pos - off : -1; } - private int eof(byte[] b, int off, int len, int oldOff) - throws IOException - { - eof = true; - if (nextin != 18) { - if (nextin == 12) - throw new IOException("Base64 stream has one un-decoded dangling byte."); - // treat ending xx/xxx without padding character legal. - // same logic as v == '=' below - b[off++] = (byte)(bits >> (16)); - if (nextin == 0) { // only one padding byte - if (len == 1) { // no enough output space - bits >>= 8; // shift to lowest byte - nextout = 0; - } else { - b[off++] = (byte) (bits >> 8); - } - } + private int eos(byte[] b, int off, int pos, int limit) throws IOException { + // wpos == 18: x dangling single x, invalid unit + // accept ending xx or xxx without padding characters + if (wpos == 18) { + throw new IOException("Base64 stream has one un-decoded dangling byte"); } - return off == oldOff ? -1 : off - oldOff; + rpos = 24; + return leftovers(b, off, pos, limit); } - private int padding(byte[] b, int off, int len, int oldOff) - throws IOException - { - // = shiftto==18 unnecessary padding - // x= shiftto==12 dangling x, invalid unit - // xx= shiftto==6 && missing last '=' - // xx=y or last is not '=' - if (nextin == 18 || nextin == 12 || - nextin == 6 && is.read() != '=') { - throw new IOException("Illegal base64 ending sequence:" + nextin); + private int padding(byte[] b, int off, int pos, int limit) throws IOException { + // wpos == 24: = (unnecessary padding) + // wpos == 18: x= (dangling single x, invalid unit) + // wpos == 12 and missing last '=': xx= (invalid padding) + // wpos == 12 and last is not '=': xx=x (invalid padding) + if (wpos >= 18 || wpos == 12 && is.read() != '=') { + throw new IOException("Illegal base64 ending sequence"); } - b[off++] = (byte)(bits >> (16)); - if (nextin == 0) { // only one padding byte - if (len == 1) { // no enough output space - bits >>= 8; // shift to lowest byte - nextout = 0; - } else { - b[off++] = (byte) (bits >> 8); - } - } - eof = true; - return off - oldOff; + rpos = 24; + return leftovers(b, off, pos, limit); } @Override public int read(byte[] b, int off, int len) throws IOException { - if (closed) + if (closed) { throw new IOException("Stream is closed"); - if (eof && nextout < 0) // eof and no leftover - return -1; - if (off < 0 || len < 0 || len > b.length - off) - throw new IndexOutOfBoundsException(); - int oldOff = off; - while (nextout >= 0) { // leftover output byte(s) in bits buf - if (len == 0) - return off - oldOff; - b[off++] = (byte)(bits >> nextout); - len--; - nextout -= 8; + } + Objects.checkFromIndexSize(off, len, b.length); + + // limit can overflow to Integer.MIN_VALUE. However, as long + // as comparisons with pos are done as coded, there's no harm. + int pos = off; + int limit = off + len; + if (eos) { + return leftovers(b, off, pos, limit); + } + + // leftovers from previous invocation; here, wpos == 0 + while (rpos - 8 >= 0 && pos != limit) { + b[pos++] = (byte) (bits >> (rpos -= 8)); } + if (pos == limit) { + return limit - off; + } + bits = 0; - while (len > 0) { - int v = is.read(); - if (v == -1) { - return eof(b, off, len, oldOff); + wpos = 24; + while (pos != limit) { + int i = is.read(); + if (i < 0) { + return eos(b, off, pos, limit); } - if ((v = base64[v]) < 0) { - if (v == -2) { // padding byte(s) - return padding(b, off, len, oldOff); - } + int v = base64[i]; + if (v < 0) { + // i not in alphabet + // v = -2 (i is '=') or v = -1 (i is other, e.g., CR or LF) if (v == -1) { - if (!isMIME) - throw new IOException("Illegal base64 character " + - Integer.toString(v, 16)); - continue; // skip if for rfc2045 + if (isMIME) { + continue; + } + throw new IOException("Illegal base64 byte 0x" + + Integer.toHexString(i)); } - // neve be here + return padding(b, off, pos, limit); } - bits |= (v << nextin); - if (nextin == 0) { - nextin = 18; // clear for next in - b[off++] = (byte)(bits >> 16); - if (len == 1) { - nextout = 8; // 2 bytes left in bits - break; + bits |= (v << (wpos -= 6)); + if (wpos == 0) { + if (limit - pos >= 3) { + // frequently taken fast path, no need to track rpos + b[pos++] = (byte) (bits >> 16); + b[pos++] = (byte) (bits >> 8); + b[pos++] = (byte) bits; + bits = 0; + wpos = 24; + } else { + rpos = 24; + while (pos != limit) { + b[pos++] = (byte) (bits >> (rpos -= 8)); + } } - b[off++] = (byte)(bits >> 8); - if (len == 2) { - nextout = 0; // 1 byte left in bits - break; - } - b[off++] = (byte)bits; - len -= 3; - bits = 0; - } else { - nextin -= 6; } } - return off - oldOff; + return limit - off; } @Override diff --git a/test/jdk/java/util/Base64/TestBase64.java b/test/jdk/java/util/Base64/TestBase64.java --- a/test/jdk/java/util/Base64/TestBase64.java +++ b/test/jdk/java/util/Base64/TestBase64.java @@ -144,6 +144,10 @@ testDecoderKeepsAbstinence(Base64.getDecoder()); testDecoderKeepsAbstinence(Base64.getUrlDecoder()); testDecoderKeepsAbstinence(Base64.getMimeDecoder()); + + // tests patch addressing JDK-8222187 + // https://bugs.openjdk.java.net/browse/JDK-8222187 + testJDK_8222187(); } private static void test(Base64.Encoder enc, Base64.Decoder dec, @@ -607,4 +611,27 @@ } } } + + private static void testJDK_8222187() throws Throwable { + byte[] orig = "12345678".getBytes("US-ASCII"); + byte[] encoded = Base64.getEncoder().encode(orig); + // decode using different buffer sizes + for (int bufferSize = 1; bufferSize <= 100; bufferSize++) { + try ( + InputStream in = Base64.getDecoder().wrap( + new ByteArrayInputStream(encoded)); + ByteArrayOutputStream baos = new ByteArrayOutputStream(); + ) { + byte[] buffer = new byte[bufferSize]; + int read; + while ((read = in.read(buffer, 0, bufferSize)) >= 0) { + baos.write(buffer, 0, read); + } + // compare result, output info if lengths do not match + byte[] decoded = baos.toByteArray(); + checkEqual(decoded, orig, "Base64 stream decoding failed!"); + } + } + + } } On 2020-06-09 09:20, Raffaello Giulietti wrote: > Hi Lance, > > before working on a fix, I just wanted to make sure that I'm not > interfering with existing efforts. Thus, I don't have a fix, yet. > > I'll be using the example provided in the bug report as a basic test. > > I'll show up here once the fix is ready. > > > Greetings > Raffaello > > > > On 2020-06-08 22:34, Lance Andersen wrote: >> Hi?Raffaello, >> >> If you are interested in providing a fix, you are more than welcome to >> do so. ?Please include a test which validates your fix. >> >> Best >> lance >> >>> On Jun 8, 2020, at 3:45 PM, Raffaello Giulietti >>> >> > wrote: >>> >>> Raffaello >> >> >> >> >> Lance >> Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle?Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> >> -------------- next part -------------- A non-text attachment was scrubbed... Name: JDK-8222187.patch Type: text/x-patch Size: 21471 bytes Desc: not available URL: From raffaello.giulietti at gmail.com Mon Jun 29 17:58:22 2020 From: raffaello.giulietti at gmail.com (Raffaello Giulietti) Date: Mon, 29 Jun 2020 19:58:22 +0200 Subject: Contributing to JEP 356: Enhanced Pseudo-Random Number Generators ? In-Reply-To: <01ae5908-c7b9-a10e-9139-a90c8ba37d6c@gmail.com> References: <01ae5908-c7b9-a10e-9139-a90c8ba37d6c@gmail.com> Message-ID: <9c90fb0e-aef1-f78c-8a5b-2bb588042ddc@gmail.com> Hello, does anybody know what the progress of this JEP is? How could I contribute? Greetings Raffaello On 2020-06-24 19:22, Raffaello Giulietti wrote: > Hi, > > seems like the last activity in JBS related to this JEP [1, 2] dates > back to June 2019. > > Are there major stumbling blocks? > Can I contribute in any way? > > > Greetings > Raffaello > > ---- > > [1] http://openjdk.java.net/jeps/356 > [2] https://bugs.openjdk.java.net/browse/JDK-8193209 From alexander.matveev at oracle.com Mon Jun 29 21:37:21 2020 From: alexander.matveev at oracle.com (alexander.matveev at oracle.com) Date: Mon, 29 Jun 2020 14:37:21 -0700 Subject: RFR: 8248059: [macos] EmptyFolderPackageTest.java failed "hdiutil: create failed - No child processes" In-Reply-To: References: <2610c5a5-d360-9bd1-cfc6-71b36e6caf72@oracle.com> <03307ab2-dacb-c379-a7dc-1b477e691df1@oracle.com> Message-ID: Hi Alexey, No, it is not used. I forgot to remove it. Patch updated: http://cr.openjdk.java.net/~almatvee/8248059/webrev.02/ Thanks, Alexander On 6/29/20 7:47 AM, Alexey Semenyuk wrote: > Alexander, > > Is `error.cannot-calculate-folder-size` l10n property still used in > the new patch? > > - Alexey > > On 6/27/2020 12:00 AM, alexander.matveev at oracle.com wrote: >> Hi Alexey, >> >> http://cr.openjdk.java.net/~almatvee/8248059/webrev.01/ >> - Removed getFilesSizeInFolder() method and size will be calculated >> as per your suggestion. >> >> Thanks, >> Alexander >> >> On 6/26/20 8:16 PM, Alexey Semenyuk wrote: >>> Hi Alexander, >>> >>> MacDmgBundler.java: >>> --- >>> ? 94???????? } catch (IOException | PackagerException ex) { >>> ? 95???????????? Log.verbose(ex); >>> ? 96???????????? throw new PackagerException(ex); >>> ? 97???????? } >>> --- >>> >>> There is no need to add IOUtils.getFilesSizeInFolder() method. >>> >>> --- >>> new PathGroup(Map.of(new Object(), srcFolder.toPath())).sizeInBytes(); >>> --- >>> can be used to get the size of all regular files in `srcFolder` >>> folder recursively. Besides there is resource leak in >>> IOUtils.getFilesSizeInFolder() - close() should be called on the >>> Stream object returned by Files.walk() call. >>> >>> - Alexey >>> >>> On 6/26/2020 9:22 PM, alexander.matveev at oracle.com wrote: >>>> Please review the jpackage fix for bug [1] at [2]. >>>> >>>> Added fallback for creating DMG if original approach fails. In >>>> original approach DMG will be created by providing app image to >>>> hdiutil. It was noticed that helper sub-processes run by hdiutil to >>>> copy app image sometimes crashed or failed during automated >>>> testing. Fallback approach will create empty DMG and copy files >>>> manually. It was observed that in this case hdiutil does not run >>>> sub-process that caused tests to fail, so hopefully this workaround >>>> will fix issue. Both cases produces same working DMG. Tests which >>>> failed due to this issue was removed from ProblemList. >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8248059 >>>> [2] http://cr.openjdk.java.net/~almatvee/8248059/webrev.00/ >>>> >>>> Thanks, >>>> Alexander >>> >> > From alexey.semenyuk at oracle.com Tue Jun 30 00:12:12 2020 From: alexey.semenyuk at oracle.com (Alexey Semenyuk) Date: Mon, 29 Jun 2020 20:12:12 -0400 Subject: RFR: 8248059: [macos] EmptyFolderPackageTest.java failed "hdiutil: create failed - No child processes" In-Reply-To: References: <2610c5a5-d360-9bd1-cfc6-71b36e6caf72@oracle.com> <03307ab2-dacb-c379-a7dc-1b477e691df1@oracle.com> Message-ID: <1ba79dd6-87bb-3a23-5d04-51d967b20dff@oracle.com> Looks good. - Alexey On 6/29/2020 5:37 PM, alexander.matveev at oracle.com wrote: > Hi Alexey, > > No, it is not used. I forgot to remove it. Patch updated: > http://cr.openjdk.java.net/~almatvee/8248059/webrev.02/ > > Thanks, > Alexander > > On 6/29/20 7:47 AM, Alexey Semenyuk wrote: >> Alexander, >> >> Is `error.cannot-calculate-folder-size` l10n property still used in >> the new patch? >> >> - Alexey >> >> On 6/27/2020 12:00 AM, alexander.matveev at oracle.com wrote: >>> Hi Alexey, >>> >>> http://cr.openjdk.java.net/~almatvee/8248059/webrev.01/ >>> - Removed getFilesSizeInFolder() method and size will be calculated >>> as per your suggestion. >>> >>> Thanks, >>> Alexander >>> >>> On 6/26/20 8:16 PM, Alexey Semenyuk wrote: >>>> Hi Alexander, >>>> >>>> MacDmgBundler.java: >>>> --- >>>> ? 94???????? } catch (IOException | PackagerException ex) { >>>> ? 95???????????? Log.verbose(ex); >>>> ? 96???????????? throw new PackagerException(ex); >>>> ? 97???????? } >>>> --- >>>> >>>> There is no need to add IOUtils.getFilesSizeInFolder() method. >>>> >>>> --- >>>> new PathGroup(Map.of(new Object(), srcFolder.toPath())).sizeInBytes(); >>>> --- >>>> can be used to get the size of all regular files in `srcFolder` >>>> folder recursively. Besides there is resource leak in >>>> IOUtils.getFilesSizeInFolder() - close() should be called on the >>>> Stream object returned by Files.walk() call. >>>> >>>> - Alexey >>>> >>>> On 6/26/2020 9:22 PM, alexander.matveev at oracle.com wrote: >>>>> Please review the jpackage fix for bug [1] at [2]. >>>>> >>>>> Added fallback for creating DMG if original approach fails. In >>>>> original approach DMG will be created by providing app image to >>>>> hdiutil. It was noticed that helper sub-processes run by hdiutil >>>>> to copy app image sometimes crashed or failed during automated >>>>> testing. Fallback approach will create empty DMG and copy files >>>>> manually. It was observed that in this case hdiutil does not run >>>>> sub-process that caused tests to fail, so hopefully this >>>>> workaround will fix issue. Both cases produces same working DMG. >>>>> Tests which failed due to this issue was removed from ProblemList. >>>>> >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8248059 >>>>> [2] http://cr.openjdk.java.net/~almatvee/8248059/webrev.00/ >>>>> >>>>> Thanks, >>>>> Alexander >>>> >>> >> > From andy.herrick at oracle.com Tue Jun 30 11:49:58 2020 From: andy.herrick at oracle.com (Andy Herrick) Date: Tue, 30 Jun 2020 07:49:58 -0400 Subject: RFR: 8248059: [macos] EmptyFolderPackageTest.java failed "hdiutil: create failed - No child processes" In-Reply-To: References: <2610c5a5-d360-9bd1-cfc6-71b36e6caf72@oracle.com> <03307ab2-dacb-c379-a7dc-1b477e691df1@oracle.com> Message-ID: <7c481548-8991-9029-2433-77463b3bb219@oracle.com> looks good. On 6/29/2020 5:37 PM, alexander.matveev at oracle.com wrote: > Hi Alexey, > > No, it is not used. I forgot to remove it. Patch updated: > http://cr.openjdk.java.net/~almatvee/8248059/webrev.02/ > > Thanks, > Alexander > > On 6/29/20 7:47 AM, Alexey Semenyuk wrote: >> Alexander, >> >> Is `error.cannot-calculate-folder-size` l10n property still used in >> the new patch? >> >> - Alexey >> >> On 6/27/2020 12:00 AM, alexander.matveev at oracle.com wrote: >>> Hi Alexey, >>> >>> http://cr.openjdk.java.net/~almatvee/8248059/webrev.01/ >>> - Removed getFilesSizeInFolder() method and size will be calculated >>> as per your suggestion. >>> >>> Thanks, >>> Alexander >>> >>> On 6/26/20 8:16 PM, Alexey Semenyuk wrote: >>>> Hi Alexander, >>>> >>>> MacDmgBundler.java: >>>> --- >>>> ? 94???????? } catch (IOException | PackagerException ex) { >>>> ? 95???????????? Log.verbose(ex); >>>> ? 96???????????? throw new PackagerException(ex); >>>> ? 97???????? } >>>> --- >>>> >>>> There is no need to add IOUtils.getFilesSizeInFolder() method. >>>> >>>> --- >>>> new PathGroup(Map.of(new Object(), srcFolder.toPath())).sizeInBytes(); >>>> --- >>>> can be used to get the size of all regular files in `srcFolder` >>>> folder recursively. Besides there is resource leak in >>>> IOUtils.getFilesSizeInFolder() - close() should be called on the >>>> Stream object returned by Files.walk() call. >>>> >>>> - Alexey >>>> >>>> On 6/26/2020 9:22 PM, alexander.matveev at oracle.com wrote: >>>>> Please review the jpackage fix for bug [1] at [2]. >>>>> >>>>> Added fallback for creating DMG if original approach fails. In >>>>> original approach DMG will be created by providing app image to >>>>> hdiutil. It was noticed that helper sub-processes run by hdiutil >>>>> to copy app image sometimes crashed or failed during automated >>>>> testing. Fallback approach will create empty DMG and copy files >>>>> manually. It was observed that in this case hdiutil does not run >>>>> sub-process that caused tests to fail, so hopefully this >>>>> workaround will fix issue. Both cases produces same working DMG. >>>>> Tests which failed due to this issue was removed from ProblemList. >>>>> >>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8248059 >>>>> [2] http://cr.openjdk.java.net/~almatvee/8248059/webrev.00/ >>>>> >>>>> Thanks, >>>>> Alexander >>>> >>> >> > From alexey at azul.com Tue Jun 30 12:22:55 2020 From: alexey at azul.com (Alexey Bakhtin) Date: Tue, 30 Jun 2020 12:22:55 +0000 Subject: RFR: 8245527: LDAP Cnannel Binding support for Java GSS/Kerberos In-Reply-To: <959ffeaa-a8ef-87fb-6a21-0f125fa3d891@oracle.com> References: <9B4D357B-43E7-4AE3-AE5C-48432309AE24@oracle.com> <3E55A2A4-6B7B-4F2A-90B6-A32CE691DDF2@azul.com> <9013C8D0-33D9-4D09-8953-F8618020A7F7@oracle.com> <8A2C4DEF-FC61-4574-B49D-1133F5349BCE@azul.com> <15fa46f2-0352-a111-a9c7-0d5fc31d9159@oracle.com> <6167ea24-e7f9-c259-e2c5-8be96d74a40d@oracle.com> <228BA21D-836A-46D7-B273-738AFFE639E4@azul.com> <97a4f719-e0f5-96f7-6637-432b4b746685@oracle.com> <567C1F82-0EE7-46F3-90E9-3A1B5FDD5E13@azul.com> <959ffeaa-a8ef-87fb-6a21-0f125fa3d891@oracle.com> Message-ID: Hello Daniel, Thank you for review. I have updated LdapCBPropertiesTest according to your comments. Updated webrev at http://cr.openjdk.java.net/~abakhtin/8245527/webrev.v9/ Regards Alexey From ivan.sipka at oracle.com Tue Jun 30 12:38:21 2020 From: ivan.sipka at oracle.com (Ivan Sipka) Date: Tue, 30 Jun 2020 12:38:21 +0000 (UTC) Subject: request for review JDK-8211290 Message-ID: Hi all, kind reminder for RFR for JDK-8211974. thank you, ----- Original Message ----- From: ivan.sipka at oracle.com To: core-libs-dev at openjdk.java.net Cc: igor.ignatyev at oracle.com Sent: Thursday, June 4, 2020 7:40:27 PM GMT +00:00 GMT Britain, Ireland, Portugal Subject: request for review JDK-8211290 Hi all, please review the following changeset: http://cr.openjdk.java.net/~iignatyev/isipka/8211974/webrev.01/index.html for the JBS issue https://bugs.openjdk.java.net/browse/JDK-8211974 which moves the files: open/test/jdk/lib/testlibrary/java/util/jar/Compiler.java open/test/jdk/lib/testlibrary/java/util/jar/JarBuilder.java to files: open/test/lib/jdk/test/lib/util/JarBuilder.java open/test/lib/jdk/test/lib/compiler/Compiler.java and changes the relevant jtreg specification tags in dependent tests: open/test/jdk/sun/net/www/protocol/jar/MultiReleaseJarURLConnection.java open/test/jdk/jdk/nio/zipfs/jarfs/JFSTester.java open/test/jdk/jdk/nio/zipfs/jarfs/MultiReleaseJarTest.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarAPI.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarHttpProperties.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarProperties.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarSecurity.java open/test/jdk/java/util/jar/JarFile/JarBacktickManifest.java open/test/jdk/java/net/URL/JarHandlerPkgPrefix/JarHandlerPkgPrefix.java and adds relevant package imports to dependent file: open/test/jdk/lib/testlibrary/java/util/jar/CreateMultiReleaseTestJars.java jtreg command to run all dependent tests from repository root jtreg -verbose:summary -ea -esa -a -conc:9 -jdk:build/linux-x64/jdk/ open/test/jdk/sun/net/www/protocol/jar/MultiReleaseJarURLConnection.java open/test/jdk/jdk/nio/zipfs/jarfs/JFSTester.java open/test/jdk/jdk/nio/zipfs/jarfs/MultiReleaseJarTest.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarAPI.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarHttpProperties.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarProperties.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarSecurity.java open/test/jdk/java/util/jar/JarFile/JarBacktickManifest.java open/test/jdk/java/net/URL/JarHandlerPkgPrefix/JarHandlerPkgPrefix.java thank you, From vladimir.x.ivanov at oracle.com Tue Jun 30 13:48:28 2020 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 30 Jun 2020 16:48:28 +0300 Subject: Scalar replacement issue in JDK 14.0.1 In-Reply-To: <426551593172556@mail.yandex.ru> References: <426551593172556@mail.yandex.ru> Message-ID: Hi Sergey, I took a look at the benchmark and I think there's more than 11 vs 14 in play here. When I compiled the benchmark with jdk8, I saw the following in the compilation log (irrespective of jdk version used): 1221 67 b org.openjdk.ea.StringCompositeKeyBenchmark::compositeKey (22 bytes) ... @ 15 org.openjdk.ea.StringCompositeKeyBenchmark$Key:: (7 bytes) unloaded signature classes ... The constructor is not inlined, so even if the Key instance doesn't escape globally, it escapes into a call and C2 can't scalar replace it. The reason why inlining fails is private constructor can't be accessed directly, but requires a bridge method. Bridge method has additional method argument which has a non-existent type (with unique name). Inlining heuristics don't inline methods which have unresolved classes in their signatures. But if you recompile the benchmark with jdk11 (or later), inlining happends and the allocation is eliminated [1]. If you look at the bytecodes, there's no bridge method anymore. Javac generates NestMembers attribute instead which is enough to make private constructor accessible from the enclosing class: $ javap -verbose -private target/classes//org/openjdk/ea/StringCompositeKeyBenchmark.class ... NestMembers: org/openjdk/ea/StringCompositeKeyBenchmark$Key org/openjdk/ea/StringCompositeKeyBenchmark$Data ... So, it boils down to the target language level being used. Starting 11, javac doesn't emit bridge methods anymore and it helps with getting EA in C2 to eliminate the allocation. Best regards, Vladimir Ivanov [1] $ javap -verbose -private target/classes//org/openjdk/ea/StringCompositeKeyBenchmark.class public java.lang.Object compositeKey(org.openjdk.ea.StringCompositeKeyBenchmark$Data); descriptor: (Lorg/openjdk/ea/StringCompositeKeyBenchmark$Data;)Ljava/lang/Object; flags: (0x0001) ACC_PUBLIC Code: stack=6, locals=2, args_size=2 0: aload_1 1: invokestatic #9 // Method org/openjdk/ea/StringCompositeKeyBenchmark$Data.access$200:(Lorg/openjdk/ea/StringCompositeKeyBenchmark$Data;)Ljava/util/HashMap; 4: new #13 // class org/openjdk/ea/StringCompositeKeyBenchmark$Key 7: dup 8: ldc #15 // String code1 10: aload_1 11: invokestatic #17 // Method org/openjdk/ea/StringCompositeKeyBenchmark$Data.access$000:(Lorg/openjdk/ea/StringCompositeKeyBenchmark$Data;)Ljava/util/Locale; 14: aconst_null 15: invokespecial #21 // Method org/openjdk/ea/StringCompositeKeyBenchmark$Key."":(Ljava/lang/String;Ljava/util/Locale;Lorg/openjdk/ea/StringCompositeKeyBenchmark$1;)V 18: invokevirtual #24 // Method java/util/HashMap.get:(Ljava/lang/Object;)Ljava/lang/Object; 21: areturn $ javap -verbose -private target/classes//org/openjdk/ea/StringCompositeKeyBenchmark\$Key.class ... org.openjdk.ea.StringCompositeKeyBenchmark$Key(java.lang.String, java.util.Locale, org.openjdk.ea.StringCompositeKeyBenchmark$1); descriptor: (Ljava/lang/String;Ljava/util/Locale;Lorg/openjdk/ea/StringCompositeKeyBenchmark$1;)V ... [2] 1426 67 b org.openjdk.ea.StringCompositeKeyBenchmark::compositeKey (26 bytes) ======== Connection graph for org.openjdk.ea.StringCompositeKeyBenchmark::compositeKey JavaObject NoEscape(NoEscape) [ 148F 142F 144F 137F [ 58 63 ]] 46 Allocate === 29 6 7 8 1 ( 44 43 39 1 1 37 42 ) [[ 47 48 49 56 57 58 ]] rawptr:NotNull ( int:>=0, java/lang/Object:NotNull *, bool, top ) StringCompositeKeyBenchmark::compositeKey @ bci:4 !jvms: StringCompositeKeyBenchmark::compositeKey @ bci:4 LocalVar [ 46P [ 63 148b 142b ]] 58 Proj === 46 [[ 59 63 142 148 ]] #5 !jvms: StringCompositeKeyBenchmark::compositeKey @ bci:4 LocalVar [ 58 46P [ 144b 137b ]] 63 CheckCastPP === 60 58 [[ 906 865 814 814 865 717 689 689 144 678 717 437 137 137 144 155 166 185 678 678 649 450 223 241 649 634 618 598 576 522 522 990 990 478 478 450 371 371 424 424 437 ]] #org/openjdk/ea/StringCompositeKeyBenchmark$Key:NotNull:exact * Oop:org/openjdk/ea/StringCompositeKeyBenchmark$Key:NotNull:exact * !jvms: StringCompositeKeyBenchmark::compositeKey @ bci:4 Scalar 63 CheckCastPP === 60 58 [[ 906 865 814 814 865 717 689 689 424 990 717 437 522 522 424 437 166 185 990 478 478 450 223 241 450 634 618 598 576 ]] #org/openjdk/ea/StringCompositeKeyBenchmark$Key:NotNull:exact *,iid=46 Oop:org/openjdk/ea/StringCompositeKeyBenchmark$Key:NotNull:exact *,iid=46 !jvms: StringCompositeKeyBenchmark::compositeKey @ bci:4 ++++ Eliminated: 46 Allocate @ 9 java.util.Objects::requireNonNull (14 bytes) inline (hot) @ 19 org.openjdk.ea.StringCompositeKeyBenchmark$Key:: (15 bytes) inline (hot) @ 1 java.lang.Object:: (1 bytes) inline (hot) On 26.06.2020 15:06, ?????? ??????? wrote: > Hello, > > while looking into an issue I've found out that scalar replacement is not working in trivial case on JDK 14.0.1. > > This benchmark illustrates the issue: > > @State(Scope.Thread) > @BenchmarkMode(Mode.AverageTime) > @OutputTimeUnit(TimeUnit.NANOSECONDS) > @Fork(jvmArgsAppend = {"-Xms2g", "-Xmx2g"}) > public class StringCompositeKeyBenchmark { > @Benchmark > public Object compositeKey(Data data) { > return data.keyObjectMap.get(new Key(data.code, data.locale)); > } > > > @State(Scope.Thread) > public static class Data { > private final String code = "code1"; > private final Locale locale = Locale.getDefault(); > > private final HashMap keyObjectMap = new HashMap<>(); > > @Setup > public void setUp() { > keyObjectMap.put(new Key(code, locale), new Object()); > } > } > > private static final class Key { > private final String code; > private final Locale locale; > > private Key(String code, Locale locale) { > this.code = code; > this.locale = locale; > } > > @Override > public boolean equals(Object o) { > if (this == o) return true; > if (o == null || getClass() != o.getClass()) return false; > > Key key = (Key) o; > > if (!code.equals(key.code)) return false; > return locale.equals(key.locale); > } > > @Override > public int hashCode() { > return 31 * code.hashCode() + locale.hashCode(); > } > } > } > > When I run this on JDK 11 (JDK 11.0.7, OpenJDK 64-Bit Server VM, 11.0.7+10-post-Ubuntu-2ubuntu218.04) I get this output: > > Benchmark Mode Cnt Score Error Units > StringCompositeKeyBenchmark.compositeKey avgt 10 5.510 ? 0.121 ns/op > StringCompositeKeyBenchmark.compositeKey:?gc.alloc.rate avgt 10 ? 10?? MB/sec > StringCompositeKeyBenchmark.compositeKey:?gc.alloc.rate.norm avgt 10 ? 10?? B/op > StringCompositeKeyBenchmark.compositeKey:?gc.count avgt 10 ? 0 counts > > As I understand Java runtime erases object allocation here and we don't use additional memory. > > Same run on JDK 14 (JDK 14.0.1, Java HotSpot(TM) 64-Bit Server VM, 14.0.1+7) demonstrate object allocation per each method call: > > Benchmark Mode Cnt Score Error Units > StringCompositeKeyBenchmark.compositeKey avgt 10 7.958 ? 1.360 ns/op > StringCompositeKeyBenchmark.compositeKey:?gc.alloc.rate avgt 10 1937.551 ? 320.718 MB/sec > StringCompositeKeyBenchmark.compositeKey:?gc.alloc.rate.norm avgt 10 24.001 ? 0.001 B/op > StringCompositeKeyBenchmark.compositeKey:?gc.churn.G1_Eden_Space avgt 10 1879.111 ? 596.770 MB/sec > StringCompositeKeyBenchmark.compositeKey:?gc.churn.G1_Eden_Space.norm avgt 10 23.244 ? 5.509 B/op > StringCompositeKeyBenchmark.compositeKey:?gc.churn.G1_Survivor_Space avgt 10 0.267 ? 0.750 MB/sec > StringCompositeKeyBenchmark.compositeKey:?gc.churn.G1_Survivor_Space.norm avgt 10 0.003 ? 0.009 B/op > StringCompositeKeyBenchmark.compositeKey:?gc.count avgt 10 23.000 counts > StringCompositeKeyBenchmark.compositeKey:?gc.time avgt 10 44.000 ms > > At the same time in more trivial scenario like > > @Benchmark > public int compositeKey(Data data) { > return new Key(data.code, data.locale).hashCode(); > } > > scalar replacement again eliminates allocation of object. > > So I'm curious whether this is normal behaviour or a bug? > > Regards, > Sergey Tsypanov > From sean.coffey at oracle.com Tue Jun 30 13:51:57 2020 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Tue, 30 Jun 2020 14:51:57 +0100 Subject: RFR: 8218021: Have jarsigner preserve posix permission attributes In-Reply-To: References: <6a34a99c-ed06-5e38-0932-788e944a70df@oracle.com> <22AD74BE-6E90-45CA-AEEF-73E5FCCC0BF5@oracle.com> <60ef9824-bb1c-9ff1-11a8-76290f8f8882@oracle.com> Message-ID: <9903a925-a93e-d712-1d6c-80db047e2907@oracle.com> Thanks Lance. During the CSR review, a suggestion was made to have jarsigner preserve such attributes by default. Warnings about these attributes will also be added during signing and verify operations (if detected). webrev: https://cr.openjdk.java.net/~coffeys/webrev.8218021.v4/webrev/ regards, Sean. On 22/06/2020 17:17, Lance Andersen wrote: > HI Sean, > > Looks OK based on our exchanges. ?Thank you for your time on this one! > > Best > Lance > >> On Jun 22, 2020, at 7:22 AM, Se?n Coffey > > wrote: >> >> Thanks Lance. >> >> I've updated the patch with some extra offline feedback from yourself >> and Max. >> A new warning is printed with use of the new flag. A warning is also >> printed when file posix permissions are detected on resources being >> signed. Test updated for that also. >> >> https://cr.openjdk.java.net/~coffeys/webrev.8218021.v3/webrev/ >> >> regards, >> Sean. >> >> On 12/06/2020 17:05, Lance Andersen wrote: >>> Hi Sean, >>> >>> I think your changes look fine so all good FMPOV. >>> >>> Best >>> Lance >>> >>>> On Jun 12, 2020, at 6:21 AM, Se?n Coffey >>> > wrote: >>>> >>>> Hi, >>>> >>>> I'd like to reboot this jarsigner enhancement request[1]. I've >>>> removed the problem references to zip file name extensions. >>>> Instead, there's a new JDK implementation specific jarsigner >>>> option: -keepposixperms >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8218021 >>>> https://cr.openjdk.java.net/~coffeys/webrev.8218021.v2/webrev/ >>>> >>>> regards, >>>> Sean. >>>> >>>> [1] >>>> http://mail.openjdk.java.net/pipermail/security-dev/2020-January/021141.html >>>> >>> >>> >>> >>> >>> Lance >>> Andersen| Principal Member of Technical Staff | +1.781.442.2037 >>> Oracle?Java Engineering >>> 1 Network Drive >>> Burlington, MA 01803 >>> Lance.Andersen at oracle.com >>> >>> >>> > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle?Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From sandhya.viswanathan at intel.com Tue Jun 30 18:56:34 2020 From: sandhya.viswanathan at intel.com (Viswanathan, Sandhya) Date: Tue, 30 Jun 2020 18:56:34 +0000 Subject: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes In-Reply-To: References: <275eb57c-51c0-675e-c32a-91b198023559@redhat.com> <719F9169-ABC4-408E-B732-F1BD9A84337F@oracle.com> <9a13f5df-d946-579d-4282-917dc7338dc8@redhat.com> <09BC0693-80E0-4F87-855E-0B38A6F5EFA2@oracle.com> <668e500e-f621-5a2c-a41e-f73536880f73@redhat.com> <1909fa9d-98bb-c2fb-45d8-540247d1ca8b@redhat.com> Message-ID: Hi Yang, I have merged vectorIntrinsics with changes from panama/default. Hope this helps. Best Regards, Sandhya -----Original Message----- From: Yang Zhang Sent: Monday, June 29, 2020 12:49 AM To: Viswanathan, Sandhya ; Andrew Haley ; Paul Sandoz Cc: nd ; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net Subject: RE: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes Hi Andrew, 1. Instructions that can be matched with NEON instructions directly. MulVB, SqrtVF and AbsV have been merged into jdk master already. 2. Instructions that jdk master has middle end support for, but they cannot be matched with NEON instructions directly. Such as AddReductionVL, MulReductionVL, And/Or/XorReductionV These new instructions can be moved into jdk master first, but for auto-vectorization, the performance might not get improved. 3. Panama/Vector API specific instructions such as Load/StoreVector ( 16 bits), VectorReinterpret, VectorMaskCmp, MaxV/MinV, VectorBlend etc. These instructions cannot be moved into jdk master first because there isn't middle-end support. I will put 2 and 3 in a new ad file aarch64_neon.ad. I will also update aarch64_asmtest.py and macroassemler.cpp. When the patch is ready, I will send it again. Hi Sandhya, Could you please help to manual merge panama vectorIntrinsics/vector-unstable to jdk master? So that I can update this patch based on latest jdk master. Regards Yang -----Original Message----- From: Viswanathan, Sandhya Sent: Thursday, June 25, 2020 3:04 AM To: Yang Zhang ; Andrew Haley ; Paul Sandoz Cc: nd ; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net Subject: RE: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes Hi Andrew/Yang, We couldn?t propose Vector API to target in time for JDK 15 and hoping to do so early in JDK 16 timeframe. The implementation reviews on other components have made good progress. We have so far ok to PPT from (runtime, shared compiler changes, x86 backend). Java API implementation review is in progress. I wanted to check with you both if we have a go ahead from aarch64 backed point of view. Best Regards, Sandhya -----Original Message----- From: hotspot-compiler-dev On Behalf Of Yang Zhang Sent: Tuesday, May 26, 2020 7:59 PM To: Andrew Haley ; Paul Sandoz Cc: nd ; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net Subject: RE: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes > But to my earlier question. please: can the new instructions be moved into jdk head first, and then merged into the Panama branch, or not? The new instructions can be classified as: 1. Instructions that can be matched with NEON instructions directly. MulVB and SqrtVF have been merged into jdk master already. The patch of AbsV is in review [1]. 2. Instructions that Jdk master has middle end support for, but they cannot be matched with NEON instructions directly. Such as AddReductionVL, MulReductionVL, And/Or/XorReductionV These new instructions can be moved into jdk master first, but for auto-vectorization, the performance might not get improved. May I have a new patch for these? 3. Panama/Vector API specific instructions Such as Load/StoreVector ( 16 bits), VectorReinterpret, VectorMaskCmp, MaxV/MinV, VectorBlend etc. These instructions cannot be moved into jdk master first because there isn't middle-end support. Regards Yang [1] https://mail.openjdk.java.net/pipermail/aarch64-port-dev/2020-May/008861.html -----Original Message----- From: Andrew Haley Sent: Tuesday, May 26, 2020 4:25 PM To: Yang Zhang ; Paul Sandoz Cc: hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; nd Subject: Re: [aarch64-port-dev ] RFR (XXL): 8223347: Integration of Vector API (Incubator): AArch64 backend changes On 25/05/2020 09:26, Yang Zhang wrote: > In jdk master, what we need to do is that writing m4 file for existing > vector instructions and placed them to a new file aarch64_neon.ad. > If no question, I will do it right away. I'm not entirely sure that such a change is necessary now. In particular, reorganizing the existing vector instructions is IMO excessive, but I admit that it might be an improvement. But to my earlier question. please: can the new instructions be moved into jdk head first, and then merged into the Panama branch, or not? It'd help if this was possible. -- Andrew Haley (he/him) Java Platform Lead Engineer Red Hat UK Ltd. https://keybase.io/andrewhaley EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From brent.christian at oracle.com Tue Jun 30 21:15:57 2020 From: brent.christian at oracle.com (Brent Christian) Date: Tue, 30 Jun 2020 14:15:57 -0700 Subject: request for review JDK-8211290 In-Reply-To: References: Message-ID: Hi, Ivan The changes look fine to me. Please see that an automated test run that includes all the changed tests is performed. Thanks, -Brent On 6/30/20 5:38 AM, Ivan Sipka wrote: > Hi all, > > kind reminder for RFR for JDK-8211974. > > thank you, > > ----- Original Message ----- > From: ivan.sipka at oracle.com > To: core-libs-dev at openjdk.java.net > Cc: igor.ignatyev at oracle.com > Sent: Thursday, June 4, 2020 7:40:27 PM GMT +00:00 GMT Britain, Ireland, Portugal > Subject: request for review JDK-8211290 > > Hi all, > > please review the following changeset: http://cr.openjdk.java.net/~iignatyev/isipka/8211974/webrev.01/index.html > for the JBS issue https://bugs.openjdk.java.net/browse/JDK-8211974 > > which moves the files: > > open/test/jdk/lib/testlibrary/java/util/jar/Compiler.java > open/test/jdk/lib/testlibrary/java/util/jar/JarBuilder.java > > to files: > > open/test/lib/jdk/test/lib/util/JarBuilder.java > open/test/lib/jdk/test/lib/compiler/Compiler.java > > and changes the relevant jtreg specification tags in dependent tests: > > open/test/jdk/sun/net/www/protocol/jar/MultiReleaseJarURLConnection.java > open/test/jdk/jdk/nio/zipfs/jarfs/JFSTester.java > open/test/jdk/jdk/nio/zipfs/jarfs/MultiReleaseJarTest.java > open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarAPI.java > open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarHttpProperties.java > open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarProperties.java > open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarSecurity.java > open/test/jdk/java/util/jar/JarFile/JarBacktickManifest.java > open/test/jdk/java/net/URL/JarHandlerPkgPrefix/JarHandlerPkgPrefix.java > > and adds relevant package imports to dependent file: > > open/test/jdk/lib/testlibrary/java/util/jar/CreateMultiReleaseTestJars.java > > jtreg command to run all dependent tests from repository root > > jtreg -verbose:summary -ea -esa -a -conc:9 -jdk:build/linux-x64/jdk/ open/test/jdk/sun/net/www/protocol/jar/MultiReleaseJarURLConnection.java open/test/jdk/jdk/nio/zipfs/jarfs/JFSTester.java open/test/jdk/jdk/nio/zipfs/jarfs/MultiReleaseJarTest.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarAPI.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarHttpProperties.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarProperties.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarSecurity.java open/test/jdk/java/util/jar/JarFile/JarBacktickManifest.java open/test/jdk/java/net/URL/JarHandlerPkgPrefix/JarHandlerPkgPrefix.java > > thank you, > From Roger.Riggs at oracle.com Tue Jun 30 22:03:01 2020 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Tue, 30 Jun 2020 18:03:01 -0400 Subject: request for review JDK-8211290 In-Reply-To: References: Message-ID: Hi Ivan, In test/jdk/java/util/jar/JarFile/JarBacktickManifest.java, I don't think you want the new blank line after the removed @summary. Otherwise, looks fine. Roger On 6/30/20 5:15 PM, Brent Christian wrote: > Hi, Ivan > > The changes look fine to me. > Please see that an automated test run that includes all the changed > tests is performed. > > Thanks, > -Brent > > On 6/30/20 5:38 AM, Ivan Sipka wrote: >> Hi all, >> >> kind reminder for RFR for JDK-8211974. >> >> thank you, >> >> ----- Original Message ----- >> From: ivan.sipka at oracle.com >> To: core-libs-dev at openjdk.java.net >> Cc: igor.ignatyev at oracle.com >> Sent: Thursday, June 4, 2020 7:40:27 PM GMT +00:00 GMT Britain, >> Ireland, Portugal >> Subject: request for review JDK-8211290 >> >> Hi all, >> >> please review the following changeset: >> http://cr.openjdk.java.net/~iignatyev/isipka/8211974/webrev.01/index.html >> for the JBS issue https://bugs.openjdk.java.net/browse/JDK-8211974 >> >> which moves the files: >> >> open/test/jdk/lib/testlibrary/java/util/jar/Compiler.java >> open/test/jdk/lib/testlibrary/java/util/jar/JarBuilder.java >> >> to files: >> >> open/test/lib/jdk/test/lib/util/JarBuilder.java >> open/test/lib/jdk/test/lib/compiler/Compiler.java >> >> and changes the relevant jtreg specification tags in dependent tests: >> >> open/test/jdk/sun/net/www/protocol/jar/MultiReleaseJarURLConnection.java >> open/test/jdk/jdk/nio/zipfs/jarfs/JFSTester.java >> open/test/jdk/jdk/nio/zipfs/jarfs/MultiReleaseJarTest.java >> open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarAPI.java >> open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarHttpProperties.java >> >> open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarProperties.java >> open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarSecurity.java >> open/test/jdk/java/util/jar/JarFile/JarBacktickManifest.java >> open/test/jdk/java/net/URL/JarHandlerPkgPrefix/JarHandlerPkgPrefix.java >> >> and adds relevant package imports to dependent file: >> >> open/test/jdk/lib/testlibrary/java/util/jar/CreateMultiReleaseTestJars.java >> >> >> jtreg command to run all dependent tests from repository root >> >> jtreg -verbose:summary -ea -esa -a -conc:9 -jdk:build/linux-x64/jdk/ >> open/test/jdk/sun/net/www/protocol/jar/MultiReleaseJarURLConnection.java >> open/test/jdk/jdk/nio/zipfs/jarfs/JFSTester.java >> open/test/jdk/jdk/nio/zipfs/jarfs/MultiReleaseJarTest.java >> open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarAPI.java >> open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarHttpProperties.java >> open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarProperties.java >> open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarSecurity.java >> open/test/jdk/java/util/jar/JarFile/JarBacktickManifest.java >> open/test/jdk/java/net/URL/JarHandlerPkgPrefix/JarHandlerPkgPrefix.java >> >> thank you, >> From lance.andersen at oracle.com Tue Jun 30 22:15:30 2020 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 30 Jun 2020 18:15:30 -0400 Subject: request for review JDK-8211290 In-Reply-To: References: Message-ID: <105EDE3D-E8A4-4798-BDA4-2EDE6803F964@oracle.com> Hi Ivan, Looks fine to me also outside of the one extra line that Roger caught. Best Lance > On Jun 30, 2020, at 8:38 AM, Ivan Sipka wrote: > > Hi all, > > kind reminder for RFR for JDK-8211974. > > thank you, > > ----- Original Message ----- > From: ivan.sipka at oracle.com > To: core-libs-dev at openjdk.java.net > Cc: igor.ignatyev at oracle.com > Sent: Thursday, June 4, 2020 7:40:27 PM GMT +00:00 GMT Britain, Ireland, Portugal > Subject: request for review JDK-8211290 > > Hi all, > > please review the following changeset: http://cr.openjdk.java.net/~iignatyev/isipka/8211974/webrev.01/index.html > for the JBS issue https://bugs.openjdk.java.net/browse/JDK-8211974 > > which moves the files: > > open/test/jdk/lib/testlibrary/java/util/jar/Compiler.java > open/test/jdk/lib/testlibrary/java/util/jar/JarBuilder.java > > to files: > > open/test/lib/jdk/test/lib/util/JarBuilder.java > open/test/lib/jdk/test/lib/compiler/Compiler.java > > and changes the relevant jtreg specification tags in dependent tests: > > open/test/jdk/sun/net/www/protocol/jar/MultiReleaseJarURLConnection.java > open/test/jdk/jdk/nio/zipfs/jarfs/JFSTester.java > open/test/jdk/jdk/nio/zipfs/jarfs/MultiReleaseJarTest.java > open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarAPI.java > open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarHttpProperties.java > open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarProperties.java > open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarSecurity.java > open/test/jdk/java/util/jar/JarFile/JarBacktickManifest.java > open/test/jdk/java/net/URL/JarHandlerPkgPrefix/JarHandlerPkgPrefix.java > > and adds relevant package imports to dependent file: > > open/test/jdk/lib/testlibrary/java/util/jar/CreateMultiReleaseTestJars.java > > jtreg command to run all dependent tests from repository root > > jtreg -verbose:summary -ea -esa -a -conc:9 -jdk:build/linux-x64/jdk/ open/test/jdk/sun/net/www/protocol/jar/MultiReleaseJarURLConnection.java open/test/jdk/jdk/nio/zipfs/jarfs/JFSTester.java open/test/jdk/jdk/nio/zipfs/jarfs/MultiReleaseJarTest.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarAPI.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarHttpProperties.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarProperties.java open/test/jdk/java/util/jar/JarFile/mrjar/MultiReleaseJarSecurity.java open/test/jdk/java/util/jar/JarFile/JarBacktickManifest.java open/test/jdk/java/net/URL/JarHandlerPkgPrefix/JarHandlerPkgPrefix.java > > thank you, Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com