From mandy.chung at oracle.com  Wed Apr  1 03:01:10 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Tue, 31 Mar 2020 20:01:10 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
Message-ID: <1a5eb022-dd7e-b0bb-750f-c43be195eb42@oracle.com>

Thanks to the review feedbacks.

Updated webrev:
http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.04/
Delta between webrev.03 and webrev.04:
http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03-delta/

Summary of changes is:
1. Update javac to retain the previous behavior when compiling to target 
release <= 14 where lambda proxy class is not a nestmate
2. Rename Class::isHiddenClass to Class::isHidden
3. Address Joe's feedback on the CSR to document the behavior of the 
relevant methods in java.lang.Class for hidden classes
4. Add test case for unloadable class with nest host error
5. Add test cases for hidden classes with different kinds of class or 
interface
6. Update dcmd to drop "weak hidden class" and refer it as "hidden class"
7. Small changes in response to Remi, Coleen, Paul's review comments
8. Fix copyright headers
9. Fix @modules in tests

Most of the changes above have also been reviewed as separate patches.

Thanks
Mandy

On 3/26/20 4:57 PM, Mandy Chung wrote:
> Please review the implementation of JEP 371: Hidden Classes. The main 
> changes are in core-libs and hotspot runtime area.? Small changes are 
> made in javac, VM compiler (intrinsification of Class::isHiddenClass), 
> JFR, JDI, and jcmd.? CSR [1]has been reviewed and is in the finalized 
> state (see specdiff and javadoc below for reference).
>
> Webrev:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
>
>
> javadoc/specdiff
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
>
>
> JVMS 5.4.4 change:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
>
>
> CSR:
> https://bugs.openjdk.java.net/browse/JDK-8238359
>
> Thanks
> Mandy
> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
> [3] https://bugs.openjdk.java.net/browse/JDK-8230502


From denghui.ddh at alibaba-inc.com  Wed Apr  1 03:14:12 2020
From: denghui.ddh at alibaba-inc.com (Denghui Dong)
Date: Wed, 01 Apr 2020 11:14:12 +0800
Subject: =?UTF-8?B?UmU6IFBJTkc6IFJGUihTKTogODI0MDM2MDogTmF0aXZlTGlicmFyeUV2ZW50IGhhcyB3cm9u?=
 =?UTF-8?B?ZyBsaWJyYXJ5IG5hbWUgb24gTGludXg=?=
In-Reply-To: <ae0e9d38-b560-425f-a571-97a54803f11b.denghui.ddh@alibaba-inc.com>
References: <0741e423-3bfc-4bbc-8929-54500f387cb7.denghui.ddh@alibaba-inc.com>
 <9f586a0e-11ff-1dce-575a-508686a3427f@oss.nttdata.com>
 <b8a5a357-40af-4359-bf6f-f978a85524ae.denghui.ddh@alibaba-inc.com>,
 <774cc959-d635-0699-34d4-f85bb83de764@oss.nttdata.com>,
 <ae0e9d38-b560-425f-a571-97a54803f11b.denghui.ddh@alibaba-inc.com>
Message-ID: <1a54a719-ef71-4749-b0c3-8a76ed5086a9.denghui.ddh@alibaba-inc.com>

Hi,

Could anyone review it, need one more reviewer.

Webrev: http://cr.openjdk.java.net/~ddong/8240360/webrev.01/ 
JBS: https://bugs.openjdk.java.net/browse/JDK-8240360

Summary:
the device number in my Linux environment is 103:01, the format passed to sscanf can't identify it.
My fix is very small, use "%*s" to ignore useless information.


Thanks
Denghui


------------------------------------------------------------------
From:???(??) <denghui.ddh at alibaba-inc.com>
Send Time:2020?3?23?(???) 21:26
To:hotspot-jfr-dev <hotspot-jfr-dev at openjdk.java.net>; Yasumasa Suenaga <suenaga at oss.nttdata.com>
Subject:Re: PING: RFR(S): 8240360: NativeLibraryEvent has wrong library name on Linux

Thank you Yasumasa.

Can I get one more reviewer please?

Cheers,
Denghui Dong
------------------------------------------------------------------
From:Yasumasa Suenaga <suenaga at oss.nttdata.com>
Send Time:2020?3?23?(???) 20:25
To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-jfr-dev <hotspot-jfr-dev at openjdk.java.net>
Subject:Re: PING: RFR(S): 8240360: NativeLibraryEvent has wrong library name on Linux

Hi Denghui,

On 2020/03/23 18:08, Denghui Dong wrote:
> Hi Yasumasa,
> 
> Thanks for the review.
> 
> I agree with you that it's necessary to filter those lines in which the file is not a shared object by check the exec bit.
> I think we should create another bug to trace the problem that you mentioned.

I filed it:
https://bugs.openjdk.java.net/browse/JDK-8241439

I think we might use dl_iterate_phdr() for the fix. I will try it.


> BTW, Could you sponsor it?

Sure, but you need to get one more reviewer before pushing.


Thanks,

Yasumasa


> Thanks,
> Denghui Dong
> 
>     ------------------------------------------------------------------
>     From:Yasumasa Suenaga <suenaga at oss.nttdata.com>
>     Send Time:2020?3?23?(???) 16:31
>     To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-jfr-dev <hotspot-jfr-dev at openjdk.java.net>
>     Subject:Re: PING: RFR(S): 8240360: NativeLibraryEvent has wrong library name on Linux
> 
>     Hi Denghui,
> 
>     Your change looks good.
> 
>     However, we might need to update os::get_loaded_modules_info() more.
>     Currently, we would get multiple jdk.NativeLibrary events from 1 library as below:
> 
>     ```
>     jdk.NativeLibrary {
>         startTime = 16:59:06.621
>         name = "/usr/lib64/libc-2.30.so"
>         baseAddress = 0x7FDB06ACC000
>         topAddress = 0x7FDB06AF1000
>     }
> 
>     jdk.NativeLibrary {
>         startTime = 16:59:06.621
>         name = "/usr/lib64/libc-2.30.so"
>         baseAddress = 0x7FDB06AF1000
>         topAddress = 0x7FDB06C40000
>     }
> 
>          :
> 
>     ```
> 
>     In addition, we would get the events even if they are not shared library as below:
> 
>     ```
>     jdk.NativeLibrary {
>         startTime = 16:59:06.621
>         name = "/home/ysuenaga/OpenJDK/jdk/build/linux-x86_64-server-fastdebug/images/jdk/lib/server/cl
>     asses.jsa"
>         baseAddress = 0x800000000
>         topAddress = 0x800007000
>     }
>     ```
> 
>     os::get_loaded_modules_info() has introduced in JDK-8056242, and it says this function is for dll/so/dylibs.
>     In fact, os::get_loaded_modules_info() uses EnumProcessModules() in Windows.
> 
>     So I think we need to check exec bit, and merge same library to one event.
>     But I think it is another issue, and I'm not sure my think is correct.
> 
>     Thus I agree with your change now.
> 
> 
>     Thanks,
> 
>     Yasumasa (ysuenaga)
> 
> 
>     On 2020/03/20 11:12, Denghui Dong wrote:
>      > PING: Could you review it?
>      >
>      > Webrev: http://cr.openjdk.java.net/~ddong/8240360/webrev.01/
>      > JBS: https://bugs.openjdk.java.net/browse/JDK-8240360
>      >
>      > In Linux, EventNativeLibrary has a wrong name if the target library is located in a device whose device number (major or minor) exceeds the range of two digits.
>      >
>      > My fix is small, use "%*s" to ignore perms, offset and device.
>      >
>      > --- old/src/hotspot/os/linux/os_linux.cpp 2020-03-10 14:35:41.224134041 +0800
>      > +++ new/src/hotspot/os/linux/os_linux.cpp 2020-03-10 14:35:41.090129104 +0800
>      > @@ -2078,20 +2078,18 @@
>      >
>      >       // Read line by line from 'file'
>      >       while (fgets(line, sizeof(line), procmapsFile) != NULL) {
>      > -      u8 base, top, offset, inode;
>      > -      char permissions[5];
>      > -      char device[6];
>      > +      u8 base, top, inode;
>      >         char name[sizeof(line)];
>      >
>      > -      // Parse fields from line
>      > -      int matches = sscanf(line, UINT64_FORMAT_X "-" UINT64_FORMAT_X " %4s " UINT64_FORMAT_X " %5s " INT64_FORMAT " %s",
>      > -             &base, &top, permissions, &offset, device, &inode, name);
>      > -      // the last entry 'name' is empty for some entries, so we might have 6 matches instead of 7 for some lines
>      > -      if (matches < 6) continue;
>      > -      if (matches == 6) name[0] = '\0';
>      > +      // Parse fields from line, discard perms, offset and device
>      > +      int matches = sscanf(line, UINT64_FORMAT_X "-" UINT64_FORMAT_X " %*s %*s %*s " INT64_FORMAT " %s",
>      > +             &base, &top, &inode, name);
>      > +      // the last entry 'name' is empty for some entries, so we might have 3 matches instead of 4 for some lines
>      > +      if (matches < 3) continue;
>      > +      if (matches == 3) name[0] = '\0';
>      >
>      > -      // Filter by device id '00:00' so that we only get file system mapped files.
>      > -      if (strcmp(device, "00:00") != 0) {
>      > +      // Filter by inode 0 so that we only get file system mapped files.
>      > +      if (inode != 0) {
>      >
>      >           // Call callback with the fields of interest
>      >           if(callback(name, (address)base, (address)top, param)) {
>      >
>      >
>      > Thanks,
>      > Denghui Dong
>      >
>      >
>      >
>      > ------------------------------------------------------------------
>      > From:???(??) <denghui.ddh at alibaba-inc.com>
>      > Send Time:2020?3?10?(???) 14:16
>      > To:hotspot-jfr-dev <hotspot-jfr-dev at openjdk.java.net>
>      > Subject:Re: RFR(S): 8240360: NativeLibraryEvent has wrong library name on Linux
>      >
>      > Hi,
>      > I used a new way to fix this problem.
>      > Ignore "perms", "offset" and "device" fields by "%*s" so that we don't need to be concerned about the length of the device,
>      > and use "inode" instead of "device" for filtering.
>      >
>      > Webrev: http://cr.openjdk.java.net/~ddong/8240360/webrev.01/
>      >
>      > Could you review it?
>      >
>      > Thanks,
>      > Denghui Dong
>      > ------------------------------------------------------------------
>      > From:???(??) <denghui.ddh at alibaba-inc.com>
>      > Send Time:2020?3?4?(???) 12:05
>      > To:hotspot-jfr-dev <hotspot-jfr-dev at openjdk.java.net>
>      > Subject:Re: RFR(S): 8240360: NativeLibraryEvent has wrong library name on Linux
>      >
>      > Testing: jdk/jfr all passed.
>      > ------------------------------------------------------------------
>      > From:???(??) <denghui.ddh at alibaba-inc.com>
>      > Send Time:2020?3?3?(???) 22:25
>      > To:hotspot-jfr-dev <hotspot-jfr-dev at openjdk.java.net>
>      > Subject:RFR(S): 8240360: NativeLibraryEvent has wrong library name on Linux
>      >
>      > Hi team,
>      >
>      > I found TestNativeLibrariesEvent is always failed in my Linux environment.
>      >
>      > The root cause is that os::get_loaded_modules_info in os_linux.cpp can not read library name correctly,
>      > because the device number in my Linux environment is 103:01, the format passed to sscanf can't identify
>      > it.
>      >
>      > I found the limit of major and minor device number in Linux from
>      > https://github.com/torvalds/linux/blob/master/include/linux/kdev_t.h, and made this patch.
>      >
>      > Please review this small change that fixes this problem.
>      >
>      > Bug: https://bugs.openjdk.java.net/browse/JDK-8240360
>      > Webrev: http://cr.openjdk.java.net/~ddong/8240360/webrev.00/
>      >
>      >
>      > Thanks
>      > Denghui Dong
>      > 
> 

From matthias.baesken at sap.com  Wed Apr  1 06:44:28 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 1 Apr 2020 06:44:28 +0000
Subject: RFR [XXS] : 8241948: enhance list of environment variables printed in
 hs_err file
Message-ID: <AM0PR02MB4227B7BAB12015D6E43FE19693C90@AM0PR02MB4227.eurprd02.prod.outlook.com>

Hello, please review this small change .
It enhances the list of environment variables printed in hs_err file  by locale / timezone related
variables. They are often of interest in support cases.

Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8241948

http://cr.openjdk.java.net/~mbaesken/webrevs/8241948.0/

Thanks, Matthias



From stumon01 at arm.com  Wed Apr  1 09:29:02 2020
From: stumon01 at arm.com (Stuart Monteith)
Date: Wed, 1 Apr 2020 10:29:02 +0100
Subject: RFR(S): 8241587: Aarch64: remove x86 specifics from
 os_linux.cpp/hpp/inline.hpp
Message-ID: <9be2bcfe-faa4-1dc0-53fe-989c962f0ad7@arm.com>

Hello,
        This patch removes a couple of x86 specifics from aarch64 code. Tested with hotspot tier1.

Webrev:
        http://cr.openjdk.java.net/~smonteith/8241587/webrev.0/
Bug:
        https://bugs.openjdk.java.net/browse/JDK-8241587

Thanks,
        Stuart

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

From david.holmes at oracle.com  Wed Apr  1 09:47:28 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 1 Apr 2020 19:47:28 +1000
Subject: RFR [XXS] : 8241948: enhance list of environment variables
 printed in hs_err file
In-Reply-To: <AM0PR02MB4227B7BAB12015D6E43FE19693C90@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB4227B7BAB12015D6E43FE19693C90@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <83c8b1fc-50fc-db6d-6c75-98a7a1962ffe@oracle.com>

Hi Matthias,

On 1/04/2020 4:44 pm, Baesken, Matthias wrote:
> Hello, please review this small change .
> It enhances the list of environment variables printed in hs_err file  by locale / timezone related
> variables. They are often of interest in support cases.
> 
> Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8241948
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8241948.0/

That seems harmless enough :)

Those lists need some additional TLC:


   93   // defined on Linux
   94   "LD_ASSUME_KERNEL", "_JAVA_SR_SIGNUM",

I think we can dispense with LD_ASSUME_KERNEL, and _JAVA_SR_SIGNUM is 
used on Linux, BSD and AIX. :) Not for this change of course.

Cheers,
David

> Thanks, Matthias
> 
> 

From david.holmes at oracle.com  Wed Apr  1 10:03:32 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 1 Apr 2020 20:03:32 +1000
Subject: RFR(S): 8241587: Aarch64: remove x86 specifics from
 os_linux.cpp/hpp/inline.hpp
In-Reply-To: <9be2bcfe-faa4-1dc0-53fe-989c962f0ad7@arm.com>
References: <9be2bcfe-faa4-1dc0-53fe-989c962f0ad7@arm.com>
Message-ID: <4c10a542-f629-3a37-3d11-8809d70ebeea@oracle.com>

Hi Stuart,

On 1/04/2020 7:29 pm, Stuart Monteith wrote:
> Hello,
>          This patch removes a couple of x86 specifics from aarch64 code. Tested with hotspot tier1.
> 
> Webrev:
>          http://cr.openjdk.java.net/~smonteith/8241587/webrev.0/
> Bug:
>          https://bugs.openjdk.java.net/browse/JDK-8241587

That clean up seems good to me.

> Thanks,
>          Stuart
> 
> IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

That footer seems inappropriate for OpenJDK emails.

Cheers,
David


From stumon01 at arm.com  Wed Apr  1 10:09:40 2020
From: stumon01 at arm.com (Stuart Monteith)
Date: Wed, 1 Apr 2020 11:09:40 +0100
Subject: RFR(S): 8241587: Aarch64: remove x86 specifics from
 os_linux.cpp/hpp/inline.hpp
In-Reply-To: <4c10a542-f629-3a37-3d11-8809d70ebeea@oracle.com>
References: <9be2bcfe-faa4-1dc0-53fe-989c962f0ad7@arm.com>
 <4c10a542-f629-3a37-3d11-8809d70ebeea@oracle.com>
Message-ID: <a2b0e31b-ad4b-3bf4-91fa-4b9d285d32f9@arm.com>

On 01/04/2020 11:03, David Holmes wrote:
> Hi Stuart,
>
> On 1/04/2020 7:29 pm, Stuart Monteith wrote:
>> Hello,
>>          This patch removes a couple of x86 specifics from aarch64 code. Tested with hotspot tier1.
>>
>> Webrev:
>>          http://cr.openjdk.java.net/~smonteith/8241587/webrev.0/
>> Bug:
>>          https://bugs.openjdk.java.net/browse/JDK-8241587
>
> That clean up seems good to me.
>

Thanks.

>> Thanks,
>>          Stuart
>>
>> IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you
>> are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other
>> person, use it for any purpose, or store or copy the information in any medium. Thank you.
>
> That footer seems inappropriate for OpenJDK emails.
>

Apologies - dismiss that. I'd ordinarily send the email from my other machine.

> Cheers,
> David
>

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

From matthias.baesken at sap.com  Wed Apr  1 11:10:04 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 1 Apr 2020 11:10:04 +0000
Subject: RFR [XXS] : 8241948: enhance list of environment variables
 printed in hs_err file
In-Reply-To: <83c8b1fc-50fc-db6d-6c75-98a7a1962ffe@oracle.com>
References: <AM0PR02MB4227B7BAB12015D6E43FE19693C90@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <83c8b1fc-50fc-db6d-6c75-98a7a1962ffe@oracle.com>
Message-ID: <AM0PR02MB42271D64E1E7ED3FF6B9498E93C90@AM0PR02MB4227.eurprd02.prod.outlook.com>

Hi David, thanks for the review .

>I think we can dispense with LD_ASSUME_KERNEL, and _JAVA_SR_SIGNUM is 
>used on Linux, BSD and AIX. :) Not for this change of course.

Should I open another JBS issue for this ?

Best regards, Matthias

-----Original Message-----
From: David Holmes <david.holmes at oracle.com> 
Sent: Mittwoch, 1. April 2020 11:47
To: Baesken, Matthias <matthias.baesken at sap.com>; 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
Subject: Re: RFR [XXS] : 8241948: enhance list of environment variables printed in hs_err file

Hi Matthias,

On 1/04/2020 4:44 pm, Baesken, Matthias wrote:
> Hello, please review this small change .
> It enhances the list of environment variables printed in hs_err file  by locale / timezone related
> variables. They are often of interest in support cases.
> 
> Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8241948
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8241948.0/

That seems harmless enough :)

Those lists need some additional TLC:


   93   // defined on Linux
   94   "LD_ASSUME_KERNEL", "_JAVA_SR_SIGNUM",

I think we can dispense with LD_ASSUME_KERNEL, and _JAVA_SR_SIGNUM is 
used on Linux, BSD and AIX. :) Not for this change of course.

Cheers,
David

> Thanks, Matthias
> 
> 

From harold.seigel at oracle.com  Wed Apr  1 13:18:00 2020
From: harold.seigel at oracle.com (Harold Seigel)
Date: Wed, 1 Apr 2020 09:18:00 -0400
Subject: RFR [XXS] : 8241948: enhance list of environment variables
 printed in hs_err file
In-Reply-To: <AM0PR02MB4227B7BAB12015D6E43FE19693C90@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB4227B7BAB12015D6E43FE19693C90@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <f229aba0-7e83-cc2f-2817-a2e4ca334332@oracle.com>

Looks good.

Thanks, Harold

On 4/1/2020 2:44 AM, Baesken, Matthias wrote:
> Hello, please review this small change .
> It enhances the list of environment variables printed in hs_err file  by locale / timezone related
> variables. They are often of interest in support cases.
>
> Bug/webrev :
>
> https://bugs.openjdk.java.net/browse/JDK-8241948
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8241948.0/
>
> Thanks, Matthias
>
>

From david.holmes at oracle.com  Wed Apr  1 13:23:53 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 1 Apr 2020 23:23:53 +1000
Subject: RFR [XXS] : 8241948: enhance list of environment variables
 printed in hs_err file
In-Reply-To: <AM0PR02MB42271D64E1E7ED3FF6B9498E93C90@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB4227B7BAB12015D6E43FE19693C90@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <83c8b1fc-50fc-db6d-6c75-98a7a1962ffe@oracle.com>
 <AM0PR02MB42271D64E1E7ED3FF6B9498E93C90@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <aed6af65-e3f9-0955-3087-7a220ecb67b5@oracle.com>

On 1/04/2020 9:10 pm, Baesken, Matthias wrote:
> Hi David, thanks for the review .
> 
>> I think we can dispense with LD_ASSUME_KERNEL, and _JAVA_SR_SIGNUM is
>> used on Linux, BSD and AIX. :) Not for this change of course.
> 
> Should I open another JBS issue for this ?

Sure :)

Thanks,
David

> Best regards, Matthias
> 
> -----Original Message-----
> From: David Holmes <david.holmes at oracle.com>
> Sent: Mittwoch, 1. April 2020 11:47
> To: Baesken, Matthias <matthias.baesken at sap.com>; 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: Re: RFR [XXS] : 8241948: enhance list of environment variables printed in hs_err file
> 
> Hi Matthias,
> 
> On 1/04/2020 4:44 pm, Baesken, Matthias wrote:
>> Hello, please review this small change .
>> It enhances the list of environment variables printed in hs_err file  by locale / timezone related
>> variables. They are often of interest in support cases.
>>
>> Bug/webrev :
>>
>> https://bugs.openjdk.java.net/browse/JDK-8241948
>>
>> http://cr.openjdk.java.net/~mbaesken/webrevs/8241948.0/
> 
> That seems harmless enough :)
> 
> Those lists need some additional TLC:
> 
> 
>     93   // defined on Linux
>     94   "LD_ASSUME_KERNEL", "_JAVA_SR_SIGNUM",
> 
> I think we can dispense with LD_ASSUME_KERNEL, and _JAVA_SR_SIGNUM is
> used on Linux, BSD and AIX. :) Not for this change of course.
> 
> Cheers,
> David
> 
>> Thanks, Matthias
>>
>>

From matthias.baesken at sap.com  Wed Apr  1 13:35:12 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 1 Apr 2020 13:35:12 +0000
Subject: RFR: 8241996: on linux set full relro in the linker flags
Message-ID: <AM0PR02MB4227D7F68410C965BE9620FD93C90@AM0PR02MB4227.eurprd02.prod.outlook.com>

Hello, please review this binary hardening related change.

To improve binary hardening, we should enable full relro in the OpenJDK builds. Currently
our build settings enable only partial relro (they miss z,now).
See https://www.redhat.com/en/blog/hardening-elf-binaries-using-relocation-read-only-relro

"Both partial and full RELRO reorder the ELF internal data sections to protect them from being overwritten in the event of a buffer-overflow,
but only full RELRO mitigates the above mentioned popular technique of overwriting the GOT entry to get control of program execution."

See also :
https://wiki.debian.org/Hardening

Some documentations/blogs mention slight performance impact of full relro (for startup performance) .

My quick checks on an example Linux server show not much impact (checked on linux x86_64) .
1)time on a   java HelloWorld  varies   (for both a patched and  unpatched  JDK)    between 0,6 and 0,7 seconds  ;
2) perf - runs on a java HelloWorld   show  a bit less  cycles (not clear why) but more  instructions :


> "normal  JVM" :

>         185,085,660      cycles                    #    2.424 GHz                      ( +-  0.54% )  (83.18%)

>         128,415,594      stalled-cycles-frontend   #   69.38% frontend cycles idle     ( +-  0.80% )  (80.98%)

>          84,990,433      stalled-cycles-backend    #   45.92% backend  cycles idle     ( +-  1.78% )  (65.38%)

>         102,950,894      instructions              #    0.56  insns per cycle

>                                                    #    1.25  stalled cycles per insn  ( +-  1.48% )  (86.90%)

>

> Changed JVM with z,now  set :

>

>         182,514,813      cycles                    #    2.394 GHz                      ( +-  0.58% )  (80.14%)

>         126,879,112      stalled-cycles-frontend   #   69.52% frontend cycles idle     ( +-  0.81% )  (81.24%)

>          82,691,295      stalled-cycles-backend    #   45.31% backend  cycles idle     ( +-  1.72% )  (69.16%)

>         103,958,399      instructions              #    0.57  insns per cycle

>                                                    #    1.22  stalled cycles per insn  ( +-  1.21% )  (89.47%)


Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8241996

http://cr.openjdk.java.net/~mbaesken/webrevs/8241996.0/


Best regards, Matthias

From claes.redestad at oracle.com  Wed Apr  1 13:53:45 2020
From: claes.redestad at oracle.com (Claes Redestad)
Date: Wed, 1 Apr 2020 15:53:45 +0200
Subject: RFR: 8241996: on linux set full relro in the linker flags
In-Reply-To: <AM0PR02MB4227D7F68410C965BE9620FD93C90@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB4227D7F68410C965BE9620FD93C90@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <fe5f654b-4192-f523-08a5-3c05f81addd0@oracle.com>

Hi,

I took this for a spin on my setup, and see a ~500k increase in both
instructions and cycles - I guess one of your runs saw more
interference than the other.

 From a startup performance point of view the advertised cost seems
more or less negligible. (For context we've optimized away 10M
instructions since JDK 14 GA - a 0.5M step in the wrong direction won't
be noticeable)

/Claes

On 2020-04-01 15:35, Baesken, Matthias wrote:
> Hello, please review this binary hardening related change.
> 
> To improve binary hardening, we should enable full relro in the OpenJDK 
> builds. Currently
> our build settings enable only partial relro (they miss z,now).
> See 
> https://www.redhat.com/en/blog/hardening-elf-binaries-using-relocation-read-only-relro
> 
> "Both partial and full RELRO reorder the ELF internal data sections to 
> protect them from being overwritten in the event of a buffer-overflow,
> but only full RELRO mitigates the above mentioned popular technique of 
> overwriting the GOT entry to get control of program execution."
> 
> See also :
> https://wiki.debian.org/Hardening
> 
> Some documentations/blogs mention slight performance impact of full 
> relro (for startup performance) .
> 
> 
> My quick checks on an example Linux server show not much impact (checked 
> on linux x86_64) .
> 
> 1)time on a?? java HelloWorld? varies ??(for both a patched and  
> unpatched? JDK)?? ?between 0,6 and 0,7 seconds ?;
> 
> 2) perf - runs on a java HelloWorld ??show? a bit less? cycles (not 
> clear why) but more? instructions :
> 
>> "normal? JVM" :
> 
>>???????? 185,085,660????? cycles??????????????????? #??? 2.424 GHz???????????? ?????????( +-? 0.54% )? (83.18%)
> 
>>???????? 128,415,594????? stalled-cycles-frontend?? #?? 69.38% frontend cycles idle???? ( +-? 0.80% )? (80.98%)
> 
>>????????? 84,990,433????? stalled-cycles-backend??? #?? 45.92% backend? cycles idle???? ( +-? 1.78% )? (65.38%)
> 
>>???????? 102,950,894????? instructions????????????? #??? 0.56? insns per cycle
> 
>>??????????????????????????????????????????????????? #??? 1.25? stalled cycles per insn? ( +-? 1.48% )? (86.90%)
> 
>> 
> 
>> Changed JVM with z,now ?set :
> 
>> 
> 
>>???????? 182,514,813????? cycles??????????????????? #??? 2.394 GHz????????????????????? ( +-? 0.58% )? (80.14%)
> 
>>???????? 126,879,112????? stalled-cycles-frontend?? #?? 69.52% frontend cycles idle???? ( +-? 0.81% )? (81.24%)
> 
>>????????? 82,691,295????? stalled-cycles-backend??? #?? 45.31% backend? cycles idle???? ( +-? 1.72% )? (69.16%)
> 
>>???????? 103,958,399????? instructions????????????? #??? 0.57? insns per cycle
> 
>>??????????????????????????????????????????????????? #??? 1.22? stalled cycles per insn? ( +-? 1.21% )? (89.47%)
> 
> Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8241996
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8241996.0/
> 
> Best regards, Matthias
> 

From Alan.Bateman at oracle.com  Wed Apr  1 14:26:08 2020
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Wed, 1 Apr 2020 15:26:08 +0100
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <cfe9c4ae-f70e-6408-9d8b-70f02158ff3a@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <940c6907-612e-8744-376c-5362991d4a42@oracle.com>
 <cfe9c4ae-f70e-6408-9d8b-70f02158ff3a@oracle.com>
Message-ID: <cad4ee20-f009-46d9-0ab2-5c68e595dbf1@oracle.com>



On 31/03/2020 20:25, Mandy Chung wrote:
> Alex's feedback:? rename isHiddenClass to isHidden as it can be a 
> hidden class or interface.
>
> `isLocalClass` and `sAnonymousClass` are correct because the Java 
> language only has local classes and anon classes, not local interfaces 
> or anon. interfaces.? `isHidden` is like `isSynthetic`, it could be a 
> class or interface.
>
> Although isHiddenClass seems clearer, I'm okay to rename it to 
> `isHidden`.
I went through the java.base changes in webrev.03-delta. The rename to 
isHidden and the updated javadoc looks good. There are a few additional 
drive-by updates to the javadoc, including isSynthetic, they all look 
good too.

Maybe I missed it going by, but why is the isHidden check in the field 
offset methods on sun.misc.Unsafe rather than the internal Unsafe?

-Alan.

From erik.joelsson at oracle.com  Wed Apr  1 14:33:22 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Wed, 1 Apr 2020 07:33:22 -0700
Subject: RFR: 8241996: on linux set full relro in the linker flags
In-Reply-To: <AM0PR02MB4227D7F68410C965BE9620FD93C90@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB4227D7F68410C965BE9620FD93C90@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <1d2fcac0-2ab8-c60e-4fda-d38a11b49b27@oracle.com>

Hello Matthias,

We are currently setting -z now for slowdebug builds. That should be 
removed if it's now set by default for all configs.

/Erik

On 2020-04-01 06:35, Baesken, Matthias wrote:
> Hello, please review this binary hardening related change.
>
> To improve binary hardening, we should enable full relro in the OpenJDK builds. Currently
> our build settings enable only partial relro (they miss z,now).
> See https://www.redhat.com/en/blog/hardening-elf-binaries-using-relocation-read-only-relro
>
> "Both partial and full RELRO reorder the ELF internal data sections to protect them from being overwritten in the event of a buffer-overflow,
> but only full RELRO mitigates the above mentioned popular technique of overwriting the GOT entry to get control of program execution."
>
> See also :
> https://wiki.debian.org/Hardening
>
> Some documentations/blogs mention slight performance impact of full relro (for startup performance) .
>
> My quick checks on an example Linux server show not much impact (checked on linux x86_64) .
> 1)time on a   java HelloWorld  varies   (for both a patched and  unpatched  JDK)    between 0,6 and 0,7 seconds  ;
> 2) perf - runs on a java HelloWorld   show  a bit less  cycles (not clear why) but more  instructions :
>
>
>> "normal  JVM" :
>>          185,085,660      cycles                    #    2.424 GHz                      ( +-  0.54% )  (83.18%)
>>          128,415,594      stalled-cycles-frontend   #   69.38% frontend cycles idle     ( +-  0.80% )  (80.98%)
>>           84,990,433      stalled-cycles-backend    #   45.92% backend  cycles idle     ( +-  1.78% )  (65.38%)
>>          102,950,894      instructions              #    0.56  insns per cycle
>>                                                     #    1.25  stalled cycles per insn  ( +-  1.48% )  (86.90%)
>> Changed JVM with z,now  set :
>>          182,514,813      cycles                    #    2.394 GHz                      ( +-  0.58% )  (80.14%)
>>          126,879,112      stalled-cycles-frontend   #   69.52% frontend cycles idle     ( +-  0.81% )  (81.24%)
>>           82,691,295      stalled-cycles-backend    #   45.31% backend  cycles idle     ( +-  1.72% )  (69.16%)
>>          103,958,399      instructions              #    0.57  insns per cycle
>>                                                     #    1.22  stalled cycles per insn  ( +-  1.21% )  (89.47%)
>
> Bug/webrev :
>
> https://bugs.openjdk.java.net/browse/JDK-8241996
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8241996.0/
>
>
> Best regards, Matthias

From mandy.chung at oracle.com  Wed Apr  1 17:42:57 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Wed, 1 Apr 2020 10:42:57 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <cad4ee20-f009-46d9-0ab2-5c68e595dbf1@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <940c6907-612e-8744-376c-5362991d4a42@oracle.com>
 <cfe9c4ae-f70e-6408-9d8b-70f02158ff3a@oracle.com>
 <cad4ee20-f009-46d9-0ab2-5c68e595dbf1@oracle.com>
Message-ID: <d2d01595-2665-57ce-aef4-8c1d6af30921@oracle.com>



On 4/1/20 7:26 AM, Alan Bateman wrote:
> Maybe I missed it going by, but why is the isHidden check in the field 
> offset methods on sun.misc.Unsafe rather than the internal Unsafe? 

The reflection and VarHandle implementation uses 
jdk.internal.misc.Unsafe to do field access (both read and write). For 
fields of a hidden declaring class, Field::get works on final and 
non-final fields.? Field::set will work on non-final fields and throw 
IAE on final fields.?? That's why no change to the internal Unsafe to 
support reflective field access.

The check in the field offset methods in `sun.misc.Unsafe` intends to 
constrain the existing use of jdk.unsupported unsafe field offset 
methods to existing classes but not hidden classes (perhaps same to 
apply to inline classes) moving toward "trusted non-instance finals".

Mandy

From igor.ignatyev at oracle.com  Wed Apr  1 19:13:09 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Wed, 1 Apr 2020 12:13:09 -0700
Subject: RFR(XS): 8174768: Make ProcessTools print executed process output
 into a separate file
In-Reply-To: <ec26b257-4518-419e-83a3-b64f920f766b@default>
References: <ec26b257-4518-419e-83a3-b64f920f766b@default>
Message-ID: <C980DAFB-8F75-4D8A-A0BB-B3227D01F54D@oracle.com>

Hi Evgeny,

(widening the audience, given this affects not just hotspot compiler, but hotspot tests as well as core libs tests in general)

overall that looks good to me. one suggestion, for the ease of failure analysis it might be worth to print out the names of created files, although this might potentially clutter the output, I don't think it'll be a problem given we already print out things like 'Gathering output for process ...' , 'Waiting for completion...' in LazyOutputBuffer.

> The change has been tested via a mach5 test runs (jdk-tier1 through 4) on the 4 common platforms (linux-x64, windows-x64, macosx-x64, sparcv9). 
this doesn't include any of hotspot tiers, could you please also run hs-tier1--4? 
// you can use tierN jobs which include both jdk and hs parts.

Thanks,
-- Igor

> On Mar 30, 2020, at 3:55 AM, Evgeny Nikitin <evgeny.nikitin at oracle.com> wrote:
> 
> 
> Hi, 
> 
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8174768 
> 
> Webrev: http://cr.openjdk.java.net/~iignatyev/enikitin/8174768/webrev.00/ 
> 
> 
> The bug had been created as a request to simplify investigation for compiler control tests failures. 
> I found the functionality pretty generic and useful and made ProcessTools dump output as well as some diagnostic information for every executed process into a separate file. 
> The diagnostic information contains cmdline, exit code, stdout and stderr. The output files are named like 'pid-<PID>-output.log'. 
> 
> The change has been tested via a mach5 test runs (jdk-tier1 through 4) on the 4 common platforms (linux-x64, windows-x64, macosx-x64, sparcv9). 
> 
> Please review, 
> /Evgeny Nikitin. 


From paul.sandoz at oracle.com  Wed Apr  1 22:46:55 2020
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Wed, 1 Apr 2020 15:46:55 -0700
Subject: RFR JDK-8223347 Integration of Vector API (Incubator): Java API,
 implementation, and tests
Message-ID: <49F81016-55DF-4E3C-940A-A8DF8C712259@oracle.com>

Hi,

A prior email sent out a request for review of the Vector API in preparation for JEP 338: Vector API (Incubator) [1] to be proposed for target:

  https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065345.html <https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065345.html>

This email expands the review of the API to the Java implementation and Java tests:

  http://cr.openjdk.java.net/~psandoz/panama/JDK-8223347-vector-api-integration-java/jdk_src_webrev/webrev/ <http://cr.openjdk.java.net/~psandoz/panama/JDK-8223347-vector-api-integration-java/jdk_src_webrev/webrev/>

  http://cr.openjdk.java.net/~psandoz/panama/JDK-8223347-vector-api-integration-java/jdk_test_webrev/webrev/ <http://cr.openjdk.java.net/~psandoz/panama/JDK-8223347-vector-api-integration-java/jdk_test_webrev/webrev/>

(Further emails will sent for review of general HotSpot changes CPU architecture specific HotSpot changes, x64 and aarch64, and performance tests.  For an early peek see the links in the description of JDK-8223347.)

?

The Vector API provides

- the public Vector class with vector operations common to all supported primitive types

- public primitive specializations, such as IntVector, with common operations associated with the primitive type

- internal concrete specializations for the vector size, such as Int256Vector, for a vector holding 8 ints.

Operations generally defer to one of approximately 20 vector intrinsics.  Some operations may be composed of other operations.

Explicit casts are performed by vector operations to ensure vectors arguments are of the required shape (bit size) and element type.

A vector intrinsic is an internal low-level vector operation. The last argument to the intrinsic is fall back behavior in Java, implementing the scalar operation over the number of elements held by the vector.  Thus, If the intrinsic is not supported in C2 for the other arguments then the Java implementation is executed (the Java implementation is always executed when running in the interpreter or for C1).

The public primitive specializations and the internal concrete implementations are generated from SSP template files, X-Vector.java.template and X-VectorBits.java.template respectively. 

Overall the implementation approach is quite formulaic and rather repetitive.  Once you grok the pattern It should be easier to review if a little laborious.

Unit tests are auto-generated by composing templates for each operation into an SSP template file which is then used to generate unit test files for each concrete vector class.  The tests are quite extensive and have found many a HotSpot issue in development across a wide range of platforms.

Paul.

[1] https://bugs.openjdk.java.net/browse/JDK-8201271 <https://bugs.openjdk.java.net/browse/JDK-8201271> 


From ioi.lam at oracle.com  Thu Apr  2 00:00:21 2020
From: ioi.lam at oracle.com (Ioi Lam)
Date: Wed, 1 Apr 2020 17:00:21 -0700
Subject: RFR(XS): 8174768: Make ProcessTools print executed process output
 into a separate file
In-Reply-To: <C980DAFB-8F75-4D8A-A0BB-B3227D01F54D@oracle.com>
References: <ec26b257-4518-419e-83a3-b64f920f766b@default>
 <C980DAFB-8F75-4D8A-A0BB-B3227D01F54D@oracle.com>
Message-ID: <3bbe30fd-aae0-f55f-15f4-6a92ef918617@oracle.com>



On 4/1/20 12:13 PM, Igor Ignatyev wrote:
> Hi Evgeny,
>
> (widening the audience, given this affects not just hotspot compiler, but hotspot tests as well as core libs tests in general)
>
> overall that looks good to me. one suggestion, for the ease of failure analysis it might be worth to print out the names of created files, although this might potentially clutter the output, I don't think it'll be a problem given we already print out things like 'Gathering output for process ...' , 'Waiting for completion...' in LazyOutputBuffer.
>

FYI,

We've been doing a similar thing with all the CDS tests -- all the logs 
from ProcessTools are saved, and we print out the name of stdout/stderr 
files in the .jtr files. It's been very valuable in diagnosing failures.

Command line: [/home/iklam/jdk/bld/fre-fastdebug/images/jdk/bin/java -cp 
/jdk2/tmp/jtreg/work/classes/13/runtime/cds/appcds/HelloTest.d:/jdk2/fre/open/test/hotspot/jtreg/runtime/cds/appcds:/jdk2/tmp/jtreg/work/classes/13/test/lib:/jdk/tools/jtreg/5.0-b01/lib/javatest.jar:/jdk/tools/jtreg/5.0-b01/lib/jtreg.jar 
-XX:MaxRAM=8g -cp 
/jdk2/tmp/jtreg/work/classes/13/runtime/cds/appcds/HelloTest.d/hello.jar 
-Xshare:dump -Xlog:cds 
-XX:SharedArchiveFile=/jdk2/tmp/jtreg/work/scratch/2/appcds-23h24m40s432.jsa 
-XX:ExtraSharedClassListFile=/jdk2/tmp/jtreg/work/classes/13/runtime/cds/appcds/HelloTest.d/HelloTest-test.classlist 
]
[2020-04-01T06:24:40.530164Z] Gathering output for process 22666
[ELAPSED: 3068 ms]
[logging stdout to HelloTest-0000-dump.stdout]
[logging stderr to HelloTest-0000-dump.stderr]

Thanks
- Ioi




>> The change has been tested via a mach5 test runs (jdk-tier1 through 4) on the 4 common platforms (linux-x64, windows-x64, macosx-x64, sparcv9).
> this doesn't include any of hotspot tiers, could you please also run hs-tier1--4?
> // you can use tierN jobs which include both jdk and hs parts.
>
> Thanks,
> -- Igor
>
>> On Mar 30, 2020, at 3:55 AM, Evgeny Nikitin <evgeny.nikitin at oracle.com> wrote:
>>
>>
>> Hi,
>>
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8174768
>>
>> Webrev: http://cr.openjdk.java.net/~iignatyev/enikitin/8174768/webrev.00/
>>
>>
>> The bug had been created as a request to simplify investigation for compiler control tests failures.
>> I found the functionality pretty generic and useful and made ProcessTools dump output as well as some diagnostic information for every executed process into a separate file.
>> The diagnostic information contains cmdline, exit code, stdout and stderr. The output files are named like 'pid-<PID>-output.log'.
>>
>> The change has been tested via a mach5 test runs (jdk-tier1 through 4) on the 4 common platforms (linux-x64, windows-x64, macosx-x64, sparcv9).
>>
>> Please review,
>> /Evgeny Nikitin.


From david.holmes at oracle.com  Thu Apr  2 00:07:31 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 1 Apr 2020 17:07:31 -0700 (PDT)
Subject: RFR(XS): 8174768: Make ProcessTools print executed process output
 into a separate file
In-Reply-To: <C980DAFB-8F75-4D8A-A0BB-B3227D01F54D@oracle.com>
References: <ec26b257-4518-419e-83a3-b64f920f766b@default>
 <C980DAFB-8F75-4D8A-A0BB-B3227D01F54D@oracle.com>
Message-ID: <70147008-45b8-0b7f-6691-50f8429c5369@oracle.com>

Thanks for sharing this Igor!

I'm not at all sure this is generally what we want for every single test 
that uses ProcessTools! But I'm willing it to see it trialed.

Evgeny: Please run full tier testing at least to tier 6 and ideally 
beyond before pushing this. There are potential implications for 
temporary (and more permanent) disk usage as well as additional time 
needed to write files out to disk. (Hopefully these are generally small 
enough that this doesn't make a noticeable difference.)

Thanks,
David

On 2/04/2020 5:13 am, Igor Ignatyev wrote:
> Hi Evgeny,
> 
> (widening the audience, given this affects not just hotspot compiler, but hotspot tests as well as core libs tests in general)
> 
> overall that looks good to me. one suggestion, for the ease of failure analysis it might be worth to print out the names of created files, although this might potentially clutter the output, I don't think it'll be a problem given we already print out things like 'Gathering output for process ...' , 'Waiting for completion...' in LazyOutputBuffer.
> 
>> The change has been tested via a mach5 test runs (jdk-tier1 through 4) on the 4 common platforms (linux-x64, windows-x64, macosx-x64, sparcv9).
> this doesn't include any of hotspot tiers, could you please also run hs-tier1--4?
> // you can use tierN jobs which include both jdk and hs parts.
> 
> Thanks,
> -- Igor
> 
>> On Mar 30, 2020, at 3:55 AM, Evgeny Nikitin <evgeny.nikitin at oracle.com> wrote:
>>
>>
>> Hi,
>>
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8174768
>>
>> Webrev: http://cr.openjdk.java.net/~iignatyev/enikitin/8174768/webrev.00/
>>
>>
>> The bug had been created as a request to simplify investigation for compiler control tests failures.
>> I found the functionality pretty generic and useful and made ProcessTools dump output as well as some diagnostic information for every executed process into a separate file.
>> The diagnostic information contains cmdline, exit code, stdout and stderr. The output files are named like 'pid-<PID>-output.log'.
>>
>> The change has been tested via a mach5 test runs (jdk-tier1 through 4) on the 4 common platforms (linux-x64, windows-x64, macosx-x64, sparcv9).
>>
>> Please review,
>> /Evgeny Nikitin.
> 

From david.holmes at oracle.com  Thu Apr  2 04:52:54 2020
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 2 Apr 2020 14:52:54 +1000
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <1a5eb022-dd7e-b0bb-750f-c43be195eb42@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <1a5eb022-dd7e-b0bb-750f-c43be195eb42@oracle.com>
Message-ID: <319b5917-397f-5ac1-9c94-56f558653797@oracle.com>

Hi Mandy,

On 1/04/2020 1:01 pm, Mandy Chung wrote:
> Thanks to the review feedbacks.
> 
> Updated webrev:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.04/

I've had a good look through all the hotspot related changes. All looks 
good.

A few minor comments:

src/hotspot/share/ci/ciField.cpp

    // Trust VM hidden and unsafe anonymous classes.

should say

    // Trust hidden and VM unsafe anonymous classes.


   // the private API (jdk.internal.misc.Unsafe)

s/the/a/  or else change the () to commas or perhaps even better:

   // the private jdk.internal.misc.Unsafe API,

---

src/hotspot/share/ci/ciInstanceKlass.cpp

   // VM weak hidden and unsafe anonymous classes

should say

   // weak hidden and VM unsafe anonymous classes

---

src/hotspot/share/classfile/classLoaderData.hpp

!  // the CLDs lifecycle.  For example, a non-strong hidden class or an
...
!  // Used for weak hidden classes, unsafe anonymous classes and the

There is an inconsistency in terminology, so far most code comments 
refer to "weak hidden" but now you are using "non-strong hidden".

! for an weak hidden

s/an/a/   multiple locations

---

src/hotspot/share/interpreter/linkResolver.cpp

Can you fix one of my comments please (in two places) :)

+     // For private access check if there was a problem with nest host

would read better as:

+     // For private access see if there was a problem with nest host

---

Thanks,
David
------

> Delta between webrev.03 and webrev.04:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03-delta/
> 
> Summary of changes is:
> 1. Update javac to retain the previous behavior when compiling to target 
> release <= 14 where lambda proxy class is not a nestmate
> 2. Rename Class::isHiddenClass to Class::isHidden
> 3. Address Joe's feedback on the CSR to document the behavior of the 
> relevant methods in java.lang.Class for hidden classes
> 4. Add test case for unloadable class with nest host error
> 5. Add test cases for hidden classes with different kinds of class or 
> interface
> 6. Update dcmd to drop "weak hidden class" and refer it as "hidden class"
> 7. Small changes in response to Remi, Coleen, Paul's review comments
> 8. Fix copyright headers
> 9. Fix @modules in tests
> 
> Most of the changes above have also been reviewed as separate patches.
> 
> Thanks
> Mandy
> 
> On 3/26/20 4:57 PM, Mandy Chung wrote:
>> Please review the implementation of JEP 371: Hidden Classes. The main 
>> changes are in core-libs and hotspot runtime area.? Small changes are 
>> made in javac, VM compiler (intrinsification of Class::isHiddenClass), 
>> JFR, JDI, and jcmd.? CSR [1]has been reviewed and is in the finalized 
>> state (see specdiff and javadoc below for reference).
>>
>> Webrev:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
>>
>>
>> javadoc/specdiff
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
>>
>>
>> JVMS 5.4.4 change:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
>>
>>
>> CSR:
>> https://bugs.openjdk.java.net/browse/JDK-8238359
>>
>> Thanks
>> Mandy
>> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
>> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
>> [3] https://bugs.openjdk.java.net/browse/JDK-8230502
> 

From mandy.chung at oracle.com  Thu Apr  2 05:17:09 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Thu, 2 Apr 2020 05:17:09 +0000 (UTC)
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <319b5917-397f-5ac1-9c94-56f558653797@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <1a5eb022-dd7e-b0bb-750f-c43be195eb42@oracle.com>
 <319b5917-397f-5ac1-9c94-56f558653797@oracle.com>
Message-ID: <bf7f63a9-5969-1f9b-bb2b-a2b869333884@oracle.com>

Hi David,

Thanks for the edits to the comments.?? "weak hidden" were missed to be 
changed to "non-strong hidden".? Here is the patch fixing the comments.

diff --git a/src/hotspot/share/ci/ciField.cpp 
b/src/hotspot/share/ci/ciField.cpp
--- a/src/hotspot/share/ci/ciField.cpp
+++ b/src/hotspot/share/ci/ciField.cpp
@@ -223,8 +223,8 @@
 ?????? holder->is_in_package("jdk/internal/foreign") || 
holder->is_in_package("jdk/incubator/foreign") ||
 ?????? holder->is_in_package("java/lang"))
 ???? return true;
-? // Trust VM hidden and unsafe anonymous classes. They are created via 
Lookup.defineClass or
-? // the private API (jdk.internal.misc.Unsafe) and can't be 
serialized, so there is no hacking
+? // Trust hidden and VM unsafe anonymous classes. They are created via 
Lookup.defineClass or
+? // the private jdk.internal.misc.Unsafe API and can't be serialized, 
so there is no hacking
 ?? // of finals going on with them.
 ?? if (holder->is_hidden() || holder->is_unsafe_anonymous())
 ???? return true;
diff --git a/src/hotspot/share/ci/ciInstanceKlass.cpp 
b/src/hotspot/share/ci/ciInstanceKlass.cpp
--- a/src/hotspot/share/ci/ciInstanceKlass.cpp
+++ b/src/hotspot/share/ci/ciInstanceKlass.cpp
@@ -76,7 +76,7 @@
 ?? oop holder = ik->klass_holder();
 ?? if (ik->class_loader_data()->has_class_mirror_holder()) {
 ???? // Though ciInstanceKlass records class loader oop, it's not 
enough to keep
-??? // VM weak hidden and unsafe anonymous classes alive (loader == 
NULL). Klass holder should
+??? // non-strong hidden class and VM unsafe anonymous classes alive 
(loader == NULL). Klass holder should
 ???? // be used instead. It is enough to record a ciObject, since 
cached elements are never removed
 ???? // during ciObjectFactory lifetime. ciObjectFactory itself is 
created for
 ???? // every compilation and lives for the whole duration of the 
compilation.
diff --git a/src/hotspot/share/classfile/classLoaderData.hpp 
b/src/hotspot/share/classfile/classLoaderData.hpp
--- a/src/hotspot/share/classfile/classLoaderData.hpp
+++ b/src/hotspot/share/classfile/classLoaderData.hpp
@@ -127,9 +127,10 @@
 ?? bool _accumulated_modified_oops; // Mod Union Equivalent (CMS support)

 ?? int _keep_alive;???????? // if this CLD is kept alive.
-?????????????????????????? // Used for weak hidden classes, unsafe 
anonymous classes and the
+?????????????????????????? // Used for non-strong hidden classes, 
unsafe anonymous classes and the
 ??????????????????????????? // boot class loader. _keep_alive does not 
need to be volatile or
-?????????????????????????? // atomic since there is one unique CLD per 
weak hidden or unsafe anonymous class.
+?????????????????????????? // atomic since there is one unique CLD per 
non-strong hidden class
+?????????????????????????? // or unsafe anonymous class.

 ?? volatile int _claim; // non-zero if claimed, for example during GC 
traces.
 ??????????????????????? // To avoid applying oop closure more than once.
@@ -242,15 +243,15 @@
 ?? }

 ?? // Returns true if this class loader data is for the system class 
loader.
-? // (Note that the class loader data may be for an weak hidden or 
unsafe anonymous class)
+? // (Note that the class loader data may be for a non-strong hidden 
class or unsafe anonymous class)
 ?? bool is_system_class_loader_data() const;

 ?? // Returns true if this class loader data is for the platform class 
loader.
-? // (Note that the class loader data may be for an weak hidden or 
unsafe anonymous class)
+? // (Note that the class loader data may be for a non-strong hidden 
class or unsafe anonymous class)
 ?? bool is_platform_class_loader_data() const;

 ?? // Returns true if this class loader data is for the boot class loader.
-? // (Note that the class loader data may be for an weak hidden unsafe 
anonymous class)
+? // (Note that the class loader data may be for a non-strong hidden 
class or unsafe anonymous class)
 ?? inline bool is_boot_class_loader_data() const;

 ?? bool is_builtin_class_loader_data() const;
@@ -271,7 +272,7 @@
 ???? return _unloading;
 ?? }

-? // Used to refcount an weak hidden or unsafe anonymous class's CLD in 
order to
+? // Used to refcount a non-strong hidden class's or unsafe anonymous 
class's CLD in order to
 ?? // indicate their aliveness.
 ?? void inc_keep_alive();
 ?? void dec_keep_alive();

diff --git a/src/hotspot/share/interpreter/linkResolver.cpp 
b/src/hotspot/share/interpreter/linkResolver.cpp
--- a/src/hotspot/share/interpreter/linkResolver.cpp
+++ b/src/hotspot/share/interpreter/linkResolver.cpp
@@ -611,7 +611,7 @@
 ????????????? (same_module) ? "" : sel_klass->class_in_module_of_loader()
 ????????????? );

-??? // For private access check if there was a problem with nest host
+??? // For private access see if there was a problem with nest host
 ???? // resolution, and if so report that as part of the message.
 ???? if (sel_method->is_private()) {
 ?????? print_nest_host_error_on(&ss, ref_klass, sel_klass, THREAD);
@@ -951,7 +951,7 @@
 ????????????? (same_module) ? "" : "; ",
 ????????????? (same_module) ? "" : sel_klass->class_in_module_of_loader()
 ????????????? );
-??? // For private access check if there was a problem with nest host
+??? // For private access see if there was a problem with nest host
 ???? // resolution, and if so report that as part of the message.
 ???? if (fd.is_private()) {
 ?????? print_nest_host_error_on(&ss, ref_klass, sel_klass, THREAD);

Mandy

On 4/1/20 9:52 PM, David Holmes wrote:
> Hi Mandy,
>
> On 1/04/2020 1:01 pm, Mandy Chung wrote:
>> Thanks to the review feedbacks.
>>
>> Updated webrev:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.04/ 
>>
>
> I've had a good look through all the hotspot related changes. All 
> looks good.
>
> A few minor comments:
>
> src/hotspot/share/ci/ciField.cpp
>
> ?? // Trust VM hidden and unsafe anonymous classes.
>
> should say
>
> ?? // Trust hidden and VM unsafe anonymous classes.
>
>
> ? // the private API (jdk.internal.misc.Unsafe)
>
> s/the/a/? or else change the () to commas or perhaps even better:
>
> ? // the private jdk.internal.misc.Unsafe API,
>
> ---
>
> src/hotspot/share/ci/ciInstanceKlass.cpp
>
> ? // VM weak hidden and unsafe anonymous classes
>
> should say
>
> ? // weak hidden and VM unsafe anonymous classes
>
> ---
>
> src/hotspot/share/classfile/classLoaderData.hpp
>
> !? // the CLDs lifecycle.? For example, a non-strong hidden class or an
> ...
> !? // Used for weak hidden classes, unsafe anonymous classes and the
>
> There is an inconsistency in terminology, so far most code comments 
> refer to "weak hidden" but now you are using "non-strong hidden".
>
> ! for an weak hidden
>
> s/an/a/?? multiple locations
>
> ---
>
> src/hotspot/share/interpreter/linkResolver.cpp
>
> Can you fix one of my comments please (in two places) :)
>
> +???? // For private access check if there was a problem with nest host
>
> would read better as:
>
> +???? // For private access see if there was a problem with nest host
>
> ---
>
> Thanks,
> David
>


From david.holmes at oracle.com  Thu Apr  2 06:21:10 2020
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 2 Apr 2020 16:21:10 +1000
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <bf7f63a9-5969-1f9b-bb2b-a2b869333884@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <1a5eb022-dd7e-b0bb-750f-c43be195eb42@oracle.com>
 <319b5917-397f-5ac1-9c94-56f558653797@oracle.com>
 <bf7f63a9-5969-1f9b-bb2b-a2b869333884@oracle.com>
Message-ID: <c1cd2330-2bfe-f92d-531d-35fb64a65e25@oracle.com>

Hi Mandy,

On 2/04/2020 3:17 pm, Mandy Chung wrote:
> Hi David,
> 
> Thanks for the edits to the comments.?? "weak hidden" were missed to be 
> changed to "non-strong hidden".? Here is the patch fixing the comments.

There are 33 cases of "weak hidden" in the patch I reviewed and also 
some "hidden weak". Should they all be "non-strong hidden" now? In some 
cases it also appears in variables and associated logic ie.:

src/hotspot/share/classfile/classLoaderHierarchyDCmd.cpp

+      _hidden_weak_classes(NULL), _num_hidden_weak_classes(0),

I'm not clear how far this terminology change needs to extend ??

Otherwise patch below seems fine.

Thanks,
David
-----


> diff --git a/src/hotspot/share/ci/ciField.cpp 
> b/src/hotspot/share/ci/ciField.cpp
> --- a/src/hotspot/share/ci/ciField.cpp
> +++ b/src/hotspot/share/ci/ciField.cpp
> @@ -223,8 +223,8 @@
>  ?????? holder->is_in_package("jdk/internal/foreign") || 
> holder->is_in_package("jdk/incubator/foreign") ||
>  ?????? holder->is_in_package("java/lang"))
>  ???? return true;
> -? // Trust VM hidden and unsafe anonymous classes. They are created via 
> Lookup.defineClass or
> -? // the private API (jdk.internal.misc.Unsafe) and can't be 
> serialized, so there is no hacking
> +? // Trust hidden and VM unsafe anonymous classes. They are created via 
> Lookup.defineClass or
> +? // the private jdk.internal.misc.Unsafe API and can't be serialized, 
> so there is no hacking
>  ?? // of finals going on with them.
>  ?? if (holder->is_hidden() || holder->is_unsafe_anonymous())
>  ???? return true;
> diff --git a/src/hotspot/share/ci/ciInstanceKlass.cpp 
> b/src/hotspot/share/ci/ciInstanceKlass.cpp
> --- a/src/hotspot/share/ci/ciInstanceKlass.cpp
> +++ b/src/hotspot/share/ci/ciInstanceKlass.cpp
> @@ -76,7 +76,7 @@
>  ?? oop holder = ik->klass_holder();
>  ?? if (ik->class_loader_data()->has_class_mirror_holder()) {
>  ???? // Though ciInstanceKlass records class loader oop, it's not 
> enough to keep
> -??? // VM weak hidden and unsafe anonymous classes alive (loader == 
> NULL). Klass holder should
> +??? // non-strong hidden class and VM unsafe anonymous classes alive 
> (loader == NULL). Klass holder should
>  ???? // be used instead. It is enough to record a ciObject, since 
> cached elements are never removed
>  ???? // during ciObjectFactory lifetime. ciObjectFactory itself is 
> created for
>  ???? // every compilation and lives for the whole duration of the 
> compilation.
> diff --git a/src/hotspot/share/classfile/classLoaderData.hpp 
> b/src/hotspot/share/classfile/classLoaderData.hpp
> --- a/src/hotspot/share/classfile/classLoaderData.hpp
> +++ b/src/hotspot/share/classfile/classLoaderData.hpp
> @@ -127,9 +127,10 @@
>  ?? bool _accumulated_modified_oops; // Mod Union Equivalent (CMS support)
> 
>  ?? int _keep_alive;???????? // if this CLD is kept alive.
> -?????????????????????????? // Used for weak hidden classes, unsafe 
> anonymous classes and the
> +?????????????????????????? // Used for non-strong hidden classes, 
> unsafe anonymous classes and the
>  ??????????????????????????? // boot class loader. _keep_alive does not 
> need to be volatile or
> -?????????????????????????? // atomic since there is one unique CLD per 
> weak hidden or unsafe anonymous class.
> +?????????????????????????? // atomic since there is one unique CLD per 
> non-strong hidden class
> +?????????????????????????? // or unsafe anonymous class.
> 
>  ?? volatile int _claim; // non-zero if claimed, for example during GC 
> traces.
>  ??????????????????????? // To avoid applying oop closure more than once.
> @@ -242,15 +243,15 @@
>  ?? }
> 
>  ?? // Returns true if this class loader data is for the system class 
> loader.
> -? // (Note that the class loader data may be for an weak hidden or 
> unsafe anonymous class)
> +? // (Note that the class loader data may be for a non-strong hidden 
> class or unsafe anonymous class)
>  ?? bool is_system_class_loader_data() const;
> 
>  ?? // Returns true if this class loader data is for the platform class 
> loader.
> -? // (Note that the class loader data may be for an weak hidden or 
> unsafe anonymous class)
> +? // (Note that the class loader data may be for a non-strong hidden 
> class or unsafe anonymous class)
>  ?? bool is_platform_class_loader_data() const;
> 
>  ?? // Returns true if this class loader data is for the boot class loader.
> -? // (Note that the class loader data may be for an weak hidden unsafe 
> anonymous class)
> +? // (Note that the class loader data may be for a non-strong hidden 
> class or unsafe anonymous class)
>  ?? inline bool is_boot_class_loader_data() const;
> 
>  ?? bool is_builtin_class_loader_data() const;
> @@ -271,7 +272,7 @@
>  ???? return _unloading;
>  ?? }
> 
> -? // Used to refcount an weak hidden or unsafe anonymous class's CLD in 
> order to
> +? // Used to refcount a non-strong hidden class's or unsafe anonymous 
> class's CLD in order to
>  ?? // indicate their aliveness.
>  ?? void inc_keep_alive();
>  ?? void dec_keep_alive();
> 
> diff --git a/src/hotspot/share/interpreter/linkResolver.cpp 
> b/src/hotspot/share/interpreter/linkResolver.cpp
> --- a/src/hotspot/share/interpreter/linkResolver.cpp
> +++ b/src/hotspot/share/interpreter/linkResolver.cpp
> @@ -611,7 +611,7 @@
>  ????????????? (same_module) ? "" : sel_klass->class_in_module_of_loader()
>  ????????????? );
> 
> -??? // For private access check if there was a problem with nest host
> +??? // For private access see if there was a problem with nest host
>  ???? // resolution, and if so report that as part of the message.
>  ???? if (sel_method->is_private()) {
>  ?????? print_nest_host_error_on(&ss, ref_klass, sel_klass, THREAD);
> @@ -951,7 +951,7 @@
>  ????????????? (same_module) ? "" : "; ",
>  ????????????? (same_module) ? "" : sel_klass->class_in_module_of_loader()
>  ????????????? );
> -??? // For private access check if there was a problem with nest host
> +??? // For private access see if there was a problem with nest host
>  ???? // resolution, and if so report that as part of the message.
>  ???? if (fd.is_private()) {
>  ?????? print_nest_host_error_on(&ss, ref_klass, sel_klass, THREAD);
> 
> Mandy
> 
> On 4/1/20 9:52 PM, David Holmes wrote:
>> Hi Mandy,
>>
>> On 1/04/2020 1:01 pm, Mandy Chung wrote:
>>> Thanks to the review feedbacks.
>>>
>>> Updated webrev:
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.04/ 
>>>
>>
>> I've had a good look through all the hotspot related changes. All 
>> looks good.
>>
>> A few minor comments:
>>
>> src/hotspot/share/ci/ciField.cpp
>>
>> ?? // Trust VM hidden and unsafe anonymous classes.
>>
>> should say
>>
>> ?? // Trust hidden and VM unsafe anonymous classes.
>>
>>
>> ? // the private API (jdk.internal.misc.Unsafe)
>>
>> s/the/a/? or else change the () to commas or perhaps even better:
>>
>> ? // the private jdk.internal.misc.Unsafe API,
>>
>> ---
>>
>> src/hotspot/share/ci/ciInstanceKlass.cpp
>>
>> ? // VM weak hidden and unsafe anonymous classes
>>
>> should say
>>
>> ? // weak hidden and VM unsafe anonymous classes
>>
>> ---
>>
>> src/hotspot/share/classfile/classLoaderData.hpp
>>
>> !? // the CLDs lifecycle.? For example, a non-strong hidden class or an
>> ...
>> !? // Used for weak hidden classes, unsafe anonymous classes and the
>>
>> There is an inconsistency in terminology, so far most code comments 
>> refer to "weak hidden" but now you are using "non-strong hidden".
>>
>> ! for an weak hidden
>>
>> s/an/a/?? multiple locations
>>
>> ---
>>
>> src/hotspot/share/interpreter/linkResolver.cpp
>>
>> Can you fix one of my comments please (in two places) :)
>>
>> +???? // For private access check if there was a problem with nest host
>>
>> would read better as:
>>
>> +???? // For private access see if there was a problem with nest host
>>
>> ---
>>
>> Thanks,
>> David
>>
> 

From matthias.baesken at sap.com  Thu Apr  2 06:55:07 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Thu, 2 Apr 2020 06:55:07 +0000
Subject: RFR [XXS]: 8242000: clean up list of environment variables printed in
 hs_err file
Message-ID: <AM0PR02MB42277D8406509AD8FDF83D7593C60@AM0PR02MB4227.eurprd02.prod.outlook.com>

Hello, please review this very small cleanup change.

When discussing the environment variables printed in the hs_err file, we found that
some cleanup should be done , David  pointed out  :

>> I think we can dispense with LD_ASSUME_KERNEL, and _JAVA_SR_SIGNUM is
>> used on Linux, BSD and AIX. :) Not for this change of course.


Bug/webrev :
https://bugs.openjdk.java.net/browse/JDK-8242000

http://cr.openjdk.java.net/~mbaesken/webrevs/8242000.0/

Thanks, Matthias

From matthias.baesken at sap.com  Thu Apr  2 07:41:51 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Thu, 2 Apr 2020 07:41:51 +0000
Subject: RFR: 8241996: on linux set full relro in the linker flags
In-Reply-To: <1d2fcac0-2ab8-c60e-4fda-d38a11b49b27@oracle.com>
References: <AM0PR02MB4227D7F68410C965BE9620FD93C90@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <1d2fcac0-2ab8-c60e-4fda-d38a11b49b27@oracle.com>
Message-ID: <AM0PR02MB4227FC0F6CE46A7F61B35DD793C60@AM0PR02MB4227.eurprd02.prod.outlook.com>

Hi Erik, that's a good point .
New webrev :

http://cr.openjdk.java.net/~mbaesken/webrevs/8241996.1/


Best regards, Matthias


> Hello Matthias,
>
> We are currently setting -z now for slowdebug builds. That should be 
> removed if it's now set by default for all configs.



From thomas.stuefe at gmail.com  Thu Apr  2 08:33:54 2020
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Thu, 2 Apr 2020 10:33:54 +0200
Subject: RFR [XXS]: 8242000: clean up list of environment variables
 printed in hs_err file
In-Reply-To: <AM0PR02MB42277D8406509AD8FDF83D7593C60@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB42277D8406509AD8FDF83D7593C60@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <CAA-vtUyFzYyWPNDhBWJp6JumY7xxykra3UkDXwTZ9Ee2Fiboag@mail.gmail.com>

LGTM

..Thomas

On Thu, Apr 2, 2020 at 8:57 AM Baesken, Matthias <matthias.baesken at sap.com>
wrote:

> Hello, please review this very small cleanup change.
>
> When discussing the environment variables printed in the hs_err file, we
> found that
> some cleanup should be done , David  pointed out  :
>
> >> I think we can dispense with LD_ASSUME_KERNEL, and _JAVA_SR_SIGNUM is
> >> used on Linux, BSD and AIX. :) Not for this change of course.
>
>
> Bug/webrev :
> https://bugs.openjdk.java.net/browse/JDK-8242000
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8242000.0/
>
> Thanks, Matthias
>

From claes.redestad at oracle.com  Thu Apr  2 08:43:46 2020
From: claes.redestad at oracle.com (Claes Redestad)
Date: Thu, 2 Apr 2020 10:43:46 +0200
Subject: RFR [XXS]: 8242000: clean up list of environment variables
 printed in hs_err file
In-Reply-To: <CAA-vtUyFzYyWPNDhBWJp6JumY7xxykra3UkDXwTZ9Ee2Fiboag@mail.gmail.com>
References: <AM0PR02MB42277D8406509AD8FDF83D7593C60@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <CAA-vtUyFzYyWPNDhBWJp6JumY7xxykra3UkDXwTZ9Ee2Fiboag@mail.gmail.com>
Message-ID: <d417971f-bf9b-dc97-b0b3-b675724be629@oracle.com>

+1

/Claes

On 2020-04-02 10:33, Thomas St?fe wrote:
> LGTM
> 
> ..Thomas
> 
> On Thu, Apr 2, 2020 at 8:57 AM Baesken, Matthias <matthias.baesken at sap.com>
> wrote:
> 
>> Hello, please review this very small cleanup change.
>>
>> When discussing the environment variables printed in the hs_err file, we
>> found that
>> some cleanup should be done , David  pointed out  :
>>
>>>> I think we can dispense with LD_ASSUME_KERNEL, and _JAVA_SR_SIGNUM is
>>>> used on Linux, BSD and AIX. :) Not for this change of course.
>>
>>
>> Bug/webrev :
>> https://bugs.openjdk.java.net/browse/JDK-8242000
>>
>> http://cr.openjdk.java.net/~mbaesken/webrevs/8242000.0/
>>
>> Thanks, Matthias
>>

From matthias.baesken at sap.com  Thu Apr  2 08:47:56 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Thu, 2 Apr 2020 08:47:56 +0000
Subject: RFR [XXS]: 8242000: clean up list of environment variables
 printed in hs_err file
In-Reply-To: <d417971f-bf9b-dc97-b0b3-b675724be629@oracle.com>
References: <AM0PR02MB42277D8406509AD8FDF83D7593C60@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <CAA-vtUyFzYyWPNDhBWJp6JumY7xxykra3UkDXwTZ9Ee2Fiboag@mail.gmail.com>
 <d417971f-bf9b-dc97-b0b3-b675724be629@oracle.com>
Message-ID: <AM0PR02MB4227842BA3880E9A26B7FE1693C60@AM0PR02MB4227.eurprd02.prod.outlook.com>

Hi Claes and Thomas , thanks for the reviews !


-----Original Message-----
From: Claes Redestad <claes.redestad at oracle.com> 
Sent: Donnerstag, 2. April 2020 10:44
To: Thomas St?fe <thomas.stuefe at gmail.com>; Baesken, Matthias <matthias.baesken at sap.com>
Cc: hotspot-dev at openjdk.java.net
Subject: Re: RFR [XXS]: 8242000: clean up list of environment variables printed in hs_err file

+1

/Claes

On 2020-04-02 10:33, Thomas St?fe wrote:
> LGTM
> 
> ..Thomas
> 
> On Thu, Apr 2, 2020 at 8:57 AM Baesken, Matthias <matthias.baesken at sap.com>
> wrote:


From david.holmes at oracle.com  Thu Apr  2 09:41:01 2020
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 2 Apr 2020 19:41:01 +1000
Subject: RFR [XXS]: 8242000: clean up list of environment variables
 printed in hs_err file
In-Reply-To: <AM0PR02MB42277D8406509AD8FDF83D7593C60@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB42277D8406509AD8FDF83D7593C60@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <2be6613a-76db-0c1c-9b67-34baff18ce1a@oracle.com>

Hi Matthias,

On 2/04/2020 4:55 pm, Baesken, Matthias wrote:
> Hello, please review this very small cleanup change.
> 
> When discussing the environment variables printed in the hs_err file, we 
> found that
> some cleanup should be done , David? pointed out? :
> 
>>> I think we can dispense with LD_ASSUME_KERNEL, and _JAVA_SR_SIGNUM is
>>> used on Linux, BSD and AIX. :) Not for this change of course.
> 
> Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8242000
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8242000.0/

That fixes the two examples I already found but the intent was to do a 
more thorough examination of all the variables to see if they are still 
relevant e.g is JRE_HOME still meaningful? What does JAVA_COMPILER do? 
The intent is to report env vars that can affect the execution of the JVM.

Cheers,
David

> Thanks, Matthias
> 

From matthias.baesken at sap.com  Thu Apr  2 10:11:16 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Thu, 2 Apr 2020 10:11:16 +0000
Subject: RFR [XXS]: 8242000: clean up list of environment variables
 printed in hs_err file
In-Reply-To: <2be6613a-76db-0c1c-9b67-34baff18ce1a@oracle.com>
References: <AM0PR02MB42277D8406509AD8FDF83D7593C60@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <2be6613a-76db-0c1c-9b67-34baff18ce1a@oracle.com>
Message-ID: <AM0PR02MB4227F9B979F2A0DB37BCAEDD93C60@AM0PR02MB4227.eurprd02.prod.outlook.com>

Hi David ,

I think  "JAVA_COMPILER" is no longer needed,
 (see a remark here that it was used to configure the JIT compiler  https://www.ibm.com/support/knowledgecenter/SSGU8G_12.1.0/com.ibm.sqlr.doc/ids_sqr_280.htm 
 but I cannot  find this any more in the current jdk/jdk codebase).

Regarding JRE_HOME - I find  still some references in the source code, but not many .
Could it be that some installers / tools outside the JDK still set/use it ?

jdk/src/hotspot/share/jfr/dcmd/jfrDcmds.cpp
345 _settings("settings", "Settings file(s), e.g. profile or default. See JRE_HOME/lib/jfr", "STRING SET", false),

jdk/src/hotspot/share/utilities/vmError.cpp
83 "JAVA_HOME", "JRE_HOME", "JAVA_TOOL_OPTIONS", "_JAVA_OPTIONS", "CLASSPATH",

/openjdk-jdk/src/java.base/share/man/java.1
1813 \f[CB]JRE_HOME/lib/jfr\f[R].

So I would suggest to remove JAVA_COMPILER.  Maybe JRE_HOME too (but then also the other 2 references).


Best regards, Matthias



-----Original Message-----
From: David Holmes <david.holmes at oracle.com> 
Sent: Donnerstag, 2. April 2020 11:41
To: Baesken, Matthias <matthias.baesken at sap.com>; 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
Subject: Re: RFR [XXS]: 8242000: clean up list of environment variables printed in hs_err file

Hi Matthias,

On 2/04/2020 4:55 pm, Baesken, Matthias wrote:
> Hello, please review this very small cleanup change.
> 
> When discussing the environment variables printed in the hs_err file, we 
> found that
> some cleanup should be done , David? pointed out? :
> 
>>> I think we can dispense with LD_ASSUME_KERNEL, and _JAVA_SR_SIGNUM is
>>> used on Linux, BSD and AIX. :) Not for this change of course.
> 
> Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8242000
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8242000.0/

That fixes the two examples I already found but the intent was to do a 
more thorough examination of all the variables to see if they are still 
relevant e.g is JRE_HOME still meaningful? What does JAVA_COMPILER do? 
The intent is to report env vars that can affect the execution of the JVM.

Cheers,
David

> Thanks, Matthias
> 

From david.holmes at oracle.com  Thu Apr  2 10:18:43 2020
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 2 Apr 2020 20:18:43 +1000
Subject: RFR [XXS]: 8242000: clean up list of environment variables
 printed in hs_err file
In-Reply-To: <AM0PR02MB4227F9B979F2A0DB37BCAEDD93C60@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB42277D8406509AD8FDF83D7593C60@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <2be6613a-76db-0c1c-9b67-34baff18ce1a@oracle.com>
 <AM0PR02MB4227F9B979F2A0DB37BCAEDD93C60@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <18b11103-96f1-eee7-4d21-21826a42f758@oracle.com>

On 2/04/2020 8:11 pm, Baesken, Matthias wrote:
> Hi David ,
> 
> I think  "JAVA_COMPILER" is no longer needed,
>   (see a remark here that it was used to configure the JIT compiler  https://www.ibm.com/support/knowledgecenter/SSGU8G_12.1.0/com.ibm.sqlr.doc/ids_sqr_280.htm
>   but I cannot  find this any more in the current jdk/jdk codebase).

JAVA_COMPILER usage is ancient, back in 1.1 and 1.2 days IIRC :)

> Regarding JRE_HOME - I find  still some references in the source code, but not many .
> Could it be that some installers / tools outside the JDK still set/use it ?
> 
> jdk/src/hotspot/share/jfr/dcmd/jfrDcmds.cpp
> 345 _settings("settings", "Settings file(s), e.g. profile or default. See JRE_HOME/lib/jfr", "STRING SET", false),
> 
> jdk/src/hotspot/share/utilities/vmError.cpp
> 83 "JAVA_HOME", "JRE_HOME", "JAVA_TOOL_OPTIONS", "_JAVA_OPTIONS", "CLASSPATH",
> 
> /openjdk-jdk/src/java.base/share/man/java.1
> 1813 \f[CB]JRE_HOME/lib/jfr\f[R].

These uses are just commentary, I would suggest removing JRE_HOME and 
file a RFE for JFR to cleanup its references to JRE_HOME in jfrDcmds.cpp 
and the java command manpage.

But again these are just two more that I looked at in more depth. There 
may be others that need removing too.

Cheers,
David
-----

> So I would suggest to remove JAVA_COMPILER.  Maybe JRE_HOME too (but then also the other 2 references).
> 
> 
> Best regards, Matthias
> 
> 
> 
> -----Original Message-----
> From: David Holmes <david.holmes at oracle.com>
> Sent: Donnerstag, 2. April 2020 11:41
> To: Baesken, Matthias <matthias.baesken at sap.com>; 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: Re: RFR [XXS]: 8242000: clean up list of environment variables printed in hs_err file
> 
> Hi Matthias,
> 
> On 2/04/2020 4:55 pm, Baesken, Matthias wrote:
>> Hello, please review this very small cleanup change.
>>
>> When discussing the environment variables printed in the hs_err file, we
>> found that
>> some cleanup should be done , David? pointed out? :
>>
>>>> I think we can dispense with LD_ASSUME_KERNEL, and _JAVA_SR_SIGNUM is
>>>> used on Linux, BSD and AIX. :) Not for this change of course.
>>
>> Bug/webrev :
>>
>> https://bugs.openjdk.java.net/browse/JDK-8242000
>>
>> http://cr.openjdk.java.net/~mbaesken/webrevs/8242000.0/
> 
> That fixes the two examples I already found but the intent was to do a
> more thorough examination of all the variables to see if they are still
> relevant e.g is JRE_HOME still meaningful? What does JAVA_COMPILER do?
> The intent is to report env vars that can affect the execution of the JVM.
> 
> Cheers,
> David
> 
>> Thanks, Matthias
>>

From matthias.baesken at sap.com  Thu Apr  2 10:59:48 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Thu, 2 Apr 2020 10:59:48 +0000
Subject: RFR [XXS]: 8242000: clean up list of environment variables
 printed in hs_err file
In-Reply-To: <18b11103-96f1-eee7-4d21-21826a42f758@oracle.com>
References: <AM0PR02MB42277D8406509AD8FDF83D7593C60@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <2be6613a-76db-0c1c-9b67-34baff18ce1a@oracle.com>
 <AM0PR02MB4227F9B979F2A0DB37BCAEDD93C60@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <18b11103-96f1-eee7-4d21-21826a42f758@oracle.com>
Message-ID: <AM0PR02MB42276ADDD2FF6F723CB1A86E93C60@AM0PR02MB4227.eurprd02.prod.outlook.com>

Hi David ,

new  webrev  :

http://cr.openjdk.java.net/~mbaesken/webrevs/8242000.1/


I looked at the other env variables listed.
The ones from the "All platforms",  "AIX"  and "Windows"  sections have to stay .

I am not sure about Mac, maybe some Mac expert could comment.

In the Linux/Solaris/BSD   section ,
  "HOSTTYPE", "OSTYPE", "ARCH", "MACHTYPE",
have only limited use from what I see  ,  but you find them usually  in the shell and they describe the system  a bit more.
So I would keep  them  .

Regarding JRE_HOME I opened

https://bugs.openjdk.java.net/browse/JDK-8242034

Remove JRE_HOME references

Best regards, Matthias



-----Original Message-----
From: David Holmes <david.holmes at oracle.com> 
Sent: Donnerstag, 2. April 2020 12:19
To: Baesken, Matthias <matthias.baesken at sap.com>; 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
Subject: Re: RFR [XXS]: 8242000: clean up list of environment variables printed in hs_err file

On 2/04/2020 8:11 pm, Baesken, Matthias wrote:
> Hi David ,
> 
> I think  "JAVA_COMPILER" is no longer needed,
>   (see a remark here that it was used to configure the JIT compiler  https://www.ibm.com/support/knowledgecenter/SSGU8G_12.1.0/com.ibm.sqlr.doc/ids_sqr_280.htm
>   but I cannot  find this any more in the current jdk/jdk codebase).

JAVA_COMPILER usage is ancient, back in 1.1 and 1.2 days IIRC :)

> Regarding JRE_HOME - I find  still some references in the source code, but not many .
> Could it be that some installers / tools outside the JDK still set/use it ?
> 
> jdk/src/hotspot/share/jfr/dcmd/jfrDcmds.cpp
> 345 _settings("settings", "Settings file(s), e.g. profile or default. See JRE_HOME/lib/jfr", "STRING SET", false),
> 
> jdk/src/hotspot/share/utilities/vmError.cpp
> 83 "JAVA_HOME", "JRE_HOME", "JAVA_TOOL_OPTIONS", "_JAVA_OPTIONS", "CLASSPATH",
> 
> /openjdk-jdk/src/java.base/share/man/java.1
> 1813 \f[CB]JRE_HOME/lib/jfr\f[R].

These uses are just commentary, I would suggest removing JRE_HOME and 
file a RFE for JFR to cleanup its references to JRE_HOME in jfrDcmds.cpp 
and the java command manpage.

But again these are just two more that I looked at in more depth. There 
may be others that need removing too.

Cheers,
David
-----



From erik.joelsson at oracle.com  Thu Apr  2 13:31:19 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Thu, 2 Apr 2020 06:31:19 -0700
Subject: RFR: 8241996: on linux set full relro in the linker flags
In-Reply-To: <AM0PR02MB4227FC0F6CE46A7F61B35DD793C60@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB4227D7F68410C965BE9620FD93C90@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <1d2fcac0-2ab8-c60e-4fda-d38a11b49b27@oracle.com>
 <AM0PR02MB4227FC0F6CE46A7F61B35DD793C60@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <7c4c21b6-b339-9534-cb75-a89edfd16100@oracle.com>

Looks good to me.

/Erik

On 2020-04-02 00:41, Baesken, Matthias wrote:
> Hi Erik, that's a good point .
> New webrev :
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8241996.1/
>
>
> Best regards, Matthias
>
>
>> Hello Matthias,
>>
>> We are currently setting -z now for slowdebug builds. That should be
>> removed if it's now set by default for all configs.
>

From matthias.baesken at sap.com  Thu Apr  2 13:33:03 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Thu, 2 Apr 2020 13:33:03 +0000
Subject: RFR: 8241996: on linux set full relro in the linker flags
In-Reply-To: <7c4c21b6-b339-9534-cb75-a89edfd16100@oracle.com>
References: <AM0PR02MB4227D7F68410C965BE9620FD93C90@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <1d2fcac0-2ab8-c60e-4fda-d38a11b49b27@oracle.com>
 <AM0PR02MB4227FC0F6CE46A7F61B35DD793C60@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <7c4c21b6-b339-9534-cb75-a89edfd16100@oracle.com>
Message-ID: <AM0PR02MB4227E85AF23891DDDF4042A993C60@AM0PR02MB4227.eurprd02.prod.outlook.com>

Thanks for the review !

-----Original Message-----
From: Erik Joelsson <erik.joelsson at oracle.com> 
Sent: Donnerstag, 2. April 2020 15:31
To: Baesken, Matthias <matthias.baesken at sap.com>; 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>; 'build-dev at openjdk.java.net' <build-dev at openjdk.java.net>
Subject: Re: RFR: 8241996: on linux set full relro in the linker flags

Looks good to me.

/Erik

On 2020-04-02 00:41, Baesken, Matthias wrote:
> Hi Erik, that's a good point .
> New webrev :
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8241996.1/
>
>
> Best regards, Matthias
>
>
>> Hello Matthias,
>>
>> We are currently setting -z now for slowdebug builds. That should be
>> removed if it's now set by default for all configs.
>

From erik.osterlund at oracle.com  Thu Apr  2 16:28:51 2020
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Thu, 2 Apr 2020 18:28:51 +0200
Subject: RFR: 8240693: Sweeper should not examine dying metadata in
 is_unloading() nmethod during static call stub cleaning
Message-ID: <a067ea89-86f1-269c-fd9b-33900158b961@oracle.com>

Hi,

There is currently an unfortunate race condition between concurrent code 
cache unloading and the Sweeper
(affecting ZGC and Shenandoah).

First of all, big thanks to Nils for the initial analysis, requiring 
downloading a 90 GB core file. Ouch.
He discovered that we crashed accessing a freed metadata relocation in 
an unloaded nmethod from the sweeper.

I am fairly certain it is due to the following race condition that can 
indeed happen today:

1. GC terminates marking in a safepoint
2. GC performs concurrent code cache unlinking. In this step, inline 
caches are cleaned for is_alive() &&
 ?? !is_unloading() nmethods. The key is that it is not performed for 
is_alive() && is_unloading() nmethods.
 ?? So metadata relocations of their static/opt virtual call stubs 
remain in the nmethod, and could potentially
 ?? be dying.
3. A handshake operation is issued to separate unlinking from purging. 
The sweeper replies to the handshake.
4. GC performs concurrent code cache purging, making all nmethods 
unloaded. But just before one nmethod is unloaded...
5. The sweeper starts processing the nmethod. It is is_alive() && 
is_unloading() and the sweeper correctly
 ?? decides to clean its inline caches. As part of that inline cache 
cleaning, it examines if the metadata
 ?? relocations of static/opt virtual call stubs need clearing 
(conditional of the holder of the target method being unloading).
6. Just before examining the metadata relocation, the sweeper gets 
preempted.
7. The GC finishes concurrent code cache purging.
8. The GC finishes concurrent metaspace purging (the Method* is now dead).
9. The sweeper wakes up from its slumber and starts processing the now 
freed Method* of the metadata relocation.
10. Crash.

The key here is that the reason why the sweeper clears metadata 
relocations of static/opt virtual call stubs,
is to unlink the metadata of is_alive() && !is_unloading() nmethods, so 
that they are not used after the class
unloading handshake. However, the GC does not unlink said metadata from 
is_alive() && is_unloading() nmethods,
because it is assumed that nobody should follow metadata from 
is_unloading() nmethods. This violation in the
sweeper is the exception to that rule.

The fix is to skip the processing of metadata relocations of 
is_unloading() nmethods, during inline cache cleaning.
I also noticed another related but harmless violation of this rule by 
class redefinition, where metadata only filters
is_alive(), not && !is_unloading(). I figured that this is actually 
harmless, because class redefinition happens in
a safepoint, and code cache purging is done with the suspendible thread 
set taken, blocking out safepoints. Therefore
it will happen either completely before or after code cache purging. If 
it happens before, then the metadata is not
yet deleted, and if it happens after, then it is no longer is_alive(). 
However, I would like to fix that anyway,
just to make the model more consistent.

Bug:
https://bugs.openjdk.java.net/browse/JDK-8240693

Webrev:
http://cr.openjdk.java.net/~eosterlund/8240693/webrev.00/

Sanity checked with hs-tier1-8 with ZGC.

Thanks,
/Erik

From mandy.chung at oracle.com  Thu Apr  2 18:56:47 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Thu, 2 Apr 2020 11:56:47 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <c1cd2330-2bfe-f92d-531d-35fb64a65e25@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <1a5eb022-dd7e-b0bb-750f-c43be195eb42@oracle.com>
 <319b5917-397f-5ac1-9c94-56f558653797@oracle.com>
 <bf7f63a9-5969-1f9b-bb2b-a2b869333884@oracle.com>
 <c1cd2330-2bfe-f92d-531d-35fb64a65e25@oracle.com>
Message-ID: <f888dca4-e39e-7310-5a8a-b82a658c0215@oracle.com>

Hi David,

Thank you for checking thoroughly.?? I now grep on src/hotspot and clean 
all of them.

Updated delta webrev:
http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.04-delta/

On 4/1/20 11:21 PM, David Holmes wrote:
> Hi Mandy,
>
> On 2/04/2020 3:17 pm, Mandy Chung wrote:
>> Hi David,
>>
>> Thanks for the edits to the comments.?? "weak hidden" were missed to 
>> be changed to "non-strong hidden".? Here is the patch fixing the 
>> comments.
>
> There are 33 cases of "weak hidden" in the patch I reviewed and also 
> some "hidden weak". Should they all be "non-strong hidden" now? 

yes, supposed to.?? All should be fixed in webrev.04-delta.

> In some cases it also appears in variables and associated logic ie.:
>
> src/hotspot/share/classfile/classLoaderHierarchyDCmd.cpp
>
> +????? _hidden_weak_classes(NULL), _num_hidden_weak_classes(0),
>

Are you reading webrev.03???? This has been fixed in webrev.04.

I found Klass::is_hidden_weak that should have been renamed too.

> I'm not clear how far this terminology change needs to extend ??
>

I hope consistently used in the source.

> Otherwise patch below seems fine.
>

Revised the patch.

Thanks
Mandy
> Thanks,
> David
> -----
>
>
>> diff --git a/src/hotspot/share/ci/ciField.cpp 
>> b/src/hotspot/share/ci/ciField.cpp
>> --- a/src/hotspot/share/ci/ciField.cpp
>> +++ b/src/hotspot/share/ci/ciField.cpp
>> @@ -223,8 +223,8 @@
>> ??????? holder->is_in_package("jdk/internal/foreign") || 
>> holder->is_in_package("jdk/incubator/foreign") ||
>> ??????? holder->is_in_package("java/lang"))
>> ????? return true;
>> -? // Trust VM hidden and unsafe anonymous classes. They are created 
>> via Lookup.defineClass or
>> -? // the private API (jdk.internal.misc.Unsafe) and can't be 
>> serialized, so there is no hacking
>> +? // Trust hidden and VM unsafe anonymous classes. They are created 
>> via Lookup.defineClass or
>> +? // the private jdk.internal.misc.Unsafe API and can't be 
>> serialized, so there is no hacking
>> ??? // of finals going on with them.
>> ??? if (holder->is_hidden() || holder->is_unsafe_anonymous())
>> ????? return true;
>> diff --git a/src/hotspot/share/ci/ciInstanceKlass.cpp 
>> b/src/hotspot/share/ci/ciInstanceKlass.cpp
>> --- a/src/hotspot/share/ci/ciInstanceKlass.cpp
>> +++ b/src/hotspot/share/ci/ciInstanceKlass.cpp
>> @@ -76,7 +76,7 @@
>> ??? oop holder = ik->klass_holder();
>> ??? if (ik->class_loader_data()->has_class_mirror_holder()) {
>> ????? // Though ciInstanceKlass records class loader oop, it's not 
>> enough to keep
>> -??? // VM weak hidden and unsafe anonymous classes alive (loader == 
>> NULL). Klass holder should
>> +??? // non-strong hidden class and VM unsafe anonymous classes alive 
>> (loader == NULL). Klass holder should
>> ????? // be used instead. It is enough to record a ciObject, since 
>> cached elements are never removed
>> ????? // during ciObjectFactory lifetime. ciObjectFactory itself is 
>> created for
>> ????? // every compilation and lives for the whole duration of the 
>> compilation.
>> diff --git a/src/hotspot/share/classfile/classLoaderData.hpp 
>> b/src/hotspot/share/classfile/classLoaderData.hpp
>> --- a/src/hotspot/share/classfile/classLoaderData.hpp
>> +++ b/src/hotspot/share/classfile/classLoaderData.hpp
>> @@ -127,9 +127,10 @@
>> ??? bool _accumulated_modified_oops; // Mod Union Equivalent (CMS 
>> support)
>>
>> ??? int _keep_alive;???????? // if this CLD is kept alive.
>> -?????????????????????????? // Used for weak hidden classes, unsafe 
>> anonymous classes and the
>> +?????????????????????????? // Used for non-strong hidden classes, 
>> unsafe anonymous classes and the
>> ???????????????????????????? // boot class loader. _keep_alive does 
>> not need to be volatile or
>> -?????????????????????????? // atomic since there is one unique CLD 
>> per weak hidden or unsafe anonymous class.
>> +?????????????????????????? // atomic since there is one unique CLD 
>> per non-strong hidden class
>> +?????????????????????????? // or unsafe anonymous class.
>>
>> ??? volatile int _claim; // non-zero if claimed, for example during 
>> GC traces.
>> ???????????????????????? // To avoid applying oop closure more than 
>> once.
>> @@ -242,15 +243,15 @@
>> ??? }
>>
>> ??? // Returns true if this class loader data is for the system class 
>> loader.
>> -? // (Note that the class loader data may be for an weak hidden or 
>> unsafe anonymous class)
>> +? // (Note that the class loader data may be for a non-strong hidden 
>> class or unsafe anonymous class)
>> ??? bool is_system_class_loader_data() const;
>>
>> ??? // Returns true if this class loader data is for the platform 
>> class loader.
>> -? // (Note that the class loader data may be for an weak hidden or 
>> unsafe anonymous class)
>> +? // (Note that the class loader data may be for a non-strong hidden 
>> class or unsafe anonymous class)
>> ??? bool is_platform_class_loader_data() const;
>>
>> ??? // Returns true if this class loader data is for the boot class 
>> loader.
>> -? // (Note that the class loader data may be for an weak hidden 
>> unsafe anonymous class)
>> +? // (Note that the class loader data may be for a non-strong hidden 
>> class or unsafe anonymous class)
>> ??? inline bool is_boot_class_loader_data() const;
>>
>> ??? bool is_builtin_class_loader_data() const;
>> @@ -271,7 +272,7 @@
>> ????? return _unloading;
>> ??? }
>>
>> -? // Used to refcount an weak hidden or unsafe anonymous class's CLD 
>> in order to
>> +? // Used to refcount a non-strong hidden class's or unsafe 
>> anonymous class's CLD in order to
>> ??? // indicate their aliveness.
>> ??? void inc_keep_alive();
>> ??? void dec_keep_alive();
>>
>> diff --git a/src/hotspot/share/interpreter/linkResolver.cpp 
>> b/src/hotspot/share/interpreter/linkResolver.cpp
>> --- a/src/hotspot/share/interpreter/linkResolver.cpp
>> +++ b/src/hotspot/share/interpreter/linkResolver.cpp
>> @@ -611,7 +611,7 @@
>> ?????????????? (same_module) ? "" : 
>> sel_klass->class_in_module_of_loader()
>> ?????????????? );
>>
>> -??? // For private access check if there was a problem with nest host
>> +??? // For private access see if there was a problem with nest host
>> ????? // resolution, and if so report that as part of the message.
>> ????? if (sel_method->is_private()) {
>> ??????? print_nest_host_error_on(&ss, ref_klass, sel_klass, THREAD);
>> @@ -951,7 +951,7 @@
>> ?????????????? (same_module) ? "" : "; ",
>> ?????????????? (same_module) ? "" : 
>> sel_klass->class_in_module_of_loader()
>> ?????????????? );
>> -??? // For private access check if there was a problem with nest host
>> +??? // For private access see if there was a problem with nest host
>> ????? // resolution, and if so report that as part of the message.
>> ????? if (fd.is_private()) {
>> ??????? print_nest_host_error_on(&ss, ref_klass, sel_klass, THREAD);
>>
>> Mandy
>>
>> On 4/1/20 9:52 PM, David Holmes wrote:
>>> Hi Mandy,
>>>
>>> On 1/04/2020 1:01 pm, Mandy Chung wrote:
>>>> Thanks to the review feedbacks.
>>>>
>>>> Updated webrev:
>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.04/ 
>>>>
>>>
>>> I've had a good look through all the hotspot related changes. All 
>>> looks good.
>>>
>>> A few minor comments:
>>>
>>> src/hotspot/share/ci/ciField.cpp
>>>
>>> ?? // Trust VM hidden and unsafe anonymous classes.
>>>
>>> should say
>>>
>>> ?? // Trust hidden and VM unsafe anonymous classes.
>>>
>>>
>>> ? // the private API (jdk.internal.misc.Unsafe)
>>>
>>> s/the/a/? or else change the () to commas or perhaps even better:
>>>
>>> ? // the private jdk.internal.misc.Unsafe API,
>>>
>>> ---
>>>
>>> src/hotspot/share/ci/ciInstanceKlass.cpp
>>>
>>> ? // VM weak hidden and unsafe anonymous classes
>>>
>>> should say
>>>
>>> ? // weak hidden and VM unsafe anonymous classes
>>>
>>> ---
>>>
>>> src/hotspot/share/classfile/classLoaderData.hpp
>>>
>>> !? // the CLDs lifecycle.? For example, a non-strong hidden class or an
>>> ...
>>> !? // Used for weak hidden classes, unsafe anonymous classes and the
>>>
>>> There is an inconsistency in terminology, so far most code comments 
>>> refer to "weak hidden" but now you are using "non-strong hidden".
>>>
>>> ! for an weak hidden
>>>
>>> s/an/a/?? multiple locations
>>>
>>> ---
>>>
>>> src/hotspot/share/interpreter/linkResolver.cpp
>>>
>>> Can you fix one of my comments please (in two places) :)
>>>
>>> +???? // For private access check if there was a problem with nest host
>>>
>>> would read better as:
>>>
>>> +???? // For private access see if there was a problem with nest host
>>>
>>> ---
>>>
>>> Thanks,
>>> David
>>>
>>


From coleen.phillimore at oracle.com  Thu Apr  2 18:59:40 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Thu, 2 Apr 2020 14:59:40 -0400
Subject: RFR: 8240693: Sweeper should not examine dying metadata in
 is_unloading() nmethod during static call stub cleaning
In-Reply-To: <a067ea89-86f1-269c-fd9b-33900158b961@oracle.com>
References: <a067ea89-86f1-269c-fd9b-33900158b961@oracle.com>
Message-ID: <085ace74-cc0f-8ae8-9950-995692306c89@oracle.com>


Erik,

The redefinition part looks good to me.? It should have been is_alive 
and not unloading, like the code cache walk to determine evol dependent 
nmethods.

thanks,
Coleen

On 4/2/20 12:28 PM, Erik ?sterlund wrote:
> Hi,
>
> There is currently an unfortunate race condition between concurrent 
> code cache unloading and the Sweeper
> (affecting ZGC and Shenandoah).
>
> First of all, big thanks to Nils for the initial analysis, requiring 
> downloading a 90 GB core file. Ouch.
> He discovered that we crashed accessing a freed metadata relocation in 
> an unloaded nmethod from the sweeper.
>
> I am fairly certain it is due to the following race condition that can 
> indeed happen today:
>
> 1. GC terminates marking in a safepoint
> 2. GC performs concurrent code cache unlinking. In this step, inline 
> caches are cleaned for is_alive() &&
> ?? !is_unloading() nmethods. The key is that it is not performed for 
> is_alive() && is_unloading() nmethods.
> ?? So metadata relocations of their static/opt virtual call stubs 
> remain in the nmethod, and could potentially
> ?? be dying.
> 3. A handshake operation is issued to separate unlinking from purging. 
> The sweeper replies to the handshake.
> 4. GC performs concurrent code cache purging, making all nmethods 
> unloaded. But just before one nmethod is unloaded...
> 5. The sweeper starts processing the nmethod. It is is_alive() && 
> is_unloading() and the sweeper correctly
> ?? decides to clean its inline caches. As part of that inline cache 
> cleaning, it examines if the metadata
> ?? relocations of static/opt virtual call stubs need clearing 
> (conditional of the holder of the target method being unloading).
> 6. Just before examining the metadata relocation, the sweeper gets 
> preempted.
> 7. The GC finishes concurrent code cache purging.
> 8. The GC finishes concurrent metaspace purging (the Method* is now 
> dead).
> 9. The sweeper wakes up from its slumber and starts processing the now 
> freed Method* of the metadata relocation.
> 10. Crash.
>
> The key here is that the reason why the sweeper clears metadata 
> relocations of static/opt virtual call stubs,
> is to unlink the metadata of is_alive() && !is_unloading() nmethods, 
> so that they are not used after the class
> unloading handshake. However, the GC does not unlink said metadata 
> from is_alive() && is_unloading() nmethods,
> because it is assumed that nobody should follow metadata from 
> is_unloading() nmethods. This violation in the
> sweeper is the exception to that rule.
>
> The fix is to skip the processing of metadata relocations of 
> is_unloading() nmethods, during inline cache cleaning.
> I also noticed another related but harmless violation of this rule by 
> class redefinition, where metadata only filters
> is_alive(), not && !is_unloading(). I figured that this is actually 
> harmless, because class redefinition happens in
> a safepoint, and code cache purging is done with the suspendible 
> thread set taken, blocking out safepoints. Therefore
> it will happen either completely before or after code cache purging. 
> If it happens before, then the metadata is not
> yet deleted, and if it happens after, then it is no longer is_alive(). 
> However, I would like to fix that anyway,
> just to make the model more consistent.
>
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8240693
>
> Webrev:
> http://cr.openjdk.java.net/~eosterlund/8240693/webrev.00/
>
> Sanity checked with hs-tier1-8 with ZGC.
>
> Thanks,
> /Erik


From bob.vandette at oracle.com  Thu Apr  2 19:57:04 2020
From: bob.vandette at oracle.com (Bob Vandette)
Date: Thu, 2 Apr 2020 15:57:04 -0400
Subject: RFR: 8241423: NUMA APIs may fail to work in the docker due to
 operation not permitted
In-Reply-To: <8F3CA0ED-1051-4BF4-B61B-C91856878992@tencent.com>
References: <8F3CA0ED-1051-4BF4-B61B-C91856878992@tencent.com>
Message-ID: <A9180A01-1155-4766-8443-3943A079E61E@oracle.com>

Jie,

Before we discuss this specific fix, I?d like to know if you have confirmed that Hotspot?s NUMA
support actually functions properly when running in containers (with proper privs)?

Also, do the libnuma functions work properly in response to cgroup limitations imposed by docker run --cpuset-mems?

Some of the traditional kernel functions reporting resource limits only report host values and do not
correctly report limits specified for containers.   To resolve this issue I have added an osContainer
class to hotspot.  Included in this class is a function that reports the memory nodes available 
to hotspot when running in a container.   It might be necessary to query this function when
trying to configure the hotspot NUMA support.

Back to your webrev, is it not possible to get the address for numa_available and
then try to calling it in order to determine if NUMA can be used?

If it is determined that you don?t have sufficient access, I would suggest disabling UseNUMA
all together.

Bob

> On Mar 23, 2020, at 11:58 AM, jiefu(??) <jiefu at tencent.com> wrote:
> 
> Hi all,
> 
> JBS:    https://bugs.openjdk.java.net/browse/JDK-8241423
> Webrev: http://cr.openjdk.java.net/~jiefu/8241423/webrev.00/
> 
> A VM fatal error may be observed if ZGC is used (see JDK-8241354).
> The background is that some of our products run in the docker.
> And for safety reasons, SYS_get_mempolicy is not allowed by default [1].
> 
> At first, we thought it just a zgc-only problem and filed JDK-8241354.
> But Thomas had reminded me that other collectors are also affected [2].
> So it would be better to fix them together.
> 
> After more investigation, we found that NUMA APIs are actually dependent on several syscalls, such as get_mempolicy, mbind and set_mempolicy.
> When the required syscalls are unavailable, NUMA APIs fail to work as expected.
> 
> The fix is to check whether the required syscalls are available.
> In theory, all NUMA-related syscalls should be checked.
> But it seems hard to do so because some of them will cause unexpected side effect.
> To fix our issue, checking get_mempolicy is enough.
> And just as Per suggested that we can refine this later if it turns out to be a problem [3].
> 
> Please review it and give me some advice.
> 
> Thanks a lot.
> Best regards,
> Jie
> 
> [1] https://docs.docker.com/engine/security/seccomp/
> [2] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028923.html
> [3] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028933.html


From lois.foltan at oracle.com  Thu Apr  2 20:28:07 2020
From: lois.foltan at oracle.com (Lois Foltan)
Date: Thu, 2 Apr 2020 16:28:07 -0400
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <f888dca4-e39e-7310-5a8a-b82a658c0215@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <1a5eb022-dd7e-b0bb-750f-c43be195eb42@oracle.com>
 <319b5917-397f-5ac1-9c94-56f558653797@oracle.com>
 <bf7f63a9-5969-1f9b-bb2b-a2b869333884@oracle.com>
 <c1cd2330-2bfe-f92d-531d-35fb64a65e25@oracle.com>
 <f888dca4-e39e-7310-5a8a-b82a658c0215@oracle.com>
Message-ID: <333b05fb-65c6-e8ef-a804-b624c2da56c5@oracle.com>

On 4/2/2020 2:56 PM, Mandy Chung wrote:
> Hi David,
>
> Thank you for checking thoroughly.?? I now grep on src/hotspot and 
> clean all of them.
>
> Updated delta webrev:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.04-delta/ 
>

Patch looks good Mandy.
Lois

>
> On 4/1/20 11:21 PM, David Holmes wrote:
>> Hi Mandy,
>>
>> On 2/04/2020 3:17 pm, Mandy Chung wrote:
>>> Hi David,
>>>
>>> Thanks for the edits to the comments.?? "weak hidden" were missed to 
>>> be changed to "non-strong hidden".? Here is the patch fixing the 
>>> comments.
>>
>> There are 33 cases of "weak hidden" in the patch I reviewed and also 
>> some "hidden weak". Should they all be "non-strong hidden" now? 
>
> yes, supposed to.?? All should be fixed in webrev.04-delta.
>
>> In some cases it also appears in variables and associated logic ie.:
>>
>> src/hotspot/share/classfile/classLoaderHierarchyDCmd.cpp
>>
>> +????? _hidden_weak_classes(NULL), _num_hidden_weak_classes(0),
>>
>
> Are you reading webrev.03???? This has been fixed in webrev.04.
>
> I found Klass::is_hidden_weak that should have been renamed too.
>
>> I'm not clear how far this terminology change needs to extend ??
>>
>
> I hope consistently used in the source.
>
>> Otherwise patch below seems fine.
>>
>
> Revised the patch.
>
> Thanks
> Mandy
>> Thanks,
>> David
>> -----
>>
>>
>>> diff --git a/src/hotspot/share/ci/ciField.cpp 
>>> b/src/hotspot/share/ci/ciField.cpp
>>> --- a/src/hotspot/share/ci/ciField.cpp
>>> +++ b/src/hotspot/share/ci/ciField.cpp
>>> @@ -223,8 +223,8 @@
>>> ??????? holder->is_in_package("jdk/internal/foreign") || 
>>> holder->is_in_package("jdk/incubator/foreign") ||
>>> ??????? holder->is_in_package("java/lang"))
>>> ????? return true;
>>> -? // Trust VM hidden and unsafe anonymous classes. They are created 
>>> via Lookup.defineClass or
>>> -? // the private API (jdk.internal.misc.Unsafe) and can't be 
>>> serialized, so there is no hacking
>>> +? // Trust hidden and VM unsafe anonymous classes. They are created 
>>> via Lookup.defineClass or
>>> +? // the private jdk.internal.misc.Unsafe API and can't be 
>>> serialized, so there is no hacking
>>> ??? // of finals going on with them.
>>> ??? if (holder->is_hidden() || holder->is_unsafe_anonymous())
>>> ????? return true;
>>> diff --git a/src/hotspot/share/ci/ciInstanceKlass.cpp 
>>> b/src/hotspot/share/ci/ciInstanceKlass.cpp
>>> --- a/src/hotspot/share/ci/ciInstanceKlass.cpp
>>> +++ b/src/hotspot/share/ci/ciInstanceKlass.cpp
>>> @@ -76,7 +76,7 @@
>>> ??? oop holder = ik->klass_holder();
>>> ??? if (ik->class_loader_data()->has_class_mirror_holder()) {
>>> ????? // Though ciInstanceKlass records class loader oop, it's not 
>>> enough to keep
>>> -??? // VM weak hidden and unsafe anonymous classes alive (loader == 
>>> NULL). Klass holder should
>>> +??? // non-strong hidden class and VM unsafe anonymous classes 
>>> alive (loader == NULL). Klass holder should
>>> ????? // be used instead. It is enough to record a ciObject, since 
>>> cached elements are never removed
>>> ????? // during ciObjectFactory lifetime. ciObjectFactory itself is 
>>> created for
>>> ????? // every compilation and lives for the whole duration of the 
>>> compilation.
>>> diff --git a/src/hotspot/share/classfile/classLoaderData.hpp 
>>> b/src/hotspot/share/classfile/classLoaderData.hpp
>>> --- a/src/hotspot/share/classfile/classLoaderData.hpp
>>> +++ b/src/hotspot/share/classfile/classLoaderData.hpp
>>> @@ -127,9 +127,10 @@
>>> ??? bool _accumulated_modified_oops; // Mod Union Equivalent (CMS 
>>> support)
>>>
>>> ??? int _keep_alive;???????? // if this CLD is kept alive.
>>> -?????????????????????????? // Used for weak hidden classes, unsafe 
>>> anonymous classes and the
>>> +?????????????????????????? // Used for non-strong hidden classes, 
>>> unsafe anonymous classes and the
>>> ???????????????????????????? // boot class loader. _keep_alive does 
>>> not need to be volatile or
>>> -?????????????????????????? // atomic since there is one unique CLD 
>>> per weak hidden or unsafe anonymous class.
>>> +?????????????????????????? // atomic since there is one unique CLD 
>>> per non-strong hidden class
>>> +?????????????????????????? // or unsafe anonymous class.
>>>
>>> ??? volatile int _claim; // non-zero if claimed, for example during 
>>> GC traces.
>>> ???????????????????????? // To avoid applying oop closure more than 
>>> once.
>>> @@ -242,15 +243,15 @@
>>> ??? }
>>>
>>> ??? // Returns true if this class loader data is for the system 
>>> class loader.
>>> -? // (Note that the class loader data may be for an weak hidden or 
>>> unsafe anonymous class)
>>> +? // (Note that the class loader data may be for a non-strong 
>>> hidden class or unsafe anonymous class)
>>> ??? bool is_system_class_loader_data() const;
>>>
>>> ??? // Returns true if this class loader data is for the platform 
>>> class loader.
>>> -? // (Note that the class loader data may be for an weak hidden or 
>>> unsafe anonymous class)
>>> +? // (Note that the class loader data may be for a non-strong 
>>> hidden class or unsafe anonymous class)
>>> ??? bool is_platform_class_loader_data() const;
>>>
>>> ??? // Returns true if this class loader data is for the boot class 
>>> loader.
>>> -? // (Note that the class loader data may be for an weak hidden 
>>> unsafe anonymous class)
>>> +? // (Note that the class loader data may be for a non-strong 
>>> hidden class or unsafe anonymous class)
>>> ??? inline bool is_boot_class_loader_data() const;
>>>
>>> ??? bool is_builtin_class_loader_data() const;
>>> @@ -271,7 +272,7 @@
>>> ????? return _unloading;
>>> ??? }
>>>
>>> -? // Used to refcount an weak hidden or unsafe anonymous class's 
>>> CLD in order to
>>> +? // Used to refcount a non-strong hidden class's or unsafe 
>>> anonymous class's CLD in order to
>>> ??? // indicate their aliveness.
>>> ??? void inc_keep_alive();
>>> ??? void dec_keep_alive();
>>>
>>> diff --git a/src/hotspot/share/interpreter/linkResolver.cpp 
>>> b/src/hotspot/share/interpreter/linkResolver.cpp
>>> --- a/src/hotspot/share/interpreter/linkResolver.cpp
>>> +++ b/src/hotspot/share/interpreter/linkResolver.cpp
>>> @@ -611,7 +611,7 @@
>>> ?????????????? (same_module) ? "" : 
>>> sel_klass->class_in_module_of_loader()
>>> ?????????????? );
>>>
>>> -??? // For private access check if there was a problem with nest host
>>> +??? // For private access see if there was a problem with nest host
>>> ????? // resolution, and if so report that as part of the message.
>>> ????? if (sel_method->is_private()) {
>>> ??????? print_nest_host_error_on(&ss, ref_klass, sel_klass, THREAD);
>>> @@ -951,7 +951,7 @@
>>> ?????????????? (same_module) ? "" : "; ",
>>> ?????????????? (same_module) ? "" : 
>>> sel_klass->class_in_module_of_loader()
>>> ?????????????? );
>>> -??? // For private access check if there was a problem with nest host
>>> +??? // For private access see if there was a problem with nest host
>>> ????? // resolution, and if so report that as part of the message.
>>> ????? if (fd.is_private()) {
>>> ??????? print_nest_host_error_on(&ss, ref_klass, sel_klass, THREAD);
>>>
>>> Mandy
>>>
>>> On 4/1/20 9:52 PM, David Holmes wrote:
>>>> Hi Mandy,
>>>>
>>>> On 1/04/2020 1:01 pm, Mandy Chung wrote:
>>>>> Thanks to the review feedbacks.
>>>>>
>>>>> Updated webrev:
>>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.04/ 
>>>>>
>>>>
>>>> I've had a good look through all the hotspot related changes. All 
>>>> looks good.
>>>>
>>>> A few minor comments:
>>>>
>>>> src/hotspot/share/ci/ciField.cpp
>>>>
>>>> ?? // Trust VM hidden and unsafe anonymous classes.
>>>>
>>>> should say
>>>>
>>>> ?? // Trust hidden and VM unsafe anonymous classes.
>>>>
>>>>
>>>> ? // the private API (jdk.internal.misc.Unsafe)
>>>>
>>>> s/the/a/? or else change the () to commas or perhaps even better:
>>>>
>>>> ? // the private jdk.internal.misc.Unsafe API,
>>>>
>>>> ---
>>>>
>>>> src/hotspot/share/ci/ciInstanceKlass.cpp
>>>>
>>>> ? // VM weak hidden and unsafe anonymous classes
>>>>
>>>> should say
>>>>
>>>> ? // weak hidden and VM unsafe anonymous classes
>>>>
>>>> ---
>>>>
>>>> src/hotspot/share/classfile/classLoaderData.hpp
>>>>
>>>> !? // the CLDs lifecycle.? For example, a non-strong hidden class 
>>>> or an
>>>> ...
>>>> !? // Used for weak hidden classes, unsafe anonymous classes and the
>>>>
>>>> There is an inconsistency in terminology, so far most code comments 
>>>> refer to "weak hidden" but now you are using "non-strong hidden".
>>>>
>>>> ! for an weak hidden
>>>>
>>>> s/an/a/?? multiple locations
>>>>
>>>> ---
>>>>
>>>> src/hotspot/share/interpreter/linkResolver.cpp
>>>>
>>>> Can you fix one of my comments please (in two places) :)
>>>>
>>>> +???? // For private access check if there was a problem with nest 
>>>> host
>>>>
>>>> would read better as:
>>>>
>>>> +???? // For private access see if there was a problem with nest host
>>>>
>>>> ---
>>>>
>>>> Thanks,
>>>> David
>>>>
>>>
>


From erik.osterlund at oracle.com  Thu Apr  2 20:35:52 2020
From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=)
Date: Thu, 2 Apr 2020 22:35:52 +0200
Subject: RFR: 8240693: Sweeper should not examine dying metadata in
 is_unloading() nmethod during static call stub cleaning
In-Reply-To: <085ace74-cc0f-8ae8-9950-995692306c89@oracle.com>
References: <085ace74-cc0f-8ae8-9950-995692306c89@oracle.com>
Message-ID: <8617E701-337A-4C2E-B632-B3F30F240C41@oracle.com>

Hi Coleen,

Thanks for the review.

/Erik

> On 2 Apr 2020, at 21:00, coleen.phillimore at oracle.com wrote:
> 
> ?
> Erik,
> 
> The redefinition part looks good to me.  It should have been is_alive and not unloading, like the code cache walk to determine evol dependent nmethods.
> 
> thanks,
> Coleen
> 
>> On 4/2/20 12:28 PM, Erik ?sterlund wrote:
>> Hi,
>> 
>> There is currently an unfortunate race condition between concurrent code cache unloading and the Sweeper
>> (affecting ZGC and Shenandoah).
>> 
>> First of all, big thanks to Nils for the initial analysis, requiring downloading a 90 GB core file. Ouch.
>> He discovered that we crashed accessing a freed metadata relocation in an unloaded nmethod from the sweeper.
>> 
>> I am fairly certain it is due to the following race condition that can indeed happen today:
>> 
>> 1. GC terminates marking in a safepoint
>> 2. GC performs concurrent code cache unlinking. In this step, inline caches are cleaned for is_alive() &&
>>    !is_unloading() nmethods. The key is that it is not performed for is_alive() && is_unloading() nmethods.
>>    So metadata relocations of their static/opt virtual call stubs remain in the nmethod, and could potentially
>>    be dying.
>> 3. A handshake operation is issued to separate unlinking from purging. The sweeper replies to the handshake.
>> 4. GC performs concurrent code cache purging, making all nmethods unloaded. But just before one nmethod is unloaded...
>> 5. The sweeper starts processing the nmethod. It is is_alive() && is_unloading() and the sweeper correctly
>>    decides to clean its inline caches. As part of that inline cache cleaning, it examines if the metadata
>>    relocations of static/opt virtual call stubs need clearing (conditional of the holder of the target method being unloading).
>> 6. Just before examining the metadata relocation, the sweeper gets preempted.
>> 7. The GC finishes concurrent code cache purging.
>> 8. The GC finishes concurrent metaspace purging (the Method* is now dead).
>> 9. The sweeper wakes up from its slumber and starts processing the now freed Method* of the metadata relocation.
>> 10. Crash.
>> 
>> The key here is that the reason why the sweeper clears metadata relocations of static/opt virtual call stubs,
>> is to unlink the metadata of is_alive() && !is_unloading() nmethods, so that they are not used after the class
>> unloading handshake. However, the GC does not unlink said metadata from is_alive() && is_unloading() nmethods,
>> because it is assumed that nobody should follow metadata from is_unloading() nmethods. This violation in the
>> sweeper is the exception to that rule.
>> 
>> The fix is to skip the processing of metadata relocations of is_unloading() nmethods, during inline cache cleaning.
>> I also noticed another related but harmless violation of this rule by class redefinition, where metadata only filters
>> is_alive(), not && !is_unloading(). I figured that this is actually harmless, because class redefinition happens in
>> a safepoint, and code cache purging is done with the suspendible thread set taken, blocking out safepoints. Therefore
>> it will happen either completely before or after code cache purging. If it happens before, then the metadata is not
>> yet deleted, and if it happens after, then it is no longer is_alive(). However, I would like to fix that anyway,
>> just to make the model more consistent.
>> 
>> Bug:
>> https://bugs.openjdk.java.net/browse/JDK-8240693
>> 
>> Webrev:
>> http://cr.openjdk.java.net/~eosterlund/8240693/webrev.00/
>> 
>> Sanity checked with hs-tier1-8 with ZGC.
>> 
>> Thanks,
>> /Erik
> 


From david.holmes at oracle.com  Fri Apr  3 01:36:03 2020
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 3 Apr 2020 11:36:03 +1000
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <f888dca4-e39e-7310-5a8a-b82a658c0215@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <1a5eb022-dd7e-b0bb-750f-c43be195eb42@oracle.com>
 <319b5917-397f-5ac1-9c94-56f558653797@oracle.com>
 <bf7f63a9-5969-1f9b-bb2b-a2b869333884@oracle.com>
 <c1cd2330-2bfe-f92d-531d-35fb64a65e25@oracle.com>
 <f888dca4-e39e-7310-5a8a-b82a658c0215@oracle.com>
Message-ID: <10e51a8a-6d69-db08-d4f9-f578ec591ee2@oracle.com>

Hi Mandy,

Update seems fine.

Thanks,
David

On 3/04/2020 4:56 am, Mandy Chung wrote:
> Hi David,
> 
> Thank you for checking thoroughly.?? I now grep on src/hotspot and clean 
> all of them.
> 
> Updated delta webrev:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.04-delta/
> 
> On 4/1/20 11:21 PM, David Holmes wrote:
>> Hi Mandy,
>>
>> On 2/04/2020 3:17 pm, Mandy Chung wrote:
>>> Hi David,
>>>
>>> Thanks for the edits to the comments.?? "weak hidden" were missed to 
>>> be changed to "non-strong hidden".? Here is the patch fixing the 
>>> comments.
>>
>> There are 33 cases of "weak hidden" in the patch I reviewed and also 
>> some "hidden weak". Should they all be "non-strong hidden" now? 
> 
> yes, supposed to.?? All should be fixed in webrev.04-delta.
> 
>> In some cases it also appears in variables and associated logic ie.:
>>
>> src/hotspot/share/classfile/classLoaderHierarchyDCmd.cpp
>>
>> +????? _hidden_weak_classes(NULL), _num_hidden_weak_classes(0),
>>
> 
> Are you reading webrev.03???? This has been fixed in webrev.04.
> 
> I found Klass::is_hidden_weak that should have been renamed too.
> 
>> I'm not clear how far this terminology change needs to extend ??
>>
> 
> I hope consistently used in the source.
> 
>> Otherwise patch below seems fine.
>>
> 
> Revised the patch.
> 
> Thanks
> Mandy
>> Thanks,
>> David
>> -----
>>
>>
>>> diff --git a/src/hotspot/share/ci/ciField.cpp 
>>> b/src/hotspot/share/ci/ciField.cpp
>>> --- a/src/hotspot/share/ci/ciField.cpp
>>> +++ b/src/hotspot/share/ci/ciField.cpp
>>> @@ -223,8 +223,8 @@
>>> ??????? holder->is_in_package("jdk/internal/foreign") || 
>>> holder->is_in_package("jdk/incubator/foreign") ||
>>> ??????? holder->is_in_package("java/lang"))
>>> ????? return true;
>>> -? // Trust VM hidden and unsafe anonymous classes. They are created 
>>> via Lookup.defineClass or
>>> -? // the private API (jdk.internal.misc.Unsafe) and can't be 
>>> serialized, so there is no hacking
>>> +? // Trust hidden and VM unsafe anonymous classes. They are created 
>>> via Lookup.defineClass or
>>> +? // the private jdk.internal.misc.Unsafe API and can't be 
>>> serialized, so there is no hacking
>>> ??? // of finals going on with them.
>>> ??? if (holder->is_hidden() || holder->is_unsafe_anonymous())
>>> ????? return true;
>>> diff --git a/src/hotspot/share/ci/ciInstanceKlass.cpp 
>>> b/src/hotspot/share/ci/ciInstanceKlass.cpp
>>> --- a/src/hotspot/share/ci/ciInstanceKlass.cpp
>>> +++ b/src/hotspot/share/ci/ciInstanceKlass.cpp
>>> @@ -76,7 +76,7 @@
>>> ??? oop holder = ik->klass_holder();
>>> ??? if (ik->class_loader_data()->has_class_mirror_holder()) {
>>> ????? // Though ciInstanceKlass records class loader oop, it's not 
>>> enough to keep
>>> -??? // VM weak hidden and unsafe anonymous classes alive (loader == 
>>> NULL). Klass holder should
>>> +??? // non-strong hidden class and VM unsafe anonymous classes alive 
>>> (loader == NULL). Klass holder should
>>> ????? // be used instead. It is enough to record a ciObject, since 
>>> cached elements are never removed
>>> ????? // during ciObjectFactory lifetime. ciObjectFactory itself is 
>>> created for
>>> ????? // every compilation and lives for the whole duration of the 
>>> compilation.
>>> diff --git a/src/hotspot/share/classfile/classLoaderData.hpp 
>>> b/src/hotspot/share/classfile/classLoaderData.hpp
>>> --- a/src/hotspot/share/classfile/classLoaderData.hpp
>>> +++ b/src/hotspot/share/classfile/classLoaderData.hpp
>>> @@ -127,9 +127,10 @@
>>> ??? bool _accumulated_modified_oops; // Mod Union Equivalent (CMS 
>>> support)
>>>
>>> ??? int _keep_alive;???????? // if this CLD is kept alive.
>>> -?????????????????????????? // Used for weak hidden classes, unsafe 
>>> anonymous classes and the
>>> +?????????????????????????? // Used for non-strong hidden classes, 
>>> unsafe anonymous classes and the
>>> ???????????????????????????? // boot class loader. _keep_alive does 
>>> not need to be volatile or
>>> -?????????????????????????? // atomic since there is one unique CLD 
>>> per weak hidden or unsafe anonymous class.
>>> +?????????????????????????? // atomic since there is one unique CLD 
>>> per non-strong hidden class
>>> +?????????????????????????? // or unsafe anonymous class.
>>>
>>> ??? volatile int _claim; // non-zero if claimed, for example during 
>>> GC traces.
>>> ???????????????????????? // To avoid applying oop closure more than 
>>> once.
>>> @@ -242,15 +243,15 @@
>>> ??? }
>>>
>>> ??? // Returns true if this class loader data is for the system class 
>>> loader.
>>> -? // (Note that the class loader data may be for an weak hidden or 
>>> unsafe anonymous class)
>>> +? // (Note that the class loader data may be for a non-strong hidden 
>>> class or unsafe anonymous class)
>>> ??? bool is_system_class_loader_data() const;
>>>
>>> ??? // Returns true if this class loader data is for the platform 
>>> class loader.
>>> -? // (Note that the class loader data may be for an weak hidden or 
>>> unsafe anonymous class)
>>> +? // (Note that the class loader data may be for a non-strong hidden 
>>> class or unsafe anonymous class)
>>> ??? bool is_platform_class_loader_data() const;
>>>
>>> ??? // Returns true if this class loader data is for the boot class 
>>> loader.
>>> -? // (Note that the class loader data may be for an weak hidden 
>>> unsafe anonymous class)
>>> +? // (Note that the class loader data may be for a non-strong hidden 
>>> class or unsafe anonymous class)
>>> ??? inline bool is_boot_class_loader_data() const;
>>>
>>> ??? bool is_builtin_class_loader_data() const;
>>> @@ -271,7 +272,7 @@
>>> ????? return _unloading;
>>> ??? }
>>>
>>> -? // Used to refcount an weak hidden or unsafe anonymous class's CLD 
>>> in order to
>>> +? // Used to refcount a non-strong hidden class's or unsafe 
>>> anonymous class's CLD in order to
>>> ??? // indicate their aliveness.
>>> ??? void inc_keep_alive();
>>> ??? void dec_keep_alive();
>>>
>>> diff --git a/src/hotspot/share/interpreter/linkResolver.cpp 
>>> b/src/hotspot/share/interpreter/linkResolver.cpp
>>> --- a/src/hotspot/share/interpreter/linkResolver.cpp
>>> +++ b/src/hotspot/share/interpreter/linkResolver.cpp
>>> @@ -611,7 +611,7 @@
>>> ?????????????? (same_module) ? "" : 
>>> sel_klass->class_in_module_of_loader()
>>> ?????????????? );
>>>
>>> -??? // For private access check if there was a problem with nest host
>>> +??? // For private access see if there was a problem with nest host
>>> ????? // resolution, and if so report that as part of the message.
>>> ????? if (sel_method->is_private()) {
>>> ??????? print_nest_host_error_on(&ss, ref_klass, sel_klass, THREAD);
>>> @@ -951,7 +951,7 @@
>>> ?????????????? (same_module) ? "" : "; ",
>>> ?????????????? (same_module) ? "" : 
>>> sel_klass->class_in_module_of_loader()
>>> ?????????????? );
>>> -??? // For private access check if there was a problem with nest host
>>> +??? // For private access see if there was a problem with nest host
>>> ????? // resolution, and if so report that as part of the message.
>>> ????? if (fd.is_private()) {
>>> ??????? print_nest_host_error_on(&ss, ref_klass, sel_klass, THREAD);
>>>
>>> Mandy
>>>
>>> On 4/1/20 9:52 PM, David Holmes wrote:
>>>> Hi Mandy,
>>>>
>>>> On 1/04/2020 1:01 pm, Mandy Chung wrote:
>>>>> Thanks to the review feedbacks.
>>>>>
>>>>> Updated webrev:
>>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.04/ 
>>>>>
>>>>
>>>> I've had a good look through all the hotspot related changes. All 
>>>> looks good.
>>>>
>>>> A few minor comments:
>>>>
>>>> src/hotspot/share/ci/ciField.cpp
>>>>
>>>> ?? // Trust VM hidden and unsafe anonymous classes.
>>>>
>>>> should say
>>>>
>>>> ?? // Trust hidden and VM unsafe anonymous classes.
>>>>
>>>>
>>>> ? // the private API (jdk.internal.misc.Unsafe)
>>>>
>>>> s/the/a/? or else change the () to commas or perhaps even better:
>>>>
>>>> ? // the private jdk.internal.misc.Unsafe API,
>>>>
>>>> ---
>>>>
>>>> src/hotspot/share/ci/ciInstanceKlass.cpp
>>>>
>>>> ? // VM weak hidden and unsafe anonymous classes
>>>>
>>>> should say
>>>>
>>>> ? // weak hidden and VM unsafe anonymous classes
>>>>
>>>> ---
>>>>
>>>> src/hotspot/share/classfile/classLoaderData.hpp
>>>>
>>>> !? // the CLDs lifecycle.? For example, a non-strong hidden class or an
>>>> ...
>>>> !? // Used for weak hidden classes, unsafe anonymous classes and the
>>>>
>>>> There is an inconsistency in terminology, so far most code comments 
>>>> refer to "weak hidden" but now you are using "non-strong hidden".
>>>>
>>>> ! for an weak hidden
>>>>
>>>> s/an/a/?? multiple locations
>>>>
>>>> ---
>>>>
>>>> src/hotspot/share/interpreter/linkResolver.cpp
>>>>
>>>> Can you fix one of my comments please (in two places) :)
>>>>
>>>> +???? // For private access check if there was a problem with nest host
>>>>
>>>> would read better as:
>>>>
>>>> +???? // For private access see if there was a problem with nest host
>>>>
>>>> ---
>>>>
>>>> Thanks,
>>>> David
>>>>
>>>
> 

From jiefu at tencent.com  Fri Apr  3 07:58:18 2020
From: jiefu at tencent.com (=?utf-8?B?amllZnUo5YKF5p2wKQ==?=)
Date: Fri, 3 Apr 2020 07:58:18 +0000
Subject: RFR: 8241423: NUMA APIs may fail to work in the docker due to
 operation not permitted
Message-ID: <97987050-6175-44EC-8ABE-B51D60E513AF@tencent.com>

Hi Bob,

Thanks for your review and helpful comments. 
                    
I'm not a docker expert. 
Apart from the zgc crash [1], we didn't come across other problems in the docker.
                    
It seems that this bug has nothing to do with the resource limit.
The root cause is that some NUMA-related syscalls are disabled in the docker for safety reasons.

Please note that we already have numa_available() check here [2].
But it failed to detect such cases.

What do you think?

Thanks a lot.
Best regards,
Jie
                      
[1] https://bugs.openjdk.java.net/browse/JDK-8241354
[2] http://hg.openjdk.java.net/jdk/jdk/file/f50a7df94744/src/hotspot/os/linux/os_linux.cpp#l3182

?On 2020/4/3, 3:58 AM, "Bob Vandette" <bob.vandette at oracle.com> wrote:

    Jie,
    
    Before we discuss this specific fix, I?d like to know if you have confirmed that Hotspot?s NUMA
    support actually functions properly when running in containers (with proper privs)?
    
    Also, do the libnuma functions work properly in response to cgroup limitations imposed by docker run --cpuset-mems?
    
    Some of the traditional kernel functions reporting resource limits only report host values and do not
    correctly report limits specified for containers.   To resolve this issue I have added an osContainer
    class to hotspot.  Included in this class is a function that reports the memory nodes available 
    to hotspot when running in a container.   It might be necessary to query this function when
    trying to configure the hotspot NUMA support.
    
    Back to your webrev, is it not possible to get the address for numa_available and
    then try to calling it in order to determine if NUMA can be used?
    
    If it is determined that you don?t have sufficient access, I would suggest disabling UseNUMA
    all together.
    
    Bob
    
    > On Mar 23, 2020, at 11:58 AM, jiefu(??) <jiefu at tencent.com> wrote:
    > 
    > Hi all,
    > 
    > JBS:    https://bugs.openjdk.java.net/browse/JDK-8241423
    > Webrev: http://cr.openjdk.java.net/~jiefu/8241423/webrev.00/
    > 
    > A VM fatal error may be observed if ZGC is used (see JDK-8241354).
    > The background is that some of our products run in the docker.
    > And for safety reasons, SYS_get_mempolicy is not allowed by default [1].
    > 
    > At first, we thought it just a zgc-only problem and filed JDK-8241354.
    > But Thomas had reminded me that other collectors are also affected [2].
    > So it would be better to fix them together.
    > 
    > After more investigation, we found that NUMA APIs are actually dependent on several syscalls, such as get_mempolicy, mbind and set_mempolicy.
    > When the required syscalls are unavailable, NUMA APIs fail to work as expected.
    > 
    > The fix is to check whether the required syscalls are available.
    > In theory, all NUMA-related syscalls should be checked.
    > But it seems hard to do so because some of them will cause unexpected side effect.
    > To fix our issue, checking get_mempolicy is enough.
    > And just as Per suggested that we can refine this later if it turns out to be a problem [3].
    > 
    > Please review it and give me some advice.
    > 
    > Thanks a lot.
    > Best regards,
    > Jie
    > 
    > [1] https://docs.docker.com/engine/security/seccomp/
    > [2] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028923.html
    > [3] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028933.html
    
    
    


From peter.levart at gmail.com  Fri Apr  3 09:11:04 2020
From: peter.levart at gmail.com (Peter Levart)
Date: Fri, 3 Apr 2020 11:11:04 +0200
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
Message-ID: <b74f5458-94e2-02f6-5799-722910efd906@gmail.com>

Hi Mandy,

Good work.

I'm trying to find out which language use-case is covered by the 
InnerClassLambdaMetafactory needing to inject method handle into the 
generated proxy class to be invoked instead of proxy class directly 
invoking the method:

 ??????? useImplMethodHandle = 
!implClass.getPackageName().equals(implInfo.getDeclaringClass().getPackageName())
 ??????????????????????????????? && 
!Modifier.isPublic(implInfo.getModifiers());

If I check what implClass and implInfo get resolved to in 
AbstractValidatingLambdaMetafactory, there are several cases:

 ??????? this.implInfo = caller.revealDirect(implMethod);
 ??????? switch (implInfo.getReferenceKind()) {
 ??????????? case REF_invokeVirtual:
 ??????????? case REF_invokeInterface:
 ??????????????? this.implClass = implMethodType.parameterType(0);
 ??????????????? // reference kind reported by implInfo may not match 
implMethodType's first param
 ??????????????? // Example: implMethodType is (Cloneable)String, 
implInfo is for Object.toString
 ??????????????? this.implKind = implClass.isInterface() ? 
REF_invokeInterface : REF_invokeVirtual;
 ??????????????? this.implIsInstanceMethod = true;
 ??????????????? break;
 ??????????? case REF_invokeSpecial:
 ??????????????? // JDK-8172817: should use referenced class here, but 
we don't know what it was
 ??????????????? this.implClass = implInfo.getDeclaringClass();
 ??????????????? this.implIsInstanceMethod = true;

 ??????????????? // Classes compiled prior to dynamic nestmate support 
invokes a private instance
 ??????????????? // method with REF_invokeSpecial.
 ??????????????? //
 ??????????????? // invokespecial should only be used to invoke private 
nestmate constructors.
 ??????????????? // The lambda proxy class will be defined as a nestmate 
of targetClass.
 ??????????????? // If the method to be invoked is an instance method of 
targetClass, then
 ??????????????? // convert to use invokevirtual or invokeinterface.
 ??????????????? if (targetClass == implClass && 
!implInfo.getName().equals("<init>")) {
 ??????????????????? this.implKind = implClass.isInterface() ? 
REF_invokeInterface : REF_invokeVirtual;
 ??????????????? } else {
 ??????????????????? this.implKind = REF_invokeSpecial;
 ??????????????? }
 ??????????????? break;
 ??????????? case REF_invokeStatic:
 ??????????? case REF_newInvokeSpecial:
 ??????????????? // JDK-8172817: should use referenced class here for 
invokestatic, but we don't know what it was
 ??????????????? this.implClass = implInfo.getDeclaringClass();
 ??????????????? this.implKind = implInfo.getReferenceKind();
 ??????????????? this.implIsInstanceMethod = false;
 ??????????????? break;
 ??????????? default:
 ??????????????? throw new 
LambdaConversionException(String.format("Unsupported MethodHandle kind: 
%s", implInfo));
 ??????? }


For majority of cases (REF_invokeSpecial, REF_invokeStatic, 
REF_newInvokeSpecial) the this.implClass = implInfo.getDeclaringClass();

Only REF_invokeVirtual and REF_invokeInterface are possible kandidates, 
right?

So when does the implMethod type of parameter 0 differ from the 
declaring class of the MethodHandleInfo cracked from that same 
implMethod and in addition those two types leave in different packages?

Regards, Peter


On 3/27/20 12:57 AM, Mandy Chung wrote:
> Please review the implementation of JEP 371: Hidden Classes. The main 
> changes are in core-libs and hotspot runtime area.? Small changes are 
> made in javac, VM compiler (intrinsification of Class::isHiddenClass), 
> JFR, JDI, and jcmd.? CSR [1]has been reviewed and is in the finalized 
> state (see specdiff and javadoc below for reference).
>
> Webrev:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
>
>
> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's point
> of view, a hidden class is a normal class except the following:
>
> - A hidden class has no initiating class loader and is not registered 
> in any dictionary.
> - A hidden class has a name containing an illegal character 
> `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature` 
> returns "Lp/Foo.0x1234;".
> - A hidden class is not modifiable, i.e. cannot be redefined or 
> retransformed. JVM TI IsModifableClass returns false on a hidden.
> - Final fields in a hidden class is "final".? The value of final 
> fields cannot be overriden via reflection.? setAccessible(true) can 
> still be called on reflected objects representing final fields in a 
> hidden class and its access check will be suppressed but only have 
> read-access (i.e. can do Field::getXXX but not setXXX).
>
> Brief summary of this patch:
>
> 1. A new Lookup::defineHiddenClass method is the API to create a 
> hidden class.
> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG 
> option that
> ?? can be specified when creating a hidden class.
> 3. A new Class::isHiddenClass method tests if a class is a hidden class.
> 4. Field::setXXX method will throw IAE on a final field of a hidden class
> ?? regardless of the value of the accessible flag.
> 5. JVM_LookupDefineClass is the new JVM entry point for 
> Lookup::defineClass
> ?? and defineHiddenClass to create a class from the given bytes.
> 6. ClassLoaderData implementation is not changed.? There is one 
> primary CLD
> ?? that holds the classes strongly referenced by its defining loader.? 
> There
> ?? can be zero or more additional CLDs - one per weak class.
> 7. Nest host determination is updated per revised JVMS 5.4.4. Access 
> control
> ?? check no longer throws LinkageError but instead it will throw IAE with
> ?? a clear message if a class fails to resolve/validate the nest host 
> declared
> ?? in NestHost/NestMembers attribute.
> 8. JFR, jcmd, JDI are updated to support hidden classes.
> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
> ?? and generate a bridge method to desuger a method reference to a 
> protected
> ?? method in its supertype in a different package
>
> This patch also updates StringConcatFactory, LambdaMetaFactory, and 
> LambdaForms
> to use hidden classes.? The webrev includes changes in nashorn to 
> hidden class
> and I will update the webrev if JEP 372 removes it any time soon.
>
> We uncovered a bug in Lookup::defineClass spec throws LinkageError and 
> intends
> to have the newly created class linked.? However, the implementation 
> in 14
> does not link the class.? A separate CSR [2] proposes to update the
> implementation to match the spec.? This patch fixes the implementation.
>
> The spec update on JVM TI, JDI and Instrumentation will be done as
> a separate RFE [3].? This patch includes new tests for JVM TI and
> java.instrument that validates how the existing APIs work for hidden 
> classes.
>
> javadoc/specdiff
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
>
>
> JVMS 5.4.4 change:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
>
>
> CSR:
> https://bugs.openjdk.java.net/browse/JDK-8238359
>
> Thanks
> Mandy
> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
> [3] https://bugs.openjdk.java.net/browse/JDK-8230502


From peter.levart at gmail.com  Fri Apr  3 11:40:54 2020
From: peter.levart at gmail.com (Peter Levart)
Date: Fri, 3 Apr 2020 13:40:54 +0200
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <b74f5458-94e2-02f6-5799-722910efd906@gmail.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <b74f5458-94e2-02f6-5799-722910efd906@gmail.com>
Message-ID: <958f4c01-85a2-5d96-7221-c9549fae76f3@gmail.com>

Ok, I think I found one such use-case. In the following example:

package test;
public class LambdaTest {
 ??? protected void m() {
 ??? }
}

package test.sub;
public class LambdaTestSub extends test.LambdaTest {
 ??? public void test() {
 ??????? Runnable r = this::m;
 ??????? r.run();
 ??? }
}

...when compiled with JDK 14 javac. In this case the implClass is 
test.sub.LambdaTestSub while implInfo is "invokeVirtual 
test.LambdaTest.m:()void" and the method is not public.

Anyway, the name of the proxy class is derived from the targetClass (and 
therefore shares the same package with targetClass) which is caller's 
lookup class. Is targetClass always the same as implClass in the 
invokeVirtual/invokeInterface case?

I also noticed that JDK 15 patched javac transforms method reference in 
above code into a lambda method. But looking at the javac changes in the 
patch, I don't quite see where this distinction between JDK 14 and 
patched JDK 15 javac comes from. From the changes to method 
com.sun.tools.javac.comp.LambdaToMethod.LambdaAnalyzerPreprocessor.ReferenceTranslationContext#needsConversionToLambda:

 ??????????? final boolean needsConversionToLambda() {
 ??????????????? return interfaceParameterIsIntersectionOrUnionType() ||
 ??????????????????????? isSuper ||
 ??????????????????????? needsVarArgsConversion() ||
 ??????????????????????? isArrayOp() ||
#??????????????????????? isPrivateInOtherClass() ||
isProtectedInSuperClassOfEnclosingClassInOtherPackage() ||
 ??????????????????????? !receiverAccessible() ||
 ??????????????????????? (tree.getMode() == ReferenceMode.NEW &&
 ????????????????????????? tree.kind != ReferenceKind.ARRAY_CTOR &&
 ????????????????????????? (tree.sym.owner.isLocal() || 
tree.sym.owner.isInner()));
 ??????????? }

...I would draw the conclusion that conversion to lambda is performed in 
less cases not in more. Hm.

Regards, Peter

On 4/3/20 11:11 AM, Peter Levart wrote:
> Hi Mandy,
>
> Good work.
>
> I'm trying to find out which language use-case is covered by the 
> InnerClassLambdaMetafactory needing to inject method handle into the 
> generated proxy class to be invoked instead of proxy class directly 
> invoking the method:
>
> ??????? useImplMethodHandle = 
> !implClass.getPackageName().equals(implInfo.getDeclaringClass().getPackageName())
> ??????????????????????????????? && 
> !Modifier.isPublic(implInfo.getModifiers());
>
> If I check what implClass and implInfo get resolved to in 
> AbstractValidatingLambdaMetafactory, there are several cases:
>
> ??????? this.implInfo = caller.revealDirect(implMethod);
> ??????? switch (implInfo.getReferenceKind()) {
> ??????????? case REF_invokeVirtual:
> ??????????? case REF_invokeInterface:
> ??????????????? this.implClass = implMethodType.parameterType(0);
> ??????????????? // reference kind reported by implInfo may not match 
> implMethodType's first param
> ??????????????? // Example: implMethodType is (Cloneable)String, 
> implInfo is for Object.toString
> ??????????????? this.implKind = implClass.isInterface() ? 
> REF_invokeInterface : REF_invokeVirtual;
> ??????????????? this.implIsInstanceMethod = true;
> ??????????????? break;
> ??????????? case REF_invokeSpecial:
> ??????????????? // JDK-8172817: should use referenced class here, but 
> we don't know what it was
> ??????????????? this.implClass = implInfo.getDeclaringClass();
> ??????????????? this.implIsInstanceMethod = true;
>
> ??????????????? // Classes compiled prior to dynamic nestmate support 
> invokes a private instance
> ??????????????? // method with REF_invokeSpecial.
> ??????????????? //
> ??????????????? // invokespecial should only be used to invoke private 
> nestmate constructors.
> ??????????????? // The lambda proxy class will be defined as a 
> nestmate of targetClass.
> ??????????????? // If the method to be invoked is an instance method 
> of targetClass, then
> ??????????????? // convert to use invokevirtual or invokeinterface.
> ??????????????? if (targetClass == implClass && 
> !implInfo.getName().equals("<init>")) {
> ??????????????????? this.implKind = implClass.isInterface() ? 
> REF_invokeInterface : REF_invokeVirtual;
> ??????????????? } else {
> ??????????????????? this.implKind = REF_invokeSpecial;
> ??????????????? }
> ??????????????? break;
> ??????????? case REF_invokeStatic:
> ??????????? case REF_newInvokeSpecial:
> ??????????????? // JDK-8172817: should use referenced class here for 
> invokestatic, but we don't know what it was
> ??????????????? this.implClass = implInfo.getDeclaringClass();
> ??????????????? this.implKind = implInfo.getReferenceKind();
> ??????????????? this.implIsInstanceMethod = false;
> ??????????????? break;
> ??????????? default:
> ??????????????? throw new 
> LambdaConversionException(String.format("Unsupported MethodHandle 
> kind: %s", implInfo));
> ??????? }
>
>
> For majority of cases (REF_invokeSpecial, REF_invokeStatic, 
> REF_newInvokeSpecial) the this.implClass = implInfo.getDeclaringClass();
>
> Only REF_invokeVirtual and REF_invokeInterface are possible 
> kandidates, right?
>
> So when does the implMethod type of parameter 0 differ from the 
> declaring class of the MethodHandleInfo cracked from that same 
> implMethod and in addition those two types leave in different packages?
>
> Regards, Peter
>
>
> On 3/27/20 12:57 AM, Mandy Chung wrote:
>> Please review the implementation of JEP 371: Hidden Classes. The main 
>> changes are in core-libs and hotspot runtime area.? Small changes are 
>> made in javac, VM compiler (intrinsification of 
>> Class::isHiddenClass), JFR, JDI, and jcmd.? CSR [1]has been reviewed 
>> and is in the finalized state (see specdiff and javadoc below for 
>> reference).
>>
>> Webrev:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
>>
>>
>> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's 
>> point
>> of view, a hidden class is a normal class except the following:
>>
>> - A hidden class has no initiating class loader and is not registered 
>> in any dictionary.
>> - A hidden class has a name containing an illegal character 
>> `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature` 
>> returns "Lp/Foo.0x1234;".
>> - A hidden class is not modifiable, i.e. cannot be redefined or 
>> retransformed. JVM TI IsModifableClass returns false on a hidden.
>> - Final fields in a hidden class is "final".? The value of final 
>> fields cannot be overriden via reflection.? setAccessible(true) can 
>> still be called on reflected objects representing final fields in a 
>> hidden class and its access check will be suppressed but only have 
>> read-access (i.e. can do Field::getXXX but not setXXX).
>>
>> Brief summary of this patch:
>>
>> 1. A new Lookup::defineHiddenClass method is the API to create a 
>> hidden class.
>> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG 
>> option that
>> ?? can be specified when creating a hidden class.
>> 3. A new Class::isHiddenClass method tests if a class is a hidden class.
>> 4. Field::setXXX method will throw IAE on a final field of a hidden 
>> class
>> ?? regardless of the value of the accessible flag.
>> 5. JVM_LookupDefineClass is the new JVM entry point for 
>> Lookup::defineClass
>> ?? and defineHiddenClass to create a class from the given bytes.
>> 6. ClassLoaderData implementation is not changed.? There is one 
>> primary CLD
>> ?? that holds the classes strongly referenced by its defining 
>> loader.? There
>> ?? can be zero or more additional CLDs - one per weak class.
>> 7. Nest host determination is updated per revised JVMS 5.4.4. Access 
>> control
>> ?? check no longer throws LinkageError but instead it will throw IAE 
>> with
>> ?? a clear message if a class fails to resolve/validate the nest host 
>> declared
>> ?? in NestHost/NestMembers attribute.
>> 8. JFR, jcmd, JDI are updated to support hidden classes.
>> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
>> ?? and generate a bridge method to desuger a method reference to a 
>> protected
>> ?? method in its supertype in a different package
>>
>> This patch also updates StringConcatFactory, LambdaMetaFactory, and 
>> LambdaForms
>> to use hidden classes.? The webrev includes changes in nashorn to 
>> hidden class
>> and I will update the webrev if JEP 372 removes it any time soon.
>>
>> We uncovered a bug in Lookup::defineClass spec throws LinkageError 
>> and intends
>> to have the newly created class linked.? However, the implementation 
>> in 14
>> does not link the class.? A separate CSR [2] proposes to update the
>> implementation to match the spec.? This patch fixes the implementation.
>>
>> The spec update on JVM TI, JDI and Instrumentation will be done as
>> a separate RFE [3].? This patch includes new tests for JVM TI and
>> java.instrument that validates how the existing APIs work for hidden 
>> classes.
>>
>> javadoc/specdiff
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
>>
>>
>> JVMS 5.4.4 change:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
>>
>>
>> CSR:
>> https://bugs.openjdk.java.net/browse/JDK-8238359
>>
>> Thanks
>> Mandy
>> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
>> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
>> [3] https://bugs.openjdk.java.net/browse/JDK-8230502
>


From mandy.chung at oracle.com  Fri Apr  3 21:32:48 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Fri, 3 Apr 2020 14:32:48 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <958f4c01-85a2-5d96-7221-c9549fae76f3@gmail.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <b74f5458-94e2-02f6-5799-722910efd906@gmail.com>
 <958f4c01-85a2-5d96-7221-c9549fae76f3@gmail.com>
Message-ID: <1490090a-9e0e-b51b-03d6-088e479315be@oracle.com>

Hi Peter,

I added a JBS comment [1] to describe this special case trying to put 
the story together (let me know if it needs more explanation). More 
comment inline below.

On 4/3/20 4:40 AM, Peter Levart wrote:
> Ok, I think I found one such use-case. In the following example:
>
> package test;
> public class LambdaTest {
> ??? protected void m() {
> ??? }
> }
>
> package test.sub;
> public class LambdaTestSub extends test.LambdaTest {
> ??? public void test() {
> ??????? Runnable r = this::m;
> ??????? r.run();
> ??? }
> }

Yes.

This is specific for binary compatibility.?? the invocation of a 
protected method inherited from its supertype in a different package.

The lambda proxy is in the same package as the target class (`test.sub` 
in the example above) but it has no access to `test.LambdaTest::m`.

>
> ...when compiled with JDK 14 javac. In this case the implClass is 
> test.sub.LambdaTestSub while implInfo is "invokeVirtual 
> test.LambdaTest.m:()void" and the method is not public.
>

In JDK 14, a lambda proxy `test.sub.LambdaTestSub$Lambda$$1234` is VM 
anonymous class which has a special powerful access as if the host 
class.?? This proxy class, even though it's not an instance of 
`test.LambdaTest`, can invoke? the protected`test.LambdaTest.m:()void` 
member.

> Anyway, the name of the proxy class is derived from the targetClass 
> (and therefore shares the same package with targetClass) which is 
> caller's lookup class. Is targetClass always the same as implClass in 
> the invokeVirtual/invokeInterface case?
>

implMethod is the direct method handle describing the implementation 
method resolved by the VM.?? So it depends.

In the above example, it's `test.LambdaTest.m:()void` and implClass is 
test.LambdaTest.? The targetClass is test.sub.LambdaTestSub which is 
*NOT* the same as implClass.? That's why we change if they are in the 
same package.

It can be invoking a method in targetClass or a method in another class 
in the same package with package access, then implClass may or may not 
be the same as targetClass.

> I also noticed that JDK 15 patched javac transforms method reference 
> in above code into a lambda method. But looking at the javac changes 
> in the patch, I don't quite see where this distinction between JDK 14 
> and patched JDK 15 javac comes from. 

javac has been changed in JDK 14 to synthesize a bridge method if it's a 
method reference to access a protected member in a remote supertype? 
(JDK-8234729).

BTW, the new tests relevant to this scenario are under 
test/jdk/java/lang/invoke/lambda/superProtectedMethod.

> From the changes to method 
> com.sun.tools.javac.comp.LambdaToMethod.LambdaAnalyzerPreprocessor.ReferenceTranslationContext#needsConversionToLambda:
>
> ??????????? final boolean needsConversionToLambda() {
> ??????????????? return interfaceParameterIsIntersectionOrUnionType() ||
> ??????????????????????? isSuper ||
> ??????????????????????? needsVarArgsConversion() ||
> ??????????????????????? isArrayOp() ||
> #??????????????????????? isPrivateInOtherClass() ||
> isProtectedInSuperClassOfEnclosingClassInOtherPackage() ||
> ??????????????????????? !receiverAccessible() ||
> ??????????????????????? (tree.getMode() == ReferenceMode.NEW &&
> ????????????????????????? tree.kind != ReferenceKind.ARRAY_CTOR &&
> ????????????????????????? (tree.sym.owner.isLocal() || 
> tree.sym.owner.isInner()));
> ??????????? }
>
> ...I would draw the conclusion that conversion to lambda is performed 
> in less cases not in more. 

Jan and Srikanath may be able to explain this further.

> Hm.
>
> Regards, Peter
>
> On 4/3/20 11:11 AM, Peter Levart wrote:
>> Hi Mandy,
>>
>> Good work.
>>
>> I'm trying to find out which language use-case is covered by the 
>> InnerClassLambdaMetafactory needing to inject method handle into the 
>> generated proxy class to be invoked instead of proxy class directly 
>> invoking the method:
>>
>> ??????? useImplMethodHandle = 
>> !implClass.getPackageName().equals(implInfo.getDeclaringClass().getPackageName())
>> ??????????????????????????????? && 
>> !Modifier.isPublic(implInfo.getModifiers());
>>
>> If I check what implClass and implInfo get resolved to in 
>> AbstractValidatingLambdaMetafactory, there are several cases:
>>
>> ??????? this.implInfo = caller.revealDirect(implMethod);
>> ??????? switch (implInfo.getReferenceKind()) {
>> ??????????? case REF_invokeVirtual:
>> ??????????? case REF_invokeInterface:
>> ??????????????? this.implClass = implMethodType.parameterType(0);
>> ??????????????? // reference kind reported by implInfo may not match 
>> implMethodType's first param
>> ??????????????? // Example: implMethodType is (Cloneable)String, 
>> implInfo is for Object.toString
>> ??????????????? this.implKind = implClass.isInterface() ? 
>> REF_invokeInterface : REF_invokeVirtual;
>> ??????????????? this.implIsInstanceMethod = true;
>> ??????????????? break;
>> ??????????? case REF_invokeSpecial:
>> ??????????????? // JDK-8172817: should use referenced class here, but 
>> we don't know what it was
>> ??????????????? this.implClass = implInfo.getDeclaringClass();
>> ??????????????? this.implIsInstanceMethod = true;
>>
>> ??????????????? // Classes compiled prior to dynamic nestmate support 
>> invokes a private instance
>> ??????????????? // method with REF_invokeSpecial.
>> ??????????????? //
>> ??????????????? // invokespecial should only be used to invoke 
>> private nestmate constructors.
>> ??????????????? // The lambda proxy class will be defined as a 
>> nestmate of targetClass.
>> ??????????????? // If the method to be invoked is an instance method 
>> of targetClass, then
>> ??????????????? // convert to use invokevirtual or invokeinterface.
>> ??????????????? if (targetClass == implClass && 
>> !implInfo.getName().equals("<init>")) {
>> ??????????????????? this.implKind = implClass.isInterface() ? 
>> REF_invokeInterface : REF_invokeVirtual;
>> ??????????????? } else {
>> ??????????????????? this.implKind = REF_invokeSpecial;
>> ??????????????? }
>> ??????????????? break;
>> ??????????? case REF_invokeStatic:
>> ??????????? case REF_newInvokeSpecial:
>> ??????????????? // JDK-8172817: should use referenced class here for 
>> invokestatic, but we don't know what it was
>> ??????????????? this.implClass = implInfo.getDeclaringClass();
>> ??????????????? this.implKind = implInfo.getReferenceKind();
>> ??????????????? this.implIsInstanceMethod = false;
>> ??????????????? break;
>> ??????????? default:
>> ??????????????? throw new 
>> LambdaConversionException(String.format("Unsupported MethodHandle 
>> kind: %s", implInfo));
>> ??????? }
>>
>>
>> For majority of cases (REF_invokeSpecial, REF_invokeStatic, 
>> REF_newInvokeSpecial) the this.implClass = implInfo.getDeclaringClass();
>>
>> Only REF_invokeVirtual and REF_invokeInterface are possible 
>> kandidates, right?
>>
>> So when does the implMethod type of parameter 0 differ from the 
>> declaring class of the MethodHandleInfo cracked from that same 
>> implMethod and in addition those two types leave in different packages?
>>

This is concerning the instance method and so parameter 0 is what it 
wants to look at.

>> Regards, Peter
>>

Hope this helps.
Mandy
[1] 
https://bugs.openjdk.java.net/browse/JDK-8239384?focusedCommentId=14328369&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14328369



From vladimir.x.ivanov at oracle.com  Fri Apr  3 23:12:30 2020
From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov)
Date: Sat, 4 Apr 2020 02:12:30 +0300
Subject: RFR (XXL): 8223347: Integration of Vector API (Incubator): General
 HotSpot changes
Message-ID: <c1bdf88c-5de2-d069-5f31-5a95c6988bf8@oracle.com>

Hi,

Following up on review requests of API [0] and Java implementation [1] 
for Vector API (JEP 338 [2]), here's a request for review of general 
HotSpot changes (in shared code) required for supporting the API:

 
http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/all.00-03/

(First of all, to set proper expectations: since the JEP is still in 
Candidate state, the intention is to initiate preliminary round(s) of 
review to inform the community and gather feedback before sending out 
final/official RFRs once the JEP is Targeted to a release.)

Vector API (being developed in Project Panama [3]) relies on JVM support 
to utilize optimal vector hardware instructions at runtime. It interacts 
with JVM through intrinsics (declared in 
jdk.internal.vm.vector.VectorSupport [4]) which expose vector operations 
support in C2 JIT-compiler.

As Paul wrote earlier: "A vector intrinsic is an internal low-level 
vector operation. The last argument to the intrinsic is fall back 
behavior in Java, implementing the scalar operation over the number of 
elements held by the vector.  Thus, If the intrinsic is not supported in 
C2 for the other arguments then the Java implementation is executed (the 
Java implementation is always executed when running in the interpreter 
or for C1)."

The rest of JVM support is about aggressively optimizing vector boxes to 
minimize (ideally eliminate) the overhead of boxing for vector values.
It's a stop-the-gap solution for vector box elimination problem until 
inline classes arrive. Vector classes are value-based and in the longer 
term will be migrated to inline classes once the support becomes available.

Vector API talk from JVMLS'18 [5] contains brief overview of JVM 
implementation and some details.

Complete implementation resides in vector-unstable branch of panama/dev 
repository [6].

Now to gory details (the patch is split in multiple "sub-webrevs"):

===========================================================

(1) 
http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/00.backend.shared/

Ideal vector nodes for new operations introduced by Vector API.

(Platform-specific back end support will be posted for review separately).

===========================================================

(2) 
http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/01.intrinsics/

JVM Java interface (VectorSupport) and intrinsic support in C2.

Vector instances are initially represented as VectorBox macro nodes and 
"unboxing" is represented by VectorUnbox node. It simplifies vector box 
elimination analysis and the nodes are expanded later right before EA pass.

Vectors have 2-level on-heap representation: for the vector value 
primitive array is used as a backing storage and it is encapsulated in a 
typed wrapper (e.g., Int256Vector - vector of 8 ints - contains a int[8] 
instance which is used to store vector value).

Unless VectorBox node goes away, it needs to be expanded into an 
allocation eventually, but it is a pure node and doesn't have any JVM 
state associated with it. The problem is solved by keeping JVM state 
separately in a VectorBoxAllocate node associated with VectorBox node 
and use it during expansion.

Also, to simplify vector box elimination, inlining of vector reboxing 
calls (VectorSupport::maybeRebox) is delayed until the analysis is over.

===========================================================

(3) 
http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/02.vbox_elimination/

Vector box elimination analysis implementation. (Brief overview: slides 
#36-42 [5].)

The main part is devoted to scalarization across safepoints and 
rematerialization support during deoptimization. In C2-generated code 
vector operations work with raw vector values which live in registers or 
spilled on the stack and it allows to avoid boxing/unboxing when a 
vector value is alive across a safepoint. As with other values, there's 
just a location of the vector value at the safepoint and vector type 
information recorded in the relevant nmethod metadata and all the 
heavy-lifting happens only when rematerialization takes place.

The analysis preserves object identity invariants except during 
aggressive reboxing (guarded by -XX:+EnableAggressiveReboxing).

(Aggressive reboxing is crucial for cases when vectors "escape": it 
allocates a fresh instance at every escape point thus enabling original 
instance to go away.)

===========================================================

(4) 
http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/03.module.hotspot/

HotSpot changes for jdk.incubator.vector module. Vector support is 
makred experimental and turned off by default. JEP 338 proposes the API 
to be released as an incubator module, so a user has to specify 
"--add-module jdk.incubator.vector" on the command line to be able to 
use it.
When user does that, JVM automatically enables Vector API support.
It improves usability (user doesn't need to separately "open" the API 
and enable JVM support) while minimizing risks of destabilitzation from 
new code when the API is not used.


That's it! Will be happy to answer any questions.

And thanks in advance for any feedback!

Best regards,
Vladimir Ivanov

[0] 
https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065345.html

[1] 
https://mail.openjdk.java.net/pipermail/hotspot-dev/2020-April/041228.html

[2] https://openjdk.java.net/jeps/338

[3] https://openjdk.java.net/projects/panama/

[4] 
http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/01.intrinsics/src/java.base/share/classes/jdk/internal/vm/vector/VectorSupport.java.html

[5] http://cr.openjdk.java.net/~vlivanov/talks/2018_JVMLS_VectorAPI.pdf

[6] http://hg.openjdk.java.net/panama/dev/shortlog/92bbd44386e9

     $ hg clone http://hg.openjdk.java.net/panama/dev/ -b vector-unstable

From forax at univ-mlv.fr  Fri Apr  3 23:31:11 2020
From: forax at univ-mlv.fr (Remi Forax)
Date: Sat, 4 Apr 2020 01:31:11 +0200 (CEST)
Subject: RFR (XXL): 8223347: Integration of Vector API (Incubator):
 General HotSpot changes
In-Reply-To: <c1bdf88c-5de2-d069-5f31-5a95c6988bf8@oracle.com>
References: <c1bdf88c-5de2-d069-5f31-5a95c6988bf8@oracle.com>
Message-ID: <579633672.1221535.1585956671384.JavaMail.zimbra@u-pem.fr>

[...]

> (4)
> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/03.module.hotspot/
> 
> HotSpot changes for jdk.incubator.vector module. Vector support is
> makred experimental and turned off by default. JEP 338 proposes the API
> to be released as an incubator module, so a user has to specify
> "--add-module jdk.incubator.vector" on the command line to be able to
> use it.

Typo, it's --add-modules

> When user does that, JVM automatically enables Vector API support.
> It improves usability (user doesn't need to separately "open" the API
> and enable JVM support) while minimizing risks of destabilitzation from
> new code when the API is not used.

Question, what if i declare a module-info that requires "jdk.incubator.vector", because in that case, i don't have to add --add-modules jdk.incubator.vector on the command line, but does the VM will enable the Vector API support ?

> 
> 
> That's it! Will be happy to answer any questions.
> 
> And thanks in advance for any feedback!

regards,
R?mi

> 
> Best regards,
> Vladimir Ivanov
> 
> [0]
> https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065345.html
> 
> [1]
> https://mail.openjdk.java.net/pipermail/hotspot-dev/2020-April/041228.html
> 
> [2] https://openjdk.java.net/jeps/338
> 
> [3] https://openjdk.java.net/projects/panama/
> 
> [4]
> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/01.intrinsics/src/java.base/share/classes/jdk/internal/vm/vector/VectorSupport.java.html
> 
> [5] http://cr.openjdk.java.net/~vlivanov/talks/2018_JVMLS_VectorAPI.pdf
> 
> [6] http://hg.openjdk.java.net/panama/dev/shortlog/92bbd44386e9
> 
>      $ hg clone http://hg.openjdk.java.net/panama/dev/ -b vector-unstable

From vladimir.x.ivanov at oracle.com  Fri Apr  3 23:52:03 2020
From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov)
Date: Sat, 4 Apr 2020 02:52:03 +0300
Subject: RFR (XXL): 8223347: Integration of Vector API (Incubator):
 General HotSpot changes
In-Reply-To: <579633672.1221535.1585956671384.JavaMail.zimbra@u-pem.fr>
References: <c1bdf88c-5de2-d069-5f31-5a95c6988bf8@oracle.com>
 <579633672.1221535.1585956671384.JavaMail.zimbra@u-pem.fr>
Message-ID: <4e61bd37-8f8d-1f35-c1db-7826db2b0f53@oracle.com>


> Typo, it's --add-modules

Good catch, Remi. Thanks for the correction.

> 
>> When user does that, JVM automatically enables Vector API support.
>> It improves usability (user doesn't need to separately "open" the API
>> and enable JVM support) while minimizing risks of destabilitzation from
>> new code when the API is not used.
> 
> Question, what if i declare a module-info that requires "jdk.incubator.vector", because in that case, i don't have to add --add-modules jdk.incubator.vector on the command line, but does the VM will enable the Vector API support ?

Good point. JEP 11: "Incubator Modules" [1] states the following:

"Applications on the class path must use the --add-modules command-line 
option to request that an incubator module be resolved. Applications 
developed as modules can specify requires or requires transitive 
dependences upon an incubator module directly."

Current implementation doesn't distinguish whether the module is 
resolved for an application on the class path or by another module, so 
JVM support will be enabled by default in both cases. Do you see any 
problems with that?

Best regards,
Vladimir Ivanov

[1] https://openjdk.java.net/jeps/11

>> That's it! Will be happy to answer any questions.
>>
>> And thanks in advance for any feedback!
> 
> regards,
> R?mi
> 
>>
>> Best regards,
>> Vladimir Ivanov
>>
>> [0]
>> https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065345.html
>>
>> [1]
>> https://mail.openjdk.java.net/pipermail/hotspot-dev/2020-April/041228.html
>>
>> [2] https://openjdk.java.net/jeps/338
>>
>> [3] https://openjdk.java.net/projects/panama/
>>
>> [4]
>> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/01.intrinsics/src/java.base/share/classes/jdk/internal/vm/vector/VectorSupport.java.html
>>
>> [5] http://cr.openjdk.java.net/~vlivanov/talks/2018_JVMLS_VectorAPI.pdf
>>
>> [6] http://hg.openjdk.java.net/panama/dev/shortlog/92bbd44386e9
>>
>>       $ hg clone http://hg.openjdk.java.net/panama/dev/ -b vector-unstable

From peter.levart at gmail.com  Sat Apr  4 10:58:15 2020
From: peter.levart at gmail.com (Peter Levart)
Date: Sat, 4 Apr 2020 12:58:15 +0200
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <1490090a-9e0e-b51b-03d6-088e479315be@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <b74f5458-94e2-02f6-5799-722910efd906@gmail.com>
 <958f4c01-85a2-5d96-7221-c9549fae76f3@gmail.com>
 <1490090a-9e0e-b51b-03d6-088e479315be@oracle.com>
Message-ID: <c941480c-c76e-0695-4a18-20d9aff224d7@gmail.com>

Hi Mandy,

On 4/3/20 11:32 PM, Mandy Chung wrote:
> Hi Peter,
>
> I added a JBS comment [1] to describe this special case trying to put 
> the story together (let me know if it needs more explanation). ? More 
> comment inline below.

Thanks, this helps in establishing the historical context.

>
> On 4/3/20 4:40 AM, Peter Levart wrote:
>> Ok, I think I found one such use-case. In the following example:
>>
>> package test;
>> public class LambdaTest {
>> ??? protected void m() {
>> ??? }
>> }
>>
>> package test.sub;
>> public class LambdaTestSub extends test.LambdaTest {
>> ??? public void test() {
>> ??????? Runnable r = this::m;
>> ??????? r.run();
>> ??? }
>> }
>
> Yes.
>
> This is specific for binary compatibility.?? the invocation of a 
> protected method inherited from its supertype in a different package.
>
> The lambda proxy is in the same package as the target class 
> (`test.sub` in the example above) but it has no access to 
> `test.LambdaTest::m`.
>
>>
>> ...when compiled with JDK 14 javac. In this case the implClass is 
>> test.sub.LambdaTestSub while implInfo is "invokeVirtual 
>> test.LambdaTest.m:()void" and the method is not public.
>>
>
> In JDK 14, a lambda proxy `test.sub.LambdaTestSub$Lambda$$1234` is VM 
> anonymous class which has a special powerful access as if the host 
> class.?? This proxy class, even though it's not an instance of 
> `test.LambdaTest`, can invoke? the protected`test.LambdaTest.m:()void` 
> member.

Right, the VM anonymous class "inherits" all access rights from the host 
class, which in above example is the subclass of test.LambdaTes.

>
>> Anyway, the name of the proxy class is derived from the targetClass 
>> (and therefore shares the same package with targetClass) which is 
>> caller's lookup class. Is targetClass always the same as implClass in 
>> the invokeVirtual/invokeInterface case?
>>
>
> implMethod is the direct method handle describing the implementation 
> method resolved by the VM.?? So it depends.
>
> In the above example, it's `test.LambdaTest.m:()void` and implClass is 
> test.LambdaTest.

Here I think, you are not quite right. First I need to clarify that we 
are talking about the case where the method reference in above example 
is not converted to lambda by javac, so the proxy class needs to invoke 
the superclass method directly (without the help of lambda bridge). I 
did an experiment and compiled the example with JDK 13 javac, where the 
patch for (JDK-8234729) is not applied yet. What I get from this 
compilation is the following metafactory bootstrap method invocation:

 ? 0: #35 REF_invokeStatic 
java/lang/invoke/LambdaMetafactory.metafactory:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite;
 ??? Method arguments:
 ????? #42 ()V
 ????? #43 REF_invokeVirtual test/LambdaTest.m:()V
 ????? #42 ()V

The #43 is the implMethod method handle and it is the following:

 ? #43 = MethodHandle?????? 5:#44????????? // REF_invokeVirtual 
test/LambdaTest.m:()V
 ? #44 = Methodref????????? #2.#45???????? // test/LambdaTest.m:()V
 ? #45 = NameAndType??????? #46:#6???????? // m:()V
 ? #46 = Utf8?????????????? m
 ?? #2 = Class????????????? #4???????????? // test/LambdaTest
 ?? #4 = Utf8?????????????? test/LambdaTest

*BUT* the class that looks up this MH from the constant pool is the 
subclass (test.sub.LambdaTestSub) which is equivalent to the following 
programmatic lookup:

 ??????? var mh = MethodHandles.lookup().findVirtual(LambdaTest.class, 
"m", MethodType.methodType(void.class));
 ??????? System.out.println(mh.type());

and this results in a method handle of the following type: 
(LambdaTestSub)void

which is correct since the method handle *MUST* check that the passed in 
instance (this) is of type LambdaTestSub or subtype or else the 
"protected" access would be violated.

And since the ref type is REF_invokeVirtual, the 
AbstractValidatingLambdaMetafactory assigns the following to the implClass:

 ??????? this.implMethodType = implMethod.type();
 ??????? this.implInfo = caller.revealDirect(implMethod);
 ??????? switch (implInfo.getReferenceKind()) {
 ??????????? case REF_invokeVirtual:
 ??????????? case REF_invokeInterface:
 ??????????????? this.implClass = implMethodType.parameterType(0);

...which makes the implClass be LambdaTestSub in this case. Which is 
correct again since we want InnerClassLambdaMetafactory to decide that 
this is the special case for proxy to invoke the method via method handle:

 ??????? useImplMethodHandle = 
!implClass.getPackageName().equals(implInfo.getDeclaringClass().getPackageName())
 ??????????????????????????????? && 
!Modifier.isPublic(implInfo.getModifiers());

If implClass was test.LambdaTest as you said above, this condition would 
evaluate to false, since implInfo is "invokeVirtual 
test.LambdaTest.m:()void" in above case.

So everything is OK, but my original question was the following: The 
name of the generated proxy class is derived from the targetClass which 
is the caller's lookup class. In this example the caller is 
LambdaTestSub and this is the same as implClass in this case. Would 
those two classes always be the same in the case where the evaluation of 
the above `useImplMethodHandle` boolean matters? I mean, the decision 
about whether to base the proxy invocation strategy on method handle or 
direct bytecode invocation is based on one class (implClass), but the 
actual package of the proxy class which effectively influences the 
bytecode invocation rights is taken from another class (targetClass).

On one hand the package of the proxy class has to be the same as 
targetClass if the proxy wants to be the nestmate of the targetClass 
(for example to have private access to the members of the nest). But 
OTOH it needs to be the same package also with implClass so that the 
above decision of the proxy invocation strategy is correct. I have a 
feeling that for protected virtual methods, this is true, because the 
type of the 0-argument of such method handle is always the lookup class. 
But am I right?

What do you think of another alternative to invoking the super protected 
method in other package: What if the LMF would decide to base the name 
of the proxy class on the implInfo.getDeclaringClass() in such case? It 
would not have to be a nestmate of any class, just in the same package 
with the method's declaring class. Consequently it would be in the same 
module as the declaring class and loaded by the same class loader. 
Therefore it would have to be WEAKLY referenced from the class loader. 
And the Lookup instance passed to bootstrap LMF method would not be 
enough for defining such class. But LMF could obtain special powers 
since it is JDK internal class...

Well, I don't know for myself. Is this situation rare enough so that 
invoking via method handle is not a drawback? It only happens when 
running JDK 13- compiled classes with JDK 15+ and in addition the method 
reference must point to protected method in a distant class.

> The targetClass is test.sub.LambdaTestSub which is *NOT* the same as 
> implClass.? That's why we change if they are in the same package.
>
> It can be invoking a method in targetClass or a method in another 
> class in the same package with package access, then implClass may or 
> may not be the same as targetClass.
>
>> I also noticed that JDK 15 patched javac transforms method reference 
>> in above code into a lambda method. But looking at the javac changes 
>> in the patch, I don't quite see where this distinction between JDK 14 
>> and patched JDK 15 javac comes from. 
>
> javac has been changed in JDK 14 to synthesize a bridge method if it's 
> a method reference to access a protected member in a remote supertype? 
> (JDK-8234729).

Ah, that was the reason I haven't seen the change in your patch. I used 
the JDK 13 javac and thought it would be the same as JDK 14 javac. My 
mistake.

So this answers the question below...


Regards, Peter

>
> BTW, the new tests relevant to this scenario are under 
> test/jdk/java/lang/invoke/lambda/superProtectedMethod.
>
>> From the changes to method 
>> com.sun.tools.javac.comp.LambdaToMethod.LambdaAnalyzerPreprocessor.ReferenceTranslationContext#needsConversionToLambda:
>>
>> ??????????? final boolean needsConversionToLambda() {
>> ??????????????? return interfaceParameterIsIntersectionOrUnionType() ||
>> ??????????????????????? isSuper ||
>> ??????????????????????? needsVarArgsConversion() ||
>> ??????????????????????? isArrayOp() ||
>> #??????????????????????? isPrivateInOtherClass() ||
>> isProtectedInSuperClassOfEnclosingClassInOtherPackage() ||
>> ??????????????????????? !receiverAccessible() ||
>> ??????????????????????? (tree.getMode() == ReferenceMode.NEW &&
>> ????????????????????????? tree.kind != ReferenceKind.ARRAY_CTOR &&
>> ????????????????????????? (tree.sym.owner.isLocal() || 
>> tree.sym.owner.isInner()));
>> ??????????? }
>>
>> ...I would draw the conclusion that conversion to lambda is performed 
>> in less cases not in more. 
>
> Jan and Srikanath may be able to explain this further.
>
>> Hm.
>>
>> Regards, Peter
>>
>> On 4/3/20 11:11 AM, Peter Levart wrote:
>>> Hi Mandy,
>>>
>>> Good work.
>>>
>>> I'm trying to find out which language use-case is covered by the 
>>> InnerClassLambdaMetafactory needing to inject method handle into the 
>>> generated proxy class to be invoked instead of proxy class directly 
>>> invoking the method:
>>>
>>> ??????? useImplMethodHandle = 
>>> !implClass.getPackageName().equals(implInfo.getDeclaringClass().getPackageName())
>>> ??????????????????????????????? && 
>>> !Modifier.isPublic(implInfo.getModifiers());
>>>
>>> If I check what implClass and implInfo get resolved to in 
>>> AbstractValidatingLambdaMetafactory, there are several cases:
>>>
>>> ??????? this.implInfo = caller.revealDirect(implMethod);
>>> ??????? switch (implInfo.getReferenceKind()) {
>>> ??????????? case REF_invokeVirtual:
>>> ??????????? case REF_invokeInterface:
>>> ??????????????? this.implClass = implMethodType.parameterType(0);
>>> ??????????????? // reference kind reported by implInfo may not match 
>>> implMethodType's first param
>>> ??????????????? // Example: implMethodType is (Cloneable)String, 
>>> implInfo is for Object.toString
>>> ??????????????? this.implKind = implClass.isInterface() ? 
>>> REF_invokeInterface : REF_invokeVirtual;
>>> ??????????????? this.implIsInstanceMethod = true;
>>> ??????????????? break;
>>> ??????????? case REF_invokeSpecial:
>>> ??????????????? // JDK-8172817: should use referenced class here, 
>>> but we don't know what it was
>>> ??????????????? this.implClass = implInfo.getDeclaringClass();
>>> ??????????????? this.implIsInstanceMethod = true;
>>>
>>> ??????????????? // Classes compiled prior to dynamic nestmate 
>>> support invokes a private instance
>>> ??????????????? // method with REF_invokeSpecial.
>>> ??????????????? //
>>> ??????????????? // invokespecial should only be used to invoke 
>>> private nestmate constructors.
>>> ??????????????? // The lambda proxy class will be defined as a 
>>> nestmate of targetClass.
>>> ??????????????? // If the method to be invoked is an instance method 
>>> of targetClass, then
>>> ??????????????? // convert to use invokevirtual or invokeinterface.
>>> ??????????????? if (targetClass == implClass && 
>>> !implInfo.getName().equals("<init>")) {
>>> ??????????????????? this.implKind = implClass.isInterface() ? 
>>> REF_invokeInterface : REF_invokeVirtual;
>>> ??????????????? } else {
>>> ??????????????????? this.implKind = REF_invokeSpecial;
>>> ??????????????? }
>>> ??????????????? break;
>>> ??????????? case REF_invokeStatic:
>>> ??????????? case REF_newInvokeSpecial:
>>> ??????????????? // JDK-8172817: should use referenced class here for 
>>> invokestatic, but we don't know what it was
>>> ??????????????? this.implClass = implInfo.getDeclaringClass();
>>> ??????????????? this.implKind = implInfo.getReferenceKind();
>>> ??????????????? this.implIsInstanceMethod = false;
>>> ??????????????? break;
>>> ??????????? default:
>>> ??????????????? throw new 
>>> LambdaConversionException(String.format("Unsupported MethodHandle 
>>> kind: %s", implInfo));
>>> ??????? }
>>>
>>>
>>> For majority of cases (REF_invokeSpecial, REF_invokeStatic, 
>>> REF_newInvokeSpecial) the this.implClass = 
>>> implInfo.getDeclaringClass();
>>>
>>> Only REF_invokeVirtual and REF_invokeInterface are possible 
>>> kandidates, right?
>>>
>>> So when does the implMethod type of parameter 0 differ from the 
>>> declaring class of the MethodHandleInfo cracked from that same 
>>> implMethod and in addition those two types leave in different packages?
>>>
>
> This is concerning the instance method and so parameter 0 is what it 
> wants to look at.
>
>>> Regards, Peter
>>>
>
> Hope this helps.
> Mandy
> [1] 
> https://bugs.openjdk.java.net/browse/JDK-8239384?focusedCommentId=14328369&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14328369
>
>


From forax at univ-mlv.fr  Sat Apr  4 12:18:34 2020
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Sat, 4 Apr 2020 14:18:34 +0200 (CEST)
Subject: RFR (XXL): 8223347: Integration of Vector API (Incubator):
 General HotSpot changes
In-Reply-To: <4e61bd37-8f8d-1f35-c1db-7826db2b0f53@oracle.com>
References: <c1bdf88c-5de2-d069-5f31-5a95c6988bf8@oracle.com>
 <579633672.1221535.1585956671384.JavaMail.zimbra@u-pem.fr>
 <4e61bd37-8f8d-1f35-c1db-7826db2b0f53@oracle.com>
Message-ID: <801330324.1421417.1586002714453.JavaMail.zimbra@u-pem.fr>

----- Mail original -----
> De: "Vladimir Ivanov" <vladimir.x.ivanov at oracle.com>
> ?: "Remi Forax" <forax at univ-mlv.fr>
> Cc: "hotspot-dev" <hotspot-dev at openjdk.java.net>, "hotspot compiler" <hotspot-compiler-dev at openjdk.java.net>,
> "panama-dev at openjdk.java.net'" <panama-dev at openjdk.java.net>
> Envoy?: Samedi 4 Avril 2020 01:52:03
> Objet: Re: RFR (XXL): 8223347: Integration of Vector API (Incubator): General HotSpot changes

>> Typo, it's --add-modules
> 
> Good catch, Remi. Thanks for the correction.
> 
>> 
>>> When user does that, JVM automatically enables Vector API support.
>>> It improves usability (user doesn't need to separately "open" the API
>>> and enable JVM support) while minimizing risks of destabilitzation from
>>> new code when the API is not used.
>> 
>> Question, what if i declare a module-info that requires "jdk.incubator.vector",
>> because in that case, i don't have to add --add-modules jdk.incubator.vector on
>> the command line, but does the VM will enable the Vector API support ?
> 
> Good point. JEP 11: "Incubator Modules" [1] states the following:
> 
> "Applications on the class path must use the --add-modules command-line
> option to request that an incubator module be resolved. Applications
> developed as modules can specify requires or requires transitive
> dependences upon an incubator module directly."
> 
> Current implementation doesn't distinguish whether the module is
> resolved for an application on the class path or by another module, so
> JVM support will be enabled by default in both cases. Do you see any
> problems with that?

So the VM supports is enabled either because there is an explicit --add-modules or because the module is transitively reachable from the root modules.
It means that it doesn't work if the module jdk.incubator.vector is loaded using a ModuleLayer. Users has to use XX:+EnableVectorSupport in that case.

regards,
R?mi

> 
> Best regards,
> Vladimir Ivanov
> 
> [1] https://openjdk.java.net/jeps/11
> 
>>> That's it! Will be happy to answer any questions.
>>>
>>> And thanks in advance for any feedback!
>> 
>> regards,
>> R?mi
>> 
>>>
>>> Best regards,
>>> Vladimir Ivanov
>>>
>>> [0]
>>> https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065345.html
>>>
>>> [1]
>>> https://mail.openjdk.java.net/pipermail/hotspot-dev/2020-April/041228.html
>>>
>>> [2] https://openjdk.java.net/jeps/338
>>>
>>> [3] https://openjdk.java.net/projects/panama/
>>>
>>> [4]
>>> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/01.intrinsics/src/java.base/share/classes/jdk/internal/vm/vector/VectorSupport.java.html
>>>
>>> [5] http://cr.openjdk.java.net/~vlivanov/talks/2018_JVMLS_VectorAPI.pdf
>>>
>>> [6] http://hg.openjdk.java.net/panama/dev/shortlog/92bbd44386e9
>>>
> >>       $ hg clone http://hg.openjdk.java.net/panama/dev/ -b vector-unstable

From Alan.Bateman at oracle.com  Sat Apr  4 12:37:29 2020
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Sat, 4 Apr 2020 13:37:29 +0100
Subject: RFR (XXL): 8223347: Integration of Vector API (Incubator):
 General HotSpot changes
In-Reply-To: <801330324.1421417.1586002714453.JavaMail.zimbra@u-pem.fr>
References: <c1bdf88c-5de2-d069-5f31-5a95c6988bf8@oracle.com>
 <579633672.1221535.1585956671384.JavaMail.zimbra@u-pem.fr>
 <4e61bd37-8f8d-1f35-c1db-7826db2b0f53@oracle.com>
 <801330324.1421417.1586002714453.JavaMail.zimbra@u-pem.fr>
Message-ID: <baae6c2d-b9b6-81f6-cdf2-1da0cf7078c3@oracle.com>

On 04/04/2020 13:18, forax at univ-mlv.fr wrote:
> :
> So the VM supports is enabled either because there is an explicit --add-modules or because the module is transitively reachable from the root modules.
> It means that it doesn't work if the module jdk.incubator.vector is loaded using a ModuleLayer. Users has to use XX:+EnableVectorSupport in that case.
>
Is jdk.incubator.vector is mapped to the boot loader? If so then it 
can't be loaded into a child layer.

-Alan


From peter.levart at gmail.com  Sat Apr  4 16:16:15 2020
From: peter.levart at gmail.com (Peter Levart)
Date: Sat, 4 Apr 2020 18:16:15 +0200
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <c941480c-c76e-0695-4a18-20d9aff224d7@gmail.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <b74f5458-94e2-02f6-5799-722910efd906@gmail.com>
 <958f4c01-85a2-5d96-7221-c9549fae76f3@gmail.com>
 <1490090a-9e0e-b51b-03d6-088e479315be@oracle.com>
 <c941480c-c76e-0695-4a18-20d9aff224d7@gmail.com>
Message-ID: <60700e61-11da-164c-c192-eb8bb5ee71bb@gmail.com>

Hi Mandy,

Just another observation of the code in InnerClassLambdaMetafactory...

For the useImplMethodHandle case you generate the protectedImplMethod 
static field to hold the MH and a static setter method:

 ??????????? mv = cw.visitMethod(ACC_PRIVATE + ACC_STATIC,
 ??????????????????????????????? "setImplMethod", DESCR_SET_IMPL_METHOD,
 ??????????????????????????????? null, null);

...but then later after you define the class you inject the MH via a 
"staticSetter" method handle:

 ??????????????? MethodHandle mh = 
lookup.findStaticSetter(lookup.lookupClass(), NAME_FIELD_IMPL_METHOD, 
MethodHandle.class);
 ??????????????? mh.invokeExact(implMethod);

So you don't invoke the generated setter method but set the field via 
special kind of method handle (equivalent to putstatic bytecode).
You can remove the setImplMethod method generation code.

Regards, Peter

On 4/4/20 12:58 PM, Peter Levart wrote:
> Hi Mandy,
>
> On 4/3/20 11:32 PM, Mandy Chung wrote:
>> Hi Peter,
>>
>> I added a JBS comment [1] to describe this special case trying to put 
>> the story together (let me know if it needs more explanation). ? More 
>> comment inline below.
>
> Thanks, this helps in establishing the historical context.
>
>>
>> On 4/3/20 4:40 AM, Peter Levart wrote:
>>> Ok, I think I found one such use-case. In the following example:
>>>
>>> package test;
>>> public class LambdaTest {
>>> ??? protected void m() {
>>> ??? }
>>> }
>>>
>>> package test.sub;
>>> public class LambdaTestSub extends test.LambdaTest {
>>> ??? public void test() {
>>> ??????? Runnable r = this::m;
>>> ??????? r.run();
>>> ??? }
>>> }
>>
>> Yes.
>>
>> This is specific for binary compatibility.?? the invocation of a 
>> protected method inherited from its supertype in a different package.
>>
>> The lambda proxy is in the same package as the target class 
>> (`test.sub` in the example above) but it has no access to 
>> `test.LambdaTest::m`.
>>
>>>
>>> ...when compiled with JDK 14 javac. In this case the implClass is 
>>> test.sub.LambdaTestSub while implInfo is "invokeVirtual 
>>> test.LambdaTest.m:()void" and the method is not public.
>>>
>>
>> In JDK 14, a lambda proxy `test.sub.LambdaTestSub$Lambda$$1234` is VM 
>> anonymous class which has a special powerful access as if the host 
>> class.?? This proxy class, even though it's not an instance of 
>> `test.LambdaTest`, can invoke? the 
>> protected`test.LambdaTest.m:()void` member.
>
> Right, the VM anonymous class "inherits" all access rights from the 
> host class, which in above example is the subclass of test.LambdaTes.
>
>>
>>> Anyway, the name of the proxy class is derived from the targetClass 
>>> (and therefore shares the same package with targetClass) which is 
>>> caller's lookup class. Is targetClass always the same as implClass 
>>> in the invokeVirtual/invokeInterface case?
>>>
>>
>> implMethod is the direct method handle describing the implementation 
>> method resolved by the VM.?? So it depends.
>>
>> In the above example, it's `test.LambdaTest.m:()void` and implClass 
>> is test.LambdaTest.
>
> Here I think, you are not quite right. First I need to clarify that we 
> are talking about the case where the method reference in above example 
> is not converted to lambda by javac, so the proxy class needs to 
> invoke the superclass method directly (without the help of lambda 
> bridge). I did an experiment and compiled the example with JDK 13 
> javac, where the patch for (JDK-8234729) is not applied yet. What I 
> get from this compilation is the following metafactory bootstrap 
> method invocation:
>
> ? 0: #35 REF_invokeStatic 
> java/lang/invoke/LambdaMetafactory.metafactory:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite;
> ??? Method arguments:
> ????? #42 ()V
> ????? #43 REF_invokeVirtual test/LambdaTest.m:()V
> ????? #42 ()V
>
> The #43 is the implMethod method handle and it is the following:
>
> ? #43 = MethodHandle?????? 5:#44????????? // REF_invokeVirtual 
> test/LambdaTest.m:()V
> ? #44 = Methodref????????? #2.#45???????? // test/LambdaTest.m:()V
> ? #45 = NameAndType??????? #46:#6???????? // m:()V
> ? #46 = Utf8?????????????? m
> ?? #2 = Class????????????? #4???????????? // test/LambdaTest
> ?? #4 = Utf8?????????????? test/LambdaTest
>
> *BUT* the class that looks up this MH from the constant pool is the 
> subclass (test.sub.LambdaTestSub) which is equivalent to the following 
> programmatic lookup:
>
> ??????? var mh = MethodHandles.lookup().findVirtual(LambdaTest.class, 
> "m", MethodType.methodType(void.class));
> ??????? System.out.println(mh.type());
>
> and this results in a method handle of the following type: 
> (LambdaTestSub)void
>
> which is correct since the method handle *MUST* check that the passed 
> in instance (this) is of type LambdaTestSub or subtype or else the 
> "protected" access would be violated.
>
> And since the ref type is REF_invokeVirtual, the 
> AbstractValidatingLambdaMetafactory assigns the following to the 
> implClass:
>
> ??????? this.implMethodType = implMethod.type();
> ??????? this.implInfo = caller.revealDirect(implMethod);
> ??????? switch (implInfo.getReferenceKind()) {
> ??????????? case REF_invokeVirtual:
> ??????????? case REF_invokeInterface:
> ??????????????? this.implClass = implMethodType.parameterType(0);
>
> ...which makes the implClass be LambdaTestSub in this case. Which is 
> correct again since we want InnerClassLambdaMetafactory to decide that 
> this is the special case for proxy to invoke the method via method 
> handle:
>
> ??????? useImplMethodHandle = 
> !implClass.getPackageName().equals(implInfo.getDeclaringClass().getPackageName())
> ??????????????????????????????? && 
> !Modifier.isPublic(implInfo.getModifiers());
>
> If implClass was test.LambdaTest as you said above, this condition 
> would evaluate to false, since implInfo is "invokeVirtual 
> test.LambdaTest.m:()void" in above case.
>
> So everything is OK, but my original question was the following: The 
> name of the generated proxy class is derived from the targetClass 
> which is the caller's lookup class. In this example the caller is 
> LambdaTestSub and this is the same as implClass in this case. Would 
> those two classes always be the same in the case where the evaluation 
> of the above `useImplMethodHandle` boolean matters? I mean, the 
> decision about whether to base the proxy invocation strategy on method 
> handle or direct bytecode invocation is based on one class 
> (implClass), but the actual package of the proxy class which 
> effectively influences the bytecode invocation rights is taken from 
> another class (targetClass).
>
> On one hand the package of the proxy class has to be the same as 
> targetClass if the proxy wants to be the nestmate of the targetClass 
> (for example to have private access to the members of the nest). But 
> OTOH it needs to be the same package also with implClass so that the 
> above decision of the proxy invocation strategy is correct. I have a 
> feeling that for protected virtual methods, this is true, because the 
> type of the 0-argument of such method handle is always the lookup 
> class. But am I right?
>
> What do you think of another alternative to invoking the super 
> protected method in other package: What if the LMF would decide to 
> base the name of the proxy class on the implInfo.getDeclaringClass() 
> in such case? It would not have to be a nestmate of any class, just in 
> the same package with the method's declaring class. Consequently it 
> would be in the same module as the declaring class and loaded by the 
> same class loader. Therefore it would have to be WEAKLY referenced 
> from the class loader. And the Lookup instance passed to bootstrap LMF 
> method would not be enough for defining such class. But LMF could 
> obtain special powers since it is JDK internal class...
>
> Well, I don't know for myself. Is this situation rare enough so that 
> invoking via method handle is not a drawback? It only happens when 
> running JDK 13- compiled classes with JDK 15+ and in addition the 
> method reference must point to protected method in a distant class.
>
>> The targetClass is test.sub.LambdaTestSub which is *NOT* the same as 
>> implClass.? That's why we change if they are in the same package.
>>
>> It can be invoking a method in targetClass or a method in another 
>> class in the same package with package access, then implClass may or 
>> may not be the same as targetClass.
>>
>>> I also noticed that JDK 15 patched javac transforms method reference 
>>> in above code into a lambda method. But looking at the javac changes 
>>> in the patch, I don't quite see where this distinction between JDK 
>>> 14 and patched JDK 15 javac comes from. 
>>
>> javac has been changed in JDK 14 to synthesize a bridge method if 
>> it's a method reference to access a protected member in a remote 
>> supertype? (JDK-8234729).
>
> Ah, that was the reason I haven't seen the change in your patch. I 
> used the JDK 13 javac and thought it would be the same as JDK 14 
> javac. My mistake.
>
> So this answers the question below...
>
>
> Regards, Peter
>
>>
>> BTW, the new tests relevant to this scenario are under 
>> test/jdk/java/lang/invoke/lambda/superProtectedMethod.
>>
>>> From the changes to method 
>>> com.sun.tools.javac.comp.LambdaToMethod.LambdaAnalyzerPreprocessor.ReferenceTranslationContext#needsConversionToLambda:
>>>
>>> ??????????? final boolean needsConversionToLambda() {
>>> ??????????????? return interfaceParameterIsIntersectionOrUnionType() ||
>>> ??????????????????????? isSuper ||
>>> ??????????????????????? needsVarArgsConversion() ||
>>> ??????????????????????? isArrayOp() ||
>>> #??????????????????????? isPrivateInOtherClass() ||
>>> isProtectedInSuperClassOfEnclosingClassInOtherPackage() ||
>>> ??????????????????????? !receiverAccessible() ||
>>> ??????????????????????? (tree.getMode() == ReferenceMode.NEW &&
>>> ????????????????????????? tree.kind != ReferenceKind.ARRAY_CTOR &&
>>> ????????????????????????? (tree.sym.owner.isLocal() || 
>>> tree.sym.owner.isInner()));
>>> ??????????? }
>>>
>>> ...I would draw the conclusion that conversion to lambda is 
>>> performed in less cases not in more. 
>>
>> Jan and Srikanath may be able to explain this further.
>>
>>> Hm.
>>>
>>> Regards, Peter
>>>
>>> On 4/3/20 11:11 AM, Peter Levart wrote:
>>>> Hi Mandy,
>>>>
>>>> Good work.
>>>>
>>>> I'm trying to find out which language use-case is covered by the 
>>>> InnerClassLambdaMetafactory needing to inject method handle into 
>>>> the generated proxy class to be invoked instead of proxy class 
>>>> directly invoking the method:
>>>>
>>>> ??????? useImplMethodHandle = 
>>>> !implClass.getPackageName().equals(implInfo.getDeclaringClass().getPackageName())
>>>> ??????????????????????????????? && 
>>>> !Modifier.isPublic(implInfo.getModifiers());
>>>>
>>>> If I check what implClass and implInfo get resolved to in 
>>>> AbstractValidatingLambdaMetafactory, there are several cases:
>>>>
>>>> ??????? this.implInfo = caller.revealDirect(implMethod);
>>>> ??????? switch (implInfo.getReferenceKind()) {
>>>> ??????????? case REF_invokeVirtual:
>>>> ??????????? case REF_invokeInterface:
>>>> ??????????????? this.implClass = implMethodType.parameterType(0);
>>>> ??????????????? // reference kind reported by implInfo may not 
>>>> match implMethodType's first param
>>>> ??????????????? // Example: implMethodType is (Cloneable)String, 
>>>> implInfo is for Object.toString
>>>> ??????????????? this.implKind = implClass.isInterface() ? 
>>>> REF_invokeInterface : REF_invokeVirtual;
>>>> ??????????????? this.implIsInstanceMethod = true;
>>>> ??????????????? break;
>>>> ??????????? case REF_invokeSpecial:
>>>> ??????????????? // JDK-8172817: should use referenced class here, 
>>>> but we don't know what it was
>>>> ??????????????? this.implClass = implInfo.getDeclaringClass();
>>>> ??????????????? this.implIsInstanceMethod = true;
>>>>
>>>> ??????????????? // Classes compiled prior to dynamic nestmate 
>>>> support invokes a private instance
>>>> ??????????????? // method with REF_invokeSpecial.
>>>> ??????????????? //
>>>> ??????????????? // invokespecial should only be used to invoke 
>>>> private nestmate constructors.
>>>> ??????????????? // The lambda proxy class will be defined as a 
>>>> nestmate of targetClass.
>>>> ??????????????? // If the method to be invoked is an instance 
>>>> method of targetClass, then
>>>> ??????????????? // convert to use invokevirtual or invokeinterface.
>>>> ??????????????? if (targetClass == implClass && 
>>>> !implInfo.getName().equals("<init>")) {
>>>> ??????????????????? this.implKind = implClass.isInterface() ? 
>>>> REF_invokeInterface : REF_invokeVirtual;
>>>> ??????????????? } else {
>>>> ??????????????????? this.implKind = REF_invokeSpecial;
>>>> ??????????????? }
>>>> ??????????????? break;
>>>> ??????????? case REF_invokeStatic:
>>>> ??????????? case REF_newInvokeSpecial:
>>>> ??????????????? // JDK-8172817: should use referenced class here 
>>>> for invokestatic, but we don't know what it was
>>>> ??????????????? this.implClass = implInfo.getDeclaringClass();
>>>> ??????????????? this.implKind = implInfo.getReferenceKind();
>>>> ??????????????? this.implIsInstanceMethod = false;
>>>> ??????????????? break;
>>>> ??????????? default:
>>>> ??????????????? throw new 
>>>> LambdaConversionException(String.format("Unsupported MethodHandle 
>>>> kind: %s", implInfo));
>>>> ??????? }
>>>>
>>>>
>>>> For majority of cases (REF_invokeSpecial, REF_invokeStatic, 
>>>> REF_newInvokeSpecial) the this.implClass = 
>>>> implInfo.getDeclaringClass();
>>>>
>>>> Only REF_invokeVirtual and REF_invokeInterface are possible 
>>>> kandidates, right?
>>>>
>>>> So when does the implMethod type of parameter 0 differ from the 
>>>> declaring class of the MethodHandleInfo cracked from that same 
>>>> implMethod and in addition those two types leave in different 
>>>> packages?
>>>>
>>
>> This is concerning the instance method and so parameter 0 is what it 
>> wants to look at.
>>
>>>> Regards, Peter
>>>>
>>
>> Hope this helps.
>> Mandy
>> [1] 
>> https://bugs.openjdk.java.net/browse/JDK-8239384?focusedCommentId=14328369&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14328369
>>
>>
>


From mandy.chung at oracle.com  Sat Apr  4 16:45:20 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Sat, 4 Apr 2020 09:45:20 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <60700e61-11da-164c-c192-eb8bb5ee71bb@gmail.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <b74f5458-94e2-02f6-5799-722910efd906@gmail.com>
 <958f4c01-85a2-5d96-7221-c9549fae76f3@gmail.com>
 <1490090a-9e0e-b51b-03d6-088e479315be@oracle.com>
 <c941480c-c76e-0695-4a18-20d9aff224d7@gmail.com>
 <60700e61-11da-164c-c192-eb8bb5ee71bb@gmail.com>
Message-ID: <53d28cd4-8e4e-b1bb-f218-a9f382ea9df6@oracle.com>



On 4/4/20 9:16 AM, Peter Levart wrote:
> Hi Mandy,
>
> Just another observation of the code in InnerClassLambdaMetafactory...
>
> For the useImplMethodHandle case you generate the protectedImplMethod 
> static field to hold the MH and a static setter method:
>
> ??????????? mv = cw.visitMethod(ACC_PRIVATE + ACC_STATIC,
> ??????????????????????????????? "setImplMethod", DESCR_SET_IMPL_METHOD,
> ??????????????????????????????? null, null);
>
> ...but then later after you define the class you inject the MH via a 
> "staticSetter" method handle:
>
> ??????????????? MethodHandle mh = 
> lookup.findStaticSetter(lookup.lookupClass(), NAME_FIELD_IMPL_METHOD, 
> MethodHandle.class);
> ??????????????? mh.invokeExact(implMethod);
>
> So you don't invoke the generated setter method but set the field via 
> special kind of method handle (equivalent to putstatic bytecode).
> You can remove the setImplMethod method generation code.
>

Good catch.?? Removed the unused method.

Mandy

> Regards, Peter


From mandy.chung at oracle.com  Sat Apr  4 18:15:14 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Sat, 4 Apr 2020 11:15:14 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <c941480c-c76e-0695-4a18-20d9aff224d7@gmail.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <b74f5458-94e2-02f6-5799-722910efd906@gmail.com>
 <958f4c01-85a2-5d96-7221-c9549fae76f3@gmail.com>
 <1490090a-9e0e-b51b-03d6-088e479315be@oracle.com>
 <c941480c-c76e-0695-4a18-20d9aff224d7@gmail.com>
Message-ID: <5fb01f1d-7d4d-b024-095a-4f429b2a06af@oracle.com>

Hi Peter,

On 4/4/20 3:58 AM, Peter Levart wrote:
>
> Here I think, you are not quite right. First I need to clarify that we 
> are talking about the case where the method reference in above example 
> is not converted to lambda by javac, so the proxy class needs to 
> invoke the superclass method directly (without the help of lambda 
> bridge). I did an experiment and compiled the example with JDK 13 
> javac, where the patch for (JDK-8234729) is not applied yet. What I 
> get from this compilation is the following metafactory bootstrap 
> method invocation:
>
> ? 0: #35 REF_invokeStatic 
> java/lang/invoke/LambdaMetafactory.metafactory:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite;
> ??? Method arguments:
> ????? #42 ()V
> ????? #43 REF_invokeVirtual test/LambdaTest.m:()V
> ????? #42 ()V
>
> The #43 is the implMethod method handle and it is the following:
>
> ? #43 = MethodHandle?????? 5:#44????????? // REF_invokeVirtual 
> test/LambdaTest.m:()V
> ? #44 = Methodref????????? #2.#45???????? // test/LambdaTest.m:()V
> ? #45 = NameAndType??????? #46:#6???????? // m:()V
> ? #46 = Utf8?????????????? m
> ?? #2 = Class????????????? #4???????????? // test/LambdaTest
> ?? #4 = Utf8?????????????? test/LambdaTest
>
> *BUT* the class that looks up this MH from the constant pool is the 
> subclass (test.sub.LambdaTestSub) which is equivalent to the following 
> programmatic lookup:
>
> ??????? var mh = MethodHandles.lookup().findVirtual(LambdaTest.class, 
> "m", MethodType.methodType(void.class));
> ??????? System.out.println(mh.type());
>
> and this results in a method handle of the following type: 
> (LambdaTestSub)void
>
> which is correct since the method handle *MUST* check that the passed 
> in instance (this) is of type LambdaTestSub or subtype or else the 
> "protected" access would be violated.
>
> And since the ref type is REF_invokeVirtual, the 
> AbstractValidatingLambdaMetafactory assigns the following to the 
> implClass:
>
> ??????? this.implMethodType = implMethod.type();
> ??????? this.implInfo = caller.revealDirect(implMethod);
> ??????? switch (implInfo.getReferenceKind()) {
> ??????????? case REF_invokeVirtual:
> ??????????? case REF_invokeInterface:
> ??????????????? this.implClass = implMethodType.parameterType(0);
>
> ...which makes the implClass be LambdaTestSub in this case. Which is 
> correct again since we want InnerClassLambdaMetafactory to decide that 
> this is the special case for proxy to invoke the method via method 
> handle:
>
> ??????? useImplMethodHandle = 
> !implClass.getPackageName().equals(implInfo.getDeclaringClass().getPackageName())
> ??????????????????????????????? && 
> !Modifier.isPublic(implInfo.getModifiers());
>
> If implClass was test.LambdaTest as you said above, this condition 
> would evaluate to false, since implInfo is "invokeVirtual 
> test.LambdaTest.m:()void" in above case.
>

My bad.? I mixed up with implClass and implInfo cracked from implMethod 
in your question.

implInfo::getDeclaringClass() returns the declaring class of the 
resolved method, which is the superclass (which is what I have been 
thinking for test.LambdaTest).

implClass is the target type of the method reference. 
AbstractValidatingLambdaMetafactory has clear comment:

 ??? final MethodType implMethodType;? // Type of the implMethod 
MethodHandle "(CC,int)String"
 ??? final Class<?> implClass;???????????????? // Class for referencing 
the implementation method "class CC"


> So everything is OK, but my original question was the following: The 
> name of the generated proxy class is derived from the targetClass 
> which is the caller's lookup class. In this example the caller is 
> LambdaTestSub and this is the same as implClass in this case.

Yes.

> Would those two classes always be the same in the case where the 
> evaluation of the above `useImplMethodHandle` boolean matters? I mean, 
> the decision about whether to base the proxy invocation strategy on 
> method handle or direct bytecode invocation is based on one class 
> (implClass), but the actual package of the proxy class which 
> effectively influences the bytecode invocation rights is taken from 
> another class (targetClass).
>
> On one hand the package of the proxy class has to be the same as 
> targetClass if the proxy wants to be the nestmate of the targetClass 
> (for example to have private access to the members of the nest). But 
> OTOH it needs to be the same package also with implClass so that the 
> above decision of the proxy invocation strategy is correct. I have a 
> feeling that for protected virtual methods, this is true, because the 
> type of the 0-argument of such method handle is always the lookup 
> class. But am I right?
>
> What do you think of another alternative to invoking the super 
> protected method in other package: What if the LMF would decide to 
> base the name of the proxy class on the implInfo.getDeclaringClass() 
> in such case? It would not have to be a nestmate of any class, just in 
> the same package with the method's declaring class. Consequently it 
> would be in the same module as the declaring class and loaded by the 
> same class loader. Therefore it would have to be WEAKLY referenced 
> from the class loader. And the Lookup instance passed to bootstrap LMF 
> method would not be enough for defining such class. But LMF could 
> obtain special powers since it is JDK internal class...
>

The implementation of the method reference invocation is logical part of 
the caller class.? I don't think spinning the proxy class in a remote 
package is the desirable thing to do (injecting a class from package A 
to package B).


> Well, I don't know for myself. Is this situation rare enough so that 
> invoking via method handle is not a drawback? It only happens when 
> running JDK 13- compiled classes with JDK 15+ and in addition the 
> method reference must point to protected method in a distant class.

There are other alternatives we considered.? This implementation is a 
just short term solution.? We plan to follow up some future enhancements 
that are the possible longer-term solutions for this issue.

1. JDK-8239580 evaluate the performance of direct method handle 
invocation rather than bytecode invocation for ALL cases

Direct invocation of the `implMethod` method handle by the lambda proxy 
was explored in JDK 8 lambda development time. It is time to remeasure 
the performance of direct method handle invocation and re-evaluate that 
approach.

2.? JDK-8230501 class data support.? The live MethodHandle can be passed 
to the hidden class to replace the current implementation to set the 
implMethod in a static field of proxy class after it's defined.

3. JDK-8199386 enhance Lookup::in to support nestmates

Hope this helps.
Mandy


From david.holmes at oracle.com  Sun Apr  5 22:55:26 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 6 Apr 2020 08:55:26 +1000
Subject: RFR [XXS]: 8242000: clean up list of environment variables
 printed in hs_err file
In-Reply-To: <AM0PR02MB42276ADDD2FF6F723CB1A86E93C60@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB42277D8406509AD8FDF83D7593C60@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <2be6613a-76db-0c1c-9b67-34baff18ce1a@oracle.com>
 <AM0PR02MB4227F9B979F2A0DB37BCAEDD93C60@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <18b11103-96f1-eee7-4d21-21826a42f758@oracle.com>
 <AM0PR02MB42276ADDD2FF6F723CB1A86E93C60@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <3ddf5f22-d2f2-d194-5dcb-78f925199651@oracle.com>

Hi Matthias,

On 2/04/2020 8:59 pm, Baesken, Matthias wrote:
> Hi David ,
> 
> new  webrev  :
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8242000.1/
> 
> 
> I looked at the other env variables listed.
> The ones from the "All platforms",  "AIX"  and "Windows"  sections have to stay .

Agreed.

It would be clearer to me (not that I'm asking you to do this) if we 
actually split these up into three semantic sections:

1. Env vars defined by the JDK that can be used to affect the behaviour 
of the JDK - e.g. the JAVA_* and CLASSPATH vars

2. Env vars defined for the system that will affect how the system 
operates in relation to the JDK e.g. LIBPATH, LD_LIBRARY_PATH, 
LD_PRELOAD, LANG etc.

3. Informational env vars that might be useful to see: USERNAME, HOSTTYPE

> I am not sure about Mac, maybe some Mac expert could comment.

I've no idea either, so fine with them staying.

> In the Linux/Solaris/BSD   section ,
>    "HOSTTYPE", "OSTYPE", "ARCH", "MACHTYPE",
> have only limited use from what I see  ,  but you find them usually  in the shell and they describe the system  a bit more.
> So I would keep  them  .

These are harmless to print and informational but I think we already 
report that information via uname. Possibly of interest if the env vars 
don't match what uname shows.

> Regarding JRE_HOME I opened
> 
> https://bugs.openjdk.java.net/browse/JDK-8242034
> 
> Remove JRE_HOME references

Thanks for doing that.

This change seems fine to me as-is.

David

> Best regards, Matthias
> 
> 
> 
> -----Original Message-----
> From: David Holmes <david.holmes at oracle.com>
> Sent: Donnerstag, 2. April 2020 12:19
> To: Baesken, Matthias <matthias.baesken at sap.com>; 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: Re: RFR [XXS]: 8242000: clean up list of environment variables printed in hs_err file
> 
> On 2/04/2020 8:11 pm, Baesken, Matthias wrote:
>> Hi David ,
>>
>> I think  "JAVA_COMPILER" is no longer needed,
>>    (see a remark here that it was used to configure the JIT compiler  https://www.ibm.com/support/knowledgecenter/SSGU8G_12.1.0/com.ibm.sqlr.doc/ids_sqr_280.htm
>>    but I cannot  find this any more in the current jdk/jdk codebase).
> 
> JAVA_COMPILER usage is ancient, back in 1.1 and 1.2 days IIRC :)
> 
>> Regarding JRE_HOME - I find  still some references in the source code, but not many .
>> Could it be that some installers / tools outside the JDK still set/use it ?
>>
>> jdk/src/hotspot/share/jfr/dcmd/jfrDcmds.cpp
>> 345 _settings("settings", "Settings file(s), e.g. profile or default. See JRE_HOME/lib/jfr", "STRING SET", false),
>>
>> jdk/src/hotspot/share/utilities/vmError.cpp
>> 83 "JAVA_HOME", "JRE_HOME", "JAVA_TOOL_OPTIONS", "_JAVA_OPTIONS", "CLASSPATH",
>>
>> /openjdk-jdk/src/java.base/share/man/java.1
>> 1813 \f[CB]JRE_HOME/lib/jfr\f[R].
> 
> These uses are just commentary, I would suggest removing JRE_HOME and
> file a RFE for JFR to cleanup its references to JRE_HOME in jfrDcmds.cpp
> and the java command manpage.
> 
> But again these are just two more that I looked at in more depth. There
> may be others that need removing too.
> 
> Cheers,
> David
> -----
> 
> 

From david.holmes at oracle.com  Sun Apr  5 23:37:32 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 6 Apr 2020 09:37:32 +1000
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <bbb2b5ef-62f2-6507-95f7-03712a382153@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
 <135a6496-5b54-9117-5cbf-e3211ddc32dd@oracle.com>
 <bbb2b5ef-62f2-6507-95f7-03712a382153@oracle.com>
Message-ID: <e242e057-de6e-493a-eb00-068b2db8c9fa@oracle.com>

Hi Leo,

On 1/04/2020 5:10 am, Leo Korinth wrote:
> Thanks for taking time to look through this!
> 
> On 31/03/2020 12:04, David Holmes wrote:
>> Hi Leo,
>>
>> I looked at the "combined change". Overall I like the cleanup, but I 
>> did find a few questionable items:
>>
>> src/hotspot/cpu/arm/interpreterRT_arm.cpp
>>
>> -????? _toGP[_last_gp++] = (*(intptr_t*)from_addr == 0) ? NULL : 
>> from_addr;
>> +????? _toGP[_last_gp++] = (*(intptr_t*)from_addr == 0) ? NULL_WORD : 
>> from_addr;
>> ????? } else {
>> -????? *_to++ = (*(intptr_t*)from_addr == 0) ? NULL : from_addr;
>> +????? *_to++ = (*(intptr_t*)from_addr == 0) ? NULL_WORD : from_addr;
>>
>> If from_addr is being treated as intptr_t* then shouldn't it be 
>> compared against NULL_WORD rather than 0?
> 
> I agree that it could be compared against NULL_WORD, do you want me to 
> change it? (I have no opinion on this issue)

If we're trying to clean up the use of 0, NULL, NULL_WORD then I think 
it should be changed.

>>
>> ---
>>
>> General query: are changes like:
>>
>> Foo f = NULL;
>>
>> to
>>
>> Foo f(NULL);
>>
>> just making the implicit constructor selection explicit? (I have to 
>> say it would nice if whenever you saw " x = NULL" that you knew x was 
>> a pointer type.)
> 
> Yes, with my original NULL two conversions would not compile. Assigning 
> is adding one extra step from direct construction. The new style is IMO 
> easier to read.

I agree.

>> ---
>>
>> src/hotspot/cpu/sparc/c1_CodeStubs_sparc.cpp
>>
>> -??? AddressLiteral addrlit(NULL, metadata_Relocation::spec(_index));
>> +??? AddressLiteral addrlit(NULL_WORD, 
>> metadata_Relocation::spec(_index));
>>
>> If the first constructor arg is an "address" which seems to be the 
>> pointer type "uchar*" then surely NULL is correct here, not NULL_WORD
> The first argument (to the constructor) can be address (u_char*), 
> jobject (_jobject*), or intptr_t (not a pointer). Surely if we send a 
> pointer (NULL), the compiler would have a hard time choosing between the 
> first two pointer constructors, right? Therefore we send a non-pointer!

Ah I looked at the wrong platform definition. sparc is a bit of mess 
here. Under the covers they all get cast to address anyway. But yes 
using NULL_WORD to explicitly select the intptr_t variant seems fine.

> The reason this worked is because I believe integer /promotion/ has 
> precedence over 0-to-pointer /implicit conversion/, but I am not too 
> knowledgable about these things... Thus the old NULL prefer to be 
> promoted to integral type! Do you agree or am I misunderstanding something?

I'm not sure how the compiler handles this.

> This is /highly/ non-intuitive and one /strong/ argument to change NULL 
> to be of the future c++11 nullptr_t type! (or my failed null_t type).
> 
>>
>> Same applies to other AddressLiteral changes.
> 
> After above reasoning, I should keep my changes right?

Yes - fine for sparc.

>> ---
>>
>> src/hotspot/share/code/codeHeapState.cpp
>>
>> This doesn't look right:
>>
>> -??? if (((_termString) != NULL) && (strlen(_termString) > 0)){\
>> +??? if (((_termString) != 0) && (strlen(_termString) > 0)){?? \
>>
>> _termString should be a "const char*" so comparison against NULL is 
>> correct.
> 
> No, _termString is mostly a string literal and not a "const char*" and 
> /can not/ in the instances of literal be NULL. Thus the (windows) 
> compiler (rightly) will complain if you use a "proper" pointer-type 
> NULL. The "technical" reason is that the array will not decay to a 
> pointer and my original NULL will thus not convert to the array. Which 
> makes sense.

Sorry yes, I was thinking of this as if it were a parameter but of 
course this is a macro with direct substitution.

> The reason I did not remove the test is that I was afraid that I either 
> missed some invocation of the macro where _termString really was of 
> pointer type, or that someone in the future would use it as such. The 
> real problem here is macros (and c strings).

Agreed.

> It is a pity that neither gcc nor clang complains (I am not sure about 
> the standard), I am bit unsure about nullptr behaviour in the future in 
> this regard, but I hope it will complain as good as my failed NULL!
> 
>> ---
>>
>> share/compiler/directivesParser.cpp
>>
>> ?> new NULL value does unfortunately not convert to method pointers
>>
>> Is that a temporary limitation? I'm not sure what progress were are 
>> making if some pointers can use NULL and others must use 0.
> 
> I believe this to be a temporary limitation, my understanding is that 
> when we get c++11 nullptr will convert to method pointers as well, I 
> could and think I should revert these changes as I did revert my 
> implementation of NULL, would you like that? My null_t type could, to my 
> knowledge, not auto convert to method pointers at least not without 
> template varargs stuff that comes in later versions of the standard.

This would look better to me as NULL.

>>
>> ---
>>
>> src/hotspot/share/jfr/jni/jfrJavaCall.cpp
>>
>> I don't understand why the NULL check has been removed:
>>
>> -? assert(_storage != NULL, "invariant");
>> +? assert(_storage_index < (int) ARRAY_SIZE(_storage), "invariant");
> 
> Because _storage can not be NULL (it is an array and not a pointer).

Oops! Yep missed that.

> I 
> feel the new assert is strictly superior! Also an example of the power 
> of static typing and more precise types.
> 
>>
>> ---
>>
>> src/hotspot/share/jvmci/jvmciCompilerToVMInit.cpp
>>
>> -??? if (value != NULL) { \
>> +??? if ((value) != 0) { \
>>
>> Unclear what type value is - in some cases char* in others I can't 
>> determine.
> 
> This is the same issue as before with string literals and macros, my 
> arguing is that I will not try to prove that value is (or in the future 
> always will be) a string literal, instead I will make the code work for 
> both string literals and pointers. I hope you do not mind the extra 
> parentheses I added around "value".

Normal style rules for macros say to use the macro arg in parentheses - 
but there are three other uses in the same macro.

>>
>> ---
>>
>> src/hotspot/share/jvmci/jvmciExceptions.hpp
>>
>> -#define JVMCI_CHECK_0????????????????????????????????? JVMCI_CHECK_(0)
>> -#define JVMCI_CHECK_NULL                               
>> JVMCI_CHECK_(NULL)
>> +#define JVMCI_CHECK_0 JVMCI_CHECK_(NULL_WORD)
>> +#define JVMCI_CHECK_NULL?????????????????????????????? JVMCI_CHECK_(0)
>>
>> These just seem plain wrong. CHECK_0 is for returning a integral zero 
>> value (e.g. jint, jbtye etc), nothing to do with intptr_t and NULL_WORD.
> 
> I feel NULL_WORD act as a more restricted 0 integral literal than the 
> zero literal itself (it does not auto converts to pointers). The size of 
> the zero constant is of less interest to me.
> 
>> Having CHECK_NULL become CHECK_(0) seems completely wrong when you 
>> consider that if you were expanding by hand in a function returning a 
>> pointer then you would write "return NULL".
> 
> I agree it is not as distinct as NULL. Now for example, JVMCIObjectArray 
> is not a pointer (strictly at least, it is a class). With my first NULL 
> it would need to go through two conversions to become a JVMCIObjectArray 
> (NULL converted to any pointer and any pointer converted to 
> JVMCIObjectArray), thus it would not compile.
> 
> There are several solutions to this including:
> 1) use 0 literal as I did.
> 2) change JVMCIObjectArray constructor to take a null(ptr)_t instead of 
> a pointer (this is a nice example of how a specific null type can make 
> that constructor /total/ and not require the assert(v == NULL, "must be 
> NULL");). Although this is a tempting solution, I had to abandon my own 
> NULL and this solution is no longer possible until c++11.
> 3) use NULL as before, and fix it when we get nullptr_t.
> 4) create another macro in some way?
> 
>> Further if this chasnge is necessary/desirable for JVMCI_CHECK_NULL 
>> then surely the same is true for the basic CHECK_NULL in exceptions.hpp ?
> 
> No, the same is not true for CHECK_NULL. I have mostly changed stuff 
> that does not compile. This still compiles.

My point is that semantically the bare CHECK_ macros and the 
JVMCI_CHECK_ macros are supposed to be doing exactly the same thing 
(which makes me wonder why they defined the JVMCI variants?). It seems 
the exact usage contexts are determining whether you need to use NULL or 
NULL_WORD and that seems wrong to me.

IMO the functions that return JVMCIObjectArray should not be returning 
"null" unless it does return an actual "null" instance of the class. 
Otherwise it should be returning JVMCIObjectArray*

> I could change CHECK_0 to use NULL_WORD though and if I do, I get 
> compiler errors. CHECK_0 is used for both pointers and integral zeroes 
> and types that implicitly converts from either. What the original 
> thought was is unclear to me; is CHECK_0 to be used to return things 
> convertible from pointers? If so, what is CHECK_NULL for?

No CHECK_0 is intended for int-returning functions (which would also 
extend to intptr_t).

>>
>> Aside:
>>
>> ./share/jvmci/jvmciCompilerToVM.cpp
>>
>> C2V_VMENTRY_0(jboolean, isInternedString, (JNIEnv* env, jobject, 
>> jobject object))
>> ?? Handle str = JVMCIENV->asConstant(JVMCIENV->wrap(object), 
>> JVMCI_CHECK_0);
>> ?? if (!java_lang_String::is_instance(str())) {
>> ???? return false;
>>
>> CHECK_0 should be CHECK_false. Ditto in isInstance, 
>> isCurrentThreadAttached,? attachCurrentThread.
>>
> 
> Maybe all this should be cleaned up separately? Maybe this macro 
> solution could be improved? And preferably not in this change?

This misuse just jumped out at me when looking at the code.

>>
>> src/hotspot/share/runtime/sweeper.hpp
>>
>> ?> gcc does unfortunately not know that our NULL is a safe format string
>>
>> -? static void log_sweep(const char* msg, const char* format = NULL, 
>> ...) ATTRIBUTE_PRINTF(2, 3);
>> +? static void log_sweep(const char* msg, const char* format = 0, ...) 
>> ATTRIBUTE_PRINTF(2, 3);
>>
>> I don't follow. This isn't "our NULL" we're using the toolchain's 
>> NULL, and format is a pointer type so this should be fine. Is gcc broken?
> 
> Sorry, my comment is unclear, this was from my original failed NULL 
> implementation. I will revert this to NULL as I expect this to work with 
> nullptr_t in the future.

Ok.

> 
>> ---
>>
>> src/hotspot/share/utilities/globalDefinitions_solstudio.hpp
>>
>> It isn't clear that the issue of passing 0 as NULL in a varargs 
>> situation is now fixed in Sol Studio.
>>
> 
> I guess tier1-3 would fail if not? Maybe that is a false assumption of 
> mine?

I would consider that a necessary but not sufficient condition.

>>
>> src/hotspot/share/utilities/linkedlist.hpp
>>
>> -??? return equal<E>(_data, t, NULL);
>> +??? return equal<E>(_data, t, 0);
>>
>> Again concerned about the method/function pointer situation.
> 
> I am not concerned :-) but I will revert this as it is not necessary now 
> or in the future as I reverted my original implementation and c++11 will 
> fix this.

Ok.

>>
>> ----
>>
>> That's all from me.
>>
>> Thanks,
>> David
>> -----
> 
> Before ending this mail, I also want to raise one question. Maybe I 
> ought to #ifdef a #define NULL __null for the gcc tool chain to give 
> better typing support until the real nullptr? I would hate to wait, but 
> maybe changing to c++11 will take longer than I hope for.

That sounds like a good intermediate step until we have C++11 support.

My main concern here is that this looks clean from a developer 
perspective: use NULL for pointers, NULL_WORD for integers and hopefully 
avoid needing to use 0 where the type system can't distinguish the two.

Thanks,
David

> 
> Many thanks!
> Leo
>>
>>
>>
>> On 30/03/2020 10:24 am, Leo Korinth wrote:
>>> Hi, could I have a review on this change to improve NULL?
>>>
>>> I try to:
>>> 1) make NULL only convert to pointer types for better type safety, 
>>> __null on gcc achieves some of this, but can be casted away.
>>> 2) define NULL in one place, in a platform independent way, no need 
>>> to define it for every tool chain.
>>> 3) make NULL be of a special type (null_t) so that one can template 
>>> specialise against it. This is for the future!
>>>
>>> I fail with this, but continue to read...
>>>
>>> I try to achieve this by defining NULL as a variable of type null_t 
>>> in the header null.hpp using ODR. null_t converts to (almost) any 
>>> pointer type.
>>>
>>> Minor negative issues:
>>> 1) No conversion to method pointers, they are only used in three 
>>> places though, I use a literal 0 instead.
>>> 2) gcc does unfortunately not know that our NULL is a safe format 
>>> string, only needed to change one place.
>>> 3) No constexpr yet, so I needed to use 0 to initialise a thread 
>>> variable.
>>>
>>> IMO positive issues:
>>> 1) I have fixed many bad usages of NULL for integer types and instead 
>>> used NULL_WORD or 0.
>>> 2) Too many implicit conversions in certain places makes the code not 
>>> compile, the changed code does look better and more idiomatic IMO.
>>> 3) Possible benign bug found where fields are initialised to NULL 
>>> (not in constructor), this should not be possible right?
>>> 4) NULL does not convert to array types, on Windows this results in 
>>> compile time error as the windows compiler does not decay arrays when 
>>> comparing to null_t, this did locate one bug, and possibly some 
>>> unnecessary comparisons in xmacros that I kept by changing NULL to 0.
>>> 5) Found a reinterpret_cast on NULL that is not allowed by the standard.
>>>
>>> I also made a new NULL_WORD in the same way as I did NULL, but 
>>> instead converting to a intptr_t of value 0. I was about to release 
>>> NULL_WORD as a separate patch, but due to the current practice of 
>>> defining NULL_WORD as NULL on many platforms, it was easier to merge 
>>> these two changes into one. Only small issues in JVMCI code for this 
>>> change, and one fix in sparc assembler. I find NULL_WORD 
>>> questionable, but it is widely used.
>>>
>>> Something to notice is the strange comment in the solstudio header 
>>> about intptr_t being wider than int on some systems that is to my 
>>> understanding false (for systems we support). I have a static assert 
>>> to check that intptr_t is indeed of (char) pointer size. I did remove 
>>> an assert that checks the size of NULL.
>>>
>>> Now, the sad thing is that my change does not work fully, I can not 
>>> make NULL constexpr before c++ 11, and thus several initialisation 
>>> order bugs appear on global variables set to NULL. I could set them 
>>> to zero or not set them at all, and they would work, but it feels 
>>> brittle and could cause future bugs when people expect NULLs on 
>>> globals to be initialised directly. Thus instead, I propose that I 
>>> revert somewhat and define NULL as 0 and change NULL_WORD to a 
>>> constant 0 of type intptr_t.
>>>
>>> When we get c++>=11 we can define NULL to nullptr of type nullptr_t, 
>>> and hopefully that will be a simple fix after this change.
>>>
>>> What do you think?
>>>
>>> Below I have given a short description of the changes:
>>> src/hotspot/cpu/aarch64/interpreterRT_aarch64.cpp
>>> correct the usage of integer type from NULL to NULL_WORD
>>>
>>> src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp
>>> too many implicit conversions
>>>
>>> src/hotspot/cpu/arm/interpreterRT_arm.cpp
>>> correct the usage of integer type from NULL to NULL_WORD
>>>
>>> src/hotspot/cpu/ppc/c1_LIRAssembler_ppc.cpp
>>> too many implicit conversions
>>>
>>> src/hotspot/cpu/sparc/c1_CodeStubs_sparc.cpp
>>> correct the usage of integer type from NULL to NULL_WORD
>>>
>>> src/hotspot/cpu/sparc/c1_LIRAssembler_sparc.cpp
>>> correct the usage of integer type from NULL to NULL_WORD, and the
>>> opposite way as well!
>>>
>>> src/hotspot/cpu/sparc/interpreterRT_sparc.cpp
>>> correct the usage of integer type from NULL to NULL_WORD
>>>
>>> src/hotspot/cpu/x86/interp_masm_x86.cpp
>>> correct the usage of integer type from NULL to NULL_WORD
>>>
>>> src/hotspot/cpu/x86/interpreterRT_x86_64.cpp
>>> correct the usage of integer type from NULL to NULL_WORD
>>>
>>> src/hotspot/cpu/x86/stubGenerator_x86_64.cpp
>>> correct the usage of integer type from NULL to NULL_WORD
>>>
>>> src/hotspot/os/linux/cgroupV1Subsystem_linux.hpp
>>> correct member initialisation to be in initialise list, gcc bug?
>>>
>>> src/hotspot/os/linux/cgroupV2Subsystem_linux.hpp
>>> correct member initialisation to be in initialise list, gcc bug?
>>>
>>> src/hotspot/os/solaris/attachListener_solaris.cpp
>>> correct the usage of integer type from NULL to 0
>>>
>>> src/hotspot/os/windows/osThread_windows.cpp
>>> correct the usage of integer type from NULL to 0
>>>
>>> src/hotspot/os/windows/os_windows.cpp
>>> correct the usage of integer type from NULL to 0
>>>
>>> src/hotspot/os/windows/pdh_interface.cpp
>>> new NULL value does unfortunately not convert to method pointers
>>>
>>> src/hotspot/share/classfile/javaClasses.cpp
>>> too many implicit conversions
>>>
>>> src/hotspot/share/classfile/moduleEntry.hpp
>>> too many implicit conversions
>>>
>>> src/hotspot/share/classfile/modules.cpp
>>> too many implicit conversions
>>>
>>> src/hotspot/share/code/codeHeapState.cpp
>>> new NULL does not compare with arrays witch is great, maybe (xmacros) 
>>> the NULL check can be removed, this change is safe though
>>>
>>> src/hotspot/share/compiler/directivesParser.cpp
>>> new NULL value does unfortunately not convert to method pointers
>>>
>>> src/hotspot/share/gc/g1/g1ConcurrentRefine.cpp
>>> remove unnecessary system include, might redefine NULL
>>>
>>> src/hotspot/share/gc/parallel/psAdaptiveSizePolicy.cpp
>>> remove unnecessary system include, might redefine NULL
>>>
>>> src/hotspot/share/gc/parallel/psParallelCompact.cpp
>>> remove unnecessary system include, might redefine NULL
>>>
>>> src/hotspot/share/gc/shared/memset_with_concurrent_readers.hpp
>>> remove unnecessary system include, might redefine NULL
>>>
>>> src/hotspot/share/gc/z/zErrno.cpp
>>> remove unnecessary system include, might redefine NULL
>>>
>>> src/hotspot/share/jfr/jni/jfrGetAllEventClasses.cpp
>>> fix bad assert found due to new NULL
>>>
>>> src/hotspot/share/jvmci/jvmciCompilerToVM.cpp
>>> too many implicit conversions, and misc changes
>>>
>>> src/hotspot/share/jvmci/jvmciCompilerToVMInit.cpp
>>> new NULL does not compare with arrays witch is great, maybe (xmacros) 
>>> the NULL check can be removed, this change is safe though
>>>
>>> src/hotspot/share/jvmci/jvmciExceptions.hpp
>>> too many implicit conversions
>>>
>>> src/hotspot/share/oops/access.hpp reinterpret_cast is not safe on 
>>> NULL, rewrite
>>>
>>> src/hotspot/share/oops/constantPool.cpp
>>> too many implicit conversions
>>>
>>> src/hotspot/share/oops/cpCache.cpp
>>> too many implicit conversions
>>>
>>> src/hotspot/share/oops/klass.cpp
>>> too many implicit conversions
>>>
>>> src/hotspot/share/oops/oopsHierarchy.hpp
>>> add null_t overload to oop class
>>>
>>> src/hotspot/share/prims/jni.cpp
>>> too many implicit conversions
>>>
>>> src/hotspot/share/prims/jvm.cpp
>>> too many implicit conversions
>>>
>>> src/hotspot/share/prims/methodHandles.cpp
>>> too many implicit conversions
>>>
>>> src/hotspot/share/runtime/os.cpp
>>> correct the usage of integer type from NULL to 0
>>>
>>> src/hotspot/share/runtime/os.inline.hpp
>>> #undef NULL set by system headers, use the new NULL instead
>>>
>>> src/hotspot/share/runtime/sweeper.hpp
>>> gcc does unfortunately not know that our NULL is a safe format string
>>>
>>> src/hotspot/share/runtime/thread.cpp
>>> our conversion can not be a constexpr before c++11, use 0 instead
>>>
>>> src/hotspot/share/utilities/debug.hpp
>>> stop using system headers for NULL
>>>
>>> src/hotspot/share/utilities/globalDefinitions.cpp
>>> change to static asserts, and remove one sizeof(NULL) assert.
>>> I will change the assert on sizeof(uintptr_t) to sizeof(NULL_WORD) in 
>>> the next webrev as it is both wrong and not clear (I defined 
>>> NULL_WORD to be of type intptr_t and not uintptr_t)
>>>
>>> src/hotspot/share/utilities/globalDefinitions.hpp
>>> include the new definition of NULL *after* system includes, also do 
>>> redefinition after include guard end
>>>
>>> src/hotspot/share/utilities/globalDefinitions_gcc.hpp
>>> remove old definition of NULL
>>>
>>> src/hotspot/share/utilities/globalDefinitions_solstudio.hpp
>>> remove old definition of NULL
>>>
>>> src/hotspot/share/utilities/globalDefinitions_visCPP.hpp
>>> remove old definition of NULL
>>>
>>> src/hotspot/share/utilities/globalDefinitions_xlc.hpp
>>> remove old definition of NULL
>>>
>>> src/hotspot/share/utilities/linkedlist.hpp
>>> new NULL value does unfortunately not convert to method pointers
>>>
>>> src/hotspot/share/utilities/null.hpp
>>> new NULL
>>>
>>> test/hotspot/gtest/gc/z/test_zAddress.cpp
>>> correct the usage of integer type from NULL to NULL_WORD
>>>
>>> test/hotspot/gtest/unittest.hpp
>>> do not use system NULL
>>>
>>> I have a follow up patch that tries to (totally) remove usage of 
>>> system header NULL, but that is orthogonal to this patch so I did not 
>>> want to mix them if the approach is not liked.
>>>
>>> Bug:
>>> https://bugs.openjdk.java.net/browse/JDK-8240110
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~lkorinth/8240110/part1 (NULL)
>>> http://cr.openjdk.java.net/~lkorinth/8240110/part2 (NULL_WORD)
>>> http://cr.openjdk.java.net/~lkorinth/8240110/part3 (reverse somewhat)
>>> http://cr.openjdk.java.net/~lkorinth/8240110/0???? (combined change 
>>> that I will create incremental changes against)
>>>
>>> Testing:
>>> tier 1-3
>>>
>>> Thanks,
>>> Leo

From vladimir.x.ivanov at oracle.com  Mon Apr  6 08:02:10 2020
From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov)
Date: Mon, 6 Apr 2020 11:02:10 +0300
Subject: RFR (XXL): 8223347: Integration of Vector API (Incubator):
 General HotSpot changes
In-Reply-To: <baae6c2d-b9b6-81f6-cdf2-1da0cf7078c3@oracle.com>
References: <c1bdf88c-5de2-d069-5f31-5a95c6988bf8@oracle.com>
 <579633672.1221535.1585956671384.JavaMail.zimbra@u-pem.fr>
 <4e61bd37-8f8d-1f35-c1db-7826db2b0f53@oracle.com>
 <801330324.1421417.1586002714453.JavaMail.zimbra@u-pem.fr>
 <baae6c2d-b9b6-81f6-cdf2-1da0cf7078c3@oracle.com>
Message-ID: <cbfc0480-1eb4-94ed-1171-23eea68bf080@oracle.com>


>> So the VM supports is enabled either because there is an explicit 
>> --add-modules or because the module is transitively reachable from the 
>> root modules.
>> It means that it doesn't work if the module jdk.incubator.vector is 
>> loaded using a ModuleLayer. Users has to use XX:+EnableVectorSupport 
>> in that case.
>>
> Is jdk.incubator.vector is mapped to the boot loader? If so then it 
> can't be loaded into a child layer.

Yes, jdk.incubator.vector is a boot module. The reason to put it there 
is to be able to trust final instance fields by the JVM.

Since the module extensively uses VM annotations, it should be either 
boot or platform module in order to have access to them, but in case of 
platform module existing logic for trusting final instance fields 
doesn't work and all such fields should be marked as @Stable instead.

Best regards,
Vladimir Ivanov

From alexandr.miloslavskiy at gmail.com  Mon Apr  6 11:03:34 2020
From: alexandr.miloslavskiy at gmail.com (Alexander Miloslavskiy)
Date: Mon, 6 Apr 2020 13:03:34 +0200
Subject: Linux crash native stacks only have 1 line
Message-ID: <805a02c4-a765-dbd7-52f7-553e232f87d0@gmail.com>

Hello,

I'm working on fixing various crashes in our Java-based product.

There's one thing that consistently cripples my research: hs_err_pid.log 
files will always contain just _one_ line of native stack on Linux. On 
macOS, stack is fine.

So in most cases, I have no useful stack (the nearest Java frame is 
something like GTK message loop).

Is there any switch to fix that? Can someone explain why is this happening?

For some background, this is what I see in a typical crash log:
--------
Stack: [0x00007fcab1a14000,0x00007fcab1c14000],  sp=0x00007fcab1c117a0, 
free space=2037k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, 
C=native code)
C  [libgobject-2.0.so.0+0x36471]  g_type_check_instance+0x41

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
J 2857  org.eclipse.swt.internal.gtk.OS._g_main_context_iteration(JZ)Z 
(0 bytes) @ 0x00007fca9d9f6ec4 [0x00007fca9d9f6e80+0x44]
J 6486 C2 org.eclipse.swt.widgets.Display.readAndDispatch()Z (73 bytes) 
@ 0x00007fca9dbf3a24 [0x00007fca9dbf3760+0x2c4]
<...>
--------

This is pretty much as good as "it crashed somewhere".

And this is the actual stack when I finally learn how to reproduce and 
attach GDB:

--------
#0  g_type_check_instance
#1  g_signal_emit_by_name
#2  gtk_im_multicontext_retrieve_surrounding_cb
#3  _gtk_marshal_BOOLEAN__VOID
#4  g_closure_invoke
#5  signal_emit_unlocked_R
#6  g_signal_emit_valist
#7  g_signal_emit
#8  _request_surrounding_text
#9  _request_surrounding_text
#10 _request_surrounding_text_after_focus
#11 gdk_threads_dispatch
#12 g_main_dispatch
#13 g_main_context_dispatch
#14 g_main_context_iterate
#15 g_main_context_iteration
#16 Java_org_eclipse_swt_internal_gtk_OS__1g_1main_1context_1iteration
--------

From david.holmes at oracle.com  Mon Apr  6 13:21:28 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 6 Apr 2020 23:21:28 +1000
Subject: Linux crash native stacks only have 1 line
In-Reply-To: <805a02c4-a765-dbd7-52f7-553e232f87d0@gmail.com>
References: <805a02c4-a765-dbd7-52f7-553e232f87d0@gmail.com>
Message-ID: <6da1abd3-9b77-9daf-c21c-4913227fbce2@oracle.com>

Hello,

On 6/04/2020 9:03 pm, Alexander Miloslavskiy wrote:
> Hello,
> 
> I'm working on fixing various crashes in our Java-based product.
> 
> There's one thing that consistently cripples my research: hs_err_pid.log 
> files will always contain just _one_ line of native stack on Linux. On 
> macOS, stack is fine.
> 
> So in most cases, I have no useful stack (the nearest Java frame is 
> something like GTK message loop).
> 
> Is there any switch to fix that? Can someone explain why is this happening?

I suggest you ask the Eclipse/SWT folk the answer to this as it seems to 
be their native code that isn't allowing an informative stacktrace to be 
obtained. It may have been compiled without the necessary frame 
back-pointers or otherwise lacks the metadata needed to reconstruct the 
stack.

David

> For some background, this is what I see in a typical crash log:
> --------
> Stack: [0x00007fcab1a14000,0x00007fcab1c14000],? sp=0x00007fcab1c117a0, 
> free space=2037k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, 
> C=native code)
> C? [libgobject-2.0.so.0+0x36471]? g_type_check_instance+0x41
> 
> Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
> J 2857? org.eclipse.swt.internal.gtk.OS._g_main_context_iteration(JZ)Z 
> (0 bytes) @ 0x00007fca9d9f6ec4 [0x00007fca9d9f6e80+0x44]
> J 6486 C2 org.eclipse.swt.widgets.Display.readAndDispatch()Z (73 bytes) 
> @ 0x00007fca9dbf3a24 [0x00007fca9dbf3760+0x2c4]
> <...>
> --------
> 
> This is pretty much as good as "it crashed somewhere".
> 
> And this is the actual stack when I finally learn how to reproduce and 
> attach GDB:
> 
> --------
> #0? g_type_check_instance
> #1? g_signal_emit_by_name
> #2? gtk_im_multicontext_retrieve_surrounding_cb
> #3? _gtk_marshal_BOOLEAN__VOID
> #4? g_closure_invoke
> #5? signal_emit_unlocked_R
> #6? g_signal_emit_valist
> #7? g_signal_emit
> #8? _request_surrounding_text
> #9? _request_surrounding_text
> #10 _request_surrounding_text_after_focus
> #11 gdk_threads_dispatch
> #12 g_main_dispatch
> #13 g_main_context_dispatch
> #14 g_main_context_iterate
> #15 g_main_context_iteration
> #16 Java_org_eclipse_swt_internal_gtk_OS__1g_1main_1context_1iteration
> --------

From alexandr.miloslavskiy at gmail.com  Mon Apr  6 13:25:03 2020
From: alexandr.miloslavskiy at gmail.com (Alexander Miloslavskiy)
Date: Mon, 6 Apr 2020 15:25:03 +0200
Subject: Linux crash native stacks only have 1 line
In-Reply-To: <6da1abd3-9b77-9daf-c21c-4913227fbce2@oracle.com>
References: <805a02c4-a765-dbd7-52f7-553e232f87d0@gmail.com>
 <6da1abd3-9b77-9daf-c21c-4913227fbce2@oracle.com>
Message-ID: <d29477cb-0596-6204-fc07-23ef7ec61f5b@gmail.com>

On 06.04.2020 15:21, David Holmes wrote:
> I suggest you ask the Eclipse/SWT folk the answer to this as it seems to 
> be their native code that isn't allowing an informative stacktrace to be 
> obtained. It may have been compiled without the necessary frame 
> back-pointers or otherwise lacks the metadata needed to reconstruct the 
> stack.

I'm also dealing with SWT, so there's no need to ask other guys :)

Just to make sure, is it really possible that _bottom_ stack frame can 
break stack unwinding? I expected that stack unwinding starts from
$RIP $RSP (that is top frame), and these frames are linux GTK in my case.

From volker.simonis at gmail.com  Mon Apr  6 13:42:21 2020
From: volker.simonis at gmail.com (Volker Simonis)
Date: Mon, 6 Apr 2020 15:42:21 +0200
Subject: Linux crash native stacks only have 1 line
In-Reply-To: <d29477cb-0596-6204-fc07-23ef7ec61f5b@gmail.com>
References: <805a02c4-a765-dbd7-52f7-553e232f87d0@gmail.com>
 <6da1abd3-9b77-9daf-c21c-4913227fbce2@oracle.com>
 <d29477cb-0596-6204-fc07-23ef7ec61f5b@gmail.com>
Message-ID: <CA+3eh11tSrjTHCMFSSOeUdk=z1WKUdTkxpWoaH9-6k6wmsxccw@mail.gmail.com>

Hi Alexander,

one of the problems may be that the native code you're crashed in was
compiled with "-fomit-frame-pointer" which means that the frame pointer
register can't be used for unwinding. gdb can use other, additional sources
of information from the various ELF tables or from the debugging
information which aren't available when HotSpot creates its native stack
trace.

I you already managed to attach gdb and get a correct back trace at the
point where your program crashes, you can just as well debug the HotSpot
stack tracing routine "print_native_stack()" in the file "debug.cpp" to see
what's the actual problem :) Before doing  that, I'd check your reproducer
with jdk14 or better the tip revision just to make sure you're not hunting
a problem which has already been fixed upstream.

In general it's always advisable to mention the exact jdk version and
distribution you've used.

Best regards,
Volker


On Mon, Apr 6, 2020 at 3:25 PM Alexander Miloslavskiy <
alexandr.miloslavskiy at gmail.com> wrote:

> On 06.04.2020 15:21, David Holmes wrote:
> > I suggest you ask the Eclipse/SWT folk the answer to this as it seems to
> > be their native code that isn't allowing an informative stacktrace to be
> > obtained. It may have been compiled without the necessary frame
> > back-pointers or otherwise lacks the metadata needed to reconstruct the
> > stack.
>
> I'm also dealing with SWT, so there's no need to ask other guys :)
>
> Just to make sure, is it really possible that _bottom_ stack frame can
> break stack unwinding? I expected that stack unwinding starts from
> $RIP $RSP (that is top frame), and these frames are linux GTK in my case.
>

From coleen.phillimore at oracle.com  Mon Apr  6 14:45:22 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Mon, 6 Apr 2020 10:45:22 -0400
Subject: RFR (XXL): 8223347: Integration of Vector API (Incubator):
 General HotSpot changes
In-Reply-To: <c1bdf88c-5de2-d069-5f31-5a95c6988bf8@oracle.com>
References: <c1bdf88c-5de2-d069-5f31-5a95c6988bf8@oracle.com>
Message-ID: <32e55f12-c705-60ea-4306-fadf6cf97faa@oracle.com>


http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/all.00-03/src/hotspot/share/runtime/vmStructs.cpp.udiff.html

Since I don't see SA support for VectorNodes in the SA, and I presume 
you can debug this with gdb, does this code need to be added?

Thanks,
Coleen

On 4/3/20 7:12 PM, Vladimir Ivanov wrote:
> Hi,
>
> Following up on review requests of API [0] and Java implementation [1] 
> for Vector API (JEP 338 [2]), here's a request for review of general 
> HotSpot changes (in shared code) required for supporting the API:
>
>
> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/all.00-03/ 
>
>
> (First of all, to set proper expectations: since the JEP is still in 
> Candidate state, the intention is to initiate preliminary round(s) of 
> review to inform the community and gather feedback before sending out 
> final/official RFRs once the JEP is Targeted to a release.)
>
> Vector API (being developed in Project Panama [3]) relies on JVM 
> support to utilize optimal vector hardware instructions at runtime. It 
> interacts with JVM through intrinsics (declared in 
> jdk.internal.vm.vector.VectorSupport [4]) which expose vector 
> operations support in C2 JIT-compiler.
>
> As Paul wrote earlier: "A vector intrinsic is an internal low-level 
> vector operation. The last argument to the intrinsic is fall back 
> behavior in Java, implementing the scalar operation over the number of 
> elements held by the vector.? Thus, If the intrinsic is not supported 
> in C2 for the other arguments then the Java implementation is executed 
> (the Java implementation is always executed when running in the 
> interpreter or for C1)."
>
> The rest of JVM support is about aggressively optimizing vector boxes 
> to minimize (ideally eliminate) the overhead of boxing for vector values.
> It's a stop-the-gap solution for vector box elimination problem until 
> inline classes arrive. Vector classes are value-based and in the 
> longer term will be migrated to inline classes once the support 
> becomes available.
>
> Vector API talk from JVMLS'18 [5] contains brief overview of JVM 
> implementation and some details.
>
> Complete implementation resides in vector-unstable branch of 
> panama/dev repository [6].
>
> Now to gory details (the patch is split in multiple "sub-webrevs"):
>
> ===========================================================
>
> (1) 
> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/00.backend.shared/
>
> Ideal vector nodes for new operations introduced by Vector API.
>
> (Platform-specific back end support will be posted for review 
> separately).
>
> ===========================================================
>
> (2) 
> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/01.intrinsics/
>
> JVM Java interface (VectorSupport) and intrinsic support in C2.
>
> Vector instances are initially represented as VectorBox macro nodes 
> and "unboxing" is represented by VectorUnbox node. It simplifies 
> vector box elimination analysis and the nodes are expanded later right 
> before EA pass.
>
> Vectors have 2-level on-heap representation: for the vector value 
> primitive array is used as a backing storage and it is encapsulated in 
> a typed wrapper (e.g., Int256Vector - vector of 8 ints - contains a 
> int[8] instance which is used to store vector value).
>
> Unless VectorBox node goes away, it needs to be expanded into an 
> allocation eventually, but it is a pure node and doesn't have any JVM 
> state associated with it. The problem is solved by keeping JVM state 
> separately in a VectorBoxAllocate node associated with VectorBox node 
> and use it during expansion.
>
> Also, to simplify vector box elimination, inlining of vector reboxing 
> calls (VectorSupport::maybeRebox) is delayed until the analysis is over.
>
> ===========================================================
>
> (3) 
> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/02.vbox_elimination/
>
> Vector box elimination analysis implementation. (Brief overview: 
> slides #36-42 [5].)
>
> The main part is devoted to scalarization across safepoints and 
> rematerialization support during deoptimization. In C2-generated code 
> vector operations work with raw vector values which live in registers 
> or spilled on the stack and it allows to avoid boxing/unboxing when a 
> vector value is alive across a safepoint. As with other values, 
> there's just a location of the vector value at the safepoint and 
> vector type information recorded in the relevant nmethod metadata and 
> all the heavy-lifting happens only when rematerialization takes place.
>
> The analysis preserves object identity invariants except during 
> aggressive reboxing (guarded by -XX:+EnableAggressiveReboxing).
>
> (Aggressive reboxing is crucial for cases when vectors "escape": it 
> allocates a fresh instance at every escape point thus enabling 
> original instance to go away.)
>
> ===========================================================
>
> (4) 
> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/03.module.hotspot/
>
> HotSpot changes for jdk.incubator.vector module. Vector support is 
> makred experimental and turned off by default. JEP 338 proposes the 
> API to be released as an incubator module, so a user has to specify 
> "--add-module jdk.incubator.vector" on the command line to be able to 
> use it.
> When user does that, JVM automatically enables Vector API support.
> It improves usability (user doesn't need to separately "open" the API 
> and enable JVM support) while minimizing risks of destabilitzation 
> from new code when the API is not used.
>
>
> That's it! Will be happy to answer any questions.
>
> And thanks in advance for any feedback!
>
> Best regards,
> Vladimir Ivanov
>
> [0] 
> https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065345.html
>
> [1] 
> https://mail.openjdk.java.net/pipermail/hotspot-dev/2020-April/041228.html
>
> [2] https://openjdk.java.net/jeps/338
>
> [3] https://openjdk.java.net/projects/panama/
>
> [4] 
> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/01.intrinsics/src/java.base/share/classes/jdk/internal/vm/vector/VectorSupport.java.html
>
> [5] http://cr.openjdk.java.net/~vlivanov/talks/2018_JVMLS_VectorAPI.pdf
>
> [6] http://hg.openjdk.java.net/panama/dev/shortlog/92bbd44386e9
>
> ??? $ hg clone http://hg.openjdk.java.net/panama/dev/ -b vector-unstable


From vladimir.x.ivanov at oracle.com  Mon Apr  6 16:00:03 2020
From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov)
Date: Mon, 6 Apr 2020 19:00:03 +0300
Subject: RFR (XXL): 8223347: Integration of Vector API (Incubator):
 General HotSpot changes
In-Reply-To: <32e55f12-c705-60ea-4306-fadf6cf97faa@oracle.com>
References: <c1bdf88c-5de2-d069-5f31-5a95c6988bf8@oracle.com>
 <32e55f12-c705-60ea-4306-fadf6cf97faa@oracle.com>
Message-ID: <a74fa57c-805a-0743-3f54-bc62c9daa1f1@oracle.com>

Hi Coleen,

> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/all.00-03/src/hotspot/share/runtime/vmStructs.cpp.udiff.html 
> 
> 
> Since I don't see SA support for VectorNodes in the SA, and I presume 
> you can debug this with gdb, does this code need to be added?

No, the code is neither required for Vector API implementation nor for 
C2 support.

The declarations are added to mainly to align newly introduced classes 
with existing classes which are exposed to SA.

Best regards,
Vladimir Ivanov

> On 4/3/20 7:12 PM, Vladimir Ivanov wrote:
>> Hi,
>>
>> Following up on review requests of API [0] and Java implementation [1] 
>> for Vector API (JEP 338 [2]), here's a request for review of general 
>> HotSpot changes (in shared code) required for supporting the API:
>>
>>
>> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/all.00-03/ 
>>
>>
>> (First of all, to set proper expectations: since the JEP is still in 
>> Candidate state, the intention is to initiate preliminary round(s) of 
>> review to inform the community and gather feedback before sending out 
>> final/official RFRs once the JEP is Targeted to a release.)
>>
>> Vector API (being developed in Project Panama [3]) relies on JVM 
>> support to utilize optimal vector hardware instructions at runtime. It 
>> interacts with JVM through intrinsics (declared in 
>> jdk.internal.vm.vector.VectorSupport [4]) which expose vector 
>> operations support in C2 JIT-compiler.
>>
>> As Paul wrote earlier: "A vector intrinsic is an internal low-level 
>> vector operation. The last argument to the intrinsic is fall back 
>> behavior in Java, implementing the scalar operation over the number of 
>> elements held by the vector.? Thus, If the intrinsic is not supported 
>> in C2 for the other arguments then the Java implementation is executed 
>> (the Java implementation is always executed when running in the 
>> interpreter or for C1)."
>>
>> The rest of JVM support is about aggressively optimizing vector boxes 
>> to minimize (ideally eliminate) the overhead of boxing for vector values.
>> It's a stop-the-gap solution for vector box elimination problem until 
>> inline classes arrive. Vector classes are value-based and in the 
>> longer term will be migrated to inline classes once the support 
>> becomes available.
>>
>> Vector API talk from JVMLS'18 [5] contains brief overview of JVM 
>> implementation and some details.
>>
>> Complete implementation resides in vector-unstable branch of 
>> panama/dev repository [6].
>>
>> Now to gory details (the patch is split in multiple "sub-webrevs"):
>>
>> ===========================================================
>>
>> (1) 
>> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/00.backend.shared/ 
>>
>>
>> Ideal vector nodes for new operations introduced by Vector API.
>>
>> (Platform-specific back end support will be posted for review 
>> separately).
>>
>> ===========================================================
>>
>> (2) 
>> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/01.intrinsics/ 
>>
>>
>> JVM Java interface (VectorSupport) and intrinsic support in C2.
>>
>> Vector instances are initially represented as VectorBox macro nodes 
>> and "unboxing" is represented by VectorUnbox node. It simplifies 
>> vector box elimination analysis and the nodes are expanded later right 
>> before EA pass.
>>
>> Vectors have 2-level on-heap representation: for the vector value 
>> primitive array is used as a backing storage and it is encapsulated in 
>> a typed wrapper (e.g., Int256Vector - vector of 8 ints - contains a 
>> int[8] instance which is used to store vector value).
>>
>> Unless VectorBox node goes away, it needs to be expanded into an 
>> allocation eventually, but it is a pure node and doesn't have any JVM 
>> state associated with it. The problem is solved by keeping JVM state 
>> separately in a VectorBoxAllocate node associated with VectorBox node 
>> and use it during expansion.
>>
>> Also, to simplify vector box elimination, inlining of vector reboxing 
>> calls (VectorSupport::maybeRebox) is delayed until the analysis is over.
>>
>> ===========================================================
>>
>> (3) 
>> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/02.vbox_elimination/ 
>>
>>
>> Vector box elimination analysis implementation. (Brief overview: 
>> slides #36-42 [5].)
>>
>> The main part is devoted to scalarization across safepoints and 
>> rematerialization support during deoptimization. In C2-generated code 
>> vector operations work with raw vector values which live in registers 
>> or spilled on the stack and it allows to avoid boxing/unboxing when a 
>> vector value is alive across a safepoint. As with other values, 
>> there's just a location of the vector value at the safepoint and 
>> vector type information recorded in the relevant nmethod metadata and 
>> all the heavy-lifting happens only when rematerialization takes place.
>>
>> The analysis preserves object identity invariants except during 
>> aggressive reboxing (guarded by -XX:+EnableAggressiveReboxing).
>>
>> (Aggressive reboxing is crucial for cases when vectors "escape": it 
>> allocates a fresh instance at every escape point thus enabling 
>> original instance to go away.)
>>
>> ===========================================================
>>
>> (4) 
>> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/03.module.hotspot/ 
>>
>>
>> HotSpot changes for jdk.incubator.vector module. Vector support is 
>> makred experimental and turned off by default. JEP 338 proposes the 
>> API to be released as an incubator module, so a user has to specify 
>> "--add-module jdk.incubator.vector" on the command line to be able to 
>> use it.
>> When user does that, JVM automatically enables Vector API support.
>> It improves usability (user doesn't need to separately "open" the API 
>> and enable JVM support) while minimizing risks of destabilitzation 
>> from new code when the API is not used.
>>
>>
>> That's it! Will be happy to answer any questions.
>>
>> And thanks in advance for any feedback!
>>
>> Best regards,
>> Vladimir Ivanov
>>
>> [0] 
>> https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065345.html 
>>
>>
>> [1] 
>> https://mail.openjdk.java.net/pipermail/hotspot-dev/2020-April/041228.html 
>>
>>
>> [2] https://openjdk.java.net/jeps/338
>>
>> [3] https://openjdk.java.net/projects/panama/
>>
>> [4] 
>> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/01.intrinsics/src/java.base/share/classes/jdk/internal/vm/vector/VectorSupport.java.html 
>>
>>
>> [5] http://cr.openjdk.java.net/~vlivanov/talks/2018_JVMLS_VectorAPI.pdf
>>
>> [6] http://hg.openjdk.java.net/panama/dev/shortlog/92bbd44386e9
>>
>> ??? $ hg clone http://hg.openjdk.java.net/panama/dev/ -b vector-unstable
> 

From glaubitz at physik.fu-berlin.de  Mon Apr  6 18:09:01 2020
From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz)
Date: Mon, 6 Apr 2020 20:09:01 +0200
Subject: RFR: 8199138: Add RISC-V support to Zero
Message-ID: <12d60510-3e13-8ba1-e50b-e145126e7ff9@physik.fu-berlin.de>

Hello!

Please review this small change which adds basic support for the riscv64
target for Linux/Zero [1].

I have reduced the complexity of the patch as some of the changes from
the previous change set are not necessary, in particular the changes
to config.{guess,sub}, the definition of EM_RISCV (which is already defined
by the Linux kernel headers now provided the kernel is recent enough).

Additionally, I'm leaving the change for os::get_summary_cpu_info() out
as I would like to clean up this code a bit first - it's rather inconsistent
considering whether the arch override is used for Hotspot arches only or
also for Zero arches.

Thanks,
Adrian

> [1] http://cr.openjdk.java.net/~glaubitz/8199138/webrev.02/

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz at debian.org
`. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

From erik.joelsson at oracle.com  Mon Apr  6 18:20:30 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Mon, 6 Apr 2020 11:20:30 -0700
Subject: RFR: 8199138: Add RISC-V support to Zero
In-Reply-To: <12d60510-3e13-8ba1-e50b-e145126e7ff9@physik.fu-berlin.de>
References: <12d60510-3e13-8ba1-e50b-e145126e7ff9@physik.fu-berlin.de>
Message-ID: <ec69612c-1bbd-43ff-c39e-19faaf4eea11@oracle.com>

Build change looks good.

/Erik

On 2020-04-06 11:09, John Paul Adrian Glaubitz wrote:
> Hello!
>
> Please review this small change which adds basic support for the riscv64
> target for Linux/Zero [1].
>
> I have reduced the complexity of the patch as some of the changes from
> the previous change set are not necessary, in particular the changes
> to config.{guess,sub}, the definition of EM_RISCV (which is already defined
> by the Linux kernel headers now provided the kernel is recent enough).
>
> Additionally, I'm leaving the change for os::get_summary_cpu_info() out
> as I would like to clean up this code a bit first - it's rather inconsistent
> considering whether the arch override is used for Hotspot arches only or
> also for Zero arches.
>
> Thanks,
> Adrian
>
>> [1] http://cr.openjdk.java.net/~glaubitz/8199138/webrev.02/

From thomas.stuefe at gmail.com  Mon Apr  6 18:44:07 2020
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Mon, 6 Apr 2020 20:44:07 +0200
Subject: RFR: 8199138: Add RISC-V support to Zero
In-Reply-To: <12d60510-3e13-8ba1-e50b-e145126e7ff9@physik.fu-berlin.de>
References: <12d60510-3e13-8ba1-e50b-e145126e7ff9@physik.fu-berlin.de>
Message-ID: <CAA-vtUz0MQnYkZbMv2Zag8ZZbb_V3d419=CDKKwSAXQ8DfwGtg@mail.gmail.com>

Hi Adrian, looks good to me.

Cheers, Thomas

On Mon, Apr 6, 2020, 20:11 John Paul Adrian Glaubitz <
glaubitz at physik.fu-berlin.de> wrote:

> Hello!
>
> Please review this small change which adds basic support for the riscv64
> target for Linux/Zero [1].
>
> I have reduced the complexity of the patch as some of the changes from
> the previous change set are not necessary, in particular the changes
> to config.{guess,sub}, the definition of EM_RISCV (which is already defined
> by the Linux kernel headers now provided the kernel is recent enough).
>
> Additionally, I'm leaving the change for os::get_summary_cpu_info() out
> as I would like to clean up this code a bit first - it's rather
> inconsistent
> considering whether the arch override is used for Hotspot arches only or
> also for Zero arches.
>
> Thanks,
> Adrian
>
> > [1] http://cr.openjdk.java.net/~glaubitz/8199138/webrev.02/
>
> --
>  .''`.  John Paul Adrian Glaubitz
> : :' :  Debian Developer - glaubitz at debian.org
> `. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
>   `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913
>

From mandy.chung at oracle.com  Mon Apr  6 18:54:11 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Mon, 6 Apr 2020 11:54:11 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <e99edbae-99f2-cd81-10fd-3b261057449c@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <bd511487-c84f-2973-98ff-03b1bedbd92d@oracle.com>
 <d84c046a-6425-8d58-aea7-9d5c5d28a493@oracle.com>
 <b85c551a-f76c-f520-129a-984409678098@oracle.com>
 <93aca329-cf62-a199-4744-5a86f0b96998@oracle.com>
 <e99edbae-99f2-cd81-10fd-3b261057449c@oracle.com>
Message-ID: <f045ee81-ed2f-3374-647d-80fa6abcab16@oracle.com>

On 4/6/20 9:56 AM, serguei.spitsyn at oracle.com wrote:
>
> The suggested fix is:
> http://cr.openjdk.java.net/~sspitsyn/webrevs/2020/valhalla-jdi-regression-8242166.1/
>

This patch looks okay. I'll include in my local patch.

On 4/6/20 11:00 AM, Chris Plummer wrote:
>
> I think that's fine but I don't think it should be done in the context 
> of this Vahalla webrev since it has nothing to do with Vahalla. I'd 
> suggest filing an RFE and pushing it to jdk/jdk. Easier to track that 
> way if there are issues down the road.
>

I am okay to follow up as a separate RFE.

thanks
Mandy

From sandhya.viswanathan at intel.com  Mon Apr  6 19:01:17 2020
From: sandhya.viswanathan at intel.com (Viswanathan, Sandhya)
Date: Mon, 6 Apr 2020 19:01:17 +0000
Subject: RFR (XXL): 8223347: Integration of Vector API (Incubator): x86
 backend changes
In-Reply-To: <CACR9jGOVPZjEGCO9ots2BEgxVOgtNj2oeA8WKbrH-aK=7wdQbw@mail.gmail.com>
References: <MW3PR11MB47452D16DFBF1AD1B18A79ACEFC40@MW3PR11MB4745.namprd11.prod.outlook.com>
 <CACR9jGOVPZjEGCO9ots2BEgxVOgtNj2oeA8WKbrH-aK=7wdQbw@mail.gmail.com>
Message-ID: <MW3PR11MB4745B9E2A67EB557946D3AFAEFC20@MW3PR11MB4745.namprd11.prod.outlook.com>

Hi Vivek,

Thanks for the feedback. This patch is for mainline jdk.

Best Regards,
Sandhya

From: Vivek Deshpande <viv.desh at gmail.com>
Sent: Monday, April 06, 2020 11:55 AM
To: Viswanathan, Sandhya <sandhya.viswanathan at intel.com>
Cc: hotspot-compiler-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; hotspot-dev <hotspot-dev at openjdk.java.net>
Subject: Re: RFR (XXL): 8223347: Integration of Vector API (Incubator): x86 backend changes

Hi Sandhya

I looked at the patch over the weekend. It looks good to me and a lot of work is involved.
I have a question. Is this patch intended to panama/dev or mainline jdk?
Nit: macroAssembler_x86.cpp has extra line at 115.

Regards,
Vivek
OpenJDK id: vdeshpande

On Fri, Apr 3, 2020 at 5:18 PM Viswanathan, Sandhya <sandhya.viswanathan at intel.com<mailto:sandhya.viswanathan at intel.com>> wrote:
Hi,


Following up on review requests of API [0], Java implementation [1] and

General Hotspot changes[3] for Vector API, here's a request for review

of x86 backend changes required for supporting the API:



JEP: https://openjdk.java.net/jeps/338

JBS: https://bugs.openjdk.java.net/browse/JDK-8223347

Webrev:http://cr.openjdk.java.net/~sviswanathan/VAPI_RFR/x86_webrev/webrev.00/



Complete implementation resides in vector-unstable branch of

panama/dev repository [3].

Looking forward to your feedback.

Best Regards,
Sandhya


[0]  https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065345.html



[1]  https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-April/065587.html



[2]  https://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2020-April/037798.html



[3]  https://openjdk.java.net/projects/panama/

       $ hg clone http://hg.openjdk.java.net/panama/dev/ -b vector-unstable






--
Thanks and Regards,

Vivek Deshpande
viv.desh at gmail.com<mailto:viv.desh at gmail.com>

From serguei.spitsyn at oracle.com  Mon Apr  6 19:08:46 2020
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Mon, 6 Apr 2020 12:08:46 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <f045ee81-ed2f-3374-647d-80fa6abcab16@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <bd511487-c84f-2973-98ff-03b1bedbd92d@oracle.com>
 <d84c046a-6425-8d58-aea7-9d5c5d28a493@oracle.com>
 <b85c551a-f76c-f520-129a-984409678098@oracle.com>
 <93aca329-cf62-a199-4744-5a86f0b96998@oracle.com>
 <e99edbae-99f2-cd81-10fd-3b261057449c@oracle.com>
 <f045ee81-ed2f-3374-647d-80fa6abcab16@oracle.com>
Message-ID: <a68c6ea6-cf46-637c-1bb9-77f4d79953c3@oracle.com>

On 4/6/20 11:54, Mandy Chung wrote:
> On 4/6/20 9:56 AM, serguei.spitsyn at oracle.com wrote:
>>
>> The suggested fix is:
>> http://cr.openjdk.java.net/~sspitsyn/webrevs/2020/valhalla-jdi-regression-8242166.1/
>>
>
> This patch looks okay. I'll include in my local patch.
>
> On 4/6/20 11:00 AM, Chris Plummer wrote:
>>
>> I think that's fine but I don't think it should be done in the 
>> context of this Vahalla webrev since it has nothing to do with 
>> Vahalla. I'd suggest filing an RFE and pushing it to jdk/jdk. Easier 
>> to track that way if there are issues down the road.
>>
>
> I am okay to follow up as a separate RFE.

Filed RFE:
 ? https://bugs.openjdk.java.net/browse/JDK-8242241
 ??? add assert to ClassUnloadEventImpl::className

Thanks,
Serguei

>
> thanks
> Mandy


From chris.plummer at oracle.com  Mon Apr  6 20:10:35 2020
From: chris.plummer at oracle.com (Chris Plummer)
Date: Mon, 6 Apr 2020 13:10:35 -0700
Subject: RFR(S) 8242235: Disable SA testing on Solaris. Remove JDK-8193639
 entries from ProblemList.txt
Message-ID: <213cbe04-beb1-bc55-a34e-14789426a294@oracle.com>

Hello,

Please review the following:

https://bugs.openjdk.java.net/browse/JDK-8242235
http://cr.openjdk.java.net/~cjplummer/8242235/webrev.00

The SA problem list entries due to JDK-8193639 [1] have been a burden to 
maintain, and are badly out of date. Easiest solution is to remove them 
and disable SA testing on Solaris, which is what all the entries were 
suppose to be doing in the first place (although not always correctly).

Once this change is pushed, I'll add a note to JDK-8193639 [1] so anyone 
working on it will know to undo the change in Platform.java.

[1] https://bugs.openjdk.java.net/browse/JDK-8193639

thanks,

Chris

From harold.seigel at oracle.com  Mon Apr  6 20:12:20 2020
From: harold.seigel at oracle.com (Harold Seigel)
Date: Mon, 6 Apr 2020 16:12:20 -0400
Subject: RFR(S) 8242235: Disable SA testing on Solaris. Remove JDK-8193639
 entries from ProblemList.txt
In-Reply-To: <213cbe04-beb1-bc55-a34e-14789426a294@oracle.com>
References: <213cbe04-beb1-bc55-a34e-14789426a294@oracle.com>
Message-ID: <086fc26a-c9bb-3ebc-9965-af829caafab2@oracle.com>

Looks good.

Thanks, Harold

On 4/6/2020 4:10 PM, Chris Plummer wrote:
> Hello,
>
> Please review the following:
>
> https://bugs.openjdk.java.net/browse/JDK-8242235
> http://cr.openjdk.java.net/~cjplummer/8242235/webrev.00
>
> The SA problem list entries due to JDK-8193639 [1] have been a burden 
> to maintain, and are badly out of date. Easiest solution is to remove 
> them and disable SA testing on Solaris, which is what all the entries 
> were suppose to be doing in the first place (although not always 
> correctly).
>
> Once this change is pushed, I'll add a note to JDK-8193639 [1] so 
> anyone working on it will know to undo the change in Platform.java.
>
> [1] https://bugs.openjdk.java.net/browse/JDK-8193639
>
> thanks,
>
> Chris

From daniel.daugherty at oracle.com  Mon Apr  6 20:21:30 2020
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Mon, 6 Apr 2020 16:21:30 -0400
Subject: RFR(S) 8242235: Disable SA testing on Solaris. Remove JDK-8193639
 entries from ProblemList.txt
In-Reply-To: <213cbe04-beb1-bc55-a34e-14789426a294@oracle.com>
References: <213cbe04-beb1-bc55-a34e-14789426a294@oracle.com>
Message-ID: <28a65798-b46e-d482-4ece-09906d0ada78@oracle.com>

On 4/6/20 4:10 PM, Chris Plummer wrote:
> Hello,
>
> Please review the following:
>
> https://bugs.openjdk.java.net/browse/JDK-8242235
> http://cr.openjdk.java.net/~cjplummer/8242235/webrev.00

test/hotspot/jtreg/ProblemList.txt
 ??? No comments.

test/jdk/ProblemList.txt
 ??? No comments.

test/lib/jdk/test/lib/Platform.java
 ??? No comments.

Thumbs up.

In the bug report, you said:

> although due to test renames, moves, and additions, there are actually
> still about 8 tests running on Solaris, but they shouldn't be. 

Perhaps those 8 tests are not affected by the bug that causes JDK-8193639.
I don't see any recent links to JDK-8193639 that are valid failures.
There are some links to JDK-8193639 for non-Solaris platforms which means
those links are wrong, but none for Solaris (that I see).

Dan


>
> The SA problem list entries due to JDK-8193639 [1] have been a burden 
> to maintain, and are badly out of date. Easiest solution is to remove 
> them and disable SA testing on Solaris, which is what all the entries 
> were suppose to be doing in the first place (although not always 
> correctly).
>
> Once this change is pushed, I'll add a note to JDK-8193639 [1] so 
> anyone working on it will know to undo the change in Platform.java.
>
> [1] https://bugs.openjdk.java.net/browse/JDK-8193639
>
> thanks,
>
> Chris


From alexey.menkov at oracle.com  Mon Apr  6 20:35:18 2020
From: alexey.menkov at oracle.com (Alex Menkov)
Date: Mon, 6 Apr 2020 13:35:18 -0700
Subject: RFR(S) 8242235: Disable SA testing on Solaris. Remove JDK-8193639
 entries from ProblemList.txt
In-Reply-To: <213cbe04-beb1-bc55-a34e-14789426a294@oracle.com>
References: <213cbe04-beb1-bc55-a34e-14789426a294@oracle.com>
Message-ID: <7d0794cb-ef75-d4a1-4a24-90ad2028bdd5@oracle.com>

LGTM

--alex

On 04/06/2020 13:10, Chris Plummer wrote:
> Hello,
> 
> Please review the following:
> 
> https://bugs.openjdk.java.net/browse/JDK-8242235
> http://cr.openjdk.java.net/~cjplummer/8242235/webrev.00
> 
> The SA problem list entries due to JDK-8193639 [1] have been a burden to 
> maintain, and are badly out of date. Easiest solution is to remove them 
> and disable SA testing on Solaris, which is what all the entries were 
> suppose to be doing in the first place (although not always correctly).
> 
> Once this change is pushed, I'll add a note to JDK-8193639 [1] so anyone 
> working on it will know to undo the change in Platform.java.
> 
> [1] https://bugs.openjdk.java.net/browse/JDK-8193639
> 
> thanks,
> 
> Chris

From chris.plummer at oracle.com  Mon Apr  6 21:08:11 2020
From: chris.plummer at oracle.com (Chris Plummer)
Date: Mon, 6 Apr 2020 14:08:11 -0700
Subject: RFR(S) 8242235: Disable SA testing on Solaris. Remove JDK-8193639
 entries from ProblemList.txt
In-Reply-To: <28a65798-b46e-d482-4ece-09906d0ada78@oracle.com>
References: <213cbe04-beb1-bc55-a34e-14789426a294@oracle.com>
 <28a65798-b46e-d482-4ece-09906d0ada78@oracle.com>
Message-ID: <7782696d-c456-20a8-f70a-7376300e6498@oracle.com>

On 4/6/20 1:21 PM, Daniel D. Daugherty wrote:
> On 4/6/20 4:10 PM, Chris Plummer wrote:
>> Hello,
>>
>> Please review the following:
>>
>> https://bugs.openjdk.java.net/browse/JDK-8242235
>> http://cr.openjdk.java.net/~cjplummer/8242235/webrev.00
>
> test/hotspot/jtreg/ProblemList.txt
> ??? No comments.
>
> test/jdk/ProblemList.txt
> ??? No comments.
>
> test/lib/jdk/test/lib/Platform.java
> ??? No comments.
>
> Thumbs up.
>
> In the bug report, you said:
>
>> although due to test renames, moves, and additions, there are actually
>> still about 8 tests running on Solaris, but they shouldn't be. 
>
> Perhaps those 8 tests are not affected by the bug that causes 
> JDK-8193639.
> I don't see any recent links to JDK-8193639 that are valid failures.
> There are some links to JDK-8193639 for non-Solaris platforms which means
> those links are wrong, but none for Solaris (that I see).
Hi Dan,

Thanks for the review.

On closer look it's more like 3 tests since 5 of the 8 don't really 
count. One was a new test I wrote that I haven't pushed yet (and didn't 
problem list because I wanted to do this CR first). Two are tests that 
are not run for other reasons (but are also incorrectly problem listed 
for 8193639). Two are tests that only recently started running on 
solaris due to splitting out into #id0 and #id1 runs, so the problem 
list is not properly filtering them.

So maybe 3 tests have been running for a while without causing issues, 
but I honestly don't think it's worth the effort to figure out which 
tests are truly affected by JDK-8193639, and try to make the problem 
list correct to allow them to run.

thanks,

Chris

>
> Dan
>
>
>>
>> The SA problem list entries due to JDK-8193639 [1] have been a burden 
>> to maintain, and are badly out of date. Easiest solution is to remove 
>> them and disable SA testing on Solaris, which is what all the entries 
>> were suppose to be doing in the first place (although not always 
>> correctly).
>>
>> Once this change is pushed, I'll add a note to JDK-8193639 [1] so 
>> anyone working on it will know to undo the change in Platform.java.
>>
>> [1] https://bugs.openjdk.java.net/browse/JDK-8193639
>>
>> thanks,
>>
>> Chris
>



From daniel.daugherty at oracle.com  Mon Apr  6 21:17:46 2020
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Mon, 6 Apr 2020 17:17:46 -0400
Subject: RFR(S) 8242235: Disable SA testing on Solaris. Remove JDK-8193639
 entries from ProblemList.txt
In-Reply-To: <7782696d-c456-20a8-f70a-7376300e6498@oracle.com>
References: <213cbe04-beb1-bc55-a34e-14789426a294@oracle.com>
 <28a65798-b46e-d482-4ece-09906d0ada78@oracle.com>
 <7782696d-c456-20a8-f70a-7376300e6498@oracle.com>
Message-ID: <28b2f8ac-6c5e-dcf9-eef6-9a35cdbc2055@oracle.com>

On 4/6/20 5:08 PM, Chris Plummer wrote:
> On 4/6/20 1:21 PM, Daniel D. Daugherty wrote:
>> On 4/6/20 4:10 PM, Chris Plummer wrote:
>>> Hello,
>>>
>>> Please review the following:
>>>
>>> https://bugs.openjdk.java.net/browse/JDK-8242235
>>> http://cr.openjdk.java.net/~cjplummer/8242235/webrev.00
>>
>> test/hotspot/jtreg/ProblemList.txt
>> ??? No comments.
>>
>> test/jdk/ProblemList.txt
>> ??? No comments.
>>
>> test/lib/jdk/test/lib/Platform.java
>> ??? No comments.
>>
>> Thumbs up.
>>
>> In the bug report, you said:
>>
>>> although due to test renames, moves, and additions, there are actually
>>> still about 8 tests running on Solaris, but they shouldn't be. 
>>
>> Perhaps those 8 tests are not affected by the bug that causes 
>> JDK-8193639.
>> I don't see any recent links to JDK-8193639 that are valid failures.
>> There are some links to JDK-8193639 for non-Solaris platforms which 
>> means
>> those links are wrong, but none for Solaris (that I see).
> Hi Dan,
>
> Thanks for the review.
>
> On closer look it's more like 3 tests since 5 of the 8 don't really 
> count. One was a new test I wrote that I haven't pushed yet (and 
> didn't problem list because I wanted to do this CR first). Two are 
> tests that are not run for other reasons (but are also incorrectly 
> problem listed for 8193639). Two are tests that only recently started 
> running on solaris due to splitting out into #id0 and #id1 runs, so 
> the problem list is not properly filtering them.
>
> So maybe 3 tests have been running for a while without causing issues, 
> but I honestly don't think it's worth the effort to figure out which 
> tests are truly affected by JDK-8193639, and try to make the problem 
> list correct to allow them to run.

No problem. Your call on what to do here.

Dan


>
> thanks,
>
> Chris
>
>>
>> Dan
>>
>>
>>>
>>> The SA problem list entries due to JDK-8193639 [1] have been a 
>>> burden to maintain, and are badly out of date. Easiest solution is 
>>> to remove them and disable SA testing on Solaris, which is what all 
>>> the entries were suppose to be doing in the first place (although 
>>> not always correctly).
>>>
>>> Once this change is pushed, I'll add a note to JDK-8193639 [1] so 
>>> anyone working on it will know to undo the change in Platform.java.
>>>
>>> [1] https://bugs.openjdk.java.net/browse/JDK-8193639
>>>
>>> thanks,
>>>
>>> Chris
>>
>
>


From glaubitz at physik.fu-berlin.de  Mon Apr  6 22:08:20 2020
From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz)
Date: Tue, 7 Apr 2020 00:08:20 +0200
Subject: RFR: 8242244: Remove redundant ELF machine definitions
Message-ID: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>

Hi!

Please review this small change which removes redundant ELF machine
definitions for EM_AARCH64 and EM_486 on Linux and Solaris [1].

Both EM_AARCH64 and EM_486 are defined by the kernel headers on both
Solaris 11.3 or higher (verified on gcc211.fsffrance.org) and Linux 3.12
or higher [2], so I think it's safe to remove those definitions from
the JDK codebase unless the JDK is supposed to be built on systems
older than that.

Thanks,
Adrian

> [1] http://cr.openjdk.java.net/~glaubitz/8242244/webrev.01/
> [2] https://elixir.bootlin.com/linux/v3.12/source/include/uapi/linux/elf-em.h

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz at debian.org
`. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

From glaubitz at physik.fu-berlin.de  Mon Apr  6 23:19:41 2020
From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz)
Date: Tue, 7 Apr 2020 01:19:41 +0200
Subject: RFR: 8199138: Add RISC-V support to Zero
In-Reply-To: <12d60510-3e13-8ba1-e50b-e145126e7ff9@physik.fu-berlin.de>
References: <12d60510-3e13-8ba1-e50b-e145126e7ff9@physik.fu-berlin.de>
Message-ID: <14005cc5-b39f-937d-f09e-ae97a255f26b@physik.fu-berlin.de>

Hello!

On 4/6/20 8:09 PM, John Paul Adrian Glaubitz wrote:
> I have reduced the complexity of the patch as some of the changes from
> the previous change set are not necessary, in particular the changes
> to config.{guess,sub}, the definition of EM_RISCV (which is already defined
> by the Linux kernel headers now provided the kernel is recent enough).

After checking various kernel headers of older but still supported enterprise
Linux distributions such as SLE-12 and RHEL-7, I think it's probably better
to include the redundant EM_RISCV definition to avoid build problems on these
platforms [1]. I just want to be on the safe side.

The build changes are unchanged.

Adrian

> [1] http://cr.openjdk.java.net/~glaubitz/8199138/webrev.03/

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz at debian.org
`. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

From ekaterina.pavlova at oracle.com  Tue Apr  7 03:12:49 2020
From: ekaterina.pavlova at oracle.com (Ekaterina Pavlova)
Date: Mon, 6 Apr 2020 20:12:49 -0700
Subject: RFR (XXL): 8223347: Integration of Vector API (Incubator):
 General HotSpot changes
In-Reply-To: <c1bdf88c-5de2-d069-5f31-5a95c6988bf8@oracle.com>
References: <c1bdf88c-5de2-d069-5f31-5a95c6988bf8@oracle.com>
Message-ID: <3b6f8501-5d90-0a14-ed21-5978567d95d2@oracle.com>

Hi Vladimir,

what kind of testing has been done to verify these changes?
Taking into account the changes are quite large and touch share code
running hs compiler and perhaps runtime tiers would be very advisable.

thanks,
-katya

On 4/3/20 4:12 PM, Vladimir Ivanov wrote:
> Hi,
> 
> Following up on review requests of API [0] and Java implementation [1] for Vector API (JEP 338 [2]), here's a request for review of general HotSpot changes (in shared code) required for supporting the API:
> 
> 
> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/all.00-03/
> 
> (First of all, to set proper expectations: since the JEP is still in Candidate state, the intention is to initiate preliminary round(s) of review to inform the community and gather feedback before sending out final/official RFRs once the JEP is Targeted to a release.)
> 
> Vector API (being developed in Project Panama [3]) relies on JVM support to utilize optimal vector hardware instructions at runtime. It interacts with JVM through intrinsics (declared in jdk.internal.vm.vector.VectorSupport [4]) which expose vector operations support in C2 JIT-compiler.
> 
> As Paul wrote earlier: "A vector intrinsic is an internal low-level vector operation. The last argument to the intrinsic is fall back behavior in Java, implementing the scalar operation over the number of elements held by the vector.? Thus, If the intrinsic is not supported in C2 for the other arguments then the Java implementation is executed (the Java implementation is always executed when running in the interpreter or for C1)."
> 
> The rest of JVM support is about aggressively optimizing vector boxes to minimize (ideally eliminate) the overhead of boxing for vector values.
> It's a stop-the-gap solution for vector box elimination problem until inline classes arrive. Vector classes are value-based and in the longer term will be migrated to inline classes once the support becomes available.
> 
> Vector API talk from JVMLS'18 [5] contains brief overview of JVM implementation and some details.
> 
> Complete implementation resides in vector-unstable branch of panama/dev repository [6].
> 
> Now to gory details (the patch is split in multiple "sub-webrevs"):
> 
> ===========================================================
> 
> (1) http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/00.backend.shared/
> 
> Ideal vector nodes for new operations introduced by Vector API.
> 
> (Platform-specific back end support will be posted for review separately).
> 
> ===========================================================
> 
> (2) http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/01.intrinsics/
> 
> JVM Java interface (VectorSupport) and intrinsic support in C2.
> 
> Vector instances are initially represented as VectorBox macro nodes and "unboxing" is represented by VectorUnbox node. It simplifies vector box elimination analysis and the nodes are expanded later right before EA pass.
> 
> Vectors have 2-level on-heap representation: for the vector value primitive array is used as a backing storage and it is encapsulated in a typed wrapper (e.g., Int256Vector - vector of 8 ints - contains a int[8] instance which is used to store vector value).
> 
> Unless VectorBox node goes away, it needs to be expanded into an allocation eventually, but it is a pure node and doesn't have any JVM state associated with it. The problem is solved by keeping JVM state separately in a VectorBoxAllocate node associated with VectorBox node and use it during expansion.
> 
> Also, to simplify vector box elimination, inlining of vector reboxing calls (VectorSupport::maybeRebox) is delayed until the analysis is over.
> 
> ===========================================================
> 
> (3) http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/02.vbox_elimination/
> 
> Vector box elimination analysis implementation. (Brief overview: slides #36-42 [5].)
> 
> The main part is devoted to scalarization across safepoints and rematerialization support during deoptimization. In C2-generated code vector operations work with raw vector values which live in registers or spilled on the stack and it allows to avoid boxing/unboxing when a vector value is alive across a safepoint. As with other values, there's just a location of the vector value at the safepoint and vector type information recorded in the relevant nmethod metadata and all the heavy-lifting happens only when rematerialization takes place.
> 
> The analysis preserves object identity invariants except during aggressive reboxing (guarded by -XX:+EnableAggressiveReboxing).
> 
> (Aggressive reboxing is crucial for cases when vectors "escape": it allocates a fresh instance at every escape point thus enabling original instance to go away.)
> 
> ===========================================================
> 
> (4) http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/03.module.hotspot/
> 
> HotSpot changes for jdk.incubator.vector module. Vector support is makred experimental and turned off by default. JEP 338 proposes the API to be released as an incubator module, so a user has to specify "--add-module jdk.incubator.vector" on the command line to be able to use it.
> When user does that, JVM automatically enables Vector API support.
> It improves usability (user doesn't need to separately "open" the API and enable JVM support) while minimizing risks of destabilitzation from new code when the API is not used.
> 
> 
> That's it! Will be happy to answer any questions.
> 
> And thanks in advance for any feedback!
> 
> Best regards,
> Vladimir Ivanov
> 
> [0] https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065345.html
> 
> [1] https://mail.openjdk.java.net/pipermail/hotspot-dev/2020-April/041228.html
> 
> [2] https://openjdk.java.net/jeps/338
> 
> [3] https://openjdk.java.net/projects/panama/
> 
> [4] http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/01.intrinsics/src/java.base/share/classes/jdk/internal/vm/vector/VectorSupport.java.html
> 
> [5] http://cr.openjdk.java.net/~vlivanov/talks/2018_JVMLS_VectorAPI.pdf
> 
> [6] http://hg.openjdk.java.net/panama/dev/shortlog/92bbd44386e9
> 
>  ??? $ hg clone http://hg.openjdk.java.net/panama/dev/ -b vector-unstable


From thomas.stuefe at gmail.com  Tue Apr  7 04:51:37 2020
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Tue, 7 Apr 2020 06:51:37 +0200
Subject: RFR: 8199138: Add RISC-V support to Zero
In-Reply-To: <14005cc5-b39f-937d-f09e-ae97a255f26b@physik.fu-berlin.de>
References: <12d60510-3e13-8ba1-e50b-e145126e7ff9@physik.fu-berlin.de>
 <14005cc5-b39f-937d-f09e-ae97a255f26b@physik.fu-berlin.de>
Message-ID: <CAA-vtUxn8w6khRMF2AnTk=a73s0vYk2yLPN+4iEoH3AOTihcMg@mail.gmail.com>

Looks still good.

..Thomas

On Tue 7. Apr 2020 at 01:20, John Paul Adrian Glaubitz <
glaubitz at physik.fu-berlin.de> wrote:

> Hello!
>
> On 4/6/20 8:09 PM, John Paul Adrian Glaubitz wrote:
> > I have reduced the complexity of the patch as some of the changes from
> > the previous change set are not necessary, in particular the changes
> > to config.{guess,sub}, the definition of EM_RISCV (which is already
> defined
> > by the Linux kernel headers now provided the kernel is recent enough).
>
> After checking various kernel headers of older but still supported
> enterprise
> Linux distributions such as SLE-12 and RHEL-7, I think it's probably better
> to include the redundant EM_RISCV definition to avoid build problems on
> these
> platforms [1]. I just want to be on the safe side.
>
> The build changes are unchanged.
>
> Adrian
>
> > [1] http://cr.openjdk.java.net/~glaubitz/8199138/webrev.03/
>
> --
>  .''`.  John Paul Adrian Glaubitz
> : :' :  Debian Developer - glaubitz at debian.org
> `. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
>   `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913
>

From christoph.langer at sap.com  Tue Apr  7 06:17:03 2020
From: christoph.langer at sap.com (Langer, Christoph)
Date: Tue, 7 Apr 2020 06:17:03 +0000
Subject: PING: RFR(S): 8240360: NativeLibraryEvent has wrong library name
 on Linux
In-Reply-To: <872c2924-db54-452d-aa73-1a52e4919b5d.denghui.ddh@alibaba-inc.com>
References: <0741e423-3bfc-4bbc-8929-54500f387cb7.denghui.ddh@alibaba-inc.com>
 <9f586a0e-11ff-1dce-575a-508686a3427f@oss.nttdata.com>
 <b8a5a357-40af-4359-bf6f-f978a85524ae.denghui.ddh@alibaba-inc.com>,
 <774cc959-d635-0699-34d4-f85bb83de764@oss.nttdata.com>,
 <ae0e9d38-b560-425f-a571-97a54803f11b.denghui.ddh@alibaba-inc.com>
 <872c2924-db54-452d-aa73-1a52e4919b5d.denghui.ddh@alibaba-inc.com>
Message-ID: <AM0PR02MB5714C7BE26C54F959CA973238AC30@AM0PR02MB5714.eurprd02.prod.outlook.com>

Hi Denghui,

I think your fix is good.

Best regards
Christoph

> -----Original Message-----
> From: hotspot-runtime-dev <hotspot-runtime-dev-
> bounces at openjdk.java.net> On Behalf Of Denghui Dong
> Sent: Dienstag, 31. M?rz 2020 12:02
> To: hotspot-jfr-dev <hotspot-jfr-dev at openjdk.java.net>; hotspot-runtime-
> dev at openjdk.java.net; hotspot-dev at openjdk.java.net
> Subject: Re: PING: RFR(S): 8240360: NativeLibraryEvent has wrong library
> name on Linux
> 
> Hi,
> 
> Could anyone review it, need one more reviewer.
> 
> Webrev: http://cr.openjdk.java.net/~ddong/8240360/webrev.01/
> JBS: https://bugs.openjdk.java.net/browse/JDK-8240360
> 
> 
> Thanks
> Denghui
> 
> 
> ------------------------------------------------------------------
> From:???(??) <denghui.ddh at alibaba-inc.com>
> Send Time:2020?3?23?(???) 21:26
> To:hotspot-jfr-dev <hotspot-jfr-dev at openjdk.java.net>; Yasumasa Suenaga
> <suenaga at oss.nttdata.com>
> Subject:Re: PING: RFR(S): 8240360: NativeLibraryEvent has wrong library
> name on Linux
> 
> Thank you Yasumasa.
> 
> Can I get one more reviewer please?
> 
> Cheers,
> Denghui Dong
> ------------------------------------------------------------------
> From:Yasumasa Suenaga <suenaga at oss.nttdata.com>
> Send Time:2020?3?23?(???) 20:25
> To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-jfr-dev
> <hotspot-jfr-dev at openjdk.java.net>
> Subject:Re: PING: RFR(S): 8240360: NativeLibraryEvent has wrong library
> name on Linux
> 
> Hi Denghui,
> 
> On 2020/03/23 18:08, Denghui Dong wrote:
> > Hi Yasumasa,
> >
> > Thanks for the review.
> >
> > I agree with you that it's necessary to filter those lines in which the file is
> not a shared object by check the exec bit.
> > I think we should create another bug to trace the problem that you
> mentioned.
> 
> I filed it:
> https://bugs.openjdk.java.net/browse/JDK-8241439
> 
> I think we might use dl_iterate_phdr() for the fix. I will try it.
> 
> 
> > BTW, Could you sponsor it?
> 
> Sure, but you need to get one more reviewer before pushing.
> 
> 
> Thanks,
> 
> Yasumasa
> 
> 
> > Thanks,
> > Denghui Dong
> >
> >     ------------------------------------------------------------------
> >     From:Yasumasa Suenaga <suenaga at oss.nttdata.com>
> >     Send Time:2020?3?23?(???) 16:31
> >     To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-jfr-dev
> <hotspot-jfr-dev at openjdk.java.net>
> >     Subject:Re: PING: RFR(S): 8240360: NativeLibraryEvent has wrong library
> name on Linux
> >
> >     Hi Denghui,
> >
> >     Your change looks good.
> >
> >     However, we might need to update os::get_loaded_modules_info()
> more.
> >     Currently, we would get multiple jdk.NativeLibrary events from 1 library
> as below:
> >
> >     ```
> >     jdk.NativeLibrary {
> >         startTime = 16:59:06.621
> >         name = "/usr/lib64/libc-2.30.so"
> >         baseAddress = 0x7FDB06ACC000
> >         topAddress = 0x7FDB06AF1000
> >     }
> >
> >     jdk.NativeLibrary {
> >         startTime = 16:59:06.621
> >         name = "/usr/lib64/libc-2.30.so"
> >         baseAddress = 0x7FDB06AF1000
> >         topAddress = 0x7FDB06C40000
> >     }
> >
> >          :
> >
> >     ```
> >
> >     In addition, we would get the events even if they are not shared library
> as below:
> >
> >     ```
> >     jdk.NativeLibrary {
> >         startTime = 16:59:06.621
> >         name = "/home/ysuenaga/OpenJDK/jdk/build/linux-x86_64-server-
> fastdebug/images/jdk/lib/server/cl
> >     asses.jsa"
> >         baseAddress = 0x800000000
> >         topAddress = 0x800007000
> >     }
> >     ```
> >
> >     os::get_loaded_modules_info() has introduced in JDK-8056242, and it
> says this function is for dll/so/dylibs.
> >     In fact, os::get_loaded_modules_info() uses EnumProcessModules() in
> Windows.
> >
> >     So I think we need to check exec bit, and merge same library to one
> event.
> >     But I think it is another issue, and I'm not sure my think is correct.
> >
> >     Thus I agree with your change now.
> >
> >
> >     Thanks,
> >
> >     Yasumasa (ysuenaga)
> >
> >
> >     On 2020/03/20 11:12, Denghui Dong wrote:
> >      > PING: Could you review it?
> >      >
> >      > Webrev: http://cr.openjdk.java.net/~ddong/8240360/webrev.01/
> >      > JBS: https://bugs.openjdk.java.net/browse/JDK-8240360
> >      >
> >      > In Linux, EventNativeLibrary has a wrong name if the target library is
> located in a device whose device number (major or minor) exceeds the
> range of two digits.
> >      >
> >      > My fix is small, use "%*s" to ignore perms, offset and device.
> >      >
> >      > --- old/src/hotspot/os/linux/os_linux.cpp 2020-03-10
> 14:35:41.224134041 +0800
> >      > +++ new/src/hotspot/os/linux/os_linux.cpp 2020-03-10
> 14:35:41.090129104 +0800
> >      > @@ -2078,20 +2078,18 @@
> >      >
> >      >       // Read line by line from 'file'
> >      >       while (fgets(line, sizeof(line), procmapsFile) != NULL) {
> >      > -      u8 base, top, offset, inode;
> >      > -      char permissions[5];
> >      > -      char device[6];
> >      > +      u8 base, top, inode;
> >      >         char name[sizeof(line)];
> >      >
> >      > -      // Parse fields from line
> >      > -      int matches = sscanf(line, UINT64_FORMAT_X "-"
> UINT64_FORMAT_X " %4s " UINT64_FORMAT_X " %5s " INT64_FORMAT "
> %s",
> >      > -             &base, &top, permissions, &offset, device, &inode, name);
> >      > -      // the last entry 'name' is empty for some entries, so we might
> have 6 matches instead of 7 for some lines
> >      > -      if (matches < 6) continue;
> >      > -      if (matches == 6) name[0] = '\0';
> >      > +      // Parse fields from line, discard perms, offset and device
> >      > +      int matches = sscanf(line, UINT64_FORMAT_X "-"
> UINT64_FORMAT_X " %*s %*s %*s " INT64_FORMAT " %s",
> >      > +             &base, &top, &inode, name);
> >      > +      // the last entry 'name' is empty for some entries, so we might
> have 3 matches instead of 4 for some lines
> >      > +      if (matches < 3) continue;
> >      > +      if (matches == 3) name[0] = '\0';
> >      >
> >      > -      // Filter by device id '00:00' so that we only get file system mapped
> files.
> >      > -      if (strcmp(device, "00:00") != 0) {
> >      > +      // Filter by inode 0 so that we only get file system mapped files.
> >      > +      if (inode != 0) {
> >      >
> >      >           // Call callback with the fields of interest
> >      >           if(callback(name, (address)base, (address)top, param)) {
> >      >
> >      >
> >      > Thanks,
> >      > Denghui Dong
> >      >
> >      >
> >      >
> >      > ------------------------------------------------------------------
> >      > From:???(??) <denghui.ddh at alibaba-inc.com>
> >      > Send Time:2020?3?10?(???) 14:16
> >      > To:hotspot-jfr-dev <hotspot-jfr-dev at openjdk.java.net>
> >      > Subject:Re: RFR(S): 8240360: NativeLibraryEvent has wrong library
> name on Linux
> >      >
> >      > Hi,
> >      > I used a new way to fix this problem.
> >      > Ignore "perms", "offset" and "device" fields by "%*s" so that we don't
> need to be concerned about the length of the device,
> >      > and use "inode" instead of "device" for filtering.
> >      >
> >      > Webrev: http://cr.openjdk.java.net/~ddong/8240360/webrev.01/
> >      >
> >      > Could you review it?
> >      >
> >      > Thanks,
> >      > Denghui Dong
> >      > ------------------------------------------------------------------
> >      > From:???(??) <denghui.ddh at alibaba-inc.com>
> >      > Send Time:2020?3?4?(???) 12:05
> >      > To:hotspot-jfr-dev <hotspot-jfr-dev at openjdk.java.net>
> >      > Subject:Re: RFR(S): 8240360: NativeLibraryEvent has wrong library
> name on Linux
> >      >
> >      > Testing: jdk/jfr all passed.
> >      > ------------------------------------------------------------------
> >      > From:???(??) <denghui.ddh at alibaba-inc.com>
> >      > Send Time:2020?3?3?(???) 22:25
> >      > To:hotspot-jfr-dev <hotspot-jfr-dev at openjdk.java.net>
> >      > Subject:RFR(S): 8240360: NativeLibraryEvent has wrong library name on
> Linux
> >      >
> >      > Hi team,
> >      >
> >      > I found TestNativeLibrariesEvent is always failed in my Linux
> environment.
> >      >
> >      > The root cause is that os::get_loaded_modules_info in os_linux.cpp
> can not read library name correctly,
> >      > because the device number in my Linux environment is 103:01, the
> format passed to sscanf can't identify
> >      > it.
> >      >
> >      > I found the limit of major and minor device number in Linux from
> >      >
> https://github.com/torvalds/linux/blob/master/include/linux/kdev_t.h, and
> made this patch.
> >      >
> >      > Please review this small change that fixes this problem.
> >      >
> >      > Bug: https://bugs.openjdk.java.net/browse/JDK-8240360
> >      > Webrev: http://cr.openjdk.java.net/~ddong/8240360/webrev.00/
> >      >
> >      >
> >      > Thanks
> >      > Denghui Dong
> >      >
> >


From vladimir.x.ivanov at oracle.com  Tue Apr  7 09:39:32 2020
From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov)
Date: Tue, 7 Apr 2020 12:39:32 +0300
Subject: RFR (XXL): 8223347: Integration of Vector API (Incubator):
 General HotSpot changes
In-Reply-To: <3b6f8501-5d90-0a14-ed21-5978567d95d2@oracle.com>
References: <c1bdf88c-5de2-d069-5f31-5a95c6988bf8@oracle.com>
 <3b6f8501-5d90-0a14-ed21-5978567d95d2@oracle.com>
Message-ID: <7dc065c6-b8c4-3d83-4b5d-788e07d8d6e5@oracle.com>

Hi Katya,

> what kind of testing has been done to verify these changes?
> Taking into account the changes are quite large and touch share code
> running hs compiler and perhaps runtime tiers would be very advisable.

The changes (and previous versions) were tested in 2 modes:

   * ran through tier1-tier4 with the functionality turned OFF; (also, 
some previous version went through tier1-tier6 once)

   * unit tests on Vector API were run on different x86 hardware in the 
following modes: -XX:UseAVX=[3,2,1,0] -XX:UseSSE=[4,3,2]. Arm engineers 
tested the version in vector-unstable branch on AArch64 hardware.

As of now, the only known test failure is 
compiler/graalunit/HotspotTest.java in 
org.graalvm.compiler.hotspot.test.CheckGraalIntrinsics which should be 
taught about new JVM intrinsics added.

Best regards,
Vladimir Ivanov

> On 4/3/20 4:12 PM, Vladimir Ivanov wrote:
>> Hi,
>>
>> Following up on review requests of API [0] and Java implementation [1] 
>> for Vector API (JEP 338 [2]), here's a request for review of general 
>> HotSpot changes (in shared code) required for supporting the API:
>>
>>
>> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/all.00-03/ 
>>
>>
>> (First of all, to set proper expectations: since the JEP is still in 
>> Candidate state, the intention is to initiate preliminary round(s) of 
>> review to inform the community and gather feedback before sending out 
>> final/official RFRs once the JEP is Targeted to a release.)
>>
>> Vector API (being developed in Project Panama [3]) relies on JVM 
>> support to utilize optimal vector hardware instructions at runtime. It 
>> interacts with JVM through intrinsics (declared in 
>> jdk.internal.vm.vector.VectorSupport [4]) which expose vector 
>> operations support in C2 JIT-compiler.
>>
>> As Paul wrote earlier: "A vector intrinsic is an internal low-level 
>> vector operation. The last argument to the intrinsic is fall back 
>> behavior in Java, implementing the scalar operation over the number of 
>> elements held by the vector.? Thus, If the intrinsic is not supported 
>> in C2 for the other arguments then the Java implementation is executed 
>> (the Java implementation is always executed when running in the 
>> interpreter or for C1)."
>>
>> The rest of JVM support is about aggressively optimizing vector boxes 
>> to minimize (ideally eliminate) the overhead of boxing for vector values.
>> It's a stop-the-gap solution for vector box elimination problem until 
>> inline classes arrive. Vector classes are value-based and in the 
>> longer term will be migrated to inline classes once the support 
>> becomes available.
>>
>> Vector API talk from JVMLS'18 [5] contains brief overview of JVM 
>> implementation and some details.
>>
>> Complete implementation resides in vector-unstable branch of 
>> panama/dev repository [6].
>>
>> Now to gory details (the patch is split in multiple "sub-webrevs"):
>>
>> ===========================================================
>>
>> (1) 
>> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/00.backend.shared/ 
>>
>>
>> Ideal vector nodes for new operations introduced by Vector API.
>>
>> (Platform-specific back end support will be posted for review 
>> separately).
>>
>> ===========================================================
>>
>> (2) 
>> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/01.intrinsics/ 
>>
>>
>> JVM Java interface (VectorSupport) and intrinsic support in C2.
>>
>> Vector instances are initially represented as VectorBox macro nodes 
>> and "unboxing" is represented by VectorUnbox node. It simplifies 
>> vector box elimination analysis and the nodes are expanded later right 
>> before EA pass.
>>
>> Vectors have 2-level on-heap representation: for the vector value 
>> primitive array is used as a backing storage and it is encapsulated in 
>> a typed wrapper (e.g., Int256Vector - vector of 8 ints - contains a 
>> int[8] instance which is used to store vector value).
>>
>> Unless VectorBox node goes away, it needs to be expanded into an 
>> allocation eventually, but it is a pure node and doesn't have any JVM 
>> state associated with it. The problem is solved by keeping JVM state 
>> separately in a VectorBoxAllocate node associated with VectorBox node 
>> and use it during expansion.
>>
>> Also, to simplify vector box elimination, inlining of vector reboxing 
>> calls (VectorSupport::maybeRebox) is delayed until the analysis is over.
>>
>> ===========================================================
>>
>> (3) 
>> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/02.vbox_elimination/ 
>>
>>
>> Vector box elimination analysis implementation. (Brief overview: 
>> slides #36-42 [5].)
>>
>> The main part is devoted to scalarization across safepoints and 
>> rematerialization support during deoptimization. In C2-generated code 
>> vector operations work with raw vector values which live in registers 
>> or spilled on the stack and it allows to avoid boxing/unboxing when a 
>> vector value is alive across a safepoint. As with other values, 
>> there's just a location of the vector value at the safepoint and 
>> vector type information recorded in the relevant nmethod metadata and 
>> all the heavy-lifting happens only when rematerialization takes place.
>>
>> The analysis preserves object identity invariants except during 
>> aggressive reboxing (guarded by -XX:+EnableAggressiveReboxing).
>>
>> (Aggressive reboxing is crucial for cases when vectors "escape": it 
>> allocates a fresh instance at every escape point thus enabling 
>> original instance to go away.)
>>
>> ===========================================================
>>
>> (4) 
>> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/03.module.hotspot/ 
>>
>>
>> HotSpot changes for jdk.incubator.vector module. Vector support is 
>> makred experimental and turned off by default. JEP 338 proposes the 
>> API to be released as an incubator module, so a user has to specify 
>> "--add-module jdk.incubator.vector" on the command line to be able to 
>> use it.
>> When user does that, JVM automatically enables Vector API support.
>> It improves usability (user doesn't need to separately "open" the API 
>> and enable JVM support) while minimizing risks of destabilitzation 
>> from new code when the API is not used.
>>
>>
>> That's it! Will be happy to answer any questions.
>>
>> And thanks in advance for any feedback!
>>
>> Best regards,
>> Vladimir Ivanov
>>
>> [0] 
>> https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065345.html 
>>
>>
>> [1] 
>> https://mail.openjdk.java.net/pipermail/hotspot-dev/2020-April/041228.html 
>>
>>
>> [2] https://openjdk.java.net/jeps/338
>>
>> [3] https://openjdk.java.net/projects/panama/
>>
>> [4] 
>> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/01.intrinsics/src/java.base/share/classes/jdk/internal/vm/vector/VectorSupport.java.html 
>>
>>
>> [5] http://cr.openjdk.java.net/~vlivanov/talks/2018_JVMLS_VectorAPI.pdf
>>
>> [6] http://hg.openjdk.java.net/panama/dev/shortlog/92bbd44386e9
>>
>> ???? $ hg clone http://hg.openjdk.java.net/panama/dev/ -b vector-unstable
> 

From per.liden at oracle.com  Tue Apr  7 10:46:36 2020
From: per.liden at oracle.com (Per Liden)
Date: Tue, 7 Apr 2020 12:46:36 +0200
Subject: RFR: 8240693: Sweeper should not examine dying metadata in
 is_unloading() nmethod during static call stub cleaning
In-Reply-To: <a067ea89-86f1-269c-fd9b-33900158b961@oracle.com>
References: <a067ea89-86f1-269c-fd9b-33900158b961@oracle.com>
Message-ID: <355ed4bc-74ee-f48a-7042-738b2a143dee@oracle.com>

Looks good!

/Per

On 4/2/20 6:28 PM, Erik ?sterlund wrote:
> Hi,
> 
> There is currently an unfortunate race condition between concurrent code 
> cache unloading and the Sweeper
> (affecting ZGC and Shenandoah).
> 
> First of all, big thanks to Nils for the initial analysis, requiring 
> downloading a 90 GB core file. Ouch.
> He discovered that we crashed accessing a freed metadata relocation in 
> an unloaded nmethod from the sweeper.
> 
> I am fairly certain it is due to the following race condition that can 
> indeed happen today:
> 
> 1. GC terminates marking in a safepoint
> 2. GC performs concurrent code cache unlinking. In this step, inline 
> caches are cleaned for is_alive() &&
>  ?? !is_unloading() nmethods. The key is that it is not performed for 
> is_alive() && is_unloading() nmethods.
>  ?? So metadata relocations of their static/opt virtual call stubs 
> remain in the nmethod, and could potentially
>  ?? be dying.
> 3. A handshake operation is issued to separate unlinking from purging. 
> The sweeper replies to the handshake.
> 4. GC performs concurrent code cache purging, making all nmethods 
> unloaded. But just before one nmethod is unloaded...
> 5. The sweeper starts processing the nmethod. It is is_alive() && 
> is_unloading() and the sweeper correctly
>  ?? decides to clean its inline caches. As part of that inline cache 
> cleaning, it examines if the metadata
>  ?? relocations of static/opt virtual call stubs need clearing 
> (conditional of the holder of the target method being unloading).
> 6. Just before examining the metadata relocation, the sweeper gets 
> preempted.
> 7. The GC finishes concurrent code cache purging.
> 8. The GC finishes concurrent metaspace purging (the Method* is now dead).
> 9. The sweeper wakes up from its slumber and starts processing the now 
> freed Method* of the metadata relocation.
> 10. Crash.
> 
> The key here is that the reason why the sweeper clears metadata 
> relocations of static/opt virtual call stubs,
> is to unlink the metadata of is_alive() && !is_unloading() nmethods, so 
> that they are not used after the class
> unloading handshake. However, the GC does not unlink said metadata from 
> is_alive() && is_unloading() nmethods,
> because it is assumed that nobody should follow metadata from 
> is_unloading() nmethods. This violation in the
> sweeper is the exception to that rule.
> 
> The fix is to skip the processing of metadata relocations of 
> is_unloading() nmethods, during inline cache cleaning.
> I also noticed another related but harmless violation of this rule by 
> class redefinition, where metadata only filters
> is_alive(), not && !is_unloading(). I figured that this is actually 
> harmless, because class redefinition happens in
> a safepoint, and code cache purging is done with the suspendible thread 
> set taken, blocking out safepoints. Therefore
> it will happen either completely before or after code cache purging. If 
> it happens before, then the metadata is not
> yet deleted, and if it happens after, then it is no longer is_alive(). 
> However, I would like to fix that anyway,
> just to make the model more consistent.
> 
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8240693
> 
> Webrev:
> http://cr.openjdk.java.net/~eosterlund/8240693/webrev.00/
> 
> Sanity checked with hs-tier1-8 with ZGC.
> 
> Thanks,
> /Erik

From david.holmes at oracle.com  Tue Apr  7 12:50:51 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 7 Apr 2020 22:50:51 +1000
Subject: RFR(XS): 8174768: Make ProcessTools print executed process output
 into a separate file
In-Reply-To: <dba5980d-42dd-0a81-6ddf-eb53da02679c@oracle.com>
References: <ec26b257-4518-419e-83a3-b64f920f766b@default>
 <C980DAFB-8F75-4D8A-A0BB-B3227D01F54D@oracle.com>
 <70147008-45b8-0b7f-6691-50f8429c5369@oracle.com>
 <dba5980d-42dd-0a81-6ddf-eb53da02679c@oracle.com>
Message-ID: <48fa1432-2931-bf03-5eb6-ef853950f98c@oracle.com>

Hi Evgeny,

On 7/04/2020 6:33 pm, Evgeny Nikitin wrote:
> Hi David,
> 
>> I'm not at all sure this is generally what we want for every single 
>> test that uses ProcessTools! But I'm willing it to see it trialed. 
> 
> Well, it's mostly used for test VM runs. In runs I performed (artificial 
> "failures" included) the amounts of output were very small.
> 
>> Please run full tier testing at least to tier 6 and ideally beyond 
>> before pushing this. There are potential implications for temporary 
>> (and more permanent) disk usage as well as additional time needed to 
>> write files out to disk. (Hopefully these are generally small enough 
>> that this doesn't make a noticeable difference.)
> Done, thanks for suggestion. The additional runs showed no problems.

Good to know - thanks.

> I've provided Igor with a slightly modified version (Added notification 
> about the output file into the main test's log). Please review:
> 
> http://cr.openjdk.java.net/~iignatyev/enikitin/8174768/webrev.01/

A couple of minor nits:

  397             {

why did you introduce a new block scope?

  404                 System.out.println(String.format(

You can use printf rather than making a separate call to format.

No need to see any update if you chose to make it.

Thanks,
David
-----

> Best Regards,
> 
> Evgeny.
> 
> On 2020-04-02 02:07, David Holmes wrote:
>> Thanks for sharing this Igor!
>>
>> I'm not at all sure this is generally what we want for every single 
>> test that uses ProcessTools! But I'm willing it to see it trialed.
>>
>> Evgeny: Please run full tier testing at least to tier 6 and ideally 
>> beyond before pushing this. There are potential implications for 
>> temporary (and more permanent) disk usage as well as additional time 
>> needed to write files out to disk. (Hopefully these are generally 
>> small enough that this doesn't make a noticeable difference.)
>>
>> Thanks,
>> David
>>
>> On 2/04/2020 5:13 am, Igor Ignatyev wrote:
>>> Hi Evgeny,
>>>
>>> (widening the audience, given this affects not just hotspot compiler, 
>>> but hotspot tests as well as core libs tests in general)
>>>
>>> overall that looks good to me. one suggestion, for the ease of 
>>> failure analysis it might be worth to print out the names of created 
>>> files, although this might potentially clutter the output, I don't 
>>> think it'll be a problem given we already print out things like 
>>> 'Gathering output for process ...' , 'Waiting for completion...' in 
>>> LazyOutputBuffer.
>>>
>>>> The change has been tested via a mach5 test runs (jdk-tier1 through 
>>>> 4) on the 4 common platforms (linux-x64, windows-x64, macosx-x64, 
>>>> sparcv9).
>>> this doesn't include any of hotspot tiers, could you please also run 
>>> hs-tier1--4?
>>> // you can use tierN jobs which include both jdk and hs parts.
>>>
>>> Thanks,
>>> -- Igor
>>>
>>>> On Mar 30, 2020, at 3:55 AM, Evgeny Nikitin 
>>>> <evgeny.nikitin at oracle.com> wrote:
>>>>
>>>>
>>>> Hi,
>>>>
>>>>
>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8174768
>>>>
>>>> Webrev: 
>>>> http://cr.openjdk.java.net/~iignatyev/enikitin/8174768/webrev.00/
>>>>
>>>>
>>>> The bug had been created as a request to simplify investigation for 
>>>> compiler control tests failures.
>>>> I found the functionality pretty generic and useful and made 
>>>> ProcessTools dump output as well as some diagnostic information for 
>>>> every executed process into a separate file.
>>>> The diagnostic information contains cmdline, exit code, stdout and 
>>>> stderr. The output files are named like 'pid-<PID>-output.log'.
>>>>
>>>> The change has been tested via a mach5 test runs (jdk-tier1 through 
>>>> 4) on the 4 common platforms (linux-x64, windows-x64, macosx-x64, 
>>>> sparcv9).
>>>>
>>>> Please review,
>>>> /Evgeny Nikitin.
>>>

From erik.osterlund at oracle.com  Tue Apr  7 16:30:00 2020
From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=)
Date: Tue, 7 Apr 2020 18:30:00 +0200
Subject: RFR: 8240693: Sweeper should not examine dying metadata in
 is_unloading() nmethod during static call stub cleaning
In-Reply-To: <355ed4bc-74ee-f48a-7042-738b2a143dee@oracle.com>
References: <355ed4bc-74ee-f48a-7042-738b2a143dee@oracle.com>
Message-ID: <003B930D-9E58-4422-9341-13FFBF629150@oracle.com>

Hi Per,

Thanks for the review.

/Erik

> On 7 Apr 2020, at 12:46, Per Liden <per.liden at oracle.com> wrote:
> 
> ?Looks good!
> 
> /Per
> 
>> On 4/2/20 6:28 PM, Erik ?sterlund wrote:
>> Hi,
>> There is currently an unfortunate race condition between concurrent code cache unloading and the Sweeper
>> (affecting ZGC and Shenandoah).
>> First of all, big thanks to Nils for the initial analysis, requiring downloading a 90 GB core file. Ouch.
>> He discovered that we crashed accessing a freed metadata relocation in an unloaded nmethod from the sweeper.
>> I am fairly certain it is due to the following race condition that can indeed happen today:
>> 1. GC terminates marking in a safepoint
>> 2. GC performs concurrent code cache unlinking. In this step, inline caches are cleaned for is_alive() &&
>>    !is_unloading() nmethods. The key is that it is not performed for is_alive() && is_unloading() nmethods.
>>    So metadata relocations of their static/opt virtual call stubs remain in the nmethod, and could potentially
>>    be dying.
>> 3. A handshake operation is issued to separate unlinking from purging. The sweeper replies to the handshake.
>> 4. GC performs concurrent code cache purging, making all nmethods unloaded. But just before one nmethod is unloaded...
>> 5. The sweeper starts processing the nmethod. It is is_alive() && is_unloading() and the sweeper correctly
>>    decides to clean its inline caches. As part of that inline cache cleaning, it examines if the metadata
>>    relocations of static/opt virtual call stubs need clearing (conditional of the holder of the target method being unloading).
>> 6. Just before examining the metadata relocation, the sweeper gets preempted.
>> 7. The GC finishes concurrent code cache purging.
>> 8. The GC finishes concurrent metaspace purging (the Method* is now dead).
>> 9. The sweeper wakes up from its slumber and starts processing the now freed Method* of the metadata relocation.
>> 10. Crash.
>> The key here is that the reason why the sweeper clears metadata relocations of static/opt virtual call stubs,
>> is to unlink the metadata of is_alive() && !is_unloading() nmethods, so that they are not used after the class
>> unloading handshake. However, the GC does not unlink said metadata from is_alive() && is_unloading() nmethods,
>> because it is assumed that nobody should follow metadata from is_unloading() nmethods. This violation in the
>> sweeper is the exception to that rule.
>> The fix is to skip the processing of metadata relocations of is_unloading() nmethods, during inline cache cleaning.
>> I also noticed another related but harmless violation of this rule by class redefinition, where metadata only filters
>> is_alive(), not && !is_unloading(). I figured that this is actually harmless, because class redefinition happens in
>> a safepoint, and code cache purging is done with the suspendible thread set taken, blocking out safepoints. Therefore
>> it will happen either completely before or after code cache purging. If it happens before, then the metadata is not
>> yet deleted, and if it happens after, then it is no longer is_alive(). However, I would like to fix that anyway,
>> just to make the model more consistent.
>> Bug:
>> https://bugs.openjdk.java.net/browse/JDK-8240693
>> Webrev:
>> http://cr.openjdk.java.net/~eosterlund/8240693/webrev.00/
>> Sanity checked with hs-tier1-8 with ZGC.
>> Thanks,
>> /Erik


From alexandr.miloslavskiy at gmail.com  Tue Apr  7 16:48:27 2020
From: alexandr.miloslavskiy at gmail.com (Alexander Miloslavskiy)
Date: Tue, 7 Apr 2020 18:48:27 +0200
Subject: Linux crash native stacks only have 1 line
In-Reply-To: <CA+3eh11tSrjTHCMFSSOeUdk=z1WKUdTkxpWoaH9-6k6wmsxccw@mail.gmail.com>
References: <805a02c4-a765-dbd7-52f7-553e232f87d0@gmail.com>
 <6da1abd3-9b77-9daf-c21c-4913227fbce2@oracle.com>
 <d29477cb-0596-6204-fc07-23ef7ec61f5b@gmail.com>
 <CA+3eh11tSrjTHCMFSSOeUdk=z1WKUdTkxpWoaH9-6k6wmsxccw@mail.gmail.com>
Message-ID: <3e5f2300-5f9b-43e4-0ea7-c923af24df9f@gmail.com>

> one of the problems may be that the native code you're crashed in was 
> compiled with "-fomit-frame-pointer" which means that the frame pointer 
> register can't be used for unwinding. 

I found that gcc `-O` already implies `-fomit-frame-pointer` [1].
gcc `-O2` includes all optimizations from `-O`.

I have tested that on a small program and confirmed that
1) `-O2` omits frame pointers,
2) `-O2 -fno-omit-frame-pointer` preserves frame pointers.

Since most "production" code is compiled with -O2, this effectively 
means that if native crash occurs outside JRE then crash log will fail 
to unwind stack :(

In my case, most crashes occur in GTK or GLib, both omit frame pointers.

> I you already managed to attach gdb and get a correct back trace at the 
> point where your program crashes, you can just as well debug the HotSpot 
> stack tracing routine "print_native_stack()" in the file "debug.cpp" to 
> see what's the actual problem :)

I didn't debug because I understand that JDK can't unwind stack without 
frame pointers.

> Before doing? that, I'd check your 
> reproducer with jdk14 or better the tip revision just to make sure 
> you're not hunting a problem which has already been fixed upstream.

I have tested with JDK14+36 and alas, no changes.

> In general it's always advisable to mention the exact jdk version and 
> distribution you've used.

Sorry, didn't mention because I observed this on all JDK's from JDK8 to 
JDK13 (now also confirmed on JDK14)

I think I could try to implement better stack walking. Is there any 
interest to accept such patches, provided that they pass code review?

[1] 
https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#Optimize-Options

From vladimir.kozlov at oracle.com  Tue Apr  7 17:33:40 2020
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Tue, 7 Apr 2020 10:33:40 -0700
Subject: RFR: 8240693: Sweeper should not examine dying metadata in
 is_unloading() nmethod during static call stub cleaning
In-Reply-To: <355ed4bc-74ee-f48a-7042-738b2a143dee@oracle.com>
References: <a067ea89-86f1-269c-fd9b-33900158b961@oracle.com>
 <355ed4bc-74ee-f48a-7042-738b2a143dee@oracle.com>
Message-ID: <93267de7-6971-97e3-06e6-d53ab6659bb9@oracle.com>

+1

Thanks,
Vladimir

On 4/7/20 3:46 AM, Per Liden wrote:
> Looks good!
> 
> /Per
> 
> On 4/2/20 6:28 PM, Erik ?sterlund wrote:
>> Hi,
>>
>> There is currently an unfortunate race condition between concurrent code cache unloading and the Sweeper
>> (affecting ZGC and Shenandoah).
>>
>> First of all, big thanks to Nils for the initial analysis, requiring downloading a 90 GB core file. Ouch.
>> He discovered that we crashed accessing a freed metadata relocation in an unloaded nmethod from the sweeper.
>>
>> I am fairly certain it is due to the following race condition that can indeed happen today:
>>
>> 1. GC terminates marking in a safepoint
>> 2. GC performs concurrent code cache unlinking. In this step, inline caches are cleaned for is_alive() &&
>> ??? !is_unloading() nmethods. The key is that it is not performed for is_alive() && is_unloading() nmethods.
>> ??? So metadata relocations of their static/opt virtual call stubs remain in the nmethod, and could potentially
>> ??? be dying.
>> 3. A handshake operation is issued to separate unlinking from purging. The sweeper replies to the handshake.
>> 4. GC performs concurrent code cache purging, making all nmethods unloaded. But just before one nmethod is unloaded...
>> 5. The sweeper starts processing the nmethod. It is is_alive() && is_unloading() and the sweeper correctly
>> ??? decides to clean its inline caches. As part of that inline cache cleaning, it examines if the metadata
>> ??? relocations of static/opt virtual call stubs need clearing (conditional of the holder of the target method being 
>> unloading).
>> 6. Just before examining the metadata relocation, the sweeper gets preempted.
>> 7. The GC finishes concurrent code cache purging.
>> 8. The GC finishes concurrent metaspace purging (the Method* is now dead).
>> 9. The sweeper wakes up from its slumber and starts processing the now freed Method* of the metadata relocation.
>> 10. Crash.
>>
>> The key here is that the reason why the sweeper clears metadata relocations of static/opt virtual call stubs,
>> is to unlink the metadata of is_alive() && !is_unloading() nmethods, so that they are not used after the class
>> unloading handshake. However, the GC does not unlink said metadata from is_alive() && is_unloading() nmethods,
>> because it is assumed that nobody should follow metadata from is_unloading() nmethods. This violation in the
>> sweeper is the exception to that rule.
>>
>> The fix is to skip the processing of metadata relocations of is_unloading() nmethods, during inline cache cleaning.
>> I also noticed another related but harmless violation of this rule by class redefinition, where metadata only filters
>> is_alive(), not && !is_unloading(). I figured that this is actually harmless, because class redefinition happens in
>> a safepoint, and code cache purging is done with the suspendible thread set taken, blocking out safepoints. Therefore
>> it will happen either completely before or after code cache purging. If it happens before, then the metadata is not
>> yet deleted, and if it happens after, then it is no longer is_alive(). However, I would like to fix that anyway,
>> just to make the model more consistent.
>>
>> Bug:
>> https://bugs.openjdk.java.net/browse/JDK-8240693
>>
>> Webrev:
>> http://cr.openjdk.java.net/~eosterlund/8240693/webrev.00/
>>
>> Sanity checked with hs-tier1-8 with ZGC.
>>
>> Thanks,
>> /Erik

From volker.simonis at gmail.com  Tue Apr  7 17:35:48 2020
From: volker.simonis at gmail.com (Volker Simonis)
Date: Tue, 7 Apr 2020 19:35:48 +0200
Subject: Linux crash native stacks only have 1 line
In-Reply-To: <3e5f2300-5f9b-43e4-0ea7-c923af24df9f@gmail.com>
References: <805a02c4-a765-dbd7-52f7-553e232f87d0@gmail.com>
 <6da1abd3-9b77-9daf-c21c-4913227fbce2@oracle.com>
 <d29477cb-0596-6204-fc07-23ef7ec61f5b@gmail.com>
 <CA+3eh11tSrjTHCMFSSOeUdk=z1WKUdTkxpWoaH9-6k6wmsxccw@mail.gmail.com>
 <3e5f2300-5f9b-43e4-0ea7-c923af24df9f@gmail.com>
Message-ID: <CA+3eh12tyQ_vNt5CvqTNdzV_iy16g2Fpz9VkKuJ9nCF9+y2soA@mail.gmail.com>

On Tue, Apr 7, 2020 at 6:48 PM Alexander Miloslavskiy
<alexandr.miloslavskiy at gmail.com> wrote:
>
> > one of the problems may be that the native code you're crashed in was
> > compiled with "-fomit-frame-pointer" which means that the frame pointer
> > register can't be used for unwinding.
>
> I found that gcc `-O` already implies `-fomit-frame-pointer` [1].
> gcc `-O2` includes all optimizations from `-O`.
>
> I have tested that on a small program and confirmed that
> 1) `-O2` omits frame pointers,
> 2) `-O2 -fno-omit-frame-pointer` preserves frame pointers.
>
> Since most "production" code is compiled with -O2, this effectively
> means that if native crash occurs outside JRE then crash log will fail
> to unwind stack :(
>
> In my case, most crashes occur in GTK or GLib, both omit frame pointers.
>
> > I you already managed to attach gdb and get a correct back trace at the
> > point where your program crashes, you can just as well debug the HotSpot
> > stack tracing routine "print_native_stack()" in the file "debug.cpp" to
> > see what's the actual problem :)
>
> I didn't debug because I understand that JDK can't unwind stack without
> frame pointers.
>
> > Before doing  that, I'd check your
> > reproducer with jdk14 or better the tip revision just to make sure
> > you're not hunting a problem which has already been fixed upstream.
>
> I have tested with JDK14+36 and alas, no changes.
>
> > In general it's always advisable to mention the exact jdk version and
> > distribution you've used.
>
> Sorry, didn't mention because I observed this on all JDK's from JDK8 to
> JDK13 (now also confirmed on JDK14)
>
> I think I could try to implement better stack walking. Is there any
> interest to accept such patches, provided that they pass code review?
>

Sure, why not - you can definitively give it a try. On thing you
should be aware of is that this code will be usually called from the
signal handler, when the VM is already crashing. This means that the
VM may already be in an unstable state and we should be very careful
in order to not risk another error. You can start by looking at
ElfDecoder/ElfFile in decoder_elf.hpp/elfFile.hpp which already parses
an ELF file to get symbol information for a certain address. It has
the following comment :)

// ElfFile is basically an elf file parser, which can lookup the symbol
// that is the nearest to the given address.
// Beware, this code is called from vm error reporting code, when vm is already
// in "error" state, so there are scenarios, lookup will fail. We want this
// part of code to be very defensive, and bait out if anything went wrong.

I think what you'll have to do is to find out the frame size of a
native function and use that for unwinding if you have no FP. But this
can be tricky because glibc sometimes uses frameless functions. I
think if you look at gdb's unwinding code you will see that it is
quite complex.

Please also take into account that using a third party library like
libunwind is not an option for OpenJDK because of licensing issues.

Good luck and best regards,
Volker


> [1]
> https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#Optimize-Options

From daniel.smith at oracle.com  Tue Apr  7 19:08:58 2020
From: daniel.smith at oracle.com (Dan Smith)
Date: Tue, 7 Apr 2020 13:08:58 -0600
Subject: 2020 JVM Language Summit Cancelled
Message-ID: <DC2AB5F8-D69F-4A13-8D83-816DADCBA992@oracle.com>

Due to increasing concerns about the ongoing COVID-19 outbreak, and to help protect the health and safety of our customers, employees, partners, and everyone who helps host the event, Oracle is postponing JVMLS in Santa Clara, California this July, but we look forward to seeing you in 2021. In the meantime, we are evaluating other ways to communicate the exciting developments that have happened since JVMLS 2019.

From alexandr.miloslavskiy at gmail.com  Tue Apr  7 19:37:42 2020
From: alexandr.miloslavskiy at gmail.com (Alexander Miloslavskiy)
Date: Tue, 7 Apr 2020 21:37:42 +0200
Subject: Linux crash native stacks only have 1 line
In-Reply-To: <CA+3eh12tyQ_vNt5CvqTNdzV_iy16g2Fpz9VkKuJ9nCF9+y2soA@mail.gmail.com>
References: <805a02c4-a765-dbd7-52f7-553e232f87d0@gmail.com>
 <6da1abd3-9b77-9daf-c21c-4913227fbce2@oracle.com>
 <d29477cb-0596-6204-fc07-23ef7ec61f5b@gmail.com>
 <CA+3eh11tSrjTHCMFSSOeUdk=z1WKUdTkxpWoaH9-6k6wmsxccw@mail.gmail.com>
 <3e5f2300-5f9b-43e4-0ea7-c923af24df9f@gmail.com>
 <CA+3eh12tyQ_vNt5CvqTNdzV_iy16g2Fpz9VkKuJ9nCF9+y2soA@mail.gmail.com>
Message-ID: <0d660ac9-ace6-f085-3d1c-baff52c37004@gmail.com>


On 07.04.2020 19:35, Volker Simonis wrote:

> Sure, why not - you can definitively give it a try. On thing you
> should be aware of is that this code will be usually called from the
> signal handler, when the VM is already crashing. This means that the
> VM may already be in an unstable state and we should be very careful
> in order to not risk another error. You can start by looking at
> ElfDecoder/ElfFile in decoder_elf.hpp/elfFile.hpp which already parses
> an ELF file to get symbol information for a certain address. It has
> the following comment :)
> 
> // ElfFile is basically an elf file parser, which can lookup the symbol
> // that is the nearest to the given address.
> // Beware, this code is called from vm error reporting code, when vm is already
> // in "error" state, so there are scenarios, lookup will fail. We want this
> // part of code to be very defensive, and bait out if anything went wrong.
> 
> I think what you'll have to do is to find out the frame size of a
> native function and use that for unwinding if you have no FP. But this
> can be tricky because glibc sometimes uses frameless functions. I
> think if you look at gdb's unwinding code you will see that it is
> quite complex.
> 
> Please also take into account that using a third party library like
> libunwind is not an option for OpenJDK because of licensing issues.

I looked into default glibc `backtrace()` implementation and found that 
it loads libgcc and uses `_Unwind_Backtrace` from it. I understand that 
libgcc is expected to be installed on most Linux systems, then.

So it sounds promising to also use `_Unwind_Backtrace` in JDK. What do 
you think?

From igor.ignatyev at oracle.com  Tue Apr  7 20:26:39 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Tue, 7 Apr 2020 13:26:39 -0700
Subject: RFR(T) : 8241707 : introduce randomness k/w to hotspot test suite
Message-ID: <27D72A05-4B4D-4751-AAAA-45CAD8E15C68@oracle.com>

http://cr.openjdk.java.net/~iignatyev//8241707/webrev.00
> 5 lines changed: 2 ins; 0 del; 3 mod;

Hi all,

could you please review this small and trivial patch which adds "randomness" k/w into hotspot test suite and marks one testlibrary(/shared/common/sanity) test w/ this k/w?

other tests are to be updated by other subtasks of 8241623[*]. the end goal of this activity is to improve reproducibility of hotspot test suite and mark all the tests whose behavior might vary from run to run (for the reasons unrelated to the product) w/ "randomness" k/w so it will be easier to identify such tests and be more cautious when they fail and/or are used for fix verification.
       
from JBS:
> introduce 'randomness' k/w to mark tests which may vary from run to run due to usage of (pseudo)random. it's important to mention that the same k/w is used w/ the exact same meaning in the jdk test suite, so this will also further reduce differences b/w these two test bases. 
> 
> this is just a preparation state, the actual marking will be done by other sub-tasks of JDK-8241623 (most probably component-by-component).



[*] https://bugs.openjdk.java.net/browse/JDK-8241623
JBS: https://bugs.openjdk.java.net/browse/JDK-8241707
webrev: http://cr.openjdk.java.net/~iignatyev//8241707/webrev.00

Thanks,
-- Igor


From mikhailo.seledtsov at oracle.com  Tue Apr  7 20:34:13 2020
From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com)
Date: Tue, 7 Apr 2020 13:34:13 -0700
Subject: RFR(T) : 8241707 : introduce randomness k/w to hotspot test suite
In-Reply-To: <27D72A05-4B4D-4751-AAAA-45CAD8E15C68@oracle.com>
References: <27D72A05-4B4D-4751-AAAA-45CAD8E15C68@oracle.com>
Message-ID: <1293fe48-2165-5f57-eadf-24ce9c077b03@oracle.com>

Looks good,

Misha

On 4/7/20 1:26 PM, Igor Ignatyev wrote:
> http://cr.openjdk.java.net/~iignatyev//8241707/webrev.00
>> 5 lines changed: 2 ins; 0 del; 3 mod;
> Hi all,
>
> could you please review this small and trivial patch which adds "randomness" k/w into hotspot test suite and marks one testlibrary(/shared/common/sanity) test w/ this k/w?
>
> other tests are to be updated by other subtasks of 8241623[*]. the end goal of this activity is to improve reproducibility of hotspot test suite and mark all the tests whose behavior might vary from run to run (for the reasons unrelated to the product) w/ "randomness" k/w so it will be easier to identify such tests and be more cautious when they fail and/or are used for fix verification.
>         
> from JBS:
>> introduce 'randomness' k/w to mark tests which may vary from run to run due to usage of (pseudo)random. it's important to mention that the same k/w is used w/ the exact same meaning in the jdk test suite, so this will also further reduce differences b/w these two test bases.
>>
>> this is just a preparation state, the actual marking will be done by other sub-tasks of JDK-8241623 (most probably component-by-component).
>
>
> [*] https://bugs.openjdk.java.net/browse/JDK-8241623
> JBS: https://bugs.openjdk.java.net/browse/JDK-8241707
> webrev: http://cr.openjdk.java.net/~iignatyev//8241707/webrev.00
>
> Thanks,
> -- Igor
>

From erik.osterlund at oracle.com  Tue Apr  7 20:56:18 2020
From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=)
Date: Tue, 7 Apr 2020 22:56:18 +0200
Subject: RFR: 8240693: Sweeper should not examine dying metadata in
 is_unloading() nmethod during static call stub cleaning
In-Reply-To: <93267de7-6971-97e3-06e6-d53ab6659bb9@oracle.com>
References: <93267de7-6971-97e3-06e6-d53ab6659bb9@oracle.com>
Message-ID: <9AB8B912-AD09-47FC-9673-0666B34B08AA@oracle.com>

Hi Vladimir,

Thanks for the review!

/Erik

> On 7 Apr 2020, at 19:34, Vladimir Kozlov <vladimir.kozlov at oracle.com> wrote:
> 
> ?+1
> 
> Thanks,
> Vladimir
> 
>>> On 4/7/20 3:46 AM, Per Liden wrote:
>> Looks good!
>> /Per
>>> On 4/2/20 6:28 PM, Erik ?sterlund wrote:
>>> Hi,
>>> There is currently an unfortunate race condition between concurrent code cache unloading and the Sweeper
>>> (affecting ZGC and Shenandoah).
>>> First of all, big thanks to Nils for the initial analysis, requiring downloading a 90 GB core file. Ouch.
>>> He discovered that we crashed accessing a freed metadata relocation in an unloaded nmethod from the sweeper.
>>> I am fairly certain it is due to the following race condition that can indeed happen today:
>>> 1. GC terminates marking in a safepoint
>>> 2. GC performs concurrent code cache unlinking. In this step, inline caches are cleaned for is_alive() &&
>>>    !is_unloading() nmethods. The key is that it is not performed for is_alive() && is_unloading() nmethods.
>>>    So metadata relocations of their static/opt virtual call stubs remain in the nmethod, and could potentially
>>>    be dying.
>>> 3. A handshake operation is issued to separate unlinking from purging. The sweeper replies to the handshake.
>>> 4. GC performs concurrent code cache purging, making all nmethods unloaded. But just before one nmethod is unloaded...
>>> 5. The sweeper starts processing the nmethod. It is is_alive() && is_unloading() and the sweeper correctly
>>>    decides to clean its inline caches. As part of that inline cache cleaning, it examines if the metadata
>>>    relocations of static/opt virtual call stubs need clearing (conditional of the holder of the target method being unloading).
>>> 6. Just before examining the metadata relocation, the sweeper gets preempted.
>>> 7. The GC finishes concurrent code cache purging.
>>> 8. The GC finishes concurrent metaspace purging (the Method* is now dead).
>>> 9. The sweeper wakes up from its slumber and starts processing the now freed Method* of the metadata relocation.
>>> 10. Crash.
>>> The key here is that the reason why the sweeper clears metadata relocations of static/opt virtual call stubs,
>>> is to unlink the metadata of is_alive() && !is_unloading() nmethods, so that they are not used after the class
>>> unloading handshake. However, the GC does not unlink said metadata from is_alive() && is_unloading() nmethods,
>>> because it is assumed that nobody should follow metadata from is_unloading() nmethods. This violation in the
>>> sweeper is the exception to that rule.
>>> The fix is to skip the processing of metadata relocations of is_unloading() nmethods, during inline cache cleaning.
>>> I also noticed another related but harmless violation of this rule by class redefinition, where metadata only filters
>>> is_alive(), not && !is_unloading(). I figured that this is actually harmless, because class redefinition happens in
>>> a safepoint, and code cache purging is done with the suspendible thread set taken, blocking out safepoints. Therefore
>>> it will happen either completely before or after code cache purging. If it happens before, then the metadata is not
>>> yet deleted, and if it happens after, then it is no longer is_alive(). However, I would like to fix that anyway,
>>> just to make the model more consistent.
>>> Bug:
>>> https://bugs.openjdk.java.net/browse/JDK-8240693
>>> Webrev:
>>> http://cr.openjdk.java.net/~eosterlund/8240693/webrev.00/
>>> Sanity checked with hs-tier1-8 with ZGC.
>>> Thanks,
>>> /Erik


From ekaterina.pavlova at oracle.com  Tue Apr  7 21:05:55 2020
From: ekaterina.pavlova at oracle.com (Ekaterina Pavlova)
Date: Tue, 7 Apr 2020 14:05:55 -0700
Subject: RFR (XXL): 8223347: Integration of Vector API (Incubator):
 General HotSpot changes
In-Reply-To: <7dc065c6-b8c4-3d83-4b5d-788e07d8d6e5@oracle.com>
References: <c1bdf88c-5de2-d069-5f31-5a95c6988bf8@oracle.com>
 <3b6f8501-5d90-0a14-ed21-5978567d95d2@oracle.com>
 <7dc065c6-b8c4-3d83-4b5d-788e07d8d6e5@oracle.com>
Message-ID: <b5140094-f5ce-c3b9-814e-8a949f788d1a@oracle.com>

Thanks Vladimir,

Running tier1-tier4 tests and not getting any regressions is very good.
I would also recommend to run other tiers as they contain more stress tests as well as jck.
Doing it at least once before the integration would be very helpful and prevents us from
getting late issues.

Please let me know if you need any help with this.

regards,
-katya

On 4/7/20 2:39 AM, Vladimir Ivanov wrote:
> Hi Katya,
> 
>> what kind of testing has been done to verify these changes?
>> Taking into account the changes are quite large and touch share code
>> running hs compiler and perhaps runtime tiers would be very advisable.
> 
> The changes (and previous versions) were tested in 2 modes:
> 
>  ? * ran through tier1-tier4 with the functionality turned OFF; (also, some previous version went through tier1-tier6 once)
> 
>  ? * unit tests on Vector API were run on different x86 hardware in the following modes: -XX:UseAVX=[3,2,1,0] -XX:UseSSE=[4,3,2]. Arm engineers tested the version in vector-unstable branch on AArch64 hardware.
> 
> As of now, the only known test failure is compiler/graalunit/HotspotTest.java in org.graalvm.compiler.hotspot.test.CheckGraalIntrinsics which should be taught about new JVM intrinsics added.
> 
> Best regards,
> Vladimir Ivanov
> 
>> On 4/3/20 4:12 PM, Vladimir Ivanov wrote:
>>> Hi,
>>>
>>> Following up on review requests of API [0] and Java implementation [1] for Vector API (JEP 338 [2]), here's a request for review of general HotSpot changes (in shared code) required for supporting the API:
>>>
>>>
>>> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/all.00-03/
>>>
>>> (First of all, to set proper expectations: since the JEP is still in Candidate state, the intention is to initiate preliminary round(s) of review to inform the community and gather feedback before sending out final/official RFRs once the JEP is Targeted to a release.)
>>>
>>> Vector API (being developed in Project Panama [3]) relies on JVM support to utilize optimal vector hardware instructions at runtime. It interacts with JVM through intrinsics (declared in jdk.internal.vm.vector.VectorSupport [4]) which expose vector operations support in C2 JIT-compiler.
>>>
>>> As Paul wrote earlier: "A vector intrinsic is an internal low-level vector operation. The last argument to the intrinsic is fall back behavior in Java, implementing the scalar operation over the number of elements held by the vector.? Thus, If the intrinsic is not supported in C2 for the other arguments then the Java implementation is executed (the Java implementation is always executed when running in the interpreter or for C1)."
>>>
>>> The rest of JVM support is about aggressively optimizing vector boxes to minimize (ideally eliminate) the overhead of boxing for vector values.
>>> It's a stop-the-gap solution for vector box elimination problem until inline classes arrive. Vector classes are value-based and in the longer term will be migrated to inline classes once the support becomes available.
>>>
>>> Vector API talk from JVMLS'18 [5] contains brief overview of JVM implementation and some details.
>>>
>>> Complete implementation resides in vector-unstable branch of panama/dev repository [6].
>>>
>>> Now to gory details (the patch is split in multiple "sub-webrevs"):
>>>
>>> ===========================================================
>>>
>>> (1) http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/00.backend.shared/
>>>
>>> Ideal vector nodes for new operations introduced by Vector API.
>>>
>>> (Platform-specific back end support will be posted for review separately).
>>>
>>> ===========================================================
>>>
>>> (2) http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/01.intrinsics/
>>>
>>> JVM Java interface (VectorSupport) and intrinsic support in C2.
>>>
>>> Vector instances are initially represented as VectorBox macro nodes and "unboxing" is represented by VectorUnbox node. It simplifies vector box elimination analysis and the nodes are expanded later right before EA pass.
>>>
>>> Vectors have 2-level on-heap representation: for the vector value primitive array is used as a backing storage and it is encapsulated in a typed wrapper (e.g., Int256Vector - vector of 8 ints - contains a int[8] instance which is used to store vector value).
>>>
>>> Unless VectorBox node goes away, it needs to be expanded into an allocation eventually, but it is a pure node and doesn't have any JVM state associated with it. The problem is solved by keeping JVM state separately in a VectorBoxAllocate node associated with VectorBox node and use it during expansion.
>>>
>>> Also, to simplify vector box elimination, inlining of vector reboxing calls (VectorSupport::maybeRebox) is delayed until the analysis is over.
>>>
>>> ===========================================================
>>>
>>> (3) http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/02.vbox_elimination/
>>>
>>> Vector box elimination analysis implementation. (Brief overview: slides #36-42 [5].)
>>>
>>> The main part is devoted to scalarization across safepoints and rematerialization support during deoptimization. In C2-generated code vector operations work with raw vector values which live in registers or spilled on the stack and it allows to avoid boxing/unboxing when a vector value is alive across a safepoint. As with other values, there's just a location of the vector value at the safepoint and vector type information recorded in the relevant nmethod metadata and all the heavy-lifting happens only when rematerialization takes place.
>>>
>>> The analysis preserves object identity invariants except during aggressive reboxing (guarded by -XX:+EnableAggressiveReboxing).
>>>
>>> (Aggressive reboxing is crucial for cases when vectors "escape": it allocates a fresh instance at every escape point thus enabling original instance to go away.)
>>>
>>> ===========================================================
>>>
>>> (4) http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/03.module.hotspot/
>>>
>>> HotSpot changes for jdk.incubator.vector module. Vector support is makred experimental and turned off by default. JEP 338 proposes the API to be released as an incubator module, so a user has to specify "--add-module jdk.incubator.vector" on the command line to be able to use it.
>>> When user does that, JVM automatically enables Vector API support.
>>> It improves usability (user doesn't need to separately "open" the API and enable JVM support) while minimizing risks of destabilitzation from new code when the API is not used.
>>>
>>>
>>> That's it! Will be happy to answer any questions.
>>>
>>> And thanks in advance for any feedback!
>>>
>>> Best regards,
>>> Vladimir Ivanov
>>>
>>> [0] https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065345.html
>>>
>>> [1] https://mail.openjdk.java.net/pipermail/hotspot-dev/2020-April/041228.html
>>>
>>> [2] https://openjdk.java.net/jeps/338
>>>
>>> [3] https://openjdk.java.net/projects/panama/
>>>
>>> [4] http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/01.intrinsics/src/java.base/share/classes/jdk/internal/vm/vector/VectorSupport.java.html
>>>
>>> [5] http://cr.openjdk.java.net/~vlivanov/talks/2018_JVMLS_VectorAPI.pdf
>>>
>>> [6] http://hg.openjdk.java.net/panama/dev/shortlog/92bbd44386e9
>>>
>>> ???? $ hg clone http://hg.openjdk.java.net/panama/dev/ -b vector-unstable
>>


From volker.simonis at gmail.com  Wed Apr  8 14:25:35 2020
From: volker.simonis at gmail.com (Volker Simonis)
Date: Wed, 8 Apr 2020 16:25:35 +0200
Subject: Linux crash native stacks only have 1 line
In-Reply-To: <0d660ac9-ace6-f085-3d1c-baff52c37004@gmail.com>
References: <805a02c4-a765-dbd7-52f7-553e232f87d0@gmail.com>
 <6da1abd3-9b77-9daf-c21c-4913227fbce2@oracle.com>
 <d29477cb-0596-6204-fc07-23ef7ec61f5b@gmail.com>
 <CA+3eh11tSrjTHCMFSSOeUdk=z1WKUdTkxpWoaH9-6k6wmsxccw@mail.gmail.com>
 <3e5f2300-5f9b-43e4-0ea7-c923af24df9f@gmail.com>
 <CA+3eh12tyQ_vNt5CvqTNdzV_iy16g2Fpz9VkKuJ9nCF9+y2soA@mail.gmail.com>
 <0d660ac9-ace6-f085-3d1c-baff52c37004@gmail.com>
Message-ID: <CA+3eh107v1ysB91f_yHJdB3TT4asaepd46TrOpe=nsBxUwo9Eg@mail.gmail.com>

On Tue, Apr 7, 2020 at 9:37 PM Alexander Miloslavskiy
<alexandr.miloslavskiy at gmail.com> wrote:
>
>
> On 07.04.2020 19:35, Volker Simonis wrote:
>
> > Sure, why not - you can definitively give it a try. On thing you
> > should be aware of is that this code will be usually called from the
> > signal handler, when the VM is already crashing. This means that the
> > VM may already be in an unstable state and we should be very careful
> > in order to not risk another error. You can start by looking at
> > ElfDecoder/ElfFile in decoder_elf.hpp/elfFile.hpp which already parses
> > an ELF file to get symbol information for a certain address. It has
> > the following comment :)
> >
> > // ElfFile is basically an elf file parser, which can lookup the symbol
> > // that is the nearest to the given address.
> > // Beware, this code is called from vm error reporting code, when vm is already
> > // in "error" state, so there are scenarios, lookup will fail. We want this
> > // part of code to be very defensive, and bait out if anything went wrong.
> >
> > I think what you'll have to do is to find out the frame size of a
> > native function and use that for unwinding if you have no FP. But this
> > can be tricky because glibc sometimes uses frameless functions. I
> > think if you look at gdb's unwinding code you will see that it is
> > quite complex.
> >
> > Please also take into account that using a third party library like
> > libunwind is not an option for OpenJDK because of licensing issues.
>
> I looked into default glibc `backtrace()` implementation and found that
> it loads libgcc and uses `_Unwind_Backtrace` from it. I understand that
> libgcc is expected to be installed on most Linux systems, then.
>
> So it sounds promising to also use `_Unwind_Backtrace` in JDK. What do
> you think?

Libjvm.so is already linked against libgcc anyway (either statically
or dynamically, depending on the "--with-stdc++lib" configure option".
But the man page for "backtrace()" mentions;

These functions make some assumptions about how a function's return
address is stored on the stack.  Note the following:
 * Omission  of  the frame pointers (as implied by any of gcc(1)'s
nonzero optimization levels) may cause these assumptions to be
violated.
 * Inlined functions do not have stack frames.
 * Tail-call optimization causes one stack frame to replace another.

So I'm not sure if the result will be really better.

Also notice that "VMError::print_native_stack()" prints a mixed
Native/Java stack which ideally looks something like:

Stack: [0x00007ffff7ed5000,0x00007ffff7fd5000],
sp=0x00007ffff7fd2a70,  free space=1014k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0xa5038d]  Unsafe_SetInt+0xad
j  sun.misc.Unsafe.putInt(Ljava/lang/Object;JI)V+0
j  CrashNative.crashIt(Lsun/misc/Unsafe;I)V+53
j  CrashNative.doIt()V+44
v  ~StubRoutines::call_stub
V  [libjvm.so+0x66bafa]  JavaCalls::call_helper(JavaValue*,
methodHandle*, JavaCallArguments*, Thread*)+0xe1a
V  [libjvm.so+0x94cea8]  Reflection::invoke(instanceKlassHandle,
methodHandle, Handle, bool, objArrayHandle, BasicType, objArrayHandle,
bool, Thread*)+0xae8
V  [libjvm.so+0x94e876]  Reflection::invoke_method(oopDesc*, Handle,
objArrayHandle, Thread*)+0x126
V  [libjvm.so+0x6b7915]  JVM_InvokeMethod+0x115
j  sun.reflect.NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+0
j  sun.reflect.NativeMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+100
j  sun.reflect.DelegatingMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+6
j  java.lang.reflect.Method.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+56
j  CrashNative.mainJava()V+32
v  ~StubRoutines::call_stub
V  [libjvm.so+0x66bafa]  JavaCalls::call_helper(JavaValue*,
methodHandle*, JavaCallArguments*, Thread*)+0xe1a
V  [libjvm.so+0x68c390]  jni_invoke_static(JNIEnv_*, JavaValue*,
_jobject*, JNICallType, _jmethodID*, JNI_ArgumentPusher*, Thread*)
[clone .isra.92] [clone .constprop.98]+0x1f0
V  [libjvm.so+0x68e979]  jni_CallStaticVoidMethodV+0xd9
C  [libCrashNative.so+0xa81]  JNIEnv_::CallStaticVoidMethod(_jclass*,
_jmethodID*, ...)+0xc7
C  [libCrashNative.so+0xb00]  step3(JNIEnv_*, _jobject*)+0x65
C  [libCrashNative.so+0xb5a]  step2(JNIEnv_*, _jobject*)+0x57
C  [libCrashNative.so+0xb45]  step2(JNIEnv_*, _jobject*)+0x42
C  [libCrashNative.so+0xb45]  step2(JNIEnv_*, _jobject*)+0x42
C  [libCrashNative.so+0xb45]  step2(JNIEnv_*, _jobject*)+0x42
C  [libCrashNative.so+0xb45]  step2(JNIEnv_*, _jobject*)+0x42
C  [libCrashNative.so+0xb45]  step2(JNIEnv_*, _jobject*)+0x42
C  [libCrashNative.so+0xb80]  step1(JNIEnv_*, _jobject*)+0x23
C  [libCrashNative.so+0x94d]  Java_CrashNative_nativeMethod+0x23
j  CrashNative.nativeMethod()V+0
j  CrashNative.main([Ljava/lang/String;)V+23
v  ~StubRoutines::call_stub
V  [libjvm.so+0x66bafa]  JavaCalls::call_helper(JavaValue*,
methodHandle*, JavaCallArguments*, Thread*)+0xe1a
V  [libjvm.so+0x68c390]  jni_invoke_static(JNIEnv_*, JavaValue*,
_jobject*, JNICallType, _jmethodID*, JNI_ArgumentPusher*, Thread*)
[clone .isra.92] [clone .constprop.98]+0x1f0
V  [libjvm.so+0x68e84b]  jni_CallStaticVoidMethod+0x15b
C  [libjli.so+0x88fc]  JavaMain+0xa3c
C  [libpthread.so.0+0x76db]  start_thread+0xdb

If you use a helper function like "backtrace()", it will only unwind
native frames until it reaches the first Java frame because it won't
be able to unwind Java frames. Even gdb can not do that , because it
doesn't know the frame layout of HotSpots JIT compiled or interpreted
frames.

Best regards,
Volker

From alexandr.miloslavskiy at gmail.com  Wed Apr  8 15:08:32 2020
From: alexandr.miloslavskiy at gmail.com (Alexander Miloslavskiy)
Date: Wed, 8 Apr 2020 17:08:32 +0200
Subject: Linux crash native stacks only have 1 line
In-Reply-To: <CA+3eh107v1ysB91f_yHJdB3TT4asaepd46TrOpe=nsBxUwo9Eg@mail.gmail.com>
References: <805a02c4-a765-dbd7-52f7-553e232f87d0@gmail.com>
 <6da1abd3-9b77-9daf-c21c-4913227fbce2@oracle.com>
 <d29477cb-0596-6204-fc07-23ef7ec61f5b@gmail.com>
 <CA+3eh11tSrjTHCMFSSOeUdk=z1WKUdTkxpWoaH9-6k6wmsxccw@mail.gmail.com>
 <3e5f2300-5f9b-43e4-0ea7-c923af24df9f@gmail.com>
 <CA+3eh12tyQ_vNt5CvqTNdzV_iy16g2Fpz9VkKuJ9nCF9+y2soA@mail.gmail.com>
 <0d660ac9-ace6-f085-3d1c-baff52c37004@gmail.com>
 <CA+3eh107v1ysB91f_yHJdB3TT4asaepd46TrOpe=nsBxUwo9Eg@mail.gmail.com>
Message-ID: <6731a867-7a5a-eaee-7ffc-55f0f803ca4f@gmail.com>

On 08.04.2020 16:25, Volker Simonis wrote:
> Libjvm.so is already linked against libgcc anyway (either statically
> or dynamically, depending on the "--with-stdc++lib" configure option".

I understand that this means that yes, libgcc is fine to use.

> But the man page for "backtrace()" mentions;
> 
> These functions make some assumptions about how a function's return
> address is stored on the stack.  Note the following:
>   * Omission  of  the frame pointers (as implied by any of gcc(1)'s
> nonzero optimization levels) may cause these assumptions to be
> violated.
>   * Inlined functions do not have stack frames.
>   * Tail-call optimization causes one stack frame to replace another.
> 
> So I'm not sure if the result will be really better.

Luckily, 'backtrace()' is good enough. In fact, I would be surprised if 
it was too dumb.

Try it yourself:
----------------
#include <gtk/gtk.h>
#include <stdio.h>
#include <csignal>
#include <execinfo.h>

void OnButtonClicked(GtkWidget*, gpointer*) {
	void* myBacktrace[256];
	int length = backtrace(myBacktrace, 256);
	for (int i = 0; i < length; i++) {
		printf("backtrace[%d] = %p\n", i, myBacktrace[i]);
	}

	std::raise(SIGINT);
}

int main(int argc, char **argv) {
	gtk_init(&argc, &argv);
	GtkWidget *window = gtk_window_new(GTK_WINDOW_TOPLEVEL);
	g_signal_connect(window, "delete_event", gtk_main_quit, NULL);
	gtk_window_resize(GTK_WINDOW(window), 200, 100);

	GtkWidget *button = gtk_button_new_with_label("Test");
	gtk_container_add(GTK_CONTAINER(window), button);
	g_signal_connect(button, "clicked", G_CALLBACK(OnButtonClicked), 0);

	gtk_widget_show_all(window);
	gtk_main();
	return 0;
}
----------------

Compile with -O2:
g++ -O2 -g Snippet.cpp -o Snippet `pkg-config --cflags gtk+-3.0` 
`pkg-config --libs gtk+-3.0`

Run under GDB. When it stops, run gdb `bt`.

I see that `backtrace()` is almost perfect!

It's easy to verify that there are no frame pointers, either by 
inspecting stack with `x/32a $rsp` or by disassembling like `disas 
g_signal_emit`.

> 
> Also notice that "VMError::print_native_stack()" prints a mixed
> Native/Java stack which ideally looks something like:

> If you use a helper function like "backtrace()", it will only unwind
> native frames until it reaches the first Java frame because it won't
> be able to unwind Java frames. Even gdb can not do that , because it
> doesn't know the frame layout of HotSpots JIT compiled or interpreted
> frames.

Right, that's why I intend to use `_Unwind_Backtrace` directly instead 
of `backtrace()`. The first is a powerful iterator-like API, second is 
simple to use but without any advanced stuff.

How does that sound to you?

From igor.ignatyev at oracle.com  Wed Apr  8 15:19:40 2020
From: igor.ignatyev at oracle.com (Igor Ignatev)
Date: Wed, 8 Apr 2020 08:19:40 -0700
Subject: RFR(XS): 8174768: Make ProcessTools print executed process output
 into a separate file
In-Reply-To: <428675e0-f09c-ac6b-f5ec-afc4190a6b45@oracle.com>
References: <428675e0-f09c-ac6b-f5ec-afc4190a6b45@oracle.com>
Message-ID: <16A3371B-EABC-4955-902C-78EB079094DB@oracle.com>

You can use %n in printf to get a system-specific newline.

? Igor

> On Apr 8, 2020, at 8:12 AM, Evgeny Nikitin <evgeny.nikitin at oracle.com> wrote:
> 
> ?Hi David,
> 
> >  > You can use printf rather than making a separate call to format.
> > Good point, I've missed that method. I'll fix it and ask Igor to change
> > the webrev.
> 
> I finally decided to leave it as it is now, since printf doesn't add a newline and System.lineSeparator() is not shorter than String.format() :). Please let me know if you disagree.
> 
> Best regards,
> Evgeny.
> 
> 
>> On 2020-04-07 16:09, Evgeny Nikitin wrote:
>> Hi David,
>> > why did you introduce a new block scope?
>> To separate the action block from the other code. "A Poor/lazy man's method". I've preferred to lay it out this way because it makes the code more compact and easy to read (as opposing to looking for a only-once used method in some other part of the file) and due to the confusing name of OutputAnalyzer type (which I need as a storage for output, not as some analyzer). Creating a method with OutputAnalyzer as a parameter would make this method's purpose even more confusing.
>> > You can use printf rather than making a separate call to format.
>> Good point, I've missed that method. I'll fix it and ask Igor to change the webrev.
>> Regards,
>> Evgeny.
>>> On 2020-04-07 14:50, David Holmes wrote:
>>> Hi Evgeny,
>>> 
>>> On 7/04/2020 6:33 pm, Evgeny Nikitin wrote:
>>>> Hi David,
>>>> 
>>>>> I'm not at all sure this is generally what we want for every single test that uses ProcessTools! But I'm willing it to see it trialed. 
>>>> 
>>>> Well, it's mostly used for test VM runs. In runs I performed (artificial "failures" included) the amounts of output were very small.
>>>> 
>>>>> Please run full tier testing at least to tier 6 and ideally beyond before pushing this. There are potential implications for temporary (and more permanent) disk usage as well as additional time needed to write files out to disk. (Hopefully these are generally small enough that this doesn't make a noticeable difference.)
>>>> Done, thanks for suggestion. The additional runs showed no problems.
>>> 
>>> Good to know - thanks.
>>> 
>>>> I've provided Igor with a slightly modified version (Added notification about the output file into the main test's log). Please review:
>>>> 
>>>> http://cr.openjdk.java.net/~iignatyev/enikitin/8174768/webrev.01/
>>> 
>>> A couple of minor nits:
>>> 
>>>  397             {
>>> 
>>> why did you introduce a new block scope?
>>> 
>>>  404                 System.out.println(String.format(
>>> 
>>> You can use printf rather than making a separate call to format.
>>> 
>>> No need to see any update if you chose to make it.
>>> 
>>> Thanks,
>>> David
>>> -----
>>> 
>>>> Best Regards,
>>>> 
>>>> Evgeny.
>>>> 
>>>> On 2020-04-02 02:07, David Holmes wrote:
>>>>> Thanks for sharing this Igor!
>>>>> 
>>>>> I'm not at all sure this is generally what we want for every single test that uses ProcessTools! But I'm willing it to see it trialed.
>>>>> 
>>>>> Evgeny: Please run full tier testing at least to tier 6 and ideally beyond before pushing this. There are potential implications for temporary (and more permanent) disk usage as well as additional time needed to write files out to disk. (Hopefully these are generally small enough that this doesn't make a noticeable difference.)
>>>>> 
>>>>> Thanks,
>>>>> David
>>>>> 
>>>>> On 2/04/2020 5:13 am, Igor Ignatyev wrote:
>>>>>> Hi Evgeny,
>>>>>> 
>>>>>> (widening the audience, given this affects not just hotspot compiler, but hotspot tests as well as core libs tests in general)
>>>>>> 
>>>>>> overall that looks good to me. one suggestion, for the ease of failure analysis it might be worth to print out the names of created files, although this might potentially clutter the output, I don't think it'll be a problem given we already print out things like 'Gathering output for process ...' , 'Waiting for completion...' in LazyOutputBuffer.
>>>>>> 
>>>>>>> The change has been tested via a mach5 test runs (jdk-tier1 through 4) on the 4 common platforms (linux-x64, windows-x64, macosx-x64, sparcv9).
>>>>>> this doesn't include any of hotspot tiers, could you please also run hs-tier1--4?
>>>>>> // you can use tierN jobs which include both jdk and hs parts.
>>>>>> 
>>>>>> Thanks,
>>>>>> -- Igor
>>>>>> 
>>>>>>> On Mar 30, 2020, at 3:55 AM, Evgeny Nikitin <evgeny.nikitin at oracle.com> wrote:
>>>>>>> 
>>>>>>> 
>>>>>>> Hi,
>>>>>>> 
>>>>>>> 
>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8174768
>>>>>>> 
>>>>>>> Webrev: http://cr.openjdk.java.net/~iignatyev/enikitin/8174768/webrev.00/
>>>>>>> 
>>>>>>> 
>>>>>>> The bug had been created as a request to simplify investigation for compiler control tests failures.
>>>>>>> I found the functionality pretty generic and useful and made ProcessTools dump output as well as some diagnostic information for every executed process into a separate file.
>>>>>>> The diagnostic information contains cmdline, exit code, stdout and stderr. The output files are named like 'pid-<PID>-output.log'.
>>>>>>> 
>>>>>>> The change has been tested via a mach5 test runs (jdk-tier1 through 4) on the 4 common platforms (linux-x64, windows-x64, macosx-x64, sparcv9).
>>>>>>> 
>>>>>>> Please review,
>>>>>>> /Evgeny Nikitin.
>>>>>> 


From evgeny.nikitin at oracle.com  Wed Apr  8 17:27:16 2020
From: evgeny.nikitin at oracle.com (Evgeny Nikitin)
Date: Wed, 8 Apr 2020 19:27:16 +0200
Subject: RFR(XS): 8174768: Make ProcessTools print executed process output
 into a separate file
In-Reply-To: <16A3371B-EABC-4955-902C-78EB079094DB@oracle.com>
References: <428675e0-f09c-ac6b-f5ec-afc4190a6b45@oracle.com>
 <16A3371B-EABC-4955-902C-78EB079094DB@oracle.com>
Message-ID: <da4ab22e-b126-4b27-e744-67e572326608@oracle.com>

Thanks to Igor Ignatyev, a new version is available at

http://cr.openjdk.java.net/~iignatyev/enikitin/8174768/webrev.02

Regards,
Evgeny.


On 2020-04-08 17:19, Igor Ignatev wrote:
> You can use %n in printf to get a system-specific newline.
> 
> ? Igor
> 
>> On Apr 8, 2020, at 8:12 AM, Evgeny Nikitin <evgeny.nikitin at oracle.com> wrote:
>>
>> ?Hi David,
>>
>>>   > You can use printf rather than making a separate call to format.
>>> Good point, I've missed that method. I'll fix it and ask Igor to change
>>> the webrev.
>>
>> I finally decided to leave it as it is now, since printf doesn't add a newline and System.lineSeparator() is not shorter than String.format() :). Please let me know if you disagree.
>>
>> Best regards,
>> Evgeny.
>>
>>
>>> On 2020-04-07 16:09, Evgeny Nikitin wrote:
>>> Hi David,
>>>> why did you introduce a new block scope?
>>> To separate the action block from the other code. "A Poor/lazy man's method". I've preferred to lay it out this way because it makes the code more compact and easy to read (as opposing to looking for a only-once used method in some other part of the file) and due to the confusing name of OutputAnalyzer type (which I need as a storage for output, not as some analyzer). Creating a method with OutputAnalyzer as a parameter would make this method's purpose even more confusing.
>>>> You can use printf rather than making a separate call to format.
>>> Good point, I've missed that method. I'll fix it and ask Igor to change the webrev.
>>> Regards,
>>> Evgeny.
>>>> On 2020-04-07 14:50, David Holmes wrote:
>>>> Hi Evgeny,
>>>>
>>>> On 7/04/2020 6:33 pm, Evgeny Nikitin wrote:
>>>>> Hi David,
>>>>>
>>>>>> I'm not at all sure this is generally what we want for every single test that uses ProcessTools! But I'm willing it to see it trialed.
>>>>>
>>>>> Well, it's mostly used for test VM runs. In runs I performed (artificial "failures" included) the amounts of output were very small.
>>>>>
>>>>>> Please run full tier testing at least to tier 6 and ideally beyond before pushing this. There are potential implications for temporary (and more permanent) disk usage as well as additional time needed to write files out to disk. (Hopefully these are generally small enough that this doesn't make a noticeable difference.)
>>>>> Done, thanks for suggestion. The additional runs showed no problems.
>>>>
>>>> Good to know - thanks.
>>>>
>>>>> I've provided Igor with a slightly modified version (Added notification about the output file into the main test's log). Please review:
>>>>>
>>>>> http://cr.openjdk.java.net/~iignatyev/enikitin/8174768/webrev.01/
>>>>
>>>> A couple of minor nits:
>>>>
>>>>   397             {
>>>>
>>>> why did you introduce a new block scope?
>>>>
>>>>   404                 System.out.println(String.format(
>>>>
>>>> You can use printf rather than making a separate call to format.
>>>>
>>>> No need to see any update if you chose to make it.
>>>>
>>>> Thanks,
>>>> David
>>>> -----
>>>>
>>>>> Best Regards,
>>>>>
>>>>> Evgeny.
>>>>>
>>>>> On 2020-04-02 02:07, David Holmes wrote:
>>>>>> Thanks for sharing this Igor!
>>>>>>
>>>>>> I'm not at all sure this is generally what we want for every single test that uses ProcessTools! But I'm willing it to see it trialed.
>>>>>>
>>>>>> Evgeny: Please run full tier testing at least to tier 6 and ideally beyond before pushing this. There are potential implications for temporary (and more permanent) disk usage as well as additional time needed to write files out to disk. (Hopefully these are generally small enough that this doesn't make a noticeable difference.)
>>>>>>
>>>>>> Thanks,
>>>>>> David
>>>>>>
>>>>>> On 2/04/2020 5:13 am, Igor Ignatyev wrote:
>>>>>>> Hi Evgeny,
>>>>>>>
>>>>>>> (widening the audience, given this affects not just hotspot compiler, but hotspot tests as well as core libs tests in general)
>>>>>>>
>>>>>>> overall that looks good to me. one suggestion, for the ease of failure analysis it might be worth to print out the names of created files, although this might potentially clutter the output, I don't think it'll be a problem given we already print out things like 'Gathering output for process ...' , 'Waiting for completion...' in LazyOutputBuffer.
>>>>>>>
>>>>>>>> The change has been tested via a mach5 test runs (jdk-tier1 through 4) on the 4 common platforms (linux-x64, windows-x64, macosx-x64, sparcv9).
>>>>>>> this doesn't include any of hotspot tiers, could you please also run hs-tier1--4?
>>>>>>> // you can use tierN jobs which include both jdk and hs parts.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> -- Igor
>>>>>>>
>>>>>>>> On Mar 30, 2020, at 3:55 AM, Evgeny Nikitin <evgeny.nikitin at oracle.com> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>>
>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8174768
>>>>>>>>
>>>>>>>> Webrev: http://cr.openjdk.java.net/~iignatyev/enikitin/8174768/webrev.00/
>>>>>>>>
>>>>>>>>
>>>>>>>> The bug had been created as a request to simplify investigation for compiler control tests failures.
>>>>>>>> I found the functionality pretty generic and useful and made ProcessTools dump output as well as some diagnostic information for every executed process into a separate file.
>>>>>>>> The diagnostic information contains cmdline, exit code, stdout and stderr. The output files are named like 'pid-<PID>-output.log'.
>>>>>>>>
>>>>>>>> The change has been tested via a mach5 test runs (jdk-tier1 through 4) on the 4 common platforms (linux-x64, windows-x64, macosx-x64, sparcv9).
>>>>>>>>
>>>>>>>> Please review,
>>>>>>>> /Evgeny Nikitin.
>>>>>>>
> 

From fweimer at redhat.com  Wed Apr  8 17:32:04 2020
From: fweimer at redhat.com (Florian Weimer)
Date: Wed, 08 Apr 2020 19:32:04 +0200
Subject: Linux crash native stacks only have 1 line
In-Reply-To: <6731a867-7a5a-eaee-7ffc-55f0f803ca4f@gmail.com> (Alexander
 Miloslavskiy's message of "Wed, 8 Apr 2020 17:08:32 +0200")
References: <805a02c4-a765-dbd7-52f7-553e232f87d0@gmail.com>
 <6da1abd3-9b77-9daf-c21c-4913227fbce2@oracle.com>
 <d29477cb-0596-6204-fc07-23ef7ec61f5b@gmail.com>
 <CA+3eh11tSrjTHCMFSSOeUdk=z1WKUdTkxpWoaH9-6k6wmsxccw@mail.gmail.com>
 <3e5f2300-5f9b-43e4-0ea7-c923af24df9f@gmail.com>
 <CA+3eh12tyQ_vNt5CvqTNdzV_iy16g2Fpz9VkKuJ9nCF9+y2soA@mail.gmail.com>
 <0d660ac9-ace6-f085-3d1c-baff52c37004@gmail.com>
 <CA+3eh107v1ysB91f_yHJdB3TT4asaepd46TrOpe=nsBxUwo9Eg@mail.gmail.com>
 <6731a867-7a5a-eaee-7ffc-55f0f803ca4f@gmail.com>
Message-ID: <87a73lx4rv.fsf@oldenburg2.str.redhat.com>

* Alexander Miloslavskiy:

> On 08.04.2020 16:25, Volker Simonis wrote:
>> Libjvm.so is already linked against libgcc anyway (either statically
>> or dynamically, depending on the "--with-stdc++lib" configure option".
>
> I understand that this means that yes, libgcc is fine to use.

Not really.  You need libgcc_s.  You can link the unwinder statically,
but you really, really do not want to do that because sometimes, the
operating system has newer unwinding information than what is understood
by the statically linked copy of the unwinder.  Having multiple copies
of the unwinder in the same process can cause issues as well (but the
way OpenJDK uses hidden symbols tends to reduce that particular risk).

> I see that `backtrace()` is almost perfect!

Some distributions build with -fno-unwind-tables, or do not patch GCC to
generate the tables by default.  I expect you will get very different
results on such distributions.  But if the tables are there, the
unwinder (whether used via backtrace or directly via the _Unwind_*
functions) gives very good results for valid call stacks.

(Personally, I think stripping unwind tables is wrong, and this happens
to align with what we try to do in Fedora and downstream, but that is
not the only opinion out there.)

Thanks,
Florian


From alexandr.miloslavskiy at gmail.com  Wed Apr  8 17:53:24 2020
From: alexandr.miloslavskiy at gmail.com (Alexander Miloslavskiy)
Date: Wed, 8 Apr 2020 19:53:24 +0200
Subject: Linux crash native stacks only have 1 line
In-Reply-To: <87a73lx4rv.fsf@oldenburg2.str.redhat.com>
References: <805a02c4-a765-dbd7-52f7-553e232f87d0@gmail.com>
 <6da1abd3-9b77-9daf-c21c-4913227fbce2@oracle.com>
 <d29477cb-0596-6204-fc07-23ef7ec61f5b@gmail.com>
 <CA+3eh11tSrjTHCMFSSOeUdk=z1WKUdTkxpWoaH9-6k6wmsxccw@mail.gmail.com>
 <3e5f2300-5f9b-43e4-0ea7-c923af24df9f@gmail.com>
 <CA+3eh12tyQ_vNt5CvqTNdzV_iy16g2Fpz9VkKuJ9nCF9+y2soA@mail.gmail.com>
 <0d660ac9-ace6-f085-3d1c-baff52c37004@gmail.com>
 <CA+3eh107v1ysB91f_yHJdB3TT4asaepd46TrOpe=nsBxUwo9Eg@mail.gmail.com>
 <6731a867-7a5a-eaee-7ffc-55f0f803ca4f@gmail.com>
 <87a73lx4rv.fsf@oldenburg2.str.redhat.com>
Message-ID: <a3db52b6-3651-6c14-f55e-fd8e3f0245ea@gmail.com>

On 08.04.2020 19:32, Florian Weimer wrote:
> Not really.  You need libgcc_s.  You can link the unwinder statically,
> but you really, really do not want to do that because sometimes, the
> operating system has newer unwinding information than what is understood
> by the statically linked copy of the unwinder.  Having multiple copies
> of the unwinder in the same process can cause issues as well (but the
> way OpenJDK uses hidden symbols tends to reduce that particular risk).

Right, I actually meant that. I am going to implement it similar to 
libc [1] where it uses dlopen(LIBGCC_S_SO).
			
> Some distributions build with -fno-unwind-tables, or do not patch GCC to
> generate the tables by default.  I expect you will get very different
> results on such distributions.  But if the tables are there, the
> unwinder (whether used via backtrace or directly via the_Unwind_*
> functions) gives very good results for valid call stacks.

Solving the problem on a majority of systems will already be a 
significant improvement compared to how currently it fails on the 
majority of systems.

[1] https://github.com/bminor/glibc/blob/master/debug/backtrace.c

From volker.simonis at gmail.com  Wed Apr  8 18:49:53 2020
From: volker.simonis at gmail.com (Volker Simonis)
Date: Wed, 8 Apr 2020 20:49:53 +0200
Subject: Linux crash native stacks only have 1 line
In-Reply-To: <6731a867-7a5a-eaee-7ffc-55f0f803ca4f@gmail.com>
References: <805a02c4-a765-dbd7-52f7-553e232f87d0@gmail.com>
 <6da1abd3-9b77-9daf-c21c-4913227fbce2@oracle.com>
 <d29477cb-0596-6204-fc07-23ef7ec61f5b@gmail.com>
 <CA+3eh11tSrjTHCMFSSOeUdk=z1WKUdTkxpWoaH9-6k6wmsxccw@mail.gmail.com>
 <3e5f2300-5f9b-43e4-0ea7-c923af24df9f@gmail.com>
 <CA+3eh12tyQ_vNt5CvqTNdzV_iy16g2Fpz9VkKuJ9nCF9+y2soA@mail.gmail.com>
 <0d660ac9-ace6-f085-3d1c-baff52c37004@gmail.com>
 <CA+3eh107v1ysB91f_yHJdB3TT4asaepd46TrOpe=nsBxUwo9Eg@mail.gmail.com>
 <6731a867-7a5a-eaee-7ffc-55f0f803ca4f@gmail.com>
Message-ID: <CA+3eh13TatCOTj-no=geQQE_+FD2OOO8gYLQGg8cqAuq7ZOdnQ@mail.gmail.com>

On Wed, Apr 8, 2020 at 5:08 PM Alexander Miloslavskiy
<alexandr.miloslavskiy at gmail.com> wrote:
>
> On 08.04.2020 16:25, Volker Simonis wrote:
> > Libjvm.so is already linked against libgcc anyway (either statically
> > or dynamically, depending on the "--with-stdc++lib" configure option".
>
> I understand that this means that yes, libgcc is fine to use.
>
> > But the man page for "backtrace()" mentions;
> >
> > These functions make some assumptions about how a function's return
> > address is stored on the stack.  Note the following:
> >   * Omission  of  the frame pointers (as implied by any of gcc(1)'s
> > nonzero optimization levels) may cause these assumptions to be
> > violated.
> >   * Inlined functions do not have stack frames.
> >   * Tail-call optimization causes one stack frame to replace another.
> >
> > So I'm not sure if the result will be really better.
>
> Luckily, 'backtrace()' is good enough. In fact, I would be surprised if
> it was too dumb.
>
> Try it yourself:
> ----------------
> #include <gtk/gtk.h>
> #include <stdio.h>
> #include <csignal>
> #include <execinfo.h>
>
> void OnButtonClicked(GtkWidget*, gpointer*) {
>         void* myBacktrace[256];
>         int length = backtrace(myBacktrace, 256);
>         for (int i = 0; i < length; i++) {
>                 printf("backtrace[%d] = %p\n", i, myBacktrace[i]);
>         }
>
>         std::raise(SIGINT);
> }
>
> int main(int argc, char **argv) {
>         gtk_init(&argc, &argv);
>         GtkWidget *window = gtk_window_new(GTK_WINDOW_TOPLEVEL);
>         g_signal_connect(window, "delete_event", gtk_main_quit, NULL);
>         gtk_window_resize(GTK_WINDOW(window), 200, 100);
>
>         GtkWidget *button = gtk_button_new_with_label("Test");
>         gtk_container_add(GTK_CONTAINER(window), button);
>         g_signal_connect(button, "clicked", G_CALLBACK(OnButtonClicked), 0);
>
>         gtk_widget_show_all(window);
>         gtk_main();
>         return 0;
> }
> ----------------
>
> Compile with -O2:
> g++ -O2 -g Snippet.cpp -o Snippet `pkg-config --cflags gtk+-3.0`
> `pkg-config --libs gtk+-3.0`
>
> Run under GDB. When it stops, run gdb `bt`.
>

Are you sure that the "backtrace" command in gdb is using the
"backtrace()" function from libgcc?

Just asking because I'm not sure. I thought gdb has its own stack
unwinding logic and applies some extra tricks but it's actually quite
a while since I've last looked into the sources :)

> I see that `backtrace()` is almost perfect!
>
> It's easy to verify that there are no frame pointers, either by
> inspecting stack with `x/32a $rsp` or by disassembling like `disas
> g_signal_emit`.
>
> >
> > Also notice that "VMError::print_native_stack()" prints a mixed
> > Native/Java stack which ideally looks something like:
>
> > If you use a helper function like "backtrace()", it will only unwind
> > native frames until it reaches the first Java frame because it won't
> > be able to unwind Java frames. Even gdb can not do that , because it
> > doesn't know the frame layout of HotSpots JIT compiled or interpreted
> > frames.
>
> Right, that's why I intend to use `_Unwind_Backtrace` directly instead
> of `backtrace()`. The first is a powerful iterator-like API, second is
> simple to use but without any advanced stuff.
>
> How does that sound to you?

From alexandr.miloslavskiy at gmail.com  Wed Apr  8 18:52:37 2020
From: alexandr.miloslavskiy at gmail.com (Alexander Miloslavskiy)
Date: Wed, 8 Apr 2020 20:52:37 +0200
Subject: Linux crash native stacks only have 1 line
In-Reply-To: <CA+3eh13TatCOTj-no=geQQE_+FD2OOO8gYLQGg8cqAuq7ZOdnQ@mail.gmail.com>
References: <805a02c4-a765-dbd7-52f7-553e232f87d0@gmail.com>
 <6da1abd3-9b77-9daf-c21c-4913227fbce2@oracle.com>
 <d29477cb-0596-6204-fc07-23ef7ec61f5b@gmail.com>
 <CA+3eh11tSrjTHCMFSSOeUdk=z1WKUdTkxpWoaH9-6k6wmsxccw@mail.gmail.com>
 <3e5f2300-5f9b-43e4-0ea7-c923af24df9f@gmail.com>
 <CA+3eh12tyQ_vNt5CvqTNdzV_iy16g2Fpz9VkKuJ9nCF9+y2soA@mail.gmail.com>
 <0d660ac9-ace6-f085-3d1c-baff52c37004@gmail.com>
 <CA+3eh107v1ysB91f_yHJdB3TT4asaepd46TrOpe=nsBxUwo9Eg@mail.gmail.com>
 <6731a867-7a5a-eaee-7ffc-55f0f803ca4f@gmail.com>
 <CA+3eh13TatCOTj-no=geQQE_+FD2OOO8gYLQGg8cqAuq7ZOdnQ@mail.gmail.com>
Message-ID: <5ebae9f0-3bfd-41bf-ac6a-1c6cd2f60cda@gmail.com>

On 08.04.2020 20:49, Volker Simonis wrote:

> Are you sure that the "backtrace" command in gdb is using the
> "backtrace()" function from libgcc?
> 
> Just asking because I'm not sure. I thought gdb has its own stack
> unwinding logic and applies some extra tricks but it's actually quite
> a while since I've last looked into the sources :)

Apparently you were lazy and didn't try it yourself ;)

The snippet prints backtrace() to stdout.
gdb's `bt` gives gdb backtrace with all the tricks applied.

Then you compare the two and find that they are almost identical.

From volker.simonis at gmail.com  Wed Apr  8 19:06:32 2020
From: volker.simonis at gmail.com (Volker Simonis)
Date: Wed, 8 Apr 2020 21:06:32 +0200
Subject: Linux crash native stacks only have 1 line
In-Reply-To: <5ebae9f0-3bfd-41bf-ac6a-1c6cd2f60cda@gmail.com>
References: <805a02c4-a765-dbd7-52f7-553e232f87d0@gmail.com>
 <6da1abd3-9b77-9daf-c21c-4913227fbce2@oracle.com>
 <d29477cb-0596-6204-fc07-23ef7ec61f5b@gmail.com>
 <CA+3eh11tSrjTHCMFSSOeUdk=z1WKUdTkxpWoaH9-6k6wmsxccw@mail.gmail.com>
 <3e5f2300-5f9b-43e4-0ea7-c923af24df9f@gmail.com>
 <CA+3eh12tyQ_vNt5CvqTNdzV_iy16g2Fpz9VkKuJ9nCF9+y2soA@mail.gmail.com>
 <0d660ac9-ace6-f085-3d1c-baff52c37004@gmail.com>
 <CA+3eh107v1ysB91f_yHJdB3TT4asaepd46TrOpe=nsBxUwo9Eg@mail.gmail.com>
 <6731a867-7a5a-eaee-7ffc-55f0f803ca4f@gmail.com>
 <CA+3eh13TatCOTj-no=geQQE_+FD2OOO8gYLQGg8cqAuq7ZOdnQ@mail.gmail.com>
 <5ebae9f0-3bfd-41bf-ac6a-1c6cd2f60cda@gmail.com>
Message-ID: <CA+3eh10gFvLzzXfrv_Ay_CcyaW11N_G+joc5UbQPUuy27W-e1Q@mail.gmail.com>

On Wed, Apr 8, 2020 at 8:52 PM Alexander Miloslavskiy
<alexandr.miloslavskiy at gmail.com> wrote:
>
> On 08.04.2020 20:49, Volker Simonis wrote:
>
> > Are you sure that the "backtrace" command in gdb is using the
> > "backtrace()" function from libgcc?
> >
> > Just asking because I'm not sure. I thought gdb has its own stack
> > unwinding logic and applies some extra tricks but it's actually quite
> > a while since I've last looked into the sources :)
>
> Apparently you were lazy and didn't try it yourself ;)
>

No, I didn't and yes, I am :)

> The snippet prints backtrace() to stdout.
> gdb's `bt` gives gdb backtrace with all the tricks applied.
>
> Then you compare the two and find that they are almost identical.

Sounds good.

From vladimir.kozlov at oracle.com  Wed Apr  8 22:38:16 2020
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Wed, 8 Apr 2020 15:38:16 -0700
Subject: RFR(T) : 8241707 : introduce randomness k/w to hotspot test suite
In-Reply-To: <27D72A05-4B4D-4751-AAAA-45CAD8E15C68@oracle.com>
References: <27D72A05-4B4D-4751-AAAA-45CAD8E15C68@oracle.com>
Message-ID: <ddacdf5f-a789-3a68-4521-3f75a3174dea@oracle.com>

Looks good.

Thanks,
Vladimir K

On 4/7/20 1:26 PM, Igor Ignatyev wrote:
> http://cr.openjdk.java.net/~iignatyev//8241707/webrev.00
>> 5 lines changed: 2 ins; 0 del; 3 mod;
> 
> Hi all,
> 
> could you please review this small and trivial patch which adds "randomness" k/w into hotspot test suite and marks one testlibrary(/shared/common/sanity) test w/ this k/w?
> 
> other tests are to be updated by other subtasks of 8241623[*]. the end goal of this activity is to improve reproducibility of hotspot test suite and mark all the tests whose behavior might vary from run to run (for the reasons unrelated to the product) w/ "randomness" k/w so it will be easier to identify such tests and be more cautious when they fail and/or are used for fix verification.
>         
> from JBS:
>> introduce 'randomness' k/w to mark tests which may vary from run to run due to usage of (pseudo)random. it's important to mention that the same k/w is used w/ the exact same meaning in the jdk test suite, so this will also further reduce differences b/w these two test bases.
>>
>> this is just a preparation state, the actual marking will be done by other sub-tasks of JDK-8241623 (most probably component-by-component).
> 
> 
> 
> [*] https://bugs.openjdk.java.net/browse/JDK-8241623
> JBS: https://bugs.openjdk.java.net/browse/JDK-8241707
> webrev: http://cr.openjdk.java.net/~iignatyev//8241707/webrev.00
> 
> Thanks,
> -- Igor
> 

From igor.ignatyev at oracle.com  Wed Apr  8 22:43:36 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Wed, 8 Apr 2020 15:43:36 -0700
Subject: RFR(T) : 8241707 : introduce randomness k/w to hotspot test suite
In-Reply-To: <ddacdf5f-a789-3a68-4521-3f75a3174dea@oracle.com>
References: <27D72A05-4B4D-4751-AAAA-45CAD8E15C68@oracle.com>
 <ddacdf5f-a789-3a68-4521-3f75a3174dea@oracle.com>
Message-ID: <DB433DD0-64A8-4A2A-ACCA-C35655162B94@oracle.com>

Vladimir, Misha,

Thank you!

-- Igor

> On Apr 8, 2020, at 3:38 PM, Vladimir Kozlov <vladimir.kozlov at oracle.com> wrote:
> 
> Looks good.
> 
> Thanks,
> Vladimir K

> On Apr 7, 2020, at 1:34 PM, mikhailo.seledtsov at oracle.com wrote:
> 
> Looks good,
> 
> Misha

> 
> On 4/7/20 1:26 PM, Igor Ignatyev wrote:
>> http://cr.openjdk.java.net/~iignatyev//8241707/webrev.00
>>> 5 lines changed: 2 ins; 0 del; 3 mod;
>> Hi all,
>> could you please review this small and trivial patch which adds "randomness" k/w into hotspot test suite and marks one testlibrary(/shared/common/sanity) test w/ this k/w?
>> other tests are to be updated by other subtasks of 8241623[*]. the end goal of this activity is to improve reproducibility of hotspot test suite and mark all the tests whose behavior might vary from run to run (for the reasons unrelated to the product) w/ "randomness" k/w so it will be easier to identify such tests and be more cautious when they fail and/or are used for fix verification.
>>        from JBS:
>>> introduce 'randomness' k/w to mark tests which may vary from run to run due to usage of (pseudo)random. it's important to mention that the same k/w is used w/ the exact same meaning in the jdk test suite, so this will also further reduce differences b/w these two test bases.
>>> 
>>> this is just a preparation state, the actual marking will be done by other sub-tasks of JDK-8241623 (most probably component-by-component).
>> [*] https://bugs.openjdk.java.net/browse/JDK-8241623
>> JBS: https://bugs.openjdk.java.net/browse/JDK-8241707
>> webrev: http://cr.openjdk.java.net/~iignatyev//8241707/webrev.00
>> Thanks,
>> -- Igor


From alexandr.miloslavskiy at gmail.com  Wed Apr  8 22:43:44 2020
From: alexandr.miloslavskiy at gmail.com (Alexander Miloslavskiy)
Date: Thu, 9 Apr 2020 00:43:44 +0200
Subject: Linux crash native stacks only have 1 line
In-Reply-To: <CA+3eh10gFvLzzXfrv_Ay_CcyaW11N_G+joc5UbQPUuy27W-e1Q@mail.gmail.com>
References: <805a02c4-a765-dbd7-52f7-553e232f87d0@gmail.com>
 <6da1abd3-9b77-9daf-c21c-4913227fbce2@oracle.com>
 <d29477cb-0596-6204-fc07-23ef7ec61f5b@gmail.com>
 <CA+3eh11tSrjTHCMFSSOeUdk=z1WKUdTkxpWoaH9-6k6wmsxccw@mail.gmail.com>
 <3e5f2300-5f9b-43e4-0ea7-c923af24df9f@gmail.com>
 <CA+3eh12tyQ_vNt5CvqTNdzV_iy16g2Fpz9VkKuJ9nCF9+y2soA@mail.gmail.com>
 <0d660ac9-ace6-f085-3d1c-baff52c37004@gmail.com>
 <CA+3eh107v1ysB91f_yHJdB3TT4asaepd46TrOpe=nsBxUwo9Eg@mail.gmail.com>
 <6731a867-7a5a-eaee-7ffc-55f0f803ca4f@gmail.com>
 <CA+3eh13TatCOTj-no=geQQE_+FD2OOO8gYLQGg8cqAuq7ZOdnQ@mail.gmail.com>
 <5ebae9f0-3bfd-41bf-ac6a-1c6cd2f60cda@gmail.com>
 <CA+3eh10gFvLzzXfrv_Ay_CcyaW11N_G+joc5UbQPUuy27W-e1Q@mail.gmail.com>
Message-ID: <693e8a1f-e782-b4f6-6eb3-e4ddd6264c44@gmail.com>

Hmm, after trying to cobble something together with `_Unwind_Backtrace`, 
I found that it's not going to be too easy: it's undocumented, so will 
be subject to trial&error, which doesn't sound too good.

Back to libunwind, I noticed that its license says:
----
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:

The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
----

Is that really a licensing problem for JDK?

From david.holmes at oracle.com  Thu Apr  9 00:35:27 2020
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 9 Apr 2020 10:35:27 +1000
Subject: RFR(XS): 8174768: Make ProcessTools print executed process output
 into a separate file
In-Reply-To: <da4ab22e-b126-4b27-e744-67e572326608@oracle.com>
References: <428675e0-f09c-ac6b-f5ec-afc4190a6b45@oracle.com>
 <16A3371B-EABC-4955-902C-78EB079094DB@oracle.com>
 <da4ab22e-b126-4b27-e744-67e572326608@oracle.com>
Message-ID: <23e587ec-4673-e0a2-3345-f0420c9b3eeb@oracle.com>

Looks good.

Thanks,
David

On 9/04/2020 3:27 am, Evgeny Nikitin wrote:
> Thanks to Igor Ignatyev, a new version is available at
> 
> http://cr.openjdk.java.net/~iignatyev/enikitin/8174768/webrev.02
> 
> Regards,
> Evgeny.
> 
> 
> On 2020-04-08 17:19, Igor Ignatev wrote:
>> You can use %n in printf to get a system-specific newline.
>>
>> ? Igor
>>
>>> On Apr 8, 2020, at 8:12 AM, Evgeny Nikitin 
>>> <evgeny.nikitin at oracle.com> wrote:
>>>
>>> ?Hi David,
>>>
>>>> ? > You can use printf rather than making a separate call to format.
>>>> Good point, I've missed that method. I'll fix it and ask Igor to change
>>>> the webrev.
>>>
>>> I finally decided to leave it as it is now, since printf doesn't add 
>>> a newline and System.lineSeparator() is not shorter than 
>>> String.format() :). Please let me know if you disagree.
>>>
>>> Best regards,
>>> Evgeny.
>>>
>>>
>>>> On 2020-04-07 16:09, Evgeny Nikitin wrote:
>>>> Hi David,
>>>>> why did you introduce a new block scope?
>>>> To separate the action block from the other code. "A Poor/lazy man's 
>>>> method". I've preferred to lay it out this way because it makes the 
>>>> code more compact and easy to read (as opposing to looking for a 
>>>> only-once used method in some other part of the file) and due to the 
>>>> confusing name of OutputAnalyzer type (which I need as a storage for 
>>>> output, not as some analyzer). Creating a method with OutputAnalyzer 
>>>> as a parameter would make this method's purpose even more confusing.
>>>>> You can use printf rather than making a separate call to format.
>>>> Good point, I've missed that method. I'll fix it and ask Igor to 
>>>> change the webrev.
>>>> Regards,
>>>> Evgeny.
>>>>> On 2020-04-07 14:50, David Holmes wrote:
>>>>> Hi Evgeny,
>>>>>
>>>>> On 7/04/2020 6:33 pm, Evgeny Nikitin wrote:
>>>>>> Hi David,
>>>>>>
>>>>>>> I'm not at all sure this is generally what we want for every 
>>>>>>> single test that uses ProcessTools! But I'm willing it to see it 
>>>>>>> trialed.
>>>>>>
>>>>>> Well, it's mostly used for test VM runs. In runs I performed 
>>>>>> (artificial "failures" included) the amounts of output were very 
>>>>>> small.
>>>>>>
>>>>>>> Please run full tier testing at least to tier 6 and ideally 
>>>>>>> beyond before pushing this. There are potential implications for 
>>>>>>> temporary (and more permanent) disk usage as well as additional 
>>>>>>> time needed to write files out to disk. (Hopefully these are 
>>>>>>> generally small enough that this doesn't make a noticeable 
>>>>>>> difference.)
>>>>>> Done, thanks for suggestion. The additional runs showed no problems.
>>>>>
>>>>> Good to know - thanks.
>>>>>
>>>>>> I've provided Igor with a slightly modified version (Added 
>>>>>> notification about the output file into the main test's log). 
>>>>>> Please review:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~iignatyev/enikitin/8174768/webrev.01/
>>>>>
>>>>> A couple of minor nits:
>>>>>
>>>>> ? 397???????????? {
>>>>>
>>>>> why did you introduce a new block scope?
>>>>>
>>>>> ? 404???????????????? System.out.println(String.format(
>>>>>
>>>>> You can use printf rather than making a separate call to format.
>>>>>
>>>>> No need to see any update if you chose to make it.
>>>>>
>>>>> Thanks,
>>>>> David
>>>>> -----
>>>>>
>>>>>> Best Regards,
>>>>>>
>>>>>> Evgeny.
>>>>>>
>>>>>> On 2020-04-02 02:07, David Holmes wrote:
>>>>>>> Thanks for sharing this Igor!
>>>>>>>
>>>>>>> I'm not at all sure this is generally what we want for every 
>>>>>>> single test that uses ProcessTools! But I'm willing it to see it 
>>>>>>> trialed.
>>>>>>>
>>>>>>> Evgeny: Please run full tier testing at least to tier 6 and 
>>>>>>> ideally beyond before pushing this. There are potential 
>>>>>>> implications for temporary (and more permanent) disk usage as 
>>>>>>> well as additional time needed to write files out to disk. 
>>>>>>> (Hopefully these are generally small enough that this doesn't 
>>>>>>> make a noticeable difference.)
>>>>>>>
>>>>>>> Thanks,
>>>>>>> David
>>>>>>>
>>>>>>> On 2/04/2020 5:13 am, Igor Ignatyev wrote:
>>>>>>>> Hi Evgeny,
>>>>>>>>
>>>>>>>> (widening the audience, given this affects not just hotspot 
>>>>>>>> compiler, but hotspot tests as well as core libs tests in general)
>>>>>>>>
>>>>>>>> overall that looks good to me. one suggestion, for the ease of 
>>>>>>>> failure analysis it might be worth to print out the names of 
>>>>>>>> created files, although this might potentially clutter the 
>>>>>>>> output, I don't think it'll be a problem given we already print 
>>>>>>>> out things like 'Gathering output for process ...' , 'Waiting 
>>>>>>>> for completion...' in LazyOutputBuffer.
>>>>>>>>
>>>>>>>>> The change has been tested via a mach5 test runs (jdk-tier1 
>>>>>>>>> through 4) on the 4 common platforms (linux-x64, windows-x64, 
>>>>>>>>> macosx-x64, sparcv9).
>>>>>>>> this doesn't include any of hotspot tiers, could you please also 
>>>>>>>> run hs-tier1--4?
>>>>>>>> // you can use tierN jobs which include both jdk and hs parts.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -- Igor
>>>>>>>>
>>>>>>>>> On Mar 30, 2020, at 3:55 AM, Evgeny Nikitin 
>>>>>>>>> <evgeny.nikitin at oracle.com> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8174768
>>>>>>>>>
>>>>>>>>> Webrev: 
>>>>>>>>> http://cr.openjdk.java.net/~iignatyev/enikitin/8174768/webrev.00/
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> The bug had been created as a request to simplify investigation 
>>>>>>>>> for compiler control tests failures.
>>>>>>>>> I found the functionality pretty generic and useful and made 
>>>>>>>>> ProcessTools dump output as well as some diagnostic information 
>>>>>>>>> for every executed process into a separate file.
>>>>>>>>> The diagnostic information contains cmdline, exit code, stdout 
>>>>>>>>> and stderr. The output files are named like 
>>>>>>>>> 'pid-<PID>-output.log'.
>>>>>>>>>
>>>>>>>>> The change has been tested via a mach5 test runs (jdk-tier1 
>>>>>>>>> through 4) on the 4 common platforms (linux-x64, windows-x64, 
>>>>>>>>> macosx-x64, sparcv9).
>>>>>>>>>
>>>>>>>>> Please review,
>>>>>>>>> /Evgeny Nikitin.
>>>>>>>>
>>

From Roger.Riggs at oracle.com  Thu Apr  9 13:48:29 2020
From: Roger.Riggs at oracle.com (Roger Riggs)
Date: Thu, 9 Apr 2020 09:48:29 -0400
Subject: RFR 15 8242462: Residual Cleanup of rmic removal
Message-ID: <90bb7028-922a-a322-a2cf-844d43424b18@oracle.com>

Please review a few cleanups I missed on the removal of rmic.
I didn't get a reply on the changes to the hotspot (cds) tests.
(I did get an ok on the javadoc changes)

Webrev:
http://cr.openjdk.java.net/~rriggs/webrev-remove-rmic-8225319-misc-1/

Issue:
 ? https://bugs.openjdk.java.net/browse/JDK-8242462

Thanks, Roger


From lois.foltan at oracle.com  Thu Apr  9 18:42:21 2020
From: lois.foltan at oracle.com (Lois Foltan)
Date: Thu, 9 Apr 2020 14:42:21 -0400
Subject: RFR (M) JDK-8210012: Implement Unified Logging Option for
 -XX:+TraceMethodHandles and -XX:+TraceInvokeDynamic
Message-ID: <07752bc9-3c6f-6b49-6403-bce3080e54cf@oracle.com>

Please review the following change to implement unified logging options 
for -XX:+TraceMethodHandles and -XX:+TraceInvokeDynamic. The new options 
map as follows:

-XX:+TraceMethodHandles --> -Xlog:methodhandles=info
-XX:+TraceInvokeDynamic -->? -Xlog:methodhandles+indy=debug

and in addition dynamic constants can now be viewed under their own 
option via -Xlog:methodhandles+condy=debug

open webrev at: http://cr.openjdk.java.net/~lfoltan/bug_jdk8210012.0/webrev/
bug link: https://bugs.openjdk.java.net/browse/JDK-8210012

Testing: hs-tier1-5

Thanks,
Lois

From stuart.monteith at arm.com  Thu Apr  9 19:18:01 2020
From: stuart.monteith at arm.com (Stuart Monteith)
Date: Thu, 9 Apr 2020 20:18:01 +0100
Subject: RFR(S): 8241587: Aarch64: remove x86 specifics from
 os_linux.cpp/hpp/inline.hpp
In-Reply-To: <4c10a542-f629-3a37-3d11-8809d70ebeea@oracle.com>
References: <9be2bcfe-faa4-1dc0-53fe-989c962f0ad7@arm.com>
 <4c10a542-f629-3a37-3d11-8809d70ebeea@oracle.com>
Message-ID: <ec4efd73-4d23-3d9b-5542-8cd269f18cdd@arm.com>

Thanks David. I'll need someone to push this for me. Ningsheng - would
you be able to?

Thanks,
	Stuart

On 01/04/2020 11:03, David Holmes wrote:
> Hi Stuart,
> 
> On 1/04/2020 7:29 pm, Stuart Monteith wrote:
>> Hello,
>> ???????? This patch removes a couple of x86 specifics from aarch64
>> code. Tested with hotspot tier1.
>>
>> Webrev:
>> ???????? http://cr.openjdk.java.net/~smonteith/8241587/webrev.0/
>> Bug:
>> ???????? https://bugs.openjdk.java.net/browse/JDK-8241587
> 
> That clean up seems good to me.
> 
>> Thanks,
>> ???????? Stuart
>>
>> IMPORTANT NOTICE: The contents of this email and any attachments are
>> confidential and may also be privileged. If you are not the intended
>> recipient, please notify the sender immediately and do not disclose
>> the contents to any other person, use it for any purpose, or store or
>> copy the information in any medium. Thank you.
> 
> That footer seems inappropriate for OpenJDK emails.
> 
> Cheers,
> David
> 


From john.r.rose at oracle.com  Thu Apr  9 21:03:28 2020
From: john.r.rose at oracle.com (John Rose)
Date: Thu, 9 Apr 2020 14:03:28 -0700
Subject: RFR (M) JDK-8210012: Implement Unified Logging Option for
 -XX:+TraceMethodHandles and -XX:+TraceInvokeDynamic
In-Reply-To: <07752bc9-3c6f-6b49-6403-bce3080e54cf@oracle.com>
References: <07752bc9-3c6f-6b49-6403-bce3080e54cf@oracle.com>
Message-ID: <ACCBE810-8524-4B40-9871-C37A2E66A0A1@oracle.com>

Reviewed.  Nice things I saw:

 - fewer uses of ttyLocker
 - indy-related diagnostics getting into UL
 - UL being strong enough to handle complex multi-line formatted output

Side comment about UL:  Last time I looked, which was a
while ago, UL was not friendly to multi-line outputs.

Eventually, UL should be applied to -XX:+LogCompilation, but
that will require UL to manage large structured segments of
logging data.  -XX:+LC produces a large XML-flavored report
for each distinct compilation task.

> On Apr 9, 2020, at 11:42 AM, Lois Foltan <lois.foltan at oracle.com> wrote:
> 
> Please review the following change to implement unified logging options for -XX:+TraceMethodHandles and -XX:+TraceInvokeDynamic. The new options map as follows:
> 
> -XX:+TraceMethodHandles --> -Xlog:methodhandles=info
> -XX:+TraceInvokeDynamic -->  -Xlog:methodhandles+indy=debug
> 
> and in addition dynamic constants can now be viewed under their own option via -Xlog:methodhandles+condy=debug
> 
> open webrev at: http://cr.openjdk.java.net/~lfoltan/bug_jdk8210012.0/webrev/
> bug link: https://bugs.openjdk.java.net/browse/JDK-8210012
> 
> Testing: hs-tier1-5
> 
> Thanks,
> Lois


From ioi.lam at oracle.com  Fri Apr 10 00:06:56 2020
From: ioi.lam at oracle.com (Ioi Lam)
Date: Thu, 9 Apr 2020 17:06:56 -0700
Subject: RFC 8236988 - Modular Design for JVM Flags
Message-ID: <d067ca79-5cad-58b9-11e4-e078982ba15d@oracle.com>

The code for specifying HotSpot command-line flags used to be simple. At 
the very beginning it was something like

 ??? #define ALL_FLAGS(develop, develop) \
 ? ? ?? product(int, SomeFlag, def_value, "some doc") \
 ?????? develop(int, MoreFlag, def_value, "more doc") \

but like all good ideas, it has gradually degraded into a mess [1][2]. 
It's now very difficult to add any new functionality (e.g., add 
"manageable" annotations to the flags [3]).

I think it's time for a complete rewrite :-)

My proposal is here. Please comment if it's a good or bad idea.

https://wiki.openjdk.java.net/display/HotSpot/HotSpot+Command-Line+Flags+Overhaul+-+Design+Doc



tldr; ---


OLD:

 ? product(intx, AliasLevel, 3,????????????????????????????????????????? \
 ????????? "0 for no aliasing, 1 for oop/field/static/array split, 
"???????? \
 ????????? "2 for class split, 3 for unique 
instances")????????????????????? \
 ????????? range(0, 3) 
/*optional*/???????????????????????????????????????? \
 ????????? constraint(AliasLevelConstraintFunc,AfterErgo) /*optional*/ ? 
 ??? \


NEW:

 ? PRODUCT_FLAG(intx,?? AliasLevel, 3, JVMFlag::RANGE | JVMFlag::CONSTRAINT,
 ?????????????????????? "0 for no aliasing, 1 for oop/field/static/array 
split, "
 ?????????????????????? "2 for class split, 3 for unique instances");
 ?? // optional
 ?? FLAG_RANGE(???????? AliasLevel, 0, 3);
 ?? // optional
 ?? FLAG_CONSTRAINT(??? AliasLevel, (void*)AliasLevelConstraintFunc, 
JVMFlag::AfterErgo);




[1] 
http://hg.openjdk.java.net/jdk/jdk/file/7af6364e1792/src/hotspot/share/runtime/globals.hpp#l115
[2] 
http://hg.openjdk.java.net/jdk/jdk/file/7af6364e1792/src/hotspot/share/runtime/flags/jvmFlag.cpp#l635
[3] https://bugs.openjdk.java.net/browse/JDK-7123237

Thanks
- Ioi

From ningsheng.jian at arm.com  Fri Apr 10 02:15:48 2020
From: ningsheng.jian at arm.com (Ningsheng Jian)
Date: Fri, 10 Apr 2020 10:15:48 +0800
Subject: RFR(S): 8241587: Aarch64: remove x86 specifics from
 os_linux.cpp/hpp/inline.hpp
In-Reply-To: <ec4efd73-4d23-3d9b-5542-8cd269f18cdd@arm.com>
References: <9be2bcfe-faa4-1dc0-53fe-989c962f0ad7@arm.com>
 <4c10a542-f629-3a37-3d11-8809d70ebeea@oracle.com>
 <ec4efd73-4d23-3d9b-5542-8cd269f18cdd@arm.com>
Message-ID: <7641c396-fde8-dc35-c355-18faba5c5a39@arm.com>

On 4/10/20 3:18 AM, Stuart Monteith wrote:
> Thanks David. I'll need someone to push this for me. Ningsheng - would
> you be able to?
> 

Pushed.

Thanks,
Ningsheng

From david.holmes at oracle.com  Fri Apr 10 03:12:09 2020
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 10 Apr 2020 13:12:09 +1000
Subject: RFR (M) JDK-8210012: Implement Unified Logging Option for
 -XX:+TraceMethodHandles and -XX:+TraceInvokeDynamic
In-Reply-To: <07752bc9-3c6f-6b49-6403-bce3080e54cf@oracle.com>
References: <07752bc9-3c6f-6b49-6403-bce3080e54cf@oracle.com>
Message-ID: <41e05f04-e6f3-cba4-d1d3-6bc1b156071d@oracle.com>

Hi Lois,

On 10/04/2020 4:42 am, Lois Foltan wrote:
> Please review the following change to implement unified logging options 
> for -XX:+TraceMethodHandles and -XX:+TraceInvokeDynamic. The new options 
> map as follows:
> 
> -XX:+TraceMethodHandles --> -Xlog:methodhandles=info
> -XX:+TraceInvokeDynamic -->? -Xlog:methodhandles+indy=debug
> 
> and in addition dynamic constants can now be viewed under their own 
> option via -Xlog:methodhandles+condy=debug
> 
> open webrev at: 
> http://cr.openjdk.java.net/~lfoltan/bug_jdk8210012.0/webrev/
> bug link: https://bugs.openjdk.java.net/browse/JDK-8210012

Generally looks good. A couple of things.

First the change in compileTask.cpp seems wrong: you should get the 
timestamp from the specified stream not the tty.

Second the changes to wrap_dynamic_exception seem rather awkward, both 
in terms of having to remember what the initial boolean arg represents 
at the call site, and more so in the logic to determine which log stream 
to use. I was wondering if it might be better to add 
wrap_dynamic_exception_for_indy to keep the call sites clean? Otherwise 
perhaps document like "true /* indy */" and "false /* not indy */.

For the log stream logic I was assuming there must be a better/simpler 
way, but after looking at it in some detail it seems not. :(

Thanks,
David

> Testing: hs-tier1-5
> 
> Thanks,
> Lois

From ioi.lam at oracle.com  Fri Apr 10 04:07:07 2020
From: ioi.lam at oracle.com (Ioi Lam)
Date: Thu, 9 Apr 2020 21:07:07 -0700
Subject: RFR (M) JDK-8210012: Implement Unified Logging Option for
 -XX:+TraceMethodHandles and -XX:+TraceInvokeDynamic
In-Reply-To: <41e05f04-e6f3-cba4-d1d3-6bc1b156071d@oracle.com>
References: <07752bc9-3c6f-6b49-6403-bce3080e54cf@oracle.com>
 <41e05f04-e6f3-cba4-d1d3-6bc1b156071d@oracle.com>
Message-ID: <9f678d60-93c1-f2e3-84a1-51feb74c6961@oracle.com>



On 4/9/20 8:12 PM, David Holmes wrote:
> Hi Lois,
>
> On 10/04/2020 4:42 am, Lois Foltan wrote:
>> Please review the following change to implement unified logging 
>> options for -XX:+TraceMethodHandles and -XX:+TraceInvokeDynamic. The 
>> new options map as follows:
>>
>> -XX:+TraceMethodHandles --> -Xlog:methodhandles=info
>> -XX:+TraceInvokeDynamic -->? -Xlog:methodhandles+indy=debug
>>
>> and in addition dynamic constants can now be viewed under their own 
>> option via -Xlog:methodhandles+condy=debug
>>
>> open webrev at: 
>> http://cr.openjdk.java.net/~lfoltan/bug_jdk8210012.0/webrev/
>> bug link: https://bugs.openjdk.java.net/browse/JDK-8210012
>
> Generally looks good. A couple of things.
>
> First the change in compileTask.cpp seems wrong: you should get the 
> timestamp from the specified stream not the tty.
>
> Second the changes to wrap_dynamic_exception seem rather awkward, both 
> in terms of having to remember what the initial boolean arg represents 
> at the call site, and more so in the logic to determine which log 
> stream to use. I was wondering if it might be better to add 
> wrap_dynamic_exception_for_indy to keep the call sites clean? 
> Otherwise perhaps document like "true /* indy */" and "false /* not 
> indy */.
>
> For the log stream logic I was assuming there must be a better/simpler 
> way, but after looking at it in some detail it seems not. :(
>

Hi Lois,


LinkResolver::lookup_polymorphic_method()
+? ResourceMark rm(THREAD);

Is this ResourceMark used only for logging?

=================
For code like this in ConstantPool::resolve_constant_at_impl:

 ??? if (log_is_enabled(Debug, methodhandles, condy)) {
 ????? LogTarget(Debug, methodhandles, condy) lt;
 ????? LogStream ls(lt);
 ????? bootstrap_specifier.print_msg_on(&ls, "resolve_constant_at_impl");
 ??? }

The second set of tags can be avoided by declaring LogTarget(Debug, 
methodhandles, condy) first. There's an example of this style in 
FileMapInfo::log_paths().

 ??? LogTarget(Debug, methodhandles, condy) lt;
 ??? if (lt.is_enabled()) {
 ????? LogStream ls(lt);
 ????? bootstrap_specifier.print_msg_on(&ls, "resolve_constant_at_impl");
 ??? }

=================

unpack_method_and_appendix() {

 ????? if (log_develop_is_enabled(Info, methodhandles)) {
 ??????? ResourceMark rm(THREAD);
 ??????? LogTarget(Info, methodhandles) lt;
 ??????? LogStream ls(lt);
 ??????? ls->print_on(.....);

This can be fixed similarly, if we add a new function to LogTargetImpl:

 ?? class LogTargetImpl {
 ? ?? static bool develop_is_enabled() {
 ?? ? ? NOT_PRODUCT(return LogImpl<T0, T1, T2, T3, T4, 
GuardTag>::is_level(level));
 ?????? PRODUCT_ONLY(return false);
 ? ?? }
 ???? ...
 ?? }

 ? ? ? LogTarget(Info, methodhandles) lt;
 ????? if (lt.develop_is_enabled()) {
 ??????? ResourceMark rm(THREAD);
 ??????? LogStream ls(lt);
 ??????? ls->print_on(.....);

=================
void ConstantPoolCacheEntry::set_method_handle_common()

 ? LogStreamHandle(Debug, methodhandles, indy) lsh;
 ? if (log_is_enabled(Debug, methodhandles, indy)) {

Maybe we should add a new function LogStream::is_enabled so we can:

 ? LogStreamHandle(Debug, methodhandles, indy) lsh;
 ? if (lsh.is_enabled()) {


=================
Not a comment about this patch, but in general, I am not happy with the 
verbosity when a ResourceMark is needed in logging.

I think the best way to avoid the verbosity would be to use C++ lambda 
expressions (are we allowed??), or (oh gosh!) maybe a macro? (Probably 
in a separate RFE).

 ??? LOG_WITH_RESOURCE_MARK((Info, methodhandles), THREAD, ls, {
 ????? ? ls->print_on(.....);
 ?? ?? });


Thanks
- Ioi


> Thanks,
> David
>
>> Testing: hs-tier1-5
>>
>> Thanks,
>> Lois


From denghui.ddh at alibaba-inc.com  Fri Apr 10 12:34:28 2020
From: denghui.ddh at alibaba-inc.com (Denghui Dong)
Date: Fri, 10 Apr 2020 20:34:28 +0800
Subject: =?UTF-8?B?UkZSKFhYUyk6IDgyNDI0ODU6IE51bGwgX2ZpbGUgY2hlY2tpbmcgaW4gZmlsZVN0cmVhbTo6?=
 =?UTF-8?B?Zmx1c2goKQ==?=
Message-ID: <e8d66934-57b9-44e5-a1b2-c650ea101220.denghui.ddh@alibaba-inc.com>

Hi team,

Could you please review this small patch?

JBS: https://bugs.openjdk.java.net/browse/JDK-8242485
webrev: http://cr.openjdk.java.net/~ddong/8242485/webrev.00/

I found the method "fileStream::flush()" lacks null check before fflush  which will cause
all open output streams are flushed by fflush() if _file is NULL.

Thanks,
Denghui Dong

From bob.vandette at oracle.com  Fri Apr 10 14:27:52 2020
From: bob.vandette at oracle.com (Bob Vandette)
Date: Fri, 10 Apr 2020 10:27:52 -0400
Subject: RFR: 8241423: NUMA APIs may fail to work in the docker due to
 operation not permitted
In-Reply-To: <97987050-6175-44EC-8ABE-B51D60E513AF@tencent.com>
References: <97987050-6175-44EC-8ABE-B51D60E513AF@tencent.com>
Message-ID: <312806A9-2A4F-42CE-8DA1-2EF351971DFC@oracle.com>


> On Apr 3, 2020, at 3:58 AM, jiefu(??) <jiefu at tencent.com> wrote:
> 
> Hi Bob,
> 
> Thanks for your review and helpful comments. 
> 
> I'm not a docker expert. 
> Apart from the zgc crash [1], we didn't come across other problems in the docker.
> 
> It seems that this bug has nothing to do with the resource limit.
> The root cause is that some NUMA-related syscalls are disabled in the docker for safety reasons.

I understand but it would be useful to ensure that all aspects of supporting NUMA in containers
work while addressing this issue.  What good is it to enable NUMA but have it not work properly.
I?m only asking for some validation since you appear to have a proper setup.
 
> 
> Please note that we already have numa_available() check here [2].
> But it failed to detect such cases.
Ah yes, I see.

> 
> What do you think?

It would be better if we could find a more supported way  of determining if our process has
the required access rather than counting on a failing syscall.

What about using prctl?

http://man7.org/linux/man-pages/man2/prctl.2.html

This call can be used to query if a specific capability is enabled.

Bob.


> 
> Thanks a lot.
> Best regards,
> Jie
> 
> [1] https://bugs.openjdk.java.net/browse/JDK-8241354
> [2] http://hg.openjdk.java.net/jdk/jdk/file/f50a7df94744/src/hotspot/os/linux/os_linux.cpp#l3182
> 
> ?On 2020/4/3, 3:58 AM, "Bob Vandette" <bob.vandette at oracle.com> wrote:
> 
>    Jie,
> 
>    Before we discuss this specific fix, I?d like to know if you have confirmed that Hotspot?s NUMA
>    support actually functions properly when running in containers (with proper privs)?
> 
>    Also, do the libnuma functions work properly in response to cgroup limitations imposed by docker run --cpuset-mems?
> 
>    Some of the traditional kernel functions reporting resource limits only report host values and do not
>    correctly report limits specified for containers.   To resolve this issue I have added an osContainer
>    class to hotspot.  Included in this class is a function that reports the memory nodes available 
>    to hotspot when running in a container.   It might be necessary to query this function when
>    trying to configure the hotspot NUMA support.
> 
>    Back to your webrev, is it not possible to get the address for numa_available and
>    then try to calling it in order to determine if NUMA can be used?
> 
>    If it is determined that you don?t have sufficient access, I would suggest disabling UseNUMA
>    all together.
> 
>    Bob
> 
>> On Mar 23, 2020, at 11:58 AM, jiefu(??) <jiefu at tencent.com> wrote:
>> 
>> Hi all,
>> 
>> JBS:    https://bugs.openjdk.java.net/browse/JDK-8241423
>> Webrev: http://cr.openjdk.java.net/~jiefu/8241423/webrev.00/
>> 
>> A VM fatal error may be observed if ZGC is used (see JDK-8241354).
>> The background is that some of our products run in the docker.
>> And for safety reasons, SYS_get_mempolicy is not allowed by default [1].
>> 
>> At first, we thought it just a zgc-only problem and filed JDK-8241354.
>> But Thomas had reminded me that other collectors are also affected [2].
>> So it would be better to fix them together.
>> 
>> After more investigation, we found that NUMA APIs are actually dependent on several syscalls, such as get_mempolicy, mbind and set_mempolicy.
>> When the required syscalls are unavailable, NUMA APIs fail to work as expected.
>> 
>> The fix is to check whether the required syscalls are available.
>> In theory, all NUMA-related syscalls should be checked.
>> But it seems hard to do so because some of them will cause unexpected side effect.
>> To fix our issue, checking get_mempolicy is enough.
>> And just as Per suggested that we can refine this later if it turns out to be a problem [3].
>> 
>> Please review it and give me some advice.
>> 
>> Thanks a lot.
>> Best regards,
>> Jie
>> 
>> [1] https://docs.docker.com/engine/security/seccomp/
>> [2] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028923.html
>> [3] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028933.html
> 
> 
> 
> 


From xxinliu at amazon.com  Fri Apr 10 17:15:11 2020
From: xxinliu at amazon.com (Liu, Xin)
Date: Fri, 10 Apr 2020 17:15:11 +0000
Subject: RFR(XXS): 8242485: Null _file checking in fileStream::flush()
In-Reply-To: <e8d66934-57b9-44e5-a1b2-c650ea101220.denghui.ddh@alibaba-inc.com>
References: <e8d66934-57b9-44e5-a1b2-c650ea101220.denghui.ddh@alibaba-inc.com>
Message-ID: <BC32AE94-F647-45C0-83EC-92E29FDC0411@amazon.com>

Hi, Dong, 

LGTM. (I am not reviewer).  
I just realize that fflush accepts NULL, but "If the stream argument is NULL, fflush() flushes all open output streams."
Although it doesn't cause runtime error, it may cause unexpected IO latency. 

Thanks,
--lx

?On 4/10/20, 5:36 AM, "hotspot-dev on behalf of Denghui Dong" <hotspot-dev-bounces at openjdk.java.net on behalf of denghui.ddh at alibaba-inc.com> wrote:

    CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
    
    
    
    Hi team,
    
    Could you please review this small patch?
    
    JBS: https://bugs.openjdk.java.net/browse/JDK-8242485
    webrev: http://cr.openjdk.java.net/~ddong/8242485/webrev.00/
    
    I found the method "fileStream::flush()" lacks null check before fflush  which will cause
    all open output streams are flushed by fflush() if _file is NULL.
    
    Thanks,
    Denghui Dong
    


From igor.ignatyev at oracle.com  Fri Apr 10 18:44:08 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Fri, 10 Apr 2020 11:44:08 -0700
Subject: RFR 15 8242462: Residual Cleanup of rmic removal
In-Reply-To: <90bb7028-922a-a322-a2cf-844d43424b18@oracle.com>
References: <90bb7028-922a-a322-a2cf-844d43424b18@oracle.com>
Message-ID: <B1B7C622-636A-4BCE-A598-D1BF2730B0EE@oracle.com>

Hi Roger,

removal of applications/ctw/modules/jdk_rmic.java and changes in doc (assuming .html was generated from .md) look good to me.

adding hotspot-runtime alias to bring attention of runtime team. I'm not sure who are the right people to review bin/unshuffle_list.txt though,.. build team (cc'ed them)?

Cheers,
-- Igor

> On Apr 9, 2020, at 6:48 AM, Roger Riggs <Roger.Riggs at Oracle.com> wrote:
> 
> Please review a few cleanups I missed on the removal of rmic.
> I didn't get a reply on the changes to the hotspot (cds) tests.
> (I did get an ok on the javadoc changes)
> 
> Webrev:
> http://cr.openjdk.java.net/~rriggs/webrev-remove-rmic-8225319-misc-1/
> 
> Issue:
>   https://bugs.openjdk.java.net/browse/JDK-8242462
> 
> Thanks, Roger
> 


From erik.joelsson at oracle.com  Fri Apr 10 18:52:18 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Fri, 10 Apr 2020 11:52:18 -0700
Subject: RFR 15 8242462: Residual Cleanup of rmic removal
In-Reply-To: <B1B7C622-636A-4BCE-A598-D1BF2730B0EE@oracle.com>
References: <90bb7028-922a-a322-a2cf-844d43424b18@oracle.com>
 <B1B7C622-636A-4BCE-A598-D1BF2730B0EE@oracle.com>
Message-ID: <740845b0-9ec5-dced-982d-d97fab60993a@oracle.com>

unshuffle_list.txt looks good to me.

/Erik

On 2020-04-10 11:44, Igor Ignatyev wrote:
> Hi Roger,
>
> removal of applications/ctw/modules/jdk_rmic.java and changes in doc (assuming .html was generated from .md) look good to me.
>
> adding hotspot-runtime alias to bring attention of runtime team. I'm not sure who are the right people to review bin/unshuffle_list.txt though,.. build team (cc'ed them)?
>
> Cheers,
> -- Igor
>
>> On Apr 9, 2020, at 6:48 AM, Roger Riggs <Roger.Riggs at Oracle.com> wrote:
>>
>> Please review a few cleanups I missed on the removal of rmic.
>> I didn't get a reply on the changes to the hotspot (cds) tests.
>> (I did get an ok on the javadoc changes)
>>
>> Webrev:
>> http://cr.openjdk.java.net/~rriggs/webrev-remove-rmic-8225319-misc-1/
>>
>> Issue:
>>    https://bugs.openjdk.java.net/browse/JDK-8242462
>>
>> Thanks, Roger
>>

From mikhailo.seledtsov at oracle.com  Fri Apr 10 19:28:05 2020
From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com)
Date: Fri, 10 Apr 2020 12:28:05 -0700
Subject: RFR 15 8242462: Residual Cleanup of rmic removal
In-Reply-To: <B1B7C622-636A-4BCE-A598-D1BF2730B0EE@oracle.com>
References: <90bb7028-922a-a322-a2cf-844d43424b18@oracle.com>
 <B1B7C622-636A-4BCE-A598-D1BF2730B0EE@oracle.com>
Message-ID: <c6a28086-6eb9-c9ec-d78e-84a9d108ff3b@oracle.com>

Runtime test changes look good to me,

Misha

On 4/10/20 11:44 AM, Igor Ignatyev wrote:
> Hi Roger,
>
> removal of applications/ctw/modules/jdk_rmic.java and changes in doc (assuming .html was generated from .md) look good to me.
>
> adding hotspot-runtime alias to bring attention of runtime team. I'm not sure who are the right people to review bin/unshuffle_list.txt though,.. build team (cc'ed them)?
>
> Cheers,
> -- Igor
>
>> On Apr 9, 2020, at 6:48 AM, Roger Riggs <Roger.Riggs at Oracle.com> wrote:
>>
>> Please review a few cleanups I missed on the removal of rmic.
>> I didn't get a reply on the changes to the hotspot (cds) tests.
>> (I did get an ok on the javadoc changes)
>>
>> Webrev:
>> http://cr.openjdk.java.net/~rriggs/webrev-remove-rmic-8225319-misc-1/
>>
>> Issue:
>>    https://bugs.openjdk.java.net/browse/JDK-8242462
>>
>> Thanks, Roger
>>

From lois.foltan at oracle.com  Fri Apr 10 21:55:38 2020
From: lois.foltan at oracle.com (Lois Foltan)
Date: Fri, 10 Apr 2020 17:55:38 -0400
Subject: RFR (M) JDK-8210012: Implement Unified Logging Option for
 -XX:+TraceMethodHandles and -XX:+TraceInvokeDynamic
In-Reply-To: <41e05f04-e6f3-cba4-d1d3-6bc1b156071d@oracle.com>
References: <07752bc9-3c6f-6b49-6403-bce3080e54cf@oracle.com>
 <41e05f04-e6f3-cba4-d1d3-6bc1b156071d@oracle.com>
Message-ID: <0319c697-10b0-9aaf-7e9f-2762d2cd1c16@oracle.com>

On 4/9/2020 11:12 PM, David Holmes wrote:
> Hi Lois,
>
> On 10/04/2020 4:42 am, Lois Foltan wrote:
>> Please review the following change to implement unified logging 
>> options for -XX:+TraceMethodHandles and -XX:+TraceInvokeDynamic. The 
>> new options map as follows:
>>
>> -XX:+TraceMethodHandles --> -Xlog:methodhandles=info
>> -XX:+TraceInvokeDynamic -->? -Xlog:methodhandles+indy=debug
>>
>> and in addition dynamic constants can now be viewed under their own 
>> option via -Xlog:methodhandles+condy=debug
>>
>> open webrev at: 
>> http://cr.openjdk.java.net/~lfoltan/bug_jdk8210012.0/webrev/
>> bug link: https://bugs.openjdk.java.net/browse/JDK-8210012
>
> Generally looks good. A couple of things.

Thanks David for the review!

>
> First the change in compileTask.cpp seems wrong: you should get the 
> timestamp from the specified stream not the tty.

At first it looked wrong to me as well, but I was receiving an assert 
about the TimeStamp associated with the outputStream being cleared, (the 
first assert in TimeStamp::milliseconds).? After investigation it turns 
out that CompileTask::print_impl() has never been called with any 
outputStream other than tty.? And 3-or 4 levels up when I call it with a 
LogStream (which inherits from outputStream and constructs with a 
default ctor that does not set a time stamp), it seems odd at that point 
that I have to determine a timestamp, one that does not seem relevant to 
CompileTask.? What I believe CompileTask wants, and what it seems other 
calls in the compiler directory use is the time since the vm was 
started.? See the support for PrintCompilation in 
compiler/compileBroker.cpp for the use of tty->timestamp() vs. 
CompileBroker's _time* timestamps.? Thus I think my change to use tty in 
CompileTask::print_impl() is correct.

>
> Second the changes to wrap_dynamic_exception seem rather awkward, both 
> in terms of having to remember what the initial boolean arg represents 
> at the call site, and more so in the logic to determine which log 
> stream to use. I was wondering if it might be better to add 
> wrap_dynamic_exception_for_indy to keep the call sites clean? 
> Otherwise perhaps document like "true /* indy */" and "false /* not 
> indy */.

I would rather not break wrap_dynamic_exception into 2 methods since 
this is a JVMS behavior to wrap certain exceptions into a 
BootstrapMethodError for both indy and condy.? So I have instead opted 
for your suggestion of using comments when calling wrap_dynamic_exception.

New webrev at: 
http://cr.openjdk.java.net/~lfoltan/bug_jdk8210012.1/webrev/index.html

Thanks,
Lois

>
> For the log stream logic I was assuming there must be a better/simpler 
> way, but after looking at it in some detail it seems not. :(
>
> Thanks,
> David
>
>> Testing: hs-tier1-5
>>
>> Thanks,
>> Lois


From lois.foltan at oracle.com  Fri Apr 10 22:05:13 2020
From: lois.foltan at oracle.com (Lois Foltan)
Date: Fri, 10 Apr 2020 18:05:13 -0400
Subject: RFR (M) JDK-8210012: Implement Unified Logging Option for
 -XX:+TraceMethodHandles and -XX:+TraceInvokeDynamic
In-Reply-To: <3e7cc93f-8c9b-85a4-ec68-39720bf876b4@oracle.com>
References: <9f678d60-93c1-f2e3-84a1-51feb74c6961@oracle.com>
 <3e7cc93f-8c9b-85a4-ec68-39720bf876b4@oracle.com>
Message-ID: <ad36856f-5e45-f636-9c11-0bd9292da05e@oracle.com>


> On 4/9/20 8:12 PM, David Holmes wrote:
>> Hi Lois,
>>
>> On 10/04/2020 4:42 am, Lois Foltan wrote:
>>> Please review the following change to implement unified logging 
>>> options for -XX:+TraceMethodHandles and -XX:+TraceInvokeDynamic. The 
>>> new options map as follows:
>>>
>>> -XX:+TraceMethodHandles --> -Xlog:methodhandles=info
>>> -XX:+TraceInvokeDynamic --> -Xlog:methodhandles+indy=debug
>>>
>>> and in addition dynamic constants can now be viewed under their own 
>>> option via -Xlog:methodhandles+condy=debug
>>>
>>> open webrev at: 
>>> http://cr.openjdk.java.net/~lfoltan/bug_jdk8210012.0/webrev/
>>> bug link: https://bugs.openjdk.java.net/browse/JDK-8210012
>>
>> Generally looks good. A couple of things.
>>
>> First the change in compileTask.cpp seems wrong: you should get the 
>> timestamp from the specified stream not the tty.
>>
>> Second the changes to wrap_dynamic_exception seem rather awkward, 
>> both in terms of having to remember what the initial boolean arg 
>> represents at the call site, and more so in the logic to determine 
>> which log stream to use. I was wondering if it might be better to add 
>> wrap_dynamic_exception_for_indy to keep the call sites clean? 
>> Otherwise perhaps document like "true /* indy */" and "false /* not 
>> indy */.
>>
>> For the log stream logic I was assuming there must be a 
>> better/simpler way, but after looking at it in some detail it seems 
>> not. :(
>>
>
> Hi Lois,

Hi Ioi,

Thanks for the review!

>
>
> LinkResolver::lookup_polymorphic_method()
> +? ResourceMark rm(THREAD);
>
> Is this ResourceMark used only for logging?

Yes.? As a matter of fact specifying -XX:+TraceMethodHandles or 
-XX:+TraceInvokeDynamic currently on my sample test case results today 
in crashes concerning the lack of allocating without a ResourceMark.

>
> =================
> For code like this in ConstantPool::resolve_constant_at_impl:
>
> ??? if (log_is_enabled(Debug, methodhandles, condy)) {
> ????? LogTarget(Debug, methodhandles, condy) lt;
> ????? LogStream ls(lt);
> ????? bootstrap_specifier.print_msg_on(&ls, "resolve_constant_at_impl");
> ??? }
>
> The second set of tags can be avoided by declaring LogTarget(Debug, 
> methodhandles, condy) first. There's an example of this style in 
> FileMapInfo::log_paths().
>
> ??? LogTarget(Debug, methodhandles, condy) lt;
> ??? if (lt.is_enabled()) {
> ????? LogStream ls(lt);
> ????? bootstrap_specifier.print_msg_on(&ls, "resolve_constant_at_impl");
> ??? }

Fixed, I have changed all checks in this webrev to consistently use this 
form of checking if LogTarget is enabled.

>
> =================
>
> unpack_method_and_appendix() {
>
> ????? if (log_develop_is_enabled(Info, methodhandles)) {
> ??????? ResourceMark rm(THREAD);
> ??????? LogTarget(Info, methodhandles) lt;
> ??????? LogStream ls(lt);
> ??????? ls->print_on(.....);
>
> This can be fixed similarly, if we add a new function to LogTargetImpl:
>
> ?? class LogTargetImpl {
> ? ?? static bool develop_is_enabled() {
> ?? ? ? NOT_PRODUCT(return LogImpl<T0, T1, T2, T3, T4, 
> GuardTag>::is_level(level));
> ?????? PRODUCT_ONLY(return false);
> ? ?? }
> ???? ...
> ?? }
>
> ? ? ? LogTarget(Info, methodhandles) lt;
> ????? if (lt.develop_is_enabled()) {
> ??????? ResourceMark rm(THREAD);
> ??????? LogStream ls(lt);
> ??????? ls->print_on(.....);

Fixed. Added the method "develop_is_enabled()" to class LogTargetImpl.? 
However my method looks more like this:

static bool develop_is_enabled() {
 ? NOT_PRODUCT(return is_enabled();)
 ? PRODUCT_ONLY(return false);
}

>
> =================
> void ConstantPoolCacheEntry::set_method_handle_common()
>
> ? LogStreamHandle(Debug, methodhandles, indy) lsh;
> ? if (log_is_enabled(Debug, methodhandles, indy)) {
>
> Maybe we should add a new function LogStream::is_enabled so we can:
>
> ? LogStreamHandle(Debug, methodhandles, indy) lsh;
> ? if (lsh.is_enabled()) {

Fixed. Turns out that a LogTargetHandle already has an "is_enabled()" 
method.? So I added an "is_enabled()" method to class LogStream to check 
if its field, _log_handle, whose type is LogTargetHandle, is enabled.

>
>
> =================
> Not a comment about this patch, but in general, I am not happy with 
> the verbosity when a ResourceMark is needed in logging.
>
> I think the best way to avoid the verbosity would be to use C++ lambda 
> expressions (are we allowed??), or (oh gosh!) maybe a macro? (Probably 
> in a separate RFE).
>
> ??? LOG_WITH_RESOURCE_MARK((Info, methodhandles), THREAD,ls, {
> ????? ? ls->print_on(.....);
> ?? ?? });

Good idea, but will create an RFE for this.

New webrev at: 
http://cr.openjdk.java.net/~lfoltan/bug_jdk8210012.1/webrev/index.html

Thanks,
Lois

>
>
> Thanks
> - Ioi
>
>
>> Thanks,
>> David
>>
>>> Testing: hs-tier1-5
>>>
>>> Thanks,
>>> Lois
>


From lois.foltan at oracle.com  Fri Apr 10 22:08:54 2020
From: lois.foltan at oracle.com (Lois Foltan)
Date: Fri, 10 Apr 2020 18:08:54 -0400
Subject: RFR (M) JDK-8210012: Implement Unified Logging Option for
 -XX:+TraceMethodHandles and -XX:+TraceInvokeDynamic
In-Reply-To: <ACCBE810-8524-4B40-9871-C37A2E66A0A1@oracle.com>
References: <07752bc9-3c6f-6b49-6403-bce3080e54cf@oracle.com>
 <ACCBE810-8524-4B40-9871-C37A2E66A0A1@oracle.com>
Message-ID: <6d3b062f-24b3-07d4-9e96-deb906b194bd@oracle.com>

On 4/9/2020 5:03 PM, John Rose wrote:
> Reviewed.  Nice things I saw:
>
>   - fewer uses of ttyLocker
>   - indy-related diagnostics getting into UL
>   - UL being strong enough to handle complex multi-line formatted output

Thanks John for the review!? New webrev at 
http://cr.openjdk.java.net/~lfoltan/bug_jdk8210012.1/webrev/index.html 
that addresses both David and Ioi's comments in case you would like to 
take a look too.
Lois

>
> Side comment about UL:  Last time I looked, which was a
> while ago, UL was not friendly to multi-line outputs.
>
> Eventually, UL should be applied to -XX:+LogCompilation, but
> that will require UL to manage large structured segments of
> logging data.  -XX:+LC produces a large XML-flavored report
> for each distinct compilation task.
>
>> On Apr 9, 2020, at 11:42 AM, Lois Foltan <lois.foltan at oracle.com> wrote:
>>
>> Please review the following change to implement unified logging options for -XX:+TraceMethodHandles and -XX:+TraceInvokeDynamic. The new options map as follows:
>>
>> -XX:+TraceMethodHandles --> -Xlog:methodhandles=info
>> -XX:+TraceInvokeDynamic -->  -Xlog:methodhandles+indy=debug
>>
>> and in addition dynamic constants can now be viewed under their own option via -Xlog:methodhandles+condy=debug
>>
>> open webrev at: http://cr.openjdk.java.net/~lfoltan/bug_jdk8210012.0/webrev/
>> bug link: https://bugs.openjdk.java.net/browse/JDK-8210012
>>
>> Testing: hs-tier1-5
>>
>> Thanks,
>> Lois


From david.holmes at oracle.com  Fri Apr 10 22:53:39 2020
From: david.holmes at oracle.com (David Holmes)
Date: Sat, 11 Apr 2020 08:53:39 +1000
Subject: RFR(XXS): 8242485: Null _file checking in fileStream::flush()
In-Reply-To: <e8d66934-57b9-44e5-a1b2-c650ea101220.denghui.ddh@alibaba-inc.com>
References: <e8d66934-57b9-44e5-a1b2-c650ea101220.denghui.ddh@alibaba-inc.com>
Message-ID: <654c6f00-b39b-6115-fa19-8e5b67255744@oracle.com>

Hi,

On 10/04/2020 10:34 pm, Denghui Dong wrote:
> Hi team,
> 
> Could you please review this small patch?
> 
> JBS: https://bugs.openjdk.java.net/browse/JDK-8242485
> webrev: http://cr.openjdk.java.net/~ddong/8242485/webrev.00/
> 
> I found the method "fileStream::flush()" lacks null check before fflush  which will cause
> all open output streams are flushed by fflush() if _file is NULL.

Good catch. Our fileStream abstraction seems somewhat broken when it 
comes to errors! I think fileStream::readln also needs a NULL check for 
clarity if nothing else.

Thanks,
David

> Thanks,
> Denghui Dong
> 

From david.holmes at oracle.com  Sat Apr 11 02:57:42 2020
From: david.holmes at oracle.com (David Holmes)
Date: Sat, 11 Apr 2020 12:57:42 +1000
Subject: RFR (M) JDK-8210012: Implement Unified Logging Option for
 -XX:+TraceMethodHandles and -XX:+TraceInvokeDynamic
In-Reply-To: <0319c697-10b0-9aaf-7e9f-2762d2cd1c16@oracle.com>
References: <07752bc9-3c6f-6b49-6403-bce3080e54cf@oracle.com>
 <41e05f04-e6f3-cba4-d1d3-6bc1b156071d@oracle.com>
 <0319c697-10b0-9aaf-7e9f-2762d2cd1c16@oracle.com>
Message-ID: <28cd4d46-ab4d-c5fe-7993-0a0da61fade2@oracle.com>

Hi Lois,

On 11/04/2020 7:55 am, Lois Foltan wrote:
> On 4/9/2020 11:12 PM, David Holmes wrote:
>> Hi Lois,
>>
>> On 10/04/2020 4:42 am, Lois Foltan wrote:
>>> Please review the following change to implement unified logging 
>>> options for -XX:+TraceMethodHandles and -XX:+TraceInvokeDynamic. The 
>>> new options map as follows:
>>>
>>> -XX:+TraceMethodHandles --> -Xlog:methodhandles=info
>>> -XX:+TraceInvokeDynamic -->? -Xlog:methodhandles+indy=debug
>>>
>>> and in addition dynamic constants can now be viewed under their own 
>>> option via -Xlog:methodhandles+condy=debug
>>>
>>> open webrev at: 
>>> http://cr.openjdk.java.net/~lfoltan/bug_jdk8210012.0/webrev/
>>> bug link: https://bugs.openjdk.java.net/browse/JDK-8210012
>>
>> Generally looks good. A couple of things.
> 
> Thanks David for the review!
> 
>>
>> First the change in compileTask.cpp seems wrong: you should get the 
>> timestamp from the specified stream not the tty.
> 
> At first it looked wrong to me as well, but I was receiving an assert 
> about the TimeStamp associated with the outputStream being cleared, (the 
> first assert in TimeStamp::milliseconds).? After investigation it turns 
> out that CompileTask::print_impl() has never been called with any 
> outputStream other than tty.? And 3-or 4 levels up when I call it with a 
> LogStream (which inherits from outputStream and constructs with a 
> default ctor that does not set a time stamp), it seems odd at that point 
> that I have to determine a timestamp, one that does not seem relevant to 
> CompileTask.? What I believe CompileTask wants, and what it seems other 
> calls in the compiler directory use is the time since the vm was 
> started.? See the support for PrintCompilation in 
> compiler/compileBroker.cpp for the use of tty->timestamp() vs. 
> CompileBroker's _time* timestamps.? Thus I think my change to use tty in 
> CompileTask::print_impl() is correct.

So we have an outputStream API for timestamps, but in practice it is 
enabled on a per-instance basis (only the tty?) and we have whole 
subclasses (stringStream, LogStream) that don't support it. Yet we have 
code that claims to generically operate on any outputstream but calls 
timestamp() which may assert if not called on a supporting instance! 
What a mess. :(

We also have two notions of "uptime" in the VM:
- one is the timestamp held within the tty
- the other is os::elapsedTime()  (which is used by UL)

Probably some scope for some cleanup here ... like get rid of timestamp 
from the tty/outputStream as it is barely used and used incorrectly in a 
couple of places. Further, it's main use was consistency across all the 
logging output to the tty, but now very little is logged to the tty so 
we have inconsistencies across different log output anyway.

But not for this RFE.

>>
>> Second the changes to wrap_dynamic_exception seem rather awkward, both 
>> in terms of having to remember what the initial boolean arg represents 
>> at the call site, and more so in the logic to determine which log 
>> stream to use. I was wondering if it might be better to add 
>> wrap_dynamic_exception_for_indy to keep the call sites clean? 
>> Otherwise perhaps document like "true /* indy */" and "false /* not 
>> indy */.
> 
> I would rather not break wrap_dynamic_exception into 2 methods since 
> this is a JVMS behavior to wrap certain exceptions into a 
> BootstrapMethodError for both indy and condy.? So I have instead opted 
> for your suggestion of using comments when calling wrap_dynamic_exception.
> 
> New webrev at: 
> http://cr.openjdk.java.net/~lfoltan/bug_jdk8210012.1/webrev/index.html

Updates look fine.

Thanks,
David

> Thanks,
> Lois
> 
>>
>> For the log stream logic I was assuming there must be a better/simpler 
>> way, but after looking at it in some detail it seems not. :(
>>
>> Thanks,
>> David
>>
>>> Testing: hs-tier1-5
>>>
>>> Thanks,
>>> Lois
> 

From denghui.ddh at alibaba-inc.com  Sat Apr 11 15:52:55 2020
From: denghui.ddh at alibaba-inc.com (Denghui Dong)
Date: Sat, 11 Apr 2020 23:52:55 +0800
Subject: =?UTF-8?B?UmU6IFJGUihYWFMpOiA4MjQyNDg1OiBOdWxsIF9maWxlIGNoZWNraW5nIGluIGZpbGVTdHJl?=
 =?UTF-8?B?YW06OmZsdXNoKCk=?=
In-Reply-To: <a8bd28fd-619a-d974-621d-45873edc7fa0@oss.nttdata.com>
References: <e8d66934-57b9-44e5-a1b2-c650ea101220.denghui.ddh@alibaba-inc.com>,
 <a8bd28fd-619a-d974-621d-45873edc7fa0@oss.nttdata.com>
Message-ID: <a5758e50-0d65-4c90-8512-c41415b7aed7.denghui.ddh@alibaba-inc.com>

Hi,
Thank you for the review and point out other methods need to check _file.
In addition to "fileStream::flush", I found there are still four methods need to check _file in fileStream: read, readln, eof, rewind.
but those methods will crash if _file is NULL, and I didn't find any caller of them(Maybe I am wrong),
so I think it's more appropriate to add assert check to those methods.

Webrev: http://cr.openjdk.java.net/~ddong/8242485/webrev.01/

Could you review it again, and sponsor it if everything is okay?

Testing: teir1
==============================
Test summary
==============================
   TEST                                              TOTAL  PASS  FAIL ERROR
   jtreg:test/hotspot/jtreg:tier1                     1516  1516     0     0
   jtreg:test/jdk:tier1                               1904  1904     0     0
>> jtreg:test/langtools:tier1                         4031  4029     2     0 <<
   jtreg:test/nashorn:tier1                              0     0     0     0
   jtreg:test/jaxp:tier1                                 0     0     0     0
==============================
There are two unrelated test failures in langtools: jdk/javadoc/tool/CheckResourceKeys.java  and tools/javac/processing/model/TestSymtabItems.java 



------------------------------------------------------------------
From:Yasumasa Suenaga <suenaga at oss.nttdata.com>
Send Time:2020?4?11?(???) 09:22
To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net <hotspot-runtime-dev at openjdk.java.net>
Subject:Re: RFR(XXS): 8242485: Null _file checking in fileStream::flush()

Hi Denghui,

null check lacks in other place too.
Can you fix it?

   http://hg.openjdk.java.net/jdk/jdk/file/97d5d0cd1085/src/hotspot/share/utilities/ostream.cpp#l554


Thanks,

Yasumasa


On 2020/04/10 21:34, Denghui Dong wrote:
> Hi team,
> 
> Could you please review this small patch?
> 
> JBS: https://bugs.openjdk.java.net/browse/JDK-8242485
> webrev: http://cr.openjdk.java.net/~ddong/8242485/webrev.00/
> 
> I found the method "fileStream::flush()" lacks null check before fflush  which will cause
> all open output streams are flushed by fflush() if _file is NULL.
> 
> Thanks,
> Denghui Dong
> 

From david.holmes at oracle.com  Sun Apr 12 02:38:29 2020
From: david.holmes at oracle.com (David Holmes)
Date: Sun, 12 Apr 2020 12:38:29 +1000
Subject: RFR(XXS): 8242485: Null _file checking in fileStream::flush()
In-Reply-To: <a5758e50-0d65-4c90-8512-c41415b7aed7.denghui.ddh@alibaba-inc.com>
References: <e8d66934-57b9-44e5-a1b2-c650ea101220.denghui.ddh@alibaba-inc.com>
 <a8bd28fd-619a-d974-621d-45873edc7fa0@oss.nttdata.com>
 <a5758e50-0d65-4c90-8512-c41415b7aed7.denghui.ddh@alibaba-inc.com>
Message-ID: <d7f0d3b2-1fc7-3601-b651-a4bdb0244d23@oracle.com>

Hi,

On 12/04/2020 1:52 am, Denghui Dong wrote:
> Hi,
> Thank?you?for?the?review?and?point?out?other?methods?need?to?check?_file.
> In?addition?to?"fileStream::flush",?I?found?there?are?still?four?methods?need?to?check?_file?in?fileStream:?read,?readln,?eof,?rewind.
> but?those?methods?will?crash?if?_file?is?NULL,?and?I?didn't?find?any?caller?of?them(Maybe?I?am?wrong),
> so?I?think?it's?more?appropriate?to?add?assert?check?to?those?methods.

Assertions are really only appropriate if they will be exercised by 
tests - the intent being that tests will expose any cases where the 
assertion fails.

Also it doesn't make sense to me to have assertions in most places but a 
true NULL check in others - that suggests we expect some methods to be 
called when _file is NULL, but others not. I'd prefer just to see the 
missing NULL checks added, rather than using assertions.

Thanks,
David

> Webrev: http://cr.openjdk.java.net/~ddong/8242485/webrev.01/
> 
> Could you review it again, and sponsor it if everything is okay?
> 
> Testing:?teir1
> ==============================
> Test?summary
> ==============================
>  ???TEST??????????????????????????????????????????????TOTAL??PASS??FAIL?ERROR
>  ???jtreg:test/hotspot/jtreg:tier1?????????????????????1516??1516?????0?????0
>  ???jtreg:test/jdk:tier1???????????????????????????????1904??1904?????0?????0
>  >>?jtreg:test/langtools:tier1?????????????????????????4031??4029?????2?????0?<<
>  ???jtreg:test/nashorn:tier1??????????????????????????????0?????0?????0?????0
>  ???jtreg:test/jaxp:tier1?????????????????????????????????0?????0?????0?????0
> ==============================
> There?are?two?unrelated test failures in 
> langtools:?jdk/javadoc/tool/CheckResourceKeys.java??and?tools/javac/processing/model/TestSymtabItems.java 
> 
> 
> 
>     ------------------------------------------------------------------
>     From:Yasumasa Suenaga <suenaga at oss.nttdata.com>
>     Send Time:2020?4?11?(???) 09:22
>     To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev
>     <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>     <hotspot-runtime-dev at openjdk.java.net>
>     Subject:Re: RFR(XXS): 8242485: Null _file checking in
>     fileStream::flush()
> 
>     Hi?Denghui,
> 
>     null?check?lacks?in?other?place?too.
>     Can?you?fix?it?
> 
>      ???http://hg.openjdk.java.net/jdk/jdk/file/97d5d0cd1085/src/hotspot/share/utilities/ostream.cpp#l554
> 
> 
>     Thanks,
> 
>     Yasumasa
> 
> 
>     On?2020/04/10?21:34,?Denghui?Dong?wrote:
>      >?Hi?team,
>      >
>      >?Could?you?please?review?this?small?patch?
>      >
>      >?JBS:?https://bugs.openjdk.java.net/browse/JDK-8242485
>      >?webrev:?http://cr.openjdk.java.net/~ddong/8242485/webrev.00/
>      >
>      >?I?found?the?method?"fileStream::flush()"?lacks?null?check?before?fflush??which?will?cause
>      >?all?open?output?streams?are?flushed?by?fflush()?if?_file?is?NULL.
>      >
>      >?Thanks,
>      >?Denghui?Dong
>      > 
> 

From mandy.chung at oracle.com  Sun Apr 12 03:13:07 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Sat, 11 Apr 2020 20:13:07 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <1a5eb022-dd7e-b0bb-750f-c43be195eb42@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <1a5eb022-dd7e-b0bb-750f-c43be195eb42@oracle.com>
Message-ID: <da9f80ff-6d63-aebe-a9fa-1a683eb42bbc@oracle.com>

Please review the delta patch:
http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.06-delta/

Incremental specdiff:
http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff-inc/overview-summary.html
This is to follow a discussion [1] on Class::descriptorString and 
MethodType::descriptorString for hidden classes.?? The proposal is to 
define the descriptor string for a hidden class of this form:
 ??? "L" + N + ";" + "/" + <suffix>

The spec of `Lookup::defineHiddenClass`, `Class::descriptorString` and 
`MethodType::descriptorString` are updated to return the descriptor of 
this form for a hidden class.?? To support hidden class, 
`java.lang.invoke.TypeDescriptor` spec is revised such that a 
`TypeDescriptor` object can represent an entity that may not be 
described in nominal form. ??? This change affects JVM TI, JDWP and 
JDI.??? The spec change includes a couple other JVM TI and 
java.instrument spec clarification w.r.t. hidden classes that Serguei 
has been working on.



Mandy
[1] 
https://mail.openjdk.java.net/pipermail/valhalla-dev/2020-April/007093.html

----------------
As a record, the above patch is applied on the top:
http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.06/

webrev.06 includes the following changes that have been reviewed:
1. rename "weak hidden" and Klass::is_hidden_weak to is_non_strong_hidden
2. remove unused `setImplMethod` method from lambda proxy class
3. include Serguei's patch to fix JDK-8242166: regression in JDI 
ClassUnload events
4. drop the uniqueness guarantee of the suffix of the hidden class's name

On 3/31/20 8:01 PM, Mandy Chung wrote:
> Thanks to the review feedbacks.
>
> Updated webrev:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.04/ 
>
> Delta between webrev.03 and webrev.04:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03-delta/ 
>
>
> Summary of changes is:
> 1. Update javac to retain the previous behavior when compiling to 
> target release <= 14 where lambda proxy class is not a nestmate
> 2. Rename Class::isHiddenClass to Class::isHidden
> 3. Address Joe's feedback on the CSR to document the behavior of the 
> relevant methods in java.lang.Class for hidden classes
> 4. Add test case for unloadable class with nest host error
> 5. Add test cases for hidden classes with different kinds of class or 
> interface
> 6. Update dcmd to drop "weak hidden class" and refer it as "hidden class"
> 7. Small changes in response to Remi, Coleen, Paul's review comments
> 8. Fix copyright headers
> 9. Fix @modules in tests
>
> Most of the changes above have also been reviewed as separate patches.
>
> Thanks
> Mandy
>
> On 3/26/20 4:57 PM, Mandy Chung wrote:
>> Please review the implementation of JEP 371: Hidden Classes. The main 
>> changes are in core-libs and hotspot runtime area.? Small changes are 
>> made in javac, VM compiler (intrinsification of 
>> Class::isHiddenClass), JFR, JDI, and jcmd.? CSR [1]has been reviewed 
>> and is in the finalized state (see specdiff and javadoc below for 
>> reference).
>>
>> Webrev:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
>>
>>
>> javadoc/specdiff
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
>>
>>
>> JVMS 5.4.4 change:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
>>
>>
>> CSR:
>> https://bugs.openjdk.java.net/browse/JDK-8238359
>>
>> Thanks
>> Mandy
>> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
>> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
>> [3] https://bugs.openjdk.java.net/browse/JDK-8230502
>


From denghui.ddh at alibaba-inc.com  Sun Apr 12 04:41:31 2020
From: denghui.ddh at alibaba-inc.com (Denghui Dong)
Date: Sun, 12 Apr 2020 12:41:31 +0800
Subject: =?UTF-8?B?UmU6IFJGUihYWFMpOiA4MjQyNDg1OiBOdWxsIF9maWxlIGNoZWNraW5nIGluIGZpbGVTdHJl?=
 =?UTF-8?B?YW06OmZsdXNoKCk=?=
In-Reply-To: <d7f0d3b2-1fc7-3601-b651-a4bdb0244d23@oracle.com>
References: <e8d66934-57b9-44e5-a1b2-c650ea101220.denghui.ddh@alibaba-inc.com>
 <a8bd28fd-619a-d974-621d-45873edc7fa0@oss.nttdata.com>
 <a5758e50-0d65-4c90-8512-c41415b7aed7.denghui.ddh@alibaba-inc.com>,
 <d7f0d3b2-1fc7-3601-b651-a4bdb0244d23@oracle.com>
Message-ID: <d483f6ff-c51e-4eed-b11d-d7e8e427d8e4.denghui.ddh@alibaba-inc.com>

Hi David,

Thanks for the comments
new webrev: http://cr.openjdk.java.net/~ddong/8242485/webrev.02

when _file is NULL:
- fileStream::read return 0
- fileStream::readln return NULL
- fileStream::eof return -1 (like fileStream::fileSize, not sure if appropriate, feof returns nonzero if target file is end)

Testing: tier1

Cheers,
Denghui Dong

------------------------------------------------------------------
From:David Holmes <david.holmes at oracle.com>
Send Time:2020?4?12?(???) 10:40
To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net <hotspot-runtime-dev at openjdk.java.net>; Yasumasa Suenaga <suenaga at oss.nttdata.com>; "Liu, Xin" <xxinliu at amazon.com>
Subject:Re: RFR(XXS): 8242485: Null _file checking in fileStream::flush()

Hi,

On 12/04/2020 1:52 am, Denghui Dong wrote:
> Hi,
> Thank you for the review and point out other methods need to check _file.
> In addition to "fileStream::flush", I found there are still four methods need to check _file in fileStream: read, readln, eof, rewind.
> but those methods will crash if _file is NULL, and I didn't find any caller of them(Maybe I am wrong),
> so I think it's more appropriate to add assert check to those methods.

Assertions are really only appropriate if they will be exercised by 
tests - the intent being that tests will expose any cases where the 
assertion fails.

Also it doesn't make sense to me to have assertions in most places but a 
true NULL check in others - that suggests we expect some methods to be 
called when _file is NULL, but others not. I'd prefer just to see the 
missing NULL checks added, rather than using assertions.

Thanks,
David

> Webrev: http://cr.openjdk.java.net/~ddong/8242485/webrev.01/
> 
> Could you review it again, and sponsor it if everything is okay?
> 
> Testing: teir1
> ==============================
> Test summary
> ==============================
>     TEST                                              TOTAL  PASS  FAIL ERROR
>     jtreg:test/hotspot/jtreg:tier1                     1516  1516     0     0
>     jtreg:test/jdk:tier1                               1904  1904     0     0
>  >> jtreg:test/langtools:tier1                         4031  4029     2     0 <<
>     jtreg:test/nashorn:tier1                              0     0     0     0
>     jtreg:test/jaxp:tier1                                 0     0     0     0
> ==============================
> There are two unrelated test failures in 
> langtools: jdk/javadoc/tool/CheckResourceKeys.java  and tools/javac/processing/model/TestSymtabItems.java 
> 
> 
> 
>     ------------------------------------------------------------------
>     From:Yasumasa Suenaga <suenaga at oss.nttdata.com>
>     Send Time:2020?4?11?(???) 09:22
>     To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev
>     <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>     <hotspot-runtime-dev at openjdk.java.net>
>     Subject:Re: RFR(XXS): 8242485: Null _file checking in
>     fileStream::flush()
> 
>     Hi Denghui,
> 
>     null check lacks in other place too.
>     Can you fix it?
> 
>         http://hg.openjdk.java.net/jdk/jdk/file/97d5d0cd1085/src/hotspot/share/utilities/ostream.cpp#l554
> 
> 
>     Thanks,
> 
>     Yasumasa
> 
> 
>     On 2020/04/10 21:34, Denghui Dong wrote:
>      > Hi team,
>      >
>      > Could you please review this small patch?
>      >
>      > JBS: https://bugs.openjdk.java.net/browse/JDK-8242485
>      > webrev: http://cr.openjdk.java.net/~ddong/8242485/webrev.00/
>      >
>      > I found the method "fileStream::flush()" lacks null check before fflush  which will cause
>      > all open output streams are flushed by fflush() if _file is NULL.
>      >
>      > Thanks,
>      > Denghui Dong
>      > 
> 

From david.holmes at oracle.com  Sun Apr 12 07:04:15 2020
From: david.holmes at oracle.com (David Holmes)
Date: Sun, 12 Apr 2020 17:04:15 +1000
Subject: RFR(XXS): 8242485: Null _file checking in fileStream::flush()
In-Reply-To: <d483f6ff-c51e-4eed-b11d-d7e8e427d8e4.denghui.ddh@alibaba-inc.com>
References: <e8d66934-57b9-44e5-a1b2-c650ea101220.denghui.ddh@alibaba-inc.com>
 <a8bd28fd-619a-d974-621d-45873edc7fa0@oss.nttdata.com>
 <a5758e50-0d65-4c90-8512-c41415b7aed7.denghui.ddh@alibaba-inc.com>
 <d7f0d3b2-1fc7-3601-b651-a4bdb0244d23@oracle.com>
 <d483f6ff-c51e-4eed-b11d-d7e8e427d8e4.denghui.ddh@alibaba-inc.com>
Message-ID: <618b239a-2a21-a56f-021a-74df5e8d433c@oracle.com>

Hi Denghui,

On 12/04/2020 2:41 pm, Denghui Dong wrote:
> Hi?David,
> 
> Thanks?for?the?comments
> new?webrev: http://cr.openjdk.java.net/~ddong/8242485/webrev.02
> 
> when?_file?is?NULL:
> -?fileStream::read?return?0
> -?fileStream::readln?return?NULL
> -?fileStream::eof?return?-1?(like 
> fileStream::fileSize,?not?sure?if?appropriate,?feof?returns?nonzero?if?target?file?is?end)

That is all great - thanks!

Unfortunately the more I look at this code the more problems I see with 
it. :( We have:

void fileStream::write(const char* s, size_t len) {
   if (_file != NULL)  {
     // Make an unused local variable to avoid warning from gcc compiler.
     size_t count = fwrite(s, 1, len, _file);
   }
   update_position(s, len);
}

but the update_position should be inside the if-statement - no need to 
see a new webrev for that.

We should also be watching for errors from fwrite - but that in itself 
just opens up a can-of-worms with regard to error handling in general 
with this class, so that can be left as-is.

Thanks,
David
-----

> Testing:?tier1
> 
> Cheers,
> Denghui?Dong
> 
>     ------------------------------------------------------------------
>     From:David Holmes <david.holmes at oracle.com>
>     Send Time:2020?4?12?(???) 10:40
>     To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev
>     <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>     <hotspot-runtime-dev at openjdk.java.net>; Yasumasa Suenaga
>     <suenaga at oss.nttdata.com>; "Liu, Xin" <xxinliu at amazon.com>
>     Subject:Re: RFR(XXS): 8242485: Null _file checking in
>     fileStream::flush()
> 
>     Hi,
> 
>     On?12/04/2020?1:52?am,?Denghui?Dong?wrote:
>      >?Hi,
>      >?Thank?you?for?the?review?and?point?out?other?methods?need?to?check?_file.
>      >?In?addition?to?"fileStream::flush",?I?found?there?are?still?four?methods?need?to?check?_file?in?fileStream:?read,?readln,?eof,?rewind.
>      >?but?those?methods?will?crash?if?_file?is?NULL,?and?I?didn't?find?any?caller?of?them(Maybe?I?am?wrong),
>      >?so?I?think?it's?more?appropriate?to?add?assert?check?to?those?methods.
> 
>     Assertions?are?really?only?appropriate?if?they?will?be?exercised?by
>     tests?-?the?intent?being?that?tests?will?expose?any?cases?where?the
>     assertion?fails.
> 
>     Also?it?doesn't?make?sense?to?me?to?have?assertions?in?most?places?but?a
> 
>     true?NULL?check?in?others?-?that?suggests?we?expect?some?methods?to?be
>     called?when?_file?is?NULL,?but?others?not.?I'd?prefer?just?to?see?the
>     missing?NULL?checks?added,?rather?than?using?assertions.
> 
>     Thanks,
>     David
> 
>      >?Webrev:?http://cr.openjdk.java.net/~ddong/8242485/webrev.01/
>      >
>      >?Could?you?review?it?again,?and?sponsor?it?if?everything?is?okay?
>      >
>      >?Testing:?teir1
>      >?==============================
>      >?Test?summary
>      >?==============================
>      >?????TEST??????????????????????????????????????????????TOTAL??PASS??FAIL?ERROR
>      >?????jtreg:test/hotspot/jtreg:tier1?????????????????????1516??1516?????0?????0
>      >?????jtreg:test/jdk:tier1???????????????????????????????1904??1904?????0?????0
>      >??>>?jtreg:test/langtools:tier1?????????????????????????4031??4029?????2?????0?<<
>      >?????jtreg:test/nashorn:tier1??????????????????????????????0?????0?????0?????0
>      >?????jtreg:test/jaxp:tier1?????????????????????????????????0?????0?????0?????0
>      >?==============================
>      >?There?are?two?unrelated?test?failures?in
>      >?langtools:?jdk/javadoc/tool/CheckResourceKeys.java??and?tools/javac/processing/model/TestSymtabItems.java
>      >
>      >
>      >
>      >?????------------------------------------------------------------------
>      >?????From:Yasumasa?Suenaga?<suenaga at oss.nttdata.com>
>      >?????Send?Time:2020?4?11?(???)?09:22
>      >?????To:???(??)?<denghui.ddh at alibaba-inc.com>;?hotspot-dev
>      >?????<hotspot-dev at openjdk.java.net>;?hotspot-runtime-dev at openjdk.java.net
>      >?????<hotspot-runtime-dev at openjdk.java.net>
>      >?????Subject:Re:?RFR(XXS):?8242485:?Null?_file?checking?in
>      >?????fileStream::flush()
>      >
>      >?????Hi?Denghui,
>      >
>      >?????null?check?lacks?in?other?place?too.
>      >?????Can?you?fix?it?
>      >
>      >?????????http://hg.openjdk.java.net/jdk/jdk/file/97d5d0cd1085/src/hotspot/share/utilities/ostream.cpp#l554
>      >
>      >
>      >?????Thanks,
>      >
>      >?????Yasumasa
>      >
>      >
>      >?????On?2020/04/10?21:34,?Denghui?Dong?wrote:
>      >??????>?Hi?team,
>      >??????>
>      >??????>?Could?you?please?review?this?small?patch?
>      >??????>
>      >??????>?JBS:?https://bugs.openjdk.java.net/browse/JDK-8242485
>      >??????>?webrev:?http://cr.openjdk.java.net/~ddong/8242485/webrev.00/
>      >??????>
>      >??????>?I?found?the?method?"fileStream::flush()"?lacks?null?check?before?fflush??which?will?cause
>      >??????>?all?open?output?streams?are?flushed?by?fflush()?if?_file?is?NULL.
>      >??????>
>      >??????>?Thanks,
>      >??????>?Denghui?Dong
>      >??????>
>      > 
> 

From xxinliu at amazon.com  Sun Apr 12 07:59:36 2020
From: xxinliu at amazon.com (Liu, Xin)
Date: Sun, 12 Apr 2020 07:59:36 +0000
Subject: RFC 8236988 - Modular Design for JVM Flags
Message-ID: <6EE91323-A69B-44E1-8215-411250EB16DD@amazon.com>

Hi, Ioi, 

I'm recently refactoring intrinsics options for compiler[1]. I introduced a new option type called "intrinsics".  The biggest trouble was that I had to modify multiple places.  I found your patch beautifully replaces jvmConstraintList.* &jvmFlagRangeList.* with some macros in jvmFlag.inline.hpp. great!

I have some questions or suggests.

1. generate a document of options
There're hundreds of JVM options.  It's hard to learn them by heart. I feel developers uses globals.hpp and sub header files as a de factor manual. 
if you break up the monolithic header, which is good,  I think developers deserve a manual.  
Off the top of my head, I think you can provide a mini tool, which consume a header aggerates all sub-header files and automatically generates a manual.

2. change FLAG_RANGE and FLAG_CONTRAINT to optional parameters of PRODUCT_FLAG.
If you use variadic macro, it's possible to select RANGE or CONTRAINT or NIL from __VA_ARGS__.
I used to propose it before. https://cr.openjdk.java.net/~xliu/8230669/00/webrev/src/hotspot/share/utilities/macro_overloading.hpp.html

I think it's a good idea because it make the declaration section pure. 
If we replace those macros eg. PRODUCT_FLAG with C++ classes of same names, it's very easy to write a standalone tool to generate a manual of JVM options from header files, right? 

2. There's a TCL script.  Isn't C Processor powerful enough for it?  I don't know TCL, I think an external script will bring dependency and learning  burden.

3. let's say we choose a script generator. It looks like those globals_xxx.cpp files are auto-gen files. 
Will you consider to untrack those generated files in the repo? 

4. Probably, it's not a bad idea to still use product/develop instead of PRODUCT_FLAG, DEVELOP_FLAG. 
Those options will be read everyday by developers, I feel lower-case names are more friendly. 

[1]: https://cr.openjdk.java.net/~xliu/8151779/00/webrev/

Thanks,
--lx

?On 4/9/20, 5:08 PM, "hotspot-dev on behalf of Ioi Lam" <hotspot-dev-bounces at openjdk.java.net on behalf of ioi.lam at oracle.com> wrote:

    CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
    
    
    
    The code for specifying HotSpot command-line flags used to be simple. At
    the very beginning it was something like
    
         #define ALL_FLAGS(develop, develop) \
            product(int, SomeFlag, def_value, "some doc") \
            develop(int, MoreFlag, def_value, "more doc") \
    
    but like all good ideas, it has gradually degraded into a mess [1][2].
    It's now very difficult to add any new functionality (e.g., add
    "manageable" annotations to the flags [3]).
    
    I think it's time for a complete rewrite :-)
    
    My proposal is here. Please comment if it's a good or bad idea.
    
    https://wiki.openjdk.java.net/display/HotSpot/HotSpot+Command-Line+Flags+Overhaul+-+Design+Doc
    
    
    
    tldr; ---
    
    
    OLD:
    
       product(intx, AliasLevel, 3,                                          \
               "0 for no aliasing, 1 for oop/field/static/array split,
    "         \
               "2 for class split, 3 for unique
    instances")                      \
               range(0, 3)
    /*optional*/                                         \
               constraint(AliasLevelConstraintFunc,AfterErgo) /*optional*/
         \
    
    
    NEW:
    
       PRODUCT_FLAG(intx,   AliasLevel, 3, JVMFlag::RANGE | JVMFlag::CONSTRAINT,
                            "0 for no aliasing, 1 for oop/field/static/array
    split, "
                            "2 for class split, 3 for unique instances");
        // optional
        FLAG_RANGE(         AliasLevel, 0, 3);
        // optional
        FLAG_CONSTRAINT(    AliasLevel, (void*)AliasLevelConstraintFunc,
    JVMFlag::AfterErgo);
    
    
    
    
    [1]
    http://hg.openjdk.java.net/jdk/jdk/file/7af6364e1792/src/hotspot/share/runtime/globals.hpp#l115
    [2]
    http://hg.openjdk.java.net/jdk/jdk/file/7af6364e1792/src/hotspot/share/runtime/flags/jvmFlag.cpp#l635
    [3] https://bugs.openjdk.java.net/browse/JDK-7123237
    
    Thanks
    - Ioi
    


From denghui.ddh at alibaba-inc.com  Sun Apr 12 09:01:23 2020
From: denghui.ddh at alibaba-inc.com (Denghui Dong)
Date: Sun, 12 Apr 2020 17:01:23 +0800
Subject: =?UTF-8?B?UmU6IFJGUihYWFMpOiA4MjQyNDg1OiBOdWxsIF9maWxlIGNoZWNraW5nIGluIGZpbGVTdHJl?=
 =?UTF-8?B?YW06OmZsdXNoKCk=?=
In-Reply-To: <618b239a-2a21-a56f-021a-74df5e8d433c@oracle.com>
References: <e8d66934-57b9-44e5-a1b2-c650ea101220.denghui.ddh@alibaba-inc.com>
 <a8bd28fd-619a-d974-621d-45873edc7fa0@oss.nttdata.com>
 <a5758e50-0d65-4c90-8512-c41415b7aed7.denghui.ddh@alibaba-inc.com>
 <d7f0d3b2-1fc7-3601-b651-a4bdb0244d23@oracle.com>
 <d483f6ff-c51e-4eed-b11d-d7e8e427d8e4.denghui.ddh@alibaba-inc.com>,
 <618b239a-2a21-a56f-021a-74df5e8d433c@oracle.com>
Message-ID: <f86225d5-993a-40bf-98d1-fdb47e113877.denghui.ddh@alibaba-inc.com>

Hi David,

Thanks for the comments!

> Unfortunately the more I look at this code the more problems I see with 
> it. :( We have:
> 
> void fileStream::write(const char* s, size_t len) {
>    if (_file != NULL)  {
>      // Make an unused local variable to avoid warning from gcc compiler.
>      size_t count = fwrite(s, 1, len, _file);
>    }
>    update_position(s, len);
> }
> 
> but the update_position should be inside the if-statement - no need to 
> see a new webrev for that.

update_position is also used in fdStream::write? bufferedStream::write?and stringStream::write,
and according to the comments in stringStream::write

=====
  // Note that the following does not depend on write_len.
  // This means that position and count get updated
  // even when overflow occurs.
  update_position(s, len);
=====

If my understanding is correct, whether or not to call update_position does not depend on the success of the write action.

If there's a problem here, we should also check the usage of update_position in other methods to ensure consistent usage.


Hence,  I think we shouldn't put update_position inside the if-statement, at least for the current issue.





------------------------------------------------------------------
From:David Holmes <david.holmes at oracle.com>
Send Time:2020?4?12?(???) 15:04
To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net <hotspot-runtime-dev at openjdk.java.net>; Yasumasa Suenaga <suenaga at oss.nttdata.com>; "Liu, Xin" <xxinliu at amazon.com>
Subject:Re: RFR(XXS): 8242485: Null _file checking in fileStream::flush()

Hi Denghui,

On 12/04/2020 2:41 pm, Denghui Dong wrote:
> Hi David,
> 
> Thanks for the comments
> new webrev: http://cr.openjdk.java.net/~ddong/8242485/webrev.02
> 
> when _file is NULL:
> - fileStream::read return 0
> - fileStream::readln return NULL
> - fileStream::eof return -1 (like 
> fileStream::fileSize, not sure if appropriate, feof returns nonzero if target file is end)

That is all great - thanks!

Unfortunately the more I look at this code the more problems I see with 
it. :( We have:

void fileStream::write(const char* s, size_t len) {
   if (_file != NULL)  {
     // Make an unused local variable to avoid warning from gcc compiler.
     size_t count = fwrite(s, 1, len, _file);
   }
   update_position(s, len);
}

but the update_position should be inside the if-statement - no need to 
see a new webrev for that.

We should also be watching for errors from fwrite - but that in itself 
just opens up a can-of-worms with regard to error handling in general 
with this class, so that can be left as-is.

Thanks,
David
-----

> Testing: tier1
> 
> Cheers,
> Denghui Dong
> 
>     ------------------------------------------------------------------
>     From:David Holmes <david.holmes at oracle.com>
>     Send Time:2020?4?12?(???) 10:40
>     To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev
>     <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>     <hotspot-runtime-dev at openjdk.java.net>; Yasumasa Suenaga
>     <suenaga at oss.nttdata.com>; "Liu, Xin" <xxinliu at amazon.com>
>     Subject:Re: RFR(XXS): 8242485: Null _file checking in
>     fileStream::flush()
> 
>     Hi,
> 
>     On 12/04/2020 1:52 am, Denghui Dong wrote:
>      > Hi,
>      > Thank you for the review and point out other methods need to check _file.
>      > In addition to "fileStream::flush", I found there are still four methods need to check _file in fileStream: read, readln, eof, rewind.
>      > but those methods will crash if _file is NULL, and I didn't find any caller of them(Maybe I am wrong),
>      > so I think it's more appropriate to add assert check to those methods.
> 
>     Assertions are really only appropriate if they will be exercised by
>     tests - the intent being that tests will expose any cases where the
>     assertion fails.
> 
>     Also it doesn't make sense to me to have assertions in most places but a
> 
>     true NULL check in others - that suggests we expect some methods to be
>     called when _file is NULL, but others not. I'd prefer just to see the
>     missing NULL checks added, rather than using assertions.
> 
>     Thanks,
>     David
> 
>      > Webrev: http://cr.openjdk.java.net/~ddong/8242485/webrev.01/
>      >
>      > Could you review it again, and sponsor it if everything is okay?
>      >
>      > Testing: teir1
>      > ==============================
>      > Test summary
>      > ==============================
>      >     TEST                                              TOTAL  PASS  FAIL ERROR
>      >     jtreg:test/hotspot/jtreg:tier1                     1516  1516     0     0
>      >     jtreg:test/jdk:tier1                               1904  1904     0     0
>      >  >> jtreg:test/langtools:tier1                         4031  4029     2     0 <<
>      >     jtreg:test/nashorn:tier1                              0     0     0     0
>      >     jtreg:test/jaxp:tier1                                 0     0     0     0
>      > ==============================
>      > There are two unrelated test failures in
>      > langtools: jdk/javadoc/tool/CheckResourceKeys.java  and tools/javac/processing/model/TestSymtabItems.java
>      >
>      >
>      >
>      >     ------------------------------------------------------------------
>      >     From:Yasumasa Suenaga <suenaga at oss.nttdata.com>
>      >     Send Time:2020?4?11?(???) 09:22
>      >     To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev
>      >     <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>      >     <hotspot-runtime-dev at openjdk.java.net>
>      >     Subject:Re: RFR(XXS): 8242485: Null _file checking in
>      >     fileStream::flush()
>      >
>      >     Hi Denghui,
>      >
>      >     null check lacks in other place too.
>      >     Can you fix it?
>      >
>      >         http://hg.openjdk.java.net/jdk/jdk/file/97d5d0cd1085/src/hotspot/share/utilities/ostream.cpp#l554
>      >
>      >
>      >     Thanks,
>      >
>      >     Yasumasa
>      >
>      >
>      >     On 2020/04/10 21:34, Denghui Dong wrote:
>      >      > Hi team,
>      >      >
>      >      > Could you please review this small patch?
>      >      >
>      >      > JBS: https://bugs.openjdk.java.net/browse/JDK-8242485
>      >      > webrev: http://cr.openjdk.java.net/~ddong/8242485/webrev.00/
>      >      >
>      >      > I found the method "fileStream::flush()" lacks null check before fflush  which will cause
>      >      > all open output streams are flushed by fflush() if _file is NULL.
>      >      >
>      >      > Thanks,
>      >      > Denghui Dong
>      >      >
>      > 
> 

From cnewland at chrisnewland.com  Sun Apr 12 12:46:16 2020
From: cnewland at chrisnewland.com (Chris Newland)
Date: Sun, 12 Apr 2020 13:46:16 +0100
Subject: RFC 8236988 - Modular Design for JVM Flags
In-Reply-To: <6EE91323-A69B-44E1-8215-411250EB16DD@amazon.com>
References: <6EE91323-A69B-44E1-8215-411250EB16DD@amazon.com>
Message-ID: <7eb3c0b97a2e732d7f9374c38537438e@chrisnewland.com>

Hi Ioi, Liu Xin,

In case it is any help to this effort I built an open source tool which 
parses VM options.

VMOptionsExplorer https://github.com/chriswhocodes/VMOptionsExplorer 
parses the OpenJDK 6-15 HotSpot sources (as well as OpenJ9 sources and 
the switch outputs from Zing and GraalVM) to build VM switch 
documentation that can be searched and compared here 
https://chriswhocodes.com/vm-options-explorer.html

It also parses intrinsics information from OpenJDK sources and documents 
them here https://chriswhocodes.com/hotspot_intrinsics_jdk11.html

The parsed outputs also power an interactive VM switch inspector 
https://jacoline.dev/inspect

If any of this is useful to your goals please let me know if I can help?

Kind regards,

Chris

--
@chriswhocodes

JITWatch - Log analyser / visualiser for JVM JIT compilers.

VMOptionsExplorer - Look up JVM options and intrinsics.

JaCoLine - Java Command Line Inspector


On 2020-04-12 08:59, Liu, Xin wrote:
> Hi, Ioi,
> 
> I'm recently refactoring intrinsics options for compiler[1]. I
> introduced a new option type called "intrinsics".  The biggest trouble
> was that I had to modify multiple places.  I found your patch
> beautifully replaces jvmConstraintList.* &jvmFlagRangeList.* with some
> macros in jvmFlag.inline.hpp. great!
> 
> I have some questions or suggests.
> 
> 1. generate a document of options
> There're hundreds of JVM options.  It's hard to learn them by heart. I
> feel developers uses globals.hpp and sub header files as a de factor
> manual.
> if you break up the monolithic header, which is good,  I think
> developers deserve a manual.
> Off the top of my head, I think you can provide a mini tool, which
> consume a header aggerates all sub-header files and automatically
> generates a manual.
> 
> 2. change FLAG_RANGE and FLAG_CONTRAINT to optional parameters of 
> PRODUCT_FLAG.
> If you use variadic macro, it's possible to select RANGE or CONTRAINT
> or NIL from __VA_ARGS__.
> I used to propose it before.
> https://cr.openjdk.java.net/~xliu/8230669/00/webrev/src/hotspot/share/utilities/macro_overloading.hpp.html
> 
> I think it's a good idea because it make the declaration section pure.
> If we replace those macros eg. PRODUCT_FLAG with C++ classes of same
> names, it's very easy to write a standalone tool to generate a manual
> of JVM options from header files, right?
> 
> 2. There's a TCL script.  Isn't C Processor powerful enough for it?  I
> don't know TCL, I think an external script will bring dependency and
> learning  burden.
> 
> 3. let's say we choose a script generator. It looks like those
> globals_xxx.cpp files are auto-gen files.
> Will you consider to untrack those generated files in the repo?
> 
> 4. Probably, it's not a bad idea to still use product/develop instead
> of PRODUCT_FLAG, DEVELOP_FLAG.
> Those options will be read everyday by developers, I feel lower-case
> names are more friendly.
> 
> [1]: https://cr.openjdk.java.net/~xliu/8151779/00/webrev/
> 
> Thanks,
> --lx
> 
> ?On 4/9/20, 5:08 PM, "hotspot-dev on behalf of Ioi Lam"
> <hotspot-dev-bounces at openjdk.java.net on behalf of ioi.lam at oracle.com>
> wrote:
> 
>     CAUTION: This email originated from outside of the organization.
> Do not click links or open attachments unless you can confirm the
> sender and know the content is safe.
> 
> 
> 
>     The code for specifying HotSpot command-line flags used to be 
> simple. At
>     the very beginning it was something like
> 
>          #define ALL_FLAGS(develop, develop) \
>             product(int, SomeFlag, def_value, "some doc") \
>             develop(int, MoreFlag, def_value, "more doc") \
> 
>     but like all good ideas, it has gradually degraded into a mess 
> [1][2].
>     It's now very difficult to add any new functionality (e.g., add
>     "manageable" annotations to the flags [3]).
> 
>     I think it's time for a complete rewrite :-)
> 
>     My proposal is here. Please comment if it's a good or bad idea.
> 
> 
> https://wiki.openjdk.java.net/display/HotSpot/HotSpot+Command-Line+Flags+Overhaul+-+Design+Doc
> 
> 
> 
>     tldr; ---
> 
> 
>     OLD:
> 
>        product(intx, AliasLevel, 3,                                     
>      \
>                "0 for no aliasing, 1 for oop/field/static/array split,
>     "         \
>                "2 for class split, 3 for unique
>     instances")                      \
>                range(0, 3)
>     /*optional*/                                         \
>                constraint(AliasLevelConstraintFunc,AfterErgo) 
> /*optional*/
>          \
> 
> 
>     NEW:
> 
>        PRODUCT_FLAG(intx,   AliasLevel, 3, JVMFlag::RANGE | 
> JVMFlag::CONSTRAINT,
>                             "0 for no aliasing, 1 for 
> oop/field/static/array
>     split, "
>                             "2 for class split, 3 for unique 
> instances");
>         // optional
>         FLAG_RANGE(         AliasLevel, 0, 3);
>         // optional
>         FLAG_CONSTRAINT(    AliasLevel, 
> (void*)AliasLevelConstraintFunc,
>     JVMFlag::AfterErgo);
> 
> 
> 
> 
>     [1]
> 
> http://hg.openjdk.java.net/jdk/jdk/file/7af6364e1792/src/hotspot/share/runtime/globals.hpp#l115
>     [2]
> 
> http://hg.openjdk.java.net/jdk/jdk/file/7af6364e1792/src/hotspot/share/runtime/flags/jvmFlag.cpp#l635
>     [3] https://bugs.openjdk.java.net/browse/JDK-7123237
> 
>     Thanks
>     - Ioi

From david.holmes at oracle.com  Sun Apr 12 22:23:28 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 13 Apr 2020 08:23:28 +1000
Subject: RFR(XXS): 8242485: Null _file checking in fileStream::flush()
In-Reply-To: <f86225d5-993a-40bf-98d1-fdb47e113877.denghui.ddh@alibaba-inc.com>
References: <e8d66934-57b9-44e5-a1b2-c650ea101220.denghui.ddh@alibaba-inc.com>
 <a8bd28fd-619a-d974-621d-45873edc7fa0@oss.nttdata.com>
 <a5758e50-0d65-4c90-8512-c41415b7aed7.denghui.ddh@alibaba-inc.com>
 <d7f0d3b2-1fc7-3601-b651-a4bdb0244d23@oracle.com>
 <d483f6ff-c51e-4eed-b11d-d7e8e427d8e4.denghui.ddh@alibaba-inc.com>
 <618b239a-2a21-a56f-021a-74df5e8d433c@oracle.com>
 <f86225d5-993a-40bf-98d1-fdb47e113877.denghui.ddh@alibaba-inc.com>
Message-ID: <25546944-b9e7-bca8-aff3-9cd90c443dcd@oracle.com>

On 12/04/2020 7:01 pm, Denghui Dong wrote:
> Hi David,
> 
> Thanks for the comments!
> 
>> Unfortunately?the?more?I?look?at?this?code?the?more?problems?I?see?with 
>> it.?:(?We?have:
>> 
>> void?fileStream::write(const?char*?s,?size_t?len)?{
>>? ? if?(_file?!=?NULL)??{
>>? ? ? //?Make?an?unused?local?variable?to?avoid?warning?from?gcc?compiler.
>>? ? ? size_t?count?=?fwrite(s,?1,?len,?_file);
>>? ? }
>>? ? update_position(s,?len);
>> }
>> 
>> but?the?update_position?should?be?inside?the?if-statement?-?no?need?to 
>> see?a?new?webrev?for?that.
> 
> update_position is also used in fdStream::write? 
> bufferedStream::write?and stringStream::write,
> and according to the comments in stringStream::write
> 
> =====
>  ??//?Note?that?the?following?does?not?depend?on?write_len.
>  ??//?This?means?that?position?and?count?get?updated
>  ??//?even?when?overflow?occurs.
>  ??update_position(s,?len);
> =====
> 
> If my understanding is correct, whether?or?not?to?call update_position 
> does?not?depend?on?the?success?of?the?write?action.

Every fileStream operation should be a no-op if _file is NULL. Whether 
other streams have issues is a different matter. It makes absolutely no 
sense to update any state of the filestream in write if nothing is ever 
going to examine it anyway - just a waste of time.

Le's see if there are any other opinions on this.

Thanks,
David
-----

> If?there's?a?problem?here, we should also check the usage of 
> update_position in other methods to ensure?consistent?usage.
> 
> Hence,? I think we shouldn't put update_position inside the 
> if-statement, at?least?for?the?current?issue.
> 
> 
> 
> 
>     ------------------------------------------------------------------
>     From:David Holmes <david.holmes at oracle.com>
>     Send Time:2020?4?12?(???) 15:04
>     To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev
>     <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>     <hotspot-runtime-dev at openjdk.java.net>; Yasumasa Suenaga
>     <suenaga at oss.nttdata.com>; "Liu, Xin" <xxinliu at amazon.com>
>     Subject:Re: RFR(XXS): 8242485: Null _file checking in
>     fileStream::flush()
> 
>     Hi?Denghui,
> 
>     On?12/04/2020?2:41?pm,?Denghui?Dong?wrote:
>      >?Hi?David,
>      >
>      >?Thanks?for?the?comments
>      >?new?webrev:?http://cr.openjdk.java.net/~ddong/8242485/webrev.02
>      >
>      >?when?_file?is?NULL:
>      >?-?fileStream::read?return?0
>      >?-?fileStream::readln?return?NULL
>      >?-?fileStream::eof?return?-1?(like
>      >?fileStream::fileSize,?not?sure?if?appropriate,?feof?returns?nonzero?if?target?file?is?end)
> 
>     That?is?all?great?-?thanks!
> 
>     Unfortunately?the?more?I?look?at?this?code?the?more?problems?I?see?with
>     it.?:(?We?have:
> 
>     void?fileStream::write(const?char*?s,?size_t?len)?{
>      ???if?(_file?!=?NULL)??{
>      ?????//?Make?an?unused?local?variable?to?avoid?warning?from?gcc?compiler.
>      ?????size_t?count?=?fwrite(s,?1,?len,?_file);
>      ???}
>      ???update_position(s,?len);
>     }
> 
>     but?the?update_position?should?be?inside?the?if-statement?-?no?need?to
>     see?a?new?webrev?for?that.
> 
>     We?should?also?be?watching?for?errors?from?fwrite?-?but?that?in?itself
>     just?opens?up?a?can-of-worms?with?regard?to?error?handling?in?general
>     with?this?class,?so?that?can?be?left?as-is.
> 
>     Thanks,
>     David
>     -----
> 
>      >?Testing:?tier1
>      >
>      >?Cheers,
>      >?Denghui?Dong
>      >
>      >?????------------------------------------------------------------------
>      >?????From:David?Holmes?<david.holmes at oracle.com>
>      >?????Send?Time:2020?4?12?(???)?10:40
>      >?????To:???(??)?<denghui.ddh at alibaba-inc.com>;?hotspot-dev
>      >?????<hotspot-dev at openjdk.java.net>;?hotspot-runtime-dev at openjdk.java.net
>      >?????<hotspot-runtime-dev at openjdk.java.net>;?Yasumasa?Suenaga
>      >?????<suenaga at oss.nttdata.com>;?"Liu,?Xin"?<xxinliu at amazon.com>
>      >?????Subject:Re:?RFR(XXS):?8242485:?Null?_file?checking?in
>      >?????fileStream::flush()
>      >
>      >?????Hi,
>      >
>      >?????On?12/04/2020?1:52?am,?Denghui?Dong?wrote:
>      >??????>?Hi,
>      >??????>?Thank?you?for?the?review?and?point?out?other?methods?need?to?check?_file.
>      >??????>?In?addition?to?"fileStream::flush",?I?found?there?are?still?four?methods?need?to?check?_file?in?fileStream:?read,?readln,?eof,?rewind.
>      >??????>?but?those?methods?will?crash?if?_file?is?NULL,?and?I?didn't?find?any?caller?of?them(Maybe?I?am?wrong),
>      >??????>?so?I?think?it's?more?appropriate?to?add?assert?check?to?those?methods.
>      >
>      >?????Assertions?are?really?only?appropriate?if?they?will?be?exercised?by
>      >?????tests?-?the?intent?being?that?tests?will?expose?any?cases?where?the
>      >?????assertion?fails.
>      >
>      >?????Also?it?doesn't?make?sense?to?me?to?have?assertions?in?most?places?but?a
>      >
>      >?????true?NULL?check?in?others?-?that?suggests?we?expect?some?methods?to?be
>      >?????called?when?_file?is?NULL,?but?others?not.?I'd?prefer?just?to?see?the
>      >?????missing?NULL?checks?added,?rather?than?using?assertions.
>      >
>      >?????Thanks,
>      >?????David
>      >
>      >??????>?Webrev:?http://cr.openjdk.java.net/~ddong/8242485/webrev.01/
>      >??????>
>      >??????>?Could?you?review?it?again,?and?sponsor?it?if?everything?is?okay?
>      >??????>
>      >??????>?Testing:?teir1
>      >??????>?==============================
>      >??????>?Test?summary
>      >??????>?==============================
>      >??????>?????TEST??????????????????????????????????????????????TOTAL??PASS??FAIL?ERROR
>      >??????>?????jtreg:test/hotspot/jtreg:tier1?????????????????????1516??1516?????0?????0
>      >??????>?????jtreg:test/jdk:tier1???????????????????????????????1904??1904?????0?????0
>      >??????>??>>?jtreg:test/langtools:tier1?????????????????????????4031??4029?????2?????0?<<
>      >??????>?????jtreg:test/nashorn:tier1??????????????????????????????0?????0?????0?????0
>      >??????>?????jtreg:test/jaxp:tier1?????????????????????????????????0?????0?????0?????0
>      >??????>?==============================
>      >??????>?There?are?two?unrelated?test?failures?in
>      >??????>?langtools:?jdk/javadoc/tool/CheckResourceKeys.java??and?tools/javac/processing/model/TestSymtabItems.java
>      >??????>
>      >??????>
>      >??????>
>      >??????>?????------------------------------------------------------------------
>      >??????>?????From:Yasumasa?Suenaga?<suenaga at oss.nttdata.com>
>      >??????>?????Send?Time:2020?4?11?(???)?09:22
>      >??????>?????To:???(??)?<denghui.ddh at alibaba-inc.com>;?hotspot-dev
>      >??????>?????<hotspot-dev at openjdk.java.net>;?hotspot-runtime-dev at openjdk.java.net
>      >??????>?????<hotspot-runtime-dev at openjdk.java.net>
>      >??????>?????Subject:Re:?RFR(XXS):?8242485:?Null?_file?checking?in
>      >??????>?????fileStream::flush()
>      >??????>
>      >??????>?????Hi?Denghui,
>      >??????>
>      >??????>?????null?check?lacks?in?other?place?too.
>      >??????>?????Can?you?fix?it?
>      >??????>
>      >??????>?????????http://hg.openjdk.java.net/jdk/jdk/file/97d5d0cd1085/src/hotspot/share/utilities/ostream.cpp#l554
>      >??????>
>      >??????>
>      >??????>?????Thanks,
>      >??????>
>      >??????>?????Yasumasa
>      >??????>
>      >??????>
>      >??????>?????On?2020/04/10?21:34,?Denghui?Dong?wrote:
>      >??????>??????>?Hi?team,
>      >??????>??????>
>      >??????>??????>?Could?you?please?review?this?small?patch?
>      >??????>??????>
>      >??????>??????>?JBS:?https://bugs.openjdk.java.net/browse/JDK-8242485
>      >??????>??????>?webrev:?http://cr.openjdk.java.net/~ddong/8242485/webrev.00/
>      >??????>??????>
>      >??????>??????>?I?found?the?method?"fileStream::flush()"?lacks?null?check?before?fflush??which?will?cause
>      >??????>??????>?all?open?output?streams?are?flushed?by?fflush()?if?_file?is?NULL.
>      >??????>??????>
>      >??????>??????>?Thanks,
>      >??????>??????>?Denghui?Dong
>      >??????>??????>
>      >??????>
>      > 
> 

From ioi.lam at oracle.com  Mon Apr 13 00:22:11 2020
From: ioi.lam at oracle.com (Ioi Lam)
Date: Sun, 12 Apr 2020 17:22:11 -0700
Subject: RFC 8236988 - Modular Design for JVM Flags
In-Reply-To: <6EE91323-A69B-44E1-8215-411250EB16DD@amazon.com>
References: <6EE91323-A69B-44E1-8215-411250EB16DD@amazon.com>
Message-ID: <e0a4bcc5-e116-c901-85f9-78360307e243@oracle.com>

Hi Xin,

Thanks for the feedback. It's very helpful.

On 4/12/20 12:59 AM, Liu, Xin wrote:
> Hi, Ioi,
>
> I'm recently refactoring intrinsics options for compiler[1]. I introduced a new option type called "intrinsics".  The biggest trouble was that I had to modify multiple places.  I found your patch beautifully replaces jvmConstraintList.* &jvmFlagRangeList.* with some macros in jvmFlag.inline.hpp. great!
>
> I have some questions or suggests.
>
> 1. generate a document of options
> There're hundreds of JVM options.  It's hard to learn them by heart. I feel developers uses globals.hpp and sub header files as a de factor manual.
> if you break up the monolithic header, which is good,  I think developers deserve a manual.
> Off the top of my head, I think you can provide a mini tool, which consume a header aggerates all sub-header files and automatically generates a manual.

Even today the flags are scattered around in various *.hpp files. There 
are already ways of finding information about the all the flags:

 ??? java -XX:+PrintFlagsWithComments?? (debug build only)
 ??? java -XX:+PrintFlagsInitial
 ??? java -XX:+PrintFlagsFinal
 ??? java -XX:+PrintFlagsRanges

These flags will continue to work after my patch. Would these be sufficient?

Also, you can use this to look for a flag:

 ??? find . -name \*.hpp | xargs grep FLAG.*SplitIfBlocks

> 2. change FLAG_RANGE and FLAG_CONTRAINT to optional parameters of PRODUCT_FLAG.
> If you use variadic macro, it's possible to select RANGE or CONTRAINT or NIL from __VA_ARGS__.
> I used to propose it before. https://cr.openjdk.java.net/~xliu/8230669/00/webrev/src/hotspot/share/utilities/macro_overloading.hpp.html
>
> I think it's a good idea because it make the declaration section pure.
> If we replace those macros eg. PRODUCT_FLAG with C++ classes of same names, it's very easy to write a standalone tool to generate a manual of JVM options from header files, right?

That's a good suggestion. I've uploaded an updated version of the design 
that uses __VA_ARGS__

http://cr.openjdk.java.net/~iklam/design/jvm_flags_overhaul/flags.hpp
http://cr.openjdk.java.net/~iklam/design/jvm_flags_overhaul/flags.cpp

To see how the macros work, try doing this, and look at the file flags.ii

 ??? gcc -DPRODUCT -O3 -o flags -save-temps flags.cpp && ./flags
 ??? gcc -g -o flags_g -save-temps flags.cpp && ./flags_g

I think we can try different designs using the above as a template. When 
we have a more concrete design I can implement it in the VM.

Now the declaration is:

 ??? PRODUCT_FLAG(bool,???? SplitIfBlocks, true, JVMFlag::DEFAULT,
 ?????????????????????????? "Clone compares and control flow through merge "
 ?????????????????????????? "points to fold some branches");

 ??? DEVELOP_FLAG(intx,???? FreqCountInvocations, 1, JVMFlag::DEFAULT,
 ?????????????????????????? "Scaling factor for branch frequencies",
 ?????????????????????????? FLAG_RANGE(1, max_intx))

 ??? PRODUCT_FLAG(intx,???? NodeLimitFudgeFactor, 2000, JVMFlag::CONSTRAINT,
 ?????????????????????????? "Fudge Factor for certain optimizations",
FLAG_CONSTRAINT(NodeLimitFudgeFactorConstraintFunc,
 ? ? ? ? ? ? ? ? ? ? ? ? ?? JVMFlag::AfterErgo))

 ??? PRODUCT_FLAG(intx,???? AliasLevel, 3, JVMFlag::DEFAULT,
 ?????????????????????????? "0 for no aliasing, 1 for .... split, "
 ?????????????????????????? "2 for class split, 3 for unique instances",
 ?????????????????????????? FLAG_RANGE(0, 3),
FLAG_CONSTRAINT(AliasLevelConstraintFunc,
 ? ? ? ? ? ? ? ? ? ? ? ? ?? JVMFlag::AfterErgo))

and the implementation:

 ??? DEFN_PRODUCT_FLAG(SplitIfBlocks);
 ??? DEFN_DEVELOP_FLAG(FreqCountInvocations);
 ??? DEFN_PRODUCT_FLAG(NodeLimitFudgeFactor);
 ??? DEFN_PRODUCT_FLAG(AliasLevel);

My goal is still to put all the information about each flag in a single 
place (the hpp file), so I need to pass the info to the cpp files using 
an inline function:

 ? inline const ModifiableFlag<FLAG_TYPE_##name> init_##name() { \
 ??? return ModifiableFlag<FLAG_TYPE_##name>( \
 ????? JVMFlag::TYPE_##type, #name, attr, &name, __VA_ARGS__); \
 ? }


But the problem is this inline function could theoretically allocate the 
instance on the stack, so I cannot save its address into the _head of 
the global flags list. As I result, I needed to split the _next pointer 
from JVMFlag into AbstractFlagLink::_link.

If anyone has a better suggestion, I would be happy to try it out.

> 2. There's a TCL script.  Isn't C Processor powerful enough for it?  I don't know TCL, I think an external script will bring dependency and learning  burden.
>
> 3. let's say we choose a script generator. It looks like those globals_xxx.cpp files are auto-gen files.
> Will you consider to untrack those generated files in the repo?

The whole purpose of the TCL script is to convert the old style macros 
in the hpp files into the new coding style. That way I can keep track of 
changes in the upstream repo, and also easily change the design of the 
new code. The script is used by me only (that's why it's messy). It's 
not used during the build and will not be needed in the final patch.

The globals_xxx.cpp files will be checked into the repo. Afterwards, to 
add/remove flags, you would need to edit the xxx_globals.hpp/cpp files 
directly.

As mentioned in the design doc, my goal is to avoid C++ code generation 
as many people are against it.

Personally, I prefer code generation. I think that can result in a 
smaller binary footprint that will start up faster, but the delta is 
probably negligible. We can probably leave that for the future. As long 
as we use the macros in a consistent manner, we can easily extract the 
flag definitions and put them into external input files that are 
processed by a C++ code generator.

> 4. Probably, it's not a bad idea to still use product/develop instead of PRODUCT_FLAG, DEVELOP_FLAG.
> Those options will be read everyday by developers, I feel lower-case names are more friendly.

I don't have a strong preference, but I think that adding "product" 
globally (even as a macro) may cause problems with code like this:

 ??? static int product(int a, int b) {return a*b;}

as the C pre-processor would try to expand "product".

Thanks
- Ioi


>
> [1]: https://cr.openjdk.java.net/~xliu/8151779/00/webrev/
>
> Thanks,
> --lx
>
> ?On 4/9/20, 5:08 PM, "hotspot-dev on behalf of Ioi Lam" <hotspot-dev-bounces at openjdk.java.net on behalf of ioi.lam at oracle.com> wrote:
>
>      CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>      
>      
>      
>      The code for specifying HotSpot command-line flags used to be simple. At
>      the very beginning it was something like
>      
>           #define ALL_FLAGS(develop, develop) \
>              product(int, SomeFlag, def_value, "some doc") \
>              develop(int, MoreFlag, def_value, "more doc") \
>      
>      but like all good ideas, it has gradually degraded into a mess [1][2].
>      It's now very difficult to add any new functionality (e.g., add
>      "manageable" annotations to the flags [3]).
>      
>      I think it's time for a complete rewrite :-)
>      
>      My proposal is here. Please comment if it's a good or bad idea.
>      
>      https://wiki.openjdk.java.net/display/HotSpot/HotSpot+Command-Line+Flags+Overhaul+-+Design+Doc
>      
>      
>      
>      tldr; ---
>      
>      
>      OLD:
>      
>         product(intx, AliasLevel, 3,                                          \
>                 "0 for no aliasing, 1 for oop/field/static/array split,
>      "         \
>                 "2 for class split, 3 for unique
>      instances")                      \
>                 range(0, 3)
>      /*optional*/                                         \
>                 constraint(AliasLevelConstraintFunc,AfterErgo) /*optional*/
>           \
>      
>      
>      NEW:
>      
>         PRODUCT_FLAG(intx,   AliasLevel, 3, JVMFlag::RANGE | JVMFlag::CONSTRAINT,
>                              "0 for no aliasing, 1 for oop/field/static/array
>      split, "
>                              "2 for class split, 3 for unique instances");
>          // optional
>          FLAG_RANGE(         AliasLevel, 0, 3);
>          // optional
>          FLAG_CONSTRAINT(    AliasLevel, (void*)AliasLevelConstraintFunc,
>      JVMFlag::AfterErgo);
>      
>      
>      
>      
>      [1]
>      http://hg.openjdk.java.net/jdk/jdk/file/7af6364e1792/src/hotspot/share/runtime/globals.hpp#l115
>      [2]
>      http://hg.openjdk.java.net/jdk/jdk/file/7af6364e1792/src/hotspot/share/runtime/flags/jvmFlag.cpp#l635
>      [3] https://bugs.openjdk.java.net/browse/JDK-7123237
>      
>      Thanks
>      - Ioi
>      
>


From ioi.lam at oracle.com  Mon Apr 13 05:40:46 2020
From: ioi.lam at oracle.com (Ioi Lam)
Date: Sun, 12 Apr 2020 22:40:46 -0700
Subject: RFR (M) JDK-8210012: Implement Unified Logging Option for
 -XX:+TraceMethodHandles and -XX:+TraceInvokeDynamic
In-Reply-To: <ad36856f-5e45-f636-9c11-0bd9292da05e@oracle.com>
References: <9f678d60-93c1-f2e3-84a1-51feb74c6961@oracle.com>
 <3e7cc93f-8c9b-85a4-ec68-39720bf876b4@oracle.com>
 <ad36856f-5e45-f636-9c11-0bd9292da05e@oracle.com>
Message-ID: <ab0a778e-f26c-3180-474c-5580af87463a@oracle.com>



On 4/10/20 3:05 PM, Lois Foltan wrote:
>
>> On 4/9/20 8:12 PM, David Holmes wrote:
>>> Hi Lois,
>>>
>>> On 10/04/2020 4:42 am, Lois Foltan wrote:
>>>> Please review the following change to implement unified logging 
>>>> options for -XX:+TraceMethodHandles and -XX:+TraceInvokeDynamic. 
>>>> The new options map as follows:
>>>>
>>>> -XX:+TraceMethodHandles --> -Xlog:methodhandles=info
>>>> -XX:+TraceInvokeDynamic --> -Xlog:methodhandles+indy=debug
>>>>
>>>> and in addition dynamic constants can now be viewed under their own 
>>>> option via -Xlog:methodhandles+condy=debug
>>>>
>>>> open webrev at: 
>>>> http://cr.openjdk.java.net/~lfoltan/bug_jdk8210012.0/webrev/
>>>> bug link: https://bugs.openjdk.java.net/browse/JDK-8210012
>>>
>>> Generally looks good. A couple of things.
>>>
>>> First the change in compileTask.cpp seems wrong: you should get the 
>>> timestamp from the specified stream not the tty.
>>>
>>> Second the changes to wrap_dynamic_exception seem rather awkward, 
>>> both in terms of having to remember what the initial boolean arg 
>>> represents at the call site, and more so in the logic to determine 
>>> which log stream to use. I was wondering if it might be better to 
>>> add wrap_dynamic_exception_for_indy to keep the call sites clean? 
>>> Otherwise perhaps document like "true /* indy */" and "false /* not 
>>> indy */.
>>>
>>> For the log stream logic I was assuming there must be a 
>>> better/simpler way, but after looking at it in some detail it seems 
>>> not. :(
>>>
>>
>> Hi Lois,
>
> Hi Ioi,
>
> Thanks for the review!
>
>>
>>
>> LinkResolver::lookup_polymorphic_method()
>> +? ResourceMark rm(THREAD);
>>
>> Is this ResourceMark used only for logging?
>
> Yes.? As a matter of fact specifying -XX:+TraceMethodHandles or 
> -XX:+TraceInvokeDynamic currently on my sample test case results today 
> in crashes concerning the lack of allocating without a ResourceMark.
>
>>
>> =================
>> For code like this in ConstantPool::resolve_constant_at_impl:
>>
>> ??? if (log_is_enabled(Debug, methodhandles, condy)) {
>> ????? LogTarget(Debug, methodhandles, condy) lt;
>> ????? LogStream ls(lt);
>> ????? bootstrap_specifier.print_msg_on(&ls, "resolve_constant_at_impl");
>> ??? }
>>
>> The second set of tags can be avoided by declaring LogTarget(Debug, 
>> methodhandles, condy) first. There's an example of this style in 
>> FileMapInfo::log_paths().
>>
>> ??? LogTarget(Debug, methodhandles, condy) lt;
>> ??? if (lt.is_enabled()) {
>> ????? LogStream ls(lt);
>> ????? bootstrap_specifier.print_msg_on(&ls, "resolve_constant_at_impl");
>> ??? }
>
> Fixed, I have changed all checks in this webrev to consistently use 
> this form of checking if LogTarget is enabled.
>
>>
>> =================
>>
>> unpack_method_and_appendix() {
>>
>> ????? if (log_develop_is_enabled(Info, methodhandles)) {
>> ??????? ResourceMark rm(THREAD);
>> ??????? LogTarget(Info, methodhandles) lt;
>> ??????? LogStream ls(lt);
>> ??????? ls->print_on(.....);
>>
>> This can be fixed similarly, if we add a new function to LogTargetImpl:
>>
>> ?? class LogTargetImpl {
>> ? ?? static bool develop_is_enabled() {
>> ?? ? ? NOT_PRODUCT(return LogImpl<T0, T1, T2, T3, T4, 
>> GuardTag>::is_level(level));
>> ?????? PRODUCT_ONLY(return false);
>> ? ?? }
>> ???? ...
>> ?? }
>>
>> ? ? ? LogTarget(Info, methodhandles) lt;
>> ????? if (lt.develop_is_enabled()) {
>> ??????? ResourceMark rm(THREAD);
>> ??????? LogStream ls(lt);
>> ??????? ls->print_on(.....);
>
> Fixed. Added the method "develop_is_enabled()" to class 
> LogTargetImpl.? However my method looks more like this:
>
> static bool develop_is_enabled() {
> ? NOT_PRODUCT(return is_enabled();)
> ? PRODUCT_ONLY(return false);
> }
>
>>
>> =================
>> void ConstantPoolCacheEntry::set_method_handle_common()
>>
>> ? LogStreamHandle(Debug, methodhandles, indy) lsh;
>> ? if (log_is_enabled(Debug, methodhandles, indy)) {
>>
>> Maybe we should add a new function LogStream::is_enabled so we can:
>>
>> ? LogStreamHandle(Debug, methodhandles, indy) lsh;
>> ? if (lsh.is_enabled()) {
>
> Fixed. Turns out that a LogTargetHandle already has an "is_enabled()" 
> method.? So I added an "is_enabled()" method to class LogStream to 
> check if its field, _log_handle, whose type is LogTargetHandle, is 
> enabled.
>
>>
>>
>> =================
>> Not a comment about this patch, but in general, I am not happy with 
>> the verbosity when a ResourceMark is needed in logging.
>>
>> I think the best way to avoid the verbosity would be to use C++ 
>> lambda expressions (are we allowed??), or (oh gosh!) maybe a macro? 
>> (Probably in a separate RFE).
>>
>> ??? LOG_WITH_RESOURCE_MARK((Info, methodhandles), THREAD,ls, {
>> ????? ? ls->print_on(.....);
>> ?? ?? });
>
> Good idea, but will create an RFE for this.
>
> New webrev at: 
> http://cr.openjdk.java.net/~lfoltan/bug_jdk8210012.1/webrev/index.html
>

Hi Lois,

This new version looks good to me.

Thanks
- Ioi


> Thanks,
> Lois
>
>>
>>
>> Thanks
>> - Ioi
>>
>>
>>> Thanks,
>>> David
>>>
>>>> Testing: hs-tier1-5
>>>>
>>>> Thanks,
>>>> Lois
>>
>


From coleen.phillimore at oracle.com  Mon Apr 13 14:34:01 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Mon, 13 Apr 2020 10:34:01 -0400
Subject: RFR (S) 8074292: nsk/jdb/kill/kill001: generateOopMap.cpp
 assert(bb->is_reachable()) failed
Message-ID: <f66b8de6-d9db-a90f-bfc2-5bf42a19cada@oracle.com>

Summary: Do not install async exceptions at_safepoint for each bytecode.

See CR for a lot more details.? This change calls a new 
InterpreterRuntime::at_safepoint_async_safe() which installs the async 
exception in the interpreter at backward branches and returns.? This 
uses safepoint polling code in the interpreter for each platform.? These 
changes (cross) compile on platforms that Oracle doesn't support but I 
don't know if they work.

I'm not convinced the platform specific changes are necessary, because 
calls to the runtime from many bytecodes will install the async 
exception, so it's essentially installed "enough" for this deprecated 
feature.? I tested the changes with *and* without the platform specific 
changes with no failure, which included the jdb, jdi and jvmti 
serviceability tests.

This change also makes InterpreterRuntime::monitorexit a JRT_LEAF 
bytecode. The code to check for exceptions is outside the runtime call.? 
I ran the JCK vm and lang tests on this change with no failure.

Tested with tier1-6.

open webrev at http://cr.openjdk.java.net/~coleenp/2020/8074292.01/webrev
bug link https://bugs.openjdk.java.net/browse/JDK-8074292

Thanks,
Coleen


From lois.foltan at oracle.com  Mon Apr 13 17:53:33 2020
From: lois.foltan at oracle.com (Lois Foltan)
Date: Mon, 13 Apr 2020 13:53:33 -0400
Subject: RFR (M) JDK-8210012: Implement Unified Logging Option for
 -XX:+TraceMethodHandles and -XX:+TraceInvokeDynamic
In-Reply-To: <28cd4d46-ab4d-c5fe-7993-0a0da61fade2@oracle.com>
References: <07752bc9-3c6f-6b49-6403-bce3080e54cf@oracle.com>
 <41e05f04-e6f3-cba4-d1d3-6bc1b156071d@oracle.com>
 <0319c697-10b0-9aaf-7e9f-2762d2cd1c16@oracle.com>
 <28cd4d46-ab4d-c5fe-7993-0a0da61fade2@oracle.com>
Message-ID: <b3af30fd-4bdb-2f14-8cec-f493df366a7c@oracle.com>

On 4/10/2020 10:57 PM, David Holmes wrote:
> Hi Lois,
>
> On 11/04/2020 7:55 am, Lois Foltan wrote:
>> On 4/9/2020 11:12 PM, David Holmes wrote:
>>> Hi Lois,
>>>
>>> On 10/04/2020 4:42 am, Lois Foltan wrote:
>>>> Please review the following change to implement unified logging 
>>>> options for -XX:+TraceMethodHandles and -XX:+TraceInvokeDynamic. 
>>>> The new options map as follows:
>>>>
>>>> -XX:+TraceMethodHandles --> -Xlog:methodhandles=info
>>>> -XX:+TraceInvokeDynamic --> -Xlog:methodhandles+indy=debug
>>>>
>>>> and in addition dynamic constants can now be viewed under their own 
>>>> option via -Xlog:methodhandles+condy=debug
>>>>
>>>> open webrev at: 
>>>> http://cr.openjdk.java.net/~lfoltan/bug_jdk8210012.0/webrev/
>>>> bug link: https://bugs.openjdk.java.net/browse/JDK-8210012
>>>
>>> Generally looks good. A couple of things.
>>
>> Thanks David for the review!
>>
>>>
>>> First the change in compileTask.cpp seems wrong: you should get the 
>>> timestamp from the specified stream not the tty.
>>
>> At first it looked wrong to me as well, but I was receiving an assert 
>> about the TimeStamp associated with the outputStream being cleared, 
>> (the first assert in TimeStamp::milliseconds). After investigation it 
>> turns out that CompileTask::print_impl() has never been called with 
>> any outputStream other than tty.? And 3-or 4 levels up when I call it 
>> with a LogStream (which inherits from outputStream and constructs 
>> with a default ctor that does not set a time stamp), it seems odd at 
>> that point that I have to determine a timestamp, one that does not 
>> seem relevant to CompileTask.? What I believe CompileTask wants, and 
>> what it seems other calls in the compiler directory use is the time 
>> since the vm was started.? See the support for PrintCompilation in 
>> compiler/compileBroker.cpp for the use of tty->timestamp() vs. 
>> CompileBroker's _time* timestamps.? Thus I think my change to use tty 
>> in CompileTask::print_impl() is correct.
>
> So we have an outputStream API for timestamps, but in practice it is 
> enabled on a per-instance basis (only the tty?) and we have whole 
> subclasses (stringStream, LogStream) that don't support it. Yet we 
> have code that claims to generically operate on any outputstream but 
> calls timestamp() which may assert if not called on a supporting 
> instance! What a mess. :(
>
> We also have two notions of "uptime" in the VM:
> - one is the timestamp held within the tty
> - the other is os::elapsedTime()? (which is used by UL)
>
> Probably some scope for some cleanup here ... like get rid of 
> timestamp from the tty/outputStream as it is barely used and used 
> incorrectly in a couple of places. Further, it's main use was 
> consistency across all the logging output to the tty, but now very 
> little is logged to the tty so we have inconsistencies across 
> different log output anyway.
>
> But not for this RFE.

Exactly and when I stated that my use of tty is correct, I meant correct 
for this current situation but hopefully not long term.

>
>>>
>>> Second the changes to wrap_dynamic_exception seem rather awkward, 
>>> both in terms of having to remember what the initial boolean arg 
>>> represents at the call site, and more so in the logic to determine 
>>> which log stream to use. I was wondering if it might be better to 
>>> add wrap_dynamic_exception_for_indy to keep the call sites clean? 
>>> Otherwise perhaps document like "true /* indy */" and "false /* not 
>>> indy */.
>>
>> I would rather not break wrap_dynamic_exception into 2 methods since 
>> this is a JVMS behavior to wrap certain exceptions into a 
>> BootstrapMethodError for both indy and condy.? So I have instead 
>> opted for your suggestion of using comments when calling 
>> wrap_dynamic_exception.
>>
>> New webrev at: 
>> http://cr.openjdk.java.net/~lfoltan/bug_jdk8210012.1/webrev/index.html
>
> Updates look fine.

Thanks again for the review!
Lois

>
> Thanks,
> David
>
>> Thanks,
>> Lois
>>
>>>
>>> For the log stream logic I was assuming there must be a 
>>> better/simpler way, but after looking at it in some detail it seems 
>>> not. :(
>>>
>>> Thanks,
>>> David
>>>
>>>> Testing: hs-tier1-5
>>>>
>>>> Thanks,
>>>> Lois
>>


From lois.foltan at oracle.com  Mon Apr 13 18:30:12 2020
From: lois.foltan at oracle.com (Lois Foltan)
Date: Mon, 13 Apr 2020 14:30:12 -0400
Subject: RFR (M) JDK-8210012: Implement Unified Logging Option for
 -XX:+TraceMethodHandles and -XX:+TraceInvokeDynamic
In-Reply-To: <ab0a778e-f26c-3180-474c-5580af87463a@oracle.com>
References: <9f678d60-93c1-f2e3-84a1-51feb74c6961@oracle.com>
 <3e7cc93f-8c9b-85a4-ec68-39720bf876b4@oracle.com>
 <ad36856f-5e45-f636-9c11-0bd9292da05e@oracle.com>
 <ab0a778e-f26c-3180-474c-5580af87463a@oracle.com>
Message-ID: <390c484c-a838-1a6b-69fd-813a71b23fa7@oracle.com>

On 4/13/2020 1:40 AM, Ioi Lam wrote:
>
>
> On 4/10/20 3:05 PM, Lois Foltan wrote:
>>
>>> On 4/9/20 8:12 PM, David Holmes wrote:
>>>> Hi Lois,
>>>>
>>>> On 10/04/2020 4:42 am, Lois Foltan wrote:
>>>>> Please review the following change to implement unified logging 
>>>>> options for -XX:+TraceMethodHandles and -XX:+TraceInvokeDynamic. 
>>>>> The new options map as follows:
>>>>>
>>>>> -XX:+TraceMethodHandles --> -Xlog:methodhandles=info
>>>>> -XX:+TraceInvokeDynamic --> -Xlog:methodhandles+indy=debug
>>>>>
>>>>> and in addition dynamic constants can now be viewed under their 
>>>>> own option via -Xlog:methodhandles+condy=debug
>>>>>
>>>>> open webrev at: 
>>>>> http://cr.openjdk.java.net/~lfoltan/bug_jdk8210012.0/webrev/
>>>>> bug link: https://bugs.openjdk.java.net/browse/JDK-8210012
>>>>
>>>> Generally looks good. A couple of things.
>>>>
>>>> First the change in compileTask.cpp seems wrong: you should get the 
>>>> timestamp from the specified stream not the tty.
>>>>
>>>> Second the changes to wrap_dynamic_exception seem rather awkward, 
>>>> both in terms of having to remember what the initial boolean arg 
>>>> represents at the call site, and more so in the logic to determine 
>>>> which log stream to use. I was wondering if it might be better to 
>>>> add wrap_dynamic_exception_for_indy to keep the call sites clean? 
>>>> Otherwise perhaps document like "true /* indy */" and "false /* not 
>>>> indy */.
>>>>
>>>> For the log stream logic I was assuming there must be a 
>>>> better/simpler way, but after looking at it in some detail it seems 
>>>> not. :(
>>>>
>>>
>>> Hi Lois,
>>
>> Hi Ioi,
>>
>> Thanks for the review!
>>
>>>
>>>
>>> LinkResolver::lookup_polymorphic_method()
>>> +? ResourceMark rm(THREAD);
>>>
>>> Is this ResourceMark used only for logging?
>>
>> Yes.? As a matter of fact specifying -XX:+TraceMethodHandles or 
>> -XX:+TraceInvokeDynamic currently on my sample test case results 
>> today in crashes concerning the lack of allocating without a 
>> ResourceMark.
>>
>>>
>>> =================
>>> For code like this in ConstantPool::resolve_constant_at_impl:
>>>
>>> ??? if (log_is_enabled(Debug, methodhandles, condy)) {
>>> ????? LogTarget(Debug, methodhandles, condy) lt;
>>> ????? LogStream ls(lt);
>>> ????? bootstrap_specifier.print_msg_on(&ls, 
>>> "resolve_constant_at_impl");
>>> ??? }
>>>
>>> The second set of tags can be avoided by declaring LogTarget(Debug, 
>>> methodhandles, condy) first. There's an example of this style in 
>>> FileMapInfo::log_paths().
>>>
>>> ??? LogTarget(Debug, methodhandles, condy) lt;
>>> ??? if (lt.is_enabled()) {
>>> ????? LogStream ls(lt);
>>> ????? bootstrap_specifier.print_msg_on(&ls, 
>>> "resolve_constant_at_impl");
>>> ??? }
>>
>> Fixed, I have changed all checks in this webrev to consistently use 
>> this form of checking if LogTarget is enabled.
>>
>>>
>>> =================
>>>
>>> unpack_method_and_appendix() {
>>>
>>> ????? if (log_develop_is_enabled(Info, methodhandles)) {
>>> ??????? ResourceMark rm(THREAD);
>>> ??????? LogTarget(Info, methodhandles) lt;
>>> ??????? LogStream ls(lt);
>>> ??????? ls->print_on(.....);
>>>
>>> This can be fixed similarly, if we add a new function to LogTargetImpl:
>>>
>>> ?? class LogTargetImpl {
>>> ? ?? static bool develop_is_enabled() {
>>> ?? ? ? NOT_PRODUCT(return LogImpl<T0, T1, T2, T3, T4, 
>>> GuardTag>::is_level(level));
>>> ?????? PRODUCT_ONLY(return false);
>>> ? ?? }
>>> ???? ...
>>> ?? }
>>>
>>> ? ? ? LogTarget(Info, methodhandles) lt;
>>> ????? if (lt.develop_is_enabled()) {
>>> ??????? ResourceMark rm(THREAD);
>>> ??????? LogStream ls(lt);
>>> ??????? ls->print_on(.....);
>>
>> Fixed. Added the method "develop_is_enabled()" to class 
>> LogTargetImpl.? However my method looks more like this:
>>
>> static bool develop_is_enabled() {
>> ? NOT_PRODUCT(return is_enabled();)
>> ? PRODUCT_ONLY(return false);
>> }
>>
>>>
>>> =================
>>> void ConstantPoolCacheEntry::set_method_handle_common()
>>>
>>> ? LogStreamHandle(Debug, methodhandles, indy) lsh;
>>> ? if (log_is_enabled(Debug, methodhandles, indy)) {
>>>
>>> Maybe we should add a new function LogStream::is_enabled so we can:
>>>
>>> ? LogStreamHandle(Debug, methodhandles, indy) lsh;
>>> ? if (lsh.is_enabled()) {
>>
>> Fixed. Turns out that a LogTargetHandle already has an "is_enabled()" 
>> method.? So I added an "is_enabled()" method to class LogStream to 
>> check if its field, _log_handle, whose type is LogTargetHandle, is 
>> enabled.
>>
>>>
>>>
>>> =================
>>> Not a comment about this patch, but in general, I am not happy with 
>>> the verbosity when a ResourceMark is needed in logging.
>>>
>>> I think the best way to avoid the verbosity would be to use C++ 
>>> lambda expressions (are we allowed??), or (oh gosh!) maybe a macro? 
>>> (Probably in a separate RFE).
>>>
>>> ??? LOG_WITH_RESOURCE_MARK((Info, methodhandles), THREAD,ls, {
>>> ????? ? ls->print_on(.....);
>>> ?? ?? });
>>
>> Good idea, but will create an RFE for this.

Created https://bugs.openjdk.java.net/browse/JDK-8242589 for this RFE.

>>
>> New webrev at: 
>> http://cr.openjdk.java.net/~lfoltan/bug_jdk8210012.1/webrev/index.html
>>
>
> Hi Lois,
>
> This new version looks good to me.

Thanks again for the review Ioi!
Lois

>
> Thanks
> - Ioi
>
>
>> Thanks,
>> Lois
>>
>>>
>>>
>>> Thanks
>>> - Ioi
>>>
>>>
>>>> Thanks,
>>>> David
>>>>
>>>>> Testing: hs-tier1-5
>>>>>
>>>>> Thanks,
>>>>> Lois
>>>
>>
>


From martin.doerr at sap.com  Mon Apr 13 18:32:08 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Mon, 13 Apr 2020 18:32:08 +0000
Subject: RFR (S) 8074292: nsk/jdb/kill/kill001: generateOopMap.cpp
 assert(bb->is_reachable()) failed
In-Reply-To: <f66b8de6-d9db-a90f-bfc2-5bf42a19cada@oracle.com>
References: <f66b8de6-d9db-a90f-bfc2-5bf42a19cada@oracle.com>
Message-ID: <AM0PR0202MB329791E930A720FE320D13A99ADD0@AM0PR0202MB3297.eurprd02.prod.outlook.com>

Hi Coleen,

thanks for implementing it for all platforms.

PPC64: looks good.

s390: The tm instruction produces a special condition code. z_brnaz should be used (please don't omit the 'a').

Tested interpreter safepoint polling by:
jdk/bin/java -Xint -XX:-UseBiasedLocking -XX:+SafepointTimeout -XX:+SafepointALot -XX:+AbortVMOnSafepointTimeout -XX:SafepointTimeoutDelay=300 -XX:GuaranteedSafepointInterval=300 TestLoop

(With some long running TestLoop.)

This test crashed with z_brnz as expected, but worked with z_brnaz.

Thanks and best regards,
Martin


> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
> coleen.phillimore at oracle.com
> Sent: Montag, 13. April 2020 16:34
> To: hotspot-dev developers <hotspot-dev at openjdk.java.net>;
> serviceability-dev at openjdk.java.net
> Subject: RFR (S) 8074292: nsk/jdb/kill/kill001: generateOopMap.cpp
> assert(bb->is_reachable()) failed
> 
> Summary: Do not install async exceptions at_safepoint for each bytecode.
> 
> See CR for a lot more details.? This change calls a new
> InterpreterRuntime::at_safepoint_async_safe() which installs the async
> exception in the interpreter at backward branches and returns.? This
> uses safepoint polling code in the interpreter for each platform.? These
> changes (cross) compile on platforms that Oracle doesn't support but I
> don't know if they work.
> 
> I'm not convinced the platform specific changes are necessary, because
> calls to the runtime from many bytecodes will install the async
> exception, so it's essentially installed "enough" for this deprecated
> feature.? I tested the changes with *and* without the platform specific
> changes with no failure, which included the jdb, jdi and jvmti
> serviceability tests.
> 
> This change also makes InterpreterRuntime::monitorexit a JRT_LEAF
> bytecode. The code to check for exceptions is outside the runtime call.
> I ran the JCK vm and lang tests on this change with no failure.
> 
> Tested with tier1-6.
> 
> open webrev at
> http://cr.openjdk.java.net/~coleenp/2020/8074292.01/webrev
> bug link https://bugs.openjdk.java.net/browse/JDK-8074292
> 
> Thanks,
> Coleen


From coleen.phillimore at oracle.com  Mon Apr 13 18:42:35 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Mon, 13 Apr 2020 14:42:35 -0400
Subject: RFR (S) 8074292: nsk/jdb/kill/kill001: generateOopMap.cpp
 assert(bb->is_reachable()) failed
In-Reply-To: <AM0PR0202MB329791E930A720FE320D13A99ADD0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
References: <f66b8de6-d9db-a90f-bfc2-5bf42a19cada@oracle.com>
 <AM0PR0202MB329791E930A720FE320D13A99ADD0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
Message-ID: <bb63abb2-831c-ca02-7412-289160fed0bf@oracle.com>


Hi Martin,

Thank you for testing this, the correction, and the code review.

Coleen

On 4/13/20 2:32 PM, Doerr, Martin wrote:
> Hi Coleen,
>
> thanks for implementing it for all platforms.
>
> PPC64: looks good.
>
> s390: The tm instruction produces a special condition code. z_brnaz should be used (please don't omit the 'a').
>
> Tested interpreter safepoint polling by:
> jdk/bin/java -Xint -XX:-UseBiasedLocking -XX:+SafepointTimeout -XX:+SafepointALot -XX:+AbortVMOnSafepointTimeout -XX:SafepointTimeoutDelay=300 -XX:GuaranteedSafepointInterval=300 TestLoop
>
> (With some long running TestLoop.)
>
> This test crashed with z_brnz as expected, but worked with z_brnaz.
>
> Thanks and best regards,
> Martin
>
>
>> -----Original Message-----
>> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
>> coleen.phillimore at oracle.com
>> Sent: Montag, 13. April 2020 16:34
>> To: hotspot-dev developers <hotspot-dev at openjdk.java.net>;
>> serviceability-dev at openjdk.java.net
>> Subject: RFR (S) 8074292: nsk/jdb/kill/kill001: generateOopMap.cpp
>> assert(bb->is_reachable()) failed
>>
>> Summary: Do not install async exceptions at_safepoint for each bytecode.
>>
>> See CR for a lot more details.? This change calls a new
>> InterpreterRuntime::at_safepoint_async_safe() which installs the async
>> exception in the interpreter at backward branches and returns.? This
>> uses safepoint polling code in the interpreter for each platform.? These
>> changes (cross) compile on platforms that Oracle doesn't support but I
>> don't know if they work.
>>
>> I'm not convinced the platform specific changes are necessary, because
>> calls to the runtime from many bytecodes will install the async
>> exception, so it's essentially installed "enough" for this deprecated
>> feature.? I tested the changes with *and* without the platform specific
>> changes with no failure, which included the jdb, jdi and jvmti
>> serviceability tests.
>>
>> This change also makes InterpreterRuntime::monitorexit a JRT_LEAF
>> bytecode. The code to check for exceptions is outside the runtime call.
>> I ran the JCK vm and lang tests on this change with no failure.
>>
>> Tested with tier1-6.
>>
>> open webrev at
>> http://cr.openjdk.java.net/~coleenp/2020/8074292.01/webrev
>> bug link https://bugs.openjdk.java.net/browse/JDK-8074292
>>
>> Thanks,
>> Coleen


From david.holmes at oracle.com  Tue Apr 14 02:49:37 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 14 Apr 2020 12:49:37 +1000
Subject: RFR (S) 8074292: nsk/jdb/kill/kill001: generateOopMap.cpp
 assert(bb->is_reachable()) failed
In-Reply-To: <f66b8de6-d9db-a90f-bfc2-5bf42a19cada@oracle.com>
References: <f66b8de6-d9db-a90f-bfc2-5bf42a19cada@oracle.com>
Message-ID: <6e38070c-f166-a0c2-8b8d-d698e5f5bcf3@oracle.com>

Hi Coleen,

On 14/04/2020 12:34 am, coleen.phillimore at oracle.com wrote:
> Summary: Do not install async exceptions at_safepoint for each bytecode.

I'm still not certain that we have to go this far to solve this problem, 
but it does sound like a relatively simple solution provided there are 
no unintended consequences.

> See CR for a lot more details.? This change calls a new 
> InterpreterRuntime::at_safepoint_async_safe() which installs the async 
> exception in the interpreter at backward branches and returns.? This 
> uses safepoint polling code in the interpreter for each platform.? These 
> changes (cross) compile on platforms that Oracle doesn't support but I 
> don't know if they work.
> 
> I'm not convinced the platform specific changes are necessary, because 
> calls to the runtime from many bytecodes will install the async 
> exception, so it's essentially installed "enough" for this deprecated 
> feature.? I tested the changes with *and* without the platform specific 
> changes with no failure, which included the jdb, jdi and jvmti 
> serviceability tests.

I don't understand what you mean here. If the whole basis of this fix is 
"don't install async exceptions other than at backward branches and 
returns" then how is that implemented without the changes in the 
interpreter code?

If this can be fixed just by adjusting the actual monitor code then I 
would much prefer that. It took me a while to get my head around the 
dispatch changes in interpreter code and even then I don't see how those 
changes only impact backward branches and returns ??

> 
> This change also makes InterpreterRuntime::monitorexit a JRT_LEAF 
> bytecode. The code to check for exceptions is outside the runtime call. 
> I ran the JCK vm and lang tests on this change with no failure.
> 
> Tested with tier1-6.
> 
> open webrev at http://cr.openjdk.java.net/~coleenp/2020/8074292.01/webrev
> bug link https://bugs.openjdk.java.net/browse/JDK-8074292

./cpu/x86/interp_masm_x86.cpp

It took me a long time to figure out how the new logic worked compared 
to the old logic. Even then I'm unclear about the effective recursive 
dispatch path: dispatch_base(generate_poll=true) -> dispatch_via -> 
dispatch_base(generate_poll=false) - does it work okay with 
VerifyActivationFrameSize? It seems a rather convoluted way to 
effectively just execute:

  858   lea(rscratch1, ExternalAddress((address)table));
  859   jmp(Address(rscratch1, rbx, Address::times_8));

---

src/hotspot/share/interpreter/interpreterRuntime.cpp

How were you able to drop this code:

791   if (elem == NULL || h_obj()->is_unlocked()) {
792     THROW(vmSymbols::java_lang_IllegalMonitorStateException());
793   }

?
  and this:

798 #ifdef ASSERT
799   thread->last_frame().interpreter_frame_verify_monitor(elem);
800 #endif

?

Thanks,
David

> Thanks,
> Coleen
> 

From denghui.ddh at alibaba-inc.com  Tue Apr 14 05:28:53 2020
From: denghui.ddh at alibaba-inc.com (Denghui Dong)
Date: Tue, 14 Apr 2020 13:28:53 +0800
Subject: =?UTF-8?B?UmU6IFJGUihYWFMpOiA4MjQyNDg1OiBOdWxsIF9maWxlIGNoZWNraW5nIGluIGZpbGVTdHJl?=
 =?UTF-8?B?YW06OmZsdXNoKCk=?=
In-Reply-To: <25546944-b9e7-bca8-aff3-9cd90c443dcd@oracle.com>
References: <e8d66934-57b9-44e5-a1b2-c650ea101220.denghui.ddh@alibaba-inc.com>
 <a8bd28fd-619a-d974-621d-45873edc7fa0@oss.nttdata.com>
 <a5758e50-0d65-4c90-8512-c41415b7aed7.denghui.ddh@alibaba-inc.com>
 <d7f0d3b2-1fc7-3601-b651-a4bdb0244d23@oracle.com>
 <d483f6ff-c51e-4eed-b11d-d7e8e427d8e4.denghui.ddh@alibaba-inc.com>
 <618b239a-2a21-a56f-021a-74df5e8d433c@oracle.com>
 <f86225d5-993a-40bf-98d1-fdb47e113877.denghui.ddh@alibaba-inc.com>,
 <25546944-b9e7-bca8-aff3-9cd90c443dcd@oracle.com>
Message-ID: <09f66dc8-e613-4aad-8c2c-4d77c45f33b1.denghui.ddh@alibaba-inc.com>

Hi David,

After discussing with Yasumasa,  I decided to put update_position into if-statement,
I also modified fdStream::write, because it obviously has the same problem.

Could you review it again?
still at http://cr.openjdk.java.net/~ddong/8242485/webrev.02

Thanks,
Denghui Dong


------------------------------------------------------------------
From:David Holmes <david.holmes at oracle.com>
Send Time:2020?4?13?(???) 06:23
To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net <hotspot-runtime-dev at openjdk.java.net>; Yasumasa Suenaga <suenaga at oss.nttdata.com>; "Liu, Xin" <xxinliu at amazon.com>
Subject:Re: RFR(XXS): 8242485: Null _file checking in fileStream::flush()

On 12/04/2020 7:01 pm, Denghui Dong wrote:
> Hi David,
> 
> Thanks for the comments!
> 
>> Unfortunately the more I look at this code the more problems I see with 
>> it. :( We have:
>> 
>> void fileStream::write(const char* s, size_t len) {
>>    if (_file != NULL)  {
>>      // Make an unused local variable to avoid warning from gcc compiler.
>>      size_t count = fwrite(s, 1, len, _file);
>>    }
>>    update_position(s, len);
>> }
>> 
>> but the update_position should be inside the if-statement - no need to 
>> see a new webrev for that.
> 
> update_position is also used in fdStream::write? 
> bufferedStream::write?and stringStream::write,
> and according to the comments in stringStream::write
> 
> =====
>    // Note that the following does not depend on write_len.
>    // This means that position and count get updated
>    // even when overflow occurs.
>    update_position(s, len);
> =====
> 
> If my understanding is correct, whether or not to call update_position 
> does not depend on the success of the write action.

Every fileStream operation should be a no-op if _file is NULL. Whether 
other streams have issues is a different matter. It makes absolutely no 
sense to update any state of the filestream in write if nothing is ever 
going to examine it anyway - just a waste of time.

Le's see if there are any other opinions on this.

Thanks,
David
-----

> If there's a problem here, we should also check the usage of 
> update_position in other methods to ensure consistent usage.
> 
> Hence,  I think we shouldn't put update_position inside the 
> if-statement, at least for the current issue.
> 
> 
> 
> 
>     ------------------------------------------------------------------
>     From:David Holmes <david.holmes at oracle.com>
>     Send Time:2020?4?12?(???) 15:04
>     To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev
>     <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>     <hotspot-runtime-dev at openjdk.java.net>; Yasumasa Suenaga
>     <suenaga at oss.nttdata.com>; "Liu, Xin" <xxinliu at amazon.com>
>     Subject:Re: RFR(XXS): 8242485: Null _file checking in
>     fileStream::flush()
> 
>     Hi Denghui,
> 
>     On 12/04/2020 2:41 pm, Denghui Dong wrote:
>      > Hi David,
>      >
>      > Thanks for the comments
>      > new webrev: http://cr.openjdk.java.net/~ddong/8242485/webrev.02
>      >
>      > when _file is NULL:
>      > - fileStream::read return 0
>      > - fileStream::readln return NULL
>      > - fileStream::eof return -1 (like
>      > fileStream::fileSize, not sure if appropriate, feof returns nonzero if target file is end)
> 
>     That is all great - thanks!
> 
>     Unfortunately the more I look at this code the more problems I see with
>     it. :( We have:
> 
>     void fileStream::write(const char* s, size_t len) {
>         if (_file != NULL)  {
>           // Make an unused local variable to avoid warning from gcc compiler.
>           size_t count = fwrite(s, 1, len, _file);
>         }
>         update_position(s, len);
>     }
> 
>     but the update_position should be inside the if-statement - no need to
>     see a new webrev for that.
> 
>     We should also be watching for errors from fwrite - but that in itself
>     just opens up a can-of-worms with regard to error handling in general
>     with this class, so that can be left as-is.
> 
>     Thanks,
>     David
>     -----
> 
>      > Testing: tier1
>      >
>      > Cheers,
>      > Denghui Dong
>      >
>      >     ------------------------------------------------------------------
>      >     From:David Holmes <david.holmes at oracle.com>
>      >     Send Time:2020?4?12?(???) 10:40
>      >     To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev
>      >     <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>      >     <hotspot-runtime-dev at openjdk.java.net>; Yasumasa Suenaga
>      >     <suenaga at oss.nttdata.com>; "Liu, Xin" <xxinliu at amazon.com>
>      >     Subject:Re: RFR(XXS): 8242485: Null _file checking in
>      >     fileStream::flush()
>      >
>      >     Hi,
>      >
>      >     On 12/04/2020 1:52 am, Denghui Dong wrote:
>      >      > Hi,
>      >      > Thank you for the review and point out other methods need to check _file.
>      >      > In addition to "fileStream::flush", I found there are still four methods need to check _file in fileStream: read, readln, eof, rewind.
>      >      > but those methods will crash if _file is NULL, and I didn't find any caller of them(Maybe I am wrong),
>      >      > so I think it's more appropriate to add assert check to those methods.
>      >
>      >     Assertions are really only appropriate if they will be exercised by
>      >     tests - the intent being that tests will expose any cases where the
>      >     assertion fails.
>      >
>      >     Also it doesn't make sense to me to have assertions in most places but a
>      >
>      >     true NULL check in others - that suggests we expect some methods to be
>      >     called when _file is NULL, but others not. I'd prefer just to see the
>      >     missing NULL checks added, rather than using assertions.
>      >
>      >     Thanks,
>      >     David
>      >
>      >      > Webrev: http://cr.openjdk.java.net/~ddong/8242485/webrev.01/
>      >      >
>      >      > Could you review it again, and sponsor it if everything is okay?
>      >      >
>      >      > Testing: teir1
>      >      > ==============================
>      >      > Test summary
>      >      > ==============================
>      >      >     TEST                                              TOTAL  PASS  FAIL ERROR
>      >      >     jtreg:test/hotspot/jtreg:tier1                     1516  1516     0     0
>      >      >     jtreg:test/jdk:tier1                               1904  1904     0     0
>      >      >  >> jtreg:test/langtools:tier1                         4031  4029     2     0 <<
>      >      >     jtreg:test/nashorn:tier1                              0     0     0     0
>      >      >     jtreg:test/jaxp:tier1                                 0     0     0     0
>      >      > ==============================
>      >      > There are two unrelated test failures in
>      >      > langtools: jdk/javadoc/tool/CheckResourceKeys.java  and tools/javac/processing/model/TestSymtabItems.java
>      >      >
>      >      >
>      >      >
>      >      >     ------------------------------------------------------------------
>      >      >     From:Yasumasa Suenaga <suenaga at oss.nttdata.com>
>      >      >     Send Time:2020?4?11?(???) 09:22
>      >      >     To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev
>      >      >     <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>      >      >     <hotspot-runtime-dev at openjdk.java.net>
>      >      >     Subject:Re: RFR(XXS): 8242485: Null _file checking in
>      >      >     fileStream::flush()
>      >      >
>      >      >     Hi Denghui,
>      >      >
>      >      >     null check lacks in other place too.
>      >      >     Can you fix it?
>      >      >
>      >      >         http://hg.openjdk.java.net/jdk/jdk/file/97d5d0cd1085/src/hotspot/share/utilities/ostream.cpp#l554
>      >      >
>      >      >
>      >      >     Thanks,
>      >      >
>      >      >     Yasumasa
>      >      >
>      >      >
>      >      >     On 2020/04/10 21:34, Denghui Dong wrote:
>      >      >      > Hi team,
>      >      >      >
>      >      >      > Could you please review this small patch?
>      >      >      >
>      >      >      > JBS: https://bugs.openjdk.java.net/browse/JDK-8242485
>      >      >      > webrev: http://cr.openjdk.java.net/~ddong/8242485/webrev.00/
>      >      >      >
>      >      >      > I found the method "fileStream::flush()" lacks null check before fflush  which will cause
>      >      >      > all open output streams are flushed by fflush() if _file is NULL.
>      >      >      >
>      >      >      > Thanks,
>      >      >      > Denghui Dong
>      >      >      >
>      >      >
>      > 
> 


From david.holmes at oracle.com  Tue Apr 14 05:54:35 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 14 Apr 2020 15:54:35 +1000
Subject: RFR(XXS): 8242485: Null _file checking in fileStream::flush()
In-Reply-To: <09f66dc8-e613-4aad-8c2c-4d77c45f33b1.denghui.ddh@alibaba-inc.com>
References: <e8d66934-57b9-44e5-a1b2-c650ea101220.denghui.ddh@alibaba-inc.com>
 <a8bd28fd-619a-d974-621d-45873edc7fa0@oss.nttdata.com>
 <a5758e50-0d65-4c90-8512-c41415b7aed7.denghui.ddh@alibaba-inc.com>
 <d7f0d3b2-1fc7-3601-b651-a4bdb0244d23@oracle.com>
 <d483f6ff-c51e-4eed-b11d-d7e8e427d8e4.denghui.ddh@alibaba-inc.com>
 <618b239a-2a21-a56f-021a-74df5e8d433c@oracle.com>
 <f86225d5-993a-40bf-98d1-fdb47e113877.denghui.ddh@alibaba-inc.com>
 <25546944-b9e7-bca8-aff3-9cd90c443dcd@oracle.com>
 <09f66dc8-e613-4aad-8c2c-4d77c45f33b1.denghui.ddh@alibaba-inc.com>
Message-ID: <ebeb9566-bd3e-3a7b-4c1c-2e89c9f1c571@oracle.com>

Hi Denghui,

This looks fine to me.

Thanks!

David

On 14/04/2020 3:28 pm, Denghui Dong wrote:
> Hi David,
> 
> After discussing with?Yasumasa,? I decided to put?update_position into 
> if-statement,
> I also modified fdStream::write, because it obviously has the same problem.
> 
> Could you review it again?
> still at http://cr.openjdk.java.net/~ddong/8242485/webrev.02
> 
> Thanks,
> Denghui Dong
> 
>     ------------------------------------------------------------------
>     From:David Holmes <david.holmes at oracle.com>
>     Send Time:2020?4?13?(???) 06:23
>     To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev
>     <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>     <hotspot-runtime-dev at openjdk.java.net>; Yasumasa Suenaga
>     <suenaga at oss.nttdata.com>; "Liu, Xin" <xxinliu at amazon.com>
>     Subject:Re: RFR(XXS): 8242485: Null _file checking in
>     fileStream::flush()
> 
>     On?12/04/2020?7:01?pm,?Denghui?Dong?wrote:
>      >?Hi?David,
>      >
>      >?Thanks?for?the?comments!
>      >
>      >>?Unfortunately?the?more?I?look?at?this?code?the?more?problems?I?see?with
>      >>?it.?:(?We?have:
>      >>
>      >>?void?fileStream::write(const?char*?s,?size_t?len)?{
>      >>????if?(_file?!=?NULL)??{
>      >>??????//?Make?an?unused?local?variable?to?avoid?warning?from?gcc?compiler.
>      >>??????size_t?count?=?fwrite(s,?1,?len,?_file);
>      >>????}
>      >>????update_position(s,?len);
>      >>?}
>      >>
>      >>?but?the?update_position?should?be?inside?the?if-statement?-?no?need?to
>      >>?see?a?new?webrev?for?that.
>      >
>      >?update_position?is?also?used?in?fdStream::write?
>      >?bufferedStream::write?and?stringStream::write,
>      >?and?according?to?the?comments?in?stringStream::write
>      >
>      >?=====
>      >????//?Note?that?the?following?does?not?depend?on?write_len.
>      >????//?This?means?that?position?and?count?get?updated
>      >????//?even?when?overflow?occurs.
>      >????update_position(s,?len);
>      >?=====
>      >
>      >?If?my?understanding?is?correct,?whether?or?not?to?call?update_position
>      >?does?not?depend?on?the?success?of?the?write?action.
> 
>     Every?fileStream?operation?should?be?a?no-op?if?_file?is?NULL.?Whether
>     other?streams?have?issues?is?a?different?matter.?It?makes?absolutely?no
>     sense?to?update?any?state?of?the?filestream?in?write?if?nothing?is?ever
>     going?to?examine?it?anyway?-?just?a?waste?of?time.
> 
>     Le's?see?if?there?are?any?other?opinions?on?this.
> 
>     Thanks,
>     David
>     -----
> 
>      >?If?there's?a?problem?here,?we?should?also?check?the?usage?of
>      >?update_position?in?other?methods?to?ensure?consistent?usage.
>      >
>      >?Hence,??I?think?we?shouldn't?put?update_position?inside?the
>      >?if-statement,?at?least?for?the?current?issue.
>      >
>      >
>      >
>      >
>      >?????------------------------------------------------------------------
>      >?????From:David?Holmes?<david.holmes at oracle.com>
>      >?????Send?Time:2020?4?12?(???)?15:04
>      >?????To:???(??)?<denghui.ddh at alibaba-inc.com>;?hotspot-dev
>      >?????<hotspot-dev at openjdk.java.net>;?hotspot-runtime-dev at openjdk.java.net
>      >?????<hotspot-runtime-dev at openjdk.java.net>;?Yasumasa?Suenaga
>      >?????<suenaga at oss.nttdata.com>;?"Liu,?Xin"?<xxinliu at amazon.com>
>      >?????Subject:Re:?RFR(XXS):?8242485:?Null?_file?checking?in
>      >?????fileStream::flush()
>      >
>      >?????Hi?Denghui,
>      >
>      >?????On?12/04/2020?2:41?pm,?Denghui?Dong?wrote:
>      >??????>?Hi?David,
>      >??????>
>      >??????>?Thanks?for?the?comments
>      >??????>?new?webrev:?http://cr.openjdk.java.net/~ddong/8242485/webrev.02
>      >??????>
>      >??????>?when?_file?is?NULL:
>      >??????>?-?fileStream::read?return?0
>      >??????>?-?fileStream::readln?return?NULL
>      >??????>?-?fileStream::eof?return?-1?(like
>      >??????>?fileStream::fileSize,?not?sure?if?appropriate,?feof?returns?nonzero?if?target?file?is?end)
>      >
>      >?????That?is?all?great?-?thanks!
>      >
>      >?????Unfortunately?the?more?I?look?at?this?code?the?more?problems?I?see?with
>      >?????it.?:(?We?have:
>      >
>      >?????void?fileStream::write(const?char*?s,?size_t?len)?{
>      >?????????if?(_file?!=?NULL)??{
>      >???????????//?Make?an?unused?local?variable?to?avoid?warning?from?gcc?compiler.
>      >???????????size_t?count?=?fwrite(s,?1,?len,?_file);
>      >?????????}
>      >?????????update_position(s,?len);
>      >?????}
>      >
>      >?????but?the?update_position?should?be?inside?the?if-statement?-?no?need?to
>      >?????see?a?new?webrev?for?that.
>      >
>      >?????We?should?also?be?watching?for?errors?from?fwrite?-?but?that?in?itself
>      >?????just?opens?up?a?can-of-worms?with?regard?to?error?handling?in?general
>      >?????with?this?class,?so?that?can?be?left?as-is.
>      >
>      >?????Thanks,
>      >?????David
>      >?????-----
>      >
>      >??????>?Testing:?tier1
>      >??????>
>      >??????>?Cheers,
>      >??????>?Denghui?Dong
>      >??????>
>      >??????>?????------------------------------------------------------------------
>      >??????>?????From:David?Holmes?<david.holmes at oracle.com>
>      >??????>?????Send?Time:2020?4?12?(???)?10:40
>      >??????>?????To:???(??)?<denghui.ddh at alibaba-inc.com>;?hotspot-dev
>      >??????>?????<hotspot-dev at openjdk.java.net>;?hotspot-runtime-dev at openjdk.java.net
>      >??????>?????<hotspot-runtime-dev at openjdk.java.net>;?Yasumasa?Suenaga
>      >??????>?????<suenaga at oss.nttdata.com>;?"Liu,?Xin"?<xxinliu at amazon.com>
>      >??????>?????Subject:Re:?RFR(XXS):?8242485:?Null?_file?checking?in
>      >??????>?????fileStream::flush()
>      >??????>
>      >??????>?????Hi,
>      >??????>
>      >??????>?????On?12/04/2020?1:52?am,?Denghui?Dong?wrote:
>      >??????>??????>?Hi,
>      >??????>??????>?Thank?you?for?the?review?and?point?out?other?methods?need?to?check?_file.
>      >??????>??????>?In?addition?to?"fileStream::flush",?I?found?there?are?still?four?methods?need?to?check?_file?in?fileStream:?read,?readln,?eof,?rewind.
>      >??????>??????>?but?those?methods?will?crash?if?_file?is?NULL,?and?I?didn't?find?any?caller?of?them(Maybe?I?am?wrong),
>      >??????>??????>?so?I?think?it's?more?appropriate?to?add?assert?check?to?those?methods.
>      >??????>
>      >??????>?????Assertions?are?really?only?appropriate?if?they?will?be?exercised?by
>      >??????>?????tests?-?the?intent?being?that?tests?will?expose?any?cases?where?the
>      >??????>?????assertion?fails.
>      >??????>
>      >??????>?????Also?it?doesn't?make?sense?to?me?to?have?assertions?in?most?places?but?a
>      >??????>
>      >??????>?????true?NULL?check?in?others?-?that?suggests?we?expect?some?methods?to?be
>      >??????>?????called?when?_file?is?NULL,?but?others?not.?I'd?prefer?just?to?see?the
>      >??????>?????missing?NULL?checks?added,?rather?than?using?assertions.
>      >??????>
>      >??????>?????Thanks,
>      >??????>?????David
>      >??????>
>      >??????>??????>?Webrev:?http://cr.openjdk.java.net/~ddong/8242485/webrev.01/
>      >??????>??????>
>      >??????>??????>?Could?you?review?it?again,?and?sponsor?it?if?everything?is?okay?
>      >??????>??????>
>      >??????>??????>?Testing:?teir1
>      >??????>??????>?==============================
>      >??????>??????>?Test?summary
>      >??????>??????>?==============================
>      >??????>??????>?????TEST??????????????????????????????????????????????TOTAL??PASS??FAIL?ERROR
>      >??????>??????>?????jtreg:test/hotspot/jtreg:tier1?????????????????????1516??1516?????0?????0
>      >??????>??????>?????jtreg:test/jdk:tier1???????????????????????????????1904??1904?????0?????0
>      >??????>??????>??>>?jtreg:test/langtools:tier1?????????????????????????4031??4029?????2?????0?<<
>      >??????>??????>?????jtreg:test/nashorn:tier1??????????????????????????????0?????0?????0?????0
>      >??????>??????>?????jtreg:test/jaxp:tier1?????????????????????????????????0?????0?????0?????0
>      >??????>??????>?==============================
>      >??????>??????>?There?are?two?unrelated?test?failures?in
>      >??????>??????>?langtools:?jdk/javadoc/tool/CheckResourceKeys.java??and?tools/javac/processing/model/TestSymtabItems.java
>      >??????>??????>
>      >??????>??????>
>      >??????>??????>
>      >??????>??????>?????------------------------------------------------------------------
>      >??????>??????>?????From:Yasumasa?Suenaga?<suenaga at oss.nttdata.com>
>      >??????>??????>?????Send?Time:2020?4?11?(???)?09:22
>      >??????>??????>?????To:???(??)?<denghui.ddh at alibaba-
>     inc.com>;?hotspot-dev
>      >??????>??????>?????<hotspot-dev at openjdk.java.net>;?hotspot-runtime-dev at openjdk.java.net
>      >??????>??????>?????<hotspot-runtime-dev at openjdk.java.net>
>      >??????>??????>?????Subject:Re:?RFR(XXS):?8242485:?Null?_file?checking?in
>      >??????>??????>?????fileStream::flush()
>      >??????>??????>
>      >??????>??????>?????Hi?Denghui,
>      >??????>??????>
>      >??????>??????>?????null?check?lacks?in?other?place?too.
>      >??????>??????>?????Can?you?fix?it?
>      >??????>??????>
>      >??????>??????>?????????http://hg.openjdk.java.net/jdk/jdk/file/97d5d0cd1085/src/hotspot/share/utilities/ostream.cpp#l554
>      >??????>??????>
>      >??????>??????>
>      >??????>??????>?????Thanks,
>      >??????>??????>
>      >??????>??????>?????Yasumasa
>      >??????>??????>
>      >??????>??????>
>      >??????>??????>?????On?2020/04/10?21:34,?Denghui?Dong?wrote:
>      >??????>??????>??????>?Hi?team,
>      >??????>??????>??????>
>      >??????>??????>??????>?Could?you?please?review?this?small?patch?
>      >??????>??????>??????>
>      >??????>??????>??????>?JBS:?https://bugs.openjdk.java.net/browse/JDK-8242485
>      >??????>??????>??????>?webrev:?http://cr.openjdk.java.net/~ddong/8242485/webrev.00/
>      >??????>??????>??????>
>      >??????>??????>??????>?I?found?the?method?"fileStream::flush()"?lacks?null?check?before?fflush??which?will?cause
>      >??????>??????>??????>?all?open?output?streams?are?flushed?by?fflush()?if?_file?is?NULL.
>      >??????>??????>??????>
>      >??????>??????>??????>?Thanks,
>      >??????>??????>??????>?Denghui?Dong
>      >??????>??????>??????>
>      >??????>??????>
>      >??????>
>      > 
> 
> 

From denghui.ddh at alibaba-inc.com  Tue Apr 14 06:04:34 2020
From: denghui.ddh at alibaba-inc.com (Denghui Dong)
Date: Tue, 14 Apr 2020 14:04:34 +0800
Subject: =?UTF-8?B?UmU6IFJGUihYWFMpOiA4MjQyNDg1OiBOdWxsIF9maWxlIGNoZWNraW5nIGluIGZpbGVTdHJl?=
 =?UTF-8?B?YW06OmZsdXNoKCk=?=
In-Reply-To: <ebeb9566-bd3e-3a7b-4c1c-2e89c9f1c571@oracle.com>
References: <e8d66934-57b9-44e5-a1b2-c650ea101220.denghui.ddh@alibaba-inc.com>
 <a8bd28fd-619a-d974-621d-45873edc7fa0@oss.nttdata.com>
 <a5758e50-0d65-4c90-8512-c41415b7aed7.denghui.ddh@alibaba-inc.com>
 <d7f0d3b2-1fc7-3601-b651-a4bdb0244d23@oracle.com>
 <d483f6ff-c51e-4eed-b11d-d7e8e427d8e4.denghui.ddh@alibaba-inc.com>
 <618b239a-2a21-a56f-021a-74df5e8d433c@oracle.com>
 <f86225d5-993a-40bf-98d1-fdb47e113877.denghui.ddh@alibaba-inc.com>
 <25546944-b9e7-bca8-aff3-9cd90c443dcd@oracle.com>
 <09f66dc8-e613-4aad-8c2c-4d77c45f33b1.denghui.ddh@alibaba-inc.com>,
 <ebeb9566-bd3e-3a7b-4c1c-2e89c9f1c571@oracle.com>
Message-ID: <f499226c-0bc5-4518-b757-58eed29876bb.denghui.ddh@alibaba-inc.com>

Thanks!

Could you sponsor it?
------------------------------------------------------------------
From:David Holmes <david.holmes at oracle.com>
Send Time:2020?4?14?(???) 13:54
To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net <hotspot-runtime-dev at openjdk.java.net>; Yasumasa Suenaga <suenaga at oss.nttdata.com>; "Liu, Xin" <xxinliu at amazon.com>
Subject:Re: RFR(XXS): 8242485: Null _file checking in fileStream::flush()

Hi Denghui,

This looks fine to me.

Thanks!

David

On 14/04/2020 3:28 pm, Denghui Dong wrote:
> Hi David,
> 
> After discussing with Yasumasa,  I decided to put update_position into 
> if-statement,
> I also modified fdStream::write, because it obviously has the same problem.
> 
> Could you review it again?
> still at http://cr.openjdk.java.net/~ddong/8242485/webrev.02
> 
> Thanks,
> Denghui Dong
> 
>     ------------------------------------------------------------------
>     From:David Holmes <david.holmes at oracle.com>
>     Send Time:2020?4?13?(???) 06:23
>     To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev
>     <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>     <hotspot-runtime-dev at openjdk.java.net>; Yasumasa Suenaga
>     <suenaga at oss.nttdata.com>; "Liu, Xin" <xxinliu at amazon.com>
>     Subject:Re: RFR(XXS): 8242485: Null _file checking in
>     fileStream::flush()
> 
>     On 12/04/2020 7:01 pm, Denghui Dong wrote:
>      > Hi David,
>      >
>      > Thanks for the comments!
>      >
>      >> Unfortunately the more I look at this code the more problems I see with
>      >> it. :( We have:
>      >>
>      >> void fileStream::write(const char* s, size_t len) {
>      >>    if (_file != NULL)  {
>      >>      // Make an unused local variable to avoid warning from gcc compiler.
>      >>      size_t count = fwrite(s, 1, len, _file);
>      >>    }
>      >>    update_position(s, len);
>      >> }
>      >>
>      >> but the update_position should be inside the if-statement - no need to
>      >> see a new webrev for that.
>      >
>      > update_position is also used in fdStream::write?
>      > bufferedStream::write?and stringStream::write,
>      > and according to the comments in stringStream::write
>      >
>      > =====
>      >    // Note that the following does not depend on write_len.
>      >    // This means that position and count get updated
>      >    // even when overflow occurs.
>      >    update_position(s, len);
>      > =====
>      >
>      > If my understanding is correct, whether or not to call update_position
>      > does not depend on the success of the write action.
> 
>     Every fileStream operation should be a no-op if _file is NULL. Whether
>     other streams have issues is a different matter. It makes absolutely no
>     sense to update any state of the filestream in write if nothing is ever
>     going to examine it anyway - just a waste of time.
> 
>     Le's see if there are any other opinions on this.
> 
>     Thanks,
>     David
>     -----
> 
>      > If there's a problem here, we should also check the usage of
>      > update_position in other methods to ensure consistent usage.
>      >
>      > Hence,  I think we shouldn't put update_position inside the
>      > if-statement, at least for the current issue.
>      >
>      >
>      >
>      >
>      >     ------------------------------------------------------------------
>      >     From:David Holmes <david.holmes at oracle.com>
>      >     Send Time:2020?4?12?(???) 15:04
>      >     To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev
>      >     <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>      >     <hotspot-runtime-dev at openjdk.java.net>; Yasumasa Suenaga
>      >     <suenaga at oss.nttdata.com>; "Liu, Xin" <xxinliu at amazon.com>
>      >     Subject:Re: RFR(XXS): 8242485: Null _file checking in
>      >     fileStream::flush()
>      >
>      >     Hi Denghui,
>      >
>      >     On 12/04/2020 2:41 pm, Denghui Dong wrote:
>      >      > Hi David,
>      >      >
>      >      > Thanks for the comments
>      >      > new webrev: http://cr.openjdk.java.net/~ddong/8242485/webrev.02
>      >      >
>      >      > when _file is NULL:
>      >      > - fileStream::read return 0
>      >      > - fileStream::readln return NULL
>      >      > - fileStream::eof return -1 (like
>      >      > fileStream::fileSize, not sure if appropriate, feof returns nonzero if target file is end)
>      >
>      >     That is all great - thanks!
>      >
>      >     Unfortunately the more I look at this code the more problems I see with
>      >     it. :( We have:
>      >
>      >     void fileStream::write(const char* s, size_t len) {
>      >         if (_file != NULL)  {
>      >           // Make an unused local variable to avoid warning from gcc compiler.
>      >           size_t count = fwrite(s, 1, len, _file);
>      >         }
>      >         update_position(s, len);
>      >     }
>      >
>      >     but the update_position should be inside the if-statement - no need to
>      >     see a new webrev for that.
>      >
>      >     We should also be watching for errors from fwrite - but that in itself
>      >     just opens up a can-of-worms with regard to error handling in general
>      >     with this class, so that can be left as-is.
>      >
>      >     Thanks,
>      >     David
>      >     -----
>      >
>      >      > Testing: tier1
>      >      >
>      >      > Cheers,
>      >      > Denghui Dong
>      >      >
>      >      >     ------------------------------------------------------------------
>      >      >     From:David Holmes <david.holmes at oracle.com>
>      >      >     Send Time:2020?4?12?(???) 10:40
>      >      >     To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev
>      >      >     <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>      >      >     <hotspot-runtime-dev at openjdk.java.net>; Yasumasa Suenaga
>      >      >     <suenaga at oss.nttdata.com>; "Liu, Xin" <xxinliu at amazon.com>
>      >      >     Subject:Re: RFR(XXS): 8242485: Null _file checking in
>      >      >     fileStream::flush()
>      >      >
>      >      >     Hi,
>      >      >
>      >      >     On 12/04/2020 1:52 am, Denghui Dong wrote:
>      >      >      > Hi,
>      >      >      > Thank you for the review and point out other methods need to check _file.
>      >      >      > In addition to "fileStream::flush", I found there are still four methods need to check _file in fileStream: read, readln, eof, rewind.
>      >      >      > but those methods will crash if _file is NULL, and I didn't find any caller of them(Maybe I am wrong),
>      >      >      > so I think it's more appropriate to add assert check to those methods.
>      >      >
>      >      >     Assertions are really only appropriate if they will be exercised by
>      >      >     tests - the intent being that tests will expose any cases where the
>      >      >     assertion fails.
>      >      >
>      >      >     Also it doesn't make sense to me to have assertions in most places but a
>      >      >
>      >      >     true NULL check in others - that suggests we expect some methods to be
>      >      >     called when _file is NULL, but others not. I'd prefer just to see the
>      >      >     missing NULL checks added, rather than using assertions.
>      >      >
>      >      >     Thanks,
>      >      >     David
>      >      >
>      >      >      > Webrev: http://cr.openjdk.java.net/~ddong/8242485/webrev.01/
>      >      >      >
>      >      >      > Could you review it again, and sponsor it if everything is okay?
>      >      >      >
>      >      >      > Testing: teir1
>      >      >      > ==============================
>      >      >      > Test summary
>      >      >      > ==============================
>      >      >      >     TEST                                              TOTAL  PASS  FAIL ERROR
>      >      >      >     jtreg:test/hotspot/jtreg:tier1                     1516  1516     0     0
>      >      >      >     jtreg:test/jdk:tier1                               1904  1904     0     0
>      >      >      >  >> jtreg:test/langtools:tier1                         4031  4029     2     0 <<
>      >      >      >     jtreg:test/nashorn:tier1                              0     0     0     0
>      >      >      >     jtreg:test/jaxp:tier1                                 0     0     0     0
>      >      >      > ==============================
>      >      >      > There are two unrelated test failures in
>      >      >      > langtools: jdk/javadoc/tool/CheckResourceKeys.java  and tools/javac/processing/model/TestSymtabItems.java
>      >      >      >
>      >      >      >
>      >      >      >
>      >      >      >     ------------------------------------------------------------------
>      >      >      >     From:Yasumasa Suenaga <suenaga at oss.nttdata.com>
>      >      >      >     Send Time:2020?4?11?(???) 09:22
>      >      >      >     To:???(??) <denghui.ddh at alibaba-
>     inc.com>; hotspot-dev
>      >      >      >     <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>      >      >      >     <hotspot-runtime-dev at openjdk.java.net>
>      >      >      >     Subject:Re: RFR(XXS): 8242485: Null _file checking in
>      >      >      >     fileStream::flush()
>      >      >      >
>      >      >      >     Hi Denghui,
>      >      >      >
>      >      >      >     null check lacks in other place too.
>      >      >      >     Can you fix it?
>      >      >      >
>      >      >      >         http://hg.openjdk.java.net/jdk/jdk/file/97d5d0cd1085/src/hotspot/share/utilities/ostream.cpp#l554
>      >      >      >
>      >      >      >
>      >      >      >     Thanks,
>      >      >      >
>      >      >      >     Yasumasa
>      >      >      >
>      >      >      >
>      >      >      >     On 2020/04/10 21:34, Denghui Dong wrote:
>      >      >      >      > Hi team,
>      >      >      >      >
>      >      >      >      > Could you please review this small patch?
>      >      >      >      >
>      >      >      >      > JBS: https://bugs.openjdk.java.net/browse/JDK-8242485
>      >      >      >      > webrev: http://cr.openjdk.java.net/~ddong/8242485/webrev.00/
>      >      >      >      >
>      >      >      >      > I found the method "fileStream::flush()" lacks null check before fflush  which will cause
>      >      >      >      > all open output streams are flushed by fflush() if _file is NULL.
>      >      >      >      >
>      >      >      >      > Thanks,
>      >      >      >      > Denghui Dong
>      >      >      >      >
>      >      >      >
>      >      >
>      > 
> 
> 

From david.holmes at oracle.com  Tue Apr 14 07:28:09 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 14 Apr 2020 17:28:09 +1000
Subject: RFR(XXS): 8242485: Null _file checking in fileStream::flush()
In-Reply-To: <f499226c-0bc5-4518-b757-58eed29876bb.denghui.ddh@alibaba-inc.com>
References: <e8d66934-57b9-44e5-a1b2-c650ea101220.denghui.ddh@alibaba-inc.com>
 <a8bd28fd-619a-d974-621d-45873edc7fa0@oss.nttdata.com>
 <a5758e50-0d65-4c90-8512-c41415b7aed7.denghui.ddh@alibaba-inc.com>
 <d7f0d3b2-1fc7-3601-b651-a4bdb0244d23@oracle.com>
 <d483f6ff-c51e-4eed-b11d-d7e8e427d8e4.denghui.ddh@alibaba-inc.com>
 <618b239a-2a21-a56f-021a-74df5e8d433c@oracle.com>
 <f86225d5-993a-40bf-98d1-fdb47e113877.denghui.ddh@alibaba-inc.com>
 <25546944-b9e7-bca8-aff3-9cd90c443dcd@oracle.com>
 <09f66dc8-e613-4aad-8c2c-4d77c45f33b1.denghui.ddh@alibaba-inc.com>
 <ebeb9566-bd3e-3a7b-4c1c-2e89c9f1c571@oracle.com>
 <f499226c-0bc5-4518-b757-58eed29876bb.denghui.ddh@alibaba-inc.com>
Message-ID: <adfabe15-25a8-6076-1d06-4a4c4baaff24@oracle.com>

On 14/04/2020 4:04 pm, Denghui Dong wrote:
> Thanks!
> 
> Could you sponsor it?

Sure. Please send me a link to the finalized changeset properly 
formatted with reviewers etc.

Thanks,
David

>     ------------------------------------------------------------------
>     From:David Holmes <david.holmes at oracle.com>
>     Send Time:2020?4?14?(???) 13:54
>     To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev
>     <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>     <hotspot-runtime-dev at openjdk.java.net>; Yasumasa Suenaga
>     <suenaga at oss.nttdata.com>; "Liu, Xin" <xxinliu at amazon.com>
>     Subject:Re: RFR(XXS): 8242485: Null _file checking in
>     fileStream::flush()
> 
>     Hi?Denghui,
> 
>     This?looks?fine?to?me.
> 
>     Thanks!
> 
>     David
> 
>     On?14/04/2020?3:28?pm,?Denghui?Dong?wrote:
>      >?Hi?David,
>      >
>      >?After?discussing?with?Yasumasa,??I?decided?to?put?update_position?into
>      >?if-statement,
>      >?I?also?modified?fdStream::write,?because?it?obviously?has?the?same?problem.
>      >
>      >?Could?you?review?it?again?
>      >?still?at?http://cr.openjdk.java.net/~ddong/8242485/webrev.02
>      >
>      >?Thanks,
>      >?Denghui?Dong
>      >
>      >?????------------------------------------------------------------------
>      >?????From:David?Holmes?<david.holmes at oracle.com>
>      >?????Send?Time:2020?4?13?(???)?06:23
>      >?????To:???(??)?<denghui.ddh at alibaba-inc.com>;?hotspot-dev
>      >?????<hotspot-dev at openjdk.java.net>;?hotspot-runtime-dev at openjdk.java.net
>      >?????<hotspot-runtime-dev at openjdk.java.net>;?Yasumasa?Suenaga
>      >?????<suenaga at oss.nttdata.com>;?"Liu,?Xin"?<xxinliu at amazon.com>
>      >?????Subject:Re:?RFR(XXS):?8242485:?Null?_file?checking?in
>      >?????fileStream::flush()
>      >
>      >?????On?12/04/2020?7:01?pm,?Denghui?Dong?wrote:
>      >??????>?Hi?David,
>      >??????>
>      >??????>?Thanks?for?the?comments!
>      >??????>
>      >??????>>?Unfortunately?the?more?I?look?at?this?code?the?more?problems?I?see?with
>      >??????>>?it.?:(?We?have:
>      >??????>>
>      >??????>>?void?fileStream::write(const?char*?s,?size_t?len)?{
>      >??????>>????if?(_file?!=?NULL)??{
>      >??????>>??????//?Make?an?unused?local?variable?to?avoid?warning?from?gcc?compiler.
>      >??????>>??????size_t?count?=?fwrite(s,?1,?len,?_file);
>      >??????>>????}
>      >??????>>????update_position(s,?len);
>      >??????>>?}
>      >??????>>
>      >??????>>?but?the?update_position?should?be?inside?the?if-statement?-?no?need?to
>      >??????>>?see?a?new?webrev?for?that.
>      >??????>
>      >??????>?update_position?is?also?used?in?fdStream::write?
>      >??????>?bufferedStream::write?and?stringStream::write,
>      >??????>?and?according?to?the?comments?in?stringStream::write
>      >??????>
>      >??????>?=====
>      >??????>????//?Note?that?the?following?does?not?depend?on?write_len.
>      >??????>????//?This?means?that?position?and?count?get?updated
>      >??????>????//?even?when?overflow?occurs.
>      >??????>????update_position(s,?len);
>      >??????>?=====
>      >??????>
>      >??????>?If?my?understanding?is?correct,?whether?or?not?to?call?update_position
>      >??????>?does?not?depend?on?the?success?of?the?write?action.
>      >
>      >?????Every?fileStream?operation?should?be?a?no-op?if?_file?is?NULL.?Whether
>      >?????other?streams?have?issues?is?a?different?matter.?It?makes?absolutely?no
>      >?????sense?to?update?any?state?of?the?filestream?in?write?if?nothing?is?ever
>      >?????going?to?examine?it?anyway?-?just?a?waste?of?time.
>      >
>      >?????Le's?see?if?there?are?any?other?opinions?on?this.
>      >
>      >?????Thanks,
>      >?????David
>      >?????-----
>      >
>      >??????>?If?there's?a?problem?here,?we?should?also?check?the?usage?of
>      >??????>?update_position?in?other?methods?to?ensure?consistent?usage.
>      >??????>
>      >??????>?Hence,??I?think?we?shouldn't?put?update_position?inside?the
>      >??????>?if-statement,?at?least?for?the?current?issue.
>      >??????>
>      >??????>
>      >??????>
>      >??????>
>      >??????>?????------------------------------------------------------------------
>      >??????>?????From:David?Holmes?<david.holmes at oracle.com>
>      >??????>?????Send?Time:2020?4?12?(???)?15:04
>      >??????>?????To:???(??)?<denghui.ddh at alibaba-inc.com>;?hotspot-dev
>      >??????>?????<hotspot-dev at openjdk.java.net>;?hotspot-runtime-dev at openjdk.java.net
>      >??????>?????<hotspot-runtime-dev at openjdk.java.net>;?Yasumasa?Suenaga
>      >??????>?????<suenaga at oss.nttdata.com>;?"Liu,?Xin"?<xxinliu at amazon.com>
>      >??????>?????Subject:Re:?RFR(XXS):?8242485:?Null?_file?checking?in
>      >??????>?????fileStream::flush()
>      >??????>
>      >??????>?????Hi?Denghui,
>      >??????>
>      >??????>?????On?12/04/2020?2:41?pm,?Denghui?Dong?wrote:
>      >??????>??????>?Hi?David,
>      >??????>??????>
>      >??????>??????>?Thanks?for?the?comments
>      >??????>??????>?new?webrev:?http://cr.openjdk.java.net/~ddong/8242485/webrev.02
>      >??????>??????>
>      >??????>??????>?when?_file?is?NULL:
>      >??????>??????>?-?fileStream::read?return?0
>      >??????>??????>?-?fileStream::readln?return?NULL
>      >??????>??????>?-?fileStream::eof?return?-1?(like
>      >??????>??????>?fileStream::fileSize,?not?sure?if?appropriate,?feof?returns?nonzero?if?target?file?is?end)
>      >??????>
>      >??????>?????That?is?all?great?-?thanks!
>      >??????>
>      >??????>?????Unfortunately?the?more?I?look?at?this?code?the?more?problems?I?see?with
>      >??????>?????it.?:(?We?have:
>      >??????>
>      >??????>?????void?fileStream::write(const?char*?s,?size_t?len)?{
>      >??????>?????????if?(_file?!=?NULL)??{
>      >??????>???????????//?Make?an?unused?local?variable?to?avoid?warning?from?gcc?compiler.
>      >??????>???????????size_t?count?=?fwrite(s,?1,?len,?_file);
>      >??????>?????????}
>      >??????>?????????update_position(s,?len);
>      >??????>?????}
>      >??????>
>      >??????>?????but?the?update_position?should?be?inside?the?if-statement?-?no?need?to
>      >??????>?????see?a?new?webrev?for?that.
>      >??????>
>      >??????>?????We?should?also?be?watching?for?errors?from?fwrite?-?but?that?in?itself
>      >??????>?????just?opens?up?a?can-of-worms?with?regard?to?error?handling?in?general
>      >??????>?????with?this?class,?so?that?can?be?left?as-is.
>      >??????>
>      >??????>?????Thanks,
>      >??????>?????David
>      >??????>?????-----
>      >??????>
>      >??????>??????>?Testing:?tier1
>      >??????>??????>
>      >??????>??????>?Cheers,
>      >??????>??????>?Denghui?Dong
>      >??????>??????>
>      >??????>??????>?????------------------------------------------------------------------
>      >??????>??????>?????From:David?Holmes?<david.holmes at oracle.com>
>      >??????>??????>?????Send?Time:2020?4?12?(???)?10:40
>      >??????>??????>?????To:???(??)?<denghui.ddh at alibaba-
>     inc.com>;?hotspot-dev
>      >??????>??????>?????<hotspot-dev at openjdk.java.net>;?hotspot-runtime-dev at openjdk.java.net
>      >??????>??????>?????<hotspot-runtime-dev at openjdk.java.net>;?Yasumasa?Suenaga
>      >??????>??????>?????<suenaga at oss.nttdata.com>;?"Liu,?Xin"?<xxinliu at amazon.com>
>      >??????>??????>?????Subject:Re:?RFR(XXS):?8242485:?Null?_file?checking?in
>      >??????>??????>?????fileStream::flush()
>      >??????>??????>
>      >??????>??????>?????Hi,
>      >??????>??????>
>      >??????>??????>?????On?12/04/2020?1:52?am,?Denghui?Dong?wrote:
>      >??????>??????>??????>?Hi,
>      >??????>??????>??????>?Thank?you?for?the?review?and?point?out?other?methods?need?to?check?_file.
>      >??????>??????>??????>?In?addition?to?"fileStream::flush",?I?found?there?are?still?four?methods?need?to?check?_file?in?fileStream:?read,?readln,?eof,?rewind.
>      >??????>??????>??????>?but?those?methods?will?crash?if?_file?is?NULL,?and?I?didn't?find?any?caller?of?them(Maybe?I?am?wrong),
>      >??????>??????>??????>?so?I?think?it's?more?appropriate?to?add?assert?check?to?those?methods.
>      >??????>??????>
>      >??????>??????>?????Assertions?are?really?only?appropriate?if?they?will?be?exercised?by
>      >??????>??????>?????tests?-?the?intent?being?that?tests?will?expose?any?cases?where?the
>      >??????>??????>?????assertion?fails.
>      >??????>??????>
>      >??????>??????>?????Also?it?doesn't?make?sense?to?me?to?have?assertions?in?most?places?but?a
>      >??????>??????>
>      >??????>??????>?????true?NULL?check?in?others?-?that?suggests?we?expect?some?methods?to?be
>      >??????>??????>?????called?when?_file?is?NULL,?but?others?not.?I'd?prefer?just?to?see?the
>      >??????>??????>?????missing?NULL?checks?added,?rather?than?using?assertions.
>      >??????>??????>
>      >??????>??????>?????Thanks,
>      >??????>??????>?????David
>      >??????>??????>
>      >??????>??????>??????>?Webrev:?http://cr.openjdk.java.net/~ddong/8242485/webrev.01/
>      >??????>??????>??????>
>      >??????>??????>??????>?Could?you?review?it?again,?and?sponsor?it?if?everything?is?okay?
>      >??????>??????>??????>
>      >??????>??????>??????>?Testing:?teir1
>      >??????>??????>??????>?==============================
>      >??????>??????>??????>?Test?summary
>      >??????>??????>??????>?==============================
>      >??????>??????>??????>?????TEST??????????????????????????????????????????????TOTAL??PASS??FAIL?ERROR
>      >??????>??????>??????>?????jtreg:test/hotspot/jtreg:tier1?????????????????????1516??1516?????0?????0
>      >??????>??????>??????>?????jtreg:test/jdk:tier1???????????????????????????????1904??1904?????0?????0
>      >??????>??????>??????>??>>?jtreg:test/langtools:tier1?????????????????????????4031??4029?????2?????0?<<
>      >??????>??????>??????>?????jtreg:test/nashorn:tier1??????????????????????????????0?????0?????0?????0
>      >??????>??????>??????>?????jtreg:test/jaxp:tier1?????????????????????????????????0?????0?????0?????0
>      >??????>??????>??????>?==============================
>      >??????>??????>??????>?There?are?two?unrelated?test?failures?in
>      >??????>??????>??????>?langtools:?jdk/javadoc/tool/CheckResourceKeys.java??and?tools/javac/processing/model/TestSymtabItems.java
>      >??????>??????>??????>
>      >??????>??????>??????>
>      >??????>??????>??????>
>      >??????>??????>??????>?????------------------------------------------------------------------
>      >??????>??????>??????>?????From:Yasumasa?Suenaga?<suenaga at oss.nttdata.com>
>      >??????>??????>??????>?????Send?Time:2020?4?11?(???)?09:22
>      >??????>??????>??????>?????To:???(??)?<denghui.ddh at alibaba-
>      >?????inc.com>;?hotspot-dev
>      >??????>??????>??????>?????<hotspot-dev at openjdk.java.net>;?hotspot-runtime-dev at openjdk.java.net
>      >??????>??????>??????>?????<hotspot-runtime-dev at openjdk.java.net>
>      >??????>??????>??????>?????Subject:Re:?RFR(XXS):?8242485:?Null?_file?checking?in
>      >??????>??????>??????>?????fileStream::flush()
>      >??????>??????>??????>
>      >??????>??????>??????>?????Hi?Denghui,
>      >??????>??????>??????>
>      >??????>??????>??????>?????null?check?lacks?in?other?place?too.
>      >??????>??????>??????>?????Can?you?fix?it?
>      >??????>??????>??????>
>      >??????>??????>??????>?????????http://hg.openjdk.java.net/jdk/jdk/file/97d5d0cd1085/src/hotspot/share/utilities/ostream.cpp#l554
>      >??????>??????>??????>
>      >??????>??????>??????>
>      >??????>??????>??????>?????Thanks,
>      >??????>??????>??????>
>      >??????>??????>??????>?????Yasumasa
>      >??????>??????>??????>
>      >??????>??????>??????>
>      >??????>??????>??????>?????On?2020/04/10?21:34,?Denghui?Dong?wrote:
>      >??????>??????>??????>??????>?Hi?team,
>      >??????>??????>??????>??????>
>      >??????>??????>??????>??????>?Could?you?please?review?this?small?patch?
>      >??????>??????>??????>??????>
>      >??????>??????>??????>??????>?JBS:?https://bugs.openjdk.java.net/browse/JDK-8242485
>      >??????>??????>??????>??????>?webrev:?http://cr.openjdk.java.net/~ddong/8242485/webrev.00/
>      >??????>??????>??????>??????>
>      >??????>??????>??????>??????>?I?found?the?method?"fileStream::flush()"?lacks?null?check?before?fflush??which?will?cause
>      >??????>??????>??????>??????>?all?open?output?streams?are?flushed?by?fflush()?if?_file?is?NULL.
>      >??????>??????>??????>??????>
>      >??????>??????>??????>??????>?Thanks,
>      >??????>??????>??????>??????>?Denghui?Dong
>      >??????>??????>??????>??????>
>      >??????>??????>??????>
>      >??????>??????>
>      >??????>
>      >
>      > 
> 

From denghui.ddh at alibaba-inc.com  Tue Apr 14 08:48:08 2020
From: denghui.ddh at alibaba-inc.com (Denghui Dong)
Date: Tue, 14 Apr 2020 16:48:08 +0800
Subject: =?UTF-8?B?UmU6IFJGUihYWFMpOiA4MjQyNDg1OiBOdWxsIF9maWxlIGNoZWNraW5nIGluIGZpbGVTdHJl?=
 =?UTF-8?B?YW06OmZsdXNoKCk=?=
In-Reply-To: <adfabe15-25a8-6076-1d06-4a4c4baaff24@oracle.com>
References: <e8d66934-57b9-44e5-a1b2-c650ea101220.denghui.ddh@alibaba-inc.com>
 <a8bd28fd-619a-d974-621d-45873edc7fa0@oss.nttdata.com>
 <a5758e50-0d65-4c90-8512-c41415b7aed7.denghui.ddh@alibaba-inc.com>
 <d7f0d3b2-1fc7-3601-b651-a4bdb0244d23@oracle.com>
 <d483f6ff-c51e-4eed-b11d-d7e8e427d8e4.denghui.ddh@alibaba-inc.com>
 <618b239a-2a21-a56f-021a-74df5e8d433c@oracle.com>
 <f86225d5-993a-40bf-98d1-fdb47e113877.denghui.ddh@alibaba-inc.com>
 <25546944-b9e7-bca8-aff3-9cd90c443dcd@oracle.com>
 <09f66dc8-e613-4aad-8c2c-4d77c45f33b1.denghui.ddh@alibaba-inc.com>
 <ebeb9566-bd3e-3a7b-4c1c-2e89c9f1c571@oracle.com>
 <f499226c-0bc5-4518-b757-58eed29876bb.denghui.ddh@alibaba-inc.com>,
 <adfabe15-25a8-6076-1d06-4a4c4baaff24@oracle.com>
Message-ID: <5c187808-6f99-4293-a6f1-6440c2eaa102.denghui.ddh@alibaba-inc.com>

http://cr.openjdk.java.net/~ddong/8242485/webrev.02/

Added review info.

Thanks,
Denghui Dong


------------------------------------------------------------------
From:David Holmes <david.holmes at oracle.com>
Send Time:2020?4?14?(???) 15:28
To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net <hotspot-runtime-dev at openjdk.java.net>; Yasumasa Suenaga <suenaga at oss.nttdata.com>; "Liu, Xin" <xxinliu at amazon.com>
Subject:Re: RFR(XXS): 8242485: Null _file checking in fileStream::flush()

On 14/04/2020 4:04 pm, Denghui Dong wrote:
> Thanks!
> 
> Could you sponsor it?

Sure. Please send me a link to the finalized changeset properly 
formatted with reviewers etc.

Thanks,
David

>     ------------------------------------------------------------------
>     From:David Holmes <david.holmes at oracle.com>
>     Send Time:2020?4?14?(???) 13:54
>     To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev
>     <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>     <hotspot-runtime-dev at openjdk.java.net>; Yasumasa Suenaga
>     <suenaga at oss.nttdata.com>; "Liu, Xin" <xxinliu at amazon.com>
>     Subject:Re: RFR(XXS): 8242485: Null _file checking in
>     fileStream::flush()
> 
>     Hi Denghui,
> 
>     This looks fine to me.
> 
>     Thanks!
> 
>     David
> 
>     On 14/04/2020 3:28 pm, Denghui Dong wrote:
>      > Hi David,
>      >
>      > After discussing with Yasumasa,  I decided to put update_position into
>      > if-statement,
>      > I also modified fdStream::write, because it obviously has the same problem.
>      >
>      > Could you review it again?
>      > still at http://cr.openjdk.java.net/~ddong/8242485/webrev.02
>      >
>      > Thanks,
>      > Denghui Dong
>      >
>      >     ------------------------------------------------------------------
>      >     From:David Holmes <david.holmes at oracle.com>
>      >     Send Time:2020?4?13?(???) 06:23
>      >     To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev
>      >     <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>      >     <hotspot-runtime-dev at openjdk.java.net>; Yasumasa Suenaga
>      >     <suenaga at oss.nttdata.com>; "Liu, Xin" <xxinliu at amazon.com>
>      >     Subject:Re: RFR(XXS): 8242485: Null _file checking in
>      >     fileStream::flush()
>      >
>      >     On 12/04/2020 7:01 pm, Denghui Dong wrote:
>      >      > Hi David,
>      >      >
>      >      > Thanks for the comments!
>      >      >
>      >      >> Unfortunately the more I look at this code the more problems I see with
>      >      >> it. :( We have:
>      >      >>
>      >      >> void fileStream::write(const char* s, size_t len) {
>      >      >>    if (_file != NULL)  {
>      >      >>      // Make an unused local variable to avoid warning from gcc compiler.
>      >      >>      size_t count = fwrite(s, 1, len, _file);
>      >      >>    }
>      >      >>    update_position(s, len);
>      >      >> }
>      >      >>
>      >      >> but the update_position should be inside the if-statement - no need to
>      >      >> see a new webrev for that.
>      >      >
>      >      > update_position is also used in fdStream::write?
>      >      > bufferedStream::write?and stringStream::write,
>      >      > and according to the comments in stringStream::write
>      >      >
>      >      > =====
>      >      >    // Note that the following does not depend on write_len.
>      >      >    // This means that position and count get updated
>      >      >    // even when overflow occurs.
>      >      >    update_position(s, len);
>      >      > =====
>      >      >
>      >      > If my understanding is correct, whether or not to call update_position
>      >      > does not depend on the success of the write action.
>      >
>      >     Every fileStream operation should be a no-op if _file is NULL. Whether
>      >     other streams have issues is a different matter. It makes absolutely no
>      >     sense to update any state of the filestream in write if nothing is ever
>      >     going to examine it anyway - just a waste of time.
>      >
>      >     Le's see if there are any other opinions on this.
>      >
>      >     Thanks,
>      >     David
>      >     -----
>      >
>      >      > If there's a problem here, we should also check the usage of
>      >      > update_position in other methods to ensure consistent usage.
>      >      >
>      >      > Hence,  I think we shouldn't put update_position inside the
>      >      > if-statement, at least for the current issue.
>      >      >
>      >      >
>      >      >
>      >      >
>      >      >     ------------------------------------------------------------------
>      >      >     From:David Holmes <david.holmes at oracle.com>
>      >      >     Send Time:2020?4?12?(???) 15:04
>      >      >     To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev
>      >      >     <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>      >      >     <hotspot-runtime-dev at openjdk.java.net>; Yasumasa Suenaga
>      >      >     <suenaga at oss.nttdata.com>; "Liu, Xin" <xxinliu at amazon.com>
>      >      >     Subject:Re: RFR(XXS): 8242485: Null _file checking in
>      >      >     fileStream::flush()
>      >      >
>      >      >     Hi Denghui,
>      >      >
>      >      >     On 12/04/2020 2:41 pm, Denghui Dong wrote:
>      >      >      > Hi David,
>      >      >      >
>      >      >      > Thanks for the comments
>      >      >      > new webrev: http://cr.openjdk.java.net/~ddong/8242485/webrev.02
>      >      >      >
>      >      >      > when _file is NULL:
>      >      >      > - fileStream::read return 0
>      >      >      > - fileStream::readln return NULL
>      >      >      > - fileStream::eof return -1 (like
>      >      >      > fileStream::fileSize, not sure if appropriate, feof returns nonzero if target file is end)
>      >      >
>      >      >     That is all great - thanks!
>      >      >
>      >      >     Unfortunately the more I look at this code the more problems I see with
>      >      >     it. :( We have:
>      >      >
>      >      >     void fileStream::write(const char* s, size_t len) {
>      >      >         if (_file != NULL)  {
>      >      >           // Make an unused local variable to avoid warning from gcc compiler.
>      >      >           size_t count = fwrite(s, 1, len, _file);
>      >      >         }
>      >      >         update_position(s, len);
>      >      >     }
>      >      >
>      >      >     but the update_position should be inside the if-statement - no need to
>      >      >     see a new webrev for that.
>      >      >
>      >      >     We should also be watching for errors from fwrite - but that in itself
>      >      >     just opens up a can-of-worms with regard to error handling in general
>      >      >     with this class, so that can be left as-is.
>      >      >
>      >      >     Thanks,
>      >      >     David
>      >      >     -----
>      >      >
>      >      >      > Testing: tier1
>      >      >      >
>      >      >      > Cheers,
>      >      >      > Denghui Dong
>      >      >      >
>      >      >      >     ------------------------------------------------------------------
>      >      >      >     From:David Holmes <david.holmes at oracle.com>
>      >      >      >     Send Time:2020?4?12?(???) 10:40
>      >      >      >     To:???(??) <denghui.ddh at alibaba-
>     inc.com>; hotspot-dev
>      >      >      >     <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>      >      >      >     <hotspot-runtime-dev at openjdk.java.net>; Yasumasa Suenaga
>      >      >      >     <suenaga at oss.nttdata.com>; "Liu, Xin" <xxinliu at amazon.com>
>      >      >      >     Subject:Re: RFR(XXS): 8242485: Null _file checking in
>      >      >      >     fileStream::flush()
>      >      >      >
>      >      >      >     Hi,
>      >      >      >
>      >      >      >     On 12/04/2020 1:52 am, Denghui Dong wrote:
>      >      >      >      > Hi,
>      >      >      >      > Thank you for the review and point out other methods need to check _file.
>      >      >      >      > In addition to "fileStream::flush", I found there are still four methods need to check _file in fileStream: read, readln, eof, rewind.
>      >      >      >      > but those methods will crash if _file is NULL, and I didn't find any caller of them(Maybe I am wrong),
>      >      >      >      > so I think it's more appropriate to add assert check to those methods.
>      >      >      >
>      >      >      >     Assertions are really only appropriate if they will be exercised by
>      >      >      >     tests - the intent being that tests will expose any cases where the
>      >      >      >     assertion fails.
>      >      >      >
>      >      >      >     Also it doesn't make sense to me to have assertions in most places but a
>      >      >      >
>      >      >      >     true NULL check in others - that suggests we expect some methods to be
>      >      >      >     called when _file is NULL, but others not. I'd prefer just to see the
>      >      >      >     missing NULL checks added, rather than using assertions.
>      >      >      >
>      >      >      >     Thanks,
>      >      >      >     David
>      >      >      >
>      >      >      >      > Webrev: http://cr.openjdk.java.net/~ddong/8242485/webrev.01/
>      >      >      >      >
>      >      >      >      > Could you review it again, and sponsor it if everything is okay?
>      >      >      >      >
>      >      >      >      > Testing: teir1
>      >      >      >      > ==============================
>      >      >      >      > Test summary
>      >      >      >      > ==============================
>      >      >      >      >     TEST                                              TOTAL  PASS  FAIL ERROR
>      >      >      >      >     jtreg:test/hotspot/jtreg:tier1                     1516  1516     0     0
>      >      >      >      >     jtreg:test/jdk:tier1                               1904  1904     0     0
>      >      >      >      >  >> jtreg:test/langtools:tier1                         4031  4029     2     0 <<
>      >      >      >      >     jtreg:test/nashorn:tier1                              0     0     0     0
>      >      >      >      >     jtreg:test/jaxp:tier1                                 0     0     0     0
>      >      >      >      > ==============================
>      >      >      >      > There are two unrelated test failures in
>      >      >      >      > langtools: jdk/javadoc/tool/CheckResourceKeys.java  and tools/javac/processing/model/TestSymtabItems.java
>      >      >      >      >
>      >      >      >      >
>      >      >      >      >
>      >      >      >      >     ------------------------------------------------------------------
>      >      >      >      >     From:Yasumasa Suenaga <suenaga at oss.nttdata.com>
>      >      >      >      >     Send Time:2020?4?11?(???) 09:22
>      >      >      >      >     To:???(??) <denghui.ddh at alibaba-
>      >     inc.com>; hotspot-dev
>      >      >      >      >     <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>      >      >      >      >     <hotspot-runtime-dev at openjdk.java.net>
>      >      >      >      >     Subject:Re: RFR(XXS): 8242485: Null _file checking in
>      >      >      >      >     fileStream::flush()
>      >      >      >      >
>      >      >      >      >     Hi Denghui,
>      >      >      >      >
>      >      >      >      >     null check lacks in other place too.
>      >      >      >      >     Can you fix it?
>      >      >      >      >
>      >      >      >      >         http://hg.openjdk.java.net/jdk/jdk/file/97d5d0cd1085/src/hotspot/share/utilities/ostream.cpp#l554
>      >      >      >      >
>      >      >      >      >
>      >      >      >      >     Thanks,
>      >      >      >      >
>      >      >      >      >     Yasumasa
>      >      >      >      >
>      >      >      >      >
>      >      >      >      >     On 2020/04/10 21:34, Denghui Dong wrote:
>      >      >      >      >      > Hi team,
>      >      >      >      >      >
>      >      >      >      >      > Could you please review this small patch?
>      >      >      >      >      >
>      >      >      >      >      > JBS: https://bugs.openjdk.java.net/browse/JDK-8242485
>      >      >      >      >      > webrev: http://cr.openjdk.java.net/~ddong/8242485/webrev.00/
>      >      >      >      >      >
>      >      >      >      >      > I found the method "fileStream::flush()" lacks null check before fflush  which will cause
>      >      >      >      >      > all open output streams are flushed by fflush() if _file is NULL.
>      >      >      >      >      >
>      >      >      >      >      > Thanks,
>      >      >      >      >      > Denghui Dong
>      >      >      >      >      >
>      >      >      >      >
>      >      >      >
>      >      >
>      >
>      > 
> 

From coleen.phillimore at oracle.com  Tue Apr 14 12:36:13 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Tue, 14 Apr 2020 08:36:13 -0400
Subject: RFR (S) 8074292: nsk/jdb/kill/kill001: generateOopMap.cpp
 assert(bb->is_reachable()) failed
In-Reply-To: <6e38070c-f166-a0c2-8b8d-d698e5f5bcf3@oracle.com>
References: <f66b8de6-d9db-a90f-bfc2-5bf42a19cada@oracle.com>
 <6e38070c-f166-a0c2-8b8d-d698e5f5bcf3@oracle.com>
Message-ID: <1e3d4907-d54a-df0e-e0d3-8bbda8c5394a@oracle.com>



On 4/13/20 10:49 PM, David Holmes wrote:
> Hi Coleen,
>
> On 14/04/2020 12:34 am, coleen.phillimore at oracle.com wrote:
>> Summary: Do not install async exceptions at_safepoint for each bytecode.
>
> I'm still not certain that we have to go this far to solve this 
> problem, but it does sound like a relatively simple solution provided 
> there are no unintended consequences.
>
>> See CR for a lot more details.? This change calls a new 
>> InterpreterRuntime::at_safepoint_async_safe() which installs the 
>> async exception in the interpreter at backward branches and returns.? 
>> This uses safepoint polling code in the interpreter for each 
>> platform.? These changes (cross) compile on platforms that Oracle 
>> doesn't support but I don't know if they work.
>>
>> I'm not convinced the platform specific changes are necessary, 
>> because calls to the runtime from many bytecodes will install the 
>> async exception, so it's essentially installed "enough" for this 
>> deprecated feature.? I tested the changes with *and* without the 
>> platform specific changes with no failure, which included the jdb, 
>> jdi and jvmti serviceability tests.
>
> I don't understand what you mean here. If the whole basis of this fix 
> is "don't install async exceptions other than at backward branches and 
> returns" then how is that implemented without the changes in the 
> interpreter code?
>
> If this can be fixed just by adjusting the actual monitor code then I 
> would much prefer that. It took me a while to get my head around the 
> dispatch changes in interpreter code and even then I don't see how 
> those changes only impact backward branches and returns ??

You have to read the comments in the bug again.? There *is* special code 
to not install the async exception in the monitorexit code. That is not 
enough to prevent this bug.? You can also read the old bug report you 
linked to this one.

The interpreter code dispatch_next passes "true" if it's a backwards 
branch, that's how it can tell.

My point was that there are enough code paths that install async 
exceptions *other than monitorenter and monitorexit* that maybe it's not 
necessary to install them at backwards branches and returns.? I suppose 
someone could construct a test case to show otherwise.
>
>>
>> This change also makes InterpreterRuntime::monitorexit a JRT_LEAF 
>> bytecode. The code to check for exceptions is outside the runtime 
>> call. I ran the JCK vm and lang tests on this change with no failure.
>>
>> Tested with tier1-6.
>>
>> open webrev at 
>> http://cr.openjdk.java.net/~coleenp/2020/8074292.01/webrev
>> bug link https://bugs.openjdk.java.net/browse/JDK-8074292
>
> ./cpu/x86/interp_masm_x86.cpp
>
> It took me a long time to figure out how the new logic worked compared 
> to the old logic. Even then I'm unclear about the effective recursive 
> dispatch path: dispatch_base(generate_poll=true) -> dispatch_via -> 
> dispatch_base(generate_poll=false) - does it work okay with 
> VerifyActivationFrameSize? It seems a rather convoluted way to 
> effectively just execute:
>
> ?858?? lea(rscratch1, ExternalAddress((address)table));
> ?859?? jmp(Address(rscratch1, rbx, Address::times_8));

I could test it with VerifyActivationFrameSize.? Or I could just add the 
two instructions per platform.? I mostly copied the code in 
generate_safept_entry_for.? It might be better to just copy the 
instructions.

>
> ---
>
> src/hotspot/share/interpreter/interpreterRuntime.cpp
>
> How were you able to drop this code:
>
> 791?? if (elem == NULL || h_obj()->is_unlocked()) {
> 792 THROW(vmSymbols::java_lang_IllegalMonitorStateException());
> 793?? }
>
> ?
> ?and this:

The caller throws the exception for these cases.

>
> 798 #ifdef ASSERT
> 799 thread->last_frame().interpreter_frame_verify_monitor(elem);
> 800 #endif
>

This seemed redundant.

Coleen
> ?
>
> Thanks,
> David
>
>> Thanks,
>> Coleen
>>


From kim.barrett at oracle.com  Tue Apr 14 13:02:02 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Tue, 14 Apr 2020 09:02:02 -0400
Subject: [trivial] RFR: 8242631: Missing but used special functions for some
 classes 
Message-ID: <13190093-8B8E-4D9C-804A-F130B9C3C2EF@oracle.com>

Please review this trivial change to a couple of classes to eliminate
gcc9 deprecation warnings for them when compiling for C++11 or later.
See the CR for more details about the warnings.

BCEscapeAnalyzer::ArgumentMap has an implicitly defined copy
constructor and a user-defined copy assignment operator.  Since the
copy assignment operator is functionally equivalent to the default,
removed it.

Various subtypes of oop had an implicitly defined copy constructor;
made it explicit.

There are a few more like these, but I may address them differently.

CR:
https://bugs.openjdk.java.net/browse/JDK-8242631

Webrev:
https://cr.openjdk.java.net/~kbarrett/8242631/open.00/

Testing:
mach5 tier1



From leo.korinth at oracle.com  Tue Apr 14 13:53:09 2020
From: leo.korinth at oracle.com (Leo Korinth)
Date: Tue, 14 Apr 2020 15:53:09 +0200
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <e242e057-de6e-493a-eb00-068b2db8c9fa@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
 <135a6496-5b54-9117-5cbf-e3211ddc32dd@oracle.com>
 <bbb2b5ef-62f2-6507-95f7-03712a382153@oracle.com>
 <e242e057-de6e-493a-eb00-068b2db8c9fa@oracle.com>
Message-ID: <df621fd2-0eaf-cb24-788f-12166b271303@oracle.com>

Hi!

On 06/04/2020 01:37, David Holmes wrote:

>>
>> I agree that it could be compared against NULL_WORD, do you want me to 
>> change it? (I have no opinion on this issue)
> 
> If we're trying to clean up the use of 0, NULL, NULL_WORD then I think 
> it should be changed.

Fixed in several locations.

>>>
>>> ?> new NULL value does unfortunately not convert to method pointers
>>>
>>> Is that a temporary limitation? I'm not sure what progress were are 
>>> making if some pointers can use NULL and others must use 0.
>>
>> I believe this to be a temporary limitation, my understanding is that 
>> when we get c++11 nullptr will convert to method pointers as well, I 
>> could and think I should revert these changes as I did revert my 
>> implementation of NULL, would you like that? My null_t type could, to 
>> my knowledge, not auto convert to method pointers at least not without 
>> template varargs stuff that comes in later versions of the standard.
> 
> This would look better to me as NULL.

I have reverted this and a few other places.


>>
>> This is the same issue as before with string literals and macros, my 
>> arguing is that I will not try to prove that value is (or in the 
>> future always will be) a string literal, instead I will make the code 
>> work for both string literals and pointers. I hope you do not mind the 
>> extra parentheses I added around "value".
> 
> Normal style rules for macros say to use the macro arg in parentheses - 
> but there are three other uses in the same macro.
> 

I will keep the parenthesis on the line I changed and do no additional 
such change.


> 
>> I could change CHECK_0 to use NULL_WORD though and if I do, I get 
>> compiler errors. CHECK_0 is used for both pointers and integral zeroes 
>> and types that implicitly converts from either. What the original 
>> thought was is unclear to me; is CHECK_0 to be used to return things 
>> convertible from pointers? If so, what is CHECK_NULL for?
> 
> No CHECK_0 is intended for int-returning functions (which would also 
> extend to intptr_t).

It might be intended for it, but it is not used in such a way (therefore 
my remark on compile failures when using NULL_WORD)

I will do no change here.


>>
>>> ---
>>>
>>> src/hotspot/share/utilities/globalDefinitions_solstudio.hpp
>>>
>>> It isn't clear that the issue of passing 0 as NULL in a varargs 
>>> situation is now fixed in Sol Studio.
>>>
>>
>> I guess tier1-3 would fail if not? Maybe that is a false assumption of 
>> mine?
> 
> I would consider that a necessary but not sufficient condition.
> 

I reverted my NULL/NULL_WORD redefinition so it is no issue any more at 
least.

>>>
>>> src/hotspot/share/utilities/linkedlist.hpp
>>>
>>> -??? return equal<E>(_data, t, NULL);
>>> +??? return equal<E>(_data, t, 0);
>>>
>>> Again concerned about the method/function pointer situation.
>>
>> I am not concerned :-) but I will revert this as it is not necessary 
>> now or in the future as I reverted my original implementation and 
>> c++11 will fix this.
> 
> Ok.

Fixed

In additions to these changes I have also /reverted/ all changes to 
NULL/NULL_WORD definitions. In the end Kim did not like them, and they 
can be fixed later when we get a more modern C++. To reflect this I will 
also change the CR title to "Improve NULL usage"

Webrev:
http://cr.openjdk.java.net/~lkorinth/8240110/part1 (NULL)
http://cr.openjdk.java.net/~lkorinth/8240110/part2 (NULL_WORD)
http://cr.openjdk.java.net/~lkorinth/8240110/part3 (reverse somewhat)

new in this mail:
http://cr.openjdk.java.net/~lkorinth/8240110/part4 (improvements 
suggested by David)
http://cr.openjdk.java.net/~lkorinth/8240110/part5 (reverse my changes 
to NULL/NULL_WORD itself)

total combined change:
http://cr.openjdk.java.net/~lkorinth/8240110/1

Testing:
running tier 1-3 (looks good so far)

Thanks,
Leo

From magnus.ihse.bursie at oracle.com  Tue Apr 14 14:18:09 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Tue, 14 Apr 2020 16:18:09 +0200
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <df621fd2-0eaf-cb24-788f-12166b271303@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
 <135a6496-5b54-9117-5cbf-e3211ddc32dd@oracle.com>
 <bbb2b5ef-62f2-6507-95f7-03712a382153@oracle.com>
 <e242e057-de6e-493a-eb00-068b2db8c9fa@oracle.com>
 <df621fd2-0eaf-cb24-788f-12166b271303@oracle.com>
Message-ID: <e4fed1a3-addd-133c-fc68-74cf47fa6086@oracle.com>



On 2020-04-14 15:53, Leo Korinth wrote:
> Hi!
>
> On 06/04/2020 01:37, David Holmes wrote:
>
>>>
>>> I agree that it could be compared against NULL_WORD, do you want me 
>>> to change it? (I have no opinion on this issue)
>>
>> If we're trying to clean up the use of 0, NULL, NULL_WORD then I 
>> think it should be changed.
>
> Fixed in several locations.
>
>>>>
>>>> ?> new NULL value does unfortunately not convert to method pointers
>>>>
>>>> Is that a temporary limitation? I'm not sure what progress were are 
>>>> making if some pointers can use NULL and others must use 0.
>>>
>>> I believe this to be a temporary limitation, my understanding is 
>>> that when we get c++11 nullptr will convert to method pointers as 
>>> well, I could and think I should revert these changes as I did 
>>> revert my implementation of NULL, would you like that? My null_t 
>>> type could, to my knowledge, not auto convert to method pointers at 
>>> least not without template varargs stuff that comes in later 
>>> versions of the standard.
>>
>> This would look better to me as NULL.
>
> I have reverted this and a few other places.
>
>
>>>
>>> This is the same issue as before with string literals and macros, my 
>>> arguing is that I will not try to prove that value is (or in the 
>>> future always will be) a string literal, instead I will make the 
>>> code work for both string literals and pointers. I hope you do not 
>>> mind the extra parentheses I added around "value".
>>
>> Normal style rules for macros say to use the macro arg in parentheses 
>> - but there are three other uses in the same macro.
>>
>
> I will keep the parenthesis on the line I changed and do no additional 
> such change.
>
>
>>
>>> I could change CHECK_0 to use NULL_WORD though and if I do, I get 
>>> compiler errors. CHECK_0 is used for both pointers and integral 
>>> zeroes and types that implicitly converts from either. What the 
>>> original thought was is unclear to me; is CHECK_0 to be used to 
>>> return things convertible from pointers? If so, what is CHECK_NULL for?
>>
>> No CHECK_0 is intended for int-returning functions (which would also 
>> extend to intptr_t).
>
> It might be intended for it, but it is not used in such a way 
> (therefore my remark on compile failures when using NULL_WORD)
>
> I will do no change here.
>
>
>>>
>>>> ---
>>>>
>>>> src/hotspot/share/utilities/globalDefinitions_solstudio.hpp
>>>>
>>>> It isn't clear that the issue of passing 0 as NULL in a varargs 
>>>> situation is now fixed in Sol Studio.
>>>>
>>>
>>> I guess tier1-3 would fail if not? Maybe that is a false assumption 
>>> of mine?
>>
>> I would consider that a necessary but not sufficient condition.
>>
>
> I reverted my NULL/NULL_WORD redefinition so it is no issue any more 
> at least.
>
>>>>
>>>> src/hotspot/share/utilities/linkedlist.hpp
>>>>
>>>> -??? return equal<E>(_data, t, NULL);
>>>> +??? return equal<E>(_data, t, 0);
>>>>
>>>> Again concerned about the method/function pointer situation.
>>>
>>> I am not concerned :-) but I will revert this as it is not necessary 
>>> now or in the future as I reverted my original implementation and 
>>> c++11 will fix this.
>>
>> Ok.
>
> Fixed
>
> In additions to these changes I have also /reverted/ all changes to 
> NULL/NULL_WORD definitions. In the end Kim did not like them, and they 
> can be fixed later when we get a more modern C++. To reflect this I 
> will also change the CR title to "Improve NULL usage"
>
> Webrev:
> http://cr.openjdk.java.net/~lkorinth/8240110/part1 (NULL)
> http://cr.openjdk.java.net/~lkorinth/8240110/part2 (NULL_WORD)
> http://cr.openjdk.java.net/~lkorinth/8240110/part3 (reverse somewhat)
>
> new in this mail:
> http://cr.openjdk.java.net/~lkorinth/8240110/part4 (improvements 
> suggested by David)
> http://cr.openjdk.java.net/~lkorinth/8240110/part5 (reverse my changes 
> to NULL/NULL_WORD itself)
>
> total combined change:
> http://cr.openjdk.java.net/~lkorinth/8240110/1

Combined change ended up containing updated copyright in 
src/hotspot/share/runtime/sweeper.hpp, even though no actual change were 
done there in the end.

/Magnus
>
> Testing:
> running tier 1-3 (looks good so far)
>
> Thanks,
> Leo


From matthias.baesken at sap.com  Tue Apr 14 15:27:14 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 14 Apr 2020 15:27:14 +0000
Subject: RFR: 8242626: enhance posix print_rlimit_info
Message-ID: <AM0PR02MB4227F897CBF6AA514A7343D293DA0@AM0PR02MB4227.eurprd02.prod.outlook.com>

Please review this small enhancement of  os::Posix::print_rlimit_info.

It adds  return code handling  of  getrlimit ,  output of some more limit values and  output of  hard limits (additionally to the existing soft limits) .

Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8242626

http://cr.openjdk.java.net/~mbaesken/webrevs/8242626.1/

Thanks, Matthias

From paul.sandoz at oracle.com  Tue Apr 14 18:51:44 2020
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Tue, 14 Apr 2020 11:51:44 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <da9f80ff-6d63-aebe-a9fa-1a683eb42bbc@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <1a5eb022-dd7e-b0bb-750f-c43be195eb42@oracle.com>
 <da9f80ff-6d63-aebe-a9fa-1a683eb42bbc@oracle.com>
Message-ID: <4A360464-1873-43A8-B423-C93D1BEF9895@oracle.com>

Looks good to me (not familiar with all the code areas.

Minor suggestion:

MethodHandles.java

1811          * ASCII periods. For the instance of {@link java.lang.Class} representing {@code C}:
1812          * <ul>
1813          * <li> {@link Class#getName()} returns the string {@code GN + "/" + <suffix>},
1814          *      even though this is not a valid binary class or interface name.</li>
1815          * <li> {@link Class#descriptorString()} returns the string
1816          *      {@code "L" + N + ";" + "/" + <suffix> },
1817          *      even though this is not a valid type descriptor name.</li>
1818          * </ul>

Add another bullet:

?
even though this is not a valid type descriptor name; and

- therefore {@link Class#describeConstable} returns an empty {@code Optional}.
?

?

Paul.

> On Apr 11, 2020, at 8:13 PM, Mandy Chung <mandy.chung at oracle.com> wrote:
> 
> Please review the delta patch:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.06-delta/
> 
> Incremental specdiff:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff-inc/overview-summary.html
> This is to follow a discussion [1] on Class::descriptorString and MethodType::descriptorString for hidden classes.   The proposal is to define the descriptor string for a hidden class of this form:
>     "L" + N + ";" + "/" + <suffix>
> 
> The spec of `Lookup::defineHiddenClass`, `Class::descriptorString` and `MethodType::descriptorString` are updated to return the descriptor of this form for a hidden class.   To support hidden class, `java.lang.invoke.TypeDescriptor` spec is revised such that a `TypeDescriptor` object can represent an entity that may not be described in nominal form.     This change affects JVM TI, JDWP and JDI.    The spec change includes a couple other JVM TI and java.instrument spec clarification w.r.t. hidden classes that Serguei has been working on.
> 
> 
> 
> Mandy
> [1] https://mail.openjdk.java.net/pipermail/valhalla-dev/2020-April/007093.html
> 
> ----------------
> As a record, the above patch is applied on the top:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.06/
> 
> webrev.06 includes the following changes that have been reviewed:
> 1. rename "weak hidden" and Klass::is_hidden_weak to is_non_strong_hidden
> 2. remove unused `setImplMethod` method from lambda proxy class
> 3. include Serguei's patch to fix JDK-8242166: regression in JDI ClassUnload events
> 4. drop the uniqueness guarantee of the suffix of the hidden class's name
> 
> On 3/31/20 8:01 PM, Mandy Chung wrote:
>> Thanks to the review feedbacks.
>> 
>> Updated webrev:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.04/ 
>> Delta between webrev.03 and webrev.04:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03-delta/ 
>> 
>> Summary of changes is:
>> 1. Update javac to retain the previous behavior when compiling to target release <= 14 where lambda proxy class is not a nestmate
>> 2. Rename Class::isHiddenClass to Class::isHidden
>> 3. Address Joe's feedback on the CSR to document the behavior of the relevant methods in java.lang.Class for hidden classes
>> 4. Add test case for unloadable class with nest host error
>> 5. Add test cases for hidden classes with different kinds of class or interface
>> 6. Update dcmd to drop "weak hidden class" and refer it as "hidden class"
>> 7. Small changes in response to Remi, Coleen, Paul's review comments
>> 8. Fix copyright headers
>> 9. Fix @modules in tests
>> 
>> Most of the changes above have also been reviewed as separate patches.
>> 
>> Thanks
>> Mandy
>> 
>> On 3/26/20 4:57 PM, Mandy Chung wrote:
>>> Please review the implementation of JEP 371: Hidden Classes. The main changes are in core-libs and hotspot runtime area.  Small changes are made in javac, VM compiler (intrinsification of Class::isHiddenClass), JFR, JDI, and jcmd.  CSR [1]has been reviewed and is in the finalized state (see specdiff and javadoc below for reference).
>>> 
>>> Webrev:
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
>>> 
>>> javadoc/specdiff
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
>>> 
>>> JVMS 5.4.4 change:
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
>>> 
>>> CSR:
>>> https://bugs.openjdk.java.net/browse/JDK-8238359
>>> 
>>> Thanks
>>> Mandy
>>> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
>>> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
>>> [3] https://bugs.openjdk.java.net/browse/JDK-8230502
>> 
> 


From xxinliu at amazon.com  Tue Apr 14 19:28:18 2020
From: xxinliu at amazon.com (Liu, Xin)
Date: Tue, 14 Apr 2020 19:28:18 +0000
Subject: RFC 8236988 - Modular Design for JVM Flags
In-Reply-To: <e0a4bcc5-e116-c901-85f9-78360307e243@oracle.com>
References: <6EE91323-A69B-44E1-8215-411250EB16DD@amazon.com>
 <e0a4bcc5-e116-c901-85f9-78360307e243@oracle.com>
Message-ID: <A01A906F-392D-4105-8868-EF76B2A278AE@amazon.com>

Hi, Chris, 

Your project is very interesting. Nice work! I've starred it.
Ioi also show me some useful flags I don't know that. thanks!

Hi, Ioi, 
I got some comments inline. 

?On 4/12/20, 5:22 PM, "Ioi Lam" <ioi.lam at oracle.com> wrote:

    CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
    
    
    
    Hi Xin,
    
    Thanks for the feedback. It's very helpful.
    
    On 4/12/20 12:59 AM, Liu, Xin wrote:
    > Hi, Ioi,
    >
    > I'm recently refactoring intrinsics options for compiler[1]. I introduced a new option type called "intrinsics".  The biggest trouble was that I had to modify multiple places.  I found your patch beautifully replaces jvmConstraintList.* &jvmFlagRangeList.* with some macros in jvmFlag.inline.hpp. great!
    >
    > I have some questions or suggests.
    >
    > 1. generate a document of options
    > There're hundreds of JVM options.  It's hard to learn them by heart. I feel developers uses globals.hpp and sub header files as a de factor manual.
    > if you break up the monolithic header, which is good,  I think developers deserve a manual.
    > Off the top of my head, I think you can provide a mini tool, which consume a header aggerates all sub-header files and automatically generates a manual.
    
    Even today the flags are scattered around in various *.hpp files. There
    are already ways of finding information about the all the flags:
    
         java -XX:+PrintFlagsWithComments   (debug build only)
         java -XX:+PrintFlagsInitial
         java -XX:+PrintFlagsFinal
         java -XX:+PrintFlagsRanges
    
    These flags will continue to work after my patch. Would these be sufficient?
    
    Also, you can use this to look for a flag:
    
         find . -name \*.hpp | xargs grep FLAG.*SplitIfBlocks
    
    > 2. change FLAG_RANGE and FLAG_CONTRAINT to optional parameters of PRODUCT_FLAG.
    > If you use variadic macro, it's possible to select RANGE or CONTRAINT or NIL from __VA_ARGS__.
    > I used to propose it before. https://cr.openjdk.java.net/~xliu/8230669/00/webrev/src/hotspot/share/utilities/macro_overloading.hpp.html
    >
    > I think it's a good idea because it make the declaration section pure.
    > If we replace those macros eg. PRODUCT_FLAG with C++ classes of same names, it's very easy to write a standalone tool to generate a manual of JVM options from header files, right?
    
    That's a good suggestion. I've uploaded an updated version of the design
    that uses __VA_ARGS__
    
    http://cr.openjdk.java.net/~iklam/design/jvm_flags_overhaul/flags.hpp
    http://cr.openjdk.java.net/~iklam/design/jvm_flags_overhaul/flags.cpp
    
    To see how the macros work, try doing this, and look at the file flags.ii
    
         gcc -DPRODUCT -O3 -o flags -save-temps flags.cpp && ./flags
         gcc -g -o flags_g -save-temps flags.cpp && ./flags_g
    
--
Oh, you use overloaded constructors to select optional parameters. 
Interesting, I didn't come up with that idea. 
--
    I think we can try different designs using the above as a template. When
    we have a more concrete design I can implement it in the VM.
    
    Now the declaration is:
    
         PRODUCT_FLAG(bool,     SplitIfBlocks, true, JVMFlag::DEFAULT,
                                "Clone compares and control flow through merge "
                                "points to fold some branches");
    
         DEVELOP_FLAG(intx,     FreqCountInvocations, 1, JVMFlag::DEFAULT,
                                "Scaling factor for branch frequencies",
                                FLAG_RANGE(1, max_intx))
    
         PRODUCT_FLAG(intx,     NodeLimitFudgeFactor, 2000, JVMFlag::CONSTRAINT,
                                "Fudge Factor for certain optimizations",
    FLAG_CONSTRAINT(NodeLimitFudgeFactorConstraintFunc,
                                JVMFlag::AfterErgo))
    
         PRODUCT_FLAG(intx,     AliasLevel, 3, JVMFlag::DEFAULT,
                                "0 for no aliasing, 1 for .... split, "
                                "2 for class split, 3 for unique instances",
                                FLAG_RANGE(0, 3),
    FLAG_CONSTRAINT(AliasLevelConstraintFunc,
                                JVMFlag::AfterErgo))
    
    and the implementation:
    
         DEFN_PRODUCT_FLAG(SplitIfBlocks);
         DEFN_DEVELOP_FLAG(FreqCountInvocations);
         DEFN_PRODUCT_FLAG(NodeLimitFudgeFactor);
         DEFN_PRODUCT_FLAG(AliasLevel);
    
    My goal is still to put all the information about each flag in a single
    place (the hpp file), so I need to pass the info to the cpp files using
    an inline function:
    
       inline const ModifiableFlag<FLAG_TYPE_##name> init_##name() { \
         return ModifiableFlag<FLAG_TYPE_##name>( \
           JVMFlag::TYPE_##type, #name, attr, &name, __VA_ARGS__); \
       }
    
    But the problem is this inline function could theoretically allocate the
    instance on the stack, so I cannot save its address into the _head of
    the global flags list. As I result, I needed to split the _next pointer
    from JVMFlag into AbstractFlagLink::_link.
    
    If anyone has a better suggestion, I would be happy to try it out.
--
You can't get addresses of temps, but you can get addresses of static variables.  Is this trick useful for your aim?

Eg.
       inline const ModifiableFlag<FLAG_TYPE_##name>* init_##name() { \
         static ModifiableFlag<FLAG_TYPE_##name>( \
           JVMFlag::TYPE_##type, #name, attr, &name, __VA_ARGS__) t; \
           Return &t;                                                                                              \
       }

Maybe Linkedlist isn't your only choice. I use enum to count how many options JVM will have and allocate a static array. 
In this way, I can even sort the array and use binary-search. 
Check out line #89 of https://cr.openjdk.java.net/~xliu/8151779/00/webrev/src/hotspot/share/compiler/intrinsics.cpp.html
--
   
    > 2. There's a TCL script.  Isn't C Processor powerful enough for it?  I don't know TCL, I think an external script will bring dependency and learning  burden.
    >
    > 3. let's say we choose a script generator. It looks like those globals_xxx.cpp files are auto-gen files.
    > Will you consider to untrack those generated files in the repo?
    
    The whole purpose of the TCL script is to convert the old style macros
    in the hpp files into the new coding style. That way I can keep track of
    changes in the upstream repo, and also easily change the design of the
    new code. The script is used by me only (that's why it's messy). It's
    not used during the build and will not be needed in the final patch.

    The globals_xxx.cpp files will be checked into the repo. Afterwards, to
    add/remove flags, you would need to edit the xxx_globals.hpp/cpp files
    directly.
    
    As mentioned in the design doc, my goal is to avoid C++ code generation
    as many people are against it.
    
    Personally, I prefer code generation. I think that can result in a
    smaller binary footprint that will start up faster, but the delta is
    probably negligible. We can probably leave that for the future. As long
    as we use the macros in a consistent manner, we can easily extract the
    flag definitions and put them into external input files that are
    processed by a C++ code generator.
---
But your current webrev leads to have a codegen path.

You have to iterate all options at least twice, right? one is in header file, the other one is in the cpp file.
The list is long and the order doesn't matter. I can't imagine you will check in those hpp/cpp pairs.  It's very natural to have a script like tcl or m4 to generate cpp from hpp, right? 
There was a high-order macro RUNTIME_FLAGS in previous globle.hpp. That's why it avoids to iterate options again & again. C-preprocessor did the "codegen" job by expanding macros. 

Why do you ditch that design? it still can archive all your design goals. 
---   
    > 4. Probably, it's not a bad idea to still use product/develop instead of PRODUCT_FLAG, DEVELOP_FLAG.
    > Those options will be read everyday by developers, I feel lower-case names are more friendly.
    
    I don't have a strong preference, but I think that adding "product"
    globally (even as a macro) may cause problems with code like this:
    
         static int product(int a, int b) {return a*b;}
    
    as the C pre-processor would try to expand "product".
--
In previous globals.hpp, 'product' is just a macro parameter.  That's why current implementation doesn't have any conflict with the function product.
--    

    Thanks
    - Ioi
    
    
    >
    > [1]: https://cr.openjdk.java.net/~xliu/8151779/00/webrev/
    >
    > Thanks,
    > --lx
    >
    > On 4/9/20, 5:08 PM, "hotspot-dev on behalf of Ioi Lam" <hotspot-dev-bounces at openjdk.java.net on behalf of ioi.lam at oracle.com> wrote:
    >
    >      CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
    >
    >
    >
    >      The code for specifying HotSpot command-line flags used to be simple. At
    >      the very beginning it was something like
    >
    >           #define ALL_FLAGS(develop, develop) \
    >              product(int, SomeFlag, def_value, "some doc") \
    >              develop(int, MoreFlag, def_value, "more doc") \
    >
    >      but like all good ideas, it has gradually degraded into a mess [1][2].
    >      It's now very difficult to add any new functionality (e.g., add
    >      "manageable" annotations to the flags [3]).
    >
    >      I think it's time for a complete rewrite :-)
    >
    >      My proposal is here. Please comment if it's a good or bad idea.
    >
    >      https://wiki.openjdk.java.net/display/HotSpot/HotSpot+Command-Line+Flags+Overhaul+-+Design+Doc
    >
    >
    >
    >      tldr; ---
    >
    >
    >      OLD:
    >
    >         product(intx, AliasLevel, 3,                                          \
    >                 "0 for no aliasing, 1 for oop/field/static/array split,
    >      "         \
    >                 "2 for class split, 3 for unique
    >      instances")                      \
    >                 range(0, 3)
    >      /*optional*/                                         \
    >                 constraint(AliasLevelConstraintFunc,AfterErgo) /*optional*/
    >           \
    >
    >
    >      NEW:
    >
    >         PRODUCT_FLAG(intx,   AliasLevel, 3, JVMFlag::RANGE | JVMFlag::CONSTRAINT,
    >                              "0 for no aliasing, 1 for oop/field/static/array
    >      split, "
    >                              "2 for class split, 3 for unique instances");
    >          // optional
    >          FLAG_RANGE(         AliasLevel, 0, 3);
    >          // optional
    >          FLAG_CONSTRAINT(    AliasLevel, (void*)AliasLevelConstraintFunc,
    >      JVMFlag::AfterErgo);
    >
    >
    >
    >
    >      [1]
    >      http://hg.openjdk.java.net/jdk/jdk/file/7af6364e1792/src/hotspot/share/runtime/globals.hpp#l115
    >      [2]
    >      http://hg.openjdk.java.net/jdk/jdk/file/7af6364e1792/src/hotspot/share/runtime/flags/jvmFlag.cpp#l635
    >      [3] https://bugs.openjdk.java.net/browse/JDK-7123237
    >
    >      Thanks
    >      - Ioi
    >
    >
    
    


From david.holmes at oracle.com  Wed Apr 15 02:11:40 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 15 Apr 2020 12:11:40 +1000
Subject: RFR(XXS): 8242485: Null _file checking in fileStream::flush()
In-Reply-To: <5c187808-6f99-4293-a6f1-6440c2eaa102.denghui.ddh@alibaba-inc.com>
References: <e8d66934-57b9-44e5-a1b2-c650ea101220.denghui.ddh@alibaba-inc.com>
 <a8bd28fd-619a-d974-621d-45873edc7fa0@oss.nttdata.com>
 <a5758e50-0d65-4c90-8512-c41415b7aed7.denghui.ddh@alibaba-inc.com>
 <d7f0d3b2-1fc7-3601-b651-a4bdb0244d23@oracle.com>
 <d483f6ff-c51e-4eed-b11d-d7e8e427d8e4.denghui.ddh@alibaba-inc.com>
 <618b239a-2a21-a56f-021a-74df5e8d433c@oracle.com>
 <f86225d5-993a-40bf-98d1-fdb47e113877.denghui.ddh@alibaba-inc.com>
 <25546944-b9e7-bca8-aff3-9cd90c443dcd@oracle.com>
 <09f66dc8-e613-4aad-8c2c-4d77c45f33b1.denghui.ddh@alibaba-inc.com>
 <ebeb9566-bd3e-3a7b-4c1c-2e89c9f1c571@oracle.com>
 <f499226c-0bc5-4518-b757-58eed29876bb.denghui.ddh@alibaba-inc.com>
 <adfabe15-25a8-6076-1d06-4a4c4baaff24@oracle.com>
 <5c187808-6f99-4293-a6f1-6440c2eaa102.denghui.ddh@alibaba-inc.com>
Message-ID: <fc6a94ba-9c3a-ff4f-8b66-dd5ed20cb3c8@oracle.com>

Changes pushed.

David

On 14/04/2020 6:48 pm, Denghui Dong wrote:
> http://cr.openjdk.java.net/~ddong/8242485/webrev.02/
> 
> Added review info.
> 
> Thanks,
> <http://cr.openjdk.java.net/~ddong/8242485/webrev.02/>Denghui Dong
> 
>     ------------------------------------------------------------------
>     From:David Holmes <david.holmes at oracle.com>
>     Send Time:2020?4?14?(???) 15:28
>     To:???(??) <denghui.ddh at alibaba-inc.com>; hotspot-dev
>     <hotspot-dev at openjdk.java.net>; hotspot-runtime-dev at openjdk.java.net
>     <hotspot-runtime-dev at openjdk.java.net>; Yasumasa Suenaga
>     <suenaga at oss.nttdata.com>; "Liu, Xin" <xxinliu at amazon.com>
>     Subject:Re: RFR(XXS): 8242485: Null _file checking in
>     fileStream::flush()
> 
>     On?14/04/2020?4:04?pm,?Denghui?Dong?wrote:
>      >?Thanks!
>      >
>      >?Could?you?sponsor?it?
> 
>     Sure.?Please?send?me?a?link?to?the?finalized?changeset?properly
>     formatted?with?reviewers?etc.
> 
>     Thanks,
>     David
> 
>      >?????------------------------------------------------------------------
>      >?????From:David?Holmes?<david.holmes at oracle.com>
>      >?????Send?Time:2020?4?14?(???)?13:54
>      >?????To:???(??)?<denghui.ddh at alibaba-inc.com>;?hotspot-dev
>      >?????<hotspot-dev at openjdk.java.net>;?hotspot-runtime-dev at openjdk.java.net
>      >?????<hotspot-runtime-dev at openjdk.java.net>;?Yasumasa?Suenaga
>      >?????<suenaga at oss.nttdata.com>;?"Liu,?Xin"?<xxinliu at amazon.com>
>      >?????Subject:Re:?RFR(XXS):?8242485:?Null?_file?checking?in
>      >?????fileStream::flush()
>      >
>      >?????Hi?Denghui,
>      >
>      >?????This?looks?fine?to?me.
>      >
>      >?????Thanks!
>      >
>      >?????David
>      >
>      >?????On?14/04/2020?3:28?pm,?Denghui?Dong?wrote:
>      >??????>?Hi?David,
>      >??????>
>      >??????>?After?discussing?with?Yasumasa,??I?decided?to?put?update_position?into
>      >??????>?if-statement,
>      >??????>?I?also?modified?fdStream::write,?because?it?obviously?has?the?same?problem.
>      >??????>
>      >??????>?Could?you?review?it?again?
>      >??????>?still?at?http://cr.openjdk.java.net/~ddong/8242485/webrev.02
>      >??????>
>      >??????>?Thanks,
>      >??????>?Denghui?Dong
>      >??????>
>      >??????>?????------------------------------------------------------------------
>      >??????>?????From:David?Holmes?<david.holmes at oracle.com>
>      >??????>?????Send?Time:2020?4?13?(???)?06:23
>      >??????>?????To:???(??)?<denghui.ddh at alibaba-inc.com>;?hotspot-dev
>      >??????>?????<hotspot-dev at openjdk.java.net>;?hotspot-runtime-dev at openjdk.java.net
>      >??????>?????<hotspot-runtime-dev at openjdk.java.net>;?Yasumasa?Suenaga
>      >??????>?????<suenaga at oss.nttdata.com>;?"Liu,?Xin"?<xxinliu at amazon.com>
>      >??????>?????Subject:Re:?RFR(XXS):?8242485:?Null?_file?checking?in
>      >??????>?????fileStream::flush()
>      >??????>
>      >??????>?????On?12/04/2020?7:01?pm,?Denghui?Dong?wrote:
>      >??????>??????>?Hi?David,
>      >??????>??????>
>      >??????>??????>?Thanks?for?the?comments!
>      >??????>??????>
>      >??????>??????>>?Unfortunately?the?more?I?look?at?this?code?the?more?problems?I?see?with
>      >??????>??????>>?it.?:(?We?have:
>      >??????>??????>>
>      >??????>??????>>?void?fileStream::write(const?char*?s,?size_t?len)?{
>      >??????>??????>>????if?(_file?!=?NULL)??{
>      >??????>??????>>??????//?Make?an?unused?local?variable?to?avoid?warning?from?gcc?compiler.
>      >??????>??????>>??????size_t?count?=?fwrite(s,?1,?len,?_file);
>      >??????>??????>>????}
>      >??????>??????>>????update_position(s,?len);
>      >??????>??????>>?}
>      >??????>??????>>
>      >??????>??????>>?but?the?update_position?should?be?inside?the?if-statement?-?no?need?to
>      >??????>??????>>?see?a?new?webrev?for?that.
>      >??????>??????>
>      >??????>??????>?update_position?is?also?used?in?fdStream::write?
>      >??????>??????>?bufferedStream::write?and?stringStream::write,
>      >??????>??????>?and?according?to?the?comments?in?stringStream::write
>      >??????>??????>
>      >??????>??????>?=====
>      >??????>??????>????//?Note?that?the?following?does?not?depend?on?write_len.
>      >??????>??????>????//?This?means?that?position?and?count?get?updated
>      >??????>??????>????//?even?when?overflow?occurs.
>      >??????>??????>????update_position(s,?len);
>      >??????>??????>?=====
>      >??????>??????>
>      >??????>??????>?If?my?understanding?is?correct,?whether?or?not?to?call?update_position
>      >??????>??????>?does?not?depend?on?the?success?of?the?write?action.
>      >??????>
>      >??????>?????Every?fileStream?operation?should?be?a?no-op?if?_file?is?NULL.?Whether
>      >??????>?????other?streams?have?issues?is?a?different?matter.?It?makes?absolutely?no
>      >??????>?????sense?to?update?any?state?of?the?filestream?in?write?if?nothing?is?ever
>      >??????>?????going?to?examine?it?anyway?-?just?a?waste?of?time.
>      >??????>
>      >??????>?????Le's?see?if?there?are?any?other?opinions?on?this.
>      >??????>
>      >??????>?????Thanks,
>      >??????>?????David
>      >??????>?????-----
>      >??????>
>      >??????>??????>?If?there's?a?problem?here,?we?should?also?check?the?usage?of
>      >??????>??????>?update_position?in?other?methods?to?ensure?consistent?usage.
>      >??????>??????>
>      >??????>??????>?Hence,??I?think?we?shouldn't?put?update_position?inside?the
>      >??????>??????>?if-statement,?at?least?for?the?current?issue.
>      >??????>??????>
>      >??????>??????>
>      >??????>??????>
>      >??????>??????>
>      >??????>??????>?????------------------------------------------------------------------
>      >??????>??????>?????From:David?Holmes?<david.holmes at oracle.com>
>      >??????>??????>?????Send?Time:2020?4?12?(???)?15:04
>      >??????>??????>?????To:???(??)?<denghui.ddh at alibaba-
>     inc.com>;?hotspot-dev
>      >??????>??????>?????<hotspot-dev at openjdk.java.net>;?hotspot-runtime-dev at openjdk.java.net
>      >??????>??????>?????<hotspot-runtime-dev at openjdk.java.net>;?Yasumasa?Suenaga
>      >??????>??????>?????<suenaga at oss.nttdata.com>;?"Liu,?Xin"?<xxinliu at amazon.com>
>      >??????>??????>?????Subject:Re:?RFR(XXS):?8242485:?Null?_file?checking?in
>      >??????>??????>?????fileStream::flush()
>      >??????>??????>
>      >??????>??????>?????Hi?Denghui,
>      >??????>??????>
>      >??????>??????>?????On?12/04/2020?2:41?pm,?Denghui?Dong?wrote:
>      >??????>??????>??????>?Hi?David,
>      >??????>??????>??????>
>      >??????>??????>??????>?Thanks?for?the?comments
>      >??????>??????>??????>?new?webrev:?http://cr.openjdk.java.net/~ddong/8242485/webrev.02
>      >??????>??????>??????>
>      >??????>??????>??????>?when?_file?is?NULL:
>      >??????>??????>??????>?-?fileStream::read?return?0
>      >??????>??????>??????>?-?fileStream::readln?return?NULL
>      >??????>??????>??????>?-?fileStream::eof?return?-1?(like
>      >??????>??????>??????>?fileStream::fileSize,?not?sure?if?appropriate,?feof?returns?nonzero?if?target?file?is?end)
>      >??????>??????>
>      >??????>??????>?????That?is?all?great?-?thanks!
>      >??????>??????>
>      >??????>??????>?????Unfortunately?the?more?I?look?at?this?code?the?more?problems?I?see?with
>      >??????>??????>?????it.?:(?We?have:
>      >??????>??????>
>      >??????>??????>?????void?fileStream::write(const?char*?s,?size_t?len)?{
>      >??????>??????>?????????if?(_file?!=?NULL)??{
>      >??????>??????>???????????//?Make?an?unused?local?variable?to?avoid?warning?from?gcc?compiler.
>      >??????>??????>???????????size_t?count?=?fwrite(s,?1,?len,?_file);
>      >??????>??????>?????????}
>      >??????>??????>?????????update_position(s,?len);
>      >??????>??????>?????}
>      >??????>??????>
>      >??????>??????>?????but?the?update_position?should?be?inside?the?if-statement?-?no?need?to
>      >??????>??????>?????see?a?new?webrev?for?that.
>      >??????>??????>
>      >??????>??????>?????We?should?also?be?watching?for?errors?from?fwrite?-?but?that?in?itself
>      >??????>??????>?????just?opens?up?a?can-of-worms?with?regard?to?error?handling?in?general
>      >??????>??????>?????with?this?class,?so?that?can?be?left?as-is.
>      >??????>??????>
>      >??????>??????>?????Thanks,
>      >??????>??????>?????David
>      >??????>??????>?????-----
>      >??????>??????>
>      >??????>??????>??????>?Testing:?tier1
>      >??????>??????>??????>
>      >??????>??????>??????>?Cheers,
>      >??????>??????>??????>?Denghui?Dong
>      >??????>??????>??????>
>      >??????>??????>??????>?????------------------------------------------------------------------
>      >??????>??????>??????>?????From:David?Holmes?<david.holmes at oracle.com>
>      >??????>??????>??????>?????Send?Time:2020?4?12?(???)?10:40
>      >??????>??????>??????>?????To:???(??)?<denghui.ddh at alibaba-
>      >?????inc.com>;?hotspot-dev
>      >??????>??????>??????>?????<hotspot-dev at openjdk.java.net>;?hotspot-runtime-dev at openjdk.java.net
>      >??????>??????>??????>?????<hotspot-runtime-dev at openjdk.java.net>;?Yasumasa?Suenaga
>      >??????>??????>??????>?????<suenaga at oss.nttdata.com>;?"Liu,?Xin"?<xxinliu at amazon.com>
>      >??????>??????>??????>?????Subject:Re:?RFR(XXS):?8242485:?Null?_file?checking?in
>      >??????>??????>??????>?????fileStream::flush()
>      >??????>??????>??????>
>      >??????>??????>??????>?????Hi,
>      >??????>??????>??????>
>      >??????>??????>??????>?????On?12/04/2020?1:52?am,?Denghui?Dong?wrote:
>      >??????>??????>??????>??????>?Hi,
>      >??????>??????>??????>??????>?Thank?you?for?the?review?and?point?out?other?methods?need?to?check?_file.
>      >??????>??????>??????>??????>?In?addition?to?"fileStream::flush",?I?found?there?are?still?four?methods?need?to?check?_file?in?fileStream:?read,?readln,?eof,?rewind.
>      >??????>??????>??????>??????>?but?those?methods?will?crash?if?_file?is?NULL,?and?I?didn't?find?any?caller?of?them(Maybe?I?am?wrong),
>      >??????>??????>??????>??????>?so?I?think?it's?more?appropriate?to?add?assert?check?to?those?methods.
>      >??????>??????>??????>
>      >??????>??????>??????>?????Assertions?are?really?only?appropriate?if?they?will?be?exercised?by
>      >??????>??????>??????>?????tests?-?the?intent?being?that?tests?will?expose?any?cases?where?the
>      >??????>??????>??????>?????assertion?fails.
>      >??????>??????>??????>
>      >??????>??????>??????>?????Also?it?doesn't?make?sense?to?me?to?have?assertions?in?most?places?but?a
>      >??????>??????>??????>
>      >??????>??????>??????>?????true?NULL?check?in?others?-?that?suggests?we?expect?some?methods?to?be
>      >??????>??????>??????>?????called?when?_file?is?NULL,?but?others?not.?I'd?prefer?just?to?see?the
>      >??????>??????>??????>?????missing?NULL?checks?added,?rather?than?using?assertions.
>      >??????>??????>??????>
>      >??????>??????>??????>?????Thanks,
>      >??????>??????>??????>?????David
>      >??????>??????>??????>
>      >??????>??????>??????>??????>?Webrev:?http://cr.openjdk.java.net/~ddong/8242485/webrev.01/
>      >??????>??????>??????>??????>
>      >??????>??????>??????>??????>?Could?you?review?it?again,?and?sponsor?it?if?everything?is?okay?
>      >??????>??????>??????>??????>
>      >??????>??????>??????>??????>?Testing:?teir1
>      >??????>??????>??????>??????>?==============================
>      >??????>??????>??????>??????>?Test?summary
>      >??????>??????>??????>??????>?==============================
>      >??????>??????>??????>??????>?????TEST??????????????????????????????????????????????TOTAL??PASS??FAIL?ERROR
>      >??????>??????>??????>??????>?????jtreg:test/hotspot/jtreg:tier1?????????????????????1516??1516?????0?????0
>      >??????>??????>??????>??????>?????jtreg:test/jdk:tier1???????????????????????????????1904??1904?????0?????0
>      >??????>??????>??????>??????>??>>?jtreg:test/langtools:tier1?????????????????????????4031??4029?????2?????0?<<
>      >??????>??????>??????>??????>?????jtreg:test/nashorn:tier1??????????????????????????????0?????0?????0?????0
>      >??????>??????>??????>??????>?????jtreg:test/jaxp:tier1?????????????????????????????????0?????0?????0?????0
>      >??????>??????>??????>??????>?==============================
>      >??????>??????>??????>??????>?There?are?two?unrelated?test?failures?in
>      >??????>??????>??????>??????>?langtools:?jdk/javadoc/tool/CheckResourceKeys.java??and?tools/javac/processing/model/TestSymtabItems.java
>      >??????>??????>??????>??????>
>      >??????>??????>??????>??????>
>      >??????>??????>??????>??????>
>      >??????>??????>??????>??????>?????------------------------------------------------------------------
>      >??????>??????>??????>??????>?????From:Yasumasa?Suenaga?<suenaga at oss.nttdata.com>
>      >??????>??????>??????>??????>?????Send?Time:2020?4?11?(???)?09:22
>      >??????>??????>??????>??????>?????To:???(??)?<denghui.ddh at alibaba-
>      >??????>?????inc.com>;?hotspot-dev
>      >??????>??????>??????>??????>?????<hotspot-dev at openjdk.java.net>;?hotspot-runtime-dev at openjdk.java.net
>      >??????>??????>??????>??????>?????<hotspot-runtime-dev at openjdk.java.net>
>      >??????>??????>??????>??????>?????Subject:Re:?RFR(XXS):?8242485:?Null?_file?checking?in
>      >??????>??????>??????>??????>?????fileStream::flush()
>      >??????>??????>??????>??????>
>      >??????>??????>??????>??????>?????Hi?Denghui,
>      >??????>??????>??????>??????>
>      >??????>??????>??????>??????>?????null?check?lacks?in?other?place?too.
>      >??????>??????>??????>??????>?????Can?you?fix?it?
>      >??????>??????>??????>??????>
>      >??????>??????>??????>??????>?????????http://hg.openjdk.java.net/jdk/jdk/file/97d5d0cd1085/src/hotspot/share/utilities/ostream.cpp#l554
>      >??????>??????>??????>??????>
>      >??????>??????>??????>??????>
>      >??????>??????>??????>??????>?????Thanks,
>      >??????>??????>??????>??????>
>      >??????>??????>??????>??????>?????Yasumasa
>      >??????>??????>??????>??????>
>      >??????>??????>??????>??????>
>      >??????>??????>??????>??????>?????On?2020/04/10?21:34,?Denghui?Dong?wrote:
>      >??????>??????>??????>??????>??????>?Hi?team,
>      >??????>??????>??????>??????>??????>
>      >??????>??????>??????>??????>??????>?Could?you?please?review?this?small?patch?
>      >??????>??????>??????>??????>??????>
>      >??????>??????>??????>??????>??????>?JBS:?https://bugs.openjdk.java.net/browse/JDK-8242485
>      >??????>??????>??????>??????>??????>?webrev:?http://cr.openjdk.java.net/~ddong/8242485/webrev.00/
>      >??????>??????>??????>??????>??????>
>      >??????>??????>??????>??????>??????>?I?found?the?method?"fileStream::flush()"?lacks?null?check?before?fflush??which?will?cause
>      >??????>??????>??????>??????>??????>?all?open?output?streams?are?flushed?by?fflush()?if?_file?is?NULL.
>      >??????>??????>??????>??????>??????>
>      >??????>??????>??????>??????>??????>?Thanks,
>      >??????>??????>??????>??????>??????>?Denghui?Dong
>      >??????>??????>??????>??????>??????>
>      >??????>??????>??????>??????>
>      >??????>??????>??????>
>      >??????>??????>
>      >??????>
>      >??????>
>      > 
> 

From david.holmes at oracle.com  Wed Apr 15 02:26:02 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 15 Apr 2020 12:26:02 +1000
Subject: [trivial] RFR: 8242631: Missing but used special functions for
 some classes
In-Reply-To: <13190093-8B8E-4D9C-804A-F130B9C3C2EF@oracle.com>
References: <13190093-8B8E-4D9C-804A-F130B9C3C2EF@oracle.com>
Message-ID: <80185f2d-e428-ebb9-9117-a34af96e8e0b@oracle.com>

Hi Kim,

Looks fine and trivial.

Thanks,
David

On 14/04/2020 11:02 pm, Kim Barrett wrote:
> Please review this trivial change to a couple of classes to eliminate
> gcc9 deprecation warnings for them when compiling for C++11 or later.
> See the CR for more details about the warnings.
> 
> BCEscapeAnalyzer::ArgumentMap has an implicitly defined copy
> constructor and a user-defined copy assignment operator.  Since the
> copy assignment operator is functionally equivalent to the default,
> removed it.
> 
> Various subtypes of oop had an implicitly defined copy constructor;
> made it explicit.
> 
> There are a few more like these, but I may address them differently.
> 
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8242631
> 
> Webrev:
> https://cr.openjdk.java.net/~kbarrett/8242631/open.00/
> 
> Testing:
> mach5 tier1
> 
> 

From david.holmes at oracle.com  Wed Apr 15 05:28:40 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 15 Apr 2020 15:28:40 +1000
Subject: RFR (S) 8074292: nsk/jdb/kill/kill001: generateOopMap.cpp
 assert(bb->is_reachable()) failed
In-Reply-To: <1e3d4907-d54a-df0e-e0d3-8bbda8c5394a@oracle.com>
References: <f66b8de6-d9db-a90f-bfc2-5bf42a19cada@oracle.com>
 <6e38070c-f166-a0c2-8b8d-d698e5f5bcf3@oracle.com>
 <1e3d4907-d54a-df0e-e0d3-8bbda8c5394a@oracle.com>
Message-ID: <0a87b7cd-7dfe-0831-05e1-8d48f5eb6755@oracle.com>

On 14/04/2020 10:36 pm, coleen.phillimore at oracle.com wrote:
> On 4/13/20 10:49 PM, David Holmes wrote:
>> Hi Coleen,
>>
>> On 14/04/2020 12:34 am, coleen.phillimore at oracle.com wrote:
>>> Summary: Do not install async exceptions at_safepoint for each bytecode.
>>
>> I'm still not certain that we have to go this far to solve this 
>> problem, but it does sound like a relatively simple solution provided 
>> there are no unintended consequences.
>>
>>> See CR for a lot more details.? This change calls a new 
>>> InterpreterRuntime::at_safepoint_async_safe() which installs the 
>>> async exception in the interpreter at backward branches and returns. 
>>> This uses safepoint polling code in the interpreter for each 
>>> platform.? These changes (cross) compile on platforms that Oracle 
>>> doesn't support but I don't know if they work.
>>>
>>> I'm not convinced the platform specific changes are necessary, 
>>> because calls to the runtime from many bytecodes will install the 
>>> async exception, so it's essentially installed "enough" for this 
>>> deprecated feature.? I tested the changes with *and* without the 
>>> platform specific changes with no failure, which included the jdb, 
>>> jdi and jvmti serviceability tests.
>>
>> I don't understand what you mean here. If the whole basis of this fix 
>> is "don't install async exceptions other than at backward branches and 
>> returns" then how is that implemented without the changes in the 
>> interpreter code?
>>
>> If this can be fixed just by adjusting the actual monitor code then I 
>> would much prefer that. It took me a while to get my head around the 
>> dispatch changes in interpreter code and even then I don't see how 
>> those changes only impact backward branches and returns ??
> 
> You have to read the comments in the bug again.? There *is* special code 
> to not install the async exception in the monitorexit code. That is not 
> enough to prevent this bug.? You can also read the old bug report you 
> linked to this one.

I know there is some special handling in monitorexit already, I was 
referring to additional special handling around the monitorexit to 
disable whatever piece of code is currently installing the async exception.

> The interpreter code dispatch_next passes "true" if it's a backwards 
> branch, that's how it can tell.

Okay - I see dispatch_next is passed generate_poll=true for returns but 
I can't see where backward branches come into play. Your changes cause 
the at_safepoint_async_safe to be called in that case, whereas any other 
code paths that lead to at_safepoint no longer install the async 
exception. So things are a bit clearer in that regard.

> My point was that there are enough code paths that install async 
> exceptions *other than monitorenter and monitorexit* that maybe it's not 
> necessary to install them at backwards branches and returns.? I suppose 
> someone could construct a test case to show otherwise.

Yes I understood that was the query, but without knowing exactly where 
those code paths are it is hard to comment on whether there is adequate 
coverage. Even with installing on backward branches there is a risk that 
small loops can be unrolled and so lose the check (indeed that is what 
"counted loops" in the JIT does) and thus we rely on these other code 
paths anyway.

>>> This change also makes InterpreterRuntime::monitorexit a JRT_LEAF 
>>> bytecode. The code to check for exceptions is outside the runtime 
>>> call. I ran the JCK vm and lang tests on this change with no failure.
>>>
>>> Tested with tier1-6.
>>>
>>> open webrev at 
>>> http://cr.openjdk.java.net/~coleenp/2020/8074292.01/webrev
>>> bug link https://bugs.openjdk.java.net/browse/JDK-8074292
>>
>> ./cpu/x86/interp_masm_x86.cpp
>>
>> It took me a long time to figure out how the new logic worked compared 
>> to the old logic. Even then I'm unclear about the effective recursive 
>> dispatch path: dispatch_base(generate_poll=true) -> dispatch_via -> 
>> dispatch_base(generate_poll=false) - does it work okay with 
>> VerifyActivationFrameSize? It seems a rather convoluted way to 
>> effectively just execute:
>>
>> ?858?? lea(rscratch1, ExternalAddress((address)table));
>> ?859?? jmp(Address(rscratch1, rbx, Address::times_8));
> 
> I could test it with VerifyActivationFrameSize.? Or I could just add the 
> two instructions per platform.? I mostly copied the code in 
> generate_safept_entry_for.? It might be better to just copy the 
> instructions.
> 
>>
>> ---
>>
>> src/hotspot/share/interpreter/interpreterRuntime.cpp
>>
>> How were you able to drop this code:
>>
>> 791?? if (elem == NULL || h_obj()->is_unlocked()) {
>> 792 THROW(vmSymbols::java_lang_IllegalMonitorStateException());
>> 793?? }
>>
>> ?
>> ?and this:
> 
> The caller throws the exception for these cases.

Sorry but I don't see that in the caller e.g. 
InterpreterMacroAssembler::unlock_object. Further you are not allowing 
for elem to be NULL which will lead to a crash when you dereference it.

>>
>> 798 #ifdef ASSERT
>> 799 thread->last_frame().interpreter_frame_verify_monitor(elem);
>> 800 #endif
>>
> 
> This seemed redundant.

Possibly, but I assume the pairing in monitorenter and monitorexit 
around the actual monitor operation is to ensure that the stack frame 
info is not unexpectedly messed up. Maybe it is redundant to call before 
and after, but then maybe the same is true in monitorenter.

David
-----

> Coleen
>> ?
>>
>> Thanks,
>> David
>>
>>> Thanks,
>>> Coleen
>>>
> 

From david.holmes at oracle.com  Wed Apr 15 05:45:09 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 15 Apr 2020 15:45:09 +1000
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <df621fd2-0eaf-cb24-788f-12166b271303@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
 <135a6496-5b54-9117-5cbf-e3211ddc32dd@oracle.com>
 <bbb2b5ef-62f2-6507-95f7-03712a382153@oracle.com>
 <e242e057-de6e-493a-eb00-068b2db8c9fa@oracle.com>
 <df621fd2-0eaf-cb24-788f-12166b271303@oracle.com>
Message-ID: <91f6fba9-56c1-31d7-12de-4f2b75d38ede@oracle.com>

Hi Leo,

Before I look at the webrev one query ..

<trimming>

On 14/04/2020 11:53 pm, Leo Korinth wrote:
> On 06/04/2020 01:37, David Holmes wrote:
>>> I could change CHECK_0 to use NULL_WORD though and if I do, I get 
>>> compiler errors. CHECK_0 is used for both pointers and integral 
>>> zeroes and types that implicitly converts from either. What the 
>>> original thought was is unclear to me; is CHECK_0 to be used to 
>>> return things convertible from pointers? If so, what is CHECK_NULL for?
>>
>> No CHECK_0 is intended for int-returning functions (which would also 
>> extend to intptr_t).
> 
> It might be intended for it, but it is not used in such a way (therefore 
> my remark on compile failures when using NULL_WORD)
> 
> I will do no change here.

Please explain what you mean by "it is not used in such a way". Other 
than a few misuses in ./share/jvmci/jvmciCompilerToVM.cpp where it is 
used for jboolean and the jobject case you are fixing, it is used in 
methods that return jint, jbyte, jlong ...

Thanks,
David
-----

From ioi.lam at oracle.com  Wed Apr 15 06:04:46 2020
From: ioi.lam at oracle.com (Ioi Lam)
Date: Tue, 14 Apr 2020 23:04:46 -0700
Subject: RFC 8236988 - Modular Design for JVM Flags
In-Reply-To: <A01A906F-392D-4105-8868-EF76B2A278AE@amazon.com>
References: <6EE91323-A69B-44E1-8215-411250EB16DD@amazon.com>
 <e0a4bcc5-e116-c901-85f9-78360307e243@oracle.com>
 <A01A906F-392D-4105-8868-EF76B2A278AE@amazon.com>
Message-ID: <369acb14-38d6-c2e6-aeb5-7285b62b16fc@oracle.com>

Hi Xin,

Thanks for your comments. I have uploaded a new prototype that no longer 
requires the DEFN_XXX_FLAG macros, so we don't need any of the new 
xxx_globals.cpp files anymore. You will still need one new file 
(allGlobals.cpp) file to instantiate all the flags.

http://cr.openjdk.java.net/~iklam/design/jvm_flags_overhaul/design-09a/

 ??? make run
 ??? make rung

 > There was a high-order macro RUNTIME_FLAGS in previous globle.hpp.
 > That's why it avoids to iterate options again & again. C-preprocessor
 > did the "codegen" job by expanding macros.
 >
 > Why do you ditch that design? it still can archive all your design goals.

As I explained in the design wiki page, my foremost goal is to 
modularize the flags.

https://wiki.openjdk.java.net/display/HotSpot/HotSpot+Command-Line+Flags+Overhaul+-+Design+Doc

I want to avoid using a single giant macro that has all the flags. Such 
a macro makes it impossible to include just the flags that a particular 
.cpp file needs. Today, every .cpp or .inline.hpp files that needs just 
a single flag must include the declaration of all 1000+ flags.

Another advantage is now flags can be easily excluded like

 ??? #if INCLUDE_CDS
 ??? PRODUCT_FLAG(bool, UseSharedSpaces, ...);
 ??? #endif

This was possible before, but was extremely clumsy. So we ended up no 
doing any fine-grained subdivision and globals.hpp became the kitchen 
sink of 600+ flags.

The downside is, as you observed, it's no longer possible to put all the 
flags in a simple C++ array. I have to build a linked list during VM 
bootstrap. But that has just the cost of 1 instruction per flag. The C++ 
compiler/linker is able to fold the linked-list building to simple 
assignments:

 ???? A->next = &B;
 ???? B->next = &C;
 ???? C->next = &D;
 ???? D->next = &E;
 ???? E->next = head;
 ?? ? head = A;

And the computation of num_flags is folded into a single assignment

 ???? num_flags += 5;

You can see this from the disassembly of the "jvm" executable from the 
new prototype. I tested with both gcc versions 5.4, 7.3 and 8.3.

So I believe the cost is negligible.

Another advantage of not using a simple C++ array is, now the FLAG_XXX 
instances can have variable size. E.g., I have

 ??? template <typename T> class ModifiableFlag ...{
 ? ? ?? T _min;
 ?????? T _max;
 ???? }

so this will use less space for the smaller-sized flags such as bool and 
int.

If you want to use a C++ array of variable-sized elements, you will end 
up doing this:

 ??? JVMFlag* flags[] = {
 ??????? & FLAG_A,
 ??????? & FLAG_B,
 ??????? & FLAG_C, ...
 ??? };

but this will compiled to the same amount of instructions as the linked 
list ....

BTW, my plan is, after the initial integration, to build a hash table of 
the flags by statically calculating the hashcodes during build time. 
This should generate the exact same number of instructions as the 
current design (except there are 129 different heads instead of a single 
head). So the performance should be comparable/faster than an 
alphabetically sorted table. This can be done with fancy macros, or 
better, with C++ constexpr that will soon be available for HotSpot.

What do you think?

thanks
- Ioi

On 4/14/20 12:28 PM, Liu, Xin wrote:
> Hi, Chris,
>
> Your project is very interesting. Nice work! I've starred it.
> Ioi also show me some useful flags I don't know that. thanks!
>
> Hi, Ioi,
> I got some comments inline.
>
> ?On 4/12/20, 5:22 PM, "Ioi Lam" <ioi.lam at oracle.com> wrote:
>
>      CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>      
>      
>      
>      Hi Xin,
>      
>      Thanks for the feedback. It's very helpful.
>      
>      On 4/12/20 12:59 AM, Liu, Xin wrote:
>      > Hi, Ioi,
>      >
>      > I'm recently refactoring intrinsics options for compiler[1]. I introduced a new option type called "intrinsics".  The biggest trouble was that I had to modify multiple places.  I found your patch beautifully replaces jvmConstraintList.* &jvmFlagRangeList.* with some macros in jvmFlag.inline.hpp. great!
>      >
>      > I have some questions or suggests.
>      >
>      > 1. generate a document of options
>      > There're hundreds of JVM options.  It's hard to learn them by heart. I feel developers uses globals.hpp and sub header files as a de factor manual.
>      > if you break up the monolithic header, which is good,  I think developers deserve a manual.
>      > Off the top of my head, I think you can provide a mini tool, which consume a header aggerates all sub-header files and automatically generates a manual.
>      
>      Even today the flags are scattered around in various *.hpp files. There
>      are already ways of finding information about the all the flags:
>      
>           java -XX:+PrintFlagsWithComments   (debug build only)
>           java -XX:+PrintFlagsInitial
>           java -XX:+PrintFlagsFinal
>           java -XX:+PrintFlagsRanges
>      
>      These flags will continue to work after my patch. Would these be sufficient?
>      
>      Also, you can use this to look for a flag:
>      
>           find . -name \*.hpp | xargs grep FLAG.*SplitIfBlocks
>      
>      > 2. change FLAG_RANGE and FLAG_CONTRAINT to optional parameters of PRODUCT_FLAG.
>      > If you use variadic macro, it's possible to select RANGE or CONTRAINT or NIL from __VA_ARGS__.
>      > I used to propose it before. https://cr.openjdk.java.net/~xliu/8230669/00/webrev/src/hotspot/share/utilities/macro_overloading.hpp.html
>      >
>      > I think it's a good idea because it make the declaration section pure.
>      > If we replace those macros eg. PRODUCT_FLAG with C++ classes of same names, it's very easy to write a standalone tool to generate a manual of JVM options from header files, right?
>      
>      That's a good suggestion. I've uploaded an updated version of the design
>      that uses __VA_ARGS__
>      
>      http://cr.openjdk.java.net/~iklam/design/jvm_flags_overhaul/flags.hpp
>      http://cr.openjdk.java.net/~iklam/design/jvm_flags_overhaul/flags.cpp
>      
>      To see how the macros work, try doing this, and look at the file flags.ii
>      
>           gcc -DPRODUCT -O3 -o flags -save-temps flags.cpp && ./flags
>           gcc -g -o flags_g -save-temps flags.cpp && ./flags_g
>      
> --
> Oh, you use overloaded constructors to select optional parameters.
> Interesting, I didn't come up with that idea.
> --
>      I think we can try different designs using the above as a template. When
>      we have a more concrete design I can implement it in the VM.
>      
>      Now the declaration is:
>      
>           PRODUCT_FLAG(bool,     SplitIfBlocks, true, JVMFlag::DEFAULT,
>                                  "Clone compares and control flow through merge "
>                                  "points to fold some branches");
>      
>           DEVELOP_FLAG(intx,     FreqCountInvocations, 1, JVMFlag::DEFAULT,
>                                  "Scaling factor for branch frequencies",
>                                  FLAG_RANGE(1, max_intx))
>      
>           PRODUCT_FLAG(intx,     NodeLimitFudgeFactor, 2000, JVMFlag::CONSTRAINT,
>                                  "Fudge Factor for certain optimizations",
>      FLAG_CONSTRAINT(NodeLimitFudgeFactorConstraintFunc,
>                                  JVMFlag::AfterErgo))
>      
>           PRODUCT_FLAG(intx,     AliasLevel, 3, JVMFlag::DEFAULT,
>                                  "0 for no aliasing, 1 for .... split, "
>                                  "2 for class split, 3 for unique instances",
>                                  FLAG_RANGE(0, 3),
>      FLAG_CONSTRAINT(AliasLevelConstraintFunc,
>                                  JVMFlag::AfterErgo))
>      
>      and the implementation:
>      
>           DEFN_PRODUCT_FLAG(SplitIfBlocks);
>           DEFN_DEVELOP_FLAG(FreqCountInvocations);
>           DEFN_PRODUCT_FLAG(NodeLimitFudgeFactor);
>           DEFN_PRODUCT_FLAG(AliasLevel);
>      
>      My goal is still to put all the information about each flag in a single
>      place (the hpp file), so I need to pass the info to the cpp files using
>      an inline function:
>      
>         inline const ModifiableFlag<FLAG_TYPE_##name> init_##name() { \
>           return ModifiableFlag<FLAG_TYPE_##name>( \
>             JVMFlag::TYPE_##type, #name, attr, &name, __VA_ARGS__); \
>         }
>      
>      But the problem is this inline function could theoretically allocate the
>      instance on the stack, so I cannot save its address into the _head of
>      the global flags list. As I result, I needed to split the _next pointer
>      from JVMFlag into AbstractFlagLink::_link.
>      
>      If anyone has a better suggestion, I would be happy to try it out.
> --
> You can't get addresses of temps, but you can get addresses of static variables.  Is this trick useful for your aim?
>
> Eg.
>         inline const ModifiableFlag<FLAG_TYPE_##name>* init_##name() { \
>           static ModifiableFlag<FLAG_TYPE_##name>( \
>             JVMFlag::TYPE_##type, #name, attr, &name, __VA_ARGS__) t; \
>             Return &t;                                                                                              \
>         }
>
> Maybe Linkedlist isn't your only choice. I use enum to count how many options JVM will have and allocate a static array.
> In this way, I can even sort the array and use binary-search.
> Check out line #89 of https://cr.openjdk.java.net/~xliu/8151779/00/webrev/src/hotspot/share/compiler/intrinsics.cpp.html
> --
>     
>      > 2. There's a TCL script.  Isn't C Processor powerful enough for it?  I don't know TCL, I think an external script will bring dependency and learning  burden.
>      >
>      > 3. let's say we choose a script generator. It looks like those globals_xxx.cpp files are auto-gen files.
>      > Will you consider to untrack those generated files in the repo?
>      
>      The whole purpose of the TCL script is to convert the old style macros
>      in the hpp files into the new coding style. That way I can keep track of
>      changes in the upstream repo, and also easily change the design of the
>      new code. The script is used by me only (that's why it's messy). It's
>      not used during the build and will not be needed in the final patch.
>
>      The globals_xxx.cpp files will be checked into the repo. Afterwards, to
>      add/remove flags, you would need to edit the xxx_globals.hpp/cpp files
>      directly.
>      
>      As mentioned in the design doc, my goal is to avoid C++ code generation
>      as many people are against it.
>      
>      Personally, I prefer code generation. I think that can result in a
>      smaller binary footprint that will start up faster, but the delta is
>      probably negligible. We can probably leave that for the future. As long
>      as we use the macros in a consistent manner, we can easily extract the
>      flag definitions and put them into external input files that are
>      processed by a C++ code generator.
> ---
> But your current webrev leads to have a codegen path.
>
> You have to iterate all options at least twice, right? one is in header file, the other one is in the cpp file.
> The list is long and the order doesn't matter. I can't imagine you will check in those hpp/cpp pairs.  It's very natural to have a script like tcl or m4 to generate cpp from hpp, right?
> There was a high-order macro RUNTIME_FLAGS in previous globle.hpp. That's why it avoids to iterate options again & again. C-preprocessor did the "codegen" job by expanding macros.
>
> Why do you ditch that design? it still can archive all your design goals.
> ---
>      > 4. Probably, it's not a bad idea to still use product/develop instead of PRODUCT_FLAG, DEVELOP_FLAG.
>      > Those options will be read everyday by developers, I feel lower-case names are more friendly.
>      
>      I don't have a strong preference, but I think that adding "product"
>      globally (even as a macro) may cause problems with code like this:
>      
>           static int product(int a, int b) {return a*b;}
>      
>      as the C pre-processor would try to expand "product".
> --
> In previous globals.hpp, 'product' is just a macro parameter.  That's why current implementation doesn't have any conflict with the function product.
> --
>
>      Thanks
>      - Ioi
>      
>      
>      >
>      > [1]: https://cr.openjdk.java.net/~xliu/8151779/00/webrev/
>      >
>      > Thanks,
>      > --lx
>      >
>      > On 4/9/20, 5:08 PM, "hotspot-dev on behalf of Ioi Lam" <hotspot-dev-bounces at openjdk.java.net on behalf of ioi.lam at oracle.com> wrote:
>      >
>      >      CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>      >
>      >
>      >
>      >      The code for specifying HotSpot command-line flags used to be simple. At
>      >      the very beginning it was something like
>      >
>      >           #define ALL_FLAGS(develop, develop) \
>      >              product(int, SomeFlag, def_value, "some doc") \
>      >              develop(int, MoreFlag, def_value, "more doc") \
>      >
>      >      but like all good ideas, it has gradually degraded into a mess [1][2].
>      >      It's now very difficult to add any new functionality (e.g., add
>      >      "manageable" annotations to the flags [3]).
>      >
>      >      I think it's time for a complete rewrite :-)
>      >
>      >      My proposal is here. Please comment if it's a good or bad idea.
>      >
>      >      https://wiki.openjdk.java.net/display/HotSpot/HotSpot+Command-Line+Flags+Overhaul+-+Design+Doc
>      >
>      >
>      >
>      >      tldr; ---
>      >
>      >
>      >      OLD:
>      >
>      >         product(intx, AliasLevel, 3,                                          \
>      >                 "0 for no aliasing, 1 for oop/field/static/array split,
>      >      "         \
>      >                 "2 for class split, 3 for unique
>      >      instances")                      \
>      >                 range(0, 3)
>      >      /*optional*/                                         \
>      >                 constraint(AliasLevelConstraintFunc,AfterErgo) /*optional*/
>      >           \
>      >
>      >
>      >      NEW:
>      >
>      >         PRODUCT_FLAG(intx,   AliasLevel, 3, JVMFlag::RANGE | JVMFlag::CONSTRAINT,
>      >                              "0 for no aliasing, 1 for oop/field/static/array
>      >      split, "
>      >                              "2 for class split, 3 for unique instances");
>      >          // optional
>      >          FLAG_RANGE(         AliasLevel, 0, 3);
>      >          // optional
>      >          FLAG_CONSTRAINT(    AliasLevel, (void*)AliasLevelConstraintFunc,
>      >      JVMFlag::AfterErgo);
>      >
>      >
>      >
>      >
>      >      [1]
>      >      http://hg.openjdk.java.net/jdk/jdk/file/7af6364e1792/src/hotspot/share/runtime/globals.hpp#l115
>      >      [2]
>      >      http://hg.openjdk.java.net/jdk/jdk/file/7af6364e1792/src/hotspot/share/runtime/flags/jvmFlag.cpp#l635
>      >      [3] https://bugs.openjdk.java.net/browse/JDK-7123237
>      >
>      >      Thanks
>      >      - Ioi
>      >
>      >
>      
>      
>


From xxinliu at amazon.com  Wed Apr 15 22:42:12 2020
From: xxinliu at amazon.com (Liu, Xin)
Date: Wed, 15 Apr 2020 22:42:12 +0000
Subject: RFC 8236988 - Modular Design for JVM Flags
In-Reply-To: <369acb14-38d6-c2e6-aeb5-7285b62b16fc@oracle.com>
References: <6EE91323-A69B-44E1-8215-411250EB16DD@amazon.com>
 <e0a4bcc5-e116-c901-85f9-78360307e243@oracle.com>
 <A01A906F-392D-4105-8868-EF76B2A278AE@amazon.com>
 <369acb14-38d6-c2e6-aeb5-7285b62b16fc@oracle.com>
Message-ID: <C8428A81-4F1A-4D53-A5D4-D93E864075C2@amazon.com>



?On 4/14/20, 11:07 PM, "Ioi Lam" <ioi.lam at oracle.com> wrote:

    CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
    
    
    
    Hi Xin,
    
    Thanks for your comments. I have uploaded a new prototype that no longer
    requires the DEFN_XXX_FLAG macros, so we don't need any of the new
    xxx_globals.cpp files anymore. You will still need one new file
    (allGlobals.cpp) file to instantiate all the flags.
    
    http://cr.openjdk.java.net/~iklam/design/jvm_flags_overhaul/design-09a/
    
         make run
         make rung
---
I also thought you could use repeating inclusion if you don't like high-order macro. 
I hesitated because I feel people don't like tricks here.  
Actually, I feel it's a perfect solution now because it fits your design goal. allGlobals.cpp is smart.
your prototype looks awesome to me. 
---
     > There was a high-order macro RUNTIME_FLAGS in previous globle.hpp.
     > That's why it avoids to iterate options again & again. C-preprocessor
     > did the "codegen" job by expanding macros.
     >
     > Why do you ditch that design? it still can archive all your design goals.
    
    As I explained in the design wiki page, my foremost goal is to
    modularize the flags.
    
    https://wiki.openjdk.java.net/display/HotSpot/HotSpot+Command-Line+Flags+Overhaul+-+Design+Doc
    
    I want to avoid using a single giant macro that has all the flags. Such
    a macro makes it impossible to include just the flags that a particular
    .cpp file needs. Today, every .cpp or .inline.hpp files that needs just
    a single flag must include the declaration of all 1000+ flags.
    
    Another advantage is now flags can be easily excluded like
    
         #if INCLUDE_CDS
         PRODUCT_FLAG(bool, UseSharedSpaces, ...);
         #endif
    
    This was possible before, but was extremely clumsy. So we ended up no
    doing any fine-grained subdivision and globals.hpp became the kitchen
    sink of 600+ flags.
    
    The downside is, as you observed, it's no longer possible to put all the
    flags in a simple C++ array. I have to build a linked list during VM
    bootstrap. But that has just the cost of 1 instruction per flag. The C++
    compiler/linker is able to fold the linked-list building to simple
    assignments:
    
          A->next = &B;
          B->next = &C;
          C->next = &D;
          D->next = &E;
          E->next = head;
          head = A;
    
    And the computation of num_flags is folded into a single assignment
    
          num_flags += 5;
    
    You can see this from the disassembly of the "jvm" executable from the
    new prototype. I tested with both gcc versions 5.4, 7.3 and 8.3.
    
    So I believe the cost is negligible.
    
    Another advantage of not using a simple C++ array is, now the FLAG_XXX
    instances can have variable size. E.g., I have
    
         template <typename T> class ModifiableFlag ...{
            T _min;
            T _max;
          }
    
    so this will use less space for the smaller-sized flags such as bool and
    int.
    
    If you want to use a C++ array of variable-sized elements, you will end
    up doing this:
    
         JVMFlag* flags[] = {
             & FLAG_A,
             & FLAG_B,
             & FLAG_C, ...
         };
    
    but this will compiled to the same amount of instructions as the linked
    list ....
    
    BTW, my plan is, after the initial integration, to build a hash table of
    the flags by statically calculating the hashcodes during build time.
    This should generate the exact same number of instructions as the
    current design (except there are 129 different heads instead of a single
    head). So the performance should be comparable/faster than an
    alphabetically sorted table. This can be done with fancy macros, or
    better, with C++ constexpr that will soon be available for HotSpot.
    
    What do you think?
--
Good idea.  I really think it's very possible in C/C++. It can be phrase-2.

I have seen that you actually use C++ template parameters for ModifiableFlag.
Without hashtable, each type still can have its own list,  so you can have NUM_TYPES heads. 

Thanks,
--lx
--
    thanks
    - Ioi
    
    On 4/14/20 12:28 PM, Liu, Xin wrote:
    > Hi, Chris,
    >
    > Your project is very interesting. Nice work! I've starred it.
    > Ioi also show me some useful flags I don't know that. thanks!
    >
    > Hi, Ioi,
    > I got some comments inline.
    >
    > On 4/12/20, 5:22 PM, "Ioi Lam" <ioi.lam at oracle.com> wrote:
    >
    >      CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
    >
    >
    >
    >      Hi Xin,
    >
    >      Thanks for the feedback. It's very helpful.
    >
    >      On 4/12/20 12:59 AM, Liu, Xin wrote:
    >      > Hi, Ioi,
    >      >
    >      > I'm recently refactoring intrinsics options for compiler[1]. I introduced a new option type called "intrinsics".  The biggest trouble was that I had to modify multiple places.  I found your patch beautifully replaces jvmConstraintList.* &jvmFlagRangeList.* with some macros in jvmFlag.inline.hpp. great!
    >      >
    >      > I have some questions or suggests.
    >      >
    >      > 1. generate a document of options
    >      > There're hundreds of JVM options.  It's hard to learn them by heart. I feel developers uses globals.hpp and sub header files as a de factor manual.
    >      > if you break up the monolithic header, which is good,  I think developers deserve a manual.
    >      > Off the top of my head, I think you can provide a mini tool, which consume a header aggerates all sub-header files and automatically generates a manual.
    >
    >      Even today the flags are scattered around in various *.hpp files. There
    >      are already ways of finding information about the all the flags:
    >
    >           java -XX:+PrintFlagsWithComments   (debug build only)
    >           java -XX:+PrintFlagsInitial
    >           java -XX:+PrintFlagsFinal
    >           java -XX:+PrintFlagsRanges
    >
    >      These flags will continue to work after my patch. Would these be sufficient?
    >
    >      Also, you can use this to look for a flag:
    >
    >           find . -name \*.hpp | xargs grep FLAG.*SplitIfBlocks
    >
    >      > 2. change FLAG_RANGE and FLAG_CONTRAINT to optional parameters of PRODUCT_FLAG.
    >      > If you use variadic macro, it's possible to select RANGE or CONTRAINT or NIL from __VA_ARGS__.
    >      > I used to propose it before. https://cr.openjdk.java.net/~xliu/8230669/00/webrev/src/hotspot/share/utilities/macro_overloading.hpp.html
    >      >
    >      > I think it's a good idea because it make the declaration section pure.
    >      > If we replace those macros eg. PRODUCT_FLAG with C++ classes of same names, it's very easy to write a standalone tool to generate a manual of JVM options from header files, right?
    >
    >      That's a good suggestion. I've uploaded an updated version of the design
    >      that uses __VA_ARGS__
    >
    >      http://cr.openjdk.java.net/~iklam/design/jvm_flags_overhaul/flags.hpp
    >      http://cr.openjdk.java.net/~iklam/design/jvm_flags_overhaul/flags.cpp
    >
    >      To see how the macros work, try doing this, and look at the file flags.ii
    >
    >           gcc -DPRODUCT -O3 -o flags -save-temps flags.cpp && ./flags
    >           gcc -g -o flags_g -save-temps flags.cpp && ./flags_g
    >
    > --
    > Oh, you use overloaded constructors to select optional parameters.
    > Interesting, I didn't come up with that idea.
    > --
    >      I think we can try different designs using the above as a template. When
    >      we have a more concrete design I can implement it in the VM.
    >
    >      Now the declaration is:
    >
    >           PRODUCT_FLAG(bool,     SplitIfBlocks, true, JVMFlag::DEFAULT,
    >                                  "Clone compares and control flow through merge "
    >                                  "points to fold some branches");
    >
    >           DEVELOP_FLAG(intx,     FreqCountInvocations, 1, JVMFlag::DEFAULT,
    >                                  "Scaling factor for branch frequencies",
    >                                  FLAG_RANGE(1, max_intx))
    >
    >           PRODUCT_FLAG(intx,     NodeLimitFudgeFactor, 2000, JVMFlag::CONSTRAINT,
    >                                  "Fudge Factor for certain optimizations",
    >      FLAG_CONSTRAINT(NodeLimitFudgeFactorConstraintFunc,
    >                                  JVMFlag::AfterErgo))
    >
    >           PRODUCT_FLAG(intx,     AliasLevel, 3, JVMFlag::DEFAULT,
    >                                  "0 for no aliasing, 1 for .... split, "
    >                                  "2 for class split, 3 for unique instances",
    >                                  FLAG_RANGE(0, 3),
    >      FLAG_CONSTRAINT(AliasLevelConstraintFunc,
    >                                  JVMFlag::AfterErgo))
    >
    >      and the implementation:
    >
    >           DEFN_PRODUCT_FLAG(SplitIfBlocks);
    >           DEFN_DEVELOP_FLAG(FreqCountInvocations);
    >           DEFN_PRODUCT_FLAG(NodeLimitFudgeFactor);
    >           DEFN_PRODUCT_FLAG(AliasLevel);
    >
    >      My goal is still to put all the information about each flag in a single
    >      place (the hpp file), so I need to pass the info to the cpp files using
    >      an inline function:
    >
    >         inline const ModifiableFlag<FLAG_TYPE_##name> init_##name() { \
    >           return ModifiableFlag<FLAG_TYPE_##name>( \
    >             JVMFlag::TYPE_##type, #name, attr, &name, __VA_ARGS__); \
    >         }
    >
    >      But the problem is this inline function could theoretically allocate the
    >      instance on the stack, so I cannot save its address into the _head of
    >      the global flags list. As I result, I needed to split the _next pointer
    >      from JVMFlag into AbstractFlagLink::_link.
    >
    >      If anyone has a better suggestion, I would be happy to try it out.
    > --
    > You can't get addresses of temps, but you can get addresses of static variables.  Is this trick useful for your aim?
    >
    > Eg.
    >         inline const ModifiableFlag<FLAG_TYPE_##name>* init_##name() { \
    >           static ModifiableFlag<FLAG_TYPE_##name>( \
    >             JVMFlag::TYPE_##type, #name, attr, &name, __VA_ARGS__) t; \
    >             Return &t;                                                                                              \
    >         }
    >
    > Maybe Linkedlist isn't your only choice. I use enum to count how many options JVM will have and allocate a static array.
    > In this way, I can even sort the array and use binary-search.
    > Check out line #89 of https://cr.openjdk.java.net/~xliu/8151779/00/webrev/src/hotspot/share/compiler/intrinsics.cpp.html
    > --
    >
    >      > 2. There's a TCL script.  Isn't C Processor powerful enough for it?  I don't know TCL, I think an external script will bring dependency and learning  burden.
    >      >
    >      > 3. let's say we choose a script generator. It looks like those globals_xxx.cpp files are auto-gen files.
    >      > Will you consider to untrack those generated files in the repo?
    >
    >      The whole purpose of the TCL script is to convert the old style macros
    >      in the hpp files into the new coding style. That way I can keep track of
    >      changes in the upstream repo, and also easily change the design of the
    >      new code. The script is used by me only (that's why it's messy). It's
    >      not used during the build and will not be needed in the final patch.
    >
    >      The globals_xxx.cpp files will be checked into the repo. Afterwards, to
    >      add/remove flags, you would need to edit the xxx_globals.hpp/cpp files
    >      directly.
    >
    >      As mentioned in the design doc, my goal is to avoid C++ code generation
    >      as many people are against it.
    >
    >      Personally, I prefer code generation. I think that can result in a
    >      smaller binary footprint that will start up faster, but the delta is
    >      probably negligible. We can probably leave that for the future. As long
    >      as we use the macros in a consistent manner, we can easily extract the
    >      flag definitions and put them into external input files that are
    >      processed by a C++ code generator.
    > ---
    > But your current webrev leads to have a codegen path.
    >
    > You have to iterate all options at least twice, right? one is in header file, the other one is in the cpp file.
    > The list is long and the order doesn't matter. I can't imagine you will check in those hpp/cpp pairs.  It's very natural to have a script like tcl or m4 to generate cpp from hpp, right?
    > There was a high-order macro RUNTIME_FLAGS in previous globle.hpp. That's why it avoids to iterate options again & again. C-preprocessor did the "codegen" job by expanding macros.
    >
    > Why do you ditch that design? it still can archive all your design goals.
    > ---
    >      > 4. Probably, it's not a bad idea to still use product/develop instead of PRODUCT_FLAG, DEVELOP_FLAG.
    >      > Those options will be read everyday by developers, I feel lower-case names are more friendly.
    >
    >      I don't have a strong preference, but I think that adding "product"
    >      globally (even as a macro) may cause problems with code like this:
    >
    >           static int product(int a, int b) {return a*b;}
    >
    >      as the C pre-processor would try to expand "product".
    > --
    > In previous globals.hpp, 'product' is just a macro parameter.  That's why current implementation doesn't have any conflict with the function product.
    > --
    >
    >      Thanks
    >      - Ioi
    >
    >
    >      >
    >      > [1]: https://cr.openjdk.java.net/~xliu/8151779/00/webrev/
    >      >
    >      > Thanks,
    >      > --lx
    >      >
    >      > On 4/9/20, 5:08 PM, "hotspot-dev on behalf of Ioi Lam" <hotspot-dev-bounces at openjdk.java.net on behalf of ioi.lam at oracle.com> wrote:
    >      >
    >      >      CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
    >      >
    >      >
    >      >
    >      >      The code for specifying HotSpot command-line flags used to be simple. At
    >      >      the very beginning it was something like
    >      >
    >      >           #define ALL_FLAGS(develop, develop) \
    >      >              product(int, SomeFlag, def_value, "some doc") \
    >      >              develop(int, MoreFlag, def_value, "more doc") \
    >      >
    >      >      but like all good ideas, it has gradually degraded into a mess [1][2].
    >      >      It's now very difficult to add any new functionality (e.g., add
    >      >      "manageable" annotations to the flags [3]).
    >      >
    >      >      I think it's time for a complete rewrite :-)
    >      >
    >      >      My proposal is here. Please comment if it's a good or bad idea.
    >      >
    >      >      https://wiki.openjdk.java.net/display/HotSpot/HotSpot+Command-Line+Flags+Overhaul+-+Design+Doc
    >      >
    >      >
    >      >
    >      >      tldr; ---
    >      >
    >      >
    >      >      OLD:
    >      >
    >      >         product(intx, AliasLevel, 3,                                          \
    >      >                 "0 for no aliasing, 1 for oop/field/static/array split,
    >      >      "         \
    >      >                 "2 for class split, 3 for unique
    >      >      instances")                      \
    >      >                 range(0, 3)
    >      >      /*optional*/                                         \
    >      >                 constraint(AliasLevelConstraintFunc,AfterErgo) /*optional*/
    >      >           \
    >      >
    >      >
    >      >      NEW:
    >      >
    >      >         PRODUCT_FLAG(intx,   AliasLevel, 3, JVMFlag::RANGE | JVMFlag::CONSTRAINT,
    >      >                              "0 for no aliasing, 1 for oop/field/static/array
    >      >      split, "
    >      >                              "2 for class split, 3 for unique instances");
    >      >          // optional
    >      >          FLAG_RANGE(         AliasLevel, 0, 3);
    >      >          // optional
    >      >          FLAG_CONSTRAINT(    AliasLevel, (void*)AliasLevelConstraintFunc,
    >      >      JVMFlag::AfterErgo);
    >      >
    >      >
    >      >
    >      >
    >      >      [1]
    >      >      http://hg.openjdk.java.net/jdk/jdk/file/7af6364e1792/src/hotspot/share/runtime/globals.hpp#l115
    >      >      [2]
    >      >      http://hg.openjdk.java.net/jdk/jdk/file/7af6364e1792/src/hotspot/share/runtime/flags/jvmFlag.cpp#l635
    >      >      [3] https://bugs.openjdk.java.net/browse/JDK-7123237
    >      >
    >      >      Thanks
    >      >      - Ioi
    >      >
    >      >
    >
    >
    >
    
    


From kim.barrett at oracle.com  Wed Apr 15 23:31:07 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 15 Apr 2020 19:31:07 -0400
Subject: [trivial] RFR: 8242631: Missing but used special functions for
 some classes
In-Reply-To: <80185f2d-e428-ebb9-9117-a34af96e8e0b@oracle.com>
References: <13190093-8B8E-4D9C-804A-F130B9C3C2EF@oracle.com>
 <80185f2d-e428-ebb9-9117-a34af96e8e0b@oracle.com>
Message-ID: <64CA88C7-5C3E-4C2F-AA29-021D2AB85909@oracle.com>

> On Apr 14, 2020, at 10:26 PM, David Holmes <david.holmes at oracle.com> wrote:
> 
> Hi Kim,
> 
> Looks fine and trivial.

Thanks.

> 
> Thanks,
> David
> 
> On 14/04/2020 11:02 pm, Kim Barrett wrote:
>> Please review this trivial change to a couple of classes to eliminate
>> gcc9 deprecation warnings for them when compiling for C++11 or later.
>> See the CR for more details about the warnings.
>> BCEscapeAnalyzer::ArgumentMap has an implicitly defined copy
>> constructor and a user-defined copy assignment operator.  Since the
>> copy assignment operator is functionally equivalent to the default,
>> removed it.
>> Various subtypes of oop had an implicitly defined copy constructor;
>> made it explicit.
>> There are a few more like these, but I may address them differently.
>> CR:
>> https://bugs.openjdk.java.net/browse/JDK-8242631
>> Webrev:
>> https://cr.openjdk.java.net/~kbarrett/8242631/open.00/
>> Testing:
>> mach5 tier1



From coleen.phillimore at oracle.com  Thu Apr 16 02:40:13 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Wed, 15 Apr 2020 22:40:13 -0400
Subject: RFR (S) 8074292: nsk/jdb/kill/kill001: generateOopMap.cpp
 assert(bb->is_reachable()) failed
In-Reply-To: <0a87b7cd-7dfe-0831-05e1-8d48f5eb6755@oracle.com>
References: <f66b8de6-d9db-a90f-bfc2-5bf42a19cada@oracle.com>
 <6e38070c-f166-a0c2-8b8d-d698e5f5bcf3@oracle.com>
 <1e3d4907-d54a-df0e-e0d3-8bbda8c5394a@oracle.com>
 <0a87b7cd-7dfe-0831-05e1-8d48f5eb6755@oracle.com>
Message-ID: <102f0511-9172-ef0e-5f26-003600c322c2@oracle.com>



On 4/15/20 1:28 AM, David Holmes wrote:
> On 14/04/2020 10:36 pm, coleen.phillimore at oracle.com wrote:
>> On 4/13/20 10:49 PM, David Holmes wrote:
>>> Hi Coleen,
>>>
>>> On 14/04/2020 12:34 am, coleen.phillimore at oracle.com wrote:
>>>> Summary: Do not install async exceptions at_safepoint for each 
>>>> bytecode.
>>>
>>> I'm still not certain that we have to go this far to solve this 
>>> problem, but it does sound like a relatively simple solution 
>>> provided there are no unintended consequences.
>>>
>>>> See CR for a lot more details.? This change calls a new 
>>>> InterpreterRuntime::at_safepoint_async_safe() which installs the 
>>>> async exception in the interpreter at backward branches and 
>>>> returns. This uses safepoint polling code in the interpreter for 
>>>> each platform.? These changes (cross) compile on platforms that 
>>>> Oracle doesn't support but I don't know if they work.
>>>>
>>>> I'm not convinced the platform specific changes are necessary, 
>>>> because calls to the runtime from many bytecodes will install the 
>>>> async exception, so it's essentially installed "enough" for this 
>>>> deprecated feature.? I tested the changes with *and* without the 
>>>> platform specific changes with no failure, which included the jdb, 
>>>> jdi and jvmti serviceability tests.
>>>
>>> I don't understand what you mean here. If the whole basis of this 
>>> fix is "don't install async exceptions other than at backward 
>>> branches and returns" then how is that implemented without the 
>>> changes in the interpreter code?
>>>
>>> If this can be fixed just by adjusting the actual monitor code then 
>>> I would much prefer that. It took me a while to get my head around 
>>> the dispatch changes in interpreter code and even then I don't see 
>>> how those changes only impact backward branches and returns ??
>>
>> You have to read the comments in the bug again.? There *is* special 
>> code to not install the async exception in the monitorexit code. That 
>> is not enough to prevent this bug.? You can also read the old bug 
>> report you linked to this one.
>
> I know there is some special handling in monitorexit already, I was 
> referring to additional special handling around the monitorexit to 
> disable whatever piece of code is currently installing the async 
> exception.

David pointed out to me offline that the bytecode after monitorexit was 
not part of the exception region (see bug comments for details).

I'm withdrawing this RFR.

Coleen
>
>> The interpreter code dispatch_next passes "true" if it's a backwards 
>> branch, that's how it can tell.
>
> Okay - I see dispatch_next is passed generate_poll=true for returns 
> but I can't see where backward branches come into play. Your changes 
> cause the at_safepoint_async_safe to be called in that case, whereas 
> any other code paths that lead to at_safepoint no longer install the 
> async exception. So things are a bit clearer in that regard.
>
>> My point was that there are enough code paths that install async 
>> exceptions *other than monitorenter and monitorexit* that maybe it's 
>> not necessary to install them at backwards branches and returns.? I 
>> suppose someone could construct a test case to show otherwise.
>
> Yes I understood that was the query, but without knowing exactly where 
> those code paths are it is hard to comment on whether there is 
> adequate coverage. Even with installing on backward branches there is 
> a risk that small loops can be unrolled and so lose the check (indeed 
> that is what "counted loops" in the JIT does) and thus we rely on 
> these other code paths anyway.
>
>>>> This change also makes InterpreterRuntime::monitorexit a JRT_LEAF 
>>>> bytecode. The code to check for exceptions is outside the runtime 
>>>> call. I ran the JCK vm and lang tests on this change with no failure.
>>>>
>>>> Tested with tier1-6.
>>>>
>>>> open webrev at 
>>>> http://cr.openjdk.java.net/~coleenp/2020/8074292.01/webrev
>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8074292
>>>
>>> ./cpu/x86/interp_masm_x86.cpp
>>>
>>> It took me a long time to figure out how the new logic worked 
>>> compared to the old logic. Even then I'm unclear about the effective 
>>> recursive dispatch path: dispatch_base(generate_poll=true) -> 
>>> dispatch_via -> dispatch_base(generate_poll=false) - does it work 
>>> okay with VerifyActivationFrameSize? It seems a rather convoluted 
>>> way to effectively just execute:
>>>
>>> ?858?? lea(rscratch1, ExternalAddress((address)table));
>>> ?859?? jmp(Address(rscratch1, rbx, Address::times_8));
>>
>> I could test it with VerifyActivationFrameSize.? Or I could just add 
>> the two instructions per platform.? I mostly copied the code in 
>> generate_safept_entry_for.? It might be better to just copy the 
>> instructions.
>>
>>>
>>> ---
>>>
>>> src/hotspot/share/interpreter/interpreterRuntime.cpp
>>>
>>> How were you able to drop this code:
>>>
>>> 791?? if (elem == NULL || h_obj()->is_unlocked()) {
>>> 792 THROW(vmSymbols::java_lang_IllegalMonitorStateException());
>>> 793?? }
>>>
>>> ?
>>> ?and this:
>>
>> The caller throws the exception for these cases.
>
> Sorry but I don't see that in the caller e.g. 
> InterpreterMacroAssembler::unlock_object. Further you are not allowing 
> for elem to be NULL which will lead to a crash when you dereference it.
>
>>>
>>> 798 #ifdef ASSERT
>>> 799 thread->last_frame().interpreter_frame_verify_monitor(elem);
>>> 800 #endif
>>>
>>
>> This seemed redundant.
>
> Possibly, but I assume the pairing in monitorenter and monitorexit 
> around the actual monitor operation is to ensure that the stack frame 
> info is not unexpectedly messed up. Maybe it is redundant to call 
> before and after, but then maybe the same is true in monitorenter.
>
> David
> -----
>
>> Coleen
>>> ?
>>>
>>> Thanks,
>>> David
>>>
>>>> Thanks,
>>>> Coleen
>>>>
>>


From vladimir.x.ivanov at oracle.com  Thu Apr 16 12:32:52 2020
From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov)
Date: Thu, 16 Apr 2020 15:32:52 +0300
Subject: RFR (XXL): 8223347: Integration of Vector API (Incubator):
 General HotSpot changes
In-Reply-To: <c1bdf88c-5de2-d069-5f31-5a95c6988bf8@oracle.com>
References: <c1bdf88c-5de2-d069-5f31-5a95c6988bf8@oracle.com>
Message-ID: <25a564a1-7f40-6988-060f-86b06e02ad21@oracle.com>

Hi,

Any more reviews, please? Especially, compiler and runtime-related changes.

Thanks in advance!

Best regards,
Vladimir Ivanov

On 04.04.2020 02:12, Vladimir Ivanov wrote:
> Hi,
> 
> Following up on review requests of API [0] and Java implementation [1] 
> for Vector API (JEP 338 [2]), here's a request for review of general 
> HotSpot changes (in shared code) required for supporting the API:
> 
> 
> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/all.00-03/ 
> 
> 
> (First of all, to set proper expectations: since the JEP is still in 
> Candidate state, the intention is to initiate preliminary round(s) of 
> review to inform the community and gather feedback before sending out 
> final/official RFRs once the JEP is Targeted to a release.)
> 
> Vector API (being developed in Project Panama [3]) relies on JVM support 
> to utilize optimal vector hardware instructions at runtime. It interacts 
> with JVM through intrinsics (declared in 
> jdk.internal.vm.vector.VectorSupport [4]) which expose vector operations 
> support in C2 JIT-compiler.
> 
> As Paul wrote earlier: "A vector intrinsic is an internal low-level 
> vector operation. The last argument to the intrinsic is fall back 
> behavior in Java, implementing the scalar operation over the number of 
> elements held by the vector.? Thus, If the intrinsic is not supported in 
> C2 for the other arguments then the Java implementation is executed (the 
> Java implementation is always executed when running in the interpreter 
> or for C1)."
> 
> The rest of JVM support is about aggressively optimizing vector boxes to 
> minimize (ideally eliminate) the overhead of boxing for vector values.
> It's a stop-the-gap solution for vector box elimination problem until 
> inline classes arrive. Vector classes are value-based and in the longer 
> term will be migrated to inline classes once the support becomes available.
> 
> Vector API talk from JVMLS'18 [5] contains brief overview of JVM 
> implementation and some details.
> 
> Complete implementation resides in vector-unstable branch of panama/dev 
> repository [6].
> 
> Now to gory details (the patch is split in multiple "sub-webrevs"):
> 
> ===========================================================
> 
> (1) 
> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/00.backend.shared/ 
> 
> 
> Ideal vector nodes for new operations introduced by Vector API.
> 
> (Platform-specific back end support will be posted for review separately).
> 
> ===========================================================
> 
> (2) 
> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/01.intrinsics/ 
> 
> 
> JVM Java interface (VectorSupport) and intrinsic support in C2.
> 
> Vector instances are initially represented as VectorBox macro nodes and 
> "unboxing" is represented by VectorUnbox node. It simplifies vector box 
> elimination analysis and the nodes are expanded later right before EA pass.
> 
> Vectors have 2-level on-heap representation: for the vector value 
> primitive array is used as a backing storage and it is encapsulated in a 
> typed wrapper (e.g., Int256Vector - vector of 8 ints - contains a int[8] 
> instance which is used to store vector value).
> 
> Unless VectorBox node goes away, it needs to be expanded into an 
> allocation eventually, but it is a pure node and doesn't have any JVM 
> state associated with it. The problem is solved by keeping JVM state 
> separately in a VectorBoxAllocate node associated with VectorBox node 
> and use it during expansion.
> 
> Also, to simplify vector box elimination, inlining of vector reboxing 
> calls (VectorSupport::maybeRebox) is delayed until the analysis is over.
> 
> ===========================================================
> 
> (3) 
> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/02.vbox_elimination/ 
> 
> 
> Vector box elimination analysis implementation. (Brief overview: slides 
> #36-42 [5].)
> 
> The main part is devoted to scalarization across safepoints and 
> rematerialization support during deoptimization. In C2-generated code 
> vector operations work with raw vector values which live in registers or 
> spilled on the stack and it allows to avoid boxing/unboxing when a 
> vector value is alive across a safepoint. As with other values, there's 
> just a location of the vector value at the safepoint and vector type 
> information recorded in the relevant nmethod metadata and all the 
> heavy-lifting happens only when rematerialization takes place.
> 
> The analysis preserves object identity invariants except during 
> aggressive reboxing (guarded by -XX:+EnableAggressiveReboxing).
> 
> (Aggressive reboxing is crucial for cases when vectors "escape": it 
> allocates a fresh instance at every escape point thus enabling original 
> instance to go away.)
> 
> ===========================================================
> 
> (4) 
> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/03.module.hotspot/ 
> 
> 
> HotSpot changes for jdk.incubator.vector module. Vector support is 
> makred experimental and turned off by default. JEP 338 proposes the API 
> to be released as an incubator module, so a user has to specify 
> "--add-module jdk.incubator.vector" on the command line to be able to 
> use it.
> When user does that, JVM automatically enables Vector API support.
> It improves usability (user doesn't need to separately "open" the API 
> and enable JVM support) while minimizing risks of destabilitzation from 
> new code when the API is not used.
> 
> 
> That's it! Will be happy to answer any questions.
> 
> And thanks in advance for any feedback!
> 
> Best regards,
> Vladimir Ivanov
> 
> [0] 
> https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065345.html 
> 
> 
> [1] 
> https://mail.openjdk.java.net/pipermail/hotspot-dev/2020-April/041228.html
> 
> [2] https://openjdk.java.net/jeps/338
> 
> [3] https://openjdk.java.net/projects/panama/
> 
> [4] 
> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/01.intrinsics/src/java.base/share/classes/jdk/internal/vm/vector/VectorSupport.java.html 
> 
> 
> [5] http://cr.openjdk.java.net/~vlivanov/talks/2018_JVMLS_VectorAPI.pdf
> 
> [6] http://hg.openjdk.java.net/panama/dev/shortlog/92bbd44386e9
> 
>  ??? $ hg clone http://hg.openjdk.java.net/panama/dev/ -b vector-unstable

From sundararajan.athijegannathan at oracle.com  Thu Apr 16 13:06:05 2020
From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com)
Date: Thu, 16 Apr 2020 18:36:05 +0530
Subject: RFR 8242931: Few more tests that use nashorn have been missed
In-Reply-To: <f819ece5-83e7-74e0-4aa3-7c16f3e974ca@oracle.com>
References: <f819ece5-83e7-74e0-4aa3-7c16f3e974ca@oracle.com>
Message-ID: <5a4c6841-a0b2-a596-0a19-e4a7f0222c43@oracle.com>

Mail bounced back because I had not subscribed to hotspot-dev. Please 
review.

Thanks

-Sundar

On 16/04/20 6:29 pm, sundararajan.athijegannathan at oracle.com wrote:
> Nashorn engine removal fix (8241749: Remove the Nashorn JavaScript 
> Engine) missed updating few tests, config files.
>
> Please review.
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8242931
>
> Webrev: http://cr.openjdk.java.net/~sundar/8242931/webrev.00/
>
> PS I'm disabling tests by adding @ignore. I've filed bugs for the 
> tests. Please revisit port or remove these nashorn using tests
>
> Thanks,
>
> -Sundar
>

From Alan.Bateman at oracle.com  Thu Apr 16 13:10:14 2020
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Thu, 16 Apr 2020 14:10:14 +0100
Subject: RFR 8242931: Few more tests that use nashorn have been missed
In-Reply-To: <f819ece5-83e7-74e0-4aa3-7c16f3e974ca@oracle.com>
References: <f819ece5-83e7-74e0-4aa3-7c16f3e974ca@oracle.com>
Message-ID: <6e45f66e-4b23-b367-dfc1-ed473a6f39d6@oracle.com>

On 16/04/2020 13:59, sundararajan.athijegannathan at oracle.com wrote:
> Nashorn engine removal fix (8241749: Remove the Nashorn JavaScript 
> Engine) missed updating few tests, config files.
>
> Please review.
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8242931
>
> Webrev: http://cr.openjdk.java.net/~sundar/8242931/webrev.00/
>
> PS I'm disabling tests by adding @ignore. I've filed bugs for the 
> tests. Please revisit port or remove these nashorn using tests
Can you use the jtreg exclude lists (ProblemList.txt files) instead? We 
moved away from @ignore a few ago to make it easier to track excluded tests.

-Alan

From david.holmes at oracle.com  Thu Apr 16 13:19:51 2020
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 16 Apr 2020 23:19:51 +1000
Subject: RFR 8242931: Few more tests that use nashorn have been missed
In-Reply-To: <6e45f66e-4b23-b367-dfc1-ed473a6f39d6@oracle.com>
References: <f819ece5-83e7-74e0-4aa3-7c16f3e974ca@oracle.com>
 <6e45f66e-4b23-b367-dfc1-ed473a6f39d6@oracle.com>
Message-ID: <d3114e6a-4306-6cd1-bdd9-eda60b27ba82@oracle.com>

+1 Please problem list the tests under the associated bug ids.

Thanks,
David

On 16/04/2020 11:10 pm, Alan Bateman wrote:
> On 16/04/2020 13:59, sundararajan.athijegannathan at oracle.com wrote:
>> Nashorn engine removal fix (8241749: Remove the Nashorn JavaScript 
>> Engine) missed updating few tests, config files.
>>
>> Please review.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8242931
>>
>> Webrev: http://cr.openjdk.java.net/~sundar/8242931/webrev.00/
>>
>> PS I'm disabling tests by adding @ignore. I've filed bugs for the 
>> tests. Please revisit port or remove these nashorn using tests
> Can you use the jtreg exclude lists (ProblemList.txt files) instead? We 
> moved away from @ignore a few ago to make it easier to track excluded 
> tests.
> 
> -Alan

From erik.gahlin at oracle.com  Thu Apr 16 13:27:42 2020
From: erik.gahlin at oracle.com (Erik Gahlin)
Date: Thu, 16 Apr 2020 15:27:42 +0200
Subject: RFR 8242931: Few more tests that use nashorn have been missed
In-Reply-To: <f819ece5-83e7-74e0-4aa3-7c16f3e974ca@oracle.com>
References: <f819ece5-83e7-74e0-4aa3-7c16f3e974ca@oracle.com>
Message-ID: <7F6F4FFB-DF42-469B-961A-B7EF56E6C01C@oracle.com>

Hi Sundar,

Will there be a replacement for Nashorn, i.e. V8 with added Java bindings?

I am asking because one of the JFR test uses Nashorn for parsing JSON and if another parser is coming in the near future, it could be used. instead of adding a parser for the test.

Thanks
Erik

> On 16 Apr 2020, at 14:59, sundararajan.athijegannathan at oracle.com wrote:
> 
> Nashorn engine removal fix (8241749: Remove the Nashorn JavaScript Engine) missed updating few tests, config files.
> 
> Please review.
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8242931
> 
> Webrev: http://cr.openjdk.java.net/~sundar/8242931/webrev.00/
> 
> PS I'm disabling tests by adding @ignore. I've filed bugs for the tests. Please revisit port or remove these nashorn using tests
> 
> Thanks,
> 
> -Sundar
> 


From leo.korinth at oracle.com  Thu Apr 16 15:39:03 2020
From: leo.korinth at oracle.com (Leo Korinth)
Date: Thu, 16 Apr 2020 17:39:03 +0200
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <e4fed1a3-addd-133c-fc68-74cf47fa6086@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
 <135a6496-5b54-9117-5cbf-e3211ddc32dd@oracle.com>
 <bbb2b5ef-62f2-6507-95f7-03712a382153@oracle.com>
 <e242e057-de6e-493a-eb00-068b2db8c9fa@oracle.com>
 <df621fd2-0eaf-cb24-788f-12166b271303@oracle.com>
 <e4fed1a3-addd-133c-fc68-74cf47fa6086@oracle.com>
Message-ID: <83cfd8cb-0f75-81c0-82b3-b3a8874ca87c@oracle.com>



On 14/04/2020 16:18, Magnus Ihse Bursie wrote:

> Combined change ended up containing updated copyright in 
> src/hotspot/share/runtime/sweeper.hpp, even though no actual change were 
> done there in the end.
> 
> /Magnus

Thanks, I will fix that.

/Leo

From gerard.ziemski at oracle.com  Thu Apr 16 15:53:21 2020
From: gerard.ziemski at oracle.com (gerard ziemski)
Date: Thu, 16 Apr 2020 10:53:21 -0500
Subject: RFR: 8242626: enhance posix print_rlimit_info
In-Reply-To: <AM0PR02MB4227F897CBF6AA514A7343D293DA0@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB4227F897CBF6AA514A7343D293DA0@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <c8c56831-cd1e-861f-52a8-a66a23ca2858@oracle.com>

hi Matthias,

Nice enhancement, I like that you added the hard limit and that we now 
check the return code.

+1 from me (small r reviewer)

ps. I checked our tests to see whether any one of them parses the rlimit 
output and I didn't find any, but did you run any tests just in case?


cheers

On 4/14/20 10:27 AM, Baesken, Matthias wrote:
> Please review this small enhancement of  os::Posix::print_rlimit_info.
>
> It adds  return code handling  of  getrlimit ,  output of some more limit values and  output of  hard limits (additionally to the existing soft limits) .
>
> Bug/webrev :
>
> https://bugs.openjdk.java.net/browse/JDK-8242626
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8242626.1/
>
> Thanks, Matthias


From leo.korinth at oracle.com  Thu Apr 16 16:04:58 2020
From: leo.korinth at oracle.com (Leo Korinth)
Date: Thu, 16 Apr 2020 18:04:58 +0200
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <91f6fba9-56c1-31d7-12de-4f2b75d38ede@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
 <135a6496-5b54-9117-5cbf-e3211ddc32dd@oracle.com>
 <bbb2b5ef-62f2-6507-95f7-03712a382153@oracle.com>
 <e242e057-de6e-493a-eb00-068b2db8c9fa@oracle.com>
 <df621fd2-0eaf-cb24-788f-12166b271303@oracle.com>
 <91f6fba9-56c1-31d7-12de-4f2b75d38ede@oracle.com>
Message-ID: <786bacbf-bde3-7c3f-9b57-c84b61c61d16@oracle.com>

On 15/04/2020 07:45, David Holmes wrote:
> Hi Leo,
> 
> Before I look at the webrev one query ..
> 
> <trimming>
> 
> On 14/04/2020 11:53 pm, Leo Korinth wrote:
>> On 06/04/2020 01:37, David Holmes wrote:
>>>> I could change CHECK_0 to use NULL_WORD though and if I do, I get 
>>>> compiler errors. CHECK_0 is used for both pointers and integral 
>>>> zeroes and types that implicitly converts from either. What the 
>>>> original thought was is unclear to me; is CHECK_0 to be used to 
>>>> return things convertible from pointers? If so, what is CHECK_NULL for?
>>>
>>> No CHECK_0 is intended for int-returning functions (which would also 
>>> extend to intptr_t).
>>
>> It might be intended for it, but it is not used in such a way 
>> (therefore my remark on compile failures when using NULL_WORD)
>>
>> I will do no change here.
> 
> Please explain what you mean by "it is not used in such a way". Other 
> than a few misuses in ./share/jvmci/jvmciCompilerToVM.cpp where it is 
> used for jboolean and the jobject case you are fixing, it is used in 
> methods that return jint, jbyte, jlong ...
> 
> Thanks,
> David
> -----

You mentioned in an earlier mail that the problems I fixed in 
JVMCI_CHECK_NULL should also be observed in CHECK_NULL. I answered that 
was not true because it was used differently. Though if I where to 
redefine CHECK_0 in the same way as i redefined the JVMCI counterpart I 
would get compile time errors. That is, CHECK_0 can not be defined more 
stringently to return an integral zero as call-sites rely on the 
(convertible) zero literal; the macro is thus used in (IMO) the wrong way.


What I tried to say was that I have done /no/ changes for CHECK_0 (or 
the call-sites of it) as I did not want to follow that rabbit hole, I 
only updated JVMCI_CHECK_0 as it was a minimal change. I did cut out too 
much of the mail conversation context for that to be understood, sorry.

/Leo

From martin.doerr at sap.com  Thu Apr 16 16:21:22 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Thu, 16 Apr 2020 16:21:22 +0000
Subject: RFR: 8242626: enhance posix print_rlimit_info
In-Reply-To: <c8c56831-cd1e-861f-52a8-a66a23ca2858@oracle.com>
References: <AM0PR02MB4227F897CBF6AA514A7343D293DA0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <c8c56831-cd1e-861f-52a8-a66a23ca2858@oracle.com>
Message-ID: <AM0PR0202MB329764EC55E8688516D582359AD80@AM0PR0202MB3297.eurprd02.prod.outlook.com>

Hi Matthias,

thanks for improving it.

A very minor hotspot coding style issue:
Curly brackets should be used for the if and else parts.
And the indentation looks odd in line 432.

It'd be nice to add an example output to the issue description.

Otherwise, looks good. I don't need to see another webrev.

Best regards,
Martin


> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
> gerard ziemski
> Sent: Donnerstag, 16. April 2020 17:53
> To: Baesken, Matthias <matthias.baesken at sap.com>; 'hotspot-
> dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: Re: RFR: 8242626: enhance posix print_rlimit_info
> 
> hi Matthias,
> 
> Nice enhancement, I like that you added the hard limit and that we now
> check the return code.
> 
> +1 from me (small r reviewer)
> 
> ps. I checked our tests to see whether any one of them parses the rlimit
> output and I didn't find any, but did you run any tests just in case?
> 
> 
> cheers
> 
> On 4/14/20 10:27 AM, Baesken, Matthias wrote:
> > Please review this small enhancement of  os::Posix::print_rlimit_info.
> >
> > It adds  return code handling  of  getrlimit ,  output of some more limit
> values and  output of  hard limits (additionally to the existing soft limits) .
> >
> > Bug/webrev :
> >
> > https://bugs.openjdk.java.net/browse/JDK-8242626
> >
> > http://cr.openjdk.java.net/~mbaesken/webrevs/8242626.1/
> >
> > Thanks, Matthias


From sundararajan.athijegannathan at oracle.com  Thu Apr 16 16:26:44 2020
From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com)
Date: Thu, 16 Apr 2020 21:56:44 +0530
Subject: RFR 8242931: Few more tests that use nashorn have been missed
In-Reply-To: <d3114e6a-4306-6cd1-bdd9-eda60b27ba82@oracle.com>
References: <f819ece5-83e7-74e0-4aa3-7c16f3e974ca@oracle.com>
 <6e45f66e-4b23-b367-dfc1-ed473a6f39d6@oracle.com>
 <d3114e6a-4306-6cd1-bdd9-eda60b27ba82@oracle.com>
Message-ID: <1707876a-a671-d7cc-6af3-59dcc11f6aab@oracle.com>

Please review updated webrev: 
http://cr.openjdk.java.net/~sundar/8242931/webrev.01/

Thanks,

-Sundar

On 16/04/20 6:49 pm, David Holmes wrote:
> +1 Please problem list the tests under the associated bug ids.
>
> Thanks,
> David
>
> On 16/04/2020 11:10 pm, Alan Bateman wrote:
>> On 16/04/2020 13:59, sundararajan.athijegannathan at oracle.com wrote:
>>> Nashorn engine removal fix (8241749: Remove the Nashorn JavaScript 
>>> Engine) missed updating few tests, config files.
>>>
>>> Please review.
>>>
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8242931
>>>
>>> Webrev: http://cr.openjdk.java.net/~sundar/8242931/webrev.00/
>>>
>>> PS I'm disabling tests by adding @ignore. I've filed bugs for the 
>>> tests. Please revisit port or remove these nashorn using tests
>> Can you use the jtreg exclude lists (ProblemList.txt files) instead? 
>> We moved away from @ignore a few ago to make it easier to track 
>> excluded tests.
>>
>> -Alan

From mandy.chung at oracle.com  Thu Apr 16 16:45:27 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Thu, 16 Apr 2020 09:45:27 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <4A360464-1873-43A8-B423-C93D1BEF9895@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <1a5eb022-dd7e-b0bb-750f-c43be195eb42@oracle.com>
 <da9f80ff-6d63-aebe-a9fa-1a683eb42bbc@oracle.com>
 <4A360464-1873-43A8-B423-C93D1BEF9895@oracle.com>
Message-ID: <51adeaaf-9c85-240c-ed0d-f460c117ba46@oracle.com>

On 4/14/20 11:51 AM, Paul Sandoz wrote:
> Looks good to me (not familiar with all the code areas.
>
> Minor suggestion:
>
> MethodHandles.java
>
> 1811          * ASCII periods. For the instance of {@link java.lang.Class} representing {@code C}:
> 1812          * <ul>
> 1813          * <li> {@link Class#getName()} returns the string {@code GN + "/" + <suffix>},
> 1814          *      even though this is not a valid binary class or interface name.</li>
> 1815          * <li> {@link Class#descriptorString()} returns the string
> 1816          *      {@code "L" + N + ";" + "/" + <suffix> },
> 1817          *      even though this is not a valid type descriptor name.</li>
> 1818          * </ul>
>
> Add another bullet:
>
> ?
> even though this is not a valid type descriptor name; and
>
> - therefore {@link Class#describeConstable} returns an empty {@code Optional}.
> ?
>
> ?

OK.?? I add this bullet:

- Class.describeConstable() returns an empty optional as C cannot be 
described in nominal form.

The webrev and spec was updated [1] for descriptor string to be of the 
form "Lfoo/Foo.1234;" to mitigate the compatibility risk.? Th

Specdiff with serviceability spec change:
http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff-inc/
Specdiff without svc spec change:
http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff-descriptor-string/overview-summary.html

Webrev:
http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.06-descriptor-string/

Svc spec change webrev:
http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.06-svc-spec-changes/

thanks
Mandy
[1] 
https://mail.openjdk.java.net/pipermail/valhalla-dev/2020-April/007155.html


>
> Paul.
>
>> On Apr 11, 2020, at 8:13 PM, Mandy Chung <mandy.chung at oracle.com> wrote:
>>
>> Please review the delta patch:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.06-delta/
>>
>> Incremental specdiff:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff-inc/overview-summary.html
>> This is to follow a discussion [1] on Class::descriptorString and MethodType::descriptorString for hidden classes.   The proposal is to define the descriptor string for a hidden class of this form:
>>      "L" + N + ";" + "/" + <suffix>
>>
>> The spec of `Lookup::defineHiddenClass`, `Class::descriptorString` and `MethodType::descriptorString` are updated to return the descriptor of this form for a hidden class.   To support hidden class, `java.lang.invoke.TypeDescriptor` spec is revised such that a `TypeDescriptor` object can represent an entity that may not be described in nominal form.     This change affects JVM TI, JDWP and JDI.    The spec change includes a couple other JVM TI and java.instrument spec clarification w.r.t. hidden classes that Serguei has been working on.
>>
>>
>>
>> Mandy
>> [1] https://mail.openjdk.java.net/pipermail/valhalla-dev/2020-April/007093.html
>>
>> ----------------
>> As a record, the above patch is applied on the top:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.06/
>>
>> webrev.06 includes the following changes that have been reviewed:
>> 1. rename "weak hidden" and Klass::is_hidden_weak to is_non_strong_hidden
>> 2. remove unused `setImplMethod` method from lambda proxy class
>> 3. include Serguei's patch to fix JDK-8242166: regression in JDI ClassUnload events
>> 4. drop the uniqueness guarantee of the suffix of the hidden class's name
>>
>> On 3/31/20 8:01 PM, Mandy Chung wrote:
>>> Thanks to the review feedbacks.
>>>
>>> Updated webrev:
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.04/
>>> Delta between webrev.03 and webrev.04:
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03-delta/
>>>
>>> Summary of changes is:
>>> 1. Update javac to retain the previous behavior when compiling to target release <= 14 where lambda proxy class is not a nestmate
>>> 2. Rename Class::isHiddenClass to Class::isHidden
>>> 3. Address Joe's feedback on the CSR to document the behavior of the relevant methods in java.lang.Class for hidden classes
>>> 4. Add test case for unloadable class with nest host error
>>> 5. Add test cases for hidden classes with different kinds of class or interface
>>> 6. Update dcmd to drop "weak hidden class" and refer it as "hidden class"
>>> 7. Small changes in response to Remi, Coleen, Paul's review comments
>>> 8. Fix copyright headers
>>> 9. Fix @modules in tests
>>>
>>> Most of the changes above have also been reviewed as separate patches.
>>>
>>> Thanks
>>> Mandy
>>>
>>> On 3/26/20 4:57 PM, Mandy Chung wrote:
>>>> Please review the implementation of JEP 371: Hidden Classes. The main changes are in core-libs and hotspot runtime area.  Small changes are made in javac, VM compiler (intrinsification of Class::isHiddenClass), JFR, JDI, and jcmd.  CSR [1]has been reviewed and is in the finalized state (see specdiff and javadoc below for reference).
>>>>
>>>> Webrev:
>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03
>>>>
>>>> javadoc/specdiff
>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/
>>>>
>>>> JVMS 5.4.4 change:
>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf
>>>>
>>>> CSR:
>>>> https://bugs.openjdk.java.net/browse/JDK-8238359
>>>>
>>>> Thanks
>>>> Mandy
>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
>>>> [3] https://bugs.openjdk.java.net/browse/JDK-8230502


From daniel.fuchs at oracle.com  Thu Apr 16 17:46:05 2020
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Thu, 16 Apr 2020 18:46:05 +0100
Subject: RFR 8242931: Few more tests that use nashorn have been missed
In-Reply-To: <1707876a-a671-d7cc-6af3-59dcc11f6aab@oracle.com>
References: <f819ece5-83e7-74e0-4aa3-7c16f3e974ca@oracle.com>
 <6e45f66e-4b23-b367-dfc1-ed473a6f39d6@oracle.com>
 <d3114e6a-4306-6cd1-bdd9-eda60b27ba82@oracle.com>
 <1707876a-a671-d7cc-6af3-59dcc11f6aab@oracle.com>
Message-ID: <54b64a4b-5d4f-0fc9-3242-b50fdcb4b0e1@oracle.com>

Hi Sundar,

I spotted a typo (generial vs generic) test/jdk/ProblemList.txt:

  936 java/util/ServiceLoader/ReloadTest.java 
8242935 generial-all

This is not an area I know well - so maybe wait for at least
another opinion (reviewer or not) but what I saw looked
reasonable.

You have my review.

best regards,

-- daniel

On 16/04/2020 17:26, sundararajan.athijegannathan at oracle.com wrote:
> Please review updated webrev: 
> http://cr.openjdk.java.net/~sundar/8242931/webrev.01/
> 
> Thanks,
> 
> -Sundar
> 
> On 16/04/20 6:49 pm, David Holmes wrote:
>> +1 Please problem list the tests under the associated bug ids.
>>
>> Thanks,
>> David
>>
>> On 16/04/2020 11:10 pm, Alan Bateman wrote:
>>> On 16/04/2020 13:59, sundararajan.athijegannathan at oracle.com wrote:
>>>> Nashorn engine removal fix (8241749: Remove the Nashorn JavaScript 
>>>> Engine) missed updating few tests, config files.
>>>>
>>>> Please review.
>>>>
>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8242931
>>>>
>>>> Webrev: http://cr.openjdk.java.net/~sundar/8242931/webrev.00/
>>>>
>>>> PS I'm disabling tests by adding @ignore. I've filed bugs for the 
>>>> tests. Please revisit port or remove these nashorn using tests
>>> Can you use the jtreg exclude lists (ProblemList.txt files) instead? 
>>> We moved away from @ignore a few ago to make it easier to track 
>>> excluded tests.
>>>
>>> -Alan




From sundararajan.athijegannathan at oracle.com  Thu Apr 16 18:00:44 2020
From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com)
Date: Thu, 16 Apr 2020 23:30:44 +0530
Subject: RFR 8242931: Few more tests that use nashorn have been missed
In-Reply-To: <54b64a4b-5d4f-0fc9-3242-b50fdcb4b0e1@oracle.com>
References: <f819ece5-83e7-74e0-4aa3-7c16f3e974ca@oracle.com>
 <6e45f66e-4b23-b367-dfc1-ed473a6f39d6@oracle.com>
 <d3114e6a-4306-6cd1-bdd9-eda60b27ba82@oracle.com>
 <1707876a-a671-d7cc-6af3-59dcc11f6aab@oracle.com>
 <54b64a4b-5d4f-0fc9-3242-b50fdcb4b0e1@oracle.com>
Message-ID: <0c86efad-1cbb-55a3-7872-bd9453a596c2@oracle.com>

Thanks Daniel.

Fixed it. Updated webrev: 
http://cr.openjdk.java.net/~sundar/8242931/webrev.02/

PS. Submitting mach5 job concurrently.

Thanks,

-Sundar


On 16/04/20 11:16 pm, Daniel Fuchs wrote:
> Hi Sundar,
>
> I spotted a typo (generial vs generic) test/jdk/ProblemList.txt:
>
> ?936 java/util/ServiceLoader/ReloadTest.java 8242935 generial-all
>
> This is not an area I know well - so maybe wait for at least
> another opinion (reviewer or not) but what I saw looked
> reasonable.
>
> You have my review.
>
> best regards,
>
> -- daniel
>
> On 16/04/2020 17:26, sundararajan.athijegannathan at oracle.com wrote:
>> Please review updated webrev: 
>> http://cr.openjdk.java.net/~sundar/8242931/webrev.01/
>>
>> Thanks,
>>
>> -Sundar
>>
>> On 16/04/20 6:49 pm, David Holmes wrote:
>>> +1 Please problem list the tests under the associated bug ids.
>>>
>>> Thanks,
>>> David
>>>
>>> On 16/04/2020 11:10 pm, Alan Bateman wrote:
>>>> On 16/04/2020 13:59, sundararajan.athijegannathan at oracle.com wrote:
>>>>> Nashorn engine removal fix (8241749: Remove the Nashorn JavaScript 
>>>>> Engine) missed updating few tests, config files.
>>>>>
>>>>> Please review.
>>>>>
>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8242931
>>>>>
>>>>> Webrev: http://cr.openjdk.java.net/~sundar/8242931/webrev.00/
>>>>>
>>>>> PS I'm disabling tests by adding @ignore. I've filed bugs for the 
>>>>> tests. Please revisit port or remove these nashorn using tests
>>>> Can you use the jtreg exclude lists (ProblemList.txt files) 
>>>> instead? We moved away from @ignore a few ago to make it easier to 
>>>> track excluded tests.
>>>>
>>>> -Alan
>
>
>

From daniel.fuchs at oracle.com  Thu Apr 16 18:11:10 2020
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Thu, 16 Apr 2020 19:11:10 +0100
Subject: RFR 8242931: Few more tests that use nashorn have been missed
In-Reply-To: <0c86efad-1cbb-55a3-7872-bd9453a596c2@oracle.com>
References: <f819ece5-83e7-74e0-4aa3-7c16f3e974ca@oracle.com>
 <6e45f66e-4b23-b367-dfc1-ed473a6f39d6@oracle.com>
 <d3114e6a-4306-6cd1-bdd9-eda60b27ba82@oracle.com>
 <1707876a-a671-d7cc-6af3-59dcc11f6aab@oracle.com>
 <54b64a4b-5d4f-0fc9-3242-b50fdcb4b0e1@oracle.com>
 <0c86efad-1cbb-55a3-7872-bd9453a596c2@oracle.com>
Message-ID: <527b338d-066e-f3e5-66a5-1b8e1c0745d1@oracle.com>

On 16/04/2020 19:00, sundararajan.athijegannathan at oracle.com wrote:
> Thanks Daniel.
> 
> Fixed it. Updated webrev: 
> http://cr.openjdk.java.net/~sundar/8242931/webrev.02/

LGTM

best regards,

-- daniel

> 
> PS. Submitting mach5 job concurrently.
> 
> Thanks,
> 
> -Sundar

From mandy.chung at oracle.com  Thu Apr 16 18:48:41 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Thu, 16 Apr 2020 11:48:41 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <330b7f12-4357-5f2f-7fa3-7896a1c3415e@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <1a5eb022-dd7e-b0bb-750f-c43be195eb42@oracle.com>
 <da9f80ff-6d63-aebe-a9fa-1a683eb42bbc@oracle.com>
 <4A360464-1873-43A8-B423-C93D1BEF9895@oracle.com>
 <51adeaaf-9c85-240c-ed0d-f460c117ba46@oracle.com>
 <330b7f12-4357-5f2f-7fa3-7896a1c3415e@oracle.com>
Message-ID: <d88e0a80-52bf-ee50-aad1-698a49a25bcf@oracle.com>



On 4/16/20 11:42 AM, serguei.spitsyn at oracle.com wrote:
> Hi Mandy,
>
> I have a couple of minor comments on the Serviceability spec update.
>
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.06-svc-spec-changes/src/jdk.jdi/share/classes/com/sun/jdi/ReferenceType.java.udiff.html
>
> ?Replace: "The returned name is the same form as ..." => "The returned 
> name is of the same form as ..."
>
>
> Example is:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.06-svc-spec-changes/src/jdk.jdi/share/classes/com/sun/jdi/Type.java.udiff.html
>
> ? + * Returns the name of this type. The result is of the same form as
>
>
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.06-svc-spec-changes/src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java.udiff.html
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.06-svc-spec-changes/src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java.udiff.html
>
> Both files above have this: "a `class` file representation".
> It feels like something is wrong with the `class` part.
> Should it say `class file` or maybe use some other formatting?
>
> Otherwise, the Serviceability spec update looks great.
> I'll look at the non-Serviceability spec changes too.

Thanks for catching these typo/format issue.? I have fixed them in my 
local repo:

diff --git 
a/src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java 
b/src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java
--- 
a/src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java
+++ 
b/src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java
@@ -391,7 +391,7 @@
 ????? * <p>
 ????? * A class or interface creation can be triggered by one of the 
following:
 ????? * <ul>
-???? * <li>by loading and deriving a class from a `class` file 
representation
+???? * <li>by loading and deriving a class from a {@code class} file 
representation
 ????? *???? using class loader (see JVMS {@jvms 5.3}).
 ????? * <li>by invoking {@link 
java.lang.invoke.MethodHandles.Lookup#defineHiddenClass(byte[], boolean, 
java.lang.invoke.MethodHandles.Lookup.ClassOption...)
 ????? *???? Lookup::defineHiddenClass} that creates a {@link Class#isHidden
diff --git a/src/jdk.jdi/share/classes/com/sun/jdi/ReferenceType.java 
b/src/jdk.jdi/share/classes/com/sun/jdi/ReferenceType.java
--- a/src/jdk.jdi/share/classes/com/sun/jdi/ReferenceType.java
+++ b/src/jdk.jdi/share/classes/com/sun/jdi/ReferenceType.java
@@ -85,7 +85,7 @@
 ?{
 ???? /**
 ????? * Returns the name of this {@code ReferenceType} object.
-???? * The returned name is the same form as the name returned by
+???? * The returned name is of the same form as the name returned by
 ????? * {@link Class#getName()}.
 ????? *
 ????? * @return a string containing the type name.
diff --git a/src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java 
b/src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java
--- a/src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java
+++ b/src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java
@@ -137,7 +137,7 @@
 ????? * <p>
 ????? * A class or interface creation can be triggered by one of the 
following:
 ????? * <ul>
-???? * <li>by loading and deriving a class from a `class` file 
representation
+???? * <li>by loading and deriving a class from a {@code class} file 
representation
 ????? *???? using class loader (see JVMS {@jvms 5.3}).
 ????? * <li>by invoking {@link 
java.lang.invoke.MethodHandles.Lookup#defineHiddenClass(byte[], boolean, 
java.lang.invoke.MethodHandles.Lookup.ClassOption...)
 ????? *???? Lookup::defineHiddenClass} that creates a {@link Class#isHidden

Mandy

From david.holmes at oracle.com  Fri Apr 17 02:19:08 2020
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 17 Apr 2020 12:19:08 +1000
Subject: RFR 8242931: Few more tests that use nashorn have been missed
In-Reply-To: <0c86efad-1cbb-55a3-7872-bd9453a596c2@oracle.com>
References: <f819ece5-83e7-74e0-4aa3-7c16f3e974ca@oracle.com>
 <6e45f66e-4b23-b367-dfc1-ed473a6f39d6@oracle.com>
 <d3114e6a-4306-6cd1-bdd9-eda60b27ba82@oracle.com>
 <1707876a-a671-d7cc-6af3-59dcc11f6aab@oracle.com>
 <54b64a4b-5d4f-0fc9-3242-b50fdcb4b0e1@oracle.com>
 <0c86efad-1cbb-55a3-7872-bd9453a596c2@oracle.com>
Message-ID: <9d6b5ca2-022b-42e4-9dcf-d43078ca68eb@oracle.com>

Hi Sundar,

On 17/04/2020 4:00 am, sundararajan.athijegannathan at oracle.com wrote:
> Thanks Daniel.
> 
> Fixed it. Updated webrev: 
> http://cr.openjdk.java.net/~sundar/8242931/webrev.02/

These changes seem okay in themselves.

Thanks,
David

> PS. Submitting mach5 job concurrently.
> 
> Thanks,
> 
> -Sundar
> 
> 
> On 16/04/20 11:16 pm, Daniel Fuchs wrote:
>> Hi Sundar,
>>
>> I spotted a typo (generial vs generic) test/jdk/ProblemList.txt:
>>
>> ?936 java/util/ServiceLoader/ReloadTest.java 8242935 generial-all
>>
>> This is not an area I know well - so maybe wait for at least
>> another opinion (reviewer or not) but what I saw looked
>> reasonable.
>>
>> You have my review.
>>
>> best regards,
>>
>> -- daniel
>>
>> On 16/04/2020 17:26, sundararajan.athijegannathan at oracle.com wrote:
>>> Please review updated webrev: 
>>> http://cr.openjdk.java.net/~sundar/8242931/webrev.01/
>>>
>>> Thanks,
>>>
>>> -Sundar
>>>
>>> On 16/04/20 6:49 pm, David Holmes wrote:
>>>> +1 Please problem list the tests under the associated bug ids.
>>>>
>>>> Thanks,
>>>> David
>>>>
>>>> On 16/04/2020 11:10 pm, Alan Bateman wrote:
>>>>> On 16/04/2020 13:59, sundararajan.athijegannathan at oracle.com wrote:
>>>>>> Nashorn engine removal fix (8241749: Remove the Nashorn JavaScript 
>>>>>> Engine) missed updating few tests, config files.
>>>>>>
>>>>>> Please review.
>>>>>>
>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8242931
>>>>>>
>>>>>> Webrev: http://cr.openjdk.java.net/~sundar/8242931/webrev.00/
>>>>>>
>>>>>> PS I'm disabling tests by adding @ignore. I've filed bugs for the 
>>>>>> tests. Please revisit port or remove these nashorn using tests
>>>>> Can you use the jtreg exclude lists (ProblemList.txt files) 
>>>>> instead? We moved away from @ignore a few ago to make it easier to 
>>>>> track excluded tests.
>>>>>
>>>>> -Alan
>>
>>
>>

From nick.gasson at arm.com  Fri Apr 17 05:12:11 2020
From: nick.gasson at arm.com (Nick Gasson)
Date: Fri, 17 Apr 2020 13:12:11 +0800
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <71a1abc6-cbd8-9a1e-2027-d5ac5ba9ab9b@redhat.com>
References: <CA+3eh11or=pxZ7D752GJbNm7a7=OofxX74Gmza2HJSmgD_9ZNA@mail.gmail.com>
 <5B92DB3F-A436-4358-A8B4-56977A6EDE6E@oracle.com>
 <7fe5e846-8157-0d53-69a7-f2c8cf878757@redhat.com>
 <71a1abc6-cbd8-9a1e-2027-d5ac5ba9ab9b@redhat.com>
Message-ID: <854ktisnkk.fsf@arm.com>

On 03/13/20 17:42 pm, Andrew Dinn wrote:
> On 13/03/2020 09:08, Andrew Haley wrote:
>> I could submit a patch to the offending files so that everyone editing
>> them with emacs will benefit. I'm not sure if that'll offend anyone's
>> sense of purity, though. It only needs this at the end of the files:
>>
>> // Local Variables:
>> // mode: fundamental
>> // End:
>>
>> WDYT?
> All doubt re that question can be readily resolved by my handy,
> /prescriptive/ emacs-as-one-true-OpenJDK-IDE-of-choice guide. Any takers?
>

Seems like many of us were suffering in silence. I reported this to the
cc-mode author: the patch here is a huge improvement, although it's
still a bit slow.

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=40052#8


Thanks,
Nick

From matthias.baesken at sap.com  Fri Apr 17 06:28:40 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 17 Apr 2020 06:28:40 +0000
Subject: RFR: 8242626: enhance posix print_rlimit_info
In-Reply-To: <c8c56831-cd1e-861f-52a8-a66a23ca2858@oracle.com>
References: <AM0PR02MB4227F897CBF6AA514A7343D293DA0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <c8c56831-cd1e-861f-52a8-a66a23ca2858@oracle.com>
Message-ID: <AM0PR02MB422759FE453DB2F89B60360393D90@AM0PR02MB4227.eurprd02.prod.outlook.com>

Hi Gerard,  thanks for the review !

> output and I didn't find any, but did you run any tests just in case?

Yes I had the patch in our internal build/test queue (incl JCK/jtreg and others) , I did not notice any issues with the patch .

Best regards, Matthias


-----Original Message-----
From: gerard ziemski <gerard.ziemski at oracle.com> 
Sent: Donnerstag, 16. April 2020 17:53
To: Baesken, Matthias <matthias.baesken at sap.com>; 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
Subject: Re: RFR: 8242626: enhance posix print_rlimit_info

hi Matthias,

Nice enhancement, I like that you added the hard limit and that we now 
check the return code.

+1 from me (small r reviewer)

ps. I checked our tests to see whether any one of them parses the rlimit 
output and I didn't find any, but did you run any tests just in case?


cheers

On 4/14/20 10:27 AM, Baesken, Matthias wrote:
> Please review this small enhancement of  os::Posix::print_rlimit_info.
>
> It adds  return code handling  of  getrlimit ,  output of some more limit values and  output of  hard limits (additionally to the existing soft limits) .
>
> Bug/webrev :
>
> https://bugs.openjdk.java.net/browse/JDK-8242626
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8242626.1/
>
> Thanks, Matthias


From david.holmes at oracle.com  Fri Apr 17 06:45:58 2020
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 17 Apr 2020 16:45:58 +1000
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <df621fd2-0eaf-cb24-788f-12166b271303@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
 <135a6496-5b54-9117-5cbf-e3211ddc32dd@oracle.com>
 <bbb2b5ef-62f2-6507-95f7-03712a382153@oracle.com>
 <e242e057-de6e-493a-eb00-068b2db8c9fa@oracle.com>
 <df621fd2-0eaf-cb24-788f-12166b271303@oracle.com>
Message-ID: <26105810-b9be-faec-6577-616ef26e84a7@oracle.com>

Hi Leo,

I've taken a look at the complete set of changes and the only thing I 
strongly object to is the change to:

src/hotspot/share/jvmci/jvmciExceptions.hpp

as it just doesn't make any sense to me. If you were to write the 
expanded macros out in full you would write the code as it currently is 
defined in the macro. If a method returns a pointer you would write:

return NULL;

If a method returns a jint/jbyte/jlong etc you would write

return 0;

Cheers,
David

On 14/04/2020 11:53 pm, Leo Korinth wrote:
> Hi!
> 
> On 06/04/2020 01:37, David Holmes wrote:
> 
>>>
>>> I agree that it could be compared against NULL_WORD, do you want me 
>>> to change it? (I have no opinion on this issue)
>>
>> If we're trying to clean up the use of 0, NULL, NULL_WORD then I think 
>> it should be changed.
> 
> Fixed in several locations.
> 
>>>>
>>>> ?> new NULL value does unfortunately not convert to method pointers
>>>>
>>>> Is that a temporary limitation? I'm not sure what progress were are 
>>>> making if some pointers can use NULL and others must use 0.
>>>
>>> I believe this to be a temporary limitation, my understanding is that 
>>> when we get c++11 nullptr will convert to method pointers as well, I 
>>> could and think I should revert these changes as I did revert my 
>>> implementation of NULL, would you like that? My null_t type could, to 
>>> my knowledge, not auto convert to method pointers at least not 
>>> without template varargs stuff that comes in later versions of the 
>>> standard.
>>
>> This would look better to me as NULL.
> 
> I have reverted this and a few other places.
> 
> 
>>>
>>> This is the same issue as before with string literals and macros, my 
>>> arguing is that I will not try to prove that value is (or in the 
>>> future always will be) a string literal, instead I will make the code 
>>> work for both string literals and pointers. I hope you do not mind 
>>> the extra parentheses I added around "value".
>>
>> Normal style rules for macros say to use the macro arg in parentheses 
>> - but there are three other uses in the same macro.
>>
> 
> I will keep the parenthesis on the line I changed and do no additional 
> such change.
> 
> 
>>
>>> I could change CHECK_0 to use NULL_WORD though and if I do, I get 
>>> compiler errors. CHECK_0 is used for both pointers and integral 
>>> zeroes and types that implicitly converts from either. What the 
>>> original thought was is unclear to me; is CHECK_0 to be used to 
>>> return things convertible from pointers? If so, what is CHECK_NULL for?
>>
>> No CHECK_0 is intended for int-returning functions (which would also 
>> extend to intptr_t).
> 
> It might be intended for it, but it is not used in such a way (therefore 
> my remark on compile failures when using NULL_WORD)
> 
> I will do no change here.
> 
> 
>>>
>>>> ---
>>>>
>>>> src/hotspot/share/utilities/globalDefinitions_solstudio.hpp
>>>>
>>>> It isn't clear that the issue of passing 0 as NULL in a varargs 
>>>> situation is now fixed in Sol Studio.
>>>>
>>>
>>> I guess tier1-3 would fail if not? Maybe that is a false assumption 
>>> of mine?
>>
>> I would consider that a necessary but not sufficient condition.
>>
> 
> I reverted my NULL/NULL_WORD redefinition so it is no issue any more at 
> least.
> 
>>>>
>>>> src/hotspot/share/utilities/linkedlist.hpp
>>>>
>>>> -??? return equal<E>(_data, t, NULL);
>>>> +??? return equal<E>(_data, t, 0);
>>>>
>>>> Again concerned about the method/function pointer situation.
>>>
>>> I am not concerned :-) but I will revert this as it is not necessary 
>>> now or in the future as I reverted my original implementation and 
>>> c++11 will fix this.
>>
>> Ok.
> 
> Fixed
> 
> In additions to these changes I have also /reverted/ all changes to 
> NULL/NULL_WORD definitions. In the end Kim did not like them, and they 
> can be fixed later when we get a more modern C++. To reflect this I will 
> also change the CR title to "Improve NULL usage"
> 
> Webrev:
> http://cr.openjdk.java.net/~lkorinth/8240110/part1 (NULL)
> http://cr.openjdk.java.net/~lkorinth/8240110/part2 (NULL_WORD)
> http://cr.openjdk.java.net/~lkorinth/8240110/part3 (reverse somewhat)
> 
> new in this mail:
> http://cr.openjdk.java.net/~lkorinth/8240110/part4 (improvements 
> suggested by David)
> http://cr.openjdk.java.net/~lkorinth/8240110/part5 (reverse my changes 
> to NULL/NULL_WORD itself)
> 
> total combined change:
> http://cr.openjdk.java.net/~lkorinth/8240110/1
> 
> Testing:
> running tier 1-3 (looks good so far)
> 
> Thanks,
> Leo

From matthias.baesken at sap.com  Fri Apr 17 07:07:09 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 17 Apr 2020 07:07:09 +0000
Subject: RFR: 8242626: enhance posix print_rlimit_info
In-Reply-To: <AM0PR0202MB329764EC55E8688516D582359AD80@AM0PR0202MB3297.eurprd02.prod.outlook.com>
References: <AM0PR02MB4227F897CBF6AA514A7343D293DA0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <c8c56831-cd1e-861f-52a8-a66a23ca2858@oracle.com>
 <AM0PR0202MB329764EC55E8688516D582359AD80@AM0PR0202MB3297.eurprd02.prod.outlook.com>
Message-ID: <AM0PR02MB422766AA0FE32FE0211CBFAC93D90@AM0PR02MB4227.eurprd02.prod.outlook.com>

Hi Martin , thanks for the review .

>A very minor hotspot coding style issue:
>Curly brackets should be used for the if and else parts.
>And the indentation looks odd in line 432.

I adjusted this.

>It'd be nice to add an example output to the issue description.

I added an example limits output to the JBS issue .

Best regards, Matthias

From kim.barrett at oracle.com  Fri Apr 17 07:46:33 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Fri, 17 Apr 2020 03:46:33 -0400
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <df621fd2-0eaf-cb24-788f-12166b271303@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
 <135a6496-5b54-9117-5cbf-e3211ddc32dd@oracle.com>
 <bbb2b5ef-62f2-6507-95f7-03712a382153@oracle.com>
 <e242e057-de6e-493a-eb00-068b2db8c9fa@oracle.com>
 <df621fd2-0eaf-cb24-788f-12166b271303@oracle.com>
Message-ID: <98749A40-89C8-4E15-B46A-23F0D1CB2A32@oracle.com>

> On Apr 14, 2020, at 9:53 AM, Leo Korinth <leo.korinth at oracle.com> wrote:
> On 06/04/2020 01:37, David Holmes wrote:
>>> I could change CHECK_0 to use NULL_WORD though and if I do, I get compiler errors. CHECK_0 is used for both pointers and integral zeroes and types that implicitly converts from either. What the original thought was is unclear to me; is CHECK_0 to be used to return things convertible from pointers? If so, what is CHECK_NULL for?
>> No CHECK_0 is intended for int-returning functions (which would also extend to intptr_t).
> 
> It might be intended for it, but it is not used in such a way (therefore my remark on compile failures when using NULL_WORD)
> 
> I will do no change here.

It sounds like there are some places using CHECK_0 that should be
using CHECK_NULL, and they should be fixed. But that can be a separate
change. It might be aided (and backsliding prevented) by doing
something similar to what I suggest for JVMCI_CHECK_0/NULL below.

> total combined change:
> http://cr.openjdk.java.net/~lkorinth/8240110/1

I've gone through all the changes in the "total combined change", with
comments below.

------------------------------------------------------------------------------ 

In the end you seem to have backed away from fixing the definition of
NULL_WORD, and I'm not sure why. Though I think the definition that
was proposed is at least unnecessarily baroque, possibly wrong. It
seems to me that a good and sufficient change for that would be to
delete all the existing platform definitions and put the following in
globalDefinitions.hpp:

const intptr_t NULL_WORD = 0;

Then it's no longer a null pointer constant, so can't be misused in
contexts where a pointer is required, will be treated as an integral
value by templates, &etc.

I wouldn't worry about this comment in globalDefinitions_solstudio.hpp:

  "On some platforms, sizeof(intptr_t) > sizeof(void*)"

or you can be really paranoid and STATIC_ASSERT that they are the same
size. Though I really wonder what platform(s) that's talking about.
That comment comes from 6452081: 3/4 Allow for Linux builds with Sun
Studio Linux compilers.  So maybe 32bit Solaris Studio Linux is weird?

------------------------------------------------------------------------------

There is a pattern to some of the changes where I'd prefer something
different. 

We have classes with a pointer member that is initialized to NULL when
default constructed, and initialized from the argument by an implicit
conversion constructor with an appropriately typed pointer parameter.
Clearly this pattern permits implicit conversion construction from
NULL, but the proposed changes suggest one should avoid doing so. But
nothing is being done to prevent it.

Implicit conversion constructors, like implicit conversion operators,
can be problematic. I think a better change for these would be to make
the conversion from pointer constructor explict and use default
construction for the NULL pointer case. But whether a change from
implicit to explicit conversion is desirable might not be obvious in
all cases, and ought to be discussed with people who work with the
relevant parts of the code a lot.

Rather than repeat all that, I've used the shorthand "PATTERN CHANGE"
to indicate these in the comments below.

------------------------------------------------------------------------------ 
src/hotspot/cpu/sparc/c1_CodeStubs_sparc.cpp
303     AddressLiteral addrlit(NULL_WORD, metadata_Relocation::spec(_index));
318     AddressLiteral addrlit(NULL_WORD, oop_Relocation::spec(_index));

I think these should be left as NULL rather than being changed to
NULL_WORD.  The expected type is the address type, and other types are
converted to that via casts in constructor overloads.

Similarly here:
src/hotspot/cpu/sparc/c1_LIRAssembler_sparc.cpp
 426   AddressLiteral addrlit(NULL_WORD, oop_Relocation::spec(oop_index));
 445   AddressLiteral addrlit(NULL_WORD, metadata_Relocation::spec(index));

------------------------------------------------------------------------------
src/hotspot/os/linux/cgroupV1Subsystem_linux.hpp
[old code]
100     CachingCgroupController* _memory = NULL;
101     CgroupV1Controller* _cpuset = NULL;
102     CachingCgroupController* _cpu = NULL;
103     CgroupV1Controller* _cpuacct = NULL;

How does this even compile?  Non-static member initializers are a
C++11 feature.  Weirdly, g++ with -std=gnu++98 warns about this for
some types/initializers but not for others.

The proposed change is better, even with C++11.

Similarly here:
src/hotspot/os/linux/cgroupV2Subsystem_linux.hpp

------------------------------------------------------------------------------
src/hotspot/os/solaris/attachListener_solaris.cpp
461   status = ::sema_init(&_wakeup, 0, NULL, NULL);
=>
461   status = ::sema_init(&_wakeup, 0, 0, NULL);

The old code appears to have been cribbed from the documentation:
https://docs.oracle.com/cd/E36784_01/html/E36874/sema-init-3c.html
See the example of "Default semaphore initialization (intra-process)".

Rather than a literal 0, better would be USYNC_THREAD.  Presumably the
value of USYNC_THREAD is 0.

------------------------------------------------------------------------------
src/hotspot/os/windows/os_windows.cpp
1659   if (!GetFileVersionInfo(kernel32_path, 0, version_size, version_info)) {

Given that the second argument, which was changed from NULL to 0, is a
"handle" (though unused), I think using NULL was appropriate and this
change shouldn't be made.

------------------------------------------------------------------------------
src/hotspot/share/classfile/javaClasses.cpp
 851       k->set_java_mirror_handle(NULL);
=>
 851       k->set_java_mirror_handle(OopHandle(NULL));

PATTERN CHANGE

Similarly here:
src/hotspot/share/classfile/moduleEntry.hpp
 82     _module = OopHandle(NULL);
 84     _pd = OopHandle(NULL);

And here:
src/hotspot/share/classfile/modules.cpp
156     version_symbol = TempNewSymbol(NULL);
161   TempNewSymbol location_symbol(NULL);
368     version_symbol = TempNewSymbol(NULL);
373   TempNewSymbol location_symbol(NULL);

Although I'm not sure why the assignments on line 156 and line 368 is
even being done, since each version_symbol was already default
initialized.

------------------------------------------------------------------------------
src/hotspot/share/code/codeHeapState.cpp
 140     if (((_termString) != 0) && (strlen(_termString) > 0)){   \
 154     if (((_termString) != 0) && (strlen(_termString) > 0)){   \

I think these changes to compare _termstring with 0 rather than NULL
should not be made.  _termString is supposed to be a `[const] char*`,
so comparing to NULL is preferable to comparing to 0.

Though at a brief skim it looks like _termString is always provided
and never NULL, so it might be better to assert that _termString is
non-NULL.  But that's a completely different change.

------------------------------------------------------------------------------
src/hotspot/share/gc/parallel/psAdaptiveSizePolicy.cpp
src/hotspot/share/gc/parallel/psParallelCompact.cpp
src/hotspot/share/gc/shared/memset_with_concurrent_readers.hpp
src/hotspot/share/gc/z/zErrno.cpp

These #include changes seem unrelated to NULL.

------------------------------------------------------------------------------
src/hotspot/share/jfr/jni/jfrGetAllEventClasses.cpp
172   assert(add_method_signature != NULL, "invariant");
=>
172   assert(add_method_sig_sym != NULL, "invariant");

This seems to be an unrelated bug fix; the wrong value was being
compared to NULL.

------------------------------------------------------------------------------
src/hotspot/share/jfr/jni/jfrJavaCall.cpp
 57   assert(_storage != NULL, "invariant");
=>
 57   assert(_storage_index < (int) ARRAY_SIZE(_storage), "invariant");

_storage is an array, so the original assert is definitionally true,
but it's harmless to verify, in case future maintenance makes it an
allocated array rather than inline.

The size of _storage is definitionally SIZE, and _storage_index is
checked against SIZE a couple lines later, making the proposed
replacement just a duplicative check.

So I think the proposed change should not be made.

Similarly at lines 65, 73, and 91.

------------------------------------------------------------------------------ 
src/hotspot/share/jvmci/jvmciCompilerToVM.cpp  
1562   JVMCIENV->set_HotSpotStackFrameReference_localIsVirtual(hs_frame, 0);

This change to use literal 0 rather than NULL as a null pointer
constant does not seem like an improvement to me.  I think this change
should not be made.

------------------------------------------------------------------------------
src/hotspot/share/jvmci/jvmciCompilerToVM.cpp  

I looked at the changes from JVMCI_CHECK_0 to JVMCI_CHECK_NULL and
think they are correct, assuming a suitable definition of the latter.
(Though see below regarding that.)

------------------------------------------------------------------------------
src/hotspot/share/jvmci/jvmciCompilerToVM.cpp  
2572   JVMCIObjectArray current_array = NULL;
=>
2572   JVMCIObjectArray current_array(NULL); 

PATTERN CHANGE

------------------------------------------------------------------------------
src/hotspot/share/jvmci/jvmciCompilerToVMInit.cpp
267     if ((value) != 0) { \

Value is a char*; I don't think this change should be made.

------------------------------------------------------------------------------ 
src/hotspot/share/jvmci/jvmciExceptions.hpp  
50 #define JVMCI_CHECK_NULL                               JVMCI_CHECK_(NULL) 
=>
50 #define JVMCI_CHECK_NULL                               JVMCI_CHECK_(0) 

For platforms that warn about converting NULL to an integral value
this is a type safety regression. I think this change should not be
made.

------------------------------------------------------------------------------
src/hotspot/share/jvmci/jvmciExceptions.hpp 
49 #define JVMCI_CHECK_0                                  JVMCI_CHECK_(0)
=>
49 #define JVMCI_CHECK_0                                  JVMCI_CHECK_(NULL_WORD)

I kind of think this is an abuse of NULL_WORD. It also serves no real
purpose with the current definitions of NULL_WORD (though would with
the suggestion far above). If the goal is to have something that isn't
a null pointer constant so it can't be used in inappropriate
contexts (which is a reasonable goal), then replace "0" with "int(0)"
(and comment appropriately).

------------------------------------------------------------------------------
src/hotspot/share/oops/access.hpp
296 class ArrayAccess ...

How about adding the following private functions to this class, and
use them in the obvious places:

  template<typename T> static const T* const_raw_null() { return NULL; }
  template<typename T> static       T*       raw_null() { return NULL; }

------------------------------------------------------------------------------
src/hotspot/share/oops/constantPool.cpp
 270     set_resolved_references(OopHandle(NULL));
 302     set_resolved_references(OopHandle(NULL));
 376     set_resolved_references(OopHandle(NULL));

PATTERN CHANGE

Similarly here:
src/hotspot/share/oops/cpCache.cpp
766   set_resolved_references(OopHandle(NULL));

src/hotspot/share/oops/klass.cpp
549   _java_mirror = OopHandle(NULL);

src/hotspot/share/prims/jni.cpp
 319   TempNewSymbol class_name(NULL);

src/hotspot/share/prims/jvm.cpp
 954   TempNewSymbol class_name(NULL);

src/hotspot/share/prims/methodHandles.cpp
1311   TempNewSymbol name(NULL);
1312   TempNewSymbol sig(NULL);

------------------------------------------------------------------------------
src/hotspot/share/runtime/os.cpp
1324   if (path == NULL || strlen(path) == 0 || file_name_length == 0) {

`(size_t)NULL` ??  Good catch!

------------------------------------------------------------------------------


From coleen.phillimore at oracle.com  Fri Apr 17 12:21:47 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Fri, 17 Apr 2020 08:21:47 -0400
Subject: RFR (XXL): 8223347: Integration of Vector API (Incubator):
 General HotSpot changes
In-Reply-To: <25a564a1-7f40-6988-060f-86b06e02ad21@oracle.com>
References: <c1bdf88c-5de2-d069-5f31-5a95c6988bf8@oracle.com>
 <25a564a1-7f40-6988-060f-86b06e02ad21@oracle.com>
Message-ID: <c48cb1f8-0e4d-fc38-8258-9d4892466739@oracle.com>


Hi, I looked at the runtime changes and they seem fine.

http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/all.00-03/src/hotspot/share/classfile/vmSymbols.hpp.udiff.html

I really hate that intrinsics are in this header file and this change 
has to add so much in this file.? I filed 
https://bugs.openjdk.java.net/browse/JDK-8243066 for after this change.? 
Feel free to add details to the RFE.? At least, I think all of these 
symbols are shared in CDS.

Coleen

On 4/16/20 8:32 AM, Vladimir Ivanov wrote:
> Hi,
>
> Any more reviews, please? Especially, compiler and runtime-related 
> changes.
>
> Thanks in advance!
>
> Best regards,
> Vladimir Ivanov
>
> On 04.04.2020 02:12, Vladimir Ivanov wrote:
>> Hi,
>>
>> Following up on review requests of API [0] and Java implementation 
>> [1] for Vector API (JEP 338 [2]), here's a request for review of 
>> general HotSpot changes (in shared code) required for supporting the 
>> API:
>>
>>
>> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/all.00-03/ 
>>
>>
>> (First of all, to set proper expectations: since the JEP is still in 
>> Candidate state, the intention is to initiate preliminary round(s) of 
>> review to inform the community and gather feedback before sending out 
>> final/official RFRs once the JEP is Targeted to a release.)
>>
>> Vector API (being developed in Project Panama [3]) relies on JVM 
>> support to utilize optimal vector hardware instructions at runtime. 
>> It interacts with JVM through intrinsics (declared in 
>> jdk.internal.vm.vector.VectorSupport [4]) which expose vector 
>> operations support in C2 JIT-compiler.
>>
>> As Paul wrote earlier: "A vector intrinsic is an internal low-level 
>> vector operation. The last argument to the intrinsic is fall back 
>> behavior in Java, implementing the scalar operation over the number 
>> of elements held by the vector.? Thus, If the intrinsic is not 
>> supported in C2 for the other arguments then the Java implementation 
>> is executed (the Java implementation is always executed when running 
>> in the interpreter or for C1)."
>>
>> The rest of JVM support is about aggressively optimizing vector boxes 
>> to minimize (ideally eliminate) the overhead of boxing for vector 
>> values.
>> It's a stop-the-gap solution for vector box elimination problem until 
>> inline classes arrive. Vector classes are value-based and in the 
>> longer term will be migrated to inline classes once the support 
>> becomes available.
>>
>> Vector API talk from JVMLS'18 [5] contains brief overview of JVM 
>> implementation and some details.
>>
>> Complete implementation resides in vector-unstable branch of 
>> panama/dev repository [6].
>>
>> Now to gory details (the patch is split in multiple "sub-webrevs"):
>>
>> ===========================================================
>>
>> (1) 
>> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/00.backend.shared/ 
>>
>>
>> Ideal vector nodes for new operations introduced by Vector API.
>>
>> (Platform-specific back end support will be posted for review 
>> separately).
>>
>> ===========================================================
>>
>> (2) 
>> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/01.intrinsics/ 
>>
>>
>> JVM Java interface (VectorSupport) and intrinsic support in C2.
>>
>> Vector instances are initially represented as VectorBox macro nodes 
>> and "unboxing" is represented by VectorUnbox node. It simplifies 
>> vector box elimination analysis and the nodes are expanded later 
>> right before EA pass.
>>
>> Vectors have 2-level on-heap representation: for the vector value 
>> primitive array is used as a backing storage and it is encapsulated 
>> in a typed wrapper (e.g., Int256Vector - vector of 8 ints - contains 
>> a int[8] instance which is used to store vector value).
>>
>> Unless VectorBox node goes away, it needs to be expanded into an 
>> allocation eventually, but it is a pure node and doesn't have any JVM 
>> state associated with it. The problem is solved by keeping JVM state 
>> separately in a VectorBoxAllocate node associated with VectorBox node 
>> and use it during expansion.
>>
>> Also, to simplify vector box elimination, inlining of vector reboxing 
>> calls (VectorSupport::maybeRebox) is delayed until the analysis is over.
>>
>> ===========================================================
>>
>> (3) 
>> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/02.vbox_elimination/ 
>>
>>
>> Vector box elimination analysis implementation. (Brief overview: 
>> slides #36-42 [5].)
>>
>> The main part is devoted to scalarization across safepoints and 
>> rematerialization support during deoptimization. In C2-generated code 
>> vector operations work with raw vector values which live in registers 
>> or spilled on the stack and it allows to avoid boxing/unboxing when a 
>> vector value is alive across a safepoint. As with other values, 
>> there's just a location of the vector value at the safepoint and 
>> vector type information recorded in the relevant nmethod metadata and 
>> all the heavy-lifting happens only when rematerialization takes place.
>>
>> The analysis preserves object identity invariants except during 
>> aggressive reboxing (guarded by -XX:+EnableAggressiveReboxing).
>>
>> (Aggressive reboxing is crucial for cases when vectors "escape": it 
>> allocates a fresh instance at every escape point thus enabling 
>> original instance to go away.)
>>
>> ===========================================================
>>
>> (4) 
>> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/03.module.hotspot/ 
>>
>>
>> HotSpot changes for jdk.incubator.vector module. Vector support is 
>> makred experimental and turned off by default. JEP 338 proposes the 
>> API to be released as an incubator module, so a user has to specify 
>> "--add-module jdk.incubator.vector" on the command line to be able to 
>> use it.
>> When user does that, JVM automatically enables Vector API support.
>> It improves usability (user doesn't need to separately "open" the API 
>> and enable JVM support) while minimizing risks of destabilitzation 
>> from new code when the API is not used.
>>
>>
>> That's it! Will be happy to answer any questions.
>>
>> And thanks in advance for any feedback!
>>
>> Best regards,
>> Vladimir Ivanov
>>
>> [0] 
>> https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065345.html 
>>
>>
>> [1] 
>> https://mail.openjdk.java.net/pipermail/hotspot-dev/2020-April/041228.html
>>
>> [2] https://openjdk.java.net/jeps/338
>>
>> [3] https://openjdk.java.net/projects/panama/
>>
>> [4] 
>> http://cr.openjdk.java.net/~vlivanov/panama/vector/jep338/hotspot.shared/webrev.00/01.intrinsics/src/java.base/share/classes/jdk/internal/vm/vector/VectorSupport.java.html 
>>
>>
>> [5] http://cr.openjdk.java.net/~vlivanov/talks/2018_JVMLS_VectorAPI.pdf
>>
>> [6] http://hg.openjdk.java.net/panama/dev/shortlog/92bbd44386e9
>>
>> ???? $ hg clone http://hg.openjdk.java.net/panama/dev/ -b 
>> vector-unstable


From aph at redhat.com  Fri Apr 17 12:24:32 2020
From: aph at redhat.com (Andrew Haley)
Date: Fri, 17 Apr 2020 13:24:32 +0100
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <854ktisnkk.fsf@arm.com>
References: <CA+3eh11or=pxZ7D752GJbNm7a7=OofxX74Gmza2HJSmgD_9ZNA@mail.gmail.com>
 <5B92DB3F-A436-4358-A8B4-56977A6EDE6E@oracle.com>
 <7fe5e846-8157-0d53-69a7-f2c8cf878757@redhat.com>
 <71a1abc6-cbd8-9a1e-2027-d5ac5ba9ab9b@redhat.com> <854ktisnkk.fsf@arm.com>
Message-ID: <18f6fae0-6ce8-452e-4898-ef53c50677e8@redhat.com>

On 4/17/20 6:12 AM, Nick Gasson wrote:
> Seems like many of us were suffering in silence. I reported this to the
> cc-mode author: the patch here is a huge improvement, although it's
> still a bit slow.
> 
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=40052#8

You, sir, are my hero of the day.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From adinn at redhat.com  Fri Apr 17 13:04:33 2020
From: adinn at redhat.com (Andrew Dinn)
Date: Fri, 17 Apr 2020 14:04:33 +0100
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <18f6fae0-6ce8-452e-4898-ef53c50677e8@redhat.com>
References: <CA+3eh11or=pxZ7D752GJbNm7a7=OofxX74Gmza2HJSmgD_9ZNA@mail.gmail.com>
 <5B92DB3F-A436-4358-A8B4-56977A6EDE6E@oracle.com>
 <7fe5e846-8157-0d53-69a7-f2c8cf878757@redhat.com>
 <71a1abc6-cbd8-9a1e-2027-d5ac5ba9ab9b@redhat.com> <854ktisnkk.fsf@arm.com>
 <18f6fae0-6ce8-452e-4898-ef53c50677e8@redhat.com>
Message-ID: <3a69213f-ba17-8e23-f366-4ede54cf7fca@redhat.com>

On 17/04/2020 13:24, Andrew Haley wrote:
> On 4/17/20 6:12 AM, Nick Gasson wrote:
>> Seems like many of us were suffering in silence. I reported this to the
>> cc-mode author: the patch here is a huge improvement, although it's
>> still a bit slow.
>>
>> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=40052#8
> 
> You, sir, are my hero of the day.
I'll second that motion!

regards,


Andrew Dinn
-----------


From jiefu at tencent.com  Fri Apr 17 13:04:53 2020
From: jiefu at tencent.com (=?utf-8?B?amllZnUo5YKF5p2wKQ==?=)
Date: Fri, 17 Apr 2020 13:04:53 +0000
Subject: 8241423: NUMA APIs may fail to work in the docker due to
 operation not permitted(Internet mail)
Message-ID: <389A6CC1-3C5A-4B97-8532-FC2335098308@tencent.com>

Hi Bob,

Thanks for your review and sorry for the late reply.

After days of thinking, I'm still not clear how to use prctl to solve this issue.

Did you mean calling prctl with PR_GET_SECCOMP to check whether the java process is running in SECCOMP_MODE?
But I found it doesn't work in cases like:
 1) When the prctl itself is not allowed to execute
 2) When the kernel isn't configured with CONFIG_SECCOMP enabled 

Could you please make it more clear?

Thanks a lot.
Best regards,
Jie


?On 2020/4/10, 10:29 PM, "Bob Vandette" <bob.vandette at oracle.com> wrote:

    
    > On Apr 3, 2020, at 3:58 AM, jiefu(??) <jiefu at tencent.com> wrote:
    > 
    > Hi Bob,
    > 
    > Thanks for your review and helpful comments. 
    > 
    > I'm not a docker expert. 
    > Apart from the zgc crash [1], we didn't come across other problems in the docker.
    > 
    > It seems that this bug has nothing to do with the resource limit.
    > The root cause is that some NUMA-related syscalls are disabled in the docker for safety reasons.
    
    I understand but it would be useful to ensure that all aspects of supporting NUMA in containers
    work while addressing this issue.  What good is it to enable NUMA but have it not work properly.
    I?m only asking for some validation since you appear to have a proper setup.
     
    > 
    > Please note that we already have numa_available() check here [2].
    > But it failed to detect such cases.
    Ah yes, I see.
    
    > 
    > What do you think?
    
    It would be better if we could find a more supported way  of determining if our process has
    the required access rather than counting on a failing syscall.
    
    What about using prctl?
    
    http://man7.org/linux/man-pages/man2/prctl.2.html
    
    This call can be used to query if a specific capability is enabled.
    
    Bob.
    
    
    > 
    > Thanks a lot.
    > Best regards,
    > Jie
    > 
    > [1] https://bugs.openjdk.java.net/browse/JDK-8241354
    > [2] http://hg.openjdk.java.net/jdk/jdk/file/f50a7df94744/src/hotspot/os/linux/os_linux.cpp#l3182
    > 
    > On 2020/4/3, 3:58 AM, "Bob Vandette" <bob.vandette at oracle.com> wrote:
    > 
    >    Jie,
    > 
    >    Before we discuss this specific fix, I?d like to know if you have confirmed that Hotspot?s NUMA
    >    support actually functions properly when running in containers (with proper privs)?
    > 
    >    Also, do the libnuma functions work properly in response to cgroup limitations imposed by docker run --cpuset-mems?
    > 
    >    Some of the traditional kernel functions reporting resource limits only report host values and do not
    >    correctly report limits specified for containers.   To resolve this issue I have added an osContainer
    >    class to hotspot.  Included in this class is a function that reports the memory nodes available 
    >    to hotspot when running in a container.   It might be necessary to query this function when
    >    trying to configure the hotspot NUMA support.
    > 
    >    Back to your webrev, is it not possible to get the address for numa_available and
    >    then try to calling it in order to determine if NUMA can be used?
    > 
    >    If it is determined that you don?t have sufficient access, I would suggest disabling UseNUMA
    >    all together.
    > 
    >    Bob
    > 
    >> On Mar 23, 2020, at 11:58 AM, jiefu(??) <jiefu at tencent.com> wrote:
    >> 
    >> Hi all,
    >> 
    >> JBS:    https://bugs.openjdk.java.net/browse/JDK-8241423
    >> Webrev: http://cr.openjdk.java.net/~jiefu/8241423/webrev.00/
    >> 
    >> A VM fatal error may be observed if ZGC is used (see JDK-8241354).
    >> The background is that some of our products run in the docker.
    >> And for safety reasons, SYS_get_mempolicy is not allowed by default [1].
    >> 
    >> At first, we thought it just a zgc-only problem and filed JDK-8241354.
    >> But Thomas had reminded me that other collectors are also affected [2].
    >> So it would be better to fix them together.
    >> 
    >> After more investigation, we found that NUMA APIs are actually dependent on several syscalls, such as get_mempolicy, mbind and set_mempolicy.
    >> When the required syscalls are unavailable, NUMA APIs fail to work as expected.
    >> 
    >> The fix is to check whether the required syscalls are available.
    >> In theory, all NUMA-related syscalls should be checked.
    >> But it seems hard to do so because some of them will cause unexpected side effect.
    >> To fix our issue, checking get_mempolicy is enough.
    >> And just as Per suggested that we can refine this later if it turns out to be a problem [3].
    >> 
    >> Please review it and give me some advice.
    >> 
    >> Thanks a lot.
    >> Best regards,
    >> Jie
    >> 
    >> [1] https://docs.docker.com/engine/security/seccomp/
    >> [2] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028923.html
    >> [3] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028933.html
    > 
    > 
    > 
    > 
    
    
    


From fw at deneb.enyo.de  Fri Apr 17 18:27:56 2020
From: fw at deneb.enyo.de (Florian Weimer)
Date: Fri, 17 Apr 2020 20:27:56 +0200
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <854ktisnkk.fsf@arm.com> (Nick Gasson's message of "Fri, 17 Apr
 2020 13:12:11 +0800")
References: <CA+3eh11or=pxZ7D752GJbNm7a7=OofxX74Gmza2HJSmgD_9ZNA@mail.gmail.com>
 <5B92DB3F-A436-4358-A8B4-56977A6EDE6E@oracle.com>
 <7fe5e846-8157-0d53-69a7-f2c8cf878757@redhat.com>
 <71a1abc6-cbd8-9a1e-2027-d5ac5ba9ab9b@redhat.com>
 <854ktisnkk.fsf@arm.com>
Message-ID: <87mu7a9dcj.fsf@mid.deneb.enyo.de>

* Nick Gasson:

> On 03/13/20 17:42 pm, Andrew Dinn wrote:
>> On 13/03/2020 09:08, Andrew Haley wrote:
>>> I could submit a patch to the offending files so that everyone editing
>>> them with emacs will benefit. I'm not sure if that'll offend anyone's
>>> sense of purity, though. It only needs this at the end of the files:
>>>
>>> // Local Variables:
>>> // mode: fundamental
>>> // End:
>>>
>>> WDYT?
>> All doubt re that question can be readily resolved by my handy,
>> /prescriptive/ emacs-as-one-true-OpenJDK-IDE-of-choice guide. Any takers?
>>
>
> Seems like many of us were suffering in silence. I reported this to the
> cc-mode author: the patch here is a huge improvement, although it's
> still a bit slow.
>
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=40052#8

Nice.  Will the globals.hpp situation improve further as the result of
the flags modularization work?

<https://wiki.openjdk.java.net/display/HotSpot/HotSpot+Command-Line+Flags+Overhaul+-+Design+Doc>

It looks to me the large macro is removed.

From chris.plummer at oracle.com  Fri Apr 17 22:51:41 2020
From: chris.plummer at oracle.com (Chris Plummer)
Date: Fri, 17 Apr 2020 15:51:41 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <51adeaaf-9c85-240c-ed0d-f460c117ba46@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <1a5eb022-dd7e-b0bb-750f-c43be195eb42@oracle.com>
 <da9f80ff-6d63-aebe-a9fa-1a683eb42bbc@oracle.com>
 <4A360464-1873-43A8-B423-C93D1BEF9895@oracle.com>
 <51adeaaf-9c85-240c-ed0d-f460c117ba46@oracle.com>
Message-ID: <6890c88c-809b-fbea-aa10-675661ef2c68@oracle.com>

On 4/16/20 9:45 AM, Mandy Chung wrote:
> On 4/14/20 11:51 AM, Paul Sandoz wrote:
>> Looks good to me (not familiar with all the code areas.
>>
>> Minor suggestion:
>>
>> MethodHandles.java
>>
>> 1811????????? * ASCII periods. For the instance of {@link 
>> java.lang.Class} representing {@code C}:
>> 1812????????? * <ul>
>> 1813????????? * <li> {@link Class#getName()} returns the string 
>> {@code GN + "/" + <suffix>},
>> 1814????????? *????? even though this is not a valid binary class or 
>> interface name.</li>
>> 1815????????? * <li> {@link Class#descriptorString()} returns the string
>> 1816????????? *????? {@code "L" + N + ";" + "/" + <suffix> },
>> 1817????????? *????? even though this is not a valid type descriptor 
>> name.</li>
>> 1818????????? * </ul>
>>
>> Add another bullet:
>>
>> ?
>> even though this is not a valid type descriptor name; and
>>
>> - therefore {@link Class#describeConstable} returns an empty {@code 
>> Optional}.
>> ?
>>
>> ?
>
> OK.?? I add this bullet:
>
> - Class.describeConstable() returns an empty optional as C cannot be 
> described in nominal form.
>
> The webrev and spec was updated [1] for descriptor string to be of the 
> form "Lfoo/Foo.1234;" to mitigate the compatibility risk.? Th
>
> Specdiff with serviceability spec change:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff-inc/ 
>
> Specdiff without svc spec change:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff-descriptor-string/overview-summary.html 
>
>
> Webrev:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.06-descriptor-string/ 
>
>
> Svc spec change webrev:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.06-svc-spec-changes/ 
>
>
> thanks
> Mandy
> [1] 
> https://mail.openjdk.java.net/pipermail/valhalla-dev/2020-April/007155.html

Hi Mandy,

Thanks for updating the svc specs. Some comments below:


In the JDWP spec update, you changed "JNI signature" to "type signature" 
in one place, but left it as "JNI signature" everywhere else. Should 
they all be changed?


In the JDWP spec for ClassLoaderReference.VisibleClasses:

 ?"That is, this class loader has been recorded as an <i>initiating</i> 
loader of the returned classes." -> "That is, all classes for which this 
class loader has been recorded as an <i>initiating</i> loader."

This seems like too much detail to be put here. Basically the term 
"initiating ClassLoader" has turned into a short essay. Is it possible 
that all this detail could be put elsewhere and referenced? Aren't there 
other places in other specs where a similar clarification of "initiating 
ClassLoader" is needed (I see now that ClassLoaderClasses in the JVMTI 
spec, ClassLoaderReference,visibleClasses in the JDI spec, and 
Instrumentation.getInitiatedClasses are all dealing with this, but not 
all in the exact same way).


In the JVMTI spec for GetLoadedClasses:

This suffers in a way similar to ClassLoaderReference.VisibleClasses in 
the JDWP spec, although not as badly. A simple concept ends up with a 
complex description, and it seems that description should really be in a 
more centralized place. I would also suggest a bit of cleanup of these 
lines:

6866???????? An array class is created directly by Java virtual 
machine.? An array class
6867???????? creation can be triggered by using class loaders or by 
invoking methods in certain
6868???????? Java SE Platform API such as reflection.

"Created by [the] Java virtual machine" (add "the")

Change "An array class creation" to "The creation" since your are 
repeating "An array class" from the previous sentence.


In the JVMTI spec ClassLoaderClasses section:

"That is, initiating_loader has been recorded as an initiating loader of 
the returned classes." -> "That is, all classes for which 
initiating_loader has been recorded as an initiating loader."


In the JVMTI spec GetClassSignature section:

"If the class indicated by klass is ..." -> "If the class ..." (you just 
finished the previous sentence with "class indicated by Klass").

"the returned name is [the] JNI type signature" (add "the"). Also, is 
"JNI type signature" formally defined somewhere? This relates to my JDWP 
spec comment above.

" where N is the binary name encoded in internal form indicated by the 
class file". Is "binary name encoded in internal form" explained 
somewhere? Also, can you add an example of a returned hidden class 
signature?


In the JVMTI spec ClassLoad section:

"representation using [a] class loader" (add "a")

"By invoking Lookup::defineHiddenClass, that creates ..."? -> "By 
invoking Lookup::defineHiddenClass to create ..."

"certain Java SE Platform API" -> Should be "APIs"


In JDI ClassLoaderReference.definedClasses()

"loaded at least to the point of preparation and types ..." -> "loaded 
at least to the point of preparation, and types ..." (Note, this not a 
new issue with your edits)


In Instrumentation.getAllLoadedClasses:

The reference to `class` did not format properly.

"by invoking Lookup::defineHiddenClass that creates" -> "by invoking 
Lookup::defineHiddenClass, which creates"

"An array class is created directly by Java virtual machine. An array 
class creation can be triggered ..." ->"An array class is created 
directly by the Java virtual machine. Array class creation can be 
triggered ..."


In Instrumentation.getInitiatedClasses:

"That is, loader has been recorded as an initiating loader of these 
classes." -> "That is, all classes for which loader has been recorded as 
an initiating loader."

thanks,

Chris
>
>
>>
>> Paul.
>>
>>> On Apr 11, 2020, at 8:13 PM, Mandy Chung <mandy.chung at oracle.com> 
>>> wrote:
>>>
>>> Please review the delta patch:
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.06-delta/ 
>>>
>>>
>>> Incremental specdiff:
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff-inc/overview-summary.html 
>>>
>>> This is to follow a discussion [1] on Class::descriptorString and 
>>> MethodType::descriptorString for hidden classes.?? The proposal is 
>>> to define the descriptor string for a hidden class of this form:
>>> ???? "L" + N + ";" + "/" + <suffix>
>>>
>>> The spec of `Lookup::defineHiddenClass`, `Class::descriptorString` 
>>> and `MethodType::descriptorString` are updated to return the 
>>> descriptor of this form for a hidden class.?? To support hidden 
>>> class, `java.lang.invoke.TypeDescriptor` spec is revised such that a 
>>> `TypeDescriptor` object can represent an entity that may not be 
>>> described in nominal form.???? This change affects JVM TI, JDWP and 
>>> JDI.??? The spec change includes a couple other JVM TI and 
>>> java.instrument spec clarification w.r.t. hidden classes that 
>>> Serguei has been working on.
>>>
>>>
>>>
>>> Mandy
>>> [1] 
>>> https://mail.openjdk.java.net/pipermail/valhalla-dev/2020-April/007093.html
>>>
>>> ----------------
>>> As a record, the above patch is applied on the top:
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.06/ 
>>>
>>>
>>> webrev.06 includes the following changes that have been reviewed:
>>> 1. rename "weak hidden" and Klass::is_hidden_weak to 
>>> is_non_strong_hidden
>>> 2. remove unused `setImplMethod` method from lambda proxy class
>>> 3. include Serguei's patch to fix JDK-8242166: regression in JDI 
>>> ClassUnload events
>>> 4. drop the uniqueness guarantee of the suffix of the hidden class's 
>>> name
>>>
>>> On 3/31/20 8:01 PM, Mandy Chung wrote:
>>>> Thanks to the review feedbacks.
>>>>
>>>> Updated webrev:
>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.04/ 
>>>>
>>>> Delta between webrev.03 and webrev.04:
>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03-delta/ 
>>>>
>>>>
>>>> Summary of changes is:
>>>> 1. Update javac to retain the previous behavior when compiling to 
>>>> target release <= 14 where lambda proxy class is not a nestmate
>>>> 2. Rename Class::isHiddenClass to Class::isHidden
>>>> 3. Address Joe's feedback on the CSR to document the behavior of 
>>>> the relevant methods in java.lang.Class for hidden classes
>>>> 4. Add test case for unloadable class with nest host error
>>>> 5. Add test cases for hidden classes with different kinds of class 
>>>> or interface
>>>> 6. Update dcmd to drop "weak hidden class" and refer it as "hidden 
>>>> class"
>>>> 7. Small changes in response to Remi, Coleen, Paul's review comments
>>>> 8. Fix copyright headers
>>>> 9. Fix @modules in tests
>>>>
>>>> Most of the changes above have also been reviewed as separate patches.
>>>>
>>>> Thanks
>>>> Mandy
>>>>
>>>> On 3/26/20 4:57 PM, Mandy Chung wrote:
>>>>> Please review the implementation of JEP 371: Hidden Classes. The 
>>>>> main changes are in core-libs and hotspot runtime area.? Small 
>>>>> changes are made in javac, VM compiler (intrinsification of 
>>>>> Class::isHiddenClass), JFR, JDI, and jcmd.? CSR [1]has been 
>>>>> reviewed and is in the finalized state (see specdiff and javadoc 
>>>>> below for reference).
>>>>>
>>>>> Webrev:
>>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
>>>>>
>>>>>
>>>>> javadoc/specdiff
>>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/ 
>>>>>
>>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
>>>>>
>>>>>
>>>>> JVMS 5.4.4 change:
>>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
>>>>>
>>>>>
>>>>> CSR:
>>>>> https://bugs.openjdk.java.net/browse/JDK-8238359
>>>>>
>>>>> Thanks
>>>>> Mandy
>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
>>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
>>>>> [3] https://bugs.openjdk.java.net/browse/JDK-8230502
>



From mandy.chung at oracle.com  Fri Apr 17 23:52:09 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Fri, 17 Apr 2020 16:52:09 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <6890c88c-809b-fbea-aa10-675661ef2c68@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <1a5eb022-dd7e-b0bb-750f-c43be195eb42@oracle.com>
 <da9f80ff-6d63-aebe-a9fa-1a683eb42bbc@oracle.com>
 <4A360464-1873-43A8-B423-C93D1BEF9895@oracle.com>
 <51adeaaf-9c85-240c-ed0d-f460c117ba46@oracle.com>
 <6890c88c-809b-fbea-aa10-675661ef2c68@oracle.com>
Message-ID: <440544f0-8a74-7ff1-8b78-0c20307bb89a@oracle.com>



On 4/17/20 3:51 PM, Chris Plummer wrote:
> Hi Mandy,
>
> Thanks for updating the svc specs. Some comments below:
>
>
> In the JDWP spec update, you changed "JNI signature" to "type 
> signature" in one place, but left it as "JNI signature" everywhere 
> else. Should they all be changed?
>

JDWP signature is changed because there is no JNI signature representing 
a hidden class.??? I leave other references to JNI signature as is since 
those only apply for normal classes as I wanted to keep this change 
specifically due to hidden classes.? I think it's good to file a JBS 
issue to follow up on JDWP and JDI to determine if the spec should be 
upgraded to reference the new TypeDescriptor API.

>
> In the JDWP spec for ClassLoaderReference.VisibleClasses:
>
> ?"That is, this class loader has been recorded as an <i>initiating</i> 
> loader of the returned classes." -> "That is, all classes for which 
> this class loader has been recorded as an <i>initiating</i> loader."
>
> This seems like too much detail to be put here. Basically the term 
> "initiating ClassLoader" has turned into a short essay. Is it possible 
> that all this detail could be put elsewhere and referenced? 

Any suggestion??? We attempted to place those description in JVM TI 
Class section or ClassLoad event.?? However, that's not ideal place 
since that's needed by JDWP, JDI and Instrumentation.?? I found inlining 
this description is not ideal but it provides adequate clarification.

> Aren't there other places in other specs where a similar clarification 
> of "initiating ClassLoader" is needed (I see now that 
> ClassLoaderClasses in the JVMTI spec, 
> ClassLoaderReference,visibleClasses in the JDI spec, and 
> Instrumentation.getInitiatedClasses are all dealing with this, but not 
> all in the exact same way).
>

I took the conservative side and make sure the clarification is in place 
for all APIs.? I'm open to any suggestion for example having JDWP and 
JDI to link to JVM TI spec if you think appropriate.

>
> In the JVMTI spec for GetLoadedClasses:
>
> This suffers in a way similar to ClassLoaderReference.VisibleClasses 
> in the JDWP spec, although not as badly. A simple concept ends up with 
> a complex description, and it seems that description should really be 
> in a more centralized place.? I would also suggest a bit of cleanup of 
> these lines:
>
> 6866???????? An array class is created directly by Java virtual 
> machine.? An array class
> 6867???????? creation can be triggered by using class loaders or by 
> invoking methods in certain
> 6868???????? Java SE Platform API such as reflection.
>
> "Created by [the] Java virtual machine" (add "the")
> Change "An array class creation" to "The creation" since your are 
> repeating "An array class" from the previous sentence.
>
>
> In the JVMTI spec ClassLoaderClasses section:
>
> "That is, initiating_loader has been recorded as an initiating loader 
> of the returned classes." -> "That is, all classes for which 
> initiating_loader has been recorded as an initiating loader."
>
>
> In the JVMTI spec GetClassSignature section:
>
> "If the class indicated by klass is ..." -> "If the class ..." (you 
> just finished the previous sentence with "class indicated by Klass").
>
> "the returned name is [the] JNI type signature" (add "the"). Also, is 
> "JNI type signature" formally defined somewhere? This relates to my 
> JDWP spec comment above.
>

It's a link to 
https://download.java.net/java/early_access/jdk15/docs/specs/jni/types.html#type-signatures. 
This is how the current JVM TI spec defines.

> " where N is the binary name encoded in internal form indicated by the 
> class file". Is "binary name encoded in internal form" explained 
> somewhere? 

JVMS 4.2.1

https://docs.oracle.com/javase/specs/jvms/se14/html/jvms-4.html#jvms-4.2.1

> Also, can you add an example of a returned hidden class signature?
>
OK

>
> In the JVMTI spec ClassLoad section:
>
> "representation using [a] class loader" (add "a")
>
> "By invoking Lookup::defineHiddenClass, that creates ..."? -> "By 
> invoking Lookup::defineHiddenClass to create ..."
>
> "certain Java SE Platform API" -> Should be "APIs"
>
>
> In JDI ClassLoaderReference.definedClasses()
>
> "loaded at least to the point of preparation and types ..." -> "loaded 
> at least to the point of preparation, and types ..." (Note, this not a 
> new issue with your edits)
>
>
> In Instrumentation.getAllLoadedClasses:
>
> The reference to `class` did not format properly.
>

Serguei caught that one too.? I fixed it in my local repo.

> "by invoking Lookup::defineHiddenClass that creates" -> "by invoking 
> Lookup::defineHiddenClass, which creates"
>
> "An array class is created directly by Java virtual machine. An array 
> class creation can be triggered ..." ->"An array class is created 
> directly by the Java virtual machine. Array class creation can be 
> triggered ..."
>
>
> In Instrumentation.getInitiatedClasses:
>
> "That is, loader has been recorded as an initiating loader of these 
> classes." -> "That is, all classes for which loader has been recorded 
> as an initiating loader."
>
> thanks,
>
> Chris

Thanks for the review.?? See this patch of the editorial fixes.

diff --git a/make/data/jdwp/jdwp.spec b/make/data/jdwp/jdwp.spec
--- a/make/data/jdwp/jdwp.spec
+++ b/make/data/jdwp/jdwp.spec
@@ -2265,8 +2265,8 @@
 ???????? "Returns a list of all classes which this class loader can find "
 ???????? "by name via <code>ClassLoader::loadClass</code>, "
 ???????? "<code>Class::forName</code> and bytecode linkage. That is, "
-??????? "this class loader has been recorded as an <i>initiating</i> "
-??????? "loader of the returned classes. The list contains each
+??????? "all classes for which this class loader has been recorded as an "
+??????? "<i>initiating</i> loader. The list contains each "
 ???????? "reference type created by this loader and any types for which "
 ???????? "loading was delegated by this class loader to another class 
loader. "
 ???????? "The list does not include hidden classes or interfaces 
created by "
diff --git a/src/hotspot/share/prims/jvmti.xml 
b/src/hotspot/share/prims/jvmti.xml
--- a/src/hotspot/share/prims/jvmti.xml
+++ b/src/hotspot/share/prims/jvmti.xml
@@ -6857,15 +6857,15 @@
 ???????? A class or interface creation can be triggered by one of the 
following:
 ???????? <ul>
 ???????? <li>By loading and deriving a class from a <code>class</code> 
file representation
-??????????? using class loader (see <vmspec chapter="5.3"/>).</li>
+??????????? using a class loader (see <vmspec chapter="5.3"/>).</li>
 ???????? <li>By invoking <externallink 
id="../api/java.base/java/lang/invoke/MethodHandles.Lookup.html#defineHiddenClass(byte[],boolean,java.lang.invoke.MethodHandles.Lookup.ClassOption...)">Lookup::defineHiddenClass</externallink>
 ???????????? that creates a hidden class or interface from a 
<code>class</code> file representation.</li>
-??????? <li>By invoking methods in certain Java SE Platform API such as 
reflection.</li>
+??????? <li>By invoking methods in certain Java SE Platform APIs such 
as reflection.</li>
 ????????? </ul>
 ???????? <p/>
-??????? An array class is created directly by Java virtual machine.? An 
array class
-??????? creation can be triggered by using class loaders or by invoking 
methods in certain
-??????? Java SE Platform API such as reflection.
+??????? An array class is created directly by the Java virtual 
machine.? The creation
+??????? can be triggered by using class loaders or by invoking methods 
in certain
+??????? Java SE Platform APIs such as reflection.
 ???????? <p/>
 ???????? The returned list includes all classes and interfaces, including
 ???????? <externallink 
id="../api/java.base/java/lang/Class.html#isHidden()">
@@ -6904,8 +6904,8 @@
 ???????? can find by name via
 ???????? <externallink 
id="../api/java.base/java/lang/ClassLoader.html#loadClass(java.lang.String,boolean)">ClassLoader::loadClass</externallink>,
 ???????? <externallink 
id="../api/java.base/java/lang/Class.html#forName(java.lang.String,boolean,java.lang.ClassLoader)">Class::forName</externallink> 
and bytecode linkage.
-??????? That is, <code>initiating_loader</code>
-??????? has been recorded as an initiating loader of the returned classes.
+??????? That is, all classes for which <code>initiating_loader</code>
+??????? has been recorded as an initiating loader.
 ???????? Each class in the returned array was created by this class loader,
 ???????? either by defining it directly or by delegation to another 
class loader.
 ???????? See <vmspec chapter="5.3"/>.
@@ -6964,21 +6964,22 @@
 ?????? <description>
 ???????? Return the name and the generic signature of the class 
indicated by <code>klass</code>.
 ???????? <p/>
-??????? If the class indicated by <code>klass</code> is a class or 
interface, then:
+??????? If the class is a class or interface, then:
 ???????? <ul>
 ???????? <li>If the class or interface is not <externallink 
id="../api/java.base/java/lang/Class.html#isHidden()">hi
dden</externallink>,
-????????? then the returned name is <externallink 
id="jni/types.html#type-signatures">
+????????? then the returned name is the <externallink 
id="jni/types.html#type-signatures">
 ?????????? JNI type signature</externallink>.
 ?????????? For example, java.util.List is "Ljava/util/List;"
 ???????? </li>
 ???????? <li>If the class or interface is <externallink 
id="../api/java.base/java/lang/Class.html#isHidden()">hidden
</externallink>,
 ?????????? then the returned name is a string of the form:
 ?????????? <code>"L" + N + "." +? S + ";"</code>
-????????? where <code>N</code> is the binary name encoded in internal form
+????????? where <code>N</code> is the binary name encoded in internal 
form (JVMS 4.2.1)
 ?????????? indicated by the <code>class</code> file passed to
 ?????????? <externallink 
id="../api/java.base/java/lang/invoke/MethodHandles.Lookup.html#defineHiddenClass(byte[],boolean,java.lang.invoke.MethodHandles.Lookup.ClassOption...)">Lookup::defineHiddenClass</externallink>,
 ?????????? and <code>S</code> is an unqualified name.
-????????? The returned name is not a valid type descriptor.
+????????? The returned name is not a type descriptor and does not 
conform to JVMS 4.3.2.
+????????? For example, com.foo.Foo/AnySuffix is "Lcom/foo/Foo.AnySuffix;"
 ???????? </li>
 ???????? </ul>
 ???????? <p/>
@@ -12768,10 +12769,10 @@
 ?????? A class or interface is created by one of the following:
 ?????? <ul>
 ?????? <li>By loading and deriving a class from a <code>class</code> 
file representation
-????????? using class loader.</li>
+????????? using a class loader.</li>
 ?????? <li>By invoking <externallink 
id="../api/java.base/java/lang/invoke/MethodHandles.Lookup.html#defineHiddenClass(byte[],boolean,java.lang.invoke.MethodHandles.Lookup.ClassOption...)">Lookup::defineHiddenClass</externallink>,
 ?????????? that creates a hidden class or interface from a 
<code>class</code> file representation.</li>
-????? <li>By invoking methods in certain Java SE Platform API such as 
reflection.</li>
+????? <li>By invoking methods in certain Java SE Platform APIs such as 
reflection.</li>
 ?????? </ul>
 ?????? <p/>
 ?????? Array class creation does not generate a class load event.
diff --git 
a/src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java 
b/src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java
--- 
a/src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java
+++ 
b/src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java
@@ -392,15 +392,15 @@
 ????? * A class or interface creation can be triggered by one of the 
following:
 ????? * <ul>
 ????? * <li>by loading and deriving a class from a {@code class} file 
representation
-???? *???? using class loader (see JVMS {@jvms 5.3}).
+???? *???? using a class loader (see JVMS {@jvms 5.3}).
 ????? * <li>by invoking {@link 
java.lang.invoke.MethodHandles.Lookup#defineHiddenClass(byte[], boolean, 
java.lang.invoke.MethodHandles.Lookup.ClassOption...)
-???? *???? Lookup::defineHiddenClass} that creates a {@link Class#isHidden
+???? *???? Lookup::defineHiddenClass}, which creates a {@link 
Class#isHidden
 ????? *???? hidden class or interface} from a {@code class} file 
representation.
 ????? * <li>by invoking methods in certain reflection APIs such as
 ????? *???? {@link Class#forName(String) Class::forName}.
 ????? * </ul>
 ????? * <p>
-???? * An array class is created directly by Java virtual machine. An array
+???? * An array class is created directly by the Java virtual machine.? 
Array
 ????? * class creation can be triggered by using class loaders or by 
invoking
 ????? * methods in certain reflection APIs such as
 ????? * {@link java.lang.reflect.Array#newInstance(Class, int) 
Array::newInstance}
@@ -420,8 +420,8 @@
 ????? * Returns an array of all classes which {@code loader} can find 
by name
 ????? * via {@link ClassLoader#loadClass(String, boolean) 
ClassLoader::loadClass},
 ????? * {@link Class#forName(String) Class::forName} and bytecode linkage.
-???? * That is, {@code loader} has been recorded as an initiating loader
-???? * of these classes. If the supplied {@code loader} is {@code null},
+???? * That is, all classes for which {@code loader} has been recorded as
+???? * an initiating loader. If the supplied {@code loader} is {@code 
null},
 ????? * classes that the bootstrap class loader can find by name are 
returned.
 ????? *
 ????? * <p>
diff --git 
a/src/jdk.jdi/share/classes/com/sun/jdi/ClassLoaderReference.java 
b/src/jdk.jdi/share/classes/com/sun/jdi/ClassLoaderReference.java
--- a/src/jdk.jdi/share/classes/com/sun/jdi/ClassLoaderReference.java
+++ b/src/jdk.jdi/share/classes/com/sun/jdi/ClassLoaderReference.java
@@ -46,7 +46,7 @@
 ????? * No ordering of this list guaranteed.
 ????? * The returned list will include all reference types, including
 ????? * {@link Class#isHidden hidden classes or interfaces}, loaded
-???? * at least to the point of preparation and types (like array)
+???? * at least to the point of preparation, and types (like array)
 ????? * for which preparation is not defined.
 ????? *
 ????? * @return a {@code List} of {@link ReferenceType} objects 
mirroring types
@@ -59,8 +59,8 @@
 ????? * Returns a list of all classes which this class loader
 ????? * can find by name via {@link ClassLoader#loadClass(String, boolean)
 ????? * ClassLoader::loadClass}, {@link Class#forName(String) 
Class::forName}
-???? * and bytecode linkage in the target VM.? That is, this class loader
-???? * has been recorded as an initiating loader of these classes.
+???? * and bytecode linkage in the target VM.? That is, all classes for
+???? * which this class loader has been recorded as an initiating loader.
 ????? * <p>
 ????? * Each class in the returned list was created by this class loader
 ????? * either by defining it directly or by delegation to another 
class loader
diff --git a/src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java 
b/src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java
--- a/src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java
+++ b/src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java
@@ -138,9 +138,9 @@
 ????? * A class or interface creation can be triggered by one of the 
following:
 ????? * <ul>
 ????? * <li>by loading and deriving a class from a {@code class} file 
representation
-???? *???? using class loader (see JVMS {@jvms 5.3}).
+???? *???? using a class loader (see JVMS {@jvms 5.3}).
 ????? * <li>by invoking {@link 
java.lang.invoke.MethodHandles.Lookup#defineHiddenClass(byte[], boolean, 
java.lang.invoke.MethodHandles.Lookup.ClassOption...)
-???? *???? Lookup::defineHiddenClass} that creates a {@link Class#isHidden
+???? *???? Lookup::defineHiddenClass}, which creates a {@link 
Class#isHidden
 ????? *???? hidden class or interface} from a {@code class} file 
representation.
 ????? * <li>by invoking methods in certain reflection APIs such as
 ????? *???? {@link Class#forName(String) Class::forName}.

Mandy



From serguei.spitsyn at oracle.com  Sat Apr 18 04:37:21 2020
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Fri, 17 Apr 2020 21:37:21 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <440544f0-8a74-7ff1-8b78-0c20307bb89a@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <1a5eb022-dd7e-b0bb-750f-c43be195eb42@oracle.com>
 <da9f80ff-6d63-aebe-a9fa-1a683eb42bbc@oracle.com>
 <4A360464-1873-43A8-B423-C93D1BEF9895@oracle.com>
 <51adeaaf-9c85-240c-ed0d-f460c117ba46@oracle.com>
 <6890c88c-809b-fbea-aa10-675661ef2c68@oracle.com>
 <440544f0-8a74-7ff1-8b78-0c20307bb89a@oracle.com>
Message-ID: <cad852a0-0f4a-ed5b-870a-a2dfa7aa3494@oracle.com>

On 4/17/20 16:52, Mandy Chung wrote:
>
>
> On 4/17/20 3:51 PM, Chris Plummer wrote:
>> Hi Mandy,
>>
>> Thanks for updating the svc specs. Some comments below:
>>
>>
>> In the JDWP spec update, you changed "JNI signature" to "type 
>> signature" in one place, but left it as "JNI signature" everywhere 
>> else. Should they all be changed?
>>
>
> JDWP signature is changed because there is no JNI signature 
> representing a hidden class.??? I leave other references to JNI 
> signature as is since those only apply for normal classes as I wanted 
> to keep this change specifically due to hidden classes.? I think it's 
> good to file a JBS issue to follow up on JDWP and JDI to determine if 
> the spec should be upgraded to reference the new TypeDescriptor API.
>
>>
>> In the JDWP spec for ClassLoaderReference.VisibleClasses:
>>
>> ?"That is, this class loader has been recorded as an 
>> <i>initiating</i> loader of the returned classes." -> "That is, all 
>> classes for which this class loader has been recorded as an 
>> <i>initiating</i> loader."
>>
>> This seems like too much detail to be put here. Basically the term 
>> "initiating ClassLoader" has turned into a short essay. Is it 
>> possible that all this detail could be put elsewhere and referenced? 
>
> Any suggestion??? We attempted to place those description in JVM TI 
> Class section or ClassLoad event.?? However, that's not ideal place 
> since that's needed by JDWP, JDI and Instrumentation.?? I found 
> inlining this description is not ideal but it provides adequate 
> clarification.

The JDI (transitively via JDWP), JDWP and Instrumentation 
implementations are based on the JVMTI.
I've tried to suggest once to link these API's to the JVMTI.
The problem is there was no such practice in the specs of these API's 
before but we can make a step to introduce it now.
Placing this description either in JVM TI Class section or ClassLoad 
event would be good enough.

An alternative approach is to make JVMTI/JDI/JDWP/Instrument to refer to 
the java.lang.Class spec for
general information about class loading and classes defined and 
initiated by class loaders.

Thanks,
Serguei

>> Aren't there other places in other specs where a similar 
>> clarification of "initiating ClassLoader" is needed (I see now that 
>> ClassLoaderClasses in the JVMTI spec, 
>> ClassLoaderReference,visibleClasses in the JDI spec, and 
>> Instrumentation.getInitiatedClasses are all dealing with this, but 
>> not all in the exact same way).
>>
>
> I took the conservative side and make sure the clarification is in 
> place for all APIs.? I'm open to any suggestion for example having 
> JDWP and JDI to link to JVM TI spec if you think appropriate.
>
>>
>> In the JVMTI spec for GetLoadedClasses:
>>
>> This suffers in a way similar to ClassLoaderReference.VisibleClasses 
>> in the JDWP spec, although not as badly. A simple concept ends up 
>> with a complex description, and it seems that description should 
>> really be in a more centralized place.? I would also suggest a bit of 
>> cleanup of these lines:
>>
>> 6866???????? An array class is created directly by Java virtual 
>> machine.? An array class
>> 6867???????? creation can be triggered by using class loaders or by 
>> invoking methods in certain
>> 6868???????? Java SE Platform API such as reflection.
>>
>> "Created by [the] Java virtual machine" (add "the")
>> Change "An array class creation" to "The creation" since your are 
>> repeating "An array class" from the previous sentence.
>>
>>
>> In the JVMTI spec ClassLoaderClasses section:
>>
>> "That is, initiating_loader has been recorded as an initiating loader 
>> of the returned classes." -> "That is, all classes for which 
>> initiating_loader has been recorded as an initiating loader."
>>
>>
>> In the JVMTI spec GetClassSignature section:
>>
>> "If the class indicated by klass is ..." -> "If the class ..." (you 
>> just finished the previous sentence with "class indicated by Klass").
>>
>> "the returned name is [the] JNI type signature" (add "the"). Also, is 
>> "JNI type signature" formally defined somewhere? This relates to my 
>> JDWP spec comment above.
>>
>
> It's a link to 
> https://download.java.net/java/early_access/jdk15/docs/specs/jni/types.html#type-signatures. 
> This is how the current JVM TI spec defines.
>
>> " where N is the binary name encoded in internal form indicated by 
>> the class file". Is "binary name encoded in internal form" explained 
>> somewhere? 
>
> JVMS 4.2.1
>
> https://docs.oracle.com/javase/specs/jvms/se14/html/jvms-4.html#jvms-4.2.1 
>
>
>> Also, can you add an example of a returned hidden class signature?
>>
> OK
>
>>
>> In the JVMTI spec ClassLoad section:
>>
>> "representation using [a] class loader" (add "a")
>>
>> "By invoking Lookup::defineHiddenClass, that creates ..."? -> "By 
>> invoking Lookup::defineHiddenClass to create ..."
>>
>> "certain Java SE Platform API" -> Should be "APIs"
>>
>>
>> In JDI ClassLoaderReference.definedClasses()
>>
>> "loaded at least to the point of preparation and types ..." -> 
>> "loaded at least to the point of preparation, and types ..." (Note, 
>> this not a new issue with your edits)
>>
>>
>> In Instrumentation.getAllLoadedClasses:
>>
>> The reference to `class` did not format properly.
>>
>
> Serguei caught that one too.? I fixed it in my local repo.
>
>> "by invoking Lookup::defineHiddenClass that creates" -> "by invoking 
>> Lookup::defineHiddenClass, which creates"
>>
>> "An array class is created directly by Java virtual machine. An array 
>> class creation can be triggered ..." ->"An array class is created 
>> directly by the Java virtual machine. Array class creation can be 
>> triggered ..."
>>
>>
>> In Instrumentation.getInitiatedClasses:
>>
>> "That is, loader has been recorded as an initiating loader of these 
>> classes." -> "That is, all classes for which loader has been recorded 
>> as an initiating loader."
>>
>> thanks,
>>
>> Chris
>
> Thanks for the review.?? See this patch of the editorial fixes.
>
> diff --git a/make/data/jdwp/jdwp.spec b/make/data/jdwp/jdwp.spec
> --- a/make/data/jdwp/jdwp.spec
> +++ b/make/data/jdwp/jdwp.spec
> @@ -2265,8 +2265,8 @@
> ???????? "Returns a list of all classes which this class loader can 
> find "
> ???????? "by name via <code>ClassLoader::loadClass</code>, "
> ???????? "<code>Class::forName</code> and bytecode linkage. That is, "
> -??????? "this class loader has been recorded as an <i>initiating</i> "
> -??????? "loader of the returned classes. The list contains each
> +??????? "all classes for which this class loader has been recorded as 
> an "
> +??????? "<i>initiating</i> loader. The list contains each "
> ???????? "reference type created by this loader and any types for which "
> ???????? "loading was delegated by this class loader to another class 
> loader. "
> ???????? "The list does not include hidden classes or interfaces 
> created by "
> diff --git a/src/hotspot/share/prims/jvmti.xml 
> b/src/hotspot/share/prims/jvmti.xml
> --- a/src/hotspot/share/prims/jvmti.xml
> +++ b/src/hotspot/share/prims/jvmti.xml
> @@ -6857,15 +6857,15 @@
> ???????? A class or interface creation can be triggered by one of the 
> following:
> ???????? <ul>
> ???????? <li>By loading and deriving a class from a <code>class</code> 
> file representation
> -??????????? using class loader (see <vmspec chapter="5.3"/>).</li>
> +??????????? using a class loader (see <vmspec chapter="5.3"/>).</li>
> ???????? <li>By invoking <externallink 
> id="../api/java.base/java/lang/invoke/MethodHandles.Lookup.html#defineHiddenClass(byte[],boolean,java.lang.invoke.MethodHandles.Lookup.ClassOption...)">Lookup::defineHiddenClass</externallink>
> ???????????? that creates a hidden class or interface from a 
> <code>class</code> file representation.</li>
> -??????? <li>By invoking methods in certain Java SE Platform API such 
> as reflection.</li>
> +??????? <li>By invoking methods in certain Java SE Platform APIs such 
> as reflection.</li>
> ????????? </ul>
> ???????? <p/>
> -??????? An array class is created directly by Java virtual machine.? 
> An array class
> -??????? creation can be triggered by using class loaders or by 
> invoking methods in certain
> -??????? Java SE Platform API such as reflection.
> +??????? An array class is created directly by the Java virtual 
> machine.? The creation
> +??????? can be triggered by using class loaders or by invoking 
> methods in certain
> +??????? Java SE Platform APIs such as reflection.
> ???????? <p/>
> ???????? The returned list includes all classes and interfaces, including
> ???????? <externallink 
> id="../api/java.base/java/lang/Class.html#isHidden()">
> @@ -6904,8 +6904,8 @@
> ???????? can find by name via
> ???????? <externallink 
> id="../api/java.base/java/lang/ClassLoader.html#loadClass(java.lang.String,boolean)">ClassLoader::loadClass</externallink>,
> ???????? <externallink 
> id="../api/java.base/java/lang/Class.html#forName(java.lang.String,boolean,java.lang.ClassLoader)">Class::forName</externallink> 
> and bytecode linkage.
> -??????? That is, <code>initiating_loader</code>
> -??????? has been recorded as an initiating loader of the returned 
> classes.
> +??????? That is, all classes for which <code>initiating_loader</code>
> +??????? has been recorded as an initiating loader.
> ???????? Each class in the returned array was created by this class 
> loader,
> ???????? either by defining it directly or by delegation to another 
> class loader.
> ???????? See <vmspec chapter="5.3"/>.
> @@ -6964,21 +6964,22 @@
> ?????? <description>
> ???????? Return the name and the generic signature of the class 
> indicated by <code>klass</code>.
> ???????? <p/>
> -??????? If the class indicated by <code>klass</code> is a class or 
> interface, then:
> +??????? If the class is a class or interface, then:
> ???????? <ul>
> ???????? <li>If the class or interface is not <externallink 
> id="../api/java.base/java/lang/Class.html#isHidden()">hi
> dden</externallink>,
> -????????? then the returned name is <externallink 
> id="jni/types.html#type-signatures">
> +????????? then the returned name is the <externallink 
> id="jni/types.html#type-signatures">
> ?????????? JNI type signature</externallink>.
> ?????????? For example, java.util.List is "Ljava/util/List;"
> ???????? </li>
> ???????? <li>If the class or interface is <externallink 
> id="../api/java.base/java/lang/Class.html#isHidden()">hidden
> </externallink>,
> ?????????? then the returned name is a string of the form:
> ?????????? <code>"L" + N + "." +? S + ";"</code>
> -????????? where <code>N</code> is the binary name encoded in internal 
> form
> +????????? where <code>N</code> is the binary name encoded in internal 
> form (JVMS 4.2.1)
> ?????????? indicated by the <code>class</code> file passed to
> ?????????? <externallink 
> id="../api/java.base/java/lang/invoke/MethodHandles.Lookup.html#defineHiddenClass(byte[],boolean,java.lang.invoke.MethodHandles.Lookup.ClassOption...)">Lookup::defineHiddenClass</externallink>,
> ?????????? and <code>S</code> is an unqualified name.
> -????????? The returned name is not a valid type descriptor.
> +????????? The returned name is not a type descriptor and does not 
> conform to JVMS 4.3.2.
> +????????? For example, com.foo.Foo/AnySuffix is 
> "Lcom/foo/Foo.AnySuffix;"
> ???????? </li>
> ???????? </ul>
> ???????? <p/>
> @@ -12768,10 +12769,10 @@
> ?????? A class or interface is created by one of the following:
> ?????? <ul>
> ?????? <li>By loading and deriving a class from a <code>class</code> 
> file representation
> -????????? using class loader.</li>
> +????????? using a class loader.</li>
> ?????? <li>By invoking <externallink 
> id="../api/java.base/java/lang/invoke/MethodHandles.Lookup.html#defineHiddenClass(byte[],boolean,java.lang.invoke.MethodHandles.Lookup.ClassOption...)">Lookup::defineHiddenClass</externallink>,
> ?????????? that creates a hidden class or interface from a 
> <code>class</code> file representation.</li>
> -????? <li>By invoking methods in certain Java SE Platform API such as 
> reflection.</li>
> +????? <li>By invoking methods in certain Java SE Platform APIs such 
> as reflection.</li>
> ?????? </ul>
> ?????? <p/>
> ?????? Array class creation does not generate a class load event.
> diff --git 
> a/src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java 
> b/src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java 
>
> --- 
> a/src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java
> +++ 
> b/src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java
> @@ -392,15 +392,15 @@
> ????? * A class or interface creation can be triggered by one of the 
> following:
> ????? * <ul>
> ????? * <li>by loading and deriving a class from a {@code class} file 
> representation
> -???? *???? using class loader (see JVMS {@jvms 5.3}).
> +???? *???? using a class loader (see JVMS {@jvms 5.3}).
> ????? * <li>by invoking {@link 
> java.lang.invoke.MethodHandles.Lookup#defineHiddenClass(byte[], 
> boolean, java.lang.invoke.MethodHandles.Lookup.ClassOption...)
> -???? *???? Lookup::defineHiddenClass} that creates a {@link 
> Class#isHidden
> +???? *???? Lookup::defineHiddenClass}, which creates a {@link 
> Class#isHidden
> ????? *???? hidden class or interface} from a {@code class} file 
> representation.
> ????? * <li>by invoking methods in certain reflection APIs such as
> ????? *???? {@link Class#forName(String) Class::forName}.
> ????? * </ul>
> ????? * <p>
> -???? * An array class is created directly by Java virtual machine. An 
> array
> +???? * An array class is created directly by the Java virtual 
> machine.? Array
> ????? * class creation can be triggered by using class loaders or by 
> invoking
> ????? * methods in certain reflection APIs such as
> ????? * {@link java.lang.reflect.Array#newInstance(Class, int) 
> Array::newInstance}
> @@ -420,8 +420,8 @@
> ????? * Returns an array of all classes which {@code loader} can find 
> by name
> ????? * via {@link ClassLoader#loadClass(String, boolean) 
> ClassLoader::loadClass},
> ????? * {@link Class#forName(String) Class::forName} and bytecode 
> linkage.
> -???? * That is, {@code loader} has been recorded as an initiating loader
> -???? * of these classes. If the supplied {@code loader} is {@code null},
> +???? * That is, all classes for which {@code loader} has been 
> recorded as
> +???? * an initiating loader. If the supplied {@code loader} is {@code 
> null},
> ????? * classes that the bootstrap class loader can find by name are 
> returned.
> ????? *
> ????? * <p>
> diff --git 
> a/src/jdk.jdi/share/classes/com/sun/jdi/ClassLoaderReference.java 
> b/src/jdk.jdi/share/classes/com/sun/jdi/ClassLoaderReference.java
> --- a/src/jdk.jdi/share/classes/com/sun/jdi/ClassLoaderReference.java
> +++ b/src/jdk.jdi/share/classes/com/sun/jdi/ClassLoaderReference.java
> @@ -46,7 +46,7 @@
> ????? * No ordering of this list guaranteed.
> ????? * The returned list will include all reference types, including
> ????? * {@link Class#isHidden hidden classes or interfaces}, loaded
> -???? * at least to the point of preparation and types (like array)
> +???? * at least to the point of preparation, and types (like array)
> ????? * for which preparation is not defined.
> ????? *
> ????? * @return a {@code List} of {@link ReferenceType} objects 
> mirroring types
> @@ -59,8 +59,8 @@
> ????? * Returns a list of all classes which this class loader
> ????? * can find by name via {@link ClassLoader#loadClass(String, 
> boolean)
> ????? * ClassLoader::loadClass}, {@link Class#forName(String) 
> Class::forName}
> -???? * and bytecode linkage in the target VM.? That is, this class 
> loader
> -???? * has been recorded as an initiating loader of these classes.
> +???? * and bytecode linkage in the target VM.? That is, all classes for
> +???? * which this class loader has been recorded as an initiating 
> loader.
> ????? * <p>
> ????? * Each class in the returned list was created by this class loader
> ????? * either by defining it directly or by delegation to another 
> class loader
> diff --git a/src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java 
> b/src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java
> --- a/src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java
> +++ b/src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java
> @@ -138,9 +138,9 @@
> ????? * A class or interface creation can be triggered by one of the 
> following:
> ????? * <ul>
> ????? * <li>by loading and deriving a class from a {@code class} file 
> representation
> -???? *???? using class loader (see JVMS {@jvms 5.3}).
> +???? *???? using a class loader (see JVMS {@jvms 5.3}).
> ????? * <li>by invoking {@link 
> java.lang.invoke.MethodHandles.Lookup#defineHiddenClass(byte[], 
> boolean, java.lang.invoke.MethodHandles.Lookup.ClassOption...)
> -???? *???? Lookup::defineHiddenClass} that creates a {@link 
> Class#isHidden
> +???? *???? Lookup::defineHiddenClass}, which creates a {@link 
> Class#isHidden
> ????? *???? hidden class or interface} from a {@code class} file 
> representation.
> ????? * <li>by invoking methods in certain reflection APIs such as
> ????? *???? {@link Class#forName(String) Class::forName}.
>
> Mandy
>
>


From chris.plummer at oracle.com  Sat Apr 18 07:18:59 2020
From: chris.plummer at oracle.com (Chris Plummer)
Date: Sat, 18 Apr 2020 00:18:59 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <cad852a0-0f4a-ed5b-870a-a2dfa7aa3494@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <1a5eb022-dd7e-b0bb-750f-c43be195eb42@oracle.com>
 <da9f80ff-6d63-aebe-a9fa-1a683eb42bbc@oracle.com>
 <4A360464-1873-43A8-B423-C93D1BEF9895@oracle.com>
 <51adeaaf-9c85-240c-ed0d-f460c117ba46@oracle.com>
 <6890c88c-809b-fbea-aa10-675661ef2c68@oracle.com>
 <440544f0-8a74-7ff1-8b78-0c20307bb89a@oracle.com>
 <cad852a0-0f4a-ed5b-870a-a2dfa7aa3494@oracle.com>
Message-ID: <9b74fc5f-0cb0-e0f5-6a36-969db3deff0a@oracle.com>

On 4/17/20 9:37 PM, serguei.spitsyn at oracle.com wrote:
> On 4/17/20 16:52, Mandy Chung wrote:
>>
>>
>> On 4/17/20 3:51 PM, Chris Plummer wrote:
>>> Hi Mandy,
>>>
>>> Thanks for updating the svc specs. Some comments below:
>>>
>>>
>>> In the JDWP spec update, you changed "JNI signature" to "type 
>>> signature" in one place, but left it as "JNI signature" everywhere 
>>> else. Should they all be changed?
>>>
>>
>> JDWP signature is changed because there is no JNI signature 
>> representing a hidden class.??? I leave other references to JNI 
>> signature as is since those only apply for normal classes as I wanted 
>> to keep this change specifically due to hidden classes. I think it's 
>> good to file a JBS issue to follow up on JDWP and JDI to determine if 
>> the spec should be upgraded to reference the new TypeDescriptor API.
>>
>>>
>>> In the JDWP spec for ClassLoaderReference.VisibleClasses:
>>>
>>> ?"That is, this class loader has been recorded as an 
>>> <i>initiating</i> loader of the returned classes." -> "That is, all 
>>> classes for which this class loader has been recorded as an 
>>> <i>initiating</i> loader."
>>>
>>> This seems like too much detail to be put here. Basically the term 
>>> "initiating ClassLoader" has turned into a short essay. Is it 
>>> possible that all this detail could be put elsewhere and referenced? 
>>
>> Any suggestion??? We attempted to place those description in JVM TI 
>> Class section or ClassLoad event.?? However, that's not ideal place 
>> since that's needed by JDWP, JDI and Instrumentation.?? I found 
>> inlining this description is not ideal but it provides adequate 
>> clarification.
>
> The JDI (transitively via JDWP), JDWP and Instrumentation 
> implementations are based on the JVMTI.
> I've tried to suggest once to link these API's to the JVMTI.
> The problem is there was no such practice in the specs of these API's 
> before but we can make a step to introduce it now.
> Placing this description either in JVM TI Class section or ClassLoad 
> event would be good enough.
>
> An alternative approach is to make JVMTI/JDI/JDWP/Instrument to refer 
> to the java.lang.Class spec for
> general information about class loading and classes defined and 
> initiated by class loaders.
I'd first like to clarify on my comment above just in case there was any 
confusion. At the last minute I re-ordered the two paragraph and now I 
see it makes it seem like I was only referring to to the one sentence 
about initiating loaders. I was referring to pretty much all the changes 
that were made in this section and other similar APIs in the other specs.

Yes. I'd like to see all this as part of the Class/Classloading spec in 
some sort of section that gives an overview of all these topics, but 
mostly clarifies what an initiating loader is, and the (non) 
relationship to hidden classes.

thanks,

Chris
>
> Thanks,
> Serguei
>
>>> Aren't there other places in other specs where a similar 
>>> clarification of "initiating ClassLoader" is needed (I see now that 
>>> ClassLoaderClasses in the JVMTI spec, 
>>> ClassLoaderReference,visibleClasses in the JDI spec, and 
>>> Instrumentation.getInitiatedClasses are all dealing with this, but 
>>> not all in the exact same way).
>>>
>>
>> I took the conservative side and make sure the clarification is in 
>> place for all APIs.? I'm open to any suggestion for example having 
>> JDWP and JDI to link to JVM TI spec if you think appropriate.
>>
>>>
>>> In the JVMTI spec for GetLoadedClasses:
>>>
>>> This suffers in a way similar to ClassLoaderReference.VisibleClasses 
>>> in the JDWP spec, although not as badly. A simple concept ends up 
>>> with a complex description, and it seems that description should 
>>> really be in a more centralized place.? I would also suggest a bit 
>>> of cleanup of these lines:
>>>
>>> 6866???????? An array class is created directly by Java virtual 
>>> machine.? An array class
>>> 6867???????? creation can be triggered by using class loaders or by 
>>> invoking methods in certain
>>> 6868???????? Java SE Platform API such as reflection.
>>>
>>> "Created by [the] Java virtual machine" (add "the")
>>> Change "An array class creation" to "The creation" since your are 
>>> repeating "An array class" from the previous sentence.
>>>
>>>
>>> In the JVMTI spec ClassLoaderClasses section:
>>>
>>> "That is, initiating_loader has been recorded as an initiating 
>>> loader of the returned classes." -> "That is, all classes for which 
>>> initiating_loader has been recorded as an initiating loader."
>>>
>>>
>>> In the JVMTI spec GetClassSignature section:
>>>
>>> "If the class indicated by klass is ..." -> "If the class ..." (you 
>>> just finished the previous sentence with "class indicated by Klass").
>>>
>>> "the returned name is [the] JNI type signature" (add "the"). Also, 
>>> is "JNI type signature" formally defined somewhere? This relates to 
>>> my JDWP spec comment above.
>>>
>>
>> It's a link to 
>> https://download.java.net/java/early_access/jdk15/docs/specs/jni/types.html#type-signatures. 
>> This is how the current JVM TI spec defines.
>>
>>> " where N is the binary name encoded in internal form indicated by 
>>> the class file". Is "binary name encoded in internal form" explained 
>>> somewhere? 
>>
>> JVMS 4.2.1
>>
>> https://docs.oracle.com/javase/specs/jvms/se14/html/jvms-4.html#jvms-4.2.1 
>>
>>
>>> Also, can you add an example of a returned hidden class signature?
>>>
>> OK
>>
>>>
>>> In the JVMTI spec ClassLoad section:
>>>
>>> "representation using [a] class loader" (add "a")
>>>
>>> "By invoking Lookup::defineHiddenClass, that creates ..." -> "By 
>>> invoking Lookup::defineHiddenClass to create ..."
>>>
>>> "certain Java SE Platform API" -> Should be "APIs"
>>>
>>>
>>> In JDI ClassLoaderReference.definedClasses()
>>>
>>> "loaded at least to the point of preparation and types ..." -> 
>>> "loaded at least to the point of preparation, and types ..." (Note, 
>>> this not a new issue with your edits)
>>>
>>>
>>> In Instrumentation.getAllLoadedClasses:
>>>
>>> The reference to `class` did not format properly.
>>>
>>
>> Serguei caught that one too.? I fixed it in my local repo.
>>
>>> "by invoking Lookup::defineHiddenClass that creates" -> "by invoking 
>>> Lookup::defineHiddenClass, which creates"
>>>
>>> "An array class is created directly by Java virtual machine. An 
>>> array class creation can be triggered ..." ->"An array class is 
>>> created directly by the Java virtual machine. Array class creation 
>>> can be triggered ..."
>>>
>>>
>>> In Instrumentation.getInitiatedClasses:
>>>
>>> "That is, loader has been recorded as an initiating loader of these 
>>> classes." -> "That is, all classes for which loader has been 
>>> recorded as an initiating loader."
>>>
>>> thanks,
>>>
>>> Chris
>>
>> Thanks for the review.?? See this patch of the editorial fixes.
>>
>> diff --git a/make/data/jdwp/jdwp.spec b/make/data/jdwp/jdwp.spec
>> --- a/make/data/jdwp/jdwp.spec
>> +++ b/make/data/jdwp/jdwp.spec
>> @@ -2265,8 +2265,8 @@
>> ???????? "Returns a list of all classes which this class loader can 
>> find "
>> ???????? "by name via <code>ClassLoader::loadClass</code>, "
>> ???????? "<code>Class::forName</code> and bytecode linkage. That is, "
>> -??????? "this class loader has been recorded as an <i>initiating</i> "
>> -??????? "loader of the returned classes. The list contains each
>> +??????? "all classes for which this class loader has been recorded 
>> as an "
>> +??????? "<i>initiating</i> loader. The list contains each "
>> ???????? "reference type created by this loader and any types for 
>> which "
>> ???????? "loading was delegated by this class loader to another class 
>> loader. "
>> ???????? "The list does not include hidden classes or interfaces 
>> created by "
>> diff --git a/src/hotspot/share/prims/jvmti.xml 
>> b/src/hotspot/share/prims/jvmti.xml
>> --- a/src/hotspot/share/prims/jvmti.xml
>> +++ b/src/hotspot/share/prims/jvmti.xml
>> @@ -6857,15 +6857,15 @@
>> ???????? A class or interface creation can be triggered by one of the 
>> following:
>> ???????? <ul>
>> ???????? <li>By loading and deriving a class from a 
>> <code>class</code> file representation
>> -??????????? using class loader (see <vmspec chapter="5.3"/>).</li>
>> +??????????? using a class loader (see <vmspec chapter="5.3"/>).</li>
>> ???????? <li>By invoking <externallink 
>> id="../api/java.base/java/lang/invoke/MethodHandles.Lookup.html#defineHiddenClass(byte[],boolean,java.lang.invoke.MethodHandles.Lookup.ClassOption...)">Lookup::defineHiddenClass</externallink>
>> ???????????? that creates a hidden class or interface from a 
>> <code>class</code> file representation.</li>
>> -??????? <li>By invoking methods in certain Java SE Platform API such 
>> as reflection.</li>
>> +??????? <li>By invoking methods in certain Java SE Platform APIs 
>> such as reflection.</li>
>> ????????? </ul>
>> ???????? <p/>
>> -??????? An array class is created directly by Java virtual machine.? 
>> An array class
>> -??????? creation can be triggered by using class loaders or by 
>> invoking methods in certain
>> -??????? Java SE Platform API such as reflection.
>> +??????? An array class is created directly by the Java virtual 
>> machine.? The creation
>> +??????? can be triggered by using class loaders or by invoking 
>> methods in certain
>> +??????? Java SE Platform APIs such as reflection.
>> ???????? <p/>
>> ???????? The returned list includes all classes and interfaces, 
>> including
>> ???????? <externallink 
>> id="../api/java.base/java/lang/Class.html#isHidden()">
>> @@ -6904,8 +6904,8 @@
>> ???????? can find by name via
>> ???????? <externallink 
>> id="../api/java.base/java/lang/ClassLoader.html#loadClass(java.lang.String,boolean)">ClassLoader::loadClass</externallink>,
>> ???????? <externallink 
>> id="../api/java.base/java/lang/Class.html#forName(java.lang.String,boolean,java.lang.ClassLoader)">Class::forName</externallink> 
>> and bytecode linkage.
>> -??????? That is, <code>initiating_loader</code>
>> -??????? has been recorded as an initiating loader of the returned 
>> classes.
>> +??????? That is, all classes for which <code>initiating_loader</code>
>> +??????? has been recorded as an initiating loader.
>> ???????? Each class in the returned array was created by this class 
>> loader,
>> ???????? either by defining it directly or by delegation to another 
>> class loader.
>> ???????? See <vmspec chapter="5.3"/>.
>> @@ -6964,21 +6964,22 @@
>> ?????? <description>
>> ???????? Return the name and the generic signature of the class 
>> indicated by <code>klass</code>.
>> ???????? <p/>
>> -??????? If the class indicated by <code>klass</code> is a class or 
>> interface, then:
>> +??????? If the class is a class or interface, then:
>> ???????? <ul>
>> ???????? <li>If the class or interface is not <externallink 
>> id="../api/java.base/java/lang/Class.html#isHidden()">hi
>> dden</externallink>,
>> -????????? then the returned name is <externallink 
>> id="jni/types.html#type-signatures">
>> +????????? then the returned name is the <externallink 
>> id="jni/types.html#type-signatures">
>> ?????????? JNI type signature</externallink>.
>> ?????????? For example, java.util.List is "Ljava/util/List;"
>> ???????? </li>
>> ???????? <li>If the class or interface is <externallink 
>> id="../api/java.base/java/lang/Class.html#isHidden()">hidden
>> </externallink>,
>> ?????????? then the returned name is a string of the form:
>> ?????????? <code>"L" + N + "." +? S + ";"</code>
>> -????????? where <code>N</code> is the binary name encoded in 
>> internal form
>> +????????? where <code>N</code> is the binary name encoded in 
>> internal form (JVMS 4.2.1)
>> ?????????? indicated by the <code>class</code> file passed to
>> ?????????? <externallink 
>> id="../api/java.base/java/lang/invoke/MethodHandles.Lookup.html#defineHiddenClass(byte[],boolean,java.lang.invoke.MethodHandles.Lookup.ClassOption...)">Lookup::defineHiddenClass</externallink>,
>> ?????????? and <code>S</code> is an unqualified name.
>> -????????? The returned name is not a valid type descriptor.
>> +????????? The returned name is not a type descriptor and does not 
>> conform to JVMS 4.3.2.
>> +????????? For example, com.foo.Foo/AnySuffix is 
>> "Lcom/foo/Foo.AnySuffix;"
>> ???????? </li>
>> ???????? </ul>
>> ???????? <p/>
>> @@ -12768,10 +12769,10 @@
>> ?????? A class or interface is created by one of the following:
>> ?????? <ul>
>> ?????? <li>By loading and deriving a class from a <code>class</code> 
>> file representation
>> -????????? using class loader.</li>
>> +????????? using a class loader.</li>
>> ?????? <li>By invoking <externallink 
>> id="../api/java.base/java/lang/invoke/MethodHandles.Lookup.html#defineHiddenClass(byte[],boolean,java.lang.invoke.MethodHandles.Lookup.ClassOption...)">Lookup::defineHiddenClass</externallink>,
>> ?????????? that creates a hidden class or interface from a 
>> <code>class</code> file representation.</li>
>> -????? <li>By invoking methods in certain Java SE Platform API such 
>> as reflection.</li>
>> +????? <li>By invoking methods in certain Java SE Platform APIs such 
>> as reflection.</li>
>> ?????? </ul>
>> ?????? <p/>
>> ?????? Array class creation does not generate a class load event.
>> diff --git 
>> a/src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java 
>> b/src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java 
>>
>> --- 
>> a/src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java
>> +++ 
>> b/src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java
>> @@ -392,15 +392,15 @@
>> ????? * A class or interface creation can be triggered by one of the 
>> following:
>> ????? * <ul>
>> ????? * <li>by loading and deriving a class from a {@code class} file 
>> representation
>> -???? *???? using class loader (see JVMS {@jvms 5.3}).
>> +???? *???? using a class loader (see JVMS {@jvms 5.3}).
>> ????? * <li>by invoking {@link 
>> java.lang.invoke.MethodHandles.Lookup#defineHiddenClass(byte[], 
>> boolean, java.lang.invoke.MethodHandles.Lookup.ClassOption...)
>> -???? *???? Lookup::defineHiddenClass} that creates a {@link 
>> Class#isHidden
>> +???? *???? Lookup::defineHiddenClass}, which creates a {@link 
>> Class#isHidden
>> ????? *???? hidden class or interface} from a {@code class} file 
>> representation.
>> ????? * <li>by invoking methods in certain reflection APIs such as
>> ????? *???? {@link Class#forName(String) Class::forName}.
>> ????? * </ul>
>> ????? * <p>
>> -???? * An array class is created directly by Java virtual machine. 
>> An array
>> +???? * An array class is created directly by the Java virtual 
>> machine.? Array
>> ????? * class creation can be triggered by using class loaders or by 
>> invoking
>> ????? * methods in certain reflection APIs such as
>> ????? * {@link java.lang.reflect.Array#newInstance(Class, int) 
>> Array::newInstance}
>> @@ -420,8 +420,8 @@
>> ????? * Returns an array of all classes which {@code loader} can find 
>> by name
>> ????? * via {@link ClassLoader#loadClass(String, boolean) 
>> ClassLoader::loadClass},
>> ????? * {@link Class#forName(String) Class::forName} and bytecode 
>> linkage.
>> -???? * That is, {@code loader} has been recorded as an initiating 
>> loader
>> -???? * of these classes. If the supplied {@code loader} is {@code 
>> null},
>> +???? * That is, all classes for which {@code loader} has been 
>> recorded as
>> +???? * an initiating loader. If the supplied {@code loader} is 
>> {@code null},
>> ????? * classes that the bootstrap class loader can find by name are 
>> returned.
>> ????? *
>> ????? * <p>
>> diff --git 
>> a/src/jdk.jdi/share/classes/com/sun/jdi/ClassLoaderReference.java 
>> b/src/jdk.jdi/share/classes/com/sun/jdi/ClassLoaderReference.java
>> --- a/src/jdk.jdi/share/classes/com/sun/jdi/ClassLoaderReference.java
>> +++ b/src/jdk.jdi/share/classes/com/sun/jdi/ClassLoaderReference.java
>> @@ -46,7 +46,7 @@
>> ????? * No ordering of this list guaranteed.
>> ????? * The returned list will include all reference types, including
>> ????? * {@link Class#isHidden hidden classes or interfaces}, loaded
>> -???? * at least to the point of preparation and types (like array)
>> +???? * at least to the point of preparation, and types (like array)
>> ????? * for which preparation is not defined.
>> ????? *
>> ????? * @return a {@code List} of {@link ReferenceType} objects 
>> mirroring types
>> @@ -59,8 +59,8 @@
>> ????? * Returns a list of all classes which this class loader
>> ????? * can find by name via {@link ClassLoader#loadClass(String, 
>> boolean)
>> ????? * ClassLoader::loadClass}, {@link Class#forName(String) 
>> Class::forName}
>> -???? * and bytecode linkage in the target VM.? That is, this class 
>> loader
>> -???? * has been recorded as an initiating loader of these classes.
>> +???? * and bytecode linkage in the target VM.? That is, all classes for
>> +???? * which this class loader has been recorded as an initiating 
>> loader.
>> ????? * <p>
>> ????? * Each class in the returned list was created by this class loader
>> ????? * either by defining it directly or by delegation to another 
>> class loader
>> diff --git 
>> a/src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java 
>> b/src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java
>> --- a/src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java
>> +++ b/src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java
>> @@ -138,9 +138,9 @@
>> ????? * A class or interface creation can be triggered by one of the 
>> following:
>> ????? * <ul>
>> ????? * <li>by loading and deriving a class from a {@code class} file 
>> representation
>> -???? *???? using class loader (see JVMS {@jvms 5.3}).
>> +???? *???? using a class loader (see JVMS {@jvms 5.3}).
>> ????? * <li>by invoking {@link 
>> java.lang.invoke.MethodHandles.Lookup#defineHiddenClass(byte[], 
>> boolean, java.lang.invoke.MethodHandles.Lookup.ClassOption...)
>> -???? *???? Lookup::defineHiddenClass} that creates a {@link 
>> Class#isHidden
>> +???? *???? Lookup::defineHiddenClass}, which creates a {@link 
>> Class#isHidden
>> ????? *???? hidden class or interface} from a {@code class} file 
>> representation.
>> ????? * <li>by invoking methods in certain reflection APIs such as
>> ????? *???? {@link Class#forName(String) Class::forName}.
>>
>> Mandy
>>
>>
>


From chris.plummer at oracle.com  Sat Apr 18 07:47:20 2020
From: chris.plummer at oracle.com (Chris Plummer)
Date: Sat, 18 Apr 2020 00:47:20 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <440544f0-8a74-7ff1-8b78-0c20307bb89a@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <1a5eb022-dd7e-b0bb-750f-c43be195eb42@oracle.com>
 <da9f80ff-6d63-aebe-a9fa-1a683eb42bbc@oracle.com>
 <4A360464-1873-43A8-B423-C93D1BEF9895@oracle.com>
 <51adeaaf-9c85-240c-ed0d-f460c117ba46@oracle.com>
 <6890c88c-809b-fbea-aa10-675661ef2c68@oracle.com>
 <440544f0-8a74-7ff1-8b78-0c20307bb89a@oracle.com>
Message-ID: <ba5b5c63-912e-c358-8c11-9643424d91ad@oracle.com>

Hi Mandy,

Comments inline.

On 4/17/20 4:52 PM, Mandy Chung wrote:
>
>
> On 4/17/20 3:51 PM, Chris Plummer wrote:
>> Hi Mandy,
>>
>> Thanks for updating the svc specs. Some comments below:
>>
>>
>> In the JDWP spec update, you changed "JNI signature" to "type 
>> signature" in one place, but left it as "JNI signature" everywhere 
>> else. Should they all be changed?
>>
>
> JDWP signature is changed because there is no JNI signature 
> representing a hidden class.??? I leave other references to JNI 
> signature as is since those only apply for normal classes as I wanted 
> to keep this change specifically due to hidden classes.? I think it's 
> good to file a JBS issue to follow up on JDWP and JDI to determine if 
> the spec should be upgraded to reference the new TypeDescriptor API.
>
>>
>> In the JDWP spec for ClassLoaderReference.VisibleClasses:
>>
>> ?"That is, this class loader has been recorded as an 
>> <i>initiating</i> loader of the returned classes." -> "That is, all 
>> classes for which this class loader has been recorded as an 
>> <i>initiating</i> loader."
>>
>> This seems like too much detail to be put here. Basically the term 
>> "initiating ClassLoader" has turned into a short essay. Is it 
>> possible that all this detail could be put elsewhere and referenced? 
>
> Any suggestion??? We attempted to place those description in JVM TI 
> Class section or ClassLoad event.?? However, that's not ideal place 
> since that's needed by JDWP, JDI and Instrumentation.?? I found 
> inlining this description is not ideal but it provides adequate 
> clarification.
See other thread.
>
>> Aren't there other places in other specs where a similar 
>> clarification of "initiating ClassLoader" is needed (I see now that 
>> ClassLoaderClasses in the JVMTI spec, 
>> ClassLoaderReference,visibleClasses in the JDI spec, and 
>> Instrumentation.getInitiatedClasses are all dealing with this, but 
>> not all in the exact same way).
>>
>
> I took the conservative side and make sure the clarification is in 
> place for all APIs.? I'm open to any suggestion for example having 
> JDWP and JDI to link to JVM TI spec if you think appropriate.
>
>>
>> In the JVMTI spec for GetLoadedClasses:
>>
>> This suffers in a way similar to ClassLoaderReference.VisibleClasses 
>> in the JDWP spec, although not as badly. A simple concept ends up 
>> with a complex description, and it seems that description should 
>> really be in a more centralized place.? I would also suggest a bit of 
>> cleanup of these lines:
>>
>> 6866???????? An array class is created directly by Java virtual 
>> machine.? An array class
>> 6867???????? creation can be triggered by using class loaders or by 
>> invoking methods in certain
>> 6868???????? Java SE Platform API such as reflection.
>>
>> "Created by [the] Java virtual machine" (add "the")
>> Change "An array class creation" to "The creation" since your are 
>> repeating "An array class" from the previous sentence.
>>
>>
>> In the JVMTI spec ClassLoaderClasses section:
>>
>> "That is, initiating_loader has been recorded as an initiating loader 
>> of the returned classes." -> "That is, all classes for which 
>> initiating_loader has been recorded as an initiating loader."
>>
>>
>> In the JVMTI spec GetClassSignature section:
>>
>> "If the class indicated by klass is ..." -> "If the class ..." (you 
>> just finished the previous sentence with "class indicated by Klass").
>>
>> "the returned name is [the] JNI type signature" (add "the"). Also, is 
>> "JNI type signature" formally defined somewhere? This relates to my 
>> JDWP spec comment above.
>>
>
> It's a link to 
> https://download.java.net/java/early_access/jdk15/docs/specs/jni/types.html#type-signatures. 
> This is how the current JVM TI spec defines.
Since it looks like this reference was present before your changes, I 
guess it's ok to leave it, but like JDWP maybe a bug should be filed to 
clean it up.

Regarding your JNI spec reference, it say "The JNI uses the Java VM?s 
representation of type signatures" and then has a table called "Java VM 
Type Signatures". No where in the JNI spec do you see "JNI Signature" or 
"JNI Type Signature". It seems we should always just use "Type Signature"?

Even the JVMTI spec is not consistent. It has a couple of references to 
JNI Type Signature as links to the JNI spec, but also says "type 
signatures" in a couple of places, including for GetLocalVariableTable() 
which refers the the JVMS: "The local variable's type signature, encoded 
as a modified UTF-8 string. The signature format is the same as that 
defined in The Java? Virtual Machine Specification, Chapter 4.3.2. "

>
>> " where N is the binary name encoded in internal form indicated by 
>> the class file". Is "binary name encoded in internal form" explained 
>> somewhere? 
>
> JVMS 4.2.1
>
> https://docs.oracle.com/javase/specs/jvms/se14/html/jvms-4.html#jvms-4.2.1
Thanks. I see you've also added a reference in your edits below.
>
>> Also, can you add an example of a returned hidden class signature?
>>
> OK
>
>>
>> In the JVMTI spec ClassLoad section:
>>
>> "representation using [a] class loader" (add "a")
>>
>> "By invoking Lookup::defineHiddenClass, that creates ..."? -> "By 
>> invoking Lookup::defineHiddenClass to create ..."
>>
>> "certain Java SE Platform API" -> Should be "APIs"
>>
>>
>> In JDI ClassLoaderReference.definedClasses()
>>
>> "loaded at least to the point of preparation and types ..." -> 
>> "loaded at least to the point of preparation, and types ..." (Note, 
>> this not a new issue with your edits)
>>
>>
>> In Instrumentation.getAllLoadedClasses:
>>
>> The reference to `class` did not format properly.
>>
>
> Serguei caught that one too.? I fixed it in my local repo.
>
>> "by invoking Lookup::defineHiddenClass that creates" -> "by invoking 
>> Lookup::defineHiddenClass, which creates"
>>
>> "An array class is created directly by Java virtual machine. An array 
>> class creation can be triggered ..." ->"An array class is created 
>> directly by the Java virtual machine. Array class creation can be 
>> triggered ..."
>>
>>
>> In Instrumentation.getInitiatedClasses:
>>
>> "That is, loader has been recorded as an initiating loader of these 
>> classes." -> "That is, all classes for which loader has been recorded 
>> as an initiating loader."
>>
>> thanks,
>>
>> Chris
>
> Thanks for the review.?? See this patch of the editorial fixes.
>
> diff --git a/make/data/jdwp/jdwp.spec b/make/data/jdwp/jdwp.spec
> --- a/make/data/jdwp/jdwp.spec
> +++ b/make/data/jdwp/jdwp.spec
> @@ -2265,8 +2265,8 @@
> ???????? "Returns a list of all classes which this class loader can find "
> ???????? "by name via <code>ClassLoader::loadClass</code>, "
> ???????? "<code>Class::forName</code> and bytecode linkage. That is, "
> -??????? "this class loader has been recorded as an <i>initiating</i> "
> -??????? "loader of the returned classes. The list contains each
> +??????? "all classes for which this class loader has been recorded as 
> an "
> +??????? "<i>initiating</i> loader. The list contains each "
> ???????? "reference type created by this loader and any types for which "
> ???????? "loading was delegated by this class loader to another class 
> loader. "
> ???????? "The list does not include hidden classes or interfaces 
> created by "
> diff --git a/src/hotspot/share/prims/jvmti.xml 
> b/src/hotspot/share/prims/jvmti.xml
> --- a/src/hotspot/share/prims/jvmti.xml
> +++ b/src/hotspot/share/prims/jvmti.xml
> @@ -6857,15 +6857,15 @@
> ???????? A class or interface creation can be triggered by one of the 
> following:
> ???????? <ul>
> ???????? <li>By loading and deriving a class from a <code>class</code> 
> file representation
> -??????????? using class loader (see <vmspec chapter="5.3"/>).</li>
> +??????????? using a class loader (see <vmspec chapter="5.3"/>).</li>
> ???????? <li>By invoking <externallink 
> id="../api/java.base/java/lang/invoke/MethodHandles.Lookup.html#defineHiddenClass(byte[],boolean,java.lang.invoke.MethodHandles.Lookup.ClassOption...)">Lookup::defineHiddenClass</externallink>
> ???????????? that creates a hidden class or interface from a 
> <code>class</code> file representation.</li>
> -??????? <li>By invoking methods in certain Java SE Platform API such 
> as reflection.</li>
> +??????? <li>By invoking methods in certain Java SE Platform APIs such 
> as reflection.</li>
> ????????? </ul>
> ???????? <p/>
> -??????? An array class is created directly by Java virtual machine.? 
> An array class
> -??????? creation can be triggered by using class loaders or by 
> invoking methods in certain
> -??????? Java SE Platform API such as reflection.
> +??????? An array class is created directly by the Java virtual 
> machine.? The creation
> +??????? can be triggered by using class loaders or by invoking 
> methods in certain
> +??????? Java SE Platform APIs such as reflection.
> ???????? <p/>
> ???????? The returned list includes all classes and interfaces, including
> ???????? <externallink 
> id="../api/java.base/java/lang/Class.html#isHidden()">
> @@ -6904,8 +6904,8 @@
> ???????? can find by name via
> ???????? <externallink 
> id="../api/java.base/java/lang/ClassLoader.html#loadClass(java.lang.String,boolean)">ClassLoader::loadClass</externallink>,
> ???????? <externallink 
> id="../api/java.base/java/lang/Class.html#forName(java.lang.String,boolean,java.lang.ClassLoader)">Class::forName</externallink> 
> and bytecode linkage.
> -??????? That is, <code>initiating_loader</code>
> -??????? has been recorded as an initiating loader of the returned 
> classes.
> +??????? That is, all classes for which <code>initiating_loader</code>
> +??????? has been recorded as an initiating loader.
> ???????? Each class in the returned array was created by this class 
> loader,
> ???????? either by defining it directly or by delegation to another 
> class loader.
> ???????? See <vmspec chapter="5.3"/>.
> @@ -6964,21 +6964,22 @@
> ?????? <description>
> ???????? Return the name and the generic signature of the class 
> indicated by <code>klass</code>.
> ???????? <p/>
> -??????? If the class indicated by <code>klass</code> is a class or 
> interface, then:
> +??????? If the class is a class or interface, then:
> ???????? <ul>
> ???????? <li>If the class or interface is not <externallink 
> id="../api/java.base/java/lang/Class.html#isHidden()">hi
> dden</externallink>,
> -????????? then the returned name is <externallink 
> id="jni/types.html#type-signatures">
> +????????? then the returned name is the <externallink 
> id="jni/types.html#type-signatures">
> ?????????? JNI type signature</externallink>.
> ?????????? For example, java.util.List is "Ljava/util/List;"
> ???????? </li>
> ???????? <li>If the class or interface is <externallink 
> id="../api/java.base/java/lang/Class.html#isHidden()">hidden
> </externallink>,
> ?????????? then the returned name is a string of the form:
> ?????????? <code>"L" + N + "." +? S + ";"</code>
> -????????? where <code>N</code> is the binary name encoded in internal 
> form
> +????????? where <code>N</code> is the binary name encoded in internal 
> form (JVMS 4.2.1)
> ?????????? indicated by the <code>class</code> file passed to
> ?????????? <externallink 
> id="../api/java.base/java/lang/invoke/MethodHandles.Lookup.html#defineHiddenClass(byte[],boolean,java.lang.invoke.MethodHandles.Lookup.ClassOption...)">Lookup::defineHiddenClass</externallink>,
> ?????????? and <code>S</code> is an unqualified name.
> -????????? The returned name is not a valid type descriptor.
> +????????? The returned name is not a type descriptor and does not 
> conform to JVMS 4.3.2.
> +????????? For example, com.foo.Foo/AnySuffix is "Lcom/foo/Foo.AnySuffix;"
> ???????? </li>
> ???????? </ul>
> ???????? <p/>
> @@ -12768,10 +12769,10 @@
> ?????? A class or interface is created by one of the following:
> ?????? <ul>
> ?????? <li>By loading and deriving a class from a <code>class</code> 
> file representation
> -????????? using class loader.</li>
> +????????? using a class loader.</li>
> ?????? <li>By invoking <externallink 
> id="../api/java.base/java/lang/invoke/MethodHandles.Lookup.html#defineHiddenClass(byte[],boolean,java.lang.invoke.MethodHandles.Lookup.ClassOption...)">Lookup::defineHiddenClass</externallink>,
> ?????????? that creates a hidden class or interface from a 
> <code>class</code> file representation.</li>
> -????? <li>By invoking methods in certain Java SE Platform API such as 
> reflection.</li>
> +????? <li>By invoking methods in certain Java SE Platform APIs such 
> as reflection.</li>
> ?????? </ul>
> ?????? <p/>
> ?????? Array class creation does not generate a class load event.
> diff --git 
> a/src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java 
> b/src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java
> --- 
> a/src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java
> +++ 
> b/src/java.instrument/share/classes/java/lang/instrument/Instrumentation.java
> @@ -392,15 +392,15 @@
> ????? * A class or interface creation can be triggered by one of the 
> following:
> ????? * <ul>
> ????? * <li>by loading and deriving a class from a {@code class} file 
> representation
> -???? *???? using class loader (see JVMS {@jvms 5.3}).
> +???? *???? using a class loader (see JVMS {@jvms 5.3}).
> ????? * <li>by invoking {@link 
> java.lang.invoke.MethodHandles.Lookup#defineHiddenClass(byte[], 
> boolean, java.lang.invoke.MethodHandles.Lookup.ClassOption...)
> -???? *???? Lookup::defineHiddenClass} that creates a {@link 
> Class#isHidden
> +???? *???? Lookup::defineHiddenClass}, which creates a {@link 
> Class#isHidden
> ????? *???? hidden class or interface} from a {@code class} file 
> representation.
> ????? * <li>by invoking methods in certain reflection APIs such as
> ????? *???? {@link Class#forName(String) Class::forName}.
> ????? * </ul>
> ????? * <p>
> -???? * An array class is created directly by Java virtual machine.? 
> An array
> +???? * An array class is created directly by the Java virtual 
> machine.? Array
> ????? * class creation can be triggered by using class loaders or by 
> invoking
> ????? * methods in certain reflection APIs such as
> ????? * {@link java.lang.reflect.Array#newInstance(Class, int) 
> Array::newInstance}
> @@ -420,8 +420,8 @@
> ????? * Returns an array of all classes which {@code loader} can find 
> by name
> ????? * via {@link ClassLoader#loadClass(String, boolean) 
> ClassLoader::loadClass},
> ????? * {@link Class#forName(String) Class::forName} and bytecode linkage.
> -???? * That is, {@code loader} has been recorded as an initiating loader
> -???? * of these classes. If the supplied {@code loader} is {@code null},
> +???? * That is, all classes for which {@code loader} has been recorded as
> +???? * an initiating loader. If the supplied {@code loader} is {@code 
> null},
> ????? * classes that the bootstrap class loader can find by name are 
> returned.
> ????? *
> ????? * <p>
> diff --git 
> a/src/jdk.jdi/share/classes/com/sun/jdi/ClassLoaderReference.java 
> b/src/jdk.jdi/share/classes/com/sun/jdi/ClassLoaderReference.java
> --- a/src/jdk.jdi/share/classes/com/sun/jdi/ClassLoaderReference.java
> +++ b/src/jdk.jdi/share/classes/com/sun/jdi/ClassLoaderReference.java
> @@ -46,7 +46,7 @@
> ????? * No ordering of this list guaranteed.
> ????? * The returned list will include all reference types, including
> ????? * {@link Class#isHidden hidden classes or interfaces}, loaded
> -???? * at least to the point of preparation and types (like array)
> +???? * at least to the point of preparation, and types (like array)
> ????? * for which preparation is not defined.
> ????? *
> ????? * @return a {@code List} of {@link ReferenceType} objects 
> mirroring types
> @@ -59,8 +59,8 @@
> ????? * Returns a list of all classes which this class loader
> ????? * can find by name via {@link ClassLoader#loadClass(String, boolean)
> ????? * ClassLoader::loadClass}, {@link Class#forName(String) 
> Class::forName}
> -???? * and bytecode linkage in the target VM.? That is, this class loader
> -???? * has been recorded as an initiating loader of these classes.
> +???? * and bytecode linkage in the target VM.? That is, all classes for
> +???? * which this class loader has been recorded as an initiating loader.
> ????? * <p>
> ????? * Each class in the returned list was created by this class loader
> ????? * either by defining it directly or by delegation to another 
> class loader
> diff --git a/src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java 
> b/src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java
> --- a/src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java
> +++ b/src/jdk.jdi/share/classes/com/sun/jdi/VirtualMachine.java
> @@ -138,9 +138,9 @@
> ????? * A class or interface creation can be triggered by one of the 
> following:
> ????? * <ul>
> ????? * <li>by loading and deriving a class from a {@code class} file 
> representation
> -???? *???? using class loader (see JVMS {@jvms 5.3}).
> +???? *???? using a class loader (see JVMS {@jvms 5.3}).
> ????? * <li>by invoking {@link 
> java.lang.invoke.MethodHandles.Lookup#defineHiddenClass(byte[], 
> boolean, java.lang.invoke.MethodHandles.Lookup.ClassOption...)
> -???? *???? Lookup::defineHiddenClass} that creates a {@link 
> Class#isHidden
> +???? *???? Lookup::defineHiddenClass}, which creates a {@link 
> Class#isHidden
> ????? *???? hidden class or interface} from a {@code class} file 
> representation.
> ????? * <li>by invoking methods in certain reflection APIs such as
> ????? *???? {@link Class#forName(String) Class::forName}.
>
Above changes all look good.

thanks,

Chris
> Mandy
>
>


From mandy.chung at oracle.com  Sat Apr 18 21:09:57 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Sat, 18 Apr 2020 14:09:57 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <ba5b5c63-912e-c358-8c11-9643424d91ad@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <1a5eb022-dd7e-b0bb-750f-c43be195eb42@oracle.com>
 <da9f80ff-6d63-aebe-a9fa-1a683eb42bbc@oracle.com>
 <4A360464-1873-43A8-B423-C93D1BEF9895@oracle.com>
 <51adeaaf-9c85-240c-ed0d-f460c117ba46@oracle.com>
 <6890c88c-809b-fbea-aa10-675661ef2c68@oracle.com>
 <440544f0-8a74-7ff1-8b78-0c20307bb89a@oracle.com>
 <ba5b5c63-912e-c358-8c11-9643424d91ad@oracle.com>
Message-ID: <4f9a5a74-3b10-3dee-e055-feef32191c5d@oracle.com>



On 4/18/20 12:47 AM, Chris Plummer wrote:
>>
>> It's a link to 
>> https://download.java.net/java/early_access/jdk15/docs/specs/jni/types.html#type-signatures. 
>> This is how the current JVM TI spec defines.
> Since it looks like this reference was present before your changes, I 
> guess it's ok to leave it, but like JDWP maybe a bug should be filed 
> to clean it up.
>

Right, they are existing reference.? I'll let Serguei to file a JBS 
issue to follow up.

> Regarding your JNI spec reference, it say "The JNI uses the Java VM?s 
> representation of type signatures" and then has a table called "Java 
> VM Type Signatures". No where in the JNI spec do you see "JNI 
> Signature" or "JNI Type Signature". It seems we should always just use 
> "Type Signature"?
>

I agree that the svc specs should be examined and use the terminologies 
consistently.

> Even the JVMTI spec is not consistent. It has a couple of references 
> to JNI Type Signature as links to the JNI spec, but also says "type 
> signatures" in a couple of places, including for 
> GetLocalVariableTable() which refers the the JVMS: "The local 
> variable's type signature, encoded as a modified UTF-8 string. The 
> signature format is the same as that defined in The Java? Virtual 
> Machine Specification, Chapter 4.3.2. "

Mandy

From fw at deneb.enyo.de  Sat Apr 18 21:55:46 2020
From: fw at deneb.enyo.de (Florian Weimer)
Date: Sat, 18 Apr 2020 23:55:46 +0200
Subject: RFR: 8241423: NUMA APIs may fail to work in the docker due to
 operation not permitted
In-Reply-To: <312806A9-2A4F-42CE-8DA1-2EF351971DFC@oracle.com> (Bob Vandette's
 message of "Fri, 10 Apr 2020 10:27:52 -0400")
References: <97987050-6175-44EC-8ABE-B51D60E513AF@tencent.com>
 <312806A9-2A4F-42CE-8DA1-2EF351971DFC@oracle.com>
Message-ID: <87d0844fx9.fsf@mid.deneb.enyo.de>

* Bob Vandette:

> I understand but it would be useful to ensure that all aspects of
> supporting NUMA in containers work while addressing this issue.
> What good is it to enable NUMA but have it not work properly.  I?m
> only asking for some validation since you appear to have a proper
> setup.

There is a large variability among container environments, so this
seems rather futile.

> It would be better if we could find a more supported way of
> determining if our process has the required access rather than
> counting on a failing syscall.
>
> What about using prctl?
>
> http://man7.org/linux/man-pages/man2/prctl.2.html
>
> This call can be used to query if a specific capability is enabled.

I think the only feasible way would be to probe early if all the
required system calls (and flag arguments) are available.  Container
environments block prctl operations, too, or block system calls by
completely different means (such as a patched Linux kernel).

It may be more attractive to ask container environments to provide a
closer approximation to the official Linux system call interface.

From leo.korinth at oracle.com  Sun Apr 19 15:39:00 2020
From: leo.korinth at oracle.com (Leo Korinth)
Date: Sun, 19 Apr 2020 17:39:00 +0200
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <98749A40-89C8-4E15-B46A-23F0D1CB2A32@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
 <135a6496-5b54-9117-5cbf-e3211ddc32dd@oracle.com>
 <bbb2b5ef-62f2-6507-95f7-03712a382153@oracle.com>
 <e242e057-de6e-493a-eb00-068b2db8c9fa@oracle.com>
 <df621fd2-0eaf-cb24-788f-12166b271303@oracle.com>
 <98749A40-89C8-4E15-B46A-23F0D1CB2A32@oracle.com>
Message-ID: <841c7ed3-ec0b-b300-15f1-30863453f74b@oracle.com>



On 17/04/2020 09:46, Kim Barrett wrote:
>> On Apr 14, 2020, at 9:53 AM, Leo Korinth <leo.korinth at oracle.com> wrote:
>> On 06/04/2020 01:37, David Holmes wrote:
>>>> I could change CHECK_0 to use NULL_WORD though and if I do, I get compiler errors. CHECK_0 is used for both pointers and integral zeroes and types that implicitly converts from either. What the original thought was is unclear to me; is CHECK_0 to be used to return things convertible from pointers? If so, what is CHECK_NULL for?
>>> No CHECK_0 is intended for int-returning functions (which would also extend to intptr_t).
>>
>> It might be intended for it, but it is not used in such a way (therefore my remark on compile failures when using NULL_WORD)
>>
>> I will do no change here.
> 
> It sounds like there are some places using CHECK_0 that should be
> using CHECK_NULL, and they should be fixed. But that can be a separate
> change. It might be aided (and backsliding prevented) by doing
> something similar to what I suggest for JVMCI_CHECK_0/NULL below.

I believe that there are places that could use neither (the macro 
handles both integral and pointer types), but I do not want to fix that 
macro nest.

> 
>> total combined change:
>> http://cr.openjdk.java.net/~lkorinth/8240110/1
> 
> I've gone through all the changes in the "total combined change", with
> comments below.
> 
> ------------------------------------------------------------------------------
> 
> In the end you seem to have backed away from fixing the definition of
> NULL_WORD, and I'm not sure why. Though I think the definition that
> was proposed is at least unnecessarily baroque, possibly wrong. It
> seems to me that a good and sufficient change for that would be to
> delete all the existing platform definitions and put the following in
> globalDefinitions.hpp:
> 
> const intptr_t NULL_WORD = 0;
> 
> Then it's no longer a null pointer constant, so can't be misused in
> contexts where a pointer is required, will be treated as an integral
> value by templates, &etc.

my first try was (in part2):

struct null_word_t {
   operator intptr_t()  const { return 0; }
};

which IMO is nice as it:
1) mirrors my (initial) implementation of null_t
2) can be template specialized against
3) is defined in one place

Unfortunately we do not yet have constexpr.

So my (temporary) fix was (in part3):
const intptr_t NULL_WORD = 0;

This is /identical/ to what you propose. I think this is a good 
compromise. However after much effort from my side trying to get this 
solution you did not want my changes to NULL (and maybe I misunderstood 
you regarding NULL_WORD) so I reverted that in part5.

I am very much confused to what you want. Should I once again revert 
NULL_WORD and put it in globalDefinitions.hpp?

> 
> I wouldn't worry about this comment in globalDefinitions_solstudio.hpp:
> 
>    "On some platforms, sizeof(intptr_t) > sizeof(void*)"
> 
> or you can be really paranoid and STATIC_ASSERT that they are the same
> size. Though I really wonder what platform(s) that's talking about.
> That comment comes from 6452081: 3/4 Allow for Linux builds with Sun
> Studio Linux compilers.  So maybe 32bit Solaris Studio Linux is weird?

I did this originally (disregard the typo of uint vs int) in part1.

STATIC_ASSERT(sizeof(uintptr_t) == sizeof(char *)); // uintptr_t is only 
guaranteed to be at /least/ as big as a pointer

I later reverted this change as I thought you dismissed all of my NULL 
and NULL_WORD definitions. I really wanted this.

> 
> ------------------------------------------------------------------------------
> 
> There is a pattern to some of the changes where I'd prefer something
> different.
> 
> We have classes with a pointer member that is initialized to NULL when
> default constructed, and initialized from the argument by an implicit
> conversion constructor with an appropriately typed pointer parameter.
> Clearly this pattern permits implicit conversion construction from
> NULL, but the proposed changes suggest one should avoid doing so. But
> nothing is being done to prevent it.
> 
> Implicit conversion constructors, like implicit conversion operators,
> can be problematic. I think a better change for these would be to make
> the conversion from pointer constructor explict and use default
> construction for the NULL pointer case. But whether a change from
> implicit to explicit conversion is desirable might not be obvious in
> all cases, and ought to be discussed with people who work with the
> relevant parts of the code a lot.
> 
> Rather than repeat all that, I've used the shorthand "PATTERN CHANGE"
> to indicate these in the comments below.
> 
> ------------------------------------------------------------------------------
> src/hotspot/cpu/sparc/c1_CodeStubs_sparc.cpp
> 303     AddressLiteral addrlit(NULL_WORD, metadata_Relocation::spec(_index));
> 318     AddressLiteral addrlit(NULL_WORD, oop_Relocation::spec(_index));
> 
> I think these should be left as NULL rather than being changed to
> NULL_WORD.  The expected type is the address type, and other types are
> converted to that via casts in constructor overloads.
> 
> Similarly here:
> src/hotspot/cpu/sparc/c1_LIRAssembler_sparc.cpp
>   426   AddressLiteral addrlit(NULL_WORD, oop_Relocation::spec(oop_index));
>   445   AddressLiteral addrlit(NULL_WORD, metadata_Relocation::spec(index));
> 

I believe those do not at all auto convert, but promote to (integral) 
zero as I (quite thoroughly) described in my reply to David (and thus 
/should/ be NULL_WORD. I believe this is one prime example when our NULL 
definitions results in unintuitive selection of integer constructor.

> ------------------------------------------------------------------------------
> src/hotspot/os/linux/cgroupV1Subsystem_linux.hpp
> [old code]
> 100     CachingCgroupController* _memory = NULL;
> 101     CgroupV1Controller* _cpuset = NULL;
> 102     CachingCgroupController* _cpu = NULL;
> 103     CgroupV1Controller* _cpuacct = NULL;
> 
> How does this even compile?  Non-static member initializers are a
> C++11 feature.  Weirdly, g++ with -std=gnu++98 warns about this for
> some types/initializers but not for others.

As I remarked in my original mail, I believe it to be a gcc bug, but I 
am not sure.

> 
> The proposed change is better, even with C++11.
> 
> Similarly here:
> src/hotspot/os/linux/cgroupV2Subsystem_linux.hpp
> 
> ------------------------------------------------------------------------------
> src/hotspot/os/solaris/attachListener_solaris.cpp
> 461   status = ::sema_init(&_wakeup, 0, NULL, NULL);
> =>
> 461   status = ::sema_init(&_wakeup, 0, 0, NULL);
> 
> The old code appears to have been cribbed from the documentation:
> https://docs.oracle.com/cd/E36784_01/html/E36874/sema-init-3c.html
> See the example of "Default semaphore initialization (intra-process)".
> 
> Rather than a literal 0, better would be USYNC_THREAD.  Presumably the
> value of USYNC_THREAD is 0.

I could not easily find the value of the symbols, so I only changed it 
to the correct type, with resent changes to Java regarding Solaris do 
you care enough for me to check this out, or is 0 good enough?

> 
> ------------------------------------------------------------------------------
> src/hotspot/os/windows/os_windows.cpp
> 1659   if (!GetFileVersionInfo(kernel32_path, 0, version_size, version_info)) {
> 
> Given that the second argument, which was changed from NULL to 0, is a
> "handle" (though unused), I think using NULL was appropriate and this
> change shouldn't be made.

I have no windows machine, so I can not check this out 100% at the 
moment, but according to the compile error I got, and the documentation 
of GetFileVersionInfoA or GetFileVersionInfoW I find on the net:
https://docs.microsoft.com/en-us/windows/win32/api/winver/nf-winver-getfileversioninfoa

The type of dwHandle is DWORD. Defined as:
https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-dtyp/262627d8-3418-4627-9218-4ffe110850b2 


What makes you think the handle is of pointer type, to me it looks like 
a long?

> 
> ------------------------------------------------------------------------------
> src/hotspot/share/classfile/javaClasses.cpp
>   851       k->set_java_mirror_handle(NULL);
> =>
>   851       k->set_java_mirror_handle(OopHandle(NULL));
> 
> PATTERN CHANGE
> 
> Similarly here:
> src/hotspot/share/classfile/moduleEntry.hpp
>   82     _module = OopHandle(NULL);
>   84     _pd = OopHandle(NULL); >
> And here:
> src/hotspot/share/classfile/modules.cpp
> 156     version_symbol = TempNewSymbol(NULL); > 161   TempNewSymbol location_symbol(NULL);
> 368     version_symbol = TempNewSymbol(NULL);
> 373   TempNewSymbol location_symbol(NULL);
> 
> Although I'm not sure why the assignments on line 156 and line 368 is
> even being done, since each version_symbol was already default
> initialized.
Probably to /show/ that it has an initialized value (I do not 
necessarily agree that it is good). Do you want me to change any of 
those, and if so to what?

> ------------------------------------------------------------------------------
> src/hotspot/share/code/codeHeapState.cpp
>   140     if (((_termString) != 0) && (strlen(_termString) > 0)){   \
>   154     if (((_termString) != 0) && (strlen(_termString) > 0)){   \
> 
> I think these changes to compare _termstring with 0 rather than NULL
> should not be made.  _termString is supposed to be a `[const] char*`,
> so comparing to NULL is preferable to comparing to 0.

_termString is not a pointer (or at least not most of the time), see my 
thorough discussion of the topic with David. I believe my solution is 
the next safest one (after rewriting that macro nest which I am not 
going to do now).

> 
> Though at a brief skim it looks like _termString is always provided
> and never NULL, so it might be better to assert that _termString is
> non-NULL.  But that's a completely different change.

I believe we should use less macros, and I agree that is different change.

> ------------------------------------------------------------------------------
> src/hotspot/share/gc/parallel/psAdaptiveSizePolicy.cpp
> src/hotspot/share/gc/parallel/psParallelCompact.cpp
> src/hotspot/share/gc/shared/memset_with_concurrent_readers.hpp
> src/hotspot/share/gc/z/zErrno.cpp
> 
> These #include changes seem unrelated to NULL.

They are all (unnecessary) including standard headers, It is hard to 
know which headers that (indirectly) redefines NULL. It is better to not 
include them if you do not have to. If I remember correctly I removed 
all but one standard library include in the "gc" folder. The one I left 
was for <new>.

> 
> ------------------------------------------------------------------------------
> src/hotspot/share/jfr/jni/jfrGetAllEventClasses.cpp
> 172   assert(add_method_signature != NULL, "invariant");
> =>
> 172   assert(add_method_sig_sym != NULL, "invariant");
> 
> This seems to be an unrelated bug fix; the wrong value was being
> compared to NULL.

It is highly related as it did not compile on my original definition of 
NULL and was thus found.

> ------------------------------------------------------------------------------
> src/hotspot/share/jfr/jni/jfrJavaCall.cpp
>   57   assert(_storage != NULL, "invariant");
> =>
>   57   assert(_storage_index < (int) ARRAY_SIZE(_storage), "invariant");
> 
> _storage is an array, so the original assert is definitionally true,
> but it's harmless to verify, in case future maintenance makes it an
> allocated array rather than inline.

It is so statically true that it should not be a runtime check.

My replacement check is not only implicitly checking that the variable 
is not NULL, it does so at compile time (ARRAY_SIZE will fail on NULL) 
and it does an even better runtime check that actually adds value.
> 
> The size of _storage is definitionally SIZE, and _storage_index is
> checked against SIZE a couple lines later, making the proposed
> replacement just a duplicative check.

I have no problem with duplicate checks, but if anything should be 
removed, it is the SIZE assert. My assert clearly shows that the access 
is okay even though someone would change the size of the array to not be 
of size SIZE. My assert can be understood without looking outside the 
method.

> So I think the proposed change should not be made.
> 
> Similarly at lines 65, 73, and 91.
> 
I strongly disagree. Each of my replacement asserts is strictly superior 
if I am not missing something. Not only that, but they might not compile 
on windows in the future depending on how we define NULL in the future.

Most of the changes I have made are because of compile errors after my 
new NULL, not "just" because of cosmetic issues.

> ------------------------------------------------------------------------------
> src/hotspot/share/jvmci/jvmciCompilerToVM.cpp
> 1562   JVMCIENV->set_HotSpotStackFrameReference_localIsVirtual(hs_frame, 0);
> 
> This change to use literal 0 rather than NULL as a null pointer
> constant does not seem like an improvement to me.  I think this change
> should not be made.
> 
> ------------------------------------------------------------------------------
> src/hotspot/share/jvmci/jvmciCompilerToVM.cpp
> 
> I looked at the changes from JVMCI_CHECK_0 to JVMCI_CHECK_NULL and
> think they are correct, assuming a suitable definition of the latter.
> (Though see below regarding that.)
> 
> ------------------------------------------------------------------------------
> src/hotspot/share/jvmci/jvmciCompilerToVM.cpp
> 2572   JVMCIObjectArray current_array = NULL;
> =>
> 2572   JVMCIObjectArray current_array(NULL);
> 
> PATTERN CHANGE
> 
> ------------------------------------------------------------------------------
> src/hotspot/share/jvmci/jvmciCompilerToVMInit.cpp
> 267     if ((value) != 0) { \
> 
> Value is a char*; I don't think this change should be made.
> 
> ------------------------------------------------------------------------------
> src/hotspot/share/jvmci/jvmciExceptions.hpp
> 50 #define JVMCI_CHECK_NULL                               JVMCI_CHECK_(NULL)
> =>
> 50 #define JVMCI_CHECK_NULL                               JVMCI_CHECK_(0)
> 
> For platforms that warn about converting NULL to an integral value
> this is a type safety regression. I think this change should not be
> made.
> 
> ------------------------------------------------------------------------------
> src/hotspot/share/jvmci/jvmciExceptions.hpp
> 49 #define JVMCI_CHECK_0                                  JVMCI_CHECK_(0)
> =>
> 49 #define JVMCI_CHECK_0                                  JVMCI_CHECK_(NULL_WORD)
> 
> I kind of think this is an abuse of NULL_WORD. It also serves no real
> purpose with the current definitions of NULL_WORD (though would with
> the suggestion far above). If the goal is to have something that isn't
> a null pointer constant so it can't be used in inappropriate
> contexts (which is a reasonable goal), then replace "0" with "int(0)"
> (and comment appropriately).

Please read my conversation with David and see if you after that still 
have any issue with what I am proposing.

> ------------------------------------------------------------------------------
> src/hotspot/share/oops/access.hpp
> 296 class ArrayAccess ...
> 
> How about adding the following private functions to this class, and
> use them in the obvious places:
> 
>    template<typename T> static const T* const_raw_null() { return NULL; }
>    template<typename T> static       T*       raw_null() { return NULL; }

I am lacking c++ and template skills, but the problem here is that we 
need to make the type of the null pointer concrete for the template 
system. I fail to see how these template methods achieve this from 
static context and no parameters. The code fails to compile for me, and 
I am probably missing something, but I fail to see the idea behind you 
construction. The only type parameter is the return one, and that is the 
one we do not know, right?

As another solution we could specialize the type explicitly at the 
call-site instead of in the delivered parameters (I tried that first), 
but that was IMO /much/ harder to read.

Even if raw_null() does work (assuming failure from my side), the 
original is cleaner IMO.

> 
> ------------------------------------------------------------------------------
> src/hotspot/share/oops/constantPool.cpp
>   270     set_resolved_references(OopHandle(NULL));
>   302     set_resolved_references(OopHandle(NULL));
>   376     set_resolved_references(OopHandle(NULL));
> 
> PATTERN CHANGE
> 
> Similarly here:
> src/hotspot/share/oops/cpCache.cpp
> 766   set_resolved_references(OopHandle(NULL));
> 
> src/hotspot/share/oops/klass.cpp
> 549   _java_mirror = OopHandle(NULL);
> 
> src/hotspot/share/prims/jni.cpp
>   319   TempNewSymbol class_name(NULL);
> 
> src/hotspot/share/prims/jvm.cpp
>   954   TempNewSymbol class_name(NULL);
> 
> src/hotspot/share/prims/methodHandles.cpp
> 1311   TempNewSymbol name(NULL);
> 1312   TempNewSymbol sig(NULL);

I understand you have no problem with those changes, is that correct?

> 
> ------------------------------------------------------------------------------
> src/hotspot/share/runtime/os.cpp
> 1324   if (path == NULL || strlen(path) == 0 || file_name_length == 0) {
> 
> `(size_t)NULL` ??  Good catch!
> 
> ------------------------------------------------------------------------------
> 

Thanks Kim!

/Leo

From leo.korinth at oracle.com  Sun Apr 19 17:08:09 2020
From: leo.korinth at oracle.com (Leo Korinth)
Date: Sun, 19 Apr 2020 19:08:09 +0200
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <26105810-b9be-faec-6577-616ef26e84a7@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
 <135a6496-5b54-9117-5cbf-e3211ddc32dd@oracle.com>
 <bbb2b5ef-62f2-6507-95f7-03712a382153@oracle.com>
 <e242e057-de6e-493a-eb00-068b2db8c9fa@oracle.com>
 <df621fd2-0eaf-cb24-788f-12166b271303@oracle.com>
 <26105810-b9be-faec-6577-616ef26e84a7@oracle.com>
Message-ID: <fa000582-255b-f072-99bd-765d587b4a26@oracle.com>



On 17/04/2020 08:45, David Holmes wrote:
> Hi Leo,
> 
> I've taken a look at the complete set of changes and the only thing I 
> strongly object to is the change to:
> 
> src/hotspot/share/jvmci/jvmciExceptions.hpp
> 
> as it just doesn't make any sense to me. If you were to write the 
> expanded macros out in full you would write the code as it currently is 
> defined in the macro. If a method returns a pointer you would write:
> 
> return NULL;
> 
> If a method returns a jint/jbyte/jlong etc you would write
> 
> return 0;
> 
> Cheers,
> David

I totally agree with you and the change needed was smaller than I 
rememberd, I should have fixed this directly. Returning NULL for an 
object that is not of pointer type seems just wrong, but returning a 
JVMCIObjectArray(NULL) seems okay.

What about this?

#define JVMCI_CHECK_0             JVMCI_CHECK_(NULL_WORD)
#define JVMCI_CHECK_NULL          JVMCI_CHECK_(NULL)

and 3 changes of JVMCI_CHECK_NULL to JVMCI_CHECK_(JVMCIObjectArray(NULL))

(this change is so that we get one conversion less)

Thanks,
Leo



From david.holmes at oracle.com  Sun Apr 19 22:13:37 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 20 Apr 2020 08:13:37 +1000
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <fa000582-255b-f072-99bd-765d587b4a26@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
 <135a6496-5b54-9117-5cbf-e3211ddc32dd@oracle.com>
 <bbb2b5ef-62f2-6507-95f7-03712a382153@oracle.com>
 <e242e057-de6e-493a-eb00-068b2db8c9fa@oracle.com>
 <df621fd2-0eaf-cb24-788f-12166b271303@oracle.com>
 <26105810-b9be-faec-6577-616ef26e84a7@oracle.com>
 <fa000582-255b-f072-99bd-765d587b4a26@oracle.com>
Message-ID: <087a3bfa-b4ba-68d2-2509-7408e13bac1e@oracle.com>

On 20/04/2020 3:08 am, Leo Korinth wrote:
> 
> 
> On 17/04/2020 08:45, David Holmes wrote:
>> Hi Leo,
>>
>> I've taken a look at the complete set of changes and the only thing I 
>> strongly object to is the change to:
>>
>> src/hotspot/share/jvmci/jvmciExceptions.hpp
>>
>> as it just doesn't make any sense to me. If you were to write the 
>> expanded macros out in full you would write the code as it currently 
>> is defined in the macro. If a method returns a pointer you would write:
>>
>> return NULL;
>>
>> If a method returns a jint/jbyte/jlong etc you would write
>>
>> return 0;
>>
>> Cheers,
>> David
> 
> I totally agree with you and the change needed was smaller than I 
> rememberd, I should have fixed this directly. Returning NULL for an 
> object that is not of pointer type seems just wrong, but returning a 
> JVMCIObjectArray(NULL) seems okay.
> 
> What about this?
> 
> #define JVMCI_CHECK_0???????????? JVMCI_CHECK_(NULL_WORD)

Why does this integer returning macro need to return NULL_WORD? As I 
said above if writing this directly I would write "return 0;" in the 
code. Is this just an attempt under the covers to (eventually) avoid 
misusing JVMCI_CHECK_0 where JVMCI_CHECK_NULL should have been used?

> #define JVMCI_CHECK_NULL????????? JVMCI_CHECK_(NULL)
> 
> and 3 changes of JVMCI_CHECK_NULL to JVMCI_CHECK_(JVMCIObjectArray(NULL))

I'm not sure specifically where those changes are.

In general I think I prefer to see an explicit constructor passed NULL 
rather than some implicit conversion, but that is a stylistic matter and 
the implicit conversion form seems to be used quite extensively.

Thanks,
David

> (this change is so that we get one conversion less)
> 
> Thanks,
> Leo
> 
> 

From kim.barrett at oracle.com  Mon Apr 20 00:58:47 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Sun, 19 Apr 2020 20:58:47 -0400
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <087a3bfa-b4ba-68d2-2509-7408e13bac1e@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
 <135a6496-5b54-9117-5cbf-e3211ddc32dd@oracle.com>
 <bbb2b5ef-62f2-6507-95f7-03712a382153@oracle.com>
 <e242e057-de6e-493a-eb00-068b2db8c9fa@oracle.com>
 <df621fd2-0eaf-cb24-788f-12166b271303@oracle.com>
 <26105810-b9be-faec-6577-616ef26e84a7@oracle.com>
 <fa000582-255b-f072-99bd-765d587b4a26@oracle.com>
 <087a3bfa-b4ba-68d2-2509-7408e13bac1e@oracle.com>
Message-ID: <19C3FFC0-B0DE-4A19-87A4-0A071BDC64C0@oracle.com>

> On Apr 19, 2020, at 6:13 PM, David Holmes <david.holmes at oracle.com> wrote:
> 
> On 20/04/2020 3:08 am, Leo Korinth wrote:
>> On 17/04/2020 08:45, David Holmes wrote:
>>> Hi Leo,
>>> 
>>> I've taken a look at the complete set of changes and the only thing I strongly object to is the change to:
>>> 
>>> src/hotspot/share/jvmci/jvmciExceptions.hpp
>>> 
>>> as it just doesn't make any sense to me. If you were to write the expanded macros out in full you would write the code as it currently is defined in the macro. If a method returns a pointer you would write:
>>> 
>>> return NULL;
>>> 
>>> If a method returns a jint/jbyte/jlong etc you would write
>>> 
>>> return 0;
>>> 
>>> Cheers,
>>> David
>> I totally agree with you and the change needed was smaller than I rememberd, I should have fixed this directly. Returning NULL for an object that is not of pointer type seems just wrong, but returning a JVMCIObjectArray(NULL) seems okay.
>> What about this?
>> #define JVMCI_CHECK_0             JVMCI_CHECK_(NULL_WORD)
> 
> Why does this integer returning macro need to return NULL_WORD? As I said above if writing this directly I would write "return 0;" in the code. Is this just an attempt under the covers to (eventually) avoid misusing JVMCI_CHECK_0 where JVMCI_CHECK_NULL should have been used?

I agree that NULL_WORD has the wrong implication here.

As you correctly surmise, the problem with returning `0` is that it is
a null pointer constant. It would be nice if JVMCI_CHECK_0 could not
be used in a context where a pointer result is expected, and instead
require JVMCI_CHECK_NULL in such cases. That was the point of my
suggestion of using

#define JVMCI_CHECK_0  JVMCI_CHECK_(int(0))

The expression `int(0)` is not a null pointer constant, so won't
implicitly convert to a null pointer.

I realized later that there's a (currently theoretical?) problem with
this; if the result type is an unsigned integral value then this may
(someday?) produce a signed to unsigned conversion warning. (The
proposed use of NULL_WORD has the same problem.) I think there's not a
way to eliminate that possibility prior to C++11, other than by
introducing and using JVMCI_CHECK_0U to return an unsigned zero value.
But since I think that problem is currently theoretical, and can be
solved with C++11, I'm not concerned.

>> #define JVMCI_CHECK_NULL          JVMCI_CHECK_(NULL)
>> and 3 changes of JVMCI_CHECK_NULL to JVMCI_CHECK_(JVMCIObjectArray(NULL))
> 
> I'm not sure specifically where those changes are.
> 
> In general I think I prefer to see an explicit constructor passed NULL rather than some implicit conversion, but that is a stylistic matter and the implicit conversion form seems to be used quite extensively.

The question is, given class Foo, which has a conversion from pointer
constructor and is default constructable, which is preferable:

Foo foo1 = NULL;
Foo foo2(NULL);
Foo foo3;

and similarly, with functions taking a Foo:

function1(NULL)
function2(Foo(NULL)) 
function3(Foo()) 

We have occurrences of both variant #1 usages. The proposal is to
instead use the variant #2 style. My own preference would be for the
variant #3 style. Both #2 and #3 should also declare the conversion
constructor "explicit" to actively prevent variant #1; that's missing
from the current proposal.



From david.holmes at oracle.com  Mon Apr 20 02:35:34 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 20 Apr 2020 12:35:34 +1000
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <19C3FFC0-B0DE-4A19-87A4-0A071BDC64C0@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
 <135a6496-5b54-9117-5cbf-e3211ddc32dd@oracle.com>
 <bbb2b5ef-62f2-6507-95f7-03712a382153@oracle.com>
 <e242e057-de6e-493a-eb00-068b2db8c9fa@oracle.com>
 <df621fd2-0eaf-cb24-788f-12166b271303@oracle.com>
 <26105810-b9be-faec-6577-616ef26e84a7@oracle.com>
 <fa000582-255b-f072-99bd-765d587b4a26@oracle.com>
 <087a3bfa-b4ba-68d2-2509-7408e13bac1e@oracle.com>
 <19C3FFC0-B0DE-4A19-87A4-0A071BDC64C0@oracle.com>
Message-ID: <5d686fe6-661b-4636-a71d-d379ef0324ed@oracle.com>

Hi Kim,

On 20/04/2020 10:58 am, Kim Barrett wrote:
>> On Apr 19, 2020, at 6:13 PM, David Holmes <david.holmes at oracle.com> wrote:
>>
>> On 20/04/2020 3:08 am, Leo Korinth wrote:
>>> On 17/04/2020 08:45, David Holmes wrote:
>>>> Hi Leo,
>>>>
>>>> I've taken a look at the complete set of changes and the only thing I strongly object to is the change to:
>>>>
>>>> src/hotspot/share/jvmci/jvmciExceptions.hpp
>>>>
>>>> as it just doesn't make any sense to me. If you were to write the expanded macros out in full you would write the code as it currently is defined in the macro. If a method returns a pointer you would write:
>>>>
>>>> return NULL;
>>>>
>>>> If a method returns a jint/jbyte/jlong etc you would write
>>>>
>>>> return 0;
>>>>
>>>> Cheers,
>>>> David
>>> I totally agree with you and the change needed was smaller than I rememberd, I should have fixed this directly. Returning NULL for an object that is not of pointer type seems just wrong, but returning a JVMCIObjectArray(NULL) seems okay.
>>> What about this?
>>> #define JVMCI_CHECK_0             JVMCI_CHECK_(NULL_WORD)
>>
>> Why does this integer returning macro need to return NULL_WORD? As I said above if writing this directly I would write "return 0;" in the code. Is this just an attempt under the covers to (eventually) avoid misusing JVMCI_CHECK_0 where JVMCI_CHECK_NULL should have been used?
> 
> I agree that NULL_WORD has the wrong implication here.
> 
> As you correctly surmise, the problem with returning `0` is that it is
> a null pointer constant. It would be nice if JVMCI_CHECK_0 could not
> be used in a context where a pointer result is expected, and instead
> require JVMCI_CHECK_NULL in such cases. That was the point of my
> suggestion of using
> 
> #define JVMCI_CHECK_0  JVMCI_CHECK_(int(0))
> 
> The expression `int(0)` is not a null pointer constant, so won't
> implicitly convert to a null pointer.

I'm sure I tried something like that when fixing up some misuses of the 
more general CHECK_0 macro, but it did not help in detecting the misuse.

Cheers,
David
-----

> I realized later that there's a (currently theoretical?) problem with
> this; if the result type is an unsigned integral value then this may
> (someday?) produce a signed to unsigned conversion warning. (The
> proposed use of NULL_WORD has the same problem.) I think there's not a
> way to eliminate that possibility prior to C++11, other than by
> introducing and using JVMCI_CHECK_0U to return an unsigned zero value.
> But since I think that problem is currently theoretical, and can be
> solved with C++11, I'm not concerned.
> 
>>> #define JVMCI_CHECK_NULL          JVMCI_CHECK_(NULL)
>>> and 3 changes of JVMCI_CHECK_NULL to JVMCI_CHECK_(JVMCIObjectArray(NULL))
>>
>> I'm not sure specifically where those changes are.
>>
>> In general I think I prefer to see an explicit constructor passed NULL rather than some implicit conversion, but that is a stylistic matter and the implicit conversion form seems to be used quite extensively.
> 
> The question is, given class Foo, which has a conversion from pointer
> constructor and is default constructable, which is preferable:
> 
> Foo foo1 = NULL;
> Foo foo2(NULL);
> Foo foo3;
> 
> and similarly, with functions taking a Foo:
> 
> function1(NULL)
> function2(Foo(NULL))
> function3(Foo())
> 
> We have occurrences of both variant #1 usages. The proposal is to
> instead use the variant #2 style. My own preference would be for the
> variant #3 style. Both #2 and #3 should also declare the conversion
> constructor "explicit" to actively prevent variant #1; that's missing
> from the current proposal.
> 
> 

From magnus.ihse.bursie at oracle.com  Mon Apr 20 09:20:58 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Mon, 20 Apr 2020 11:20:58 +0200
Subject: RFR: JDK-8243154 Fix deprecation warnings in failure handler
Message-ID: <25c29d2f-6b71-04cd-9891-3d78b690920e@oracle.com>

The failure handler has two places where deprecated methods are used. 
This results in warnings that are not possible to suppress. This patch 
fixes the code in those places.

This patch is part of the ongoing effort to make the entire build free 
of warnings.

Bug: https://bugs.openjdk.java.net/browse/JDK-8243154
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8243154-fix-failurehandler-deprecations/webrev.01

/Magnus

From kim.barrett at oracle.com  Mon Apr 20 09:23:53 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Mon, 20 Apr 2020 05:23:53 -0400
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <5d686fe6-661b-4636-a71d-d379ef0324ed@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
 <135a6496-5b54-9117-5cbf-e3211ddc32dd@oracle.com>
 <bbb2b5ef-62f2-6507-95f7-03712a382153@oracle.com>
 <e242e057-de6e-493a-eb00-068b2db8c9fa@oracle.com>
 <df621fd2-0eaf-cb24-788f-12166b271303@oracle.com>
 <26105810-b9be-faec-6577-616ef26e84a7@oracle.com>
 <fa000582-255b-f072-99bd-765d587b4a26@oracle.com>
 <087a3bfa-b4ba-68d2-2509-7408e13bac1e@oracle.com>
 <19C3FFC0-B0DE-4A19-87A4-0A071BDC64C0@oracle.com>
 <5d686fe6-661b-4636-a71d-d379ef0324ed@oracle.com>
Message-ID: <DC61E51D-C893-4E41-8F95-8396688D2C70@oracle.com>

> On Apr 19, 2020, at 10:35 PM, David Holmes <david.holmes at oracle.com> wrote:
> On 20/04/2020 10:58 am, Kim Barrett wrote:
>>> On Apr 19, 2020, at 6:13 PM, David Holmes <david.holmes at oracle.com> wrote:
>>> 
>>> On 20/04/2020 3:08 am, Leo Korinth wrote:
>>>> What about this?
>>>> #define JVMCI_CHECK_0             JVMCI_CHECK_(NULL_WORD)
>>> 
>>> Why does this integer returning macro need to return NULL_WORD? As I said above if writing this directly I would write "return 0;" in the code. Is this just an attempt under the covers to (eventually) avoid misusing JVMCI_CHECK_0 where JVMCI_CHECK_NULL should have been used?
>> I agree that NULL_WORD has the wrong implication here.
>> As you correctly surmise, the problem with returning `0` is that it is
>> a null pointer constant. It would be nice if JVMCI_CHECK_0 could not
>> be used in a context where a pointer result is expected, and instead
>> require JVMCI_CHECK_NULL in such cases. That was the point of my
>> suggestion of using
>> #define JVMCI_CHECK_0  JVMCI_CHECK_(int(0))
>> The expression `int(0)` is not a null pointer constant, so won't
>> implicitly convert to a null pointer.
> 
> I'm sure I tried something like that when fixing up some misuses of the more general CHECK_0 macro, but it did not help in detecting the misuse.

Drat, you are right; that doesn't have the desired effect, at least
with gcc.

I hadn't noticed some of the evolution of the definition of "null
pointer constant", and have mostly been looking at C++14 for reference
lately. The definition (4.10 in all three below) is:

C++98: an integral constant expression (5.19) rvalue of integer type
that evaluates to zero.

C++11: an integral constant expression (5.19) prvalue of integer type
that evaluates to zero or a prvalue of type std::nullptr_t.

C++14: an integer literal (2.14.2) with value zero or a prvalue of
type std::nullptr_t.

I've been assuming the literal 0 definition. And even so, it seems
that gcc (at least) is using the C++11 definition with -std=c++14. I
don't know what clang or Visual Studio do in C++14 mode.

Rather than using `int(0)`, I think the following would achieve the
goal of preventing JVMCI_CHECK_0 from being used in a pointer context:

inline int JVMCI_CHECK_0_value() { return 0; } // Private helper
#define JVMCI_CHECK_0  JVMCI_CHECK_(JVMCI_CHECK_0_value())

Is that worth the trouble?


From david.holmes at oracle.com  Mon Apr 20 10:09:11 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 20 Apr 2020 20:09:11 +1000
Subject: RFR: JDK-8243154 Fix deprecation warnings in failure handler
In-Reply-To: <25c29d2f-6b71-04cd-9891-3d78b690920e@oracle.com>
References: <25c29d2f-6b71-04cd-9891-3d78b690920e@oracle.com>
Message-ID: <f1514873-6bda-a673-cb65-3e7287002657@oracle.com>

Hi Magnus,

LGTM!

Thanks,
David

On 20/04/2020 7:20 pm, Magnus Ihse Bursie wrote:
> The failure handler has two places where deprecated methods are used. 
> This results in warnings that are not possible to suppress. This patch 
> fixes the code in those places.
> 
> This patch is part of the ongoing effort to make the entire build free 
> of warnings.
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8243154
> WebRev: 
> http://cr.openjdk.java.net/~ihse/JDK-8243154-fix-failurehandler-deprecations/webrev.01 
> 
> 
> /Magnus

From leo.korinth at oracle.com  Mon Apr 20 12:14:19 2020
From: leo.korinth at oracle.com (Leo Korinth)
Date: Mon, 20 Apr 2020 14:14:19 +0200
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <DC61E51D-C893-4E41-8F95-8396688D2C70@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
 <135a6496-5b54-9117-5cbf-e3211ddc32dd@oracle.com>
 <bbb2b5ef-62f2-6507-95f7-03712a382153@oracle.com>
 <e242e057-de6e-493a-eb00-068b2db8c9fa@oracle.com>
 <df621fd2-0eaf-cb24-788f-12166b271303@oracle.com>
 <26105810-b9be-faec-6577-616ef26e84a7@oracle.com>
 <fa000582-255b-f072-99bd-765d587b4a26@oracle.com>
 <087a3bfa-b4ba-68d2-2509-7408e13bac1e@oracle.com>
 <19C3FFC0-B0DE-4A19-87A4-0A071BDC64C0@oracle.com>
 <5d686fe6-661b-4636-a71d-d379ef0324ed@oracle.com>
 <DC61E51D-C893-4E41-8F95-8396688D2C70@oracle.com>
Message-ID: <cda962a7-1821-ed4b-ea4c-fbf43d47f9f5@oracle.com>

On 20/04/2020 11:23, Kim Barrett wrote:
>> On Apr 19, 2020, at 10:35 PM, David Holmes <david.holmes at oracle.com> wrote:
>> On 20/04/2020 10:58 am, Kim Barrett wrote:
>>>> On Apr 19, 2020, at 6:13 PM, David Holmes <david.holmes at oracle.com> wrote:
>>>>
>>>> On 20/04/2020 3:08 am, Leo Korinth wrote:
>>>>> What about this?
>>>>> #define JVMCI_CHECK_0             JVMCI_CHECK_(NULL_WORD)
>>>>
>>>> Why does this integer returning macro need to return NULL_WORD? As I said above if writing this directly I would write "return 0;" in the code. Is this just an attempt under the covers to (eventually) avoid misusing JVMCI_CHECK_0 where JVMCI_CHECK_NULL should have been used?
>>> I agree that NULL_WORD has the wrong implication here.
>>> As you correctly surmise, the problem with returning `0` is that it is
>>> a null pointer constant. It would be nice if JVMCI_CHECK_0 could not
>>> be used in a context where a pointer result is expected, and instead
>>> require JVMCI_CHECK_NULL in such cases. That was the point of my
>>> suggestion of using
>>> #define JVMCI_CHECK_0  JVMCI_CHECK_(int(0))
>>> The expression `int(0)` is not a null pointer constant, so won't
>>> implicitly convert to a null pointer.
>>
>> I'm sure I tried something like that when fixing up some misuses of the more general CHECK_0 macro, but it did not help in detecting the misuse.
> 
> Drat, you are right; that doesn't have the desired effect, at least
> with gcc.
> 
> I hadn't noticed some of the evolution of the definition of "null
> pointer constant", and have mostly been looking at C++14 for reference
> lately. The definition (4.10 in all three below) is:
> 
> C++98: an integral constant expression (5.19) rvalue of integer type
> that evaluates to zero.
> 
> C++11: an integral constant expression (5.19) prvalue of integer type
> that evaluates to zero or a prvalue of type std::nullptr_t.
> 
> C++14: an integer literal (2.14.2) with value zero or a prvalue of
> type std::nullptr_t.
> 
> I've been assuming the literal 0 definition. And even so, it seems
> that gcc (at least) is using the C++11 definition with -std=c++14. I
> don't know what clang or Visual Studio do in C++14 mode.
> 
> Rather than using `int(0)`, I think the following would achieve the
> goal of preventing JVMCI_CHECK_0 from being used in a pointer context:
> 
> inline int JVMCI_CHECK_0_value() { return 0; } // Private helper
> #define JVMCI_CHECK_0  JVMCI_CHECK_(JVMCI_CHECK_0_value())
> 
> Is that worth the trouble?
> 

I do not think so. I dislike these macros and i just wanted to make some 
trivial improvements. If we can not agree, I propose we leave the macros 
as they were.

New webrev including copyright year fix proposed by Magnus:
http://cr.openjdk.java.net/~lkorinth/8240110/part6

I keep the old definitions of JVMCI_CHECK_0 although I can not 
understand the dislike to use NULL_WORD.

/Leo






From erik.joelsson at oracle.com  Mon Apr 20 12:54:48 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Mon, 20 Apr 2020 05:54:48 -0700
Subject: RFR: JDK-8243154 Fix deprecation warnings in failure handler
In-Reply-To: <25c29d2f-6b71-04cd-9891-3d78b690920e@oracle.com>
References: <25c29d2f-6b71-04cd-9891-3d78b690920e@oracle.com>
Message-ID: <00be6793-5841-3217-4368-be5a58ef3082@oracle.com>

Looks good.

/Erik

On 2020-04-20 02:20, Magnus Ihse Bursie wrote:
> The failure handler has two places where deprecated methods are used. 
> This results in warnings that are not possible to suppress. This patch 
> fixes the code in those places.
>
> This patch is part of the ongoing effort to make the entire build free 
> of warnings.
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8243154
> WebRev: 
> http://cr.openjdk.java.net/~ihse/JDK-8243154-fix-failurehandler-deprecations/webrev.01
>
> /Magnus

From forax at univ-mlv.fr  Mon Apr 20 13:31:55 2020
From: forax at univ-mlv.fr (Remi Forax)
Date: Mon, 20 Apr 2020 15:31:55 +0200 (CEST)
Subject: RFR JDK-8223347 Integration of Vector API (Incubator): Java
 API, implementation, and tests
In-Reply-To: <49F81016-55DF-4E3C-940A-A8DF8C712259@oracle.com>
References: <49F81016-55DF-4E3C-940A-A8DF8C712259@oracle.com>
Message-ID: <1886691513.405349.1587389515699.JavaMail.zimbra@u-pem.fr>

Hi Paul,
about the API part (not the implementation),
there are location where the same concept is described with a different names which doesn't help to understand how thing work
- vector length <=> vector lane count
- vector shape <=> vector bits size
- element size <=> lane size

"size" should never be used (VectorSpecies.elementSize() by example ) because it's never clear if it's the byte size or the bits size, using byteSize and bitsSize as used several times in the API should be used everywhere.

Initially, having VectorSpecies and VectorShape confuse me, it's not the shape of the vector but only it's bits size,
renaming VectorShape to VectorBitsSize will help, i think. 

You have the same values defined at several places in the API (sometimes with different names), usually it's aliases but it makes the API bigger that it should.
I think all the reference to the element type, vector byte size, vector bit size, element byte size, element byte size, vector lane count should not appear on Vector because they are accessible from Species,
and to go from a vector to the corresponding species, species() is enough.

You have also the same methods defined at several places, on VectorSpecies and as a static method taking a species as parameter, 
all the methods of VectorSpecies that takes or return a vector/mask/shuffle should be static in vector/mask/shuffle.

I think Binary and Associative should be merged to one VectorOperation, given that the difference is subtle and the whole point of this API is that if the hardware do not provide a way to reduce a binary op, it can be done in Java. 

I have no idea what a Conversion.rangeType() is ?

More specially:
- VectorSpecies.loopbound() is not used in the example ?
- VectorSpecies.arrayType()/genericElementType() are more for implementers than users , no ? at least arrayType should be arrayElementType.
- VectorSpecies.withLanes() => withElementType()

- VectorMask.check* are implementer methods ?
- VectorMask.equal(VectorMask) => equiv

- VectorShuffle.check* are implementer methods ?
- VectorShuffle.vectorSpecies() => VectorShuffle.species()  (as in Vector and VectorMask)
- VectorShuffle.toVector() should return a IntVector
- VectorShuffle.laneIsValid(), it seems to be an operation on Vector, not VectorShuffle.

- IntVector.max, why there is no binary version that takes a mask ? It's documented but not why it's not available.


and i'm also not a big fans of the method that returns a long instead of a vector and only works on 64 bits values.

regards,
R?mi

----- Mail original -----
> De: "Paul Sandoz" <paul.sandoz at oracle.com>
> ?: "core-libs-dev" <core-libs-dev at openjdk.java.net>, "hotspot-dev" <hotspot-dev at openjdk.java.net>
> Envoy?: Jeudi 2 Avril 2020 00:46:55
> Objet: RFR JDK-8223347 Integration of Vector API (Incubator): Java API, implementation, and tests

> Hi,
> 
> A prior email sent out a request for review of the Vector API in preparation for
> JEP 338: Vector API (Incubator) [1] to be proposed for target:
> 
>  https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065345.html
>  <https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065345.html>
> 
> This email expands the review of the API to the Java implementation and Java
> tests:
> 
>  http://cr.openjdk.java.net/~psandoz/panama/JDK-8223347-vector-api-integration-java/jdk_src_webrev/webrev/
>  <http://cr.openjdk.java.net/~psandoz/panama/JDK-8223347-vector-api-integration-java/jdk_src_webrev/webrev/>
> 
>  http://cr.openjdk.java.net/~psandoz/panama/JDK-8223347-vector-api-integration-java/jdk_test_webrev/webrev/
>  <http://cr.openjdk.java.net/~psandoz/panama/JDK-8223347-vector-api-integration-java/jdk_test_webrev/webrev/>
> 
> (Further emails will sent for review of general HotSpot changes CPU architecture
> specific HotSpot changes, x64 and aarch64, and performance tests.  For an early
> peek see the links in the description of JDK-8223347.)
> 
> ?
> 
> The Vector API provides
> 
> - the public Vector class with vector operations common to all supported
> primitive types
> 
> - public primitive specializations, such as IntVector, with common operations
> associated with the primitive type
> 
> - internal concrete specializations for the vector size, such as Int256Vector,
> for a vector holding 8 ints.
> 
> Operations generally defer to one of approximately 20 vector intrinsics.  Some
> operations may be composed of other operations.
> 
> Explicit casts are performed by vector operations to ensure vectors arguments
> are of the required shape (bit size) and element type.
> 
> A vector intrinsic is an internal low-level vector operation. The last argument
> to the intrinsic is fall back behavior in Java, implementing the scalar
> operation over the number of elements held by the vector.  Thus, If the
> intrinsic is not supported in C2 for the other arguments then the Java
> implementation is executed (the Java implementation is always executed when
> running in the interpreter or for C1).
> 
> The public primitive specializations and the internal concrete implementations
> are generated from SSP template files, X-Vector.java.template and
> X-VectorBits.java.template respectively.
> 
> Overall the implementation approach is quite formulaic and rather repetitive.
> Once you grok the pattern It should be easier to review if a little laborious.
> 
> Unit tests are auto-generated by composing templates for each operation into an
> SSP template file which is then used to generate unit test files for each
> concrete vector class.  The tests are quite extensive and have found many a
> HotSpot issue in development across a wide range of platforms.
> 
> Paul.
> 
> [1] https://bugs.openjdk.java.net/browse/JDK-8201271
> <https://bugs.openjdk.java.net/browse/JDK-8201271>

From forax at univ-mlv.fr  Mon Apr 20 14:04:45 2020
From: forax at univ-mlv.fr (Remi Forax)
Date: Mon, 20 Apr 2020 16:04:45 +0200 (CEST)
Subject: RFR JDK-8223347 Integration of Vector API (Incubator): Java
 API, implementation, and tests
In-Reply-To: <49F81016-55DF-4E3C-940A-A8DF8C712259@oracle.com>
References: <49F81016-55DF-4E3C-940A-A8DF8C712259@oracle.com>
Message-ID: <1162249813.435958.1587391485813.JavaMail.zimbra@u-pem.fr>

BTW, did you try to add an intermediary private non visible class between Vector and IntVector to avoid all hand-written cast inside the implementation of IntVector,
something along that line

  public abstract class Vector<E> {
    abstract  void foo(Vector<E> vector);
    abstract Vector<E> bar();
  }

  /* non public */ abstract class AbstractVector<E, V extends AbstractVector<E, V>> extends Vector<E> {
    public abstract void foo(V vector);
    public final void foo(Vector<E> vector) {   // bridge by hand
      foo((V)vector);
    }
    public abstract V bar();
  }

  public abstract class IntVector extends AbstractVector<Integer, IntVector> {

  }

R?mi

----- Mail original -----
> De: "Paul Sandoz" <paul.sandoz at oracle.com>
> ?: "core-libs-dev" <core-libs-dev at openjdk.java.net>, "hotspot-dev" <hotspot-dev at openjdk.java.net>
> Envoy?: Jeudi 2 Avril 2020 00:46:55
> Objet: RFR JDK-8223347 Integration of Vector API (Incubator): Java API, implementation, and tests

> Hi,
> 
> A prior email sent out a request for review of the Vector API in preparation for
> JEP 338: Vector API (Incubator) [1] to be proposed for target:
> 
>  https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065345.html
>  <https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065345.html>
> 
> This email expands the review of the API to the Java implementation and Java
> tests:
> 
>  http://cr.openjdk.java.net/~psandoz/panama/JDK-8223347-vector-api-integration-java/jdk_src_webrev/webrev/
>  <http://cr.openjdk.java.net/~psandoz/panama/JDK-8223347-vector-api-integration-java/jdk_src_webrev/webrev/>
> 
>  http://cr.openjdk.java.net/~psandoz/panama/JDK-8223347-vector-api-integration-java/jdk_test_webrev/webrev/
>  <http://cr.openjdk.java.net/~psandoz/panama/JDK-8223347-vector-api-integration-java/jdk_test_webrev/webrev/>
> 
> (Further emails will sent for review of general HotSpot changes CPU architecture
> specific HotSpot changes, x64 and aarch64, and performance tests.  For an early
> peek see the links in the description of JDK-8223347.)
> 
> ?
> 
> The Vector API provides
> 
> - the public Vector class with vector operations common to all supported
> primitive types
> 
> - public primitive specializations, such as IntVector, with common operations
> associated with the primitive type
> 
> - internal concrete specializations for the vector size, such as Int256Vector,
> for a vector holding 8 ints.
> 
> Operations generally defer to one of approximately 20 vector intrinsics.  Some
> operations may be composed of other operations.
> 
> Explicit casts are performed by vector operations to ensure vectors arguments
> are of the required shape (bit size) and element type.
> 
> A vector intrinsic is an internal low-level vector operation. The last argument
> to the intrinsic is fall back behavior in Java, implementing the scalar
> operation over the number of elements held by the vector.  Thus, If the
> intrinsic is not supported in C2 for the other arguments then the Java
> implementation is executed (the Java implementation is always executed when
> running in the interpreter or for C1).
> 
> The public primitive specializations and the internal concrete implementations
> are generated from SSP template files, X-Vector.java.template and
> X-VectorBits.java.template respectively.
> 
> Overall the implementation approach is quite formulaic and rather repetitive.
> Once you grok the pattern It should be easier to review if a little laborious.
> 
> Unit tests are auto-generated by composing templates for each operation into an
> SSP template file which is then used to generate unit test files for each
> concrete vector class.  The tests are quite extensive and have found many a
> HotSpot issue in development across a wide range of platforms.
> 
> Paul.
> 
> [1] https://bugs.openjdk.java.net/browse/JDK-8201271
> <https://bugs.openjdk.java.net/browse/JDK-8201271>

From lois.foltan at oracle.com  Mon Apr 20 15:51:35 2020
From: lois.foltan at oracle.com (Lois Foltan)
Date: Mon, 20 Apr 2020 11:51:35 -0400
Subject: RFR (XS) JDK-8243045: AOTCompiledMethod::print_on triggers assertion
 after JDK-8210012
Message-ID: <2a744fd7-7b78-84b9-acaf-ac66104f5832@oracle.com>

Please review the following change to fix an issue when AOT and 
-Xlog:all=trace are specified together.? This issue was found during 
hs-tier6 testing.? AOTCompiledMethod::print_on() needs the same change 
that JDK-8210012 introduced for compiler/compileTask.cpp. The time stamp 
should come from tty, not the incoming outputStream parameter.

open webrev at: http://cr.openjdk.java.net/~lfoltan/bug_jdk8243045.0/webrev/
bug link: https://bugs.openjdk.java.net/browse/JDK-8243045

Testing: hs-tier1? & hs-tier6 on all platforms, hs-tier1-7 linux only

Thanks,
Lois

From harold.seigel at oracle.com  Mon Apr 20 17:09:57 2020
From: harold.seigel at oracle.com (Harold Seigel)
Date: Mon, 20 Apr 2020 13:09:57 -0400
Subject: RFR (XS) JDK-8243045: AOTCompiledMethod::print_on triggers
 assertion after JDK-8210012
In-Reply-To: <2a744fd7-7b78-84b9-acaf-ac66104f5832@oracle.com>
References: <2a744fd7-7b78-84b9-acaf-ac66104f5832@oracle.com>
Message-ID: <a0b96958-312d-4860-46bc-29215f7d83ea@oracle.com>

Looks good and trivial.

Thanks, Harold

On 4/20/2020 11:51 AM, Lois Foltan wrote:
> Please review the following change to fix an issue when AOT and 
> -Xlog:all=trace are specified together.? This issue was found during 
> hs-tier6 testing.? AOTCompiledMethod::print_on() needs the same change 
> that JDK-8210012 introduced for compiler/compileTask.cpp. The time 
> stamp should come from tty, not the incoming outputStream parameter.
>
> open webrev at: 
> http://cr.openjdk.java.net/~lfoltan/bug_jdk8243045.0/webrev/
> bug link: https://bugs.openjdk.java.net/browse/JDK-8243045
>
> Testing: hs-tier1? & hs-tier6 on all platforms, hs-tier1-7 linux only
>
> Thanks,
> Lois

From lois.foltan at oracle.com  Mon Apr 20 17:19:49 2020
From: lois.foltan at oracle.com (Lois Foltan)
Date: Mon, 20 Apr 2020 13:19:49 -0400
Subject: RFR (XS) JDK-8243045: AOTCompiledMethod::print_on triggers
 assertion after JDK-8210012
In-Reply-To: <a0b96958-312d-4860-46bc-29215f7d83ea@oracle.com>
References: <2a744fd7-7b78-84b9-acaf-ac66104f5832@oracle.com>
 <a0b96958-312d-4860-46bc-29215f7d83ea@oracle.com>
Message-ID: <67551773-3ec0-60f8-c320-07000a077896@oracle.com>

Thanks for the review Harold.
Lois

On 4/20/2020 1:09 PM, Harold Seigel wrote:
> Looks good and trivial.
>
> Thanks, Harold
>
> On 4/20/2020 11:51 AM, Lois Foltan wrote:
>> Please review the following change to fix an issue when AOT and 
>> -Xlog:all=trace are specified together. This issue was found during 
>> hs-tier6 testing. AOTCompiledMethod::print_on() needs the same change 
>> that JDK-8210012 introduced for compiler/compileTask.cpp. The time 
>> stamp should come from tty, not the incoming outputStream parameter.
>>
>> open webrev at: 
>> http://cr.openjdk.java.net/~lfoltan/bug_jdk8243045.0/webrev/
>> bug link: https://bugs.openjdk.java.net/browse/JDK-8243045
>>
>> Testing: hs-tier1? & hs-tier6 on all platforms, hs-tier1-7 linux only
>>
>> Thanks,
>> Lois


From coleen.phillimore at oracle.com  Mon Apr 20 18:59:44 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Mon, 20 Apr 2020 14:59:44 -0400
Subject: RFR (XS) JDK-8243045: AOTCompiledMethod::print_on triggers
 assertion after JDK-8210012
In-Reply-To: <a0b96958-312d-4860-46bc-29215f7d83ea@oracle.com>
References: <2a744fd7-7b78-84b9-acaf-ac66104f5832@oracle.com>
 <a0b96958-312d-4860-46bc-29215f7d83ea@oracle.com>
Message-ID: <03fee934-27c0-44ab-78fe-c067b404f231@oracle.com>

+1.

Coleen

On 4/20/20 1:09 PM, Harold Seigel wrote:
> Looks good and trivial.
>
> Thanks, Harold
>
> On 4/20/2020 11:51 AM, Lois Foltan wrote:
>> Please review the following change to fix an issue when AOT and 
>> -Xlog:all=trace are specified together. This issue was found during 
>> hs-tier6 testing. AOTCompiledMethod::print_on() needs the same change 
>> that JDK-8210012 introduced for compiler/compileTask.cpp. The time 
>> stamp should come from tty, not the incoming outputStream parameter.
>>
>> open webrev at: 
>> http://cr.openjdk.java.net/~lfoltan/bug_jdk8243045.0/webrev/
>> bug link: https://bugs.openjdk.java.net/browse/JDK-8243045
>>
>> Testing: hs-tier1? & hs-tier6 on all platforms, hs-tier1-7 linux only
>>
>> Thanks,
>> Lois


From vladimir.kozlov at oracle.com  Mon Apr 20 19:40:39 2020
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Mon, 20 Apr 2020 12:40:39 -0700
Subject: RFR (XS) JDK-8243045: AOTCompiledMethod::print_on triggers
 assertion after JDK-8210012
In-Reply-To: <03fee934-27c0-44ab-78fe-c067b404f231@oracle.com>
References: <2a744fd7-7b78-84b9-acaf-ac66104f5832@oracle.com>
 <a0b96958-312d-4860-46bc-29215f7d83ea@oracle.com>
 <03fee934-27c0-44ab-78fe-c067b404f231@oracle.com>
Message-ID: <2ea7f1ad-df74-80b8-1fe3-ad55180a3857@oracle.com>

+1

Vladimir K

On 4/20/20 11:59 AM, coleen.phillimore at oracle.com wrote:
> +1.
> 
> Coleen
> 
> On 4/20/20 1:09 PM, Harold Seigel wrote:
>> Looks good and trivial.
>>
>> Thanks, Harold
>>
>> On 4/20/2020 11:51 AM, Lois Foltan wrote:
>>> Please review the following change to fix an issue when AOT and -Xlog:all=trace are specified together. This issue 
>>> was found during hs-tier6 testing. AOTCompiledMethod::print_on() needs the same change that JDK-8210012 introduced 
>>> for compiler/compileTask.cpp. The time stamp should come from tty, not the incoming outputStream parameter.
>>>
>>> open webrev at: http://cr.openjdk.java.net/~lfoltan/bug_jdk8243045.0/webrev/
>>> bug link: https://bugs.openjdk.java.net/browse/JDK-8243045
>>>
>>> Testing: hs-tier1? & hs-tier6 on all platforms, hs-tier1-7 linux only
>>>
>>> Thanks,
>>> Lois
> 

From lois.foltan at oracle.com  Mon Apr 20 19:53:26 2020
From: lois.foltan at oracle.com (Lois Foltan)
Date: Mon, 20 Apr 2020 15:53:26 -0400
Subject: RFR (XS) JDK-8243045: AOTCompiledMethod::print_on triggers
 assertion after JDK-8210012
In-Reply-To: <2ea7f1ad-df74-80b8-1fe3-ad55180a3857@oracle.com>
References: <2a744fd7-7b78-84b9-acaf-ac66104f5832@oracle.com>
 <a0b96958-312d-4860-46bc-29215f7d83ea@oracle.com>
 <03fee934-27c0-44ab-78fe-c067b404f231@oracle.com>
 <2ea7f1ad-df74-80b8-1fe3-ad55180a3857@oracle.com>
Message-ID: <fcaee781-a9d4-71cb-f2cc-9d9e7ab874e7@oracle.com>

Thank you Coleen and Vladimir for your reviews!
Lois

On 4/20/2020 3:40 PM, Vladimir Kozlov wrote:
> +1
>
> Vladimir K
>
> On 4/20/20 11:59 AM, coleen.phillimore at oracle.com wrote:
>> +1.
>>
>> Coleen
>>
>> On 4/20/20 1:09 PM, Harold Seigel wrote:
>>> Looks good and trivial.
>>>
>>> Thanks, Harold
>>>
>>> On 4/20/2020 11:51 AM, Lois Foltan wrote:
>>>> Please review the following change to fix an issue when AOT and 
>>>> -Xlog:all=trace are specified together. This issue was found during 
>>>> hs-tier6 testing. AOTCompiledMethod::print_on() needs the same 
>>>> change that JDK-8210012 introduced for compiler/compileTask.cpp. 
>>>> The time stamp should come from tty, not the incoming outputStream 
>>>> parameter.
>>>>
>>>> open webrev at: 
>>>> http://cr.openjdk.java.net/~lfoltan/bug_jdk8243045.0/webrev/
>>>> bug link: https://bugs.openjdk.java.net/browse/JDK-8243045
>>>>
>>>> Testing: hs-tier1? & hs-tier6 on all platforms, hs-tier1-7 linux only
>>>>
>>>> Thanks,
>>>> Lois
>>


From paul.sandoz at oracle.com  Mon Apr 20 20:25:52 2020
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Mon, 20 Apr 2020 13:25:52 -0700
Subject: RFR JDK-8223347 Integration of Vector API (Incubator): Java API, 
 implementation, and tests
In-Reply-To: <1162249813.435958.1587391485813.JavaMail.zimbra@u-pem.fr>
References: <49F81016-55DF-4E3C-940A-A8DF8C712259@oracle.com>
 <1162249813.435958.1587391485813.JavaMail.zimbra@u-pem.fr>
Message-ID: <C796F37D-0F9F-4062-BB58-853284FBD836@oracle.com>

Hi Remi,

I am not aware of this being considered.  Perhaps because the vector specializations are generated from templates.  As such I am not seeing a huge benefit to doing this.

I admit to not understanding fully how this could work for vector arguments with a bridge method. I can see what you are getting to avoid declaration of co-variant overrides for simple methods such as add + lanewise, but it could limit what JavaDoc can be referenced, for example referencing methods accepting the primitive scalar value.

Paul.

> On Apr 20, 2020, at 7:04 AM, Remi Forax <forax at univ-mlv.fr> wrote:
> 
> BTW, did you try to add an intermediary private non visible class between Vector and IntVector to avoid all hand-written cast inside the implementation of IntVector,
> something along that line
> 
>  public abstract class Vector<E> {
>    abstract  void foo(Vector<E> vector);
>    abstract Vector<E> bar();
>  }
> 
>  /* non public */ abstract class AbstractVector<E, V extends AbstractVector<E, V>> extends Vector<E> {
>    public abstract void foo(V vector);
>    public final void foo(Vector<E> vector) {   // bridge by hand
>      foo((V)vector);
>    }
>    public abstract V bar();
>  }
> 
>  public abstract class IntVector extends AbstractVector<Integer, IntVector> {
> 
>  }
> 
> R?mi
> 
> ----- Mail original -----
>> De: "Paul Sandoz" <paul.sandoz at oracle.com>
>> ?: "core-libs-dev" <core-libs-dev at openjdk.java.net>, "hotspot-dev" <hotspot-dev at openjdk.java.net>
>> Envoy?: Jeudi 2 Avril 2020 00:46:55
>> Objet: RFR JDK-8223347 Integration of Vector API (Incubator): Java API, implementation, and tests
> 
>> Hi,
>> 
>> A prior email sent out a request for review of the Vector API in preparation for
>> JEP 338: Vector API (Incubator) [1] to be proposed for target:
>> 
>> https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065345.html
>> <https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065345.html>
>> 
>> This email expands the review of the API to the Java implementation and Java
>> tests:
>> 
>> http://cr.openjdk.java.net/~psandoz/panama/JDK-8223347-vector-api-integration-java/jdk_src_webrev/webrev/
>> <http://cr.openjdk.java.net/~psandoz/panama/JDK-8223347-vector-api-integration-java/jdk_src_webrev/webrev/>
>> 
>> http://cr.openjdk.java.net/~psandoz/panama/JDK-8223347-vector-api-integration-java/jdk_test_webrev/webrev/
>> <http://cr.openjdk.java.net/~psandoz/panama/JDK-8223347-vector-api-integration-java/jdk_test_webrev/webrev/>
>> 
>> (Further emails will sent for review of general HotSpot changes CPU architecture
>> specific HotSpot changes, x64 and aarch64, and performance tests.  For an early
>> peek see the links in the description of JDK-8223347.)
>> 
>> ?
>> 
>> The Vector API provides
>> 
>> - the public Vector class with vector operations common to all supported
>> primitive types
>> 
>> - public primitive specializations, such as IntVector, with common operations
>> associated with the primitive type
>> 
>> - internal concrete specializations for the vector size, such as Int256Vector,
>> for a vector holding 8 ints.
>> 
>> Operations generally defer to one of approximately 20 vector intrinsics.  Some
>> operations may be composed of other operations.
>> 
>> Explicit casts are performed by vector operations to ensure vectors arguments
>> are of the required shape (bit size) and element type.
>> 
>> A vector intrinsic is an internal low-level vector operation. The last argument
>> to the intrinsic is fall back behavior in Java, implementing the scalar
>> operation over the number of elements held by the vector.  Thus, If the
>> intrinsic is not supported in C2 for the other arguments then the Java
>> implementation is executed (the Java implementation is always executed when
>> running in the interpreter or for C1).
>> 
>> The public primitive specializations and the internal concrete implementations
>> are generated from SSP template files, X-Vector.java.template and
>> X-VectorBits.java.template respectively.
>> 
>> Overall the implementation approach is quite formulaic and rather repetitive.
>> Once you grok the pattern It should be easier to review if a little laborious.
>> 
>> Unit tests are auto-generated by composing templates for each operation into an
>> SSP template file which is then used to generate unit test files for each
>> concrete vector class.  The tests are quite extensive and have found many a
>> HotSpot issue in development across a wide range of platforms.
>> 
>> Paul.
>> 
>> [1] https://bugs.openjdk.java.net/browse/JDK-8201271
>> <https://bugs.openjdk.java.net/browse/JDK-8201271>


From david.holmes at oracle.com  Tue Apr 21 02:48:59 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 21 Apr 2020 12:48:59 +1000
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <cda962a7-1821-ed4b-ea4c-fbf43d47f9f5@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
 <135a6496-5b54-9117-5cbf-e3211ddc32dd@oracle.com>
 <bbb2b5ef-62f2-6507-95f7-03712a382153@oracle.com>
 <e242e057-de6e-493a-eb00-068b2db8c9fa@oracle.com>
 <df621fd2-0eaf-cb24-788f-12166b271303@oracle.com>
 <26105810-b9be-faec-6577-616ef26e84a7@oracle.com>
 <fa000582-255b-f072-99bd-765d587b4a26@oracle.com>
 <087a3bfa-b4ba-68d2-2509-7408e13bac1e@oracle.com>
 <19C3FFC0-B0DE-4A19-87A4-0A071BDC64C0@oracle.com>
 <5d686fe6-661b-4636-a71d-d379ef0324ed@oracle.com>
 <DC61E51D-C893-4E41-8F95-8396688D2C70@oracle.com>
 <cda962a7-1821-ed4b-ea4c-fbf43d47f9f5@oracle.com>
Message-ID: <b6ba0370-0127-1a93-5949-f576674a37d3@oracle.com>

On 20/04/2020 10:14 pm, Leo Korinth wrote:
> On 20/04/2020 11:23, Kim Barrett wrote:
>>> On Apr 19, 2020, at 10:35 PM, David Holmes <david.holmes at oracle.com> 
>>> wrote:
>>> On 20/04/2020 10:58 am, Kim Barrett wrote:
>>>>> On Apr 19, 2020, at 6:13 PM, David Holmes <david.holmes at oracle.com> 
>>>>> wrote:
>>>>>
>>>>> On 20/04/2020 3:08 am, Leo Korinth wrote:
>>>>>> What about this?
>>>>>> #define JVMCI_CHECK_0???????????? JVMCI_CHECK_(NULL_WORD)
>>>>>
>>>>> Why does this integer returning macro need to return NULL_WORD? As 
>>>>> I said above if writing this directly I would write "return 0;" in 
>>>>> the code. Is this just an attempt under the covers to (eventually) 
>>>>> avoid misusing JVMCI_CHECK_0 where JVMCI_CHECK_NULL should have 
>>>>> been used?
>>>> I agree that NULL_WORD has the wrong implication here.
>>>> As you correctly surmise, the problem with returning `0` is that it is
>>>> a null pointer constant. It would be nice if JVMCI_CHECK_0 could not
>>>> be used in a context where a pointer result is expected, and instead
>>>> require JVMCI_CHECK_NULL in such cases. That was the point of my
>>>> suggestion of using
>>>> #define JVMCI_CHECK_0? JVMCI_CHECK_(int(0))
>>>> The expression `int(0)` is not a null pointer constant, so won't
>>>> implicitly convert to a null pointer.
>>>
>>> I'm sure I tried something like that when fixing up some misuses of 
>>> the more general CHECK_0 macro, but it did not help in detecting the 
>>> misuse.
>>
>> Drat, you are right; that doesn't have the desired effect, at least
>> with gcc.
>>
>> I hadn't noticed some of the evolution of the definition of "null
>> pointer constant", and have mostly been looking at C++14 for reference
>> lately. The definition (4.10 in all three below) is:
>>
>> C++98: an integral constant expression (5.19) rvalue of integer type
>> that evaluates to zero.
>>
>> C++11: an integral constant expression (5.19) prvalue of integer type
>> that evaluates to zero or a prvalue of type std::nullptr_t.
>>
>> C++14: an integer literal (2.14.2) with value zero or a prvalue of
>> type std::nullptr_t.
>>
>> I've been assuming the literal 0 definition. And even so, it seems
>> that gcc (at least) is using the C++11 definition with -std=c++14. I
>> don't know what clang or Visual Studio do in C++14 mode.
>>
>> Rather than using `int(0)`, I think the following would achieve the
>> goal of preventing JVMCI_CHECK_0 from being used in a pointer context:
>>
>> inline int JVMCI_CHECK_0_value() { return 0; } // Private helper
>> #define JVMCI_CHECK_0? JVMCI_CHECK_(JVMCI_CHECK_0_value())
>>
>> Is that worth the trouble?
>>
> 
> I do not think so. I dislike these macros and i just wanted to make some 
> trivial improvements. If we can not agree, I propose we leave the macros 
> as they were.
> 
> New webrev including copyright year fix proposed by Magnus:
> http://cr.openjdk.java.net/~lkorinth/8240110/part6

No further comments from me.

Thanks,
David

> I keep the old definitions of JVMCI_CHECK_0 although I can not 
> understand the dislike to use NULL_WORD.
> 
> /Leo
> 
> 
> 
> 
> 

From coleen.phillimore at oracle.com  Tue Apr 21 20:12:44 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Tue, 21 Apr 2020 16:12:44 -0400
Subject: RFR (M) Close alignment gaps in InstanceKlass
Message-ID: <cf6cc941-61ef-daa0-f7fd-3dd918ecc008@oracle.com>

Summary: moved fields around and some constant fields into ConstantPool

This is a simple change except that I moved some constant fields from 
InstanceKlass into the constant pool so they can be shared read-only in 
the CDS archive.? There are associated repercussions in SA and JVMCI, so 
please look at these changes.? Also moved similarly sized fields 
together in the class so there's less likelihood of introducing gaps in 
future InstanceKlass changes.

InstanceKlass is reduced from 544 to 520 bytes in a simple Hello World 
class.

open webrev at http://cr.openjdk.java.net/~coleenp/2020/8238048.01/webrev
bug link https://bugs.openjdk.java.net/browse/JDK-8238048

Tested with tier1-6.

Thanks,
Coleen



From chris.plummer at oracle.com  Tue Apr 21 20:41:53 2020
From: chris.plummer at oracle.com (Chris Plummer)
Date: Tue, 21 Apr 2020 13:41:53 -0700
Subject: RFR (M) Close alignment gaps in InstanceKlass
In-Reply-To: <cf6cc941-61ef-daa0-f7fd-3dd918ecc008@oracle.com>
References: <cf6cc941-61ef-daa0-f7fd-3dd918ecc008@oracle.com>
Message-ID: <d78b9f5a-893f-f338-3a23-5ebabf4b020e@oracle.com>

Hi Coleen,

The SA changes look good. BTW, I've already made the 
TestIntConstant.java change in the Loom project. 
InstanceKlass::_misc_is_unsafe_anonymous was already changed to "8" in 
loom. You might want to sync up with Ron to make sure you aren't making 
conflicting changes in this area. Your code looks like:

 ?249?? enum {
 ?250???? _misc_rewritten?????????????????????????? = 1 << 0,? // 
methods rewritten.
 ?251???? _misc_has_nonstatic_fields??????????????? = 1 << 1,? // for 
sizing with UseCompressedOops
 ?252???? _misc_should_verify_class???????????????? = 1 << 2,? // allow 
caching of preverification
 ?253???? _misc_is_unsafe_anonymous???????????????? = 1 << 3,? // has 
embedded _unsafe_anonymous_host field
 ?254???? _misc_is_contended??????????????????????? = 1 << 4,? // marked 
with contended annotation

Loom looks like:

 ? static const unsigned _misc_kind_field_size = 0;
...
 ? // Start after _misc_kind field.
 ? enum {
 ??? _misc_rewritten?????????????????????????? = 1 << 
(_misc_kind_field_size + 0),? // methods rewritten.
 ??? _misc_has_nonstatic_fields??????????????? = 1 << 
(_misc_kind_field_size + 1),? // for sizing with UseCompressedOops
 ??? _misc_should_verify_class???????????????? = 1 << 
(_misc_kind_field_size + 2),? // allow caching of preverification
 ??? _misc_is_unsafe_anonymous???????????????? = 1 << 
(_misc_kind_field_size + 3),? // has embedded _unsafe_anonymous_host field
 ??? _misc_is_contended??????????????????????? = 1 << 
(_misc_kind_field_size + 4),? // marked with contended annotation
...

At the very least this is a merge conflict that loom will need to deal 
with and it would help to know about in advance (Alan normally does the 
merges).

thanks,

Chris

On 4/21/20 1:12 PM, coleen.phillimore at oracle.com wrote:
> Summary: moved fields around and some constant fields into ConstantPool
>
> This is a simple change except that I moved some constant fields from 
> InstanceKlass into the constant pool so they can be shared read-only 
> in the CDS archive.? There are associated repercussions in SA and 
> JVMCI, so please look at these changes.? Also moved similarly sized 
> fields together in the class so there's less likelihood of introducing 
> gaps in future InstanceKlass changes.
>
> InstanceKlass is reduced from 544 to 520 bytes in a simple Hello World 
> class.
>
> open webrev at http://cr.openjdk.java.net/~coleenp/2020/8238048.01/webrev
> bug link https://bugs.openjdk.java.net/browse/JDK-8238048
>
> Tested with tier1-6.
>
> Thanks,
> Coleen
>
>


From coleen.phillimore at oracle.com  Tue Apr 21 21:36:32 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Tue, 21 Apr 2020 17:36:32 -0400
Subject: RFR (M) Close alignment gaps in InstanceKlass
In-Reply-To: <d78b9f5a-893f-f338-3a23-5ebabf4b020e@oracle.com>
References: <cf6cc941-61ef-daa0-f7fd-3dd918ecc008@oracle.com>
 <d78b9f5a-893f-f338-3a23-5ebabf4b020e@oracle.com>
Message-ID: <269efe5a-3cd2-714f-16c3-f6bd46a60b4d@oracle.com>



On 4/21/20 4:41 PM, Chris Plummer wrote:
> Hi Coleen,
>
> The SA changes look good. BTW, I've already made the 
> TestIntConstant.java change in the Loom project. 
> InstanceKlass::_misc_is_unsafe_anonymous was already changed to "8" in 
> loom. You might want to sync up with Ron to make sure you aren't 
> making conflicting changes in this area. Your code looks like:
>
> ?249?? enum {
> ?250???? _misc_rewritten?????????????????????????? = 1 << 0,? // 
> methods rewritten.
> ?251???? _misc_has_nonstatic_fields??????????????? = 1 << 1,? // for 
> sizing with UseCompressedOops
> ?252???? _misc_should_verify_class???????????????? = 1 << 2,? // allow 
> caching of preverification
> ?253???? _misc_is_unsafe_anonymous???????????????? = 1 << 3,? // has 
> embedded _unsafe_anonymous_host field
> ?254???? _misc_is_contended??????????????????????? = 1 << 4,? // 
> marked with contended annotation
>
> Loom looks like:
>
> ? static const unsigned _misc_kind_field_size = 0;
> ...
> ? // Start after _misc_kind field.
> ? enum {
> ??? _misc_rewritten?????????????????????????? = 1 << 
> (_misc_kind_field_size + 0),? // methods rewritten.
> ??? _misc_has_nonstatic_fields??????????????? = 1 << 
> (_misc_kind_field_size + 1),? // for sizing with UseCompressedOops
> ??? _misc_should_verify_class???????????????? = 1 << 
> (_misc_kind_field_size + 2),? // allow caching of preverification
> ??? _misc_is_unsafe_anonymous???????????????? = 1 << 
> (_misc_kind_field_size + 3),? // has embedded _unsafe_anonymous_host 
> field
> ??? _misc_is_contended??????????????????????? = 1 << 
> (_misc_kind_field_size + 4),? // marked with contended annotation
> ...
>
> At the very least this is a merge conflict that loom will need to deal 
> with and it would help to know about in advance (Alan normally does 
> the merges).

It looks like there are other changes in loom that motivated changing it 
like this.? Why is misc_kind_field_size = 0?

Did loom add more misc_flags ?? It turns out that there are plenty of 
available flags in the access_flags if more are needed.

Thank you for looking at the SA changes.

Coelen
>
> thanks,
>
> Chris
>
> On 4/21/20 1:12 PM, coleen.phillimore at oracle.com wrote:
>> Summary: moved fields around and some constant fields into ConstantPool
>>
>> This is a simple change except that I moved some constant fields from 
>> InstanceKlass into the constant pool so they can be shared read-only 
>> in the CDS archive.? There are associated repercussions in SA and 
>> JVMCI, so please look at these changes. Also moved similarly sized 
>> fields together in the class so there's less likelihood of 
>> introducing gaps in future InstanceKlass changes.
>>
>> InstanceKlass is reduced from 544 to 520 bytes in a simple Hello 
>> World class.
>>
>> open webrev at 
>> http://cr.openjdk.java.net/~coleenp/2020/8238048.01/webrev
>> bug link https://bugs.openjdk.java.net/browse/JDK-8238048
>>
>> Tested with tier1-6.
>>
>> Thanks,
>> Coleen
>>
>>
>


From coleen.phillimore at oracle.com  Tue Apr 21 21:45:38 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Tue, 21 Apr 2020 17:45:38 -0400
Subject: RFR (M) Close alignment gaps in InstanceKlass
In-Reply-To: <d78b9f5a-893f-f338-3a23-5ebabf4b020e@oracle.com>
References: <cf6cc941-61ef-daa0-f7fd-3dd918ecc008@oracle.com>
 <d78b9f5a-893f-f338-3a23-5ebabf4b020e@oracle.com>
Message-ID: <515a359f-d4a1-785d-4aa9-ae39c7074438@oracle.com>



On 4/21/20 4:41 PM, Chris Plummer wrote:
> Hi Coleen,
>
> The SA changes look good. BTW, I've already made the 
> TestIntConstant.java change in the Loom project. 
> InstanceKlass::_misc_is_unsafe_anonymous was already changed to "8" in 
> loom. You might want to sync up with Ron to make sure you aren't 
> making conflicting changes in this area. Your code looks like:
>
> ?249?? enum {
> ?250???? _misc_rewritten?????????????????????????? = 1 << 0,? // 
> methods rewritten.
> ?251???? _misc_has_nonstatic_fields??????????????? = 1 << 1,? // for 
> sizing with UseCompressedOops
> ?252???? _misc_should_verify_class???????????????? = 1 << 2,? // allow 
> caching of preverification
> ?253???? _misc_is_unsafe_anonymous???????????????? = 1 << 3,? // has 
> embedded _unsafe_anonymous_host field
> ?254???? _misc_is_contended??????????????????????? = 1 << 4,? // 
> marked with contended annotation
>
> Loom looks like:
>
> ? static const unsigned _misc_kind_field_size = 0;
> ...
> ? // Start after _misc_kind field.
> ? enum {
> ??? _misc_rewritten?????????????????????????? = 1 << 
> (_misc_kind_field_size + 0),? // methods rewritten.
> ??? _misc_has_nonstatic_fields??????????????? = 1 << 
> (_misc_kind_field_size + 1),? // for sizing with UseCompressedOops
> ??? _misc_should_verify_class???????????????? = 1 << 
> (_misc_kind_field_size + 2),? // allow caching of preverification
> ??? _misc_is_unsafe_anonymous???????????????? = 1 << 
> (_misc_kind_field_size + 3),? // has embedded _unsafe_anonymous_host 
> field
> ??? _misc_is_contended??????????????????????? = 1 << 
> (_misc_kind_field_size + 4),? // marked with contended annotation
> ...
>
> At the very least this is a merge conflict that loom will need to deal 
> with and it would help to know about in advance (Alan normally does 
> the merges).

Actually I just had a look at the loom instanceKlass.hpp and this change 
helps that. Loom adds a kind of InstanceKlass, so changed the field to a 
u1 as I have done. This will clean up loom too.

Thanks,
Coleen
>
> thanks,
>
> Chris
>
> On 4/21/20 1:12 PM, coleen.phillimore at oracle.com wrote:
>> Summary: moved fields around and some constant fields into ConstantPool
>>
>> This is a simple change except that I moved some constant fields from 
>> InstanceKlass into the constant pool so they can be shared read-only 
>> in the CDS archive.? There are associated repercussions in SA and 
>> JVMCI, so please look at these changes. Also moved similarly sized 
>> fields together in the class so there's less likelihood of 
>> introducing gaps in future InstanceKlass changes.
>>
>> InstanceKlass is reduced from 544 to 520 bytes in a simple Hello 
>> World class.
>>
>> open webrev at 
>> http://cr.openjdk.java.net/~coleenp/2020/8238048.01/webrev
>> bug link https://bugs.openjdk.java.net/browse/JDK-8238048
>>
>> Tested with tier1-6.
>>
>> Thanks,
>> Coleen
>>
>>
>


From dean.long at oracle.com  Wed Apr 22 01:51:19 2020
From: dean.long at oracle.com (Dean Long)
Date: Tue, 21 Apr 2020 18:51:19 -0700
Subject: RFR (M) Close alignment gaps in InstanceKlass
In-Reply-To: <cf6cc941-61ef-daa0-f7fd-3dd918ecc008@oracle.com>
References: <cf6cc941-61ef-daa0-f7fd-3dd918ecc008@oracle.com>
Message-ID: <5a8317cb-dcdd-1fc5-7174-17304d75345a@oracle.com>

Hi Coleen.? The JVMCI changes look OK.? It looks like there is a Graal 
unittest that covers getSourceFileName, but those tests don't always get 
run.? If it's not too much trouble, could you look into enabling 
getSourceFileName() testing in

test/hotspot/jtreg/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestResolvedJavaType.java

?? It's currently on the "untested" list.

thanks,

dl

On 4/21/20 1:12 PM, coleen.phillimore at oracle.com wrote:
> Summary: moved fields around and some constant fields into ConstantPool
>
> This is a simple change except that I moved some constant fields from 
> InstanceKlass into the constant pool so they can be shared read-only 
> in the CDS archive.? There are associated repercussions in SA and 
> JVMCI, so please look at these changes.? Also moved similarly sized 
> fields together in the class so there's less likelihood of introducing 
> gaps in future InstanceKlass changes.
>
> InstanceKlass is reduced from 544 to 520 bytes in a simple Hello World 
> class.
>
> open webrev at http://cr.openjdk.java.net/~coleenp/2020/8238048.01/webrev
> bug link https://bugs.openjdk.java.net/browse/JDK-8238048
>
> Tested with tier1-6.
>
> Thanks,
> Coleen
>
>


From kim.barrett at oracle.com  Wed Apr 22 13:23:19 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 22 Apr 2020 09:23:19 -0400
Subject: RFR: 8239357: Revert gcc implementation of offset_of 
Message-ID: <62523ECD-EDB3-445B-9B9C-995B1F8D19BF@oracle.com>

Please review this change to undo the (small) portion of JDK-8238281
[1] related to the HotSpot offset_of macro for gcc/clang.  That change
should not have been made.  See CR for details.

I did not reinstate the redefinition of offsetof; we shouldn't be
using that macro, and having the warning enabled will catch
questionable cases.  (I also considered but didn?t poison it.)

CR:
https://bugs.openjdk.java.net/browse/JDK-8239357

[1] https://bugs.openjdk.java.net/browse/JDK-8238281

Webrev:
https://cr.openjdk.java.net/~kbarrett/8239357/open.00/

Testing:
mach5 tier1

With a temporary change to use offsetof with a non-standard-layout class,
verified the expected build failures occur with gcc and clang.


From erik.joelsson at oracle.com  Wed Apr 22 13:28:15 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Wed, 22 Apr 2020 06:28:15 -0700
Subject: RFR: 8239357: Revert gcc implementation of offset_of
In-Reply-To: <62523ECD-EDB3-445B-9B9C-995B1F8D19BF@oracle.com>
References: <62523ECD-EDB3-445B-9B9C-995B1F8D19BF@oracle.com>
Message-ID: <eec5d4de-a1fb-5bb1-481c-04be4c86aba4@oracle.com>

Build change looks good.

/Erik

On 2020-04-22 06:23, Kim Barrett wrote:
> Please review this change to undo the (small) portion of JDK-8238281
> [1] related to the HotSpot offset_of macro for gcc/clang.  That change
> should not have been made.  See CR for details.
>
> I did not reinstate the redefinition of offsetof; we shouldn't be
> using that macro, and having the warning enabled will catch
> questionable cases.  (I also considered but didn?t poison it.)
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8239357
>
> [1] https://bugs.openjdk.java.net/browse/JDK-8238281
>
> Webrev:
> https://cr.openjdk.java.net/~kbarrett/8239357/open.00/
>
> Testing:
> mach5 tier1
>
> With a temporary change to use offsetof with a non-standard-layout class,
> verified the expected build failures occur with gcc and clang.
>

From magnus.ihse.bursie at oracle.com  Wed Apr 22 13:57:15 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Wed, 22 Apr 2020 15:57:15 +0200
Subject: RFR: 8239357: Revert gcc implementation of offset_of
In-Reply-To: <62523ECD-EDB3-445B-9B9C-995B1F8D19BF@oracle.com>
References: <62523ECD-EDB3-445B-9B9C-995B1F8D19BF@oracle.com>
Message-ID: <242c3953-e334-cee9-8d3d-01c35f84900b@oracle.com>

On 2020-04-22 15:23, Kim Barrett wrote:
> Please review this change to undo the (small) portion of JDK-8238281
> [1] related to the HotSpot offset_of macro for gcc/clang.  That change
> should not have been made.  See CR for details.
>
> I did not reinstate the redefinition of offsetof; we shouldn't be
> using that macro, and having the warning enabled will catch
> questionable cases.  (I also considered but didn?t poison it.)
>
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8239357
>
> [1] https://bugs.openjdk.java.net/browse/JDK-8238281
>
> Webrev:
> https://cr.openjdk.java.net/~kbarrett/8239357/open.00/
Looks good to me.

/Magnus
>
> Testing:
> mach5 tier1
>
> With a temporary change to use offsetof with a non-standard-layout class,
> verified the expected build failures occur with gcc and clang.
>


From coleen.phillimore at oracle.com  Wed Apr 22 15:18:07 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Wed, 22 Apr 2020 11:18:07 -0400
Subject: RFR (M) Close alignment gaps in InstanceKlass
In-Reply-To: <5a8317cb-dcdd-1fc5-7174-17304d75345a@oracle.com>
References: <cf6cc941-61ef-daa0-f7fd-3dd918ecc008@oracle.com>
 <5a8317cb-dcdd-1fc5-7174-17304d75345a@oracle.com>
Message-ID: <addf99c8-c66c-9664-a87d-0a56165ed0bf@oracle.com>


Hi Dean, Thank you for looking at the JVMCI changes and the suggestion 
to add the test.? I did this and found a bug.? The new test is quite 
limited because there's no good test to see if a source file name can 
assertNotNull(type.getSourceFileName()), so I couldn't iterate through 
the list of loaded classes like the other tests in that file.

http://cr.openjdk.java.net/~coleenp/2020/8238048.02.incr/webrev/index.html

Thanks,
Coleen


On 4/21/20 9:51 PM, Dean Long wrote:
> Hi Coleen.? The JVMCI changes look OK.? It looks like there is a Graal 
> unittest that covers getSourceFileName, but those tests don't always 
> get run.? If it's not too much trouble, could you look into enabling 
> getSourceFileName() testing in
>
> test/hotspot/jtreg/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestResolvedJavaType.java 
>
>
> ?? It's currently on the "untested" list.
>
> thanks,
>
> dl
>
> On 4/21/20 1:12 PM, coleen.phillimore at oracle.com wrote:
>> Summary: moved fields around and some constant fields into ConstantPool
>>
>> This is a simple change except that I moved some constant fields from 
>> InstanceKlass into the constant pool so they can be shared read-only 
>> in the CDS archive.? There are associated repercussions in SA and 
>> JVMCI, so please look at these changes. Also moved similarly sized 
>> fields together in the class so there's less likelihood of 
>> introducing gaps in future InstanceKlass changes.
>>
>> InstanceKlass is reduced from 544 to 520 bytes in a simple Hello 
>> World class.
>>
>> open webrev at 
>> http://cr.openjdk.java.net/~coleenp/2020/8238048.01/webrev
>> bug link https://bugs.openjdk.java.net/browse/JDK-8238048
>>
>> Tested with tier1-6.
>>
>> Thanks,
>> Coleen
>>
>>
>


From igor.ignatyev at oracle.com  Wed Apr 22 18:03:46 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Wed, 22 Apr 2020 11:03:46 -0700
Subject: RFR(S) : 8242314 : use reproducible random in vmTestbase shared code
Message-ID: <A59F9A91-D3D4-4D98-BD02-C44E1C3790E4@oracle.com>

http://cr.openjdk.java.net/~iignatyev/8242314/webrev.00/ 
> 122 lines changed: 71 ins; 19 del; 32 mod;


Hi all,

could you please review this small patch which updates shared vmTestbase  test code to use j.t.l.Utils.getRandomInstance() as a random number generator or as a source of seed values? this patch lays the ground work for updates of :vmTestbase_* test groups which are to be done by remaining subtasks of 8241623[1], such split make it possible to work on/review/test testgroup-specific changes in concurrently.  
 
unfortunately, such extra dependency on /test/lib classes broke the shaky balance and caused some tests to fail due to "split testlibrary" problem (see 8188828[2] for the definition), the solution to that is to remove all dependencies on /test/lib classes from ExecDriver class, so the patch also includes "inlining" of several jdk.test.lib.Utils and Platform methods/fields into ExecDriver.

webrev: http://cr.openjdk.java.net/~iignatyev/8242314/webrev.00/ 
testing: all :vmTestbase_* test groups on {linux,windows,macosx}-x64 and solaris-sparcv9
JBS: https://bugs.openjdk.java.net/browse/JDK-8242314

[1] https://bugs.openjdk.java.net/browse/JDK-8241623
[2] https://bugs.openjdk.java.net/browse/JDK-8188828

Thanks,
-- Igor



From dean.long at oracle.com  Thu Apr 23 00:39:46 2020
From: dean.long at oracle.com (Dean Long)
Date: Wed, 22 Apr 2020 17:39:46 -0700
Subject: RFR (M) Close alignment gaps in InstanceKlass
In-Reply-To: <addf99c8-c66c-9664-a87d-0a56165ed0bf@oracle.com>
References: <cf6cc941-61ef-daa0-f7fd-3dd918ecc008@oracle.com>
 <5a8317cb-dcdd-1fc5-7174-17304d75345a@oracle.com>
 <addf99c8-c66c-9664-a87d-0a56165ed0bf@oracle.com>
Message-ID: <988a7d89-0304-edbd-b32a-e4b8428d8b8b@oracle.com>

Can you compare the result to some string, like "Object.java"?? That 
seems to be what HotSpotResolvedObjectTypeTest.java is doing.
Also, did getSourceFileName() return null for arrays before your change?

dl

On 4/22/20 8:18 AM, coleen.phillimore at oracle.com wrote:
>
> Hi Dean, Thank you for looking at the JVMCI changes and the suggestion 
> to add the test.? I did this and found a bug.? The new test is quite 
> limited because there's no good test to see if a source file name can 
> assertNotNull(type.getSourceFileName()), so I couldn't iterate through 
> the list of loaded classes like the other tests in that file.
>
> http://cr.openjdk.java.net/~coleenp/2020/8238048.02.incr/webrev/index.html 
>
>
> Thanks,
> Coleen
>
>
> On 4/21/20 9:51 PM, Dean Long wrote:
>> Hi Coleen.? The JVMCI changes look OK.? It looks like there is a 
>> Graal unittest that covers getSourceFileName, but those tests don't 
>> always get run.? If it's not too much trouble, could you look into 
>> enabling getSourceFileName() testing in
>>
>> test/hotspot/jtreg/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestResolvedJavaType.java 
>>
>>
>> ?? It's currently on the "untested" list.
>>
>> thanks,
>>
>> dl
>>
>> On 4/21/20 1:12 PM, coleen.phillimore at oracle.com wrote:
>>> Summary: moved fields around and some constant fields into ConstantPool
>>>
>>> This is a simple change except that I moved some constant fields 
>>> from InstanceKlass into the constant pool so they can be shared 
>>> read-only in the CDS archive.? There are associated repercussions in 
>>> SA and JVMCI, so please look at these changes. Also moved similarly 
>>> sized fields together in the class so there's less likelihood of 
>>> introducing gaps in future InstanceKlass changes.
>>>
>>> InstanceKlass is reduced from 544 to 520 bytes in a simple Hello 
>>> World class.
>>>
>>> open webrev at 
>>> http://cr.openjdk.java.net/~coleenp/2020/8238048.01/webrev
>>> bug link https://bugs.openjdk.java.net/browse/JDK-8238048
>>>
>>> Tested with tier1-6.
>>>
>>> Thanks,
>>> Coleen
>>>
>>>
>>
>


From dean.long at oracle.com  Thu Apr 23 01:00:56 2020
From: dean.long at oracle.com (Dean Long)
Date: Wed, 22 Apr 2020 18:00:56 -0700
Subject: RFR (M) Close alignment gaps in InstanceKlass
In-Reply-To: <988a7d89-0304-edbd-b32a-e4b8428d8b8b@oracle.com>
References: <cf6cc941-61ef-daa0-f7fd-3dd918ecc008@oracle.com>
 <5a8317cb-dcdd-1fc5-7174-17304d75345a@oracle.com>
 <addf99c8-c66c-9664-a87d-0a56165ed0bf@oracle.com>
 <988a7d89-0304-edbd-b32a-e4b8428d8b8b@oracle.com>
Message-ID: <0d1f9056-e20e-a8bd-6e90-2012b4e37f20@oracle.com>

It looks like calling the JVMCI getSourceFileName() on an array would 
have accessed random memory because it was expecting an InstanceKlass.? 
Instead of returning null we might want to throw an exception like in 
HotSpotResolvedPrimitiveType.

dl

On 4/22/20 5:39 PM, Dean Long wrote:
> Can you compare the result to some string, like "Object.java"?? That 
> seems to be what HotSpotResolvedObjectTypeTest.java is doing.
> Also, did getSourceFileName() return null for arrays before your change?
>
> dl
>
> On 4/22/20 8:18 AM, coleen.phillimore at oracle.com wrote:
>>
>> Hi Dean, Thank you for looking at the JVMCI changes and the 
>> suggestion to add the test.? I did this and found a bug.? The new 
>> test is quite limited because there's no good test to see if a source 
>> file name can assertNotNull(type.getSourceFileName()), so I couldn't 
>> iterate through the list of loaded classes like the other tests in 
>> that file.
>>
>> http://cr.openjdk.java.net/~coleenp/2020/8238048.02.incr/webrev/index.html 
>>
>>
>> Thanks,
>> Coleen
>>
>>
>> On 4/21/20 9:51 PM, Dean Long wrote:
>>> Hi Coleen.? The JVMCI changes look OK. It looks like there is a 
>>> Graal unittest that covers getSourceFileName, but those tests don't 
>>> always get run.? If it's not too much trouble, could you look into 
>>> enabling getSourceFileName() testing in
>>>
>>> test/hotspot/jtreg/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestResolvedJavaType.java 
>>>
>>>
>>> ?? It's currently on the "untested" list.
>>>
>>> thanks,
>>>
>>> dl
>>>
>>> On 4/21/20 1:12 PM, coleen.phillimore at oracle.com wrote:
>>>> Summary: moved fields around and some constant fields into 
>>>> ConstantPool
>>>>
>>>> This is a simple change except that I moved some constant fields 
>>>> from InstanceKlass into the constant pool so they can be shared 
>>>> read-only in the CDS archive.? There are associated repercussions 
>>>> in SA and JVMCI, so please look at these changes. Also moved 
>>>> similarly sized fields together in the class so there's less 
>>>> likelihood of introducing gaps in future InstanceKlass changes.
>>>>
>>>> InstanceKlass is reduced from 544 to 520 bytes in a simple Hello 
>>>> World class.
>>>>
>>>> open webrev at 
>>>> http://cr.openjdk.java.net/~coleenp/2020/8238048.01/webrev
>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8238048
>>>>
>>>> Tested with tier1-6.
>>>>
>>>> Thanks,
>>>> Coleen
>>>>
>>>>
>>>
>>
>


From coleen.phillimore at oracle.com  Thu Apr 23 02:32:28 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Wed, 22 Apr 2020 22:32:28 -0400
Subject: RFR (M) Close alignment gaps in InstanceKlass
In-Reply-To: <0d1f9056-e20e-a8bd-6e90-2012b4e37f20@oracle.com>
References: <cf6cc941-61ef-daa0-f7fd-3dd918ecc008@oracle.com>
 <5a8317cb-dcdd-1fc5-7174-17304d75345a@oracle.com>
 <addf99c8-c66c-9664-a87d-0a56165ed0bf@oracle.com>
 <988a7d89-0304-edbd-b32a-e4b8428d8b8b@oracle.com>
 <0d1f9056-e20e-a8bd-6e90-2012b4e37f20@oracle.com>
Message-ID: <bdc757b9-ff93-7382-590d-fa1483d1b579@oracle.com>



On 4/22/20 9:00 PM, Dean Long wrote:
> It looks like calling the JVMCI getSourceFileName() on an array would 
> have accessed random memory because it was expecting an 
> InstanceKlass.? Instead of returning null we might want to throw an 
> exception like in HotSpotResolvedPrimitiveType.
It was never called, except when I tried to call it on an array in the 
test.? It caused an assert in the hotspot code. How about this? 
Something else in that file throws JVMCIError with a similar message.

 ??? public String getSourceFileName() {
 ??????? if (isArray()) {
 ??????????? throw new JVMCIError("Cannot call getSourceFileName() on an 
array klass type: %s", this);
 ??????? }
 ??????? return getConstantPool().getSourceFileName();
 ??? }

>
> dl
>
> On 4/22/20 5:39 PM, Dean Long wrote:
>> Can you compare the result to some string, like "Object.java"?? That 
>> seems to be what HotSpotResolvedObjectTypeTest.java is doing.
>> Also, did getSourceFileName() return null for arrays before your change?

Fixed:
 ??? public void getSourceFileNameTest() {
 ??????? Class<?> c = Object.class;
 ??????? ResolvedJavaType type = metaAccess.lookupJavaType(c);
 ??????? assertEquals(type.getSourceFileName(), "Object.java");
 ??? }

thanks,
Coleen

>>
>> dl
>>
>> On 4/22/20 8:18 AM, coleen.phillimore at oracle.com wrote:
>>>
>>> Hi Dean, Thank you for looking at the JVMCI changes and the 
>>> suggestion to add the test.? I did this and found a bug.? The new 
>>> test is quite limited because there's no good test to see if a 
>>> source file name can assertNotNull(type.getSourceFileName()), so I 
>>> couldn't iterate through the list of loaded classes like the other 
>>> tests in that file.
>>>
>>> http://cr.openjdk.java.net/~coleenp/2020/8238048.02.incr/webrev/index.html 
>>>
>>>
>>> Thanks,
>>> Coleen
>>>
>>>
>>> On 4/21/20 9:51 PM, Dean Long wrote:
>>>> Hi Coleen.? The JVMCI changes look OK. It looks like there is a 
>>>> Graal unittest that covers getSourceFileName, but those tests don't 
>>>> always get run.? If it's not too much trouble, could you look into 
>>>> enabling getSourceFileName() testing in
>>>>
>>>> test/hotspot/jtreg/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestResolvedJavaType.java 
>>>>
>>>>
>>>> ?? It's currently on the "untested" list.
>>>>
>>>> thanks,
>>>>
>>>> dl
>>>>
>>>> On 4/21/20 1:12 PM, coleen.phillimore at oracle.com wrote:
>>>>> Summary: moved fields around and some constant fields into 
>>>>> ConstantPool
>>>>>
>>>>> This is a simple change except that I moved some constant fields 
>>>>> from InstanceKlass into the constant pool so they can be shared 
>>>>> read-only in the CDS archive.? There are associated repercussions 
>>>>> in SA and JVMCI, so please look at these changes. Also moved 
>>>>> similarly sized fields together in the class so there's less 
>>>>> likelihood of introducing gaps in future InstanceKlass changes.
>>>>>
>>>>> InstanceKlass is reduced from 544 to 520 bytes in a simple Hello 
>>>>> World class.
>>>>>
>>>>> open webrev at 
>>>>> http://cr.openjdk.java.net/~coleenp/2020/8238048.01/webrev
>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8238048
>>>>>
>>>>> Tested with tier1-6.
>>>>>
>>>>> Thanks,
>>>>> Coleen
>>>>>
>>>>>
>>>>
>>>
>>
>


From david.holmes at oracle.com  Thu Apr 23 02:33:53 2020
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 23 Apr 2020 12:33:53 +1000
Subject: RFR: 8239357: Revert gcc implementation of offset_of
In-Reply-To: <62523ECD-EDB3-445B-9B9C-995B1F8D19BF@oracle.com>
References: <62523ECD-EDB3-445B-9B9C-995B1F8D19BF@oracle.com>
Message-ID: <1da06b1c-153f-175d-73b3-209c6be164f4@oracle.com>

On 22/04/2020 11:23 pm, Kim Barrett wrote:
> Please review this change to undo the (small) portion of JDK-8238281
> [1] related to the HotSpot offset_of macro for gcc/clang.  That change
> should not have been made.  See CR for details.
> 
> I did not reinstate the redefinition of offsetof; we shouldn't be
> using that macro, and having the warning enabled will catch
> questionable cases.  (I also considered but didn?t poison it.)

Okay. This seems reasonable.

Thanks,
David

> CR:
> https://bugs.openjdk.java.net/browse/JDK-8239357
> 
> [1] https://bugs.openjdk.java.net/browse/JDK-8238281
> 
> Webrev:
> https://cr.openjdk.java.net/~kbarrett/8239357/open.00/
> 
> Testing:
> mach5 tier1
> 
> With a temporary change to use offsetof with a non-standard-layout class,
> verified the expected build failures occur with gcc and clang.
> 

From thomas.stuefe at gmail.com  Thu Apr 23 04:45:00 2020
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Thu, 23 Apr 2020 06:45:00 +0200
Subject: UL question: combining output flags with or at code site?
Message-ID: <CAA-vtUzt9DmKp5fKyzn7J1kUpXs4sY5vvVZiZoKg88XSyhVjGA@mail.gmail.com>

Hi,

with UL, is there a way at the logging site to specify that a logging has
to happen if one of the flags is set (so, *or* combined)?

E.g. if I wanted to print something when either "cds" or "metaspace" is
active? Without forcing the user to specify an asterix at the -Xlog
command, because that would activate too many other output sites.

If I use it like this:

LogTarget(Info, cds, metaspace) lt;

both flags must be active to print the output, right?

Thanks, Thomas

From matthias.baesken at sap.com  Thu Apr 23 07:14:53 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Thu, 23 Apr 2020 07:14:53 +0000
Subject: RFR: 8243389: enhance os::pd_print_cpu_info on linux
Message-ID: <AM0PR02MB541287DA62A637AA0856474893D30@AM0PR02MB5412.eurprd02.prod.outlook.com>

Hello, please review the following enhancement of  os::pd_print_cpu_info on linux .
It adds  information about
cache
online/offline cpus
frequency details
governors

by looking at the info provided at /sys/devices/system/cpu  .


Example of added output on an x86_64 machine  :

Online cpus:0-47
Offline cpus:
cache level:1
cache type:Data
cache size:32K
cache coherency line size:64
cache level:1
cache type:Instruction
cache size:32K
cache coherency line size:64
cache level:2
cache type:Unified
cache size:256K
cache coherency line size:64
cache level:3
cache type:Unified
cache size:18432K
cache coherency line size:64
BIOS frequency limitation:1862000
Frequency switch latency (ns):10000
Available cpu frequencies:1862000 1729000 1596000 1463000 1330000 1197000 1064000
Maximum cpu frequency:1862000
Minimum cpu frequency:1064000
Current cpu frequency:1064000
Available governors:conservative userspace powersave ondemand performance
Current governor:ondemand
Core performance/turbo boost:<Not Available>

bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8243389

http://cr.openjdk.java.net/~mbaesken/webrevs/8243389.0/


Thanks, Matthias

From stefan.karlsson at oracle.com  Thu Apr 23 07:33:20 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Thu, 23 Apr 2020 09:33:20 +0200
Subject: UL question: combining output flags with or at code site?
In-Reply-To: <CAA-vtUzt9DmKp5fKyzn7J1kUpXs4sY5vvVZiZoKg88XSyhVjGA@mail.gmail.com>
References: <CAA-vtUzt9DmKp5fKyzn7J1kUpXs4sY5vvVZiZoKg88XSyhVjGA@mail.gmail.com>
Message-ID: <1107d469-2e87-0d69-419d-e4147d1e84a3@oracle.com>

Hi Thomas,

On 2020-04-23 06:45, Thomas St?fe wrote:
> Hi,
> 
> with UL, is there a way at the logging site to specify that a logging has
> to happen if one of the flags is set (so, *or* combined)?
> 
> E.g. if I wanted to print something when either "cds" or "metaspace" is
> active? Without forcing the user to specify an asterix at the -Xlog
> command, because that would activate too many other output sites.
> 
> If I use it like this:
> 
> LogTarget(Info, cds, metaspace) lt;
> 
> both flags must be active to print the output, right?

You can use the type-erasure feature of the LogTargetHandle.

   LogTarget(Info, cds) log_cds;
   LogTarget(Info, metaspace) log_metaspace;

   LogTargetHandle log_handle_cds(log_cds);
   LogTargetHandle log_handle_metaspace(log_metaspace);

   LogTargetHandle log(log_cds.is_enabled() ? log_handle_cds : 
log_handle_metaspace);

   log.print("Either cds or metaspace");

With the above you get:

$ java -Xlog:cds -XX:+UseZGC -version
[0.015s][info][cds] Either cds or metaspace

$ java -Xlog:metaspace -XX:+UseZGC -version
[0.015s][info][metaspace] Either cds or metaspace

$ java -Xlog:metaspace,cds -XX:+UseZGC -version
[0.015s][info][cds] Either cds or metaspace

And nothing if you don't specify either:
$ java -XX:+UseZGC -version

You could also use a third LogTarget/LogTargetHandle to specify the 
combination of cds+metaspace and use that as a combination above.

Maybe there's a way to shrink this down. It would have been nice to be 
able to write:

LogTargetHandle log(log_cds.is_enabled() ? log_cds.handle() : 
log_metaspace.handle());

Or you could create a helper function to hide it:
LogTargetHandle log = cds_or_metaspace_logger();

HTH,
StefanK

> 
> Thanks, Thomas
> 

From kim.barrett at oracle.com  Thu Apr 23 07:56:27 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 23 Apr 2020 03:56:27 -0400
Subject: RFR: 8239357: Revert gcc implementation of offset_of
In-Reply-To: <242c3953-e334-cee9-8d3d-01c35f84900b@oracle.com>
References: <62523ECD-EDB3-445B-9B9C-995B1F8D19BF@oracle.com>
 <242c3953-e334-cee9-8d3d-01c35f84900b@oracle.com>
Message-ID: <D00D90B0-0032-47C6-9E3B-F5DFFF2D8377@oracle.com>

> On Apr 22, 2020, at 9:57 AM, Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com> wrote:
> 
> On 2020-04-22 15:23, Kim Barrett wrote:
>> Please review this change to undo the (small) portion of JDK-8238281
>> [1] related to the HotSpot offset_of macro for gcc/clang.  That change
>> should not have been made.  See CR for details.
>> 
>> I did not reinstate the redefinition of offsetof; we shouldn't be
>> using that macro, and having the warning enabled will catch
>> questionable cases.  (I also considered but didn?t poison it.)
>> 
>> CR:
>> https://bugs.openjdk.java.net/browse/JDK-8239357
>> 
>> [1] https://bugs.openjdk.java.net/browse/JDK-8238281
>> 
>> Webrev:
>> https://cr.openjdk.java.net/~kbarrett/8239357/open.00/
> Looks good to me.
> 
> /Magnus

Thanks.

>> 
>> Testing:
>> mach5 tier1
>> 
>> With a temporary change to use offsetof with a non-standard-layout class,
>> verified the expected build failures occur with gcc and clang.



From kim.barrett at oracle.com  Thu Apr 23 07:56:14 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 23 Apr 2020 03:56:14 -0400
Subject: RFR: 8239357: Revert gcc implementation of offset_of
In-Reply-To: <eec5d4de-a1fb-5bb1-481c-04be4c86aba4@oracle.com>
References: <62523ECD-EDB3-445B-9B9C-995B1F8D19BF@oracle.com>
 <eec5d4de-a1fb-5bb1-481c-04be4c86aba4@oracle.com>
Message-ID: <1AA3207C-E016-4A6C-894B-C214B2EC179B@oracle.com>

> On Apr 22, 2020, at 9:28 AM, Erik Joelsson <erik.joelsson at oracle.com> wrote:
> 
> Build change looks good.

Thanks.

> 
> /Erik
> 
> On 2020-04-22 06:23, Kim Barrett wrote:
>> Please review this change to undo the (small) portion of JDK-8238281
>> [1] related to the HotSpot offset_of macro for gcc/clang.  That change
>> should not have been made.  See CR for details.
>> 
>> I did not reinstate the redefinition of offsetof; we shouldn't be
>> using that macro, and having the warning enabled will catch
>> questionable cases.  (I also considered but didn?t poison it.)
>> 
>> CR:
>> https://bugs.openjdk.java.net/browse/JDK-8239357
>> 
>> [1] https://bugs.openjdk.java.net/browse/JDK-8238281
>> 
>> Webrev:
>> https://cr.openjdk.java.net/~kbarrett/8239357/open.00/
>> 
>> Testing:
>> mach5 tier1
>> 
>> With a temporary change to use offsetof with a non-standard-layout class,
>> verified the expected build failures occur with gcc and clang.



From kim.barrett at oracle.com  Thu Apr 23 07:56:40 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 23 Apr 2020 03:56:40 -0400
Subject: RFR: 8239357: Revert gcc implementation of offset_of
In-Reply-To: <1da06b1c-153f-175d-73b3-209c6be164f4@oracle.com>
References: <62523ECD-EDB3-445B-9B9C-995B1F8D19BF@oracle.com>
 <1da06b1c-153f-175d-73b3-209c6be164f4@oracle.com>
Message-ID: <FAE7854A-E2AE-434E-8915-363A67DB0054@oracle.com>

> On Apr 22, 2020, at 10:33 PM, David Holmes <david.holmes at oracle.com> wrote:
> 
> On 22/04/2020 11:23 pm, Kim Barrett wrote:
>> Please review this change to undo the (small) portion of JDK-8238281
>> [1] related to the HotSpot offset_of macro for gcc/clang.  That change
>> should not have been made.  See CR for details.
>> I did not reinstate the redefinition of offsetof; we shouldn't be
>> using that macro, and having the warning enabled will catch
>> questionable cases.  (I also considered but didn?t poison it.)
> 
> Okay. This seems reasonable.
> 
> Thanks,
> David

Thanks.

> 
>> CR:
>> https://bugs.openjdk.java.net/browse/JDK-8239357
>> [1] https://bugs.openjdk.java.net/browse/JDK-8238281
>> Webrev:
>> https://cr.openjdk.java.net/~kbarrett/8239357/open.00/
>> Testing:
>> mach5 tier1
>> With a temporary change to use offsetof with a non-standard-layout class,
>> verified the expected build failures occur with gcc and clang.



From kim.barrett at oracle.com  Thu Apr 23 09:30:34 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 23 Apr 2020 05:30:34 -0400
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <841c7ed3-ec0b-b300-15f1-30863453f74b@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
 <135a6496-5b54-9117-5cbf-e3211ddc32dd@oracle.com>
 <bbb2b5ef-62f2-6507-95f7-03712a382153@oracle.com>
 <e242e057-de6e-493a-eb00-068b2db8c9fa@oracle.com>
 <df621fd2-0eaf-cb24-788f-12166b271303@oracle.com>
 <98749A40-89C8-4E15-B46A-23F0D1CB2A32@oracle.com>
 <841c7ed3-ec0b-b300-15f1-30863453f74b@oracle.com>
Message-ID: <50021864-6BFA-4B67-A16F-139F8EF10E62@oracle.com>

> On Apr 19, 2020, at 11:39 AM, Leo Korinth <leo.korinth at oracle.com> wrote:

I've not yet covered all the items; I'll note inline which ones I
haven't addressed yet.

> On 17/04/2020 09:46, Kim Barrett wrote:
>>> On Apr 14, 2020, at 9:53 AM, Leo Korinth <leo.korinth at oracle.com> wrote:
>>> On 06/04/2020 01:37, David Holmes wrote:
>>>>> I could change CHECK_0 to use NULL_WORD though and if I do, I get compiler errors. CHECK_0 is used for both pointers and integral zeroes and types that implicitly converts from either. What the original thought was is unclear to me; is CHECK_0 to be used to return things convertible from pointers? If so, what is CHECK_NULL for?
>>>> No CHECK_0 is intended for int-returning functions (which would also extend to intptr_t).
>>> 
>>> It might be intended for it, but it is not used in such a way (therefore my remark on compile failures when using NULL_WORD)
>>> 
>>> I will do no change here.
>> It sounds like there are some places using CHECK_0 that should be
>> using CHECK_NULL, and they should be fixed. But that can be a separate
>> change. It might be aided (and backsliding prevented) by doing
>> something similar to what I suggest for JVMCI_CHECK_0/NULL below.
> 
> I believe that there are places that could use neither (the macro handles both integral and pointer types), but I do not want to fix that macro nest.

I'm not sure if you are saying anything beyond "CHECK_0 is being used
in places where CHECK_NULL should be used instead"?

Unfortunately, as discussed in the part of the thread with David, my
suggestion for JVMCI_CHECK_0 turns out to not have the desired effect.

So I agree, let's just leave these macros alone (both the JVMCI_ and
unprefixed variants).

>> In the end you seem to have backed away from fixing the definition of
>> NULL_WORD, and I'm not sure why. Though I think the definition that
>> was proposed is at least unnecessarily baroque, possibly wrong. It
>> seems to me that a good and sufficient change for that would be to
>> delete all the existing platform definitions and put the following in
>> globalDefinitions.hpp:
>> const intptr_t NULL_WORD = 0;
>> Then it's no longer a null pointer constant, so can't be misused in
>> contexts where a pointer is required, will be treated as an integral
>> value by templates, &etc.
> 
> my first try was (in part2):
> 
> struct null_word_t {
>  operator intptr_t()  const { return 0; }
> };
> 
> which IMO is nice as it:
> 1) mirrors my (initial) implementation of null_t
> 2) can be template specialized against
> 3) is defined in one place
> 
> Unfortunately we do not yet have constexpr.
> 
> So my (temporary) fix was (in part3):
> const intptr_t NULL_WORD = 0;
> 
> This is /identical/ to what you propose. I think this is a good compromise. However after much effort from my side trying to get this solution you did not want my changes to NULL (and maybe I misunderstood you regarding NULL_WORD) so I reverted that in part5.
> 
> I am very much confused to what you want. Should I once again revert NULL_WORD and put it in globalDefinitions.hpp?

Sorry, but I haven't looked at all the different stages of your
changes.  I've only looked carefully through 8240110/1, to most recent
total combined change.

And it seems the discussion of NULL and the discussion of NULL_WORD
have gotten confused and mixed up.  So I'll be explicit here.

I think / agree the following definition of NULL_WORD in
globalDefinitions.hpp is fine:

const intptr_t NULL_WORD = 0;
// The Standard only guarantees intptr_t is at least big enough to
// hold a pointer value, and purportedly there are platforms where
// this will fail.  If this ever trips we'll need to think about
// whether it really matters and what to do if it does.
STATIC_ASSERT(sizeof(NULL_WORD) == sizeof(void*));

(The comment for the static assert is only intended to be suggestive
of wording; feel free to reword to suit.)

I agree there are some places that are using NULL that should be using
NULL_WORD, and vice versa.

I wonder if it would be worthwhile splitting out the NULL_WORD
defintion and the NULL <-> NULL_WORD changes as a separate smaller
change that is (I think) easily dealt with. It will then no longer
clutter the remainder.

>> I wouldn't worry about this comment in globalDefinitions_solstudio.hpp:
>>   "On some platforms, sizeof(intptr_t) > sizeof(void*)"
>> or you can be really paranoid and STATIC_ASSERT that they are the same
>> size. Though I really wonder what platform(s) that's talking about.
>> That comment comes from 6452081: 3/4 Allow for Linux builds with Sun
>> Studio Linux compilers.  So maybe 32bit Solaris Studio Linux is weird?
> 
> I did this originally (disregard the typo of uint vs int) in part1.
> 
> STATIC_ASSERT(sizeof(uintptr_t) == sizeof(char *)); // uintptr_t is only guaranteed to be at /least/ as big as a pointer
> 
> I later reverted this change as I thought you dismissed all of my NULL and NULL_WORD definitions. I really wanted this.

See above.

>> src/hotspot/cpu/sparc/c1_CodeStubs_sparc.cpp
>> 303     AddressLiteral addrlit(NULL_WORD, metadata_Relocation::spec(_index));
>> 318     AddressLiteral addrlit(NULL_WORD, oop_Relocation::spec(_index));
>> I think these should be left as NULL rather than being changed to
>> NULL_WORD.  The expected type is the address type, and other types are
>> converted to that via casts in constructor overloads.
>> Similarly here:
>> src/hotspot/cpu/sparc/c1_LIRAssembler_sparc.cpp
>>  426   AddressLiteral addrlit(NULL_WORD, oop_Relocation::spec(oop_index));
>>  445   AddressLiteral addrlit(NULL_WORD, metadata_Relocation::spec(index));
> 
> I believe those do not at all auto convert, but promote to (integral) zero as I (quite thoroughly) described in my reply to David (and thus /should/ be NULL_WORD. I believe this is one prime example when our NULL definitions results in unintuitive selection of integer constructor.

I'd failed to notice that this one is in Solaris/sparc code, where the
gcc rules don't apply.

I think the expectation here is that NULL would select the pointer
overload, but it doesn't on this platform because NULL is a literal
0/0L, and as you say, the integer overload is better.

Similar code compiled with gcc would also select the integer overload,
but with a -Wconversion-null warning.  Apparently gcc's __null, while
fairly similar to nullptr, does support conversion to an integer type
but with a warning.  And that's intentional; quoting from
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=35669#c13

  0 is still a valid definition of NULL so conversion from NULL to int
  is OK. And __null does not have type nullptr_t, changing it to have
  that type would break a lot of code

nullptr would do the right thing here.

But note that this code is slated to go away real soon (JEP 381).

My inclination would be to not make this change, in case there is a
repo of changes for that removal that will have a merge conflict if
this change were to go in first.  But I no longer care all that much.

>> src/hotspot/os/linux/cgroupV1Subsystem_linux.hpp
>> [old code]
>> 100     CachingCgroupController* _memory = NULL;
>> 101     CgroupV1Controller* _cpuset = NULL;
>> 102     CachingCgroupController* _cpu = NULL;
>> 103     CgroupV1Controller* _cpuacct = NULL;
>> How does this even compile?  Non-static member initializers are a
>> C++11 feature.  Weirdly, g++ with -std=gnu++98 warns about this for
>> some types/initializers but not for others.
> 
> As I remarked in my original mail, I believe it to be a gcc bug, but I am not sure.

Agreed that it?s a gcc bug / weirdness.  Deleting that stuff is fine with me.

>> src/hotspot/os/solaris/attachListener_solaris.cpp
>> 461   status = ::sema_init(&_wakeup, 0, NULL, NULL);
>> =>
>> 461   status = ::sema_init(&_wakeup, 0, 0, NULL);
>> The old code appears to have been cribbed from the documentation:
>> https://docs.oracle.com/cd/E36784_01/html/E36874/sema-init-3c.html
>> See the example of "Default semaphore initialization (intra-process)".
>> Rather than a literal 0, better would be USYNC_THREAD.  Presumably the
>> value of USYNC_THREAD is 0.
> 
> I could not easily find the value of the symbols, so I only changed it to the correct type, with resent changes to Java regarding Solaris do you care enough for me to check this out, or is 0 good enough?

This is another bit of Solaris code that is slated to go away real
soon now.  So I don't care all that much.

>> src/hotspot/os/windows/os_windows.cpp
>> 1659   if (!GetFileVersionInfo(kernel32_path, 0, version_size, version_info)) {
>> Given that the second argument, which was changed from NULL to 0, is a
>> "handle" (though unused), I think using NULL was appropriate and this
>> change shouldn't be made.
> 
> I have no windows machine, so I can not check this out 100% at the moment, but according to the compile error I got, and the documentation of GetFileVersionInfoA or GetFileVersionInfoW I find on the net:
> https://docs.microsoft.com/en-us/windows/win32/api/winver/nf-winver-getfileversioninfoa
> 
> The type of dwHandle is DWORD. Defined as:
> https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-dtyp/262627d8-3418-4627-9218-4ffe110850b2 
> 
> What makes you think the handle is of pointer type, to me it looks like a long?

A Windows "handle" is generally a pointer, so NULL makes sense
semantically, and that's presumably what the original author thought
too. OTOH, the examples of calling that function that I found via a
web search just now all seem to pass literal 0. So I'm okay with the
change after all.

>> src/hotspot/share/classfile/javaClasses.cpp
>>  851       k->set_java_mirror_handle(NULL);
>> =>
>>  851       k->set_java_mirror_handle(OopHandle(NULL));
>> PATTERN CHANGE
>> Similarly here:
>> src/hotspot/share/classfile/moduleEntry.hpp
>>  82     _module = OopHandle(NULL);
>>  84     _pd = OopHandle(NULL); >
>> And here:
>> src/hotspot/share/classfile/modules.cpp
>> 156     version_symbol = TempNewSymbol(NULL); > 161   TempNewSymbol location_symbol(NULL);
>> 368     version_symbol = TempNewSymbol(NULL);
>> 373   TempNewSymbol location_symbol(NULL);
>> Although I'm not sure why the assignments on line 156 and line 368 is
>> even being done, since each version_symbol was already default
>> initialized.
> Probably to /show/ that it has an initialized value (I do not necessarily agree that it is good). Do you want me to change any of those, and if so to what?

Unfinished.

>> src/hotspot/share/code/codeHeapState.cpp
>>  140     if (((_termString) != 0) && (strlen(_termString) > 0)){   \
>>  154     if (((_termString) != 0) && (strlen(_termString) > 0)){   \
>> I think these changes to compare _termstring with 0 rather than NULL
>> should not be made.  _termString is supposed to be a `[const] char*`,
>> so comparing to NULL is preferable to comparing to 0.
> 
> _termString is not a pointer (or at least not most of the time), see my thorough discussion of the topic with David. I believe my solution is the next safest one (after rewriting that macro nest which I am not going to do now).

Unfinished.

>> Though at a brief skim it looks like _termString is always provided
>> and never NULL, so it might be better to assert that _termString is
>> non-NULL.  But that's a completely different change.
> 
> I believe we should use less macros, and I agree that is different change.

Unfinished.

>> src/hotspot/share/gc/parallel/psAdaptiveSizePolicy.cpp
>> src/hotspot/share/gc/parallel/psParallelCompact.cpp
>> src/hotspot/share/gc/shared/memset_with_concurrent_readers.hpp
>> src/hotspot/share/gc/z/zErrno.cpp
>> These #include changes seem unrelated to NULL.
> 
> They are all (unnecessary) including standard headers, It is hard to know which headers that (indirectly) redefines NULL. It is better to not include them if you do not have to. If I remember correctly I removed all but one standard library include in the "gc" folder. The one I left was for <new>.

Regarding knowing which standard headers might redefine NULL, we
should not care. We should be using the native NULL and don't care who
might define it.

For zErrno.cpp, the #include of <string.h> seems unnecessary, since it
is using os::strerror.  But I think it is appropriate to #include
<errno.h> here, rather than relying on it being implicitly included by
some dependency that could change, breaking this unrelated code.

For memset_with_concurrent_readers.hpp, I'm not a big fan of
globalDefinitions.hpp (I really dislike big omnibus random collections
of stuff headers).  But okay.  I didn't re-examine the others.

>> src/hotspot/share/jfr/jni/jfrGetAllEventClasses.cpp
>> 172   assert(add_method_signature != NULL, "invariant");
>> =>
>> 172   assert(add_method_sig_sym != NULL, "invariant");
>> This seems to be an unrelated bug fix; the wrong value was being
>> compared to NULL.
> 
> It is highly related as it did not compile on my original definition of NULL and was thus found.

The change is fine, but I would have filed and fixed the bug rather than
burying it in this large change.  Much better for archeology.

>> src/hotspot/share/jfr/jni/jfrJavaCall.cpp
>>  57   assert(_storage != NULL, "invariant");
>> =>
>>  57   assert(_storage_index < (int) ARRAY_SIZE(_storage), "invariant");
>> _storage is an array, so the original assert is definitionally true,
>> but it's harmless to verify, in case future maintenance makes it an
>> allocated array rather than inline.
> 
> It is so statically true that it should not be a runtime check.
> 
> My replacement check is not only implicitly checking that the variable is not NULL, it does so at compile time (ARRAY_SIZE will fail on NULL) and it does an even better runtime check that actually adds value.

Unfinished.

>> The size of _storage is definitionally SIZE, and _storage_index is
>> checked against SIZE a couple lines later, making the proposed
>> replacement just a duplicative check.
> 
> I have no problem with duplicate checks, but if anything should be removed, it is the SIZE assert. My assert clearly shows that the access is okay even though someone would change the size of the array to not be of size SIZE. My assert can be understood without looking outside the method.

Unfinished.

>> So I think the proposed change should not be made.
>> Similarly at lines 65, 73, and 91.
> I strongly disagree. Each of my replacement asserts is strictly superior if I am not missing something. Not only that, but they might not compile on windows in the future depending on how we define NULL in the future.
> 
> Most of the changes I have made are because of compile errors after my new NULL, not "just" because of cosmetic issues.

Unfinished.

>> src/hotspot/share/jvmci/jvmciCompilerToVM.cpp
>> 1562   JVMCIENV->set_HotSpotStackFrameReference_localIsVirtual(hs_frame, 0);
>> This change to use literal 0 rather than NULL as a null pointer
>> constant does not seem like an improvement to me.  I think this change
>> should not be made.
>> ------------------------------------------------------------------------------
>> src/hotspot/share/jvmci/jvmciCompilerToVM.cpp
>> I looked at the changes from JVMCI_CHECK_0 to JVMCI_CHECK_NULL and
>> think they are correct, assuming a suitable definition of the latter.
>> (Though see below regarding that.)
>> ------------------------------------------------------------------------------
>> src/hotspot/share/jvmci/jvmciCompilerToVM.cpp
>> 2572   JVMCIObjectArray current_array = NULL;
>> =>
>> 2572   JVMCIObjectArray current_array(NULL);
>> PATTERN CHANGE
>> ------------------------------------------------------------------------------
>> src/hotspot/share/jvmci/jvmciCompilerToVMInit.cpp
>> 267     if ((value) != 0) { \
>> Value is a char*; I don't think this change should be made.
>> ------------------------------------------------------------------------------
>> src/hotspot/share/jvmci/jvmciExceptions.hpp
>> 50 #define JVMCI_CHECK_NULL                               JVMCI_CHECK_(NULL)
>> =>
>> 50 #define JVMCI_CHECK_NULL                               JVMCI_CHECK_(0)
>> For platforms that warn about converting NULL to an integral value
>> this is a type safety regression. I think this change should not be
>> made.
>> ------------------------------------------------------------------------------
>> src/hotspot/share/jvmci/jvmciExceptions.hpp
>> 49 #define JVMCI_CHECK_0                                  JVMCI_CHECK_(0)
>> =>
>> 49 #define JVMCI_CHECK_0                                  JVMCI_CHECK_(NULL_WORD)
>> I kind of think this is an abuse of NULL_WORD. It also serves no real
>> purpose with the current definitions of NULL_WORD (though would with
>> the suggestion far above). If the goal is to have something that isn't
>> a null pointer constant so it can't be used in inappropriate
>> contexts (which is a reasonable goal), then replace "0" with "int(0)"
>> (and comment appropriately).
> 
> Please read my conversation with David and see if you after that still have any issue with what I am proposing.

Unfinished.

>> src/hotspot/share/oops/access.hpp
>> 296 class ArrayAccess ...
>> How about adding the following private functions to this class, and
>> use them in the obvious places:
>>   template<typename T> static const T* const_raw_null() { return NULL; }
>>   template<typename T> static       T*       raw_null() { return NULL; }
> 
> I am lacking c++ and template skills, but the problem here is that we need to make the type of the null pointer concrete for the template system. I fail to see how these template methods achieve this from static context and no parameters. The code fails to compile for me, and I am probably missing something, but I fail to see the idea behind you construction. The only type parameter is the return one, and that is the one we do not know, right?
> 
> As another solution we could specialize the type explicitly at the call-site instead of in the delivered parameters (I tried that first), but that was IMO /much/ harder to read.
> 
> Even if raw_null() does work (assuming failure from my side), the original is cleaner IMO.

I guess my suggestion left too much as an exercise for the reader.  Here?s the
change I was contemplating, which seems to work fine:

diff -r 1fa1ec0e7048 -r 08ee68a3d2a4 src/hotspot/share/oops/access.hpp
--- a/src/hotspot/share/oops/access.hpp Sat Apr 18 20:19:45 2020 +0200
+++ b/src/hotspot/share/oops/access.hpp Sun Apr 19 21:08:43 2020 -0400
@@ -295,13 +295,17 @@
 template <DecoratorSet decorators = DECORATORS_NONE>
 class ArrayAccess: public HeapAccess<IS_ARRAY | decorators> {
   typedef HeapAccess<IS_ARRAY | decorators> AccessT;
+
+  template<typename T> static const T* const_raw_null() { return NULL; }
+  template<typename T> static       T*       raw_null() { return NULL; }
+
 public:
   template <typename T>
   static inline void arraycopy(arrayOop src_obj, size_t src_offset_in_bytes,
                                arrayOop dst_obj, size_t dst_offset_in_bytes,
                                size_t length) {
-    AccessT::arraycopy(src_obj, src_offset_in_bytes, reinterpret_cast<const T*>(NULL),
-                       dst_obj, dst_offset_in_bytes, reinterpret_cast<T*>(NULL),
+    AccessT::arraycopy(src_obj, src_offset_in_bytes, const_raw_null<T>(),
+                       dst_obj, dst_offset_in_bytes, raw_null<T>(),
                        length);
   }
 
@@ -309,7 +313,7 @@
   static inline void arraycopy_to_native(arrayOop src_obj, size_t src_offset_in_bytes,
                                          T* dst,
                                          size_t length) {
-    AccessT::arraycopy(src_obj, src_offset_in_bytes, reinterpret_cast<const T*>(NULL),
+    AccessT::arraycopy(src_obj, src_offset_in_bytes, const_raw_null<T>(),
                        NULL, 0, dst,
                        length);
   }
@@ -319,15 +323,15 @@
                                            arrayOop dst_obj, size_t dst_offset_in_bytes,
                                            size_t length) {
     AccessT::arraycopy(NULL, 0, src,
-                       dst_obj, dst_offset_in_bytes, reinterpret_cast<T*>(NULL),
+                       dst_obj, dst_offset_in_bytes, raw_null<T>(),
                        length);
   }
 
   static inline bool oop_arraycopy(arrayOop src_obj, size_t src_offset_in_bytes,
                                    arrayOop dst_obj, size_t dst_offset_in_bytes,
                                    size_t length) {
-    return AccessT::oop_arraycopy(src_obj, src_offset_in_bytes, reinterpret_cast<const HeapWord*>(NULL),
-                                  dst_obj, dst_offset_in_bytes, reinterpret_cast<HeapWord*>(NULL),
+    return AccessT::oop_arraycopy(src_obj, src_offset_in_bytes, const_raw_null<HeapWord>(),
+                                  dst_obj, dst_offset_in_bytes, raw_null<HeapWord>(),
                                   length);
   }
 
>> src/hotspot/share/oops/constantPool.cpp
>>  270     set_resolved_references(OopHandle(NULL));
>>  302     set_resolved_references(OopHandle(NULL));
>>  376     set_resolved_references(OopHandle(NULL));
>> PATTERN CHANGE
>> Similarly here:
>> src/hotspot/share/oops/cpCache.cpp
>> 766   set_resolved_references(OopHandle(NULL));
>> src/hotspot/share/oops/klass.cpp
>> 549   _java_mirror = OopHandle(NULL);
>> src/hotspot/share/prims/jni.cpp
>>  319   TempNewSymbol class_name(NULL);
>> src/hotspot/share/prims/jvm.cpp
>>  954   TempNewSymbol class_name(NULL);
>> src/hotspot/share/prims/methodHandles.cpp
>> 1311   TempNewSymbol name(NULL);
>> 1312   TempNewSymbol sig(NULL);
> 
> I understand you have no problem with those changes, is that correct?

Not correct.  See ?PATTERN CHANGE?.  Unfinished.


From thomas.stuefe at gmail.com  Thu Apr 23 10:21:01 2020
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Thu, 23 Apr 2020 12:21:01 +0200
Subject: UL question: combining output flags with or at code site?
In-Reply-To: <1107d469-2e87-0d69-419d-e4147d1e84a3@oracle.com>
References: <CAA-vtUzt9DmKp5fKyzn7J1kUpXs4sY5vvVZiZoKg88XSyhVjGA@mail.gmail.com>
 <1107d469-2e87-0d69-419d-e4147d1e84a3@oracle.com>
Message-ID: <CAA-vtUxu9rsQW4e6_up=uHQUzp+v4BHYBp_WoW3NZbHyqeOhfw@mail.gmail.com>

Thank you, Stefan. For the moment a helper function may be the simplest
solution.

Maybe long-term we could have a LogTarget variant OR-combining the flags
instead of AND-ing them...

Sometimes I think I would have prepared ORing the flags by default, I would
have found that less surprising and easier to use (e.g. if I find a
particular "gc" log output useful, I could have added a "metaspace" tag and
thus selected it for -Xlog=metaspace too. As it is now, this would just
effectively mute the output by requiring both flags to be set).

Alas, it is now what it is, and its a matter of taste anyway...

Thanks, Thomas



On Thu, Apr 23, 2020 at 9:33 AM Stefan Karlsson <stefan.karlsson at oracle.com>
wrote:

> Hi Thomas,
>
> On 2020-04-23 06:45, Thomas St?fe wrote:
> > Hi,
> >
> > with UL, is there a way at the logging site to specify that a logging has
> > to happen if one of the flags is set (so, *or* combined)?
> >
> > E.g. if I wanted to print something when either "cds" or "metaspace" is
> > active? Without forcing the user to specify an asterix at the -Xlog
> > command, because that would activate too many other output sites.
> >
> > If I use it like this:
> >
> > LogTarget(Info, cds, metaspace) lt;
> >
> > both flags must be active to print the output, right?
>
> You can use the type-erasure feature of the LogTargetHandle.
>
>    LogTarget(Info, cds) log_cds;
>    LogTarget(Info, metaspace) log_metaspace;
>
>    LogTargetHandle log_handle_cds(log_cds);
>    LogTargetHandle log_handle_metaspace(log_metaspace);
>
>    LogTargetHandle log(log_cds.is_enabled() ? log_handle_cds :
> log_handle_metaspace);
>
>    log.print("Either cds or metaspace");
>
> With the above you get:
>
> $ java -Xlog:cds -XX:+UseZGC -version
> [0.015s][info][cds] Either cds or metaspace
>
> $ java -Xlog:metaspace -XX:+UseZGC -version
> [0.015s][info][metaspace] Either cds or metaspace
>
> $ java -Xlog:metaspace,cds -XX:+UseZGC -version
> [0.015s][info][cds] Either cds or metaspace
>
> And nothing if you don't specify either:
> $ java -XX:+UseZGC -version
>
> You could also use a third LogTarget/LogTargetHandle to specify the
> combination of cds+metaspace and use that as a combination above.
>
> Maybe there's a way to shrink this down. It would have been nice to be
> able to write:
>
> LogTargetHandle log(log_cds.is_enabled() ? log_cds.handle() :
> log_metaspace.handle());
>
> Or you could create a helper function to hide it:
> LogTargetHandle log = cds_or_metaspace_logger();
>
> HTH,
> StefanK
>
> >
> > Thanks, Thomas
> >
>

From thomas.stuefe at gmail.com  Thu Apr 23 14:05:52 2020
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Thu, 23 Apr 2020 16:05:52 +0200
Subject: NMT and os::split_reserved_memory?
Message-ID: <CAA-vtUzysafi0LX=xTOkKp3L2WnMbYe_Ti7mudJsKU+ifOAzVw@mail.gmail.com>

Hi,

How does NMT handle os::split_reserved_memory?

Lets say I reserve a space, then split it into two spaces
using os::split_reserved_memory() (which really is a noop for most
platforms). Then I want to set both spaces to different flags.

Setting the second one triggers an assert: assert(reserved_rgn->flag() ==
mtNone, "Overwrite memory type");

since I assume NMT is not aware that splitting happened.

What would be the correct way to go here? Can I somehow inform NMT that
this region has been split? I assume within NMT both split regions are
still represented by one reserved_rgn object.

Thank you,

Thomas

From vkempik at azul.com  Thu Apr 23 15:26:36 2020
From: vkempik at azul.com (Vladimir Kempik)
Date: Thu, 23 Apr 2020 15:26:36 +0000
Subject: RFR: 8243470: [macos] bring back O2 opt level for unsafe.cpp
Message-ID: <DA95D3E7-1EEA-4CF8-95E8-F1A690666439@azul.com>


Hello
Please review a fix for JDK-8243470

Long time ago as part of JEP284: New HotSpot Build System this fix was applied to jdk9 https://bugs.openjdk.java.net/browse/JDK-8152666
At that time it was decided to lower optimisation level for unsafe.cpp from O2 to O1 only for clang on Macosx.

I suppose it was done due to issues in Set/Get<Object> helper functions where too optimistic optimisations were eliminating some null pointer checks. it was probably a clang bug.
That issue could be checked with test jdk/test/sun/misc/CopyMemory.java.

I believe that workaround (going from O2 to O1) produced this issue - https://bugs.openjdk.java.net/browse/JDK-8234963 (Thread.getStackTrace is slow with clang).
JDK-8234963 can only be seen on mac with libjvm compiled by clang.

Here I propose the patch which eliminates that workaround for clang 8+.

I have tested clang versions 8/9/9.1/10, all of them showed good results:
1) CopyMemory test passes fine on 11/14/15.
2) jdk11/jdk14 passed tck. Regression testing were good as well. jdk15: no new failures in tck.
3) The testRandom "benchmark" from 8234963 shows great improvements on my machine, going down from ~200 seconds to ~150 seconds (the newer clang the better result). For comparision, gcc built libjvm for jdk11 shows ~130 seconds on my machine.

The webrev: http://cr.openjdk.java.net/~vkempik/8243470/webrev.00/

getStackTrace benchmark: http://cr.openjdk.java.net/~vkempik/8243470/TestRandom.java

The bug: https://bugs.openjdk.java.net/browse/JDK-8243470

Thanks, Vladimir

From zgu at redhat.com  Thu Apr 23 15:51:33 2020
From: zgu at redhat.com (Zhengyu Gu)
Date: Thu, 23 Apr 2020 11:51:33 -0400
Subject: NMT and os::split_reserved_memory?
In-Reply-To: <CAA-vtUzysafi0LX=xTOkKp3L2WnMbYe_Ti7mudJsKU+ifOAzVw@mail.gmail.com>
References: <CAA-vtUzysafi0LX=xTOkKp3L2WnMbYe_Ti7mudJsKU+ifOAzVw@mail.gmail.com>
Message-ID: <6e389a52-85c0-675d-ab65-4f4eedc36614@redhat.com>

Hi Thomas,

On 4/23/20 10:05 AM, Thomas St?fe wrote:
> Hi,
> 
> How does NMT handle os::split_reserved_memory?

I don't think NMT handles it.

> 
> Lets say I reserve a space, then split it into two spaces
> using os::split_reserved_memory() (which really is a noop for most
> platforms). Then I want to set both spaces to different flags.

You can remove the whole region, then create two separate regions.
The intricate part is when the region contains committed regions, I am 
not so sure how virtual memory system behaves.

Thanks,

-Zhengyu

> 
> Setting the second one triggers an assert: assert(reserved_rgn->flag() ==
> mtNone, "Overwrite memory type");
> 
> since I assume NMT is not aware that splitting happened.
> 
> What would be the correct way to go here? Can I somehow inform NMT that
> this region has been split? I assume within NMT both split regions are
> still represented by one reserved_rgn object.
> 
> Thank you,
> 
> Thomas
> 


From ioi.lam at oracle.com  Thu Apr 23 16:43:58 2020
From: ioi.lam at oracle.com (Ioi Lam)
Date: Thu, 23 Apr 2020 09:43:58 -0700
Subject: RFR (M) Close alignment gaps in InstanceKlass
In-Reply-To: <cf6cc941-61ef-daa0-f7fd-3dd918ecc008@oracle.com>
References: <cf6cc941-61ef-daa0-f7fd-3dd918ecc008@oracle.com>
Message-ID: <eab4ef9f-c8b7-b917-e94b-0caf32612943@oracle.com>

Hi Coleen,

The changes look good to me.

Thanks
- Ioi

On 4/21/20 1:12 PM, coleen.phillimore at oracle.com wrote:
> Summary: moved fields around and some constant fields into ConstantPool
>
> This is a simple change except that I moved some constant fields from 
> InstanceKlass into the constant pool so they can be shared read-only 
> in the CDS archive.? There are associated repercussions in SA and 
> JVMCI, so please look at these changes.? Also moved similarly sized 
> fields together in the class so there's less likelihood of introducing 
> gaps in future InstanceKlass changes.
>
> InstanceKlass is reduced from 544 to 520 bytes in a simple Hello World 
> class.
>
> open webrev at http://cr.openjdk.java.net/~coleenp/2020/8238048.01/webrev
> bug link https://bugs.openjdk.java.net/browse/JDK-8238048
>
> Tested with tier1-6.
>
> Thanks,
> Coleen
>
>


From thomas.stuefe at gmail.com  Thu Apr 23 17:03:35 2020
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Thu, 23 Apr 2020 19:03:35 +0200
Subject: NMT and os::split_reserved_memory?
In-Reply-To: <6e389a52-85c0-675d-ab65-4f4eedc36614@redhat.com>
References: <CAA-vtUzysafi0LX=xTOkKp3L2WnMbYe_Ti7mudJsKU+ifOAzVw@mail.gmail.com>
 <6e389a52-85c0-675d-ab65-4f4eedc36614@redhat.com>
Message-ID: <CAA-vtUw1OwRthWCyaGAKhaFw1mwpcWY2_x3pRcgZd_OZrLpUjw@mail.gmail.com>

Hi Zhengyu,

thank you! That is unfortunate, but I'll see what I can do.

Cheers, Thomas

On Thu, Apr 23, 2020 at 5:51 PM Zhengyu Gu <zgu at redhat.com> wrote:

> Hi Thomas,
>
> On 4/23/20 10:05 AM, Thomas St?fe wrote:
> > Hi,
> >
> > How does NMT handle os::split_reserved_memory?
>
> I don't think NMT handles it.
>
> >
> > Lets say I reserve a space, then split it into two spaces
> > using os::split_reserved_memory() (which really is a noop for most
> > platforms). Then I want to set both spaces to different flags.
>
> You can remove the whole region, then create two separate regions.
> The intricate part is when the region contains committed regions, I am
> not so sure how virtual memory system behaves.
>
> Thanks,
>
> -Zhengyu
>
> >
> > Setting the second one triggers an assert: assert(reserved_rgn->flag() ==
> > mtNone, "Overwrite memory type");
> >
> > since I assume NMT is not aware that splitting happened.
> >
> > What would be the correct way to go here? Can I somehow inform NMT that
> > this region has been split? I assume within NMT both split regions are
> > still represented by one reserved_rgn object.
> >
> > Thank you,
> >
> > Thomas
> >
>
>

From coleen.phillimore at oracle.com  Thu Apr 23 17:36:26 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Thu, 23 Apr 2020 13:36:26 -0400
Subject: RFR (M) Close alignment gaps in InstanceKlass
In-Reply-To: <eab4ef9f-c8b7-b917-e94b-0caf32612943@oracle.com>
References: <cf6cc941-61ef-daa0-f7fd-3dd918ecc008@oracle.com>
 <eab4ef9f-c8b7-b917-e94b-0caf32612943@oracle.com>
Message-ID: <e222e997-b515-ac89-7061-24ffe35aacd4@oracle.com>

Thanks, Ioi!
Coleen

On 4/23/20 12:43 PM, Ioi Lam wrote:
> Hi Coleen,
>
> The changes look good to me.
>
> Thanks
> - Ioi
>
> On 4/21/20 1:12 PM, coleen.phillimore at oracle.com wrote:
>> Summary: moved fields around and some constant fields into ConstantPool
>>
>> This is a simple change except that I moved some constant fields from 
>> InstanceKlass into the constant pool so they can be shared read-only 
>> in the CDS archive.? There are associated repercussions in SA and 
>> JVMCI, so please look at these changes. Also moved similarly sized 
>> fields together in the class so there's less likelihood of 
>> introducing gaps in future InstanceKlass changes.
>>
>> InstanceKlass is reduced from 544 to 520 bytes in a simple Hello 
>> World class.
>>
>> open webrev at 
>> http://cr.openjdk.java.net/~coleenp/2020/8238048.01/webrev
>> bug link https://bugs.openjdk.java.net/browse/JDK-8238048
>>
>> Tested with tier1-6.
>>
>> Thanks,
>> Coleen
>>
>>
>


From dean.long at oracle.com  Thu Apr 23 20:16:06 2020
From: dean.long at oracle.com (Dean Long)
Date: Thu, 23 Apr 2020 13:16:06 -0700
Subject: RFR (M) Close alignment gaps in InstanceKlass
In-Reply-To: <bdc757b9-ff93-7382-590d-fa1483d1b579@oracle.com>
References: <cf6cc941-61ef-daa0-f7fd-3dd918ecc008@oracle.com>
 <5a8317cb-dcdd-1fc5-7174-17304d75345a@oracle.com>
 <addf99c8-c66c-9664-a87d-0a56165ed0bf@oracle.com>
 <988a7d89-0304-edbd-b32a-e4b8428d8b8b@oracle.com>
 <0d1f9056-e20e-a8bd-6e90-2012b4e37f20@oracle.com>
 <bdc757b9-ff93-7382-590d-fa1483d1b579@oracle.com>
Message-ID: <bacb5f88-17ae-0baa-e6a4-642480394545@oracle.com>

OK, thanks, looks good!

dl

On 4/22/20 7:32 PM, coleen.phillimore at oracle.com wrote:
>
>
> On 4/22/20 9:00 PM, Dean Long wrote:
>> It looks like calling the JVMCI getSourceFileName() on an array would 
>> have accessed random memory because it was expecting an 
>> InstanceKlass.? Instead of returning null we might want to throw an 
>> exception like in HotSpotResolvedPrimitiveType.
> It was never called, except when I tried to call it on an array in the 
> test.? It caused an assert in the hotspot code. How about this? 
> Something else in that file throws JVMCIError with a similar message.
>
> ??? public String getSourceFileName() {
> ??????? if (isArray()) {
> ??????????? throw new JVMCIError("Cannot call getSourceFileName() on 
> an array klass type: %s", this);
> ??????? }
> ??????? return getConstantPool().getSourceFileName();
> ??? }
>
>>
>> dl
>>
>> On 4/22/20 5:39 PM, Dean Long wrote:
>>> Can you compare the result to some string, like "Object.java"?? That 
>>> seems to be what HotSpotResolvedObjectTypeTest.java is doing.
>>> Also, did getSourceFileName() return null for arrays before your 
>>> change?
>
> Fixed:
> ??? public void getSourceFileNameTest() {
> ??????? Class<?> c = Object.class;
> ??????? ResolvedJavaType type = metaAccess.lookupJavaType(c);
> ??????? assertEquals(type.getSourceFileName(), "Object.java");
> ??? }
>
> thanks,
> Coleen
>
>>>
>>> dl
>>>
>>> On 4/22/20 8:18 AM, coleen.phillimore at oracle.com wrote:
>>>>
>>>> Hi Dean, Thank you for looking at the JVMCI changes and the 
>>>> suggestion to add the test.? I did this and found a bug. The new 
>>>> test is quite limited because there's no good test to see if a 
>>>> source file name can assertNotNull(type.getSourceFileName()), so I 
>>>> couldn't iterate through the list of loaded classes like the other 
>>>> tests in that file.
>>>>
>>>> http://cr.openjdk.java.net/~coleenp/2020/8238048.02.incr/webrev/index.html 
>>>>
>>>>
>>>> Thanks,
>>>> Coleen
>>>>
>>>>
>>>> On 4/21/20 9:51 PM, Dean Long wrote:
>>>>> Hi Coleen.? The JVMCI changes look OK. It looks like there is a 
>>>>> Graal unittest that covers getSourceFileName, but those tests 
>>>>> don't always get run. If it's not too much trouble, could you look 
>>>>> into enabling getSourceFileName() testing in
>>>>>
>>>>> test/hotspot/jtreg/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestResolvedJavaType.java 
>>>>>
>>>>>
>>>>> ?? It's currently on the "untested" list.
>>>>>
>>>>> thanks,
>>>>>
>>>>> dl
>>>>>
>>>>> On 4/21/20 1:12 PM, coleen.phillimore at oracle.com wrote:
>>>>>> Summary: moved fields around and some constant fields into 
>>>>>> ConstantPool
>>>>>>
>>>>>> This is a simple change except that I moved some constant fields 
>>>>>> from InstanceKlass into the constant pool so they can be shared 
>>>>>> read-only in the CDS archive.? There are associated repercussions 
>>>>>> in SA and JVMCI, so please look at these changes. Also moved 
>>>>>> similarly sized fields together in the class so there's less 
>>>>>> likelihood of introducing gaps in future InstanceKlass changes.
>>>>>>
>>>>>> InstanceKlass is reduced from 544 to 520 bytes in a simple Hello 
>>>>>> World class.
>>>>>>
>>>>>> open webrev at 
>>>>>> http://cr.openjdk.java.net/~coleenp/2020/8238048.01/webrev
>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8238048
>>>>>>
>>>>>> Tested with tier1-6.
>>>>>>
>>>>>> Thanks,
>>>>>> Coleen
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>


From coleen.phillimore at oracle.com  Thu Apr 23 20:27:29 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Thu, 23 Apr 2020 16:27:29 -0400
Subject: RFR (M) Close alignment gaps in InstanceKlass
In-Reply-To: <bacb5f88-17ae-0baa-e6a4-642480394545@oracle.com>
References: <cf6cc941-61ef-daa0-f7fd-3dd918ecc008@oracle.com>
 <5a8317cb-dcdd-1fc5-7174-17304d75345a@oracle.com>
 <addf99c8-c66c-9664-a87d-0a56165ed0bf@oracle.com>
 <988a7d89-0304-edbd-b32a-e4b8428d8b8b@oracle.com>
 <0d1f9056-e20e-a8bd-6e90-2012b4e37f20@oracle.com>
 <bdc757b9-ff93-7382-590d-fa1483d1b579@oracle.com>
 <bacb5f88-17ae-0baa-e6a4-642480394545@oracle.com>
Message-ID: <c73b4cb4-d802-439f-9fcd-51e3164a8017@oracle.com>

Thanks, Dean!
Coleen

On 4/23/20 4:16 PM, Dean Long wrote:
> OK, thanks, looks good!
>
> dl
>
> On 4/22/20 7:32 PM, coleen.phillimore at oracle.com wrote:
>>
>>
>> On 4/22/20 9:00 PM, Dean Long wrote:
>>> It looks like calling the JVMCI getSourceFileName() on an array 
>>> would have accessed random memory because it was expecting an 
>>> InstanceKlass.? Instead of returning null we might want to throw an 
>>> exception like in HotSpotResolvedPrimitiveType.
>> It was never called, except when I tried to call it on an array in 
>> the test.? It caused an assert in the hotspot code. How about this? 
>> Something else in that file throws JVMCIError with a similar message.
>>
>> ??? public String getSourceFileName() {
>> ??????? if (isArray()) {
>> ??????????? throw new JVMCIError("Cannot call getSourceFileName() on 
>> an array klass type: %s", this);
>> ??????? }
>> ??????? return getConstantPool().getSourceFileName();
>> ??? }
>>
>>>
>>> dl
>>>
>>> On 4/22/20 5:39 PM, Dean Long wrote:
>>>> Can you compare the result to some string, like "Object.java"?? 
>>>> That seems to be what HotSpotResolvedObjectTypeTest.java is doing.
>>>> Also, did getSourceFileName() return null for arrays before your 
>>>> change?
>>
>> Fixed:
>> ??? public void getSourceFileNameTest() {
>> ??????? Class<?> c = Object.class;
>> ??????? ResolvedJavaType type = metaAccess.lookupJavaType(c);
>> ??????? assertEquals(type.getSourceFileName(), "Object.java");
>> ??? }
>>
>> thanks,
>> Coleen
>>
>>>>
>>>> dl
>>>>
>>>> On 4/22/20 8:18 AM, coleen.phillimore at oracle.com wrote:
>>>>>
>>>>> Hi Dean, Thank you for looking at the JVMCI changes and the 
>>>>> suggestion to add the test.? I did this and found a bug. The new 
>>>>> test is quite limited because there's no good test to see if a 
>>>>> source file name can assertNotNull(type.getSourceFileName()), so I 
>>>>> couldn't iterate through the list of loaded classes like the other 
>>>>> tests in that file.
>>>>>
>>>>> http://cr.openjdk.java.net/~coleenp/2020/8238048.02.incr/webrev/index.html 
>>>>>
>>>>>
>>>>> Thanks,
>>>>> Coleen
>>>>>
>>>>>
>>>>> On 4/21/20 9:51 PM, Dean Long wrote:
>>>>>> Hi Coleen.? The JVMCI changes look OK. It looks like there is a 
>>>>>> Graal unittest that covers getSourceFileName, but those tests 
>>>>>> don't always get run. If it's not too much trouble, could you 
>>>>>> look into enabling getSourceFileName() testing in
>>>>>>
>>>>>> test/hotspot/jtreg/compiler/jvmci/jdk.vm.ci.runtime.test/src/jdk/vm/ci/runtime/test/TestResolvedJavaType.java 
>>>>>>
>>>>>>
>>>>>> ?? It's currently on the "untested" list.
>>>>>>
>>>>>> thanks,
>>>>>>
>>>>>> dl
>>>>>>
>>>>>> On 4/21/20 1:12 PM, coleen.phillimore at oracle.com wrote:
>>>>>>> Summary: moved fields around and some constant fields into 
>>>>>>> ConstantPool
>>>>>>>
>>>>>>> This is a simple change except that I moved some constant fields 
>>>>>>> from InstanceKlass into the constant pool so they can be shared 
>>>>>>> read-only in the CDS archive.? There are associated 
>>>>>>> repercussions in SA and JVMCI, so please look at these changes. 
>>>>>>> Also moved similarly sized fields together in the class so 
>>>>>>> there's less likelihood of introducing gaps in future 
>>>>>>> InstanceKlass changes.
>>>>>>>
>>>>>>> InstanceKlass is reduced from 544 to 520 bytes in a simple Hello 
>>>>>>> World class.
>>>>>>>
>>>>>>> open webrev at 
>>>>>>> http://cr.openjdk.java.net/~coleenp/2020/8238048.01/webrev
>>>>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8238048
>>>>>>>
>>>>>>> Tested with tier1-6.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Coleen
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>


From ioi.lam at oracle.com  Fri Apr 24 05:59:28 2020
From: ioi.lam at oracle.com (Ioi Lam)
Date: Thu, 23 Apr 2020 22:59:28 -0700
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <87mu7a9dcj.fsf@mid.deneb.enyo.de>
References: <CA+3eh11or=pxZ7D752GJbNm7a7=OofxX74Gmza2HJSmgD_9ZNA@mail.gmail.com>
 <5B92DB3F-A436-4358-A8B4-56977A6EDE6E@oracle.com>
 <7fe5e846-8157-0d53-69a7-f2c8cf878757@redhat.com>
 <71a1abc6-cbd8-9a1e-2027-d5ac5ba9ab9b@redhat.com> <854ktisnkk.fsf@arm.com>
 <87mu7a9dcj.fsf@mid.deneb.enyo.de>
Message-ID: <03a05a27-b979-368d-dd85-504d368cddd4@oracle.com>



On 4/17/20 11:27 AM, Florian Weimer wrote:
> * Nick Gasson:
>
>> On 03/13/20 17:42 pm, Andrew Dinn wrote:
>> Seems like many of us were suffering in silence. I reported this to the
>> cc-mode author: the patch here is a huge improvement, although it's
>> still a bit slow.
>>
>> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=40052#8
> Nice.  Will the globals.hpp situation improve further as the result of
> the flags modularization work?
>
> <https://wiki.openjdk.java.net/display/HotSpot/HotSpot+Command-Line+Flags+Overhaul+-+Design+Doc>
>
> It looks to me the large macro is removed.
Actually, I am going with a different direction with the flags clean up. 
The big macro will stay (John Rose convinced me that I can do all the 
things I wanted while keeping the existing macros largely unchanged). I 
have created a new RFE (and sub-tasks)

https://bugs.openjdk.java.net/browse/JDK-8243208

I'll post a new design doc/prototype soon.

I'll leave the tackling of the big macro to braver souls.

- Ioi

From erik.osterlund at oracle.com  Fri Apr 24 08:17:08 2020
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Fri, 24 Apr 2020 10:17:08 +0200
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64
Message-ID: <56a0cf85-9698-64b2-95b4-e5b123c3e9e2@oracle.com>

Hi,

Today, the use of compressed class pointers assumes the use of 
compressed oops.
This patch loosens up the relationship between compressed oops and 
compressed
class pointers, so that compressed class pointers can be used without 
compressed
oops. This benefits for example ZGC that wants compressed class 
pointers, but
not compressed oops, effectively giving ZGC 4 bytes smaller objects.

Much of the complexity of the change is that r12 used to be used by 
compressed
class pointers as some kind of semi-fast temp register that one would 
restore
to the compressed oops heap base (or zero). That made it effectively a 
slightly
optimized spilling mechanism used to find a temp register. I replaced that
mechanism with a plain old normal temp register that you pass in as a 
parameter.
The bad news is that I had to find available temp registers in a lot of 
places.
The good news is that almost always were there temp registers available 
for free,
and hence the new temp register is even faster than the old optimized 
spilling
mechanism. Because we almost never need any spilling at all in the 
contexts where
this is used.

Since I want the 4 new bytes to actually make objects smaller, I poked 
the new
layout code a bit so that the klass gap is made available for fields. 
That used
to be made available only with compressed oops enabled, due to 
restrictions in
the layout code. Our new layout code does not have such restrictions, and so
I will make the 4 bytes available for fields when the new layout code is 
used
and compressed class pointers are used.

Now I'm only fixing this for HotSpot x86_64. Ideally the use of 
compressed oops
and compressed class pointers should not be entangled in other platforms and
Graal. But that is beyond the scope of this change, and I will let them 
behave
the way that they used to, to be potentially fixed later.

Bug:
https://bugs.openjdk.java.net/browse/JDK-8241825

Webrev:
http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/

Testing:
hs-tier1-7

Thanks,
/Erik

From richard.reingruber at sap.com  Fri Apr 24 08:18:31 2020
From: richard.reingruber at sap.com (Reingruber, Richard)
Date: Fri, 24 Apr 2020 08:18:31 +0000
Subject: RFR(S) 8238585: Use handshake for
 JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled
 methods on stack not_entrant
In-Reply-To: <AM0PR0202MB33318BC60E3460662D42A40D9B150@AM0PR0202MB3331.eurprd02.prod.outlook.com>
References: <AM0PR0202MB33311AB7623F4164335286309B1B0@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <3c59b9f9-ec38-18c9-8f24-e1186a08a04a@oracle.com>
 <AM0PR0202MB3331FD660B315CEC25B1DC209B150@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <410eed04-e2ef-0f4f-1c56-19e6734a10f6@oracle.com>
 <AM0PR0202MB33318BC60E3460662D42A40D9B150@AM0PR0202MB3331.eurprd02.prod.outlook.com>
Message-ID: <AM0PR0202MB33319C2F9595CC3C2FA1617B9BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>

Hi Patricio, Vladimir, and Serguei,

now that direct handshakes are available, I've updated the patch to make use of them.

In addition I have done some clean-up changes I missed in the first webrev.

Finally I have implemented the workaround suggested by Patricio to avoid nesting the handshake
into the vm operation VM_SetFramePop [1]

Kindly review again:

Webrev:        http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1/
Webrev(delta): http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1.inc/

I updated the JBS item explaining why the vm operation VM_EnterInterpOnlyMode can be replaced with a
direct handshake:

JBS: https://bugs.openjdk.java.net/browse/JDK-8238585

Testing:

* JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms.

* Submit-repo: mach5-one-rrich-JDK-8238585-20200423-1436-10441737

Thanks,
Richard.

[1] An assertion in Handshake::execute_direct() fails, if called be VMThread, because it is no JavaThread.

-----Original Message-----
From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of Reingruber, Richard
Sent: Freitag, 14. Februar 2020 19:47
To: Patricio Chilano <patricio.chilano.mateo at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
Subject: RE: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant

Hi Patricio,

  > > I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a
  > > handshake cannot be nested in a vm operation. Maybe it should be asserted in the
  > > Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation?
  > >
  > >    > Alternatively I think you could do something similar to what we do in
  > >    > Deoptimization::deoptimize_all_marked():
  > >    >
  > >    >    EnterInterpOnlyModeClosure hs;
  > >    >    if (SafepointSynchronize::is_at_safepoint()) {
  > >    >      hs.do_thread(state->get_thread());
  > >    >    } else {
  > >    >      Handshake::execute(&hs, state->get_thread());
  > >    >    }
  > >    > (you could pass ?EnterInterpOnlyModeClosure? directly to the
  > >    > HandshakeClosure() constructor)
  > >
  > > Maybe this could be used also in the Handshake::execute() methods as general solution?
  > Right, we could also do that. Avoiding to clear the polling page in 
  > HandshakeState::clear_handshake() should be enough to fix this issue and 
  > execute a handshake inside a safepoint, but adding that "if" statement 
  > in Hanshake::execute() sounds good to avoid all the extra code that we 
  > go through when executing a handshake. I filed 8239084 to make that change.

Thanks for taking care of this and creating the RFE.

  > 
  > >    > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
  > >    > always called in a nested operation or just sometimes.
  > >
  > > At least one execution path without vm operation exists:
  > >
  > >    JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void
  > >      JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong
  > >        JvmtiEventControllerPrivate::recompute_enabled() : void
  > >          JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches)
  > >            JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void
  > >              JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError
  > >                jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError
  > >
  > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a
  > > handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further
  > > encouraged to do it with a handshake :)
  > Ah! I think you can still do it with a handshake with the 
  > Deoptimization::deoptimize_all_marked() like solution. I can change the 
  > if-else statement with just the Handshake::execute() call in 8239084. 
  > But up to you.  : )

Well, I think that's enough encouragement :)
I'll wait for 8239084 and try then again.
(no urgency and all)

Thanks,
Richard.

-----Original Message-----
From: Patricio Chilano <patricio.chilano.mateo at oracle.com> 
Sent: Freitag, 14. Februar 2020 15:54
To: Reingruber, Richard <richard.reingruber at sap.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant

Hi Richard,

On 2/14/20 9:58 AM, Reingruber, Richard wrote:
> Hi Patricio,
>
> thanks for having a look.
>
>    > I?m only commenting on the handshake changes.
>    > I see that operation VM_EnterInterpOnlyMode can be called inside
>    > operation VM_SetFramePop which also allows nested operations. Here is a
>    > comment in VM_SetFramePop definition:
>    >
>    > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is
>    > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled.
>    >
>    > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we
>    > could have a handshake inside a safepoint operation. The issue I see
>    > there is that at the end of the handshake the polling page of the target
>    > thread could be disarmed. So if the target thread happens to be in a
>    > blocked state just transiently and wakes up then it will not stop for
>    > the ongoing safepoint. Maybe I can file an RFE to assert that the
>    > polling page is armed at the beginning of disarm_safepoint().
>
> I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a
> handshake cannot be nested in a vm operation. Maybe it should be asserted in the
> Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation?
>
>    > Alternatively I think you could do something similar to what we do in
>    > Deoptimization::deoptimize_all_marked():
>    >
>    >    EnterInterpOnlyModeClosure hs;
>    >    if (SafepointSynchronize::is_at_safepoint()) {
>    >      hs.do_thread(state->get_thread());
>    >    } else {
>    >      Handshake::execute(&hs, state->get_thread());
>    >    }
>    > (you could pass ?EnterInterpOnlyModeClosure? directly to the
>    > HandshakeClosure() constructor)
>
> Maybe this could be used also in the Handshake::execute() methods as general solution?
Right, we could also do that. Avoiding to clear the polling page in 
HandshakeState::clear_handshake() should be enough to fix this issue and 
execute a handshake inside a safepoint, but adding that "if" statement 
in Hanshake::execute() sounds good to avoid all the extra code that we 
go through when executing a handshake. I filed 8239084 to make that change.

>    > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>    > always called in a nested operation or just sometimes.
>
> At least one execution path without vm operation exists:
>
>    JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void
>      JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong
>        JvmtiEventControllerPrivate::recompute_enabled() : void
>          JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches)
>            JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void
>              JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError
>                jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError
>
> I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a
> handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further
> encouraged to do it with a handshake :)
Ah! I think you can still do it with a handshake with the 
Deoptimization::deoptimize_all_marked() like solution. I can change the 
if-else statement with just the Handshake::execute() call in 8239084. 
But up to you.? : )

Thanks,
Patricio
> Thanks again,
> Richard.
>
> -----Original Message-----
> From: Patricio Chilano <patricio.chilano.mateo at oracle.com>
> Sent: Donnerstag, 13. Februar 2020 18:47
> To: Reingruber, Richard <richard.reingruber at sap.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>
> Hi Richard,
>
> I?m only commenting on the handshake changes.
> I see that operation VM_EnterInterpOnlyMode can be called inside
> operation VM_SetFramePop which also allows nested operations. Here is a
> comment in VM_SetFramePop definition:
>
> // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is
> // called from the JvmtiEventControllerPrivate::recompute_thread_enabled.
>
> So if we change VM_EnterInterpOnlyMode to be a handshake, then now we
> could have a handshake inside a safepoint operation. The issue I see
> there is that at the end of the handshake the polling page of the target
> thread could be disarmed. So if the target thread happens to be in a
> blocked state just transiently and wakes up then it will not stop for
> the ongoing safepoint. Maybe I can file an RFE to assert that the
> polling page is armed at the beginning of disarm_safepoint().
>
> I think one option could be to remove
> SafepointMechanism::disarm_if_needed() in
> HandshakeState::clear_handshake() and let each JavaThread disarm itself
> for the handshake case.
>
> Alternatively I think you could do something similar to what we do in
> Deoptimization::deoptimize_all_marked():
>
>   ? EnterInterpOnlyModeClosure hs;
>   ? if (SafepointSynchronize::is_at_safepoint()) {
>   ??? hs.do_thread(state->get_thread());
>   ? } else {
>   ??? Handshake::execute(&hs, state->get_thread());
>   ? }
> (you could pass ?EnterInterpOnlyModeClosure? directly to the
> HandshakeClosure() constructor)
>
> I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
> always called in a nested operation or just sometimes.
>
> Thanks,
> Patricio
>
> On 2/12/20 7:23 AM, Reingruber, Richard wrote:
>> // Repost including hotspot runtime and gc lists.
>> // Dean Long suggested to do so, because the enhancement replaces a vm operation
>> // with a handshake.
>> // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html
>>
>> Hi,
>>
>> could I please get reviews for this small enhancement in hotspot's jvmti implementation:
>>
>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/
>> Bug:    https://bugs.openjdk.java.net/browse/JDK-8238585
>>
>> The change avoids making all compiled methods on stack not_entrant when switching a java thread to
>> interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack.
>>
>> Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations.
>>
>> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms.
>>
>> Thanks, Richard.
>>
>> See also my question if anyone knows a reason for making the compiled methods not_entrant:
>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html


From suenaga at oss.nttdata.com  Fri Apr 24 11:34:22 2020
From: suenaga at oss.nttdata.com (Yasumasa Suenaga)
Date: Fri, 24 Apr 2020 20:34:22 +0900
Subject: RFR(S) 8238585: Use handshake for
 JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled
 methods on stack not_entrant
In-Reply-To: <AM0PR0202MB33319C2F9595CC3C2FA1617B9BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
References: <AM0PR0202MB33311AB7623F4164335286309B1B0@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <3c59b9f9-ec38-18c9-8f24-e1186a08a04a@oracle.com>
 <AM0PR0202MB3331FD660B315CEC25B1DC209B150@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <410eed04-e2ef-0f4f-1c56-19e6734a10f6@oracle.com>
 <AM0PR0202MB33318BC60E3460662D42A40D9B150@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <AM0PR0202MB33319C2F9595CC3C2FA1617B9BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
Message-ID: <e1244807-74b4-755b-17c2-dce3263ce337@oss.nttdata.com>

Hi Richard,

I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
Does it help you? I think it gives you to remove workaround.

(The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.)


Thanks,

Yasumasa


On 2020/04/24 17:18, Reingruber, Richard wrote:
> Hi Patricio, Vladimir, and Serguei,
> 
> now that direct handshakes are available, I've updated the patch to make use of them.
> 
> In addition I have done some clean-up changes I missed in the first webrev.
> 
> Finally I have implemented the workaround suggested by Patricio to avoid nesting the handshake
> into the vm operation VM_SetFramePop [1]
> 
> Kindly review again:
> 
> Webrev:        http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1/
> Webrev(delta): http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1.inc/
> 
> I updated the JBS item explaining why the vm operation VM_EnterInterpOnlyMode can be replaced with a
> direct handshake:
> 
> JBS: https://bugs.openjdk.java.net/browse/JDK-8238585
> 
> Testing:
> 
> * JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms.
> 
> * Submit-repo: mach5-one-rrich-JDK-8238585-20200423-1436-10441737
> 
> Thanks,
> Richard.
> 
> [1] An assertion in Handshake::execute_direct() fails, if called be VMThread, because it is no JavaThread.
> 
> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of Reingruber, Richard
> Sent: Freitag, 14. Februar 2020 19:47
> To: Patricio Chilano <patricio.chilano.mateo at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
> Subject: RE: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
> 
> Hi Patricio,
> 
>    > > I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a
>    > > handshake cannot be nested in a vm operation. Maybe it should be asserted in the
>    > > Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation?
>    > >
>    > >    > Alternatively I think you could do something similar to what we do in
>    > >    > Deoptimization::deoptimize_all_marked():
>    > >    >
>    > >    >    EnterInterpOnlyModeClosure hs;
>    > >    >    if (SafepointSynchronize::is_at_safepoint()) {
>    > >    >      hs.do_thread(state->get_thread());
>    > >    >    } else {
>    > >    >      Handshake::execute(&hs, state->get_thread());
>    > >    >    }
>    > >    > (you could pass ?EnterInterpOnlyModeClosure? directly to the
>    > >    > HandshakeClosure() constructor)
>    > >
>    > > Maybe this could be used also in the Handshake::execute() methods as general solution?
>    > Right, we could also do that. Avoiding to clear the polling page in
>    > HandshakeState::clear_handshake() should be enough to fix this issue and
>    > execute a handshake inside a safepoint, but adding that "if" statement
>    > in Hanshake::execute() sounds good to avoid all the extra code that we
>    > go through when executing a handshake. I filed 8239084 to make that change.
> 
> Thanks for taking care of this and creating the RFE.
> 
>    >
>    > >    > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>    > >    > always called in a nested operation or just sometimes.
>    > >
>    > > At least one execution path without vm operation exists:
>    > >
>    > >    JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void
>    > >      JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong
>    > >        JvmtiEventControllerPrivate::recompute_enabled() : void
>    > >          JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches)
>    > >            JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void
>    > >              JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError
>    > >                jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError
>    > >
>    > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a
>    > > handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further
>    > > encouraged to do it with a handshake :)
>    > Ah! I think you can still do it with a handshake with the
>    > Deoptimization::deoptimize_all_marked() like solution. I can change the
>    > if-else statement with just the Handshake::execute() call in 8239084.
>    > But up to you.  : )
> 
> Well, I think that's enough encouragement :)
> I'll wait for 8239084 and try then again.
> (no urgency and all)
> 
> Thanks,
> Richard.
> 
> -----Original Message-----
> From: Patricio Chilano <patricio.chilano.mateo at oracle.com>
> Sent: Freitag, 14. Februar 2020 15:54
> To: Reingruber, Richard <richard.reingruber at sap.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
> 
> Hi Richard,
> 
> On 2/14/20 9:58 AM, Reingruber, Richard wrote:
>> Hi Patricio,
>>
>> thanks for having a look.
>>
>>     > I?m only commenting on the handshake changes.
>>     > I see that operation VM_EnterInterpOnlyMode can be called inside
>>     > operation VM_SetFramePop which also allows nested operations. Here is a
>>     > comment in VM_SetFramePop definition:
>>     >
>>     > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is
>>     > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled.
>>     >
>>     > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we
>>     > could have a handshake inside a safepoint operation. The issue I see
>>     > there is that at the end of the handshake the polling page of the target
>>     > thread could be disarmed. So if the target thread happens to be in a
>>     > blocked state just transiently and wakes up then it will not stop for
>>     > the ongoing safepoint. Maybe I can file an RFE to assert that the
>>     > polling page is armed at the beginning of disarm_safepoint().
>>
>> I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a
>> handshake cannot be nested in a vm operation. Maybe it should be asserted in the
>> Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation?
>>
>>     > Alternatively I think you could do something similar to what we do in
>>     > Deoptimization::deoptimize_all_marked():
>>     >
>>     >    EnterInterpOnlyModeClosure hs;
>>     >    if (SafepointSynchronize::is_at_safepoint()) {
>>     >      hs.do_thread(state->get_thread());
>>     >    } else {
>>     >      Handshake::execute(&hs, state->get_thread());
>>     >    }
>>     > (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>     > HandshakeClosure() constructor)
>>
>> Maybe this could be used also in the Handshake::execute() methods as general solution?
> Right, we could also do that. Avoiding to clear the polling page in
> HandshakeState::clear_handshake() should be enough to fix this issue and
> execute a handshake inside a safepoint, but adding that "if" statement
> in Hanshake::execute() sounds good to avoid all the extra code that we
> go through when executing a handshake. I filed 8239084 to make that change.
> 
>>     > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>     > always called in a nested operation or just sometimes.
>>
>> At least one execution path without vm operation exists:
>>
>>     JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void
>>       JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong
>>         JvmtiEventControllerPrivate::recompute_enabled() : void
>>           JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches)
>>             JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void
>>               JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError
>>                 jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError
>>
>> I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a
>> handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further
>> encouraged to do it with a handshake :)
> Ah! I think you can still do it with a handshake with the
> Deoptimization::deoptimize_all_marked() like solution. I can change the
> if-else statement with just the Handshake::execute() call in 8239084.
> But up to you.? : )
> 
> Thanks,
> Patricio
>> Thanks again,
>> Richard.
>>
>> -----Original Message-----
>> From: Patricio Chilano <patricio.chilano.mateo at oracle.com>
>> Sent: Donnerstag, 13. Februar 2020 18:47
>> To: Reingruber, Richard <richard.reingruber at sap.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>
>> Hi Richard,
>>
>> I?m only commenting on the handshake changes.
>> I see that operation VM_EnterInterpOnlyMode can be called inside
>> operation VM_SetFramePop which also allows nested operations. Here is a
>> comment in VM_SetFramePop definition:
>>
>> // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is
>> // called from the JvmtiEventControllerPrivate::recompute_thread_enabled.
>>
>> So if we change VM_EnterInterpOnlyMode to be a handshake, then now we
>> could have a handshake inside a safepoint operation. The issue I see
>> there is that at the end of the handshake the polling page of the target
>> thread could be disarmed. So if the target thread happens to be in a
>> blocked state just transiently and wakes up then it will not stop for
>> the ongoing safepoint. Maybe I can file an RFE to assert that the
>> polling page is armed at the beginning of disarm_safepoint().
>>
>> I think one option could be to remove
>> SafepointMechanism::disarm_if_needed() in
>> HandshakeState::clear_handshake() and let each JavaThread disarm itself
>> for the handshake case.
>>
>> Alternatively I think you could do something similar to what we do in
>> Deoptimization::deoptimize_all_marked():
>>
>>    ? EnterInterpOnlyModeClosure hs;
>>    ? if (SafepointSynchronize::is_at_safepoint()) {
>>    ??? hs.do_thread(state->get_thread());
>>    ? } else {
>>    ??? Handshake::execute(&hs, state->get_thread());
>>    ? }
>> (you could pass ?EnterInterpOnlyModeClosure? directly to the
>> HandshakeClosure() constructor)
>>
>> I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>> always called in a nested operation or just sometimes.
>>
>> Thanks,
>> Patricio
>>
>> On 2/12/20 7:23 AM, Reingruber, Richard wrote:
>>> // Repost including hotspot runtime and gc lists.
>>> // Dean Long suggested to do so, because the enhancement replaces a vm operation
>>> // with a handshake.
>>> // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html
>>>
>>> Hi,
>>>
>>> could I please get reviews for this small enhancement in hotspot's jvmti implementation:
>>>
>>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/
>>> Bug:    https://bugs.openjdk.java.net/browse/JDK-8238585
>>>
>>> The change avoids making all compiled methods on stack not_entrant when switching a java thread to
>>> interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack.
>>>
>>> Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations.
>>>
>>> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms.
>>>
>>> Thanks, Richard.
>>>
>>> See also my question if anyone knows a reason for making the compiled methods not_entrant:
>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html
> 

From frederic.parain at oracle.com  Fri Apr 24 13:28:28 2020
From: frederic.parain at oracle.com (Frederic Parain)
Date: Fri, 24 Apr 2020 09:28:28 -0400
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64
In-Reply-To: <56a0cf85-9698-64b2-95b4-e5b123c3e9e2@oracle.com>
References: <56a0cf85-9698-64b2-95b4-e5b123c3e9e2@oracle.com>
Message-ID: <B9EAEA33-057A-44C2-A7CF-DFB407776759@oracle.com>

Hi Erik,

Why did you removed the code handling classes with hard coded layouts?
This code was there to provide the freedom to play with layout algorithms
without having to deal with these special cases. Without this code, any
modification is now constrained by these special classes.

And I?d prefer to keep the dispatch in FieldLayoutBuilder::build_layout()
because we have more cases to handle there for the Valhalla project (the
layout algorithm for inline types is different than the one in
compute_regular_layout()).

Regards,

Fred


> On Apr 24, 2020, at 04:17, Erik ?sterlund <erik.osterlund at oracle.com> wrote:
> 
> Hi,
> 
> Today, the use of compressed class pointers assumes the use of compressed oops.
> This patch loosens up the relationship between compressed oops and compressed
> class pointers, so that compressed class pointers can be used without compressed
> oops. This benefits for example ZGC that wants compressed class pointers, but
> not compressed oops, effectively giving ZGC 4 bytes smaller objects.
> 
> Much of the complexity of the change is that r12 used to be used by compressed
> class pointers as some kind of semi-fast temp register that one would restore
> to the compressed oops heap base (or zero). That made it effectively a slightly
> optimized spilling mechanism used to find a temp register. I replaced that
> mechanism with a plain old normal temp register that you pass in as a parameter.
> The bad news is that I had to find available temp registers in a lot of places.
> The good news is that almost always were there temp registers available for free,
> and hence the new temp register is even faster than the old optimized spilling
> mechanism. Because we almost never need any spilling at all in the contexts where
> this is used.
> 
> Since I want the 4 new bytes to actually make objects smaller, I poked the new
> layout code a bit so that the klass gap is made available for fields. That used
> to be made available only with compressed oops enabled, due to restrictions in
> the layout code. Our new layout code does not have such restrictions, and so
> I will make the 4 bytes available for fields when the new layout code is used
> and compressed class pointers are used.
> 
> Now I'm only fixing this for HotSpot x86_64. Ideally the use of compressed oops
> and compressed class pointers should not be entangled in other platforms and
> Graal. But that is beyond the scope of this change, and I will let them behave
> the way that they used to, to be potentially fixed later.
> 
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8241825
> 
> Webrev:
> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/
> 
> Testing:
> hs-tier1-7
> 
> Thanks,
> /Erik


From erik.osterlund at oracle.com  Fri Apr 24 14:16:00 2020
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Fri, 24 Apr 2020 16:16:00 +0200
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64
In-Reply-To: <B9EAEA33-057A-44C2-A7CF-DFB407776759@oracle.com>
References: <56a0cf85-9698-64b2-95b4-e5b123c3e9e2@oracle.com>
 <B9EAEA33-057A-44C2-A7CF-DFB407776759@oracle.com>
Message-ID: <42f02810-7a6e-1c7d-3fe3-b892f007ddc4@oracle.com>

Hi Frederic,

Thanks for reviewing this.

On 2020-04-24 15:28, Frederic Parain wrote:
> Hi Erik,
>
> Why did you removed the code handling classes with hard coded layouts?
> This code was there to provide the freedom to play with layout algorithms
> without having to deal with these special cases. Without this code, any
> modification is now constrained by these special classes.

Good question. Since I started using the klass gap for fields, even when 
compressed oops is off, dealing with the
hardcoded offsets and layouts really became quite awkward, to the extent 
that I started wondering why do we even have
special layouts for these things anyway. I don't like special things. 
Treating it like any other object seemed far
easier, than matching the special special hardcoded layouts with the 
hardcoded offsets.

I wouldn't say that the whole layout algorithm is constrained by the 
fact that there are hardcoded offsets.
I would say that it's just a matter of whoever changes the layout 
algorithms such that the hardcoded offsets
are no longer the same, is just gonna have to update the offset to 
whatever the correct offsets are with the
new algorithm. And if you look at the classes where we actually do 
hardcode the offsets, they are rather vanilla
with relatively few ways you could change the layout other than changing 
it around for the sake of it. Especially
primitive box objects. My fantasy is quite limited how we expect a new 
layout algorithm is gonna do anything else
there really.

Philosophically, I really think it is up to the person hardcoding an 
offset to know what the offset is going to
be, ratherthan having completely different special (and worse) layouts 
produced for certain objects, just to
satisfy the expectations of the hardcoded offsets.Especially if the 
result is pretty bad memory waste in for
example Integer box objects, that can be quite plentyful in an application.

The code to figure out the right offsets if a very small amount of code, 
compared to the rather large amount of
code to create special layouts and syncing the special hardcoded layout 
code with the hardcoded offsets. That
is why my patch has a net negative amount of code.

Philosophically I also don't like having "special" layouts because we 
don't trust our ability to hardcode some
offset to the right value. Almost all "special" things are bad and 
create unnecessary complications, and hence
should have very good motivation for being allowed to be special. In 
this case, I really could not see any reason
why these layouts would be special, unless it was somehow super hard to 
figure out what the hardcoded offsets would be.
But for the classes we currently need hardcoded offsets, the 
calculations are trivial. So I don't think there is
enough motivation to have special (bad) layouts for these objects for 
that reason. It only makes things harder than
it needs to be, and makes the layout worse.

> And I?d prefer to keep the dispatch in FieldLayoutBuilder::build_layout()
> because we have more cases to handle there for the Valhalla project (the
> layout algorithm for inline types is different than the one in
> compute_regular_layout()).

Okay. There was no need for the dispatching any more since there was 
only the normal layout and nothing
else. But I can put that code back into the compute_regular_layout() 
function if that makes it easier for you
in Valhalla, and have FieldLayoutBuilder::build_layout() just call that 
function. Will update in the next webrev.

Thanks,
/Erik

> Regards,
>
> Fred
>
>
>> On Apr 24, 2020, at 04:17, Erik ?sterlund <erik.osterlund at oracle.com> wrote:
>>
>> Hi,
>>
>> Today, the use of compressed class pointers assumes the use of compressed oops.
>> This patch loosens up the relationship between compressed oops and compressed
>> class pointers, so that compressed class pointers can be used without compressed
>> oops. This benefits for example ZGC that wants compressed class pointers, but
>> not compressed oops, effectively giving ZGC 4 bytes smaller objects.
>>
>> Much of the complexity of the change is that r12 used to be used by compressed
>> class pointers as some kind of semi-fast temp register that one would restore
>> to the compressed oops heap base (or zero). That made it effectively a slightly
>> optimized spilling mechanism used to find a temp register. I replaced that
>> mechanism with a plain old normal temp register that you pass in as a parameter.
>> The bad news is that I had to find available temp registers in a lot of places.
>> The good news is that almost always were there temp registers available for free,
>> and hence the new temp register is even faster than the old optimized spilling
>> mechanism. Because we almost never need any spilling at all in the contexts where
>> this is used.
>>
>> Since I want the 4 new bytes to actually make objects smaller, I poked the new
>> layout code a bit so that the klass gap is made available for fields. That used
>> to be made available only with compressed oops enabled, due to restrictions in
>> the layout code. Our new layout code does not have such restrictions, and so
>> I will make the 4 bytes available for fields when the new layout code is used
>> and compressed class pointers are used.
>>
>> Now I'm only fixing this for HotSpot x86_64. Ideally the use of compressed oops
>> and compressed class pointers should not be entangled in other platforms and
>> Graal. But that is beyond the scope of this change, and I will let them behave
>> the way that they used to, to be potentially fixed later.
>>
>> Bug:
>> https://bugs.openjdk.java.net/browse/JDK-8241825
>>
>> Webrev:
>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/
>>
>> Testing:
>> hs-tier1-7
>>
>> Thanks,
>> /Erik


From coleen.phillimore at oracle.com  Fri Apr 24 14:36:14 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Fri, 24 Apr 2020 10:36:14 -0400
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64
In-Reply-To: <42f02810-7a6e-1c7d-3fe3-b892f007ddc4@oracle.com>
References: <56a0cf85-9698-64b2-95b4-e5b123c3e9e2@oracle.com>
 <B9EAEA33-057A-44C2-A7CF-DFB407776759@oracle.com>
 <42f02810-7a6e-1c7d-3fe3-b892f007ddc4@oracle.com>
Message-ID: <7cda1c27-e745-e3c5-8c0e-a05a588baa6e@oracle.com>



On 4/24/20 10:16 AM, Erik ?sterlund wrote:
> Hi Frederic,
>
> Thanks for reviewing this.
>
> On 2020-04-24 15:28, Frederic Parain wrote:
>> Hi Erik,
>>
>> Why did you removed the code handling classes with hard coded layouts?
>> This code was there to provide the freedom to play with layout 
>> algorithms
>> without having to deal with these special cases. Without this code, any
>> modification is now constrained by these special classes.
>
> Good question. Since I started using the klass gap for fields, even 
> when compressed oops is off, dealing with the
> hardcoded offsets and layouts really became quite awkward, to the 
> extent that I started wondering why do we even have
> special layouts for these things anyway. I don't like special things. 
> Treating it like any other object seemed far
> easier, than matching the special special hardcoded layouts with the 
> hardcoded offsets.
>
> I wouldn't say that the whole layout algorithm is constrained by the 
> fact that there are hardcoded offsets.
> I would say that it's just a matter of whoever changes the layout 
> algorithms such that the hardcoded offsets
> are no longer the same, is just gonna have to update the offset to 
> whatever the correct offsets are with the
> new algorithm. And if you look at the classes where we actually do 
> hardcode the offsets, they are rather vanilla
> with relatively few ways you could change the layout other than 
> changing it around for the sake of it. Especially
> primitive box objects. My fantasy is quite limited how we expect a new 
> layout algorithm is gonna do anything else
> there really.
>
> Philosophically, I really think it is up to the person hardcoding an 
> offset to know what the offset is going to
> be, ratherthan having completely different special (and worse) layouts 
> produced for certain objects, just to
> satisfy the expectations of the hardcoded offsets.Especially if the 
> result is pretty bad memory waste in for
> example Integer box objects, that can be quite plentyful in an 
> application.

We used to have a lot more special hardcoded layouts but most have been 
removed. The reason these few classes are hardcoded is because the 
interpreter generates code using these offsets before 
JavaClasses::compute_offsets is run.? We tried to get rid of all of them 
and maybe there's a way to indirectly refer to them in the generated 
code so we can do so in the future.
>
> The code to figure out the right offsets if a very small amount of 
> code, compared to the rather large amount of
> code to create special layouts and syncing the special hardcoded 
> layout code with the hardcoded offsets. That
> is why my patch has a net negative amount of code.
>
> Philosophically I also don't like having "special" layouts because we 
> don't trust our ability to hardcode some
> offset to the right value. Almost all "special" things are bad and 
> create unnecessary complications, and hence
> should have very good motivation for being allowed to be special. In 
> this case, I really could not see any reason
> why these layouts would be special, unless it was somehow super hard 
> to figure out what the hardcoded offsets would be.
> But for the classes we currently need hardcoded offsets, the 
> calculations are trivial. So I don't think there is
> enough motivation to have special (bad) layouts for these objects for 
> that reason. It only makes things harder than
> it needs to be, and makes the layout worse.
>
>> And I?d prefer to keep the dispatch in 
>> FieldLayoutBuilder::build_layout()
>> because we have more cases to handle there for the Valhalla project (the
>> layout algorithm for inline types is different than the one in
>> compute_regular_layout()).
>
> Okay. There was no need for the dispatching any more since there was 
> only the normal layout and nothing
> else. But I can put that code back into the compute_regular_layout() 
> function if that makes it easier for you
> in Valhalla, and have FieldLayoutBuilder::build_layout() just call 
> that function. Will update in the next webrev.

Thanks.
Coleen
>
> Thanks,
> /Erik
>
>> Regards,
>>
>> Fred
>>
>>
>>> On Apr 24, 2020, at 04:17, Erik ?sterlund 
>>> <erik.osterlund at oracle.com> wrote:
>>>
>>> Hi,
>>>
>>> Today, the use of compressed class pointers assumes the use of 
>>> compressed oops.
>>> This patch loosens up the relationship between compressed oops and 
>>> compressed
>>> class pointers, so that compressed class pointers can be used 
>>> without compressed
>>> oops. This benefits for example ZGC that wants compressed class 
>>> pointers, but
>>> not compressed oops, effectively giving ZGC 4 bytes smaller objects.
>>>
>>> Much of the complexity of the change is that r12 used to be used by 
>>> compressed
>>> class pointers as some kind of semi-fast temp register that one 
>>> would restore
>>> to the compressed oops heap base (or zero). That made it effectively 
>>> a slightly
>>> optimized spilling mechanism used to find a temp register. I 
>>> replaced that
>>> mechanism with a plain old normal temp register that you pass in as 
>>> a parameter.
>>> The bad news is that I had to find available temp registers in a lot 
>>> of places.
>>> The good news is that almost always were there temp registers 
>>> available for free,
>>> and hence the new temp register is even faster than the old 
>>> optimized spilling
>>> mechanism. Because we almost never need any spilling at all in the 
>>> contexts where
>>> this is used.
>>>
>>> Since I want the 4 new bytes to actually make objects smaller, I 
>>> poked the new
>>> layout code a bit so that the klass gap is made available for 
>>> fields. That used
>>> to be made available only with compressed oops enabled, due to 
>>> restrictions in
>>> the layout code. Our new layout code does not have such 
>>> restrictions, and so
>>> I will make the 4 bytes available for fields when the new layout 
>>> code is used
>>> and compressed class pointers are used.
>>>
>>> Now I'm only fixing this for HotSpot x86_64. Ideally the use of 
>>> compressed oops
>>> and compressed class pointers should not be entangled in other 
>>> platforms and
>>> Graal. But that is beyond the scope of this change, and I will let 
>>> them behave
>>> the way that they used to, to be potentially fixed later.
>>>
>>> Bug:
>>> https://bugs.openjdk.java.net/browse/JDK-8241825
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/
>>>
>>> Testing:
>>> hs-tier1-7
>>>
>>> Thanks,
>>> /Erik
>


From goetz.lindenmaier at sap.com  Fri Apr 24 14:36:13 2020
From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz)
Date: Fri, 24 Apr 2020 14:36:13 +0000
Subject: RFR(S): 8233014: Enable ShowCodeDetailsInExceptionMessages per
 default.
Message-ID: <AM4PR0201MB21469B9BDC047A9756AC1C6EECD00@AM4PR0201MB2146.eurprd02.prod.outlook.com>

Hi,

I would like to enable JEP 358 Helpful NullPointerExceptions by default:

http://cr.openjdk.java.net/~goetz/wr19/8233014-enable_NPE_per_default/02/

Please review.

The change passed our testing and jdk-submit.

Best regards,
 Goetz.

From richard.reingruber at sap.com  Fri Apr 24 14:44:29 2020
From: richard.reingruber at sap.com (Reingruber, Richard)
Date: Fri, 24 Apr 2020 14:44:29 +0000
Subject: RFR(S) 8238585: Use handshake for
 JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled
 methods on stack not_entrant
In-Reply-To: <e1244807-74b4-755b-17c2-dce3263ce337@oss.nttdata.com>
References: <AM0PR0202MB33311AB7623F4164335286309B1B0@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <3c59b9f9-ec38-18c9-8f24-e1186a08a04a@oracle.com>
 <AM0PR0202MB3331FD660B315CEC25B1DC209B150@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <410eed04-e2ef-0f4f-1c56-19e6734a10f6@oracle.com>
 <AM0PR0202MB33318BC60E3460662D42A40D9B150@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <AM0PR0202MB33319C2F9595CC3C2FA1617B9BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <e1244807-74b4-755b-17c2-dce3263ce337@oss.nttdata.com>
Message-ID: <AM0PR0202MB333172BA4EAF92D0BC3BA9379BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>

Hi Yasumasa,

> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
> Does it help you? I think it gives you to remove workaround.

I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake
you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to
change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes.

Also my first impression was that it won't be that easy from a synchronization point of view to
replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls
JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where
JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear
to me, how this has to be handled.

So it appears to me that it would be easier to push JDK-8242427 after this (JDK-8238585).

> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.)

Would be interesting to see how you handled the issues above :)

Thanks, Richard.

[1] See question in comment https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14302030&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14302030

-----Original Message-----
From: Yasumasa Suenaga <suenaga at oss.nttdata.com> 
Sent: Freitag, 24. April 2020 13:34
To: Reingruber, Richard <richard.reingruber at sap.com>; Patricio Chilano <patricio.chilano.mateo at oracle.com>; serguei.spitsyn at oracle.com; Vladimir Ivanov <vladimir.x.ivanov at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant

Hi Richard,

I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
Does it help you? I think it gives you to remove workaround.

(The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.)


Thanks,

Yasumasa


On 2020/04/24 17:18, Reingruber, Richard wrote:
> Hi Patricio, Vladimir, and Serguei,
> 
> now that direct handshakes are available, I've updated the patch to make use of them.
> 
> In addition I have done some clean-up changes I missed in the first webrev.
> 
> Finally I have implemented the workaround suggested by Patricio to avoid nesting the handshake
> into the vm operation VM_SetFramePop [1]
> 
> Kindly review again:
> 
> Webrev:        http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1/
> Webrev(delta): http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1.inc/
> 
> I updated the JBS item explaining why the vm operation VM_EnterInterpOnlyMode can be replaced with a
> direct handshake:
> 
> JBS: https://bugs.openjdk.java.net/browse/JDK-8238585
> 
> Testing:
> 
> * JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms.
> 
> * Submit-repo: mach5-one-rrich-JDK-8238585-20200423-1436-10441737
> 
> Thanks,
> Richard.
> 
> [1] An assertion in Handshake::execute_direct() fails, if called be VMThread, because it is no JavaThread.
> 
> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of Reingruber, Richard
> Sent: Freitag, 14. Februar 2020 19:47
> To: Patricio Chilano <patricio.chilano.mateo at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
> Subject: RE: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
> 
> Hi Patricio,
> 
>    > > I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a
>    > > handshake cannot be nested in a vm operation. Maybe it should be asserted in the
>    > > Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation?
>    > >
>    > >    > Alternatively I think you could do something similar to what we do in
>    > >    > Deoptimization::deoptimize_all_marked():
>    > >    >
>    > >    >    EnterInterpOnlyModeClosure hs;
>    > >    >    if (SafepointSynchronize::is_at_safepoint()) {
>    > >    >      hs.do_thread(state->get_thread());
>    > >    >    } else {
>    > >    >      Handshake::execute(&hs, state->get_thread());
>    > >    >    }
>    > >    > (you could pass ?EnterInterpOnlyModeClosure? directly to the
>    > >    > HandshakeClosure() constructor)
>    > >
>    > > Maybe this could be used also in the Handshake::execute() methods as general solution?
>    > Right, we could also do that. Avoiding to clear the polling page in
>    > HandshakeState::clear_handshake() should be enough to fix this issue and
>    > execute a handshake inside a safepoint, but adding that "if" statement
>    > in Hanshake::execute() sounds good to avoid all the extra code that we
>    > go through when executing a handshake. I filed 8239084 to make that change.
> 
> Thanks for taking care of this and creating the RFE.
> 
>    >
>    > >    > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>    > >    > always called in a nested operation or just sometimes.
>    > >
>    > > At least one execution path without vm operation exists:
>    > >
>    > >    JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void
>    > >      JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong
>    > >        JvmtiEventControllerPrivate::recompute_enabled() : void
>    > >          JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches)
>    > >            JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void
>    > >              JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError
>    > >                jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError
>    > >
>    > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a
>    > > handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further
>    > > encouraged to do it with a handshake :)
>    > Ah! I think you can still do it with a handshake with the
>    > Deoptimization::deoptimize_all_marked() like solution. I can change the
>    > if-else statement with just the Handshake::execute() call in 8239084.
>    > But up to you.  : )
> 
> Well, I think that's enough encouragement :)
> I'll wait for 8239084 and try then again.
> (no urgency and all)
> 
> Thanks,
> Richard.
> 
> -----Original Message-----
> From: Patricio Chilano <patricio.chilano.mateo at oracle.com>
> Sent: Freitag, 14. Februar 2020 15:54
> To: Reingruber, Richard <richard.reingruber at sap.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
> 
> Hi Richard,
> 
> On 2/14/20 9:58 AM, Reingruber, Richard wrote:
>> Hi Patricio,
>>
>> thanks for having a look.
>>
>>     > I?m only commenting on the handshake changes.
>>     > I see that operation VM_EnterInterpOnlyMode can be called inside
>>     > operation VM_SetFramePop which also allows nested operations. Here is a
>>     > comment in VM_SetFramePop definition:
>>     >
>>     > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is
>>     > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled.
>>     >
>>     > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we
>>     > could have a handshake inside a safepoint operation. The issue I see
>>     > there is that at the end of the handshake the polling page of the target
>>     > thread could be disarmed. So if the target thread happens to be in a
>>     > blocked state just transiently and wakes up then it will not stop for
>>     > the ongoing safepoint. Maybe I can file an RFE to assert that the
>>     > polling page is armed at the beginning of disarm_safepoint().
>>
>> I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a
>> handshake cannot be nested in a vm operation. Maybe it should be asserted in the
>> Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation?
>>
>>     > Alternatively I think you could do something similar to what we do in
>>     > Deoptimization::deoptimize_all_marked():
>>     >
>>     >    EnterInterpOnlyModeClosure hs;
>>     >    if (SafepointSynchronize::is_at_safepoint()) {
>>     >      hs.do_thread(state->get_thread());
>>     >    } else {
>>     >      Handshake::execute(&hs, state->get_thread());
>>     >    }
>>     > (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>     > HandshakeClosure() constructor)
>>
>> Maybe this could be used also in the Handshake::execute() methods as general solution?
> Right, we could also do that. Avoiding to clear the polling page in
> HandshakeState::clear_handshake() should be enough to fix this issue and
> execute a handshake inside a safepoint, but adding that "if" statement
> in Hanshake::execute() sounds good to avoid all the extra code that we
> go through when executing a handshake. I filed 8239084 to make that change.
> 
>>     > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>     > always called in a nested operation or just sometimes.
>>
>> At least one execution path without vm operation exists:
>>
>>     JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void
>>       JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong
>>         JvmtiEventControllerPrivate::recompute_enabled() : void
>>           JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches)
>>             JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void
>>               JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError
>>                 jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError
>>
>> I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a
>> handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further
>> encouraged to do it with a handshake :)
> Ah! I think you can still do it with a handshake with the
> Deoptimization::deoptimize_all_marked() like solution. I can change the
> if-else statement with just the Handshake::execute() call in 8239084.
> But up to you.? : )
> 
> Thanks,
> Patricio
>> Thanks again,
>> Richard.
>>
>> -----Original Message-----
>> From: Patricio Chilano <patricio.chilano.mateo at oracle.com>
>> Sent: Donnerstag, 13. Februar 2020 18:47
>> To: Reingruber, Richard <richard.reingruber at sap.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>
>> Hi Richard,
>>
>> I?m only commenting on the handshake changes.
>> I see that operation VM_EnterInterpOnlyMode can be called inside
>> operation VM_SetFramePop which also allows nested operations. Here is a
>> comment in VM_SetFramePop definition:
>>
>> // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is
>> // called from the JvmtiEventControllerPrivate::recompute_thread_enabled.
>>
>> So if we change VM_EnterInterpOnlyMode to be a handshake, then now we
>> could have a handshake inside a safepoint operation. The issue I see
>> there is that at the end of the handshake the polling page of the target
>> thread could be disarmed. So if the target thread happens to be in a
>> blocked state just transiently and wakes up then it will not stop for
>> the ongoing safepoint. Maybe I can file an RFE to assert that the
>> polling page is armed at the beginning of disarm_safepoint().
>>
>> I think one option could be to remove
>> SafepointMechanism::disarm_if_needed() in
>> HandshakeState::clear_handshake() and let each JavaThread disarm itself
>> for the handshake case.
>>
>> Alternatively I think you could do something similar to what we do in
>> Deoptimization::deoptimize_all_marked():
>>
>>    ? EnterInterpOnlyModeClosure hs;
>>    ? if (SafepointSynchronize::is_at_safepoint()) {
>>    ??? hs.do_thread(state->get_thread());
>>    ? } else {
>>    ??? Handshake::execute(&hs, state->get_thread());
>>    ? }
>> (you could pass ?EnterInterpOnlyModeClosure? directly to the
>> HandshakeClosure() constructor)
>>
>> I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>> always called in a nested operation or just sometimes.
>>
>> Thanks,
>> Patricio
>>
>> On 2/12/20 7:23 AM, Reingruber, Richard wrote:
>>> // Repost including hotspot runtime and gc lists.
>>> // Dean Long suggested to do so, because the enhancement replaces a vm operation
>>> // with a handshake.
>>> // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html
>>>
>>> Hi,
>>>
>>> could I please get reviews for this small enhancement in hotspot's jvmti implementation:
>>>
>>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/
>>> Bug:    https://bugs.openjdk.java.net/browse/JDK-8238585
>>>
>>> The change avoids making all compiled methods on stack not_entrant when switching a java thread to
>>> interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack.
>>>
>>> Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations.
>>>
>>> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms.
>>>
>>> Thanks, Richard.
>>>
>>> See also my question if anyone knows a reason for making the compiled methods not_entrant:
>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html
> 

From frederic.parain at oracle.com  Fri Apr 24 14:45:49 2020
From: frederic.parain at oracle.com (Frederic Parain)
Date: Fri, 24 Apr 2020 10:45:49 -0400
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64
In-Reply-To: <7cda1c27-e745-e3c5-8c0e-a05a588baa6e@oracle.com>
References: <56a0cf85-9698-64b2-95b4-e5b123c3e9e2@oracle.com>
 <B9EAEA33-057A-44C2-A7CF-DFB407776759@oracle.com>
 <42f02810-7a6e-1c7d-3fe3-b892f007ddc4@oracle.com>
 <7cda1c27-e745-e3c5-8c0e-a05a588baa6e@oracle.com>
Message-ID: <866D9669-4CFD-4ABC-BCC2-3F2B51BDDDA9@oracle.com>



> On Apr 24, 2020, at 10:36, coleen.phillimore at oracle.com wrote:
> 
> 
> 
> On 4/24/20 10:16 AM, Erik ?sterlund wrote:
>> Hi Frederic,
>> 
>> Thanks for reviewing this.
>> 
>> On 2020-04-24 15:28, Frederic Parain wrote:
>>> Hi Erik,
>>> 
>>> Why did you removed the code handling classes with hard coded layouts?
>>> This code was there to provide the freedom to play with layout algorithms
>>> without having to deal with these special cases. Without this code, any
>>> modification is now constrained by these special classes.
>> 
>> Good question. Since I started using the klass gap for fields, even when compressed oops is off, dealing with the
>> hardcoded offsets and layouts really became quite awkward, to the extent that I started wondering why do we even have
>> special layouts for these things anyway. I don't like special things. Treating it like any other object seemed far
>> easier, than matching the special special hardcoded layouts with the hardcoded offsets.
>> 
>> I wouldn't say that the whole layout algorithm is constrained by the fact that there are hardcoded offsets.
>> I would say that it's just a matter of whoever changes the layout algorithms such that the hardcoded offsets
>> are no longer the same, is just gonna have to update the offset to whatever the correct offsets are with the
>> new algorithm. And if you look at the classes where we actually do hardcode the offsets, they are rather vanilla
>> with relatively few ways you could change the layout other than changing it around for the sake of it. Especially
>> primitive box objects. My fantasy is quite limited how we expect a new layout algorithm is gonna do anything else
>> there really.
>> 
>> Philosophically, I really think it is up to the person hardcoding an offset to know what the offset is going to
>> be, ratherthan having completely different special (and worse) layouts produced for certain objects, just to
>> satisfy the expectations of the hardcoded offsets.Especially if the result is pretty bad memory waste in for
>> example Integer box objects, that can be quite plentyful in an application.
> 
> We used to have a lot more special hardcoded layouts but most have been removed. The reason these few classes are hardcoded is because the interpreter generates code using these offsets before JavaClasses::compute_offsets is run.  We tried to get rid of all of them and maybe there's a way to indirectly refer to them in the generated code so we can do so in the future.

The proposed changes make the whole system brittle. Restoring the special handling of these classes during
layout computation or making the generated code able to deal with dynamically computed offsets would be a
more robust solution.


>> 
>> The code to figure out the right offsets if a very small amount of code, compared to the rather large amount of
>> code to create special layouts and syncing the special hardcoded layout code with the hardcoded offsets. That
>> is why my patch has a net negative amount of code.
>> 
>> Philosophically I also don't like having "special" layouts because we don't trust our ability to hardcode some
>> offset to the right value. Almost all "special" things are bad and create unnecessary complications, and hence
>> should have very good motivation for being allowed to be special. In this case, I really could not see any reason
>> why these layouts would be special, unless it was somehow super hard to figure out what the hardcoded offsets would be.
>> But for the classes we currently need hardcoded offsets, the calculations are trivial. So I don't think there is
>> enough motivation to have special (bad) layouts for these objects for that reason. It only makes things harder than
>> it needs to be, and makes the layout worse.
>> 
>>> And I?d prefer to keep the dispatch in FieldLayoutBuilder::build_layout()
>>> because we have more cases to handle there for the Valhalla project (the
>>> layout algorithm for inline types is different than the one in
>>> compute_regular_layout()).
>> 
>> Okay. There was no need for the dispatching any more since there was only the normal layout and nothing
>> else. But I can put that code back into the compute_regular_layout() function if that makes it easier for you
>> in Valhalla, and have FieldLayoutBuilder::build_layout() just call that function. Will update in the next webrev.
> 

Thank you,

Fred

>> 
>> 
>>> Regards,
>>> 
>>> Fred
>>> 
>>> 
>>>> On Apr 24, 2020, at 04:17, Erik ?sterlund <erik.osterlund at oracle.com> wrote:
>>>> 
>>>> Hi,
>>>> 
>>>> Today, the use of compressed class pointers assumes the use of compressed oops.
>>>> This patch loosens up the relationship between compressed oops and compressed
>>>> class pointers, so that compressed class pointers can be used without compressed
>>>> oops. This benefits for example ZGC that wants compressed class pointers, but
>>>> not compressed oops, effectively giving ZGC 4 bytes smaller objects.
>>>> 
>>>> Much of the complexity of the change is that r12 used to be used by compressed
>>>> class pointers as some kind of semi-fast temp register that one would restore
>>>> to the compressed oops heap base (or zero). That made it effectively a slightly
>>>> optimized spilling mechanism used to find a temp register. I replaced that
>>>> mechanism with a plain old normal temp register that you pass in as a parameter.
>>>> The bad news is that I had to find available temp registers in a lot of places.
>>>> The good news is that almost always were there temp registers available for free,
>>>> and hence the new temp register is even faster than the old optimized spilling
>>>> mechanism. Because we almost never need any spilling at all in the contexts where
>>>> this is used.
>>>> 
>>>> Since I want the 4 new bytes to actually make objects smaller, I poked the new
>>>> layout code a bit so that the klass gap is made available for fields. That used
>>>> to be made available only with compressed oops enabled, due to restrictions in
>>>> the layout code. Our new layout code does not have such restrictions, and so
>>>> I will make the 4 bytes available for fields when the new layout code is used
>>>> and compressed class pointers are used.
>>>> 
>>>> Now I'm only fixing this for HotSpot x86_64. Ideally the use of compressed oops
>>>> and compressed class pointers should not be entangled in other platforms and
>>>> Graal. But that is beyond the scope of this change, and I will let them behave
>>>> the way that they used to, to be potentially fixed later.
>>>> 
>>>> Bug:
>>>> https://bugs.openjdk.java.net/browse/JDK-8241825
>>>> 
>>>> Webrev:
>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/
>>>> 
>>>> Testing:
>>>> hs-tier1-7
>>>> 
>>>> Thanks,
>>>> /Erik
>> 
> 


From erik.osterlund at oracle.com  Fri Apr 24 15:03:30 2020
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Fri, 24 Apr 2020 17:03:30 +0200
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64
In-Reply-To: <7cda1c27-e745-e3c5-8c0e-a05a588baa6e@oracle.com>
References: <56a0cf85-9698-64b2-95b4-e5b123c3e9e2@oracle.com>
 <B9EAEA33-057A-44C2-A7CF-DFB407776759@oracle.com>
 <42f02810-7a6e-1c7d-3fe3-b892f007ddc4@oracle.com>
 <7cda1c27-e745-e3c5-8c0e-a05a588baa6e@oracle.com>
Message-ID: <19791b81-6363-8e65-bde1-1f6f084d0d89@oracle.com>

Hi Coleen,

On 2020-04-24 16:36, coleen.phillimore at oracle.com wrote:
>
>
> On 4/24/20 10:16 AM, Erik ?sterlund wrote:
>> Hi Frederic,
>>
>> Thanks for reviewing this.
>>
>> On 2020-04-24 15:28, Frederic Parain wrote:
>>> Hi Erik,
>>>
>>> Why did you removed the code handling classes with hard coded layouts?
>>> This code was there to provide the freedom to play with layout 
>>> algorithms
>>> without having to deal with these special cases. Without this code, any
>>> modification is now constrained by these special classes.
>>
>> Good question. Since I started using the klass gap for fields, even 
>> when compressed oops is off, dealing with the
>> hardcoded offsets and layouts really became quite awkward, to the 
>> extent that I started wondering why do we even have
>> special layouts for these things anyway. I don't like special things. 
>> Treating it like any other object seemed far
>> easier, than matching the special special hardcoded layouts with the 
>> hardcoded offsets.
>>
>> I wouldn't say that the whole layout algorithm is constrained by the 
>> fact that there are hardcoded offsets.
>> I would say that it's just a matter of whoever changes the layout 
>> algorithms such that the hardcoded offsets
>> are no longer the same, is just gonna have to update the offset to 
>> whatever the correct offsets are with the
>> new algorithm. And if you look at the classes where we actually do 
>> hardcode the offsets, they are rather vanilla
>> with relatively few ways you could change the layout other than 
>> changing it around for the sake of it. Especially
>> primitive box objects. My fantasy is quite limited how we expect a 
>> new layout algorithm is gonna do anything else
>> there really.
>>
>> Philosophically, I really think it is up to the person hardcoding an 
>> offset to know what the offset is going to
>> be, ratherthan having completely different special (and worse) 
>> layouts produced for certain objects, just to
>> satisfy the expectations of the hardcoded offsets.Especially if the 
>> result is pretty bad memory waste in for
>> example Integer box objects, that can be quite plentyful in an 
>> application.
>
> We used to have a lot more special hardcoded layouts but most have 
> been removed. The reason these few classes are hardcoded is because 
> the interpreter generates code using these offsets before 
> JavaClasses::compute_offsets is run.? We tried to get rid of all of 
> them and maybe there's a way to indirectly refer to them in the 
> generated code so we can do so in the future.

Makes sense. If the hardcoded offsets of the few remaining classes ever 
become annoying to a future layout
algorithm, then removing the hardcoding seems like a good option.

Thanks,
/Erik

From suenaga at oss.nttdata.com  Fri Apr 24 15:23:06 2020
From: suenaga at oss.nttdata.com (Yasumasa Suenaga)
Date: Sat, 25 Apr 2020 00:23:06 +0900
Subject: RFR(S) 8238585: Use handshake for
 JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled
 methods on stack not_entrant
In-Reply-To: <AM0PR0202MB333172BA4EAF92D0BC3BA9379BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
References: <AM0PR0202MB33311AB7623F4164335286309B1B0@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <3c59b9f9-ec38-18c9-8f24-e1186a08a04a@oracle.com>
 <AM0PR0202MB3331FD660B315CEC25B1DC209B150@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <410eed04-e2ef-0f4f-1c56-19e6734a10f6@oracle.com>
 <AM0PR0202MB33318BC60E3460662D42A40D9B150@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <AM0PR0202MB33319C2F9595CC3C2FA1617B9BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <e1244807-74b4-755b-17c2-dce3263ce337@oss.nttdata.com>
 <AM0PR0202MB333172BA4EAF92D0BC3BA9379BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
Message-ID: <81d7caa8-4244-85f3-4d4e-78117fe5e25b@oss.nttdata.com>

Hi Richard,

On 2020/04/24 23:44, Reingruber, Richard wrote:
> Hi Yasumasa,
> 
>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
>> Does it help you? I think it gives you to remove workaround.
> 
> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake
> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to
> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes.

Thanks for your information.
I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop.
I will modify and will test it after yours.


> Also my first impression was that it won't be that easy from a synchronization point of view to
> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls
> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where
> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear
> to me, how this has to be handled.

I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event.


Thanks,

Yasumasa


> So it appears to me that it would be easier to push JDK-8242427 after this (JDK-8238585).
> 
>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.)
> 
> Would be interesting to see how you handled the issues above :)
> 
> Thanks, Richard.
> 
> [1] See question in comment https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14302030&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14302030
> 
> -----Original Message-----
> From: Yasumasa Suenaga <suenaga at oss.nttdata.com>
> Sent: Freitag, 24. April 2020 13:34
> To: Reingruber, Richard <richard.reingruber at sap.com>; Patricio Chilano <patricio.chilano.mateo at oracle.com>; serguei.spitsyn at oracle.com; Vladimir Ivanov <vladimir.x.ivanov at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
> 
> Hi Richard,
> 
> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
> Does it help you? I think it gives you to remove workaround.
> 
> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.)
> 
> 
> Thanks,
> 
> Yasumasa
> 
> 
> On 2020/04/24 17:18, Reingruber, Richard wrote:
>> Hi Patricio, Vladimir, and Serguei,
>>
>> now that direct handshakes are available, I've updated the patch to make use of them.
>>
>> In addition I have done some clean-up changes I missed in the first webrev.
>>
>> Finally I have implemented the workaround suggested by Patricio to avoid nesting the handshake
>> into the vm operation VM_SetFramePop [1]
>>
>> Kindly review again:
>>
>> Webrev:        http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1/
>> Webrev(delta): http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1.inc/
>>
>> I updated the JBS item explaining why the vm operation VM_EnterInterpOnlyMode can be replaced with a
>> direct handshake:
>>
>> JBS: https://bugs.openjdk.java.net/browse/JDK-8238585
>>
>> Testing:
>>
>> * JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms.
>>
>> * Submit-repo: mach5-one-rrich-JDK-8238585-20200423-1436-10441737
>>
>> Thanks,
>> Richard.
>>
>> [1] An assertion in Handshake::execute_direct() fails, if called be VMThread, because it is no JavaThread.
>>
>> -----Original Message-----
>> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of Reingruber, Richard
>> Sent: Freitag, 14. Februar 2020 19:47
>> To: Patricio Chilano <patricio.chilano.mateo at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>> Subject: RE: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>
>> Hi Patricio,
>>
>>     > > I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a
>>     > > handshake cannot be nested in a vm operation. Maybe it should be asserted in the
>>     > > Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation?
>>     > >
>>     > >    > Alternatively I think you could do something similar to what we do in
>>     > >    > Deoptimization::deoptimize_all_marked():
>>     > >    >
>>     > >    >    EnterInterpOnlyModeClosure hs;
>>     > >    >    if (SafepointSynchronize::is_at_safepoint()) {
>>     > >    >      hs.do_thread(state->get_thread());
>>     > >    >    } else {
>>     > >    >      Handshake::execute(&hs, state->get_thread());
>>     > >    >    }
>>     > >    > (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>     > >    > HandshakeClosure() constructor)
>>     > >
>>     > > Maybe this could be used also in the Handshake::execute() methods as general solution?
>>     > Right, we could also do that. Avoiding to clear the polling page in
>>     > HandshakeState::clear_handshake() should be enough to fix this issue and
>>     > execute a handshake inside a safepoint, but adding that "if" statement
>>     > in Hanshake::execute() sounds good to avoid all the extra code that we
>>     > go through when executing a handshake. I filed 8239084 to make that change.
>>
>> Thanks for taking care of this and creating the RFE.
>>
>>     >
>>     > >    > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>     > >    > always called in a nested operation or just sometimes.
>>     > >
>>     > > At least one execution path without vm operation exists:
>>     > >
>>     > >    JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void
>>     > >      JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong
>>     > >        JvmtiEventControllerPrivate::recompute_enabled() : void
>>     > >          JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches)
>>     > >            JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void
>>     > >              JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError
>>     > >                jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError
>>     > >
>>     > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a
>>     > > handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further
>>     > > encouraged to do it with a handshake :)
>>     > Ah! I think you can still do it with a handshake with the
>>     > Deoptimization::deoptimize_all_marked() like solution. I can change the
>>     > if-else statement with just the Handshake::execute() call in 8239084.
>>     > But up to you.  : )
>>
>> Well, I think that's enough encouragement :)
>> I'll wait for 8239084 and try then again.
>> (no urgency and all)
>>
>> Thanks,
>> Richard.
>>
>> -----Original Message-----
>> From: Patricio Chilano <patricio.chilano.mateo at oracle.com>
>> Sent: Freitag, 14. Februar 2020 15:54
>> To: Reingruber, Richard <richard.reingruber at sap.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>
>> Hi Richard,
>>
>> On 2/14/20 9:58 AM, Reingruber, Richard wrote:
>>> Hi Patricio,
>>>
>>> thanks for having a look.
>>>
>>>      > I?m only commenting on the handshake changes.
>>>      > I see that operation VM_EnterInterpOnlyMode can be called inside
>>>      > operation VM_SetFramePop which also allows nested operations. Here is a
>>>      > comment in VM_SetFramePop definition:
>>>      >
>>>      > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is
>>>      > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled.
>>>      >
>>>      > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we
>>>      > could have a handshake inside a safepoint operation. The issue I see
>>>      > there is that at the end of the handshake the polling page of the target
>>>      > thread could be disarmed. So if the target thread happens to be in a
>>>      > blocked state just transiently and wakes up then it will not stop for
>>>      > the ongoing safepoint. Maybe I can file an RFE to assert that the
>>>      > polling page is armed at the beginning of disarm_safepoint().
>>>
>>> I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a
>>> handshake cannot be nested in a vm operation. Maybe it should be asserted in the
>>> Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation?
>>>
>>>      > Alternatively I think you could do something similar to what we do in
>>>      > Deoptimization::deoptimize_all_marked():
>>>      >
>>>      >    EnterInterpOnlyModeClosure hs;
>>>      >    if (SafepointSynchronize::is_at_safepoint()) {
>>>      >      hs.do_thread(state->get_thread());
>>>      >    } else {
>>>      >      Handshake::execute(&hs, state->get_thread());
>>>      >    }
>>>      > (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>>      > HandshakeClosure() constructor)
>>>
>>> Maybe this could be used also in the Handshake::execute() methods as general solution?
>> Right, we could also do that. Avoiding to clear the polling page in
>> HandshakeState::clear_handshake() should be enough to fix this issue and
>> execute a handshake inside a safepoint, but adding that "if" statement
>> in Hanshake::execute() sounds good to avoid all the extra code that we
>> go through when executing a handshake. I filed 8239084 to make that change.
>>
>>>      > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>>      > always called in a nested operation or just sometimes.
>>>
>>> At least one execution path without vm operation exists:
>>>
>>>      JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void
>>>        JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong
>>>          JvmtiEventControllerPrivate::recompute_enabled() : void
>>>            JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches)
>>>              JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void
>>>                JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError
>>>                  jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError
>>>
>>> I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a
>>> handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further
>>> encouraged to do it with a handshake :)
>> Ah! I think you can still do it with a handshake with the
>> Deoptimization::deoptimize_all_marked() like solution. I can change the
>> if-else statement with just the Handshake::execute() call in 8239084.
>> But up to you.? : )
>>
>> Thanks,
>> Patricio
>>> Thanks again,
>>> Richard.
>>>
>>> -----Original Message-----
>>> From: Patricio Chilano <patricio.chilano.mateo at oracle.com>
>>> Sent: Donnerstag, 13. Februar 2020 18:47
>>> To: Reingruber, Richard <richard.reingruber at sap.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>>
>>> Hi Richard,
>>>
>>> I?m only commenting on the handshake changes.
>>> I see that operation VM_EnterInterpOnlyMode can be called inside
>>> operation VM_SetFramePop which also allows nested operations. Here is a
>>> comment in VM_SetFramePop definition:
>>>
>>> // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is
>>> // called from the JvmtiEventControllerPrivate::recompute_thread_enabled.
>>>
>>> So if we change VM_EnterInterpOnlyMode to be a handshake, then now we
>>> could have a handshake inside a safepoint operation. The issue I see
>>> there is that at the end of the handshake the polling page of the target
>>> thread could be disarmed. So if the target thread happens to be in a
>>> blocked state just transiently and wakes up then it will not stop for
>>> the ongoing safepoint. Maybe I can file an RFE to assert that the
>>> polling page is armed at the beginning of disarm_safepoint().
>>>
>>> I think one option could be to remove
>>> SafepointMechanism::disarm_if_needed() in
>>> HandshakeState::clear_handshake() and let each JavaThread disarm itself
>>> for the handshake case.
>>>
>>> Alternatively I think you could do something similar to what we do in
>>> Deoptimization::deoptimize_all_marked():
>>>
>>>     ? EnterInterpOnlyModeClosure hs;
>>>     ? if (SafepointSynchronize::is_at_safepoint()) {
>>>     ??? hs.do_thread(state->get_thread());
>>>     ? } else {
>>>     ??? Handshake::execute(&hs, state->get_thread());
>>>     ? }
>>> (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>> HandshakeClosure() constructor)
>>>
>>> I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>> always called in a nested operation or just sometimes.
>>>
>>> Thanks,
>>> Patricio
>>>
>>> On 2/12/20 7:23 AM, Reingruber, Richard wrote:
>>>> // Repost including hotspot runtime and gc lists.
>>>> // Dean Long suggested to do so, because the enhancement replaces a vm operation
>>>> // with a handshake.
>>>> // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html
>>>>
>>>> Hi,
>>>>
>>>> could I please get reviews for this small enhancement in hotspot's jvmti implementation:
>>>>
>>>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/
>>>> Bug:    https://bugs.openjdk.java.net/browse/JDK-8238585
>>>>
>>>> The change avoids making all compiled methods on stack not_entrant when switching a java thread to
>>>> interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack.
>>>>
>>>> Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations.
>>>>
>>>> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms.
>>>>
>>>> Thanks, Richard.
>>>>
>>>> See also my question if anyone knows a reason for making the compiled methods not_entrant:
>>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html
>>

From richard.reingruber at sap.com  Fri Apr 24 16:08:57 2020
From: richard.reingruber at sap.com (Reingruber, Richard)
Date: Fri, 24 Apr 2020 16:08:57 +0000
Subject: RFR(S) 8238585: Use handshake for
 JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled
 methods on stack not_entrant
In-Reply-To: <81d7caa8-4244-85f3-4d4e-78117fe5e25b@oss.nttdata.com>
References: <AM0PR0202MB33311AB7623F4164335286309B1B0@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <3c59b9f9-ec38-18c9-8f24-e1186a08a04a@oracle.com>
 <AM0PR0202MB3331FD660B315CEC25B1DC209B150@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <410eed04-e2ef-0f4f-1c56-19e6734a10f6@oracle.com>
 <AM0PR0202MB33318BC60E3460662D42A40D9B150@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <AM0PR0202MB33319C2F9595CC3C2FA1617B9BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <e1244807-74b4-755b-17c2-dce3263ce337@oss.nttdata.com>
 <AM0PR0202MB333172BA4EAF92D0BC3BA9379BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <81d7caa8-4244-85f3-4d4e-78117fe5e25b@oss.nttdata.com>
Message-ID: <AM0PR0202MB3331324D56A68E08CE7CDAE69BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>

Hi Yasumasa, Patricio,

> >> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
> >> Does it help you? I think it gives you to remove workaround.
> > 
> > I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake
> > you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to
> > change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes.

> Thanks for your information.
> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop.
> I will modify and will test it after yours.

Thanks :)

> > Also my first impression was that it won't be that easy from a synchronization point of view to
> > replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls
> > JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where
> > JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear
> > to me, how this has to be handled.

> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event.

Yes. To me it is unclear what synchronization is necessary, if it is called during a handshake. And
also I'm unsure if a thread should do safepoint checks while executing a handshake.

@Patricio, coming back to my question [1]:

In the example you gave in your answer [2]: the java thread would execute a vm operation during a
direct handshake operation, while the VMThread is actually in the middle of a VM_HandshakeAllThreads
operation, waiting to handshake the same handshakee: why can't the VMThread just proceed? The
handshakee would be safepoint safe, wouldn't it?

Thanks, Richard.

[1] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301677&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301677

[2] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301763&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301763

-----Original Message-----
From: Yasumasa Suenaga <suenaga at oss.nttdata.com> 
Sent: Freitag, 24. April 2020 17:23
To: Reingruber, Richard <richard.reingruber at sap.com>; Patricio Chilano <patricio.chilano.mateo at oracle.com>; serguei.spitsyn at oracle.com; Vladimir Ivanov <vladimir.x.ivanov at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant

Hi Richard,

On 2020/04/24 23:44, Reingruber, Richard wrote:
> Hi Yasumasa,
> 
>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
>> Does it help you? I think it gives you to remove workaround.
> 
> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake
> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to
> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes.

Thanks for your information.
I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop.
I will modify and will test it after yours.


> Also my first impression was that it won't be that easy from a synchronization point of view to
> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls
> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where
> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear
> to me, how this has to be handled.

I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event.


Thanks,

Yasumasa


> So it appears to me that it would be easier to push JDK-8242427 after this (JDK-8238585).
> 
>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.)
> 
> Would be interesting to see how you handled the issues above :)
> 
> Thanks, Richard.
> 
> [1] See question in comment https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14302030&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14302030
> 
> -----Original Message-----
> From: Yasumasa Suenaga <suenaga at oss.nttdata.com>
> Sent: Freitag, 24. April 2020 13:34
> To: Reingruber, Richard <richard.reingruber at sap.com>; Patricio Chilano <patricio.chilano.mateo at oracle.com>; serguei.spitsyn at oracle.com; Vladimir Ivanov <vladimir.x.ivanov at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
> 
> Hi Richard,
> 
> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
> Does it help you? I think it gives you to remove workaround.
> 
> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.)
> 
> 
> Thanks,
> 
> Yasumasa
> 
> 
> On 2020/04/24 17:18, Reingruber, Richard wrote:
>> Hi Patricio, Vladimir, and Serguei,
>>
>> now that direct handshakes are available, I've updated the patch to make use of them.
>>
>> In addition I have done some clean-up changes I missed in the first webrev.
>>
>> Finally I have implemented the workaround suggested by Patricio to avoid nesting the handshake
>> into the vm operation VM_SetFramePop [1]
>>
>> Kindly review again:
>>
>> Webrev:        http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1/
>> Webrev(delta): http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1.inc/
>>
>> I updated the JBS item explaining why the vm operation VM_EnterInterpOnlyMode can be replaced with a
>> direct handshake:
>>
>> JBS: https://bugs.openjdk.java.net/browse/JDK-8238585
>>
>> Testing:
>>
>> * JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms.
>>
>> * Submit-repo: mach5-one-rrich-JDK-8238585-20200423-1436-10441737
>>
>> Thanks,
>> Richard.
>>
>> [1] An assertion in Handshake::execute_direct() fails, if called be VMThread, because it is no JavaThread.
>>
>> -----Original Message-----
>> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of Reingruber, Richard
>> Sent: Freitag, 14. Februar 2020 19:47
>> To: Patricio Chilano <patricio.chilano.mateo at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>> Subject: RE: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>
>> Hi Patricio,
>>
>>     > > I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a
>>     > > handshake cannot be nested in a vm operation. Maybe it should be asserted in the
>>     > > Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation?
>>     > >
>>     > >    > Alternatively I think you could do something similar to what we do in
>>     > >    > Deoptimization::deoptimize_all_marked():
>>     > >    >
>>     > >    >    EnterInterpOnlyModeClosure hs;
>>     > >    >    if (SafepointSynchronize::is_at_safepoint()) {
>>     > >    >      hs.do_thread(state->get_thread());
>>     > >    >    } else {
>>     > >    >      Handshake::execute(&hs, state->get_thread());
>>     > >    >    }
>>     > >    > (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>     > >    > HandshakeClosure() constructor)
>>     > >
>>     > > Maybe this could be used also in the Handshake::execute() methods as general solution?
>>     > Right, we could also do that. Avoiding to clear the polling page in
>>     > HandshakeState::clear_handshake() should be enough to fix this issue and
>>     > execute a handshake inside a safepoint, but adding that "if" statement
>>     > in Hanshake::execute() sounds good to avoid all the extra code that we
>>     > go through when executing a handshake. I filed 8239084 to make that change.
>>
>> Thanks for taking care of this and creating the RFE.
>>
>>     >
>>     > >    > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>     > >    > always called in a nested operation or just sometimes.
>>     > >
>>     > > At least one execution path without vm operation exists:
>>     > >
>>     > >    JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void
>>     > >      JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong
>>     > >        JvmtiEventControllerPrivate::recompute_enabled() : void
>>     > >          JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches)
>>     > >            JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void
>>     > >              JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError
>>     > >                jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError
>>     > >
>>     > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a
>>     > > handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further
>>     > > encouraged to do it with a handshake :)
>>     > Ah! I think you can still do it with a handshake with the
>>     > Deoptimization::deoptimize_all_marked() like solution. I can change the
>>     > if-else statement with just the Handshake::execute() call in 8239084.
>>     > But up to you.  : )
>>
>> Well, I think that's enough encouragement :)
>> I'll wait for 8239084 and try then again.
>> (no urgency and all)
>>
>> Thanks,
>> Richard.
>>
>> -----Original Message-----
>> From: Patricio Chilano <patricio.chilano.mateo at oracle.com>
>> Sent: Freitag, 14. Februar 2020 15:54
>> To: Reingruber, Richard <richard.reingruber at sap.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>
>> Hi Richard,
>>
>> On 2/14/20 9:58 AM, Reingruber, Richard wrote:
>>> Hi Patricio,
>>>
>>> thanks for having a look.
>>>
>>>      > I?m only commenting on the handshake changes.
>>>      > I see that operation VM_EnterInterpOnlyMode can be called inside
>>>      > operation VM_SetFramePop which also allows nested operations. Here is a
>>>      > comment in VM_SetFramePop definition:
>>>      >
>>>      > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is
>>>      > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled.
>>>      >
>>>      > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we
>>>      > could have a handshake inside a safepoint operation. The issue I see
>>>      > there is that at the end of the handshake the polling page of the target
>>>      > thread could be disarmed. So if the target thread happens to be in a
>>>      > blocked state just transiently and wakes up then it will not stop for
>>>      > the ongoing safepoint. Maybe I can file an RFE to assert that the
>>>      > polling page is armed at the beginning of disarm_safepoint().
>>>
>>> I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a
>>> handshake cannot be nested in a vm operation. Maybe it should be asserted in the
>>> Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation?
>>>
>>>      > Alternatively I think you could do something similar to what we do in
>>>      > Deoptimization::deoptimize_all_marked():
>>>      >
>>>      >    EnterInterpOnlyModeClosure hs;
>>>      >    if (SafepointSynchronize::is_at_safepoint()) {
>>>      >      hs.do_thread(state->get_thread());
>>>      >    } else {
>>>      >      Handshake::execute(&hs, state->get_thread());
>>>      >    }
>>>      > (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>>      > HandshakeClosure() constructor)
>>>
>>> Maybe this could be used also in the Handshake::execute() methods as general solution?
>> Right, we could also do that. Avoiding to clear the polling page in
>> HandshakeState::clear_handshake() should be enough to fix this issue and
>> execute a handshake inside a safepoint, but adding that "if" statement
>> in Hanshake::execute() sounds good to avoid all the extra code that we
>> go through when executing a handshake. I filed 8239084 to make that change.
>>
>>>      > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>>      > always called in a nested operation or just sometimes.
>>>
>>> At least one execution path without vm operation exists:
>>>
>>>      JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void
>>>        JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong
>>>          JvmtiEventControllerPrivate::recompute_enabled() : void
>>>            JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches)
>>>              JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void
>>>                JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError
>>>                  jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError
>>>
>>> I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a
>>> handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further
>>> encouraged to do it with a handshake :)
>> Ah! I think you can still do it with a handshake with the
>> Deoptimization::deoptimize_all_marked() like solution. I can change the
>> if-else statement with just the Handshake::execute() call in 8239084.
>> But up to you.? : )
>>
>> Thanks,
>> Patricio
>>> Thanks again,
>>> Richard.
>>>
>>> -----Original Message-----
>>> From: Patricio Chilano <patricio.chilano.mateo at oracle.com>
>>> Sent: Donnerstag, 13. Februar 2020 18:47
>>> To: Reingruber, Richard <richard.reingruber at sap.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>>
>>> Hi Richard,
>>>
>>> I?m only commenting on the handshake changes.
>>> I see that operation VM_EnterInterpOnlyMode can be called inside
>>> operation VM_SetFramePop which also allows nested operations. Here is a
>>> comment in VM_SetFramePop definition:
>>>
>>> // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is
>>> // called from the JvmtiEventControllerPrivate::recompute_thread_enabled.
>>>
>>> So if we change VM_EnterInterpOnlyMode to be a handshake, then now we
>>> could have a handshake inside a safepoint operation. The issue I see
>>> there is that at the end of the handshake the polling page of the target
>>> thread could be disarmed. So if the target thread happens to be in a
>>> blocked state just transiently and wakes up then it will not stop for
>>> the ongoing safepoint. Maybe I can file an RFE to assert that the
>>> polling page is armed at the beginning of disarm_safepoint().
>>>
>>> I think one option could be to remove
>>> SafepointMechanism::disarm_if_needed() in
>>> HandshakeState::clear_handshake() and let each JavaThread disarm itself
>>> for the handshake case.
>>>
>>> Alternatively I think you could do something similar to what we do in
>>> Deoptimization::deoptimize_all_marked():
>>>
>>>     ? EnterInterpOnlyModeClosure hs;
>>>     ? if (SafepointSynchronize::is_at_safepoint()) {
>>>     ??? hs.do_thread(state->get_thread());
>>>     ? } else {
>>>     ??? Handshake::execute(&hs, state->get_thread());
>>>     ? }
>>> (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>> HandshakeClosure() constructor)
>>>
>>> I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>> always called in a nested operation or just sometimes.
>>>
>>> Thanks,
>>> Patricio
>>>
>>> On 2/12/20 7:23 AM, Reingruber, Richard wrote:
>>>> // Repost including hotspot runtime and gc lists.
>>>> // Dean Long suggested to do so, because the enhancement replaces a vm operation
>>>> // with a handshake.
>>>> // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html
>>>>
>>>> Hi,
>>>>
>>>> could I please get reviews for this small enhancement in hotspot's jvmti implementation:
>>>>
>>>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/
>>>> Bug:    https://bugs.openjdk.java.net/browse/JDK-8238585
>>>>
>>>> The change avoids making all compiled methods on stack not_entrant when switching a java thread to
>>>> interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack.
>>>>
>>>> Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations.
>>>>
>>>> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms.
>>>>
>>>> Thanks, Richard.
>>>>
>>>> See also my question if anyone knows a reason for making the compiled methods not_entrant:
>>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html
>>

From patricio.chilano.mateo at oracle.com  Fri Apr 24 17:13:43 2020
From: patricio.chilano.mateo at oracle.com (Patricio Chilano)
Date: Fri, 24 Apr 2020 14:13:43 -0300
Subject: RFR(S) 8238585: Use handshake for
 JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled
 methods on stack not_entrant
In-Reply-To: <AM0PR0202MB3331324D56A68E08CE7CDAE69BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
References: <AM0PR0202MB33311AB7623F4164335286309B1B0@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <3c59b9f9-ec38-18c9-8f24-e1186a08a04a@oracle.com>
 <AM0PR0202MB3331FD660B315CEC25B1DC209B150@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <410eed04-e2ef-0f4f-1c56-19e6734a10f6@oracle.com>
 <AM0PR0202MB33318BC60E3460662D42A40D9B150@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <AM0PR0202MB33319C2F9595CC3C2FA1617B9BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <e1244807-74b4-755b-17c2-dce3263ce337@oss.nttdata.com>
 <AM0PR0202MB333172BA4EAF92D0BC3BA9379BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <81d7caa8-4244-85f3-4d4e-78117fe5e25b@oss.nttdata.com>
 <AM0PR0202MB3331324D56A68E08CE7CDAE69BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
Message-ID: <11c78b30-de04-544d-3a10-811ebf663bf2@oracle.com>

Hi Richard,

Just jumping into your last question for now.? : )


On 4/24/20 1:08 PM, Reingruber, Richard wrote:
> Hi Yasumasa, Patricio,
>
>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
>>>> Does it help you? I think it gives you to remove workaround.
>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake
>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to
>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes.
>> Thanks for your information.
>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop.
>> I will modify and will test it after yours.
> Thanks :)
>
>>> Also my first impression was that it won't be that easy from a synchronization point of view to
>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls
>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where
>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear
>>> to me, how this has to be handled.
>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event.
> Yes. To me it is unclear what synchronization is necessary, if it is called during a handshake. And
> also I'm unsure if a thread should do safepoint checks while executing a handshake.
>
> @Patricio, coming back to my question [1]:
>
> In the example you gave in your answer [2]: the java thread would execute a vm operation during a
> direct handshake operation, while the VMThread is actually in the middle of a VM_HandshakeAllThreads
> operation, waiting to handshake the same handshakee: why can't the VMThread just proceed? The
> handshakee would be safepoint safe, wouldn't it?
Because the VMThread would not be able to decrement _processing_sem to 
claim the operation and execute the closure for that handshakee. If 
another JavaThread is doing a direct handshake with that same handshakee 
and called a new VM operation inside the execution of the 
HandshakeClosure in do_handshake(), nobody would be able to decrement 
the _processing_sem anymore until the original direct operation finished 
and the semaphore is signaled again. So this can happen despite the 
state of the handshakee is "handshake/safepoint safe". Changing the 
nested VM operation to be a direct handshake would have the same issue. 
Actually as the code is right now we would not even get pass setting the 
handshake operation because in that case we would block in the 
_handshake_turn_sem for the same reason.

So changing VM_SetFramePop to use direct handshakes in the future will 
probably create that last issue I mentioned. Now, since it is executed 
at a safepoint, with your workaround in enter_interp_only_mode() we 
avoid those nested issues in . Maybe 8239084 would have to be revisited 
to address nested operations in all cases. It is not clear to me now 
though if we should handle that in the handshake code or the caller of a 
certain operation should know it might be called in a nested scenario 
and should handle it.

I'll look a bit more at the updated patch but at first glance looks good.

Thanks!

Patricio
> Thanks, Richard.
>
> [1] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301677&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301677
>
> [2] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301763&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301763
>
> -----Original Message-----
> From: Yasumasa Suenaga <suenaga at oss.nttdata.com>
> Sent: Freitag, 24. April 2020 17:23
> To: Reingruber, Richard <richard.reingruber at sap.com>; Patricio Chilano <patricio.chilano.mateo at oracle.com>; serguei.spitsyn at oracle.com; Vladimir Ivanov <vladimir.x.ivanov at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>
> Hi Richard,
>
> On 2020/04/24 23:44, Reingruber, Richard wrote:
>> Hi Yasumasa,
>>
>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
>>> Does it help you? I think it gives you to remove workaround.
>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake
>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to
>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes.
> Thanks for your information.
> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop.
> I will modify and will test it after yours.
>
>
>> Also my first impression was that it won't be that easy from a synchronization point of view to
>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls
>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where
>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear
>> to me, how this has to be handled.
> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event.
>
>
> Thanks,
>
> Yasumasa
>
>
>> So it appears to me that it would be easier to push JDK-8242427 after this (JDK-8238585).
>>
>>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.)
>> Would be interesting to see how you handled the issues above :)
>>
>> Thanks, Richard.
>>
>> [1] See question in comment https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14302030&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14302030
>>
>> -----Original Message-----
>> From: Yasumasa Suenaga <suenaga at oss.nttdata.com>
>> Sent: Freitag, 24. April 2020 13:34
>> To: Reingruber, Richard <richard.reingruber at sap.com>; Patricio Chilano <patricio.chilano.mateo at oracle.com>; serguei.spitsyn at oracle.com; Vladimir Ivanov <vladimir.x.ivanov at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>
>> Hi Richard,
>>
>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
>> Does it help you? I think it gives you to remove workaround.
>>
>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.)
>>
>>
>> Thanks,
>>
>> Yasumasa
>>
>>
>> On 2020/04/24 17:18, Reingruber, Richard wrote:
>>> Hi Patricio, Vladimir, and Serguei,
>>>
>>> now that direct handshakes are available, I've updated the patch to make use of them.
>>>
>>> In addition I have done some clean-up changes I missed in the first webrev.
>>>
>>> Finally I have implemented the workaround suggested by Patricio to avoid nesting the handshake
>>> into the vm operation VM_SetFramePop [1]
>>>
>>> Kindly review again:
>>>
>>> Webrev:        http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1/
>>> Webrev(delta): http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1.inc/
>>>
>>> I updated the JBS item explaining why the vm operation VM_EnterInterpOnlyMode can be replaced with a
>>> direct handshake:
>>>
>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8238585
>>>
>>> Testing:
>>>
>>> * JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms.
>>>
>>> * Submit-repo: mach5-one-rrich-JDK-8238585-20200423-1436-10441737
>>>
>>> Thanks,
>>> Richard.
>>>
>>> [1] An assertion in Handshake::execute_direct() fails, if called be VMThread, because it is no JavaThread.
>>>
>>> -----Original Message-----
>>> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of Reingruber, Richard
>>> Sent: Freitag, 14. Februar 2020 19:47
>>> To: Patricio Chilano <patricio.chilano.mateo at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>>> Subject: RE: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>>
>>> Hi Patricio,
>>>
>>>      > > I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a
>>>      > > handshake cannot be nested in a vm operation. Maybe it should be asserted in the
>>>      > > Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation?
>>>      > >
>>>      > >    > Alternatively I think you could do something similar to what we do in
>>>      > >    > Deoptimization::deoptimize_all_marked():
>>>      > >    >
>>>      > >    >    EnterInterpOnlyModeClosure hs;
>>>      > >    >    if (SafepointSynchronize::is_at_safepoint()) {
>>>      > >    >      hs.do_thread(state->get_thread());
>>>      > >    >    } else {
>>>      > >    >      Handshake::execute(&hs, state->get_thread());
>>>      > >    >    }
>>>      > >    > (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>>      > >    > HandshakeClosure() constructor)
>>>      > >
>>>      > > Maybe this could be used also in the Handshake::execute() methods as general solution?
>>>      > Right, we could also do that. Avoiding to clear the polling page in
>>>      > HandshakeState::clear_handshake() should be enough to fix this issue and
>>>      > execute a handshake inside a safepoint, but adding that "if" statement
>>>      > in Hanshake::execute() sounds good to avoid all the extra code that we
>>>      > go through when executing a handshake. I filed 8239084 to make that change.
>>>
>>> Thanks for taking care of this and creating the RFE.
>>>
>>>      >
>>>      > >    > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>>      > >    > always called in a nested operation or just sometimes.
>>>      > >
>>>      > > At least one execution path without vm operation exists:
>>>      > >
>>>      > >    JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void
>>>      > >      JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong
>>>      > >        JvmtiEventControllerPrivate::recompute_enabled() : void
>>>      > >          JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches)
>>>      > >            JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void
>>>      > >              JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError
>>>      > >                jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError
>>>      > >
>>>      > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a
>>>      > > handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further
>>>      > > encouraged to do it with a handshake :)
>>>      > Ah! I think you can still do it with a handshake with the
>>>      > Deoptimization::deoptimize_all_marked() like solution. I can change the
>>>      > if-else statement with just the Handshake::execute() call in 8239084.
>>>      > But up to you.  : )
>>>
>>> Well, I think that's enough encouragement :)
>>> I'll wait for 8239084 and try then again.
>>> (no urgency and all)
>>>
>>> Thanks,
>>> Richard.
>>>
>>> -----Original Message-----
>>> From: Patricio Chilano <patricio.chilano.mateo at oracle.com>
>>> Sent: Freitag, 14. Februar 2020 15:54
>>> To: Reingruber, Richard <richard.reingruber at sap.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>>
>>> Hi Richard,
>>>
>>> On 2/14/20 9:58 AM, Reingruber, Richard wrote:
>>>> Hi Patricio,
>>>>
>>>> thanks for having a look.
>>>>
>>>>       > I?m only commenting on the handshake changes.
>>>>       > I see that operation VM_EnterInterpOnlyMode can be called inside
>>>>       > operation VM_SetFramePop which also allows nested operations. Here is a
>>>>       > comment in VM_SetFramePop definition:
>>>>       >
>>>>       > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is
>>>>       > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled.
>>>>       >
>>>>       > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we
>>>>       > could have a handshake inside a safepoint operation. The issue I see
>>>>       > there is that at the end of the handshake the polling page of the target
>>>>       > thread could be disarmed. So if the target thread happens to be in a
>>>>       > blocked state just transiently and wakes up then it will not stop for
>>>>       > the ongoing safepoint. Maybe I can file an RFE to assert that the
>>>>       > polling page is armed at the beginning of disarm_safepoint().
>>>>
>>>> I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a
>>>> handshake cannot be nested in a vm operation. Maybe it should be asserted in the
>>>> Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation?
>>>>
>>>>       > Alternatively I think you could do something similar to what we do in
>>>>       > Deoptimization::deoptimize_all_marked():
>>>>       >
>>>>       >    EnterInterpOnlyModeClosure hs;
>>>>       >    if (SafepointSynchronize::is_at_safepoint()) {
>>>>       >      hs.do_thread(state->get_thread());
>>>>       >    } else {
>>>>       >      Handshake::execute(&hs, state->get_thread());
>>>>       >    }
>>>>       > (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>>>       > HandshakeClosure() constructor)
>>>>
>>>> Maybe this could be used also in the Handshake::execute() methods as general solution?
>>> Right, we could also do that. Avoiding to clear the polling page in
>>> HandshakeState::clear_handshake() should be enough to fix this issue and
>>> execute a handshake inside a safepoint, but adding that "if" statement
>>> in Hanshake::execute() sounds good to avoid all the extra code that we
>>> go through when executing a handshake. I filed 8239084 to make that change.
>>>
>>>>       > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>>>       > always called in a nested operation or just sometimes.
>>>>
>>>> At least one execution path without vm operation exists:
>>>>
>>>>       JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void
>>>>         JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong
>>>>           JvmtiEventControllerPrivate::recompute_enabled() : void
>>>>             JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches)
>>>>               JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void
>>>>                 JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError
>>>>                   jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError
>>>>
>>>> I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a
>>>> handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further
>>>> encouraged to do it with a handshake :)
>>> Ah! I think you can still do it with a handshake with the
>>> Deoptimization::deoptimize_all_marked() like solution. I can change the
>>> if-else statement with just the Handshake::execute() call in 8239084.
>>> But up to you.? : )
>>>
>>> Thanks,
>>> Patricio
>>>> Thanks again,
>>>> Richard.
>>>>
>>>> -----Original Message-----
>>>> From: Patricio Chilano <patricio.chilano.mateo at oracle.com>
>>>> Sent: Donnerstag, 13. Februar 2020 18:47
>>>> To: Reingruber, Richard <richard.reingruber at sap.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>>>
>>>> Hi Richard,
>>>>
>>>> I?m only commenting on the handshake changes.
>>>> I see that operation VM_EnterInterpOnlyMode can be called inside
>>>> operation VM_SetFramePop which also allows nested operations. Here is a
>>>> comment in VM_SetFramePop definition:
>>>>
>>>> // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is
>>>> // called from the JvmtiEventControllerPrivate::recompute_thread_enabled.
>>>>
>>>> So if we change VM_EnterInterpOnlyMode to be a handshake, then now we
>>>> could have a handshake inside a safepoint operation. The issue I see
>>>> there is that at the end of the handshake the polling page of the target
>>>> thread could be disarmed. So if the target thread happens to be in a
>>>> blocked state just transiently and wakes up then it will not stop for
>>>> the ongoing safepoint. Maybe I can file an RFE to assert that the
>>>> polling page is armed at the beginning of disarm_safepoint().
>>>>
>>>> I think one option could be to remove
>>>> SafepointMechanism::disarm_if_needed() in
>>>> HandshakeState::clear_handshake() and let each JavaThread disarm itself
>>>> for the handshake case.
>>>>
>>>> Alternatively I think you could do something similar to what we do in
>>>> Deoptimization::deoptimize_all_marked():
>>>>
>>>>      ? EnterInterpOnlyModeClosure hs;
>>>>      ? if (SafepointSynchronize::is_at_safepoint()) {
>>>>      ??? hs.do_thread(state->get_thread());
>>>>      ? } else {
>>>>      ??? Handshake::execute(&hs, state->get_thread());
>>>>      ? }
>>>> (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>>> HandshakeClosure() constructor)
>>>>
>>>> I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>>> always called in a nested operation or just sometimes.
>>>>
>>>> Thanks,
>>>> Patricio
>>>>
>>>> On 2/12/20 7:23 AM, Reingruber, Richard wrote:
>>>>> // Repost including hotspot runtime and gc lists.
>>>>> // Dean Long suggested to do so, because the enhancement replaces a vm operation
>>>>> // with a handshake.
>>>>> // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html
>>>>>
>>>>> Hi,
>>>>>
>>>>> could I please get reviews for this small enhancement in hotspot's jvmti implementation:
>>>>>
>>>>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/
>>>>> Bug:    https://bugs.openjdk.java.net/browse/JDK-8238585
>>>>>
>>>>> The change avoids making all compiled methods on stack not_entrant when switching a java thread to
>>>>> interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack.
>>>>>
>>>>> Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations.
>>>>>
>>>>> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms.
>>>>>
>>>>> Thanks, Richard.
>>>>>
>>>>> See also my question if anyone knows a reason for making the compiled methods not_entrant:
>>>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html


From coleen.phillimore at oracle.com  Fri Apr 24 19:32:21 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Fri, 24 Apr 2020 15:32:21 -0400
Subject: RFR(S): 8233014: Enable ShowCodeDetailsInExceptionMessages per
 default.
In-Reply-To: <AM4PR0201MB21469B9BDC047A9756AC1C6EECD00@AM4PR0201MB2146.eurprd02.prod.outlook.com>
References: <AM4PR0201MB21469B9BDC047A9756AC1C6EECD00@AM4PR0201MB2146.eurprd02.prod.outlook.com>
Message-ID: <66f0cfad-b9fc-6f23-49e4-05f4066a0a23@oracle.com>

Goetz,
This looks good.
thanks,
Coleen

On 4/24/20 10:36 AM, Lindenmaier, Goetz wrote:
> Hi,
>
> I would like to enable JEP 358 Helpful NullPointerExceptions by default:
>
> http://cr.openjdk.java.net/~goetz/wr19/8233014-enable_NPE_per_default/02/
>
> Please review.
>
> The change passed our testing and jdk-submit.
>
> Best regards,
>   Goetz.


From richard.reingruber at sap.com  Fri Apr 24 21:41:45 2020
From: richard.reingruber at sap.com (Reingruber, Richard)
Date: Fri, 24 Apr 2020 21:41:45 +0000
Subject: RFR(S) 8238585: Use handshake for
 JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled
 methods on stack not_entrant
In-Reply-To: <11c78b30-de04-544d-3a10-811ebf663bf2@oracle.com>
References: <AM0PR0202MB33311AB7623F4164335286309B1B0@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <3c59b9f9-ec38-18c9-8f24-e1186a08a04a@oracle.com>
 <AM0PR0202MB3331FD660B315CEC25B1DC209B150@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <410eed04-e2ef-0f4f-1c56-19e6734a10f6@oracle.com>
 <AM0PR0202MB33318BC60E3460662D42A40D9B150@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <AM0PR0202MB33319C2F9595CC3C2FA1617B9BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <e1244807-74b4-755b-17c2-dce3263ce337@oss.nttdata.com>
 <AM0PR0202MB333172BA4EAF92D0BC3BA9379BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <81d7caa8-4244-85f3-4d4e-78117fe5e25b@oss.nttdata.com>
 <AM0PR0202MB3331324D56A68E08CE7CDAE69BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <11c78b30-de04-544d-3a10-811ebf663bf2@oracle.com>
Message-ID: <AM0PR0202MB3331610F89DFE2AD378B52289BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>

Hi Patricio,

> > @Patricio, coming back to my question [1]:
> >
> > In the example you gave in your answer [2]: the java thread would execute a vm operation during a
> > direct handshake operation, while the VMThread is actually in the middle of a VM_HandshakeAllThreads
> > operation, waiting to handshake the same handshakee: why can't the VMThread just proceed? The
> > handshakee would be safepoint safe, wouldn't it?
> Because the VMThread would not be able to decrement _processing_sem to 
> claim the operation and execute the closure for that handshakee. If 
> another JavaThread is doing a direct handshake with that same handshakee 
> and called a new VM operation inside the execution of the 
> HandshakeClosure in do_handshake(), nobody would be able to decrement 
> the _processing_sem anymore until the original direct operation finished 
> and the semaphore is signaled again.

Thanks, understood. On a higher level: a JavaThread can have at most one handshake operation being
processed at at time.

> So this can happen despite the 
> state of the handshakee is "handshake/safepoint safe". Changing the 
> nested VM operation to be a direct handshake would have the same issue. 
> Actually as the code is right now we would not even get pass setting the 
> handshake operation because in that case we would block in the 
> _handshake_turn_sem for the same reason.

Don't really understand the details here, but that's ok.
Interesting that _handshake_turn_sem gets signaled before or after do_handshake() depending if the
handshake operation is processed by handshakee. Comments say "Disarm before/after executing
operation" but not why :)

> So changing VM_SetFramePop to use direct handshakes in the future will 
> probably create that last issue I mentioned. Now, since it is executed 
> at a safepoint, with your workaround in enter_interp_only_mode() we 
> avoid those nested issues in . Maybe 8239084 would have to be revisited 
> to address nested operations in all cases. It is not clear to me now 
> though if we should handle that in the handshake code or the caller of a 
> certain operation should know it might be called in a nested scenario 
> and should handle it.

Last question: is it ok for the processor of a direct handshake operation to do safepoint/handshake
checks? I wouldn't see a reason, why not. But certainly I would avoid it.

> I'll look a bit more at the updated patch but at first glance looks good.

Thanks,
Richard.

-----Original Message-----
From: Patricio Chilano <patricio.chilano.mateo at oracle.com> 
Sent: Freitag, 24. April 2020 19:14
To: Reingruber, Richard <richard.reingruber at sap.com>; Yasumasa Suenaga <suenaga at oss.nttdata.com>; serguei.spitsyn at oracle.com; Vladimir Ivanov <vladimir.x.ivanov at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant

Hi Richard,

Just jumping into your last question for now.? : )


On 4/24/20 1:08 PM, Reingruber, Richard wrote:
> Hi Yasumasa, Patricio,
>
>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
>>>> Does it help you? I think it gives you to remove workaround.
>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake
>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to
>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes.
>> Thanks for your information.
>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop.
>> I will modify and will test it after yours.
> Thanks :)
>
>>> Also my first impression was that it won't be that easy from a synchronization point of view to
>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls
>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where
>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear
>>> to me, how this has to be handled.
>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event.
> Yes. To me it is unclear what synchronization is necessary, if it is called during a handshake. And
> also I'm unsure if a thread should do safepoint checks while executing a handshake.
>
> @Patricio, coming back to my question [1]:
>
> In the example you gave in your answer [2]: the java thread would execute a vm operation during a
> direct handshake operation, while the VMThread is actually in the middle of a VM_HandshakeAllThreads
> operation, waiting to handshake the same handshakee: why can't the VMThread just proceed? The
> handshakee would be safepoint safe, wouldn't it?
Because the VMThread would not be able to decrement _processing_sem to 
claim the operation and execute the closure for that handshakee. If 
another JavaThread is doing a direct handshake with that same handshakee 
and called a new VM operation inside the execution of the 
HandshakeClosure in do_handshake(), nobody would be able to decrement 
the _processing_sem anymore until the original direct operation finished 
and the semaphore is signaled again. So this can happen despite the 
state of the handshakee is "handshake/safepoint safe". Changing the 
nested VM operation to be a direct handshake would have the same issue. 
Actually as the code is right now we would not even get pass setting the 
handshake operation because in that case we would block in the 
_handshake_turn_sem for the same reason.

So changing VM_SetFramePop to use direct handshakes in the future will 
probably create that last issue I mentioned. Now, since it is executed 
at a safepoint, with your workaround in enter_interp_only_mode() we 
avoid those nested issues in . Maybe 8239084 would have to be revisited 
to address nested operations in all cases. It is not clear to me now 
though if we should handle that in the handshake code or the caller of a 
certain operation should know it might be called in a nested scenario 
and should handle it.

I'll look a bit more at the updated patch but at first glance looks good.

Thanks!

Patricio
> Thanks, Richard.
>
> [1] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301677&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301677
>
> [2] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301763&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301763
>
> -----Original Message-----
> From: Yasumasa Suenaga <suenaga at oss.nttdata.com>
> Sent: Freitag, 24. April 2020 17:23
> To: Reingruber, Richard <richard.reingruber at sap.com>; Patricio Chilano <patricio.chilano.mateo at oracle.com>; serguei.spitsyn at oracle.com; Vladimir Ivanov <vladimir.x.ivanov at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>
> Hi Richard,
>
> On 2020/04/24 23:44, Reingruber, Richard wrote:
>> Hi Yasumasa,
>>
>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
>>> Does it help you? I think it gives you to remove workaround.
>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake
>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to
>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes.
> Thanks for your information.
> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop.
> I will modify and will test it after yours.
>
>
>> Also my first impression was that it won't be that easy from a synchronization point of view to
>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls
>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where
>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear
>> to me, how this has to be handled.
> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event.
>
>
> Thanks,
>
> Yasumasa
>
>
>> So it appears to me that it would be easier to push JDK-8242427 after this (JDK-8238585).
>>
>>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.)
>> Would be interesting to see how you handled the issues above :)
>>
>> Thanks, Richard.
>>
>> [1] See question in comment https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14302030&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14302030
>>
>> -----Original Message-----
>> From: Yasumasa Suenaga <suenaga at oss.nttdata.com>
>> Sent: Freitag, 24. April 2020 13:34
>> To: Reingruber, Richard <richard.reingruber at sap.com>; Patricio Chilano <patricio.chilano.mateo at oracle.com>; serguei.spitsyn at oracle.com; Vladimir Ivanov <vladimir.x.ivanov at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>
>> Hi Richard,
>>
>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
>> Does it help you? I think it gives you to remove workaround.
>>
>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.)
>>
>>
>> Thanks,
>>
>> Yasumasa
>>
>>
>> On 2020/04/24 17:18, Reingruber, Richard wrote:
>>> Hi Patricio, Vladimir, and Serguei,
>>>
>>> now that direct handshakes are available, I've updated the patch to make use of them.
>>>
>>> In addition I have done some clean-up changes I missed in the first webrev.
>>>
>>> Finally I have implemented the workaround suggested by Patricio to avoid nesting the handshake
>>> into the vm operation VM_SetFramePop [1]
>>>
>>> Kindly review again:
>>>
>>> Webrev:        http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1/
>>> Webrev(delta): http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1.inc/
>>>
>>> I updated the JBS item explaining why the vm operation VM_EnterInterpOnlyMode can be replaced with a
>>> direct handshake:
>>>
>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8238585
>>>
>>> Testing:
>>>
>>> * JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms.
>>>
>>> * Submit-repo: mach5-one-rrich-JDK-8238585-20200423-1436-10441737
>>>
>>> Thanks,
>>> Richard.
>>>
>>> [1] An assertion in Handshake::execute_direct() fails, if called be VMThread, because it is no JavaThread.
>>>
>>> -----Original Message-----
>>> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of Reingruber, Richard
>>> Sent: Freitag, 14. Februar 2020 19:47
>>> To: Patricio Chilano <patricio.chilano.mateo at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>>> Subject: RE: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>>
>>> Hi Patricio,
>>>
>>>      > > I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a
>>>      > > handshake cannot be nested in a vm operation. Maybe it should be asserted in the
>>>      > > Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation?
>>>      > >
>>>      > >    > Alternatively I think you could do something similar to what we do in
>>>      > >    > Deoptimization::deoptimize_all_marked():
>>>      > >    >
>>>      > >    >    EnterInterpOnlyModeClosure hs;
>>>      > >    >    if (SafepointSynchronize::is_at_safepoint()) {
>>>      > >    >      hs.do_thread(state->get_thread());
>>>      > >    >    } else {
>>>      > >    >      Handshake::execute(&hs, state->get_thread());
>>>      > >    >    }
>>>      > >    > (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>>      > >    > HandshakeClosure() constructor)
>>>      > >
>>>      > > Maybe this could be used also in the Handshake::execute() methods as general solution?
>>>      > Right, we could also do that. Avoiding to clear the polling page in
>>>      > HandshakeState::clear_handshake() should be enough to fix this issue and
>>>      > execute a handshake inside a safepoint, but adding that "if" statement
>>>      > in Hanshake::execute() sounds good to avoid all the extra code that we
>>>      > go through when executing a handshake. I filed 8239084 to make that change.
>>>
>>> Thanks for taking care of this and creating the RFE.
>>>
>>>      >
>>>      > >    > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>>      > >    > always called in a nested operation or just sometimes.
>>>      > >
>>>      > > At least one execution path without vm operation exists:
>>>      > >
>>>      > >    JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void
>>>      > >      JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong
>>>      > >        JvmtiEventControllerPrivate::recompute_enabled() : void
>>>      > >          JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches)
>>>      > >            JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void
>>>      > >              JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError
>>>      > >                jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError
>>>      > >
>>>      > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a
>>>      > > handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further
>>>      > > encouraged to do it with a handshake :)
>>>      > Ah! I think you can still do it with a handshake with the
>>>      > Deoptimization::deoptimize_all_marked() like solution. I can change the
>>>      > if-else statement with just the Handshake::execute() call in 8239084.
>>>      > But up to you.  : )
>>>
>>> Well, I think that's enough encouragement :)
>>> I'll wait for 8239084 and try then again.
>>> (no urgency and all)
>>>
>>> Thanks,
>>> Richard.
>>>
>>> -----Original Message-----
>>> From: Patricio Chilano <patricio.chilano.mateo at oracle.com>
>>> Sent: Freitag, 14. Februar 2020 15:54
>>> To: Reingruber, Richard <richard.reingruber at sap.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>>
>>> Hi Richard,
>>>
>>> On 2/14/20 9:58 AM, Reingruber, Richard wrote:
>>>> Hi Patricio,
>>>>
>>>> thanks for having a look.
>>>>
>>>>       > I?m only commenting on the handshake changes.
>>>>       > I see that operation VM_EnterInterpOnlyMode can be called inside
>>>>       > operation VM_SetFramePop which also allows nested operations. Here is a
>>>>       > comment in VM_SetFramePop definition:
>>>>       >
>>>>       > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is
>>>>       > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled.
>>>>       >
>>>>       > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we
>>>>       > could have a handshake inside a safepoint operation. The issue I see
>>>>       > there is that at the end of the handshake the polling page of the target
>>>>       > thread could be disarmed. So if the target thread happens to be in a
>>>>       > blocked state just transiently and wakes up then it will not stop for
>>>>       > the ongoing safepoint. Maybe I can file an RFE to assert that the
>>>>       > polling page is armed at the beginning of disarm_safepoint().
>>>>
>>>> I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a
>>>> handshake cannot be nested in a vm operation. Maybe it should be asserted in the
>>>> Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation?
>>>>
>>>>       > Alternatively I think you could do something similar to what we do in
>>>>       > Deoptimization::deoptimize_all_marked():
>>>>       >
>>>>       >    EnterInterpOnlyModeClosure hs;
>>>>       >    if (SafepointSynchronize::is_at_safepoint()) {
>>>>       >      hs.do_thread(state->get_thread());
>>>>       >    } else {
>>>>       >      Handshake::execute(&hs, state->get_thread());
>>>>       >    }
>>>>       > (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>>>       > HandshakeClosure() constructor)
>>>>
>>>> Maybe this could be used also in the Handshake::execute() methods as general solution?
>>> Right, we could also do that. Avoiding to clear the polling page in
>>> HandshakeState::clear_handshake() should be enough to fix this issue and
>>> execute a handshake inside a safepoint, but adding that "if" statement
>>> in Hanshake::execute() sounds good to avoid all the extra code that we
>>> go through when executing a handshake. I filed 8239084 to make that change.
>>>
>>>>       > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>>>       > always called in a nested operation or just sometimes.
>>>>
>>>> At least one execution path without vm operation exists:
>>>>
>>>>       JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void
>>>>         JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong
>>>>           JvmtiEventControllerPrivate::recompute_enabled() : void
>>>>             JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches)
>>>>               JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void
>>>>                 JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError
>>>>                   jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError
>>>>
>>>> I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a
>>>> handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further
>>>> encouraged to do it with a handshake :)
>>> Ah! I think you can still do it with a handshake with the
>>> Deoptimization::deoptimize_all_marked() like solution. I can change the
>>> if-else statement with just the Handshake::execute() call in 8239084.
>>> But up to you.? : )
>>>
>>> Thanks,
>>> Patricio
>>>> Thanks again,
>>>> Richard.
>>>>
>>>> -----Original Message-----
>>>> From: Patricio Chilano <patricio.chilano.mateo at oracle.com>
>>>> Sent: Donnerstag, 13. Februar 2020 18:47
>>>> To: Reingruber, Richard <richard.reingruber at sap.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>>>
>>>> Hi Richard,
>>>>
>>>> I?m only commenting on the handshake changes.
>>>> I see that operation VM_EnterInterpOnlyMode can be called inside
>>>> operation VM_SetFramePop which also allows nested operations. Here is a
>>>> comment in VM_SetFramePop definition:
>>>>
>>>> // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is
>>>> // called from the JvmtiEventControllerPrivate::recompute_thread_enabled.
>>>>
>>>> So if we change VM_EnterInterpOnlyMode to be a handshake, then now we
>>>> could have a handshake inside a safepoint operation. The issue I see
>>>> there is that at the end of the handshake the polling page of the target
>>>> thread could be disarmed. So if the target thread happens to be in a
>>>> blocked state just transiently and wakes up then it will not stop for
>>>> the ongoing safepoint. Maybe I can file an RFE to assert that the
>>>> polling page is armed at the beginning of disarm_safepoint().
>>>>
>>>> I think one option could be to remove
>>>> SafepointMechanism::disarm_if_needed() in
>>>> HandshakeState::clear_handshake() and let each JavaThread disarm itself
>>>> for the handshake case.
>>>>
>>>> Alternatively I think you could do something similar to what we do in
>>>> Deoptimization::deoptimize_all_marked():
>>>>
>>>>      ? EnterInterpOnlyModeClosure hs;
>>>>      ? if (SafepointSynchronize::is_at_safepoint()) {
>>>>      ??? hs.do_thread(state->get_thread());
>>>>      ? } else {
>>>>      ??? Handshake::execute(&hs, state->get_thread());
>>>>      ? }
>>>> (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>>> HandshakeClosure() constructor)
>>>>
>>>> I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>>> always called in a nested operation or just sometimes.
>>>>
>>>> Thanks,
>>>> Patricio
>>>>
>>>> On 2/12/20 7:23 AM, Reingruber, Richard wrote:
>>>>> // Repost including hotspot runtime and gc lists.
>>>>> // Dean Long suggested to do so, because the enhancement replaces a vm operation
>>>>> // with a handshake.
>>>>> // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html
>>>>>
>>>>> Hi,
>>>>>
>>>>> could I please get reviews for this small enhancement in hotspot's jvmti implementation:
>>>>>
>>>>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/
>>>>> Bug:    https://bugs.openjdk.java.net/browse/JDK-8238585
>>>>>
>>>>> The change avoids making all compiled methods on stack not_entrant when switching a java thread to
>>>>> interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack.
>>>>>
>>>>> Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations.
>>>>>
>>>>> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms.
>>>>>
>>>>> Thanks, Richard.
>>>>>
>>>>> See also my question if anyone knows a reason for making the compiled methods not_entrant:
>>>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html


From igor.ignatyev at oracle.com  Sat Apr 25 05:33:52 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Fri, 24 Apr 2020 22:33:52 -0700
Subject: RFR(S) : 8242314 : use reproducible random in vmTestbase shared
 code
In-Reply-To: <A59F9A91-D3D4-4D98-BD02-C44E1C3790E4@oracle.com>
References: <A59F9A91-D3D4-4D98-BD02-C44E1C3790E4@oracle.com>
Message-ID: <B1831898-162E-4E75-8C7E-EEA7661C7C6F@oracle.com>

ping?

> On Apr 22, 2020, at 11:03 AM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
> 
> http://cr.openjdk.java.net/~iignatyev/8242314/webrev.00/ 
>> 122 lines changed: 71 ins; 19 del; 32 mod;
> 
> 
> Hi all,
> 
> could you please review this small patch which updates shared vmTestbase  test code to use j.t.l.Utils.getRandomInstance() as a random number generator or as a source of seed values? this patch lays the ground work for updates of :vmTestbase_* test groups which are to be done by remaining subtasks of 8241623[1], such split make it possible to work on/review/test testgroup-specific changes in concurrently.  
> 
> unfortunately, such extra dependency on /test/lib classes broke the shaky balance and caused some tests to fail due to "split testlibrary" problem (see 8188828[2] for the definition), the solution to that is to remove all dependencies on /test/lib classes from ExecDriver class, so the patch also includes "inlining" of several jdk.test.lib.Utils and Platform methods/fields into ExecDriver.
> 
> webrev: http://cr.openjdk.java.net/~iignatyev/8242314/webrev.00/ 
> testing: all :vmTestbase_* test groups on {linux,windows,macosx}-x64 and solaris-sparcv9
> JBS: https://bugs.openjdk.java.net/browse/JDK-8242314
> 
> [1] https://bugs.openjdk.java.net/browse/JDK-8241623
> [2] https://bugs.openjdk.java.net/browse/JDK-8188828
> 
> Thanks,
> -- Igor
> 
> 


From patricio.chilano.mateo at oracle.com  Sat Apr 25 09:23:03 2020
From: patricio.chilano.mateo at oracle.com (Patricio Chilano)
Date: Sat, 25 Apr 2020 06:23:03 -0300
Subject: RFR(S) 8238585: Use handshake for
 JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled
 methods on stack not_entrant
In-Reply-To: <AM0PR0202MB3331610F89DFE2AD378B52289BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
References: <AM0PR0202MB33311AB7623F4164335286309B1B0@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <3c59b9f9-ec38-18c9-8f24-e1186a08a04a@oracle.com>
 <AM0PR0202MB3331FD660B315CEC25B1DC209B150@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <410eed04-e2ef-0f4f-1c56-19e6734a10f6@oracle.com>
 <AM0PR0202MB33318BC60E3460662D42A40D9B150@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <AM0PR0202MB33319C2F9595CC3C2FA1617B9BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <e1244807-74b4-755b-17c2-dce3263ce337@oss.nttdata.com>
 <AM0PR0202MB333172BA4EAF92D0BC3BA9379BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <81d7caa8-4244-85f3-4d4e-78117fe5e25b@oss.nttdata.com>
 <AM0PR0202MB3331324D56A68E08CE7CDAE69BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <11c78b30-de04-544d-3a10-811ebf663bf2@oracle.com>
 <AM0PR0202MB3331610F89DFE2AD378B52289BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
Message-ID: <7bb8abdc-6c49-3059-23d5-9552bd80b480@oracle.com>

Hi Richard,

On 4/24/20 6:41 PM, Reingruber, Richard wrote:
> Hi Patricio,
>
>>> @Patricio, coming back to my question [1]:
>>>
>>> In the example you gave in your answer [2]: the java thread would execute a vm operation during a
>>> direct handshake operation, while the VMThread is actually in the middle of a VM_HandshakeAllThreads
>>> operation, waiting to handshake the same handshakee: why can't the VMThread just proceed? The
>>> handshakee would be safepoint safe, wouldn't it?
>> Because the VMThread would not be able to decrement _processing_sem to
>> claim the operation and execute the closure for that handshakee. If
>> another JavaThread is doing a direct handshake with that same handshakee
>> and called a new VM operation inside the execution of the
>> HandshakeClosure in do_handshake(), nobody would be able to decrement
>> the _processing_sem anymore until the original direct operation finished
>> and the semaphore is signaled again.
> Thanks, understood. On a higher level: a JavaThread can have at most one handshake operation being
> processed at at time.
Exactly. As of now we don't handle the case where another handshake 
operation on the same handshakee is called inside 
_handshake_cl->do_thread(). If this happens we will deadlock.

>> So this can happen despite the
>> state of the handshakee is "handshake/safepoint safe". Changing the
>> nested VM operation to be a direct handshake would have the same issue.
>> Actually as the code is right now we would not even get pass setting the
>> handshake operation because in that case we would block in the
>> _handshake_turn_sem for the same reason.
> Don't really understand the details here, but that's ok.
> Interesting that _handshake_turn_sem gets signaled before or after do_handshake() depending if the
> handshake operation is processed by handshakee. Comments say "Disarm before/after executing
> operation" but not why :)
Yes, that pattern actually relates with clearing _operation and predates 
direct handshakes. In theory we should always call do_handshake() first 
and then clear the handshake. This is what we do when the operation is 
processed by the handshaker, and it is necessary to be that way, 
otherwise if we clear the handshake first then the handshakee might 
transition from the safe state and never see that it actually has to 
stop for the handshake. Now, when the handshake operation is processed 
by the handshakee itself we don't have that issue, so it doesn't matter 
if we clear it before or after. The reason we do it before is to avoid 
the VMThread to execute unnecessary instructions in try_process(). This 
is specially true for the VM_HandshakeAllThreads operation case. If the 
VMThread sees that a JavaThread doesn't have an operation set, it can 
just continue to try to process the next JavaThread, instead of going 
through the unnecessary steps of checking the state of the JavaThread 
and trying to execute a try_wait() operation on the _processing_sem 
which we know won't succeed. Now for the direct handshake case doing it 
before or after doesn't really matter and so I just copied the pattern 
from the non-direct case to make it consistent in that same method.


>> So changing VM_SetFramePop to use direct handshakes in the future will
>> probably create that last issue I mentioned. Now, since it is executed
>> at a safepoint, with your workaround in enter_interp_only_mode() we
>> avoid those nested issues in . Maybe 8239084 would have to be revisited
>> to address nested operations in all cases. It is not clear to me now
>> though if we should handle that in the handshake code or the caller of a
>> certain operation should know it might be called in a nested scenario
>> and should handle it.
> Last question: is it ok for the processor of a direct handshake operation to do safepoint/handshake
> checks? I wouldn't see a reason, why not. But certainly I would avoid it.
I tried to think of possible issues with that (independent of the 
closure logic) but I couldn't find a specific one. If the handshakee 
tries to process a pending handshake, process_by_self() will just return 
without calling process_self_inner() since it will detect it is already 
inside a handshake. And that behaviour makes sense since there is no 
point in trying to execute a new handshake operation if you are in the 
middle of another one. If the handshaker inside the closure checks for 
its own pending handshakes that also seems okay (this will by itself 
also check for safepoints in the transitions). Checking for safepoints 
in both cases seems more tricky but I couldn't think of a concrete issue 
with that.
In any case I would also avoid checking for safepoints/handshakes inside 
the handshake closure. You might get issues related to the actual logic 
of the closure, like the typical deadlock because of trying to grab the 
same lock (although it's true that you always have to deal with that 
kind of problems when checking for safepoint/handshakes), or coming back 
from the safepoint/handshake and failing because some state you didn't 
expect to change in the middle of the handshake actually changed.


Thanks,
Patricio
>> I'll look a bit more at the updated patch but at first glance looks good.
> Thanks,
> Richard.
>
> -----Original Message-----
> From: Patricio Chilano <patricio.chilano.mateo at oracle.com>
> Sent: Freitag, 24. April 2020 19:14
> To: Reingruber, Richard <richard.reingruber at sap.com>; Yasumasa Suenaga <suenaga at oss.nttdata.com>; serguei.spitsyn at oracle.com; Vladimir Ivanov <vladimir.x.ivanov at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>
> Hi Richard,
>
> Just jumping into your last question for now.? : )
>
>
> On 4/24/20 1:08 PM, Reingruber, Richard wrote:
>> Hi Yasumasa, Patricio,
>>
>>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
>>>>> Does it help you? I think it gives you to remove workaround.
>>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake
>>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to
>>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes.
>>> Thanks for your information.
>>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop.
>>> I will modify and will test it after yours.
>> Thanks :)
>>
>>>> Also my first impression was that it won't be that easy from a synchronization point of view to
>>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls
>>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where
>>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear
>>>> to me, how this has to be handled.
>>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event.
>> Yes. To me it is unclear what synchronization is necessary, if it is called during a handshake. And
>> also I'm unsure if a thread should do safepoint checks while executing a handshake.
>>
>> @Patricio, coming back to my question [1]:
>>
>> In the example you gave in your answer [2]: the java thread would execute a vm operation during a
>> direct handshake operation, while the VMThread is actually in the middle of a VM_HandshakeAllThreads
>> operation, waiting to handshake the same handshakee: why can't the VMThread just proceed? The
>> handshakee would be safepoint safe, wouldn't it?
> Because the VMThread would not be able to decrement _processing_sem to
> claim the operation and execute the closure for that handshakee. If
> another JavaThread is doing a direct handshake with that same handshakee
> and called a new VM operation inside the execution of the
> HandshakeClosure in do_handshake(), nobody would be able to decrement
> the _processing_sem anymore until the original direct operation finished
> and the semaphore is signaled again. So this can happen despite the
> state of the handshakee is "handshake/safepoint safe". Changing the
> nested VM operation to be a direct handshake would have the same issue.
> Actually as the code is right now we would not even get pass setting the
> handshake operation because in that case we would block in the
> _handshake_turn_sem for the same reason.
>
> So changing VM_SetFramePop to use direct handshakes in the future will
> probably create that last issue I mentioned. Now, since it is executed
> at a safepoint, with your workaround in enter_interp_only_mode() we
> avoid those nested issues in . Maybe 8239084 would have to be revisited
> to address nested operations in all cases. It is not clear to me now
> though if we should handle that in the handshake code or the caller of a
> certain operation should know it might be called in a nested scenario
> and should handle it.
>
> I'll look a bit more at the updated patch but at first glance looks good.
>
> Thanks!
>
> Patricio
>> Thanks, Richard.
>>
>> [1] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301677&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301677
>>
>> [2] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301763&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301763
>>
>> -----Original Message-----
>> From: Yasumasa Suenaga <suenaga at oss.nttdata.com>
>> Sent: Freitag, 24. April 2020 17:23
>> To: Reingruber, Richard <richard.reingruber at sap.com>; Patricio Chilano <patricio.chilano.mateo at oracle.com>; serguei.spitsyn at oracle.com; Vladimir Ivanov <vladimir.x.ivanov at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>
>> Hi Richard,
>>
>> On 2020/04/24 23:44, Reingruber, Richard wrote:
>>> Hi Yasumasa,
>>>
>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
>>>> Does it help you? I think it gives you to remove workaround.
>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake
>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to
>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes.
>> Thanks for your information.
>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop.
>> I will modify and will test it after yours.
>>
>>
>>> Also my first impression was that it won't be that easy from a synchronization point of view to
>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls
>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where
>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear
>>> to me, how this has to be handled.
>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event.
>>
>>
>> Thanks,
>>
>> Yasumasa
>>
>>
>>> So it appears to me that it would be easier to push JDK-8242427 after this (JDK-8238585).
>>>
>>>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.)
>>> Would be interesting to see how you handled the issues above :)
>>>
>>> Thanks, Richard.
>>>
>>> [1] See question in comment https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14302030&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14302030
>>>
>>> -----Original Message-----
>>> From: Yasumasa Suenaga <suenaga at oss.nttdata.com>
>>> Sent: Freitag, 24. April 2020 13:34
>>> To: Reingruber, Richard <richard.reingruber at sap.com>; Patricio Chilano <patricio.chilano.mateo at oracle.com>; serguei.spitsyn at oracle.com; Vladimir Ivanov <vladimir.x.ivanov at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>>
>>> Hi Richard,
>>>
>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
>>> Does it help you? I think it gives you to remove workaround.
>>>
>>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.)
>>>
>>>
>>> Thanks,
>>>
>>> Yasumasa
>>>
>>>
>>> On 2020/04/24 17:18, Reingruber, Richard wrote:
>>>> Hi Patricio, Vladimir, and Serguei,
>>>>
>>>> now that direct handshakes are available, I've updated the patch to make use of them.
>>>>
>>>> In addition I have done some clean-up changes I missed in the first webrev.
>>>>
>>>> Finally I have implemented the workaround suggested by Patricio to avoid nesting the handshake
>>>> into the vm operation VM_SetFramePop [1]
>>>>
>>>> Kindly review again:
>>>>
>>>> Webrev:        http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1/
>>>> Webrev(delta): http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1.inc/
>>>>
>>>> I updated the JBS item explaining why the vm operation VM_EnterInterpOnlyMode can be replaced with a
>>>> direct handshake:
>>>>
>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8238585
>>>>
>>>> Testing:
>>>>
>>>> * JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms.
>>>>
>>>> * Submit-repo: mach5-one-rrich-JDK-8238585-20200423-1436-10441737
>>>>
>>>> Thanks,
>>>> Richard.
>>>>
>>>> [1] An assertion in Handshake::execute_direct() fails, if called be VMThread, because it is no JavaThread.
>>>>
>>>> -----Original Message-----
>>>> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of Reingruber, Richard
>>>> Sent: Freitag, 14. Februar 2020 19:47
>>>> To: Patricio Chilano <patricio.chilano.mateo at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>>>> Subject: RE: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>>>
>>>> Hi Patricio,
>>>>
>>>>       > > I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a
>>>>       > > handshake cannot be nested in a vm operation. Maybe it should be asserted in the
>>>>       > > Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation?
>>>>       > >
>>>>       > >    > Alternatively I think you could do something similar to what we do in
>>>>       > >    > Deoptimization::deoptimize_all_marked():
>>>>       > >    >
>>>>       > >    >    EnterInterpOnlyModeClosure hs;
>>>>       > >    >    if (SafepointSynchronize::is_at_safepoint()) {
>>>>       > >    >      hs.do_thread(state->get_thread());
>>>>       > >    >    } else {
>>>>       > >    >      Handshake::execute(&hs, state->get_thread());
>>>>       > >    >    }
>>>>       > >    > (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>>>       > >    > HandshakeClosure() constructor)
>>>>       > >
>>>>       > > Maybe this could be used also in the Handshake::execute() methods as general solution?
>>>>       > Right, we could also do that. Avoiding to clear the polling page in
>>>>       > HandshakeState::clear_handshake() should be enough to fix this issue and
>>>>       > execute a handshake inside a safepoint, but adding that "if" statement
>>>>       > in Hanshake::execute() sounds good to avoid all the extra code that we
>>>>       > go through when executing a handshake. I filed 8239084 to make that change.
>>>>
>>>> Thanks for taking care of this and creating the RFE.
>>>>
>>>>       >
>>>>       > >    > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>>>       > >    > always called in a nested operation or just sometimes.
>>>>       > >
>>>>       > > At least one execution path without vm operation exists:
>>>>       > >
>>>>       > >    JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void
>>>>       > >      JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong
>>>>       > >        JvmtiEventControllerPrivate::recompute_enabled() : void
>>>>       > >          JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches)
>>>>       > >            JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void
>>>>       > >              JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError
>>>>       > >                jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError
>>>>       > >
>>>>       > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a
>>>>       > > handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further
>>>>       > > encouraged to do it with a handshake :)
>>>>       > Ah! I think you can still do it with a handshake with the
>>>>       > Deoptimization::deoptimize_all_marked() like solution. I can change the
>>>>       > if-else statement with just the Handshake::execute() call in 8239084.
>>>>       > But up to you.  : )
>>>>
>>>> Well, I think that's enough encouragement :)
>>>> I'll wait for 8239084 and try then again.
>>>> (no urgency and all)
>>>>
>>>> Thanks,
>>>> Richard.
>>>>
>>>> -----Original Message-----
>>>> From: Patricio Chilano <patricio.chilano.mateo at oracle.com>
>>>> Sent: Freitag, 14. Februar 2020 15:54
>>>> To: Reingruber, Richard <richard.reingruber at sap.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>>>
>>>> Hi Richard,
>>>>
>>>> On 2/14/20 9:58 AM, Reingruber, Richard wrote:
>>>>> Hi Patricio,
>>>>>
>>>>> thanks for having a look.
>>>>>
>>>>>        > I?m only commenting on the handshake changes.
>>>>>        > I see that operation VM_EnterInterpOnlyMode can be called inside
>>>>>        > operation VM_SetFramePop which also allows nested operations. Here is a
>>>>>        > comment in VM_SetFramePop definition:
>>>>>        >
>>>>>        > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is
>>>>>        > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled.
>>>>>        >
>>>>>        > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we
>>>>>        > could have a handshake inside a safepoint operation. The issue I see
>>>>>        > there is that at the end of the handshake the polling page of the target
>>>>>        > thread could be disarmed. So if the target thread happens to be in a
>>>>>        > blocked state just transiently and wakes up then it will not stop for
>>>>>        > the ongoing safepoint. Maybe I can file an RFE to assert that the
>>>>>        > polling page is armed at the beginning of disarm_safepoint().
>>>>>
>>>>> I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a
>>>>> handshake cannot be nested in a vm operation. Maybe it should be asserted in the
>>>>> Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation?
>>>>>
>>>>>        > Alternatively I think you could do something similar to what we do in
>>>>>        > Deoptimization::deoptimize_all_marked():
>>>>>        >
>>>>>        >    EnterInterpOnlyModeClosure hs;
>>>>>        >    if (SafepointSynchronize::is_at_safepoint()) {
>>>>>        >      hs.do_thread(state->get_thread());
>>>>>        >    } else {
>>>>>        >      Handshake::execute(&hs, state->get_thread());
>>>>>        >    }
>>>>>        > (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>>>>        > HandshakeClosure() constructor)
>>>>>
>>>>> Maybe this could be used also in the Handshake::execute() methods as general solution?
>>>> Right, we could also do that. Avoiding to clear the polling page in
>>>> HandshakeState::clear_handshake() should be enough to fix this issue and
>>>> execute a handshake inside a safepoint, but adding that "if" statement
>>>> in Hanshake::execute() sounds good to avoid all the extra code that we
>>>> go through when executing a handshake. I filed 8239084 to make that change.
>>>>
>>>>>        > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>>>>        > always called in a nested operation or just sometimes.
>>>>>
>>>>> At least one execution path without vm operation exists:
>>>>>
>>>>>        JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void
>>>>>          JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong
>>>>>            JvmtiEventControllerPrivate::recompute_enabled() : void
>>>>>              JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches)
>>>>>                JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void
>>>>>                  JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError
>>>>>                    jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError
>>>>>
>>>>> I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a
>>>>> handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further
>>>>> encouraged to do it with a handshake :)
>>>> Ah! I think you can still do it with a handshake with the
>>>> Deoptimization::deoptimize_all_marked() like solution. I can change the
>>>> if-else statement with just the Handshake::execute() call in 8239084.
>>>> But up to you.? : )
>>>>
>>>> Thanks,
>>>> Patricio
>>>>> Thanks again,
>>>>> Richard.
>>>>>
>>>>> -----Original Message-----
>>>>> From: Patricio Chilano <patricio.chilano.mateo at oracle.com>
>>>>> Sent: Donnerstag, 13. Februar 2020 18:47
>>>>> To: Reingruber, Richard <richard.reingruber at sap.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>>>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>>>>
>>>>> Hi Richard,
>>>>>
>>>>> I?m only commenting on the handshake changes.
>>>>> I see that operation VM_EnterInterpOnlyMode can be called inside
>>>>> operation VM_SetFramePop which also allows nested operations. Here is a
>>>>> comment in VM_SetFramePop definition:
>>>>>
>>>>> // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is
>>>>> // called from the JvmtiEventControllerPrivate::recompute_thread_enabled.
>>>>>
>>>>> So if we change VM_EnterInterpOnlyMode to be a handshake, then now we
>>>>> could have a handshake inside a safepoint operation. The issue I see
>>>>> there is that at the end of the handshake the polling page of the target
>>>>> thread could be disarmed. So if the target thread happens to be in a
>>>>> blocked state just transiently and wakes up then it will not stop for
>>>>> the ongoing safepoint. Maybe I can file an RFE to assert that the
>>>>> polling page is armed at the beginning of disarm_safepoint().
>>>>>
>>>>> I think one option could be to remove
>>>>> SafepointMechanism::disarm_if_needed() in
>>>>> HandshakeState::clear_handshake() and let each JavaThread disarm itself
>>>>> for the handshake case.
>>>>>
>>>>> Alternatively I think you could do something similar to what we do in
>>>>> Deoptimization::deoptimize_all_marked():
>>>>>
>>>>>       ? EnterInterpOnlyModeClosure hs;
>>>>>       ? if (SafepointSynchronize::is_at_safepoint()) {
>>>>>       ??? hs.do_thread(state->get_thread());
>>>>>       ? } else {
>>>>>       ??? Handshake::execute(&hs, state->get_thread());
>>>>>       ? }
>>>>> (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>>>> HandshakeClosure() constructor)
>>>>>
>>>>> I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>>>> always called in a nested operation or just sometimes.
>>>>>
>>>>> Thanks,
>>>>> Patricio
>>>>>
>>>>> On 2/12/20 7:23 AM, Reingruber, Richard wrote:
>>>>>> // Repost including hotspot runtime and gc lists.
>>>>>> // Dean Long suggested to do so, because the enhancement replaces a vm operation
>>>>>> // with a handshake.
>>>>>> // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> could I please get reviews for this small enhancement in hotspot's jvmti implementation:
>>>>>>
>>>>>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/
>>>>>> Bug:    https://bugs.openjdk.java.net/browse/JDK-8238585
>>>>>>
>>>>>> The change avoids making all compiled methods on stack not_entrant when switching a java thread to
>>>>>> interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack.
>>>>>>
>>>>>> Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations.
>>>>>>
>>>>>> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms.
>>>>>>
>>>>>> Thanks, Richard.
>>>>>>
>>>>>> See also my question if anyone knows a reason for making the compiled methods not_entrant:
>>>>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html


From richard.reingruber at sap.com  Sat Apr 25 10:28:00 2020
From: richard.reingruber at sap.com (Reingruber, Richard)
Date: Sat, 25 Apr 2020 10:28:00 +0000
Subject: RFR(S) 8238585: Use handshake for
 JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled
 methods on stack not_entrant
In-Reply-To: <7bb8abdc-6c49-3059-23d5-9552bd80b480@oracle.com>
References: <AM0PR0202MB33311AB7623F4164335286309B1B0@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <3c59b9f9-ec38-18c9-8f24-e1186a08a04a@oracle.com>
 <AM0PR0202MB3331FD660B315CEC25B1DC209B150@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <410eed04-e2ef-0f4f-1c56-19e6734a10f6@oracle.com>
 <AM0PR0202MB33318BC60E3460662D42A40D9B150@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <AM0PR0202MB33319C2F9595CC3C2FA1617B9BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <e1244807-74b4-755b-17c2-dce3263ce337@oss.nttdata.com>
 <AM0PR0202MB333172BA4EAF92D0BC3BA9379BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <81d7caa8-4244-85f3-4d4e-78117fe5e25b@oss.nttdata.com>
 <AM0PR0202MB3331324D56A68E08CE7CDAE69BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <11c78b30-de04-544d-3a10-811ebf663bf2@oracle.com>
 <AM0PR0202MB3331610F89DFE2AD378B52289BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <7bb8abdc-6c49-3059-23d5-9552bd80b480@oracle.com>
Message-ID: <AM0PR0202MB333106369C015452B2E0283F9BD10@AM0PR0202MB3331.eurprd02.prod.outlook.com>

Hi Patricio,

thanks a lot for all the explanations. At least to me they are really helpful. :)

Cheers, Richard.

-----Original Message-----
From: Patricio Chilano <patricio.chilano.mateo at oracle.com> 
Sent: Samstag, 25. April 2020 11:23
To: Reingruber, Richard <richard.reingruber at sap.com>; Yasumasa Suenaga <suenaga at oss.nttdata.com>; serguei.spitsyn at oracle.com; Vladimir Ivanov <vladimir.x.ivanov at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant

Hi Richard,

On 4/24/20 6:41 PM, Reingruber, Richard wrote:
> Hi Patricio,
>
>>> @Patricio, coming back to my question [1]:
>>>
>>> In the example you gave in your answer [2]: the java thread would execute a vm operation during a
>>> direct handshake operation, while the VMThread is actually in the middle of a VM_HandshakeAllThreads
>>> operation, waiting to handshake the same handshakee: why can't the VMThread just proceed? The
>>> handshakee would be safepoint safe, wouldn't it?
>> Because the VMThread would not be able to decrement _processing_sem to
>> claim the operation and execute the closure for that handshakee. If
>> another JavaThread is doing a direct handshake with that same handshakee
>> and called a new VM operation inside the execution of the
>> HandshakeClosure in do_handshake(), nobody would be able to decrement
>> the _processing_sem anymore until the original direct operation finished
>> and the semaphore is signaled again.
> Thanks, understood. On a higher level: a JavaThread can have at most one handshake operation being
> processed at at time.
Exactly. As of now we don't handle the case where another handshake 
operation on the same handshakee is called inside 
_handshake_cl->do_thread(). If this happens we will deadlock.

>> So this can happen despite the
>> state of the handshakee is "handshake/safepoint safe". Changing the
>> nested VM operation to be a direct handshake would have the same issue.
>> Actually as the code is right now we would not even get pass setting the
>> handshake operation because in that case we would block in the
>> _handshake_turn_sem for the same reason.
> Don't really understand the details here, but that's ok.
> Interesting that _handshake_turn_sem gets signaled before or after do_handshake() depending if the
> handshake operation is processed by handshakee. Comments say "Disarm before/after executing
> operation" but not why :)
Yes, that pattern actually relates with clearing _operation and predates 
direct handshakes. In theory we should always call do_handshake() first 
and then clear the handshake. This is what we do when the operation is 
processed by the handshaker, and it is necessary to be that way, 
otherwise if we clear the handshake first then the handshakee might 
transition from the safe state and never see that it actually has to 
stop for the handshake. Now, when the handshake operation is processed 
by the handshakee itself we don't have that issue, so it doesn't matter 
if we clear it before or after. The reason we do it before is to avoid 
the VMThread to execute unnecessary instructions in try_process(). This 
is specially true for the VM_HandshakeAllThreads operation case. If the 
VMThread sees that a JavaThread doesn't have an operation set, it can 
just continue to try to process the next JavaThread, instead of going 
through the unnecessary steps of checking the state of the JavaThread 
and trying to execute a try_wait() operation on the _processing_sem 
which we know won't succeed. Now for the direct handshake case doing it 
before or after doesn't really matter and so I just copied the pattern 
from the non-direct case to make it consistent in that same method.


>> So changing VM_SetFramePop to use direct handshakes in the future will
>> probably create that last issue I mentioned. Now, since it is executed
>> at a safepoint, with your workaround in enter_interp_only_mode() we
>> avoid those nested issues in . Maybe 8239084 would have to be revisited
>> to address nested operations in all cases. It is not clear to me now
>> though if we should handle that in the handshake code or the caller of a
>> certain operation should know it might be called in a nested scenario
>> and should handle it.
> Last question: is it ok for the processor of a direct handshake operation to do safepoint/handshake
> checks? I wouldn't see a reason, why not. But certainly I would avoid it.
I tried to think of possible issues with that (independent of the 
closure logic) but I couldn't find a specific one. If the handshakee 
tries to process a pending handshake, process_by_self() will just return 
without calling process_self_inner() since it will detect it is already 
inside a handshake. And that behaviour makes sense since there is no 
point in trying to execute a new handshake operation if you are in the 
middle of another one. If the handshaker inside the closure checks for 
its own pending handshakes that also seems okay (this will by itself 
also check for safepoints in the transitions). Checking for safepoints 
in both cases seems more tricky but I couldn't think of a concrete issue 
with that.
In any case I would also avoid checking for safepoints/handshakes inside 
the handshake closure. You might get issues related to the actual logic 
of the closure, like the typical deadlock because of trying to grab the 
same lock (although it's true that you always have to deal with that 
kind of problems when checking for safepoint/handshakes), or coming back 
from the safepoint/handshake and failing because some state you didn't 
expect to change in the middle of the handshake actually changed.


Thanks,
Patricio
>> I'll look a bit more at the updated patch but at first glance looks good.
> Thanks,
> Richard.
>
> -----Original Message-----
> From: Patricio Chilano <patricio.chilano.mateo at oracle.com>
> Sent: Freitag, 24. April 2020 19:14
> To: Reingruber, Richard <richard.reingruber at sap.com>; Yasumasa Suenaga <suenaga at oss.nttdata.com>; serguei.spitsyn at oracle.com; Vladimir Ivanov <vladimir.x.ivanov at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>
> Hi Richard,
>
> Just jumping into your last question for now.? : )
>
>
> On 4/24/20 1:08 PM, Reingruber, Richard wrote:
>> Hi Yasumasa, Patricio,
>>
>>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
>>>>> Does it help you? I think it gives you to remove workaround.
>>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake
>>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to
>>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes.
>>> Thanks for your information.
>>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop.
>>> I will modify and will test it after yours.
>> Thanks :)
>>
>>>> Also my first impression was that it won't be that easy from a synchronization point of view to
>>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls
>>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where
>>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear
>>>> to me, how this has to be handled.
>>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event.
>> Yes. To me it is unclear what synchronization is necessary, if it is called during a handshake. And
>> also I'm unsure if a thread should do safepoint checks while executing a handshake.
>>
>> @Patricio, coming back to my question [1]:
>>
>> In the example you gave in your answer [2]: the java thread would execute a vm operation during a
>> direct handshake operation, while the VMThread is actually in the middle of a VM_HandshakeAllThreads
>> operation, waiting to handshake the same handshakee: why can't the VMThread just proceed? The
>> handshakee would be safepoint safe, wouldn't it?
> Because the VMThread would not be able to decrement _processing_sem to
> claim the operation and execute the closure for that handshakee. If
> another JavaThread is doing a direct handshake with that same handshakee
> and called a new VM operation inside the execution of the
> HandshakeClosure in do_handshake(), nobody would be able to decrement
> the _processing_sem anymore until the original direct operation finished
> and the semaphore is signaled again. So this can happen despite the
> state of the handshakee is "handshake/safepoint safe". Changing the
> nested VM operation to be a direct handshake would have the same issue.
> Actually as the code is right now we would not even get pass setting the
> handshake operation because in that case we would block in the
> _handshake_turn_sem for the same reason.
>
> So changing VM_SetFramePop to use direct handshakes in the future will
> probably create that last issue I mentioned. Now, since it is executed
> at a safepoint, with your workaround in enter_interp_only_mode() we
> avoid those nested issues in . Maybe 8239084 would have to be revisited
> to address nested operations in all cases. It is not clear to me now
> though if we should handle that in the handshake code or the caller of a
> certain operation should know it might be called in a nested scenario
> and should handle it.
>
> I'll look a bit more at the updated patch but at first glance looks good.
>
> Thanks!
>
> Patricio
>> Thanks, Richard.
>>
>> [1] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301677&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301677
>>
>> [2] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301763&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301763
>>
>> -----Original Message-----
>> From: Yasumasa Suenaga <suenaga at oss.nttdata.com>
>> Sent: Freitag, 24. April 2020 17:23
>> To: Reingruber, Richard <richard.reingruber at sap.com>; Patricio Chilano <patricio.chilano.mateo at oracle.com>; serguei.spitsyn at oracle.com; Vladimir Ivanov <vladimir.x.ivanov at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>
>> Hi Richard,
>>
>> On 2020/04/24 23:44, Reingruber, Richard wrote:
>>> Hi Yasumasa,
>>>
>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
>>>> Does it help you? I think it gives you to remove workaround.
>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake
>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to
>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes.
>> Thanks for your information.
>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop.
>> I will modify and will test it after yours.
>>
>>
>>> Also my first impression was that it won't be that easy from a synchronization point of view to
>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls
>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where
>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear
>>> to me, how this has to be handled.
>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event.
>>
>>
>> Thanks,
>>
>> Yasumasa
>>
>>
>>> So it appears to me that it would be easier to push JDK-8242427 after this (JDK-8238585).
>>>
>>>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.)
>>> Would be interesting to see how you handled the issues above :)
>>>
>>> Thanks, Richard.
>>>
>>> [1] See question in comment https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14302030&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14302030
>>>
>>> -----Original Message-----
>>> From: Yasumasa Suenaga <suenaga at oss.nttdata.com>
>>> Sent: Freitag, 24. April 2020 13:34
>>> To: Reingruber, Richard <richard.reingruber at sap.com>; Patricio Chilano <patricio.chilano.mateo at oracle.com>; serguei.spitsyn at oracle.com; Vladimir Ivanov <vladimir.x.ivanov at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>>
>>> Hi Richard,
>>>
>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
>>> Does it help you? I think it gives you to remove workaround.
>>>
>>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.)
>>>
>>>
>>> Thanks,
>>>
>>> Yasumasa
>>>
>>>
>>> On 2020/04/24 17:18, Reingruber, Richard wrote:
>>>> Hi Patricio, Vladimir, and Serguei,
>>>>
>>>> now that direct handshakes are available, I've updated the patch to make use of them.
>>>>
>>>> In addition I have done some clean-up changes I missed in the first webrev.
>>>>
>>>> Finally I have implemented the workaround suggested by Patricio to avoid nesting the handshake
>>>> into the vm operation VM_SetFramePop [1]
>>>>
>>>> Kindly review again:
>>>>
>>>> Webrev:        http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1/
>>>> Webrev(delta): http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1.inc/
>>>>
>>>> I updated the JBS item explaining why the vm operation VM_EnterInterpOnlyMode can be replaced with a
>>>> direct handshake:
>>>>
>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8238585
>>>>
>>>> Testing:
>>>>
>>>> * JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms.
>>>>
>>>> * Submit-repo: mach5-one-rrich-JDK-8238585-20200423-1436-10441737
>>>>
>>>> Thanks,
>>>> Richard.
>>>>
>>>> [1] An assertion in Handshake::execute_direct() fails, if called be VMThread, because it is no JavaThread.
>>>>
>>>> -----Original Message-----
>>>> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of Reingruber, Richard
>>>> Sent: Freitag, 14. Februar 2020 19:47
>>>> To: Patricio Chilano <patricio.chilano.mateo at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>>>> Subject: RE: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>>>
>>>> Hi Patricio,
>>>>
>>>>       > > I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a
>>>>       > > handshake cannot be nested in a vm operation. Maybe it should be asserted in the
>>>>       > > Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation?
>>>>       > >
>>>>       > >    > Alternatively I think you could do something similar to what we do in
>>>>       > >    > Deoptimization::deoptimize_all_marked():
>>>>       > >    >
>>>>       > >    >    EnterInterpOnlyModeClosure hs;
>>>>       > >    >    if (SafepointSynchronize::is_at_safepoint()) {
>>>>       > >    >      hs.do_thread(state->get_thread());
>>>>       > >    >    } else {
>>>>       > >    >      Handshake::execute(&hs, state->get_thread());
>>>>       > >    >    }
>>>>       > >    > (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>>>       > >    > HandshakeClosure() constructor)
>>>>       > >
>>>>       > > Maybe this could be used also in the Handshake::execute() methods as general solution?
>>>>       > Right, we could also do that. Avoiding to clear the polling page in
>>>>       > HandshakeState::clear_handshake() should be enough to fix this issue and
>>>>       > execute a handshake inside a safepoint, but adding that "if" statement
>>>>       > in Hanshake::execute() sounds good to avoid all the extra code that we
>>>>       > go through when executing a handshake. I filed 8239084 to make that change.
>>>>
>>>> Thanks for taking care of this and creating the RFE.
>>>>
>>>>       >
>>>>       > >    > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>>>       > >    > always called in a nested operation or just sometimes.
>>>>       > >
>>>>       > > At least one execution path without vm operation exists:
>>>>       > >
>>>>       > >    JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void
>>>>       > >      JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong
>>>>       > >        JvmtiEventControllerPrivate::recompute_enabled() : void
>>>>       > >          JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches)
>>>>       > >            JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void
>>>>       > >              JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError
>>>>       > >                jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError
>>>>       > >
>>>>       > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a
>>>>       > > handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further
>>>>       > > encouraged to do it with a handshake :)
>>>>       > Ah! I think you can still do it with a handshake with the
>>>>       > Deoptimization::deoptimize_all_marked() like solution. I can change the
>>>>       > if-else statement with just the Handshake::execute() call in 8239084.
>>>>       > But up to you.  : )
>>>>
>>>> Well, I think that's enough encouragement :)
>>>> I'll wait for 8239084 and try then again.
>>>> (no urgency and all)
>>>>
>>>> Thanks,
>>>> Richard.
>>>>
>>>> -----Original Message-----
>>>> From: Patricio Chilano <patricio.chilano.mateo at oracle.com>
>>>> Sent: Freitag, 14. Februar 2020 15:54
>>>> To: Reingruber, Richard <richard.reingruber at sap.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>>>
>>>> Hi Richard,
>>>>
>>>> On 2/14/20 9:58 AM, Reingruber, Richard wrote:
>>>>> Hi Patricio,
>>>>>
>>>>> thanks for having a look.
>>>>>
>>>>>        > I?m only commenting on the handshake changes.
>>>>>        > I see that operation VM_EnterInterpOnlyMode can be called inside
>>>>>        > operation VM_SetFramePop which also allows nested operations. Here is a
>>>>>        > comment in VM_SetFramePop definition:
>>>>>        >
>>>>>        > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is
>>>>>        > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled.
>>>>>        >
>>>>>        > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we
>>>>>        > could have a handshake inside a safepoint operation. The issue I see
>>>>>        > there is that at the end of the handshake the polling page of the target
>>>>>        > thread could be disarmed. So if the target thread happens to be in a
>>>>>        > blocked state just transiently and wakes up then it will not stop for
>>>>>        > the ongoing safepoint. Maybe I can file an RFE to assert that the
>>>>>        > polling page is armed at the beginning of disarm_safepoint().
>>>>>
>>>>> I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a
>>>>> handshake cannot be nested in a vm operation. Maybe it should be asserted in the
>>>>> Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation?
>>>>>
>>>>>        > Alternatively I think you could do something similar to what we do in
>>>>>        > Deoptimization::deoptimize_all_marked():
>>>>>        >
>>>>>        >    EnterInterpOnlyModeClosure hs;
>>>>>        >    if (SafepointSynchronize::is_at_safepoint()) {
>>>>>        >      hs.do_thread(state->get_thread());
>>>>>        >    } else {
>>>>>        >      Handshake::execute(&hs, state->get_thread());
>>>>>        >    }
>>>>>        > (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>>>>        > HandshakeClosure() constructor)
>>>>>
>>>>> Maybe this could be used also in the Handshake::execute() methods as general solution?
>>>> Right, we could also do that. Avoiding to clear the polling page in
>>>> HandshakeState::clear_handshake() should be enough to fix this issue and
>>>> execute a handshake inside a safepoint, but adding that "if" statement
>>>> in Hanshake::execute() sounds good to avoid all the extra code that we
>>>> go through when executing a handshake. I filed 8239084 to make that change.
>>>>
>>>>>        > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>>>>        > always called in a nested operation or just sometimes.
>>>>>
>>>>> At least one execution path without vm operation exists:
>>>>>
>>>>>        JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void
>>>>>          JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong
>>>>>            JvmtiEventControllerPrivate::recompute_enabled() : void
>>>>>              JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches)
>>>>>                JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void
>>>>>                  JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError
>>>>>                    jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError
>>>>>
>>>>> I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a
>>>>> handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further
>>>>> encouraged to do it with a handshake :)
>>>> Ah! I think you can still do it with a handshake with the
>>>> Deoptimization::deoptimize_all_marked() like solution. I can change the
>>>> if-else statement with just the Handshake::execute() call in 8239084.
>>>> But up to you.? : )
>>>>
>>>> Thanks,
>>>> Patricio
>>>>> Thanks again,
>>>>> Richard.
>>>>>
>>>>> -----Original Message-----
>>>>> From: Patricio Chilano <patricio.chilano.mateo at oracle.com>
>>>>> Sent: Donnerstag, 13. Februar 2020 18:47
>>>>> To: Reingruber, Richard <richard.reingruber at sap.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>>>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>>>>
>>>>> Hi Richard,
>>>>>
>>>>> I?m only commenting on the handshake changes.
>>>>> I see that operation VM_EnterInterpOnlyMode can be called inside
>>>>> operation VM_SetFramePop which also allows nested operations. Here is a
>>>>> comment in VM_SetFramePop definition:
>>>>>
>>>>> // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is
>>>>> // called from the JvmtiEventControllerPrivate::recompute_thread_enabled.
>>>>>
>>>>> So if we change VM_EnterInterpOnlyMode to be a handshake, then now we
>>>>> could have a handshake inside a safepoint operation. The issue I see
>>>>> there is that at the end of the handshake the polling page of the target
>>>>> thread could be disarmed. So if the target thread happens to be in a
>>>>> blocked state just transiently and wakes up then it will not stop for
>>>>> the ongoing safepoint. Maybe I can file an RFE to assert that the
>>>>> polling page is armed at the beginning of disarm_safepoint().
>>>>>
>>>>> I think one option could be to remove
>>>>> SafepointMechanism::disarm_if_needed() in
>>>>> HandshakeState::clear_handshake() and let each JavaThread disarm itself
>>>>> for the handshake case.
>>>>>
>>>>> Alternatively I think you could do something similar to what we do in
>>>>> Deoptimization::deoptimize_all_marked():
>>>>>
>>>>>       ? EnterInterpOnlyModeClosure hs;
>>>>>       ? if (SafepointSynchronize::is_at_safepoint()) {
>>>>>       ??? hs.do_thread(state->get_thread());
>>>>>       ? } else {
>>>>>       ??? Handshake::execute(&hs, state->get_thread());
>>>>>       ? }
>>>>> (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>>>> HandshakeClosure() constructor)
>>>>>
>>>>> I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>>>> always called in a nested operation or just sometimes.
>>>>>
>>>>> Thanks,
>>>>> Patricio
>>>>>
>>>>> On 2/12/20 7:23 AM, Reingruber, Richard wrote:
>>>>>> // Repost including hotspot runtime and gc lists.
>>>>>> // Dean Long suggested to do so, because the enhancement replaces a vm operation
>>>>>> // with a handshake.
>>>>>> // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> could I please get reviews for this small enhancement in hotspot's jvmti implementation:
>>>>>>
>>>>>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/
>>>>>> Bug:    https://bugs.openjdk.java.net/browse/JDK-8238585
>>>>>>
>>>>>> The change avoids making all compiled methods on stack not_entrant when switching a java thread to
>>>>>> interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack.
>>>>>>
>>>>>> Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations.
>>>>>>
>>>>>> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms.
>>>>>>
>>>>>> Thanks, Richard.
>>>>>>
>>>>>> See also my question if anyone knows a reason for making the compiled methods not_entrant:
>>>>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html


From thomas.stuefe at gmail.com  Sat Apr 25 17:12:11 2020
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Sat, 25 Apr 2020 19:12:11 +0200
Subject: RFR(S): 8233014: Enable ShowCodeDetailsInExceptionMessages per
 default.
In-Reply-To: <AM4PR0201MB21469B9BDC047A9756AC1C6EECD00@AM4PR0201MB2146.eurprd02.prod.outlook.com>
References: <AM4PR0201MB21469B9BDC047A9756AC1C6EECD00@AM4PR0201MB2146.eurprd02.prod.outlook.com>
Message-ID: <CAA-vtUyVwk24UKTu5caG-Vg6G=iUNWCq_72D7m48+XKiD8JgiA@mail.gmail.com>

Looks good.

Wonder what would be the point of switching it off though. At some time we
may just remove that switch.

Cheers, Thomas

On Fri, Apr 24, 2020 at 4:41 PM Lindenmaier, Goetz <
goetz.lindenmaier at sap.com> wrote:

> Hi,
>
> I would like to enable JEP 358 Helpful NullPointerExceptions by default:
>
> http://cr.openjdk.java.net/~goetz/wr19/8233014-enable_NPE_per_default/02/
>
> Please review.
>
> The change passed our testing and jdk-submit.
>
> Best regards,
>  Goetz.
>

From kim.barrett at oracle.com  Sat Apr 25 23:57:46 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Sat, 25 Apr 2020 19:57:46 -0400
Subject: RFR(S) : 8242314 : use reproducible random in vmTestbase shared
 code
In-Reply-To: <A59F9A91-D3D4-4D98-BD02-C44E1C3790E4@oracle.com>
References: <A59F9A91-D3D4-4D98-BD02-C44E1C3790E4@oracle.com>
Message-ID: <4C16AD88-51F8-43F5-BAC1-16083E4A3C71@oracle.com>

> On Apr 22, 2020, at 2:03 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
> 
> http://cr.openjdk.java.net/~iignatyev/8242314/webrev.00/ 
>> 122 lines changed: 71 ins; 19 del; 32 mod;
> 
> 
> Hi all,
> 
> could you please review this small patch which updates shared vmTestbase  test code to use j.t.l.Utils.getRandomInstance() as a random number generator or as a source of seed values? this patch lays the ground work for updates of :vmTestbase_* test groups which are to be done by remaining subtasks of 8241623[1], such split make it possible to work on/review/test testgroup-specific changes in concurrently.  
> 
> unfortunately, such extra dependency on /test/lib classes broke the shaky balance and caused some tests to fail due to "split testlibrary" problem (see 8188828[2] for the definition), the solution to that is to remove all dependencies on /test/lib classes from ExecDriver class, so the patch also includes "inlining" of several jdk.test.lib.Utils and Platform methods/fields into ExecDriver.
> 
> webrev: http://cr.openjdk.java.net/~iignatyev/8242314/webrev.00/ 
> testing: all :vmTestbase_* test groups on {linux,windows,macosx}-x64 and solaris-sparcv9
> JBS: https://bugs.openjdk.java.net/browse/JDK-8242314
> 
> [1] https://bugs.openjdk.java.net/browse/JDK-8241623
> [2] https://bugs.openjdk.java.net/browse/JDK-8188828
> 
> Thanks,
> -- Igor

The test changes look fine.

But I don't feel like I understand the "split library" problem well
enough to review the ExecDriver changes.  I will say that I don't like
the solution of copying code, and hope there is a better way.  Is
there a bug for the problem?  It seems like it could crop up again,
perhaps repeatedly, in which case the code copying becomes increasingly
bad.  And it might be a good idea to leave some breadcrumbs associated
with this code copying workaround, so an eventual better fix could also
cleanup the duplication.


From igor.ignatyev at oracle.com  Sun Apr 26 01:01:32 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Sat, 25 Apr 2020 18:01:32 -0700
Subject: RFR(S) : 8242314 : use reproducible random in vmTestbase shared
 code
In-Reply-To: <4C16AD88-51F8-43F5-BAC1-16083E4A3C71@oracle.com>
References: <A59F9A91-D3D4-4D98-BD02-C44E1C3790E4@oracle.com>
 <4C16AD88-51F8-43F5-BAC1-16083E4A3C71@oracle.com>
Message-ID: <4C10A003-6286-4272-AB7E-FC20659A6226@oracle.com>


> On Apr 25, 2020, at 4:57 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
> 
>> On Apr 22, 2020, at 2:03 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
>> 
>> http://cr.openjdk.java.net/~iignatyev/8242314/webrev.00/ 
>>> 122 lines changed: 71 ins; 19 del; 32 mod;
>> 
>> 
>> Hi all,
>> 
>> could you please review this small patch which updates shared vmTestbase  test code to use j.t.l.Utils.getRandomInstance() as a random number generator or as a source of seed values? this patch lays the ground work for updates of :vmTestbase_* test groups which are to be done by remaining subtasks of 8241623[1], such split make it possible to work on/review/test testgroup-specific changes in concurrently.  
>> 
>> unfortunately, such extra dependency on /test/lib classes broke the shaky balance and caused some tests to fail due to "split testlibrary" problem (see 8188828[2] for the definition), the solution to that is to remove all dependencies on /test/lib classes from ExecDriver class, so the patch also includes "inlining" of several jdk.test.lib.Utils and Platform methods/fields into ExecDriver.
>> 
>> webrev: http://cr.openjdk.java.net/~iignatyev/8242314/webrev.00/ 
>> testing: all :vmTestbase_* test groups on {linux,windows,macosx}-x64 and solaris-sparcv9
>> JBS: https://bugs.openjdk.java.net/browse/JDK-8242314
>> 
>> [1] https://bugs.openjdk.java.net/browse/JDK-8241623
>> [2] https://bugs.openjdk.java.net/browse/JDK-8188828
>> 
>> Thanks,
>> -- Igor
> 
> The test changes look fine.
> 
> But I don't feel like I understand the "split library" problem well
> enough to review the ExecDriver changes.  I will say that I don't like
> the solution of copying code, and hope there is a better way.  Is
> there a bug for the problem?  It seems like it could crop up again,
> perhaps repeatedly, in which case the code copying becomes increasingly
> bad.  And it might be a good idea to leave some breadcrumbs associated
> with this code copying workaround, so an eventual better fix could also
> cleanup the duplication.
> 

Hi Kim,

thanks for look at this!

"split library" problem occur when a test "have these properties: 
 - has more than one @library 
 - depends on class Foo from library A which depends on class Bar from another library B 
 - has an implicit/explicit build for class Baz from libraries which depends on class Bar"[1]
in this case A=/vmTestbase, B=/test/lib, Foo=ExecDriver, Bar=jdk.test.lib.Utils, and Baz can be pretty much any of class from jdk.test.lib, e.g. jdk.test.lib.process.ProcessTools.

there is a better way to fix that, which we discussed have discussed in the aftermath of our encounter of this problem (when 8188828 was used as a stopgap solution), and this way is to redesign/simplify and potentially modularize test libraries so it will be easier/saner to build them. unfortunately, we are not there yet, yet we are making progress; this activity is tracked by 8211289[2], 8211290[3], and 8211358[4].

as of breadcrumbs, I think I left mentions of the copied methods in all the methods, e.g. L#144; and the only copied pieces of code which don't have such breadcrumbs are TEST_CLASS_PATH and TEST_JDK static fields. as they are quite straightforward, I didn't feel that it's need to refer back to the original code. but if you like, I can add comments there as well.

Thanks,
-- Igor 


[1] https://bugs.openjdk.java.net/browse/JDK-8188828?focusedCommentId=14121228&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14121228 <https://bugs.openjdk.java.net/browse/JDK-8188828?focusedCommentId=14121228&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14121228>
[2] https://bugs.openjdk.java.net/browse/JDK-8211289 <https://bugs.openjdk.java.net/browse/JDK-8211289>
[3] https://bugs.openjdk.java.net/browse/JDK-8211290 <https://bugs.openjdk.java.net/browse/JDK-8211290>
[4] https://bugs.openjdk.java.net/browse/JDK-8211358 <https://bugs.openjdk.java.net/browse/JDK-8211358>




From kim.barrett at oracle.com  Sun Apr 26 03:45:38 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Sat, 25 Apr 2020 23:45:38 -0400
Subject: RFR(S) : 8242314 : use reproducible random in vmTestbase shared
 code
In-Reply-To: <4C10A003-6286-4272-AB7E-FC20659A6226@oracle.com>
References: <A59F9A91-D3D4-4D98-BD02-C44E1C3790E4@oracle.com>
 <4C16AD88-51F8-43F5-BAC1-16083E4A3C71@oracle.com>
 <4C10A003-6286-4272-AB7E-FC20659A6226@oracle.com>
Message-ID: <FFE3D45D-5702-403C-995F-B09748A006EA@oracle.com>

> On Apr 25, 2020, at 9:01 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
> 
> 
>> On Apr 25, 2020, at 4:57 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
>> 
>>> On Apr 22, 2020, at 2:03 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
>>> 
>>> http://cr.openjdk.java.net/~iignatyev/8242314/webrev.00/ 
>>>> 122 lines changed: 71 ins; 19 del; 32 mod;
>>> 
>>> 
>>> Hi all,
>>> 
>>> could you please review this small patch which updates shared vmTestbase  test code to use j.t.l.Utils.getRandomInstance() as a random number generator or as a source of seed values? this patch lays the ground work for updates of :vmTestbase_* test groups which are to be done by remaining subtasks of 8241623[1], such split make it possible to work on/review/test testgroup-specific changes in concurrently.  
>>> 
>>> unfortunately, such extra dependency on /test/lib classes broke the shaky balance and caused some tests to fail due to "split testlibrary" problem (see 8188828[2] for the definition), the solution to that is to remove all dependencies on /test/lib classes from ExecDriver class, so the patch also includes "inlining" of several jdk.test.lib.Utils and Platform methods/fields into ExecDriver.
>>> 
>>> webrev: http://cr.openjdk.java.net/~iignatyev/8242314/webrev.00/ 
>>> testing: all :vmTestbase_* test groups on {linux,windows,macosx}-x64 and solaris-sparcv9
>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8242314
>>> 
>>> [1] https://bugs.openjdk.java.net/browse/JDK-8241623
>>> [2] https://bugs.openjdk.java.net/browse/JDK-8188828
>>> 
>>> Thanks,
>>> -- Igor
>> 
>> The test changes look fine.
>> 
>> But I don't feel like I understand the "split library" problem well
>> enough to review the ExecDriver changes.  I will say that I don't like
>> the solution of copying code, and hope there is a better way.  Is
>> there a bug for the problem?  It seems like it could crop up again,
>> perhaps repeatedly, in which case the code copying becomes increasingly
>> bad.  And it might be a good idea to leave some breadcrumbs associated
>> with this code copying workaround, so an eventual better fix could also
>> cleanup the duplication.
>> 
> 
> Hi Kim,
> 
> thanks for look at this!
> 
> "split library" problem occur when a test "have these properties: 
>  - has more than one @library 
>  - depends on class Foo from library A which depends on class Bar from another library B 
>  - has an implicit/explicit build for class Baz from libraries which depends on class Bar"[1]
> in this case A=/vmTestbase, B=/test/lib, Foo=ExecDriver, Bar=jdk.test.lib.Utils, and Baz can be pretty much any of class from jdk.test.lib, e.g. jdk.test.lib.process.ProcessTools.

Yes, I read that already, from JDK-8188828.  But I don't understand
the problem; remember, I'm not really a Java programmer, I just fake
it sometimes :)  In particular, I don't have a good understanding of
Java packaging and modularization and such.  So I'd rather not be
counted as having reviewed that part of the change.

> there is a better way to fix that, which we discussed have discussed in the aftermath of our encounter of this problem (when 8188828 was used as a stopgap solution), and this way is to redesign/simplify and potentially modularize test libraries so it will be easier/saner to build them. unfortunately, we are not there yet, yet we are making progress; this activity is tracked by 8211289[2], 8211290[3], and 8211358[4].
> 
> as of breadcrumbs, I think I left mentions of the copied methods in all the methods, e.g. L#144; and the only copied pieces of code which don't have such breadcrumbs are TEST_CLASS_PATH and TEST_JDK static fields. as they are quite straightforward, I didn't feel that it's need to refer back to the original code. but if you like, I can add comments there as well.

Regarding the breadcrumb comments, I didn't recognize them as such.
Had they been something more than just a fully qualified method name,
i.e. something like "Copied from <fqname>", it would have been clear.
Is there a cleanup CR for later?

> Thanks,
> -- Igor 
> 
> 
> [1] https://bugs.openjdk.java.net/browse/JDK-8188828?focusedCommentId=14121228&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14121228
> [2] https://bugs.openjdk.java.net/browse/JDK-8211289
> [3] https://bugs.openjdk.java.net/browse/JDK-8211290
> [4] https://bugs.openjdk.java.net/browse/JDK-8211358



From igor.ignatyev at oracle.com  Sun Apr 26 04:22:05 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Sat, 25 Apr 2020 21:22:05 -0700
Subject: RFR(S) : 8242314 : use reproducible random in vmTestbase shared
 code
In-Reply-To: <FFE3D45D-5702-403C-995F-B09748A006EA@oracle.com>
References: <A59F9A91-D3D4-4D98-BD02-C44E1C3790E4@oracle.com>
 <4C16AD88-51F8-43F5-BAC1-16083E4A3C71@oracle.com>
 <4C10A003-6286-4272-AB7E-FC20659A6226@oracle.com>
 <FFE3D45D-5702-403C-995F-B09748A006EA@oracle.com>
Message-ID: <5FE778F4-FCCE-44E5-B856-48B374545D98@oracle.com>



> On Apr 25, 2020, at 8:45 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
> 
>> On Apr 25, 2020, at 9:01 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
>> 
>> 
>>> On Apr 25, 2020, at 4:57 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
>>> 
>>>> On Apr 22, 2020, at 2:03 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
>>>> 
>>>> http://cr.openjdk.java.net/~iignatyev/8242314/webrev.00/ 
>>>>> 122 lines changed: 71 ins; 19 del; 32 mod;
>>>> 
>>>> 
>>>> Hi all,
>>>> 
>>>> could you please review this small patch which updates shared vmTestbase  test code to use j.t.l.Utils.getRandomInstance() as a random number generator or as a source of seed values? this patch lays the ground work for updates of :vmTestbase_* test groups which are to be done by remaining subtasks of 8241623[1], such split make it possible to work on/review/test testgroup-specific changes in concurrently.  
>>>> 
>>>> unfortunately, such extra dependency on /test/lib classes broke the shaky balance and caused some tests to fail due to "split testlibrary" problem (see 8188828[2] for the definition), the solution to that is to remove all dependencies on /test/lib classes from ExecDriver class, so the patch also includes "inlining" of several jdk.test.lib.Utils and Platform methods/fields into ExecDriver.
>>>> 
>>>> webrev: http://cr.openjdk.java.net/~iignatyev/8242314/webrev.00/ 
>>>> testing: all :vmTestbase_* test groups on {linux,windows,macosx}-x64 and solaris-sparcv9
>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8242314
>>>> 
>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8241623
>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8188828
>>>> 
>>>> Thanks,
>>>> -- Igor
>>> 
>>> The test changes look fine.
>>> 
>>> But I don't feel like I understand the "split library" problem well
>>> enough to review the ExecDriver changes.  I will say that I don't like
>>> the solution of copying code, and hope there is a better way.  Is
>>> there a bug for the problem?  It seems like it could crop up again,
>>> perhaps repeatedly, in which case the code copying becomes increasingly
>>> bad.  And it might be a good idea to leave some breadcrumbs associated
>>> with this code copying workaround, so an eventual better fix could also
>>> cleanup the duplication.
>>> 
>> 
>> Hi Kim,
>> 
>> thanks for look at this!
>> 
>> "split library" problem occur when a test "have these properties: 
>> - has more than one @library 
>> - depends on class Foo from library A which depends on class Bar from another library B 
>> - has an implicit/explicit build for class Baz from libraries which depends on class Bar"[1]
>> in this case A=/vmTestbase, B=/test/lib, Foo=ExecDriver, Bar=jdk.test.lib.Utils, and Baz can be pretty much any of class from jdk.test.lib, e.g. jdk.test.lib.process.ProcessTools.
> 
> Yes, I read that already, from JDK-8188828.  But I don't understand
> the problem; remember, I'm not really a Java programmer, I just fake
> it sometimes :)  In particular, I don't have a good understanding of
> Java packaging and modularization and such.  So I'd rather not be
> counted as having reviewed that part of the change.

well, a usual Java programmer will have a little understanding of the problem either since it's kinda unique to jtreg, or rather to how we use it. let me try to elaborate more on that. @library tag tells jtreg where to look for source code during compilation, later when jtreg invokes javac to compile java source code to java bytecode it puts the result depending on the location of the class which provoked compilation, that's to say if you have '@build Bar' (or `@run main Bar`), jtreg will search for Bar in current test source directory and in all @libraries, if Bar is found in test source, jtreg will use test-specific directory as destination directory; if Bar is found in a @library, jtreg will use the directory specific for this library, and this directory is shared b/w different tests. to find which particular dependency of Bar to build, jtreg relies on javac's ability to implicitly compile dependencies, and the bytecode of all dependencies will be placed in the same directory regardless of the origin of dependencies source code. it all works great, unless you have a situation described above, as in such cases, you might end up w/ some part of @library be compiled into one directory, and some part in a directory of another @library -- a so-called "split library" problem, so the next test which uses 1st @library but doesn't use the 2nd will fail to find class files of some classes from 1st @library. I hope it makes this at least a little bit clearer. there is also a very good analyze of this problem from Ioi in CODETOOLS-7901986 -- https://bugs.openjdk.java.net/browse/CODETOOLS-7901986?focusedCommentId=14108320&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14108320 <https://bugs.openjdk.java.net/browse/CODETOOLS-7901986?focusedCommentId=14108320&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14108320>
  
> 
>> there is a better way to fix that, which we discussed have discussed in the aftermath of our encounter of this problem (when 8188828 was used as a stopgap solution), and this way is to redesign/simplify and potentially modularize test libraries so it will be easier/saner to build them. unfortunately, we are not there yet, yet we are making progress; this activity is tracked by 8211289[2], 8211290[3], and 8211358[4].
>> 
>> as of breadcrumbs, I think I left mentions of the copied methods in all the methods, e.g. L#144; and the only copied pieces of code which don't have such breadcrumbs are TEST_CLASS_PATH and TEST_JDK static fields. as they are quite straightforward, I didn't feel that it's need to refer back to the original code. but if you like, I can add comments there as well.
> 
> Regarding the breadcrumb comments, I didn't recognize them as such.
> Had they been something more than just a fully qualified method name,
> i.e. something like "Copied from <fqname>", it would have been clear.
> Is there a cleanup CR for later?
sure I'll update comments and file an RFE to clean this up. 
> 
>> Thanks,
>> -- Igor 
>> 
>> 
>> [1] https://bugs.openjdk.java.net/browse/JDK-8188828?focusedCommentId=14121228&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14121228 <https://bugs.openjdk.java.net/browse/JDK-8188828?focusedCommentId=14121228&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14121228>
>> [2] https://bugs.openjdk.java.net/browse/JDK-8211289 <https://bugs.openjdk.java.net/browse/JDK-8211289>
>> [3] https://bugs.openjdk.java.net/browse/JDK-8211290 <https://bugs.openjdk.java.net/browse/JDK-8211290>
>> [4] https://bugs.openjdk.java.net/browse/JDK-8211358 <https://bugs.openjdk.java.net/browse/JDK-8211358>

From david.holmes at oracle.com  Sun Apr 26 22:02:32 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 27 Apr 2020 08:02:32 +1000
Subject: RFR(S): 8233014: Enable ShowCodeDetailsInExceptionMessages per
 default.
In-Reply-To: <AM4PR0201MB21469B9BDC047A9756AC1C6EECD00@AM4PR0201MB2146.eurprd02.prod.outlook.com>
References: <AM4PR0201MB21469B9BDC047A9756AC1C6EECD00@AM4PR0201MB2146.eurprd02.prod.outlook.com>
Message-ID: <47f7b53e-1306-96a2-89d7-16d7dec2ee9f@oracle.com>

Hi Goetz,

This will need a CSR request to change the default flag value.

Otherwise change looks okay to me.

Thanks,
David

On 25/04/2020 12:36 am, Lindenmaier, Goetz wrote:
> Hi,
> 
> I would like to enable JEP 358 Helpful NullPointerExceptions by default:
> 
> http://cr.openjdk.java.net/~goetz/wr19/8233014-enable_NPE_per_default/02/
> 
> Please review.
> 
> The change passed our testing and jdk-submit.
> 
> Best regards,
>   Goetz.
> 

From david.holmes at oracle.com  Sun Apr 26 23:07:45 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 27 Apr 2020 09:07:45 +1000
Subject: RFR(S): 8233014: Enable ShowCodeDetailsInExceptionMessages per
 default.
In-Reply-To: <47f7b53e-1306-96a2-89d7-16d7dec2ee9f@oracle.com>
References: <AM4PR0201MB21469B9BDC047A9756AC1C6EECD00@AM4PR0201MB2146.eurprd02.prod.outlook.com>
 <47f7b53e-1306-96a2-89d7-16d7dec2ee9f@oracle.com>
Message-ID: <c8f4532f-48ff-cde4-06dc-a74a534f8f71@oracle.com>

Hi Goetz,

On 27/04/2020 8:02 am, David Holmes wrote:
> Hi Goetz,
> 
> This will need a CSR request to change the default flag value.

On that front, given this feature only just shipped in 14, on what basis 
are we ready to flip the flag in 15? We have no feedback on the feature 
and would not likely get any for some time.

> Otherwise change looks okay to me.

With regards to testing however, please ensure that turning this on does 
not inadvertently disrupt any tests. Passing jdk-submit is a minimal 
requirement here and more extensive testing is desirable.

Thanks,
David

> Thanks,
> David
> 
> On 25/04/2020 12:36 am, Lindenmaier, Goetz wrote:
>> Hi,
>>
>> I would like to enable JEP 358 Helpful NullPointerExceptions by default:
>>
>> http://cr.openjdk.java.net/~goetz/wr19/8233014-enable_NPE_per_default/02/
>>
>> Please review.
>>
>> The change passed our testing and jdk-submit.
>>
>> Best regards,
>> ? Goetz.
>>

From forax at univ-mlv.fr  Mon Apr 27 00:36:38 2020
From: forax at univ-mlv.fr (Remi Forax)
Date: Mon, 27 Apr 2020 02:36:38 +0200 (CEST)
Subject: RFR(S): 8233014: Enable ShowCodeDetailsInExceptionMessages per
 default.
In-Reply-To: <CAA-vtUyVwk24UKTu5caG-Vg6G=iUNWCq_72D7m48+XKiD8JgiA@mail.gmail.com>
References: <AM4PR0201MB21469B9BDC047A9756AC1C6EECD00@AM4PR0201MB2146.eurprd02.prod.outlook.com>
 <CAA-vtUyVwk24UKTu5caG-Vg6G=iUNWCq_72D7m48+XKiD8JgiA@mail.gmail.com>
Message-ID: <1366816673.66499.1587947798161.JavaMail.zimbra@u-pem.fr>

----- Mail original -----
> De: "Thomas St?fe" <thomas.stuefe at gmail.com>
> ?: "Goetz Lindenmaier" <goetz.lindenmaier at sap.com>
> Cc: "hotspot-dev" <hotspot-dev at openjdk.java.net>
> Envoy?: Samedi 25 Avril 2020 19:12:11
> Objet: Re: RFR(S): 8233014: Enable ShowCodeDetailsInExceptionMessages per default.

> Looks good.
> 
> Wonder what would be the point of switching it off though. At some time we
> may just remove that switch.
> 
> Cheers, Thomas

If some applications relies on
- specific error message in NPE, the application may cease to work, or log parsing will not work anymore, etc
- catching NPE, having detailed error message may slow the application.

so the point to keep the switch is to provide an escape hatch until people either fix their code or use another dependency,
nobody write this kind of code so it has to be in a dependency :)

regards
R?mi

> 
> On Fri, Apr 24, 2020 at 4:41 PM Lindenmaier, Goetz <
> goetz.lindenmaier at sap.com> wrote:
> 
>> Hi,
>>
>> I would like to enable JEP 358 Helpful NullPointerExceptions by default:
>>
>> http://cr.openjdk.java.net/~goetz/wr19/8233014-enable_NPE_per_default/02/
>>
>> Please review.
>>
>> The change passed our testing and jdk-submit.
>>
>> Best regards,
>>  Goetz.

From Alan.Bateman at oracle.com  Mon Apr 27 06:10:04 2020
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Mon, 27 Apr 2020 07:10:04 +0100
Subject: RFR(S): 8233014: Enable ShowCodeDetailsInExceptionMessages per
 default.
In-Reply-To: <47f7b53e-1306-96a2-89d7-16d7dec2ee9f@oracle.com>
References: <AM4PR0201MB21469B9BDC047A9756AC1C6EECD00@AM4PR0201MB2146.eurprd02.prod.outlook.com>
 <47f7b53e-1306-96a2-89d7-16d7dec2ee9f@oracle.com>
Message-ID: <49fe5ebf-c592-4bfb-35e1-52ac5aee69ff@oracle.com>

On 26/04/2020 23:02, David Holmes wrote:
> Hi Goetz,
>
> This will need a CSR request to change the default flag value.
Yes, a CSR is needed for this. I also suspect it needs further security 
review too (JEP 358 acknowledges the security concern).

-Alan.

From richard.reingruber at sap.com  Mon Apr 27 08:20:20 2020
From: richard.reingruber at sap.com (Reingruber, Richard)
Date: Mon, 27 Apr 2020 08:20:20 +0000
Subject: RFR(S): 8233014: Enable ShowCodeDetailsInExceptionMessages per
 default.
In-Reply-To: <AM4PR0201MB21469B9BDC047A9756AC1C6EECD00@AM4PR0201MB2146.eurprd02.prod.outlook.com>
References: <AM4PR0201MB21469B9BDC047A9756AC1C6EECD00@AM4PR0201MB2146.eurprd02.prod.outlook.com>
Message-ID: <AM0PR0202MB33319287A3A9BD5FC7FBFFD49BAF0@AM0PR0202MB3331.eurprd02.prod.outlook.com>

Hi Goetz,

Just a minor detail: the license check fails for the file SuppressMessagesTest.java

Try

  sh make/scripts/lic_check.sh -gpl $(hg log -v --rev tip |sed -n '/files:[[:space:]]*/s///p')

With

--- a/test/hotspot/jtreg/runtime/exceptionMsgs/NullPointerException/SuppressMessagesTest.java
+++ b/test/hotspot/jtreg/runtime/exceptionMsgs/NullPointerException/SuppressMessagesTest.java
@@ -1,6 +1,6 @@
 /*
+ * Copyright (c) 2019, 2020, SAP SE. All rights reserved.
  * Copyright (c) 2019, 2020, Oracle and/or its affiliates. All rights reserved.
- * Copyright (c) 2019, 2020 SAP SE. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * This code is free software; you can redistribute it and/or modify it

it succeeds. Note that I added a comma after 2020 also.

Cheers, Richard.

-----Original Message-----
From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of Lindenmaier, Goetz
Sent: Freitag, 24. April 2020 16:36
To: 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
Subject: RFR(S): 8233014: Enable ShowCodeDetailsInExceptionMessages per default.

Hi,

I would like to enable JEP 358 Helpful NullPointerExceptions by default:

http://cr.openjdk.java.net/~goetz/wr19/8233014-enable_NPE_per_default/02/

Please review.

The change passed our testing and jdk-submit.

Best regards,
 Goetz.

From david.holmes at oracle.com  Mon Apr 27 09:02:16 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 27 Apr 2020 19:02:16 +1000
Subject: RFR(S): 8233014: Enable ShowCodeDetailsInExceptionMessages per
 default.
In-Reply-To: <AM0PR0202MB33319287A3A9BD5FC7FBFFD49BAF0@AM0PR0202MB3331.eurprd02.prod.outlook.com>
References: <AM4PR0201MB21469B9BDC047A9756AC1C6EECD00@AM4PR0201MB2146.eurprd02.prod.outlook.com>
 <AM0PR0202MB33319287A3A9BD5FC7FBFFD49BAF0@AM0PR0202MB3331.eurprd02.prod.outlook.com>
Message-ID: <ef47b6db-6c5c-b292-f1a4-18a5af5ae6ac@oracle.com>

On 27/04/2020 6:20 pm, Reingruber, Richard wrote:
> Hi Goetz,
> 
> Just a minor detail: the license check fails for the file SuppressMessagesTest.java
> 
> Try
> 
>    sh make/scripts/lic_check.sh -gpl $(hg log -v --rev tip |sed -n '/files:[[:space:]]*/s///p')

Why did it fail?

> With
> 
> --- a/test/hotspot/jtreg/runtime/exceptionMsgs/NullPointerException/SuppressMessagesTest.java
> +++ b/test/hotspot/jtreg/runtime/exceptionMsgs/NullPointerException/SuppressMessagesTest.java
> @@ -1,6 +1,6 @@
>   /*
> + * Copyright (c) 2019, 2020, SAP SE. All rights reserved.
>    * Copyright (c) 2019, 2020, Oracle and/or its affiliates. All rights reserved.
> - * Copyright (c) 2019, 2020 SAP SE. All rights reserved.
>    * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>    *
>    * This code is free software; you can redistribute it and/or modify it
> 
> it succeeds. Note that I added a comma after 2020 also.

That doesn't seem to be the SAP format. (Though it is not 100% consistent.)

Cheers,
David

> Cheers, Richard.
> 
> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of Lindenmaier, Goetz
> Sent: Freitag, 24. April 2020 16:36
> To: 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: RFR(S): 8233014: Enable ShowCodeDetailsInExceptionMessages per default.
> 
> Hi,
> 
> I would like to enable JEP 358 Helpful NullPointerExceptions by default:
> 
> http://cr.openjdk.java.net/~goetz/wr19/8233014-enable_NPE_per_default/02/
> 
> Please review.
> 
> The change passed our testing and jdk-submit.
> 
> Best regards,
>   Goetz.
> 

From richard.reingruber at sap.com  Mon Apr 27 09:49:48 2020
From: richard.reingruber at sap.com (Reingruber, Richard)
Date: Mon, 27 Apr 2020 09:49:48 +0000
Subject: RFR(S): 8233014: Enable ShowCodeDetailsInExceptionMessages per
 default.
In-Reply-To: <ef47b6db-6c5c-b292-f1a4-18a5af5ae6ac@oracle.com>
References: <AM4PR0201MB21469B9BDC047A9756AC1C6EECD00@AM4PR0201MB2146.eurprd02.prod.outlook.com>
 <AM0PR0202MB33319287A3A9BD5FC7FBFFD49BAF0@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <ef47b6db-6c5c-b292-f1a4-18a5af5ae6ac@oracle.com>
Message-ID: <AM0PR0202MB33318D97F2504B4594DDD09B9BAF0@AM0PR0202MB3331.eurprd02.prod.outlook.com>

Hi David, Goetz,

> > Just a minor detail: the license check fails for the file SuppressMessagesTest.java
> > 
> > Try
> > 
> >    sh make/scripts/lic_check.sh -gpl $(hg log -v --rev tip |sed -n '/files:[[:space:]]*/s///p')

> Why did it fail?

Because there is no match for make/templates/gpl-header I reckon.

> > With
> > 
> > --- a/test/hotspot/jtreg/runtime/exceptionMsgs/NullPointerException/SuppressMessagesTest.java
> > +++ b/test/hotspot/jtreg/runtime/exceptionMsgs/NullPointerException/SuppressMessagesTest.java
> > @@ -1,6 +1,6 @@
> >   /*
> > + * Copyright (c) 2019, 2020, SAP SE. All rights reserved.
> >    * Copyright (c) 2019, 2020, Oracle and/or its affiliates. All rights reserved.
> > - * Copyright (c) 2019, 2020 SAP SE. All rights reserved.
> >    * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
> >    *
> >    * This code is free software; you can redistribute it and/or modify it
> > 
> > it succeeds. Note that I added a comma after 2020 also.

> That doesn't seem to be the SAP format. (Though it is not 100% consistent.)

Right. In all files I checked the Oracle copyright line came before other copyright lines (SAP, Red
Hat).

Maybe it is not applicable here?

Cheers, Richard.

-----Original Message-----
From: David Holmes <david.holmes at oracle.com> 
Sent: Montag, 27. April 2020 11:02
To: Reingruber, Richard <richard.reingruber at sap.com>; Lindenmaier, Goetz <goetz.lindenmaier at sap.com>; 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
Subject: Re: RFR(S): 8233014: Enable ShowCodeDetailsInExceptionMessages per default.

On 27/04/2020 6:20 pm, Reingruber, Richard wrote:
> Hi Goetz,
> 
> Just a minor detail: the license check fails for the file SuppressMessagesTest.java
> 
> Try
> 
>    sh make/scripts/lic_check.sh -gpl $(hg log -v --rev tip |sed -n '/files:[[:space:]]*/s///p')

Why did it fail?

> With
> 
> --- a/test/hotspot/jtreg/runtime/exceptionMsgs/NullPointerException/SuppressMessagesTest.java
> +++ b/test/hotspot/jtreg/runtime/exceptionMsgs/NullPointerException/SuppressMessagesTest.java
> @@ -1,6 +1,6 @@
>   /*
> + * Copyright (c) 2019, 2020, SAP SE. All rights reserved.
>    * Copyright (c) 2019, 2020, Oracle and/or its affiliates. All rights reserved.
> - * Copyright (c) 2019, 2020 SAP SE. All rights reserved.
>    * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>    *
>    * This code is free software; you can redistribute it and/or modify it
> 
> it succeeds. Note that I added a comma after 2020 also.

That doesn't seem to be the SAP format. (Though it is not 100% consistent.)

Cheers,
David

> Cheers, Richard.
> 
> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of Lindenmaier, Goetz
> Sent: Freitag, 24. April 2020 16:36
> To: 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: RFR(S): 8233014: Enable ShowCodeDetailsInExceptionMessages per default.
> 
> Hi,
> 
> I would like to enable JEP 358 Helpful NullPointerExceptions by default:
> 
> http://cr.openjdk.java.net/~goetz/wr19/8233014-enable_NPE_per_default/02/
> 
> Please review.
> 
> The change passed our testing and jdk-submit.
> 
> Best regards,
>   Goetz.
> 

From david.holmes at oracle.com  Mon Apr 27 09:58:01 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 27 Apr 2020 19:58:01 +1000
Subject: RFR(S): 8233014: Enable ShowCodeDetailsInExceptionMessages per
 default.
In-Reply-To: <AM0PR0202MB33318D97F2504B4594DDD09B9BAF0@AM0PR0202MB3331.eurprd02.prod.outlook.com>
References: <AM4PR0201MB21469B9BDC047A9756AC1C6EECD00@AM4PR0201MB2146.eurprd02.prod.outlook.com>
 <AM0PR0202MB33319287A3A9BD5FC7FBFFD49BAF0@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <ef47b6db-6c5c-b292-f1a4-18a5af5ae6ac@oracle.com>
 <AM0PR0202MB33318D97F2504B4594DDD09B9BAF0@AM0PR0202MB3331.eurprd02.prod.outlook.com>
Message-ID: <0fb381bf-0e86-3f9c-d8b3-d9fc3f0a46b2@oracle.com>

On 27/04/2020 7:49 pm, Reingruber, Richard wrote:
> Hi David, Goetz,
> 
>>> Just a minor detail: the license check fails for the file SuppressMessagesTest.java
>>>
>>> Try
>>>
>>>     sh make/scripts/lic_check.sh -gpl $(hg log -v --rev tip |sed -n '/files:[[:space:]]*/s///p')
> 
>> Why did it fail?
> 
> Because there is no match for make/templates/gpl-header I reckon.
> 
>>> With
>>>
>>> --- a/test/hotspot/jtreg/runtime/exceptionMsgs/NullPointerException/SuppressMessagesTest.java
>>> +++ b/test/hotspot/jtreg/runtime/exceptionMsgs/NullPointerException/SuppressMessagesTest.java
>>> @@ -1,6 +1,6 @@
>>>    /*
>>> + * Copyright (c) 2019, 2020, SAP SE. All rights reserved.
>>>     * Copyright (c) 2019, 2020, Oracle and/or its affiliates. All rights reserved.
>>> - * Copyright (c) 2019, 2020 SAP SE. All rights reserved.
>>>     * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>>>     *
>>>     * This code is free software; you can redistribute it and/or modify it
>>>
>>> it succeeds. Note that I added a comma after 2020 also.
> 
>> That doesn't seem to be the SAP format. (Though it is not 100% consistent.)
> 
> Right. In all files I checked the Oracle copyright line came before other copyright lines (SAP, Red
> Hat).

I was referring specifically to the comma after the second copyright 
year. It's an Oracle requirement, but not apparently a SAP one.

Most files with dual copyrights do have Oracle first, but that should 
not be a requirement for the header. There are many files without an 
Oracle copyright and if an Oracle engineer were to make significant 
changes and thus add the Oracle copyright, there's no rule I know that 
says the Oracle copyright should then come first. Sounds like a bug in 
the script to me.

Cheers,
David

> Maybe it is not applicable here?
> 
> Cheers, Richard.
> 
> -----Original Message-----
> From: David Holmes <david.holmes at oracle.com>
> Sent: Montag, 27. April 2020 11:02
> To: Reingruber, Richard <richard.reingruber at sap.com>; Lindenmaier, Goetz <goetz.lindenmaier at sap.com>; 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: Re: RFR(S): 8233014: Enable ShowCodeDetailsInExceptionMessages per default.
> 
> On 27/04/2020 6:20 pm, Reingruber, Richard wrote:
>> Hi Goetz,
>>
>> Just a minor detail: the license check fails for the file SuppressMessagesTest.java
>>
>> Try
>>
>>     sh make/scripts/lic_check.sh -gpl $(hg log -v --rev tip |sed -n '/files:[[:space:]]*/s///p')
> 
> Why did it fail?
> 
>> With
>>
>> --- a/test/hotspot/jtreg/runtime/exceptionMsgs/NullPointerException/SuppressMessagesTest.java
>> +++ b/test/hotspot/jtreg/runtime/exceptionMsgs/NullPointerException/SuppressMessagesTest.java
>> @@ -1,6 +1,6 @@
>>    /*
>> + * Copyright (c) 2019, 2020, SAP SE. All rights reserved.
>>     * Copyright (c) 2019, 2020, Oracle and/or its affiliates. All rights reserved.
>> - * Copyright (c) 2019, 2020 SAP SE. All rights reserved.
>>     * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>>     *
>>     * This code is free software; you can redistribute it and/or modify it
>>
>> it succeeds. Note that I added a comma after 2020 also.
> 
> That doesn't seem to be the SAP format. (Though it is not 100% consistent.)
> 
> Cheers,
> David
> 
>> Cheers, Richard.
>>
>> -----Original Message-----
>> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of Lindenmaier, Goetz
>> Sent: Freitag, 24. April 2020 16:36
>> To: 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
>> Subject: RFR(S): 8233014: Enable ShowCodeDetailsInExceptionMessages per default.
>>
>> Hi,
>>
>> I would like to enable JEP 358 Helpful NullPointerExceptions by default:
>>
>> http://cr.openjdk.java.net/~goetz/wr19/8233014-enable_NPE_per_default/02/
>>
>> Please review.
>>
>> The change passed our testing and jdk-submit.
>>
>> Best regards,
>>    Goetz.
>>

From robbin.ehn at oracle.com  Mon Apr 27 11:44:05 2020
From: robbin.ehn at oracle.com (Robbin Ehn)
Date: Mon, 27 Apr 2020 13:44:05 +0200
Subject: UL question: combining output flags with or at code site?
In-Reply-To: <CAA-vtUxu9rsQW4e6_up=uHQUzp+v4BHYBp_WoW3NZbHyqeOhfw@mail.gmail.com>
References: <CAA-vtUzt9DmKp5fKyzn7J1kUpXs4sY5vvVZiZoKg88XSyhVjGA@mail.gmail.com>
 <1107d469-2e87-0d69-419d-e4147d1e84a3@oracle.com>
 <CAA-vtUxu9rsQW4e6_up=uHQUzp+v4BHYBp_WoW3NZbHyqeOhfw@mail.gmail.com>
Message-ID: <727a9b17-6c1d-ab79-d845-c72f8a8b4e47@oracle.com>

On 2020-04-23 12:21, Thomas St?fe wrote:
> Thank you, Stefan. For the moment a helper function may be the simplest
> solution.
> 
> Maybe long-term we could have a LogTarget variant OR-combining the flags
> instead of AND-ing them...
> 
> Sometimes I think I would have prepared ORing the flags by default, I would
> have found that less surprising and easier to use (e.g. if I find a
> particular "gc" log output useful, I could have added a "metaspace" tag and
> thus selected it for -Xlog=metaspace too. As it is now, this would just
> effectively mute the output by requiring both flags to be set).
> 
> Alas, it is now what it is, and its a matter of taste anyway...

I agree that we should change something, as it is now I always add
asterisk. E.g. -Xlog:thread gives nothing with -version, but you most
likely wanted the os+thread output.

/Robbin

> 
> Thanks, Thomas
> 
> 
> 
> On Thu, Apr 23, 2020 at 9:33 AM Stefan Karlsson <stefan.karlsson at oracle.com>
> wrote:
> 
>> Hi Thomas,
>>
>> On 2020-04-23 06:45, Thomas St?fe wrote:
>>> Hi,
>>>
>>> with UL, is there a way at the logging site to specify that a logging has
>>> to happen if one of the flags is set (so, *or* combined)?
>>>
>>> E.g. if I wanted to print something when either "cds" or "metaspace" is
>>> active? Without forcing the user to specify an asterix at the -Xlog
>>> command, because that would activate too many other output sites.
>>>
>>> If I use it like this:
>>>
>>> LogTarget(Info, cds, metaspace) lt;
>>>
>>> both flags must be active to print the output, right?
>>
>> You can use the type-erasure feature of the LogTargetHandle.
>>
>>     LogTarget(Info, cds) log_cds;
>>     LogTarget(Info, metaspace) log_metaspace;
>>
>>     LogTargetHandle log_handle_cds(log_cds);
>>     LogTargetHandle log_handle_metaspace(log_metaspace);
>>
>>     LogTargetHandle log(log_cds.is_enabled() ? log_handle_cds :
>> log_handle_metaspace);
>>
>>     log.print("Either cds or metaspace");
>>
>> With the above you get:
>>
>> $ java -Xlog:cds -XX:+UseZGC -version
>> [0.015s][info][cds] Either cds or metaspace
>>
>> $ java -Xlog:metaspace -XX:+UseZGC -version
>> [0.015s][info][metaspace] Either cds or metaspace
>>
>> $ java -Xlog:metaspace,cds -XX:+UseZGC -version
>> [0.015s][info][cds] Either cds or metaspace
>>
>> And nothing if you don't specify either:
>> $ java -XX:+UseZGC -version
>>
>> You could also use a third LogTarget/LogTargetHandle to specify the
>> combination of cds+metaspace and use that as a combination above.
>>
>> Maybe there's a way to shrink this down. It would have been nice to be
>> able to write:
>>
>> LogTargetHandle log(log_cds.is_enabled() ? log_cds.handle() :
>> log_metaspace.handle());
>>
>> Or you could create a helper function to hide it:
>> LogTargetHandle log = cds_or_metaspace_logger();
>>
>> HTH,
>> StefanK
>>
>>>
>>> Thanks, Thomas
>>>
>>

From erik.osterlund at oracle.com  Mon Apr 27 15:28:30 2020
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Mon, 27 Apr 2020 17:28:30 +0200
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64
In-Reply-To: <866D9669-4CFD-4ABC-BCC2-3F2B51BDDDA9@oracle.com>
References: <56a0cf85-9698-64b2-95b4-e5b123c3e9e2@oracle.com>
 <B9EAEA33-057A-44C2-A7CF-DFB407776759@oracle.com>
 <42f02810-7a6e-1c7d-3fe3-b892f007ddc4@oracle.com>
 <7cda1c27-e745-e3c5-8c0e-a05a588baa6e@oracle.com>
 <866D9669-4CFD-4ABC-BCC2-3F2B51BDDDA9@oracle.com>
Message-ID: <ddbd4b25-beb1-0f4a-20b9-086eef381d1a@oracle.com>

Hi Frederic,

I hear your concern about hardcoded offsets making the JVM more fragile.

First of all, let me assure you that you would not be able to run hello 
world if the offsets ever were to be
wrong. They are verified during bootstrapping in 
JavaClasses::check_offsets(), so you have to be quite skilled
to manage to break anything without immediately noticing it. You 
literally can't have run a single test or java -version
without immediately crashing. This makes this significantly less 
fragile. So I don't think I believe that this is fragile.

Having said that, I do agree it could be annoying and a pain to maintain 
and update the layout code if such changes
are occasionally incompatible with the random selection of hardcoded 
offsets, forcing new awkward updates of said offsets,
and that hardcoding things in general is undesirable, compared to a more 
solid solution.
I had a look at an approach for removing hardcoded offsets completely. 
It looks something like this:

http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00..not_01/

The change is a bit involved and complicated though. Because of that, I 
would prefer to push it as a follow-up RFE
focused on removing hardcoded offsets, if you are okay with that. This 
change is already quite tricky, and I think
these things ought to be separate.

The fix involves splitting allocation and code generation of the 
interpreter (due to dependencies with verification code in i2i adapters
generated during method loading, requiring the interpreter to be 
allocated early), computing offsets of jlr.Reference early during
resolution of well known classes (as is done for java_lang_String and 
java_lang_Class), because it needs offsets to remove the
discovered and referent fields from oop maps before its subclasses are 
initialized, and moving initialization (and generation)
of the interpreter until after offsets have been computed (no longer 
having the dependency to method loading and linking).
This way, all offsets have always been computed (or loaded by CDS) 
before they are needed, without any need for hardcoding.

What do you think?

Thanks,
/Erik

On 2020-04-24 16:45, Frederic Parain wrote:
>
>> On Apr 24, 2020, at 10:36, coleen.phillimore at oracle.com wrote:
>>
>>
>>
>> On 4/24/20 10:16 AM, Erik ?sterlund wrote:
>>> Hi Frederic,
>>>
>>> Thanks for reviewing this.
>>>
>>> On 2020-04-24 15:28, Frederic Parain wrote:
>>>> Hi Erik,
>>>>
>>>> Why did you removed the code handling classes with hard coded layouts?
>>>> This code was there to provide the freedom to play with layout algorithms
>>>> without having to deal with these special cases. Without this code, any
>>>> modification is now constrained by these special classes.
>>> Good question. Since I started using the klass gap for fields, even when compressed oops is off, dealing with the
>>> hardcoded offsets and layouts really became quite awkward, to the extent that I started wondering why do we even have
>>> special layouts for these things anyway. I don't like special things. Treating it like any other object seemed far
>>> easier, than matching the special special hardcoded layouts with the hardcoded offsets.
>>>
>>> I wouldn't say that the whole layout algorithm is constrained by the fact that there are hardcoded offsets.
>>> I would say that it's just a matter of whoever changes the layout algorithms such that the hardcoded offsets
>>> are no longer the same, is just gonna have to update the offset to whatever the correct offsets are with the
>>> new algorithm. And if you look at the classes where we actually do hardcode the offsets, they are rather vanilla
>>> with relatively few ways you could change the layout other than changing it around for the sake of it. Especially
>>> primitive box objects. My fantasy is quite limited how we expect a new layout algorithm is gonna do anything else
>>> there really.
>>>
>>> Philosophically, I really think it is up to the person hardcoding an offset to know what the offset is going to
>>> be, ratherthan having completely different special (and worse) layouts produced for certain objects, just to
>>> satisfy the expectations of the hardcoded offsets.Especially if the result is pretty bad memory waste in for
>>> example Integer box objects, that can be quite plentyful in an application.
>> We used to have a lot more special hardcoded layouts but most have been removed. The reason these few classes are hardcoded is because the interpreter generates code using these offsets before JavaClasses::compute_offsets is run.  We tried to get rid of all of them and maybe there's a way to indirectly refer to them in the generated code so we can do so in the future.
> The proposed changes make the whole system brittle. Restoring the special handling of these classes during
> layout computation or making the generated code able to deal with dynamically computed offsets would be a
> more robust solution.
>
>
>>> The code to figure out the right offsets if a very small amount of code, compared to the rather large amount of
>>> code to create special layouts and syncing the special hardcoded layout code with the hardcoded offsets. That
>>> is why my patch has a net negative amount of code.
>>>
>>> Philosophically I also don't like having "special" layouts because we don't trust our ability to hardcode some
>>> offset to the right value. Almost all "special" things are bad and create unnecessary complications, and hence
>>> should have very good motivation for being allowed to be special. In this case, I really could not see any reason
>>> why these layouts would be special, unless it was somehow super hard to figure out what the hardcoded offsets would be.
>>> But for the classes we currently need hardcoded offsets, the calculations are trivial. So I don't think there is
>>> enough motivation to have special (bad) layouts for these objects for that reason. It only makes things harder than
>>> it needs to be, and makes the layout worse.
>>>
>>>> And I?d prefer to keep the dispatch in FieldLayoutBuilder::build_layout()
>>>> because we have more cases to handle there for the Valhalla project (the
>>>> layout algorithm for inline types is different than the one in
>>>> compute_regular_layout()).
>>> Okay. There was no need for the dispatching any more since there was only the normal layout and nothing
>>> else. But I can put that code back into the compute_regular_layout() function if that makes it easier for you
>>> in Valhalla, and have FieldLayoutBuilder::build_layout() just call that function. Will update in the next webrev.
> Thank you,
>
> Fred
>
>>>
>>>> Regards,
>>>>
>>>> Fred
>>>>
>>>>
>>>>> On Apr 24, 2020, at 04:17, Erik ?sterlund <erik.osterlund at oracle.com> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> Today, the use of compressed class pointers assumes the use of compressed oops.
>>>>> This patch loosens up the relationship between compressed oops and compressed
>>>>> class pointers, so that compressed class pointers can be used without compressed
>>>>> oops. This benefits for example ZGC that wants compressed class pointers, but
>>>>> not compressed oops, effectively giving ZGC 4 bytes smaller objects.
>>>>>
>>>>> Much of the complexity of the change is that r12 used to be used by compressed
>>>>> class pointers as some kind of semi-fast temp register that one would restore
>>>>> to the compressed oops heap base (or zero). That made it effectively a slightly
>>>>> optimized spilling mechanism used to find a temp register. I replaced that
>>>>> mechanism with a plain old normal temp register that you pass in as a parameter.
>>>>> The bad news is that I had to find available temp registers in a lot of places.
>>>>> The good news is that almost always were there temp registers available for free,
>>>>> and hence the new temp register is even faster than the old optimized spilling
>>>>> mechanism. Because we almost never need any spilling at all in the contexts where
>>>>> this is used.
>>>>>
>>>>> Since I want the 4 new bytes to actually make objects smaller, I poked the new
>>>>> layout code a bit so that the klass gap is made available for fields. That used
>>>>> to be made available only with compressed oops enabled, due to restrictions in
>>>>> the layout code. Our new layout code does not have such restrictions, and so
>>>>> I will make the 4 bytes available for fields when the new layout code is used
>>>>> and compressed class pointers are used.
>>>>>
>>>>> Now I'm only fixing this for HotSpot x86_64. Ideally the use of compressed oops
>>>>> and compressed class pointers should not be entangled in other platforms and
>>>>> Graal. But that is beyond the scope of this change, and I will let them behave
>>>>> the way that they used to, to be potentially fixed later.
>>>>>
>>>>> Bug:
>>>>> https://bugs.openjdk.java.net/browse/JDK-8241825
>>>>>
>>>>> Webrev:
>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/
>>>>>
>>>>> Testing:
>>>>> hs-tier1-7
>>>>>
>>>>> Thanks,
>>>>> /Erik


From frederic.parain at oracle.com  Mon Apr 27 15:51:19 2020
From: frederic.parain at oracle.com (Frederic Parain)
Date: Mon, 27 Apr 2020 11:51:19 -0400
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64
In-Reply-To: <ddbd4b25-beb1-0f4a-20b9-086eef381d1a@oracle.com>
References: <56a0cf85-9698-64b2-95b4-e5b123c3e9e2@oracle.com>
 <B9EAEA33-057A-44C2-A7CF-DFB407776759@oracle.com>
 <42f02810-7a6e-1c7d-3fe3-b892f007ddc4@oracle.com>
 <7cda1c27-e745-e3c5-8c0e-a05a588baa6e@oracle.com>
 <866D9669-4CFD-4ABC-BCC2-3F2B51BDDDA9@oracle.com>
 <ddbd4b25-beb1-0f4a-20b9-086eef381d1a@oracle.com>
Message-ID: <36D80B5C-9B88-4B6A-A0A1-AD87A2FB5EA9@oracle.com>

Erik,

Thank you for taking care of this issue.

> On Apr 27, 2020, at 11:28, Erik ?sterlund <erik.osterlund at oracle.com> wrote:
> 
> Hi Frederic,
> 
> I hear your concern about hardcoded offsets making the JVM more fragile.
> 
> First of all, let me assure you that you would not be able to run hello world if the offsets ever were to be
> wrong. They are verified during bootstrapping in JavaClasses::check_offsets(), so you have to be quite skilled
> to manage to break anything without immediately noticing it. You literally can't have run a single test or java -version
> without immediately crashing. This makes this significantly less fragile. So I don't think I believe that this is fragile.

I agree. I was planning to remove this code in 16, as part of the old layout code removal,
but it can still be useful for some time.

> 
> Having said that, I do agree it could be annoying and a pain to maintain and update the layout code if such changes
> are occasionally incompatible with the random selection of hardcoded offsets, forcing new awkward updates of said offsets,
> and that hardcoding things in general is undesirable, compared to a more solid solution.

I worked around the problem by isolating these classes so they don?t impact the regular
cases of field layout, but it was just a work around. Completely removing hardcoded offsets
is a cleaner and even more robust solution.

> I had a look at an approach for removing hardcoded offsets completely. It looks something like this:
> 
> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00..not_01/
> 
> The change is a bit involved and complicated though. Because of that, I would prefer to push it as a follow-up RFE
> focused on removing hardcoded offsets, if you are okay with that. This change is already quite tricky, and I think
> these things ought to be separate.

As your changes linked above touch the VM init sequence (always a tricky area),
I think dedicating a specific RFE for them is a good idea.

> 
> The fix involves splitting allocation and code generation of the interpreter (due to dependencies with verification code in i2i adapters
> generated during method loading, requiring the interpreter to be allocated early), computing offsets of jlr.Reference early during
> resolution of well known classes (as is done for java_lang_String and java_lang_Class), because it needs offsets to remove the
> discovered and referent fields from oop maps before its subclasses are initialized, and moving initialization (and generation)
> of the interpreter until after offsets have been computed (no longer having the dependency to method loading and linking).
> This way, all offsets have always been computed (or loaded by CDS) before they are needed, without any need for hardcoding.
> 
> What do you think?

Sounds like a good plan. I?ve read your changes, and they look fine to me.
However, I would be more comfortable with extended testing to cover all the
configurations (+/-CDS, templateInterpreter/cppInterpreter, C2/Graal) to be
sure we didn?t miss a dependency somewhere.

Thank you for addressing this technical debt and solving it cleanly.

Regards,

Fred


> 
> On 2020-04-24 16:45, Frederic Parain wrote:
>> 
>>> On Apr 24, 2020, at 10:36, coleen.phillimore at oracle.com wrote:
>>> 
>>> 
>>> 
>>> On 4/24/20 10:16 AM, Erik ?sterlund wrote:
>>>> Hi Frederic,
>>>> 
>>>> Thanks for reviewing this.
>>>> 
>>>> On 2020-04-24 15:28, Frederic Parain wrote:
>>>>> Hi Erik,
>>>>> 
>>>>> Why did you removed the code handling classes with hard coded layouts?
>>>>> This code was there to provide the freedom to play with layout algorithms
>>>>> without having to deal with these special cases. Without this code, any
>>>>> modification is now constrained by these special classes.
>>>> Good question. Since I started using the klass gap for fields, even when compressed oops is off, dealing with the
>>>> hardcoded offsets and layouts really became quite awkward, to the extent that I started wondering why do we even have
>>>> special layouts for these things anyway. I don't like special things. Treating it like any other object seemed far
>>>> easier, than matching the special special hardcoded layouts with the hardcoded offsets.
>>>> 
>>>> I wouldn't say that the whole layout algorithm is constrained by the fact that there are hardcoded offsets.
>>>> I would say that it's just a matter of whoever changes the layout algorithms such that the hardcoded offsets
>>>> are no longer the same, is just gonna have to update the offset to whatever the correct offsets are with the
>>>> new algorithm. And if you look at the classes where we actually do hardcode the offsets, they are rather vanilla
>>>> with relatively few ways you could change the layout other than changing it around for the sake of it. Especially
>>>> primitive box objects. My fantasy is quite limited how we expect a new layout algorithm is gonna do anything else
>>>> there really.
>>>> 
>>>> Philosophically, I really think it is up to the person hardcoding an offset to know what the offset is going to
>>>> be, ratherthan having completely different special (and worse) layouts produced for certain objects, just to
>>>> satisfy the expectations of the hardcoded offsets.Especially if the result is pretty bad memory waste in for
>>>> example Integer box objects, that can be quite plentyful in an application.
>>> We used to have a lot more special hardcoded layouts but most have been removed. The reason these few classes are hardcoded is because the interpreter generates code using these offsets before JavaClasses::compute_offsets is run.  We tried to get rid of all of them and maybe there's a way to indirectly refer to them in the generated code so we can do so in the future.
>> The proposed changes make the whole system brittle. Restoring the special handling of these classes during
>> layout computation or making the generated code able to deal with dynamically computed offsets would be a
>> more robust solution.
>> 
>> 
>>>> The code to figure out the right offsets if a very small amount of code, compared to the rather large amount of
>>>> code to create special layouts and syncing the special hardcoded layout code with the hardcoded offsets. That
>>>> is why my patch has a net negative amount of code.
>>>> 
>>>> Philosophically I also don't like having "special" layouts because we don't trust our ability to hardcode some
>>>> offset to the right value. Almost all "special" things are bad and create unnecessary complications, and hence
>>>> should have very good motivation for being allowed to be special. In this case, I really could not see any reason
>>>> why these layouts would be special, unless it was somehow super hard to figure out what the hardcoded offsets would be.
>>>> But for the classes we currently need hardcoded offsets, the calculations are trivial. So I don't think there is
>>>> enough motivation to have special (bad) layouts for these objects for that reason. It only makes things harder than
>>>> it needs to be, and makes the layout worse.
>>>> 
>>>>> And I?d prefer to keep the dispatch in FieldLayoutBuilder::build_layout()
>>>>> because we have more cases to handle there for the Valhalla project (the
>>>>> layout algorithm for inline types is different than the one in
>>>>> compute_regular_layout()).
>>>> Okay. There was no need for the dispatching any more since there was only the normal layout and nothing
>>>> else. But I can put that code back into the compute_regular_layout() function if that makes it easier for you
>>>> in Valhalla, and have FieldLayoutBuilder::build_layout() just call that function. Will update in the next webrev.
>> Thank you,
>> 
>> Fred
>> 
>>>> 
>>>>> Regards,
>>>>> 
>>>>> Fred
>>>>> 
>>>>> 
>>>>>> On Apr 24, 2020, at 04:17, Erik ?sterlund <erik.osterlund at oracle.com> wrote:
>>>>>> 
>>>>>> Hi,
>>>>>> 
>>>>>> Today, the use of compressed class pointers assumes the use of compressed oops.
>>>>>> This patch loosens up the relationship between compressed oops and compressed
>>>>>> class pointers, so that compressed class pointers can be used without compressed
>>>>>> oops. This benefits for example ZGC that wants compressed class pointers, but
>>>>>> not compressed oops, effectively giving ZGC 4 bytes smaller objects.
>>>>>> 
>>>>>> Much of the complexity of the change is that r12 used to be used by compressed
>>>>>> class pointers as some kind of semi-fast temp register that one would restore
>>>>>> to the compressed oops heap base (or zero). That made it effectively a slightly
>>>>>> optimized spilling mechanism used to find a temp register. I replaced that
>>>>>> mechanism with a plain old normal temp register that you pass in as a parameter.
>>>>>> The bad news is that I had to find available temp registers in a lot of places.
>>>>>> The good news is that almost always were there temp registers available for free,
>>>>>> and hence the new temp register is even faster than the old optimized spilling
>>>>>> mechanism. Because we almost never need any spilling at all in the contexts where
>>>>>> this is used.
>>>>>> 
>>>>>> Since I want the 4 new bytes to actually make objects smaller, I poked the new
>>>>>> layout code a bit so that the klass gap is made available for fields. That used
>>>>>> to be made available only with compressed oops enabled, due to restrictions in
>>>>>> the layout code. Our new layout code does not have such restrictions, and so
>>>>>> I will make the 4 bytes available for fields when the new layout code is used
>>>>>> and compressed class pointers are used.
>>>>>> 
>>>>>> Now I'm only fixing this for HotSpot x86_64. Ideally the use of compressed oops
>>>>>> and compressed class pointers should not be entangled in other platforms and
>>>>>> Graal. But that is beyond the scope of this change, and I will let them behave
>>>>>> the way that they used to, to be potentially fixed later.
>>>>>> 
>>>>>> Bug:
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8241825
>>>>>> 
>>>>>> Webrev:
>>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/
>>>>>> 
>>>>>> Testing:
>>>>>> hs-tier1-7
>>>>>> 
>>>>>> Thanks,
>>>>>> /Erik
> 


From martin.doerr at sap.com  Mon Apr 27 16:35:04 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Mon, 27 Apr 2020 16:35:04 +0000
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64
In-Reply-To: <36D80B5C-9B88-4B6A-A0A1-AD87A2FB5EA9@oracle.com>
References: <56a0cf85-9698-64b2-95b4-e5b123c3e9e2@oracle.com>
 <B9EAEA33-057A-44C2-A7CF-DFB407776759@oracle.com>
 <42f02810-7a6e-1c7d-3fe3-b892f007ddc4@oracle.com>
 <7cda1c27-e745-e3c5-8c0e-a05a588baa6e@oracle.com>
 <866D9669-4CFD-4ABC-BCC2-3F2B51BDDDA9@oracle.com>
 <ddbd4b25-beb1-0f4a-20b9-086eef381d1a@oracle.com>
 <36D80B5C-9B88-4B6A-A0A1-AD87A2FB5EA9@oracle.com>
Message-ID: <AM4PR02MB3057D8FB70B904172A02E7BE9AAF0@AM4PR02MB3057.eurprd02.prod.outlook.com>

Hi Erik,

I think this change makes sense (not a review).

But I guess this was a x86 specific limitation. I can't see why we're not setting COMPRESSED_CLASS_POINTERS_DEPENDS_ON_COMPRESSED_OOPS to false on other platforms.
I think it should work at least on PPC64 and s390. But let me double-check.
(Haven't checked others.)

Best regards,
Martin


> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
> Frederic Parain
> Sent: Montag, 27. April 2020 17:51
> To: Erik ?sterlund <erik.osterlund at oracle.com>
> Cc: hotspot-dev at openjdk.java.net
> Subject: Re: RFR: 8241825: Make compressed oops and compressed class
> pointers independent on x86_64
> 
> Erik,
> 
> Thank you for taking care of this issue.
> 
> > On Apr 27, 2020, at 11:28, Erik ?sterlund <erik.osterlund at oracle.com>
> wrote:
> >
> > Hi Frederic,
> >
> > I hear your concern about hardcoded offsets making the JVM more fragile.
> >
> > First of all, let me assure you that you would not be able to run hello world
> if the offsets ever were to be
> > wrong. They are verified during bootstrapping in
> JavaClasses::check_offsets(), so you have to be quite skilled
> > to manage to break anything without immediately noticing it. You literally
> can't have run a single test or java -version
> > without immediately crashing. This makes this significantly less fragile. So I
> don't think I believe that this is fragile.
> 
> I agree. I was planning to remove this code in 16, as part of the old layout
> code removal,
> but it can still be useful for some time.
> 
> >
> > Having said that, I do agree it could be annoying and a pain to maintain and
> update the layout code if such changes
> > are occasionally incompatible with the random selection of hardcoded
> offsets, forcing new awkward updates of said offsets,
> > and that hardcoding things in general is undesirable, compared to a more
> solid solution.
> 
> I worked around the problem by isolating these classes so they don?t impact
> the regular
> cases of field layout, but it was just a work around. Completely removing
> hardcoded offsets
> is a cleaner and even more robust solution.
> 
> > I had a look at an approach for removing hardcoded offsets completely. It
> looks something like this:
> >
> > http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00..not_01/
> >
> > The change is a bit involved and complicated though. Because of that, I
> would prefer to push it as a follow-up RFE
> > focused on removing hardcoded offsets, if you are okay with that. This
> change is already quite tricky, and I think
> > these things ought to be separate.
> 
> As your changes linked above touch the VM init sequence (always a tricky
> area),
> I think dedicating a specific RFE for them is a good idea.
> 
> >
> > The fix involves splitting allocation and code generation of the interpreter
> (due to dependencies with verification code in i2i adapters
> > generated during method loading, requiring the interpreter to be allocated
> early), computing offsets of jlr.Reference early during
> > resolution of well known classes (as is done for java_lang_String and
> java_lang_Class), because it needs offsets to remove the
> > discovered and referent fields from oop maps before its subclasses are
> initialized, and moving initialization (and generation)
> > of the interpreter until after offsets have been computed (no longer
> having the dependency to method loading and linking).
> > This way, all offsets have always been computed (or loaded by CDS) before
> they are needed, without any need for hardcoding.
> >
> > What do you think?
> 
> Sounds like a good plan. I?ve read your changes, and they look fine to me.
> However, I would be more comfortable with extended testing to cover all
> the
> configurations (+/-CDS, templateInterpreter/cppInterpreter, C2/Graal) to be
> sure we didn?t miss a dependency somewhere.
> 
> Thank you for addressing this technical debt and solving it cleanly.
> 
> Regards,
> 
> Fred
> 
> 
> >
> > On 2020-04-24 16:45, Frederic Parain wrote:
> >>
> >>> On Apr 24, 2020, at 10:36, coleen.phillimore at oracle.com wrote:
> >>>
> >>>
> >>>
> >>> On 4/24/20 10:16 AM, Erik ?sterlund wrote:
> >>>> Hi Frederic,
> >>>>
> >>>> Thanks for reviewing this.
> >>>>
> >>>> On 2020-04-24 15:28, Frederic Parain wrote:
> >>>>> Hi Erik,
> >>>>>
> >>>>> Why did you removed the code handling classes with hard coded
> layouts?
> >>>>> This code was there to provide the freedom to play with layout
> algorithms
> >>>>> without having to deal with these special cases. Without this code,
> any
> >>>>> modification is now constrained by these special classes.
> >>>> Good question. Since I started using the klass gap for fields, even when
> compressed oops is off, dealing with the
> >>>> hardcoded offsets and layouts really became quite awkward, to the
> extent that I started wondering why do we even have
> >>>> special layouts for these things anyway. I don't like special things.
> Treating it like any other object seemed far
> >>>> easier, than matching the special special hardcoded layouts with the
> hardcoded offsets.
> >>>>
> >>>> I wouldn't say that the whole layout algorithm is constrained by the fact
> that there are hardcoded offsets.
> >>>> I would say that it's just a matter of whoever changes the layout
> algorithms such that the hardcoded offsets
> >>>> are no longer the same, is just gonna have to update the offset to
> whatever the correct offsets are with the
> >>>> new algorithm. And if you look at the classes where we actually do
> hardcode the offsets, they are rather vanilla
> >>>> with relatively few ways you could change the layout other than
> changing it around for the sake of it. Especially
> >>>> primitive box objects. My fantasy is quite limited how we expect a new
> layout algorithm is gonna do anything else
> >>>> there really.
> >>>>
> >>>> Philosophically, I really think it is up to the person hardcoding an offset
> to know what the offset is going to
> >>>> be, ratherthan having completely different special (and worse) layouts
> produced for certain objects, just to
> >>>> satisfy the expectations of the hardcoded offsets.Especially if the result
> is pretty bad memory waste in for
> >>>> example Integer box objects, that can be quite plentyful in an
> application.
> >>> We used to have a lot more special hardcoded layouts but most have
> been removed. The reason these few classes are hardcoded is because the
> interpreter generates code using these offsets before
> JavaClasses::compute_offsets is run.  We tried to get rid of all of them and
> maybe there's a way to indirectly refer to them in the generated code so we
> can do so in the future.
> >> The proposed changes make the whole system brittle. Restoring the
> special handling of these classes during
> >> layout computation or making the generated code able to deal with
> dynamically computed offsets would be a
> >> more robust solution.
> >>
> >>
> >>>> The code to figure out the right offsets if a very small amount of code,
> compared to the rather large amount of
> >>>> code to create special layouts and syncing the special hardcoded layout
> code with the hardcoded offsets. That
> >>>> is why my patch has a net negative amount of code.
> >>>>
> >>>> Philosophically I also don't like having "special" layouts because we
> don't trust our ability to hardcode some
> >>>> offset to the right value. Almost all "special" things are bad and create
> unnecessary complications, and hence
> >>>> should have very good motivation for being allowed to be special. In
> this case, I really could not see any reason
> >>>> why these layouts would be special, unless it was somehow super hard
> to figure out what the hardcoded offsets would be.
> >>>> But for the classes we currently need hardcoded offsets, the
> calculations are trivial. So I don't think there is
> >>>> enough motivation to have special (bad) layouts for these objects for
> that reason. It only makes things harder than
> >>>> it needs to be, and makes the layout worse.
> >>>>
> >>>>> And I?d prefer to keep the dispatch in
> FieldLayoutBuilder::build_layout()
> >>>>> because we have more cases to handle there for the Valhalla project
> (the
> >>>>> layout algorithm for inline types is different than the one in
> >>>>> compute_regular_layout()).
> >>>> Okay. There was no need for the dispatching any more since there was
> only the normal layout and nothing
> >>>> else. But I can put that code back into the compute_regular_layout()
> function if that makes it easier for you
> >>>> in Valhalla, and have FieldLayoutBuilder::build_layout() just call that
> function. Will update in the next webrev.
> >> Thank you,
> >>
> >> Fred
> >>
> >>>>
> >>>>> Regards,
> >>>>>
> >>>>> Fred
> >>>>>
> >>>>>
> >>>>>> On Apr 24, 2020, at 04:17, Erik ?sterlund
> <erik.osterlund at oracle.com> wrote:
> >>>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> Today, the use of compressed class pointers assumes the use of
> compressed oops.
> >>>>>> This patch loosens up the relationship between compressed oops
> and compressed
> >>>>>> class pointers, so that compressed class pointers can be used
> without compressed
> >>>>>> oops. This benefits for example ZGC that wants compressed class
> pointers, but
> >>>>>> not compressed oops, effectively giving ZGC 4 bytes smaller objects.
> >>>>>>
> >>>>>> Much of the complexity of the change is that r12 used to be used by
> compressed
> >>>>>> class pointers as some kind of semi-fast temp register that one
> would restore
> >>>>>> to the compressed oops heap base (or zero). That made it
> effectively a slightly
> >>>>>> optimized spilling mechanism used to find a temp register. I replaced
> that
> >>>>>> mechanism with a plain old normal temp register that you pass in as
> a parameter.
> >>>>>> The bad news is that I had to find available temp registers in a lot of
> places.
> >>>>>> The good news is that almost always were there temp registers
> available for free,
> >>>>>> and hence the new temp register is even faster than the old
> optimized spilling
> >>>>>> mechanism. Because we almost never need any spilling at all in the
> contexts where
> >>>>>> this is used.
> >>>>>>
> >>>>>> Since I want the 4 new bytes to actually make objects smaller, I
> poked the new
> >>>>>> layout code a bit so that the klass gap is made available for fields.
> That used
> >>>>>> to be made available only with compressed oops enabled, due to
> restrictions in
> >>>>>> the layout code. Our new layout code does not have such
> restrictions, and so
> >>>>>> I will make the 4 bytes available for fields when the new layout code
> is used
> >>>>>> and compressed class pointers are used.
> >>>>>>
> >>>>>> Now I'm only fixing this for HotSpot x86_64. Ideally the use of
> compressed oops
> >>>>>> and compressed class pointers should not be entangled in other
> platforms and
> >>>>>> Graal. But that is beyond the scope of this change, and I will let them
> behave
> >>>>>> the way that they used to, to be potentially fixed later.
> >>>>>>
> >>>>>> Bug:
> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8241825
> >>>>>>
> >>>>>> Webrev:
> >>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/
> >>>>>>
> >>>>>> Testing:
> >>>>>> hs-tier1-7
> >>>>>>
> >>>>>> Thanks,
> >>>>>> /Erik
> >


From erik.osterlund at oracle.com  Mon Apr 27 17:37:14 2020
From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=)
Date: Mon, 27 Apr 2020 19:37:14 +0200
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64
In-Reply-To: <AM4PR02MB3057D8FB70B904172A02E7BE9AAF0@AM4PR02MB3057.eurprd02.prod.outlook.com>
References: <AM4PR02MB3057D8FB70B904172A02E7BE9AAF0@AM4PR02MB3057.eurprd02.prod.outlook.com>
Message-ID: <28F6EBD2-30EC-4635-BB28-EC233D515019@oracle.com>

Hi Martin,

I would definitely encourage enabling this on other platforms, but given the large amount of platform specific code I ran into for x86_64, I didn?t feel like I could enable it on other platforms without being able to do thorough testing. I skimmed at other platforms and it ?seems to work?, but it needs verification.

If you could sanity check PPC and S390, that would be fantastic. I?m talking to Stuart about AArch64 as well, so hopefully we can check that box too soon.

Thanks,
/Erik

> On 27 Apr 2020, at 18:35, Doerr, Martin <martin.doerr at sap.com> wrote:
> 
> ?Hi Erik,
> 
> I think this change makes sense (not a review).
> 
> But I guess this was a x86 specific limitation. I can't see why we're not setting COMPRESSED_CLASS_POINTERS_DEPENDS_ON_COMPRESSED_OOPS to false on other platforms.
> I think it should work at least on PPC64 and s390. But let me double-check.
> (Haven't checked others.)
> 
> Best regards,
> Martin
> 
> 
>> -----Original Message-----
>> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
>> Frederic Parain
>> Sent: Montag, 27. April 2020 17:51
>> To: Erik ?sterlund <erik.osterlund at oracle.com>
>> Cc: hotspot-dev at openjdk.java.net
>> Subject: Re: RFR: 8241825: Make compressed oops and compressed class
>> pointers independent on x86_64
>> 
>> Erik,
>> 
>> Thank you for taking care of this issue.
>> 
>>> On Apr 27, 2020, at 11:28, Erik ?sterlund <erik.osterlund at oracle.com>
>> wrote:
>>> 
>>> Hi Frederic,
>>> 
>>> I hear your concern about hardcoded offsets making the JVM more fragile.
>>> 
>>> First of all, let me assure you that you would not be able to run hello world
>> if the offsets ever were to be
>>> wrong. They are verified during bootstrapping in
>> JavaClasses::check_offsets(), so you have to be quite skilled
>>> to manage to break anything without immediately noticing it. You literally
>> can't have run a single test or java -version
>>> without immediately crashing. This makes this significantly less fragile. So I
>> don't think I believe that this is fragile.
>> 
>> I agree. I was planning to remove this code in 16, as part of the old layout
>> code removal,
>> but it can still be useful for some time.
>> 
>>> 
>>> Having said that, I do agree it could be annoying and a pain to maintain and
>> update the layout code if such changes
>>> are occasionally incompatible with the random selection of hardcoded
>> offsets, forcing new awkward updates of said offsets,
>>> and that hardcoding things in general is undesirable, compared to a more
>> solid solution.
>> 
>> I worked around the problem by isolating these classes so they don?t impact
>> the regular
>> cases of field layout, but it was just a work around. Completely removing
>> hardcoded offsets
>> is a cleaner and even more robust solution.
>> 
>>> I had a look at an approach for removing hardcoded offsets completely. It
>> looks something like this:
>>> 
>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00..not_01/
>>> 
>>> The change is a bit involved and complicated though. Because of that, I
>> would prefer to push it as a follow-up RFE
>>> focused on removing hardcoded offsets, if you are okay with that. This
>> change is already quite tricky, and I think
>>> these things ought to be separate.
>> 
>> As your changes linked above touch the VM init sequence (always a tricky
>> area),
>> I think dedicating a specific RFE for them is a good idea.
>> 
>>> 
>>> The fix involves splitting allocation and code generation of the interpreter
>> (due to dependencies with verification code in i2i adapters
>>> generated during method loading, requiring the interpreter to be allocated
>> early), computing offsets of jlr.Reference early during
>>> resolution of well known classes (as is done for java_lang_String and
>> java_lang_Class), because it needs offsets to remove the
>>> discovered and referent fields from oop maps before its subclasses are
>> initialized, and moving initialization (and generation)
>>> of the interpreter until after offsets have been computed (no longer
>> having the dependency to method loading and linking).
>>> This way, all offsets have always been computed (or loaded by CDS) before
>> they are needed, without any need for hardcoding.
>>> 
>>> What do you think?
>> 
>> Sounds like a good plan. I?ve read your changes, and they look fine to me.
>> However, I would be more comfortable with extended testing to cover all
>> the
>> configurations (+/-CDS, templateInterpreter/cppInterpreter, C2/Graal) to be
>> sure we didn?t miss a dependency somewhere.
>> 
>> Thank you for addressing this technical debt and solving it cleanly.
>> 
>> Regards,
>> 
>> Fred
>> 
>> 
>>> 
>>> On 2020-04-24 16:45, Frederic Parain wrote:
>>>> 
>>>>> On Apr 24, 2020, at 10:36, coleen.phillimore at oracle.com wrote:
>>>>> 
>>>>> 
>>>>> 
>>>>> On 4/24/20 10:16 AM, Erik ?sterlund wrote:
>>>>>> Hi Frederic,
>>>>>> 
>>>>>> Thanks for reviewing this.
>>>>>> 
>>>>>> On 2020-04-24 15:28, Frederic Parain wrote:
>>>>>>> Hi Erik,
>>>>>>> 
>>>>>>> Why did you removed the code handling classes with hard coded
>> layouts?
>>>>>>> This code was there to provide the freedom to play with layout
>> algorithms
>>>>>>> without having to deal with these special cases. Without this code,
>> any
>>>>>>> modification is now constrained by these special classes.
>>>>>> Good question. Since I started using the klass gap for fields, even when
>> compressed oops is off, dealing with the
>>>>>> hardcoded offsets and layouts really became quite awkward, to the
>> extent that I started wondering why do we even have
>>>>>> special layouts for these things anyway. I don't like special things.
>> Treating it like any other object seemed far
>>>>>> easier, than matching the special special hardcoded layouts with the
>> hardcoded offsets.
>>>>>> 
>>>>>> I wouldn't say that the whole layout algorithm is constrained by the fact
>> that there are hardcoded offsets.
>>>>>> I would say that it's just a matter of whoever changes the layout
>> algorithms such that the hardcoded offsets
>>>>>> are no longer the same, is just gonna have to update the offset to
>> whatever the correct offsets are with the
>>>>>> new algorithm. And if you look at the classes where we actually do
>> hardcode the offsets, they are rather vanilla
>>>>>> with relatively few ways you could change the layout other than
>> changing it around for the sake of it. Especially
>>>>>> primitive box objects. My fantasy is quite limited how we expect a new
>> layout algorithm is gonna do anything else
>>>>>> there really.
>>>>>> 
>>>>>> Philosophically, I really think it is up to the person hardcoding an offset
>> to know what the offset is going to
>>>>>> be, ratherthan having completely different special (and worse) layouts
>> produced for certain objects, just to
>>>>>> satisfy the expectations of the hardcoded offsets.Especially if the result
>> is pretty bad memory waste in for
>>>>>> example Integer box objects, that can be quite plentyful in an
>> application.
>>>>> We used to have a lot more special hardcoded layouts but most have
>> been removed. The reason these few classes are hardcoded is because the
>> interpreter generates code using these offsets before
>> JavaClasses::compute_offsets is run.  We tried to get rid of all of them and
>> maybe there's a way to indirectly refer to them in the generated code so we
>> can do so in the future.
>>>> The proposed changes make the whole system brittle. Restoring the
>> special handling of these classes during
>>>> layout computation or making the generated code able to deal with
>> dynamically computed offsets would be a
>>>> more robust solution.
>>>> 
>>>> 
>>>>>> The code to figure out the right offsets if a very small amount of code,
>> compared to the rather large amount of
>>>>>> code to create special layouts and syncing the special hardcoded layout
>> code with the hardcoded offsets. That
>>>>>> is why my patch has a net negative amount of code.
>>>>>> 
>>>>>> Philosophically I also don't like having "special" layouts because we
>> don't trust our ability to hardcode some
>>>>>> offset to the right value. Almost all "special" things are bad and create
>> unnecessary complications, and hence
>>>>>> should have very good motivation for being allowed to be special. In
>> this case, I really could not see any reason
>>>>>> why these layouts would be special, unless it was somehow super hard
>> to figure out what the hardcoded offsets would be.
>>>>>> But for the classes we currently need hardcoded offsets, the
>> calculations are trivial. So I don't think there is
>>>>>> enough motivation to have special (bad) layouts for these objects for
>> that reason. It only makes things harder than
>>>>>> it needs to be, and makes the layout worse.
>>>>>> 
>>>>>>> And I?d prefer to keep the dispatch in
>> FieldLayoutBuilder::build_layout()
>>>>>>> because we have more cases to handle there for the Valhalla project
>> (the
>>>>>>> layout algorithm for inline types is different than the one in
>>>>>>> compute_regular_layout()).
>>>>>> Okay. There was no need for the dispatching any more since there was
>> only the normal layout and nothing
>>>>>> else. But I can put that code back into the compute_regular_layout()
>> function if that makes it easier for you
>>>>>> in Valhalla, and have FieldLayoutBuilder::build_layout() just call that
>> function. Will update in the next webrev.
>>>> Thank you,
>>>> 
>>>> Fred
>>>> 
>>>>>> 
>>>>>>> Regards,
>>>>>>> 
>>>>>>> Fred
>>>>>>> 
>>>>>>> 
>>>>>>>> On Apr 24, 2020, at 04:17, Erik ?sterlund
>> <erik.osterlund at oracle.com> wrote:
>>>>>>>> 
>>>>>>>> Hi,
>>>>>>>> 
>>>>>>>> Today, the use of compressed class pointers assumes the use of
>> compressed oops.
>>>>>>>> This patch loosens up the relationship between compressed oops
>> and compressed
>>>>>>>> class pointers, so that compressed class pointers can be used
>> without compressed
>>>>>>>> oops. This benefits for example ZGC that wants compressed class
>> pointers, but
>>>>>>>> not compressed oops, effectively giving ZGC 4 bytes smaller objects.
>>>>>>>> 
>>>>>>>> Much of the complexity of the change is that r12 used to be used by
>> compressed
>>>>>>>> class pointers as some kind of semi-fast temp register that one
>> would restore
>>>>>>>> to the compressed oops heap base (or zero). That made it
>> effectively a slightly
>>>>>>>> optimized spilling mechanism used to find a temp register. I replaced
>> that
>>>>>>>> mechanism with a plain old normal temp register that you pass in as
>> a parameter.
>>>>>>>> The bad news is that I had to find available temp registers in a lot of
>> places.
>>>>>>>> The good news is that almost always were there temp registers
>> available for free,
>>>>>>>> and hence the new temp register is even faster than the old
>> optimized spilling
>>>>>>>> mechanism. Because we almost never need any spilling at all in the
>> contexts where
>>>>>>>> this is used.
>>>>>>>> 
>>>>>>>> Since I want the 4 new bytes to actually make objects smaller, I
>> poked the new
>>>>>>>> layout code a bit so that the klass gap is made available for fields.
>> That used
>>>>>>>> to be made available only with compressed oops enabled, due to
>> restrictions in
>>>>>>>> the layout code. Our new layout code does not have such
>> restrictions, and so
>>>>>>>> I will make the 4 bytes available for fields when the new layout code
>> is used
>>>>>>>> and compressed class pointers are used.
>>>>>>>> 
>>>>>>>> Now I'm only fixing this for HotSpot x86_64. Ideally the use of
>> compressed oops
>>>>>>>> and compressed class pointers should not be entangled in other
>> platforms and
>>>>>>>> Graal. But that is beyond the scope of this change, and I will let them
>> behave
>>>>>>>> the way that they used to, to be potentially fixed later.
>>>>>>>> 
>>>>>>>> Bug:
>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8241825
>>>>>>>> 
>>>>>>>> Webrev:
>>>>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/
>>>>>>>> 
>>>>>>>> Testing:
>>>>>>>> hs-tier1-7
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> /Erik
>>> 
> 


From palacsint at palacsint.hu  Mon Apr 27 21:26:05 2020
From: palacsint at palacsint.hu (=?UTF-8?B?S2FyYWvDsyBNaWtsw7Nz?=)
Date: Mon, 27 Apr 2020 23:26:05 +0200
Subject: Thread-Local Handshakes slowdown on single-cpu machines
Message-ID: <5e65ba58-b599-e83c-b4ea-3d766ed3c457@palacsint.hu>

Hello,

We bumped into a possible Thread-Local Handshakes issue with multiple 
apps. It seems that enabled TLH slows down applications running on 
(although rare) single CPU boxes. I would be grateful if you could 
confirm that this is a known trade-off for these rare setups or a 
possible JVM bug. That would save us at least a few hours of debugging.

TL;DR: Both tested apps stop frequently around "HandshakeOneThread", 
"HandshakeAllThreads" and "Revoked bias of currently-unlocked object" 
running with OpenJDK12/OpenJDK13/OpenJDK14. OpenJDK 13 with 
-XX:-ThreadLocalHandshakes fixes the issue. Adding a second CPU to the 
virtual machine fixes the issue. Enabling hyper-threading fixes the issue.

More details are available at StackOverflow: 
https://stackoverflow.com/questions/61375565/slow-application-frequent-jvm-hangs-with-single-cpu-setups-and-java-12

All thoughts are welcome.

Best,
Miklos


From david.holmes at oracle.com  Mon Apr 27 22:53:51 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 28 Apr 2020 08:53:51 +1000
Subject: RFR: 8242244: Remove redundant ELF machine definitions
In-Reply-To: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>
References: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>
Message-ID: <22fdf007-a3d5-f14f-4d27-5746ad21a88e@oracle.com>

Hi Adrian,

Apologies as I just spotted this in my mail backlog.

This seems fine and trivial.

Thanks,
David

On 7/04/2020 8:08 am, John Paul Adrian Glaubitz wrote:
> Hi!
> 
> Please review this small change which removes redundant ELF machine
> definitions for EM_AARCH64 and EM_486 on Linux and Solaris [1].
> 
> Both EM_AARCH64 and EM_486 are defined by the kernel headers on both
> Solaris 11.3 or higher (verified on gcc211.fsffrance.org) and Linux 3.12
> or higher [2], so I think it's safe to remove those definitions from
> the JDK codebase unless the JDK is supposed to be built on systems
> older than that.
> 
> Thanks,
> Adrian
> 
>> [1] http://cr.openjdk.java.net/~glaubitz/8242244/webrev.01/
>> [2] https://elixir.bootlin.com/linux/v3.12/source/include/uapi/linux/elf-em.h
> 

From david.holmes at oracle.com  Mon Apr 27 23:02:26 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 28 Apr 2020 09:02:26 +1000
Subject: Thread-Local Handshakes slowdown on single-cpu machines
In-Reply-To: <5e65ba58-b599-e83c-b4ea-3d766ed3c457@palacsint.hu>
References: <5e65ba58-b599-e83c-b4ea-3d766ed3c457@palacsint.hu>
Message-ID: <66059982-49bf-1a15-39fb-22008b069573@oracle.com>

Hi Miklos,

On 28/04/2020 7:26 am, Karak? Mikl?s wrote:
> Hello,
> 
> We bumped into a possible Thread-Local Handshakes issue with multiple 
> apps. It seems that enabled TLH slows down applications running on 
> (although rare) single CPU boxes. I would be grateful if you could 
> confirm that this is a known trade-off for these rare setups or a 
> possible JVM bug. That would save us at least a few hours of debugging.
> 
> TL;DR: Both tested apps stop frequently around "HandshakeOneThread", 
> "HandshakeAllThreads" and "Revoked bias of currently-unlocked object" 
> running with OpenJDK12/OpenJDK13/OpenJDK14. OpenJDK 13 with 
> -XX:-ThreadLocalHandshakes fixes the issue. Adding a second CPU to the 
> virtual machine fixes the issue. Enabling hyper-threading fixes the issue.

Can you try with latest JDK 15 build, just to see if this may be 
something already addressed?

Thanks,
David

> More details are available at StackOverflow: 
> https://stackoverflow.com/questions/61375565/slow-application-frequent-jvm-hangs-with-single-cpu-setups-and-java-12 
> 
> 
> All thoughts are welcome.
> 
> Best,
> Miklos
> 

From glaubitz at physik.fu-berlin.de  Mon Apr 27 23:48:04 2020
From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz)
Date: Tue, 28 Apr 2020 01:48:04 +0200
Subject: RFR: 8242244: Remove redundant ELF machine definitions
In-Reply-To: <22fdf007-a3d5-f14f-4d27-5746ad21a88e@oracle.com>
References: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>
 <22fdf007-a3d5-f14f-4d27-5746ad21a88e@oracle.com>
Message-ID: <483038a4-7517-e190-cbbe-783da5f68ecc@physik.fu-berlin.de>

Hi David!

On 4/28/20 12:53 AM, David Holmes wrote:
> Apologies as I just spotted this in my mail backlog.

No worries ;).

> This seems fine and trivial.
Thanks. I post a rebased version tomorrow - necessary because I added
the definition for EM_RISCV to make sure we don't run into build issues
with slightly older kernels has EM_RISCV hasn't been defined in the
kernel for too long yet.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz at debian.org
`. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

From igor.ignatyev at oracle.com  Tue Apr 28 00:16:22 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Mon, 27 Apr 2020 17:16:22 -0700
Subject: RFR(T) : 8243930 : update copyright years
Message-ID: <FC11738A-3DD1-42CA-A7B7-B2C42C9DFFDA@oracle.com>

http://cr.openjdk.java.net/~iignatyev/8243930/webrev.00/
> 33 lines changed: 0 ins; 0 del; 33 mod; 

Hi all,

could you please review this trivial patch which just updates copyright years in several files I modified recently? I had copyright changed stashed so they won't pollute webrevs, and somehow forgot to apply them before pushing.

JBS: https://bugs.openjdk.java.net/browse/JDK-8243930
webrev: http://cr.openjdk.java.net/~iignatyev/8243930/webrev.00/

Thanks,
-- Igor


From david.holmes at oracle.com  Tue Apr 28 00:33:27 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 28 Apr 2020 10:33:27 +1000
Subject: RFR(T) : 8243930 : update copyright years
In-Reply-To: <FC11738A-3DD1-42CA-A7B7-B2C42C9DFFDA@oracle.com>
References: <FC11738A-3DD1-42CA-A7B7-B2C42C9DFFDA@oracle.com>
Message-ID: <26e72a09-7271-b538-748c-fbd37997bbca@oracle.com>

Hi Igor,

Looks good and trivial.

Thanks,
David

On 28/04/2020 10:16 am, Igor Ignatyev wrote:
> http://cr.openjdk.java.net/~iignatyev/8243930/webrev.00/
>> 33 lines changed: 0 ins; 0 del; 33 mod;
> 
> Hi all,
> 
> could you please review this trivial patch which just updates copyright years in several files I modified recently? I had copyright changed stashed so they won't pollute webrevs, and somehow forgot to apply them before pushing.
> 
> JBS: https://bugs.openjdk.java.net/browse/JDK-8243930
> webrev: http://cr.openjdk.java.net/~iignatyev/8243930/webrev.00/
> 
> Thanks,
> -- Igor
> 

From igor.ignatyev at oracle.com  Tue Apr 28 00:58:44 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Mon, 27 Apr 2020 17:58:44 -0700
Subject: RFR(T) : 8243930 : update copyright years
In-Reply-To: <26e72a09-7271-b538-748c-fbd37997bbca@oracle.com>
References: <FC11738A-3DD1-42CA-A7B7-B2C42C9DFFDA@oracle.com>
 <26e72a09-7271-b538-748c-fbd37997bbca@oracle.com>
Message-ID: <1BC124B8-D7CE-4F89-8058-368C61ACC4E8@oracle.com>

Thanks David, pushed.

-- Igor

> On Apr 27, 2020, at 5:33 PM, David Holmes <david.holmes at oracle.com> wrote:
> 
> Hi Igor,
> 
> Looks good and trivial.
> 
> Thanks,
> David
> 
> On 28/04/2020 10:16 am, Igor Ignatyev wrote:
>> http://cr.openjdk.java.net/~iignatyev/8243930/webrev.00/
>>> 33 lines changed: 0 ins; 0 del; 33 mod;
>> Hi all,
>> could you please review this trivial patch which just updates copyright years in several files I modified recently? I had copyright changed stashed so they won't pollute webrevs, and somehow forgot to apply them before pushing.
>> JBS: https://bugs.openjdk.java.net/browse/JDK-8243930
>> webrev: http://cr.openjdk.java.net/~iignatyev/8243930/webrev.00/
>> Thanks,
>> -- Igor


From igor.ignatyev at oracle.com  Tue Apr 28 02:32:59 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Mon, 27 Apr 2020 19:32:59 -0700
Subject: RFR(M/T) : 8243935 : s.h.WhiteBoxPermission must be
 ClassFileInstall-ed together w/ s.h.WhiteBox
Message-ID: <018B98AF-1DC5-4074-AE80-6601F9FDB867@oracle.com>

http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/
> 338 lines changed: 107 ins; 0 del; 231 mod

Hi all,

could you please review the trivial yet tedious patch which adds 'sun.hotspot.WhiteBox$WhiteBoxPermission' to all 'ClassFileInstaller sun.hotspot.WhiteBox' actions?
the patch also 
 - replaces '@run main ClassFileInstaller' w/ '@run driver ClassFileInstaller'
 - unifies the way ClassFileInstaller sun.hotspot.WhiteBox are aligned

JBS: https://bugs.openjdk.java.net/browse/JDK-8243935 
webrevs: 
 - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/ -- regular webrev
 - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.including_ws -- shows "whitespace only" changes 

Thanks,
-- Igor

From igor.ignatyev at oracle.com  Tue Apr 28 04:15:49 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Mon, 27 Apr 2020 21:15:49 -0700
Subject: RFR(S) : 8242314 : use reproducible random in vmTestbase shared
 code
In-Reply-To: <5FE778F4-FCCE-44E5-B856-48B374545D98@oracle.com>
References: <A59F9A91-D3D4-4D98-BD02-C44E1C3790E4@oracle.com>
 <4C16AD88-51F8-43F5-BAC1-16083E4A3C71@oracle.com>
 <4C10A003-6286-4272-AB7E-FC20659A6226@oracle.com>
 <FFE3D45D-5702-403C-995F-B09748A006EA@oracle.com>
 <5FE778F4-FCCE-44E5-B856-48B374545D98@oracle.com>
Message-ID: <CCA27FF4-9C5D-486F-A31E-180C58483310@oracle.com>



> On Apr 25, 2020, at 9:22 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
> 
> 
> 
>> On Apr 25, 2020, at 8:45 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
>> 
>>> On Apr 25, 2020, at 9:01 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
>>> 
>>> 
>>>> On Apr 25, 2020, at 4:57 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
>>>> 
>>>>> On Apr 22, 2020, at 2:03 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
>>>>> 
>>>>> http://cr.openjdk.java.net/~iignatyev/8242314/webrev.00/ 
>>>>>> 122 lines changed: 71 ins; 19 del; 32 mod;
>>>>> 
>>>>> 
>>>>> Hi all,
>>>>> 
>>>>> could you please review this small patch which updates shared vmTestbase  test code to use j.t.l.Utils.getRandomInstance() as a random number generator or as a source of seed values? this patch lays the ground work for updates of :vmTestbase_* test groups which are to be done by remaining subtasks of 8241623[1], such split make it possible to work on/review/test testgroup-specific changes in concurrently.  
>>>>> 
>>>>> unfortunately, such extra dependency on /test/lib classes broke the shaky balance and caused some tests to fail due to "split testlibrary" problem (see 8188828[2] for the definition), the solution to that is to remove all dependencies on /test/lib classes from ExecDriver class, so the patch also includes "inlining" of several jdk.test.lib.Utils and Platform methods/fields into ExecDriver.
>>>>> 
>>>>> webrev: http://cr.openjdk.java.net/~iignatyev/8242314/webrev.00/ 
>>>>> testing: all :vmTestbase_* test groups on {linux,windows,macosx}-x64 and solaris-sparcv9
>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8242314
>>>>> 
>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8241623
>>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8188828
>>>>> 
>>>>> Thanks,
>>>>> -- Igor
>>>> 
>>>> The test changes look fine.
>>>> 
>>>> But I don't feel like I understand the "split library" problem well
>>>> enough to review the ExecDriver changes.  I will say that I don't like
>>>> the solution of copying code, and hope there is a better way.  Is
>>>> there a bug for the problem?  It seems like it could crop up again,
>>>> perhaps repeatedly, in which case the code copying becomes increasingly
>>>> bad.  And it might be a good idea to leave some breadcrumbs associated
>>>> with this code copying workaround, so an eventual better fix could also
>>>> cleanup the duplication.
>>>> 
>>> 
>>> Hi Kim,
>>> 
>>> thanks for look at this!
>>> 
>>> "split library" problem occur when a test "have these properties: 
>>> - has more than one @library 
>>> - depends on class Foo from library A which depends on class Bar from another library B 
>>> - has an implicit/explicit build for class Baz from libraries which depends on class Bar"[1]
>>> in this case A=/vmTestbase, B=/test/lib, Foo=ExecDriver, Bar=jdk.test.lib.Utils, and Baz can be pretty much any of class from jdk.test.lib, e.g. jdk.test.lib.process.ProcessTools.
>> 
>> Yes, I read that already, from JDK-8188828.  But I don't understand
>> the problem; remember, I'm not really a Java programmer, I just fake
>> it sometimes :)  In particular, I don't have a good understanding of
>> Java packaging and modularization and such.  So I'd rather not be
>> counted as having reviewed that part of the change.
> 
> well, a usual Java programmer will have a little understanding of the problem either since it's kinda unique to jtreg, or rather to how we use it. let me try to elaborate more on that. @library tag tells jtreg where to look for source code during compilation, later when jtreg invokes javac to compile java source code to java bytecode it puts the result depending on the location of the class which provoked compilation, that's to say if you have '@build Bar' (or `@run main Bar`), jtreg will search for Bar in current test source directory and in all @libraries, if Bar is found in test source, jtreg will use test-specific directory as destination directory; if Bar is found in a @library, jtreg will use the directory specific for this library, and this directory is shared b/w different tests. to find which particular dependency of Bar to build, jtreg relies on javac's ability to implicitly compile dependencies, and the bytecode of all dependencies will be placed in the same directory regardless of the origin of dependencies source code. it all works great, unless you have a situation described above, as in such cases, you might end up w/ some part of @library be compiled into one directory, and some part in a directory of another @library -- a so-called "split library" problem, so the next test which uses 1st @library but doesn't use the 2nd will fail to find class files of some classes from 1st @library. I hope it makes this at least a little bit clearer. there is also a very good analyze of this problem from Ioi in CODETOOLS-7901986 -- https://bugs.openjdk.java.net/browse/CODETOOLS-7901986?focusedCommentId=14108320&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14108320 <https://bugs.openjdk.java.net/browse/CODETOOLS-7901986?focusedCommentId=14108320&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14108320> <https://bugs.openjdk.java.net/browse/CODETOOLS-7901986?focusedCommentId=14108320&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14108320 <https://bugs.openjdk.java.net/browse/CODETOOLS-7901986?focusedCommentId=14108320&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14108320>>
> 
>> 
>>> there is a better way to fix that, which we discussed have discussed in the aftermath of our encounter of this problem (when 8188828 was used as a stopgap solution), and this way is to redesign/simplify and potentially modularize test libraries so it will be easier/saner to build them. unfortunately, we are not there yet, yet we are making progress; this activity is tracked by 8211289[2], 8211290[3], and 8211358[4].
>>> 
>>> as of breadcrumbs, I think I left mentions of the copied methods in all the methods, e.g. L#144; and the only copied pieces of code which don't have such breadcrumbs are TEST_CLASS_PATH and TEST_JDK static fields. as they are quite straightforward, I didn't feel that it's need to refer back to the original code. but if you like, I can add comments there as well.
>> 
>> Regarding the breadcrumb comments, I didn't recognize them as such.
>> Had they been something more than just a fully qualified method name,
>> i.e. something like "Copied from <fqname>", it would have been clear.
>> Is there a cleanup CR for later?
> sure I'll update comments and file an RFE to clean this up. 

filed https://bugs.openjdk.java.net/browse/JDK-8243927 <https://bugs.openjdk.java.net/browse/JDK-8243927>; and revamped the comments:
 - full webrev: http://cr.openjdk.java.net/~iignatyev//8242314/webrev.01
 - diff b/w 00 and 01: http://cr.openjdk.java.net/~iignatyev//8242314/webrev.0-1 <http://cr.openjdk.java.net/~iignatyev//8242314/webrev.0-1>

-- Igor
>> 
>>> Thanks,
>>> -- Igor 
>>> 
>>> 
>>> [1] https://bugs.openjdk.java.net/browse/JDK-8188828?focusedCommentId=14121228&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14121228 <https://bugs.openjdk.java.net/browse/JDK-8188828?focusedCommentId=14121228&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14121228> <https://bugs.openjdk.java.net/browse/JDK-8188828?focusedCommentId=14121228&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14121228 <https://bugs.openjdk.java.net/browse/JDK-8188828?focusedCommentId=14121228&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14121228>>
>>> [2] https://bugs.openjdk.java.net/browse/JDK-8211289 <https://bugs.openjdk.java.net/browse/JDK-8211289><https://bugs.openjdk.java.net/browse/JDK-8211289 <https://bugs.openjdk.java.net/browse/JDK-8211289>>
>>> [3] https://bugs.openjdk.java.net/browse/JDK-8211290 <https://bugs.openjdk.java.net/browse/JDK-8211290><https://bugs.openjdk.java.net/browse/JDK-8211290 <https://bugs.openjdk.java.net/browse/JDK-8211290>>
>>> [4] https://bugs.openjdk.java.net/browse/JDK-8211358 <https://bugs.openjdk.java.net/browse/JDK-8211358><https://bugs.openjdk.java.net/browse/JDK-8211358 <https://bugs.openjdk.java.net/browse/JDK-8211358>>


From igor.ignatyev at oracle.com  Tue Apr 28 04:24:12 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Mon, 27 Apr 2020 21:24:12 -0700
Subject: RFR(T) : 8243943 :
 testlibrary_tests/process/TestNativeProcessBuilder.java can be run in driver
 mode
Message-ID: <0C10EC56-EBBA-473E-8195-CCC89784988E@oracle.com>

http://cr.openjdk.java.net/~iignatyev//8243943/webrev.00
> 1 line changed: 0 ins; 0 del; 1 mod;

Hi all,

could you please review this trivial one-liner which updates TestNativeProcessBuilder.java test to use driver mode?

webrev: http://cr.openjdk.java.net/~iignatyev//8243943/webrev.00
JBS: https://bugs.openjdk.java.net/browse/JDK-8243943

Thanks,
-- Igor


From david.holmes at oracle.com  Tue Apr 28 05:03:20 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 28 Apr 2020 15:03:20 +1000
Subject: RFR(M/T) : 8243935 : s.h.WhiteBoxPermission must be
 ClassFileInstall-ed together w/ s.h.WhiteBox
In-Reply-To: <018B98AF-1DC5-4074-AE80-6601F9FDB867@oracle.com>
References: <018B98AF-1DC5-4074-AE80-6601F9FDB867@oracle.com>
Message-ID: <e5a0d476-c516-b627-920f-9f25f0795961@oracle.com>

Hi Igor,

On 28/04/2020 12:32 pm, Igor Ignatyev wrote:
> http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/
>> 338 lines changed: 107 ins; 0 del; 231 mod
> 
> Hi all,
> 
> could you please review the trivial yet tedious patch which adds 'sun.hotspot.WhiteBox$WhiteBoxPermission' to all 'ClassFileInstaller sun.hotspot.WhiteBox' actions?
> the patch also
>   - replaces '@run main ClassFileInstaller' w/ '@run driver ClassFileInstaller'
>   - unifies the way ClassFileInstaller sun.hotspot.WhiteBox are aligned

That alignment change is unnecessary and makes it very hard to review 
this patch.

> JBS: https://bugs.openjdk.java.net/browse/JDK-8243935
> webrevs:
>   - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/ -- regular webrev
>   - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.including_ws -- shows "whitespace only" changes

I'd like to see the patch that leaves out the whitespace-only changes - 
is that possible?

Thanks,
David

> Thanks,
> -- Igor
> 

From igor.ignatyev at oracle.com  Tue Apr 28 05:37:50 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Mon, 27 Apr 2020 22:37:50 -0700
Subject: RFR(M/T) : 8243935 : s.h.WhiteBoxPermission must be
 ClassFileInstall-ed together w/ s.h.WhiteBox
In-Reply-To: <e5a0d476-c516-b627-920f-9f25f0795961@oracle.com>
References: <018B98AF-1DC5-4074-AE80-6601F9FDB867@oracle.com>
 <e5a0d476-c516-b627-920f-9f25f0795961@oracle.com>
Message-ID: <BBD7C18E-16AC-4BD1-9660-F103FC4F0D41@oracle.com>



> On Apr 27, 2020, at 10:03 PM, David Holmes <david.holmes at oracle.com> wrote:
> 
> Hi Igor,
> 
> On 28/04/2020 12:32 pm, Igor Ignatyev wrote:
>> http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/
>>> 338 lines changed: 107 ins; 0 del; 231 mod
>> Hi all,
>> could you please review the trivial yet tedious patch which adds 'sun.hotspot.WhiteBox$WhiteBoxPermission' to all 'ClassFileInstaller sun.hotspot.WhiteBox' actions?
>> the patch also
>>  - replaces '@run main ClassFileInstaller' w/ '@run driver ClassFileInstaller'
>>  - unifies the way ClassFileInstaller sun.hotspot.WhiteBox are aligned
> 
> That alignment change is unnecessary and makes it very hard to review this patch.
> 
>> JBS: https://bugs.openjdk.java.net/browse/JDK-8243935
>> webrevs:
>>  - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/ -- regular webrev
>>  - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.including_ws -- shows "whitespace only" changes
> 
> I'd like to see the patch that leaves out the whitespace-only changes - is that possible?
sure, I've used `hg diff -w` (--ignore-all-space    ignore white space when comparing lines) to remove all whitespace changes, here is webrev --  http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.w/

-- Igor
> 
> Thanks,
> David
> 
>> Thanks,
>> -- Igor


From igor.ignatyev at oracle.com  Tue Apr 28 05:41:51 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Mon, 27 Apr 2020 22:41:51 -0700
Subject: RFR(T) : 8243943 :
 testlibrary_tests/process/TestNativeProcessBuilder.java can be run in driver
 mode
In-Reply-To: <0C10EC56-EBBA-473E-8195-CCC89784988E@oracle.com>
References: <0C10EC56-EBBA-473E-8195-CCC89784988E@oracle.com>
Message-ID: <91AB2E4C-8120-43D1-88EA-21EA8AC0F046@oracle.com>

it appears it's illegal have 'driver/native' (where /native would just mean that a test depends on something from -nativepath, so don't run it if nativepath hasn't been specified), so I'm withdrawing this RFR. sorry for the noise.

-- Igor

> On Apr 27, 2020, at 9:24 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
> 
> http://cr.openjdk.java.net/~iignatyev//8243943/webrev.00
>> 1 line changed: 0 ins; 0 del; 1 mod;
> 
> Hi all,
> 
> could you please review this trivial one-liner which updates TestNativeProcessBuilder.java test to use driver mode?
> 
> webrev: http://cr.openjdk.java.net/~iignatyev//8243943/webrev.00
> JBS: https://bugs.openjdk.java.net/browse/JDK-8243943
> 
> Thanks,
> -- Igor
> 


From david.holmes at oracle.com  Tue Apr 28 06:08:39 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 28 Apr 2020 16:08:39 +1000
Subject: RFR(M/T) : 8243935 : s.h.WhiteBoxPermission must be
 ClassFileInstall-ed together w/ s.h.WhiteBox
In-Reply-To: <BBD7C18E-16AC-4BD1-9660-F103FC4F0D41@oracle.com>
References: <018B98AF-1DC5-4074-AE80-6601F9FDB867@oracle.com>
 <e5a0d476-c516-b627-920f-9f25f0795961@oracle.com>
 <BBD7C18E-16AC-4BD1-9660-F103FC4F0D41@oracle.com>
Message-ID: <2d8ef01d-5ace-c55f-6a89-60d8b3a2ced4@oracle.com>

On 28/04/2020 3:37 pm, Igor Ignatyev wrote:
> 
> 
>> On Apr 27, 2020, at 10:03 PM, David Holmes <david.holmes at oracle.com> wrote:
>>
>> Hi Igor,
>>
>> On 28/04/2020 12:32 pm, Igor Ignatyev wrote:
>>> http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/
>>>> 338 lines changed: 107 ins; 0 del; 231 mod
>>> Hi all,
>>> could you please review the trivial yet tedious patch which adds 'sun.hotspot.WhiteBox$WhiteBoxPermission' to all 'ClassFileInstaller sun.hotspot.WhiteBox' actions?
>>> the patch also
>>>   - replaces '@run main ClassFileInstaller' w/ '@run driver ClassFileInstaller'
>>>   - unifies the way ClassFileInstaller sun.hotspot.WhiteBox are aligned
>>
>> That alignment change is unnecessary and makes it very hard to review this patch.
>>
>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8243935
>>> webrevs:
>>>   - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/ -- regular webrev
>>>   - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.including_ws -- shows "whitespace only" changes
>>
>> I'd like to see the patch that leaves out the whitespace-only changes - is that possible?
> sure, I've used `hg diff -w` (--ignore-all-space    ignore white space when comparing lines) to remove all whitespace changes, here is webrev --  http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.w/

Thanks that was somewhat better. Change seems okay.

David

> -- Igor
>>
>> Thanks,
>> David
>>
>>> Thanks,
>>> -- Igor
> 

From thomas.stuefe at gmail.com  Tue Apr 28 06:43:17 2020
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Tue, 28 Apr 2020 08:43:17 +0200
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64
In-Reply-To: <56a0cf85-9698-64b2-95b4-e5b123c3e9e2@oracle.com>
References: <56a0cf85-9698-64b2-95b4-e5b123c3e9e2@oracle.com>
Message-ID: <CAA-vtUyFiMpf2vSXLf-c9uaKYXF1tSA3At5KJySCiz2skG0w3A@mail.gmail.com>

Hi Erik,

nice change, and now I do not have to keep in mind "heap>32G = no class
space" :)

I eyeballed the metaspace changes (what little there are).

This:

-     char* base = (char*)align_up(CompressedOops::end(),
_reserve_alignment);+     char* base;+     if (UseCompressedOops) {+
    base = (char*)align_up(CompressedOops::end(),
_reserve_alignment);+     } else {+       base =
(char*)HeapBaseMinAddress;+     }

may not work as intended. You probably want the end of the reserved
heap as attach point for ccs, not the HeapBaseMinAddress.

Also, could we have some tests which exercise cds/metaspace initialization
without coops and with ccps? Easiest way would be to extend
"CompressedOops/CompressedClassPointers.java" by some cases.

Thank you, Thomas



On Fri, Apr 24, 2020 at 10:19 AM Erik ?sterlund <erik.osterlund at oracle.com>
wrote:

> Hi,
>
> Today, the use of compressed class pointers assumes the use of
> compressed oops.
> This patch loosens up the relationship between compressed oops and
> compressed
> class pointers, so that compressed class pointers can be used without
> compressed
> oops. This benefits for example ZGC that wants compressed class
> pointers, but
> not compressed oops, effectively giving ZGC 4 bytes smaller objects.
>
> Much of the complexity of the change is that r12 used to be used by
> compressed
> class pointers as some kind of semi-fast temp register that one would
> restore
> to the compressed oops heap base (or zero). That made it effectively a
> slightly
> optimized spilling mechanism used to find a temp register. I replaced that
> mechanism with a plain old normal temp register that you pass in as a
> parameter.
> The bad news is that I had to find available temp registers in a lot of
> places.
> The good news is that almost always were there temp registers available
> for free,
> and hence the new temp register is even faster than the old optimized
> spilling
> mechanism. Because we almost never need any spilling at all in the
> contexts where
> this is used.
>
> Since I want the 4 new bytes to actually make objects smaller, I poked
> the new
> layout code a bit so that the klass gap is made available for fields.
> That used
> to be made available only with compressed oops enabled, due to
> restrictions in
> the layout code. Our new layout code does not have such restrictions, and
> so
> I will make the 4 bytes available for fields when the new layout code is
> used
> and compressed class pointers are used.
>
> Now I'm only fixing this for HotSpot x86_64. Ideally the use of
> compressed oops
> and compressed class pointers should not be entangled in other platforms
> and
> Graal. But that is beyond the scope of this change, and I will let them
> behave
> the way that they used to, to be potentially fixed later.
>
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8241825
>
> Webrev:
> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/
>
> Testing:
> hs-tier1-7
>
> Thanks,
> /Erik
>

From stefan.karlsson at oracle.com  Tue Apr 28 07:34:54 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Tue, 28 Apr 2020 09:34:54 +0200
Subject: RFR(M/T) : 8243935 : s.h.WhiteBoxPermission must be
 ClassFileInstall-ed together w/ s.h.WhiteBox
In-Reply-To: <2d8ef01d-5ace-c55f-6a89-60d8b3a2ced4@oracle.com>
References: <018B98AF-1DC5-4074-AE80-6601F9FDB867@oracle.com>
 <e5a0d476-c516-b627-920f-9f25f0795961@oracle.com>
 <BBD7C18E-16AC-4BD1-9660-F103FC4F0D41@oracle.com>
 <2d8ef01d-5ace-c55f-6a89-60d8b3a2ced4@oracle.com>
Message-ID: <c1888765-ca63-64d4-e612-02fcb95932c2@oracle.com>

Hi,

I have another idea. Can't we simply remove WhiteBoxPermission and 
incorporate that functionality in the WhiteBox class? We would then be 
able to remove all these WhiteBoxPermission lines.

I've tested this and it seems to work:
https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/

Summary of changes:??? 695 lines changed: 0 ins; 657 del; 38 mod; 60752 
unchg

The interesting code:
https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/test/lib/sun/hotspot/WhiteBox.java.udiff.html

(There are some parts of the patch that was butchered by the sed-script 
- I'll fix this if we agree that this is a good thing to do)

If you think this is something we should do, I'll create an RFE, 
complete the patch, run more tests, and send it out for review.

Thanks,
StefanK

On 2020-04-28 08:08, David Holmes wrote:
> On 28/04/2020 3:37 pm, Igor Ignatyev wrote:
>>
>>
>>> On Apr 27, 2020, at 10:03 PM, David Holmes <david.holmes at oracle.com> 
>>> wrote:
>>>
>>> Hi Igor,
>>>
>>> On 28/04/2020 12:32 pm, Igor Ignatyev wrote:
>>>> http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/
>>>>> 338 lines changed: 107 ins; 0 del; 231 mod
>>>> Hi all,
>>>> could you please review the trivial yet tedious patch which adds 
>>>> 'sun.hotspot.WhiteBox$WhiteBoxPermission' to all 
>>>> 'ClassFileInstaller sun.hotspot.WhiteBox' actions?
>>>> the patch also
>>>> ? - replaces '@run main ClassFileInstaller' w/ '@run driver 
>>>> ClassFileInstaller'
>>>> ? - unifies the way ClassFileInstaller sun.hotspot.WhiteBox are 
>>>> aligned
>>>
>>> That alignment change is unnecessary and makes it very hard to 
>>> review this patch.
>>>
>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8243935
>>>> webrevs:
>>>> ? - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/ -- 
>>>> regular webrev
>>>> ? - 
>>>> http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.including_ws 
>>>> -- shows "whitespace only" changes
>>>
>>> I'd like to see the patch that leaves out the whitespace-only 
>>> changes - is that possible?
>> sure, I've used `hg diff -w` (--ignore-all-space??? ignore white 
>> space when comparing lines) to remove all whitespace changes, here is 
>> webrev -- http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.w/
>
> Thanks that was somewhat better. Change seems okay.
>
> David
>
>> -- Igor
>>>
>>> Thanks,
>>> David
>>>
>>>> Thanks,
>>>> -- Igor
>>


From erik.osterlund at oracle.com  Tue Apr 28 08:06:50 2020
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Tue, 28 Apr 2020 10:06:50 +0200
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64
In-Reply-To: <CAA-vtUyFiMpf2vSXLf-c9uaKYXF1tSA3At5KJySCiz2skG0w3A@mail.gmail.com>
References: <56a0cf85-9698-64b2-95b4-e5b123c3e9e2@oracle.com>
 <CAA-vtUyFiMpf2vSXLf-c9uaKYXF1tSA3At5KJySCiz2skG0w3A@mail.gmail.com>
Message-ID: <9a314ebe-5499-ecb6-3c5b-0c8ce7f9c8d2@oracle.com>

Hi Thomas,

On 2020-04-28 08:43, Thomas St?fe wrote:
> Hi Erik,
>
> nice change, and now I do not have to keep in mind "heap>32G = no 
> class space" :)

Glad you liked it!

> I eyeballed the metaspace changes (what little there are).
>
> This:
>
> - char* base = (char*)align_up(CompressedOops::end(), 
> _reserve_alignment);+ char* base;+ if (UseCompressedOops) {+ base = 
> (char*)align_up(CompressedOops::end(), _reserve_alignment);+ } else {+ 
> base = (char*)HeapBaseMinAddress;+ }
> may not work as intended. You probably want the end of the reserved heap as attach point for ccs, not the HeapBaseMinAddress.

Could you please expand on that? To make it clear, what this code does 
is to put metaspace at the
minimum address we are allowed to map, when compressed oops is disabled. 
The reasoning for doing
that is that if the low address bits (resulting in good compressed 
pointers) are not used by compressed oops,
then they are available to be used by compressed class pointers instead, 
such that at least compressed class
pointers can utilize this VA to get optimal encoding of class pointers.

Is there anything not desirable about this? If so, I can't see it, and 
would appreciate if you expand on why
that would be a bad idea. Perhaps there is something I do not see here.

> Also, could we have some?tests which exercise cds/metaspace 
> initialization without coops and with ccps? Easiest way would be to 
> extend "CompressedOops/CompressedClassPointers.java" by some cases.

I will have a look at that for the next webrev.

Thanks for the review.

/Erik

> Thank you, Thomas
>
> On Fri, Apr 24, 2020 at 10:19 AM Erik ?sterlund 
> <erik.osterlund at oracle.com <mailto:erik.osterlund at oracle.com>> wrote:
>
>     Hi,
>
>     Today, the use of compressed class pointers assumes the use of
>     compressed oops.
>     This patch loosens up the relationship between compressed oops and
>     compressed
>     class pointers, so that compressed class pointers can be used without
>     compressed
>     oops. This benefits for example ZGC that wants compressed class
>     pointers, but
>     not compressed oops, effectively giving ZGC 4 bytes smaller objects.
>
>     Much of the complexity of the change is that r12 used to be used by
>     compressed
>     class pointers as some kind of semi-fast temp register that one would
>     restore
>     to the compressed oops heap base (or zero). That made it
>     effectively a
>     slightly
>     optimized spilling mechanism used to find a temp register. I
>     replaced that
>     mechanism with a plain old normal temp register that you pass in as a
>     parameter.
>     The bad news is that I had to find available temp registers in a
>     lot of
>     places.
>     The good news is that almost always were there temp registers
>     available
>     for free,
>     and hence the new temp register is even faster than the old optimized
>     spilling
>     mechanism. Because we almost never need any spilling at all in the
>     contexts where
>     this is used.
>
>     Since I want the 4 new bytes to actually make objects smaller, I
>     poked
>     the new
>     layout code a bit so that the klass gap is made available for fields.
>     That used
>     to be made available only with compressed oops enabled, due to
>     restrictions in
>     the layout code. Our new layout code does not have such
>     restrictions, and so
>     I will make the 4 bytes available for fields when the new layout
>     code is
>     used
>     and compressed class pointers are used.
>
>     Now I'm only fixing this for HotSpot x86_64. Ideally the use of
>     compressed oops
>     and compressed class pointers should not be entangled in other
>     platforms and
>     Graal. But that is beyond the scope of this change, and I will let
>     them
>     behave
>     the way that they used to, to be potentially fixed later.
>
>     Bug:
>     https://bugs.openjdk.java.net/browse/JDK-8241825
>
>     Webrev:
>     http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/
>
>     Testing:
>     hs-tier1-7
>
>     Thanks,
>     /Erik
>


From martin.doerr at sap.com  Tue Apr 28 09:30:38 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Tue, 28 Apr 2020 09:30:38 +0000
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64
In-Reply-To: <9a314ebe-5499-ecb6-3c5b-0c8ce7f9c8d2@oracle.com>
References: <56a0cf85-9698-64b2-95b4-e5b123c3e9e2@oracle.com>
 <CAA-vtUyFiMpf2vSXLf-c9uaKYXF1tSA3At5KJySCiz2skG0w3A@mail.gmail.com>
 <9a314ebe-5499-ecb6-3c5b-0c8ce7f9c8d2@oracle.com>
Message-ID: <AM4PR02MB3057DC7374B9FC6777B22E309AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>

Hi Erik,

I've run your new tests + some other stuff on PPC64 and s390 and it works great.
If aarch64 is also fine, I'll vote for removing COMPRESSED_CLASS_POINTERS_DEPENDS_ON_COMPRESSED_OOPS.

Best regards,
Martin


> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
> Erik ?sterlund
> Sent: Dienstag, 28. April 2020 10:07
> To: Thomas St?fe <thomas.stuefe at gmail.com>
> Cc: hotspot-dev <hotspot-dev at openjdk.java.net>
> Subject: Re: RFR: 8241825: Make compressed oops and compressed class
> pointers independent on x86_64
> 
> Hi Thomas,
> 
> On 2020-04-28 08:43, Thomas St?fe wrote:
> > Hi Erik,
> >
> > nice change, and now I do not have to keep in mind "heap>32G = no
> > class space" :)
> 
> Glad you liked it!
> 
> > I eyeballed the metaspace changes (what little there are).
> >
> > This:
> >
> > - char* base = (char*)align_up(CompressedOops::end(),
> > _reserve_alignment);+ char* base;+ if (UseCompressedOops) {+ base =
> > (char*)align_up(CompressedOops::end(), _reserve_alignment);+ } else {+
> > base = (char*)HeapBaseMinAddress;+ }
> > may not work as intended. You probably want the end of the reserved
> heap as attach point for ccs, not the HeapBaseMinAddress.
> 
> Could you please expand on that? To make it clear, what this code does
> is to put metaspace at the
> minimum address we are allowed to map, when compressed oops is
> disabled.
> The reasoning for doing
> that is that if the low address bits (resulting in good compressed
> pointers) are not used by compressed oops,
> then they are available to be used by compressed class pointers instead,
> such that at least compressed class
> pointers can utilize this VA to get optimal encoding of class pointers.
> 
> Is there anything not desirable about this? If so, I can't see it, and
> would appreciate if you expand on why
> that would be a bad idea. Perhaps there is something I do not see here.
> 
> > Also, could we have some?tests which exercise cds/metaspace
> > initialization without coops and with ccps? Easiest way would be to
> > extend "CompressedOops/CompressedClassPointers.java" by some cases.
> 
> I will have a look at that for the next webrev.
> 
> Thanks for the review.
> 
> /Erik
> 
> > Thank you, Thomas
> >
> > On Fri, Apr 24, 2020 at 10:19 AM Erik ?sterlund
> > <erik.osterlund at oracle.com <mailto:erik.osterlund at oracle.com>> wrote:
> >
> >     Hi,
> >
> >     Today, the use of compressed class pointers assumes the use of
> >     compressed oops.
> >     This patch loosens up the relationship between compressed oops and
> >     compressed
> >     class pointers, so that compressed class pointers can be used without
> >     compressed
> >     oops. This benefits for example ZGC that wants compressed class
> >     pointers, but
> >     not compressed oops, effectively giving ZGC 4 bytes smaller objects.
> >
> >     Much of the complexity of the change is that r12 used to be used by
> >     compressed
> >     class pointers as some kind of semi-fast temp register that one would
> >     restore
> >     to the compressed oops heap base (or zero). That made it
> >     effectively a
> >     slightly
> >     optimized spilling mechanism used to find a temp register. I
> >     replaced that
> >     mechanism with a plain old normal temp register that you pass in as a
> >     parameter.
> >     The bad news is that I had to find available temp registers in a
> >     lot of
> >     places.
> >     The good news is that almost always were there temp registers
> >     available
> >     for free,
> >     and hence the new temp register is even faster than the old optimized
> >     spilling
> >     mechanism. Because we almost never need any spilling at all in the
> >     contexts where
> >     this is used.
> >
> >     Since I want the 4 new bytes to actually make objects smaller, I
> >     poked
> >     the new
> >     layout code a bit so that the klass gap is made available for fields.
> >     That used
> >     to be made available only with compressed oops enabled, due to
> >     restrictions in
> >     the layout code. Our new layout code does not have such
> >     restrictions, and so
> >     I will make the 4 bytes available for fields when the new layout
> >     code is
> >     used
> >     and compressed class pointers are used.
> >
> >     Now I'm only fixing this for HotSpot x86_64. Ideally the use of
> >     compressed oops
> >     and compressed class pointers should not be entangled in other
> >     platforms and
> >     Graal. But that is beyond the scope of this change, and I will let
> >     them
> >     behave
> >     the way that they used to, to be potentially fixed later.
> >
> >     Bug:
> >     https://bugs.openjdk.java.net/browse/JDK-8241825
> >
> >     Webrev:
> >     http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/
> >
> >     Testing:
> >     hs-tier1-7
> >
> >     Thanks,
> >     /Erik
> >


From david.holmes at oracle.com  Tue Apr 28 09:38:37 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 28 Apr 2020 19:38:37 +1000
Subject: URGENT (trivial) RFR: 8243989:
 test/hotspot/jtreg/serviceability/jvmti/CanGenerateAllClassHook/CanGenerateAllClassHook.java
 needs to use othervm
Message-ID: <a886ed77-c06f-6d26-3614-46a1127e3837@oracle.com>

https://bugs.openjdk.java.net/browse/JDK-8243989

diff -r 5dfc5e2f1bfc 
test/hotspot/jtreg/serviceability/jvmti/CanGenerateAllClassHook/CanGenerateAllClassHook.java
--- 
a/test/hotspot/jtreg/serviceability/jvmti/CanGenerateAllClassHook/CanGenerateAllClassHook.java
+++ 
b/test/hotspot/jtreg/serviceability/jvmti/CanGenerateAllClassHook/CanGenerateAllClassHook.java
@@ -30,7 +30,7 @@
   * @requires vm.cds
   * @library /test/lib
   * @compile CanGenerateAllClassHook.java
- * @run main/native CanGenerateAllClassHook
+ * @run main/othervm/native CanGenerateAllClassHook
   */


Thanks,
David

From martin.doerr at sap.com  Tue Apr 28 09:43:47 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Tue, 28 Apr 2020 09:43:47 +0000
Subject: URGENT (trivial) RFR: 8243989:
 test/hotspot/jtreg/serviceability/jvmti/CanGenerateAllClassHook/CanGenerateAllClassHook.java
 needs to use othervm
In-Reply-To: <a886ed77-c06f-6d26-3614-46a1127e3837@oracle.com>
References: <a886ed77-c06f-6d26-3614-46a1127e3837@oracle.com>
Message-ID: <AM4PR02MB30576178694B22624675B11D9AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>

Hi David,

looks good and trivial.

Best regards,
Martin


> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
> David Holmes
> Sent: Dienstag, 28. April 2020 11:39
> To: hotspot-dev developers <hotspot-dev at openjdk.java.net>;
> serviceability-dev <serviceability-dev at openjdk.java.net>
> Subject: URGENT (trivial) RFR: 8243989:
> test/hotspot/jtreg/serviceability/jvmti/CanGenerateAllClassHook/CanGener
> ateAllClassHook.java needs to use othervm
> 
> https://bugs.openjdk.java.net/browse/JDK-8243989
> 
> diff -r 5dfc5e2f1bfc
> test/hotspot/jtreg/serviceability/jvmti/CanGenerateAllClassHook/CanGener
> ateAllClassHook.java
> ---
> a/test/hotspot/jtreg/serviceability/jvmti/CanGenerateAllClassHook/CanGen
> erateAllClassHook.java
> +++
> b/test/hotspot/jtreg/serviceability/jvmti/CanGenerateAllClassHook/CanGen
> erateAllClassHook.java
> @@ -30,7 +30,7 @@
>    * @requires vm.cds
>    * @library /test/lib
>    * @compile CanGenerateAllClassHook.java
> - * @run main/native CanGenerateAllClassHook
> + * @run main/othervm/native CanGenerateAllClassHook
>    */
> 
> 
> Thanks,
> David

From matthias.baesken at sap.com  Tue Apr 28 09:45:14 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 28 Apr 2020 09:45:14 +0000
Subject: RFR: 8243389: enhance os::pd_print_cpu_info on linux
Message-ID: <AM0PR02MB541225ED5513768AED6732F093AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>

Ping ... any reviews please?

Thanks, Matthias

-------------------------------
Hello, please review the following enhancement of  os::pd_print_cpu_info on linux .
It adds  information about
cache
online/offline cpus
frequency details
governors

by looking at the info provided at /sys/devices/system/cpu  .

.....
bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8243389

http://cr.openjdk.java.net/~mbaesken/webrevs/8243389.0/


From david.holmes at oracle.com  Tue Apr 28 09:49:19 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 28 Apr 2020 19:49:19 +1000
Subject: RFR(M/T) : 8243935 : s.h.WhiteBoxPermission must be
 ClassFileInstall-ed together w/ s.h.WhiteBox
In-Reply-To: <c1888765-ca63-64d4-e612-02fcb95932c2@oracle.com>
References: <018B98AF-1DC5-4074-AE80-6601F9FDB867@oracle.com>
 <e5a0d476-c516-b627-920f-9f25f0795961@oracle.com>
 <BBD7C18E-16AC-4BD1-9660-F103FC4F0D41@oracle.com>
 <2d8ef01d-5ace-c55f-6a89-60d8b3a2ced4@oracle.com>
 <c1888765-ca63-64d4-e612-02fcb95932c2@oracle.com>
Message-ID: <ec0209dd-5a25-d5bc-4b43-b01559cc1412@oracle.com>

On 28/04/2020 5:34 pm, Stefan Karlsson wrote:
> Hi,
> 
> I have another idea. Can't we simply remove WhiteBoxPermission and 
> incorporate that functionality in the WhiteBox class? We would then be 
> able to remove all these WhiteBoxPermission lines.

Interesting though a little hackish. :)

But why is this an issue anyway we already hacked ClassfileInstaller to 
ensure it always copies the permission class!!!

  102     // Add commonly used inner classes that are often omitted by 
mistake. Currently
  103     // we support only sun.hotspot.WhiteBox$WhiteBoxPermission. 
See JDK-8199290
  104     private static String[] addInnerClasses(String[] classes, int 
startIdx) {
  105         boolean seenWB = false;
  106         boolean seenWBInner = false;
  107         final String wb = "sun.hotspot.WhiteBox";
  108         final String wbInner = 
"sun.hotspot.WhiteBox$WhiteBoxPermission";

David
-----

> I've tested this and it seems to work:
> https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/ 
> 
> 
> Summary of changes:??? 695 lines changed: 0 ins; 657 del; 38 mod; 60752 
> unchg
> 
> The interesting code:
> https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/test/lib/sun/hotspot/WhiteBox.java.udiff.html 
> 
> 
> (There are some parts of the patch that was butchered by the sed-script 
> - I'll fix this if we agree that this is a good thing to do)
> 
> If you think this is something we should do, I'll create an RFE, 
> complete the patch, run more tests, and send it out for review.
> 
> Thanks,
> StefanK
> 
> On 2020-04-28 08:08, David Holmes wrote:
>> On 28/04/2020 3:37 pm, Igor Ignatyev wrote:
>>>
>>>
>>>> On Apr 27, 2020, at 10:03 PM, David Holmes <david.holmes at oracle.com> 
>>>> wrote:
>>>>
>>>> Hi Igor,
>>>>
>>>> On 28/04/2020 12:32 pm, Igor Ignatyev wrote:
>>>>> http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/
>>>>>> 338 lines changed: 107 ins; 0 del; 231 mod
>>>>> Hi all,
>>>>> could you please review the trivial yet tedious patch which adds 
>>>>> 'sun.hotspot.WhiteBox$WhiteBoxPermission' to all 
>>>>> 'ClassFileInstaller sun.hotspot.WhiteBox' actions?
>>>>> the patch also
>>>>> ? - replaces '@run main ClassFileInstaller' w/ '@run driver 
>>>>> ClassFileInstaller'
>>>>> ? - unifies the way ClassFileInstaller sun.hotspot.WhiteBox are 
>>>>> aligned
>>>>
>>>> That alignment change is unnecessary and makes it very hard to 
>>>> review this patch.
>>>>
>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8243935
>>>>> webrevs:
>>>>> ? - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/ -- 
>>>>> regular webrev
>>>>> ? - 
>>>>> http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.including_ws 
>>>>> -- shows "whitespace only" changes
>>>>
>>>> I'd like to see the patch that leaves out the whitespace-only 
>>>> changes - is that possible?
>>> sure, I've used `hg diff -w` (--ignore-all-space??? ignore white 
>>> space when comparing lines) to remove all whitespace changes, here is 
>>> webrev -- http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.w/
>>
>> Thanks that was somewhat better. Change seems okay.
>>
>> David
>>
>>> -- Igor
>>>>
>>>> Thanks,
>>>> David
>>>>
>>>>> Thanks,
>>>>> -- Igor
>>>
> 

From david.holmes at oracle.com  Tue Apr 28 09:51:34 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 28 Apr 2020 19:51:34 +1000
Subject: URGENT (trivial) RFR: 8243989:
 test/hotspot/jtreg/serviceability/jvmti/CanGenerateAllClassHook/CanGenerateAllClassHook.java
 needs to use othervm
In-Reply-To: <AM4PR02MB30576178694B22624675B11D9AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>
References: <a886ed77-c06f-6d26-3614-46a1127e3837@oracle.com>
 <AM4PR02MB30576178694B22624675B11D9AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>
Message-ID: <d39322af-b2de-7d65-51b5-c96e4ee9c0e1@oracle.com>

Thanks Martin!

David

On 28/04/2020 7:43 pm, Doerr, Martin wrote:
> Hi David,
> 
> looks good and trivial.
> 
> Best regards,
> Martin
> 
> 
>> -----Original Message-----
>> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
>> David Holmes
>> Sent: Dienstag, 28. April 2020 11:39
>> To: hotspot-dev developers <hotspot-dev at openjdk.java.net>;
>> serviceability-dev <serviceability-dev at openjdk.java.net>
>> Subject: URGENT (trivial) RFR: 8243989:
>> test/hotspot/jtreg/serviceability/jvmti/CanGenerateAllClassHook/CanGener
>> ateAllClassHook.java needs to use othervm
>>
>> https://bugs.openjdk.java.net/browse/JDK-8243989
>>
>> diff -r 5dfc5e2f1bfc
>> test/hotspot/jtreg/serviceability/jvmti/CanGenerateAllClassHook/CanGener
>> ateAllClassHook.java
>> ---
>> a/test/hotspot/jtreg/serviceability/jvmti/CanGenerateAllClassHook/CanGen
>> erateAllClassHook.java
>> +++
>> b/test/hotspot/jtreg/serviceability/jvmti/CanGenerateAllClassHook/CanGen
>> erateAllClassHook.java
>> @@ -30,7 +30,7 @@
>>     * @requires vm.cds
>>     * @library /test/lib
>>     * @compile CanGenerateAllClassHook.java
>> - * @run main/native CanGenerateAllClassHook
>> + * @run main/othervm/native CanGenerateAllClassHook
>>     */
>>
>>
>> Thanks,
>> David

From xxinliu at amazon.com  Tue Apr 28 09:57:48 2020
From: xxinliu at amazon.com (Liu, Xin)
Date: Tue, 28 Apr 2020 09:57:48 +0000
Subject: FW: [Mach5] mach5-one-phh-JDK-8151779-20200427-2151-10554367: FAILED
In-Reply-To: <C769AA2F-55A8-4986-9EEE-0AD93FB93E00@amazon.com>
References: <C769AA2F-55A8-4986-9EEE-0AD93FB93E00@amazon.com>
Message-ID: <E6D13CFA-A9E6-4262-8D33-2FB4412413B6@amazon.com>

Hello,

I recently received some build failures from submit repo.
May I know more details ? eg. linux-x64-linux-x64-build-5, linux-x64-debug-linux-x64-build-6
I have successfully built it on the latest linux on ubuntu 18.04 with gcc 7.5.

My nightly buildbot started failing recently on aarch64.
One issue is the error message of [cds] prevents configure from bootjdk determining.
https://hg.openjdk.java.net/jdk/jdk/file/1b8f9e72b22b/make/autoconf/boot-jdk.m4#l77

./bin/java --version
[0.006s][error][cds] Unable to map CDS archive -- os::vm_allocation_granularity() expected: 65536 actual: 4096
openjdk 14.0.1 2020-04-14
OpenJDK Runtime Environment AdoptOpenJDK (build 14.0.1+7)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 14.0.1+7, mixed mode)

Are you aware of this issue? I can?t see [cds] line on my linux/x86_64 host.
Maybe it?s aarch64-only.  CC aarch-port-dev.

Thanks,
--lx

From: <do-not-reply at oracle.com> on behalf of "do-not-reply at oracle.com" <do-not-reply at oracle.com>
Reply-To: "mach5_admin_ww_grp at oracle.com" <mach5_admin_ww_grp at oracle.com>
Date: Monday, April 27, 2020 at 3:46 PM
To: "Hohensee, Paul" <hohensee at amazon.com>
Subject: [EXTERNAL] [Mach5] mach5-one-phh-JDK-8151779-20200427-2151-10554367: FAILED
Job: mach5-one-phh-JDK-8151779-20200427-2151-10554367
BuildId: 2020-04-27-2150109.hohensee.source
No failed tests
Tasks Summary

  *   NOTHING_TO_RUN: 0
  *   UNABLE_TO_RUN: 24
  *   KILLED: 0
  *   NA: 0
  *   HARNESS_ERROR: 0
  *   FAILED: 0
  *   EXECUTED_WITH_FAILURE: 9
  *   PASSED: 51

Build
2 Unable to run

     *   linux-aarch64-install-linux-aarch64-build-signing-20 Dependency task failed: mach5...-10554367-linux-aarch64-linux-aarch64-build-1
     *   linux-x64-install-linux-x64-build-signing-21 Dependency task failed: mach5...427-2151-10554367-linux-x64-linux-x64-build-5

9 Executed with failure

     *   linux-aarch64-linux-aarch64-build-1 error while building, return value: 2
     *   linux-aarch64-debug-linux-aarch64-build-2 error while building, return value: 2
     *   linux-aarch64-open-linux-aarch64-build-3 error while building, return value: 2
     *   linux-aarch64-open-debug-linux-aarch64-build-4 error while building, return value: 2
     *   linux-x64-linux-x64-build-5 error while building, return value: 2
     *   linux-x64-debug-linux-x64-build-6 error while building, return value: 2
     *   linux-x64-debug-nopch-linux-x64-build-9 error while building, return value: 2
     *   linux-x64-open-linux-x64-build-7 error while building, return value: 2
     *   linux-x64-open-debug-linux-x64-build-8 error while building, return value: 2

Test
22 Unable to run

     *   tier1-product-open_test_hotspot_jtreg_tier1_common-linux-x64-24 Dependency task failed: mach5...427-2151-10554367-linux-x64-linux-x64-build-5
     *   tier1-debug-open_test_hotspot_jtreg_tier1_common-linux-x64-debug-30 Dependency task failed: mach5...51-10554367-linux-x64-debug-linux-x64-build-6
     *   tier1-debug-open_test_hotspot_jtreg_tier1_compiler_1-linux-x64-debug-33 Dependency task failed: mach5...51-10554367-linux-x64-debug-linux-x64-build-6
     *   tier1-debug-open_test_hotspot_jtreg_tier1_compiler_2-linux-x64-debug-36 Dependency task failed: mach5...51-10554367-linux-x64-debug-linux-x64-build-6
     *   tier1-debug-open_test_hotspot_jtreg_tier1_compiler_3-linux-x64-debug-39 Dependency task failed: mach5...51-10554367-linux-x64-debug-linux-x64-build-6
     *   tier1-debug-open_test_hotspot_jtreg_tier1_compiler_graal-linux-x64-debug-45 Dependency task failed: mach5...51-10554367-linux-x64-debug-linux-x64-build-6
     *   tier1-debug-open_test_hotspot_jtreg_tier1_compiler_not_xcomp-linux-x64-debug-42 Dependency task failed: mach5...51-10554367-linux-x64-debug-linux-x64-build-6
     *   tier1-debug-open_test_hotspot_jtreg_tier1_gc_1-linux-x64-debug-48 Dependency task failed: mach5...51-10554367-linux-x64-debug-linux-x64-build-6
     *   tier1-debug-open_test_hotspot_jtreg_tier1_gc_2-linux-x64-debug-51 Dependency task failed: mach5...51-10554367-linux-x64-debug-linux-x64-build-6
     *   tier1-product-open_test_hotspot_jtreg_tier1_gc_gcbasher-linux-x64-27 Dependency task failed: mach5...427-2151-10554367-linux-x64-linux-x64-build-5
     *   See all 22...

From erik.osterlund at oracle.com  Tue Apr 28 10:06:00 2020
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Tue, 28 Apr 2020 12:06:00 +0200
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64
In-Reply-To: <AM4PR02MB3057DC7374B9FC6777B22E309AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>
References: <56a0cf85-9698-64b2-95b4-e5b123c3e9e2@oracle.com>
 <CAA-vtUyFiMpf2vSXLf-c9uaKYXF1tSA3At5KJySCiz2skG0w3A@mail.gmail.com>
 <9a314ebe-5499-ecb6-3c5b-0c8ce7f9c8d2@oracle.com>
 <AM4PR02MB3057DC7374B9FC6777B22E309AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>
Message-ID: <c016f481-7267-ea5d-e56d-9c7aeb224012@oracle.com>

Hi Martin,

I am glad to hear it. I will flip 
COMPRESSED_CLASS_POINTERS_DEPENDS_ON_COMPRESSED_OOPS for you on
your platforms. However, this did not work in e.g. Graal for x86_64. So 
regardless, I'm afraid I'm gonna have to
at least temporarily hold on to 
COMPRESSED_CLASS_POINTERS_DEPENDS_ON_COMPRESSED_OOPS at least until
the issue is resolved in Graal upstream. I can not commit atomically to 
Graal and to OpenJDK, so unfortunately
I think I have to introduce the feature incrementally anyway. But I will 
try to get rid of it as soon as possible,
once it is supported by all relevant platforms and compilers.

Thanks,
/Erik

On 2020-04-28 11:30, Doerr, Martin wrote:
> Hi Erik,
>
> I've run your new tests + some other stuff on PPC64 and s390 and it works great.
> If aarch64 is also fine, I'll vote for removing COMPRESSED_CLASS_POINTERS_DEPENDS_ON_COMPRESSED_OOPS.
>
> Best regards,
> Martin
>
>
>> -----Original Message-----
>> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
>> Erik ?sterlund
>> Sent: Dienstag, 28. April 2020 10:07
>> To: Thomas St?fe <thomas.stuefe at gmail.com>
>> Cc: hotspot-dev <hotspot-dev at openjdk.java.net>
>> Subject: Re: RFR: 8241825: Make compressed oops and compressed class
>> pointers independent on x86_64
>>
>> Hi Thomas,
>>
>> On 2020-04-28 08:43, Thomas St?fe wrote:
>>> Hi Erik,
>>>
>>> nice change, and now I do not have to keep in mind "heap>32G = no
>>> class space" :)
>> Glad you liked it!
>>
>>> I eyeballed the metaspace changes (what little there are).
>>>
>>> This:
>>>
>>> - char* base = (char*)align_up(CompressedOops::end(),
>>> _reserve_alignment);+ char* base;+ if (UseCompressedOops) {+ base =
>>> (char*)align_up(CompressedOops::end(), _reserve_alignment);+ } else {+
>>> base = (char*)HeapBaseMinAddress;+ }
>>> may not work as intended. You probably want the end of the reserved
>> heap as attach point for ccs, not the HeapBaseMinAddress.
>>
>> Could you please expand on that? To make it clear, what this code does
>> is to put metaspace at the
>> minimum address we are allowed to map, when compressed oops is
>> disabled.
>> The reasoning for doing
>> that is that if the low address bits (resulting in good compressed
>> pointers) are not used by compressed oops,
>> then they are available to be used by compressed class pointers instead,
>> such that at least compressed class
>> pointers can utilize this VA to get optimal encoding of class pointers.
>>
>> Is there anything not desirable about this? If so, I can't see it, and
>> would appreciate if you expand on why
>> that would be a bad idea. Perhaps there is something I do not see here.
>>
>>> Also, could we have some?tests which exercise cds/metaspace
>>> initialization without coops and with ccps? Easiest way would be to
>>> extend "CompressedOops/CompressedClassPointers.java" by some cases.
>> I will have a look at that for the next webrev.
>>
>> Thanks for the review.
>>
>> /Erik
>>
>>> Thank you, Thomas
>>>
>>> On Fri, Apr 24, 2020 at 10:19 AM Erik ?sterlund
>>> <erik.osterlund at oracle.com <mailto:erik.osterlund at oracle.com>> wrote:
>>>
>>>      Hi,
>>>
>>>      Today, the use of compressed class pointers assumes the use of
>>>      compressed oops.
>>>      This patch loosens up the relationship between compressed oops and
>>>      compressed
>>>      class pointers, so that compressed class pointers can be used without
>>>      compressed
>>>      oops. This benefits for example ZGC that wants compressed class
>>>      pointers, but
>>>      not compressed oops, effectively giving ZGC 4 bytes smaller objects.
>>>
>>>      Much of the complexity of the change is that r12 used to be used by
>>>      compressed
>>>      class pointers as some kind of semi-fast temp register that one would
>>>      restore
>>>      to the compressed oops heap base (or zero). That made it
>>>      effectively a
>>>      slightly
>>>      optimized spilling mechanism used to find a temp register. I
>>>      replaced that
>>>      mechanism with a plain old normal temp register that you pass in as a
>>>      parameter.
>>>      The bad news is that I had to find available temp registers in a
>>>      lot of
>>>      places.
>>>      The good news is that almost always were there temp registers
>>>      available
>>>      for free,
>>>      and hence the new temp register is even faster than the old optimized
>>>      spilling
>>>      mechanism. Because we almost never need any spilling at all in the
>>>      contexts where
>>>      this is used.
>>>
>>>      Since I want the 4 new bytes to actually make objects smaller, I
>>>      poked
>>>      the new
>>>      layout code a bit so that the klass gap is made available for fields.
>>>      That used
>>>      to be made available only with compressed oops enabled, due to
>>>      restrictions in
>>>      the layout code. Our new layout code does not have such
>>>      restrictions, and so
>>>      I will make the 4 bytes available for fields when the new layout
>>>      code is
>>>      used
>>>      and compressed class pointers are used.
>>>
>>>      Now I'm only fixing this for HotSpot x86_64. Ideally the use of
>>>      compressed oops
>>>      and compressed class pointers should not be entangled in other
>>>      platforms and
>>>      Graal. But that is beyond the scope of this change, and I will let
>>>      them
>>>      behave
>>>      the way that they used to, to be potentially fixed later.
>>>
>>>      Bug:
>>>      https://bugs.openjdk.java.net/browse/JDK-8241825
>>>
>>>      Webrev:
>>>      http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/
>>>
>>>      Testing:
>>>      hs-tier1-7
>>>
>>>      Thanks,
>>>      /Erik
>>>


From nick.gasson at arm.com  Tue Apr 28 10:11:04 2020
From: nick.gasson at arm.com (Nick Gasson)
Date: Tue, 28 Apr 2020 18:11:04 +0800
Subject: [aarch64-port-dev ] FW: [Mach5]
 mach5-one-phh-JDK-8151779-20200427-2151-10554367: FAILED
In-Reply-To: <E6D13CFA-A9E6-4262-8D33-2FB4412413B6@amazon.com>
References: <C769AA2F-55A8-4986-9EEE-0AD93FB93E00@amazon.com>
 <E6D13CFA-A9E6-4262-8D33-2FB4412413B6@amazon.com>
Message-ID: <858sifgbt3.fsf@arm.com>

>
> My nightly buildbot started failing recently on aarch64.
> One issue is the error message of [cds] prevents configure from bootjdk determining.
> https://hg.openjdk.java.net/jdk/jdk/file/1b8f9e72b22b/make/autoconf/boot-jdk.m4#l77
>
> ./bin/java --version
> [0.006s][error][cds] Unable to map CDS archive -- os::vm_allocation_granularity() expected: 65536 actual: 4096
> openjdk 14.0.1 2020-04-14
> OpenJDK Runtime Environment AdoptOpenJDK (build 14.0.1+7)
> OpenJDK 64-Bit Server VM AdoptOpenJDK (build 14.0.1+7, mixed mode)
>
> Are you aware of this issue? I can?t see [cds] line on my linux/x86_64 host.
> Maybe it?s aarch64-only.  CC aarch-port-dev.
>

Was your boot JDK built on a machine configured with a different page
size to your current machine? Looks like the CDS archive was dumped on a
machine with 64k pages but you're running with 4k pages. There's a JBS
issue for this:

https://bugs.openjdk.java.net/browse/JDK-8236847


Thanks,
Nick

From martin.doerr at sap.com  Tue Apr 28 10:12:41 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Tue, 28 Apr 2020 10:12:41 +0000
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64
In-Reply-To: <c016f481-7267-ea5d-e56d-9c7aeb224012@oracle.com>
References: <56a0cf85-9698-64b2-95b4-e5b123c3e9e2@oracle.com>
 <CAA-vtUyFiMpf2vSXLf-c9uaKYXF1tSA3At5KJySCiz2skG0w3A@mail.gmail.com>
 <9a314ebe-5499-ecb6-3c5b-0c8ce7f9c8d2@oracle.com>
 <AM4PR02MB3057DC7374B9FC6777B22E309AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>
 <c016f481-7267-ea5d-e56d-9c7aeb224012@oracle.com>
Message-ID: <AM4PR02MB30579CE36EED2AE5476B59C79AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>

Hi Erik,

ok, thank you.

Best regards,
Martin


> -----Original Message-----
> From: Erik ?sterlund <erik.osterlund at oracle.com>
> Sent: Dienstag, 28. April 2020 12:06
> To: Doerr, Martin <martin.doerr at sap.com>; Thomas St?fe
> <thomas.stuefe at gmail.com>
> Cc: hotspot-dev <hotspot-dev at openjdk.java.net>
> Subject: Re: RFR: 8241825: Make compressed oops and compressed class
> pointers independent on x86_64
> 
> Hi Martin,
> 
> I am glad to hear it. I will flip
> COMPRESSED_CLASS_POINTERS_DEPENDS_ON_COMPRESSED_OOPS for
> you on
> your platforms. However, this did not work in e.g. Graal for x86_64. So
> regardless, I'm afraid I'm gonna have to
> at least temporarily hold on to
> COMPRESSED_CLASS_POINTERS_DEPENDS_ON_COMPRESSED_OOPS at
> least until
> the issue is resolved in Graal upstream. I can not commit atomically to
> Graal and to OpenJDK, so unfortunately
> I think I have to introduce the feature incrementally anyway. But I will
> try to get rid of it as soon as possible,
> once it is supported by all relevant platforms and compilers.
> 
> Thanks,
> /Erik
> 
> On 2020-04-28 11:30, Doerr, Martin wrote:
> > Hi Erik,
> >
> > I've run your new tests + some other stuff on PPC64 and s390 and it works
> great.
> > If aarch64 is also fine, I'll vote for removing
> COMPRESSED_CLASS_POINTERS_DEPENDS_ON_COMPRESSED_OOPS.
> >
> > Best regards,
> > Martin
> >
> >
> >> -----Original Message-----
> >> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf
> Of
> >> Erik ?sterlund
> >> Sent: Dienstag, 28. April 2020 10:07
> >> To: Thomas St?fe <thomas.stuefe at gmail.com>
> >> Cc: hotspot-dev <hotspot-dev at openjdk.java.net>
> >> Subject: Re: RFR: 8241825: Make compressed oops and compressed class
> >> pointers independent on x86_64
> >>
> >> Hi Thomas,
> >>
> >> On 2020-04-28 08:43, Thomas St?fe wrote:
> >>> Hi Erik,
> >>>
> >>> nice change, and now I do not have to keep in mind "heap>32G = no
> >>> class space" :)
> >> Glad you liked it!
> >>
> >>> I eyeballed the metaspace changes (what little there are).
> >>>
> >>> This:
> >>>
> >>> - char* base = (char*)align_up(CompressedOops::end(),
> >>> _reserve_alignment);+ char* base;+ if (UseCompressedOops) {+ base =
> >>> (char*)align_up(CompressedOops::end(), _reserve_alignment);+ } else
> {+
> >>> base = (char*)HeapBaseMinAddress;+ }
> >>> may not work as intended. You probably want the end of the reserved
> >> heap as attach point for ccs, not the HeapBaseMinAddress.
> >>
> >> Could you please expand on that? To make it clear, what this code does
> >> is to put metaspace at the
> >> minimum address we are allowed to map, when compressed oops is
> >> disabled.
> >> The reasoning for doing
> >> that is that if the low address bits (resulting in good compressed
> >> pointers) are not used by compressed oops,
> >> then they are available to be used by compressed class pointers instead,
> >> such that at least compressed class
> >> pointers can utilize this VA to get optimal encoding of class pointers.
> >>
> >> Is there anything not desirable about this? If so, I can't see it, and
> >> would appreciate if you expand on why
> >> that would be a bad idea. Perhaps there is something I do not see here.
> >>
> >>> Also, could we have some?tests which exercise cds/metaspace
> >>> initialization without coops and with ccps? Easiest way would be to
> >>> extend "CompressedOops/CompressedClassPointers.java" by some
> cases.
> >> I will have a look at that for the next webrev.
> >>
> >> Thanks for the review.
> >>
> >> /Erik
> >>
> >>> Thank you, Thomas
> >>>
> >>> On Fri, Apr 24, 2020 at 10:19 AM Erik ?sterlund
> >>> <erik.osterlund at oracle.com <mailto:erik.osterlund at oracle.com>>
> wrote:
> >>>
> >>>      Hi,
> >>>
> >>>      Today, the use of compressed class pointers assumes the use of
> >>>      compressed oops.
> >>>      This patch loosens up the relationship between compressed oops and
> >>>      compressed
> >>>      class pointers, so that compressed class pointers can be used without
> >>>      compressed
> >>>      oops. This benefits for example ZGC that wants compressed class
> >>>      pointers, but
> >>>      not compressed oops, effectively giving ZGC 4 bytes smaller objects.
> >>>
> >>>      Much of the complexity of the change is that r12 used to be used by
> >>>      compressed
> >>>      class pointers as some kind of semi-fast temp register that one would
> >>>      restore
> >>>      to the compressed oops heap base (or zero). That made it
> >>>      effectively a
> >>>      slightly
> >>>      optimized spilling mechanism used to find a temp register. I
> >>>      replaced that
> >>>      mechanism with a plain old normal temp register that you pass in as a
> >>>      parameter.
> >>>      The bad news is that I had to find available temp registers in a
> >>>      lot of
> >>>      places.
> >>>      The good news is that almost always were there temp registers
> >>>      available
> >>>      for free,
> >>>      and hence the new temp register is even faster than the old
> optimized
> >>>      spilling
> >>>      mechanism. Because we almost never need any spilling at all in the
> >>>      contexts where
> >>>      this is used.
> >>>
> >>>      Since I want the 4 new bytes to actually make objects smaller, I
> >>>      poked
> >>>      the new
> >>>      layout code a bit so that the klass gap is made available for fields.
> >>>      That used
> >>>      to be made available only with compressed oops enabled, due to
> >>>      restrictions in
> >>>      the layout code. Our new layout code does not have such
> >>>      restrictions, and so
> >>>      I will make the 4 bytes available for fields when the new layout
> >>>      code is
> >>>      used
> >>>      and compressed class pointers are used.
> >>>
> >>>      Now I'm only fixing this for HotSpot x86_64. Ideally the use of
> >>>      compressed oops
> >>>      and compressed class pointers should not be entangled in other
> >>>      platforms and
> >>>      Graal. But that is beyond the scope of this change, and I will let
> >>>      them
> >>>      behave
> >>>      the way that they used to, to be potentially fixed later.
> >>>
> >>>      Bug:
> >>>      https://bugs.openjdk.java.net/browse/JDK-8241825
> >>>
> >>>      Webrev:
> >>>      http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/
> >>>
> >>>      Testing:
> >>>      hs-tier1-7
> >>>
> >>>      Thanks,
> >>>      /Erik
> >>>


From martin.doerr at sap.com  Tue Apr 28 10:27:03 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Tue, 28 Apr 2020 10:27:03 +0000
Subject: RFR: 8243389: enhance os::pd_print_cpu_info on linux
In-Reply-To: <AM0PR02MB541225ED5513768AED6732F093AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
References: <AM0PR02MB541225ED5513768AED6732F093AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
Message-ID: <AM4PR02MB30579EBA32B978AC69A3F8DD9AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>

Hi Matthias,

technically, this looks good to me.

"Maximum cpu frequency" and "Minimum cpu frequency:" seems to be redundant when we have "Available cpu frequencies:".

I think it exceeds what a typical supporter needs to know in order to analyze issues.
Please consider using ExtensiveErrorReports for some information like e.g. cache related info.

Best regards,
Martin


> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
> Baesken, Matthias
> Sent: Dienstag, 28. April 2020 11:45
> To: 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: RE: RFR: 8243389: enhance os::pd_print_cpu_info on linux
> 
> Ping ... any reviews please?
> 
> Thanks, Matthias
> 
> -------------------------------
> Hello, please review the following enhancement of  os::pd_print_cpu_info
> on linux .
> It adds  information about
> cache
> online/offline cpus
> frequency details
> governors
> 
> by looking at the info provided at /sys/devices/system/cpu  .
> 
> .....
> bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8243389
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8243389.0/


From dmitry.chuyko at bell-sw.com  Tue Apr 28 10:33:26 2020
From: dmitry.chuyko at bell-sw.com (Dmitry Chuyko)
Date: Tue, 28 Apr 2020 13:33:26 +0300
Subject: RFR: 8242244: Remove redundant ELF machine definitions
In-Reply-To: <483038a4-7517-e190-cbbe-783da5f68ecc@physik.fu-berlin.de>
References: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>
 <22fdf007-a3d5-f14f-4d27-5746ad21a88e@oracle.com>
 <483038a4-7517-e190-cbbe-783da5f68ecc@physik.fu-berlin.de>
Message-ID: <8b5812dd-78bf-f099-f066-b0bac2cf1572@bell-sw.com>

Hello,

Sorry for being late.

I have following error during the build of 15 tip:

* For target hotspot_variant-server_libjvm_objs_os_linux.o:
jdk/src/hotspot/os/linux/os_linux.cpp: In static member function 'static 
void* os::dll_load(const char*, char*, int)':
jdk/src/hotspot/os/linux/os_linux.cpp:1854:6: error: 'EM_486' was not 
declared in this scope
 ???? {EM_486,???????? EM_386,???? ELFCLASS32, ELFDATA2LSB, (char*)"IA 32"},
 ????? ^~~~~~
jdk/src/hotspot/os/linux/os_linux.cpp:1854:6: note: suggested 
alternative: 'EM_386'
 ???? {EM_486,???????? EM_386,???? ELFCLASS32, ELFDATA2LSB, (char*)"IA 32"},
 ????? ^~~~~~
 ????? EM_386

It happens on CenOS 8 and Ubuntu 20.04 (kernel headers installed), 
aarch64 and amd64.

Is any additional configuration expected?

-Dmitry

On 4/28/20 2:48 AM, John Paul Adrian Glaubitz wrote:
> Hi David!
>
> On 4/28/20 12:53 AM, David Holmes wrote:
>> Apologies as I just spotted this in my mail backlog.
> No worries ;).
>
>> This seems fine and trivial.
> Thanks. I post a rebased version tomorrow - necessary because I added
> the definition for EM_RISCV to make sure we don't run into build issues
> with slightly older kernels has EM_RISCV hasn't been defined in the
> kernel for too long yet.
>
> Adrian
>

From aoqi at loongson.cn  Tue Apr 28 10:33:20 2020
From: aoqi at loongson.cn (Ao Qi)
Date: Tue, 28 Apr 2020 18:33:20 +0800
Subject: RFR: 8242244: Remove redundant ELF machine definitions
In-Reply-To: <483038a4-7517-e190-cbbe-783da5f68ecc@physik.fu-berlin.de>
References: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>
 <22fdf007-a3d5-f14f-4d27-5746ad21a88e@oracle.com>
 <483038a4-7517-e190-cbbe-783da5f68ecc@physik.fu-berlin.de>
Message-ID: <CALjzQn7sh=kWdt7Qj3vTZ+qyS60TLNXB4HcyacsFvRhLof4agQ@mail.gmail.com>

Hi,

It failed to build on my two linux x64 machines after this fix. They
are Ubuntu 18.04.4(5.3.0-46-generic) and CentOS Linux release
8.1.1911(4.18.0-147.el8.x86_64). Should we add this patch:
$ hg diff
diff -r 7205265660b2 src/hotspot/os/linux/os_linux.cpp
--- a/src/hotspot/os/linux/os_linux.cpp Tue Apr 28 05:53:19 2020 -0400
+++ b/src/hotspot/os/linux/os_linux.cpp Tue Apr 28 18:24:45 2020 +0800
@@ -110,6 +110,7 @@
 # include <stdint.h>
 # include <inttypes.h>
 # include <sys/ioctl.h>
+# include <linux/elf-em.h>

 #ifndef _GNU_SOURCE
   #define _GNU_SOURCE

Thanks,
Ao Qi

On Tue, Apr 28, 2020 at 7:48 AM John Paul Adrian Glaubitz
<glaubitz at physik.fu-berlin.de> wrote:
>
> Hi David!
>
> On 4/28/20 12:53 AM, David Holmes wrote:
> > Apologies as I just spotted this in my mail backlog.
>
> No worries ;).
>
> > This seems fine and trivial.
> Thanks. I post a rebased version tomorrow - necessary because I added
> the definition for EM_RISCV to make sure we don't run into build issues
> with slightly older kernels has EM_RISCV hasn't been defined in the
> kernel for too long yet.
>
> Adrian
>
> --
>  .''`.  John Paul Adrian Glaubitz
> : :' :  Debian Developer - glaubitz at debian.org
> `. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
>   `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913


From glaubitz at physik.fu-berlin.de  Tue Apr 28 10:38:04 2020
From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz)
Date: Tue, 28 Apr 2020 12:38:04 +0200
Subject: RFR: 8242244: Remove redundant ELF machine definitions
In-Reply-To: <CALjzQn7sh=kWdt7Qj3vTZ+qyS60TLNXB4HcyacsFvRhLof4agQ@mail.gmail.com>
References: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>
 <22fdf007-a3d5-f14f-4d27-5746ad21a88e@oracle.com>
 <483038a4-7517-e190-cbbe-783da5f68ecc@physik.fu-berlin.de>
 <CALjzQn7sh=kWdt7Qj3vTZ+qyS60TLNXB4HcyacsFvRhLof4agQ@mail.gmail.com>
Message-ID: <3714a010-139e-68d0-b6fa-052755eeca79@physik.fu-berlin.de>

Hi Ao!

On 4/28/20 12:33 PM, Ao Qi wrote:
> It failed to build on my two linux x64 machines after this fix. They
> are Ubuntu 18.04.4(5.3.0-46-generic) and CentOS Linux release
> 8.1.1911(4.18.0-147.el8.x86_64). Should we add this patch:
> $ hg diff
> diff -r 7205265660b2 src/hotspot/os/linux/os_linux.cpp
> --- a/src/hotspot/os/linux/os_linux.cpp Tue Apr 28 05:53:19 2020 -0400
> +++ b/src/hotspot/os/linux/os_linux.cpp Tue Apr 28 18:24:45 2020 +0800
> @@ -110,6 +110,7 @@
>  # include <stdint.h>
>  # include <inttypes.h>
>  # include <sys/ioctl.h>
> +# include <linux/elf-em.h>
> 
>  #ifndef _GNU_SOURCE
>    #define _GNU_SOURCE

Apologies for the problem. I'm surprised that it failed because of EM_486
missing but not the other missing EM_ definitions.

Your change looks reasonable in any case.

@Dmitry: Does it fix the build for you as well?

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz at debian.org
`. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

From erik.osterlund at oracle.com  Tue Apr 28 10:39:09 2020
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Tue, 28 Apr 2020 12:39:09 +0200
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64
In-Reply-To: <36D80B5C-9B88-4B6A-A0A1-AD87A2FB5EA9@oracle.com>
References: <56a0cf85-9698-64b2-95b4-e5b123c3e9e2@oracle.com>
 <B9EAEA33-057A-44C2-A7CF-DFB407776759@oracle.com>
 <42f02810-7a6e-1c7d-3fe3-b892f007ddc4@oracle.com>
 <7cda1c27-e745-e3c5-8c0e-a05a588baa6e@oracle.com>
 <866D9669-4CFD-4ABC-BCC2-3F2B51BDDDA9@oracle.com>
 <ddbd4b25-beb1-0f4a-20b9-086eef381d1a@oracle.com>
 <36D80B5C-9B88-4B6A-A0A1-AD87A2FB5EA9@oracle.com>
Message-ID: <6a17efee-934d-1d43-9baf-b7ef854e9dd9@oracle.com>

Hi Frederic,

On 2020-04-27 17:51, Frederic Parain wrote:
> Erik,
>
> Thank you for taking care of this issue.
>
>> On Apr 27, 2020, at 11:28, Erik ?sterlund <erik.osterlund at oracle.com> wrote:
>>
>> Hi Frederic,
>>
>> I hear your concern about hardcoded offsets making the JVM more fragile.
>>
>> First of all, let me assure you that you would not be able to run hello world if the offsets ever were to be
>> wrong. They are verified during bootstrapping in JavaClasses::check_offsets(), so you have to be quite skilled
>> to manage to break anything without immediately noticing it. You literally can't have run a single test or java -version
>> without immediately crashing. This makes this significantly less fragile. So I don't think I believe that this is fragile.
> I agree. I was planning to remove this code in 16, as part of the old layout code removal,
> but it can still be useful for some time.
>
>> Having said that, I do agree it could be annoying and a pain to maintain and update the layout code if such changes
>> are occasionally incompatible with the random selection of hardcoded offsets, forcing new awkward updates of said offsets,
>> and that hardcoding things in general is undesirable, compared to a more solid solution.
> I worked around the problem by isolating these classes so they don?t impact the regular
> cases of field layout, but it was just a work around. Completely removing hardcoded offsets
> is a cleaner and even more robust solution.
>
>> I had a look at an approach for removing hardcoded offsets completely. It looks something like this:
>>
>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00..not_01/
>>
>> The change is a bit involved and complicated though. Because of that, I would prefer to push it as a follow-up RFE
>> focused on removing hardcoded offsets, if you are okay with that. This change is already quite tricky, and I think
>> these things ought to be separate.
> As your changes linked above touch the VM init sequence (always a tricky area),
> I think dedicating a specific RFE for them is a good idea.

Okay. I filed https://bugs.openjdk.java.net/browse/JDK-8243996 for this, 
and will split out that change to that RFE.

>
>> The fix involves splitting allocation and code generation of the interpreter (due to dependencies with verification code in i2i adapters
>> generated during method loading, requiring the interpreter to be allocated early), computing offsets of jlr.Reference early during
>> resolution of well known classes (as is done for java_lang_String and java_lang_Class), because it needs offsets to remove the
>> discovered and referent fields from oop maps before its subclasses are initialized, and moving initialization (and generation)
>> of the interpreter until after offsets have been computed (no longer having the dependency to method loading and linking).
>> This way, all offsets have always been computed (or loaded by CDS) before they are needed, without any need for hardcoding.
>>
>> What do you think?
> Sounds like a good plan. I?ve read your changes, and they look fine to me.
> However, I would be more comfortable with extended testing to cover all the
> configurations (+/-CDS, templateInterpreter/cppInterpreter, C2/Graal) to be
> sure we didn?t miss a dependency somewhere.

I presume this comment is about the new 
https://bugs.openjdk.java.net/browse/JDK-8243996 right?
If so, I will have a look at what configurations we need to cover better 
for that change.

> Thank you for addressing this technical debt and solving it cleanly.

You are welcome! Thank you for reviewing my code.

Thanks,
/Erik

> Regards,
>
> Fred
>
>
>> On 2020-04-24 16:45, Frederic Parain wrote:
>>>> On Apr 24, 2020, at 10:36, coleen.phillimore at oracle.com wrote:
>>>>
>>>>
>>>>
>>>> On 4/24/20 10:16 AM, Erik ?sterlund wrote:
>>>>> Hi Frederic,
>>>>>
>>>>> Thanks for reviewing this.
>>>>>
>>>>> On 2020-04-24 15:28, Frederic Parain wrote:
>>>>>> Hi Erik,
>>>>>>
>>>>>> Why did you removed the code handling classes with hard coded layouts?
>>>>>> This code was there to provide the freedom to play with layout algorithms
>>>>>> without having to deal with these special cases. Without this code, any
>>>>>> modification is now constrained by these special classes.
>>>>> Good question. Since I started using the klass gap for fields, even when compressed oops is off, dealing with the
>>>>> hardcoded offsets and layouts really became quite awkward, to the extent that I started wondering why do we even have
>>>>> special layouts for these things anyway. I don't like special things. Treating it like any other object seemed far
>>>>> easier, than matching the special special hardcoded layouts with the hardcoded offsets.
>>>>>
>>>>> I wouldn't say that the whole layout algorithm is constrained by the fact that there are hardcoded offsets.
>>>>> I would say that it's just a matter of whoever changes the layout algorithms such that the hardcoded offsets
>>>>> are no longer the same, is just gonna have to update the offset to whatever the correct offsets are with the
>>>>> new algorithm. And if you look at the classes where we actually do hardcode the offsets, they are rather vanilla
>>>>> with relatively few ways you could change the layout other than changing it around for the sake of it. Especially
>>>>> primitive box objects. My fantasy is quite limited how we expect a new layout algorithm is gonna do anything else
>>>>> there really.
>>>>>
>>>>> Philosophically, I really think it is up to the person hardcoding an offset to know what the offset is going to
>>>>> be, ratherthan having completely different special (and worse) layouts produced for certain objects, just to
>>>>> satisfy the expectations of the hardcoded offsets.Especially if the result is pretty bad memory waste in for
>>>>> example Integer box objects, that can be quite plentyful in an application.
>>>> We used to have a lot more special hardcoded layouts but most have been removed. The reason these few classes are hardcoded is because the interpreter generates code using these offsets before JavaClasses::compute_offsets is run.  We tried to get rid of all of them and maybe there's a way to indirectly refer to them in the generated code so we can do so in the future.
>>> The proposed changes make the whole system brittle. Restoring the special handling of these classes during
>>> layout computation or making the generated code able to deal with dynamically computed offsets would be a
>>> more robust solution.
>>>
>>>
>>>>> The code to figure out the right offsets if a very small amount of code, compared to the rather large amount of
>>>>> code to create special layouts and syncing the special hardcoded layout code with the hardcoded offsets. That
>>>>> is why my patch has a net negative amount of code.
>>>>>
>>>>> Philosophically I also don't like having "special" layouts because we don't trust our ability to hardcode some
>>>>> offset to the right value. Almost all "special" things are bad and create unnecessary complications, and hence
>>>>> should have very good motivation for being allowed to be special. In this case, I really could not see any reason
>>>>> why these layouts would be special, unless it was somehow super hard to figure out what the hardcoded offsets would be.
>>>>> But for the classes we currently need hardcoded offsets, the calculations are trivial. So I don't think there is
>>>>> enough motivation to have special (bad) layouts for these objects for that reason. It only makes things harder than
>>>>> it needs to be, and makes the layout worse.
>>>>>
>>>>>> And I?d prefer to keep the dispatch in FieldLayoutBuilder::build_layout()
>>>>>> because we have more cases to handle there for the Valhalla project (the
>>>>>> layout algorithm for inline types is different than the one in
>>>>>> compute_regular_layout()).
>>>>> Okay. There was no need for the dispatching any more since there was only the normal layout and nothing
>>>>> else. But I can put that code back into the compute_regular_layout() function if that makes it easier for you
>>>>> in Valhalla, and have FieldLayoutBuilder::build_layout() just call that function. Will update in the next webrev.
>>> Thank you,
>>>
>>> Fred
>>>
>>>>>> Regards,
>>>>>>
>>>>>> Fred
>>>>>>
>>>>>>
>>>>>>> On Apr 24, 2020, at 04:17, Erik ?sterlund <erik.osterlund at oracle.com> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Today, the use of compressed class pointers assumes the use of compressed oops.
>>>>>>> This patch loosens up the relationship between compressed oops and compressed
>>>>>>> class pointers, so that compressed class pointers can be used without compressed
>>>>>>> oops. This benefits for example ZGC that wants compressed class pointers, but
>>>>>>> not compressed oops, effectively giving ZGC 4 bytes smaller objects.
>>>>>>>
>>>>>>> Much of the complexity of the change is that r12 used to be used by compressed
>>>>>>> class pointers as some kind of semi-fast temp register that one would restore
>>>>>>> to the compressed oops heap base (or zero). That made it effectively a slightly
>>>>>>> optimized spilling mechanism used to find a temp register. I replaced that
>>>>>>> mechanism with a plain old normal temp register that you pass in as a parameter.
>>>>>>> The bad news is that I had to find available temp registers in a lot of places.
>>>>>>> The good news is that almost always were there temp registers available for free,
>>>>>>> and hence the new temp register is even faster than the old optimized spilling
>>>>>>> mechanism. Because we almost never need any spilling at all in the contexts where
>>>>>>> this is used.
>>>>>>>
>>>>>>> Since I want the 4 new bytes to actually make objects smaller, I poked the new
>>>>>>> layout code a bit so that the klass gap is made available for fields. That used
>>>>>>> to be made available only with compressed oops enabled, due to restrictions in
>>>>>>> the layout code. Our new layout code does not have such restrictions, and so
>>>>>>> I will make the 4 bytes available for fields when the new layout code is used
>>>>>>> and compressed class pointers are used.
>>>>>>>
>>>>>>> Now I'm only fixing this for HotSpot x86_64. Ideally the use of compressed oops
>>>>>>> and compressed class pointers should not be entangled in other platforms and
>>>>>>> Graal. But that is beyond the scope of this change, and I will let them behave
>>>>>>> the way that they used to, to be potentially fixed later.
>>>>>>>
>>>>>>> Bug:
>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8241825
>>>>>>>
>>>>>>> Webrev:
>>>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/
>>>>>>>
>>>>>>> Testing:
>>>>>>> hs-tier1-7
>>>>>>>
>>>>>>> Thanks,
>>>>>>> /Erik


From leo.korinth at oracle.com  Tue Apr 28 10:42:20 2020
From: leo.korinth at oracle.com (Leo Korinth)
Date: Tue, 28 Apr 2020 12:42:20 +0200
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <50021864-6BFA-4B67-A16F-139F8EF10E62@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
 <135a6496-5b54-9117-5cbf-e3211ddc32dd@oracle.com>
 <bbb2b5ef-62f2-6507-95f7-03712a382153@oracle.com>
 <e242e057-de6e-493a-eb00-068b2db8c9fa@oracle.com>
 <df621fd2-0eaf-cb24-788f-12166b271303@oracle.com>
 <98749A40-89C8-4E15-B46A-23F0D1CB2A32@oracle.com>
 <841c7ed3-ec0b-b300-15f1-30863453f74b@oracle.com>
 <50021864-6BFA-4B67-A16F-139F8EF10E62@oracle.com>
Message-ID: <a8c00bb1-70f1-7cde-1983-f8710859c045@oracle.com>



On 23/04/2020 11:30, Kim Barrett wrote:
>> On Apr 19, 2020, at 11:39 AM, Leo Korinth <leo.korinth at oracle.com> wrote:
> 
> I've not yet covered all the items; I'll note inline which ones I
> haven't addressed yet.
> 
>> On 17/04/2020 09:46, Kim Barrett wrote:
>>>> On Apr 14, 2020, at 9:53 AM, Leo Korinth <leo.korinth at oracle.com> wrote:
>>>> On 06/04/2020 01:37, David Holmes wrote:
>>>>>> I could change CHECK_0 to use NULL_WORD though and if I do, I get compiler errors. CHECK_0 is used for both pointers and integral zeroes and types that implicitly converts from either. What the original thought was is unclear to me; is CHECK_0 to be used to return things convertible from pointers? If so, what is CHECK_NULL for?
>>>>> No CHECK_0 is intended for int-returning functions (which would also extend to intptr_t).
>>>>
>>>> It might be intended for it, but it is not used in such a way (therefore my remark on compile failures when using NULL_WORD)
>>>>
>>>> I will do no change here.
>>> It sounds like there are some places using CHECK_0 that should be
>>> using CHECK_NULL, and they should be fixed. But that can be a separate
>>> change. It might be aided (and backsliding prevented) by doing
>>> something similar to what I suggest for JVMCI_CHECK_0/NULL below.
>>
>> I believe that there are places that could use neither (the macro handles both integral and pointer types), but I do not want to fix that macro nest.
> 
> I'm not sure if you are saying anything beyond "CHECK_0 is being used
> in places where CHECK_NULL should be used instead"?

I believe (if I remember correctly) CHECK_0 is used in macros that can 
be used both with integral returns and pointer returns (or possibly 
returns converted from either). I will not check it as I will not change 
those macros (but that was what I tried to convey).

> 
> Unfortunately, as discussed in the part of the thread with David, my
> suggestion for JVMCI_CHECK_0 turns out to not have the desired effect.
> 
> So I agree, let's just leave these macros alone (both the JVMCI_ and
> unprefixed variants).

Good.

> 
>>> In the end you seem to have backed away from fixing the definition of
>>> NULL_WORD, and I'm not sure why. Though I think the definition that
>>> was proposed is at least unnecessarily baroque, possibly wrong. It
>>> seems to me that a good and sufficient change for that would be to
>>> delete all the existing platform definitions and put the following in
>>> globalDefinitions.hpp:
>>> const intptr_t NULL_WORD = 0;
>>> Then it's no longer a null pointer constant, so can't be misused in
>>> contexts where a pointer is required, will be treated as an integral
>>> value by templates, &etc.
>>
>> my first try was (in part2):
>>
>> struct null_word_t {
>>   operator intptr_t()  const { return 0; }
>> };
>>
>> which IMO is nice as it:
>> 1) mirrors my (initial) implementation of null_t
>> 2) can be template specialized against
>> 3) is defined in one place
>>
>> Unfortunately we do not yet have constexpr.
>>
>> So my (temporary) fix was (in part3):
>> const intptr_t NULL_WORD = 0;
>>
>> This is /identical/ to what you propose. I think this is a good compromise. However after much effort from my side trying to get this solution you did not want my changes to NULL (and maybe I misunderstood you regarding NULL_WORD) so I reverted that in part5.
>>
>> I am very much confused to what you want. Should I once again revert NULL_WORD and put it in globalDefinitions.hpp?
> 
> Sorry, but I haven't looked at all the different stages of your
> changes.  I've only looked carefully through 8240110/1, to most recent
> total combined change.
> 
> And it seems the discussion of NULL and the discussion of NULL_WORD
> have gotten confused and mixed up.  So I'll be explicit here.
> 
> I think / agree the following definition of NULL_WORD in
> globalDefinitions.hpp is fine:

Then I will add it in a later webrev...
> 
> const intptr_t NULL_WORD = 0;
> // The Standard only guarantees intptr_t is at least big enough to
> // hold a pointer value, and purportedly there are platforms where
> // this will fail.  If this ever trips we'll need to think about
> // whether it really matters and what to do if it does.
> STATIC_ASSERT(sizeof(NULL_WORD) == sizeof(void*));
> 
> (The comment for the static assert is only intended to be suggestive
> of wording; feel free to reword to suit.)

Better than my original wording so I will use yours.

> 
> I agree there are some places that are using NULL that should be using
> NULL_WORD, and vice versa.
> 
> I wonder if it would be worthwhile splitting out the NULL_WORD
> defintion and the NULL <-> NULL_WORD changes as a separate smaller
> change that is (I think) easily dealt with. It will then no longer
> clutter the remainder.

I did present it as part1 and part2 with exactly that separation to be 
easy to review. Or do you mean actually pushing as two RFRs? I could do 
that, but I wanted them to be reviewed together as they share a lot, 
that was why I bundled them into one review. Do you want two pushes?

> 
>>> I wouldn't worry about this comment in globalDefinitions_solstudio.hpp:
>>>    "On some platforms, sizeof(intptr_t) > sizeof(void*)"
>>> or you can be really paranoid and STATIC_ASSERT that they are the same
>>> size. Though I really wonder what platform(s) that's talking about.
>>> That comment comes from 6452081: 3/4 Allow for Linux builds with Sun
>>> Studio Linux compilers.  So maybe 32bit Solaris Studio Linux is weird?
>>
>> I did this originally (disregard the typo of uint vs int) in part1.
>>
>> STATIC_ASSERT(sizeof(uintptr_t) == sizeof(char *)); // uintptr_t is only guaranteed to be at /least/ as big as a pointer
>>
>> I later reverted this change as I thought you dismissed all of my NULL and NULL_WORD definitions. I really wanted this.
> 
> See above.

I will use yours.

> 
>>> src/hotspot/cpu/sparc/c1_CodeStubs_sparc.cpp
>>> 303     AddressLiteral addrlit(NULL_WORD, metadata_Relocation::spec(_index));
>>> 318     AddressLiteral addrlit(NULL_WORD, oop_Relocation::spec(_index));
>>> I think these should be left as NULL rather than being changed to
>>> NULL_WORD.  The expected type is the address type, and other types are
>>> converted to that via casts in constructor overloads.
>>> Similarly here:
>>> src/hotspot/cpu/sparc/c1_LIRAssembler_sparc.cpp
>>>   426   AddressLiteral addrlit(NULL_WORD, oop_Relocation::spec(oop_index));
>>>   445   AddressLiteral addrlit(NULL_WORD, metadata_Relocation::spec(index));
>>
>> I believe those do not at all auto convert, but promote to (integral) zero as I (quite thoroughly) described in my reply to David (and thus /should/ be NULL_WORD. I believe this is one prime example when our NULL definitions results in unintuitive selection of integer constructor.
> 
> I'd failed to notice that this one is in Solaris/sparc code, where the
> gcc rules don't apply.
> 
> I think the expectation here is that NULL would select the pointer
> overload, but it doesn't on this platform because NULL is a literal
> 0/0L, and as you say, the integer overload is better.
> 
> Similar code compiled with gcc would also select the integer overload,
> but with a -Wconversion-null warning.  Apparently gcc's __null, while
> fairly similar to nullptr, does support conversion to an integer type
> but with a warning.  And that's intentional; quoting from
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=35669#c13
> 
>    0 is still a valid definition of NULL so conversion from NULL to int
>    is OK. And __null does not have type nullptr_t, changing it to have
>    that type would break a lot of code
> 
> nullptr would do the right thing here.
> 
> But note that this code is slated to go away real soon (JEP 381).
> 
> My inclination would be to not make this change, in case there is a
> repo of changes for that removal that will have a merge conflict if
> this change were to go in first.  But I no longer care all that much.

As both you and David questioned the validity of the fix, I think the 
code is misleading to say the least and thus the fix is important. The 
rebase conflict on a removal of a file can not be /that/ hard to resolve.
> 
>>> src/hotspot/os/linux/cgroupV1Subsystem_linux.hpp
>>> [old code]
>>> 100     CachingCgroupController* _memory = NULL;
>>> 101     CgroupV1Controller* _cpuset = NULL;
>>> 102     CachingCgroupController* _cpu = NULL;
>>> 103     CgroupV1Controller* _cpuacct = NULL;
>>> How does this even compile?  Non-static member initializers are a
>>> C++11 feature.  Weirdly, g++ with -std=gnu++98 warns about this for
>>> some types/initializers but not for others.
>>
>> As I remarked in my original mail, I believe it to be a gcc bug, but I am not sure.
> 
> Agreed that it?s a gcc bug / weirdness.  Deleting that stuff is fine with me.

Good.
> 
>>> src/hotspot/os/solaris/attachListener_solaris.cpp
>>> 461   status = ::sema_init(&_wakeup, 0, NULL, NULL);
>>> =>
>>> 461   status = ::sema_init(&_wakeup, 0, 0, NULL);
>>> The old code appears to have been cribbed from the documentation:
>>> https://docs.oracle.com/cd/E36784_01/html/E36874/sema-init-3c.html
>>> See the example of "Default semaphore initialization (intra-process)".
>>> Rather than a literal 0, better would be USYNC_THREAD.  Presumably the
>>> value of USYNC_THREAD is 0.
>>
>> I could not easily find the value of the symbols, so I only changed it to the correct type, with resent changes to Java regarding Solaris do you care enough for me to check this out, or is 0 good enough?
> 
> This is another bit of Solaris code that is slated to go away real
> soon now.  So I don't care all that much.

I will keep my change then.
> 
>>> src/hotspot/os/windows/os_windows.cpp
>>> 1659   if (!GetFileVersionInfo(kernel32_path, 0, version_size, version_info)) {
>>> Given that the second argument, which was changed from NULL to 0, is a
>>> "handle" (though unused), I think using NULL was appropriate and this
>>> change shouldn't be made.
>>
>> I have no windows machine, so I can not check this out 100% at the moment, but according to the compile error I got, and the documentation of GetFileVersionInfoA or GetFileVersionInfoW I find on the net:
>> https://docs.microsoft.com/en-us/windows/win32/api/winver/nf-winver-getfileversioninfoa
>>
>> The type of dwHandle is DWORD. Defined as:
>> https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-dtyp/262627d8-3418-4627-9218-4ffe110850b2
>>
>> What makes you think the handle is of pointer type, to me it looks like a long?
> 
> A Windows "handle" is generally a pointer, so NULL makes sense
> semantically, and that's presumably what the original author thought
> too. OTOH, the examples of calling that function that I found via a
> web search just now all seem to pass literal 0. So I'm okay with the
> change after all.

Good.

> 
>>> src/hotspot/share/classfile/javaClasses.cpp
>>>   851       k->set_java_mirror_handle(NULL);
>>> =>
>>>   851       k->set_java_mirror_handle(OopHandle(NULL));
>>> PATTERN CHANGE
>>> Similarly here:
>>> src/hotspot/share/classfile/moduleEntry.hpp
>>>   82     _module = OopHandle(NULL);
>>>   84     _pd = OopHandle(NULL); >
>>> And here:
>>> src/hotspot/share/classfile/modules.cpp
>>> 156     version_symbol = TempNewSymbol(NULL); > 161   TempNewSymbol location_symbol(NULL);
>>> 368     version_symbol = TempNewSymbol(NULL);
>>> 373   TempNewSymbol location_symbol(NULL);
>>> Although I'm not sure why the assignments on line 156 and line 368 is
>>> even being done, since each version_symbol was already default
>>> initialized.
>> Probably to /show/ that it has an initialized value (I do not necessarily agree that it is good). Do you want me to change any of those, and if so to what?
> 
> Unfinished.
> 
>>> src/hotspot/share/code/codeHeapState.cpp
>>>   140     if (((_termString) != 0) && (strlen(_termString) > 0)){   \
>>>   154     if (((_termString) != 0) && (strlen(_termString) > 0)){   \
>>> I think these changes to compare _termstring with 0 rather than NULL
>>> should not be made.  _termString is supposed to be a `[const] char*`,
>>> so comparing to NULL is preferable to comparing to 0.
>>
>> _termString is not a pointer (or at least not most of the time), see my thorough discussion of the topic with David. I believe my solution is the next safest one (after rewriting that macro nest which I am not going to do now).
> 
> Unfinished.
> 
>>> Though at a brief skim it looks like _termString is always provided
>>> and never NULL, so it might be better to assert that _termString is
>>> non-NULL.  But that's a completely different change.
>>
>> I believe we should use less macros, and I agree that is different change.
> 
> Unfinished.
> 
>>> src/hotspot/share/gc/parallel/psAdaptiveSizePolicy.cpp
>>> src/hotspot/share/gc/parallel/psParallelCompact.cpp
>>> src/hotspot/share/gc/shared/memset_with_concurrent_readers.hpp
>>> src/hotspot/share/gc/z/zErrno.cpp
>>> These #include changes seem unrelated to NULL.
>>
>> They are all (unnecessary) including standard headers, It is hard to know which headers that (indirectly) redefines NULL. It is better to not include them if you do not have to. If I remember correctly I removed all but one standard library include in the "gc" folder. The one I left was for <new>.
> 
> Regarding knowing which standard headers might redefine NULL, we
> should not care. We should be using the native NULL and don't care who
> might define it.

So, there are reasons for using native NULL and reasons for using 
typesafe NULL, but I can see no good reason for not knowing which is 
which, and that is the case now.

My stance is that if we should use native NULL, we should remove our 
definitions of NULL. I feel that would be a loss (only) because I can 
see few reviewers accepting my patch replacing /all/ NULLs with a 
typesafe alternative and in the process updating almost every source 
file (although this is /actually/ what I would prefer).

I want strongly typed NULL, nullptr or something with another name that 
does not implicitly convert to non pointer types. I want them 
/everywhere/. I want to be able to template specialize against them. How 
do I do that?

Or do you suggest that we should continue to live in this NULL-limbo 
even after the new c++? Maybe then have a forest of nulls:

NULL that is native,
NULL that is defined by us (never knowing which is which),
const_raw_null
raw_null
NULL_WORD,
0
and nullptr (or replacement)?

And I guess a handful of ones I do not yet know about.

In addition maybe never being able to unify them gradually "because it 
breaks the style of the file"? Always being locked by either too big 
changes or too small ones that breaks the local style.

> 
> For zErrno.cpp, the #include of <string.h> seems unnecessary, since it
> is using os::strerror.  But I think it is appropriate to #include
> <errno.h> here, rather than relying on it being implicitly included by
> some dependency that could change, breaking this unrelated code.
> 
> For memset_with_concurrent_readers.hpp, I'm not a big fan of
> globalDefinitions.hpp (I really dislike big omnibus random collections
> of stuff headers).  But okay.  I didn't re-examine the others.
>
globalDefinition.hpp add basically all native includes. It is a pattern 
even if we might or might not like it. It has a value in that if we use 
it we can have our own, improved NULL. It is used almost everywhere in 
the gc folder for example.

>>> src/hotspot/share/jfr/jni/jfrGetAllEventClasses.cpp
>>> 172   assert(add_method_signature != NULL, "invariant");
>>> =>
>>> 172   assert(add_method_sig_sym != NULL, "invariant");
>>> This seems to be an unrelated bug fix; the wrong value was being
>>> compared to NULL.
>>
>> It is highly related as it did not compile on my original definition of NULL and was thus found.
> 
> The change is fine, but I would have filed and fixed the bug rather than
> burying it in this large change.  Much better for archeology.
> 
Good, I will keep it.

>>> src/hotspot/share/jfr/jni/jfrJavaCall.cpp
>>>   57   assert(_storage != NULL, "invariant");
>>> =>
>>>   57   assert(_storage_index < (int) ARRAY_SIZE(_storage), "invariant");
>>> _storage is an array, so the original assert is definitionally true,
>>> but it's harmless to verify, in case future maintenance makes it an
>>> allocated array rather than inline.
>>
>> It is so statically true that it should not be a runtime check.
>>
>> My replacement check is not only implicitly checking that the variable is not NULL, it does so at compile time (ARRAY_SIZE will fail on NULL) and it does an even better runtime check that actually adds value.
> 
> Unfinished.
> 
>>> The size of _storage is definitionally SIZE, and _storage_index is
>>> checked against SIZE a couple lines later, making the proposed
>>> replacement just a duplicative check.
>>
>> I have no problem with duplicate checks, but if anything should be removed, it is the SIZE assert. My assert clearly shows that the access is okay even though someone would change the size of the array to not be of size SIZE. My assert can be understood without looking outside the method.
> 
> Unfinished.
> 
>>> So I think the proposed change should not be made.
>>> Similarly at lines 65, 73, and 91.
>> I strongly disagree. Each of my replacement asserts is strictly superior if I am not missing something. Not only that, but they might not compile on windows in the future depending on how we define NULL in the future.
>>
>> Most of the changes I have made are because of compile errors after my new NULL, not "just" because of cosmetic issues.
> 
> Unfinished.
> 
>>> src/hotspot/share/jvmci/jvmciCompilerToVM.cpp
>>> 1562   JVMCIENV->set_HotSpotStackFrameReference_localIsVirtual(hs_frame, 0);
>>> This change to use literal 0 rather than NULL as a null pointer
>>> constant does not seem like an improvement to me.  I think this change
>>> should not be made.
>>> ------------------------------------------------------------------------------
>>> src/hotspot/share/jvmci/jvmciCompilerToVM.cpp
>>> I looked at the changes from JVMCI_CHECK_0 to JVMCI_CHECK_NULL and
>>> think they are correct, assuming a suitable definition of the latter.
>>> (Though see below regarding that.)
>>> ------------------------------------------------------------------------------
>>> src/hotspot/share/jvmci/jvmciCompilerToVM.cpp
>>> 2572   JVMCIObjectArray current_array = NULL;
>>> =>
>>> 2572   JVMCIObjectArray current_array(NULL);
>>> PATTERN CHANGE
>>> ------------------------------------------------------------------------------
>>> src/hotspot/share/jvmci/jvmciCompilerToVMInit.cpp
>>> 267     if ((value) != 0) { \
>>> Value is a char*; I don't think this change should be made.
>>> ------------------------------------------------------------------------------
>>> src/hotspot/share/jvmci/jvmciExceptions.hpp
>>> 50 #define JVMCI_CHECK_NULL                               JVMCI_CHECK_(NULL)
>>> =>
>>> 50 #define JVMCI_CHECK_NULL                               JVMCI_CHECK_(0)
>>> For platforms that warn about converting NULL to an integral value
>>> this is a type safety regression. I think this change should not be
>>> made.
>>> ------------------------------------------------------------------------------
>>> src/hotspot/share/jvmci/jvmciExceptions.hpp
>>> 49 #define JVMCI_CHECK_0                                  JVMCI_CHECK_(0)
>>> =>
>>> 49 #define JVMCI_CHECK_0                                  JVMCI_CHECK_(NULL_WORD)
>>> I kind of think this is an abuse of NULL_WORD. It also serves no real
>>> purpose with the current definitions of NULL_WORD (though would with
>>> the suggestion far above). If the goal is to have something that isn't
>>> a null pointer constant so it can't be used in inappropriate
>>> contexts (which is a reasonable goal), then replace "0" with "int(0)"
>>> (and comment appropriately).
>>
>> Please read my conversation with David and see if you after that still have any issue with what I am proposing.
> 
> Unfinished.
> 
>>> src/hotspot/share/oops/access.hpp
>>> 296 class ArrayAccess ...
>>> How about adding the following private functions to this class, and
>>> use them in the obvious places:
>>>    template<typename T> static const T* const_raw_null() { return NULL; }
>>>    template<typename T> static       T*       raw_null() { return NULL; }
>>
>> I am lacking c++ and template skills, but the problem here is that we need to make the type of the null pointer concrete for the template system. I fail to see how these template methods achieve this from static context and no parameters. The code fails to compile for me, and I am probably missing something, but I fail to see the idea behind you construction. The only type parameter is the return one, and that is the one we do not know, right?
>>
>> As another solution we could specialize the type explicitly at the call-site instead of in the delivered parameters (I tried that first), but that was IMO /much/ harder to read.
>>
>> Even if raw_null() does work (assuming failure from my side), the original is cleaner IMO.
> 
> I guess my suggestion left too much as an exercise for the reader.  Here?s the
> change I was contemplating, which seems to work fine:

Yes. With your solution you not only add two methods templates, but also 
explicitly pass the type to them. Is that not one step more than just 
explicitly passing the type to the arraycopy call? That is:

AccessT::template arraycopy<T>(src_obj, src_offset_in_bytes, NULL,
                                dst_obj, dst_offset_in_bytes, NULL,
                                length);

I think that is better than than the raw_null methods, and I think my 
proposed T* null = NULL; is yet easier to read. Now, when you have seen 
all three, what do you think?

Thanks,
Leo


> 
> diff -r 1fa1ec0e7048 -r 08ee68a3d2a4 src/hotspot/share/oops/access.hpp
> --- a/src/hotspot/share/oops/access.hpp Sat Apr 18 20:19:45 2020 +0200
> +++ b/src/hotspot/share/oops/access.hpp Sun Apr 19 21:08:43 2020 -0400
> @@ -295,13 +295,17 @@
>   template <DecoratorSet decorators = DECORATORS_NONE>
>   class ArrayAccess: public HeapAccess<IS_ARRAY | decorators> {
>     typedef HeapAccess<IS_ARRAY | decorators> AccessT;
> +
> +  template<typename T> static const T* const_raw_null() { return NULL; }
> +  template<typename T> static       T*       raw_null() { return NULL; }
> +
>   public:
>     template <typename T>
>     static inline void arraycopy(arrayOop src_obj, size_t src_offset_in_bytes,
>                                  arrayOop dst_obj, size_t dst_offset_in_bytes,
>                                  size_t length) {
> -    AccessT::arraycopy(src_obj, src_offset_in_bytes, reinterpret_cast<const T*>(NULL),
> -                       dst_obj, dst_offset_in_bytes, reinterpret_cast<T*>(NULL),
> +    AccessT::arraycopy(src_obj, src_offset_in_bytes, const_raw_null<T>(),
> +                       dst_obj, dst_offset_in_bytes, raw_null<T>(),
>                          length);
>     }
>   
> @@ -309,7 +313,7 @@
>     static inline void arraycopy_to_native(arrayOop src_obj, size_t src_offset_in_bytes,
>                                            T* dst,
>                                            size_t length) {
> -    AccessT::arraycopy(src_obj, src_offset_in_bytes, reinterpret_cast<const T*>(NULL),
> +    AccessT::arraycopy(src_obj, src_offset_in_bytes, const_raw_null<T>(),
>                          NULL, 0, dst,
>                          length);
>     }
> @@ -319,15 +323,15 @@
>                                              arrayOop dst_obj, size_t dst_offset_in_bytes,
>                                              size_t length) {
>       AccessT::arraycopy(NULL, 0, src,
> -                       dst_obj, dst_offset_in_bytes, reinterpret_cast<T*>(NULL),
> +                       dst_obj, dst_offset_in_bytes, raw_null<T>(),
>                          length);
>     }
>   
>     static inline bool oop_arraycopy(arrayOop src_obj, size_t src_offset_in_bytes,
>                                      arrayOop dst_obj, size_t dst_offset_in_bytes,
>                                      size_t length) {
> -    return AccessT::oop_arraycopy(src_obj, src_offset_in_bytes, reinterpret_cast<const HeapWord*>(NULL),
> -                                  dst_obj, dst_offset_in_bytes, reinterpret_cast<HeapWord*>(NULL),
> +    return AccessT::oop_arraycopy(src_obj, src_offset_in_bytes, const_raw_null<HeapWord>(),
> +                                  dst_obj, dst_offset_in_bytes, raw_null<HeapWord>(),
>                                     length);
>     }
>   
>>> src/hotspot/share/oops/constantPool.cpp
>>>   270     set_resolved_references(OopHandle(NULL));
>>>   302     set_resolved_references(OopHandle(NULL));
>>>   376     set_resolved_references(OopHandle(NULL));
>>> PATTERN CHANGE
>>> Similarly here:
>>> src/hotspot/share/oops/cpCache.cpp
>>> 766   set_resolved_references(OopHandle(NULL));
>>> src/hotspot/share/oops/klass.cpp
>>> 549   _java_mirror = OopHandle(NULL);
>>> src/hotspot/share/prims/jni.cpp
>>>   319   TempNewSymbol class_name(NULL);
>>> src/hotspot/share/prims/jvm.cpp
>>>   954   TempNewSymbol class_name(NULL);
>>> src/hotspot/share/prims/methodHandles.cpp
>>> 1311   TempNewSymbol name(NULL);
>>> 1312   TempNewSymbol sig(NULL);
>>
>> I understand you have no problem with those changes, is that correct?
> 
> Not correct.  See ?PATTERN CHANGE?.  Unfinished.
> 

From glaubitz at physik.fu-berlin.de  Tue Apr 28 10:41:42 2020
From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz)
Date: Tue, 28 Apr 2020 12:41:42 +0200
Subject: RFR: 8242244: Remove redundant ELF machine definitions
In-Reply-To: <8b5812dd-78bf-f099-f066-b0bac2cf1572@bell-sw.com>
References: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>
 <22fdf007-a3d5-f14f-4d27-5746ad21a88e@oracle.com>
 <483038a4-7517-e190-cbbe-783da5f68ecc@physik.fu-berlin.de>
 <8b5812dd-78bf-f099-f066-b0bac2cf1572@bell-sw.com>
Message-ID: <538aabc2-158d-c8ce-08ef-0ae3c62ffc78@physik.fu-berlin.de>

Hi Dmitry!

On 4/28/20 12:33 PM, Dmitry Chuyko wrote:
> * For target hotspot_variant-server_libjvm_objs_os_linux.o:
> jdk/src/hotspot/os/linux/os_linux.cpp: In static member function 'static void* os::dll_load(const char*, char*, int)':
> jdk/src/hotspot/os/linux/os_linux.cpp:1854:6: error: 'EM_486' was not declared in this scope
> ???? {EM_486,???????? EM_386,???? ELFCLASS32, ELFDATA2LSB, (char*)"IA 32"},
> ????? ^~~~~~
> jdk/src/hotspot/os/linux/os_linux.cpp:1854:6: note: suggested alternative: 'EM_386'
> ???? {EM_486,???????? EM_386,???? ELFCLASS32, ELFDATA2LSB, (char*)"IA 32"},
> ????? ^~~~~~
> ????? EM_386
> 
> It happens on CenOS 8 and Ubuntu 20.04 (kernel headers installed), aarch64 and amd64.
> 
> Is any additional configuration expected?

Ao already found a potential fix:

diff -r 7205265660b2 src/hotspot/os/linux/os_linux.cpp
--- a/src/hotspot/os/linux/os_linux.cpp Tue Apr 28 05:53:19 2020 -0400
+++ b/src/hotspot/os/linux/os_linux.cpp Tue Apr 28 18:24:45 2020 +0800
@@ -110,6 +110,7 @@
 # include <stdint.h>
 # include <inttypes.h>
 # include <sys/ioctl.h>
+# include <linux/elf-em.h>

 #ifndef _GNU_SOURCE
   #define _GNU_SOURCE

Could you try whether this addresses your problem?

Thanks,
Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz at debian.org
`. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

From aoqi at loongson.cn  Tue Apr 28 10:44:30 2020
From: aoqi at loongson.cn (Ao Qi)
Date: Tue, 28 Apr 2020 18:44:30 +0800
Subject: RFR: 8242244: Remove redundant ELF machine definitions
In-Reply-To: <8b5812dd-78bf-f099-f066-b0bac2cf1572@bell-sw.com>
References: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>
 <22fdf007-a3d5-f14f-4d27-5746ad21a88e@oracle.com>
 <483038a4-7517-e190-cbbe-783da5f68ecc@physik.fu-berlin.de>
 <8b5812dd-78bf-f099-f066-b0bac2cf1572@bell-sw.com>
Message-ID: <CALjzQn7mbw2-SpX=aBx967tCtAKH00uj58bMc0KrnOP-B+LZMg@mail.gmail.com>

I filed a bug: https://bugs.openjdk.java.net/browse/JDK-8243997

On Tue, Apr 28, 2020 at 6:33 PM Dmitry Chuyko <dmitry.chuyko at bell-sw.com> wrote:
>
> Hello,
>
> Sorry for being late.
>
> I have following error during the build of 15 tip:
>
> * For target hotspot_variant-server_libjvm_objs_os_linux.o:
> jdk/src/hotspot/os/linux/os_linux.cpp: In static member function 'static
> void* os::dll_load(const char*, char*, int)':
> jdk/src/hotspot/os/linux/os_linux.cpp:1854:6: error: 'EM_486' was not
> declared in this scope
>       {EM_486,         EM_386,     ELFCLASS32, ELFDATA2LSB, (char*)"IA 32"},
>        ^~~~~~
> jdk/src/hotspot/os/linux/os_linux.cpp:1854:6: note: suggested
> alternative: 'EM_386'
>       {EM_486,         EM_386,     ELFCLASS32, ELFDATA2LSB, (char*)"IA 32"},
>        ^~~~~~
>        EM_386
>
> It happens on CenOS 8 and Ubuntu 20.04 (kernel headers installed),
> aarch64 and amd64.
>
> Is any additional configuration expected?
>
> -Dmitry
>
> On 4/28/20 2:48 AM, John Paul Adrian Glaubitz wrote:
> > Hi David!
> >
> > On 4/28/20 12:53 AM, David Holmes wrote:
> >> Apologies as I just spotted this in my mail backlog.
> > No worries ;).
> >
> >> This seems fine and trivial.
> > Thanks. I post a rebased version tomorrow - necessary because I added
> > the definition for EM_RISCV to make sure we don't run into build issues
> > with slightly older kernels has EM_RISCV hasn't been defined in the
> > kernel for too long yet.
> >
> > Adrian
> >


From aph at redhat.com  Tue Apr 28 10:47:02 2020
From: aph at redhat.com (Andrew Haley)
Date: Tue, 28 Apr 2020 11:47:02 +0100
Subject: [aarch64-port-dev ] FW: [Mach5]
 mach5-one-phh-JDK-8151779-20200427-2151-10554367: FAILED
In-Reply-To: <858sifgbt3.fsf@arm.com>
References: <C769AA2F-55A8-4986-9EEE-0AD93FB93E00@amazon.com>
 <E6D13CFA-A9E6-4262-8D33-2FB4412413B6@amazon.com> <858sifgbt3.fsf@arm.com>
Message-ID: <c13feff5-31e5-10fa-4915-df74c170a2e8@redhat.com>

On 4/28/20 11:11 AM, Nick Gasson wrote:
>>
>> My nightly buildbot started failing recently on aarch64.
>> One issue is the error message of [cds] prevents configure from bootjdk determining.
>> https://hg.openjdk.java.net/jdk/jdk/file/1b8f9e72b22b/make/autoconf/boot-jdk.m4#l77
>>
>> ./bin/java --version
>> [0.006s][error][cds] Unable to map CDS archive -- os::vm_allocation_granularity() expected: 65536 actual: 4096
>> openjdk 14.0.1 2020-04-14
>> OpenJDK Runtime Environment AdoptOpenJDK (build 14.0.1+7)
>> OpenJDK 64-Bit Server VM AdoptOpenJDK (build 14.0.1+7, mixed mode)
>>
>> Are you aware of this issue? I can?t see [cds] line on my linux/x86_64 host.
>> Maybe it?s aarch64-only.  CC aarch-port-dev.
> 
> Was your boot JDK built on a machine configured with a different page
> size to your current machine? Looks like the CDS archive was dumped on a
> machine with 64k pages but you're running with 4k pages. There's a JBS
> issue for this:
> 
> https://bugs.openjdk.java.net/browse/JDK-8236847

The thread seems to have died here:

http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-February/038207.html

We really need to get this fixed. If anyone reading this has machines with
both 4k and 64k pages, please do the experiment and we'll make a suitable
patch. Everything here has 64k pages.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From glaubitz at physik.fu-berlin.de  Tue Apr 28 10:52:15 2020
From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz)
Date: Tue, 28 Apr 2020 12:52:15 +0200
Subject: RFR: 8242244: Remove redundant ELF machine definitions
In-Reply-To: <8b5812dd-78bf-f099-f066-b0bac2cf1572@bell-sw.com>
References: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>
 <22fdf007-a3d5-f14f-4d27-5746ad21a88e@oracle.com>
 <483038a4-7517-e190-cbbe-783da5f68ecc@physik.fu-berlin.de>
 <8b5812dd-78bf-f099-f066-b0bac2cf1572@bell-sw.com>
Message-ID: <3e17af2e-5cd6-7bee-5147-89093d248657@physik.fu-berlin.de>

Hi!

On 4/28/20 12:33 PM, Dmitry Chuyko wrote:
> I have following error during the build of 15 tip:
> 
> * For target hotspot_variant-server_libjvm_objs_os_linux.o:
> jdk/src/hotspot/os/linux/os_linux.cpp: In static member function 'static void* os::dll_load(const char*, char*, int)':
> jdk/src/hotspot/os/linux/os_linux.cpp:1854:6: error: 'EM_486' was not declared in this scope
> ???? {EM_486,???????? EM_386,???? ELFCLASS32, ELFDATA2LSB, (char*)"IA 32"},
> ????? ^~~~~~
> jdk/src/hotspot/os/linux/os_linux.cpp:1854:6: note: suggested alternative: 'EM_386'
> ???? {EM_486,???????? EM_386,???? ELFCLASS32, ELFDATA2LSB, (char*)"IA 32"},
> ????? ^~~~~~
> ????? EM_386
> 
> It happens on CenOS 8 and Ubuntu 20.04 (kernel headers installed), aarch64 and amd64.
I have opened a bug for the issue [1].

Will publish the fix suggested by Ao Qi once you can confirm it fixes the problem
for you.

Adrian

> [1] https://bugs.openjdk.java.net/browse/JDK-8243998

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz at debian.org
`. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

From matthias.baesken at sap.com  Tue Apr 28 10:52:34 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 28 Apr 2020 10:52:34 +0000
Subject: RFR: 8243389: enhance os::pd_print_cpu_info on linux
In-Reply-To: <AM4PR02MB30579EBA32B978AC69A3F8DD9AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>
References: <AM0PR02MB541225ED5513768AED6732F093AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
 <AM4PR02MB30579EBA32B978AC69A3F8DD9AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>
Message-ID: <AM0PR02MB54121E0BA017A19057FAF78193AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>

> "Maximum cpu frequency" and "Minimum cpu frequency:" seems to be redundant when we have "Available cpu frequencies:".

Hi Martin, I added  all those intentionally because different  kernels/setups might report one but not the other.

> Please consider using ExtensiveErrorReports for some information like e.g. cache related info.

Good point, we should take this into consideration.
Maybe some other opinions ? I would then guard the cache and frequency related info with the flag (or would you like to keep the frequency related info by default) ?


Best regards, Matthias

-----Original Message-----
From: Doerr, Martin <martin.doerr at sap.com> 
Sent: Dienstag, 28. April 2020 12:27
To: Baesken, Matthias <matthias.baesken at sap.com>; 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
Subject: RE: RFR: 8243389: enhance os::pd_print_cpu_info on linux

Hi Matthias,

technically, this looks good to me.

"Maximum cpu frequency" and "Minimum cpu frequency:" seems to be redundant when we have "Available cpu frequencies:".

I think it exceeds what a typical supporter needs to know in order to analyze issues.
Please consider using ExtensiveErrorReports for some information like e.g. cache related info.

Best regards,
Martin


> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
> Baesken, Matthias
> Sent: Dienstag, 28. April 2020 11:45
> To: 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: RE: RFR: 8243389: enhance os::pd_print_cpu_info on linux
> 
> Ping ... any reviews please?
> 
> Thanks, Matthias
> 
> -------------------------------
> Hello, please review the following enhancement of  os::pd_print_cpu_info
> on linux .
> It adds  information about
> cache
> online/offline cpus
> frequency details
> governors
> 
> by looking at the info provided at /sys/devices/system/cpu  .
> 
> .....
> bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8243389
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8243389.0/


From glaubitz at physik.fu-berlin.de  Tue Apr 28 10:54:57 2020
From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz)
Date: Tue, 28 Apr 2020 12:54:57 +0200
Subject: RFR: 8242244: Remove redundant ELF machine definitions
In-Reply-To: <CALjzQn7mbw2-SpX=aBx967tCtAKH00uj58bMc0KrnOP-B+LZMg@mail.gmail.com>
References: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>
 <22fdf007-a3d5-f14f-4d27-5746ad21a88e@oracle.com>
 <483038a4-7517-e190-cbbe-783da5f68ecc@physik.fu-berlin.de>
 <8b5812dd-78bf-f099-f066-b0bac2cf1572@bell-sw.com>
 <CALjzQn7mbw2-SpX=aBx967tCtAKH00uj58bMc0KrnOP-B+LZMg@mail.gmail.com>
Message-ID: <870c10b4-61ef-5183-f8bc-db6a3750e45c@physik.fu-berlin.de>

On 4/28/20 12:44 PM, Ao Qi wrote:
> I filed a bug: https://bugs.openjdk.java.net/browse/JDK-8243997
Shall I take ownership or do you want to fix it yourself?

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz at debian.org
`. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

From aoqi at loongson.cn  Tue Apr 28 11:02:57 2020
From: aoqi at loongson.cn (Ao Qi)
Date: Tue, 28 Apr 2020 19:02:57 +0800
Subject: RFR: 8242244: Remove redundant ELF machine definitions
In-Reply-To: <870c10b4-61ef-5183-f8bc-db6a3750e45c@physik.fu-berlin.de>
References: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>
 <22fdf007-a3d5-f14f-4d27-5746ad21a88e@oracle.com>
 <483038a4-7517-e190-cbbe-783da5f68ecc@physik.fu-berlin.de>
 <8b5812dd-78bf-f099-f066-b0bac2cf1572@bell-sw.com>
 <CALjzQn7mbw2-SpX=aBx967tCtAKH00uj58bMc0KrnOP-B+LZMg@mail.gmail.com>
 <870c10b4-61ef-5183-f8bc-db6a3750e45c@physik.fu-berlin.de>
Message-ID: <CALjzQn5tW0LYBVXQ3h3f9cvhWh66oeCngFWjkk-wsLEY7w59Kw@mail.gmail.com>

On Tue, Apr 28, 2020 at 6:54 PM John Paul Adrian Glaubitz
<glaubitz at physik.fu-berlin.de> wrote:
>
> On 4/28/20 12:44 PM, Ao Qi wrote:
> > I filed a bug: https://bugs.openjdk.java.net/browse/JDK-8243997
> Shall I take ownership or do you want to fix it yourself?

I am testing the patch on Linux. Seems fine so far. But I don't have a
solaris machine, I am not sure if solaris build is ok. If you want,
feel free to take it:)

Thanks,
Ao Qi

>
> Adrian
>
> --
>  .''`.  John Paul Adrian Glaubitz
> : :' :  Debian Developer - glaubitz at debian.org
> `. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
>   `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913


From palacsint at palacsint.hu  Tue Apr 28 11:04:44 2020
From: palacsint at palacsint.hu (=?UTF-8?B?S2FyYWvDsyBNaWtsw7Nz?=)
Date: Tue, 28 Apr 2020 13:04:44 +0200
Subject: Thread-Local Handshakes slowdown on single-cpu machines
In-Reply-To: <66059982-49bf-1a15-39fb-22008b069573@oracle.com>
References: <5e65ba58-b599-e83c-b4ea-3d766ed3c457@palacsint.hu>
 <66059982-49bf-1a15-39fb-22008b069573@oracle.com>
Message-ID: <7216aa59-ad6f-dd3c-4f37-4723d3b7e9a7@palacsint.hu>

Hello David,

I've tried with OpenJDK 15 EA Build 20. Unfortunately it's slow as with 
OpenJDK14.
A colleague of mine set CPU affinity with taskset to only one CPU on 
Linux and did not experience the slowdown. So the issue seems 
Windows-related.

Miklos

On 4/28/20 1:02 AM, David Holmes wrote:
> Hi Miklos,
>
> On 28/04/2020 7:26 am, Karak? Mikl?s wrote:
>> Hello,
>>
>> We bumped into a possible Thread-Local Handshakes issue with multiple 
>> apps. It seems that enabled TLH slows down applications running on 
>> (although rare) single CPU boxes. I would be grateful if you could 
>> confirm that this is a known trade-off for these rare setups or a 
>> possible JVM bug. That would save us at least a few hours of debugging.
>>
>> TL;DR: Both tested apps stop frequently around "HandshakeOneThread", 
>> "HandshakeAllThreads" and "Revoked bias of currently-unlocked object" 
>> running with OpenJDK12/OpenJDK13/OpenJDK14. OpenJDK 13 with 
>> -XX:-ThreadLocalHandshakes fixes the issue. Adding a second CPU to 
>> the virtual machine fixes the issue. Enabling hyper-threading fixes 
>> the issue.
>
> Can you try with latest JDK 15 build, just to see if this may be 
> something already addressed?
>
> Thanks,
> David
>
>> More details are available at StackOverflow: 
>> https://stackoverflow.com/questions/61375565/slow-application-frequent-jvm-hangs-with-single-cpu-setups-and-java-12 
>>
>>
>> All thoughts are welcome.
>>
>> Best,
>> Miklos
>>


From magnus.ihse.bursie at oracle.com  Tue Apr 28 11:04:47 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Tue, 28 Apr 2020 13:04:47 +0200
Subject: RFR: 8242244: Remove redundant ELF machine definitions
In-Reply-To: <3e17af2e-5cd6-7bee-5147-89093d248657@physik.fu-berlin.de>
References: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>
 <22fdf007-a3d5-f14f-4d27-5746ad21a88e@oracle.com>
 <483038a4-7517-e190-cbbe-783da5f68ecc@physik.fu-berlin.de>
 <8b5812dd-78bf-f099-f066-b0bac2cf1572@bell-sw.com>
 <3e17af2e-5cd6-7bee-5147-89093d248657@physik.fu-berlin.de>
Message-ID: <cc679014-7637-7667-3e75-c45f4f5d6d02@oracle.com>

On 2020-04-28 12:52, John Paul Adrian Glaubitz wrote:
> Hi!
>
> On 4/28/20 12:33 PM, Dmitry Chuyko wrote:
>> I have following error during the build of 15 tip:
>>
>> * For target hotspot_variant-server_libjvm_objs_os_linux.o:
>> jdk/src/hotspot/os/linux/os_linux.cpp: In static member function 'static void* os::dll_load(const char*, char*, int)':
>> jdk/src/hotspot/os/linux/os_linux.cpp:1854:6: error: 'EM_486' was not declared in this scope
>>  ???? {EM_486,???????? EM_386,???? ELFCLASS32, ELFDATA2LSB, (char*)"IA 32"},
>>  ????? ^~~~~~
>> jdk/src/hotspot/os/linux/os_linux.cpp:1854:6: note: suggested alternative: 'EM_386'
>>  ???? {EM_486,???????? EM_386,???? ELFCLASS32, ELFDATA2LSB, (char*)"IA 32"},
>>  ????? ^~~~~~
>>  ????? EM_386
>>
>> It happens on CenOS 8 and Ubuntu 20.04 (kernel headers installed), aarch64 and amd64.
> I have opened a bug for the issue [1].
>
> Will publish the fix suggested by Ao Qi once you can confirm it fixes the problem
> for you.
The patch did not seem to solve the problem for me. I'm currently 
cleaning and rebuilding to verify, but it might be a non-starter.

/Magnus
>
> Adrian
>
>> [1] https://bugs.openjdk.java.net/browse/JDK-8243998


From magnus.ihse.bursie at oracle.com  Tue Apr 28 11:12:09 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Tue, 28 Apr 2020 13:12:09 +0200
Subject: RFR: 8242244: Remove redundant ELF machine definitions
In-Reply-To: <cc679014-7637-7667-3e75-c45f4f5d6d02@oracle.com>
References: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>
 <22fdf007-a3d5-f14f-4d27-5746ad21a88e@oracle.com>
 <483038a4-7517-e190-cbbe-783da5f68ecc@physik.fu-berlin.de>
 <8b5812dd-78bf-f099-f066-b0bac2cf1572@bell-sw.com>
 <3e17af2e-5cd6-7bee-5147-89093d248657@physik.fu-berlin.de>
 <cc679014-7637-7667-3e75-c45f4f5d6d02@oracle.com>
Message-ID: <2e9a3c14-d444-6328-30b8-6c7fa8e400bb@oracle.com>

On 2020-04-28 13:04, Magnus Ihse Bursie wrote:
> On 2020-04-28 12:52, John Paul Adrian Glaubitz wrote:
>> Hi!
>>
>> On 4/28/20 12:33 PM, Dmitry Chuyko wrote:
>>> I have following error during the build of 15 tip:
>>>
>>> * For target hotspot_variant-server_libjvm_objs_os_linux.o:
>>> jdk/src/hotspot/os/linux/os_linux.cpp: In static member function 
>>> 'static void* os::dll_load(const char*, char*, int)':
>>> jdk/src/hotspot/os/linux/os_linux.cpp:1854:6: error: 'EM_486' was 
>>> not declared in this scope
>>> ????? {EM_486,???????? EM_386,???? ELFCLASS32, ELFDATA2LSB, 
>>> (char*)"IA 32"},
>>> ?????? ^~~~~~
>>> jdk/src/hotspot/os/linux/os_linux.cpp:1854:6: note: suggested 
>>> alternative: 'EM_386'
>>> ????? {EM_486,???????? EM_386,???? ELFCLASS32, ELFDATA2LSB, 
>>> (char*)"IA 32"},
>>> ?????? ^~~~~~
>>> ?????? EM_386
>>>
>>> It happens on CenOS 8 and Ubuntu 20.04 (kernel headers installed), 
>>> aarch64 and amd64.
>> I have opened a bug for the issue [1].
>>
>> Will publish the fix suggested by Ao Qi once you can confirm it fixes 
>> the problem
>> for you.
> The patch did not seem to solve the problem for me. I'm currently 
> cleaning and rebuilding to verify, but it might be a non-starter.
I now have more information.

The patch works if I build locally on my Ubuntu system.

It does not work when I use the Oracle devkit. That linux/elf-em.h 
headers does not have the EM_AARCH64 define. Since this is a very 
actively used environment used to build OpenJDK, I'd say your assumption 
in the original bug "[...] unless the JDK is supposed to be built on 
systems older than that" was incorrect.

Recommending that you back out this change.

/Magnus



>
> /Magnus
>>
>> Adrian
>>
>>> [1] https://bugs.openjdk.java.net/browse/JDK-8243998
>


From thomas.stuefe at gmail.com  Tue Apr 28 11:14:26 2020
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Tue, 28 Apr 2020 13:14:26 +0200
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64
In-Reply-To: <9a314ebe-5499-ecb6-3c5b-0c8ce7f9c8d2@oracle.com>
References: <56a0cf85-9698-64b2-95b4-e5b123c3e9e2@oracle.com>
 <CAA-vtUyFiMpf2vSXLf-c9uaKYXF1tSA3At5KJySCiz2skG0w3A@mail.gmail.com>
 <9a314ebe-5499-ecb6-3c5b-0c8ce7f9c8d2@oracle.com>
Message-ID: <CAA-vtUwYFCwT1a0Qbcrqmo6OVLit3DROq0jC8WqAYEd3_3XY9A@mail.gmail.com>

Hi Erik,

On Tue, Apr 28, 2020 at 10:07 AM Erik ?sterlund <erik.osterlund at oracle.com>
wrote:

> Hi Thomas,
>
> On 2020-04-28 08:43, Thomas St?fe wrote:
>
> Hi Erik,
>
> nice change, and now I do not have to keep in mind "heap>32G = no class
> space" :)
>
>
> Glad you liked it!
>
> I eyeballed the metaspace changes (what little there are).
>
> This:
>
> -     char* base = (char*)align_up(CompressedOops::end(), _reserve_alignment);+     char* base;+     if (UseCompressedOops) {+       base = (char*)align_up(CompressedOops::end(), _reserve_alignment);+     } else {+       base = (char*)HeapBaseMinAddress;+     }
>
> may not work as intended. You probably want the end of the reserved heap as attach point for ccs, not the HeapBaseMinAddress.
>
>
> Could you please expand on that? To make it clear, what this code does is
> to put metaspace at the
> minimum address we are allowed to map, when compressed oops is disabled.
> The reasoning for doing
> that is that if the low address bits (resulting in good compressed
> pointers) are not used by compressed oops,
> then they are available to be used by compressed class pointers instead,
> such that at least compressed class
> pointers can utilize this VA to get optimal encoding of class pointers.
>
> Is there anything not desirable about this? If so, I can't see it, and
> would appreciate if you expand on why
> that would be a bad idea. Perhaps there is something I do not see here.
>
>
Oh, this was a long standing confusion in my head. I never noticed that
HeapBaseMinAddress exists for the sole purpose of compressed oops, and is
ignored otherwise. So I thought what you do there is just mapping ccs over
the start of where the user wants the heap to start, and since that may not
work we would fall back to map-anywhere.

So you are saying HeapBaseMinAddress is the lowest address the platform
allows us to map to, irregardless of whether we run with compressed oops or
not.

With compressed oops we continue to squeeze ccs in after the heap in the
hope that it all still fits into the lower 4G or 32G. (Do we ever bother to
check if heap is actually reserved in low area? It may have given up and
reserved upstate instead, leaving free room below...).

Now with your patch without compressed oops we attempt to map at
HeapBaseMinAddress since this is the lowest address we are allowed to map
at and the heap won't obviously need it, right?

My only misgiving would be that HeapBaseMinAddress is even more of a
misnomer (its already now a bit, witness my year long confusion :-), since
it now controls not only heap placement but ccs too. Actually if it is that
important to avoid mapping to addresses below HeapBaseMinAddress we should
probably prevent it somewhere below in the os layer but I really do not
want to open that can of worms.

--

Some more remarks about arguments.cpp:

- In Arguments::set_use_compressed_klass_ptrs(), if now compressed class
pointers are independent from compressed oops, checking
CompressedClassSpaceSize for validity should now always happen for
UseCompressedClassPointers=1.
- If HeapBaseMinAddress is now always used, we should check it always too
(see Arguments::set_heap_size)

Also, could we have some tests which exercise cds/metaspace initialization
> without coops and with ccps? Easiest way would be to extend
> "CompressedOops/CompressedClassPointers.java" by some cases.
>
>
>
Thank you. I think this makes sense especially since compressed class
pointer encoding is restricted to one mode if CDS is on (base = shared
address and shift=3), and since CDS=on is the default zerobased encoding
may not happen that often.


> I will have a look at that for the next webrev.
>
> Thanks for the review.
>
> /Erik
>
>

Thanks,

Thomas



> Thank you, Thomas
>
>
> On Fri, Apr 24, 2020 at 10:19 AM Erik ?sterlund <erik.osterlund at oracle.com>
> wrote:
>
>> Hi,
>>
>> Today, the use of compressed class pointers assumes the use of
>> compressed oops.
>> This patch loosens up the relationship between compressed oops and
>> compressed
>> class pointers, so that compressed class pointers can be used without
>> compressed
>> oops. This benefits for example ZGC that wants compressed class
>> pointers, but
>> not compressed oops, effectively giving ZGC 4 bytes smaller objects.
>>
>> Much of the complexity of the change is that r12 used to be used by
>> compressed
>> class pointers as some kind of semi-fast temp register that one would
>> restore
>> to the compressed oops heap base (or zero). That made it effectively a
>> slightly
>> optimized spilling mechanism used to find a temp register. I replaced that
>> mechanism with a plain old normal temp register that you pass in as a
>> parameter.
>> The bad news is that I had to find available temp registers in a lot of
>> places.
>> The good news is that almost always were there temp registers available
>> for free,
>> and hence the new temp register is even faster than the old optimized
>> spilling
>> mechanism. Because we almost never need any spilling at all in the
>> contexts where
>> this is used.
>>
>> Since I want the 4 new bytes to actually make objects smaller, I poked
>> the new
>> layout code a bit so that the klass gap is made available for fields.
>> That used
>> to be made available only with compressed oops enabled, due to
>> restrictions in
>> the layout code. Our new layout code does not have such restrictions, and
>> so
>> I will make the 4 bytes available for fields when the new layout code is
>> used
>> and compressed class pointers are used.
>>
>> Now I'm only fixing this for HotSpot x86_64. Ideally the use of
>> compressed oops
>> and compressed class pointers should not be entangled in other platforms
>> and
>> Graal. But that is beyond the scope of this change, and I will let them
>> behave
>> the way that they used to, to be potentially fixed later.
>>
>> Bug:
>> https://bugs.openjdk.java.net/browse/JDK-8241825
>>
>> Webrev:
>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/
>>
>> Testing:
>> hs-tier1-7
>>
>> Thanks,
>> /Erik
>>
>
>

From glaubitz at physik.fu-berlin.de  Tue Apr 28 11:14:58 2020
From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz)
Date: Tue, 28 Apr 2020 13:14:58 +0200
Subject: RFR: 8242244: Remove redundant ELF machine definitions
In-Reply-To: <2e9a3c14-d444-6328-30b8-6c7fa8e400bb@oracle.com>
References: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>
 <22fdf007-a3d5-f14f-4d27-5746ad21a88e@oracle.com>
 <483038a4-7517-e190-cbbe-783da5f68ecc@physik.fu-berlin.de>
 <8b5812dd-78bf-f099-f066-b0bac2cf1572@bell-sw.com>
 <3e17af2e-5cd6-7bee-5147-89093d248657@physik.fu-berlin.de>
 <cc679014-7637-7667-3e75-c45f4f5d6d02@oracle.com>
 <2e9a3c14-d444-6328-30b8-6c7fa8e400bb@oracle.com>
Message-ID: <a4678d4d-6c09-c24f-b572-86b1c930eeea@physik.fu-berlin.de>

Hi Magnus!

On 4/28/20 1:12 PM, Magnus Ihse Bursie wrote:
> It does not work when I use the Oracle devkit. That linux/elf-em.h headers does not have the EM_AARCH64 define. Since this is a very actively used environment used to build OpenJDK, I'd say your assumption in the original bug "[...] unless the JDK is supposed to be built on systems older than that" was incorrect.

Ok, thank you. Does the Oracle devkit define EM_486 or does that need to be there as well?

> Recommending that you back out this change.

Apologies for the breakage. I should have done more testing.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz at debian.org
`. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

From aoqi at loongson.cn  Tue Apr 28 11:17:21 2020
From: aoqi at loongson.cn (Ao Qi)
Date: Tue, 28 Apr 2020 19:17:21 +0800
Subject: RFR: 8242244: Remove redundant ELF machine definitions
In-Reply-To: <cc679014-7637-7667-3e75-c45f4f5d6d02@oracle.com>
References: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>
 <22fdf007-a3d5-f14f-4d27-5746ad21a88e@oracle.com>
 <483038a4-7517-e190-cbbe-783da5f68ecc@physik.fu-berlin.de>
 <8b5812dd-78bf-f099-f066-b0bac2cf1572@bell-sw.com>
 <3e17af2e-5cd6-7bee-5147-89093d248657@physik.fu-berlin.de>
 <cc679014-7637-7667-3e75-c45f4f5d6d02@oracle.com>
Message-ID: <CALjzQn4DxNtJQfjjr1OPsjZ5KDF4-xgmMisdqoJovhgGLnbJ6w@mail.gmail.com>

On Tue, Apr 28, 2020 at 7:04 PM Magnus Ihse Bursie
<magnus.ihse.bursie at oracle.com> wrote:
>
> On 2020-04-28 12:52, John Paul Adrian Glaubitz wrote:
> > Hi!
> >
> > On 4/28/20 12:33 PM, Dmitry Chuyko wrote:
> >> I have following error during the build of 15 tip:
> >>
> >> * For target hotspot_variant-server_libjvm_objs_os_linux.o:
> >> jdk/src/hotspot/os/linux/os_linux.cpp: In static member function 'static void* os::dll_load(const char*, char*, int)':
> >> jdk/src/hotspot/os/linux/os_linux.cpp:1854:6: error: 'EM_486' was not declared in this scope
> >>       {EM_486,         EM_386,     ELFCLASS32, ELFDATA2LSB, (char*)"IA 32"},
> >>        ^~~~~~
> >> jdk/src/hotspot/os/linux/os_linux.cpp:1854:6: note: suggested alternative: 'EM_386'
> >>       {EM_486,         EM_386,     ELFCLASS32, ELFDATA2LSB, (char*)"IA 32"},
> >>        ^~~~~~
> >>        EM_386
> >>
> >> It happens on CenOS 8 and Ubuntu 20.04 (kernel headers installed), aarch64 and amd64.
> > I have opened a bug for the issue [1].
> >
> > Will publish the fix suggested by Ao Qi once you can confirm it fixes the problem
> > for you.
> The patch did not seem to solve the problem for me. I'm currently
> cleaning and rebuilding to verify, but it might be a non-starter.

My working day is near the end. Anyone please feel free to take the
issue. Thanks:)

Cheers,
Ao Qi

>
> /Magnus
> >
> > Adrian
> >
> >> [1] https://bugs.openjdk.java.net/browse/JDK-8243998


From magnus.ihse.bursie at oracle.com  Tue Apr 28 11:23:42 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Tue, 28 Apr 2020 13:23:42 +0200
Subject: RFR: 8242244: Remove redundant ELF machine definitions
In-Reply-To: <a4678d4d-6c09-c24f-b572-86b1c930eeea@physik.fu-berlin.de>
References: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>
 <22fdf007-a3d5-f14f-4d27-5746ad21a88e@oracle.com>
 <483038a4-7517-e190-cbbe-783da5f68ecc@physik.fu-berlin.de>
 <8b5812dd-78bf-f099-f066-b0bac2cf1572@bell-sw.com>
 <3e17af2e-5cd6-7bee-5147-89093d248657@physik.fu-berlin.de>
 <cc679014-7637-7667-3e75-c45f4f5d6d02@oracle.com>
 <2e9a3c14-d444-6328-30b8-6c7fa8e400bb@oracle.com>
 <a4678d4d-6c09-c24f-b572-86b1c930eeea@physik.fu-berlin.de>
Message-ID: <ab7eff23-1f63-9a9b-7a6d-0fb022362ac5@oracle.com>

On 2020-04-28 13:14, John Paul Adrian Glaubitz wrote:
> Hi Magnus!
>
> On 4/28/20 1:12 PM, Magnus Ihse Bursie wrote:
>> It does not work when I use the Oracle devkit. That linux/elf-em.h headers does not have the EM_AARCH64 define. Since this is a very actively used environment used to build OpenJDK, I'd say your assumption in the original bug "[...] unless the JDK is supposed to be built on systems older than that" was incorrect.
> Ok, thank you. Does the Oracle devkit define EM_486 or does that need to be there as well?
I thought a simple backing out would be the best, since we have a build 
breakage.

But sure, I understand why you want to keep part of the cleaning up.

I have checked, EM_486 is defined. So this combined partial backout + 
the fix suggested by Qi works:

diff --git a/src/hotspot/os/linux/os_linux.cpp 
b/src/hotspot/os/linux/os_linux.cpp
--- a/src/hotspot/os/linux/os_linux.cpp
+++ b/src/hotspot/os/linux/os_linux.cpp
@@ -110,6 +110,7 @@
 ?# include <stdint.h>
 ?# include <inttypes.h>
 ?# include <sys/ioctl.h>
+# include <linux/elf-em.h>

 ?#ifndef _GNU_SOURCE
 ?? #define _GNU_SOURCE
@@ -1845,6 +1846,9 @@
 ???? char*???????? name;???????? // String representation
 ?? } arch_t;

+#ifndef EM_AARCH64
+? #define EM_AARCH64??? 183?????????????? /* ARM AARCH64 */
+#endif
 ?#ifndef EM_RISCV
 ?? #define EM_RISCV????? 243?????????????? /* RISC-V */
 ?#endif


I'd say that this is a trivial fix, and you can consider it reviewed by 
me (although the authorship of this patch is somewhat convoluted).

/Magnus

>
>> Recommending that you back out this change.
> Apologies for the breakage. I should have done more testing.
>
> Adrian
>


From glaubitz at physik.fu-berlin.de  Tue Apr 28 11:28:50 2020
From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz)
Date: Tue, 28 Apr 2020 13:28:50 +0200
Subject: RFR: 8242244: Remove redundant ELF machine definitions
In-Reply-To: <ab7eff23-1f63-9a9b-7a6d-0fb022362ac5@oracle.com>
References: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>
 <22fdf007-a3d5-f14f-4d27-5746ad21a88e@oracle.com>
 <483038a4-7517-e190-cbbe-783da5f68ecc@physik.fu-berlin.de>
 <8b5812dd-78bf-f099-f066-b0bac2cf1572@bell-sw.com>
 <3e17af2e-5cd6-7bee-5147-89093d248657@physik.fu-berlin.de>
 <cc679014-7637-7667-3e75-c45f4f5d6d02@oracle.com>
 <2e9a3c14-d444-6328-30b8-6c7fa8e400bb@oracle.com>
 <a4678d4d-6c09-c24f-b572-86b1c930eeea@physik.fu-berlin.de>
 <ab7eff23-1f63-9a9b-7a6d-0fb022362ac5@oracle.com>
Message-ID: <f183ef9f-6e0f-ebf6-0b03-e36e4c8d40f5@physik.fu-berlin.de>

Hi Magnus!

On 4/28/20 1:23 PM, Magnus Ihse Bursie wrote:
> I thought a simple backing out would be the best, since we have a build breakage.
> 
> But sure, I understand why you want to keep part of the cleaning up.
> 
> I have checked, EM_486 is defined. So this combined partial backout + the fix suggested by Qi works:

Thank you!

> diff --git a/src/hotspot/os/linux/os_linux.cpp b/src/hotspot/os/linux/os_linux.cpp
> --- a/src/hotspot/os/linux/os_linux.cpp
> +++ b/src/hotspot/os/linux/os_linux.cpp
> @@ -110,6 +110,7 @@
> ?# include <stdint.h>
> ?# include <inttypes.h>
> ?# include <sys/ioctl.h>
> +# include <linux/elf-em.h>
> 
> ?#ifndef _GNU_SOURCE
> ?? #define _GNU_SOURCE
> @@ -1845,6 +1846,9 @@
> ???? char*???????? name;???????? // String representation
> ?? } arch_t;
> 
> +#ifndef EM_AARCH64
> +? #define EM_AARCH64??? 183?????????????? /* ARM AARCH64 */
> +#endif
> ?#ifndef EM_RISCV
> ?? #define EM_RISCV????? 243?????????????? /* RISC-V */
> ?#endif
> 
> 
> I'd say that this is a trivial fix, and you can consider it reviewed by me (although the authorship of this patch is somewhat convoluted).

I do not insist on taking authorship, so if you like to push it yourself, I'm
more than happy with it.

It's the first time I broke the build, so I feel a little bad about it.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz at debian.org
`. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

From magnus.ihse.bursie at oracle.com  Tue Apr 28 11:33:54 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Tue, 28 Apr 2020 13:33:54 +0200
Subject: RFR: 8242244: Remove redundant ELF machine definitions
In-Reply-To: <f183ef9f-6e0f-ebf6-0b03-e36e4c8d40f5@physik.fu-berlin.de>
References: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>
 <22fdf007-a3d5-f14f-4d27-5746ad21a88e@oracle.com>
 <483038a4-7517-e190-cbbe-783da5f68ecc@physik.fu-berlin.de>
 <8b5812dd-78bf-f099-f066-b0bac2cf1572@bell-sw.com>
 <3e17af2e-5cd6-7bee-5147-89093d248657@physik.fu-berlin.de>
 <cc679014-7637-7667-3e75-c45f4f5d6d02@oracle.com>
 <2e9a3c14-d444-6328-30b8-6c7fa8e400bb@oracle.com>
 <a4678d4d-6c09-c24f-b572-86b1c930eeea@physik.fu-berlin.de>
 <ab7eff23-1f63-9a9b-7a6d-0fb022362ac5@oracle.com>
 <f183ef9f-6e0f-ebf6-0b03-e36e4c8d40f5@physik.fu-berlin.de>
Message-ID: <d8e66c85-62fc-713f-43f7-d6d7967aa5f8@oracle.com>

On 2020-04-28 13:28, John Paul Adrian Glaubitz wrote:
> Hi Magnus!
>
> On 4/28/20 1:23 PM, Magnus Ihse Bursie wrote:
>> I thought a simple backing out would be the best, since we have a build breakage.
>>
>> But sure, I understand why you want to keep part of the cleaning up.
>>
>> I have checked, EM_486 is defined. So this combined partial backout + the fix suggested by Qi works:
> Thank you!
>
>> diff --git a/src/hotspot/os/linux/os_linux.cpp b/src/hotspot/os/linux/os_linux.cpp
>> --- a/src/hotspot/os/linux/os_linux.cpp
>> +++ b/src/hotspot/os/linux/os_linux.cpp
>> @@ -110,6 +110,7 @@
>>  ?# include <stdint.h>
>>  ?# include <inttypes.h>
>>  ?# include <sys/ioctl.h>
>> +# include <linux/elf-em.h>
>>
>>  ?#ifndef _GNU_SOURCE
>>  ?? #define _GNU_SOURCE
>> @@ -1845,6 +1846,9 @@
>>  ???? char*???????? name;???????? // String representation
>>  ?? } arch_t;
>>
>> +#ifndef EM_AARCH64
>> +? #define EM_AARCH64??? 183?????????????? /* ARM AARCH64 */
>> +#endif
>>  ?#ifndef EM_RISCV
>>  ?? #define EM_RISCV????? 243?????????????? /* RISC-V */
>>  ?#endif
>>
>>
>> I'd say that this is a trivial fix, and you can consider it reviewed by me (although the authorship of this patch is somewhat convoluted).
> I do not insist on taking authorship, so if you like to push it yourself, I'm
> more than happy with it.
It's more a matter of formality. If I consign myself as author to it, 
I'd need a Reviewer to confirm it. Seems they are all asleep now, and 
I'd rather see that they do not wake up to a broken CI system. :)

/Magnus
>
> It's the first time I broke the build, so I feel a little bad about it.
>
> Adrian
>


From glaubitz at physik.fu-berlin.de  Tue Apr 28 11:37:54 2020
From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz)
Date: Tue, 28 Apr 2020 13:37:54 +0200
Subject: RFR: 8242244: Remove redundant ELF machine definitions
In-Reply-To: <d8e66c85-62fc-713f-43f7-d6d7967aa5f8@oracle.com>
References: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>
 <22fdf007-a3d5-f14f-4d27-5746ad21a88e@oracle.com>
 <483038a4-7517-e190-cbbe-783da5f68ecc@physik.fu-berlin.de>
 <8b5812dd-78bf-f099-f066-b0bac2cf1572@bell-sw.com>
 <3e17af2e-5cd6-7bee-5147-89093d248657@physik.fu-berlin.de>
 <cc679014-7637-7667-3e75-c45f4f5d6d02@oracle.com>
 <2e9a3c14-d444-6328-30b8-6c7fa8e400bb@oracle.com>
 <a4678d4d-6c09-c24f-b572-86b1c930eeea@physik.fu-berlin.de>
 <ab7eff23-1f63-9a9b-7a6d-0fb022362ac5@oracle.com>
 <f183ef9f-6e0f-ebf6-0b03-e36e4c8d40f5@physik.fu-berlin.de>
 <d8e66c85-62fc-713f-43f7-d6d7967aa5f8@oracle.com>
Message-ID: <a507677d-a2a1-5f70-1a02-5ec886a3d65c@physik.fu-berlin.de>

On 4/28/20 1:33 PM, Magnus Ihse Bursie wrote:
> It's more a matter of formality. If I consign myself as author to it, I'd need a Reviewer to confirm it. Seems they are all asleep now, and I'd rather see that they do not wake up to a broken CI system. :)

Okay, I'll create a webrev in a minute.

Thanks for your help!

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz at debian.org
`. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

From aoqi at loongson.cn  Tue Apr 28 11:43:18 2020
From: aoqi at loongson.cn (Ao Qi)
Date: Tue, 28 Apr 2020 19:43:18 +0800
Subject: RFR: 8242244: Remove redundant ELF machine definitions
In-Reply-To: <d8e66c85-62fc-713f-43f7-d6d7967aa5f8@oracle.com>
References: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>
 <22fdf007-a3d5-f14f-4d27-5746ad21a88e@oracle.com>
 <483038a4-7517-e190-cbbe-783da5f68ecc@physik.fu-berlin.de>
 <8b5812dd-78bf-f099-f066-b0bac2cf1572@bell-sw.com>
 <3e17af2e-5cd6-7bee-5147-89093d248657@physik.fu-berlin.de>
 <cc679014-7637-7667-3e75-c45f4f5d6d02@oracle.com>
 <2e9a3c14-d444-6328-30b8-6c7fa8e400bb@oracle.com>
 <a4678d4d-6c09-c24f-b572-86b1c930eeea@physik.fu-berlin.de>
 <ab7eff23-1f63-9a9b-7a6d-0fb022362ac5@oracle.com>
 <f183ef9f-6e0f-ebf6-0b03-e36e4c8d40f5@physik.fu-berlin.de>
 <d8e66c85-62fc-713f-43f7-d6d7967aa5f8@oracle.com>
Message-ID: <CALjzQn7cSwF16e=+_0B3+G5zRQoFAnr6yX=3D2JioRVpwL9Xrg@mail.gmail.com>

On Tue, Apr 28, 2020 at 7:33 PM Magnus Ihse Bursie
<magnus.ihse.bursie at oracle.com> wrote:
>
> On 2020-04-28 13:28, John Paul Adrian Glaubitz wrote:
> > Hi Magnus!
> >
> > On 4/28/20 1:23 PM, Magnus Ihse Bursie wrote:
> >> I thought a simple backing out would be the best, since we have a build breakage.
> >>
> >> But sure, I understand why you want to keep part of the cleaning up.
> >>
> >> I have checked, EM_486 is defined. So this combined partial backout + the fix suggested by Qi works:
> > Thank you!
> >
> >> diff --git a/src/hotspot/os/linux/os_linux.cpp b/src/hotspot/os/linux/os_linux.cpp
> >> --- a/src/hotspot/os/linux/os_linux.cpp
> >> +++ b/src/hotspot/os/linux/os_linux.cpp
> >> @@ -110,6 +110,7 @@
> >>   # include <stdint.h>
> >>   # include <inttypes.h>
> >>   # include <sys/ioctl.h>
> >> +# include <linux/elf-em.h>
> >>
> >>   #ifndef _GNU_SOURCE
> >>     #define _GNU_SOURCE
> >> @@ -1845,6 +1846,9 @@
> >>       char*         name;         // String representation
> >>     } arch_t;
> >>
> >> +#ifndef EM_AARCH64
> >> +  #define EM_AARCH64    183               /* ARM AARCH64 */
> >> +#endif
> >>   #ifndef EM_RISCV
> >>     #define EM_RISCV      243               /* RISC-V */
> >>   #endif
> >>
> >>
> >> I'd say that this is a trivial fix, and you can consider it reviewed by me (although the authorship of this patch is somewhat convoluted).
> > I do not insist on taking authorship, so if you like to push it yourself, I'm
> > more than happy with it.
> It's more a matter of formality. If I consign myself as author to it,
> I'd need a Reviewer to confirm it. Seems they are all asleep now, and
> I'd rather see that they do not wake up to a broken CI system. :)

Hi Magnus,

Is there anything that you suggest what to do the next? Is there
someting I can help?

Further, do we need a seperate RFR thread for the patch?

Thanks,
Ao Qi
>
> /Magnus
> >
> > It's the first time I broke the build, so I feel a little bad about it.
> >
> > Adrian
> >


From magnus.ihse.bursie at oracle.com  Tue Apr 28 11:46:43 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Tue, 28 Apr 2020 13:46:43 +0200
Subject: RFR: 8242244: Remove redundant ELF machine definitions
In-Reply-To: <CALjzQn7cSwF16e=+_0B3+G5zRQoFAnr6yX=3D2JioRVpwL9Xrg@mail.gmail.com>
References: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>
 <22fdf007-a3d5-f14f-4d27-5746ad21a88e@oracle.com>
 <483038a4-7517-e190-cbbe-783da5f68ecc@physik.fu-berlin.de>
 <8b5812dd-78bf-f099-f066-b0bac2cf1572@bell-sw.com>
 <3e17af2e-5cd6-7bee-5147-89093d248657@physik.fu-berlin.de>
 <cc679014-7637-7667-3e75-c45f4f5d6d02@oracle.com>
 <2e9a3c14-d444-6328-30b8-6c7fa8e400bb@oracle.com>
 <a4678d4d-6c09-c24f-b572-86b1c930eeea@physik.fu-berlin.de>
 <ab7eff23-1f63-9a9b-7a6d-0fb022362ac5@oracle.com>
 <f183ef9f-6e0f-ebf6-0b03-e36e4c8d40f5@physik.fu-berlin.de>
 <d8e66c85-62fc-713f-43f7-d6d7967aa5f8@oracle.com>
 <CALjzQn7cSwF16e=+_0B3+G5zRQoFAnr6yX=3D2JioRVpwL9Xrg@mail.gmail.com>
Message-ID: <f6dfed7e-963b-9ed3-9f9e-0901dd42d3c2@oracle.com>



On 2020-04-28 13:43, Ao Qi wrote:
> On Tue, Apr 28, 2020 at 7:33 PM Magnus Ihse Bursie
> <magnus.ihse.bursie at oracle.com> wrote:
>> On 2020-04-28 13:28, John Paul Adrian Glaubitz wrote:
>>> Hi Magnus!
>>>
>>> On 4/28/20 1:23 PM, Magnus Ihse Bursie wrote:
>>>> I thought a simple backing out would be the best, since we have a build breakage.
>>>>
>>>> But sure, I understand why you want to keep part of the cleaning up.
>>>>
>>>> I have checked, EM_486 is defined. So this combined partial backout + the fix suggested by Qi works:
>>> Thank you!
>>>
>>>> diff --git a/src/hotspot/os/linux/os_linux.cpp b/src/hotspot/os/linux/os_linux.cpp
>>>> --- a/src/hotspot/os/linux/os_linux.cpp
>>>> +++ b/src/hotspot/os/linux/os_linux.cpp
>>>> @@ -110,6 +110,7 @@
>>>>    # include <stdint.h>
>>>>    # include <inttypes.h>
>>>>    # include <sys/ioctl.h>
>>>> +# include <linux/elf-em.h>
>>>>
>>>>    #ifndef _GNU_SOURCE
>>>>      #define _GNU_SOURCE
>>>> @@ -1845,6 +1846,9 @@
>>>>        char*         name;         // String representation
>>>>      } arch_t;
>>>>
>>>> +#ifndef EM_AARCH64
>>>> +  #define EM_AARCH64    183               /* ARM AARCH64 */
>>>> +#endif
>>>>    #ifndef EM_RISCV
>>>>      #define EM_RISCV      243               /* RISC-V */
>>>>    #endif
>>>>
>>>>
>>>> I'd say that this is a trivial fix, and you can consider it reviewed by me (although the authorship of this patch is somewhat convoluted).
>>> I do not insist on taking authorship, so if you like to push it yourself, I'm
>>> more than happy with it.
>> It's more a matter of formality. If I consign myself as author to it,
>> I'd need a Reviewer to confirm it. Seems they are all asleep now, and
>> I'd rather see that they do not wake up to a broken CI system. :)
> Hi Magnus,
>
> Is there anything that you suggest what to do the next? Is there
> someting I can help?
No, but thank you for your offer. I think Adrian and I got this under 
control now.

>
> Further, do we need a seperate RFR thread for the patch?
That is always a good idea, for record-keeping. I believe Adrian is 
working on that right now.

/Magnus
>
> Thanks,
> Ao Qi
>> /Magnus
>>> It's the first time I broke the build, so I feel a little bad about it.
>>>
>>> Adrian
>>>


From glaubitz at physik.fu-berlin.de  Tue Apr 28 11:47:09 2020
From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz)
Date: Tue, 28 Apr 2020 13:47:09 +0200
Subject: RFR: 8243997: Linux build failed after JDK-8242244
Message-ID: <4113de5d-3c65-b999-cdd3-9e12d05955f1@physik.fu-berlin.de>

Hello!

Please review this small change which fixes the build on Linux which
broke on some systems after JDK-8242244.

Thanks,
Adrian

> [1] http://cr.openjdk.java.net/~glaubitz/8243997/webrev.00/

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz at debian.org
`. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

From stefan.karlsson at oracle.com  Tue Apr 28 11:58:35 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Tue, 28 Apr 2020 13:58:35 +0200
Subject: RFR: 8244010: Simplify usages of
 ProcessTools.createJavaProcessBuilder in our tests
Message-ID: <60c0e0e4-634b-d349-c6f3-ca1ad52926cb@oracle.com>

Hi all,

Please review this patch to simplify usages of 
ProcessTools.createJavaProcessBuilder in our tests.

https://cr.openjdk.java.net/~stefank/8244010/webrev.01/
https://bugs.openjdk.java.net/browse/JDK-8244010

I saw all this code when reviewing changes to how we pass flags in our 
tests. There are a many places where arguments are converted and passed 
back and forth in String[] and Collections.

For example:
 ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(
 ??????????argsList.toArray(new String[argsList.size()]));

If we add an overload the createJavaProcessBuilder, that takes a 
Collection<String> as an argument, then we can write the code above as:
 ???ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(argsList);

Other places temporarily put the flags in a String[], where most calls 
simply lists the arguments in the call:
 ??String[] opts = {Xmx, "-XX:NativeMemoryTracking=detail", 
"-XX:+UseParallelGC", "-version"};
 ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(opts);

And some places put the args in a temporary Collection:
 ??LinkedList<String> vmOptions = new LinkedList<>();
 ??vmOptions.add(gc);
 ??vmOptions.add("-Xmx" + minMaxHeap);
 ??vmOptions.add("-XX:+PrintFlagsFinal");
 ??vmOptions.add(VerifyHeapSize.class.getName());

 ??ProcessBuilder pb = 
ProcessTools.createJavaProcessBuilder(vmOptions.toArray(new String[0]));

I'd like to cleanup, simplify, and unify many of these usages.

I've tested this by running all the changed tests locally.

Thanks,
StefanK

From magnus.ihse.bursie at oracle.com  Tue Apr 28 12:02:25 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Tue, 28 Apr 2020 14:02:25 +0200
Subject: RFR: 8243997: Linux build failed after JDK-8242244
In-Reply-To: <4113de5d-3c65-b999-cdd3-9e12d05955f1@physik.fu-berlin.de>
References: <4113de5d-3c65-b999-cdd3-9e12d05955f1@physik.fu-berlin.de>
Message-ID: <dcbcb103-1bac-d7f7-ea18-9cbb49200966@oracle.com>

On 2020-04-28 13:47, John Paul Adrian Glaubitz wrote:
> Hello!
>
> Please review this small change which fixes the build on Linux which
> broke on some systems after JDK-8242244.
>
> Thanks,
> Adrian
>
>> [1] http://cr.openjdk.java.net/~glaubitz/8243997/webrev.00/
Looks good and trivial to me.

/Magnus

From aoqi at loongson.cn  Tue Apr 28 12:09:15 2020
From: aoqi at loongson.cn (Ao Qi)
Date: Tue, 28 Apr 2020 20:09:15 +0800
Subject: RFR: 8243997: Linux build failed after JDK-8242244
In-Reply-To: <dcbcb103-1bac-d7f7-ea18-9cbb49200966@oracle.com>
References: <4113de5d-3c65-b999-cdd3-9e12d05955f1@physik.fu-berlin.de>
 <dcbcb103-1bac-d7f7-ea18-9cbb49200966@oracle.com>
Message-ID: <CALjzQn4Q-QWEWCje1ZbdS49WcUqjrpPqX9jB_54Ei+YpRP0sHQ@mail.gmail.com>

On Tue, Apr 28, 2020 at 8:02 PM Magnus Ihse Bursie
<magnus.ihse.bursie at oracle.com> wrote:
>
> On 2020-04-28 13:47, John Paul Adrian Glaubitz wrote:
> > Hello!
> >
> > Please review this small change which fixes the build on Linux which
> > broke on some systems after JDK-8242244.
> >
> > Thanks,
> > Adrian
> >
> >> [1] http://cr.openjdk.java.net/~glaubitz/8243997/webrev.00/

Looks good to me, although I am not a Reviewer.

Thanks,
Ao Qi

> Looks good and trivial to me.
>
> /Magnus


From martin.doerr at sap.com  Tue Apr 28 12:36:30 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Tue, 28 Apr 2020 12:36:30 +0000
Subject: RFR: 8243389: enhance os::pd_print_cpu_info on linux
In-Reply-To: <AM0PR02MB54121E0BA017A19057FAF78193AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
References: <AM0PR02MB541225ED5513768AED6732F093AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
 <AM4PR02MB30579EBA32B978AC69A3F8DD9AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>
 <AM0PR02MB54121E0BA017A19057FAF78193AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
Message-ID: <AM4PR02MB3057422667C0D81A15B420059AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>

Hi Matthias,

I'd use ExtensiveErrorReports for cache related info and the following ones:
"Maximum cpu frequency:"
"Minimum cpu frequency:"
"Current cpu frequency:"
"Available governors:"

Thanks and best regards,
Martin


> -----Original Message-----
> From: Baesken, Matthias <matthias.baesken at sap.com>
> Sent: Dienstag, 28. April 2020 12:53
> To: Doerr, Martin <martin.doerr at sap.com>; 'hotspot-
> dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: RE: RFR: 8243389: enhance os::pd_print_cpu_info on linux
> 
> > "Maximum cpu frequency" and "Minimum cpu frequency:" seems to be
> redundant when we have "Available cpu frequencies:".
> 
> Hi Martin, I added  all those intentionally because different  kernels/setups
> might report one but not the other.
> 
> > Please consider using ExtensiveErrorReports for some information like e.g.
> cache related info.
> 
> Good point, we should take this into consideration.
> Maybe some other opinions ? I would then guard the cache and frequency
> related info with the flag (or would you like to keep the frequency related
> info by default) ?
> 
> 
> Best regards, Matthias
> 
> -----Original Message-----
> From: Doerr, Martin <martin.doerr at sap.com>
> Sent: Dienstag, 28. April 2020 12:27
> To: Baesken, Matthias <matthias.baesken at sap.com>; 'hotspot-
> dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: RE: RFR: 8243389: enhance os::pd_print_cpu_info on linux
> 
> Hi Matthias,
> 
> technically, this looks good to me.
> 
> "Maximum cpu frequency" and "Minimum cpu frequency:" seems to be
> redundant when we have "Available cpu frequencies:".
> 
> I think it exceeds what a typical supporter needs to know in order to analyze
> issues.
> Please consider using ExtensiveErrorReports for some information like e.g.
> cache related info.
> 
> Best regards,
> Martin
> 
> 
> > -----Original Message-----
> > From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf
> Of
> > Baesken, Matthias
> > Sent: Dienstag, 28. April 2020 11:45
> > To: 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> > Subject: RE: RFR: 8243389: enhance os::pd_print_cpu_info on linux
> >
> > Ping ... any reviews please?
> >
> > Thanks, Matthias
> >
> > -------------------------------
> > Hello, please review the following enhancement of  os::pd_print_cpu_info
> > on linux .
> > It adds  information about
> > cache
> > online/offline cpus
> > frequency details
> > governors
> >
> > by looking at the info provided at /sys/devices/system/cpu  .
> >
> > .....
> > bug/webrev :
> >
> > https://bugs.openjdk.java.net/browse/JDK-8243389
> >
> > http://cr.openjdk.java.net/~mbaesken/webrevs/8243389.0/


From glaubitz at physik.fu-berlin.de  Tue Apr 28 13:15:48 2020
From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz)
Date: Tue, 28 Apr 2020 15:15:48 +0200
Subject: RFR: 8243997: Linux build failed after JDK-8242244
In-Reply-To: <dcbcb103-1bac-d7f7-ea18-9cbb49200966@oracle.com>
References: <4113de5d-3c65-b999-cdd3-9e12d05955f1@physik.fu-berlin.de>
 <dcbcb103-1bac-d7f7-ea18-9cbb49200966@oracle.com>
Message-ID: <649afb69-a3e0-1ec6-794a-8df7eec26ea7@physik.fu-berlin.de>

On 4/28/20 2:02 PM, Magnus Ihse Bursie wrote:
> On 2020-04-28 13:47, John Paul Adrian Glaubitz wrote:
>> Hello!
>>
>> Please review this small change which fixes the build on Linux which
>> broke on some systems after JDK-8242244.
>>
>> Thanks,
>> Adrian
>>
>>> [1] http://cr.openjdk.java.net/~glaubitz/8243997/webrev.00/
> Looks good and trivial to me.

Thanks. I'm pushing to the submit repo first to get some testing
before pushing.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz at debian.org
`. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

From stefan.karlsson at oracle.com  Tue Apr 28 13:54:41 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Tue, 28 Apr 2020 15:54:41 +0200
Subject: RFR: 8244010: Simplify usages of
 ProcessTools.createJavaProcessBuilder in our tests
In-Reply-To: <60c0e0e4-634b-d349-c6f3-ca1ad52926cb@oracle.com>
References: <60c0e0e4-634b-d349-c6f3-ca1ad52926cb@oracle.com>
Message-ID: <056ff9b1-1a4f-a1fb-2727-b503707f3750@oracle.com>

Hi again,

I realized that we probably want to give ProcessTools.executeTestJvm the 
same treatment.

Side-note: It's very awkard that createJavaProcessBuilder defaults to 
not adding user-specifed flags, but executeTestJvm does. I think it 
would be good to unify this as a separate RFE. I think *a lot* of 
callers to createJavaProcessBuilder could be simplified by either using 
executeTestJvm directly, or a simplified version of that.

I'm running testing through mach5 and found a few things to fix, I might 
find more when the testing has proceeded further.

This is the current patch:
https://cr.openjdk.java.net/~stefank/8244010/webrev.02.delta
https://cr.openjdk.java.net/~stefank/8244010/webrev.02

Thanks,
StefanK

On 2020-04-28 13:58, Stefan Karlsson wrote:
> Hi all,
>
> Please review this patch to simplify usages of 
> ProcessTools.createJavaProcessBuilder in our tests.
>
> https://cr.openjdk.java.net/~stefank/8244010/webrev.01/
> https://bugs.openjdk.java.net/browse/JDK-8244010
>
> I saw all this code when reviewing changes to how we pass flags in our 
> tests. There are a many places where arguments are converted and 
> passed back and forth in String[] and Collections.
>
> For example:
> ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(
> ??????????argsList.toArray(new String[argsList.size()]));
>
> If we add an overload the createJavaProcessBuilder, that takes a 
> Collection<String> as an argument, then we can write the code above as:
> ???ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(argsList);
>
> Other places temporarily put the flags in a String[], where most calls 
> simply lists the arguments in the call:
> ??String[] opts = {Xmx, "-XX:NativeMemoryTracking=detail", 
> "-XX:+UseParallelGC", "-version"};
> ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(opts);
>
> And some places put the args in a temporary Collection:
> ??LinkedList<String> vmOptions = new LinkedList<>();
> ??vmOptions.add(gc);
> ??vmOptions.add("-Xmx" + minMaxHeap);
> ??vmOptions.add("-XX:+PrintFlagsFinal");
> ??vmOptions.add(VerifyHeapSize.class.getName());
>
> ??ProcessBuilder pb = 
> ProcessTools.createJavaProcessBuilder(vmOptions.toArray(new String[0]));
>
> I'd like to cleanup, simplify, and unify many of these usages.
>
> I've tested this by running all the changed tests locally.
>
> Thanks,
> StefanK


From dmitry.chuyko at bell-sw.com  Tue Apr 28 14:05:47 2020
From: dmitry.chuyko at bell-sw.com (Dmitry Chuyko)
Date: Tue, 28 Apr 2020 17:05:47 +0300
Subject: RFR: 8242244: Remove redundant ELF machine definitions
In-Reply-To: <538aabc2-158d-c8ce-08ef-0ae3c62ffc78@physik.fu-berlin.de>
References: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>
 <22fdf007-a3d5-f14f-4d27-5746ad21a88e@oracle.com>
 <483038a4-7517-e190-cbbe-783da5f68ecc@physik.fu-berlin.de>
 <8b5812dd-78bf-f099-f066-b0bac2cf1572@bell-sw.com>
 <538aabc2-158d-c8ce-08ef-0ae3c62ffc78@physik.fu-berlin.de>
Message-ID: <441206dd-c4cf-bc65-7e28-700939009203@bell-sw.com>

Yes, our builds work after this fix.

-Dmitry

On 4/28/20 1:41 PM, John Paul Adrian Glaubitz wrote:
> Hi Dmitry!
>
> On 4/28/20 12:33 PM, Dmitry Chuyko wrote:
>> * For target hotspot_variant-server_libjvm_objs_os_linux.o:
>> jdk/src/hotspot/os/linux/os_linux.cpp: In static member function 'static void* os::dll_load(const char*, char*, int)':
>> jdk/src/hotspot/os/linux/os_linux.cpp:1854:6: error: 'EM_486' was not declared in this scope
>>  ???? {EM_486,???????? EM_386,???? ELFCLASS32, ELFDATA2LSB, (char*)"IA 32"},
>>  ????? ^~~~~~
>> jdk/src/hotspot/os/linux/os_linux.cpp:1854:6: note: suggested alternative: 'EM_386'
>>  ???? {EM_486,???????? EM_386,???? ELFCLASS32, ELFDATA2LSB, (char*)"IA 32"},
>>  ????? ^~~~~~
>>  ????? EM_386
>>
>> It happens on CenOS 8 and Ubuntu 20.04 (kernel headers installed), aarch64 and amd64.
>>
>> Is any additional configuration expected?
> Ao already found a potential fix:
>
> diff -r 7205265660b2 src/hotspot/os/linux/os_linux.cpp
> --- a/src/hotspot/os/linux/os_linux.cpp Tue Apr 28 05:53:19 2020 -0400
> +++ b/src/hotspot/os/linux/os_linux.cpp Tue Apr 28 18:24:45 2020 +0800
> @@ -110,6 +110,7 @@
>   # include <stdint.h>
>   # include <inttypes.h>
>   # include <sys/ioctl.h>
> +# include <linux/elf-em.h>
>
>   #ifndef _GNU_SOURCE
>     #define _GNU_SOURCE
>
> Could you try whether this addresses your problem?
>
> Thanks,
> Adrian
>

From coleen.phillimore at oracle.com  Tue Apr 28 14:10:35 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Tue, 28 Apr 2020 10:10:35 -0400
Subject: RFR: 8244010: Simplify usages of
 ProcessTools.createJavaProcessBuilder in our tests
In-Reply-To: <056ff9b1-1a4f-a1fb-2727-b503707f3750@oracle.com>
References: <60c0e0e4-634b-d349-c6f3-ca1ad52926cb@oracle.com>
 <056ff9b1-1a4f-a1fb-2727-b503707f3750@oracle.com>
Message-ID: <94095aef-266d-fa88-5d36-c59d07b27f3d@oracle.com>


I clicked on the runtime and serviceability tests and they look good to 
me.? This is definitely an improvement.

- pb = ProcessTools.createJavaProcessBuilder(runJava.toArray(new 
String[0]));
+ pb = ProcessTools.createJavaProcessBuilder(runJava);



On 4/28/20 9:54 AM, Stefan Karlsson wrote:
> Hi again,
>
> I realized that we probably want to give ProcessTools.executeTestJvm 
> the same treatment.
>
> Side-note: It's very awkard that createJavaProcessBuilder defaults to 
> not adding user-specifed flags, but executeTestJvm does. I think it 
> would be good to unify this as a separate RFE. I think *a lot* of 
> callers to createJavaProcessBuilder could be simplified by either 
> using executeTestJvm directly, or a simplified version of that.

I never knew there were two.? Why are they different?? It looks like 
there are more createJavaProcessBuilder calls than executeTestjvm ones 
though.? It would be good to have just one.

Thanks,
Coleen

>
> I'm running testing through mach5 and found a few things to fix, I 
> might find more when the testing has proceeded further.
>
> This is the current patch:
> https://cr.openjdk.java.net/~stefank/8244010/webrev.02.delta
> https://cr.openjdk.java.net/~stefank/8244010/webrev.02
>
> Thanks,
> StefanK
>
> On 2020-04-28 13:58, Stefan Karlsson wrote:
>> Hi all,
>>
>> Please review this patch to simplify usages of 
>> ProcessTools.createJavaProcessBuilder in our tests.
>>
>> https://cr.openjdk.java.net/~stefank/8244010/webrev.01/
>> https://bugs.openjdk.java.net/browse/JDK-8244010
>>
>> I saw all this code when reviewing changes to how we pass flags in 
>> our tests. There are a many places where arguments are converted and 
>> passed back and forth in String[] and Collections.
>>
>> For example:
>> ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(
>> ??????????argsList.toArray(new String[argsList.size()]));
>>
>> If we add an overload the createJavaProcessBuilder, that takes a 
>> Collection<String> as an argument, then we can write the code above as:
>> ???ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(argsList);
>>
>> Other places temporarily put the flags in a String[], where most 
>> calls simply lists the arguments in the call:
>> ??String[] opts = {Xmx, "-XX:NativeMemoryTracking=detail", 
>> "-XX:+UseParallelGC", "-version"};
>> ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(opts);
>>
>> And some places put the args in a temporary Collection:
>> ??LinkedList<String> vmOptions = new LinkedList<>();
>> ??vmOptions.add(gc);
>> ??vmOptions.add("-Xmx" + minMaxHeap);
>> ??vmOptions.add("-XX:+PrintFlagsFinal");
>> ??vmOptions.add(VerifyHeapSize.class.getName());
>>
>> ??ProcessBuilder pb = 
>> ProcessTools.createJavaProcessBuilder(vmOptions.toArray(new String[0]));
>>
>> I'd like to cleanup, simplify, and unify many of these usages.
>>
>> I've tested this by running all the changed tests locally.
>>
>> Thanks,
>> StefanK
>


From glaubitz at physik.fu-berlin.de  Tue Apr 28 14:10:56 2020
From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz)
Date: Tue, 28 Apr 2020 16:10:56 +0200
Subject: RFR: 8242244: Remove redundant ELF machine definitions
In-Reply-To: <441206dd-c4cf-bc65-7e28-700939009203@bell-sw.com>
References: <71284763-35dd-6de1-8107-0e06227a2551@physik.fu-berlin.de>
 <22fdf007-a3d5-f14f-4d27-5746ad21a88e@oracle.com>
 <483038a4-7517-e190-cbbe-783da5f68ecc@physik.fu-berlin.de>
 <8b5812dd-78bf-f099-f066-b0bac2cf1572@bell-sw.com>
 <538aabc2-158d-c8ce-08ef-0ae3c62ffc78@physik.fu-berlin.de>
 <441206dd-c4cf-bc65-7e28-700939009203@bell-sw.com>
Message-ID: <d2868fd8-2460-559f-e218-5f46fd80f245@physik.fu-berlin.de>

Hi Dmitry!

On 4/28/20 4:05 PM, Dmitry Chuyko wrote:
> Yes, our builds work after this fix.

Thanks for the confirmation.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz at debian.org
`. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

From coleen.phillimore at oracle.com  Tue Apr 28 14:15:31 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Tue, 28 Apr 2020 10:15:31 -0400
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64 (closed)
In-Reply-To: <f119092f-6239-bf61-c701-b1b45612792c@oracle.com>
References: <1dd1bf71-fbb9-c57c-b033-884b5821033d@oracle.com>
 <f119092f-6239-bf61-c701-b1b45612792c@oracle.com>
Message-ID: <c604dd4c-5e36-2ea8-15da-165b0162c500@oracle.com>

Sorry, this went to the wrong list.

Coleen

On 4/24/20 11:17 AM, coleen.phillimore at oracle.com wrote:
>
> Hi Erik, well the interpreter code isn't as bad as I thought it would 
> be.? We tied UseCompressedKlassPointers to UseCompressedOops because 
> we couldn't imagine why someone would care about 4 bytes per object 
> with all 8 byte pointers to oops.? But maybe you do care.
>
> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/cpu/x86/interp_masm_x86.cpp.udiff.html
>
>        if (UseBiasedLocking) {
> - biased_locking_enter(lock_reg, obj_reg, swap_reg, tmp_reg, false, 
> done, &slow_case);
> + Register rklass_decode_tmp = LP64_ONLY(rscratch1) NOT_LP64(noreg);
> + biased_locking_enter(lock_reg, obj_reg, swap_reg, tmp_reg, 
> rscratch1, false, done, &slow_case);
>        }
> You didn't use rklass_decode_tmp here.
>
> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/cpu/x86/templateTable_x86.cpp.udiff.html
>
>
> + Register tmp_load_klass = LP64_ONLY(rscratch1) NOT_LP64(noreg);
>
> I wonder if you couldn't just pass rscratch1 into load_klass and 
> handle ignoring it inside the load_klass function, ie. fix the 
> asserts.? This gets repetitive in the templateTable and other code.? 
> Maybe people may have the opposite opinion though.
>
> This is quite nice that UseCompressedKlassPointers doesn't have to 
> reinit_heapbase.? The interpreter code looks good.
>
> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/share/memory/metaspace.cpp.frames.html
>
> I don't know if base = HeapBaseMinAddress is right for the compressed 
> class space.? It's reserved but not committed as 1G. Maybe it makes 
> sense since the heap without UseCompressedOops should be above the 
> class space.
>
> Coleen
>
> On 4/24/20 4:25 AM, Erik ?sterlund wrote:
>> Hi,
>>
>> This is the closed version of the mostly open change. There are some 
>> @Contended tests in closed
>> that needed some love with my change to make compressed class 
>> pointers usable with compressed
>> oops disabled. In particular, these tests perform calculations to 
>> ensure that things are padded
>> as expected. Part of those calculations makes assumptions about how 
>> large the object header is,
>> presuming that the number is a constant that is the same for all 
>> objects, and is determinable
>> by checking the offset of the first field. However, with my changes, 
>> the first field will end
>> up either in the klass gap, or after, depending on the field size. I 
>> updated the tests to consider
>> different header sizes depending on the type of fields being used. It 
>> still isn't perfect, but
>> it is clever enough for the purposes of the tests; they pass with 
>> these changes.
>>
>> Bug:
>> https://bugs.openjdk.java.net/browse/JDK-8241825
>>
>> Webrev:
>> http://wikifiles.se.oracle.com/dev/eosterlund/8241825/webrev.00/
>>
>> Thanks,
>> /Erik
>


From coleen.phillimore at oracle.com  Tue Apr 28 14:17:20 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Tue, 28 Apr 2020 10:17:20 -0400
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64
In-Reply-To: <ddbd4b25-beb1-0f4a-20b9-086eef381d1a@oracle.com>
References: <56a0cf85-9698-64b2-95b4-e5b123c3e9e2@oracle.com>
 <B9EAEA33-057A-44C2-A7CF-DFB407776759@oracle.com>
 <42f02810-7a6e-1c7d-3fe3-b892f007ddc4@oracle.com>
 <7cda1c27-e745-e3c5-8c0e-a05a588baa6e@oracle.com>
 <866D9669-4CFD-4ABC-BCC2-3F2B51BDDDA9@oracle.com>
 <ddbd4b25-beb1-0f4a-20b9-086eef381d1a@oracle.com>
Message-ID: <2822d95c-2fb7-a2b6-e61e-ba64614c504b@oracle.com>



On 4/27/20 11:28 AM, Erik ?sterlund wrote:
> Hi Frederic,
>
> I hear your concern about hardcoded offsets making the JVM more fragile.
>
> First of all, let me assure you that you would not be able to run 
> hello world if the offsets ever were to be
> wrong. They are verified during bootstrapping in 
> JavaClasses::check_offsets(), so you have to be quite skilled
> to manage to break anything without immediately noticing it. You 
> literally can't have run a single test or java -version
> without immediately crashing. This makes this significantly less 
> fragile. So I don't think I believe that this is fragile.
>
> Having said that, I do agree it could be annoying and a pain to 
> maintain and update the layout code if such changes
> are occasionally incompatible with the random selection of hardcoded 
> offsets, forcing new awkward updates of said offsets,
> and that hardcoding things in general is undesirable, compared to a 
> more solid solution.
> I had a look at an approach for removing hardcoded offsets completely. 
> It looks something like this:
>
> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00..not_01/
>
> The change is a bit involved and complicated though. Because of that, 
> I would prefer to push it as a follow-up RFE
> focused on removing hardcoded offsets, if you are okay with that. This 
> change is already quite tricky, and I think
> these things ought to be separate.

This is nice and should be a follow-up RFE.? I sent you some comments to 
add to it.? It'd be great to not have hc offsets anymore!

Coleen

>
> The fix involves splitting allocation and code generation of the 
> interpreter (due to dependencies with verification code in i2i adapters
> generated during method loading, requiring the interpreter to be 
> allocated early), computing offsets of jlr.Reference early during
> resolution of well known classes (as is done for java_lang_String and 
> java_lang_Class), because it needs offsets to remove the
> discovered and referent fields from oop maps before its subclasses are 
> initialized, and moving initialization (and generation)
> of the interpreter until after offsets have been computed (no longer 
> having the dependency to method loading and linking).
> This way, all offsets have always been computed (or loaded by CDS) 
> before they are needed, without any need for hardcoding.
>
> What do you think?
>
> Thanks,
> /Erik
>
> On 2020-04-24 16:45, Frederic Parain wrote:
>>
>>> On Apr 24, 2020, at 10:36, coleen.phillimore at oracle.com wrote:
>>>
>>>
>>>
>>> On 4/24/20 10:16 AM, Erik ?sterlund wrote:
>>>> Hi Frederic,
>>>>
>>>> Thanks for reviewing this.
>>>>
>>>> On 2020-04-24 15:28, Frederic Parain wrote:
>>>>> Hi Erik,
>>>>>
>>>>> Why did you removed the code handling classes with hard coded 
>>>>> layouts?
>>>>> This code was there to provide the freedom to play with layout 
>>>>> algorithms
>>>>> without having to deal with these special cases. Without this 
>>>>> code, any
>>>>> modification is now constrained by these special classes.
>>>> Good question. Since I started using the klass gap for fields, even 
>>>> when compressed oops is off, dealing with the
>>>> hardcoded offsets and layouts really became quite awkward, to the 
>>>> extent that I started wondering why do we even have
>>>> special layouts for these things anyway. I don't like special 
>>>> things. Treating it like any other object seemed far
>>>> easier, than matching the special special hardcoded layouts with 
>>>> the hardcoded offsets.
>>>>
>>>> I wouldn't say that the whole layout algorithm is constrained by 
>>>> the fact that there are hardcoded offsets.
>>>> I would say that it's just a matter of whoever changes the layout 
>>>> algorithms such that the hardcoded offsets
>>>> are no longer the same, is just gonna have to update the offset to 
>>>> whatever the correct offsets are with the
>>>> new algorithm. And if you look at the classes where we actually do 
>>>> hardcode the offsets, they are rather vanilla
>>>> with relatively few ways you could change the layout other than 
>>>> changing it around for the sake of it. Especially
>>>> primitive box objects. My fantasy is quite limited how we expect a 
>>>> new layout algorithm is gonna do anything else
>>>> there really.
>>>>
>>>> Philosophically, I really think it is up to the person hardcoding 
>>>> an offset to know what the offset is going to
>>>> be, ratherthan having completely different special (and worse) 
>>>> layouts produced for certain objects, just to
>>>> satisfy the expectations of the hardcoded offsets.Especially if the 
>>>> result is pretty bad memory waste in for
>>>> example Integer box objects, that can be quite plentyful in an 
>>>> application.
>>> We used to have a lot more special hardcoded layouts but most have 
>>> been removed. The reason these few classes are hardcoded is because 
>>> the interpreter generates code using these offsets before 
>>> JavaClasses::compute_offsets is run.? We tried to get rid of all of 
>>> them and maybe there's a way to indirectly refer to them in the 
>>> generated code so we can do so in the future.
>> The proposed changes make the whole system brittle. Restoring the 
>> special handling of these classes during
>> layout computation or making the generated code able to deal with 
>> dynamically computed offsets would be a
>> more robust solution.
>>
>>
>>>> The code to figure out the right offsets if a very small amount of 
>>>> code, compared to the rather large amount of
>>>> code to create special layouts and syncing the special hardcoded 
>>>> layout code with the hardcoded offsets. That
>>>> is why my patch has a net negative amount of code.
>>>>
>>>> Philosophically I also don't like having "special" layouts because 
>>>> we don't trust our ability to hardcode some
>>>> offset to the right value. Almost all "special" things are bad and 
>>>> create unnecessary complications, and hence
>>>> should have very good motivation for being allowed to be special. 
>>>> In this case, I really could not see any reason
>>>> why these layouts would be special, unless it was somehow super 
>>>> hard to figure out what the hardcoded offsets would be.
>>>> But for the classes we currently need hardcoded offsets, the 
>>>> calculations are trivial. So I don't think there is
>>>> enough motivation to have special (bad) layouts for these objects 
>>>> for that reason. It only makes things harder than
>>>> it needs to be, and makes the layout worse.
>>>>
>>>>> And I?d prefer to keep the dispatch in 
>>>>> FieldLayoutBuilder::build_layout()
>>>>> because we have more cases to handle there for the Valhalla 
>>>>> project (the
>>>>> layout algorithm for inline types is different than the one in
>>>>> compute_regular_layout()).
>>>> Okay. There was no need for the dispatching any more since there 
>>>> was only the normal layout and nothing
>>>> else. But I can put that code back into the 
>>>> compute_regular_layout() function if that makes it easier for you
>>>> in Valhalla, and have FieldLayoutBuilder::build_layout() just call 
>>>> that function. Will update in the next webrev.
>> Thank you,
>>
>> Fred
>>
>>>>
>>>>> Regards,
>>>>>
>>>>> Fred
>>>>>
>>>>>
>>>>>> On Apr 24, 2020, at 04:17, Erik ?sterlund 
>>>>>> <erik.osterlund at oracle.com> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Today, the use of compressed class pointers assumes the use of 
>>>>>> compressed oops.
>>>>>> This patch loosens up the relationship between compressed oops 
>>>>>> and compressed
>>>>>> class pointers, so that compressed class pointers can be used 
>>>>>> without compressed
>>>>>> oops. This benefits for example ZGC that wants compressed class 
>>>>>> pointers, but
>>>>>> not compressed oops, effectively giving ZGC 4 bytes smaller objects.
>>>>>>
>>>>>> Much of the complexity of the change is that r12 used to be used 
>>>>>> by compressed
>>>>>> class pointers as some kind of semi-fast temp register that one 
>>>>>> would restore
>>>>>> to the compressed oops heap base (or zero). That made it 
>>>>>> effectively a slightly
>>>>>> optimized spilling mechanism used to find a temp register. I 
>>>>>> replaced that
>>>>>> mechanism with a plain old normal temp register that you pass in 
>>>>>> as a parameter.
>>>>>> The bad news is that I had to find available temp registers in a 
>>>>>> lot of places.
>>>>>> The good news is that almost always were there temp registers 
>>>>>> available for free,
>>>>>> and hence the new temp register is even faster than the old 
>>>>>> optimized spilling
>>>>>> mechanism. Because we almost never need any spilling at all in 
>>>>>> the contexts where
>>>>>> this is used.
>>>>>>
>>>>>> Since I want the 4 new bytes to actually make objects smaller, I 
>>>>>> poked the new
>>>>>> layout code a bit so that the klass gap is made available for 
>>>>>> fields. That used
>>>>>> to be made available only with compressed oops enabled, due to 
>>>>>> restrictions in
>>>>>> the layout code. Our new layout code does not have such 
>>>>>> restrictions, and so
>>>>>> I will make the 4 bytes available for fields when the new layout 
>>>>>> code is used
>>>>>> and compressed class pointers are used.
>>>>>>
>>>>>> Now I'm only fixing this for HotSpot x86_64. Ideally the use of 
>>>>>> compressed oops
>>>>>> and compressed class pointers should not be entangled in other 
>>>>>> platforms and
>>>>>> Graal. But that is beyond the scope of this change, and I will 
>>>>>> let them behave
>>>>>> the way that they used to, to be potentially fixed later.
>>>>>>
>>>>>> Bug:
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8241825
>>>>>>
>>>>>> Webrev:
>>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/
>>>>>>
>>>>>> Testing:
>>>>>> hs-tier1-7
>>>>>>
>>>>>> Thanks,
>>>>>> /Erik
>


From thomas.stuefe at gmail.com  Tue Apr 28 14:54:57 2020
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Tue, 28 Apr 2020 16:54:57 +0200
Subject: RFR(M): 8243392: Remodel CDS/Metaspace storage reservation
Message-ID: <CAA-vtUyVki-DHepyAzORoTK_MwnrezFUB+QsxbWszztsx1x7NA@mail.gmail.com>

Hi all,

Could I have reviews for the following proposal of reworking cds/class
space reservation?

Bug: https://bugs.openjdk.java.net/browse/JDK-8243392

Webrev:
http://cr.openjdk.java.net/~stuefe/webrevs/rework-cds-ccs-reservation/webrev.00/webrev/

(Many thanks to Ioi Lam for so patiently explaining CDS internals to me,
and to Andrew Haley and Nick Gasson for help with aarch64!)

Reservation of the compressed class space is needlessly complicated and has
some minor issues. It can be simplified and made clearer.

The complexity stems from the fact that this area lives at the intersection
of two to three sub systems, depending on how one counts. Metaspace, CDS,
and the platform which may or may not its own view of how to reserve class
space. And this code has been growing organically over time.

One small example:

ReservedSpace Metaspace::reserve_preferred_space(size_t size, size_t
alignment,
                                                 bool large_pages, char
*requested_addr,
                                                 bool use_requested_addr)

which I spent hours decoding, resulting in a very confused mail to
hs-runtime and aarch64-port-dev [2].

This patch attempts to simplify cds and metaspace setup a bit; to comment
implicit knowledge which is not immediately clear; to cleanly abstract
platform concerns like optimized class space placement; and to disentangle
cds from metaspace to solve issues which may bite us later with Elastic
Metaspace [4].

---

The main change is the reworked reservation mechanism. This is based on
Ioi's proposal [5].

When reserving class space, three things must happen:

1) reservation of the space obviously. If cds is active that space must be
in the vicinity of cds archives to be covered by compressed class pointer
encoding.
2) setting up the internal Metaspace structures atop of that space
3) setting up compressed class pointer encoding.

In its current form, Metaspace may or may not do some or all of that in one
function (Metaspace::allocate_metaspace_compressed_klass_ptrs(ReservedSpace
metaspace_rs, char* requested_addr, address cds_base);) - if cds is active,
it will reserve the space for Metaspace and hand it in, otherwise it will
create it itself.

When discussing this in [2], Ioi proposed to move the reservation of the
class space completely out of Metaspace and make it a responsibility of the
caller always. This would reduce some complexity, and this patch follows
the proposal.

I removed Metaspace::allocate_metaspace_compressed_klass_ptrs(ReservedSpace
metaspace_rs, char* requested_addr, address cds_base); and all its sub
functions.

(1) now has to always be done outside - a ReservedSpace for class space has
to be provided by the caller. However, Metaspace now offers a utility
function for reserving space at a "nice" location, and explicitly doing
nothing else:

ReservedSpace
Metaspace::reserve_address_space_for_compressed_classes(size_t size);

this function can be redefined on a platform level for platform optimized
reservation, see below for details.

(2) is taken care of by a new function,
Metaspace::initialize_class_space(ReservedSpace rs)

(3) is taken care of a new function CompressedKlassPointers::initialize(),
see below for details.


So, class space now is set up three explicit steps:

- First, reserve a suitable space by however means you want. For
convenience you may use
Metaspace::reserve_address_space_for_compressed_classes(), or you may roll
your own reservation.
- Next, tell Metaspace to use that range as backing storage for class
space: Metaspace::initialize_class_space(ReservedSpace rs)
- Finally, set up encoding. Encoding is independent from the concept of a
ReservedSpace, it just gets an address range, see below for details.

Separating these steps and moving them out of the responsibility of
Metaspace makes this whole thing more flexible; it also removes unnecessary
knowledge (e.g. Metaspace does not need to know anything about either ccp
encoding or cds).

---

How it comes together:

If CDS is off, we just reserve a space using
Metaspace::reserve_address_space_for_compressed_classes(), initialize it
with Metaspace::initialize_class_space(ReservedSpace rs), then set up
compressed class pointer encoding covering the range of this class space.

If CDS is on (dump time), we reserve large 4G space, either at
SharedBaseAddress or using
Metaspace::reserve_address_space_for_compressed_classes(); we then split
that into 3G archive space and 1G class space; we set up that space with
Metaspace as class space; then we set up compressed class pointer encoding
covering both archive space and cds.

If CDS is on (run time), we reserve a large space, split it into archive
space (large enough to hold both archives) and class space, then basically
proceed as above.

Note that this is almost exactly how things worked before (modulo some
minor fixes, e.g. alignment issues), only the code is reformed and made
more explicit.

---

I moved compressed class pointer setup over to CompressedKlassPointers and
changed the interface:

-void Metaspace::set_narrow_klass_base_and_shift(ReservedSpace
metaspace_rs, address cds_base)
+void CompressedKlassPointers::initialize(address addr, size_t len);

Instead of feeding it a single ReservedSpace, which is supposed to
represent class space, and an optional alternate base if cds is on, now we
give it just an numeric address range. That range marks the limits to where
Klass structures are to be expected, and is the implicit promise that
outside that range no Klass structures will exist, so encoding has to cover
only this range.

This range may contain just the class space; or class space+cds; or
whatever allocation scheme we come up with in the future. Encoding does not
really care how the memory is organized as long as the input range covers
all possible Klass locations. That way we remove knowledge about class
space/cds from compressed class pointer encoding.

Moving it away from metaspace.cpp into the CompressedKlassPointers class
also mirrors CompressedOops::initialize().

---

I renamed _narrow_klass_range to just _range, because strictly speaking
this is the range un-narrow Klass pointers can have.

As for the implementation of CompressedKlassPointers::initialize(address
addr, size_t len), I mimicked very closely what happened before, so there
should be almost no differences. Since "almost no differences" sounds scary
:) here are the differences:

- When CDS is active (dump or run time) we now always, unconditionally, set
the encoding range to 4G. This fixes a theoretical bug discussed on
aarch64-port-dev [1].

- When CDS is not active, we set the encoding range to the minimum required
length. Before, it was left at its default value of 4G.

Both differences only affect aarch64, since they are currently the only one
using the range field in CompressedKlassPointers.

I wanted to add an assert somewhere to test encoding of the very last
address of the CompressedKlassPointers range, again to prevent errors like
[3]. But I did not come up with a good place for this assert which would
cover also the encoding done by C1/C2.

For the same reason I thought about introducing a mode where Klass
structures would be allocated in reverse order, starting at the end of the
ccs, but again left it out as too big a change.

---

OS abstraction: platforms may have restrictions of what constitutes a valid
compressed class pointer encoding base. Or if not, they may have at least
preferences. There was logic like this in metaspace.cpp, which I removed
and cleanly factored out into platform dependent files, giving each
platform the option to add special logic.

These are two new methods:

- bool CompressedKlassPointers::is_valid_base(address p)

to let the platform tell you whether it considers p to be a valid encoding
base. The only platform having these restrictions currently is aarch64.

- ReservedSpace
Metaspace::reserve_address_space_for_compressed_classes(size_t size);

this hands over the process of allocating a range suitable for compressed
class pointer encoding to the platform. Most platforms will allocate just
anywhere, but some platforms may have a better strategy (e.g. trying low
memory first, trying only correctly aligned addresses and so on).

Beforehand, this coding existed in a similar form in metaspace.cpp for
aarch64 and AIX. For now, I left the AIX part out - it seems only half
done, and I want to check further if we even need it, if yes why not on
Linux ppc, and C1 does not seem to support anything other than base+offset
with shift either, but I may be mistaken.

These two methods should give the platform enough control to implement
their own scheme for optimized class space placement without bothering any
shared code about it.

Note about the form, I introduced two new platform dependent files,
"metaspace_<cpu>.cpp" and "compressedOops_<cpu>.cpp". I am not happy about
this but this seems to be what we generally do in hotspot, right?

---

Metaspace reserve alignment vs cds alignment

CDS was using Metaspace reserve alignment for CDS internal purposes. I
guess this was just a copy paste issue. It never caused problems since
Metaspace reserve alignment == page size, but that is not true anymore in
the upcoming Elastic Metaspace where reserve alignment will be larger. This
causes a number of issues.

I separated those two cleanly. CDS now uses os::vm_allocation_granularity.
Metaspace::reserve_alignment is only used in those two places where it is
needed, when CDS creates the address space for class space on behalf of the
Metaspace.

---

Windows special handling in CDS

To simplify coding I removed the windows specific handling which left out
reservation of the archive. This was needed because windows cannot mmap
files into reserved regions. But fallback code exists in filemap.cpp for
this case which just reads in the region instead of mapping it.

Should that turn out to be a performance problem, I will reinstate the
feature. But a simpler way would be reserve the archive and later just
before mmapping the archive file to release the archive space. That would
not only be simpler but give us the best guarantee that that address space
is actually available. But I'd be happy to leave that part out completely
if we do not see any performance problems on windows x64.

---

NMT cannot deal with spaces which are split. This problem manifests in that
bookkeeping for class space is done under "Shared Classes", not "Classes"
as it should. This problem exists today too at dump time and randomly at
run time. But since I simplified the reservation, this problem now shows up
always, whether or not we map at the SharedBaseAddress.
While I could work around this problem, I'd prefer this problem to be
solved at the core, and NMT to have an option to recognize reservation
splits. So I'd rather not put a workaround for this into the patch but
leave it for fixing as a separate issue. I opened this issue to track it
[6].

---

Jtreg tests:

I expanded the CompressedOops/CompressedClassPointers.java. I also extended
them to Windows. The tests now optionally omit strict class space placement
tests, since these tests heavily depend on ASLR and were the reason they
were excluded on Windows. However I think even without checking for class
space placement they make sense, just to see that the VM comes up and lives
with the many different settings we can run in.

---

Tests:

- I ran the patch through Oracles submit repo
- I ran tests manually for aarch64, zero, linux 32bit and windows x64
- The whole battery of nightly tests at SAP, including ppc, ppcle and
aarch64, unfortunately excluding windows because of unrelated errors.
Windows x64 tests will be redone tonight.


Thank you,

Thomas

[1]
https://mail.openjdk.java.net/pipermail/aarch64-port-dev/2020-April/008804.html
[2]
https://mail.openjdk.java.net/pipermail/aarch64-port-dev/2020-April/008757.html
[3] https://bugs.openjdk.java.net/browse/JDK-8193266
[4] https://bugs.openjdk.java.net/browse/JDK-8221173
[5]
https://mail.openjdk.java.net/pipermail/aarch64-port-dev/2020-April/008765.html
[6] https://bugs.openjdk.java.net/browse/JDK-8243535

From matthias.baesken at sap.com  Tue Apr 28 15:29:20 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 28 Apr 2020 15:29:20 +0000
Subject: RFR: 8243389: enhance os::pd_print_cpu_info on linux
In-Reply-To: <AM4PR02MB3057422667C0D81A15B420059AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>
References: <AM0PR02MB541225ED5513768AED6732F093AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
 <AM4PR02MB30579EBA32B978AC69A3F8DD9AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>
 <AM0PR02MB54121E0BA017A19057FAF78193AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
 <AM4PR02MB3057422667C0D81A15B420059AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>
Message-ID: <AM0PR02MB5412CB78884450FCE489DF3493AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>

Thanks, here is a new webrev :

http://cr.openjdk.java.net/~mbaesken/webrevs/8243389.1/

Best regards, Matthias

>Hi Matthias,
>
>I'd use ExtensiveErrorReports for cache related info and the following ones:
>"Maximum cpu frequency:"
>"Minimum cpu frequency:"
>"Current cpu frequency:"
>"Available governors:"



From igor.ignatyev at oracle.com  Tue Apr 28 15:30:45 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Tue, 28 Apr 2020 08:30:45 -0700
Subject: RFR(M/T) : 8243935 : s.h.WhiteBoxPermission must be
 ClassFileInstall-ed together w/ s.h.WhiteBox
In-Reply-To: <ec0209dd-5a25-d5bc-4b43-b01559cc1412@oracle.com>
References: <018B98AF-1DC5-4074-AE80-6601F9FDB867@oracle.com>
 <e5a0d476-c516-b627-920f-9f25f0795961@oracle.com>
 <BBD7C18E-16AC-4BD1-9660-F103FC4F0D41@oracle.com>
 <2d8ef01d-5ace-c55f-6a89-60d8b3a2ced4@oracle.com>
 <c1888765-ca63-64d4-e612-02fcb95932c2@oracle.com>
 <ec0209dd-5a25-d5bc-4b43-b01559cc1412@oracle.com>
Message-ID: <A35F3B72-5B8D-4B8F-B40F-0A815D3F2D7C@oracle.com>

Hi David, Stefan,

there is no issue, this patch is just a spin-off of other patch I was working on. I noticed that some tests copy WhiteBoxPermission and other don't, so I decided to clean this up. I completely forgot about the hack we did in ClassFileInstaller (I guess partially b/c we still have tests which copy WhiteBoxPermission explicitly) so now I think it's better to just remove all explicit copying of WhiteBox$WhiteBoxPermission (which kinda echoes Stefan's patch). this of course touches more files, on the bright side this makes them all neater. so I run sed '/^\s*\*\s*sun.hotspot.WhiteBox\$WhiteBoxPermission/'d to remove all WhiteBoxPermission, and them removed some leftovers in 'containers/docker': 
- full webrev:
  http://cr.openjdk.java.net/~iignatyev//8243935/webrev.01/
> 656 lines changed: 0 ins; 612 del; 44 mod;

- incremental webrev:
  http://cr.openjdk.java.net/~iignatyev//8243935/webrev.0-1/
> 722 lines changed: 0 ins; 719 del; 3 mod; 

if we agree that this is the way to go, I'll repurpose 8243935 to remove WhiteBoxPermission (and obviously will retest the patch).

Thanks,
-- Igor 


> On Apr 28, 2020, at 2:49 AM, David Holmes <david.holmes at oracle.com> wrote:
> 
> On 28/04/2020 5:34 pm, Stefan Karlsson wrote:
>> Hi,
>> I have another idea. Can't we simply remove WhiteBoxPermission and incorporate that functionality in the WhiteBox class? We would then be able to remove all these WhiteBoxPermission lines.
> 
> Interesting though a little hackish. :)
> 
> But why is this an issue anyway we already hacked ClassfileInstaller to ensure it always copies the permission class!!!
> 
> 102     // Add commonly used inner classes that are often omitted by mistake. Currently
> 103     // we support only sun.hotspot.WhiteBox$WhiteBoxPermission. See JDK-8199290
> 104     private static String[] addInnerClasses(String[] classes, int startIdx) {
> 105         boolean seenWB = false;
> 106         boolean seenWBInner = false;
> 107         final String wb = "sun.hotspot.WhiteBox";
> 108         final String wbInner = "sun.hotspot.WhiteBox$WhiteBoxPermission";
> 
> David
> -----
> 
>> I've tested this and it seems to work:
>> https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/ Summary of changes:    695 lines changed: 0 ins; 657 del; 38 mod; 60752 unchg
>> The interesting code:
>> https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/test/lib/sun/hotspot/WhiteBox.java.udiff.html (There are some parts of the patch that was butchered by the sed-script - I'll fix this if we agree that this is a good thing to do)
>> If you think this is something we should do, I'll create an RFE, complete the patch, run more tests, and send it out for review.
>> Thanks,
>> StefanK
>> On 2020-04-28 08:08, David Holmes wrote:
>>> On 28/04/2020 3:37 pm, Igor Ignatyev wrote:
>>>> 
>>>> 
>>>>> On Apr 27, 2020, at 10:03 PM, David Holmes <david.holmes at oracle.com> wrote:
>>>>> 
>>>>> Hi Igor,
>>>>> 
>>>>> On 28/04/2020 12:32 pm, Igor Ignatyev wrote:
>>>>>> http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/
>>>>>>> 338 lines changed: 107 ins; 0 del; 231 mod
>>>>>> Hi all,
>>>>>> could you please review the trivial yet tedious patch which adds 'sun.hotspot.WhiteBox$WhiteBoxPermission' to all 'ClassFileInstaller sun.hotspot.WhiteBox' actions?
>>>>>> the patch also
>>>>>>   - replaces '@run main ClassFileInstaller' w/ '@run driver ClassFileInstaller'
>>>>>>   - unifies the way ClassFileInstaller sun.hotspot.WhiteBox are aligned
>>>>> 
>>>>> That alignment change is unnecessary and makes it very hard to review this patch.
>>>>> 
>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8243935
>>>>>> webrevs:
>>>>>>   - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/ -- regular webrev
>>>>>>   - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.including_ws -- shows "whitespace only" changes
>>>>> 
>>>>> I'd like to see the patch that leaves out the whitespace-only changes - is that possible?
>>>> sure, I've used `hg diff -w` (--ignore-all-space    ignore white space when comparing lines) to remove all whitespace changes, here is webrev -- http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.w/
>>> 
>>> Thanks that was somewhat better. Change seems okay.
>>> 
>>> David
>>> 
>>>> -- Igor
>>>>> 
>>>>> Thanks,
>>>>> David
>>>>> 
>>>>>> Thanks,
>>>>>> -- Igor
>>>> 


From martin.doerr at sap.com  Tue Apr 28 15:35:51 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Tue, 28 Apr 2020 15:35:51 +0000
Subject: RFR: 8243389: enhance os::pd_print_cpu_info on linux
In-Reply-To: <AM0PR02MB5412CB78884450FCE489DF3493AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
References: <AM0PR02MB541225ED5513768AED6732F093AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
 <AM4PR02MB30579EBA32B978AC69A3F8DD9AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>
 <AM0PR02MB54121E0BA017A19057FAF78193AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
 <AM4PR02MB3057422667C0D81A15B420059AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>
 <AM0PR02MB5412CB78884450FCE489DF3493AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
Message-ID: <AM4PR02MB30572CD677351CA02B0143CB9AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>

Thumbs up from my side.

Thanks and best regards,
Martin


> -----Original Message-----
> From: Baesken, Matthias <matthias.baesken at sap.com>
> Sent: Dienstag, 28. April 2020 17:29
> To: Doerr, Martin <martin.doerr at sap.com>; 'hotspot-
> dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: RE: RFR: 8243389: enhance os::pd_print_cpu_info on linux
> 
> Thanks, here is a new webrev :
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8243389.1/
> 
> Best regards, Matthias
> 
> >Hi Matthias,
> >
> >I'd use ExtensiveErrorReports for cache related info and the following ones:
> >"Maximum cpu frequency:"
> >"Minimum cpu frequency:"
> >"Current cpu frequency:"
> >"Available governors:"
> 


From matthias.baesken at sap.com  Tue Apr 28 15:45:14 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 28 Apr 2020 15:45:14 +0000
Subject: RFR: 8243389: enhance os::pd_print_cpu_info on linux
In-Reply-To: <AM4PR02MB30572CD677351CA02B0143CB9AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>
References: <AM0PR02MB541225ED5513768AED6732F093AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
 <AM4PR02MB30579EBA32B978AC69A3F8DD9AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>
 <AM0PR02MB54121E0BA017A19057FAF78193AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
 <AM4PR02MB3057422667C0D81A15B420059AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>
 <AM0PR02MB5412CB78884450FCE489DF3493AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
 <AM4PR02MB30572CD677351CA02B0143CB9AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>
Message-ID: <AM0PR02MB54120B7109B39A1598313A4493AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>


Thanks !

May I have a second review ?


Best regards, Matthias



>Thumbs up from my side.
>
>Thanks and best regards,
>Martin
>
>> 
>> Thanks, here is a new webrev :
>> 
>> http://cr.openjdk.java.net/~mbaesken/webrevs/8243389.1/
>> 


From stefan.karlsson at oracle.com  Tue Apr 28 15:46:36 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Tue, 28 Apr 2020 17:46:36 +0200
Subject: RFR: 8244010: Simplify usages of
 ProcessTools.createJavaProcessBuilder in our tests
In-Reply-To: <94095aef-266d-fa88-5d36-c59d07b27f3d@oracle.com>
References: <60c0e0e4-634b-d349-c6f3-ca1ad52926cb@oracle.com>
 <056ff9b1-1a4f-a1fb-2727-b503707f3750@oracle.com>
 <94095aef-266d-fa88-5d36-c59d07b27f3d@oracle.com>
Message-ID: <75d7f5ee-3c1c-0dc3-147f-f6d445769e64@oracle.com>

On 2020-04-28 16:10, coleen.phillimore at oracle.com wrote:
>
> I clicked on the runtime and serviceability tests and they look good 
> to me.? This is definitely an improvement.
>
> - pb = ProcessTools.createJavaProcessBuilder(runJava.toArray(new 
> String[0]));
> + pb = ProcessTools.createJavaProcessBuilder(runJava);

Thanks. See below:

>
>
>
> On 4/28/20 9:54 AM, Stefan Karlsson wrote:
>> Hi again,
>>
>> I realized that we probably want to give ProcessTools.executeTestJvm 
>> the same treatment.
>>
>> Side-note: It's very awkard that createJavaProcessBuilder defaults to 
>> not adding user-specifed flags, but executeTestJvm does. I think it 
>> would be good to unify this as a separate RFE. I think *a lot* of 
>> callers to createJavaProcessBuilder could be simplified by either 
>> using executeTestJvm directly, or a simplified version of that.
>
> I never knew there were two.? Why are they different?? It looks like 
> there are more createJavaProcessBuilder calls than executeTestjvm ones 
> though.? It would be good to have just one.

One big difference is that one provides a ProcessBuilder and the other 
hides it and returns an OutputAnalyzer:

 ??? public static OutputAnalyzer executeTestJvm(String... cmds) throws 
Exception {
 ??????? ProcessBuilder pb = createJavaProcessBuilder(true, cmds);
 ??????? return executeProcess(pb);
 ??? }

There is a difference between executeProcess and new 
OutputAnalyzer(pb.start()), but if we ignore that and focus on what the 
test code would look like, we could often completely get rid of the 
notion of a ProcessBuilder from our tests.

If we take test/hotspot/jtreg/runtime//logging/ClassResolutionTest.java 
as an example. We could change:

 ? ProcessBuilder pb = 
ProcessTools.createJavaProcessBuilder("-Xlog:class+resolve=debug",
ClassResolutionTestMain.class.getName());
 ? OutputAnalyzer o = new OutputAnalyzer(pb.start());
 ? o.shouldContain("[class,resolve] 
ClassResolutionTest$ClassResolutionTestMain$Thing1Handler 
ClassResolutionTest$ClassResolutionTestMain$Thing1");
 ? o.shouldContain("[class,resolve] resolve JVM_CONSTANT_MethodHandle");

To:

 ? OutputAnalyzer o = 
ProcessTools.executeTestJvm("-Xlog:class+resolve=debug",
ClassResolutionTestMain.class.getName());
 ? o.shouldContain("[class,resolve] 
ClassResolutionTest$ClassResolutionTestMain$Thing1Handler 
ClassResolutionTest$ClassResolutionTestMain$Thing1");
 ? o.shouldContain("[class,resolve] resolve JVM_CONSTANT_MethodHandle");

Or even get rid of the short lived variables (Not sure if it's 
preferable, but it's doable):

 ? ProcessTools.executeTestJvm("-Xlog:class+resolve=debug",
ClassResolutionTestMain.class.getName())
 ????????????? .shouldContain("[class,resolve] 
ClassResolutionTest$ClassResolutionTestMain$Thing1Handler 
ClassResolutionTest$ClassResolutionTestMain$Thing1");
 ????????????? .shouldContain("[class,resolve] resolve 
JVM_CONSTANT_MethodHandle");

StefanK
>
> Thanks,
> Coleen
>
>>
>> I'm running testing through mach5 and found a few things to fix, I 
>> might find more when the testing has proceeded further.
>>
>> This is the current patch:
>> https://cr.openjdk.java.net/~stefank/8244010/webrev.02.delta
>> https://cr.openjdk.java.net/~stefank/8244010/webrev.02
>>
>> Thanks,
>> StefanK
>>
>> On 2020-04-28 13:58, Stefan Karlsson wrote:
>>> Hi all,
>>>
>>> Please review this patch to simplify usages of 
>>> ProcessTools.createJavaProcessBuilder in our tests.
>>>
>>> https://cr.openjdk.java.net/~stefank/8244010/webrev.01/
>>> https://bugs.openjdk.java.net/browse/JDK-8244010
>>>
>>> I saw all this code when reviewing changes to how we pass flags in 
>>> our tests. There are a many places where arguments are converted and 
>>> passed back and forth in String[] and Collections.
>>>
>>> For example:
>>> ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(
>>> ??????????argsList.toArray(new String[argsList.size()]));
>>>
>>> If we add an overload the createJavaProcessBuilder, that takes a 
>>> Collection<String> as an argument, then we can write the code above as:
>>> ???ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(argsList);
>>>
>>> Other places temporarily put the flags in a String[], where most 
>>> calls simply lists the arguments in the call:
>>> ??String[] opts = {Xmx, "-XX:NativeMemoryTracking=detail", 
>>> "-XX:+UseParallelGC", "-version"};
>>> ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(opts);
>>>
>>> And some places put the args in a temporary Collection:
>>> ??LinkedList<String> vmOptions = new LinkedList<>();
>>> ??vmOptions.add(gc);
>>> ??vmOptions.add("-Xmx" + minMaxHeap);
>>> ??vmOptions.add("-XX:+PrintFlagsFinal");
>>> ??vmOptions.add(VerifyHeapSize.class.getName());
>>>
>>> ??ProcessBuilder pb = 
>>> ProcessTools.createJavaProcessBuilder(vmOptions.toArray(new 
>>> String[0]));
>>>
>>> I'd like to cleanup, simplify, and unify many of these usages.
>>>
>>> I've tested this by running all the changed tests locally.
>>>
>>> Thanks,
>>> StefanK
>>
>


From stefan.karlsson at oracle.com  Tue Apr 28 15:57:20 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Tue, 28 Apr 2020 17:57:20 +0200
Subject: RFR(M/T) : 8243935 : s.h.WhiteBoxPermission must be
 ClassFileInstall-ed together w/ s.h.WhiteBox
In-Reply-To: <A35F3B72-5B8D-4B8F-B40F-0A815D3F2D7C@oracle.com>
References: <018B98AF-1DC5-4074-AE80-6601F9FDB867@oracle.com>
 <e5a0d476-c516-b627-920f-9f25f0795961@oracle.com>
 <BBD7C18E-16AC-4BD1-9660-F103FC4F0D41@oracle.com>
 <2d8ef01d-5ace-c55f-6a89-60d8b3a2ced4@oracle.com>
 <c1888765-ca63-64d4-e612-02fcb95932c2@oracle.com>
 <ec0209dd-5a25-d5bc-4b43-b01559cc1412@oracle.com>
 <A35F3B72-5B8D-4B8F-B40F-0A815D3F2D7C@oracle.com>
Message-ID: <0c30a41f-9031-757f-4e26-93a21ed7de6f@oracle.com>

Hi Igor,

On 2020-04-28 17:30, Igor Ignatyev wrote:
> Hi David, Stefan,
>
> there is no issue, this patch is just a spin-off of other patch I was working on. I noticed that some tests copy WhiteBoxPermission and other don't, so I decided to clean this up. I completely forgot about the hack we did in ClassFileInstaller (I guess partially b/c we still have tests which copy WhiteBoxPermission explicitly) so now I think it's better to just remove all explicit copying of WhiteBox$WhiteBoxPermission (which kinda echoes Stefan's patch). this of course touches more files, on the bright side this makes them all neater. so I run sed '/^\s*\*\s*sun.hotspot.WhiteBox\$WhiteBoxPermission/'d to remove all WhiteBoxPermission, and them removed some leftovers in 'containers/docker':
> - full webrev:
>    http://cr.openjdk.java.net/~iignatyev//8243935/webrev.01/

Your open.patch is incorrectly generated and reflects the earlier patch:
https://cr.openjdk.java.net/~iignatyev//8243935/webrev.01/open.changeset

It's a bug in webrev, but I don't remember under what circumstances it 
occurs. Could you rerun with: -r qparent -c 8243935 -C -N and see if it 
works better?

Thanks,
StefanK

>> 656 lines changed: 0 ins; 612 del; 44 mod;
> - incremental webrev:
>    http://cr.openjdk.java.net/~iignatyev//8243935/webrev.0-1/
>> 722 lines changed: 0 ins; 719 del; 3 mod;
> if we agree that this is the way to go, I'll repurpose 8243935 to remove WhiteBoxPermission (and obviously will retest the patch).
>
> Thanks,
> -- Igor
>
>
>> On Apr 28, 2020, at 2:49 AM, David Holmes <david.holmes at oracle.com> wrote:
>>
>> On 28/04/2020 5:34 pm, Stefan Karlsson wrote:
>>> Hi,
>>> I have another idea. Can't we simply remove WhiteBoxPermission and incorporate that functionality in the WhiteBox class? We would then be able to remove all these WhiteBoxPermission lines.
>> Interesting though a little hackish. :)
>>
>> But why is this an issue anyway we already hacked ClassfileInstaller to ensure it always copies the permission class!!!
>>
>> 102     // Add commonly used inner classes that are often omitted by mistake. Currently
>> 103     // we support only sun.hotspot.WhiteBox$WhiteBoxPermission. See JDK-8199290
>> 104     private static String[] addInnerClasses(String[] classes, int startIdx) {
>> 105         boolean seenWB = false;
>> 106         boolean seenWBInner = false;
>> 107         final String wb = "sun.hotspot.WhiteBox";
>> 108         final String wbInner = "sun.hotspot.WhiteBox$WhiteBoxPermission";
>>
>> David
>> -----
>>
>>> I've tested this and it seems to work:
>>> https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/ Summary of changes:    695 lines changed: 0 ins; 657 del; 38 mod; 60752 unchg
>>> The interesting code:
>>> https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/test/lib/sun/hotspot/WhiteBox.java.udiff.html (There are some parts of the patch that was butchered by the sed-script - I'll fix this if we agree that this is a good thing to do)
>>> If you think this is something we should do, I'll create an RFE, complete the patch, run more tests, and send it out for review.
>>> Thanks,
>>> StefanK
>>> On 2020-04-28 08:08, David Holmes wrote:
>>>> On 28/04/2020 3:37 pm, Igor Ignatyev wrote:
>>>>>
>>>>>> On Apr 27, 2020, at 10:03 PM, David Holmes <david.holmes at oracle.com> wrote:
>>>>>>
>>>>>> Hi Igor,
>>>>>>
>>>>>> On 28/04/2020 12:32 pm, Igor Ignatyev wrote:
>>>>>>> http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/
>>>>>>>> 338 lines changed: 107 ins; 0 del; 231 mod
>>>>>>> Hi all,
>>>>>>> could you please review the trivial yet tedious patch which adds 'sun.hotspot.WhiteBox$WhiteBoxPermission' to all 'ClassFileInstaller sun.hotspot.WhiteBox' actions?
>>>>>>> the patch also
>>>>>>>    - replaces '@run main ClassFileInstaller' w/ '@run driver ClassFileInstaller'
>>>>>>>    - unifies the way ClassFileInstaller sun.hotspot.WhiteBox are aligned
>>>>>> That alignment change is unnecessary and makes it very hard to review this patch.
>>>>>>
>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8243935
>>>>>>> webrevs:
>>>>>>>    - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/ -- regular webrev
>>>>>>>    - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.including_ws -- shows "whitespace only" changes
>>>>>> I'd like to see the patch that leaves out the whitespace-only changes - is that possible?
>>>>> sure, I've used `hg diff -w` (--ignore-all-space    ignore white space when comparing lines) to remove all whitespace changes, here is webrev -- http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.w/
>>>> Thanks that was somewhat better. Change seems okay.
>>>>
>>>> David
>>>>
>>>>> -- Igor
>>>>>> Thanks,
>>>>>> David
>>>>>>
>>>>>>> Thanks,
>>>>>>> -- Igor


From igor.ignatyev at oracle.com  Tue Apr 28 16:16:20 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Tue, 28 Apr 2020 09:16:20 -0700
Subject: RFR(M/T) : 8243935 : s.h.WhiteBoxPermission must be
 ClassFileInstall-ed together w/ s.h.WhiteBox
In-Reply-To: <0c30a41f-9031-757f-4e26-93a21ed7de6f@oracle.com>
References: <018B98AF-1DC5-4074-AE80-6601F9FDB867@oracle.com>
 <e5a0d476-c516-b627-920f-9f25f0795961@oracle.com>
 <BBD7C18E-16AC-4BD1-9660-F103FC4F0D41@oracle.com>
 <2d8ef01d-5ace-c55f-6a89-60d8b3a2ced4@oracle.com>
 <c1888765-ca63-64d4-e612-02fcb95932c2@oracle.com>
 <ec0209dd-5a25-d5bc-4b43-b01559cc1412@oracle.com>
 <A35F3B72-5B8D-4B8F-B40F-0A815D3F2D7C@oracle.com>
 <0c30a41f-9031-757f-4e26-93a21ed7de6f@oracle.com>
Message-ID: <86A49CAF-3C66-4D64-801A-197E38A4BF84@oracle.com>

Hi Stefan,

I guess you are interested only in an actual patch, if so I would rather just use hg diff to get the patch and attach/upload it b/c, by some reasons, webrev takes a lot of time to be generated.

-- Igor 

> On Apr 28, 2020, at 8:57 AM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
> 
> Hi Igor,
> 
> On 2020-04-28 17:30, Igor Ignatyev wrote:
>> Hi David, Stefan,
>> 
>> there is no issue, this patch is just a spin-off of other patch I was working on. I noticed that some tests copy WhiteBoxPermission and other don't, so I decided to clean this up. I completely forgot about the hack we did in ClassFileInstaller (I guess partially b/c we still have tests which copy WhiteBoxPermission explicitly) so now I think it's better to just remove all explicit copying of WhiteBox$WhiteBoxPermission (which kinda echoes Stefan's patch). this of course touches more files, on the bright side this makes them all neater. so I run sed '/^\s*\*\s*sun.hotspot.WhiteBox\$WhiteBoxPermission/'d to remove all WhiteBoxPermission, and them removed some leftovers in 'containers/docker':
>> - full webrev:
>>   http://cr.openjdk.java.net/~iignatyev//8243935/webrev.01/
> 
> Your open.patch is incorrectly generated and reflects the earlier patch:
> https://cr.openjdk.java.net/~iignatyev//8243935/webrev.01/open.changeset
> 
> It's a bug in webrev, but I don't remember under what circumstances it occurs. Could you rerun with: -r qparent -c 8243935 -C -N and see if it works better?
> 
> Thanks,
> StefanK
> 
>>> 656 lines changed: 0 ins; 612 del; 44 mod;
>> - incremental webrev:
>>   http://cr.openjdk.java.net/~iignatyev//8243935/webrev.0-1/
>>> 722 lines changed: 0 ins; 719 del; 3 mod;
>> if we agree that this is the way to go, I'll repurpose 8243935 to remove WhiteBoxPermission (and obviously will retest the patch).
>> 
>> Thanks,
>> -- Igor
>> 
>> 
>>> On Apr 28, 2020, at 2:49 AM, David Holmes <david.holmes at oracle.com> wrote:
>>> 
>>> On 28/04/2020 5:34 pm, Stefan Karlsson wrote:
>>>> Hi,
>>>> I have another idea. Can't we simply remove WhiteBoxPermission and incorporate that functionality in the WhiteBox class? We would then be able to remove all these WhiteBoxPermission lines.
>>> Interesting though a little hackish. :)
>>> 
>>> But why is this an issue anyway we already hacked ClassfileInstaller to ensure it always copies the permission class!!!
>>> 
>>> 102     // Add commonly used inner classes that are often omitted by mistake. Currently
>>> 103     // we support only sun.hotspot.WhiteBox$WhiteBoxPermission. See JDK-8199290
>>> 104     private static String[] addInnerClasses(String[] classes, int startIdx) {
>>> 105         boolean seenWB = false;
>>> 106         boolean seenWBInner = false;
>>> 107         final String wb = "sun.hotspot.WhiteBox";
>>> 108         final String wbInner = "sun.hotspot.WhiteBox$WhiteBoxPermission";
>>> 
>>> David
>>> -----
>>> 
>>>> I've tested this and it seems to work:
>>>> https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/ Summary of changes:    695 lines changed: 0 ins; 657 del; 38 mod; 60752 unchg
>>>> The interesting code:
>>>> https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/test/lib/sun/hotspot/WhiteBox.java.udiff.html (There are some parts of the patch that was butchered by the sed-script - I'll fix this if we agree that this is a good thing to do)
>>>> If you think this is something we should do, I'll create an RFE, complete the patch, run more tests, and send it out for review.
>>>> Thanks,
>>>> StefanK
>>>> On 2020-04-28 08:08, David Holmes wrote:
>>>>> On 28/04/2020 3:37 pm, Igor Ignatyev wrote:
>>>>>> 
>>>>>>> On Apr 27, 2020, at 10:03 PM, David Holmes <david.holmes at oracle.com> wrote:
>>>>>>> 
>>>>>>> Hi Igor,
>>>>>>> 
>>>>>>> On 28/04/2020 12:32 pm, Igor Ignatyev wrote:
>>>>>>>> http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/
>>>>>>>>> 338 lines changed: 107 ins; 0 del; 231 mod
>>>>>>>> Hi all,
>>>>>>>> could you please review the trivial yet tedious patch which adds 'sun.hotspot.WhiteBox$WhiteBoxPermission' to all 'ClassFileInstaller sun.hotspot.WhiteBox' actions?
>>>>>>>> the patch also
>>>>>>>>   - replaces '@run main ClassFileInstaller' w/ '@run driver ClassFileInstaller'
>>>>>>>>   - unifies the way ClassFileInstaller sun.hotspot.WhiteBox are aligned
>>>>>>> That alignment change is unnecessary and makes it very hard to review this patch.
>>>>>>> 
>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8243935
>>>>>>>> webrevs:
>>>>>>>>   - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/ -- regular webrev
>>>>>>>>   - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.including_ws -- shows "whitespace only" changes
>>>>>>> I'd like to see the patch that leaves out the whitespace-only changes - is that possible?
>>>>>> sure, I've used `hg diff -w` (--ignore-all-space    ignore white space when comparing lines) to remove all whitespace changes, here is webrev -- http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.w/
>>>>> Thanks that was somewhat better. Change seems okay.
>>>>> 
>>>>> David
>>>>> 
>>>>>> -- Igor
>>>>>>> Thanks,
>>>>>>> David
>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> -- Igor
> 


From stefan.karlsson at oracle.com  Tue Apr 28 16:18:10 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Tue, 28 Apr 2020 18:18:10 +0200
Subject: RFR(M/T) : 8243935 : s.h.WhiteBoxPermission must be
 ClassFileInstall-ed together w/ s.h.WhiteBox
In-Reply-To: <86A49CAF-3C66-4D64-801A-197E38A4BF84@oracle.com>
References: <018B98AF-1DC5-4074-AE80-6601F9FDB867@oracle.com>
 <e5a0d476-c516-b627-920f-9f25f0795961@oracle.com>
 <BBD7C18E-16AC-4BD1-9660-F103FC4F0D41@oracle.com>
 <2d8ef01d-5ace-c55f-6a89-60d8b3a2ced4@oracle.com>
 <c1888765-ca63-64d4-e612-02fcb95932c2@oracle.com>
 <ec0209dd-5a25-d5bc-4b43-b01559cc1412@oracle.com>
 <A35F3B72-5B8D-4B8F-B40F-0A815D3F2D7C@oracle.com>
 <0c30a41f-9031-757f-4e26-93a21ed7de6f@oracle.com>
 <86A49CAF-3C66-4D64-801A-197E38A4BF84@oracle.com>
Message-ID: <90d19c71-8e60-9e24-6792-3606b0015185@oracle.com>

On 2020-04-28 18:16, Igor Ignatyev wrote:
> Hi Stefan,
>
> I guess you are interested only in an actual patch, if so I would rather just use hg diff to get the patch and attach/upload it b/c, by some reasons, webrev takes a lot of time to be generated.

Sure. hg diff works.

StefanK

>
> -- Igor
>
>> On Apr 28, 2020, at 8:57 AM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
>>
>> Hi Igor,
>>
>> On 2020-04-28 17:30, Igor Ignatyev wrote:
>>> Hi David, Stefan,
>>>
>>> there is no issue, this patch is just a spin-off of other patch I was working on. I noticed that some tests copy WhiteBoxPermission and other don't, so I decided to clean this up. I completely forgot about the hack we did in ClassFileInstaller (I guess partially b/c we still have tests which copy WhiteBoxPermission explicitly) so now I think it's better to just remove all explicit copying of WhiteBox$WhiteBoxPermission (which kinda echoes Stefan's patch). this of course touches more files, on the bright side this makes them all neater. so I run sed '/^\s*\*\s*sun.hotspot.WhiteBox\$WhiteBoxPermission/'d to remove all WhiteBoxPermission, and them removed some leftovers in 'containers/docker':
>>> - full webrev:
>>>    http://cr.openjdk.java.net/~iignatyev//8243935/webrev.01/
>> Your open.patch is incorrectly generated and reflects the earlier patch:
>> https://cr.openjdk.java.net/~iignatyev//8243935/webrev.01/open.changeset
>>
>> It's a bug in webrev, but I don't remember under what circumstances it occurs. Could you rerun with: -r qparent -c 8243935 -C -N and see if it works better?
>>
>> Thanks,
>> StefanK
>>
>>>> 656 lines changed: 0 ins; 612 del; 44 mod;
>>> - incremental webrev:
>>>    http://cr.openjdk.java.net/~iignatyev//8243935/webrev.0-1/
>>>> 722 lines changed: 0 ins; 719 del; 3 mod;
>>> if we agree that this is the way to go, I'll repurpose 8243935 to remove WhiteBoxPermission (and obviously will retest the patch).
>>>
>>> Thanks,
>>> -- Igor
>>>
>>>
>>>> On Apr 28, 2020, at 2:49 AM, David Holmes <david.holmes at oracle.com> wrote:
>>>>
>>>> On 28/04/2020 5:34 pm, Stefan Karlsson wrote:
>>>>> Hi,
>>>>> I have another idea. Can't we simply remove WhiteBoxPermission and incorporate that functionality in the WhiteBox class? We would then be able to remove all these WhiteBoxPermission lines.
>>>> Interesting though a little hackish. :)
>>>>
>>>> But why is this an issue anyway we already hacked ClassfileInstaller to ensure it always copies the permission class!!!
>>>>
>>>> 102     // Add commonly used inner classes that are often omitted by mistake. Currently
>>>> 103     // we support only sun.hotspot.WhiteBox$WhiteBoxPermission. See JDK-8199290
>>>> 104     private static String[] addInnerClasses(String[] classes, int startIdx) {
>>>> 105         boolean seenWB = false;
>>>> 106         boolean seenWBInner = false;
>>>> 107         final String wb = "sun.hotspot.WhiteBox";
>>>> 108         final String wbInner = "sun.hotspot.WhiteBox$WhiteBoxPermission";
>>>>
>>>> David
>>>> -----
>>>>
>>>>> I've tested this and it seems to work:
>>>>> https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/ Summary of changes:    695 lines changed: 0 ins; 657 del; 38 mod; 60752 unchg
>>>>> The interesting code:
>>>>> https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/test/lib/sun/hotspot/WhiteBox.java.udiff.html (There are some parts of the patch that was butchered by the sed-script - I'll fix this if we agree that this is a good thing to do)
>>>>> If you think this is something we should do, I'll create an RFE, complete the patch, run more tests, and send it out for review.
>>>>> Thanks,
>>>>> StefanK
>>>>> On 2020-04-28 08:08, David Holmes wrote:
>>>>>> On 28/04/2020 3:37 pm, Igor Ignatyev wrote:
>>>>>>>> On Apr 27, 2020, at 10:03 PM, David Holmes <david.holmes at oracle.com> wrote:
>>>>>>>>
>>>>>>>> Hi Igor,
>>>>>>>>
>>>>>>>> On 28/04/2020 12:32 pm, Igor Ignatyev wrote:
>>>>>>>>> http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/
>>>>>>>>>> 338 lines changed: 107 ins; 0 del; 231 mod
>>>>>>>>> Hi all,
>>>>>>>>> could you please review the trivial yet tedious patch which adds 'sun.hotspot.WhiteBox$WhiteBoxPermission' to all 'ClassFileInstaller sun.hotspot.WhiteBox' actions?
>>>>>>>>> the patch also
>>>>>>>>>    - replaces '@run main ClassFileInstaller' w/ '@run driver ClassFileInstaller'
>>>>>>>>>    - unifies the way ClassFileInstaller sun.hotspot.WhiteBox are aligned
>>>>>>>> That alignment change is unnecessary and makes it very hard to review this patch.
>>>>>>>>
>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8243935
>>>>>>>>> webrevs:
>>>>>>>>>    - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/ -- regular webrev
>>>>>>>>>    - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.including_ws -- shows "whitespace only" changes
>>>>>>>> I'd like to see the patch that leaves out the whitespace-only changes - is that possible?
>>>>>>> sure, I've used `hg diff -w` (--ignore-all-space    ignore white space when comparing lines) to remove all whitespace changes, here is webrev -- http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.w/
>>>>>> Thanks that was somewhat better. Change seems okay.
>>>>>>
>>>>>> David
>>>>>>
>>>>>>> -- Igor
>>>>>>>> Thanks,
>>>>>>>> David
>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> -- Igor


From igor.ignatyev at oracle.com  Tue Apr 28 16:23:50 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Tue, 28 Apr 2020 09:23:50 -0700
Subject: RFR(M/T) : 8243935 : s.h.WhiteBoxPermission must be
 ClassFileInstall-ed together w/ s.h.WhiteBox
In-Reply-To: <90d19c71-8e60-9e24-6792-3606b0015185@oracle.com>
References: <018B98AF-1DC5-4074-AE80-6601F9FDB867@oracle.com>
 <e5a0d476-c516-b627-920f-9f25f0795961@oracle.com>
 <BBD7C18E-16AC-4BD1-9660-F103FC4F0D41@oracle.com>
 <2d8ef01d-5ace-c55f-6a89-60d8b3a2ced4@oracle.com>
 <c1888765-ca63-64d4-e612-02fcb95932c2@oracle.com>
 <ec0209dd-5a25-d5bc-4b43-b01559cc1412@oracle.com>
 <A35F3B72-5B8D-4B8F-B40F-0A815D3F2D7C@oracle.com>
 <0c30a41f-9031-757f-4e26-93a21ed7de6f@oracle.com>
 <86A49CAF-3C66-4D64-801A-197E38A4BF84@oracle.com>
 <90d19c71-8e60-9e24-6792-3606b0015185@oracle.com>
Message-ID: <F1DC62C9-43AE-49EF-AF6E-D12A87D481F4@oracle.com>

uploaded http://cr.openjdk.java.net/~iignatyev/8243935/8243935.01.patch

-- Igor

> On Apr 28, 2020, at 9:18 AM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
> 
> On 2020-04-28 18:16, Igor Ignatyev wrote:
>> Hi Stefan,
>> 
>> I guess you are interested only in an actual patch, if so I would rather just use hg diff to get the patch and attach/upload it b/c, by some reasons, webrev takes a lot of time to be generated.
> 
> Sure. hg diff works.
> 
> StefanK
> 
>> 
>> -- Igor
>> 
>>> On Apr 28, 2020, at 8:57 AM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
>>> 
>>> Hi Igor,
>>> 
>>> On 2020-04-28 17:30, Igor Ignatyev wrote:
>>>> Hi David, Stefan,
>>>> 
>>>> there is no issue, this patch is just a spin-off of other patch I was working on. I noticed that some tests copy WhiteBoxPermission and other don't, so I decided to clean this up. I completely forgot about the hack we did in ClassFileInstaller (I guess partially b/c we still have tests which copy WhiteBoxPermission explicitly) so now I think it's better to just remove all explicit copying of WhiteBox$WhiteBoxPermission (which kinda echoes Stefan's patch). this of course touches more files, on the bright side this makes them all neater. so I run sed '/^\s*\*\s*sun.hotspot.WhiteBox\$WhiteBoxPermission/'d to remove all WhiteBoxPermission, and them removed some leftovers in 'containers/docker':
>>>> - full webrev:
>>>>   http://cr.openjdk.java.net/~iignatyev//8243935/webrev.01/
>>> Your open.patch is incorrectly generated and reflects the earlier patch:
>>> https://cr.openjdk.java.net/~iignatyev//8243935/webrev.01/open.changeset
>>> 
>>> It's a bug in webrev, but I don't remember under what circumstances it occurs. Could you rerun with: -r qparent -c 8243935 -C -N and see if it works better?
>>> 
>>> Thanks,
>>> StefanK
>>> 
>>>>> 656 lines changed: 0 ins; 612 del; 44 mod;
>>>> - incremental webrev:
>>>>   http://cr.openjdk.java.net/~iignatyev//8243935/webrev.0-1/
>>>>> 722 lines changed: 0 ins; 719 del; 3 mod;
>>>> if we agree that this is the way to go, I'll repurpose 8243935 to remove WhiteBoxPermission (and obviously will retest the patch).
>>>> 
>>>> Thanks,
>>>> -- Igor
>>>> 
>>>> 
>>>>> On Apr 28, 2020, at 2:49 AM, David Holmes <david.holmes at oracle.com> wrote:
>>>>> 
>>>>> On 28/04/2020 5:34 pm, Stefan Karlsson wrote:
>>>>>> Hi,
>>>>>> I have another idea. Can't we simply remove WhiteBoxPermission and incorporate that functionality in the WhiteBox class? We would then be able to remove all these WhiteBoxPermission lines.
>>>>> Interesting though a little hackish. :)
>>>>> 
>>>>> But why is this an issue anyway we already hacked ClassfileInstaller to ensure it always copies the permission class!!!
>>>>> 
>>>>> 102     // Add commonly used inner classes that are often omitted by mistake. Currently
>>>>> 103     // we support only sun.hotspot.WhiteBox$WhiteBoxPermission. See JDK-8199290
>>>>> 104     private static String[] addInnerClasses(String[] classes, int startIdx) {
>>>>> 105         boolean seenWB = false;
>>>>> 106         boolean seenWBInner = false;
>>>>> 107         final String wb = "sun.hotspot.WhiteBox";
>>>>> 108         final String wbInner = "sun.hotspot.WhiteBox$WhiteBoxPermission";
>>>>> 
>>>>> David
>>>>> -----
>>>>> 
>>>>>> I've tested this and it seems to work:
>>>>>> https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/ Summary of changes:    695 lines changed: 0 ins; 657 del; 38 mod; 60752 unchg
>>>>>> The interesting code:
>>>>>> https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/test/lib/sun/hotspot/WhiteBox.java.udiff.html (There are some parts of the patch that was butchered by the sed-script - I'll fix this if we agree that this is a good thing to do)
>>>>>> If you think this is something we should do, I'll create an RFE, complete the patch, run more tests, and send it out for review.
>>>>>> Thanks,
>>>>>> StefanK
>>>>>> On 2020-04-28 08:08, David Holmes wrote:
>>>>>>> On 28/04/2020 3:37 pm, Igor Ignatyev wrote:
>>>>>>>>> On Apr 27, 2020, at 10:03 PM, David Holmes <david.holmes at oracle.com> wrote:
>>>>>>>>> 
>>>>>>>>> Hi Igor,
>>>>>>>>> 
>>>>>>>>> On 28/04/2020 12:32 pm, Igor Ignatyev wrote:
>>>>>>>>>> http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/
>>>>>>>>>>> 338 lines changed: 107 ins; 0 del; 231 mod
>>>>>>>>>> Hi all,
>>>>>>>>>> could you please review the trivial yet tedious patch which adds 'sun.hotspot.WhiteBox$WhiteBoxPermission' to all 'ClassFileInstaller sun.hotspot.WhiteBox' actions?
>>>>>>>>>> the patch also
>>>>>>>>>>   - replaces '@run main ClassFileInstaller' w/ '@run driver ClassFileInstaller'
>>>>>>>>>>   - unifies the way ClassFileInstaller sun.hotspot.WhiteBox are aligned
>>>>>>>>> That alignment change is unnecessary and makes it very hard to review this patch.
>>>>>>>>> 
>>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8243935
>>>>>>>>>> webrevs:
>>>>>>>>>>   - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/ -- regular webrev
>>>>>>>>>>   - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.including_ws -- shows "whitespace only" changes
>>>>>>>>> I'd like to see the patch that leaves out the whitespace-only changes - is that possible?
>>>>>>>> sure, I've used `hg diff -w` (--ignore-all-space    ignore white space when comparing lines) to remove all whitespace changes, here is webrev -- http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.w/
>>>>>>> Thanks that was somewhat better. Change seems okay.
>>>>>>> 
>>>>>>> David
>>>>>>> 
>>>>>>>> -- Igor
>>>>>>>>> Thanks,
>>>>>>>>> David
>>>>>>>>> 
>>>>>>>>>> Thanks,
>>>>>>>>>> -- Igor
> 


From erik.osterlund at oracle.com  Tue Apr 28 16:28:21 2020
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Tue, 28 Apr 2020 18:28:21 +0200
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64
In-Reply-To: <CAA-vtUwYFCwT1a0Qbcrqmo6OVLit3DROq0jC8WqAYEd3_3XY9A@mail.gmail.com>
References: <56a0cf85-9698-64b2-95b4-e5b123c3e9e2@oracle.com>
 <CAA-vtUyFiMpf2vSXLf-c9uaKYXF1tSA3At5KJySCiz2skG0w3A@mail.gmail.com>
 <9a314ebe-5499-ecb6-3c5b-0c8ce7f9c8d2@oracle.com>
 <CAA-vtUwYFCwT1a0Qbcrqmo6OVLit3DROq0jC8WqAYEd3_3XY9A@mail.gmail.com>
Message-ID: <44febab6-5703-1c8b-d4ad-0f4c12517afa@oracle.com>

Hi Thomas,

Thanks for reviewing this.

On 2020-04-28 13:14, Thomas St?fe wrote:
> Hi Erik,
>
> On Tue, Apr 28, 2020 at 10:07 AM Erik ?sterlund 
> <erik.osterlund at oracle.com <mailto:erik.osterlund at oracle.com>> wrote:
>
>     Hi Thomas,
>
>     On 2020-04-28 08:43, Thomas St?fe wrote:
>>     Hi Erik,
>>
>>     nice change, and now I do not have to keep in mind "heap>32G = no
>>     class space" :)
>
>     Glad you liked it!
>
>>     I eyeballed the metaspace changes (what little there are).
>>
>>     This:
>>
>>     - char* base = (char*)align_up(CompressedOops::end(),
>>     _reserve_alignment);+ char* base;+ if (UseCompressedOops) {+ base
>>     = (char*)align_up(CompressedOops::end(), _reserve_alignment);+ }
>>     else {+ base = (char*)HeapBaseMinAddress;+ }
>>     may not work as intended. You probably want the end of the reserved heap as attach point for ccs, not the HeapBaseMinAddress.
>
>     Could you please expand on that? To make it clear, what this code
>     does is to put metaspace at the
>     minimum address we are allowed to map, when compressed oops is
>     disabled. The reasoning for doing
>     that is that if the low address bits (resulting in good compressed
>     pointers) are not used by compressed oops,
>     then they are available to be used by compressed class pointers
>     instead, such that at least compressed class
>     pointers can utilize this VA to get optimal encoding of class
>     pointers.
>
>     Is there anything not desirable about this? If so, I can't see it,
>     and would appreciate if you expand on why
>     that would be a bad idea. Perhaps there is something I do not see
>     here.
>
>
> Oh, this was a long standing confusion in my head. I never noticed 
> that HeapBaseMinAddress exists for the sole purpose of compressed 
> oops, and is ignored otherwise. So I thought what you do there is just 
> mapping ccs over the start of where the?user wants the heap to start, 
> and since that may not work we would fall back to map-anywhere.
>
> So you are saying HeapBaseMinAddress is the lowest address the 
> platform allows us to map to, irregardless of whether we run with 
> compressed oops or not.

Right, so the purpose of this flag was to be able to move the heap up, 
because if we map things too low, we ran into conflicts with sbrk, and 
hence some old suboptimal malloc implementations, such as the default 
malloc on Solaris. This called for a flag to be able to move the heap 
up, putting it out of the way from sbrk. That is how HeapBaseMinAddress 
came about, AFAIK. The situation when mapping in metaspace and 
compressed class space, without compressed oops, is very much the same: 
we want to go as low as possible for optimal encoding, but not too low 
so we run into these problems.

> With compressed oops we continue to squeeze ccs in after the heap in 
> the hope that it all still fits into the lower 4G or 32G. (Do we ever 
> bother to check if heap is actually reserved in low area? It may have 
> given up and reserved upstate instead, leaving free room below...).

Right. And yes I'm pretty sure we do check. The heap reserves memory, 
and sets the CompressedOops::end() based on that, which is fed as input 
to the metaspace/CDS placement.

> Now with your patch without compressed oops we attempt to map at 
> HeapBaseMinAddress since this is the lowest address we are allowed to 
> map at and the heap won't obviously need it,?right?

Exactly.

> My only misgiving would be that?HeapBaseMinAddress is even more of a 
> misnomer (its already now a?bit, witness my year long confusion :-), 
> since it now controls not only heap placement but ccs too. Actually if 
> it is that important to avoid mapping to addresses 
> below?HeapBaseMinAddress we should probably prevent it somewhere below 
> in the os layer but I really do not want to open that can of worms.

Yes agreed. Ideally it would be called something like MappingMinAddress 
instead, because it really isn't tied to the heap really, it just 
happened to be the primary user of mapping things at low addresses. But 
I also think changing that, and enforcing it more widely in the os layer 
sounds like a different can of worms.

> --
>
> Some more remarks about arguments.cpp:
>
> - In Arguments::set_use_compressed_klass_ptrs(), if now compressed 
> class pointers are independent from compressed oops, checking 
> CompressedClassSpaceSize for validity should now always happen for 
> UseCompressedClassPointers=1.

I think it looks like I do that.

> - If HeapBaseMinAddress is now always used, we should check it always 
> too (see Arguments::set_heap_size)

Good idea. Fixed.

>
>>     Also, could we have some?tests which exercise cds/metaspace
>>     initialization without coops and with ccps? Easiest way would be
>>     to extend "CompressedOops/CompressedClassPointers.java" by some
>>     cases.
>
>
> Thank you. I think this makes sense especially since compressed class 
> pointer encoding is restricted to one mode if CDS is on (base = shared 
> address and shift=3), and since CDS=on is the default zerobased 
> encoding may not happen that often.

Yup. I added some explicit test cases for -XX:-UseCompressedOops 
-XX:+UseCompressedClassPointers as suggested. This will also be the 
default configuration for ZGC, so that should give that configuration 
more test coverage in the various tests we have.

Here is a new webrev:
http://cr.openjdk.java.net/~eosterlund/8241825/webrev.01/

Incremental:
http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00_01/

I hope I fixed everything you wanted in this patch. I also fixed what 
Frederic and Coleen wanted.

Thanks,
/Erik

>     I will have a look at that for the next webrev.
>
>     Thanks for the review.
>
>     /Erik
>
>
>
> Thanks,
>
> Thomas
>
>>     Thank you, Thomas
>>
>>     On Fri, Apr 24, 2020 at 10:19 AM Erik ?sterlund
>>     <erik.osterlund at oracle.com <mailto:erik.osterlund at oracle.com>> wrote:
>>
>>         Hi,
>>
>>         Today, the use of compressed class pointers assumes the use of
>>         compressed oops.
>>         This patch loosens up the relationship between compressed
>>         oops and
>>         compressed
>>         class pointers, so that compressed class pointers can be used
>>         without
>>         compressed
>>         oops. This benefits for example ZGC that wants compressed class
>>         pointers, but
>>         not compressed oops, effectively giving ZGC 4 bytes smaller
>>         objects.
>>
>>         Much of the complexity of the change is that r12 used to be
>>         used by
>>         compressed
>>         class pointers as some kind of semi-fast temp register that
>>         one would
>>         restore
>>         to the compressed oops heap base (or zero). That made it
>>         effectively a
>>         slightly
>>         optimized spilling mechanism used to find a temp register. I
>>         replaced that
>>         mechanism with a plain old normal temp register that you pass
>>         in as a
>>         parameter.
>>         The bad news is that I had to find available temp registers
>>         in a lot of
>>         places.
>>         The good news is that almost always were there temp registers
>>         available
>>         for free,
>>         and hence the new temp register is even faster than the old
>>         optimized
>>         spilling
>>         mechanism. Because we almost never need any spilling at all
>>         in the
>>         contexts where
>>         this is used.
>>
>>         Since I want the 4 new bytes to actually make objects
>>         smaller, I poked
>>         the new
>>         layout code a bit so that the klass gap is made available for
>>         fields.
>>         That used
>>         to be made available only with compressed oops enabled, due to
>>         restrictions in
>>         the layout code. Our new layout code does not have such
>>         restrictions, and so
>>         I will make the 4 bytes available for fields when the new
>>         layout code is
>>         used
>>         and compressed class pointers are used.
>>
>>         Now I'm only fixing this for HotSpot x86_64. Ideally the use of
>>         compressed oops
>>         and compressed class pointers should not be entangled in
>>         other platforms and
>>         Graal. But that is beyond the scope of this change, and I
>>         will let them
>>         behave
>>         the way that they used to, to be potentially fixed later.
>>
>>         Bug:
>>         https://bugs.openjdk.java.net/browse/JDK-8241825
>>
>>         Webrev:
>>         http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/
>>
>>         Testing:
>>         hs-tier1-7
>>
>>         Thanks,
>>         /Erik
>>
>


From thomas.stuefe at gmail.com  Tue Apr 28 16:29:39 2020
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Tue, 28 Apr 2020 18:29:39 +0200
Subject: [aarch64-port-dev ] FW: [Mach5]
 mach5-one-phh-JDK-8151779-20200427-2151-10554367: FAILED
In-Reply-To: <c13feff5-31e5-10fa-4915-df74c170a2e8@redhat.com>
References: <C769AA2F-55A8-4986-9EEE-0AD93FB93E00@amazon.com>
 <E6D13CFA-A9E6-4262-8D33-2FB4412413B6@amazon.com> <858sifgbt3.fsf@arm.com>
 <c13feff5-31e5-10fa-4915-df74c170a2e8@redhat.com>
Message-ID: <CAA-vtUw8eB5+VsvZ+W+WLcFUQ4FpZETFLF++5PCDbZkb=NB4yQ@mail.gmail.com>

On a related note, I wonder whether it would be possible to fake a larger
page size than the system uses. Well, I don't wonder since we do this on
AIX, for complicated reasons.

It requires a bit of work. So, not only spoofing os::vm_page_size(), but
also fixing the places where the native page size shines thru, e.g. making
sure os::reserve_memory() returns always os::vm_page_size() aligned memory.
I wonder whether it would be worth the work, that way one could simulate a
larger page size with a switch and test for errors like this.

..Thomas


On Tue, Apr 28, 2020 at 12:50 PM Andrew Haley <aph at redhat.com> wrote:

> On 4/28/20 11:11 AM, Nick Gasson wrote:
> >>
> >> My nightly buildbot started failing recently on aarch64.
> >> One issue is the error message of [cds] prevents configure from bootjdk
> determining.
> >>
> https://hg.openjdk.java.net/jdk/jdk/file/1b8f9e72b22b/make/autoconf/boot-jdk.m4#l77
> >>
> >> ./bin/java --version
> >> [0.006s][error][cds] Unable to map CDS archive --
> os::vm_allocation_granularity() expected: 65536 actual: 4096
> >> openjdk 14.0.1 2020-04-14
> >> OpenJDK Runtime Environment AdoptOpenJDK (build 14.0.1+7)
> >> OpenJDK 64-Bit Server VM AdoptOpenJDK (build 14.0.1+7, mixed mode)
> >>
> >> Are you aware of this issue? I can?t see [cds] line on my linux/x86_64
> host.
> >> Maybe it?s aarch64-only.  CC aarch-port-dev.
> >
> > Was your boot JDK built on a machine configured with a different page
> > size to your current machine? Looks like the CDS archive was dumped on a
> > machine with 64k pages but you're running with 4k pages. There's a JBS
> > issue for this:
> >
> > https://bugs.openjdk.java.net/browse/JDK-8236847
>
> The thread seems to have died here:
>
>
> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-February/038207.html
>
> We really need to get this fixed. If anyone reading this has machines with
> both 4k and 64k pages, please do the experiment and we'll make a suitable
> patch. Everything here has 64k pages.
>
> --
> Andrew Haley  (he/him)
> Java Platform Lead Engineer
> Red Hat UK Ltd. <https://www.redhat.com>
> https://keybase.io/andrewhaley
> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671
>
>

From erik.osterlund at oracle.com  Tue Apr 28 16:30:31 2020
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Tue, 28 Apr 2020 18:30:31 +0200
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64 (closed)
In-Reply-To: <c604dd4c-5e36-2ea8-15da-165b0162c500@oracle.com>
References: <1dd1bf71-fbb9-c57c-b033-884b5821033d@oracle.com>
 <f119092f-6239-bf61-c701-b1b45612792c@oracle.com>
 <c604dd4c-5e36-2ea8-15da-165b0162c500@oracle.com>
Message-ID: <9436e27a-0518-6139-afd7-2172383f55e7@oracle.com>

Hi Coleen,

Thanks for the review.

Here is a new webrev:
http://cr.openjdk.java.net/~eosterlund/8241825/webrev.01/

Incremental:
http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00_01/

On 2020-04-28 16:15, coleen.phillimore at oracle.com wrote:
> Sorry, this went to the wrong list.
>
> Coleen
>
> On 4/24/20 11:17 AM, coleen.phillimore at oracle.comwrote:
>>
>> Hi Erik, well the interpreter code isn't as bad as I thought it would 
>> be.? We tied UseCompressedKlassPointers to UseCompressedOops because 
>> we couldn't imagine why someone would care about 4 bytes per object 
>> with all 8 byte pointers to oops.? But maybe you do care.
>>
>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/cpu/x86/interp_masm_x86.cpp.udiff.html
>>
>>        if (UseBiasedLocking) {
>> - biased_locking_enter(lock_reg, obj_reg, swap_reg, tmp_reg, false, 
>> done, &slow_case);
>> + Register rklass_decode_tmp = LP64_ONLY(rscratch1) NOT_LP64(noreg);
>> + biased_locking_enter(lock_reg, obj_reg, swap_reg, tmp_reg, 
>> rscratch1, false, done, &slow_case);
>>        }
>> You didn't use rklass_decode_tmp here.

Nice catch. Fixed.

>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/cpu/x86/templateTable_x86.cpp.udiff.html
>>
>>
>> + Register tmp_load_klass = LP64_ONLY(rscratch1) NOT_LP64(noreg);
>>
>> I wonder if you couldn't just pass rscratch1 into load_klass and 
>> handle ignoring it inside the load_klass function, ie. fix the 
>> asserts.? This gets repetitive in the templateTable and other code.? 
>> Maybe people may have the opposite opinion though.

Unfortunately, that won't compile in code that is shared between 32 bit 
and 64 bit. In 32 bit-only code I just pass in noreg, and in 64-bit only 
code I just send in the temp. But for shared code, I have to do the 
macro dance, because rscratch1 does not exist on 32 bit.

>> This is quite nice that UseCompressedKlassPointers doesn't have to 
>> reinit_heapbase.? The interpreter code looks good.

Thanks!

>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/share/memory/metaspace.cpp.frames.html
>>
>> I don't know if base = HeapBaseMinAddress is right for the compressed 
>> class space.? It's reserved but not committed as 1G.? Maybe it makes 
>> sense since the heap without UseCompressedOops should be above the 
>> class space.

Thomas had a similar comment. I think the summary is that yes it is the 
right thing to use, but the naming is unfortunate.

Hope I fixed it all.

Thanks,
/Erik

>> Coleen
>>
>> On 4/24/20 4:25 AM, Erik ?sterlund wrote:
>>> Hi,
>>>
>>> This is the closed version of the mostly open change. There are some 
>>> @Contended tests in closed
>>> that needed some love with my change to make compressed class 
>>> pointers usable with compressed
>>> oops disabled. In particular, these tests perform calculations to 
>>> ensure that things are padded
>>> as expected. Part of those calculations makes assumptions about how 
>>> large the object header is,
>>> presuming that the number is a constant that is the same for all 
>>> objects, and is determinable
>>> by checking the offset of the first field. However, with my changes, 
>>> the first field will end
>>> up either in the klass gap, or after, depending on the field size. I 
>>> updated the tests to consider
>>> different header sizes depending on the type of fields being used. 
>>> It still isn't perfect, but
>>> it is clever enough for the purposes of the tests; they pass with 
>>> these changes.
>>>
>>> Bug:
>>> https://bugs.openjdk.java.net/browse/JDK-8241825
>>>
>>> Webrev:
>>> http://wikifiles.se.oracle.com/dev/eosterlund/8241825/webrev.00/
>>>
>>> Thanks,
>>> /Erik
>>
>


From dms at samersoff.net  Tue Apr 28 16:31:03 2020
From: dms at samersoff.net (Dmitry Samersoff)
Date: Tue, 28 Apr 2020 19:31:03 +0300
Subject: [aarch64-port-dev ] FW: [Mach5]
 mach5-one-phh-JDK-8151779-20200427-2151-10554367: FAILED
In-Reply-To: <c13feff5-31e5-10fa-4915-df74c170a2e8@redhat.com>
References: <C769AA2F-55A8-4986-9EEE-0AD93FB93E00@amazon.com>
 <E6D13CFA-A9E6-4262-8D33-2FB4412413B6@amazon.com> <858sifgbt3.fsf@arm.com>
 <c13feff5-31e5-10fa-4915-df74c170a2e8@redhat.com>
Message-ID: <2fc1d25f-e854-31d9-6d14-fab24b5d24c5@samersoff.net>

Hello Andrew,

I'm working on it, based on the fix proposed by Ioi.

-Dmitry


On 28.04.2020 13:47, Andrew Haley wrote:
> On 4/28/20 11:11 AM, Nick Gasson wrote:
>>>
>>> My nightly buildbot started failing recently on aarch64.
>>> One issue is the error message of [cds] prevents configure from bootjdk determining.
>>> https://hg.openjdk.java.net/jdk/jdk/file/1b8f9e72b22b/make/autoconf/boot-jdk.m4#l77
>>>
>>> ./bin/java --version
>>> [0.006s][error][cds] Unable to map CDS archive -- os::vm_allocation_granularity() expected: 65536 actual: 4096
>>> openjdk 14.0.1 2020-04-14
>>> OpenJDK Runtime Environment AdoptOpenJDK (build 14.0.1+7)
>>> OpenJDK 64-Bit Server VM AdoptOpenJDK (build 14.0.1+7, mixed mode)
>>>
>>> Are you aware of this issue? I can?t see [cds] line on my linux/x86_64 host.
>>> Maybe it?s aarch64-only.  CC aarch-port-dev.
>>
>> Was your boot JDK built on a machine configured with a different page
>> size to your current machine? Looks like the CDS archive was dumped on a
>> machine with 64k pages but you're running with 4k pages. There's a JBS
>> issue for this:
>>
>> https://bugs.openjdk.java.net/browse/JDK-8236847
> 
> The thread seems to have died here:
> 
> http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-February/038207.html
> 
> We really need to get this fixed. If anyone reading this has machines with
> both 4k and 64k pages, please do the experiment and we'll make a suitable
> patch. Everything here has 64k pages.
> 


From leonid.mesnik at oracle.com  Tue Apr 28 16:37:30 2020
From: leonid.mesnik at oracle.com (Leonid Mesnik)
Date: Tue, 28 Apr 2020 09:37:30 -0700
Subject: RFR: 8244010: Simplify usages of
 ProcessTools.createJavaProcessBuilder in our tests
In-Reply-To: <056ff9b1-1a4f-a1fb-2727-b503707f3750@oracle.com>
References: <60c0e0e4-634b-d349-c6f3-ca1ad52926cb@oracle.com>
 <056ff9b1-1a4f-a1fb-2727-b503707f3750@oracle.com>
Message-ID: <7ea36243-294b-bfd8-a95a-7699c0b776f9@oracle.com>

Hi

The changes look good.

The only small nit. Why don't? restrict arguments to "List<String>" 
instead of "Collection<String>"? The unordered collections like 
Set<String> shouldn't be used here. It is a just proposal, no need for 
new review if you going to fix it.

Leonid

On 4/28/20 6:54 AM, Stefan Karlsson wrote:
> Hi again,
>
> I realized that we probably want to give ProcessTools.executeTestJvm 
> the same treatment.
>
> Side-note: It's very awkard that createJavaProcessBuilder defaults to 
> not adding user-specifed flags, but executeTestJvm does. I think it 
> would be good to unify this as a separate RFE. I think *a lot* of 
> callers to createJavaProcessBuilder could be simplified by either 
> using executeTestJvm directly, or a simplified version of that.
>
> I'm running testing through mach5 and found a few things to fix, I 
> might find more when the testing has proceeded further.
>
> This is the current patch:
> https://cr.openjdk.java.net/~stefank/8244010/webrev.02.delta
> https://cr.openjdk.java.net/~stefank/8244010/webrev.02
>
> Thanks,
> StefanK
>
> On 2020-04-28 13:58, Stefan Karlsson wrote:
>> Hi all,
>>
>> Please review this patch to simplify usages of 
>> ProcessTools.createJavaProcessBuilder in our tests.
>>
>> https://cr.openjdk.java.net/~stefank/8244010/webrev.01/
>> https://bugs.openjdk.java.net/browse/JDK-8244010
>>
>> I saw all this code when reviewing changes to how we pass flags in 
>> our tests. There are a many places where arguments are converted and 
>> passed back and forth in String[] and Collections.
>>
>> For example:
>> ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(
>> ??????????argsList.toArray(new String[argsList.size()]));
>>
>> If we add an overload the createJavaProcessBuilder, that takes a 
>> Collection<String> as an argument, then we can write the code above as:
>> ???ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(argsList);
>>
>> Other places temporarily put the flags in a String[], where most 
>> calls simply lists the arguments in the call:
>> ??String[] opts = {Xmx, "-XX:NativeMemoryTracking=detail", 
>> "-XX:+UseParallelGC", "-version"};
>> ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(opts);
>>
>> And some places put the args in a temporary Collection:
>> ??LinkedList<String> vmOptions = new LinkedList<>();
>> ??vmOptions.add(gc);
>> ??vmOptions.add("-Xmx" + minMaxHeap);
>> ??vmOptions.add("-XX:+PrintFlagsFinal");
>> ??vmOptions.add(VerifyHeapSize.class.getName());
>>
>> ??ProcessBuilder pb = 
>> ProcessTools.createJavaProcessBuilder(vmOptions.toArray(new String[0]));
>>
>> I'd like to cleanup, simplify, and unify many of these usages.
>>
>> I've tested this by running all the changed tests locally.
>>
>> Thanks,
>> StefanK
>

From erik.osterlund at oracle.com  Tue Apr 28 16:38:03 2020
From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=)
Date: Tue, 28 Apr 2020 18:38:03 +0200
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64
In-Reply-To: <2822d95c-2fb7-a2b6-e61e-ba64614c504b@oracle.com>
References: <2822d95c-2fb7-a2b6-e61e-ba64614c504b@oracle.com>
Message-ID: <22A79898-39EF-49A6-963D-F3257D6AA720@oracle.com>

Thanks Coleen. Will add some comments and send out as a follow-up to this.

/Erik

> On 28 Apr 2020, at 16:17, coleen.phillimore at oracle.com wrote:
> 
>> On 4/27/20 11:28 AM, Erik ?sterlund wrote:
>> Hi Frederic,
>> 
>> I hear your concern about hardcoded offsets making the JVM more fragile.
>> 
>> First of all, let me assure you that you would not be able to run hello world if the offsets ever were to be
>> wrong. They are verified during bootstrapping in JavaClasses::check_offsets(), so you have to be quite skilled
>> to manage to break anything without immediately noticing it. You literally can't have run a single test or java -version
>> without immediately crashing. This makes this significantly less fragile. So I don't think I believe that this is fragile.
>> 
>> Having said that, I do agree it could be annoying and a pain to maintain and update the layout code if such changes
>> are occasionally incompatible with the random selection of hardcoded offsets, forcing new awkward updates of said offsets,
>> and that hardcoding things in general is undesirable, compared to a more solid solution.
>> I had a look at an approach for removing hardcoded offsets completely. It looks something like this:
>> 
>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00..not_01/
>> 
>> The change is a bit involved and complicated though. Because of that, I would prefer to push it as a follow-up RFE
>> focused on removing hardcoded offsets, if you are okay with that. This change is already quite tricky, and I think
>> these things ought to be separate.
> 
> This is nice and should be a follow-up RFE.  I sent you some comments to add to it.  It'd be great to not have hc offsets anymore!
> 
> Coleen


From igor.ignatyev at oracle.com  Tue Apr 28 16:47:43 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Tue, 28 Apr 2020 09:47:43 -0700
Subject: RFR: 8244010: Simplify usages of
 ProcessTools.createJavaProcessBuilder in our tests
In-Reply-To: <7ea36243-294b-bfd8-a95a-7699c0b776f9@oracle.com>
References: <60c0e0e4-634b-d349-c6f3-ca1ad52926cb@oracle.com>
 <056ff9b1-1a4f-a1fb-2727-b503707f3750@oracle.com>
 <7ea36243-294b-bfd8-a95a-7699c0b776f9@oracle.com>
Message-ID: <2A1C30A0-7B08-4B51-8AF4-71E1C8F3BCB1@oracle.com>

Hi Stefan,

the patch looks good to me.

-- Igor

> On Apr 28, 2020, at 9:37 AM, Leonid Mesnik <leonid.mesnik at oracle.com> wrote:
> 
> Hi
> 
> The changes look good.
> 
> The only small nit. Why don't  restrict arguments to "List<String>" instead of "Collection<String>"? The unordered collections like Set<String> shouldn't be used here. It is a just proposal, no need for new review if you going to fix it.
> 
> Leonid
> 
> On 4/28/20 6:54 AM, Stefan Karlsson wrote:
>> Hi again,
>> 
>> I realized that we probably want to give ProcessTools.executeTestJvm the same treatment.
>> 
>> Side-note: It's very awkard that createJavaProcessBuilder defaults to not adding user-specifed flags, but executeTestJvm does. I think it would be good to unify this as a separate RFE. I think *a lot* of callers to createJavaProcessBuilder could be simplified by either using executeTestJvm directly, or a simplified version of that.
>> 
>> I'm running testing through mach5 and found a few things to fix, I might find more when the testing has proceeded further.
>> 
>> This is the current patch:
>> https://cr.openjdk.java.net/~stefank/8244010/webrev.02.delta
>> https://cr.openjdk.java.net/~stefank/8244010/webrev.02
>> 
>> Thanks,
>> StefanK
>> 
>> On 2020-04-28 13:58, Stefan Karlsson wrote:
>>> Hi all,
>>> 
>>> Please review this patch to simplify usages of ProcessTools.createJavaProcessBuilder in our tests.
>>> 
>>> https://cr.openjdk.java.net/~stefank/8244010/webrev.01/
>>> https://bugs.openjdk.java.net/browse/JDK-8244010
>>> 
>>> I saw all this code when reviewing changes to how we pass flags in our tests. There are a many places where arguments are converted and passed back and forth in String[] and Collections.
>>> 
>>> For example:
>>>   ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(
>>>           argsList.toArray(new String[argsList.size()]));
>>> 
>>> If we add an overload the createJavaProcessBuilder, that takes a Collection<String> as an argument, then we can write the code above as:
>>>    ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(argsList);
>>> 
>>> Other places temporarily put the flags in a String[], where most calls simply lists the arguments in the call:
>>>   String[] opts = {Xmx, "-XX:NativeMemoryTracking=detail", "-XX:+UseParallelGC", "-version"};
>>>   ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(opts);
>>> 
>>> And some places put the args in a temporary Collection:
>>>   LinkedList<String> vmOptions = new LinkedList<>();
>>>   vmOptions.add(gc);
>>>   vmOptions.add("-Xmx" + minMaxHeap);
>>>   vmOptions.add("-XX:+PrintFlagsFinal");
>>>   vmOptions.add(VerifyHeapSize.class.getName());
>>> 
>>>   ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(vmOptions.toArray(new String[0]));
>>> 
>>> I'd like to cleanup, simplify, and unify many of these usages.
>>> 
>>> I've tested this by running all the changed tests locally.
>>> 
>>> Thanks,
>>> StefanK
>> 


From coleen.phillimore at oracle.com  Tue Apr 28 16:54:24 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Tue, 28 Apr 2020 12:54:24 -0400
Subject: RFR: 8244010: Simplify usages of
 ProcessTools.createJavaProcessBuilder in our tests
In-Reply-To: <75d7f5ee-3c1c-0dc3-147f-f6d445769e64@oracle.com>
References: <60c0e0e4-634b-d349-c6f3-ca1ad52926cb@oracle.com>
 <056ff9b1-1a4f-a1fb-2727-b503707f3750@oracle.com>
 <94095aef-266d-fa88-5d36-c59d07b27f3d@oracle.com>
 <75d7f5ee-3c1c-0dc3-147f-f6d445769e64@oracle.com>
Message-ID: <92f28aa3-0d8c-2e18-ce58-fb0d0ee38931@oracle.com>



On 4/28/20 11:46 AM, Stefan Karlsson wrote:
> On 2020-04-28 16:10, coleen.phillimore at oracle.com wrote:
>>
>> I clicked on the runtime and serviceability tests and they look good 
>> to me.? This is definitely an improvement.
>>
>> - pb = ProcessTools.createJavaProcessBuilder(runJava.toArray(new 
>> String[0]));
>> + pb = ProcessTools.createJavaProcessBuilder(runJava);
>
> Thanks. See below:
>
>>
>>
>>
>> On 4/28/20 9:54 AM, Stefan Karlsson wrote:
>>> Hi again,
>>>
>>> I realized that we probably want to give ProcessTools.executeTestJvm 
>>> the same treatment.
>>>
>>> Side-note: It's very awkard that createJavaProcessBuilder defaults 
>>> to not adding user-specifed flags, but executeTestJvm does. I think 
>>> it would be good to unify this as a separate RFE. I think *a lot* of 
>>> callers to createJavaProcessBuilder could be simplified by either 
>>> using executeTestJvm directly, or a simplified version of that.
>>
>> I never knew there were two.? Why are they different?? It looks like 
>> there are more createJavaProcessBuilder calls than executeTestjvm 
>> ones though.? It would be good to have just one.
>
> One big difference is that one provides a ProcessBuilder and the other 
> hides it and returns an OutputAnalyzer:
>
> ??? public static OutputAnalyzer executeTestJvm(String... cmds) throws 
> Exception {
> ??????? ProcessBuilder pb = createJavaProcessBuilder(true, cmds);
> ??????? return executeProcess(pb);
> ??? }
>
> There is a difference between executeProcess and new 
> OutputAnalyzer(pb.start()), but if we ignore that and focus on what 
> the test code would look like, we could often completely get rid of 
> the notion of a ProcessBuilder from our tests.
>
> If we take 
> test/hotspot/jtreg/runtime//logging/ClassResolutionTest.java as an 
> example. We could change:
>
> ? ProcessBuilder pb = 
> ProcessTools.createJavaProcessBuilder("-Xlog:class+resolve=debug",
> ClassResolutionTestMain.class.getName());
> ? OutputAnalyzer o = new OutputAnalyzer(pb.start());
> ? o.shouldContain("[class,resolve] 
> ClassResolutionTest$ClassResolutionTestMain$Thing1Handler 
> ClassResolutionTest$ClassResolutionTestMain$Thing1");
> ? o.shouldContain("[class,resolve] resolve JVM_CONSTANT_MethodHandle");
>
> To:
>
> ? OutputAnalyzer o = 
> ProcessTools.executeTestJvm("-Xlog:class+resolve=debug",
> ClassResolutionTestMain.class.getName());
> ? o.shouldContain("[class,resolve] 
> ClassResolutionTest$ClassResolutionTestMain$Thing1Handler 
> ClassResolutionTest$ClassResolutionTestMain$Thing1");
> ? o.shouldContain("[class,resolve] resolve JVM_CONSTANT_MethodHandle");
>
> Or even get rid of the short lived variables (Not sure if it's 
> preferable, but it's doable):
>
> ? ProcessTools.executeTestJvm("-Xlog:class+resolve=debug",
> ClassResolutionTestMain.class.getName())
> ????????????? .shouldContain("[class,resolve] 
> ClassResolutionTest$ClassResolutionTestMain$Thing1Handler 
> ClassResolutionTest$ClassResolutionTestMain$Thing1");
> ????????????? .shouldContain("[class,resolve] resolve 
> JVM_CONSTANT_MethodHandle");

Yes, both of these are better.? Less stuff to cut/paste for writing new 
tests.

thanks!
Coleen

>
> StefanK
>>
>> Thanks,
>> Coleen
>>
>>>
>>> I'm running testing through mach5 and found a few things to fix, I 
>>> might find more when the testing has proceeded further.
>>>
>>> This is the current patch:
>>> https://cr.openjdk.java.net/~stefank/8244010/webrev.02.delta
>>> https://cr.openjdk.java.net/~stefank/8244010/webrev.02
>>>
>>> Thanks,
>>> StefanK
>>>
>>> On 2020-04-28 13:58, Stefan Karlsson wrote:
>>>> Hi all,
>>>>
>>>> Please review this patch to simplify usages of 
>>>> ProcessTools.createJavaProcessBuilder in our tests.
>>>>
>>>> https://cr.openjdk.java.net/~stefank/8244010/webrev.01/
>>>> https://bugs.openjdk.java.net/browse/JDK-8244010
>>>>
>>>> I saw all this code when reviewing changes to how we pass flags in 
>>>> our tests. There are a many places where arguments are converted 
>>>> and passed back and forth in String[] and Collections.
>>>>
>>>> For example:
>>>> ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(
>>>> ??????????argsList.toArray(new String[argsList.size()]));
>>>>
>>>> If we add an overload the createJavaProcessBuilder, that takes a 
>>>> Collection<String> as an argument, then we can write the code above 
>>>> as:
>>>> ???ProcessBuilder pb = 
>>>> ProcessTools.createJavaProcessBuilder(argsList);
>>>>
>>>> Other places temporarily put the flags in a String[], where most 
>>>> calls simply lists the arguments in the call:
>>>> ??String[] opts = {Xmx, "-XX:NativeMemoryTracking=detail", 
>>>> "-XX:+UseParallelGC", "-version"};
>>>> ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(opts);
>>>>
>>>> And some places put the args in a temporary Collection:
>>>> ??LinkedList<String> vmOptions = new LinkedList<>();
>>>> ??vmOptions.add(gc);
>>>> ??vmOptions.add("-Xmx" + minMaxHeap);
>>>> ??vmOptions.add("-XX:+PrintFlagsFinal");
>>>> ??vmOptions.add(VerifyHeapSize.class.getName());
>>>>
>>>> ??ProcessBuilder pb = 
>>>> ProcessTools.createJavaProcessBuilder(vmOptions.toArray(new 
>>>> String[0]));
>>>>
>>>> I'd like to cleanup, simplify, and unify many of these usages.
>>>>
>>>> I've tested this by running all the changed tests locally.
>>>>
>>>> Thanks,
>>>> StefanK
>>>
>>
>


From coleen.phillimore at oracle.com  Tue Apr 28 17:25:40 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Tue, 28 Apr 2020 13:25:40 -0400
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64 (closed)
In-Reply-To: <9436e27a-0518-6139-afd7-2172383f55e7@oracle.com>
References: <1dd1bf71-fbb9-c57c-b033-884b5821033d@oracle.com>
 <f119092f-6239-bf61-c701-b1b45612792c@oracle.com>
 <c604dd4c-5e36-2ea8-15da-165b0162c500@oracle.com>
 <9436e27a-0518-6139-afd7-2172383f55e7@oracle.com>
Message-ID: <12900956-dcb1-3069-80bf-0425e64ca92e@oracle.com>

Okay.? Your incremental is 4040 but looks good to me.
Thanks,
Coleen

On 4/28/20 12:30 PM, Erik ?sterlund wrote:
> Hi Coleen,
>
> Thanks for the review.
>
> Here is a new webrev:
> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.01/
>
> Incremental:
> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00_01/
>
> On 2020-04-28 16:15, coleen.phillimore at oracle.com wrote:
>> Sorry, this went to the wrong list.
>>
>> Coleen
>>
>> On 4/24/20 11:17 AM, coleen.phillimore at oracle.comwrote:
>>>
>>> Hi Erik, well the interpreter code isn't as bad as I thought it 
>>> would be.? We tied UseCompressedKlassPointers to UseCompressedOops 
>>> because we couldn't imagine why someone would care about 4 bytes per 
>>> object with all 8 byte pointers to oops.? But maybe you do care.
>>>
>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/cpu/x86/interp_masm_x86.cpp.udiff.html
>>>
>>>        if (UseBiasedLocking) {
>>> - biased_locking_enter(lock_reg, obj_reg, swap_reg, tmp_reg, false, 
>>> done, &slow_case);
>>> + Register rklass_decode_tmp = LP64_ONLY(rscratch1) NOT_LP64(noreg);
>>> + biased_locking_enter(lock_reg, obj_reg, swap_reg, tmp_reg, 
>>> rscratch1, false, done, &slow_case);
>>>        }
>>> You didn't use rklass_decode_tmp here.
>
> Nice catch. Fixed.
>
>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/cpu/x86/templateTable_x86.cpp.udiff.html
>>>
>>>
>>> + Register tmp_load_klass = LP64_ONLY(rscratch1) NOT_LP64(noreg);
>>>
>>> I wonder if you couldn't just pass rscratch1 into load_klass and 
>>> handle ignoring it inside the load_klass function, ie. fix the 
>>> asserts.? This gets repetitive in the templateTable and other code.? 
>>> Maybe people may have the opposite opinion though.
>
> Unfortunately, that won't compile in code that is shared between 32 
> bit and 64 bit. In 32 bit-only code I just pass in noreg, and in 
> 64-bit only code I just send in the temp. But for shared code, I have 
> to do the macro dance, because rscratch1 does not exist on 32 bit.
>
>>> This is quite nice that UseCompressedKlassPointers doesn't have to 
>>> reinit_heapbase.? The interpreter code looks good.
>
> Thanks!
>
>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/share/memory/metaspace.cpp.frames.html
>>>
>>> I don't know if base = HeapBaseMinAddress is right for the 
>>> compressed class space.? It's reserved but not committed as 1G.? 
>>> Maybe it makes sense since the heap without UseCompressedOops should 
>>> be above the class space.
>
> Thomas had a similar comment. I think the summary is that yes it is 
> the right thing to use, but the naming is unfortunate.
>
> Hope I fixed it all.
>
> Thanks,
> /Erik
>
>>> Coleen
>>>
>>> On 4/24/20 4:25 AM, Erik ?sterlund wrote:
>>>> Hi,
>>>>
>>>> This is the closed version of the mostly open change. There are 
>>>> some @Contended tests in closed
>>>> that needed some love with my change to make compressed class 
>>>> pointers usable with compressed
>>>> oops disabled. In particular, these tests perform calculations to 
>>>> ensure that things are padded
>>>> as expected. Part of those calculations makes assumptions about how 
>>>> large the object header is,
>>>> presuming that the number is a constant that is the same for all 
>>>> objects, and is determinable
>>>> by checking the offset of the first field. However, with my 
>>>> changes, the first field will end
>>>> up either in the klass gap, or after, depending on the field size. 
>>>> I updated the tests to consider
>>>> different header sizes depending on the type of fields being used. 
>>>> It still isn't perfect, but
>>>> it is clever enough for the purposes of the tests; they pass with 
>>>> these changes.
>>>>
>>>> Bug:
>>>> https://bugs.openjdk.java.net/browse/JDK-8241825
>>>>
>>>> Webrev:
>>>> http://wikifiles.se.oracle.com/dev/eosterlund/8241825/webrev.00/
>>>>
>>>> Thanks,
>>>> /Erik
>>>
>>
>


From coleen.phillimore at oracle.com  Tue Apr 28 17:28:42 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Tue, 28 Apr 2020 13:28:42 -0400
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64 (closed)
In-Reply-To: <12900956-dcb1-3069-80bf-0425e64ca92e@oracle.com>
References: <1dd1bf71-fbb9-c57c-b033-884b5821033d@oracle.com>
 <f119092f-6239-bf61-c701-b1b45612792c@oracle.com>
 <c604dd4c-5e36-2ea8-15da-165b0162c500@oracle.com>
 <9436e27a-0518-6139-afd7-2172383f55e7@oracle.com>
 <12900956-dcb1-3069-80bf-0425e64ca92e@oracle.com>
Message-ID: <f890552b-7b30-1495-b579-74d6b81b9845@oracle.com>



On 4/28/20 1:25 PM, coleen.phillimore at oracle.com wrote:
> Okay. Your incremental is 4040 but looks good to me.
10 times 404 I mean.
> Thanks,
> Coleen
>
> On 4/28/20 12:30 PM, Erik ?sterlund wrote:
>> Hi Coleen,
>>
>> Thanks for the review.
>>
>> Here is a new webrev:
>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.01/
>>
>> Incremental:
>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00_01/
>>
>> On 2020-04-28 16:15, coleen.phillimore at oracle.com wrote:
>>> Sorry, this went to the wrong list.
>>>
>>> Coleen
>>>
>>> On 4/24/20 11:17 AM, coleen.phillimore at oracle.comwrote:
>>>>
>>>> Hi Erik, well the interpreter code isn't as bad as I thought it 
>>>> would be.? We tied UseCompressedKlassPointers to UseCompressedOops 
>>>> because we couldn't imagine why someone would care about 4 bytes 
>>>> per object with all 8 byte pointers to oops.? But maybe you do care.
>>>>
>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/cpu/x86/interp_masm_x86.cpp.udiff.html 
>>>>
>>>>
>>>> ?????? if (UseBiasedLocking) {
>>>> - biased_locking_enter(lock_reg, obj_reg, swap_reg, tmp_reg, false, 
>>>> done, &slow_case);
>>>> + Register rklass_decode_tmp = LP64_ONLY(rscratch1) NOT_LP64(noreg);
>>>> + biased_locking_enter(lock_reg, obj_reg, swap_reg, tmp_reg, 
>>>> rscratch1, false, done, &slow_case);
>>>> ?????? }
>>>> You didn't use rklass_decode_tmp here.
>>
>> Nice catch. Fixed.
>>
>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/cpu/x86/templateTable_x86.cpp.udiff.html 
>>>>
>>>>
>>>>
>>>> + Register tmp_load_klass = LP64_ONLY(rscratch1) NOT_LP64(noreg);
>>>>
>>>> I wonder if you couldn't just pass rscratch1 into load_klass and 
>>>> handle ignoring it inside the load_klass function, ie. fix the 
>>>> asserts.? This gets repetitive in the templateTable and other 
>>>> code.? Maybe people may have the opposite opinion though.
>>
>> Unfortunately, that won't compile in code that is shared between 32 
>> bit and 64 bit. In 32 bit-only code I just pass in noreg, and in 
>> 64-bit only code I just send in the temp. But for shared code, I have 
>> to do the macro dance, because rscratch1 does not exist on 32 bit.
>>
>>>> This is quite nice that UseCompressedKlassPointers doesn't have to 
>>>> reinit_heapbase. The interpreter code looks good.
>>
>> Thanks!
>>
>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/share/memory/metaspace.cpp.frames.html 
>>>>
>>>>
>>>> I don't know if base = HeapBaseMinAddress is right for the 
>>>> compressed class space.? It's reserved but not committed as 1G.? 
>>>> Maybe it makes sense since the heap without UseCompressedOops 
>>>> should be above the class space.
>>
>> Thomas had a similar comment. I think the summary is that yes it is 
>> the right thing to use, but the naming is unfortunate.
>>
>> Hope I fixed it all.
>>
>> Thanks,
>> /Erik
>>
>>>> Coleen
>>>>
>>>> On 4/24/20 4:25 AM, Erik ?sterlund wrote:
>>>>> Hi,
>>>>>
>>>>> This is the closed version of the mostly open change. There are 
>>>>> some @Contended tests in closed
>>>>> that needed some love with my change to make compressed class 
>>>>> pointers usable with compressed
>>>>> oops disabled. In particular, these tests perform calculations to 
>>>>> ensure that things are padded
>>>>> as expected. Part of those calculations makes assumptions about 
>>>>> how large the object header is,
>>>>> presuming that the number is a constant that is the same for all 
>>>>> objects, and is determinable
>>>>> by checking the offset of the first field. However, with my 
>>>>> changes, the first field will end
>>>>> up either in the klass gap, or after, depending on the field size. 
>>>>> I updated the tests to consider
>>>>> different header sizes depending on the type of fields being used. 
>>>>> It still isn't perfect, but
>>>>> it is clever enough for the purposes of the tests; they pass with 
>>>>> these changes.
>>>>>
>>>>> Bug:
>>>>> https://bugs.openjdk.java.net/browse/JDK-8241825
>>>>>
>>>>> Webrev:
>>>>> http://wikifiles.se.oracle.com/dev/eosterlund/8241825/webrev.00/
>>>>>
>>>>> Thanks,
>>>>> /Erik
>>>>
>>>
>>
>


From erik.osterlund at oracle.com  Tue Apr 28 17:52:36 2020
From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=)
Date: Tue, 28 Apr 2020 19:52:36 +0200
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64 (closed)
In-Reply-To: <12900956-dcb1-3069-80bf-0425e64ca92e@oracle.com>
References: <12900956-dcb1-3069-80bf-0425e64ca92e@oracle.com>
Message-ID: <F9F70B1F-BAA1-4D7B-87E8-19BCA305661B@oracle.com>

Hi Coleen,

Oops, typo in the incremental URL. Here it is:

http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00..01/

Thanks for the review.

/Erik

> On 28 Apr 2020, at 19:25, coleen.phillimore at oracle.com wrote:
> 
> ? Okay.  Your incremental is 4040 but looks good to me.
> Thanks,
> Coleen
> 
> On 4/28/20 12:30 PM, Erik ?sterlund wrote:
>> Hi Coleen,
>> 
>> Thanks for the review.
>> 
>> Here is a new webrev:
>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.01/
>> 
>> Incremental:
>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00_01/
>> 
>>> On 2020-04-28 16:15, coleen.phillimore at oracle.com wrote:
>>> Sorry, this went to the wrong list.
>>> 
>>> Coleen
>>> 
>>>> On 4/24/20 11:17 AM, coleen.phillimore at oracle.com wrote:
>>>> 
>>>> Hi Erik, well the interpreter code isn't as bad as I thought it would be.  We tied UseCompressedKlassPointers to UseCompressedOops because we couldn't imagine why someone would care about 4 bytes per object with all 8 byte pointers to oops.  But maybe you do care.
>>>> 
>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/cpu/x86/interp_masm_x86.cpp.udiff.html
>>>> 
>>>>        if (UseBiasedLocking) {
>>>> -       biased_locking_enter(lock_reg, obj_reg, swap_reg, tmp_reg, false, done, &slow_case);
>>>> +       Register rklass_decode_tmp = LP64_ONLY(rscratch1) NOT_LP64(noreg);
>>>> +       biased_locking_enter(lock_reg, obj_reg, swap_reg, tmp_reg, rscratch1, false, done, &slow_case);
>>>>       }
>>>> You didn't use rklass_decode_tmp here.
>> 
>> Nice catch. Fixed.
>> 
>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/cpu/x86/templateTable_x86.cpp.udiff.html
>>>> 
>>>> 
>>>>  +   Register tmp_load_klass = LP64_ONLY(rscratch1) NOT_LP64(noreg);
>>>> 
>>>> I wonder if you couldn't just pass rscratch1 into load_klass and handle ignoring it inside the load_klass function, ie. fix the asserts.  This gets repetitive in the templateTable and other code.  Maybe people may have the opposite opinion though.
>> 
>> Unfortunately, that won't compile in code that is shared between 32 bit and 64 bit. In 32 bit-only code I just pass in noreg, and in 64-bit only code I just send in the temp. But for shared code, I have to do the macro dance, because rscratch1 does not exist on 32 bit.
>> 
>>>> This is quite nice that UseCompressedKlassPointers doesn't have to reinit_heapbase.  The interpreter code looks good.
>> 
>> Thanks!
>> 
>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/share/memory/metaspace.cpp.frames.html
>>>> 
>>>> I don't know if base = HeapBaseMinAddress is right for the compressed class space.  It's reserved but not committed as 1G.  Maybe it makes sense since the heap without UseCompressedOops should be above the class space.
>> 
>> Thomas had a similar comment. I think the summary is that yes it is the right thing to use, but the naming is unfortunate.
>> 
>> Hope I fixed it all.
>> 
>> Thanks,
>> /Erik
>> 
>>>> Coleen
>>>> 
>>>>> On 4/24/20 4:25 AM, Erik ?sterlund wrote:
>>>>> Hi, 
>>>>> 
>>>>> This is the closed version of the mostly open change. There are some @Contended tests in closed 
>>>>> that needed some love with my change to make compressed class pointers usable with compressed 
>>>>> oops disabled. In particular, these tests perform calculations to ensure that things are padded 
>>>>> as expected. Part of those calculations makes assumptions about how large the object header is, 
>>>>> presuming that the number is a constant that is the same for all objects, and is determinable 
>>>>> by checking the offset of the first field. However, with my changes, the first field will end 
>>>>> up either in the klass gap, or after, depending on the field size. I updated the tests to consider 
>>>>> different header sizes depending on the type of fields being used. It still isn't perfect, but 
>>>>> it is clever enough for the purposes of the tests; they pass with these changes. 
>>>>> 
>>>>> Bug: 
>>>>> https://bugs.openjdk.java.net/browse/JDK-8241825 
>>>>> 
>>>>> Webrev: 
>>>>> http://wikifiles.se.oracle.com/dev/eosterlund/8241825/webrev.00/ 
>>>>> 
>>>>> Thanks, 
>>>>> /Erik 
>>>> 
>>> 
>> 
> 

From coleen.phillimore at oracle.com  Tue Apr 28 19:56:54 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Tue, 28 Apr 2020 15:56:54 -0400
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64 (closed)
In-Reply-To: <F9F70B1F-BAA1-4D7B-87E8-19BCA305661B@oracle.com>
References: <12900956-dcb1-3069-80bf-0425e64ca92e@oracle.com>
 <F9F70B1F-BAA1-4D7B-87E8-19BCA305661B@oracle.com>
Message-ID: <9fd23c76-900b-bf6a-3411-64955bd4261c@oracle.com>


Hi Erik,

I was trying to figure out why this change is needed to decouple 
CompressedOops and CompressedKlassPointers, but it's not needed for the 
change, is it?

http://cr.openjdk.java.net/~eosterlund/8241825/webrev.01/src/hotspot/share/classfile/fieldLayoutBuilder.cpp.udiff.html

Can you make this change with your patch for 
https://bugs.openjdk.java.net/browse/JDK-8243996 which we really like?

Thanks,
Coleen


On 4/28/20 1:52 PM, Erik ?sterlund wrote:
> Hi Coleen,
>
> Oops, typo in the incremental URL. Here it is:
>
> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00..01/
>
> Thanks for the review.
>
> /Erik
>
>> On 28 Apr 2020, at 19:25, coleen.phillimore at oracle.com wrote:
>>
>> ? Okay.? Your incremental is 4040 but looks good to me.
>> Thanks,
>> Coleen
>>
>> On 4/28/20 12:30 PM, Erik ?sterlund wrote:
>>> Hi Coleen,
>>>
>>> Thanks for the review.
>>>
>>> Here is a new webrev:
>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.01/
>>>
>>> Incremental:
>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00_01/
>>>
>>> On 2020-04-28 16:15, coleen.phillimore at oracle.com wrote:
>>>> Sorry, this went to the wrong list.
>>>>
>>>> Coleen
>>>>
>>>> On 4/24/20 11:17 AM, coleen.phillimore at oracle.comwrote:
>>>>>
>>>>> Hi Erik, well the interpreter code isn't as bad as I thought it 
>>>>> would be.? We tied UseCompressedKlassPointers to UseCompressedOops 
>>>>> because we couldn't imagine why someone would care about 4 bytes 
>>>>> per object with all 8 byte pointers to oops.? But maybe you do care.
>>>>>
>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/cpu/x86/interp_masm_x86.cpp.udiff.html
>>>>>
>>>>>        if (UseBiasedLocking) {
>>>>> - biased_locking_enter(lock_reg, obj_reg, swap_reg, tmp_reg, 
>>>>> false, done, &slow_case);
>>>>> + Register rklass_decode_tmp = LP64_ONLY(rscratch1) NOT_LP64(noreg);
>>>>> + biased_locking_enter(lock_reg, obj_reg, swap_reg, tmp_reg, 
>>>>> rscratch1, false, done, &slow_case);
>>>>>        }
>>>>> You didn't use rklass_decode_tmp here.
>>>
>>> Nice catch. Fixed.
>>>
>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/cpu/x86/templateTable_x86.cpp.udiff.html
>>>>>
>>>>>
>>>>> + Register tmp_load_klass = LP64_ONLY(rscratch1) NOT_LP64(noreg);
>>>>>
>>>>> I wonder if you couldn't just pass rscratch1 into load_klass and 
>>>>> handle ignoring it inside the load_klass function, ie. fix the 
>>>>> asserts.? This gets repetitive in the templateTable and other 
>>>>> code.? Maybe people may have the opposite opinion though.
>>>
>>> Unfortunately, that won't compile in code that is shared between 32 
>>> bit and 64 bit. In 32 bit-only code I just pass in noreg, and in 
>>> 64-bit only code I just send in the temp. But for shared code, I 
>>> have to do the macro dance, because rscratch1 does not exist on 32 bit.
>>>
>>>>> This is quite nice that UseCompressedKlassPointers doesn't have to 
>>>>> reinit_heapbase.? The interpreter code looks good.
>>>
>>> Thanks!
>>>
>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/share/memory/metaspace.cpp.frames.html
>>>>>
>>>>> I don't know if base = HeapBaseMinAddress is right for the 
>>>>> compressed class space.? It's reserved but not committed as 1G.? 
>>>>> Maybe it makes sense since the heap without UseCompressedOops 
>>>>> should be above the class space.
>>>
>>> Thomas had a similar comment. I think the summary is that yes it is 
>>> the right thing to use, but the naming is unfortunate.
>>>
>>> Hope I fixed it all.
>>>
>>> Thanks,
>>> /Erik
>>>
>>>>> Coleen
>>>>>
>>>>> On 4/24/20 4:25 AM, Erik ?sterlund wrote:
>>>>>> Hi,
>>>>>>
>>>>>> This is the closed version of the mostly open change. There are 
>>>>>> some @Contended tests in closed
>>>>>> that needed some love with my change to make compressed class 
>>>>>> pointers usable with compressed
>>>>>> oops disabled. In particular, these tests perform calculations to 
>>>>>> ensure that things are padded
>>>>>> as expected. Part of those calculations makes assumptions about 
>>>>>> how large the object header is,
>>>>>> presuming that the number is a constant that is the same for all 
>>>>>> objects, and is determinable
>>>>>> by checking the offset of the first field. However, with my 
>>>>>> changes, the first field will end
>>>>>> up either in the klass gap, or after, depending on the field 
>>>>>> size. I updated the tests to consider
>>>>>> different header sizes depending on the type of fields being 
>>>>>> used. It still isn't perfect, but
>>>>>> it is clever enough for the purposes of the tests; they pass with 
>>>>>> these changes.
>>>>>>
>>>>>> Bug:
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8241825
>>>>>>
>>>>>> Webrev:
>>>>>> http://wikifiles.se.oracle.com/dev/eosterlund/8241825/webrev.00/
>>>>>>
>>>>>> Thanks,
>>>>>> /Erik
>>>>>
>>>>
>>>
>>


From erik.osterlund at oracle.com  Tue Apr 28 20:30:14 2020
From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=)
Date: Tue, 28 Apr 2020 22:30:14 +0200
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64 (closed)
In-Reply-To: <9fd23c76-900b-bf6a-3411-64955bd4261c@oracle.com>
References: <9fd23c76-900b-bf6a-3411-64955bd4261c@oracle.com>
Message-ID: <C92914CE-B996-44FF-B13D-3E1189C349D1@oracle.com>

Hi Coleen,

Unfortunately I do need that change already now. Because making compressed oops and compressed class pointers independent involves changing the instanceOop base with compressed oop off and compressed class pointers off. That changes the layout and offsets that are hardcoded (primitive wrappers that wrap a primitive less than 4 bytes now start in the klass gap).

My proposed solution is to remove hardcoded layouts in this patch, and remove hardcoded offsets in the next one. The alternative would be trying to fix the hardcoded layouts only to remove them one patch later. I tried to fix them but ran into issues; that is why I removed them instead in this patch. I hope you think this is okay. The net result will be the same one patch later.

Thanks,
/Erik

> On 28 Apr 2020, at 21:56, coleen.phillimore at oracle.com wrote:
> 
> ? 
> Hi Erik,
> 
> I was trying to figure out why this change is needed to decouple CompressedOops and CompressedKlassPointers, but it's not needed for the change, is it?
> 
> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.01/src/hotspot/share/classfile/fieldLayoutBuilder.cpp.udiff.html
> 
> Can you make this change with your patch for https://bugs.openjdk.java.net/browse/JDK-8243996 which we really like?
> 
> Thanks,
> Coleen
> 
> 
> On 4/28/20 1:52 PM, Erik ?sterlund wrote:
>> Hi Coleen,
>> 
>> Oops, typo in the incremental URL. Here it is:
>> 
>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00..01/
>> 
>> Thanks for the review.
>> 
>> /Erik
>> 
>>> On 28 Apr 2020, at 19:25, coleen.phillimore at oracle.com wrote:
>>> 
>>> ? Okay.  Your incremental is 4040 but looks good to me.
>>> Thanks,
>>> Coleen
>>> 
>>> On 4/28/20 12:30 PM, Erik ?sterlund wrote:
>>>> Hi Coleen,
>>>> 
>>>> Thanks for the review.
>>>> 
>>>> Here is a new webrev:
>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.01/
>>>> 
>>>> Incremental:
>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00_01/
>>>> 
>>>>> On 2020-04-28 16:15, coleen.phillimore at oracle.com wrote:
>>>>> Sorry, this went to the wrong list.
>>>>> 
>>>>> Coleen
>>>>> 
>>>>>> On 4/24/20 11:17 AM, coleen.phillimore at oracle.com wrote:
>>>>>> 
>>>>>> Hi Erik, well the interpreter code isn't as bad as I thought it would be.  We tied UseCompressedKlassPointers to UseCompressedOops because we couldn't imagine why someone would care about 4 bytes per object with all 8 byte pointers to oops.  But maybe you do care.
>>>>>> 
>>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/cpu/x86/interp_masm_x86.cpp.udiff.html
>>>>>> 
>>>>>>        if (UseBiasedLocking) {
>>>>>> -       biased_locking_enter(lock_reg, obj_reg, swap_reg, tmp_reg, false, done, &slow_case);
>>>>>> +       Register rklass_decode_tmp = LP64_ONLY(rscratch1) NOT_LP64(noreg);
>>>>>> +       biased_locking_enter(lock_reg, obj_reg, swap_reg, tmp_reg, rscratch1, false, done, &slow_case);
>>>>>>       }
>>>>>> You didn't use rklass_decode_tmp here.
>>>> 
>>>> Nice catch. Fixed.
>>>> 
>>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/cpu/x86/templateTable_x86.cpp.udiff.html
>>>>>> 
>>>>>> 
>>>>>>  +   Register tmp_load_klass = LP64_ONLY(rscratch1) NOT_LP64(noreg);
>>>>>> 
>>>>>> I wonder if you couldn't just pass rscratch1 into load_klass and handle ignoring it inside the load_klass function, ie. fix the asserts.  This gets repetitive in the templateTable and other code.  Maybe people may have the opposite opinion though.
>>>> 
>>>> Unfortunately, that won't compile in code that is shared between 32 bit and 64 bit. In 32 bit-only code I just pass in noreg, and in 64-bit only code I just send in the temp. But for shared code, I have to do the macro dance, because rscratch1 does not exist on 32 bit.
>>>> 
>>>>>> This is quite nice that UseCompressedKlassPointers doesn't have to reinit_heapbase.  The interpreter code looks good.
>>>> 
>>>> Thanks!
>>>> 
>>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/share/memory/metaspace.cpp.frames.html
>>>>>> 
>>>>>> I don't know if base = HeapBaseMinAddress is right for the compressed class space.  It's reserved but not committed as 1G.  Maybe it makes sense since the heap without UseCompressedOops should be above the class space.
>>>> 
>>>> Thomas had a similar comment. I think the summary is that yes it is the right thing to use, but the naming is unfortunate.
>>>> 
>>>> Hope I fixed it all.
>>>> 
>>>> Thanks,
>>>> /Erik
>>>> 
>>>>>> Coleen
>>>>>> 
>>>>>>> On 4/24/20 4:25 AM, Erik ?sterlund wrote:
>>>>>>> Hi, 
>>>>>>> 
>>>>>>> This is the closed version of the mostly open change. There are some @Contended tests in closed 
>>>>>>> that needed some love with my change to make compressed class pointers usable with compressed 
>>>>>>> oops disabled. In particular, these tests perform calculations to ensure that things are padded 
>>>>>>> as expected. Part of those calculations makes assumptions about how large the object header is, 
>>>>>>> presuming that the number is a constant that is the same for all objects, and is determinable 
>>>>>>> by checking the offset of the first field. However, with my changes, the first field will end 
>>>>>>> up either in the klass gap, or after, depending on the field size. I updated the tests to consider 
>>>>>>> different header sizes depending on the type of fields being used. It still isn't perfect, but 
>>>>>>> it is clever enough for the purposes of the tests; they pass with these changes. 
>>>>>>> 
>>>>>>> Bug: 
>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8241825 
>>>>>>> 
>>>>>>> Webrev: 
>>>>>>> http://wikifiles.se.oracle.com/dev/eosterlund/8241825/webrev.00/ 
>>>>>>> 
>>>>>>> Thanks, 
>>>>>>> /Erik 
>>>>>> 
>>>>> 
>>>> 
>>> 
> 

From stefan.karlsson at oracle.com  Tue Apr 28 20:31:09 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Tue, 28 Apr 2020 22:31:09 +0200
Subject: RFR(M/T) : 8243935 : s.h.WhiteBoxPermission must be
 ClassFileInstall-ed together w/ s.h.WhiteBox
In-Reply-To: <F1DC62C9-43AE-49EF-AF6E-D12A87D481F4@oracle.com>
References: <018B98AF-1DC5-4074-AE80-6601F9FDB867@oracle.com>
 <e5a0d476-c516-b627-920f-9f25f0795961@oracle.com>
 <BBD7C18E-16AC-4BD1-9660-F103FC4F0D41@oracle.com>
 <2d8ef01d-5ace-c55f-6a89-60d8b3a2ced4@oracle.com>
 <c1888765-ca63-64d4-e612-02fcb95932c2@oracle.com>
 <ec0209dd-5a25-d5bc-4b43-b01559cc1412@oracle.com>
 <A35F3B72-5B8D-4B8F-B40F-0A815D3F2D7C@oracle.com>
 <0c30a41f-9031-757f-4e26-93a21ed7de6f@oracle.com>
 <86A49CAF-3C66-4D64-801A-197E38A4BF84@oracle.com>
 <90d19c71-8e60-9e24-6792-3606b0015185@oracle.com>
 <F1DC62C9-43AE-49EF-AF6E-D12A87D481F4@oracle.com>
Message-ID: <787159ea-1901-978b-93b9-0fb91a3ebdca@oracle.com>

Looks good.

Ran hg diff -r -2 -U 0 and did a quick scan.

Are you going to fix occurrences in test/jdk, or do you leave that as an 
exercise for others?

Thanks,
StefanK

On 2020-04-28 18:23, Igor Ignatyev wrote:
> uploaded http://cr.openjdk.java.net/~iignatyev/8243935/8243935.01.patch
>
> -- Igor
>
>> On Apr 28, 2020, at 9:18 AM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
>>
>> On 2020-04-28 18:16, Igor Ignatyev wrote:
>>> Hi Stefan,
>>>
>>> I guess you are interested only in an actual patch, if so I would rather just use hg diff to get the patch and attach/upload it b/c, by some reasons, webrev takes a lot of time to be generated.
>> Sure. hg diff works.
>>
>> StefanK
>>
>>> -- Igor
>>>
>>>> On Apr 28, 2020, at 8:57 AM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
>>>>
>>>> Hi Igor,
>>>>
>>>> On 2020-04-28 17:30, Igor Ignatyev wrote:
>>>>> Hi David, Stefan,
>>>>>
>>>>> there is no issue, this patch is just a spin-off of other patch I was working on. I noticed that some tests copy WhiteBoxPermission and other don't, so I decided to clean this up. I completely forgot about the hack we did in ClassFileInstaller (I guess partially b/c we still have tests which copy WhiteBoxPermission explicitly) so now I think it's better to just remove all explicit copying of WhiteBox$WhiteBoxPermission (which kinda echoes Stefan's patch). this of course touches more files, on the bright side this makes them all neater. so I run sed '/^\s*\*\s*sun.hotspot.WhiteBox\$WhiteBoxPermission/'d to remove all WhiteBoxPermission, and them removed some leftovers in 'containers/docker':
>>>>> - full webrev:
>>>>>    http://cr.openjdk.java.net/~iignatyev//8243935/webrev.01/
>>>> Your open.patch is incorrectly generated and reflects the earlier patch:
>>>> https://cr.openjdk.java.net/~iignatyev//8243935/webrev.01/open.changeset
>>>>
>>>> It's a bug in webrev, but I don't remember under what circumstances it occurs. Could you rerun with: -r qparent -c 8243935 -C -N and see if it works better?
>>>>
>>>> Thanks,
>>>> StefanK
>>>>
>>>>>> 656 lines changed: 0 ins; 612 del; 44 mod;
>>>>> - incremental webrev:
>>>>>    http://cr.openjdk.java.net/~iignatyev//8243935/webrev.0-1/
>>>>>> 722 lines changed: 0 ins; 719 del; 3 mod;
>>>>> if we agree that this is the way to go, I'll repurpose 8243935 to remove WhiteBoxPermission (and obviously will retest the patch).
>>>>>
>>>>> Thanks,
>>>>> -- Igor
>>>>>
>>>>>
>>>>>> On Apr 28, 2020, at 2:49 AM, David Holmes <david.holmes at oracle.com> wrote:
>>>>>>
>>>>>> On 28/04/2020 5:34 pm, Stefan Karlsson wrote:
>>>>>>> Hi,
>>>>>>> I have another idea. Can't we simply remove WhiteBoxPermission and incorporate that functionality in the WhiteBox class? We would then be able to remove all these WhiteBoxPermission lines.
>>>>>> Interesting though a little hackish. :)
>>>>>>
>>>>>> But why is this an issue anyway we already hacked ClassfileInstaller to ensure it always copies the permission class!!!
>>>>>>
>>>>>> 102     // Add commonly used inner classes that are often omitted by mistake. Currently
>>>>>> 103     // we support only sun.hotspot.WhiteBox$WhiteBoxPermission. See JDK-8199290
>>>>>> 104     private static String[] addInnerClasses(String[] classes, int startIdx) {
>>>>>> 105         boolean seenWB = false;
>>>>>> 106         boolean seenWBInner = false;
>>>>>> 107         final String wb = "sun.hotspot.WhiteBox";
>>>>>> 108         final String wbInner = "sun.hotspot.WhiteBox$WhiteBoxPermission";
>>>>>>
>>>>>> David
>>>>>> -----
>>>>>>
>>>>>>> I've tested this and it seems to work:
>>>>>>> https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/ Summary of changes:    695 lines changed: 0 ins; 657 del; 38 mod; 60752 unchg
>>>>>>> The interesting code:
>>>>>>> https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/test/lib/sun/hotspot/WhiteBox.java.udiff.html (There are some parts of the patch that was butchered by the sed-script - I'll fix this if we agree that this is a good thing to do)
>>>>>>> If you think this is something we should do, I'll create an RFE, complete the patch, run more tests, and send it out for review.
>>>>>>> Thanks,
>>>>>>> StefanK
>>>>>>> On 2020-04-28 08:08, David Holmes wrote:
>>>>>>>> On 28/04/2020 3:37 pm, Igor Ignatyev wrote:
>>>>>>>>>> On Apr 27, 2020, at 10:03 PM, David Holmes <david.holmes at oracle.com> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Igor,
>>>>>>>>>>
>>>>>>>>>> On 28/04/2020 12:32 pm, Igor Ignatyev wrote:
>>>>>>>>>>> http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/
>>>>>>>>>>>> 338 lines changed: 107 ins; 0 del; 231 mod
>>>>>>>>>>> Hi all,
>>>>>>>>>>> could you please review the trivial yet tedious patch which adds 'sun.hotspot.WhiteBox$WhiteBoxPermission' to all 'ClassFileInstaller sun.hotspot.WhiteBox' actions?
>>>>>>>>>>> the patch also
>>>>>>>>>>>    - replaces '@run main ClassFileInstaller' w/ '@run driver ClassFileInstaller'
>>>>>>>>>>>    - unifies the way ClassFileInstaller sun.hotspot.WhiteBox are aligned
>>>>>>>>>> That alignment change is unnecessary and makes it very hard to review this patch.
>>>>>>>>>>
>>>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8243935
>>>>>>>>>>> webrevs:
>>>>>>>>>>>    - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/ -- regular webrev
>>>>>>>>>>>    - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.including_ws -- shows "whitespace only" changes
>>>>>>>>>> I'd like to see the patch that leaves out the whitespace-only changes - is that possible?
>>>>>>>>> sure, I've used `hg diff -w` (--ignore-all-space    ignore white space when comparing lines) to remove all whitespace changes, here is webrev -- http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.w/
>>>>>>>> Thanks that was somewhat better. Change seems okay.
>>>>>>>>
>>>>>>>> David
>>>>>>>>
>>>>>>>>> -- Igor
>>>>>>>>>> Thanks,
>>>>>>>>>> David
>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> -- Igor


From stefan.karlsson at oracle.com  Tue Apr 28 20:37:40 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Tue, 28 Apr 2020 22:37:40 +0200
Subject: RFR: 8244010: Simplify usages of
 ProcessTools.createJavaProcessBuilder in our tests
In-Reply-To: <7ea36243-294b-bfd8-a95a-7699c0b776f9@oracle.com>
References: <60c0e0e4-634b-d349-c6f3-ca1ad52926cb@oracle.com>
 <056ff9b1-1a4f-a1fb-2727-b503707f3750@oracle.com>
 <7ea36243-294b-bfd8-a95a-7699c0b776f9@oracle.com>
Message-ID: <a5ba5bed-42de-96d1-acc1-26ec615f13d7@oracle.com>

On 2020-04-28 18:37, Leonid Mesnik wrote:
> Hi
>
> The changes look good.

Thanks, Leonid.

>
> The only small nit. Why don't? restrict arguments to "List<String>" 
> instead of "Collection<String>"? The unordered collections like 
> Set<String> shouldn't be used here. It is a just proposal, no need for 
> new review if you going to fix it.

Sure, I'll change it to List<String>.

Thanks,
StefanK

>
> Leonid
>
> On 4/28/20 6:54 AM, Stefan Karlsson wrote:
>> Hi again,
>>
>> I realized that we probably want to give ProcessTools.executeTestJvm 
>> the same treatment.
>>
>> Side-note: It's very awkard that createJavaProcessBuilder defaults to 
>> not adding user-specifed flags, but executeTestJvm does. I think it 
>> would be good to unify this as a separate RFE. I think *a lot* of 
>> callers to createJavaProcessBuilder could be simplified by either 
>> using executeTestJvm directly, or a simplified version of that.
>>
>> I'm running testing through mach5 and found a few things to fix, I 
>> might find more when the testing has proceeded further.
>>
>> This is the current patch:
>> https://cr.openjdk.java.net/~stefank/8244010/webrev.02.delta
>> https://cr.openjdk.java.net/~stefank/8244010/webrev.02
>>
>> Thanks,
>> StefanK
>>
>> On 2020-04-28 13:58, Stefan Karlsson wrote:
>>> Hi all,
>>>
>>> Please review this patch to simplify usages of 
>>> ProcessTools.createJavaProcessBuilder in our tests.
>>>
>>> https://cr.openjdk.java.net/~stefank/8244010/webrev.01/
>>> https://bugs.openjdk.java.net/browse/JDK-8244010
>>>
>>> I saw all this code when reviewing changes to how we pass flags in 
>>> our tests. There are a many places where arguments are converted and 
>>> passed back and forth in String[] and Collections.
>>>
>>> For example:
>>> ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(
>>> ??????????argsList.toArray(new String[argsList.size()]));
>>>
>>> If we add an overload the createJavaProcessBuilder, that takes a 
>>> Collection<String> as an argument, then we can write the code above as:
>>> ???ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(argsList);
>>>
>>> Other places temporarily put the flags in a String[], where most 
>>> calls simply lists the arguments in the call:
>>> ??String[] opts = {Xmx, "-XX:NativeMemoryTracking=detail", 
>>> "-XX:+UseParallelGC", "-version"};
>>> ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(opts);
>>>
>>> And some places put the args in a temporary Collection:
>>> ??LinkedList<String> vmOptions = new LinkedList<>();
>>> ??vmOptions.add(gc);
>>> ??vmOptions.add("-Xmx" + minMaxHeap);
>>> ??vmOptions.add("-XX:+PrintFlagsFinal");
>>> ??vmOptions.add(VerifyHeapSize.class.getName());
>>>
>>> ??ProcessBuilder pb = 
>>> ProcessTools.createJavaProcessBuilder(vmOptions.toArray(new 
>>> String[0]));
>>>
>>> I'd like to cleanup, simplify, and unify many of these usages.
>>>
>>> I've tested this by running all the changed tests locally.
>>>
>>> Thanks,
>>> StefanK
>>


From stefan.karlsson at oracle.com  Tue Apr 28 20:37:58 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Tue, 28 Apr 2020 22:37:58 +0200
Subject: RFR: 8244010: Simplify usages of
 ProcessTools.createJavaProcessBuilder in our tests
In-Reply-To: <2A1C30A0-7B08-4B51-8AF4-71E1C8F3BCB1@oracle.com>
References: <60c0e0e4-634b-d349-c6f3-ca1ad52926cb@oracle.com>
 <056ff9b1-1a4f-a1fb-2727-b503707f3750@oracle.com>
 <7ea36243-294b-bfd8-a95a-7699c0b776f9@oracle.com>
 <2A1C30A0-7B08-4B51-8AF4-71E1C8F3BCB1@oracle.com>
Message-ID: <22c3439e-b6fd-e3b3-5b87-f9040c73eca1@oracle.com>

Thanks, Igor.

StefanK

On 2020-04-28 18:47, Igor Ignatyev wrote:
> Hi Stefan,
>
> the patch looks good to me.
>
> -- Igor
>
>> On Apr 28, 2020, at 9:37 AM, Leonid Mesnik <leonid.mesnik at oracle.com> wrote:
>>
>> Hi
>>
>> The changes look good.
>>
>> The only small nit. Why don't  restrict arguments to "List<String>" instead of "Collection<String>"? The unordered collections like Set<String> shouldn't be used here. It is a just proposal, no need for new review if you going to fix it.
>>
>> Leonid
>>
>> On 4/28/20 6:54 AM, Stefan Karlsson wrote:
>>> Hi again,
>>>
>>> I realized that we probably want to give ProcessTools.executeTestJvm the same treatment.
>>>
>>> Side-note: It's very awkard that createJavaProcessBuilder defaults to not adding user-specifed flags, but executeTestJvm does. I think it would be good to unify this as a separate RFE. I think *a lot* of callers to createJavaProcessBuilder could be simplified by either using executeTestJvm directly, or a simplified version of that.
>>>
>>> I'm running testing through mach5 and found a few things to fix, I might find more when the testing has proceeded further.
>>>
>>> This is the current patch:
>>> https://cr.openjdk.java.net/~stefank/8244010/webrev.02.delta
>>> https://cr.openjdk.java.net/~stefank/8244010/webrev.02
>>>
>>> Thanks,
>>> StefanK
>>>
>>> On 2020-04-28 13:58, Stefan Karlsson wrote:
>>>> Hi all,
>>>>
>>>> Please review this patch to simplify usages of ProcessTools.createJavaProcessBuilder in our tests.
>>>>
>>>> https://cr.openjdk.java.net/~stefank/8244010/webrev.01/
>>>> https://bugs.openjdk.java.net/browse/JDK-8244010
>>>>
>>>> I saw all this code when reviewing changes to how we pass flags in our tests. There are a many places where arguments are converted and passed back and forth in String[] and Collections.
>>>>
>>>> For example:
>>>>    ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(
>>>>            argsList.toArray(new String[argsList.size()]));
>>>>
>>>> If we add an overload the createJavaProcessBuilder, that takes a Collection<String> as an argument, then we can write the code above as:
>>>>     ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(argsList);
>>>>
>>>> Other places temporarily put the flags in a String[], where most calls simply lists the arguments in the call:
>>>>    String[] opts = {Xmx, "-XX:NativeMemoryTracking=detail", "-XX:+UseParallelGC", "-version"};
>>>>    ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(opts);
>>>>
>>>> And some places put the args in a temporary Collection:
>>>>    LinkedList<String> vmOptions = new LinkedList<>();
>>>>    vmOptions.add(gc);
>>>>    vmOptions.add("-Xmx" + minMaxHeap);
>>>>    vmOptions.add("-XX:+PrintFlagsFinal");
>>>>    vmOptions.add(VerifyHeapSize.class.getName());
>>>>
>>>>    ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(vmOptions.toArray(new String[0]));
>>>>
>>>> I'd like to cleanup, simplify, and unify many of these usages.
>>>>
>>>> I've tested this by running all the changed tests locally.
>>>>
>>>> Thanks,
>>>> StefanK


From igor.ignatyev at oracle.com  Tue Apr 28 21:56:54 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Tue, 28 Apr 2020 14:56:54 -0700
Subject: RFR(M/T) : 8243935 : s.h.WhiteBoxPermission must be
 ClassFileInstall-ed together w/ s.h.WhiteBox
In-Reply-To: <787159ea-1901-978b-93b9-0fb91a3ebdca@oracle.com>
References: <018B98AF-1DC5-4074-AE80-6601F9FDB867@oracle.com>
 <e5a0d476-c516-b627-920f-9f25f0795961@oracle.com>
 <BBD7C18E-16AC-4BD1-9660-F103FC4F0D41@oracle.com>
 <2d8ef01d-5ace-c55f-6a89-60d8b3a2ced4@oracle.com>
 <c1888765-ca63-64d4-e612-02fcb95932c2@oracle.com>
 <ec0209dd-5a25-d5bc-4b43-b01559cc1412@oracle.com>
 <A35F3B72-5B8D-4B8F-B40F-0A815D3F2D7C@oracle.com>
 <0c30a41f-9031-757f-4e26-93a21ed7de6f@oracle.com>
 <86A49CAF-3C66-4D64-801A-197E38A4BF84@oracle.com>
 <90d19c71-8e60-9e24-6792-3606b0015185@oracle.com>
 <F1DC62C9-43AE-49EF-AF6E-D12A87D481F4@oracle.com>
 <787159ea-1901-978b-93b9-0fb91a3ebdca@oracle.com>
Message-ID: <E41A5770-3A31-40BC-AB36-81D5F1B57297@oracle.com>

Thanks Stefan.

I'm going to update other test suites by separate RFEs.

-- Igor

> On Apr 28, 2020, at 1:31 PM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
> 
> Looks good.
> 
> Ran hg diff -r -2 -U 0 and did a quick scan.
> 
> Are you going to fix occurrences in test/jdk, or do you leave that as an exercise for others?
> 
> Thanks,
> StefanK
> 
> On 2020-04-28 18:23, Igor Ignatyev wrote:
>> uploaded http://cr.openjdk.java.net/~iignatyev/8243935/8243935.01.patch
>> 
>> -- Igor
>> 
>>> On Apr 28, 2020, at 9:18 AM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
>>> 
>>> On 2020-04-28 18:16, Igor Ignatyev wrote:
>>>> Hi Stefan,
>>>> 
>>>> I guess you are interested only in an actual patch, if so I would rather just use hg diff to get the patch and attach/upload it b/c, by some reasons, webrev takes a lot of time to be generated.
>>> Sure. hg diff works.
>>> 
>>> StefanK
>>> 
>>>> -- Igor
>>>> 
>>>>> On Apr 28, 2020, at 8:57 AM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
>>>>> 
>>>>> Hi Igor,
>>>>> 
>>>>> On 2020-04-28 17:30, Igor Ignatyev wrote:
>>>>>> Hi David, Stefan,
>>>>>> 
>>>>>> there is no issue, this patch is just a spin-off of other patch I was working on. I noticed that some tests copy WhiteBoxPermission and other don't, so I decided to clean this up. I completely forgot about the hack we did in ClassFileInstaller (I guess partially b/c we still have tests which copy WhiteBoxPermission explicitly) so now I think it's better to just remove all explicit copying of WhiteBox$WhiteBoxPermission (which kinda echoes Stefan's patch). this of course touches more files, on the bright side this makes them all neater. so I run sed '/^\s*\*\s*sun.hotspot.WhiteBox\$WhiteBoxPermission/'d to remove all WhiteBoxPermission, and them removed some leftovers in 'containers/docker':
>>>>>> - full webrev:
>>>>>>   http://cr.openjdk.java.net/~iignatyev//8243935/webrev.01/
>>>>> Your open.patch is incorrectly generated and reflects the earlier patch:
>>>>> https://cr.openjdk.java.net/~iignatyev//8243935/webrev.01/open.changeset
>>>>> 
>>>>> It's a bug in webrev, but I don't remember under what circumstances it occurs. Could you rerun with: -r qparent -c 8243935 -C -N and see if it works better?
>>>>> 
>>>>> Thanks,
>>>>> StefanK
>>>>> 
>>>>>>> 656 lines changed: 0 ins; 612 del; 44 mod;
>>>>>> - incremental webrev:
>>>>>>   http://cr.openjdk.java.net/~iignatyev//8243935/webrev.0-1/
>>>>>>> 722 lines changed: 0 ins; 719 del; 3 mod;
>>>>>> if we agree that this is the way to go, I'll repurpose 8243935 to remove WhiteBoxPermission (and obviously will retest the patch).
>>>>>> 
>>>>>> Thanks,
>>>>>> -- Igor
>>>>>> 
>>>>>> 
>>>>>>> On Apr 28, 2020, at 2:49 AM, David Holmes <david.holmes at oracle.com> wrote:
>>>>>>> 
>>>>>>> On 28/04/2020 5:34 pm, Stefan Karlsson wrote:
>>>>>>>> Hi,
>>>>>>>> I have another idea. Can't we simply remove WhiteBoxPermission and incorporate that functionality in the WhiteBox class? We would then be able to remove all these WhiteBoxPermission lines.
>>>>>>> Interesting though a little hackish. :)
>>>>>>> 
>>>>>>> But why is this an issue anyway we already hacked ClassfileInstaller to ensure it always copies the permission class!!!
>>>>>>> 
>>>>>>> 102     // Add commonly used inner classes that are often omitted by mistake. Currently
>>>>>>> 103     // we support only sun.hotspot.WhiteBox$WhiteBoxPermission. See JDK-8199290
>>>>>>> 104     private static String[] addInnerClasses(String[] classes, int startIdx) {
>>>>>>> 105         boolean seenWB = false;
>>>>>>> 106         boolean seenWBInner = false;
>>>>>>> 107         final String wb = "sun.hotspot.WhiteBox";
>>>>>>> 108         final String wbInner = "sun.hotspot.WhiteBox$WhiteBoxPermission";
>>>>>>> 
>>>>>>> David
>>>>>>> -----
>>>>>>> 
>>>>>>>> I've tested this and it seems to work:
>>>>>>>> https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/ Summary of changes:    695 lines changed: 0 ins; 657 del; 38 mod; 60752 unchg
>>>>>>>> The interesting code:
>>>>>>>> https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/test/lib/sun/hotspot/WhiteBox.java.udiff.html (There are some parts of the patch that was butchered by the sed-script - I'll fix this if we agree that this is a good thing to do)
>>>>>>>> If you think this is something we should do, I'll create an RFE, complete the patch, run more tests, and send it out for review.
>>>>>>>> Thanks,
>>>>>>>> StefanK
>>>>>>>> On 2020-04-28 08:08, David Holmes wrote:
>>>>>>>>> On 28/04/2020 3:37 pm, Igor Ignatyev wrote:
>>>>>>>>>>> On Apr 27, 2020, at 10:03 PM, David Holmes <david.holmes at oracle.com> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Hi Igor,
>>>>>>>>>>> 
>>>>>>>>>>> On 28/04/2020 12:32 pm, Igor Ignatyev wrote:
>>>>>>>>>>>> http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/
>>>>>>>>>>>>> 338 lines changed: 107 ins; 0 del; 231 mod
>>>>>>>>>>>> Hi all,
>>>>>>>>>>>> could you please review the trivial yet tedious patch which adds 'sun.hotspot.WhiteBox$WhiteBoxPermission' to all 'ClassFileInstaller sun.hotspot.WhiteBox' actions?
>>>>>>>>>>>> the patch also
>>>>>>>>>>>>   - replaces '@run main ClassFileInstaller' w/ '@run driver ClassFileInstaller'
>>>>>>>>>>>>   - unifies the way ClassFileInstaller sun.hotspot.WhiteBox are aligned
>>>>>>>>>>> That alignment change is unnecessary and makes it very hard to review this patch.
>>>>>>>>>>> 
>>>>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8243935
>>>>>>>>>>>> webrevs:
>>>>>>>>>>>>   - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/ -- regular webrev
>>>>>>>>>>>>   - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.including_ws -- shows "whitespace only" changes
>>>>>>>>>>> I'd like to see the patch that leaves out the whitespace-only changes - is that possible?
>>>>>>>>>> sure, I've used `hg diff -w` (--ignore-all-space    ignore white space when comparing lines) to remove all whitespace changes, here is webrev -- http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.w/
>>>>>>>>> Thanks that was somewhat better. Change seems okay.
>>>>>>>>> 
>>>>>>>>> David
>>>>>>>>> 
>>>>>>>>>> -- Igor
>>>>>>>>>>> Thanks,
>>>>>>>>>>> David
>>>>>>>>>>> 
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> -- Igor
> 


From coleen.phillimore at oracle.com  Tue Apr 28 23:19:54 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Tue, 28 Apr 2020 19:19:54 -0400
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64 (closed)
In-Reply-To: <C92914CE-B996-44FF-B13D-3E1189C349D1@oracle.com>
References: <9fd23c76-900b-bf6a-3411-64955bd4261c@oracle.com>
 <C92914CE-B996-44FF-B13D-3E1189C349D1@oracle.com>
Message-ID: <728ae6da-8c71-d26c-13a9-7b6e9816c946@oracle.com>



On 4/28/20 4:30 PM, Erik ?sterlund wrote:
> Hi Coleen,
>
> Unfortunately I do need that change already now. Because making 
> compressed oops and compressed class pointers independent involves 
> changing the instanceOop base with compressed oop off and compressed 
> class pointers off. That changes the layout and offsets that are 
> hardcoded (primitive wrappers that wrap a primitive less than 4 bytes 
> now start in the klass gap).

Okay, I didn't see how it related from the diffs.? Now I get it. Your 
patch is good then.? Thanks for removing the hardcoded offsets next.

Coleen

>
> My proposed solution is to remove hardcoded layouts in this patch, and 
> remove hardcoded offsets in the next one. The alternative would be 
> trying to fix the hardcoded layouts only to remove them one patch 
> later. I tried to fix them but ran into issues; that is why I removed 
> them instead in this patch. I hope you think this is okay. The net 
> result will be the same one patch later.
>
> Thanks,
> /Erik
>
>> On 28 Apr 2020, at 21:56, coleen.phillimore at oracle.com wrote:
>>
>> ?
>> Hi Erik,
>>
>> I was trying to figure out why this change is needed to decouple 
>> CompressedOops and CompressedKlassPointers, but it's not needed for 
>> the change, is it?
>>
>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.01/src/hotspot/share/classfile/fieldLayoutBuilder.cpp.udiff.html
>>
>> Can you make this change with your patch for 
>> https://bugs.openjdk.java.net/browse/JDK-8243996 which we really like?
>>
>> Thanks,
>> Coleen
>>
>>
>> On 4/28/20 1:52 PM, Erik ?sterlund wrote:
>>> Hi Coleen,
>>>
>>> Oops, typo in the incremental URL. Here it is:
>>>
>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00..01/
>>>
>>> Thanks for the review.
>>>
>>> /Erik
>>>
>>>> On 28 Apr 2020, at 19:25, coleen.phillimore at oracle.com wrote:
>>>>
>>>> ? Okay.? Your incremental is 4040 but looks good to me.
>>>> Thanks,
>>>> Coleen
>>>>
>>>> On 4/28/20 12:30 PM, Erik ?sterlund wrote:
>>>>> Hi Coleen,
>>>>>
>>>>> Thanks for the review.
>>>>>
>>>>> Here is a new webrev:
>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.01/
>>>>>
>>>>> Incremental:
>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00_01/
>>>>>
>>>>> On 2020-04-28 16:15, coleen.phillimore at oracle.com wrote:
>>>>>> Sorry, this went to the wrong list.
>>>>>>
>>>>>> Coleen
>>>>>>
>>>>>> On 4/24/20 11:17 AM, coleen.phillimore at oracle.comwrote:
>>>>>>>
>>>>>>> Hi Erik, well the interpreter code isn't as bad as I thought it 
>>>>>>> would be.? We tied UseCompressedKlassPointers to 
>>>>>>> UseCompressedOops because we couldn't imagine why someone would 
>>>>>>> care about 4 bytes per object with all 8 byte pointers to oops.? 
>>>>>>> But maybe you do care.
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/cpu/x86/interp_masm_x86.cpp.udiff.html
>>>>>>>
>>>>>>>        if (UseBiasedLocking) {
>>>>>>> - biased_locking_enter(lock_reg, obj_reg, swap_reg, tmp_reg, 
>>>>>>> false, done, &slow_case);
>>>>>>> + Register rklass_decode_tmp = LP64_ONLY(rscratch1) NOT_LP64(noreg);
>>>>>>> + biased_locking_enter(lock_reg, obj_reg, swap_reg, tmp_reg, 
>>>>>>> rscratch1, false, done, &slow_case);
>>>>>>>        }
>>>>>>> You didn't use rklass_decode_tmp here.
>>>>>
>>>>> Nice catch. Fixed.
>>>>>
>>>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/cpu/x86/templateTable_x86.cpp.udiff.html
>>>>>>>
>>>>>>>
>>>>>>> + Register tmp_load_klass = LP64_ONLY(rscratch1) NOT_LP64(noreg);
>>>>>>>
>>>>>>> I wonder if you couldn't just pass rscratch1 into load_klass and 
>>>>>>> handle ignoring it inside the load_klass function, ie. fix the 
>>>>>>> asserts.? This gets repetitive in the templateTable and other 
>>>>>>> code.? Maybe people may have the opposite opinion though.
>>>>>
>>>>> Unfortunately, that won't compile in code that is shared between 
>>>>> 32 bit and 64 bit. In 32 bit-only code I just pass in noreg, and 
>>>>> in 64-bit only code I just send in the temp. But for shared code, 
>>>>> I have to do the macro dance, because rscratch1 does not exist on 
>>>>> 32 bit.
>>>>>
>>>>>>> This is quite nice that UseCompressedKlassPointers doesn't have 
>>>>>>> to reinit_heapbase.? The interpreter code looks good.
>>>>>
>>>>> Thanks!
>>>>>
>>>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/share/memory/metaspace.cpp.frames.html
>>>>>>>
>>>>>>> I don't know if base = HeapBaseMinAddress is right for the 
>>>>>>> compressed class space.? It's reserved but not committed as 1G.? 
>>>>>>> Maybe it makes sense since the heap without UseCompressedOops 
>>>>>>> should be above the class space.
>>>>>
>>>>> Thomas had a similar comment. I think the summary is that yes it 
>>>>> is the right thing to use, but the naming is unfortunate.
>>>>>
>>>>> Hope I fixed it all.
>>>>>
>>>>> Thanks,
>>>>> /Erik
>>>>>
>>>>>>> Coleen
>>>>>>>
>>>>>>> On 4/24/20 4:25 AM, Erik ?sterlund wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> This is the closed version of the mostly open change. There are 
>>>>>>>> some @Contended tests in closed
>>>>>>>> that needed some love with my change to make compressed class 
>>>>>>>> pointers usable with compressed
>>>>>>>> oops disabled. In particular, these tests perform calculations 
>>>>>>>> to ensure that things are padded
>>>>>>>> as expected. Part of those calculations makes assumptions about 
>>>>>>>> how large the object header is,
>>>>>>>> presuming that the number is a constant that is the same for 
>>>>>>>> all objects, and is determinable
>>>>>>>> by checking the offset of the first field. However, with my 
>>>>>>>> changes, the first field will end
>>>>>>>> up either in the klass gap, or after, depending on the field 
>>>>>>>> size. I updated the tests to consider
>>>>>>>> different header sizes depending on the type of fields being 
>>>>>>>> used. It still isn't perfect, but
>>>>>>>> it is clever enough for the purposes of the tests; they pass 
>>>>>>>> with these changes.
>>>>>>>>
>>>>>>>> Bug:
>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8241825
>>>>>>>>
>>>>>>>> Webrev:
>>>>>>>> http://wikifiles.se.oracle.com/dev/eosterlund/8241825/webrev.00/
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> /Erik
>>>>>>>
>>>>>>
>>>>>
>>>>
>>


From igor.ignatyev at oracle.com  Wed Apr 29 00:06:40 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Tue, 28 Apr 2020 17:06:40 -0700
Subject: RFR(M/T) : 8243935 : s.h.WhiteBoxPermission must be
 ClassFileInstall-ed together w/ s.h.WhiteBox
In-Reply-To: <E41A5770-3A31-40BC-AB36-81D5F1B57297@oracle.com>
References: <018B98AF-1DC5-4074-AE80-6601F9FDB867@oracle.com>
 <e5a0d476-c516-b627-920f-9f25f0795961@oracle.com>
 <BBD7C18E-16AC-4BD1-9660-F103FC4F0D41@oracle.com>
 <2d8ef01d-5ace-c55f-6a89-60d8b3a2ced4@oracle.com>
 <c1888765-ca63-64d4-e612-02fcb95932c2@oracle.com>
 <ec0209dd-5a25-d5bc-4b43-b01559cc1412@oracle.com>
 <A35F3B72-5B8D-4B8F-B40F-0A815D3F2D7C@oracle.com>
 <0c30a41f-9031-757f-4e26-93a21ed7de6f@oracle.com>
 <86A49CAF-3C66-4D64-801A-197E38A4BF84@oracle.com>
 <90d19c71-8e60-9e24-6792-3606b0015185@oracle.com>
 <F1DC62C9-43AE-49EF-AF6E-D12A87D481F4@oracle.com>
 <787159ea-1901-978b-93b9-0fb91a3ebdca@oracle.com>
 <E41A5770-3A31-40BC-AB36-81D5F1B57297@oracle.com>
Message-ID: <D91B8CB3-107D-4CAF-96DA-975C96512252@oracle.com>

update:
 - changed the bug's title to 'remove copying of s.h.WB$WhiteBoxPermission in hotspot tests'
 - filed JDK-8244052 for test/jdk tests
 - run hs-tier[1-3] and test/hotspot/jtreg/containers which accounts for all changed tests, all tests have passed

so now I'm just one LGTM short from pushing this.

-- Igor


> On Apr 28, 2020, at 2:56 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
> 
> Thanks Stefan.
> 
> I'm going to update other test suites by separate RFEs.
> 
> -- Igor
> 
>> On Apr 28, 2020, at 1:31 PM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
>> 
>> Looks good.
>> 
>> Ran hg diff -r -2 -U 0 and did a quick scan.
>> 
>> Are you going to fix occurrences in test/jdk, or do you leave that as an exercise for others?
>> 
>> Thanks,
>> StefanK
>> 
>> On 2020-04-28 18:23, Igor Ignatyev wrote:
>>> uploaded http://cr.openjdk.java.net/~iignatyev/8243935/8243935.01.patch
>>> 
>>> -- Igor
>>> 
>>>> On Apr 28, 2020, at 9:18 AM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
>>>> 
>>>> On 2020-04-28 18:16, Igor Ignatyev wrote:
>>>>> Hi Stefan,
>>>>> 
>>>>> I guess you are interested only in an actual patch, if so I would rather just use hg diff to get the patch and attach/upload it b/c, by some reasons, webrev takes a lot of time to be generated.
>>>> Sure. hg diff works.
>>>> 
>>>> StefanK
>>>> 
>>>>> -- Igor
>>>>> 
>>>>>> On Apr 28, 2020, at 8:57 AM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
>>>>>> 
>>>>>> Hi Igor,
>>>>>> 
>>>>>> On 2020-04-28 17:30, Igor Ignatyev wrote:
>>>>>>> Hi David, Stefan,
>>>>>>> 
>>>>>>> there is no issue, this patch is just a spin-off of other patch I was working on. I noticed that some tests copy WhiteBoxPermission and other don't, so I decided to clean this up. I completely forgot about the hack we did in ClassFileInstaller (I guess partially b/c we still have tests which copy WhiteBoxPermission explicitly) so now I think it's better to just remove all explicit copying of WhiteBox$WhiteBoxPermission (which kinda echoes Stefan's patch). this of course touches more files, on the bright side this makes them all neater. so I run sed '/^\s*\*\s*sun.hotspot.WhiteBox\$WhiteBoxPermission/'d to remove all WhiteBoxPermission, and them removed some leftovers in 'containers/docker':
>>>>>>> - full webrev:
>>>>>>>  http://cr.openjdk.java.net/~iignatyev//8243935/webrev.01/
>>>>>> Your open.patch is incorrectly generated and reflects the earlier patch:
>>>>>> https://cr.openjdk.java.net/~iignatyev//8243935/webrev.01/open.changeset
>>>>>> 
>>>>>> It's a bug in webrev, but I don't remember under what circumstances it occurs. Could you rerun with: -r qparent -c 8243935 -C -N and see if it works better?
>>>>>> 
>>>>>> Thanks,
>>>>>> StefanK
>>>>>> 
>>>>>>>> 656 lines changed: 0 ins; 612 del; 44 mod;
>>>>>>> - incremental webrev:
>>>>>>>  http://cr.openjdk.java.net/~iignatyev//8243935/webrev.0-1/
>>>>>>>> 722 lines changed: 0 ins; 719 del; 3 mod;
>>>>>>> if we agree that this is the way to go, I'll repurpose 8243935 to remove WhiteBoxPermission (and obviously will retest the patch).
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> -- Igor
>>>>>>> 
>>>>>>> 
>>>>>>>> On Apr 28, 2020, at 2:49 AM, David Holmes <david.holmes at oracle.com> wrote:
>>>>>>>> 
>>>>>>>> On 28/04/2020 5:34 pm, Stefan Karlsson wrote:
>>>>>>>>> Hi,
>>>>>>>>> I have another idea. Can't we simply remove WhiteBoxPermission and incorporate that functionality in the WhiteBox class? We would then be able to remove all these WhiteBoxPermission lines.
>>>>>>>> Interesting though a little hackish. :)
>>>>>>>> 
>>>>>>>> But why is this an issue anyway we already hacked ClassfileInstaller to ensure it always copies the permission class!!!
>>>>>>>> 
>>>>>>>> 102     // Add commonly used inner classes that are often omitted by mistake. Currently
>>>>>>>> 103     // we support only sun.hotspot.WhiteBox$WhiteBoxPermission. See JDK-8199290
>>>>>>>> 104     private static String[] addInnerClasses(String[] classes, int startIdx) {
>>>>>>>> 105         boolean seenWB = false;
>>>>>>>> 106         boolean seenWBInner = false;
>>>>>>>> 107         final String wb = "sun.hotspot.WhiteBox";
>>>>>>>> 108         final String wbInner = "sun.hotspot.WhiteBox$WhiteBoxPermission";
>>>>>>>> 
>>>>>>>> David
>>>>>>>> -----
>>>>>>>> 
>>>>>>>>> I've tested this and it seems to work:
>>>>>>>>> https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/ Summary of changes:    695 lines changed: 0 ins; 657 del; 38 mod; 60752 unchg
>>>>>>>>> The interesting code:
>>>>>>>>> https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/test/lib/sun/hotspot/WhiteBox.java.udiff.html (There are some parts of the patch that was butchered by the sed-script - I'll fix this if we agree that this is a good thing to do)
>>>>>>>>> If you think this is something we should do, I'll create an RFE, complete the patch, run more tests, and send it out for review.
>>>>>>>>> Thanks,
>>>>>>>>> StefanK
>>>>>>>>> On 2020-04-28 08:08, David Holmes wrote:
>>>>>>>>>> On 28/04/2020 3:37 pm, Igor Ignatyev wrote:
>>>>>>>>>>>> On Apr 27, 2020, at 10:03 PM, David Holmes <david.holmes at oracle.com> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Hi Igor,
>>>>>>>>>>>> 
>>>>>>>>>>>> On 28/04/2020 12:32 pm, Igor Ignatyev wrote:
>>>>>>>>>>>>> http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/
>>>>>>>>>>>>>> 338 lines changed: 107 ins; 0 del; 231 mod
>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>> could you please review the trivial yet tedious patch which adds 'sun.hotspot.WhiteBox$WhiteBoxPermission' to all 'ClassFileInstaller sun.hotspot.WhiteBox' actions?
>>>>>>>>>>>>> the patch also
>>>>>>>>>>>>>  - replaces '@run main ClassFileInstaller' w/ '@run driver ClassFileInstaller'
>>>>>>>>>>>>>  - unifies the way ClassFileInstaller sun.hotspot.WhiteBox are aligned
>>>>>>>>>>>> That alignment change is unnecessary and makes it very hard to review this patch.
>>>>>>>>>>>> 
>>>>>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8243935
>>>>>>>>>>>>> webrevs:
>>>>>>>>>>>>>  - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/ -- regular webrev
>>>>>>>>>>>>>  - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.including_ws -- shows "whitespace only" changes
>>>>>>>>>>>> I'd like to see the patch that leaves out the whitespace-only changes - is that possible?
>>>>>>>>>>> sure, I've used `hg diff -w` (--ignore-all-space    ignore white space when comparing lines) to remove all whitespace changes, here is webrev -- http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.w/
>>>>>>>>>> Thanks that was somewhat better. Change seems okay.
>>>>>>>>>> 
>>>>>>>>>> David
>>>>>>>>>> 
>>>>>>>>>>> -- Igor
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> David
>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> -- Igor
>> 
> 


From patricio.chilano.mateo at oracle.com  Wed Apr 29 00:18:53 2020
From: patricio.chilano.mateo at oracle.com (Patricio Chilano)
Date: Tue, 28 Apr 2020 21:18:53 -0300
Subject: Thread-Local Handshakes slowdown on single-cpu machines
In-Reply-To: <7216aa59-ad6f-dd3c-4f37-4723d3b7e9a7@palacsint.hu>
References: <5e65ba58-b599-e83c-b4ea-3d766ed3c457@palacsint.hu>
 <66059982-49bf-1a15-39fb-22008b069573@oracle.com>
 <7216aa59-ad6f-dd3c-4f37-4723d3b7e9a7@palacsint.hu>
Message-ID: <857ffb5e-d9e5-3976-3968-124e3b47effb@oracle.com>

Hi Miklos,

On 4/28/20 8:04 AM, Karak? Mikl?s wrote:
> Hello David,
>
> I've tried with OpenJDK 15 EA Build 20. Unfortunately it's slow as 
> with OpenJDK14.
> A colleague of mine set CPU affinity with taskset to only one CPU on 
> Linux and did not experience the slowdown. So the issue seems 
> Windows-related.
I wouldn?t rule out this being a scheduling issue. With thread-local 
handshakes some operations like the ones you see in your logs 
(deoptimization, biasedlocking revocation) do not need all JavaThreads 
to be stopped at the same time in order to be executed. If you only have 
one CPU but many JavaThreads running, depending on the scheduler, maybe 
it could happen that the JavaThread you need to handshake is constantly 
being left behind in the scheduler queue and so the operation takes 
longer to execute. When you add -XX:-ThreadLocalHandshakes, those 
operations will be executed using safepoints, which means all 
JavaThreads are stopped. So even if the scheduler decides to give 
priority to other JavaThreads those will block and free the CPU. Also 
when adding more CPU that will increase the likelihood of a JavaThread 
getting scheduled and might explain why you see it gets fixed.

Alternatively the JavaThread that needs to be handshaked is being 
scheduled but is not polling for the handshake. But I think in that the 
case you should still have the same issue with 
-XX:-ThreadLocalHandshakes because the polling mechanism is the same, 
although it's true that the misbehaving JavaThread will have more CPU 
time to complete whatever is doing while others are stopped when 
safepointing.

One thing that makes me question the scheduler theory is the fact that 
you see such improvements when only disabling biased-locking in 13.0.2 
because that was still using safepoints back then. Can you check the 
logs to see in which operations you see the pauses when running with 
default options compared to -XX:-UseBiasedLocking?

I?m also surprised you see those long logging pauses because all the 
other JavaThreads were already processed in the handshake so something 
must be executing. Maybe you can switch the logging to trace instead of 
debug. Also maybe you can add logging at Java code so you can follow 
what is actually running? You could send it to the same app.txt and 
flush every time you write to see it ordered in the log. You can also 
get the uptime you see in the UL logs with 
ManagementFactory.getRuntimeMXBean().getUptime(). That logging will 
affect timing but might give some insights.

Thanks,
Patricio
> Miklos
>
> On 4/28/20 1:02 AM, David Holmes wrote:
>> Hi Miklos,
>>
>> On 28/04/2020 7:26 am, Karak? Mikl?s wrote:
>>> Hello,
>>>
>>> We bumped into a possible Thread-Local Handshakes issue with 
>>> multiple apps. It seems that enabled TLH slows down applications 
>>> running on (although rare) single CPU boxes. I would be grateful if 
>>> you could confirm that this is a known trade-off for these rare 
>>> setups or a possible JVM bug. That would save us at least a few 
>>> hours of debugging.
>>>
>>> TL;DR: Both tested apps stop frequently around "HandshakeOneThread", 
>>> "HandshakeAllThreads" and "Revoked bias of currently-unlocked 
>>> object" running with OpenJDK12/OpenJDK13/OpenJDK14. OpenJDK 13 with 
>>> -XX:-ThreadLocalHandshakes fixes the issue. Adding a second CPU to 
>>> the virtual machine fixes the issue. Enabling hyper-threading fixes 
>>> the issue.
>>
>> Can you try with latest JDK 15 build, just to see if this may be 
>> something already addressed?
>>
>> Thanks,
>> David
>>
>>> More details are available at StackOverflow: 
>>> https://stackoverflow.com/questions/61375565/slow-application-frequent-jvm-hangs-with-single-cpu-setups-and-java-12 
>>>
>>>
>>> All thoughts are welcome.
>>>
>>> Best,
>>> Miklos
>>>
>


From david.holmes at oracle.com  Wed Apr 29 02:31:23 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 29 Apr 2020 12:31:23 +1000
Subject: RFR(M/T) : 8243935 : s.h.WhiteBoxPermission must be
 ClassFileInstall-ed together w/ s.h.WhiteBox
In-Reply-To: <D91B8CB3-107D-4CAF-96DA-975C96512252@oracle.com>
References: <018B98AF-1DC5-4074-AE80-6601F9FDB867@oracle.com>
 <e5a0d476-c516-b627-920f-9f25f0795961@oracle.com>
 <BBD7C18E-16AC-4BD1-9660-F103FC4F0D41@oracle.com>
 <2d8ef01d-5ace-c55f-6a89-60d8b3a2ced4@oracle.com>
 <c1888765-ca63-64d4-e612-02fcb95932c2@oracle.com>
 <ec0209dd-5a25-d5bc-4b43-b01559cc1412@oracle.com>
 <A35F3B72-5B8D-4B8F-B40F-0A815D3F2D7C@oracle.com>
 <0c30a41f-9031-757f-4e26-93a21ed7de6f@oracle.com>
 <86A49CAF-3C66-4D64-801A-197E38A4BF84@oracle.com>
 <90d19c71-8e60-9e24-6792-3606b0015185@oracle.com>
 <F1DC62C9-43AE-49EF-AF6E-D12A87D481F4@oracle.com>
 <787159ea-1901-978b-93b9-0fb91a3ebdca@oracle.com>
 <E41A5770-3A31-40BC-AB36-81D5F1B57297@oracle.com>
 <D91B8CB3-107D-4CAF-96DA-975C96512252@oracle.com>
Message-ID: <4158fb9a-23c8-9227-9ec4-1be2b6a46975@oracle.com>

Hi Igor,

On 29/04/2020 10:06 am, Igor Ignatyev wrote:
> update:
>   - changed the bug's title to 'remove copying of s.h.WB$WhiteBoxPermission in hotspot tests'
>   - filed JDK-8244052 for test/jdk tests
>   - run hs-tier[1-3] and test/hotspot/jtreg/containers which accounts for all changed tests, all tests have passed
> 
> so now I'm just one LGTM short from pushing this.

I can't say I think this was a particularly worthwhile exercise, but 
FWIW LGTM.

However, you will also need to ensure all copyrights are updated if needed.

Thanks,
David

> -- Igor
> 
> 
>> On Apr 28, 2020, at 2:56 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
>>
>> Thanks Stefan.
>>
>> I'm going to update other test suites by separate RFEs.
>>
>> -- Igor
>>
>>> On Apr 28, 2020, at 1:31 PM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
>>>
>>> Looks good.
>>>
>>> Ran hg diff -r -2 -U 0 and did a quick scan.
>>>
>>> Are you going to fix occurrences in test/jdk, or do you leave that as an exercise for others?
>>>
>>> Thanks,
>>> StefanK
>>>
>>> On 2020-04-28 18:23, Igor Ignatyev wrote:
>>>> uploaded http://cr.openjdk.java.net/~iignatyev/8243935/8243935.01.patch
>>>>
>>>> -- Igor
>>>>
>>>>> On Apr 28, 2020, at 9:18 AM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
>>>>>
>>>>> On 2020-04-28 18:16, Igor Ignatyev wrote:
>>>>>> Hi Stefan,
>>>>>>
>>>>>> I guess you are interested only in an actual patch, if so I would rather just use hg diff to get the patch and attach/upload it b/c, by some reasons, webrev takes a lot of time to be generated.
>>>>> Sure. hg diff works.
>>>>>
>>>>> StefanK
>>>>>
>>>>>> -- Igor
>>>>>>
>>>>>>> On Apr 28, 2020, at 8:57 AM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
>>>>>>>
>>>>>>> Hi Igor,
>>>>>>>
>>>>>>> On 2020-04-28 17:30, Igor Ignatyev wrote:
>>>>>>>> Hi David, Stefan,
>>>>>>>>
>>>>>>>> there is no issue, this patch is just a spin-off of other patch I was working on. I noticed that some tests copy WhiteBoxPermission and other don't, so I decided to clean this up. I completely forgot about the hack we did in ClassFileInstaller (I guess partially b/c we still have tests which copy WhiteBoxPermission explicitly) so now I think it's better to just remove all explicit copying of WhiteBox$WhiteBoxPermission (which kinda echoes Stefan's patch). this of course touches more files, on the bright side this makes them all neater. so I run sed '/^\s*\*\s*sun.hotspot.WhiteBox\$WhiteBoxPermission/'d to remove all WhiteBoxPermission, and them removed some leftovers in 'containers/docker':
>>>>>>>> - full webrev:
>>>>>>>>   http://cr.openjdk.java.net/~iignatyev//8243935/webrev.01/
>>>>>>> Your open.patch is incorrectly generated and reflects the earlier patch:
>>>>>>> https://cr.openjdk.java.net/~iignatyev//8243935/webrev.01/open.changeset
>>>>>>>
>>>>>>> It's a bug in webrev, but I don't remember under what circumstances it occurs. Could you rerun with: -r qparent -c 8243935 -C -N and see if it works better?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> StefanK
>>>>>>>
>>>>>>>>> 656 lines changed: 0 ins; 612 del; 44 mod;
>>>>>>>> - incremental webrev:
>>>>>>>>   http://cr.openjdk.java.net/~iignatyev//8243935/webrev.0-1/
>>>>>>>>> 722 lines changed: 0 ins; 719 del; 3 mod;
>>>>>>>> if we agree that this is the way to go, I'll repurpose 8243935 to remove WhiteBoxPermission (and obviously will retest the patch).
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -- Igor
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Apr 28, 2020, at 2:49 AM, David Holmes <david.holmes at oracle.com> wrote:
>>>>>>>>>
>>>>>>>>> On 28/04/2020 5:34 pm, Stefan Karlsson wrote:
>>>>>>>>>> Hi,
>>>>>>>>>> I have another idea. Can't we simply remove WhiteBoxPermission and incorporate that functionality in the WhiteBox class? We would then be able to remove all these WhiteBoxPermission lines.
>>>>>>>>> Interesting though a little hackish. :)
>>>>>>>>>
>>>>>>>>> But why is this an issue anyway we already hacked ClassfileInstaller to ensure it always copies the permission class!!!
>>>>>>>>>
>>>>>>>>> 102     // Add commonly used inner classes that are often omitted by mistake. Currently
>>>>>>>>> 103     // we support only sun.hotspot.WhiteBox$WhiteBoxPermission. See JDK-8199290
>>>>>>>>> 104     private static String[] addInnerClasses(String[] classes, int startIdx) {
>>>>>>>>> 105         boolean seenWB = false;
>>>>>>>>> 106         boolean seenWBInner = false;
>>>>>>>>> 107         final String wb = "sun.hotspot.WhiteBox";
>>>>>>>>> 108         final String wbInner = "sun.hotspot.WhiteBox$WhiteBoxPermission";
>>>>>>>>>
>>>>>>>>> David
>>>>>>>>> -----
>>>>>>>>>
>>>>>>>>>> I've tested this and it seems to work:
>>>>>>>>>> https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/ Summary of changes:    695 lines changed: 0 ins; 657 del; 38 mod; 60752 unchg
>>>>>>>>>> The interesting code:
>>>>>>>>>> https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/test/lib/sun/hotspot/WhiteBox.java.udiff.html (There are some parts of the patch that was butchered by the sed-script - I'll fix this if we agree that this is a good thing to do)
>>>>>>>>>> If you think this is something we should do, I'll create an RFE, complete the patch, run more tests, and send it out for review.
>>>>>>>>>> Thanks,
>>>>>>>>>> StefanK
>>>>>>>>>> On 2020-04-28 08:08, David Holmes wrote:
>>>>>>>>>>> On 28/04/2020 3:37 pm, Igor Ignatyev wrote:
>>>>>>>>>>>>> On Apr 27, 2020, at 10:03 PM, David Holmes <david.holmes at oracle.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Igor,
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 28/04/2020 12:32 pm, Igor Ignatyev wrote:
>>>>>>>>>>>>>> http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/
>>>>>>>>>>>>>>> 338 lines changed: 107 ins; 0 del; 231 mod
>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>> could you please review the trivial yet tedious patch which adds 'sun.hotspot.WhiteBox$WhiteBoxPermission' to all 'ClassFileInstaller sun.hotspot.WhiteBox' actions?
>>>>>>>>>>>>>> the patch also
>>>>>>>>>>>>>>   - replaces '@run main ClassFileInstaller' w/ '@run driver ClassFileInstaller'
>>>>>>>>>>>>>>   - unifies the way ClassFileInstaller sun.hotspot.WhiteBox are aligned
>>>>>>>>>>>>> That alignment change is unnecessary and makes it very hard to review this patch.
>>>>>>>>>>>>>
>>>>>>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8243935
>>>>>>>>>>>>>> webrevs:
>>>>>>>>>>>>>>   - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/ -- regular webrev
>>>>>>>>>>>>>>   - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.including_ws -- shows "whitespace only" changes
>>>>>>>>>>>>> I'd like to see the patch that leaves out the whitespace-only changes - is that possible?
>>>>>>>>>>>> sure, I've used `hg diff -w` (--ignore-all-space    ignore white space when comparing lines) to remove all whitespace changes, here is webrev -- http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.w/
>>>>>>>>>>> Thanks that was somewhat better. Change seems okay.
>>>>>>>>>>>
>>>>>>>>>>> David
>>>>>>>>>>>
>>>>>>>>>>>> -- Igor
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> David
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> -- Igor
>>>
>>
> 

From igor.ignatyev at oracle.com  Wed Apr 29 02:58:42 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Tue, 28 Apr 2020 19:58:42 -0700
Subject: RFR(M/T) : 8243935 : s.h.WhiteBoxPermission must be
 ClassFileInstall-ed together w/ s.h.WhiteBox
In-Reply-To: <4158fb9a-23c8-9227-9ec4-1be2b6a46975@oracle.com>
References: <018B98AF-1DC5-4074-AE80-6601F9FDB867@oracle.com>
 <e5a0d476-c516-b627-920f-9f25f0795961@oracle.com>
 <BBD7C18E-16AC-4BD1-9660-F103FC4F0D41@oracle.com>
 <2d8ef01d-5ace-c55f-6a89-60d8b3a2ced4@oracle.com>
 <c1888765-ca63-64d4-e612-02fcb95932c2@oracle.com>
 <ec0209dd-5a25-d5bc-4b43-b01559cc1412@oracle.com>
 <A35F3B72-5B8D-4B8F-B40F-0A815D3F2D7C@oracle.com>
 <0c30a41f-9031-757f-4e26-93a21ed7de6f@oracle.com>
 <86A49CAF-3C66-4D64-801A-197E38A4BF84@oracle.com>
 <90d19c71-8e60-9e24-6792-3606b0015185@oracle.com>
 <F1DC62C9-43AE-49EF-AF6E-D12A87D481F4@oracle.com>
 <787159ea-1901-978b-93b9-0fb91a3ebdca@oracle.com>
 <E41A5770-3A31-40BC-AB36-81D5F1B57297@oracle.com>
 <D91B8CB3-107D-4CAF-96DA-975C96512252@oracle.com>
 <4158fb9a-23c8-9227-9ec4-1be2b6a46975@oracle.com>
Message-ID: <70CB814B-0962-46A5-BFC2-D080AAE3EF27@oracle.com>

Thanks for review David, I really appreciate you spending time on it.

sure-sure, 545 out of 638 files got copyrights update before the push. 

Thanks,
-- Igor

> On Apr 28, 2020, at 7:31 PM, David Holmes <david.holmes at oracle.com> wrote:
> 
> Hi Igor,
> 
> On 29/04/2020 10:06 am, Igor Ignatyev wrote:
>> update:
>>  - changed the bug's title to 'remove copying of s.h.WB$WhiteBoxPermission in hotspot tests'
>>  - filed JDK-8244052 for test/jdk tests
>>  - run hs-tier[1-3] and test/hotspot/jtreg/containers which accounts for all changed tests, all tests have passed
>> so now I'm just one LGTM short from pushing this.
> 
> I can't say I think this was a particularly worthwhile exercise, but FWIW LGTM.
> 
> However, you will also need to ensure all copyrights are updated if needed.
> 
> Thanks,
> David
> 
>> -- Igor
>>> On Apr 28, 2020, at 2:56 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
>>> 
>>> Thanks Stefan.
>>> 
>>> I'm going to update other test suites by separate RFEs.
>>> 
>>> -- Igor
>>> 
>>>> On Apr 28, 2020, at 1:31 PM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
>>>> 
>>>> Looks good.
>>>> 
>>>> Ran hg diff -r -2 -U 0 and did a quick scan.
>>>> 
>>>> Are you going to fix occurrences in test/jdk, or do you leave that as an exercise for others?
>>>> 
>>>> Thanks,
>>>> StefanK
>>>> 
>>>> On 2020-04-28 18:23, Igor Ignatyev wrote:
>>>>> uploaded http://cr.openjdk.java.net/~iignatyev/8243935/8243935.01.patch
>>>>> 
>>>>> -- Igor
>>>>> 
>>>>>> On Apr 28, 2020, at 9:18 AM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
>>>>>> 
>>>>>> On 2020-04-28 18:16, Igor Ignatyev wrote:
>>>>>>> Hi Stefan,
>>>>>>> 
>>>>>>> I guess you are interested only in an actual patch, if so I would rather just use hg diff to get the patch and attach/upload it b/c, by some reasons, webrev takes a lot of time to be generated.
>>>>>> Sure. hg diff works.
>>>>>> 
>>>>>> StefanK
>>>>>> 
>>>>>>> -- Igor
>>>>>>> 
>>>>>>>> On Apr 28, 2020, at 8:57 AM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
>>>>>>>> 
>>>>>>>> Hi Igor,
>>>>>>>> 
>>>>>>>> On 2020-04-28 17:30, Igor Ignatyev wrote:
>>>>>>>>> Hi David, Stefan,
>>>>>>>>> 
>>>>>>>>> there is no issue, this patch is just a spin-off of other patch I was working on. I noticed that some tests copy WhiteBoxPermission and other don't, so I decided to clean this up. I completely forgot about the hack we did in ClassFileInstaller (I guess partially b/c we still have tests which copy WhiteBoxPermission explicitly) so now I think it's better to just remove all explicit copying of WhiteBox$WhiteBoxPermission (which kinda echoes Stefan's patch). this of course touches more files, on the bright side this makes them all neater. so I run sed '/^\s*\*\s*sun.hotspot.WhiteBox\$WhiteBoxPermission/'d to remove all WhiteBoxPermission, and them removed some leftovers in 'containers/docker':
>>>>>>>>> - full webrev:
>>>>>>>>>  http://cr.openjdk.java.net/~iignatyev//8243935/webrev.01/
>>>>>>>> Your open.patch is incorrectly generated and reflects the earlier patch:
>>>>>>>> https://cr.openjdk.java.net/~iignatyev//8243935/webrev.01/open.changeset
>>>>>>>> 
>>>>>>>> It's a bug in webrev, but I don't remember under what circumstances it occurs. Could you rerun with: -r qparent -c 8243935 -C -N and see if it works better?
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> StefanK
>>>>>>>> 
>>>>>>>>>> 656 lines changed: 0 ins; 612 del; 44 mod;
>>>>>>>>> - incremental webrev:
>>>>>>>>>  http://cr.openjdk.java.net/~iignatyev//8243935/webrev.0-1/
>>>>>>>>>> 722 lines changed: 0 ins; 719 del; 3 mod;
>>>>>>>>> if we agree that this is the way to go, I'll repurpose 8243935 to remove WhiteBoxPermission (and obviously will retest the patch).
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> -- Igor
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> On Apr 28, 2020, at 2:49 AM, David Holmes <david.holmes at oracle.com> wrote:
>>>>>>>>>> 
>>>>>>>>>> On 28/04/2020 5:34 pm, Stefan Karlsson wrote:
>>>>>>>>>>> Hi,
>>>>>>>>>>> I have another idea. Can't we simply remove WhiteBoxPermission and incorporate that functionality in the WhiteBox class? We would then be able to remove all these WhiteBoxPermission lines.
>>>>>>>>>> Interesting though a little hackish. :)
>>>>>>>>>> 
>>>>>>>>>> But why is this an issue anyway we already hacked ClassfileInstaller to ensure it always copies the permission class!!!
>>>>>>>>>> 
>>>>>>>>>> 102     // Add commonly used inner classes that are often omitted by mistake. Currently
>>>>>>>>>> 103     // we support only sun.hotspot.WhiteBox$WhiteBoxPermission. See JDK-8199290
>>>>>>>>>> 104     private static String[] addInnerClasses(String[] classes, int startIdx) {
>>>>>>>>>> 105         boolean seenWB = false;
>>>>>>>>>> 106         boolean seenWBInner = false;
>>>>>>>>>> 107         final String wb = "sun.hotspot.WhiteBox";
>>>>>>>>>> 108         final String wbInner = "sun.hotspot.WhiteBox$WhiteBoxPermission";
>>>>>>>>>> 
>>>>>>>>>> David
>>>>>>>>>> -----
>>>>>>>>>> 
>>>>>>>>>>> I've tested this and it seems to work:
>>>>>>>>>>> https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/ Summary of changes:    695 lines changed: 0 ins; 657 del; 38 mod; 60752 unchg
>>>>>>>>>>> The interesting code:
>>>>>>>>>>> https://cr.openjdk.java.net/~stefank/prototype/removeWhiteBoxPermission/webrev.01/test/lib/sun/hotspot/WhiteBox.java.udiff.html (There are some parts of the patch that was butchered by the sed-script - I'll fix this if we agree that this is a good thing to do)
>>>>>>>>>>> If you think this is something we should do, I'll create an RFE, complete the patch, run more tests, and send it out for review.
>>>>>>>>>>> Thanks,
>>>>>>>>>>> StefanK
>>>>>>>>>>> On 2020-04-28 08:08, David Holmes wrote:
>>>>>>>>>>>> On 28/04/2020 3:37 pm, Igor Ignatyev wrote:
>>>>>>>>>>>>>> On Apr 27, 2020, at 10:03 PM, David Holmes <david.holmes at oracle.com> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Hi Igor,
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On 28/04/2020 12:32 pm, Igor Ignatyev wrote:
>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/
>>>>>>>>>>>>>>>> 338 lines changed: 107 ins; 0 del; 231 mod
>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>> could you please review the trivial yet tedious patch which adds 'sun.hotspot.WhiteBox$WhiteBoxPermission' to all 'ClassFileInstaller sun.hotspot.WhiteBox' actions?
>>>>>>>>>>>>>>> the patch also
>>>>>>>>>>>>>>>  - replaces '@run main ClassFileInstaller' w/ '@run driver ClassFileInstaller'
>>>>>>>>>>>>>>>  - unifies the way ClassFileInstaller sun.hotspot.WhiteBox are aligned
>>>>>>>>>>>>>> That alignment change is unnecessary and makes it very hard to review this patch.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8243935
>>>>>>>>>>>>>>> webrevs:
>>>>>>>>>>>>>>>  - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00/ -- regular webrev
>>>>>>>>>>>>>>>  - http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.including_ws -- shows "whitespace only" changes
>>>>>>>>>>>>>> I'd like to see the patch that leaves out the whitespace-only changes - is that possible?
>>>>>>>>>>>>> sure, I've used `hg diff -w` (--ignore-all-space    ignore white space when comparing lines) to remove all whitespace changes, here is webrev -- http://cr.openjdk.java.net/~iignatyev//8243935/webrev.00.w/
>>>>>>>>>>>> Thanks that was somewhat better. Change seems okay.
>>>>>>>>>>>> 
>>>>>>>>>>>> David
>>>>>>>>>>>> 
>>>>>>>>>>>>> -- Igor
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> David
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> -- Igor
>>>> 
>>> 


From palacsint at palacsint.hu  Wed Apr 29 03:26:43 2020
From: palacsint at palacsint.hu (=?UTF-8?B?S2FyYWvDsyBNaWtsw7Nz?=)
Date: Wed, 29 Apr 2020 05:26:43 +0200
Subject: Thread-Local Handshakes slowdown on single-cpu machines
In-Reply-To: <857ffb5e-d9e5-3976-3968-124e3b47effb@oracle.com>
References: <5e65ba58-b599-e83c-b4ea-3d766ed3c457@palacsint.hu>
 <66059982-49bf-1a15-39fb-22008b069573@oracle.com>
 <7216aa59-ad6f-dd3c-4f37-4723d3b7e9a7@palacsint.hu>
 <857ffb5e-d9e5-3976-3968-124e3b47effb@oracle.com>
Message-ID: <f23d9ba4-3380-050a-d772-8a8a8eeb5b15@palacsint.hu>

Hi Patricio,

Yes, it seems like a scheduling issue for us too. One of my colleagues 
spotted the following sentence in Robbin Ehn's Handshaking HotSpot 
presentation (https://youtu.be/VBCOfAJ409s?t=2222): ~"if the VM thread 
finds that all threads are unsafe, it keeps spinning"
Could it be that actually the VM thread gets more CPU time and actually 
blocks JavaThread?

I did not except that but I could reproduce the issue with a simple 
hello world:

public class Main {
 ??? public static void main(String[] args) {
 ??? ??? System.out.println("Hello world");
 ??? }
}

A few measurements on Windows 10 x64 in VirtualBox:


PS Z:\main> Measure-Command { .\main-1cpu.bat }

Minutes?????????? : 0
Seconds?????????? : 7
Milliseconds????? : 20
Ticks???????????? : 70203455
TotalMinutes????? : 0.117005758333333
TotalSeconds????? : 7.0203455
TotalMilliseconds : 7020.3455



PS Z:\main> Measure-Command { .\main-full.bat }

Minutes?????????? : 0
Seconds?????????? : 1
Milliseconds????? : 575
Ticks???????????? : 15751352
TotalSeconds????? : 1.5751352
TotalMilliseconds : 1575.1352



PS Z:\main> Measure-Command { .\main-full.bat }

Minutes?????????? : 0
Seconds?????????? : 1
Milliseconds????? : 558
Ticks???????????? : 15585384
TotalSeconds????? : 1.5585384
TotalMilliseconds : 1558.5384



PS Z:\main> Measure-Command { .\main-1cpu.bat }

Minutes?????????? : 0
Seconds?????????? : 23
Milliseconds????? : 648
Ticks???????????? : 236482685
TotalSeconds????? : 23.6482685
TotalMilliseconds : 23648.2685


main-1cpu.bat:

c:\windows\system32\cmd.exe /C start /affinity 1 \jdk-14\bin\java 
-Xlog:all=trace:file=app-1cpu.txt:uptime,tid,level,tags:filecount=50 Main

main-full.bat:

c:\windows\system32\cmd.exe /C start /affinity FF \jdk-14\bin\java 
-Xlog:all=trace:file=app-full.txt:uptime,tid,level,tags:filecount=50 Main

The files are also available here:
https://palacsint.hu/etc/jdk/jdk14-single-cpu-slowdown-reprod-poc/
Or in a zip:
https://palacsint.hu/etc/jdk/jdk14-single-cpu-slowdown-reprod-poc.zip

You can find trace level logs of three testruns in the run1-2-3 
directories as well.

Best,
Miklos


On 4/29/20 2:18 AM, Patricio Chilano wrote:
> Hi Miklos,
>
> On 4/28/20 8:04 AM, Karak? Mikl?s wrote:
>> Hello David,
>>
>> I've tried with OpenJDK 15 EA Build 20. Unfortunately it's slow as 
>> with OpenJDK14.
>> A colleague of mine set CPU affinity with taskset to only one CPU on 
>> Linux and did not experience the slowdown. So the issue seems 
>> Windows-related.
> I wouldn?t rule out this being a scheduling issue. With thread-local 
> handshakes some operations like the ones you see in your logs 
> (deoptimization, biasedlocking revocation) do not need all JavaThreads 
> to be stopped at the same time in order to be executed. If you only 
> have one CPU but many JavaThreads running, depending on the scheduler, 
> maybe it could happen that the JavaThread you need to handshake is 
> constantly being left behind in the scheduler queue and so the 
> operation takes longer to execute. When you add 
> -XX:-ThreadLocalHandshakes, those operations will be executed using 
> safepoints, which means all JavaThreads are stopped. So even if the 
> scheduler decides to give priority to other JavaThreads those will 
> block and free the CPU. Also when adding more CPU that will increase 
> the likelihood of a JavaThread getting scheduled and might explain why 
> you see it gets fixed.
>
> Alternatively the JavaThread that needs to be handshaked is being 
> scheduled but is not polling for the handshake. But I think in that 
> the case you should still have the same issue with 
> -XX:-ThreadLocalHandshakes because the polling mechanism is the same, 
> although it's true that the misbehaving JavaThread will have more CPU 
> time to complete whatever is doing while others are stopped when 
> safepointing.
>
> One thing that makes me question the scheduler theory is the fact that 
> you see such improvements when only disabling biased-locking in 13.0.2 
> because that was still using safepoints back then. Can you check the 
> logs to see in which operations you see the pauses when running with 
> default options compared to -XX:-UseBiasedLocking?
>
> I?m also surprised you see those long logging pauses because all the 
> other JavaThreads were already processed in the handshake so something 
> must be executing. Maybe you can switch the logging to trace instead 
> of debug. Also maybe you can add logging at Java code so you can 
> follow what is actually running? You could send it to the same app.txt 
> and flush every time you write to see it ordered in the log. You can 
> also get the uptime you see in the UL logs with 
> ManagementFactory.getRuntimeMXBean().getUptime(). That logging will 
> affect timing but might give some insights.
>
> Thanks,
> Patricio
>> Miklos
>>
>> On 4/28/20 1:02 AM, David Holmes wrote:
>>> Hi Miklos,
>>>
>>> On 28/04/2020 7:26 am, Karak? Mikl?s wrote:
>>>> Hello,
>>>>
>>>> We bumped into a possible Thread-Local Handshakes issue with 
>>>> multiple apps. It seems that enabled TLH slows down applications 
>>>> running on (although rare) single CPU boxes. I would be grateful if 
>>>> you could confirm that this is a known trade-off for these rare 
>>>> setups or a possible JVM bug. That would save us at least a few 
>>>> hours of debugging.
>>>>
>>>> TL;DR: Both tested apps stop frequently around 
>>>> "HandshakeOneThread", "HandshakeAllThreads" and "Revoked bias of 
>>>> currently-unlocked object" running with 
>>>> OpenJDK12/OpenJDK13/OpenJDK14. OpenJDK 13 with 
>>>> -XX:-ThreadLocalHandshakes fixes the issue. Adding a second CPU to 
>>>> the virtual machine fixes the issue. Enabling hyper-threading fixes 
>>>> the issue.
>>>
>>> Can you try with latest JDK 15 build, just to see if this may be 
>>> something already addressed?
>>>
>>> Thanks,
>>> David
>>>
>>>> More details are available at StackOverflow: 
>>>> https://stackoverflow.com/questions/61375565/slow-application-frequent-jvm-hangs-with-single-cpu-setups-and-java-12 
>>>>
>>>>
>>>> All thoughts are welcome.
>>>>
>>>> Best,
>>>> Miklos
>>>>
>>
>


From leonid.mesnik at oracle.com  Wed Apr 29 03:44:31 2020
From: leonid.mesnik at oracle.com (Leonid Mesnik)
Date: Tue, 28 Apr 2020 20:44:31 -0700
Subject: RFR(S) : 8242314 : use reproducible random in vmTestbase shared
 code
In-Reply-To: <CCA27FF4-9C5D-486F-A31E-180C58483310@oracle.com>
References: <A59F9A91-D3D4-4D98-BD02-C44E1C3790E4@oracle.com>
 <4C16AD88-51F8-43F5-BAC1-16083E4A3C71@oracle.com>
 <4C10A003-6286-4272-AB7E-FC20659A6226@oracle.com>
 <FFE3D45D-5702-403C-995F-B09748A006EA@oracle.com>
 <5FE778F4-FCCE-44E5-B856-48B374545D98@oracle.com>
 <CCA27FF4-9C5D-486F-A31E-180C58483310@oracle.com>
Message-ID: <635b9c25-856a-2cca-be7d-1de3f780a14f@oracle.com>

Looks good. (Assuming there is 
https://bugs.openjdk.java.net/browse/JDK-8243927 for next clean up.)

Leonid

On 4/27/20 9:15 PM, Igor Ignatyev wrote:
>
>> On Apr 25, 2020, at 9:22 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
>>
>>
>>
>>> On Apr 25, 2020, at 8:45 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
>>>
>>>> On Apr 25, 2020, at 9:01 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
>>>>
>>>>
>>>>> On Apr 25, 2020, at 4:57 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
>>>>>
>>>>>> On Apr 22, 2020, at 2:03 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~iignatyev/8242314/webrev.00/
>>>>>>> 122 lines changed: 71 ins; 19 del; 32 mod;
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> could you please review this small patch which updates shared vmTestbase  test code to use j.t.l.Utils.getRandomInstance() as a random number generator or as a source of seed values? this patch lays the ground work for updates of :vmTestbase_* test groups which are to be done by remaining subtasks of 8241623[1], such split make it possible to work on/review/test testgroup-specific changes in concurrently.
>>>>>>
>>>>>> unfortunately, such extra dependency on /test/lib classes broke the shaky balance and caused some tests to fail due to "split testlibrary" problem (see 8188828[2] for the definition), the solution to that is to remove all dependencies on /test/lib classes from ExecDriver class, so the patch also includes "inlining" of several jdk.test.lib.Utils and Platform methods/fields into ExecDriver.
>>>>>>
>>>>>> webrev: http://cr.openjdk.java.net/~iignatyev/8242314/webrev.00/
>>>>>> testing: all :vmTestbase_* test groups on {linux,windows,macosx}-x64 and solaris-sparcv9
>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8242314
>>>>>>
>>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8241623
>>>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8188828
>>>>>>
>>>>>> Thanks,
>>>>>> -- Igor
>>>>> The test changes look fine.
>>>>>
>>>>> But I don't feel like I understand the "split library" problem well
>>>>> enough to review the ExecDriver changes.  I will say that I don't like
>>>>> the solution of copying code, and hope there is a better way.  Is
>>>>> there a bug for the problem?  It seems like it could crop up again,
>>>>> perhaps repeatedly, in which case the code copying becomes increasingly
>>>>> bad.  And it might be a good idea to leave some breadcrumbs associated
>>>>> with this code copying workaround, so an eventual better fix could also
>>>>> cleanup the duplication.
>>>>>
>>>> Hi Kim,
>>>>
>>>> thanks for look at this!
>>>>
>>>> "split library" problem occur when a test "have these properties:
>>>> - has more than one @library
>>>> - depends on class Foo from library A which depends on class Bar from another library B
>>>> - has an implicit/explicit build for class Baz from libraries which depends on class Bar"[1]
>>>> in this case A=/vmTestbase, B=/test/lib, Foo=ExecDriver, Bar=jdk.test.lib.Utils, and Baz can be pretty much any of class from jdk.test.lib, e.g. jdk.test.lib.process.ProcessTools.
>>> Yes, I read that already, from JDK-8188828.  But I don't understand
>>> the problem; remember, I'm not really a Java programmer, I just fake
>>> it sometimes :)  In particular, I don't have a good understanding of
>>> Java packaging and modularization and such.  So I'd rather not be
>>> counted as having reviewed that part of the change.
>> well, a usual Java programmer will have a little understanding of the problem either since it's kinda unique to jtreg, or rather to how we use it. let me try to elaborate more on that. @library tag tells jtreg where to look for source code during compilation, later when jtreg invokes javac to compile java source code to java bytecode it puts the result depending on the location of the class which provoked compilation, that's to say if you have '@build Bar' (or `@run main Bar`), jtreg will search for Bar in current test source directory and in all @libraries, if Bar is found in test source, jtreg will use test-specific directory as destination directory; if Bar is found in a @library, jtreg will use the directory specific for this library, and this directory is shared b/w different tests. to find which particular dependency of Bar to build, jtreg relies on javac's ability to implicitly compile dependencies, and the bytecode of all dependencies will be placed in the same directory regardless of the origin of dependencies source code. it all works great, unless you have a situation described above, as in such cases, you might end up w/ some part of @library be compiled into one directory, and some part in a directory of another @library -- a so-called "split library" problem, so the next test which uses 1st @library but doesn't use the 2nd will fail to find class files of some classes from 1st @library. I hope it makes this at least a little bit clearer. there is also a very good analyze of this problem from Ioi in CODETOOLS-7901986 -- https://bugs.openjdk.java.net/browse/CODETOOLS-7901986?focusedCommentId=14108320&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14108320 <https://bugs.openjdk.java.net/browse/CODETOOLS-7901986?focusedCommentId=14108320&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14108320> <https://bugs.openjdk.java.net/browse/CODETOOLS-7901986?focusedCommentId=14108320&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14108320 <https://bugs.openjdk.java.net/browse/CODETOOLS-7901986?focusedCommentId=14108320&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14108320>>
>>
>>>> there is a better way to fix that, which we discussed have discussed in the aftermath of our encounter of this problem (when 8188828 was used as a stopgap solution), and this way is to redesign/simplify and potentially modularize test libraries so it will be easier/saner to build them. unfortunately, we are not there yet, yet we are making progress; this activity is tracked by 8211289[2], 8211290[3], and 8211358[4].
>>>>
>>>> as of breadcrumbs, I think I left mentions of the copied methods in all the methods, e.g. L#144; and the only copied pieces of code which don't have such breadcrumbs are TEST_CLASS_PATH and TEST_JDK static fields. as they are quite straightforward, I didn't feel that it's need to refer back to the original code. but if you like, I can add comments there as well.
>>> Regarding the breadcrumb comments, I didn't recognize them as such.
>>> Had they been something more than just a fully qualified method name,
>>> i.e. something like "Copied from <fqname>", it would have been clear.
>>> Is there a cleanup CR for later?
>> sure I'll update comments and file an RFE to clean this up.
> filed https://bugs.openjdk.java.net/browse/JDK-8243927 <https://bugs.openjdk.java.net/browse/JDK-8243927>; and revamped the comments:
>   - full webrev: http://cr.openjdk.java.net/~iignatyev//8242314/webrev.01
>   - diff b/w 00 and 01: http://cr.openjdk.java.net/~iignatyev//8242314/webrev.0-1 <http://cr.openjdk.java.net/~iignatyev//8242314/webrev.0-1>
>
> -- Igor
>>>> Thanks,
>>>> -- Igor
>>>>
>>>>
>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8188828?focusedCommentId=14121228&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14121228 <https://bugs.openjdk.java.net/browse/JDK-8188828?focusedCommentId=14121228&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14121228> <https://bugs.openjdk.java.net/browse/JDK-8188828?focusedCommentId=14121228&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14121228 <https://bugs.openjdk.java.net/browse/JDK-8188828?focusedCommentId=14121228&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14121228>>
>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8211289 <https://bugs.openjdk.java.net/browse/JDK-8211289><https://bugs.openjdk.java.net/browse/JDK-8211289 <https://bugs.openjdk.java.net/browse/JDK-8211289>>
>>>> [3] https://bugs.openjdk.java.net/browse/JDK-8211290 <https://bugs.openjdk.java.net/browse/JDK-8211290><https://bugs.openjdk.java.net/browse/JDK-8211290 <https://bugs.openjdk.java.net/browse/JDK-8211290>>
>>>> [4] https://bugs.openjdk.java.net/browse/JDK-8211358 <https://bugs.openjdk.java.net/browse/JDK-8211358><https://bugs.openjdk.java.net/browse/JDK-8211358 <https://bugs.openjdk.java.net/browse/JDK-8211358>>


From igor.ignatyev at oracle.com  Wed Apr 29 03:50:12 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Tue, 28 Apr 2020 20:50:12 -0700
Subject: RFR(S) : 8242314 : use reproducible random in vmTestbase shared
 code
In-Reply-To: <635b9c25-856a-2cca-be7d-1de3f780a14f@oracle.com>
References: <A59F9A91-D3D4-4D98-BD02-C44E1C3790E4@oracle.com>
 <4C16AD88-51F8-43F5-BAC1-16083E4A3C71@oracle.com>
 <4C10A003-6286-4272-AB7E-FC20659A6226@oracle.com>
 <FFE3D45D-5702-403C-995F-B09748A006EA@oracle.com>
 <5FE778F4-FCCE-44E5-B856-48B374545D98@oracle.com>
 <CCA27FF4-9C5D-486F-A31E-180C58483310@oracle.com>
 <635b9c25-856a-2cca-be7d-1de3f780a14f@oracle.com>
Message-ID: <05B33ACA-E470-4250-AB4C-AD83D56095E3@oracle.com>

thanks for your review Leonid. pushed

-- Igor

> On Apr 28, 2020, at 8:44 PM, Leonid Mesnik <leonid.mesnik at oracle.com> wrote:
> 
> Looks good. (Assuming there is https://bugs.openjdk.java.net/browse/JDK-8243927 <https://bugs.openjdk.java.net/browse/JDK-8243927> for next clean up.)
> 
> Leonid
> 
> On 4/27/20 9:15 PM, Igor Ignatyev wrote:
>> 
>>> On Apr 25, 2020, at 9:22 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
>>> 
>>> 
>>> 
>>>> On Apr 25, 2020, at 8:45 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
>>>> 
>>>>> On Apr 25, 2020, at 9:01 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
>>>>> 
>>>>> 
>>>>>> On Apr 25, 2020, at 4:57 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
>>>>>> 
>>>>>>> On Apr 22, 2020, at 2:03 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
>>>>>>> 
>>>>>>> http://cr.openjdk.java.net/~iignatyev/8242314/webrev.00/
>>>>>>>> 122 lines changed: 71 ins; 19 del; 32 mod;
>>>>>>> 
>>>>>>> Hi all,
>>>>>>> 
>>>>>>> could you please review this small patch which updates shared vmTestbase  test code to use j.t.l.Utils.getRandomInstance() as a random number generator or as a source of seed values? this patch lays the ground work for updates of :vmTestbase_* test groups which are to be done by remaining subtasks of 8241623[1], such split make it possible to work on/review/test testgroup-specific changes in concurrently.
>>>>>>> 
>>>>>>> unfortunately, such extra dependency on /test/lib classes broke the shaky balance and caused some tests to fail due to "split testlibrary" problem (see 8188828[2] for the definition), the solution to that is to remove all dependencies on /test/lib classes from ExecDriver class, so the patch also includes "inlining" of several jdk.test.lib.Utils and Platform methods/fields into ExecDriver.
>>>>>>> 
>>>>>>> webrev: http://cr.openjdk.java.net/~iignatyev/8242314/webrev.00/
>>>>>>> testing: all :vmTestbase_* test groups on {linux,windows,macosx}-x64 and solaris-sparcv9
>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8242314
>>>>>>> 
>>>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8241623
>>>>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8188828
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> -- Igor
>>>>>> The test changes look fine.
>>>>>> 
>>>>>> But I don't feel like I understand the "split library" problem well
>>>>>> enough to review the ExecDriver changes.  I will say that I don't like
>>>>>> the solution of copying code, and hope there is a better way.  Is
>>>>>> there a bug for the problem?  It seems like it could crop up again,
>>>>>> perhaps repeatedly, in which case the code copying becomes increasingly
>>>>>> bad.  And it might be a good idea to leave some breadcrumbs associated
>>>>>> with this code copying workaround, so an eventual better fix could also
>>>>>> cleanup the duplication.
>>>>>> 
>>>>> Hi Kim,
>>>>> 
>>>>> thanks for look at this!
>>>>> 
>>>>> "split library" problem occur when a test "have these properties:
>>>>> - has more than one @library
>>>>> - depends on class Foo from library A which depends on class Bar from another library B
>>>>> - has an implicit/explicit build for class Baz from libraries which depends on class Bar"[1]
>>>>> in this case A=/vmTestbase, B=/test/lib, Foo=ExecDriver, Bar=jdk.test.lib.Utils, and Baz can be pretty much any of class from jdk.test.lib, e.g. jdk.test.lib.process.ProcessTools.
>>>> Yes, I read that already, from JDK-8188828.  But I don't understand
>>>> the problem; remember, I'm not really a Java programmer, I just fake
>>>> it sometimes :)  In particular, I don't have a good understanding of
>>>> Java packaging and modularization and such.  So I'd rather not be
>>>> counted as having reviewed that part of the change.
>>> well, a usual Java programmer will have a little understanding of the problem either since it's kinda unique to jtreg, or rather to how we use it. let me try to elaborate more on that. @library tag tells jtreg where to look for source code during compilation, later when jtreg invokes javac to compile java source code to java bytecode it puts the result depending on the location of the class which provoked compilation, that's to say if you have '@build Bar' (or `@run main Bar`), jtreg will search for Bar in current test source directory and in all @libraries, if Bar is found in test source, jtreg will use test-specific directory as destination directory; if Bar is found in a @library, jtreg will use the directory specific for this library, and this directory is shared b/w different tests. to find which particular dependency of Bar to build, jtreg relies on javac's ability to implicitly compile dependencies, and the bytecode of all dependencies will be placed in the same directory regardless of the origin of dependencies source code. it all works great, unless you have a situation described above, as in such cases, you might end up w/ some part of @library be compiled into one directory, and some part in a directory of another @library -- a so-called "split library" problem, so the next test which uses 1st @library but doesn't use the 2nd will fail to find class files of some classes from 1st @library. I hope it makes this at least a little bit clearer. there is also a very good analyze of this problem from Ioi in CODETOOLS-7901986 -- https://bugs.openjdk.java.net/browse/CODETOOLS-7901986?focusedCommentId=14108320&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14108320 <https://bugs.openjdk.java.net/browse/CODETOOLS-7901986?focusedCommentId=14108320&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14108320><https://bugs.openjdk.java.net/browse/CODETOOLS-7901986?focusedCommentId=14108320&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14108320 <https://bugs.openjdk.java.net/browse/CODETOOLS-7901986?focusedCommentId=14108320&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14108320>> <https://bugs.openjdk.java.net/browse/CODETOOLS-7901986?focusedCommentId=14108320&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14108320 <https://bugs.openjdk.java.net/browse/CODETOOLS-7901986?focusedCommentId=14108320&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14108320><https://bugs.openjdk.java.net/browse/CODETOOLS-7901986?focusedCommentId=14108320&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14108320 <https://bugs.openjdk.java.net/browse/CODETOOLS-7901986?focusedCommentId=14108320&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14108320>>>
>>> 
>>>>> there is a better way to fix that, which we discussed have discussed in the aftermath of our encounter of this problem (when 8188828 was used as a stopgap solution), and this way is to redesign/simplify and potentially modularize test libraries so it will be easier/saner to build them. unfortunately, we are not there yet, yet we are making progress; this activity is tracked by 8211289[2], 8211290[3], and 8211358[4].
>>>>> 
>>>>> as of breadcrumbs, I think I left mentions of the copied methods in all the methods, e.g. L#144; and the only copied pieces of code which don't have such breadcrumbs are TEST_CLASS_PATH and TEST_JDK static fields. as they are quite straightforward, I didn't feel that it's need to refer back to the original code. but if you like, I can add comments there as well.
>>>> Regarding the breadcrumb comments, I didn't recognize them as such.
>>>> Had they been something more than just a fully qualified method name,
>>>> i.e. something like "Copied from <fqname>", it would have been clear.
>>>> Is there a cleanup CR for later?
>>> sure I'll update comments and file an RFE to clean this up.
>> filed https://bugs.openjdk.java.net/browse/JDK-8243927 <https://bugs.openjdk.java.net/browse/JDK-8243927><https://bugs.openjdk.java.net/browse/JDK-8243927 <https://bugs.openjdk.java.net/browse/JDK-8243927>>; and revamped the comments:
>>  - full webrev: http://cr.openjdk.java.net/~iignatyev//8242314/webrev.01 <http://cr.openjdk.java.net/~iignatyev//8242314/webrev.01>
>>  - diff b/w 00 and 01: http://cr.openjdk.java.net/~iignatyev//8242314/webrev.0-1 <http://cr.openjdk.java.net/~iignatyev//8242314/webrev.0-1><http://cr.openjdk.java.net/~iignatyev//8242314/webrev.0-1 <http://cr.openjdk.java.net/~iignatyev//8242314/webrev.0-1>>
>> 
>> -- Igor
>>>>> Thanks,
>>>>> -- Igor
>>>>> 
>>>>> 
>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8188828?focusedCommentId=14121228&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14121228 <https://bugs.openjdk.java.net/browse/JDK-8188828?focusedCommentId=14121228&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14121228><https://bugs.openjdk.java.net/browse/JDK-8188828?focusedCommentId=14121228&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14121228 <https://bugs.openjdk.java.net/browse/JDK-8188828?focusedCommentId=14121228&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14121228>> <https://bugs.openjdk.java.net/browse/JDK-8188828?focusedCommentId=14121228&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14121228 <https://bugs.openjdk.java.net/browse/JDK-8188828?focusedCommentId=14121228&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14121228><https://bugs.openjdk.java.net/browse/JDK-8188828?focusedCommentId=14121228&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14121228 <https://bugs.openjdk.java.net/browse/JDK-8188828?focusedCommentId=14121228&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14121228>>>
>>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8211289 <https://bugs.openjdk.java.net/browse/JDK-8211289><https://bugs.openjdk.java.net/browse/JDK-8211289 <https://bugs.openjdk.java.net/browse/JDK-8211289>><https://bugs.openjdk.java.net/browse/JDK-8211289 <https://bugs.openjdk.java.net/browse/JDK-8211289><https://bugs.openjdk.java.net/browse/JDK-8211289 <https://bugs.openjdk.java.net/browse/JDK-8211289>>>
>>>>> [3] https://bugs.openjdk.java.net/browse/JDK-8211290 <https://bugs.openjdk.java.net/browse/JDK-8211290><https://bugs.openjdk.java.net/browse/JDK-8211290 <https://bugs.openjdk.java.net/browse/JDK-8211290>><https://bugs.openjdk.java.net/browse/JDK-8211290 <https://bugs.openjdk.java.net/browse/JDK-8211290><https://bugs.openjdk.java.net/browse/JDK-8211290 <https://bugs.openjdk.java.net/browse/JDK-8211290>>>
>>>>> [4] https://bugs.openjdk.java.net/browse/JDK-8211358 <https://bugs.openjdk.java.net/browse/JDK-8211358><https://bugs.openjdk.java.net/browse/JDK-8211358 <https://bugs.openjdk.java.net/browse/JDK-8211358>><https://bugs.openjdk.java.net/browse/JDK-8211358 <https://bugs.openjdk.java.net/browse/JDK-8211358><https://bugs.openjdk.java.net/browse/JDK-8211358 <https://bugs.openjdk.java.net/browse/JDK-8211358>>>


From david.holmes at oracle.com  Wed Apr 29 05:12:46 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 29 Apr 2020 15:12:46 +1000
Subject: RFR: 8244010: Simplify usages of
 ProcessTools.createJavaProcessBuilder in our tests
In-Reply-To: <056ff9b1-1a4f-a1fb-2727-b503707f3750@oracle.com>
References: <60c0e0e4-634b-d349-c6f3-ca1ad52926cb@oracle.com>
 <056ff9b1-1a4f-a1fb-2727-b503707f3750@oracle.com>
Message-ID: <b91260e2-52d7-3c5a-dc39-9d7333ad56bf@oracle.com>

Hi Stefan,

On 28/04/2020 11:54 pm, Stefan Karlsson wrote:
> Hi again,
> 
> I realized that we probably want to give ProcessTools.executeTestJvm the 
> same treatment.
> 
> Side-note: It's very awkard that createJavaProcessBuilder defaults to 
> not adding user-specifed flags, but executeTestJvm does. I think it 
> would be good to unify this as a separate RFE. I think *a lot* of 
> callers to createJavaProcessBuilder could be simplified by either using 
> executeTestJvm directly, or a simplified version of that.

createJavaProcessBuilder is the primitive and over time we have added 
higher-level APIs to abstract away the boiler-plate that deal with the 
externally passed flags, gives you an OutputAnalyzer etc.

The underlying java.lang.ProcessBuilder supports both List<String> and 
String[] for passing arguments, but returns the command as List<String>, 
so this has also affected the ProcessTools API. As Leonid suggests 
adding in a List<String> overload suffices rather than Collection<String>.

> I'm running testing through mach5 and found a few things to fix, I might 
> find more when the testing has proceeded further.
> 
> This is the current patch:
> https://cr.openjdk.java.net/~stefank/8244010/webrev.02.delta
> https://cr.openjdk.java.net/~stefank/8244010/webrev.02

One thing I noticed, as you copied it, is that this comment to all the 
createJavaProcessBuilder methods seems wrong:

271      * with any platform specific arguments prepended

as we only add the other arguments if we pass "true" for 
addTestVmAndJavaOptions. ??

Also regarding the String[]/List conversions ... as ProcessBuilder 
supports both I really don't see why we are doing:

  334         return new ProcessBuilder(args.toArray(new 
String[args.size()]));

and the "reporting" section could just iterate the args rather than 
creating a StringBuilder.

Cheers,
David

> Thanks,
> StefanK
> 
> On 2020-04-28 13:58, Stefan Karlsson wrote:
>> Hi all,
>>
>> Please review this patch to simplify usages of 
>> ProcessTools.createJavaProcessBuilder in our tests.
>>
>> https://cr.openjdk.java.net/~stefank/8244010/webrev.01/
>> https://bugs.openjdk.java.net/browse/JDK-8244010
>>
>> I saw all this code when reviewing changes to how we pass flags in our 
>> tests. There are a many places where arguments are converted and 
>> passed back and forth in String[] and Collections.
>>
>> For example:
>> ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(
>> ??????????argsList.toArray(new String[argsList.size()]));
>>
>> If we add an overload the createJavaProcessBuilder, that takes a 
>> Collection<String> as an argument, then we can write the code above as:
>> ???ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(argsList);
>>
>> Other places temporarily put the flags in a String[], where most calls 
>> simply lists the arguments in the call:
>> ??String[] opts = {Xmx, "-XX:NativeMemoryTracking=detail", 
>> "-XX:+UseParallelGC", "-version"};
>> ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(opts);
>>
>> And some places put the args in a temporary Collection:
>> ??LinkedList<String> vmOptions = new LinkedList<>();
>> ??vmOptions.add(gc);
>> ??vmOptions.add("-Xmx" + minMaxHeap);
>> ??vmOptions.add("-XX:+PrintFlagsFinal");
>> ??vmOptions.add(VerifyHeapSize.class.getName());
>>
>> ??ProcessBuilder pb = 
>> ProcessTools.createJavaProcessBuilder(vmOptions.toArray(new String[0]));
>>
>> I'd like to cleanup, simplify, and unify many of these usages.
>>
>> I've tested this by running all the changed tests locally.
>>
>> Thanks,
>> StefanK
> 

From ioi.lam at oracle.com  Wed Apr 29 05:14:31 2020
From: ioi.lam at oracle.com (Ioi Lam)
Date: Tue, 28 Apr 2020 22:14:31 -0700
Subject: RFR(M): 8243392: Remodel CDS/Metaspace storage reservation
In-Reply-To: <CAA-vtUyVki-DHepyAzORoTK_MwnrezFUB+QsxbWszztsx1x7NA@mail.gmail.com>
References: <CAA-vtUyVki-DHepyAzORoTK_MwnrezFUB+QsxbWszztsx1x7NA@mail.gmail.com>
Message-ID: <6802c3af-77a5-8a91-b94c-d590dd41765f@oracle.com>

Hi Thomas,

There are a lot of changes so it will take me a while to go through 
everything. Just some initial comments:

 ? // User may have specified an invalid base address. Should we ignore 
it or assert?
guarantee(CompressedKlassPointers::is_valid_base((address)shared_base),
 ??????????? "SharedBaseAddress: " PTR_FORMAT " is not a valid base.", 
p2i(shared_base));

This will cause the VM to crash. I think it's better (1) exit the VM 
properly with an error code, or (2) override the user's input.

======

Since this is a potentially disruptive change, I want to run it in our 
CI as well. Could you tell me the tip of your repo?

========

For testing the CDS relocation code, I would suggest running:

cd test/hotspot/jtreg
jtreg -javaoption:-XX:+UnlockDiagnosticVMOptions \
 ????? -javaoption:-XX:ArchiveRelocationMode=1 \
 ????? -javaoption:-XX:NativeMemoryTracking=detail
 ????? :hotspot_cds_relocation

This will place the CCS at random locations picked by the OS.

========

metaspace.cpp:

If your intention is to "shake things up a little", it's not a good idea 
to include it in a complex change set. If things indeed go wrong, we 
don't know who caused it (your CCS changes, or old bugs triggered by 
this debug code), and we will end up backing out the entire changeset.

I would suggest putting this in a different RFE, and even push it now.

 ? // The upcoming Elastic Metaspace will have stricter alignment 
requirements.
 ? // For debug builds, increase reserve alignment to shake loose errors 
resulting
 ? // from misusing this alignment.
 ? // Note: do not increase too much (e.g. not on platforms with 64K 
pages), we do not
 ? // want to disturb tests requiring precise numbers for metaspace size 
or ccs size.
#ifdef ASSERT
 ? if (_reserve_alignment == 4 * K) {
 ??? _reserve_alignment *= 4;
 ? }
#endif


More to come ....

Thanks
- Ioi


On 4/28/20 7:54 AM, Thomas St?fe wrote:
> Hi all,
>
> Could I have reviews for the following proposal of reworking cds/class 
> space reservation?
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8243392
>
> Webrev: 
> http://cr.openjdk.java.net/~stuefe/webrevs/rework-cds-ccs-reservation/webrev.00/webrev/
>
> (Many thanks to Ioi Lam for so patiently explaining CDS internals to 
> me, and to Andrew Haley and Nick Gasson for help with aarch64!)
>
> Reservation of the compressed class space is needlessly complicated 
> and has some minor issues. It can be simplified and made clearer.
>
> The complexity stems from the fact that this area lives at the 
> intersection of two to three sub systems, depending on how one counts. 
> Metaspace, CDS, and the platform which may or may not its own view of 
> how to reserve class space. And this code has been growing organically 
> over time.
>
> One small example:
>
> ReservedSpace Metaspace::reserve_preferred_space(size_t size, size_t 
> alignment,
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?bool large_pages, 
> char *requested_addr,
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?bool use_requested_addr)
>
> which I spent hours decoding, resulting in a very confused mail to 
> hs-runtime and aarch64-port-dev [2].
>
> This patch attempts to simplify cds and metaspace setup a bit; to 
> comment implicit knowledge which is not immediately clear; to cleanly 
> abstract platform concerns like optimized class space placement; and 
> to disentangle cds from metaspace to solve issues which may bite us 
> later with Elastic Metaspace [4].
>
> ---
>
> The main change is the reworked reservation mechanism. This is based 
> on Ioi's proposal [5].
>
> When reserving class space, three things must happen:
>
> 1) reservation of the space obviously. If cds is active that space 
> must be in the vicinity of cds archives to be covered by compressed 
> class pointer encoding.
> 2) setting up the internal Metaspace structures atop of that space
> 3) setting up compressed class pointer encoding.
>
> In its current form, Metaspace may or may not do some or all of that 
> in one function 
> (Metaspace::allocate_metaspace_compressed_klass_ptrs(ReservedSpace 
> metaspace_rs, char* requested_addr, address cds_base);) - if cds is 
> active, it will reserve the space for Metaspace and hand it in, 
> otherwise it will create it itself.
>
> When discussing this in [2], Ioi proposed to move the reservation of 
> the class space completely out of Metaspace and make it a 
> responsibility of the caller always. This would reduce some 
> complexity, and this patch follows the proposal.
>
> I removed 
> Metaspace::allocate_metaspace_compressed_klass_ptrs(ReservedSpace 
> metaspace_rs, char* requested_addr, address cds_base); and all its sub 
> functions.
>
> (1) now has to always be done outside - a ReservedSpace for class 
> space has to be provided by the caller. However, Metaspace now offers 
> a utility function for reserving space at a "nice" location, and 
> explicitly doing nothing else:
>
> ReservedSpace 
> Metaspace::reserve_address_space_for_compressed_classes(size_t size);
>
> this function can be redefined on a platform level for platform 
> optimized reservation, see below for details.
>
> (2) is taken care of by a new function, 
> Metaspace::initialize_class_space(ReservedSpace rs)
>
> (3) is taken care of a new function 
> CompressedKlassPointers::initialize(), see below for details.
>
>
> So, class space now is set up three explicit steps:
>
> - First, reserve a suitable space by however means you want. For 
> convenience you may use 
> Metaspace::reserve_address_space_for_compressed_classes(), or you may 
> roll your own reservation.
> - Next, tell Metaspace to use that range as backing storage for class 
> space: Metaspace::initialize_class_space(ReservedSpace rs)
> - Finally, set up encoding. Encoding is independent from the concept 
> of a ReservedSpace, it just gets an address range, see below for details.
>
> Separating these steps and moving them out of the responsibility of 
> Metaspace makes this whole thing more flexible; it also removes 
> unnecessary knowledge (e.g. Metaspace does not need to know anything 
> about either ccp encoding or cds).
>
> ---
>
> How it comes together:
>
> If CDS is off, we just reserve a space using 
> Metaspace::reserve_address_space_for_compressed_classes(), initialize 
> it with Metaspace::initialize_class_space(ReservedSpace rs), then set 
> up compressed class pointer encoding covering the range of this class 
> space.
>
> If CDS is on (dump time), we reserve large 4G space, either at 
> SharedBaseAddress or using 
> Metaspace::reserve_address_space_for_compressed_classes(); we then 
> split that into 3G archive space and 1G class space; we set up that 
> space with Metaspace as class space; then we set up?compressed class 
> pointer encoding covering both archive space and cds.
>
> If CDS is on (run time), we reserve a large space, split it into 
> archive space (large enough to hold both archives) and class space, 
> then basically proceed as above.
>
> Note that this is almost exactly how things worked before (modulo some 
> minor fixes, e.g. alignment issues), only the code is reformed and 
> made more explicit.
>
> ---
>
> I moved compressed class pointer setup over to CompressedKlassPointers 
> and changed the interface:
>
> -void Metaspace::set_narrow_klass_base_and_shift(ReservedSpace 
> metaspace_rs, address cds_base)
> +void CompressedKlassPointers::initialize(address addr, size_t len);
>
> Instead of feeding it a single ReservedSpace, which is supposed to 
> represent class space, and an optional alternate base if cds is on, 
> now we give it just an numeric address range. That range marks the 
> limits to where Klass structures are to be expected, and is the 
> implicit promise that outside that range no Klass structures will 
> exist, so encoding has to cover only this range.
>
> This range may contain just the class space; or class space+cds; or 
> whatever allocation scheme we come up with in the future. Encoding 
> does not really care how the memory is organized as long as the input 
> range covers all possible Klass locations. That way we remove 
> knowledge about class space/cds from compressed class pointer encoding.
>
> Moving it away from metaspace.cpp into the CompressedKlassPointers 
> class also mirrors CompressedOops::initialize().
>
> ---
>
> I renamed _narrow_klass_range to just _range, because strictly 
> speaking this is the range un-narrow Klass pointers can have.
>
> As for the implementation of 
> CompressedKlassPointers::initialize(address addr, size_t len), I 
> mimicked very closely what happened before, so there should be almost 
> no differences. Since "almost no differences" sounds scary :) here are 
> the differences:
>
> - When CDS is active (dump or run time) we now always, 
> unconditionally, set the encoding range to 4G. This fixes a 
> theoretical bug discussed on aarch64-port-dev [1].
>
> - When CDS is not active, we set the encoding range to the minimum 
> required length. Before, it was left at its default value of 4G.
>
> Both differences only affect aarch64, since they are currently the 
> only one using the range field in CompressedKlassPointers.
>
> I wanted to add an assert somewhere to test encoding of the very last 
> address of the CompressedKlassPointers range, again to prevent errors 
> like [3]. But I did not come up with a good place for this assert 
> which would cover also the encoding done by C1/C2.
>
> For the same reason I thought about introducing a mode where Klass 
> structures would be allocated in reverse order, starting at the end of 
> the ccs, but again left it out as too big a change.
>
> ---
>
> OS abstraction: platforms may have restrictions of what constitutes a 
> valid compressed class pointer encoding base. Or if not, they may have 
> at least preferences. There was logic like this in metaspace.cpp, 
> which I removed and cleanly factored out into platform dependent 
> files, giving each platform the option to add special logic.
>
> These are two new methods:
>
> - bool CompressedKlassPointers::is_valid_base(address p)
>
> to let the platform tell you whether it considers p to be a valid 
> encoding base. The only platform having these restrictions currently 
> is aarch64.
>
> - ReservedSpace 
> Metaspace::reserve_address_space_for_compressed_classes(size_t size);
>
> this hands over the process of allocating a range suitable for 
> compressed class pointer encoding to the platform. Most platforms will 
> allocate just anywhere, but some platforms may have a better strategy 
> (e.g. trying low memory first, trying only correctly aligned addresses 
> and so on).
>
> Beforehand, this coding existed in a similar form in metaspace.cpp for 
> aarch64 and AIX. For now, I left the AIX part out - it seems only half 
> done, and I want to check further if we even need it, if yes why not 
> on Linux ppc, and C1 does not seem to support anything other than 
> base+offset with shift either, but I may be mistaken.
>
> These two methods should give the platform enough control to implement 
> their own scheme for optimized class space placement without bothering 
> any shared code about it.
>
> Note about the form, I introduced two new platform dependent files, 
> "metaspace_<cpu>.cpp" and "compressedOops_<cpu>.cpp". I am not happy 
> about this but this seems to be what we generally do in hotspot, right?
>
> ---
>
> Metaspace reserve alignment vs cds alignment
>
> CDS was using Metaspace reserve alignment for CDS internal purposes. I 
> guess this was just a copy paste issue. It never caused problems since 
> Metaspace reserve alignment == page size, but that is not true anymore 
> in the upcoming Elastic Metaspace where reserve alignment will be 
> larger. This causes a number of issues.
>
> I separated those two cleanly. CDS now uses 
> os::vm_allocation_granularity. Metaspace::reserve_alignment is only 
> used in those two places where it is needed, when CDS creates the 
> address space for class space on behalf of the Metaspace.
>
> ---
>
> Windows special handling in CDS
>
> To simplify coding I removed the windows specific handling which left 
> out reservation of the archive. This was needed because windows cannot 
> mmap files into reserved regions. But fallback code exists in 
> filemap.cpp for this case which just reads in the region instead of 
> mapping?it.
>
> Should that turn out to be a performance problem, I will reinstate the 
> feature. But a simpler way would be reserve the archive and later just 
> before mmapping?the archive file to release the archive space. That 
> would not only be simpler but give us the best guarantee that that 
> address space is actually available. But I'd be happy to leave that 
> part out completely if we do not see any performance problems on 
> windows x64.
>
> ---
>
> NMT cannot deal with spaces which are split. This problem manifests in 
> that bookkeeping for class space is done under "Shared Classes", not 
> "Classes" as it should. This problem exists today too at dump?time and 
> randomly at run time. But since I simplified the reservation, this 
> problem now shows up always, whether or not we map at the 
> SharedBaseAddress.
> While I could work around this problem, I'd prefer this problem to be 
> solved at the core, and NMT to have an option to recognize reservation 
> splits. So I'd rather not put a workaround for this into the patch but 
> leave it for fixing as a separate issue. I opened this issue to track 
> it [6].
>
> ---
>
> Jtreg tests:
>
> I expanded the CompressedOops/CompressedClassPointers.java. I also 
> extended them to Windows. The tests now optionally omit strict class 
> space placement tests, since these tests heavily depend on ASLR and 
> were the reason they were excluded on Windows. However I think even 
> without checking for class space placement they make sense, just to 
> see that the VM comes up and lives with the many different settings we 
> can run in.
>
> ---
>
> Tests:
>
> - I ran the patch through Oracles submit repo
> - I ran tests manually for aarch64, zero, linux 32bit and windows x64
> - The whole battery of nightly tests at SAP, including ppc, ppcle and 
> aarch64, unfortunately excluding windows because of unrelated errors. 
> Windows x64 tests will be redone tonight.
>
>
> Thank you,
>
> Thomas
>
> [1] 
> https://mail.openjdk.java.net/pipermail/aarch64-port-dev/2020-April/008804.html
> [2] 
> https://mail.openjdk.java.net/pipermail/aarch64-port-dev/2020-April/008757.html
> [3] https://bugs.openjdk.java.net/browse/JDK-8193266
> [4] https://bugs.openjdk.java.net/browse/JDK-8221173
> [5] 
> https://mail.openjdk.java.net/pipermail/aarch64-port-dev/2020-April/008765.html
> [6] https://bugs.openjdk.java.net/browse/JDK-8243535
>


From david.holmes at oracle.com  Wed Apr 29 05:31:42 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 29 Apr 2020 15:31:42 +1000
Subject: Thread-Local Handshakes slowdown on single-cpu machines
In-Reply-To: <f23d9ba4-3380-050a-d772-8a8a8eeb5b15@palacsint.hu>
References: <5e65ba58-b599-e83c-b4ea-3d766ed3c457@palacsint.hu>
 <66059982-49bf-1a15-39fb-22008b069573@oracle.com>
 <7216aa59-ad6f-dd3c-4f37-4723d3b7e9a7@palacsint.hu>
 <857ffb5e-d9e5-3976-3968-124e3b47effb@oracle.com>
 <f23d9ba4-3380-050a-d772-8a8a8eeb5b15@palacsint.hu>
Message-ID: <46fd88cd-3e68-ebbf-869f-355ebb196146@oracle.com>

On 29/04/2020 1:26 pm, Karak? Mikl?s wrote:
> Hi Patricio,
> 
> Yes, it seems like a scheduling issue for us too. One of my colleagues 
> spotted the following sentence in Robbin Ehn's Handshaking HotSpot 
> presentation (https://youtu.be/VBCOfAJ409s?t=2222): ~"if the VM thread 
> finds that all threads are unsafe, it keeps spinning"
> Could it be that actually the VM thread gets more CPU time and actually 
> blocks JavaThread?

The "spinning" is intended to be done in a scheduler-friendly way via:

void os::naked_short_nanosleep(jlong ns) {
   assert(ns > -1 && ns < NANOUNITS, "Un-interruptable sleep, short time 
use only");

   int64_t start = os::javaTimeNanos();
   do {
     if (SwitchToThread() == 0) {
       // Nothing else is ready to run on this cpu, spin a little
       SpinPause();
     }
   } while (os::javaTimeNanos() - start < ns);
}

but this won't be that scheduler-friendly because it places the VMThread 
at the head of the priority queue, so it will cede up-to a timeslice to 
another thread, then run another iteration of the loop and hence 
alternate between a JavaThread and the VMThread until they are all 
blocked at the safepoint. What is really wanted is for the VMThread to 
go to the back of the scheduling queue.

David
-----

> I did not except that but I could reproduce the issue with a simple 
> hello world:
> 
> public class Main {
>  ??? public static void main(String[] args) {
>  ??? ??? System.out.println("Hello world");
>  ??? }
> }
> 
> A few measurements on Windows 10 x64 in VirtualBox:
> 
> 
> PS Z:\main> Measure-Command { .\main-1cpu.bat }
> 
> Minutes?????????? : 0
> Seconds?????????? : 7
> Milliseconds????? : 20
> Ticks???????????? : 70203455
> TotalMinutes????? : 0.117005758333333
> TotalSeconds????? : 7.0203455
> TotalMilliseconds : 7020.3455
> 
> 
> 
> PS Z:\main> Measure-Command { .\main-full.bat }
> 
> Minutes?????????? : 0
> Seconds?????????? : 1
> Milliseconds????? : 575
> Ticks???????????? : 15751352
> TotalSeconds????? : 1.5751352
> TotalMilliseconds : 1575.1352
> 
> 
> 
> PS Z:\main> Measure-Command { .\main-full.bat }
> 
> Minutes?????????? : 0
> Seconds?????????? : 1
> Milliseconds????? : 558
> Ticks???????????? : 15585384
> TotalSeconds????? : 1.5585384
> TotalMilliseconds : 1558.5384
> 
> 
> 
> PS Z:\main> Measure-Command { .\main-1cpu.bat }
> 
> Minutes?????????? : 0
> Seconds?????????? : 23
> Milliseconds????? : 648
> Ticks???????????? : 236482685
> TotalSeconds????? : 23.6482685
> TotalMilliseconds : 23648.2685
> 
> 
> main-1cpu.bat:
> 
> c:\windows\system32\cmd.exe /C start /affinity 1 \jdk-14\bin\java 
> -Xlog:all=trace:file=app-1cpu.txt:uptime,tid,level,tags:filecount=50 Main
> 
> main-full.bat:
> 
> c:\windows\system32\cmd.exe /C start /affinity FF \jdk-14\bin\java 
> -Xlog:all=trace:file=app-full.txt:uptime,tid,level,tags:filecount=50 Main
> 
> The files are also available here:
> https://palacsint.hu/etc/jdk/jdk14-single-cpu-slowdown-reprod-poc/
> Or in a zip:
> https://palacsint.hu/etc/jdk/jdk14-single-cpu-slowdown-reprod-poc.zip
> 
> You can find trace level logs of three testruns in the run1-2-3 
> directories as well.
> 
> Best,
> Miklos
> 
> 
> On 4/29/20 2:18 AM, Patricio Chilano wrote:
>> Hi Miklos,
>>
>> On 4/28/20 8:04 AM, Karak? Mikl?s wrote:
>>> Hello David,
>>>
>>> I've tried with OpenJDK 15 EA Build 20. Unfortunately it's slow as 
>>> with OpenJDK14.
>>> A colleague of mine set CPU affinity with taskset to only one CPU on 
>>> Linux and did not experience the slowdown. So the issue seems 
>>> Windows-related.
>> I wouldn?t rule out this being a scheduling issue. With thread-local 
>> handshakes some operations like the ones you see in your logs 
>> (deoptimization, biasedlocking revocation) do not need all JavaThreads 
>> to be stopped at the same time in order to be executed. If you only 
>> have one CPU but many JavaThreads running, depending on the scheduler, 
>> maybe it could happen that the JavaThread you need to handshake is 
>> constantly being left behind in the scheduler queue and so the 
>> operation takes longer to execute. When you add 
>> -XX:-ThreadLocalHandshakes, those operations will be executed using 
>> safepoints, which means all JavaThreads are stopped. So even if the 
>> scheduler decides to give priority to other JavaThreads those will 
>> block and free the CPU. Also when adding more CPU that will increase 
>> the likelihood of a JavaThread getting scheduled and might explain why 
>> you see it gets fixed.
>>
>> Alternatively the JavaThread that needs to be handshaked is being 
>> scheduled but is not polling for the handshake. But I think in that 
>> the case you should still have the same issue with 
>> -XX:-ThreadLocalHandshakes because the polling mechanism is the same, 
>> although it's true that the misbehaving JavaThread will have more CPU 
>> time to complete whatever is doing while others are stopped when 
>> safepointing.
>>
>> One thing that makes me question the scheduler theory is the fact that 
>> you see such improvements when only disabling biased-locking in 13.0.2 
>> because that was still using safepoints back then. Can you check the 
>> logs to see in which operations you see the pauses when running with 
>> default options compared to -XX:-UseBiasedLocking?
>>
>> I?m also surprised you see those long logging pauses because all the 
>> other JavaThreads were already processed in the handshake so something 
>> must be executing. Maybe you can switch the logging to trace instead 
>> of debug. Also maybe you can add logging at Java code so you can 
>> follow what is actually running? You could send it to the same app.txt 
>> and flush every time you write to see it ordered in the log. You can 
>> also get the uptime you see in the UL logs with 
>> ManagementFactory.getRuntimeMXBean().getUptime(). That logging will 
>> affect timing but might give some insights.
>>
>> Thanks,
>> Patricio
>>> Miklos
>>>
>>> On 4/28/20 1:02 AM, David Holmes wrote:
>>>> Hi Miklos,
>>>>
>>>> On 28/04/2020 7:26 am, Karak? Mikl?s wrote:
>>>>> Hello,
>>>>>
>>>>> We bumped into a possible Thread-Local Handshakes issue with 
>>>>> multiple apps. It seems that enabled TLH slows down applications 
>>>>> running on (although rare) single CPU boxes. I would be grateful if 
>>>>> you could confirm that this is a known trade-off for these rare 
>>>>> setups or a possible JVM bug. That would save us at least a few 
>>>>> hours of debugging.
>>>>>
>>>>> TL;DR: Both tested apps stop frequently around 
>>>>> "HandshakeOneThread", "HandshakeAllThreads" and "Revoked bias of 
>>>>> currently-unlocked object" running with 
>>>>> OpenJDK12/OpenJDK13/OpenJDK14. OpenJDK 13 with 
>>>>> -XX:-ThreadLocalHandshakes fixes the issue. Adding a second CPU to 
>>>>> the virtual machine fixes the issue. Enabling hyper-threading fixes 
>>>>> the issue.
>>>>
>>>> Can you try with latest JDK 15 build, just to see if this may be 
>>>> something already addressed?
>>>>
>>>> Thanks,
>>>> David
>>>>
>>>>> More details are available at StackOverflow: 
>>>>> https://stackoverflow.com/questions/61375565/slow-application-frequent-jvm-hangs-with-single-cpu-setups-and-java-12 
>>>>>
>>>>>
>>>>> All thoughts are welcome.
>>>>>
>>>>> Best,
>>>>> Miklos
>>>>>
>>>
>>
> 

From ioi.lam at oracle.com  Wed Apr 29 05:37:50 2020
From: ioi.lam at oracle.com (Ioi Lam)
Date: Tue, 28 Apr 2020 22:37:50 -0700
Subject: RFR: 8244010: Simplify usages of
 ProcessTools.createJavaProcessBuilder in our tests
In-Reply-To: <056ff9b1-1a4f-a1fb-2727-b503707f3750@oracle.com>
References: <60c0e0e4-634b-d349-c6f3-ca1ad52926cb@oracle.com>
 <056ff9b1-1a4f-a1fb-2727-b503707f3750@oracle.com>
Message-ID: <2b745d4b-d46e-c9eb-e48d-f3020e04e6e6@oracle.com>

Hi Stefan,

For a future RFE, I would suggesting adding a new function

 ??? ProcessTools.createJavaProcessBuilder(Object... args)

Every argument must be a String, a String[], a List<String>, or NULL.

You can simplified code like this:

 ?? static ProcessBuilder exec(String... args) throws Exception {
 ??????? List<String> argsList = new ArrayList<>();
 ??????? Collections.addAll(argsList, args);
 ??????? Collections.addAll(argsList, "-Xmn8m");
 ??????? if (cond) {
 ??????????? Collections.addAll(argsList, "-Dtest.classes=" + 
System.getProperty("test.classes","."));
 ??????? }
 ??????? Collections.addAll(argsList, ClassUnloadTestMain.class.getName());
 ??????? return ProcessTools.createJavaProcessBuilder(argsList);
 ??? }

to this:

 ??? static ProcessBuilder exec(String... args) throws Exception {
 ??????? return ProcessTools.createJavaProcessBuilder(
 ????????????????? args,
 ????????????????? "-Xmn8m",
(cond) ? "-Dtest.classes=" + System.getProperty("test.classes",".") : NULL,
ClassUnloadTestMain.class.getName());
 ??? }

We can probably allow higher-level dimensions like Object[][], as long 
as the eventual element type is a String.

What do you think?

- Ioi



On 4/28/20 6:54 AM, Stefan Karlsson wrote:
> Hi again,
>
> I realized that we probably want to give ProcessTools.executeTestJvm 
> the same treatment.
>
> Side-note: It's very awkard that createJavaProcessBuilder defaults to 
> not adding user-specifed flags, but executeTestJvm does. I think it 
> would be good to unify this as a separate RFE. I think *a lot* of 
> callers to createJavaProcessBuilder could be simplified by either 
> using executeTestJvm directly, or a simplified version of that.
>
> I'm running testing through mach5 and found a few things to fix, I 
> might find more when the testing has proceeded further.
>
> This is the current patch:
> https://cr.openjdk.java.net/~stefank/8244010/webrev.02.delta
> https://cr.openjdk.java.net/~stefank/8244010/webrev.02
>
> Thanks,
> StefanK
>
> On 2020-04-28 13:58, Stefan Karlsson wrote:
>> Hi all,
>>
>> Please review this patch to simplify usages of 
>> ProcessTools.createJavaProcessBuilder in our tests.
>>
>> https://cr.openjdk.java.net/~stefank/8244010/webrev.01/
>> https://bugs.openjdk.java.net/browse/JDK-8244010
>>
>> I saw all this code when reviewing changes to how we pass flags in 
>> our tests. There are a many places where arguments are converted and 
>> passed back and forth in String[] and Collections.
>>
>> For example:
>> ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(
>> ??????????argsList.toArray(new String[argsList.size()]));
>>
>> If we add an overload the createJavaProcessBuilder, that takes a 
>> Collection<String> as an argument, then we can write the code above as:
>> ???ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(argsList);
>>
>> Other places temporarily put the flags in a String[], where most 
>> calls simply lists the arguments in the call:
>> ??String[] opts = {Xmx, "-XX:NativeMemoryTracking=detail", 
>> "-XX:+UseParallelGC", "-version"};
>> ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(opts);
>>
>> And some places put the args in a temporary Collection:
>> ??LinkedList<String> vmOptions = new LinkedList<>();
>> ??vmOptions.add(gc);
>> ??vmOptions.add("-Xmx" + minMaxHeap);
>> ??vmOptions.add("-XX:+PrintFlagsFinal");
>> ??vmOptions.add(VerifyHeapSize.class.getName());
>>
>> ??ProcessBuilder pb = 
>> ProcessTools.createJavaProcessBuilder(vmOptions.toArray(new String[0]));
>>
>> I'd like to cleanup, simplify, and unify many of these usages.
>>
>> I've tested this by running all the changed tests locally.
>>
>> Thanks,
>> StefanK
>


From thomas.stuefe at gmail.com  Wed Apr 29 06:18:58 2020
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Wed, 29 Apr 2020 08:18:58 +0200
Subject: RFR(M): 8243392: Remodel CDS/Metaspace storage reservation
In-Reply-To: <6802c3af-77a5-8a91-b94c-d590dd41765f@oracle.com>
References: <CAA-vtUyVki-DHepyAzORoTK_MwnrezFUB+QsxbWszztsx1x7NA@mail.gmail.com>
 <6802c3af-77a5-8a91-b94c-d590dd41765f@oracle.com>
Message-ID: <CAA-vtUwvm_29aMpffCLrSaPp9TFL6ngXAaLQe5Atjyj4qd9DSg@mail.gmail.com>

Hi Ioi,

thanks for looking at this. Of course I'm happy if you run it through your
CI too.

The patch is based on

changeset:   59009:2b3b41fff837
tag:         qparent
user:        egahlin
date:        Mon Apr 27 15:01:22 2020 +0200
summary:     8242034: Remove JRE_HOME references


On Wed, Apr 29, 2020 at 7:14 AM Ioi Lam <ioi.lam at oracle.com> wrote:

> Hi Thomas,
>
> There are a lot of changes so it will take me a while to go through
> everything. Just some initial comments:
>
>    // User may have specified an invalid base address. Should we ignore
> it or assert?
> guarantee(CompressedKlassPointers::is_valid_base((address)shared_base),
>              "SharedBaseAddress: " PTR_FORMAT " is not a valid base.",
> p2i(shared_base));
>
> This will cause the VM to crash. I think it's better (1) exit the VM
> properly with an error code, or (2) override the user's input.
>
> ======
>
> Since this is a potentially disruptive change, I want to run it in our
> CI as well. Could you tell me the tip of your repo?
>
> ========
>
> For testing the CDS relocation code, I would suggest running:
>
> cd test/hotspot/jtreg
> jtreg -javaoption:-XX:+UnlockDiagnosticVMOptions \
>        -javaoption:-XX:ArchiveRelocationMode=1 \
>        -javaoption:-XX:NativeMemoryTracking=detail
>        :hotspot_cds_relocation
>
> This will place the CCS at random locations picked by the OS.
>
> ========
>
> metaspace.cpp:
>
> If your intention is to "shake things up a little", it's not a good idea
> to include it in a complex change set. If things indeed go wrong, we
> don't know who caused it (your CCS changes, or old bugs triggered by
> this debug code), and we will end up backing out the entire changeset.
>
> I would suggest putting this in a different RFE, and even push it now.
>
>    // The upcoming Elastic Metaspace will have stricter alignment
> requirements.
>    // For debug builds, increase reserve alignment to shake loose errors
> resulting
>    // from misusing this alignment.
>    // Note: do not increase too much (e.g. not on platforms with 64K
> pages), we do not
>    // want to disturb tests requiring precise numbers for metaspace size
> or ccs size.
> #ifdef ASSERT
>    if (_reserve_alignment == 4 * K) {
>      _reserve_alignment *= 4;
>    }
> #endif
>
>
> More to come ....
>
> Thanks
> - Ioi
>
>
>
All good points, I'll wait for your final review.

Cheers, Thomas


> On 4/28/20 7:54 AM, Thomas St?fe wrote:
> > Hi all,
> >
> > Could I have reviews for the following proposal of reworking cds/class
> > space reservation?
> >
> > Bug: https://bugs.openjdk.java.net/browse/JDK-8243392
> >
> > Webrev:
> >
> http://cr.openjdk.java.net/~stuefe/webrevs/rework-cds-ccs-reservation/webrev.00/webrev/
> >
> > (Many thanks to Ioi Lam for so patiently explaining CDS internals to
> > me, and to Andrew Haley and Nick Gasson for help with aarch64!)
> >
> > Reservation of the compressed class space is needlessly complicated
> > and has some minor issues. It can be simplified and made clearer.
> >
> > The complexity stems from the fact that this area lives at the
> > intersection of two to three sub systems, depending on how one counts.
> > Metaspace, CDS, and the platform which may or may not its own view of
> > how to reserve class space. And this code has been growing organically
> > over time.
> >
> > One small example:
> >
> > ReservedSpace Metaspace::reserve_preferred_space(size_t size, size_t
> > alignment,
> >                                                  bool large_pages,
> > char *requested_addr,
> >                                                  bool use_requested_addr)
> >
> > which I spent hours decoding, resulting in a very confused mail to
> > hs-runtime and aarch64-port-dev [2].
> >
> > This patch attempts to simplify cds and metaspace setup a bit; to
> > comment implicit knowledge which is not immediately clear; to cleanly
> > abstract platform concerns like optimized class space placement; and
> > to disentangle cds from metaspace to solve issues which may bite us
> > later with Elastic Metaspace [4].
> >
> > ---
> >
> > The main change is the reworked reservation mechanism. This is based
> > on Ioi's proposal [5].
> >
> > When reserving class space, three things must happen:
> >
> > 1) reservation of the space obviously. If cds is active that space
> > must be in the vicinity of cds archives to be covered by compressed
> > class pointer encoding.
> > 2) setting up the internal Metaspace structures atop of that space
> > 3) setting up compressed class pointer encoding.
> >
> > In its current form, Metaspace may or may not do some or all of that
> > in one function
> > (Metaspace::allocate_metaspace_compressed_klass_ptrs(ReservedSpace
> > metaspace_rs, char* requested_addr, address cds_base);) - if cds is
> > active, it will reserve the space for Metaspace and hand it in,
> > otherwise it will create it itself.
> >
> > When discussing this in [2], Ioi proposed to move the reservation of
> > the class space completely out of Metaspace and make it a
> > responsibility of the caller always. This would reduce some
> > complexity, and this patch follows the proposal.
> >
> > I removed
> > Metaspace::allocate_metaspace_compressed_klass_ptrs(ReservedSpace
> > metaspace_rs, char* requested_addr, address cds_base); and all its sub
> > functions.
> >
> > (1) now has to always be done outside - a ReservedSpace for class
> > space has to be provided by the caller. However, Metaspace now offers
> > a utility function for reserving space at a "nice" location, and
> > explicitly doing nothing else:
> >
> > ReservedSpace
> > Metaspace::reserve_address_space_for_compressed_classes(size_t size);
> >
> > this function can be redefined on a platform level for platform
> > optimized reservation, see below for details.
> >
> > (2) is taken care of by a new function,
> > Metaspace::initialize_class_space(ReservedSpace rs)
> >
> > (3) is taken care of a new function
> > CompressedKlassPointers::initialize(), see below for details.
> >
> >
> > So, class space now is set up three explicit steps:
> >
> > - First, reserve a suitable space by however means you want. For
> > convenience you may use
> > Metaspace::reserve_address_space_for_compressed_classes(), or you may
> > roll your own reservation.
> > - Next, tell Metaspace to use that range as backing storage for class
> > space: Metaspace::initialize_class_space(ReservedSpace rs)
> > - Finally, set up encoding. Encoding is independent from the concept
> > of a ReservedSpace, it just gets an address range, see below for details.
> >
> > Separating these steps and moving them out of the responsibility of
> > Metaspace makes this whole thing more flexible; it also removes
> > unnecessary knowledge (e.g. Metaspace does not need to know anything
> > about either ccp encoding or cds).
> >
> > ---
> >
> > How it comes together:
> >
> > If CDS is off, we just reserve a space using
> > Metaspace::reserve_address_space_for_compressed_classes(), initialize
> > it with Metaspace::initialize_class_space(ReservedSpace rs), then set
> > up compressed class pointer encoding covering the range of this class
> > space.
> >
> > If CDS is on (dump time), we reserve large 4G space, either at
> > SharedBaseAddress or using
> > Metaspace::reserve_address_space_for_compressed_classes(); we then
> > split that into 3G archive space and 1G class space; we set up that
> > space with Metaspace as class space; then we set up compressed class
> > pointer encoding covering both archive space and cds.
> >
> > If CDS is on (run time), we reserve a large space, split it into
> > archive space (large enough to hold both archives) and class space,
> > then basically proceed as above.
> >
> > Note that this is almost exactly how things worked before (modulo some
> > minor fixes, e.g. alignment issues), only the code is reformed and
> > made more explicit.
> >
> > ---
> >
> > I moved compressed class pointer setup over to CompressedKlassPointers
> > and changed the interface:
> >
> > -void Metaspace::set_narrow_klass_base_and_shift(ReservedSpace
> > metaspace_rs, address cds_base)
> > +void CompressedKlassPointers::initialize(address addr, size_t len);
> >
> > Instead of feeding it a single ReservedSpace, which is supposed to
> > represent class space, and an optional alternate base if cds is on,
> > now we give it just an numeric address range. That range marks the
> > limits to where Klass structures are to be expected, and is the
> > implicit promise that outside that range no Klass structures will
> > exist, so encoding has to cover only this range.
> >
> > This range may contain just the class space; or class space+cds; or
> > whatever allocation scheme we come up with in the future. Encoding
> > does not really care how the memory is organized as long as the input
> > range covers all possible Klass locations. That way we remove
> > knowledge about class space/cds from compressed class pointer encoding.
> >
> > Moving it away from metaspace.cpp into the CompressedKlassPointers
> > class also mirrors CompressedOops::initialize().
> >
> > ---
> >
> > I renamed _narrow_klass_range to just _range, because strictly
> > speaking this is the range un-narrow Klass pointers can have.
> >
> > As for the implementation of
> > CompressedKlassPointers::initialize(address addr, size_t len), I
> > mimicked very closely what happened before, so there should be almost
> > no differences. Since "almost no differences" sounds scary :) here are
> > the differences:
> >
> > - When CDS is active (dump or run time) we now always,
> > unconditionally, set the encoding range to 4G. This fixes a
> > theoretical bug discussed on aarch64-port-dev [1].
> >
> > - When CDS is not active, we set the encoding range to the minimum
> > required length. Before, it was left at its default value of 4G.
> >
> > Both differences only affect aarch64, since they are currently the
> > only one using the range field in CompressedKlassPointers.
> >
> > I wanted to add an assert somewhere to test encoding of the very last
> > address of the CompressedKlassPointers range, again to prevent errors
> > like [3]. But I did not come up with a good place for this assert
> > which would cover also the encoding done by C1/C2.
> >
> > For the same reason I thought about introducing a mode where Klass
> > structures would be allocated in reverse order, starting at the end of
> > the ccs, but again left it out as too big a change.
> >
> > ---
> >
> > OS abstraction: platforms may have restrictions of what constitutes a
> > valid compressed class pointer encoding base. Or if not, they may have
> > at least preferences. There was logic like this in metaspace.cpp,
> > which I removed and cleanly factored out into platform dependent
> > files, giving each platform the option to add special logic.
> >
> > These are two new methods:
> >
> > - bool CompressedKlassPointers::is_valid_base(address p)
> >
> > to let the platform tell you whether it considers p to be a valid
> > encoding base. The only platform having these restrictions currently
> > is aarch64.
> >
> > - ReservedSpace
> > Metaspace::reserve_address_space_for_compressed_classes(size_t size);
> >
> > this hands over the process of allocating a range suitable for
> > compressed class pointer encoding to the platform. Most platforms will
> > allocate just anywhere, but some platforms may have a better strategy
> > (e.g. trying low memory first, trying only correctly aligned addresses
> > and so on).
> >
> > Beforehand, this coding existed in a similar form in metaspace.cpp for
> > aarch64 and AIX. For now, I left the AIX part out - it seems only half
> > done, and I want to check further if we even need it, if yes why not
> > on Linux ppc, and C1 does not seem to support anything other than
> > base+offset with shift either, but I may be mistaken.
> >
> > These two methods should give the platform enough control to implement
> > their own scheme for optimized class space placement without bothering
> > any shared code about it.
> >
> > Note about the form, I introduced two new platform dependent files,
> > "metaspace_<cpu>.cpp" and "compressedOops_<cpu>.cpp". I am not happy
> > about this but this seems to be what we generally do in hotspot, right?
> >
> > ---
> >
> > Metaspace reserve alignment vs cds alignment
> >
> > CDS was using Metaspace reserve alignment for CDS internal purposes. I
> > guess this was just a copy paste issue. It never caused problems since
> > Metaspace reserve alignment == page size, but that is not true anymore
> > in the upcoming Elastic Metaspace where reserve alignment will be
> > larger. This causes a number of issues.
> >
> > I separated those two cleanly. CDS now uses
> > os::vm_allocation_granularity. Metaspace::reserve_alignment is only
> > used in those two places where it is needed, when CDS creates the
> > address space for class space on behalf of the Metaspace.
> >
> > ---
> >
> > Windows special handling in CDS
> >
> > To simplify coding I removed the windows specific handling which left
> > out reservation of the archive. This was needed because windows cannot
> > mmap files into reserved regions. But fallback code exists in
> > filemap.cpp for this case which just reads in the region instead of
> > mapping it.
> >
> > Should that turn out to be a performance problem, I will reinstate the
> > feature. But a simpler way would be reserve the archive and later just
> > before mmapping the archive file to release the archive space. That
> > would not only be simpler but give us the best guarantee that that
> > address space is actually available. But I'd be happy to leave that
> > part out completely if we do not see any performance problems on
> > windows x64.
> >
> > ---
> >
> > NMT cannot deal with spaces which are split. This problem manifests in
> > that bookkeeping for class space is done under "Shared Classes", not
> > "Classes" as it should. This problem exists today too at dump time and
> > randomly at run time. But since I simplified the reservation, this
> > problem now shows up always, whether or not we map at the
> > SharedBaseAddress.
> > While I could work around this problem, I'd prefer this problem to be
> > solved at the core, and NMT to have an option to recognize reservation
> > splits. So I'd rather not put a workaround for this into the patch but
> > leave it for fixing as a separate issue. I opened this issue to track
> > it [6].
> >
> > ---
> >
> > Jtreg tests:
> >
> > I expanded the CompressedOops/CompressedClassPointers.java. I also
> > extended them to Windows. The tests now optionally omit strict class
> > space placement tests, since these tests heavily depend on ASLR and
> > were the reason they were excluded on Windows. However I think even
> > without checking for class space placement they make sense, just to
> > see that the VM comes up and lives with the many different settings we
> > can run in.
> >
> > ---
> >
> > Tests:
> >
> > - I ran the patch through Oracles submit repo
> > - I ran tests manually for aarch64, zero, linux 32bit and windows x64
> > - The whole battery of nightly tests at SAP, including ppc, ppcle and
> > aarch64, unfortunately excluding windows because of unrelated errors.
> > Windows x64 tests will be redone tonight.
> >
> >
> > Thank you,
> >
> > Thomas
> >
> > [1]
> >
> https://mail.openjdk.java.net/pipermail/aarch64-port-dev/2020-April/008804.html
> > [2]
> >
> https://mail.openjdk.java.net/pipermail/aarch64-port-dev/2020-April/008757.html
> > [3] https://bugs.openjdk.java.net/browse/JDK-8193266
> > [4] https://bugs.openjdk.java.net/browse/JDK-8221173
> > [5]
> >
> https://mail.openjdk.java.net/pipermail/aarch64-port-dev/2020-April/008765.html
> > [6] https://bugs.openjdk.java.net/browse/JDK-8243535
> >
>
>

From thomas.stuefe at gmail.com  Wed Apr 29 06:27:31 2020
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Wed, 29 Apr 2020 08:27:31 +0200
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64
In-Reply-To: <44febab6-5703-1c8b-d4ad-0f4c12517afa@oracle.com>
References: <56a0cf85-9698-64b2-95b4-e5b123c3e9e2@oracle.com>
 <CAA-vtUyFiMpf2vSXLf-c9uaKYXF1tSA3At5KJySCiz2skG0w3A@mail.gmail.com>
 <9a314ebe-5499-ecb6-3c5b-0c8ce7f9c8d2@oracle.com>
 <CAA-vtUwYFCwT1a0Qbcrqmo6OVLit3DROq0jC8WqAYEd3_3XY9A@mail.gmail.com>
 <44febab6-5703-1c8b-d4ad-0f4c12517afa@oracle.com>
Message-ID: <CAA-vtUz46XkNN0vOnR+KJzvUO=A0h+fE8A0TpZ0a56V-+Y10YQ@mail.gmail.com>

Hi Erik,

On Tue, Apr 28, 2020 at 6:28 PM Erik ?sterlund <erik.osterlund at oracle.com>
wrote:

> Hi Thomas,
>
> Thanks for reviewing this.
>
> On 2020-04-28 13:14, Thomas St?fe wrote:
>
> Hi Erik,
>
> On Tue, Apr 28, 2020 at 10:07 AM Erik ?sterlund <erik.osterlund at oracle.com>
> wrote:
>
>> Hi Thomas,
>>
>> On 2020-04-28 08:43, Thomas St?fe wrote:
>>
>> Hi Erik,
>>
>> nice change, and now I do not have to keep in mind "heap>32G = no class
>> space" :)
>>
>>
>> Glad you liked it!
>>
>> I eyeballed the metaspace changes (what little there are).
>>
>> This:
>>
>> -     char* base = (char*)align_up(CompressedOops::end(), _reserve_alignment);+     char* base;+     if (UseCompressedOops) {+       base = (char*)align_up(CompressedOops::end(), _reserve_alignment);+     } else {+       base = (char*)HeapBaseMinAddress;+     }
>>
>> may not work as intended. You probably want the end of the reserved heap as attach point for ccs, not the HeapBaseMinAddress.
>>
>>
>> Could you please expand on that? To make it clear, what this code does is
>> to put metaspace at the
>> minimum address we are allowed to map, when compressed oops is disabled.
>> The reasoning for doing
>> that is that if the low address bits (resulting in good compressed
>> pointers) are not used by compressed oops,
>> then they are available to be used by compressed class pointers instead,
>> such that at least compressed class
>> pointers can utilize this VA to get optimal encoding of class pointers.
>>
>> Is there anything not desirable about this? If so, I can't see it, and
>> would appreciate if you expand on why
>> that would be a bad idea. Perhaps there is something I do not see here.
>>
>>
> Oh, this was a long standing confusion in my head. I never noticed that
> HeapBaseMinAddress exists for the sole purpose of compressed oops, and is
> ignored otherwise. So I thought what you do there is just mapping ccs over
> the start of where the user wants the heap to start, and since that may not
> work we would fall back to map-anywhere.
>
> So you are saying HeapBaseMinAddress is the lowest address the platform
> allows us to map to, irregardless of whether we run with compressed oops or
> not.
>
>
> Right, so the purpose of this flag was to be able to move the heap up,
> because if we map things too low, we ran into conflicts with sbrk, and
> hence some old suboptimal malloc implementations, such as the default
> malloc on Solaris. This called for a flag to be able to move the heap up,
> putting it out of the way from sbrk. That is how HeapBaseMinAddress came
> about, AFAIK. The situation when mapping in metaspace and compressed class
> space, without compressed oops, is very much the same: we want to go as low
> as possible for optimal encoding, but not too low so we run into these
> problems.
>
> With compressed oops we continue to squeeze ccs in after the heap in the
> hope that it all still fits into the lower 4G or 32G. (Do we ever bother to
> check if heap is actually reserved in low area? It may have given up and
> reserved upstate instead, leaving free room below...).
>
>
> Right. And yes I'm pretty sure we do check. The heap reserves memory, and
> sets the CompressedOops::end() based on that, which is fed as input to the
> metaspace/CDS placement.
>
>
No, what I meant was: since we reserve at CompressedOops::end(), if the
heap was too large to place it in the lower address regions, the class
space will not live there either even if it could, since it is smaller and
might fit.

For example:

thomas at mainframe:/shared/projects/openjdk/jdk-jdk/source$
../../jdks/sapmachine14/bin/java -Xlog:gc+metaspace=trace -Xshare:off
-Xmx31g
[0.003s][trace][gc,metaspace] node @0x00007fca441b9f90: reserved=1048576.00
KB, committed=0.00 KB (  0%), used=0.00 KB (  0%)
[0.003s][trace][gc,metaspace]    [0x00000017c0400000, 0x00000017c0400000,
0x00000017c0400000, 0x0000001800400000)
[0.003s][trace][gc,metaspace] Narrow klass base: 0x00000017c0400000, Narrow
klass shift: 0
[0.003s][trace][gc,metaspace] Compressed class space size: 1073741824
Address: 0x00000017c0400000 Req Addr: 0x00000017c0400000
[0.003s][trace][gc,metaspace] node @0x00007fca441ba190: reserved=8192.00
KB, committed=0.00 KB (  0%), used=0.00 KB (  0%)
[0.003s][trace][gc,metaspace]    [0x00007fc9cb800000, 0x00007fc9cb800000,
0x00007fc9cb800000, 0x00007fc9cc000000)

(large heap and no CDS, since it prevents zero based mode).

In that case narrow klass base is not zero based even it if probably could
be, if we were to allocate class space in the lower regions, which are now
unoccupied since the large heap does not fit there. A missed opportunity to
go zero based. Not a big deal, since we mostly run with CDS now anyway.

Now with your patch without compressed oops we attempt to map at
> HeapBaseMinAddress since this is the lowest address we are allowed to map
> at and the heap won't obviously need it, right?
>
>
> Exactly.
>
> My only misgiving would be that HeapBaseMinAddress is even more of a
> misnomer (its already now a bit, witness my year long confusion :-), since
> it now controls not only heap placement but ccs too. Actually if it is that
> important to avoid mapping to addresses below HeapBaseMinAddress we should
> probably prevent it somewhere below in the os layer but I really do not
> want to open that can of worms.
>
>
> Yes agreed. Ideally it would be called something like MappingMinAddress
> instead, because it really isn't tied to the heap really, it just happened
> to be the primary user of mapping things at low addresses. But I also think
> changing that, and enforcing it more widely in the os layer sounds like a
> different can of worms.
>
> --
>
> Some more remarks about arguments.cpp:
>
> - In Arguments::set_use_compressed_klass_ptrs(), if now compressed class
> pointers are independent from compressed oops, checking
> CompressedClassSpaceSize for validity should now always happen for
> UseCompressedClassPointers=1.
>
>
> I think it looks like I do that.
>
> - If HeapBaseMinAddress is now always used, we should check it always too
> (see Arguments::set_heap_size)
>
>
> Good idea. Fixed.
>
>
> Also, could we have some tests which exercise cds/metaspace initialization
>> without coops and with ccps? Easiest way would be to extend
>> "CompressedOops/CompressedClassPointers.java" by some cases.
>>
>>
>>
> Thank you. I think this makes sense especially since compressed class
> pointer encoding is restricted to one mode if CDS is on (base = shared
> address and shift=3), and since CDS=on is the default zerobased encoding
> may not happen that often.
>
>
> Yup. I added some explicit test cases for -XX:-UseCompressedOops
> -XX:+UseCompressedClassPointers as suggested. This will also be the default
> configuration for ZGC, so that should give that configuration more test
> coverage in the various tests we have.
>
> Here is a new webrev:
> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.01/
>
> Incremental:
> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00_01/
>
> I hope I fixed everything you wanted in this patch. I also fixed what
> Frederic and Coleen wanted.
>
>
arguments.cpp, the metaspace stuff and the test look good to me now. Thank
you for your work!

..Thomas


> Thanks,
> /Erik
>
> I will have a look at that for the next webrev.
>>
>> Thanks for the review.
>>
>> /Erik
>>
>>
>
> Thanks,
>
> Thomas
>
>
>
>> Thank you, Thomas
>>
>>
>> On Fri, Apr 24, 2020 at 10:19 AM Erik ?sterlund <
>> erik.osterlund at oracle.com> wrote:
>>
>>> Hi,
>>>
>>> Today, the use of compressed class pointers assumes the use of
>>> compressed oops.
>>> This patch loosens up the relationship between compressed oops and
>>> compressed
>>> class pointers, so that compressed class pointers can be used without
>>> compressed
>>> oops. This benefits for example ZGC that wants compressed class
>>> pointers, but
>>> not compressed oops, effectively giving ZGC 4 bytes smaller objects.
>>>
>>> Much of the complexity of the change is that r12 used to be used by
>>> compressed
>>> class pointers as some kind of semi-fast temp register that one would
>>> restore
>>> to the compressed oops heap base (or zero). That made it effectively a
>>> slightly
>>> optimized spilling mechanism used to find a temp register. I replaced
>>> that
>>> mechanism with a plain old normal temp register that you pass in as a
>>> parameter.
>>> The bad news is that I had to find available temp registers in a lot of
>>> places.
>>> The good news is that almost always were there temp registers available
>>> for free,
>>> and hence the new temp register is even faster than the old optimized
>>> spilling
>>> mechanism. Because we almost never need any spilling at all in the
>>> contexts where
>>> this is used.
>>>
>>> Since I want the 4 new bytes to actually make objects smaller, I poked
>>> the new
>>> layout code a bit so that the klass gap is made available for fields.
>>> That used
>>> to be made available only with compressed oops enabled, due to
>>> restrictions in
>>> the layout code. Our new layout code does not have such restrictions,
>>> and so
>>> I will make the 4 bytes available for fields when the new layout code is
>>> used
>>> and compressed class pointers are used.
>>>
>>> Now I'm only fixing this for HotSpot x86_64. Ideally the use of
>>> compressed oops
>>> and compressed class pointers should not be entangled in other platforms
>>> and
>>> Graal. But that is beyond the scope of this change, and I will let them
>>> behave
>>> the way that they used to, to be potentially fixed later.
>>>
>>> Bug:
>>> https://bugs.openjdk.java.net/browse/JDK-8241825
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/
>>>
>>> Testing:
>>> hs-tier1-7
>>>
>>> Thanks,
>>> /Erik
>>>
>>
>>
>

From stefan.karlsson at oracle.com  Wed Apr 29 06:29:02 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Wed, 29 Apr 2020 08:29:02 +0200
Subject: RFR: 8244010: Simplify usages of
 ProcessTools.createJavaProcessBuilder in our tests
In-Reply-To: <b91260e2-52d7-3c5a-dc39-9d7333ad56bf@oracle.com>
References: <60c0e0e4-634b-d349-c6f3-ca1ad52926cb@oracle.com>
 <056ff9b1-1a4f-a1fb-2727-b503707f3750@oracle.com>
 <b91260e2-52d7-3c5a-dc39-9d7333ad56bf@oracle.com>
Message-ID: <bde49df7-184b-b67b-5136-3eb9e7da6f6d@oracle.com>

On 2020-04-29 07:12, David Holmes wrote:
> Hi Stefan,
>
> On 28/04/2020 11:54 pm, Stefan Karlsson wrote:
>> Hi again,
>>
>> I realized that we probably want to give ProcessTools.executeTestJvm 
>> the same treatment.
>>
>> Side-note: It's very awkard that createJavaProcessBuilder defaults to 
>> not adding user-specifed flags, but executeTestJvm does. I think it 
>> would be good to unify this as a separate RFE. I think *a lot* of 
>> callers to createJavaProcessBuilder could be simplified by either 
>> using executeTestJvm directly, or a simplified version of that.
>
> createJavaProcessBuilder is the primitive and over time we have added 
> higher-level APIs to abstract away the boiler-plate that deal with the 
> externally passed flags, gives you an OutputAnalyzer etc.
Right, and what I'm saying is that I think a lot of the code could 
benefit from using the higher-level APIs. Though I'm not sure the once 
we have are an exact match to what we need, so we might have to add some 
more, or tweak some of the old once.

>
> The underlying java.lang.ProcessBuilder supports both List<String> and 
> String[] for passing arguments, but returns the command as 
> List<String>, so this has also affected the ProcessTools API. As 
> Leonid suggests adding in a List<String> overload suffices rather than 
> Collection<String>.

Yes, and I answered Leonid's review and said that I would change it to 
List<String>. I've tested that version with tier1-5.

>
>
>> I'm running testing through mach5 and found a few things to fix, I 
>> might find more when the testing has proceeded further.
>>
>> This is the current patch:
>> https://cr.openjdk.java.net/~stefank/8244010/webrev.02.delta
>> https://cr.openjdk.java.net/~stefank/8244010/webrev.02
>
> One thing I noticed, as you copied it, is that this comment to all the 
> createJavaProcessBuilder methods seems wrong:
>
> 271????? * with any platform specific arguments prepended
>
> as we only add the other arguments if we pass "true" for 
> addTestVmAndJavaOptions. ??

I saw that, but thought that the @param addTestVmAndJavaOptions 
disambiguated this. I'll remove that part from all the comments.

>
> Also regarding the String[]/List conversions ... as ProcessBuilder 
> supports both I really don't see why we are doing:
>
> ?334???????? return new ProcessBuilder(args.toArray(new 
> String[args.size()]));

I'll fix that as well.

>
>
> and the "reporting" section could just iterate the args rather than 
> creating a StringBuilder.

I'm unsure about that. I can't tell if this is done to ensure that the 
reporting is atomic. I'll leave this for someone else to fix.

Thanks,
StefanK

>
> Cheers,
> David
>
>> Thanks,
>> StefanK
>>
>> On 2020-04-28 13:58, Stefan Karlsson wrote:
>>> Hi all,
>>>
>>> Please review this patch to simplify usages of 
>>> ProcessTools.createJavaProcessBuilder in our tests.
>>>
>>> https://cr.openjdk.java.net/~stefank/8244010/webrev.01/
>>> https://bugs.openjdk.java.net/browse/JDK-8244010
>>>
>>> I saw all this code when reviewing changes to how we pass flags in 
>>> our tests. There are a many places where arguments are converted and 
>>> passed back and forth in String[] and Collections.
>>>
>>> For example:
>>> ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(
>>> ??????????argsList.toArray(new String[argsList.size()]));
>>>
>>> If we add an overload the createJavaProcessBuilder, that takes a 
>>> Collection<String> as an argument, then we can write the code above as:
>>> ???ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(argsList);
>>>
>>> Other places temporarily put the flags in a String[], where most 
>>> calls simply lists the arguments in the call:
>>> ??String[] opts = {Xmx, "-XX:NativeMemoryTracking=detail", 
>>> "-XX:+UseParallelGC", "-version"};
>>> ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(opts);
>>>
>>> And some places put the args in a temporary Collection:
>>> ??LinkedList<String> vmOptions = new LinkedList<>();
>>> ??vmOptions.add(gc);
>>> ??vmOptions.add("-Xmx" + minMaxHeap);
>>> ??vmOptions.add("-XX:+PrintFlagsFinal");
>>> ??vmOptions.add(VerifyHeapSize.class.getName());
>>>
>>> ??ProcessBuilder pb = 
>>> ProcessTools.createJavaProcessBuilder(vmOptions.toArray(new 
>>> String[0]));
>>>
>>> I'd like to cleanup, simplify, and unify many of these usages.
>>>
>>> I've tested this by running all the changed tests locally.
>>>
>>> Thanks,
>>> StefanK
>>


From david.holmes at oracle.com  Wed Apr 29 06:31:51 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 29 Apr 2020 16:31:51 +1000
Subject: RFR: 8244010: Simplify usages of
 ProcessTools.createJavaProcessBuilder in our tests
In-Reply-To: <bde49df7-184b-b67b-5136-3eb9e7da6f6d@oracle.com>
References: <60c0e0e4-634b-d349-c6f3-ca1ad52926cb@oracle.com>
 <056ff9b1-1a4f-a1fb-2727-b503707f3750@oracle.com>
 <b91260e2-52d7-3c5a-dc39-9d7333ad56bf@oracle.com>
 <bde49df7-184b-b67b-5136-3eb9e7da6f6d@oracle.com>
Message-ID: <2827832c-5282-e6d3-04b1-b14ed1e9960b@oracle.com>

One follow up at end ...

On 29/04/2020 4:29 pm, Stefan Karlsson wrote:
> On 2020-04-29 07:12, David Holmes wrote:
>> Hi Stefan,
>>
>> On 28/04/2020 11:54 pm, Stefan Karlsson wrote:
>>> Hi again,
>>>
>>> I realized that we probably want to give ProcessTools.executeTestJvm 
>>> the same treatment.
>>>
>>> Side-note: It's very awkard that createJavaProcessBuilder defaults to 
>>> not adding user-specifed flags, but executeTestJvm does. I think it 
>>> would be good to unify this as a separate RFE. I think *a lot* of 
>>> callers to createJavaProcessBuilder could be simplified by either 
>>> using executeTestJvm directly, or a simplified version of that.
>>
>> createJavaProcessBuilder is the primitive and over time we have added 
>> higher-level APIs to abstract away the boiler-plate that deal with the 
>> externally passed flags, gives you an OutputAnalyzer etc.
> Right, and what I'm saying is that I think a lot of the code could 
> benefit from using the higher-level APIs. Though I'm not sure the once 
> we have are an exact match to what we need, so we might have to add some 
> more, or tweak some of the old once.
> 
>>
>> The underlying java.lang.ProcessBuilder supports both List<String> and 
>> String[] for passing arguments, but returns the command as 
>> List<String>, so this has also affected the ProcessTools API. As 
>> Leonid suggests adding in a List<String> overload suffices rather than 
>> Collection<String>.
> 
> Yes, and I answered Leonid's review and said that I would change it to 
> List<String>. I've tested that version with tier1-5.
> 
>>
>>
>>> I'm running testing through mach5 and found a few things to fix, I 
>>> might find more when the testing has proceeded further.
>>>
>>> This is the current patch:
>>> https://cr.openjdk.java.net/~stefank/8244010/webrev.02.delta
>>> https://cr.openjdk.java.net/~stefank/8244010/webrev.02
>>
>> One thing I noticed, as you copied it, is that this comment to all the 
>> createJavaProcessBuilder methods seems wrong:
>>
>> 271????? * with any platform specific arguments prepended
>>
>> as we only add the other arguments if we pass "true" for 
>> addTestVmAndJavaOptions. ??
> 
> I saw that, but thought that the @param addTestVmAndJavaOptions 
> disambiguated this. I'll remove that part from all the comments.
> 
>>
>> Also regarding the String[]/List conversions ... as ProcessBuilder 
>> supports both I really don't see why we are doing:
>>
>> ?334???????? return new ProcessBuilder(args.toArray(new 
>> String[args.size()]));
> 
> I'll fix that as well.
> 
>>
>>
>> and the "reporting" section could just iterate the args rather than 
>> creating a StringBuilder.
> 
> I'm unsure about that. I can't tell if this is done to ensure that the 
> reporting is atomic. I'll leave this for someone else to fix.

Good point on atomicity - that's important for our test logs.

Thanks,
David

> Thanks,
> StefanK
> 
>>
>> Cheers,
>> David
>>
>>> Thanks,
>>> StefanK
>>>
>>> On 2020-04-28 13:58, Stefan Karlsson wrote:
>>>> Hi all,
>>>>
>>>> Please review this patch to simplify usages of 
>>>> ProcessTools.createJavaProcessBuilder in our tests.
>>>>
>>>> https://cr.openjdk.java.net/~stefank/8244010/webrev.01/
>>>> https://bugs.openjdk.java.net/browse/JDK-8244010
>>>>
>>>> I saw all this code when reviewing changes to how we pass flags in 
>>>> our tests. There are a many places where arguments are converted and 
>>>> passed back and forth in String[] and Collections.
>>>>
>>>> For example:
>>>> ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(
>>>> ??????????argsList.toArray(new String[argsList.size()]));
>>>>
>>>> If we add an overload the createJavaProcessBuilder, that takes a 
>>>> Collection<String> as an argument, then we can write the code above as:
>>>> ???ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(argsList);
>>>>
>>>> Other places temporarily put the flags in a String[], where most 
>>>> calls simply lists the arguments in the call:
>>>> ??String[] opts = {Xmx, "-XX:NativeMemoryTracking=detail", 
>>>> "-XX:+UseParallelGC", "-version"};
>>>> ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(opts);
>>>>
>>>> And some places put the args in a temporary Collection:
>>>> ??LinkedList<String> vmOptions = new LinkedList<>();
>>>> ??vmOptions.add(gc);
>>>> ??vmOptions.add("-Xmx" + minMaxHeap);
>>>> ??vmOptions.add("-XX:+PrintFlagsFinal");
>>>> ??vmOptions.add(VerifyHeapSize.class.getName());
>>>>
>>>> ??ProcessBuilder pb = 
>>>> ProcessTools.createJavaProcessBuilder(vmOptions.toArray(new 
>>>> String[0]));
>>>>
>>>> I'd like to cleanup, simplify, and unify many of these usages.
>>>>
>>>> I've tested this by running all the changed tests locally.
>>>>
>>>> Thanks,
>>>> StefanK
>>>
> 

From stefan.karlsson at oracle.com  Wed Apr 29 06:36:10 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Wed, 29 Apr 2020 08:36:10 +0200
Subject: RFR: 8244010: Simplify usages of
 ProcessTools.createJavaProcessBuilder in our tests
In-Reply-To: <2b745d4b-d46e-c9eb-e48d-f3020e04e6e6@oracle.com>
References: <60c0e0e4-634b-d349-c6f3-ca1ad52926cb@oracle.com>
 <056ff9b1-1a4f-a1fb-2727-b503707f3750@oracle.com>
 <2b745d4b-d46e-c9eb-e48d-f3020e04e6e6@oracle.com>
Message-ID: <18e6a384-c947-1514-d766-bb6c09bd3cae@oracle.com>

On 2020-04-29 07:37, Ioi Lam wrote:
> Hi Stefan,
>
> For a future RFE, I would suggesting adding a new function
>
> ??? ProcessTools.createJavaProcessBuilder(Object... args)
>
> Every argument must be a String, a String[], a List<String>, or NULL.
>
> You can simplified code like this:
>
> ?? static ProcessBuilder exec(String... args) throws Exception {
> ??????? List<String> argsList = new ArrayList<>();
> ??????? Collections.addAll(argsList, args);
> ??????? Collections.addAll(argsList, "-Xmn8m");
> ??????? if (cond) {
> ??????????? Collections.addAll(argsList, "-Dtest.classes=" + 
> System.getProperty("test.classes","."));
> ??????? }
> ??????? Collections.addAll(argsList, 
> ClassUnloadTestMain.class.getName());
> ??????? return ProcessTools.createJavaProcessBuilder(argsList);
> ??? }
>
> to this:
>
> ??? static ProcessBuilder exec(String... args) throws Exception {
> ??????? return ProcessTools.createJavaProcessBuilder(
> ????????????????? args,
> ????????????????? "-Xmn8m",
> (cond) ? "-Dtest.classes=" + System.getProperty("test.classes",".") : 
> NULL,
> ClassUnloadTestMain.class.getName());
> ??? }
>
> We can probably allow higher-level dimensions like Object[][], as long 
> as the eventual element type is a String.
>
> What do you think?

I had similar thoughts while doing this patch. I'm on the fence about 
this. It makes the call sites much nicer, but it makes the API less 
clear and more error-prone.

StefanK

>
> - Ioi
>
>
>
> On 4/28/20 6:54 AM, Stefan Karlsson wrote:
>> Hi again,
>>
>> I realized that we probably want to give ProcessTools.executeTestJvm 
>> the same treatment.
>>
>> Side-note: It's very awkard that createJavaProcessBuilder defaults to 
>> not adding user-specifed flags, but executeTestJvm does. I think it 
>> would be good to unify this as a separate RFE. I think *a lot* of 
>> callers to createJavaProcessBuilder could be simplified by either 
>> using executeTestJvm directly, or a simplified version of that.
>>
>> I'm running testing through mach5 and found a few things to fix, I 
>> might find more when the testing has proceeded further.
>>
>> This is the current patch:
>> https://cr.openjdk.java.net/~stefank/8244010/webrev.02.delta
>> https://cr.openjdk.java.net/~stefank/8244010/webrev.02
>>
>> Thanks,
>> StefanK
>>
>> On 2020-04-28 13:58, Stefan Karlsson wrote:
>>> Hi all,
>>>
>>> Please review this patch to simplify usages of 
>>> ProcessTools.createJavaProcessBuilder in our tests.
>>>
>>> https://cr.openjdk.java.net/~stefank/8244010/webrev.01/
>>> https://bugs.openjdk.java.net/browse/JDK-8244010
>>>
>>> I saw all this code when reviewing changes to how we pass flags in 
>>> our tests. There are a many places where arguments are converted and 
>>> passed back and forth in String[] and Collections.
>>>
>>> For example:
>>> ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(
>>> ??????????argsList.toArray(new String[argsList.size()]));
>>>
>>> If we add an overload the createJavaProcessBuilder, that takes a 
>>> Collection<String> as an argument, then we can write the code above as:
>>> ???ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(argsList);
>>>
>>> Other places temporarily put the flags in a String[], where most 
>>> calls simply lists the arguments in the call:
>>> ??String[] opts = {Xmx, "-XX:NativeMemoryTracking=detail", 
>>> "-XX:+UseParallelGC", "-version"};
>>> ??ProcessBuilder pb = ProcessTools.createJavaProcessBuilder(opts);
>>>
>>> And some places put the args in a temporary Collection:
>>> ??LinkedList<String> vmOptions = new LinkedList<>();
>>> ??vmOptions.add(gc);
>>> ??vmOptions.add("-Xmx" + minMaxHeap);
>>> ??vmOptions.add("-XX:+PrintFlagsFinal");
>>> ??vmOptions.add(VerifyHeapSize.class.getName());
>>>
>>> ??ProcessBuilder pb = 
>>> ProcessTools.createJavaProcessBuilder(vmOptions.toArray(new 
>>> String[0]));
>>>
>>> I'd like to cleanup, simplify, and unify many of these usages.
>>>
>>> I've tested this by running all the changed tests locally.
>>>
>>> Thanks,
>>> StefanK
>>
>


From david.holmes at oracle.com  Wed Apr 29 06:40:28 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 29 Apr 2020 16:40:28 +1000
Subject: RFR: 8243389: enhance os::pd_print_cpu_info on linux
In-Reply-To: <AM0PR02MB54120B7109B39A1598313A4493AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
References: <AM0PR02MB541225ED5513768AED6732F093AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
 <AM4PR02MB30579EBA32B978AC69A3F8DD9AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>
 <AM0PR02MB54121E0BA017A19057FAF78193AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
 <AM4PR02MB3057422667C0D81A15B420059AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>
 <AM0PR02MB5412CB78884450FCE489DF3493AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
 <AM4PR02MB30572CD677351CA02B0143CB9AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>
 <AM0PR02MB54120B7109B39A1598313A4493AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
Message-ID: <e8ee7786-f902-f33b-7bad-28072e0bfc02@oracle.com>

On 29/04/2020 1:45 am, Baesken, Matthias wrote:
> 
> Thanks !
> 
> May I have a second review ?

Seems reasonable.

Minor style nit: i=0; i<10;

spaces around binary operators please.

Thanks,
David

> 
> Best regards, Matthias
> 
> 
> 
>> Thumbs up from my side.
>>
>> Thanks and best regards,
>> Martin
>>
>>>
>>> Thanks, here is a new webrev :
>>>
>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8243389.1/
>>>
> 

From matthias.baesken at sap.com  Wed Apr 29 06:59:44 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 29 Apr 2020 06:59:44 +0000
Subject: RFR: 8243389: enhance os::pd_print_cpu_info on linux
In-Reply-To: <e8ee7786-f902-f33b-7bad-28072e0bfc02@oracle.com>
References: <AM0PR02MB541225ED5513768AED6732F093AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
 <AM4PR02MB30579EBA32B978AC69A3F8DD9AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>
 <AM0PR02MB54121E0BA017A19057FAF78193AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
 <AM4PR02MB3057422667C0D81A15B420059AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>
 <AM0PR02MB5412CB78884450FCE489DF3493AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
 <AM4PR02MB30572CD677351CA02B0143CB9AAC0@AM4PR02MB3057.eurprd02.prod.outlook.com>
 <AM0PR02MB54120B7109B39A1598313A4493AC0@AM0PR02MB5412.eurprd02.prod.outlook.com>
 <e8ee7786-f902-f33b-7bad-28072e0bfc02@oracle.com>
Message-ID: <AM0PR02MB5412ACDEE8C28E7197F2C2A893AD0@AM0PR02MB5412.eurprd02.prod.outlook.com>

Hi David, thanks for the review !
I'll add the spaces.

Best regards, Matthias


>Seems reasonable.
>
>Minor style nit: i=0; i<10;
>
>spaces around binary operators please.

 

From erik.osterlund at oracle.com  Wed Apr 29 07:39:48 2020
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Wed, 29 Apr 2020 09:39:48 +0200
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64
In-Reply-To: <CAA-vtUz46XkNN0vOnR+KJzvUO=A0h+fE8A0TpZ0a56V-+Y10YQ@mail.gmail.com>
References: <56a0cf85-9698-64b2-95b4-e5b123c3e9e2@oracle.com>
 <CAA-vtUyFiMpf2vSXLf-c9uaKYXF1tSA3At5KJySCiz2skG0w3A@mail.gmail.com>
 <9a314ebe-5499-ecb6-3c5b-0c8ce7f9c8d2@oracle.com>
 <CAA-vtUwYFCwT1a0Qbcrqmo6OVLit3DROq0jC8WqAYEd3_3XY9A@mail.gmail.com>
 <44febab6-5703-1c8b-d4ad-0f4c12517afa@oracle.com>
 <CAA-vtUz46XkNN0vOnR+KJzvUO=A0h+fE8A0TpZ0a56V-+Y10YQ@mail.gmail.com>
Message-ID: <e34303ab-9ad3-476f-2072-67e3e860a0ce@oracle.com>

Hi Thomas,

On 2020-04-29 08:27, Thomas St?fe wrote:
> Hi Erik,
>
> On Tue, Apr 28, 2020 at 6:28 PM Erik ?sterlund 
> <erik.osterlund at oracle.com <mailto:erik.osterlund at oracle.com>> wrote:
>
>     Hi Thomas,
>
>     Thanks for reviewing this.
>
>     On 2020-04-28 13:14, Thomas St?fe wrote:
>
>>     With compressed oops we continue to squeeze ccs in after the heap
>>     in the hope that it all still fits into the lower 4G or 32G. (Do
>>     we ever bother to check if heap is actually reserved in low area?
>>     It may have given up and reserved upstate instead, leaving free
>>     room below...).
>
>     Right. And yes I'm pretty sure we do check. The heap reserves
>     memory, and sets the CompressedOops::end() based on that, which is
>     fed as input to the metaspace/CDS placement.
>
>
> No, what I meant was: since we reserve at CompressedOops::end(), if 
> the heap was too large to place it in the lower address regions, the 
> class space will not live there either even if it could, since it is 
> smaller and might fit.
>
> For example:
>
> thomas at mainframe:/shared/projects/openjdk/jdk-jdk/source$ 
> ../../jdks/sapmachine14/bin/java -Xlog:gc+metaspace=trace -Xshare:off 
> -Xmx31g
> [0.003s][trace][gc,metaspace] node @0x00007fca441b9f90: 
> reserved=1048576.00 KB, committed=0.00 KB ( ?0%), used=0.00 KB ( ?0%)
> [0.003s][trace][gc,metaspace] ?[0x00000017c0400000, 
> 0x00000017c0400000, 0x00000017c0400000, 0x0000001800400000)
> [0.003s][trace][gc,metaspace] Narrow klass base: 0x00000017c0400000, 
> Narrow klass shift: 0
> [0.003s][trace][gc,metaspace] Compressed class space size: 1073741824 
> Address: 0x00000017c0400000 Req Addr: 0x00000017c0400000
> [0.003s][trace][gc,metaspace] node @0x00007fca441ba190: 
> reserved=8192.00 KB, committed=0.00 KB ( ?0%), used=0.00 KB ( ?0%)
> [0.003s][trace][gc,metaspace] ?[0x00007fc9cb800000, 
> 0x00007fc9cb800000, 0x00007fc9cb800000, 0x00007fc9cc000000)
>
> (large heap and no CDS, since it prevents zero based mode).
>
> In that case narrow klass base is not zero based even it if probably 
> could be, if we were to allocate class space in the lower regions, 
> which are now unoccupied since the large heap does not fit there. A 
> missed opportunity to go zero based. Not a big deal, since we mostly 
> run with CDS now anyway.

I see. Yeah, there is room for improvement there. Although, as you say, 
people mostly run with CDS on, so it probably won't matter even if that 
is improved.

>>     Now with your patch without compressed oops we attempt to map at
>>     HeapBaseMinAddress since this is the lowest address we are
>>     allowed to map at and the heap won't obviously need it,?right?
>
>     Exactly.
>
>>     My only misgiving would be that?HeapBaseMinAddress is even more
>>     of a misnomer (its already now a?bit, witness my year long
>>     confusion :-), since it now controls not only heap placement but
>>     ccs too. Actually if it is that important to avoid mapping to
>>     addresses below?HeapBaseMinAddress we should probably prevent it
>>     somewhere below in the os layer but I really do not want to open
>>     that can of worms.
>
>     Yes agreed. Ideally it would be called something like
>     MappingMinAddress instead, because it really isn't tied to the
>     heap really, it just happened to be the primary user of mapping
>     things at low addresses. But I also think changing that, and
>     enforcing it more widely in the os layer sounds like a different
>     can of worms.
>
>>     --
>>
>>     Some more remarks about arguments.cpp:
>>
>>     - In Arguments::set_use_compressed_klass_ptrs(), if now
>>     compressed class pointers are independent from compressed oops,
>>     checking CompressedClassSpaceSize for validity should now always
>>     happen for UseCompressedClassPointers=1.
>
>     I think it looks like I do that.
>
>>     - If HeapBaseMinAddress is now always used, we should check it
>>     always too (see Arguments::set_heap_size)
>
>     Good idea. Fixed.
>
>>
>>>         Also, could we have some?tests which exercise cds/metaspace
>>>         initialization without coops and with ccps? Easiest way
>>>         would be to extend
>>>         "CompressedOops/CompressedClassPointers.java" by some cases.
>>
>>
>>     Thank you. I think this makes sense especially since compressed
>>     class pointer encoding is restricted to one mode if CDS is on
>>     (base = shared address and shift=3), and since CDS=on is the
>>     default zerobased encoding may not happen that often.
>
>     Yup. I added some explicit test cases for -XX:-UseCompressedOops
>     -XX:+UseCompressedClassPointers as suggested. This will also be
>     the default configuration for ZGC, so that should give that
>     configuration more test coverage in the various tests we have.
>
>     Here is a new webrev:
>     http://cr.openjdk.java.net/~eosterlund/8241825/webrev.01/
>
>     Incremental:
>     http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00_01/
>
>     I hope I fixed everything you wanted in this patch. I also fixed
>     what Frederic and Coleen wanted.
>
>
> arguments.cpp, the metaspace stuff and the test look good to me now. 
> Thank you for your work!

Thanks for the review Thomas!

/Erik

From robbin.ehn at oracle.com  Wed Apr 29 07:45:44 2020
From: robbin.ehn at oracle.com (Robbin Ehn)
Date: Wed, 29 Apr 2020 09:45:44 +0200
Subject: Thread-Local Handshakes slowdown on single-cpu machines
In-Reply-To: <46fd88cd-3e68-ebbf-869f-355ebb196146@oracle.com>
References: <5e65ba58-b599-e83c-b4ea-3d766ed3c457@palacsint.hu>
 <66059982-49bf-1a15-39fb-22008b069573@oracle.com>
 <7216aa59-ad6f-dd3c-4f37-4723d3b7e9a7@palacsint.hu>
 <857ffb5e-d9e5-3976-3968-124e3b47effb@oracle.com>
 <f23d9ba4-3380-050a-d772-8a8a8eeb5b15@palacsint.hu>
 <46fd88cd-3e68-ebbf-869f-355ebb196146@oracle.com>
Message-ID: <5fb7355d-52ab-c8bb-1fdd-fd0b13a9de85@oracle.com>

Hi,

On 2020-04-29 07:31, David Holmes wrote:
> On 29/04/2020 1:26 pm, Karak? Mikl?s wrote:
>> Hi Patricio,
>>
>> Yes, it seems like a scheduling issue for us too. One of my colleagues 
>> spotted the following sentence in Robbin Ehn's Handshaking HotSpot 
>> presentation (https://youtu.be/VBCOfAJ409s?t=2222): ~"if the VM thread 
>> finds that all threads are unsafe, it keeps spinning"
>> Could it be that actually the VM thread gets more CPU time and 
>> actually blocks JavaThread?
> 

At the moment handshake do not use yield or sleep.
http://hg.openjdk.java.net/jdk/jdk/file/ddb12e77e8dd/src/hotspot/share/runtime/handshake.cpp
Do while loop L184 -> L201.

I have tested at least three different approaches trying to ease off the
CPU, all off them suffered from performance regression on MP machines.
We have internally seen this on very rare occasions on MP (we do not
have UPs). Since this have been so rare, not trivial to fix without
performance regression and I was under the assumption that UP were
basically dead :), it have not been prioritized.

To just fix the UP case we could add something like:
diff -r f7919010c7c2 src/hotspot/share/runtime/handshake.cpp
--- a/src/hotspot/share/runtime/handshake.cpp   Tue Apr 28 09:14:17 2020 
+0200
+++ b/src/hotspot/share/runtime/handshake.cpp   Wed Apr 29 09:36:42 2020 
+0200
@@ -200,2 +200,5 @@
        }
+      if (!os::is_MP()) {
+        os::naked_yield();
+      }
      } while (!_op->is_completed());

Since no thread can progress (assuming that one iteration is much
shorter than a time-slice, this is extremely unlikely) there is no need
for a second iteration until we executed something else.

And should be trivial to backport to JDK 11.

Thanks, Robbin



> The "spinning" is intended to be done in a scheduler-friendly way via:
> 
> void os::naked_short_nanosleep(jlong ns) {
>  ? assert(ns > -1 && ns < NANOUNITS, "Un-interruptable sleep, short time 
> use only");
> 
>  ? int64_t start = os::javaTimeNanos();
>  ? do {
>  ??? if (SwitchToThread() == 0) {
>  ????? // Nothing else is ready to run on this cpu, spin a little
>  ????? SpinPause();
>  ??? }
>  ? } while (os::javaTimeNanos() - start < ns);
> }
> 
> but this won't be that scheduler-friendly because it places the VMThread 
> at the head of the priority queue, so it will cede up-to a timeslice to 
> another thread, then run another iteration of the loop and hence 
> alternate between a JavaThread and the VMThread until they are all 
> blocked at the safepoint. What is really wanted is for the VMThread to 
> go to the back of the scheduling queue.
> 
> David
> -----
> 
>> I did not except that but I could reproduce the issue with a simple 
>> hello world:
>>
>> public class Main {
>> ???? public static void main(String[] args) {
>> ???? ??? System.out.println("Hello world");
>> ???? }
>> }
>>
>> A few measurements on Windows 10 x64 in VirtualBox:
>>
>>
>> PS Z:\main> Measure-Command { .\main-1cpu.bat }
>>
>> Minutes?????????? : 0
>> Seconds?????????? : 7
>> Milliseconds????? : 20
>> Ticks???????????? : 70203455
>> TotalMinutes????? : 0.117005758333333
>> TotalSeconds????? : 7.0203455
>> TotalMilliseconds : 7020.3455
>>
>>
>>
>> PS Z:\main> Measure-Command { .\main-full.bat }
>>
>> Minutes?????????? : 0
>> Seconds?????????? : 1
>> Milliseconds????? : 575
>> Ticks???????????? : 15751352
>> TotalSeconds????? : 1.5751352
>> TotalMilliseconds : 1575.1352
>>
>>
>>
>> PS Z:\main> Measure-Command { .\main-full.bat }
>>
>> Minutes?????????? : 0
>> Seconds?????????? : 1
>> Milliseconds????? : 558
>> Ticks???????????? : 15585384
>> TotalSeconds????? : 1.5585384
>> TotalMilliseconds : 1558.5384
>>
>>
>>
>> PS Z:\main> Measure-Command { .\main-1cpu.bat }
>>
>> Minutes?????????? : 0
>> Seconds?????????? : 23
>> Milliseconds????? : 648
>> Ticks???????????? : 236482685
>> TotalSeconds????? : 23.6482685
>> TotalMilliseconds : 23648.2685
>>
>>
>> main-1cpu.bat:
>>
>> c:\windows\system32\cmd.exe /C start /affinity 1 \jdk-14\bin\java 
>> -Xlog:all=trace:file=app-1cpu.txt:uptime,tid,level,tags:filecount=50 Main
>>
>> main-full.bat:
>>
>> c:\windows\system32\cmd.exe /C start /affinity FF \jdk-14\bin\java 
>> -Xlog:all=trace:file=app-full.txt:uptime,tid,level,tags:filecount=50 Main
>>
>> The files are also available here:
>> https://palacsint.hu/etc/jdk/jdk14-single-cpu-slowdown-reprod-poc/
>> Or in a zip:
>> https://palacsint.hu/etc/jdk/jdk14-single-cpu-slowdown-reprod-poc.zip
>>
>> You can find trace level logs of three testruns in the run1-2-3 
>> directories as well.
>>
>> Best,
>> Miklos
>>
>>
>> On 4/29/20 2:18 AM, Patricio Chilano wrote:
>>> Hi Miklos,
>>>
>>> On 4/28/20 8:04 AM, Karak? Mikl?s wrote:
>>>> Hello David,
>>>>
>>>> I've tried with OpenJDK 15 EA Build 20. Unfortunately it's slow as 
>>>> with OpenJDK14.
>>>> A colleague of mine set CPU affinity with taskset to only one CPU on 
>>>> Linux and did not experience the slowdown. So the issue seems 
>>>> Windows-related.
>>> I wouldn?t rule out this being a scheduling issue. With thread-local 
>>> handshakes some operations like the ones you see in your logs 
>>> (deoptimization, biasedlocking revocation) do not need all 
>>> JavaThreads to be stopped at the same time in order to be executed. 
>>> If you only have one CPU but many JavaThreads running, depending on 
>>> the scheduler, maybe it could happen that the JavaThread you need to 
>>> handshake is constantly being left behind in the scheduler queue and 
>>> so the operation takes longer to execute. When you add 
>>> -XX:-ThreadLocalHandshakes, those operations will be executed using 
>>> safepoints, which means all JavaThreads are stopped. So even if the 
>>> scheduler decides to give priority to other JavaThreads those will 
>>> block and free the CPU. Also when adding more CPU that will increase 
>>> the likelihood of a JavaThread getting scheduled and might explain 
>>> why you see it gets fixed.
>>>
>>> Alternatively the JavaThread that needs to be handshaked is being 
>>> scheduled but is not polling for the handshake. But I think in that 
>>> the case you should still have the same issue with 
>>> -XX:-ThreadLocalHandshakes because the polling mechanism is the same, 
>>> although it's true that the misbehaving JavaThread will have more CPU 
>>> time to complete whatever is doing while others are stopped when 
>>> safepointing.
>>>
>>> One thing that makes me question the scheduler theory is the fact 
>>> that you see such improvements when only disabling biased-locking in 
>>> 13.0.2 because that was still using safepoints back then. Can you 
>>> check the logs to see in which operations you see the pauses when 
>>> running with default options compared to -XX:-UseBiasedLocking?
>>>
>>> I?m also surprised you see those long logging pauses because all the 
>>> other JavaThreads were already processed in the handshake so 
>>> something must be executing. Maybe you can switch the logging to 
>>> trace instead of debug. Also maybe you can add logging at Java code 
>>> so you can follow what is actually running? You could send it to the 
>>> same app.txt and flush every time you write to see it ordered in the 
>>> log. You can also get the uptime you see in the UL logs with 
>>> ManagementFactory.getRuntimeMXBean().getUptime(). That logging will 
>>> affect timing but might give some insights.
>>>
>>> Thanks,
>>> Patricio
>>>> Miklos
>>>>
>>>> On 4/28/20 1:02 AM, David Holmes wrote:
>>>>> Hi Miklos,
>>>>>
>>>>> On 28/04/2020 7:26 am, Karak? Mikl?s wrote:
>>>>>> Hello,
>>>>>>
>>>>>> We bumped into a possible Thread-Local Handshakes issue with 
>>>>>> multiple apps. It seems that enabled TLH slows down applications 
>>>>>> running on (although rare) single CPU boxes. I would be grateful 
>>>>>> if you could confirm that this is a known trade-off for these rare 
>>>>>> setups or a possible JVM bug. That would save us at least a few 
>>>>>> hours of debugging.
>>>>>>
>>>>>> TL;DR: Both tested apps stop frequently around 
>>>>>> "HandshakeOneThread", "HandshakeAllThreads" and "Revoked bias of 
>>>>>> currently-unlocked object" running with 
>>>>>> OpenJDK12/OpenJDK13/OpenJDK14. OpenJDK 13 with 
>>>>>> -XX:-ThreadLocalHandshakes fixes the issue. Adding a second CPU to 
>>>>>> the virtual machine fixes the issue. Enabling hyper-threading 
>>>>>> fixes the issue.
>>>>>
>>>>> Can you try with latest JDK 15 build, just to see if this may be 
>>>>> something already addressed?
>>>>>
>>>>> Thanks,
>>>>> David
>>>>>
>>>>>> More details are available at StackOverflow: 
>>>>>> https://stackoverflow.com/questions/61375565/slow-application-frequent-jvm-hangs-with-single-cpu-setups-and-java-12 
>>>>>>
>>>>>>
>>>>>> All thoughts are welcome.
>>>>>>
>>>>>> Best,
>>>>>> Miklos
>>>>>>
>>>>
>>>
>>

From nick.gasson at arm.com  Wed Apr 29 07:47:02 2020
From: nick.gasson at arm.com (Nick Gasson)
Date: Wed, 29 Apr 2020 15:47:02 +0800
Subject: [aarch64-port-dev ] RFR(M): 8243392: Remodel CDS/Metaspace
 storage reservation
In-Reply-To: <CAA-vtUyVki-DHepyAzORoTK_MwnrezFUB+QsxbWszztsx1x7NA@mail.gmail.com>
References: <CAA-vtUyVki-DHepyAzORoTK_MwnrezFUB+QsxbWszztsx1x7NA@mail.gmail.com>
Message-ID: <857dxyg2dl.fsf@arm.com>

Hi Thomas,

On 04/28/20 22:54 pm, Thomas St?fe wrote:
>
> These are two new methods:
>
> - bool CompressedKlassPointers::is_valid_base(address p)
>
> to let the platform tell you whether it considers p to be a valid encoding
> base. The only platform having these restrictions currently is aarch64.
>
> - ReservedSpace
> Metaspace::reserve_address_space_for_compressed_classes(size_t size);
>
> this hands over the process of allocating a range suitable for compressed
> class pointer encoding to the platform. Most platforms will allocate just
> anywhere, but some platforms may have a better strategy (e.g. trying low
> memory first, trying only correctly aligned addresses and so on).
>
> Beforehand, this coding existed in a similar form in metaspace.cpp for
> aarch64 and AIX. For now, I left the AIX part out - it seems only half
> done, and I want to check further if we even need it, if yes why not on
> Linux ppc, and C1 does not seem to support anything other than base+offset
> with shift either, but I may be mistaken.

Just a small comment:

  33 bool CompressedKlassPointers::is_valid_base(address p) {
  34
  35   // Below 32G, base must be aligned to 4G.
  36   // Above that point, base must be aligned to 32G
  37
  38   if (p < (address)(32 * G)) {
  39     return is_aligned(p, 4 * G);
  40   }
  41
  42   return is_aligned(p, 32 * G);
  43
  44 }

On line 42 I'd prefer to use (4 << LogKlassAlignmentInBytes)*G as it
currently is in metaspace.cpp instead of the literal 32. This makes the
relationship with the compressed class decode logic a bit clearer as the
restriction comes from the MOV and MOVK instructions we use to
decompress the pointer: we have to ensure the bits of the base and bits
of the offset after shifting do not overlap. Similarly for the
`increment` field in metaspace_aarch64.cpp line 51.


Thanks,
Nick

From aph at redhat.com  Wed Apr 29 08:36:03 2020
From: aph at redhat.com (Andrew Haley)
Date: Wed, 29 Apr 2020 09:36:03 +0100
Subject: [aarch64-port-dev ] RFR(M): 8243392: Remodel CDS/Metaspace
 storage reservation
In-Reply-To: <857dxyg2dl.fsf@arm.com>
References: <CAA-vtUyVki-DHepyAzORoTK_MwnrezFUB+QsxbWszztsx1x7NA@mail.gmail.com>
 <857dxyg2dl.fsf@arm.com>
Message-ID: <463e11c5-8c0d-3dad-060b-2a3d9e80fd40@redhat.com>

On 4/29/20 8:47 AM, Nick Gasson wrote:
> This makes the
> relationship with the compressed class decode logic a bit clearer as the
> restriction comes from the MOV and MOVK instructions we use to
> decompress the pointer: we have to ensure the bits of the base and bits
> of the offset after shifting do not overlap.

This seems a bit crazy. Whyever would anyone want shifted
CompressedKlassPointers with an offset? I guess I'm going to have to
look very closely at this patch.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From nick.gasson at arm.com  Wed Apr 29 08:51:23 2020
From: nick.gasson at arm.com (Nick Gasson)
Date: Wed, 29 Apr 2020 16:51:23 +0800
Subject: [aarch64-port-dev ] RFR(M): 8243392: Remodel CDS/Metaspace
 storage reservation
In-Reply-To: <463e11c5-8c0d-3dad-060b-2a3d9e80fd40@redhat.com>
References: <CAA-vtUyVki-DHepyAzORoTK_MwnrezFUB+QsxbWszztsx1x7NA@mail.gmail.com>
 <857dxyg2dl.fsf@arm.com> <463e11c5-8c0d-3dad-060b-2a3d9e80fd40@redhat.com>
Message-ID: <854kt2fzec.fsf@arm.com>


On 04/29/20 16:36 pm, Andrew Haley wrote:
> On 4/29/20 8:47 AM, Nick Gasson wrote:
>> This makes the
>> relationship with the compressed class decode logic a bit clearer as the
>> restriction comes from the MOV and MOVK instructions we use to
>> decompress the pointer: we have to ensure the bits of the base and bits
>> of the offset after shifting do not overlap.
>
> This seems a bit crazy. Whyever would anyone want shifted
> CompressedKlassPointers with an offset? I guess I'm going to have to
> look very closely at this patch.

The compressed class shift is always set to LogKlassAlignmentInBytes
when CDS is enabled. It's for compatibility with AOT. See this comment
in Metaspace::set_narrow_klass_base_and_shift():

  // CDS uses LogKlassAlignmentInBytes for narrow_klass_shift. See
  // MetaspaceShared::initialize_dumptime_shared_and_meta_spaces() for
  // how dump time narrow_klass_shift is set. Although, CDS can work
  // with zero-shift mode also, to be consistent with AOT it uses
  // LogKlassAlignmentInBytes for klass shift so archived java heap objects
  // can be used at same time as AOT code.

For AOT this is set up in AOTGraalHotSpotVMConfig.java:

  // AOT captures VM settings during compilation. For compressed oops this
  // presents a problem for the case when the VM selects a zero-shift mode
  // (i.e., when the heap is less than 4G). Compiling an AOT binary with
  // zero-shift limits its usability. As such we force the shift to be
  // always equal to alignment to avoid emitting zero-shift AOT code.
  CompressEncoding vmOopEncoding = super.getOopEncoding();
  aotOopEncoding = new CompressEncoding(vmOopEncoding.getBase(), logMinObjAlignment());
  CompressEncoding vmKlassEncoding = super.getKlassEncoding();
  aotKlassEncoding = new CompressEncoding(vmKlassEncoding.getBase(), logKlassAlignment);

For compressed OOPs it makes sense because it allows a larger heap
without changing the encoding mode. But for compressed class pointers we
never need to address more than 4G so maybe it's better to use 0 shift
instead of logKlassAlignment above? With CDS the default shared base
address is 0x80000000 which doesn't allow a zero base anyway.


Thanks,
Nick

From palacsint at palacsint.hu  Wed Apr 29 12:48:00 2020
From: palacsint at palacsint.hu (=?UTF-8?B?S2FyYWvDsyBNaWtsw7Nz?=)
Date: Wed, 29 Apr 2020 14:48:00 +0200
Subject: Thread-Local Handshakes slowdown on single-cpu machines
In-Reply-To: <5fb7355d-52ab-c8bb-1fdd-fd0b13a9de85@oracle.com>
References: <5e65ba58-b599-e83c-b4ea-3d766ed3c457@palacsint.hu>
 <66059982-49bf-1a15-39fb-22008b069573@oracle.com>
 <7216aa59-ad6f-dd3c-4f37-4723d3b7e9a7@palacsint.hu>
 <857ffb5e-d9e5-3976-3968-124e3b47effb@oracle.com>
 <f23d9ba4-3380-050a-d772-8a8a8eeb5b15@palacsint.hu>
 <46fd88cd-3e68-ebbf-869f-355ebb196146@oracle.com>
 <5fb7355d-52ab-c8bb-1fdd-fd0b13a9de85@oracle.com>
Message-ID: <c2c19fd7-84ce-3c39-3f2e-81a6c1879cf8@palacsint.hu>

Hello,

Wow, thank you everyone! Can we expect that the fix be released with JDK15?

Feel free to answer my Stack Overflow question 
(https://stackoverflow.com/q/61375565/843804) too for the bounty 
(expires in 2 days).

Best,
Miklos

On 4/29/20 9:45 AM, Robbin Ehn wrote:
> Hi,
>
> On 2020-04-29 07:31, David Holmes wrote:
>> On 29/04/2020 1:26 pm, Karak? Mikl?s wrote:
>>> Hi Patricio,
>>>
>>> Yes, it seems like a scheduling issue for us too. One of my 
>>> colleagues spotted the following sentence in Robbin Ehn's 
>>> Handshaking HotSpot presentation 
>>> (https://youtu.be/VBCOfAJ409s?t=2222): ~"if the VM thread finds that 
>>> all threads are unsafe, it keeps spinning"
>>> Could it be that actually the VM thread gets more CPU time and 
>>> actually blocks JavaThread?
>>
>
> At the moment handshake do not use yield or sleep.
> http://hg.openjdk.java.net/jdk/jdk/file/ddb12e77e8dd/src/hotspot/share/runtime/handshake.cpp 
>
> Do while loop L184 -> L201.
>
> I have tested at least three different approaches trying to ease off the
> CPU, all off them suffered from performance regression on MP machines.
> We have internally seen this on very rare occasions on MP (we do not
> have UPs). Since this have been so rare, not trivial to fix without
> performance regression and I was under the assumption that UP were
> basically dead :), it have not been prioritized.
>
> To just fix the UP case we could add something like:
> diff -r f7919010c7c2 src/hotspot/share/runtime/handshake.cpp
> --- a/src/hotspot/share/runtime/handshake.cpp?? Tue Apr 28 09:14:17 
> 2020 +0200
> +++ b/src/hotspot/share/runtime/handshake.cpp?? Wed Apr 29 09:36:42 
> 2020 +0200
> @@ -200,2 +200,5 @@
> ?????? }
> +????? if (!os::is_MP()) {
> +??????? os::naked_yield();
> +????? }
> ???? } while (!_op->is_completed());
>
> Since no thread can progress (assuming that one iteration is much
> shorter than a time-slice, this is extremely unlikely) there is no need
> for a second iteration until we executed something else.
>
> And should be trivial to backport to JDK 11.
>
> Thanks, Robbin


From aph at redhat.com  Wed Apr 29 12:56:10 2020
From: aph at redhat.com (Andrew Haley)
Date: Wed, 29 Apr 2020 13:56:10 +0100
Subject: [aarch64-port-dev ] RFR(M): 8243392: Remodel CDS/Metaspace
 storage reservation
In-Reply-To: <854kt2fzec.fsf@arm.com>
References: <CAA-vtUyVki-DHepyAzORoTK_MwnrezFUB+QsxbWszztsx1x7NA@mail.gmail.com>
 <857dxyg2dl.fsf@arm.com> <463e11c5-8c0d-3dad-060b-2a3d9e80fd40@redhat.com>
 <854kt2fzec.fsf@arm.com>
Message-ID: <25829763-7485-5208-58e8-1d51f7068816@redhat.com>

On 4/29/20 9:51 AM, Nick Gasson wrote:
> For compressed OOPs it makes sense because it allows a larger heap
> without changing the encoding mode.

That's right: I'm looking at AOT-compiled code (after applying your
patch) and by default it uses a shift of 3, no offset. If I then run
the AOT-compiled code with -Xmx31G I get:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x0000ffffa142bd3c, pid=9965, tid=10174
#
# JRE version:  (15.0) (slowdebug build )
# Java VM: OpenJDK 64-Bit Server VM (slowdebug 15-internal+0-adhoc.aph.jdk-tmp, mixed mode, aot, tiered, compressed oops, g1 gc, linux-aarch64)
# Problematic frame:
# A 388  java.lang.Thread.setPriority(I)V java.base (56 bytes) @ 0x0000ffffa142bd3c [0x0000ffffa142bac0+0x000000000000027c]

   0x0000ffffa142bd30 <+624>:	ldr	w1, [x4, #56]
   0x0000ffffa142bd34 <+628>:	cbz	w1, 0xffffa142bd84 <java.lang.Thread.setPriority(I)V+708>
   0x0000ffffa142bd38 <+632>:	lsl	x1, x1, #3
   0x0000ffffa142bd3c <+636>:	ldr	w0, [x1, #12]

... so the AOT-compiled code is still trying to use the shift of 3,
but it is not adding in the base, which is 0x1000000000. I guess this
is pilot error, but I'm trying to understand what gets checked and
when.

> But for compressed class pointers we never need to address more than
> 4G so maybe it's better to use 0 shift instead of logKlassAlignment
> above? With CDS the default shared base address is 0x80000000 which
> doesn't allow a zero base anyway.

Maybe. What actually happens when we decode compressed class pointers
in AOT-compiled code is:

Load the klass pointer from an Object:

  532440:       b940082a        ldr     w10, [x1,#8]

Load the compressed class base:

  532444:       90055e68        adrp    x8, b0fe000 <A.meta.got+0x16e000>
  532448:       9104c108        add     x8, x8, #0x130
  53244c:       f9400108        ldr     x8, [x8]

Shift and add:

  532450:       8b2a6d0a        add     x10, x8, x10, uxtx #3

... none of which is very nice, but the expensive part is loading the
compressed classw base and doing the add, so I guess we don't care
that there is a shift as well.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From david.holmes at oracle.com  Wed Apr 29 13:04:11 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 29 Apr 2020 23:04:11 +1000
Subject: Thread-Local Handshakes slowdown on single-cpu machines
In-Reply-To: <5fb7355d-52ab-c8bb-1fdd-fd0b13a9de85@oracle.com>
References: <5e65ba58-b599-e83c-b4ea-3d766ed3c457@palacsint.hu>
 <66059982-49bf-1a15-39fb-22008b069573@oracle.com>
 <7216aa59-ad6f-dd3c-4f37-4723d3b7e9a7@palacsint.hu>
 <857ffb5e-d9e5-3976-3968-124e3b47effb@oracle.com>
 <f23d9ba4-3380-050a-d772-8a8a8eeb5b15@palacsint.hu>
 <46fd88cd-3e68-ebbf-869f-355ebb196146@oracle.com>
 <5fb7355d-52ab-c8bb-1fdd-fd0b13a9de85@oracle.com>
Message-ID: <a18d569a-ac1a-1f19-dcef-35f2f0003541@oracle.com>

Hi Robbin,

On 29/04/2020 5:45 pm, Robbin Ehn wrote:
> Hi,
> 
> On 2020-04-29 07:31, David Holmes wrote:
>> On 29/04/2020 1:26 pm, Karak? Mikl?s wrote:
>>> Hi Patricio,
>>>
>>> Yes, it seems like a scheduling issue for us too. One of my 
>>> colleagues spotted the following sentence in Robbin Ehn's Handshaking 
>>> HotSpot presentation (https://youtu.be/VBCOfAJ409s?t=2222): ~"if the 
>>> VM thread finds that all threads are unsafe, it keeps spinning"
>>> Could it be that actually the VM thread gets more CPU time and 
>>> actually blocks JavaThread?
>>
> 
> At the moment handshake do not use yield or sleep.
> http://hg.openjdk.java.net/jdk/jdk/file/ddb12e77e8dd/src/hotspot/share/runtime/handshake.cpp 
> 
> Do while loop L184 -> L201.
> 
> I have tested at least three different approaches trying to ease off the
> CPU, all off them suffered from performance regression on MP machines.
> We have internally seen this on very rare occasions on MP (we do not
> have UPs). Since this have been so rare, not trivial to fix without
> performance regression and I was under the assumption that UP were
> basically dead :), it have not been prioritized.

So I have to wonder why we did not use the same VMThread back_off() 
strategy with handshakes?

Cheers,
David
-----

> To just fix the UP case we could add something like:
> diff -r f7919010c7c2 src/hotspot/share/runtime/handshake.cpp
> --- a/src/hotspot/share/runtime/handshake.cpp?? Tue Apr 28 09:14:17 2020 
> +0200
> +++ b/src/hotspot/share/runtime/handshake.cpp?? Wed Apr 29 09:36:42 2020 
> +0200
> @@ -200,2 +200,5 @@
>  ?????? }
> +????? if (!os::is_MP()) {
> +??????? os::naked_yield();
> +????? }
>  ???? } while (!_op->is_completed());
> 
> Since no thread can progress (assuming that one iteration is much
> shorter than a time-slice, this is extremely unlikely) there is no need
> for a second iteration until we executed something else.
> 
> And should be trivial to backport to JDK 11.
> 
> Thanks, Robbin
> 
> 
> 
>> The "spinning" is intended to be done in a scheduler-friendly way via:
>>
>> void os::naked_short_nanosleep(jlong ns) {
>> ?? assert(ns > -1 && ns < NANOUNITS, "Un-interruptable sleep, short 
>> time use only");
>>
>> ?? int64_t start = os::javaTimeNanos();
>> ?? do {
>> ???? if (SwitchToThread() == 0) {
>> ?????? // Nothing else is ready to run on this cpu, spin a little
>> ?????? SpinPause();
>> ???? }
>> ?? } while (os::javaTimeNanos() - start < ns);
>> }
>>
>> but this won't be that scheduler-friendly because it places the 
>> VMThread at the head of the priority queue, so it will cede up-to a 
>> timeslice to another thread, then run another iteration of the loop 
>> and hence alternate between a JavaThread and the VMThread until they 
>> are all blocked at the safepoint. What is really wanted is for the 
>> VMThread to go to the back of the scheduling queue.
>>
>> David
>> -----
>>
>>> I did not except that but I could reproduce the issue with a simple 
>>> hello world:
>>>
>>> public class Main {
>>> ???? public static void main(String[] args) {
>>> ???? ??? System.out.println("Hello world");
>>> ???? }
>>> }
>>>
>>> A few measurements on Windows 10 x64 in VirtualBox:
>>>
>>>
>>> PS Z:\main> Measure-Command { .\main-1cpu.bat }
>>>
>>> Minutes?????????? : 0
>>> Seconds?????????? : 7
>>> Milliseconds????? : 20
>>> Ticks???????????? : 70203455
>>> TotalMinutes????? : 0.117005758333333
>>> TotalSeconds????? : 7.0203455
>>> TotalMilliseconds : 7020.3455
>>>
>>>
>>>
>>> PS Z:\main> Measure-Command { .\main-full.bat }
>>>
>>> Minutes?????????? : 0
>>> Seconds?????????? : 1
>>> Milliseconds????? : 575
>>> Ticks???????????? : 15751352
>>> TotalSeconds????? : 1.5751352
>>> TotalMilliseconds : 1575.1352
>>>
>>>
>>>
>>> PS Z:\main> Measure-Command { .\main-full.bat }
>>>
>>> Minutes?????????? : 0
>>> Seconds?????????? : 1
>>> Milliseconds????? : 558
>>> Ticks???????????? : 15585384
>>> TotalSeconds????? : 1.5585384
>>> TotalMilliseconds : 1558.5384
>>>
>>>
>>>
>>> PS Z:\main> Measure-Command { .\main-1cpu.bat }
>>>
>>> Minutes?????????? : 0
>>> Seconds?????????? : 23
>>> Milliseconds????? : 648
>>> Ticks???????????? : 236482685
>>> TotalSeconds????? : 23.6482685
>>> TotalMilliseconds : 23648.2685
>>>
>>>
>>> main-1cpu.bat:
>>>
>>> c:\windows\system32\cmd.exe /C start /affinity 1 \jdk-14\bin\java 
>>> -Xlog:all=trace:file=app-1cpu.txt:uptime,tid,level,tags:filecount=50 
>>> Main
>>>
>>> main-full.bat:
>>>
>>> c:\windows\system32\cmd.exe /C start /affinity FF \jdk-14\bin\java 
>>> -Xlog:all=trace:file=app-full.txt:uptime,tid,level,tags:filecount=50 
>>> Main
>>>
>>> The files are also available here:
>>> https://palacsint.hu/etc/jdk/jdk14-single-cpu-slowdown-reprod-poc/
>>> Or in a zip:
>>> https://palacsint.hu/etc/jdk/jdk14-single-cpu-slowdown-reprod-poc.zip
>>>
>>> You can find trace level logs of three testruns in the run1-2-3 
>>> directories as well.
>>>
>>> Best,
>>> Miklos
>>>
>>>
>>> On 4/29/20 2:18 AM, Patricio Chilano wrote:
>>>> Hi Miklos,
>>>>
>>>> On 4/28/20 8:04 AM, Karak? Mikl?s wrote:
>>>>> Hello David,
>>>>>
>>>>> I've tried with OpenJDK 15 EA Build 20. Unfortunately it's slow as 
>>>>> with OpenJDK14.
>>>>> A colleague of mine set CPU affinity with taskset to only one CPU 
>>>>> on Linux and did not experience the slowdown. So the issue seems 
>>>>> Windows-related.
>>>> I wouldn?t rule out this being a scheduling issue. With thread-local 
>>>> handshakes some operations like the ones you see in your logs 
>>>> (deoptimization, biasedlocking revocation) do not need all 
>>>> JavaThreads to be stopped at the same time in order to be executed. 
>>>> If you only have one CPU but many JavaThreads running, depending on 
>>>> the scheduler, maybe it could happen that the JavaThread you need to 
>>>> handshake is constantly being left behind in the scheduler queue and 
>>>> so the operation takes longer to execute. When you add 
>>>> -XX:-ThreadLocalHandshakes, those operations will be executed using 
>>>> safepoints, which means all JavaThreads are stopped. So even if the 
>>>> scheduler decides to give priority to other JavaThreads those will 
>>>> block and free the CPU. Also when adding more CPU that will increase 
>>>> the likelihood of a JavaThread getting scheduled and might explain 
>>>> why you see it gets fixed.
>>>>
>>>> Alternatively the JavaThread that needs to be handshaked is being 
>>>> scheduled but is not polling for the handshake. But I think in that 
>>>> the case you should still have the same issue with 
>>>> -XX:-ThreadLocalHandshakes because the polling mechanism is the 
>>>> same, although it's true that the misbehaving JavaThread will have 
>>>> more CPU time to complete whatever is doing while others are stopped 
>>>> when safepointing.
>>>>
>>>> One thing that makes me question the scheduler theory is the fact 
>>>> that you see such improvements when only disabling biased-locking in 
>>>> 13.0.2 because that was still using safepoints back then. Can you 
>>>> check the logs to see in which operations you see the pauses when 
>>>> running with default options compared to -XX:-UseBiasedLocking?
>>>>
>>>> I?m also surprised you see those long logging pauses because all the 
>>>> other JavaThreads were already processed in the handshake so 
>>>> something must be executing. Maybe you can switch the logging to 
>>>> trace instead of debug. Also maybe you can add logging at Java code 
>>>> so you can follow what is actually running? You could send it to the 
>>>> same app.txt and flush every time you write to see it ordered in the 
>>>> log. You can also get the uptime you see in the UL logs with 
>>>> ManagementFactory.getRuntimeMXBean().getUptime(). That logging will 
>>>> affect timing but might give some insights.
>>>>
>>>> Thanks,
>>>> Patricio
>>>>> Miklos
>>>>>
>>>>> On 4/28/20 1:02 AM, David Holmes wrote:
>>>>>> Hi Miklos,
>>>>>>
>>>>>> On 28/04/2020 7:26 am, Karak? Mikl?s wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> We bumped into a possible Thread-Local Handshakes issue with 
>>>>>>> multiple apps. It seems that enabled TLH slows down applications 
>>>>>>> running on (although rare) single CPU boxes. I would be grateful 
>>>>>>> if you could confirm that this is a known trade-off for these 
>>>>>>> rare setups or a possible JVM bug. That would save us at least a 
>>>>>>> few hours of debugging.
>>>>>>>
>>>>>>> TL;DR: Both tested apps stop frequently around 
>>>>>>> "HandshakeOneThread", "HandshakeAllThreads" and "Revoked bias of 
>>>>>>> currently-unlocked object" running with 
>>>>>>> OpenJDK12/OpenJDK13/OpenJDK14. OpenJDK 13 with 
>>>>>>> -XX:-ThreadLocalHandshakes fixes the issue. Adding a second CPU 
>>>>>>> to the virtual machine fixes the issue. Enabling hyper-threading 
>>>>>>> fixes the issue.
>>>>>>
>>>>>> Can you try with latest JDK 15 build, just to see if this may be 
>>>>>> something already addressed?
>>>>>>
>>>>>> Thanks,
>>>>>> David
>>>>>>
>>>>>>> More details are available at StackOverflow: 
>>>>>>> https://stackoverflow.com/questions/61375565/slow-application-frequent-jvm-hangs-with-single-cpu-setups-and-java-12 
>>>>>>>
>>>>>>>
>>>>>>> All thoughts are welcome.
>>>>>>>
>>>>>>> Best,
>>>>>>> Miklos
>>>>>>>
>>>>>
>>>>
>>>

From robbin.ehn at oracle.com  Wed Apr 29 13:32:18 2020
From: robbin.ehn at oracle.com (Robbin Ehn)
Date: Wed, 29 Apr 2020 15:32:18 +0200
Subject: Thread-Local Handshakes slowdown on single-cpu machines
In-Reply-To: <a18d569a-ac1a-1f19-dcef-35f2f0003541@oracle.com>
References: <5e65ba58-b599-e83c-b4ea-3d766ed3c457@palacsint.hu>
 <66059982-49bf-1a15-39fb-22008b069573@oracle.com>
 <7216aa59-ad6f-dd3c-4f37-4723d3b7e9a7@palacsint.hu>
 <857ffb5e-d9e5-3976-3968-124e3b47effb@oracle.com>
 <f23d9ba4-3380-050a-d772-8a8a8eeb5b15@palacsint.hu>
 <46fd88cd-3e68-ebbf-869f-355ebb196146@oracle.com>
 <5fb7355d-52ab-c8bb-1fdd-fd0b13a9de85@oracle.com>
 <a18d569a-ac1a-1f19-dcef-35f2f0003541@oracle.com>
Message-ID: <f6ce7bcc-bff3-bbe5-63b9-324fbb60530b@oracle.com>

Hi David,

>> I have tested at least three different approaches trying to ease off the
>> CPU, all off them suffered from performance regression on MP machines.
>> We have internally seen this on very rare occasions on MP (we do not
>> have UPs). Since this have been so rare, not trivial to fix without
>> performance regression and I was under the assumption that UP were
>> basically dead :), it have not been prioritized.
> 
> So I have to wonder why we did not use the same VMThread back_off() 
> strategy with handshakes?

That is one of the variants I have tested.
I think the main problem is that we do not consider if we did any 
progress or not. Also a big difference is if VM thread backs off the CPU
during a handshake, a thread that have already completed the handshake
may run instead of the VM thread.

I'm coding a draft of an algorithm that takes into account if we did any
progress or not. That involves knowning for each thread why the
operation have not been executed. So any backporting will not be
trivial.

 From what I recall the performance regression was mostly startup.
In startup +- BiasedLocking and early deoptimize play major roll if and
how many handshakes we need to perform.

I'll create a jira issue for this.

What you think about just fixing UP first with something simple like below?

Btw I think code should be:
if (!os::is_MP() && !_op->is_completed()) {
   os::naked_yield();
}
And it's also needed one single target thread case.

Thanks, Robbin


> 
> Cheers,
> David
> -----
> 
>> To just fix the UP case we could add something like:
>> diff -r f7919010c7c2 src/hotspot/share/runtime/handshake.cpp
>> --- a/src/hotspot/share/runtime/handshake.cpp?? Tue Apr 28 09:14:17 
>> 2020 +0200
>> +++ b/src/hotspot/share/runtime/handshake.cpp?? Wed Apr 29 09:36:42 
>> 2020 +0200
>> @@ -200,2 +200,5 @@
>> ??????? }
>> +????? if (!os::is_MP()) {
>> +??????? os::naked_yield();
>> +????? }
>> ????? } while (!_op->is_completed());
>>
>> Since no thread can progress (assuming that one iteration is much
>> shorter than a time-slice, this is extremely unlikely) there is no need
>> for a second iteration until we executed something else.
>>
>> And should be trivial to backport to JDK 11.
>>
>> Thanks, Robbin
>>
>>
>>
>>> The "spinning" is intended to be done in a scheduler-friendly way via:
>>>
>>> void os::naked_short_nanosleep(jlong ns) {
>>> ?? assert(ns > -1 && ns < NANOUNITS, "Un-interruptable sleep, short 
>>> time use only");
>>>
>>> ?? int64_t start = os::javaTimeNanos();
>>> ?? do {
>>> ???? if (SwitchToThread() == 0) {
>>> ?????? // Nothing else is ready to run on this cpu, spin a little
>>> ?????? SpinPause();
>>> ???? }
>>> ?? } while (os::javaTimeNanos() - start < ns);
>>> }
>>>
>>> but this won't be that scheduler-friendly because it places the 
>>> VMThread at the head of the priority queue, so it will cede up-to a 
>>> timeslice to another thread, then run another iteration of the loop 
>>> and hence alternate between a JavaThread and the VMThread until they 
>>> are all blocked at the safepoint. What is really wanted is for the 
>>> VMThread to go to the back of the scheduling queue.
>>>
>>> David
>>> -----
>>>
>>>> I did not except that but I could reproduce the issue with a simple 
>>>> hello world:
>>>>
>>>> public class Main {
>>>> ???? public static void main(String[] args) {
>>>> ???? ??? System.out.println("Hello world");
>>>> ???? }
>>>> }
>>>>
>>>> A few measurements on Windows 10 x64 in VirtualBox:
>>>>
>>>>
>>>> PS Z:\main> Measure-Command { .\main-1cpu.bat }
>>>>
>>>> Minutes?????????? : 0
>>>> Seconds?????????? : 7
>>>> Milliseconds????? : 20
>>>> Ticks???????????? : 70203455
>>>> TotalMinutes????? : 0.117005758333333
>>>> TotalSeconds????? : 7.0203455
>>>> TotalMilliseconds : 7020.3455
>>>>
>>>>
>>>>
>>>> PS Z:\main> Measure-Command { .\main-full.bat }
>>>>
>>>> Minutes?????????? : 0
>>>> Seconds?????????? : 1
>>>> Milliseconds????? : 575
>>>> Ticks???????????? : 15751352
>>>> TotalSeconds????? : 1.5751352
>>>> TotalMilliseconds : 1575.1352
>>>>
>>>>
>>>>
>>>> PS Z:\main> Measure-Command { .\main-full.bat }
>>>>
>>>> Minutes?????????? : 0
>>>> Seconds?????????? : 1
>>>> Milliseconds????? : 558
>>>> Ticks???????????? : 15585384
>>>> TotalSeconds????? : 1.5585384
>>>> TotalMilliseconds : 1558.5384
>>>>
>>>>
>>>>
>>>> PS Z:\main> Measure-Command { .\main-1cpu.bat }
>>>>
>>>> Minutes?????????? : 0
>>>> Seconds?????????? : 23
>>>> Milliseconds????? : 648
>>>> Ticks???????????? : 236482685
>>>> TotalSeconds????? : 23.6482685
>>>> TotalMilliseconds : 23648.2685
>>>>
>>>>
>>>> main-1cpu.bat:
>>>>
>>>> c:\windows\system32\cmd.exe /C start /affinity 1 \jdk-14\bin\java 
>>>> -Xlog:all=trace:file=app-1cpu.txt:uptime,tid,level,tags:filecount=50 
>>>> Main
>>>>
>>>> main-full.bat:
>>>>
>>>> c:\windows\system32\cmd.exe /C start /affinity FF \jdk-14\bin\java 
>>>> -Xlog:all=trace:file=app-full.txt:uptime,tid,level,tags:filecount=50 
>>>> Main
>>>>
>>>> The files are also available here:
>>>> https://palacsint.hu/etc/jdk/jdk14-single-cpu-slowdown-reprod-poc/
>>>> Or in a zip:
>>>> https://palacsint.hu/etc/jdk/jdk14-single-cpu-slowdown-reprod-poc.zip
>>>>
>>>> You can find trace level logs of three testruns in the run1-2-3 
>>>> directories as well.
>>>>
>>>> Best,
>>>> Miklos
>>>>
>>>>
>>>> On 4/29/20 2:18 AM, Patricio Chilano wrote:
>>>>> Hi Miklos,
>>>>>
>>>>> On 4/28/20 8:04 AM, Karak? Mikl?s wrote:
>>>>>> Hello David,
>>>>>>
>>>>>> I've tried with OpenJDK 15 EA Build 20. Unfortunately it's slow as 
>>>>>> with OpenJDK14.
>>>>>> A colleague of mine set CPU affinity with taskset to only one CPU 
>>>>>> on Linux and did not experience the slowdown. So the issue seems 
>>>>>> Windows-related.
>>>>> I wouldn?t rule out this being a scheduling issue. With 
>>>>> thread-local handshakes some operations like the ones you see in 
>>>>> your logs (deoptimization, biasedlocking revocation) do not need 
>>>>> all JavaThreads to be stopped at the same time in order to be 
>>>>> executed. If you only have one CPU but many JavaThreads running, 
>>>>> depending on the scheduler, maybe it could happen that the 
>>>>> JavaThread you need to handshake is constantly being left behind in 
>>>>> the scheduler queue and so the operation takes longer to execute. 
>>>>> When you add -XX:-ThreadLocalHandshakes, those operations will be 
>>>>> executed using safepoints, which means all JavaThreads are stopped. 
>>>>> So even if the scheduler decides to give priority to other 
>>>>> JavaThreads those will block and free the CPU. Also when adding 
>>>>> more CPU that will increase the likelihood of a JavaThread getting 
>>>>> scheduled and might explain why you see it gets fixed.
>>>>>
>>>>> Alternatively the JavaThread that needs to be handshaked is being 
>>>>> scheduled but is not polling for the handshake. But I think in that 
>>>>> the case you should still have the same issue with 
>>>>> -XX:-ThreadLocalHandshakes because the polling mechanism is the 
>>>>> same, although it's true that the misbehaving JavaThread will have 
>>>>> more CPU time to complete whatever is doing while others are 
>>>>> stopped when safepointing.
>>>>>
>>>>> One thing that makes me question the scheduler theory is the fact 
>>>>> that you see such improvements when only disabling biased-locking 
>>>>> in 13.0.2 because that was still using safepoints back then. Can 
>>>>> you check the logs to see in which operations you see the pauses 
>>>>> when running with default options compared to -XX:-UseBiasedLocking?
>>>>>
>>>>> I?m also surprised you see those long logging pauses because all 
>>>>> the other JavaThreads were already processed in the handshake so 
>>>>> something must be executing. Maybe you can switch the logging to 
>>>>> trace instead of debug. Also maybe you can add logging at Java code 
>>>>> so you can follow what is actually running? You could send it to 
>>>>> the same app.txt and flush every time you write to see it ordered 
>>>>> in the log. You can also get the uptime you see in the UL logs with 
>>>>> ManagementFactory.getRuntimeMXBean().getUptime(). That logging will 
>>>>> affect timing but might give some insights.
>>>>>
>>>>> Thanks,
>>>>> Patricio
>>>>>> Miklos
>>>>>>
>>>>>> On 4/28/20 1:02 AM, David Holmes wrote:
>>>>>>> Hi Miklos,
>>>>>>>
>>>>>>> On 28/04/2020 7:26 am, Karak? Mikl?s wrote:
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> We bumped into a possible Thread-Local Handshakes issue with 
>>>>>>>> multiple apps. It seems that enabled TLH slows down applications 
>>>>>>>> running on (although rare) single CPU boxes. I would be grateful 
>>>>>>>> if you could confirm that this is a known trade-off for these 
>>>>>>>> rare setups or a possible JVM bug. That would save us at least a 
>>>>>>>> few hours of debugging.
>>>>>>>>
>>>>>>>> TL;DR: Both tested apps stop frequently around 
>>>>>>>> "HandshakeOneThread", "HandshakeAllThreads" and "Revoked bias of 
>>>>>>>> currently-unlocked object" running with 
>>>>>>>> OpenJDK12/OpenJDK13/OpenJDK14. OpenJDK 13 with 
>>>>>>>> -XX:-ThreadLocalHandshakes fixes the issue. Adding a second CPU 
>>>>>>>> to the virtual machine fixes the issue. Enabling hyper-threading 
>>>>>>>> fixes the issue.
>>>>>>>
>>>>>>> Can you try with latest JDK 15 build, just to see if this may be 
>>>>>>> something already addressed?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> David
>>>>>>>
>>>>>>>> More details are available at StackOverflow: 
>>>>>>>> https://stackoverflow.com/questions/61375565/slow-application-frequent-jvm-hangs-with-single-cpu-setups-and-java-12 
>>>>>>>>
>>>>>>>>
>>>>>>>> All thoughts are welcome.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Miklos
>>>>>>>>
>>>>>>
>>>>>
>>>>

From aph at redhat.com  Wed Apr 29 13:32:56 2020
From: aph at redhat.com (Andrew Haley)
Date: Wed, 29 Apr 2020 14:32:56 +0100
Subject: [aarch64-port-dev ] RFR(M): 8243392: Remodel CDS/Metaspace
 storage reservation
In-Reply-To: <25829763-7485-5208-58e8-1d51f7068816@redhat.com>
References: <CAA-vtUyVki-DHepyAzORoTK_MwnrezFUB+QsxbWszztsx1x7NA@mail.gmail.com>
 <857dxyg2dl.fsf@arm.com> <463e11c5-8c0d-3dad-060b-2a3d9e80fd40@redhat.com>
 <854kt2fzec.fsf@arm.com> <25829763-7485-5208-58e8-1d51f7068816@redhat.com>
Message-ID: <040930ce-2008-455f-4427-82c2795492c7@redhat.com>

On 4/29/20 1:56 PM, Andrew Haley wrote:
> Maybe. What actually happens when we decode compressed class pointers
> in AOT-compiled code is:
> 
> Load the klass pointer from an Object:
> 
>   532440:       b940082a        ldr     w10, [x1,#8]
> 
> Load the compressed class base:
> 
>   532444:       90055e68        adrp    x8, b0fe000 <A.meta.got+0x16e000>
>   532448:       9104c108        add     x8, x8, #0x130
>   53244c:       f9400108        ldr     x8, [x8]
> 
> Shift and add:
> 
>   532450:       8b2a6d0a        add     x10, x8, x10, uxtx #3
> 
> ... none of which is very nice, but the expensive part is loading the
> compressed class base and doing the add, so I guess we don't care
> that there is a shift as well.

Argh. s/compressed class/compressed oop/g

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From aph at redhat.com  Wed Apr 29 14:21:44 2020
From: aph at redhat.com (Andrew Haley)
Date: Wed, 29 Apr 2020 15:21:44 +0100
Subject: [aarch64-port-dev ] RFR(M): 8243392: Remodel CDS/Metaspace
 storage reservation
In-Reply-To: <CAA-vtUyVki-DHepyAzORoTK_MwnrezFUB+QsxbWszztsx1x7NA@mail.gmail.com>
References: <CAA-vtUyVki-DHepyAzORoTK_MwnrezFUB+QsxbWszztsx1x7NA@mail.gmail.com>
Message-ID: <d6a94952-4d45-9fc3-2b45-f0ba46fda0c5@redhat.com>

On 4/28/20 3:54 PM, Thomas St?fe wrote:
> These two methods should give the platform enough control to implement
> their own scheme for optimized class space placement without bothering any
> shared code about it.

There's still something I don't like. If we have a compressed class space
in the lower 32G but above 4G, we do this:

    // Otherwise we attempt to use a zero base if the range fits in lower 32G.
    if (end <= (address)ClassEncodingMetaspaceMax) {
      base = 0;
    } else {
      base = addr;
    }

    // Highest offset a Class* can ever have in relation to base.
    range = end - base;

    // We may not even need a shift if the range fits into 32bit:
    const uint64_t UnscaledClassSpaceMax = (uint64_t(max_juint) + 1);
    if (range < UnscaledClassSpaceMax) {
      shift = 0;
    } else {
      shift = LogClassAlignmentInBytes;
    }

... which means that we end up with zero base, shifted compressed
class pointers, *despite the fact* that we carefully chose a
nicely-aligned compressed class base we could encode efficiently.

I guess this code above is really optimized for x86; it certainly
seems to prefer shifts to offsets, which makes sense on that part. It
doesn't make much difference (if any) for AArch64, I admit, but it is
odd.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From thomas.stuefe at gmail.com  Wed Apr 29 16:14:30 2020
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Wed, 29 Apr 2020 18:14:30 +0200
Subject: [aarch64-port-dev ] RFR(M): 8243392: Remodel CDS/Metaspace
 storage reservation
In-Reply-To: <d6a94952-4d45-9fc3-2b45-f0ba46fda0c5@redhat.com>
References: <CAA-vtUyVki-DHepyAzORoTK_MwnrezFUB+QsxbWszztsx1x7NA@mail.gmail.com>
 <d6a94952-4d45-9fc3-2b45-f0ba46fda0c5@redhat.com>
Message-ID: <CAA-vtUwKc57J=xinDrES0tES+8eYD+nTzjqaqFyxz3E3fzMtVg@mail.gmail.com>

Hi Andrew,

On Wed, Apr 29, 2020 at 4:21 PM Andrew Haley <aph at redhat.com> wrote:

> On 4/28/20 3:54 PM, Thomas St?fe wrote:
> > These two methods should give the platform enough control to implement
> > their own scheme for optimized class space placement without bothering
> any
> > shared code about it.
>
> There's still something I don't like. If we have a compressed class space
> in the lower 32G but above 4G, we do this:
>
>     // Otherwise we attempt to use a zero base if the range fits in lower
> 32G.
>     if (end <= (address)ClassEncodingMetaspaceMax) {
>       base = 0;
>     } else {
>       base = addr;
>     }
>
>     // Highest offset a Class* can ever have in relation to base.
>     range = end - base;
>
>     // We may not even need a shift if the range fits into 32bit:
>     const uint64_t UnscaledClassSpaceMax = (uint64_t(max_juint) + 1);
>     if (range < UnscaledClassSpaceMax) {
>       shift = 0;
>     } else {
>       shift = LogClassAlignmentInBytes;
>     }
>
> ... which means that we end up with zero base, shifted compressed
> class pointers, *despite the fact* that we carefully chose a
> nicely-aligned compressed class base we could encode efficiently.
>
> I guess this code above is really optimized for x86; it certainly
> seems to prefer shifts to offsets, which makes sense on that part. It
> doesn't make much difference (if any) for AArch64, I admit, but it is
> odd.
>
>
I understand.

First off, this patch is supposed to be an almost clean code reshuffle. It
was not the intent of this patch to change functionality, at least not by
much, since the patch is complicated enough as it is. It just wants to
improve maintainability, so that we can improve the code in the future
easier. So, compressed class pointer encoding should work as it did before,
modulo those little details about CompressedKlassPointers::range.

That said, I agree with you, this is not optimal. There are other
possibilities to improve matters, e.g. we miss opportunities to go zero
based if the heap is very large since the class space gets always allocated
behind the heap.

Since the intent of this code is to give platforms greater leeway to do
their thing without disturbing shared code, maybe we should make
CompressedKlassPointers::initialize() platform dependent too? Or, add a
hook at the end of it allowing platforms to overwrite the default behavior.
If aarch64 prefers shift=0 and base=<nice base address> it could do so.

..Thomas


-- 
> Andrew Haley  (he/him)
> Java Platform Lead Engineer
> Red Hat UK Ltd. <https://www.redhat.com>
> https://keybase.io/andrewhaley
> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671
>
>

From thomas.stuefe at gmail.com  Wed Apr 29 16:16:24 2020
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Wed, 29 Apr 2020 18:16:24 +0200
Subject: [aarch64-port-dev ] RFR(M): 8243392: Remodel CDS/Metaspace
 storage reservation
In-Reply-To: <857dxyg2dl.fsf@arm.com>
References: <CAA-vtUyVki-DHepyAzORoTK_MwnrezFUB+QsxbWszztsx1x7NA@mail.gmail.com>
 <857dxyg2dl.fsf@arm.com>
Message-ID: <CAA-vtUxguP9_1QF7wEn7qAhQF6UZ9srP0krY0_mFHq=yjrJ-og@mail.gmail.com>

Thank you Nick. You are of course right, I should use speaking constants
instead. I will change that.

..Thomas

On Wed, Apr 29, 2020 at 10:14 AM Nick Gasson <nick.gasson at arm.com> wrote:

> Hi Thomas,
>
> On 04/28/20 22:54 pm, Thomas St?fe wrote:
> >
> > These are two new methods:
> >
> > - bool CompressedKlassPointers::is_valid_base(address p)
> >
> > to let the platform tell you whether it considers p to be a valid
> encoding
> > base. The only platform having these restrictions currently is aarch64.
> >
> > - ReservedSpace
> > Metaspace::reserve_address_space_for_compressed_classes(size_t size);
> >
> > this hands over the process of allocating a range suitable for compressed
> > class pointer encoding to the platform. Most platforms will allocate just
> > anywhere, but some platforms may have a better strategy (e.g. trying low
> > memory first, trying only correctly aligned addresses and so on).
> >
> > Beforehand, this coding existed in a similar form in metaspace.cpp for
> > aarch64 and AIX. For now, I left the AIX part out - it seems only half
> > done, and I want to check further if we even need it, if yes why not on
> > Linux ppc, and C1 does not seem to support anything other than
> base+offset
> > with shift either, but I may be mistaken.
>
> Just a small comment:
>
>   33 bool CompressedKlassPointers::is_valid_base(address p) {
>   34
>   35   // Below 32G, base must be aligned to 4G.
>   36   // Above that point, base must be aligned to 32G
>   37
>   38   if (p < (address)(32 * G)) {
>   39     return is_aligned(p, 4 * G);
>   40   }
>   41
>   42   return is_aligned(p, 32 * G);
>   43
>   44 }
>
> On line 42 I'd prefer to use (4 << LogKlassAlignmentInBytes)*G as it
> currently is in metaspace.cpp instead of the literal 32. This makes the
> relationship with the compressed class decode logic a bit clearer as the
> restriction comes from the MOV and MOVK instructions we use to
> decompress the pointer: we have to ensure the bits of the base and bits
> of the offset after shifting do not overlap. Similarly for the
> `increment` field in metaspace_aarch64.cpp line 51.
>
>
> Thanks,
> Nick
>

From frederic.parain at oracle.com  Wed Apr 29 17:15:02 2020
From: frederic.parain at oracle.com (Frederic Parain)
Date: Wed, 29 Apr 2020 13:15:02 -0400
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64 (closed)
In-Reply-To: <728ae6da-8c71-d26c-13a9-7b6e9816c946@oracle.com>
References: <9fd23c76-900b-bf6a-3411-64955bd4261c@oracle.com>
 <C92914CE-B996-44FF-B13D-3E1189C349D1@oracle.com>
 <728ae6da-8c71-d26c-13a9-7b6e9816c946@oracle.com>
Message-ID: <6D696686-EBAB-4FAF-A590-499430184065@oracle.com>


Looks good to me.
My comment about testing was related to the follow-up patch to remove hard coded
offsets, but Coleen told me we have good coverage about the different initialization
paths.

Thank you,

Fred


> On Apr 28, 2020, at 19:19, coleen.phillimore at oracle.com wrote:
> 
> 
> 
> On 4/28/20 4:30 PM, Erik ?sterlund wrote:
>> Hi Coleen,
>> 
>> Unfortunately I do need that change already now. Because making compressed oops and compressed class pointers independent involves changing the instanceOop base with compressed oop off and compressed class pointers off. That changes the layout and offsets that are hardcoded (primitive wrappers that wrap a primitive less than 4 bytes now start in the klass gap).
> 
> Okay, I didn't see how it related from the diffs.  Now I get it. Your patch is good then.  Thanks for removing the hardcoded offsets next.
> 
> Coleen
> 
>> 
>> My proposed solution is to remove hardcoded layouts in this patch, and remove hardcoded offsets in the next one. The alternative would be trying to fix the hardcoded layouts only to remove them one patch later. I tried to fix them but ran into issues; that is why I removed them instead in this patch. I hope you think this is okay. The net result will be the same one patch later.
>> 
>> Thanks,
>> /Erik
>> 
>>> On 28 Apr 2020, at 21:56, coleen.phillimore at oracle.com wrote:
>>> 
>>> ?
>>> Hi Erik,
>>> 
>>> I was trying to figure out why this change is needed to decouple CompressedOops and CompressedKlassPointers, but it's not needed for the change, is it?
>>> 
>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.01/src/hotspot/share/classfile/fieldLayoutBuilder.cpp.udiff.html
>>> 
>>> Can you make this change with your patch for https://bugs.openjdk.java.net/browse/JDK-8243996 which we really like?
>>> 
>>> Thanks,
>>> Coleen
>>> 
>>> 
>>> On 4/28/20 1:52 PM, Erik ?sterlund wrote:
>>>> Hi Coleen,
>>>> 
>>>> Oops, typo in the incremental URL. Here it is:
>>>> 
>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00..01/
>>>> 
>>>> Thanks for the review.
>>>> 
>>>> /Erik
>>>> 
>>>>> On 28 Apr 2020, at 19:25, coleen.phillimore at oracle.com wrote:
>>>>> 
>>>>> ? Okay.  Your incremental is 4040 but looks good to me.
>>>>> Thanks,
>>>>> Coleen
>>>>> 
>>>>> On 4/28/20 12:30 PM, Erik ?sterlund wrote:
>>>>>> Hi Coleen,
>>>>>> 
>>>>>> Thanks for the review.
>>>>>> 
>>>>>> Here is a new webrev:
>>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.01/
>>>>>> 
>>>>>> Incremental:
>>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00_01/
>>>>>> 
>>>>>> On 2020-04-28 16:15, coleen.phillimore at oracle.com wrote:
>>>>>>> Sorry, this went to the wrong list.
>>>>>>> 
>>>>>>> Coleen
>>>>>>> 
>>>>>>> On 4/24/20 11:17 AM, coleen.phillimore at oracle.comwrote:
>>>>>>>> 
>>>>>>>> Hi Erik, well the interpreter code isn't as bad as I thought it would be.  We tied UseCompressedKlassPointers to UseCompressedOops because we couldn't imagine why someone would care about 4 bytes per object with all 8 byte pointers to oops.  But maybe you do care.
>>>>>>>> 
>>>>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/cpu/x86/interp_masm_x86.cpp.udiff.html
>>>>>>>> 
>>>>>>>>       if (UseBiasedLocking) {
>>>>>>>> - biased_locking_enter(lock_reg, obj_reg, swap_reg, tmp_reg, false, done, &slow_case);
>>>>>>>> + Register rklass_decode_tmp = LP64_ONLY(rscratch1) NOT_LP64(noreg);
>>>>>>>> + biased_locking_enter(lock_reg, obj_reg, swap_reg, tmp_reg, rscratch1, false, done, &slow_case);
>>>>>>>>       }
>>>>>>>> You didn't use rklass_decode_tmp here.
>>>>>> 
>>>>>> Nice catch. Fixed.
>>>>>> 
>>>>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/cpu/x86/templateTable_x86.cpp.udiff.html
>>>>>>>> 
>>>>>>>> 
>>>>>>>> + Register tmp_load_klass = LP64_ONLY(rscratch1) NOT_LP64(noreg);
>>>>>>>> 
>>>>>>>> I wonder if you couldn't just pass rscratch1 into load_klass and handle ignoring it inside the load_klass function, ie. fix the asserts.  This gets repetitive in the templateTable and other code.  Maybe people may have the opposite opinion though.
>>>>>> 
>>>>>> Unfortunately, that won't compile in code that is shared between 32 bit and 64 bit. In 32 bit-only code I just pass in noreg, and in 64-bit only code I just send in the temp. But for shared code, I have to do the macro dance, because rscratch1 does not exist on 32 bit.
>>>>>> 
>>>>>>>> This is quite nice that UseCompressedKlassPointers doesn't have to reinit_heapbase.  The interpreter code looks good.
>>>>>> 
>>>>>> Thanks!
>>>>>> 
>>>>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/share/memory/metaspace.cpp.frames.html
>>>>>>>> 
>>>>>>>> I don't know if base = HeapBaseMinAddress is right for the compressed class space.  It's reserved but not committed as 1G.  Maybe it makes sense since the heap without UseCompressedOops should be above the class space.
>>>>>> 
>>>>>> Thomas had a similar comment. I think the summary is that yes it is the right thing to use, but the naming is unfortunate.
>>>>>> 
>>>>>> Hope I fixed it all.
>>>>>> 
>>>>>> Thanks,
>>>>>> /Erik
>>>>>> 
>>>>>>>> Coleen
>>>>>>>> 
>>>>>>>> On 4/24/20 4:25 AM, Erik ?sterlund wrote:
>>>>>>>>> Hi,
>>>>>>>>> 
>>>>>>>>> This is the closed version of the mostly open change. There are some @Contended tests in closed
>>>>>>>>> that needed some love with my change to make compressed class pointers usable with compressed
>>>>>>>>> oops disabled. In particular, these tests perform calculations to ensure that things are padded
>>>>>>>>> as expected. Part of those calculations makes assumptions about how large the object header is,
>>>>>>>>> presuming that the number is a constant that is the same for all objects, and is determinable
>>>>>>>>> by checking the offset of the first field. However, with my changes, the first field will end
>>>>>>>>> up either in the klass gap, or after, depending on the field size. I updated the tests to consider
>>>>>>>>> different header sizes depending on the type of fields being used. It still isn't perfect, but
>>>>>>>>> it is clever enough for the purposes of the tests; they pass with these changes.
>>>>>>>>> 
>>>>>>>>> Bug:
>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8241825
>>>>>>>>> 
>>>>>>>>> Webrev:
>>>>>>>>> http://wikifiles.se.oracle.com/dev/eosterlund/8241825/webrev.00/
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> /Erik
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>> 
> 


From erik.osterlund at oracle.com  Wed Apr 29 18:15:52 2020
From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=)
Date: Wed, 29 Apr 2020 20:15:52 +0200
Subject: RFR: 8241825: Make compressed oops and compressed class pointers
 independent on x86_64 (closed)
In-Reply-To: <6D696686-EBAB-4FAF-A590-499430184065@oracle.com>
References: <6D696686-EBAB-4FAF-A590-499430184065@oracle.com>
Message-ID: <BB0F3014-B904-4C1C-8D04-060B0264039B@oracle.com>

Hi Frederic,

Thanks for the review.

/Erik

> On 29 Apr 2020, at 19:15, Frederic Parain <frederic.parain at oracle.com> wrote:
> 
> ?
> Looks good to me.
> My comment about testing was related to the follow-up patch to remove hard coded
> offsets, but Coleen told me we have good coverage about the different initialization
> paths.
> 
> Thank you,
> 
> Fred
> 
> 
>> On Apr 28, 2020, at 19:19, coleen.phillimore at oracle.com wrote:
>> 
>> 
>> 
>>> On 4/28/20 4:30 PM, Erik ?sterlund wrote:
>>> Hi Coleen,
>>> 
>>> Unfortunately I do need that change already now. Because making compressed oops and compressed class pointers independent involves changing the instanceOop base with compressed oop off and compressed class pointers off. That changes the layout and offsets that are hardcoded (primitive wrappers that wrap a primitive less than 4 bytes now start in the klass gap).
>> 
>> Okay, I didn't see how it related from the diffs.  Now I get it. Your patch is good then.  Thanks for removing the hardcoded offsets next.
>> 
>> Coleen
>> 
>>> 
>>> My proposed solution is to remove hardcoded layouts in this patch, and remove hardcoded offsets in the next one. The alternative would be trying to fix the hardcoded layouts only to remove them one patch later. I tried to fix them but ran into issues; that is why I removed them instead in this patch. I hope you think this is okay. The net result will be the same one patch later.
>>> 
>>> Thanks,
>>> /Erik
>>> 
>>>> On 28 Apr 2020, at 21:56, coleen.phillimore at oracle.com wrote:
>>>> 
>>>> ?
>>>> Hi Erik,
>>>> 
>>>> I was trying to figure out why this change is needed to decouple CompressedOops and CompressedKlassPointers, but it's not needed for the change, is it?
>>>> 
>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.01/src/hotspot/share/classfile/fieldLayoutBuilder.cpp.udiff.html
>>>> 
>>>> Can you make this change with your patch for https://bugs.openjdk.java.net/browse/JDK-8243996 which we really like?
>>>> 
>>>> Thanks,
>>>> Coleen
>>>> 
>>>> 
>>>> On 4/28/20 1:52 PM, Erik ?sterlund wrote:
>>>>> Hi Coleen,
>>>>> 
>>>>> Oops, typo in the incremental URL. Here it is:
>>>>> 
>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00..01/
>>>>> 
>>>>> Thanks for the review.
>>>>> 
>>>>> /Erik
>>>>> 
>>>>>> On 28 Apr 2020, at 19:25, coleen.phillimore at oracle.com wrote:
>>>>>> 
>>>>>> ? Okay.  Your incremental is 4040 but looks good to me.
>>>>>> Thanks,
>>>>>> Coleen
>>>>>> 
>>>>>> On 4/28/20 12:30 PM, Erik ?sterlund wrote:
>>>>>>> Hi Coleen,
>>>>>>> 
>>>>>>> Thanks for the review.
>>>>>>> 
>>>>>>> Here is a new webrev:
>>>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.01/
>>>>>>> 
>>>>>>> Incremental:
>>>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00_01/
>>>>>>> 
>>>>>>> On 2020-04-28 16:15, coleen.phillimore at oracle.com wrote:
>>>>>>>> Sorry, this went to the wrong list.
>>>>>>>> 
>>>>>>>> Coleen
>>>>>>>> 
>>>>>>>> On 4/24/20 11:17 AM, coleen.phillimore at oracle.comwrote:
>>>>>>>>> 
>>>>>>>>> Hi Erik, well the interpreter code isn't as bad as I thought it would be.  We tied UseCompressedKlassPointers to UseCompressedOops because we couldn't imagine why someone would care about 4 bytes per object with all 8 byte pointers to oops.  But maybe you do care.
>>>>>>>>> 
>>>>>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/cpu/x86/interp_masm_x86.cpp.udiff.html
>>>>>>>>> 
>>>>>>>>>      if (UseBiasedLocking) {
>>>>>>>>> - biased_locking_enter(lock_reg, obj_reg, swap_reg, tmp_reg, false, done, &slow_case);
>>>>>>>>> + Register rklass_decode_tmp = LP64_ONLY(rscratch1) NOT_LP64(noreg);
>>>>>>>>> + biased_locking_enter(lock_reg, obj_reg, swap_reg, tmp_reg, rscratch1, false, done, &slow_case);
>>>>>>>>>      }
>>>>>>>>> You didn't use rklass_decode_tmp here.
>>>>>>> 
>>>>>>> Nice catch. Fixed.
>>>>>>> 
>>>>>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/cpu/x86/templateTable_x86.cpp.udiff.html
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> + Register tmp_load_klass = LP64_ONLY(rscratch1) NOT_LP64(noreg);
>>>>>>>>> 
>>>>>>>>> I wonder if you couldn't just pass rscratch1 into load_klass and handle ignoring it inside the load_klass function, ie. fix the asserts.  This gets repetitive in the templateTable and other code.  Maybe people may have the opposite opinion though.
>>>>>>> 
>>>>>>> Unfortunately, that won't compile in code that is shared between 32 bit and 64 bit. In 32 bit-only code I just pass in noreg, and in 64-bit only code I just send in the temp. But for shared code, I have to do the macro dance, because rscratch1 does not exist on 32 bit.
>>>>>>> 
>>>>>>>>> This is quite nice that UseCompressedKlassPointers doesn't have to reinit_heapbase.  The interpreter code looks good.
>>>>>>> 
>>>>>>> Thanks!
>>>>>>> 
>>>>>>>>> http://cr.openjdk.java.net/~eosterlund/8241825/webrev.00/src/hotspot/share/memory/metaspace.cpp.frames.html
>>>>>>>>> 
>>>>>>>>> I don't know if base = HeapBaseMinAddress is right for the compressed class space.  It's reserved but not committed as 1G.  Maybe it makes sense since the heap without UseCompressedOops should be above the class space.
>>>>>>> 
>>>>>>> Thomas had a similar comment. I think the summary is that yes it is the right thing to use, but the naming is unfortunate.
>>>>>>> 
>>>>>>> Hope I fixed it all.
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> /Erik
>>>>>>> 
>>>>>>>>> Coleen
>>>>>>>>> 
>>>>>>>>> On 4/24/20 4:25 AM, Erik ?sterlund wrote:
>>>>>>>>>> Hi,
>>>>>>>>>> 
>>>>>>>>>> This is the closed version of the mostly open change. There are some @Contended tests in closed
>>>>>>>>>> that needed some love with my change to make compressed class pointers usable with compressed
>>>>>>>>>> oops disabled. In particular, these tests perform calculations to ensure that things are padded
>>>>>>>>>> as expected. Part of those calculations makes assumptions about how large the object header is,
>>>>>>>>>> presuming that the number is a constant that is the same for all objects, and is determinable
>>>>>>>>>> by checking the offset of the first field. However, with my changes, the first field will end
>>>>>>>>>> up either in the klass gap, or after, depending on the field size. I updated the tests to consider
>>>>>>>>>> different header sizes depending on the type of fields being used. It still isn't perfect, but
>>>>>>>>>> it is clever enough for the purposes of the tests; they pass with these changes.
>>>>>>>>>> 
>>>>>>>>>> Bug:
>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8241825
>>>>>>>>>> 
>>>>>>>>>> Webrev:
>>>>>>>>>> http://wikifiles.se.oracle.com/dev/eosterlund/8241825/webrev.00/
>>>>>>>>>> 
>>>>>>>>>> Thanks,
>>>>>>>>>> /Erik
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>> 
>> 
> 


From coleen.phillimore at oracle.com  Wed Apr 29 21:58:02 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Wed, 29 Apr 2020 17:58:02 -0400
Subject: RFR(M): 8243392: Remodel CDS/Metaspace storage reservation
In-Reply-To: <CAA-vtUyVki-DHepyAzORoTK_MwnrezFUB+QsxbWszztsx1x7NA@mail.gmail.com>
References: <CAA-vtUyVki-DHepyAzORoTK_MwnrezFUB+QsxbWszztsx1x7NA@mail.gmail.com>
Message-ID: <7811f408-2d57-5943-1eeb-5671318dd0a4@oracle.com>


Hi Thomas, Like Ioi, I need more time with this change.? I like the idea 
of cleaning up the interaction of compressed class space, metaspace and 
CDS code.

My initial comment is that I don't like that there's 14 new files to 
have one different implementation for aarch64.? I would much rather see 
an ifdef in compressedOops.cpp, and in metaspace.cpp for 
reserve_address_space_for_compressed_classes too.? It's hard to see how 
you changed the implementation from the code that was under 
PREFERRED_METASPACE_ALIGNMENT.? Can that be changed in place?

thanks,
Coleen

On 4/28/20 10:54 AM, Thomas St?fe wrote:
> Hi all,
>
> Could I have reviews for the following proposal of reworking cds/class
> space reservation?
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8243392
>
> Webrev:
> http://cr.openjdk.java.net/~stuefe/webrevs/rework-cds-ccs-reservation/webrev.00/webrev/
>
> (Many thanks to Ioi Lam for so patiently explaining CDS internals to me,
> and to Andrew Haley and Nick Gasson for help with aarch64!)
>
> Reservation of the compressed class space is needlessly complicated and has
> some minor issues. It can be simplified and made clearer.
>
> The complexity stems from the fact that this area lives at the intersection
> of two to three sub systems, depending on how one counts. Metaspace, CDS,
> and the platform which may or may not its own view of how to reserve class
> space. And this code has been growing organically over time.
>
> One small example:
>
> ReservedSpace Metaspace::reserve_preferred_space(size_t size, size_t
> alignment,
>                                                   bool large_pages, char
> *requested_addr,
>                                                   bool use_requested_addr)
>
> which I spent hours decoding, resulting in a very confused mail to
> hs-runtime and aarch64-port-dev [2].
>
> This patch attempts to simplify cds and metaspace setup a bit; to comment
> implicit knowledge which is not immediately clear; to cleanly abstract
> platform concerns like optimized class space placement; and to disentangle
> cds from metaspace to solve issues which may bite us later with Elastic
> Metaspace [4].
>
> ---
>
> The main change is the reworked reservation mechanism. This is based on
> Ioi's proposal [5].
>
> When reserving class space, three things must happen:
>
> 1) reservation of the space obviously. If cds is active that space must be
> in the vicinity of cds archives to be covered by compressed class pointer
> encoding.
> 2) setting up the internal Metaspace structures atop of that space
> 3) setting up compressed class pointer encoding.
>
> In its current form, Metaspace may or may not do some or all of that in one
> function (Metaspace::allocate_metaspace_compressed_klass_ptrs(ReservedSpace
> metaspace_rs, char* requested_addr, address cds_base);) - if cds is active,
> it will reserve the space for Metaspace and hand it in, otherwise it will
> create it itself.
>
> When discussing this in [2], Ioi proposed to move the reservation of the
> class space completely out of Metaspace and make it a responsibility of the
> caller always. This would reduce some complexity, and this patch follows
> the proposal.
>
> I removed Metaspace::allocate_metaspace_compressed_klass_ptrs(ReservedSpace
> metaspace_rs, char* requested_addr, address cds_base); and all its sub
> functions.
>
> (1) now has to always be done outside - a ReservedSpace for class space has
> to be provided by the caller. However, Metaspace now offers a utility
> function for reserving space at a "nice" location, and explicitly doing
> nothing else:
>
> ReservedSpace
> Metaspace::reserve_address_space_for_compressed_classes(size_t size);
>
> this function can be redefined on a platform level for platform optimized
> reservation, see below for details.
>
> (2) is taken care of by a new function,
> Metaspace::initialize_class_space(ReservedSpace rs)
>
> (3) is taken care of a new function CompressedKlassPointers::initialize(),
> see below for details.
>
>
> So, class space now is set up three explicit steps:
>
> - First, reserve a suitable space by however means you want. For
> convenience you may use
> Metaspace::reserve_address_space_for_compressed_classes(), or you may roll
> your own reservation.
> - Next, tell Metaspace to use that range as backing storage for class
> space: Metaspace::initialize_class_space(ReservedSpace rs)
> - Finally, set up encoding. Encoding is independent from the concept of a
> ReservedSpace, it just gets an address range, see below for details.
>
> Separating these steps and moving them out of the responsibility of
> Metaspace makes this whole thing more flexible; it also removes unnecessary
> knowledge (e.g. Metaspace does not need to know anything about either ccp
> encoding or cds).
>
> ---
>
> How it comes together:
>
> If CDS is off, we just reserve a space using
> Metaspace::reserve_address_space_for_compressed_classes(), initialize it
> with Metaspace::initialize_class_space(ReservedSpace rs), then set up
> compressed class pointer encoding covering the range of this class space.
>
> If CDS is on (dump time), we reserve large 4G space, either at
> SharedBaseAddress or using
> Metaspace::reserve_address_space_for_compressed_classes(); we then split
> that into 3G archive space and 1G class space; we set up that space with
> Metaspace as class space; then we set up compressed class pointer encoding
> covering both archive space and cds.
>
> If CDS is on (run time), we reserve a large space, split it into archive
> space (large enough to hold both archives) and class space, then basically
> proceed as above.
>
> Note that this is almost exactly how things worked before (modulo some
> minor fixes, e.g. alignment issues), only the code is reformed and made
> more explicit.
>
> ---
>
> I moved compressed class pointer setup over to CompressedKlassPointers and
> changed the interface:
>
> -void Metaspace::set_narrow_klass_base_and_shift(ReservedSpace
> metaspace_rs, address cds_base)
> +void CompressedKlassPointers::initialize(address addr, size_t len);
>
> Instead of feeding it a single ReservedSpace, which is supposed to
> represent class space, and an optional alternate base if cds is on, now we
> give it just an numeric address range. That range marks the limits to where
> Klass structures are to be expected, and is the implicit promise that
> outside that range no Klass structures will exist, so encoding has to cover
> only this range.
>
> This range may contain just the class space; or class space+cds; or
> whatever allocation scheme we come up with in the future. Encoding does not
> really care how the memory is organized as long as the input range covers
> all possible Klass locations. That way we remove knowledge about class
> space/cds from compressed class pointer encoding.
>
> Moving it away from metaspace.cpp into the CompressedKlassPointers class
> also mirrors CompressedOops::initialize().
>
> ---
>
> I renamed _narrow_klass_range to just _range, because strictly speaking
> this is the range un-narrow Klass pointers can have.
>
> As for the implementation of CompressedKlassPointers::initialize(address
> addr, size_t len), I mimicked very closely what happened before, so there
> should be almost no differences. Since "almost no differences" sounds scary
> :) here are the differences:
>
> - When CDS is active (dump or run time) we now always, unconditionally, set
> the encoding range to 4G. This fixes a theoretical bug discussed on
> aarch64-port-dev [1].
>
> - When CDS is not active, we set the encoding range to the minimum required
> length. Before, it was left at its default value of 4G.
>
> Both differences only affect aarch64, since they are currently the only one
> using the range field in CompressedKlassPointers.
>
> I wanted to add an assert somewhere to test encoding of the very last
> address of the CompressedKlassPointers range, again to prevent errors like
> [3]. But I did not come up with a good place for this assert which would
> cover also the encoding done by C1/C2.
>
> For the same reason I thought about introducing a mode where Klass
> structures would be allocated in reverse order, starting at the end of the
> ccs, but again left it out as too big a change.
>
> ---
>
> OS abstraction: platforms may have restrictions of what constitutes a valid
> compressed class pointer encoding base. Or if not, they may have at least
> preferences. There was logic like this in metaspace.cpp, which I removed
> and cleanly factored out into platform dependent files, giving each
> platform the option to add special logic.
>
> These are two new methods:
>
> - bool CompressedKlassPointers::is_valid_base(address p)
>
> to let the platform tell you whether it considers p to be a valid encoding
> base. The only platform having these restrictions currently is aarch64.
>
> - ReservedSpace
> Metaspace::reserve_address_space_for_compressed_classes(size_t size);
>
> this hands over the process of allocating a range suitable for compressed
> class pointer encoding to the platform. Most platforms will allocate just
> anywhere, but some platforms may have a better strategy (e.g. trying low
> memory first, trying only correctly aligned addresses and so on).
>
> Beforehand, this coding existed in a similar form in metaspace.cpp for
> aarch64 and AIX. For now, I left the AIX part out - it seems only half
> done, and I want to check further if we even need it, if yes why not on
> Linux ppc, and C1 does not seem to support anything other than base+offset
> with shift either, but I may be mistaken.
>
> These two methods should give the platform enough control to implement
> their own scheme for optimized class space placement without bothering any
> shared code about it.
>
> Note about the form, I introduced two new platform dependent files,
> "metaspace_<cpu>.cpp" and "compressedOops_<cpu>.cpp". I am not happy about
> this but this seems to be what we generally do in hotspot, right?
>
> ---
>
> Metaspace reserve alignment vs cds alignment
>
> CDS was using Metaspace reserve alignment for CDS internal purposes. I
> guess this was just a copy paste issue. It never caused problems since
> Metaspace reserve alignment == page size, but that is not true anymore in
> the upcoming Elastic Metaspace where reserve alignment will be larger. This
> causes a number of issues.
>
> I separated those two cleanly. CDS now uses os::vm_allocation_granularity.
> Metaspace::reserve_alignment is only used in those two places where it is
> needed, when CDS creates the address space for class space on behalf of the
> Metaspace.
>
> ---
>
> Windows special handling in CDS
>
> To simplify coding I removed the windows specific handling which left out
> reservation of the archive. This was needed because windows cannot mmap
> files into reserved regions. But fallback code exists in filemap.cpp for
> this case which just reads in the region instead of mapping it.
>
> Should that turn out to be a performance problem, I will reinstate the
> feature. But a simpler way would be reserve the archive and later just
> before mmapping the archive file to release the archive space. That would
> not only be simpler but give us the best guarantee that that address space
> is actually available. But I'd be happy to leave that part out completely
> if we do not see any performance problems on windows x64.
>
> ---
>
> NMT cannot deal with spaces which are split. This problem manifests in that
> bookkeeping for class space is done under "Shared Classes", not "Classes"
> as it should. This problem exists today too at dump time and randomly at
> run time. But since I simplified the reservation, this problem now shows up
> always, whether or not we map at the SharedBaseAddress.
> While I could work around this problem, I'd prefer this problem to be
> solved at the core, and NMT to have an option to recognize reservation
> splits. So I'd rather not put a workaround for this into the patch but
> leave it for fixing as a separate issue. I opened this issue to track it
> [6].
>
> ---
>
> Jtreg tests:
>
> I expanded the CompressedOops/CompressedClassPointers.java. I also extended
> them to Windows. The tests now optionally omit strict class space placement
> tests, since these tests heavily depend on ASLR and were the reason they
> were excluded on Windows. However I think even without checking for class
> space placement they make sense, just to see that the VM comes up and lives
> with the many different settings we can run in.
>
> ---
>
> Tests:
>
> - I ran the patch through Oracles submit repo
> - I ran tests manually for aarch64, zero, linux 32bit and windows x64
> - The whole battery of nightly tests at SAP, including ppc, ppcle and
> aarch64, unfortunately excluding windows because of unrelated errors.
> Windows x64 tests will be redone tonight.
>
>
> Thank you,
>
> Thomas
>
> [1]
> https://mail.openjdk.java.net/pipermail/aarch64-port-dev/2020-April/008804.html
> [2]
> https://mail.openjdk.java.net/pipermail/aarch64-port-dev/2020-April/008757.html
> [3] https://bugs.openjdk.java.net/browse/JDK-8193266
> [4] https://bugs.openjdk.java.net/browse/JDK-8221173
> [5]
> https://mail.openjdk.java.net/pipermail/aarch64-port-dev/2020-April/008765.html
> [6] https://bugs.openjdk.java.net/browse/JDK-8243535


From xxinliu at amazon.com  Wed Apr 29 23:43:13 2020
From: xxinliu at amazon.com (Liu, Xin)
Date: Wed, 29 Apr 2020 23:43:13 +0000
Subject: [aarch64-port-dev ] FW: [Mach5]
 mach5-one-phh-JDK-8151779-20200427-2151-10554367: FAILED
In-Reply-To: <858sifgbt3.fsf@arm.com>
References: <C769AA2F-55A8-4986-9EEE-0AD93FB93E00@amazon.com>
 <E6D13CFA-A9E6-4262-8D33-2FB4412413B6@amazon.com> <858sifgbt3.fsf@arm.com>
Message-ID: <57BB18E5-1F35-45AC-88BF-8D8C6A0767E3@amazon.com>

Hi, Nick,

Thanks for taking a look at this issue.  I figure out why.

There?re two separated issues. 
1. My toolchain was too old. That?s why I can?t reproduce the building failures feedbacked from submit repo. 

OpenJDK wiki says jdk13 Linux x86_64 supports gcc 8.2, so I believe it means the building hosts must use gcc 8.2+. 
My WIP patch does have a C++ issue captured by g++-8+.

2.  I can?t use adoptOpenJDK?s aarch64 jdk14 as boot-jdk.  Thanks for helping me to understand the problem. 
The pagesize of my aarch64 host is 4k.  I don't have access of adoptOpenJDK's build hosts. I have filed an issue to adoptJDK https://github.com/AdoptOpenJDK/openjdk14-binaries/issues/1
 
One trick here. It's very easy to cheat configure by hacking the boot-jdk.m4 to "$HEAD -n 2". Everything looks fine then. 
 
Thanks,
--lx

?On 4/28/20, 3:14 AM, "hotspot-dev on behalf of Nick Gasson" <hotspot-dev-bounces at openjdk.java.net on behalf of nick.gasson at arm.com> wrote:

    CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.



    >
    > My nightly buildbot started failing recently on aarch64.
    > One issue is the error message of [cds] prevents configure from bootjdk determining.
    > https://hg.openjdk.java.net/jdk/jdk/file/1b8f9e72b22b/make/autoconf/boot-jdk.m4#l77
    >
    > ./bin/java --version
    > [0.006s][error][cds] Unable to map CDS archive -- os::vm_allocation_granularity() expected: 65536 actual: 4096
    > openjdk 14.0.1 2020-04-14
    > OpenJDK Runtime Environment AdoptOpenJDK (build 14.0.1+7)
    > OpenJDK 64-Bit Server VM AdoptOpenJDK (build 14.0.1+7, mixed mode)
    >
    > Are you aware of this issue? I can?t see [cds] line on my linux/x86_64 host.
    > Maybe it?s aarch64-only.  CC aarch-port-dev.
    >

    Was your boot JDK built on a machine configured with a different page
    size to your current machine? Looks like the CDS archive was dumped on a
    machine with 64k pages but you're running with 4k pages. There's a JBS
    issue for this:

    https://bugs.openjdk.java.net/browse/JDK-8236847


    Thanks,
    Nick


From nick.gasson at arm.com  Thu Apr 30 05:55:16 2020
From: nick.gasson at arm.com (Nick Gasson)
Date: Thu, 30 Apr 2020 13:55:16 +0800
Subject: [aarch64-port-dev ] RFR(M): 8243392: Remodel CDS/Metaspace
 storage reservation
In-Reply-To: <25829763-7485-5208-58e8-1d51f7068816@redhat.com>
References: <CAA-vtUyVki-DHepyAzORoTK_MwnrezFUB+QsxbWszztsx1x7NA@mail.gmail.com>
 <857dxyg2dl.fsf@arm.com> <463e11c5-8c0d-3dad-060b-2a3d9e80fd40@redhat.com>
 <854kt2fzec.fsf@arm.com> <25829763-7485-5208-58e8-1d51f7068816@redhat.com>
Message-ID: <85368lfrgb.fsf@arm.com>

On 04/29/20 20:56 pm, Andrew Haley wrote:
>
> That's right: I'm looking at AOT-compiled code (after applying your
> patch) and by default it uses a shift of 3, no offset. If I then run
> the AOT-compiled code with -Xmx31G I get:
>
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x0000ffffa142bd3c, pid=9965, tid=10174
> #
> # JRE version:  (15.0) (slowdebug build )
> # Java VM: OpenJDK 64-Bit Server VM (slowdebug 15-internal+0-adhoc.aph.jdk-tmp, mixed mode, aot, tiered, compressed oops, g1 gc, linux-aarch64)
> # Problematic frame:
> # A 388  java.lang.Thread.setPriority(I)V java.base (56 bytes) @ 0x0000ffffa142bd3c [0x0000ffffa142bac0+0x000000000000027c]
>
>    0x0000ffffa142bd30 <+624>:	ldr	w1, [x4, #56]
>    0x0000ffffa142bd34 <+628>:	cbz	w1, 0xffffa142bd84 <java.lang.Thread.setPriority(I)V+708>
>    0x0000ffffa142bd38 <+632>:	lsl	x1, x1, #3
>    0x0000ffffa142bd3c <+636>:	ldr	w0, [x1, #12]
>
> ... so the AOT-compiled code is still trying to use the shift of 3,
> but it is not adding in the base, which is 0x1000000000. I guess this
> is pilot error, but I'm trying to understand what gets checked and
> when.

No this looks like a real bug: jaotc is using the value of the heap base
in the VM where jaotc is run to decide whether to emit the add or
not. If you run `jaotc -J-Xmx31g` it works.

I'm not very familiar with Graal but I believe this fixes it:

--- a/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot.aarch64/src/org/graalvm/compiler/hotspot/aarch64/AArch64HotSpotMove.java
+++ b/src/jdk.internal.vm.compiler/share/classes/org.graalvm.compiler.hotspot.aarch64/src/org/graalvm/compiler/hotspot/aarch64/AArch64HotSpotMove.java
@@ -139,8 +139,9 @@ public class AArch64HotSpotMove {
             Register resultRegister = asRegister(result);
             Register ptr = asRegister(input);
             Register base = (isRegister(baseRegister) ? asRegister(baseRegister) : zr);
+            boolean pic = GeneratePIC.getValue(crb.getOptions());
             // result = (ptr - base) >> shift
-            if (!encoding.hasBase()) {
+            if (!pic && !encoding.hasBase()) {
                 if (encoding.hasShift()) {
                     masm.lshr(64, resultRegister, ptr, encoding.getShift());
                 } else {
@@ -189,7 +190,8 @@ public class AArch64HotSpotMove {
         public void emitCode(CompilationResultBuilder crb, AArch64MacroAssembler masm) {
             Register inputRegister = asRegister(input);
             Register resultRegister = asRegister(result);
-            Register base = encoding.hasBase() ? asRegister(baseRegister) : null;
+            boolean pic = GeneratePIC.getValue(crb.getOptions());
+            Register base = pic || encoding.hasBase() ? asRegister(baseRegister) : null;
             emitUncompressCode(masm, inputRegister, resultRegister, base, encoding.getShift(), nonNull);
         }


I've made a JBS issue to track this:
https://bugs.openjdk.java.net/browse/JDK-8244164

>
>> But for compressed class pointers we never need to address more than
>> 4G so maybe it's better to use 0 shift instead of logKlassAlignment
>> above? With CDS the default shared base address is 0x80000000 which
>> doesn't allow a zero base anyway.
>
> Maybe. What actually happens when we decode compressed class pointers
> in AOT-compiled code is:
>
> Load the klass pointer from an Object:
>
>   532440:       b940082a        ldr     w10, [x1,#8]
>
> Load the compressed class base:
>
>   532444:       90055e68        adrp    x8, b0fe000 <A.meta.got+0x16e000>
>   532448:       9104c108        add     x8, x8, #0x130
>   53244c:       f9400108        ldr     x8, [x8]
>
> Shift and add:
>
>   532450:       8b2a6d0a        add     x10, x8, x10, uxtx #3
>
> ... none of which is very nice, but the expensive part is loading the
> compressed classw base and doing the add, so I guess we don't care
> that there is a shift as well.

Yes but if we can avoid the shift here then CDS can also use zero shift
by default. Which avoids the problem of having compressed class pointers
with both shift and base non-zero in the Hotspot-generated code.


Thanks,
Nick

From aph at redhat.com  Thu Apr 30 07:57:31 2020
From: aph at redhat.com (Andrew Haley)
Date: Thu, 30 Apr 2020 08:57:31 +0100
Subject: [aarch64-port-dev ] RFR(M): 8243392: Remodel CDS/Metaspace
 storage reservation
In-Reply-To: <CAA-vtUwKc57J=xinDrES0tES+8eYD+nTzjqaqFyxz3E3fzMtVg@mail.gmail.com>
References: <CAA-vtUyVki-DHepyAzORoTK_MwnrezFUB+QsxbWszztsx1x7NA@mail.gmail.com>
 <d6a94952-4d45-9fc3-2b45-f0ba46fda0c5@redhat.com>
 <CAA-vtUwKc57J=xinDrES0tES+8eYD+nTzjqaqFyxz3E3fzMtVg@mail.gmail.com>
Message-ID: <617e5a37-3282-fb9e-6904-a03bd1f71411@redhat.com>

On 4/29/20 5:14 PM, Thomas St?fe wrote:
> Since the intent of this code is to give platforms greater leeway to do
> their thing without disturbing shared code, maybe we should make
> CompressedKlassPointers::initialize() platform dependent too? 

That would be very nice. If we can fix things so that we never shift,
then a lot of things become easier.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From aph at redhat.com  Thu Apr 30 07:57:43 2020
From: aph at redhat.com (Andrew Haley)
Date: Thu, 30 Apr 2020 08:57:43 +0100
Subject: [aarch64-port-dev ] RFR(M): 8243392: Remodel CDS/Metaspace
 storage reservation
In-Reply-To: <85368lfrgb.fsf@arm.com>
References: <CAA-vtUyVki-DHepyAzORoTK_MwnrezFUB+QsxbWszztsx1x7NA@mail.gmail.com>
 <857dxyg2dl.fsf@arm.com> <463e11c5-8c0d-3dad-060b-2a3d9e80fd40@redhat.com>
 <854kt2fzec.fsf@arm.com> <25829763-7485-5208-58e8-1d51f7068816@redhat.com>
 <85368lfrgb.fsf@arm.com>
Message-ID: <45cf5ef0-3939-38df-7959-0e19d79d9e39@redhat.com>

On 4/30/20 6:55 AM, Nick Gasson wrote:
> Yes but if we can avoid the shift here then CDS can also use zero shift
> by default. Which avoids the problem of having compressed class pointers
> with both shift and base non-zero in the Hotspot-generated code.

That sounds good.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From aph at redhat.com  Thu Apr 30 08:18:51 2020
From: aph at redhat.com (Andrew Haley)
Date: Thu, 30 Apr 2020 09:18:51 +0100
Subject: [aarch64-port-dev ] FW: [Mach5]
 mach5-one-phh-JDK-8151779-20200427-2151-10554367: FAILED
In-Reply-To: <57BB18E5-1F35-45AC-88BF-8D8C6A0767E3@amazon.com>
References: <C769AA2F-55A8-4986-9EEE-0AD93FB93E00@amazon.com>
 <E6D13CFA-A9E6-4262-8D33-2FB4412413B6@amazon.com> <858sifgbt3.fsf@arm.com>
 <57BB18E5-1F35-45AC-88BF-8D8C6A0767E3@amazon.com>
Message-ID: <56d85fb9-6ce8-7c00-ff4f-bf3fe40718a4@redhat.com>

On 4/30/20 12:43 AM, Liu, Xin wrote:
> One trick here. It's very easy to cheat configure by hacking the boot-jdk.m4 to "$HEAD -n 2". Everything looks fine then. 

The fix should be submitted to build-dev.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From stefan.karlsson at oracle.com  Thu Apr 30 09:07:54 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Thu, 30 Apr 2020 11:07:54 +0200
Subject: RFR: 8244078: ProcessTools executeTestJvm and
 createJavaProcessBuilder have inconsistent handling of test.*.opts
Message-ID: <44e3e1e7-7135-1892-12d3-cbf3707cfce0@oracle.com>

Hi all,

Please review this patch to make it less likely that we accidentally add 
or fail to add test.java.opts and test.vm.opts to our spawned test JVMs.

https://cr.openjdk.java.net/~stefank/8244078/webrev.01/
https://bugs.openjdk.java.net/browse/JDK-8244078

ProcessTools.createJavaProcessBuilder(cmd) creates a ProcessBuilder 
*without* test.java.opts and test.vm.opts. There is a 
(addTestVmAndJavaOptions, cmd) overload that allows the caller to opt-in 
to the addition of these flags. The created ProcessBuilder is then used 
to start the JVM, and almost always fed into an OutputAnalyzer.

There's another function executeTestJvm, that both creates a 
ProcessBuilder and then feeds it into an OutputAnalyzer (plus a bit 
more). This function uses createJavaProcessBuilder(true, cmd), and 
thereby adds the test.java.opts and test.vm.opts flags.

This means that one has to know about this difference when reading code 
using createJavaProcessBuilder(cmd) and executeTestJvm(cmd), and when 
creating (copying) code using these functions.

It has been suggested that createJavaProcessBuilder is intended to be a 
lower-level, building block API and that it's not confusing that they 
have different behavior. I don't really agree, but I'm buying into the 
notion of lower-level vs higher-level APIs here. So, my proposal is to 
remove the addTestVmAndJavaOptions feature from 
createJavaProcessBuilder, and instead create a new function called 
createTestJvm that adds test.java.opts and test.vm.opts. The name is 
intentionally similar to executeTestJvm, and in fact, the executeTestJvm 
implementation will use createTestJvm:

      public static OutputAnalyzer executeTestJvm(String... cmds) throws Exception {
- ProcessBuilder pb = createJavaProcessBuilder(true, cmds);
+ ProcessBuilder pb = createTestJvm(cmds);
          return executeProcess(pb);
      }

The rest of the patch is mainly:
- leaving createJavaProcessBuilder(cmd) as is
- replacing createJavaProcessBuilder(false, cmd) with 
createJavaProcessBuilder(cmd)
- replacing createJavaProcessBuilder(true, cmd) with createTestJvm(cmd)

There was one odd thing in jdi that requires extra scrutiny:
https://cr.openjdk.java.net/~stefank/8244078/webrev.01/test/jdk/com/sun/jdi/lib/jdb/Debuggee.java.udiff.html

I've run this through tier1-3, and are currently running this through 
higher tiers.

Thanks,
StefanK

From Alan.Bateman at oracle.com  Thu Apr 30 09:24:44 2020
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Thu, 30 Apr 2020 10:24:44 +0100
Subject: RFR: 8244078: ProcessTools executeTestJvm and
 createJavaProcessBuilder have inconsistent handling of test.*.opts
In-Reply-To: <44e3e1e7-7135-1892-12d3-cbf3707cfce0@oracle.com>
References: <44e3e1e7-7135-1892-12d3-cbf3707cfce0@oracle.com>
Message-ID: <a6844af2-a51d-1f60-a299-eaac0a47a032@oracle.com>

On 30/04/2020 10:07, Stefan Karlsson wrote:
> Hi all,
>
> Please review this patch to make it less likely that we accidentally 
> add or fail to add test.java.opts and test.vm.opts to our spawned test 
> JVMs.
>
> https://cr.openjdk.java.net/~stefank/8244078/webrev.01/
> https://bugs.openjdk.java.net/browse/JDK-8244078
We tried a few years ago to get the tests in the libraries areas moved 
to using the xxxJava variants of these methods to make it clear that it 
exec'ing the java launcher of the JDK under test. Methods such as 
executeTestJvm were deprecated as the naming didn't make it clear that 
it exec's the java launcher. Looks like some of this has been lost with 
the combining of the test infrastructure. So while not directly to this 
webrev, I think we need to go back to the naming issue at some point and 
avoid having two sets of methods that do the same thing.

-Alan

From david.holmes at oracle.com  Thu Apr 30 09:59:48 2020
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 30 Apr 2020 19:59:48 +1000
Subject: RFR: 8244078: ProcessTools executeTestJvm and
 createJavaProcessBuilder have inconsistent handling of test.*.opts
In-Reply-To: <44e3e1e7-7135-1892-12d3-cbf3707cfce0@oracle.com>
References: <44e3e1e7-7135-1892-12d3-cbf3707cfce0@oracle.com>
Message-ID: <ab75575b-0995-8460-9f4e-8ac8f401062d@oracle.com>

Hi Stefan,

On 30/04/2020 7:07 pm, Stefan Karlsson wrote:
> Hi all,
> 
> Please review this patch to make it less likely that we accidentally add 
> or fail to add test.java.opts and test.vm.opts to our spawned test JVMs.
> 
> https://cr.openjdk.java.net/~stefank/8244078/webrev.01/
> https://bugs.openjdk.java.net/browse/JDK-8244078
> 
> ProcessTools.createJavaProcessBuilder(cmd) creates a ProcessBuilder 
> *without* test.java.opts and test.vm.opts. There is a 
> (addTestVmAndJavaOptions, cmd) overload that allows the caller to opt-in 
> to the addition of these flags. The created ProcessBuilder is then used 
> to start the JVM, and almost always fed into an OutputAnalyzer.
> 
> There's another function executeTestJvm, that both creates a 
> ProcessBuilder and then feeds it into an OutputAnalyzer (plus a bit 
> more). This function uses createJavaProcessBuilder(true, cmd), and 
> thereby adds the test.java.opts and test.vm.opts flags.
> 
> This means that one has to know about this difference when reading code 
> using createJavaProcessBuilder(cmd) and executeTestJvm(cmd), and when 
> creating (copying) code using these functions.
> 
> It has been suggested that createJavaProcessBuilder is intended to be a 
> lower-level, building block API and that it's not confusing that they 
> have different behavior. I don't really agree, but I'm buying into the 
> notion of lower-level vs higher-level APIs here. So, my proposal is to 
> remove the addTestVmAndJavaOptions feature from 
> createJavaProcessBuilder, and instead create a new function called 
> createTestJvm that adds test.java.opts and test.vm.opts. The name is 
> intentionally similar to executeTestJvm, and in fact, the executeTestJvm 
> implementation will use createTestJvm:
> 
>  ???? public static OutputAnalyzer executeTestJvm(String... cmds) throws 
> Exception {
> - ProcessBuilder pb = createJavaProcessBuilder(true, cmds);
> + ProcessBuilder pb = createTestJvm(cmds);
>  ???????? return executeProcess(pb);
>  ???? }
> 
> The rest of the patch is mainly:
> - leaving createJavaProcessBuilder(cmd) as is
> - replacing createJavaProcessBuilder(false, cmd) with 
> createJavaProcessBuilder(cmd)
> - replacing createJavaProcessBuilder(true, cmd) with createTestJvm(cmd)

This all looks good to me. I think this high/low API makes things much 
clearer.

An observation from compiler/runtime/cr8015436/Driver8015436.java - can:

oa = ProcessTools.executeProcess(ProcessTools.createTestJvm(...));

be replaced with

oa = executeTestJvm(...);

?

---

test/hotspot/jtreg/gc/arguments/GCArguments.java

Isn't the String[] <-> List<String> conversion already handled in 
ProcessTools?

This looks like an area where GC added its own helper utilities early on 
and they aren't really needed any more. Future RFE?

---

test/hotspot/jtreg/gc/g1/mixedgc/TestLogging.java
test/hotspot/jtreg/gc/nvdimm/*

Just observation - these tests also use a code pattern that looks like 
it could use executeTestJvm instead.

---

test/lib/jdk/test/lib/process/ProcessTools.java

This looks a bit odd:

    /**
      * @see #executeTestJvm(String...)
      * @param cmds User specified arguments.
      * @return The output from the process.
      */
     public static OutputAnalyzer executeTestJava(String... cmds) throws 
Exception {
         return executeTestJvm(cmds);
     }

I assume this exists because this was the name used in the JDK test 
library originally and many (most?) of the JDK tests call it rather than 
executeTestJvm. There are a couple of hotspot callers that should 
probably be converted over:

./jtreg/compiler/jvmci/errors/TestInvalidTieredStopAtLevel.java
./jtreg/compiler/jvmci/TestValidateModules.java

otherwise another future RFE to switch over to single API.

> There was one odd thing in jdi that requires extra scrutiny:
> https://cr.openjdk.java.net/~stefank/8244078/webrev.01/test/jdk/com/sun/jdi/lib/jdb/Debuggee.java.udiff.html 

As there are no callers of the addTestVmAndJavaOptions(boolean) API 
these changes seem fine. I can understand why the ability to select 
whether or not to add the test harness args was made available, but it 
seems no test is concerned about using it, so it can go.

Thanks,
David
-----

> 
> I've run this through tier1-3, and are currently running this through 
> higher tiers.
> 
> Thanks,
> StefanK

From stefan.karlsson at oracle.com  Thu Apr 30 10:08:37 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Thu, 30 Apr 2020 12:08:37 +0200
Subject: RFR: 8244078: ProcessTools executeTestJvm and
 createJavaProcessBuilder have inconsistent handling of test.*.opts
In-Reply-To: <a6844af2-a51d-1f60-a299-eaac0a47a032@oracle.com>
References: <44e3e1e7-7135-1892-12d3-cbf3707cfce0@oracle.com>
 <a6844af2-a51d-1f60-a299-eaac0a47a032@oracle.com>
Message-ID: <06bc26af-d877-c1fa-c736-20eb0a76d1a5@oracle.com>

On 2020-04-30 11:24, Alan Bateman wrote:
> On 30/04/2020 10:07, Stefan Karlsson wrote:
>> Hi all,
>>
>> Please review this patch to make it less likely that we accidentally 
>> add or fail to add test.java.opts and test.vm.opts to our spawned 
>> test JVMs.
>>
>> https://cr.openjdk.java.net/~stefank/8244078/webrev.01/
>> https://bugs.openjdk.java.net/browse/JDK-8244078
> We tried a few years ago to get the tests in the libraries areas moved 
> to using the xxxJava variants of these methods to make it clear that 
> it exec'ing the java launcher of the JDK under test. Methods such as 
> executeTestJvm were deprecated as the naming didn't make it clear that 
> it exec's the java launcher. Looks like some of this has been lost 
> with the combining of the test infrastructure. So while not directly 
> to this webrev, I think we need to go back to the naming issue at some 
> point and avoid having two sets of methods that do the same thing.

Are you specifically referring to executeTestJvm vs executeTestJava?

We could take a step back and look at all these functions and try to 
find good names and/or default values. If we're going to clean this up, 
I think we need to figure out good and, preferably, concise naming for 
all of these:
- create test java launcher with test.*.opts
- create test java launcher without test.*.opts
- execute test java launcher with test.*opts
- execute test java launcher without test.*.opts

Do you want me to hold off on this patch until we've resolved this?

StefanK

>
> -Alan


From stefan.karlsson at oracle.com  Thu Apr 30 10:22:19 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Thu, 30 Apr 2020 12:22:19 +0200
Subject: RFR: 8244078: ProcessTools executeTestJvm and
 createJavaProcessBuilder have inconsistent handling of test.*.opts
In-Reply-To: <ab75575b-0995-8460-9f4e-8ac8f401062d@oracle.com>
References: <44e3e1e7-7135-1892-12d3-cbf3707cfce0@oracle.com>
 <ab75575b-0995-8460-9f4e-8ac8f401062d@oracle.com>
Message-ID: <43cc7b8d-7038-e77e-553a-22c76ca80b78@oracle.com>

Hi David,

On 2020-04-30 11:59, David Holmes wrote:
> Hi Stefan,
>
> On 30/04/2020 7:07 pm, Stefan Karlsson wrote:
>> Hi all,
>>
>> Please review this patch to make it less likely that we accidentally 
>> add or fail to add test.java.opts and test.vm.opts to our spawned 
>> test JVMs.
>>
>> https://cr.openjdk.java.net/~stefank/8244078/webrev.01/
>> https://bugs.openjdk.java.net/browse/JDK-8244078
>>
>> ProcessTools.createJavaProcessBuilder(cmd) creates a ProcessBuilder 
>> *without* test.java.opts and test.vm.opts. There is a 
>> (addTestVmAndJavaOptions, cmd) overload that allows the caller to 
>> opt-in to the addition of these flags. The created ProcessBuilder is 
>> then used to start the JVM, and almost always fed into an 
>> OutputAnalyzer.
>>
>> There's another function executeTestJvm, that both creates a 
>> ProcessBuilder and then feeds it into an OutputAnalyzer (plus a bit 
>> more). This function uses createJavaProcessBuilder(true, cmd), and 
>> thereby adds the test.java.opts and test.vm.opts flags.
>>
>> This means that one has to know about this difference when reading 
>> code using createJavaProcessBuilder(cmd) and executeTestJvm(cmd), and 
>> when creating (copying) code using these functions.
>>
>> It has been suggested that createJavaProcessBuilder is intended to be 
>> a lower-level, building block API and that it's not confusing that 
>> they have different behavior. I don't really agree, but I'm buying 
>> into the notion of lower-level vs higher-level APIs here. So, my 
>> proposal is to remove the addTestVmAndJavaOptions feature from 
>> createJavaProcessBuilder, and instead create a new function called 
>> createTestJvm that adds test.java.opts and test.vm.opts. The name is 
>> intentionally similar to executeTestJvm, and in fact, the 
>> executeTestJvm implementation will use createTestJvm:
>>
>> ????? public static OutputAnalyzer executeTestJvm(String... cmds) 
>> throws Exception {
>> - ProcessBuilder pb = createJavaProcessBuilder(true, cmds);
>> + ProcessBuilder pb = createTestJvm(cmds);
>> ????????? return executeProcess(pb);
>> ????? }
>>
>> The rest of the patch is mainly:
>> - leaving createJavaProcessBuilder(cmd) as is
>> - replacing createJavaProcessBuilder(false, cmd) with 
>> createJavaProcessBuilder(cmd)
>> - replacing createJavaProcessBuilder(true, cmd) with createTestJvm(cmd)
>
> This all looks good to me. I think this high/low API makes things much 
> clearer.
>
> An observation from compiler/runtime/cr8015436/Driver8015436.java - can:
>
> oa = ProcessTools.executeProcess(ProcessTools.createTestJvm(...));
>
> be replaced with
>
> oa = executeTestJvm(...);
>
> ?

Yes. Good point.

>
> ---
>
> test/hotspot/jtreg/gc/arguments/GCArguments.java
>
> Isn't the String[] <-> List<String> conversion already handled in 
> ProcessTools?
>
> This looks like an area where GC added its own helper utilities early 
> on and they aren't really needed any more. Future RFE?
I thought the same when I first looked at this, but there's a subtle 
withDefaults(...) call in there that filters out some of the passed in 
flags.

However, it's weird because it doesn't filter the test.vm.opts and 
test.java.opts, only the flags that the tests explicitly passes in. I 
think we need to take a closer look at this, as a separate RFE.

>
> ---
>
> test/hotspot/jtreg/gc/g1/mixedgc/TestLogging.java
> test/hotspot/jtreg/gc/nvdimm/*
>
> Just observation - these tests also use a code pattern that looks like 
> it could use executeTestJvm instead.
Are you thinking about:
 ??? ProcessBuilder pb = ProcessTools.createTestJvm(flags);
 ??? OutputAnalyzer output = new OutputAnalyzer(pb.start());

Yes, my goal is to be able to replace those line with a one-liner. There 
is huge number of places were we have this pattern.

However, note that executeTestJvm waits for the process to finish and 
dumps the output to files. new OutputAnalyzer(pb.start()) doesn't 
AFAICT. Either we think executeTestJvm is the right approach, or we have 
to create yet another function that performs the above code.

>
> ---
>
> test/lib/jdk/test/lib/process/ProcessTools.java
>
> This looks a bit odd:
>
> ?? /**
> ???? * @see #executeTestJvm(String...)
> ???? * @param cmds User specified arguments.
> ???? * @return The output from the process.
> ???? */
> ??? public static OutputAnalyzer executeTestJava(String... cmds) 
> throws Exception {
> ??????? return executeTestJvm(cmds);
> ??? }
>
> I assume this exists because this was the name used in the JDK test 
> library originally and many (most?) of the JDK tests call it rather 
> than executeTestJvm. There are a couple of hotspot callers that should 
> probably be converted over:
>
> ./jtreg/compiler/jvmci/errors/TestInvalidTieredStopAtLevel.java
> ./jtreg/compiler/jvmci/TestValidateModules.java
>
> otherwise another future RFE to switch over to single API.

I think this is what Alan mentioned in his mail. I thought the 
executeTestJava call was the deprecated one, but apparently it's the 
other way around.

>
>> There was one odd thing in jdi that requires extra scrutiny:
>> https://cr.openjdk.java.net/~stefank/8244078/webrev.01/test/jdk/com/sun/jdi/lib/jdb/Debuggee.java.udiff.html 
>
>
> As there are no callers of the addTestVmAndJavaOptions(boolean) API 
> these changes seem fine. I can understand why the ability to select 
> whether or not to add the test harness args was made available, but it 
> seems no test is concerned about using it, so it can go.

Thanks,
StefanK

>
>
> Thanks,
> David
> -----
>
>>
>> I've run this through tier1-3, and are currently running this through 
>> higher tiers.
>>
>> Thanks,
>> StefanK


From stefan.karlsson at oracle.com  Thu Apr 30 10:26:52 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Thu, 30 Apr 2020 03:26:52 -0700 (PDT)
Subject: RFR: 8244078: ProcessTools executeTestJvm and
 createJavaProcessBuilder have inconsistent handling of test.*.opts
In-Reply-To: <43cc7b8d-7038-e77e-553a-22c76ca80b78@oracle.com>
References: <44e3e1e7-7135-1892-12d3-cbf3707cfce0@oracle.com>
 <ab75575b-0995-8460-9f4e-8ac8f401062d@oracle.com>
 <43cc7b8d-7038-e77e-553a-22c76ca80b78@oracle.com>
Message-ID: <b08d07e1-9de9-7770-d400-2ccd51b3a628@oracle.com>

On 2020-04-30 12:22, Stefan Karlsson wrote:
> Hi David,
>
> On 2020-04-30 11:59, David Holmes wrote:
...
>>
>> ---
>>
>> test/hotspot/jtreg/gc/arguments/GCArguments.java
>>
>> Isn't the String[] <-> List<String> conversion already handled in 
>> ProcessTools?
>>
>> This looks like an area where GC added its own helper utilities early 
>> on and they aren't really needed any more. Future RFE?
> I thought the same when I first looked at this, but there's a subtle 
> withDefaults(...) call in there that filters out some of the passed in 
> flags.
>
> However, it's weird because it doesn't filter the test.vm.opts and 
> test.java.opts, only the flags that the tests explicitly passes in. I 
> think we need to take a closer look at this, as a separate RFE.

I read the disableX part to quickly. This *adds* flags to turn off features.

StefanK

From Alan.Bateman at oracle.com  Thu Apr 30 10:36:16 2020
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Thu, 30 Apr 2020 11:36:16 +0100
Subject: RFR: 8244078: ProcessTools executeTestJvm and
 createJavaProcessBuilder have inconsistent handling of test.*.opts
In-Reply-To: <06bc26af-d877-c1fa-c736-20eb0a76d1a5@oracle.com>
References: <44e3e1e7-7135-1892-12d3-cbf3707cfce0@oracle.com>
 <a6844af2-a51d-1f60-a299-eaac0a47a032@oracle.com>
 <06bc26af-d877-c1fa-c736-20eb0a76d1a5@oracle.com>
Message-ID: <43412b9f-331a-7097-d854-d5cc8196e5ea@oracle.com>

On 30/04/2020 11:08, Stefan Karlsson wrote:
>
> Are you specifically referring to executeTestJvm vs executeTestJava?
Yes, executeTestJvm was deprecated in the 
test/lib/testlibrary/jdk/testlibrary/ProcessTools.java but it seems to 
have come back, maybe when we move to a single repo. I only bring it up 
because there are a couple of tests in your webrev that are changed to 
use executeTestJvm when I expected they would use executeTestJava.

>
> We could take a step back and look at all these functions and try to 
> find good names and/or default values. If we're going to clean this 
> up, I think we need to figure out good and, preferably, concise naming 
> for all of these:
> - create test java launcher with test.*.opts
> - create test java launcher without test.*.opts
> - execute test java launcher with test.*opts
> - execute test java launcher without test.*.opts
>
> Do you want me to hold off on this patch until we've resolved this?
I don't want to delay your work. I think we just need to create a few 
issues to do some cleanup and maybe remove the methods from ProcessTools 
that have misleading names.

-Alan

From matthias.baesken at sap.com  Thu Apr 30 15:46:39 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Thu, 30 Apr 2020 15:46:39 +0000
Subject: RFR [XS]:  8244196:  adjust output in os_linux
Message-ID: <AM0PR02MB54128AA97E9F71F88AB15A0793AA0@AM0PR02MB5412.eurprd02.prod.outlook.com>

Hello, please review this small change to os_linux.cpp  .

It  uses the  recently introduced    _print_ascii_file_h
Function t more places (makes the code  a bit  shorter ) .


Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8244196

http://cr.openjdk.java.net/~mbaesken/webrevs/8244196.0/

Thanks, Matthias

From igor.ignatyev at oracle.com  Thu Apr 30 17:15:04 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Thu, 30 Apr 2020 10:15:04 -0700
Subject: RFR(S) : 8243429 : use reproducible random in :vmTestbase_nsk_stress
Message-ID: <82A98A70-3E9B-4A63-BE6D-293EF64A5C8B@oracle.com>

http://cr.openjdk.java.net/~iignatyev/8243429/webrev.00
> 82 lines changed: 41 ins; 4 del; 37 mod;

Hi all,

could you please review this small patch?
from JBS: 
> this subtask is to use j.t.l.Utils.getRandomInstance() as a random number generator, where applicable, in :vmTestbase_nsk_stress test group and marking the tests which make use of "randomness" with a proper k/w.

testing: : vmTestbase_nsk_stress test group
JBS: https://bugs.openjdk.java.net/browse/JDK-8243429
webrevs:
- code changes: http://cr.openjdk.java.net/~iignatyev//8243429/webrev.00.code
> 58 lines changed: 41 ins; 4 del; 13 mod; 

- adding k/w: http://cr.openjdk.java.net/~iignatyev//8243429/webrev.00.kw
> 11 lines changed: 0 ins; 0 del; 11 mod;

- full: http://cr.openjdk.java.net/~iignatyev//8243429/webrev.00
> 82 lines changed: 41 ins; 4 del; 37 mod;


Thanks,
-- Igor

From ekaterina.pavlova at oracle.com  Thu Apr 30 20:59:11 2020
From: ekaterina.pavlova at oracle.com (Ekaterina Pavlova)
Date: Thu, 30 Apr 2020 13:59:11 -0700
Subject: RFR(S) : 8243429 : use reproducible random in
 :vmTestbase_nsk_stress
In-Reply-To: <82A98A70-3E9B-4A63-BE6D-293EF64A5C8B@oracle.com>
References: <82A98A70-3E9B-4A63-BE6D-293EF64A5C8B@oracle.com>
Message-ID: <0ac6619d-4b8b-7b23-8a03-9018b3f0b659@oracle.com>

Seems to be good.

-katya

On 4/30/20 10:15 AM, Igor Ignatyev wrote:
> http://cr.openjdk.java.net/~iignatyev/8243429/webrev.00
>> 82 lines changed: 41 ins; 4 del; 37 mod;
> 
> Hi all,
> 
> could you please review this small patch?
> from JBS:
>> this subtask is to use j.t.l.Utils.getRandomInstance() as a random number generator, where applicable, in :vmTestbase_nsk_stress test group and marking the tests which make use of "randomness" with a proper k/w.
> 
> testing: : vmTestbase_nsk_stress test group
> JBS: https://bugs.openjdk.java.net/browse/JDK-8243429
> webrevs:
> - code changes: http://cr.openjdk.java.net/~iignatyev//8243429/webrev.00.code
>> 58 lines changed: 41 ins; 4 del; 13 mod;
> 
> - adding k/w: http://cr.openjdk.java.net/~iignatyev//8243429/webrev.00.kw
>> 11 lines changed: 0 ins; 0 del; 11 mod;
> 
> - full: http://cr.openjdk.java.net/~iignatyev//8243429/webrev.00
>> 82 lines changed: 41 ins; 4 del; 37 mod;
> 
> 
> Thanks,
> -- Igor
> 


From xxinliu at amazon.com  Thu Apr 30 21:48:07 2020
From: xxinliu at amazon.com (Liu, Xin)
Date: Thu, 30 Apr 2020 21:48:07 +0000
Subject: [aarch64-port-dev ] FW: [Mach5]
 mach5-one-phh-JDK-8151779-20200427-2151-10554367: FAILED
In-Reply-To: <56d85fb9-6ce8-7c00-ff4f-bf3fe40718a4@redhat.com>
References: <C769AA2F-55A8-4986-9EEE-0AD93FB93E00@amazon.com>
 <E6D13CFA-A9E6-4262-8D33-2FB4412413B6@amazon.com> <858sifgbt3.fsf@arm.com>
 <57BB18E5-1F35-45AC-88BF-8D8C6A0767E3@amazon.com>
 <56d85fb9-6ce8-7c00-ff4f-bf3fe40718a4@redhat.com>
Message-ID: <28A344AC-8FF4-49FB-96B9-6AC886C05930@amazon.com>

Hi, Andrew, 

That's a hack. A general way should use grep or sed to capture the needed line instead of hardcoding first or second line. 
Okay, Let me try to do that. 

Thanks, 
--lx


?On 4/30/20, 1:19 AM, "aph at redhat.com" <aph at redhat.com> wrote:

    CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.



    On 4/30/20 12:43 AM, Liu, Xin wrote:
    > One trick here. It's very easy to cheat configure by hacking the boot-jdk.m4 to "$HEAD -n 2". Everything looks fine then.

    The fix should be submitted to build-dev.

    --
    Andrew Haley  (he/him)
    Java Platform Lead Engineer
    Red Hat UK Ltd. <https://www.redhat.com>
    https://keybase.io/andrewhaley
    EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671



From david.holmes at oracle.com  Thu Apr 30 23:31:05 2020
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 1 May 2020 09:31:05 +1000
Subject: RFR [XS]: 8244196: adjust output in os_linux
In-Reply-To: <AM0PR02MB54128AA97E9F71F88AB15A0793AA0@AM0PR02MB5412.eurprd02.prod.outlook.com>
References: <AM0PR02MB54128AA97E9F71F88AB15A0793AA0@AM0PR02MB5412.eurprd02.prod.outlook.com>
Message-ID: <9f7e3d4c-d6e0-6a6e-16ae-51fba319fcfa@oracle.com>

Hi Matthias,

On 1/05/2020 1:46 am, Baesken, Matthias wrote:
> Hello, please review this small change to os_linux.cpp  .
> 
> It  uses the  recently introduced    _print_ascii_file_h
> Function t more places (makes the code  a bit  shorter ) .

I like the intent here but I think we have lost a few newlines in the 
process. It isn't even clear to me that the result of print_ascii_file 
will end with a newline being printed. Can you check the before and 
after layout for readability please (and add to bug report).

Thanks,
David

> 
> Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8244196
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8244196.0/
> 
> Thanks, Matthias
> 

From xxinliu at amazon.com  Thu Apr 30 23:48:05 2020
From: xxinliu at amazon.com (Liu, Xin)
Date: Thu, 30 Apr 2020 23:48:05 +0000
Subject: [aarch64-port-dev ] FW: [Mach5]
 mach5-one-phh-JDK-8151779-20200427-2151-10554367: FAILED
In-Reply-To: <28A344AC-8FF4-49FB-96B9-6AC886C05930@amazon.com>
References: <C769AA2F-55A8-4986-9EEE-0AD93FB93E00@amazon.com>
 <E6D13CFA-A9E6-4262-8D33-2FB4412413B6@amazon.com> <858sifgbt3.fsf@arm.com>
 <57BB18E5-1F35-45AC-88BF-8D8C6A0767E3@amazon.com>
 <56d85fb9-6ce8-7c00-ff4f-bf3fe40718a4@redhat.com>
 <28A344AC-8FF4-49FB-96B9-6AC886C05930@amazon.com>
Message-ID: <9725176C-5E5A-499B-8093-A5865C4AC443@amazon.com>

Hi, Andrew, 

How about this?  I can use awk to capture java -version.  There're 2 cases.

I) openjdk
openjdk version "14.0.1" 2020-04-14
2) oraclejdk
java 14.0.1 2020-04-14 
 
if somehow java displays some error/warning messages,  awk can filter them out and capture the version line.
Eg. 
$ ~/builds/jdk-14.0.1+7/bin/java -version
[0.009s][error][cds] Unable to map CDS archive -- os::vm_allocation_granularity() expected: 65536 actual: 4096
openjdk version "14.0.1" 2020-04-14
OpenJDK Runtime Environment AdoptOpenJDK (build 14.0.1+7)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 14.0.1+7, mixed mode)
$ ~/builds/jdk-14.0.1+7/bin/java -version 2>&1 | awk '/^(openjdk version|java)/ {print $0}'
openjdk version "14.0.1" 2020-04-14

I think this awk stmt is portable, but it's always good to ask experts to review it, so I cc build-dev. 
 
Hers is the change. 

diff --git a/make/autoconf/boot-jdk.m4 b/make/autoconf/boot-jdk.m4
--- a/make/autoconf/boot-jdk.m4
+++ b/make/autoconf/boot-jdk.m4
@@ -74,7 +74,7 @@
           BOOT_JDK_FOUND=no
         else
           # Oh, this is looking good! We probably have found a proper JDK. Is it the correct version?
-          BOOT_JDK_VERSION=`"$BOOT_JDK/bin/java$EXE_SUFFIX" $USER_BOOT_JDK_OPTIONS -version 2>&1 | $HEAD -n 1`
+          BOOT_JDK_VERSION=`"$BOOT_JDK/bin/java$EXE_SUFFIX" $USER_BOOT_JDK_OPTIONS -version 2>&1 | $AWK '/^(openjdk version|java)/ {print [$]0}'`
           if [ [[ "$BOOT_JDK_VERSION" =~ "Picked up" ]] ]; then
             AC_MSG_NOTICE([You have _JAVA_OPTIONS or JAVA_TOOL_OPTIONS set. This can mess up the build. Please use --with-boot-jdk-jvmargs instead.])
             AC_MSG_NOTICE([Java reports: "$BOOT_JDK_VERSION".])
@@ -529,7 +529,7 @@
         BUILD_JDK_FOUND=no
       else
         # Oh, this is looking good! We probably have found a proper JDK. Is it the correct version?
-        BUILD_JDK_VERSION=`"$BUILD_JDK/bin/java" -version 2>&1 | $HEAD -n 1`
+        BUILD_JDK_VERSION=`"$BUILD_JDK/bin/java" -version 2>&1 | $AWK '/^(openjdk version|java)/ {print [$]0}'`

         # Extra M4 quote needed to protect [] in grep expression.
         [FOUND_CORRECT_VERSION=`echo $BUILD_JDK_VERSION | $EGREP "\"$VERSION_FEATURE([\.+-].*)?\""`]



?On 4/30/20, 2:52 PM, "aarch64-port-dev on behalf of Liu, Xin" <aarch64-port-dev-bounces at openjdk.java.net on behalf of xxinliu at amazon.com> wrote:

    Hi, Andrew, 

    That's a hack. A general way should use grep or sed to capture the needed line instead of hardcoding first or second line. 
    Okay, Let me try to do that. 

    Thanks, 
    --lx


    On 4/30/20, 1:19 AM, "aph at redhat.com" <aph at redhat.com> wrote:

        CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.



        On 4/30/20 12:43 AM, Liu, Xin wrote:
        > One trick here. It's very easy to cheat configure by hacking the boot-jdk.m4 to "$HEAD -n 2". Everything looks fine then.

        The fix should be submitted to build-dev.

        --
        Andrew Haley  (he/him)
        Java Platform Lead Engineer
        Red Hat UK Ltd. <https://www.redhat.com>
        https://keybase.io/andrewhaley
        EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671




From david.holmes at oracle.com  Mon Apr 27 05:17:02 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 27 Apr 2020 05:17:02 -0000
Subject: RFR(S) 8238585: Use handshake for
 JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled
 methods on stack not_entrant
In-Reply-To: <AM0PR0202MB3331324D56A68E08CE7CDAE69BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
References: <AM0PR0202MB33311AB7623F4164335286309B1B0@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <3c59b9f9-ec38-18c9-8f24-e1186a08a04a@oracle.com>
 <AM0PR0202MB3331FD660B315CEC25B1DC209B150@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <410eed04-e2ef-0f4f-1c56-19e6734a10f6@oracle.com>
 <AM0PR0202MB33318BC60E3460662D42A40D9B150@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <AM0PR0202MB33319C2F9595CC3C2FA1617B9BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <e1244807-74b4-755b-17c2-dce3263ce337@oss.nttdata.com>
 <AM0PR0202MB333172BA4EAF92D0BC3BA9379BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <81d7caa8-4244-85f3-4d4e-78117fe5e25b@oss.nttdata.com>
 <AM0PR0202MB3331324D56A68E08CE7CDAE69BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
Message-ID: <550b95ac-8b29-1eb8-a507-533e81d02322@oracle.com>

Hi all,

Not a review but some general commentary ...

On 25/04/2020 2:08 am, Reingruber, Richard wrote:
> Hi Yasumasa, Patricio,
> 
>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
>>>> Does it help you? I think it gives you to remove workaround.
>>>
>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake
>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to
>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes.
> 
>> Thanks for your information.
>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop.
>> I will modify and will test it after yours.
> 
> Thanks :)
> 
>>> Also my first impression was that it won't be that easy from a synchronization point of view to
>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls
>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where
>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear
>>> to me, how this has to be handled.
> 
>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event.
> 
> Yes. To me it is unclear what synchronization is necessary, if it is called during a handshake. And
> also I'm unsure if a thread should do safepoint checks while executing a handshake.

I'm growing increasingly concerned that use of direct handshakes to 
replace VM operations needs a much greater examination for correctness 
than might initially be thought. I see a number of issues:

First, the VMThread executes (most) VM operations with a clean stack in 
a clean state, so it has lots of room to work. If we now execute the 
same logic in a JavaThread then we risk hitting stackoverflows if 
nothing else. But we are also now executing code in a JavaThread and so 
we have to be sure that code is not going to act differently (in a bad 
way) if executed by a JavaThread rather than the VMThread. For example, 
may it be possible that if executing in the VMThread we defer some 
activity that might require execution of Java code, or else hand it off 
to one of the service threads? If we execute that code directly in the 
current JavaThread instead we may not be in a valid state (e.g. consider 
re-entrancy to various subsystems that is not allowed).

Second, we have this question mark over what happens if the operation 
hits further safepoint or handshake polls/checks? Are there constraints 
on what is allowed here? How can we recognise this problem may exist and 
so deal with it?

Third, while we are generally considering what appear to be 
single-thread operations, which should be amenable to a direct 
handshake, we also have to be careful that some of the code involved 
doesn't already expect/assume we are at a safepoint - e.g. a VM op may 
not need to take a lock where a direct handshake might!

Cheers,
David
-----

> @Patricio, coming back to my question [1]:
> 
> In the example you gave in your answer [2]: the java thread would execute a vm operation during a
> direct handshake operation, while the VMThread is actually in the middle of a VM_HandshakeAllThreads
> operation, waiting to handshake the same handshakee: why can't the VMThread just proceed? The
> handshakee would be safepoint safe, wouldn't it?
> 
> Thanks, Richard.
> 
> [1] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301677&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301677
> 
> [2] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301763&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301763
> 
> -----Original Message-----
> From: Yasumasa Suenaga <suenaga at oss.nttdata.com>
> Sent: Freitag, 24. April 2020 17:23
> To: Reingruber, Richard <richard.reingruber at sap.com>; Patricio Chilano <patricio.chilano.mateo at oracle.com>; serguei.spitsyn at oracle.com; Vladimir Ivanov <vladimir.x.ivanov at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
> 
> Hi Richard,
> 
> On 2020/04/24 23:44, Reingruber, Richard wrote:
>> Hi Yasumasa,
>>
>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
>>> Does it help you? I think it gives you to remove workaround.
>>
>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake
>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to
>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes.
> 
> Thanks for your information.
> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop.
> I will modify and will test it after yours.
> 
> 
>> Also my first impression was that it won't be that easy from a synchronization point of view to
>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls
>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where
>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear
>> to me, how this has to be handled.
> 
> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event.
> 
> 
> Thanks,
> 
> Yasumasa
> 
> 
>> So it appears to me that it would be easier to push JDK-8242427 after this (JDK-8238585).
>>
>>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.)
>>
>> Would be interesting to see how you handled the issues above :)
>>
>> Thanks, Richard.
>>
>> [1] See question in comment https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14302030&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14302030
>>
>> -----Original Message-----
>> From: Yasumasa Suenaga <suenaga at oss.nttdata.com>
>> Sent: Freitag, 24. April 2020 13:34
>> To: Reingruber, Richard <richard.reingruber at sap.com>; Patricio Chilano <patricio.chilano.mateo at oracle.com>; serguei.spitsyn at oracle.com; Vladimir Ivanov <vladimir.x.ivanov at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>
>> Hi Richard,
>>
>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
>> Does it help you? I think it gives you to remove workaround.
>>
>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.)
>>
>>
>> Thanks,
>>
>> Yasumasa
>>
>>
>> On 2020/04/24 17:18, Reingruber, Richard wrote:
>>> Hi Patricio, Vladimir, and Serguei,
>>>
>>> now that direct handshakes are available, I've updated the patch to make use of them.
>>>
>>> In addition I have done some clean-up changes I missed in the first webrev.
>>>
>>> Finally I have implemented the workaround suggested by Patricio to avoid nesting the handshake
>>> into the vm operation VM_SetFramePop [1]
>>>
>>> Kindly review again:
>>>
>>> Webrev:        http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1/
>>> Webrev(delta): http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1.inc/
>>>
>>> I updated the JBS item explaining why the vm operation VM_EnterInterpOnlyMode can be replaced with a
>>> direct handshake:
>>>
>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8238585
>>>
>>> Testing:
>>>
>>> * JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms.
>>>
>>> * Submit-repo: mach5-one-rrich-JDK-8238585-20200423-1436-10441737
>>>
>>> Thanks,
>>> Richard.
>>>
>>> [1] An assertion in Handshake::execute_direct() fails, if called be VMThread, because it is no JavaThread.
>>>
>>> -----Original Message-----
>>> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of Reingruber, Richard
>>> Sent: Freitag, 14. Februar 2020 19:47
>>> To: Patricio Chilano <patricio.chilano.mateo at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>>> Subject: RE: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>>
>>> Hi Patricio,
>>>
>>>      > > I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a
>>>      > > handshake cannot be nested in a vm operation. Maybe it should be asserted in the
>>>      > > Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation?
>>>      > >
>>>      > >    > Alternatively I think you could do something similar to what we do in
>>>      > >    > Deoptimization::deoptimize_all_marked():
>>>      > >    >
>>>      > >    >    EnterInterpOnlyModeClosure hs;
>>>      > >    >    if (SafepointSynchronize::is_at_safepoint()) {
>>>      > >    >      hs.do_thread(state->get_thread());
>>>      > >    >    } else {
>>>      > >    >      Handshake::execute(&hs, state->get_thread());
>>>      > >    >    }
>>>      > >    > (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>>      > >    > HandshakeClosure() constructor)
>>>      > >
>>>      > > Maybe this could be used also in the Handshake::execute() methods as general solution?
>>>      > Right, we could also do that. Avoiding to clear the polling page in
>>>      > HandshakeState::clear_handshake() should be enough to fix this issue and
>>>      > execute a handshake inside a safepoint, but adding that "if" statement
>>>      > in Hanshake::execute() sounds good to avoid all the extra code that we
>>>      > go through when executing a handshake. I filed 8239084 to make that change.
>>>
>>> Thanks for taking care of this and creating the RFE.
>>>
>>>      >
>>>      > >    > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>>      > >    > always called in a nested operation or just sometimes.
>>>      > >
>>>      > > At least one execution path without vm operation exists:
>>>      > >
>>>      > >    JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void
>>>      > >      JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong
>>>      > >        JvmtiEventControllerPrivate::recompute_enabled() : void
>>>      > >          JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches)
>>>      > >            JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void
>>>      > >              JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError
>>>      > >                jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError
>>>      > >
>>>      > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a
>>>      > > handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further
>>>      > > encouraged to do it with a handshake :)
>>>      > Ah! I think you can still do it with a handshake with the
>>>      > Deoptimization::deoptimize_all_marked() like solution. I can change the
>>>      > if-else statement with just the Handshake::execute() call in 8239084.
>>>      > But up to you.  : )
>>>
>>> Well, I think that's enough encouragement :)
>>> I'll wait for 8239084 and try then again.
>>> (no urgency and all)
>>>
>>> Thanks,
>>> Richard.
>>>
>>> -----Original Message-----
>>> From: Patricio Chilano <patricio.chilano.mateo at oracle.com>
>>> Sent: Freitag, 14. Februar 2020 15:54
>>> To: Reingruber, Richard <richard.reingruber at sap.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>>
>>> Hi Richard,
>>>
>>> On 2/14/20 9:58 AM, Reingruber, Richard wrote:
>>>> Hi Patricio,
>>>>
>>>> thanks for having a look.
>>>>
>>>>       > I?m only commenting on the handshake changes.
>>>>       > I see that operation VM_EnterInterpOnlyMode can be called inside
>>>>       > operation VM_SetFramePop which also allows nested operations. Here is a
>>>>       > comment in VM_SetFramePop definition:
>>>>       >
>>>>       > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is
>>>>       > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled.
>>>>       >
>>>>       > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we
>>>>       > could have a handshake inside a safepoint operation. The issue I see
>>>>       > there is that at the end of the handshake the polling page of the target
>>>>       > thread could be disarmed. So if the target thread happens to be in a
>>>>       > blocked state just transiently and wakes up then it will not stop for
>>>>       > the ongoing safepoint. Maybe I can file an RFE to assert that the
>>>>       > polling page is armed at the beginning of disarm_safepoint().
>>>>
>>>> I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a
>>>> handshake cannot be nested in a vm operation. Maybe it should be asserted in the
>>>> Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation?
>>>>
>>>>       > Alternatively I think you could do something similar to what we do in
>>>>       > Deoptimization::deoptimize_all_marked():
>>>>       >
>>>>       >    EnterInterpOnlyModeClosure hs;
>>>>       >    if (SafepointSynchronize::is_at_safepoint()) {
>>>>       >      hs.do_thread(state->get_thread());
>>>>       >    } else {
>>>>       >      Handshake::execute(&hs, state->get_thread());
>>>>       >    }
>>>>       > (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>>>       > HandshakeClosure() constructor)
>>>>
>>>> Maybe this could be used also in the Handshake::execute() methods as general solution?
>>> Right, we could also do that. Avoiding to clear the polling page in
>>> HandshakeState::clear_handshake() should be enough to fix this issue and
>>> execute a handshake inside a safepoint, but adding that "if" statement
>>> in Hanshake::execute() sounds good to avoid all the extra code that we
>>> go through when executing a handshake. I filed 8239084 to make that change.
>>>
>>>>       > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>>>       > always called in a nested operation or just sometimes.
>>>>
>>>> At least one execution path without vm operation exists:
>>>>
>>>>       JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void
>>>>         JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong
>>>>           JvmtiEventControllerPrivate::recompute_enabled() : void
>>>>             JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches)
>>>>               JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void
>>>>                 JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError
>>>>                   jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError
>>>>
>>>> I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a
>>>> handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further
>>>> encouraged to do it with a handshake :)
>>> Ah! I think you can still do it with a handshake with the
>>> Deoptimization::deoptimize_all_marked() like solution. I can change the
>>> if-else statement with just the Handshake::execute() call in 8239084.
>>> But up to you.? : )
>>>
>>> Thanks,
>>> Patricio
>>>> Thanks again,
>>>> Richard.
>>>>
>>>> -----Original Message-----
>>>> From: Patricio Chilano <patricio.chilano.mateo at oracle.com>
>>>> Sent: Donnerstag, 13. Februar 2020 18:47
>>>> To: Reingruber, Richard <richard.reingruber at sap.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>>>
>>>> Hi Richard,
>>>>
>>>> I?m only commenting on the handshake changes.
>>>> I see that operation VM_EnterInterpOnlyMode can be called inside
>>>> operation VM_SetFramePop which also allows nested operations. Here is a
>>>> comment in VM_SetFramePop definition:
>>>>
>>>> // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is
>>>> // called from the JvmtiEventControllerPrivate::recompute_thread_enabled.
>>>>
>>>> So if we change VM_EnterInterpOnlyMode to be a handshake, then now we
>>>> could have a handshake inside a safepoint operation. The issue I see
>>>> there is that at the end of the handshake the polling page of the target
>>>> thread could be disarmed. So if the target thread happens to be in a
>>>> blocked state just transiently and wakes up then it will not stop for
>>>> the ongoing safepoint. Maybe I can file an RFE to assert that the
>>>> polling page is armed at the beginning of disarm_safepoint().
>>>>
>>>> I think one option could be to remove
>>>> SafepointMechanism::disarm_if_needed() in
>>>> HandshakeState::clear_handshake() and let each JavaThread disarm itself
>>>> for the handshake case.
>>>>
>>>> Alternatively I think you could do something similar to what we do in
>>>> Deoptimization::deoptimize_all_marked():
>>>>
>>>>      ? EnterInterpOnlyModeClosure hs;
>>>>      ? if (SafepointSynchronize::is_at_safepoint()) {
>>>>      ??? hs.do_thread(state->get_thread());
>>>>      ? } else {
>>>>      ??? Handshake::execute(&hs, state->get_thread());
>>>>      ? }
>>>> (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>>> HandshakeClosure() constructor)
>>>>
>>>> I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>>> always called in a nested operation or just sometimes.
>>>>
>>>> Thanks,
>>>> Patricio
>>>>
>>>> On 2/12/20 7:23 AM, Reingruber, Richard wrote:
>>>>> // Repost including hotspot runtime and gc lists.
>>>>> // Dean Long suggested to do so, because the enhancement replaces a vm operation
>>>>> // with a handshake.
>>>>> // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html
>>>>>
>>>>> Hi,
>>>>>
>>>>> could I please get reviews for this small enhancement in hotspot's jvmti implementation:
>>>>>
>>>>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/
>>>>> Bug:    https://bugs.openjdk.java.net/browse/JDK-8238585
>>>>>
>>>>> The change avoids making all compiled methods on stack not_entrant when switching a java thread to
>>>>> interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack.
>>>>>
>>>>> Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations.
>>>>>
>>>>> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms.
>>>>>
>>>>> Thanks, Richard.
>>>>>
>>>>> See also my question if anyone knows a reason for making the compiled methods not_entrant:
>>>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html
>>>

From richard.reingruber at sap.com  Mon Apr 27 14:10:04 2020
From: richard.reingruber at sap.com (Reingruber, Richard)
Date: Mon, 27 Apr 2020 14:10:04 -0000
Subject: RFR(S) 8238585: Use handshake for
 JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled
 methods on stack not_entrant
In-Reply-To: <550b95ac-8b29-1eb8-a507-533e81d02322@oracle.com>
References: <AM0PR0202MB33311AB7623F4164335286309B1B0@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <3c59b9f9-ec38-18c9-8f24-e1186a08a04a@oracle.com>
 <AM0PR0202MB3331FD660B315CEC25B1DC209B150@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <410eed04-e2ef-0f4f-1c56-19e6734a10f6@oracle.com>
 <AM0PR0202MB33318BC60E3460662D42A40D9B150@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <AM0PR0202MB33319C2F9595CC3C2FA1617B9BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <e1244807-74b4-755b-17c2-dce3263ce337@oss.nttdata.com>
 <AM0PR0202MB333172BA4EAF92D0BC3BA9379BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <81d7caa8-4244-85f3-4d4e-78117fe5e25b@oss.nttdata.com>
 <AM0PR0202MB3331324D56A68E08CE7CDAE69BD00@AM0PR0202MB3331.eurprd02.prod.outlook.com>
 <550b95ac-8b29-1eb8-a507-533e81d02322@oracle.com>
Message-ID: <AM0PR0202MB33312E168ECCFD38BAC38FD99BAF0@AM0PR0202MB3331.eurprd02.prod.outlook.com>

Hi David,

> Not a review but some general commentary ...

That's welcome.

> On 25/04/2020 2:08 am, Reingruber, Richard wrote:
> > Hi Yasumasa, Patricio,
> > 
> >>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
> >>>> Does it help you? I think it gives you to remove workaround.
> >>>
> >>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake
> >>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to
> >>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes.
> > 
> >> Thanks for your information.
> >> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop.
> >> I will modify and will test it after yours.
> > 
> > Thanks :)
> > 
> >>> Also my first impression was that it won't be that easy from a synchronization point of view to
> >>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls
> >>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where
> >>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear
> >>> to me, how this has to be handled.
> > 
> >> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event.
> > 
> > Yes. To me it is unclear what synchronization is necessary, if it is called during a handshake. And
> > also I'm unsure if a thread should do safepoint checks while executing a handshake.

> I'm growing increasingly concerned that use of direct handshakes to 
> replace VM operations needs a much greater examination for correctness 
> than might initially be thought. I see a number of issues:

I agree. I'll address your concerns in the context of this review thread for JDK-8238585 below.

In addition I would suggest to take the general part of the discussion to a dedicated thread or to
the review thread for JDK-8242427. I would like to keep this thread closer to its subject.

> First, the VMThread executes (most) VM operations with a clean stack in 
> a clean state, so it has lots of room to work. If we now execute the 
> same logic in a JavaThread then we risk hitting stackoverflows if 
> nothing else. But we are also now executing code in a JavaThread and so 
> we have to be sure that code is not going to act differently (in a bad 
> way) if executed by a JavaThread rather than the VMThread. For example, 
> may it be possible that if executing in the VMThread we defer some 
> activity that might require execution of Java code, or else hand it off 
> to one of the service threads? If we execute that code directly in the 
> current JavaThread instead we may not be in a valid state (e.g. consider 
> re-entrancy to various subsystems that is not allowed).

It is not too complex, what EnterInterpOnlyModeClosure::do_thread() is doing. I already added a
paragraph to the JBS-Item [1] explaining why the direct handshake is sufficient from a
synchronization point of view.

Furthermore the stack is walked and the return pc of compiled frames is replaced with the address of
the deopt handler.

I can't see why this cannot be done with a direct handshake. Something very similar is already done
in JavaThread::deoptimize_marked_methods() which is executed as part of an ordinary handshake.

The demand on stack-space should be very modest. I would not expect a higher risk for stackoverflow.

> Second, we have this question mark over what happens if the operation 
> hits further safepoint or handshake polls/checks? Are there constraints 
> on what is allowed here? How can we recognise this problem may exist and 
> so deal with it?

The thread in EnterInterpOnlyModeClosure::do_thread() can't become safepoint/handshake safe. I
tested locally test/hotspot/jtreg:vmTestbase_nsk_jvmti with a NoSafepointVerifier.

> Third, while we are generally considering what appear to be 
> single-thread operations, which should be amenable to a direct 
> handshake, we also have to be careful that some of the code involved 
> doesn't already expect/assume we are at a safepoint - e.g. a VM op may 
> not need to take a lock where a direct handshake might!

See again my arguments in the JBS item [1].

Thanks,
Richard.

[1] https://bugs.openjdk.java.net/browse/JDK-8238585

-----Original Message-----
From: David Holmes <david.holmes at oracle.com> 
Sent: Montag, 27. April 2020 07:16
To: Reingruber, Richard <richard.reingruber at sap.com>; Yasumasa Suenaga <suenaga at oss.nttdata.com>; Patricio Chilano <patricio.chilano.mateo at oracle.com>; serguei.spitsyn at oracle.com; Vladimir Ivanov <vladimir.x.ivanov at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant

Hi all,

Not a review but some general commentary ...

On 25/04/2020 2:08 am, Reingruber, Richard wrote:
> Hi Yasumasa, Patricio,
> 
>>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
>>>> Does it help you? I think it gives you to remove workaround.
>>>
>>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake
>>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to
>>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes.
> 
>> Thanks for your information.
>> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop.
>> I will modify and will test it after yours.
> 
> Thanks :)
> 
>>> Also my first impression was that it won't be that easy from a synchronization point of view to
>>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls
>>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where
>>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear
>>> to me, how this has to be handled.
> 
>> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event.
> 
> Yes. To me it is unclear what synchronization is necessary, if it is called during a handshake. And
> also I'm unsure if a thread should do safepoint checks while executing a handshake.

I'm growing increasingly concerned that use of direct handshakes to 
replace VM operations needs a much greater examination for correctness 
than might initially be thought. I see a number of issues:

First, the VMThread executes (most) VM operations with a clean stack in 
a clean state, so it has lots of room to work. If we now execute the 
same logic in a JavaThread then we risk hitting stackoverflows if 
nothing else. But we are also now executing code in a JavaThread and so 
we have to be sure that code is not going to act differently (in a bad 
way) if executed by a JavaThread rather than the VMThread. For example, 
may it be possible that if executing in the VMThread we defer some 
activity that might require execution of Java code, or else hand it off 
to one of the service threads? If we execute that code directly in the 
current JavaThread instead we may not be in a valid state (e.g. consider 
re-entrancy to various subsystems that is not allowed).

Second, we have this question mark over what happens if the operation 
hits further safepoint or handshake polls/checks? Are there constraints 
on what is allowed here? How can we recognise this problem may exist and 
so deal with it?

Third, while we are generally considering what appear to be 
single-thread operations, which should be amenable to a direct 
handshake, we also have to be careful that some of the code involved 
doesn't already expect/assume we are at a safepoint - e.g. a VM op may 
not need to take a lock where a direct handshake might!

Cheers,
David
-----

> @Patricio, coming back to my question [1]:
> 
> In the example you gave in your answer [2]: the java thread would execute a vm operation during a
> direct handshake operation, while the VMThread is actually in the middle of a VM_HandshakeAllThreads
> operation, waiting to handshake the same handshakee: why can't the VMThread just proceed? The
> handshakee would be safepoint safe, wouldn't it?
> 
> Thanks, Richard.
> 
> [1] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301677&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301677
> 
> [2] https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14301763&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14301763
> 
> -----Original Message-----
> From: Yasumasa Suenaga <suenaga at oss.nttdata.com>
> Sent: Freitag, 24. April 2020 17:23
> To: Reingruber, Richard <richard.reingruber at sap.com>; Patricio Chilano <patricio.chilano.mateo at oracle.com>; serguei.spitsyn at oracle.com; Vladimir Ivanov <vladimir.x.ivanov at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
> 
> Hi Richard,
> 
> On 2020/04/24 23:44, Reingruber, Richard wrote:
>> Hi Yasumasa,
>>
>>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
>>> Does it help you? I think it gives you to remove workaround.
>>
>> I think it would not help that much. Note that when replacing VM_SetFramePop with a direct handshake
>> you could not just execute VM_EnterInterpOnlyMode as a nested vm operation [1]. So you would have to
>> change/replace VM_EnterInterpOnlyMode and I would have to adapt to these changes.
> 
> Thanks for your information.
> I tested my patch with both vmTestbase/nsk/jvmti/PopFrame and vmTestbase/nsk/jvmti/NotifyFramePop.
> I will modify and will test it after yours.
> 
> 
>> Also my first impression was that it won't be that easy from a synchronization point of view to
>> replace VM_SetFramePop with a direct handshake. E.g. VM_SetFramePop::doit() indirectly calls
>> JvmtiEventController::set_frame_pop(JvmtiEnvThreadState *ets, JvmtiFramePop fpop) where
>> JvmtiThreadState_lock is acquired with safepoint check, if not at safepoint. It's not directly clear
>> to me, how this has to be handled.
> 
> I think JvmtiEventController::set_frame_pop() should hold JvmtiThreadState_lock because it affects other JVMTI operation especially FramePop event.
> 
> 
> Thanks,
> 
> Yasumasa
> 
> 
>> So it appears to me that it would be easier to push JDK-8242427 after this (JDK-8238585).
>>
>>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.)
>>
>> Would be interesting to see how you handled the issues above :)
>>
>> Thanks, Richard.
>>
>> [1] See question in comment https://bugs.openjdk.java.net/browse/JDK-8230594?focusedCommentId=14302030&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14302030
>>
>> -----Original Message-----
>> From: Yasumasa Suenaga <suenaga at oss.nttdata.com>
>> Sent: Freitag, 24. April 2020 13:34
>> To: Reingruber, Richard <richard.reingruber at sap.com>; Patricio Chilano <patricio.chilano.mateo at oracle.com>; serguei.spitsyn at oracle.com; Vladimir Ivanov <vladimir.x.ivanov at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>
>> Hi Richard,
>>
>> I will send review request to replace VM_SetFramePop to handshake in early next week in JDK-8242427.
>> Does it help you? I think it gives you to remove workaround.
>>
>> (The patch is available, but I want to see the result of PIT in this weekend whether JDK-8242425 works fine.)
>>
>>
>> Thanks,
>>
>> Yasumasa
>>
>>
>> On 2020/04/24 17:18, Reingruber, Richard wrote:
>>> Hi Patricio, Vladimir, and Serguei,
>>>
>>> now that direct handshakes are available, I've updated the patch to make use of them.
>>>
>>> In addition I have done some clean-up changes I missed in the first webrev.
>>>
>>> Finally I have implemented the workaround suggested by Patricio to avoid nesting the handshake
>>> into the vm operation VM_SetFramePop [1]
>>>
>>> Kindly review again:
>>>
>>> Webrev:        http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1/
>>> Webrev(delta): http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.1.inc/
>>>
>>> I updated the JBS item explaining why the vm operation VM_EnterInterpOnlyMode can be replaced with a
>>> direct handshake:
>>>
>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8238585
>>>
>>> Testing:
>>>
>>> * JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms.
>>>
>>> * Submit-repo: mach5-one-rrich-JDK-8238585-20200423-1436-10441737
>>>
>>> Thanks,
>>> Richard.
>>>
>>> [1] An assertion in Handshake::execute_direct() fails, if called be VMThread, because it is no JavaThread.
>>>
>>> -----Original Message-----
>>> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of Reingruber, Richard
>>> Sent: Freitag, 14. Februar 2020 19:47
>>> To: Patricio Chilano <patricio.chilano.mateo at oracle.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>>> Subject: RE: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>>
>>> Hi Patricio,
>>>
>>>      > > I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a
>>>      > > handshake cannot be nested in a vm operation. Maybe it should be asserted in the
>>>      > > Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation?
>>>      > >
>>>      > >    > Alternatively I think you could do something similar to what we do in
>>>      > >    > Deoptimization::deoptimize_all_marked():
>>>      > >    >
>>>      > >    >    EnterInterpOnlyModeClosure hs;
>>>      > >    >    if (SafepointSynchronize::is_at_safepoint()) {
>>>      > >    >      hs.do_thread(state->get_thread());
>>>      > >    >    } else {
>>>      > >    >      Handshake::execute(&hs, state->get_thread());
>>>      > >    >    }
>>>      > >    > (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>>      > >    > HandshakeClosure() constructor)
>>>      > >
>>>      > > Maybe this could be used also in the Handshake::execute() methods as general solution?
>>>      > Right, we could also do that. Avoiding to clear the polling page in
>>>      > HandshakeState::clear_handshake() should be enough to fix this issue and
>>>      > execute a handshake inside a safepoint, but adding that "if" statement
>>>      > in Hanshake::execute() sounds good to avoid all the extra code that we
>>>      > go through when executing a handshake. I filed 8239084 to make that change.
>>>
>>> Thanks for taking care of this and creating the RFE.
>>>
>>>      >
>>>      > >    > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>>      > >    > always called in a nested operation or just sometimes.
>>>      > >
>>>      > > At least one execution path without vm operation exists:
>>>      > >
>>>      > >    JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void
>>>      > >      JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong
>>>      > >        JvmtiEventControllerPrivate::recompute_enabled() : void
>>>      > >          JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches)
>>>      > >            JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void
>>>      > >              JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError
>>>      > >                jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError
>>>      > >
>>>      > > I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a
>>>      > > handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further
>>>      > > encouraged to do it with a handshake :)
>>>      > Ah! I think you can still do it with a handshake with the
>>>      > Deoptimization::deoptimize_all_marked() like solution. I can change the
>>>      > if-else statement with just the Handshake::execute() call in 8239084.
>>>      > But up to you.  : )
>>>
>>> Well, I think that's enough encouragement :)
>>> I'll wait for 8239084 and try then again.
>>> (no urgency and all)
>>>
>>> Thanks,
>>> Richard.
>>>
>>> -----Original Message-----
>>> From: Patricio Chilano <patricio.chilano.mateo at oracle.com>
>>> Sent: Freitag, 14. Februar 2020 15:54
>>> To: Reingruber, Richard <richard.reingruber at sap.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>>
>>> Hi Richard,
>>>
>>> On 2/14/20 9:58 AM, Reingruber, Richard wrote:
>>>> Hi Patricio,
>>>>
>>>> thanks for having a look.
>>>>
>>>>       > I?m only commenting on the handshake changes.
>>>>       > I see that operation VM_EnterInterpOnlyMode can be called inside
>>>>       > operation VM_SetFramePop which also allows nested operations. Here is a
>>>>       > comment in VM_SetFramePop definition:
>>>>       >
>>>>       > // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is
>>>>       > // called from the JvmtiEventControllerPrivate::recompute_thread_enabled.
>>>>       >
>>>>       > So if we change VM_EnterInterpOnlyMode to be a handshake, then now we
>>>>       > could have a handshake inside a safepoint operation. The issue I see
>>>>       > there is that at the end of the handshake the polling page of the target
>>>>       > thread could be disarmed. So if the target thread happens to be in a
>>>>       > blocked state just transiently and wakes up then it will not stop for
>>>>       > the ongoing safepoint. Maybe I can file an RFE to assert that the
>>>>       > polling page is armed at the beginning of disarm_safepoint().
>>>>
>>>> I'm really glad you noticed the problematic nesting. This seems to be a general issue: currently a
>>>> handshake cannot be nested in a vm operation. Maybe it should be asserted in the
>>>> Handshake::execute() methods that they are not called by the vm thread evaluating a vm operation?
>>>>
>>>>       > Alternatively I think you could do something similar to what we do in
>>>>       > Deoptimization::deoptimize_all_marked():
>>>>       >
>>>>       >    EnterInterpOnlyModeClosure hs;
>>>>       >    if (SafepointSynchronize::is_at_safepoint()) {
>>>>       >      hs.do_thread(state->get_thread());
>>>>       >    } else {
>>>>       >      Handshake::execute(&hs, state->get_thread());
>>>>       >    }
>>>>       > (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>>>       > HandshakeClosure() constructor)
>>>>
>>>> Maybe this could be used also in the Handshake::execute() methods as general solution?
>>> Right, we could also do that. Avoiding to clear the polling page in
>>> HandshakeState::clear_handshake() should be enough to fix this issue and
>>> execute a handshake inside a safepoint, but adding that "if" statement
>>> in Hanshake::execute() sounds good to avoid all the extra code that we
>>> go through when executing a handshake. I filed 8239084 to make that change.
>>>
>>>>       > I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>>>       > always called in a nested operation or just sometimes.
>>>>
>>>> At least one execution path without vm operation exists:
>>>>
>>>>       JvmtiEventControllerPrivate::enter_interp_only_mode(JvmtiThreadState *) : void
>>>>         JvmtiEventControllerPrivate::recompute_thread_enabled(JvmtiThreadState *) : jlong
>>>>           JvmtiEventControllerPrivate::recompute_enabled() : void
>>>>             JvmtiEventControllerPrivate::change_field_watch(jvmtiEvent, bool) : void (2 matches)
>>>>               JvmtiEventController::change_field_watch(jvmtiEvent, bool) : void
>>>>                 JvmtiEnv::SetFieldAccessWatch(fieldDescriptor *) : jvmtiError
>>>>                   jvmti_SetFieldAccessWatch(jvmtiEnv *, jclass, jfieldID) : jvmtiError
>>>>
>>>> I tend to revert back to VM_EnterInterpOnlyMode as it wasn't my main intent to replace it with a
>>>> handshake, but to avoid making the compiled methods on stack not_entrant.... unless I'm further
>>>> encouraged to do it with a handshake :)
>>> Ah! I think you can still do it with a handshake with the
>>> Deoptimization::deoptimize_all_marked() like solution. I can change the
>>> if-else statement with just the Handshake::execute() call in 8239084.
>>> But up to you.? : )
>>>
>>> Thanks,
>>> Patricio
>>>> Thanks again,
>>>> Richard.
>>>>
>>>> -----Original Message-----
>>>> From: Patricio Chilano <patricio.chilano.mateo at oracle.com>
>>>> Sent: Donnerstag, 13. Februar 2020 18:47
>>>> To: Reingruber, Richard <richard.reingruber at sap.com>; serviceability-dev at openjdk.java.net; hotspot-compiler-dev at openjdk.java.net; hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net
>>>> Subject: Re: RFR(S) 8238585: Use handshake for JvmtiEventControllerPrivate::enter_interp_only_mode() and don't make compiled methods on stack not_entrant
>>>>
>>>> Hi Richard,
>>>>
>>>> I?m only commenting on the handshake changes.
>>>> I see that operation VM_EnterInterpOnlyMode can be called inside
>>>> operation VM_SetFramePop which also allows nested operations. Here is a
>>>> comment in VM_SetFramePop definition:
>>>>
>>>> // Nested operation must be allowed for the VM_EnterInterpOnlyMode that is
>>>> // called from the JvmtiEventControllerPrivate::recompute_thread_enabled.
>>>>
>>>> So if we change VM_EnterInterpOnlyMode to be a handshake, then now we
>>>> could have a handshake inside a safepoint operation. The issue I see
>>>> there is that at the end of the handshake the polling page of the target
>>>> thread could be disarmed. So if the target thread happens to be in a
>>>> blocked state just transiently and wakes up then it will not stop for
>>>> the ongoing safepoint. Maybe I can file an RFE to assert that the
>>>> polling page is armed at the beginning of disarm_safepoint().
>>>>
>>>> I think one option could be to remove
>>>> SafepointMechanism::disarm_if_needed() in
>>>> HandshakeState::clear_handshake() and let each JavaThread disarm itself
>>>> for the handshake case.
>>>>
>>>> Alternatively I think you could do something similar to what we do in
>>>> Deoptimization::deoptimize_all_marked():
>>>>
>>>>      ? EnterInterpOnlyModeClosure hs;
>>>>      ? if (SafepointSynchronize::is_at_safepoint()) {
>>>>      ??? hs.do_thread(state->get_thread());
>>>>      ? } else {
>>>>      ??? Handshake::execute(&hs, state->get_thread());
>>>>      ? }
>>>> (you could pass ?EnterInterpOnlyModeClosure? directly to the
>>>> HandshakeClosure() constructor)
>>>>
>>>> I don?t know JVMTI code so I?m not sure if VM_EnterInterpOnlyMode is
>>>> always called in a nested operation or just sometimes.
>>>>
>>>> Thanks,
>>>> Patricio
>>>>
>>>> On 2/12/20 7:23 AM, Reingruber, Richard wrote:
>>>>> // Repost including hotspot runtime and gc lists.
>>>>> // Dean Long suggested to do so, because the enhancement replaces a vm operation
>>>>> // with a handshake.
>>>>> // Original thread: http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-February/030359.html
>>>>>
>>>>> Hi,
>>>>>
>>>>> could I please get reviews for this small enhancement in hotspot's jvmti implementation:
>>>>>
>>>>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/8238585/webrev.0/
>>>>> Bug:    https://bugs.openjdk.java.net/browse/JDK-8238585
>>>>>
>>>>> The change avoids making all compiled methods on stack not_entrant when switching a java thread to
>>>>> interpreter only execution for jvmti purposes. It is sufficient to deoptimize the compiled frames on stack.
>>>>>
>>>>> Additionally a handshake is used instead of a vm operation to walk the stack and do the deoptimizations.
>>>>>
>>>>> Testing: JCK and JTREG tests, also in Xcomp mode with fastdebug and release builds on all platforms.
>>>>>
>>>>> Thanks, Richard.
>>>>>
>>>>> See also my question if anyone knows a reason for making the compiled methods not_entrant:
>>>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2020-January/030339.html
>>>