From erik.joelsson at oracle.com  Thu Jan  2 07:57:07 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Thu, 2 Jan 2020 08:57:07 +0100
Subject: <AWT Dev> [15] Review Request: 8235975 Update copyright year to
 match last edit in jdk repository for 2014/15/16/17/18
In-Reply-To: <3460a6f6-6178-cc45-5840-0f215eebc53f@oracle.com>
References: <1e7d0395-fc57-4d5b-9cfa-c33e0f6462d5@oracle.com>
 <3460a6f6-6178-cc45-5840-0f215eebc53f@oracle.com>
Message-ID: <7b67e39d-752c-50a8-8e18-9a8f86bd641c@oracle.com>

Build files look good.

/Erik

On 2019-12-24 19:22, Sergey Bylokhov wrote:
> Hello.
>
> Here is an updated version:
> ? Bug: https://bugs.openjdk.java.net/browse/JDK-8235975
> ? Patch (2 Mb): 
> http://cr.openjdk.java.net/~serb/8235975/webrev.03/open.patch
> ? Fix: http://cr.openjdk.java.net/~serb/8235975/webrev.03/
>
> ?- "jdk.internal.vm.compiler" is removed from the patch.
> ?- "Aes128CtsHmacSha2EType.java" is updated to "Copyright (c) 2018"
>
> On 12/22/19 11:24 pm, Sergey Bylokhov wrote:
>> Hello.
>> Please review the fix for JDK 15.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8235975
>> Patch (2 Mb): 
>> http://cr.openjdk.java.net/~serb/8235975/webrev.02/open.patch
>> Fix: http://cr.openjdk.java.net/~serb/8235975/webrev.02
>>
>> I have updated the source code copyrights by the 
>> "update_copyright_year.sh"
>> script for 2014/15/16/18/19 years, unfortunately, cannot run it for 2017
>> because of: "JDK-8187443: Forest Consolidation: Move files to unified 
>> layout"
>> which touched all files.
>>
>>
>
>

From Sergey.Bylokhov at oracle.com  Thu Jan  2 12:02:14 2020
From: Sergey.Bylokhov at oracle.com (Sergey Bylokhov)
Date: Thu, 2 Jan 2020 15:02:14 +0300
Subject: <AWT Dev> [15] Review Request: 8235975 Update copyright year to
 match last edit in jdk repository for 2014/15/16/17/18
In-Reply-To: <7b67e39d-752c-50a8-8e18-9a8f86bd641c@oracle.com>
References: <1e7d0395-fc57-4d5b-9cfa-c33e0f6462d5@oracle.com>
 <3460a6f6-6178-cc45-5840-0f215eebc53f@oracle.com>
 <7b67e39d-752c-50a8-8e18-9a8f86bd641c@oracle.com>
Message-ID: <0caab700-28e3-16e7-db00-b698557443f0@oracle.com>

I guess it is too late to fix it, will need to update the files at the end of 2020.

On 1/2/20 10:57 am, Erik Joelsson wrote:
> Build files look good.
> 
> /Erik
> 
> On 2019-12-24 19:22, Sergey Bylokhov wrote:
>> Hello.
>>
>> Here is an updated version:
>> ? Bug: https://bugs.openjdk.java.net/browse/JDK-8235975
>> ? Patch (2 Mb): http://cr.openjdk.java.net/~serb/8235975/webrev.03/open.patch
>> ? Fix: http://cr.openjdk.java.net/~serb/8235975/webrev.03/
>>
>> ?- "jdk.internal.vm.compiler" is removed from the patch.
>> ?- "Aes128CtsHmacSha2EType.java" is updated to "Copyright (c) 2018"
>>
>> On 12/22/19 11:24 pm, Sergey Bylokhov wrote:
>>> Hello.
>>> Please review the fix for JDK 15.
>>>
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8235975
>>> Patch (2 Mb): http://cr.openjdk.java.net/~serb/8235975/webrev.02/open.patch
>>> Fix: http://cr.openjdk.java.net/~serb/8235975/webrev.02
>>>
>>> I have updated the source code copyrights by the "update_copyright_year.sh"
>>> script for 2014/15/16/18/19 years, unfortunately, cannot run it for 2017
>>> because of: "JDK-8187443: Forest Consolidation: Move files to unified layout"
>>> which touched all files.
>>>
>>>
>>
>>


-- 
Best regards, Sergey.

From matthias.baesken at sap.com  Thu Jan  2 13:26:15 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Thu, 2 Jan 2020 13:26:15 +0000
Subject: RFR: 8236617: jtreg test containers/docker/TestMemoryAwareness.java
 fails after 8226575
Message-ID: <AM6PR02MB420053D86671425CAFEC64CF93200@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello, please review this small adjustment to  jtreg test  containers/docker/TestMemoryAwareness.java .

After change   "8226575: OperatingSystemMXBean should be made container aware"    has been pushed,
we observe failures on linux s390x / ppc64le in the docker related jtreg tests  .


The test runs into  the following error :
java.lang.RuntimeException: 'OperatingSystemMXBean.getTotalSwapSpaceSize: 52428800' missing from stdout/stderr

at jdk.test.lib.process.OutputAnalyzer.shouldContain(OutputAnalyzer.java:187)
at TestMemoryAwareness.testOperatingSystemMXBeanAwareness(TestMemoryAwareness.java:154)
at TestMemoryAwareness.main(TestMemoryAwareness.java:65)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
at java.base/java.lang.Thread.run(Thread.java:832)


The reason is that the value found is instead     OperatingSystemMXBean.getTotalSwapSpaceSize: -104857600    .
When looking into the getTotalSwapSpaceSize() function, we get values of 0 for "limit" and 104857600 for "memLimit" :

57 long limit = containerMetrics.getMemoryAndSwapLimit();
 ....
62 long memLimit = containerMetrics.getMemoryLimit();
63 if (limit >= 0 && memLimit >= 0) {
64 return limit - memLimit;
65 }

That explains the value "-104857600" . We see messages    "Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap" , this most likely
causes the unexpected limit == 0  value  .


Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8236617

http://cr.openjdk.java.net/~mbaesken/webrevs/8236617.0/


Thanks, Matthias

From martin.doerr at sap.com  Thu Jan  2 13:46:51 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Thu, 2 Jan 2020 13:46:51 +0000
Subject: RFR: 8236617: jtreg test
 containers/docker/TestMemoryAwareness.java fails after 8226575
In-Reply-To: <AM6PR02MB420053D86671425CAFEC64CF93200@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB420053D86671425CAFEC64CF93200@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <AM0PR0202MB3297AC6EDB0465B22758F8269A200@AM0PR0202MB3297.eurprd02.prod.outlook.com>

Hi Matthias,

thanks for fixing it.

I suggest to put "Your kernel does not support swap limit capabilities or the cgroup is not mounted" into a String instead of having it 3 times.

Looks good to me otherwise.

Best regards,
Martin


> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
> Baesken, Matthias
> Sent: Donnerstag, 2. Januar 2020 14:26
> To: 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: RFR: 8236617: jtreg test
> containers/docker/TestMemoryAwareness.java fails after 8226575
> 
> Hello, please review this small adjustment to  jtreg test
> containers/docker/TestMemoryAwareness.java .
> 
> After change   "8226575: OperatingSystemMXBean should be made container
> aware"    has been pushed,
> we observe failures on linux s390x / ppc64le in the docker related jtreg tests
> .
> 
> 
> The test runs into  the following error :
> java.lang.RuntimeException:
> 'OperatingSystemMXBean.getTotalSwapSpaceSize: 52428800' missing from
> stdout/stderr
> 
> at
> jdk.test.lib.process.OutputAnalyzer.shouldContain(OutputAnalyzer.java:187)
> at
> TestMemoryAwareness.testOperatingSystemMXBeanAwareness(TestMem
> oryAwareness.java:154)
> at TestMemoryAwareness.main(TestMemoryAwareness.java:65)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMet
> hodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Delega
> tingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:564)
> at
> com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapp
> er.java:127)
> at java.base/java.lang.Thread.run(Thread.java:832)
> 
> 
> The reason is that the value found is instead
> OperatingSystemMXBean.getTotalSwapSpaceSize: -104857600    .
> When looking into the getTotalSwapSpaceSize() function, we get values of 0
> for "limit" and 104857600 for "memLimit" :
> 
> 57 long limit = containerMetrics.getMemoryAndSwapLimit();
>  ....
> 62 long memLimit = containerMetrics.getMemoryLimit();
> 63 if (limit >= 0 && memLimit >= 0) {
> 64 return limit - memLimit;
> 65 }
> 
> That explains the value "-104857600" . We see messages    "Your kernel does
> not support swap limit capabilities or the cgroup is not mounted. Memory
> limited without swap" , this most likely
> causes the unexpected limit == 0  value  .
> 
> 
> Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8236617
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8236617.0/
> 
> 
> Thanks, Matthias

From bob.vandette at oracle.com  Thu Jan  2 16:45:24 2020
From: bob.vandette at oracle.com (Bob Vandette)
Date: Thu, 2 Jan 2020 11:45:24 -0500
Subject: RFR: 8236617: jtreg test
 containers/docker/TestMemoryAwareness.java fails after 8226575
In-Reply-To: <AM6PR02MB420053D86671425CAFEC64CF93200@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB420053D86671425CAFEC64CF93200@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <FFBF6590-7E38-4DB7-BCA4-867DC19F4C2A@oracle.com>

Matthias,

I really don?t like testing for some Docker message that could possibly change or go away in the future.
There may be other reasons that the getTotalSwapSpaceSize function will fail and return 0.

The real problem here is that the OperatingSystemMXBean.getTotalSwapSpaceSize is returning a 
negative value when there is no swap available.  

I?d prefer that we fix this problem by correcting the getTotalSwapSpaceSize function to properly return 0
under these conditions and allow 0 to be a valid expected result in the test.

 if (limit >= 0 && memLimit >= 0) {
   return (limit < memLimit) ? 0 : limit - memLimit;
 }

Note:  My suggestion assumes that there is no swap available when the kernel swap limit capability is not enabled.
           I have not verified this.  The message does claim that this is the case "Memory limited without swap?.

Bob.


> On Jan 2, 2020, at 8:26 AM, Baesken, Matthias <matthias.baesken at sap.com> wrote:
> 
> Hello, please review this small adjustment to  jtreg test  containers/docker/TestMemoryAwareness.java .
> 
> After change   "8226575: OperatingSystemMXBean should be made container aware"    has been pushed,
> we observe failures on linux s390x / ppc64le in the docker related jtreg tests  .
> 
> 
> The test runs into  the following error :
> java.lang.RuntimeException: 'OperatingSystemMXBean.getTotalSwapSpaceSize: 52428800' missing from stdout/stderr
> 
> at jdk.test.lib.process.OutputAnalyzer.shouldContain(OutputAnalyzer.java:187)
> at TestMemoryAwareness.testOperatingSystemMXBeanAwareness(TestMemoryAwareness.java:154)
> at TestMemoryAwareness.main(TestMemoryAwareness.java:65)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:564)
> at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
> at java.base/java.lang.Thread.run(Thread.java:832)
> 
> 
> The reason is that the value found is instead     OperatingSystemMXBean.getTotalSwapSpaceSize: -104857600    .
> When looking into the getTotalSwapSpaceSize() function, we get values of 0 for "limit" and 104857600 for "memLimit" :
> 
> 57 long limit = containerMetrics.getMemoryAndSwapLimit();
> ....
> 62 long memLimit = containerMetrics.getMemoryLimit();
> 63 if (limit >= 0 && memLimit >= 0) {
> 64 return limit - memLimit;
> 65 }
> 
> That explains the value "-104857600" . We see messages    "Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap" , this most likely
> causes the unexpected limit == 0  value  .
> 
> 
> Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8236617
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8236617.0/
> 
> 
> Thanks, Matthias


From joe.darcy at oracle.com  Thu Jan  2 21:26:36 2020
From: joe.darcy at oracle.com (Joe Darcy)
Date: Thu, 2 Jan 2020 13:26:36 -0800
Subject: RFR(S): 8236111 : narrow allowSmartActionArgs disabling
In-Reply-To: <0BA46866-3DEA-44BF-B87C-2B59B84196C9@oracle.com>
References: <F0F5A79A-2AF8-496F-94CB-26C3CC06A30D@oracle.com>
 <ECA06D8F-0F7A-4FBC-B502-4BDC52F019AA@oracle.com>
 <423ea31a-ebf8-4cba-72a4-6fbb934f7789@oracle.com>
 <0BA46866-3DEA-44BF-B87C-2B59B84196C9@oracle.com>
Message-ID: <916e0375-abba-9945-b845-0fd4198513f0@oracle.com>

The removal of the existing TEST.properties files look fine.

Please also solicit feedback from the security libs team as their area 
is affected.

Roger, FYI the serial filter tests are updated as part of this changeset.

Cheers,

-Joe

On 12/23/2019 8:13 PM, Igor Ignatyev wrote:
> Thanks David.
>
> core-libs folks, could you please review jdk part of this patch?
>
> Thanks,
> -- Igor
>
>> On Dec 23, 2019, at 1:33 PM, David Holmes <david.holmes at oracle.com> wrote:
>>
>> Hi Igor,
>>
>> Hotspot changes seem fine. Can't comment on jdk tests.
>>
>> Thanks,
>> David
>>
>> On 24/12/2019 6:42 am, Igor Ignatyev wrote:
>>> ping?
>>>> On Dec 17, 2019, at 11:30 AM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
>>>>
>>>> http://cr.openjdk.java.net/~iignatyev/8236111/webrev.00/
>>>>> 31 lines changed: 20 ins; 11 del; 0 mod;
>>>> Hi all,
>>>>
>>>> could you please review this small patch which enables allowSmartActionArgs in hotspot and jdk test suites and disables them in a small number of test directories? the patch also removes TEST.properties files which enabled allowSmartActionArgs as they aren't needed anymore.
>>>>
>>>> from JBS:
>>>>> currently, allowSmartActionArgs is disabled for the whole hotspot and jdk test suites and enabled just in few places. this makes it a bit harder for people to use smart action arguments in these test suites as they have to not to forget to enable them. and given in all the other test suites, smart action arguments are enabled, it can be confusing and frustrating.
>>>>
>>>> testing: tier1-5
>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8236111
>>>> webrev: http://cr.openjdk.java.net/~iignatyev/8236111/webrev.00/
>>>>
>>>> Thanks,
>>>> -- Igor

From matthias.baesken at sap.com  Fri Jan  3 10:15:16 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 3 Jan 2020 10:15:16 +0000
Subject: RFR: 8236617: jtreg test
 containers/docker/TestMemoryAwareness.java fails after 8226575
In-Reply-To: <FFBF6590-7E38-4DB7-BCA4-867DC19F4C2A@oracle.com>
References: <AM6PR02MB420053D86671425CAFEC64CF93200@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <FFBF6590-7E38-4DB7-BCA4-867DC19F4C2A@oracle.com>
Message-ID: <AM6PR02MB4200FFFB14E85F3BC144F35993230@AM6PR02MB4200.eurprd02.prod.outlook.com>

HI Bob,  
Looking at the docker sources,  the message seems to come  from here :

daemon_unix.go  :

// verifyPlatformContainerResources performs platform-specific validation of the container's resource-configuration
func verifyPlatformContainerResources(resources *containertypes.Resources, sysInfo *sysinfo.SysInfo, update bool) (warnings []string, err error) {
  .....
    //
    // means resources have positive Memory limit,  memory+swap is not unlimited  AND  SwapLimit (memory.memsw.limit_in_bytes ?) is not enabled   [comment added by me)
	if resources.Memory > 0 && resources.MemorySwap != -1 && !sysInfo.SwapLimit {
		warnings = append(warnings, "Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.")
		resources.MemorySwap = -1
	}

with Resources  from hostconfig.go  :

// Resources contains container's resources (cgroups config, ulimits...)
type Resources struct {
	Memory    int64 // Memory limit (in bytes)
 ...
	MemorySwap           int64           // Total memory usage (memory + swap); set `-1` to enable unlimited swap



So I think  your suggestion  to return  0  in that special case in function   getTotalSwapSpaceSize   sounds reasonable to me ( at least better than return a large negative value ).
New webrev :


http://cr.openjdk.java.net/~mbaesken/webrevs/8236617.2/



Thanks, Matthias




> 
> Matthias,
> 
> I really don?t like testing for some Docker message that could possibly change
> or go away in the future.
> There may be other reasons that the getTotalSwapSpaceSize function will fail
> and return 0.
> 
> The real problem here is that the
> OperatingSystemMXBean.getTotalSwapSpaceSize is returning a
> negative value when there is no swap available.
> 
> I?d prefer that we fix this problem by correcting the getTotalSwapSpaceSize
> function to properly return 0
> under these conditions and allow 0 to be a valid expected result in the test.
> 
>  if (limit >= 0 && memLimit >= 0) {
>    return (limit < memLimit) ? 0 : limit - memLimit;
>  }
> 
> Note:  My suggestion assumes that there is no swap available when the
> kernel swap limit capability is not enabled.
>            I have not verified this.  The message does claim that this is the case
> "Memory limited without swap?.
> 
> Bob.
> 
> 
> > On Jan 2, 2020, at 8:26 AM, Baesken, Matthias
> <matthias.baesken at sap.com> wrote:
> >
> > Hello, please review this small adjustment to  jtreg test
> containers/docker/TestMemoryAwareness.java .
> >
> > After change   "8226575: OperatingSystemMXBean should be made
> container aware"    has been pushed,
> > we observe failures on linux s390x / ppc64le in the docker related jtreg
> tests  .
> >
> >
> > The test runs into  the following error :
> > java.lang.RuntimeException:
> 'OperatingSystemMXBean.getTotalSwapSpaceSize: 52428800' missing from
> stdout/stderr
> >
> > at
> jdk.test.lib.process.OutputAnalyzer.shouldContain(OutputAnalyzer.java:187)
> > at
> TestMemoryAwareness.testOperatingSystemMXBeanAwareness(TestMem
> oryAwareness.java:154)
> > at TestMemoryAwareness.main(TestMemoryAwareness.java:65)
> > at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> > at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMet
> hodAccessorImpl.java:62)
> > at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Delega
> tingMethodAccessorImpl.java:43)
> > at java.base/java.lang.reflect.Method.invoke(Method.java:564)
> > at
> com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapp
> er.java:127)
> > at java.base/java.lang.Thread.run(Thread.java:832)
> >
> >
> > The reason is that the value found is instead
> OperatingSystemMXBean.getTotalSwapSpaceSize: -104857600    .
> > When looking into the getTotalSwapSpaceSize() function, we get values of
> 0 for "limit" and 104857600 for "memLimit" :
> >
> > 57 long limit = containerMetrics.getMemoryAndSwapLimit();
> > ....
> > 62 long memLimit = containerMetrics.getMemoryLimit();
> > 63 if (limit >= 0 && memLimit >= 0) {
> > 64 return limit - memLimit;
> > 65 }
> >
> > That explains the value "-104857600" . We see messages    "Your kernel
> does not support swap limit capabilities or the cgroup is not mounted.
> Memory limited without swap" , this most likely
> > causes the unexpected limit == 0  value  .
> >
> >
> > Bug/webrev :
> >
> > https://bugs.openjdk.java.net/browse/JDK-8236617
> >
> > http://cr.openjdk.java.net/~mbaesken/webrevs/8236617.0/
> >
> >
> > Thanks, Matthias


From glaubitz at physik.fu-berlin.de  Sat Jan  4 11:18:16 2020
From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz)
Date: Sat, 4 Jan 2020 12:18:16 +0100
Subject: Status for JDK-8199138, riscv64 support for Zero
Message-ID: <96a56c15-5952-ab7c-8427-4860918f3c4a@physik.fu-berlin.de>

Hi!

Debian and several other distribution already have already been bootstrapped
for riscv64 with a large number of packages building fine.

Support for OpenJDK Zero has been added through a slightly modified version
of JDK-8199138 [1]. Looking at the bug report for JDK-8199138, it seems that
the patch was retracted back in 2018.

However, since the Debian version of the patch works fine, I was wondering
whether we could get it merged in one form or another?

Thanks,
Adrian

> [1] https://git.launchpad.net/~openjdk/ubuntu/+source/openjdk/+git/openjdk/tree/debian/patches/riscv64.diff?h=openjdk-13
> [2] https://bugs.openjdk.java.net/browse/JDK-8199138

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz at debian.org
`. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

From christoph.langer at sap.com  Sun Jan  5 22:21:40 2020
From: christoph.langer at sap.com (Langer, Christoph)
Date: Sun, 5 Jan 2020 22:21:40 +0000
Subject: RFR: 8236617: jtreg test
 containers/docker/TestMemoryAwareness.java fails after 8226575
In-Reply-To: <AM6PR02MB4200FFFB14E85F3BC144F35993230@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB420053D86671425CAFEC64CF93200@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <FFBF6590-7E38-4DB7-BCA4-867DC19F4C2A@oracle.com>
 <AM6PR02MB4200FFFB14E85F3BC144F35993230@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <AM6PR02MB498486C2DD5EC876F94A45928A3D0@AM6PR02MB4984.eurprd02.prod.outlook.com>

Hi Matthias,

this change looks good to me.

Best regards
Christoph

> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
> Baesken, Matthias
> Sent: Freitag, 3. Januar 2020 11:15
> To: Bob Vandette <bob.vandette at oracle.com>
> Cc: hotspot-dev at openjdk.java.net
> Subject: RE: RFR: 8236617: jtreg test
> containers/docker/TestMemoryAwareness.java fails after 8226575
> 
> HI Bob,
> Looking at the docker sources,  the message seems to come  from here :
> 
> daemon_unix.go  :
> 
> // verifyPlatformContainerResources performs platform-specific validation of
> the container's resource-configuration
> func verifyPlatformContainerResources(resources
> *containertypes.Resources, sysInfo *sysinfo.SysInfo, update bool) (warnings
> []string, err error) {
>   .....
>     //
>     // means resources have positive Memory limit,  memory+swap is not
> unlimited  AND  SwapLimit (memory.memsw.limit_in_bytes ?) is not enabled
> [comment added by me)
> 	if resources.Memory > 0 && resources.MemorySwap != -1 &&
> !sysInfo.SwapLimit {
> 		warnings = append(warnings, "Your kernel does not support
> swap limit capabilities or the cgroup is not mounted. Memory limited without
> swap.")
> 		resources.MemorySwap = -1
> 	}
> 
> with Resources  from hostconfig.go  :
> 
> // Resources contains container's resources (cgroups config, ulimits...)
> type Resources struct {
> 	Memory    int64 // Memory limit (in bytes)
>  ...
> 	MemorySwap           int64           // Total memory usage (memory +
> swap); set `-1` to enable unlimited swap
> 
> 
> 
> So I think  your suggestion  to return  0  in that special case in function
> getTotalSwapSpaceSize   sounds reasonable to me ( at least better than
> return a large negative value ).
> New webrev :
> 
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8236617.2/
> 
> 
> 
> Thanks, Matthias
> 
> 
> 
> 
> >
> > Matthias,
> >
> > I really don?t like testing for some Docker message that could possibly
> change
> > or go away in the future.
> > There may be other reasons that the getTotalSwapSpaceSize function will
> fail
> > and return 0.
> >
> > The real problem here is that the
> > OperatingSystemMXBean.getTotalSwapSpaceSize is returning a
> > negative value when there is no swap available.
> >
> > I?d prefer that we fix this problem by correcting the getTotalSwapSpaceSize
> > function to properly return 0
> > under these conditions and allow 0 to be a valid expected result in the test.
> >
> >  if (limit >= 0 && memLimit >= 0) {
> >    return (limit < memLimit) ? 0 : limit - memLimit;
> >  }
> >
> > Note:  My suggestion assumes that there is no swap available when the
> > kernel swap limit capability is not enabled.
> >            I have not verified this.  The message does claim that this is the case
> > "Memory limited without swap?.
> >
> > Bob.
> >
> >
> > > On Jan 2, 2020, at 8:26 AM, Baesken, Matthias
> > <matthias.baesken at sap.com> wrote:
> > >
> > > Hello, please review this small adjustment to  jtreg test
> > containers/docker/TestMemoryAwareness.java .
> > >
> > > After change   "8226575: OperatingSystemMXBean should be made
> > container aware"    has been pushed,
> > > we observe failures on linux s390x / ppc64le in the docker related jtreg
> > tests  .
> > >
> > >
> > > The test runs into  the following error :
> > > java.lang.RuntimeException:
> > 'OperatingSystemMXBean.getTotalSwapSpaceSize: 52428800' missing from
> > stdout/stderr
> > >
> > > at
> >
> jdk.test.lib.process.OutputAnalyzer.shouldContain(OutputAnalyzer.java:187)
> > > at
> >
> TestMemoryAwareness.testOperatingSystemMXBeanAwareness(TestMem
> > oryAwareness.java:154)
> > > at TestMemoryAwareness.main(TestMemoryAwareness.java:65)
> > > at
> > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> > Method)
> > > at
> >
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMet
> > hodAccessorImpl.java:62)
> > > at
> >
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Delega
> > tingMethodAccessorImpl.java:43)
> > > at java.base/java.lang.reflect.Method.invoke(Method.java:564)
> > > at
> >
> com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapp
> > er.java:127)
> > > at java.base/java.lang.Thread.run(Thread.java:832)
> > >
> > >
> > > The reason is that the value found is instead
> > OperatingSystemMXBean.getTotalSwapSpaceSize: -104857600    .
> > > When looking into the getTotalSwapSpaceSize() function, we get values
> of
> > 0 for "limit" and 104857600 for "memLimit" :
> > >
> > > 57 long limit = containerMetrics.getMemoryAndSwapLimit();
> > > ....
> > > 62 long memLimit = containerMetrics.getMemoryLimit();
> > > 63 if (limit >= 0 && memLimit >= 0) {
> > > 64 return limit - memLimit;
> > > 65 }
> > >
> > > That explains the value "-104857600" . We see messages    "Your kernel
> > does not support swap limit capabilities or the cgroup is not mounted.
> > Memory limited without swap" , this most likely
> > > causes the unexpected limit == 0  value  .
> > >
> > >
> > > Bug/webrev :
> > >
> > > https://bugs.openjdk.java.net/browse/JDK-8236617
> > >
> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8236617.0/
> > >
> > >
> > > Thanks, Matthias


From aph at redhat.com  Mon Jan  6 18:16:37 2020
From: aph at redhat.com (Andrew Haley)
Date: Mon, 6 Jan 2020 18:16:37 +0000
Subject: RFR: 8210498: nmethod entry barriers
In-Reply-To: <5BBB538B.5070404@oracle.com>
References: <5BB77E18.6040401@oracle.com>
 <399d9f0b-e66b-30e1-16b3-4873845367a9@redhat.com>
 <5BBB538B.5070404@oracle.com>
Message-ID: <22b7f727-929c-b159-316e-78b76543b4fd@redhat.com>

On 10/8/18 1:54 PM, Erik ?sterlund wrote:
> Also note that the implementation space of the barrier itself has some 
> flexibility. Rickard's first prototype involved having an unconditional 
> branch patched in over a nop. Since the nop is removed in the frontend, 
> it seemed like the most conservative starting point. But since there was 
> no measurable difference to the conditional branch, that was more 
> favourable in the end, since it hade the additional advantage of not 
> requiring a code cache walk in the safepoint. But if you have a platform 
> where the trade off is not as obvious, both mechanisms could easily be 
> supported.

Can you describe this mechanism a little more? I don't really understand
how that would work, even on x86.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From kim.barrett at oracle.com  Tue Jan  7 02:46:22 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Mon, 6 Jan 2020 21:46:22 -0500
Subject: RFR: 8235669: G1: Stack walking API can expose AS_NO_KEEPALIVE
 oops
In-Reply-To: <3b2a6c47-b958-5e41-d7c3-a4d25000b17e@oracle.com>
References: <3b2a6c47-b958-5e41-d7c3-a4d25000b17e@oracle.com>
Message-ID: <76376D1F-0BDE-41CF-A702-CC121E01C1FE@oracle.com>

> On Dec 10, 2019, at 11:02 AM, erik.osterlund at oracle.com wrote:
> 
> Hi,
> 
> When the stack is walked and e.g. locals are turned into StackValues, it might be that said local was made a constant oop by the JIT. In such cases, it is read from the nmethod using ON_STRONG_OOP_REF | AS_NO_KEEPALIVE. However, these oops need to be kept alive when concurrent marking is ongoing.
> While I haven't seen crashes obviously linked to this yet, I don't think we should wait until we do, because it certainly will eventually.
> 
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8235669
> 
> Webrev:
> http://cr.openjdk.java.net/~eosterlund/8235669/webrev.00/
> 
> Thanks,
> /Erik

Change looks good.

I think it's kind of gross that oop_at returns an AS_NO_KEEPALIVE
value to some more or less arbitrary context without any indication
that this can happen.  The scope of AS_NO_KEEPALIVE values really
ought to be more constrained than that.

I wonder if oop_at should do the phantom access, and there should be a
different function for use by those places that want / can cope with
an AS_NO_KEEPALIVE value. So I think there might be some naming / API
issues in this neighborhood, but that can be addressed in a post-14 RFE.



From coleen.phillimore at oracle.com  Tue Jan  7 03:25:21 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Mon, 6 Jan 2020 22:25:21 -0500
Subject: RFR (S) 8236224: Obsolete the FieldsAllocationStyle and CompactFields
 options
Message-ID: <2de68ab4-f913-29d9-97ca-a8fcc5dc94b7@oracle.com>

Summary: Remove the options and code for options deprecated in JDK 14

open webrev at http://cr.openjdk.java.net/~coleenp/2019/8236224.01/webrev
bug link https://bugs.openjdk.java.net/browse/JDK-8236224

Ran tier1 on all oracle platforms, and 2, 3 on linux/windows-x64-debug 
and hs-tier4-graal because there were jvmci changes.

thanks,
Coleen

From david.holmes at oracle.com  Tue Jan  7 05:14:27 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 7 Jan 2020 15:14:27 +1000
Subject: RFR (S) 8236224: Obsolete the FieldsAllocationStyle and
 CompactFields options
In-Reply-To: <2de68ab4-f913-29d9-97ca-a8fcc5dc94b7@oracle.com>
References: <2de68ab4-f913-29d9-97ca-a8fcc5dc94b7@oracle.com>
Message-ID: <731c3127-46ce-15a5-f7b3-02b11dd5cbf3@oracle.com>

Hi Coleen,

On 7/01/2020 1:25 pm, coleen.phillimore at oracle.com wrote:
> Summary: Remove the options and code for options deprecated in JDK 14

Generally looks good.

> open webrev at http://cr.openjdk.java.net/~coleenp/2019/8236224.01/webrev
> bug link https://bugs.openjdk.java.net/browse/JDK-8236224

src/hotspot/share/aot/aotCodeHeap.hpp

   typedef struct {
!   enum { CONFIG_SIZE = 7 * jintSize + 9 };
     // 8 int values

Now 7 int values

     // byte[11] array map to boolean values here

Now byte[10]. Or should that be byte[9]? I think the original code may 
be off by one.

---

src/hotspot/share/classfile/classFileParser.cpp

4133   bool allocate_oops_first = false; // was allocation_style == 0

The comment has no context now that there is no selectable allocation style.

I don't understand why you removed a bunch of classes from this check:

4143       (_class_name == 
vmSymbols::java_lang_AssertionStatusDirectives() ||
4144        _class_name == vmSymbols::java_lang_Class() ||
4145        _class_name == vmSymbols::java_lang_ClassLoader() ||

4147        _class_name == vmSymbols::java_lang_ref_SoftReference() ||
4148        _class_name == vmSymbols::java_lang_StackTraceElement() ||
4149        _class_name == vmSymbols::java_lang_String() ||
4150        _class_name == vmSymbols::java_lang_Throwable() ||

??

---

Thanks,
David

> Ran tier1 on all oracle platforms, and 2, 3 on linux/windows-x64-debug 
> and hs-tier4-graal because there were jvmci changes.
> 
> thanks,
> Coleen

From coleen.phillimore at oracle.com  Tue Jan  7 05:25:08 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Tue, 7 Jan 2020 00:25:08 -0500
Subject: RFR (S) 8236224: Obsolete the FieldsAllocationStyle and
 CompactFields options
In-Reply-To: <731c3127-46ce-15a5-f7b3-02b11dd5cbf3@oracle.com>
References: <2de68ab4-f913-29d9-97ca-a8fcc5dc94b7@oracle.com>
 <731c3127-46ce-15a5-f7b3-02b11dd5cbf3@oracle.com>
Message-ID: <7cc12a90-159b-cdaf-2a48-7fec1e2540c9@oracle.com>



On 1/7/20 12:14 AM, David Holmes wrote:
> Hi Coleen,
>
> On 7/01/2020 1:25 pm, coleen.phillimore at oracle.com wrote:
>> Summary: Remove the options and code for options deprecated in JDK 14
>
> Generally looks good.
>
>> open webrev at 
>> http://cr.openjdk.java.net/~coleenp/2019/8236224.01/webrev
>> bug link https://bugs.openjdk.java.net/browse/JDK-8236224
>
> src/hotspot/share/aot/aotCodeHeap.hpp
>
> ? typedef struct {
> !?? enum { CONFIG_SIZE = 7 * jintSize + 9 };
> ??? // 8 int values
>
> Now 7 int values
>
> ??? // byte[11] array map to boolean values here
>
> Now byte[10]. Or should that be byte[9]? I think the original code may 
> be off by one.

Yes, it was wrong. I fixed the comments.
>
> ---
>
> src/hotspot/share/classfile/classFileParser.cpp
>
> 4133?? bool allocate_oops_first = false; // was allocation_style == 0
>
> The comment has no context now that there is no selectable allocation 
> style.
>
Removed.? It was mostly to remind myself.
> I don't understand why you removed a bunch of classes from this check:
>
> 4143?????? (_class_name == 
> vmSymbols::java_lang_AssertionStatusDirectives() ||
> 4144??????? _class_name == vmSymbols::java_lang_Class() ||
> 4145??????? _class_name == vmSymbols::java_lang_ClassLoader() ||
>
> 4147??????? _class_name == vmSymbols::java_lang_ref_SoftReference() ||
> 4148??????? _class_name == vmSymbols::java_lang_StackTraceElement() ||
> 4149??????? _class_name == vmSymbols::java_lang_String() ||
> 4150??????? _class_name == vmSymbols::java_lang_Throwable() ||
>
> ??

The classes removed no longer have hardcoded offsets so did not need to 
follow the oops-first allocation style.? This was not cleaned up when 
the hardcoded offsets were removed from these classes. ? Fred also fixes 
this with his field layout patch in perhaps another place.

Thanks,
Coleen
>
> ---
>
> Thanks,
> David
>
>> Ran tier1 on all oracle platforms, and 2, 3 on 
>> linux/windows-x64-debug and hs-tier4-graal because there were jvmci 
>> changes.
>>
>> thanks,
>> Coleen


From david.holmes at oracle.com  Tue Jan  7 06:05:49 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 7 Jan 2020 16:05:49 +1000
Subject: RFR (S) 8236224: Obsolete the FieldsAllocationStyle and
 CompactFields options
In-Reply-To: <7cc12a90-159b-cdaf-2a48-7fec1e2540c9@oracle.com>
References: <2de68ab4-f913-29d9-97ca-a8fcc5dc94b7@oracle.com>
 <731c3127-46ce-15a5-f7b3-02b11dd5cbf3@oracle.com>
 <7cc12a90-159b-cdaf-2a48-7fec1e2540c9@oracle.com>
Message-ID: <cd6e1c54-fec9-22d0-998e-38f6757fe072@oracle.com>

Hi Coleen,

On 7/01/2020 3:25 pm, coleen.phillimore at oracle.com wrote:
> 
> 
> On 1/7/20 12:14 AM, David Holmes wrote:
>> Hi Coleen,
>>
>> On 7/01/2020 1:25 pm, coleen.phillimore at oracle.com wrote:
>>> Summary: Remove the options and code for options deprecated in JDK 14
>>
>> Generally looks good.
>>
>>> open webrev at 
>>> http://cr.openjdk.java.net/~coleenp/2019/8236224.01/webrev
>>> bug link https://bugs.openjdk.java.net/browse/JDK-8236224
>>
>> src/hotspot/share/aot/aotCodeHeap.hpp
>>
>> ? typedef struct {
>> !?? enum { CONFIG_SIZE = 7 * jintSize + 9 };
>> ??? // 8 int values
>>
>> Now 7 int values
>>
>> ??? // byte[11] array map to boolean values here
>>
>> Now byte[10]. Or should that be byte[9]? I think the original code may 
>> be off by one.
> 
> Yes, it was wrong. I fixed the comments.
>>
>> ---
>>
>> src/hotspot/share/classfile/classFileParser.cpp
>>
>> 4133?? bool allocate_oops_first = false; // was allocation_style == 0
>>
>> The comment has no context now that there is no selectable allocation 
>> style.
>>
> Removed.? It was mostly to remind myself.
>> I don't understand why you removed a bunch of classes from this check:
>>
>> 4143?????? (_class_name == 
>> vmSymbols::java_lang_AssertionStatusDirectives() ||
>> 4144??????? _class_name == vmSymbols::java_lang_Class() ||
>> 4145??????? _class_name == vmSymbols::java_lang_ClassLoader() ||
>>
>> 4147??????? _class_name == vmSymbols::java_lang_ref_SoftReference() ||
>> 4148??????? _class_name == vmSymbols::java_lang_StackTraceElement() ||
>> 4149??????? _class_name == vmSymbols::java_lang_String() ||
>> 4150??????? _class_name == vmSymbols::java_lang_Throwable() ||
>>
>> ??
> 
> The classes removed no longer have hardcoded offsets so did not need to 
> follow the oops-first allocation style.? This was not cleaned up when 
> the hardcoded offsets were removed from these classes. ? Fred also fixes 
> this with his field layout patch in perhaps another place.

Okay thanks for clarifying.

David

> Thanks,
> Coleen
>>
>> ---
>>
>> Thanks,
>> David
>>
>>> Ran tier1 on all oracle platforms, and 2, 3 on 
>>> linux/windows-x64-debug and hs-tier4-graal because there were jvmci 
>>> changes.
>>>
>>> thanks,
>>> Coleen
> 

From matthias.baesken at sap.com  Tue Jan  7 08:09:13 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 7 Jan 2020 08:09:13 +0000
Subject: RFR: 8236617: jtreg test
 containers/docker/TestMemoryAwareness.java fails after 8226575
In-Reply-To: <AM6PR02MB498486C2DD5EC876F94A45928A3D0@AM6PR02MB4984.eurprd02.prod.outlook.com>
References: <AM6PR02MB420053D86671425CAFEC64CF93200@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <FFBF6590-7E38-4DB7-BCA4-867DC19F4C2A@oracle.com>
 <AM6PR02MB4200FFFB14E85F3BC144F35993230@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <AM6PR02MB498486C2DD5EC876F94A45928A3D0@AM6PR02MB4984.eurprd02.prod.outlook.com>
Message-ID: <AM6PR02MB4200D9DAF1D775990A49EF90933F0@AM6PR02MB4200.eurprd02.prod.outlook.com>


Hi Christoph, thanks for the review !

Bob are you fine with the latest version ?

Best regards, Matthias


> Hi Matthias,
> 
> this change looks good to me.
> 
> Best regards
> Christoph
> 
> > -----Original Message-----
> > From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf
> Of
> > Baesken, Matthias
> > Sent: Freitag, 3. Januar 2020 11:15
> > To: Bob Vandette <bob.vandette at oracle.com>
> > Cc: hotspot-dev at openjdk.java.net
> > Subject: RE: RFR: 8236617: jtreg test
> > containers/docker/TestMemoryAwareness.java fails after 8226575
> >
> > HI Bob,
> > Looking at the docker sources,  the message seems to come  from here :
> >
> > daemon_unix.go  :
> >
> > // verifyPlatformContainerResources performs platform-specific validation
> of
> > the container's resource-configuration
> > func verifyPlatformContainerResources(resources
> > *containertypes.Resources, sysInfo *sysinfo.SysInfo, update bool)
> (warnings
> > []string, err error) {
> >   .....
> >     //
> >     // means resources have positive Memory limit,  memory+swap is not
> > unlimited  AND  SwapLimit (memory.memsw.limit_in_bytes ?) is not
> enabled
> > [comment added by me)
> > 	if resources.Memory > 0 && resources.MemorySwap != -1 &&
> > !sysInfo.SwapLimit {
> > 		warnings = append(warnings, "Your kernel does not support
> > swap limit capabilities or the cgroup is not mounted. Memory limited
> without
> > swap.")
> > 		resources.MemorySwap = -1
> > 	}
> >
> > with Resources  from hostconfig.go  :
> >
> > // Resources contains container's resources (cgroups config, ulimits...)
> > type Resources struct {
> > 	Memory    int64 // Memory limit (in bytes)
> >  ...
> > 	MemorySwap           int64           // Total memory usage (memory +
> > swap); set `-1` to enable unlimited swap
> >
> >
> >
> > So I think  your suggestion  to return  0  in that special case in function
> > getTotalSwapSpaceSize   sounds reasonable to me ( at least better than
> > return a large negative value ).
> > New webrev :
> >
> >
> > http://cr.openjdk.java.net/~mbaesken/webrevs/8236617.2/
> >
> >
> >
> > Thanks, Matthias
> >
> >
> >
> >
> > >
> > > Matthias,
> > >
> > > I really don?t like testing for some Docker message that could possibly
> > change
> > > or go away in the future.
> > > There may be other reasons that the getTotalSwapSpaceSize function will
> > fail
> > > and return 0.
> > >
> > > The real problem here is that the
> > > OperatingSystemMXBean.getTotalSwapSpaceSize is returning a
> > > negative value when there is no swap available.
> > >
> > > I?d prefer that we fix this problem by correcting the
> getTotalSwapSpaceSize
> > > function to properly return 0
> > > under these conditions and allow 0 to be a valid expected result in the
> test.
> > >
> > >  if (limit >= 0 && memLimit >= 0) {
> > >    return (limit < memLimit) ? 0 : limit - memLimit;
> > >  }
> > >
> > > Note:  My suggestion assumes that there is no swap available when the
> > > kernel swap limit capability is not enabled.
> > >            I have not verified this.  The message does claim that this is the case
> > > "Memory limited without swap?.
> > >
> > > Bob.
> > >
> > >
> > > > On Jan 2, 2020, at 8:26 AM, Baesken, Matthias
> > > <matthias.baesken at sap.com> wrote:
> > > >
> > > > Hello, please review this small adjustment to  jtreg test
> > > containers/docker/TestMemoryAwareness.java .
> > > >
> > > > After change   "8226575: OperatingSystemMXBean should be made
> > > container aware"    has been pushed,
> > > > we observe failures on linux s390x / ppc64le in the docker related jtreg
> > > tests  .
> > > >
> > > >
> > > > The test runs into  the following error :
> > > > java.lang.RuntimeException:
> > > 'OperatingSystemMXBean.getTotalSwapSpaceSize: 52428800' missing
> from
> > > stdout/stderr
> > > >
> > > > at
> > >
> >
> jdk.test.lib.process.OutputAnalyzer.shouldContain(OutputAnalyzer.java:187)
> > > > at
> > >
> >
> TestMemoryAwareness.testOperatingSystemMXBeanAwareness(TestMem
> > > oryAwareness.java:154)
> > > > at TestMemoryAwareness.main(TestMemoryAwareness.java:65)
> > > > at
> > >
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> > > Method)
> > > > at
> > >
> >
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMet
> > > hodAccessorImpl.java:62)
> > > > at
> > >
> >
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Delega
> > > tingMethodAccessorImpl.java:43)
> > > > at java.base/java.lang.reflect.Method.invoke(Method.java:564)
> > > > at
> > >
> >
> com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapp
> > > er.java:127)
> > > > at java.base/java.lang.Thread.run(Thread.java:832)
> > > >
> > > >
> > > > The reason is that the value found is instead
> > > OperatingSystemMXBean.getTotalSwapSpaceSize: -104857600    .
> > > > When looking into the getTotalSwapSpaceSize() function, we get values
> > of
> > > 0 for "limit" and 104857600 for "memLimit" :
> > > >
> > > > 57 long limit = containerMetrics.getMemoryAndSwapLimit();
> > > > ....
> > > > 62 long memLimit = containerMetrics.getMemoryLimit();
> > > > 63 if (limit >= 0 && memLimit >= 0) {
> > > > 64 return limit - memLimit;
> > > > 65 }
> > > >
> > > > That explains the value "-104857600" . We see messages    "Your kernel
> > > does not support swap limit capabilities or the cgroup is not mounted.
> > > Memory limited without swap" , this most likely
> > > > causes the unexpected limit == 0  value  .
> > > >
> > > >
> > > > Bug/webrev :
> > > >
> > > > https://bugs.openjdk.java.net/browse/JDK-8236617
> > > >
> > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8236617.0/
> > > >
> > > >
> > > > Thanks, Matthias


From erik.osterlund at oracle.com  Tue Jan  7 08:55:30 2020
From: erik.osterlund at oracle.com (erik.osterlund at oracle.com)
Date: Tue, 7 Jan 2020 09:55:30 +0100
Subject: RFR: 8235669: G1: Stack walking API can expose AS_NO_KEEPALIVE
 oops
In-Reply-To: <76376D1F-0BDE-41CF-A702-CC121E01C1FE@oracle.com>
References: <3b2a6c47-b958-5e41-d7c3-a4d25000b17e@oracle.com>
 <76376D1F-0BDE-41CF-A702-CC121E01C1FE@oracle.com>
Message-ID: <0949e751-cae1-45f0-bbc8-4ba9abc54406@oracle.com>

Hi Kim,

Thanks for the review. I agree the naming should be fixed (in 14).

Thanks,
/Erik

On 1/7/20 3:46 AM, Kim Barrett wrote:
>> On Dec 10, 2019, at 11:02 AM, erik.osterlund at oracle.com wrote:
>>
>> Hi,
>>
>> When the stack is walked and e.g. locals are turned into StackValues, it might be that said local was made a constant oop by the JIT. In such cases, it is read from the nmethod using ON_STRONG_OOP_REF | AS_NO_KEEPALIVE. However, these oops need to be kept alive when concurrent marking is ongoing.
>> While I haven't seen crashes obviously linked to this yet, I don't think we should wait until we do, because it certainly will eventually.
>>
>> Bug:
>> https://bugs.openjdk.java.net/browse/JDK-8235669
>>
>> Webrev:
>> http://cr.openjdk.java.net/~eosterlund/8235669/webrev.00/
>>
>> Thanks,
>> /Erik
> Change looks good.
>
> I think it's kind of gross that oop_at returns an AS_NO_KEEPALIVE
> value to some more or less arbitrary context without any indication
> that this can happen.  The scope of AS_NO_KEEPALIVE values really
> ought to be more constrained than that.
>
> I wonder if oop_at should do the phantom access, and there should be a
> different function for use by those places that want / can cope with
> an AS_NO_KEEPALIVE value. So I think there might be some naming / API
> issues in this neighborhood, but that can be addressed in a post-14 RFE.
>
>


From erik.osterlund at oracle.com  Tue Jan  7 09:22:44 2020
From: erik.osterlund at oracle.com (erik.osterlund at oracle.com)
Date: Tue, 7 Jan 2020 10:22:44 +0100
Subject: RFR: 8210498: nmethod entry barriers
In-Reply-To: <22b7f727-929c-b159-316e-78b76543b4fd@redhat.com>
References: <5BB77E18.6040401@oracle.com>
 <399d9f0b-e66b-30e1-16b3-4873845367a9@redhat.com>
 <5BBB538B.5070404@oracle.com>
 <22b7f727-929c-b159-316e-78b76543b4fd@redhat.com>
Message-ID: <f89259d1-54a5-cd8e-99e2-7f570b127cf2@oracle.com>

Hi Andrew,

On 1/6/20 7:16 PM, Andrew Haley wrote:
> On 10/8/18 1:54 PM, Erik ?sterlund wrote:
>> Also note that the implementation space of the barrier itself has some
>> flexibility. Rickard's first prototype involved having an unconditional
>> branch patched in over a nop. Since the nop is removed in the frontend,
>> it seemed like the most conservative starting point. But since there was
>> no measurable difference to the conditional branch, that was more
>> favourable in the end, since it hade the additional advantage of not
>> requiring a code cache walk in the safepoint. But if you have a platform
>> where the trade off is not as obvious, both mechanisms could easily be
>> supported.
> Can you describe this mechanism a little more? I don't really understand
> how that would work, even on x86.

Presuming you would like to hear about the solution we didn't go for 
(unconditional branch)...

The nmethod entry barriers are armed in a safepoint operation. Today 
that safepoint operation
flips some epoch counter that the conditional branch will consider armed 
once the safepoint is
released.

In the alternative solution that biases the cost towards arming, instead 
of calling, you would
instead walk the code cache and explicitly arm nmethods by patching in a 
jump over nops in the
verified entry (for all nmethods).

Disarming would be done by patching back nops over the jump on 
individual nmethods as they
become safe to disarm.

In the end, the hypothetical overhead of performing a conditional branch 
instead of executing
nops was never observed to make a difference, and therefore we went with 
the conditional branch
as the latency cost of walking the code cache was conversely not 
hypothetical.

Note that since the entry barrier is used to protect mutators from 
observing stale oops, the
current solution (and this alternative solution) relies on instruction 
cache coherency. Since
there are oops embedded in the code stream, we rely on the disarming 
being a code modification
such that a mutator observing the disarmed barrier implies it will also 
observe the fixed oops.

If you are looking for an AArch64 solution, Stuart Monteith is cooking 
up a solution that we
discussed, which does not rely on that for AArch64, which you might be 
interested in. Although,
perhaps that is not what you are fishing for.

Hope this helps.

Thanks,
/Erik

From matthias.baesken at sap.com  Tue Jan  7 09:27:50 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 7 Jan 2020 09:27:50 +0000
Subject: RFR [XS]: 8236709: struct SwitchRange in HS violates C++ One
 Definition Rule  - was RE: struct SwitchRange  and  C++ One Definition Rule
 [-Wodr]
Message-ID: <AM6PR02MB420004E4B2B911D7ADA71EAD933F0@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello, please review  this small fix for an issue  that I was running into  when experimenting  with  gcc8  and   the -flto compiler flag .
When building with those flags,  the gcc8 warns that  the  SwitchRange   classes  in HS code   violate the C++ One Definition Rule .
So I renamed one of those 2  SwitchRange   classes .

 Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8236709

http://cr.openjdk.java.net/~mbaesken/webrevs/8236709.0/


Thanks, Matthias


> > Hello,  when  experimenting  with  gcc8  and   the -flto compiler flag I was running into these  warnings  in the c1  coding :
> >
> > /open_jdk/jdk_3/jdk/src/hotspot/share/c1/c1_LIRGenerator.hpp:50:7:
> > warning: type 'struct SwitchRange' violates the C++ One Definition Rule [-
> > Wodr]
> > class SwitchRange: public CompilationResourceObj {
> >        ^
> > /open_jdk/jdk_3/jdk/src/hotspot/share/opto/parse2.cpp:319: note: a
> > different type is defined in another translation unit
> > class SwitchRange : public StackObj {
> >
> >
> >
> /usr/work/d040975/open_jdk/jdk_3/jdk/src/hotspot/share/c1/c1_LIRGener
> > ator.hpp:52:7: note: the first difference of corresponding definitions is field
> > '_low_key'
> >    int _low_key;
> >        ^
> > /open_jdk/jdk_3/jdk/src/hotspot/share/opto/parse2.cpp:321: note: a
> > field with different name is defined in another translation unit
> >    jint _lo;                     // inclusive lower limit
> >
> >
> > Do  you think this should be fixed ( renaming  one SwitchRange  ) ?
> >
> >
> > Martin suggested that  even without  flto added   it could be problematic .
>
> Yes, please file a bug and let's get this fixed quickly.  This sort of thing can
> lead
> to really weird looking problems, like using the same vtable (which one
> picked
> ?at random?) for instances of both classes.


From aph at redhat.com  Tue Jan  7 10:04:22 2020
From: aph at redhat.com (Andrew Haley)
Date: Tue, 7 Jan 2020 10:04:22 +0000
Subject: RFR: 8210498: nmethod entry barriers
In-Reply-To: <f89259d1-54a5-cd8e-99e2-7f570b127cf2@oracle.com>
References: <5BB77E18.6040401@oracle.com>
 <399d9f0b-e66b-30e1-16b3-4873845367a9@redhat.com>
 <5BBB538B.5070404@oracle.com>
 <22b7f727-929c-b159-316e-78b76543b4fd@redhat.com>
 <f89259d1-54a5-cd8e-99e2-7f570b127cf2@oracle.com>
Message-ID: <b4d1ca17-aa2b-b3e1-a8b8-104dd3fc60c9@redhat.com>

On 1/7/20 9:22 AM, erik.osterlund at oracle.com wrote:
> Presuming you would like to hear about the solution we didn't go for 
> (unconditional branch)...
> 
> The nmethod entry barriers are armed in a safepoint operation. Today 
> that safepoint operation
> flips some epoch counter that the conditional branch will consider armed 
> once the safepoint is
> released.
> 
> In the alternative solution that biases the cost towards arming, instead 
> of calling, you would
> instead walk the code cache and explicitly arm nmethods by patching in a 
> jump over nops in the
> verified entry (for all nmethods).
> 
> Disarming would be done by patching back nops over the jump on 
> individual nmethods as they
> become safe to disarm.

Aha! That'd be a much simpler method for AArch64, for sure. We already have
a nop at the start of every method, so we could rewrite it as a simple
jump.

> In the end, the hypothetical overhead of performing a conditional branch 
> instead of executing
> nops was never observed to make a difference, and therefore we went with 
> the conditional branch
> as the latency cost of walking the code cache was conversely not 
> hypothetical.

Totally. However, that walk is not inline in the mutator code, and there's
no reason not to run it concurrently.

> Note that since the entry barrier is used to protect mutators from 
> observing stale oops, the
> current solution (and this alternative solution) relies on instruction 
> cache coherency.

I'm not sure it [the alternative] does, exactly. It requires that
mutators see the changed jump once the cache flush has been done, but
that's less of a requirement than icache coherency.

> Since
> there are oops embedded in the code stream, we rely on the disarming 
> being a code modification
> such that a mutator observing the disarmed barrier implies it will also 
> observe the fixed oops.

Sure, but there's little reason that oops should be embedded in the code
stream. It's an optimization, but a pretty minor one.

> If you are looking for an AArch64 solution, Stuart Monteith is cooking 
> up a solution that we
> discussed, which does not rely on that for AArch64, which you might be 
> interested in.

i haven't seen that. Was it discussed anywhere? I'll ask him.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From per.liden at oracle.com  Tue Jan  7 10:25:08 2020
From: per.liden at oracle.com (Per Liden)
Date: Tue, 7 Jan 2020 11:25:08 +0100
Subject: RFR: 8210498: nmethod entry barriers
In-Reply-To: <b4d1ca17-aa2b-b3e1-a8b8-104dd3fc60c9@redhat.com>
References: <5BB77E18.6040401@oracle.com>
 <399d9f0b-e66b-30e1-16b3-4873845367a9@redhat.com>
 <5BBB538B.5070404@oracle.com>
 <22b7f727-929c-b159-316e-78b76543b4fd@redhat.com>
 <f89259d1-54a5-cd8e-99e2-7f570b127cf2@oracle.com>
 <b4d1ca17-aa2b-b3e1-a8b8-104dd3fc60c9@redhat.com>
Message-ID: <1b5081ca-7f4b-64ec-b212-b7dc4110ac4c@oracle.com>

Hi,

On 1/7/20 11:04 AM, Andrew Haley wrote:
> On 1/7/20 9:22 AM, erik.osterlund at oracle.com wrote:
[...]
>> In the alternative solution that biases the cost towards arming, instead
>> of calling, you would
>> instead walk the code cache and explicitly arm nmethods by patching in a
>> jump over nops in the
>> verified entry (for all nmethods).
>>
>> Disarming would be done by patching back nops over the jump on
>> individual nmethods as they
>> become safe to disarm.
> 
> Aha! That'd be a much simpler method for AArch64, for sure. We already have
> a nop at the start of every method, so we could rewrite it as a simple
> jump.

But as Erik hinted, the main problem with this alternative is that 
arming becomes an O(n) stop-the-world operation (where n is the number 
of nmethods) rather than O(1), which is highly undesirable for ZGC.

cheers,
Per

From aph at redhat.com  Tue Jan  7 10:36:57 2020
From: aph at redhat.com (Andrew Haley)
Date: Tue, 7 Jan 2020 10:36:57 +0000
Subject: RFR: 8210498: nmethod entry barriers
In-Reply-To: <1b5081ca-7f4b-64ec-b212-b7dc4110ac4c@oracle.com>
References: <5BB77E18.6040401@oracle.com>
 <399d9f0b-e66b-30e1-16b3-4873845367a9@redhat.com>
 <5BBB538B.5070404@oracle.com>
 <22b7f727-929c-b159-316e-78b76543b4fd@redhat.com>
 <f89259d1-54a5-cd8e-99e2-7f570b127cf2@oracle.com>
 <b4d1ca17-aa2b-b3e1-a8b8-104dd3fc60c9@redhat.com>
 <1b5081ca-7f4b-64ec-b212-b7dc4110ac4c@oracle.com>
Message-ID: <772a1cb2-9486-6d38-32fa-a7f6bb56b7fd@redhat.com>

On 1/7/20 10:25 AM, Per Liden wrote:
> 
> On 1/7/20 11:04 AM, Andrew Haley wrote:
>> On 1/7/20 9:22 AM, erik.osterlund at oracle.com wrote:
> [...]
>>> In the alternative solution that biases the cost towards arming, instead
>>> of calling, you would
>>> instead walk the code cache and explicitly arm nmethods by patching in a
>>> jump over nops in the
>>> verified entry (for all nmethods).
>>>
>>> Disarming would be done by patching back nops over the jump on
>>> individual nmethods as they
>>> become safe to disarm.
>>
>> Aha! That'd be a much simpler method for AArch64, for sure. We already have
>> a nop at the start of every method, so we could rewrite it as a simple
>> jump.
> 
> But as Erik hinted, the main problem with this alternative is that 
> arming becomes an O(n) stop-the-world operation

O(n) I understand, but why stop the world?

> (where n is the number 
> of nmethods) rather than O(1), which is highly undesirable for ZGC.

Yeah, I get that. It's a choice between the devil and the deep blue sea.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From erik.osterlund at oracle.com  Tue Jan  7 11:15:10 2020
From: erik.osterlund at oracle.com (erik.osterlund at oracle.com)
Date: Tue, 7 Jan 2020 12:15:10 +0100
Subject: RFR: 8210498: nmethod entry barriers
In-Reply-To: <b4d1ca17-aa2b-b3e1-a8b8-104dd3fc60c9@redhat.com>
References: <5BB77E18.6040401@oracle.com>
 <399d9f0b-e66b-30e1-16b3-4873845367a9@redhat.com>
 <5BBB538B.5070404@oracle.com>
 <22b7f727-929c-b159-316e-78b76543b4fd@redhat.com>
 <f89259d1-54a5-cd8e-99e2-7f570b127cf2@oracle.com>
 <b4d1ca17-aa2b-b3e1-a8b8-104dd3fc60c9@redhat.com>
Message-ID: <384efad9-2957-1159-1577-ab0207ad0d66@oracle.com>

Hi Andrew,

On 1/7/20 11:04 AM, Andrew Haley wrote:
> On 1/7/20 9:22 AM, erik.osterlund at oracle.com wrote:
>> Presuming you would like to hear about the solution we didn't go for
>> (unconditional branch)...
>>
>> The nmethod entry barriers are armed in a safepoint operation. Today
>> that safepoint operation
>> flips some epoch counter that the conditional branch will consider armed
>> once the safepoint is
>> released.
>>
>> In the alternative solution that biases the cost towards arming, instead
>> of calling, you would
>> instead walk the code cache and explicitly arm nmethods by patching in a
>> jump over nops in the
>> verified entry (for all nmethods).
>>
>> Disarming would be done by patching back nops over the jump on
>> individual nmethods as they
>> become safe to disarm.
> Aha! That'd be a much simpler method for AArch64, for sure. We already have
> a nop at the start of every method, so we could rewrite it as a simple
> jump.

I'm presuming you are referring to the nop that we plaster a jump over 
when making thenmethod not_entrant.
Conceptually that would be absolutely fine. However, note that
1) This nop is before the frame of the callee is constructed. The way 
the barrier works today is that
 ?? we wait for the frame to be constructed before calling the slow 
path, mostly out of convenience because
 ?? then the dispatch machinery has selected the callee nmethod of the 
call, and we can easily acquire the
 ?? callee nmethod from the slowpath code. Other miss handlers used in 
the VM typically resolve the call instead
 ?? figuring out what the callee nmethod should be (before the selection 
is done).
 ?? I think it's possible to rewrite the code in a style where this is 
done before the frame is constructed,
 ?? but I'm just noting that there might be some headache involved in that.
2) Unless care is taken, you might run into scenarios where two threads 
race, one making the nmethod not_entrant,
 ?? and another one disarming it. If the same nop is reused, you will 
need some additional synchronization to ensure
 ?? the monotonicity of the jump injected by not_entrant transitions.

In other words, while reusing that nop is possible, I think it will be 
significantly more painful compared
to putting another one right at the end of frame construction.


>
>> In the end, the hypothetical overhead of performing a conditional branch
>> instead of executing
>> nops was never observed to make a difference, and therefore we went with
>> the conditional branch
>> as the latency cost of walking the code cache was conversely not
>> hypothetical.
> Totally. However, that walk is not inline in the mutator code, and there's
> no reason not to run it concurrently.

There is. The disarming of nmethods must happen in the safepoint.The 
reason is that if an nmethod
dies due to class unloading (an oop in the nmethod is dead), then 
subsequent calls to that nmethod
from stale inline caches must be trapped so that we can unroll the frame 
and re-resolve the call.
Since marking terminates in a safepoint for all current GCs, that same 
safepoint must disarm the
nmethods before being released.

>> Note that since the entry barrier is used to protect mutators from
>> observing stale oops, the
>> current solution (and this alternative solution) relies on instruction
>> cache coherency.
> I'm not sure it [the alternative] does, exactly. It requires that
> mutators see the changed jump once the cache flush has been done, but
> that's less of a requirement than icache coherency.

Consider the following race during concurrent execution:

JavaThread 1: Take nmethod entry barrier slow path
JavaThread 1: Patch instruction oops
JavaThread 1: Patch barrier jump to nop (disarm)

JavaThread 2: Execute nop written by JavaThread 1
JavaThread 2: <--- surely we need at least isb here --->
JavaThread 2: Execute instruction oop

As long as the oops are embedded as instructions, I presume we need at 
least an isb as indicated
in my example above. Perhaps you are talking about if oops are data 
instead, in which case I am still not sure
that there are global cross-CPU acquire-like semantics when performing 
instruction cache flushing. I certainly
don't know of any such guarantees, but perhaps you know better. So I 
really don't know how we expect the oop loaded
(which is concurrently modified) to be the new value and not a stale value.

Also, as mentioned above, the arm operation really has to become 
globally observable in the safepoint.

>> Since
>> there are oops embedded in the code stream, we rely on the disarming
>> being a code modification
>> such that a mutator observing the disarmed barrier implies it will also
>> observe the fixed oops.
> Sure, but there's little reason that oops should be embedded in the code
> stream. It's an optimization, but a pretty minor one.

Agreed. I would love to see that disappear.

>> If you are looking for an AArch64 solution, Stuart Monteith is cooking
>> up a solution that we
>> discussed, which does not rely on that for AArch64, which you might be
>> interested in.
> i haven't seen that. Was it discussed anywhere? I'll ask him.

Stuart and I discussed it off-list. I proposed to him the following 
crazy solution:

1) Move oops to data by reserving a table of content (TOC) register, 
which is initialized
 ?? at nmethod entry time by loading the TOC from the nmethod (ldr). 
Each oop used by JIT
 ?? is simply loaded with ldr relative to the TOC register (it's like a 
lookup table).

2) Let the entry barrier compare the established TOC low order bits to 
the current GC phase
 ?? (load a global/TLS-local bit battern similar to the cmpl used in the 
x86 code) and take
 ?? the slow path if TOC has the wrong low order bits.

3) The TOC reserves N-1 extra slots, where N is the number of states 
observed by the barrier
 ?? (N == 3 for ZGC). This allows having different TOC pointers for each 
phase.

4) The nmethod entry barrier slow path selects a new TOC pointer and 
copies the oops in-place
 ?? such that after selecting the new TOC, each offset points as the 
same oops as before.

In this scenario, the entry barrier dodges an ldar by relying on 
dependent loads not reordering
instead. If the correct TOC is observed, then subsequent ldr of its oops 
will observe the correct
oop as well (due to being dependent). Oh, and returns into compiled code 
from non-leaf calls must
re-establish the TOC pointer with a new load in case a safepoint flipped it.

Stuart is working on something similar ish to that, but instead of the 
TOC dependent load trick,
he is exploring using ldar instead in the VEP and not reserving a TOC 
register, instead performing
PC relative loads when accessing oops, which is sanity checking if my 
solution is a premature
optimization or not before considering doing that fully, which seems to 
make sense to me as
what I proposed is a bit tricky to cook up.

So in both solutions the idea is to keep both the barrier check and the 
oops as data, and possibly
optimize away acquire with some data dependency trick.

Hope this makes sense.

Thanks,
/Erik



From thomas.schatzl at oracle.com  Tue Jan  7 11:15:41 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 7 Jan 2020 12:15:41 +0100
Subject: RFR: 8235669: G1: Stack walking API can expose AS_NO_KEEPALIVE
 oops
In-Reply-To: <76376D1F-0BDE-41CF-A702-CC121E01C1FE@oracle.com>
References: <3b2a6c47-b958-5e41-d7c3-a4d25000b17e@oracle.com>
 <76376D1F-0BDE-41CF-A702-CC121E01C1FE@oracle.com>
Message-ID: <9b97e138-885d-2a3f-a82d-da94a51010ed@oracle.com>

Hi,

On 07.01.20 03:46, Kim Barrett wrote:
>> On Dec 10, 2019, at 11:02 AM, erik.osterlund at oracle.com wrote:
>>
>> Hi,
>>
>> When the stack is walked and e.g. locals are turned into StackValues, it might be that said local was made a constant oop by the JIT. In such cases, it is read from the nmethod using ON_STRONG_OOP_REF | AS_NO_KEEPALIVE. However, these oops need to be kept alive when concurrent marking is ongoing.
>> While I haven't seen crashes obviously linked to this yet, I don't think we should wait until we do, because it certainly will eventually.
>>
>> Bug:
>> https://bugs.openjdk.java.net/browse/JDK-8235669
>>
>> Webrev:
>> http://cr.openjdk.java.net/~eosterlund/8235669/webrev.00/
>>
>> Thanks,
>> /Erik
> 
> Change looks good.

To me too.

> 
> I think it's kind of gross that oop_at returns an AS_NO_KEEPALIVE
> value to some more or less arbitrary context without any indication
> that this can happen.  The scope of AS_NO_KEEPALIVE values really
> ought to be more constrained than that.
> 
> I wonder if oop_at should do the phantom access, and there should be a
> different function for use by those places that want / can cope with
> an AS_NO_KEEPALIVE value. So I think there might be some naming / API
> issues in this neighborhood, but that can be addressed in a post-14 RFE.
> 
> 

+1

Thanks,
   Thomas

From erik.osterlund at oracle.com  Tue Jan  7 11:16:34 2020
From: erik.osterlund at oracle.com (erik.osterlund at oracle.com)
Date: Tue, 7 Jan 2020 12:16:34 +0100
Subject: RFR: 8235669: G1: Stack walking API can expose AS_NO_KEEPALIVE
 oops
In-Reply-To: <9b97e138-885d-2a3f-a82d-da94a51010ed@oracle.com>
References: <3b2a6c47-b958-5e41-d7c3-a4d25000b17e@oracle.com>
 <76376D1F-0BDE-41CF-A702-CC121E01C1FE@oracle.com>
 <9b97e138-885d-2a3f-a82d-da94a51010ed@oracle.com>
Message-ID: <13797ce3-7e95-b598-80fa-fa830ab03e02@oracle.com>

Hi Thomas,

Thanks for the review!

/Erik

On 1/7/20 12:15 PM, Thomas Schatzl wrote:
> Hi,
>
> On 07.01.20 03:46, Kim Barrett wrote:
>>> On Dec 10, 2019, at 11:02 AM, erik.osterlund at oracle.com wrote:
>>>
>>> Hi,
>>>
>>> When the stack is walked and e.g. locals are turned into 
>>> StackValues, it might be that said local was made a constant oop by 
>>> the JIT. In such cases, it is read from the nmethod using 
>>> ON_STRONG_OOP_REF | AS_NO_KEEPALIVE. However, these oops need to be 
>>> kept alive when concurrent marking is ongoing.
>>> While I haven't seen crashes obviously linked to this yet, I don't 
>>> think we should wait until we do, because it certainly will eventually.
>>>
>>> Bug:
>>> https://bugs.openjdk.java.net/browse/JDK-8235669
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~eosterlund/8235669/webrev.00/
>>>
>>> Thanks,
>>> /Erik
>>
>> Change looks good.
>
> To me too.
>
>>
>> I think it's kind of gross that oop_at returns an AS_NO_KEEPALIVE
>> value to some more or less arbitrary context without any indication
>> that this can happen.? The scope of AS_NO_KEEPALIVE values really
>> ought to be more constrained than that.
>>
>> I wonder if oop_at should do the phantom access, and there should be a
>> different function for use by those places that want / can cope with
>> an AS_NO_KEEPALIVE value. So I think there might be some naming / API
>> issues in this neighborhood, but that can be addressed in a post-14 RFE.
>>
>>
>
> +1
>
> Thanks,
> ? Thomas


From harold.seigel at oracle.com  Tue Jan  7 14:14:46 2020
From: harold.seigel at oracle.com (Harold Seigel)
Date: Tue, 7 Jan 2020 09:14:46 -0500
Subject: RFR (S) 8236224: Obsolete the FieldsAllocationStyle and
 CompactFields options
In-Reply-To: <7cc12a90-159b-cdaf-2a48-7fec1e2540c9@oracle.com>
References: <2de68ab4-f913-29d9-97ca-a8fcc5dc94b7@oracle.com>
 <731c3127-46ce-15a5-f7b3-02b11dd5cbf3@oracle.com>
 <7cc12a90-159b-cdaf-2a48-7fec1e2540c9@oracle.com>
Message-ID: <ca99ec27-2fbe-70d6-f82b-c2a54a4f55e3@oracle.com>

Hi Coleen,

The change looks good!

Thanks, Harold

On 1/7/2020 12:25 AM, coleen.phillimore at oracle.com wrote:
>
>
> On 1/7/20 12:14 AM, David Holmes wrote:
>> Hi Coleen,
>>
>> On 7/01/2020 1:25 pm, coleen.phillimore at oracle.com wrote:
>>> Summary: Remove the options and code for options deprecated in JDK 14
>>
>> Generally looks good.
>>
>>> open webrev at 
>>> http://cr.openjdk.java.net/~coleenp/2019/8236224.01/webrev
>>> bug link https://bugs.openjdk.java.net/browse/JDK-8236224
>>
>> src/hotspot/share/aot/aotCodeHeap.hpp
>>
>> ? typedef struct {
>> !?? enum { CONFIG_SIZE = 7 * jintSize + 9 };
>> ??? // 8 int values
>>
>> Now 7 int values
>>
>> ??? // byte[11] array map to boolean values here
>>
>> Now byte[10]. Or should that be byte[9]? I think the original code 
>> may be off by one.
>
> Yes, it was wrong. I fixed the comments.
>>
>> ---
>>
>> src/hotspot/share/classfile/classFileParser.cpp
>>
>> 4133?? bool allocate_oops_first = false; // was allocation_style == 0
>>
>> The comment has no context now that there is no selectable allocation 
>> style.
>>
> Removed.? It was mostly to remind myself.
>> I don't understand why you removed a bunch of classes from this check:
>>
>> 4143?????? (_class_name == 
>> vmSymbols::java_lang_AssertionStatusDirectives() ||
>> 4144??????? _class_name == vmSymbols::java_lang_Class() ||
>> 4145??????? _class_name == vmSymbols::java_lang_ClassLoader() ||
>>
>> 4147??????? _class_name == vmSymbols::java_lang_ref_SoftReference() ||
>> 4148??????? _class_name == vmSymbols::java_lang_StackTraceElement() ||
>> 4149??????? _class_name == vmSymbols::java_lang_String() ||
>> 4150??????? _class_name == vmSymbols::java_lang_Throwable() ||
>>
>> ??
>
> The classes removed no longer have hardcoded offsets so did not need 
> to follow the oops-first allocation style.? This was not cleaned up 
> when the hardcoded offsets were removed from these classes. ? Fred 
> also fixes this with his field layout patch in perhaps another place.
>
> Thanks,
> Coleen
>>
>> ---
>>
>> Thanks,
>> David
>>
>>> Ran tier1 on all oracle platforms, and 2, 3 on 
>>> linux/windows-x64-debug and hs-tier4-graal because there were jvmci 
>>> changes.
>>>
>>> thanks,
>>> Coleen
>

From frederic.parain at oracle.com  Tue Jan  7 14:33:46 2020
From: frederic.parain at oracle.com (Frederic Parain)
Date: Tue, 7 Jan 2020 09:33:46 -0500
Subject: RFR (S) 8236224: Obsolete the FieldsAllocationStyle and
 CompactFields options
In-Reply-To: <7cc12a90-159b-cdaf-2a48-7fec1e2540c9@oracle.com>
References: <2de68ab4-f913-29d9-97ca-a8fcc5dc94b7@oracle.com>
 <731c3127-46ce-15a5-f7b3-02b11dd5cbf3@oracle.com>
 <7cc12a90-159b-cdaf-2a48-7fec1e2540c9@oracle.com>
Message-ID: <fa3e8f7f-2407-ae85-0a43-5fb1586433ca@oracle.com>

Coleen,

Thank you for cleaning up this code.
Changes look good to me.

Fred

On 1/7/20 12:25 AM, coleen.phillimore at oracle.com wrote:
> 
> 
> On 1/7/20 12:14 AM, David Holmes wrote:
>> Hi Coleen,
>>
>> On 7/01/2020 1:25 pm, coleen.phillimore at oracle.com wrote:
>>> Summary: Remove the options and code for options deprecated in JDK 14
>>
>> Generally looks good.
>>
>>> open webrev at 
>>> http://cr.openjdk.java.net/~coleenp/2019/8236224.01/webrev
>>> bug link https://bugs.openjdk.java.net/browse/JDK-8236224
>>
>> src/hotspot/share/aot/aotCodeHeap.hpp
>>
>> ? typedef struct {
>> !?? enum { CONFIG_SIZE = 7 * jintSize + 9 };
>> ??? // 8 int values
>>
>> Now 7 int values
>>
>> ??? // byte[11] array map to boolean values here
>>
>> Now byte[10]. Or should that be byte[9]? I think the original code may 
>> be off by one.
> 
> Yes, it was wrong. I fixed the comments.
>>
>> ---
>>
>> src/hotspot/share/classfile/classFileParser.cpp
>>
>> 4133?? bool allocate_oops_first = false; // was allocation_style == 0
>>
>> The comment has no context now that there is no selectable allocation 
>> style.
>>
> Removed.? It was mostly to remind myself.
>> I don't understand why you removed a bunch of classes from this check:
>>
>> 4143?????? (_class_name == 
>> vmSymbols::java_lang_AssertionStatusDirectives() ||
>> 4144??????? _class_name == vmSymbols::java_lang_Class() ||
>> 4145??????? _class_name == vmSymbols::java_lang_ClassLoader() ||
>>
>> 4147??????? _class_name == vmSymbols::java_lang_ref_SoftReference() ||
>> 4148??????? _class_name == vmSymbols::java_lang_StackTraceElement() ||
>> 4149??????? _class_name == vmSymbols::java_lang_String() ||
>> 4150??????? _class_name == vmSymbols::java_lang_Throwable() ||
>>
>> ??
> 
> The classes removed no longer have hardcoded offsets so did not need to 
> follow the oops-first allocation style.? This was not cleaned up when 
> the hardcoded offsets were removed from these classes. ? Fred also fixes 
> this with his field layout patch in perhaps another place.
> 
> Thanks,
> Coleen
>>
>> ---
>>
>> Thanks,
>> David
>>
>>> Ran tier1 on all oracle platforms, and 2, 3 on 
>>> linux/windows-x64-debug and hs-tier4-graal because there were jvmci 
>>> changes.
>>>
>>> thanks,
>>> Coleen
> 

From coleen.phillimore at oracle.com  Tue Jan  7 14:41:57 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Tue, 7 Jan 2020 09:41:57 -0500
Subject: RFR (S) 8236224: Obsolete the FieldsAllocationStyle and
 CompactFields options
In-Reply-To: <ca99ec27-2fbe-70d6-f82b-c2a54a4f55e3@oracle.com>
References: <2de68ab4-f913-29d9-97ca-a8fcc5dc94b7@oracle.com>
 <731c3127-46ce-15a5-f7b3-02b11dd5cbf3@oracle.com>
 <7cc12a90-159b-cdaf-2a48-7fec1e2540c9@oracle.com>
 <ca99ec27-2fbe-70d6-f82b-c2a54a4f55e3@oracle.com>
Message-ID: <a23a290b-e80f-b00e-e917-4a2f16145e07@oracle.com>

Thanks Harold!
Coleen

On 1/7/20 9:14 AM, Harold Seigel wrote:
> Hi Coleen,
>
> The change looks good!
>
> Thanks, Harold
>
> On 1/7/2020 12:25 AM, coleen.phillimore at oracle.com wrote:
>>
>>
>> On 1/7/20 12:14 AM, David Holmes wrote:
>>> Hi Coleen,
>>>
>>> On 7/01/2020 1:25 pm, coleen.phillimore at oracle.com wrote:
>>>> Summary: Remove the options and code for options deprecated in JDK 14
>>>
>>> Generally looks good.
>>>
>>>> open webrev at 
>>>> http://cr.openjdk.java.net/~coleenp/2019/8236224.01/webrev
>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8236224
>>>
>>> src/hotspot/share/aot/aotCodeHeap.hpp
>>>
>>> ? typedef struct {
>>> !?? enum { CONFIG_SIZE = 7 * jintSize + 9 };
>>> ??? // 8 int values
>>>
>>> Now 7 int values
>>>
>>> ??? // byte[11] array map to boolean values here
>>>
>>> Now byte[10]. Or should that be byte[9]? I think the original code 
>>> may be off by one.
>>
>> Yes, it was wrong. I fixed the comments.
>>>
>>> ---
>>>
>>> src/hotspot/share/classfile/classFileParser.cpp
>>>
>>> 4133?? bool allocate_oops_first = false; // was allocation_style == 0
>>>
>>> The comment has no context now that there is no selectable 
>>> allocation style.
>>>
>> Removed.? It was mostly to remind myself.
>>> I don't understand why you removed a bunch of classes from this check:
>>>
>>> 4143?????? (_class_name == 
>>> vmSymbols::java_lang_AssertionStatusDirectives() ||
>>> 4144??????? _class_name == vmSymbols::java_lang_Class() ||
>>> 4145??????? _class_name == vmSymbols::java_lang_ClassLoader() ||
>>>
>>> 4147??????? _class_name == vmSymbols::java_lang_ref_SoftReference() ||
>>> 4148??????? _class_name == vmSymbols::java_lang_StackTraceElement() ||
>>> 4149??????? _class_name == vmSymbols::java_lang_String() ||
>>> 4150??????? _class_name == vmSymbols::java_lang_Throwable() ||
>>>
>>> ??
>>
>> The classes removed no longer have hardcoded offsets so did not need 
>> to follow the oops-first allocation style.? This was not cleaned up 
>> when the hardcoded offsets were removed from these classes. ? Fred 
>> also fixes this with his field layout patch in perhaps another place.
>>
>> Thanks,
>> Coleen
>>>
>>> ---
>>>
>>> Thanks,
>>> David
>>>
>>>> Ran tier1 on all oracle platforms, and 2, 3 on 
>>>> linux/windows-x64-debug and hs-tier4-graal because there were jvmci 
>>>> changes.
>>>>
>>>> thanks,
>>>> Coleen
>>


From coleen.phillimore at oracle.com  Tue Jan  7 14:42:33 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Tue, 7 Jan 2020 09:42:33 -0500
Subject: RFR (S) 8236224: Obsolete the FieldsAllocationStyle and
 CompactFields options
In-Reply-To: <fa3e8f7f-2407-ae85-0a43-5fb1586433ca@oracle.com>
References: <2de68ab4-f913-29d9-97ca-a8fcc5dc94b7@oracle.com>
 <731c3127-46ce-15a5-f7b3-02b11dd5cbf3@oracle.com>
 <7cc12a90-159b-cdaf-2a48-7fec1e2540c9@oracle.com>
 <fa3e8f7f-2407-ae85-0a43-5fb1586433ca@oracle.com>
Message-ID: <d8723ad0-ef38-84a0-c544-65e44913afca@oracle.com>

Thanks Fred and for the prereview and hope it helps you a little. Now 
you don't have to worry about these flags.
Coleen

On 1/7/20 9:33 AM, Frederic Parain wrote:
> Coleen,
>
> Thank you for cleaning up this code.
> Changes look good to me.
>
> Fred
>
> On 1/7/20 12:25 AM, coleen.phillimore at oracle.com wrote:
>>
>>
>> On 1/7/20 12:14 AM, David Holmes wrote:
>>> Hi Coleen,
>>>
>>> On 7/01/2020 1:25 pm, coleen.phillimore at oracle.com wrote:
>>>> Summary: Remove the options and code for options deprecated in JDK 14
>>>
>>> Generally looks good.
>>>
>>>> open webrev at 
>>>> http://cr.openjdk.java.net/~coleenp/2019/8236224.01/webrev
>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8236224
>>>
>>> src/hotspot/share/aot/aotCodeHeap.hpp
>>>
>>> ? typedef struct {
>>> !?? enum { CONFIG_SIZE = 7 * jintSize + 9 };
>>> ??? // 8 int values
>>>
>>> Now 7 int values
>>>
>>> ??? // byte[11] array map to boolean values here
>>>
>>> Now byte[10]. Or should that be byte[9]? I think the original code 
>>> may be off by one.
>>
>> Yes, it was wrong. I fixed the comments.
>>>
>>> ---
>>>
>>> src/hotspot/share/classfile/classFileParser.cpp
>>>
>>> 4133?? bool allocate_oops_first = false; // was allocation_style == 0
>>>
>>> The comment has no context now that there is no selectable 
>>> allocation style.
>>>
>> Removed.? It was mostly to remind myself.
>>> I don't understand why you removed a bunch of classes from this check:
>>>
>>> 4143?????? (_class_name == 
>>> vmSymbols::java_lang_AssertionStatusDirectives() ||
>>> 4144??????? _class_name == vmSymbols::java_lang_Class() ||
>>> 4145??????? _class_name == vmSymbols::java_lang_ClassLoader() ||
>>>
>>> 4147??????? _class_name == vmSymbols::java_lang_ref_SoftReference() ||
>>> 4148??????? _class_name == vmSymbols::java_lang_StackTraceElement() ||
>>> 4149??????? _class_name == vmSymbols::java_lang_String() ||
>>> 4150??????? _class_name == vmSymbols::java_lang_Throwable() ||
>>>
>>> ??
>>
>> The classes removed no longer have hardcoded offsets so did not need 
>> to follow the oops-first allocation style.? This was not cleaned up 
>> when the hardcoded offsets were removed from these classes. ? Fred 
>> also fixes this with his field layout patch in perhaps another place.
>>
>> Thanks,
>> Coleen
>>>
>>> ---
>>>
>>> Thanks,
>>> David
>>>
>>>> Ran tier1 on all oracle platforms, and 2, 3 on 
>>>> linux/windows-x64-debug and hs-tier4-graal because there were jvmci 
>>>> changes.
>>>>
>>>> thanks,
>>>> Coleen
>>


From coleen.phillimore at oracle.com  Tue Jan  7 14:43:10 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Tue, 7 Jan 2020 09:43:10 -0500
Subject: RFR (S) 8236224: Obsolete the FieldsAllocationStyle and
 CompactFields options
In-Reply-To: <cd6e1c54-fec9-22d0-998e-38f6757fe072@oracle.com>
References: <2de68ab4-f913-29d9-97ca-a8fcc5dc94b7@oracle.com>
 <731c3127-46ce-15a5-f7b3-02b11dd5cbf3@oracle.com>
 <7cc12a90-159b-cdaf-2a48-7fec1e2540c9@oracle.com>
 <cd6e1c54-fec9-22d0-998e-38f6757fe072@oracle.com>
Message-ID: <8c288b90-ea49-1eb9-aa10-d02d061ca22d@oracle.com>

Thanks for the code review, David.
Coleen

On 1/7/20 1:05 AM, David Holmes wrote:
> Hi Coleen,
>
> On 7/01/2020 3:25 pm, coleen.phillimore at oracle.com wrote:
>>
>>
>> On 1/7/20 12:14 AM, David Holmes wrote:
>>> Hi Coleen,
>>>
>>> On 7/01/2020 1:25 pm, coleen.phillimore at oracle.com wrote:
>>>> Summary: Remove the options and code for options deprecated in JDK 14
>>>
>>> Generally looks good.
>>>
>>>> open webrev at 
>>>> http://cr.openjdk.java.net/~coleenp/2019/8236224.01/webrev
>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8236224
>>>
>>> src/hotspot/share/aot/aotCodeHeap.hpp
>>>
>>> ? typedef struct {
>>> !?? enum { CONFIG_SIZE = 7 * jintSize + 9 };
>>> ??? // 8 int values
>>>
>>> Now 7 int values
>>>
>>> ??? // byte[11] array map to boolean values here
>>>
>>> Now byte[10]. Or should that be byte[9]? I think the original code 
>>> may be off by one.
>>
>> Yes, it was wrong. I fixed the comments.
>>>
>>> ---
>>>
>>> src/hotspot/share/classfile/classFileParser.cpp
>>>
>>> 4133?? bool allocate_oops_first = false; // was allocation_style == 0
>>>
>>> The comment has no context now that there is no selectable 
>>> allocation style.
>>>
>> Removed.? It was mostly to remind myself.
>>> I don't understand why you removed a bunch of classes from this check:
>>>
>>> 4143?????? (_class_name == 
>>> vmSymbols::java_lang_AssertionStatusDirectives() ||
>>> 4144??????? _class_name == vmSymbols::java_lang_Class() ||
>>> 4145??????? _class_name == vmSymbols::java_lang_ClassLoader() ||
>>>
>>> 4147??????? _class_name == vmSymbols::java_lang_ref_SoftReference() ||
>>> 4148??????? _class_name == vmSymbols::java_lang_StackTraceElement() ||
>>> 4149??????? _class_name == vmSymbols::java_lang_String() ||
>>> 4150??????? _class_name == vmSymbols::java_lang_Throwable() ||
>>>
>>> ??
>>
>> The classes removed no longer have hardcoded offsets so did not need 
>> to follow the oops-first allocation style.? This was not cleaned up 
>> when the hardcoded offsets were removed from these classes. ? Fred 
>> also fixes this with his field layout patch in perhaps another place.
>
> Okay thanks for clarifying.
>
> David
>
>> Thanks,
>> Coleen
>>>
>>> ---
>>>
>>> Thanks,
>>> David
>>>
>>>> Ran tier1 on all oracle platforms, and 2, 3 on 
>>>> linux/windows-x64-debug and hs-tier4-graal because there were jvmci 
>>>> changes.
>>>>
>>>> thanks,
>>>> Coleen
>>


From david.holmes at oracle.com  Wed Jan  8 00:23:46 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 8 Jan 2020 10:23:46 +1000
Subject: RFR [XS]: 8236709: struct SwitchRange in HS violates C++ One
 Definition Rule - was RE: struct SwitchRange and C++ One Definition Rule
 [-Wodr]
In-Reply-To: <AM6PR02MB420004E4B2B911D7ADA71EAD933F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB420004E4B2B911D7ADA71EAD933F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <5d1e6853-ae4d-e229-c6b6-a8ea66402e8a@oracle.com>

Hi Matthias,

On 7/01/2020 7:27 pm, Baesken, Matthias wrote:
> Hello, please review  this small fix for an issue  that I was running into  when experimenting  with  gcc8  and   the -flto compiler flag .
> When building with those flags,  the gcc8 warns that  the  SwitchRange   classes  in HS code   violate the C++ One Definition Rule .
> So I renamed one of those 2  SwitchRange   classes .

Could you instead put the ./share/opto/parse2.cpp version inside an 
anonymous namespace? It seems to me that both SwitchRange classes are 
intended for use in a single compilation unit, so if we can make that 
more obvious that seems better than solving the name clash.

Otherwise I have to wonder whether you could rename the C1 version as 
you have in the hpp file (to avoid the global name clash) but add a 
typedef to allow the cpp file (and latter part of the header) to be 
unchanged? Also minor nit:

   class Invoke;
! class SwitchRangeC1;
   class LIRItem;

The forward declaration seems unnecessary.

Thanks,
David
-----

>   Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8236709
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8236709.0/
> 
> 
> Thanks, Matthias
> 
> 
>>> Hello,  when  experimenting  with  gcc8  and   the -flto compiler flag I was running into these  warnings  in the c1  coding :
>>>
>>> /open_jdk/jdk_3/jdk/src/hotspot/share/c1/c1_LIRGenerator.hpp:50:7:
>>> warning: type 'struct SwitchRange' violates the C++ One Definition Rule [-
>>> Wodr]
>>> class SwitchRange: public CompilationResourceObj {
>>>         ^
>>> /open_jdk/jdk_3/jdk/src/hotspot/share/opto/parse2.cpp:319: note: a
>>> different type is defined in another translation unit
>>> class SwitchRange : public StackObj {
>>>
>>>
>>>
>> /usr/work/d040975/open_jdk/jdk_3/jdk/src/hotspot/share/c1/c1_LIRGener
>>> ator.hpp:52:7: note: the first difference of corresponding definitions is field
>>> '_low_key'
>>>     int _low_key;
>>>         ^
>>> /open_jdk/jdk_3/jdk/src/hotspot/share/opto/parse2.cpp:321: note: a
>>> field with different name is defined in another translation unit
>>>     jint _lo;                     // inclusive lower limit
>>>
>>>
>>> Do  you think this should be fixed ( renaming  one SwitchRange  ) ?
>>>
>>>
>>> Martin suggested that  even without  flto added   it could be problematic .
>>
>> Yes, please file a bug and let's get this fixed quickly.  This sort of thing can
>> lead
>> to really weird looking problems, like using the same vtable (which one
>> picked
>> ?at random?) for instances of both classes.
> 

From coleen.phillimore at oracle.com  Wed Jan  8 00:23:27 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Tue, 7 Jan 2020 19:23:27 -0500
Subject: RFR 8232759: Remove or simplify GC.class_stats
Message-ID: <2faa21b1-c5d3-0caf-d7ed-6e7852814669@oracle.com>

Summary: Make the GC.class_stats option obsolete

open webrev at http://cr.openjdk.java.net/~coleenp/2019/8232759.01/webrev
bug link https://bugs.openjdk.java.net/browse/JDK-8232759

Tested with tier1 on all Oracle platforms and tier2,3 on linux-x64-debug.

Thanks,
Coleen


From kim.barrett at oracle.com  Wed Jan  8 00:58:30 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Tue, 7 Jan 2020 19:58:30 -0500
Subject: RFR [XS]: 8236709: struct SwitchRange in HS violates C++ One
 Definition Rule - was RE: struct SwitchRange and C++ One Definition Rule
 [-Wodr]
In-Reply-To: <5d1e6853-ae4d-e229-c6b6-a8ea66402e8a@oracle.com>
References: <AM6PR02MB420004E4B2B911D7ADA71EAD933F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <5d1e6853-ae4d-e229-c6b6-a8ea66402e8a@oracle.com>
Message-ID: <37D2B3E8-2CB8-47DB-824F-78ABD737B094@oracle.com>

> On Jan 7, 2020, at 7:23 PM, David Holmes <david.holmes at oracle.com> wrote:
> 
> Hi Matthias,
> 
> On 7/01/2020 7:27 pm, Baesken, Matthias wrote:
>> Hello, please review  this small fix for an issue  that I was running into  when experimenting  with  gcc8  and   the -flto compiler flag .
>> When building with those flags,  the gcc8 warns that  the  SwitchRange   classes  in HS code   violate the C++ One Definition Rule .
>> So I renamed one of those 2  SwitchRange   classes .
> 
> Could you instead put the ./share/opto/parse2.cpp version inside an anonymous namespace? It seems to me that both SwitchRange classes are intended for use in a single compilation unit, so if we can make that more obvious that seems better than solving the name clash.

At one time I considered proposing changing the HotSpot Style Guide to
permit (and indeed encourage, as there are some performance benefits too)
the use of anonymous namespaces. However, I discovered that debuggers
don't seem to like them at all, so dropped that idea.

https://groups.google.com/forum/#!topic/mozilla.dev.platform/KsaG3lEEaRM
Suggests Visual Studio debugger might not be able to refer to
anonoymous namespace symbols, so can't set breakpoints in them &etc.
Though the discussion seems to go back and forth on that.

https://firefox-source-docs.mozilla.org/tools/lint/coding-style/coding_style_cpp.html
Search for "Anonymous namespaces"
Suggests preferring "static" to anonymous namespaces where applicable,
because of poor debugger support for anonymous namespaces.

https://sourceware.org/bugzilla/show_bug.cgi?id=16874
Bug for similar gdb problems.


From kim.barrett at oracle.com  Wed Jan  8 01:16:31 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Tue, 7 Jan 2020 20:16:31 -0500
Subject: RFR [XS]: 8236709: struct SwitchRange in HS violates C++ One
 Definition Rule  - was RE: struct SwitchRange  and  C++ One Definition Rule
 [-Wodr]
In-Reply-To: <AM6PR02MB420004E4B2B911D7ADA71EAD933F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB420004E4B2B911D7ADA71EAD933F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <062FA66D-4C78-4B11-AB2B-AE9B792596A6@oracle.com>

> On Jan 7, 2020, at 4:27 AM, Baesken, Matthias <matthias.baesken at sap.com> wrote:
> 
> Hello, please review  this small fix for an issue  that I was running into  when experimenting  with  gcc8  and   the -flto compiler flag .
> When building with those flags,  the gcc8 warns that  the  SwitchRange   classes  in HS code   violate the C++ One Definition Rule .
> So I renamed one of those 2  SwitchRange   classes .
> 
> Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8236709
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8236709.0/

I think I would prefer the C1 "namespace" disambiguator to be a prefix
rather than a suffix, to be consistent with usage in other parts of
HotSpot.  (GC code does that somewhat consistently, for example.)  I
don't recall seeing "namespace" suffixes used anywhere in HotSpot.

I've occasionally considered proposing that C2 code be wrapped in a
namespace. It grabs a bunch of very generic global type names ("Type",
"Block", &etc; really!?) that can be annoying. (Though the diet for
precompiled.hpp helped with that.) But anything like that is probably
out of scope for the immediate problem.

Giving C2 primacy of place (renaming the C1 class and letting C2 keep
the "good" name) seems consistent with existing practice.


From david.holmes at oracle.com  Wed Jan  8 05:55:49 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 8 Jan 2020 15:55:49 +1000
Subject: RFR 8232759: Remove or simplify GC.class_stats
In-Reply-To: <2faa21b1-c5d3-0caf-d7ed-6e7852814669@oracle.com>
References: <2faa21b1-c5d3-0caf-d7ed-6e7852814669@oracle.com>
Message-ID: <6b3b3556-1ae2-41a1-bfef-30c60e7e4526@oracle.com>

Hi Coleen,

On 8/01/2020 10:23 am, coleen.phillimore at oracle.com wrote:
> Summary: Make the GC.class_stats option obsolete
> 
> open webrev at http://cr.openjdk.java.net/~coleenp/2019/8232759.01/webrev
> bug link https://bugs.openjdk.java.net/browse/JDK-8232759

The fan out from that was larger than I was expecting :)

Change appears fine. May I suggest updating the bug to get rid of the 
"or simplify" part.

Thanks,
David

> Tested with tier1 on all Oracle platforms and tier2,3 on linux-x64-debug.
> 
> Thanks,
> Coleen
> 

From matthias.baesken at sap.com  Wed Jan  8 08:10:30 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 8 Jan 2020 08:10:30 +0000
Subject: RFR [XS]: 8236709: struct SwitchRange in HS violates C++ One
 Definition Rule  - was RE: struct SwitchRange  and  C++ One Definition Rule
 [-Wodr]
In-Reply-To: <062FA66D-4C78-4B11-AB2B-AE9B792596A6@oracle.com>
References: <AM6PR02MB420004E4B2B911D7ADA71EAD933F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <062FA66D-4C78-4B11-AB2B-AE9B792596A6@oracle.com>
Message-ID: <AM6PR02MB42008DB43063A59D1DC845E7933E0@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello, thanks for the input .

I renamed SwitchRangeC1   to  C1SwitchRange  and  removed the unnecessary forward declaration .

New webrev :

http://cr.openjdk.java.net/~mbaesken/webrevs/8236709.1/


Best regards, Matthias




> >
> > Hello, please review  this small fix for an issue  that I was running into  when
> experimenting  with  gcc8  and   the -flto compiler flag .
> > When building with those flags,  the gcc8 warns that  the  SwitchRange
> classes  in HS code   violate the C++ One Definition Rule .
> > So I renamed one of those 2  SwitchRange   classes .
> >
> > Bug/webrev :
> >
> > https://bugs.openjdk.java.net/browse/JDK-8236709
> >
> > http://cr.openjdk.java.net/~mbaesken/webrevs/8236709.0/
> 
> I think I would prefer the C1 "namespace" disambiguator to be a prefix
> rather than a suffix, to be consistent with usage in other parts of
> HotSpot.  (GC code does that somewhat consistently, for example.)  I
> don't recall seeing "namespace" suffixes used anywhere in HotSpot.
> 
> I've occasionally considered proposing that C2 code be wrapped in a
> namespace. It grabs a bunch of very generic global type names ("Type",
> "Block", &etc; really!?) that can be annoying. (Though the diet for
> precompiled.hpp helped with that.) But anything like that is probably
> out of scope for the immediate problem.
> 
> Giving C2 primacy of place (renaming the C1 class and letting C2 keep
> the "good" name) seems consistent with existing practice.


From thomas.schatzl at oracle.com  Wed Jan  8 08:34:13 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 8 Jan 2020 09:34:13 +0100
Subject: RFR [XS]: 8236709: struct SwitchRange in HS violates C++ One
 Definition Rule - was RE: struct SwitchRange and C++ One Definition Rule
 [-Wodr]
In-Reply-To: <AM6PR02MB42008DB43063A59D1DC845E7933E0@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB420004E4B2B911D7ADA71EAD933F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <062FA66D-4C78-4B11-AB2B-AE9B792596A6@oracle.com>
 <AM6PR02MB42008DB43063A59D1DC845E7933E0@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <ef923885-c1ed-9077-ea12-220339f54524@oracle.com>

Hi,

On 08.01.20 09:10, Baesken, Matthias wrote:
> Hello, thanks for the input .
> 
> I renamed SwitchRangeC1   to  C1SwitchRange  and  removed the unnecessary forward declaration .
> 
> New webrev :
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8236709.1/
> 

   looks good.

Thomas

From david.holmes at oracle.com  Wed Jan  8 09:51:11 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 8 Jan 2020 19:51:11 +1000
Subject: RFR [XS]: 8236709: struct SwitchRange in HS violates C++ One
 Definition Rule - was RE: struct SwitchRange and C++ One Definition Rule
 [-Wodr]
In-Reply-To: <AM6PR02MB42008DB43063A59D1DC845E7933E0@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB420004E4B2B911D7ADA71EAD933F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <062FA66D-4C78-4B11-AB2B-AE9B792596A6@oracle.com>
 <AM6PR02MB42008DB43063A59D1DC845E7933E0@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <56bd4688-7e48-c443-86f8-554539c3f325@oracle.com>

On 8/01/2020 6:10 pm, Baesken, Matthias wrote:
> Hello, thanks for the input .
> 
> I renamed SwitchRangeC1   to  C1SwitchRange  and  removed the unnecessary forward declaration .
> 
> New webrev :
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8236709.1/

Seems fine.

Thanks,
David

> 
> Best regards, Matthias
> 
> 
> 
> 
>>>
>>> Hello, please review  this small fix for an issue  that I was running into  when
>> experimenting  with  gcc8  and   the -flto compiler flag .
>>> When building with those flags,  the gcc8 warns that  the  SwitchRange
>> classes  in HS code   violate the C++ One Definition Rule .
>>> So I renamed one of those 2  SwitchRange   classes .
>>>
>>> Bug/webrev :
>>>
>>> https://bugs.openjdk.java.net/browse/JDK-8236709
>>>
>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8236709.0/
>>
>> I think I would prefer the C1 "namespace" disambiguator to be a prefix
>> rather than a suffix, to be consistent with usage in other parts of
>> HotSpot.  (GC code does that somewhat consistently, for example.)  I
>> don't recall seeing "namespace" suffixes used anywhere in HotSpot.
>>
>> I've occasionally considered proposing that C2 code be wrapped in a
>> namespace. It grabs a bunch of very generic global type names ("Type",
>> "Block", &etc; really!?) that can be annoying. (Though the diet for
>> precompiled.hpp helped with that.) But anything like that is probably
>> out of scope for the immediate problem.
>>
>> Giving C2 primacy of place (renaming the C1 class and letting C2 keep
>> the "good" name) seems consistent with existing practice.
> 

From matthias.baesken at sap.com  Wed Jan  8 12:08:29 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 8 Jan 2020 12:08:29 +0000
Subject: RFR [XS]: 8236709: struct SwitchRange in HS violates C++ One
 Definition Rule - was RE: struct SwitchRange and C++ One Definition Rule
 [-Wodr]
In-Reply-To: <56bd4688-7e48-c443-86f8-554539c3f325@oracle.com>
References: <AM6PR02MB420004E4B2B911D7ADA71EAD933F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <062FA66D-4C78-4B11-AB2B-AE9B792596A6@oracle.com>
 <AM6PR02MB42008DB43063A59D1DC845E7933E0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <56bd4688-7e48-c443-86f8-554539c3f325@oracle.com>
Message-ID: <AM6PR02MB42000E77C3801EE4E7B962F9933E0@AM6PR02MB4200.eurprd02.prod.outlook.com>

Thanks !

Kim are you fine as well  with the latest webrev   ?

Best regards, Matthias


> On 8/01/2020 6:10 pm, Baesken, Matthias wrote:
> > Hello, thanks for the input .
> >
> > I renamed SwitchRangeC1   to  C1SwitchRange  and  removed the
> unnecessary forward declaration .
> >
> > New webrev :
> >
> > http://cr.openjdk.java.net/~mbaesken/webrevs/8236709.1/
> 
> Seems fine.
> 
> Thanks,
> David
> 


From kim.barrett at oracle.com  Wed Jan  8 13:44:20 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 8 Jan 2020 08:44:20 -0500
Subject: RFR [XS]: 8236709: struct SwitchRange in HS violates C++ One
 Definition Rule  - was RE: struct SwitchRange  and  C++ One Definition Rule
 [-Wodr]
In-Reply-To: <AM6PR02MB42008DB43063A59D1DC845E7933E0@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB420004E4B2B911D7ADA71EAD933F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <062FA66D-4C78-4B11-AB2B-AE9B792596A6@oracle.com>
 <AM6PR02MB42008DB43063A59D1DC845E7933E0@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <85D461D2-400E-4814-A250-19A124AA37F8@oracle.com>

> On Jan 8, 2020, at 3:10 AM, Baesken, Matthias <matthias.baesken at sap.com> wrote:
> 
> Hello, thanks for the input .
> 
> I renamed SwitchRangeC1   to  C1SwitchRange  and  removed the unnecessary forward declaration .
> 
> New webrev :
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8236709.1/

Looks good.


From coleen.phillimore at oracle.com  Wed Jan  8 14:26:05 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Wed, 8 Jan 2020 09:26:05 -0500
Subject: RFR 8232759: Remove or simplify GC.class_stats
In-Reply-To: <6b3b3556-1ae2-41a1-bfef-30c60e7e4526@oracle.com>
References: <2faa21b1-c5d3-0caf-d7ed-6e7852814669@oracle.com>
 <6b3b3556-1ae2-41a1-bfef-30c60e7e4526@oracle.com>
Message-ID: <9ec8d0f8-c92d-5896-8482-98a2e80f3321@oracle.com>



On 1/8/20 12:55 AM, David Holmes wrote:
> Hi Coleen,
>
> On 8/01/2020 10:23 am, coleen.phillimore at oracle.com wrote:
>> Summary: Make the GC.class_stats option obsolete
>>
>> open webrev at 
>> http://cr.openjdk.java.net/~coleenp/2019/8232759.01/webrev
>> bug link https://bugs.openjdk.java.net/browse/JDK-8232759
>
> The fan out from that was larger than I was expecting :)

Yes.? If we wanted such a thing to affect all metadata classes like 
this, there is metaspace_pointers_do() now that should be used instead.
>
> Change appears fine. May I suggest updating the bug to get rid of the 
> "or simplify" part.

Ok.
Thanks!
Coleen
>
> Thanks,
> David
>
>> Tested with tier1 on all Oracle platforms and tier2,3 on 
>> linux-x64-debug.
>>
>> Thanks,
>> Coleen
>>


From stefan.karlsson at oracle.com  Wed Jan  8 15:34:05 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Wed, 8 Jan 2020 16:34:05 +0100
Subject: RFR: 8236778: Add Atomic::fetch_and_add
Message-ID: <3ca1aee9-0edf-e093-68d3-f77ce4dc02e4@oracle.com>

Hi all,

Please review this patch to introduce Atomic::fetch_and_add.

https://cr.openjdk.java.net/~stefank/8236778/webrev.01
https://bugs.openjdk.java.net/browse/JDK-8236778

There are a number of places where we have this pattern:
int result = Atomic::add(_index, amount) - amount;

I'd like to introduce Atomic::fetch_and_add so that we can write:
int result = Atomic::fetch_and_add(_index, amount);

The current implementation already has support for both "add and fetch" 
and "fetch and add" but it's not exposed to the upper layers.

Previously, the platform-specific code either implemented "add and 
fetch" or "fetch and add", and then exposed it as an "add and fetch" 
implementation by using CRTP and inheriting from either AddAndFetch or 
FetchAndAdd.

My first implementation of this continued in this track, but got 
push-back because the code was non-intuitive and/or used non-intuitive 
names. Therefore, I've removed FetchAndAdd/AddAndFetch and opted to 
duplicate the trivial functionality in the platform files instead. For 
example:

+  template<typename D, typename I>
+  D add_and_fetch(D volatile* dest, I add_value, atomic_memory_order 
order) const {
+    return fetch_and_add(dest, add_value, order) + add_value;
+  }

There has been some thoughts that maybe we should have:

void Atomic::add(...)
D    Atomic::add_and_fetch(...)
D    Atomic::fetch_and_add(...)

Not sure if it's worth changing to this, but if others think this is 
good, I can do that change.

Tested with tier123, but only compiled on platforms I have access to.

Thanks,
StefanK

From ioi.lam at oracle.com  Wed Jan  8 17:05:28 2020
From: ioi.lam at oracle.com (Ioi Lam)
Date: Wed, 8 Jan 2020 09:05:28 -0800
Subject: RFR 8232759: Remove or simplify GC.class_stats
In-Reply-To: <9ec8d0f8-c92d-5896-8482-98a2e80f3321@oracle.com>
References: <2faa21b1-c5d3-0caf-d7ed-6e7852814669@oracle.com>
 <6b3b3556-1ae2-41a1-bfef-30c60e7e4526@oracle.com>
 <9ec8d0f8-c92d-5896-8482-98a2e80f3321@oracle.com>
Message-ID: <9789ea24-3f4c-6c82-1d69-670856f6e28d@oracle.com>

Hi Coleen,

The changes look good to me. Thanks for cleaning up the mess that I 
introduced :-)

- Ioi

On 1/8/20 6:26 AM, coleen.phillimore at oracle.com wrote:
>
>
> On 1/8/20 12:55 AM, David Holmes wrote:
>> Hi Coleen,
>>
>> On 8/01/2020 10:23 am, coleen.phillimore at oracle.com wrote:
>>> Summary: Make the GC.class_stats option obsolete
>>>
>>> open webrev at 
>>> http://cr.openjdk.java.net/~coleenp/2019/8232759.01/webrev
>>> bug link https://bugs.openjdk.java.net/browse/JDK-8232759
>>
>> The fan out from that was larger than I was expecting :)
>
> Yes.? If we wanted such a thing to affect all metadata classes like 
> this, there is metaspace_pointers_do() now that should be used instead.
>>
>> Change appears fine. May I suggest updating the bug to get rid of the 
>> "or simplify" part.
>
> Ok.
> Thanks!
> Coleen
>>
>> Thanks,
>> David
>>
>>> Tested with tier1 on all Oracle platforms and tier2,3 on 
>>> linux-x64-debug.
>>>
>>> Thanks,
>>> Coleen
>>>
>


From coleen.phillimore at oracle.com  Wed Jan  8 21:14:21 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Wed, 8 Jan 2020 16:14:21 -0500
Subject: RFR 8232759: Remove or simplify GC.class_stats
In-Reply-To: <9789ea24-3f4c-6c82-1d69-670856f6e28d@oracle.com>
References: <2faa21b1-c5d3-0caf-d7ed-6e7852814669@oracle.com>
 <6b3b3556-1ae2-41a1-bfef-30c60e7e4526@oracle.com>
 <9ec8d0f8-c92d-5896-8482-98a2e80f3321@oracle.com>
 <9789ea24-3f4c-6c82-1d69-670856f6e28d@oracle.com>
Message-ID: <89e463a9-0845-e0c9-8774-3af045bf5a72@oracle.com>



On 1/8/20 12:05 PM, Ioi Lam wrote:
> Hi Coleen,
>
> The changes look good to me. Thanks for cleaning up the mess that I 
> introduced :-)

Thanks Ioi!? It was useful for a while though.
Coleen

>
> - Ioi
>
> On 1/8/20 6:26 AM, coleen.phillimore at oracle.com wrote:
>>
>>
>> On 1/8/20 12:55 AM, David Holmes wrote:
>>> Hi Coleen,
>>>
>>> On 8/01/2020 10:23 am, coleen.phillimore at oracle.com wrote:
>>>> Summary: Make the GC.class_stats option obsolete
>>>>
>>>> open webrev at 
>>>> http://cr.openjdk.java.net/~coleenp/2019/8232759.01/webrev
>>>> bug link https://bugs.openjdk.java.net/browse/JDK-8232759
>>>
>>> The fan out from that was larger than I was expecting :)
>>
>> Yes.? If we wanted such a thing to affect all metadata classes like 
>> this, there is metaspace_pointers_do() now that should be used instead.
>>>
>>> Change appears fine. May I suggest updating the bug to get rid of 
>>> the "or simplify" part.
>>
>> Ok.
>> Thanks!
>> Coleen
>>>
>>> Thanks,
>>> David
>>>
>>>> Tested with tier1 on all Oracle platforms and tier2,3 on 
>>>> linux-x64-debug.
>>>>
>>>> Thanks,
>>>> Coleen
>>>>
>>
>


From kim.barrett at oracle.com  Thu Jan  9 00:00:34 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 8 Jan 2020 19:00:34 -0500
Subject: RFR: 8236778: Add Atomic::fetch_and_add
In-Reply-To: <3ca1aee9-0edf-e093-68d3-f77ce4dc02e4@oracle.com>
References: <3ca1aee9-0edf-e093-68d3-f77ce4dc02e4@oracle.com>
Message-ID: <E63FAE9D-D556-4BE6-A583-A6F94FD36D86@oracle.com>

> On Jan 8, 2020, at 10:34 AM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
> 
> Hi all,
> 
> Please review this patch to introduce Atomic::fetch_and_add.
> 
> https://cr.openjdk.java.net/~stefank/8236778/webrev.01
> https://bugs.openjdk.java.net/browse/JDK-8236778
> 
> There are a number of places where we have this pattern:
> int result = Atomic::add(_index, amount) - amount;
> 
> I'd like to introduce Atomic::fetch_and_add so that we can write:
> int result = Atomic::fetch_and_add(_index, amount);

------------------------------------------------------------------------------
src/hotspot/share/runtime/atomic.hpp
Removed:
 240   // - platform_add is an object of type PlatformAdd<sizeof(D)>.
 241   //
 242   // Then
 243   //   platform_add(dest, add_value)
 244   // must be a valid expression, returning a result convertible to D.

and

 250   // Helper base classes for defining PlatformAdd.  To use, define
...
 275   // caller.

These comments should have been updated to describe the new protocol
for PlatformAdd, rather than simply removed.  Documentation of
extension points like this is important.  Something like

  // - platform_add is an object of type PlatformAdd<sizeof(D)>.
  //
  // Then both
  //   platform_add.add_and_fetch(dest, add_value)
  //   platform_add.fetch_and_add(dest, add_value)
  // must be valid expressions returning a result convertible to D.
  //
  // add_and_fetch atomically adds add_value to the value of dest,
  // returning the new value.
  //
  // fetch_and_add atomically adds add_value to the value of dest,
  // returning the old value.
  //
  // When D is a pointer type P*, both add_and_fetch and fetch_and_add
  // treat it as if it were an uintptr_t; they do not perform any
  // scaling of add_value, as that has already been done by the caller.

------------------------------------------------------------------------------
src/hotspot/share/runtime/atomic.hpp 
 679   static I scale_addend(I add_value) {
 680     return add_value * sizeof(P);
 681   }
 682 
 683   static P* add_and_fetch(P* volatile* dest, I add_value, atomic_memory_order order) {
 684     CI addend = add_value;
 685     return PlatformAdd<sizeof(P*)>().add_and_fetch(dest, scale_addend(addend), order);
 686   }

This is converting add_value from I to CI then back to I, the latter
possibly being a narrowing conversion.  Better would be

  static CI scale_addend(CI add_value) {
    return add_value * sizeof(P);
  }

and then either

  static P* add_and_fetch(P* volatile* dest, I add_value, atomic_memory_order order) {
    CI addend = scale_addend(add_value);
    return PlatformAdd<sizeof(P*)>().add_and_fetch(dest, addend, order);
  }

or don't bother with the addend variable and just pass the scaled
result directly to add_and_fetch / fetch_and_add.

------------------------------------------------------------------------------
src/hotspot/os_cpu/windows_x86/atomic_windows_x86.hpp
  54 #pragma warning(disable: 4035) // Disables warnings reporting missing return statement

Pre-existing: This warning suppression (and the corresponding
restoration) appears to only be needed for the !defined(AMD64) case
(where __asm statements are being used), and should be moved accordingly.

------------------------------------------------------------------------------

> The current implementation already has support for both "add and fetch" and "fetch and add" but it's not exposed to the upper layers.
> 
> Previously, the platform-specific code either implemented "add and fetch" or "fetch and add", and then exposed it as an "add and fetch" implementation by using CRTP and inheriting from either AddAndFetch or FetchAndAdd.
> 
> My first implementation of this continued in this track, but got push-back because the code was non-intuitive and/or used non-intuitive names. Therefore, I've removed FetchAndAdd/AddAndFetch and opted to duplicate the trivial functionality in the platform files instead. For example:
> 
> +  template<typename D, typename I>
> +  D add_and_fetch(D volatile* dest, I add_value, atomic_memory_order order) const {
> +    return fetch_and_add(dest, add_value, order) + add_value;
> +  }

For the record, I would have been fine with (and actually preferred; I
dislike roughly a dozen copies of one or the other of the translation
functions) CRTP-based AddAndFetch and FetchAndAdd that provided one
operation in terms of the other.

The move of the scaling of add_value in the pointer case to AddImpl is
an improvement (even on the pre-existing code) that might have made
such a CRTP approach clearer, possibly with some name changes.  Then
the helper CRTP base class just provides one of the functions in terms
of Derived's other function, and nothing else.

Perhaps the hardest part of that approach is naming the helper base
classes.  FetchAndAddUsingAddAndFetch is explicit but quite a
mouthful.  Perhaps FetchAndAddHelper, which provides fetch_and_add in
terms of Derived's add_and_fetch?

Apologies for not having followed the internal pre-review discussion
of this and so not commenting there.

> There has been some thoughts that maybe we should have:
> 
> void Atomic::add(...)
> D    Atomic::add_and_fetch(...)
> D    Atomic::fetch_and_add(...)
> 
> Not sure if it's worth changing to this, but if others think this is good, I can do that change.

I would be okay with renaming "add" to "add_and_fetch", though I don't
at the moment have a strong preference either way.  I'm not sure also
providing "add" that returns void is really all that useful.  We have
"inc" and "dec" that return void; my recollection is that they were at
one time thought to provide an opportunity for a more efficient
implementation on some platforms, but discussion during the
templatization project showed that compilers could eliminate an unused
return value anyway.

For symmetry it seems like we should have fetch_and_sub, but I don't
see any current uses of sub that would need that.

If add_and_fetch is added (either instead of or in addition to add),
then we should definitely treat the subtraction case similarly.

> Tested with tier123, but only compiled on platforms I have access to.

mach5 has the ability to cross-compile to several OpenJDK platforms
not otherwise supported by Oracle, though can't run tests.  That can
be a useful smoke test to at least avoid things like typos that don't
build.  For example, "-b linux-aarch64-debug?.


From robbin.ehn at oracle.com  Thu Jan  9 09:32:12 2020
From: robbin.ehn at oracle.com (Robbin Ehn)
Date: Thu, 9 Jan 2020 10:32:12 +0100
Subject: Status for JDK-8199138, riscv64 support for Zero
In-Reply-To: <96a56c15-5952-ab7c-8427-4860918f3c4a@physik.fu-berlin.de>
References: <96a56c15-5952-ab7c-8427-4860918f3c4a@physik.fu-berlin.de>
Message-ID: <44d74403-9424-6d91-b090-f8c626436562@oracle.com>

Hi Adrian,

As Magnus wrote:
"For this patch to be accepted in mainline, you will need the remove the 
changes to the build-aux/autoconf-config* files (we cannot change them 
due to copyright), and instead patch the OpenJDK wrapper scripts 
build-aux/config*."

Which you seem to have done in the current debian patch.

So AFAICT just re-open and send-out that patch. (note I'm not a build 
file guy...)

Thanks, Robbin

On 2020-01-04 12:18, John Paul Adrian Glaubitz wrote:
> Hi!
> 
> Debian and several other distribution already have already been bootstrapped
> for riscv64 with a large number of packages building fine.
> 
> Support for OpenJDK Zero has been added through a slightly modified version
> of JDK-8199138 [1]. Looking at the bug report for JDK-8199138, it seems that
> the patch was retracted back in 2018.
> 
> However, since the Debian version of the patch works fine, I was wondering
> whether we could get it merged in one form or another?
> 
> Thanks,
> Adrian
> 
>> [1] https://git.launchpad.net/~openjdk/ubuntu/+source/openjdk/+git/openjdk/tree/debian/patches/riscv64.diff?h=openjdk-13
>> [2] https://bugs.openjdk.java.net/browse/JDK-8199138
> 

From aph at redhat.com  Thu Jan  9 15:04:02 2020
From: aph at redhat.com (Andrew Haley)
Date: Thu, 9 Jan 2020 15:04:02 +0000
Subject: 8236856: AArch64: Spurious GCC warnings
Message-ID: <7a729234-0f79-e39d-00c8-27fd48b51bd4@redhat.com>

With some versions of GCC we get this at compile time, which causes build failures
when warnings-as-errors is enabled. It's a false positive, and should be fixed in
GCC, but we need to shut it up.

Compiling macroAssembler_aarch64.cpp (for libjvm.so)
In file included from /home/aph/jdk-jdk/src/hotspot/share/runtime/atomic.hpp:632:0,
                 from /home/aph/jdk-jdk/src/hotspot/share/oops/oop.hpp:33,
                 from /home/aph/jdk-jdk/src/hotspot/share/runtime/handles.hpp:29,
                 from /home/aph/jdk-jdk/src/hotspot/share/code/oopRecorder.hpp:28,
                 from /home/aph/jdk-jdk/src/hotspot/share/asm/codeBuffer.hpp:28,
                 from /home/aph/jdk-jdk/src/hotspot/share/asm/assembler.hpp:28,
                 from /home/aph/jdk-jdk/src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp:30:
/home/aph/jdk-jdk/src/hotspot/os_cpu/linux_aarch64/atomic_linux_aarch64.hpp: In instantiation of 'T Atomic::PlatformCmpxchg<byte_size>::operator()(volatile T*, T, T, atomic_memory_order) const [with T = signed char; long unsigned int byte_size = 1ul]':
/home/aph/jdk-jdk/src/hotspot/share/runtime/atomic.hpp:783:46: required from 'T Atomic::CmpxchgImpl<T, T, T, typename EnableIf<(IsIntegral<T>::value || IsRegisteredEnum<T>::value)>::type>::operator()(volatile T*, T, T, atomic_memory_order) const [with T = signed char; typename EnableIf<(IsIntegral<T>::value || IsRegisteredEnum<T>::value)>::type = void]'
/home/aph/jdk-jdk/src/hotspot/share/runtime/atomic.hpp:756:75: required from 'static D Atomic::cmpxchg(volatile D*, U, T, atomic_memory_order) [with D = signed char; U = signed char; T = signed char]'
/home/aph/jdk-jdk/src/hotspot/share/gc/shenandoah/shenandoahSharedVariables.hpp:77:113: required from here
/home/aph/jdk-jdk/src/hotspot/os_cpu/linux_aarch64/atomic_linux_aarch64.hpp:60:10:

Fixed thusly. OK?

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671
-------------- next part --------------
# HG changeset patch
# User aph
# Date 1578582061 18000
#      Thu Jan 09 10:01:01 2020 -0500
# Node ID a6c0679606c37ad3c7c21537bd338f4d272aa1e3
# Parent  6d23020e3da0ed7b276e10f60e0c8d178d7c049f
8236856: AArch64: Spurious GCC warnings
Reviewed-by: adinn

diff -r 6d23020e3da0 -r a6c0679606c3 src/hotspot/os_cpu/linux_aarch64/atomic_linux_aarch64.hpp
--- a/src/hotspot/os_cpu/linux_aarch64/atomic_linux_aarch64.hpp	Thu Jan 09 09:30:49 2020 -0500
+++ b/src/hotspot/os_cpu/linux_aarch64/atomic_linux_aarch64.hpp	Thu Jan 09 10:01:01 2020 -0500
@@ -55,9 +55,10 @@
   return res;
 }
 
+// __attribute__((unused)) on dest is to get rid of spurious GCC warnings.
 template<size_t byte_size>
 template<typename T>
-inline T Atomic::PlatformCmpxchg<byte_size>::operator()(T volatile* dest,
+inline T Atomic::PlatformCmpxchg<byte_size>::operator()(T volatile* dest __attribute__((unused)),
                                                         T compare_value,
                                                         T exchange_value,
                                                         atomic_memory_order order) const {

From glaubitz at physik.fu-berlin.de  Thu Jan  9 15:25:14 2020
From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz)
Date: Thu, 9 Jan 2020 16:25:14 +0100
Subject: Status for JDK-8199138, riscv64 support for Zero
In-Reply-To: <44d74403-9424-6d91-b090-f8c626436562@oracle.com>
References: <96a56c15-5952-ab7c-8427-4860918f3c4a@physik.fu-berlin.de>
 <44d74403-9424-6d91-b090-f8c626436562@oracle.com>
Message-ID: <f883f347-532a-35c3-fe40-82ba650f33f5@physik.fu-berlin.de>

Hi!

On 1/9/20 10:32 AM, Robbin Ehn wrote:
> As Magnus wrote:
> "For this patch to be accepted in mainline, you will need the remove the changes to the build-aux/autoconf-config* files (we cannot change them due to copyright), and instead patch the OpenJDK wrapper scripts build-aux/config*."
> 
> Which you seem to have done in the current debian patch.
> 
> So AFAICT just re-open and send-out that patch. (note I'm not a build file guy...)
Okay, I'll put myself as owner for the bug report first, then post a new RFR.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz at debian.org
`. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

From aph at redhat.com  Thu Jan  9 16:09:52 2020
From: aph at redhat.com (Andrew Haley)
Date: Thu, 9 Jan 2020 16:09:52 +0000
Subject: [resend] 8236856: AArch64: Spurious GCC warnings
Message-ID: <2c9e41f3-44e3-9a56-1c6b-9ff86ebfb21a@redhat.com>

With some versions of GCC we get this at compile time, which causes build failures
when warnings-as-errors is enabled. It's a false positive, and should be fixed in
GCC, but we need to shut it up.

Compiling macroAssembler_aarch64.cpp (for libjvm.so)
In file included from /home/aph/jdk-jdk/src/hotspot/share/runtime/atomic.hpp:632:0,
                 from /home/aph/jdk-jdk/src/hotspot/share/oops/oop.hpp:33,
                 from /home/aph/jdk-jdk/src/hotspot/share/runtime/handles.hpp:29,
                 from /home/aph/jdk-jdk/src/hotspot/share/code/oopRecorder.hpp:28,
                 from /home/aph/jdk-jdk/src/hotspot/share/asm/codeBuffer.hpp:28,
                 from /home/aph/jdk-jdk/src/hotspot/share/asm/assembler.hpp:28,
                 from /home/aph/jdk-jdk/src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp:30:
/home/aph/jdk-jdk/src/hotspot/os_cpu/linux_aarch64/atomic_linux_aarch64.hpp: In instantiation of 'T Atomic::PlatformCmpxchg<byte_size>::operator()(volatile T*, T, T, atomic_memory_order) const [with T = signed char; long unsigned int byte_size = 1ul]':
/home/aph/jdk-jdk/src/hotspot/share/runtime/atomic.hpp:783:46: required from 'T Atomic::CmpxchgImpl<T, T, T, typename EnableIf<(IsIntegral<T>::value || IsRegisteredEnum<T>::value)>::type>::operator()(volatile T*, T, T, atomic_memory_order) const [with T = signed char; typename EnableIf<(IsIntegral<T>::value || IsRegisteredEnum<T>::value)>::type = void]'
/home/aph/jdk-jdk/src/hotspot/share/runtime/atomic.hpp:756:75: required from 'static D Atomic::cmpxchg(volatile D*, U, T, atomic_memory_order) [with D = signed char; U = signed char; T = signed char]'
/home/aph/jdk-jdk/src/hotspot/share/gc/shenandoah/shenandoahSharedVariables.hpp:77:113: required from here
/home/aph/jdk-jdk/src/hotspot/os_cpu/linux_aarch64/atomic_linux_aarch64.hpp:60:10: warning: parameter 'dest' set but not used [-Wunused-but-set-parameter]

Fixed thusly. OK?

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

-------------- next part --------------
# HG changeset patch
# User aph
# Date 1578582061 18000
#      Thu Jan 09 10:01:01 2020 -0500
# Node ID a6c0679606c37ad3c7c21537bd338f4d272aa1e3
# Parent  6d23020e3da0ed7b276e10f60e0c8d178d7c049f
8236856: AArch64: Spurious GCC warnings
Reviewed-by: adinn

diff -r 6d23020e3da0 -r a6c0679606c3 src/hotspot/os_cpu/linux_aarch64/atomic_linux_aarch64.hpp
--- a/src/hotspot/os_cpu/linux_aarch64/atomic_linux_aarch64.hpp	Thu Jan 09 09:30:49 2020 -0500
+++ b/src/hotspot/os_cpu/linux_aarch64/atomic_linux_aarch64.hpp	Thu Jan 09 10:01:01 2020 -0500
@@ -55,9 +55,10 @@
   return res;
 }
 
+// __attribute__((unused)) on dest is to get rid of spurious GCC warnings.
 template<size_t byte_size>
 template<typename T>
-inline T Atomic::PlatformCmpxchg<byte_size>::operator()(T volatile* dest,
+inline T Atomic::PlatformCmpxchg<byte_size>::operator()(T volatile* dest __attribute__((unused)),
                                                         T compare_value,
                                                         T exchange_value,
                                                         atomic_memory_order order) const {

From adinn at redhat.com  Thu Jan  9 16:12:46 2020
From: adinn at redhat.com (Andrew Dinn)
Date: Thu, 9 Jan 2020 16:12:46 +0000
Subject: [resend] 8236856: AArch64: Spurious GCC warnings
In-Reply-To: <2c9e41f3-44e3-9a56-1c6b-9ff86ebfb21a@redhat.com>
References: <2c9e41f3-44e3-9a56-1c6b-9ff86ebfb21a@redhat.com>
Message-ID: <8577e9bd-acf6-3536-056f-a3043e040bf2@redhat.com>

On 09/01/2020 16:09, Andrew Haley wrote:
> With some versions of GCC we get this at compile time, which causes build failures
> when warnings-as-errors is enabled. It's a false positive, and should be fixed in
> GCC, but we need to shut it up.
> 
> Compiling macroAssembler_aarch64.cpp (for libjvm.so)
> In file included from /home/aph/jdk-jdk/src/hotspot/share/runtime/atomic.hpp:632:0,
>                  from /home/aph/jdk-jdk/src/hotspot/share/oops/oop.hpp:33,
>                  from /home/aph/jdk-jdk/src/hotspot/share/runtime/handles.hpp:29,
>                  from /home/aph/jdk-jdk/src/hotspot/share/code/oopRecorder.hpp:28,
>                  from /home/aph/jdk-jdk/src/hotspot/share/asm/codeBuffer.hpp:28,
>                  from /home/aph/jdk-jdk/src/hotspot/share/asm/assembler.hpp:28,
>                  from /home/aph/jdk-jdk/src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp:30:
> /home/aph/jdk-jdk/src/hotspot/os_cpu/linux_aarch64/atomic_linux_aarch64.hpp: In instantiation of 'T Atomic::PlatformCmpxchg<byte_size>::operator()(volatile T*, T, T, atomic_memory_order) const [with T = signed char; long unsigned int byte_size = 1ul]':
> /home/aph/jdk-jdk/src/hotspot/share/runtime/atomic.hpp:783:46: required from 'T Atomic::CmpxchgImpl<T, T, T, typename EnableIf<(IsIntegral<T>::value || IsRegisteredEnum<T>::value)>::type>::operator()(volatile T*, T, T, atomic_memory_order) const [with T = signed char; typename EnableIf<(IsIntegral<T>::value || IsRegisteredEnum<T>::value)>::type = void]'
> /home/aph/jdk-jdk/src/hotspot/share/runtime/atomic.hpp:756:75: required from 'static D Atomic::cmpxchg(volatile D*, U, T, atomic_memory_order) [with D = signed char; U = signed char; T = signed char]'
> /home/aph/jdk-jdk/src/hotspot/share/gc/shenandoah/shenandoahSharedVariables.hpp:77:113: required from here
> /home/aph/jdk-jdk/src/hotspot/os_cpu/linux_aarch64/atomic_linux_aarch64.hpp:60:10: warning: parameter 'dest' set but not used [-Wunused-but-set-parameter]
> 
> Fixed thusly. OK?
Yes, that looks fine /and/ trivial.

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill


From stefan.karlsson at oracle.com  Thu Jan  9 22:17:40 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Thu, 9 Jan 2020 23:17:40 +0100
Subject: RFR: 8236778: Add Atomic::fetch_and_add
In-Reply-To: <E63FAE9D-D556-4BE6-A583-A6F94FD36D86@oracle.com>
References: <3ca1aee9-0edf-e093-68d3-f77ce4dc02e4@oracle.com>
 <E63FAE9D-D556-4BE6-A583-A6F94FD36D86@oracle.com>
Message-ID: <7b2a936a-e4c0-55fe-3c8a-4627fb59475a@oracle.com>

Updated webrev:
 ?https://cr.openjdk.java.net/~stefank/8236778/webrev.02.delta
 ?https://cr.openjdk.java.net/~stefank/8236778/webrev.02

Comments below:

On 2020-01-09 01:00, Kim Barrett wrote:
>> On Jan 8, 2020, at 10:34 AM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
>>
>> Hi all,
>>
>> Please review this patch to introduce Atomic::fetch_and_add.
>>
>> https://cr.openjdk.java.net/~stefank/8236778/webrev.01
>> https://bugs.openjdk.java.net/browse/JDK-8236778
>>
>> There are a number of places where we have this pattern:
>> int result = Atomic::add(_index, amount) - amount;
>>
>> I'd like to introduce Atomic::fetch_and_add so that we can write:
>> int result = Atomic::fetch_and_add(_index, amount);
> ------------------------------------------------------------------------------
> src/hotspot/share/runtime/atomic.hpp
> Removed:
>   240   // - platform_add is an object of type PlatformAdd<sizeof(D)>.
>   241   //
>   242   // Then
>   243   //   platform_add(dest, add_value)
>   244   // must be a valid expression, returning a result convertible to D.
>
> and
>
>   250   // Helper base classes for defining PlatformAdd.  To use, define
> ...
>   275   // caller.
>
> These comments should have been updated to describe the new protocol
> for PlatformAdd, rather than simply removed.  Documentation of
> extension points like this is important.

The comments seemed to already have started deteriorating. It's seems to 
be missing an *if* that matches the *then, and it doesn't mention 
*order*. So, instead of trying to figure out what the comment tried to 
say, I opted from getting rid of those problems by getting rid of that 
part of the comment. I thought the code was pretty self-explanatory, and 
didn't need that part. I'm not opposed to fixing the comments if you 
have suggestions.

>    Something like
>
>    // - platform_add is an object of type PlatformAdd<sizeof(D)>.
>    //
>    // Then both
>    //   platform_add.add_and_fetch(dest, add_value)
>    //   platform_add.fetch_and_add(dest, add_value)
>    // must be valid expressions returning a result convertible to D.
>    //
>    // add_and_fetch atomically adds add_value to the value of dest,
>    // returning the new value.
>    //
>    // fetch_and_add atomically adds add_value to the value of dest,
>    // returning the old value.
>    //
>    // When D is a pointer type P*, both add_and_fetch and fetch_and_add
>    // treat it as if it were an uintptr_t; they do not perform any
>    // scaling of add_value, as that has already been done by the caller.

I'll update the code verbatim with what you've suggested.

>
> ------------------------------------------------------------------------------
> src/hotspot/share/runtime/atomic.hpp
>   679   static I scale_addend(I add_value) {
>   680     return add_value * sizeof(P);
>   681   }
>   682
>   683   static P* add_and_fetch(P* volatile* dest, I add_value, atomic_memory_order order) {
>   684     CI addend = add_value;
>   685     return PlatformAdd<sizeof(P*)>().add_and_fetch(dest, scale_addend(addend), order);
>   686   }
>
> This is converting add_value from I to CI then back to I, the latter
> possibly being a narrowing conversion.  Better would be
>
>    static CI scale_addend(CI add_value) {
>      return add_value * sizeof(P);
>    }
>
> and then either
>
>    static P* add_and_fetch(P* volatile* dest, I add_value, atomic_memory_order order) {
>      CI addend = scale_addend(add_value);
>      return PlatformAdd<sizeof(P*)>().add_and_fetch(dest, addend, order);
>    }
>
> or don't bother with the addend variable and just pass the scaled
> result directly to add_and_fetch / fetch_and_add.

Thanks. This was an unintentional change from the original code.
>
> ------------------------------------------------------------------------------
> src/hotspot/os_cpu/windows_x86/atomic_windows_x86.hpp
>    54 #pragma warning(disable: 4035) // Disables warnings reporting missing return statement
>
> Pre-existing: This warning suppression (and the corresponding
> restoration) appears to only be needed for the !defined(AMD64) case
> (where __asm statements are being used), and should be moved accordingly.

Will you create a bug report for that?

>
> ------------------------------------------------------------------------------
>
>> The current implementation already has support for both "add and fetch" and "fetch and add" but it's not exposed to the upper layers.
>>
>> Previously, the platform-specific code either implemented "add and fetch" or "fetch and add", and then exposed it as an "add and fetch" implementation by using CRTP and inheriting from either AddAndFetch or FetchAndAdd.
>>
>> My first implementation of this continued in this track, but got push-back because the code was non-intuitive and/or used non-intuitive names. Therefore, I've removed FetchAndAdd/AddAndFetch and opted to duplicate the trivial functionality in the platform files instead. For example:
>>
>> +  template<typename D, typename I>
>> +  D add_and_fetch(D volatile* dest, I add_value, atomic_memory_order order) const {
>> +    return fetch_and_add(dest, add_value, order) + add_value;
>> +  }
> For the record, I would have been fine with (and actually preferred; I
> dislike roughly a dozen copies of one or the other of the translation
> functions) CRTP-based AddAndFetch and FetchAndAdd that provided one
> operation in terms of the other.
>
> The move of the scaling of add_value in the pointer case to AddImpl is
> an improvement (even on the pre-existing code) that might have made
> such a CRTP approach clearer, possibly with some name changes.  Then
> the helper CRTP base class just provides one of the functions in terms
> of Derived's other function, and nothing else.
>
> Perhaps the hardest part of that approach is naming the helper base
> classes.  FetchAndAddUsingAddAndFetch is explicit but quite a
> mouthful.  Perhaps FetchAndAddHelper, which provides fetch_and_add in
> terms of Derived's add_and_fetch?
>
> Apologies for not having followed the internal pre-review discussion
> of this and so not commenting there.
>
>> There has been some thoughts that maybe we should have:
>>
>> void Atomic::add(...)
>> D    Atomic::add_and_fetch(...)
>> D    Atomic::fetch_and_add(...)
>>
>> Not sure if it's worth changing to this, but if others think this is good, I can do that change.
> I would be okay with renaming "add" to "add_and_fetch", though I don't
> at the moment have a strong preference either way.  I'm not sure also
> providing "add" that returns void is really all that useful.  We have
> "inc" and "dec" that return void; my recollection is that they were at
> one time thought to provide an opportunity for a more efficient
> implementation on some platforms, but discussion during the
> templatization project showed that compilers could eliminate an unused
> return value anyway.
>
> For symmetry it seems like we should have fetch_and_sub, but I don't
> see any current uses of sub that would need that.
>
> If add_and_fetch is added (either instead of or in addition to add),
> then we should definitely treat the subtraction case similarly.

I'll interpret your comments above that there might be opportunities to 
do more changes, but none of those needs to be done for this patch.

>
>> Tested with tier123, but only compiled on platforms I have access to.
> mach5 has the ability to cross-compile to several OpenJDK platforms
> not otherwise supported by Oracle, though can't run tests.  That can
> be a useful smoke test to at least avoid things like typos that don't
> build.  For example, "-b linux-aarch64-debug?.

Right. I wasn't explicit about it, but I've compiled locally (linux) for 
aarch64, arm32, ppc64, s390, zero, minimal.

Thanks,
StefanK
>


From kim.barrett at oracle.com  Fri Jan 10 01:50:08 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 9 Jan 2020 20:50:08 -0500
Subject: RFR: 8236778: Add Atomic::fetch_and_add
In-Reply-To: <7b2a936a-e4c0-55fe-3c8a-4627fb59475a@oracle.com>
References: <3ca1aee9-0edf-e093-68d3-f77ce4dc02e4@oracle.com>
 <E63FAE9D-D556-4BE6-A583-A6F94FD36D86@oracle.com>
 <7b2a936a-e4c0-55fe-3c8a-4627fb59475a@oracle.com>
Message-ID: <4F9B2E5D-DF2E-492C-A384-430C09B0DB5B@oracle.com>

> On Jan 9, 2020, at 5:17 PM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
> 
> Updated webrev:
>  https://cr.openjdk.java.net/~stefank/8236778/webrev.02.delta
>  https://cr.openjdk.java.net/~stefank/8236778/webrev.02

Looks good except for a couple more comment tweaks described below.

>> src/hotspot/share/runtime/atomic.hpp
>> Removed:
>>  240   // - platform_add is an object of type PlatformAdd<sizeof(D)>.
>>  241   //
>>  242   // Then
>>  243   //   platform_add(dest, add_value)
>>  244   // must be a valid expression, returning a result convertible to D.
>> 
>> and
>> 
>>  250   // Helper base classes for defining PlatformAdd.  To use, define
>> ...
>>  275   // caller.
>> 
>> These comments should have been updated to describe the new protocol
>> for PlatformAdd, rather than simply removed.  Documentation of
>> extension points like this is important.
> 
> The comments seemed to already have started deteriorating. It's seems to be missing an *if* that matches the *then,

The structure is "Given <list of requirements> then <implications>";
there was never an "if".  All of the similar comments in this file
were written that way.

But there's a further update that I just noticed is needed by your changes:

 233   // class is a function object that must be default constructable,

Delete "is a function object that" in the above.

>  and it doesn't mention *order*.

It's true that *order* isn't mentioned; that seems to have been missed
when the order argument was added here.  (It *is* described in the
xchg and cmpxchg commentary, and we could do similarly here.)

Add

  // - order is of type atomic_memory_order.

to the requirements list, and change the expressions to include an
order argument, e.g.

  // Then both
  //   platform_add.add_and_fetch(dest, add_value, order)
  //   platform_add.fetch_and_add(dest, add_value, order)

> So, instead of trying to figure out what the comment tried to say, I opted from getting rid of those problems by getting rid of that part of the comment. I thought the code was pretty self-explanatory, and didn't need that part. I'm not opposed to fixing the comments if you have suggestions.

Unfortunately, documenting templates can be pretty hard. But because
of the looseness of templates, differentiating between intentional
semantics and accidents of today's implementation based only on the
code can also be pretty hard. Yet failure to do so leads to
brittleness and uncertainties, especially when there are third-party
specializations. So I think it's worth making an attempt at
documentation, even if it's hard and the results not always perfect.

> I'll update the code verbatim with what you've suggested.

Plus the couple further modifications mentioned above?

>> src/hotspot/os_cpu/windows_x86/atomic_windows_x86.hpp
>>   54 #pragma warning(disable: 4035) // Disables warnings reporting missing return statement
>> 
>> Pre-existing: This warning suppression (and the corresponding
>> restoration) appears to only be needed for the !defined(AMD64) case
>> (where __asm statements are being used), and should be moved accordingly.
> 
> Will you create a bug report for that?

https://bugs.openjdk.java.net/browse/JDK-8236900
atomic_windows_x86.hpp disables warning 4035 more widely than needed

> I'll interpret your comments above that there might be opportunities to do more changes, but none of those needs to be done for this patch.

OK.

> Right. I wasn't explicit about it, but I've compiled locally (linux) for aarch64, arm32, ppc64, s390, zero, minimal.

Ah, that wasn?t clear to me from what you said.  Good.


From david.holmes at oracle.com  Fri Jan 10 02:05:01 2020
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 10 Jan 2020 12:05:01 +1000
Subject: RFR: 8236778: Add Atomic::fetch_and_add
In-Reply-To: <7b2a936a-e4c0-55fe-3c8a-4627fb59475a@oracle.com>
References: <3ca1aee9-0edf-e093-68d3-f77ce4dc02e4@oracle.com>
 <E63FAE9D-D556-4BE6-A583-A6F94FD36D86@oracle.com>
 <7b2a936a-e4c0-55fe-3c8a-4627fb59475a@oracle.com>
Message-ID: <48a620f5-a562-b3b6-3c80-7c6c33fa8ba9@oracle.com>

Hi Stefan,

On 10/01/2020 8:17 am, Stefan Karlsson wrote:
> Updated webrev:
>  ?https://cr.openjdk.java.net/~stefank/8236778/webrev.02.delta
>  ?https://cr.openjdk.java.net/~stefank/8236778/webrev.02

That all seems okay to me (not that I fully understand the details of 
the template code). I'll leave you and Kim to work out the details of 
the commentary.

Thanks,
David

> Comments below:
> 
> On 2020-01-09 01:00, Kim Barrett wrote:
>>> On Jan 8, 2020, at 10:34 AM, Stefan Karlsson 
>>> <stefan.karlsson at oracle.com> wrote:
>>>
>>> Hi all,
>>>
>>> Please review this patch to introduce Atomic::fetch_and_add.
>>>
>>> https://cr.openjdk.java.net/~stefank/8236778/webrev.01
>>> https://bugs.openjdk.java.net/browse/JDK-8236778
>>>
>>> There are a number of places where we have this pattern:
>>> int result = Atomic::add(_index, amount) - amount;
>>>
>>> I'd like to introduce Atomic::fetch_and_add so that we can write:
>>> int result = Atomic::fetch_and_add(_index, amount);
>> ------------------------------------------------------------------------------ 
>>
>> src/hotspot/share/runtime/atomic.hpp
>> Removed:
>> ? 240?? // - platform_add is an object of type PlatformAdd<sizeof(D)>.
>> ? 241?? //
>> ? 242?? // Then
>> ? 243?? //?? platform_add(dest, add_value)
>> ? 244?? // must be a valid expression, returning a result convertible 
>> to D.
>>
>> and
>>
>> ? 250?? // Helper base classes for defining PlatformAdd.? To use, define
>> ...
>> ? 275?? // caller.
>>
>> These comments should have been updated to describe the new protocol
>> for PlatformAdd, rather than simply removed.? Documentation of
>> extension points like this is important.
> 
> The comments seemed to already have started deteriorating. It's seems to 
> be missing an *if* that matches the *then, and it doesn't mention 
> *order*. So, instead of trying to figure out what the comment tried to 
> say, I opted from getting rid of those problems by getting rid of that 
> part of the comment. I thought the code was pretty self-explanatory, and 
> didn't need that part. I'm not opposed to fixing the comments if you 
> have suggestions.
> 
>> ?? Something like
>>
>> ?? // - platform_add is an object of type PlatformAdd<sizeof(D)>.
>> ?? //
>> ?? // Then both
>> ?? //?? platform_add.add_and_fetch(dest, add_value)
>> ?? //?? platform_add.fetch_and_add(dest, add_value)
>> ?? // must be valid expressions returning a result convertible to D.
>> ?? //
>> ?? // add_and_fetch atomically adds add_value to the value of dest,
>> ?? // returning the new value.
>> ?? //
>> ?? // fetch_and_add atomically adds add_value to the value of dest,
>> ?? // returning the old value.
>> ?? //
>> ?? // When D is a pointer type P*, both add_and_fetch and fetch_and_add
>> ?? // treat it as if it were an uintptr_t; they do not perform any
>> ?? // scaling of add_value, as that has already been done by the caller.
> 
> I'll update the code verbatim with what you've suggested.
> 
>>
>> ------------------------------------------------------------------------------ 
>>
>> src/hotspot/share/runtime/atomic.hpp
>> ? 679?? static I scale_addend(I add_value) {
>> ? 680???? return add_value * sizeof(P);
>> ? 681?? }
>> ? 682
>> ? 683?? static P* add_and_fetch(P* volatile* dest, I add_value, 
>> atomic_memory_order order) {
>> ? 684???? CI addend = add_value;
>> ? 685???? return PlatformAdd<sizeof(P*)>().add_and_fetch(dest, 
>> scale_addend(addend), order);
>> ? 686?? }
>>
>> This is converting add_value from I to CI then back to I, the latter
>> possibly being a narrowing conversion.? Better would be
>>
>> ?? static CI scale_addend(CI add_value) {
>> ???? return add_value * sizeof(P);
>> ?? }
>>
>> and then either
>>
>> ?? static P* add_and_fetch(P* volatile* dest, I add_value, 
>> atomic_memory_order order) {
>> ???? CI addend = scale_addend(add_value);
>> ???? return PlatformAdd<sizeof(P*)>().add_and_fetch(dest, addend, order);
>> ?? }
>>
>> or don't bother with the addend variable and just pass the scaled
>> result directly to add_and_fetch / fetch_and_add.
> 
> Thanks. This was an unintentional change from the original code.
>>
>> ------------------------------------------------------------------------------ 
>>
>> src/hotspot/os_cpu/windows_x86/atomic_windows_x86.hpp
>> ?? 54 #pragma warning(disable: 4035) // Disables warnings reporting 
>> missing return statement
>>
>> Pre-existing: This warning suppression (and the corresponding
>> restoration) appears to only be needed for the !defined(AMD64) case
>> (where __asm statements are being used), and should be moved accordingly.
> 
> Will you create a bug report for that?
> 
>>
>> ------------------------------------------------------------------------------ 
>>
>>
>>> The current implementation already has support for both "add and 
>>> fetch" and "fetch and add" but it's not exposed to the upper layers.
>>>
>>> Previously, the platform-specific code either implemented "add and 
>>> fetch" or "fetch and add", and then exposed it as an "add and fetch" 
>>> implementation by using CRTP and inheriting from either AddAndFetch 
>>> or FetchAndAdd.
>>>
>>> My first implementation of this continued in this track, but got 
>>> push-back because the code was non-intuitive and/or used 
>>> non-intuitive names. Therefore, I've removed FetchAndAdd/AddAndFetch 
>>> and opted to duplicate the trivial functionality in the platform 
>>> files instead. For example:
>>>
>>> +? template<typename D, typename I>
>>> +? D add_and_fetch(D volatile* dest, I add_value, atomic_memory_order 
>>> order) const {
>>> +??? return fetch_and_add(dest, add_value, order) + add_value;
>>> +? }
>> For the record, I would have been fine with (and actually preferred; I
>> dislike roughly a dozen copies of one or the other of the translation
>> functions) CRTP-based AddAndFetch and FetchAndAdd that provided one
>> operation in terms of the other.
>>
>> The move of the scaling of add_value in the pointer case to AddImpl is
>> an improvement (even on the pre-existing code) that might have made
>> such a CRTP approach clearer, possibly with some name changes.? Then
>> the helper CRTP base class just provides one of the functions in terms
>> of Derived's other function, and nothing else.
>>
>> Perhaps the hardest part of that approach is naming the helper base
>> classes.? FetchAndAddUsingAddAndFetch is explicit but quite a
>> mouthful.? Perhaps FetchAndAddHelper, which provides fetch_and_add in
>> terms of Derived's add_and_fetch?
>>
>> Apologies for not having followed the internal pre-review discussion
>> of this and so not commenting there.
>>
>>> There has been some thoughts that maybe we should have:
>>>
>>> void Atomic::add(...)
>>> D??? Atomic::add_and_fetch(...)
>>> D??? Atomic::fetch_and_add(...)
>>>
>>> Not sure if it's worth changing to this, but if others think this is 
>>> good, I can do that change.
>> I would be okay with renaming "add" to "add_and_fetch", though I don't
>> at the moment have a strong preference either way.? I'm not sure also
>> providing "add" that returns void is really all that useful.? We have
>> "inc" and "dec" that return void; my recollection is that they were at
>> one time thought to provide an opportunity for a more efficient
>> implementation on some platforms, but discussion during the
>> templatization project showed that compilers could eliminate an unused
>> return value anyway.
>>
>> For symmetry it seems like we should have fetch_and_sub, but I don't
>> see any current uses of sub that would need that.
>>
>> If add_and_fetch is added (either instead of or in addition to add),
>> then we should definitely treat the subtraction case similarly.
> 
> I'll interpret your comments above that there might be opportunities to 
> do more changes, but none of those needs to be done for this patch.
> 
>>
>>> Tested with tier123, but only compiled on platforms I have access to.
>> mach5 has the ability to cross-compile to several OpenJDK platforms
>> not otherwise supported by Oracle, though can't run tests.? That can
>> be a useful smoke test to at least avoid things like typos that don't
>> build.? For example, "-b linux-aarch64-debug?.
> 
> Right. I wasn't explicit about it, but I've compiled locally (linux) for 
> aarch64, arm32, ppc64, s390, zero, minimal.
> 
> Thanks,
> StefanK
>>
> 

From kim.barrett at oracle.com  Fri Jan 10 07:38:35 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Fri, 10 Jan 2020 02:38:35 -0500
Subject: RFR: 8236778: Add Atomic::fetch_and_add
In-Reply-To: <4F9B2E5D-DF2E-492C-A384-430C09B0DB5B@oracle.com>
References: <3ca1aee9-0edf-e093-68d3-f77ce4dc02e4@oracle.com>
 <E63FAE9D-D556-4BE6-A583-A6F94FD36D86@oracle.com>
 <7b2a936a-e4c0-55fe-3c8a-4627fb59475a@oracle.com>
 <4F9B2E5D-DF2E-492C-A384-430C09B0DB5B@oracle.com>
Message-ID: <8A76CC7A-7CD1-470C-A008-9E3AB94C0BFF@oracle.com>

> On Jan 9, 2020, at 8:50 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
> 
>> On Jan 9, 2020, at 5:17 PM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
>> 
>> Updated webrev:
>> https://cr.openjdk.java.net/~stefank/8236778/webrev.02.delta
>> https://cr.openjdk.java.net/~stefank/8236778/webrev.02
> 
> Looks good except for a couple more comment tweaks described below.

BTW, I don?t need a new webrev for those additional comment tweaks.


From matthias.baesken at sap.com  Fri Jan 10 08:51:29 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 10 Jan 2020 08:51:29 +0000
Subject: RFR: 8236617: jtreg test
 containers/docker/TestMemoryAwareness.java fails after 8226575
References: <AM6PR02MB420053D86671425CAFEC64CF93200@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <FFBF6590-7E38-4DB7-BCA4-867DC19F4C2A@oracle.com>
 <AM6PR02MB4200FFFB14E85F3BC144F35993230@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <AM6PR02MB498486C2DD5EC876F94A45928A3D0@AM6PR02MB4984.eurprd02.prod.outlook.com>
Message-ID: <AM6PR02MB42000534495A178630090C5493380@AM6PR02MB4200.eurprd02.prod.outlook.com>

Ping ...  Martin/Bob are you  fine  with the latest rev too ?

Best regards, Matthias

> Hi Christoph, thanks for the review !
> 
> Bob are you fine with the latest version ?
> 
> Best regards, Matthias
> 
> 
> > >
> > > So I think  your suggestion  to return  0  in that special case in function
> > > getTotalSwapSpaceSize   sounds reasonable to me ( at least better than
> > > return a large negative value ).
> > > New webrev :
> > >
> > >
> > > http://cr.openjdk.java.net/~mbaesken/webrevs/8236617.2/
> > >
> > >
> > >


From stefan.karlsson at oracle.com  Fri Jan 10 08:56:06 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Fri, 10 Jan 2020 09:56:06 +0100
Subject: RFR: 8236778: Add Atomic::fetch_and_add
In-Reply-To: <4F9B2E5D-DF2E-492C-A384-430C09B0DB5B@oracle.com>
References: <3ca1aee9-0edf-e093-68d3-f77ce4dc02e4@oracle.com>
 <E63FAE9D-D556-4BE6-A583-A6F94FD36D86@oracle.com>
 <7b2a936a-e4c0-55fe-3c8a-4627fb59475a@oracle.com>
 <4F9B2E5D-DF2E-492C-A384-430C09B0DB5B@oracle.com>
Message-ID: <e1b256e3-4f56-a1ee-fec7-c4076b2b17d8@oracle.com>

On 2020-01-10 02:50, Kim Barrett wrote:
>> On Jan 9, 2020, at 5:17 PM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
>>
>> Updated webrev:
>>   https://cr.openjdk.java.net/~stefank/8236778/webrev.02.delta
>>   https://cr.openjdk.java.net/~stefank/8236778/webrev.02
> Looks good except for a couple more comment tweaks described below.
>
>>> src/hotspot/share/runtime/atomic.hpp
>>> Removed:
>>>   240   // - platform_add is an object of type PlatformAdd<sizeof(D)>.
>>>   241   //
>>>   242   // Then
>>>   243   //   platform_add(dest, add_value)
>>>   244   // must be a valid expression, returning a result convertible to D.
>>>
>>> and
>>>
>>>   250   // Helper base classes for defining PlatformAdd.  To use, define
>>> ...
>>>   275   // caller.
>>>
>>> These comments should have been updated to describe the new protocol
>>> for PlatformAdd, rather than simply removed.  Documentation of
>>> extension points like this is important.
>> The comments seemed to already have started deteriorating. It's seems to be missing an *if* that matches the *then,
> The structure is "Given <list of requirements> then <implications>";
> there was never an "if".  All of the similar comments in this file
> were written that way.

Well, there's no explicit *given* in the comments either. So, to me, it 
still sounds weird. Updated webrev with your comments below:
 ?https://cr.openjdk.java.net/~stefank/8236778/webrev.03.delta
 ?https://cr.openjdk.java.net/~stefank/8236778/webrev.03

StefanK

> But there's a further update that I just noticed is needed by your changes:
>
>   233   // class is a function object that must be default constructable,
>
> Delete "is a function object that" in the above.
>
>>   and it doesn't mention *order*.
> It's true that *order* isn't mentioned; that seems to have been missed
> when the order argument was added here.  (It *is* described in the
> xchg and cmpxchg commentary, and we could do similarly here.)
>
> Add
>
>    // - order is of type atomic_memory_order.
>
> to the requirements list, and change the expressions to include an
> order argument, e.g.
>
>    // Then both
>    //   platform_add.add_and_fetch(dest, add_value, order)
>    //   platform_add.fetch_and_add(dest, add_value, order)
>
>> So, instead of trying to figure out what the comment tried to say, I opted from getting rid of those problems by getting rid of that part of the comment. I thought the code was pretty self-explanatory, and didn't need that part. I'm not opposed to fixing the comments if you have suggestions.
> Unfortunately, documenting templates can be pretty hard. But because
> of the looseness of templates, differentiating between intentional
> semantics and accidents of today's implementation based only on the
> code can also be pretty hard. Yet failure to do so leads to
> brittleness and uncertainties, especially when there are third-party
> specializations. So I think it's worth making an attempt at
> documentation, even if it's hard and the results not always perfect.
>
>> I'll update the code verbatim with what you've suggested.
> Plus the couple further modifications mentioned above?
>
>>> src/hotspot/os_cpu/windows_x86/atomic_windows_x86.hpp
>>>    54 #pragma warning(disable: 4035) // Disables warnings reporting missing return statement
>>>
>>> Pre-existing: This warning suppression (and the corresponding
>>> restoration) appears to only be needed for the !defined(AMD64) case
>>> (where __asm statements are being used), and should be moved accordingly.
>> Will you create a bug report for that?
> https://bugs.openjdk.java.net/browse/JDK-8236900
> atomic_windows_x86.hpp disables warning 4035 more widely than needed
>
>> I'll interpret your comments above that there might be opportunities to do more changes, but none of those needs to be done for this patch.
> OK.
>
>> Right. I wasn't explicit about it, but I've compiled locally (linux) for aarch64, arm32, ppc64, s390, zero, minimal.
> Ah, that wasn?t clear to me from what you said.  Good.
>


From stefan.karlsson at oracle.com  Fri Jan 10 08:57:26 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Fri, 10 Jan 2020 09:57:26 +0100
Subject: RFR: 8236778: Add Atomic::fetch_and_add
In-Reply-To: <48a620f5-a562-b3b6-3c80-7c6c33fa8ba9@oracle.com>
References: <3ca1aee9-0edf-e093-68d3-f77ce4dc02e4@oracle.com>
 <E63FAE9D-D556-4BE6-A583-A6F94FD36D86@oracle.com>
 <7b2a936a-e4c0-55fe-3c8a-4627fb59475a@oracle.com>
 <48a620f5-a562-b3b6-3c80-7c6c33fa8ba9@oracle.com>
Message-ID: <e875549b-d64f-2f8f-f5d3-8551adde118e@oracle.com>

Thanks for reviewing, David.

StefanK

On 2020-01-10 03:05, David Holmes wrote:
> Hi Stefan,
>
> On 10/01/2020 8:17 am, Stefan Karlsson wrote:
>> Updated webrev:
>> ??https://cr.openjdk.java.net/~stefank/8236778/webrev.02.delta
>> ??https://cr.openjdk.java.net/~stefank/8236778/webrev.02
>
> That all seems okay to me (not that I fully understand the details of 
> the template code). I'll leave you and Kim to work out the details of 
> the commentary.
>
> Thanks,
> David
>
>> Comments below:
>>
>> On 2020-01-09 01:00, Kim Barrett wrote:
>>>> On Jan 8, 2020, at 10:34 AM, Stefan Karlsson 
>>>> <stefan.karlsson at oracle.com> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> Please review this patch to introduce Atomic::fetch_and_add.
>>>>
>>>> https://cr.openjdk.java.net/~stefank/8236778/webrev.01
>>>> https://bugs.openjdk.java.net/browse/JDK-8236778
>>>>
>>>> There are a number of places where we have this pattern:
>>>> int result = Atomic::add(_index, amount) - amount;
>>>>
>>>> I'd like to introduce Atomic::fetch_and_add so that we can write:
>>>> int result = Atomic::fetch_and_add(_index, amount);
>>> ------------------------------------------------------------------------------ 
>>>
>>> src/hotspot/share/runtime/atomic.hpp
>>> Removed:
>>> ? 240?? // - platform_add is an object of type PlatformAdd<sizeof(D)>.
>>> ? 241?? //
>>> ? 242?? // Then
>>> ? 243?? //?? platform_add(dest, add_value)
>>> ? 244?? // must be a valid expression, returning a result 
>>> convertible to D.
>>>
>>> and
>>>
>>> ? 250?? // Helper base classes for defining PlatformAdd.? To use, 
>>> define
>>> ...
>>> ? 275?? // caller.
>>>
>>> These comments should have been updated to describe the new protocol
>>> for PlatformAdd, rather than simply removed.? Documentation of
>>> extension points like this is important.
>>
>> The comments seemed to already have started deteriorating. It's seems 
>> to be missing an *if* that matches the *then, and it doesn't mention 
>> *order*. So, instead of trying to figure out what the comment tried 
>> to say, I opted from getting rid of those problems by getting rid of 
>> that part of the comment. I thought the code was pretty 
>> self-explanatory, and didn't need that part. I'm not opposed to 
>> fixing the comments if you have suggestions.
>>
>>> ?? Something like
>>>
>>> ?? // - platform_add is an object of type PlatformAdd<sizeof(D)>.
>>> ?? //
>>> ?? // Then both
>>> ?? //?? platform_add.add_and_fetch(dest, add_value)
>>> ?? //?? platform_add.fetch_and_add(dest, add_value)
>>> ?? // must be valid expressions returning a result convertible to D.
>>> ?? //
>>> ?? // add_and_fetch atomically adds add_value to the value of dest,
>>> ?? // returning the new value.
>>> ?? //
>>> ?? // fetch_and_add atomically adds add_value to the value of dest,
>>> ?? // returning the old value.
>>> ?? //
>>> ?? // When D is a pointer type P*, both add_and_fetch and fetch_and_add
>>> ?? // treat it as if it were an uintptr_t; they do not perform any
>>> ?? // scaling of add_value, as that has already been done by the 
>>> caller.
>>
>> I'll update the code verbatim with what you've suggested.
>>
>>>
>>> ------------------------------------------------------------------------------ 
>>>
>>> src/hotspot/share/runtime/atomic.hpp
>>> ? 679?? static I scale_addend(I add_value) {
>>> ? 680???? return add_value * sizeof(P);
>>> ? 681?? }
>>> ? 682
>>> ? 683?? static P* add_and_fetch(P* volatile* dest, I add_value, 
>>> atomic_memory_order order) {
>>> ? 684???? CI addend = add_value;
>>> ? 685???? return PlatformAdd<sizeof(P*)>().add_and_fetch(dest, 
>>> scale_addend(addend), order);
>>> ? 686?? }
>>>
>>> This is converting add_value from I to CI then back to I, the latter
>>> possibly being a narrowing conversion.? Better would be
>>>
>>> ?? static CI scale_addend(CI add_value) {
>>> ???? return add_value * sizeof(P);
>>> ?? }
>>>
>>> and then either
>>>
>>> ?? static P* add_and_fetch(P* volatile* dest, I add_value, 
>>> atomic_memory_order order) {
>>> ???? CI addend = scale_addend(add_value);
>>> ???? return PlatformAdd<sizeof(P*)>().add_and_fetch(dest, addend, 
>>> order);
>>> ?? }
>>>
>>> or don't bother with the addend variable and just pass the scaled
>>> result directly to add_and_fetch / fetch_and_add.
>>
>> Thanks. This was an unintentional change from the original code.
>>>
>>> ------------------------------------------------------------------------------ 
>>>
>>> src/hotspot/os_cpu/windows_x86/atomic_windows_x86.hpp
>>> ?? 54 #pragma warning(disable: 4035) // Disables warnings reporting 
>>> missing return statement
>>>
>>> Pre-existing: This warning suppression (and the corresponding
>>> restoration) appears to only be needed for the !defined(AMD64) case
>>> (where __asm statements are being used), and should be moved 
>>> accordingly.
>>
>> Will you create a bug report for that?
>>
>>>
>>> ------------------------------------------------------------------------------ 
>>>
>>>
>>>> The current implementation already has support for both "add and 
>>>> fetch" and "fetch and add" but it's not exposed to the upper layers.
>>>>
>>>> Previously, the platform-specific code either implemented "add and 
>>>> fetch" or "fetch and add", and then exposed it as an "add and 
>>>> fetch" implementation by using CRTP and inheriting from either 
>>>> AddAndFetch or FetchAndAdd.
>>>>
>>>> My first implementation of this continued in this track, but got 
>>>> push-back because the code was non-intuitive and/or used 
>>>> non-intuitive names. Therefore, I've removed 
>>>> FetchAndAdd/AddAndFetch and opted to duplicate the trivial 
>>>> functionality in the platform files instead. For example:
>>>>
>>>> +? template<typename D, typename I>
>>>> +? D add_and_fetch(D volatile* dest, I add_value, 
>>>> atomic_memory_order order) const {
>>>> +??? return fetch_and_add(dest, add_value, order) + add_value;
>>>> +? }
>>> For the record, I would have been fine with (and actually preferred; I
>>> dislike roughly a dozen copies of one or the other of the translation
>>> functions) CRTP-based AddAndFetch and FetchAndAdd that provided one
>>> operation in terms of the other.
>>>
>>> The move of the scaling of add_value in the pointer case to AddImpl is
>>> an improvement (even on the pre-existing code) that might have made
>>> such a CRTP approach clearer, possibly with some name changes.? Then
>>> the helper CRTP base class just provides one of the functions in terms
>>> of Derived's other function, and nothing else.
>>>
>>> Perhaps the hardest part of that approach is naming the helper base
>>> classes.? FetchAndAddUsingAddAndFetch is explicit but quite a
>>> mouthful.? Perhaps FetchAndAddHelper, which provides fetch_and_add in
>>> terms of Derived's add_and_fetch?
>>>
>>> Apologies for not having followed the internal pre-review discussion
>>> of this and so not commenting there.
>>>
>>>> There has been some thoughts that maybe we should have:
>>>>
>>>> void Atomic::add(...)
>>>> D??? Atomic::add_and_fetch(...)
>>>> D??? Atomic::fetch_and_add(...)
>>>>
>>>> Not sure if it's worth changing to this, but if others think this 
>>>> is good, I can do that change.
>>> I would be okay with renaming "add" to "add_and_fetch", though I don't
>>> at the moment have a strong preference either way.? I'm not sure also
>>> providing "add" that returns void is really all that useful. We have
>>> "inc" and "dec" that return void; my recollection is that they were at
>>> one time thought to provide an opportunity for a more efficient
>>> implementation on some platforms, but discussion during the
>>> templatization project showed that compilers could eliminate an unused
>>> return value anyway.
>>>
>>> For symmetry it seems like we should have fetch_and_sub, but I don't
>>> see any current uses of sub that would need that.
>>>
>>> If add_and_fetch is added (either instead of or in addition to add),
>>> then we should definitely treat the subtraction case similarly.
>>
>> I'll interpret your comments above that there might be opportunities 
>> to do more changes, but none of those needs to be done for this patch.
>>
>>>
>>>> Tested with tier123, but only compiled on platforms I have access to.
>>> mach5 has the ability to cross-compile to several OpenJDK platforms
>>> not otherwise supported by Oracle, though can't run tests. That can
>>> be a useful smoke test to at least avoid things like typos that don't
>>> build.? For example, "-b linux-aarch64-debug?.
>>
>> Right. I wasn't explicit about it, but I've compiled locally (linux) 
>> for aarch64, arm32, ppc64, s390, zero, minimal.
>>
>> Thanks,
>> StefanK
>>>
>>


From matthias.baesken at sap.com  Fri Jan 10 10:01:47 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 10 Jan 2020 10:01:47 +0000
Subject: serviceability agent : problems when using  gcc LTO (link time
 optimization)
Message-ID: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello,   I recently looked into  the  gcc  lto  optimization mode (see for some details https://gcc.gnu.org/onlinedocs/gccint/LTO-Overview.html  and  http://hubicka.blogspot.com/2019/05/gcc-9-link-time-and-inter-procedural.html  ).
This mode can lead to more compact binaries (~10% smaller)  , it also might bring  small performance improvements  but that wasn't my (main)  goal  .

The changes for this are rather small , one needs to use a recent gcc  , add  -flto   to the compile flags  , for example

--- a/make/autoconf/flags-cflags.m4      Wed Jan 01 03:08:45 2020 +0100
+++ b/make/autoconf/flags-cflags.m4   Wed Jan 08 17:39:10 2020 +0100
@@ -530,8 +530,13 @@
   fi
   if test "x$TOOLCHAIN_TYPE" = xgcc; then
-    TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new -fstack-protector"
-    TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector"
+    TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new -fstack-protector -flto"
+    TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector -flto"

  .... and you have to make sure  to use  gcc-ar  and  gcc-nm instead   of  ar / nm .
Build and test(s)  work,  however with  one exception.
The  serviceability   tests like  serviceability/sa   seems to rely   heavily  on the "normal"   structure  of   libjvm.so   (from what I   understand  e.g. in  LinuxVtblAccess  it is attempted to access  internal symbols  like  _ZTV ).

Errors in the sa  tests look like :


java.lang.InternalError: Metadata does not appear to be polymorphic
         at jdk.hotspot.agent/sun.jvm.hotspot.types.basic.BasicTypeDataBase.findDynamicTypeForAddress(BasicTypeDataBase.java:279)
         at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VirtualBaseConstructor.instantiateWrapperFor(VirtualBaseConstructor.java:102)
         at jdk.hotspot.agent/sun.jvm.hotspot.oops.Metadata.instantiateWrapperFor(Metadata.java:74)
         at jdk.hotspot.agent/sun.jvm.hotspot.memory.SystemDictionary.getClassLoaderKlass(SystemDictionary.java:96)
         at jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.printClassLoaderStatistics(ClassLoaderStats.java:93)
         at jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.run(ClassLoaderStats.java:78)
         at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.run(JMap.java:115)
         at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:262)
         at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:225)
         at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
         at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.main(JMap.java:176)
         at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJMAP(SALauncher.java:321)
         at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:406)

Has anyone experimented with LTO optimization ?

And to the  serviceability   agent experts -  any idea  how to make the  jdk.hotspot.agent   more independent from  optimization settings ?


Best regards, Matthias

From martin.doerr at sap.com  Fri Jan 10 10:02:20 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Fri, 10 Jan 2020 10:02:20 +0000
Subject: RFR: 8236617: jtreg test
 containers/docker/TestMemoryAwareness.java fails after 8226575
In-Reply-To: <AM6PR02MB42000534495A178630090C5493380@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB420053D86671425CAFEC64CF93200@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <FFBF6590-7E38-4DB7-BCA4-867DC19F4C2A@oracle.com>
 <AM6PR02MB4200FFFB14E85F3BC144F35993230@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <AM6PR02MB498486C2DD5EC876F94A45928A3D0@AM6PR02MB4984.eurprd02.prod.outlook.com>
 <AM6PR02MB42000534495A178630090C5493380@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <AM0PR0202MB3297B8487572BE9B6B0DCD059A380@AM0PR0202MB3297.eurprd02.prod.outlook.com>

Hi Matthias,

yes, looks good to me.

Thanks,
Martin


> -----Original Message-----
> From: Baesken, Matthias <matthias.baesken at sap.com>
> Sent: Freitag, 10. Januar 2020 09:51
> To: Langer, Christoph <christoph.langer at sap.com>; Bob Vandette
> <bob.vandette at oracle.com>; Doerr, Martin <martin.doerr at sap.com>
> Cc: hotspot-dev at openjdk.java.net
> Subject: RE: RFR: 8236617: jtreg test
> containers/docker/TestMemoryAwareness.java fails after 8226575
> 
> Ping ...  Martin/Bob are you  fine  with the latest rev too ?
> 
> Best regards, Matthias
> 
> > Hi Christoph, thanks for the review !
> >
> > Bob are you fine with the latest version ?
> >
> > Best regards, Matthias
> >
> >
> > > >
> > > > So I think  your suggestion  to return  0  in that special case in function
> > > > getTotalSwapSpaceSize   sounds reasonable to me ( at least better than
> > > > return a large negative value ).
> > > > New webrev :
> > > >
> > > >
> > > > http://cr.openjdk.java.net/~mbaesken/webrevs/8236617.2/
> > > >
> > > >
> > > >


From bob.vandette at oracle.com  Fri Jan 10 16:56:25 2020
From: bob.vandette at oracle.com (Bob Vandette)
Date: Fri, 10 Jan 2020 11:56:25 -0500
Subject: RFR: 8236617: jtreg test
 containers/docker/TestMemoryAwareness.java fails after 8226575
In-Reply-To: <AM6PR02MB42000534495A178630090C5493380@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB420053D86671425CAFEC64CF93200@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <FFBF6590-7E38-4DB7-BCA4-867DC19F4C2A@oracle.com>
 <AM6PR02MB4200FFFB14E85F3BC144F35993230@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <AM6PR02MB498486C2DD5EC876F94A45928A3D0@AM6PR02MB4984.eurprd02.prod.outlook.com>
 <AM6PR02MB42000534495A178630090C5493380@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <43D28D45-87E2-42CD-BE49-3575EC1D5153@oracle.com>

Yes, the fix is fine.

Bob.


> On Jan 10, 2020, at 3:51 AM, Baesken, Matthias <matthias.baesken at sap.com> wrote:
> 
> Ping ...  Martin/Bob are you  fine  with the latest rev too ?
> 
> Best regards, Matthias
> 
>> Hi Christoph, thanks for the review !
>> 
>> Bob are you fine with the latest version ?
>> 
>> Best regards, Matthias
>> 
>> 
>>>> 
>>>> So I think  your suggestion  to return  0  in that special case in function
>>>> getTotalSwapSpaceSize   sounds reasonable to me ( at least better than
>>>> return a large negative value ).
>>>> New webrev :
>>>> 
>>>> 
>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8236617.2/
>>>> 
>>>> 
>>>> 
> 


From volker.simonis at gmail.com  Sat Jan 11 13:38:12 2020
From: volker.simonis at gmail.com (Volker Simonis)
Date: Sat, 11 Jan 2020 14:38:12 +0100
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <CA+3eh11OCjjxZ1Yfx8CWTNgF8i0DixPxMcsAa-cf5ufZfsrMWw@mail.gmail.com>

SA pretends to know the exact types of objects in the JVM and for
polymorphic objects it wants to read their vtable from the shared library.
If LTO de-virtulizes methods and thus changes polymorphic to
non-polymorphic types, this won't work. But if LTO can de-virtulizes a
type, maybe you can do that manually (and update the corresponding
representation in the SA), because it doesn't seem to be needed.

Notice that other places in the VM may also rely on this. E.g. CDS stores
Metadata objects in the CDS archive and restores their vtable pointers when
they are loaded. On the other hand, if the CDS tests have passed, this
doesn't seem to be a problem.

Baesken, Matthias <matthias.baesken at sap.com> schrieb am Fr., 10. Jan. 2020,
11:03:

> Hello,   I recently looked into  the  gcc  lto  optimization mode (see for
> some details https://gcc.gnu.org/onlinedocs/gccint/LTO-Overview.html
> and
> http://hubicka.blogspot.com/2019/05/gcc-9-link-time-and-inter-procedural.html
> ).
> This mode can lead to more compact binaries (~10% smaller)  , it also
> might bring  small performance improvements  but that wasn't my (main)
> goal  .
>
> The changes for this are rather small , one needs to use a recent gcc  ,
> add  -flto   to the compile flags  , for example
>
> --- a/make/autoconf/flags-cflags.m4      Wed Jan 01 03:08:45 2020 +0100
> +++ b/make/autoconf/flags-cflags.m4   Wed Jan 08 17:39:10 2020 +0100
> @@ -530,8 +530,13 @@
>    fi
>    if test "x$TOOLCHAIN_TYPE" = xgcc; then
> -    TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new
> -fstack-protector"
> -    TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector"
> +    TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new
> -fstack-protector -flto"
> +    TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector -flto"
>
>   .... and you have to make sure  to use  gcc-ar  and  gcc-nm instead
>  of  ar / nm .
> Build and test(s)  work,  however with  one exception.
> The  serviceability   tests like  serviceability/sa   seems to rely
>  heavily  on the "normal"   structure  of   libjvm.so   (from what I
>  understand  e.g. in  LinuxVtblAccess  it is attempted to access  internal
> symbols  like  _ZTV ).
>
> Errors in the sa  tests look like :
>
>
> java.lang.InternalError: Metadata does not appear to be polymorphic
>          at
> jdk.hotspot.agent/sun.jvm.hotspot.types.basic.BasicTypeDataBase.findDynamicTypeForAddress(BasicTypeDataBase.java:279)
>          at
> jdk.hotspot.agent/sun.jvm.hotspot.runtime.VirtualBaseConstructor.instantiateWrapperFor(VirtualBaseConstructor.java:102)
>          at
> jdk.hotspot.agent/sun.jvm.hotspot.oops.Metadata.instantiateWrapperFor(Metadata.java:74)
>          at
> jdk.hotspot.agent/sun.jvm.hotspot.memory.SystemDictionary.getClassLoaderKlass(SystemDictionary.java:96)
>          at
> jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.printClassLoaderStatistics(ClassLoaderStats.java:93)
>          at
> jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.run(ClassLoaderStats.java:78)
>          at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.run(JMap.java:115)
>          at
> jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:262)
>          at
> jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:225)
>          at
> jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
>          at
> jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.main(JMap.java:176)
>          at
> jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJMAP(SALauncher.java:321)
>          at
> jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:406)
>
> Has anyone experimented with LTO optimization ?
>
> And to the  serviceability   agent experts -  any idea  how to make the
> jdk.hotspot.agent   more independent from  optimization settings ?
>
>
> Best regards, Matthias
>

From chris.plummer at oracle.com  Sat Jan 11 18:27:03 2020
From: chris.plummer at oracle.com (Chris Plummer)
Date: Sat, 11 Jan 2020 10:27:03 -0800
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <CA+3eh11OCjjxZ1Yfx8CWTNgF8i0DixPxMcsAa-cf5ufZfsrMWw@mail.gmail.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <CA+3eh11OCjjxZ1Yfx8CWTNgF8i0DixPxMcsAa-cf5ufZfsrMWw@mail.gmail.com>
Message-ID: <00f43a29-1000-e5b7-1e36-c6b9b6177d21@oracle.com>

cds is also disabled for minimalVM so testing of cds with LTO probably 
has not been done. There are a number of features that minimalVM 
excludes such as jvmti, cds and SA (which I think falls under 
"services"), and there was very little testing done with these features 
individually disabled. They would all at least build (if any one was 
disabled) and I think heartbeat testing was done, but probably no more 
than that. Also various combinations were not tested, other than the one 
combination that minimalVM used. Search for NON_MINIMAL_FEATURES in 
hotspot.m4 to see which features are disabled for minimalVM.

Chris

On 1/11/20 5:38 AM, Volker Simonis wrote:
> SA pretends to know the exact types of objects in the JVM and for
> polymorphic objects it wants to read their vtable from the shared library.
> If LTO de-virtulizes methods and thus changes polymorphic to
> non-polymorphic types, this won't work. But if LTO can de-virtulizes a
> type, maybe you can do that manually (and update the corresponding
> representation in the SA), because it doesn't seem to be needed.
>
> Notice that other places in the VM may also rely on this. E.g. CDS stores
> Metadata objects in the CDS archive and restores their vtable pointers when
> they are loaded. On the other hand, if the CDS tests have passed, this
> doesn't seem to be a problem.
>
> Baesken, Matthias <matthias.baesken at sap.com> schrieb am Fr., 10. Jan. 2020,
> 11:03:
>
>> Hello,   I recently looked into  the  gcc  lto  optimization mode (see for
>> some details https://gcc.gnu.org/onlinedocs/gccint/LTO-Overview.html
>> and
>> http://hubicka.blogspot.com/2019/05/gcc-9-link-time-and-inter-procedural.html
>> ).
>> This mode can lead to more compact binaries (~10% smaller)  , it also
>> might bring  small performance improvements  but that wasn't my (main)
>> goal  .
>>
>> The changes for this are rather small , one needs to use a recent gcc  ,
>> add  -flto   to the compile flags  , for example
>>
>> --- a/make/autoconf/flags-cflags.m4      Wed Jan 01 03:08:45 2020 +0100
>> +++ b/make/autoconf/flags-cflags.m4   Wed Jan 08 17:39:10 2020 +0100
>> @@ -530,8 +530,13 @@
>>     fi
>>     if test "x$TOOLCHAIN_TYPE" = xgcc; then
>> -    TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new
>> -fstack-protector"
>> -    TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector"
>> +    TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new
>> -fstack-protector -flto"
>> +    TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector -flto"
>>
>>    .... and you have to make sure  to use  gcc-ar  and  gcc-nm instead
>>   of  ar / nm .
>> Build and test(s)  work,  however with  one exception.
>> The  serviceability   tests like  serviceability/sa   seems to rely
>>   heavily  on the "normal"   structure  of   libjvm.so   (from what I
>>   understand  e.g. in  LinuxVtblAccess  it is attempted to access  internal
>> symbols  like  _ZTV ).
>>
>> Errors in the sa  tests look like :
>>
>>
>> java.lang.InternalError: Metadata does not appear to be polymorphic
>>           at
>> jdk.hotspot.agent/sun.jvm.hotspot.types.basic.BasicTypeDataBase.findDynamicTypeForAddress(BasicTypeDataBase.java:279)
>>           at
>> jdk.hotspot.agent/sun.jvm.hotspot.runtime.VirtualBaseConstructor.instantiateWrapperFor(VirtualBaseConstructor.java:102)
>>           at
>> jdk.hotspot.agent/sun.jvm.hotspot.oops.Metadata.instantiateWrapperFor(Metadata.java:74)
>>           at
>> jdk.hotspot.agent/sun.jvm.hotspot.memory.SystemDictionary.getClassLoaderKlass(SystemDictionary.java:96)
>>           at
>> jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.printClassLoaderStatistics(ClassLoaderStats.java:93)
>>           at
>> jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.run(ClassLoaderStats.java:78)
>>           at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.run(JMap.java:115)
>>           at
>> jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:262)
>>           at
>> jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:225)
>>           at
>> jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
>>           at
>> jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.main(JMap.java:176)
>>           at
>> jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJMAP(SALauncher.java:321)
>>           at
>> jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:406)
>>
>> Has anyone experimented with LTO optimization ?
>>
>> And to the  serviceability   agent experts -  any idea  how to make the
>> jdk.hotspot.agent   more independent from  optimization settings ?
>>
>>
>> Best regards, Matthias
>>


From david.holmes at oracle.com  Mon Jan 13 07:13:34 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 13 Jan 2020 17:13:34 +1000
Subject: RFR (M): 8235741: Inappropriate uses of os::javaTimeMillis()
Message-ID: <f99194ad-3dad-48db-d8d4-de0cc9161701@oracle.com>

webrev: http://cr.openjdk.java.net/~dholmes/8235741/webrev/
bug: https://bugs.openjdk.java.net/browse/JDK-8235741

Full details in the bug report about the existing uses of 
javaTimeMillis(), many of which just want an elapsed time in ms and so 
should be using javaTimeNanos() and convert to ms. This covers areas all 
across the VM.

Only non-simple change is in os_perf_linux.cpp (and the same code will 
be in os_perf_aix.cpp once it has been validated). There we are tracking 
an elapsed time in ms but relative to the boot time, which is seconds 
since the epoch. Consequently the first interval has to be calculated 
using javaTimeMillis, but after that we can use javaTimeNanos (using a 
new 'first time' captured at the same time we used javaTimeMillis). I 
think I have the logic right but other than through JFR this code seems 
unused and I have limited means of testing it. The JFR test 
jdk/jfr/event/os/TestThreadContextSwitches.java exercises the code but 
the results of running that test seems to exhibit arbitrary randomness 
in the rates reported - e.g. 0 to 16000Hz - both with and without my 
change, so not really that useful. Stefan K. suggested a gtest which I 
may look into - though it is frustrating to have to expend such effort 
to validate this.

Other testing tiers 1-3.

Thanks,
David

From shade at redhat.com  Mon Jan 13 07:52:03 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Mon, 13 Jan 2020 08:52:03 +0100
Subject: RFR (M): 8235741: Inappropriate uses of os::javaTimeMillis()
In-Reply-To: <f99194ad-3dad-48db-d8d4-de0cc9161701@oracle.com>
References: <f99194ad-3dad-48db-d8d4-de0cc9161701@oracle.com>
Message-ID: <7e4e82f2-b1e6-e79a-755e-39c86d957e9c@redhat.com>

On 1/13/20 8:13 AM, David Holmes wrote:
> webrev: http://cr.openjdk.java.net/~dholmes/8235741/webrev/
> bug: https://bugs.openjdk.java.net/browse/JDK-8235741

Shenandoah change looks good.

>From the cursory look over the other changes, those look good too.

-- 
Thanks,
-Aleksey


From matthias.baesken at sap.com  Mon Jan 13 08:27:28 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Mon, 13 Jan 2020 08:27:28 +0000
Subject: RFR: 8236617: jtreg test
 containers/docker/TestMemoryAwareness.java fails after 8226575
In-Reply-To: <43D28D45-87E2-42CD-BE49-3575EC1D5153@oracle.com>
References: <AM6PR02MB420053D86671425CAFEC64CF93200@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <FFBF6590-7E38-4DB7-BCA4-867DC19F4C2A@oracle.com>
 <AM6PR02MB4200FFFB14E85F3BC144F35993230@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <AM6PR02MB498486C2DD5EC876F94A45928A3D0@AM6PR02MB4984.eurprd02.prod.outlook.com>
 <AM6PR02MB42000534495A178630090C5493380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <43D28D45-87E2-42CD-BE49-3575EC1D5153@oracle.com>
Message-ID: <AM6PR02MB4200380E89A98E838DE35C6693350@AM6PR02MB4200.eurprd02.prod.outlook.com>

Thanks for the reviews !

> 
> Yes, the fix is fine.
> 
> Bob.
> 
> 
> > On Jan 10, 2020, at 3:51 AM, Baesken, Matthias
> <matthias.baesken at sap.com> wrote:
> >
> > Ping ...  Martin/Bob are you  fine  with the latest rev too ?
> >
> > Best regards, Matthias
> >
> >> Hi Christoph, thanks for the review !
> >>
> >> Bob are you fine with the latest version ?
> >>
> >> Best regards, Matthias
> >>
> >>
> >>>>
> >>>> So I think  your suggestion  to return  0  in that special case in function
> >>>> getTotalSwapSpaceSize   sounds reasonable to me ( at least better than
> >>>> return a large negative value ).
> >>>> New webrev :
> >>>>
> >>>>
> >>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8236617.2/
> >>>>
> >>>>
> >>>>
> >


From matthias.baesken at sap.com  Mon Jan 13 09:28:22 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Mon, 13 Jan 2020 09:28:22 +0000
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <00f43a29-1000-e5b7-1e36-c6b9b6177d21@oracle.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <CA+3eh11OCjjxZ1Yfx8CWTNgF8i0DixPxMcsAa-cf5ufZfsrMWw@mail.gmail.com>
 <00f43a29-1000-e5b7-1e36-c6b9b6177d21@oracle.com>
Message-ID: <AM6PR02MB4200939AB9AB4BFB1BF91D3A93350@AM6PR02MB4200.eurprd02.prod.outlook.com>


Hello, thanks for  the info -  seems that for the minimal  VM ,  lto is fine but  currently not for the other (server/...) VM builds .

Btw.   I noticed similar issues with the SA  when using  link-time-gc . Looks like this eliminates the vtable info too that tha SA-coding  ( LinuxVtblAccess class ?)  wants to look into .


Best regards, Matthias


> cds is also disabled for minimalVM so testing of cds with LTO probably
> has not been done. There are a number of features that minimalVM
> excludes such as jvmti, cds and SA (which I think falls under
> "services"), and there was very little testing done with these features
> individually disabled. They would all at least build (if any one was
> disabled) and I think heartbeat testing was done, but probably no more
> than that. Also various combinations were not tested, other than the one
> combination that minimalVM used. Search for NON_MINIMAL_FEATURES in
> hotspot.m4 to see which features are disabled for minimalVM.
> 
> Chris
> 
> On 1/11/20 5:38 AM, Volker Simonis wrote:
> > SA pretends to know the exact types of objects in the JVM and for
> > polymorphic objects it wants to read their vtable from the shared library.
> > If LTO de-virtulizes methods and thus changes polymorphic to
> > non-polymorphic types, this won't work. But if LTO can de-virtulizes a
> > type, maybe you can do that manually (and update the corresponding
> > representation in the SA), because it doesn't seem to be needed.
> >
> > Notice that other places in the VM may also rely on this. E.g. CDS stores
> > Metadata objects in the CDS archive and restores their vtable pointers
> when
> > they are loaded. On the other hand, if the CDS tests have passed, this
> > doesn't seem to be a problem.
> >
> > Baesken, Matthias <matthias.baesken at sap.com> schrieb am Fr., 10. Jan.
> 2020,
> > 11:03:
> >
> >> Hello,   I recently looked into  the  gcc  lto  optimization mode (see for
> >> some details https://gcc.gnu.org/onlinedocs/gccint/LTO-Overview.html
> >> and
> >> http://hubicka.blogspot.com/2019/05/gcc-9-link-time-and-inter-
> procedural.html
> >> ).
> >> This mode can lead to more compact binaries (~10% smaller)  , it also
> >> might bring  small performance improvements  but that wasn't my (main)
> >> goal  .
> >>
> >> The changes for this are rather small , one needs to use a recent gcc  ,
> >> add  -flto   to the compile flags  , for example
> >>
> >> --- a/make/autoconf/flags-cflags.m4      Wed Jan 01 03:08:45 2020 +0100
> >> +++ b/make/autoconf/flags-cflags.m4   Wed Jan 08 17:39:10 2020 +0100
> >> @@ -530,8 +530,13 @@
> >>     fi
> >>     if test "x$TOOLCHAIN_TYPE" = xgcc; then
> >> -    TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new
> >> -fstack-protector"
> >> -    TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector"
> >> +    TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new
> >> -fstack-protector -flto"
> >> +    TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector -flto"
> >>
> >>    .... and you have to make sure  to use  gcc-ar  and  gcc-nm instead
> >>   of  ar / nm .
> >> Build and test(s)  work,  however with  one exception.
> >> The  serviceability   tests like  serviceability/sa   seems to rely
> >>   heavily  on the "normal"   structure  of   libjvm.so   (from what I
> >>   understand  e.g. in  LinuxVtblAccess  it is attempted to access  internal
> >> symbols  like  _ZTV ).
> >>
> >> Errors in the sa  tests look like :
> >>
> >>
> >> java.lang.InternalError: Metadata does not appear to be polymorphic
> >>           at
> >>
> jdk.hotspot.agent/sun.jvm.hotspot.types.basic.BasicTypeDataBase.findDyna
> micTypeForAddress(BasicTypeDataBase.java:279)
> >>           at
> >>
> jdk.hotspot.agent/sun.jvm.hotspot.runtime.VirtualBaseConstructor.instanti
> ateWrapperFor(VirtualBaseConstructor.java:102)
> >>           at
> >>
> jdk.hotspot.agent/sun.jvm.hotspot.oops.Metadata.instantiateWrapperFor(
> Metadata.java:74)
> >>           at
> >>
> jdk.hotspot.agent/sun.jvm.hotspot.memory.SystemDictionary.getClassLoad
> erKlass(SystemDictionary.java:96)
> >>           at
> >>
> jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.printClassLoaderS
> tatistics(ClassLoaderStats.java:93)
> >>           at
> >>
> jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.run(ClassLoaderS
> tats.java:78)
> >>           at
> jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.run(JMap.java:115)
> >>           at
> >> jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:262)
> >>           at
> >> jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:225)
> >>           at
> >> jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
> >>           at
> >> jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.main(JMap.java:176)
> >>           at
> >>
> jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJMAP(SALauncher.java:3
> 21)
> >>           at
> >>
> jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:406)
> >>
> >> Has anyone experimented with LTO optimization ?
> >>
> >> And to the  serviceability   agent experts -  any idea  how to make the
> >> jdk.hotspot.agent   more independent from  optimization settings ?
> >>
> >>
> >> Best regards, Matthias
> >>


From christoph.langer at sap.com  Mon Jan 13 10:23:38 2020
From: christoph.langer at sap.com (Langer, Christoph)
Date: Mon, 13 Jan 2020 10:23:38 +0000
Subject: RFR (XS): 8237008: Exclude
 serviceability/sa/TestInstanceKlassSizeForInterface.java on linuxppc64 and
 linuxppc64le
Message-ID: <AM0PR02MB5714B60661BB2222D5ECC3488A350@AM0PR02MB5714.eurprd02.prod.outlook.com>

Hi,

after JDK-8232759 "Remove GC.class_stats" [0] was pushed, we see the test "serviceability/sa/TestInstanceKlassSizeForInterface.java" failing on linuxppc64 and linuxppc64le the same way as "serviceability/sa/TestInstanceKlassSize.java". Hence, I'm requesting the same exclusion of TestInstanceKlassSizeForInterface (referring to JDK-8230664 [1] for resolution).

Bug: https://bugs.openjdk.java.net/browse/JDK-8237008
Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8237008.0/

Thanks
Christoph

[0] https://hg.openjdk.java.net/jdk/jdk/rev/d8f6e926cedc
[1] https://bugs.openjdk.java.net/browse/JDK-8230664


From martin.doerr at sap.com  Mon Jan 13 10:36:34 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Mon, 13 Jan 2020 10:36:34 +0000
Subject: RFR (XS): 8237008: Exclude
 serviceability/sa/TestInstanceKlassSizeForInterface.java on linuxppc64 and
 linuxppc64le
In-Reply-To: <AM0PR02MB5714B60661BB2222D5ECC3488A350@AM0PR02MB5714.eurprd02.prod.outlook.com>
References: <AM0PR02MB5714B60661BB2222D5ECC3488A350@AM0PR02MB5714.eurprd02.prod.outlook.com>
Message-ID: <AM0PR0202MB329728CDCD40049B497A7EBA9A350@AM0PR0202MB3297.eurprd02.prod.outlook.com>

Hi Christoph,

looks good to me.

Thanks,
Martin


From: Langer, Christoph <christoph.langer at sap.com>
Sent: Montag, 13. Januar 2020 11:24
To: hotspot-dev at openjdk.java.net; Doerr, Martin <martin.doerr at sap.com>
Cc: OpenJDK Serviceability <serviceability-dev at openjdk.java.net>
Subject: RFR (XS): 8237008: Exclude serviceability/sa/TestInstanceKlassSizeForInterface.java on linuxppc64 and linuxppc64le

Hi,

after JDK-8232759 "Remove GC.class_stats" [0] was pushed, we see the test "serviceability/sa/TestInstanceKlassSizeForInterface.java" failing on linuxppc64 and linuxppc64le the same way as "serviceability/sa/TestInstanceKlassSize.java". Hence, I'm requesting the same exclusion of TestInstanceKlassSizeForInterface (referring to JDK-8230664 [1] for resolution).

Bug: https://bugs.openjdk.java.net/browse/JDK-8237008
Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8237008.0/

Thanks
Christoph

[0] https://hg.openjdk.java.net/jdk/jdk/rev/d8f6e926cedc
[1] https://bugs.openjdk.java.net/browse/JDK-8230664


From david.holmes at oracle.com  Mon Jan 13 12:52:26 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 13 Jan 2020 22:52:26 +1000
Subject: RFR (M): 8235741: Inappropriate uses of os::javaTimeMillis()
In-Reply-To: <7e4e82f2-b1e6-e79a-755e-39c86d957e9c@redhat.com>
References: <f99194ad-3dad-48db-d8d4-de0cc9161701@oracle.com>
 <7e4e82f2-b1e6-e79a-755e-39c86d957e9c@redhat.com>
Message-ID: <2ca62f20-8748-c043-d3f1-ab3f3a78a2a6@oracle.com>

On 13/01/2020 5:52 pm, Aleksey Shipilev wrote:
> On 1/13/20 8:13 AM, David Holmes wrote:
>> webrev: http://cr.openjdk.java.net/~dholmes/8235741/webrev/
>> bug: https://bugs.openjdk.java.net/browse/JDK-8235741
> 
> Shenandoah change looks good.
> 
>  From the cursory look over the other changes, those look good too.

Thanks Aleksey!

David


From david.holmes at oracle.com  Mon Jan 13 12:57:00 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 13 Jan 2020 22:57:00 +1000
Subject: RFR (XS): 8237008: Exclude
 serviceability/sa/TestInstanceKlassSizeForInterface.java on linuxppc64 and
 linuxppc64le
In-Reply-To: <AM0PR02MB5714B60661BB2222D5ECC3488A350@AM0PR02MB5714.eurprd02.prod.outlook.com>
References: <AM0PR02MB5714B60661BB2222D5ECC3488A350@AM0PR02MB5714.eurprd02.prod.outlook.com>
Message-ID: <8a6a9f3d-88ab-9733-7c7f-f822da148d1e@oracle.com>

Hi Christoph,

I think those tests are currnetly failing on all platforms - see 
JDK-8236917. The failures after GC.class_stats removal are unrelated to 
the failures reported in 8230664 AFAICS.

David

On 13/01/2020 8:23 pm, Langer, Christoph wrote:
> Hi,
> 
> after JDK-8232759 ?Remove GC.class_stats? [0] was pushed, we see the 
> test ?serviceability/sa/TestInstanceKlassSizeForInterface.java? failing 
> on linuxppc64 and linuxppc64le the same way as 
> ?serviceability/sa/TestInstanceKlassSize.java?. Hence, I?m requesting 
> the same exclusion of TestInstanceKlassSizeForInterface (referring to 
> JDK-8230664 [1] for resolution).
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8237008
> 
> Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8237008.0/
> 
> Thanks
> 
> Christoph
> 
> [0] https://hg.openjdk.java.net/jdk/jdk/rev/d8f6e926cedc
> 
> [1] https://bugs.openjdk.java.net/browse/JDK-8230664
> 

From robbin.ehn at oracle.com  Mon Jan 13 13:08:21 2020
From: robbin.ehn at oracle.com (Robbin Ehn)
Date: Mon, 13 Jan 2020 14:08:21 +0100
Subject: RFR (M): 8235741: Inappropriate uses of os::javaTimeMillis()
In-Reply-To: <f99194ad-3dad-48db-d8d4-de0cc9161701@oracle.com>
References: <f99194ad-3dad-48db-d8d4-de0cc9161701@oracle.com>
Message-ID: <75f4cd8d-d291-2cf5-b0dd-91a1d76a6130@oracle.com>

Hi David, looks good, thanks!

/Robbin

On 1/13/20 8:13 AM, David Holmes wrote:
> webrev: http://cr.openjdk.java.net/~dholmes/8235741/webrev/
> bug: https://bugs.openjdk.java.net/browse/JDK-8235741
> 
> Full details in the bug report about the existing uses of javaTimeMillis(), many 
> of which just want an elapsed time in ms and so should be using javaTimeNanos() 
> and convert to ms. This covers areas all across the VM.
> 
> Only non-simple change is in os_perf_linux.cpp (and the same code will be in 
> os_perf_aix.cpp once it has been validated). There we are tracking an elapsed 
> time in ms but relative to the boot time, which is seconds since the epoch. 
> Consequently the first interval has to be calculated using javaTimeMillis, but 
> after that we can use javaTimeNanos (using a new 'first time' captured at the 
> same time we used javaTimeMillis). I think I have the logic right but other than 
> through JFR this code seems unused and I have limited means of testing it. The 
> JFR test jdk/jfr/event/os/TestThreadContextSwitches.java exercises the code but 
> the results of running that test seems to exhibit arbitrary randomness in the 
> rates reported - e.g. 0 to 16000Hz - both with and without my change, so not 
> really that useful. Stefan K. suggested a gtest which I may look into - though 
> it is frustrating to have to expend such effort to validate this.
> 
> Other testing tiers 1-3.
> 
> Thanks,
> David

From christoph.langer at sap.com  Mon Jan 13 13:26:46 2020
From: christoph.langer at sap.com (Langer, Christoph)
Date: Mon, 13 Jan 2020 13:26:46 +0000
Subject: RFR (XS): 8237008: Exclude
 serviceability/sa/TestInstanceKlassSizeForInterface.java on linuxppc64 and
 linuxppc64le
In-Reply-To: <8a6a9f3d-88ab-9733-7c7f-f822da148d1e@oracle.com>
References: <AM0PR02MB5714B60661BB2222D5ECC3488A350@AM0PR02MB5714.eurprd02.prod.outlook.com>
 <8a6a9f3d-88ab-9733-7c7f-f822da148d1e@oracle.com>
Message-ID: <AM0PR02MB57148C98A930DCC8D4E73E888A350@AM0PR02MB5714.eurprd02.prod.outlook.com>

Hi David,

thanks for the heads up. However, in our CI, these tests pass on all platforms except linuxppc64/linuxppc64le consistently.

I think I'll push the exclusion and once JDK-8236917 has been resolved I'll try on the ppc linuxes again...

Best regards
Christoph

> -----Original Message-----
> From: David Holmes <david.holmes at oracle.com>
> Sent: Montag, 13. Januar 2020 13:57
> To: Langer, Christoph <christoph.langer at sap.com>; hotspot-
> dev at openjdk.java.net; Doerr, Martin <martin.doerr at sap.com>
> Cc: OpenJDK Serviceability <serviceability-dev at openjdk.java.net>
> Subject: Re: RFR (XS): 8237008: Exclude
> serviceability/sa/TestInstanceKlassSizeForInterface.java on linuxppc64 and
> linuxppc64le
> 
> Hi Christoph,
> 
> I think those tests are currnetly failing on all platforms - see
> JDK-8236917. The failures after GC.class_stats removal are unrelated to
> the failures reported in 8230664 AFAICS.
> 
> David
> 
> On 13/01/2020 8:23 pm, Langer, Christoph wrote:
> > Hi,
> >
> > after JDK-8232759 "Remove GC.class_stats" [0] was pushed, we see the
> > test "serviceability/sa/TestInstanceKlassSizeForInterface.java" failing
> > on linuxppc64 and linuxppc64le the same way as
> > "serviceability/sa/TestInstanceKlassSize.java". Hence, I'm requesting
> > the same exclusion of TestInstanceKlassSizeForInterface (referring to
> > JDK-8230664 [1] for resolution).
> >
> > Bug: https://bugs.openjdk.java.net/browse/JDK-8237008
> >
> > Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8237008.0/
> >
> > Thanks
> >
> > Christoph
> >
> > [0] https://hg.openjdk.java.net/jdk/jdk/rev/d8f6e926cedc
> >
> > [1] https://bugs.openjdk.java.net/browse/JDK-8230664
> >

From coleen.phillimore at oracle.com  Mon Jan 13 14:22:07 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Mon, 13 Jan 2020 09:22:07 -0500
Subject: RFR (XS): 8237008: Exclude
 serviceability/sa/TestInstanceKlassSizeForInterface.java on linuxppc64 and
 linuxppc64le
In-Reply-To: <AM0PR02MB57148C98A930DCC8D4E73E888A350@AM0PR02MB5714.eurprd02.prod.outlook.com>
References: <AM0PR02MB5714B60661BB2222D5ECC3488A350@AM0PR02MB5714.eurprd02.prod.outlook.com>
 <8a6a9f3d-88ab-9733-7c7f-f822da148d1e@oracle.com>
 <AM0PR02MB57148C98A930DCC8D4E73E888A350@AM0PR02MB5714.eurprd02.prod.outlook.com>
Message-ID: <60869e5e-d0ae-86e7-b514-897b5ae0bdc4@oracle.com>

Hi, I didn't see this with my filtering.? Do you know why it fails for 
ppcle?

It fails for us on all platforms because test.vm.opts isn't set in our 
CI jobs.

Thanks,
Coleen

On 1/13/20 8:26 AM, Langer, Christoph wrote:
> Hi David,
>
> thanks for the heads up. However, in our CI, these tests pass on all platforms except linuxppc64/linuxppc64le consistently.
>
> I think I'll push the exclusion and once JDK-8236917 has been resolved I'll try on the ppc linuxes again...
>
> Best regards
> Christoph
>
>> -----Original Message-----
>> From: David Holmes <david.holmes at oracle.com>
>> Sent: Montag, 13. Januar 2020 13:57
>> To: Langer, Christoph <christoph.langer at sap.com>; hotspot-
>> dev at openjdk.java.net; Doerr, Martin <martin.doerr at sap.com>
>> Cc: OpenJDK Serviceability <serviceability-dev at openjdk.java.net>
>> Subject: Re: RFR (XS): 8237008: Exclude
>> serviceability/sa/TestInstanceKlassSizeForInterface.java on linuxppc64 and
>> linuxppc64le
>>
>> Hi Christoph,
>>
>> I think those tests are currnetly failing on all platforms - see
>> JDK-8236917. The failures after GC.class_stats removal are unrelated to
>> the failures reported in 8230664 AFAICS.
>>
>> David
>>
>> On 13/01/2020 8:23 pm, Langer, Christoph wrote:
>>> Hi,
>>>
>>> after JDK-8232759 "Remove GC.class_stats" [0] was pushed, we see the
>>> test "serviceability/sa/TestInstanceKlassSizeForInterface.java" failing
>>> on linuxppc64 and linuxppc64le the same way as
>>> "serviceability/sa/TestInstanceKlassSize.java". Hence, I'm requesting
>>> the same exclusion of TestInstanceKlassSizeForInterface (referring to
>>> JDK-8230664 [1] for resolution).
>>>
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8237008
>>>
>>> Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8237008.0/
>>>
>>> Thanks
>>>
>>> Christoph
>>>
>>> [0] https://hg.openjdk.java.net/jdk/jdk/rev/d8f6e926cedc
>>>
>>> [1] https://bugs.openjdk.java.net/browse/JDK-8230664
>>>


From christoph.langer at sap.com  Mon Jan 13 14:49:34 2020
From: christoph.langer at sap.com (Langer, Christoph)
Date: Mon, 13 Jan 2020 14:49:34 +0000
Subject: RFR (XS): 8237008: Exclude
 serviceability/sa/TestInstanceKlassSizeForInterface.java on linuxppc64 and
 linuxppc64le
In-Reply-To: <60869e5e-d0ae-86e7-b514-897b5ae0bdc4@oracle.com>
References: <AM0PR02MB5714B60661BB2222D5ECC3488A350@AM0PR02MB5714.eurprd02.prod.outlook.com>
 <8a6a9f3d-88ab-9733-7c7f-f822da148d1e@oracle.com>
 <AM0PR02MB57148C98A930DCC8D4E73E888A350@AM0PR02MB5714.eurprd02.prod.outlook.com>
 <60869e5e-d0ae-86e7-b514-897b5ae0bdc4@oracle.com>
Message-ID: <AM0PR02MB571467212E2425CC2564F08B8A350@AM0PR02MB5714.eurprd02.prod.outlook.com>

Hi Coleen,

why it's failing on ppc or ppcle was analyzed in https://bugs.openjdk.java.net/browse/JDK-8230664. You can read in the description:

The test retrieves the size of Java classes by 2 different APIs and expects the result to be equal:
- SA reports the size of the reserved memory. Rounded up to a multiple of 16 on PPC64.
- Jcmd GC.class_stats reports the size of the space needed for a number of 8 Byte blocks. (The number is from an internal statistic.)
So there may be a difference of 8 Bytes on platforms which reserve memory 16 Byte wise.

I assume that the functionality from Jcmd GC.class_stats was replaced in your change but goes back to the same statistics.

In our tests we set test.vm.opts, so I guess that's why we don't see failures on other platforms.

Best regards
Christoph




> -----Original Message-----
> From: coleen.phillimore at oracle.com <coleen.phillimore at oracle.com>
> Sent: Montag, 13. Januar 2020 15:22
> To: Langer, Christoph <christoph.langer at sap.com>; David Holmes
> <david.holmes at oracle.com>; hotspot-dev at openjdk.java.net; Doerr, Martin
> <martin.doerr at sap.com>
> Cc: OpenJDK Serviceability <serviceability-dev at openjdk.java.net>
> Subject: Re: RFR (XS): 8237008: Exclude
> serviceability/sa/TestInstanceKlassSizeForInterface.java on linuxppc64 and
> linuxppc64le
> 
> Hi, I didn't see this with my filtering.? Do you know why it fails for
> ppcle?
> 
> It fails for us on all platforms because test.vm.opts isn't set in our
> CI jobs.
> 
> Thanks,
> Coleen
> 
> On 1/13/20 8:26 AM, Langer, Christoph wrote:
> > Hi David,
> >
> > thanks for the heads up. However, in our CI, these tests pass on all
> platforms except linuxppc64/linuxppc64le consistently.
> >
> > I think I'll push the exclusion and once JDK-8236917 has been resolved I'll
> try on the ppc linuxes again...
> >
> > Best regards
> > Christoph
> >
> >> -----Original Message-----
> >> From: David Holmes <david.holmes at oracle.com>
> >> Sent: Montag, 13. Januar 2020 13:57
> >> To: Langer, Christoph <christoph.langer at sap.com>; hotspot-
> >> dev at openjdk.java.net; Doerr, Martin <martin.doerr at sap.com>
> >> Cc: OpenJDK Serviceability <serviceability-dev at openjdk.java.net>
> >> Subject: Re: RFR (XS): 8237008: Exclude
> >> serviceability/sa/TestInstanceKlassSizeForInterface.java on linuxppc64
> and
> >> linuxppc64le
> >>
> >> Hi Christoph,
> >>
> >> I think those tests are currnetly failing on all platforms - see
> >> JDK-8236917. The failures after GC.class_stats removal are unrelated to
> >> the failures reported in 8230664 AFAICS.
> >>
> >> David
> >>
> >> On 13/01/2020 8:23 pm, Langer, Christoph wrote:
> >>> Hi,
> >>>
> >>> after JDK-8232759 "Remove GC.class_stats" [0] was pushed, we see the
> >>> test "serviceability/sa/TestInstanceKlassSizeForInterface.java" failing
> >>> on linuxppc64 and linuxppc64le the same way as
> >>> "serviceability/sa/TestInstanceKlassSize.java". Hence, I'm requesting
> >>> the same exclusion of TestInstanceKlassSizeForInterface (referring to
> >>> JDK-8230664 [1] for resolution).
> >>>
> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8237008
> >>>
> >>> Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8237008.0/
> >>>
> >>> Thanks
> >>>
> >>> Christoph
> >>>
> >>> [0] https://hg.openjdk.java.net/jdk/jdk/rev/d8f6e926cedc
> >>>
> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8230664
> >>>


From ioi.lam at oracle.com  Mon Jan 13 17:46:02 2020
From: ioi.lam at oracle.com (Ioi Lam)
Date: Mon, 13 Jan 2020 09:46:02 -0800
Subject: RFR (XS): 8237008: Exclude
 serviceability/sa/TestInstanceKlassSizeForInterface.java on linuxppc64 and
 linuxppc64le
In-Reply-To: <60869e5e-d0ae-86e7-b514-897b5ae0bdc4@oracle.com>
References: <AM0PR02MB5714B60661BB2222D5ECC3488A350@AM0PR02MB5714.eurprd02.prod.outlook.com>
 <8a6a9f3d-88ab-9733-7c7f-f822da148d1e@oracle.com>
 <AM0PR02MB57148C98A930DCC8D4E73E888A350@AM0PR02MB5714.eurprd02.prod.outlook.com>
 <60869e5e-d0ae-86e7-b514-897b5ae0bdc4@oracle.com>
Message-ID: <75785e0b-adef-b61c-707d-5ff9604711b1@oracle.com>

This test passed on linux/x64 on my machine, but failed on our linux/x64 
CI as well. I think there's a fundamental bug that's triggered only by 
certain CPU and/or environment combinations. So this test should be 
excluded on all platforms for now until the fundamental issue is fixed.

Thanks
- Ioi

On 1/13/20 6:22 AM, coleen.phillimore at oracle.com wrote:
> Hi, I didn't see this with my filtering.? Do you know why it fails for 
> ppcle?
>
> It fails for us on all platforms because test.vm.opts isn't set in our 
> CI jobs.
>
> Thanks,
> Coleen
>
> On 1/13/20 8:26 AM, Langer, Christoph wrote:
>> Hi David,
>>
>> thanks for the heads up. However, in our CI, these tests pass on all 
>> platforms except linuxppc64/linuxppc64le consistently.
>>
>> I think I'll push the exclusion and once JDK-8236917 has been 
>> resolved I'll try on the ppc linuxes again...
>>
>> Best regards
>> Christoph
>>
>>> -----Original Message-----
>>> From: David Holmes <david.holmes at oracle.com>
>>> Sent: Montag, 13. Januar 2020 13:57
>>> To: Langer, Christoph <christoph.langer at sap.com>; hotspot-
>>> dev at openjdk.java.net; Doerr, Martin <martin.doerr at sap.com>
>>> Cc: OpenJDK Serviceability <serviceability-dev at openjdk.java.net>
>>> Subject: Re: RFR (XS): 8237008: Exclude
>>> serviceability/sa/TestInstanceKlassSizeForInterface.java on 
>>> linuxppc64 and
>>> linuxppc64le
>>>
>>> Hi Christoph,
>>>
>>> I think those tests are currnetly failing on all platforms - see
>>> JDK-8236917. The failures after GC.class_stats removal are unrelated to
>>> the failures reported in 8230664 AFAICS.
>>>
>>> David
>>>
>>> On 13/01/2020 8:23 pm, Langer, Christoph wrote:
>>>> Hi,
>>>>
>>>> after JDK-8232759 "Remove GC.class_stats" [0] was pushed, we see the
>>>> test "serviceability/sa/TestInstanceKlassSizeForInterface.java" 
>>>> failing
>>>> on linuxppc64 and linuxppc64le the same way as
>>>> "serviceability/sa/TestInstanceKlassSize.java". Hence, I'm requesting
>>>> the same exclusion of TestInstanceKlassSizeForInterface (referring to
>>>> JDK-8230664 [1] for resolution).
>>>>
>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8237008
>>>>
>>>> Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8237008.0/
>>>>
>>>> Thanks
>>>>
>>>> Christoph
>>>>
>>>> [0] https://hg.openjdk.java.net/jdk/jdk/rev/d8f6e926cedc
>>>>
>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8230664
>>>>
>


From david.holmes at oracle.com  Mon Jan 13 21:35:56 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 14 Jan 2020 07:35:56 +1000
Subject: RFR (XS): 8237008: Exclude
 serviceability/sa/TestInstanceKlassSizeForInterface.java on linuxppc64 and
 linuxppc64le
In-Reply-To: <AM0PR02MB57148C98A930DCC8D4E73E888A350@AM0PR02MB5714.eurprd02.prod.outlook.com>
References: <AM0PR02MB5714B60661BB2222D5ECC3488A350@AM0PR02MB5714.eurprd02.prod.outlook.com>
 <8a6a9f3d-88ab-9733-7c7f-f822da148d1e@oracle.com>
 <AM0PR02MB57148C98A930DCC8D4E73E888A350@AM0PR02MB5714.eurprd02.prod.outlook.com>
Message-ID: <0d2c6e0c-9f68-c478-ebfd-69de072d0e80@oracle.com>

On 13/01/2020 11:26 pm, Langer, Christoph wrote:
> Hi David,
> 
> thanks for the heads up. However, in our CI, these tests pass on all platforms except linuxppc64/linuxppc64le consistently.

Sorry I wasn't specific enough. This test fails on all platforms in some 
configuration due to a problem with test.vm.opts/test.java.opts not 
getting passed through as expected and causing a problem when CDS is 
disabled. All in all the failure behaviour is very confusing. :(

David

> I think I'll push the exclusion and once JDK-8236917 has been resolved I'll try on the ppc linuxes again...
> 
> Best regards
> Christoph
> 
>> -----Original Message-----
>> From: David Holmes <david.holmes at oracle.com>
>> Sent: Montag, 13. Januar 2020 13:57
>> To: Langer, Christoph <christoph.langer at sap.com>; hotspot-
>> dev at openjdk.java.net; Doerr, Martin <martin.doerr at sap.com>
>> Cc: OpenJDK Serviceability <serviceability-dev at openjdk.java.net>
>> Subject: Re: RFR (XS): 8237008: Exclude
>> serviceability/sa/TestInstanceKlassSizeForInterface.java on linuxppc64 and
>> linuxppc64le
>>
>> Hi Christoph,
>>
>> I think those tests are currnetly failing on all platforms - see
>> JDK-8236917. The failures after GC.class_stats removal are unrelated to
>> the failures reported in 8230664 AFAICS.
>>
>> David
>>
>> On 13/01/2020 8:23 pm, Langer, Christoph wrote:
>>> Hi,
>>>
>>> after JDK-8232759 "Remove GC.class_stats" [0] was pushed, we see the
>>> test "serviceability/sa/TestInstanceKlassSizeForInterface.java" failing
>>> on linuxppc64 and linuxppc64le the same way as
>>> "serviceability/sa/TestInstanceKlassSize.java". Hence, I'm requesting
>>> the same exclusion of TestInstanceKlassSizeForInterface (referring to
>>> JDK-8230664 [1] for resolution).
>>>
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8237008
>>>
>>> Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8237008.0/
>>>
>>> Thanks
>>>
>>> Christoph
>>>
>>> [0] https://hg.openjdk.java.net/jdk/jdk/rev/d8f6e926cedc
>>>
>>> [1] https://bugs.openjdk.java.net/browse/JDK-8230664
>>>

From david.holmes at oracle.com  Mon Jan 13 22:31:12 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 14 Jan 2020 08:31:12 +1000
Subject: RFR (M): 8235741: Inappropriate uses of os::javaTimeMillis()
In-Reply-To: <75f4cd8d-d291-2cf5-b0dd-91a1d76a6130@oracle.com>
References: <f99194ad-3dad-48db-d8d4-de0cc9161701@oracle.com>
 <75f4cd8d-d291-2cf5-b0dd-91a1d76a6130@oracle.com>
Message-ID: <9e2804c6-3441-2e0a-5c38-7e5ecc96ecae@oracle.com>

Hi Robbin,

Thanks for looking at this.

I'm going to wait to get more eyes on this as it does cover various 
parts of the VM.

David

On 13/01/2020 11:08 pm, Robbin Ehn wrote:
> Hi David, looks good, thanks!
> 
> /Robbin
> 
> On 1/13/20 8:13 AM, David Holmes wrote:
>> webrev: http://cr.openjdk.java.net/~dholmes/8235741/webrev/
>> bug: https://bugs.openjdk.java.net/browse/JDK-8235741
>>
>> Full details in the bug report about the existing uses of 
>> javaTimeMillis(), many of which just want an elapsed time in ms and so 
>> should be using javaTimeNanos() and convert to ms. This covers areas 
>> all across the VM.
>>
>> Only non-simple change is in os_perf_linux.cpp (and the same code will 
>> be in os_perf_aix.cpp once it has been validated). There we are 
>> tracking an elapsed time in ms but relative to the boot time, which is 
>> seconds since the epoch. Consequently the first interval has to be 
>> calculated using javaTimeMillis, but after that we can use 
>> javaTimeNanos (using a new 'first time' captured at the same time we 
>> used javaTimeMillis). I think I have the logic right but other than 
>> through JFR this code seems unused and I have limited means of testing 
>> it. The JFR test jdk/jfr/event/os/TestThreadContextSwitches.java 
>> exercises the code but the results of running that test seems to 
>> exhibit arbitrary randomness in the rates reported - e.g. 0 to 16000Hz 
>> - both with and without my change, so not really that useful. Stefan 
>> K. suggested a gtest which I may look into - though it is frustrating 
>> to have to expend such effort to validate this.
>>
>> Other testing tiers 1-3.
>>
>> Thanks,
>> David

From bwmat.reloaded at gmail.com  Mon Jan 13 23:56:28 2020
From: bwmat.reloaded at gmail.com (Bwmat .)
Date: Mon, 13 Jan 2020 15:56:28 -0800
Subject: Need some help in debugging jvm corruption/crashes in C++/JNI code
In-Reply-To: <MWHPR11MB1471FDB6688D30DB07E215CEC2350@MWHPR11MB1471.namprd11.prod.outlook.com>
References: <MWHPR11MB1471EF77810DF9DD84A39305C2350@MWHPR11MB1471.namprd11.prod.outlook.com>
 <MWHPR11MB1471FDB6688D30DB07E215CEC2350@MWHPR11MB1471.namprd11.prod.outlook.com>
Message-ID: <CANXFMXEcPHe70ERzPCVwmRsJaGfuZYy595R=C7S1H2XrH-YGrA@mail.gmail.com>

Hello,


I just joined this mailing list to hopefully get a bit of help from someone
familiar with hotspot.



We found a crash in a C++ library we have that uses JNI, and I?ve been
trying to debug it for a couple of days. I?m working on windows, and I
managed to get a TTD (time travel debugging, a feature in the preview
version of WinDBG) trace of when the problem occurs, and I built myself a
debug JVM from source to help debug, but I?m completely unfamiliar with
hotspot.



I?ve also attached a log file created one time that the crash occurred (the
symptoms aren?t always the same, sometimes it doesn?t crash at all, but
just return an invalid value from a method call, so my test app quits
early, sometimes it crashes from an assertion from within hotpot, not
always the same one).



One weird thing is that I found the issue while doing some testing, and a
certain SQL query triggers the issue, but another, analogous query does
not. This is significant, since all of the SQL processing is written in
Java, so very little changes in the native part of the application between
the case that works and the case that fails, which makes me think it?s a
jvm issue. I?m currently debugging a java 8 openjdk (since I used an
article about how to build it that used that version, and that?s what we?re
using internally anyways, but I also reproduced the issue on a java 13
oracle jvm, so if it IS a JVM bug, doesn?t seem like it's been fixed yet.
I?m quite aware that it?s probably still our fault somehow though.)



The issue seems to somehow be caused by the wrong java method being invoked
by a JNI call, or maybe the right method, but on the wrong object.



Early on, while debugging using Eclipse?s remote debugger, I found that,
right before crashing, I was able to hit a breakpoint in a method on a type
T, but the debugger told me that the ?this? reference was actually of type
String!



Later on, while debugging the internals of hotspot in my TTD trace (which
allowed me to get quite far without much understanding of what?s going
one), I found that the crash in the trace occurs when the wrong method is
invoked on an object, returning a long, which is later interpreted as a
string reference, and the jvm notices it isn?t actually a string when the
native code tries to get the string length.



To lay out the situation without going into too many details of our code,
we have an interface IColumn, that has a few methods that get attributes
about a SQL column. Some methods return primitives, some return strings.
The native code is trying to call IColumn.GetLabel(), so it passes a
jmethodID that was generated via a successful call to JNI?s GetMethodID,
passing the class object of IColumn. For the receiver of the call, it
passes an instance of a type which implements IColumn, and has no
superclasses, but is a private static inner class.



In jni_invoke_nonstatic(), it goes into the ?else if
(!m->has_itable_index())? branch, which seems wrong from what little I
gleamed from https://wiki.openjdk.java.net/display/HotSpot/InterfaceCalls

It then resolves the wrong method, getDisplaySize(), which returns a long
(I note that getDisplaySize() is the method declared directly before
getLabel() in the declaration of IColumn, so maybe an off-by-one error
somewhere?), and then invokes it, dooming the process to a future crash.



If I go backwards in time to when the _*vtable*_index field of the Method
(the one originally resolved using GetMethodID, and later passed into
CallObjectMethodV) is set, it?s in KlassVtable::initialize_vtable(), on the
line ?mh()->set_vtable_index(initialized); // set primary vtable index?,
and that?s the last time it?s set.

Am I right in thinking that it should have instead been set to a negative
value (so that it was an ?itable index?), since it?s an interface method?
If so, where would that happen? What else should I check?



Any suggestions are welcome, thanks in advance.

From david.holmes at oracle.com  Tue Jan 14 02:22:15 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 14 Jan 2020 12:22:15 +1000
Subject: Need some help in debugging jvm corruption/crashes in C++/JNI code
In-Reply-To: <CANXFMXEcPHe70ERzPCVwmRsJaGfuZYy595R=C7S1H2XrH-YGrA@mail.gmail.com>
References: <MWHPR11MB1471EF77810DF9DD84A39305C2350@MWHPR11MB1471.namprd11.prod.outlook.com>
 <MWHPR11MB1471FDB6688D30DB07E215CEC2350@MWHPR11MB1471.namprd11.prod.outlook.com>
 <CANXFMXEcPHe70ERzPCVwmRsJaGfuZYy595R=C7S1H2XrH-YGrA@mail.gmail.com>
Message-ID: <83b068d7-464c-56ac-e0e0-56db59c31648@oracle.com>

Hi,

On 14/01/2020 9:56 am, Bwmat . wrote:
> Hello,
> 
> 
> I just joined this mailing list to hopefully get a bit of help from someone
> familiar with hotspot.

These mailing lists are not really for end user application debugging 
help. However, as you have deep dived into hotspot internals ... :)

In a debug build you can use itable/vtable logging to see how the 
itable/vtable is constructed and see if anything odd appears there.

-Xlog:itables*=trace,vtables*=trace

You can also try running with -Xcheck:jni.

FYI most file attachments are stripped by the mailing list so your log 
did not get included.

Cheers,
David

> 
> 
> We found a crash in a C++ library we have that uses JNI, and I?ve been
> trying to debug it for a couple of days. I?m working on windows, and I
> managed to get a TTD (time travel debugging, a feature in the preview
> version of WinDBG) trace of when the problem occurs, and I built myself a
> debug JVM from source to help debug, but I?m completely unfamiliar with
> hotspot.
> 
> 
> 
> I?ve also attached a log file created one time that the crash occurred (the
> symptoms aren?t always the same, sometimes it doesn?t crash at all, but
> just return an invalid value from a method call, so my test app quits
> early, sometimes it crashes from an assertion from within hotpot, not
> always the same one).
> 
> 
> 
> One weird thing is that I found the issue while doing some testing, and a
> certain SQL query triggers the issue, but another, analogous query does
> not. This is significant, since all of the SQL processing is written in
> Java, so very little changes in the native part of the application between
> the case that works and the case that fails, which makes me think it?s a
> jvm issue. I?m currently debugging a java 8 openjdk (since I used an
> article about how to build it that used that version, and that?s what we?re
> using internally anyways, but I also reproduced the issue on a java 13
> oracle jvm, so if it IS a JVM bug, doesn?t seem like it's been fixed yet.
> I?m quite aware that it?s probably still our fault somehow though.)
> 
> 
> 
> The issue seems to somehow be caused by the wrong java method being invoked
> by a JNI call, or maybe the right method, but on the wrong object.
> 
> 
> 
> Early on, while debugging using Eclipse?s remote debugger, I found that,
> right before crashing, I was able to hit a breakpoint in a method on a type
> T, but the debugger told me that the ?this? reference was actually of type
> String!
> 
> 
> 
> Later on, while debugging the internals of hotspot in my TTD trace (which
> allowed me to get quite far without much understanding of what?s going
> one), I found that the crash in the trace occurs when the wrong method is
> invoked on an object, returning a long, which is later interpreted as a
> string reference, and the jvm notices it isn?t actually a string when the
> native code tries to get the string length.
> 
> 
> 
> To lay out the situation without going into too many details of our code,
> we have an interface IColumn, that has a few methods that get attributes
> about a SQL column. Some methods return primitives, some return strings.
> The native code is trying to call IColumn.GetLabel(), so it passes a
> jmethodID that was generated via a successful call to JNI?s GetMethodID,
> passing the class object of IColumn. For the receiver of the call, it
> passes an instance of a type which implements IColumn, and has no
> superclasses, but is a private static inner class.
> 
> 
> 
> In jni_invoke_nonstatic(), it goes into the ?else if
> (!m->has_itable_index())? branch, which seems wrong from what little I
> gleamed from https://wiki.openjdk.java.net/display/HotSpot/InterfaceCalls
> 
> It then resolves the wrong method, getDisplaySize(), which returns a long
> (I note that getDisplaySize() is the method declared directly before
> getLabel() in the declaration of IColumn, so maybe an off-by-one error
> somewhere?), and then invokes it, dooming the process to a future crash.
> 
> 
> 
> If I go backwards in time to when the _*vtable*_index field of the Method
> (the one originally resolved using GetMethodID, and later passed into
> CallObjectMethodV) is set, it?s in KlassVtable::initialize_vtable(), on the
> line ?mh()->set_vtable_index(initialized); // set primary vtable index?,
> and that?s the last time it?s set.
> 
> Am I right in thinking that it should have instead been set to a negative
> value (so that it was an ?itable index?), since it?s an interface method?
> If so, where would that happen? What else should I check?
> 
> 
> 
> Any suggestions are welcome, thanks in advance.
> 

From bwmat.reloaded at gmail.com  Tue Jan 14 02:35:43 2020
From: bwmat.reloaded at gmail.com (Bwmat .)
Date: Mon, 13 Jan 2020 18:35:43 -0800
Subject: Need some help in debugging jvm corruption/crashes in C++/JNI code
In-Reply-To: <83b068d7-464c-56ac-e0e0-56db59c31648@oracle.com>
References: <MWHPR11MB1471EF77810DF9DD84A39305C2350@MWHPR11MB1471.namprd11.prod.outlook.com>
 <MWHPR11MB1471FDB6688D30DB07E215CEC2350@MWHPR11MB1471.namprd11.prod.outlook.com>
 <CANXFMXEcPHe70ERzPCVwmRsJaGfuZYy595R=C7S1H2XrH-YGrA@mail.gmail.com>
 <83b068d7-464c-56ac-e0e0-56db59c31648@oracle.com>
Message-ID: <CANXFMXHYfDOE=UeCsFBXg6vqfCPw4GTJY61XR+N-pBK7QmQ=UA@mail.gmail.com>

Thanks for the response, but I JUST figured it out.

It was our fault, of course. I was staring at the trace, wondering why it
wasn't doing an interface call, when I realized that what I had described
as IColumn wasn't actually an interface, it was a class! I had assumed we
had properly written the compatibility layer to use the interface, but the
*real* Interface is actually rarely used, with the aforementioned class
being used in the vast majority of cases... (and I haven't used this code
much for a couple of years). Not here though! Really kicking myself for not
realizing this sooner.

Kinda wish that -Xcheck:jni checked this (or the debug assertions in
hotspot, though maybe they do in a newer version). Has this been
considered, anyone know?

Thanks again,
Matthew w.


On Mon., Jan. 13, 2020, 6:22 p.m. David Holmes, <david.holmes at oracle.com>
wrote:

> Hi,
>
> On 14/01/2020 9:56 am, Bwmat . wrote:
> > Hello,
> >
> >
> > I just joined this mailing list to hopefully get a bit of help from
> someone
> > familiar with hotspot.
>
> These mailing lists are not really for end user application debugging
> help. However, as you have deep dived into hotspot internals ... :)
>
> In a debug build you can use itable/vtable logging to see how the
> itable/vtable is constructed and see if anything odd appears there.
>
> -Xlog:itables*=trace,vtables*=trace
>
> You can also try running with -Xcheck:jni.
>
> FYI most file attachments are stripped by the mailing list so your log
> did not get included.
>
> Cheers,
> David
>
> >
> >
> > We found a crash in a C++ library we have that uses JNI, and I?ve been
> > trying to debug it for a couple of days. I?m working on windows, and I
> > managed to get a TTD (time travel debugging, a feature in the preview
> > version of WinDBG) trace of when the problem occurs, and I built myself a
> > debug JVM from source to help debug, but I?m completely unfamiliar with
> > hotspot.
> >
> >
> >
> > I?ve also attached a log file created one time that the crash occurred
> (the
> > symptoms aren?t always the same, sometimes it doesn?t crash at all, but
> > just return an invalid value from a method call, so my test app quits
> > early, sometimes it crashes from an assertion from within hotpot, not
> > always the same one).
> >
> >
> >
> > One weird thing is that I found the issue while doing some testing, and a
> > certain SQL query triggers the issue, but another, analogous query does
> > not. This is significant, since all of the SQL processing is written in
> > Java, so very little changes in the native part of the application
> between
> > the case that works and the case that fails, which makes me think it?s a
> > jvm issue. I?m currently debugging a java 8 openjdk (since I used an
> > article about how to build it that used that version, and that?s what
> we?re
> > using internally anyways, but I also reproduced the issue on a java 13
> > oracle jvm, so if it IS a JVM bug, doesn?t seem like it's been fixed yet.
> > I?m quite aware that it?s probably still our fault somehow though.)
> >
> >
> >
> > The issue seems to somehow be caused by the wrong java method being
> invoked
> > by a JNI call, or maybe the right method, but on the wrong object.
> >
> >
> >
> > Early on, while debugging using Eclipse?s remote debugger, I found that,
> > right before crashing, I was able to hit a breakpoint in a method on a
> type
> > T, but the debugger told me that the ?this? reference was actually of
> type
> > String!
> >
> >
> >
> > Later on, while debugging the internals of hotspot in my TTD trace (which
> > allowed me to get quite far without much understanding of what?s going
> > one), I found that the crash in the trace occurs when the wrong method is
> > invoked on an object, returning a long, which is later interpreted as a
> > string reference, and the jvm notices it isn?t actually a string when the
> > native code tries to get the string length.
> >
> >
> >
> > To lay out the situation without going into too many details of our code,
> > we have an interface IColumn, that has a few methods that get attributes
> > about a SQL column. Some methods return primitives, some return strings.
> > The native code is trying to call IColumn.GetLabel(), so it passes a
> > jmethodID that was generated via a successful call to JNI?s GetMethodID,
> > passing the class object of IColumn. For the receiver of the call, it
> > passes an instance of a type which implements IColumn, and has no
> > superclasses, but is a private static inner class.
> >
> >
> >
> > In jni_invoke_nonstatic(), it goes into the ?else if
> > (!m->has_itable_index())? branch, which seems wrong from what little I
> > gleamed from
> https://wiki.openjdk.java.net/display/HotSpot/InterfaceCalls
> >
> > It then resolves the wrong method, getDisplaySize(), which returns a long
> > (I note that getDisplaySize() is the method declared directly before
> > getLabel() in the declaration of IColumn, so maybe an off-by-one error
> > somewhere?), and then invokes it, dooming the process to a future crash.
> >
> >
> >
> > If I go backwards in time to when the _*vtable*_index field of the Method
> > (the one originally resolved using GetMethodID, and later passed into
> > CallObjectMethodV) is set, it?s in KlassVtable::initialize_vtable(), on
> the
> > line ?mh()->set_vtable_index(initialized); // set primary vtable index?,
> > and that?s the last time it?s set.
> >
> > Am I right in thinking that it should have instead been set to a negative
> > value (so that it was an ?itable index?), since it?s an interface method?
> > If so, where would that happen? What else should I check?
> >
> >
> >
> > Any suggestions are welcome, thanks in advance.
> >
>

From david.holmes at oracle.com  Tue Jan 14 04:21:33 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 14 Jan 2020 14:21:33 +1000
Subject: Need some help in debugging jvm corruption/crashes in C++/JNI code
In-Reply-To: <CANXFMXHYfDOE=UeCsFBXg6vqfCPw4GTJY61XR+N-pBK7QmQ=UA@mail.gmail.com>
References: <MWHPR11MB1471EF77810DF9DD84A39305C2350@MWHPR11MB1471.namprd11.prod.outlook.com>
 <MWHPR11MB1471FDB6688D30DB07E215CEC2350@MWHPR11MB1471.namprd11.prod.outlook.com>
 <CANXFMXEcPHe70ERzPCVwmRsJaGfuZYy595R=C7S1H2XrH-YGrA@mail.gmail.com>
 <83b068d7-464c-56ac-e0e0-56db59c31648@oracle.com>
 <CANXFMXHYfDOE=UeCsFBXg6vqfCPw4GTJY61XR+N-pBK7QmQ=UA@mail.gmail.com>
Message-ID: <1f77a5f2-9056-d14e-ca95-3d4c68ce5397@oracle.com>

On 14/01/2020 12:35 pm, Bwmat . wrote:
> Thanks for the response, but I JUST figured it out.
> 
> It was our fault, of course. I was staring at the trace, wondering why 
> it wasn't doing an interface call, when I realized that what I had 
> described as IColumn wasn't actually an interface, it was a class! I had 
> assumed we had properly written the compatibility layer to use the 
> interface, but the *real* Interface is actually rarely used, with the 
> aforementioned class being used in the vast majority of cases... (and I 
> haven't used this code much for a couple of years). Not here though! 
> Really kicking myself for not realizing this sooner.
> 
> Kinda wish that -Xcheck:jni checked this (or the debug assertions in 
> hotspot, though maybe they do in a newer version). Has this been 
> considered, anyone know?

Funny you should mention that:

https://bugs.openjdk.java.net/browse/JDK-8229900

a change in JDK 14, should provide the receiver check you were looking for.

Cheers,
David

> Thanks again,
> Matthew w.
> 
> 
> On Mon., Jan. 13, 2020, 6:22 p.m. David Holmes, <david.holmes at oracle.com 
> <mailto:david.holmes at oracle.com>> wrote:
> 
>     Hi,
> 
>     On 14/01/2020 9:56 am, Bwmat . wrote:
>      > Hello,
>      >
>      >
>      > I just joined this mailing list to hopefully get a bit of help
>     from someone
>      > familiar with hotspot.
> 
>     These mailing lists are not really for end user application debugging
>     help. However, as you have deep dived into hotspot internals ... :)
> 
>     In a debug build you can use itable/vtable logging to see how the
>     itable/vtable is constructed and see if anything odd appears there.
> 
>     -Xlog:itables*=trace,vtables*=trace
> 
>     You can also try running with -Xcheck:jni.
> 
>     FYI most file attachments are stripped by the mailing list so your log
>     did not get included.
> 
>     Cheers,
>     David
> 
>      >
>      >
>      > We found a crash in a C++ library we have that uses JNI, and I?ve
>     been
>      > trying to debug it for a couple of days. I?m working on windows,
>     and I
>      > managed to get a TTD (time travel debugging, a feature in the preview
>      > version of WinDBG) trace of when the problem occurs, and I built
>     myself a
>      > debug JVM from source to help debug, but I?m completely
>     unfamiliar with
>      > hotspot.
>      >
>      >
>      >
>      > I?ve also attached a log file created one time that the crash
>     occurred (the
>      > symptoms aren?t always the same, sometimes it doesn?t crash at
>     all, but
>      > just return an invalid value from a method call, so my test app quits
>      > early, sometimes it crashes from an assertion from within hotpot, not
>      > always the same one).
>      >
>      >
>      >
>      > One weird thing is that I found the issue while doing some
>     testing, and a
>      > certain SQL query triggers the issue, but another, analogous
>     query does
>      > not. This is significant, since all of the SQL processing is
>     written in
>      > Java, so very little changes in the native part of the
>     application between
>      > the case that works and the case that fails, which makes me think
>     it?s a
>      > jvm issue. I?m currently debugging a java 8 openjdk (since I used an
>      > article about how to build it that used that version, and that?s
>     what we?re
>      > using internally anyways, but I also reproduced the issue on a
>     java 13
>      > oracle jvm, so if it IS a JVM bug, doesn?t seem like it's been
>     fixed yet.
>      > I?m quite aware that it?s probably still our fault somehow though.)
>      >
>      >
>      >
>      > The issue seems to somehow be caused by the wrong java method
>     being invoked
>      > by a JNI call, or maybe the right method, but on the wrong object.
>      >
>      >
>      >
>      > Early on, while debugging using Eclipse?s remote debugger, I
>     found that,
>      > right before crashing, I was able to hit a breakpoint in a method
>     on a type
>      > T, but the debugger told me that the ?this? reference was
>     actually of type
>      > String!
>      >
>      >
>      >
>      > Later on, while debugging the internals of hotspot in my TTD
>     trace (which
>      > allowed me to get quite far without much understanding of what?s
>     going
>      > one), I found that the crash in the trace occurs when the wrong
>     method is
>      > invoked on an object, returning a long, which is later
>     interpreted as a
>      > string reference, and the jvm notices it isn?t actually a string
>     when the
>      > native code tries to get the string length.
>      >
>      >
>      >
>      > To lay out the situation without going into too many details of
>     our code,
>      > we have an interface IColumn, that has a few methods that get
>     attributes
>      > about a SQL column. Some methods return primitives, some return
>     strings.
>      > The native code is trying to call IColumn.GetLabel(), so it passes a
>      > jmethodID that was generated via a successful call to JNI?s
>     GetMethodID,
>      > passing the class object of IColumn. For the receiver of the call, it
>      > passes an instance of a type which implements IColumn, and has no
>      > superclasses, but is a private static inner class.
>      >
>      >
>      >
>      > In jni_invoke_nonstatic(), it goes into the ?else if
>      > (!m->has_itable_index())? branch, which seems wrong from what
>     little I
>      > gleamed from
>     https://wiki.openjdk.java.net/display/HotSpot/InterfaceCalls
>      >
>      > It then resolves the wrong method, getDisplaySize(), which
>     returns a long
>      > (I note that getDisplaySize() is the method declared directly before
>      > getLabel() in the declaration of IColumn, so maybe an off-by-one
>     error
>      > somewhere?), and then invokes it, dooming the process to a future
>     crash.
>      >
>      >
>      >
>      > If I go backwards in time to when the _*vtable*_index field of
>     the Method
>      > (the one originally resolved using GetMethodID, and later passed into
>      > CallObjectMethodV) is set, it?s in
>     KlassVtable::initialize_vtable(), on the
>      > line ?mh()->set_vtable_index(initialized); // set primary vtable
>     index?,
>      > and that?s the last time it?s set.
>      >
>      > Am I right in thinking that it should have instead been set to a
>     negative
>      > value (so that it was an ?itable index?), since it?s an interface
>     method?
>      > If so, where would that happen? What else should I check?
>      >
>      >
>      >
>      > Any suggestions are welcome, thanks in advance.
>      >
> 

From magnus.ihse.bursie at oracle.com  Tue Jan 14 11:27:52 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Tue, 14 Jan 2020 12:27:52 +0100
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>

On 2020-01-10 11:01, Baesken, Matthias wrote:
> Hello,   I recently looked into  the  gcc  lto  optimization mode (see for some details https://gcc.gnu.org/onlinedocs/gccint/LTO-Overview.html  and  http://hubicka.blogspot.com/2019/05/gcc-9-link-time-and-inter-procedural.html  ).
> This mode can lead to more compact binaries (~10% smaller)  , it also might bring  small performance improvements  but that wasn't my (main)  goal  .
>
> The changes for this are rather small , one needs to use a recent gcc  , add  -flto   to the compile flags  , for example
>
> --- a/make/autoconf/flags-cflags.m4      Wed Jan 01 03:08:45 2020 +0100
> +++ b/make/autoconf/flags-cflags.m4   Wed Jan 08 17:39:10 2020 +0100
> @@ -530,8 +530,13 @@
>     fi
>     if test "x$TOOLCHAIN_TYPE" = xgcc; then
> -    TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new -fstack-protector"
> -    TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector"
> +    TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new -fstack-protector -flto"
> +    TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector -flto"
>
>    .... and you have to make sure  to use  gcc-ar  and  gcc-nm instead   of  ar / nm .
> Build and test(s)  work,  however with  one exception.
> The  serviceability   tests like  serviceability/sa   seems to rely   heavily  on the "normal"   structure  of   libjvm.so   (from what I   understand  e.g. in  LinuxVtblAccess  it is attempted to access  internal symbols  like  _ZTV ).
>
> Errors in the sa  tests look like :
>
>
> java.lang.InternalError: Metadata does not appear to be polymorphic
>           at jdk.hotspot.agent/sun.jvm.hotspot.types.basic.BasicTypeDataBase.findDynamicTypeForAddress(BasicTypeDataBase.java:279)
>           at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VirtualBaseConstructor.instantiateWrapperFor(VirtualBaseConstructor.java:102)
>           at jdk.hotspot.agent/sun.jvm.hotspot.oops.Metadata.instantiateWrapperFor(Metadata.java:74)
>           at jdk.hotspot.agent/sun.jvm.hotspot.memory.SystemDictionary.getClassLoaderKlass(SystemDictionary.java:96)
>           at jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.printClassLoaderStatistics(ClassLoaderStats.java:93)
>           at jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.run(ClassLoaderStats.java:78)
>           at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.run(JMap.java:115)
>           at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:262)
>           at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:225)
>           at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
>           at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.main(JMap.java:176)
>           at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJMAP(SALauncher.java:321)
>           at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:406)
>
> Has anyone experimented with LTO optimization ?

Hi Matthias,

We used to have LTO enabled on the old, closed-source Oracle arm-32 
builds. There is still a "link-time-opt" JVM feature present; afaik it 
still works and adds the -flto flag. The main drawback of this is the 
*extremely* long link times of libjvm.so.

I don't think servicability was ever supported for that platform, so I'm 
not surprised this does not work.

/Magnus

>
> And to the  serviceability   agent experts -  any idea  how to make the  jdk.hotspot.agent   more independent from  optimization settings ?
>
>
> Best regards, Matthias


From matthias.baesken at sap.com  Tue Jan 14 12:49:43 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 14 Jan 2020 12:49:43 +0000
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
Message-ID: <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hi Magnus, thanks for the info , I already noticed yesterday the setting for arm-32 in the minimal build .
Do you think  we could set it too for the other Linux platforms  in the minimal build  ( serviceability agent is not supported there as well so the  observed issue wouldn?t be a problem).

Best regards, Matthias




On 2020-01-10 11:01, Baesken, Matthias wrote:

Hello,   I recently looked into  the  gcc  lto  optimization mode (see for some details https://gcc.gnu.org/onlinedocs/gccint/LTO-Overview.html  and  http://hubicka.blogspot.com/2019/05/gcc-9-link-time-and-inter-procedural.html  ).

This mode can lead to more compact binaries (~10% smaller)  , it also might bring  small performance improvements  but that wasn't my (main)  goal  .



The changes for this are rather small , one needs to use a recent gcc  , add  -flto   to the compile flags  , for example



--- a/make/autoconf/flags-cflags.m4      Wed Jan 01 03:08:45 2020 +0100

+++ b/make/autoconf/flags-cflags.m4   Wed Jan 08 17:39:10 2020 +0100

@@ -530,8 +530,13 @@

   fi

   if test "x$TOOLCHAIN_TYPE" = xgcc; then

-    TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new -fstack-protector"

-    TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector"

+    TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new -fstack-protector -flto"

+    TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector -flto"



  .... and you have to make sure  to use  gcc-ar  and  gcc-nm instead   of  ar / nm .

Build and test(s)  work,  however with  one exception.

The  serviceability   tests like  serviceability/sa   seems to rely   heavily  on the "normal"   structure  of   libjvm.so   (from what I   understand  e.g. in  LinuxVtblAccess  it is attempted to access  internal symbols  like  _ZTV ).



Errors in the sa  tests look like :





java.lang.InternalError: Metadata does not appear to be polymorphic

         at jdk.hotspot.agent/sun.jvm.hotspot.types.basic.BasicTypeDataBase.findDynamicTypeForAddress(BasicTypeDataBase.java:279)

         at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VirtualBaseConstructor.instantiateWrapperFor(VirtualBaseConstructor.java:102)

         at jdk.hotspot.agent/sun.jvm.hotspot.oops.Metadata.instantiateWrapperFor(Metadata.java:74)

         at jdk.hotspot.agent/sun.jvm.hotspot.memory.SystemDictionary.getClassLoaderKlass(SystemDictionary.java:96)

         at jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.printClassLoaderStatistics(ClassLoaderStats.java:93)

         at jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.run(ClassLoaderStats.java:78)

         at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.run(JMap.java:115)

         at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:262)

         at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:225)

         at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)

         at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.main(JMap.java:176)

         at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJMAP(SALauncher.java:321)

         at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:406)



Has anyone experimented with LTO optimization ?

Hi Matthias,

We used to have LTO enabled on the old, closed-source Oracle arm-32 builds. There is still a "link-time-opt" JVM feature present; afaik it still works and adds the -flto flag. The main drawback of this is the *extremely* long link times of libjvm.so.

I don't think servicability was ever supported for that platform, so I'm not surprised this does not work.

/Magnus







And to the  serviceability   agent experts -  any idea  how to make the  jdk.hotspot.agent   more independent from  optimization settings ?





Best regards, Matthias


From harold.seigel at oracle.com  Tue Jan 14 14:00:13 2020
From: harold.seigel at oracle.com (Harold Seigel)
Date: Tue, 14 Jan 2020 09:00:13 -0500
Subject: RFR 8235678: Remove unnecessary calls to Thread::current() in
 MutexLocker calls
Message-ID: <0a7cbbe4-3613-bd5a-962b-78dee06dde8a@oracle.com>

Hi,

Please review this small change, to reduce unnecessary calls to 
Thread::current() in MutexLocker calls, by passing the current thread as 
an argument.? A few ResoureMark declarations were also changed.

Open Webrev: 
http://cr.openjdk.java.net/~hseigel/bug_8235678/webrev/index.html

JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8235678

The fix was regression tested by running Mach5 tiers 1 and 2 tests and 
builds on Linux-x64, Solaris, Windows, and Mac OS X, by running Mach5 
tiers 3-5 tests on Linux-x64, and JCK lang and VM tests on Linux-x64.

Thanks, Harold


From magnus.ihse.bursie at oracle.com  Tue Jan 14 14:04:11 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Tue, 14 Jan 2020 15:04:11 +0100
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
 <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>

On 2020-01-14 13:49, Baesken, Matthias wrote:
>
> Hi Magnus, thanks for the info , I already noticed yesterday the 
> setting for arm-32 in the minimal build .
>
> Do you think? we could set it too for the other Linux platforms? in 
> the minimal build? ( serviceability agent is not supported there as 
> well so the? observed issue wouldn?t be a problem).
>

You mean if you could enable it on your builds without any issues? I'd 
guess so, but I don't know. Just try it: 
--with-jvm-features="link-time-opt".

If you mean that it should be turned on by default on minimal builds for 
all platforms? No, I don't think that's a good idea. The link time is 
really a killer. I remember arm-32 going from like a couple of minutes 
to half an hour for linking libjvm.so.

Things might be different with gold, though. I know they have done work 
with at least some kind of "lightweight" LTO, that might be worth at 
least looking into.

/Magnus


> Best regards, Matthias
>
> On 2020-01-10 11:01, Baesken, Matthias wrote:
>
>     Hello,?? I recently looked into? the? gcc? lto? optimization mode (see for some detailshttps://gcc.gnu.org/onlinedocs/gccint/LTO-Overview.html ? andhttp://hubicka.blogspot.com/2019/05/gcc-9-link-time-and-inter-procedural.html ? ).
>
>     This mode can lead to more compact binaries (~10% smaller)? , it also might bring? small performance improvements? but that wasn't my (main)? goal? .
>
>     The changes for this are rather small , one needs to use a recent gcc? , add? -flto?? to the compile flags? , for example
>
>     --- a/make/autoconf/flags-cflags.m4????? Wed Jan 01 03:08:45 2020 +0100
>
>     +++ b/make/autoconf/flags-cflags.m4?? Wed Jan 08 17:39:10 2020 +0100
>
>     @@ -530,8 +530,13 @@
>
>      ?? fi
>
>      ?? if test "x$TOOLCHAIN_TYPE" = xgcc; then
>
>     -??? TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new -fstack-protector"
>
>     -??? TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector"
>
>     +??? TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new -fstack-protector -flto"
>
>     +??? TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector -flto"
>
>      ? .... and you have to make sure? to use? gcc-ar? and? gcc-nm instead?? of? ar / nm .
>
>     Build and test(s)? work,? however with? one exception.
>
>     The? serviceability?? tests like? serviceability/sa?? seems to rely?? heavily? on the "normal"?? structure? of?? libjvm.so?? (from what I?? understand? e.g. in? LinuxVtblAccess? it is attempted to access? internal symbols? like? _ZTV ).
>
>     Errors in the sa? tests look like :
>
>     java.lang.InternalError: Metadata does not appear to be polymorphic
>
>      ???????? at jdk.hotspot.agent/sun.jvm.hotspot.types.basic.BasicTypeDataBase.findDynamicTypeForAddress(BasicTypeDataBase.java:279)
>
>      ???????? at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VirtualBaseConstructor.instantiateWrapperFor(VirtualBaseConstructor.java:102)
>
>      ???? ????at jdk.hotspot.agent/sun.jvm.hotspot.oops.Metadata.instantiateWrapperFor(Metadata.java:74)
>
>      ???????? at jdk.hotspot.agent/sun.jvm.hotspot.memory.SystemDictionary.getClassLoaderKlass(SystemDictionary.java:96)
>
>      ???????? at jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.printClassLoaderStatistics(ClassLoaderStats.java:93)
>
>      ???????? at jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.run(ClassLoaderStats.java:78)
>
>      ???????? at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.run(JMap.java:115)
>
>      ??? ?????at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:262)
>
>      ???????? at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:225)
>
>      ???????? at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
>
>      ???????? at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.main(JMap.java:176)
>
>      ???????? at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJMAP(SALauncher.java:321)
>
>      ???????? at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:406)
>
>     Has anyone experimented with LTO optimization ?
>
>
> Hi Matthias,
>
> We used to have LTO enabled on the old, closed-source Oracle arm-32 
> builds. There is still a "link-time-opt" JVM feature present; afaik it 
> still works and adds the -flto flag. The main drawback of this is the 
> *extremely* long link times of libjvm.so.
>
> I don't think servicability was ever supported for that platform, so 
> I'm not surprised this does not work.
>
> /Magnus
>
>
>     And to the? serviceability?? agent experts -? any idea? how to make the? jdk.hotspot.agent?? more independent from? optimization settings ?
>
>     Best regards, Matthias
>


From matthias.baesken at sap.com  Tue Jan 14 14:07:16 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 14 Jan 2020 14:07:16 +0000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove unused
 code
Message-ID: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello,  the following change enables the  link-time section-gc for linux .

gcc and ld support enabling "garbage collection" of unused input sections.
This can be used to eliminate unused coding from native libraries (especially when already compiling the objects with compiler flags -ffunction-sections -fdata-sections .
See for details the --gc-sections and --print-gc-sections parts of the ld documentation :

https://linux.die.net/man/1/ld


We had this enabled already  for  linux s390x ,  with https://bugs.openjdk.java.net/browse/JDK-8234525
8234525: enable link-time section-gc for linux s390x to remove unused code .

This  time we enable it too for  the other linux platforms .

For the other platforms I do not enable it for JVM, just for the JDK libs.  The reason is that the serviceability agent  (not supported on linux s390x )    is not  (yet) ready for the optimization .
Below you see the results , for some libraries  a significant  size reduction can be achieved .


Results from linux x86_64 product builds :

without / with ltgc

320K / 300K    /images/jdk/lib/libsunec.so       <-------------------------
36K  / 36K     /images/jdk/lib/libdt_socket.so
280K / 276K   /images/jdk/lib/libjdwp.so
23M  / 23M    /images/jdk/lib/server/libjvm.so    <---- not set for libjvm.so for x86_64
16K  / 16K    /images/jdk/lib/server/libjsig.so
72K  / 72M    /images/jdk/lib/libverify.so
84K  / 84M   /images/jdk/lib/libjli.so
16K  / 16K    /images/jdk/lib/libjsig.so
196K / 196K   /images/jdk/lib/libjava.so
44K  / 44K    /images/jdk/lib/libzip.so
144K / 136K   /images/jdk/lib/libjimage.so
112K / 112K   /images/jdk/lib/libnet.so
100K / 100K   /images/jdk/lib/libnio.so
36K  / 36K    /images/jdk/lib/libsctp.so
576K / 556K   /images/jdk/lib/libmlib_image.so
752K / 752K   /images/jdk/lib/libawt.so
260K / 252K   /images/jdk/lib/libjavajpeg.so
784K / 784K   /images/jdk/lib/libfreetype.so
368K / 236K /images/jdk/lib/libsplashscreen.so   <-------------------------
88K / 88K    /images/jdk/lib/libjsound.so
472K / 468K    /images/jdk/lib/libawt_xawt.so
564K / 404K   /images/jdk/lib/liblcms.so         <--------------------------
48K / 48K    /images/jdk/lib/libawt_headless.so
12K / 12K    /images/jdk/lib/libjawt.so
1.5M / 900K   /images/jdk/lib/libfontmanager.so  <------------------------------
12K / 12K    /images/jdk/lib/libjaas.so
92K / 92K    /images/jdk/lib/libj2pkcs11.so
16K / 16K    /images/jdk/lib/libattach.so
8.0K / 8.0K   /images/jdk/lib/librmi.so
56K / 56K    /images/jdk/lib/libinstrument.so
16K / 16K    /images/jdk/lib/libprefs.so
52K / 52K    /images/jdk/lib/libj2gss.so
12K / 12K    /images/jdk/lib/libmanagement_agent.so
36K / 32K    /images/jdk/lib/libmanagement.so
16K / 16K    /images/jdk/lib/libextnet.so
20K / 20K    /images/jdk/lib/libj2pcsc.so
40K / 40K    /images/jdk/lib/libmanagement_ext.so
60K / 60K    /images/jdk/lib/libsaproc.so


Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8236714

http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.2/


Thanks, Matthias

From lois.foltan at oracle.com  Tue Jan 14 15:05:34 2020
From: lois.foltan at oracle.com (Lois Foltan)
Date: Tue, 14 Jan 2020 10:05:34 -0500
Subject: RFR 8235678: Remove unnecessary calls to Thread::current() in
 MutexLocker calls
In-Reply-To: <0a7cbbe4-3613-bd5a-962b-78dee06dde8a@oracle.com>
References: <0a7cbbe4-3613-bd5a-962b-78dee06dde8a@oracle.com>
Message-ID: <e5553f18-f9d2-ecf5-2473-f49f1dcae828@oracle.com>

On 1/14/2020 9:00 AM, Harold Seigel wrote:
> Hi,
>
> Please review this small change, to reduce unnecessary calls to 
> Thread::current() in MutexLocker calls, by passing the current thread 
> as an argument.? A few ResoureMark declarations were also changed.
>
> Open Webrev: 
> http://cr.openjdk.java.net/~hseigel/bug_8235678/webrev/index.html
>
> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8235678
>
> The fix was regression tested by running Mach5 tiers 1 and 2 tests and 
> builds on Linux-x64, Solaris, Windows, and Mac OS X, by running Mach5 
> tiers 3-5 tests on Linux-x64, and JCK lang and VM tests on Linux-x64.
>
> Thanks, Harold
>

Overall looks great.? One comment:

- prims/methodHandles.cpp
line #1507: Curious to know why you use "THREAD" and the MutexLocker mu1 
at line #1502 uses "thread"?

Thanks,
Lois

From aleksei.voitylov at bell-sw.com  Tue Jan 14 15:15:14 2020
From: aleksei.voitylov at bell-sw.com (Aleksei Voitylov)
Date: Tue, 14 Jan 2020 18:15:14 +0300
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
 <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>
Message-ID: <249cba82-1469-03f8-d205-e5895eab2cc3@bell-sw.com>

Magnus, Matthias,

for me, lto is a little heavyweight for development. x86_64 build time
with gcc 7:

Server 1m32.484s
Server+Minimal 1m42.166s
Server+Minimal (--with-jvm-features="link-time-opt") 5m29.422s

If the change to enable lto by default is proposed, what would be the
recommended strategy for development?

For ARM32 Minimal, please keep in mind that it's not uncommon to disable
LTO plugin in commodity ARM32 gcc compiler distributions, so for some it
does not matter what settings we have in OpenJDK. I believe there could
be other reasons for that on top of build time (bugs?).

-Aleksei

On 14/01/2020 17:04, Magnus Ihse Bursie wrote:
> On 2020-01-14 13:49, Baesken, Matthias wrote:
>>
>> Hi Magnus, thanks for the info , I already noticed yesterday the
>> setting for arm-32 in the minimal build .
>>
>> Do you think? we could set it too for the other Linux platforms? in
>> the minimal build? ( serviceability agent is not supported there as
>> well so the? observed issue wouldn?t be a problem).
>>
>
> You mean if you could enable it on your builds without any issues? I'd
> guess so, but I don't know. Just try it:
> --with-jvm-features="link-time-opt".
>
> If you mean that it should be turned on by default on minimal builds
> for all platforms? No, I don't think that's a good idea. The link time
> is really a killer. I remember arm-32 going from like a couple of
> minutes to half an hour for linking libjvm.so.
>
> Things might be different with gold, though. I know they have done
> work with at least some kind of "lightweight" LTO, that might be worth
> at least looking into.
>
> /Magnus
>
>
>> Best regards, Matthias
>>
>> On 2020-01-10 11:01, Baesken, Matthias wrote:
>>
>> ??? Hello,?? I recently looked into? the? gcc? lto? optimization mode
>> (see for some
>> detailshttps://gcc.gnu.org/onlinedocs/gccint/LTO-Overview.html ?
>> andhttp://hubicka.blogspot.com/2019/05/gcc-9-link-time-and-inter-procedural.html
>> ? ).
>>
>> ??? This mode can lead to more compact binaries (~10% smaller)? , it
>> also might bring? small performance improvements? but that wasn't my
>> (main)? goal? .
>>
>> ??? The changes for this are rather small , one needs to use a recent
>> gcc? , add? -flto?? to the compile flags? , for example
>>
>> ??? --- a/make/autoconf/flags-cflags.m4????? Wed Jan 01 03:08:45 2020
>> +0100
>>
>> ??? +++ b/make/autoconf/flags-cflags.m4?? Wed Jan 08 17:39:10 2020 +0100
>>
>> ??? @@ -530,8 +530,13 @@
>>
>> ???? ?? fi
>>
>> ???? ?? if test "x$TOOLCHAIN_TYPE" = xgcc; then
>>
>> ??? -??? TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new
>> -fstack-protector"
>>
>> ??? -??? TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector"
>>
>> ??? +??? TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new
>> -fstack-protector -flto"
>>
>> ??? +??? TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector -flto"
>>
>> ???? ? .... and you have to make sure? to use? gcc-ar? and? gcc-nm
>> instead?? of? ar / nm .
>>
>> ??? Build and test(s)? work,? however with? one exception.
>>
>> ??? The? serviceability?? tests like? serviceability/sa?? seems to
>> rely?? heavily? on the "normal"?? structure? of?? libjvm.so?? (from
>> what I?? understand? e.g. in? LinuxVtblAccess? it is attempted to
>> access? internal symbols? like? _ZTV ).
>>
>> ??? Errors in the sa? tests look like :
>>
>> ??? java.lang.InternalError: Metadata does not appear to be polymorphic
>>
>> ???? ???????? at
>> jdk.hotspot.agent/sun.jvm.hotspot.types.basic.BasicTypeDataBase.findDynamicTypeForAddress(BasicTypeDataBase.java:279)
>>
>> ???? ???????? at
>> jdk.hotspot.agent/sun.jvm.hotspot.runtime.VirtualBaseConstructor.instantiateWrapperFor(VirtualBaseConstructor.java:102)
>>
>> ???? ???? ????at
>> jdk.hotspot.agent/sun.jvm.hotspot.oops.Metadata.instantiateWrapperFor(Metadata.java:74)
>>
>> ???? ???????? at
>> jdk.hotspot.agent/sun.jvm.hotspot.memory.SystemDictionary.getClassLoaderKlass(SystemDictionary.java:96)
>>
>> ???? ???????? at
>> jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.printClassLoaderStatistics(ClassLoaderStats.java:93)
>>
>> ???? ???????? at
>> jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.run(ClassLoaderStats.java:78)
>>
>> ???? ???????? at
>> jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.run(JMap.java:115)
>>
>> ???? ??? ?????at
>> jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:262)
>>
>> ???? ???????? at
>> jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:225)
>>
>> ???? ???????? at
>> jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
>>
>> ???? ???????? at
>> jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.main(JMap.java:176)
>>
>> ???? ???????? at
>> jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJMAP(SALauncher.java:321)
>>
>> ???? ???????? at
>> jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:406)
>>
>> ??? Has anyone experimented with LTO optimization ?
>>
>>
>> Hi Matthias,
>>
>> We used to have LTO enabled on the old, closed-source Oracle arm-32
>> builds. There is still a "link-time-opt" JVM feature present; afaik
>> it still works and adds the -flto flag. The main drawback of this is
>> the *extremely* long link times of libjvm.so.
>>
>> I don't think servicability was ever supported for that platform, so
>> I'm not surprised this does not work.
>>
>> /Magnus
>>
>>
>> ??? And to the? serviceability?? agent experts -? any idea? how to
>> make the? jdk.hotspot.agent?? more independent from? optimization
>> settings ?
>>
>> ??? Best regards, Matthias
>>
>

From erik.joelsson at oracle.com  Tue Jan 14 16:12:24 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Tue, 14 Jan 2020 08:12:24 -0800
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <b8f01c3f-27d1-f055-ba49-448a47ca4960@oracle.com>

(adding core-libs-dev)

Change looks good to me, but would like input from at least someone in 
core-libs.

/Erik

On 2020-01-14 06:07, Baesken, Matthias wrote:
> Hello,  the following change enables the  link-time section-gc for linux .
>
> gcc and ld support enabling "garbage collection" of unused input sections.
> This can be used to eliminate unused coding from native libraries (especially when already compiling the objects with compiler flags -ffunction-sections -fdata-sections .
> See for details the --gc-sections and --print-gc-sections parts of the ld documentation :
>
> https://linux.die.net/man/1/ld
>
>
> We had this enabled already  for  linux s390x ,  with https://bugs.openjdk.java.net/browse/JDK-8234525
> 8234525: enable link-time section-gc for linux s390x to remove unused code .
>
> This  time we enable it too for  the other linux platforms .
>
> For the other platforms I do not enable it for JVM, just for the JDK libs.  The reason is that the serviceability agent  (not supported on linux s390x )    is not  (yet) ready for the optimization .
> Below you see the results , for some libraries  a significant  size reduction can be achieved .
>
>
> Results from linux x86_64 product builds :
>
> without / with ltgc
>
> 320K / 300K    /images/jdk/lib/libsunec.so       <-------------------------
> 36K  / 36K     /images/jdk/lib/libdt_socket.so
> 280K / 276K   /images/jdk/lib/libjdwp.so
> 23M  / 23M    /images/jdk/lib/server/libjvm.so    <---- not set for libjvm.so for x86_64
> 16K  / 16K    /images/jdk/lib/server/libjsig.so
> 72K  / 72M    /images/jdk/lib/libverify.so
> 84K  / 84M   /images/jdk/lib/libjli.so
> 16K  / 16K    /images/jdk/lib/libjsig.so
> 196K / 196K   /images/jdk/lib/libjava.so
> 44K  / 44K    /images/jdk/lib/libzip.so
> 144K / 136K   /images/jdk/lib/libjimage.so
> 112K / 112K   /images/jdk/lib/libnet.so
> 100K / 100K   /images/jdk/lib/libnio.so
> 36K  / 36K    /images/jdk/lib/libsctp.so
> 576K / 556K   /images/jdk/lib/libmlib_image.so
> 752K / 752K   /images/jdk/lib/libawt.so
> 260K / 252K   /images/jdk/lib/libjavajpeg.so
> 784K / 784K   /images/jdk/lib/libfreetype.so
> 368K / 236K /images/jdk/lib/libsplashscreen.so   <-------------------------
> 88K / 88K    /images/jdk/lib/libjsound.so
> 472K / 468K    /images/jdk/lib/libawt_xawt.so
> 564K / 404K   /images/jdk/lib/liblcms.so         <--------------------------
> 48K / 48K    /images/jdk/lib/libawt_headless.so
> 12K / 12K    /images/jdk/lib/libjawt.so
> 1.5M / 900K   /images/jdk/lib/libfontmanager.so  <------------------------------
> 12K / 12K    /images/jdk/lib/libjaas.so
> 92K / 92K    /images/jdk/lib/libj2pkcs11.so
> 16K / 16K    /images/jdk/lib/libattach.so
> 8.0K / 8.0K   /images/jdk/lib/librmi.so
> 56K / 56K    /images/jdk/lib/libinstrument.so
> 16K / 16K    /images/jdk/lib/libprefs.so
> 52K / 52K    /images/jdk/lib/libj2gss.so
> 12K / 12K    /images/jdk/lib/libmanagement_agent.so
> 36K / 32K    /images/jdk/lib/libmanagement.so
> 16K / 16K    /images/jdk/lib/libextnet.so
> 20K / 20K    /images/jdk/lib/libj2pcsc.so
> 40K / 40K    /images/jdk/lib/libmanagement_ext.so
> 60K / 60K    /images/jdk/lib/libsaproc.so
>
>
> Bug/webrev :
>
> https://bugs.openjdk.java.net/browse/JDK-8236714
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.2/
>
>
> Thanks, Matthias

From coleen.phillimore at oracle.com  Tue Jan 14 16:24:43 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Tue, 14 Jan 2020 11:24:43 -0500
Subject: RFR 8235678: Remove unnecessary calls to Thread::current() in
 MutexLocker calls
In-Reply-To: <0a7cbbe4-3613-bd5a-962b-78dee06dde8a@oracle.com>
References: <0a7cbbe4-3613-bd5a-962b-78dee06dde8a@oracle.com>
Message-ID: <1fcf8439-2b5e-a776-9142-d9dc14a58faf@oracle.com>


Hi Harold,

I really wanted this change to move Thread to the first argument like 
many of the other calls in the VM that take THREAD as an argument.

Written like this:

+ MutexLocker mu(Threads_lock, THREAD);


It's too easy for someone who's cut/pasting to think the last THREAD 
argument should really be CHECK, which is completely wrong.

Can you switch the arguments?

Thanks,
Coleen

On 1/14/20 9:00 AM, Harold Seigel wrote:
> Hi,
>
> Please review this small change, to reduce unnecessary calls to 
> Thread::current() in MutexLocker calls, by passing the current thread 
> as an argument.? A few ResoureMark declarations were also changed.
>
> Open Webrev: 
> http://cr.openjdk.java.net/~hseigel/bug_8235678/webrev/index.html
>
> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8235678
>
> The fix was regression tested by running Mach5 tiers 1 and 2 tests and 
> builds on Linux-x64, Solaris, Windows, and Mac OS X, by running Mach5 
> tiers 3-5 tests on Linux-x64, and JCK lang and VM tests on Linux-x64.
>
> Thanks, Harold
>


From harold.seigel at oracle.com  Tue Jan 14 16:27:17 2020
From: harold.seigel at oracle.com (Harold Seigel)
Date: Tue, 14 Jan 2020 11:27:17 -0500
Subject: RFR 8235678: Remove unnecessary calls to Thread::current() in
 MutexLocker calls
In-Reply-To: <e5553f18-f9d2-ecf5-2473-f49f1dcae828@oracle.com>
References: <0a7cbbe4-3613-bd5a-962b-78dee06dde8a@oracle.com>
 <e5553f18-f9d2-ecf5-2473-f49f1dcae828@oracle.com>
Message-ID: <50019f6b-d94f-eb1a-de1c-622b437edf9f@oracle.com>

Hi Lois,

Thanks for the review!

 >> line #1507: Curious to know why you use "THREAD" and the MutexLocker 
mu1 at line #1502 uses "thread"?

I used THREAD when available because it is more typically used, but I 
didn't change those calls that used JavaThread* objects.

Harold

On 1/14/2020 10:05 AM, Lois Foltan wrote:
> On 1/14/2020 9:00 AM, Harold Seigel wrote:
>> Hi,
>>
>> Please review this small change, to reduce unnecessary calls to 
>> Thread::current() in MutexLocker calls, by passing the current thread 
>> as an argument.? A few ResoureMark declarations were also changed.
>>
>> Open Webrev: 
>> http://cr.openjdk.java.net/~hseigel/bug_8235678/webrev/index.html
>>
>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8235678
>>
>> The fix was regression tested by running Mach5 tiers 1 and 2 tests 
>> and builds on Linux-x64, Solaris, Windows, and Mac OS X, by running 
>> Mach5 tiers 3-5 tests on Linux-x64, and JCK lang and VM tests on 
>> Linux-x64.
>>
>> Thanks, Harold
>>
>
> Overall looks great.? One comment:
>
> - prims/methodHandles.cpp
> line #1507: Curious to know why you use "THREAD" and the MutexLocker 
> mu1 at line #1502 uses "thread"?
>
> Thanks,
> Lois

From harold.seigel at oracle.com  Tue Jan 14 16:33:00 2020
From: harold.seigel at oracle.com (Harold Seigel)
Date: Tue, 14 Jan 2020 11:33:00 -0500
Subject: RFR 8235678: Remove unnecessary calls to Thread::current() in
 MutexLocker calls
In-Reply-To: <1fcf8439-2b5e-a776-9142-d9dc14a58faf@oracle.com>
References: <0a7cbbe4-3613-bd5a-962b-78dee06dde8a@oracle.com>
 <1fcf8439-2b5e-a776-9142-d9dc14a58faf@oracle.com>
Message-ID: <e918a0ee-4285-881f-7527-63dc01eb0e16@oracle.com>

Hi Coleen,

I'll go ahead and switch the order and put out a new webrev.

Thanks for looking at it.

Harold

On 1/14/2020 11:24 AM, coleen.phillimore at oracle.com wrote:
>
> Hi Harold,
>
> I really wanted this change to move Thread to the first argument like 
> many of the other calls in the VM that take THREAD as an argument.
>
> Written like this:
>
> + MutexLocker mu(Threads_lock, THREAD);
>
>
> It's too easy for someone who's cut/pasting to think the last THREAD 
> argument should really be CHECK, which is completely wrong.
>
> Can you switch the arguments?
>
> Thanks,
> Coleen
>
> On 1/14/20 9:00 AM, Harold Seigel wrote:
>> Hi,
>>
>> Please review this small change, to reduce unnecessary calls to 
>> Thread::current() in MutexLocker calls, by passing the current thread 
>> as an argument.? A few ResoureMark declarations were also changed.
>>
>> Open Webrev: 
>> http://cr.openjdk.java.net/~hseigel/bug_8235678/webrev/index.html
>>
>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8235678
>>
>> The fix was regression tested by running Mach5 tiers 1 and 2 tests 
>> and builds on Linux-x64, Solaris, Windows, and Mac OS X, by running 
>> Mach5 tiers 3-5 tests on Linux-x64, and JCK lang and VM tests on 
>> Linux-x64.
>>
>> Thanks, Harold
>>
>

From matthias.baesken at sap.com  Tue Jan 14 16:57:33 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 14 Jan 2020 16:57:33 +0000
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <249cba82-1469-03f8-d205-e5895eab2cc3@bell-sw.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
 <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>
 <249cba82-1469-03f8-d205-e5895eab2cc3@bell-sw.com>
Message-ID: <AM6PR02MB4200457722633979CA9EC83093340@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello  Magnus and Aleksei,  thanks for the input .

The times you  provided really look like they make a big difference  at least for people  often  building   minimal-vm  .
Guess I have to measure myself a bit  (maybe the difference is not that big on our linux s390x / ppc64(le) ) .

>
> If the change to enable lto by default is proposed, what would be the
> recommended strategy for development?
>

Probably  we should a)   do not enable it by default but just make sure it can be enabled easily and works  for  the minimal-vm     or  b)  take it easy to disable it for local development.

Best regards, Matthias



> 
> Magnus, Matthias,
> 
> for me, lto is a little heavyweight for development. x86_64 build time
> with gcc 7:
> 
> Server 1m32.484s
> Server+Minimal 1m42.166s
> Server+Minimal (--with-jvm-features="link-time-opt") 5m29.422s
> 
> If the change to enable lto by default is proposed, what would be the
> recommended strategy for development?
> 
> For ARM32 Minimal, please keep in mind that it's not uncommon to disable
> LTO plugin in commodity ARM32 gcc compiler distributions, so for some it
> does not matter what settings we have in OpenJDK. I believe there could
> be other reasons for that on top of build time (bugs?).
> 


From igor.ignatyev at oracle.com  Tue Jan 14 17:03:38 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Tue, 14 Jan 2020 09:03:38 -0800
Subject: RFR(S): 8236111 : narrow allowSmartActionArgs disabling
In-Reply-To: <916e0375-abba-9945-b845-0fd4198513f0@oracle.com>
References: <F0F5A79A-2AF8-496F-94CB-26C3CC06A30D@oracle.com>
 <ECA06D8F-0F7A-4FBC-B502-4BDC52F019AA@oracle.com>
 <423ea31a-ebf8-4cba-72a4-6fbb934f7789@oracle.com>
 <0BA46866-3DEA-44BF-B87C-2B59B84196C9@oracle.com>
 <916e0375-abba-9945-b845-0fd4198513f0@oracle.com>
Message-ID: <AA6D290C-92AF-42F1-857C-AE1F87BB9815@oracle.com>

Joe and Roger, thank you for your reviews.

security-libs guys, could you please take a look?

Thanks,
-- Igor

> On Jan 2, 2020, at 12:58 PM, Roger Riggs <Roger.Riggs at Oracle.com> wrote:
> 
> The core lib changes look ok.
> 
> Roger

> On Jan 2, 2020, at 1:26 PM, Joe Darcy <joe.darcy at oracle.com> wrote:
> 
> The removal of the existing TEST.properties files look fine.
> 
> Please also solicit feedback from the security libs team as their area is affected.
> 
> Roger, FYI the serial filter tests are updated as part of this changeset.
> 
> Cheers,
> 
> -Joe
> 
> On 12/23/2019 8:13 PM, Igor Ignatyev wrote:
>> Thanks David.
>> 
>> core-libs folks, could you please review jdk part of this patch?
>> 
>> Thanks,
>> -- Igor
>> 
>>> On Dec 23, 2019, at 1:33 PM, David Holmes <david.holmes at oracle.com> wrote:
>>> 
>>> Hi Igor,
>>> 
>>> Hotspot changes seem fine. Can't comment on jdk tests.
>>> 
>>> Thanks,
>>> David
>>> 
>>> On 24/12/2019 6:42 am, Igor Ignatyev wrote:
>>>> ping?
>>>>> On Dec 17, 2019, at 11:30 AM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
>>>>> 
>>>>> http://cr.openjdk.java.net/~iignatyev/8236111/webrev.00/
>>>>>> 31 lines changed: 20 ins; 11 del; 0 mod;
>>>>> Hi all,
>>>>> 
>>>>> could you please review this small patch which enables allowSmartActionArgs in hotspot and jdk test suites and disables them in a small number of test directories? the patch also removes TEST.properties files which enabled allowSmartActionArgs as they aren't needed anymore.
>>>>> 
>>>>> from JBS:
>>>>>> currently, allowSmartActionArgs is disabled for the whole hotspot and jdk test suites and enabled just in few places. this makes it a bit harder for people to use smart action arguments in these test suites as they have to not to forget to enable them. and given in all the other test suites, smart action arguments are enabled, it can be confusing and frustrating.
>>>>> 
>>>>> testing: tier1-5
>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8236111
>>>>> webrev: http://cr.openjdk.java.net/~iignatyev/8236111/webrev.00/
>>>>> 
>>>>> Thanks,
>>>>> -- Igor


From aleksei.voitylov at bell-sw.com  Tue Jan 14 17:54:36 2020
From: aleksei.voitylov at bell-sw.com (Aleksei Voitylov)
Date: Tue, 14 Jan 2020 20:54:36 +0300
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <AM6PR02MB4200457722633979CA9EC83093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
 <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>
 <249cba82-1469-03f8-d205-e5895eab2cc3@bell-sw.com>
 <AM6PR02MB4200457722633979CA9EC83093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <9908f346-f376-5bfc-8d5b-bc250e336032@bell-sw.com>


On 14/01/2020 19:57, Baesken, Matthias wrote:
> Hello  Magnus and Aleksei,  thanks for the input .
>
> The times you  provided really look like they make a big difference  at least for people  often  building   minimal-vm  .
> Guess I have to measure myself a bit  (maybe the difference is not that big on our linux s390x / ppc64(le) ) .
>
>> If the change to enable lto by default is proposed, what would be the
>> recommended strategy for development?
>>
> Probably  we should a)   do not enable it by default but just make sure it can be enabled easily and works  for  the minimal-vm   
That would be welcome. I have high hopes to LTO the VM some time by
default, and the tendency observed is that the compiler time overhead
for GCC becomes smaller. At the same time there is no reason why vendors
that invested in testing and can absorb the build time hit could provide
binaries with LTO built VMs by passing an additional option flag.
>   or  b)  take it easy to disable it for local development.
>
> Best regards, Matthias
>
>
>
>> Magnus, Matthias,
>>
>> for me, lto is a little heavyweight for development. x86_64 build time
>> with gcc 7:
>>
>> Server 1m32.484s
>> Server+Minimal 1m42.166s
>> Server+Minimal (--with-jvm-features="link-time-opt") 5m29.422s
>>
>> If the change to enable lto by default is proposed, what would be the
>> recommended strategy for development?
>>
>> For ARM32 Minimal, please keep in mind that it's not uncommon to disable
>> LTO plugin in commodity ARM32 gcc compiler distributions, so for some it
>> does not matter what settings we have in OpenJDK. I believe there could
>> be other reasons for that on top of build time (bugs?).
>>


From bob.vandette at oracle.com  Tue Jan 14 20:04:55 2020
From: bob.vandette at oracle.com (Bob Vandette)
Date: Tue, 14 Jan 2020 15:04:55 -0500
Subject: [PING2!] RFR: 8230305: Cgroups v2: Container awareness
In-Reply-To: <b575b061cf4b7b0d9c0656aa1a726b2dd5a8a98c.camel@redhat.com>
References: <072f66ee8c44034831b4e38f6470da4bff6edd07.camel@redhat.com>
 <7540a208e306ab957032b18178a53c6afa105d33.camel@redhat.com>
 <CC473B8C-E6CB-4293-BE01-BEDD09BEB358@oracle.com>
 <baa4c4821174dc0dfa43190d862d74b22fbf35d4.camel@redhat.com>
 <d504e04dd277ef46f42064d2bbd54b1ea94da0c2.camel@redhat.com>
 <5eec97c04d86562346243c1db3832e86e13697a1.camel@redhat.com>
 <b575b061cf4b7b0d9c0656aa1a726b2dd5a8a98c.camel@redhat.com>
Message-ID: <52110EA5-26C9-43F2-8C2A-21D4E03ED3CC@oracle.com>


Cgroup V2 is about to go mainstream this year for popular distros such as Oracle Linux 8, Redhat Linux 8 and Fedora
so this fix it?s important to get into JDK 15 so we can start shaking out this support.

Please take a look and help get this change reviewed.

Thanks,
Bob Vandette



> On Nov 29, 2019, at 4:04 AM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
> 
> On Fri, 2019-11-15 at 17:56 +0100, Severin Gehwolf wrote:
>> On Fri, 2019-11-08 at 15:21 +0100, Severin Gehwolf wrote:
>>> Hi Bob,
>>> 
>>> On Wed, 2019-11-06 at 10:47 +0100, Severin Gehwolf wrote:
>>>> On Tue, 2019-11-05 at 16:54 -0500, Bob Vandette wrote:
>>>>> Severin,
>>>>> 
>>>>> Thanks for taking on this cgroup v2 improvement.  
>>>>> 
>>>>> In general I like the implementation and the refactoring.  The CachedMetric class is nice.
>>>>> We can add any metric we want to cache in a more general way.
>>>>> 
>>>>> Is this the latest version of the webrev?
>>>>> 
>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/cgroupsv2-hotspot/03/webrev/src/hotspot/os/linux/cgroupV2Subsystem_linux.cpp.html
>>>>> 
>>>>> It looks like you need to add the caching support for active_processor_count (JDK-8227006).
>>> [...]
>>>> I'll do a proper rebase ASAP.
>>> 
>>> Latest webrev:
>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/cgroupsv2-hotspot/05/webrev/
>>> 
>>>>> I?m not sure it?s worth providing different strings for Unlimited versus Max or Scaled shares.
>>>>> I?d just try to be compatible with the cgroupv2 output so you don?t have to change the test.
>>>> 
>>>> OK. Will do.
>>> 
>>> Unfortunately, there is no way of NOT changing TestCPUAwareness.java as
>>> it expects CPU Shares to be written to the cgroup filesystem verbatim.
>>> That's no longer the case for cgroups v2 (at least for crun). Either
>>> way, most test changes are gone now.
>>> 
>>>>> I wonder if it?s worth trying to synthesize memory_max_usage_in_bytes() by keeping the highest
>>>>> value ever returned by the API.
>>>> 
>>>> Interesting idea. I'll ponder this a bit and get back to you.
>>> 
>>> This has been implemented. I'm not sure this is correct, though. It
>>> merely piggy-backs on calls to memory_usage_in_bytes() and keeps the
>>> high watermark value of that.
>>> 
>>> Testing passed on F31 with cgroups v2 controllers properly configured
>>> (podman) and hybrid (legacy hierarchy) with docker/podman.
>>> 
>>> Thoughts?
>> 
>> Ping?
> 
> Anyone willing to review this? It would be nice to make some progress.
> 
> Thanks,
> Severin
> 
>> Metrics work proposed for RFR here:
>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2019-November/063464.html
>> 
>> Thanks,
>> Severin
> 


From david.holmes at oracle.com  Tue Jan 14 23:04:46 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 15 Jan 2020 09:04:46 +1000
Subject: RFR 8235678: Remove unnecessary calls to Thread::current() in
 MutexLocker calls
In-Reply-To: <e5553f18-f9d2-ecf5-2473-f49f1dcae828@oracle.com>
References: <0a7cbbe4-3613-bd5a-962b-78dee06dde8a@oracle.com>
 <e5553f18-f9d2-ecf5-2473-f49f1dcae828@oracle.com>
Message-ID: <c97a4062-fa96-39ca-7453-e60956e93be6@oracle.com>

Hi Lois,

On 15/01/2020 1:05 am, Lois Foltan wrote:
> On 1/14/2020 9:00 AM, Harold Seigel wrote:
>> Hi,
>>
>> Please review this small change, to reduce unnecessary calls to 
>> Thread::current() in MutexLocker calls, by passing the current thread 
>> as an argument.? A few ResoureMark declarations were also changed.
>>
>> Open Webrev: 
>> http://cr.openjdk.java.net/~hseigel/bug_8235678/webrev/index.html
>>
>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8235678
>>
>> The fix was regression tested by running Mach5 tiers 1 and 2 tests and 
>> builds on Linux-x64, Solaris, Windows, and Mac OS X, by running Mach5 
>> tiers 3-5 tests on Linux-x64, and JCK lang and VM tests on Linux-x64.
>>
>> Thanks, Harold
>>
> 
> Overall looks great.? One comment:
> 
> - prims/methodHandles.cpp
> line #1507: Curious to know why you use "THREAD" and the MutexLocker mu1 
> at line #1502 uses "thread"?

Just for reference THREAD is a Thread*, but thread is a JavaThread* 
introduced by some of the *ENTRY macros. It can be confusing when they 
both get used by code that only needs a Thread* - as per the line you 
quoted. We probably have a few places where we explicitly cast THREAD to 
JavaThread* unnecessarily because it isn't obvious that the thread 
variable exists.

David

> Thanks,
> Lois

From kim.barrett at oracle.com  Wed Jan 15 06:24:40 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 15 Jan 2020 01:24:40 -0500
Subject: RFR (M): 8235741: Inappropriate uses of os::javaTimeMillis()
In-Reply-To: <f99194ad-3dad-48db-d8d4-de0cc9161701@oracle.com>
References: <f99194ad-3dad-48db-d8d4-de0cc9161701@oracle.com>
Message-ID: <160821DF-BE15-435C-BDBE-AB553F794590@oracle.com>

> On Jan 13, 2020, at 2:13 AM, David Holmes <david.holmes at oracle.com> wrote:
> 
> webrev: http://cr.openjdk.java.net/~dholmes/8235741/webrev/
> bug: https://bugs.openjdk.java.net/browse/JDK-8235741
> 
> Full details in the bug report about the existing uses of javaTimeMillis(), many of which just want an elapsed time in ms and so should be using javaTimeNanos() and convert to ms. This covers areas all across the VM.
> 
> Only non-simple change is in os_perf_linux.cpp (and the same code will be in os_perf_aix.cpp once it has been validated). There we are tracking an elapsed time in ms but relative to the boot time, which is seconds since the epoch. Consequently the first interval has to be calculated using javaTimeMillis, but after that we can use javaTimeNanos (using a new 'first time' captured at the same time we used javaTimeMillis). I think I have the logic right but other than through JFR this code seems unused and I have limited means of testing it. The JFR test jdk/jfr/event/os/TestThreadContextSwitches.java exercises the code but the results of running that test seems to exhibit arbitrary randomness in the rates reported - e.g. 0 to 16000Hz - both with and without my change, so not really that useful. Stefan K. suggested a gtest which I may look into - though it is frustrating to have to expend such effort to validate this.
> 
> Other testing tiers 1-3.
> 
> Thanks,
> David

Thanks for the audit of uses of os::javaTimeMillis() in the bug report.
I wonder if some of that ought to be captured as comments in the
relevant code.  It's not always obvious to me that an external time
base is involved and thus making javaTimeMillis not a mistake.

There are a lot of places where conversions from nanoseconds to
milliseconds are being done to maintain existing units.  Some of those
places look like they could just as well be in nanoseconds.  But I can
see how changing the units for some of those could lead to a lot of
fannout, so okay.

------------------------------------------------------------------------------
src/hotspot/os/windows/os_perf_windows.cpp
 100   s8     lastUpdate; // Last time query was updated (current millis). 
...
 290   const s8 now = os::javaTimeNanos();
 291   if (NANOS_TO_MILLIS(now - update_query->lastUpdate) > min_update_interval_millis) {
...
 295     update_query->lastUpdate = now;

now and update_query->lastUpdate are now in nanos, but comment for
lastUpdate still says it's in millis.  Looks like the comment needs
updating. 

------------------------------------------------------------------------------
src/hotspot/share/utilities/globalDefinitions.hpp
 262 // time unit conversion macros
 263 
 264 #define NANOS_TO_MILLIS(ns) ((ns) / NANOSECS_PER_MILLISEC)
 265 #define MILLIS_TO_NANOS(ms) ((ms) * NANOSECS_PER_MILLISEC)

Why are these macros, rather than (template) functions?

Also, depending on the type and value of ms, MILLIS_TO_NANOS could
easily overflow, e.g. if ms type is a 32 bit type with a value of more
than ~4 seconds.  (I checked the two uses, and they happen to be okay.)

inline int64_t nanos_to_millis(int64_t ns) {
  return ns / NANOSECS_PER_MILLISECOND;
}

inline int64_t millis_to_nanos(int64_t ms) {
  return ms * NANOSECONDS_PER_MILLISEC;
}

Also, the names don't suggest time conversions, but potentially
arbitrary unit conversions, e.g. between something in NANOUNITS and
something in MILLIUNITS.

------------------------------------------------------------------------------
Regarding this from the audit:

--- begin ---
./share/gc/parallel/psParallelCompact.cpp: // os::javaTimeMillis() does not guarantee monotonicity.
...
./share/gc/shared/referenceProcessor.cpp: // os::javaTimeMillis() does not guarantee monotonicity.

These are all describing why the subsequent code uses javaTimeNanos not javaTimeMillis.
--- end ---

Do we really still support platforms that don't have a monotonic
clock?  I guess we appear to at least try.  But it's really wrong that
callers of os::javaTimeNanos should even think they need to cope with
that function being non-monotonic.

Hm, I always thought System.nanoTime() was a monotonic clock, but I
don't see any such guarantee. So I guess Java just doesn't have such a
thing. Wow!

So I guess none of this is really relevant to the change at hand after all.

------------------------------------------------------------------------------



From david.holmes at oracle.com  Wed Jan 15 07:12:05 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 15 Jan 2020 17:12:05 +1000
Subject: RFR (M): 8235741: Inappropriate uses of os::javaTimeMillis()
In-Reply-To: <160821DF-BE15-435C-BDBE-AB553F794590@oracle.com>
References: <f99194ad-3dad-48db-d8d4-de0cc9161701@oracle.com>
 <160821DF-BE15-435C-BDBE-AB553F794590@oracle.com>
Message-ID: <a04e7b9f-3fe8-01ea-9784-fcf21613340d@oracle.com>

Hi Kim,

Thanks for taking a look at this.

On 15/01/2020 4:24 pm, Kim Barrett wrote:
>> On Jan 13, 2020, at 2:13 AM, David Holmes <david.holmes at oracle.com> wrote:
>>
>> webrev: http://cr.openjdk.java.net/~dholmes/8235741/webrev/
>> bug: https://bugs.openjdk.java.net/browse/JDK-8235741
>>
>> Full details in the bug report about the existing uses of javaTimeMillis(), many of which just want an elapsed time in ms and so should be using javaTimeNanos() and convert to ms. This covers areas all across the VM.
>>
>> Only non-simple change is in os_perf_linux.cpp (and the same code will be in os_perf_aix.cpp once it has been validated). There we are tracking an elapsed time in ms but relative to the boot time, which is seconds since the epoch. Consequently the first interval has to be calculated using javaTimeMillis, but after that we can use javaTimeNanos (using a new 'first time' captured at the same time we used javaTimeMillis). I think I have the logic right but other than through JFR this code seems unused and I have limited means of testing it. The JFR test jdk/jfr/event/os/TestThreadContextSwitches.java exercises the code but the results of running that test seems to exhibit arbitrary randomness in the rates reported - e.g. 0 to 16000Hz - both with and without my change, so not really that useful. Stefan K. suggested a gtest which I may look into - though it is frustrating to have to expend such effort to validate this.
>>
>> Other testing tiers 1-3.
>>
>> Thanks,
>> David
> 
> Thanks for the audit of uses of os::javaTimeMillis() in the bug report.
> I wonder if some of that ought to be captured as comments in the
> relevant code.  It's not always obvious to me that an external time
> base is involved and thus making javaTimeMillis not a mistake.

Okay, I will add comments to the other uses of currentTimeMillis().

> There are a lot of places where conversions from nanoseconds to
> milliseconds are being done to maintain existing units.  Some of those
> places look like they could just as well be in nanoseconds.  But I can
> see how changing the units for some of those could lead to a lot of
> fannout, so okay.

Yes I tried to minimise the changes. In many cases a granularity of ms 
seems somewhat arbitrary.

> ------------------------------------------------------------------------------
> src/hotspot/os/windows/os_perf_windows.cpp
>   100   s8     lastUpdate; // Last time query was updated (current millis).
> ...
>   290   const s8 now = os::javaTimeNanos();
>   291   if (NANOS_TO_MILLIS(now - update_query->lastUpdate) > min_update_interval_millis) {
> ...
>   295     update_query->lastUpdate = now;
> 
> now and update_query->lastUpdate are now in nanos, but comment for
> lastUpdate still says it's in millis.  Looks like the comment needs
> updating.

Yes - good catch.

> ------------------------------------------------------------------------------
> src/hotspot/share/utilities/globalDefinitions.hpp
>   262 // time unit conversion macros
>   263
>   264 #define NANOS_TO_MILLIS(ns) ((ns) / NANOSECS_PER_MILLISEC)
>   265 #define MILLIS_TO_NANOS(ms) ((ms) * NANOSECS_PER_MILLISEC)
> 
> Why are these macros, rather than (template) functions?

Just because I just wanted a simple textual replacement to make it 
clearer that I'm converting from millis to nanos or vice versa. I reach 
for macros for such simple cases.

> Also, depending on the type and value of ms, MILLIS_TO_NANOS could
> easily overflow, e.g. if ms type is a 32 bit type with a value of more
> than ~4 seconds.  (I checked the two uses, and they happen to be okay.)

These are not trying to be mathematically sound. The conversion from 
millis to nanos is used in two cases:

1. Converting a current timestamp in ms to ns. Unless the current time 
is set far in the future I don't think we have any issue with overflow 
of  such a value.

2. converting an elapsed time in ms to ns. These will be small values so 
no overflow is possible.

> inline int64_t nanos_to_millis(int64_t ns) {
>    return ns / NANOSECS_PER_MILLISECOND;
> }
> 
> inline int64_t millis_to_nanos(int64_t ms) {
>    return ms * NANOSECONDS_PER_MILLISEC;
> }
> 
> Also, the names don't suggest time conversions, but potentially
> arbitrary unit conversions, e.g. between something in NANOUNITS and
> something in MILLIUNITS.

They don't have to be time conversions - the calculation is unit-less in 
practice. The fact we have NANOSEC_PER_MILLISECOND et al is just an 
artifact of introducing those values for timeout 
calculations/conversions - it could just be NANOS_PER_MILLI etc

> ------------------------------------------------------------------------------
> Regarding this from the audit:
> 
> --- begin ---
> ./share/gc/parallel/psParallelCompact.cpp: // os::javaTimeMillis() does not guarantee monotonicity.
> ...
> ./share/gc/shared/referenceProcessor.cpp: // os::javaTimeMillis() does not guarantee monotonicity.
> 
> These are all describing why the subsequent code uses javaTimeNanos not javaTimeMillis.
> --- end ---
> 
> Do we really still support platforms that don't have a monotonic
> clock?  I guess we appear to at least try.  But it's really wrong that
> callers of os::javaTimeNanos should even think they need to cope with
> that function being non-monotonic.
> 
> Hm, I always thought System.nanoTime() was a monotonic clock, but I
> don't see any such guarantee. So I guess Java just doesn't have such a
> thing. Wow!
> 
> So I guess none of this is really relevant to the change at hand after all.

I think you read the comments the wrong way round. The code uses 
javaTimeNanos not javaTimeMillis because javaTimeMillis is not monotonic 
and the code wants a monotonic clock. These comments were mostly 
inserted when the incorrect use of javaTimeMillis was replaced with 
javaTimeNanos.

Thanks,
David
-----

> ------------------------------------------------------------------------------
> 
> 

From matthias.baesken at sap.com  Wed Jan 15 08:27:09 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 15 Jan 2020 08:27:09 +0000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <b8f01c3f-27d1-f055-ba49-448a47ca4960@oracle.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <b8f01c3f-27d1-f055-ba49-448a47ca4960@oracle.com>
Message-ID: <AM6PR02MB420099D1C75E508D1BE1CD8D93370@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hi Erik, thanks for the review and for forwarding ,  you are correct  corelibs-dev is probably  interested in this as well .

Best regards, Matthias


> (adding core-libs-dev)
> 
> Change looks good to me, but would like input from at least someone in
> core-libs.
> 
> /Erik
> 
> On 2020-01-14 06:07, Baesken, Matthias wrote:
> > Hello,  the following change enables the  link-time section-gc for linux .
> >
> > gcc and ld support enabling "garbage collection" of unused input sections.
> > This can be used to eliminate unused coding from native libraries
> (especially when already compiling the objects with compiler flags -ffunction-
> sections -fdata-sections .
> > See for details the --gc-sections and --print-gc-sections parts of the ld
> documentation :
> >
> > https://linux.die.net/man/1/ld
> >
> >
> > We had this enabled already  for  linux s390x ,  with
> https://bugs.openjdk.java.net/browse/JDK-8234525
> > 8234525: enable link-time section-gc for linux s390x to remove unused code
> .
> >
> > This  time we enable it too for  the other linux platforms .
> >
> > For the other platforms I do not enable it for JVM, just for the JDK libs.  The
> reason is that the serviceability agent  (not supported on linux s390x )    is not
> (yet) ready for the optimization .
> > Below you see the results , for some libraries  a significant  size reduction
> can be achieved .
> >
> >
> > Results from linux x86_64 product builds :
> >
> > without / with ltgc
> >
> > 320K / 300K    /images/jdk/lib/libsunec.so       <-------------------------
> > 36K  / 36K     /images/jdk/lib/libdt_socket.so
> > 280K / 276K   /images/jdk/lib/libjdwp.so
> > 23M  / 23M    /images/jdk/lib/server/libjvm.so    <---- not set for libjvm.so
> for x86_64
> > 16K  / 16K    /images/jdk/lib/server/libjsig.so
> > 72K  / 72M    /images/jdk/lib/libverify.so
> > 84K  / 84M   /images/jdk/lib/libjli.so
> > 16K  / 16K    /images/jdk/lib/libjsig.so
> > 196K / 196K   /images/jdk/lib/libjava.so
> > 44K  / 44K    /images/jdk/lib/libzip.so
> > 144K / 136K   /images/jdk/lib/libjimage.so
> > 112K / 112K   /images/jdk/lib/libnet.so
> > 100K / 100K   /images/jdk/lib/libnio.so
> > 36K  / 36K    /images/jdk/lib/libsctp.so
> > 576K / 556K   /images/jdk/lib/libmlib_image.so
> > 752K / 752K   /images/jdk/lib/libawt.so
> > 260K / 252K   /images/jdk/lib/libjavajpeg.so
> > 784K / 784K   /images/jdk/lib/libfreetype.so
> > 368K / 236K /images/jdk/lib/libsplashscreen.so   <-------------------------
> > 88K / 88K    /images/jdk/lib/libjsound.so
> > 472K / 468K    /images/jdk/lib/libawt_xawt.so
> > 564K / 404K   /images/jdk/lib/liblcms.so         <--------------------------
> > 48K / 48K    /images/jdk/lib/libawt_headless.so
> > 12K / 12K    /images/jdk/lib/libjawt.so
> > 1.5M / 900K   /images/jdk/lib/libfontmanager.so  <------------------------------
> > 12K / 12K    /images/jdk/lib/libjaas.so
> > 92K / 92K    /images/jdk/lib/libj2pkcs11.so
> > 16K / 16K    /images/jdk/lib/libattach.so
> > 8.0K / 8.0K   /images/jdk/lib/librmi.so
> > 56K / 56K    /images/jdk/lib/libinstrument.so
> > 16K / 16K    /images/jdk/lib/libprefs.so
> > 52K / 52K    /images/jdk/lib/libj2gss.so
> > 12K / 12K    /images/jdk/lib/libmanagement_agent.so
> > 36K / 32K    /images/jdk/lib/libmanagement.so
> > 16K / 16K    /images/jdk/lib/libextnet.so
> > 20K / 20K    /images/jdk/lib/libj2pcsc.so
> > 40K / 40K    /images/jdk/lib/libmanagement_ext.so
> > 60K / 60K    /images/jdk/lib/libsaproc.so
> >
> >
> > Bug/webrev :
> >
> > https://bugs.openjdk.java.net/browse/JDK-8236714
> >
> > http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.2/
> >
> >
> > Thanks, Matthias

From harold.seigel at oracle.com  Wed Jan 15 12:57:24 2020
From: harold.seigel at oracle.com (Harold Seigel)
Date: Wed, 15 Jan 2020 07:57:24 -0500
Subject: RFR 8235678: Remove unnecessary calls to Thread::current() in
 MutexLocker calls
In-Reply-To: <1fcf8439-2b5e-a776-9142-d9dc14a58faf@oracle.com>
References: <0a7cbbe4-3613-bd5a-962b-78dee06dde8a@oracle.com>
 <1fcf8439-2b5e-a776-9142-d9dc14a58faf@oracle.com>
Message-ID: <0758d1d1-b74b-31e6-82fb-a1223a3481f7@oracle.com>

Hi,

Please review this new webrev that also makes Thread* the first argument 
to the relevant MutexLocker and MonitorLocker constructors as requested 
by Coleen.

Updated Webrev: 
http://cr.openjdk.java.net/~hseigel/bug_8235678.2/webrev/index.html

Thanks, Harold

On 1/14/2020 11:24 AM, coleen.phillimore at oracle.com wrote:
>
> Hi Harold,
>
> I really wanted this change to move Thread to the first argument like 
> many of the other calls in the VM that take THREAD as an argument.
>
> Written like this:
>
> + MutexLocker mu(Threads_lock, THREAD);
>
>
> It's too easy for someone who's cut/pasting to think the last THREAD 
> argument should really be CHECK, which is completely wrong.
>
> Can you switch the arguments?
>
> Thanks,
> Coleen
>
> On 1/14/20 9:00 AM, Harold Seigel wrote:
>> Hi,
>>
>> Please review this small change, to reduce unnecessary calls to 
>> Thread::current() in MutexLocker calls, by passing the current thread 
>> as an argument.? A few ResoureMark declarations were also changed.
>>
>> Open Webrev: 
>> http://cr.openjdk.java.net/~hseigel/bug_8235678/webrev/index.html
>>
>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8235678
>>
>> The fix was regression tested by running Mach5 tiers 1 and 2 tests 
>> and builds on Linux-x64, Solaris, Windows, and Mac OS X, by running 
>> Mach5 tiers 3-5 tests on Linux-x64, and JCK lang and VM tests on 
>> Linux-x64.
>>
>> Thanks, Harold
>>
>

From volker.simonis at gmail.com  Wed Jan 15 13:40:02 2020
From: volker.simonis at gmail.com (Volker Simonis)
Date: Wed, 15 Jan 2020 05:40:02 -0800
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <9908f346-f376-5bfc-8d5b-bc250e336032@bell-sw.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
 <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>
 <249cba82-1469-03f8-d205-e5895eab2cc3@bell-sw.com>
 <AM6PR02MB4200457722633979CA9EC83093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <9908f346-f376-5bfc-8d5b-bc250e336032@bell-sw.com>
Message-ID: <CA+3eh11mg+UgbwgF7qCHkq3hvr73H=3X4+sDTaZjdvZ3kTuejQ@mail.gmail.com>

While we are speaking about all the drawbacks of LTO, it's still not clear
what the benefits are? In the very first mail Matthias mentioned that there
might be performance improvements but that performance is not the main
driving factor behind this initiative. So is it the reduced code size
(Matthias mentioned something around ~10%)?

It would be nice to see some real numbers on various platform for both, the
performance improvements for native parts like JIT/GC as well as for the
size reduction.

Aleksei Voitylov <aleksei.voitylov at bell-sw.com> schrieb am Di., 14. Jan.
2020, 09:54:

>
> On 14/01/2020 19:57, Baesken, Matthias wrote:
> > Hello  Magnus and Aleksei,  thanks for the input .
> >
> > The times you  provided really look like they make a big difference  at
> least for people  often  building   minimal-vm  .
> > Guess I have to measure myself a bit  (maybe the difference is not that
> big on our linux s390x / ppc64(le) ) .
> >
> >> If the change to enable lto by default is proposed, what would be the
> >> recommended strategy for development?
> >>
> > Probably  we should a)   do not enable it by default but just make sure
> it can be enabled easily and works  for  the minimal-vm
> That would be welcome. I have high hopes to LTO the VM some time by
> default, and the tendency observed is that the compiler time overhead
> for GCC becomes smaller. At the same time there is no reason why vendors
> that invested in testing and can absorb the build time hit could provide
> binaries with LTO built VMs by passing an additional option flag.
> >   or  b)  take it easy to disable it for local development.
> >
> > Best regards, Matthias
> >
> >
> >
> >> Magnus, Matthias,
> >>
> >> for me, lto is a little heavyweight for development. x86_64 build time
> >> with gcc 7:
> >>
> >> Server 1m32.484s
> >> Server+Minimal 1m42.166s
> >> Server+Minimal (--with-jvm-features="link-time-opt") 5m29.422s
> >>
> >> If the change to enable lto by default is proposed, what would be the
> >> recommended strategy for development?
> >>
> >> For ARM32 Minimal, please keep in mind that it's not uncommon to disable
> >> LTO plugin in commodity ARM32 gcc compiler distributions, so for some it
> >> does not matter what settings we have in OpenJDK. I believe there could
> >> be other reasons for that on top of build time (bugs?).
> >>
>
>

From erik.joelsson at oracle.com  Wed Jan 15 14:06:59 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Wed, 15 Jan 2020 06:06:59 -0800
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <AM6PR02MB420099D1C75E508D1BE1CD8D93370@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <b8f01c3f-27d1-f055-ba49-448a47ca4960@oracle.com>
 <AM6PR02MB420099D1C75E508D1BE1CD8D93370@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <998c4a08-5670-f51d-4625-9c7e984b4b5d@oracle.com>

Given the discussion regarding lto on hotspot and the extreme increased 
build time, have you noticed any difference in build times with this patch?

/Erik

On 2020-01-15 00:27, Baesken, Matthias wrote:
> Hi Erik, thanks for the review and for forwarding ,  you are correct  corelibs-dev is probably  interested in this as well .
>
> Best regards, Matthias
>
>
>> (adding core-libs-dev)
>>
>> Change looks good to me, but would like input from at least someone in
>> core-libs.
>>
>> /Erik
>>
>> On 2020-01-14 06:07, Baesken, Matthias wrote:
>>> Hello,  the following change enables the  link-time section-gc for linux .
>>>
>>> gcc and ld support enabling "garbage collection" of unused input sections.
>>> This can be used to eliminate unused coding from native libraries
>> (especially when already compiling the objects with compiler flags -ffunction-
>> sections -fdata-sections .
>>> See for details the --gc-sections and --print-gc-sections parts of the ld
>> documentation :
>>> https://linux.die.net/man/1/ld
>>>
>>>
>>> We had this enabled already  for  linux s390x ,  with
>> https://bugs.openjdk.java.net/browse/JDK-8234525
>>> 8234525: enable link-time section-gc for linux s390x to remove unused code
>> .
>>> This  time we enable it too for  the other linux platforms .
>>>
>>> For the other platforms I do not enable it for JVM, just for the JDK libs.  The
>> reason is that the serviceability agent  (not supported on linux s390x )    is not
>> (yet) ready for the optimization .
>>> Below you see the results , for some libraries  a significant  size reduction
>> can be achieved .
>>>
>>> Results from linux x86_64 product builds :
>>>
>>> without / with ltgc
>>>
>>> 320K / 300K    /images/jdk/lib/libsunec.so       <-------------------------
>>> 36K  / 36K     /images/jdk/lib/libdt_socket.so
>>> 280K / 276K   /images/jdk/lib/libjdwp.so
>>> 23M  / 23M    /images/jdk/lib/server/libjvm.so    <---- not set for libjvm.so
>> for x86_64
>>> 16K  / 16K    /images/jdk/lib/server/libjsig.so
>>> 72K  / 72M    /images/jdk/lib/libverify.so
>>> 84K  / 84M   /images/jdk/lib/libjli.so
>>> 16K  / 16K    /images/jdk/lib/libjsig.so
>>> 196K / 196K   /images/jdk/lib/libjava.so
>>> 44K  / 44K    /images/jdk/lib/libzip.so
>>> 144K / 136K   /images/jdk/lib/libjimage.so
>>> 112K / 112K   /images/jdk/lib/libnet.so
>>> 100K / 100K   /images/jdk/lib/libnio.so
>>> 36K  / 36K    /images/jdk/lib/libsctp.so
>>> 576K / 556K   /images/jdk/lib/libmlib_image.so
>>> 752K / 752K   /images/jdk/lib/libawt.so
>>> 260K / 252K   /images/jdk/lib/libjavajpeg.so
>>> 784K / 784K   /images/jdk/lib/libfreetype.so
>>> 368K / 236K /images/jdk/lib/libsplashscreen.so   <-------------------------
>>> 88K / 88K    /images/jdk/lib/libjsound.so
>>> 472K / 468K    /images/jdk/lib/libawt_xawt.so
>>> 564K / 404K   /images/jdk/lib/liblcms.so         <--------------------------
>>> 48K / 48K    /images/jdk/lib/libawt_headless.so
>>> 12K / 12K    /images/jdk/lib/libjawt.so
>>> 1.5M / 900K   /images/jdk/lib/libfontmanager.so  <------------------------------
>>> 12K / 12K    /images/jdk/lib/libjaas.so
>>> 92K / 92K    /images/jdk/lib/libj2pkcs11.so
>>> 16K / 16K    /images/jdk/lib/libattach.so
>>> 8.0K / 8.0K   /images/jdk/lib/librmi.so
>>> 56K / 56K    /images/jdk/lib/libinstrument.so
>>> 16K / 16K    /images/jdk/lib/libprefs.so
>>> 52K / 52K    /images/jdk/lib/libj2gss.so
>>> 12K / 12K    /images/jdk/lib/libmanagement_agent.so
>>> 36K / 32K    /images/jdk/lib/libmanagement.so
>>> 16K / 16K    /images/jdk/lib/libextnet.so
>>> 20K / 20K    /images/jdk/lib/libj2pcsc.so
>>> 40K / 40K    /images/jdk/lib/libmanagement_ext.so
>>> 60K / 60K    /images/jdk/lib/libsaproc.so
>>>
>>>
>>> Bug/webrev :
>>>
>>> https://bugs.openjdk.java.net/browse/JDK-8236714
>>>
>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.2/
>>>
>>>
>>> Thanks, Matthias

From aleksei.voitylov at bell-sw.com  Wed Jan 15 14:57:11 2020
From: aleksei.voitylov at bell-sw.com (Aleksei Voitylov)
Date: Wed, 15 Jan 2020 17:57:11 +0300
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <CA+3eh11mg+UgbwgF7qCHkq3hvr73H=3X4+sDTaZjdvZ3kTuejQ@mail.gmail.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
 <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>
 <249cba82-1469-03f8-d205-e5895eab2cc3@bell-sw.com>
 <AM6PR02MB4200457722633979CA9EC83093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <9908f346-f376-5bfc-8d5b-bc250e336032@bell-sw.com>
 <CA+3eh11mg+UgbwgF7qCHkq3hvr73H=3X4+sDTaZjdvZ3kTuejQ@mail.gmail.com>
Message-ID: <58b2c73c-c49e-49cf-71fc-7d6c2225b880@bell-sw.com>

Volker,

not a full answer, but here is some static size stats:

Server ??? x86_64? AArch64
regular ??? 23M ?? ?? 20M
lto ??? ??? ?? 17M ? ??? 14M

Minimal?? x86_64? AArch64
regular???? 4.9M????? 3.9M
lto??????????? 4.7M????? 3.6M

-Aleksei

On 15/01/2020 16:40, Volker Simonis wrote:
> While we are speaking about all the drawbacks of LTO, it's still not
> clear what the benefits are? In the very first mail Matthias mentioned
> that there might be performance improvements but that performance is
> not the main driving factor behind this initiative. So is it the
> reduced code size (Matthias mentioned something around ~10%)?
>
> It would be nice to see some real numbers on various platform for
> both, the performance improvements for native parts like JIT/GC as
> well as for the size reduction.
>
> Aleksei Voitylov <aleksei.voitylov at bell-sw.com
> <mailto:aleksei.voitylov at bell-sw.com>> schrieb am Di., 14. Jan. 2020,
> 09:54:
>
>
>     On 14/01/2020 19:57, Baesken, Matthias wrote:
>     > Hello? Magnus and Aleksei,? thanks for the input .
>     >
>     > The times you? provided really look like they make a big
>     difference? at least for people? often? building? ?minimal-vm? .
>     > Guess I have to measure myself a bit? (maybe the difference is
>     not that big on our linux s390x / ppc64(le) ) .
>     >
>     >> If the change to enable lto by default is proposed, what would
>     be the
>     >> recommended strategy for development?
>     >>
>     > Probably? we should a)? ?do not enable it by default but just
>     make sure it can be enabled easily and works? for? the minimal-vm? ?
>     That would be welcome. I have high hopes to LTO the VM some time by
>     default, and the tendency observed is that the compiler time overhead
>     for GCC becomes smaller. At the same time there is no reason why
>     vendors
>     that invested in testing and can absorb the build time hit could
>     provide
>     binaries with LTO built VMs by passing an additional option flag.
>     >? ?or? b)? take it easy to disable it for local development.
>     >
>     > Best regards, Matthias
>     >
>     >
>     >
>     >> Magnus, Matthias,
>     >>
>     >> for me, lto is a little heavyweight for development. x86_64
>     build time
>     >> with gcc 7:
>     >>
>     >> Server 1m32.484s
>     >> Server+Minimal 1m42.166s
>     >> Server+Minimal (--with-jvm-features="link-time-opt") 5m29.422s
>     >>
>     >> If the change to enable lto by default is proposed, what would
>     be the
>     >> recommended strategy for development?
>     >>
>     >> For ARM32 Minimal, please keep in mind that it's not uncommon
>     to disable
>     >> LTO plugin in commodity ARM32 gcc compiler distributions, so
>     for some it
>     >> does not matter what settings we have in OpenJDK. I believe
>     there could
>     >> be other reasons for that on top of build time (bugs?).
>     >>
>

From matthias.baesken at sap.com  Wed Jan 15 15:02:37 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 15 Jan 2020 15:02:37 +0000
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <CA+3eh11mg+UgbwgF7qCHkq3hvr73H=3X4+sDTaZjdvZ3kTuejQ@mail.gmail.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
 <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>
 <249cba82-1469-03f8-d205-e5895eab2cc3@bell-sw.com>
 <AM6PR02MB4200457722633979CA9EC83093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <9908f346-f376-5bfc-8d5b-bc250e336032@bell-sw.com>
 <CA+3eh11mg+UgbwgF7qCHkq3hvr73H=3X4+sDTaZjdvZ3kTuejQ@mail.gmail.com>
Message-ID: <AM6PR02MB42007C34069D56491E86D8B893370@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello , I can comment on   the  code size .  This is what I get when comparing  a build  without  and  with  -flto .

gcc7 linux x86_64  product build, normal / with -flto
----------------------------------------------------------------------------------

du -sh on the *.so files gives :

16K / 16K      ./lib/libattach.so
48K / 44K      ./lib/libawt_headless.so
752K / 760K    ./lib/libawt.so            <------------------ this one gets a bit larger with flto
472K / 456K    ./lib/libawt_xawt.so       <------------------ small gain
36K / 32K      ./lib/libdt_socket.so
16K /16K       ./lib/libextnet.so
1.5M / 824K    ./lib/libfontmanager.so     <------------------ HUGE gain
784K / 792K    ./lib/libfreetype.so        <------------------ this one gets a bit larger with flto
56K / 56K      ./lib/libinstrument.so
52K / 52K      ./lib/libj2gss.so
20K / 20K      ./lib/libj2pcsc.so
92K / 84K      ./lib/libj2pkcs11.so
12K / 12k      ./lib/libjaas.so
260K / 244K    ./lib/libjavajpeg.so         <----------------- small gain
196K / 188K    ./lib/libjava.so
12K / 12K      ./lib/libjawt.so
280K / 256K    ./lib/libjdwp.so             <----------------- small gain
144K / 140K    ./lib/libjimage.so
84K / 76K      ./lib/libjli.so
16K / 16K      ./lib/libjsig.so
88K / 80K      ./lib/libjsound.so
564K / 420K    ./lib/liblcms.so             <----------------- large gain
12K / 12K      ./lib/libmanagement_agent.so
40K / 36K      ./lib/libmanagement_ext.so
36K / 32K      ./lib/libmanagement.so
576K / 496K    ./lib/libmlib_image.so       <----------------- large gain
112K / 108K    ./lib/libnet.so
100K / 100K    ./lib/libnio.so
16K  / 16K     ./lib/libprefs.so
8.0K / 8.0K    ./lib/librmi.so
60K / 60K      ./lib/libsaproc.so
36K / 32K      ./lib/libsctp.so
368K / 212K    ./lib/libsplashscreen.so     <----------------- large gain
320K / 296K    ./lib/libsunec.so            <----------------- medium gain
72K / 72K      ./lib/libverify.so
44K / 44K      ./lib/libzip.so
16K / 16K      ./lib/server/libjsig.so
23M / 17M      ./lib/server/libjvm.so        <----------------- big gain maybe because it is C++ ?


So  for  some libs  you see  10% and more , but not for all .   But  most  large  libs  like   libjvm.so,  libfontmanager.so      or   liblcms.so    we see good results regarding reduced code size.

I Cannot say much about performance improvements , probably it would be small .

For SPEC  you find something at

http://hubicka.blogspot.com/2019/05/gcc-9-link-time-and-inter-procedural.html

(not that these results would say too much about  JVM performance ).


Best regards, Matthias

From: Volker Simonis <volker.simonis at gmail.com>
Sent: Mittwoch, 15. Januar 2020 14:40
To: Aleksei Voitylov <aleksei.voitylov at bell-sw.com>
Cc: Baesken, Matthias <matthias.baesken at sap.com>; Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>; serviceability-dev at openjdk.java.net; build-dev <build-dev at openjdk.java.net>; hotspot-dev at openjdk.java.net
Subject: Re: serviceability agent : problems when using gcc LTO (link time optimization)

While we are speaking about all the drawbacks of LTO, it's still not clear what the benefits are? In the very first mail Matthias mentioned that there might be performance improvements but that performance is not the main driving factor behind this initiative. So is it the reduced code size (Matthias mentioned something around ~10%)?

It would be nice to see some real numbers on various platform for both, the performance improvements for native parts like JIT/GC as well as for the size reduction.
Aleksei Voitylov <aleksei.voitylov at bell-sw.com<mailto:aleksei.voitylov at bell-sw.com>> schrieb am Di., 14. Jan. 2020, 09:54:

On 14/01/2020 19:57, Baesken, Matthias wrote:
> Hello  Magnus and Aleksei,  thanks for the input .
>
> The times you  provided really look like they make a big difference  at least for people  often  building   minimal-vm  .
> Guess I have to measure myself a bit  (maybe the difference is not that big on our linux s390x / ppc64(le) ) .
>
>> If the change to enable lto by default is proposed, what would be the
>> recommended strategy for development?
>>
> Probably  we should a)   do not enable it by default but just make sure it can be enabled easily and works  for  the minimal-vm
That would be welcome. I have high hopes to LTO the VM some time by
default, and the tendency observed is that the compiler time overhead
for GCC becomes smaller. At the same time there is no reason why vendors
that invested in testing and can absorb the build time hit could provide
binaries with LTO built VMs by passing an additional option flag.
>   or  b)  take it easy to disable it for local development.
>
> Best regards, Matthias
>
>
>
>> Magnus, Matthias,
>>
>> for me, lto is a little heavyweight for development. x86_64 build time
>> with gcc 7:
>>
>> Server 1m32.484s
>> Server+Minimal 1m42.166s
>> Server+Minimal (--with-jvm-features="link-time-opt") 5m29.422s
>>
>> If the change to enable lto by default is proposed, what would be the
>> recommended strategy for development?
>>
>> For ARM32 Minimal, please keep in mind that it's not uncommon to disable
>> LTO plugin in commodity ARM32 gcc compiler distributions, so for some it
>> does not matter what settings we have in OpenJDK. I believe there could
>> be other reasons for that on top of build time (bugs?).
>>

From volker.simonis at gmail.com  Wed Jan 15 15:29:48 2020
From: volker.simonis at gmail.com (Volker Simonis)
Date: Wed, 15 Jan 2020 07:29:48 -0800
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <AM6PR02MB42007C34069D56491E86D8B893370@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
 <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>
 <249cba82-1469-03f8-d205-e5895eab2cc3@bell-sw.com>
 <AM6PR02MB4200457722633979CA9EC83093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <9908f346-f376-5bfc-8d5b-bc250e336032@bell-sw.com>
 <CA+3eh11mg+UgbwgF7qCHkq3hvr73H=3X4+sDTaZjdvZ3kTuejQ@mail.gmail.com>
 <AM6PR02MB42007C34069D56491E86D8B893370@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <CA+3eh12gmh2-xOhbPf9Gc=92mfWE1XcRJeUo32Aam01ipKx49w@mail.gmail.com>

Aleksei, Matthias,

thanks for the numbers. The size reduction on libjvm.so looks not bad,
indeed.

Do you know if newer versions of GCC use the gold linker by default? I
remember from some experiments which I did many years ago that gold was
considerably faster compared to the default ld linker.

Unfortunately, the documentation I found about LTO/ld/gold [1,2] seems to
be quite old and not very precise. Do you have gained any experience with
LTO/gold and know if gold could maybe improve linking times with LTO?

[1] https://gcc.gnu.org/wiki/LinkTimeOptimization
[2] https://stackoverflow.com/questions/31688069/requirements-to-use-flto


Baesken, Matthias <matthias.baesken at sap.com> schrieb am Mi., 15. Jan. 2020,
07:02:

> Hello , I can comment on   the  code size .  This is what I get when
> comparing  a build  without  and  with  -flto .
>
>
>
> gcc7 linux x86_64  product build, normal / with -flto
>
>
> ----------------------------------------------------------------------------------
>
>
>
> du -sh on the *.so files gives :
>
>
>
> 16K / 16K      ./lib/libattach.so
>
> 48K / 44K      ./lib/libawt_headless.so
>
> 752K / 760K    ./lib/libawt.so            <------------------ this one
> gets a bit larger with flto
>
> 472K / 456K    ./lib/libawt_xawt.so       <------------------ small gain
>
> 36K / 32K      ./lib/libdt_socket.so
>
> 16K /16K       ./lib/libextnet.so
>
> 1.5M / 824K    ./lib/libfontmanager.so     <------------------ HUGE gain
>
> 784K / 792K    ./lib/libfreetype.so        <------------------ this one
> gets a bit larger with flto
>
> 56K / 56K      ./lib/libinstrument.so
>
> 52K / 52K      ./lib/libj2gss.so
>
> 20K / 20K      ./lib/libj2pcsc.so
>
> 92K / 84K      ./lib/libj2pkcs11.so
>
> 12K / 12k      ./lib/libjaas.so
>
> 260K / 244K    ./lib/libjavajpeg.so         <----------------- small gain
>
> 196K / 188K    ./lib/libjava.so
>
> 12K / 12K      ./lib/libjawt.so
>
> 280K / 256K    ./lib/libjdwp.so             <----------------- small gain
>
> 144K / 140K    ./lib/libjimage.so
>
> 84K / 76K      ./lib/libjli.so
>
> 16K / 16K      ./lib/libjsig.so
>
> 88K / 80K      ./lib/libjsound.so
>
> 564K / 420K    ./lib/liblcms.so             <----------------- large gain
>
> 12K / 12K      ./lib/libmanagement_agent.so
>
> 40K / 36K      ./lib/libmanagement_ext.so
>
> 36K / 32K      ./lib/libmanagement.so
>
> 576K / 496K    ./lib/libmlib_image.so       <----------------- large gain
>
> 112K / 108K    ./lib/libnet.so
>
> 100K / 100K    ./lib/libnio.so
>
> 16K  / 16K     ./lib/libprefs.so
>
> 8.0K / 8.0K    ./lib/librmi.so
>
> 60K / 60K      ./lib/libsaproc.so
>
> 36K / 32K      ./lib/libsctp.so
>
> 368K / 212K    ./lib/libsplashscreen.so     <----------------- large gain
>
> 320K / 296K    ./lib/libsunec.so            <----------------- medium gain
>
> 72K / 72K      ./lib/libverify.so
>
> 44K / 44K      ./lib/libzip.so
>
> 16K / 16K      ./lib/server/libjsig.so
>
> 23M / 17M      ./lib/server/libjvm.so        <----------------- big gain
> maybe because it is C++ ?
>
>
>
>
>
> So  for  some libs  you see  10% and more , but not for all .   But  most
> large  libs  like   libjvm.so,  libfontmanager.so      or   liblcms.so
> we see good results regarding reduced code size.
>
>
>
> I Cannot say much about performance improvements , probably it would be
> small .
>
>
>
> For SPEC  you find something at
>
>
>
>
> http://hubicka.blogspot.com/2019/05/gcc-9-link-time-and-inter-procedural.html
>
>
>
> (not that these results would say too much about  JVM performance ).
>
>
>
>
>
> Best regards, Matthias
>
>
>
> *From:* Volker Simonis <volker.simonis at gmail.com>
> *Sent:* Mittwoch, 15. Januar 2020 14:40
> *To:* Aleksei Voitylov <aleksei.voitylov at bell-sw.com>
> *Cc:* Baesken, Matthias <matthias.baesken at sap.com>; Magnus Ihse Bursie <
> magnus.ihse.bursie at oracle.com>; serviceability-dev at openjdk.java.net;
> build-dev <build-dev at openjdk.java.net>; hotspot-dev at openjdk.java.net
> *Subject:* Re: serviceability agent : problems when using gcc LTO (link
> time optimization)
>
>
>
> While we are speaking about all the drawbacks of LTO, it's still not clear
> what the benefits are? In the very first mail Matthias mentioned that there
> might be performance improvements but that performance is not the main
> driving factor behind this initiative. So is it the reduced code size
> (Matthias mentioned something around ~10%)?
>
>
>
> It would be nice to see some real numbers on various platform for both,
> the performance improvements for native parts like JIT/GC as well as for
> the size reduction.
>
> Aleksei Voitylov <aleksei.voitylov at bell-sw.com> schrieb am Di., 14. Jan.
> 2020, 09:54:
>
>
> On 14/01/2020 19:57, Baesken, Matthias wrote:
> > Hello  Magnus and Aleksei,  thanks for the input .
> >
> > The times you  provided really look like they make a big difference  at
> least for people  often  building   minimal-vm  .
> > Guess I have to measure myself a bit  (maybe the difference is not that
> big on our linux s390x / ppc64(le) ) .
> >
> >> If the change to enable lto by default is proposed, what would be the
> >> recommended strategy for development?
> >>
> > Probably  we should a)   do not enable it by default but just make sure
> it can be enabled easily and works  for  the minimal-vm
> That would be welcome. I have high hopes to LTO the VM some time by
> default, and the tendency observed is that the compiler time overhead
> for GCC becomes smaller. At the same time there is no reason why vendors
> that invested in testing and can absorb the build time hit could provide
> binaries with LTO built VMs by passing an additional option flag.
> >   or  b)  take it easy to disable it for local development.
> >
> > Best regards, Matthias
> >
> >
> >
> >> Magnus, Matthias,
> >>
> >> for me, lto is a little heavyweight for development. x86_64 build time
> >> with gcc 7:
> >>
> >> Server 1m32.484s
> >> Server+Minimal 1m42.166s
> >> Server+Minimal (--with-jvm-features="link-time-opt") 5m29.422s
> >>
> >> If the change to enable lto by default is proposed, what would be the
> >> recommended strategy for development?
> >>
> >> For ARM32 Minimal, please keep in mind that it's not uncommon to disable
> >> LTO plugin in commodity ARM32 gcc compiler distributions, so for some it
> >> does not matter what settings we have in OpenJDK. I believe there could
> >> be other reasons for that on top of build time (bugs?).
> >>
>
>

From erik.joelsson at oracle.com  Wed Jan 15 15:47:57 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Wed, 15 Jan 2020 07:47:57 -0800
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <CA+3eh12gmh2-xOhbPf9Gc=92mfWE1XcRJeUo32Aam01ipKx49w@mail.gmail.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
 <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>
 <249cba82-1469-03f8-d205-e5895eab2cc3@bell-sw.com>
 <AM6PR02MB4200457722633979CA9EC83093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <9908f346-f376-5bfc-8d5b-bc250e336032@bell-sw.com>
 <CA+3eh11mg+UgbwgF7qCHkq3hvr73H=3X4+sDTaZjdvZ3kTuejQ@mail.gmail.com>
 <AM6PR02MB42007C34069D56491E86D8B893370@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <CA+3eh12gmh2-xOhbPf9Gc=92mfWE1XcRJeUo32Aam01ipKx49w@mail.gmail.com>
Message-ID: <824316f1-43f6-b61b-c764-ad0c0996b325@oracle.com>

On 2020-01-15 07:29, Volker Simonis wrote:
> Do you know if newer versions of GCC use the gold linker by default? I
> remember from some experiments which I did many years ago that gold was
> considerably faster compared to the default ld linker.

The default linker is system configured so it depends on your Linux 
distro. The devkits generated by the current devkit makefiles configures 
gold as default.

/Erik



From matthias.baesken at sap.com  Wed Jan 15 16:00:38 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 15 Jan 2020 16:00:38 +0000
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <824316f1-43f6-b61b-c764-ad0c0996b325@oracle.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
 <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>
 <249cba82-1469-03f8-d205-e5895eab2cc3@bell-sw.com>
 <AM6PR02MB4200457722633979CA9EC83093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <9908f346-f376-5bfc-8d5b-bc250e336032@bell-sw.com>
 <CA+3eh11mg+UgbwgF7qCHkq3hvr73H=3X4+sDTaZjdvZ3kTuejQ@mail.gmail.com>
 <AM6PR02MB42007C34069D56491E86D8B893370@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <CA+3eh12gmh2-xOhbPf9Gc=92mfWE1XcRJeUo32Aam01ipKx49w@mail.gmail.com>
 <824316f1-43f6-b61b-c764-ad0c0996b325@oracle.com>
Message-ID: <AM6PR02MB42005E8E0E9B8FB4C10991AB93370@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello, I  used  the  "normal"  linker so I think  what 

https://stackoverflow.com/questions/31688069/requirements-to-use-flto

says is true ,  one can use  also the "normal"  linker .
Haven't checked  for any performance  (or other) improvements   when using gold  instead .  



Best regards, Matthias


> On 2020-01-15 07:29, Volker Simonis wrote:
> > Do you know if newer versions of GCC use the gold linker by default? I
> > remember from some experiments which I did many years ago that gold
> was
> > considerably faster compared to the default ld linker.
> 
> The default linker is system configured so it depends on your Linux
> distro. The devkits generated by the current devkit makefiles configures
> gold as default.
> 
> /Erik
> 


From matthias.baesken at sap.com  Wed Jan 15 16:11:03 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 15 Jan 2020 16:11:03 +0000
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <CA+3eh12gmh2-xOhbPf9Gc=92mfWE1XcRJeUo32Aam01ipKx49w@mail.gmail.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
 <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>
 <249cba82-1469-03f8-d205-e5895eab2cc3@bell-sw.com>
 <AM6PR02MB4200457722633979CA9EC83093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <9908f346-f376-5bfc-8d5b-bc250e336032@bell-sw.com>
 <CA+3eh11mg+UgbwgF7qCHkq3hvr73H=3X4+sDTaZjdvZ3kTuejQ@mail.gmail.com>
 <AM6PR02MB42007C34069D56491E86D8B893370@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <CA+3eh12gmh2-xOhbPf9Gc=92mfWE1XcRJeUo32Aam01ipKx49w@mail.gmail.com>
Message-ID: <AM6PR02MB4200A3D1523DB8800DCC194893370@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello,    here is another  comparison for the larger  JDK  shared libs,  this time  with the sizes  of   build with linktime-gc (--gc-sections)   added .
( just for the larger libs )
(  I had not  enabled  linktime-gc  for libjvm   in our  test build , just for the JDK libs . )

Linuxx86_64 / gcc7

normal / with -flto / with linktime-gc (--gc-sections)
-----------------------------------------------------------
752K / 760K / 752K   ./lib/libawt.so            <------------------ this one gets a bit larger but only with flto
472K / 456K / 468K   ./lib/libawt_xawt.so       <------------------ small gain
1.5M / 824K / 900K   ./lib/libfontmanager.so     <------------------ HUGE gain , not as good with ltgc but still good
784K / 792K / 784K  ./lib/libfreetype.so        <------------------ this one gets a bit larger  (but not with ltgc)
260K / 244K / 252K   ./lib/libjavajpeg.so         <----------------- small gain
196K / 188K / 196K   ./lib/libjava.so
280K / 256K / 276K   ./lib/libjdwp.so             <----------------- small gain
144K / 140K / 136K   ./lib/libjimage.so
564K / 420K / 404K   ./lib/liblcms.so             <----------------- large gain ,  even better with  ltgc
576K / 496K / 556K   ./lib/libmlib_image.so       <----------------- large gain with flto , small one with ltgc
368K / 212K / 236K   ./lib/libsplashscreen.so     <----------------- large gain
320K / 296K / 300K   ./lib/libsunec.so            <----------------- medium gain
23M / 17M   /  --not enabled---    ./lib/server/libjvm.so        <----------------- big gain maybe because it is C++ ?


So   one can see,  that   flto   is usually  a bit better  than link-time-gc  when it comes to  improving lib sizes, but not always .
However  linktime-gc   seems to be faster when comparing build times   , I did not really notice much  build  time slowdown because of it .
( we have it enabled  for linux  s390x  for some time in OpenJDK ).
The  linktime-gc   also offers a nice feature  to print out the eliminated stuff ,   that can be used  to remove  unused code cross-platform .
e.g.  the removed symbols  from   https://bugs.openjdk.java.net/browse/JDK-8234629    has been found this way .


Best regards, Matthias



Aleksei, Matthias,

thanks for the numbers. The size reduction on libjvm.so looks not bad, indeed.

Do you know if newer versions of GCC use the gold linker by default? I remember from some experiments which I did many years ago that gold was considerably faster compared to the default ld linker.

Unfortunately, the documentation I found about LTO/ld/gold [1,2] seems to be quite old and not very precise. Do you have gained any experience with LTO/gold and know if gold could maybe improve linking times with LTO?

[1] https://gcc.gnu.org/wiki/LinkTimeOptimization
[2] https://stackoverflow.com/questions/31688069/requirements-to-use-flto


Baesken, Matthias <matthias.baesken at sap.com<mailto:matthias.baesken at sap.com>> schrieb am Mi., 15. Jan. 2020, 07:02:
Hello , I can comment on   the  code size .  This is what I get when comparing  a build  without  and  with  -flto .

gcc7 linux x86_64  product build, normal / with -flto
----------------------------------------------------------------------------------

du -sh on the *.so files gives :

16K / 16K      ./lib/libattach.so
48K / 44K      ./lib/libawt_headless.so
752K / 760K    ./lib/libawt.so            <------------------ this one gets a bit larger with flto
472K / 456K    ./lib/libawt_xawt.so       <------------------ small gain
36K / 32K      ./lib/libdt_socket.so
16K /16K       ./lib/libextnet.so
1.5M / 824K    ./lib/libfontmanager.so     <------------------ HUGE gain
784K / 792K    ./lib/libfreetype.so        <------------------ this one gets a bit larger with flto
56K / 56K      ./lib/libinstrument.so
52K / 52K      ./lib/libj2gss.so
20K / 20K      ./lib/libj2pcsc.so
92K / 84K      ./lib/libj2pkcs11.so
12K / 12k      ./lib/libjaas.so
260K / 244K    ./lib/libjavajpeg.so         <----------------- small gain
196K / 188K    ./lib/libjava.so
12K / 12K      ./lib/libjawt.so
280K / 256K    ./lib/libjdwp.so             <----------------- small gain
144K / 140K    ./lib/libjimage.so
84K / 76K      ./lib/libjli.so
16K / 16K      ./lib/libjsig.so
88K / 80K      ./lib/libjsound.so
564K / 420K    ./lib/liblcms.so             <----------------- large gain
12K / 12K      ./lib/libmanagement_agent.so
40K / 36K      ./lib/libmanagement_ext.so
36K / 32K      ./lib/libmanagement.so
576K / 496K    ./lib/libmlib_image.so       <----------------- large gain
112K / 108K    ./lib/libnet.so
100K / 100K    ./lib/libnio.so
16K  / 16K     ./lib/libprefs.so
8.0K / 8.0K    ./lib/librmi.so
60K / 60K      ./lib/libsaproc.so
36K / 32K      ./lib/libsctp.so
368K / 212K    ./lib/libsplashscreen.so     <----------------- large gain
320K / 296K    ./lib/libsunec.so            <----------------- medium gain
72K / 72K      ./lib/libverify.so
44K / 44K      ./lib/libzip.so
16K / 16K      ./lib/server/libjsig.so
23M / 17M      ./lib/server/libjvm.so        <----------------- big gain maybe because it is C++ ?


So  for  some libs  you see  10% and more , but not for all .   But  most  large  libs  like   libjvm.so,  libfontmanager.so      or   liblcms.so    we see good results regarding reduced code size.

I Cannot say much about performance improvements , probably it would be small .

For SPEC  you find something at

http://hubicka.blogspot.com/2019/05/gcc-9-link-time-and-inter-procedural.html

(not that these results would say too much about  JVM performance ).


Best regards, Matthias

From: Volker Simonis <volker.simonis at gmail.com<mailto:volker.simonis at gmail.com>>
Sent: Mittwoch, 15. Januar 2020 14:40
To: Aleksei Voitylov <aleksei.voitylov at bell-sw.com<mailto:aleksei.voitylov at bell-sw.com>>
Cc: Baesken, Matthias <matthias.baesken at sap.com<mailto:matthias.baesken at sap.com>>; Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com<mailto:magnus.ihse.bursie at oracle.com>>; serviceability-dev at openjdk.java.net<mailto:serviceability-dev at openjdk.java.net>; build-dev <build-dev at openjdk.java.net<mailto:build-dev at openjdk.java.net>>; hotspot-dev at openjdk.java.net<mailto:hotspot-dev at openjdk.java.net>
Subject: Re: serviceability agent : problems when using gcc LTO (link time optimization)

While we are speaking about all the drawbacks of LTO, it's still not clear what the benefits are? In the very first mail Matthias mentioned that there might be performance improvements but that performance is not the main driving factor behind this initiative. So is it the reduced code size (Matthias mentioned something around ~10%)?

It would be nice to see some real numbers on various platform for both, the performance improvements for native parts like JIT/GC as well as for the size reduction.
Aleksei Voitylov <aleksei.voitylov at bell-sw.com<mailto:aleksei.voitylov at bell-sw.com>> schrieb am Di., 14. Jan. 2020, 09:54:

On 14/01/2020 19:57, Baesken, Matthias wrote:
> Hello  Magnus and Aleksei,  thanks for the input .
>
> The times you  provided really look like they make a big difference  at least for people  often  building   minimal-vm  .
> Guess I have to measure myself a bit  (maybe the difference is not that big on our linux s390x / ppc64(le) ) .
>
>> If the change to enable lto by default is proposed, what would be the
>> recommended strategy for development?
>>
> Probably  we should a)   do not enable it by default but just make sure it can be enabled easily and works  for  the minimal-vm
That would be welcome. I have high hopes to LTO the VM some time by
default, and the tendency observed is that the compiler time overhead
for GCC becomes smaller. At the same time there is no reason why vendors
that invested in testing and can absorb the build time hit could provide
binaries with LTO built VMs by passing an additional option flag.
>   or  b)  take it easy to disable it for local development.
>
> Best regards, Matthias
>
>
>
>> Magnus, Matthias,
>>
>> for me, lto is a little heavyweight for development. x86_64 build time
>> with gcc 7:
>>
>> Server 1m32.484s
>> Server+Minimal 1m42.166s
>> Server+Minimal (--with-jvm-features="link-time-opt") 5m29.422s
>>
>> If the change to enable lto by default is proposed, what would be the
>> recommended strategy for development?
>>
>> For ARM32 Minimal, please keep in mind that it's not uncommon to disable
>> LTO plugin in commodity ARM32 gcc compiler distributions, so for some it
>> does not matter what settings we have in OpenJDK. I believe there could
>> be other reasons for that on top of build time (bugs?).
>>

From matthias.baesken at sap.com  Wed Jan 15 16:14:54 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 15 Jan 2020 16:14:54 +0000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <998c4a08-5670-f51d-4625-9c7e984b4b5d@oracle.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <b8f01c3f-27d1-f055-ba49-448a47ca4960@oracle.com>
 <AM6PR02MB420099D1C75E508D1BE1CD8D93370@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <998c4a08-5670-f51d-4625-9c7e984b4b5d@oracle.com>
Message-ID: <AM6PR02MB420070AEDB1159EC92DDE58493370@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hi Erik, I did not notice  slowdowns in our night makes .

Looking at a specific  test machine  I use  (x86_64,  build JOBS hardwired  set  to 12 )   I  get around  6 minutes build time  with and without the feature .

( but you have to take into account  that  the  link-time section-gc   on  x86_64    in my patch is only enabled  for the smaller JDK libs and not libjvm.so  )  

Best regards, Matthias


> 
> Given the discussion regarding lto on hotspot and the extreme increased
> build time, have you noticed any difference in build times with this patch?
> 
> /Erik
> 


From david.holmes at oracle.com  Wed Jan 15 22:13:29 2020
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 16 Jan 2020 08:13:29 +1000
Subject: [PING2!] RFR: 8230305: Cgroups v2: Container awareness
In-Reply-To: <52110EA5-26C9-43F2-8C2A-21D4E03ED3CC@oracle.com>
References: <072f66ee8c44034831b4e38f6470da4bff6edd07.camel@redhat.com>
 <7540a208e306ab957032b18178a53c6afa105d33.camel@redhat.com>
 <CC473B8C-E6CB-4293-BE01-BEDD09BEB358@oracle.com>
 <baa4c4821174dc0dfa43190d862d74b22fbf35d4.camel@redhat.com>
 <d504e04dd277ef46f42064d2bbd54b1ea94da0c2.camel@redhat.com>
 <5eec97c04d86562346243c1db3832e86e13697a1.camel@redhat.com>
 <b575b061cf4b7b0d9c0656aa1a726b2dd5a8a98c.camel@redhat.com>
 <52110EA5-26C9-43F2-8C2A-21D4E03ED3CC@oracle.com>
Message-ID: <fb194acf-43c3-e6a0-5f89-71b9cbea3b5c@oracle.com>

Hi Bob, Severin,

On 15/01/2020 6:04 am, Bob Vandette wrote:
> 
> Cgroup V2 is about to go mainstream this year for popular distros such as Oracle Linux 8, Redhat Linux 8 and Fedora
> so this fix it?s important to get into JDK 15 so we can start shaking out this support.
> 
> Please take a look and help get this change reviewed.

I've taken a look and the overall structure and approach seems fine. I 
can't attest to the details of cgroups v1 versus v2 but trust the exports.

Reviewed.

Please ensure all copyrights updated to 2020.

Bob: I assume you have tested this on our systems?

Thanks,
David

> Thanks,
> Bob Vandette
> 
> 
> 
>> On Nov 29, 2019, at 4:04 AM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
>>
>> On Fri, 2019-11-15 at 17:56 +0100, Severin Gehwolf wrote:
>>> On Fri, 2019-11-08 at 15:21 +0100, Severin Gehwolf wrote:
>>>> Hi Bob,
>>>>
>>>> On Wed, 2019-11-06 at 10:47 +0100, Severin Gehwolf wrote:
>>>>> On Tue, 2019-11-05 at 16:54 -0500, Bob Vandette wrote:
>>>>>> Severin,
>>>>>>
>>>>>> Thanks for taking on this cgroup v2 improvement.
>>>>>>
>>>>>> In general I like the implementation and the refactoring.  The CachedMetric class is nice.
>>>>>> We can add any metric we want to cache in a more general way.
>>>>>>
>>>>>> Is this the latest version of the webrev?
>>>>>>
>>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/cgroupsv2-hotspot/03/webrev/src/hotspot/os/linux/cgroupV2Subsystem_linux.cpp.html
>>>>>>
>>>>>> It looks like you need to add the caching support for active_processor_count (JDK-8227006).
>>>> [...]
>>>>> I'll do a proper rebase ASAP.
>>>>
>>>> Latest webrev:
>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/cgroupsv2-hotspot/05/webrev/
>>>>
>>>>>> I?m not sure it?s worth providing different strings for Unlimited versus Max or Scaled shares.
>>>>>> I?d just try to be compatible with the cgroupv2 output so you don?t have to change the test.
>>>>>
>>>>> OK. Will do.
>>>>
>>>> Unfortunately, there is no way of NOT changing TestCPUAwareness.java as
>>>> it expects CPU Shares to be written to the cgroup filesystem verbatim.
>>>> That's no longer the case for cgroups v2 (at least for crun). Either
>>>> way, most test changes are gone now.
>>>>
>>>>>> I wonder if it?s worth trying to synthesize memory_max_usage_in_bytes() by keeping the highest
>>>>>> value ever returned by the API.
>>>>>
>>>>> Interesting idea. I'll ponder this a bit and get back to you.
>>>>
>>>> This has been implemented. I'm not sure this is correct, though. It
>>>> merely piggy-backs on calls to memory_usage_in_bytes() and keeps the
>>>> high watermark value of that.
>>>>
>>>> Testing passed on F31 with cgroups v2 controllers properly configured
>>>> (podman) and hybrid (legacy hierarchy) with docker/podman.
>>>>
>>>> Thoughts?
>>>
>>> Ping?
>>
>> Anyone willing to review this? It would be nice to make some progress.
>>
>> Thanks,
>> Severin
>>
>>> Metrics work proposed for RFR here:
>>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2019-November/063464.html
>>>
>>> Thanks,
>>> Severin
>>
> 

From david.holmes at oracle.com  Wed Jan 15 22:25:09 2020
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 16 Jan 2020 08:25:09 +1000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>

Hi Matthias,

I have reservations about turning this on by default and with no way to 
control it. This seems like it should be something you have to opt-in to 
initially while we gain some experience with it and ensure there are no 
unexpected side-effects. After that it could be enabled by default.

David

On 15/01/2020 12:07 am, Baesken, Matthias wrote:
> Hello,  the following change enables the  link-time section-gc for linux .
> 
> gcc and ld support enabling "garbage collection" of unused input sections.
> This can be used to eliminate unused coding from native libraries (especially when already compiling the objects with compiler flags -ffunction-sections -fdata-sections .
> See for details the --gc-sections and --print-gc-sections parts of the ld documentation :
> 
> https://linux.die.net/man/1/ld
> 
> 
> We had this enabled already  for  linux s390x ,  with https://bugs.openjdk.java.net/browse/JDK-8234525
> 8234525: enable link-time section-gc for linux s390x to remove unused code .
> 
> This  time we enable it too for  the other linux platforms .
> 
> For the other platforms I do not enable it for JVM, just for the JDK libs.  The reason is that the serviceability agent  (not supported on linux s390x )    is not  (yet) ready for the optimization .
> Below you see the results , for some libraries  a significant  size reduction can be achieved .
> 
> 
> Results from linux x86_64 product builds :
> 
> without / with ltgc
> 
> 320K / 300K    /images/jdk/lib/libsunec.so       <-------------------------
> 36K  / 36K     /images/jdk/lib/libdt_socket.so
> 280K / 276K   /images/jdk/lib/libjdwp.so
> 23M  / 23M    /images/jdk/lib/server/libjvm.so    <---- not set for libjvm.so for x86_64
> 16K  / 16K    /images/jdk/lib/server/libjsig.so
> 72K  / 72M    /images/jdk/lib/libverify.so
> 84K  / 84M   /images/jdk/lib/libjli.so
> 16K  / 16K    /images/jdk/lib/libjsig.so
> 196K / 196K   /images/jdk/lib/libjava.so
> 44K  / 44K    /images/jdk/lib/libzip.so
> 144K / 136K   /images/jdk/lib/libjimage.so
> 112K / 112K   /images/jdk/lib/libnet.so
> 100K / 100K   /images/jdk/lib/libnio.so
> 36K  / 36K    /images/jdk/lib/libsctp.so
> 576K / 556K   /images/jdk/lib/libmlib_image.so
> 752K / 752K   /images/jdk/lib/libawt.so
> 260K / 252K   /images/jdk/lib/libjavajpeg.so
> 784K / 784K   /images/jdk/lib/libfreetype.so
> 368K / 236K /images/jdk/lib/libsplashscreen.so   <-------------------------
> 88K / 88K    /images/jdk/lib/libjsound.so
> 472K / 468K    /images/jdk/lib/libawt_xawt.so
> 564K / 404K   /images/jdk/lib/liblcms.so         <--------------------------
> 48K / 48K    /images/jdk/lib/libawt_headless.so
> 12K / 12K    /images/jdk/lib/libjawt.so
> 1.5M / 900K   /images/jdk/lib/libfontmanager.so  <------------------------------
> 12K / 12K    /images/jdk/lib/libjaas.so
> 92K / 92K    /images/jdk/lib/libj2pkcs11.so
> 16K / 16K    /images/jdk/lib/libattach.so
> 8.0K / 8.0K   /images/jdk/lib/librmi.so
> 56K / 56K    /images/jdk/lib/libinstrument.so
> 16K / 16K    /images/jdk/lib/libprefs.so
> 52K / 52K    /images/jdk/lib/libj2gss.so
> 12K / 12K    /images/jdk/lib/libmanagement_agent.so
> 36K / 32K    /images/jdk/lib/libmanagement.so
> 16K / 16K    /images/jdk/lib/libextnet.so
> 20K / 20K    /images/jdk/lib/libj2pcsc.so
> 40K / 40K    /images/jdk/lib/libmanagement_ext.so
> 60K / 60K    /images/jdk/lib/libsaproc.so
> 
> 
> Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8236714
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.2/
> 
> 
> Thanks, Matthias
> 

From david.holmes at oracle.com  Wed Jan 15 23:19:21 2020
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 16 Jan 2020 09:19:21 +1000
Subject: RFR 8235678: Remove unnecessary calls to Thread::current() in
 MutexLocker calls
In-Reply-To: <0758d1d1-b74b-31e6-82fb-a1223a3481f7@oracle.com>
References: <0a7cbbe4-3613-bd5a-962b-78dee06dde8a@oracle.com>
 <1fcf8439-2b5e-a776-9142-d9dc14a58faf@oracle.com>
 <0758d1d1-b74b-31e6-82fb-a1223a3481f7@oracle.com>
Message-ID: <b87a1d34-749b-5dba-ec8b-61cd812352de@oracle.com>

Hi Harold,

That all seems fine to me.

Thanks,
David

On 15/01/2020 10:57 pm, Harold Seigel wrote:
> Hi,
> 
> Please review this new webrev that also makes Thread* the first argument 
> to the relevant MutexLocker and MonitorLocker constructors as requested 
> by Coleen.
> 
> Updated Webrev: 
> http://cr.openjdk.java.net/~hseigel/bug_8235678.2/webrev/index.html
> 
> Thanks, Harold
> 
> On 1/14/2020 11:24 AM, coleen.phillimore at oracle.com wrote:
>>
>> Hi Harold,
>>
>> I really wanted this change to move Thread to the first argument like 
>> many of the other calls in the VM that take THREAD as an argument.
>>
>> Written like this:
>>
>> + MutexLocker mu(Threads_lock, THREAD);
>>
>>
>> It's too easy for someone who's cut/pasting to think the last THREAD 
>> argument should really be CHECK, which is completely wrong.
>>
>> Can you switch the arguments?
>>
>> Thanks,
>> Coleen
>>
>> On 1/14/20 9:00 AM, Harold Seigel wrote:
>>> Hi,
>>>
>>> Please review this small change, to reduce unnecessary calls to 
>>> Thread::current() in MutexLocker calls, by passing the current thread 
>>> as an argument.? A few ResoureMark declarations were also changed.
>>>
>>> Open Webrev: 
>>> http://cr.openjdk.java.net/~hseigel/bug_8235678/webrev/index.html
>>>
>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8235678
>>>
>>> The fix was regression tested by running Mach5 tiers 1 and 2 tests 
>>> and builds on Linux-x64, Solaris, Windows, and Mac OS X, by running 
>>> Mach5 tiers 3-5 tests on Linux-x64, and JCK lang and VM tests on 
>>> Linux-x64.
>>>
>>> Thanks, Harold
>>>
>>

From coleen.phillimore at oracle.com  Wed Jan 15 23:51:51 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Wed, 15 Jan 2020 18:51:51 -0500
Subject: RFR 8235678: Remove unnecessary calls to Thread::current() in
 MutexLocker calls
In-Reply-To: <0758d1d1-b74b-31e6-82fb-a1223a3481f7@oracle.com>
References: <0a7cbbe4-3613-bd5a-962b-78dee06dde8a@oracle.com>
 <1fcf8439-2b5e-a776-9142-d9dc14a58faf@oracle.com>
 <0758d1d1-b74b-31e6-82fb-a1223a3481f7@oracle.com>
Message-ID: <1f4d63f0-ac4f-7b4a-cad2-5f9d35803b4f@oracle.com>


This looks really good!? Thanks Harold.

I wonder if we see any reduction in instructions in startup, since some 
similar change did.? Maybe we can get Claes to measure it. :)

Thanks,
Coleen

On 1/15/20 7:57 AM, Harold Seigel wrote:
> Hi,
>
> Please review this new webrev that also makes Thread* the first 
> argument to the relevant MutexLocker and MonitorLocker constructors as 
> requested by Coleen.
>
> Updated Webrev: 
> http://cr.openjdk.java.net/~hseigel/bug_8235678.2/webrev/index.html
>
> Thanks, Harold
>
> On 1/14/2020 11:24 AM, coleen.phillimore at oracle.com wrote:
>>
>> Hi Harold,
>>
>> I really wanted this change to move Thread to the first argument like 
>> many of the other calls in the VM that take THREAD as an argument.
>>
>> Written like this:
>>
>> + MutexLocker mu(Threads_lock, THREAD);
>>
>>
>> It's too easy for someone who's cut/pasting to think the last THREAD 
>> argument should really be CHECK, which is completely wrong.
>>
>> Can you switch the arguments?
>>
>> Thanks,
>> Coleen
>>
>> On 1/14/20 9:00 AM, Harold Seigel wrote:
>>> Hi,
>>>
>>> Please review this small change, to reduce unnecessary calls to 
>>> Thread::current() in MutexLocker calls, by passing the current 
>>> thread as an argument.? A few ResoureMark declarations were also 
>>> changed.
>>>
>>> Open Webrev: 
>>> http://cr.openjdk.java.net/~hseigel/bug_8235678/webrev/index.html
>>>
>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8235678
>>>
>>> The fix was regression tested by running Mach5 tiers 1 and 2 tests 
>>> and builds on Linux-x64, Solaris, Windows, and Mac OS X, by running 
>>> Mach5 tiers 3-5 tests on Linux-x64, and JCK lang and VM tests on 
>>> Linux-x64.
>>>
>>> Thanks, Harold
>>>
>>


From david.holmes at oracle.com  Thu Jan 16 04:26:02 2020
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 16 Jan 2020 14:26:02 +1000
Subject: RFR (M): 8235741: Inappropriate uses of os::javaTimeMillis()
In-Reply-To: <a04e7b9f-3fe8-01ea-9784-fcf21613340d@oracle.com>
References: <f99194ad-3dad-48db-d8d4-de0cc9161701@oracle.com>
 <160821DF-BE15-435C-BDBE-AB553F794590@oracle.com>
 <a04e7b9f-3fe8-01ea-9784-fcf21613340d@oracle.com>
Message-ID: <3b9fd351-6222-b425-fada-382edc4799ff@oracle.com>

While awaiting further comments/feedback I've made some updates:

- added comments to existing use of javaTimeMillis() where it wasn't 
(fairly) obvious why it was being used
- fixed now incorrect comment pointed out by Kim
- copied os_per_linux.cpp changes to os_perf_aix.cpp (optimistically 
hoping for a thumbs up from Erik G. on this part :) ).

Full and incr webrevs at:

http://cr.openjdk.java.net/~dholmes/8235741/webrev.v2/
http://cr.openjdk.java.net/~dholmes/8235741/webrev.v2-incr/

Thanks,
David

On 15/01/2020 5:12 pm, David Holmes wrote:
> Hi Kim,
> 
> Thanks for taking a look at this.
> 
> On 15/01/2020 4:24 pm, Kim Barrett wrote:
>>> On Jan 13, 2020, at 2:13 AM, David Holmes <david.holmes at oracle.com> 
>>> wrote:
>>>
>>> webrev: http://cr.openjdk.java.net/~dholmes/8235741/webrev/
>>> bug: https://bugs.openjdk.java.net/browse/JDK-8235741
>>>
>>> Full details in the bug report about the existing uses of 
>>> javaTimeMillis(), many of which just want an elapsed time in ms and 
>>> so should be using javaTimeNanos() and convert to ms. This covers 
>>> areas all across the VM.
>>>
>>> Only non-simple change is in os_perf_linux.cpp (and the same code 
>>> will be in os_perf_aix.cpp once it has been validated). There we are 
>>> tracking an elapsed time in ms but relative to the boot time, which 
>>> is seconds since the epoch. Consequently the first interval has to be 
>>> calculated using javaTimeMillis, but after that we can use 
>>> javaTimeNanos (using a new 'first time' captured at the same time we 
>>> used javaTimeMillis). I think I have the logic right but other than 
>>> through JFR this code seems unused and I have limited means of 
>>> testing it. The JFR test 
>>> jdk/jfr/event/os/TestThreadContextSwitches.java exercises the code 
>>> but the results of running that test seems to exhibit arbitrary 
>>> randomness in the rates reported - e.g. 0 to 16000Hz - both with and 
>>> without my change, so not really that useful. Stefan K. suggested a 
>>> gtest which I may look into - though it is frustrating to have to 
>>> expend such effort to validate this.
>>>
>>> Other testing tiers 1-3.
>>>
>>> Thanks,
>>> David
>>
>> Thanks for the audit of uses of os::javaTimeMillis() in the bug report.
>> I wonder if some of that ought to be captured as comments in the
>> relevant code.? It's not always obvious to me that an external time
>> base is involved and thus making javaTimeMillis not a mistake.
> 
> Okay, I will add comments to the other uses of currentTimeMillis().
> 
>> There are a lot of places where conversions from nanoseconds to
>> milliseconds are being done to maintain existing units.? Some of those
>> places look like they could just as well be in nanoseconds.? But I can
>> see how changing the units for some of those could lead to a lot of
>> fannout, so okay.
> 
> Yes I tried to minimise the changes. In many cases a granularity of ms 
> seems somewhat arbitrary.
> 
>> ------------------------------------------------------------------------------ 
>>
>> src/hotspot/os/windows/os_perf_windows.cpp
>> ? 100?? s8???? lastUpdate; // Last time query was updated (current 
>> millis).
>> ...
>> ? 290?? const s8 now = os::javaTimeNanos();
>> ? 291?? if (NANOS_TO_MILLIS(now - update_query->lastUpdate) > 
>> min_update_interval_millis) {
>> ...
>> ? 295???? update_query->lastUpdate = now;
>>
>> now and update_query->lastUpdate are now in nanos, but comment for
>> lastUpdate still says it's in millis.? Looks like the comment needs
>> updating.
> 
> Yes - good catch.
> 
>> ------------------------------------------------------------------------------ 
>>
>> src/hotspot/share/utilities/globalDefinitions.hpp
>> ? 262 // time unit conversion macros
>> ? 263
>> ? 264 #define NANOS_TO_MILLIS(ns) ((ns) / NANOSECS_PER_MILLISEC)
>> ? 265 #define MILLIS_TO_NANOS(ms) ((ms) * NANOSECS_PER_MILLISEC)
>>
>> Why are these macros, rather than (template) functions?
> 
> Just because I just wanted a simple textual replacement to make it 
> clearer that I'm converting from millis to nanos or vice versa. I reach 
> for macros for such simple cases.
> 
>> Also, depending on the type and value of ms, MILLIS_TO_NANOS could
>> easily overflow, e.g. if ms type is a 32 bit type with a value of more
>> than ~4 seconds.? (I checked the two uses, and they happen to be okay.)
> 
> These are not trying to be mathematically sound. The conversion from 
> millis to nanos is used in two cases:
> 
> 1. Converting a current timestamp in ms to ns. Unless the current time 
> is set far in the future I don't think we have any issue with overflow 
> of? such a value.
> 
> 2. converting an elapsed time in ms to ns. These will be small values so 
> no overflow is possible.
> 
>> inline int64_t nanos_to_millis(int64_t ns) {
>> ?? return ns / NANOSECS_PER_MILLISECOND;
>> }
>>
>> inline int64_t millis_to_nanos(int64_t ms) {
>> ?? return ms * NANOSECONDS_PER_MILLISEC;
>> }
>>
>> Also, the names don't suggest time conversions, but potentially
>> arbitrary unit conversions, e.g. between something in NANOUNITS and
>> something in MILLIUNITS.
> 
> They don't have to be time conversions - the calculation is unit-less in 
> practice. The fact we have NANOSEC_PER_MILLISECOND et al is just an 
> artifact of introducing those values for timeout 
> calculations/conversions - it could just be NANOS_PER_MILLI etc
> 
>> ------------------------------------------------------------------------------ 
>>
>> Regarding this from the audit:
>>
>> --- begin ---
>> ./share/gc/parallel/psParallelCompact.cpp: // os::javaTimeMillis() 
>> does not guarantee monotonicity.
>> ...
>> ./share/gc/shared/referenceProcessor.cpp: // os::javaTimeMillis() does 
>> not guarantee monotonicity.
>>
>> These are all describing why the subsequent code uses javaTimeNanos 
>> not javaTimeMillis.
>> --- end ---
>>
>> Do we really still support platforms that don't have a monotonic
>> clock?? I guess we appear to at least try.? But it's really wrong that
>> callers of os::javaTimeNanos should even think they need to cope with
>> that function being non-monotonic.
>>
>> Hm, I always thought System.nanoTime() was a monotonic clock, but I
>> don't see any such guarantee. So I guess Java just doesn't have such a
>> thing. Wow!
>>
>> So I guess none of this is really relevant to the change at hand after 
>> all.
> 
> I think you read the comments the wrong way round. The code uses 
> javaTimeNanos not javaTimeMillis because javaTimeMillis is not monotonic 
> and the code wants a monotonic clock. These comments were mostly 
> inserted when the incorrect use of javaTimeMillis was replaced with 
> javaTimeNanos.
> 
> Thanks,
> David
> -----
> 
>> ------------------------------------------------------------------------------ 
>>
>>
>>

From david.holmes at oracle.com  Thu Jan 16 05:57:12 2020
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 16 Jan 2020 15:57:12 +1000
Subject: RFR: 8235966: Process obsolete flags less aggressively
In-Reply-To: <b8e70c4d-af76-a682-b4a9-839dd2671958@oracle.com>
References: <db63e5c9-a0c1-1575-e8dc-88429d6216fe@oracle.com>
 <674e4f0d-563f-3bb4-dee0-74aaacb9b270@oracle.com>
 <b32a2e50-62e3-ed94-7773-a6594a76e1aa@oracle.com>
 <c1a2f1e1-e86b-1fda-4632-ce9734e6e75f@oracle.com>
 <b8e70c4d-af76-a682-b4a9-839dd2671958@oracle.com>
Message-ID: <6a3de8da-746d-7e5d-9a0e-2157a0963ed8@oracle.com>

Getting back to this ...

Please see updated webrev at:

http://cr.openjdk.java.net/~dholmes/8235966/webrev/

Apologies as I mistakenly overwrote the original instead of creating v3.

This version expands on the original proposal by uncommenting the 
warnings about obsolete/expired flags that have not been removed from 
globals*.hpp, so that we don't forget to do this work. However these 
warnings are only enabled from build 20. I used 20 as being approx 2/3 
through the release cycle - long enough that the work should have been 
performed by then, whilst still leaving time to perform the work before 
RDP2. Of course we can tweak this number if there are issues with that 
choice.

Thanks,
David

On 20/12/2019 8:20 am, David Holmes wrote:
> Thanks Dan.
> 
> FTR I've updated the bug report with an extension to this proposal, 
> which is to add back the flag table validation checks to use via a gtest 
> that we only enable after a certain build in the release cycle (it 
> always passes before then). That way we avoid the problems I've outlined 
> with the initial version bump but also have a safety net in place to 
> ensure we don't forget to actually obsolete/expire flags.
> 
> Cheers,
> David
> 
> On 19/12/2019 3:37 am, Daniel D. Daugherty wrote:
>> Hi David,
>>
>> On 12/17/19 5:03 PM, David Holmes wrote:
>>> Hi Dan,
>>>
>>> Thanks for taking a look. Updated webrev:
>>>
>>> http://cr.openjdk.java.net/~dholmes/8235966/webrev.v2/
>>
>> src/hotspot/share/runtime/arguments.cpp
>> ???? I like the updates to header comment for verify_special_jvm_flags().
>>
>> Thumbs up.
>>
>>
>>>
>>> Discussion below.
>>
>> Replies below.
>>
>>
>>>
>>> On 18/12/2019 1:47 am, Daniel D. Daugherty wrote:
>>>> On 12/16/19 12:16 AM, David Holmes wrote:
>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8235966
>>>>> webrev: http://cr.openjdk.java.net/~dholmes/8235966/webrev/
>>>>
>>>> src/hotspot/share/runtime/arguments.cpp
>>>> ???? L745: ????? // if flag has become obsolete it should not have a 
>>>> "globals" flag defined anymore.
>>>> ???? L746: ????? if (!version_less_than(JDK_Version::current(), 
>>>> flag.obsolete_in)) {
>>>> ???? L747: ??????? if (JVMFlag::find_declared_flag(flag.name) != 
>>>> NULL) {
>>>> ???? L748: ????????? // Temporarily disable the warning: 8196739
>>>> ???? L749: ????????? // warning("Global variable for obsolete 
>>>> special flag entry \"%s\" should be removed", flag.name);
>>>> ???? L750: ??????? }
>>>> ???? L751: ????? }
>>>> ???????? It seems like we've been down a similar road before:
>>>>
>>>> ???????? JDK-8196739 Disable obsolete/expired VM flag transitional 
>>>> warnings
>>>> ???????? https://bugs.openjdk.java.net/browse/JDK-8196739
>>>>
>>>> ???????? This one may ring a bell... Fixed by dholmes in 
>>>> jdk11-b01... :-)
>>>>
>>>> ???????? And this followup sub-task to re-enable that warning:
>>>>
>>>> ???????? JDK-8196741 Re-enable obsolete/expired VM flag transitional 
>>>> warnings
>>>> ???????? https://bugs.openjdk.java.net/browse/JDK-8196741
>>>>
>>>> ???????? was closed as "Won't fix" on 2019.08.02.
>>>>
>>>> ???????? So the obvious questions:
>>>>
>>>> ???????? - Why is the new warning less problematic to tests that don't
>>>> ?????????? tolerate unexpected output?
>>>
>>> Two different situations. The commented out warning happens 
>>> unconditionally when you run the VM and it finds any flag marked 
>>> obsolete that hasn't been removed. Hence every single test will 
>>> encounter this warning.
>>
>> Ouch on such verbosity.
>>
>>
>>> The situation I am modifying is when a test uses a flag that is 
>>> marked for obsoletion. In the majority of cases the flag is already 
>>> deprecated and so already issuing a deprecation warning that the test 
>>> has to handle. Without my change there would still be an obsoletion 
>>> warning, so this test is in for a warning no matter what.
>>
>> Good that your change only comes into play when the flag is used.
>>
>>
>>> Also note that for hotspot at least we have strived to make tests 
>>> tolerate unexpected output. The reason JDK-8196741 was closed as 
>>> "won't fix" was because other areas wouldn't commit to doing that.
>>
>> Yup. Got it.
>>
>>
>>>
>>>> ???????? - If you move forward with this fix, then I think think code
>>>> ?????????? block needs to be removed or modified or am I missing 
>>>> something?
>>>
>>> I've rewritten the comment at the head of verify_special_jvm_flags to 
>>> explain why we can't issue a warning, and have deleted the block.
>>
>> Thanks for deleting the stale code.
>>
>>
>>>
>>>> ???????? There's a similar commented out check on L757-L765, but 
>>>> that one
>>>> ???????? is for an expired flag... You might want to adjust/delete 
>>>> it also?
>>>
>>> Deleted.
>>
>> Thanks.
>>
>>
>>>
>>>> ???? L753: ??????? warning("Special flag entry \"%s\" must be 
>>>> explicitly obsoleted before expired.", flag.name);
>>>> ???? L754: ??????? success = false;
>>>> ???????? nit - s/before expired/before being expired/
>>>> ???????? Update: I now see that "style" is in several places in this
>>>> ???????????? function. I'm not sure what to think here... it grates,
>>>> ?? ? ? ? ? ? but I can live with it.
>>>>
>>>> ???????? nit - L75[34] indented too much by two spaces.
>>>
>>> Fixed.
>>>
>>>> ???? L962: ????????? return real_name;
>>>> ???????? nit - indented too much by two spaces.
>>>
>>> Fixed.
>>>
>>>>
>>>> Trying to understand the modified logic in argument processing is
>>>> making my head spin...
>>>
>>> Mine too. It took a few attempts to put the logic in the right place 
>>> and make adjustments so that it all works as expected for a correctly 
>>> specified flag and an erroneous one.
>>
>> I keep trying to convince myself that we're improving this flag and
>> options code with each release... :-)
>>
>>
>>>
>>>> - You've added a call to is_obsolete_flag() in
>>>> ?? handle_aliases_and_deprecation() and is_obsolete_flag()
>>>> ?? is where the new warning is output:
>>>>
>>>> ???? warning("Temporarily processing option %s; support is scheduled 
>>>> for removal in %s"
>>>>
>>>> ?? handle_aliases_and_deprecation() is called from six different 
>>>> places,
>>>> ?? but the call sites are different based on the argument pattern so I
>>>> ?? have (mostly) convinced myself that there should not be any 
>>>> duplicate
>>>> ?? warning lines.
>>>
>>> Right - handle_aliases_and_deprecation is only called for a 
>>> syntactically correct flag based on those patterns. It normally 
>>> filters out obsoleted/expired flags and lets them fall through to 
>>> later error processing (in process_argument after parse_arg returns 
>>> false). That error processing is where the normal obsoletion check is 
>>> performed. So I had to not filter the flag in 
>>> handle_aliases_and_deprecation in this case, but still produce the 
>>> warning for a malformed flag. E.g.
>>>
>>> java -XX:+UseParallelOldGC -version
>>> Java HotSpot(TM) 64-Bit Server VM warning: Temporarily processing 
>>> option UseParallelOldGC; support is scheduled for removal in 15.0
>>> java version "15-internal" 2020-09-15
>>>
>>> java -XX:UseParallelOldGC -version
>>> Java HotSpot(TM) 64-Bit Server VM warning: Temporarily processing 
>>> option UseParallelOldGC; support is scheduled for removal in 15.0
>>> Missing +/- setting for VM option 'UseParallelOldGC'
>>> Error: Could not create the Java Virtual Machine.
>>
>> Thanks for the example. That helps a lot.
>>
>>
>>>
>>>> So I now understand the new logic that allows an obsoleted option
>>>> to be specified with a warning as long as the option still exists.
>>>> I'm good with the technical change, but...
>>>>
>>>> I'm worried about tests that don't tolerate the new warning mesg,
>>>> i.e., why wouldn't this become an issue again:
>>>>
>>>> JDK-8196739 Disable obsolete/expired VM flag transitional warnings
>>>
>>> Explained above.
>>
>> Yup and thanks.
>>
>> Dan
>>
>>
>>>
>>> Thanks,
>>> David
>>>
>>>> Dan
>>>>
>>>>
>>>>>
>>>>> When a flag is marked as obsolete in the special-flags table we 
>>>>> will ignore it and issue a warning that it is being ignored, as 
>>>>> soon as we bump the version of the JDK. That means that any tests 
>>>>> still using the obsolete flag may start to fail, leading to a surge 
>>>>> of test failures at the start of a release cycle. For example for 
>>>>> JDK 15 we have a whole bunch of JFR tests that fail because they 
>>>>> still try to work with UseParallelOldGC. In another case 
>>>>> runtime/cds/appcds/FieldLayoutFlags.java passes only be accident.
>>>>>
>>>>> When a flag is marked as obsolete for a release, all code involving 
>>>>> that flag (including tests that use it) must be updated within that 
>>>>> release and the flag itself removed. Whilst this is typically 
>>>>> scheduled early in a release cycle it isn't reasonable to expect it 
>>>>> to all occur within the first couple of days of the release cycle, 
>>>>> nor do we want to have to ProblemList a bunch of tests when they 
>>>>> start failing.
>>>>>
>>>>> What I propose is to instead allow an obsolete flag to continue to 
>>>>> be processed as long as that code removal has not actually occurred 
>>>>> - with an adjusted warning. The change I propose:
>>>>>
>>>>> - only treats an obsolete flag as obsolete if the flag cannot be found
>>>>> - added a new flag verification rule that disallows obsoletion in 
>>>>> an undefined version, but expiration in a specific version i.e. we 
>>>>> must always explicitly obsolete a flag before we expire it.
>>>>>
>>>>> The only downside here is that if we actually forget to file an 
>>>>> issue for the actual obsoletion work we may not notice via testing. 
>>>>> Of course whenever a change is made to the flags table to add an 
>>>>> entry then the issue to do the obsoletion should be filed at the 
>>>>> same time.
>>>>>
>>>>> Thanks,
>>>>> David
>>>>> -----
>>>>>
>>>>
>>

From matthias.baesken at sap.com  Thu Jan 16 08:10:30 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Thu, 16 Jan 2020 08:10:30 +0000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
Message-ID: <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hi David, sure we can introduce a way to switch this on/off.
There is already something similar for the link-time optimization (flto) , see the feature 

JvmFeatures.gmk
180 ifeq ($(call check-jvm-feature, link-time-opt), true)
190 ifeq ($(call check-jvm-feature, link-time-opt), false)

hotspot.m4
29 static-build link-time-opt aot jfr"
502 JVM_FEATURES_link_time_opt="link-time-opt"


Should we have  "link-time-gc"  additionally to   " link-time-opt"  ?   (however it  would  be a  bit misleading   that it is a "JVM" feature , but except linux s390x  it is only changing the build of the JDK libs) .

Best regards, Matthias



> 
> Hi Matthias,
> 
> I have reservations about turning this on by default and with no way to
> control it. This seems like it should be something you have to opt-in to
> initially while we gain some experience with it and ensure there are no
> unexpected side-effects. After that it could be enabled by default.
> 


From sgehwolf at redhat.com  Thu Jan 16 09:19:36 2020
From: sgehwolf at redhat.com (Severin Gehwolf)
Date: Thu, 16 Jan 2020 10:19:36 +0100
Subject: [PING2!] RFR: 8230305: Cgroups v2: Container awareness
In-Reply-To: <fb194acf-43c3-e6a0-5f89-71b9cbea3b5c@oracle.com>
References: <072f66ee8c44034831b4e38f6470da4bff6edd07.camel@redhat.com>
 <7540a208e306ab957032b18178a53c6afa105d33.camel@redhat.com>
 <CC473B8C-E6CB-4293-BE01-BEDD09BEB358@oracle.com>
 <baa4c4821174dc0dfa43190d862d74b22fbf35d4.camel@redhat.com>
 <d504e04dd277ef46f42064d2bbd54b1ea94da0c2.camel@redhat.com>
 <5eec97c04d86562346243c1db3832e86e13697a1.camel@redhat.com>
 <b575b061cf4b7b0d9c0656aa1a726b2dd5a8a98c.camel@redhat.com>
 <52110EA5-26C9-43F2-8C2A-21D4E03ED3CC@oracle.com>
 <fb194acf-43c3-e6a0-5f89-71b9cbea3b5c@oracle.com>
Message-ID: <5fdc573eab5d2368e23fff99b181d3d10f51edcd.camel@redhat.com>

Hi David,

On Thu, 2020-01-16 at 08:13 +1000, David Holmes wrote:
> Hi Bob, Severin,
> 
> On 15/01/2020 6:04 am, Bob Vandette wrote:
> > Cgroup V2 is about to go mainstream this year for popular distros such as Oracle Linux 8, Redhat Linux 8 and Fedora
> > so this fix it?s important to get into JDK 15 so we can start shaking out this support.
> > 
> > Please take a look and help get this change reviewed.
> 
> I've taken a look and the overall structure and approach seems fine. I 
> can't attest to the details of cgroups v1 versus v2 but trust the exports.
> 
> Reviewed.

Thanks for the review!

> Please ensure all copyrights updated to 2020.

Yes, I'll update copyrights, remove the synthesized memory_max_usage
(to preserve symmetry with Java changes), run through jdk/submit, and
my local testing (cgroup v2 and cgroup v1) and push.

Thanks,
Severin


From david.holmes at oracle.com  Thu Jan 16 09:30:38 2020
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 16 Jan 2020 19:30:38 +1000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>

Hi Matthias,

On 16/01/2020 6:10 pm, Baesken, Matthias wrote:
> Hi David, sure we can introduce a way to switch this on/off.

Thanks.

> There is already something similar for the link-time optimization (flto) , see the feature
> 
> JvmFeatures.gmk
> 180 ifeq ($(call check-jvm-feature, link-time-opt), true)
> 190 ifeq ($(call check-jvm-feature, link-time-opt), false)
> 
> hotspot.m4
> 29 static-build link-time-opt aot jfr"
> 502 JVM_FEATURES_link_time_opt="link-time-opt"

Yep familiar with that from Minimal VM and SE Embedded days :)

> Should we have  "link-time-gc"  additionally to   " link-time-opt"  ?   (however it  would  be a  bit misleading   that it is a "JVM" feature , but except linux s390x  it is only changing the build of the JDK libs) .

I agree the definition of this as a "JVM" feature is a bit 
odd/misleading. Perhaps the build folk have a suggestion on how to 
refactor this kind of option into something more general? In the 
meantime having link-time-gc sit alongside link-time-opt seems 
acceptable to me.

Thanks,
David

> Best regards, Matthias
> 
> 
> 
>>
>> Hi Matthias,
>>
>> I have reservations about turning this on by default and with no way to
>> control it. This seems like it should be something you have to opt-in to
>> initially while we gain some experience with it and ensure there are no
>> unexpected side-effects. After that it could be enabled by default.
>>
> 

From erik.gahlin at oracle.com  Thu Jan 16 11:54:18 2020
From: erik.gahlin at oracle.com (Erik Gahlin)
Date: Thu, 16 Jan 2020 12:54:18 +0100
Subject: RFR (M): 8235741: Inappropriate uses of os::javaTimeMillis()
In-Reply-To: <3b9fd351-6222-b425-fada-382edc4799ff@oracle.com>
References: <f99194ad-3dad-48db-d8d4-de0cc9161701@oracle.com>
 <160821DF-BE15-435C-BDBE-AB553F794590@oracle.com>
 <a04e7b9f-3fe8-01ea-9784-fcf21613340d@oracle.com>
 <3b9fd351-6222-b425-fada-382edc4799ff@oracle.com>
Message-ID: <0af8dd6d-f84b-7d0a-a99e-ea75e269398e@oracle.com>

HI David,

I'm reasonable convinced the JFR code works as before. We should 
probably rewrite the code, but it's outside the scope of this bug.

Thanks for fixing
Erik

On 2020-01-16 05:26, David Holmes wrote:
> While awaiting further comments/feedback I've made some updates:
>
> - added comments to existing use of javaTimeMillis() where it wasn't 
> (fairly) obvious why it was being used
> - fixed now incorrect comment pointed out by Kim
> - copied os_per_linux.cpp changes to os_perf_aix.cpp (optimistically 
> hoping for a thumbs up from Erik G. on this part :) ).
>
> Full and incr webrevs at:
>
> http://cr.openjdk.java.net/~dholmes/8235741/webrev.v2/
> http://cr.openjdk.java.net/~dholmes/8235741/webrev.v2-incr/
>
> Thanks,
> David
>
> On 15/01/2020 5:12 pm, David Holmes wrote:
>> Hi Kim,
>>
>> Thanks for taking a look at this.
>>
>> On 15/01/2020 4:24 pm, Kim Barrett wrote:
>>>> On Jan 13, 2020, at 2:13 AM, David Holmes <david.holmes at oracle.com> 
>>>> wrote:
>>>>
>>>> webrev: http://cr.openjdk.java.net/~dholmes/8235741/webrev/
>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8235741
>>>>
>>>> Full details in the bug report about the existing uses of 
>>>> javaTimeMillis(), many of which just want an elapsed time in ms and 
>>>> so should be using javaTimeNanos() and convert to ms. This covers 
>>>> areas all across the VM.
>>>>
>>>> Only non-simple change is in os_perf_linux.cpp (and the same code 
>>>> will be in os_perf_aix.cpp once it has been validated). There we 
>>>> are tracking an elapsed time in ms but relative to the boot time, 
>>>> which is seconds since the epoch. Consequently the first interval 
>>>> has to be calculated using javaTimeMillis, but after that we can 
>>>> use javaTimeNanos (using a new 'first time' captured at the same 
>>>> time we used javaTimeMillis). I think I have the logic right but 
>>>> other than through JFR this code seems unused and I have limited 
>>>> means of testing it. The JFR test 
>>>> jdk/jfr/event/os/TestThreadContextSwitches.java exercises the code 
>>>> but the results of running that test seems to exhibit arbitrary 
>>>> randomness in the rates reported - e.g. 0 to 16000Hz - both with 
>>>> and without my change, so not really that useful. Stefan K. 
>>>> suggested a gtest which I may look into - though it is frustrating 
>>>> to have to expend such effort to validate this.
>>>>
>>>> Other testing tiers 1-3.
>>>>
>>>> Thanks,
>>>> David
>>>
>>> Thanks for the audit of uses of os::javaTimeMillis() in the bug report.
>>> I wonder if some of that ought to be captured as comments in the
>>> relevant code.? It's not always obvious to me that an external time
>>> base is involved and thus making javaTimeMillis not a mistake.
>>
>> Okay, I will add comments to the other uses of currentTimeMillis().
>>
>>> There are a lot of places where conversions from nanoseconds to
>>> milliseconds are being done to maintain existing units.? Some of those
>>> places look like they could just as well be in nanoseconds. But I can
>>> see how changing the units for some of those could lead to a lot of
>>> fannout, so okay.
>>
>> Yes I tried to minimise the changes. In many cases a granularity of 
>> ms seems somewhat arbitrary.
>>
>>> ------------------------------------------------------------------------------ 
>>>
>>> src/hotspot/os/windows/os_perf_windows.cpp
>>> ? 100?? s8???? lastUpdate; // Last time query was updated (current 
>>> millis).
>>> ...
>>> ? 290?? const s8 now = os::javaTimeNanos();
>>> ? 291?? if (NANOS_TO_MILLIS(now - update_query->lastUpdate) > 
>>> min_update_interval_millis) {
>>> ...
>>> ? 295???? update_query->lastUpdate = now;
>>>
>>> now and update_query->lastUpdate are now in nanos, but comment for
>>> lastUpdate still says it's in millis.? Looks like the comment needs
>>> updating.
>>
>> Yes - good catch.
>>
>>> ------------------------------------------------------------------------------ 
>>>
>>> src/hotspot/share/utilities/globalDefinitions.hpp
>>> ? 262 // time unit conversion macros
>>> ? 263
>>> ? 264 #define NANOS_TO_MILLIS(ns) ((ns) / NANOSECS_PER_MILLISEC)
>>> ? 265 #define MILLIS_TO_NANOS(ms) ((ms) * NANOSECS_PER_MILLISEC)
>>>
>>> Why are these macros, rather than (template) functions?
>>
>> Just because I just wanted a simple textual replacement to make it 
>> clearer that I'm converting from millis to nanos or vice versa. I 
>> reach for macros for such simple cases.
>>
>>> Also, depending on the type and value of ms, MILLIS_TO_NANOS could
>>> easily overflow, e.g. if ms type is a 32 bit type with a value of more
>>> than ~4 seconds.? (I checked the two uses, and they happen to be okay.)
>>
>> These are not trying to be mathematically sound. The conversion from 
>> millis to nanos is used in two cases:
>>
>> 1. Converting a current timestamp in ms to ns. Unless the current 
>> time is set far in the future I don't think we have any issue with 
>> overflow of? such a value.
>>
>> 2. converting an elapsed time in ms to ns. These will be small values 
>> so no overflow is possible.
>>
>>> inline int64_t nanos_to_millis(int64_t ns) {
>>> ?? return ns / NANOSECS_PER_MILLISECOND;
>>> }
>>>
>>> inline int64_t millis_to_nanos(int64_t ms) {
>>> ?? return ms * NANOSECONDS_PER_MILLISEC;
>>> }
>>>
>>> Also, the names don't suggest time conversions, but potentially
>>> arbitrary unit conversions, e.g. between something in NANOUNITS and
>>> something in MILLIUNITS.
>>
>> They don't have to be time conversions - the calculation is unit-less 
>> in practice. The fact we have NANOSEC_PER_MILLISECOND et al is just 
>> an artifact of introducing those values for timeout 
>> calculations/conversions - it could just be NANOS_PER_MILLI etc
>>
>>> ------------------------------------------------------------------------------ 
>>>
>>> Regarding this from the audit:
>>>
>>> --- begin ---
>>> ./share/gc/parallel/psParallelCompact.cpp: // os::javaTimeMillis() 
>>> does not guarantee monotonicity.
>>> ...
>>> ./share/gc/shared/referenceProcessor.cpp: // os::javaTimeMillis() 
>>> does not guarantee monotonicity.
>>>
>>> These are all describing why the subsequent code uses javaTimeNanos 
>>> not javaTimeMillis.
>>> --- end ---
>>>
>>> Do we really still support platforms that don't have a monotonic
>>> clock?? I guess we appear to at least try.? But it's really wrong that
>>> callers of os::javaTimeNanos should even think they need to cope with
>>> that function being non-monotonic.
>>>
>>> Hm, I always thought System.nanoTime() was a monotonic clock, but I
>>> don't see any such guarantee. So I guess Java just doesn't have such a
>>> thing. Wow!
>>>
>>> So I guess none of this is really relevant to the change at hand 
>>> after all.
>>
>> I think you read the comments the wrong way round. The code uses 
>> javaTimeNanos not javaTimeMillis because javaTimeMillis is not 
>> monotonic and the code wants a monotonic clock. These comments were 
>> mostly inserted when the incorrect use of javaTimeMillis was replaced 
>> with javaTimeNanos.
>>
>> Thanks,
>> David
>> -----
>>
>>> ------------------------------------------------------------------------------ 
>>>
>>>
>>>

From harold.seigel at oracle.com  Thu Jan 16 13:35:30 2020
From: harold.seigel at oracle.com (Harold Seigel)
Date: Thu, 16 Jan 2020 08:35:30 -0500
Subject: RFR 8235678: Remove unnecessary calls to Thread::current() in
 MutexLocker calls
In-Reply-To: <b87a1d34-749b-5dba-ec8b-61cd812352de@oracle.com>
References: <0a7cbbe4-3613-bd5a-962b-78dee06dde8a@oracle.com>
 <1fcf8439-2b5e-a776-9142-d9dc14a58faf@oracle.com>
 <0758d1d1-b74b-31e6-82fb-a1223a3481f7@oracle.com>
 <b87a1d34-749b-5dba-ec8b-61cd812352de@oracle.com>
Message-ID: <807a034c-41ed-82af-1bba-f120245b1ce1@oracle.com>

Thanks David!

Harold

On 1/15/2020 6:19 PM, David Holmes wrote:
> Hi Harold,
>
> That all seems fine to me.
>
> Thanks,
> David
>
> On 15/01/2020 10:57 pm, Harold Seigel wrote:
>> Hi,
>>
>> Please review this new webrev that also makes Thread* the first 
>> argument to the relevant MutexLocker and MonitorLocker constructors 
>> as requested by Coleen.
>>
>> Updated Webrev: 
>> http://cr.openjdk.java.net/~hseigel/bug_8235678.2/webrev/index.html
>>
>> Thanks, Harold
>>
>> On 1/14/2020 11:24 AM, coleen.phillimore at oracle.com wrote:
>>>
>>> Hi Harold,
>>>
>>> I really wanted this change to move Thread to the first argument 
>>> like many of the other calls in the VM that take THREAD as an argument.
>>>
>>> Written like this:
>>>
>>> + MutexLocker mu(Threads_lock, THREAD);
>>>
>>>
>>> It's too easy for someone who's cut/pasting to think the last THREAD 
>>> argument should really be CHECK, which is completely wrong.
>>>
>>> Can you switch the arguments?
>>>
>>> Thanks,
>>> Coleen
>>>
>>> On 1/14/20 9:00 AM, Harold Seigel wrote:
>>>> Hi,
>>>>
>>>> Please review this small change, to reduce unnecessary calls to 
>>>> Thread::current() in MutexLocker calls, by passing the current 
>>>> thread as an argument.? A few ResoureMark declarations were also 
>>>> changed.
>>>>
>>>> Open Webrev: 
>>>> http://cr.openjdk.java.net/~hseigel/bug_8235678/webrev/index.html
>>>>
>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8235678
>>>>
>>>> The fix was regression tested by running Mach5 tiers 1 and 2 tests 
>>>> and builds on Linux-x64, Solaris, Windows, and Mac OS X, by running 
>>>> Mach5 tiers 3-5 tests on Linux-x64, and JCK lang and VM tests on 
>>>> Linux-x64.
>>>>
>>>> Thanks, Harold
>>>>
>>>

From harold.seigel at oracle.com  Thu Jan 16 13:36:16 2020
From: harold.seigel at oracle.com (Harold Seigel)
Date: Thu, 16 Jan 2020 08:36:16 -0500
Subject: RFR 8235678: Remove unnecessary calls to Thread::current() in
 MutexLocker calls
In-Reply-To: <1f4d63f0-ac4f-7b4a-cad2-5f9d35803b4f@oracle.com>
References: <0a7cbbe4-3613-bd5a-962b-78dee06dde8a@oracle.com>
 <1fcf8439-2b5e-a776-9142-d9dc14a58faf@oracle.com>
 <0758d1d1-b74b-31e6-82fb-a1223a3481f7@oracle.com>
 <1f4d63f0-ac4f-7b4a-cad2-5f9d35803b4f@oracle.com>
Message-ID: <cb0bba7b-1cef-013a-4d9f-6d2fa9e013a1@oracle.com>

Thanks Coleen!

Harold

On 1/15/2020 6:51 PM, coleen.phillimore at oracle.com wrote:
>
> This looks really good!? Thanks Harold.
>
> I wonder if we see any reduction in instructions in startup, since 
> some similar change did.? Maybe we can get Claes to measure it. :)
>
> Thanks,
> Coleen
>
> On 1/15/20 7:57 AM, Harold Seigel wrote:
>> Hi,
>>
>> Please review this new webrev that also makes Thread* the first 
>> argument to the relevant MutexLocker and MonitorLocker constructors 
>> as requested by Coleen.
>>
>> Updated Webrev: 
>> http://cr.openjdk.java.net/~hseigel/bug_8235678.2/webrev/index.html
>>
>> Thanks, Harold
>>
>> On 1/14/2020 11:24 AM, coleen.phillimore at oracle.com wrote:
>>>
>>> Hi Harold,
>>>
>>> I really wanted this change to move Thread to the first argument 
>>> like many of the other calls in the VM that take THREAD as an argument.
>>>
>>> Written like this:
>>>
>>> + MutexLocker mu(Threads_lock, THREAD);
>>>
>>>
>>> It's too easy for someone who's cut/pasting to think the last THREAD 
>>> argument should really be CHECK, which is completely wrong.
>>>
>>> Can you switch the arguments?
>>>
>>> Thanks,
>>> Coleen
>>>
>>> On 1/14/20 9:00 AM, Harold Seigel wrote:
>>>> Hi,
>>>>
>>>> Please review this small change, to reduce unnecessary calls to 
>>>> Thread::current() in MutexLocker calls, by passing the current 
>>>> thread as an argument.? A few ResoureMark declarations were also 
>>>> changed.
>>>>
>>>> Open Webrev: 
>>>> http://cr.openjdk.java.net/~hseigel/bug_8235678/webrev/index.html
>>>>
>>>> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8235678
>>>>
>>>> The fix was regression tested by running Mach5 tiers 1 and 2 tests 
>>>> and builds on Linux-x64, Solaris, Windows, and Mac OS X, by running 
>>>> Mach5 tiers 3-5 tests on Linux-x64, and JCK lang and VM tests on 
>>>> Linux-x64.
>>>>
>>>> Thanks, Harold
>>>>
>>>
>

From bob.vandette at oracle.com  Thu Jan 16 14:13:02 2020
From: bob.vandette at oracle.com (Bob Vandette)
Date: Thu, 16 Jan 2020 09:13:02 -0500
Subject: [PING2!] RFR: 8230305: Cgroups v2: Container awareness
In-Reply-To: <fb194acf-43c3-e6a0-5f89-71b9cbea3b5c@oracle.com>
References: <072f66ee8c44034831b4e38f6470da4bff6edd07.camel@redhat.com>
 <7540a208e306ab957032b18178a53c6afa105d33.camel@redhat.com>
 <CC473B8C-E6CB-4293-BE01-BEDD09BEB358@oracle.com>
 <baa4c4821174dc0dfa43190d862d74b22fbf35d4.camel@redhat.com>
 <d504e04dd277ef46f42064d2bbd54b1ea94da0c2.camel@redhat.com>
 <5eec97c04d86562346243c1db3832e86e13697a1.camel@redhat.com>
 <b575b061cf4b7b0d9c0656aa1a726b2dd5a8a98c.camel@redhat.com>
 <52110EA5-26C9-43F2-8C2A-21D4E03ED3CC@oracle.com>
 <fb194acf-43c3-e6a0-5f89-71b9cbea3b5c@oracle.com>
Message-ID: <DE00BFD8-4E0F-4538-AC67-D853D5B2D77A@oracle.com>



> On Jan 15, 2020, at 5:13 PM, David Holmes <david.holmes at oracle.com> wrote:
> 
> Hi Bob, Severin,
> 
> On 15/01/2020 6:04 am, Bob Vandette wrote:
>> Cgroup V2 is about to go mainstream this year for popular distros such as Oracle Linux 8, Redhat Linux 8 and Fedora
>> so this fix it?s important to get into JDK 15 so we can start shaking out this support.
>> Please take a look and help get this change reviewed.
> 
> I've taken a look and the overall structure and approach seems fine. I can't attest to the details of cgroups v1 versus v2 but trust the exports.
> 
> Reviewed.
> 
> Please ensure all copyrights updated to 2020.
> 
> Bob: I assume you have tested this on our systems?

I?m in the process of setting up a cgroup v2 system to test with but I?ll grab the patch and verify the
container tests on my cgroup v1 setup today.

Bob.


> 
> Thanks,
> David
> 
>> Thanks,
>> Bob Vandette
>>> On Nov 29, 2019, at 4:04 AM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
>>> 
>>> On Fri, 2019-11-15 at 17:56 +0100, Severin Gehwolf wrote:
>>>> On Fri, 2019-11-08 at 15:21 +0100, Severin Gehwolf wrote:
>>>>> Hi Bob,
>>>>> 
>>>>> On Wed, 2019-11-06 at 10:47 +0100, Severin Gehwolf wrote:
>>>>>> On Tue, 2019-11-05 at 16:54 -0500, Bob Vandette wrote:
>>>>>>> Severin,
>>>>>>> 
>>>>>>> Thanks for taking on this cgroup v2 improvement.
>>>>>>> 
>>>>>>> In general I like the implementation and the refactoring.  The CachedMetric class is nice.
>>>>>>> We can add any metric we want to cache in a more general way.
>>>>>>> 
>>>>>>> Is this the latest version of the webrev?
>>>>>>> 
>>>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/cgroupsv2-hotspot/03/webrev/src/hotspot/os/linux/cgroupV2Subsystem_linux.cpp.html
>>>>>>> 
>>>>>>> It looks like you need to add the caching support for active_processor_count (JDK-8227006).
>>>>> [...]
>>>>>> I'll do a proper rebase ASAP.
>>>>> 
>>>>> Latest webrev:
>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/cgroupsv2-hotspot/05/webrev/
>>>>> 
>>>>>>> I?m not sure it?s worth providing different strings for Unlimited versus Max or Scaled shares.
>>>>>>> I?d just try to be compatible with the cgroupv2 output so you don?t have to change the test.
>>>>>> 
>>>>>> OK. Will do.
>>>>> 
>>>>> Unfortunately, there is no way of NOT changing TestCPUAwareness.java as
>>>>> it expects CPU Shares to be written to the cgroup filesystem verbatim.
>>>>> That's no longer the case for cgroups v2 (at least for crun). Either
>>>>> way, most test changes are gone now.
>>>>> 
>>>>>>> I wonder if it?s worth trying to synthesize memory_max_usage_in_bytes() by keeping the highest
>>>>>>> value ever returned by the API.
>>>>>> 
>>>>>> Interesting idea. I'll ponder this a bit and get back to you.
>>>>> 
>>>>> This has been implemented. I'm not sure this is correct, though. It
>>>>> merely piggy-backs on calls to memory_usage_in_bytes() and keeps the
>>>>> high watermark value of that.
>>>>> 
>>>>> Testing passed on F31 with cgroups v2 controllers properly configured
>>>>> (podman) and hybrid (legacy hierarchy) with docker/podman.
>>>>> 
>>>>> Thoughts?
>>>> 
>>>> Ping?
>>> 
>>> Anyone willing to review this? It would be nice to make some progress.
>>> 
>>> Thanks,
>>> Severin
>>> 
>>>> Metrics work proposed for RFR here:
>>>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2019-November/063464.html
>>>> 
>>>> Thanks,
>>>> Severin
>>> 


From zgu at redhat.com  Thu Jan 16 17:47:36 2020
From: zgu at redhat.com (Zhengyu Gu)
Date: Thu, 16 Jan 2020 12:47:36 -0500
Subject: Submit repo status
Message-ID: <32a462b0-8f1d-4cbc-4a4a-13e3899b8971@redhat.com>

Hi,

What's the status of submit repo? I did not receive any test results 
[0][1][2], or just me?


[0] 
http://mail.openjdk.java.net/pipermail/jdk-submit-changes/2020-January/009312.html
[1] 
http://mail.openjdk.java.net/pipermail/jdk-submit-changes/2020-January/009351.html
[2] 
http://mail.openjdk.java.net/pipermail/jdk-submit-changes/2020-January/009419.html

Thanks,

-Zhengyu


From stanislav.smirnov at oracle.com  Thu Jan 16 18:50:05 2020
From: stanislav.smirnov at oracle.com (Stanislav Smirnov)
Date: Thu, 16 Jan 2020 13:50:05 -0500
Subject: Submit repo status
In-Reply-To: <32a462b0-8f1d-4cbc-4a4a-13e3899b8971@redhat.com>
References: <32a462b0-8f1d-4cbc-4a4a-13e3899b8971@redhat.com>
Message-ID: <3BAB0D44-7251-453D-8B1D-CC655FF246D9@oracle.com>

Hi Zhengyu,

Next emails with results were sent to you

[Mach5] mach5-one-zgu-JDK-8236878-1-20200113-1356-8016418: FAILED
sent: zgu at redhat.com
date: 01/15

[Mach5] mach5-one-zgu-JDK-8236878-2-20200116-1349-8103008: PASSED
sent: zgu at redhat.com
date: 01/16

Recently I received similar requests from your colleagues from Redhat.
Is it possible, that there is some firewall on your side, that prevents such emails from being delivered?

Best regards,
Stanislav Smirnov


> On Jan 16, 2020, at 12:47 PM, Zhengyu Gu <zgu at redhat.com> wrote:
> 
> Hi,
> 
> What's the status of submit repo? I did not receive any test results [0][1][2], or just me?
> 
> 
> [0] http://mail.openjdk.java.net/pipermail/jdk-submit-changes/2020-January/009312.html
> [1] http://mail.openjdk.java.net/pipermail/jdk-submit-changes/2020-January/009351.html
> [2] http://mail.openjdk.java.net/pipermail/jdk-submit-changes/2020-January/009419.html
> 
> Thanks,
> 
> -Zhengyu
> 


From zgu at redhat.com  Thu Jan 16 19:00:15 2020
From: zgu at redhat.com (Zhengyu Gu)
Date: Thu, 16 Jan 2020 14:00:15 -0500
Subject: Submit repo status
In-Reply-To: <3BAB0D44-7251-453D-8B1D-CC655FF246D9@oracle.com>
References: <32a462b0-8f1d-4cbc-4a4a-13e3899b8971@redhat.com>
 <3BAB0D44-7251-453D-8B1D-CC655FF246D9@oracle.com>
Message-ID: <9258e91a-ff84-3f2c-d511-30fcd5ba24a3@redhat.com>


Thanks, Stanislav.


On 1/16/20 1:50 PM, Stanislav Smirnov wrote:
> Hi Zhengyu,
> 
> Next emails with results were sent to you
> 
> [Mach5] mach5-one-zgu-JDK-8236878-1-20200113-1356-8016418: FAILED
> sent: zgu at redhat.com <mailto:zgu at redhat.com>
> date: 01/15
> 
> [Mach5] mach5-one-zgu-JDK-8236878-2-20200116-1349-8103008: PASSED
> sent: zgu at redhat.com <mailto:zgu at redhat.com>
> date: 01/16
> 
> Recently I received similar requests from your colleagues from Redhat.
> Is it possible, that there is some firewall on your side, that prevents 
> such emails from being delivered?

I will try to find it out.

-Zhengyu




> 
> Best regards,
> Stanislav Smirnov
> 
> 
>> On Jan 16, 2020, at 12:47 PM, Zhengyu Gu <zgu at redhat.com 
>> <mailto:zgu at redhat.com>> wrote:
>>
>> Hi,
>>
>> What's the status of submit repo? I did not receive any test results 
>> [0][1][2], or just me?
>>
>>
>> [0] 
>> http://mail.openjdk.java.net/pipermail/jdk-submit-changes/2020-January/009312.html
>> [1] 
>> http://mail.openjdk.java.net/pipermail/jdk-submit-changes/2020-January/009351.html
>> [2] 
>> http://mail.openjdk.java.net/pipermail/jdk-submit-changes/2020-January/009419.html
>>
>> Thanks,
>>
>> -Zhengyu
>>
> 


From sgehwolf at redhat.com  Thu Jan 16 20:08:15 2020
From: sgehwolf at redhat.com (Severin Gehwolf)
Date: Thu, 16 Jan 2020 21:08:15 +0100
Subject: [PING2!] RFR: 8230305: Cgroups v2: Container awareness
In-Reply-To: <DE00BFD8-4E0F-4538-AC67-D853D5B2D77A@oracle.com>
References: <072f66ee8c44034831b4e38f6470da4bff6edd07.camel@redhat.com>
 <7540a208e306ab957032b18178a53c6afa105d33.camel@redhat.com>
 <CC473B8C-E6CB-4293-BE01-BEDD09BEB358@oracle.com>
 <baa4c4821174dc0dfa43190d862d74b22fbf35d4.camel@redhat.com>
 <d504e04dd277ef46f42064d2bbd54b1ea94da0c2.camel@redhat.com>
 <5eec97c04d86562346243c1db3832e86e13697a1.camel@redhat.com>
 <b575b061cf4b7b0d9c0656aa1a726b2dd5a8a98c.camel@redhat.com>
 <52110EA5-26C9-43F2-8C2A-21D4E03ED3CC@oracle.com>
 <fb194acf-43c3-e6a0-5f89-71b9cbea3b5c@oracle.com>
 <DE00BFD8-4E0F-4538-AC67-D853D5B2D77A@oracle.com>
Message-ID: <d5a4e60a028f7a9c843245ae5511d840a25723c7.camel@redhat.com>

Hi Bob, David,

On Thu, 2020-01-16 at 09:13 -0500, Bob Vandette wrote:
> > On Jan 15, 2020, at 5:13 PM, David Holmes <david.holmes at oracle.com> wrote:
> > 
> > Hi Bob, Severin,
> > 
> > On 15/01/2020 6:04 am, Bob Vandette wrote:
> > > Cgroup V2 is about to go mainstream this year for popular distros such as Oracle Linux 8, Redhat Linux 8 and Fedora
> > > so this fix it?s important to get into JDK 15 so we can start shaking out this support.
> > > Please take a look and help get this change reviewed.
> > 
> > I've taken a look and the overall structure and approach seems fine. I can't attest to the details of cgroups v1 versus v2 but trust the exports.
> > 
> > Reviewed.
> > 
> > Please ensure all copyrights updated to 2020.
> > 
> > Bob: I assume you have tested this on our systems?
> 
> I?m in the process of setting up a cgroup v2 system to test with but I?ll grab the patch and verify the
> container tests on my cgroup v1 setup today.

Final webrev I intend to push tomorrow for this:
http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8230305/07/webrev/

This passed jdk/submit and testing on cgroups v1 hotspot docker tests.
On cgroups v2 this fails the following test since it also tests the
OperatingSystemMXBean's memory awareness which uses core-libs Metrics
code not updated with this webrev (passes all others):

hotspot/jtreg/containers/docker/TestMemoryAwareness.java

It's caused by JDK-8226575 which went in during the time I've started
this and intented push time. This test failure will be fixed with JDK-
8231111 once I get another review from a Reviewer. I doubt many people
will notice though since cgroupv2 enabled systems aren't that
widespread just yet.

Please let me know if there are any objections.

Thanks,
Severin


> Bob.
> 
> 
> > Thanks,
> > David
> > 
> > > Thanks,
> > > Bob Vandette
> > > > On Nov 29, 2019, at 4:04 AM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
> > > > 
> > > > On Fri, 2019-11-15 at 17:56 +0100, Severin Gehwolf wrote:
> > > > > On Fri, 2019-11-08 at 15:21 +0100, Severin Gehwolf wrote:
> > > > > > Hi Bob,
> > > > > > 
> > > > > > On Wed, 2019-11-06 at 10:47 +0100, Severin Gehwolf wrote:
> > > > > > > On Tue, 2019-11-05 at 16:54 -0500, Bob Vandette wrote:
> > > > > > > > Severin,
> > > > > > > > 
> > > > > > > > Thanks for taking on this cgroup v2 improvement.
> > > > > > > > 
> > > > > > > > In general I like the implementation and the refactoring.  The CachedMetric class is nice.
> > > > > > > > We can add any metric we want to cache in a more general way.
> > > > > > > > 
> > > > > > > > Is this the latest version of the webrev?
> > > > > > > > 
> > > > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/cgroupsv2-hotspot/03/webrev/src/hotspot/os/linux/cgroupV2Subsystem_linux.cpp.html
> > > > > > > > 
> > > > > > > > It looks like you need to add the caching support for active_processor_count (JDK-8227006).
> > > > > > [...]
> > > > > > > I'll do a proper rebase ASAP.
> > > > > > 
> > > > > > Latest webrev:
> > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/cgroupsv2-hotspot/05/webrev/
> > > > > > 
> > > > > > > > I?m not sure it?s worth providing different strings for Unlimited versus Max or Scaled shares.
> > > > > > > > I?d just try to be compatible with the cgroupv2 output so you don?t have to change the test.
> > > > > > > 
> > > > > > > OK. Will do.
> > > > > > 
> > > > > > Unfortunately, there is no way of NOT changing TestCPUAwareness.java as
> > > > > > it expects CPU Shares to be written to the cgroup filesystem verbatim.
> > > > > > That's no longer the case for cgroups v2 (at least for crun). Either
> > > > > > way, most test changes are gone now.
> > > > > > 
> > > > > > > > I wonder if it?s worth trying to synthesize memory_max_usage_in_bytes() by keeping the highest
> > > > > > > > value ever returned by the API.
> > > > > > > 
> > > > > > > Interesting idea. I'll ponder this a bit and get back to you.
> > > > > > 
> > > > > > This has been implemented. I'm not sure this is correct, though. It
> > > > > > merely piggy-backs on calls to memory_usage_in_bytes() and keeps the
> > > > > > high watermark value of that.
> > > > > > 
> > > > > > Testing passed on F31 with cgroups v2 controllers properly configured
> > > > > > (podman) and hybrid (legacy hierarchy) with docker/podman.
> > > > > > 
> > > > > > Thoughts?
> > > > > 
> > > > > Ping?
> > > > 
> > > > Anyone willing to review this? It would be nice to make some progress.
> > > > 
> > > > Thanks,
> > > > Severin
> > > > 
> > > > > Metrics work proposed for RFR here:
> > > > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2019-November/063464.html
> > > > > 
> > > > > Thanks,
> > > > > Severin


From bob.vandette at oracle.com  Thu Jan 16 20:18:54 2020
From: bob.vandette at oracle.com (Bob Vandette)
Date: Thu, 16 Jan 2020 15:18:54 -0500
Subject: [PING2!] RFR: 8230305: Cgroups v2: Container awareness
In-Reply-To: <d5a4e60a028f7a9c843245ae5511d840a25723c7.camel@redhat.com>
References: <072f66ee8c44034831b4e38f6470da4bff6edd07.camel@redhat.com>
 <7540a208e306ab957032b18178a53c6afa105d33.camel@redhat.com>
 <CC473B8C-E6CB-4293-BE01-BEDD09BEB358@oracle.com>
 <baa4c4821174dc0dfa43190d862d74b22fbf35d4.camel@redhat.com>
 <d504e04dd277ef46f42064d2bbd54b1ea94da0c2.camel@redhat.com>
 <5eec97c04d86562346243c1db3832e86e13697a1.camel@redhat.com>
 <b575b061cf4b7b0d9c0656aa1a726b2dd5a8a98c.camel@redhat.com>
 <52110EA5-26C9-43F2-8C2A-21D4E03ED3CC@oracle.com>
 <fb194acf-43c3-e6a0-5f89-71b9cbea3b5c@oracle.com>
 <DE00BFD8-4E0F-4538-AC67-D853D5B2D77A@oracle.com>
 <d5a4e60a028f7a9c843245ae5511d840a25723c7.camel@redhat.com>
Message-ID: <67D9E772-9949-4DE2-95B4-F4811E4EA06C@oracle.com>


> On Jan 16, 2020, at 3:08 PM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
> 
> Hi Bob, David,
> 
> On Thu, 2020-01-16 at 09:13 -0500, Bob Vandette wrote:
>>> On Jan 15, 2020, at 5:13 PM, David Holmes <david.holmes at oracle.com> wrote:
>>> 
>>> Hi Bob, Severin,
>>> 
>>> On 15/01/2020 6:04 am, Bob Vandette wrote:
>>>> Cgroup V2 is about to go mainstream this year for popular distros such as Oracle Linux 8, Redhat Linux 8 and Fedora
>>>> so this fix it?s important to get into JDK 15 so we can start shaking out this support.
>>>> Please take a look and help get this change reviewed.
>>> 
>>> I've taken a look and the overall structure and approach seems fine. I can't attest to the details of cgroups v1 versus v2 but trust the exports.
>>> 
>>> Reviewed.
>>> 
>>> Please ensure all copyrights updated to 2020.
>>> 
>>> Bob: I assume you have tested this on our systems?
>> 
>> I?m in the process of setting up a cgroup v2 system to test with but I?ll grab the patch and verify the
>> container tests on my cgroup v1 setup today.
> 
> Final webrev I intend to push tomorrow for this:
> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8230305/07/webrev/
> 
> This passed jdk/submit and testing on cgroups v1 hotspot docker tests.
> On cgroups v2 this fails the following test since it also tests the
> OperatingSystemMXBean's memory awareness which uses core-libs Metrics
> code not updated with this webrev (passes all others):
> 
> hotspot/jtreg/containers/docker/TestMemoryAwareness.java
> 
> It's caused by JDK-8226575 which went in during the time I've started
> this and intented push time. This test failure will be fixed with JDK-
> 8231111 once I get another review from a Reviewer. I doubt many people
> will notice though since cgroupv2 enabled systems aren't that
> widespread just yet.
> 
> Please let me know if there are any objections.

I ran your changes on a local cgroupv1 enabled system and didn?t see any failures
specific to your changes.

I saw one test failure on my system but it was pre-existing.  The TestDockerMemoryMetrics.testMemoryFailCount
test gets killed by OOM Killer on my system.  This needs more investigation.  There was an existing
bug filed for this issue which is resolved but the problem persists on my system.

    https://bugs.openjdk.java.net/browse/JDK-8224506 <https://bugs.openjdk.java.net/browse/JDK-8224506>


I?m good with the Metrics problem since that code is not cgroupv2 aware yet.

Bob.


> 
> Thanks,
> Severin
> 
> 
>> Bob.
>> 
>> 
>>> Thanks,
>>> David
>>> 
>>>> Thanks,
>>>> Bob Vandette
>>>>> On Nov 29, 2019, at 4:04 AM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
>>>>> 
>>>>> On Fri, 2019-11-15 at 17:56 +0100, Severin Gehwolf wrote:
>>>>>> On Fri, 2019-11-08 at 15:21 +0100, Severin Gehwolf wrote:
>>>>>>> Hi Bob,
>>>>>>> 
>>>>>>> On Wed, 2019-11-06 at 10:47 +0100, Severin Gehwolf wrote:
>>>>>>>> On Tue, 2019-11-05 at 16:54 -0500, Bob Vandette wrote:
>>>>>>>>> Severin,
>>>>>>>>> 
>>>>>>>>> Thanks for taking on this cgroup v2 improvement.
>>>>>>>>> 
>>>>>>>>> In general I like the implementation and the refactoring.  The CachedMetric class is nice.
>>>>>>>>> We can add any metric we want to cache in a more general way.
>>>>>>>>> 
>>>>>>>>> Is this the latest version of the webrev?
>>>>>>>>> 
>>>>>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/cgroupsv2-hotspot/03/webrev/src/hotspot/os/linux/cgroupV2Subsystem_linux.cpp.html
>>>>>>>>> 
>>>>>>>>> It looks like you need to add the caching support for active_processor_count (JDK-8227006).
>>>>>>> [...]
>>>>>>>> I'll do a proper rebase ASAP.
>>>>>>> 
>>>>>>> Latest webrev:
>>>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/cgroupsv2-hotspot/05/webrev/
>>>>>>> 
>>>>>>>>> I?m not sure it?s worth providing different strings for Unlimited versus Max or Scaled shares.
>>>>>>>>> I?d just try to be compatible with the cgroupv2 output so you don?t have to change the test.
>>>>>>>> 
>>>>>>>> OK. Will do.
>>>>>>> 
>>>>>>> Unfortunately, there is no way of NOT changing TestCPUAwareness.java as
>>>>>>> it expects CPU Shares to be written to the cgroup filesystem verbatim.
>>>>>>> That's no longer the case for cgroups v2 (at least for crun). Either
>>>>>>> way, most test changes are gone now.
>>>>>>> 
>>>>>>>>> I wonder if it?s worth trying to synthesize memory_max_usage_in_bytes() by keeping the highest
>>>>>>>>> value ever returned by the API.
>>>>>>>> 
>>>>>>>> Interesting idea. I'll ponder this a bit and get back to you.
>>>>>>> 
>>>>>>> This has been implemented. I'm not sure this is correct, though. It
>>>>>>> merely piggy-backs on calls to memory_usage_in_bytes() and keeps the
>>>>>>> high watermark value of that.
>>>>>>> 
>>>>>>> Testing passed on F31 with cgroups v2 controllers properly configured
>>>>>>> (podman) and hybrid (legacy hierarchy) with docker/podman.
>>>>>>> 
>>>>>>> Thoughts?
>>>>>> 
>>>>>> Ping?
>>>>> 
>>>>> Anyone willing to review this? It would be nice to make some progress.
>>>>> 
>>>>> Thanks,
>>>>> Severin
>>>>> 
>>>>>> Metrics work proposed for RFR here:
>>>>>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2019-November/063464.html
>>>>>> 
>>>>>> Thanks,
>>>>>> Severin
> 


From david.holmes at oracle.com  Thu Jan 16 22:01:56 2020
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 17 Jan 2020 08:01:56 +1000
Subject: RFR (M): 8235741: Inappropriate uses of os::javaTimeMillis()
In-Reply-To: <0af8dd6d-f84b-7d0a-a99e-ea75e269398e@oracle.com>
References: <f99194ad-3dad-48db-d8d4-de0cc9161701@oracle.com>
 <160821DF-BE15-435C-BDBE-AB553F794590@oracle.com>
 <a04e7b9f-3fe8-01ea-9784-fcf21613340d@oracle.com>
 <3b9fd351-6222-b425-fada-382edc4799ff@oracle.com>
 <0af8dd6d-f84b-7d0a-a99e-ea75e269398e@oracle.com>
Message-ID: <2ddf2d38-2c51-b1ae-8c7b-8fafa9772c17@oracle.com>

Hi Erik,

On 16/01/2020 9:54 pm, Erik Gahlin wrote:
> HI David,
> 
> I'm reasonable convinced the JFR code works as before. We should 
> probably rewrite the code, but it's outside the scope of this bug.

Great! Thanks for taking a detailed look at that part.

David

> Thanks for fixing
> Erik
> 
> On 2020-01-16 05:26, David Holmes wrote:
>> While awaiting further comments/feedback I've made some updates:
>>
>> - added comments to existing use of javaTimeMillis() where it wasn't 
>> (fairly) obvious why it was being used
>> - fixed now incorrect comment pointed out by Kim
>> - copied os_per_linux.cpp changes to os_perf_aix.cpp (optimistically 
>> hoping for a thumbs up from Erik G. on this part :) ).
>>
>> Full and incr webrevs at:
>>
>> http://cr.openjdk.java.net/~dholmes/8235741/webrev.v2/
>> http://cr.openjdk.java.net/~dholmes/8235741/webrev.v2-incr/
>>
>> Thanks,
>> David
>>
>> On 15/01/2020 5:12 pm, David Holmes wrote:
>>> Hi Kim,
>>>
>>> Thanks for taking a look at this.
>>>
>>> On 15/01/2020 4:24 pm, Kim Barrett wrote:
>>>>> On Jan 13, 2020, at 2:13 AM, David Holmes <david.holmes at oracle.com> 
>>>>> wrote:
>>>>>
>>>>> webrev: http://cr.openjdk.java.net/~dholmes/8235741/webrev/
>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8235741
>>>>>
>>>>> Full details in the bug report about the existing uses of 
>>>>> javaTimeMillis(), many of which just want an elapsed time in ms and 
>>>>> so should be using javaTimeNanos() and convert to ms. This covers 
>>>>> areas all across the VM.
>>>>>
>>>>> Only non-simple change is in os_perf_linux.cpp (and the same code 
>>>>> will be in os_perf_aix.cpp once it has been validated). There we 
>>>>> are tracking an elapsed time in ms but relative to the boot time, 
>>>>> which is seconds since the epoch. Consequently the first interval 
>>>>> has to be calculated using javaTimeMillis, but after that we can 
>>>>> use javaTimeNanos (using a new 'first time' captured at the same 
>>>>> time we used javaTimeMillis). I think I have the logic right but 
>>>>> other than through JFR this code seems unused and I have limited 
>>>>> means of testing it. The JFR test 
>>>>> jdk/jfr/event/os/TestThreadContextSwitches.java exercises the code 
>>>>> but the results of running that test seems to exhibit arbitrary 
>>>>> randomness in the rates reported - e.g. 0 to 16000Hz - both with 
>>>>> and without my change, so not really that useful. Stefan K. 
>>>>> suggested a gtest which I may look into - though it is frustrating 
>>>>> to have to expend such effort to validate this.
>>>>>
>>>>> Other testing tiers 1-3.
>>>>>
>>>>> Thanks,
>>>>> David
>>>>
>>>> Thanks for the audit of uses of os::javaTimeMillis() in the bug report.
>>>> I wonder if some of that ought to be captured as comments in the
>>>> relevant code.? It's not always obvious to me that an external time
>>>> base is involved and thus making javaTimeMillis not a mistake.
>>>
>>> Okay, I will add comments to the other uses of currentTimeMillis().
>>>
>>>> There are a lot of places where conversions from nanoseconds to
>>>> milliseconds are being done to maintain existing units.? Some of those
>>>> places look like they could just as well be in nanoseconds. But I can
>>>> see how changing the units for some of those could lead to a lot of
>>>> fannout, so okay.
>>>
>>> Yes I tried to minimise the changes. In many cases a granularity of 
>>> ms seems somewhat arbitrary.
>>>
>>>> ------------------------------------------------------------------------------ 
>>>>
>>>> src/hotspot/os/windows/os_perf_windows.cpp
>>>> ? 100?? s8???? lastUpdate; // Last time query was updated (current 
>>>> millis).
>>>> ...
>>>> ? 290?? const s8 now = os::javaTimeNanos();
>>>> ? 291?? if (NANOS_TO_MILLIS(now - update_query->lastUpdate) > 
>>>> min_update_interval_millis) {
>>>> ...
>>>> ? 295???? update_query->lastUpdate = now;
>>>>
>>>> now and update_query->lastUpdate are now in nanos, but comment for
>>>> lastUpdate still says it's in millis.? Looks like the comment needs
>>>> updating.
>>>
>>> Yes - good catch.
>>>
>>>> ------------------------------------------------------------------------------ 
>>>>
>>>> src/hotspot/share/utilities/globalDefinitions.hpp
>>>> ? 262 // time unit conversion macros
>>>> ? 263
>>>> ? 264 #define NANOS_TO_MILLIS(ns) ((ns) / NANOSECS_PER_MILLISEC)
>>>> ? 265 #define MILLIS_TO_NANOS(ms) ((ms) * NANOSECS_PER_MILLISEC)
>>>>
>>>> Why are these macros, rather than (template) functions?
>>>
>>> Just because I just wanted a simple textual replacement to make it 
>>> clearer that I'm converting from millis to nanos or vice versa. I 
>>> reach for macros for such simple cases.
>>>
>>>> Also, depending on the type and value of ms, MILLIS_TO_NANOS could
>>>> easily overflow, e.g. if ms type is a 32 bit type with a value of more
>>>> than ~4 seconds.? (I checked the two uses, and they happen to be okay.)
>>>
>>> These are not trying to be mathematically sound. The conversion from 
>>> millis to nanos is used in two cases:
>>>
>>> 1. Converting a current timestamp in ms to ns. Unless the current 
>>> time is set far in the future I don't think we have any issue with 
>>> overflow of? such a value.
>>>
>>> 2. converting an elapsed time in ms to ns. These will be small values 
>>> so no overflow is possible.
>>>
>>>> inline int64_t nanos_to_millis(int64_t ns) {
>>>> ?? return ns / NANOSECS_PER_MILLISECOND;
>>>> }
>>>>
>>>> inline int64_t millis_to_nanos(int64_t ms) {
>>>> ?? return ms * NANOSECONDS_PER_MILLISEC;
>>>> }
>>>>
>>>> Also, the names don't suggest time conversions, but potentially
>>>> arbitrary unit conversions, e.g. between something in NANOUNITS and
>>>> something in MILLIUNITS.
>>>
>>> They don't have to be time conversions - the calculation is unit-less 
>>> in practice. The fact we have NANOSEC_PER_MILLISECOND et al is just 
>>> an artifact of introducing those values for timeout 
>>> calculations/conversions - it could just be NANOS_PER_MILLI etc
>>>
>>>> ------------------------------------------------------------------------------ 
>>>>
>>>> Regarding this from the audit:
>>>>
>>>> --- begin ---
>>>> ./share/gc/parallel/psParallelCompact.cpp: // os::javaTimeMillis() 
>>>> does not guarantee monotonicity.
>>>> ...
>>>> ./share/gc/shared/referenceProcessor.cpp: // os::javaTimeMillis() 
>>>> does not guarantee monotonicity.
>>>>
>>>> These are all describing why the subsequent code uses javaTimeNanos 
>>>> not javaTimeMillis.
>>>> --- end ---
>>>>
>>>> Do we really still support platforms that don't have a monotonic
>>>> clock?? I guess we appear to at least try.? But it's really wrong that
>>>> callers of os::javaTimeNanos should even think they need to cope with
>>>> that function being non-monotonic.
>>>>
>>>> Hm, I always thought System.nanoTime() was a monotonic clock, but I
>>>> don't see any such guarantee. So I guess Java just doesn't have such a
>>>> thing. Wow!
>>>>
>>>> So I guess none of this is really relevant to the change at hand 
>>>> after all.
>>>
>>> I think you read the comments the wrong way round. The code uses 
>>> javaTimeNanos not javaTimeMillis because javaTimeMillis is not 
>>> monotonic and the code wants a monotonic clock. These comments were 
>>> mostly inserted when the incorrect use of javaTimeMillis was replaced 
>>> with javaTimeNanos.
>>>
>>> Thanks,
>>> David
>>> -----
>>>
>>>> ------------------------------------------------------------------------------ 
>>>>
>>>>
>>>>

From david.holmes at oracle.com  Thu Jan 16 22:09:33 2020
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 17 Jan 2020 08:09:33 +1000
Subject: [PING2!] RFR: 8230305: Cgroups v2: Container awareness
In-Reply-To: <d5a4e60a028f7a9c843245ae5511d840a25723c7.camel@redhat.com>
References: <072f66ee8c44034831b4e38f6470da4bff6edd07.camel@redhat.com>
 <7540a208e306ab957032b18178a53c6afa105d33.camel@redhat.com>
 <CC473B8C-E6CB-4293-BE01-BEDD09BEB358@oracle.com>
 <baa4c4821174dc0dfa43190d862d74b22fbf35d4.camel@redhat.com>
 <d504e04dd277ef46f42064d2bbd54b1ea94da0c2.camel@redhat.com>
 <5eec97c04d86562346243c1db3832e86e13697a1.camel@redhat.com>
 <b575b061cf4b7b0d9c0656aa1a726b2dd5a8a98c.camel@redhat.com>
 <52110EA5-26C9-43F2-8C2A-21D4E03ED3CC@oracle.com>
 <fb194acf-43c3-e6a0-5f89-71b9cbea3b5c@oracle.com>
 <DE00BFD8-4E0F-4538-AC67-D853D5B2D77A@oracle.com>
 <d5a4e60a028f7a9c843245ae5511d840a25723c7.camel@redhat.com>
Message-ID: <ed5975ae-e789-2292-a13d-cbe5f64cccc8@oracle.com>

No objections from me.

Thanks,
David

On 17/01/2020 6:08 am, Severin Gehwolf wrote:
> Hi Bob, David,
> 
> On Thu, 2020-01-16 at 09:13 -0500, Bob Vandette wrote:
>>> On Jan 15, 2020, at 5:13 PM, David Holmes <david.holmes at oracle.com> wrote:
>>>
>>> Hi Bob, Severin,
>>>
>>> On 15/01/2020 6:04 am, Bob Vandette wrote:
>>>> Cgroup V2 is about to go mainstream this year for popular distros such as Oracle Linux 8, Redhat Linux 8 and Fedora
>>>> so this fix it?s important to get into JDK 15 so we can start shaking out this support.
>>>> Please take a look and help get this change reviewed.
>>>
>>> I've taken a look and the overall structure and approach seems fine. I can't attest to the details of cgroups v1 versus v2 but trust the exports.
>>>
>>> Reviewed.
>>>
>>> Please ensure all copyrights updated to 2020.
>>>
>>> Bob: I assume you have tested this on our systems?
>>
>> I?m in the process of setting up a cgroup v2 system to test with but I?ll grab the patch and verify the
>> container tests on my cgroup v1 setup today.
> 
> Final webrev I intend to push tomorrow for this:
> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8230305/07/webrev/
> 
> This passed jdk/submit and testing on cgroups v1 hotspot docker tests.
> On cgroups v2 this fails the following test since it also tests the
> OperatingSystemMXBean's memory awareness which uses core-libs Metrics
> code not updated with this webrev (passes all others):
> 
> hotspot/jtreg/containers/docker/TestMemoryAwareness.java
> 
> It's caused by JDK-8226575 which went in during the time I've started
> this and intented push time. This test failure will be fixed with JDK-
> 8231111 once I get another review from a Reviewer. I doubt many people
> will notice though since cgroupv2 enabled systems aren't that
> widespread just yet.
> 
> Please let me know if there are any objections.
> 
> Thanks,
> Severin
> 
> 
>> Bob.
>>
>>
>>> Thanks,
>>> David
>>>
>>>> Thanks,
>>>> Bob Vandette
>>>>> On Nov 29, 2019, at 4:04 AM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
>>>>>
>>>>> On Fri, 2019-11-15 at 17:56 +0100, Severin Gehwolf wrote:
>>>>>> On Fri, 2019-11-08 at 15:21 +0100, Severin Gehwolf wrote:
>>>>>>> Hi Bob,
>>>>>>>
>>>>>>> On Wed, 2019-11-06 at 10:47 +0100, Severin Gehwolf wrote:
>>>>>>>> On Tue, 2019-11-05 at 16:54 -0500, Bob Vandette wrote:
>>>>>>>>> Severin,
>>>>>>>>>
>>>>>>>>> Thanks for taking on this cgroup v2 improvement.
>>>>>>>>>
>>>>>>>>> In general I like the implementation and the refactoring.  The CachedMetric class is nice.
>>>>>>>>> We can add any metric we want to cache in a more general way.
>>>>>>>>>
>>>>>>>>> Is this the latest version of the webrev?
>>>>>>>>>
>>>>>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/cgroupsv2-hotspot/03/webrev/src/hotspot/os/linux/cgroupV2Subsystem_linux.cpp.html
>>>>>>>>>
>>>>>>>>> It looks like you need to add the caching support for active_processor_count (JDK-8227006).
>>>>>>> [...]
>>>>>>>> I'll do a proper rebase ASAP.
>>>>>>>
>>>>>>> Latest webrev:
>>>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/cgroupsv2-hotspot/05/webrev/
>>>>>>>
>>>>>>>>> I?m not sure it?s worth providing different strings for Unlimited versus Max or Scaled shares.
>>>>>>>>> I?d just try to be compatible with the cgroupv2 output so you don?t have to change the test.
>>>>>>>>
>>>>>>>> OK. Will do.
>>>>>>>
>>>>>>> Unfortunately, there is no way of NOT changing TestCPUAwareness.java as
>>>>>>> it expects CPU Shares to be written to the cgroup filesystem verbatim.
>>>>>>> That's no longer the case for cgroups v2 (at least for crun). Either
>>>>>>> way, most test changes are gone now.
>>>>>>>
>>>>>>>>> I wonder if it?s worth trying to synthesize memory_max_usage_in_bytes() by keeping the highest
>>>>>>>>> value ever returned by the API.
>>>>>>>>
>>>>>>>> Interesting idea. I'll ponder this a bit and get back to you.
>>>>>>>
>>>>>>> This has been implemented. I'm not sure this is correct, though. It
>>>>>>> merely piggy-backs on calls to memory_usage_in_bytes() and keeps the
>>>>>>> high watermark value of that.
>>>>>>>
>>>>>>> Testing passed on F31 with cgroups v2 controllers properly configured
>>>>>>> (podman) and hybrid (legacy hierarchy) with docker/podman.
>>>>>>>
>>>>>>> Thoughts?
>>>>>>
>>>>>> Ping?
>>>>>
>>>>> Anyone willing to review this? It would be nice to make some progress.
>>>>>
>>>>> Thanks,
>>>>> Severin
>>>>>
>>>>>> Metrics work proposed for RFR here:
>>>>>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2019-November/063464.html
>>>>>>
>>>>>> Thanks,
>>>>>> Severin
> 

From kim.barrett at oracle.com  Fri Jan 17 01:39:48 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 16 Jan 2020 20:39:48 -0500
Subject: RFR (M): 8235741: Inappropriate uses of os::javaTimeMillis()
In-Reply-To: <a04e7b9f-3fe8-01ea-9784-fcf21613340d@oracle.com>
References: <f99194ad-3dad-48db-d8d4-de0cc9161701@oracle.com>
 <160821DF-BE15-435C-BDBE-AB553F794590@oracle.com>
 <a04e7b9f-3fe8-01ea-9784-fcf21613340d@oracle.com>
Message-ID: <EDE2324D-DA2D-4CF7-BAE0-EF59BEEB960B@oracle.com>

> On Jan 15, 2020, at 2:12 AM, David Holmes <david.holmes at oracle.com> wrote:
>> src/hotspot/share/utilities/globalDefinitions.hpp
>>  262 // time unit conversion macros
>>  263
>>  264 #define NANOS_TO_MILLIS(ns) ((ns) / NANOSECS_PER_MILLISEC)
>>  265 #define MILLIS_TO_NANOS(ms) ((ms) * NANOSECS_PER_MILLISEC)
>> Why are these macros, rather than (template) functions?
> 
> Just because I just wanted a simple textual replacement to make it clearer that I'm converting from millis to nanos or vice versa. I reach for macros for such simple cases.
> 
>> Also, depending on the type and value of ms, MILLIS_TO_NANOS could
>> easily overflow, e.g. if ms type is a 32 bit type with a value of more
>> than ~4 seconds.  (I checked the two uses, and they happen to be okay.)
> 
> These are not trying to be mathematically sound. The conversion from millis to nanos is used in two cases:
> 
> 1. Converting a current timestamp in ms to ns. Unless the current time is set far in the future I don't think we have any issue with overflow of  such a value.
> 
> 2. converting an elapsed time in ms to ns. These will be small values so no overflow is possible.

In this case, "not trying to be mathematically sound" is equivalent to
actively but subtly dangerous to use, because of possible overflow
leading to either UB or catastrophically wrong results.

Consider "int millis = 10 * 1000;", e.g. 10 seconds worth of
milliseconds.  I hope you will agree that this falls under your case
2, e.g. that 10 seconds is not an especially big interval.  I hope you
will also agree that declaring a variable that holds a millisecond
interval as an int is reasonable in some contexts. Applying
MILLIS_TO_NANOS with the jint-typed NANOSECS_PER_MILLISEC, and the
result about 1.4 seconds worth of nanoseconds, assuming the compiler
doesn't do something strange because of the UB on integer overflow.

(This is all in addition to the Style Guide's admonishment to use
inline functions rather than macros.  Macros should only be used for
syntactic extensions.)

>> inline int64_t nanos_to_millis(int64_t ns) {
>>   return ns / NANOSECS_PER_MILLISECOND;
>> }
>> inline int64_t millis_to_nanos(int64_t ms) {
>>   return ms * NANOSECONDS_PER_MILLISEC;
>> }
>> Also, the names don't suggest time conversions, but potentially
>> arbitrary unit conversions, e.g. between something in NANOUNITS and
>> something in MILLIUNITS.
> 
> They don't have to be time conversions - the calculation is unit-less in practice. The fact we have NANOSEC_PER_MILLISECOND et al is just an artifact of introducing those values for timeout calculations/conversions - it could just be NANOS_PER_MILLI etc

If they aren't time conversions then I don't think they should be
using the time factors; just use NANOUNITS/MILLIUNITS.

>> Regarding this from the audit:
>> --- begin ---
>> ./share/gc/parallel/psParallelCompact.cpp: // os::javaTimeMillis() does not guarantee monotonicity.
>> ...
>> ./share/gc/shared/referenceProcessor.cpp: // os::javaTimeMillis() does not guarantee monotonicity.
>> These are all describing why the subsequent code uses javaTimeNanos not javaTimeMillis.
>> --- end ---
>> Do we really still support platforms that don't have a monotonic
>> clock?  I guess we appear to at least try.  But it's really wrong that
>> callers of os::javaTimeNanos should even think they need to cope with
>> that function being non-monotonic.
>> Hm, I always thought System.nanoTime() was a monotonic clock, but I
>> don't see any such guarantee. So I guess Java just doesn't have such a
>> thing. Wow!
>> So I guess none of this is really relevant to the change at hand after all.
> 
> I think you read the comments the wrong way round. The code uses javaTimeNanos not javaTimeMillis because javaTimeMillis is not monotonic and the code wants a monotonic clock. These comments were mostly inserted when the incorrect use of javaTimeMillis was replaced with javaTimeNanos.

No, I did not read the comments the wrong way around.

I was pointing out that

(1) Some uses of os::javaTimeNanos attempt to cope with a possible
lack of monotonicity, while some do not and may be assuming or even
requiring monotonicity.  I'm wondering whether the coping attempts are
dead code, or instead, the latter are bugs.  That is, do we require
all platforms to provide a monotonic os::javaTimeNanos or not?
Personally, I think in today's world we should make such a requirement.

(2) There is no documented monotonicity guarantee for System.nanoTime,
nor any hint of a possible issue there, which surprised me.



From david.holmes at oracle.com  Fri Jan 17 02:48:59 2020
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 17 Jan 2020 12:48:59 +1000
Subject: RFR (M): 8235741: Inappropriate uses of os::javaTimeMillis()
In-Reply-To: <EDE2324D-DA2D-4CF7-BAE0-EF59BEEB960B@oracle.com>
References: <f99194ad-3dad-48db-d8d4-de0cc9161701@oracle.com>
 <160821DF-BE15-435C-BDBE-AB553F794590@oracle.com>
 <a04e7b9f-3fe8-01ea-9784-fcf21613340d@oracle.com>
 <EDE2324D-DA2D-4CF7-BAE0-EF59BEEB960B@oracle.com>
Message-ID: <f3e75dc8-4ee9-9fad-d5a3-94c0e813abd5@oracle.com>

Hi Kim,

On 17/01/2020 11:39 am, Kim Barrett wrote:
>> On Jan 15, 2020, at 2:12 AM, David Holmes <david.holmes at oracle.com> wrote:
>>> src/hotspot/share/utilities/globalDefinitions.hpp
>>>   262 // time unit conversion macros
>>>   263
>>>   264 #define NANOS_TO_MILLIS(ns) ((ns) / NANOSECS_PER_MILLISEC)
>>>   265 #define MILLIS_TO_NANOS(ms) ((ms) * NANOSECS_PER_MILLISEC)
>>> Why are these macros, rather than (template) functions?
>>
>> Just because I just wanted a simple textual replacement to make it clearer that I'm converting from millis to nanos or vice versa. I reach for macros for such simple cases.
>>
>>> Also, depending on the type and value of ms, MILLIS_TO_NANOS could
>>> easily overflow, e.g. if ms type is a 32 bit type with a value of more
>>> than ~4 seconds.  (I checked the two uses, and they happen to be okay.)
>>
>> These are not trying to be mathematically sound. The conversion from millis to nanos is used in two cases:
>>
>> 1. Converting a current timestamp in ms to ns. Unless the current time is set far in the future I don't think we have any issue with overflow of  such a value.
>>
>> 2. converting an elapsed time in ms to ns. These will be small values so no overflow is possible.
> 
> In this case, "not trying to be mathematically sound" is equivalent to
> actively but subtly dangerous to use, because of possible overflow
> leading to either UB or catastrophically wrong results.
> 
> Consider "int millis = 10 * 1000;", e.g. 10 seconds worth of
> milliseconds.  I hope you will agree that this falls under your case
> 2, e.g. that 10 seconds is not an especially big interval.  I hope you
> will also agree that declaring a variable that holds a millisecond
> interval as an int is reasonable in some contexts. Applying
> MILLIS_TO_NANOS with the jint-typed NANOSECS_PER_MILLISEC, and the
> result about 1.4 seconds worth of nanoseconds, assuming the compiler
> doesn't do something strange because of the UB on integer overflow.

So your concern is that someone will use the macros outside of existing 
usage contexts (where time values always using 64-bit variables) and 
thus introduce potential overflow bugs. Okay I will use the typed functions.

> (This is all in addition to the Style Guide's admonishment to use
> inline functions rather than macros.  Macros should only be used for
> syntactic extensions.)


>>> inline int64_t nanos_to_millis(int64_t ns) {
>>>    return ns / NANOSECS_PER_MILLISECOND;
>>> }
>>> inline int64_t millis_to_nanos(int64_t ms) {
>>>    return ms * NANOSECONDS_PER_MILLISEC;
>>> }
>>> Also, the names don't suggest time conversions, but potentially
>>> arbitrary unit conversions, e.g. between something in NANOUNITS and
>>> something in MILLIUNITS.
>>
>> They don't have to be time conversions - the calculation is unit-less in practice. The fact we have NANOSEC_PER_MILLISECOND et al is just an artifact of introducing those values for timeout calculations/conversions - it could just be NANOS_PER_MILLI etc
> 
> If they aren't time conversions then I don't think they should be
> using the time factors; just use NANOUNITS/MILLIUNITS.

Okay.


>>> Regarding this from the audit:
>>> --- begin ---
>>> ./share/gc/parallel/psParallelCompact.cpp: // os::javaTimeMillis() does not guarantee monotonicity.
>>> ...
>>> ./share/gc/shared/referenceProcessor.cpp: // os::javaTimeMillis() does not guarantee monotonicity.
>>> These are all describing why the subsequent code uses javaTimeNanos not javaTimeMillis.
>>> --- end ---
>>> Do we really still support platforms that don't have a monotonic
>>> clock?  I guess we appear to at least try.  But it's really wrong that
>>> callers of os::javaTimeNanos should even think they need to cope with
>>> that function being non-monotonic.
>>> Hm, I always thought System.nanoTime() was a monotonic clock, but I
>>> don't see any such guarantee. So I guess Java just doesn't have such a
>>> thing. Wow!
>>> So I guess none of this is really relevant to the change at hand after all.
>>
>> I think you read the comments the wrong way round. The code uses javaTimeNanos not javaTimeMillis because javaTimeMillis is not monotonic and the code wants a monotonic clock. These comments were mostly inserted when the incorrect use of javaTimeMillis was replaced with javaTimeNanos.
> 
> No, I did not read the comments the wrong way around.
> 
> I was pointing out that
> 
> (1) Some uses of os::javaTimeNanos attempt to cope with a possible
> lack of monotonicity, while some do not and may be assuming or even
> requiring monotonicity.  I'm wondering whether the coping attempts are
> dead code, or instead, the latter are bugs.  That is, do we require
> all platforms to provide a monotonic os::javaTimeNanos or not?
> Personally, I think in today's world we should make such a requirement.

I assume you are looking at code where we explicitly ensure that time 
never jumps backwards? This is historical. The sad fact is that over the 
past 20+ years we have seen bug after bug after bug in the OS and 
virtualization layers that caused the system monotonic clock source to 
actually not behave monotonically. These bugs were prevalent enough in 
some situations that we had to implement the workarounds in the JVM. 
These workarounds have had costs and have varied over time. Ensuring 
that javaTimeNanos() itself is guaranteed monotonic imposes too high a 
cost - see related discussions in:

https://bugs.openjdk.java.net/browse/JDK-6864866
https://bugs.openjdk.java.net/browse/JDK-6784100

regarding the situation on Solaris. In general we try to trust the 
system monotonic clock and deal with issues if they arise.

So in a bugfree world:
- os::javaTimeNanos is "guaranteed" monotonic (assuming the underlying 
monotonic clock exists on that platform)
- os::currentTimeMillis() is not monotonic as it is (required to be) 
subject to time-of-day adjustments (e.g. ntp)

> (2) There is no documented monotonicity guarantee for System.nanoTime,
> nor any hint of a possible issue there, which surprised me.

Again historical. Back in 2003 when we (JSR166 EG) specified this we 
also had to consider the implications for Java ME and the constrained 
environments in which it would execute, as well as the various 
environments supported by Java SE**. As a practical consideration it was 
necessary that a valid/conforming implementation of nanoTime() could be 
currentTimeMillis()*1000000 and without making heroic, and costly, 
efforts to ensure monotonicity - which was basically left as a quality 
of implementation issue.

** Windows versions without QueryPerformanceCounter; Linux 2.x kernels 
without CLOCK_MONOTONIC

Thanks,
David
----

> 

From david.holmes at oracle.com  Fri Jan 17 05:48:14 2020
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 17 Jan 2020 15:48:14 +1000
Subject: RFR (M): 8235741: Inappropriate uses of os::javaTimeMillis()
In-Reply-To: <f3e75dc8-4ee9-9fad-d5a3-94c0e813abd5@oracle.com>
References: <f99194ad-3dad-48db-d8d4-de0cc9161701@oracle.com>
 <160821DF-BE15-435C-BDBE-AB553F794590@oracle.com>
 <a04e7b9f-3fe8-01ea-9784-fcf21613340d@oracle.com>
 <EDE2324D-DA2D-4CF7-BAE0-EF59BEEB960B@oracle.com>
 <f3e75dc8-4ee9-9fad-d5a3-94c0e813abd5@oracle.com>
Message-ID: <50a1ee24-61a9-44fa-2590-69173765e9de@oracle.com>

I tried but failed to generate an incremental webrev at:

http://cr.openjdk.java.net/~dholmes/8235741/webrev.v3-incr/

I hadn't qfresh'd and so it contains v2 and v3 changes :(

Full webrev at:

http://cr.openjdk.java.net/~dholmes/8235741/webrev.v3/

Changes:
- conversion macros are now typed functions
- os_posix.cpp had a name clash so fixed that and updated to use the new 
conversion functions

Thanks,
David

On 17/01/2020 12:48 pm, David Holmes wrote:
> Hi Kim,
> 
> On 17/01/2020 11:39 am, Kim Barrett wrote:
>>> On Jan 15, 2020, at 2:12 AM, David Holmes <david.holmes at oracle.com> 
>>> wrote:
>>>> src/hotspot/share/utilities/globalDefinitions.hpp
>>>> ? 262 // time unit conversion macros
>>>> ? 263
>>>> ? 264 #define NANOS_TO_MILLIS(ns) ((ns) / NANOSECS_PER_MILLISEC)
>>>> ? 265 #define MILLIS_TO_NANOS(ms) ((ms) * NANOSECS_PER_MILLISEC)
>>>> Why are these macros, rather than (template) functions?
>>>
>>> Just because I just wanted a simple textual replacement to make it 
>>> clearer that I'm converting from millis to nanos or vice versa. I 
>>> reach for macros for such simple cases.
>>>
>>>> Also, depending on the type and value of ms, MILLIS_TO_NANOS could
>>>> easily overflow, e.g. if ms type is a 32 bit type with a value of more
>>>> than ~4 seconds.? (I checked the two uses, and they happen to be okay.)
>>>
>>> These are not trying to be mathematically sound. The conversion from 
>>> millis to nanos is used in two cases:
>>>
>>> 1. Converting a current timestamp in ms to ns. Unless the current 
>>> time is set far in the future I don't think we have any issue with 
>>> overflow of? such a value.
>>>
>>> 2. converting an elapsed time in ms to ns. These will be small values 
>>> so no overflow is possible.
>>
>> In this case, "not trying to be mathematically sound" is equivalent to
>> actively but subtly dangerous to use, because of possible overflow
>> leading to either UB or catastrophically wrong results.
>>
>> Consider "int millis = 10 * 1000;", e.g. 10 seconds worth of
>> milliseconds.? I hope you will agree that this falls under your case
>> 2, e.g. that 10 seconds is not an especially big interval.? I hope you
>> will also agree that declaring a variable that holds a millisecond
>> interval as an int is reasonable in some contexts. Applying
>> MILLIS_TO_NANOS with the jint-typed NANOSECS_PER_MILLISEC, and the
>> result about 1.4 seconds worth of nanoseconds, assuming the compiler
>> doesn't do something strange because of the UB on integer overflow.
> 
> So your concern is that someone will use the macros outside of existing 
> usage contexts (where time values always using 64-bit variables) and 
> thus introduce potential overflow bugs. Okay I will use the typed 
> functions.
> 
>> (This is all in addition to the Style Guide's admonishment to use
>> inline functions rather than macros.? Macros should only be used for
>> syntactic extensions.)
> 
> 
>>>> inline int64_t nanos_to_millis(int64_t ns) {
>>>> ?? return ns / NANOSECS_PER_MILLISECOND;
>>>> }
>>>> inline int64_t millis_to_nanos(int64_t ms) {
>>>> ?? return ms * NANOSECONDS_PER_MILLISEC;
>>>> }
>>>> Also, the names don't suggest time conversions, but potentially
>>>> arbitrary unit conversions, e.g. between something in NANOUNITS and
>>>> something in MILLIUNITS.
>>>
>>> They don't have to be time conversions - the calculation is unit-less 
>>> in practice. The fact we have NANOSEC_PER_MILLISECOND et al is just 
>>> an artifact of introducing those values for timeout 
>>> calculations/conversions - it could just be NANOS_PER_MILLI etc
>>
>> If they aren't time conversions then I don't think they should be
>> using the time factors; just use NANOUNITS/MILLIUNITS.
> 
> Okay.
> 
> 
>>>> Regarding this from the audit:
>>>> --- begin ---
>>>> ./share/gc/parallel/psParallelCompact.cpp: // os::javaTimeMillis() 
>>>> does not guarantee monotonicity.
>>>> ...
>>>> ./share/gc/shared/referenceProcessor.cpp: // os::javaTimeMillis() 
>>>> does not guarantee monotonicity.
>>>> These are all describing why the subsequent code uses javaTimeNanos 
>>>> not javaTimeMillis.
>>>> --- end ---
>>>> Do we really still support platforms that don't have a monotonic
>>>> clock?? I guess we appear to at least try.? But it's really wrong that
>>>> callers of os::javaTimeNanos should even think they need to cope with
>>>> that function being non-monotonic.
>>>> Hm, I always thought System.nanoTime() was a monotonic clock, but I
>>>> don't see any such guarantee. So I guess Java just doesn't have such a
>>>> thing. Wow!
>>>> So I guess none of this is really relevant to the change at hand 
>>>> after all.
>>>
>>> I think you read the comments the wrong way round. The code uses 
>>> javaTimeNanos not javaTimeMillis because javaTimeMillis is not 
>>> monotonic and the code wants a monotonic clock. These comments were 
>>> mostly inserted when the incorrect use of javaTimeMillis was replaced 
>>> with javaTimeNanos.
>>
>> No, I did not read the comments the wrong way around.
>>
>> I was pointing out that
>>
>> (1) Some uses of os::javaTimeNanos attempt to cope with a possible
>> lack of monotonicity, while some do not and may be assuming or even
>> requiring monotonicity.? I'm wondering whether the coping attempts are
>> dead code, or instead, the latter are bugs.? That is, do we require
>> all platforms to provide a monotonic os::javaTimeNanos or not?
>> Personally, I think in today's world we should make such a requirement.
> 
> I assume you are looking at code where we explicitly ensure that time 
> never jumps backwards? This is historical. The sad fact is that over the 
> past 20+ years we have seen bug after bug after bug in the OS and 
> virtualization layers that caused the system monotonic clock source to 
> actually not behave monotonically. These bugs were prevalent enough in 
> some situations that we had to implement the workarounds in the JVM. 
> These workarounds have had costs and have varied over time. Ensuring 
> that javaTimeNanos() itself is guaranteed monotonic imposes too high a 
> cost - see related discussions in:
> 
> https://bugs.openjdk.java.net/browse/JDK-6864866
> https://bugs.openjdk.java.net/browse/JDK-6784100
> 
> regarding the situation on Solaris. In general we try to trust the 
> system monotonic clock and deal with issues if they arise.
> 
> So in a bugfree world:
> - os::javaTimeNanos is "guaranteed" monotonic (assuming the underlying 
> monotonic clock exists on that platform)
> - os::currentTimeMillis() is not monotonic as it is (required to be) 
> subject to time-of-day adjustments (e.g. ntp)
> 
>> (2) There is no documented monotonicity guarantee for System.nanoTime,
>> nor any hint of a possible issue there, which surprised me.
> 
> Again historical. Back in 2003 when we (JSR166 EG) specified this we 
> also had to consider the implications for Java ME and the constrained 
> environments in which it would execute, as well as the various 
> environments supported by Java SE**. As a practical consideration it was 
> necessary that a valid/conforming implementation of nanoTime() could be 
> currentTimeMillis()*1000000 and without making heroic, and costly, 
> efforts to ensure monotonicity - which was basically left as a quality 
> of implementation issue.
> 
> ** Windows versions without QueryPerformanceCounter; Linux 2.x kernels 
> without CLOCK_MONOTONIC
> 
> Thanks,
> David
> ----
> 
>>

From matthias.baesken at sap.com  Fri Jan 17 06:47:36 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 17 Jan 2020 06:47:36 +0000
Subject: RFR [XXS]: 8237382: Cleanup the OPT_SPEED_SRC file list in
 JvmFeatures.gmk
Message-ID: <AM6PR02MB420001BB07A53C75A12C2B2093310@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello, please review this very small change .

It removes file that are not present any more from the  OPT_SPEED_SRC file list in JvmFeatures.gmk  .

( this is a list of files to be optimized for speed when we otherwise optimize for size  in the minimal-JVM build)


Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8237382

http://cr.openjdk.java.net/~mbaesken/webrevs/8237382.0/

Thanks, Matthias


From matthias.baesken at sap.com  Fri Jan 17 08:07:49 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 17 Jan 2020 08:07:49 +0000
Subject: RFR [XS]: 8237374: configuring with
 --with-jvm-variants=minimal,server makes cds disappear in server
Message-ID: <AM6PR02MB4200A055393D4CA7B3FA06E393310@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello, please review  this small patch .

When building 2 VM variants   minimal and server  in one build and using

  --with-jvm-variants=minimal,server

to configure this setup  , the build works nicely. But I notice that in the server VM, cds is removed.
Instead of

checking if cds should be enabled... yes

I get (with some tracing added ) :

configure: WARNING: ENABLE_CDS set to false because we found a minimal, core or zero JVM.
checking if cds should be enabled... no

The checks in hotspot.m4 disables cds by error for the server v,  because it matches to minimal  in the string  "minimal,server"  .

The configure option  --with-jvm-variants=minimal,server   enables  a  multi-JVM variants build  (variable BUILDING_MULTIPLE_JVM_VARIANTS in hotspot.m4) .
This   special build is only  supported  for   VALID_MULTIPLE_JVM_VARIANTS="server client minimal" .
So  we  better do not  disable  cds  in  a  BUILDING_MULTIPLE_JVM_VARIANTS - build   (means minimal + server/client ) ;   minimal has cds disabled by default anyway.


Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8237374


http://cr.openjdk.java.net/~mbaesken/webrevs/8237374.0/


Thanks, Matthias

From matthias.baesken at sap.com  Fri Jan 17 08:44:29 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 17 Jan 2020 08:44:29 +0000
Subject: RFR:  8237192: Generate stripped/public pdbs on Windows for jdk images
Message-ID: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello, please review this change related to stripped/"public" pdb file generation on Windows .

Currently the JDK bundle on Windows does not contain pdb files (full pdb files are in a separate symbols bundle).
This leads currently to bad native stack traces e.g. when crashes occur.
One reason not to deliver the full pdb files might be the large size of these files.

However there exist also "public" or stripped pdb files on Windows, see :

https://docs.microsoft.com/en-us/cpp/build/reference/pdbstripped-strip-private-symbols?view=vs-2017

Those are much smaller (often only 10-20% of the full pdb files) and they offer a good compromise (no "file:linenumber" info in the native stacks but at least the function name+hex-offset is visible)
to delivering full pdbs in the JDK.

Example sizes for the currently built full pdbs / stripped pdbs from VS2017 based 64bit build of jdk/jdk :
jvm.pdb : 73,1 MB / 9,46 MB
awt.pdb : 7,05 MB / 1,48 MB

The patch  adds  generation  of stripped  pdb files  to the Windows  build.
Additionally those files are put into the JDK  bundle    (while the symbols bundle still gets the full pdb files ) .


Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8237192

http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.0/


Thanks, Matthias

From magnus.ihse.bursie at oracle.com  Fri Jan 17 09:22:15 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Fri, 17 Jan 2020 10:22:15 +0100
Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk
 images
In-Reply-To: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <d4923478-38e3-60b5-9025-c9af9026c4d2@oracle.com>

On 2020-01-17 09:44, Baesken, Matthias wrote:
> Hello, please review this change related to stripped/"public" pdb file generation on Windows .
>
> Currently the JDK bundle on Windows does not contain pdb files (full pdb files are in a separate symbols bundle).
> This leads currently to bad native stack traces e.g. when crashes occur.
> One reason not to deliver the full pdb files might be the large size of these files.
>
> However there exist also "public" or stripped pdb files on Windows, see :
>
> https://docs.microsoft.com/en-us/cpp/build/reference/pdbstripped-strip-private-symbols?view=vs-2017
>
> Those are much smaller (often only 10-20% of the full pdb files) and they offer a good compromise (no "file:linenumber" info in the native stacks but at least the function name+hex-offset is visible)
> to delivering full pdbs in the JDK.
>
> Example sizes for the currently built full pdbs / stripped pdbs from VS2017 based 64bit build of jdk/jdk :
> jvm.pdb : 73,1 MB / 9,46 MB
> awt.pdb : 7,05 MB / 1,48 MB
>
> The patch  adds  generation  of stripped  pdb files  to the Windows  build.
> Additionally those files are put into the JDK  bundle    (while the symbols bundle still gets the full pdb files ) .
>
>
> Bug/webrev :
>
> https://bugs.openjdk.java.net/browse/JDK-8237192
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.0/

What is the extra payload of all the *.stripped.pdb files together?

/Magnus

>
>
> Thanks, Matthias


From matthias.baesken at sap.com  Fri Jan 17 09:25:01 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 17 Jan 2020 09:25:01 +0000
Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk
 images
In-Reply-To: <d4923478-38e3-60b5-9025-c9af9026c4d2@oracle.com>
References: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <d4923478-38e3-60b5-9025-c9af9026c4d2@oracle.com>
Message-ID: <AM6PR02MB420098C02FEC93014868B68A93310@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello,  my example product build (64 bit Windows / VS2017) shows the following sizes for the uncompressed pdb files :
sum of size of all full pdbs : 117 MB (jvm.pdb is 73,1 MB )
sum of size of all stripped pdbs: 18,2 MB (jvm.pdb is 9,46 MB = ~ 50 % of all)

Best regards, Matthias


On 2020-01-17 09:44, Baesken, Matthias wrote:

Hello, please review this change related to stripped/"public" pdb file generation on Windows .



Currently the JDK bundle on Windows does not contain pdb files (full pdb files are in a separate symbols bundle).

This leads currently to bad native stack traces e.g. when crashes occur.

One reason not to deliver the full pdb files might be the large size of these files.



However there exist also "public" or stripped pdb files on Windows, see :



https://docs.microsoft.com/en-us/cpp/build/reference/pdbstripped-strip-private-symbols?view=vs-2017



Those are much smaller (often only 10-20% of the full pdb files) and they offer a good compromise (no "file:linenumber"<file://linenumber> info in the native stacks but at least the function name+hex-offset is visible)

to delivering full pdbs in the JDK.



Example sizes for the currently built full pdbs / stripped pdbs from VS2017 based 64bit build of jdk/jdk :

jvm.pdb : 73,1 MB / 9,46 MB

awt.pdb : 7,05 MB / 1,48 MB



The patch  adds  generation  of stripped  pdb files  to the Windows  build.

Additionally those files are put into the JDK  bundle    (while the symbols bundle still gets the full pdb files ) .





Bug/webrev :



https://bugs.openjdk.java.net/browse/JDK-8237192



http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.0/

What is the extra payload of all the *.stripped.pdb files together?




From magnus.ihse.bursie at oracle.com  Fri Jan 17 09:31:06 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Fri, 17 Jan 2020 10:31:06 +0100
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
Message-ID: <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>

On 2020-01-16 10:30, David Holmes wrote:
> Hi Matthias,
>
> On 16/01/2020 6:10 pm, Baesken, Matthias wrote:
>> Hi David, sure we can introduce a way to switch this on/off.
>
> Thanks.
>
>> There is already something similar for the link-time optimization 
>> (flto) , see the feature
>>
>> JvmFeatures.gmk
>> 180 ifeq ($(call check-jvm-feature, link-time-opt), true)
>> 190 ifeq ($(call check-jvm-feature, link-time-opt), false)
>>
>> hotspot.m4
>> 29 static-build link-time-opt aot jfr"
>> 502 JVM_FEATURES_link_time_opt="link-time-opt"
>
> Yep familiar with that from Minimal VM and SE Embedded days :)
>
>> Should we have? "link-time-gc" additionally to?? " link-time-opt"? 
>> ??? (however it? would? be a? bit misleading?? that it is a "JVM" 
>> feature , but except linux s390x? it is only changing the build of 
>> the JDK libs) .
>
> I agree the definition of this as a "JVM" feature is a bit 
> odd/misleading. Perhaps the build folk have a suggestion on how to 
> refactor this kind of option into something more general? In the 
> meantime having link-time-gc sit alongside link-time-opt seems 
> acceptable to me.

We don't have the concept of "JDK features", akin to "JVM features". 
Maybe we should have. It's an idea worth exploring, anyway.

The way we currently do on/off features for the entire JDK is by using 
autoconf options. So, in this case, --enable-link-time-gc, or something 
like that. It might just as well be on by default, which gives us a 
--disable-link-time-gc instead. (I understand David's reservation not 
about this being the default, just that it is not possible to simply 
turn off.)

Matthias: Have a look at some recently added option to get an indication 
of the best practice in adding new options. There are some ways to 
easily make this incorrect, and our code is full of old examples that 
does this unnecessary complex, or downright wrong. (These should be 
fixed, and we should probably introduce a simpler API for doing this, 
and so on... I'll address those as soon as time permits.)

/Magnus


>
> Thanks,
> David
>
>> Best regards, Matthias
>>
>>
>>
>>>
>>> Hi Matthias,
>>>
>>> I have reservations about turning this on by default and with no way to
>>> control it. This seems like it should be something you have to 
>>> opt-in to
>>> initially while we gain some experience with it and ensure there are no
>>> unexpected side-effects. After that it could be enabled by default.
>>>
>>


From matthias.baesken at sap.com  Fri Jan 17 11:44:28 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 17 Jan 2020 11:44:28 +0000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
 <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>
Message-ID: <AM6PR02MB4200B721BD0C7728B21E2C4193310@AM6PR02MB4200.eurprd02.prod.outlook.com>



  *   Matthias: Have a look at some recently added option to get an indication of the best practice in adding new options. There are some ways to easily make this incorrect

Hi Magnus, do you have a good/?best practice?  example  (not that I catch a bad one ?? )  ?

Best regards, Matthias



On 2020-01-16 10:30, David Holmes wrote:
Hi Matthias,

On 16/01/2020 6:10 pm, Baesken, Matthias wrote:

Hi David, sure we can introduce a way to switch this on/off.

Thanks.


There is already something similar for the link-time optimization (flto) , see the feature

JvmFeatures.gmk
180 ifeq ($(call check-jvm-feature, link-time-opt), true)
190 ifeq ($(call check-jvm-feature, link-time-opt), false)

hotspot.m4
29 static-build link-time-opt aot jfr"
502 JVM_FEATURES_link_time_opt="link-time-opt"

Yep familiar with that from Minimal VM and SE Embedded days :)


Should we have  "link-time-gc"  additionally to   " link-time-opt"  ?   (however it  would  be a  bit misleading   that it is a "JVM" feature , but except linux s390x  it is only changing the build of the JDK libs) .

I agree the definition of this as a "JVM" feature is a bit odd/misleading. Perhaps the build folk have a suggestion on how to refactor this kind of option into something more general? In the meantime having link-time-gc sit alongside link-time-opt seems acceptable to me.

We don't have the concept of "JDK features", akin to "JVM features". Maybe we should have. It's an idea worth exploring, anyway.

The way we currently do on/off features for the entire JDK is by using autoconf options. So, in this case, --enable-link-time-gc, or something like that. It might just as well be on by default, which gives us a --disable-link-time-gc instead. (I understand David's reservation not about this being the default, just that it is not possible to simply turn off.)

Matthias: Have a look at some recently added option to get an indication of the best practice in adding new options. There are some ways to easily make this incorrect, and our code is full of old examples that does this unnecessary complex, or downright wrong. (These should be fixed, and we should probably introduce a simpler API for doing this, and so on... I'll address those as soon as time permits.)

/Magnus



From erik.joelsson at oracle.com  Fri Jan 17 14:09:05 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Fri, 17 Jan 2020 06:09:05 -0800
Subject: RFR [XXS]: 8237382: Cleanup the OPT_SPEED_SRC file list in
 JvmFeatures.gmk
In-Reply-To: <AM6PR02MB420001BB07A53C75A12C2B2093310@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB420001BB07A53C75A12C2B2093310@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <aaa794a1-80ac-41cf-880a-f9ae04bfb714@oracle.com>

Looks good.

/Erik

On 2020-01-16 22:47, Baesken, Matthias wrote:
> Hello, please review this very small change .
>
> It removes file that are not present any more from the  OPT_SPEED_SRC file list in JvmFeatures.gmk  .
>
> ( this is a list of files to be optimized for speed when we otherwise optimize for size  in the minimal-JVM build)
>
>
> Bug/webrev :
>
> https://bugs.openjdk.java.net/browse/JDK-8237382
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8237382.0/
>
> Thanks, Matthias
>

From erik.joelsson at oracle.com  Fri Jan 17 14:17:34 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Fri, 17 Jan 2020 06:17:34 -0800
Subject: RFR [XS]: 8237374: configuring with
 --with-jvm-variants=minimal,server makes cds disappear in server
In-Reply-To: <AM6PR02MB4200A055393D4CA7B3FA06E393310@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB4200A055393D4CA7B3FA06E393310@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <57f43f5a-ae37-5d98-e9e4-bb30c3607300@oracle.com>

Hello Matthias,

Using BUILDING_MULTIPLE_JVM_VARIANTS as condition is clever and happens 
to coincide with the set of variants that also support CDS, but I would 
say this correlation is incidental. I would still prefer an explicit 
test for if any of the variants that do support CDS is in the set of 
variants being built. This will make it much easier to read and 
understand the logic. Simply:

if ! HOTSPOT_CHECK_JVM_VARIANT(server) && ! 
HOTSPOT_CHECK_JVM_VARIANT(client); then
 ? ENABLE_CDS="false"
 ? ...

/Erik

On 2020-01-17 00:07, Baesken, Matthias wrote:
> Hello, please review  this small patch .
>
> When building 2 VM variants   minimal and server  in one build and using
>
>    --with-jvm-variants=minimal,server
>
> to configure this setup  , the build works nicely. But I notice that in the server VM, cds is removed.
> Instead of
>
> checking if cds should be enabled... yes
>
> I get (with some tracing added ) :
>
> configure: WARNING: ENABLE_CDS set to false because we found a minimal, core or zero JVM.
> checking if cds should be enabled... no
>
> The checks in hotspot.m4 disables cds by error for the server v,  because it matches to minimal  in the string  "minimal,server"  .
>
> The configure option  --with-jvm-variants=minimal,server   enables  a  multi-JVM variants build  (variable BUILDING_MULTIPLE_JVM_VARIANTS in hotspot.m4) .
> This   special build is only  supported  for   VALID_MULTIPLE_JVM_VARIANTS="server client minimal" .
> So  we  better do not  disable  cds  in  a  BUILDING_MULTIPLE_JVM_VARIANTS - build   (means minimal + server/client ) ;   minimal has cds disabled by default anyway.
>
>
> Bug/webrev :
>
> https://bugs.openjdk.java.net/browse/JDK-8237374
>
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8237374.0/
>
>
> Thanks, Matthias

From erik.joelsson at oracle.com  Fri Jan 17 15:20:09 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Fri, 17 Jan 2020 07:20:09 -0800
Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk
 images
In-Reply-To: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <eb0310f3-21e9-ae31-e797-542c17d9c858@oracle.com>

Hello Matthias,

Providing these stripped pdb files in the distribution is a good idea, 
but finding a good solution in the build is unfortunately more 
complicated than this. The JDK image we ship should (with very few 
exceptions) be the result of running jlink on all the jmods. If a user 
runs jlink and includes all the jmods we ship with the JDK, the result 
should be essentially equivalent to the original JDK image. The way the 
stripped pdb files are included in the bundles sort of at the last 
second of the build here breaks this property. Any user generated image 
would miss the stripped pdb files since they aren't packaged in the jmods.

To properly implement this, care will need to be taken to juggle the two 
sets of pdb files around, making sure each build and test use case has 
the correct one in place where and when it's needed. Quite possibly, we 
cannot cover all use cases with one build configuration. Developers 
needing the full debug symbols when debugging locally would likely need 
to disable the stripped symbols so they get the full symbols everywhere. 
Possibly this would need to be the default for debug builds and 
configurable for release builds.

/Erik

On 2020-01-17 00:44, Baesken, Matthias wrote:

> Hello, please review this change related to stripped/"public" pdb file generation on Windows .
>
> Currently the JDK bundle on Windows does not contain pdb files (full pdb files are in a separate symbols bundle).
> This leads currently to bad native stack traces e.g. when crashes occur.
> One reason not to deliver the full pdb files might be the large size of these files.
>
> However there exist also "public" or stripped pdb files on Windows, see :
>
> https://docs.microsoft.com/en-us/cpp/build/reference/pdbstripped-strip-private-symbols?view=vs-2017
>
> Those are much smaller (often only 10-20% of the full pdb files) and they offer a good compromise (no "file:linenumber" info in the native stacks but at least the function name+hex-offset is visible)
> to delivering full pdbs in the JDK.
>
> Example sizes for the currently built full pdbs / stripped pdbs from VS2017 based 64bit build of jdk/jdk :
> jvm.pdb : 73,1 MB / 9,46 MB
> awt.pdb : 7,05 MB / 1,48 MB
>
> The patch  adds  generation  of stripped  pdb files  to the Windows  build.
> Additionally those files are put into the JDK  bundle    (while the symbols bundle still gets the full pdb files ) .
>
>
> Bug/webrev :
>
> https://bugs.openjdk.java.net/browse/JDK-8237192
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.0/
>
>
> Thanks, Matthias

From daniel.daugherty at oracle.com  Fri Jan 17 15:37:51 2020
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Fri, 17 Jan 2020 10:37:51 -0500
Subject: RFR: 8235966: Process obsolete flags less aggressively
In-Reply-To: <6a3de8da-746d-7e5d-9a0e-2157a0963ed8@oracle.com>
References: <db63e5c9-a0c1-1575-e8dc-88429d6216fe@oracle.com>
 <674e4f0d-563f-3bb4-dee0-74aaacb9b270@oracle.com>
 <b32a2e50-62e3-ed94-7773-a6594a76e1aa@oracle.com>
 <c1a2f1e1-e86b-1fda-4632-ce9734e6e75f@oracle.com>
 <b8e70c4d-af76-a682-b4a9-839dd2671958@oracle.com>
 <6a3de8da-746d-7e5d-9a0e-2157a0963ed8@oracle.com>
Message-ID: <b7f7df86-50f0-523e-6cb2-e6b7d5fdddde@oracle.com>

On 1/16/20 12:57 AM, David Holmes wrote:
> Getting back to this ...

You added this update to the bug report:

> Update: after further discussion it has been proposed that we use the 
> build number as the trigger for a whitebox or gtest that performs the 
> currently disabled full verification of the flag table. So if a flag 
> has not be obsoleted or expired as it should by build N** then we fail 
> the gtest. This will complement the relaxing of the obsoletion check 
> at the start of the release cycle. 

I was expecting a new test in this latest webrev that would start failing
at Build 20... let me see what the latest webrev says...

>
> Please see updated webrev at:
>
> http://cr.openjdk.java.net/~dholmes/8235966/webrev/

src/hotspot/share/runtime/arguments.cpp
 ??? No comments.

So the changes are the same as the last round with the addition of
enabling the following at B20:

L755: ????????? warning("Global variable for obsolete special flag entry 
\"%s\" should be removed", flag.name);
L769: ????????? warning("Global variable for expired flag entry \"%s\" 
should be removed", flag.name);


> Apologies as I mistakenly overwrote the original instead of creating v3.
>
> This version expands on the original proposal by uncommenting the 
> warnings about obsolete/expired flags that have not been removed from 
> globals*.hpp, so that we don't forget to do this work. However these 
> warnings are only enabled from build 20. I used 20 as being approx 2/3 
> through the release cycle - long enough that the work should have been 
> performed by then, whilst still leaving time to perform the work 
> before RDP2. Of course we can tweak this number if there are issues 
> with that choice.

Okay... but doesn't this mean that every test would issue these warnings
as of B20 if we have not completed the work? So we have the potential of
a raft (some unknown number) of test failures due to unexpected output in
the form of these warning messages. And worse, these failures would be
unrelated to actual issue... :-(

How about adding a diagnostic flag that enables these two warning
messages (in addition to the B20 check). Add a single test that runs:

 ? java -XX:+UnlockDiagnosticVMOptions -XX:+FlagShouldNotBeDefinedCheck 
-version

and when we hit B20, if there are still flags that haven't been removed,
then the test will fail and we'll have one test that fails (X the number
of configs that run the test).

Dan


>
> Thanks,
> David
>
> On 20/12/2019 8:20 am, David Holmes wrote:
>> Thanks Dan.
>>
>> FTR I've updated the bug report with an extension to this proposal, 
>> which is to add back the flag table validation checks to use via a 
>> gtest that we only enable after a certain build in the release cycle 
>> (it always passes before then). That way we avoid the problems I've 
>> outlined with the initial version bump but also have a safety net in 
>> place to ensure we don't forget to actually obsolete/expire flags.
>>
>> Cheers,
>> David
>>
>> On 19/12/2019 3:37 am, Daniel D. Daugherty wrote:
>>> Hi David,
>>>
>>> On 12/17/19 5:03 PM, David Holmes wrote:
>>>> Hi Dan,
>>>>
>>>> Thanks for taking a look. Updated webrev:
>>>>
>>>> http://cr.openjdk.java.net/~dholmes/8235966/webrev.v2/
>>>
>>> src/hotspot/share/runtime/arguments.cpp
>>> ???? I like the updates to header comment for 
>>> verify_special_jvm_flags().
>>>
>>> Thumbs up.
>>>
>>>
>>>>
>>>> Discussion below.
>>>
>>> Replies below.
>>>
>>>
>>>>
>>>> On 18/12/2019 1:47 am, Daniel D. Daugherty wrote:
>>>>> On 12/16/19 12:16 AM, David Holmes wrote:
>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8235966
>>>>>> webrev: http://cr.openjdk.java.net/~dholmes/8235966/webrev/
>>>>>
>>>>> src/hotspot/share/runtime/arguments.cpp
>>>>> ???? L745: ????? // if flag has become obsolete it should not have 
>>>>> a "globals" flag defined anymore.
>>>>> ???? L746: ????? if (!version_less_than(JDK_Version::current(), 
>>>>> flag.obsolete_in)) {
>>>>> ???? L747: ??????? if (JVMFlag::find_declared_flag(flag.name) != 
>>>>> NULL) {
>>>>> ???? L748: ????????? // Temporarily disable the warning: 8196739
>>>>> ???? L749: ????????? // warning("Global variable for obsolete 
>>>>> special flag entry \"%s\" should be removed", flag.name);
>>>>> ???? L750: ??????? }
>>>>> ???? L751: ????? }
>>>>> ???????? It seems like we've been down a similar road before:
>>>>>
>>>>> ???????? JDK-8196739 Disable obsolete/expired VM flag transitional 
>>>>> warnings
>>>>> ???????? https://bugs.openjdk.java.net/browse/JDK-8196739
>>>>>
>>>>> ???????? This one may ring a bell... Fixed by dholmes in 
>>>>> jdk11-b01... :-)
>>>>>
>>>>> ???????? And this followup sub-task to re-enable that warning:
>>>>>
>>>>> ???????? JDK-8196741 Re-enable obsolete/expired VM flag 
>>>>> transitional warnings
>>>>> ???????? https://bugs.openjdk.java.net/browse/JDK-8196741
>>>>>
>>>>> ???????? was closed as "Won't fix" on 2019.08.02.
>>>>>
>>>>> ???????? So the obvious questions:
>>>>>
>>>>> ???????? - Why is the new warning less problematic to tests that 
>>>>> don't
>>>>> ?????????? tolerate unexpected output?
>>>>
>>>> Two different situations. The commented out warning happens 
>>>> unconditionally when you run the VM and it finds any flag marked 
>>>> obsolete that hasn't been removed. Hence every single test will 
>>>> encounter this warning.
>>>
>>> Ouch on such verbosity.
>>>
>>>
>>>> The situation I am modifying is when a test uses a flag that is 
>>>> marked for obsoletion. In the majority of cases the flag is already 
>>>> deprecated and so already issuing a deprecation warning that the 
>>>> test has to handle. Without my change there would still be an 
>>>> obsoletion warning, so this test is in for a warning no matter what.
>>>
>>> Good that your change only comes into play when the flag is used.
>>>
>>>
>>>> Also note that for hotspot at least we have strived to make tests 
>>>> tolerate unexpected output. The reason JDK-8196741 was closed as 
>>>> "won't fix" was because other areas wouldn't commit to doing that.
>>>
>>> Yup. Got it.
>>>
>>>
>>>>
>>>>> ???????? - If you move forward with this fix, then I think think code
>>>>> ?????????? block needs to be removed or modified or am I missing 
>>>>> something?
>>>>
>>>> I've rewritten the comment at the head of verify_special_jvm_flags 
>>>> to explain why we can't issue a warning, and have deleted the block.
>>>
>>> Thanks for deleting the stale code.
>>>
>>>
>>>>
>>>>> ???????? There's a similar commented out check on L757-L765, but 
>>>>> that one
>>>>> ???????? is for an expired flag... You might want to adjust/delete 
>>>>> it also?
>>>>
>>>> Deleted.
>>>
>>> Thanks.
>>>
>>>
>>>>
>>>>> ???? L753: ??????? warning("Special flag entry \"%s\" must be 
>>>>> explicitly obsoleted before expired.", flag.name);
>>>>> ???? L754: ??????? success = false;
>>>>> ???????? nit - s/before expired/before being expired/
>>>>> ???????? Update: I now see that "style" is in several places in this
>>>>> ???????????? function. I'm not sure what to think here... it grates,
>>>>> ?? ? ? ? ? ? but I can live with it.
>>>>>
>>>>> ???????? nit - L75[34] indented too much by two spaces.
>>>>
>>>> Fixed.
>>>>
>>>>> ???? L962: ????????? return real_name;
>>>>> ???????? nit - indented too much by two spaces.
>>>>
>>>> Fixed.
>>>>
>>>>>
>>>>> Trying to understand the modified logic in argument processing is
>>>>> making my head spin...
>>>>
>>>> Mine too. It took a few attempts to put the logic in the right 
>>>> place and make adjustments so that it all works as expected for a 
>>>> correctly specified flag and an erroneous one.
>>>
>>> I keep trying to convince myself that we're improving this flag and
>>> options code with each release... :-)
>>>
>>>
>>>>
>>>>> - You've added a call to is_obsolete_flag() in
>>>>> ?? handle_aliases_and_deprecation() and is_obsolete_flag()
>>>>> ?? is where the new warning is output:
>>>>>
>>>>> ???? warning("Temporarily processing option %s; support is 
>>>>> scheduled for removal in %s"
>>>>>
>>>>> ?? handle_aliases_and_deprecation() is called from six different 
>>>>> places,
>>>>> ?? but the call sites are different based on the argument pattern 
>>>>> so I
>>>>> ?? have (mostly) convinced myself that there should not be any 
>>>>> duplicate
>>>>> ?? warning lines.
>>>>
>>>> Right - handle_aliases_and_deprecation is only called for a 
>>>> syntactically correct flag based on those patterns. It normally 
>>>> filters out obsoleted/expired flags and lets them fall through to 
>>>> later error processing (in process_argument after parse_arg returns 
>>>> false). That error processing is where the normal obsoletion check 
>>>> is performed. So I had to not filter the flag in 
>>>> handle_aliases_and_deprecation in this case, but still produce the 
>>>> warning for a malformed flag. E.g.
>>>>
>>>> java -XX:+UseParallelOldGC -version
>>>> Java HotSpot(TM) 64-Bit Server VM warning: Temporarily processing 
>>>> option UseParallelOldGC; support is scheduled for removal in 15.0
>>>> java version "15-internal" 2020-09-15
>>>>
>>>> java -XX:UseParallelOldGC -version
>>>> Java HotSpot(TM) 64-Bit Server VM warning: Temporarily processing 
>>>> option UseParallelOldGC; support is scheduled for removal in 15.0
>>>> Missing +/- setting for VM option 'UseParallelOldGC'
>>>> Error: Could not create the Java Virtual Machine.
>>>
>>> Thanks for the example. That helps a lot.
>>>
>>>
>>>>
>>>>> So I now understand the new logic that allows an obsoleted option
>>>>> to be specified with a warning as long as the option still exists.
>>>>> I'm good with the technical change, but...
>>>>>
>>>>> I'm worried about tests that don't tolerate the new warning mesg,
>>>>> i.e., why wouldn't this become an issue again:
>>>>>
>>>>> JDK-8196739 Disable obsolete/expired VM flag transitional warnings
>>>>
>>>> Explained above.
>>>
>>> Yup and thanks.
>>>
>>> Dan
>>>
>>>
>>>>
>>>> Thanks,
>>>> David
>>>>
>>>>> Dan
>>>>>
>>>>>
>>>>>>
>>>>>> When a flag is marked as obsolete in the special-flags table we 
>>>>>> will ignore it and issue a warning that it is being ignored, as 
>>>>>> soon as we bump the version of the JDK. That means that any tests 
>>>>>> still using the obsolete flag may start to fail, leading to a 
>>>>>> surge of test failures at the start of a release cycle. For 
>>>>>> example for JDK 15 we have a whole bunch of JFR tests that fail 
>>>>>> because they still try to work with UseParallelOldGC. In another 
>>>>>> case runtime/cds/appcds/FieldLayoutFlags.java passes only be 
>>>>>> accident.
>>>>>>
>>>>>> When a flag is marked as obsolete for a release, all code 
>>>>>> involving that flag (including tests that use it) must be updated 
>>>>>> within that release and the flag itself removed. Whilst this is 
>>>>>> typically scheduled early in a release cycle it isn't reasonable 
>>>>>> to expect it to all occur within the first couple of days of the 
>>>>>> release cycle, nor do we want to have to ProblemList a bunch of 
>>>>>> tests when they start failing.
>>>>>>
>>>>>> What I propose is to instead allow an obsolete flag to continue 
>>>>>> to be processed as long as that code removal has not actually 
>>>>>> occurred - with an adjusted warning. The change I propose:
>>>>>>
>>>>>> - only treats an obsolete flag as obsolete if the flag cannot be 
>>>>>> found
>>>>>> - added a new flag verification rule that disallows obsoletion in 
>>>>>> an undefined version, but expiration in a specific version i.e. 
>>>>>> we must always explicitly obsolete a flag before we expire it.
>>>>>>
>>>>>> The only downside here is that if we actually forget to file an 
>>>>>> issue for the actual obsoletion work we may not notice via 
>>>>>> testing. Of course whenever a change is made to the flags table 
>>>>>> to add an entry then the issue to do the obsoletion should be 
>>>>>> filed at the same time.
>>>>>>
>>>>>> Thanks,
>>>>>> David
>>>>>> -----
>>>>>>
>>>>>
>>>


From david.holmes at oracle.com  Fri Jan 17 22:25:16 2020
From: david.holmes at oracle.com (David Holmes)
Date: Sat, 18 Jan 2020 08:25:16 +1000
Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk
 images
In-Reply-To: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com>

Hi Matthias,

This also needs to be a configurable option not one done by default as 
there can be non-technical issues relating to shipping symbol files in a 
product.

Thanks,
David

On 17/01/2020 6:44 pm, Baesken, Matthias wrote:
> Hello, please review this change related to stripped/"public" pdb file generation on Windows .
> 
> Currently the JDK bundle on Windows does not contain pdb files (full pdb files are in a separate symbols bundle).
> This leads currently to bad native stack traces e.g. when crashes occur.
> One reason not to deliver the full pdb files might be the large size of these files.
> 
> However there exist also "public" or stripped pdb files on Windows, see :
> 
> https://docs.microsoft.com/en-us/cpp/build/reference/pdbstripped-strip-private-symbols?view=vs-2017
> 
> Those are much smaller (often only 10-20% of the full pdb files) and they offer a good compromise (no "file:linenumber" info in the native stacks but at least the function name+hex-offset is visible)
> to delivering full pdbs in the JDK.
> 
> Example sizes for the currently built full pdbs / stripped pdbs from VS2017 based 64bit build of jdk/jdk :
> jvm.pdb : 73,1 MB / 9,46 MB
> awt.pdb : 7,05 MB / 1,48 MB
> 
> The patch  adds  generation  of stripped  pdb files  to the Windows  build.
> Additionally those files are put into the JDK  bundle    (while the symbols bundle still gets the full pdb files ) .
> 
> 
> Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8237192
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.0/
> 
> 
> Thanks, Matthias
> 

From sgehwolf at redhat.com  Sat Jan 18 20:57:55 2020
From: sgehwolf at redhat.com (Severin Gehwolf)
Date: Sat, 18 Jan 2020 21:57:55 +0100
Subject: [15] RFR(XS): 8237479: 8230305 causes slowdebug build failure
Message-ID: <ad05c980e1adde8bb13216069a9dd3910e7bbd64.camel@redhat.com>

Hi,

My push of hotspot's cgroup v2 support (JDK-8230305) broke the
slowdebug build. Sorry about this :-/ The issue is that
CgroupController and CgroupSubsystem are abstract classes and I missed
to declare virtual functions as pure.

This patch fixes that.

Bug: https://bugs.openjdk.java.net/browse/JDK-8237479
webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8237479/01/webrev/

Testing: slowdebug/fastdebug/release builds work on Linux x86_64.
         Currently running through jdk/submit.

Thoughts?

Thanks,
Severin


From david.holmes at oracle.com  Sun Jan 19 00:07:55 2020
From: david.holmes at oracle.com (David Holmes)
Date: Sun, 19 Jan 2020 10:07:55 +1000
Subject: [15] RFR(XS): 8237479: 8230305 causes slowdebug build failure
In-Reply-To: <ad05c980e1adde8bb13216069a9dd3910e7bbd64.camel@redhat.com>
References: <ad05c980e1adde8bb13216069a9dd3910e7bbd64.camel@redhat.com>
Message-ID: <9405f2f3-d573-0b79-8f92-77f7e221d433@oracle.com>

Hi Severin,

Looks fine and trivial.

Thanks,
David

On 19/01/2020 6:57 am, Severin Gehwolf wrote:
> Hi,
> 
> My push of hotspot's cgroup v2 support (JDK-8230305) broke the
> slowdebug build. Sorry about this :-/ The issue is that
> CgroupController and CgroupSubsystem are abstract classes and I missed
> to declare virtual functions as pure.
> 
> This patch fixes that.
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8237479
> webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8237479/01/webrev/
> 
> Testing: slowdebug/fastdebug/release builds work on Linux x86_64.
>           Currently running through jdk/submit.
> 
> Thoughts?
> 
> Thanks,
> Severin
> 

From daniel.daugherty at oracle.com  Sun Jan 19 14:16:32 2020
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Sun, 19 Jan 2020 09:16:32 -0500
Subject: [15] RFR(XS): 8237479: 8230305 causes slowdebug build failure
In-Reply-To: <9405f2f3-d573-0b79-8f92-77f7e221d433@oracle.com>
References: <ad05c980e1adde8bb13216069a9dd3910e7bbd64.camel@redhat.com>
 <9405f2f3-d573-0b79-8f92-77f7e221d433@oracle.com>
Message-ID: <c5f22c29-585f-fa62-4850-d1920dad3326@oracle.com>

Thumbs up!

Please push as soon as you can to unbreak our CI's Tier2 builds. Thanks!

Dan


On 1/18/20 7:07 PM, David Holmes wrote:
> Hi Severin,
>
> Looks fine and trivial.
>
> Thanks,
> David
>
> On 19/01/2020 6:57 am, Severin Gehwolf wrote:
>> Hi,
>>
>> My push of hotspot's cgroup v2 support (JDK-8230305) broke the
>> slowdebug build. Sorry about this :-/ The issue is that
>> CgroupController and CgroupSubsystem are abstract classes and I missed
>> to declare virtual functions as pure.
>>
>> This patch fixes that.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8237479
>> webrev: 
>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8237479/01/webrev/
>>
>> Testing: slowdebug/fastdebug/release builds work on Linux x86_64.
>> ????????? Currently running through jdk/submit.
>>
>> Thoughts?
>>
>> Thanks,
>> Severin
>>


From sgehwolf at redhat.com  Sun Jan 19 19:38:13 2020
From: sgehwolf at redhat.com (Severin Gehwolf)
Date: Sun, 19 Jan 2020 20:38:13 +0100
Subject: [15] RFR(XS): 8237479: 8230305 causes slowdebug build failure
In-Reply-To: <c5f22c29-585f-fa62-4850-d1920dad3326@oracle.com>
References: <ad05c980e1adde8bb13216069a9dd3910e7bbd64.camel@redhat.com>
 <9405f2f3-d573-0b79-8f92-77f7e221d433@oracle.com>
 <c5f22c29-585f-fa62-4850-d1920dad3326@oracle.com>
Message-ID: <48cfeafd45448bda4232382dc540cf5df0118a5f.camel@redhat.com>

Thanks for the quick reviews! Pushed.

Cheers,
Severin

On Sun, 2020-01-19 at 09:16 -0500, Daniel D. Daugherty wrote:
> Thumbs up!
> 
> Please push as soon as you can to unbreak our CI's Tier2 builds.
> Thanks!
> 
> Dan
> 
> 
> On 1/18/20 7:07 PM, David Holmes wrote:
> > Hi Severin,
> > 
> > Looks fine and trivial.
> > 
> > Thanks,
> > David
> > 
> > On 19/01/2020 6:57 am, Severin Gehwolf wrote:
> > > Hi,
> > > 
> > > My push of hotspot's cgroup v2 support (JDK-8230305) broke the
> > > slowdebug build. Sorry about this :-/ The issue is that
> > > CgroupController and CgroupSubsystem are abstract classes and I
> > > missed
> > > to declare virtual functions as pure.
> > > 
> > > This patch fixes that.
> > > 
> > > Bug: https://bugs.openjdk.java.net/browse/JDK-8237479
> > > webrev: 
> > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8237479/01/webrev/
> > > 
> > > Testing: slowdebug/fastdebug/release builds work on Linux x86_64.
> > >           Currently running through jdk/submit.
> > > 
> > > Thoughts?
> > > 
> > > Thanks,
> > > Severin
> > > 


From david.holmes at oracle.com  Sun Jan 19 21:46:53 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 20 Jan 2020 07:46:53 +1000
Subject: RFR: 8235966: Process obsolete flags less aggressively
In-Reply-To: <b7f7df86-50f0-523e-6cb2-e6b7d5fdddde@oracle.com>
References: <db63e5c9-a0c1-1575-e8dc-88429d6216fe@oracle.com>
 <674e4f0d-563f-3bb4-dee0-74aaacb9b270@oracle.com>
 <b32a2e50-62e3-ed94-7773-a6594a76e1aa@oracle.com>
 <c1a2f1e1-e86b-1fda-4632-ce9734e6e75f@oracle.com>
 <b8e70c4d-af76-a682-b4a9-839dd2671958@oracle.com>
 <6a3de8da-746d-7e5d-9a0e-2157a0963ed8@oracle.com>
 <b7f7df86-50f0-523e-6cb2-e6b7d5fdddde@oracle.com>
Message-ID: <25156ff2-9dd2-53ae-16bc-83d60c0afb1a@oracle.com>

Hi Dan,

On 18/01/2020 1:37 am, Daniel D. Daugherty wrote:
> On 1/16/20 12:57 AM, David Holmes wrote:
>> Getting back to this ...
> 
> You added this update to the bug report:
> 
>> Update: after further discussion it has been proposed that we use the 
>> build number as the trigger for a whitebox or gtest that performs the 
>> currently disabled full verification of the flag table. So if a flag 
>> has not be obsoleted or expired as it should by build N** then we fail 
>> the gtest. This will complement the relaxing of the obsoletion check 
>> at the start of the release cycle. 
> 
> I was expecting a new test in this latest webrev that would start failing
> at Build 20... let me see what the latest webrev says...

Mea culpa. When I started thinking about the test it was evident the 
test logic would just involve the existing commented out warning logic. 
So I thought lets just turn those warnings on at build 20 but ...

>>
>> Please see updated webrev at:
>>
>> http://cr.openjdk.java.net/~dholmes/8235966/webrev/
> 
> src/hotspot/share/runtime/arguments.cpp
>  ??? No comments.
> 
> So the changes are the same as the last round with the addition of
> enabling the following at B20:
> 
> L755: ????????? warning("Global variable for obsolete special flag entry 
> \"%s\" should be removed", flag.name);
> L769: ????????? warning("Global variable for expired flag entry \"%s\" 
> should be removed", flag.name);
> 
> 
>> Apologies as I mistakenly overwrote the original instead of creating v3.
>>
>> This version expands on the original proposal by uncommenting the 
>> warnings about obsolete/expired flags that have not been removed from 
>> globals*.hpp, so that we don't forget to do this work. However these 
>> warnings are only enabled from build 20. I used 20 as being approx 2/3 
>> through the release cycle - long enough that the work should have been 
>> performed by then, whilst still leaving time to perform the work 
>> before RDP2. Of course we can tweak this number if there are issues 
>> with that choice.
> 
> Okay... but doesn't this mean that every test would issue these warnings
> as of B20 if we have not completed the work? So we have the potential of
> a raft (some unknown number) of test failures due to unexpected output in
> the form of these warning messages. And worse, these failures would be
> unrelated to actual issue... :-(

... as you note the end result is not really what we want - a single 
clearly failing test.

> How about adding a diagnostic flag that enables these two warning
> messages (in addition to the B20 check). Add a single test that runs:
> 
>  ? java -XX:+UnlockDiagnosticVMOptions -XX:+FlagShouldNotBeDefinedCheck 
> -version
> 
> and when we hit B20, if there are still flags that haven't been removed,
> then the test will fail and we'll have one test that fails (X the number
> of configs that run the test).

I definitely do not want to add another VM flag :) but will look at how 
to (re)run that logic as part of a gtest.

Thanks,
David

> Dan
> 
> 
>>
>> Thanks,
>> David
>>
>> On 20/12/2019 8:20 am, David Holmes wrote:
>>> Thanks Dan.
>>>
>>> FTR I've updated the bug report with an extension to this proposal, 
>>> which is to add back the flag table validation checks to use via a 
>>> gtest that we only enable after a certain build in the release cycle 
>>> (it always passes before then). That way we avoid the problems I've 
>>> outlined with the initial version bump but also have a safety net in 
>>> place to ensure we don't forget to actually obsolete/expire flags.
>>>
>>> Cheers,
>>> David
>>>
>>> On 19/12/2019 3:37 am, Daniel D. Daugherty wrote:
>>>> Hi David,
>>>>
>>>> On 12/17/19 5:03 PM, David Holmes wrote:
>>>>> Hi Dan,
>>>>>
>>>>> Thanks for taking a look. Updated webrev:
>>>>>
>>>>> http://cr.openjdk.java.net/~dholmes/8235966/webrev.v2/
>>>>
>>>> src/hotspot/share/runtime/arguments.cpp
>>>> ???? I like the updates to header comment for 
>>>> verify_special_jvm_flags().
>>>>
>>>> Thumbs up.
>>>>
>>>>
>>>>>
>>>>> Discussion below.
>>>>
>>>> Replies below.
>>>>
>>>>
>>>>>
>>>>> On 18/12/2019 1:47 am, Daniel D. Daugherty wrote:
>>>>>> On 12/16/19 12:16 AM, David Holmes wrote:
>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8235966
>>>>>>> webrev: http://cr.openjdk.java.net/~dholmes/8235966/webrev/
>>>>>>
>>>>>> src/hotspot/share/runtime/arguments.cpp
>>>>>> ???? L745: ????? // if flag has become obsolete it should not have 
>>>>>> a "globals" flag defined anymore.
>>>>>> ???? L746: ????? if (!version_less_than(JDK_Version::current(), 
>>>>>> flag.obsolete_in)) {
>>>>>> ???? L747: ??????? if (JVMFlag::find_declared_flag(flag.name) != 
>>>>>> NULL) {
>>>>>> ???? L748: ????????? // Temporarily disable the warning: 8196739
>>>>>> ???? L749: ????????? // warning("Global variable for obsolete 
>>>>>> special flag entry \"%s\" should be removed", flag.name);
>>>>>> ???? L750: ??????? }
>>>>>> ???? L751: ????? }
>>>>>> ???????? It seems like we've been down a similar road before:
>>>>>>
>>>>>> ???????? JDK-8196739 Disable obsolete/expired VM flag transitional 
>>>>>> warnings
>>>>>> ???????? https://bugs.openjdk.java.net/browse/JDK-8196739
>>>>>>
>>>>>> ???????? This one may ring a bell... Fixed by dholmes in 
>>>>>> jdk11-b01... :-)
>>>>>>
>>>>>> ???????? And this followup sub-task to re-enable that warning:
>>>>>>
>>>>>> ???????? JDK-8196741 Re-enable obsolete/expired VM flag 
>>>>>> transitional warnings
>>>>>> ???????? https://bugs.openjdk.java.net/browse/JDK-8196741
>>>>>>
>>>>>> ???????? was closed as "Won't fix" on 2019.08.02.
>>>>>>
>>>>>> ???????? So the obvious questions:
>>>>>>
>>>>>> ???????? - Why is the new warning less problematic to tests that 
>>>>>> don't
>>>>>> ?????????? tolerate unexpected output?
>>>>>
>>>>> Two different situations. The commented out warning happens 
>>>>> unconditionally when you run the VM and it finds any flag marked 
>>>>> obsolete that hasn't been removed. Hence every single test will 
>>>>> encounter this warning.
>>>>
>>>> Ouch on such verbosity.
>>>>
>>>>
>>>>> The situation I am modifying is when a test uses a flag that is 
>>>>> marked for obsoletion. In the majority of cases the flag is already 
>>>>> deprecated and so already issuing a deprecation warning that the 
>>>>> test has to handle. Without my change there would still be an 
>>>>> obsoletion warning, so this test is in for a warning no matter what.
>>>>
>>>> Good that your change only comes into play when the flag is used.
>>>>
>>>>
>>>>> Also note that for hotspot at least we have strived to make tests 
>>>>> tolerate unexpected output. The reason JDK-8196741 was closed as 
>>>>> "won't fix" was because other areas wouldn't commit to doing that.
>>>>
>>>> Yup. Got it.
>>>>
>>>>
>>>>>
>>>>>> ???????? - If you move forward with this fix, then I think think code
>>>>>> ?????????? block needs to be removed or modified or am I missing 
>>>>>> something?
>>>>>
>>>>> I've rewritten the comment at the head of verify_special_jvm_flags 
>>>>> to explain why we can't issue a warning, and have deleted the block.
>>>>
>>>> Thanks for deleting the stale code.
>>>>
>>>>
>>>>>
>>>>>> ???????? There's a similar commented out check on L757-L765, but 
>>>>>> that one
>>>>>> ???????? is for an expired flag... You might want to adjust/delete 
>>>>>> it also?
>>>>>
>>>>> Deleted.
>>>>
>>>> Thanks.
>>>>
>>>>
>>>>>
>>>>>> ???? L753: ??????? warning("Special flag entry \"%s\" must be 
>>>>>> explicitly obsoleted before expired.", flag.name);
>>>>>> ???? L754: ??????? success = false;
>>>>>> ???????? nit - s/before expired/before being expired/
>>>>>> ???????? Update: I now see that "style" is in several places in this
>>>>>> ???????????? function. I'm not sure what to think here... it grates,
>>>>>> ?? ? ? ? ? ? but I can live with it.
>>>>>>
>>>>>> ???????? nit - L75[34] indented too much by two spaces.
>>>>>
>>>>> Fixed.
>>>>>
>>>>>> ???? L962: ????????? return real_name;
>>>>>> ???????? nit - indented too much by two spaces.
>>>>>
>>>>> Fixed.
>>>>>
>>>>>>
>>>>>> Trying to understand the modified logic in argument processing is
>>>>>> making my head spin...
>>>>>
>>>>> Mine too. It took a few attempts to put the logic in the right 
>>>>> place and make adjustments so that it all works as expected for a 
>>>>> correctly specified flag and an erroneous one.
>>>>
>>>> I keep trying to convince myself that we're improving this flag and
>>>> options code with each release... :-)
>>>>
>>>>
>>>>>
>>>>>> - You've added a call to is_obsolete_flag() in
>>>>>> ?? handle_aliases_and_deprecation() and is_obsolete_flag()
>>>>>> ?? is where the new warning is output:
>>>>>>
>>>>>> ???? warning("Temporarily processing option %s; support is 
>>>>>> scheduled for removal in %s"
>>>>>>
>>>>>> ?? handle_aliases_and_deprecation() is called from six different 
>>>>>> places,
>>>>>> ?? but the call sites are different based on the argument pattern 
>>>>>> so I
>>>>>> ?? have (mostly) convinced myself that there should not be any 
>>>>>> duplicate
>>>>>> ?? warning lines.
>>>>>
>>>>> Right - handle_aliases_and_deprecation is only called for a 
>>>>> syntactically correct flag based on those patterns. It normally 
>>>>> filters out obsoleted/expired flags and lets them fall through to 
>>>>> later error processing (in process_argument after parse_arg returns 
>>>>> false). That error processing is where the normal obsoletion check 
>>>>> is performed. So I had to not filter the flag in 
>>>>> handle_aliases_and_deprecation in this case, but still produce the 
>>>>> warning for a malformed flag. E.g.
>>>>>
>>>>> java -XX:+UseParallelOldGC -version
>>>>> Java HotSpot(TM) 64-Bit Server VM warning: Temporarily processing 
>>>>> option UseParallelOldGC; support is scheduled for removal in 15.0
>>>>> java version "15-internal" 2020-09-15
>>>>>
>>>>> java -XX:UseParallelOldGC -version
>>>>> Java HotSpot(TM) 64-Bit Server VM warning: Temporarily processing 
>>>>> option UseParallelOldGC; support is scheduled for removal in 15.0
>>>>> Missing +/- setting for VM option 'UseParallelOldGC'
>>>>> Error: Could not create the Java Virtual Machine.
>>>>
>>>> Thanks for the example. That helps a lot.
>>>>
>>>>
>>>>>
>>>>>> So I now understand the new logic that allows an obsoleted option
>>>>>> to be specified with a warning as long as the option still exists.
>>>>>> I'm good with the technical change, but...
>>>>>>
>>>>>> I'm worried about tests that don't tolerate the new warning mesg,
>>>>>> i.e., why wouldn't this become an issue again:
>>>>>>
>>>>>> JDK-8196739 Disable obsolete/expired VM flag transitional warnings
>>>>>
>>>>> Explained above.
>>>>
>>>> Yup and thanks.
>>>>
>>>> Dan
>>>>
>>>>
>>>>>
>>>>> Thanks,
>>>>> David
>>>>>
>>>>>> Dan
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> When a flag is marked as obsolete in the special-flags table we 
>>>>>>> will ignore it and issue a warning that it is being ignored, as 
>>>>>>> soon as we bump the version of the JDK. That means that any tests 
>>>>>>> still using the obsolete flag may start to fail, leading to a 
>>>>>>> surge of test failures at the start of a release cycle. For 
>>>>>>> example for JDK 15 we have a whole bunch of JFR tests that fail 
>>>>>>> because they still try to work with UseParallelOldGC. In another 
>>>>>>> case runtime/cds/appcds/FieldLayoutFlags.java passes only be 
>>>>>>> accident.
>>>>>>>
>>>>>>> When a flag is marked as obsolete for a release, all code 
>>>>>>> involving that flag (including tests that use it) must be updated 
>>>>>>> within that release and the flag itself removed. Whilst this is 
>>>>>>> typically scheduled early in a release cycle it isn't reasonable 
>>>>>>> to expect it to all occur within the first couple of days of the 
>>>>>>> release cycle, nor do we want to have to ProblemList a bunch of 
>>>>>>> tests when they start failing.
>>>>>>>
>>>>>>> What I propose is to instead allow an obsolete flag to continue 
>>>>>>> to be processed as long as that code removal has not actually 
>>>>>>> occurred - with an adjusted warning. The change I propose:
>>>>>>>
>>>>>>> - only treats an obsolete flag as obsolete if the flag cannot be 
>>>>>>> found
>>>>>>> - added a new flag verification rule that disallows obsoletion in 
>>>>>>> an undefined version, but expiration in a specific version i.e. 
>>>>>>> we must always explicitly obsolete a flag before we expire it.
>>>>>>>
>>>>>>> The only downside here is that if we actually forget to file an 
>>>>>>> issue for the actual obsoletion work we may not notice via 
>>>>>>> testing. Of course whenever a change is made to the flags table 
>>>>>>> to add an entry then the issue to do the obsoletion should be 
>>>>>>> filed at the same time.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> David
>>>>>>> -----
>>>>>>>
>>>>>>
>>>>
> 

From daniel.daugherty at oracle.com  Sun Jan 19 22:09:37 2020
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Sun, 19 Jan 2020 17:09:37 -0500
Subject: [15] RFR(XS): 8237479: 8230305 causes slowdebug build failure
In-Reply-To: <48cfeafd45448bda4232382dc540cf5df0118a5f.camel@redhat.com>
References: <ad05c980e1adde8bb13216069a9dd3910e7bbd64.camel@redhat.com>
 <9405f2f3-d573-0b79-8f92-77f7e221d433@oracle.com>
 <c5f22c29-585f-fa62-4850-d1920dad3326@oracle.com>
 <48cfeafd45448bda4232382dc540cf5df0118a5f.camel@redhat.com>
Message-ID: <f61038c7-cfb1-452f-e3ea-6c2c592f878e@oracle.com>

Thanks. Tier2 passed...

Dan


On 1/19/20 2:38 PM, Severin Gehwolf wrote:
> Thanks for the quick reviews! Pushed.
>
> Cheers,
> Severin
>
> On Sun, 2020-01-19 at 09:16 -0500, Daniel D. Daugherty wrote:
>> Thumbs up!
>>
>> Please push as soon as you can to unbreak our CI's Tier2 builds.
>> Thanks!
>>
>> Dan
>>
>>
>> On 1/18/20 7:07 PM, David Holmes wrote:
>>> Hi Severin,
>>>
>>> Looks fine and trivial.
>>>
>>> Thanks,
>>> David
>>>
>>> On 19/01/2020 6:57 am, Severin Gehwolf wrote:
>>>> Hi,
>>>>
>>>> My push of hotspot's cgroup v2 support (JDK-8230305) broke the
>>>> slowdebug build. Sorry about this :-/ The issue is that
>>>> CgroupController and CgroupSubsystem are abstract classes and I
>>>> missed
>>>> to declare virtual functions as pure.
>>>>
>>>> This patch fixes that.
>>>>
>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8237479
>>>> webrev:
>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8237479/01/webrev/
>>>>
>>>> Testing: slowdebug/fastdebug/release builds work on Linux x86_64.
>>>>            Currently running through jdk/submit.
>>>>
>>>> Thoughts?
>>>>
>>>> Thanks,
>>>> Severin
>>>>


From david.holmes at oracle.com  Mon Jan 20 01:49:59 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 20 Jan 2020 11:49:59 +1000
Subject: RFR: 8235966: Process obsolete flags less aggressively
In-Reply-To: <25156ff2-9dd2-53ae-16bc-83d60c0afb1a@oracle.com>
References: <db63e5c9-a0c1-1575-e8dc-88429d6216fe@oracle.com>
 <674e4f0d-563f-3bb4-dee0-74aaacb9b270@oracle.com>
 <b32a2e50-62e3-ed94-7773-a6594a76e1aa@oracle.com>
 <c1a2f1e1-e86b-1fda-4632-ce9734e6e75f@oracle.com>
 <b8e70c4d-af76-a682-b4a9-839dd2671958@oracle.com>
 <6a3de8da-746d-7e5d-9a0e-2157a0963ed8@oracle.com>
 <b7f7df86-50f0-523e-6cb2-e6b7d5fdddde@oracle.com>
 <25156ff2-9dd2-53ae-16bc-83d60c0afb1a@oracle.com>
Message-ID: <d70c939e-db2f-8fb1-2333-19fdd6153527@oracle.com>

gtest version:

http://cr.openjdk.java.net/~dholmes/8235966/webrev.v4/

Bug report updated to show gtest output.

Thanks,
David

On 20/01/2020 7:46 am, David Holmes wrote:
> Hi Dan,
> 
> On 18/01/2020 1:37 am, Daniel D. Daugherty wrote:
>> On 1/16/20 12:57 AM, David Holmes wrote:
>>> Getting back to this ...
>>
>> You added this update to the bug report:
>>
>>> Update: after further discussion it has been proposed that we use the 
>>> build number as the trigger for a whitebox or gtest that performs the 
>>> currently disabled full verification of the flag table. So if a flag 
>>> has not be obsoleted or expired as it should by build N** then we 
>>> fail the gtest. This will complement the relaxing of the obsoletion 
>>> check at the start of the release cycle. 
>>
>> I was expecting a new test in this latest webrev that would start failing
>> at Build 20... let me see what the latest webrev says...
> 
> Mea culpa. When I started thinking about the test it was evident the 
> test logic would just involve the existing commented out warning logic. 
> So I thought lets just turn those warnings on at build 20 but ...
> 
>>>
>>> Please see updated webrev at:
>>>
>>> http://cr.openjdk.java.net/~dholmes/8235966/webrev/
>>
>> src/hotspot/share/runtime/arguments.cpp
>> ???? No comments.
>>
>> So the changes are the same as the last round with the addition of
>> enabling the following at B20:
>>
>> L755: ????????? warning("Global variable for obsolete special flag 
>> entry \"%s\" should be removed", flag.name);
>> L769: ????????? warning("Global variable for expired flag entry \"%s\" 
>> should be removed", flag.name);
>>
>>
>>> Apologies as I mistakenly overwrote the original instead of creating v3.
>>>
>>> This version expands on the original proposal by uncommenting the 
>>> warnings about obsolete/expired flags that have not been removed from 
>>> globals*.hpp, so that we don't forget to do this work. However these 
>>> warnings are only enabled from build 20. I used 20 as being approx 
>>> 2/3 through the release cycle - long enough that the work should have 
>>> been performed by then, whilst still leaving time to perform the work 
>>> before RDP2. Of course we can tweak this number if there are issues 
>>> with that choice.
>>
>> Okay... but doesn't this mean that every test would issue these warnings
>> as of B20 if we have not completed the work? So we have the potential of
>> a raft (some unknown number) of test failures due to unexpected output in
>> the form of these warning messages. And worse, these failures would be
>> unrelated to actual issue... :-(
> 
> ... as you note the end result is not really what we want - a single 
> clearly failing test.
> 
>> How about adding a diagnostic flag that enables these two warning
>> messages (in addition to the B20 check). Add a single test that runs:
>>
>> ?? java -XX:+UnlockDiagnosticVMOptions 
>> -XX:+FlagShouldNotBeDefinedCheck -version
>>
>> and when we hit B20, if there are still flags that haven't been removed,
>> then the test will fail and we'll have one test that fails (X the number
>> of configs that run the test).
> 
> I definitely do not want to add another VM flag :) but will look at how 
> to (re)run that logic as part of a gtest.
> 
> Thanks,
> David
> 
>> Dan
>>
>>
>>>
>>> Thanks,
>>> David
>>>
>>> On 20/12/2019 8:20 am, David Holmes wrote:
>>>> Thanks Dan.
>>>>
>>>> FTR I've updated the bug report with an extension to this proposal, 
>>>> which is to add back the flag table validation checks to use via a 
>>>> gtest that we only enable after a certain build in the release cycle 
>>>> (it always passes before then). That way we avoid the problems I've 
>>>> outlined with the initial version bump but also have a safety net in 
>>>> place to ensure we don't forget to actually obsolete/expire flags.
>>>>
>>>> Cheers,
>>>> David
>>>>
>>>> On 19/12/2019 3:37 am, Daniel D. Daugherty wrote:
>>>>> Hi David,
>>>>>
>>>>> On 12/17/19 5:03 PM, David Holmes wrote:
>>>>>> Hi Dan,
>>>>>>
>>>>>> Thanks for taking a look. Updated webrev:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dholmes/8235966/webrev.v2/
>>>>>
>>>>> src/hotspot/share/runtime/arguments.cpp
>>>>> ???? I like the updates to header comment for 
>>>>> verify_special_jvm_flags().
>>>>>
>>>>> Thumbs up.
>>>>>
>>>>>
>>>>>>
>>>>>> Discussion below.
>>>>>
>>>>> Replies below.
>>>>>
>>>>>
>>>>>>
>>>>>> On 18/12/2019 1:47 am, Daniel D. Daugherty wrote:
>>>>>>> On 12/16/19 12:16 AM, David Holmes wrote:
>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8235966
>>>>>>>> webrev: http://cr.openjdk.java.net/~dholmes/8235966/webrev/
>>>>>>>
>>>>>>> src/hotspot/share/runtime/arguments.cpp
>>>>>>> ???? L745: ????? // if flag has become obsolete it should not 
>>>>>>> have a "globals" flag defined anymore.
>>>>>>> ???? L746: ????? if (!version_less_than(JDK_Version::current(), 
>>>>>>> flag.obsolete_in)) {
>>>>>>> ???? L747: ??????? if (JVMFlag::find_declared_flag(flag.name) != 
>>>>>>> NULL) {
>>>>>>> ???? L748: ????????? // Temporarily disable the warning: 8196739
>>>>>>> ???? L749: ????????? // warning("Global variable for obsolete 
>>>>>>> special flag entry \"%s\" should be removed", flag.name);
>>>>>>> ???? L750: ??????? }
>>>>>>> ???? L751: ????? }
>>>>>>> ???????? It seems like we've been down a similar road before:
>>>>>>>
>>>>>>> ???????? JDK-8196739 Disable obsolete/expired VM flag 
>>>>>>> transitional warnings
>>>>>>> ???????? https://bugs.openjdk.java.net/browse/JDK-8196739
>>>>>>>
>>>>>>> ???????? This one may ring a bell... Fixed by dholmes in 
>>>>>>> jdk11-b01... :-)
>>>>>>>
>>>>>>> ???????? And this followup sub-task to re-enable that warning:
>>>>>>>
>>>>>>> ???????? JDK-8196741 Re-enable obsolete/expired VM flag 
>>>>>>> transitional warnings
>>>>>>> ???????? https://bugs.openjdk.java.net/browse/JDK-8196741
>>>>>>>
>>>>>>> ???????? was closed as "Won't fix" on 2019.08.02.
>>>>>>>
>>>>>>> ???????? So the obvious questions:
>>>>>>>
>>>>>>> ???????? - Why is the new warning less problematic to tests that 
>>>>>>> don't
>>>>>>> ?????????? tolerate unexpected output?
>>>>>>
>>>>>> Two different situations. The commented out warning happens 
>>>>>> unconditionally when you run the VM and it finds any flag marked 
>>>>>> obsolete that hasn't been removed. Hence every single test will 
>>>>>> encounter this warning.
>>>>>
>>>>> Ouch on such verbosity.
>>>>>
>>>>>
>>>>>> The situation I am modifying is when a test uses a flag that is 
>>>>>> marked for obsoletion. In the majority of cases the flag is 
>>>>>> already deprecated and so already issuing a deprecation warning 
>>>>>> that the test has to handle. Without my change there would still 
>>>>>> be an obsoletion warning, so this test is in for a warning no 
>>>>>> matter what.
>>>>>
>>>>> Good that your change only comes into play when the flag is used.
>>>>>
>>>>>
>>>>>> Also note that for hotspot at least we have strived to make tests 
>>>>>> tolerate unexpected output. The reason JDK-8196741 was closed as 
>>>>>> "won't fix" was because other areas wouldn't commit to doing that.
>>>>>
>>>>> Yup. Got it.
>>>>>
>>>>>
>>>>>>
>>>>>>> ???????? - If you move forward with this fix, then I think think 
>>>>>>> code
>>>>>>> ?????????? block needs to be removed or modified or am I missing 
>>>>>>> something?
>>>>>>
>>>>>> I've rewritten the comment at the head of verify_special_jvm_flags 
>>>>>> to explain why we can't issue a warning, and have deleted the block.
>>>>>
>>>>> Thanks for deleting the stale code.
>>>>>
>>>>>
>>>>>>
>>>>>>> ???????? There's a similar commented out check on L757-L765, but 
>>>>>>> that one
>>>>>>> ???????? is for an expired flag... You might want to 
>>>>>>> adjust/delete it also?
>>>>>>
>>>>>> Deleted.
>>>>>
>>>>> Thanks.
>>>>>
>>>>>
>>>>>>
>>>>>>> ???? L753: ??????? warning("Special flag entry \"%s\" must be 
>>>>>>> explicitly obsoleted before expired.", flag.name);
>>>>>>> ???? L754: ??????? success = false;
>>>>>>> ???????? nit - s/before expired/before being expired/
>>>>>>> ???????? Update: I now see that "style" is in several places in this
>>>>>>> ???????????? function. I'm not sure what to think here... it grates,
>>>>>>> ?? ? ? ? ? ? but I can live with it.
>>>>>>>
>>>>>>> ???????? nit - L75[34] indented too much by two spaces.
>>>>>>
>>>>>> Fixed.
>>>>>>
>>>>>>> ???? L962: ????????? return real_name;
>>>>>>> ???????? nit - indented too much by two spaces.
>>>>>>
>>>>>> Fixed.
>>>>>>
>>>>>>>
>>>>>>> Trying to understand the modified logic in argument processing is
>>>>>>> making my head spin...
>>>>>>
>>>>>> Mine too. It took a few attempts to put the logic in the right 
>>>>>> place and make adjustments so that it all works as expected for a 
>>>>>> correctly specified flag and an erroneous one.
>>>>>
>>>>> I keep trying to convince myself that we're improving this flag and
>>>>> options code with each release... :-)
>>>>>
>>>>>
>>>>>>
>>>>>>> - You've added a call to is_obsolete_flag() in
>>>>>>> ?? handle_aliases_and_deprecation() and is_obsolete_flag()
>>>>>>> ?? is where the new warning is output:
>>>>>>>
>>>>>>> ???? warning("Temporarily processing option %s; support is 
>>>>>>> scheduled for removal in %s"
>>>>>>>
>>>>>>> ?? handle_aliases_and_deprecation() is called from six different 
>>>>>>> places,
>>>>>>> ?? but the call sites are different based on the argument pattern 
>>>>>>> so I
>>>>>>> ?? have (mostly) convinced myself that there should not be any 
>>>>>>> duplicate
>>>>>>> ?? warning lines.
>>>>>>
>>>>>> Right - handle_aliases_and_deprecation is only called for a 
>>>>>> syntactically correct flag based on those patterns. It normally 
>>>>>> filters out obsoleted/expired flags and lets them fall through to 
>>>>>> later error processing (in process_argument after parse_arg 
>>>>>> returns false). That error processing is where the normal 
>>>>>> obsoletion check is performed. So I had to not filter the flag in 
>>>>>> handle_aliases_and_deprecation in this case, but still produce the 
>>>>>> warning for a malformed flag. E.g.
>>>>>>
>>>>>> java -XX:+UseParallelOldGC -version
>>>>>> Java HotSpot(TM) 64-Bit Server VM warning: Temporarily processing 
>>>>>> option UseParallelOldGC; support is scheduled for removal in 15.0
>>>>>> java version "15-internal" 2020-09-15
>>>>>>
>>>>>> java -XX:UseParallelOldGC -version
>>>>>> Java HotSpot(TM) 64-Bit Server VM warning: Temporarily processing 
>>>>>> option UseParallelOldGC; support is scheduled for removal in 15.0
>>>>>> Missing +/- setting for VM option 'UseParallelOldGC'
>>>>>> Error: Could not create the Java Virtual Machine.
>>>>>
>>>>> Thanks for the example. That helps a lot.
>>>>>
>>>>>
>>>>>>
>>>>>>> So I now understand the new logic that allows an obsoleted option
>>>>>>> to be specified with a warning as long as the option still exists.
>>>>>>> I'm good with the technical change, but...
>>>>>>>
>>>>>>> I'm worried about tests that don't tolerate the new warning mesg,
>>>>>>> i.e., why wouldn't this become an issue again:
>>>>>>>
>>>>>>> JDK-8196739 Disable obsolete/expired VM flag transitional warnings
>>>>>>
>>>>>> Explained above.
>>>>>
>>>>> Yup and thanks.
>>>>>
>>>>> Dan
>>>>>
>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> David
>>>>>>
>>>>>>> Dan
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> When a flag is marked as obsolete in the special-flags table we 
>>>>>>>> will ignore it and issue a warning that it is being ignored, as 
>>>>>>>> soon as we bump the version of the JDK. That means that any 
>>>>>>>> tests still using the obsolete flag may start to fail, leading 
>>>>>>>> to a surge of test failures at the start of a release cycle. For 
>>>>>>>> example for JDK 15 we have a whole bunch of JFR tests that fail 
>>>>>>>> because they still try to work with UseParallelOldGC. In another 
>>>>>>>> case runtime/cds/appcds/FieldLayoutFlags.java passes only be 
>>>>>>>> accident.
>>>>>>>>
>>>>>>>> When a flag is marked as obsolete for a release, all code 
>>>>>>>> involving that flag (including tests that use it) must be 
>>>>>>>> updated within that release and the flag itself removed. Whilst 
>>>>>>>> this is typically scheduled early in a release cycle it isn't 
>>>>>>>> reasonable to expect it to all occur within the first couple of 
>>>>>>>> days of the release cycle, nor do we want to have to ProblemList 
>>>>>>>> a bunch of tests when they start failing.
>>>>>>>>
>>>>>>>> What I propose is to instead allow an obsolete flag to continue 
>>>>>>>> to be processed as long as that code removal has not actually 
>>>>>>>> occurred - with an adjusted warning. The change I propose:
>>>>>>>>
>>>>>>>> - only treats an obsolete flag as obsolete if the flag cannot be 
>>>>>>>> found
>>>>>>>> - added a new flag verification rule that disallows obsoletion 
>>>>>>>> in an undefined version, but expiration in a specific version 
>>>>>>>> i.e. we must always explicitly obsolete a flag before we expire it.
>>>>>>>>
>>>>>>>> The only downside here is that if we actually forget to file an 
>>>>>>>> issue for the actual obsoletion work we may not notice via 
>>>>>>>> testing. Of course whenever a change is made to the flags table 
>>>>>>>> to add an entry then the issue to do the obsoletion should be 
>>>>>>>> filed at the same time.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> David
>>>>>>>> -----
>>>>>>>>
>>>>>>>
>>>>>
>>

From nick.gasson at arm.com  Mon Jan 20 08:15:08 2020
From: nick.gasson at arm.com (Nick Gasson)
Date: Mon, 20 Jan 2020 16:15:08 +0800
Subject: RFR: 8237512: AArch64: aarch64TestHook leaks a BufferBlob
Message-ID: <20ac5b8d-4e38-db8d-4579-ff4a742365ef@arm.com>

Hello,

Bug: https://bugs.openjdk.java.net/browse/JDK-8237512
Webrev: http://cr.openjdk.java.net/~ngasson/8237512/webrev.0/

aarch64TestHook() is called from ICache::initialize() and allocates a
500k BufferBlob that it passes to entry() in assembler_aarch64.cpp which
performs a self-test of the assembler. The BufferBlob needs to be
explicitly freed or else it will hang around forever. Also we can just
skip the allocation if ASSERT is not defined as the test is not
performed in this case.

java -XX:+PrintCodeCache -Xint -version before:

   CodeCache: size=49152Kb used=1427Kb max_used=1427Kb free=47724Kb

After:

   CodeCache: size=49152Kb used=940Kb max_used=940Kb free=48212Kb

Tested jtreg tier1.


Thanks,
Nick

From adinn at redhat.com  Mon Jan 20 08:40:15 2020
From: adinn at redhat.com (Andrew Dinn)
Date: Mon, 20 Jan 2020 08:40:15 +0000
Subject: RFR: 8237512: AArch64: aarch64TestHook leaks a BufferBlob
In-Reply-To: <20ac5b8d-4e38-db8d-4579-ff4a742365ef@arm.com>
References: <20ac5b8d-4e38-db8d-4579-ff4a742365ef@arm.com>
Message-ID: <fa11cf2e-152d-ce54-6a2c-26bcefa05ed0@redhat.com>

On 20/01/2020 08:15, Nick Gasson wrote:
> Hello,
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8237512
> Webrev: http://cr.openjdk.java.net/~ngasson/8237512/webrev.0/
> 
> aarch64TestHook() is called from ICache::initialize() and allocates a
> 500k BufferBlob that it passes to entry() in assembler_aarch64.cpp which
> performs a self-test of the assembler. The BufferBlob needs to be
> explicitly freed or else it will hang around forever. Also we can just
> skip the allocation if ASSERT is not defined as the test is not
> performed in this case.
> 
> java -XX:+PrintCodeCache -Xint -version before:
> 
> ? CodeCache: size=49152Kb used=1427Kb max_used=1427Kb free=47724Kb
> 
> After:
> 
> ? CodeCache: size=49152Kb used=940Kb max_used=940Kb free=48212Kb
> 
> Tested jtreg tier1.
Reviewed, thanks.

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill


From nick.gasson at arm.com  Mon Jan 20 08:47:46 2020
From: nick.gasson at arm.com (Nick Gasson)
Date: Mon, 20 Jan 2020 16:47:46 +0800
Subject: RFR: 8237512: AArch64: aarch64TestHook leaks a BufferBlob
In-Reply-To: <fa11cf2e-152d-ce54-6a2c-26bcefa05ed0@redhat.com>
References: <20ac5b8d-4e38-db8d-4579-ff4a742365ef@arm.com>
 <fa11cf2e-152d-ce54-6a2c-26bcefa05ed0@redhat.com>
Message-ID: <e38fbed1-6954-d5b3-88c8-ea741c035def@arm.com>

On 20/01/2020 16:40, Andrew Dinn wrote:
> Reviewed, thanks.
> 

Thanks Andrew. Would you mind pushing it for me? My colleague with 
commit access is away the moment.


Nick

From aph at redhat.com  Mon Jan 20 09:54:33 2020
From: aph at redhat.com (Andrew Haley)
Date: Mon, 20 Jan 2020 09:54:33 +0000
Subject: [aarch64-port-dev ] RFR: 8237512: AArch64: aarch64TestHook leaks
 a BufferBlob
In-Reply-To: <e38fbed1-6954-d5b3-88c8-ea741c035def@arm.com>
References: <20ac5b8d-4e38-db8d-4579-ff4a742365ef@arm.com>
 <fa11cf2e-152d-ce54-6a2c-26bcefa05ed0@redhat.com>
 <e38fbed1-6954-d5b3-88c8-ea741c035def@arm.com>
Message-ID: <2e202576-00f9-8ca1-e52b-33968059e8a2@redhat.com>

On 1/20/20 8:47 AM, Nick Gasson wrote:
> On 20/01/2020 16:40, Andrew Dinn wrote:
>> Reviewed, thanks.
>>
> 
> Thanks Andrew. Would you mind pushing it for me? My colleague with 
> commit access is away the moment.

Please check the copyright dates: the RH copyright might want updating,
but the Oracle one probably doesn't.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From nick.gasson at arm.com  Mon Jan 20 10:07:19 2020
From: nick.gasson at arm.com (Nick Gasson)
Date: Mon, 20 Jan 2020 18:07:19 +0800
Subject: [aarch64-port-dev ] RFR: 8237512: AArch64: aarch64TestHook leaks
 a BufferBlob
In-Reply-To: <2e202576-00f9-8ca1-e52b-33968059e8a2@redhat.com>
References: <20ac5b8d-4e38-db8d-4579-ff4a742365ef@arm.com>
 <fa11cf2e-152d-ce54-6a2c-26bcefa05ed0@redhat.com>
 <e38fbed1-6954-d5b3-88c8-ea741c035def@arm.com>
 <2e202576-00f9-8ca1-e52b-33968059e8a2@redhat.com>
Message-ID: <b4bf5539-9660-76f0-6ba3-549c95f9393d@arm.com>

On 20/01/2020 17:54, Andrew Haley wrote:
> 
> Please check the copyright dates: the RH copyright might want updating,
> but the Oracle one probably doesn't.
> 

I'm not sure what the rules are for this: I bumped the copyright year in 
the first line of each file I modified to 2020, as this seems to be what 
everyone else does. Is aarch64Test.cpp missing an "Oracle and/or its 
affiliates" copyright line?

Thanks,
Nick

From adinn at redhat.com  Mon Jan 20 10:29:23 2020
From: adinn at redhat.com (Andrew Dinn)
Date: Mon, 20 Jan 2020 10:29:23 +0000
Subject: [aarch64-port-dev ] RFR: 8237512: AArch64: aarch64TestHook leaks
 a BufferBlob
In-Reply-To: <b4bf5539-9660-76f0-6ba3-549c95f9393d@arm.com>
References: <20ac5b8d-4e38-db8d-4579-ff4a742365ef@arm.com>
 <fa11cf2e-152d-ce54-6a2c-26bcefa05ed0@redhat.com>
 <e38fbed1-6954-d5b3-88c8-ea741c035def@arm.com>
 <2e202576-00f9-8ca1-e52b-33968059e8a2@redhat.com>
 <b4bf5539-9660-76f0-6ba3-549c95f9393d@arm.com>
Message-ID: <ccdd781c-e2f8-30bf-f79c-ec72deabc81b@redhat.com>

On 20/01/2020 10:07, Nick Gasson wrote:
> On 20/01/2020 17:54, Andrew Haley wrote:
>>
>> Please check the copyright dates: the RH copyright might want updating,
>> but the Oracle one probably doesn't.
>>
> 
> I'm not sure what the rules are for this: I bumped the copyright year in
> the first line of each file I modified to 2020, as this seems to be what
> everyone else does. Is aarch64Test.cpp missing an "Oracle and/or its
> affiliates" copyright line?
Well, I'm confused by this. As I understand it the Oracle copyright is
supposed to be up to date with whatever is the latest change while the
Red Hat copyright does not need to be kept up to date (in which case
before your patch the other files were in the erroneous state of having
an Oracle copyright that was behind the Red Hat one). However, Andrew's
comment does not seem to support my view.

Perhaps he or someone else could clarify? At which point I will be happy
to push your patch with whatever copyrights are appropriate.

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill


From martin.doerr at sap.com  Mon Jan 20 11:10:53 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Mon, 20 Jan 2020 11:10:53 +0000
Subject: RFR [XXS]: 8237382: Cleanup the OPT_SPEED_SRC file list in
 JvmFeatures.gmk
In-Reply-To: <aaa794a1-80ac-41cf-880a-f9ae04bfb714@oracle.com>
References: <AM6PR02MB420001BB07A53C75A12C2B2093310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <aaa794a1-80ac-41cf-880a-f9ae04bfb714@oracle.com>
Message-ID: <AM0PR0202MB32977EE6D5E302FB82B4C0749A320@AM0PR0202MB3297.eurprd02.prod.outlook.com>

Hi Matthias,

thanks for removing no longer existing files from the list.
I guess the list will need further updates to become really useful, but your change looks good.

Best regards,
Martin


> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
> Erik Joelsson
> Sent: Freitag, 17. Januar 2020 15:09
> To: Baesken, Matthias <matthias.baesken at sap.com>; 'build-
> dev at openjdk.java.net' <build-dev at openjdk.java.net>; 'hotspot-
> dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: Re: RFR [XXS]: 8237382: Cleanup the OPT_SPEED_SRC file list in
> JvmFeatures.gmk
> 
> Looks good.
> 
> /Erik
> 
> On 2020-01-16 22:47, Baesken, Matthias wrote:
> > Hello, please review this very small change .
> >
> > It removes file that are not present any more from the  OPT_SPEED_SRC
> file list in JvmFeatures.gmk  .
> >
> > ( this is a list of files to be optimized for speed when we otherwise optimize
> for size  in the minimal-JVM build)
> >
> >
> > Bug/webrev :
> >
> > https://bugs.openjdk.java.net/browse/JDK-8237382
> >
> > http://cr.openjdk.java.net/~mbaesken/webrevs/8237382.0/
> >
> > Thanks, Matthias
> >

From aph at redhat.com  Tue Jan 21 09:36:23 2020
From: aph at redhat.com (Andrew Haley)
Date: Tue, 21 Jan 2020 09:36:23 +0000
Subject: [aarch64-port-dev ] RFR: 8237512: AArch64: aarch64TestHook leaks
 a BufferBlob
In-Reply-To: <ccdd781c-e2f8-30bf-f79c-ec72deabc81b@redhat.com>
References: <20ac5b8d-4e38-db8d-4579-ff4a742365ef@arm.com>
 <fa11cf2e-152d-ce54-6a2c-26bcefa05ed0@redhat.com>
 <e38fbed1-6954-d5b3-88c8-ea741c035def@arm.com>
 <2e202576-00f9-8ca1-e52b-33968059e8a2@redhat.com>
 <b4bf5539-9660-76f0-6ba3-549c95f9393d@arm.com>
 <ccdd781c-e2f8-30bf-f79c-ec72deabc81b@redhat.com>
Message-ID: <ef7241e5-cf8f-84ab-c15c-c0dfd248731a@redhat.com>

On 1/20/20 10:29 AM, Andrew Dinn wrote:
> On 20/01/2020 10:07, Nick Gasson wrote:
>> On 20/01/2020 17:54, Andrew Haley wrote:
>>>
>>> Please check the copyright dates: the RH copyright might want updating,
>>> but the Oracle one probably doesn't.
>>>
>>
>> I'm not sure what the rules are for this: I bumped the copyright year in
>> the first line of each file I modified to 2020, as this seems to be what
>> everyone else does. Is aarch64Test.cpp missing an "Oracle and/or its
>> affiliates" copyright line?
>
> Well, I'm confused by this. As I understand it the Oracle copyright is
> supposed to be up to date with whatever is the latest change while the
> Red Hat copyright does not need to be kept up to date

I have never seen this rule before. What is your source for it? But
in any case, the Oracle copyright message here is because this file was
originally derived from Oracle code; the code being patches is by us.

> (in which case before your patch the other files were in the
> erroneous state of having an Oracle copyright that was behind the
> Red Hat one). However, Andrew's comment does not seem to support my
> view.
>
> Perhaps he or someone else could clarify? At which point I will be happy
> to push your patch with whatever copyrights are appropriate.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From matthias.baesken at sap.com  Tue Jan 21 10:02:48 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 21 Jan 2020 10:02:48 +0000
Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk
 images
In-Reply-To: <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com>
References: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com>
Message-ID: <AM6PR02MB4200C166288C01E34F192C93930D0@AM6PR02MB4200.eurprd02.prod.outlook.com>


Hi David ,  yes I think it makes sense to have a configure option for this .
Not everyone would like to have a larger JDK (even it is only a bit larger).

Best regards, Matthias


> 
> Hi Matthias,
> 
> This also needs to be a configurable option not one done by default as
> there can be non-technical issues relating to shipping symbol files in a
> product.
> 
> Thanks,
> David
> 
> On 17/01/2020 6:44 pm, Baesken, Matthias wrote:
> > Hello, please review this change related to stripped/"public" pdb file
> generation on Windows .
> >
> > Currently the JDK bundle on Windows does not contain pdb files (full pdb
> files are in a separate symbols bundle).
> > This leads currently to bad native stack traces e.g. when crashes occur.
> > One reason not to deliver the full pdb files might be the large size of these
> files.
> >
> > However there exist also "public" or stripped pdb files on Windows, see :
> >
> > https://docs.microsoft.com/en-us/cpp/build/reference/pdbstripped-strip-
> private-symbols?view=vs-2017
> >
> > Those are much smaller (often only 10-20% of the full pdb files) and they
> offer a good compromise (no "file:linenumber" info in the native stacks but
> at least the function name+hex-offset is visible)
> > to delivering full pdbs in the JDK.
> >
> > Example sizes for the currently built full pdbs / stripped pdbs from VS2017
> based 64bit build of jdk/jdk :
> > jvm.pdb : 73,1 MB / 9,46 MB
> > awt.pdb : 7,05 MB / 1,48 MB
> >
> > The patch  adds  generation  of stripped  pdb files  to the Windows  build.
> > Additionally those files are put into the JDK  bundle    (while the symbols
> bundle still gets the full pdb files ) .
> >
> >
> > Bug/webrev :
> >
> > https://bugs.openjdk.java.net/browse/JDK-8237192
> >
> > http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.0/
> >
> >
> > Thanks, Matthias
> >

From matthias.baesken at sap.com  Tue Jan 21 14:45:47 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 21 Jan 2020 14:45:47 +0000
Subject: RFR [XS]: 8237374: configuring with
 --with-jvm-variants=minimal,server makes cds disappear in server
In-Reply-To: <57f43f5a-ae37-5d98-e9e4-bb30c3607300@oracle.com>
References: <AM6PR02MB4200A055393D4CA7B3FA06E393310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <57f43f5a-ae37-5d98-e9e4-bb30c3607300@oracle.com>
Message-ID: <AM6PR02MB4200CC0697112F35A6585DE6930D0@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hi Erik, new webrev :

http://cr.openjdk.java.net/~mbaesken/webrevs/8237374.1/

Best regards, Matthias

-----Original Message-----
From: Erik Joelsson <erik.joelsson at oracle.com> 
Sent: Freitag, 17. Januar 2020 15:18
To: Baesken, Matthias <matthias.baesken at sap.com>; 'build-dev at openjdk.java.net' <build-dev at openjdk.java.net>; 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
Subject: Re: RFR [XS]: 8237374: configuring with --with-jvm-variants=minimal,server makes cds disappear in server

Hello Matthias,

Using BUILDING_MULTIPLE_JVM_VARIANTS as condition is clever and happens 
to coincide with the set of variants that also support CDS, but I would 
say this correlation is incidental. I would still prefer an explicit 
test for if any of the variants that do support CDS is in the set of 
variants being built. This will make it much easier to read and 
understand the logic. Simply:

if ! HOTSPOT_CHECK_JVM_VARIANT(server) && ! 
HOTSPOT_CHECK_JVM_VARIANT(client); then
 ? ENABLE_CDS="false"
 ? ...

/Erik



From daniel.daugherty at oracle.com  Tue Jan 21 15:37:03 2020
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Tue, 21 Jan 2020 10:37:03 -0500
Subject: RFR: 8235966: Process obsolete flags less aggressively
In-Reply-To: <d70c939e-db2f-8fb1-2333-19fdd6153527@oracle.com>
References: <db63e5c9-a0c1-1575-e8dc-88429d6216fe@oracle.com>
 <674e4f0d-563f-3bb4-dee0-74aaacb9b270@oracle.com>
 <b32a2e50-62e3-ed94-7773-a6594a76e1aa@oracle.com>
 <c1a2f1e1-e86b-1fda-4632-ce9734e6e75f@oracle.com>
 <b8e70c4d-af76-a682-b4a9-839dd2671958@oracle.com>
 <6a3de8da-746d-7e5d-9a0e-2157a0963ed8@oracle.com>
 <b7f7df86-50f0-523e-6cb2-e6b7d5fdddde@oracle.com>
 <25156ff2-9dd2-53ae-16bc-83d60c0afb1a@oracle.com>
 <d70c939e-db2f-8fb1-2333-19fdd6153527@oracle.com>
Message-ID: <3d066cd2-87f3-56ef-803b-a1f194ac761e@oracle.com>

On 1/19/20 8:49 PM, David Holmes wrote:
> gtest version:
>
> http://cr.openjdk.java.net/~dholmes/8235966/webrev.v4/

src/hotspot/share/runtime/arguments.hpp
 ??? No comments.

src/hotspot/share/runtime/arguments.cpp
 ??? L715: // check for stale flags when we hit build 20 (which is far 
enough into the 6 monthly
 ??????? typo: s/monthly/month/

test/hotspot/gtest/runtime/test_special_flags.cpp
 ??? Nice!

Thumbs up. I don't need to see a new webrev to fix the typo.

Dan


>
> Bug report updated to show gtest output.
>
> Thanks,
> David
>
> On 20/01/2020 7:46 am, David Holmes wrote:
>> Hi Dan,
>>
>> On 18/01/2020 1:37 am, Daniel D. Daugherty wrote:
>>> On 1/16/20 12:57 AM, David Holmes wrote:
>>>> Getting back to this ...
>>>
>>> You added this update to the bug report:
>>>
>>>> Update: after further discussion it has been proposed that we use 
>>>> the build number as the trigger for a whitebox or gtest that 
>>>> performs the currently disabled full verification of the flag 
>>>> table. So if a flag has not be obsoleted or expired as it should by 
>>>> build N** then we fail the gtest. This will complement the relaxing 
>>>> of the obsoletion check at the start of the release cycle. 
>>>
>>> I was expecting a new test in this latest webrev that would start 
>>> failing
>>> at Build 20... let me see what the latest webrev says...
>>
>> Mea culpa. When I started thinking about the test it was evident the 
>> test logic would just involve the existing commented out warning 
>> logic. So I thought lets just turn those warnings on at build 20 but ...
>>
>>>>
>>>> Please see updated webrev at:
>>>>
>>>> http://cr.openjdk.java.net/~dholmes/8235966/webrev/
>>>
>>> src/hotspot/share/runtime/arguments.cpp
>>> ???? No comments.
>>>
>>> So the changes are the same as the last round with the addition of
>>> enabling the following at B20:
>>>
>>> L755: ????????? warning("Global variable for obsolete special flag 
>>> entry \"%s\" should be removed", flag.name);
>>> L769: ????????? warning("Global variable for expired flag entry 
>>> \"%s\" should be removed", flag.name);
>>>
>>>
>>>> Apologies as I mistakenly overwrote the original instead of 
>>>> creating v3.
>>>>
>>>> This version expands on the original proposal by uncommenting the 
>>>> warnings about obsolete/expired flags that have not been removed 
>>>> from globals*.hpp, so that we don't forget to do this work. However 
>>>> these warnings are only enabled from build 20. I used 20 as being 
>>>> approx 2/3 through the release cycle - long enough that the work 
>>>> should have been performed by then, whilst still leaving time to 
>>>> perform the work before RDP2. Of course we can tweak this number if 
>>>> there are issues with that choice.
>>>
>>> Okay... but doesn't this mean that every test would issue these 
>>> warnings
>>> as of B20 if we have not completed the work? So we have the 
>>> potential of
>>> a raft (some unknown number) of test failures due to unexpected 
>>> output in
>>> the form of these warning messages. And worse, these failures would be
>>> unrelated to actual issue... :-(
>>
>> ... as you note the end result is not really what we want - a single 
>> clearly failing test.
>>
>>> How about adding a diagnostic flag that enables these two warning
>>> messages (in addition to the B20 check). Add a single test that runs:
>>>
>>> ?? java -XX:+UnlockDiagnosticVMOptions 
>>> -XX:+FlagShouldNotBeDefinedCheck -version
>>>
>>> and when we hit B20, if there are still flags that haven't been 
>>> removed,
>>> then the test will fail and we'll have one test that fails (X the 
>>> number
>>> of configs that run the test).
>>
>> I definitely do not want to add another VM flag :) but will look at 
>> how to (re)run that logic as part of a gtest.
>>
>> Thanks,
>> David
>>
>>> Dan
>>>
>>>
>>>>
>>>> Thanks,
>>>> David
>>>>
>>>> On 20/12/2019 8:20 am, David Holmes wrote:
>>>>> Thanks Dan.
>>>>>
>>>>> FTR I've updated the bug report with an extension to this 
>>>>> proposal, which is to add back the flag table validation checks to 
>>>>> use via a gtest that we only enable after a certain build in the 
>>>>> release cycle (it always passes before then). That way we avoid 
>>>>> the problems I've outlined with the initial version bump but also 
>>>>> have a safety net in place to ensure we don't forget to actually 
>>>>> obsolete/expire flags.
>>>>>
>>>>> Cheers,
>>>>> David
>>>>>
>>>>> On 19/12/2019 3:37 am, Daniel D. Daugherty wrote:
>>>>>> Hi David,
>>>>>>
>>>>>> On 12/17/19 5:03 PM, David Holmes wrote:
>>>>>>> Hi Dan,
>>>>>>>
>>>>>>> Thanks for taking a look. Updated webrev:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dholmes/8235966/webrev.v2/
>>>>>>
>>>>>> src/hotspot/share/runtime/arguments.cpp
>>>>>> ???? I like the updates to header comment for 
>>>>>> verify_special_jvm_flags().
>>>>>>
>>>>>> Thumbs up.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Discussion below.
>>>>>>
>>>>>> Replies below.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> On 18/12/2019 1:47 am, Daniel D. Daugherty wrote:
>>>>>>>> On 12/16/19 12:16 AM, David Holmes wrote:
>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8235966
>>>>>>>>> webrev: http://cr.openjdk.java.net/~dholmes/8235966/webrev/
>>>>>>>>
>>>>>>>> src/hotspot/share/runtime/arguments.cpp
>>>>>>>> ???? L745: ????? // if flag has become obsolete it should not 
>>>>>>>> have a "globals" flag defined anymore.
>>>>>>>> ???? L746: ????? if (!version_less_than(JDK_Version::current(), 
>>>>>>>> flag.obsolete_in)) {
>>>>>>>> ???? L747: ??????? if (JVMFlag::find_declared_flag(flag.name) 
>>>>>>>> != NULL) {
>>>>>>>> ???? L748: ????????? // Temporarily disable the warning: 8196739
>>>>>>>> ???? L749: ????????? // warning("Global variable for obsolete 
>>>>>>>> special flag entry \"%s\" should be removed", flag.name);
>>>>>>>> ???? L750: ??????? }
>>>>>>>> ???? L751: ????? }
>>>>>>>> ???????? It seems like we've been down a similar road before:
>>>>>>>>
>>>>>>>> ???????? JDK-8196739 Disable obsolete/expired VM flag 
>>>>>>>> transitional warnings
>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8196739
>>>>>>>>
>>>>>>>> ???????? This one may ring a bell... Fixed by dholmes in 
>>>>>>>> jdk11-b01... :-)
>>>>>>>>
>>>>>>>> ???????? And this followup sub-task to re-enable that warning:
>>>>>>>>
>>>>>>>> ???????? JDK-8196741 Re-enable obsolete/expired VM flag 
>>>>>>>> transitional warnings
>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8196741
>>>>>>>>
>>>>>>>> ???????? was closed as "Won't fix" on 2019.08.02.
>>>>>>>>
>>>>>>>> ???????? So the obvious questions:
>>>>>>>>
>>>>>>>> ???????? - Why is the new warning less problematic to tests 
>>>>>>>> that don't
>>>>>>>> ?????????? tolerate unexpected output?
>>>>>>>
>>>>>>> Two different situations. The commented out warning happens 
>>>>>>> unconditionally when you run the VM and it finds any flag marked 
>>>>>>> obsolete that hasn't been removed. Hence every single test will 
>>>>>>> encounter this warning.
>>>>>>
>>>>>> Ouch on such verbosity.
>>>>>>
>>>>>>
>>>>>>> The situation I am modifying is when a test uses a flag that is 
>>>>>>> marked for obsoletion. In the majority of cases the flag is 
>>>>>>> already deprecated and so already issuing a deprecation warning 
>>>>>>> that the test has to handle. Without my change there would still 
>>>>>>> be an obsoletion warning, so this test is in for a warning no 
>>>>>>> matter what.
>>>>>>
>>>>>> Good that your change only comes into play when the flag is used.
>>>>>>
>>>>>>
>>>>>>> Also note that for hotspot at least we have strived to make 
>>>>>>> tests tolerate unexpected output. The reason JDK-8196741 was 
>>>>>>> closed as "won't fix" was because other areas wouldn't commit to 
>>>>>>> doing that.
>>>>>>
>>>>>> Yup. Got it.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>> ???????? - If you move forward with this fix, then I think 
>>>>>>>> think code
>>>>>>>> ?????????? block needs to be removed or modified or am I 
>>>>>>>> missing something?
>>>>>>>
>>>>>>> I've rewritten the comment at the head of 
>>>>>>> verify_special_jvm_flags to explain why we can't issue a 
>>>>>>> warning, and have deleted the block.
>>>>>>
>>>>>> Thanks for deleting the stale code.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>> ???????? There's a similar commented out check on L757-L765, 
>>>>>>>> but that one
>>>>>>>> ???????? is for an expired flag... You might want to 
>>>>>>>> adjust/delete it also?
>>>>>>>
>>>>>>> Deleted.
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>> ???? L753: warning("Special flag entry \"%s\" must be 
>>>>>>>> explicitly obsoleted before expired.", flag.name);
>>>>>>>> ???? L754: ??????? success = false;
>>>>>>>> ???????? nit - s/before expired/before being expired/
>>>>>>>> ???????? Update: I now see that "style" is in several places in 
>>>>>>>> this
>>>>>>>> ???????????? function. I'm not sure what to think here... it 
>>>>>>>> grates,
>>>>>>>> ?? ? ? ? ? ? but I can live with it.
>>>>>>>>
>>>>>>>> ???????? nit - L75[34] indented too much by two spaces.
>>>>>>>
>>>>>>> Fixed.
>>>>>>>
>>>>>>>> ???? L962: ????????? return real_name;
>>>>>>>> ???????? nit - indented too much by two spaces.
>>>>>>>
>>>>>>> Fixed.
>>>>>>>
>>>>>>>>
>>>>>>>> Trying to understand the modified logic in argument processing is
>>>>>>>> making my head spin...
>>>>>>>
>>>>>>> Mine too. It took a few attempts to put the logic in the right 
>>>>>>> place and make adjustments so that it all works as expected for 
>>>>>>> a correctly specified flag and an erroneous one.
>>>>>>
>>>>>> I keep trying to convince myself that we're improving this flag and
>>>>>> options code with each release... :-)
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>> - You've added a call to is_obsolete_flag() in
>>>>>>>> ?? handle_aliases_and_deprecation() and is_obsolete_flag()
>>>>>>>> ?? is where the new warning is output:
>>>>>>>>
>>>>>>>> ???? warning("Temporarily processing option %s; support is 
>>>>>>>> scheduled for removal in %s"
>>>>>>>>
>>>>>>>> ?? handle_aliases_and_deprecation() is called from six 
>>>>>>>> different places,
>>>>>>>> ?? but the call sites are different based on the argument 
>>>>>>>> pattern so I
>>>>>>>> ?? have (mostly) convinced myself that there should not be any 
>>>>>>>> duplicate
>>>>>>>> ?? warning lines.
>>>>>>>
>>>>>>> Right - handle_aliases_and_deprecation is only called for a 
>>>>>>> syntactically correct flag based on those patterns. It normally 
>>>>>>> filters out obsoleted/expired flags and lets them fall through 
>>>>>>> to later error processing (in process_argument after parse_arg 
>>>>>>> returns false). That error processing is where the normal 
>>>>>>> obsoletion check is performed. So I had to not filter the flag 
>>>>>>> in handle_aliases_and_deprecation in this case, but still 
>>>>>>> produce the warning for a malformed flag. E.g.
>>>>>>>
>>>>>>> java -XX:+UseParallelOldGC -version
>>>>>>> Java HotSpot(TM) 64-Bit Server VM warning: Temporarily 
>>>>>>> processing option UseParallelOldGC; support is scheduled for 
>>>>>>> removal in 15.0
>>>>>>> java version "15-internal" 2020-09-15
>>>>>>>
>>>>>>> java -XX:UseParallelOldGC -version
>>>>>>> Java HotSpot(TM) 64-Bit Server VM warning: Temporarily 
>>>>>>> processing option UseParallelOldGC; support is scheduled for 
>>>>>>> removal in 15.0
>>>>>>> Missing +/- setting for VM option 'UseParallelOldGC'
>>>>>>> Error: Could not create the Java Virtual Machine.
>>>>>>
>>>>>> Thanks for the example. That helps a lot.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>> So I now understand the new logic that allows an obsoleted option
>>>>>>>> to be specified with a warning as long as the option still exists.
>>>>>>>> I'm good with the technical change, but...
>>>>>>>>
>>>>>>>> I'm worried about tests that don't tolerate the new warning mesg,
>>>>>>>> i.e., why wouldn't this become an issue again:
>>>>>>>>
>>>>>>>> JDK-8196739 Disable obsolete/expired VM flag transitional warnings
>>>>>>>
>>>>>>> Explained above.
>>>>>>
>>>>>> Yup and thanks.
>>>>>>
>>>>>> Dan
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>> David
>>>>>>>
>>>>>>>> Dan
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> When a flag is marked as obsolete in the special-flags table 
>>>>>>>>> we will ignore it and issue a warning that it is being 
>>>>>>>>> ignored, as soon as we bump the version of the JDK. That means 
>>>>>>>>> that any tests still using the obsolete flag may start to 
>>>>>>>>> fail, leading to a surge of test failures at the start of a 
>>>>>>>>> release cycle. For example for JDK 15 we have a whole bunch of 
>>>>>>>>> JFR tests that fail because they still try to work with 
>>>>>>>>> UseParallelOldGC. In another case 
>>>>>>>>> runtime/cds/appcds/FieldLayoutFlags.java passes only be accident.
>>>>>>>>>
>>>>>>>>> When a flag is marked as obsolete for a release, all code 
>>>>>>>>> involving that flag (including tests that use it) must be 
>>>>>>>>> updated within that release and the flag itself removed. 
>>>>>>>>> Whilst this is typically scheduled early in a release cycle it 
>>>>>>>>> isn't reasonable to expect it to all occur within the first 
>>>>>>>>> couple of days of the release cycle, nor do we want to have to 
>>>>>>>>> ProblemList a bunch of tests when they start failing.
>>>>>>>>>
>>>>>>>>> What I propose is to instead allow an obsolete flag to 
>>>>>>>>> continue to be processed as long as that code removal has not 
>>>>>>>>> actually occurred - with an adjusted warning. The change I 
>>>>>>>>> propose:
>>>>>>>>>
>>>>>>>>> - only treats an obsolete flag as obsolete if the flag cannot 
>>>>>>>>> be found
>>>>>>>>> - added a new flag verification rule that disallows obsoletion 
>>>>>>>>> in an undefined version, but expiration in a specific version 
>>>>>>>>> i.e. we must always explicitly obsolete a flag before we 
>>>>>>>>> expire it.
>>>>>>>>>
>>>>>>>>> The only downside here is that if we actually forget to file 
>>>>>>>>> an issue for the actual obsoletion work we may not notice via 
>>>>>>>>> testing. Of course whenever a change is made to the flags 
>>>>>>>>> table to add an entry then the issue to do the obsoletion 
>>>>>>>>> should be filed at the same time.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> David
>>>>>>>>> -----
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>


From erik.joelsson at oracle.com  Tue Jan 21 16:35:20 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Tue, 21 Jan 2020 08:35:20 -0800
Subject: RFR [XS]: 8237374: configuring with
 --with-jvm-variants=minimal,server makes cds disappear in server
In-Reply-To: <AM6PR02MB4200CC0697112F35A6585DE6930D0@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB4200A055393D4CA7B3FA06E393310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <57f43f5a-ae37-5d98-e9e4-bb30c3607300@oracle.com>
 <AM6PR02MB4200CC0697112F35A6585DE6930D0@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <2f32b930-79b2-d043-faf3-ba3975b72af1@oracle.com>

That looks good.

/Erik


On 2020-01-21 06:45, Baesken, Matthias wrote:
> Hi Erik, new webrev :
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8237374.1/
>
> Best regards, Matthias
>
> -----Original Message-----
> From: Erik Joelsson <erik.joelsson at oracle.com>
> Sent: Freitag, 17. Januar 2020 15:18
> To: Baesken, Matthias <matthias.baesken at sap.com>; 'build-dev at openjdk.java.net' <build-dev at openjdk.java.net>; 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: Re: RFR [XS]: 8237374: configuring with --with-jvm-variants=minimal,server makes cds disappear in server
>
> Hello Matthias,
>
> Using BUILDING_MULTIPLE_JVM_VARIANTS as condition is clever and happens
> to coincide with the set of variants that also support CDS, but I would
> say this correlation is incidental. I would still prefer an explicit
> test for if any of the variants that do support CDS is in the set of
> variants being built. This will make it much easier to read and
> understand the logic. Simply:
>
> if ! HOTSPOT_CHECK_JVM_VARIANT(server) && !
> HOTSPOT_CHECK_JVM_VARIANT(client); then
>   ? ENABLE_CDS="false"
>   ? ...
>
> /Erik
>
>

From aph at redhat.com  Tue Jan 21 17:12:10 2020
From: aph at redhat.com (Andrew Haley)
Date: Tue, 21 Jan 2020 17:12:10 +0000
Subject: 8230392: Define AArch64 as MULTI_COPY_ATOMIC
Message-ID: <a8d3f689-dde7-e5ac-c06e-eb3dc1355553@redhat.com>

http://cr.openjdk.java.net/~aph/8230392/

I tested this in a bunch of ways, including bootstrapping and jtreg, but
I'm not sure anything really stress tests this. I expect it'll be fine.

OK to commit?

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From david.holmes at oracle.com  Tue Jan 21 22:58:21 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 22 Jan 2020 08:58:21 +1000
Subject: 8230392: Define AArch64 as MULTI_COPY_ATOMIC
In-Reply-To: <a8d3f689-dde7-e5ac-c06e-eb3dc1355553@redhat.com>
References: <a8d3f689-dde7-e5ac-c06e-eb3dc1355553@redhat.com>
Message-ID: <053adeb2-5f00-5cf6-c96e-97d46d37f5a3@oracle.com>

Andrew,

On 22/01/2020 3:12 am, Andrew Haley wrote:
> http://cr.openjdk.java.net/~aph/8230392/
> 
> I tested this in a bunch of ways, including bootstrapping and jtreg, but
> I'm not sure anything really stress tests this. I expect it'll be fine.
> 
> OK to commit?

You obviously missed my response on the other email thread regarding the 
taskQueue code.

> "acquire" isn't used to order loads it is used to pair with a "release" associated with the store of the variable now being loaded.
> 
> If this is the code referred to:
> 
>   Age oldAge = _age.get();
>   // Architectures with weak memory model require a barrier here
>   // to guarantee that bottom is not older than age,
>   // which is crucial for the correctness of the algorithm.
> #ifndef CPU_MULTI_COPY_ATOMIC
>   OrderAccess::fence();
> #endif
>   uint localBot = Atomic::load_acquire(&_bottom);
> 
> then I think there is an assumption (perhaps incorrect) that the load_acquire will prevent reordering as well as performing the necessary "acquire" semantics. If the load_acquire doesn't prevent reordering then surely a loadload() barrier is what is needed.

Cheers,
David
-----




From david.holmes at oracle.com  Tue Jan 21 23:10:37 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 22 Jan 2020 09:10:37 +1000
Subject: RFR: 8235966: Process obsolete flags less aggressively
In-Reply-To: <3d066cd2-87f3-56ef-803b-a1f194ac761e@oracle.com>
References: <db63e5c9-a0c1-1575-e8dc-88429d6216fe@oracle.com>
 <674e4f0d-563f-3bb4-dee0-74aaacb9b270@oracle.com>
 <b32a2e50-62e3-ed94-7773-a6594a76e1aa@oracle.com>
 <c1a2f1e1-e86b-1fda-4632-ce9734e6e75f@oracle.com>
 <b8e70c4d-af76-a682-b4a9-839dd2671958@oracle.com>
 <6a3de8da-746d-7e5d-9a0e-2157a0963ed8@oracle.com>
 <b7f7df86-50f0-523e-6cb2-e6b7d5fdddde@oracle.com>
 <25156ff2-9dd2-53ae-16bc-83d60c0afb1a@oracle.com>
 <d70c939e-db2f-8fb1-2333-19fdd6153527@oracle.com>
 <3d066cd2-87f3-56ef-803b-a1f194ac761e@oracle.com>
Message-ID: <e33ab2a9-52d1-a0fa-601b-31c99370c217@oracle.com>

Thanks Dan.

Will fix the "typo" - though not actually a typo as it was a deliberate 
(mis)use of "monthly". I have always quantified "monthly" this way, but 
seems it is not correct. :)

Cheers,
David

On 22/01/2020 1:37 am, Daniel D. Daugherty wrote:
> On 1/19/20 8:49 PM, David Holmes wrote:
>> gtest version:
>>
>> http://cr.openjdk.java.net/~dholmes/8235966/webrev.v4/
> 
> src/hotspot/share/runtime/arguments.hpp
>  ??? No comments.
> 
> src/hotspot/share/runtime/arguments.cpp
>  ??? L715: // check for stale flags when we hit build 20 (which is far 
> enough into the 6 monthly
>  ??????? typo: s/monthly/month/
> 
> test/hotspot/gtest/runtime/test_special_flags.cpp
>  ??? Nice!
> 
> Thumbs up. I don't need to see a new webrev to fix the typo.
> 
> Dan
> 
> 
>>
>> Bug report updated to show gtest output.
>>
>> Thanks,
>> David
>>
>> On 20/01/2020 7:46 am, David Holmes wrote:
>>> Hi Dan,
>>>
>>> On 18/01/2020 1:37 am, Daniel D. Daugherty wrote:
>>>> On 1/16/20 12:57 AM, David Holmes wrote:
>>>>> Getting back to this ...
>>>>
>>>> You added this update to the bug report:
>>>>
>>>>> Update: after further discussion it has been proposed that we use 
>>>>> the build number as the trigger for a whitebox or gtest that 
>>>>> performs the currently disabled full verification of the flag 
>>>>> table. So if a flag has not be obsoleted or expired as it should by 
>>>>> build N** then we fail the gtest. This will complement the relaxing 
>>>>> of the obsoletion check at the start of the release cycle. 
>>>>
>>>> I was expecting a new test in this latest webrev that would start 
>>>> failing
>>>> at Build 20... let me see what the latest webrev says...
>>>
>>> Mea culpa. When I started thinking about the test it was evident the 
>>> test logic would just involve the existing commented out warning 
>>> logic. So I thought lets just turn those warnings on at build 20 but ...
>>>
>>>>>
>>>>> Please see updated webrev at:
>>>>>
>>>>> http://cr.openjdk.java.net/~dholmes/8235966/webrev/
>>>>
>>>> src/hotspot/share/runtime/arguments.cpp
>>>> ???? No comments.
>>>>
>>>> So the changes are the same as the last round with the addition of
>>>> enabling the following at B20:
>>>>
>>>> L755: ????????? warning("Global variable for obsolete special flag 
>>>> entry \"%s\" should be removed", flag.name);
>>>> L769: ????????? warning("Global variable for expired flag entry 
>>>> \"%s\" should be removed", flag.name);
>>>>
>>>>
>>>>> Apologies as I mistakenly overwrote the original instead of 
>>>>> creating v3.
>>>>>
>>>>> This version expands on the original proposal by uncommenting the 
>>>>> warnings about obsolete/expired flags that have not been removed 
>>>>> from globals*.hpp, so that we don't forget to do this work. However 
>>>>> these warnings are only enabled from build 20. I used 20 as being 
>>>>> approx 2/3 through the release cycle - long enough that the work 
>>>>> should have been performed by then, whilst still leaving time to 
>>>>> perform the work before RDP2. Of course we can tweak this number if 
>>>>> there are issues with that choice.
>>>>
>>>> Okay... but doesn't this mean that every test would issue these 
>>>> warnings
>>>> as of B20 if we have not completed the work? So we have the 
>>>> potential of
>>>> a raft (some unknown number) of test failures due to unexpected 
>>>> output in
>>>> the form of these warning messages. And worse, these failures would be
>>>> unrelated to actual issue... :-(
>>>
>>> ... as you note the end result is not really what we want - a single 
>>> clearly failing test.
>>>
>>>> How about adding a diagnostic flag that enables these two warning
>>>> messages (in addition to the B20 check). Add a single test that runs:
>>>>
>>>> ?? java -XX:+UnlockDiagnosticVMOptions 
>>>> -XX:+FlagShouldNotBeDefinedCheck -version
>>>>
>>>> and when we hit B20, if there are still flags that haven't been 
>>>> removed,
>>>> then the test will fail and we'll have one test that fails (X the 
>>>> number
>>>> of configs that run the test).
>>>
>>> I definitely do not want to add another VM flag :) but will look at 
>>> how to (re)run that logic as part of a gtest.
>>>
>>> Thanks,
>>> David
>>>
>>>> Dan
>>>>
>>>>
>>>>>
>>>>> Thanks,
>>>>> David
>>>>>
>>>>> On 20/12/2019 8:20 am, David Holmes wrote:
>>>>>> Thanks Dan.
>>>>>>
>>>>>> FTR I've updated the bug report with an extension to this 
>>>>>> proposal, which is to add back the flag table validation checks to 
>>>>>> use via a gtest that we only enable after a certain build in the 
>>>>>> release cycle (it always passes before then). That way we avoid 
>>>>>> the problems I've outlined with the initial version bump but also 
>>>>>> have a safety net in place to ensure we don't forget to actually 
>>>>>> obsolete/expire flags.
>>>>>>
>>>>>> Cheers,
>>>>>> David
>>>>>>
>>>>>> On 19/12/2019 3:37 am, Daniel D. Daugherty wrote:
>>>>>>> Hi David,
>>>>>>>
>>>>>>> On 12/17/19 5:03 PM, David Holmes wrote:
>>>>>>>> Hi Dan,
>>>>>>>>
>>>>>>>> Thanks for taking a look. Updated webrev:
>>>>>>>>
>>>>>>>> http://cr.openjdk.java.net/~dholmes/8235966/webrev.v2/
>>>>>>>
>>>>>>> src/hotspot/share/runtime/arguments.cpp
>>>>>>> ???? I like the updates to header comment for 
>>>>>>> verify_special_jvm_flags().
>>>>>>>
>>>>>>> Thumbs up.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Discussion below.
>>>>>>>
>>>>>>> Replies below.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> On 18/12/2019 1:47 am, Daniel D. Daugherty wrote:
>>>>>>>>> On 12/16/19 12:16 AM, David Holmes wrote:
>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8235966
>>>>>>>>>> webrev: http://cr.openjdk.java.net/~dholmes/8235966/webrev/
>>>>>>>>>
>>>>>>>>> src/hotspot/share/runtime/arguments.cpp
>>>>>>>>> ???? L745: ????? // if flag has become obsolete it should not 
>>>>>>>>> have a "globals" flag defined anymore.
>>>>>>>>> ???? L746: ????? if (!version_less_than(JDK_Version::current(), 
>>>>>>>>> flag.obsolete_in)) {
>>>>>>>>> ???? L747: ??????? if (JVMFlag::find_declared_flag(flag.name) 
>>>>>>>>> != NULL) {
>>>>>>>>> ???? L748: ????????? // Temporarily disable the warning: 8196739
>>>>>>>>> ???? L749: ????????? // warning("Global variable for obsolete 
>>>>>>>>> special flag entry \"%s\" should be removed", flag.name);
>>>>>>>>> ???? L750: ??????? }
>>>>>>>>> ???? L751: ????? }
>>>>>>>>> ???????? It seems like we've been down a similar road before:
>>>>>>>>>
>>>>>>>>> ???????? JDK-8196739 Disable obsolete/expired VM flag 
>>>>>>>>> transitional warnings
>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8196739
>>>>>>>>>
>>>>>>>>> ???????? This one may ring a bell... Fixed by dholmes in 
>>>>>>>>> jdk11-b01... :-)
>>>>>>>>>
>>>>>>>>> ???????? And this followup sub-task to re-enable that warning:
>>>>>>>>>
>>>>>>>>> ???????? JDK-8196741 Re-enable obsolete/expired VM flag 
>>>>>>>>> transitional warnings
>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8196741
>>>>>>>>>
>>>>>>>>> ???????? was closed as "Won't fix" on 2019.08.02.
>>>>>>>>>
>>>>>>>>> ???????? So the obvious questions:
>>>>>>>>>
>>>>>>>>> ???????? - Why is the new warning less problematic to tests 
>>>>>>>>> that don't
>>>>>>>>> ?????????? tolerate unexpected output?
>>>>>>>>
>>>>>>>> Two different situations. The commented out warning happens 
>>>>>>>> unconditionally when you run the VM and it finds any flag marked 
>>>>>>>> obsolete that hasn't been removed. Hence every single test will 
>>>>>>>> encounter this warning.
>>>>>>>
>>>>>>> Ouch on such verbosity.
>>>>>>>
>>>>>>>
>>>>>>>> The situation I am modifying is when a test uses a flag that is 
>>>>>>>> marked for obsoletion. In the majority of cases the flag is 
>>>>>>>> already deprecated and so already issuing a deprecation warning 
>>>>>>>> that the test has to handle. Without my change there would still 
>>>>>>>> be an obsoletion warning, so this test is in for a warning no 
>>>>>>>> matter what.
>>>>>>>
>>>>>>> Good that your change only comes into play when the flag is used.
>>>>>>>
>>>>>>>
>>>>>>>> Also note that for hotspot at least we have strived to make 
>>>>>>>> tests tolerate unexpected output. The reason JDK-8196741 was 
>>>>>>>> closed as "won't fix" was because other areas wouldn't commit to 
>>>>>>>> doing that.
>>>>>>>
>>>>>>> Yup. Got it.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>> ???????? - If you move forward with this fix, then I think 
>>>>>>>>> think code
>>>>>>>>> ?????????? block needs to be removed or modified or am I 
>>>>>>>>> missing something?
>>>>>>>>
>>>>>>>> I've rewritten the comment at the head of 
>>>>>>>> verify_special_jvm_flags to explain why we can't issue a 
>>>>>>>> warning, and have deleted the block.
>>>>>>>
>>>>>>> Thanks for deleting the stale code.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>> ???????? There's a similar commented out check on L757-L765, 
>>>>>>>>> but that one
>>>>>>>>> ???????? is for an expired flag... You might want to 
>>>>>>>>> adjust/delete it also?
>>>>>>>>
>>>>>>>> Deleted.
>>>>>>>
>>>>>>> Thanks.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>> ???? L753: warning("Special flag entry \"%s\" must be 
>>>>>>>>> explicitly obsoleted before expired.", flag.name);
>>>>>>>>> ???? L754: ??????? success = false;
>>>>>>>>> ???????? nit - s/before expired/before being expired/
>>>>>>>>> ???????? Update: I now see that "style" is in several places in 
>>>>>>>>> this
>>>>>>>>> ???????????? function. I'm not sure what to think here... it 
>>>>>>>>> grates,
>>>>>>>>> ?? ? ? ? ? ? but I can live with it.
>>>>>>>>>
>>>>>>>>> ???????? nit - L75[34] indented too much by two spaces.
>>>>>>>>
>>>>>>>> Fixed.
>>>>>>>>
>>>>>>>>> ???? L962: ????????? return real_name;
>>>>>>>>> ???????? nit - indented too much by two spaces.
>>>>>>>>
>>>>>>>> Fixed.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Trying to understand the modified logic in argument processing is
>>>>>>>>> making my head spin...
>>>>>>>>
>>>>>>>> Mine too. It took a few attempts to put the logic in the right 
>>>>>>>> place and make adjustments so that it all works as expected for 
>>>>>>>> a correctly specified flag and an erroneous one.
>>>>>>>
>>>>>>> I keep trying to convince myself that we're improving this flag and
>>>>>>> options code with each release... :-)
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>> - You've added a call to is_obsolete_flag() in
>>>>>>>>> ?? handle_aliases_and_deprecation() and is_obsolete_flag()
>>>>>>>>> ?? is where the new warning is output:
>>>>>>>>>
>>>>>>>>> ???? warning("Temporarily processing option %s; support is 
>>>>>>>>> scheduled for removal in %s"
>>>>>>>>>
>>>>>>>>> ?? handle_aliases_and_deprecation() is called from six 
>>>>>>>>> different places,
>>>>>>>>> ?? but the call sites are different based on the argument 
>>>>>>>>> pattern so I
>>>>>>>>> ?? have (mostly) convinced myself that there should not be any 
>>>>>>>>> duplicate
>>>>>>>>> ?? warning lines.
>>>>>>>>
>>>>>>>> Right - handle_aliases_and_deprecation is only called for a 
>>>>>>>> syntactically correct flag based on those patterns. It normally 
>>>>>>>> filters out obsoleted/expired flags and lets them fall through 
>>>>>>>> to later error processing (in process_argument after parse_arg 
>>>>>>>> returns false). That error processing is where the normal 
>>>>>>>> obsoletion check is performed. So I had to not filter the flag 
>>>>>>>> in handle_aliases_and_deprecation in this case, but still 
>>>>>>>> produce the warning for a malformed flag. E.g.
>>>>>>>>
>>>>>>>> java -XX:+UseParallelOldGC -version
>>>>>>>> Java HotSpot(TM) 64-Bit Server VM warning: Temporarily 
>>>>>>>> processing option UseParallelOldGC; support is scheduled for 
>>>>>>>> removal in 15.0
>>>>>>>> java version "15-internal" 2020-09-15
>>>>>>>>
>>>>>>>> java -XX:UseParallelOldGC -version
>>>>>>>> Java HotSpot(TM) 64-Bit Server VM warning: Temporarily 
>>>>>>>> processing option UseParallelOldGC; support is scheduled for 
>>>>>>>> removal in 15.0
>>>>>>>> Missing +/- setting for VM option 'UseParallelOldGC'
>>>>>>>> Error: Could not create the Java Virtual Machine.
>>>>>>>
>>>>>>> Thanks for the example. That helps a lot.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>> So I now understand the new logic that allows an obsoleted option
>>>>>>>>> to be specified with a warning as long as the option still exists.
>>>>>>>>> I'm good with the technical change, but...
>>>>>>>>>
>>>>>>>>> I'm worried about tests that don't tolerate the new warning mesg,
>>>>>>>>> i.e., why wouldn't this become an issue again:
>>>>>>>>>
>>>>>>>>> JDK-8196739 Disable obsolete/expired VM flag transitional warnings
>>>>>>>>
>>>>>>>> Explained above.
>>>>>>>
>>>>>>> Yup and thanks.
>>>>>>>
>>>>>>> Dan
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> David
>>>>>>>>
>>>>>>>>> Dan
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> When a flag is marked as obsolete in the special-flags table 
>>>>>>>>>> we will ignore it and issue a warning that it is being 
>>>>>>>>>> ignored, as soon as we bump the version of the JDK. That means 
>>>>>>>>>> that any tests still using the obsolete flag may start to 
>>>>>>>>>> fail, leading to a surge of test failures at the start of a 
>>>>>>>>>> release cycle. For example for JDK 15 we have a whole bunch of 
>>>>>>>>>> JFR tests that fail because they still try to work with 
>>>>>>>>>> UseParallelOldGC. In another case 
>>>>>>>>>> runtime/cds/appcds/FieldLayoutFlags.java passes only be accident.
>>>>>>>>>>
>>>>>>>>>> When a flag is marked as obsolete for a release, all code 
>>>>>>>>>> involving that flag (including tests that use it) must be 
>>>>>>>>>> updated within that release and the flag itself removed. 
>>>>>>>>>> Whilst this is typically scheduled early in a release cycle it 
>>>>>>>>>> isn't reasonable to expect it to all occur within the first 
>>>>>>>>>> couple of days of the release cycle, nor do we want to have to 
>>>>>>>>>> ProblemList a bunch of tests when they start failing.
>>>>>>>>>>
>>>>>>>>>> What I propose is to instead allow an obsolete flag to 
>>>>>>>>>> continue to be processed as long as that code removal has not 
>>>>>>>>>> actually occurred - with an adjusted warning. The change I 
>>>>>>>>>> propose:
>>>>>>>>>>
>>>>>>>>>> - only treats an obsolete flag as obsolete if the flag cannot 
>>>>>>>>>> be found
>>>>>>>>>> - added a new flag verification rule that disallows obsoletion 
>>>>>>>>>> in an undefined version, but expiration in a specific version 
>>>>>>>>>> i.e. we must always explicitly obsolete a flag before we 
>>>>>>>>>> expire it.
>>>>>>>>>>
>>>>>>>>>> The only downside here is that if we actually forget to file 
>>>>>>>>>> an issue for the actual obsoletion work we may not notice via 
>>>>>>>>>> testing. Of course whenever a change is made to the flags 
>>>>>>>>>> table to add an entry then the issue to do the obsoletion 
>>>>>>>>>> should be filed at the same time.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> David
>>>>>>>>>> -----
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>
> 

From david.holmes at oracle.com  Wed Jan 22 02:06:23 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 22 Jan 2020 12:06:23 +1000
Subject: RFR: 8235966: Process obsolete flags less aggressively
In-Reply-To: <e33ab2a9-52d1-a0fa-601b-31c99370c217@oracle.com>
References: <db63e5c9-a0c1-1575-e8dc-88429d6216fe@oracle.com>
 <674e4f0d-563f-3bb4-dee0-74aaacb9b270@oracle.com>
 <b32a2e50-62e3-ed94-7773-a6594a76e1aa@oracle.com>
 <c1a2f1e1-e86b-1fda-4632-ce9734e6e75f@oracle.com>
 <b8e70c4d-af76-a682-b4a9-839dd2671958@oracle.com>
 <6a3de8da-746d-7e5d-9a0e-2157a0963ed8@oracle.com>
 <b7f7df86-50f0-523e-6cb2-e6b7d5fdddde@oracle.com>
 <25156ff2-9dd2-53ae-16bc-83d60c0afb1a@oracle.com>
 <d70c939e-db2f-8fb1-2333-19fdd6153527@oracle.com>
 <3d066cd2-87f3-56ef-803b-a1f194ac761e@oracle.com>
 <e33ab2a9-52d1-a0fa-601b-31c99370c217@oracle.com>
Message-ID: <311314c6-7e70-49be-41de-23ae3fa7f23e@oracle.com>

Can I get a second review please?

Thanks,
David

On 22/01/2020 9:10 am, David Holmes wrote:
> Thanks Dan.
> 
> Will fix the "typo" - though not actually a typo as it was a deliberate 
> (mis)use of "monthly". I have always quantified "monthly" this way, but 
> seems it is not correct. :)
> 
> Cheers,
> David
> 
> On 22/01/2020 1:37 am, Daniel D. Daugherty wrote:
>> On 1/19/20 8:49 PM, David Holmes wrote:
>>> gtest version:
>>>
>>> http://cr.openjdk.java.net/~dholmes/8235966/webrev.v4/
>>
>> src/hotspot/share/runtime/arguments.hpp
>> ???? No comments.
>>
>> src/hotspot/share/runtime/arguments.cpp
>> ???? L715: // check for stale flags when we hit build 20 (which is far 
>> enough into the 6 monthly
>> ???????? typo: s/monthly/month/
>>
>> test/hotspot/gtest/runtime/test_special_flags.cpp
>> ???? Nice!
>>
>> Thumbs up. I don't need to see a new webrev to fix the typo.
>>
>> Dan
>>
>>
>>>
>>> Bug report updated to show gtest output.
>>>
>>> Thanks,
>>> David
>>>
>>> On 20/01/2020 7:46 am, David Holmes wrote:
>>>> Hi Dan,
>>>>
>>>> On 18/01/2020 1:37 am, Daniel D. Daugherty wrote:
>>>>> On 1/16/20 12:57 AM, David Holmes wrote:
>>>>>> Getting back to this ...
>>>>>
>>>>> You added this update to the bug report:
>>>>>
>>>>>> Update: after further discussion it has been proposed that we use 
>>>>>> the build number as the trigger for a whitebox or gtest that 
>>>>>> performs the currently disabled full verification of the flag 
>>>>>> table. So if a flag has not be obsoleted or expired as it should 
>>>>>> by build N** then we fail the gtest. This will complement the 
>>>>>> relaxing of the obsoletion check at the start of the release cycle. 
>>>>>
>>>>> I was expecting a new test in this latest webrev that would start 
>>>>> failing
>>>>> at Build 20... let me see what the latest webrev says...
>>>>
>>>> Mea culpa. When I started thinking about the test it was evident the 
>>>> test logic would just involve the existing commented out warning 
>>>> logic. So I thought lets just turn those warnings on at build 20 but 
>>>> ...
>>>>
>>>>>>
>>>>>> Please see updated webrev at:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dholmes/8235966/webrev/
>>>>>
>>>>> src/hotspot/share/runtime/arguments.cpp
>>>>> ???? No comments.
>>>>>
>>>>> So the changes are the same as the last round with the addition of
>>>>> enabling the following at B20:
>>>>>
>>>>> L755: ????????? warning("Global variable for obsolete special flag 
>>>>> entry \"%s\" should be removed", flag.name);
>>>>> L769: ????????? warning("Global variable for expired flag entry 
>>>>> \"%s\" should be removed", flag.name);
>>>>>
>>>>>
>>>>>> Apologies as I mistakenly overwrote the original instead of 
>>>>>> creating v3.
>>>>>>
>>>>>> This version expands on the original proposal by uncommenting the 
>>>>>> warnings about obsolete/expired flags that have not been removed 
>>>>>> from globals*.hpp, so that we don't forget to do this work. 
>>>>>> However these warnings are only enabled from build 20. I used 20 
>>>>>> as being approx 2/3 through the release cycle - long enough that 
>>>>>> the work should have been performed by then, whilst still leaving 
>>>>>> time to perform the work before RDP2. Of course we can tweak this 
>>>>>> number if there are issues with that choice.
>>>>>
>>>>> Okay... but doesn't this mean that every test would issue these 
>>>>> warnings
>>>>> as of B20 if we have not completed the work? So we have the 
>>>>> potential of
>>>>> a raft (some unknown number) of test failures due to unexpected 
>>>>> output in
>>>>> the form of these warning messages. And worse, these failures would be
>>>>> unrelated to actual issue... :-(
>>>>
>>>> ... as you note the end result is not really what we want - a single 
>>>> clearly failing test.
>>>>
>>>>> How about adding a diagnostic flag that enables these two warning
>>>>> messages (in addition to the B20 check). Add a single test that runs:
>>>>>
>>>>> ?? java -XX:+UnlockDiagnosticVMOptions 
>>>>> -XX:+FlagShouldNotBeDefinedCheck -version
>>>>>
>>>>> and when we hit B20, if there are still flags that haven't been 
>>>>> removed,
>>>>> then the test will fail and we'll have one test that fails (X the 
>>>>> number
>>>>> of configs that run the test).
>>>>
>>>> I definitely do not want to add another VM flag :) but will look at 
>>>> how to (re)run that logic as part of a gtest.
>>>>
>>>> Thanks,
>>>> David
>>>>
>>>>> Dan
>>>>>
>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> David
>>>>>>
>>>>>> On 20/12/2019 8:20 am, David Holmes wrote:
>>>>>>> Thanks Dan.
>>>>>>>
>>>>>>> FTR I've updated the bug report with an extension to this 
>>>>>>> proposal, which is to add back the flag table validation checks 
>>>>>>> to use via a gtest that we only enable after a certain build in 
>>>>>>> the release cycle (it always passes before then). That way we 
>>>>>>> avoid the problems I've outlined with the initial version bump 
>>>>>>> but also have a safety net in place to ensure we don't forget to 
>>>>>>> actually obsolete/expire flags.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> David
>>>>>>>
>>>>>>> On 19/12/2019 3:37 am, Daniel D. Daugherty wrote:
>>>>>>>> Hi David,
>>>>>>>>
>>>>>>>> On 12/17/19 5:03 PM, David Holmes wrote:
>>>>>>>>> Hi Dan,
>>>>>>>>>
>>>>>>>>> Thanks for taking a look. Updated webrev:
>>>>>>>>>
>>>>>>>>> http://cr.openjdk.java.net/~dholmes/8235966/webrev.v2/
>>>>>>>>
>>>>>>>> src/hotspot/share/runtime/arguments.cpp
>>>>>>>> ???? I like the updates to header comment for 
>>>>>>>> verify_special_jvm_flags().
>>>>>>>>
>>>>>>>> Thumbs up.
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Discussion below.
>>>>>>>>
>>>>>>>> Replies below.
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 18/12/2019 1:47 am, Daniel D. Daugherty wrote:
>>>>>>>>>> On 12/16/19 12:16 AM, David Holmes wrote:
>>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8235966
>>>>>>>>>>> webrev: http://cr.openjdk.java.net/~dholmes/8235966/webrev/
>>>>>>>>>>
>>>>>>>>>> src/hotspot/share/runtime/arguments.cpp
>>>>>>>>>> ???? L745: ????? // if flag has become obsolete it should not 
>>>>>>>>>> have a "globals" flag defined anymore.
>>>>>>>>>> ???? L746: ????? if 
>>>>>>>>>> (!version_less_than(JDK_Version::current(), flag.obsolete_in)) {
>>>>>>>>>> ???? L747: ??????? if (JVMFlag::find_declared_flag(flag.name) 
>>>>>>>>>> != NULL) {
>>>>>>>>>> ???? L748: ????????? // Temporarily disable the warning: 8196739
>>>>>>>>>> ???? L749: ????????? // warning("Global variable for obsolete 
>>>>>>>>>> special flag entry \"%s\" should be removed", flag.name);
>>>>>>>>>> ???? L750: ??????? }
>>>>>>>>>> ???? L751: ????? }
>>>>>>>>>> ???????? It seems like we've been down a similar road before:
>>>>>>>>>>
>>>>>>>>>> ???????? JDK-8196739 Disable obsolete/expired VM flag 
>>>>>>>>>> transitional warnings
>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8196739
>>>>>>>>>>
>>>>>>>>>> ???????? This one may ring a bell... Fixed by dholmes in 
>>>>>>>>>> jdk11-b01... :-)
>>>>>>>>>>
>>>>>>>>>> ???????? And this followup sub-task to re-enable that warning:
>>>>>>>>>>
>>>>>>>>>> ???????? JDK-8196741 Re-enable obsolete/expired VM flag 
>>>>>>>>>> transitional warnings
>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8196741
>>>>>>>>>>
>>>>>>>>>> ???????? was closed as "Won't fix" on 2019.08.02.
>>>>>>>>>>
>>>>>>>>>> ???????? So the obvious questions:
>>>>>>>>>>
>>>>>>>>>> ???????? - Why is the new warning less problematic to tests 
>>>>>>>>>> that don't
>>>>>>>>>> ?????????? tolerate unexpected output?
>>>>>>>>>
>>>>>>>>> Two different situations. The commented out warning happens 
>>>>>>>>> unconditionally when you run the VM and it finds any flag 
>>>>>>>>> marked obsolete that hasn't been removed. Hence every single 
>>>>>>>>> test will encounter this warning.
>>>>>>>>
>>>>>>>> Ouch on such verbosity.
>>>>>>>>
>>>>>>>>
>>>>>>>>> The situation I am modifying is when a test uses a flag that is 
>>>>>>>>> marked for obsoletion. In the majority of cases the flag is 
>>>>>>>>> already deprecated and so already issuing a deprecation warning 
>>>>>>>>> that the test has to handle. Without my change there would 
>>>>>>>>> still be an obsoletion warning, so this test is in for a 
>>>>>>>>> warning no matter what.
>>>>>>>>
>>>>>>>> Good that your change only comes into play when the flag is used.
>>>>>>>>
>>>>>>>>
>>>>>>>>> Also note that for hotspot at least we have strived to make 
>>>>>>>>> tests tolerate unexpected output. The reason JDK-8196741 was 
>>>>>>>>> closed as "won't fix" was because other areas wouldn't commit 
>>>>>>>>> to doing that.
>>>>>>>>
>>>>>>>> Yup. Got it.
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>> ???????? - If you move forward with this fix, then I think 
>>>>>>>>>> think code
>>>>>>>>>> ?????????? block needs to be removed or modified or am I 
>>>>>>>>>> missing something?
>>>>>>>>>
>>>>>>>>> I've rewritten the comment at the head of 
>>>>>>>>> verify_special_jvm_flags to explain why we can't issue a 
>>>>>>>>> warning, and have deleted the block.
>>>>>>>>
>>>>>>>> Thanks for deleting the stale code.
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>> ???????? There's a similar commented out check on L757-L765, 
>>>>>>>>>> but that one
>>>>>>>>>> ???????? is for an expired flag... You might want to 
>>>>>>>>>> adjust/delete it also?
>>>>>>>>>
>>>>>>>>> Deleted.
>>>>>>>>
>>>>>>>> Thanks.
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>> ???? L753: warning("Special flag entry \"%s\" must be 
>>>>>>>>>> explicitly obsoleted before expired.", flag.name);
>>>>>>>>>> ???? L754: ??????? success = false;
>>>>>>>>>> ???????? nit - s/before expired/before being expired/
>>>>>>>>>> ???????? Update: I now see that "style" is in several places 
>>>>>>>>>> in this
>>>>>>>>>> ???????????? function. I'm not sure what to think here... it 
>>>>>>>>>> grates,
>>>>>>>>>> ?? ? ? ? ? ? but I can live with it.
>>>>>>>>>>
>>>>>>>>>> ???????? nit - L75[34] indented too much by two spaces.
>>>>>>>>>
>>>>>>>>> Fixed.
>>>>>>>>>
>>>>>>>>>> ???? L962: ????????? return real_name;
>>>>>>>>>> ???????? nit - indented too much by two spaces.
>>>>>>>>>
>>>>>>>>> Fixed.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Trying to understand the modified logic in argument processing is
>>>>>>>>>> making my head spin...
>>>>>>>>>
>>>>>>>>> Mine too. It took a few attempts to put the logic in the right 
>>>>>>>>> place and make adjustments so that it all works as expected for 
>>>>>>>>> a correctly specified flag and an erroneous one.
>>>>>>>>
>>>>>>>> I keep trying to convince myself that we're improving this flag and
>>>>>>>> options code with each release... :-)
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>> - You've added a call to is_obsolete_flag() in
>>>>>>>>>> ?? handle_aliases_and_deprecation() and is_obsolete_flag()
>>>>>>>>>> ?? is where the new warning is output:
>>>>>>>>>>
>>>>>>>>>> ???? warning("Temporarily processing option %s; support is 
>>>>>>>>>> scheduled for removal in %s"
>>>>>>>>>>
>>>>>>>>>> ?? handle_aliases_and_deprecation() is called from six 
>>>>>>>>>> different places,
>>>>>>>>>> ?? but the call sites are different based on the argument 
>>>>>>>>>> pattern so I
>>>>>>>>>> ?? have (mostly) convinced myself that there should not be any 
>>>>>>>>>> duplicate
>>>>>>>>>> ?? warning lines.
>>>>>>>>>
>>>>>>>>> Right - handle_aliases_and_deprecation is only called for a 
>>>>>>>>> syntactically correct flag based on those patterns. It normally 
>>>>>>>>> filters out obsoleted/expired flags and lets them fall through 
>>>>>>>>> to later error processing (in process_argument after parse_arg 
>>>>>>>>> returns false). That error processing is where the normal 
>>>>>>>>> obsoletion check is performed. So I had to not filter the flag 
>>>>>>>>> in handle_aliases_and_deprecation in this case, but still 
>>>>>>>>> produce the warning for a malformed flag. E.g.
>>>>>>>>>
>>>>>>>>> java -XX:+UseParallelOldGC -version
>>>>>>>>> Java HotSpot(TM) 64-Bit Server VM warning: Temporarily 
>>>>>>>>> processing option UseParallelOldGC; support is scheduled for 
>>>>>>>>> removal in 15.0
>>>>>>>>> java version "15-internal" 2020-09-15
>>>>>>>>>
>>>>>>>>> java -XX:UseParallelOldGC -version
>>>>>>>>> Java HotSpot(TM) 64-Bit Server VM warning: Temporarily 
>>>>>>>>> processing option UseParallelOldGC; support is scheduled for 
>>>>>>>>> removal in 15.0
>>>>>>>>> Missing +/- setting for VM option 'UseParallelOldGC'
>>>>>>>>> Error: Could not create the Java Virtual Machine.
>>>>>>>>
>>>>>>>> Thanks for the example. That helps a lot.
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>> So I now understand the new logic that allows an obsoleted option
>>>>>>>>>> to be specified with a warning as long as the option still 
>>>>>>>>>> exists.
>>>>>>>>>> I'm good with the technical change, but...
>>>>>>>>>>
>>>>>>>>>> I'm worried about tests that don't tolerate the new warning mesg,
>>>>>>>>>> i.e., why wouldn't this become an issue again:
>>>>>>>>>>
>>>>>>>>>> JDK-8196739 Disable obsolete/expired VM flag transitional 
>>>>>>>>>> warnings
>>>>>>>>>
>>>>>>>>> Explained above.
>>>>>>>>
>>>>>>>> Yup and thanks.
>>>>>>>>
>>>>>>>> Dan
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> David
>>>>>>>>>
>>>>>>>>>> Dan
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> When a flag is marked as obsolete in the special-flags table 
>>>>>>>>>>> we will ignore it and issue a warning that it is being 
>>>>>>>>>>> ignored, as soon as we bump the version of the JDK. That 
>>>>>>>>>>> means that any tests still using the obsolete flag may start 
>>>>>>>>>>> to fail, leading to a surge of test failures at the start of 
>>>>>>>>>>> a release cycle. For example for JDK 15 we have a whole bunch 
>>>>>>>>>>> of JFR tests that fail because they still try to work with 
>>>>>>>>>>> UseParallelOldGC. In another case 
>>>>>>>>>>> runtime/cds/appcds/FieldLayoutFlags.java passes only be 
>>>>>>>>>>> accident.
>>>>>>>>>>>
>>>>>>>>>>> When a flag is marked as obsolete for a release, all code 
>>>>>>>>>>> involving that flag (including tests that use it) must be 
>>>>>>>>>>> updated within that release and the flag itself removed. 
>>>>>>>>>>> Whilst this is typically scheduled early in a release cycle 
>>>>>>>>>>> it isn't reasonable to expect it to all occur within the 
>>>>>>>>>>> first couple of days of the release cycle, nor do we want to 
>>>>>>>>>>> have to ProblemList a bunch of tests when they start failing.
>>>>>>>>>>>
>>>>>>>>>>> What I propose is to instead allow an obsolete flag to 
>>>>>>>>>>> continue to be processed as long as that code removal has not 
>>>>>>>>>>> actually occurred - with an adjusted warning. The change I 
>>>>>>>>>>> propose:
>>>>>>>>>>>
>>>>>>>>>>> - only treats an obsolete flag as obsolete if the flag cannot 
>>>>>>>>>>> be found
>>>>>>>>>>> - added a new flag verification rule that disallows 
>>>>>>>>>>> obsoletion in an undefined version, but expiration in a 
>>>>>>>>>>> specific version i.e. we must always explicitly obsolete a 
>>>>>>>>>>> flag before we expire it.
>>>>>>>>>>>
>>>>>>>>>>> The only downside here is that if we actually forget to file 
>>>>>>>>>>> an issue for the actual obsoletion work we may not notice via 
>>>>>>>>>>> testing. Of course whenever a change is made to the flags 
>>>>>>>>>>> table to add an entry then the issue to do the obsoletion 
>>>>>>>>>>> should be filed at the same time.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> David
>>>>>>>>>>> -----
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>
>>

From kim.barrett at oracle.com  Wed Jan 22 05:01:41 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 22 Jan 2020 00:01:41 -0500
Subject: RFR (M): 8235741: Inappropriate uses of os::javaTimeMillis()
In-Reply-To: <50a1ee24-61a9-44fa-2590-69173765e9de@oracle.com>
References: <f99194ad-3dad-48db-d8d4-de0cc9161701@oracle.com>
 <160821DF-BE15-435C-BDBE-AB553F794590@oracle.com>
 <a04e7b9f-3fe8-01ea-9784-fcf21613340d@oracle.com>
 <EDE2324D-DA2D-4CF7-BAE0-EF59BEEB960B@oracle.com>
 <f3e75dc8-4ee9-9fad-d5a3-94c0e813abd5@oracle.com>
 <50a1ee24-61a9-44fa-2590-69173765e9de@oracle.com>
Message-ID: <AF1C900A-B2CD-4EC9-9F6D-B9E9AD0BC3BC@oracle.com>

> On Jan 17, 2020, at 12:48 AM, David Holmes <david.holmes at oracle.com> wrote:
> 
> I tried but failed to generate an incremental webrev at:
> 
> http://cr.openjdk.java.net/~dholmes/8235741/webrev.v3-incr/
> 
> I hadn't qfresh'd and so it contains v2 and v3 changes :(
> 
> Full webrev at:
> 
> http://cr.openjdk.java.net/~dholmes/8235741/webrev.v3/
> 
> Changes:
> - conversion macros are now typed functions
> - os_posix.cpp had a name clash so fixed that and updated to use the new conversion functions

Looks good.


From david.holmes at oracle.com  Wed Jan 22 06:53:57 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 22 Jan 2020 16:53:57 +1000
Subject: RFR (M): 8235741: Inappropriate uses of os::javaTimeMillis()
In-Reply-To: <AF1C900A-B2CD-4EC9-9F6D-B9E9AD0BC3BC@oracle.com>
References: <f99194ad-3dad-48db-d8d4-de0cc9161701@oracle.com>
 <160821DF-BE15-435C-BDBE-AB553F794590@oracle.com>
 <a04e7b9f-3fe8-01ea-9784-fcf21613340d@oracle.com>
 <EDE2324D-DA2D-4CF7-BAE0-EF59BEEB960B@oracle.com>
 <f3e75dc8-4ee9-9fad-d5a3-94c0e813abd5@oracle.com>
 <50a1ee24-61a9-44fa-2590-69173765e9de@oracle.com>
 <AF1C900A-B2CD-4EC9-9F6D-B9E9AD0BC3BC@oracle.com>
Message-ID: <86333217-20d5-0c3f-e3ea-7330cfbdc677@oracle.com>

Thanks Kim :)

David

On 22/01/2020 3:01 pm, Kim Barrett wrote:
>> On Jan 17, 2020, at 12:48 AM, David Holmes <david.holmes at oracle.com> wrote:
>>
>> I tried but failed to generate an incremental webrev at:
>>
>> http://cr.openjdk.java.net/~dholmes/8235741/webrev.v3-incr/
>>
>> I hadn't qfresh'd and so it contains v2 and v3 changes :(
>>
>> Full webrev at:
>>
>> http://cr.openjdk.java.net/~dholmes/8235741/webrev.v3/
>>
>> Changes:
>> - conversion macros are now typed functions
>> - os_posix.cpp had a name clash so fixed that and updated to use the new conversion functions
> 
> Looks good.
> 

From david.holmes at oracle.com  Wed Jan 22 06:56:48 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 22 Jan 2020 16:56:48 +1000
Subject: RFR (M): 8235741: Inappropriate uses of os::javaTimeMillis()
In-Reply-To: <75f4cd8d-d291-2cf5-b0dd-91a1d76a6130@oracle.com>
References: <f99194ad-3dad-48db-d8d4-de0cc9161701@oracle.com>
 <75f4cd8d-d291-2cf5-b0dd-91a1d76a6130@oracle.com>
Message-ID: <90b6c9fe-d0fb-3889-ec42-b9db607a9334@oracle.com>

Hi Robbin,

Can you confirm you are okay with latest changes please.

http://cr.openjdk.java.net/~dholmes/8235741/webrev.v3/

Thanks,
David
-----

On 13/01/2020 11:08 pm, Robbin Ehn wrote:
> Hi David, looks good, thanks!
> 
> /Robbin
> 
> On 1/13/20 8:13 AM, David Holmes wrote:
>> webrev: http://cr.openjdk.java.net/~dholmes/8235741/webrev/
>> bug: https://bugs.openjdk.java.net/browse/JDK-8235741
>>
>> Full details in the bug report about the existing uses of 
>> javaTimeMillis(), many of which just want an elapsed time in ms and so 
>> should be using javaTimeNanos() and convert to ms. This covers areas 
>> all across the VM.
>>
>> Only non-simple change is in os_perf_linux.cpp (and the same code will 
>> be in os_perf_aix.cpp once it has been validated). There we are 
>> tracking an elapsed time in ms but relative to the boot time, which is 
>> seconds since the epoch. Consequently the first interval has to be 
>> calculated using javaTimeMillis, but after that we can use 
>> javaTimeNanos (using a new 'first time' captured at the same time we 
>> used javaTimeMillis). I think I have the logic right but other than 
>> through JFR this code seems unused and I have limited means of testing 
>> it. The JFR test jdk/jfr/event/os/TestThreadContextSwitches.java 
>> exercises the code but the results of running that test seems to 
>> exhibit arbitrary randomness in the rates reported - e.g. 0 to 16000Hz 
>> - both with and without my change, so not really that useful. Stefan 
>> K. suggested a gtest which I may look into - though it is frustrating 
>> to have to expend such effort to validate this.
>>
>> Other testing tiers 1-3.
>>
>> Thanks,
>> David

From robbin.ehn at oracle.com  Wed Jan 22 08:54:50 2020
From: robbin.ehn at oracle.com (Robbin Ehn)
Date: Wed, 22 Jan 2020 09:54:50 +0100
Subject: RFR (M): 8235741: Inappropriate uses of os::javaTimeMillis()
In-Reply-To: <90b6c9fe-d0fb-3889-ec42-b9db607a9334@oracle.com>
References: <f99194ad-3dad-48db-d8d4-de0cc9161701@oracle.com>
 <75f4cd8d-d291-2cf5-b0dd-91a1d76a6130@oracle.com>
 <90b6c9fe-d0fb-3889-ec42-b9db607a9334@oracle.com>
Message-ID: <32ef609e-c43b-76e9-f801-e70dac35cf80@oracle.com>

Hi David, yes, thanks!

/Robbin

On 1/22/20 7:56 AM, David Holmes wrote:
> Hi Robbin,
> 
> Can you confirm you are okay with latest changes please.
> 
> http://cr.openjdk.java.net/~dholmes/8235741/webrev.v3/
> 
> Thanks,
> David
> -----
> 
> On 13/01/2020 11:08 pm, Robbin Ehn wrote:
>> Hi David, looks good, thanks!
>>
>> /Robbin
>>
>> On 1/13/20 8:13 AM, David Holmes wrote:
>>> webrev: http://cr.openjdk.java.net/~dholmes/8235741/webrev/
>>> bug: https://bugs.openjdk.java.net/browse/JDK-8235741
>>>
>>> Full details in the bug report about the existing uses of javaTimeMillis(), 
>>> many of which just want an elapsed time in ms and so should be using 
>>> javaTimeNanos() and convert to ms. This covers areas all across the VM.
>>>
>>> Only non-simple change is in os_perf_linux.cpp (and the same code will be in 
>>> os_perf_aix.cpp once it has been validated). There we are tracking an elapsed 
>>> time in ms but relative to the boot time, which is seconds since the epoch. 
>>> Consequently the first interval has to be calculated using javaTimeMillis, 
>>> but after that we can use javaTimeNanos (using a new 'first time' captured at 
>>> the same time we used javaTimeMillis). I think I have the logic right but 
>>> other than through JFR this code seems unused and I have limited means of 
>>> testing it. The JFR test jdk/jfr/event/os/TestThreadContextSwitches.java 
>>> exercises the code but the results of running that test seems to exhibit 
>>> arbitrary randomness in the rates reported - e.g. 0 to 16000Hz - both with 
>>> and without my change, so not really that useful. Stefan K. suggested a gtest 
>>> which I may look into - though it is frustrating to have to expend such 
>>> effort to validate this.
>>>
>>> Other testing tiers 1-3.
>>>
>>> Thanks,
>>> David

From david.holmes at oracle.com  Wed Jan 22 09:48:46 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 22 Jan 2020 19:48:46 +1000
Subject: RFR (M): 8235741: Inappropriate uses of os::javaTimeMillis()
In-Reply-To: <32ef609e-c43b-76e9-f801-e70dac35cf80@oracle.com>
References: <f99194ad-3dad-48db-d8d4-de0cc9161701@oracle.com>
 <75f4cd8d-d291-2cf5-b0dd-91a1d76a6130@oracle.com>
 <90b6c9fe-d0fb-3889-ec42-b9db607a9334@oracle.com>
 <32ef609e-c43b-76e9-f801-e70dac35cf80@oracle.com>
Message-ID: <9ba98e60-8069-d37d-3ed4-74408f10306f@oracle.com>

Thanks Robbin!

David

On 22/01/2020 6:54 pm, Robbin Ehn wrote:
> Hi David, yes, thanks!
> 
> /Robbin
> 
> On 1/22/20 7:56 AM, David Holmes wrote:
>> Hi Robbin,
>>
>> Can you confirm you are okay with latest changes please.
>>
>> http://cr.openjdk.java.net/~dholmes/8235741/webrev.v3/
>>
>> Thanks,
>> David
>> -----
>>
>> On 13/01/2020 11:08 pm, Robbin Ehn wrote:
>>> Hi David, looks good, thanks!
>>>
>>> /Robbin
>>>
>>> On 1/13/20 8:13 AM, David Holmes wrote:
>>>> webrev: http://cr.openjdk.java.net/~dholmes/8235741/webrev/
>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8235741
>>>>
>>>> Full details in the bug report about the existing uses of 
>>>> javaTimeMillis(), many of which just want an elapsed time in ms and 
>>>> so should be using javaTimeNanos() and convert to ms. This covers 
>>>> areas all across the VM.
>>>>
>>>> Only non-simple change is in os_perf_linux.cpp (and the same code 
>>>> will be in os_perf_aix.cpp once it has been validated). There we are 
>>>> tracking an elapsed time in ms but relative to the boot time, which 
>>>> is seconds since the epoch. Consequently the first interval has to 
>>>> be calculated using javaTimeMillis, but after that we can use 
>>>> javaTimeNanos (using a new 'first time' captured at the same time we 
>>>> used javaTimeMillis). I think I have the logic right but other than 
>>>> through JFR this code seems unused and I have limited means of 
>>>> testing it. The JFR test 
>>>> jdk/jfr/event/os/TestThreadContextSwitches.java exercises the code 
>>>> but the results of running that test seems to exhibit arbitrary 
>>>> randomness in the rates reported - e.g. 0 to 16000Hz - both with and 
>>>> without my change, so not really that useful. Stefan K. suggested a 
>>>> gtest which I may look into - though it is frustrating to have to 
>>>> expend such effort to validate this.
>>>>
>>>> Other testing tiers 1-3.
>>>>
>>>> Thanks,
>>>> David

From vladimir.x.ivanov at oracle.com  Wed Jan 22 11:52:25 2020
From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov)
Date: Wed, 22 Jan 2020 14:52:25 +0300
Subject: [15] RFR (L): 7175279: Don't use x87 FPU on x86-64
In-Reply-To: <7063EB29-D415-4A48-BA4F-B16C5A1F52F8@oracle.com>
References: <0b0897e0-1dbc-306d-b2cb-31de13fb8b34@oracle.com>
 <7063EB29-D415-4A48-BA4F-B16C5A1F52F8@oracle.com>
Message-ID: <16d75ed8-896b-25fe-0a0a-babc9087e631@oracle.com>

Thanks, Vladimir.

Anybody for a second (R)eview, please?

   http://cr.openjdk.java.net/~vlivanov/7175279/webrev.00/
   +
   http://cr.openjdk.java.net/~vlivanov/7175279/webrev.01-00/

Best regards,
Vladimir Ivanov

On 20.12.2019 01:00, Vladimir Kozlov wrote:
> Good
> 
> Thanks
> Vladimir
> 
>> On Dec 19, 2019, at 1:58 PM, Vladimir Ivanov <vladimir.x.ivanov at oracle.com> wrote:
>>
>> ?
>>>>> Make ConversionStub x86_32-specific only if possible. From what I see it is only LIR_OpConvert in c1_LIR.hpp we have to deal with. I actually can't see how it could be only 32-specific. Hmm?
>>>>
>>>> I experimented with it, but it requires #ifdefs in c1_LIR.cpp/hpp which I don't like. So, I don't consider it as an  > option right now.
>>> Okay, NOT_IA32( ShouldNotReachHere() ) with comment in c1_CodeStubs.hpp should be enough for now.
>>
>> Incremental diff:
>>   http://cr.openjdk.java.net/~vlivanov/7175279/webrev.01-00/
>>
>> Best regards,
>> Vladimir Ivanov
>>
>>>>>>> c1_LinearScan.cpp - I think IA64 is used for Itanium. For 64-bit x86 we use AMD64:
>>>>>>>
>>>>>>> https://hg.openjdk.java.net/jdk/jdk/file/cfaa2457a60a/src/hotspot/share/utilities/macros.hpp#l456
>>>>>>
>>>>>>
>>>>>>
>>>>>> Yes, you are right. Good catch! :-)
>>>>>>
>>>>>> Best regards,
>>>>>> Vladimir Ivanov
>>>>>>
>>>>>>> On 12/17/19 4:50 AM, Vladimir Ivanov wrote:
>>>>>>>> http://cr.openjdk.java.net/~vlivanov/7175279/webrev.00/
>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-7175279
>>>>>>>>
>>>>>>>> There was a major rewrite of math intrinsics which in 9 time frame which almost completely eliminated x87 code in x86-64 code base.
>>>>>>>>
>>>>>>>> Proposed patch removes the rest and makes x86-64 code x87-free.
>>>>>>>>
>>>>>>>> The main motivation for the patch is to completely eliminate non-strictfp behaving code in order to prepare the JVM for JEP 306 [1] and related enhancements [2].
>>>>>>>>
>>>>>>>> Most of the changes are in C1, but there is one case in template interpreter (java_lang_math_abs) which now uses StubRoutines::x86::double_sign_mask(). It forces its initialization to be moved to StubRoutines::initialize1().
>>>>>>>>
>>>>>>>> x87 instructions are made available only on x86-32.
>>>>>>>>
>>>>>>>> C1 changes involve removing FPU support on x86-64 and effectively make x86-specific support in linear scan allocator [2] x86-32-only.
>>>>>>>>
>>>>>>>> Testing: tier1-6, build on x86-23, linux-aarch64, linux-arm32.
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>> Vladimir Ivanov
>>>>>>>>
>>>>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8175916
>>>>>>>>
>>>>>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8136414
>>>>>>>>
>>>>>>>> [3] http://hg.openjdk.java.net/jdk/jdk/file/ff7cd49f2aef/src/hotspot/cpu/x86/c1_LinearScan_x86.cpp
> 

From stefan.karlsson at oracle.com  Wed Jan 22 13:33:19 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Wed, 22 Jan 2020 14:33:19 +0100
Subject: RFR: 8237637: Remove dubious type conversions from oop
Message-ID: <0d5e0216-6de5-2bd9-d258-0e5ef7b3a3f3@oracle.com>

Hi all,

Please review this patch to remove some dubious type conversions from oop.

https://bugs.openjdk.java.net/browse/JDK-8237637

When running with fastdebug and CHECK_UNHANDLED_OOPS oop is not a simple 
oopDesc* but a class. This class has a number of type conversions with 
the comment:

   // Explict user conversions

However, these are not only *explicit* conversions. They can be invoked 
implicitly as well.

I propose that we get rid of most of these and only leave these two:
  operator void* () const
  operator oopDesc* () const

so that we get better type safety in the code.

I've split this up into multiple webrevs to make it easier to review:

https://cr.openjdk.java.net/~stefank/8237637/webrev.01/00.promotedObject/
https://cr.openjdk.java.net/~stefank/8237637/webrev.01/01.oopStar/
https://cr.openjdk.java.net/~stefank/8237637/webrev.01/02.jobject/
https://cr.openjdk.java.net/~stefank/8237637/webrev.01/03.address/
https://cr.openjdk.java.net/~stefank/8237637/webrev.01/03.address/
https://cr.openjdk.java.net/~stefank/8237637/webrev.01/05.voidStarVolatiles/
https://cr.openjdk.java.net/~stefank/8237637/webrev.01/06.heapWord/

All changes combined:
https://cr.openjdk.java.net/~stefank/8237637/webrev.01/all/

Testing: builds pass, tier1-3 running

Thanks,
StefanK


From matthias.baesken at sap.com  Wed Jan 22 13:33:24 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 22 Jan 2020 13:33:24 +0000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <AM6PR02MB4200B721BD0C7728B21E2C4193310@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
 <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>
 <AM6PR02MB4200B721BD0C7728B21E2C4193310@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <AM6PR02MB4200BCFF64D8C46340B17A6F930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hi Magnus / David,  here is a new webrev :


http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.4/


it supports now a  configure switch  --enable-linktime-gc=yes  that needs to be set  to enable the link time section gc  .

Exception is linuxs390x  where we already have the  feature enabled  (and keep it enabled always for LIB_JVM).

Best regards, Matthias



From: Baesken, Matthias
Sent: Freitag, 17. Januar 2020 12:44
To: Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>; David Holmes <david.holmes at oracle.com>; 'build-dev at openjdk.java.net' <build-dev at openjdk.java.net>; 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
Subject: RE: RFR: 8236714: enable link-time section-gc for linux to remove unused code



  *   Matthias: Have a look at some recently added option to get an indication of the best practice in adding new options. There are some ways to easily make this incorrect

Hi Magnus, do you have a good/?best practice?  example  (not that I catch a bad one ?? )  ?

Best regards, Matthias


From coleen.phillimore at oracle.com  Wed Jan 22 13:52:37 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Wed, 22 Jan 2020 08:52:37 -0500
Subject: RFR: 8237637: Remove dubious type conversions from oop
In-Reply-To: <0d5e0216-6de5-2bd9-d258-0e5ef7b3a3f3@oracle.com>
References: <0d5e0216-6de5-2bd9-d258-0e5ef7b3a3f3@oracle.com>
Message-ID: <9a2f3283-a807-8a5c-f122-958662909e97@oracle.com>


I really like this and have only started clicking.

  174 template <class T> inline T cast_from_oop(oop o) {
  175   return (T)(CHECK_UNHANDLED_OOPS_ONLY((void*))o);
  176 }


Why did you leave void* as a conversion??? Can you define 
cast_from_oop<> to not need it?

Thanks,
Coleen

On 1/22/20 8:33 AM, Stefan Karlsson wrote:
> Hi all,
>
> Please review this patch to remove some dubious type conversions from 
> oop.
>
> https://bugs.openjdk.java.net/browse/JDK-8237637
>
> When running with fastdebug and CHECK_UNHANDLED_OOPS oop is not a 
> simple oopDesc* but a class. This class has a number of type 
> conversions with the comment:
>
> ? // Explict user conversions
>
> However, these are not only *explicit* conversions. They can be 
> invoked implicitly as well.
>
> I propose that we get rid of most of these and only leave these two:
> ?operator void* () const
> ?operator oopDesc* () const
>
> so that we get better type safety in the code.
>
> I've split this up into multiple webrevs to make it easier to review:
>
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/00.promotedObject/
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/01.oopStar/
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/02.jobject/
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/03.address/
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/03.address/
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/05.voidStarVolatiles/ 
>
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/06.heapWord/
>
> All changes combined:
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/all/
>
> Testing: builds pass, tier1-3 running
>
> Thanks,
> StefanK
>


From stefan.karlsson at oracle.com  Wed Jan 22 14:33:07 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Wed, 22 Jan 2020 15:33:07 +0100
Subject: RFR: 8237637: Remove dubious type conversions from oop
In-Reply-To: <9a2f3283-a807-8a5c-f122-958662909e97@oracle.com>
References: <0d5e0216-6de5-2bd9-d258-0e5ef7b3a3f3@oracle.com>
 <9a2f3283-a807-8a5c-f122-958662909e97@oracle.com>
Message-ID: <d4df98ff-ff91-2a7d-ed25-bbfaf70f0a18@oracle.com>

On 2020-01-22 14:52, coleen.phillimore at oracle.com wrote:
> 
> I really like this and have only started clicking.
> 
>  ?174 template <class T> inline T cast_from_oop(oop o) {
>  ?175?? return (T)(CHECK_UNHANDLED_OOPS_ONLY((void*))o);
>  ?176 }
> 
> 
> Why did you leave void* as a conversion??? Can you define 
> cast_from_oop<> to not need it?

I thought it made sense to allow it to be converted to void*, but I now 
see that it already works because it can be converted through oopDesc: 
oop -> oopDesc* -> void*.

I removed the following line and compiled locally:
   operator void* () const             { return (void *)obj(); }

I'll make sure that it compiles with other compilers as well.

Thanks,
StefanK

> 
> Thanks,
> Coleen
> 
> On 1/22/20 8:33 AM, Stefan Karlsson wrote:
>> Hi all,
>>
>> Please review this patch to remove some dubious type conversions from 
>> oop.
>>
>> https://bugs.openjdk.java.net/browse/JDK-8237637
>>
>> When running with fastdebug and CHECK_UNHANDLED_OOPS oop is not a 
>> simple oopDesc* but a class. This class has a number of type 
>> conversions with the comment:
>>
>> ? // Explict user conversions
>>
>> However, these are not only *explicit* conversions. They can be 
>> invoked implicitly as well.
>>
>> I propose that we get rid of most of these and only leave these two:
>> ?operator void* () const
>> ?operator oopDesc* () const
>>
>> so that we get better type safety in the code.
>>
>> I've split this up into multiple webrevs to make it easier to review:
>>
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/00.promotedObject/
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/01.oopStar/
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/02.jobject/
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/03.address/
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/03.address/
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/05.voidStarVolatiles/ 
>>
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/06.heapWord/
>>
>> All changes combined:
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/all/
>>
>> Testing: builds pass, tier1-3 running
>>
>> Thanks,
>> StefanK
>>
> 

From erik.joelsson at oracle.com  Wed Jan 22 14:38:55 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Wed, 22 Jan 2020 06:38:55 -0800
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <AM6PR02MB4200BCFF64D8C46340B17A6F930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
 <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>
 <AM6PR02MB4200B721BD0C7728B21E2C4193310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <AM6PR02MB4200BCFF64D8C46340B17A6F930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <d3e3641f-d6ae-979d-d53f-93136da6d62a@oracle.com>

Hello Matthias,

You can keep the setting up of all the flags in flags-cflags.m4 and 
flags-ldflags.m4 based on the value of ENABLE_LINKTIME_GC. You can also 
default the value of this new parameter to true for s390x to keep the 
current behavior for that platform. As it is in this patch, the JVM 
flags for s390x are setup in configure while the JDK flags are in make, 
which gets confusing I think.

/Erik


On 2020-01-22 05:33, Baesken, Matthias wrote:
> Hi Magnus / David,  here is a new webrev :
>
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.4/
>
>
> it supports now a  configure switch  --enable-linktime-gc=yes  that needs to be set  to enable the link time section gc  .
>
> Exception is linuxs390x  where we already have the  feature enabled  (and keep it enabled always for LIB_JVM).
>
> Best regards, Matthias
>
>
>
> From: Baesken, Matthias
> Sent: Freitag, 17. Januar 2020 12:44
> To: Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>; David Holmes <david.holmes at oracle.com>; 'build-dev at openjdk.java.net' <build-dev at openjdk.java.net>; 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: RE: RFR: 8236714: enable link-time section-gc for linux to remove unused code
>
>
>
>    *   Matthias: Have a look at some recently added option to get an indication of the best practice in adding new options. There are some ways to easily make this incorrect
>
> Hi Magnus, do you have a good/?best practice?  example  (not that I catch a bad one ?? )  ?
>
> Best regards, Matthias
>

From erik.osterlund at oracle.com  Wed Jan 22 14:50:53 2020
From: erik.osterlund at oracle.com (erik.osterlund at oracle.com)
Date: Wed, 22 Jan 2020 15:50:53 +0100
Subject: RFR: 8237637: Remove dubious type conversions from oop
In-Reply-To: <d4df98ff-ff91-2a7d-ed25-bbfaf70f0a18@oracle.com>
References: <0d5e0216-6de5-2bd9-d258-0e5ef7b3a3f3@oracle.com>
 <9a2f3283-a807-8a5c-f122-958662909e97@oracle.com>
 <d4df98ff-ff91-2a7d-ed25-bbfaf70f0a18@oracle.com>
Message-ID: <b0ec2955-59ec-059f-a74d-509d3cc1c6db@oracle.com>

Hi Stefan,

Nice! Ship it!

Thanks,
/Erik

On 1/22/20 3:33 PM, Stefan Karlsson wrote:
> On 2020-01-22 14:52, coleen.phillimore at oracle.com wrote:
>>
>> I really like this and have only started clicking.
>>
>> ??174 template <class T> inline T cast_from_oop(oop o) {
>> ??175?? return (T)(CHECK_UNHANDLED_OOPS_ONLY((void*))o);
>> ??176 }
>>
>>
>> Why did you leave void* as a conversion??? Can you define 
>> cast_from_oop<> to not need it?
>
> I thought it made sense to allow it to be converted to void*, but I 
> now see that it already works because it can be converted through 
> oopDesc: oop -> oopDesc* -> void*.
>
> I removed the following line and compiled locally:
> ? operator void* () const???????????? { return (void *)obj(); }
>
> I'll make sure that it compiles with other compilers as well.
>
> Thanks,
> StefanK
>
>>
>> Thanks,
>> Coleen
>>
>> On 1/22/20 8:33 AM, Stefan Karlsson wrote:
>>> Hi all,
>>>
>>> Please review this patch to remove some dubious type conversions 
>>> from oop.
>>>
>>> https://bugs.openjdk.java.net/browse/JDK-8237637
>>>
>>> When running with fastdebug and CHECK_UNHANDLED_OOPS oop is not a 
>>> simple oopDesc* but a class. This class has a number of type 
>>> conversions with the comment:
>>>
>>> ? // Explict user conversions
>>>
>>> However, these are not only *explicit* conversions. They can be 
>>> invoked implicitly as well.
>>>
>>> I propose that we get rid of most of these and only leave these two:
>>> ?operator void* () const
>>> ?operator oopDesc* () const
>>>
>>> so that we get better type safety in the code.
>>>
>>> I've split this up into multiple webrevs to make it easier to review:
>>>
>>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/00.promotedObject/ 
>>>
>>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/01.oopStar/
>>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/02.jobject/
>>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/03.address/
>>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/03.address/
>>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/05.voidStarVolatiles/ 
>>>
>>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/06.heapWord/
>>>
>>> All changes combined:
>>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/all/
>>>
>>> Testing: builds pass, tier1-3 running
>>>
>>> Thanks,
>>> StefanK
>>>
>>


From matthias.baesken at sap.com  Wed Jan 22 15:46:48 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 22 Jan 2020 15:46:48 +0000
Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk
 images
References: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com> 
Message-ID: <AM6PR02MB42007DB6FF70591EC151174A930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello,  here is an updated version  :

http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.3/

this one supports a configure switch  "--enable-stripped-pdbs"    to enable the feature .

Best regards, Matthias


> -----Original Message-----
> From: Baesken, Matthias
> Sent: Dienstag, 21. Januar 2020 11:03
> To: 'David Holmes' <david.holmes at oracle.com>; 'build-
> dev at openjdk.java.net' <build-dev at openjdk.java.net>; 'hotspot-
> dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: RE: RFR: 8237192: Generate stripped/public pdbs on Windows for
> jdk images
> 
> 
> Hi David ,  yes I think it makes sense to have a configure option for this .
> Not everyone would like to have a larger JDK (even it is only a bit larger).
> 


From matthias.baesken at sap.com  Wed Jan 22 16:01:59 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 22 Jan 2020 16:01:59 +0000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <d3e3641f-d6ae-979d-d53f-93136da6d62a@oracle.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
 <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>
 <AM6PR02MB4200B721BD0C7728B21E2C4193310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <AM6PR02MB4200BCFF64D8C46340B17A6F930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <d3e3641f-d6ae-979d-d53f-93136da6d62a@oracle.com>
Message-ID: <AM6PR02MB4200FB39C1F2AA0C1B6D8279930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>


Hi Erik, okay I'll check that .
I had the impression I would have ordering  issues of the m4 files and how they end up in the generated-configure.sh  but  looks like that?s not the case .

Best regards, Matthias


> 
> Hello Matthias,
> 
> You can keep the setting up of all the flags in flags-cflags.m4 and
> flags-ldflags.m4 based on the value of ENABLE_LINKTIME_GC. You can also
> default the value of this new parameter to true for s390x to keep the
> current behavior for that platform. As it is in this patch, the JVM
> flags for s390x are setup in configure while the JDK flags are in make,
> which gets confusing I think.
> 
> /Erik
> 


From erik.joelsson at oracle.com  Wed Jan 22 16:36:50 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Wed, 22 Jan 2020 08:36:50 -0800
Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk
 images
In-Reply-To: <AM6PR02MB42007DB6FF70591EC151174A930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com>
 <AM6PR02MB42007DB6FF70591EC151174A930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <df71f609-64a6-4997-0ff8-562a81be6134@oracle.com>

This still does not address anything in my objection.

/Erik

On 2020-01-22 07:46, Baesken, Matthias wrote:
> Hello,  here is an updated version  :
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.3/
>
> this one supports a configure switch  "--enable-stripped-pdbs"    to enable the feature .
>
> Best regards, Matthias
>
>
>> -----Original Message-----
>> From: Baesken, Matthias
>> Sent: Dienstag, 21. Januar 2020 11:03
>> To: 'David Holmes' <david.holmes at oracle.com>; 'build-
>> dev at openjdk.java.net' <build-dev at openjdk.java.net>; 'hotspot-
>> dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
>> Subject: RE: RFR: 8237192: Generate stripped/public pdbs on Windows for
>> jdk images
>>
>>
>> Hi David ,  yes I think it makes sense to have a configure option for this .
>> Not everyone would like to have a larger JDK (even it is only a bit larger).
>>

From david.holmes at oracle.com  Wed Jan 22 23:56:42 2020
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 23 Jan 2020 09:56:42 +1000
Subject: RFR: 8237637: Remove dubious type conversions from oop
In-Reply-To: <0d5e0216-6de5-2bd9-d258-0e5ef7b3a3f3@oracle.com>
References: <0d5e0216-6de5-2bd9-d258-0e5ef7b3a3f3@oracle.com>
Message-ID: <63365943-2b6a-2388-5934-62d8f179b291@oracle.com>

Hi Stefan,

I don't have any concerns with this, but if cast_to_oop is for 
"numerical conversions" then can't we get rid of the secondary casts to 
jlong and u8 where used i.e can


--- 
old/src/hotspot/share/jfr/leakprofiler/checkpoint/objectSampleWriter.cpp 
2020-01-22 14:24:21.194074776 +0100
+++ 
new/src/hotspot/share/jfr/leakprofiler/checkpoint/objectSampleWriter.cpp 
2020-01-22 14:24:20.714066939 +0100
@@ -206,7 +206,7 @@
    oop object = oosi->_data._object;
    assert(object != NULL, "invariant");
    writer->write(oosi->_id);
-  writer->write((u8)(const HeapWord*)object);
+  writer->write((u8)cast_from_oop<HeapWord*>(object));

not use:

writer->write(cast_from_oop<u8>(object));

and:

--- old/src/hotspot/share/jvmci/jvmciCompilerToVM.cpp	2020-01-22 
14:24:22.674098943 +0100
+++ new/src/hotspot/share/jvmci/jvmciCompilerToVM.cpp	2020-01-22 
14:24:22.142090256 +0100
@@ -431,7 +431,7 @@
        } else if (JVMCIENV->isa_HotSpotObjectConstantImpl(base_object)) {
          Handle base_oop = JVMCIENV->asConstant(base_object, 
JVMCI_CHECK_NULL);
          if (base_oop->is_a(SystemDictionary::Class_klass())) {
-          base_address = (jlong) (address) base_oop();
+          base_address = (jlong) cast_from_oop<address>(base_oop());

not use

           base_address = cast_from_oop<jlong>(base_oop());

?

Thanks,
David

On 22/01/2020 11:33 pm, Stefan Karlsson wrote:
> Hi all,
> 
> Please review this patch to remove some dubious type conversions from oop.
> 
> https://bugs.openjdk.java.net/browse/JDK-8237637
> 
> When running with fastdebug and CHECK_UNHANDLED_OOPS oop is not a simple 
> oopDesc* but a class. This class has a number of type conversions with 
> the comment:
> 
>  ? // Explict user conversions
> 
> However, these are not only *explicit* conversions. They can be invoked 
> implicitly as well.
> 
> I propose that we get rid of most of these and only leave these two:
>  ?operator void* () const
>  ?operator oopDesc* () const
> 
> so that we get better type safety in the code.
> 
> I've split this up into multiple webrevs to make it easier to review:
> 
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/00.promotedObject/
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/01.oopStar/
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/02.jobject/
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/03.address/
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/03.address/
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/05.voidStarVolatiles/ 
> 
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/06.heapWord/
> 
> All changes combined:
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/all/
> 
> Testing: builds pass, tier1-3 running
> 
> Thanks,
> StefanK
> 

From kim.barrett at oracle.com  Thu Jan 23 07:06:57 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 23 Jan 2020 02:06:57 -0500
Subject: RFR: 8237637: Remove dubious type conversions from oop
In-Reply-To: <0d5e0216-6de5-2bd9-d258-0e5ef7b3a3f3@oracle.com>
References: <0d5e0216-6de5-2bd9-d258-0e5ef7b3a3f3@oracle.com>
Message-ID: <B7BE471C-6E59-4EDC-BC1B-B4705E496643@oracle.com>

> On Jan 22, 2020, at 8:33 AM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
> 
> Hi all,
> 
> Please review this patch to remove some dubious type conversions from oop.
> 
> https://bugs.openjdk.java.net/browse/JDK-8237637
> 
> When running with fastdebug and CHECK_UNHANDLED_OOPS oop is not a simple oopDesc* but a class. This class has a number of type conversions with the comment:
> 
>  // Explict user conversions
> 
> However, these are not only *explicit* conversions. They can be invoked implicitly as well.
> 
> I propose that we get rid of most of these and only leave these two:
> operator void* () const
> operator oopDesc* () const
> 
> so that we get better type safety in the code.
> 
> I've split this up into multiple webrevs to make it easier to review:
> 
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/00.promotedObject/
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/01.oopStar/
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/02.jobject/
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/03.address/
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/03.address/
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/05.voidStarVolatiles/
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/06.heapWord/
> 
> All changes combined:
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/all/
> 
> Testing: builds pass, tier1-3 running
> 
> Thanks,
> StefanK

I like this.  A few minor cleanups below.  I have some additional
suggestions, but I think they are more suited to followup RFEs that
I'm planning to write up.

------------------------------------------------------------------------------
src/hotspot/share/compiler/oopMap.cpp
 795     *derived_loc = (oop)(cast_from_oop<address>(base) + offset);

Seems like that should also be using cast_to_oop.

------------------------------------------------------------------------------
src/hotspot/share/gc/g1/g1FullGCOopClosures.cpp
  80         HeapRegion* to   = _g1h->heap_region_containing(cast_from_oop<HeapWord*>(obj));

src/hotspot/share/gc/g1/g1ParScanThreadState.hpp
 132     HeapRegion* const hr_obj = _g1h->heap_region_containing(cast_from_oop<HeapWord*>(o));

src/hotspot/share/gc/g1/heapRegion.cpp
 569           HeapRegion* to = _g1h->heap_region_containing(cast_from_oop<HeapWord*>(obj));

No casts were needed in these places, so remove them rather than
converting them to cast_from_oop<HeapWord*>.

Similarly, there are some calls to operations on mark bit maps that
don't need casts because there are overloads on oop, so should have
the casts removed rather than being converted to cast_from_oop<HeapWord*>.

Similarly for is_in_cset.  And there might be other functions like
this that I didn't notice.

------------------------------------------------------------------------------
src/hotspot/share/gc/parallel/psCardTable.cpp
  53         _unmarked_addr = (HeapWord*)(void*)p;

The addition of the cast to void* seems unnecessary.

------------------------------------------------------------------------------
src/hotspot/share/oops/oopsHierarchy.hpp

When removing this line (as was mentioned in the thread)
 113   operator void* () const             { return (void *)obj(); }

I suggest also changing cast_from_oop to use oopDesc* rather than void*:
 175   return (T)(CHECK_UNHANDLED_OOPS_ONLY((oopDesc*))o);

Note that there are a lot of implicit conversions (mostly to void* I
think) that prevent us from removing (or making explicit with C++11)
the oopDesc* conversion operator.

------------------------------------------------------------------------------


From matthias.baesken at sap.com  Thu Jan 23 08:03:10 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Thu, 23 Jan 2020 08:03:10 +0000
Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk
 images
In-Reply-To: <df71f609-64a6-4997-0ff8-562a81be6134@oracle.com>
References: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com>
 <AM6PR02MB42007DB6FF70591EC151174A930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <df71f609-64a6-4997-0ff8-562a81be6134@oracle.com>
Message-ID: <AM6PR02MB420064F6F67387E6596F0E67930F0@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hi Erik,  yes true sorry for answering  your comments a bit late .

> If a user runs jlink and includes all the jmods we ship with the JDK, the result 
> should be essentially equivalent to the original JDK image. The way the 
> stripped pdb files are included in the bundles sort of at the last 
> second of the build here breaks this property.

I think we should address this in a separate bug/CR .
Looking for example  into a Linux  build, I see  a lot of debuginfo files  in the jdk image (more or less for every shared lib)  .
But when looking into the jmods  of that jdk image ,  no debuginfo files are in there ( I checked the java.base jmod).
So  putting  the  files with debug information into the jmods  seems to be something that was not done so far  cross platform  (or is there some build switch  for it that I did not check?) .
Maybe to keep the jmods as small as possible .


> To properly implement this, care will need to be taken to juggle the two 
> sets of pdb files around, making sure each build and test use case has 
> the correct one in place where and when it's needed. Quite possibly, we 
> cannot cover all use cases with one build configuration. Developers 
> needing the full debug symbols when debugging locally would likely need 
> to disable the stripped symbols so they get the full symbols everywhere. 
> Possibly this would need to be the default for debug builds and 
> configurable for release builds.

From my limited experience , the developers  do not work with the bundles (that  would contain now after my patch the stripped pds)  but with a "normal" jdk image that  is created my make all.

Best regards, Matthias



> 
> This still does not address anything in my objection.
> 
> /Erik
> 
> On 2020-01-22 07:46, Baesken, Matthias wrote:
> > Hello,  here is an updated version  :
> >
> > http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.3/
> >
> > this one supports a configure switch  "--enable-stripped-pdbs"    to enable
> the feature .
> >
> > Best regards, Matthias
> >
> >
> >> -----Original Message-----
> >> From: Baesken, Matthias
> >> Sent: Dienstag, 21. Januar 2020 11:03
> >> To: 'David Holmes' <david.holmes at oracle.com>; 'build-
> >> dev at openjdk.java.net' <build-dev at openjdk.java.net>; 'hotspot-
> >> dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> >> Subject: RE: RFR: 8237192: Generate stripped/public pdbs on Windows for
> >> jdk images
> >>
> >>
> >> Hi David ,  yes I think it makes sense to have a configure option for this .
> >> Not everyone would like to have a larger JDK (even it is only a bit larger).
> >>

From rwestrel at redhat.com  Thu Jan 23 08:05:57 2020
From: rwestrel at redhat.com (Roland Westrelin)
Date: Thu, 23 Jan 2020 09:05:57 +0100
Subject: [15] RFR (L): 7175279: Don't use x87 FPU on x86-64
In-Reply-To: <16d75ed8-896b-25fe-0a0a-babc9087e631@oracle.com>
References: <0b0897e0-1dbc-306d-b2cb-31de13fb8b34@oracle.com>
 <7063EB29-D415-4A48-BA4F-B16C5A1F52F8@oracle.com>
 <16d75ed8-896b-25fe-0a0a-babc9087e631@oracle.com>
Message-ID: <87h80my4uy.fsf@redhat.com>


>    http://cr.openjdk.java.net/~vlivanov/7175279/webrev.00/
>    +
>    http://cr.openjdk.java.net/~vlivanov/7175279/webrev.01-00/

Looks good to me.

Roland.


From stefan.karlsson at oracle.com  Thu Jan 23 08:59:09 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Thu, 23 Jan 2020 09:59:09 +0100
Subject: RFR: 8237637: Remove dubious type conversions from oop
In-Reply-To: <B7BE471C-6E59-4EDC-BC1B-B4705E496643@oracle.com>
References: <0d5e0216-6de5-2bd9-d258-0e5ef7b3a3f3@oracle.com>
 <B7BE471C-6E59-4EDC-BC1B-B4705E496643@oracle.com>
Message-ID: <f3baab49-393b-6837-8473-f6f06f828e36@oracle.com>

On 2020-01-23 08:06, Kim Barrett wrote:
>> On Jan 22, 2020, at 8:33 AM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
>>
>> Hi all,
>>
>> Please review this patch to remove some dubious type conversions from oop.
>>
>> https://bugs.openjdk.java.net/browse/JDK-8237637
>>
>> When running with fastdebug and CHECK_UNHANDLED_OOPS oop is not a simple oopDesc* but a class. This class has a number of type conversions with the comment:
>>
>>   // Explict user conversions
>>
>> However, these are not only *explicit* conversions. They can be invoked implicitly as well.
>>
>> I propose that we get rid of most of these and only leave these two:
>> operator void* () const
>> operator oopDesc* () const
>>
>> so that we get better type safety in the code.
>>
>> I've split this up into multiple webrevs to make it easier to review:
>>
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/00.promotedObject/
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/01.oopStar/
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/02.jobject/
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/03.address/
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/03.address/
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/05.voidStarVolatiles/
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/06.heapWord/
>>
>> All changes combined:
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/all/
>>
>> Testing: builds pass, tier1-3 running
>>
>> Thanks,
>> StefanK
> 
> I like this.  A few minor cleanups below.  I have some additional
> suggestions, but I think they are more suited to followup RFEs that
> I'm planning to write up.
> 
> ------------------------------------------------------------------------------
> src/hotspot/share/compiler/oopMap.cpp
>   795     *derived_loc = (oop)(cast_from_oop<address>(base) + offset);
> 
> Seems like that should also be using cast_to_oop.
> 

This patch is about fixing conversions *from* oop to other types. There 
are numerous places in the VM where we convert *to* oop with casts 
instead of cast_to_oop. I don't want to do those kind of changes in this 
RFE.

> ------------------------------------------------------------------------------
> src/hotspot/share/gc/g1/g1FullGCOopClosures.cpp
>    80         HeapRegion* to   = _g1h->heap_region_containing(cast_from_oop<HeapWord*>(obj));
> 
> src/hotspot/share/gc/g1/g1ParScanThreadState.hpp
>   132     HeapRegion* const hr_obj = _g1h->heap_region_containing(cast_from_oop<HeapWord*>(o));
> 
> src/hotspot/share/gc/g1/heapRegion.cpp
>   569           HeapRegion* to = _g1h->heap_region_containing(cast_from_oop<HeapWord*>(obj));
> 
> No casts were needed in these places, so remove them rather than
> converting them to cast_from_oop<HeapWord*>.

Changed.

> 
> Similarly, there are some calls to operations on mark bit maps that
> don't need casts because there are overloads on oop, so should have
> the casts removed rather than being converted to cast_from_oop<HeapWord*>.

Changed.

> 
> Similarly for is_in_cset.  And there might be other functions like
> this that I didn't notice.

I can't find what you are referring to.

> 
> ------------------------------------------------------------------------------
> src/hotspot/share/gc/parallel/psCardTable.cpp
>    53         _unmarked_addr = (HeapWord*)(void*)p;
> 
> The addition of the cast to void* seems unnecessary.
>

Reverted.

> ------------------------------------------------------------------------------
> src/hotspot/share/oops/oopsHierarchy.hpp
> 
> When removing this line (as was mentioned in the thread)
>   113   operator void* () const             { return (void *)obj(); }
> 

Done.

> I suggest also changing cast_from_oop to use oopDesc* rather than void*:
>   175   return (T)(CHECK_UNHANDLED_OOPS_ONLY((oopDesc*))o);

Done.

> 
> Note that there are a lot of implicit conversions (mostly to void* I
> think) that prevent us from removing (or making explicit with C++11)
> the oopDesc* conversion operator.

It's not obvious to me that we want to remove the implicit conversions 
to oopDesc*.

I'm going to run this (and David's suggestions) through tier1, and then 
I'll push the patch.

Thanks,
StefanK


> 
> ------------------------------------------------------------------------------
> 

From stefan.karlsson at oracle.com  Thu Jan 23 09:27:53 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Thu, 23 Jan 2020 10:27:53 +0100
Subject: RFR: 8237637: Remove dubious type conversions from oop
In-Reply-To: <63365943-2b6a-2388-5934-62d8f179b291@oracle.com>
References: <0d5e0216-6de5-2bd9-d258-0e5ef7b3a3f3@oracle.com>
 <63365943-2b6a-2388-5934-62d8f179b291@oracle.com>
Message-ID: <03eb9621-75c2-f583-bd3e-c3064b3daddc@oracle.com>

Hi David,

Thanks for looking at this. I've incorporated your suggestions.

StefanK

On 2020-01-23 00:56, David Holmes wrote:
> Hi Stefan,
> 
> I don't have any concerns with this, but if cast_to_oop is for 
> "numerical conversions" then can't we get rid of the secondary casts to 
> jlong and u8 where used i.e can
> 
> 
> --- 
> old/src/hotspot/share/jfr/leakprofiler/checkpoint/objectSampleWriter.cpp 
> 2020-01-22 14:24:21.194074776 +0100
> +++ 
> new/src/hotspot/share/jfr/leakprofiler/checkpoint/objectSampleWriter.cpp 
> 2020-01-22 14:24:20.714066939 +0100
> @@ -206,7 +206,7 @@
>  ?? oop object = oosi->_data._object;
>  ?? assert(object != NULL, "invariant");
>  ?? writer->write(oosi->_id);
> -? writer->write((u8)(const HeapWord*)object);
> +? writer->write((u8)cast_from_oop<HeapWord*>(object));
> 
> not use:
> 
> writer->write(cast_from_oop<u8>(object));
> 
> and:
> 
> --- old/src/hotspot/share/jvmci/jvmciCompilerToVM.cpp??? 2020-01-22 
> 14:24:22.674098943 +0100
> +++ new/src/hotspot/share/jvmci/jvmciCompilerToVM.cpp??? 2020-01-22 
> 14:24:22.142090256 +0100
> @@ -431,7 +431,7 @@
>  ?????? } else if (JVMCIENV->isa_HotSpotObjectConstantImpl(base_object)) {
>  ???????? Handle base_oop = JVMCIENV->asConstant(base_object, 
> JVMCI_CHECK_NULL);
>  ???????? if (base_oop->is_a(SystemDictionary::Class_klass())) {
> -????????? base_address = (jlong) (address) base_oop();
> +????????? base_address = (jlong) cast_from_oop<address>(base_oop());
> 
> not use
> 
>  ????????? base_address = cast_from_oop<jlong>(base_oop());
> 
> ?
> 
> Thanks,
> David
> 
> On 22/01/2020 11:33 pm, Stefan Karlsson wrote:
>> Hi all,
>>
>> Please review this patch to remove some dubious type conversions from 
>> oop.
>>
>> https://bugs.openjdk.java.net/browse/JDK-8237637
>>
>> When running with fastdebug and CHECK_UNHANDLED_OOPS oop is not a 
>> simple oopDesc* but a class. This class has a number of type 
>> conversions with the comment:
>>
>> ?? // Explict user conversions
>>
>> However, these are not only *explicit* conversions. They can be 
>> invoked implicitly as well.
>>
>> I propose that we get rid of most of these and only leave these two:
>> ??operator void* () const
>> ??operator oopDesc* () const
>>
>> so that we get better type safety in the code.
>>
>> I've split this up into multiple webrevs to make it easier to review:
>>
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/00.promotedObject/
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/01.oopStar/
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/02.jobject/
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/03.address/
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/03.address/
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/05.voidStarVolatiles/ 
>>
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/06.heapWord/
>>
>> All changes combined:
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/all/
>>
>> Testing: builds pass, tier1-3 running
>>
>> Thanks,
>> StefanK
>>

From martin.doerr at sap.com  Thu Jan 23 11:57:17 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Thu, 23 Jan 2020 11:57:17 +0000
Subject: RFR: 8237637: Remove dubious type conversions from oop
In-Reply-To: <03eb9621-75c2-f583-bd3e-c3064b3daddc@oracle.com>
References: <0d5e0216-6de5-2bd9-d258-0e5ef7b3a3f3@oracle.com>
 <63365943-2b6a-2388-5934-62d8f179b291@oracle.com>
 <03eb9621-75c2-f583-bd3e-c3064b3daddc@oracle.com>
Message-ID: <AM0PR0202MB32976815132CF292E18706F29A0F0@AM0PR0202MB3297.eurprd02.prod.outlook.com>

Hi Stefan,

thanks a lot for cleaning this up.
Could you add the following s390 part, please?
Otherwise it would break the build. PPC64 seems to be fine.

Best regards,
Martin


diff -r 55fa1be40f39 src/hotspot/cpu/s390/assembler_s390.hpp
--- a/src/hotspot/cpu/s390/assembler_s390.hpp   Thu Jan 23 12:44:02 2020 +0100
+++ b/src/hotspot/cpu/s390/assembler_s390.hpp   Thu Jan 23 12:54:47 2020 +0100
@@ -351,14 +351,6 @@
     : _address((address) addr),
       _rspec(rspec_from_rtype(rtype, (address) addr)) {}

-  AddressLiteral(oop addr, relocInfo::relocType rtype = relocInfo::none)
-    : _address((address) addr),
-      _rspec(rspec_from_rtype(rtype, (address) addr)) {}
-
-  AddressLiteral(oop* addr, relocInfo::relocType rtype = relocInfo::none)
-    : _address((address) addr),
-      _rspec(rspec_from_rtype(rtype, (address) addr)) {}
-
   AddressLiteral(float* addr, relocInfo::relocType rtype = relocInfo::none)
     : _address((address) addr),
       _rspec(rspec_from_rtype(rtype, (address) addr)) {}
@@ -390,7 +382,6 @@

  public:
   ExternalAddress(address target) : AddressLiteral(target, reloc_for_target(          target)) {}
-  ExternalAddress(oop*    target) : AddressLiteral(target, reloc_for_target((address) target)) {}
 };

 // Argument is an abstraction used to represent an outgoing actual



> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
> Stefan Karlsson
> Sent: Donnerstag, 23. Januar 2020 10:28
> To: David Holmes <david.holmes at oracle.com>; hotspot-dev <hotspot-
> dev at openjdk.java.net>
> Subject: Re: RFR: 8237637: Remove dubious type conversions from oop
> 
> Hi David,
> 
> Thanks for looking at this. I've incorporated your suggestions.
> 
> StefanK
> 
> On 2020-01-23 00:56, David Holmes wrote:
> > Hi Stefan,
> >
> > I don't have any concerns with this, but if cast_to_oop is for
> > "numerical conversions" then can't we get rid of the secondary casts to
> > jlong and u8 where used i.e can
> >
> >
> > ---
> >
> old/src/hotspot/share/jfr/leakprofiler/checkpoint/objectSampleWriter.cpp
> > 2020-01-22 14:24:21.194074776 +0100
> > +++
> >
> new/src/hotspot/share/jfr/leakprofiler/checkpoint/objectSampleWriter.cpp
> > 2020-01-22 14:24:20.714066939 +0100
> > @@ -206,7 +206,7 @@
> >  ?? oop object = oosi->_data._object;
> >  ?? assert(object != NULL, "invariant");
> >  ?? writer->write(oosi->_id);
> > -? writer->write((u8)(const HeapWord*)object);
> > +? writer->write((u8)cast_from_oop<HeapWord*>(object));
> >
> > not use:
> >
> > writer->write(cast_from_oop<u8>(object));
> >
> > and:
> >
> > --- old/src/hotspot/share/jvmci/jvmciCompilerToVM.cpp??? 2020-01-22
> > 14:24:22.674098943 +0100
> > +++ new/src/hotspot/share/jvmci/jvmciCompilerToVM.cpp??? 2020-01-22
> > 14:24:22.142090256 +0100
> > @@ -431,7 +431,7 @@
> >  ?????? } else if (JVMCIENV->isa_HotSpotObjectConstantImpl(base_object)) {
> >  ???????? Handle base_oop = JVMCIENV->asConstant(base_object,
> > JVMCI_CHECK_NULL);
> >  ???????? if (base_oop->is_a(SystemDictionary::Class_klass())) {
> > -????????? base_address = (jlong) (address) base_oop();
> > +????????? base_address = (jlong) cast_from_oop<address>(base_oop());
> >
> > not use
> >
> >  ????????? base_address = cast_from_oop<jlong>(base_oop());
> >
> > ?
> >
> > Thanks,
> > David
> >
> > On 22/01/2020 11:33 pm, Stefan Karlsson wrote:
> >> Hi all,
> >>
> >> Please review this patch to remove some dubious type conversions from
> >> oop.
> >>
> >> https://bugs.openjdk.java.net/browse/JDK-8237637
> >>
> >> When running with fastdebug and CHECK_UNHANDLED_OOPS oop is not
> a
> >> simple oopDesc* but a class. This class has a number of type
> >> conversions with the comment:
> >>
> >> ?? // Explict user conversions
> >>
> >> However, these are not only *explicit* conversions. They can be
> >> invoked implicitly as well.
> >>
> >> I propose that we get rid of most of these and only leave these two:
> >> ??operator void* () const
> >> ??operator oopDesc* () const
> >>
> >> so that we get better type safety in the code.
> >>
> >> I've split this up into multiple webrevs to make it easier to review:
> >>
> >>
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/00.promotedObje
> ct/
> >> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/01.oopStar/
> >> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/02.jobject/
> >> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/03.address/
> >> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/03.address/
> >>
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/05.voidStarVolatil
> es/
> >>
> >> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/06.heapWord/
> >>
> >> All changes combined:
> >> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/all/
> >>
> >> Testing: builds pass, tier1-3 running
> >>
> >> Thanks,
> >> StefanK
> >>

From matthias.baesken at sap.com  Thu Jan 23 13:15:28 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Thu, 23 Jan 2020 13:15:28 +0000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <d3e3641f-d6ae-979d-d53f-93136da6d62a@oracle.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
 <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>
 <AM6PR02MB4200B721BD0C7728B21E2C4193310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <AM6PR02MB4200BCFF64D8C46340B17A6F930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <d3e3641f-d6ae-979d-d53f-93136da6d62a@oracle.com>
Message-ID: <AM6PR02MB42000810C8986AA738DB5EA3930F0@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hi Erik,  new webrev :

http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.6/

I moved the settings back into the  m4 files .

Best regards, Matthias

> 
> Hello Matthias,
> 
> You can keep the setting up of all the flags in flags-cflags.m4 and
> flags-ldflags.m4 based on the value of ENABLE_LINKTIME_GC. You can also
> default the value of this new parameter to true for s390x to keep the
> current behavior for that platform. As it is in this patch, the JVM
> flags for s390x are setup in configure while the JDK flags are in make,
> which gets confusing I think.
> 
> /Erik
> 
> 
> On 2020-01-22 05:33, Baesken, Matthias wrote:
> > Hi Magnus / David,  here is a new webrev :
> >
> >
> > http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.4/
> >
> >
> > it supports now a  configure switch  --enable-linktime-gc=yes  that needs to
> be set  to enable the link time section gc  .
> >
> > Exception is linuxs390x  where we already have the  feature enabled  (and
> keep it enabled always for LIB_JVM).
> >
> > Best regards, Matthias
> >
> >
> >
> > From: Baesken, Matthias
> > Sent: Freitag, 17. Januar 2020 12:44
> > To: Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>; David Holmes
> <david.holmes at oracle.com>; 'build-dev at openjdk.java.net' <build-
> dev at openjdk.java.net>; 'hotspot-dev at openjdk.java.net' <hotspot-
> dev at openjdk.java.net>
> > Subject: RE: RFR: 8236714: enable link-time section-gc for linux to remove
> unused code
> >
> >
> >
> >    *   Matthias: Have a look at some recently added option to get an
> indication of the best practice in adding new options. There are some ways to
> easily make this incorrect
> >
> > Hi Magnus, do you have a good/?best practice?  example  (not that I catch a
> bad one ?? )  ?
> >
> > Best regards, Matthias
> >

From vladimir.x.ivanov at oracle.com  Thu Jan 23 13:23:20 2020
From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov)
Date: Thu, 23 Jan 2020 16:23:20 +0300
Subject: [15] RFR (L): 7175279: Don't use x87 FPU on x86-64
In-Reply-To: <87h80my4uy.fsf@redhat.com>
References: <0b0897e0-1dbc-306d-b2cb-31de13fb8b34@oracle.com>
 <7063EB29-D415-4A48-BA4F-B16C5A1F52F8@oracle.com>
 <16d75ed8-896b-25fe-0a0a-babc9087e631@oracle.com> <87h80my4uy.fsf@redhat.com>
Message-ID: <c8556d91-af15-64b6-4f82-127e67003a1a@oracle.com>

Thanks, Roland.

Best regards,
Vladimir Ivanov

On 23.01.2020 11:05, Roland Westrelin wrote:
> 
>>     http://cr.openjdk.java.net/~vlivanov/7175279/webrev.00/
>>     +
>>     http://cr.openjdk.java.net/~vlivanov/7175279/webrev.01-00/
> 
> Looks good to me.
> 
> Roland.
> 

From stefan.karlsson at oracle.com  Thu Jan 23 13:33:21 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Thu, 23 Jan 2020 14:33:21 +0100
Subject: RFR: 8237637: Remove dubious type conversions from oop
In-Reply-To: <AM0PR0202MB32976815132CF292E18706F29A0F0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
References: <0d5e0216-6de5-2bd9-d258-0e5ef7b3a3f3@oracle.com>
 <63365943-2b6a-2388-5934-62d8f179b291@oracle.com>
 <03eb9621-75c2-f583-bd3e-c3064b3daddc@oracle.com>
 <AM0PR0202MB32976815132CF292E18706F29A0F0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
Message-ID: <d585e508-a499-7153-1f95-f0819d81030e@oracle.com>

Hi Martin,

On 2020-01-23 12:57, Doerr, Martin wrote:
> Hi Stefan,
> 
> thanks a lot for cleaning this up.
> Could you add the following s390 part, please?
> Otherwise it would break the build. PPC64 seems to be fine.

Thanks for testing the patch. I compiled against aarch64 and Shenandoah 
and found more places that needed to be fixed. I've rolled up all 
changes into a new webrev:

https://cr.openjdk.java.net/~stefank/8237637/webrev.02.delta/
https://cr.openjdk.java.net/~stefank/8237637/webrev.02/

Thanks,
StefanK

> 
> Best regards,
> Martin
> 
> 
> diff -r 55fa1be40f39 src/hotspot/cpu/s390/assembler_s390.hpp
> --- a/src/hotspot/cpu/s390/assembler_s390.hpp   Thu Jan 23 12:44:02 2020 +0100
> +++ b/src/hotspot/cpu/s390/assembler_s390.hpp   Thu Jan 23 12:54:47 2020 +0100
> @@ -351,14 +351,6 @@
>       : _address((address) addr),
>         _rspec(rspec_from_rtype(rtype, (address) addr)) {}
> 
> -  AddressLiteral(oop addr, relocInfo::relocType rtype = relocInfo::none)
> -    : _address((address) addr),
> -      _rspec(rspec_from_rtype(rtype, (address) addr)) {}
> -
> -  AddressLiteral(oop* addr, relocInfo::relocType rtype = relocInfo::none)
> -    : _address((address) addr),
> -      _rspec(rspec_from_rtype(rtype, (address) addr)) {}
> -
>     AddressLiteral(float* addr, relocInfo::relocType rtype = relocInfo::none)
>       : _address((address) addr),
>         _rspec(rspec_from_rtype(rtype, (address) addr)) {}
> @@ -390,7 +382,6 @@
> 
>    public:
>     ExternalAddress(address target) : AddressLiteral(target, reloc_for_target(          target)) {}
> -  ExternalAddress(oop*    target) : AddressLiteral(target, reloc_for_target((address) target)) {}
>   };
> 
>   // Argument is an abstraction used to represent an outgoing actual
> 
> 
> 
>> -----Original Message-----
>> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
>> Stefan Karlsson
>> Sent: Donnerstag, 23. Januar 2020 10:28
>> To: David Holmes <david.holmes at oracle.com>; hotspot-dev <hotspot-
>> dev at openjdk.java.net>
>> Subject: Re: RFR: 8237637: Remove dubious type conversions from oop
>>
>> Hi David,
>>
>> Thanks for looking at this. I've incorporated your suggestions.
>>
>> StefanK
>>
>> On 2020-01-23 00:56, David Holmes wrote:
>>> Hi Stefan,
>>>
>>> I don't have any concerns with this, but if cast_to_oop is for
>>> "numerical conversions" then can't we get rid of the secondary casts to
>>> jlong and u8 where used i.e can
>>>
>>>
>>> ---
>>>
>> old/src/hotspot/share/jfr/leakprofiler/checkpoint/objectSampleWriter.cpp
>>> 2020-01-22 14:24:21.194074776 +0100
>>> +++
>>>
>> new/src/hotspot/share/jfr/leakprofiler/checkpoint/objectSampleWriter.cpp
>>> 2020-01-22 14:24:20.714066939 +0100
>>> @@ -206,7 +206,7 @@
>>>   ?? oop object = oosi->_data._object;
>>>   ?? assert(object != NULL, "invariant");
>>>   ?? writer->write(oosi->_id);
>>> -? writer->write((u8)(const HeapWord*)object);
>>> +? writer->write((u8)cast_from_oop<HeapWord*>(object));
>>>
>>> not use:
>>>
>>> writer->write(cast_from_oop<u8>(object));
>>>
>>> and:
>>>
>>> --- old/src/hotspot/share/jvmci/jvmciCompilerToVM.cpp??? 2020-01-22
>>> 14:24:22.674098943 +0100
>>> +++ new/src/hotspot/share/jvmci/jvmciCompilerToVM.cpp??? 2020-01-22
>>> 14:24:22.142090256 +0100
>>> @@ -431,7 +431,7 @@
>>>   ?????? } else if (JVMCIENV->isa_HotSpotObjectConstantImpl(base_object)) {
>>>   ???????? Handle base_oop = JVMCIENV->asConstant(base_object,
>>> JVMCI_CHECK_NULL);
>>>   ???????? if (base_oop->is_a(SystemDictionary::Class_klass())) {
>>> -????????? base_address = (jlong) (address) base_oop();
>>> +????????? base_address = (jlong) cast_from_oop<address>(base_oop());
>>>
>>> not use
>>>
>>>   ????????? base_address = cast_from_oop<jlong>(base_oop());
>>>
>>> ?
>>>
>>> Thanks,
>>> David
>>>
>>> On 22/01/2020 11:33 pm, Stefan Karlsson wrote:
>>>> Hi all,
>>>>
>>>> Please review this patch to remove some dubious type conversions from
>>>> oop.
>>>>
>>>> https://bugs.openjdk.java.net/browse/JDK-8237637
>>>>
>>>> When running with fastdebug and CHECK_UNHANDLED_OOPS oop is not
>> a
>>>> simple oopDesc* but a class. This class has a number of type
>>>> conversions with the comment:
>>>>
>>>>  ?? // Explict user conversions
>>>>
>>>> However, these are not only *explicit* conversions. They can be
>>>> invoked implicitly as well.
>>>>
>>>> I propose that we get rid of most of these and only leave these two:
>>>>  ??operator void* () const
>>>>  ??operator oopDesc* () const
>>>>
>>>> so that we get better type safety in the code.
>>>>
>>>> I've split this up into multiple webrevs to make it easier to review:
>>>>
>>>>
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/00.promotedObje
>> ct/
>>>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/01.oopStar/
>>>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/02.jobject/
>>>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/03.address/
>>>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/03.address/
>>>>
>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/05.voidStarVolatil
>> es/
>>>>
>>>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/06.heapWord/
>>>>
>>>> All changes combined:
>>>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/all/
>>>>
>>>> Testing: builds pass, tier1-3 running
>>>>
>>>> Thanks,
>>>> StefanK
>>>>

From martin.doerr at sap.com  Thu Jan 23 14:40:23 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Thu, 23 Jan 2020 14:40:23 +0000
Subject: RFR: 8237637: Remove dubious type conversions from oop
In-Reply-To: <d585e508-a499-7153-1f95-f0819d81030e@oracle.com>
References: <0d5e0216-6de5-2bd9-d258-0e5ef7b3a3f3@oracle.com>
 <63365943-2b6a-2388-5934-62d8f179b291@oracle.com>
 <03eb9621-75c2-f583-bd3e-c3064b3daddc@oracle.com>
 <AM0PR0202MB32976815132CF292E18706F29A0F0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
 <d585e508-a499-7153-1f95-f0819d81030e@oracle.com>
Message-ID: <AM0PR0202MB3297FDFCD29E7BE87B3859B79A0F0@AM0PR0202MB3297.eurprd02.prod.outlook.com>

Hi Stefan,

thanks for adding my patch and for checking other platforms.

Best regards,
Martin


> -----Original Message-----
> From: Stefan Karlsson <stefan.karlsson at oracle.com>
> Sent: Donnerstag, 23. Januar 2020 14:33
> To: Doerr, Martin <martin.doerr at sap.com>; David Holmes
> <david.holmes at oracle.com>; hotspot-dev <hotspot-
> dev at openjdk.java.net>
> Subject: Re: RFR: 8237637: Remove dubious type conversions from oop
> 
> Hi Martin,
> 
> On 2020-01-23 12:57, Doerr, Martin wrote:
> > Hi Stefan,
> >
> > thanks a lot for cleaning this up.
> > Could you add the following s390 part, please?
> > Otherwise it would break the build. PPC64 seems to be fine.
> 
> Thanks for testing the patch. I compiled against aarch64 and Shenandoah
> and found more places that needed to be fixed. I've rolled up all
> changes into a new webrev:
> 
> https://cr.openjdk.java.net/~stefank/8237637/webrev.02.delta/
> https://cr.openjdk.java.net/~stefank/8237637/webrev.02/
> 
> Thanks,
> StefanK
> 
> >
> > Best regards,
> > Martin
> >
> >
> > diff -r 55fa1be40f39 src/hotspot/cpu/s390/assembler_s390.hpp
> > --- a/src/hotspot/cpu/s390/assembler_s390.hpp   Thu Jan 23 12:44:02 2020
> +0100
> > +++ b/src/hotspot/cpu/s390/assembler_s390.hpp   Thu Jan 23 12:54:47
> 2020 +0100
> > @@ -351,14 +351,6 @@
> >       : _address((address) addr),
> >         _rspec(rspec_from_rtype(rtype, (address) addr)) {}
> >
> > -  AddressLiteral(oop addr, relocInfo::relocType rtype = relocInfo::none)
> > -    : _address((address) addr),
> > -      _rspec(rspec_from_rtype(rtype, (address) addr)) {}
> > -
> > -  AddressLiteral(oop* addr, relocInfo::relocType rtype = relocInfo::none)
> > -    : _address((address) addr),
> > -      _rspec(rspec_from_rtype(rtype, (address) addr)) {}
> > -
> >     AddressLiteral(float* addr, relocInfo::relocType rtype = relocInfo::none)
> >       : _address((address) addr),
> >         _rspec(rspec_from_rtype(rtype, (address) addr)) {}
> > @@ -390,7 +382,6 @@
> >
> >    public:
> >     ExternalAddress(address target) : AddressLiteral(target,
> reloc_for_target(          target)) {}
> > -  ExternalAddress(oop*    target) : AddressLiteral(target,
> reloc_for_target((address) target)) {}
> >   };
> >
> >   // Argument is an abstraction used to represent an outgoing actual
> >
> >
> >
> >> -----Original Message-----
> >> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf
> Of
> >> Stefan Karlsson
> >> Sent: Donnerstag, 23. Januar 2020 10:28
> >> To: David Holmes <david.holmes at oracle.com>; hotspot-dev <hotspot-
> >> dev at openjdk.java.net>
> >> Subject: Re: RFR: 8237637: Remove dubious type conversions from oop
> >>
> >> Hi David,
> >>
> >> Thanks for looking at this. I've incorporated your suggestions.
> >>
> >> StefanK
> >>
> >> On 2020-01-23 00:56, David Holmes wrote:
> >>> Hi Stefan,
> >>>
> >>> I don't have any concerns with this, but if cast_to_oop is for
> >>> "numerical conversions" then can't we get rid of the secondary casts to
> >>> jlong and u8 where used i.e can
> >>>
> >>>
> >>> ---
> >>>
> >>
> old/src/hotspot/share/jfr/leakprofiler/checkpoint/objectSampleWriter.cpp
> >>> 2020-01-22 14:24:21.194074776 +0100
> >>> +++
> >>>
> >>
> new/src/hotspot/share/jfr/leakprofiler/checkpoint/objectSampleWriter.cpp
> >>> 2020-01-22 14:24:20.714066939 +0100
> >>> @@ -206,7 +206,7 @@
> >>>   ?? oop object = oosi->_data._object;
> >>>   ?? assert(object != NULL, "invariant");
> >>>   ?? writer->write(oosi->_id);
> >>> -? writer->write((u8)(const HeapWord*)object);
> >>> +? writer->write((u8)cast_from_oop<HeapWord*>(object));
> >>>
> >>> not use:
> >>>
> >>> writer->write(cast_from_oop<u8>(object));
> >>>
> >>> and:
> >>>
> >>> --- old/src/hotspot/share/jvmci/jvmciCompilerToVM.cpp??? 2020-01-22
> >>> 14:24:22.674098943 +0100
> >>> +++ new/src/hotspot/share/jvmci/jvmciCompilerToVM.cpp??? 2020-01-22
> >>> 14:24:22.142090256 +0100
> >>> @@ -431,7 +431,7 @@
> >>>   ?????? } else if (JVMCIENV-
> >isa_HotSpotObjectConstantImpl(base_object)) {
> >>>   ???????? Handle base_oop = JVMCIENV->asConstant(base_object,
> >>> JVMCI_CHECK_NULL);
> >>>   ???????? if (base_oop->is_a(SystemDictionary::Class_klass())) {
> >>> -????????? base_address = (jlong) (address) base_oop();
> >>> +????????? base_address = (jlong) cast_from_oop<address>(base_oop());
> >>>
> >>> not use
> >>>
> >>>   ????????? base_address = cast_from_oop<jlong>(base_oop());
> >>>
> >>> ?
> >>>
> >>> Thanks,
> >>> David
> >>>
> >>> On 22/01/2020 11:33 pm, Stefan Karlsson wrote:
> >>>> Hi all,
> >>>>
> >>>> Please review this patch to remove some dubious type conversions
> from
> >>>> oop.
> >>>>
> >>>> https://bugs.openjdk.java.net/browse/JDK-8237637
> >>>>
> >>>> When running with fastdebug and CHECK_UNHANDLED_OOPS oop is
> not
> >> a
> >>>> simple oopDesc* but a class. This class has a number of type
> >>>> conversions with the comment:
> >>>>
> >>>>  ?? // Explict user conversions
> >>>>
> >>>> However, these are not only *explicit* conversions. They can be
> >>>> invoked implicitly as well.
> >>>>
> >>>> I propose that we get rid of most of these and only leave these two:
> >>>>  ??operator void* () const
> >>>>  ??operator oopDesc* () const
> >>>>
> >>>> so that we get better type safety in the code.
> >>>>
> >>>> I've split this up into multiple webrevs to make it easier to review:
> >>>>
> >>>>
> >>
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/00.promotedObje
> >> ct/
> >>>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/01.oopStar/
> >>>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/02.jobject/
> >>>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/03.address/
> >>>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/03.address/
> >>>>
> >>
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/05.voidStarVolatil
> >> es/
> >>>>
> >>>>
> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/06.heapWord/
> >>>>
> >>>> All changes combined:
> >>>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/all/
> >>>>
> >>>> Testing: builds pass, tier1-3 running
> >>>>
> >>>> Thanks,
> >>>> StefanK
> >>>>

From frederic.parain at oracle.com  Thu Jan 23 15:03:11 2020
From: frederic.parain at oracle.com (Frederic Parain)
Date: Thu, 23 Jan 2020 10:03:11 -0500
Subject: RFR[L]: 8237767 Field layout computation overhaul 
Message-ID: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>

Greetings,

Please review this change proposing a new code to compute field layouts.

CR: https://bugs.openjdk.java.net/browse/JDK-8237767
webrev: http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/

The CR includes a detailed description of the motivation and the implementation.

The current version keeps the old code accessible (with a VM flag) in case the
small changes in computed layouts cause troubles to some applications or tools.
The goal is to get rid of this old code, preferably as soon as possible.

Testing tier1-8.

Thank you,

Fred


From aph at redhat.com  Thu Jan 23 16:47:52 2020
From: aph at redhat.com (Andrew Haley)
Date: Thu, 23 Jan 2020 16:47:52 +0000
Subject: 8230392: Define AArch64 as MULTI_COPY_ATOMIC
In-Reply-To: <053adeb2-5f00-5cf6-c96e-97d46d37f5a3@oracle.com>
References: <a8d3f689-dde7-e5ac-c06e-eb3dc1355553@redhat.com>
 <053adeb2-5f00-5cf6-c96e-97d46d37f5a3@oracle.com>
Message-ID: <7c242604-aac9-243f-5a89-70853351aa12@redhat.com>

Hi,

On 1/21/20 10:58 PM, David Holmes wrote:
> On 22/01/2020 3:12 am, Andrew Haley wrote:
>> http://cr.openjdk.java.net/~aph/8230392/
>>
>> I tested this in a bunch of ways, including bootstrapping and jtreg, but
>> I'm not sure anything really stress tests this. I expect it'll be fine.
>>
>> OK to commit?
> You obviously missed my response on the other email thread regarding the 
> taskQueue code.
> 

http://cr.openjdk.java.net/~aph/8230392-2/

Better?

Thanks,

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From erik.joelsson at oracle.com  Thu Jan 23 17:06:34 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Thu, 23 Jan 2020 09:06:34 -0800
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <AM6PR02MB42000810C8986AA738DB5EA3930F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
 <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>
 <AM6PR02MB4200B721BD0C7728B21E2C4193310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <AM6PR02MB4200BCFF64D8C46340B17A6F930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <d3e3641f-d6ae-979d-d53f-93136da6d62a@oracle.com>
 <AM6PR02MB42000810C8986AA738DB5EA3930F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <e770b014-2562-ed73-b258-d017d6f603a4@oracle.com>

Hello,

That's better, but there are still some issues.

flags-cflags.m4

Code is repeated in both if and else block.

jdk-options.m4

The default is now true for all platforms. I would suggest moving the 
s390x conditional down into an elif after the elif for "no".

LibCommon.gmk

Please revert whole file.

/Erik

On 2020-01-23 05:15, Baesken, Matthias wrote:
> Hi Erik,  new webrev :
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.6/
>
> I moved the settings back into the  m4 files .
>
> Best regards, Matthias
>
>> Hello Matthias,
>>
>> You can keep the setting up of all the flags in flags-cflags.m4 and
>> flags-ldflags.m4 based on the value of ENABLE_LINKTIME_GC. You can also
>> default the value of this new parameter to true for s390x to keep the
>> current behavior for that platform. As it is in this patch, the JVM
>> flags for s390x are setup in configure while the JDK flags are in make,
>> which gets confusing I think.
>>
>> /Erik
>>
>>
>> On 2020-01-22 05:33, Baesken, Matthias wrote:
>>> Hi Magnus / David,  here is a new webrev :
>>>
>>>
>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.4/
>>>
>>>
>>> it supports now a  configure switch  --enable-linktime-gc=yes  that needs to
>> be set  to enable the link time section gc  .
>>> Exception is linuxs390x  where we already have the  feature enabled  (and
>> keep it enabled always for LIB_JVM).
>>> Best regards, Matthias
>>>
>>>
>>>
>>> From: Baesken, Matthias
>>> Sent: Freitag, 17. Januar 2020 12:44
>>> To: Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>; David Holmes
>> <david.holmes at oracle.com>; 'build-dev at openjdk.java.net' <build-
>> dev at openjdk.java.net>; 'hotspot-dev at openjdk.java.net' <hotspot-
>> dev at openjdk.java.net>
>>> Subject: RE: RFR: 8236714: enable link-time section-gc for linux to remove
>> unused code
>>>
>>>
>>>     *   Matthias: Have a look at some recently added option to get an
>> indication of the best practice in adding new options. There are some ways to
>> easily make this incorrect
>>> Hi Magnus, do you have a good/?best practice?  example  (not that I catch a
>> bad one ?? )  ?
>>> Best regards, Matthias
>>>

From erik.joelsson at oracle.com  Thu Jan 23 17:48:34 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Thu, 23 Jan 2020 09:48:34 -0800
Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk
 images
In-Reply-To: <AM6PR02MB420064F6F67387E6596F0E67930F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com>
 <AM6PR02MB42007DB6FF70591EC151174A930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <df71f609-64a6-4997-0ff8-562a81be6134@oracle.com>
 <AM6PR02MB420064F6F67387E6596F0E67930F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <e9372084-eabc-8f57-981f-b95f9795582e@oracle.com>


On 2020-01-23 00:03, Baesken, Matthias wrote:
> Hi Erik,  yes true sorry for answering  your comments a bit late .
>
>> If a user runs jlink and includes all the jmods we ship with the JDK, the result
>> should be essentially equivalent to the original JDK image. The way the
>> stripped pdb files are included in the bundles sort of at the last
>> second of the build here breaks this property.
> I think we should address this in a separate bug/CR .
Maybe. I realize that my proposal below is quite a big task. But on the 
other hand, I don't think breaking the relationship between the jmods 
and the distribution bundles is on the table really.
> Looking for example  into a Linux  build, I see  a lot of debuginfo files  in the jdk image (more or less for every shared lib)  .
> But when looking into the jmods  of that jdk image ,  no debuginfo files are in there ( I checked the java.base jmod).
> So  putting  the  files with debug information into the jmods  seems to be something that was not done so far  cross platform  (or is there some build switch  for it that I did not check?) .
> Maybe to keep the jmods as small as possible .

No, we do not put the debuginfo files in the jmods nor the bundles 
because those are not intended to be shipped to customers. We are 
currently overlaying them into images/jdk in the build output to make 
them available for local debugging. (This is convoluted and I would very 
much like to get away from this practice at some point so that there is 
a 1-1 mapping between images/jdk and bundles/jdk*-bin.tar.gz.) The 
stripped pdb files you are proposing are on the contrary intended for 
shipping to customers (as I understand your proposal) so comparing them 
with the debuginfo files is not relevant.

Now if MS had been kind enough to define a separate file type for the 
stripped pdbs, so that they could live alongside the full pdbs, we 
wouldn't have this issue. The heart of the problem is that only one set 
of files (either stripped or full) can be present and usable in 
images/jdk at a time. We have 2 main uses for images/jdk.

1. Developer running and debugging locally

2. Serve as the source for generating the distribution bundles

We currently have one image serving both of these purposes, which is 
already creating complicated and convoluted build steps. To properly 
solve this I would argue for splitting these two apart into two 
different images for the two different purposes. The build procedure 
would then be, first build the images for distribution, only containing 
what should go into each bundle. Then create the developer jdk image by 
copying files from the distribution images. On Windows, the first image 
would contain the stripped pdbs and when building the second, those 
would get overwritten with the full pdbs.

Now that I figured out a working model that would solve a bunch of other 
problems as well, I would love to implement it, but I doubt I will have 
time in the near future.

/Erik

>
>> To properly implement this, care will need to be taken to juggle the two
>> sets of pdb files around, making sure each build and test use case has
>> the correct one in place where and when it's needed. Quite possibly, we
>> cannot cover all use cases with one build configuration. Developers
>> needing the full debug symbols when debugging locally would likely need
>> to disable the stripped symbols so they get the full symbols everywhere.
>> Possibly this would need to be the default for debug builds and
>> configurable for release builds.
>  From my limited experience , the developers  do not work with the bundles (that  would contain now after my patch the stripped pds)  but with a "normal" jdk image that  is created my make all.
>
> Best regards, Matthias
>
>
>
>> This still does not address anything in my objection.
>>
>> /Erik
>>
>> On 2020-01-22 07:46, Baesken, Matthias wrote:
>>> Hello,  here is an updated version  :
>>>
>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.3/
>>>
>>> this one supports a configure switch  "--enable-stripped-pdbs"    to enable
>> the feature .
>>> Best regards, Matthias
>>>
>>>
>>>> -----Original Message-----
>>>> From: Baesken, Matthias
>>>> Sent: Dienstag, 21. Januar 2020 11:03
>>>> To: 'David Holmes' <david.holmes at oracle.com>; 'build-
>>>> dev at openjdk.java.net' <build-dev at openjdk.java.net>; 'hotspot-
>>>> dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
>>>> Subject: RE: RFR: 8237192: Generate stripped/public pdbs on Windows for
>>>> jdk images
>>>>
>>>>
>>>> Hi David ,  yes I think it makes sense to have a configure option for this .
>>>> Not everyone would like to have a larger JDK (even it is only a bit larger).
>>>>

From kim.barrett at oracle.com  Thu Jan 23 19:51:52 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 23 Jan 2020 14:51:52 -0500
Subject: RFR: 8237637: Remove dubious type conversions from oop
In-Reply-To: <f3baab49-393b-6837-8473-f6f06f828e36@oracle.com>
References: <0d5e0216-6de5-2bd9-d258-0e5ef7b3a3f3@oracle.com>
 <B7BE471C-6E59-4EDC-BC1B-B4705E496643@oracle.com>
 <f3baab49-393b-6837-8473-f6f06f828e36@oracle.com>
Message-ID: <06CDB92D-5D76-4566-B96E-DFEAFC475F81@oracle.com>

> On Jan 23, 2020, at 3:59 AM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
> 
> On 2020-01-23 08:06, Kim Barrett wrote:
>>> On Jan 22, 2020, at 8:33 AM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
>>> [?]
>>> All changes combined:
>>> https://cr.openjdk.java.net/~stefank/8237637/webrev.01/all/
>>> 
>>> Testing: builds pass, tier1-3 running
>>> 
>>> Thanks,
>>> StefanK
>> src/hotspot/share/compiler/oopMap.cpp
>>  795     *derived_loc = (oop)(cast_from_oop<address>(base) + offset);
>> Seems like that should also be using cast_to_oop.
> 
> This patch is about fixing conversions *from* oop to other types. There are numerous places in the VM where we convert *to* oop with casts instead of cast_to_oop. I don't want to do those kind of changes in this RFE.

OK.

>> Similarly for is_in_cset.  And there might be other functions like
>> this that I didn't notice.
> 
> I can't find what you are referring to.

Neither can I now.  Sorry for the noise.

>> Note that there are a lot of implicit conversions (mostly to void* I
>> think) that prevent us from removing (or making explicit with C++11)
>> the oopDesc* conversion operator.
> 
> It's not obvious to me that we want to remove the implicit conversions to oopDesc*.

I was looking at that possibility on the basis that implicit
conversions are often problematic, so wondering if we really need this
one.  It turns out we do use it quite a bit.  That's not surprising,
since it makes the CHECK_UNHANDLED_OOPS case more like the
!CHECK_UNHANDLED_OOPS case, where oop is just a typedef for oopDesc*.
But it seemed worth the check.  I should have mentioned the behavioral
consistency as another reason to keep it though.

> I'm going to run this (and David's suggestions) through tier1,

I'm somewhat ambivalent about David's suggestions.  One of my not yet
written up suggestions for followup involves putting some constraints
on the types that can be used in cast_to/from_oop, rather than allow
arbitrary types (some of which are probably at least questionable,
including some that are currently used).  But any issues there can be
dealt with as part of that followup.

> and then I'll push the patch.

OK.


From david.holmes at oracle.com  Thu Jan 23 22:41:14 2020
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 24 Jan 2020 08:41:14 +1000
Subject: 8230392: Define AArch64 as MULTI_COPY_ATOMIC
In-Reply-To: <7c242604-aac9-243f-5a89-70853351aa12@redhat.com>
References: <a8d3f689-dde7-e5ac-c06e-eb3dc1355553@redhat.com>
 <053adeb2-5f00-5cf6-c96e-97d46d37f5a3@oracle.com>
 <7c242604-aac9-243f-5a89-70853351aa12@redhat.com>
Message-ID: <53962163-d04e-e742-20c1-441ea47ea234@oracle.com>

Hi Andrew,

On 24/01/2020 2:47 am, Andrew Haley wrote:
> Hi,
> 
> On 1/21/20 10:58 PM, David Holmes wrote:
>> On 22/01/2020 3:12 am, Andrew Haley wrote:
>>> http://cr.openjdk.java.net/~aph/8230392/
>>>
>>> I tested this in a bunch of ways, including bootstrapping and jtreg, but
>>> I'm not sure anything really stress tests this. I expect it'll be fine.
>>>
>>> OK to commit?
>> You obviously missed my response on the other email thread regarding the
>> taskQueue code.
>>
> 
> http://cr.openjdk.java.net/~aph/8230392-2/
> 
> Better?

That seems fine to me.

This does make me wonder whether other lock-free code in the VM needs 
special handling for non-CPU_MULTI_COPY_ATOMIC ??

Thanks,
David

> Thanks,
> 

From david.holmes at oracle.com  Thu Jan 23 22:51:59 2020
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 24 Jan 2020 08:51:59 +1000
Subject: RFR: 8235966: Process obsolete flags less aggressively
In-Reply-To: <311314c6-7e70-49be-41de-23ae3fa7f23e@oracle.com>
References: <db63e5c9-a0c1-1575-e8dc-88429d6216fe@oracle.com>
 <674e4f0d-563f-3bb4-dee0-74aaacb9b270@oracle.com>
 <b32a2e50-62e3-ed94-7773-a6594a76e1aa@oracle.com>
 <c1a2f1e1-e86b-1fda-4632-ce9734e6e75f@oracle.com>
 <b8e70c4d-af76-a682-b4a9-839dd2671958@oracle.com>
 <6a3de8da-746d-7e5d-9a0e-2157a0963ed8@oracle.com>
 <b7f7df86-50f0-523e-6cb2-e6b7d5fdddde@oracle.com>
 <25156ff2-9dd2-53ae-16bc-83d60c0afb1a@oracle.com>
 <d70c939e-db2f-8fb1-2333-19fdd6153527@oracle.com>
 <3d066cd2-87f3-56ef-803b-a1f194ac761e@oracle.com>
 <e33ab2a9-52d1-a0fa-601b-31c99370c217@oracle.com>
 <311314c6-7e70-49be-41de-23ae3fa7f23e@oracle.com>
Message-ID: <5e54498c-5eae-76f6-3661-f053d75d0f98@oracle.com>

Ping! Anyone?

David

On 22/01/2020 12:06 pm, David Holmes wrote:
> Can I get a second review please?
> 
> Thanks,
> David
> 
> On 22/01/2020 9:10 am, David Holmes wrote:
>> Thanks Dan.
>>
>> Will fix the "typo" - though not actually a typo as it was a 
>> deliberate (mis)use of "monthly". I have always quantified "monthly" 
>> this way, but seems it is not correct. :)
>>
>> Cheers,
>> David
>>
>> On 22/01/2020 1:37 am, Daniel D. Daugherty wrote:
>>> On 1/19/20 8:49 PM, David Holmes wrote:
>>>> gtest version:
>>>>
>>>> http://cr.openjdk.java.net/~dholmes/8235966/webrev.v4/
>>>
>>> src/hotspot/share/runtime/arguments.hpp
>>> ???? No comments.
>>>
>>> src/hotspot/share/runtime/arguments.cpp
>>> ???? L715: // check for stale flags when we hit build 20 (which is 
>>> far enough into the 6 monthly
>>> ???????? typo: s/monthly/month/
>>>
>>> test/hotspot/gtest/runtime/test_special_flags.cpp
>>> ???? Nice!
>>>
>>> Thumbs up. I don't need to see a new webrev to fix the typo.
>>>
>>> Dan
>>>
>>>
>>>>
>>>> Bug report updated to show gtest output.
>>>>
>>>> Thanks,
>>>> David
>>>>
>>>> On 20/01/2020 7:46 am, David Holmes wrote:
>>>>> Hi Dan,
>>>>>
>>>>> On 18/01/2020 1:37 am, Daniel D. Daugherty wrote:
>>>>>> On 1/16/20 12:57 AM, David Holmes wrote:
>>>>>>> Getting back to this ...
>>>>>>
>>>>>> You added this update to the bug report:
>>>>>>
>>>>>>> Update: after further discussion it has been proposed that we use 
>>>>>>> the build number as the trigger for a whitebox or gtest that 
>>>>>>> performs the currently disabled full verification of the flag 
>>>>>>> table. So if a flag has not be obsoleted or expired as it should 
>>>>>>> by build N** then we fail the gtest. This will complement the 
>>>>>>> relaxing of the obsoletion check at the start of the release cycle. 
>>>>>>
>>>>>> I was expecting a new test in this latest webrev that would start 
>>>>>> failing
>>>>>> at Build 20... let me see what the latest webrev says...
>>>>>
>>>>> Mea culpa. When I started thinking about the test it was evident 
>>>>> the test logic would just involve the existing commented out 
>>>>> warning logic. So I thought lets just turn those warnings on at 
>>>>> build 20 but ...
>>>>>
>>>>>>>
>>>>>>> Please see updated webrev at:
>>>>>>>
>>>>>>> http://cr.openjdk.java.net/~dholmes/8235966/webrev/
>>>>>>
>>>>>> src/hotspot/share/runtime/arguments.cpp
>>>>>> ???? No comments.
>>>>>>
>>>>>> So the changes are the same as the last round with the addition of
>>>>>> enabling the following at B20:
>>>>>>
>>>>>> L755: ????????? warning("Global variable for obsolete special flag 
>>>>>> entry \"%s\" should be removed", flag.name);
>>>>>> L769: ????????? warning("Global variable for expired flag entry 
>>>>>> \"%s\" should be removed", flag.name);
>>>>>>
>>>>>>
>>>>>>> Apologies as I mistakenly overwrote the original instead of 
>>>>>>> creating v3.
>>>>>>>
>>>>>>> This version expands on the original proposal by uncommenting the 
>>>>>>> warnings about obsolete/expired flags that have not been removed 
>>>>>>> from globals*.hpp, so that we don't forget to do this work. 
>>>>>>> However these warnings are only enabled from build 20. I used 20 
>>>>>>> as being approx 2/3 through the release cycle - long enough that 
>>>>>>> the work should have been performed by then, whilst still leaving 
>>>>>>> time to perform the work before RDP2. Of course we can tweak this 
>>>>>>> number if there are issues with that choice.
>>>>>>
>>>>>> Okay... but doesn't this mean that every test would issue these 
>>>>>> warnings
>>>>>> as of B20 if we have not completed the work? So we have the 
>>>>>> potential of
>>>>>> a raft (some unknown number) of test failures due to unexpected 
>>>>>> output in
>>>>>> the form of these warning messages. And worse, these failures 
>>>>>> would be
>>>>>> unrelated to actual issue... :-(
>>>>>
>>>>> ... as you note the end result is not really what we want - a 
>>>>> single clearly failing test.
>>>>>
>>>>>> How about adding a diagnostic flag that enables these two warning
>>>>>> messages (in addition to the B20 check). Add a single test that runs:
>>>>>>
>>>>>> ?? java -XX:+UnlockDiagnosticVMOptions 
>>>>>> -XX:+FlagShouldNotBeDefinedCheck -version
>>>>>>
>>>>>> and when we hit B20, if there are still flags that haven't been 
>>>>>> removed,
>>>>>> then the test will fail and we'll have one test that fails (X the 
>>>>>> number
>>>>>> of configs that run the test).
>>>>>
>>>>> I definitely do not want to add another VM flag :) but will look at 
>>>>> how to (re)run that logic as part of a gtest.
>>>>>
>>>>> Thanks,
>>>>> David
>>>>>
>>>>>> Dan
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>> David
>>>>>>>
>>>>>>> On 20/12/2019 8:20 am, David Holmes wrote:
>>>>>>>> Thanks Dan.
>>>>>>>>
>>>>>>>> FTR I've updated the bug report with an extension to this 
>>>>>>>> proposal, which is to add back the flag table validation checks 
>>>>>>>> to use via a gtest that we only enable after a certain build in 
>>>>>>>> the release cycle (it always passes before then). That way we 
>>>>>>>> avoid the problems I've outlined with the initial version bump 
>>>>>>>> but also have a safety net in place to ensure we don't forget to 
>>>>>>>> actually obsolete/expire flags.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> David
>>>>>>>>
>>>>>>>> On 19/12/2019 3:37 am, Daniel D. Daugherty wrote:
>>>>>>>>> Hi David,
>>>>>>>>>
>>>>>>>>> On 12/17/19 5:03 PM, David Holmes wrote:
>>>>>>>>>> Hi Dan,
>>>>>>>>>>
>>>>>>>>>> Thanks for taking a look. Updated webrev:
>>>>>>>>>>
>>>>>>>>>> http://cr.openjdk.java.net/~dholmes/8235966/webrev.v2/
>>>>>>>>>
>>>>>>>>> src/hotspot/share/runtime/arguments.cpp
>>>>>>>>> ???? I like the updates to header comment for 
>>>>>>>>> verify_special_jvm_flags().
>>>>>>>>>
>>>>>>>>> Thumbs up.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Discussion below.
>>>>>>>>>
>>>>>>>>> Replies below.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 18/12/2019 1:47 am, Daniel D. Daugherty wrote:
>>>>>>>>>>> On 12/16/19 12:16 AM, David Holmes wrote:
>>>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8235966
>>>>>>>>>>>> webrev: http://cr.openjdk.java.net/~dholmes/8235966/webrev/
>>>>>>>>>>>
>>>>>>>>>>> src/hotspot/share/runtime/arguments.cpp
>>>>>>>>>>> ???? L745: ????? // if flag has become obsolete it should not 
>>>>>>>>>>> have a "globals" flag defined anymore.
>>>>>>>>>>> ???? L746: ????? if 
>>>>>>>>>>> (!version_less_than(JDK_Version::current(), flag.obsolete_in)) {
>>>>>>>>>>> ???? L747: ??????? if (JVMFlag::find_declared_flag(flag.name) 
>>>>>>>>>>> != NULL) {
>>>>>>>>>>> ???? L748: ????????? // Temporarily disable the warning: 8196739
>>>>>>>>>>> ???? L749: ????????? // warning("Global variable for obsolete 
>>>>>>>>>>> special flag entry \"%s\" should be removed", flag.name);
>>>>>>>>>>> ???? L750: ??????? }
>>>>>>>>>>> ???? L751: ????? }
>>>>>>>>>>> ???????? It seems like we've been down a similar road before:
>>>>>>>>>>>
>>>>>>>>>>> ???????? JDK-8196739 Disable obsolete/expired VM flag 
>>>>>>>>>>> transitional warnings
>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8196739
>>>>>>>>>>>
>>>>>>>>>>> ???????? This one may ring a bell... Fixed by dholmes in 
>>>>>>>>>>> jdk11-b01... :-)
>>>>>>>>>>>
>>>>>>>>>>> ???????? And this followup sub-task to re-enable that warning:
>>>>>>>>>>>
>>>>>>>>>>> ???????? JDK-8196741 Re-enable obsolete/expired VM flag 
>>>>>>>>>>> transitional warnings
>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8196741
>>>>>>>>>>>
>>>>>>>>>>> ???????? was closed as "Won't fix" on 2019.08.02.
>>>>>>>>>>>
>>>>>>>>>>> ???????? So the obvious questions:
>>>>>>>>>>>
>>>>>>>>>>> ???????? - Why is the new warning less problematic to tests 
>>>>>>>>>>> that don't
>>>>>>>>>>> ?????????? tolerate unexpected output?
>>>>>>>>>>
>>>>>>>>>> Two different situations. The commented out warning happens 
>>>>>>>>>> unconditionally when you run the VM and it finds any flag 
>>>>>>>>>> marked obsolete that hasn't been removed. Hence every single 
>>>>>>>>>> test will encounter this warning.
>>>>>>>>>
>>>>>>>>> Ouch on such verbosity.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> The situation I am modifying is when a test uses a flag that 
>>>>>>>>>> is marked for obsoletion. In the majority of cases the flag is 
>>>>>>>>>> already deprecated and so already issuing a deprecation 
>>>>>>>>>> warning that the test has to handle. Without my change there 
>>>>>>>>>> would still be an obsoletion warning, so this test is in for a 
>>>>>>>>>> warning no matter what.
>>>>>>>>>
>>>>>>>>> Good that your change only comes into play when the flag is used.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Also note that for hotspot at least we have strived to make 
>>>>>>>>>> tests tolerate unexpected output. The reason JDK-8196741 was 
>>>>>>>>>> closed as "won't fix" was because other areas wouldn't commit 
>>>>>>>>>> to doing that.
>>>>>>>>>
>>>>>>>>> Yup. Got it.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> ???????? - If you move forward with this fix, then I think 
>>>>>>>>>>> think code
>>>>>>>>>>> ?????????? block needs to be removed or modified or am I 
>>>>>>>>>>> missing something?
>>>>>>>>>>
>>>>>>>>>> I've rewritten the comment at the head of 
>>>>>>>>>> verify_special_jvm_flags to explain why we can't issue a 
>>>>>>>>>> warning, and have deleted the block.
>>>>>>>>>
>>>>>>>>> Thanks for deleting the stale code.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> ???????? There's a similar commented out check on L757-L765, 
>>>>>>>>>>> but that one
>>>>>>>>>>> ???????? is for an expired flag... You might want to 
>>>>>>>>>>> adjust/delete it also?
>>>>>>>>>>
>>>>>>>>>> Deleted.
>>>>>>>>>
>>>>>>>>> Thanks.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> ???? L753: warning("Special flag entry \"%s\" must be 
>>>>>>>>>>> explicitly obsoleted before expired.", flag.name);
>>>>>>>>>>> ???? L754: ??????? success = false;
>>>>>>>>>>> ???????? nit - s/before expired/before being expired/
>>>>>>>>>>> ???????? Update: I now see that "style" is in several places 
>>>>>>>>>>> in this
>>>>>>>>>>> ???????????? function. I'm not sure what to think here... it 
>>>>>>>>>>> grates,
>>>>>>>>>>> ?? ? ? ? ? ? but I can live with it.
>>>>>>>>>>>
>>>>>>>>>>> ???????? nit - L75[34] indented too much by two spaces.
>>>>>>>>>>
>>>>>>>>>> Fixed.
>>>>>>>>>>
>>>>>>>>>>> ???? L962: ????????? return real_name;
>>>>>>>>>>> ???????? nit - indented too much by two spaces.
>>>>>>>>>>
>>>>>>>>>> Fixed.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Trying to understand the modified logic in argument 
>>>>>>>>>>> processing is
>>>>>>>>>>> making my head spin...
>>>>>>>>>>
>>>>>>>>>> Mine too. It took a few attempts to put the logic in the right 
>>>>>>>>>> place and make adjustments so that it all works as expected 
>>>>>>>>>> for a correctly specified flag and an erroneous one.
>>>>>>>>>
>>>>>>>>> I keep trying to convince myself that we're improving this flag 
>>>>>>>>> and
>>>>>>>>> options code with each release... :-)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> - You've added a call to is_obsolete_flag() in
>>>>>>>>>>> ?? handle_aliases_and_deprecation() and is_obsolete_flag()
>>>>>>>>>>> ?? is where the new warning is output:
>>>>>>>>>>>
>>>>>>>>>>> ???? warning("Temporarily processing option %s; support is 
>>>>>>>>>>> scheduled for removal in %s"
>>>>>>>>>>>
>>>>>>>>>>> ?? handle_aliases_and_deprecation() is called from six 
>>>>>>>>>>> different places,
>>>>>>>>>>> ?? but the call sites are different based on the argument 
>>>>>>>>>>> pattern so I
>>>>>>>>>>> ?? have (mostly) convinced myself that there should not be 
>>>>>>>>>>> any duplicate
>>>>>>>>>>> ?? warning lines.
>>>>>>>>>>
>>>>>>>>>> Right - handle_aliases_and_deprecation is only called for a 
>>>>>>>>>> syntactically correct flag based on those patterns. It 
>>>>>>>>>> normally filters out obsoleted/expired flags and lets them 
>>>>>>>>>> fall through to later error processing (in process_argument 
>>>>>>>>>> after parse_arg returns false). That error processing is where 
>>>>>>>>>> the normal obsoletion check is performed. So I had to not 
>>>>>>>>>> filter the flag in handle_aliases_and_deprecation in this 
>>>>>>>>>> case, but still produce the warning for a malformed flag. E.g.
>>>>>>>>>>
>>>>>>>>>> java -XX:+UseParallelOldGC -version
>>>>>>>>>> Java HotSpot(TM) 64-Bit Server VM warning: Temporarily 
>>>>>>>>>> processing option UseParallelOldGC; support is scheduled for 
>>>>>>>>>> removal in 15.0
>>>>>>>>>> java version "15-internal" 2020-09-15
>>>>>>>>>>
>>>>>>>>>> java -XX:UseParallelOldGC -version
>>>>>>>>>> Java HotSpot(TM) 64-Bit Server VM warning: Temporarily 
>>>>>>>>>> processing option UseParallelOldGC; support is scheduled for 
>>>>>>>>>> removal in 15.0
>>>>>>>>>> Missing +/- setting for VM option 'UseParallelOldGC'
>>>>>>>>>> Error: Could not create the Java Virtual Machine.
>>>>>>>>>
>>>>>>>>> Thanks for the example. That helps a lot.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> So I now understand the new logic that allows an obsoleted 
>>>>>>>>>>> option
>>>>>>>>>>> to be specified with a warning as long as the option still 
>>>>>>>>>>> exists.
>>>>>>>>>>> I'm good with the technical change, but...
>>>>>>>>>>>
>>>>>>>>>>> I'm worried about tests that don't tolerate the new warning 
>>>>>>>>>>> mesg,
>>>>>>>>>>> i.e., why wouldn't this become an issue again:
>>>>>>>>>>>
>>>>>>>>>>> JDK-8196739 Disable obsolete/expired VM flag transitional 
>>>>>>>>>>> warnings
>>>>>>>>>>
>>>>>>>>>> Explained above.
>>>>>>>>>
>>>>>>>>> Yup and thanks.
>>>>>>>>>
>>>>>>>>> Dan
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> David
>>>>>>>>>>
>>>>>>>>>>> Dan
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> When a flag is marked as obsolete in the special-flags table 
>>>>>>>>>>>> we will ignore it and issue a warning that it is being 
>>>>>>>>>>>> ignored, as soon as we bump the version of the JDK. That 
>>>>>>>>>>>> means that any tests still using the obsolete flag may start 
>>>>>>>>>>>> to fail, leading to a surge of test failures at the start of 
>>>>>>>>>>>> a release cycle. For example for JDK 15 we have a whole 
>>>>>>>>>>>> bunch of JFR tests that fail because they still try to work 
>>>>>>>>>>>> with UseParallelOldGC. In another case 
>>>>>>>>>>>> runtime/cds/appcds/FieldLayoutFlags.java passes only be 
>>>>>>>>>>>> accident.
>>>>>>>>>>>>
>>>>>>>>>>>> When a flag is marked as obsolete for a release, all code 
>>>>>>>>>>>> involving that flag (including tests that use it) must be 
>>>>>>>>>>>> updated within that release and the flag itself removed. 
>>>>>>>>>>>> Whilst this is typically scheduled early in a release cycle 
>>>>>>>>>>>> it isn't reasonable to expect it to all occur within the 
>>>>>>>>>>>> first couple of days of the release cycle, nor do we want to 
>>>>>>>>>>>> have to ProblemList a bunch of tests when they start failing.
>>>>>>>>>>>>
>>>>>>>>>>>> What I propose is to instead allow an obsolete flag to 
>>>>>>>>>>>> continue to be processed as long as that code removal has 
>>>>>>>>>>>> not actually occurred - with an adjusted warning. The change 
>>>>>>>>>>>> I propose:
>>>>>>>>>>>>
>>>>>>>>>>>> - only treats an obsolete flag as obsolete if the flag 
>>>>>>>>>>>> cannot be found
>>>>>>>>>>>> - added a new flag verification rule that disallows 
>>>>>>>>>>>> obsoletion in an undefined version, but expiration in a 
>>>>>>>>>>>> specific version i.e. we must always explicitly obsolete a 
>>>>>>>>>>>> flag before we expire it.
>>>>>>>>>>>>
>>>>>>>>>>>> The only downside here is that if we actually forget to file 
>>>>>>>>>>>> an issue for the actual obsoletion work we may not notice 
>>>>>>>>>>>> via testing. Of course whenever a change is made to the 
>>>>>>>>>>>> flags table to add an entry then the issue to do the 
>>>>>>>>>>>> obsoletion should be filed at the same time.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> David
>>>>>>>>>>>> -----
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>
>>>

From igor.ignatyev at oracle.com  Thu Jan 23 23:00:13 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Thu, 23 Jan 2020 15:00:13 -0800
Subject: RFR: 8235966: Process obsolete flags less aggressively
In-Reply-To: <5e54498c-5eae-76f6-3661-f053d75d0f98@oracle.com>
References: <db63e5c9-a0c1-1575-e8dc-88429d6216fe@oracle.com>
 <674e4f0d-563f-3bb4-dee0-74aaacb9b270@oracle.com>
 <b32a2e50-62e3-ed94-7773-a6594a76e1aa@oracle.com>
 <c1a2f1e1-e86b-1fda-4632-ce9734e6e75f@oracle.com>
 <b8e70c4d-af76-a682-b4a9-839dd2671958@oracle.com>
 <6a3de8da-746d-7e5d-9a0e-2157a0963ed8@oracle.com>
 <b7f7df86-50f0-523e-6cb2-e6b7d5fdddde@oracle.com>
 <25156ff2-9dd2-53ae-16bc-83d60c0afb1a@oracle.com>
 <d70c939e-db2f-8fb1-2333-19fdd6153527@oracle.com>
 <3d066cd2-87f3-56ef-803b-a1f194ac761e@oracle.com>
 <e33ab2a9-52d1-a0fa-601b-31c99370c217@oracle.com>
 <311314c6-7e70-49be-41de-23ae3fa7f23e@oracle.com>
 <5e54498c-5eae-76f6-3661-f053d75d0f98@oracle.com>
Message-ID: <A3F37A5C-0BD1-4791-9D1E-F32447930A42@oracle.com>

Hi David,

in test_special_flags.cpp, I'd put the whole TEST_VM(special_flags, verify_special_flags) in #ifdef ASSERT / endif, other than that LGTM.

-- Igor

> On Jan 23, 2020, at 2:51 PM, David Holmes <david.holmes at oracle.com> wrote:
> 
> Ping! Anyone?
> 
> David
> 
> On 22/01/2020 12:06 pm, David Holmes wrote:
>> Can I get a second review please?
>> Thanks,
>> David
>> On 22/01/2020 9:10 am, David Holmes wrote:
>>> Thanks Dan.
>>> 
>>> Will fix the "typo" - though not actually a typo as it was a deliberate (mis)use of "monthly". I have always quantified "monthly" this way, but seems it is not correct. :)
>>> 
>>> Cheers,
>>> David
>>> 
>>> On 22/01/2020 1:37 am, Daniel D. Daugherty wrote:
>>>> On 1/19/20 8:49 PM, David Holmes wrote:
>>>>> gtest version:
>>>>> 
>>>>> http://cr.openjdk.java.net/~dholmes/8235966/webrev.v4/
>>>> 
>>>> src/hotspot/share/runtime/arguments.hpp
>>>>      No comments.
>>>> 
>>>> src/hotspot/share/runtime/arguments.cpp
>>>>      L715: // check for stale flags when we hit build 20 (which is far enough into the 6 monthly
>>>>          typo: s/monthly/month/
>>>> 
>>>> test/hotspot/gtest/runtime/test_special_flags.cpp
>>>>      Nice!
>>>> 
>>>> Thumbs up. I don't need to see a new webrev to fix the typo.
>>>> 
>>>> Dan
>>>> 
>>>> 
>>>>> 
>>>>> Bug report updated to show gtest output.
>>>>> 
>>>>> Thanks,
>>>>> David
>>>>> 
>>>>> On 20/01/2020 7:46 am, David Holmes wrote:
>>>>>> Hi Dan,
>>>>>> 
>>>>>> On 18/01/2020 1:37 am, Daniel D. Daugherty wrote:
>>>>>>> On 1/16/20 12:57 AM, David Holmes wrote:
>>>>>>>> Getting back to this ...
>>>>>>> 
>>>>>>> You added this update to the bug report:
>>>>>>> 
>>>>>>>> Update: after further discussion it has been proposed that we use the build number as the trigger for a whitebox or gtest that performs the currently disabled full verification of the flag table. So if a flag has not be obsoleted or expired as it should by build N** then we fail the gtest. This will complement the relaxing of the obsoletion check at the start of the release cycle. 
>>>>>>> 
>>>>>>> I was expecting a new test in this latest webrev that would start failing
>>>>>>> at Build 20... let me see what the latest webrev says...
>>>>>> 
>>>>>> Mea culpa. When I started thinking about the test it was evident the test logic would just involve the existing commented out warning logic. So I thought lets just turn those warnings on at build 20 but ...
>>>>>> 
>>>>>>>> 
>>>>>>>> Please see updated webrev at:
>>>>>>>> 
>>>>>>>> http://cr.openjdk.java.net/~dholmes/8235966/webrev/
>>>>>>> 
>>>>>>> src/hotspot/share/runtime/arguments.cpp
>>>>>>>      No comments.
>>>>>>> 
>>>>>>> So the changes are the same as the last round with the addition of
>>>>>>> enabling the following at B20:
>>>>>>> 
>>>>>>> L755:           warning("Global variable for obsolete special flag entry \"%s\" should be removed", flag.name);
>>>>>>> L769:           warning("Global variable for expired flag entry \"%s\" should be removed", flag.name);
>>>>>>> 
>>>>>>> 
>>>>>>>> Apologies as I mistakenly overwrote the original instead of creating v3.
>>>>>>>> 
>>>>>>>> This version expands on the original proposal by uncommenting the warnings about obsolete/expired flags that have not been removed from globals*.hpp, so that we don't forget to do this work. However these warnings are only enabled from build 20. I used 20 as being approx 2/3 through the release cycle - long enough that the work should have been performed by then, whilst still leaving time to perform the work before RDP2. Of course we can tweak this number if there are issues with that choice.
>>>>>>> 
>>>>>>> Okay... but doesn't this mean that every test would issue these warnings
>>>>>>> as of B20 if we have not completed the work? So we have the potential of
>>>>>>> a raft (some unknown number) of test failures due to unexpected output in
>>>>>>> the form of these warning messages. And worse, these failures would be
>>>>>>> unrelated to actual issue... :-(
>>>>>> 
>>>>>> ... as you note the end result is not really what we want - a single clearly failing test.
>>>>>> 
>>>>>>> How about adding a diagnostic flag that enables these two warning
>>>>>>> messages (in addition to the B20 check). Add a single test that runs:
>>>>>>> 
>>>>>>>    java -XX:+UnlockDiagnosticVMOptions -XX:+FlagShouldNotBeDefinedCheck -version
>>>>>>> 
>>>>>>> and when we hit B20, if there are still flags that haven't been removed,
>>>>>>> then the test will fail and we'll have one test that fails (X the number
>>>>>>> of configs that run the test).
>>>>>> 
>>>>>> I definitely do not want to add another VM flag :) but will look at how to (re)run that logic as part of a gtest.
>>>>>> 
>>>>>> Thanks,
>>>>>> David
>>>>>> 
>>>>>>> Dan
>>>>>>> 
>>>>>>> 
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> David
>>>>>>>> 
>>>>>>>> On 20/12/2019 8:20 am, David Holmes wrote:
>>>>>>>>> Thanks Dan.
>>>>>>>>> 
>>>>>>>>> FTR I've updated the bug report with an extension to this proposal, which is to add back the flag table validation checks to use via a gtest that we only enable after a certain build in the release cycle (it always passes before then). That way we avoid the problems I've outlined with the initial version bump but also have a safety net in place to ensure we don't forget to actually obsolete/expire flags.
>>>>>>>>> 
>>>>>>>>> Cheers,
>>>>>>>>> David
>>>>>>>>> 
>>>>>>>>> On 19/12/2019 3:37 am, Daniel D. Daugherty wrote:
>>>>>>>>>> Hi David,
>>>>>>>>>> 
>>>>>>>>>> On 12/17/19 5:03 PM, David Holmes wrote:
>>>>>>>>>>> Hi Dan,
>>>>>>>>>>> 
>>>>>>>>>>> Thanks for taking a look. Updated webrev:
>>>>>>>>>>> 
>>>>>>>>>>> http://cr.openjdk.java.net/~dholmes/8235966/webrev.v2/
>>>>>>>>>> 
>>>>>>>>>> src/hotspot/share/runtime/arguments.cpp
>>>>>>>>>>      I like the updates to header comment for verify_special_jvm_flags().
>>>>>>>>>> 
>>>>>>>>>> Thumbs up.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> Discussion below.
>>>>>>>>>> 
>>>>>>>>>> Replies below.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On 18/12/2019 1:47 am, Daniel D. Daugherty wrote:
>>>>>>>>>>>> On 12/16/19 12:16 AM, David Holmes wrote:
>>>>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8235966
>>>>>>>>>>>>> webrev: http://cr.openjdk.java.net/~dholmes/8235966/webrev/
>>>>>>>>>>>> 
>>>>>>>>>>>> src/hotspot/share/runtime/arguments.cpp
>>>>>>>>>>>>      L745:       // if flag has become obsolete it should not have a "globals" flag defined anymore.
>>>>>>>>>>>>      L746:       if (!version_less_than(JDK_Version::current(), flag.obsolete_in)) {
>>>>>>>>>>>>      L747:         if (JVMFlag::find_declared_flag(flag.name) != NULL) {
>>>>>>>>>>>>      L748:           // Temporarily disable the warning: 8196739
>>>>>>>>>>>>      L749:           // warning("Global variable for obsolete special flag entry \"%s\" should be removed", flag.name);
>>>>>>>>>>>>      L750:         }
>>>>>>>>>>>>      L751:       }
>>>>>>>>>>>>          It seems like we've been down a similar road before:
>>>>>>>>>>>> 
>>>>>>>>>>>>          JDK-8196739 Disable obsolete/expired VM flag transitional warnings
>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8196739
>>>>>>>>>>>> 
>>>>>>>>>>>>          This one may ring a bell... Fixed by dholmes in jdk11-b01... :-)
>>>>>>>>>>>> 
>>>>>>>>>>>>          And this followup sub-task to re-enable that warning:
>>>>>>>>>>>> 
>>>>>>>>>>>>          JDK-8196741 Re-enable obsolete/expired VM flag transitional warnings
>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8196741
>>>>>>>>>>>> 
>>>>>>>>>>>>          was closed as "Won't fix" on 2019.08.02.
>>>>>>>>>>>> 
>>>>>>>>>>>>          So the obvious questions:
>>>>>>>>>>>> 
>>>>>>>>>>>>          - Why is the new warning less problematic to tests that don't
>>>>>>>>>>>>            tolerate unexpected output?
>>>>>>>>>>> 
>>>>>>>>>>> Two different situations. The commented out warning happens unconditionally when you run the VM and it finds any flag marked obsolete that hasn't been removed. Hence every single test will encounter this warning.
>>>>>>>>>> 
>>>>>>>>>> Ouch on such verbosity.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> The situation I am modifying is when a test uses a flag that is marked for obsoletion. In the majority of cases the flag is already deprecated and so already issuing a deprecation warning that the test has to handle. Without my change there would still be an obsoletion warning, so this test is in for a warning no matter what.
>>>>>>>>>> 
>>>>>>>>>> Good that your change only comes into play when the flag is used.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> Also note that for hotspot at least we have strived to make tests tolerate unexpected output. The reason JDK-8196741 was closed as "won't fix" was because other areas wouldn't commit to doing that.
>>>>>>>>>> 
>>>>>>>>>> Yup. Got it.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>>          - If you move forward with this fix, then I think think code
>>>>>>>>>>>>            block needs to be removed or modified or am I missing something?
>>>>>>>>>>> 
>>>>>>>>>>> I've rewritten the comment at the head of verify_special_jvm_flags to explain why we can't issue a warning, and have deleted the block.
>>>>>>>>>> 
>>>>>>>>>> Thanks for deleting the stale code.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>>          There's a similar commented out check on L757-L765, but that one
>>>>>>>>>>>>          is for an expired flag... You might want to adjust/delete it also?
>>>>>>>>>>> 
>>>>>>>>>>> Deleted.
>>>>>>>>>> 
>>>>>>>>>> Thanks.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>>      L753: warning("Special flag entry \"%s\" must be explicitly obsoleted before expired.", flag.name);
>>>>>>>>>>>>      L754:         success = false;
>>>>>>>>>>>>          nit - s/before expired/before being expired/
>>>>>>>>>>>>          Update: I now see that "style" is in several places in this
>>>>>>>>>>>>              function. I'm not sure what to think here... it grates,
>>>>>>>>>>>>              but I can live with it.
>>>>>>>>>>>> 
>>>>>>>>>>>>          nit - L75[34] indented too much by two spaces.
>>>>>>>>>>> 
>>>>>>>>>>> Fixed.
>>>>>>>>>>> 
>>>>>>>>>>>>      L962:           return real_name;
>>>>>>>>>>>>          nit - indented too much by two spaces.
>>>>>>>>>>> 
>>>>>>>>>>> Fixed.
>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> Trying to understand the modified logic in argument processing is
>>>>>>>>>>>> making my head spin...
>>>>>>>>>>> 
>>>>>>>>>>> Mine too. It took a few attempts to put the logic in the right place and make adjustments so that it all works as expected for a correctly specified flag and an erroneous one.
>>>>>>>>>> 
>>>>>>>>>> I keep trying to convince myself that we're improving this flag and
>>>>>>>>>> options code with each release... :-)
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>> - You've added a call to is_obsolete_flag() in
>>>>>>>>>>>>    handle_aliases_and_deprecation() and is_obsolete_flag()
>>>>>>>>>>>>    is where the new warning is output:
>>>>>>>>>>>> 
>>>>>>>>>>>>      warning("Temporarily processing option %s; support is scheduled for removal in %s"
>>>>>>>>>>>> 
>>>>>>>>>>>>    handle_aliases_and_deprecation() is called from six different places,
>>>>>>>>>>>>    but the call sites are different based on the argument pattern so I
>>>>>>>>>>>>    have (mostly) convinced myself that there should not be any duplicate
>>>>>>>>>>>>    warning lines.
>>>>>>>>>>> 
>>>>>>>>>>> Right - handle_aliases_and_deprecation is only called for a syntactically correct flag based on those patterns. It normally filters out obsoleted/expired flags and lets them fall through to later error processing (in process_argument after parse_arg returns false). That error processing is where the normal obsoletion check is performed. So I had to not filter the flag in handle_aliases_and_deprecation in this case, but still produce the warning for a malformed flag. E.g.
>>>>>>>>>>> 
>>>>>>>>>>> java -XX:+UseParallelOldGC -version
>>>>>>>>>>> Java HotSpot(TM) 64-Bit Server VM warning: Temporarily processing option UseParallelOldGC; support is scheduled for removal in 15.0
>>>>>>>>>>> java version "15-internal" 2020-09-15
>>>>>>>>>>> 
>>>>>>>>>>> java -XX:UseParallelOldGC -version
>>>>>>>>>>> Java HotSpot(TM) 64-Bit Server VM warning: Temporarily processing option UseParallelOldGC; support is scheduled for removal in 15.0
>>>>>>>>>>> Missing +/- setting for VM option 'UseParallelOldGC'
>>>>>>>>>>> Error: Could not create the Java Virtual Machine.
>>>>>>>>>> 
>>>>>>>>>> Thanks for the example. That helps a lot.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>> So I now understand the new logic that allows an obsoleted option
>>>>>>>>>>>> to be specified with a warning as long as the option still exists.
>>>>>>>>>>>> I'm good with the technical change, but...
>>>>>>>>>>>> 
>>>>>>>>>>>> I'm worried about tests that don't tolerate the new warning mesg,
>>>>>>>>>>>> i.e., why wouldn't this become an issue again:
>>>>>>>>>>>> 
>>>>>>>>>>>> JDK-8196739 Disable obsolete/expired VM flag transitional warnings
>>>>>>>>>>> 
>>>>>>>>>>> Explained above.
>>>>>>>>>> 
>>>>>>>>>> Yup and thanks.
>>>>>>>>>> 
>>>>>>>>>> Dan
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> Thanks,
>>>>>>>>>>> David
>>>>>>>>>>> 
>>>>>>>>>>>> Dan
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> When a flag is marked as obsolete in the special-flags table we will ignore it and issue a warning that it is being ignored, as soon as we bump the version of the JDK. That means that any tests still using the obsolete flag may start to fail, leading to a surge of test failures at the start of a release cycle. For example for JDK 15 we have a whole bunch of JFR tests that fail because they still try to work with UseParallelOldGC. In another case runtime/cds/appcds/FieldLayoutFlags.java passes only be accident.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> When a flag is marked as obsolete for a release, all code involving that flag (including tests that use it) must be updated within that release and the flag itself removed. Whilst this is typically scheduled early in a release cycle it isn't reasonable to expect it to all occur within the first couple of days of the release cycle, nor do we want to have to ProblemList a bunch of tests when they start failing.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> What I propose is to instead allow an obsolete flag to continue to be processed as long as that code removal has not actually occurred - with an adjusted warning. The change I propose:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> - only treats an obsolete flag as obsolete if the flag cannot be found
>>>>>>>>>>>>> - added a new flag verification rule that disallows obsoletion in an undefined version, but expiration in a specific version i.e. we must always explicitly obsolete a flag before we expire it.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> The only downside here is that if we actually forget to file an issue for the actual obsoletion work we may not notice via testing. Of course whenever a change is made to the flags table to add an entry then the issue to do the obsoletion should be filed at the same time.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> David
>>>>>>>>>>>>> -----
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>> 
>>>> 


From david.holmes at oracle.com  Thu Jan 23 23:10:17 2020
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 24 Jan 2020 09:10:17 +1000
Subject: RFR: 8235966: Process obsolete flags less aggressively
In-Reply-To: <A3F37A5C-0BD1-4791-9D1E-F32447930A42@oracle.com>
References: <db63e5c9-a0c1-1575-e8dc-88429d6216fe@oracle.com>
 <674e4f0d-563f-3bb4-dee0-74aaacb9b270@oracle.com>
 <b32a2e50-62e3-ed94-7773-a6594a76e1aa@oracle.com>
 <c1a2f1e1-e86b-1fda-4632-ce9734e6e75f@oracle.com>
 <b8e70c4d-af76-a682-b4a9-839dd2671958@oracle.com>
 <6a3de8da-746d-7e5d-9a0e-2157a0963ed8@oracle.com>
 <b7f7df86-50f0-523e-6cb2-e6b7d5fdddde@oracle.com>
 <25156ff2-9dd2-53ae-16bc-83d60c0afb1a@oracle.com>
 <d70c939e-db2f-8fb1-2333-19fdd6153527@oracle.com>
 <3d066cd2-87f3-56ef-803b-a1f194ac761e@oracle.com>
 <e33ab2a9-52d1-a0fa-601b-31c99370c217@oracle.com>
 <311314c6-7e70-49be-41de-23ae3fa7f23e@oracle.com>
 <5e54498c-5eae-76f6-3661-f053d75d0f98@oracle.com>
 <A3F37A5C-0BD1-4791-9D1E-F32447930A42@oracle.com>
Message-ID: <4a8f2653-ee54-6d05-bf8e-0332840807cf@oracle.com>

Hi Igor,

On 24/01/2020 9:00 am, Igor Ignatyev wrote:
> Hi David,
> 
> in test_special_flags.cpp, I'd put the whole TEST_VM(special_flags, verify_special_flags) in #ifdef ASSERT / endif, other than that LGTM.

I can try that - I wasn't sure if an empty test file would cause a problem.

Thanks for the review!

David

> -- Igor
> 
>> On Jan 23, 2020, at 2:51 PM, David Holmes <david.holmes at oracle.com> wrote:
>>
>> Ping! Anyone?
>>
>> David
>>
>> On 22/01/2020 12:06 pm, David Holmes wrote:
>>> Can I get a second review please?
>>> Thanks,
>>> David
>>> On 22/01/2020 9:10 am, David Holmes wrote:
>>>> Thanks Dan.
>>>>
>>>> Will fix the "typo" - though not actually a typo as it was a deliberate (mis)use of "monthly". I have always quantified "monthly" this way, but seems it is not correct. :)
>>>>
>>>> Cheers,
>>>> David
>>>>
>>>> On 22/01/2020 1:37 am, Daniel D. Daugherty wrote:
>>>>> On 1/19/20 8:49 PM, David Holmes wrote:
>>>>>> gtest version:
>>>>>>
>>>>>> http://cr.openjdk.java.net/~dholmes/8235966/webrev.v4/
>>>>>
>>>>> src/hotspot/share/runtime/arguments.hpp
>>>>>       No comments.
>>>>>
>>>>> src/hotspot/share/runtime/arguments.cpp
>>>>>       L715: // check for stale flags when we hit build 20 (which is far enough into the 6 monthly
>>>>>           typo: s/monthly/month/
>>>>>
>>>>> test/hotspot/gtest/runtime/test_special_flags.cpp
>>>>>       Nice!
>>>>>
>>>>> Thumbs up. I don't need to see a new webrev to fix the typo.
>>>>>
>>>>> Dan
>>>>>
>>>>>
>>>>>>
>>>>>> Bug report updated to show gtest output.
>>>>>>
>>>>>> Thanks,
>>>>>> David
>>>>>>
>>>>>> On 20/01/2020 7:46 am, David Holmes wrote:
>>>>>>> Hi Dan,
>>>>>>>
>>>>>>> On 18/01/2020 1:37 am, Daniel D. Daugherty wrote:
>>>>>>>> On 1/16/20 12:57 AM, David Holmes wrote:
>>>>>>>>> Getting back to this ...
>>>>>>>>
>>>>>>>> You added this update to the bug report:
>>>>>>>>
>>>>>>>>> Update: after further discussion it has been proposed that we use the build number as the trigger for a whitebox or gtest that performs the currently disabled full verification of the flag table. So if a flag has not be obsoleted or expired as it should by build N** then we fail the gtest. This will complement the relaxing of the obsoletion check at the start of the release cycle.
>>>>>>>>
>>>>>>>> I was expecting a new test in this latest webrev that would start failing
>>>>>>>> at Build 20... let me see what the latest webrev says...
>>>>>>>
>>>>>>> Mea culpa. When I started thinking about the test it was evident the test logic would just involve the existing commented out warning logic. So I thought lets just turn those warnings on at build 20 but ...
>>>>>>>
>>>>>>>>>
>>>>>>>>> Please see updated webrev at:
>>>>>>>>>
>>>>>>>>> http://cr.openjdk.java.net/~dholmes/8235966/webrev/
>>>>>>>>
>>>>>>>> src/hotspot/share/runtime/arguments.cpp
>>>>>>>>       No comments.
>>>>>>>>
>>>>>>>> So the changes are the same as the last round with the addition of
>>>>>>>> enabling the following at B20:
>>>>>>>>
>>>>>>>> L755:           warning("Global variable for obsolete special flag entry \"%s\" should be removed", flag.name);
>>>>>>>> L769:           warning("Global variable for expired flag entry \"%s\" should be removed", flag.name);
>>>>>>>>
>>>>>>>>
>>>>>>>>> Apologies as I mistakenly overwrote the original instead of creating v3.
>>>>>>>>>
>>>>>>>>> This version expands on the original proposal by uncommenting the warnings about obsolete/expired flags that have not been removed from globals*.hpp, so that we don't forget to do this work. However these warnings are only enabled from build 20. I used 20 as being approx 2/3 through the release cycle - long enough that the work should have been performed by then, whilst still leaving time to perform the work before RDP2. Of course we can tweak this number if there are issues with that choice.
>>>>>>>>
>>>>>>>> Okay... but doesn't this mean that every test would issue these warnings
>>>>>>>> as of B20 if we have not completed the work? So we have the potential of
>>>>>>>> a raft (some unknown number) of test failures due to unexpected output in
>>>>>>>> the form of these warning messages. And worse, these failures would be
>>>>>>>> unrelated to actual issue... :-(
>>>>>>>
>>>>>>> ... as you note the end result is not really what we want - a single clearly failing test.
>>>>>>>
>>>>>>>> How about adding a diagnostic flag that enables these two warning
>>>>>>>> messages (in addition to the B20 check). Add a single test that runs:
>>>>>>>>
>>>>>>>>     java -XX:+UnlockDiagnosticVMOptions -XX:+FlagShouldNotBeDefinedCheck -version
>>>>>>>>
>>>>>>>> and when we hit B20, if there are still flags that haven't been removed,
>>>>>>>> then the test will fail and we'll have one test that fails (X the number
>>>>>>>> of configs that run the test).
>>>>>>>
>>>>>>> I definitely do not want to add another VM flag :) but will look at how to (re)run that logic as part of a gtest.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> David
>>>>>>>
>>>>>>>> Dan
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> David
>>>>>>>>>
>>>>>>>>> On 20/12/2019 8:20 am, David Holmes wrote:
>>>>>>>>>> Thanks Dan.
>>>>>>>>>>
>>>>>>>>>> FTR I've updated the bug report with an extension to this proposal, which is to add back the flag table validation checks to use via a gtest that we only enable after a certain build in the release cycle (it always passes before then). That way we avoid the problems I've outlined with the initial version bump but also have a safety net in place to ensure we don't forget to actually obsolete/expire flags.
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>> David
>>>>>>>>>>
>>>>>>>>>> On 19/12/2019 3:37 am, Daniel D. Daugherty wrote:
>>>>>>>>>>> Hi David,
>>>>>>>>>>>
>>>>>>>>>>> On 12/17/19 5:03 PM, David Holmes wrote:
>>>>>>>>>>>> Hi Dan,
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for taking a look. Updated webrev:
>>>>>>>>>>>>
>>>>>>>>>>>> http://cr.openjdk.java.net/~dholmes/8235966/webrev.v2/
>>>>>>>>>>>
>>>>>>>>>>> src/hotspot/share/runtime/arguments.cpp
>>>>>>>>>>>       I like the updates to header comment for verify_special_jvm_flags().
>>>>>>>>>>>
>>>>>>>>>>> Thumbs up.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Discussion below.
>>>>>>>>>>>
>>>>>>>>>>> Replies below.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 18/12/2019 1:47 am, Daniel D. Daugherty wrote:
>>>>>>>>>>>>> On 12/16/19 12:16 AM, David Holmes wrote:
>>>>>>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8235966
>>>>>>>>>>>>>> webrev: http://cr.openjdk.java.net/~dholmes/8235966/webrev/
>>>>>>>>>>>>>
>>>>>>>>>>>>> src/hotspot/share/runtime/arguments.cpp
>>>>>>>>>>>>>       L745:       // if flag has become obsolete it should not have a "globals" flag defined anymore.
>>>>>>>>>>>>>       L746:       if (!version_less_than(JDK_Version::current(), flag.obsolete_in)) {
>>>>>>>>>>>>>       L747:         if (JVMFlag::find_declared_flag(flag.name) != NULL) {
>>>>>>>>>>>>>       L748:           // Temporarily disable the warning: 8196739
>>>>>>>>>>>>>       L749:           // warning("Global variable for obsolete special flag entry \"%s\" should be removed", flag.name);
>>>>>>>>>>>>>       L750:         }
>>>>>>>>>>>>>       L751:       }
>>>>>>>>>>>>>           It seems like we've been down a similar road before:
>>>>>>>>>>>>>
>>>>>>>>>>>>>           JDK-8196739 Disable obsolete/expired VM flag transitional warnings
>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8196739
>>>>>>>>>>>>>
>>>>>>>>>>>>>           This one may ring a bell... Fixed by dholmes in jdk11-b01... :-)
>>>>>>>>>>>>>
>>>>>>>>>>>>>           And this followup sub-task to re-enable that warning:
>>>>>>>>>>>>>
>>>>>>>>>>>>>           JDK-8196741 Re-enable obsolete/expired VM flag transitional warnings
>>>>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-8196741
>>>>>>>>>>>>>
>>>>>>>>>>>>>           was closed as "Won't fix" on 2019.08.02.
>>>>>>>>>>>>>
>>>>>>>>>>>>>           So the obvious questions:
>>>>>>>>>>>>>
>>>>>>>>>>>>>           - Why is the new warning less problematic to tests that don't
>>>>>>>>>>>>>             tolerate unexpected output?
>>>>>>>>>>>>
>>>>>>>>>>>> Two different situations. The commented out warning happens unconditionally when you run the VM and it finds any flag marked obsolete that hasn't been removed. Hence every single test will encounter this warning.
>>>>>>>>>>>
>>>>>>>>>>> Ouch on such verbosity.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> The situation I am modifying is when a test uses a flag that is marked for obsoletion. In the majority of cases the flag is already deprecated and so already issuing a deprecation warning that the test has to handle. Without my change there would still be an obsoletion warning, so this test is in for a warning no matter what.
>>>>>>>>>>>
>>>>>>>>>>> Good that your change only comes into play when the flag is used.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Also note that for hotspot at least we have strived to make tests tolerate unexpected output. The reason JDK-8196741 was closed as "won't fix" was because other areas wouldn't commit to doing that.
>>>>>>>>>>>
>>>>>>>>>>> Yup. Got it.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>           - If you move forward with this fix, then I think think code
>>>>>>>>>>>>>             block needs to be removed or modified or am I missing something?
>>>>>>>>>>>>
>>>>>>>>>>>> I've rewritten the comment at the head of verify_special_jvm_flags to explain why we can't issue a warning, and have deleted the block.
>>>>>>>>>>>
>>>>>>>>>>> Thanks for deleting the stale code.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>           There's a similar commented out check on L757-L765, but that one
>>>>>>>>>>>>>           is for an expired flag... You might want to adjust/delete it also?
>>>>>>>>>>>>
>>>>>>>>>>>> Deleted.
>>>>>>>>>>>
>>>>>>>>>>> Thanks.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>       L753: warning("Special flag entry \"%s\" must be explicitly obsoleted before expired.", flag.name);
>>>>>>>>>>>>>       L754:         success = false;
>>>>>>>>>>>>>           nit - s/before expired/before being expired/
>>>>>>>>>>>>>           Update: I now see that "style" is in several places in this
>>>>>>>>>>>>>               function. I'm not sure what to think here... it grates,
>>>>>>>>>>>>>               but I can live with it.
>>>>>>>>>>>>>
>>>>>>>>>>>>>           nit - L75[34] indented too much by two spaces.
>>>>>>>>>>>>
>>>>>>>>>>>> Fixed.
>>>>>>>>>>>>
>>>>>>>>>>>>>       L962:           return real_name;
>>>>>>>>>>>>>           nit - indented too much by two spaces.
>>>>>>>>>>>>
>>>>>>>>>>>> Fixed.
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Trying to understand the modified logic in argument processing is
>>>>>>>>>>>>> making my head spin...
>>>>>>>>>>>>
>>>>>>>>>>>> Mine too. It took a few attempts to put the logic in the right place and make adjustments so that it all works as expected for a correctly specified flag and an erroneous one.
>>>>>>>>>>>
>>>>>>>>>>> I keep trying to convince myself that we're improving this flag and
>>>>>>>>>>> options code with each release... :-)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> - You've added a call to is_obsolete_flag() in
>>>>>>>>>>>>>     handle_aliases_and_deprecation() and is_obsolete_flag()
>>>>>>>>>>>>>     is where the new warning is output:
>>>>>>>>>>>>>
>>>>>>>>>>>>>       warning("Temporarily processing option %s; support is scheduled for removal in %s"
>>>>>>>>>>>>>
>>>>>>>>>>>>>     handle_aliases_and_deprecation() is called from six different places,
>>>>>>>>>>>>>     but the call sites are different based on the argument pattern so I
>>>>>>>>>>>>>     have (mostly) convinced myself that there should not be any duplicate
>>>>>>>>>>>>>     warning lines.
>>>>>>>>>>>>
>>>>>>>>>>>> Right - handle_aliases_and_deprecation is only called for a syntactically correct flag based on those patterns. It normally filters out obsoleted/expired flags and lets them fall through to later error processing (in process_argument after parse_arg returns false). That error processing is where the normal obsoletion check is performed. So I had to not filter the flag in handle_aliases_and_deprecation in this case, but still produce the warning for a malformed flag. E.g.
>>>>>>>>>>>>
>>>>>>>>>>>> java -XX:+UseParallelOldGC -version
>>>>>>>>>>>> Java HotSpot(TM) 64-Bit Server VM warning: Temporarily processing option UseParallelOldGC; support is scheduled for removal in 15.0
>>>>>>>>>>>> java version "15-internal" 2020-09-15
>>>>>>>>>>>>
>>>>>>>>>>>> java -XX:UseParallelOldGC -version
>>>>>>>>>>>> Java HotSpot(TM) 64-Bit Server VM warning: Temporarily processing option UseParallelOldGC; support is scheduled for removal in 15.0
>>>>>>>>>>>> Missing +/- setting for VM option 'UseParallelOldGC'
>>>>>>>>>>>> Error: Could not create the Java Virtual Machine.
>>>>>>>>>>>
>>>>>>>>>>> Thanks for the example. That helps a lot.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> So I now understand the new logic that allows an obsoleted option
>>>>>>>>>>>>> to be specified with a warning as long as the option still exists.
>>>>>>>>>>>>> I'm good with the technical change, but...
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'm worried about tests that don't tolerate the new warning mesg,
>>>>>>>>>>>>> i.e., why wouldn't this become an issue again:
>>>>>>>>>>>>>
>>>>>>>>>>>>> JDK-8196739 Disable obsolete/expired VM flag transitional warnings
>>>>>>>>>>>>
>>>>>>>>>>>> Explained above.
>>>>>>>>>>>
>>>>>>>>>>> Yup and thanks.
>>>>>>>>>>>
>>>>>>>>>>> Dan
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> David
>>>>>>>>>>>>
>>>>>>>>>>>>> Dan
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> When a flag is marked as obsolete in the special-flags table we will ignore it and issue a warning that it is being ignored, as soon as we bump the version of the JDK. That means that any tests still using the obsolete flag may start to fail, leading to a surge of test failures at the start of a release cycle. For example for JDK 15 we have a whole bunch of JFR tests that fail because they still try to work with UseParallelOldGC. In another case runtime/cds/appcds/FieldLayoutFlags.java passes only be accident.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> When a flag is marked as obsolete for a release, all code involving that flag (including tests that use it) must be updated within that release and the flag itself removed. Whilst this is typically scheduled early in a release cycle it isn't reasonable to expect it to all occur within the first couple of days of the release cycle, nor do we want to have to ProblemList a bunch of tests when they start failing.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> What I propose is to instead allow an obsolete flag to continue to be processed as long as that code removal has not actually occurred - with an adjusted warning. The change I propose:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - only treats an obsolete flag as obsolete if the flag cannot be found
>>>>>>>>>>>>>> - added a new flag verification rule that disallows obsoletion in an undefined version, but expiration in a specific version i.e. we must always explicitly obsolete a flag before we expire it.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The only downside here is that if we actually forget to file an issue for the actual obsoletion work we may not notice via testing. Of course whenever a change is made to the flags table to add an entry then the issue to do the obsoletion should be filed at the same time.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> David
>>>>>>>>>>>>>> -----
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>
> 

From shade at redhat.com  Fri Jan 24 08:46:05 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Fri, 24 Jan 2020 09:46:05 +0100
Subject: RFR[L]: 8237767 Field layout computation overhaul
In-Reply-To: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
Message-ID: <aeed1fc3-87f3-f979-fa92-e81d524b4772@redhat.com>

Hi Frederic,

On 1/23/20 4:03 PM, Frederic Parain wrote:
> CR: https://bugs.openjdk.java.net/browse/JDK-8237767
> webrev: http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/
> 
> The CR includes a detailed description of the motivation and the implementation.

I ran the patched jdk through JOL Samples [1], and it works impressively well. I think these RFRs
can be closed as duplicates now:
  https://bugs.openjdk.java.net/browse/JDK-8024912
  https://bugs.openjdk.java.net/browse/JDK-8024912

Nit: I think the flag should be UseNew*Field*Layout.

-- 
Thanks,
-Aleksey

[1]
https://hg.openjdk.java.net/code-tools/jol/file/f1e978b895a0/jol-samples/src/main/java/org/openjdk/jol/samples


From shade at redhat.com  Fri Jan 24 08:46:55 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Fri, 24 Jan 2020 09:46:55 +0100
Subject: RFR[L]: 8237767 Field layout computation overhaul
In-Reply-To: <aeed1fc3-87f3-f979-fa92-e81d524b4772@redhat.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
 <aeed1fc3-87f3-f979-fa92-e81d524b4772@redhat.com>
Message-ID: <668eab07-adc7-548e-546d-1cc6b314f13e@redhat.com>

On 1/24/20 9:46 AM, Aleksey Shipilev wrote:
> I ran the patched jdk through JOL Samples [1], and it works impressively well. I think these RFRs
> can be closed as duplicates now:
>   https://bugs.openjdk.java.net/browse/JDK-8024912
>   https://bugs.openjdk.java.net/browse/JDK-8024912

I meant https://bugs.openjdk.java.net/browse/JDK-8024913 on the second line, of course.

-- 
Thanks,
-Aleksey


From adinn at redhat.com  Fri Jan 24 09:01:09 2020
From: adinn at redhat.com (Andrew Dinn)
Date: Fri, 24 Jan 2020 09:01:09 +0000
Subject: 8230392: Define AArch64 as MULTI_COPY_ATOMIC
In-Reply-To: <53962163-d04e-e742-20c1-441ea47ea234@oracle.com>
References: <a8d3f689-dde7-e5ac-c06e-eb3dc1355553@redhat.com>
 <053adeb2-5f00-5cf6-c96e-97d46d37f5a3@oracle.com>
 <7c242604-aac9-243f-5a89-70853351aa12@redhat.com>
 <53962163-d04e-e742-20c1-441ea47ea234@oracle.com>
Message-ID: <82bdb7e1-2526-75e6-709a-f3eb74678fa9@redhat.com>

On 23/01/2020 22:41, David Holmes wrote:
> On 24/01/2020 2:47 am, Andrew Haley wrote:
>> http://cr.openjdk.java.net/~aph/8230392-2/
>>
>> Better?
> 
> That seems fine to me.

Yes, indeed, it looks good.

> This does make me wonder whether other lock-free code in the VM needs
> special handling for non-CPU_MULTI_COPY_ATOMIC ??
The only other place I could find a mention (not a use) of symbol
CPU_MULTI_COPY_ATOMIC -- excluding the point where it is defined -- was
in a comment in parse1.cpp. That comment talks about barriers for final
fields which I believe is handled via definition of
support_IRIW_for_not_multiple_copy_atomic_cpu.

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill


From adinn at redhat.com  Fri Jan 24 09:06:18 2020
From: adinn at redhat.com (Andrew Dinn)
Date: Fri, 24 Jan 2020 09:06:18 +0000
Subject: 8230392: Define AArch64 as MULTI_COPY_ATOMIC
In-Reply-To: <82bdb7e1-2526-75e6-709a-f3eb74678fa9@redhat.com>
References: <a8d3f689-dde7-e5ac-c06e-eb3dc1355553@redhat.com>
 <053adeb2-5f00-5cf6-c96e-97d46d37f5a3@oracle.com>
 <7c242604-aac9-243f-5a89-70853351aa12@redhat.com>
 <53962163-d04e-e742-20c1-441ea47ea234@oracle.com>
 <82bdb7e1-2526-75e6-709a-f3eb74678fa9@redhat.com>
Message-ID: <3861243d-828b-fd9e-dfd2-8b5ea60ab12e@redhat.com>

On 24/01/2020 09:01, Andrew Dinn wrote:
> On 23/01/2020 22:41, David Holmes wrote:
>> On 24/01/2020 2:47 am, Andrew Haley wrote:
>>> http://cr.openjdk.java.net/~aph/8230392-2/
>>>
>>> Better?
>>
>> That seems fine to me.
> 
> Yes, indeed, it looks good.
> 
>> This does make me wonder whether other lock-free code in the VM needs
>> special handling for non-CPU_MULTI_COPY_ATOMIC ??
> The only other place I could find a mention (not a use) of symbol
> CPU_MULTI_COPY_ATOMIC -- excluding the point where it is defined -- was
> in a comment in parse1.cpp. That comment talks about barriers for final
> fields which I believe is handled via definition of
> support_IRIW_for_not_multiple_copy_atomic_cpu.
Ah sorry, David, having reread I see now your concern may be whether
other lock-free code /should/ be considering the setting of
CPU_MULTI_COPY_ATOMIC but currently may /not/ be. That's an interesting
question . . . :-)

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill


From aph at redhat.com  Fri Jan 24 09:06:29 2020
From: aph at redhat.com (Andrew Haley)
Date: Fri, 24 Jan 2020 09:06:29 +0000
Subject: 8230392: Define AArch64 as MULTI_COPY_ATOMIC
In-Reply-To: <53962163-d04e-e742-20c1-441ea47ea234@oracle.com>
References: <a8d3f689-dde7-e5ac-c06e-eb3dc1355553@redhat.com>
 <053adeb2-5f00-5cf6-c96e-97d46d37f5a3@oracle.com>
 <7c242604-aac9-243f-5a89-70853351aa12@redhat.com>
 <53962163-d04e-e742-20c1-441ea47ea234@oracle.com>
Message-ID: <d8c3ec8f-5e62-e726-9529-b940552b280c@redhat.com>

On 1/23/20 10:41 PM, David Holmes wrote:
> This does make me wonder whether other lock-free code in the VM needs
> special handling for non-CPU_MULTI_COPY_ATOMIC ??

I don't think we should do exactly that. I would have thoght that
where an algorithm needs anything stronger than acquire/release
consistency we should use explicit sequentially-consistent loads and
stores.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From aph at redhat.com  Fri Jan 24 09:08:38 2020
From: aph at redhat.com (Andrew Haley)
Date: Fri, 24 Jan 2020 09:08:38 +0000
Subject: 8230392: Define AArch64 as MULTI_COPY_ATOMIC
In-Reply-To: <82bdb7e1-2526-75e6-709a-f3eb74678fa9@redhat.com>
References: <a8d3f689-dde7-e5ac-c06e-eb3dc1355553@redhat.com>
 <053adeb2-5f00-5cf6-c96e-97d46d37f5a3@oracle.com>
 <7c242604-aac9-243f-5a89-70853351aa12@redhat.com>
 <53962163-d04e-e742-20c1-441ea47ea234@oracle.com>
 <82bdb7e1-2526-75e6-709a-f3eb74678fa9@redhat.com>
Message-ID: <bcd5854e-e46f-7376-2d59-bead9b4b461e@redhat.com>

On 1/24/20 9:01 AM, Andrew Dinn wrote:
> On 23/01/2020 22:41, David Holmes wrote:
>
>> This does make me wonder whether other lock-free code in the VM needs
>> special handling for non-CPU_MULTI_COPY_ATOMIC ??
>
> The only other place I could find a mention (not a use) of symbol
> CPU_MULTI_COPY_ATOMIC -- excluding the point where it is defined -- was
> in a comment in parse1.cpp. That comment talks about barriers for final
> fields which I believe is handled via definition of
> support_IRIW_for_not_multiple_copy_atomic_cpu.

Mmm, but that's an answer to a different question. I'm wondering how
many places there are where multi-copy atomicity is assumed but not
stated (or even realized by the programmer.)

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From matthias.baesken at sap.com  Fri Jan 24 09:27:19 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 24 Jan 2020 09:27:19 +0000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <e770b014-2562-ed73-b258-d017d6f603a4@oracle.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
 <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>
 <AM6PR02MB4200B721BD0C7728B21E2C4193310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <AM6PR02MB4200BCFF64D8C46340B17A6F930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <d3e3641f-d6ae-979d-d53f-93136da6d62a@oracle.com>
 <AM6PR02MB42000810C8986AA738DB5EA3930F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <e770b014-2562-ed73-b258-d017d6f603a4@oracle.com>
Message-ID: <AM6PR02MB4200F337259362841A4360E3930E0@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hi Erik, thanks for the comments, new webrev :

http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.7/

Best regards, Matthias



> Hello,
> 
> That's better, but there are still some issues.
> 
> flags-cflags.m4
> 
> Code is repeated in both if and else block.
> 
> jdk-options.m4
> 
> The default is now true for all platforms. I would suggest moving the
> s390x conditional down into an elif after the elif for "no".
> 
> LibCommon.gmk
> 
> Please revert whole file.
> 
> /Erik
> 
> On 2020-01-23 05:15, Baesken, Matthias wrote:
> > Hi Erik,  new webrev :
> >
> > http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.6/
> >
> > I moved the settings back into the  m4 files .
> >
> > Best regards, Matthias
> >
> >> Hello Matthias,
> >>
> >> You can keep the setting up of all the flags in flags-cflags.m4 and
> >> flags-ldflags.m4 based on the value of ENABLE_LINKTIME_GC. You can
> also
> >> default the value of this new parameter to true for s390x to keep the
> >> current behavior for that platform. As it is in this patch, the JVM
> >> flags for s390x are setup in configure while the JDK flags are in make,
> >> which gets confusing I think.
> >>
> >> /Erik
> >>
> >>
> >> On 2020-01-22 05:33, Baesken, Matthias wrote:
> >>> Hi Magnus / David,  here is a new webrev :
> >>>
> >>>
> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.4/
> >>>
> >>>
> >>> it supports now a  configure switch  --enable-linktime-gc=yes  that needs
> to
> >> be set  to enable the link time section gc  .
> >>> Exception is linuxs390x  where we already have the  feature enabled
> (and
> >> keep it enabled always for LIB_JVM).
> >>> Best regards, Matthias
> >>>
> >>>
> >>>
> >>> From: Baesken, Matthias
> >>> Sent: Freitag, 17. Januar 2020 12:44
> >>> To: Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>; David
> Holmes
> >> <david.holmes at oracle.com>; 'build-dev at openjdk.java.net' <build-
> >> dev at openjdk.java.net>; 'hotspot-dev at openjdk.java.net' <hotspot-
> >> dev at openjdk.java.net>
> >>> Subject: RE: RFR: 8236714: enable link-time section-gc for linux to
> remove
> >> unused code
> >>>
> >>>
> >>>     *   Matthias: Have a look at some recently added option to get an
> >> indication of the best practice in adding new options. There are some
> ways to
> >> easily make this incorrect
> >>> Hi Magnus, do you have a good/?best practice?  example  (not that I
> catch a
> >> bad one ?? )  ?
> >>> Best regards, Matthias
> >>>

From magnus.ihse.bursie at oracle.com  Fri Jan 24 10:40:13 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Fri, 24 Jan 2020 11:40:13 +0100
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <AM6PR02MB4200F337259362841A4360E3930E0@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
 <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>
 <AM6PR02MB4200B721BD0C7728B21E2C4193310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <AM6PR02MB4200BCFF64D8C46340B17A6F930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <d3e3641f-d6ae-979d-d53f-93136da6d62a@oracle.com>
 <AM6PR02MB42000810C8986AA738DB5EA3930F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <e770b014-2562-ed73-b258-d017d6f603a4@oracle.com>
 <AM6PR02MB4200F337259362841A4360E3930E0@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <d4ee5f0f-ab52-e77e-a44a-b21f7e554743@oracle.com>

On 2020-01-24 10:27, Baesken, Matthias wrote:
> Hi Erik, thanks for the comments, new webrev :
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.7/
Looks good to me. Sorry for not being able to provide an example in a 
timely matter. Good thing you found one yourself. :)

/Magnus
>
> Best regards, Matthias
>
>
>
>> Hello,
>>
>> That's better, but there are still some issues.
>>
>> flags-cflags.m4
>>
>> Code is repeated in both if and else block.
>>
>> jdk-options.m4
>>
>> The default is now true for all platforms. I would suggest moving the
>> s390x conditional down into an elif after the elif for "no".
>>
>> LibCommon.gmk
>>
>> Please revert whole file.
>>
>> /Erik
>>
>> On 2020-01-23 05:15, Baesken, Matthias wrote:
>>> Hi Erik,  new webrev :
>>>
>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.6/
>>>
>>> I moved the settings back into the  m4 files .
>>>
>>> Best regards, Matthias
>>>
>>>> Hello Matthias,
>>>>
>>>> You can keep the setting up of all the flags in flags-cflags.m4 and
>>>> flags-ldflags.m4 based on the value of ENABLE_LINKTIME_GC. You can
>> also
>>>> default the value of this new parameter to true for s390x to keep the
>>>> current behavior for that platform. As it is in this patch, the JVM
>>>> flags for s390x are setup in configure while the JDK flags are in make,
>>>> which gets confusing I think.
>>>>
>>>> /Erik
>>>>
>>>>
>>>> On 2020-01-22 05:33, Baesken, Matthias wrote:
>>>>> Hi Magnus / David,  here is a new webrev :
>>>>>
>>>>>
>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.4/
>>>>>
>>>>>
>>>>> it supports now a  configure switch  --enable-linktime-gc=yes  that needs
>> to
>>>> be set  to enable the link time section gc  .
>>>>> Exception is linuxs390x  where we already have the  feature enabled
>> (and
>>>> keep it enabled always for LIB_JVM).
>>>>> Best regards, Matthias
>>>>>
>>>>>
>>>>>
>>>>> From: Baesken, Matthias
>>>>> Sent: Freitag, 17. Januar 2020 12:44
>>>>> To: Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>; David
>> Holmes
>>>> <david.holmes at oracle.com>; 'build-dev at openjdk.java.net' <build-
>>>> dev at openjdk.java.net>; 'hotspot-dev at openjdk.java.net' <hotspot-
>>>> dev at openjdk.java.net>
>>>>> Subject: RE: RFR: 8236714: enable link-time section-gc for linux to
>> remove
>>>> unused code
>>>>>
>>>>>      *   Matthias: Have a look at some recently added option to get an
>>>> indication of the best practice in adding new options. There are some
>> ways to
>>>> easily make this incorrect
>>>>> Hi Magnus, do you have a good/?best practice?  example  (not that I
>> catch a
>>>> bad one ?? )  ?
>>>>> Best regards, Matthias
>>>>>


From david.holmes at oracle.com  Fri Jan 24 13:19:10 2020
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 24 Jan 2020 23:19:10 +1000
Subject: RFR[L]: 8237767 Field layout computation overhaul
In-Reply-To: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
Message-ID: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com>

Hi Fred,

This is quite a mini-project. :)

On 24/01/2020 1:03 am, Frederic Parain wrote:
> Greetings,
> 
> Please review this change proposing a new code to compute field layouts.
> 
> CR: https://bugs.openjdk.java.net/browse/JDK-8237767
> webrev: http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/
> 
> The CR includes a detailed description of the motivation and the implementation.

Thanks for the detailed discussion. I can't comment on the details. This 
is enabling technology for Valhalla so as long as it is functionally 
correct without significant performance impact, then it seems good.

I have a number of comments, mainly code style issues and typos etc.

Thanks,
David
-----

src/hotspot/share/ci/ciInstanceKlass.cpp

216   assert(self->is_loaded(), "must be loaded to field info");

Seems to be a word missing in the message - "access field info" ?

465     int super_fsize  = super->nonstatic_field_size() * heapOopSize;

Seems to be an unused local variable now.

  466     int super_flen   = super->nof_nonstatic_fields();

Could be folded directly into the assert so we don't call in product.

---

src/hotspot/share/ci/ciInstanceKlass.hpp

  229   bool contains_field_offset(int offset) {
  230     fieldDescriptor fd;
  231     return 
this->get_instanceKlass()->find_field_from_offset(offset, false, &fd);
  232   }

Seems odd to fill in the fieldDescriptor but not use it at all. Suggests 
to me that instanceKlass may need a similar query for 
contains_field_offset. Oh it does - which has the same problem but at 
least the above code could be:

return get_instanceKlass()->contains_field_offset(offset);

Style nit: no need for this->

---

src/hotspot/share/classfile/classFileParser.cpp

3935 OopMapBlocksBuilder::OopMapBlocksBuilder(unsigned int  max_blocks, 
TRAPS) {

Style nit: extra space before max_blocks.

General style issue: when breaking a long line with a method call, the 
new line (containing arguments) should be indented to the opening ( of 
the method call e.g.

3941     nonstatic_oop_maps = NEW_RESOURCE_ARRAY_IN_THREAD(
3942         THREAD, OopMapBlock, max_nonstatic_oop_maps);

should be

3941     nonstatic_oop_maps = NEW_RESOURCE_ARRAY_IN_THREAD(THREAD,
3942                                                       OopMapBlock, 
max_nonstatic_oop_maps);

or alternatively something like:

3941     nonstatic_oop_maps =
3942         NEW_RESOURCE_ARRAY_IN_THREAD(THREAD, OopMapBlock, 
max_nonstatic_oop_maps);

depending on line length and number of lines.

Second example:

3954   assert(nof_blocks && nonstatic_oop_map_count == 0 &&
3955       nof_blocks <= max_nonstatic_oop_maps, "invariant");

should be:

3954   assert(nof_blocks && nonstatic_oop_map_count == 0 &&
3955          nof_blocks <= max_nonstatic_oop_maps, "invariant");

etc. This applies across all files.

--

Style nit:

4137   int super_oop_map_count = (_super_klass == NULL) ? 0 
:_super_klass->nonstatic_oop_map_count();
4138   int max_oop_map_count =
4139       super_oop_map_count +
4140       fac->count[NONSTATIC_OOP];

Given the length of 4137, splitting 4138 over three lines seems odd. At 
most you want two lines.

---

src/hotspot/share/oops/instanceKlass.hpp

You need to be careful with _extra_flags usage if there can be 
concurrently updated bits. At the moment it looks like redefinition is a 
mutable dynamic property, whilst "contended annotations" should be a 
static immutable property - is that right?

  761     return (_extra_flags & _extra_has_contended_annotations);

Style nit: use of implicit boolean, please change to:

  761     return (_extra_flags & _extra_has_contended_annotations != 0);

Ditto for  line 774.

---

src/jdk.internal.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedObjectTypeImpl.java

  714         public int compare(ResolvedJavaField a, ResolvedJavaField b)
  715         {

Style nit: opening brace should be on the same line.

  746                 Arrays.sort(result, new SortByOffset());

No need to create a comparator on every call. Declare a static instance 
in the SortByOffset class. Ditto line 808.

I have to wonder about the performance of this code now that it has to 
do a linear search and post-sort the results? (But that's for the 
compiler folk to raise. :) ).

---

src/hotspot/share/classfile/fieldLayoutBuilder.hpp

  41 // All RawBlock must have a size and a alignment.

s/RawBlocks/RawBlocks/
s/and a/and an/

   42 // exact size of the field expressed in bytes. The alignment is
   43 // the alignment constraint of the field (1 for byte, 2 for short,
   44 // 4 for int, 8 for long, etc.)

I thought we could have stronger alignment constraints than that. For 
example if an int field is subject to atomic updates via CAS then 
depending on platform it may need to be 64-bit aligned? Or are we lucky 
that all (both :) ) of our platforms with hardware cas support direct 
32-bit cas? (Good job we don't support AtomicByte or AtomicShort :) ).

  53 class LayoutRawBlock : public ResourceObj {

Why is the class LayoutRawBlock when you refer to plain RawBlock when 
describing things? Is there some other kind of RawBlock that will come 
in with inline types?

   61     FLATTENED,     // flattened field

Does this have any meaning before inline types come in?

  205 // and two classes with hard coded offsets (java,lang.ref.Reference

s/two/two for/

207 // is that each kind of classes has a different set goals regarding

s/classes/class/

215 //   2 - Field sorting: fields are sorted out according to their

s/out//

220 //   4 - Epilogue: oopmaps are generated, layout information are

s/are/is/

221 //       prepared so other VM components can use them (instance size,

s/them/it/

---

src/hotspot/share/classfile/fieldLayoutBuilder.cpp

  2  * Copyright (c) 2019, 2019, Oracle and/or its affiliates. All 
rights reserved.

Copyright format is incorrect but will be okay when the second 2019 is 
changed to 2020. :)

In FieldLayoutBuilder::epilogue you have a number of calls to 
Thread::current() as well as an implicit call when you use 
ResourceMarks. You should capture the current thread once in a local and 
reuse it.

---

That's it.


> The current version keeps the old code accessible (with a VM flag) in case the
> small changes in computed layouts cause troubles to some applications or tools.
> The goal is to get rid of this old code, preferably as soon as possible.
> 
> Testing tier1-8.
> 
> Thank you,
> 
> Fred
> 

From matthias.baesken at sap.com  Fri Jan 24 13:21:02 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 24 Jan 2020 13:21:02 +0000
Subject: RFR [XXS]: 8237819: s390x - remove unused pd_zero_to_words_large
Message-ID: <AM6PR02MB42001BAF97936A122413D2DC930E0@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello, please review this small change .

On s390x we have an unused function pd_zero_to_words_large in the code that can be removed, this is done here .


Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8237819

http://cr.openjdk.java.net/~mbaesken/webrevs/8237819.0/


Thanks, Matthias

From david.holmes at oracle.com  Fri Jan 24 13:23:49 2020
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 24 Jan 2020 23:23:49 +1000
Subject: 8230392: Define AArch64 as MULTI_COPY_ATOMIC
In-Reply-To: <d8c3ec8f-5e62-e726-9529-b940552b280c@redhat.com>
References: <a8d3f689-dde7-e5ac-c06e-eb3dc1355553@redhat.com>
 <053adeb2-5f00-5cf6-c96e-97d46d37f5a3@oracle.com>
 <7c242604-aac9-243f-5a89-70853351aa12@redhat.com>
 <53962163-d04e-e742-20c1-441ea47ea234@oracle.com>
 <d8c3ec8f-5e62-e726-9529-b940552b280c@redhat.com>
Message-ID: <7b36d33f-e3b9-e572-be38-1dd3e72ef6f5@oracle.com>

On 24/01/2020 7:06 pm, Andrew Haley wrote:
> On 1/23/20 10:41 PM, David Holmes wrote:
>> This does make me wonder whether other lock-free code in the VM needs
>> special handling for non-CPU_MULTI_COPY_ATOMIC ??
> 
> I don't think we should do exactly that. I would have thoght that
> where an algorithm needs anything stronger than acquire/release
> consistency we should use explicit sequentially-consistent loads and
> stores.

What I meant was, "I wonder whether other lock-free code in the VM is 
assuming multi-copy-atomicity?".

David


From matthias.baesken at sap.com  Fri Jan 24 13:36:05 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 24 Jan 2020 13:36:05 +0000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <d4ee5f0f-ab52-e77e-a44a-b21f7e554743@oracle.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
 <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>
 <AM6PR02MB4200B721BD0C7728B21E2C4193310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <AM6PR02MB4200BCFF64D8C46340B17A6F930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <d3e3641f-d6ae-979d-d53f-93136da6d62a@oracle.com>
 <AM6PR02MB42000810C8986AA738DB5EA3930F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <e770b014-2562-ed73-b258-d017d6f603a4@oracle.com>
 <AM6PR02MB4200F337259362841A4360E3930E0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <d4ee5f0f-ab52-e77e-a44a-b21f7e554743@oracle.com>
Message-ID: <AM6PR02MB4200830D38B0FBF3F89347C0930E0@AM6PR02MB4200.eurprd02.prod.outlook.com>

Thanks for the review !
Erik,  are you fine with the latest change too ?

Best regards, Matthias



> On 2020-01-24 10:27, Baesken, Matthias wrote:
> > Hi Erik, thanks for the comments, new webrev :
> >
> > http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.7/
> Looks good to me. Sorry for not being able to provide an example in a
> timely matter. Good thing you found one yourself. :)
> 
> /Magnus
> >
> > Best regards, Matthias


From matthias.baesken at sap.com  Fri Jan 24 13:48:16 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 24 Jan 2020 13:48:16 +0000
Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk
 images
In-Reply-To: <e9372084-eabc-8f57-981f-b95f9795582e@oracle.com>
References: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com>
 <AM6PR02MB42007DB6FF70591EC151174A930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <df71f609-64a6-4997-0ff8-562a81be6134@oracle.com>
 <AM6PR02MB420064F6F67387E6596F0E67930F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <e9372084-eabc-8f57-981f-b95f9795582e@oracle.com>
Message-ID: <AM6PR02MB4200DAC4C1A1D363F8049D05930E0@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hi Erik, maybe we can just rename the configure option to

--enable-stripped-pdbs-for-bundle

AND make the default  = no/false .
Then without setting the configure flag,   everything stays as it is  for JDK vendors/distributors  who do not want  the stripped pdbs in the bundle.

Others who set the flag,  have to "teach"  the developers that the bundle already contains stripped pdbs that need to be replaced by full/"private" pdbs  in case better  symbols/stacks are wanted .
I think that?s a good compromise.

> Now if MS had been kind enough to define a separate file type for the
> stripped pdbs, so that they could live alongside the full pdbs, we
> wouldn't have this issue.

Unfortunately that seems not to work,  I tried to use the stripped pdb-files with another extension  but no success ?  .

An alternative could be to create 2 bundles  when  "--enable-stripped-pdbs-for-bundle" is set to yes , one with one without   stripped pdbs . 

Best regards, Matthias


> 
> On 2020-01-23 00:03, Baesken, Matthias wrote:
> > Hi Erik,  yes true sorry for answering  your comments a bit late .
> >
> >> If a user runs jlink and includes all the jmods we ship with the JDK, the
> result
> >> should be essentially equivalent to the original JDK image. The way the
> >> stripped pdb files are included in the bundles sort of at the last
> >> second of the build here breaks this property.
> > I think we should address this in a separate bug/CR .
> Maybe. I realize that my proposal below is quite a big task. But on the
> other hand, I don't think breaking the relationship between the jmods
> and the distribution bundles is on the table really.
> > Looking for example  into a Linux  build, I see  a lot of debuginfo files  in the
> jdk image (more or less for every shared lib)  .
> > But when looking into the jmods  of that jdk image ,  no debuginfo files are
> in there ( I checked the java.base jmod).
> > So  putting  the  files with debug information into the jmods  seems to be
> something that was not done so far  cross platform  (or is there some build
> switch  for it that I did not check?) .
> > Maybe to keep the jmods as small as possible .
> 
> No, we do not put the debuginfo files in the jmods nor the bundles
> because those are not intended to be shipped to customers. We are
> currently overlaying them into images/jdk in the build output to make
> them available for local debugging. (This is convoluted and I would very
> much like to get away from this practice at some point so that there is
> a 1-1 mapping between images/jdk and bundles/jdk*-bin.tar.gz.) The
> stripped pdb files you are proposing are on the contrary intended for
> shipping to customers (as I understand your proposal) so comparing them
> with the debuginfo files is not relevant.
> 
> Now if MS had been kind enough to define a separate file type for the
> stripped pdbs, so that they could live alongside the full pdbs, we
> wouldn't have this issue. The heart of the problem is that only one set
> of files (either stripped or full) can be present and usable in
> images/jdk at a time. We have 2 main uses for images/jdk.
> 
> 1. Developer running and debugging locally
> 
> 2. Serve as the source for generating the distribution bundles
> 
> We currently have one image serving both of these purposes, which is
> already creating complicated and convoluted build steps. To properly
> solve this I would argue for splitting these two apart into two
> different images for the two different purposes. The build procedure
> would then be, first build the images for distribution, only containing
> what should go into each bundle. Then create the developer jdk image by
> copying files from the distribution images. On Windows, the first image
> would contain the stripped pdbs and when building the second, those
> would get overwritten with the full pdbs.
> 
> Now that I figured out a working model that would solve a bunch of other
> problems as well, I would love to implement it, but I doubt I will have
> time in the near future.
> 
> /Erik
> 
> >
> >> To properly implement this, care will need to be taken to juggle the two
> >> sets of pdb files around, making sure each build and test use case has
> >> the correct one in place where and when it's needed. Quite possibly, we
> >> cannot cover all use cases with one build configuration. Developers
> >> needing the full debug symbols when debugging locally would likely need
> >> to disable the stripped symbols so they get the full symbols everywhere.
> >> Possibly this would need to be the default for debug builds and
> >> configurable for release builds.
> >  From my limited experience , the developers  do not work with the
> bundles (that  would contain now after my patch the stripped pds)  but with
> a "normal" jdk image that  is created my make all.
> >
> > Best regards, Matthias
> >
> >
> >
> >> This still does not address anything in my objection.
> >>
> >> /Erik
> >>
> >> On 2020-01-22 07:46, Baesken, Matthias wrote:
> >>> Hello,  here is an updated version  :
> >>>
> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.3/
> >>>
> >>> this one supports a configure switch  "--enable-stripped-pdbs"    to
> enable
> >> the feature .
> >>> Best regards, Matthias
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Baesken, Matthias
> >>>> Sent: Dienstag, 21. Januar 2020 11:03
> >>>> To: 'David Holmes' <david.holmes at oracle.com>; 'build-
> >>>> dev at openjdk.java.net' <build-dev at openjdk.java.net>; 'hotspot-
> >>>> dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> >>>> Subject: RE: RFR: 8237192: Generate stripped/public pdbs on Windows
> for
> >>>> jdk images
> >>>>
> >>>>
> >>>> Hi David ,  yes I think it makes sense to have a configure option for this .
> >>>> Not everyone would like to have a larger JDK (even it is only a bit
> larger).
> >>>>

From aph at redhat.com  Fri Jan 24 14:00:10 2020
From: aph at redhat.com (Andrew Haley)
Date: Fri, 24 Jan 2020 14:00:10 +0000
Subject: 8230392: Define AArch64 as MULTI_COPY_ATOMIC
In-Reply-To: <7b36d33f-e3b9-e572-be38-1dd3e72ef6f5@oracle.com>
References: <a8d3f689-dde7-e5ac-c06e-eb3dc1355553@redhat.com>
 <053adeb2-5f00-5cf6-c96e-97d46d37f5a3@oracle.com>
 <7c242604-aac9-243f-5a89-70853351aa12@redhat.com>
 <53962163-d04e-e742-20c1-441ea47ea234@oracle.com>
 <d8c3ec8f-5e62-e726-9529-b940552b280c@redhat.com>
 <7b36d33f-e3b9-e572-be38-1dd3e72ef6f5@oracle.com>
Message-ID: <8fa806dc-5bf5-aa82-814b-47cb21bbeca3@redhat.com>

On 1/24/20 1:23 PM, David Holmes wrote:
> On 24/01/2020 7:06 pm, Andrew Haley wrote:
>> On 1/23/20 10:41 PM, David Holmes wrote:
>>> This does make me wonder whether other lock-free code in the VM needs
>>> special handling for non-CPU_MULTI_COPY_ATOMIC ??
>>
>> I don't think we should do exactly that. I would have thoght that
>> where an algorithm needs anything stronger than acquire/release
>> consistency we should use explicit sequentially-consistent loads and
>> stores.
> 
> What I meant was, "I wonder whether other lock-free code in the VM is 
> assuming multi-copy-atomicity?".

Yes, I did understand that. What I meant to say is that even if we could
find it (which we probably can't) we shouldn't special-case it.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From erik.joelsson at oracle.com  Fri Jan 24 14:22:53 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Fri, 24 Jan 2020 06:22:53 -0800
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <AM6PR02MB4200F337259362841A4360E3930E0@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
 <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>
 <AM6PR02MB4200B721BD0C7728B21E2C4193310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <AM6PR02MB4200BCFF64D8C46340B17A6F930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <d3e3641f-d6ae-979d-d53f-93136da6d62a@oracle.com>
 <AM6PR02MB42000810C8986AA738DB5EA3930F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <e770b014-2562-ed73-b258-d017d6f603a4@oracle.com>
 <AM6PR02MB4200F337259362841A4360E3930E0@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <4067336a-cd36-7ec3-97e7-00cd3b8e86ab@oracle.com>

Looks good.

/Erik

On 2020-01-24 01:27, Baesken, Matthias wrote:
> Hi Erik, thanks for the comments, new webrev :
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.7/
>
> Best regards, Matthias
>
>
>
>> Hello,
>>
>> That's better, but there are still some issues.
>>
>> flags-cflags.m4
>>
>> Code is repeated in both if and else block.
>>
>> jdk-options.m4
>>
>> The default is now true for all platforms. I would suggest moving the
>> s390x conditional down into an elif after the elif for "no".
>>
>> LibCommon.gmk
>>
>> Please revert whole file.
>>
>> /Erik
>>
>> On 2020-01-23 05:15, Baesken, Matthias wrote:
>>> Hi Erik,  new webrev :
>>>
>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.6/
>>>
>>> I moved the settings back into the  m4 files .
>>>
>>> Best regards, Matthias
>>>
>>>> Hello Matthias,
>>>>
>>>> You can keep the setting up of all the flags in flags-cflags.m4 and
>>>> flags-ldflags.m4 based on the value of ENABLE_LINKTIME_GC. You can
>> also
>>>> default the value of this new parameter to true for s390x to keep the
>>>> current behavior for that platform. As it is in this patch, the JVM
>>>> flags for s390x are setup in configure while the JDK flags are in make,
>>>> which gets confusing I think.
>>>>
>>>> /Erik
>>>>
>>>>
>>>> On 2020-01-22 05:33, Baesken, Matthias wrote:
>>>>> Hi Magnus / David,  here is a new webrev :
>>>>>
>>>>>
>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.4/
>>>>>
>>>>>
>>>>> it supports now a  configure switch  --enable-linktime-gc=yes  that needs
>> to
>>>> be set  to enable the link time section gc  .
>>>>> Exception is linuxs390x  where we already have the  feature enabled
>> (and
>>>> keep it enabled always for LIB_JVM).
>>>>> Best regards, Matthias
>>>>>
>>>>>
>>>>>
>>>>> From: Baesken, Matthias
>>>>> Sent: Freitag, 17. Januar 2020 12:44
>>>>> To: Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>; David
>> Holmes
>>>> <david.holmes at oracle.com>; 'build-dev at openjdk.java.net' <build-
>>>> dev at openjdk.java.net>; 'hotspot-dev at openjdk.java.net' <hotspot-
>>>> dev at openjdk.java.net>
>>>>> Subject: RE: RFR: 8236714: enable link-time section-gc for linux to
>> remove
>>>> unused code
>>>>>
>>>>>      *   Matthias: Have a look at some recently added option to get an
>>>> indication of the best practice in adding new options. There are some
>> ways to
>>>> easily make this incorrect
>>>>> Hi Magnus, do you have a good/?best practice?  example  (not that I
>> catch a
>>>> bad one ?? )  ?
>>>>> Best regards, Matthias
>>>>>

From martin.doerr at sap.com  Fri Jan 24 15:18:11 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Fri, 24 Jan 2020 15:18:11 +0000
Subject: RFR [XXS]: 8237819: s390x - remove unused pd_zero_to_words_large
In-Reply-To: <AM6PR02MB42001BAF97936A122413D2DC930E0@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB42001BAF97936A122413D2DC930E0@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <AM0PR0202MB32970010D89DAE6F5C13C9E89A0E0@AM0PR0202MB3297.eurprd02.prod.outlook.com>

Hi Matthias,

looks good.

Best regards,
Martin


> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
> Baesken, Matthias
> Sent: Freitag, 24. Januar 2020 14:21
> To: 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: RFR [XXS]: 8237819: s390x - remove unused
> pd_zero_to_words_large
> 
> Hello, please review this small change .
> 
> On s390x we have an unused function pd_zero_to_words_large in the code
> that can be removed, this is done here .
> 
> 
> Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8237819
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8237819.0/
> 
> 
> Thanks, Matthias

From frederic.parain at oracle.com  Fri Jan 24 17:20:15 2020
From: frederic.parain at oracle.com (Frederic Parain)
Date: Fri, 24 Jan 2020 12:20:15 -0500
Subject: RFR[L]: 8237767 Field layout computation overhaul 
In-Reply-To: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
 <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com>
Message-ID: <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com>

Hi David,

> On Jan 24, 2020, at 08:19, David Holmes <david.holmes at oracle.com> wrote:
> 
> Hi Fred,
> 
> This is quite a mini-project. :)

Yes, indeed!

> 
> On 24/01/2020 1:03 am, Frederic Parain wrote:
>> Greetings,
>> Please review this change proposing a new code to compute field layouts.
>> CR: https://bugs.openjdk.java.net/browse/JDK-8237767
>> webrev: http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/
>> The CR includes a detailed description of the motivation and the implementation.
> 
> Thanks for the detailed discussion. I can't comment on the details. This is enabling technology for Valhalla so as long as it is functionally correct without significant performance impact, then it seems good.
> 
> I have a number of comments, mainly code style issues and typos etc.

Thank you for the detailed review, I?ve fixed all issues you mentioned and
my answers to your questions are inlined below.

The new webrev is available here:
http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.05/index.html

Rerunning tier1-3 right now.

> 
> Thanks,
> David
> -----
> 
> src/hotspot/share/ci/ciInstanceKlass.cpp
> 
> 216   assert(self->is_loaded(), "must be loaded to field info");
> 
> Seems to be a word missing in the message - "access field info? ?

Fixed

> 
> 465     int super_fsize  = super->nonstatic_field_size() * heapOopSize;
> 
> Seems to be an unused local variable now.

Removed.

> 
> 466     int super_flen   = super->nof_nonstatic_fields();
> 
> Could be folded directly into the assert so we don't call in product.

Calling not_nonstatic_fields() has the side effect to compute non-static fields,
which is required to get a correct value when reading super->_nonstatic_fields,
so the call is needed even in product builds.

> 
> ---
> 
> src/hotspot/share/ci/ciInstanceKlass.hpp
> 
> 229   bool contains_field_offset(int offset) {
> 230     fieldDescriptor fd;
> 231     return this->get_instanceKlass()->find_field_from_offset(offset, false, &fd);
> 232   }
> 
> Seems odd to fill in the fieldDescriptor but not use it at all. Suggests to me that instanceKlass may need a similar query for contains_field_offset. Oh it does - which has the same problem but at least the above code could be:
> 
> return get_instanceKlass()->contains_field_offset(offset);
> 
> Style nit: no need for this->

Fixed

> 
> ---
> 
> src/hotspot/share/classfile/classFileParser.cpp
> 
> 3935 OopMapBlocksBuilder::OopMapBlocksBuilder(unsigned int  max_blocks, TRAPS) {
> 
> Style nit: extra space before max_blocks.

Fixed

> 
> General style issue: when breaking a long line with a method call, the new line (containing arguments) should be indented to the opening ( of the method call e.g.
> 
> 3941     nonstatic_oop_maps = NEW_RESOURCE_ARRAY_IN_THREAD(
> 3942         THREAD, OopMapBlock, max_nonstatic_oop_maps);
> 
> should be
> 
> 3941     nonstatic_oop_maps = NEW_RESOURCE_ARRAY_IN_THREAD(THREAD,
> 3942                                                       OopMapBlock, max_nonstatic_oop_maps);
> 
> or alternatively something like:
> 
> 3941     nonstatic_oop_maps =
> 3942         NEW_RESOURCE_ARRAY_IN_THREAD(THREAD, OopMapBlock, max_nonstatic_oop_maps);
> 
> depending on line length and number of lines.

Fixed

> 
> Second example:
> 
> 3954   assert(nof_blocks && nonstatic_oop_map_count == 0 &&
> 3955       nof_blocks <= max_nonstatic_oop_maps, "invariant");
> 
> should be:
> 
> 3954   assert(nof_blocks && nonstatic_oop_map_count == 0 &&
> 3955          nof_blocks <= max_nonstatic_oop_maps, "invariant?);

Fixed

> 
> etc. This applies across all files.

Fixes applied lines 4003, 4011, 4041, 4138, 4143.

> 
> --
> 
> Style nit:
> 
> 4137   int super_oop_map_count = (_super_klass == NULL) ? 0 :_super_klass->nonstatic_oop_map_count();
> 4138   int max_oop_map_count =
> 4139       super_oop_map_count +
> 4140       fac->count[NONSTATIC_OOP];
> 
> Given the length of 4137, splitting 4138 over three lines seems odd. At most you want two lines.

Fixed

> 
> ---
> 
> src/hotspot/share/oops/instanceKlass.hpp
> 
> You need to be careful with _extra_flags usage if there can be concurrently updated bits. At the moment it looks like redefinition is a mutable dynamic property, whilst "contended annotations" should be a static immutable property - is that right?

Correct, _has_contended_annotations is a static immutable property, while _is_being_redefined is a mutable one.

> 
> 761     return (_extra_flags & _extra_has_contended_annotations);
> 
> Style nit: use of implicit boolean, please change to:
> 
> 761     return (_extra_flags & _extra_has_contended_annotations != 0);
> 
> Ditto for  line 774.

Fixed

> 
> ---
> 
> src/jdk.internal.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedObjectTypeImpl.java
> 
> 714         public int compare(ResolvedJavaField a, ResolvedJavaField b)
> 715         {
> 
> Style nit: opening brace should be on the same line.

Fixed

> 
> 746                 Arrays.sort(result, new SortByOffset());
> 
> No need to create a comparator on every call. Declare a static instance in the SortByOffset class. Ditto line 808.

Fixed

> 
> I have to wonder about the performance of this code now that it has to do a linear search and post-sort the results? (But that's for the compiler folk to raise. :) ).

The Graal team already had a look at this code (the new field layout had an impact on some Graal unit
tests, so I?ve discussed the issue with them and tests have been fixed), and even if this not an
official review yet, they didn?t express strong objections. I?ll double check with them again before
pushing the code.

> 
> ---
> 
> src/hotspot/share/classfile/fieldLayoutBuilder.hpp
> 
> 41 // All RawBlock must have a size and a alignment.
> 
> s/RawBlocks/RawBlocks/
> s/and a/and an/

Fixed

> 
>  42 // exact size of the field expressed in bytes. The alignment is
>  43 // the alignment constraint of the field (1 for byte, 2 for short,
>  44 // 4 for int, 8 for long, etc.)
> 
> I thought we could have stronger alignment constraints than that. For example if an int field is subject to atomic updates via CAS then depending on platform it may need to be 64-bit aligned? Or are we lucky that all (both :) ) of our platforms with hardware cas support direct 32-bit cas? (Good job we don't support AtomicByte or AtomicShort :) ).

There?s no such stronger alignment constraints in the old field layout code.
Fortunately, the new code allows adding such constraints if needed (it was designed
to support hyper-alignment in the future).

> 
> 53 class LayoutRawBlock : public ResourceObj {
> 
> Why is the class LayoutRawBlock when you refer to plain RawBlock when describing things? Is there some other kind of RawBlock that will come in with inline types?

During a code walk through last week, Lois and Harold suggested that ?RawBlock? was too generic
and could cause name conflicts, so it has been changed to ?LayoutRawBlock?. Unfortunately, the
refactoring didn?t apply automatically to comments, I?ve fixed that.
There?s no other kind of data structures coming in with Valhalla, only new methods and
additional cases in existing methods.

> 
>  61     FLATTENED,     // flattened field
> 
> Does this have any meaning before inline types come in?

Yes, I wanted to reserved the entry in the enum.

> 
> 205 // and two classes with hard coded offsets (java,lang.ref.Reference
> 
> s/two/two for/

Fixed

> 
> 207 // is that each kind of classes has a different set goals regarding
> 
> s/classes/class/

Fixed

> 
> 215 //   2 - Field sorting: fields are sorted out according to their
> 
> s/out//

Fixed

> 
> 220 //   4 - Epilogue: oopmaps are generated, layout information are
> 
> s/are/is/

Fixed

> 
> 221 //       prepared so other VM components can use them (instance size,
> 
> s/them/it/

Fixed

> 
> ---
> 
> src/hotspot/share/classfile/fieldLayoutBuilder.cpp
> 
> 2  * Copyright (c) 2019, 2019, Oracle and/or its affiliates. All rights reserved.
> 
> Copyright format is incorrect but will be okay when the second 2019 is changed to 2020. :)

Fixed with a single year format in  .cpp and .hpp and test files.

> 
> In FieldLayoutBuilder::epilogue you have a number of calls to Thread::current() as well as an implicit call when you use ResourceMarks. You should capture the current thread once in a local and reuse it.

Fixed

> 
> ---
> 
> That's it.

Thank you,

Fred

> 
> 
>> The current version keeps the old code accessible (with a VM flag) in case the
>> small changes in computed layouts cause troubles to some applications or tools.
>> The goal is to get rid of this old code, preferably as soon as possible.
>> Testing tier1-8.
>> Thank you,
>> Fred


From frederic.parain at oracle.com  Fri Jan 24 17:40:44 2020
From: frederic.parain at oracle.com (Frederic Parain)
Date: Fri, 24 Jan 2020 12:40:44 -0500
Subject: RFR[L]: 8237767 Field layout computation overhaul 
In-Reply-To: <aeed1fc3-87f3-f979-fa92-e81d524b4772@redhat.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
 <aeed1fc3-87f3-f979-fa92-e81d524b4772@redhat.com>
Message-ID: <46146AB6-5300-44DF-AA44-3E7C379DCD28@oracle.com>

Hi Aleksey,

Thank you for giving this change a try.

I?ve updated and the two RFRs as duplicates, and I?ve changed the flag name.
The new webrev is here (with updates from David?s review):
http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.05/index.

Regards,

Fred


> On Jan 24, 2020, at 03:46, Aleksey Shipilev <shade at redhat.com> wrote:
> 
> Hi Frederic,
> 
> On 1/23/20 4:03 PM, Frederic Parain wrote:
>> CR: https://bugs.openjdk.java.net/browse/JDK-8237767
>> webrev: http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/
>> 
>> The CR includes a detailed description of the motivation and the implementation.
> 
> I ran the patched jdk through JOL Samples [1], and it works impressively well. I think these RFRs
> can be closed as duplicates now:
>  https://bugs.openjdk.java.net/browse/JDK-8024912
>  https://bugs.openjdk.java.net/browse/JDK-8024912
> 
> Nit: I think the flag should be UseNew*Field*Layout.
> 
> -- 
> Thanks,
> -Aleksey
> 
> [1]
> https://hg.openjdk.java.net/code-tools/jol/file/f1e978b895a0/jol-samples/src/main/java/org/openjdk/jol/samples
> 


From coleen.phillimore at oracle.com  Fri Jan 24 21:17:06 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Fri, 24 Jan 2020 16:17:06 -0500
Subject: RFR[L]: 8237767 Field layout computation overhaul
In-Reply-To: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
Message-ID: <fbcc6b4c-10a9-2201-4594-0155d014e615@oracle.com>


http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/ci/ciInstanceKlass.hpp.udiff.html

    bool contains_field_offset(int offset) {
- return instanceOopDesc::contains_field_offset(offset, 
nonstatic_field_size());
+ fieldDescriptor fd;
+ return this->get_instanceKlass()->find_field_from_offset(offset, 
false, &fd);
    }


This has to go into the VM if it's going to access metadata, with 
VM_ENTRY_MARK, so probably belongs in the cpp file. Also, why doesn't 
this call contains_field_offset() from InstanceKlass? from here:

http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/oops/instanceKlass.cpp.udiff.html

http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/oops/instanceKlass.hpp.udiff.html

+ public:
+ enum {
+ _extra_is_being_redefined = 1 << 0, // used for locking redefinition
+ _extra_has_contended_annotations = 1 << 1 // has @Contended annotation
+ };
+


Why is this enum public?? Also, I think you should make these misc_flags 
and make _flags a u4.? There's already an alignment gap in InstanceKlass 
and we can file an RFE to fix that after this change.

http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/runtime/globals.hpp.udiff.html

I think you should make UseNewLayout a diagnostic flag since we would 
like to remove the old code once it has gotten more testing. The only 
reason someone would use it would be to diagnose some differing behavior.

http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/jdk.internal.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedObjectTypeImpl.java.udiff.html

Someone who knows the compilers should have a peek at this.

http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/classfile/fieldLayoutBuilder.hpp.html

   37 // A RawBlock describes an element of a layout.
   38 // Each field is represented by a RawBlock.
  

Thank you for changing RawBlock to LayoutRawBlock.?? Can you update the 
comments here and the description in the RFE?

   29 #include "classfile/classLoaderData.inline.hpp"


.hpp files shouldn't include .inline.hpp files.? Whatever uses the 
inline code should go into the cpp file (or if critical, add an 
inline.hpp file).

  132   static const int INITIAL_LIST_SIZE;


It's odd to see this without an initialization.

  227 class FieldLayoutBuilder : public ResourceObj {


FieldLayoutBuilder is a StackObj isn't it?

It might be nice to line up the fields, which is part of the coding 
standard that I only somewhat agree on.? Here it would enhance readability.

  264   protected:


Why are these functions "protected"??? I don't see anything that 
inherits from FieldLayoutBuilder that might want to call these functions 
and not the other functions.

http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/classfile/fieldLayoutBuilder.cpp.html

  544             _contended_groups = new (ResourceObj::RESOURCE_AREA, mtInternal) GrowableArray<FieldGroup*>(8);


Growable arrays are default ResourceObj, so I think the 'new' arguments 
aren't needed (there were a couple of these).

  698   OopMapBlocksBuilder* nonstatic_oop_maps =
  699       new OopMapBlocksBuilder(max_oop_map_count, Thread::current());

The code uses OopMapBlocksBuilder as a StackObj, which seems to make 
more sense.? Can you make it StackObj and not have it allocate from 
resourceArea?

  531 void FieldLayoutBuilder::regular_field_sorting(TRAPS) {

  635   regular_field_sorting(CHECK);

None of these functions throw Java class exceptions.? They shouldn't 
pass TRAPS and CHECK which are for Java exceptions.? If you want to 
avoid JavaThread::current, you can pass JavaThread* thread as the first 
parameter.? I think for this change, it might be better to let this:

+ OopMapBlock* oop_maps_copy = NEW_RESOURCE_ARRAY_IN_THREAD(THREAD, 
OopMapBlock,

be

+ OopMapBlock* oop_maps_copy = NEW_RESOURCE_ARRAY(OopMapBlock,


and get it's own Thread::current().? I don't think it saves many 
instructions and definitely not any time.

+void OopMapBlocksBuilder::compact(TRAPS) {


This shouldn't take a thread argument, so shouldn't be declared with 
TRAPS, see above.

  748   if (PrintFieldLayout) {


In some future change, this should be turned into unified logging.

http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/classfile/classFileParser.cpp.udiff.html

Why was AnnotationCollector moved to the header file?? It seems only 
used by classFileParser.cpp?

This is a really nice change and your comments are really good. These 
are all pretty small comments.

Thanks,
Coleen

On 1/23/20 10:03 AM, Frederic Parain wrote:
> Greetings,
>
> Please review this change proposing a new code to compute field layouts.
>
> CR: https://bugs.openjdk.java.net/browse/JDK-8237767
> webrev: http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/
>
> The CR includes a detailed description of the motivation and the implementation.
>
> The current version keeps the old code accessible (with a VM flag) in case the
> small changes in computed layouts cause troubles to some applications or tools.
> The goal is to get rid of this old code, preferably as soon as possible.
>
> Testing tier1-8.
>
> Thank you,
>
> Fred
>


From coleen.phillimore at oracle.com  Fri Jan 24 21:21:04 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Fri, 24 Jan 2020 16:21:04 -0500
Subject: RFR[L]: 8237767 Field layout computation overhaul
In-Reply-To: <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
 <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com>
Message-ID: <63ae9764-9d47-4305-a4f4-aa66df7bb79c@oracle.com>


One comment below on David's comments.

On 1/24/20 8:19 AM, David Holmes wrote:
> Hi Fred,
>
> This is quite a mini-project. :)
>
> On 24/01/2020 1:03 am, Frederic Parain wrote:
>> Greetings,
>>
>> Please review this change proposing a new code to compute field layouts.
>>
>> CR: https://bugs.openjdk.java.net/browse/JDK-8237767
>> webrev: http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/
>>
>> The CR includes a detailed description of the motivation and the 
>> implementation.
>
> Thanks for the detailed discussion. I can't comment on the details. 
> This is enabling technology for Valhalla so as long as it is 
> functionally correct without significant performance impact, then it 
> seems good.
>
> I have a number of comments, mainly code style issues and typos etc.
>
> Thanks,
> David
> -----
>
> src/hotspot/share/ci/ciInstanceKlass.cpp
>
> 216?? assert(self->is_loaded(), "must be loaded to field info");
>
> Seems to be a word missing in the message - "access field info" ?
>
> 465???? int super_fsize? = super->nonstatic_field_size() * heapOopSize;
>
> Seems to be an unused local variable now.
>
> ?466???? int super_flen?? = super->nof_nonstatic_fields();
>
> Could be folded directly into the assert so we don't call in product.
>
> ---
>
> src/hotspot/share/ci/ciInstanceKlass.hpp
>
> ?229?? bool contains_field_offset(int offset) {
> ?230???? fieldDescriptor fd;
> ?231???? return 
> this->get_instanceKlass()->find_field_from_offset(offset, false, &fd);
> ?232?? }
>
> Seems odd to fill in the fieldDescriptor but not use it at all. 
> Suggests to me that instanceKlass may need a similar query for 
> contains_field_offset. Oh it does - which has the same problem but at 
> least the above code could be:
>
> return get_instanceKlass()->contains_field_offset(offset);
>
> Style nit: no need for this->
>
> ---
>
> src/hotspot/share/classfile/classFileParser.cpp
>
> 3935 OopMapBlocksBuilder::OopMapBlocksBuilder(unsigned int max_blocks, 
> TRAPS) {
>
> Style nit: extra space before max_blocks.
>
> General style issue: when breaking a long line with a method call, the 
> new line (containing arguments) should be indented to the opening ( of 
> the method call e.g.
>
> 3941???? nonstatic_oop_maps = NEW_RESOURCE_ARRAY_IN_THREAD(
> 3942???????? THREAD, OopMapBlock, max_nonstatic_oop_maps);
>
> should be
>
> 3941???? nonstatic_oop_maps = NEW_RESOURCE_ARRAY_IN_THREAD(THREAD,
> 3942 OopMapBlock, max_nonstatic_oop_maps);
>
> or alternatively something like:
>
> 3941???? nonstatic_oop_maps =
> 3942???????? NEW_RESOURCE_ARRAY_IN_THREAD(THREAD, OopMapBlock, 
> max_nonstatic_oop_maps);
>
> depending on line length and number of lines.
>
> Second example:
>
> 3954?? assert(nof_blocks && nonstatic_oop_map_count == 0 &&
> 3955?????? nof_blocks <= max_nonstatic_oop_maps, "invariant");
>
> should be:
>
> 3954?? assert(nof_blocks && nonstatic_oop_map_count == 0 &&
> 3955????????? nof_blocks <= max_nonstatic_oop_maps, "invariant");
>
> etc. This applies across all files.
>
> -- 
>
> Style nit:
>
> 4137?? int super_oop_map_count = (_super_klass == NULL) ? 0 
> :_super_klass->nonstatic_oop_map_count();
> 4138?? int max_oop_map_count =
> 4139?????? super_oop_map_count +
> 4140?????? fac->count[NONSTATIC_OOP];
>
> Given the length of 4137, splitting 4138 over three lines seems odd. 
> At most you want two lines.
>
> ---
>
> src/hotspot/share/oops/instanceKlass.hpp
>
> You need to be careful with _extra_flags usage if there can be 
> concurrently updated bits. At the moment it looks like redefinition is 
> a mutable dynamic property, whilst "contended annotations" should be a 
> static immutable property - is that right?
>
> ?761???? return (_extra_flags & _extra_has_contended_annotations);
>
> Style nit: use of implicit boolean, please change to:
>
> ?761???? return (_extra_flags & _extra_has_contended_annotations != 0);
>
> Ditto for? line 774.

I had a look at this because I had originally added _is_being_redefined 
thinking it would be something that multiple threads tested and should 
be volatile and have memory ordering.? So that's why it had it's own 
bool field.? Looking again, all threads access this inside of the 
RedefineClasses_lock so it's okay for this to be a non-atomic bit field.

Coleen
>
> ---
>
> src/jdk.internal.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedObjectTypeImpl.java 
>
>
> ?714???????? public int compare(ResolvedJavaField a, ResolvedJavaField b)
> ?715???????? {
>
> Style nit: opening brace should be on the same line.
>
> ?746???????????????? Arrays.sort(result, new SortByOffset());
>
> No need to create a comparator on every call. Declare a static 
> instance in the SortByOffset class. Ditto line 808.
>
> I have to wonder about the performance of this code now that it has to 
> do a linear search and post-sort the results? (But that's for the 
> compiler folk to raise. :) ).
>
> ---
>
> src/hotspot/share/classfile/fieldLayoutBuilder.hpp
>
> ?41 // All RawBlock must have a size and a alignment.
>
> s/RawBlocks/RawBlocks/
> s/and a/and an/
>
> ? 42 // exact size of the field expressed in bytes. The alignment is
> ? 43 // the alignment constraint of the field (1 for byte, 2 for short,
> ? 44 // 4 for int, 8 for long, etc.)
>
> I thought we could have stronger alignment constraints than that. For 
> example if an int field is subject to atomic updates via CAS then 
> depending on platform it may need to be 64-bit aligned? Or are we 
> lucky that all (both :) ) of our platforms with hardware cas support 
> direct 32-bit cas? (Good job we don't support AtomicByte or 
> AtomicShort :) ).
>
> ?53 class LayoutRawBlock : public ResourceObj {
>
> Why is the class LayoutRawBlock when you refer to plain RawBlock when 
> describing things? Is there some other kind of RawBlock that will come 
> in with inline types?
>
> ? 61???? FLATTENED,???? // flattened field
>
> Does this have any meaning before inline types come in?
>
> ?205 // and two classes with hard coded offsets (java,lang.ref.Reference
>
> s/two/two for/
>
> 207 // is that each kind of classes has a different set goals regarding
>
> s/classes/class/
>
> 215 //?? 2 - Field sorting: fields are sorted out according to their
>
> s/out//
>
> 220 //?? 4 - Epilogue: oopmaps are generated, layout information are
>
> s/are/is/
>
> 221 //?????? prepared so other VM components can use them (instance size,
>
> s/them/it/
>
> ---
>
> src/hotspot/share/classfile/fieldLayoutBuilder.cpp
>
> ?2? * Copyright (c) 2019, 2019, Oracle and/or its affiliates. All 
> rights reserved.
>
> Copyright format is incorrect but will be okay when the second 2019 is 
> changed to 2020. :)
>
> In FieldLayoutBuilder::epilogue you have a number of calls to 
> Thread::current() as well as an implicit call when you use 
> ResourceMarks. You should capture the current thread once in a local 
> and reuse it.
>
> ---
>
> That's it.
>
>
>> The current version keeps the old code accessible (with a VM flag) in 
>> case the
>> small changes in computed layouts cause troubles to some applications 
>> or tools.
>> The goal is to get rid of this old code, preferably as soon as possible.
>>
>> Testing tier1-8.
>>
>> Thank you,
>>
>> Fred
>>


From david.holmes at oracle.com  Fri Jan 24 22:56:07 2020
From: david.holmes at oracle.com (David Holmes)
Date: Sat, 25 Jan 2020 08:56:07 +1000
Subject: Fwd: New candidate JEP: 371: Hidden Classes
In-Reply-To: <20200124210734.D1173315210@eggemoggin.niobe.net>
References: <20200124210734.D1173315210@eggemoggin.niobe.net>
Message-ID: <f8d2a865-9564-e8ca-8a5f-6c37b89a2133@oracle.com>

FYI.

Cheers,
David
-------------- next part --------------
An embedded message was scrubbed...
From: mark.reinhold at oracle.com
Subject: New candidate JEP: 371: Hidden Classes
Date: Fri, 24 Jan 2020 13:07:34 -0800 (PST)
Size: 3883
URL: <https://mail.openjdk.java.net/pipermail/hotspot-dev/attachments/20200125/adaa5d80/HiddenClasses.eml>

From shade at redhat.com  Sat Jan 25 09:46:59 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Sat, 25 Jan 2020 10:46:59 +0100
Subject: RFR (XS/T) 8237847: Zero builds fail after JDK-8237637 (Remove
 dubious type conversions from oop)
Message-ID: <05de70b1-33fb-6fa2-f78a-c491f9122e9e@redhat.com>

Bug:
  https://bugs.openjdk.java.net/browse/JDK-8237847

Seems like JDK-8237637 missed the spot there.

Fix:

diff -r d3cdf4b2b45b src/hotspot/share/interpreter/bytecodeInterpreter.cpp
--- a/src/hotspot/share/interpreter/bytecodeInterpreter.cpp     Fri Jan 24 21:57:19 2020 +0000
+++ b/src/hotspot/share/interpreter/bytecodeInterpreter.cpp     Sat Jan 25 10:18:22 2020 +0100
@@ -2171,11 +2171,11 @@
             }
 #endif
             if (result != NULL) {
               // Initialize object (if nonzero size and need) and then the header
               if (need_zero ) {
-                HeapWord* to_zero = (HeapWord*) result + sizeof(oopDesc) / oopSize;
+                HeapWord* to_zero = cast_from_oop<HeapWord*>(result) + sizeof(oopDesc) / oopSize;
                 obj_size -= sizeof(oopDesc) / oopSize;
                 if (obj_size > 0 ) {
                   memset(to_zero, 0, obj_size * HeapWordSize);
                 }
               }


Testing: Linux x86_64 zero {fastdebug,release} build

-- 
Thanks,
-Aleksey


From daniel.daugherty at oracle.com  Sat Jan 25 13:52:52 2020
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Sat, 25 Jan 2020 08:52:52 -0500
Subject: RFR (XS/T) 8237847: Zero builds fail after JDK-8237637 (Remove
 dubious type conversions from oop)
In-Reply-To: <05de70b1-33fb-6fa2-f78a-c491f9122e9e@redhat.com>
References: <05de70b1-33fb-6fa2-f78a-c491f9122e9e@redhat.com>
Message-ID: <26dd305e-a605-042b-521d-9e7640acc059@oracle.com>

Thumbs up. I agree that this is a trivial change.

Dan


On 1/25/20 4:46 AM, Aleksey Shipilev wrote:
> Bug:
>    https://bugs.openjdk.java.net/browse/JDK-8237847
>
> Seems like JDK-8237637 missed the spot there.
>
> Fix:
>
> diff -r d3cdf4b2b45b src/hotspot/share/interpreter/bytecodeInterpreter.cpp
> --- a/src/hotspot/share/interpreter/bytecodeInterpreter.cpp     Fri Jan 24 21:57:19 2020 +0000
> +++ b/src/hotspot/share/interpreter/bytecodeInterpreter.cpp     Sat Jan 25 10:18:22 2020 +0100
> @@ -2171,11 +2171,11 @@
>               }
>   #endif
>               if (result != NULL) {
>                 // Initialize object (if nonzero size and need) and then the header
>                 if (need_zero ) {
> -                HeapWord* to_zero = (HeapWord*) result + sizeof(oopDesc) / oopSize;
> +                HeapWord* to_zero = cast_from_oop<HeapWord*>(result) + sizeof(oopDesc) / oopSize;
>                   obj_size -= sizeof(oopDesc) / oopSize;
>                   if (obj_size > 0 ) {
>                     memset(to_zero, 0, obj_size * HeapWordSize);
>                   }
>                 }
>
>
> Testing: Linux x86_64 zero {fastdebug,release} build
>


From christoph.langer at sap.com  Sat Jan 25 14:16:15 2020
From: christoph.langer at sap.com (Langer, Christoph)
Date: Sat, 25 Jan 2020 14:16:15 +0000
Subject: RFR [XXS]: 8237819: s390x - remove unused pd_zero_to_words_large
In-Reply-To: <AM6PR02MB42001BAF97936A122413D2DC930E0@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB42001BAF97936A122413D2DC930E0@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <AM0PR02MB57143399D575069CE7A10EBE8A090@AM0PR02MB5714.eurprd02.prod.outlook.com>

Hi Matthias,

looks good. Small nit: The copyright year for SAP should read "Copyright (c) 2016, 2020 SAP SE. All rights reserved.". There's no comma after 2020 as opposed to the Oracle line.

Best regards
Christoph

> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
> Baesken, Matthias
> Sent: Freitag, 24. Januar 2020 14:21
> To: 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: RFR [XXS]: 8237819: s390x - remove unused
> pd_zero_to_words_large
> 
> Hello, please review this small change .
> 
> On s390x we have an unused function pd_zero_to_words_large in the code
> that can be removed, this is done here .
> 
> 
> Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8237819
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8237819.0/
> 
> 
> Thanks, Matthias

From shade at redhat.com  Sun Jan 26 16:09:48 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Sun, 26 Jan 2020 17:09:48 +0100
Subject: RFR (XS/T) 8237847: Zero builds fail after JDK-8237637 (Remove
 dubious type conversions from oop)
In-Reply-To: <26dd305e-a605-042b-521d-9e7640acc059@oracle.com>
References: <05de70b1-33fb-6fa2-f78a-c491f9122e9e@redhat.com>
 <26dd305e-a605-042b-521d-9e7640acc059@oracle.com>
Message-ID: <1965888e-aeab-569a-c7cf-01c2a011329d@redhat.com>

On 1/25/20 2:52 PM, Daniel D. Daugherty wrote:
> Thumbs up. I agree that this is a trivial change.

Thanks. Pushed!

-- 
-Aleksey


From matthias.baesken at sap.com  Mon Jan 27 07:58:24 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Mon, 27 Jan 2020 07:58:24 +0000
Subject: RFR [XXS]: 8237819: s390x - remove unused pd_zero_to_words_large
In-Reply-To: <AM0PR02MB57143399D575069CE7A10EBE8A090@AM0PR02MB5714.eurprd02.prod.outlook.com>
References: <AM6PR02MB42001BAF97936A122413D2DC930E0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <AM0PR02MB57143399D575069CE7A10EBE8A090@AM0PR02MB5714.eurprd02.prod.outlook.com>
Message-ID: <AM0PR02MB422768B77C57142E6AF31CD7930B0@AM0PR02MB4227.eurprd02.prod.outlook.com>

Thanks for the review !
I removed the comma .

Best regards, Matthias


> -----Original Message-----
> From: Langer, Christoph <christoph.langer at sap.com>
> Sent: Samstag, 25. Januar 2020 15:16
> To: Baesken, Matthias <matthias.baesken at sap.com>; 'hotspot-
> dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: RE: RFR [XXS]: 8237819: s390x - remove unused
> pd_zero_to_words_large
> 
> Hi Matthias,
> 
> looks good. Small nit: The copyright year for SAP should read "Copyright (c)
> 2016, 2020 SAP SE. All rights reserved.". There's no comma after 2020 as
> opposed to the Oracle line.
> 
> Best regards
> Christoph


From matthias.baesken at sap.com  Mon Jan 27 12:47:12 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Mon, 27 Jan 2020 12:47:12 +0000
Subject: RFR: 8237830:  support O_CLOEXEC in os::open on other OS than Linux
Message-ID: <AM0PR02MB422762F68A242BBC5D9AC6EC930B0@AM0PR02MB4227.eurprd02.prod.outlook.com>

Hello, please review my change that adds O_CLOEXEC  in HS os::open  for AIX and BSD/macOS .

Change

https://bugs.openjdk.java.net/browse/JDK-8043780

introduced the usage of O_CLOEXEC in os::open on Linux.

We might do the same on other UNIX OS for example BSD/Mac and AIX (AIX 7.X  has  O_CLOEXEC).


Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8237830

http://cr.openjdk.java.net/~mbaesken/webrevs/8237830.0/

Thanks, Matthias



From david.lloyd at redhat.com  Mon Jan 27 13:56:57 2020
From: david.lloyd at redhat.com (David Lloyd)
Date: Mon, 27 Jan 2020 07:56:57 -0600
Subject: New candidate JEP: 371: Hidden Classes
In-Reply-To: <f8d2a865-9564-e8ca-8a5f-6c37b89a2133@oracle.com>
References: <20200124210734.D1173315210@eggemoggin.niobe.net>
 <f8d2a865-9564-e8ca-8a5f-6c37b89a2133@oracle.com>
Message-ID: <CANghgrTDmCOEvhL=q75zcQAp1NSNu7BCoYT4Q9-F9nmbUY9UnA@mail.gmail.com>

On Fri, Jan 24, 2020 at 4:56 PM David Holmes <david.holmes at oracle.com> wrote:
>
> FYI.

I'm a bit curious about this:

> To invoke private nestmate instance methods from code in the hidden class, use invokevirtual or invokeinterface instead of invokespecial. Generated bytecode that uses invokespecial to invoke a private nestmate instance method will fail verification. invokespecial should only be used to invoke private nestmate constructors.

This seems pretty unusual.  Don't we normally use invokespecial for
private method invocations?  What is the reasoning for this?

-- 
- DML


From forax at univ-mlv.fr  Mon Jan 27 14:31:38 2020
From: forax at univ-mlv.fr (Remi Forax)
Date: Mon, 27 Jan 2020 15:31:38 +0100 (CET)
Subject: New candidate JEP: 371: Hidden Classes
In-Reply-To: <CANghgrTDmCOEvhL=q75zcQAp1NSNu7BCoYT4Q9-F9nmbUY9UnA@mail.gmail.com>
References: <20200124210734.D1173315210@eggemoggin.niobe.net>
 <f8d2a865-9564-e8ca-8a5f-6c37b89a2133@oracle.com>
 <CANghgrTDmCOEvhL=q75zcQAp1NSNu7BCoYT4Q9-F9nmbUY9UnA@mail.gmail.com>
Message-ID: <1721041649.427206.1580135498603.JavaMail.zimbra@u-pem.fr>

----- Mail original -----
> De: "David Lloyd" <david.lloyd at redhat.com>
> ?: "David Holmes" <david.holmes at oracle.com>
> Cc: "hotspot-dev" <hotspot-dev at openjdk.java.net>, "core-libs-dev" <core-libs-dev at openjdk.java.net>
> Envoy?: Lundi 27 Janvier 2020 14:56:57
> Objet: Re: New candidate JEP: 371: Hidden Classes

> On Fri, Jan 24, 2020 at 4:56 PM David Holmes <david.holmes at oracle.com> wrote:
>>
>> FYI.
> 
> I'm a bit curious about this:
> 
>> To invoke private nestmate instance methods from code in the hidden class, use
>> invokevirtual or invokeinterface instead of invokespecial. Generated bytecode
>> that uses invokespecial to invoke a private nestmate instance method will fail
>> verification. invokespecial should only be used to invoke private nestmate
>> constructors.
> 
> This seems pretty unusual.  Don't we normally use invokespecial for
> private method invocations?  What is the reasoning for this?

It's the new usual since 11 and the introduction of nestmates,
before javac was generating an accessor method (the access$000 methods).

class Foo {
  private void privateMethod() {
  }

  public static class Bar {
    public void m() {
      new Foo().privateMethod();
    }
  }

  public static void main(String[] args) {
    new Bar().m();
  }
}

The bytecode of 'm()' is

public void m();
    Code:
       0: new           #7                  // class Foo
       3: dup
       4: invokespecial #9                  // Method Foo."<init>":()V
       7: invokevirtual #10                 // Method Foo.privateMethod:()V
      10: return

When calling a method (this is different for constructor),  invokespecial semantics has no been extended to be able to call private method of classes of the same nest, only invokevirtual/invokeinterface and invokestatic have that capability. 

The reason is that invokespecial semantics is already a patchwork of several semantics, you can call a constructor, a private method of the same class and a method of the super class and each semantics has it's own verification rule.
Given that invokevirtual already had to be extended, adding a new mode for invokespecial was not necessary.

As a trivia, if you take a look to the dex file used by Android, the opcode invokespecial doesn't exist, instead you have several other opcodes, one for each semantics.

> 
> --
> - DML

R?mi

From mandy.chung at oracle.com  Mon Jan 27 18:29:51 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Mon, 27 Jan 2020 10:29:51 -0800
Subject: New candidate JEP: 371: Hidden Classes
In-Reply-To: <1721041649.427206.1580135498603.JavaMail.zimbra@u-pem.fr>
References: <20200124210734.D1173315210@eggemoggin.niobe.net>
 <f8d2a865-9564-e8ca-8a5f-6c37b89a2133@oracle.com>
 <CANghgrTDmCOEvhL=q75zcQAp1NSNu7BCoYT4Q9-F9nmbUY9UnA@mail.gmail.com>
 <1721041649.427206.1580135498603.JavaMail.zimbra@u-pem.fr>
Message-ID: <76d55980-1e9e-5f3c-1340-6784448496c1@oracle.com>



On 1/27/20 6:31 AM, Remi Forax wrote:
>> I'm a bit curious about this:
>>
>>> To invoke private nestmate instance methods from code in the hidden class, use
>>> invokevirtual or invokeinterface instead of invokespecial. Generated bytecode
>>> that uses invokespecial to invoke a private nestmate instance method will fail
>>> verification. invokespecial should only be used to invoke private nestmate
>>> constructors.
>> This seems pretty unusual.  Don't we normally use invokespecial for
>> private method invocations?  What is the reasoning for this?
> It's the new usual since 11 and the introduction of nestmates,
> before javac was generating an accessor method (the access$000 methods).
>
> :
>

As Remi said, this feature has been there since 11.? You can reference 
JEP 181 for more details.

Mandy
[1] https://openjdk.java.net/jeps/181


From david.lloyd at redhat.com  Mon Jan 27 19:58:25 2020
From: david.lloyd at redhat.com (David Lloyd)
Date: Mon, 27 Jan 2020 13:58:25 -0600
Subject: New candidate JEP: 371: Hidden Classes
In-Reply-To: <76d55980-1e9e-5f3c-1340-6784448496c1@oracle.com>
References: <20200124210734.D1173315210@eggemoggin.niobe.net>
 <f8d2a865-9564-e8ca-8a5f-6c37b89a2133@oracle.com>
 <CANghgrTDmCOEvhL=q75zcQAp1NSNu7BCoYT4Q9-F9nmbUY9UnA@mail.gmail.com>
 <1721041649.427206.1580135498603.JavaMail.zimbra@u-pem.fr>
 <76d55980-1e9e-5f3c-1340-6784448496c1@oracle.com>
Message-ID: <CANghgrRXPfzLv-Ym+9M-OWE4bqND3CjfvhBZvbnmJKceETuLUg@mail.gmail.com>

On Mon, Jan 27, 2020 at 12:30 PM Mandy Chung <mandy.chung at oracle.com> wrote:
>
>
>
> On 1/27/20 6:31 AM, Remi Forax wrote:
>
> I'm a bit curious about this:
>
> To invoke private nestmate instance methods from code in the hidden class, use
> invokevirtual or invokeinterface instead of invokespecial. Generated bytecode
> that uses invokespecial to invoke a private nestmate instance method will fail
> verification. invokespecial should only be used to invoke private nestmate
> constructors.
>
> This seems pretty unusual.  Don't we normally use invokespecial for
> private method invocations?  What is the reasoning for this?
>
> It's the new usual since 11 and the introduction of nestmates,
> before javac was generating an accessor method (the access$000 methods).
>
> :
>
>
>
> As Remi said, this feature has been there since 11.  You can reference JEP 181 for more details.
>
> Mandy
> [1] https://openjdk.java.net/jeps/181

This is very helpful, thanks!
-- 
- DML


From david.holmes at oracle.com  Tue Jan 28 04:26:44 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 28 Jan 2020 14:26:44 +1000
Subject: RFR: 8237830: support O_CLOEXEC in os::open on other OS than Linux
In-Reply-To: <AM0PR02MB422762F68A242BBC5D9AC6EC930B0@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB422762F68A242BBC5D9AC6EC930B0@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <acfbeabb-6f51-9707-2136-5ae275fafde9@oracle.com>

Hi Matthias,
On 27/01/2020 10:47 pm, Baesken, Matthias wrote:
> Hello, please review my change that adds O_CLOEXEC  in HS os::open  for AIX and BSD/macOS .

Do we know when O_CLOEXEC support was added to those platforms? Do we 
know that the flag is simply ignored if used on a platform prior to that 
support existing?

Thanks,
David

> Change
> 
> https://bugs.openjdk.java.net/browse/JDK-8043780
> 
> introduced the usage of O_CLOEXEC in os::open on Linux.
> 
> We might do the same on other UNIX OS for example BSD/Mac and AIX (AIX 7.X  has  O_CLOEXEC).
> 
> 
> Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8237830
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8237830.0/
> 
> Thanks, Matthias
> 
> 

From matthias.baesken at sap.com  Tue Jan 28 08:22:41 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 28 Jan 2020 08:22:41 +0000
Subject: RFR: 8223699: cleanup perfMemory_aix.cpp O_NOFOLLOW coding on aix
Message-ID: <AM0PR02MB42277D40B0E81BD41C6B4A05930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>

Hello, please review  this AIX specific cleanup.

In perfMemory_aix.cpp we still have O_NOFOLLOW related fallback coding for AIX version < 7.X.
This is no longer needed, because we do not support any more AIX versions < 7.X in coming jdk13, so cleanup is possible.


Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8223699

http://cr.openjdk.java.net/~mbaesken/webrevs/8223699.1/


Thanks, Matthias

From matthias.baesken at sap.com  Tue Jan 28 08:52:38 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 28 Jan 2020 08:52:38 +0000
Subject: RFR: 8237830: support O_CLOEXEC in os::open on other OS than Linux
In-Reply-To: <acfbeabb-6f51-9707-2136-5ae275fafde9@oracle.com>
References: <AM0PR02MB422762F68A242BBC5D9AC6EC930B0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <acfbeabb-6f51-9707-2136-5ae275fafde9@oracle.com>
Message-ID: <AM0PR02MB42270590FDE0D2EB509027BE930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>

Hi David , on AIX  I see  O_CLOEXEC  since   AIX 7.1.  Seems it was not supported in 6.1, at least I cannot find it there .

Regarding macOS,   this thread 

https://groups.google.com/forum/#!topic/golang-dev/7mmT7o9GYb4

claims it is there since  10.7 .
Maybe someone can provide more detailed information ?

Best regards, Matthias



> 
> Hi Matthias,
> On 27/01/2020 10:47 pm, Baesken, Matthias wrote:
> > Hello, please review my change that adds O_CLOEXEC  in HS os::open  for
> AIX and BSD/macOS .
> 
> Do we know when O_CLOEXEC support was added to those platforms? Do
> we
> know that the flag is simply ignored if used on a platform prior to that
> support existing?
> 
> Thanks,
> David
> 
> > Change
> >
> > https://bugs.openjdk.java.net/browse/JDK-8043780
> >
> > introduced the usage of O_CLOEXEC in os::open on Linux.
> >
> > We might do the same on other UNIX OS for example BSD/Mac and AIX
> (AIX 7.X  has  O_CLOEXEC).
> >
> >
> > Bug/webrev :
> >
> > https://bugs.openjdk.java.net/browse/JDK-8237830
> >
> > http://cr.openjdk.java.net/~mbaesken/webrevs/8237830.0/
> >
> > Thanks, Matthias
> >
> >

From martin.doerr at sap.com  Tue Jan 28 10:34:40 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Tue, 28 Jan 2020 10:34:40 +0000
Subject: RFR: 8223699: cleanup perfMemory_aix.cpp O_NOFOLLOW coding on aix
In-Reply-To: <AM0PR02MB42277D40B0E81BD41C6B4A05930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB42277D40B0E81BD41C6B4A05930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <AM0PR0202MB3297A64BF6DD35373D6CAC1E9A0A0@AM0PR0202MB3297.eurprd02.prod.outlook.com>

Hi Matthias,

I agree.

According to
https://wiki.openjdk.java.net/display/Build/Supported+Build+Platforms
we require AIX 7.1 for JDK11 and 7.2 since JDK13.

Early version of these AIX version were affected by the following O_NOFOLLOW issue:
https://bugs.openjdk.java.net/browse/JDK-8223701

But the affected Tech Levels have already reached "End of Service Pack Support":
https://www.ibm.com/support/pages/aix-support-lifecycle-information
Only AIX 7.1 TL5 and AIX 7.2 TL2 (and later) are still supported.

Change looks good.

Best regards,
Martin


> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
> Baesken, Matthias
> Sent: Dienstag, 28. Januar 2020 09:23
> To: 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: RFR: 8223699: cleanup perfMemory_aix.cpp O_NOFOLLOW coding
> on aix
> 
> Hello, please review  this AIX specific cleanup.
> 
> In perfMemory_aix.cpp we still have O_NOFOLLOW related fallback coding
> for AIX version < 7.X.
> This is no longer needed, because we do not support any more AIX versions
> < 7.X in coming jdk13, so cleanup is possible.
> 
> 
> Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8223699
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8223699.1/
> 
> 
> Thanks, Matthias

From matthias.baesken at sap.com  Tue Jan 28 11:31:44 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 28 Jan 2020 11:31:44 +0000
Subject: RFR: 8237830: support O_CLOEXEC in os::open on other OS than Linux
References: <AM0PR02MB422762F68A242BBC5D9AC6EC930B0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <acfbeabb-6f51-9707-2136-5ae275fafde9@oracle.com> 
Message-ID: <AM0PR02MB4227ED513E82E9E24FE88523930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>

New webrev :

http://cr.openjdk.java.net/~mbaesken/webrevs/8237830.1/

(one 'c' got lost somehow in the older webrev)

Best regards, Matthias


> 
> Hi David , on AIX  I see  O_CLOEXEC  since   AIX 7.1.  Seems it was not
> supported in 6.1, at least I cannot find it there .
> 
> Regarding macOS,   this thread
> 
> https://groups.google.com/forum/#!topic/golang-dev/7mmT7o9GYb4
> 
> claims it is there since  10.7 .
> Maybe someone can provide more detailed information ?
> 
> Best regards, Matthias
> 
> 
> 


From matthias.baesken at sap.com  Tue Jan 28 12:51:46 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 28 Jan 2020 12:51:46 +0000
Subject: fopen vs. os::fopen and automatic closing of the file on exec
Message-ID: <AM0PR02MB42275083AC9BA465ADCF75B4930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>

Hello, I noticed   while looking at   https://bugs.openjdk.java.net/browse/JDK-8237830   ( support O_CLOEXEC in os::open on other OS than Linux )
that os::fopen  also  has  some support  for setting  FD_CLOEXEC / O_CLOEXEC  on the file opened .
See :

1253// This function is a proxy to fopen, it tries to add a non standard flag ('e' or 'N')
1254// that ensures automatic closing of the file on exec. If it can not find support in
1255// the underlying c library, it will make an extra system call (fcntl) to ensure automatic
1256// closing of the file on exec.
1257FILE* os::fopen(const char* path, const char* mode) {
1258  char modified_mode[20];
1259  assert(strlen(mode) + 1 < sizeof(modified_mode), "mode chars plus one extra must fit in buffer");
1260  sprintf(modified_mode, "%s" LINUX_ONLY("e") BSD_ONLY("e") WINDOWS_ONLY("N"), mode);
1261  FILE* file = ::fopen(path, modified_mode);
1262
1263#if !(defined LINUX || defined BSD || defined _WINDOWS)
1264  // assume fcntl FD_CLOEXEC support as a backup solution when 'e' or 'N'
1265  // is not supported as mode in fopen
1266  if (file != NULL) {
1267    int fd = fileno(file);
1268    if (fd != -1) {
1269      int fd_flags = fcntl(fd, F_GETFD);
1270      if (fd_flags != -1) {
1271        fcntl(fd, F_SETFD, fd_flags | FD_CLOEXEC);
1272      }
1273    }
1274  }
1275#endif

However some questions arise here :

  1.  Usage :        os::fopen   is  only used sometimes  in HS code ,  should most of the calls to  fopen be adjusted to os::fopen   (see list  below )
  2.  ::fopen vs. ::fcntl     :    is  os_linux  os::open   we try to  set  the  "closing of the file on exec"   flag  when calling ::open  but we later check  that it really  worked so we seem not to trust it fully ;
Should this be done here too  for Linux ? Or is that  checking in os_linux os::open  these days not needed any more ?

Best regards, Matthias



Grep showed  me these fopen-calls in HS, a lot go not to os::fopen ?

cpu/aarch64/vm_version_aarch64.cpp:171:  if (FILE *f = fopen("/proc/cpuinfo", "r")) {
cpu/ppc/vm_version_ppc.cpp:412:  FILE* fp = fopen(info_file, "r");
os/aix/os_aix.cpp:3756:  // - might cause an fopen in the subprocess to fail on a system
os/aix/os_perf_aix.cpp:243:  if ((f = fopen(procfile, "r")) == NULL) {
os/aix/os_perf_aix.cpp:666:  if ((fp = fopen(buffer, "r")) != NULL) {
os/aix/os_perf_aix.cpp:694:  if ((fp = fopen(buffer, "r")) != NULL) {
os/bsd/os_bsd.cpp:3479:  // - might cause an fopen in the subprocess to fail on a system
os/linux/decoder_linux.cpp:61:  FILE* file = fopen(filepath, "r");
os/linux/os_linux.cpp:257:  if ((fh = fopen("/proc/stat", "r")) == NULL) {
os/linux/os_linux.cpp:364:    FILE *fp = fopen(fname, "r");
os/linux/os_linux.cpp:1106:  FILE *fp = fopen("/proc/self/maps", "r");
os/linux/os_linux.cpp:1218:    fp = fopen("/proc/self/stat", "r");
os/linux/os_linux.cpp:2075:  if ((procmapsFile = fopen("/proc/self/maps", "r")) != NULL) {
os/linux/os_linux.cpp:2244:  FILE* fp = fopen(file, "r");
os/linux/os_linux.cpp:2458:  FILE *fp = fopen("/proc/cpuinfo", "r");
os/linux/os_linux.cpp:2515:  FILE* fp = fopen("/proc/cpuinfo", "r");
os/linux/os_linux.cpp:3645:    FILE *fp = fopen("/proc/self/maps", "r");
os/linux/os_linux.cpp:3689:  if ((f = fopen("/proc/self/coredump_filter", "r+")) == NULL) {
os/linux/os_linux.cpp:3741:  FILE *fp = fopen("/proc/meminfo", "r");
os/linux/os_linux.cpp:5572:  // - might cause an fopen in the subprocess to fail on a system
os/linux/os_linux.cpp:5797:  fp = fopen(proc_name, "r");
os/linux/os_perf_linux.cpp:238:  if ((f = fopen(procfile, "r")) == NULL) {
os/linux/os_perf_linux.cpp:275:  if ((f = fopen("/proc/stat", "r")) == NULL) {
os/linux/os_perf_linux.cpp:726:  if ((fp = fopen(buffer, "r")) != NULL) {
os/linux/os_perf_linux.cpp:754:  if ((fp = fopen(buffer, "r")) != NULL) {
os/linux/perfMemory_linux.cpp:659:  FILE *fp = fopen(fname, "r");
os/linux/gc/z/zPhysicalMemoryBacking_linux.cpp:312:  FILE* const file = fopen(filename, "r");
os/linux/gc/z/zMountPoint_linux.cpp:72:  FILE* fd = fopen(PROC_SELF_MOUNTINFO, "r");
os/linux/cgroupSubsystem_linux.cpp:66:  cgroups = fopen("/proc/cgroups", "r");
os/linux/cgroupSubsystem_linux.cpp:121:  cgroup = fopen("/proc/self/cgroup", "r");
os/linux/cgroupSubsystem_linux.cpp:170:    mntinfo = fopen("/proc/self/mountinfo", "r");
os/linux/cgroupSubsystem_linux.cpp:224:  mntinfo = fopen("/proc/self/mountinfo", "r");
os/linux/cgroupSubsystem_linux.hpp:96:  fp = fopen(file, "r");
os/solaris/os_perf_solaris.cpp:521:      if ((fp = fopen(buffer, "r")) != NULL) {
os/solaris/os_perf_solaris.cpp:557:  if ((fp = fopen(psinfo_path, "r")) == NULL) {
os/solaris/os_solaris.cpp:1605:  FILE* fp = fopen("/etc/release", "r");
os/solaris/os_solaris.cpp:4091:  //   fopen must be less than 256, _even_ when the first limit above
os/solaris/os_solaris.cpp:4092:  //   has been raised.  This can cause calls to fopen (but not calls to
os/solaris/os_solaris.cpp:4094:  //   native code (although the JDK itself uses fopen).  One can hardly
os/solaris/os_solaris.cpp:4103:  //   stdio fopen limit by calling function enable_extended_FILE_stdio.
os/solaris/os_solaris.cpp:4138:  // - might cause an fopen in the subprocess to fail on a system
os_cpu/linux_sparc/vm_version_linux_sparc.cpp:39:    FILE* fp = fopen("/proc/cpuinfo", "r");
share/logging/logFileOutput.cpp:272:  _stream = os::fopen(_file_name, FileOpenMode);
share/logging/logFileOutput.cpp:361:  _stream = os::fopen(_file_name, FileOpenMode);
share/runtime/arguments.cpp:1345:  FILE* stream = fopen(file_name, "rb");
share/runtime/memprofiler.cpp:76:    _log_fp = fopen(log_name , "w+");
share/runtime/os.cpp:1253:// This function is a proxy to fopen, it tries to add a non standard flag ('e' or 'N')
share/runtime/os.cpp:1257:FILE* os::fopen(const char* path, const char* mode) {
share/runtime/os.cpp:1261:  FILE* file = ::fopen(path, modified_mode);
share/runtime/os.cpp:1265:  // is not supported as mode in fopen
share/runtime/os.hpp:510:  static FILE* fopen(const char* path, const char* mode);
share/runtime/abstract_vm_version.cpp:308:  FILE* fp = fopen(filename, "r");
share/utilities/elfFile.cpp:171:  _file = fopen(filepath, "r");
share/utilities/ostream.cpp:513:  _file = fopen(file_name, "w");
share/utilities/ostream.cpp:523:  _file = fopen(file_name, opentype);
share/adlc/main.cpp:363:      (ADF._fp = fopen(ADF._name, action)) == NULL) {
share/c1/c1_Compilation.cpp:697:  fileStream stream(fopen("c1_compile_only", "wt"));
share/c1/c1_Compilation.cpp:713:  fileStream stream(fopen(".hotspot_compiler", "at"));
share/ci/ciReplay.cpp:127:    _stream = fopen(filename, "rt");
share/classfile/classListParser.cpp:51:  // Use os::open() because neither fopen() nor os::fopen()
share/compiler/compileBroker.cpp:1891:      fp = fopen(file_name, "wt");
share/compiler/compilerOracle.cpp:703:  FILE* stream = fopen(cc_file(), "rt");
share/compiler/disassembler.cpp:272:        if ((fp = fopen(file, "r")) == NULL) {

From david.holmes at oracle.com  Tue Jan 28 13:06:38 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 28 Jan 2020 23:06:38 +1000
Subject: fopen vs. os::fopen and automatic closing of the file on exec
In-Reply-To: <AM0PR02MB42275083AC9BA465ADCF75B4930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB42275083AC9BA465ADCF75B4930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <462ea46a-cef8-79a0-ccf0-71699e619089@oracle.com>

Hi Matthias,

I don't have any info of most below but one follow up ....

On 28/01/2020 10:51 pm, Baesken, Matthias wrote:
> Hello, I noticed   while looking at   https://bugs.openjdk.java.net/browse/JDK-8237830   ( support O_CLOEXEC in os::open on other OS than Linux )
> that os::fopen  also  has  some support  for setting  FD_CLOEXEC / O_CLOEXEC  on the file opened .
> See :
> 
> 1253// This function is a proxy to fopen, it tries to add a non standard flag ('e' or 'N')
> 1254// that ensures automatic closing of the file on exec. If it can not find support in
> 1255// the underlying c library, it will make an extra system call (fcntl) to ensure automatic
> 1256// closing of the file on exec.
> 1257FILE* os::fopen(const char* path, const char* mode) {
> 1258  char modified_mode[20];
> 1259  assert(strlen(mode) + 1 < sizeof(modified_mode), "mode chars plus one extra must fit in buffer");
> 1260  sprintf(modified_mode, "%s" LINUX_ONLY("e") BSD_ONLY("e") WINDOWS_ONLY("N"), mode);
> 1261  FILE* file = ::fopen(path, modified_mode);
> 1262
> 1263#if !(defined LINUX || defined BSD || defined _WINDOWS)
> 1264  // assume fcntl FD_CLOEXEC support as a backup solution when 'e' or 'N'
> 1265  // is not supported as mode in fopen
> 1266  if (file != NULL) {
> 1267    int fd = fileno(file);
> 1268    if (fd != -1) {
> 1269      int fd_flags = fcntl(fd, F_GETFD);
> 1270      if (fd_flags != -1) {
> 1271        fcntl(fd, F_SETFD, fd_flags | FD_CLOEXEC);
> 1272      }
> 1273    }
> 1274  }
> 1275#endif
> 
> However some questions arise here :
> 
>    1.  Usage :        os::fopen   is  only used sometimes  in HS code ,  should most of the calls to  fopen be adjusted to os::fopen   (see list  below )
>    2.  ::fopen vs. ::fcntl     :    is  os_linux  os::open   we try to  set  the  "closing of the file on exec"   flag  when calling ::open  but we later check  that it really  worked so we seem not to trust it fully ;

The check is for running on older Linuxes that do not support O_CLOEXEC 
- where the flag is ignored. That is why I asked about what happens on 
BSD/macOS and AIX in that situation.

> Should this be done here too  for Linux ? Or is that  checking in os_linux os::open  these days not needed any more ?

It's possible the most recent version of Linux without O_CLOEXEC 
supported is no longer supported by OpenJDK, in which case we can remove 
it. But I'm not sure what that version is. I have no idea if fopen with 
"e" support has the same history as ::open and O_CLOEXEC.

David

> Best regards, Matthias
> 
> 
> 
> Grep showed  me these fopen-calls in HS, a lot go not to os::fopen ?
> 
> cpu/aarch64/vm_version_aarch64.cpp:171:  if (FILE *f = fopen("/proc/cpuinfo", "r")) {
> cpu/ppc/vm_version_ppc.cpp:412:  FILE* fp = fopen(info_file, "r");
> os/aix/os_aix.cpp:3756:  // - might cause an fopen in the subprocess to fail on a system
> os/aix/os_perf_aix.cpp:243:  if ((f = fopen(procfile, "r")) == NULL) {
> os/aix/os_perf_aix.cpp:666:  if ((fp = fopen(buffer, "r")) != NULL) {
> os/aix/os_perf_aix.cpp:694:  if ((fp = fopen(buffer, "r")) != NULL) {
> os/bsd/os_bsd.cpp:3479:  // - might cause an fopen in the subprocess to fail on a system
> os/linux/decoder_linux.cpp:61:  FILE* file = fopen(filepath, "r");
> os/linux/os_linux.cpp:257:  if ((fh = fopen("/proc/stat", "r")) == NULL) {
> os/linux/os_linux.cpp:364:    FILE *fp = fopen(fname, "r");
> os/linux/os_linux.cpp:1106:  FILE *fp = fopen("/proc/self/maps", "r");
> os/linux/os_linux.cpp:1218:    fp = fopen("/proc/self/stat", "r");
> os/linux/os_linux.cpp:2075:  if ((procmapsFile = fopen("/proc/self/maps", "r")) != NULL) {
> os/linux/os_linux.cpp:2244:  FILE* fp = fopen(file, "r");
> os/linux/os_linux.cpp:2458:  FILE *fp = fopen("/proc/cpuinfo", "r");
> os/linux/os_linux.cpp:2515:  FILE* fp = fopen("/proc/cpuinfo", "r");
> os/linux/os_linux.cpp:3645:    FILE *fp = fopen("/proc/self/maps", "r");
> os/linux/os_linux.cpp:3689:  if ((f = fopen("/proc/self/coredump_filter", "r+")) == NULL) {
> os/linux/os_linux.cpp:3741:  FILE *fp = fopen("/proc/meminfo", "r");
> os/linux/os_linux.cpp:5572:  // - might cause an fopen in the subprocess to fail on a system
> os/linux/os_linux.cpp:5797:  fp = fopen(proc_name, "r");
> os/linux/os_perf_linux.cpp:238:  if ((f = fopen(procfile, "r")) == NULL) {
> os/linux/os_perf_linux.cpp:275:  if ((f = fopen("/proc/stat", "r")) == NULL) {
> os/linux/os_perf_linux.cpp:726:  if ((fp = fopen(buffer, "r")) != NULL) {
> os/linux/os_perf_linux.cpp:754:  if ((fp = fopen(buffer, "r")) != NULL) {
> os/linux/perfMemory_linux.cpp:659:  FILE *fp = fopen(fname, "r");
> os/linux/gc/z/zPhysicalMemoryBacking_linux.cpp:312:  FILE* const file = fopen(filename, "r");
> os/linux/gc/z/zMountPoint_linux.cpp:72:  FILE* fd = fopen(PROC_SELF_MOUNTINFO, "r");
> os/linux/cgroupSubsystem_linux.cpp:66:  cgroups = fopen("/proc/cgroups", "r");
> os/linux/cgroupSubsystem_linux.cpp:121:  cgroup = fopen("/proc/self/cgroup", "r");
> os/linux/cgroupSubsystem_linux.cpp:170:    mntinfo = fopen("/proc/self/mountinfo", "r");
> os/linux/cgroupSubsystem_linux.cpp:224:  mntinfo = fopen("/proc/self/mountinfo", "r");
> os/linux/cgroupSubsystem_linux.hpp:96:  fp = fopen(file, "r");
> os/solaris/os_perf_solaris.cpp:521:      if ((fp = fopen(buffer, "r")) != NULL) {
> os/solaris/os_perf_solaris.cpp:557:  if ((fp = fopen(psinfo_path, "r")) == NULL) {
> os/solaris/os_solaris.cpp:1605:  FILE* fp = fopen("/etc/release", "r");
> os/solaris/os_solaris.cpp:4091:  //   fopen must be less than 256, _even_ when the first limit above
> os/solaris/os_solaris.cpp:4092:  //   has been raised.  This can cause calls to fopen (but not calls to
> os/solaris/os_solaris.cpp:4094:  //   native code (although the JDK itself uses fopen).  One can hardly
> os/solaris/os_solaris.cpp:4103:  //   stdio fopen limit by calling function enable_extended_FILE_stdio.
> os/solaris/os_solaris.cpp:4138:  // - might cause an fopen in the subprocess to fail on a system
> os_cpu/linux_sparc/vm_version_linux_sparc.cpp:39:    FILE* fp = fopen("/proc/cpuinfo", "r");
> share/logging/logFileOutput.cpp:272:  _stream = os::fopen(_file_name, FileOpenMode);
> share/logging/logFileOutput.cpp:361:  _stream = os::fopen(_file_name, FileOpenMode);
> share/runtime/arguments.cpp:1345:  FILE* stream = fopen(file_name, "rb");
> share/runtime/memprofiler.cpp:76:    _log_fp = fopen(log_name , "w+");
> share/runtime/os.cpp:1253:// This function is a proxy to fopen, it tries to add a non standard flag ('e' or 'N')
> share/runtime/os.cpp:1257:FILE* os::fopen(const char* path, const char* mode) {
> share/runtime/os.cpp:1261:  FILE* file = ::fopen(path, modified_mode);
> share/runtime/os.cpp:1265:  // is not supported as mode in fopen
> share/runtime/os.hpp:510:  static FILE* fopen(const char* path, const char* mode);
> share/runtime/abstract_vm_version.cpp:308:  FILE* fp = fopen(filename, "r");
> share/utilities/elfFile.cpp:171:  _file = fopen(filepath, "r");
> share/utilities/ostream.cpp:513:  _file = fopen(file_name, "w");
> share/utilities/ostream.cpp:523:  _file = fopen(file_name, opentype);
> share/adlc/main.cpp:363:      (ADF._fp = fopen(ADF._name, action)) == NULL) {
> share/c1/c1_Compilation.cpp:697:  fileStream stream(fopen("c1_compile_only", "wt"));
> share/c1/c1_Compilation.cpp:713:  fileStream stream(fopen(".hotspot_compiler", "at"));
> share/ci/ciReplay.cpp:127:    _stream = fopen(filename, "rt");
> share/classfile/classListParser.cpp:51:  // Use os::open() because neither fopen() nor os::fopen()
> share/compiler/compileBroker.cpp:1891:      fp = fopen(file_name, "wt");
> share/compiler/compilerOracle.cpp:703:  FILE* stream = fopen(cc_file(), "rt");
> share/compiler/disassembler.cpp:272:        if ((fp = fopen(file, "r")) == NULL) {
> 

From matthias.baesken at sap.com  Tue Jan 28 13:44:19 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 28 Jan 2020 13:44:19 +0000
Subject: fopen vs. os::fopen and automatic closing of the file on exec
In-Reply-To: <462ea46a-cef8-79a0-ccf0-71699e619089@oracle.com>
References: <AM0PR02MB42275083AC9BA465ADCF75B4930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <462ea46a-cef8-79a0-ccf0-71699e619089@oracle.com>
Message-ID: <AM0PR02MB4227CA25D9DBEDC5A1E81B31930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>

Hi David, thanks for looking into it .

While checking other  open(64) calls, I found  the fileOpen / handleOpen   calls  in java.base .
Looks like  they  miss  so far  setting  FD_CLOEXEC /   O_CLOEXEC  , it isn?t done in the callers of those functions  and it is also not (silently) added in the fileOpen/handleOpen function itself  (like HS does in os::open ).
See :

73FD
74handleOpen(const char *path, int oflag, int mode) {
75    FD fd;
76    RESTARTABLE(open64(path, oflag, mode), fd);
77    if (fd != -1) {
78        struct stat64 buf64;
79        int result;
80        RESTARTABLE(fstat64(fd, &buf64), result);
81        if (result != -1) {
82            if (S_ISDIR(buf64.st_mode)) {
83                close(fd);
84                errno = EISDIR;
85                fd = -1;
86            }
87        } else {
88            close(fd);
89            fd = -1;
90        }
91    }
92    return fd;
93}

95void
96fileOpen(JNIEnv *env, jobject this, jstring path, jfieldID fid, int flags)
97{
.....
107        fd = handleOpen(ps, flags, 0666);
....
122}

56JNIEXPORT void JNICALL
57Java_java_io_FileOutputStream_open0(JNIEnv *env, jobject this,
58                                    jstring path, jboolean append) {
59    fileOpen(env, this, path, fos_fd,
60             O_WRONLY | O_CREAT | (append ? O_APPEND : O_TRUNC));
61}

59JNIEXPORT void JNICALL
60Java_java_io_FileInputStream_open0(JNIEnv *env, jobject this, jstring path) {
61    fileOpen(env, this, path, fis_fd, O_RDONLY);
62}


Is this something that should be changed too ? 

Best regards, Matthias


> 
> Hi Matthias,
> 
> I don't have any info of most below but one follow up ....
> 
> On 28/01/2020 10:51 pm, Baesken, Matthias wrote:
> > Hello, I noticed   while looking at
> https://bugs.openjdk.java.net/browse/JDK-8237830   ( support O_CLOEXEC
> in os::open on other OS than Linux )
> > that os::fopen  also  has  some support  for setting  FD_CLOEXEC /
> O_CLOEXEC  on the file opened .
> > See :
> >
> > 1253// This function is a proxy to fopen, it tries to add a non standard flag
> ('e' or 'N')
> > 1254// that ensures automatic closing of the file on exec. If it can not find
> support in
> > 1255// the underlying c library, it will make an extra system call (fcntl) to
> ensure automatic
> > 1256// closing of the file on exec.
> > 1257FILE* os::fopen(const char* path, const char* mode) {
> > 1258  char modified_mode[20];
> > 1259  assert(strlen(mode) + 1 < sizeof(modified_mode), "mode chars plus
> one extra must fit in buffer");
> > 1260  sprintf(modified_mode, "%s" LINUX_ONLY("e") BSD_ONLY("e")
> WINDOWS_ONLY("N"), mode);
> > 1261  FILE* file = ::fopen(path, modified_mode);
> > 1262
> > 1263#if !(defined LINUX || defined BSD || defined _WINDOWS)
> > 1264  // assume fcntl FD_CLOEXEC support as a backup solution when 'e' or
> 'N'
> > 1265  // is not supported as mode in fopen
> > 1266  if (file != NULL) {
> > 1267    int fd = fileno(file);
> > 1268    if (fd != -1) {
> > 1269      int fd_flags = fcntl(fd, F_GETFD);
> > 1270      if (fd_flags != -1) {
> > 1271        fcntl(fd, F_SETFD, fd_flags | FD_CLOEXEC);
> > 1272      }
> > 1273    }
> > 1274  }
> > 1275#endif
> >
> > However some questions arise here :
> >
> >    1.  Usage :        os::fopen   is  only used sometimes  in HS code ,  should
> most of the calls to  fopen be adjusted to os::fopen   (see list  below )
> >    2.  ::fopen vs. ::fcntl     :    is  os_linux  os::open   we try to  set  the  "closing
> of the file on exec"   flag  when calling ::open  but we later check  that it really
> worked so we seem not to trust it fully ;
> 
> The check is for running on older Linuxes that do not support O_CLOEXEC
> - where the flag is ignored. That is why I asked about what happens on
> BSD/macOS and AIX in that situation.
> 
> > Should this be done here too  for Linux ? Or is that  checking in os_linux
> os::open  these days not needed any more ?
> 
> It's possible the most recent version of Linux without O_CLOEXEC
> supported is no longer supported by OpenJDK, in which case we can remove
> it. But I'm not sure what that version is. I have no idea if fopen with
> "e" support has the same history as ::open and O_CLOEXEC.
> 
> David
> 


From frederic.parain at oracle.com  Tue Jan 28 14:09:06 2020
From: frederic.parain at oracle.com (Frederic Parain)
Date: Tue, 28 Jan 2020 09:09:06 -0500
Subject: RFR[L]: 8237767 Field layout computation overhaul 
In-Reply-To: <fbcc6b4c-10a9-2201-4594-0155d014e615@oracle.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
 <fbcc6b4c-10a9-2201-4594-0155d014e615@oracle.com>
Message-ID: <3649812A-E1D6-43A5-88BE-DD3E392FA0EB@oracle.com>

Coleen,

Thank you for reviewing this change.

New webrev:
http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.06/index.html

My answers are inlined below.

> On Jan 24, 2020, at 16:17, coleen.phillimore at oracle.com wrote:
> 
> 
> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/ci/ciInstanceKlass.hpp.udiff.html
> 
>   bool contains_field_offset(int offset) {
> - return instanceOopDesc::contains_field_offset(offset, nonstatic_field_size());
> + fieldDescriptor fd;
> + return this->get_instanceKlass()->find_field_from_offset(offset, false, &fd);
>   }
> 
> 
> This has to go into the VM if it's going to access metadata, with VM_ENTRY_MARK, so probably belongs in the cpp file. Also, why doesn't this call contains_field_offset() from InstanceKlass? from here:
> 
> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/oops/instanceKlass.cpp.udiff.html
> 
> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/oops/instanceKlass.hpp.udiff.html

Good catch! I?ve changed the code to:

bool ciInstanceKlass::contains_field_offset(int offset) {
  VM_ENTRY_MARK;
  return get_instanceKlass()->contains_field_offset(offset);
}

> 
> + public:
> + enum {
> + _extra_is_being_redefined = 1 << 0, // used for locking redefinition
> + _extra_has_contended_annotations = 1 << 1 // has @Contended annotation
> + };
> +
> 
> 
> Why is this enum public?  Also, I think you should make these misc_flags and make _flags a u4.  There's already an alignment gap in InstanceKlass and we can file an RFE to fix that after this change.

I?ve extended _misc_flags to a u4 and moved all flags to it.
Note: the type change impacts vmStruct and JVMCI.

> 
> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/runtime/globals.hpp.udiff.html
> 
> I think you should make UseNewLayout a diagnostic flag since we would like to remove the old code once it has gotten more testing. The only reason someone would use it would be to diagnose some differing behavior.

Done.

> 
> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/jdk.internal.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedObjectTypeImpl.java.udiff.html
> 
> Someone who knows the compilers should have a peek at this.
> 
> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/classfile/fieldLayoutBuilder.hpp.html
> 
>  37 // A RawBlock describes an element of a layout.
>  38 // Each field is represented by a RawBlock.
> 
> Thank you for changing RawBlock to LayoutRawBlock.   Can you update the comments here and the description in the RFE?

Comments and RFE fixed.

> 
>  29 #include "classfile/classLoaderData.inline.hpp"
> 
> 
> .hpp files shouldn't include .inline.hpp files.  Whatever uses the inline code should go into the cpp file (or if critical, add an inline.hpp file).

Fixed.

> 
> 132   static const int INITIAL_LIST_SIZE;
> 
> 
> It's odd to see this without an initialization.

I moved the initialization from the .cpp to the .hpp file.

> 
> 227 class FieldLayoutBuilder : public ResourceObj {
> 
> 
> FieldLayoutBuilder is a StackObj isn't it?
> 
> It might be nice to line up the fields, which is part of the coding standard that I only somewhat agree on.  Here it would enhance readability.

Done.

> 
> 264   protected:
> 
> 
> Why are these functions "protected"?   I don't see anything that inherits from FieldLayoutBuilder that might want to call these functions and not the other functions.

It?s a left over from a previous design.
Changed to private.

> 
> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/classfile/fieldLayoutBuilder.cpp.html
> 
> 544             _contended_groups = new (ResourceObj::RESOURCE_AREA, mtInternal) GrowableArray<FieldGroup*>(8);
> 
> 
> Growable arrays are default ResourceObj, so I think the 'new' arguments aren't needed (there were a couple of these).

I?ve changed _contended_groups? type to GrowableArray<FieldGroup*> instead of GrowableArray<FieldGroup*>*
It removes the ?new? and simplifies the management of the _contended_groups field.

> 
> 698   OopMapBlocksBuilder* nonstatic_oop_maps =
> 699       new OopMapBlocksBuilder(max_oop_map_count, Thread::current());
> 
> The code uses OopMapBlocksBuilder as a StackObj, which seems to make more sense.  Can you make it StackObj and not have it allocate from resourceArea?

nonstatic_oop_maps is not used as StackObj, but as a ResourceObj, it escapes the method at the end,
when the FieldLayoutInfo is filled.

> 
> 531 void FieldLayoutBuilder::regular_field_sorting(TRAPS) {
> 
> 635   regular_field_sorting(CHECK);
> 
> None of these functions throw Java class exceptions.  They shouldn't pass TRAPS and CHECK which are for Java exceptions.  If you want to avoid JavaThread::current, you can pass JavaThread* thread as the first parameter.

Fixed

>  I think for this change, it might be better to let this:
> 
> + OopMapBlock* oop_maps_copy = NEW_RESOURCE_ARRAY_IN_THREAD(THREAD, OopMapBlock,
> 
> be
> 
> + OopMapBlock* oop_maps_copy = NEW_RESOURCE_ARRAY(OopMapBlock,
> 
> 
> and get it's own Thread::current().  I don't think it saves many instructions and definitely not any time.

Fixed

> 
> +void OopMapBlocksBuilder::compact(TRAPS) {
> 
> 
> This shouldn't take a thread argument, so shouldn't be declared with TRAPS, see above.

Fixed

> 
> 748   if (PrintFieldLayout) {
> 
> 
> In some future change, this should be turned into unified logging.

Regarding the complexity of the task (considering the amount of data that can be generated
and the variations in the output format), I?d prefer to address this conversion in a separate
changeset.

> 
> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/classfile/classFileParser.cpp.udiff.html
> 
> Why was AnnotationCollector moved to the header file?  It seems only used by classFileParser.cpp?


I needed it here during some steps of the development, but this move is not required anymore,
I moved the class back to the .cpp file.

> 
> This is a really nice change and your comments are really good. These are all pretty small comments.
> 

Thank you for this in depth review.

Fred

> Thanks,
> Coleen
> 
> On 1/23/20 10:03 AM, Frederic Parain wrote:
>> Greetings,
>> 
>> Please review this change proposing a new code to compute field layouts.
>> 
>> CR: https://bugs.openjdk.java.net/browse/JDK-8237767
>> webrev: http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/
>> 
>> The CR includes a detailed description of the motivation and the implementation.
>> 
>> The current version keeps the old code accessible (with a VM flag) in case the
>> small changes in computed layouts cause troubles to some applications or tools.
>> The goal is to get rid of this old code, preferably as soon as possible.
>> 
>> Testing tier1-8.
>> 
>> Thank you,
>> 
>> Fred
>> 
> 


From leo.korinth at oracle.com  Tue Jan 28 15:36:44 2020
From: leo.korinth at oracle.com (Leo Korinth)
Date: Tue, 28 Jan 2020 16:36:44 +0100
Subject: fopen vs. os::fopen and automatic closing of the file on exec
In-Reply-To: <462ea46a-cef8-79a0-ccf0-71699e619089@oracle.com>
References: <AM0PR02MB42275083AC9BA465ADCF75B4930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <462ea46a-cef8-79a0-ccf0-71699e619089@oracle.com>
Message-ID: <76ff9efd-4274-29dd-c048-02e55aa70890@oracle.com>



On 28/01/2020 14:06, David Holmes wrote:
> Hi Matthias,
> 
> I don't have any info of most below but one follow up ....
> 
> On 28/01/2020 10:51 pm, Baesken, Matthias wrote:
>> Hello, I noticed?? while looking at   
>> https://bugs.openjdk.java.net/browse/JDK-8237830?? ( support O_CLOEXEC 
>> in os::open on other OS than Linux )
>> that os::fopen? also? has? some support? for setting? FD_CLOEXEC / 
>> O_CLOEXEC? on the file opened .
>> See :
>>
>> 1253// This function is a proxy to fopen, it tries to add a non 
>> standard flag ('e' or 'N')
>> 1254// that ensures automatic closing of the file on exec. If it can 
>> not find support in
>> 1255// the underlying c library, it will make an extra system call 
>> (fcntl) to ensure automatic
>> 1256// closing of the file on exec.
>> 1257FILE* os::fopen(const char* path, const char* mode) {
>> 1258? char modified_mode[20];
>> 1259? assert(strlen(mode) + 1 < sizeof(modified_mode), "mode chars 
>> plus one extra must fit in buffer");
>> 1260? sprintf(modified_mode, "%s" LINUX_ONLY("e") BSD_ONLY("e") 
>> WINDOWS_ONLY("N"), mode);
>> 1261? FILE* file = ::fopen(path, modified_mode);
>> 1262
>> 1263#if !(defined LINUX || defined BSD || defined _WINDOWS)
>> 1264? // assume fcntl FD_CLOEXEC support as a backup solution when 'e' 
>> or 'N'
>> 1265? // is not supported as mode in fopen
>> 1266? if (file != NULL) {
>> 1267??? int fd = fileno(file);
>> 1268??? if (fd != -1) {
>> 1269????? int fd_flags = fcntl(fd, F_GETFD);
>> 1270????? if (fd_flags != -1) {
>> 1271??????? fcntl(fd, F_SETFD, fd_flags | FD_CLOEXEC);
>> 1272????? }
>> 1273??? }
>> 1274? }
>> 1275#endif
>>
>> However some questions arise here :
>>
>> ?? 1.? Usage :??????? os::fopen?? is? only used sometimes? in HS code 
>> ,? should most of the calls to? fopen be adjusted to os::fopen?? (see 
>> list? below )
>> ?? 2.? ::fopen vs. ::fcntl???? :??? is? os_linux? os::open?? we try 
>> to? set? the? "closing of the file on exec"?? flag? when calling 
>> ::open? but we later check? that it really? worked so we seem not to 
>> trust it fully ;
> 
> The check is for running on older Linuxes that do not support O_CLOEXEC 
> - where the flag is ignored. That is why I asked about what happens on 
> BSD/macOS and AIX in that situation.
> 
>> Should this be done here too? for Linux ? Or is that? checking in 
>> os_linux os::open? these days not needed any more ?
> 
> It's possible the most recent version of Linux without O_CLOEXEC 
> supported is no longer supported by OpenJDK, in which case we can remove 
> it. But I'm not sure what that version is. I have no idea if fopen with 
> "e" support has the same history as ::open and O_CLOEXEC.

"e" is supported since glibc 2.7, released in 2007. Any support of libc 
versions older than 2.7 today would suprise me.

Something that is not obvious is that on unix-like operating systems, 
ProcessImpl_md.c tries to close (most) open files between fork and exec. 
That is not the case for Windows (I opened 
https://bugs.openjdk.java.net/browse/JDK-8202720 for this). Thus (if I 
understand correctly) the impact on unix-like operating systems will be 
less for adding this support than it is for Windows. os::fopen was 
created to solve a specific bug on windows (logging), and was renamed to 
the more generic os::fopen during review.

I guess most uses of ::fopen /should/ use the more restricted os::fopen, 
but the gain would probably be small.

Thanks,
Leo
> David
> 
>> Best regards, Matthias
>>
>>
>>
>> Grep showed? me these fopen-calls in HS, a lot go not to os::fopen ?
>>
>> cpu/aarch64/vm_version_aarch64.cpp:171:? if (FILE *f = 
>> fopen("/proc/cpuinfo", "r")) {
>> cpu/ppc/vm_version_ppc.cpp:412:? FILE* fp = fopen(info_file, "r");
>> os/aix/os_aix.cpp:3756:? // - might cause an fopen in the subprocess 
>> to fail on a system
>> os/aix/os_perf_aix.cpp:243:? if ((f = fopen(procfile, "r")) == NULL) {
>> os/aix/os_perf_aix.cpp:666:? if ((fp = fopen(buffer, "r")) != NULL) {
>> os/aix/os_perf_aix.cpp:694:? if ((fp = fopen(buffer, "r")) != NULL) {
>> os/bsd/os_bsd.cpp:3479:? // - might cause an fopen in the subprocess 
>> to fail on a system
>> os/linux/decoder_linux.cpp:61:? FILE* file = fopen(filepath, "r");
>> os/linux/os_linux.cpp:257:? if ((fh = fopen("/proc/stat", "r")) == 
>> NULL) {
>> os/linux/os_linux.cpp:364:??? FILE *fp = fopen(fname, "r");
>> os/linux/os_linux.cpp:1106:? FILE *fp = fopen("/proc/self/maps", "r");
>> os/linux/os_linux.cpp:1218:??? fp = fopen("/proc/self/stat", "r");
>> os/linux/os_linux.cpp:2075:? if ((procmapsFile = 
>> fopen("/proc/self/maps", "r")) != NULL) {
>> os/linux/os_linux.cpp:2244:? FILE* fp = fopen(file, "r");
>> os/linux/os_linux.cpp:2458:? FILE *fp = fopen("/proc/cpuinfo", "r");
>> os/linux/os_linux.cpp:2515:? FILE* fp = fopen("/proc/cpuinfo", "r");
>> os/linux/os_linux.cpp:3645:??? FILE *fp = fopen("/proc/self/maps", "r");
>> os/linux/os_linux.cpp:3689:? if ((f = 
>> fopen("/proc/self/coredump_filter", "r+")) == NULL) {
>> os/linux/os_linux.cpp:3741:? FILE *fp = fopen("/proc/meminfo", "r");
>> os/linux/os_linux.cpp:5572:? // - might cause an fopen in the 
>> subprocess to fail on a system
>> os/linux/os_linux.cpp:5797:? fp = fopen(proc_name, "r");
>> os/linux/os_perf_linux.cpp:238:? if ((f = fopen(procfile, "r")) == 
>> NULL) {
>> os/linux/os_perf_linux.cpp:275:? if ((f = fopen("/proc/stat", "r")) == 
>> NULL) {
>> os/linux/os_perf_linux.cpp:726:? if ((fp = fopen(buffer, "r")) != NULL) {
>> os/linux/os_perf_linux.cpp:754:? if ((fp = fopen(buffer, "r")) != NULL) {
>> os/linux/perfMemory_linux.cpp:659:? FILE *fp = fopen(fname, "r");
>> os/linux/gc/z/zPhysicalMemoryBacking_linux.cpp:312:? FILE* const file 
>> = fopen(filename, "r");
>> os/linux/gc/z/zMountPoint_linux.cpp:72:? FILE* fd = 
>> fopen(PROC_SELF_MOUNTINFO, "r");
>> os/linux/cgroupSubsystem_linux.cpp:66:? cgroups = 
>> fopen("/proc/cgroups", "r");
>> os/linux/cgroupSubsystem_linux.cpp:121:? cgroup = 
>> fopen("/proc/self/cgroup", "r");
>> os/linux/cgroupSubsystem_linux.cpp:170:??? mntinfo = 
>> fopen("/proc/self/mountinfo", "r");
>> os/linux/cgroupSubsystem_linux.cpp:224:? mntinfo = 
>> fopen("/proc/self/mountinfo", "r");
>> os/linux/cgroupSubsystem_linux.hpp:96:? fp = fopen(file, "r");
>> os/solaris/os_perf_solaris.cpp:521:????? if ((fp = fopen(buffer, "r")) 
>> != NULL) {
>> os/solaris/os_perf_solaris.cpp:557:? if ((fp = fopen(psinfo_path, 
>> "r")) == NULL) {
>> os/solaris/os_solaris.cpp:1605:? FILE* fp = fopen("/etc/release", "r");
>> os/solaris/os_solaris.cpp:4091:? //?? fopen must be less than 256, 
>> _even_ when the first limit above
>> os/solaris/os_solaris.cpp:4092:? //?? has been raised.? This can cause 
>> calls to fopen (but not calls to
>> os/solaris/os_solaris.cpp:4094:? //?? native code (although the JDK 
>> itself uses fopen).? One can hardly
>> os/solaris/os_solaris.cpp:4103:? //?? stdio fopen limit by calling 
>> function enable_extended_FILE_stdio.
>> os/solaris/os_solaris.cpp:4138:? // - might cause an fopen in the 
>> subprocess to fail on a system
>> os_cpu/linux_sparc/vm_version_linux_sparc.cpp:39:??? FILE* fp = 
>> fopen("/proc/cpuinfo", "r");
>> share/logging/logFileOutput.cpp:272:? _stream = os::fopen(_file_name, 
>> FileOpenMode);
>> share/logging/logFileOutput.cpp:361:? _stream = os::fopen(_file_name, 
>> FileOpenMode);
>> share/runtime/arguments.cpp:1345:? FILE* stream = fopen(file_name, "rb");
>> share/runtime/memprofiler.cpp:76:??? _log_fp = fopen(log_name , "w+");
>> share/runtime/os.cpp:1253:// This function is a proxy to fopen, it 
>> tries to add a non standard flag ('e' or 'N')
>> share/runtime/os.cpp:1257:FILE* os::fopen(const char* path, const 
>> char* mode) {
>> share/runtime/os.cpp:1261:? FILE* file = ::fopen(path, modified_mode);
>> share/runtime/os.cpp:1265:? // is not supported as mode in fopen
>> share/runtime/os.hpp:510:? static FILE* fopen(const char* path, const 
>> char* mode);
>> share/runtime/abstract_vm_version.cpp:308:? FILE* fp = fopen(filename, 
>> "r");
>> share/utilities/elfFile.cpp:171:? _file = fopen(filepath, "r");
>> share/utilities/ostream.cpp:513:? _file = fopen(file_name, "w");
>> share/utilities/ostream.cpp:523:? _file = fopen(file_name, opentype);
>> share/adlc/main.cpp:363:????? (ADF._fp = fopen(ADF._name, action)) == 
>> NULL) {
>> share/c1/c1_Compilation.cpp:697:? fileStream 
>> stream(fopen("c1_compile_only", "wt"));
>> share/c1/c1_Compilation.cpp:713:? fileStream 
>> stream(fopen(".hotspot_compiler", "at"));
>> share/ci/ciReplay.cpp:127:??? _stream = fopen(filename, "rt");
>> share/classfile/classListParser.cpp:51:? // Use os::open() because 
>> neither fopen() nor os::fopen()
>> share/compiler/compileBroker.cpp:1891:????? fp = fopen(file_name, "wt");
>> share/compiler/compilerOracle.cpp:703:? FILE* stream = 
>> fopen(cc_file(), "rt");
>> share/compiler/disassembler.cpp:272:??????? if ((fp = fopen(file, 
>> "r")) == NULL) {
>>

From thomas.stuefe at gmail.com  Tue Jan 28 18:10:44 2020
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Tue, 28 Jan 2020 19:10:44 +0100
Subject: fopen vs. os::fopen and automatic closing of the file on exec
In-Reply-To: <76ff9efd-4274-29dd-c048-02e55aa70890@oracle.com>
References: <AM0PR02MB42275083AC9BA465ADCF75B4930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <462ea46a-cef8-79a0-ccf0-71699e619089@oracle.com>
 <76ff9efd-4274-29dd-c048-02e55aa70890@oracle.com>
Message-ID: <CAA-vtUx=G2p71JFL9O5QyVL=ztAuFkcn2aS0ztZY7z+i20UGFg@mail.gmail.com>

Hi Leo,

On Tue, Jan 28, 2020 at 4:37 PM Leo Korinth <leo.korinth at oracle.com> wrote:
...


> Something that is not obvious is that on unix-like operating systems,
> ProcessImpl_md.c tries to close (most) open files between fork and exec.
> That is not the case for Windows (I opened
> https://bugs.openjdk.java.net/browse/JDK-8202720 for this). Thus (if I
> understand correctly) the impact on unix-like operating systems will be
> less for adding this support than it is for Windows. os::fopen was
> created to solve a specific bug on windows (logging), and was renamed to
> the more generic os::fopen during review.
>
>
Just a note, it is still possible for file descriptors to escape into child
processes since you cannot guarantee that all forks happen via
Runtime.exec():
- native third party code may fork.
- hotspot may fork to run tools for error reporting

Also note that the code which closes fds in Runtime.exec() may in theory
fail to close all fds.

So I think Matthias makes sense on Posix platforms.


> I guess most uses of ::fopen /should/ use the more restricted os::fopen,
> but the gain would probably be small.
>
> Thanks,
> Leo
> > David
> >
> >> Best regards, Matthias
> >>
> >>
> >>
> >> Grep showed  me these fopen-calls in HS, a lot go not to os::fopen ?
> >>
> >> cpu/aarch64/vm_version_aarch64.cpp:171:  if (FILE *f =
> >> fopen("/proc/cpuinfo", "r")) {
> >> cpu/ppc/vm_version_ppc.cpp:412:  FILE* fp = fopen(info_file, "r");
> >> os/aix/os_aix.cpp:3756:  // - might cause an fopen in the subprocess
> >> to fail on a system
> >> os/aix/os_perf_aix.cpp:243:  if ((f = fopen(procfile, "r")) == NULL) {
> >> os/aix/os_perf_aix.cpp:666:  if ((fp = fopen(buffer, "r")) != NULL) {
> >> os/aix/os_perf_aix.cpp:694:  if ((fp = fopen(buffer, "r")) != NULL) {
> >> os/bsd/os_bsd.cpp:3479:  // - might cause an fopen in the subprocess
> >> to fail on a system
> >> os/linux/decoder_linux.cpp:61:  FILE* file = fopen(filepath, "r");
> >> os/linux/os_linux.cpp:257:  if ((fh = fopen("/proc/stat", "r")) ==
> >> NULL) {
> >> os/linux/os_linux.cpp:364:    FILE *fp = fopen(fname, "r");
> >> os/linux/os_linux.cpp:1106:  FILE *fp = fopen("/proc/self/maps", "r");
> >> os/linux/os_linux.cpp:1218:    fp = fopen("/proc/self/stat", "r");
> >> os/linux/os_linux.cpp:2075:  if ((procmapsFile =
> >> fopen("/proc/self/maps", "r")) != NULL) {
> >> os/linux/os_linux.cpp:2244:  FILE* fp = fopen(file, "r");
> >> os/linux/os_linux.cpp:2458:  FILE *fp = fopen("/proc/cpuinfo", "r");
> >> os/linux/os_linux.cpp:2515:  FILE* fp = fopen("/proc/cpuinfo", "r");
> >> os/linux/os_linux.cpp:3645:    FILE *fp = fopen("/proc/self/maps", "r");
> >> os/linux/os_linux.cpp:3689:  if ((f =
> >> fopen("/proc/self/coredump_filter", "r+")) == NULL) {
> >> os/linux/os_linux.cpp:3741:  FILE *fp = fopen("/proc/meminfo", "r");
> >> os/linux/os_linux.cpp:5572:  // - might cause an fopen in the
> >> subprocess to fail on a system
> >> os/linux/os_linux.cpp:5797:  fp = fopen(proc_name, "r");
> >> os/linux/os_perf_linux.cpp:238:  if ((f = fopen(procfile, "r")) ==
> >> NULL) {
> >> os/linux/os_perf_linux.cpp:275:  if ((f = fopen("/proc/stat", "r")) ==
> >> NULL) {
> >> os/linux/os_perf_linux.cpp:726:  if ((fp = fopen(buffer, "r")) != NULL)
> {
> >> os/linux/os_perf_linux.cpp:754:  if ((fp = fopen(buffer, "r")) != NULL)
> {
> >> os/linux/perfMemory_linux.cpp:659:  FILE *fp = fopen(fname, "r");
> >> os/linux/gc/z/zPhysicalMemoryBacking_linux.cpp:312:  FILE* const file
> >> = fopen(filename, "r");
> >> os/linux/gc/z/zMountPoint_linux.cpp:72:  FILE* fd =
> >> fopen(PROC_SELF_MOUNTINFO, "r");
> >> os/linux/cgroupSubsystem_linux.cpp:66:  cgroups =
> >> fopen("/proc/cgroups", "r");
> >> os/linux/cgroupSubsystem_linux.cpp:121:  cgroup =
> >> fopen("/proc/self/cgroup", "r");
> >> os/linux/cgroupSubsystem_linux.cpp:170:    mntinfo =
> >> fopen("/proc/self/mountinfo", "r");
> >> os/linux/cgroupSubsystem_linux.cpp:224:  mntinfo =
> >> fopen("/proc/self/mountinfo", "r");
> >> os/linux/cgroupSubsystem_linux.hpp:96:  fp = fopen(file, "r");
> >> os/solaris/os_perf_solaris.cpp:521:      if ((fp = fopen(buffer, "r"))
> >> != NULL) {
> >> os/solaris/os_perf_solaris.cpp:557:  if ((fp = fopen(psinfo_path,
> >> "r")) == NULL) {
> >> os/solaris/os_solaris.cpp:1605:  FILE* fp = fopen("/etc/release", "r");
> >> os/solaris/os_solaris.cpp:4091:  //   fopen must be less than 256,
> >> _even_ when the first limit above
> >> os/solaris/os_solaris.cpp:4092:  //   has been raised.  This can cause
> >> calls to fopen (but not calls to
> >> os/solaris/os_solaris.cpp:4094:  //   native code (although the JDK
> >> itself uses fopen).  One can hardly
> >> os/solaris/os_solaris.cpp:4103:  //   stdio fopen limit by calling
> >> function enable_extended_FILE_stdio.
> >> os/solaris/os_solaris.cpp:4138:  // - might cause an fopen in the
> >> subprocess to fail on a system
> >> os_cpu/linux_sparc/vm_version_linux_sparc.cpp:39:    FILE* fp =
> >> fopen("/proc/cpuinfo", "r");
> >> share/logging/logFileOutput.cpp:272:  _stream = os::fopen(_file_name,
> >> FileOpenMode);
> >> share/logging/logFileOutput.cpp:361:  _stream = os::fopen(_file_name,
> >> FileOpenMode);
> >> share/runtime/arguments.cpp:1345:  FILE* stream = fopen(file_name,
> "rb");
> >> share/runtime/memprofiler.cpp:76:    _log_fp = fopen(log_name , "w+");
> >> share/runtime/os.cpp:1253:// This function is a proxy to fopen, it
> >> tries to add a non standard flag ('e' or 'N')
> >> share/runtime/os.cpp:1257:FILE* os::fopen(const char* path, const
> >> char* mode) {
> >> share/runtime/os.cpp:1261:  FILE* file = ::fopen(path, modified_mode);
> >> share/runtime/os.cpp:1265:  // is not supported as mode in fopen
> >> share/runtime/os.hpp:510:  static FILE* fopen(const char* path, const
> >> char* mode);
> >> share/runtime/abstract_vm_version.cpp:308:  FILE* fp = fopen(filename,
> >> "r");
> >> share/utilities/elfFile.cpp:171:  _file = fopen(filepath, "r");
> >> share/utilities/ostream.cpp:513:  _file = fopen(file_name, "w");
> >> share/utilities/ostream.cpp:523:  _file = fopen(file_name, opentype);
> >> share/adlc/main.cpp:363:      (ADF._fp = fopen(ADF._name, action)) ==
> >> NULL) {
> >> share/c1/c1_Compilation.cpp:697:  fileStream
> >> stream(fopen("c1_compile_only", "wt"));
> >> share/c1/c1_Compilation.cpp:713:  fileStream
> >> stream(fopen(".hotspot_compiler", "at"));
> >> share/ci/ciReplay.cpp:127:    _stream = fopen(filename, "rt");
> >> share/classfile/classListParser.cpp:51:  // Use os::open() because
> >> neither fopen() nor os::fopen()
> >> share/compiler/compileBroker.cpp:1891:      fp = fopen(file_name, "wt");
> >> share/compiler/compilerOracle.cpp:703:  FILE* stream =
> >> fopen(cc_file(), "rt");
> >> share/compiler/disassembler.cpp:272:        if ((fp = fopen(file,
> >> "r")) == NULL) {
> >>
>

From coleen.phillimore at oracle.com  Tue Jan 28 18:57:41 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Tue, 28 Jan 2020 13:57:41 -0500
Subject: RFR[L]: 8237767 Field layout computation overhaul
In-Reply-To: <3649812A-E1D6-43A5-88BE-DD3E392FA0EB@oracle.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
 <fbcc6b4c-10a9-2201-4594-0155d014e615@oracle.com>
 <3649812A-E1D6-43A5-88BE-DD3E392FA0EB@oracle.com>
Message-ID: <c1ccdce8-81e9-ea9d-3fdf-9b9d26c14529@oracle.com>


Fred,? Some more minor comments.

http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.06/src/hotspot/share/classfile/classFileParser.cpp.udiff.html

+#include <classfile/fieldLayoutBuilder.hpp>

This has brackets.

http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.06/src/hotspot/share/classfile/classFileParser.hpp.udiff.html

OopMapBlockBuilder and FieldLayoutInfo field names should probably start 
with _ (preexisting problem but since you moved it, it should follow the 
coding style).

+#include "oops/instanceKlass.hpp"


This is out of alphabetical order.


http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.06/src/hotspot/share/classfile/fieldLayoutBuilder.cpp.html

My comment about TRAPS/CHECK also applied to compute_regular_layout, 
compute_java_lang_ref_Reference_layout, compute_boxing_class_layout and 
build_layout too.? None of these throw C++ exceptions and appear to need 
a THREAD parameter passed in.

http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.06/test/hotspot/jtreg/runtime/FieldLayout/FieldDensityTest.java.html

I think this needs @bug 8237767

Some small comments below.

On 1/28/20 9:09 AM, Frederic Parain wrote:
> Coleen,
>
> Thank you for reviewing this change.
>
> New webrev:
> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.06/index.html
>
> My answers are inlined below.
>
>> On Jan 24, 2020, at 16:17, coleen.phillimore at oracle.com wrote:
>>
>>
>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/ci/ciInstanceKlass.hpp.udiff.html
>>
>>    bool contains_field_offset(int offset) {
>> - return instanceOopDesc::contains_field_offset(offset, nonstatic_field_size());
>> + fieldDescriptor fd;
>> + return this->get_instanceKlass()->find_field_from_offset(offset, false, &fd);
>>    }
>>
>>
>> This has to go into the VM if it's going to access metadata, with VM_ENTRY_MARK, so probably belongs in the cpp file. Also, why doesn't this call contains_field_offset() from InstanceKlass? from here:
>>
>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/oops/instanceKlass.cpp.udiff.html
>>
>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/oops/instanceKlass.hpp.udiff.html
> Good catch! I?ve changed the code to:
>
> bool ciInstanceKlass::contains_field_offset(int offset) {
>    VM_ENTRY_MARK;
>    return get_instanceKlass()->contains_field_offset(offset);
> }
>
>> + public:
>> + enum {
>> + _extra_is_being_redefined = 1 << 0, // used for locking redefinition
>> + _extra_has_contended_annotations = 1 << 1 // has @Contended annotation
>> + };
>> +
>>
>>
>> Why is this enum public?  Also, I think you should make these misc_flags and make _flags a u4.  There's already an alignment gap in InstanceKlass and we can file an RFE to fix that after this change.
> I?ve extended _misc_flags to a u4 and moved all flags to it.
> Note: the type change impacts vmStruct and JVMCI.

Great.? I'll file a new bug to reduce the alignment gaps.
>
>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/runtime/globals.hpp.udiff.html
>>
>> I think you should make UseNewLayout a diagnostic flag since we would like to remove the old code once it has gotten more testing. The only reason someone would use it would be to diagnose some differing behavior.
> Done.
>
>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/jdk.internal.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedObjectTypeImpl.java.udiff.html
>>
>> Someone who knows the compilers should have a peek at this.
>>
>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/classfile/fieldLayoutBuilder.hpp.html
>>
>>   37 // A RawBlock describes an element of a layout.
>>   38 // Each field is represented by a RawBlock.
>>
>> Thank you for changing RawBlock to LayoutRawBlock.   Can you update the comments here and the description in the RFE?
> Comments and RFE fixed.
>
>>   29 #include "classfile/classLoaderData.inline.hpp"
>>
>>
>> .hpp files shouldn't include .inline.hpp files.  Whatever uses the inline code should go into the cpp file (or if critical, add an inline.hpp file).
> Fixed.
>
>> 132   static const int INITIAL_LIST_SIZE;
>>
>>
>> It's odd to see this without an initialization.
> I moved the initialization from the .cpp to the .hpp file.
>
>> 227 class FieldLayoutBuilder : public ResourceObj {
>>
>>
>> FieldLayoutBuilder is a StackObj isn't it?
>>
>> It might be nice to line up the fields, which is part of the coding standard that I only somewhat agree on.  Here it would enhance readability.
> Done.
>
>> 264   protected:
>>
>>
>> Why are these functions "protected"?   I don't see anything that inherits from FieldLayoutBuilder that might want to call these functions and not the other functions.
> It?s a left over from a previous design.
> Changed to private.
>
>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/classfile/fieldLayoutBuilder.cpp.html
>>
>> 544             _contended_groups = new (ResourceObj::RESOURCE_AREA, mtInternal) GrowableArray<FieldGroup*>(8);
>>
>>
>> Growable arrays are default ResourceObj, so I think the 'new' arguments aren't needed (there were a couple of these).
> I?ve changed _contended_groups? type to GrowableArray<FieldGroup*> instead of GrowableArray<FieldGroup*>*
> It removes the ?new? and simplifies the management of the _contended_groups field.
>
>> 698   OopMapBlocksBuilder* nonstatic_oop_maps =
>> 699       new OopMapBlocksBuilder(max_oop_map_count, Thread::current());
>>
>> The code uses OopMapBlocksBuilder as a StackObj, which seems to make more sense.  Can you make it StackObj and not have it allocate from resourceArea?
> nonstatic_oop_maps is not used as StackObj, but as a ResourceObj, it escapes the method at the end,
> when the FieldLayoutInfo is filled.

Ok.
>
>> 531 void FieldLayoutBuilder::regular_field_sorting(TRAPS) {
>>
>> 635   regular_field_sorting(CHECK);
>>
>> None of these functions throw Java class exceptions.  They shouldn't pass TRAPS and CHECK which are for Java exceptions.  If you want to avoid JavaThread::current, you can pass JavaThread* thread as the first parameter.
> Fixed
>
>>   I think for this change, it might be better to let this:
>>
>> + OopMapBlock* oop_maps_copy = NEW_RESOURCE_ARRAY_IN_THREAD(THREAD, OopMapBlock,
>>
>> be
>>
>> + OopMapBlock* oop_maps_copy = NEW_RESOURCE_ARRAY(OopMapBlock,
>>
>>
>> and get it's own Thread::current().  I don't think it saves many instructions and definitely not any time.
> Fixed
>
>> +void OopMapBlocksBuilder::compact(TRAPS) {
>>
>>
>> This shouldn't take a thread argument, so shouldn't be declared with TRAPS, see above.
> Fixed
>
>> 748   if (PrintFieldLayout) {
>>
>>
>> In some future change, this should be turned into unified logging.
> Regarding the complexity of the task (considering the amount of data that can be generated
> and the variations in the output format), I?d prefer to address this conversion in a separate
> changeset.

Definitely should be a follow-up RFE.?? This was one Print* flag that we 
weren't sure if we wanted to convert in the first pass, so it should be 
looked at on its own.

>
>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/classfile/classFileParser.cpp.udiff.html
>>
>> Why was AnnotationCollector moved to the header file?  It seems only used by classFileParser.cpp?
>
> I needed it here during some steps of the development, but this move is not required anymore,
> I moved the class back to the .cpp file.

Great.
>
>> This is a really nice change and your comments are really good. These are all pretty small comments.
>>
> Thank you for this in depth review.

Thanks for making the changes.

Coleen

>
> Fred
>
>> Thanks,
>> Coleen
>>
>> On 1/23/20 10:03 AM, Frederic Parain wrote:
>>> Greetings,
>>>
>>> Please review this change proposing a new code to compute field layouts.
>>>
>>> CR: https://bugs.openjdk.java.net/browse/JDK-8237767
>>> webrev: http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/
>>>
>>> The CR includes a detailed description of the motivation and the implementation.
>>>
>>> The current version keeps the old code accessible (with a VM flag) in case the
>>> small changes in computed layouts cause troubles to some applications or tools.
>>> The goal is to get rid of this old code, preferably as soon as possible.
>>>
>>> Testing tier1-8.
>>>
>>> Thank you,
>>>
>>> Fred
>>>


From frederic.parain at oracle.com  Tue Jan 28 21:30:18 2020
From: frederic.parain at oracle.com (Frederic Parain)
Date: Tue, 28 Jan 2020 16:30:18 -0500
Subject: RFR[L]: 8237767 Field layout computation overhaul 
In-Reply-To: <c1ccdce8-81e9-ea9d-3fdf-9b9d26c14529@oracle.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
 <fbcc6b4c-10a9-2201-4594-0155d014e615@oracle.com>
 <3649812A-E1D6-43A5-88BE-DD3E392FA0EB@oracle.com>
 <c1ccdce8-81e9-ea9d-3fdf-9b9d26c14529@oracle.com>
Message-ID: <66190B15-943A-4A74-B2C5-453FDCD56FDA@oracle.com>

Coleen,

Thank you for this second review:

> On Jan 28, 2020, at 13:57, coleen.phillimore at oracle.com wrote:
> 
> 
> Fred,  Some more minor comments.
> 
> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.06/src/hotspot/share/classfile/classFileParser.cpp.udiff.html
> 
> +#include <classfile/fieldLayoutBuilder.hpp>
> This has brackets.

Fixed.

> 
> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.06/src/hotspot/share/classfile/classFileParser.hpp.udiff.html
> 
> OopMapBlockBuilder and FieldLayoutInfo field names should probably start with _ (preexisting problem but since you moved it, it should follow the coding style).
> 

Fixed.

> +#include "oops/instanceKlass.hpp"
> 
> This is out of alphabetical order.
> 

Fixed

> 
> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.06/src/hotspot/share/classfile/fieldLayoutBuilder.cpp.html
> 
> My comment about TRAPS/CHECK also applied to compute_regular_layout, compute_java_lang_ref_Reference_layout, compute_boxing_class_layout and build_layout too.  None of these throw C++ exceptions and appear to need a THREAD parameter passed in.
> 

I missed the scope of your comment in your first review.
Fixed now, none of the methods in fieldLayoutBuilder.[ch]pp is using TRAPS/CHECK now.

> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.06/test/hotspot/jtreg/runtime/FieldLayout/FieldDensityTest.java.html
> 
> I think this needs @bug 8237767

Added.

New webrev with changes above and updated copyright years:

http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.07/index.html

Thank you,

Fred


> 
> On 1/28/20 9:09 AM, Frederic Parain wrote:
>> Coleen,
>> 
>> Thank you for reviewing this change.
>> 
>> New webrev:
>> 
>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.06/index.html
>> 
>> 
>> My answers are inlined below.
>> 
>> 
>>> On Jan 24, 2020, at 16:17, coleen.phillimore at oracle.com
>>>  wrote:
>>> 
>>> 
>>> 
>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/ci/ciInstanceKlass.hpp.udiff.html
>>> 
>>> 
>>>   bool contains_field_offset(int offset) {
>>> - return instanceOopDesc::contains_field_offset(offset, nonstatic_field_size());
>>> + fieldDescriptor fd;
>>> + return this->get_instanceKlass()->find_field_from_offset(offset, false, &fd);
>>>   }
>>> 
>>> 
>>> This has to go into the VM if it's going to access metadata, with VM_ENTRY_MARK, so probably belongs in the cpp file. Also, why doesn't this call contains_field_offset() from InstanceKlass? from here:
>>> 
>>> 
>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/oops/instanceKlass.cpp.udiff.html
>>> 
>>> 
>>> 
>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/oops/instanceKlass.hpp.udiff.html
>> Good catch! I?ve changed the code to:
>> 
>> bool ciInstanceKlass::contains_field_offset(int offset) {
>>   VM_ENTRY_MARK;
>>   return get_instanceKlass()->contains_field_offset(offset);
>> }
>> 
>> 
>>> + public:
>>> + enum {
>>> + _extra_is_being_redefined = 1 << 0, // used for locking redefinition
>>> + _extra_has_contended_annotations = 1 << 1 // has @Contended annotation
>>> + };
>>> +
>>> 
>>> 
>>> Why is this enum public?  Also, I think you should make these misc_flags and make _flags a u4.  There's already an alignment gap in InstanceKlass and we can file an RFE to fix that after this change.
>>> 
>> I?ve extended _misc_flags to a u4 and moved all flags to it.
>> Note: the type change impacts vmStruct and JVMCI.
>> 
> 
> Great.  I'll file a new bug to reduce the alignment gaps.
>> 
>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/runtime/globals.hpp.udiff.html
>>> 
>>> 
>>> I think you should make UseNewLayout a diagnostic flag since we would like to remove the old code once it has gotten more testing. The only reason someone would use it would be to diagnose some differing behavior.
>>> 
>> Done.
>> 
>> 
>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/jdk.internal.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedObjectTypeImpl.java.udiff.html
>>> 
>>> 
>>> Someone who knows the compilers should have a peek at this.
>>> 
>>> 
>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/classfile/fieldLayoutBuilder.hpp.html
>>> 
>>> 
>>>  37 // A RawBlock describes an element of a layout.
>>>  38 // Each field is represented by a RawBlock.
>>> 
>>> Thank you for changing RawBlock to LayoutRawBlock.   Can you update the comments here and the description in the RFE?
>>> 
>> Comments and RFE fixed.
>> 
>> 
>>>  29 #include "classfile/classLoaderData.inline.hpp"
>>> 
>>> 
>>> .hpp files shouldn't include .inline.hpp files.  Whatever uses the inline code should go into the cpp file (or if critical, add an inline.hpp file).
>>> 
>> Fixed.
>> 
>> 
>>> 132   static const int INITIAL_LIST_SIZE;
>>> 
>>> 
>>> It's odd to see this without an initialization.
>>> 
>> I moved the initialization from the .cpp to the .hpp file.
>> 
>> 
>>> 227 class FieldLayoutBuilder : public ResourceObj {
>>> 
>>> 
>>> FieldLayoutBuilder is a StackObj isn't it?
>>> 
>>> It might be nice to line up the fields, which is part of the coding standard that I only somewhat agree on.  Here it would enhance readability.
>>> 
>> Done.
>> 
>> 
>>> 264   protected:
>>> 
>>> 
>>> Why are these functions "protected"?   I don't see anything that inherits from FieldLayoutBuilder that might want to call these functions and not the other functions.
>>> 
>> It?s a left over from a previous design.
>> Changed to private.
>> 
>> 
>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/classfile/fieldLayoutBuilder.cpp.html
>>> 
>>> 
>>> 544             _contended_groups = new (ResourceObj::RESOURCE_AREA, mtInternal) GrowableArray<FieldGroup*>(8);
>>> 
>>> 
>>> Growable arrays are default ResourceObj, so I think the 'new' arguments aren't needed (there were a couple of these).
>>> 
>> I?ve changed _contended_groups? type to GrowableArray<FieldGroup*> instead of GrowableArray<FieldGroup*>*
>> It removes the ?new? and simplifies the management of the _contended_groups field.
>> 
>> 
>>> 698   OopMapBlocksBuilder* nonstatic_oop_maps =
>>> 699       new OopMapBlocksBuilder(max_oop_map_count, Thread::current());
>>> 
>>> The code uses OopMapBlocksBuilder as a StackObj, which seems to make more sense.  Can you make it StackObj and not have it allocate from resourceArea?
>>> 
>> nonstatic_oop_maps is not used as StackObj, but as a ResourceObj, it escapes the method at the end,
>> when the FieldLayoutInfo is filled.
>> 
> 
> Ok.
>> 
>>> 531 void FieldLayoutBuilder::regular_field_sorting(TRAPS) {
>>> 
>>> 635   regular_field_sorting(CHECK);
>>> 
>>> None of these functions throw Java class exceptions.  They shouldn't pass TRAPS and CHECK which are for Java exceptions.  If you want to avoid JavaThread::current, you can pass JavaThread* thread as the first parameter.
>>> 
>> Fixed
>> 
>> 
>>>  I think for this change, it might be better to let this:
>>> 
>>> + OopMapBlock* oop_maps_copy = NEW_RESOURCE_ARRAY_IN_THREAD(THREAD, OopMapBlock,
>>> 
>>> be
>>> 
>>> + OopMapBlock* oop_maps_copy = NEW_RESOURCE_ARRAY(OopMapBlock,
>>> 
>>> 
>>> and get it's own Thread::current().  I don't think it saves many instructions and definitely not any time.
>>> 
>> Fixed
>> 
>> 
>>> +void OopMapBlocksBuilder::compact(TRAPS) {
>>> 
>>> 
>>> This shouldn't take a thread argument, so shouldn't be declared with TRAPS, see above.
>>> 
>> Fixed
>> 
>> 
>>> 748   if (PrintFieldLayout) {
>>> 
>>> 
>>> In some future change, this should be turned into unified logging.
>>> 
>> Regarding the complexity of the task (considering the amount of data that can be generated
>> and the variations in the output format), I?d prefer to address this conversion in a separate
>> changeset.
>> 
> 
> Definitely should be a follow-up RFE.   This was one Print* flag that we weren't sure if we wanted to convert in the first pass, so it should be looked at on its own.
> 
>> 
>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/classfile/classFileParser.cpp.udiff.html
>>> 
>>> 
>>> Why was AnnotationCollector moved to the header file?  It seems only used by classFileParser.cpp?
>>> 
>> 
>> I needed it here during some steps of the development, but this move is not required anymore,
>> I moved the class back to the .cpp file.
>> 
> 
> Great.
>> 
>>> This is a really nice change and your comments are really good. These are all pretty small comments.
>>> 
>>> 
>> Thank you for this in depth review.
> 
> Thanks for making the changes.
> 
> Coleen
> 
>> 
>> Fred
>> 
>> 
>>> Thanks,
>>> Coleen
>>> 
>>> On 1/23/20 10:03 AM, Frederic Parain wrote:
>>> 
>>>> Greetings,
>>>> 
>>>> Please review this change proposing a new code to compute field layouts.
>>>> 
>>>> CR: 
>>>> https://bugs.openjdk.java.net/browse/JDK-8237767
>>>> 
>>>> webrev: 
>>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/
>>>> 
>>>> 
>>>> The CR includes a detailed description of the motivation and the implementation.
>>>> 
>>>> The current version keeps the old code accessible (with a VM flag) in case the
>>>> small changes in computed layouts cause troubles to some applications or tools.
>>>> The goal is to get rid of this old code, preferably as soon as possible.
>>>> 
>>>> Testing tier1-8.
>>>> 
>>>> Thank you,
>>>> 
>>>> Fred
>>>> 
>>>> 
> 


From coleen.phillimore at oracle.com  Tue Jan 28 21:58:43 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Tue, 28 Jan 2020 21:58:43 +0000 (UTC)
Subject: RFR[L]: 8237767 Field layout computation overhaul
In-Reply-To: <66190B15-943A-4A74-B2C5-453FDCD56FDA@oracle.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
 <fbcc6b4c-10a9-2201-4594-0155d014e615@oracle.com>
 <3649812A-E1D6-43A5-88BE-DD3E392FA0EB@oracle.com>
 <c1ccdce8-81e9-ea9d-3fdf-9b9d26c14529@oracle.com>
 <66190B15-943A-4A74-B2C5-453FDCD56FDA@oracle.com>
Message-ID: <910ad01f-5c55-aa11-41d6-d220f645d800@oracle.com>

Looks great!? Thanks!
Coleen

On 1/28/20 4:30 PM, Frederic Parain wrote:
> Coleen,
>
> Thank you for this second review:
>
>> On Jan 28, 2020, at 13:57, coleen.phillimore at oracle.com wrote:
>>
>>
>> Fred,  Some more minor comments.
>>
>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.06/src/hotspot/share/classfile/classFileParser.cpp.udiff.html
>>
>> +#include <classfile/fieldLayoutBuilder.hpp>
>> This has brackets.
> Fixed.
>
>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.06/src/hotspot/share/classfile/classFileParser.hpp.udiff.html
>>
>> OopMapBlockBuilder and FieldLayoutInfo field names should probably start with _ (preexisting problem but since you moved it, it should follow the coding style).
>>
> Fixed.
>
>> +#include "oops/instanceKlass.hpp"
>>
>> This is out of alphabetical order.
>>
> Fixed
>
>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.06/src/hotspot/share/classfile/fieldLayoutBuilder.cpp.html
>>
>> My comment about TRAPS/CHECK also applied to compute_regular_layout, compute_java_lang_ref_Reference_layout, compute_boxing_class_layout and build_layout too.  None of these throw C++ exceptions and appear to need a THREAD parameter passed in.
>>
> I missed the scope of your comment in your first review.
> Fixed now, none of the methods in fieldLayoutBuilder.[ch]pp is using TRAPS/CHECK now.
>
>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.06/test/hotspot/jtreg/runtime/FieldLayout/FieldDensityTest.java.html
>>
>> I think this needs @bug 8237767
> Added.
>
> New webrev with changes above and updated copyright years:
>
> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.07/index.html
>
> Thank you,
>
> Fred
>
>
>> On 1/28/20 9:09 AM, Frederic Parain wrote:
>>> Coleen,
>>>
>>> Thank you for reviewing this change.
>>>
>>> New webrev:
>>>
>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.06/index.html
>>>
>>>
>>> My answers are inlined below.
>>>
>>>
>>>> On Jan 24, 2020, at 16:17, coleen.phillimore at oracle.com
>>>>   wrote:
>>>>
>>>>
>>>>
>>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/ci/ciInstanceKlass.hpp.udiff.html
>>>>
>>>>
>>>>    bool contains_field_offset(int offset) {
>>>> - return instanceOopDesc::contains_field_offset(offset, nonstatic_field_size());
>>>> + fieldDescriptor fd;
>>>> + return this->get_instanceKlass()->find_field_from_offset(offset, false, &fd);
>>>>    }
>>>>
>>>>
>>>> This has to go into the VM if it's going to access metadata, with VM_ENTRY_MARK, so probably belongs in the cpp file. Also, why doesn't this call contains_field_offset() from InstanceKlass? from here:
>>>>
>>>>
>>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/oops/instanceKlass.cpp.udiff.html
>>>>
>>>>
>>>>
>>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/oops/instanceKlass.hpp.udiff.html
>>> Good catch! I?ve changed the code to:
>>>
>>> bool ciInstanceKlass::contains_field_offset(int offset) {
>>>    VM_ENTRY_MARK;
>>>    return get_instanceKlass()->contains_field_offset(offset);
>>> }
>>>
>>>
>>>> + public:
>>>> + enum {
>>>> + _extra_is_being_redefined = 1 << 0, // used for locking redefinition
>>>> + _extra_has_contended_annotations = 1 << 1 // has @Contended annotation
>>>> + };
>>>> +
>>>>
>>>>
>>>> Why is this enum public?  Also, I think you should make these misc_flags and make _flags a u4.  There's already an alignment gap in InstanceKlass and we can file an RFE to fix that after this change.
>>>>
>>> I?ve extended _misc_flags to a u4 and moved all flags to it.
>>> Note: the type change impacts vmStruct and JVMCI.
>>>
>> Great.  I'll file a new bug to reduce the alignment gaps.
>>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/runtime/globals.hpp.udiff.html
>>>>
>>>>
>>>> I think you should make UseNewLayout a diagnostic flag since we would like to remove the old code once it has gotten more testing. The only reason someone would use it would be to diagnose some differing behavior.
>>>>
>>> Done.
>>>
>>>
>>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/jdk.internal.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotResolvedObjectTypeImpl.java.udiff.html
>>>>
>>>>
>>>> Someone who knows the compilers should have a peek at this.
>>>>
>>>>
>>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/classfile/fieldLayoutBuilder.hpp.html
>>>>
>>>>
>>>>   37 // A RawBlock describes an element of a layout.
>>>>   38 // Each field is represented by a RawBlock.
>>>>
>>>> Thank you for changing RawBlock to LayoutRawBlock.   Can you update the comments here and the description in the RFE?
>>>>
>>> Comments and RFE fixed.
>>>
>>>
>>>>   29 #include "classfile/classLoaderData.inline.hpp"
>>>>
>>>>
>>>> .hpp files shouldn't include .inline.hpp files.  Whatever uses the inline code should go into the cpp file (or if critical, add an inline.hpp file).
>>>>
>>> Fixed.
>>>
>>>
>>>> 132   static const int INITIAL_LIST_SIZE;
>>>>
>>>>
>>>> It's odd to see this without an initialization.
>>>>
>>> I moved the initialization from the .cpp to the .hpp file.
>>>
>>>
>>>> 227 class FieldLayoutBuilder : public ResourceObj {
>>>>
>>>>
>>>> FieldLayoutBuilder is a StackObj isn't it?
>>>>
>>>> It might be nice to line up the fields, which is part of the coding standard that I only somewhat agree on.  Here it would enhance readability.
>>>>
>>> Done.
>>>
>>>
>>>> 264   protected:
>>>>
>>>>
>>>> Why are these functions "protected"?   I don't see anything that inherits from FieldLayoutBuilder that might want to call these functions and not the other functions.
>>>>
>>> It?s a left over from a previous design.
>>> Changed to private.
>>>
>>>
>>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/classfile/fieldLayoutBuilder.cpp.html
>>>>
>>>>
>>>> 544             _contended_groups = new (ResourceObj::RESOURCE_AREA, mtInternal) GrowableArray<FieldGroup*>(8);
>>>>
>>>>
>>>> Growable arrays are default ResourceObj, so I think the 'new' arguments aren't needed (there were a couple of these).
>>>>
>>> I?ve changed _contended_groups? type to GrowableArray<FieldGroup*> instead of GrowableArray<FieldGroup*>*
>>> It removes the ?new? and simplifies the management of the _contended_groups field.
>>>
>>>
>>>> 698   OopMapBlocksBuilder* nonstatic_oop_maps =
>>>> 699       new OopMapBlocksBuilder(max_oop_map_count, Thread::current());
>>>>
>>>> The code uses OopMapBlocksBuilder as a StackObj, which seems to make more sense.  Can you make it StackObj and not have it allocate from resourceArea?
>>>>
>>> nonstatic_oop_maps is not used as StackObj, but as a ResourceObj, it escapes the method at the end,
>>> when the FieldLayoutInfo is filled.
>>>
>> Ok.
>>>> 531 void FieldLayoutBuilder::regular_field_sorting(TRAPS) {
>>>>
>>>> 635   regular_field_sorting(CHECK);
>>>>
>>>> None of these functions throw Java class exceptions.  They shouldn't pass TRAPS and CHECK which are for Java exceptions.  If you want to avoid JavaThread::current, you can pass JavaThread* thread as the first parameter.
>>>>
>>> Fixed
>>>
>>>
>>>>   I think for this change, it might be better to let this:
>>>>
>>>> + OopMapBlock* oop_maps_copy = NEW_RESOURCE_ARRAY_IN_THREAD(THREAD, OopMapBlock,
>>>>
>>>> be
>>>>
>>>> + OopMapBlock* oop_maps_copy = NEW_RESOURCE_ARRAY(OopMapBlock,
>>>>
>>>>
>>>> and get it's own Thread::current().  I don't think it saves many instructions and definitely not any time.
>>>>
>>> Fixed
>>>
>>>
>>>> +void OopMapBlocksBuilder::compact(TRAPS) {
>>>>
>>>>
>>>> This shouldn't take a thread argument, so shouldn't be declared with TRAPS, see above.
>>>>
>>> Fixed
>>>
>>>
>>>> 748   if (PrintFieldLayout) {
>>>>
>>>>
>>>> In some future change, this should be turned into unified logging.
>>>>
>>> Regarding the complexity of the task (considering the amount of data that can be generated
>>> and the variations in the output format), I?d prefer to address this conversion in a separate
>>> changeset.
>>>
>> Definitely should be a follow-up RFE.   This was one Print* flag that we weren't sure if we wanted to convert in the first pass, so it should be looked at on its own.
>>
>>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/src/hotspot/share/classfile/classFileParser.cpp.udiff.html
>>>>
>>>>
>>>> Why was AnnotationCollector moved to the header file?  It seems only used by classFileParser.cpp?
>>>>
>>> I needed it here during some steps of the development, but this move is not required anymore,
>>> I moved the class back to the .cpp file.
>>>
>> Great.
>>>> This is a really nice change and your comments are really good. These are all pretty small comments.
>>>>
>>>>
>>> Thank you for this in depth review.
>> Thanks for making the changes.
>>
>> Coleen
>>
>>> Fred
>>>
>>>
>>>> Thanks,
>>>> Coleen
>>>>
>>>> On 1/23/20 10:03 AM, Frederic Parain wrote:
>>>>
>>>>> Greetings,
>>>>>
>>>>> Please review this change proposing a new code to compute field layouts.
>>>>>
>>>>> CR:
>>>>> https://bugs.openjdk.java.net/browse/JDK-8237767
>>>>>
>>>>> webrev:
>>>>> http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.04/
>>>>>
>>>>>
>>>>> The CR includes a detailed description of the motivation and the implementation.
>>>>>
>>>>> The current version keeps the old code accessible (with a VM flag) in case the
>>>>> small changes in computed layouts cause troubles to some applications or tools.
>>>>> The goal is to get rid of this old code, preferably as soon as possible.
>>>>>
>>>>> Testing tier1-8.
>>>>>
>>>>> Thank you,
>>>>>
>>>>> Fred
>>>>>
>>>>>


From david.holmes at oracle.com  Tue Jan 28 23:27:03 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 29 Jan 2020 09:27:03 +1000
Subject: fopen vs. os::fopen and automatic closing of the file on exec
In-Reply-To: <AM0PR02MB4227CA25D9DBEDC5A1E81B31930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB42275083AC9BA465ADCF75B4930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <462ea46a-cef8-79a0-ccf0-71699e619089@oracle.com>
 <AM0PR02MB4227CA25D9DBEDC5A1E81B31930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <852b12ce-8a80-eba0-4b6c-ca13a487d7c3@oracle.com>

Hi Matthias,

On 28/01/2020 11:44 pm, Baesken, Matthias wrote:
> Hi David, thanks for looking into it .
> 
> While checking other  open(64) calls, I found  the fileOpen / handleOpen   calls  in java.base .
> Looks like  they  miss  so far  setting  FD_CLOEXEC /   O_CLOEXEC  , it isn?t done in the callers of those functions  and it is also not (silently) added in the fileOpen/handleOpen function itself  (like HS does in os::open ).
> See :
> 
> 73FD
> 74handleOpen(const char *path, int oflag, int mode) {
> 75    FD fd;
> 76    RESTARTABLE(open64(path, oflag, mode), fd);
> 77    if (fd != -1) {
> 78        struct stat64 buf64;
> 79        int result;
> 80        RESTARTABLE(fstat64(fd, &buf64), result);
> 81        if (result != -1) {
> 82            if (S_ISDIR(buf64.st_mode)) {
> 83                close(fd);
> 84                errno = EISDIR;
> 85                fd = -1;
> 86            }
> 87        } else {
> 88            close(fd);
> 89            fd = -1;
> 90        }
> 91    }
> 92    return fd;
> 93}
> 
> 95void
> 96fileOpen(JNIEnv *env, jobject this, jstring path, jfieldID fid, int flags)
> 97{
> .....
> 107        fd = handleOpen(ps, flags, 0666);
> ....
> 122}
> 
> 56JNIEXPORT void JNICALL
> 57Java_java_io_FileOutputStream_open0(JNIEnv *env, jobject this,
> 58                                    jstring path, jboolean append) {
> 59    fileOpen(env, this, path, fos_fd,
> 60             O_WRONLY | O_CREAT | (append ? O_APPEND : O_TRUNC));
> 61}
> 
> 59JNIEXPORT void JNICALL
> 60Java_java_io_FileInputStream_open0(JNIEnv *env, jobject this, jstring path) {
> 61    fileOpen(env, this, path, fis_fd, O_RDONLY);
> 62}
> 
> 
> Is this something that should be changed too ?

I think not. Unless those public API's are specified to open a file in a 
specific mode, like close-on-exec, then they should not do that by default.

We should only be doing close-on-exec on internally used files.

David
-----

> Best regards, Matthias
> 
> 
>>
>> Hi Matthias,
>>
>> I don't have any info of most below but one follow up ....
>>
>> On 28/01/2020 10:51 pm, Baesken, Matthias wrote:
>>> Hello, I noticed   while looking at
>> https://bugs.openjdk.java.net/browse/JDK-8237830   ( support O_CLOEXEC
>> in os::open on other OS than Linux )
>>> that os::fopen  also  has  some support  for setting  FD_CLOEXEC /
>> O_CLOEXEC  on the file opened .
>>> See :
>>>
>>> 1253// This function is a proxy to fopen, it tries to add a non standard flag
>> ('e' or 'N')
>>> 1254// that ensures automatic closing of the file on exec. If it can not find
>> support in
>>> 1255// the underlying c library, it will make an extra system call (fcntl) to
>> ensure automatic
>>> 1256// closing of the file on exec.
>>> 1257FILE* os::fopen(const char* path, const char* mode) {
>>> 1258  char modified_mode[20];
>>> 1259  assert(strlen(mode) + 1 < sizeof(modified_mode), "mode chars plus
>> one extra must fit in buffer");
>>> 1260  sprintf(modified_mode, "%s" LINUX_ONLY("e") BSD_ONLY("e")
>> WINDOWS_ONLY("N"), mode);
>>> 1261  FILE* file = ::fopen(path, modified_mode);
>>> 1262
>>> 1263#if !(defined LINUX || defined BSD || defined _WINDOWS)
>>> 1264  // assume fcntl FD_CLOEXEC support as a backup solution when 'e' or
>> 'N'
>>> 1265  // is not supported as mode in fopen
>>> 1266  if (file != NULL) {
>>> 1267    int fd = fileno(file);
>>> 1268    if (fd != -1) {
>>> 1269      int fd_flags = fcntl(fd, F_GETFD);
>>> 1270      if (fd_flags != -1) {
>>> 1271        fcntl(fd, F_SETFD, fd_flags | FD_CLOEXEC);
>>> 1272      }
>>> 1273    }
>>> 1274  }
>>> 1275#endif
>>>
>>> However some questions arise here :
>>>
>>>     1.  Usage :        os::fopen   is  only used sometimes  in HS code ,  should
>> most of the calls to  fopen be adjusted to os::fopen   (see list  below )
>>>     2.  ::fopen vs. ::fcntl     :    is  os_linux  os::open   we try to  set  the  "closing
>> of the file on exec"   flag  when calling ::open  but we later check  that it really
>> worked so we seem not to trust it fully ;
>>
>> The check is for running on older Linuxes that do not support O_CLOEXEC
>> - where the flag is ignored. That is why I asked about what happens on
>> BSD/macOS and AIX in that situation.
>>
>>> Should this be done here too  for Linux ? Or is that  checking in os_linux
>> os::open  these days not needed any more ?
>>
>> It's possible the most recent version of Linux without O_CLOEXEC
>> supported is no longer supported by OpenJDK, in which case we can remove
>> it. But I'm not sure what that version is. I have no idea if fopen with
>> "e" support has the same history as ::open and O_CLOEXEC.
>>
>> David
>>
> 

From david.holmes at oracle.com  Wed Jan 29 05:13:29 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 29 Jan 2020 15:13:29 +1000
Subject: RFR (S): 8237857: LogDecorations::uptimenanos is implemented
 incorrectly
Message-ID: <42f25d16-7117-ae55-2dd5-2c5929f5340d@oracle.com>

Bug: https://bugs.openjdk.java.net/browse/JDK-8237857
Webrev: http://cr.openjdk.java.net/~dholmes/8237857/webrev/

After my changes in JDK-8235741 which changed uptimemillis to be the 
same as uptimenanos*1000000 the gtest for the logging time decorators 
started to fail intermittently on Windows where the uptimemillis values 
was not advancing sometimes after a 5ms sleep. It turned out this was 
actually a day one bug in the logging code that my change had exposed - 
the code uptimenanos was using os::elapsed_counter() but that isn't a 
time value it's just a counter value. On Linux/AIX/BSD it happens to 
count at 1ns resolution but not so on Solaris and in particular not so 
on Windows. When this counter value was truncated to "millis" there was 
no guarantee the value would increase as expected by the test.

Changes:
- use os::elapsedTimer() not os::elapsed_counter() for uptimemillis and 
uptimenanos
- remove dead code that should have been removed in JDK-8235741
- update the test by:
   - adding diagnostic printouts of the time values read
   - fix a bug where the sleep time can be too short for Windows (see 
details in bug report)

Testing:
   - gtests on all platforms with manual analysis of log files

Thanks,
David
-----

From david.holmes at oracle.com  Wed Jan 29 07:06:58 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 29 Jan 2020 17:06:58 +1000
Subject: RFR[L]: 8237767 Field layout computation overhaul
In-Reply-To: <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
 <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com>
 <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com>
Message-ID: <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com>

Hi Fred,

I've looked at the v7 version. A few more stylistic comments on that 
first. Note, no need for an item by item response unless that makes it 
easier for you to track :)

src/hotspot/share/classfile/classFileParser.cpp

  #include "classfile/defaultMethods.hpp"
+#include "classfile/fieldLayoutBuilder.hpp"
  #include "classfile/dictionary.hpp"

Include files are not in alphabetical order.

+   * This may well change: FixMe if doesn't,

s/if/if it/

+  //Make a temp copy, and iterate through and copy back into the orig

Space after //

s/orig/original/

+  OopMapBlock*  nonstatic_oop_map = _nonstatic_oop_maps;

Extra space after *

+  assert(ik->nonstatic_oop_map_count() == 
_field_info->oop_map_blocks->_nonstatic_oop_map_count,
+     "sanity");

Second line needs to be indented further:

    assert(ik->nonstatic_oop_map_count() == 
_field_info->oop_map_blocks->_nonstatic_oop_map_count,
           "sanity");

---

src/hotspot/share/classfile/classFileParser.hpp

+public:
+  OopMapBlock* _nonstatic_oop_maps;
+  unsigned int _nonstatic_oop_map_count;
+  unsigned int _max_nonstatic_oop_maps;
+
+ public:

Second public uneeded. First public may be indented wrong (I'm not sure 
what the rule is - single space indent?)

  class ClassFileParser {
+  friend class FieldLayoutBuilder;
+  friend class FieldLayout;

   class ClassAnnotationCollector;
   class FieldAllocationCount;
   class FieldAnnotationCollector;

Indents are different. I think the class forward declarations should 
have extra space.

---

src/hotspot/share/oops/instanceKlass.hpp

+  void increment_count(int diff)     { _count += diff; }

Extra spaces before {

---

src/hotspot/share/runtime/globals.hpp

+  diagnostic(bool, UseNewFieldLayout, true, 
     \
+               "Use new algorithm to compute layouts") 
     \
+ 
     \
+  product(bool, UseEmptySlotsInSupers, true,

Not sure I see why one flag is diagnostic and the other product. Do you 
expect people to need to disable using empty slots more so than needing 
to disable using the new field layout altogether?

---

src/hotspot/share/classfile/fieldLayoutBuilder.cpp

+  assert(kind == EMPTY || kind == RESERVED || kind == PADDING || kind 
== INHERITED,
+      "Otherwise, should use the constructor with a field index argument");

Indentation of second line is wrong.

+  assert(kind == REGULAR || kind == FLATTENED || kind == INHERITED,
+      "Other kind do not have a field index");

Ditto.


+  if (list == NULL) return;
+  if (start == NULL) {
+    start = this->_start;
+  }

Inconsistent style for single statement if-blocks. Same thing later in 
the file.

+      output->print_cr(" @%d \"%s\" %s %d/%d %s",
+          b->offset(),
+          fi->name(_cp)->as_C_string(),
+          fi->signature(_cp)->as_C_string(),
+          b->size(),
+          b->alignment(),
+          "REGULAR");

Incorrect identation of continuing line. Same for all the following 
print blocks.

+  } else if (_classname == vmSymbols::java_lang_Boolean() ||
+      _classname == vmSymbols::java_lang_Character() ||
+      _classname == vmSymbols::java_lang_Float() ||
+      _classname == vmSymbols::java_lang_Double() ||
+      _classname == vmSymbols::java_lang_Byte() ||
+      _classname == vmSymbols::java_lang_Short() ||
+      _classname == vmSymbols::java_lang_Integer() ||
+      _classname == vmSymbols::java_lang_Long()) {

Incorrect identation of continuing line.

---

src/hotspot/share/classfile/fieldLayoutBuilder.hpp

+// and the boxing classes). The rational for having multiple methods

s/rational/rationale/

+  FieldLayoutBuilder(const Symbol* classname, const InstanceKlass* 
super_klass, ConstantPool* constant_pool,
+      Array<u2>* fields, bool is_contended, FieldLayoutInfo* info);

Indentation wrong for continuing line.

+  int get_alignment() {
+     assert(_alignment != -1, "Uninitialized");
+     return _alignment;
+   }

Indenting appears off by one.

---

test/hotspot/jtreg/runtime/FieldLayout/FieldDensityTest.java

+ * @run main/othervm -XX:+UseCompressedOops 
-XX:+UseCompressedClassPointers FieldDensityTest
+ * @run main/othervm -XX:+UseCompressedOops 
-XX:-UseCompressedClassPointers FieldDensityTest
+ * @run main/othervm -XX:-UseCompressedOops 
-XX:-UseCompressedClassPointers FieldDensityTest

The test won't run on 32-bit platforms as the compressed oops flags 
won't exist.

---

Some follow up comments below ...

With trimming ...

On 25/01/2020 3:20 am, Frederic Parain wrote:
>> On Jan 24, 2020, at 08:19, David Holmes <david.holmes at oracle.com> wrote:
>>
>> 466     int super_flen   = super->nof_nonstatic_fields();
>>
>> Could be folded directly into the assert so we don't call in product.
> 
> Calling not_nonstatic_fields() has the side effect to compute non-static fields,
> which is required to get a correct value when reading super->_nonstatic_fields,
> so the call is needed even in product builds.

Yuck! That's a horrible side-effect - but not your fault obviously. :) 
It would be better to have a nonstatic_fields() accessor that has the 
same lazy initialization side-effect.

>> General style issue: when breaking a long line with a method call, the new line (containing arguments) should be indented to the opening ( of the method call e.g.
...
>> etc. This applies across all files.
> 
> Fixes applied lines 4003, 4011, 4041, 4138, 4143.

Fix was also needed in other files. Current issues highlighted above.

>>
>> src/hotspot/share/oops/instanceKlass.hpp
>>
>> You need to be careful with _extra_flags usage if there can be concurrently updated bits. At the moment it looks like redefinition is a mutable dynamic property, whilst "contended annotations" should be a static immutable property - is that right?
> 
> Correct, _has_contended_annotations is a static immutable property, while _is_being_redefined is a mutable one.

Good to know. My concern is that if someone adds a new mutable flag bit 
the need for atomic updates may not be noticed. We got bitten by this in 
the past with a flag field and I think we eventually migrated all of the 
mutable bits out into their own field. (Coleen should recall that :) ).

>>   61     FLATTENED,     // flattened field
>>
>> Does this have any meaning before inline types come in?
> 
> Yes, I wanted to reserved the entry in the enum.

Hmmm a tenuous "okay". Seems odd to require this now to support code 
that is still some way from joining mainline.

>> In FieldLayoutBuilder::epilogue you have a number of calls to Thread::current() as well as an implicit call when you use ResourceMarks. You should capture the current thread once in a local and reuse it.
> 
> Fixed

It seems that this fix is now not needed as there is only one use left 
of the new "thread" variable in the ResourceMark. So that can return to 
being:

ResourceMark rm;

Thanks,
David
-----


From matthias.baesken at sap.com  Wed Jan 29 09:38:58 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 29 Jan 2020 09:38:58 +0000
Subject: fopen vs. os::fopen and automatic closing of the file on exec
In-Reply-To: <852b12ce-8a80-eba0-4b6c-ca13a487d7c3@oracle.com>
References: <AM0PR02MB42275083AC9BA465ADCF75B4930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <462ea46a-cef8-79a0-ccf0-71699e619089@oracle.com>
 <AM0PR02MB4227CA25D9DBEDC5A1E81B31930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <852b12ce-8a80-eba0-4b6c-ca13a487d7c3@oracle.com>
Message-ID: <AM0PR02MB4227901491A303191CB55AB793050@AM0PR02MB4227.eurprd02.prod.outlook.com>

Hi David , yes you are probably right, simply adding it always in the jdk  coding  would lead to problems .
On the other hand,  do I  have the  option  to set O_CLOEXEC in some way for  the jdk  open calls , in case I want it ?

Best regards, Matthias



> Hi Matthias,
> 
> On 28/01/2020 11:44 pm, Baesken, Matthias wrote:
> > Hi David, thanks for looking into it .
> >
> > While checking other  open(64) calls, I found  the fileOpen / handleOpen
> calls  in java.base .
> > Looks like  they  miss  so far  setting  FD_CLOEXEC /   O_CLOEXEC  , it isn?t
> done in the callers of those functions  and it is also not (silently) added in the
> fileOpen/handleOpen function itself  (like HS does in os::open ).
> > See :
> >
> > 73FD
> > 74handleOpen(const char *path, int oflag, int mode) {
> > 75    FD fd;
> > 76    RESTARTABLE(open64(path, oflag, mode), fd);
> > 77    if (fd != -1) {
> > 78        struct stat64 buf64;
> > 79        int result;
> > 80        RESTARTABLE(fstat64(fd, &buf64), result);
> > 81        if (result != -1) {
> > 82            if (S_ISDIR(buf64.st_mode)) {
> > 83                close(fd);
> > 84                errno = EISDIR;
> > 85                fd = -1;
> > 86            }
> > 87        } else {
> > 88            close(fd);
> > 89            fd = -1;
> > 90        }
> > 91    }
> > 92    return fd;
> > 93}
> >
> > 95void
> > 96fileOpen(JNIEnv *env, jobject this, jstring path, jfieldID fid, int flags)
> > 97{
> > .....
> > 107        fd = handleOpen(ps, flags, 0666);
> > ....
> > 122}
> >
> > 56JNIEXPORT void JNICALL
> > 57Java_java_io_FileOutputStream_open0(JNIEnv *env, jobject this,
> > 58                                    jstring path, jboolean append) {
> > 59    fileOpen(env, this, path, fos_fd,
> > 60             O_WRONLY | O_CREAT | (append ? O_APPEND : O_TRUNC));
> > 61}
> >
> > 59JNIEXPORT void JNICALL
> > 60Java_java_io_FileInputStream_open0(JNIEnv *env, jobject this, jstring
> path) {
> > 61    fileOpen(env, this, path, fis_fd, O_RDONLY);
> > 62}
> >
> >
> > Is this something that should be changed too ?
> 
> I think not. Unless those public API's are specified to open a file in a
> specific mode, like close-on-exec, then they should not do that by default.
> 
> We should only be doing close-on-exec on internally used files.
> 
> David
> -----


From thomas.stuefe at gmail.com  Wed Jan 29 10:35:15 2020
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Wed, 29 Jan 2020 11:35:15 +0100
Subject: fopen vs. os::fopen and automatic closing of the file on exec
In-Reply-To: <852b12ce-8a80-eba0-4b6c-ca13a487d7c3@oracle.com>
References: <AM0PR02MB42275083AC9BA465ADCF75B4930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <462ea46a-cef8-79a0-ccf0-71699e619089@oracle.com>
 <AM0PR02MB4227CA25D9DBEDC5A1E81B31930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <852b12ce-8a80-eba0-4b6c-ca13a487d7c3@oracle.com>
Message-ID: <CAA-vtUw4k-MZj5s4NP=s9cXt-D9UjfX8oefhOADZFvCSKN8SLA@mail.gmail.com>

Hi David and Matthias,

On Wed 29. Jan 2020 at 00:27, David Holmes <david.holmes at oracle.com> wrote:

> Hi Matthias,
>
> On 28/01/2020 11:44 pm, Baesken, Matthias wrote:
> > Hi David, thanks for looking into it .
> >
> > While checking other  open(64) calls, I found  the fileOpen /
> handleOpen   calls  in java.base .
> > Looks like  they  miss  so far  setting  FD_CLOEXEC /   O_CLOEXEC  , it
> isn?t done in the callers of those functions  and it is also not (silently)
> added in the fileOpen/handleOpen function itself  (like HS does in os::open
> ).
> > See :
> >
> > 73FD
> > 74handleOpen(const char *path, int oflag, int mode) {
> > 75    FD fd;
> > 76    RESTARTABLE(open64(path, oflag, mode), fd);
> > 77    if (fd != -1) {
> > 78        struct stat64 buf64;
> > 79        int result;
> > 80        RESTARTABLE(fstat64(fd, &buf64), result);
> > 81        if (result != -1) {
> > 82            if (S_ISDIR(buf64.st_mode)) {
> > 83                close(fd);
> > 84                errno = EISDIR;
> > 85                fd = -1;
> > 86            }
> > 87        } else {
> > 88            close(fd);
> > 89            fd = -1;
> > 90        }
> > 91    }
> > 92    return fd;
> > 93}
> >
> > 95void
> > 96fileOpen(JNIEnv *env, jobject this, jstring path, jfieldID fid, int
> flags)
> > 97{
> > .....
> > 107        fd = handleOpen(ps, flags, 0666);
> > ....
> > 122}
> >
> > 56JNIEXPORT void JNICALL
> > 57Java_java_io_FileOutputStream_open0(JNIEnv *env, jobject this,
> > 58                                    jstring path, jboolean append) {
> > 59    fileOpen(env, this, path, fos_fd,
> > 60             O_WRONLY | O_CREAT | (append ? O_APPEND : O_TRUNC));
> > 61}
> >
> > 59JNIEXPORT void JNICALL
> > 60Java_java_io_FileInputStream_open0(JNIEnv *env, jobject this, jstring
> path) {
> > 61    fileOpen(env, this, path, fis_fd, O_RDONLY);
> > 62}
> >
> >
> > Is this something that should be changed too ?
>
> I think not. Unless those public API's are specified to open a file in a
> specific mode, like close-on-exec, then they should not do that by default.
>
> We should only be doing close-on-exec on internally used files.
>
> David
> -----


Sorry for chiming in sideways, but I am not so sure. I think it makes sense
to specify close on exec here.

When the Vm forks via Runtime.exec we take pains to manually close all open
file handles in the child process. That means to me that this is the
desired behavior. Leaking file descriptors to child processes is not wanted.

However, we cannot close file descriptors if someone does a naked fork. I?d
think this is a design problem we?d like to fix. The chance that someone
deliberately relies on nakedly forked children inheriting descriptors
opened in the parent vm is smaller than the chance that leaked file
descriptors in child?s cause trouble. These errors are also hard to find.

Btw, maybe that would be better discusses in core libs?

Cheers, thomas


> >
>

From matthias.baesken at sap.com  Wed Jan 29 13:03:19 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 29 Jan 2020 13:03:19 +0000
Subject: fopen vs. os::fopen and automatic closing of the file on exec
In-Reply-To: <CAA-vtUx=G2p71JFL9O5QyVL=ztAuFkcn2aS0ztZY7z+i20UGFg@mail.gmail.com>
References: <AM0PR02MB42275083AC9BA465ADCF75B4930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <462ea46a-cef8-79a0-ccf0-71699e619089@oracle.com>
 <76ff9efd-4274-29dd-c048-02e55aa70890@oracle.com>
 <CAA-vtUx=G2p71JFL9O5QyVL=ztAuFkcn2aS0ztZY7z+i20UGFg@mail.gmail.com>
Message-ID: <AM0PR02MB422786812770E5D21B7C547893050@AM0PR02MB4227.eurprd02.prod.outlook.com>

Hi  Leo / Thomas / David,
 Not sure if it is a small  or ?medium? gain ,  but after the discussion so far I think it makes sense to use  os::fopen in HS code  where possible
 (  what (f)open does in the  jdk C coding is a different thing, we might dicuss this on corelibs-dev  in more detail ).

I opened
https://bugs.openjdk.java.net/browse/JDK-8238161

use os::fopen in HS code where possible


Best regards, Matthias




Hi Leo,

On Tue, Jan 28, 2020 at 4:37 PM Leo Korinth <leo.korinth at oracle.com<mailto:leo.korinth at oracle.com>> wrote:
...

Something that is not obvious is that on unix-like operating systems,
ProcessImpl_md.c tries to close (most) open files between fork and exec.
That is not the case for Windows (I opened
https://bugs.openjdk.java.net/browse/JDK-8202720 for this). Thus (if I
understand correctly) the impact on unix-like operating systems will be
less for adding this support than it is for Windows. os::fopen was
created to solve a specific bug on windows (logging), and was renamed to
the more generic os::fopen during review.

Just a note, it is still possible for file descriptors to escape into child processes since you cannot guarantee that all forks happen via Runtime.exec():
- native third party code may fork.
- hotspot may fork to run tools for error reporting

Also note that the code which closes fds in Runtime.exec() may in theory fail to close all fds.

So I think Matthias makes sense on Posix platforms.

I guess most uses of ::fopen /should/ use the more restricted os::fopen,
but the gain would probably be small.

Thanks,
Leo


From david.holmes at oracle.com  Wed Jan 29 13:22:33 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 29 Jan 2020 23:22:33 +1000
Subject: fopen vs. os::fopen and automatic closing of the file on exec
In-Reply-To: <AM0PR02MB4227901491A303191CB55AB793050@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB42275083AC9BA465ADCF75B4930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <462ea46a-cef8-79a0-ccf0-71699e619089@oracle.com>
 <AM0PR02MB4227CA25D9DBEDC5A1E81B31930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <852b12ce-8a80-eba0-4b6c-ca13a487d7c3@oracle.com>
 <AM0PR02MB4227901491A303191CB55AB793050@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <48a38575-b9d1-97ec-7872-c0d11e3131a5@oracle.com>

On 29/01/2020 7:38 pm, Baesken, Matthias wrote:
> Hi David , yes you are probably right, simply adding it always in the jdk  coding  would lead to problems .
> On the other hand,  do I  have the  option  to set O_CLOEXEC in some way for  the jdk  open calls , in case I want it ?

Not O_CLOEXEC AFAICS:

https://docs.oracle.com/en/java/javase/13/docs/api/java.base/java/nio/file/StandardOpenOption.html

David

> Best regards, Matthias
> 
> 
> 
>> Hi Matthias,
>>
>> On 28/01/2020 11:44 pm, Baesken, Matthias wrote:
>>> Hi David, thanks for looking into it .
>>>
>>> While checking other  open(64) calls, I found  the fileOpen / handleOpen
>> calls  in java.base .
>>> Looks like  they  miss  so far  setting  FD_CLOEXEC /   O_CLOEXEC  , it isn?t
>> done in the callers of those functions  and it is also not (silently) added in the
>> fileOpen/handleOpen function itself  (like HS does in os::open ).
>>> See :
>>>
>>> 73FD
>>> 74handleOpen(const char *path, int oflag, int mode) {
>>> 75    FD fd;
>>> 76    RESTARTABLE(open64(path, oflag, mode), fd);
>>> 77    if (fd != -1) {
>>> 78        struct stat64 buf64;
>>> 79        int result;
>>> 80        RESTARTABLE(fstat64(fd, &buf64), result);
>>> 81        if (result != -1) {
>>> 82            if (S_ISDIR(buf64.st_mode)) {
>>> 83                close(fd);
>>> 84                errno = EISDIR;
>>> 85                fd = -1;
>>> 86            }
>>> 87        } else {
>>> 88            close(fd);
>>> 89            fd = -1;
>>> 90        }
>>> 91    }
>>> 92    return fd;
>>> 93}
>>>
>>> 95void
>>> 96fileOpen(JNIEnv *env, jobject this, jstring path, jfieldID fid, int flags)
>>> 97{
>>> .....
>>> 107        fd = handleOpen(ps, flags, 0666);
>>> ....
>>> 122}
>>>
>>> 56JNIEXPORT void JNICALL
>>> 57Java_java_io_FileOutputStream_open0(JNIEnv *env, jobject this,
>>> 58                                    jstring path, jboolean append) {
>>> 59    fileOpen(env, this, path, fos_fd,
>>> 60             O_WRONLY | O_CREAT | (append ? O_APPEND : O_TRUNC));
>>> 61}
>>>
>>> 59JNIEXPORT void JNICALL
>>> 60Java_java_io_FileInputStream_open0(JNIEnv *env, jobject this, jstring
>> path) {
>>> 61    fileOpen(env, this, path, fis_fd, O_RDONLY);
>>> 62}
>>>
>>>
>>> Is this something that should be changed too ?
>>
>> I think not. Unless those public API's are specified to open a file in a
>> specific mode, like close-on-exec, then they should not do that by default.
>>
>> We should only be doing close-on-exec on internally used files.
>>
>> David
>> -----
> 

From david.holmes at oracle.com  Wed Jan 29 13:25:52 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 29 Jan 2020 23:25:52 +1000
Subject: fopen vs. os::fopen and automatic closing of the file on exec
In-Reply-To: <CAA-vtUw4k-MZj5s4NP=s9cXt-D9UjfX8oefhOADZFvCSKN8SLA@mail.gmail.com>
References: <AM0PR02MB42275083AC9BA465ADCF75B4930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <462ea46a-cef8-79a0-ccf0-71699e619089@oracle.com>
 <AM0PR02MB4227CA25D9DBEDC5A1E81B31930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <852b12ce-8a80-eba0-4b6c-ca13a487d7c3@oracle.com>
 <CAA-vtUw4k-MZj5s4NP=s9cXt-D9UjfX8oefhOADZFvCSKN8SLA@mail.gmail.com>
Message-ID: <580ee126-e180-f04f-3ef7-7f2e3e525121@oracle.com>

Hi Thomas,

On 29/01/2020 8:35 pm, Thomas St?fe wrote:
> Hi David and Matthias,
> 
> On Wed 29. Jan 2020 at 00:27, David Holmes <david.holmes at oracle.com 
> <mailto:david.holmes at oracle.com>> wrote:
> 
>     Hi Matthias,
> 
>     On 28/01/2020 11:44 pm, Baesken, Matthias wrote:
>      > Hi David, thanks for looking into it .
>      >
>      > While checking other? open(64) calls, I found? the fileOpen /
>     handleOpen? ?calls? in java.base .
>      > Looks like? they? miss? so far? setting? FD_CLOEXEC / 
>      ?O_CLOEXEC? , it isn?t done in the callers of those functions? and
>     it is also not (silently) added in the fileOpen/handleOpen function
>     itself? (like HS does in os::open ).
>      > See :
>      >
>      > 73FD
>      > 74handleOpen(const char *path, int oflag, int mode) {
>      > 75? ? FD fd;
>      > 76? ? RESTARTABLE(open64(path, oflag, mode), fd);
>      > 77? ? if (fd != -1) {
>      > 78? ? ? ? struct stat64 buf64;
>      > 79? ? ? ? int result;
>      > 80? ? ? ? RESTARTABLE(fstat64(fd, &buf64), result);
>      > 81? ? ? ? if (result != -1) {
>      > 82? ? ? ? ? ? if (S_ISDIR(buf64.st_mode)) {
>      > 83? ? ? ? ? ? ? ? close(fd);
>      > 84? ? ? ? ? ? ? ? errno = EISDIR;
>      > 85? ? ? ? ? ? ? ? fd = -1;
>      > 86? ? ? ? ? ? }
>      > 87? ? ? ? } else {
>      > 88? ? ? ? ? ? close(fd);
>      > 89? ? ? ? ? ? fd = -1;
>      > 90? ? ? ? }
>      > 91? ? }
>      > 92? ? return fd;
>      > 93}
>      >
>      > 95void
>      > 96fileOpen(JNIEnv *env, jobject this, jstring path, jfieldID fid,
>     int flags)
>      > 97{
>      > .....
>      > 107? ? ? ? fd = handleOpen(ps, flags, 0666);
>      > ....
>      > 122}
>      >
>      > 56JNIEXPORT void JNICALL
>      > 57Java_java_io_FileOutputStream_open0(JNIEnv *env, jobject this,
>      > 58? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? jstring path, jboolean
>     append) {
>      > 59? ? fileOpen(env, this, path, fos_fd,
>      > 60? ? ? ? ? ? ?O_WRONLY | O_CREAT | (append ? O_APPEND : O_TRUNC));
>      > 61}
>      >
>      > 59JNIEXPORT void JNICALL
>      > 60Java_java_io_FileInputStream_open0(JNIEnv *env, jobject this,
>     jstring path) {
>      > 61? ? fileOpen(env, this, path, fis_fd, O_RDONLY);
>      > 62}
>      >
>      >
>      > Is this something that should be changed too ?
> 
>     I think not. Unless those public API's are specified to open a file
>     in a
>     specific mode, like close-on-exec, then they should not do that by
>     default.
> 
>     We should only be doing close-on-exec on internally used files.
> 
>     David
>     -----
> 
> 
> Sorry for chiming in sideways, but I am not so sure. I think it makes 
> sense to specify close on exec here.

It may well make sense but it isn't specified behaviour so would require 
an API specification change.

> When the Vm forks via Runtime.exec we take pains to manually close all 
> open file handles in the child process. That means to me that this is 
> the desired behavior. Leaking file descriptors to child processes is not 
> wanted.
> 
> However, we cannot close file descriptors if someone does a naked fork. 
> I?d think this is a design problem we?d like to fix. The chance that 
> someone deliberately relies on nakedly forked children inheriting 
> descriptors opened in the parent vm is smaller than the chance that 
> leaked file descriptors in child?s cause trouble. These errors are also 
> hard to find.
> 
> Btw, maybe that would be better discusses in core libs?

Yes this is a topic for core-libs or nio-dev. I'm certain this has been 
considered at some point.

Cheers,
David

> Cheers, thomas
> 
> 
>      >
> 

From coleen.phillimore at oracle.com  Wed Jan 29 13:37:42 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Wed, 29 Jan 2020 08:37:42 -0500
Subject: RFR[L]: 8237767 Field layout computation overhaul
In-Reply-To: <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
 <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com>
 <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com>
 <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com>
Message-ID: <a764b642-098b-b0ce-6232-1eed41caf78b@oracle.com>


Snippet reply.

On 1/29/20 2:06 AM, David Holmes wrote:
>
>>>
>>> src/hotspot/share/oops/instanceKlass.hpp
>>>
>>> You need to be careful with _extra_flags usage if there can be 
>>> concurrently updated bits. At the moment it looks like redefinition 
>>> is a mutable dynamic property, whilst "contended annotations" should 
>>> be a static immutable property - is that right?
>>
>> Correct, _has_contended_annotations is a static immutable property, 
>> while _is_being_redefined is a mutable one.
>
> Good to know. My concern is that if someone adds a new mutable flag 
> bit the need for atomic updates may not be noticed. We got bitten by 
> this in the past with a flag field and I think we eventually migrated 
> all of the mutable bits out into their own field. (Coleen should 
> recall that :) ).

A lot of the _misc_flags are mutable.? I think we should notice if 
someone adds a new flag that requires concurrent access, and not waste 
space in InstanceKlass for this flag.?? The other mutable bool doesn't 
seem to have any memory synchronization, iirc, which seems wrong.
>
>>> ? 61 FLATTENED,???? // flattened field
>>>
>>> Does this have any meaning before inline types come in?
>>
>> Yes, I wanted to reserved the entry in the enum.
>
> Hmmm a tenuous "okay". Seems odd to require this now to support code 
> that is still some way from joining mainline.

I didn't think it's a big deal if some small pieces of future code are 
in this change.

Coleen




From david.holmes at oracle.com  Wed Jan 29 13:31:56 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 29 Jan 2020 23:31:56 +1000
Subject: RFR[L]: 8237767 Field layout computation overhaul
In-Reply-To: <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
 <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com>
 <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com>
 <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com>
Message-ID: <77ca9ddd-e633-6b72-4301-b6d16c61509d@oracle.com>

PS. I've now seen the CSR and have commented there. I don't agree with 
the use of a diagnostic flag. It doesn't buy us anything other than not 
needing a CSR request when we decide to deprecate it. But if we're going 
to tell people to use these flags to revert behaviour then they will 
need to follow the deprecation process regardless - and a product flag 
is better from an end user perspective.

Cheers,
David

On 29/01/2020 5:06 pm, David Holmes wrote:
> Hi Fred,
> 
> I've looked at the v7 version. A few more stylistic comments on that 
> first. Note, no need for an item by item response unless that makes it 
> easier for you to track :)
> 
> src/hotspot/share/classfile/classFileParser.cpp
> 
>  ?#include "classfile/defaultMethods.hpp"
> +#include "classfile/fieldLayoutBuilder.hpp"
>  ?#include "classfile/dictionary.hpp"
> 
> Include files are not in alphabetical order.
> 
> +?? * This may well change: FixMe if doesn't,
> 
> s/if/if it/
> 
> +? //Make a temp copy, and iterate through and copy back into the orig
> 
> Space after //
> 
> s/orig/original/
> 
> +? OopMapBlock*? nonstatic_oop_map = _nonstatic_oop_maps;
> 
> Extra space after *
> 
> +? assert(ik->nonstatic_oop_map_count() == 
> _field_info->oop_map_blocks->_nonstatic_oop_map_count,
> +???? "sanity");
> 
> Second line needs to be indented further:
> 
>  ?? assert(ik->nonstatic_oop_map_count() == 
> _field_info->oop_map_blocks->_nonstatic_oop_map_count,
>  ????????? "sanity");
> 
> ---
> 
> src/hotspot/share/classfile/classFileParser.hpp
> 
> +public:
> +? OopMapBlock* _nonstatic_oop_maps;
> +? unsigned int _nonstatic_oop_map_count;
> +? unsigned int _max_nonstatic_oop_maps;
> +
> + public:
> 
> Second public uneeded. First public may be indented wrong (I'm not sure 
> what the rule is - single space indent?)
> 
>  ?class ClassFileParser {
> +? friend class FieldLayoutBuilder;
> +? friend class FieldLayout;
> 
>  ? class ClassAnnotationCollector;
>  ? class FieldAllocationCount;
>  ? class FieldAnnotationCollector;
> 
> Indents are different. I think the class forward declarations should 
> have extra space.
> 
> ---
> 
> src/hotspot/share/oops/instanceKlass.hpp
> 
> +? void increment_count(int diff)???? { _count += diff; }
> 
> Extra spaces before {
> 
> ---
> 
> src/hotspot/share/runtime/globals.hpp
> 
> +? diagnostic(bool, UseNewFieldLayout, true, ??? \
> +?????????????? "Use new algorithm to compute layouts") ??? \
> + ??? \
> +? product(bool, UseEmptySlotsInSupers, true,
> 
> Not sure I see why one flag is diagnostic and the other product. Do you 
> expect people to need to disable using empty slots more so than needing 
> to disable using the new field layout altogether?
> 
> ---
> 
> src/hotspot/share/classfile/fieldLayoutBuilder.cpp
> 
> +? assert(kind == EMPTY || kind == RESERVED || kind == PADDING || kind 
> == INHERITED,
> +????? "Otherwise, should use the constructor with a field index 
> argument");
> 
> Indentation of second line is wrong.
> 
> +? assert(kind == REGULAR || kind == FLATTENED || kind == INHERITED,
> +????? "Other kind do not have a field index");
> 
> Ditto.
> 
> 
> +? if (list == NULL) return;
> +? if (start == NULL) {
> +??? start = this->_start;
> +? }
> 
> Inconsistent style for single statement if-blocks. Same thing later in 
> the file.
> 
> +????? output->print_cr(" @%d \"%s\" %s %d/%d %s",
> +????????? b->offset(),
> +????????? fi->name(_cp)->as_C_string(),
> +????????? fi->signature(_cp)->as_C_string(),
> +????????? b->size(),
> +????????? b->alignment(),
> +????????? "REGULAR");
> 
> Incorrect identation of continuing line. Same for all the following 
> print blocks.
> 
> +? } else if (_classname == vmSymbols::java_lang_Boolean() ||
> +????? _classname == vmSymbols::java_lang_Character() ||
> +????? _classname == vmSymbols::java_lang_Float() ||
> +????? _classname == vmSymbols::java_lang_Double() ||
> +????? _classname == vmSymbols::java_lang_Byte() ||
> +????? _classname == vmSymbols::java_lang_Short() ||
> +????? _classname == vmSymbols::java_lang_Integer() ||
> +????? _classname == vmSymbols::java_lang_Long()) {
> 
> Incorrect identation of continuing line.
> 
> ---
> 
> src/hotspot/share/classfile/fieldLayoutBuilder.hpp
> 
> +// and the boxing classes). The rational for having multiple methods
> 
> s/rational/rationale/
> 
> +? FieldLayoutBuilder(const Symbol* classname, const InstanceKlass* 
> super_klass, ConstantPool* constant_pool,
> +????? Array<u2>* fields, bool is_contended, FieldLayoutInfo* info);
> 
> Indentation wrong for continuing line.
> 
> +? int get_alignment() {
> +???? assert(_alignment != -1, "Uninitialized");
> +???? return _alignment;
> +?? }
> 
> Indenting appears off by one.
> 
> ---
> 
> test/hotspot/jtreg/runtime/FieldLayout/FieldDensityTest.java
> 
> + * @run main/othervm -XX:+UseCompressedOops 
> -XX:+UseCompressedClassPointers FieldDensityTest
> + * @run main/othervm -XX:+UseCompressedOops 
> -XX:-UseCompressedClassPointers FieldDensityTest
> + * @run main/othervm -XX:-UseCompressedOops 
> -XX:-UseCompressedClassPointers FieldDensityTest
> 
> The test won't run on 32-bit platforms as the compressed oops flags 
> won't exist.
> 
> ---
> 
> Some follow up comments below ...
> 
> With trimming ...
> 
> On 25/01/2020 3:20 am, Frederic Parain wrote:
>>> On Jan 24, 2020, at 08:19, David Holmes <david.holmes at oracle.com> wrote:
>>>
>>> 466???? int super_flen?? = super->nof_nonstatic_fields();
>>>
>>> Could be folded directly into the assert so we don't call in product.
>>
>> Calling not_nonstatic_fields() has the side effect to compute 
>> non-static fields,
>> which is required to get a correct value when reading 
>> super->_nonstatic_fields,
>> so the call is needed even in product builds.
> 
> Yuck! That's a horrible side-effect - but not your fault obviously. :) 
> It would be better to have a nonstatic_fields() accessor that has the 
> same lazy initialization side-effect.
> 
>>> General style issue: when breaking a long line with a method call, 
>>> the new line (containing arguments) should be indented to the opening 
>>> ( of the method call e.g.
> ...
>>> etc. This applies across all files.
>>
>> Fixes applied lines 4003, 4011, 4041, 4138, 4143.
> 
> Fix was also needed in other files. Current issues highlighted above.
> 
>>>
>>> src/hotspot/share/oops/instanceKlass.hpp
>>>
>>> You need to be careful with _extra_flags usage if there can be 
>>> concurrently updated bits. At the moment it looks like redefinition 
>>> is a mutable dynamic property, whilst "contended annotations" should 
>>> be a static immutable property - is that right?
>>
>> Correct, _has_contended_annotations is a static immutable property, 
>> while _is_being_redefined is a mutable one.
> 
> Good to know. My concern is that if someone adds a new mutable flag bit 
> the need for atomic updates may not be noticed. We got bitten by this in 
> the past with a flag field and I think we eventually migrated all of the 
> mutable bits out into their own field. (Coleen should recall that :) ).
> 
>>> ? 61???? FLATTENED,???? // flattened field
>>>
>>> Does this have any meaning before inline types come in?
>>
>> Yes, I wanted to reserved the entry in the enum.
> 
> Hmmm a tenuous "okay". Seems odd to require this now to support code 
> that is still some way from joining mainline.
> 
>>> In FieldLayoutBuilder::epilogue you have a number of calls to 
>>> Thread::current() as well as an implicit call when you use 
>>> ResourceMarks. You should capture the current thread once in a local 
>>> and reuse it.
>>
>> Fixed
> 
> It seems that this fix is now not needed as there is only one use left 
> of the new "thread" variable in the ResourceMark. So that can return to 
> being:
> 
> ResourceMark rm;
> 
> Thanks,
> David
> -----
> 

From david.holmes at oracle.com  Wed Jan 29 13:33:14 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 29 Jan 2020 23:33:14 +1000
Subject: RFR[L]: 8237767 Field layout computation overhaul
In-Reply-To: <a764b642-098b-b0ce-6232-1eed41caf78b@oracle.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
 <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com>
 <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com>
 <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com>
 <a764b642-098b-b0ce-6232-1eed41caf78b@oracle.com>
Message-ID: <4b1fd6f6-b0a6-b2bb-8eb6-71ac2cf49f3a@oracle.com>

Hi Coleen,

On 29/01/2020 11:37 pm, coleen.phillimore at oracle.com wrote:
> 
> Snippet reply.
> 
> On 1/29/20 2:06 AM, David Holmes wrote:
>>
>>>>
>>>> src/hotspot/share/oops/instanceKlass.hpp
>>>>
>>>> You need to be careful with _extra_flags usage if there can be 
>>>> concurrently updated bits. At the moment it looks like redefinition 
>>>> is a mutable dynamic property, whilst "contended annotations" should 
>>>> be a static immutable property - is that right?
>>>
>>> Correct, _has_contended_annotations is a static immutable property, 
>>> while _is_being_redefined is a mutable one.
>>
>> Good to know. My concern is that if someone adds a new mutable flag 
>> bit the need for atomic updates may not be noticed. We got bitten by 
>> this in the past with a flag field and I think we eventually migrated 
>> all of the mutable bits out into their own field. (Coleen should 
>> recall that :) ).
> 
> A lot of the _misc_flags are mutable.? I think we should notice if 
> someone adds a new flag that requires concurrent access, and not waste 
> space in InstanceKlass for this flag.?? The other mutable bool doesn't 
> seem to have any memory synchronization, iirc, which seems wrong.

I have a distinct recollection that we were bitten by this in the past.

Cheers,
David

>>
>>>> ? 61 FLATTENED,???? // flattened field
>>>>
>>>> Does this have any meaning before inline types come in?
>>>
>>> Yes, I wanted to reserved the entry in the enum.
>>
>> Hmmm a tenuous "okay". Seems odd to require this now to support code 
>> that is still some way from joining mainline.
> 
> I didn't think it's a big deal if some small pieces of future code are 
> in this change.
> 
> Coleen
> 
> 
> 

From coleen.phillimore at oracle.com  Wed Jan 29 13:44:57 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Wed, 29 Jan 2020 08:44:57 -0500
Subject: RFR[L]: 8237767 Field layout computation overhaul
In-Reply-To: <77ca9ddd-e633-6b72-4301-b6d16c61509d@oracle.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
 <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com>
 <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com>
 <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com>
 <77ca9ddd-e633-6b72-4301-b6d16c61509d@oracle.com>
Message-ID: <4bd88392-338a-6287-a16e-1fb3949b85f7@oracle.com>



On 1/29/20 8:31 AM, David Holmes wrote:
> PS. I've now seen the CSR and have commented there. I don't agree with 
> the use of a diagnostic flag. It doesn't buy us anything other than 
> not needing a CSR request when we decide to deprecate it. But if we're 
> going to tell people to use these flags to revert behaviour then they 
> will need to follow the deprecation process regardless - and a product 
> flag is better from an end user perspective.

Obviously I disagree.? We don't want to publicise this flag *unless* 
someone has a problem, which makes it a diagnostic flag.?? We don't want 
to have to carry the old code 2 releases when there is honestly no 
reason to suspect it is problematic.

Coleen

>
> Cheers,
> David
>
> On 29/01/2020 5:06 pm, David Holmes wrote:
>> Hi Fred,
>>
>> I've looked at the v7 version. A few more stylistic comments on that 
>> first. Note, no need for an item by item response unless that makes 
>> it easier for you to track :)
>>
>> src/hotspot/share/classfile/classFileParser.cpp
>>
>> ??#include "classfile/defaultMethods.hpp"
>> +#include "classfile/fieldLayoutBuilder.hpp"
>> ??#include "classfile/dictionary.hpp"
>>
>> Include files are not in alphabetical order.
>>
>> +?? * This may well change: FixMe if doesn't,
>>
>> s/if/if it/
>>
>> +? //Make a temp copy, and iterate through and copy back into the orig
>>
>> Space after //
>>
>> s/orig/original/
>>
>> +? OopMapBlock*? nonstatic_oop_map = _nonstatic_oop_maps;
>>
>> Extra space after *
>>
>> +? assert(ik->nonstatic_oop_map_count() == 
>> _field_info->oop_map_blocks->_nonstatic_oop_map_count,
>> +???? "sanity");
>>
>> Second line needs to be indented further:
>>
>> ??? assert(ik->nonstatic_oop_map_count() == 
>> _field_info->oop_map_blocks->_nonstatic_oop_map_count,
>> ?????????? "sanity");
>>
>> ---
>>
>> src/hotspot/share/classfile/classFileParser.hpp
>>
>> +public:
>> +? OopMapBlock* _nonstatic_oop_maps;
>> +? unsigned int _nonstatic_oop_map_count;
>> +? unsigned int _max_nonstatic_oop_maps;
>> +
>> + public:
>>
>> Second public uneeded. First public may be indented wrong (I'm not 
>> sure what the rule is - single space indent?)
>>
>> ??class ClassFileParser {
>> +? friend class FieldLayoutBuilder;
>> +? friend class FieldLayout;
>>
>> ?? class ClassAnnotationCollector;
>> ?? class FieldAllocationCount;
>> ?? class FieldAnnotationCollector;
>>
>> Indents are different. I think the class forward declarations should 
>> have extra space.
>>
>> ---
>>
>> src/hotspot/share/oops/instanceKlass.hpp
>>
>> +? void increment_count(int diff)???? { _count += diff; }
>>
>> Extra spaces before {
>>
>> ---
>>
>> src/hotspot/share/runtime/globals.hpp
>>
>> +? diagnostic(bool, UseNewFieldLayout, true, ??? \
>> +?????????????? "Use new algorithm to compute layouts") ??? \
>> + ??? \
>> +? product(bool, UseEmptySlotsInSupers, true,
>>
>> Not sure I see why one flag is diagnostic and the other product. Do 
>> you expect people to need to disable using empty slots more so than 
>> needing to disable using the new field layout altogether?
>>
>> ---
>>
>> src/hotspot/share/classfile/fieldLayoutBuilder.cpp
>>
>> +? assert(kind == EMPTY || kind == RESERVED || kind == PADDING || 
>> kind == INHERITED,
>> +????? "Otherwise, should use the constructor with a field index 
>> argument");
>>
>> Indentation of second line is wrong.
>>
>> +? assert(kind == REGULAR || kind == FLATTENED || kind == INHERITED,
>> +????? "Other kind do not have a field index");
>>
>> Ditto.
>>
>>
>> +? if (list == NULL) return;
>> +? if (start == NULL) {
>> +??? start = this->_start;
>> +? }
>>
>> Inconsistent style for single statement if-blocks. Same thing later 
>> in the file.
>>
>> +????? output->print_cr(" @%d \"%s\" %s %d/%d %s",
>> +????????? b->offset(),
>> +????????? fi->name(_cp)->as_C_string(),
>> +????????? fi->signature(_cp)->as_C_string(),
>> +????????? b->size(),
>> +????????? b->alignment(),
>> +????????? "REGULAR");
>>
>> Incorrect identation of continuing line. Same for all the following 
>> print blocks.
>>
>> +? } else if (_classname == vmSymbols::java_lang_Boolean() ||
>> +????? _classname == vmSymbols::java_lang_Character() ||
>> +????? _classname == vmSymbols::java_lang_Float() ||
>> +????? _classname == vmSymbols::java_lang_Double() ||
>> +????? _classname == vmSymbols::java_lang_Byte() ||
>> +????? _classname == vmSymbols::java_lang_Short() ||
>> +????? _classname == vmSymbols::java_lang_Integer() ||
>> +????? _classname == vmSymbols::java_lang_Long()) {
>>
>> Incorrect identation of continuing line.
>>
>> ---
>>
>> src/hotspot/share/classfile/fieldLayoutBuilder.hpp
>>
>> +// and the boxing classes). The rational for having multiple methods
>>
>> s/rational/rationale/
>>
>> +? FieldLayoutBuilder(const Symbol* classname, const InstanceKlass* 
>> super_klass, ConstantPool* constant_pool,
>> +????? Array<u2>* fields, bool is_contended, FieldLayoutInfo* info);
>>
>> Indentation wrong for continuing line.
>>
>> +? int get_alignment() {
>> +???? assert(_alignment != -1, "Uninitialized");
>> +???? return _alignment;
>> +?? }
>>
>> Indenting appears off by one.
>>
>> ---
>>
>> test/hotspot/jtreg/runtime/FieldLayout/FieldDensityTest.java
>>
>> + * @run main/othervm -XX:+UseCompressedOops 
>> -XX:+UseCompressedClassPointers FieldDensityTest
>> + * @run main/othervm -XX:+UseCompressedOops 
>> -XX:-UseCompressedClassPointers FieldDensityTest
>> + * @run main/othervm -XX:-UseCompressedOops 
>> -XX:-UseCompressedClassPointers FieldDensityTest
>>
>> The test won't run on 32-bit platforms as the compressed oops flags 
>> won't exist.
>>
>> ---
>>
>> Some follow up comments below ...
>>
>> With trimming ...
>>
>> On 25/01/2020 3:20 am, Frederic Parain wrote:
>>>> On Jan 24, 2020, at 08:19, David Holmes <david.holmes at oracle.com> 
>>>> wrote:
>>>>
>>>> 466???? int super_flen?? = super->nof_nonstatic_fields();
>>>>
>>>> Could be folded directly into the assert so we don't call in product.
>>>
>>> Calling not_nonstatic_fields() has the side effect to compute 
>>> non-static fields,
>>> which is required to get a correct value when reading 
>>> super->_nonstatic_fields,
>>> so the call is needed even in product builds.
>>
>> Yuck! That's a horrible side-effect - but not your fault obviously. 
>> :) It would be better to have a nonstatic_fields() accessor that has 
>> the same lazy initialization side-effect.
>>
>>>> General style issue: when breaking a long line with a method call, 
>>>> the new line (containing arguments) should be indented to the 
>>>> opening ( of the method call e.g.
>> ...
>>>> etc. This applies across all files.
>>>
>>> Fixes applied lines 4003, 4011, 4041, 4138, 4143.
>>
>> Fix was also needed in other files. Current issues highlighted above.
>>
>>>>
>>>> src/hotspot/share/oops/instanceKlass.hpp
>>>>
>>>> You need to be careful with _extra_flags usage if there can be 
>>>> concurrently updated bits. At the moment it looks like redefinition 
>>>> is a mutable dynamic property, whilst "contended annotations" 
>>>> should be a static immutable property - is that right?
>>>
>>> Correct, _has_contended_annotations is a static immutable property, 
>>> while _is_being_redefined is a mutable one.
>>
>> Good to know. My concern is that if someone adds a new mutable flag 
>> bit the need for atomic updates may not be noticed. We got bitten by 
>> this in the past with a flag field and I think we eventually migrated 
>> all of the mutable bits out into their own field. (Coleen should 
>> recall that :) ).
>>
>>>> ? 61???? FLATTENED,???? // flattened field
>>>>
>>>> Does this have any meaning before inline types come in?
>>>
>>> Yes, I wanted to reserved the entry in the enum.
>>
>> Hmmm a tenuous "okay". Seems odd to require this now to support code 
>> that is still some way from joining mainline.
>>
>>>> In FieldLayoutBuilder::epilogue you have a number of calls to 
>>>> Thread::current() as well as an implicit call when you use 
>>>> ResourceMarks. You should capture the current thread once in a 
>>>> local and reuse it.
>>>
>>> Fixed
>>
>> It seems that this fix is now not needed as there is only one use 
>> left of the new "thread" variable in the ResourceMark. So that can 
>> return to being:
>>
>> ResourceMark rm;
>>
>> Thanks,
>> David
>> -----
>>


From coleen.phillimore at oracle.com  Wed Jan 29 13:49:38 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Wed, 29 Jan 2020 08:49:38 -0500
Subject: RFR[L]: 8237767 Field layout computation overhaul
In-Reply-To: <4b1fd6f6-b0a6-b2bb-8eb6-71ac2cf49f3a@oracle.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
 <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com>
 <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com>
 <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com>
 <a764b642-098b-b0ce-6232-1eed41caf78b@oracle.com>
 <4b1fd6f6-b0a6-b2bb-8eb6-71ac2cf49f3a@oracle.com>
Message-ID: <4702c40b-1bff-0135-1c5e-e37813ae4ee8@oracle.com>



On 1/29/20 8:33 AM, David Holmes wrote:
> Hi Coleen,
>
> On 29/01/2020 11:37 pm, coleen.phillimore at oracle.com wrote:
>>
>> Snippet reply.
>>
>> On 1/29/20 2:06 AM, David Holmes wrote:
>>>
>>>>>
>>>>> src/hotspot/share/oops/instanceKlass.hpp
>>>>>
>>>>> You need to be careful with _extra_flags usage if there can be 
>>>>> concurrently updated bits. At the moment it looks like 
>>>>> redefinition is a mutable dynamic property, whilst "contended 
>>>>> annotations" should be a static immutable property - is that right?
>>>>
>>>> Correct, _has_contended_annotations is a static immutable property, 
>>>> while _is_being_redefined is a mutable one.
>>>
>>> Good to know. My concern is that if someone adds a new mutable flag 
>>> bit the need for atomic updates may not be noticed. We got bitten by 
>>> this in the past with a flag field and I think we eventually 
>>> migrated all of the mutable bits out into their own field. (Coleen 
>>> should recall that :) ).
>>
>> A lot of the _misc_flags are mutable.? I think we should notice if 
>> someone adds a new flag that requires concurrent access, and not 
>> waste space in InstanceKlass for this flag.?? The other mutable bool 
>> doesn't seem to have any memory synchronization, iirc, which seems 
>> wrong.
>
> I have a distinct recollection that we were bitten by this in the past.

We were bitten by this one:
 ? bool??????????? _is_marked_dependent;? // used for marking during 
flushing and deoptimization

We think, but my memory is bad.? If we want to make some access to the 
misc flags thread safe, we can use the same mechanism as accessFlags.?? 
This seems unnecessary at this time.

Coleen
>
> Cheers,
> David
>
>>>
>>>>> ? 61 FLATTENED,???? // flattened field
>>>>>
>>>>> Does this have any meaning before inline types come in?
>>>>
>>>> Yes, I wanted to reserved the entry in the enum.
>>>
>>> Hmmm a tenuous "okay". Seems odd to require this now to support code 
>>> that is still some way from joining mainline.
>>
>> I didn't think it's a big deal if some small pieces of future code 
>> are in this change.
>>
>> Coleen
>>
>>
>>


From daniel.daugherty at oracle.com  Wed Jan 29 14:34:05 2020
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Wed, 29 Jan 2020 09:34:05 -0500
Subject: RFR[L]: 8237767 Field layout computation overhaul
In-Reply-To: <4bd88392-338a-6287-a16e-1fb3949b85f7@oracle.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
 <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com>
 <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com>
 <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com>
 <77ca9ddd-e633-6b72-4301-b6d16c61509d@oracle.com>
 <4bd88392-338a-6287-a16e-1fb3949b85f7@oracle.com>
Message-ID: <0813a9ec-24e9-a407-d7dc-2e0d7380b0ca@oracle.com>

On 1/29/20 8:44 AM, coleen.phillimore at oracle.com wrote:
> On 1/29/20 8:31 AM, David Holmes wrote:
>> PS. I've now seen the CSR and have commented there. I don't agree 
>> with the use of a diagnostic flag. It doesn't buy us anything other 
>> than not needing a CSR request when we decide to deprecate it. But if 
>> we're going to tell people to use these flags to revert behaviour 
>> then they will need to follow the deprecation process regardless - 
>> and a product flag is better from an end user perspective.
>
> Obviously I disagree.? We don't want to publicise this flag *unless* 
> someone has a problem, which makes it a diagnostic flag.?? We don't 
> want to have to carry the old code 2 releases when there is honestly 
> no reason to suspect it is problematic.
>
> Coleen

I'm only chiming in on this one snippet from the review.

I'm obviously a big fan of diagnostic flags instead of product
flags. Based on what I've read about this new flag, I would vote
that it should be a diagnostic flag and not a product flag. To
be a product flag, you should have to justify the expense of
keeping/maintaining the alternate code for the two releases that
it would take to remove it.

My $0.02...

Dan


From frederic.parain at oracle.com  Wed Jan 29 14:57:43 2020
From: frederic.parain at oracle.com (Frederic Parain)
Date: Wed, 29 Jan 2020 09:57:43 -0500
Subject: RFR[L]: 8237767 Field layout computation overhaul
In-Reply-To: <4bd88392-338a-6287-a16e-1fb3949b85f7@oracle.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
 <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com>
 <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com>
 <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com>
 <77ca9ddd-e633-6b72-4301-b6d16c61509d@oracle.com>
 <4bd88392-338a-6287-a16e-1fb3949b85f7@oracle.com>
Message-ID: <BFA088F7-E55B-4B47-A1CB-61A9E6900D19@oracle.com>

The kind of the flag is a secondary issue, the main issue is ?how
fast do we want to remove the old code??. I wanted to keep the old
code for a short transition period in case new field layouts cause
issues for some tools or customers. The goal was not to keep it
forever and allow users to select which algorithm they want to use.

Note that keeping the old code and its activation with a normal
product flag will be an issue for the first release of Valhalla.
The way flattened fields are allocated in the old code is very
inefficient, which means that instead of seeing an improvement
in data density, users will see a significant increase in space
consumption if they use the old code.

So, here?s the three possible paths forward:
  1 - push the new code and remove the old code in a single change,
      so no need for a VM option to select which code to use
  2 - push the new code while keeping the old one for a short
      period of time, to remove the old code before Valhalla code
      is pushed
  3 - push the new code while keeping the old one with a long
      deprecation period before removing the old code

Option 3 will be an issue for project Valhalla, as explained
above.

Option 1 sounds a big risky to me, from the experience I had
while doing this change in the JVM and the unexpected places
where it caused issues.

Option 2 sounded like a good compromise, however, it seems
there?s no consensus how to implement it. Could we refine
the details on how a short transition period could be implemented?

Thank you,

Fred


> On Jan 29, 2020, at 08:44, coleen.phillimore at oracle.com wrote:
> 
> 
> 
> On 1/29/20 8:31 AM, David Holmes wrote:
>> PS. I've now seen the CSR and have commented there. I don't agree with the use of a diagnostic flag. It doesn't buy us anything other than not needing a CSR request when we decide to deprecate it. But if we're going to tell people to use these flags to revert behaviour then they will need to follow the deprecation process regardless - and a product flag is better from an end user perspective.
> 
> Obviously I disagree.  We don't want to publicise this flag *unless* someone has a problem, which makes it a diagnostic flag.   We don't want to have to carry the old code 2 releases when there is honestly no reason to suspect it is problematic.
> 
> Coleen
> 
>> 
>> Cheers,
>> David
>> 
>> On 29/01/2020 5:06 pm, David Holmes wrote:
>>> Hi Fred,
>>> 
>>> I've looked at the v7 version. A few more stylistic comments on that first. Note, no need for an item by item response unless that makes it easier for you to track :)
>>> 
>>> src/hotspot/share/classfile/classFileParser.cpp
>>> 
>>>   #include "classfile/defaultMethods.hpp"
>>> +#include "classfile/fieldLayoutBuilder.hpp"
>>>   #include "classfile/dictionary.hpp"
>>> 
>>> Include files are not in alphabetical order.
>>> 
>>> +   * This may well change: FixMe if doesn't,
>>> 
>>> s/if/if it/
>>> 
>>> +  //Make a temp copy, and iterate through and copy back into the orig
>>> 
>>> Space after //
>>> 
>>> s/orig/original/
>>> 
>>> +  OopMapBlock*  nonstatic_oop_map = _nonstatic_oop_maps;
>>> 
>>> Extra space after *
>>> 
>>> +  assert(ik->nonstatic_oop_map_count() == _field_info->oop_map_blocks->_nonstatic_oop_map_count,
>>> +     "sanity");
>>> 
>>> Second line needs to be indented further:
>>> 
>>>     assert(ik->nonstatic_oop_map_count() == _field_info->oop_map_blocks->_nonstatic_oop_map_count,
>>>            "sanity");
>>> 
>>> ---
>>> 
>>> src/hotspot/share/classfile/classFileParser.hpp
>>> 
>>> +public:
>>> +  OopMapBlock* _nonstatic_oop_maps;
>>> +  unsigned int _nonstatic_oop_map_count;
>>> +  unsigned int _max_nonstatic_oop_maps;
>>> +
>>> + public:
>>> 
>>> Second public uneeded. First public may be indented wrong (I'm not sure what the rule is - single space indent?)
>>> 
>>>   class ClassFileParser {
>>> +  friend class FieldLayoutBuilder;
>>> +  friend class FieldLayout;
>>> 
>>>    class ClassAnnotationCollector;
>>>    class FieldAllocationCount;
>>>    class FieldAnnotationCollector;
>>> 
>>> Indents are different. I think the class forward declarations should have extra space.
>>> 
>>> ---
>>> 
>>> src/hotspot/share/oops/instanceKlass.hpp
>>> 
>>> +  void increment_count(int diff)     { _count += diff; }
>>> 
>>> Extra spaces before {
>>> 
>>> ---
>>> 
>>> src/hotspot/share/runtime/globals.hpp
>>> 
>>> +  diagnostic(bool, UseNewFieldLayout, true,     \
>>> +               "Use new algorithm to compute layouts")     \
>>> +     \
>>> +  product(bool, UseEmptySlotsInSupers, true,
>>> 
>>> Not sure I see why one flag is diagnostic and the other product. Do you expect people to need to disable using empty slots more so than needing to disable using the new field layout altogether?
>>> 
>>> ---
>>> 
>>> src/hotspot/share/classfile/fieldLayoutBuilder.cpp
>>> 
>>> +  assert(kind == EMPTY || kind == RESERVED || kind == PADDING || kind == INHERITED,
>>> +      "Otherwise, should use the constructor with a field index argument");
>>> 
>>> Indentation of second line is wrong.
>>> 
>>> +  assert(kind == REGULAR || kind == FLATTENED || kind == INHERITED,
>>> +      "Other kind do not have a field index");
>>> 
>>> Ditto.
>>> 
>>> 
>>> +  if (list == NULL) return;
>>> +  if (start == NULL) {
>>> +    start = this->_start;
>>> +  }
>>> 
>>> Inconsistent style for single statement if-blocks. Same thing later in the file.
>>> 
>>> +      output->print_cr(" @%d \"%s\" %s %d/%d %s",
>>> +          b->offset(),
>>> +          fi->name(_cp)->as_C_string(),
>>> +          fi->signature(_cp)->as_C_string(),
>>> +          b->size(),
>>> +          b->alignment(),
>>> +          "REGULAR");
>>> 
>>> Incorrect identation of continuing line. Same for all the following print blocks.
>>> 
>>> +  } else if (_classname == vmSymbols::java_lang_Boolean() ||
>>> +      _classname == vmSymbols::java_lang_Character() ||
>>> +      _classname == vmSymbols::java_lang_Float() ||
>>> +      _classname == vmSymbols::java_lang_Double() ||
>>> +      _classname == vmSymbols::java_lang_Byte() ||
>>> +      _classname == vmSymbols::java_lang_Short() ||
>>> +      _classname == vmSymbols::java_lang_Integer() ||
>>> +      _classname == vmSymbols::java_lang_Long()) {
>>> 
>>> Incorrect identation of continuing line.
>>> 
>>> ---
>>> 
>>> src/hotspot/share/classfile/fieldLayoutBuilder.hpp
>>> 
>>> +// and the boxing classes). The rational for having multiple methods
>>> 
>>> s/rational/rationale/
>>> 
>>> +  FieldLayoutBuilder(const Symbol* classname, const InstanceKlass* super_klass, ConstantPool* constant_pool,
>>> +      Array<u2>* fields, bool is_contended, FieldLayoutInfo* info);
>>> 
>>> Indentation wrong for continuing line.
>>> 
>>> +  int get_alignment() {
>>> +     assert(_alignment != -1, "Uninitialized");
>>> +     return _alignment;
>>> +   }
>>> 
>>> Indenting appears off by one.
>>> 
>>> ---
>>> 
>>> test/hotspot/jtreg/runtime/FieldLayout/FieldDensityTest.java
>>> 
>>> + * @run main/othervm -XX:+UseCompressedOops -XX:+UseCompressedClassPointers FieldDensityTest
>>> + * @run main/othervm -XX:+UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest
>>> + * @run main/othervm -XX:-UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest
>>> 
>>> The test won't run on 32-bit platforms as the compressed oops flags won't exist.
>>> 
>>> ---
>>> 
>>> Some follow up comments below ...
>>> 
>>> With trimming ...
>>> 
>>> On 25/01/2020 3:20 am, Frederic Parain wrote:
>>>>> On Jan 24, 2020, at 08:19, David Holmes <david.holmes at oracle.com> wrote:
>>>>> 
>>>>> 466     int super_flen   = super->nof_nonstatic_fields();
>>>>> 
>>>>> Could be folded directly into the assert so we don't call in product.
>>>> 
>>>> Calling not_nonstatic_fields() has the side effect to compute non-static fields,
>>>> which is required to get a correct value when reading super->_nonstatic_fields,
>>>> so the call is needed even in product builds.
>>> 
>>> Yuck! That's a horrible side-effect - but not your fault obviously. :) It would be better to have a nonstatic_fields() accessor that has the same lazy initialization side-effect.
>>> 
>>>>> General style issue: when breaking a long line with a method call, the new line (containing arguments) should be indented to the opening ( of the method call e.g.
>>> ...
>>>>> etc. This applies across all files.
>>>> 
>>>> Fixes applied lines 4003, 4011, 4041, 4138, 4143.
>>> 
>>> Fix was also needed in other files. Current issues highlighted above.
>>> 
>>>>> 
>>>>> src/hotspot/share/oops/instanceKlass.hpp
>>>>> 
>>>>> You need to be careful with _extra_flags usage if there can be concurrently updated bits. At the moment it looks like redefinition is a mutable dynamic property, whilst "contended annotations" should be a static immutable property - is that right?
>>>> 
>>>> Correct, _has_contended_annotations is a static immutable property, while _is_being_redefined is a mutable one.
>>> 
>>> Good to know. My concern is that if someone adds a new mutable flag bit the need for atomic updates may not be noticed. We got bitten by this in the past with a flag field and I think we eventually migrated all of the mutable bits out into their own field. (Coleen should recall that :) ).
>>> 
>>>>>   61     FLATTENED,     // flattened field
>>>>> 
>>>>> Does this have any meaning before inline types come in?
>>>> 
>>>> Yes, I wanted to reserved the entry in the enum.
>>> 
>>> Hmmm a tenuous "okay". Seems odd to require this now to support code that is still some way from joining mainline.
>>> 
>>>>> In FieldLayoutBuilder::epilogue you have a number of calls to Thread::current() as well as an implicit call when you use ResourceMarks. You should capture the current thread once in a local and reuse it.
>>>> 
>>>> Fixed
>>> 
>>> It seems that this fix is now not needed as there is only one use left of the new "thread" variable in the ResourceMark. So that can return to being:
>>> 
>>> ResourceMark rm;
>>> 
>>> Thanks,
>>> David
>>> -----
>>> 
> 


From lutz.schmidt at sap.com  Wed Jan 29 16:48:47 2020
From: lutz.schmidt at sap.com (Schmidt, Lutz)
Date: Wed, 29 Jan 2020 16:48:47 +0000
Subject: 8223699: cleanup perfMemory_aix.cpp O_NOFOLLOW coding on aix
Message-ID: <FA58D6FC-6902-45C1-92E7-1AD741211930@sap.com>

Hi Matthias,

the change looks good. Please note: I'm not a Reviewer!

Thanks for cleaning this up.
Lutz

?On 28.01.20, 11:34, "hotspot-dev on behalf of Doerr, Martin" <hotspot-dev-bounces at openjdk.java.net on behalf of martin.doerr at sap.com> wrote:

    Hi Matthias,
    
    I agree.
    
    According to
    https://wiki.openjdk.java.net/display/Build/Supported+Build+Platforms
    we require AIX 7.1 for JDK11 and 7.2 since JDK13.
    
    Early version of these AIX version were affected by the following O_NOFOLLOW issue:
    https://bugs.openjdk.java.net/browse/JDK-8223701
    
    But the affected Tech Levels have already reached "End of Service Pack Support":
    https://www.ibm.com/support/pages/aix-support-lifecycle-information
    Only AIX 7.1 TL5 and AIX 7.2 TL2 (and later) are still supported.
    
    Change looks good.
    
    Best regards,
    Martin
    
    
    > -----Original Message-----
    > From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
    > Baesken, Matthias
    > Sent: Dienstag, 28. Januar 2020 09:23
    > To: 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
    > Subject: RFR: 8223699: cleanup perfMemory_aix.cpp O_NOFOLLOW coding
    > on aix
    > 
    > Hello, please review  this AIX specific cleanup.
    > 
    > In perfMemory_aix.cpp we still have O_NOFOLLOW related fallback coding
    > for AIX version < 7.X.
    > This is no longer needed, because we do not support any more AIX versions
    > < 7.X in coming jdk13, so cleanup is possible.
    > 
    > 
    > Bug/webrev :
    > 
    > https://bugs.openjdk.java.net/browse/JDK-8223699
    > 
    > http://cr.openjdk.java.net/~mbaesken/webrevs/8223699.1/
    > 
    > 
    > Thanks, Matthias
    


From daniel.daugherty at oracle.com  Wed Jan 29 18:29:43 2020
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Wed, 29 Jan 2020 13:29:43 -0500
Subject: RFR (S): 8237857: LogDecorations::uptimenanos is implemented
 incorrectly
In-Reply-To: <42f25d16-7117-ae55-2dd5-2c5929f5340d@oracle.com>
References: <42f25d16-7117-ae55-2dd5-2c5929f5340d@oracle.com>
Message-ID: <8b25be64-94ce-32ab-7794-92aed8fd5189@oracle.com>

On 1/29/20 12:13 AM, David Holmes wrote:
> Bug: https://bugs.openjdk.java.net/browse/JDK-8237857
> Webrev: http://cr.openjdk.java.net/~dholmes/8237857/webrev/

src/hotspot/share/logging/logConfiguration.cpp
 ??? No comments.

src/hotspot/share/logging/logDecorations.hpp
 ??? No comments.

src/hotspot/share/logging/logDecorations.cpp
 ??? I was trying to figure out where you were going with this fix
 ??? and now I see. It would be good if you mentioned in the bug
 ??? report why you are using os::elapsedTime(). Perhaps even
 ??? mention your elapsed_time() helper

test/hotspot/gtest/logging/test_logDecorations.cpp
 ??? Thanks for adding this comment:

 ??? L117: ????? // The sleep needs to be longer than the timer 
resolution to ensure
 ??? L118: ????? // we see updates to 'timemillis'. Windows has the 
lowest resolution
 ??? L119: ????? // at 15-16ms, so we use 20.

Thumbs up.

Dan


>
> After my changes in JDK-8235741 which changed uptimemillis to be the 
> same as uptimenanos*1000000 the gtest for the logging time decorators 
> started to fail intermittently on Windows where the uptimemillis 
> values was not advancing sometimes after a 5ms sleep. It turned out 
> this was actually a day one bug in the logging code that my change had 
> exposed - the code uptimenanos was using os::elapsed_counter() but 
> that isn't a time value it's just a counter value. On Linux/AIX/BSD it 
> happens to count at 1ns resolution but not so on Solaris and in 
> particular not so on Windows. When this counter value was truncated to 
> "millis" there was no guarantee the value would increase as expected 
> by the test.
>
> Changes:
> - use os::elapsedTimer() not os::elapsed_counter() for uptimemillis 
> and uptimenanos
> - remove dead code that should have been removed in JDK-8235741
> - update the test by:
> ? - adding diagnostic printouts of the time values read
> ? - fix a bug where the sleep time can be too short for Windows (see 
> details in bug report)
>
> Testing:
> ? - gtests on all platforms with manual analysis of log files
>
> Thanks,
> David
> -----


From david.holmes at oracle.com  Wed Jan 29 23:16:53 2020
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 30 Jan 2020 09:16:53 +1000
Subject: RFR (S): 8237857: LogDecorations::uptimenanos is implemented
 incorrectly
In-Reply-To: <8b25be64-94ce-32ab-7794-92aed8fd5189@oracle.com>
References: <42f25d16-7117-ae55-2dd5-2c5929f5340d@oracle.com>
 <8b25be64-94ce-32ab-7794-92aed8fd5189@oracle.com>
Message-ID: <8a75a825-d8b9-9818-0875-53f68e13bfd3@oracle.com>

Thanks for the review Dan!

I updated one of my comments in the bug report to explicitly list the fix.

David

On 30/01/2020 4:29 am, Daniel D. Daugherty wrote:
> On 1/29/20 12:13 AM, David Holmes wrote:
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8237857
>> Webrev: http://cr.openjdk.java.net/~dholmes/8237857/webrev/
> 
> src/hotspot/share/logging/logConfiguration.cpp
>  ??? No comments.
> 
> src/hotspot/share/logging/logDecorations.hpp
>  ??? No comments.
> 
> src/hotspot/share/logging/logDecorations.cpp
>  ??? I was trying to figure out where you were going with this fix
>  ??? and now I see. It would be good if you mentioned in the bug
>  ??? report why you are using os::elapsedTime(). Perhaps even
>  ??? mention your elapsed_time() helper
> 
> test/hotspot/gtest/logging/test_logDecorations.cpp
>  ??? Thanks for adding this comment:
> 
>  ??? L117: ????? // The sleep needs to be longer than the timer 
> resolution to ensure
>  ??? L118: ????? // we see updates to 'timemillis'. Windows has the 
> lowest resolution
>  ??? L119: ????? // at 15-16ms, so we use 20.
> 
> Thumbs up.
> 
> Dan
> 
> 
>>
>> After my changes in JDK-8235741 which changed uptimemillis to be the 
>> same as uptimenanos*1000000 the gtest for the logging time decorators 
>> started to fail intermittently on Windows where the uptimemillis 
>> values was not advancing sometimes after a 5ms sleep. It turned out 
>> this was actually a day one bug in the logging code that my change had 
>> exposed - the code uptimenanos was using os::elapsed_counter() but 
>> that isn't a time value it's just a counter value. On Linux/AIX/BSD it 
>> happens to count at 1ns resolution but not so on Solaris and in 
>> particular not so on Windows. When this counter value was truncated to 
>> "millis" there was no guarantee the value would increase as expected 
>> by the test.
>>
>> Changes:
>> - use os::elapsedTimer() not os::elapsed_counter() for uptimemillis 
>> and uptimenanos
>> - remove dead code that should have been removed in JDK-8235741
>> - update the test by:
>> ? - adding diagnostic printouts of the time values read
>> ? - fix a bug where the sleep time can be too short for Windows (see 
>> details in bug report)
>>
>> Testing:
>> ? - gtests on all platforms with manual analysis of log files
>>
>> Thanks,
>> David
>> -----
> 

From david.holmes at oracle.com  Wed Jan 29 23:44:58 2020
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 30 Jan 2020 09:44:58 +1000
Subject: RFR[L]: 8237767 Field layout computation overhaul
In-Reply-To: <BFA088F7-E55B-4B47-A1CB-61A9E6900D19@oracle.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
 <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com>
 <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com>
 <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com>
 <77ca9ddd-e633-6b72-4301-b6d16c61509d@oracle.com>
 <4bd88392-338a-6287-a16e-1fb3949b85f7@oracle.com>
 <BFA088F7-E55B-4B47-A1CB-61A9E6900D19@oracle.com>
Message-ID: <e4ca9eb9-21ce-fe65-c29d-31da69c535e6@oracle.com>

Hi Fred,

On 30/01/2020 12:57 am, Frederic Parain wrote:
> The kind of the flag is a secondary issue, the main issue is ?how
> fast do we want to remove the old code??. I wanted to keep the old
> code for a short transition period in case new field layouts cause
> issues for some tools or customers. The goal was not to keep it
> forever and allow users to select which algorithm they want to use.

Right. The problem with our 6 month release cadence is that we're not 
realistically going to get this new code exposed to users who will find 
potential problems in a single release. Even if we think this might 
mainly impact tools and those developers do tend to try and keep up, it 
is still unlikely to get enough visibility in a single release so ...

> Note that keeping the old code and its activation with a normal
> product flag will be an issue for the first release of Valhalla.
> The way flattened fields are allocated in the old code is very
> inefficient, which means that instead of seeing an improvement
> in data density, users will see a significant increase in space
> consumption if they use the old code.
> 
> So, here?s the three possible paths forward:
>    1 - push the new code and remove the old code in a single change,
>        so no need for a VM option to select which code to use

Seems potentially dangerous when we don't know the impact.

>    2 - push the new code while keeping the old one for a short
>        period of time, to remove the old code before Valhalla code
>        is pushed

Begs the question: when do we think something of Valhalla will ship? I'm 
guessing not before 17. If so then we deprecate the flag in 16 and 
obsolete in 17 - thus zero impact on Valhalla. If Valhalla comes earlier 
in 16 then we have some impact but will still have this out of the way 
before 17 (which should be next LTS release).

>    3 - push the new code while keeping the old one with a long
>        deprecation period before removing the old code
> 
> Option 3 will be an issue for project Valhalla, as explained
> above.
> 
> Option 1 sounds a big risky to me, from the experience I had
> while doing this change in the JVM and the unexpected places
> where it caused issues.
> 
> Option 2 sounded like a good compromise, however, it seems
> there?s no consensus how to implement it. Could we refine
> the details on how a short transition period could be implemented?

Add flag for 15; deprecate in 16; obsolete in 17; expire in 18.

I'm concerned we are developing a tendency to make flags diagnostic 
because it is considered convenient for us. There are customers whose 
own policies disallow running their production systems with anything 
other than product flags. To me a diagnostic flag aids in diagnosing a 
problem more than being a switch between old and new behaviour - but I 
agree the lines are blurry. Historically there was much more difference 
between product flags and diagnostic, when product flags would be 
supported in a number of releases spanning many years. That is no longer 
the case. We can effectively kill off a product flag after 12 months: 6 
months full "support"; 6 months deprecated; then obsolete. We can even 
deprecate a flag in the release we introduce it if we really think that 
is warranted.

If we intend to tell anyone to use the flag then we have to ensure both 
code paths remain fully functional during that period.

Cheers,
David

> Thank you,
> 
> Fred
> 
> 
>> On Jan 29, 2020, at 08:44, coleen.phillimore at oracle.com wrote:
>>
>>
>>
>> On 1/29/20 8:31 AM, David Holmes wrote:
>>> PS. I've now seen the CSR and have commented there. I don't agree with the use of a diagnostic flag. It doesn't buy us anything other than not needing a CSR request when we decide to deprecate it. But if we're going to tell people to use these flags to revert behaviour then they will need to follow the deprecation process regardless - and a product flag is better from an end user perspective.
>>
>> Obviously I disagree.  We don't want to publicise this flag *unless* someone has a problem, which makes it a diagnostic flag.   We don't want to have to carry the old code 2 releases when there is honestly no reason to suspect it is problematic.
>>
>> Coleen
>>
>>>
>>> Cheers,
>>> David
>>>
>>> On 29/01/2020 5:06 pm, David Holmes wrote:
>>>> Hi Fred,
>>>>
>>>> I've looked at the v7 version. A few more stylistic comments on that first. Note, no need for an item by item response unless that makes it easier for you to track :)
>>>>
>>>> src/hotspot/share/classfile/classFileParser.cpp
>>>>
>>>>    #include "classfile/defaultMethods.hpp"
>>>> +#include "classfile/fieldLayoutBuilder.hpp"
>>>>    #include "classfile/dictionary.hpp"
>>>>
>>>> Include files are not in alphabetical order.
>>>>
>>>> +   * This may well change: FixMe if doesn't,
>>>>
>>>> s/if/if it/
>>>>
>>>> +  //Make a temp copy, and iterate through and copy back into the orig
>>>>
>>>> Space after //
>>>>
>>>> s/orig/original/
>>>>
>>>> +  OopMapBlock*  nonstatic_oop_map = _nonstatic_oop_maps;
>>>>
>>>> Extra space after *
>>>>
>>>> +  assert(ik->nonstatic_oop_map_count() == _field_info->oop_map_blocks->_nonstatic_oop_map_count,
>>>> +     "sanity");
>>>>
>>>> Second line needs to be indented further:
>>>>
>>>>      assert(ik->nonstatic_oop_map_count() == _field_info->oop_map_blocks->_nonstatic_oop_map_count,
>>>>             "sanity");
>>>>
>>>> ---
>>>>
>>>> src/hotspot/share/classfile/classFileParser.hpp
>>>>
>>>> +public:
>>>> +  OopMapBlock* _nonstatic_oop_maps;
>>>> +  unsigned int _nonstatic_oop_map_count;
>>>> +  unsigned int _max_nonstatic_oop_maps;
>>>> +
>>>> + public:
>>>>
>>>> Second public uneeded. First public may be indented wrong (I'm not sure what the rule is - single space indent?)
>>>>
>>>>    class ClassFileParser {
>>>> +  friend class FieldLayoutBuilder;
>>>> +  friend class FieldLayout;
>>>>
>>>>     class ClassAnnotationCollector;
>>>>     class FieldAllocationCount;
>>>>     class FieldAnnotationCollector;
>>>>
>>>> Indents are different. I think the class forward declarations should have extra space.
>>>>
>>>> ---
>>>>
>>>> src/hotspot/share/oops/instanceKlass.hpp
>>>>
>>>> +  void increment_count(int diff)     { _count += diff; }
>>>>
>>>> Extra spaces before {
>>>>
>>>> ---
>>>>
>>>> src/hotspot/share/runtime/globals.hpp
>>>>
>>>> +  diagnostic(bool, UseNewFieldLayout, true,     \
>>>> +               "Use new algorithm to compute layouts")     \
>>>> +     \
>>>> +  product(bool, UseEmptySlotsInSupers, true,
>>>>
>>>> Not sure I see why one flag is diagnostic and the other product. Do you expect people to need to disable using empty slots more so than needing to disable using the new field layout altogether?
>>>>
>>>> ---
>>>>
>>>> src/hotspot/share/classfile/fieldLayoutBuilder.cpp
>>>>
>>>> +  assert(kind == EMPTY || kind == RESERVED || kind == PADDING || kind == INHERITED,
>>>> +      "Otherwise, should use the constructor with a field index argument");
>>>>
>>>> Indentation of second line is wrong.
>>>>
>>>> +  assert(kind == REGULAR || kind == FLATTENED || kind == INHERITED,
>>>> +      "Other kind do not have a field index");
>>>>
>>>> Ditto.
>>>>
>>>>
>>>> +  if (list == NULL) return;
>>>> +  if (start == NULL) {
>>>> +    start = this->_start;
>>>> +  }
>>>>
>>>> Inconsistent style for single statement if-blocks. Same thing later in the file.
>>>>
>>>> +      output->print_cr(" @%d \"%s\" %s %d/%d %s",
>>>> +          b->offset(),
>>>> +          fi->name(_cp)->as_C_string(),
>>>> +          fi->signature(_cp)->as_C_string(),
>>>> +          b->size(),
>>>> +          b->alignment(),
>>>> +          "REGULAR");
>>>>
>>>> Incorrect identation of continuing line. Same for all the following print blocks.
>>>>
>>>> +  } else if (_classname == vmSymbols::java_lang_Boolean() ||
>>>> +      _classname == vmSymbols::java_lang_Character() ||
>>>> +      _classname == vmSymbols::java_lang_Float() ||
>>>> +      _classname == vmSymbols::java_lang_Double() ||
>>>> +      _classname == vmSymbols::java_lang_Byte() ||
>>>> +      _classname == vmSymbols::java_lang_Short() ||
>>>> +      _classname == vmSymbols::java_lang_Integer() ||
>>>> +      _classname == vmSymbols::java_lang_Long()) {
>>>>
>>>> Incorrect identation of continuing line.
>>>>
>>>> ---
>>>>
>>>> src/hotspot/share/classfile/fieldLayoutBuilder.hpp
>>>>
>>>> +// and the boxing classes). The rational for having multiple methods
>>>>
>>>> s/rational/rationale/
>>>>
>>>> +  FieldLayoutBuilder(const Symbol* classname, const InstanceKlass* super_klass, ConstantPool* constant_pool,
>>>> +      Array<u2>* fields, bool is_contended, FieldLayoutInfo* info);
>>>>
>>>> Indentation wrong for continuing line.
>>>>
>>>> +  int get_alignment() {
>>>> +     assert(_alignment != -1, "Uninitialized");
>>>> +     return _alignment;
>>>> +   }
>>>>
>>>> Indenting appears off by one.
>>>>
>>>> ---
>>>>
>>>> test/hotspot/jtreg/runtime/FieldLayout/FieldDensityTest.java
>>>>
>>>> + * @run main/othervm -XX:+UseCompressedOops -XX:+UseCompressedClassPointers FieldDensityTest
>>>> + * @run main/othervm -XX:+UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest
>>>> + * @run main/othervm -XX:-UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest
>>>>
>>>> The test won't run on 32-bit platforms as the compressed oops flags won't exist.
>>>>
>>>> ---
>>>>
>>>> Some follow up comments below ...
>>>>
>>>> With trimming ...
>>>>
>>>> On 25/01/2020 3:20 am, Frederic Parain wrote:
>>>>>> On Jan 24, 2020, at 08:19, David Holmes <david.holmes at oracle.com> wrote:
>>>>>>
>>>>>> 466     int super_flen   = super->nof_nonstatic_fields();
>>>>>>
>>>>>> Could be folded directly into the assert so we don't call in product.
>>>>>
>>>>> Calling not_nonstatic_fields() has the side effect to compute non-static fields,
>>>>> which is required to get a correct value when reading super->_nonstatic_fields,
>>>>> so the call is needed even in product builds.
>>>>
>>>> Yuck! That's a horrible side-effect - but not your fault obviously. :) It would be better to have a nonstatic_fields() accessor that has the same lazy initialization side-effect.
>>>>
>>>>>> General style issue: when breaking a long line with a method call, the new line (containing arguments) should be indented to the opening ( of the method call e.g.
>>>> ...
>>>>>> etc. This applies across all files.
>>>>>
>>>>> Fixes applied lines 4003, 4011, 4041, 4138, 4143.
>>>>
>>>> Fix was also needed in other files. Current issues highlighted above.
>>>>
>>>>>>
>>>>>> src/hotspot/share/oops/instanceKlass.hpp
>>>>>>
>>>>>> You need to be careful with _extra_flags usage if there can be concurrently updated bits. At the moment it looks like redefinition is a mutable dynamic property, whilst "contended annotations" should be a static immutable property - is that right?
>>>>>
>>>>> Correct, _has_contended_annotations is a static immutable property, while _is_being_redefined is a mutable one.
>>>>
>>>> Good to know. My concern is that if someone adds a new mutable flag bit the need for atomic updates may not be noticed. We got bitten by this in the past with a flag field and I think we eventually migrated all of the mutable bits out into their own field. (Coleen should recall that :) ).
>>>>
>>>>>>    61     FLATTENED,     // flattened field
>>>>>>
>>>>>> Does this have any meaning before inline types come in?
>>>>>
>>>>> Yes, I wanted to reserved the entry in the enum.
>>>>
>>>> Hmmm a tenuous "okay". Seems odd to require this now to support code that is still some way from joining mainline.
>>>>
>>>>>> In FieldLayoutBuilder::epilogue you have a number of calls to Thread::current() as well as an implicit call when you use ResourceMarks. You should capture the current thread once in a local and reuse it.
>>>>>
>>>>> Fixed
>>>>
>>>> It seems that this fix is now not needed as there is only one use left of the new "thread" variable in the ResourceMark. So that can return to being:
>>>>
>>>> ResourceMark rm;
>>>>
>>>> Thanks,
>>>> David
>>>> -----
>>>>
>>
> 

From kim.barrett at oracle.com  Thu Jan 30 01:21:45 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 29 Jan 2020 20:21:45 -0500
Subject: RFR (S): 8237857: LogDecorations::uptimenanos is implemented
 incorrectly
In-Reply-To: <42f25d16-7117-ae55-2dd5-2c5929f5340d@oracle.com>
References: <42f25d16-7117-ae55-2dd5-2c5929f5340d@oracle.com>
Message-ID: <5AF580E4-DA77-43D6-95FE-3E65366361E2@oracle.com>

> On Jan 29, 2020, at 12:13 AM, David Holmes <david.holmes at oracle.com> wrote:
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8237857
> Webrev: http://cr.openjdk.java.net/~dholmes/8237857/webrev/

Looks good.


From david.holmes at oracle.com  Thu Jan 30 01:39:30 2020
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 30 Jan 2020 11:39:30 +1000
Subject: RFR (S): 8237857: LogDecorations::uptimenanos is implemented
 incorrectly
In-Reply-To: <5AF580E4-DA77-43D6-95FE-3E65366361E2@oracle.com>
References: <42f25d16-7117-ae55-2dd5-2c5929f5340d@oracle.com>
 <5AF580E4-DA77-43D6-95FE-3E65366361E2@oracle.com>
Message-ID: <7cad94c0-29ca-0077-cd50-d1a5a3ed548e@oracle.com>

Thanks Kim!

David

On 30/01/2020 11:21 am, Kim Barrett wrote:
>> On Jan 29, 2020, at 12:13 AM, David Holmes <david.holmes at oracle.com> wrote:
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8237857
>> Webrev: http://cr.openjdk.java.net/~dholmes/8237857/webrev/
> 
> Looks good.
> 

From leo.korinth at oracle.com  Thu Jan 30 09:43:39 2020
From: leo.korinth at oracle.com (Leo Korinth)
Date: Thu, 30 Jan 2020 10:43:39 +0100
Subject: fopen vs. os::fopen and automatic closing of the file on exec
In-Reply-To: <AM0PR02MB422786812770E5D21B7C547893050@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB42275083AC9BA465ADCF75B4930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <462ea46a-cef8-79a0-ccf0-71699e619089@oracle.com>
 <76ff9efd-4274-29dd-c048-02e55aa70890@oracle.com>
 <CAA-vtUx=G2p71JFL9O5QyVL=ztAuFkcn2aS0ztZY7z+i20UGFg@mail.gmail.com>
 <AM0PR02MB422786812770E5D21B7C547893050@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <f494a795-c77b-44c7-f758-469b5e74eb2c@oracle.com>

On 29/01/2020 14:03, Baesken, Matthias wrote:
> Hi ?Leo / Thomas / David,
> 
>  ?Not sure if it is a small? or ?medium? gain ,? but after the 
> discussion so far I think it makes sense to use? os::fopen in HS code  
> where possible
> 
>  ?( ?what (f)open does in the ?jdk C coding is a different thing, we 
> might dicuss this on corelibs-dev? in more detail ).
> 
> I opened
> 
> https://bugs.openjdk.java.net/browse/JDK-8238161

I linked your bug to https://bugs.openjdk.java.net/browse/JDK-8202720 
(process) Clarify file handler behaviour

I have no idea why we (explicitly) keeps files open on Windows on exec, 
I think maybe that issue should be dealt with first as it might affect 
this proposed change.

Thanks,
Leo

> 
> use os::fopen in HS code where possible
> 
> Best regards, Matthias
> 
> Hi Leo,
> 
> On Tue, Jan 28, 2020 at 4:37 PM Leo Korinth <leo.korinth at oracle.com 
> <mailto:leo.korinth at oracle.com>> wrote:
> 
> ...
> 
>     Something that is not obvious is that on unix-like operating systems,
>     ProcessImpl_md.c tries to close (most) open files between fork and
>     exec.
>     That is not the case for Windows (I opened
>     https://bugs.openjdk.java.net/browse/JDK-8202720 for this). Thus (if I
>     understand correctly) the impact on unix-like operating systems will be
>     less for adding this support than it is for Windows. os::fopen was
>     created to solve a specific bug on windows (logging), and was
>     renamed to
>     the more generic os::fopen during review.
> 
> Just a note, it is still possible for file descriptors to escape into 
> child processes since you cannot guarantee that all forks happen via 
> Runtime.exec():
> 
> - native third party code may fork.
> 
> - hotspot may fork to run tools for error reporting
> 
> Also note that the code which closes fds in Runtime.exec() may in theory 
> fail to close all fds.
> 
> So I think Matthias makes sense on Posix platforms.
> 
>     I guess most uses of ::fopen /should/ use the more restricted
>     os::fopen,
>     but the gain would probably be small.
> 
>     Thanks,
>     Leo
> 

From fweimer at redhat.com  Thu Jan 30 13:37:46 2020
From: fweimer at redhat.com (Florian Weimer)
Date: Thu, 30 Jan 2020 14:37:46 +0100
Subject: fopen vs. os::fopen and automatic closing of the file on exec
In-Reply-To: <76ff9efd-4274-29dd-c048-02e55aa70890@oracle.com> (Leo Korinth's
 message of "Tue, 28 Jan 2020 16:36:44 +0100")
References: <AM0PR02MB42275083AC9BA465ADCF75B4930A0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <462ea46a-cef8-79a0-ccf0-71699e619089@oracle.com>
 <76ff9efd-4274-29dd-c048-02e55aa70890@oracle.com>
Message-ID: <874kwd3w0l.fsf@oldenburg2.str.redhat.com>

* Leo Korinth:

>> It's possible the most recent version of Linux without O_CLOEXEC
>> supported is no longer supported by OpenJDK, in which case we can
>> remove it. But I'm not sure what that version is. I have no idea if
>> fopen with "e" support has the same history as ::open and O_CLOEXEC.
>
> "e" is supported since glibc 2.7, released in 2007. Any support of
> libc versions older than 2.7 today would suprise me.

O_CLOEXEC has not been backported into the Red Hat Enterprise Linux 5
kernel (which is based on Linux 2.6.18).  Its glibc (based on the 2.5
upstream version) does not support it, either.

However, I expect that no one is going to support the next OpenJDK (LTS)
release on that platform, so I don't think this is something that
matters in the OpenJDK context.

Thanks,
Florian


From coleen.phillimore at oracle.com  Thu Jan 30 15:04:43 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Thu, 30 Jan 2020 10:04:43 -0500
Subject: RFR[L]: 8237767 Field layout computation overhaul
In-Reply-To: <e4ca9eb9-21ce-fe65-c29d-31da69c535e6@oracle.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
 <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com>
 <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com>
 <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com>
 <77ca9ddd-e633-6b72-4301-b6d16c61509d@oracle.com>
 <4bd88392-338a-6287-a16e-1fb3949b85f7@oracle.com>
 <BFA088F7-E55B-4B47-A1CB-61A9E6900D19@oracle.com>
 <e4ca9eb9-21ce-fe65-c29d-31da69c535e6@oracle.com>
Message-ID: <88d83416-58c1-7026-aaff-208929d9b9d6@oracle.com>



On 1/29/20 6:44 PM, David Holmes wrote:
> Hi Fred,
>
> On 30/01/2020 12:57 am, Frederic Parain wrote:
>> The kind of the flag is a secondary issue, the main issue is ?how
>> fast do we want to remove the old code??. I wanted to keep the old
>> code for a short transition period in case new field layouts cause
>> issues for some tools or customers. The goal was not to keep it
>> forever and allow users to select which algorithm they want to use.
>
> Right. The problem with our 6 month release cadence is that we're not 
> realistically going to get this new code exposed to users who will 
> find potential problems in a single release. Even if we think this 
> might mainly impact tools and those developers do tend to try and keep 
> up, it is still unlikely to get enough visibility in a single release 
> so ...
>
>> Note that keeping the old code and its activation with a normal
>> product flag will be an issue for the first release of Valhalla.
>> The way flattened fields are allocated in the old code is very
>> inefficient, which means that instead of seeing an improvement
>> in data density, users will see a significant increase in space
>> consumption if they use the old code.
>>
>> So, here?s the three possible paths forward:
>> ?? 1 - push the new code and remove the old code in a single change,
>> ?????? so no need for a VM option to select which code to use
>
> Seems potentially dangerous when we don't know the impact.
>
>> ?? 2 - push the new code while keeping the old one for a short
>> ?????? period of time, to remove the old code before Valhalla code
>> ?????? is pushed
>
> Begs the question: when do we think something of Valhalla will ship? 
> I'm guessing not before 17. If so then we deprecate the flag in 16 and 
> obsolete in 17 - thus zero impact on Valhalla. If Valhalla comes 
> earlier in 16 then we have some impact but will still have this out of 
> the way before 17 (which should be next LTS release).
>
>> ?? 3 - push the new code while keeping the old one with a long
>> ?????? deprecation period before removing the old code
>>
>> Option 3 will be an issue for project Valhalla, as explained
>> above.
>>
>> Option 1 sounds a big risky to me, from the experience I had
>> while doing this change in the JVM and the unexpected places
>> where it caused issues.
>>
>> Option 2 sounded like a good compromise, however, it seems
>> there?s no consensus how to implement it. Could we refine
>> the details on how a short transition period could be implemented?
>
> Add flag for 15; deprecate in 16; obsolete in 17; expire in 18.

How about add flag for 15, pre-deprecated, obsolete in 16 so that 
valhalla work can go forward, especially if there's a preview in 16.? 
Either way the old code is removed in JDK 17, the LTS release.

>
> I'm concerned we are developing a tendency to make flags diagnostic 
> because it is considered convenient for us. There are customers whose 
> own policies disallow running their production systems with anything 
> other than product flags. To me a diagnostic flag aids in diagnosing a 
> problem more than being a switch between old and new behaviour - but I 
> agree the lines are blurry. Historically there was much more 
> difference between product flags and diagnostic, when product flags 
> would be supported in a number of releases spanning many years. That 
> is no longer the case. We can effectively kill off a product flag 
> after 12 months: 6 months full "support"; 6 months deprecated; then 
> obsolete. We can even deprecate a flag in the release we introduce it 
> if we really think that is warranted.

This 6 months of support vs. 9 months isn't going to make a practical 
difference to the customers.? They can fall back on JDK 11, the previous 
LTS release until they fix their code.? 6 months of "support" is better 
for us however since we don't have code paths that we should be testing, 
and will be a broken (or preferably disabled) code path in the valhalla 
repo that we're working on and asking people to try right now.
>
> If we intend to tell anyone to use the flag then we have to ensure 
> both code paths remain fully functional during that period.

That's the thing.? We don't _want_ to tell anyone to use this flag 
unless they are diagnosing a problem with their code.? The only reason I 
can see making it a product flag is because we want customers to fix 
their code, and not us to add hacks to our new code to support any 
corner case layout that _might_ exist.

Coleen
>
> Cheers,
> David
>
>> Thank you,
>>
>> Fred
>>
>>
>>> On Jan 29, 2020, at 08:44, coleen.phillimore at oracle.com wrote:
>>>
>>>
>>>
>>> On 1/29/20 8:31 AM, David Holmes wrote:
>>>> PS. I've now seen the CSR and have commented there. I don't agree 
>>>> with the use of a diagnostic flag. It doesn't buy us anything other 
>>>> than not needing a CSR request when we decide to deprecate it. But 
>>>> if we're going to tell people to use these flags to revert 
>>>> behaviour then they will need to follow the deprecation process 
>>>> regardless - and a product flag is better from an end user 
>>>> perspective.
>>>
>>> Obviously I disagree.? We don't want to publicise this flag *unless* 
>>> someone has a problem, which makes it a diagnostic flag.?? We don't 
>>> want to have to carry the old code 2 releases when there is honestly 
>>> no reason to suspect it is problematic.
>>>
>>> Coleen
>>>
>>>>
>>>> Cheers,
>>>> David
>>>>
>>>> On 29/01/2020 5:06 pm, David Holmes wrote:
>>>>> Hi Fred,
>>>>>
>>>>> I've looked at the v7 version. A few more stylistic comments on 
>>>>> that first. Note, no need for an item by item response unless that 
>>>>> makes it easier for you to track :)
>>>>>
>>>>> src/hotspot/share/classfile/classFileParser.cpp
>>>>>
>>>>> ?? #include "classfile/defaultMethods.hpp"
>>>>> +#include "classfile/fieldLayoutBuilder.hpp"
>>>>> ?? #include "classfile/dictionary.hpp"
>>>>>
>>>>> Include files are not in alphabetical order.
>>>>>
>>>>> +?? * This may well change: FixMe if doesn't,
>>>>>
>>>>> s/if/if it/
>>>>>
>>>>> +? //Make a temp copy, and iterate through and copy back into the 
>>>>> orig
>>>>>
>>>>> Space after //
>>>>>
>>>>> s/orig/original/
>>>>>
>>>>> +? OopMapBlock*? nonstatic_oop_map = _nonstatic_oop_maps;
>>>>>
>>>>> Extra space after *
>>>>>
>>>>> +? assert(ik->nonstatic_oop_map_count() == 
>>>>> _field_info->oop_map_blocks->_nonstatic_oop_map_count,
>>>>> +???? "sanity");
>>>>>
>>>>> Second line needs to be indented further:
>>>>>
>>>>> ???? assert(ik->nonstatic_oop_map_count() == 
>>>>> _field_info->oop_map_blocks->_nonstatic_oop_map_count,
>>>>> ??????????? "sanity");
>>>>>
>>>>> ---
>>>>>
>>>>> src/hotspot/share/classfile/classFileParser.hpp
>>>>>
>>>>> +public:
>>>>> +? OopMapBlock* _nonstatic_oop_maps;
>>>>> +? unsigned int _nonstatic_oop_map_count;
>>>>> +? unsigned int _max_nonstatic_oop_maps;
>>>>> +
>>>>> + public:
>>>>>
>>>>> Second public uneeded. First public may be indented wrong (I'm not 
>>>>> sure what the rule is - single space indent?)
>>>>>
>>>>> ?? class ClassFileParser {
>>>>> +? friend class FieldLayoutBuilder;
>>>>> +? friend class FieldLayout;
>>>>>
>>>>> ??? class ClassAnnotationCollector;
>>>>> ??? class FieldAllocationCount;
>>>>> ??? class FieldAnnotationCollector;
>>>>>
>>>>> Indents are different. I think the class forward declarations 
>>>>> should have extra space.
>>>>>
>>>>> ---
>>>>>
>>>>> src/hotspot/share/oops/instanceKlass.hpp
>>>>>
>>>>> +? void increment_count(int diff)???? { _count += diff; }
>>>>>
>>>>> Extra spaces before {
>>>>>
>>>>> ---
>>>>>
>>>>> src/hotspot/share/runtime/globals.hpp
>>>>>
>>>>> +? diagnostic(bool, UseNewFieldLayout, true,???? \
>>>>> +?????????????? "Use new algorithm to compute layouts")???? \
>>>>> +???? \
>>>>> +? product(bool, UseEmptySlotsInSupers, true,
>>>>>
>>>>> Not sure I see why one flag is diagnostic and the other product. 
>>>>> Do you expect people to need to disable using empty slots more so 
>>>>> than needing to disable using the new field layout altogether?
>>>>>
>>>>> ---
>>>>>
>>>>> src/hotspot/share/classfile/fieldLayoutBuilder.cpp
>>>>>
>>>>> +? assert(kind == EMPTY || kind == RESERVED || kind == PADDING || 
>>>>> kind == INHERITED,
>>>>> +????? "Otherwise, should use the constructor with a field index 
>>>>> argument");
>>>>>
>>>>> Indentation of second line is wrong.
>>>>>
>>>>> +? assert(kind == REGULAR || kind == FLATTENED || kind == INHERITED,
>>>>> +????? "Other kind do not have a field index");
>>>>>
>>>>> Ditto.
>>>>>
>>>>>
>>>>> +? if (list == NULL) return;
>>>>> +? if (start == NULL) {
>>>>> +??? start = this->_start;
>>>>> +? }
>>>>>
>>>>> Inconsistent style for single statement if-blocks. Same thing 
>>>>> later in the file.
>>>>>
>>>>> +????? output->print_cr(" @%d \"%s\" %s %d/%d %s",
>>>>> +????????? b->offset(),
>>>>> +????????? fi->name(_cp)->as_C_string(),
>>>>> +????????? fi->signature(_cp)->as_C_string(),
>>>>> +????????? b->size(),
>>>>> +????????? b->alignment(),
>>>>> +????????? "REGULAR");
>>>>>
>>>>> Incorrect identation of continuing line. Same for all the 
>>>>> following print blocks.
>>>>>
>>>>> +? } else if (_classname == vmSymbols::java_lang_Boolean() ||
>>>>> +????? _classname == vmSymbols::java_lang_Character() ||
>>>>> +????? _classname == vmSymbols::java_lang_Float() ||
>>>>> +????? _classname == vmSymbols::java_lang_Double() ||
>>>>> +????? _classname == vmSymbols::java_lang_Byte() ||
>>>>> +????? _classname == vmSymbols::java_lang_Short() ||
>>>>> +????? _classname == vmSymbols::java_lang_Integer() ||
>>>>> +????? _classname == vmSymbols::java_lang_Long()) {
>>>>>
>>>>> Incorrect identation of continuing line.
>>>>>
>>>>> ---
>>>>>
>>>>> src/hotspot/share/classfile/fieldLayoutBuilder.hpp
>>>>>
>>>>> +// and the boxing classes). The rational for having multiple methods
>>>>>
>>>>> s/rational/rationale/
>>>>>
>>>>> +? FieldLayoutBuilder(const Symbol* classname, const 
>>>>> InstanceKlass* super_klass, ConstantPool* constant_pool,
>>>>> +????? Array<u2>* fields, bool is_contended, FieldLayoutInfo* info);
>>>>>
>>>>> Indentation wrong for continuing line.
>>>>>
>>>>> +? int get_alignment() {
>>>>> +???? assert(_alignment != -1, "Uninitialized");
>>>>> +???? return _alignment;
>>>>> +?? }
>>>>>
>>>>> Indenting appears off by one.
>>>>>
>>>>> ---
>>>>>
>>>>> test/hotspot/jtreg/runtime/FieldLayout/FieldDensityTest.java
>>>>>
>>>>> + * @run main/othervm -XX:+UseCompressedOops 
>>>>> -XX:+UseCompressedClassPointers FieldDensityTest
>>>>> + * @run main/othervm -XX:+UseCompressedOops 
>>>>> -XX:-UseCompressedClassPointers FieldDensityTest
>>>>> + * @run main/othervm -XX:-UseCompressedOops 
>>>>> -XX:-UseCompressedClassPointers FieldDensityTest
>>>>>
>>>>> The test won't run on 32-bit platforms as the compressed oops 
>>>>> flags won't exist.
>>>>>
>>>>> ---
>>>>>
>>>>> Some follow up comments below ...
>>>>>
>>>>> With trimming ...
>>>>>
>>>>> On 25/01/2020 3:20 am, Frederic Parain wrote:
>>>>>>> On Jan 24, 2020, at 08:19, David Holmes 
>>>>>>> <david.holmes at oracle.com> wrote:
>>>>>>>
>>>>>>> 466???? int super_flen?? = super->nof_nonstatic_fields();
>>>>>>>
>>>>>>> Could be folded directly into the assert so we don't call in 
>>>>>>> product.
>>>>>>
>>>>>> Calling not_nonstatic_fields() has the side effect to compute 
>>>>>> non-static fields,
>>>>>> which is required to get a correct value when reading 
>>>>>> super->_nonstatic_fields,
>>>>>> so the call is needed even in product builds.
>>>>>
>>>>> Yuck! That's a horrible side-effect - but not your fault 
>>>>> obviously. :) It would be better to have a nonstatic_fields() 
>>>>> accessor that has the same lazy initialization side-effect.
>>>>>
>>>>>>> General style issue: when breaking a long line with a method 
>>>>>>> call, the new line (containing arguments) should be indented to 
>>>>>>> the opening ( of the method call e.g.
>>>>> ...
>>>>>>> etc. This applies across all files.
>>>>>>
>>>>>> Fixes applied lines 4003, 4011, 4041, 4138, 4143.
>>>>>
>>>>> Fix was also needed in other files. Current issues highlighted above.
>>>>>
>>>>>>>
>>>>>>> src/hotspot/share/oops/instanceKlass.hpp
>>>>>>>
>>>>>>> You need to be careful with _extra_flags usage if there can be 
>>>>>>> concurrently updated bits. At the moment it looks like 
>>>>>>> redefinition is a mutable dynamic property, whilst "contended 
>>>>>>> annotations" should be a static immutable property - is that right?
>>>>>>
>>>>>> Correct, _has_contended_annotations is a static immutable 
>>>>>> property, while _is_being_redefined is a mutable one.
>>>>>
>>>>> Good to know. My concern is that if someone adds a new mutable 
>>>>> flag bit the need for atomic updates may not be noticed. We got 
>>>>> bitten by this in the past with a flag field and I think we 
>>>>> eventually migrated all of the mutable bits out into their own 
>>>>> field. (Coleen should recall that :) ).
>>>>>
>>>>>>> ?? 61???? FLATTENED,???? // flattened field
>>>>>>>
>>>>>>> Does this have any meaning before inline types come in?
>>>>>>
>>>>>> Yes, I wanted to reserved the entry in the enum.
>>>>>
>>>>> Hmmm a tenuous "okay". Seems odd to require this now to support 
>>>>> code that is still some way from joining mainline.
>>>>>
>>>>>>> In FieldLayoutBuilder::epilogue you have a number of calls to 
>>>>>>> Thread::current() as well as an implicit call when you use 
>>>>>>> ResourceMarks. You should capture the current thread once in a 
>>>>>>> local and reuse it.
>>>>>>
>>>>>> Fixed
>>>>>
>>>>> It seems that this fix is now not needed as there is only one use 
>>>>> left of the new "thread" variable in the ResourceMark. So that can 
>>>>> return to being:
>>>>>
>>>>> ResourceMark rm;
>>>>>
>>>>> Thanks,
>>>>> David
>>>>> -----
>>>>>
>>>
>>


From matthias.baesken at sap.com  Thu Jan 30 15:47:50 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Thu, 30 Jan 2020 15:47:50 +0000
Subject: RFR: 8238161: use os::fopen in HS code where possible
Message-ID: <AM0PR02MB4227F2803B02F5134CFBF98193040@AM0PR02MB4227.eurprd02.prod.outlook.com>

Please review this change which adjusts a number of calls (in HS code)  from using  fopen  to os::fopen .

There exists for some time  a function os::fopen in HS code . This function sets "close-on-exec" flags (when possible) to the opened filedescriptor.
It might have some benefits to use os::fopen more in HS code, currently there are still quite a few calls directly to fopen .
(however there are also  already a number of calls to os::fopen  e.g.    some calls   in  os_linux.cpp /os_solaris.cpp should already look up  os::fopen  , and  logFileOutput.cpp  uses os::fopen directly   )


see also the thread
https://mail.openjdk.java.net/pipermail/hotspot-dev/2020-January/040641.html
on hotspot-dev about the topic.



Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8238161

http://cr.openjdk.java.net/~mbaesken/webrevs/8238161.0/


Thanks, Matthias

From david.holmes at oracle.com  Thu Jan 30 23:18:34 2020
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 31 Jan 2020 09:18:34 +1000
Subject: RFR: 8238161: use os::fopen in HS code where possible
In-Reply-To: <AM0PR02MB4227F2803B02F5134CFBF98193040@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB4227F2803B02F5134CFBF98193040@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <e9e878d4-d878-a528-0ead-d79813000a0d@oracle.com>

Hi Matthias,

Does close-on-exec have any meaning when applied to /proc filesystem ??

Thanks,
David

On 31/01/2020 1:47 am, Baesken, Matthias wrote:
> Please review this change which adjusts a number of calls (in HS code)  from using  fopen  to os::fopen .
> 
> There exists for some time  a function os::fopen in HS code . This function sets "close-on-exec" flags (when possible) to the opened filedescriptor.
> It might have some benefits to use os::fopen more in HS code, currently there are still quite a few calls directly to fopen .
> (however there are also  already a number of calls to os::fopen  e.g.    some calls   in  os_linux.cpp /os_solaris.cpp should already look up  os::fopen  , and  logFileOutput.cpp  uses os::fopen directly   )
> 
> 
> see also the thread
> https://mail.openjdk.java.net/pipermail/hotspot-dev/2020-January/040641.html
> on hotspot-dev about the topic.
> 
> 
> 
> Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8238161
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8238161.0/
> 
> 
> Thanks, Matthias
> 

From david.holmes at oracle.com  Thu Jan 30 23:40:58 2020
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 31 Jan 2020 09:40:58 +1000
Subject: RFR[L]: 8237767 Field layout computation overhaul
In-Reply-To: <88d83416-58c1-7026-aaff-208929d9b9d6@oracle.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
 <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com>
 <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com>
 <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com>
 <77ca9ddd-e633-6b72-4301-b6d16c61509d@oracle.com>
 <4bd88392-338a-6287-a16e-1fb3949b85f7@oracle.com>
 <BFA088F7-E55B-4B47-A1CB-61A9E6900D19@oracle.com>
 <e4ca9eb9-21ce-fe65-c29d-31da69c535e6@oracle.com>
 <88d83416-58c1-7026-aaff-208929d9b9d6@oracle.com>
Message-ID: <fe0db242-2610-c612-4213-e63c53803ccd@oracle.com>

On 31/01/2020 1:04 am, coleen.phillimore at oracle.com wrote:
> On 1/29/20 6:44 PM, David Holmes wrote:
>> Hi Fred,
>>
>> On 30/01/2020 12:57 am, Frederic Parain wrote:
>>> The kind of the flag is a secondary issue, the main issue is ?how
>>> fast do we want to remove the old code??. I wanted to keep the old
>>> code for a short transition period in case new field layouts cause
>>> issues for some tools or customers. The goal was not to keep it
>>> forever and allow users to select which algorithm they want to use.
>>
>> Right. The problem with our 6 month release cadence is that we're not 
>> realistically going to get this new code exposed to users who will 
>> find potential problems in a single release. Even if we think this 
>> might mainly impact tools and those developers do tend to try and keep 
>> up, it is still unlikely to get enough visibility in a single release 
>> so ...
>>
>>> Note that keeping the old code and its activation with a normal
>>> product flag will be an issue for the first release of Valhalla.
>>> The way flattened fields are allocated in the old code is very
>>> inefficient, which means that instead of seeing an improvement
>>> in data density, users will see a significant increase in space
>>> consumption if they use the old code.
>>>
>>> So, here?s the three possible paths forward:
>>> ?? 1 - push the new code and remove the old code in a single change,
>>> ?????? so no need for a VM option to select which code to use
>>
>> Seems potentially dangerous when we don't know the impact.
>>
>>> ?? 2 - push the new code while keeping the old one for a short
>>> ?????? period of time, to remove the old code before Valhalla code
>>> ?????? is pushed
>>
>> Begs the question: when do we think something of Valhalla will ship? 
>> I'm guessing not before 17. If so then we deprecate the flag in 16 and 
>> obsolete in 17 - thus zero impact on Valhalla. If Valhalla comes 
>> earlier in 16 then we have some impact but will still have this out of 
>> the way before 17 (which should be next LTS release).
>>
>>> ?? 3 - push the new code while keeping the old one with a long
>>> ?????? deprecation period before removing the old code
>>>
>>> Option 3 will be an issue for project Valhalla, as explained
>>> above.
>>>
>>> Option 1 sounds a big risky to me, from the experience I had
>>> while doing this change in the JVM and the unexpected places
>>> where it caused issues.
>>>
>>> Option 2 sounded like a good compromise, however, it seems
>>> there?s no consensus how to implement it. Could we refine
>>> the details on how a short transition period could be implemented?
>>
>> Add flag for 15; deprecate in 16; obsolete in 17; expire in 18.
> 
> How about add flag for 15, pre-deprecated, obsolete in 16 so that 
> valhalla work can go forward, especially if there's a preview in 16. 
> Either way the old code is removed in JDK 17, the LTS release.

That is fine too.

>> I'm concerned we are developing a tendency to make flags diagnostic 
>> because it is considered convenient for us. There are customers whose 
>> own policies disallow running their production systems with anything 
>> other than product flags. To me a diagnostic flag aids in diagnosing a 
>> problem more than being a switch between old and new behaviour - but I 
>> agree the lines are blurry. Historically there was much more 
>> difference between product flags and diagnostic, when product flags 
>> would be supported in a number of releases spanning many years. That 
>> is no longer the case. We can effectively kill off a product flag 
>> after 12 months: 6 months full "support"; 6 months deprecated; then 
>> obsolete. We can even deprecate a flag in the release we introduce it 
>> if we really think that is warranted.
> 
> This 6 months of support vs. 9 months isn't going to make a practical 
> difference to the customers.? They can fall back on JDK 11, the previous 
> LTS release until they fix their code.? 6 months of "support" is better 
> for us however since we don't have code paths that we should be testing, 
> and will be a broken (or preferably disabled) code path in the valhalla 
> repo that we're working on and asking people to try right now.
>>
>> If we intend to tell anyone to use the flag then we have to ensure 
>> both code paths remain fully functional during that period.
> 
> That's the thing.? We don't _want_ to tell anyone to use this flag 
> unless they are diagnosing a problem with their code.  

Regardless of the "why" if we tell people to use this flag as a 
workaround then the code path has to be functional.

> The only reason I 
> can see making it a product flag is because we want customers to fix 
> their code, and not us to add hacks to our new code to support any 
> corner case layout that _might_ exist.

A product flag is good for users and has no downside for us. A 
diagnostic flag is not so good for users and has no upside for us. The 
old code is going to remain, and must continue to work, for the exact 
same length of time. Having one product flag and one diagnostic flag 
makes no sense to me.

Cheers,
David

> Coleen
>>
>> Cheers,
>> David
>>
>>> Thank you,
>>>
>>> Fred
>>>
>>>
>>>> On Jan 29, 2020, at 08:44, coleen.phillimore at oracle.com wrote:
>>>>
>>>>
>>>>
>>>> On 1/29/20 8:31 AM, David Holmes wrote:
>>>>> PS. I've now seen the CSR and have commented there. I don't agree 
>>>>> with the use of a diagnostic flag. It doesn't buy us anything other 
>>>>> than not needing a CSR request when we decide to deprecate it. But 
>>>>> if we're going to tell people to use these flags to revert 
>>>>> behaviour then they will need to follow the deprecation process 
>>>>> regardless - and a product flag is better from an end user 
>>>>> perspective.
>>>>
>>>> Obviously I disagree.? We don't want to publicise this flag *unless* 
>>>> someone has a problem, which makes it a diagnostic flag.?? We don't 
>>>> want to have to carry the old code 2 releases when there is honestly 
>>>> no reason to suspect it is problematic.
>>>>
>>>> Coleen
>>>>
>>>>>
>>>>> Cheers,
>>>>> David
>>>>>
>>>>> On 29/01/2020 5:06 pm, David Holmes wrote:
>>>>>> Hi Fred,
>>>>>>
>>>>>> I've looked at the v7 version. A few more stylistic comments on 
>>>>>> that first. Note, no need for an item by item response unless that 
>>>>>> makes it easier for you to track :)
>>>>>>
>>>>>> src/hotspot/share/classfile/classFileParser.cpp
>>>>>>
>>>>>> ?? #include "classfile/defaultMethods.hpp"
>>>>>> +#include "classfile/fieldLayoutBuilder.hpp"
>>>>>> ?? #include "classfile/dictionary.hpp"
>>>>>>
>>>>>> Include files are not in alphabetical order.
>>>>>>
>>>>>> +?? * This may well change: FixMe if doesn't,
>>>>>>
>>>>>> s/if/if it/
>>>>>>
>>>>>> +? //Make a temp copy, and iterate through and copy back into the 
>>>>>> orig
>>>>>>
>>>>>> Space after //
>>>>>>
>>>>>> s/orig/original/
>>>>>>
>>>>>> +? OopMapBlock*? nonstatic_oop_map = _nonstatic_oop_maps;
>>>>>>
>>>>>> Extra space after *
>>>>>>
>>>>>> +? assert(ik->nonstatic_oop_map_count() == 
>>>>>> _field_info->oop_map_blocks->_nonstatic_oop_map_count,
>>>>>> +???? "sanity");
>>>>>>
>>>>>> Second line needs to be indented further:
>>>>>>
>>>>>> ???? assert(ik->nonstatic_oop_map_count() == 
>>>>>> _field_info->oop_map_blocks->_nonstatic_oop_map_count,
>>>>>> ??????????? "sanity");
>>>>>>
>>>>>> ---
>>>>>>
>>>>>> src/hotspot/share/classfile/classFileParser.hpp
>>>>>>
>>>>>> +public:
>>>>>> +? OopMapBlock* _nonstatic_oop_maps;
>>>>>> +? unsigned int _nonstatic_oop_map_count;
>>>>>> +? unsigned int _max_nonstatic_oop_maps;
>>>>>> +
>>>>>> + public:
>>>>>>
>>>>>> Second public uneeded. First public may be indented wrong (I'm not 
>>>>>> sure what the rule is - single space indent?)
>>>>>>
>>>>>> ?? class ClassFileParser {
>>>>>> +? friend class FieldLayoutBuilder;
>>>>>> +? friend class FieldLayout;
>>>>>>
>>>>>> ??? class ClassAnnotationCollector;
>>>>>> ??? class FieldAllocationCount;
>>>>>> ??? class FieldAnnotationCollector;
>>>>>>
>>>>>> Indents are different. I think the class forward declarations 
>>>>>> should have extra space.
>>>>>>
>>>>>> ---
>>>>>>
>>>>>> src/hotspot/share/oops/instanceKlass.hpp
>>>>>>
>>>>>> +? void increment_count(int diff)???? { _count += diff; }
>>>>>>
>>>>>> Extra spaces before {
>>>>>>
>>>>>> ---
>>>>>>
>>>>>> src/hotspot/share/runtime/globals.hpp
>>>>>>
>>>>>> +? diagnostic(bool, UseNewFieldLayout, true,???? \
>>>>>> +?????????????? "Use new algorithm to compute layouts")???? \
>>>>>> +???? \
>>>>>> +? product(bool, UseEmptySlotsInSupers, true,
>>>>>>
>>>>>> Not sure I see why one flag is diagnostic and the other product. 
>>>>>> Do you expect people to need to disable using empty slots more so 
>>>>>> than needing to disable using the new field layout altogether?
>>>>>>
>>>>>> ---
>>>>>>
>>>>>> src/hotspot/share/classfile/fieldLayoutBuilder.cpp
>>>>>>
>>>>>> +? assert(kind == EMPTY || kind == RESERVED || kind == PADDING || 
>>>>>> kind == INHERITED,
>>>>>> +????? "Otherwise, should use the constructor with a field index 
>>>>>> argument");
>>>>>>
>>>>>> Indentation of second line is wrong.
>>>>>>
>>>>>> +? assert(kind == REGULAR || kind == FLATTENED || kind == INHERITED,
>>>>>> +????? "Other kind do not have a field index");
>>>>>>
>>>>>> Ditto.
>>>>>>
>>>>>>
>>>>>> +? if (list == NULL) return;
>>>>>> +? if (start == NULL) {
>>>>>> +??? start = this->_start;
>>>>>> +? }
>>>>>>
>>>>>> Inconsistent style for single statement if-blocks. Same thing 
>>>>>> later in the file.
>>>>>>
>>>>>> +????? output->print_cr(" @%d \"%s\" %s %d/%d %s",
>>>>>> +????????? b->offset(),
>>>>>> +????????? fi->name(_cp)->as_C_string(),
>>>>>> +????????? fi->signature(_cp)->as_C_string(),
>>>>>> +????????? b->size(),
>>>>>> +????????? b->alignment(),
>>>>>> +????????? "REGULAR");
>>>>>>
>>>>>> Incorrect identation of continuing line. Same for all the 
>>>>>> following print blocks.
>>>>>>
>>>>>> +? } else if (_classname == vmSymbols::java_lang_Boolean() ||
>>>>>> +????? _classname == vmSymbols::java_lang_Character() ||
>>>>>> +????? _classname == vmSymbols::java_lang_Float() ||
>>>>>> +????? _classname == vmSymbols::java_lang_Double() ||
>>>>>> +????? _classname == vmSymbols::java_lang_Byte() ||
>>>>>> +????? _classname == vmSymbols::java_lang_Short() ||
>>>>>> +????? _classname == vmSymbols::java_lang_Integer() ||
>>>>>> +????? _classname == vmSymbols::java_lang_Long()) {
>>>>>>
>>>>>> Incorrect identation of continuing line.
>>>>>>
>>>>>> ---
>>>>>>
>>>>>> src/hotspot/share/classfile/fieldLayoutBuilder.hpp
>>>>>>
>>>>>> +// and the boxing classes). The rational for having multiple methods
>>>>>>
>>>>>> s/rational/rationale/
>>>>>>
>>>>>> +? FieldLayoutBuilder(const Symbol* classname, const 
>>>>>> InstanceKlass* super_klass, ConstantPool* constant_pool,
>>>>>> +????? Array<u2>* fields, bool is_contended, FieldLayoutInfo* info);
>>>>>>
>>>>>> Indentation wrong for continuing line.
>>>>>>
>>>>>> +? int get_alignment() {
>>>>>> +???? assert(_alignment != -1, "Uninitialized");
>>>>>> +???? return _alignment;
>>>>>> +?? }
>>>>>>
>>>>>> Indenting appears off by one.
>>>>>>
>>>>>> ---
>>>>>>
>>>>>> test/hotspot/jtreg/runtime/FieldLayout/FieldDensityTest.java
>>>>>>
>>>>>> + * @run main/othervm -XX:+UseCompressedOops 
>>>>>> -XX:+UseCompressedClassPointers FieldDensityTest
>>>>>> + * @run main/othervm -XX:+UseCompressedOops 
>>>>>> -XX:-UseCompressedClassPointers FieldDensityTest
>>>>>> + * @run main/othervm -XX:-UseCompressedOops 
>>>>>> -XX:-UseCompressedClassPointers FieldDensityTest
>>>>>>
>>>>>> The test won't run on 32-bit platforms as the compressed oops 
>>>>>> flags won't exist.
>>>>>>
>>>>>> ---
>>>>>>
>>>>>> Some follow up comments below ...
>>>>>>
>>>>>> With trimming ...
>>>>>>
>>>>>> On 25/01/2020 3:20 am, Frederic Parain wrote:
>>>>>>>> On Jan 24, 2020, at 08:19, David Holmes 
>>>>>>>> <david.holmes at oracle.com> wrote:
>>>>>>>>
>>>>>>>> 466???? int super_flen?? = super->nof_nonstatic_fields();
>>>>>>>>
>>>>>>>> Could be folded directly into the assert so we don't call in 
>>>>>>>> product.
>>>>>>>
>>>>>>> Calling not_nonstatic_fields() has the side effect to compute 
>>>>>>> non-static fields,
>>>>>>> which is required to get a correct value when reading 
>>>>>>> super->_nonstatic_fields,
>>>>>>> so the call is needed even in product builds.
>>>>>>
>>>>>> Yuck! That's a horrible side-effect - but not your fault 
>>>>>> obviously. :) It would be better to have a nonstatic_fields() 
>>>>>> accessor that has the same lazy initialization side-effect.
>>>>>>
>>>>>>>> General style issue: when breaking a long line with a method 
>>>>>>>> call, the new line (containing arguments) should be indented to 
>>>>>>>> the opening ( of the method call e.g.
>>>>>> ...
>>>>>>>> etc. This applies across all files.
>>>>>>>
>>>>>>> Fixes applied lines 4003, 4011, 4041, 4138, 4143.
>>>>>>
>>>>>> Fix was also needed in other files. Current issues highlighted above.
>>>>>>
>>>>>>>>
>>>>>>>> src/hotspot/share/oops/instanceKlass.hpp
>>>>>>>>
>>>>>>>> You need to be careful with _extra_flags usage if there can be 
>>>>>>>> concurrently updated bits. At the moment it looks like 
>>>>>>>> redefinition is a mutable dynamic property, whilst "contended 
>>>>>>>> annotations" should be a static immutable property - is that right?
>>>>>>>
>>>>>>> Correct, _has_contended_annotations is a static immutable 
>>>>>>> property, while _is_being_redefined is a mutable one.
>>>>>>
>>>>>> Good to know. My concern is that if someone adds a new mutable 
>>>>>> flag bit the need for atomic updates may not be noticed. We got 
>>>>>> bitten by this in the past with a flag field and I think we 
>>>>>> eventually migrated all of the mutable bits out into their own 
>>>>>> field. (Coleen should recall that :) ).
>>>>>>
>>>>>>>> ?? 61???? FLATTENED,???? // flattened field
>>>>>>>>
>>>>>>>> Does this have any meaning before inline types come in?
>>>>>>>
>>>>>>> Yes, I wanted to reserved the entry in the enum.
>>>>>>
>>>>>> Hmmm a tenuous "okay". Seems odd to require this now to support 
>>>>>> code that is still some way from joining mainline.
>>>>>>
>>>>>>>> In FieldLayoutBuilder::epilogue you have a number of calls to 
>>>>>>>> Thread::current() as well as an implicit call when you use 
>>>>>>>> ResourceMarks. You should capture the current thread once in a 
>>>>>>>> local and reuse it.
>>>>>>>
>>>>>>> Fixed
>>>>>>
>>>>>> It seems that this fix is now not needed as there is only one use 
>>>>>> left of the new "thread" variable in the ResourceMark. So that can 
>>>>>> return to being:
>>>>>>
>>>>>> ResourceMark rm;
>>>>>>
>>>>>> Thanks,
>>>>>> David
>>>>>> -----
>>>>>>
>>>>
>>>
> 

From david.holmes at oracle.com  Fri Jan 31 00:23:50 2020
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 31 Jan 2020 10:23:50 +1000
Subject: RFR[L]: 8237767 Field layout computation overhaul
In-Reply-To: <fe0db242-2610-c612-4213-e63c53803ccd@oracle.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
 <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com>
 <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com>
 <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com>
 <77ca9ddd-e633-6b72-4301-b6d16c61509d@oracle.com>
 <4bd88392-338a-6287-a16e-1fb3949b85f7@oracle.com>
 <BFA088F7-E55B-4B47-A1CB-61A9E6900D19@oracle.com>
 <e4ca9eb9-21ce-fe65-c29d-31da69c535e6@oracle.com>
 <88d83416-58c1-7026-aaff-208929d9b9d6@oracle.com>
 <fe0db242-2610-c612-4213-e63c53803ccd@oracle.com>
Message-ID: <e75263ae-083c-9c34-b233-f5a2d463d1a3@oracle.com>

Correction ...

On 31/01/2020 9:40 am, David Holmes wrote:
> On 31/01/2020 1:04 am, coleen.phillimore at oracle.com wrote:
>> On 1/29/20 6:44 PM, David Holmes wrote:
>>> Hi Fred,
>>>
>>> On 30/01/2020 12:57 am, Frederic Parain wrote:
>>>> The kind of the flag is a secondary issue, the main issue is ?how
>>>> fast do we want to remove the old code??. I wanted to keep the old
>>>> code for a short transition period in case new field layouts cause
>>>> issues for some tools or customers. The goal was not to keep it
>>>> forever and allow users to select which algorithm they want to use.
>>>
>>> Right. The problem with our 6 month release cadence is that we're not 
>>> realistically going to get this new code exposed to users who will 
>>> find potential problems in a single release. Even if we think this 
>>> might mainly impact tools and those developers do tend to try and 
>>> keep up, it is still unlikely to get enough visibility in a single 
>>> release so ...
>>>
>>>> Note that keeping the old code and its activation with a normal
>>>> product flag will be an issue for the first release of Valhalla.
>>>> The way flattened fields are allocated in the old code is very
>>>> inefficient, which means that instead of seeing an improvement
>>>> in data density, users will see a significant increase in space
>>>> consumption if they use the old code.
>>>>
>>>> So, here?s the three possible paths forward:
>>>> ?? 1 - push the new code and remove the old code in a single change,
>>>> ?????? so no need for a VM option to select which code to use
>>>
>>> Seems potentially dangerous when we don't know the impact.
>>>
>>>> ?? 2 - push the new code while keeping the old one for a short
>>>> ?????? period of time, to remove the old code before Valhalla code
>>>> ?????? is pushed
>>>
>>> Begs the question: when do we think something of Valhalla will ship? 
>>> I'm guessing not before 17. If so then we deprecate the flag in 16 
>>> and obsolete in 17 - thus zero impact on Valhalla. If Valhalla comes 
>>> earlier in 16 then we have some impact but will still have this out 
>>> of the way before 17 (which should be next LTS release).
>>>
>>>> ?? 3 - push the new code while keeping the old one with a long
>>>> ?????? deprecation period before removing the old code
>>>>
>>>> Option 3 will be an issue for project Valhalla, as explained
>>>> above.
>>>>
>>>> Option 1 sounds a big risky to me, from the experience I had
>>>> while doing this change in the JVM and the unexpected places
>>>> where it caused issues.
>>>>
>>>> Option 2 sounded like a good compromise, however, it seems
>>>> there?s no consensus how to implement it. Could we refine
>>>> the details on how a short transition period could be implemented?
>>>
>>> Add flag for 15; deprecate in 16; obsolete in 17; expire in 18.
>>
>> How about add flag for 15, pre-deprecated, obsolete in 16 so that 
>> valhalla work can go forward, especially if there's a preview in 16. 
>> Either way the old code is removed in JDK 17, the LTS release.
> 
> That is fine too.
> 
>>> I'm concerned we are developing a tendency to make flags diagnostic 
>>> because it is considered convenient for us. There are customers whose 
>>> own policies disallow running their production systems with anything 
>>> other than product flags. To me a diagnostic flag aids in diagnosing 
>>> a problem more than being a switch between old and new behaviour - 
>>> but I agree the lines are blurry. Historically there was much more 
>>> difference between product flags and diagnostic, when product flags 
>>> would be supported in a number of releases spanning many years. That 
>>> is no longer the case. We can effectively kill off a product flag 
>>> after 12 months: 6 months full "support"; 6 months deprecated; then 
>>> obsolete. We can even deprecate a flag in the release we introduce it 
>>> if we really think that is warranted.
>>
>> This 6 months of support vs. 9 months isn't going to make a practical 
>> difference to the customers.? They can fall back on JDK 11, the 
>> previous LTS release until they fix their code.? 6 months of "support" 
>> is better for us however since we don't have code paths that we should 
>> be testing, and will be a broken (or preferably disabled) code path in 
>> the valhalla repo that we're working on and asking people to try right 
>> now.
>>>
>>> If we intend to tell anyone to use the flag then we have to ensure 
>>> both code paths remain fully functional during that period.
>>
>> That's the thing.? We don't _want_ to tell anyone to use this flag 
>> unless they are diagnosing a problem with their code. 
> 
> Regardless of the "why" if we tell people to use this flag as a 
> workaround then the code path has to be functional.
> 
>> The only reason I can see making it a product flag is because we want 
>> customers to fix their code, and not us to add hacks to our new code 
>> to support any corner case layout that _might_ exist.
> 
> A product flag is good for users and has no downside for us. A 
> diagnostic flag is not so good for users and has no upside for us. The 
> old code is going to remain, and must continue to work, for the exact 
> same length of time. Having one product flag and one diagnostic flag 
> makes no sense to me.

Sorry I just realized that the two flags are unrelated. One pertains to 
using the old code versus new; the other controls a more aggressive 
optimisation in the new code.

David

> Cheers,
> David
> 
>> Coleen
>>>
>>> Cheers,
>>> David
>>>
>>>> Thank you,
>>>>
>>>> Fred
>>>>
>>>>
>>>>> On Jan 29, 2020, at 08:44, coleen.phillimore at oracle.com wrote:
>>>>>
>>>>>
>>>>>
>>>>> On 1/29/20 8:31 AM, David Holmes wrote:
>>>>>> PS. I've now seen the CSR and have commented there. I don't agree 
>>>>>> with the use of a diagnostic flag. It doesn't buy us anything 
>>>>>> other than not needing a CSR request when we decide to deprecate 
>>>>>> it. But if we're going to tell people to use these flags to revert 
>>>>>> behaviour then they will need to follow the deprecation process 
>>>>>> regardless - and a product flag is better from an end user 
>>>>>> perspective.
>>>>>
>>>>> Obviously I disagree.? We don't want to publicise this flag 
>>>>> *unless* someone has a problem, which makes it a diagnostic flag.   
>>>>> We don't want to have to carry the old code 2 releases when there 
>>>>> is honestly no reason to suspect it is problematic.
>>>>>
>>>>> Coleen
>>>>>
>>>>>>
>>>>>> Cheers,
>>>>>> David
>>>>>>
>>>>>> On 29/01/2020 5:06 pm, David Holmes wrote:
>>>>>>> Hi Fred,
>>>>>>>
>>>>>>> I've looked at the v7 version. A few more stylistic comments on 
>>>>>>> that first. Note, no need for an item by item response unless 
>>>>>>> that makes it easier for you to track :)
>>>>>>>
>>>>>>> src/hotspot/share/classfile/classFileParser.cpp
>>>>>>>
>>>>>>> ?? #include "classfile/defaultMethods.hpp"
>>>>>>> +#include "classfile/fieldLayoutBuilder.hpp"
>>>>>>> ?? #include "classfile/dictionary.hpp"
>>>>>>>
>>>>>>> Include files are not in alphabetical order.
>>>>>>>
>>>>>>> +?? * This may well change: FixMe if doesn't,
>>>>>>>
>>>>>>> s/if/if it/
>>>>>>>
>>>>>>> +? //Make a temp copy, and iterate through and copy back into the 
>>>>>>> orig
>>>>>>>
>>>>>>> Space after //
>>>>>>>
>>>>>>> s/orig/original/
>>>>>>>
>>>>>>> +? OopMapBlock*? nonstatic_oop_map = _nonstatic_oop_maps;
>>>>>>>
>>>>>>> Extra space after *
>>>>>>>
>>>>>>> +? assert(ik->nonstatic_oop_map_count() == 
>>>>>>> _field_info->oop_map_blocks->_nonstatic_oop_map_count,
>>>>>>> +???? "sanity");
>>>>>>>
>>>>>>> Second line needs to be indented further:
>>>>>>>
>>>>>>> ???? assert(ik->nonstatic_oop_map_count() == 
>>>>>>> _field_info->oop_map_blocks->_nonstatic_oop_map_count,
>>>>>>> ??????????? "sanity");
>>>>>>>
>>>>>>> ---
>>>>>>>
>>>>>>> src/hotspot/share/classfile/classFileParser.hpp
>>>>>>>
>>>>>>> +public:
>>>>>>> +? OopMapBlock* _nonstatic_oop_maps;
>>>>>>> +? unsigned int _nonstatic_oop_map_count;
>>>>>>> +? unsigned int _max_nonstatic_oop_maps;
>>>>>>> +
>>>>>>> + public:
>>>>>>>
>>>>>>> Second public uneeded. First public may be indented wrong (I'm 
>>>>>>> not sure what the rule is - single space indent?)
>>>>>>>
>>>>>>> ?? class ClassFileParser {
>>>>>>> +? friend class FieldLayoutBuilder;
>>>>>>> +? friend class FieldLayout;
>>>>>>>
>>>>>>> ??? class ClassAnnotationCollector;
>>>>>>> ??? class FieldAllocationCount;
>>>>>>> ??? class FieldAnnotationCollector;
>>>>>>>
>>>>>>> Indents are different. I think the class forward declarations 
>>>>>>> should have extra space.
>>>>>>>
>>>>>>> ---
>>>>>>>
>>>>>>> src/hotspot/share/oops/instanceKlass.hpp
>>>>>>>
>>>>>>> +? void increment_count(int diff)???? { _count += diff; }
>>>>>>>
>>>>>>> Extra spaces before {
>>>>>>>
>>>>>>> ---
>>>>>>>
>>>>>>> src/hotspot/share/runtime/globals.hpp
>>>>>>>
>>>>>>> +? diagnostic(bool, UseNewFieldLayout, true,???? \
>>>>>>> +?????????????? "Use new algorithm to compute layouts")???? \
>>>>>>> +???? \
>>>>>>> +? product(bool, UseEmptySlotsInSupers, true,
>>>>>>>
>>>>>>> Not sure I see why one flag is diagnostic and the other product. 
>>>>>>> Do you expect people to need to disable using empty slots more so 
>>>>>>> than needing to disable using the new field layout altogether?
>>>>>>>
>>>>>>> ---
>>>>>>>
>>>>>>> src/hotspot/share/classfile/fieldLayoutBuilder.cpp
>>>>>>>
>>>>>>> +? assert(kind == EMPTY || kind == RESERVED || kind == PADDING || 
>>>>>>> kind == INHERITED,
>>>>>>> +????? "Otherwise, should use the constructor with a field index 
>>>>>>> argument");
>>>>>>>
>>>>>>> Indentation of second line is wrong.
>>>>>>>
>>>>>>> +? assert(kind == REGULAR || kind == FLATTENED || kind == INHERITED,
>>>>>>> +????? "Other kind do not have a field index");
>>>>>>>
>>>>>>> Ditto.
>>>>>>>
>>>>>>>
>>>>>>> +? if (list == NULL) return;
>>>>>>> +? if (start == NULL) {
>>>>>>> +??? start = this->_start;
>>>>>>> +? }
>>>>>>>
>>>>>>> Inconsistent style for single statement if-blocks. Same thing 
>>>>>>> later in the file.
>>>>>>>
>>>>>>> +????? output->print_cr(" @%d \"%s\" %s %d/%d %s",
>>>>>>> +????????? b->offset(),
>>>>>>> +????????? fi->name(_cp)->as_C_string(),
>>>>>>> +????????? fi->signature(_cp)->as_C_string(),
>>>>>>> +????????? b->size(),
>>>>>>> +????????? b->alignment(),
>>>>>>> +????????? "REGULAR");
>>>>>>>
>>>>>>> Incorrect identation of continuing line. Same for all the 
>>>>>>> following print blocks.
>>>>>>>
>>>>>>> +? } else if (_classname == vmSymbols::java_lang_Boolean() ||
>>>>>>> +????? _classname == vmSymbols::java_lang_Character() ||
>>>>>>> +????? _classname == vmSymbols::java_lang_Float() ||
>>>>>>> +????? _classname == vmSymbols::java_lang_Double() ||
>>>>>>> +????? _classname == vmSymbols::java_lang_Byte() ||
>>>>>>> +????? _classname == vmSymbols::java_lang_Short() ||
>>>>>>> +????? _classname == vmSymbols::java_lang_Integer() ||
>>>>>>> +????? _classname == vmSymbols::java_lang_Long()) {
>>>>>>>
>>>>>>> Incorrect identation of continuing line.
>>>>>>>
>>>>>>> ---
>>>>>>>
>>>>>>> src/hotspot/share/classfile/fieldLayoutBuilder.hpp
>>>>>>>
>>>>>>> +// and the boxing classes). The rational for having multiple 
>>>>>>> methods
>>>>>>>
>>>>>>> s/rational/rationale/
>>>>>>>
>>>>>>> +? FieldLayoutBuilder(const Symbol* classname, const 
>>>>>>> InstanceKlass* super_klass, ConstantPool* constant_pool,
>>>>>>> +????? Array<u2>* fields, bool is_contended, FieldLayoutInfo* info);
>>>>>>>
>>>>>>> Indentation wrong for continuing line.
>>>>>>>
>>>>>>> +? int get_alignment() {
>>>>>>> +???? assert(_alignment != -1, "Uninitialized");
>>>>>>> +???? return _alignment;
>>>>>>> +?? }
>>>>>>>
>>>>>>> Indenting appears off by one.
>>>>>>>
>>>>>>> ---
>>>>>>>
>>>>>>> test/hotspot/jtreg/runtime/FieldLayout/FieldDensityTest.java
>>>>>>>
>>>>>>> + * @run main/othervm -XX:+UseCompressedOops 
>>>>>>> -XX:+UseCompressedClassPointers FieldDensityTest
>>>>>>> + * @run main/othervm -XX:+UseCompressedOops 
>>>>>>> -XX:-UseCompressedClassPointers FieldDensityTest
>>>>>>> + * @run main/othervm -XX:-UseCompressedOops 
>>>>>>> -XX:-UseCompressedClassPointers FieldDensityTest
>>>>>>>
>>>>>>> The test won't run on 32-bit platforms as the compressed oops 
>>>>>>> flags won't exist.
>>>>>>>
>>>>>>> ---
>>>>>>>
>>>>>>> Some follow up comments below ...
>>>>>>>
>>>>>>> With trimming ...
>>>>>>>
>>>>>>> On 25/01/2020 3:20 am, Frederic Parain wrote:
>>>>>>>>> On Jan 24, 2020, at 08:19, David Holmes 
>>>>>>>>> <david.holmes at oracle.com> wrote:
>>>>>>>>>
>>>>>>>>> 466???? int super_flen?? = super->nof_nonstatic_fields();
>>>>>>>>>
>>>>>>>>> Could be folded directly into the assert so we don't call in 
>>>>>>>>> product.
>>>>>>>>
>>>>>>>> Calling not_nonstatic_fields() has the side effect to compute 
>>>>>>>> non-static fields,
>>>>>>>> which is required to get a correct value when reading 
>>>>>>>> super->_nonstatic_fields,
>>>>>>>> so the call is needed even in product builds.
>>>>>>>
>>>>>>> Yuck! That's a horrible side-effect - but not your fault 
>>>>>>> obviously. :) It would be better to have a nonstatic_fields() 
>>>>>>> accessor that has the same lazy initialization side-effect.
>>>>>>>
>>>>>>>>> General style issue: when breaking a long line with a method 
>>>>>>>>> call, the new line (containing arguments) should be indented to 
>>>>>>>>> the opening ( of the method call e.g.
>>>>>>> ...
>>>>>>>>> etc. This applies across all files.
>>>>>>>>
>>>>>>>> Fixes applied lines 4003, 4011, 4041, 4138, 4143.
>>>>>>>
>>>>>>> Fix was also needed in other files. Current issues highlighted 
>>>>>>> above.
>>>>>>>
>>>>>>>>>
>>>>>>>>> src/hotspot/share/oops/instanceKlass.hpp
>>>>>>>>>
>>>>>>>>> You need to be careful with _extra_flags usage if there can be 
>>>>>>>>> concurrently updated bits. At the moment it looks like 
>>>>>>>>> redefinition is a mutable dynamic property, whilst "contended 
>>>>>>>>> annotations" should be a static immutable property - is that 
>>>>>>>>> right?
>>>>>>>>
>>>>>>>> Correct, _has_contended_annotations is a static immutable 
>>>>>>>> property, while _is_being_redefined is a mutable one.
>>>>>>>
>>>>>>> Good to know. My concern is that if someone adds a new mutable 
>>>>>>> flag bit the need for atomic updates may not be noticed. We got 
>>>>>>> bitten by this in the past with a flag field and I think we 
>>>>>>> eventually migrated all of the mutable bits out into their own 
>>>>>>> field. (Coleen should recall that :) ).
>>>>>>>
>>>>>>>>> ?? 61???? FLATTENED,???? // flattened field
>>>>>>>>>
>>>>>>>>> Does this have any meaning before inline types come in?
>>>>>>>>
>>>>>>>> Yes, I wanted to reserved the entry in the enum.
>>>>>>>
>>>>>>> Hmmm a tenuous "okay". Seems odd to require this now to support 
>>>>>>> code that is still some way from joining mainline.
>>>>>>>
>>>>>>>>> In FieldLayoutBuilder::epilogue you have a number of calls to 
>>>>>>>>> Thread::current() as well as an implicit call when you use 
>>>>>>>>> ResourceMarks. You should capture the current thread once in a 
>>>>>>>>> local and reuse it.
>>>>>>>>
>>>>>>>> Fixed
>>>>>>>
>>>>>>> It seems that this fix is now not needed as there is only one use 
>>>>>>> left of the new "thread" variable in the ResourceMark. So that 
>>>>>>> can return to being:
>>>>>>>
>>>>>>> ResourceMark rm;
>>>>>>>
>>>>>>> Thanks,
>>>>>>> David
>>>>>>> -----
>>>>>>>
>>>>>
>>>>
>>

From kim.barrett at oracle.com  Fri Jan 31 06:51:28 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Fri, 31 Jan 2020 01:51:28 -0500
Subject: RFR[XS]: 8238272: Eliminate cast_from_oop to narrowOop* 
Message-ID: <267AF7FC-B6EB-4232-A51F-035A7217BCC6@oracle.com>

Please review this change to ObjArrayKlass::oop_oop_iterate_range to
eliminate a conditional cast_from_oop to either either narrowOop* or
oop*.  Casting an oop to either of those types seems pretty
questionable, and is not actually necessary here.

(This appears to be the *only* place where cast_from_oop<narrowOop*>()
is used.  There are other casts to oop* that should be examined.)

This change has the additional minor benefit that it allows the
compiler to eliminate the lower bounds clamp when start is a constant
zero, and eliminates the conditional test for zero when it isn't a
constant.

CR:
https://bugs.openjdk.java.net/browse/JDK-8238272

Webrev:
https://cr.openjdk.java.net/~kbarrett/8238272/open.00/

Testing:
mach5 tier1-5
Compared gcc generated code for linux-x64 with and without the change
for calls with and without a literal 0 start index.


From shade at redhat.com  Fri Jan 31 07:07:18 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Fri, 31 Jan 2020 08:07:18 +0100
Subject: RFR[XS]: 8238272: Eliminate cast_from_oop to narrowOop*
In-Reply-To: <267AF7FC-B6EB-4232-A51F-035A7217BCC6@oracle.com>
References: <267AF7FC-B6EB-4232-A51F-035A7217BCC6@oracle.com>
Message-ID: <44ad6bd5-b66a-3306-0cb9-f518d1b8e2c4@redhat.com>

On 1/31/20 7:51 AM, Kim Barrett wrote:
> Webrev:
> https://cr.openjdk.java.net/~kbarrett/8238272/open.00/

Hm, I am puzzled a bit.

a->obj_at_addr_raw(0) seems to point to the first array element (objArrayOopDesc::base_raw + 0 =
arrayOopDesc::base_raw(T_OBJECT) = cast_from_oop<intptr_t>(as_oop()) + base_offset_in_bytes(type) =
header_size(type) * HeapWordSize). Whereas cast_from_oop<T*>(a) points to the object header.

So start == 0 is really a special case here, do we know why?

> Compared gcc generated code for linux-x64 with and without the change
> for calls with and without a literal 0 start index.

And it was the same, contrary to the idea above?

-- 
Thanks,
-Aleksey


From kim.barrett at oracle.com  Fri Jan 31 07:35:45 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Fri, 31 Jan 2020 02:35:45 -0500
Subject: RFR[XS]: 8238272: Eliminate cast_from_oop to narrowOop*
In-Reply-To: <44ad6bd5-b66a-3306-0cb9-f518d1b8e2c4@redhat.com>
References: <267AF7FC-B6EB-4232-A51F-035A7217BCC6@oracle.com>
 <44ad6bd5-b66a-3306-0cb9-f518d1b8e2c4@redhat.com>
Message-ID: <805C4111-CF21-4EAF-AC78-742045A3DD73@oracle.com>

> On Jan 31, 2020, at 2:07 AM, Aleksey Shipilev <shade at redhat.com> wrote:
> 
> On 1/31/20 7:51 AM, Kim Barrett wrote:
>> Webrev:
>> https://cr.openjdk.java.net/~kbarrett/8238272/open.00/
> 
> Hm, I am puzzled a bit.
> 
> a->obj_at_addr_raw(0) seems to point to the first array element (objArrayOopDesc::base_raw + 0 =
> arrayOopDesc::base_raw(T_OBJECT) = cast_from_oop<intptr_t>(as_oop()) + base_offset_in_bytes(type) =
> header_size(type) * HeapWordSize). Whereas cast_from_oop<T*>(a) points to the object header.
> 
> So start == 0 is really a special case here, do we know why?

I realized there?s a problem shortly after sending out the RFR, and was starting to write
this when I saw your reply.

I should have realized this after looking at the end computation in the line after the change.
obj_at_addr_raw asserts the index is in range, which is exclusive on the length.  That?s
why the end computation hand-inlines the same address calculation as done by
obj_at_addr_raw, rather than using that function.  With my change we?ll get an assert
if the array length is also 0, even if start is also 0.

The fix is to use the same hand-inlined address calculation for start as for end, perhaps
packaging that up in a new function.  I?ll get back to this next week.  Sorry for the noise.



From jiefu at tencent.com  Fri Jan 31 11:01:16 2020
From: jiefu at tencent.com (=?utf-8?B?amllZnUo5YKF5p2wKQ==?=)
Date: Fri, 31 Jan 2020 11:01:16 +0000
Subject: RFR: 8238284: [macos] Zero VM build fails due to an obvious typo
Message-ID: <688CD21F-736B-4615-BC02-76DC6FB05D27@tencent.com>

Hi all,

JBS:    https://bugs.openjdk.java.net/browse/JDK-8238284
Webrev: http://cr.openjdk.java.net/~jiefu/8238284/webrev.00/

Zero VM build is broken due to an obvious typo [1].

More errors can be triggered with clang 11, which can be fixed by adding `const`.
------------------------------------------
/Users/fool/workspace/open/jdk/src/hotspot/os_cpu/bsd_zero/os_bsd_zero.cpp:369:8: error: conflicting types for '_Copy_conjoint_jshorts_atomic'
  void _Copy_conjoint_jshorts_atomic(jshort* from, jshort* to, size_t count) {
       ^
/Users/fool/workspace/open/jdk/src/hotspot/share/utilities/copy.hpp:47:8: note: previous declaration is here
  void _Copy_conjoint_jshorts_atomic(const jshort* from, jshort* to, size_t count);
       ^
...

/Users/fool/workspace/open/jdk/src/hotspot/os_cpu/bsd_zero/os_bsd_zero.cpp:371:15: error: cannot initialize a variable of type 'jshort *' (aka 'short *') with an rvalue of type 'const jshort *' (aka 'const short *')
      jshort *end = from + count;
              ^     ~~~~~~~~~~~~
...
------------------------------------------

Could you please review it and give me some advice?

Thanks a lot.
Best regards,
Jie

[1] http://hg.openjdk.java.net/jdk/jdk/file/0905868db490/src/hotspot/os_cpu/bsd_zero/atomic_bsd_zero.hpp#l192

From david.holmes at oracle.com  Fri Jan 31 11:59:37 2020
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 31 Jan 2020 21:59:37 +1000
Subject: RFR: 8238284: [macos] Zero VM build fails due to an obvious typo
In-Reply-To: <688CD21F-736B-4615-BC02-76DC6FB05D27@tencent.com>
References: <688CD21F-736B-4615-BC02-76DC6FB05D27@tencent.com>
Message-ID: <7c1194e6-0659-2694-2822-d3ea87dea296@oracle.com>

Hi Jie,

On 31/01/2020 9:01 pm, jiefu(??) wrote:
> Hi all,
> 
> JBS:    https://bugs.openjdk.java.net/browse/JDK-8238284
> Webrev: http://cr.openjdk.java.net/~jiefu/8238284/webrev.00/
> 
> Zero VM build is broken due to an obvious typo [1].
> 
> More errors can be triggered with clang 11, which can be fixed by adding `const`.
> ------------------------------------------
> /Users/fool/workspace/open/jdk/src/hotspot/os_cpu/bsd_zero/os_bsd_zero.cpp:369:8: error: conflicting types for '_Copy_conjoint_jshorts_atomic'
>    void _Copy_conjoint_jshorts_atomic(jshort* from, jshort* to, size_t count) {
>         ^
> /Users/fool/workspace/open/jdk/src/hotspot/share/utilities/copy.hpp:47:8: note: previous declaration is here
>    void _Copy_conjoint_jshorts_atomic(const jshort* from, jshort* to, size_t count);
>         ^
> ...
> 
> /Users/fool/workspace/open/jdk/src/hotspot/os_cpu/bsd_zero/os_bsd_zero.cpp:371:15: error: cannot initialize a variable of type 'jshort *' (aka 'short *') with an rvalue of type 'const jshort *' (aka 'const short *')
>        jshort *end = from + count;
>                ^     ~~~~~~~~~~~~
> ...
> ------------------------------------------
> 
> Could you please review it and give me some advice?

Those fixes all look obviously correct.

I note however that the same problem regarding lack of const exists in 
other zero implementations. Should they also be fixed?

Thanks,
David

> Thanks a lot.
> Best regards,
> Jie
> 
> [1] http://hg.openjdk.java.net/jdk/jdk/file/0905868db490/src/hotspot/os_cpu/bsd_zero/atomic_bsd_zero.hpp#l192
> 

From jiefu at tencent.com  Fri Jan 31 13:55:41 2020
From: jiefu at tencent.com (=?utf-8?B?amllZnUo5YKF5p2wKQ==?=)
Date: Fri, 31 Jan 2020 13:55:41 +0000
Subject: RFR: 8238284: [macos] Zero VM build fails due to an obvious typo
Message-ID: <85817F78-9D26-4FA0-8ABD-58830A714B15@tencent.com>

Hi David,

Thanks for your review and nice suggestion.

Updated: http://cr.openjdk.java.net/~jiefu/8238284/webrev.01/

Testing:
  - zero build tests on both macos and linux/x64.

Thanks a lot.
Best regards,
Jie


?On 2020/1/31, 8:00 PM, "David Holmes" <david.holmes at oracle.com> wrote:

    Hi Jie,
    
    I note however that the same problem regarding lack of const exists in 
    other zero implementations. Should they also be fixed?
    
Fixed. Thanks.    
    


From frederic.parain at oracle.com  Fri Jan 31 16:03:54 2020
From: frederic.parain at oracle.com (Frederic Parain)
Date: Fri, 31 Jan 2020 11:03:54 -0500
Subject: RFR[L]: 8237767 Field layout computation overhaul 
In-Reply-To: <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com>
References: <AE044ED9-D523-4FCC-BCD6-5F2D03931A04@oracle.com>
 <26a3be7e-71d2-848a-be62-31c9c52c17ac@oracle.com>
 <50C2C1B1-53C8-4C0A-984C-4A4F7416E1F5@oracle.com>
 <498a199d-ad3e-8bf9-df6a-8386b9bc22cd@oracle.com>
Message-ID: <B13DA421-E463-4B29-AD23-F297E0056636@oracle.com>

Hi David,

I?ve fixed all the issues you mentioned below.

According to the discussion about the VM flags, the following modifications
have also been made:

globals.hpp:
  UseNewFieldLayout is now a deprecated product flag

fieldLayoutBuilder.cpp
  lines 350-362:
  	UseEmptySlotsInSupers VM option now controls both new optimizations
  lines 449-464
	Fix an issue with layout printing (inherited fields were printed
	only for the direct super-class, now all inherited fields are printed)

New webrev:
http://cr.openjdk.java.net/~fparain/jdk_layout/webrev.08/index.html


CR and CSR have been updated accordingly (new VM flags explanations in the CSR).

Thank you,

Fred


> On Jan 29, 2020, at 02:06, David Holmes <david.holmes at oracle.com> wrote:
> 
> Hi Fred,
> 
> I've looked at the v7 version. A few more stylistic comments on that first. Note, no need for an item by item response unless that makes it easier for you to track :)
> 
> src/hotspot/share/classfile/classFileParser.cpp
> 
> #include "classfile/defaultMethods.hpp"
> +#include "classfile/fieldLayoutBuilder.hpp"
> #include "classfile/dictionary.hpp"
> 
> Include files are not in alphabetical order.
> 
> +   * This may well change: FixMe if doesn't,
> 
> s/if/if it/
> 
> +  //Make a temp copy, and iterate through and copy back into the orig
> 
> Space after //
> 
> s/orig/original/
> 
> +  OopMapBlock*  nonstatic_oop_map = _nonstatic_oop_maps;
> 
> Extra space after *
> 
> +  assert(ik->nonstatic_oop_map_count() == _field_info->oop_map_blocks->_nonstatic_oop_map_count,
> +     "sanity");
> 
> Second line needs to be indented further:
> 
>   assert(ik->nonstatic_oop_map_count() == _field_info->oop_map_blocks->_nonstatic_oop_map_count,
>          "sanity");
> 
> ---
> 
> src/hotspot/share/classfile/classFileParser.hpp
> 
> +public:
> +  OopMapBlock* _nonstatic_oop_maps;
> +  unsigned int _nonstatic_oop_map_count;
> +  unsigned int _max_nonstatic_oop_maps;
> +
> + public:
> 
> Second public uneeded. First public may be indented wrong (I'm not sure what the rule is - single space indent?)
> 
> class ClassFileParser {
> +  friend class FieldLayoutBuilder;
> +  friend class FieldLayout;
> 
>  class ClassAnnotationCollector;
>  class FieldAllocationCount;
>  class FieldAnnotationCollector;
> 
> Indents are different. I think the class forward declarations should have extra space.
> 
> ---
> 
> src/hotspot/share/oops/instanceKlass.hpp
> 
> +  void increment_count(int diff)     { _count += diff; }
> 
> Extra spaces before {
> 
> ---
> 
> src/hotspot/share/runtime/globals.hpp
> 
> +  diagnostic(bool, UseNewFieldLayout, true,     \
> +               "Use new algorithm to compute layouts")     \
> +     \
> +  product(bool, UseEmptySlotsInSupers, true,
> 
> Not sure I see why one flag is diagnostic and the other product. Do you expect people to need to disable using empty slots more so than needing to disable using the new field layout altogether?
> 
> ---
> 
> src/hotspot/share/classfile/fieldLayoutBuilder.cpp
> 
> +  assert(kind == EMPTY || kind == RESERVED || kind == PADDING || kind == INHERITED,
> +      "Otherwise, should use the constructor with a field index argument");
> 
> Indentation of second line is wrong.
> 
> +  assert(kind == REGULAR || kind == FLATTENED || kind == INHERITED,
> +      "Other kind do not have a field index");
> 
> Ditto.
> 
> 
> +  if (list == NULL) return;
> +  if (start == NULL) {
> +    start = this->_start;
> +  }
> 
> Inconsistent style for single statement if-blocks. Same thing later in the file.
> 
> +      output->print_cr(" @%d \"%s\" %s %d/%d %s",
> +          b->offset(),
> +          fi->name(_cp)->as_C_string(),
> +          fi->signature(_cp)->as_C_string(),
> +          b->size(),
> +          b->alignment(),
> +          "REGULAR");
> 
> Incorrect identation of continuing line. Same for all the following print blocks.
> 
> +  } else if (_classname == vmSymbols::java_lang_Boolean() ||
> +      _classname == vmSymbols::java_lang_Character() ||
> +      _classname == vmSymbols::java_lang_Float() ||
> +      _classname == vmSymbols::java_lang_Double() ||
> +      _classname == vmSymbols::java_lang_Byte() ||
> +      _classname == vmSymbols::java_lang_Short() ||
> +      _classname == vmSymbols::java_lang_Integer() ||
> +      _classname == vmSymbols::java_lang_Long()) {
> 
> Incorrect identation of continuing line.
> 
> ---
> 
> src/hotspot/share/classfile/fieldLayoutBuilder.hpp
> 
> +// and the boxing classes). The rational for having multiple methods
> 
> s/rational/rationale/
> 
> +  FieldLayoutBuilder(const Symbol* classname, const InstanceKlass* super_klass, ConstantPool* constant_pool,
> +      Array<u2>* fields, bool is_contended, FieldLayoutInfo* info);
> 
> Indentation wrong for continuing line.
> 
> +  int get_alignment() {
> +     assert(_alignment != -1, "Uninitialized");
> +     return _alignment;
> +   }
> 
> Indenting appears off by one.
> 
> ---
> 
> test/hotspot/jtreg/runtime/FieldLayout/FieldDensityTest.java
> 
> + * @run main/othervm -XX:+UseCompressedOops -XX:+UseCompressedClassPointers FieldDensityTest
> + * @run main/othervm -XX:+UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest
> + * @run main/othervm -XX:-UseCompressedOops -XX:-UseCompressedClassPointers FieldDensityTest
> 
> The test won't run on 32-bit platforms as the compressed oops flags won't exist.
> 
> ---
> 
> Some follow up comments below ...
> 
> With trimming ...
> 
> On 25/01/2020 3:20 am, Frederic Parain wrote:
>>> On Jan 24, 2020, at 08:19, David Holmes <david.holmes at oracle.com> wrote:
>>> 
>>> 466     int super_flen   = super->nof_nonstatic_fields();
>>> 
>>> Could be folded directly into the assert so we don't call in product.
>> Calling not_nonstatic_fields() has the side effect to compute non-static fields,
>> which is required to get a correct value when reading super->_nonstatic_fields,
>> so the call is needed even in product builds.
> 
> Yuck! That's a horrible side-effect - but not your fault obviously. :) It would be better to have a nonstatic_fields() accessor that has the same lazy initialization side-effect.
> 
>>> General style issue: when breaking a long line with a method call, the new line (containing arguments) should be indented to the opening ( of the method call e.g.
> ...
>>> etc. This applies across all files.
>> Fixes applied lines 4003, 4011, 4041, 4138, 4143.
> 
> Fix was also needed in other files. Current issues highlighted above.
> 
>>> 
>>> src/hotspot/share/oops/instanceKlass.hpp
>>> 
>>> You need to be careful with _extra_flags usage if there can be concurrently updated bits. At the moment it looks like redefinition is a mutable dynamic property, whilst "contended annotations" should be a static immutable property - is that right?
>> Correct, _has_contended_annotations is a static immutable property, while _is_being_redefined is a mutable one.
> 
> Good to know. My concern is that if someone adds a new mutable flag bit the need for atomic updates may not be noticed. We got bitten by this in the past with a flag field and I think we eventually migrated all of the mutable bits out into their own field. (Coleen should recall that :) ).
> 
>>>  61     FLATTENED,     // flattened field
>>> 
>>> Does this have any meaning before inline types come in?
>> Yes, I wanted to reserved the entry in the enum.
> 
> Hmmm a tenuous "okay". Seems odd to require this now to support code that is still some way from joining mainline.
> 
>>> In FieldLayoutBuilder::epilogue you have a number of calls to Thread::current() as well as an implicit call when you use ResourceMarks. You should capture the current thread once in a local and reuse it.
>> Fixed
> 
> It seems that this fix is now not needed as there is only one use left of the new "thread" variable in the ResourceMark. So that can return to being:
> 
> ResourceMark rm;
> 
> Thanks,
> David
> -----
> 


From kim.barrett at oracle.com  Fri Jan 31 22:12:14 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Fri, 31 Jan 2020 17:12:14 -0500
Subject: RFR[XS]: 8238272: Eliminate cast_from_oop to narrowOop*
In-Reply-To: <805C4111-CF21-4EAF-AC78-742045A3DD73@oracle.com>
References: <267AF7FC-B6EB-4232-A51F-035A7217BCC6@oracle.com>
 <44ad6bd5-b66a-3306-0cb9-f518d1b8e2c4@redhat.com>
 <805C4111-CF21-4EAF-AC78-742045A3DD73@oracle.com>
Message-ID: <0F754E34-462D-4183-8CE0-F2933D4F310F@oracle.com>

> On Jan 31, 2020, at 2:35 AM, Kim Barrett <kim.barrett at oracle.com> wrote:
> 
>> On Jan 31, 2020, at 2:07 AM, Aleksey Shipilev <shade at redhat.com> wrote:
>> 
>> On 1/31/20 7:51 AM, Kim Barrett wrote:
>>> Webrev:
>>> https://cr.openjdk.java.net/~kbarrett/8238272/open.00/
>> 
>> Hm, I am puzzled a bit.
>> 
>> a->obj_at_addr_raw(0) seems to point to the first array element (objArrayOopDesc::base_raw + 0 =
>> arrayOopDesc::base_raw(T_OBJECT) = cast_from_oop<intptr_t>(as_oop()) + base_offset_in_bytes(type) =
>> header_size(type) * HeapWordSize). Whereas cast_from_oop<T*>(a) points to the object header.
>> 
>> So start == 0 is really a special case here, do we know why?
> 
> I realized there?s a problem shortly after sending out the RFR, and was starting to write
> this when I saw your reply.
> 
> I should have realized this after looking at the end computation in the line after the change.
> obj_at_addr_raw asserts the index is in range, which is exclusive on the length.  That?s
> why the end computation hand-inlines the same address calculation as done by
> obj_at_addr_raw, rather than using that function.  With my change we?ll get an assert
> if the array length is also 0, even if start is also 0.
> 
> The fix is to use the same hand-inlined address calculation for start as for end, perhaps
> packaging that up in a new function.  I?ll get back to this next week.  Sorry for the noise.

Here's a new webrev:

https://cr.openjdk.java.net/~kbarrett/8238272/open.01/

I didn't bother with an incremental webrev, since the new version is just
a one-line change replacing the previous one-line change.

Testing: tier1