From david.holmes at oracle.com  Sun Mar  1 23:13:03 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 2 Mar 2020 09:13:03 +1000
Subject: RFR[T]: 8240246: Avoid cast_to_oop from char*
In-Reply-To: <EC012E8D-E6EB-4F60-BB69-650610BCCF09@oracle.com>
References: <6AEAFE1F-8318-415C-AC4B-070D891FCE13@oracle.com>
 <3c8258e8-81ba-87a5-f3c7-18bc45fd3f71@oracle.com>
 <EC012E8D-E6EB-4F60-BB69-650610BCCF09@oracle.com>
Message-ID: <32181063-531c-c6d8-d68d-f19005da921a@oracle.com>

On 29/02/2020 6:24 pm, Kim Barrett wrote:
>> On Feb 29, 2020, at 12:11 AM, David Holmes <david.holmes at oracle.com> wrote:
>>
>> Hi Kim,
>>
>> On 29/02/2020 11:44 am, Kim Barrett wrote:
>>> Please review this trivial change of a cast_to_oop argument from char*
>>> to address.  This removes the only call of that function with a char*
>>> argument.
>>> CR:
>>> https://bugs.openjdk.java.net/browse/JDK-8240246
>>> Webrev:
>>> https://cr.openjdk.java.net/~kbarrett/8240246/open.00/
>>
>> Okay - trivial change. But any reason not to simply use uint8_t directly?
> 
> The desired type here is ?address?.
> 
> These are in globalDefinitions.hpp:
> typedef unsigned char u_char;
> typedef u_char* address;

I misinterpreted the comment in the bug about equivalence to uint8_t*.

Thanks,
David

> I could have used RemovePointer<address>::type :)
> 
>>
>> Thanks,
>> David
>>
>>> Testing:
>>> mach5 tier1
> 
> 

From aph at redhat.com  Mon Mar  2 12:00:07 2020
From: aph at redhat.com (Andrew Haley)
Date: Mon, 2 Mar 2020 12:00:07 +0000
Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features
In-Reply-To: <F350B991-892F-4201-B8F5-F4218BD058D4@oracle.com>
References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com>
 <A54AF282-2C37-40D5-825D-2BFD7A7BADD0@oracle.com>
 <2310d10b-64ec-5680-a780-71138252a259@oracle.com>
 <AA8399FB-2675-4D6D-8E36-97E984B1B070@oracle.com>
 <3e097a38-2bad-0fa2-e16a-993a4a2718e5@oracle.com>
 <F350B991-892F-4201-B8F5-F4218BD058D4@oracle.com>
Message-ID: <3923dfe9-a3e5-49ab-dd65-0cf8950b9cc4@redhat.com>

On 11/19/18 8:39 PM, Kim Barrett wrote:
> I don't see any benefit to making the first C++ code change that uses
> a new feature also incorporate the needed build system changes.
> Indeed, how does one develop that feature usage unless one has the
> build system changes already in hand?
>
> It seems to me that if the documentation says one can use some new
> language feature but the build system hasn't been updated to actually
> support doing so, then the documentation is wrong. And recall that
> this JEP includes making some new features available immediately.
> (That initial list might be modified as part of this JEP discussion.)

What is the status of this? I thought we'd decide, but I still see

   $1_CXXSTD_CXXFLAG="-std=gnu++98"

in flags-cflags.m4

I need std::make_unsigned in order to fix some undefined behaviour in
HotSpot. I could implement it myself in share/metaprogramming but if
we've agreed to allow C++14, can we please get it done now? Thx.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From kim.barrett at oracle.com  Mon Mar  2 21:51:38 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Mon, 2 Mar 2020 16:51:38 -0500
Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features
In-Reply-To: <3923dfe9-a3e5-49ab-dd65-0cf8950b9cc4@redhat.com>
References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com>
 <A54AF282-2C37-40D5-825D-2BFD7A7BADD0@oracle.com>
 <2310d10b-64ec-5680-a780-71138252a259@oracle.com>
 <AA8399FB-2675-4D6D-8E36-97E984B1B070@oracle.com>
 <3e097a38-2bad-0fa2-e16a-993a4a2718e5@oracle.com>
 <F350B991-892F-4201-B8F5-F4218BD058D4@oracle.com>
 <3923dfe9-a3e5-49ab-dd65-0cf8950b9cc4@redhat.com>
Message-ID: <7343D525-DB1F-4FB0-AF3E-0159D3C97DB6@oracle.com>

> On Mar 2, 2020, at 7:00 AM, Andrew Haley <aph at redhat.com> wrote:
> 
> On 11/19/18 8:39 PM, Kim Barrett wrote:
>> I don't see any benefit to making the first C++ code change that uses
>> a new feature also incorporate the needed build system changes.
>> Indeed, how does one develop that feature usage unless one has the
>> build system changes already in hand?
>> 
>> It seems to me that if the documentation says one can use some new
>> language feature but the build system hasn't been updated to actually
>> support doing so, then the documentation is wrong. And recall that
>> this JEP includes making some new features available immediately.
>> (That initial list might be modified as part of this JEP discussion.)
> 
> What is the status of this? I thought we'd decide, but I still see
> 
>   $1_CXXSTD_CXXFLAG="-std=gnu++98"
> 
> in flags-cflags.m4
> 
> I need std::make_unsigned in order to fix some undefined behaviour in
> HotSpot. I could implement it myself in share/metaprogramming but if
> we've agreed to allow C++14, can we please get it done now? Thx.

In light of JEP 362 "Deprecate the Solaris and SPARC Ports"
https://bugs.openjdk.java.net/browse/JDK-8231554
with the intended removal of Oracle Developer Studio (aka Solaris
Studio) as a supported compiler sometime soon, we don't want to spend
effort getting HotSpot to production quality on that platform with
C++11/14 features enabled and in use.


From volker.simonis at gmail.com  Mon Mar  2 22:46:57 2020
From: volker.simonis at gmail.com (Volker Simonis)
Date: Mon, 2 Mar 2020 23:46:57 +0100
Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features
In-Reply-To: <3923dfe9-a3e5-49ab-dd65-0cf8950b9cc4@redhat.com>
References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com>
 <A54AF282-2C37-40D5-825D-2BFD7A7BADD0@oracle.com>
 <2310d10b-64ec-5680-a780-71138252a259@oracle.com>
 <AA8399FB-2675-4D6D-8E36-97E984B1B070@oracle.com>
 <3e097a38-2bad-0fa2-e16a-993a4a2718e5@oracle.com>
 <F350B991-892F-4201-B8F5-F4218BD058D4@oracle.com>
 <3923dfe9-a3e5-49ab-dd65-0cf8950b9cc4@redhat.com>
Message-ID: <CA+3eh12hj8jvfpEo4bpWG1Yqc5_b0Q-dchuvqdy=ptMav7G78w@mail.gmail.com>

Andrew Haley <aph at redhat.com> schrieb am Mo., 2. M?rz 2020, 13:00:

> On 11/19/18 8:39 PM, Kim Barrett wrote:
> > I don't see any benefit to making the first C++ code change that uses
> > a new feature also incorporate the needed build system changes.
> > Indeed, how does one develop that feature usage unless one has the
> > build system changes already in hand?
> >
> > It seems to me that if the documentation says one can use some new
> > language feature but the build system hasn't been updated to actually
> > support doing so, then the documentation is wrong. And recall that
> > this JEP includes making some new features available immediately.
> > (That initial list might be modified as part of this JEP discussion.)
>
> What is the status of this? I thought we'd decide, but I still see
>
>    $1_CXXSTD_CXXFLAG="-std=gnu++98"
>
> in flags-cflags.m4
>
> I need std::make_unsigned in order to fix some undefined behaviour in
> HotSpot. I could implement it myself in share/metaprogramming but if
> we've agreed to allow C++14, can we please get it done now? Thx.
>

As lead of the 8 and 11 update projects you probably know best, if this fix
will eventually be considered for backporting and choose your means wisely
:)


> --
> Andrew Haley  (he/him)
> Java Platform Lead Engineer
> Red Hat UK Ltd. <https://www.redhat.com>
> https://keybase.io/andrewhaley
> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671
>
>

From aph at redhat.com  Tue Mar  3 09:19:28 2020
From: aph at redhat.com (Andrew Haley)
Date: Tue, 3 Mar 2020 09:19:28 +0000
Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features
In-Reply-To: <7343D525-DB1F-4FB0-AF3E-0159D3C97DB6@oracle.com>
References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com>
 <A54AF282-2C37-40D5-825D-2BFD7A7BADD0@oracle.com>
 <2310d10b-64ec-5680-a780-71138252a259@oracle.com>
 <AA8399FB-2675-4D6D-8E36-97E984B1B070@oracle.com>
 <3e097a38-2bad-0fa2-e16a-993a4a2718e5@oracle.com>
 <F350B991-892F-4201-B8F5-F4218BD058D4@oracle.com>
 <3923dfe9-a3e5-49ab-dd65-0cf8950b9cc4@redhat.com>
 <7343D525-DB1F-4FB0-AF3E-0159D3C97DB6@oracle.com>
Message-ID: <99f5cc63-8d3f-5678-5077-d4fa2a3e817e@redhat.com>

On 3/2/20 9:51 PM, Kim Barrett wrote:
>> On Mar 2, 2020, at 7:00 AM, Andrew Haley <aph at redhat.com> wrote:
>>
>> On 11/19/18 8:39 PM, Kim Barrett wrote:
>>> I don't see any benefit to making the first C++ code change that uses
>>> a new feature also incorporate the needed build system changes.
>>> Indeed, how does one develop that feature usage unless one has the
>>> build system changes already in hand?
>>>
>>> It seems to me that if the documentation says one can use some new
>>> language feature but the build system hasn't been updated to actually
>>> support doing so, then the documentation is wrong. And recall that
>>> this JEP includes making some new features available immediately.
>>> (That initial list might be modified as part of this JEP discussion.)
>>
>> What is the status of this? I thought we'd decide, but I still see
>>
>>   $1_CXXSTD_CXXFLAG="-std=gnu++98"
>>
>> in flags-cflags.m4
>>
>> I need std::make_unsigned in order to fix some undefined behaviour in
>> HotSpot. I could implement it myself in share/metaprogramming but if
>> we've agreed to allow C++14, can we please get it done now? Thx.
> 
> In light of JEP 362 "Deprecate the Solaris and SPARC Ports"
> https://bugs.openjdk.java.net/browse/JDK-8231554
> with the intended removal of Oracle Developer Studio (aka Solaris
> Studio) as a supported compiler sometime soon, we don't want to spend
> effort getting HotSpot to production quality on that platform with
> C++11/14 features enabled and in use.

OK. So, I guess this means that we're stuck with C++98 (!) for a while.

It's not too hard for me to write the support for make_unsigned and add
it to share/metaprogramming. I'll do that and see what people say.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From aph at redhat.com  Tue Mar  3 09:26:44 2020
From: aph at redhat.com (Andrew Haley)
Date: Tue, 3 Mar 2020 09:26:44 +0000
Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features
In-Reply-To: <CA+3eh12hj8jvfpEo4bpWG1Yqc5_b0Q-dchuvqdy=ptMav7G78w@mail.gmail.com>
References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com>
 <A54AF282-2C37-40D5-825D-2BFD7A7BADD0@oracle.com>
 <2310d10b-64ec-5680-a780-71138252a259@oracle.com>
 <AA8399FB-2675-4D6D-8E36-97E984B1B070@oracle.com>
 <3e097a38-2bad-0fa2-e16a-993a4a2718e5@oracle.com>
 <F350B991-892F-4201-B8F5-F4218BD058D4@oracle.com>
 <3923dfe9-a3e5-49ab-dd65-0cf8950b9cc4@redhat.com>
 <CA+3eh12hj8jvfpEo4bpWG1Yqc5_b0Q-dchuvqdy=ptMav7G78w@mail.gmail.com>
Message-ID: <4081af4f-372d-911e-840b-de97847cba3a@redhat.com>

On 3/2/20 10:46 PM, Volker Simonis wrote:

> As lead of the 8 and 11 update projects you probably know best, if this fix
> will eventually be considered for backporting and choose your means wisely
> :)

Yeah, I know. Srsly. :-)

But one of the few things of which I am certain is that we must not
allow the needs of backporting to determine the future of Java. That's
the way of staying in the past.

As Patrick Head put it, ?Some people tell me that Formula 1 would be
better if the drivers still used stick shifts, but that?s a bit like
saying, 'isn?t it a pity we don?t still walk around in clogs!'?

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From volker.simonis at gmail.com  Tue Mar  3 10:22:25 2020
From: volker.simonis at gmail.com (Volker Simonis)
Date: Tue, 3 Mar 2020 11:22:25 +0100
Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features
In-Reply-To: <4081af4f-372d-911e-840b-de97847cba3a@redhat.com>
References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com>
 <A54AF282-2C37-40D5-825D-2BFD7A7BADD0@oracle.com>
 <2310d10b-64ec-5680-a780-71138252a259@oracle.com>
 <AA8399FB-2675-4D6D-8E36-97E984B1B070@oracle.com>
 <3e097a38-2bad-0fa2-e16a-993a4a2718e5@oracle.com>
 <F350B991-892F-4201-B8F5-F4218BD058D4@oracle.com>
 <3923dfe9-a3e5-49ab-dd65-0cf8950b9cc4@redhat.com>
 <CA+3eh12hj8jvfpEo4bpWG1Yqc5_b0Q-dchuvqdy=ptMav7G78w@mail.gmail.com>
 <4081af4f-372d-911e-840b-de97847cba3a@redhat.com>
Message-ID: <CA+3eh1044Ou3g6KxTELnKvADCfA7Q357zpJtzxSgNitq9JYNPQ@mail.gmail.com>

On Tue, Mar 3, 2020 at 10:26 AM Andrew Haley <aph at redhat.com> wrote:
>
> On 3/2/20 10:46 PM, Volker Simonis wrote:
>
> > As lead of the 8 and 11 update projects you probably know best, if this fix
> > will eventually be considered for backporting and choose your means wisely
> > :)
>
> Yeah, I know. Srsly. :-)
>
> But one of the few things of which I am certain is that we must not
> allow the needs of backporting to determine the future of Java. That's
> the way of staying in the past.
>
> As Patrick Head put it, ?Some people tell me that Formula 1 would be
> better if the drivers still used stick shifts, but that?s a bit like
> saying, 'isn?t it a pity we don?t still walk around in clogs!'?
>

Totally agree.

> --
> Andrew Haley  (he/him)
> Java Platform Lead Engineer
> Red Hat UK Ltd. <https://www.redhat.com>
> https://keybase.io/andrewhaley
> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671
>

From glaubitz at physik.fu-berlin.de  Tue Mar  3 22:18:57 2020
From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz)
Date: Tue, 3 Mar 2020 23:18:57 +0100
Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features
In-Reply-To: <4081af4f-372d-911e-840b-de97847cba3a@redhat.com>
References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com>
 <A54AF282-2C37-40D5-825D-2BFD7A7BADD0@oracle.com>
 <2310d10b-64ec-5680-a780-71138252a259@oracle.com>
 <AA8399FB-2675-4D6D-8E36-97E984B1B070@oracle.com>
 <3e097a38-2bad-0fa2-e16a-993a4a2718e5@oracle.com>
 <F350B991-892F-4201-B8F5-F4218BD058D4@oracle.com>
 <3923dfe9-a3e5-49ab-dd65-0cf8950b9cc4@redhat.com>
 <CA+3eh12hj8jvfpEo4bpWG1Yqc5_b0Q-dchuvqdy=ptMav7G78w@mail.gmail.com>
 <4081af4f-372d-911e-840b-de97847cba3a@redhat.com>
Message-ID: <179814ed-649b-93c4-1894-a0f16423d3d3@physik.fu-berlin.de>

On 3/3/20 10:26 AM, Andrew Haley wrote:
> But one of the few things of which I am certain is that we must not
> allow the needs of backporting to determine the future of Java. That's
> the way of staying in the past.

Unpopular opinion: It's the enterprise customers that mainly pay for the
development of software, not the users of rolling release distributions.

I know that maintaining old stuff is boring but that's where the money
is made. Too many developers unfortunately seem to forget that.

> As Patrick Head put it, ?Some people tell me that Formula 1 would be
> better if the drivers still used stick shifts, but that?s a bit like
> saying, 'isn?t it a pity we don?t still walk around in clogs!'?

Maintenance of stable software isn't done for nostalgic reasons, it's
because enterprise customers need a reliable and stable platform to
run their businesses on. A lot of businesses are still running on JDK-8
for this reason.

Running bleeding edge software might be an option for a single desktop
user, but not in an enterprise environment with thousands of users
or in a critical environment like the booking system of an airline.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz at debian.org
`. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

From kim.barrett at oracle.com  Wed Mar  4 00:29:49 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Tue, 3 Mar 2020 19:29:49 -0500
Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features
In-Reply-To: <f1542bce-3133-20f0-d116-88133144a9f3@oracle.com>
References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com>
 <A54AF282-2C37-40D5-825D-2BFD7A7BADD0@oracle.com>
 <2310d10b-64ec-5680-a780-71138252a259@oracle.com>
 <AA8399FB-2675-4D6D-8E36-97E984B1B070@oracle.com>
 <3e097a38-2bad-0fa2-e16a-993a4a2718e5@oracle.com>
 <F350B991-892F-4201-B8F5-F4218BD058D4@oracle.com>
 <3923dfe9-a3e5-49ab-dd65-0cf8950b9cc4@redhat.com>
 <CA+3eh12hj8jvfpEo4bpWG1Yqc5_b0Q-dchuvqdy=ptMav7G78w@mail.gmail.com>
 <4081af4f-372d-911e-840b-de97847cba3a@redhat.com>
 <CA+3eh1044Ou3g6KxTELnKvADCfA7Q357zpJtzxSgNitq9JYNPQ@mail.gmail.com>
 <f1542bce-3133-20f0-d116-88133144a9f3@oracle.com>
Message-ID: <81CDF382-1442-4721-AEDD-4F3A051AF192@oracle.com>

> On Mar 3, 2020, at 4:49 PM, Alexey Semenyuk <alexey.semenyuk at oracle.com> wrote:
> 
> How about C++11? I have a pending patch for jpackage that depends on C++11 features that I hesitate to pull in jdk15.

The reasons for HotSpot (at least) not already being on C++14 (cost of
switching over the Solaris Studio based platform) also apply to C++11.
A big part of the work involves dealing with changes to the compilation
model, the runtime libraries, and the ABI, all of which are part of
Solaris Studio's transition from C++98 to C++11.


From aph at redhat.com  Wed Mar  4 11:33:19 2020
From: aph at redhat.com (Andrew Haley)
Date: Wed, 4 Mar 2020 11:33:19 +0000
Subject: RFC: JEP JDK-8208089: Implement C++14 Language Features
In-Reply-To: <179814ed-649b-93c4-1894-a0f16423d3d3@physik.fu-berlin.de>
References: <7D0FB905-E515-4FD5-941D-5973BB04D0AA@oracle.com>
 <A54AF282-2C37-40D5-825D-2BFD7A7BADD0@oracle.com>
 <2310d10b-64ec-5680-a780-71138252a259@oracle.com>
 <AA8399FB-2675-4D6D-8E36-97E984B1B070@oracle.com>
 <3e097a38-2bad-0fa2-e16a-993a4a2718e5@oracle.com>
 <F350B991-892F-4201-B8F5-F4218BD058D4@oracle.com>
 <3923dfe9-a3e5-49ab-dd65-0cf8950b9cc4@redhat.com>
 <CA+3eh12hj8jvfpEo4bpWG1Yqc5_b0Q-dchuvqdy=ptMav7G78w@mail.gmail.com>
 <4081af4f-372d-911e-840b-de97847cba3a@redhat.com>
 <179814ed-649b-93c4-1894-a0f16423d3d3@physik.fu-berlin.de>
Message-ID: <f104aa81-bf99-7430-c17e-3ff8a42932b9@redhat.com>

On 3/3/20 10:18 PM, John Paul Adrian Glaubitz wrote:
> On 3/3/20 10:26 AM, Andrew Haley wrote:
>> But one of the few things of which I am certain is that we must not
>> allow the needs of backporting to determine the future of Java. That's
>> the way of staying in the past.
>
> Unpopular opinion: It's the enterprise customers that mainly pay for the
> development of software, not the users of rolling release distributions.

I don't think that's an unpopular opinion at all! It's why most of us
who work on Java updates get paid.

Continuing to depend on C++98 is not good for us in the long term. It
means we're using an obsolete programming language, and that
inevitably carries its own risks, and we miss opportunities to do
things better.

> I know that maintaining old stuff is boring but that's where the money
> is made. Too many developers unfortunately seem to forget that.

Seriously now, who are you talking about? We spend a tremendous amount
of time and money maintaining old stuff.

>> As Patrick Head put it, ?Some people tell me that Formula 1 would be
>> better if the drivers still used stick shifts, but that?s a bit like
>> saying, 'isn?t it a pity we don?t still walk around in clogs!'?
>
> Maintenance of stable software isn't done for nostalgic reasons, it's
> because enterprise customers need a reliable and stable platform to
> run their businesses on. A lot of businesses are still running on JDK-8
> for this reason.

Yes, that's why we do it.

> Running bleeding edge software might be an option for a single desktop
> user, but not in an enterprise environment with thousands of users
> or in a critical environment like the booking system of an airline.

Bear in mind that, if people hadn't pressed forward after JDK 7, there
wouldn't be a "legacy" JDK 8 to maintain.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From stefan.karlsson at oracle.com  Wed Mar  4 17:12:21 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Wed, 4 Mar 2020 18:12:21 +0100
Subject: RFR: 8240532: heap inspection prints trailing @ after name of module
 without version
Message-ID: <40f5d677-2308-3826-2b44-713af87c8f58@oracle.com>

Hi all,

Please review this patch to remove a trailing @ after module names of 
modules without a version.

https://cr.openjdk.java.net/~stefank/8240532/webrev.01/
https://bugs.openjdk.java.net/browse/JDK-8240532

The heap inspection functionality generates and prints a class histogram.

Full image builds show module name and version as 'name at version':
[3,640s][trace][gc,classhisto????? ] GC(1)??? 4: 5067???????? 162144? 
java.lang.String (java.base at 15-internal)

Exploded builds don't have a version and prints 'name@'
[4,361s][trace][gc,classhisto????? ] GC(1)??? 3: 7144???????? 228608? 
java.lang.String (java.base@)

This looks weird and I propose that we drop the trailing @ when there's 
no version string:
[4,361s][trace][gc,classhisto????? ] GC(1)??? 3: 7144???????? 228608? 
java.lang.String (java.base)

This affects the output of:
* -Xlog:classhisto*=trace
* jcmd <pid> GC.class_histogram
* inspectheap attach operation
* jmap -histo

I've run the tests I could find that were using these operations. Other 
than that I intend to run tier1-3.

Do others agree with this change?

Thanks,
StefanK

From lois.foltan at oracle.com  Wed Mar  4 17:34:18 2020
From: lois.foltan at oracle.com (Lois Foltan)
Date: Wed, 4 Mar 2020 12:34:18 -0500
Subject: RFR: 8240532: heap inspection prints trailing @ after name of
 module without version
In-Reply-To: <40f5d677-2308-3826-2b44-713af87c8f58@oracle.com>
References: <40f5d677-2308-3826-2b44-713af87c8f58@oracle.com>
Message-ID: <47a1919a-597f-d42a-7377-bc9c65b1c0e2@oracle.com>

On 3/4/2020 12:12 PM, Stefan Karlsson wrote:
> Hi all,
>
> Please review this patch to remove a trailing @ after module names of 
> modules without a version.
>
> https://cr.openjdk.java.net/~stefank/8240532/webrev.01/
> https://bugs.openjdk.java.net/browse/JDK-8240532
>
> The heap inspection functionality generates and prints a class histogram.
>
> Full image builds show module name and version as 'name at version':
> [3,640s][trace][gc,classhisto????? ] GC(1)??? 4: 5067 162144? 
> java.lang.String (java.base at 15-internal)
>
> Exploded builds don't have a version and prints 'name@'
> [4,361s][trace][gc,classhisto????? ] GC(1)??? 3: 7144 228608? 
> java.lang.String (java.base@)
>
> This looks weird and I propose that we drop the trailing @ when 
> there's no version string:
> [4,361s][trace][gc,classhisto????? ] GC(1)??? 3: 7144 228608? 
> java.lang.String (java.base)
>
> This affects the output of:
> * -Xlog:classhisto*=trace
> * jcmd <pid> GC.class_histogram
> * inspectheap attach operation
> * jmap -histo
>
> I've run the tests I could find that were using these operations. 
> Other than that I intend to run tier1-3.
>
> Do others agree with this change?

Yes, I think this change looks good and is consistent with how it is 
currently handled in javaClasses.cpp/print_stack_element_to_stream() and 
frame.cpp/frame::print_on_error() for example.? However, all of these 
places should really be calling ModuleEntry::should_show_version() 
instead of directly checking if the version() is NULL.? But that could 
be addressed in a differing RFE.

Thanks,
Lois

>
> Thanks,
> StefanK


From sgehwolf at redhat.com  Wed Mar  4 18:59:02 2020
From: sgehwolf at redhat.com (Severin Gehwolf)
Date: Wed, 04 Mar 2020 19:59:02 +0100
Subject: RFR(S): 8239785: Cgroups: Incorrect detection logic on old
 systems in hotspot
In-Reply-To: <EF45A2BB-360E-4767-B542-B420BF76FD2E@oracle.com>
References: <362678e8bc8231c205c6b3327d08dd5179154e23.camel@redhat.com>
 <2A934185-9B81-4371-B985-F0C3C39B3FD3@oracle.com>
 <215e52601b2da8600cbfd47ef73634720a7736d2.camel@redhat.com>
 <EF45A2BB-360E-4767-B542-B420BF76FD2E@oracle.com>
Message-ID: <d1683592b67f99e7625713d2db2c71641d50a738.camel@redhat.com>

Hi Bob,

Updated webrev:
http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/

Testing: hotspot docker tests on Linux fastdebug/release cgroup v1 and
cgroup v2. New regression test. jdk submit. All pass.

On Fri, 2020-02-28 at 15:08 -0500, Bob Vandette wrote:
> It still looks like the _mount_path can be corrupted if both cgroupv1 and cgroupv2 are mounted.
> You?ll only initialize the _mount_path if cgroupv2 is detected in the cgroups file but if cgroupv1
> is also mounted, you?ll still clobber the _mount_path.

How so? When we start processing /proc/self/mountinfo on line 170, then
we have already looked at /proc/cgroups, thus is_cgroupsV2 is set to
true (A) or false (B).

Cases A:
 1. We are on an old system with *no* cgroup controllers (v1 or v2)
    mounted (this bug). We are fine to continue as no cgroups entry
    will be found in /proc/self/mountinfo. Thus, no clobbering.
 2. We are on a true unified hierarchy only system. In that case
    the sscanf for v1 would match (early match), but 'memory' et.al
    controllers would not be found. Thus, no clobbering.

Cases B:
 1. Hybrid hierarchy: controllers mounted via cgroup v1, but
    cgroup v2 mounted in addition for system use. is_cgroupV2 will
    prevent the cgroup v2 controller path lookup logic to be entered.
    No clobbering.
 2. cgroup v1 only hierarchy (legacy): Same logic applies. V2 setting
    of mount paths won't be entered since is_cgroupV2 == false.
    No clobbering.

What am I missing? FWIW, I've played around with the idea of keeping
cgroup v1 and cgroup v2 mount paths separate[1]. If that's preferred,
happy to go this route.

While thinking about this issue here, I've decided to actually write a
test (using WhiteBox) so that all these cases are covered and will
continue to work. Running the test, CgroupSubsystemFactory, with
fastdebug proves that there is no clobbering if that matters. It's also
a way to reproduce the issue. So we have a regression test now too
(testCgroupv1NoMounts). Line 325-332 in [2] is the actual fix. Without
it, the test fails.

> Maybe you should just do the cgroupv2 processing if /proc/cgroups has
> cgroupv2 enabled and if that fails continue looking for cgroup v1
> mounts.

FWIW, I've amended the cgroup v1 logic to also check for the file
system type. Which needs to be 'cgroup' (not 'cgroup2') for cgroup v1.
This isn't strictly needed but more accurately reflects the intent IMO.

> 
> Do you really need to do this?  The BSS section should be zeroed.
> 
> 55   for (int i = 0; i < CG_INFO_LENGTH; i++) {
>   56     cg_infos[i]._name = NULL;
>   57     cg_infos[i]._cgroup_path = NULL;
>   58     cg_infos[i]._root_mount_path = NULL;
>   59     cg_infos[i]._mount_path = NULL;
>   60   }

I believe so. CgroupInfo array is stack allocated and we should
initialize values properly. On the other hand, we should do that in the
constructor. Done now.

> You don?t need to check for NULL in cleanup, os::free can handle NULL
> pointers.
> 
>  339 void CgroupSubsystemFactory::cleanup(CgroupInfo* cg_infos) {
>  340   assert(cg_infos != NULL, "Invariant");
>  341   for (int i = 0; i < CG_INFO_LENGTH; i++) {
>  342     if (cg_infos[i]._name != NULL) {
>  343       os::free(cg_infos[i]._name);
>  344     }
>  345     if (cg_infos[i]._cgroup_path != NULL) {
>  346       os::free(cg_infos[i]._cgroup_path);
>  347     }
>  348     if (cg_infos[i]._root_mount_path != NULL) {
>  349       os::free(cg_infos[i]._root_mount_path);
>  350     }
>  351     if (cg_infos[i]._mount_path != NULL) {
>  352       os::free(cg_infos[i]._mount_path);
>  353     }
>  354   }
>  355 }

Fixed.

Let me know what you think about the updated webrev.

Thanks,
Severin

[1] http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/02/webrev/
[2] http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/src/hotspot/os/linux/cgroupSubsystem_linux.cpp.html

> Bob.
> 
> 
> 
> > On Feb 28, 2020, at 11:42 AM, Severin Gehwolf <sgehwolf at redhat.com>
> > wrote:
> > 
> > Hi Bob,
> > 
> > Thanks for the review!
> > 
> > On Tue, 2020-02-25 at 15:08 -0500, Bob Vandette wrote:
> > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/01/webrev/src/hotspot/os/linux/cgroupSubsystem_linux.cpp.sdiff.html
> > > 
> > > If both cgroupv1 and v2 controllers are mounted, you might have
> > > some problems.  You might want to only look for
> > > cgroup2 mounts if is_cgroupV2 is true otherwise only look for
> > > cgroup mounts.
> > 
> > OK.
> > 
> > > 194           cg_infos[i]._mount_path =
> > > os::strdup(tmp_mount_point);
> > > This will stomp the pointer:
> > > 213           cg_infos[memory_idx]._mount_path =
> > > os::strdup(tmpmount);
> > 
> > You are right. If we are on a hybrid hierarchy and for some reason
> > the
> > cgroup v2 mount shows up *after* cgroup mounts in
> > /proc/self/mountinfo,
> > we'd get the mount path wrong. So ordering matters! The reason this
> > worked for me on hybrid is that cgroup2 mounts come *before* cgroup
> > mounts on the system I've tested.
> > 
> > > You should free all strdup?d pointers in cg_infos in places where
> > > you fail and return NULL.
> > > 
> > > 81       cg_infos[memory_idx]._name = os::strdup(name);
> > > 213           cg_infos[memory_idx]._mount_path =
> > > os::strdup(tmpmount);
> > > 214           cg_infos[memory_idx]._root_mount_path =
> > > os::strdup(tmproot);
> > 
> > Done now. More thoughts?
> > 
> > Updated webrev: 
> > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/03/webrev/
> > 
> > Thanks,
> > Severin
> > 
> > > > On Feb 25, 2020, at 2:22 PM, Severin Gehwolf <
> > > > sgehwolf at redhat.com> wrote:
> > > > 
> > > > Hi,
> > > > 
> > > > Could I please get reviews of this cgroup-related patch? After
> > > > JDK-
> > > > 8230305 old systems with no mounted cgroup controllers would
> > > > get
> > > > detected as cgroups v2 (wrong). Then, when initializing the v2
> > > > subsystem, it would fail. The trace about cgroupv2 mount point
> > > > not
> > > > found is misleading in this case. While the outcome is similar
> > > > pre/post
> > > > patch (NULL cgroup subsystem), I'd like to be explicit about
> > > > this case.
> > > > 
> > > > The suggested fix is to look at /proc/self/mountinfo in order
> > > > to
> > > > correctly detect whether this is cgroup v2 or cgroup v1 with no
> > > > mounted
> > > > controllers. In the latter case we just stop initialization as
> > > > we'd
> > > > fail later in cgroupv1 code anyway. This aligns hotspot code
> > > > with core-
> > > > libs after JDK-8239559.
> > > > 
> > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8239785
> > > > webrev: 
> > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/01/webrev/
> > > > 
> > > > Testing: jdk-submit, hotspot docker tests on cgroup v1 and
> > > > cgroup v2. All pass.
> > > > 
> > > > Thoughts?
> > > > 
> > > > Thanks,
> > > > Severin
> > > > 
> 
> 





From bob.vandette at oracle.com  Wed Mar  4 20:39:30 2020
From: bob.vandette at oracle.com (Bob Vandette)
Date: Wed, 4 Mar 2020 15:39:30 -0500
Subject: RFR(S): 8239785: Cgroups: Incorrect detection logic on old
 systems in hotspot
In-Reply-To: <d1683592b67f99e7625713d2db2c71641d50a738.camel@redhat.com>
References: <362678e8bc8231c205c6b3327d08dd5179154e23.camel@redhat.com>
 <2A934185-9B81-4371-B985-F0C3C39B3FD3@oracle.com>
 <215e52601b2da8600cbfd47ef73634720a7736d2.camel@redhat.com>
 <EF45A2BB-360E-4767-B542-B420BF76FD2E@oracle.com>
 <d1683592b67f99e7625713d2db2c71641d50a738.camel@redhat.com>
Message-ID: <1B61ACE7-AFB9-458C-B6AB-465259BD31C7@oracle.com>


> On Mar 4, 2020, at 1:59 PM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
> 
> Hi Bob,
> 
> Updated webrev:
> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/
> 
> Testing: hotspot docker tests on Linux fastdebug/release cgroup v1 and
> cgroup v2. New regression test. jdk submit. All pass.
> 
> On Fri, 2020-02-28 at 15:08 -0500, Bob Vandette wrote:
>> It still looks like the _mount_path can be corrupted if both cgroupv1 and cgroupv2 are mounted.
>> You?ll only initialize the _mount_path if cgroupv2 is detected in the cgroups file but if cgroupv1
>> is also mounted, you?ll still clobber the _mount_path.
> 
> How so? When we start processing /proc/self/mountinfo on line 170, then
> we have already looked at /proc/cgroups, thus is_cgroupsV2 is set to
> true (A) or false (B).
> 
> Cases A:
> 1. We are on an old system with *no* cgroup controllers (v1 or v2)
>    mounted (this bug). We are fine to continue as no cgroups entry
>    will be found in /proc/self/mountinfo. Thus, no clobbering.
> 2. We are on a true unified hierarchy only system. In that case
>    the sscanf for v1 would match (early match), but 'memory' et.al
>    controllers would not be found. Thus, no clobbering.
> 
> Cases B:
> 1. Hybrid hierarchy: controllers mounted via cgroup v1, but
>    cgroup v2 mounted in addition for system use. is_cgroupV2 will
>    prevent the cgroup v2 controller path lookup logic to be entered.
>    No clobbering.
> 2. cgroup v1 only hierarchy (legacy): Same logic applies. V2 setting
>    of mount paths won't be entered since is_cgroupV2 == false.
>    No clobbering.
> 
> What am I missing? FWIW, I've played around with the idea of keeping
> cgroup v1 and cgroup v2 mount paths separate[1]. If that's preferred,
> happy to go this route.

The case that I think is a problem is ?

cgroupv2 is detected in /proc/cgroups so is_cgroupv2 is true
both cgroupv1 and cgroupv2 devices are mounted

At line cgroupSubsystem_linux.cpp:292 in your new webrev, you process the cgroupv1
devices and clobber _mount_path.

What keeps this from occurring?

> 
> While thinking about this issue here, I've decided to actually write a
> test (using WhiteBox) so that all these cases are covered and will
> continue to work. Running the test, CgroupSubsystemFactory, with
> fastdebug proves that there is no clobbering if that matters. It's also
> a way to reproduce the issue. So we have a regression test now too
> (testCgroupv1NoMounts). Line 325-332 in [2] is the actual fix. Without
> it, the test fails.
> 
>> Maybe you should just do the cgroupv2 processing if /proc/cgroups has
>> cgroupv2 enabled and if that fails continue looking for cgroup v1
>> mounts.
> 
> FWIW, I've amended the cgroup v1 logic to also check for the file
> system type. Which needs to be 'cgroup' (not 'cgroup2') for cgroup v1.
> This isn't strictly needed but more accurately reflects the intent IMO.

Sounds good.

> 
>> 
>> Do you really need to do this?  The BSS section should be zeroed.
>> 
>> 55   for (int i = 0; i < CG_INFO_LENGTH; i++) {
>>  56     cg_infos[i]._name = NULL;
>>  57     cg_infos[i]._cgroup_path = NULL;
>>  58     cg_infos[i]._root_mount_path = NULL;
>>  59     cg_infos[i]._mount_path = NULL;
>>  60   }
> 
> I believe so. CgroupInfo array is stack allocated and we should
> initialize values properly. On the other hand, we should do that in the
> constructor. Done now.

You are correct.  I was only looking at the diffs and not the full source.

> 
>> You don?t need to check for NULL in cleanup, os::free can handle NULL
>> pointers.
>> 
>> 339 void CgroupSubsystemFactory::cleanup(CgroupInfo* cg_infos) {
>> 340   assert(cg_infos != NULL, "Invariant");
>> 341   for (int i = 0; i < CG_INFO_LENGTH; i++) {
>> 342     if (cg_infos[i]._name != NULL) {
>> 343       os::free(cg_infos[i]._name);
>> 344     }
>> 345     if (cg_infos[i]._cgroup_path != NULL) {
>> 346       os::free(cg_infos[i]._cgroup_path);
>> 347     }
>> 348     if (cg_infos[i]._root_mount_path != NULL) {
>> 349       os::free(cg_infos[i]._root_mount_path);
>> 350     }
>> 351     if (cg_infos[i]._mount_path != NULL) {
>> 352       os::free(cg_infos[i]._mount_path);
>> 353     }
>> 354   }
>> 355 }
> 
> Fixed.
> 
> Let me know what you think about the updated webrev.
> 
> Thanks,
> Severin
> 
> [1] http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/02/webrev/
> [2] http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/src/hotspot/os/linux/cgroupSubsystem_linux.cpp.html
> 
>> Bob.
>> 
>> 
>> 
>>> On Feb 28, 2020, at 11:42 AM, Severin Gehwolf <sgehwolf at redhat.com>
>>> wrote:
>>> 
>>> Hi Bob,
>>> 
>>> Thanks for the review!
>>> 
>>> On Tue, 2020-02-25 at 15:08 -0500, Bob Vandette wrote:
>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/01/webrev/src/hotspot/os/linux/cgroupSubsystem_linux.cpp.sdiff.html
>>>> 
>>>> If both cgroupv1 and v2 controllers are mounted, you might have
>>>> some problems.  You might want to only look for
>>>> cgroup2 mounts if is_cgroupV2 is true otherwise only look for
>>>> cgroup mounts.
>>> 
>>> OK.
>>> 
>>>> 194           cg_infos[i]._mount_path =
>>>> os::strdup(tmp_mount_point);
>>>> This will stomp the pointer:
>>>> 213           cg_infos[memory_idx]._mount_path =
>>>> os::strdup(tmpmount);
>>> 
>>> You are right. If we are on a hybrid hierarchy and for some reason
>>> the
>>> cgroup v2 mount shows up *after* cgroup mounts in
>>> /proc/self/mountinfo,
>>> we'd get the mount path wrong. So ordering matters! The reason this
>>> worked for me on hybrid is that cgroup2 mounts come *before* cgroup
>>> mounts on the system I've tested.
>>> 
>>>> You should free all strdup?d pointers in cg_infos in places where
>>>> you fail and return NULL.
>>>> 
>>>> 81       cg_infos[memory_idx]._name = os::strdup(name);
>>>> 213           cg_infos[memory_idx]._mount_path =
>>>> os::strdup(tmpmount);
>>>> 214           cg_infos[memory_idx]._root_mount_path =
>>>> os::strdup(tmproot);
>>> 
>>> Done now. More thoughts?
>>> 
>>> Updated webrev: 
>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/03/webrev/
>>> 
>>> Thanks,
>>> Severin
>>> 
>>>>> On Feb 25, 2020, at 2:22 PM, Severin Gehwolf <
>>>>> sgehwolf at redhat.com> wrote:
>>>>> 
>>>>> Hi,
>>>>> 
>>>>> Could I please get reviews of this cgroup-related patch? After
>>>>> JDK-
>>>>> 8230305 old systems with no mounted cgroup controllers would
>>>>> get
>>>>> detected as cgroups v2 (wrong). Then, when initializing the v2
>>>>> subsystem, it would fail. The trace about cgroupv2 mount point
>>>>> not
>>>>> found is misleading in this case. While the outcome is similar
>>>>> pre/post
>>>>> patch (NULL cgroup subsystem), I'd like to be explicit about
>>>>> this case.
>>>>> 
>>>>> The suggested fix is to look at /proc/self/mountinfo in order
>>>>> to
>>>>> correctly detect whether this is cgroup v2 or cgroup v1 with no
>>>>> mounted
>>>>> controllers. In the latter case we just stop initialization as
>>>>> we'd
>>>>> fail later in cgroupv1 code anyway. This aligns hotspot code
>>>>> with core-
>>>>> libs after JDK-8239559.
>>>>> 
>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8239785
>>>>> webrev: 
>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/01/webrev/
>>>>> 
>>>>> Testing: jdk-submit, hotspot docker tests on cgroup v1 and
>>>>> cgroup v2. All pass.
>>>>> 
>>>>> Thoughts?
>>>>> 
>>>>> Thanks,
>>>>> Severin
>>>>> 
>> 
>> 
> 
> 
> 
> 


From sgehwolf at redhat.com  Thu Mar  5 09:36:55 2020
From: sgehwolf at redhat.com (Severin Gehwolf)
Date: Thu, 05 Mar 2020 10:36:55 +0100
Subject: RFR(S): 8239785: Cgroups: Incorrect detection logic on old
 systems in hotspot
In-Reply-To: <1B61ACE7-AFB9-458C-B6AB-465259BD31C7@oracle.com>
References: <362678e8bc8231c205c6b3327d08dd5179154e23.camel@redhat.com>
 <2A934185-9B81-4371-B985-F0C3C39B3FD3@oracle.com>
 <215e52601b2da8600cbfd47ef73634720a7736d2.camel@redhat.com>
 <EF45A2BB-360E-4767-B542-B420BF76FD2E@oracle.com>
 <d1683592b67f99e7625713d2db2c71641d50a738.camel@redhat.com>
 <1B61ACE7-AFB9-458C-B6AB-465259BD31C7@oracle.com>
Message-ID: <264b4edf94d60a3256a9ff8d5de6a078c3a0819b.camel@redhat.com>

Hi Bob,

On Wed, 2020-03-04 at 15:39 -0500, Bob Vandette wrote:
> > On Mar 4, 2020, at 1:59 PM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
> > 
> > Hi Bob,
> > 
> > Updated webrev:
> > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/
> > 
> > Testing: hotspot docker tests on Linux fastdebug/release cgroup v1 and
> > cgroup v2. New regression test. jdk submit. All pass.
> > 
> > On Fri, 2020-02-28 at 15:08 -0500, Bob Vandette wrote:
> > > It still looks like the _mount_path can be corrupted if both cgroupv1 and cgroupv2 are mounted.
> > > You?ll only initialize the _mount_path if cgroupv2 is detected in the cgroups file but if cgroupv1
> > > is also mounted, you?ll still clobber the _mount_path.
> > 
> > How so? When we start processing /proc/self/mountinfo on line 170, then
> > we have already looked at /proc/cgroups, thus is_cgroupsV2 is set to
> > true (A) or false (B).
> > 
> > Cases A:
> > 1. We are on an old system with *no* cgroup controllers (v1 or v2)
> >    mounted (this bug). We are fine to continue as no cgroups entry
> >    will be found in /proc/self/mountinfo. Thus, no clobbering.
> > 2. We are on a true unified hierarchy only system. In that case
> >    the sscanf for v1 would match (early match), but 'memory' et.al
> >    controllers would not be found. Thus, no clobbering.
> > 
> > Cases B:
> > 1. Hybrid hierarchy: controllers mounted via cgroup v1, but
> >    cgroup v2 mounted in addition for system use. is_cgroupV2 will
> >    prevent the cgroup v2 controller path lookup logic to be entered.
> >    No clobbering.
> > 2. cgroup v1 only hierarchy (legacy): Same logic applies. V2 setting
> >    of mount paths won't be entered since is_cgroupV2 == false.
> >    No clobbering.
> > 
> > What am I missing? FWIW, I've played around with the idea of keeping
> > cgroup v1 and cgroup v2 mount paths separate[1]. If that's preferred,
> > happy to go this route.
> 
> The case that I think is a problem is ?
> 
> cgroupv2 is detected in /proc/cgroups so is_cgroupv2 is true
> both cgroupv1 and cgroupv2 devices are mounted

That's not possible. is_cgroupv2 cannot be true in that case.

> At line cgroupSubsystem_linux.cpp:292 in your new webrev, you process the cgroupv1
> devices and clobber _mount_path.
> 
> What keeps this from occurring?

In that case 'is_cgroupv2 == false', because having any cgroups v1
controllers mounted will result in some non-zero hierarchy ids in
/proc/cgroups. That in turn will set is_cgroup2 == false.

>From man 7 cgroups:

""""
   /proc files
       /proc/cgroups (since Linux 2.6.24)

              The fields in this file are, from left to right:

              1. The name of the controller.

              2. The unique ID of the cgroup hierarchy on which this controller is mounted.  If multiple cgroups v1 controllers are bound to the same hierarchy, then each will show the same hierarchy ID in
                 this field.  The value in this field will be 0 if:

                   a) the controller is not mounted on a cgroups v1 hierarchy;

                   b) the controller is bound to the cgroups v2 single unified hierarchy; or

                   c) the controller is disabled (see below).
"""

The only way hierarchy id == 0 for a controller on cgroups v1 is a).
It's never 0 when cgroup v1 and cgroup v2 are *both* mounted (hybrid).
One example of such a /proc/cgroups is on my F30 workstation (hybrid):

$ cat /proc/cgroups 
#subsys_name	hierarchy	num_cgroups	enabled
cpuset	5	1	1
cpu	3	1	1
cpuacct	3	1	1
blkio	11	1	1
memory	6	136	1
devices	9	75	1
freezer	7	1	1
net_cls	4	1	1
perf_event	2	1	1
net_prio	4	1	1
hugetlb	10	1	1
pids	8	81	1

And the corresponding mountinfo:

$ grep cgroup /proc/self/mountinfo 
30 23 0:26 / /sys/fs/cgroup ro,nosuid,nodev,noexec shared:4 - tmpfs tmpfs ro,seclabel,mode=755
31 30 0:27 / /sys/fs/cgroup/unified rw,nosuid,nodev,noexec,relatime shared:5 - cgroup2 cgroup2 rw,seclabel,nsdelegate
32 30 0:28 / /sys/fs/cgroup/systemd rw,nosuid,nodev,noexec,relatime shared:6 - cgroup cgroup rw,seclabel,xattr,name=systemd
35 30 0:31 / /sys/fs/cgroup/perf_event rw,nosuid,nodev,noexec,relatime shared:7 - cgroup cgroup rw,seclabel,perf_event
36 30 0:32 / /sys/fs/cgroup/cpu,cpuacct rw,nosuid,nodev,noexec,relatime shared:8 - cgroup cgroup rw,seclabel,cpu,cpuacct
37 30 0:33 / /sys/fs/cgroup/net_cls,net_prio rw,nosuid,nodev,noexec,relatime shared:9 - cgroup cgroup rw,seclabel,net_cls,net_prio
38 30 0:34 / /sys/fs/cgroup/cpuset rw,nosuid,nodev,noexec,relatime shared:10 - cgroup cgroup rw,seclabel,cpuset
39 30 0:35 / /sys/fs/cgroup/memory rw,nosuid,nodev,noexec,relatime shared:11 - cgroup cgroup rw,seclabel,memory
40 30 0:36 / /sys/fs/cgroup/freezer rw,nosuid,nodev,noexec,relatime shared:12 - cgroup cgroup rw,seclabel,freezer
41 30 0:37 / /sys/fs/cgroup/pids rw,nosuid,nodev,noexec,relatime shared:13 - cgroup cgroup rw,seclabel,pids
42 30 0:38 / /sys/fs/cgroup/devices rw,nosuid,nodev,noexec,relatime shared:14 - cgroup cgroup rw,seclabel,devices
43 30 0:39 / /sys/fs/cgroup/hugetlb rw,nosuid,nodev,noexec,relatime shared:15 - cgroup cgroup rw,seclabel,hugetlb
44 30 0:40 / /sys/fs/cgroup/blkio rw,nosuid,nodev,noexec,relatime shared:16 - cgroup cgroup rw,seclabel,blkio

Does that make sense?

Thanks,
Severin

> > While thinking about this issue here, I've decided to actually write a
> > test (using WhiteBox) so that all these cases are covered and will
> > continue to work. Running the test, CgroupSubsystemFactory, with
> > fastdebug proves that there is no clobbering if that matters. It's also
> > a way to reproduce the issue. So we have a regression test now too
> > (testCgroupv1NoMounts). Line 325-332 in [2] is the actual fix. Without
> > it, the test fails.
> > 
> > > Maybe you should just do the cgroupv2 processing if /proc/cgroups has
> > > cgroupv2 enabled and if that fails continue looking for cgroup v1
> > > mounts.
> > 
> > FWIW, I've amended the cgroup v1 logic to also check for the file
> > system type. Which needs to be 'cgroup' (not 'cgroup2') for cgroup v1.
> > This isn't strictly needed but more accurately reflects the intent IMO.
> 
> Sounds good.
> 
> > > Do you really need to do this?  The BSS section should be zeroed.
> > > 
> > > 55   for (int i = 0; i < CG_INFO_LENGTH; i++) {
> > >  56     cg_infos[i]._name = NULL;
> > >  57     cg_infos[i]._cgroup_path = NULL;
> > >  58     cg_infos[i]._root_mount_path = NULL;
> > >  59     cg_infos[i]._mount_path = NULL;
> > >  60   }
> > 
> > I believe so. CgroupInfo array is stack allocated and we should
> > initialize values properly. On the other hand, we should do that in the
> > constructor. Done now.
> 
> You are correct.  I was only looking at the diffs and not the full source.
> 
> > > You don?t need to check for NULL in cleanup, os::free can handle NULL
> > > pointers.
> > > 
> > > 339 void CgroupSubsystemFactory::cleanup(CgroupInfo* cg_infos) {
> > > 340   assert(cg_infos != NULL, "Invariant");
> > > 341   for (int i = 0; i < CG_INFO_LENGTH; i++) {
> > > 342     if (cg_infos[i]._name != NULL) {
> > > 343       os::free(cg_infos[i]._name);
> > > 344     }
> > > 345     if (cg_infos[i]._cgroup_path != NULL) {
> > > 346       os::free(cg_infos[i]._cgroup_path);
> > > 347     }
> > > 348     if (cg_infos[i]._root_mount_path != NULL) {
> > > 349       os::free(cg_infos[i]._root_mount_path);
> > > 350     }
> > > 351     if (cg_infos[i]._mount_path != NULL) {
> > > 352       os::free(cg_infos[i]._mount_path);
> > > 353     }
> > > 354   }
> > > 355 }
> > 
> > Fixed.
> > 
> > Let me know what you think about the updated webrev.
> > 
> > Thanks,
> > Severin
> > 
> > [1] http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/02/webrev/
> > [2] http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/src/hotspot/os/linux/cgroupSubsystem_linux.cpp.html
> > 
> > > Bob.
> > > 
> > > 
> > > 
> > > > On Feb 28, 2020, at 11:42 AM, Severin Gehwolf <
> > > > sgehwolf at redhat.com>
> > > > wrote:
> > > > 
> > > > Hi Bob,
> > > > 
> > > > Thanks for the review!
> > > > 
> > > > On Tue, 2020-02-25 at 15:08 -0500, Bob Vandette wrote:
> > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/01/webrev/src/hotspot/os/linux/cgroupSubsystem_linux.cpp.sdiff.html
> > > > > 
> > > > > If both cgroupv1 and v2 controllers are mounted, you might
> > > > > have
> > > > > some problems.  You might want to only look for
> > > > > cgroup2 mounts if is_cgroupV2 is true otherwise only look for
> > > > > cgroup mounts.
> > > > 
> > > > OK.
> > > > 
> > > > > 194           cg_infos[i]._mount_path =
> > > > > os::strdup(tmp_mount_point);
> > > > > This will stomp the pointer:
> > > > > 213           cg_infos[memory_idx]._mount_path =
> > > > > os::strdup(tmpmount);
> > > > 
> > > > You are right. If we are on a hybrid hierarchy and for some
> > > > reason
> > > > the
> > > > cgroup v2 mount shows up *after* cgroup mounts in
> > > > /proc/self/mountinfo,
> > > > we'd get the mount path wrong. So ordering matters! The reason
> > > > this
> > > > worked for me on hybrid is that cgroup2 mounts come *before*
> > > > cgroup
> > > > mounts on the system I've tested.
> > > > 
> > > > > You should free all strdup?d pointers in cg_infos in places
> > > > > where
> > > > > you fail and return NULL.
> > > > > 
> > > > > 81       cg_infos[memory_idx]._name = os::strdup(name);
> > > > > 213           cg_infos[memory_idx]._mount_path =
> > > > > os::strdup(tmpmount);
> > > > > 214           cg_infos[memory_idx]._root_mount_path =
> > > > > os::strdup(tmproot);
> > > > 
> > > > Done now. More thoughts?
> > > > 
> > > > Updated webrev: 
> > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/03/webrev/
> > > > 
> > > > Thanks,
> > > > Severin
> > > > 
> > > > > > On Feb 25, 2020, at 2:22 PM, Severin Gehwolf <
> > > > > > sgehwolf at redhat.com> wrote:
> > > > > > 
> > > > > > Hi,
> > > > > > 
> > > > > > Could I please get reviews of this cgroup-related patch?
> > > > > > After
> > > > > > JDK-
> > > > > > 8230305 old systems with no mounted cgroup controllers
> > > > > > would
> > > > > > get
> > > > > > detected as cgroups v2 (wrong). Then, when initializing the
> > > > > > v2
> > > > > > subsystem, it would fail. The trace about cgroupv2 mount
> > > > > > point
> > > > > > not
> > > > > > found is misleading in this case. While the outcome is
> > > > > > similar
> > > > > > pre/post
> > > > > > patch (NULL cgroup subsystem), I'd like to be explicit
> > > > > > about
> > > > > > this case.
> > > > > > 
> > > > > > The suggested fix is to look at /proc/self/mountinfo in
> > > > > > order
> > > > > > to
> > > > > > correctly detect whether this is cgroup v2 or cgroup v1
> > > > > > with no
> > > > > > mounted
> > > > > > controllers. In the latter case we just stop initialization
> > > > > > as
> > > > > > we'd
> > > > > > fail later in cgroupv1 code anyway. This aligns hotspot
> > > > > > code
> > > > > > with core-
> > > > > > libs after JDK-8239559.
> > > > > > 
> > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8239785
> > > > > > webrev: 
> > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/01/webrev/
> > > > > > 
> > > > > > Testing: jdk-submit, hotspot docker tests on cgroup v1 and
> > > > > > cgroup v2. All pass.
> > > > > > 
> > > > > > Thoughts?
> > > > > > 
> > > > > > Thanks,
> > > > > > Severin
> > > > > > 
> > 
> > 
> > 


From stefan.karlsson at oracle.com  Thu Mar  5 10:13:08 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Thu, 5 Mar 2020 11:13:08 +0100
Subject: RFR: 8240532: heap inspection prints trailing @ after name of
 module without version
In-Reply-To: <47a1919a-597f-d42a-7377-bc9c65b1c0e2@oracle.com>
References: <40f5d677-2308-3826-2b44-713af87c8f58@oracle.com>
 <47a1919a-597f-d42a-7377-bc9c65b1c0e2@oracle.com>
Message-ID: <3e5fc606-d1b6-b309-9c1d-9a881c8f57d9@oracle.com>

Thanks for reviewing, Lois.

StefanK

On 2020-03-04 18:34, Lois Foltan wrote:
> On 3/4/2020 12:12 PM, Stefan Karlsson wrote:
>> Hi all,
>>
>> Please review this patch to remove a trailing @ after module names of 
>> modules without a version.
>>
>> https://cr.openjdk.java.net/~stefank/8240532/webrev.01/
>> https://bugs.openjdk.java.net/browse/JDK-8240532
>>
>> The heap inspection functionality generates and prints a class 
>> histogram.
>>
>> Full image builds show module name and version as 'name at version':
>> [3,640s][trace][gc,classhisto????? ] GC(1)??? 4: 5067 162144 
>> java.lang.String (java.base at 15-internal)
>>
>> Exploded builds don't have a version and prints 'name@'
>> [4,361s][trace][gc,classhisto????? ] GC(1)??? 3: 7144 228608 
>> java.lang.String (java.base@)
>>
>> This looks weird and I propose that we drop the trailing @ when 
>> there's no version string:
>> [4,361s][trace][gc,classhisto????? ] GC(1)??? 3: 7144 228608 
>> java.lang.String (java.base)
>>
>> This affects the output of:
>> * -Xlog:classhisto*=trace
>> * jcmd <pid> GC.class_histogram
>> * inspectheap attach operation
>> * jmap -histo
>>
>> I've run the tests I could find that were using these operations. 
>> Other than that I intend to run tier1-3.
>>
>> Do others agree with this change?
>
> Yes, I think this change looks good and is consistent with how it is 
> currently handled in javaClasses.cpp/print_stack_element_to_stream() 
> and frame.cpp/frame::print_on_error() for example.? However, all of 
> these places should really be calling 
> ModuleEntry::should_show_version() instead of directly checking if the 
> version() is NULL.? But that could be addressed in a differing RFE.
>
> Thanks,
> Lois
>
>>
>> Thanks,
>> StefanK
>


From matthias.baesken at sap.com  Thu Mar  5 12:25:08 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Thu, 5 Mar 2020 12:25:08 +0000
Subject: RFR [XS]: 8240603: Windows 32bit compile error after 8238676
Message-ID: <AM0PR02MB422786674B9394D718FC016C93E20@AM0PR02MB4227.eurprd02.prod.outlook.com>

Hello ,  please look into the following small change .
We noticed that the Windows 32 bit builds fail  now  in jdk/jdk .

Error :
libatExit.c(107): error C2373: 'JNI_OnLoad': redefinition; different type modifiers
d:\output-jdk-dev\support\modules_include\java.base\jni.h(1970): note: see declaration of 'JNI_OnLoad'

Looks like the recent change  "8238676: jni crashes on accessing it from process exit hook"
caused this; we need on Windows 32 bit

JNIEXPORT jint JNICALL
JNI_OnLoad(JavaVM *jvm, void *reserved) {  ... }


to avoid this .

Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8240603

http://cr.openjdk.java.net/~mbaesken/webrevs/8240603.0/


Thanks, Matthias

From david.holmes at oracle.com  Thu Mar  5 13:23:59 2020
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 5 Mar 2020 23:23:59 +1000
Subject: RFR [XS]: 8240603: Windows 32bit compile error after 8238676
In-Reply-To: <AM0PR02MB422786674B9394D718FC016C93E20@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB422786674B9394D718FC016C93E20@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <dfb6ac7e-0348-fca8-b382-31a744aa205a@oracle.com>

Sorry about that oversight.

Looks good - thanks for fixing.

I'd classify this as trivial.

David

On 5/03/2020 10:25 pm, Baesken, Matthias wrote:
> Hello ,  please look into the following small change .
> We noticed that the Windows 32 bit builds fail  now  in jdk/jdk .
> 
> Error :
> libatExit.c(107): error C2373: 'JNI_OnLoad': redefinition; different type modifiers
> d:\output-jdk-dev\support\modules_include\java.base\jni.h(1970): note: see declaration of 'JNI_OnLoad'
> 
> Looks like the recent change  "8238676: jni crashes on accessing it from process exit hook"
> caused this; we need on Windows 32 bit
> 
> JNIEXPORT jint JNICALL
> JNI_OnLoad(JavaVM *jvm, void *reserved) {  ... }
> 
> 
> to avoid this .
> 
> Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8240603
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8240603.0/
> 
> 
> Thanks, Matthias
> 

From bob.vandette at oracle.com  Thu Mar  5 13:41:26 2020
From: bob.vandette at oracle.com (Bob Vandette)
Date: Thu, 5 Mar 2020 08:41:26 -0500
Subject: RFR(S): 8239785: Cgroups: Incorrect detection logic on old
 systems in hotspot
In-Reply-To: <264b4edf94d60a3256a9ff8d5de6a078c3a0819b.camel@redhat.com>
References: <362678e8bc8231c205c6b3327d08dd5179154e23.camel@redhat.com>
 <2A934185-9B81-4371-B985-F0C3C39B3FD3@oracle.com>
 <215e52601b2da8600cbfd47ef73634720a7736d2.camel@redhat.com>
 <EF45A2BB-360E-4767-B542-B420BF76FD2E@oracle.com>
 <d1683592b67f99e7625713d2db2c71641d50a738.camel@redhat.com>
 <1B61ACE7-AFB9-458C-B6AB-465259BD31C7@oracle.com>
 <264b4edf94d60a3256a9ff8d5de6a078c3a0819b.camel@redhat.com>
Message-ID: <A6EE0016-E33B-451D-BF0F-5F9C13A4DB32@oracle.com>



> On Mar 5, 2020, at 4:36 AM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
> 
> Hi Bob,
> 
> On Wed, 2020-03-04 at 15:39 -0500, Bob Vandette wrote:
>>> On Mar 4, 2020, at 1:59 PM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
>>> 
>>> Hi Bob,
>>> 
>>> Updated webrev:
>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/
>>> 
>>> Testing: hotspot docker tests on Linux fastdebug/release cgroup v1 and
>>> cgroup v2. New regression test. jdk submit. All pass.
>>> 
>>> On Fri, 2020-02-28 at 15:08 -0500, Bob Vandette wrote:
>>>> It still looks like the _mount_path can be corrupted if both cgroupv1 and cgroupv2 are mounted.
>>>> You?ll only initialize the _mount_path if cgroupv2 is detected in the cgroups file but if cgroupv1
>>>> is also mounted, you?ll still clobber the _mount_path.
>>> 
>>> How so? When we start processing /proc/self/mountinfo on line 170, then
>>> we have already looked at /proc/cgroups, thus is_cgroupsV2 is set to
>>> true (A) or false (B).
>>> 
>>> Cases A:
>>> 1. We are on an old system with *no* cgroup controllers (v1 or v2)
>>>   mounted (this bug). We are fine to continue as no cgroups entry
>>>   will be found in /proc/self/mountinfo. Thus, no clobbering.
>>> 2. We are on a true unified hierarchy only system. In that case
>>>   the sscanf for v1 would match (early match), but 'memory' et.al
>>>   controllers would not be found. Thus, no clobbering.
>>> 
>>> Cases B:
>>> 1. Hybrid hierarchy: controllers mounted via cgroup v1, but
>>>   cgroup v2 mounted in addition for system use. is_cgroupV2 will
>>>   prevent the cgroup v2 controller path lookup logic to be entered.
>>>   No clobbering.
>>> 2. cgroup v1 only hierarchy (legacy): Same logic applies. V2 setting
>>>   of mount paths won't be entered since is_cgroupV2 == false.
>>>   No clobbering.
>>> 
>>> What am I missing? FWIW, I've played around with the idea of keeping
>>> cgroup v1 and cgroup v2 mount paths separate[1]. If that's preferred,
>>> happy to go this route.
>> 
>> The case that I think is a problem is ?
>> 
>> cgroupv2 is detected in /proc/cgroups so is_cgroupv2 is true
>> both cgroupv1 and cgroupv2 devices are mounted
> 
> That's not possible. is_cgroupv2 cannot be true in that case.
> 
>> At line cgroupSubsystem_linux.cpp:292 in your new webrev, you process the cgroupv1
>> devices and clobber _mount_path.
>> 
>> What keeps this from occurring?
> 
> In that case 'is_cgroupv2 == false', because having any cgroups v1
> controllers mounted will result in some non-zero hierarchy ids in
> /proc/cgroups. That in turn will set is_cgroup2 == false.
> 
> From man 7 cgroups:
> 
> """"
>   /proc files
>       /proc/cgroups (since Linux 2.6.24)
> 
>              The fields in this file are, from left to right:
> 
>              1. The name of the controller.
> 
>              2. The unique ID of the cgroup hierarchy on which this controller is mounted.  If multiple cgroups v1 controllers are bound to the same hierarchy, then each will show the same hierarchy ID in
>                 this field.  The value in this field will be 0 if:
> 
>                   a) the controller is not mounted on a cgroups v1 hierarchy;
> 
>                   b) the controller is bound to the cgroups v2 single unified hierarchy; or
> 
>                   c) the controller is disabled (see below).
> """
> 
> The only way hierarchy id == 0 for a controller on cgroups v1 is a).
> It's never 0 when cgroup v1 and cgroup v2 are *both* mounted (hybrid).
> One example of such a /proc/cgroups is on my F30 workstation (hybrid):
> 
> $ cat /proc/cgroups 
> #subsys_name	hierarchy	num_cgroups	enabled
> cpuset	5	1	1
> cpu	3	1	1
> cpuacct	3	1	1
> blkio	11	1	1
> memory	6	136	1
> devices	9	75	1
> freezer	7	1	1
> net_cls	4	1	1
> perf_event	2	1	1
> net_prio	4	1	1
> hugetlb	10	1	1
> pids	8	81	1
> 
> And the corresponding mountinfo:
> 
> $ grep cgroup /proc/self/mountinfo 
> 30 23 0:26 / /sys/fs/cgroup ro,nosuid,nodev,noexec shared:4 - tmpfs tmpfs ro,seclabel,mode=755
> 31 30 0:27 / /sys/fs/cgroup/unified rw,nosuid,nodev,noexec,relatime shared:5 - cgroup2 cgroup2 rw,seclabel,nsdelegate
> 32 30 0:28 / /sys/fs/cgroup/systemd rw,nosuid,nodev,noexec,relatime shared:6 - cgroup cgroup rw,seclabel,xattr,name=systemd
> 35 30 0:31 / /sys/fs/cgroup/perf_event rw,nosuid,nodev,noexec,relatime shared:7 - cgroup cgroup rw,seclabel,perf_event
> 36 30 0:32 / /sys/fs/cgroup/cpu,cpuacct rw,nosuid,nodev,noexec,relatime shared:8 - cgroup cgroup rw,seclabel,cpu,cpuacct
> 37 30 0:33 / /sys/fs/cgroup/net_cls,net_prio rw,nosuid,nodev,noexec,relatime shared:9 - cgroup cgroup rw,seclabel,net_cls,net_prio
> 38 30 0:34 / /sys/fs/cgroup/cpuset rw,nosuid,nodev,noexec,relatime shared:10 - cgroup cgroup rw,seclabel,cpuset
> 39 30 0:35 / /sys/fs/cgroup/memory rw,nosuid,nodev,noexec,relatime shared:11 - cgroup cgroup rw,seclabel,memory
> 40 30 0:36 / /sys/fs/cgroup/freezer rw,nosuid,nodev,noexec,relatime shared:12 - cgroup cgroup rw,seclabel,freezer
> 41 30 0:37 / /sys/fs/cgroup/pids rw,nosuid,nodev,noexec,relatime shared:13 - cgroup cgroup rw,seclabel,pids
> 42 30 0:38 / /sys/fs/cgroup/devices rw,nosuid,nodev,noexec,relatime shared:14 - cgroup cgroup rw,seclabel,devices
> 43 30 0:39 / /sys/fs/cgroup/hugetlb rw,nosuid,nodev,noexec,relatime shared:15 - cgroup cgroup rw,seclabel,hugetlb
> 44 30 0:40 / /sys/fs/cgroup/blkio rw,nosuid,nodev,noexec,relatime shared:16 - cgroup cgroup rw,seclabel,blkio
> 
> Does that make sense?

Ok, that makes sense.

Thanks for the clarification.

Bob.

> 
> Thanks,
> Severin
> 
>>> While thinking about this issue here, I've decided to actually write a
>>> test (using WhiteBox) so that all these cases are covered and will
>>> continue to work. Running the test, CgroupSubsystemFactory, with
>>> fastdebug proves that there is no clobbering if that matters. It's also
>>> a way to reproduce the issue. So we have a regression test now too
>>> (testCgroupv1NoMounts). Line 325-332 in [2] is the actual fix. Without
>>> it, the test fails.
>>> 
>>>> Maybe you should just do the cgroupv2 processing if /proc/cgroups has
>>>> cgroupv2 enabled and if that fails continue looking for cgroup v1
>>>> mounts.
>>> 
>>> FWIW, I've amended the cgroup v1 logic to also check for the file
>>> system type. Which needs to be 'cgroup' (not 'cgroup2') for cgroup v1.
>>> This isn't strictly needed but more accurately reflects the intent IMO.
>> 
>> Sounds good.
>> 
>>>> Do you really need to do this?  The BSS section should be zeroed.
>>>> 
>>>> 55   for (int i = 0; i < CG_INFO_LENGTH; i++) {
>>>> 56     cg_infos[i]._name = NULL;
>>>> 57     cg_infos[i]._cgroup_path = NULL;
>>>> 58     cg_infos[i]._root_mount_path = NULL;
>>>> 59     cg_infos[i]._mount_path = NULL;
>>>> 60   }
>>> 
>>> I believe so. CgroupInfo array is stack allocated and we should
>>> initialize values properly. On the other hand, we should do that in the
>>> constructor. Done now.
>> 
>> You are correct.  I was only looking at the diffs and not the full source.
>> 
>>>> You don?t need to check for NULL in cleanup, os::free can handle NULL
>>>> pointers.
>>>> 
>>>> 339 void CgroupSubsystemFactory::cleanup(CgroupInfo* cg_infos) {
>>>> 340   assert(cg_infos != NULL, "Invariant");
>>>> 341   for (int i = 0; i < CG_INFO_LENGTH; i++) {
>>>> 342     if (cg_infos[i]._name != NULL) {
>>>> 343       os::free(cg_infos[i]._name);
>>>> 344     }
>>>> 345     if (cg_infos[i]._cgroup_path != NULL) {
>>>> 346       os::free(cg_infos[i]._cgroup_path);
>>>> 347     }
>>>> 348     if (cg_infos[i]._root_mount_path != NULL) {
>>>> 349       os::free(cg_infos[i]._root_mount_path);
>>>> 350     }
>>>> 351     if (cg_infos[i]._mount_path != NULL) {
>>>> 352       os::free(cg_infos[i]._mount_path);
>>>> 353     }
>>>> 354   }
>>>> 355 }
>>> 
>>> Fixed.
>>> 
>>> Let me know what you think about the updated webrev.
>>> 
>>> Thanks,
>>> Severin
>>> 
>>> [1] http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/02/webrev/
>>> [2] http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/src/hotspot/os/linux/cgroupSubsystem_linux.cpp.html
>>> 
>>>> Bob.
>>>> 
>>>> 
>>>> 
>>>>> On Feb 28, 2020, at 11:42 AM, Severin Gehwolf <
>>>>> sgehwolf at redhat.com>
>>>>> wrote:
>>>>> 
>>>>> Hi Bob,
>>>>> 
>>>>> Thanks for the review!
>>>>> 
>>>>> On Tue, 2020-02-25 at 15:08 -0500, Bob Vandette wrote:
>>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/01/webrev/src/hotspot/os/linux/cgroupSubsystem_linux.cpp.sdiff.html
>>>>>> 
>>>>>> If both cgroupv1 and v2 controllers are mounted, you might
>>>>>> have
>>>>>> some problems.  You might want to only look for
>>>>>> cgroup2 mounts if is_cgroupV2 is true otherwise only look for
>>>>>> cgroup mounts.
>>>>> 
>>>>> OK.
>>>>> 
>>>>>> 194           cg_infos[i]._mount_path =
>>>>>> os::strdup(tmp_mount_point);
>>>>>> This will stomp the pointer:
>>>>>> 213           cg_infos[memory_idx]._mount_path =
>>>>>> os::strdup(tmpmount);
>>>>> 
>>>>> You are right. If we are on a hybrid hierarchy and for some
>>>>> reason
>>>>> the
>>>>> cgroup v2 mount shows up *after* cgroup mounts in
>>>>> /proc/self/mountinfo,
>>>>> we'd get the mount path wrong. So ordering matters! The reason
>>>>> this
>>>>> worked for me on hybrid is that cgroup2 mounts come *before*
>>>>> cgroup
>>>>> mounts on the system I've tested.
>>>>> 
>>>>>> You should free all strdup?d pointers in cg_infos in places
>>>>>> where
>>>>>> you fail and return NULL.
>>>>>> 
>>>>>> 81       cg_infos[memory_idx]._name = os::strdup(name);
>>>>>> 213           cg_infos[memory_idx]._mount_path =
>>>>>> os::strdup(tmpmount);
>>>>>> 214           cg_infos[memory_idx]._root_mount_path =
>>>>>> os::strdup(tmproot);
>>>>> 
>>>>> Done now. More thoughts?
>>>>> 
>>>>> Updated webrev: 
>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/03/webrev/
>>>>> 
>>>>> Thanks,
>>>>> Severin
>>>>> 
>>>>>>> On Feb 25, 2020, at 2:22 PM, Severin Gehwolf <
>>>>>>> sgehwolf at redhat.com> wrote:
>>>>>>> 
>>>>>>> Hi,
>>>>>>> 
>>>>>>> Could I please get reviews of this cgroup-related patch?
>>>>>>> After
>>>>>>> JDK-
>>>>>>> 8230305 old systems with no mounted cgroup controllers
>>>>>>> would
>>>>>>> get
>>>>>>> detected as cgroups v2 (wrong). Then, when initializing the
>>>>>>> v2
>>>>>>> subsystem, it would fail. The trace about cgroupv2 mount
>>>>>>> point
>>>>>>> not
>>>>>>> found is misleading in this case. While the outcome is
>>>>>>> similar
>>>>>>> pre/post
>>>>>>> patch (NULL cgroup subsystem), I'd like to be explicit
>>>>>>> about
>>>>>>> this case.
>>>>>>> 
>>>>>>> The suggested fix is to look at /proc/self/mountinfo in
>>>>>>> order
>>>>>>> to
>>>>>>> correctly detect whether this is cgroup v2 or cgroup v1
>>>>>>> with no
>>>>>>> mounted
>>>>>>> controllers. In the latter case we just stop initialization
>>>>>>> as
>>>>>>> we'd
>>>>>>> fail later in cgroupv1 code anyway. This aligns hotspot
>>>>>>> code
>>>>>>> with core-
>>>>>>> libs after JDK-8239559.
>>>>>>> 
>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8239785
>>>>>>> webrev: 
>>>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/01/webrev/
>>>>>>> 
>>>>>>> Testing: jdk-submit, hotspot docker tests on cgroup v1 and
>>>>>>> cgroup v2. All pass.
>>>>>>> 
>>>>>>> Thoughts?
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> Severin
>>>>>>> 
>>> 
>>> 
>>> 
> 


From christoph.langer at sap.com  Thu Mar  5 15:40:54 2020
From: christoph.langer at sap.com (Langer, Christoph)
Date: Thu, 5 Mar 2020 15:40:54 +0000
Subject: RFR [XS]: 8240603: Windows 32bit compile error after 8238676
In-Reply-To: <dfb6ac7e-0348-fca8-b382-31a744aa205a@oracle.com>
References: <AM0PR02MB422786674B9394D718FC016C93E20@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <dfb6ac7e-0348-fca8-b382-31a744aa205a@oracle.com>
Message-ID: <AM0PR02MB57146015FC9C5AFCFB2368F88AE20@AM0PR02MB5714.eurprd02.prod.outlook.com>

+1

-----Original Message-----
From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of David Holmes
Sent: Donnerstag, 5. M?rz 2020 14:24
To: Baesken, Matthias <matthias.baesken at sap.com>; 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
Subject: Re: RFR [XS]: 8240603: Windows 32bit compile error after 8238676

Sorry about that oversight.

Looks good - thanks for fixing.

I'd classify this as trivial.

David

On 5/03/2020 10:25 pm, Baesken, Matthias wrote:
> Hello ,  please look into the following small change .
> We noticed that the Windows 32 bit builds fail  now  in jdk/jdk .
> 
> Error :
> libatExit.c(107): error C2373: 'JNI_OnLoad': redefinition; different type modifiers
> d:\output-jdk-dev\support\modules_include\java.base\jni.h(1970): note: see declaration of 'JNI_OnLoad'
> 
> Looks like the recent change  "8238676: jni crashes on accessing it from process exit hook"
> caused this; we need on Windows 32 bit
> 
> JNIEXPORT jint JNICALL
> JNI_OnLoad(JavaVM *jvm, void *reserved) {  ... }
> 
> 
> to avoid this .
> 
> Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8240603
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8240603.0/
> 
> 
> Thanks, Matthias
> 

From sgehwolf at redhat.com  Thu Mar  5 16:00:46 2020
From: sgehwolf at redhat.com (Severin Gehwolf)
Date: Thu, 05 Mar 2020 17:00:46 +0100
Subject: RFR(S): 8239785: Cgroups: Incorrect detection logic on old
 systems in hotspot
In-Reply-To: <A6EE0016-E33B-451D-BF0F-5F9C13A4DB32@oracle.com>
References: <362678e8bc8231c205c6b3327d08dd5179154e23.camel@redhat.com>
 <2A934185-9B81-4371-B985-F0C3C39B3FD3@oracle.com>
 <215e52601b2da8600cbfd47ef73634720a7736d2.camel@redhat.com>
 <EF45A2BB-360E-4767-B542-B420BF76FD2E@oracle.com>
 <d1683592b67f99e7625713d2db2c71641d50a738.camel@redhat.com>
 <1B61ACE7-AFB9-458C-B6AB-465259BD31C7@oracle.com>
 <264b4edf94d60a3256a9ff8d5de6a078c3a0819b.camel@redhat.com>
 <A6EE0016-E33B-451D-BF0F-5F9C13A4DB32@oracle.com>
Message-ID: <94c06270838cb3c090a227f2688a5c3f0aade64c.camel@redhat.com>

Hi Bob,

On Thu, 2020-03-05 at 08:41 -0500, Bob Vandette wrote:
> > On Mar 5, 2020, at 4:36 AM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
> > 
> > Hi Bob,
> > 
> > On Wed, 2020-03-04 at 15:39 -0500, Bob Vandette wrote:
> > > > On Mar 4, 2020, at 1:59 PM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
> > > > 
> > > > Hi Bob,
> > > > 
> > > > Updated webrev:
> > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/
> > > > 
> > > > Testing: hotspot docker tests on Linux fastdebug/release cgroup v1 and
> > > > cgroup v2. New regression test. jdk submit. All pass.
> > > > 
> > > > On Fri, 2020-02-28 at 15:08 -0500, Bob Vandette wrote:
> > > > > It still looks like the _mount_path can be corrupted if both cgroupv1 and cgroupv2 are mounted.
> > > > > You?ll only initialize the _mount_path if cgroupv2 is detected in the cgroups file but if cgroupv1
> > > > > is also mounted, you?ll still clobber the _mount_path.
> > > > 
> > > > How so? When we start processing /proc/self/mountinfo on line 170, then
> > > > we have already looked at /proc/cgroups, thus is_cgroupsV2 is set to
> > > > true (A) or false (B).
> > > > 
> > > > Cases A:
> > > > 1. We are on an old system with *no* cgroup controllers (v1 or v2)
> > > >   mounted (this bug). We are fine to continue as no cgroups entry
> > > >   will be found in /proc/self/mountinfo. Thus, no clobbering.
> > > > 2. We are on a true unified hierarchy only system. In that case
> > > >   the sscanf for v1 would match (early match), but 'memory' et.al
> > > >   controllers would not be found. Thus, no clobbering.
> > > > 
> > > > Cases B:
> > > > 1. Hybrid hierarchy: controllers mounted via cgroup v1, but
> > > >   cgroup v2 mounted in addition for system use. is_cgroupV2 will
> > > >   prevent the cgroup v2 controller path lookup logic to be entered.
> > > >   No clobbering.
> > > > 2. cgroup v1 only hierarchy (legacy): Same logic applies. V2 setting
> > > >   of mount paths won't be entered since is_cgroupV2 == false.
> > > >   No clobbering.
> > > > 
> > > > What am I missing? FWIW, I've played around with the idea of keeping
> > > > cgroup v1 and cgroup v2 mount paths separate[1]. If that's preferred,
> > > > happy to go this route.
> > > 
> > > The case that I think is a problem is ?
> > > 
> > > cgroupv2 is detected in /proc/cgroups so is_cgroupv2 is true
> > > both cgroupv1 and cgroupv2 devices are mounted
> > 
> > That's not possible. is_cgroupv2 cannot be true in that case.
> > 
> > > At line cgroupSubsystem_linux.cpp:292 in your new webrev, you process the cgroupv1
> > > devices and clobber _mount_path.
> > > 
> > > What keeps this from occurring?
> > 
> > In that case 'is_cgroupv2 == false', because having any cgroups v1
> > controllers mounted will result in some non-zero hierarchy ids in
> > /proc/cgroups. That in turn will set is_cgroup2 == false.
> > 
> > From man 7 cgroups:
> > 
> > """"
> >   /proc files
> >       /proc/cgroups (since Linux 2.6.24)
> > 
> >              The fields in this file are, from left to right:
> > 
> >              1. The name of the controller.
> > 
> >              2. The unique ID of the cgroup hierarchy on which this controller is mounted.  If multiple cgroups v1 controllers are bound to the same hierarchy, then each will show the same hierarchy ID in
> >                 this field.  The value in this field will be 0 if:
> > 
> >                   a) the controller is not mounted on a cgroups v1 hierarchy;
> > 
> >                   b) the controller is bound to the cgroups v2 single unified hierarchy; or
> > 
> >                   c) the controller is disabled (see below).
> > """
> > 
> > The only way hierarchy id == 0 for a controller on cgroups v1 is a).
> > It's never 0 when cgroup v1 and cgroup v2 are *both* mounted (hybrid).
> > One example of such a /proc/cgroups is on my F30 workstation (hybrid):
> > 
> > $ cat /proc/cgroups 
> > #subsys_name	hierarchy	num_cgroups	enabled
> > cpuset	5	1	1
> > cpu	3	1	1
> > cpuacct	3	1	1
> > blkio	11	1	1
> > memory	6	136	1
> > devices	9	75	1
> > freezer	7	1	1
> > net_cls	4	1	1
> > perf_event	2	1	1
> > net_prio	4	1	1
> > hugetlb	10	1	1
> > pids	8	81	1
> > 
> > And the corresponding mountinfo:
> > 
> > $ grep cgroup /proc/self/mountinfo 
> > 30 23 0:26 / /sys/fs/cgroup ro,nosuid,nodev,noexec shared:4 - tmpfs tmpfs ro,seclabel,mode=755
> > 31 30 0:27 / /sys/fs/cgroup/unified rw,nosuid,nodev,noexec,relatime shared:5 - cgroup2 cgroup2 rw,seclabel,nsdelegate
> > 32 30 0:28 / /sys/fs/cgroup/systemd rw,nosuid,nodev,noexec,relatime shared:6 - cgroup cgroup rw,seclabel,xattr,name=systemd
> > 35 30 0:31 / /sys/fs/cgroup/perf_event rw,nosuid,nodev,noexec,relatime shared:7 - cgroup cgroup rw,seclabel,perf_event
> > 36 30 0:32 / /sys/fs/cgroup/cpu,cpuacct rw,nosuid,nodev,noexec,relatime shared:8 - cgroup cgroup rw,seclabel,cpu,cpuacct
> > 37 30 0:33 / /sys/fs/cgroup/net_cls,net_prio rw,nosuid,nodev,noexec,relatime shared:9 - cgroup cgroup rw,seclabel,net_cls,net_prio
> > 38 30 0:34 / /sys/fs/cgroup/cpuset rw,nosuid,nodev,noexec,relatime shared:10 - cgroup cgroup rw,seclabel,cpuset
> > 39 30 0:35 / /sys/fs/cgroup/memory rw,nosuid,nodev,noexec,relatime shared:11 - cgroup cgroup rw,seclabel,memory
> > 40 30 0:36 / /sys/fs/cgroup/freezer rw,nosuid,nodev,noexec,relatime shared:12 - cgroup cgroup rw,seclabel,freezer
> > 41 30 0:37 / /sys/fs/cgroup/pids rw,nosuid,nodev,noexec,relatime shared:13 - cgroup cgroup rw,seclabel,pids
> > 42 30 0:38 / /sys/fs/cgroup/devices rw,nosuid,nodev,noexec,relatime shared:14 - cgroup cgroup rw,seclabel,devices
> > 43 30 0:39 / /sys/fs/cgroup/hugetlb rw,nosuid,nodev,noexec,relatime shared:15 - cgroup cgroup rw,seclabel,hugetlb
> > 44 30 0:40 / /sys/fs/cgroup/blkio rw,nosuid,nodev,noexec,relatime shared:16 - cgroup cgroup rw,seclabel,blkio
> > 
> > Does that make sense?
> 
> Ok, that makes sense.
> 
> Thanks for the clarification.

Anytime. Can I put you down as a reviewer (the non-census one)?

Thanks,
Severin

> Bob.
> 
> > Thanks,
> > Severin
> > 
> > > > While thinking about this issue here, I've decided to actually write a
> > > > test (using WhiteBox) so that all these cases are covered and will
> > > > continue to work. Running the test, CgroupSubsystemFactory, with
> > > > fastdebug proves that there is no clobbering if that matters. It's also
> > > > a way to reproduce the issue. So we have a regression test now too
> > > > (testCgroupv1NoMounts). Line 325-332 in [2] is the actual fix. Without
> > > > it, the test fails.
> > > > 
> > > > > Maybe you should just do the cgroupv2 processing if /proc/cgroups has
> > > > > cgroupv2 enabled and if that fails continue looking for cgroup v1
> > > > > mounts.
> > > > 
> > > > FWIW, I've amended the cgroup v1 logic to also check for the file
> > > > system type. Which needs to be 'cgroup' (not 'cgroup2') for cgroup v1.
> > > > This isn't strictly needed but more accurately reflects the intent IMO.
> > > 
> > > Sounds good.
> > > 
> > > > > Do you really need to do this?  The BSS section should be zeroed.
> > > > > 
> > > > > 55   for (int i = 0; i < CG_INFO_LENGTH; i++) {
> > > > > 56     cg_infos[i]._name = NULL;
> > > > > 57     cg_infos[i]._cgroup_path = NULL;
> > > > > 58     cg_infos[i]._root_mount_path = NULL;
> > > > > 59     cg_infos[i]._mount_path = NULL;
> > > > > 60   }
> > > > 
> > > > I believe so. CgroupInfo array is stack allocated and we should
> > > > initialize values properly. On the other hand, we should do that in the
> > > > constructor. Done now.
> > > 
> > > You are correct.  I was only looking at the diffs and not the full source.
> > > 
> > > > > You don?t need to check for NULL in cleanup, os::free can handle NULL
> > > > > pointers.
> > > > > 
> > > > > 339 void CgroupSubsystemFactory::cleanup(CgroupInfo* cg_infos) {
> > > > > 340   assert(cg_infos != NULL, "Invariant");
> > > > > 341   for (int i = 0; i < CG_INFO_LENGTH; i++) {
> > > > > 342     if (cg_infos[i]._name != NULL) {
> > > > > 343       os::free(cg_infos[i]._name);
> > > > > 344     }
> > > > > 345     if (cg_infos[i]._cgroup_path != NULL) {
> > > > > 346       os::free(cg_infos[i]._cgroup_path);
> > > > > 347     }
> > > > > 348     if (cg_infos[i]._root_mount_path != NULL) {
> > > > > 349       os::free(cg_infos[i]._root_mount_path);
> > > > > 350     }
> > > > > 351     if (cg_infos[i]._mount_path != NULL) {
> > > > > 352       os::free(cg_infos[i]._mount_path);
> > > > > 353     }
> > > > > 354   }
> > > > > 355 }
> > > > 
> > > > Fixed.
> > > > 
> > > > Let me know what you think about the updated webrev.
> > > > 
> > > > Thanks,
> > > > Severin
> > > > 
> > > > [1] http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/02/webrev/
> > > > [2] http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/src/hotspot/os/linux/cgroupSubsystem_linux.cpp.html
> > > > 
> > > > > Bob.
> > > > > 
> > > > > 
> > > > > 
> > > > > > On Feb 28, 2020, at 11:42 AM, Severin Gehwolf <
> > > > > > sgehwolf at redhat.com>
> > > > > > wrote:
> > > > > > 
> > > > > > Hi Bob,
> > > > > > 
> > > > > > Thanks for the review!
> > > > > > 
> > > > > > On Tue, 2020-02-25 at 15:08 -0500, Bob Vandette wrote:
> > > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/01/webrev/src/hotspot/os/linux/cgroupSubsystem_linux.cpp.sdiff.html
> > > > > > > 
> > > > > > > If both cgroupv1 and v2 controllers are mounted, you might
> > > > > > > have
> > > > > > > some problems.  You might want to only look for
> > > > > > > cgroup2 mounts if is_cgroupV2 is true otherwise only look for
> > > > > > > cgroup mounts.
> > > > > > 
> > > > > > OK.
> > > > > > 
> > > > > > > 194           cg_infos[i]._mount_path =
> > > > > > > os::strdup(tmp_mount_point);
> > > > > > > This will stomp the pointer:
> > > > > > > 213           cg_infos[memory_idx]._mount_path =
> > > > > > > os::strdup(tmpmount);
> > > > > > 
> > > > > > You are right. If we are on a hybrid hierarchy and for some
> > > > > > reason
> > > > > > the
> > > > > > cgroup v2 mount shows up *after* cgroup mounts in
> > > > > > /proc/self/mountinfo,
> > > > > > we'd get the mount path wrong. So ordering matters! The reason
> > > > > > this
> > > > > > worked for me on hybrid is that cgroup2 mounts come *before*
> > > > > > cgroup
> > > > > > mounts on the system I've tested.
> > > > > > 
> > > > > > > You should free all strdup?d pointers in cg_infos in places
> > > > > > > where
> > > > > > > you fail and return NULL.
> > > > > > > 
> > > > > > > 81       cg_infos[memory_idx]._name = os::strdup(name);
> > > > > > > 213           cg_infos[memory_idx]._mount_path =
> > > > > > > os::strdup(tmpmount);
> > > > > > > 214           cg_infos[memory_idx]._root_mount_path =
> > > > > > > os::strdup(tmproot);
> > > > > > 
> > > > > > Done now. More thoughts?
> > > > > > 
> > > > > > Updated webrev: 
> > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/03/webrev/
> > > > > > 
> > > > > > Thanks,
> > > > > > Severin
> > > > > > 
> > > > > > > > On Feb 25, 2020, at 2:22 PM, Severin Gehwolf <
> > > > > > > > sgehwolf at redhat.com> wrote:
> > > > > > > > 
> > > > > > > > Hi,
> > > > > > > > 
> > > > > > > > Could I please get reviews of this cgroup-related patch?
> > > > > > > > After
> > > > > > > > JDK-
> > > > > > > > 8230305 old systems with no mounted cgroup controllers
> > > > > > > > would
> > > > > > > > get
> > > > > > > > detected as cgroups v2 (wrong). Then, when initializing the
> > > > > > > > v2
> > > > > > > > subsystem, it would fail. The trace about cgroupv2 mount
> > > > > > > > point
> > > > > > > > not
> > > > > > > > found is misleading in this case. While the outcome is
> > > > > > > > similar
> > > > > > > > pre/post
> > > > > > > > patch (NULL cgroup subsystem), I'd like to be explicit
> > > > > > > > about
> > > > > > > > this case.
> > > > > > > > 
> > > > > > > > The suggested fix is to look at /proc/self/mountinfo in
> > > > > > > > order
> > > > > > > > to
> > > > > > > > correctly detect whether this is cgroup v2 or cgroup v1
> > > > > > > > with no
> > > > > > > > mounted
> > > > > > > > controllers. In the latter case we just stop initialization
> > > > > > > > as
> > > > > > > > we'd
> > > > > > > > fail later in cgroupv1 code anyway. This aligns hotspot
> > > > > > > > code
> > > > > > > > with core-
> > > > > > > > libs after JDK-8239559.
> > > > > > > > 
> > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8239785
> > > > > > > > webrev: 
> > > > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/01/webrev/
> > > > > > > > 
> > > > > > > > Testing: jdk-submit, hotspot docker tests on cgroup v1 and
> > > > > > > > cgroup v2. All pass.
> > > > > > > > 
> > > > > > > > Thoughts?
> > > > > > > > 
> > > > > > > > Thanks,
> > > > > > > > Severin
> > > > > > > > 
> > > > 
> > > > 


From bob.vandette at oracle.com  Thu Mar  5 16:13:07 2020
From: bob.vandette at oracle.com (Bob Vandette)
Date: Thu, 5 Mar 2020 11:13:07 -0500
Subject: RFR(S): 8239785: Cgroups: Incorrect detection logic on old
 systems in hotspot
In-Reply-To: <94c06270838cb3c090a227f2688a5c3f0aade64c.camel@redhat.com>
References: <362678e8bc8231c205c6b3327d08dd5179154e23.camel@redhat.com>
 <2A934185-9B81-4371-B985-F0C3C39B3FD3@oracle.com>
 <215e52601b2da8600cbfd47ef73634720a7736d2.camel@redhat.com>
 <EF45A2BB-360E-4767-B542-B420BF76FD2E@oracle.com>
 <d1683592b67f99e7625713d2db2c71641d50a738.camel@redhat.com>
 <1B61ACE7-AFB9-458C-B6AB-465259BD31C7@oracle.com>
 <264b4edf94d60a3256a9ff8d5de6a078c3a0819b.camel@redhat.com>
 <A6EE0016-E33B-451D-BF0F-5F9C13A4DB32@oracle.com>
 <94c06270838cb3c090a227f2688a5c3f0aade64c.camel@redhat.com>
Message-ID: <C81DB191-91BF-4ED6-81E1-51605FAE6693@oracle.com>



> On Mar 5, 2020, at 11:00 AM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
> 
> Hi Bob,
> 
> On Thu, 2020-03-05 at 08:41 -0500, Bob Vandette wrote:
>>> On Mar 5, 2020, at 4:36 AM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
>>> 
>>> Hi Bob,
>>> 
>>> On Wed, 2020-03-04 at 15:39 -0500, Bob Vandette wrote:
>>>>> On Mar 4, 2020, at 1:59 PM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
>>>>> 
>>>>> Hi Bob,
>>>>> 
>>>>> Updated webrev:
>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/
>>>>> 
>>>>> Testing: hotspot docker tests on Linux fastdebug/release cgroup v1 and
>>>>> cgroup v2. New regression test. jdk submit. All pass.
>>>>> 
>>>>> On Fri, 2020-02-28 at 15:08 -0500, Bob Vandette wrote:
>>>>>> It still looks like the _mount_path can be corrupted if both cgroupv1 and cgroupv2 are mounted.
>>>>>> You?ll only initialize the _mount_path if cgroupv2 is detected in the cgroups file but if cgroupv1
>>>>>> is also mounted, you?ll still clobber the _mount_path.
>>>>> 
>>>>> How so? When we start processing /proc/self/mountinfo on line 170, then
>>>>> we have already looked at /proc/cgroups, thus is_cgroupsV2 is set to
>>>>> true (A) or false (B).
>>>>> 
>>>>> Cases A:
>>>>> 1. We are on an old system with *no* cgroup controllers (v1 or v2)
>>>>>  mounted (this bug). We are fine to continue as no cgroups entry
>>>>>  will be found in /proc/self/mountinfo. Thus, no clobbering.
>>>>> 2. We are on a true unified hierarchy only system. In that case
>>>>>  the sscanf for v1 would match (early match), but 'memory' et.al
>>>>>  controllers would not be found. Thus, no clobbering.
>>>>> 
>>>>> Cases B:
>>>>> 1. Hybrid hierarchy: controllers mounted via cgroup v1, but
>>>>>  cgroup v2 mounted in addition for system use. is_cgroupV2 will
>>>>>  prevent the cgroup v2 controller path lookup logic to be entered.
>>>>>  No clobbering.
>>>>> 2. cgroup v1 only hierarchy (legacy): Same logic applies. V2 setting
>>>>>  of mount paths won't be entered since is_cgroupV2 == false.
>>>>>  No clobbering.
>>>>> 
>>>>> What am I missing? FWIW, I've played around with the idea of keeping
>>>>> cgroup v1 and cgroup v2 mount paths separate[1]. If that's preferred,
>>>>> happy to go this route.
>>>> 
>>>> The case that I think is a problem is ?
>>>> 
>>>> cgroupv2 is detected in /proc/cgroups so is_cgroupv2 is true
>>>> both cgroupv1 and cgroupv2 devices are mounted
>>> 
>>> That's not possible. is_cgroupv2 cannot be true in that case.
>>> 
>>>> At line cgroupSubsystem_linux.cpp:292 in your new webrev, you process the cgroupv1
>>>> devices and clobber _mount_path.
>>>> 
>>>> What keeps this from occurring?
>>> 
>>> In that case 'is_cgroupv2 == false', because having any cgroups v1
>>> controllers mounted will result in some non-zero hierarchy ids in
>>> /proc/cgroups. That in turn will set is_cgroup2 == false.
>>> 
>>> From man 7 cgroups:
>>> 
>>> """"
>>>  /proc files
>>>      /proc/cgroups (since Linux 2.6.24)
>>> 
>>>             The fields in this file are, from left to right:
>>> 
>>>             1. The name of the controller.
>>> 
>>>             2. The unique ID of the cgroup hierarchy on which this controller is mounted.  If multiple cgroups v1 controllers are bound to the same hierarchy, then each will show the same hierarchy ID in
>>>                this field.  The value in this field will be 0 if:
>>> 
>>>                  a) the controller is not mounted on a cgroups v1 hierarchy;
>>> 
>>>                  b) the controller is bound to the cgroups v2 single unified hierarchy; or
>>> 
>>>                  c) the controller is disabled (see below).
>>> """
>>> 
>>> The only way hierarchy id == 0 for a controller on cgroups v1 is a).
>>> It's never 0 when cgroup v1 and cgroup v2 are *both* mounted (hybrid).
>>> One example of such a /proc/cgroups is on my F30 workstation (hybrid):
>>> 
>>> $ cat /proc/cgroups 
>>> #subsys_name	hierarchy	num_cgroups	enabled
>>> cpuset	5	1	1
>>> cpu	3	1	1
>>> cpuacct	3	1	1
>>> blkio	11	1	1
>>> memory	6	136	1
>>> devices	9	75	1
>>> freezer	7	1	1
>>> net_cls	4	1	1
>>> perf_event	2	1	1
>>> net_prio	4	1	1
>>> hugetlb	10	1	1
>>> pids	8	81	1
>>> 
>>> And the corresponding mountinfo:
>>> 
>>> $ grep cgroup /proc/self/mountinfo 
>>> 30 23 0:26 / /sys/fs/cgroup ro,nosuid,nodev,noexec shared:4 - tmpfs tmpfs ro,seclabel,mode=755
>>> 31 30 0:27 / /sys/fs/cgroup/unified rw,nosuid,nodev,noexec,relatime shared:5 - cgroup2 cgroup2 rw,seclabel,nsdelegate
>>> 32 30 0:28 / /sys/fs/cgroup/systemd rw,nosuid,nodev,noexec,relatime shared:6 - cgroup cgroup rw,seclabel,xattr,name=systemd
>>> 35 30 0:31 / /sys/fs/cgroup/perf_event rw,nosuid,nodev,noexec,relatime shared:7 - cgroup cgroup rw,seclabel,perf_event
>>> 36 30 0:32 / /sys/fs/cgroup/cpu,cpuacct rw,nosuid,nodev,noexec,relatime shared:8 - cgroup cgroup rw,seclabel,cpu,cpuacct
>>> 37 30 0:33 / /sys/fs/cgroup/net_cls,net_prio rw,nosuid,nodev,noexec,relatime shared:9 - cgroup cgroup rw,seclabel,net_cls,net_prio
>>> 38 30 0:34 / /sys/fs/cgroup/cpuset rw,nosuid,nodev,noexec,relatime shared:10 - cgroup cgroup rw,seclabel,cpuset
>>> 39 30 0:35 / /sys/fs/cgroup/memory rw,nosuid,nodev,noexec,relatime shared:11 - cgroup cgroup rw,seclabel,memory
>>> 40 30 0:36 / /sys/fs/cgroup/freezer rw,nosuid,nodev,noexec,relatime shared:12 - cgroup cgroup rw,seclabel,freezer
>>> 41 30 0:37 / /sys/fs/cgroup/pids rw,nosuid,nodev,noexec,relatime shared:13 - cgroup cgroup rw,seclabel,pids
>>> 42 30 0:38 / /sys/fs/cgroup/devices rw,nosuid,nodev,noexec,relatime shared:14 - cgroup cgroup rw,seclabel,devices
>>> 43 30 0:39 / /sys/fs/cgroup/hugetlb rw,nosuid,nodev,noexec,relatime shared:15 - cgroup cgroup rw,seclabel,hugetlb
>>> 44 30 0:40 / /sys/fs/cgroup/blkio rw,nosuid,nodev,noexec,relatime shared:16 - cgroup cgroup rw,seclabel,blkio
>>> 
>>> Does that make sense?
>> 
>> Ok, that makes sense.
>> 
>> Thanks for the clarification.
> 
> Anytime. Can I put you down as a reviewer (the non-census one)?

Yes, absolutely.

Bob.

> 
> Thanks,
> Severin
> 
>> Bob.
>> 
>>> Thanks,
>>> Severin
>>> 
>>>>> While thinking about this issue here, I've decided to actually write a
>>>>> test (using WhiteBox) so that all these cases are covered and will
>>>>> continue to work. Running the test, CgroupSubsystemFactory, with
>>>>> fastdebug proves that there is no clobbering if that matters. It's also
>>>>> a way to reproduce the issue. So we have a regression test now too
>>>>> (testCgroupv1NoMounts). Line 325-332 in [2] is the actual fix. Without
>>>>> it, the test fails.
>>>>> 
>>>>>> Maybe you should just do the cgroupv2 processing if /proc/cgroups has
>>>>>> cgroupv2 enabled and if that fails continue looking for cgroup v1
>>>>>> mounts.
>>>>> 
>>>>> FWIW, I've amended the cgroup v1 logic to also check for the file
>>>>> system type. Which needs to be 'cgroup' (not 'cgroup2') for cgroup v1.
>>>>> This isn't strictly needed but more accurately reflects the intent IMO.
>>>> 
>>>> Sounds good.
>>>> 
>>>>>> Do you really need to do this?  The BSS section should be zeroed.
>>>>>> 
>>>>>> 55   for (int i = 0; i < CG_INFO_LENGTH; i++) {
>>>>>> 56     cg_infos[i]._name = NULL;
>>>>>> 57     cg_infos[i]._cgroup_path = NULL;
>>>>>> 58     cg_infos[i]._root_mount_path = NULL;
>>>>>> 59     cg_infos[i]._mount_path = NULL;
>>>>>> 60   }
>>>>> 
>>>>> I believe so. CgroupInfo array is stack allocated and we should
>>>>> initialize values properly. On the other hand, we should do that in the
>>>>> constructor. Done now.
>>>> 
>>>> You are correct.  I was only looking at the diffs and not the full source.
>>>> 
>>>>>> You don?t need to check for NULL in cleanup, os::free can handle NULL
>>>>>> pointers.
>>>>>> 
>>>>>> 339 void CgroupSubsystemFactory::cleanup(CgroupInfo* cg_infos) {
>>>>>> 340   assert(cg_infos != NULL, "Invariant");
>>>>>> 341   for (int i = 0; i < CG_INFO_LENGTH; i++) {
>>>>>> 342     if (cg_infos[i]._name != NULL) {
>>>>>> 343       os::free(cg_infos[i]._name);
>>>>>> 344     }
>>>>>> 345     if (cg_infos[i]._cgroup_path != NULL) {
>>>>>> 346       os::free(cg_infos[i]._cgroup_path);
>>>>>> 347     }
>>>>>> 348     if (cg_infos[i]._root_mount_path != NULL) {
>>>>>> 349       os::free(cg_infos[i]._root_mount_path);
>>>>>> 350     }
>>>>>> 351     if (cg_infos[i]._mount_path != NULL) {
>>>>>> 352       os::free(cg_infos[i]._mount_path);
>>>>>> 353     }
>>>>>> 354   }
>>>>>> 355 }
>>>>> 
>>>>> Fixed.
>>>>> 
>>>>> Let me know what you think about the updated webrev.
>>>>> 
>>>>> Thanks,
>>>>> Severin
>>>>> 
>>>>> [1] http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/02/webrev/
>>>>> [2] http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/src/hotspot/os/linux/cgroupSubsystem_linux.cpp.html
>>>>> 
>>>>>> Bob.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> On Feb 28, 2020, at 11:42 AM, Severin Gehwolf <
>>>>>>> sgehwolf at redhat.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>> Hi Bob,
>>>>>>> 
>>>>>>> Thanks for the review!
>>>>>>> 
>>>>>>> On Tue, 2020-02-25 at 15:08 -0500, Bob Vandette wrote:
>>>>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/01/webrev/src/hotspot/os/linux/cgroupSubsystem_linux.cpp.sdiff.html
>>>>>>>> 
>>>>>>>> If both cgroupv1 and v2 controllers are mounted, you might
>>>>>>>> have
>>>>>>>> some problems.  You might want to only look for
>>>>>>>> cgroup2 mounts if is_cgroupV2 is true otherwise only look for
>>>>>>>> cgroup mounts.
>>>>>>> 
>>>>>>> OK.
>>>>>>> 
>>>>>>>> 194           cg_infos[i]._mount_path =
>>>>>>>> os::strdup(tmp_mount_point);
>>>>>>>> This will stomp the pointer:
>>>>>>>> 213           cg_infos[memory_idx]._mount_path =
>>>>>>>> os::strdup(tmpmount);
>>>>>>> 
>>>>>>> You are right. If we are on a hybrid hierarchy and for some
>>>>>>> reason
>>>>>>> the
>>>>>>> cgroup v2 mount shows up *after* cgroup mounts in
>>>>>>> /proc/self/mountinfo,
>>>>>>> we'd get the mount path wrong. So ordering matters! The reason
>>>>>>> this
>>>>>>> worked for me on hybrid is that cgroup2 mounts come *before*
>>>>>>> cgroup
>>>>>>> mounts on the system I've tested.
>>>>>>> 
>>>>>>>> You should free all strdup?d pointers in cg_infos in places
>>>>>>>> where
>>>>>>>> you fail and return NULL.
>>>>>>>> 
>>>>>>>> 81       cg_infos[memory_idx]._name = os::strdup(name);
>>>>>>>> 213           cg_infos[memory_idx]._mount_path =
>>>>>>>> os::strdup(tmpmount);
>>>>>>>> 214           cg_infos[memory_idx]._root_mount_path =
>>>>>>>> os::strdup(tmproot);
>>>>>>> 
>>>>>>> Done now. More thoughts?
>>>>>>> 
>>>>>>> Updated webrev: 
>>>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/03/webrev/
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> Severin
>>>>>>> 
>>>>>>>>> On Feb 25, 2020, at 2:22 PM, Severin Gehwolf <
>>>>>>>>> sgehwolf at redhat.com> wrote:
>>>>>>>>> 
>>>>>>>>> Hi,
>>>>>>>>> 
>>>>>>>>> Could I please get reviews of this cgroup-related patch?
>>>>>>>>> After
>>>>>>>>> JDK-
>>>>>>>>> 8230305 old systems with no mounted cgroup controllers
>>>>>>>>> would
>>>>>>>>> get
>>>>>>>>> detected as cgroups v2 (wrong). Then, when initializing the
>>>>>>>>> v2
>>>>>>>>> subsystem, it would fail. The trace about cgroupv2 mount
>>>>>>>>> point
>>>>>>>>> not
>>>>>>>>> found is misleading in this case. While the outcome is
>>>>>>>>> similar
>>>>>>>>> pre/post
>>>>>>>>> patch (NULL cgroup subsystem), I'd like to be explicit
>>>>>>>>> about
>>>>>>>>> this case.
>>>>>>>>> 
>>>>>>>>> The suggested fix is to look at /proc/self/mountinfo in
>>>>>>>>> order
>>>>>>>>> to
>>>>>>>>> correctly detect whether this is cgroup v2 or cgroup v1
>>>>>>>>> with no
>>>>>>>>> mounted
>>>>>>>>> controllers. In the latter case we just stop initialization
>>>>>>>>> as
>>>>>>>>> we'd
>>>>>>>>> fail later in cgroupv1 code anyway. This aligns hotspot
>>>>>>>>> code
>>>>>>>>> with core-
>>>>>>>>> libs after JDK-8239559.
>>>>>>>>> 
>>>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8239785
>>>>>>>>> webrev: 
>>>>>>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/01/webrev/
>>>>>>>>> 
>>>>>>>>> Testing: jdk-submit, hotspot docker tests on cgroup v1 and
>>>>>>>>> cgroup v2. All pass.
>>>>>>>>> 
>>>>>>>>> Thoughts?
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> Severin
>>>>>>>>> 
>>>>> 
>>>>> 
> 


From aph at redhat.com  Thu Mar  5 18:06:49 2020
From: aph at redhat.com (Andrew Haley)
Date: Thu, 5 Mar 2020 18:06:49 +0000
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
Message-ID: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>

is_power_of_2() has always behaved oddly, and it still behaves oddly
but in a different way since 8183574: Unify the is_power_of_2
functions. And this change breaks TCK on AArch64 systems.

Previously, is_power_of_2(long) returned true for LONG_MIN, the most
-ve signed long, but returned false for INT_MIN, the most -ve signed
int.

Also, is_power_of_2(long) had Undefined Behaviour (UB) for LONG_MIN:
it overflowed from the most -ve long to the most +ve long.

Now, is_power_of_2() has Undefined Behaviour for both int and long
arguments when applied to their respective most negative values. In
testing it returns true for both, but as this is UB it is not
guaranteed.

previous definition:

inline bool is_power_of_2(intptr_t x) {
  return ((x != NoBits) && (mask_bits(x, x - 1) == NoBits));
}

since JDK-8240615:

template <typename T>
bool is_power_of_2(T x) {
  return (x != T(0)) && ((x & (x - 1)) == T(0));
}

I do not believe that this behavioural change was deliberate.

I suggest that we should change the definition of is_power_of_2() for
all signed types such that it returns false for negative
arguments. This makes the behaviour consistent and also avoids
undefined behaviour.

This is important because is_power_of_2 is sometimes used as a guard
in ADfile patterns:

instruct cmpI_branch_bit(cmpOpEqNe cmp, iRegIorL2I op1, immI op2, immI0 op3, label labl) %{
  match(If cmp (CmpI (AndI op1 op2) op3));
  predicate(is_power_of_2(n->in(2)->in(1)->in(2)->get_int()));
  effect(USE labl);

  ins_cost(BRANCH_COST);
  format %{ "tb$cmp $op1, $op2, $labl" %}
  ins_encode %{
    Label* L = $labl$$label;
    Assembler::Condition cond = (Assembler::Condition)$cmp$$cmpcode;
    int bit = exact_log2($op2$$constant);
   ...

This has a runtime error since JDK-8240615 because the initial
is_power_of_2 test succeeds, but the later exact_log2
fails. exact_log2() takes an intptr_t, not an int. On 64-bit systems,
intptr_t is a 64-bit signed type, so the first call to is_power_of_2
succeeds, whereas exact_log2 fails because its arg is sign-extended
to 64 bits.

http://cr.openjdk.java.net/~aph/8240615-1/

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From stefan.karlsson at oracle.com  Thu Mar  5 18:23:31 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Thu, 5 Mar 2020 19:23:31 +0100
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
Message-ID: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>

Hi Andrew,

The implementation of is_power_of_2 looks good to me.

Since this changes the value of is_power_of_2(LONG_MIN), did you try to 
figure out if any of the existing code relies on this returning true? It 
might be prudent to run this patch through testing with an assert 
catching any places where LONG_MIN is used as the argument. I can run 
that on our side, but need help for full platform coverage.

You used std::numeric_limits<T>::min() in the test. There's a comment in 
powerOfTwo.hpp stating:

// Helper function to get the maximum positive value. Implemented here
// since using std::numeric_limits<T>::max() seems problematic on some
// platforms.

I don't know if this means that min() is problematic as well?

Thanks for fixing this.

StefanK

On 2020-03-05 19:06, Andrew Haley wrote:
> is_power_of_2() has always behaved oddly, and it still behaves oddly
> but in a different way since 8183574: Unify the is_power_of_2
> functions. And this change breaks TCK on AArch64 systems.
>
> Previously, is_power_of_2(long) returned true for LONG_MIN, the most
> -ve signed long, but returned false for INT_MIN, the most -ve signed
> int.
>
> Also, is_power_of_2(long) had Undefined Behaviour (UB) for LONG_MIN:
> it overflowed from the most -ve long to the most +ve long.
>
> Now, is_power_of_2() has Undefined Behaviour for both int and long
> arguments when applied to their respective most negative values. In
> testing it returns true for both, but as this is UB it is not
> guaranteed.
>
> previous definition:
>
> inline bool is_power_of_2(intptr_t x) {
>    return ((x != NoBits) && (mask_bits(x, x - 1) == NoBits));
> }
>
> since JDK-8240615:
>
> template <typename T>
> bool is_power_of_2(T x) {
>    return (x != T(0)) && ((x & (x - 1)) == T(0));
> }
>
> I do not believe that this behavioural change was deliberate.
>
> I suggest that we should change the definition of is_power_of_2() for
> all signed types such that it returns false for negative
> arguments. This makes the behaviour consistent and also avoids
> undefined behaviour.
>
> This is important because is_power_of_2 is sometimes used as a guard
> in ADfile patterns:
>
> instruct cmpI_branch_bit(cmpOpEqNe cmp, iRegIorL2I op1, immI op2, immI0 op3, label labl) %{
>    match(If cmp (CmpI (AndI op1 op2) op3));
>    predicate(is_power_of_2(n->in(2)->in(1)->in(2)->get_int()));
>    effect(USE labl);
>
>    ins_cost(BRANCH_COST);
>    format %{ "tb$cmp $op1, $op2, $labl" %}
>    ins_encode %{
>      Label* L = $labl$$label;
>      Assembler::Condition cond = (Assembler::Condition)$cmp$$cmpcode;
>      int bit = exact_log2($op2$$constant);
>     ...
>
> This has a runtime error since JDK-8240615 because the initial
> is_power_of_2 test succeeds, but the later exact_log2
> fails. exact_log2() takes an intptr_t, not an int. On 64-bit systems,
> intptr_t is a 64-bit signed type, so the first call to is_power_of_2
> succeeds, whereas exact_log2 fails because its arg is sign-extended
> to 64 bits.
>
> http://cr.openjdk.java.net/~aph/8240615-1/
>


From claes.redestad at oracle.com  Thu Mar  5 18:42:30 2020
From: claes.redestad at oracle.com (Claes Redestad)
Date: Thu, 5 Mar 2020 19:42:30 +0100
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
 <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
Message-ID: <702314ab-17bc-1a3d-7384-6810beaec7b2@oracle.com>



On 2020-03-05 19:23, Stefan Karlsson wrote:
> You used std::numeric_limits<T>::min() in the test. There's a comment in 
> powerOfTwo.hpp stating:
> 
> // Helper function to get the maximum positive value. Implemented here
> // since using std::numeric_limits<T>::max() seems problematic on some
> // platforms.
> 
> I don't know if this means that min() is problematic as well?

Me neither, but I think for test use it should be fine. There are some
uses of ::max() in a few gtests already.

For context I saw Solaris slowdebug builds fail when implementing
powerOfTwo with use of std::numeric_limits. The build issue appeared
only in slowdebug builds when numeric_limits was used in code that get
heavily inlined, causing some linking issue.

/Claes

From stefan.karlsson at oracle.com  Thu Mar  5 19:46:00 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Thu, 5 Mar 2020 20:46:00 +0100
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
 <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
Message-ID: <e96653f2-3f5d-42cc-6d5e-8199a3caf8e8@oracle.com>

I ran with this patch:
diff --git a/src/hotspot/share/utilities/powerOfTwo.hpp 
b/src/hotspot/share/utilities/powerOfTwo.hpp
--- a/src/hotspot/share/utilities/powerOfTwo.hpp
+++ b/src/hotspot/share/utilities/powerOfTwo.hpp
@@ -36,6 +36,7 @@

 ?template <typename T>
 ?bool is_power_of_2(T x) {
+? assert(sizeof(x) != 8 || !IsSigned<T>::value || x != 
std::numeric_limits<T>::min(), "Caught min value");
 ?? return (x > T(0)) && ((x & (x - 1)) == T(0));
 ?}

and it asserts in DaCapo in immL_Pow2 below:

0x00007f33727e26ea in State::_sub_Op_ConL (this=0x7f3340842f60, 
n=0x7f336cbb04b8) at src/hotspot/cpu/x86/x86_64.ad:3123

operand immL_Pow2()
%{
 ? predicate(is_power_of_2(n->get_long()));
 ? match(ConL);

 ? op_cost(15);
 ? format %{ %}
 ? interface(CONST_INTER);
%}

operand immL_NotPow2()
%{
 ? predicate(is_power_of_2(~n->get_long()));
 ? match(ConL);

 ? op_cost(15);
 ? format %{ %}
 ? interface(CONST_INTER);
%}

The predicate for Pow2 and NotPow2 will both be false because of this 
change. Is that a problem?

StefanK

On 2020-03-05 19:23, Stefan Karlsson wrote:
> Hi Andrew,
>
> The implementation of is_power_of_2 looks good to me.
>
> Since this changes the value of is_power_of_2(LONG_MIN), did you try 
> to figure out if any of the existing code relies on this returning 
> true? It might be prudent to run this patch through testing with an 
> assert catching any places where LONG_MIN is used as the argument. I 
> can run that on our side, but need help for full platform coverage.
>
> You used std::numeric_limits<T>::min() in the test. There's a comment 
> in powerOfTwo.hpp stating:
>
> // Helper function to get the maximum positive value. Implemented here
> // since using std::numeric_limits<T>::max() seems problematic on some
> // platforms.
>
> I don't know if this means that min() is problematic as well?
>
> Thanks for fixing this.
>
> StefanK
>
> On 2020-03-05 19:06, Andrew Haley wrote:
>> is_power_of_2() has always behaved oddly, and it still behaves oddly
>> but in a different way since 8183574: Unify the is_power_of_2
>> functions. And this change breaks TCK on AArch64 systems.
>>
>> Previously, is_power_of_2(long) returned true for LONG_MIN, the most
>> -ve signed long, but returned false for INT_MIN, the most -ve signed
>> int.
>>
>> Also, is_power_of_2(long) had Undefined Behaviour (UB) for LONG_MIN:
>> it overflowed from the most -ve long to the most +ve long.
>>
>> Now, is_power_of_2() has Undefined Behaviour for both int and long
>> arguments when applied to their respective most negative values. In
>> testing it returns true for both, but as this is UB it is not
>> guaranteed.
>>
>> previous definition:
>>
>> inline bool is_power_of_2(intptr_t x) {
>> ?? return ((x != NoBits) && (mask_bits(x, x - 1) == NoBits));
>> }
>>
>> since JDK-8240615:
>>
>> template <typename T>
>> bool is_power_of_2(T x) {
>> ?? return (x != T(0)) && ((x & (x - 1)) == T(0));
>> }
>>
>> I do not believe that this behavioural change was deliberate.
>>
>> I suggest that we should change the definition of is_power_of_2() for
>> all signed types such that it returns false for negative
>> arguments. This makes the behaviour consistent and also avoids
>> undefined behaviour.
>>
>> This is important because is_power_of_2 is sometimes used as a guard
>> in ADfile patterns:
>>
>> instruct cmpI_branch_bit(cmpOpEqNe cmp, iRegIorL2I op1, immI op2, 
>> immI0 op3, label labl) %{
>> ?? match(If cmp (CmpI (AndI op1 op2) op3));
>> predicate(is_power_of_2(n->in(2)->in(1)->in(2)->get_int()));
>> ?? effect(USE labl);
>>
>> ?? ins_cost(BRANCH_COST);
>> ?? format %{ "tb$cmp $op1, $op2, $labl" %}
>> ?? ins_encode %{
>> ???? Label* L = $labl$$label;
>> ???? Assembler::Condition cond = (Assembler::Condition)$cmp$$cmpcode;
>> ???? int bit = exact_log2($op2$$constant);
>> ??? ...
>>
>> This has a runtime error since JDK-8240615 because the initial
>> is_power_of_2 test succeeds, but the later exact_log2
>> fails. exact_log2() takes an intptr_t, not an int. On 64-bit systems,
>> intptr_t is a 64-bit signed type, so the first call to is_power_of_2
>> succeeds, whereas exact_log2 fails because its arg is sign-extended
>> to 64 bits.
>>
>> http://cr.openjdk.java.net/~aph/8240615-1/
>>
>


From sgehwolf at redhat.com  Fri Mar  6 13:06:46 2020
From: sgehwolf at redhat.com (Severin Gehwolf)
Date: Fri, 06 Mar 2020 14:06:46 +0100
Subject: PING? RFR(S): 8239785: Cgroups: Incorrect detection logic on
 old systems in hotspot
In-Reply-To: <C81DB191-91BF-4ED6-81E1-51605FAE6693@oracle.com>
References: <362678e8bc8231c205c6b3327d08dd5179154e23.camel@redhat.com>
 <2A934185-9B81-4371-B985-F0C3C39B3FD3@oracle.com>
 <215e52601b2da8600cbfd47ef73634720a7736d2.camel@redhat.com>
 <EF45A2BB-360E-4767-B542-B420BF76FD2E@oracle.com>
 <d1683592b67f99e7625713d2db2c71641d50a738.camel@redhat.com>
 <1B61ACE7-AFB9-458C-B6AB-465259BD31C7@oracle.com>
 <264b4edf94d60a3256a9ff8d5de6a078c3a0819b.camel@redhat.com>
 <A6EE0016-E33B-451D-BF0F-5F9C13A4DB32@oracle.com>
 <94c06270838cb3c090a227f2688a5c3f0aade64c.camel@redhat.com>
 <C81DB191-91BF-4ED6-81E1-51605FAE6693@oracle.com>
Message-ID: <9376bf1ea972c00f8983db018a66cf7578a4a60c.camel@redhat.com>

Could I ask for a second *R*eview of this please? Bob Vandette already reviewed it.

webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/

Thanks,
Severin

On Thu, 2020-03-05 at 11:13 -0500, Bob Vandette wrote:
> > On Mar 5, 2020, at 11:00 AM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
> > 
> > Hi Bob,
> > 
> > On Thu, 2020-03-05 at 08:41 -0500, Bob Vandette wrote:
> > > > On Mar 5, 2020, at 4:36 AM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
> > > > 
> > > > Hi Bob,
> > > > 
> > > > On Wed, 2020-03-04 at 15:39 -0500, Bob Vandette wrote:
> > > > > > On Mar 4, 2020, at 1:59 PM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
> > > > > > 
> > > > > > Hi Bob,
> > > > > > 
> > > > > > Updated webrev:
> > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/
> > > > > > 
> > > > > > Testing: hotspot docker tests on Linux fastdebug/release cgroup v1 and
> > > > > > cgroup v2. New regression test. jdk submit. All pass.
> > > > > > 
> > > > > > On Fri, 2020-02-28 at 15:08 -0500, Bob Vandette wrote:
> > > > > > > It still looks like the _mount_path can be corrupted if both cgroupv1 and cgroupv2 are mounted.
> > > > > > > You?ll only initialize the _mount_path if cgroupv2 is detected in the cgroups file but if cgroupv1
> > > > > > > is also mounted, you?ll still clobber the _mount_path.
> > > > > > 
> > > > > > How so? When we start processing /proc/self/mountinfo on line 170, then
> > > > > > we have already looked at /proc/cgroups, thus is_cgroupsV2 is set to
> > > > > > true (A) or false (B).
> > > > > > 
> > > > > > Cases A:
> > > > > > 1. We are on an old system with *no* cgroup controllers (v1 or v2)
> > > > > >  mounted (this bug). We are fine to continue as no cgroups entry
> > > > > >  will be found in /proc/self/mountinfo. Thus, no clobbering.
> > > > > > 2. We are on a true unified hierarchy only system. In that case
> > > > > >  the sscanf for v1 would match (early match), but 'memory' et.al
> > > > > >  controllers would not be found. Thus, no clobbering.
> > > > > > 
> > > > > > Cases B:
> > > > > > 1. Hybrid hierarchy: controllers mounted via cgroup v1, but
> > > > > >  cgroup v2 mounted in addition for system use. is_cgroupV2 will
> > > > > >  prevent the cgroup v2 controller path lookup logic to be entered.
> > > > > >  No clobbering.
> > > > > > 2. cgroup v1 only hierarchy (legacy): Same logic applies. V2 setting
> > > > > >  of mount paths won't be entered since is_cgroupV2 == false.
> > > > > >  No clobbering.
> > > > > > 
> > > > > > What am I missing? FWIW, I've played around with the idea of keeping
> > > > > > cgroup v1 and cgroup v2 mount paths separate[1]. If that's preferred,
> > > > > > happy to go this route.
> > > > > 
> > > > > The case that I think is a problem is ?
> > > > > 
> > > > > cgroupv2 is detected in /proc/cgroups so is_cgroupv2 is true
> > > > > both cgroupv1 and cgroupv2 devices are mounted
> > > > 
> > > > That's not possible. is_cgroupv2 cannot be true in that case.
> > > > 
> > > > > At line cgroupSubsystem_linux.cpp:292 in your new webrev, you process the cgroupv1
> > > > > devices and clobber _mount_path.
> > > > > 
> > > > > What keeps this from occurring?
> > > > 
> > > > In that case 'is_cgroupv2 == false', because having any cgroups v1
> > > > controllers mounted will result in some non-zero hierarchy ids in
> > > > /proc/cgroups. That in turn will set is_cgroup2 == false.
> > > > 
> > > > From man 7 cgroups:
> > > > 
> > > > """"
> > > >  /proc files
> > > >      /proc/cgroups (since Linux 2.6.24)
> > > > 
> > > >             The fields in this file are, from left to right:
> > > > 
> > > >             1. The name of the controller.
> > > > 
> > > >             2. The unique ID of the cgroup hierarchy on which this controller is mounted.  If multiple cgroups v1 controllers are bound to the same hierarchy, then each will show the same hierarchy ID in
> > > >                this field.  The value in this field will be 0 if:
> > > > 
> > > >                  a) the controller is not mounted on a cgroups v1 hierarchy;
> > > > 
> > > >                  b) the controller is bound to the cgroups v2 single unified hierarchy; or
> > > > 
> > > >                  c) the controller is disabled (see below).
> > > > """
> > > > 
> > > > The only way hierarchy id == 0 for a controller on cgroups v1 is a).
> > > > It's never 0 when cgroup v1 and cgroup v2 are *both* mounted (hybrid).
> > > > One example of such a /proc/cgroups is on my F30 workstation (hybrid):
> > > > 
> > > > $ cat /proc/cgroups 
> > > > #subsys_name	hierarchy	num_cgroups	enabled
> > > > cpuset	5	1	1
> > > > cpu	3	1	1
> > > > cpuacct	3	1	1
> > > > blkio	11	1	1
> > > > memory	6	136	1
> > > > devices	9	75	1
> > > > freezer	7	1	1
> > > > net_cls	4	1	1
> > > > perf_event	2	1	1
> > > > net_prio	4	1	1
> > > > hugetlb	10	1	1
> > > > pids	8	81	1
> > > > 
> > > > And the corresponding mountinfo:
> > > > 
> > > > $ grep cgroup /proc/self/mountinfo 
> > > > 30 23 0:26 / /sys/fs/cgroup ro,nosuid,nodev,noexec shared:4 - tmpfs tmpfs ro,seclabel,mode=755
> > > > 31 30 0:27 / /sys/fs/cgroup/unified rw,nosuid,nodev,noexec,relatime shared:5 - cgroup2 cgroup2 rw,seclabel,nsdelegate
> > > > 32 30 0:28 / /sys/fs/cgroup/systemd rw,nosuid,nodev,noexec,relatime shared:6 - cgroup cgroup rw,seclabel,xattr,name=systemd
> > > > 35 30 0:31 / /sys/fs/cgroup/perf_event rw,nosuid,nodev,noexec,relatime shared:7 - cgroup cgroup rw,seclabel,perf_event
> > > > 36 30 0:32 / /sys/fs/cgroup/cpu,cpuacct rw,nosuid,nodev,noexec,relatime shared:8 - cgroup cgroup rw,seclabel,cpu,cpuacct
> > > > 37 30 0:33 / /sys/fs/cgroup/net_cls,net_prio rw,nosuid,nodev,noexec,relatime shared:9 - cgroup cgroup rw,seclabel,net_cls,net_prio
> > > > 38 30 0:34 / /sys/fs/cgroup/cpuset rw,nosuid,nodev,noexec,relatime shared:10 - cgroup cgroup rw,seclabel,cpuset
> > > > 39 30 0:35 / /sys/fs/cgroup/memory rw,nosuid,nodev,noexec,relatime shared:11 - cgroup cgroup rw,seclabel,memory
> > > > 40 30 0:36 / /sys/fs/cgroup/freezer rw,nosuid,nodev,noexec,relatime shared:12 - cgroup cgroup rw,seclabel,freezer
> > > > 41 30 0:37 / /sys/fs/cgroup/pids rw,nosuid,nodev,noexec,relatime shared:13 - cgroup cgroup rw,seclabel,pids
> > > > 42 30 0:38 / /sys/fs/cgroup/devices rw,nosuid,nodev,noexec,relatime shared:14 - cgroup cgroup rw,seclabel,devices
> > > > 43 30 0:39 / /sys/fs/cgroup/hugetlb rw,nosuid,nodev,noexec,relatime shared:15 - cgroup cgroup rw,seclabel,hugetlb
> > > > 44 30 0:40 / /sys/fs/cgroup/blkio rw,nosuid,nodev,noexec,relatime shared:16 - cgroup cgroup rw,seclabel,blkio
> > > > 
> > > > Does that make sense?
> > > 
> > > Ok, that makes sense.
> > > 
> > > Thanks for the clarification.
> > 
> > Anytime. Can I put you down as a reviewer (the non-census one)?
> 
> Yes, absolutely.
> 
> Bob.
> 
> > Thanks,
> > Severin
> > 
> > > Bob.
> > > 
> > > > Thanks,
> > > > Severin
> > > > 
> > > > > > While thinking about this issue here, I've decided to actually write a
> > > > > > test (using WhiteBox) so that all these cases are covered and will
> > > > > > continue to work. Running the test, CgroupSubsystemFactory, with
> > > > > > fastdebug proves that there is no clobbering if that matters. It's also
> > > > > > a way to reproduce the issue. So we have a regression test now too
> > > > > > (testCgroupv1NoMounts). Line 325-332 in [2] is the actual fix. Without
> > > > > > it, the test fails.
> > > > > > 
> > > > > > > Maybe you should just do the cgroupv2 processing if /proc/cgroups has
> > > > > > > cgroupv2 enabled and if that fails continue looking for cgroup v1
> > > > > > > mounts.
> > > > > > 
> > > > > > FWIW, I've amended the cgroup v1 logic to also check for the file
> > > > > > system type. Which needs to be 'cgroup' (not 'cgroup2') for cgroup v1.
> > > > > > This isn't strictly needed but more accurately reflects the intent IMO.
> > > > > 
> > > > > Sounds good.
> > > > > 
> > > > > > > Do you really need to do this?  The BSS section should be zeroed.
> > > > > > > 
> > > > > > > 55   for (int i = 0; i < CG_INFO_LENGTH; i++) {
> > > > > > > 56     cg_infos[i]._name = NULL;
> > > > > > > 57     cg_infos[i]._cgroup_path = NULL;
> > > > > > > 58     cg_infos[i]._root_mount_path = NULL;
> > > > > > > 59     cg_infos[i]._mount_path = NULL;
> > > > > > > 60   }
> > > > > > 
> > > > > > I believe so. CgroupInfo array is stack allocated and we should
> > > > > > initialize values properly. On the other hand, we should do that in the
> > > > > > constructor. Done now.
> > > > > 
> > > > > You are correct.  I was only looking at the diffs and not the full source.
> > > > > 
> > > > > > > You don?t need to check for NULL in cleanup, os::free can handle NULL
> > > > > > > pointers.
> > > > > > > 
> > > > > > > 339 void CgroupSubsystemFactory::cleanup(CgroupInfo* cg_infos) {
> > > > > > > 340   assert(cg_infos != NULL, "Invariant");
> > > > > > > 341   for (int i = 0; i < CG_INFO_LENGTH; i++) {
> > > > > > > 342     if (cg_infos[i]._name != NULL) {
> > > > > > > 343       os::free(cg_infos[i]._name);
> > > > > > > 344     }
> > > > > > > 345     if (cg_infos[i]._cgroup_path != NULL) {
> > > > > > > 346       os::free(cg_infos[i]._cgroup_path);
> > > > > > > 347     }
> > > > > > > 348     if (cg_infos[i]._root_mount_path != NULL) {
> > > > > > > 349       os::free(cg_infos[i]._root_mount_path);
> > > > > > > 350     }
> > > > > > > 351     if (cg_infos[i]._mount_path != NULL) {
> > > > > > > 352       os::free(cg_infos[i]._mount_path);
> > > > > > > 353     }
> > > > > > > 354   }
> > > > > > > 355 }
> > > > > > 
> > > > > > Fixed.
> > > > > > 
> > > > > > Let me know what you think about the updated webrev.
> > > > > > 
> > > > > > Thanks,
> > > > > > Severin
> > > > > > 
> > > > > > [1] http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/02/webrev/
> > > > > > [2] http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/src/hotspot/os/linux/cgroupSubsystem_linux.cpp.html
> > > > > > 
> > > > > > > Bob.
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > > On Feb 28, 2020, at 11:42 AM, Severin Gehwolf <
> > > > > > > > sgehwolf at redhat.com>
> > > > > > > > wrote:
> > > > > > > > 
> > > > > > > > Hi Bob,
> > > > > > > > 
> > > > > > > > Thanks for the review!
> > > > > > > > 
> > > > > > > > On Tue, 2020-02-25 at 15:08 -0500, Bob Vandette wrote:
> > > > > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/01/webrev/src/hotspot/os/linux/cgroupSubsystem_linux.cpp.sdiff.html
> > > > > > > > > 
> > > > > > > > > If both cgroupv1 and v2 controllers are mounted, you might
> > > > > > > > > have
> > > > > > > > > some problems.  You might want to only look for
> > > > > > > > > cgroup2 mounts if is_cgroupV2 is true otherwise only look for
> > > > > > > > > cgroup mounts.
> > > > > > > > 
> > > > > > > > OK.
> > > > > > > > 
> > > > > > > > > 194           cg_infos[i]._mount_path =
> > > > > > > > > os::strdup(tmp_mount_point);
> > > > > > > > > This will stomp the pointer:
> > > > > > > > > 213           cg_infos[memory_idx]._mount_path =
> > > > > > > > > os::strdup(tmpmount);
> > > > > > > > 
> > > > > > > > You are right. If we are on a hybrid hierarchy and for some
> > > > > > > > reason
> > > > > > > > the
> > > > > > > > cgroup v2 mount shows up *after* cgroup mounts in
> > > > > > > > /proc/self/mountinfo,
> > > > > > > > we'd get the mount path wrong. So ordering matters! The reason
> > > > > > > > this
> > > > > > > > worked for me on hybrid is that cgroup2 mounts come *before*
> > > > > > > > cgroup
> > > > > > > > mounts on the system I've tested.
> > > > > > > > 
> > > > > > > > > You should free all strdup?d pointers in cg_infos in places
> > > > > > > > > where
> > > > > > > > > you fail and return NULL.
> > > > > > > > > 
> > > > > > > > > 81       cg_infos[memory_idx]._name = os::strdup(name);
> > > > > > > > > 213           cg_infos[memory_idx]._mount_path =
> > > > > > > > > os::strdup(tmpmount);
> > > > > > > > > 214           cg_infos[memory_idx]._root_mount_path =
> > > > > > > > > os::strdup(tmproot);
> > > > > > > > 
> > > > > > > > Done now. More thoughts?
> > > > > > > > 
> > > > > > > > Updated webrev: 
> > > > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/03/webrev/
> > > > > > > > 
> > > > > > > > Thanks,
> > > > > > > > Severin
> > > > > > > > 
> > > > > > > > > > On Feb 25, 2020, at 2:22 PM, Severin Gehwolf <
> > > > > > > > > > sgehwolf at redhat.com> wrote:
> > > > > > > > > > 
> > > > > > > > > > Hi,
> > > > > > > > > > 
> > > > > > > > > > Could I please get reviews of this cgroup-related patch?
> > > > > > > > > > After
> > > > > > > > > > JDK-
> > > > > > > > > > 8230305 old systems with no mounted cgroup controllers
> > > > > > > > > > would
> > > > > > > > > > get
> > > > > > > > > > detected as cgroups v2 (wrong). Then, when initializing the
> > > > > > > > > > v2
> > > > > > > > > > subsystem, it would fail. The trace about cgroupv2 mount
> > > > > > > > > > point
> > > > > > > > > > not
> > > > > > > > > > found is misleading in this case. While the outcome is
> > > > > > > > > > similar
> > > > > > > > > > pre/post
> > > > > > > > > > patch (NULL cgroup subsystem), I'd like to be explicit
> > > > > > > > > > about
> > > > > > > > > > this case.
> > > > > > > > > > 
> > > > > > > > > > The suggested fix is to look at /proc/self/mountinfo in
> > > > > > > > > > order
> > > > > > > > > > to
> > > > > > > > > > correctly detect whether this is cgroup v2 or cgroup v1
> > > > > > > > > > with no
> > > > > > > > > > mounted
> > > > > > > > > > controllers. In the latter case we just stop initialization
> > > > > > > > > > as
> > > > > > > > > > we'd
> > > > > > > > > > fail later in cgroupv1 code anyway. This aligns hotspot
> > > > > > > > > > code
> > > > > > > > > > with core-
> > > > > > > > > > libs after JDK-8239559.
> > > > > > > > > > 
> > > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8239785
> > > > > > > > > > webrev: 
> > > > > > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/01/webrev/
> > > > > > > > > > 
> > > > > > > > > > Testing: jdk-submit, hotspot docker tests on cgroup v1 and
> > > > > > > > > > cgroup v2. All pass.
> > > > > > > > > > 
> > > > > > > > > > Thoughts?
> > > > > > > > > > 
> > > > > > > > > > Thanks,
> > > > > > > > > > Severin
> > > > > > > > > > 


From aph at redhat.com  Fri Mar  6 14:48:42 2020
From: aph at redhat.com (Andrew Haley)
Date: Fri, 6 Mar 2020 14:48:42 +0000
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <e96653f2-3f5d-42cc-6d5e-8199a3caf8e8@oracle.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
 <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
 <e96653f2-3f5d-42cc-6d5e-8199a3caf8e8@oracle.com>
Message-ID: <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com>

On 3/5/20 7:46 PM, Stefan Karlsson wrote:
> I ran with this patch:
> diff --git a/src/hotspot/share/utilities/powerOfTwo.hpp 
> b/src/hotspot/share/utilities/powerOfTwo.hpp
> --- a/src/hotspot/share/utilities/powerOfTwo.hpp
> +++ b/src/hotspot/share/utilities/powerOfTwo.hpp
> @@ -36,6 +36,7 @@
> 
>  ?template <typename T>
>  ?bool is_power_of_2(T x) {
> +? assert(sizeof(x) != 8 || !IsSigned<T>::value || x != 
> std::numeric_limits<T>::min(), "Caught min value");
>  ?? return (x > T(0)) && ((x & (x - 1)) == T(0));
>  ?}
> 
> and it asserts in DaCapo in immL_Pow2 below:
> 
> 0x00007f33727e26ea in State::_sub_Op_ConL (this=0x7f3340842f60, 
> n=0x7f336cbb04b8) at src/hotspot/cpu/x86/x86_64.ad:3123
> 
> operand immL_Pow2()
> %{
>  ? predicate(is_power_of_2(n->get_long()));
>  ? match(ConL);
> 
>  ? op_cost(15);
>  ? format %{ %}
>  ? interface(CONST_INTER);
> %}
> 
> operand immL_NotPow2()
> %{
>  ? predicate(is_power_of_2(~n->get_long()));
>  ? match(ConL);
> 
>  ? op_cost(15);
>  ? format %{ %}
>  ? interface(CONST_INTER);
> %}
> 
> The predicate for Pow2 and NotPow2 will both be false because of this 
> change. Is that a problem?

Probably. I'll do some more digging.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From aph at redhat.com  Fri Mar  6 17:11:20 2020
From: aph at redhat.com (Andrew Haley)
Date: Fri, 6 Mar 2020 17:11:20 +0000
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
 <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
 <e96653f2-3f5d-42cc-6d5e-8199a3caf8e8@oracle.com>
 <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com>
Message-ID: <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com>

On 3/6/20 2:48 PM, Andrew Haley wrote:
>> The predicate for Pow2 and NotPow2 will both be false because of this
>> change. Is that a problem?
> Probably. I'll do some more digging.

The pattern genertates a bitset instruction. It'll generate slightly
less efficient code after my change. Maybe.

It's pretty damm hard to persuade C2 actually to use this pattern, but
this works:

    @Benchmark
    public void test_btsL_mem_imm(BenchmarkState state) {
        for (int i = 0; i < 1000; i += 3) {
            state.array[i] |= 0x4000_0000_0000_0000l;
            state.array[i+1] |= 0x8000_0000_0000_0000l;
            state.array[i+2] |= 0x1000_0000_0000_0000l;
        }
    }

These three |= ops turn into:

  0x00007fe65cc58f40:   bts    QWORD PTR [r8+r10*8+0x10],0x3e;*lastore {reexecute=0 rethrow=0 return_oop=0}
  0x00007fe65cc58f47:   or     QWORD PTR [r8+r10*8+0x18],rdi;*lastore {reexecute=0 rethrow=0 return_oop=0}
  0x00007fe65cc58f4c:   bts    QWORD PTR [r8+r10*8+0x20],0x3c;*lastore {reexecute=0 rethrow=0 return_oop=0}

There is a very slight pessimization in that RDI is now occupied, so
the register pressure is slightly increased, and there's the cost of a
64-bit load immediate operation.

We can get back to the status quo with this patch:

--- a/src/hotspot/cpu/x86/x86_64.ad     Thu Mar 05 17:56:08 2020 +0000
+++ b/src/hotspot/cpu/x86/x86_64.ad     Fri Mar 06 17:06:20 2020 +0000
@@ -3120,7 +3120,7 @@

 operand immL_Pow2()
 %{
-  predicate(is_power_of_2(n->get_long()));
+  predicate(is_power_of_2((julong)n->get_long()));
   match(ConL);

   op_cost(15);
@@ -3130,7 +3130,7 @@

 operand immL_NotPow2()
 %{
-  predicate(is_power_of_2(~n->get_long()));
+  predicate(is_power_of_2((julong)~n->get_long()));
   match(ConL);

   op_cost(15);

  0x00007fc688bf8010:   bts    QWORD PTR [r10+r8*8+0x10],0x3e;*lastore {reexecute=0 rethrow=0 return_oop=0}
  0x00007fc688bf8017:   bts    QWORD PTR [r10+r8*8+0x18],0x3f;*lastore {reexecute=0 rethrow=0 return_oop=0}
  0x00007fc688bf801e:   bts    QWORD PTR [r10+r8*8+0x20],0x3c;*lastore {reexecute=0 rethrow=0 return_oop=0}

There will be more examples of this, stuff that just works "by
accident". If we want the previous weird behaviour we could explictly
special-case that behaviour and test it. I'd rather not...

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From vladimir.kozlov at oracle.com  Fri Mar  6 17:51:20 2020
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Fri, 6 Mar 2020 09:51:20 -0800
Subject: RFR: 8235908: omit ThreadPriorityPolicy warning when value is set
 from image
In-Reply-To: <48754173-0e4e-736e-6895-6d3051225c7c@oracle.com>
References: <48754173-0e4e-736e-6895-6d3051225c7c@oracle.com>
Message-ID: <c29731b1-4b29-c4c9-8532-b63b1485a976@oracle.com>

I changed mailing list to wider audience.

Note for reviews, Java SE builds and products are not affected by this change.

This change affects only GraalVM which sets ThreadPriorityPolicy=1 by default in their build.
GraalVM release notes have statement about that to notify GraalVM users because it may affect applications performance.

RFE describes why GraalVM needs that.

Change looks good to me.

Thanks,
Vladimir K

On 3/6/20 12:56 AM, Matthias Neugschwandtner wrote:
> Hello everyone,
> 
> Please review this change, which omits the warning that ThreadPriorityPolicy=1 requires system level permission if the 
> value is set via jlink, the same way it is omitted if the value is set by default.
> 
> https://bugs.openjdk.java.net/browse/JDK-8235908
> http://cr.openjdk.java.net/~mneugschwand/8235908
> 
> Thank you,
> Matthias

From john.r.rose at oracle.com  Fri Mar  6 19:48:58 2020
From: john.r.rose at oracle.com (John Rose)
Date: Fri, 6 Mar 2020 11:48:58 -0800
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
 <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
 <e96653f2-3f5d-42cc-6d5e-8199a3caf8e8@oracle.com>
 <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com>
 <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com>
Message-ID: <F9E331DA-B9FC-40E0-81FD-85CC25F45FD0@oracle.com>

On Mar 6, 2020, at 9:11 AM, Andrew Haley <aph at redhat.com> wrote:
> 
> +  predicate(is_power_of_2((julong)n->get_long()));

To me this is an easy ?yes?.  The hardware instruction in question
uses the unsigned convention.

The harder part is (as you say) figuring out where we were implicitly
relying on unsigned conventions and behavior will change.  But I
think we?ve had plenty of eyeballs on this, and it?s easy to fix more
stuff like this if we find it later, right?

? John

From aph at redhat.com  Sat Mar  7 10:54:32 2020
From: aph at redhat.com (Andrew Haley)
Date: Sat, 7 Mar 2020 10:54:32 +0000
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <F9E331DA-B9FC-40E0-81FD-85CC25F45FD0@oracle.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
 <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
 <e96653f2-3f5d-42cc-6d5e-8199a3caf8e8@oracle.com>
 <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com>
 <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com>
 <F9E331DA-B9FC-40E0-81FD-85CC25F45FD0@oracle.com>
Message-ID: <b3fbc9fc-ec01-3b1c-7e63-6008d5281895@redhat.com>

On 3/6/20 7:48 PM, John Rose wrote:
> On Mar 6, 2020, at 9:11 AM, Andrew Haley <aph at redhat.com> wrote:
>>
>> +  predicate(is_power_of_2((julong)n->get_long()));
>
> To me this is an easy ?yes?.  The hardware instruction in question
> uses the unsigned convention.
>
> The harder part is (as you say) figuring out where we were implicitly
> relying on unsigned conventions and behavior will change.  But I
> think we?ve had plenty of eyeballs on this, and it?s easy to fix more
> stuff like this if we find it later, right?

I think we'll have to do that.

For the record:

This is a Hard Problem, for a reason that is not obvious.
In C2, you often have patterns like this:

  match(If cmp (CmpI (AndI op1 op2) op3));
  predicate(is_power_of_2(n->in(2)->in(1)->in(2)->get_int()));
  ...
  ins_encode %{
    ...
    int bit = exact_log2($op2$$constant);

Very unfortunately, (n->in(2)->in(1)->in(2)->get_int()) is a jint, but
$op2$$constant is an intptr_t. This is problematic if we're running on
a 64-bit machine because intptr_t is signed long.

At first I wanted to define exact_log2() and is_power_of_2() in a way
that treats all arguments as unsigned -- I wrote a patch which does
that -- but it doesn't help. Because of the above, even with a jint
operand representing 0x8000_0000, $op2$$constant returns
0xffff_ffff_8000_0000. This is a historic mistake in the design of C2,
IMO, but very hard to fix today.

So, AFAICS the only way to fix this is to treat all negative integer
values, whatever their size, as non-powers of 2.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From david.holmes at oracle.com  Mon Mar  9 04:33:09 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 9 Mar 2020 14:33:09 +1000
Subject: RFR: 8235908: omit ThreadPriorityPolicy warning when value is set
 from image
In-Reply-To: <c29731b1-4b29-c4c9-8532-b63b1485a976@oracle.com>
References: <48754173-0e4e-736e-6895-6d3051225c7c@oracle.com>
 <c29731b1-4b29-c4c9-8532-b63b1485a976@oracle.com>
Message-ID: <c74ffb25-2e2a-fb8f-01e8-085753525299@oracle.com>

The changes in relation to exposing the origin of the flag as a 
JIMAGE_RESOURCE look fine. (Could potentially have been its own RFE but 
that's just bookkeeping.)

No further comment on the actual ThreadPriorityPolicy change. :)

Cheers,
David

On 7/03/2020 3:51 am, Vladimir Kozlov wrote:
> I changed mailing list to wider audience.
> 
> Note for reviews, Java SE builds and products are not affected by this 
> change.
> 
> This change affects only GraalVM which sets ThreadPriorityPolicy=1 by 
> default in their build.
> GraalVM release notes have statement about that to notify GraalVM users 
> because it may affect applications performance.
> 
> RFE describes why GraalVM needs that.
> 
> Change looks good to me.
> 
> Thanks,
> Vladimir K
> 
> On 3/6/20 12:56 AM, Matthias Neugschwandtner wrote:
>> Hello everyone,
>>
>> Please review this change, which omits the warning that 
>> ThreadPriorityPolicy=1 requires system level permission if the value 
>> is set via jlink, the same way it is omitted if the value is set by 
>> default.
>>
>> https://bugs.openjdk.java.net/browse/JDK-8235908
>> http://cr.openjdk.java.net/~mneugschwand/8235908
>>
>> Thank you,
>> Matthias

From david.holmes at oracle.com  Mon Mar  9 05:03:14 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 9 Mar 2020 15:03:14 +1000
Subject: RFR: 8240222: [TESTBUG] gtest/jfr/test_networkUtilization.cpp
 failed when the number of tests is greater than or equal to 2
In-Reply-To: <4f6c2613-de5a-cc28-f208-1a763270a5d0@loongson.cn>
References: <64960cc9-5021-d265-ab30-9cf9a91cc6e3@loongson.cn>
 <8f143c78-6bba-5f11-21ad-3096115287e1@oracle.com>
 <4f6c2613-de5a-cc28-f208-1a763270a5d0@loongson.cn>
Message-ID: <44483657-df1f-f9cc-9e6a-f73b0e602f72@oracle.com>

Hi Jia,

I see this is getting no response so have cc'd hotspot-jfr-dev as they 
should have a better idea of how this gtest can and cannot be used.

Just because the GTest framework supports "REPEAT" it doesn't mean all 
tests can, or should, be repeatable.

Cheers,
David

On 29/02/2020 1:53 pm, Jia Huang wrote:
> Hi David,
> 
> ? 2020/2/28 21:16, David Holmes ??:
>> Hi Jia,
>>
>> On 28/02/2020 11:08 pm, Jia Huang wrote:
>>> Hi all,
>>>
>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8240222
>>> Webrev: http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/
>>>
>>> gtest/jfr/test_networkUtilization.cpp failed when the number of tests 
>>> is greater than or equal to 2.
>>> As shown in the following code, the test failed because "traceid id" 
>>> is a static local variable.
>>> When the test runs a second time, "id" continues to grow, resulting 
>>> in the return value of "i" being "_interfaces.end()".
>>
>> Interesting! I'm not at all familiar with the details of gtest and how 
>> it actually runs things. But I would expect that we have many tests 
>> that rely on starting from a cleanly initialized VM state, or test 
>> state, and so can't simply be repeated (it would be like editing an 
>> arbitrary test and putting the body of main into a for-loop!).
> 
> I'm sorry that my last description may be a little incomplete.
> JfrTestNetworkUtilization is a group of tests,it includes four 
> individual tests:
>  ? --JfrTestNetworkUtilization.RequestFunctionBasic_test_vm
>  ? --JfrTestNetworkUtilization.RequestFunctionMultiple_test_vm
>  ? --JfrTestNetworkUtilization.InterfaceRemoved_test_vm
>  ? --JfrTestNetworkUtilization.InterfaceReset_test_vm
> 
> If you only run a individual test once, such as the following Testing, 
> it still fails.
> The reason for the failure, as I said last time, is also because of 
> static local variables.
> In the "RequestFunctionMultiple_test_vm" test, the sequence number of 
> "eth0" is 2.
> However, the initial value of the static local variable "id" is 0.
> 
> Testting:
>  ?? --make run-test 
> TEST=gtest:JfrTestNetworkUtilization.RequestFunctionMultiple_test_vm 
> GTEST=REPEAT=1 CONF=release
>  ?? --make run-test 
> TEST=gtest:JfrTestNetworkUtilization.InterfaceRemoved_test_vm 
> GTEST=REPEAT=1 CONF=release
>  ?? --make run-test 
> TEST=gtest:JfrTestNetworkUtilization.InterfaceReset_test_vm 
> GTEST=REPEAT=1 CONF=release
> 
>>
>> So I'm not sure what the right way to fix this is:
>> - don't use REPEAT?
>> - declare the test as not repeatable somehow?
>> - change the test to make it repeatable?
> 
> In the doc/testing.md or doc/testing.html, Gtest keywords inclue "REPEAT",
> so I think this means that all GTEST tests should be able to use the 
> "REPEAT parameter.
> 
> 
> Best regards,
> Jia
> 
> 
>>
>> Cheers,
>> David
>>
>>> ---------------------------------------------------------------------------- 
>>>
>>> test/hotspot/gtest/jfr/test_networkUtilization.cpp
>>>
>>> static const MockNetworkInterface& get_interface(traceid id) {
>>> ?? std::list<MockNetworkInterface>::const_iterator i = 
>>> _interfaces.begin();
>>> ?? for (; i != _interfaces.end(); ++i) {
>>> ???? if (i->id == id) {
>>> ?????? break;
>>> ???? }
>>> ?? }
>>> ?? return *i;
>>> }
>>> ---------------------------------------------------------------------------- 
>>>
>>>
>>>
>>> Testing:
>>> ???? - make run-test TEST=gtest:JfrTestNetworkUtilization 
>>> GTEST=REPEAT=2 CONF=server-release
>>> ???? - make run-test TEST=gtest:JfrTestNetworkUtilization 
>>> GTEST=REPEAT=2 CONF=server-fastdebug
>>>
>>> Thanks a lot
>>>
>>> Best regards,
>>> Jia
>>>
> 

From david.holmes at oracle.com  Mon Mar  9 05:33:44 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 9 Mar 2020 15:33:44 +1000
Subject: RFR(L): JDK-8229517: Support for optional asynchronous/buffered
 logging
In-Reply-To: <77FA4BF1-DC2E-4AC8-987B-56578E5D33B9@amazon.com>
References: <77FA4BF1-DC2E-4AC8-987B-56578E5D33B9@amazon.com>
Message-ID: <614b4e2d-3d66-2731-c35b-53a3c96de71f@oracle.com>

Hi,

On 5/03/2020 8:04 pm, Liu, Xin wrote:
> Hi, hotspot reviewers,
> 
> Could you take a look at my webrev?
> Bug: https://bugs.openjdk.java.net/browse/JDK-8229517
> Webrev: https://cr.openjdk.java.net/~xliu/8229517/01/webrev/
> 
> Previously, we had discussion at runtime-dev. https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-February/038211.html
> I would like to continue it because I redo my implementation on the tip of jdk and have made it a general solution.

Not a review as such but a couple of comments/queries.

The async logging is done as a periodic task that will be executed the 
Watcher thread. This task requires a mutex that is also used by the 
other threads doing the log operations. So potentially the watcher 
thread could experience a significant delay waiting for the mutex.

I'm not clear what the actual lifecycle of the LogAsyncFlusher instance 
is. I would have expected a single instance initialized at VM startup 
and cleaned up at VM exit. The code gives the appearance that this can 
actually be much more dynamic with multiple instances coming and going 
through the lifetime of the VM. Deletion of the instance is particularly 
problematic due to the Mutex, which has to be guaranteed not to be 
accessed when it gets deleted.

> This webrev  provides <logging/asynclog.hpp> on apar with <log.hpp>. Even though it doesn?t touch any existing logging calls, it?s very possible to offload any callsite of logging to the asynchronous buffer.
> My rationale is that we only need to redirect some time-critical logs to asynchronous buffer to dodge unexpected filesystem delay.  Eg. g1gc has some loggings at safepoint. We mitigate those long-tailed GC latencies rooted by filesystem writing to gclog.

So this isn't an opt-in facility at runtime, but rather something that 
has to be decided a-priori by the developer.

How does this interact with the synchronous logging that may be used for 
a given set of log tags?

Thanks,
David
-----

> I got feedback from my our internal users. They have a concern about my approach of dynamic memory. Indeed, I gave up ringbuffer design in previous version and turn to use a dynamic allocated linkedlist.  If writing to filesystem stalls infinitely, JVM may implode because of memory usage.  I plan to solve it by adding a memory quota to the FIFO of logAsyncFlusher.hpp. It will start dropping early messages of  the FIFO if it has consumed too many free memories. Another potential improvement is changing the FIFO from mutex protected to lock-free.  It make sense if we redirect high volume of logging messages to it and mutex becomes hot.
> 
> Thanks,
> --lx
>   
> 
> 

From huangjia at loongson.cn  Mon Mar  9 07:29:29 2020
From: huangjia at loongson.cn (Jia Huang)
Date: Mon, 9 Mar 2020 15:29:29 +0800
Subject: RFR: 8240222: [TESTBUG] gtest/jfr/test_networkUtilization.cpp
 failed when the number of tests is greater than or equal to 2
In-Reply-To: <44483657-df1f-f9cc-9e6a-f73b0e602f72@oracle.com>
References: <64960cc9-5021-d265-ab30-9cf9a91cc6e3@loongson.cn>
 <8f143c78-6bba-5f11-21ad-3096115287e1@oracle.com>
 <4f6c2613-de5a-cc28-f208-1a763270a5d0@loongson.cn>
 <44483657-df1f-f9cc-9e6a-f73b0e602f72@oracle.com>
Message-ID: <931ecb2e-9898-a371-0aa9-bf463b9fdcf6@loongson.cn>

Haha, I'm also trying to email hotspot-jfr-dev.
Thank you, David.

? 2020/3/9 13:03, David Holmes ??:
> Hi Jia,
>
> I see this is getting no response so have cc'd hotspot-jfr-dev as they 
> should have a better idea of how this gtest can and cannot be used.
>
> Just because the GTest framework supports "REPEAT" it doesn't mean all 
> tests can, or should, be repeatable.
>
> Cheers,
> David
>
> On 29/02/2020 1:53 pm, Jia Huang wrote:
>> Hi David,
>>
>> ? 2020/2/28 21:16, David Holmes ??:
>>> Hi Jia,
>>>
>>> On 28/02/2020 11:08 pm, Jia Huang wrote:
>>>> Hi all,
>>>>
>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8240222
>>>> Webrev: http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/
>>>>
>>>> gtest/jfr/test_networkUtilization.cpp failed when the number of 
>>>> tests is greater than or equal to 2.
>>>> As shown in the following code, the test failed because "traceid 
>>>> id" is a static local variable.
>>>> When the test runs a second time, "id" continues to grow, resulting 
>>>> in the return value of "i" being "_interfaces.end()".
>>>
>>> Interesting! I'm not at all familiar with the details of gtest and 
>>> how it actually runs things. But I would expect that we have many 
>>> tests that rely on starting from a cleanly initialized VM state, or 
>>> test state, and so can't simply be repeated (it would be like 
>>> editing an arbitrary test and putting the body of main into a 
>>> for-loop!).
>>
>> I'm sorry that my last description may be a little incomplete.
>> JfrTestNetworkUtilization is a group of tests,it includes four 
>> individual tests:
>> ?? --JfrTestNetworkUtilization.RequestFunctionBasic_test_vm
>> ?? --JfrTestNetworkUtilization.RequestFunctionMultiple_test_vm
>> ?? --JfrTestNetworkUtilization.InterfaceRemoved_test_vm
>> ?? --JfrTestNetworkUtilization.InterfaceReset_test_vm
>>
>> If you only run a individual test once, such as the following 
>> Testing, it still fails.
>> The reason for the failure, as I said last time, is also because of 
>> static local variables.
>> In the "RequestFunctionMultiple_test_vm" test, the sequence number of 
>> "eth0" is 2.
>> However, the initial value of the static local variable "id" is 0.
>>
>> Testting:
>> ??? --make run-test 
>> TEST=gtest:JfrTestNetworkUtilization.RequestFunctionMultiple_test_vm 
>> GTEST=REPEAT=1 CONF=release
>> ??? --make run-test 
>> TEST=gtest:JfrTestNetworkUtilization.InterfaceRemoved_test_vm 
>> GTEST=REPEAT=1 CONF=release
>> ??? --make run-test 
>> TEST=gtest:JfrTestNetworkUtilization.InterfaceReset_test_vm 
>> GTEST=REPEAT=1 CONF=release
>>
>>>
>>> So I'm not sure what the right way to fix this is:
>>> - don't use REPEAT?
>>> - declare the test as not repeatable somehow?
>>> - change the test to make it repeatable?
>>
>> In the doc/testing.md or doc/testing.html, Gtest keywords inclue 
>> "REPEAT",
>> so I think this means that all GTEST tests should be able to use the 
>> "REPEAT parameter.
>>
>>
>> Best regards,
>> Jia
>>
>>
>>>
>>> Cheers,
>>> David
>>>
>>>> ---------------------------------------------------------------------------- 
>>>>
>>>> test/hotspot/gtest/jfr/test_networkUtilization.cpp
>>>>
>>>> static const MockNetworkInterface& get_interface(traceid id) {
>>>> ?? std::list<MockNetworkInterface>::const_iterator i = 
>>>> _interfaces.begin();
>>>> ?? for (; i != _interfaces.end(); ++i) {
>>>> ???? if (i->id == id) {
>>>> ?????? break;
>>>> ???? }
>>>> ?? }
>>>> ?? return *i;
>>>> }
>>>> ---------------------------------------------------------------------------- 
>>>>
>>>>
>>>>
>>>> Testing:
>>>> ???? - make run-test TEST=gtest:JfrTestNetworkUtilization 
>>>> GTEST=REPEAT=2 CONF=server-release
>>>> ???? - make run-test TEST=gtest:JfrTestNetworkUtilization 
>>>> GTEST=REPEAT=2 CONF=server-fastdebug
>>>>
>>>> Thanks a lot
>>>>
>>>> Best regards,
>>>> Jia
>>>>
>>


From christoph.langer at sap.com  Mon Mar  9 07:49:16 2020
From: christoph.langer at sap.com (Langer, Christoph)
Date: Mon, 9 Mar 2020 07:49:16 +0000
Subject: RFR: 8235908: omit ThreadPriorityPolicy warning when value is set
 from image
In-Reply-To: <c74ffb25-2e2a-fb8f-01e8-085753525299@oracle.com>
References: <48754173-0e4e-736e-6895-6d3051225c7c@oracle.com>
 <c29731b1-4b29-c4c9-8532-b63b1485a976@oracle.com>
 <c74ffb25-2e2a-fb8f-01e8-085753525299@oracle.com>
Message-ID: <AM0PR02MB57143DB95ECAC4BA617865FE8AFE0@AM0PR02MB5714.eurprd02.prod.outlook.com>

Hi,

the changes look good. However, I think we should make this behavior consistent and also handle Windows, Solaris and AIX the same way.

Cheers
Christoph

-----Original Message-----
From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of David Holmes
Sent: Montag, 9. M?rz 2020 05:33
To: Vladimir Kozlov <vladimir.kozlov at oracle.com>; Matthias Neugschwandtner <matthias.neugschwandtner at oracle.com>; 'hotspot-dev developers' <hotspot-dev at openjdk.java.net>
Subject: Re: RFR: 8235908: omit ThreadPriorityPolicy warning when value is set from image

The changes in relation to exposing the origin of the flag as a 
JIMAGE_RESOURCE look fine. (Could potentially have been its own RFE but 
that's just bookkeeping.)

No further comment on the actual ThreadPriorityPolicy change. :)

Cheers,
David

On 7/03/2020 3:51 am, Vladimir Kozlov wrote:
> I changed mailing list to wider audience.
> 
> Note for reviews, Java SE builds and products are not affected by this 
> change.
> 
> This change affects only GraalVM which sets ThreadPriorityPolicy=1 by 
> default in their build.
> GraalVM release notes have statement about that to notify GraalVM users 
> because it may affect applications performance.
> 
> RFE describes why GraalVM needs that.
> 
> Change looks good to me.
> 
> Thanks,
> Vladimir K
> 
> On 3/6/20 12:56 AM, Matthias Neugschwandtner wrote:
>> Hello everyone,
>>
>> Please review this change, which omits the warning that 
>> ThreadPriorityPolicy=1 requires system level permission if the value 
>> is set via jlink, the same way it is omitted if the value is set by 
>> default.
>>
>> https://bugs.openjdk.java.net/browse/JDK-8235908
>> http://cr.openjdk.java.net/~mneugschwand/8235908
>>
>> Thank you,
>> Matthias

From david.holmes at oracle.com  Mon Mar  9 08:56:52 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 9 Mar 2020 18:56:52 +1000
Subject: RFR: 8235908: omit ThreadPriorityPolicy warning when value is set
 from image
In-Reply-To: <AM0PR02MB57143DB95ECAC4BA617865FE8AFE0@AM0PR02MB5714.eurprd02.prod.outlook.com>
References: <48754173-0e4e-736e-6895-6d3051225c7c@oracle.com>
 <c29731b1-4b29-c4c9-8532-b63b1485a976@oracle.com>
 <c74ffb25-2e2a-fb8f-01e8-085753525299@oracle.com>
 <AM0PR02MB57143DB95ECAC4BA617865FE8AFE0@AM0PR02MB5714.eurprd02.prod.outlook.com>
Message-ID: <cd8d85de-feca-363d-4e00-7d257db7b1d0@oracle.com>

Hi Christoph,

On 9/03/2020 5:49 pm, Langer, Christoph wrote:
> Hi,
> 
> the changes look good. However, I think we should make this behavior consistent and also handle Windows, Solaris and AIX the same way.

The warning is primarily a Linux thing that has been copy'n'pasted into 
the BSD and AIX ports. There is nothing to do on Solaris or Windows in 
that regard.

Cheers,
David

> Cheers
> Christoph
> 
> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of David Holmes
> Sent: Montag, 9. M?rz 2020 05:33
> To: Vladimir Kozlov <vladimir.kozlov at oracle.com>; Matthias Neugschwandtner <matthias.neugschwandtner at oracle.com>; 'hotspot-dev developers' <hotspot-dev at openjdk.java.net>
> Subject: Re: RFR: 8235908: omit ThreadPriorityPolicy warning when value is set from image
> 
> The changes in relation to exposing the origin of the flag as a
> JIMAGE_RESOURCE look fine. (Could potentially have been its own RFE but
> that's just bookkeeping.)
> 
> No further comment on the actual ThreadPriorityPolicy change. :)
> 
> Cheers,
> David
> 
> On 7/03/2020 3:51 am, Vladimir Kozlov wrote:
>> I changed mailing list to wider audience.
>>
>> Note for reviews, Java SE builds and products are not affected by this
>> change.
>>
>> This change affects only GraalVM which sets ThreadPriorityPolicy=1 by
>> default in their build.
>> GraalVM release notes have statement about that to notify GraalVM users
>> because it may affect applications performance.
>>
>> RFE describes why GraalVM needs that.
>>
>> Change looks good to me.
>>
>> Thanks,
>> Vladimir K
>>
>> On 3/6/20 12:56 AM, Matthias Neugschwandtner wrote:
>>> Hello everyone,
>>>
>>> Please review this change, which omits the warning that
>>> ThreadPriorityPolicy=1 requires system level permission if the value
>>> is set via jlink, the same way it is omitted if the value is set by
>>> default.
>>>
>>> https://bugs.openjdk.java.net/browse/JDK-8235908
>>> http://cr.openjdk.java.net/~mneugschwand/8235908
>>>
>>> Thank you,
>>> Matthias

From christoph.langer at sap.com  Mon Mar  9 09:02:14 2020
From: christoph.langer at sap.com (Langer, Christoph)
Date: Mon, 9 Mar 2020 09:02:14 +0000
Subject: RFR: 8235908: omit ThreadPriorityPolicy warning when value is set
 from image
In-Reply-To: <cd8d85de-feca-363d-4e00-7d257db7b1d0@oracle.com>
References: <48754173-0e4e-736e-6895-6d3051225c7c@oracle.com>
 <c29731b1-4b29-c4c9-8532-b63b1485a976@oracle.com>
 <c74ffb25-2e2a-fb8f-01e8-085753525299@oracle.com>
 <AM0PR02MB57143DB95ECAC4BA617865FE8AFE0@AM0PR02MB5714.eurprd02.prod.outlook.com>
 <cd8d85de-feca-363d-4e00-7d257db7b1d0@oracle.com>
Message-ID: <AM0PR02MB57147057A4572B1304CC8BD68AFE0@AM0PR02MB5714.eurprd02.prod.outlook.com>

Hi David,

you're right, I didn't look too careful. ?? But AIX should not be forgotten then.

Best regards
Christoph

-----Original Message-----
From: David Holmes <david.holmes at oracle.com> 
Sent: Montag, 9. M?rz 2020 09:57
To: Langer, Christoph <christoph.langer at sap.com>; Vladimir Kozlov <vladimir.kozlov at oracle.com>; Matthias Neugschwandtner <matthias.neugschwandtner at oracle.com>; 'hotspot-dev developers' <hotspot-dev at openjdk.java.net>
Subject: Re: RFR: 8235908: omit ThreadPriorityPolicy warning when value is set from image

Hi Christoph,

On 9/03/2020 5:49 pm, Langer, Christoph wrote:
> Hi,
> 
> the changes look good. However, I think we should make this behavior consistent and also handle Windows, Solaris and AIX the same way.

The warning is primarily a Linux thing that has been copy'n'pasted into 
the BSD and AIX ports. There is nothing to do on Solaris or Windows in 
that regard.

Cheers,
David

> Cheers
> Christoph
> 
> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of David Holmes
> Sent: Montag, 9. M?rz 2020 05:33
> To: Vladimir Kozlov <vladimir.kozlov at oracle.com>; Matthias Neugschwandtner <matthias.neugschwandtner at oracle.com>; 'hotspot-dev developers' <hotspot-dev at openjdk.java.net>
> Subject: Re: RFR: 8235908: omit ThreadPriorityPolicy warning when value is set from image
> 
> The changes in relation to exposing the origin of the flag as a
> JIMAGE_RESOURCE look fine. (Could potentially have been its own RFE but
> that's just bookkeeping.)
> 
> No further comment on the actual ThreadPriorityPolicy change. :)
> 
> Cheers,
> David
> 
> On 7/03/2020 3:51 am, Vladimir Kozlov wrote:
>> I changed mailing list to wider audience.
>>
>> Note for reviews, Java SE builds and products are not affected by this
>> change.
>>
>> This change affects only GraalVM which sets ThreadPriorityPolicy=1 by
>> default in their build.
>> GraalVM release notes have statement about that to notify GraalVM users
>> because it may affect applications performance.
>>
>> RFE describes why GraalVM needs that.
>>
>> Change looks good to me.
>>
>> Thanks,
>> Vladimir K
>>
>> On 3/6/20 12:56 AM, Matthias Neugschwandtner wrote:
>>> Hello everyone,
>>>
>>> Please review this change, which omits the warning that
>>> ThreadPriorityPolicy=1 requires system level permission if the value
>>> is set via jlink, the same way it is omitted if the value is set by
>>> default.
>>>
>>> https://bugs.openjdk.java.net/browse/JDK-8235908
>>> http://cr.openjdk.java.net/~mneugschwand/8235908
>>>
>>> Thank you,
>>> Matthias

From huangjia at loongson.cn  Mon Mar  9 09:57:20 2020
From: huangjia at loongson.cn (Jia Huang)
Date: Mon, 9 Mar 2020 17:57:20 +0800
Subject: RFR: 8240222: [TESTBUG] gtest/jfr/test_networkUtilization.cpp
 failed when the number of tests is greater than or equal to 2
In-Reply-To: <931ecb2e-9898-a371-0aa9-bf463b9fdcf6@loongson.cn>
References: <64960cc9-5021-d265-ab30-9cf9a91cc6e3@loongson.cn>
 <8f143c78-6bba-5f11-21ad-3096115287e1@oracle.com>
 <4f6c2613-de5a-cc28-f208-1a763270a5d0@loongson.cn>
 <44483657-df1f-f9cc-9e6a-f73b0e602f72@oracle.com>
 <931ecb2e-9898-a371-0aa9-bf463b9fdcf6@loongson.cn>
Message-ID: <e9d5cca3-1f35-7708-1bdb-bb4b5e673d27@loongson.cn>

In addition, the static local variable "id" is assigned in the function 
of "new_entry", that is in
share/jfr/periodic/jfrNetworkUtilization.cpp.

---------------------------------------------------------------
share/jfr/periodic/jfrNetworkUtilization.cpp

static InterfaceEntry& new_entry(const NetworkInterface* iface, 
GrowableArray<InterfaceEntry>* interfaces) {
 ? assert(iface != NULL, "invariant");
 ? assert(interfaces != NULL, "invariant");

 ? // single threaded premise
 ? static traceid interface_id = 0;????????????????? ? ?? // this is the 
static local variable

 ? const char* name = iface->get_name();
 ? assert(name != NULL, "invariant");

 ? InterfaceEntry entry;
 ? const size_t length = strlen(name);
 ? entry.name = NEW_C_HEAP_ARRAY(char, length + 1, mtInternal);
 ? strncpy(entry.name, name, length + 1);
 ? entry.id = ++interface_id;
 ? entry.bytes_in = iface->get_bytes_in();
 ? entry.bytes_out = iface->get_bytes_out();
 ? entry.written = false;
 ? return _interfaces->at(_interfaces->append(entry));
}
---------------------------------------------------------------

? 2020/3/9 15:29, Jia Huang ??:
> Haha, I'm also trying to email hotspot-jfr-dev.
> Thank you, David.
>
> ? 2020/3/9 13:03, David Holmes ??:
>> Hi Jia,
>>
>> I see this is getting no response so have cc'd hotspot-jfr-dev as 
>> they should have a better idea of how this gtest can and cannot be used.
>>
>> Just because the GTest framework supports "REPEAT" it doesn't mean 
>> all tests can, or should, be repeatable.
>>
>> Cheers,
>> David
>>
>> On 29/02/2020 1:53 pm, Jia Huang wrote:
>>> Hi David,
>>>
>>> ? 2020/2/28 21:16, David Holmes ??:
>>>> Hi Jia,
>>>>
>>>> On 28/02/2020 11:08 pm, Jia Huang wrote:
>>>>> Hi all,
>>>>>
>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8240222
>>>>> Webrev: http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/
>>>>>
>>>>> gtest/jfr/test_networkUtilization.cpp failed when the number of 
>>>>> tests is greater than or equal to 2.
>>>>> As shown in the following code, the test failed because "traceid 
>>>>> id" is a static local variable.
>>>>> When the test runs a second time, "id" continues to grow, 
>>>>> resulting in the return value of "i" being "_interfaces.end()".
>>>>
>>>> Interesting! I'm not at all familiar with the details of gtest and 
>>>> how it actually runs things. But I would expect that we have many 
>>>> tests that rely on starting from a cleanly initialized VM state, or 
>>>> test state, and so can't simply be repeated (it would be like 
>>>> editing an arbitrary test and putting the body of main into a 
>>>> for-loop!).
>>>
>>> I'm sorry that my last description may be a little incomplete.
>>> JfrTestNetworkUtilization is a group of tests,it includes four 
>>> individual tests:
>>> ?? --JfrTestNetworkUtilization.RequestFunctionBasic_test_vm
>>> ?? --JfrTestNetworkUtilization.RequestFunctionMultiple_test_vm
>>> ?? --JfrTestNetworkUtilization.InterfaceRemoved_test_vm
>>> ?? --JfrTestNetworkUtilization.InterfaceReset_test_vm
>>>
>>> If you only run a individual test once, such as the following 
>>> Testing, it still fails.
>>> The reason for the failure, as I said last time, is also because of 
>>> static local variables.
>>> In the "RequestFunctionMultiple_test_vm" test, the sequence number 
>>> of "eth0" is 2.
>>> However, the initial value of the static local variable "id" is 0.
>>>
>>> Testting:
>>> ??? --make run-test 
>>> TEST=gtest:JfrTestNetworkUtilization.RequestFunctionMultiple_test_vm 
>>> GTEST=REPEAT=1 CONF=release
>>> ??? --make run-test 
>>> TEST=gtest:JfrTestNetworkUtilization.InterfaceRemoved_test_vm 
>>> GTEST=REPEAT=1 CONF=release
>>> ??? --make run-test 
>>> TEST=gtest:JfrTestNetworkUtilization.InterfaceReset_test_vm 
>>> GTEST=REPEAT=1 CONF=release
>>>
>>>>
>>>> So I'm not sure what the right way to fix this is:
>>>> - don't use REPEAT?
>>>> - declare the test as not repeatable somehow?
>>>> - change the test to make it repeatable?
>>>
>>> In the doc/testing.md or doc/testing.html, Gtest keywords inclue 
>>> "REPEAT",
>>> so I think this means that all GTEST tests should be able to use the 
>>> "REPEAT parameter.
>>>
>>>
>>> Best regards,
>>> Jia
>>>
>>>
>>>>
>>>> Cheers,
>>>> David
>>>>
>>>>> ---------------------------------------------------------------------------- 
>>>>>
>>>>> test/hotspot/gtest/jfr/test_networkUtilization.cpp
>>>>>
>>>>> static const MockNetworkInterface& get_interface(traceid id) {
>>>>> ?? std::list<MockNetworkInterface>::const_iterator i = 
>>>>> _interfaces.begin();
>>>>> ?? for (; i != _interfaces.end(); ++i) {
>>>>> ???? if (i->id == id) {
>>>>> ?????? break;
>>>>> ???? }
>>>>> ?? }
>>>>> ?? return *i;
>>>>> }
>>>>> ---------------------------------------------------------------------------- 
>>>>>
>>>>>
>>>>>
>>>>> Testing:
>>>>> ???? - make run-test TEST=gtest:JfrTestNetworkUtilization 
>>>>> GTEST=REPEAT=2 CONF=server-release
>>>>> ???? - make run-test TEST=gtest:JfrTestNetworkUtilization 
>>>>> GTEST=REPEAT=2 CONF=server-fastdebug
>>>>>
>>>>> Thanks a lot
>>>>>
>>>>> Best regards,
>>>>> Jia
>>>>>
>>>


From aph at redhat.com  Mon Mar  9 10:54:19 2020
From: aph at redhat.com (Andrew Haley)
Date: Mon, 9 Mar 2020 10:54:19 +0000
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <F9E331DA-B9FC-40E0-81FD-85CC25F45FD0@oracle.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
 <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
 <e96653f2-3f5d-42cc-6d5e-8199a3caf8e8@oracle.com>
 <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com>
 <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com>
 <F9E331DA-B9FC-40E0-81FD-85CC25F45FD0@oracle.com>
Message-ID: <a88c261a-799e-ba93-fbb8-79e488ee2a3b@redhat.com>

So, my plan is to push http://cr.openjdk.java.net/~aph/8240615-1/ and
subsequently patch a few AArch64 cases plus the x86_64 immL_Pow2 and
immL_NotPow2 cases that we already identified.

I do not intend to do bulk changes to any other clients of
is_power_of_2() because it's often unclear exactly what behaviour is
required, and I'd risk breaking something.

OK?

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From markus.gronlund at oracle.com  Mon Mar  9 11:41:05 2020
From: markus.gronlund at oracle.com (Markus Gronlund)
Date: Mon, 9 Mar 2020 04:41:05 -0700 (PDT)
Subject: RFR: 8240222: [TESTBUG] gtest/jfr/test_networkUtilization.cpp
 failed when the number of tests is greater than or equal to 2
In-Reply-To: <e9d5cca3-1f35-7708-1bdb-bb4b5e673d27@loongson.cn>
References: <64960cc9-5021-d265-ab30-9cf9a91cc6e3@loongson.cn>
 <8f143c78-6bba-5f11-21ad-3096115287e1@oracle.com>
 <4f6c2613-de5a-cc28-f208-1a763270a5d0@loongson.cn>
 <44483657-df1f-f9cc-9e6a-f73b0e602f72@oracle.com>
 <931ecb2e-9898-a371-0aa9-bf463b9fdcf6@loongson.cn>
 <e9d5cca3-1f35-7708-1bdb-bb4b5e673d27@loongson.cn>
Message-ID: <a5b687be-a9b4-4bc7-bace-b94f723cbdab@default>

The interface_id assigns a unique primary key for each interface.

To support GTEST=REPEAT, I think the easiest way is to add a corresponding counter in the gtest itself that mirror the production code:

That is, as an example, instead of the "add_interface("eth0", 1), that hardcodes the traceid = 1, have it take a value from the corresponding counter, i.e. "add_interface("eth0", next_id++)";

Thanks
Markus



-----Original Message-----
From: Jia Huang <huangjia at loongson.cn> 
Sent: den 9 mars 2020 10:57
To: hotspot-dev at openjdk.java.net; hotspot-jfr-dev at openjdk.java.net
Subject: Re: RFR: 8240222: [TESTBUG] gtest/jfr/test_networkUtilization.cpp failed when the number of tests is greater than or equal to 2

In addition, the static local variable "id" is assigned in the function of "new_entry", that is in share/jfr/periodic/jfrNetworkUtilization.cpp.

---------------------------------------------------------------
share/jfr/periodic/jfrNetworkUtilization.cpp

static InterfaceEntry& new_entry(const NetworkInterface* iface,
GrowableArray<InterfaceEntry>* interfaces) {
 ? assert(iface != NULL, "invariant");
 ? assert(interfaces != NULL, "invariant");

 ? // single threaded premise
 ? static traceid interface_id = 0;????????????????? ? ?? // this is the static local variable

 ? const char* name = iface->get_name();
 ? assert(name != NULL, "invariant");

 ? InterfaceEntry entry;
 ? const size_t length = strlen(name);
 ? entry.name = NEW_C_HEAP_ARRAY(char, length + 1, mtInternal);
 ? strncpy(entry.name, name, length + 1);
 ? entry.id = ++interface_id;
 ? entry.bytes_in = iface->get_bytes_in();
 ? entry.bytes_out = iface->get_bytes_out();
 ? entry.written = false;
 ? return _interfaces->at(_interfaces->append(entry));
}
---------------------------------------------------------------

? 2020/3/9 15:29, Jia Huang ??:
> Haha, I'm also trying to email hotspot-jfr-dev.
> Thank you, David.
>
> ? 2020/3/9 13:03, David Holmes ??:
>> Hi Jia,
>>
>> I see this is getting no response so have cc'd hotspot-jfr-dev as 
>> they should have a better idea of how this gtest can and cannot be used.
>>
>> Just because the GTest framework supports "REPEAT" it doesn't mean 
>> all tests can, or should, be repeatable.
>>
>> Cheers,
>> David
>>
>> On 29/02/2020 1:53 pm, Jia Huang wrote:
>>> Hi David,
>>>
>>> ? 2020/2/28 21:16, David Holmes ??:
>>>> Hi Jia,
>>>>
>>>> On 28/02/2020 11:08 pm, Jia Huang wrote:
>>>>> Hi all,
>>>>>
>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8240222
>>>>> Webrev: http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/
>>>>>
>>>>> gtest/jfr/test_networkUtilization.cpp failed when the number of 
>>>>> tests is greater than or equal to 2.
>>>>> As shown in the following code, the test failed because "traceid 
>>>>> id" is a static local variable.
>>>>> When the test runs a second time, "id" continues to grow, 
>>>>> resulting in the return value of "i" being "_interfaces.end()".
>>>>
>>>> Interesting! I'm not at all familiar with the details of gtest and 
>>>> how it actually runs things. But I would expect that we have many 
>>>> tests that rely on starting from a cleanly initialized VM state, or 
>>>> test state, and so can't simply be repeated (it would be like 
>>>> editing an arbitrary test and putting the body of main into a 
>>>> for-loop!).
>>>
>>> I'm sorry that my last description may be a little incomplete.
>>> JfrTestNetworkUtilization is a group of tests,it includes four 
>>> individual tests:
>>> ?? --JfrTestNetworkUtilization.RequestFunctionBasic_test_vm
>>> ?? --JfrTestNetworkUtilization.RequestFunctionMultiple_test_vm
>>> ?? --JfrTestNetworkUtilization.InterfaceRemoved_test_vm
>>> ?? --JfrTestNetworkUtilization.InterfaceReset_test_vm
>>>
>>> If you only run a individual test once, such as the following 
>>> Testing, it still fails.
>>> The reason for the failure, as I said last time, is also because of 
>>> static local variables.
>>> In the "RequestFunctionMultiple_test_vm" test, the sequence number 
>>> of "eth0" is 2.
>>> However, the initial value of the static local variable "id" is 0.
>>>
>>> Testting:
>>> ??? --make run-test
>>> TEST=gtest:JfrTestNetworkUtilization.RequestFunctionMultiple_test_vm
>>> GTEST=REPEAT=1 CONF=release
>>> ??? --make run-test
>>> TEST=gtest:JfrTestNetworkUtilization.InterfaceRemoved_test_vm
>>> GTEST=REPEAT=1 CONF=release
>>> ??? --make run-test
>>> TEST=gtest:JfrTestNetworkUtilization.InterfaceReset_test_vm
>>> GTEST=REPEAT=1 CONF=release
>>>
>>>>
>>>> So I'm not sure what the right way to fix this is:
>>>> - don't use REPEAT?
>>>> - declare the test as not repeatable somehow?
>>>> - change the test to make it repeatable?
>>>
>>> In the doc/testing.md or doc/testing.html, Gtest keywords inclue 
>>> "REPEAT", so I think this means that all GTEST tests should be able 
>>> to use the "REPEAT parameter.
>>>
>>>
>>> Best regards,
>>> Jia
>>>
>>>
>>>>
>>>> Cheers,
>>>> David
>>>>
>>>>> ---------------------------------------------------------------------------- 
>>>>>
>>>>> test/hotspot/gtest/jfr/test_networkUtilization.cpp
>>>>>
>>>>> static const MockNetworkInterface& get_interface(traceid id) {
>>>>> ?? std::list<MockNetworkInterface>::const_iterator i = 
>>>>> _interfaces.begin();
>>>>> ?? for (; i != _interfaces.end(); ++i) {
>>>>> ???? if (i->id == id) {
>>>>> ?????? break;
>>>>> ???? }
>>>>> ?? }
>>>>> ?? return *i;
>>>>> }
>>>>> ---------------------------------------------------------------------------- 
>>>>>
>>>>>
>>>>>
>>>>> Testing:
>>>>> ???? - make run-test TEST=gtest:JfrTestNetworkUtilization 
>>>>> GTEST=REPEAT=2 CONF=server-release
>>>>> ???? - make run-test TEST=gtest:JfrTestNetworkUtilization 
>>>>> GTEST=REPEAT=2 CONF=server-fastdebug
>>>>>
>>>>> Thanks a lot
>>>>>
>>>>> Best regards,
>>>>> Jia
>>>>>
>>>


From stefan.karlsson at oracle.com  Mon Mar  9 12:20:56 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Mon, 9 Mar 2020 13:20:56 +0100
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <a88c261a-799e-ba93-fbb8-79e488ee2a3b@redhat.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
 <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
 <e96653f2-3f5d-42cc-6d5e-8199a3caf8e8@oracle.com>
 <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com>
 <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com>
 <F9E331DA-B9FC-40E0-81FD-85CC25F45FD0@oracle.com>
 <a88c261a-799e-ba93-fbb8-79e488ee2a3b@redhat.com>
Message-ID: <f500862b-80d9-a5eb-dd80-3cbbd7980e1a@oracle.com>

On 2020-03-09 11:54, Andrew Haley wrote:
> So, my plan is to push http://cr.openjdk.java.net/~aph/8240615-1/ and
> subsequently patch a few AArch64 cases plus the x86_64 immL_Pow2 and
> immL_NotPow2 cases that we already identified.
> 
> I do not intend to do bulk changes to any other clients of
> is_power_of_2() because it's often unclear exactly what behaviour is
> required, and I'd risk breaking something.
> 
> OK?

I would prefer if the x86_64 and aarch64 patches were pushed first, or 
folded into this patch. Otherwise, we knowingly introduce a regression 
by pushing the proposed patch above. However, I'll leave it up to the 
compiler devs/maintainers to decide if this is OK.

StefanK

> 

From aph at redhat.com  Mon Mar  9 12:37:05 2020
From: aph at redhat.com (Andrew Haley)
Date: Mon, 9 Mar 2020 12:37:05 +0000
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <f500862b-80d9-a5eb-dd80-3cbbd7980e1a@oracle.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
 <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
 <e96653f2-3f5d-42cc-6d5e-8199a3caf8e8@oracle.com>
 <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com>
 <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com>
 <F9E331DA-B9FC-40E0-81FD-85CC25F45FD0@oracle.com>
 <a88c261a-799e-ba93-fbb8-79e488ee2a3b@redhat.com>
 <f500862b-80d9-a5eb-dd80-3cbbd7980e1a@oracle.com>
Message-ID: <e0792f13-25b6-8d57-50f6-50239ef2150d@redhat.com>

On 3/9/20 12:20 PM, Stefan Karlsson wrote:
> On 2020-03-09 11:54, Andrew Haley wrote:
>> So, my plan is to push http://cr.openjdk.java.net/~aph/8240615-1/ and
>> subsequently patch a few AArch64 cases plus the x86_64 immL_Pow2 and
>> immL_NotPow2 cases that we already identified.
>>
>> I do not intend to do bulk changes to any other clients of
>> is_power_of_2() because it's often unclear exactly what behaviour is
>> required, and I'd risk breaking something.
>>
>> OK?
> 
> I would prefer if the x86_64 and aarch64 patches were pushed first, or 
> folded into this patch. Otherwise, we knowingly introduce a regression 
> by pushing the proposed patch above. However, I'll leave it up to the 
> compiler devs/maintainers to decide if this is OK.

We already have a regression in that AArch64 generates incorrect code.
IMO it's likely other targets do too.

However, I'm happy to push the x86_64 and aarch64 patches first or
as part of this patch.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From martin.doerr at sap.com  Mon Mar  9 14:33:50 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Mon, 9 Mar 2020 14:33:50 +0000
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <e0792f13-25b6-8d57-50f6-50239ef2150d@redhat.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
 <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
 <e96653f2-3f5d-42cc-6d5e-8199a3caf8e8@oracle.com>
 <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com>
 <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com>
 <F9E331DA-B9FC-40E0-81FD-85CC25F45FD0@oracle.com>
 <a88c261a-799e-ba93-fbb8-79e488ee2a3b@redhat.com>
 <f500862b-80d9-a5eb-dd80-3cbbd7980e1a@oracle.com>
 <e0792f13-25b6-8d57-50f6-50239ef2150d@redhat.com>
Message-ID: <AM0PR0202MB3297ED1D3059110035E679DF9AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>

Hi Andrew,

I'm ok with your proposal (regardless of which change gets pushed first), too.

In addition to that we should also fix:
log2_int(int x)
log2_jint(jint x)
which use a wrong type conversion violating
"If x < 0, the function returns 31 on a 32-bit machine and 63 on a 64-bit machine.".

I guess I should open a new issue for that.

Best regards,
Martin


> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
> Andrew Haley
> Sent: Montag, 9. M?rz 2020 13:37
> To: Stefan Karlsson <stefan.karlsson at oracle.com>; John Rose
> <john.r.rose at oracle.com>
> Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net
> Subject: Re: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
> inconsistent
> 
> On 3/9/20 12:20 PM, Stefan Karlsson wrote:
> > On 2020-03-09 11:54, Andrew Haley wrote:
> >> So, my plan is to push http://cr.openjdk.java.net/~aph/8240615-1/ and
> >> subsequently patch a few AArch64 cases plus the x86_64 immL_Pow2 and
> >> immL_NotPow2 cases that we already identified.
> >>
> >> I do not intend to do bulk changes to any other clients of
> >> is_power_of_2() because it's often unclear exactly what behaviour is
> >> required, and I'd risk breaking something.
> >>
> >> OK?
> >
> > I would prefer if the x86_64 and aarch64 patches were pushed first, or
> > folded into this patch. Otherwise, we knowingly introduce a regression
> > by pushing the proposed patch above. However, I'll leave it up to the
> > compiler devs/maintainers to decide if this is OK.
> 
> We already have a regression in that AArch64 generates incorrect code.
> IMO it's likely other targets do too.
> 
> However, I'm happy to push the x86_64 and aarch64 patches first or
> as part of this patch.
> 
> --
> Andrew Haley  (he/him)
> Java Platform Lead Engineer
> Red Hat UK Ltd. <https://www.redhat.com>
> https://keybase.io/andrewhaley
> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From martin.doerr at sap.com  Mon Mar  9 14:46:56 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Mon, 9 Mar 2020 14:46:56 +0000
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <AM0PR0202MB3297ED1D3059110035E679DF9AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
 <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
 <e96653f2-3f5d-42cc-6d5e-8199a3caf8e8@oracle.com>
 <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com>
 <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com>
 <F9E331DA-B9FC-40E0-81FD-85CC25F45FD0@oracle.com>
 <a88c261a-799e-ba93-fbb8-79e488ee2a3b@redhat.com>
 <f500862b-80d9-a5eb-dd80-3cbbd7980e1a@oracle.com>
 <e0792f13-25b6-8d57-50f6-50239ef2150d@redhat.com>
 <AM0PR0202MB3297ED1D3059110035E679DF9AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
Message-ID: <AM0PR0202MB3297D74513C3E2C7F4C8B4459AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>

Sorry, my comment about log2_int + log2_jint was not precise.

The comment
"If x < 0, the function returns 31 on a 32-bit machine and 63 on a 64-bit machine."
belongs to log2_intptr, but it also applies to log2_int + log2_jint which is not desirable IMHO.
I guess using zero extend would be better for these 2 functions: "If x < 0, the function returns 31 on all machines."

Best regards,
Martin


> -----Original Message-----
> From: hotspot-runtime-dev <hotspot-runtime-dev-
> bounces at openjdk.java.net> On Behalf Of Doerr, Martin
> Sent: Montag, 9. M?rz 2020 15:34
> To: Andrew Haley <aph at redhat.com>; Stefan Karlsson
> <stefan.karlsson at oracle.com>; John Rose <john.r.rose at oracle.com>
> Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net
> Subject: RE: RFR: 8240615: is_power_of_2() has Undefined
> Behaviour and is inconsistent
> 
> Hi Andrew,
> 
> I'm ok with your proposal (regardless of which change gets pushed first), too.
> 
> In addition to that we should also fix:
> log2_int(int x)
> log2_jint(jint x)
> which use a wrong type conversion violating
> "If x < 0, the function returns 31 on a 32-bit machine and 63 on a 64-bit
> machine.".
> 
> I guess I should open a new issue for that.
> 
> Best regards,
> Martin
> 
> 
> > -----Original Message-----
> > From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf
> Of
> > Andrew Haley
> > Sent: Montag, 9. M?rz 2020 13:37
> > To: Stefan Karlsson <stefan.karlsson at oracle.com>; John Rose
> > <john.r.rose at oracle.com>
> > Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-
> dev at openjdk.java.net
> > Subject: Re: RFR: 8240615: is_power_of_2() has Undefined Behaviour and
> is
> > inconsistent
> >
> > On 3/9/20 12:20 PM, Stefan Karlsson wrote:
> > > On 2020-03-09 11:54, Andrew Haley wrote:
> > >> So, my plan is to push http://cr.openjdk.java.net/~aph/8240615-1/ and
> > >> subsequently patch a few AArch64 cases plus the x86_64 immL_Pow2
> and
> > >> immL_NotPow2 cases that we already identified.
> > >>
> > >> I do not intend to do bulk changes to any other clients of
> > >> is_power_of_2() because it's often unclear exactly what behaviour is
> > >> required, and I'd risk breaking something.
> > >>
> > >> OK?
> > >
> > > I would prefer if the x86_64 and aarch64 patches were pushed first, or
> > > folded into this patch. Otherwise, we knowingly introduce a regression
> > > by pushing the proposed patch above. However, I'll leave it up to the
> > > compiler devs/maintainers to decide if this is OK.
> >
> > We already have a regression in that AArch64 generates incorrect code.
> > IMO it's likely other targets do too.
> >
> > However, I'm happy to push the x86_64 and aarch64 patches first or
> > as part of this patch.
> >
> > --
> > Andrew Haley  (he/him)
> > Java Platform Lead Engineer
> > Red Hat UK Ltd. <https://www.redhat.com>
> > https://keybase.io/andrewhaley
> > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From aph at redhat.com  Mon Mar  9 15:42:12 2020
From: aph at redhat.com (Andrew Haley)
Date: Mon, 9 Mar 2020 15:42:12 +0000
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <AM0PR0202MB3297D74513C3E2C7F4C8B4459AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
 <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
 <e96653f2-3f5d-42cc-6d5e-8199a3caf8e8@oracle.com>
 <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com>
 <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com>
 <F9E331DA-B9FC-40E0-81FD-85CC25F45FD0@oracle.com>
 <a88c261a-799e-ba93-fbb8-79e488ee2a3b@redhat.com>
 <f500862b-80d9-a5eb-dd80-3cbbd7980e1a@oracle.com>
 <e0792f13-25b6-8d57-50f6-50239ef2150d@redhat.com>
 <AM0PR0202MB3297ED1D3059110035E679DF9AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
 <AM0PR0202MB3297D74513C3E2C7F4C8B4459AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
Message-ID: <1c40a7d1-fa36-b5d2-6072-eaa243cc9397@redhat.com>

On 3/9/20 2:46 PM, Doerr, Martin wrote:
> Sorry, my comment about log2_int + log2_jint was not precise.
> 
> The comment
> "If x < 0, the function returns 31 on a 32-bit machine and 63 on a 64-bit machine."
> belongs to log2_intptr,

Indeed.

> but it also applies to log2_int + log2_jint which is not desirable IMHO.
> I guess using zero extend would be better for these 2 functions:
> "If x < 0, the function returns 31 on all machines."

As far as I can see, log2_jint() does the right thing: it returns
31 for 0x8000_0000-0xffff_ffff, and it does this with no undefined
behaviour.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From martin.doerr at sap.com  Mon Mar  9 15:56:33 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Mon, 9 Mar 2020 15:56:33 +0000
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <1c40a7d1-fa36-b5d2-6072-eaa243cc9397@redhat.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
 <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
 <e96653f2-3f5d-42cc-6d5e-8199a3caf8e8@oracle.com>
 <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com>
 <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com>
 <F9E331DA-B9FC-40E0-81FD-85CC25F45FD0@oracle.com>
 <a88c261a-799e-ba93-fbb8-79e488ee2a3b@redhat.com>
 <f500862b-80d9-a5eb-dd80-3cbbd7980e1a@oracle.com>
 <e0792f13-25b6-8d57-50f6-50239ef2150d@redhat.com>
 <AM0PR0202MB3297ED1D3059110035E679DF9AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
 <AM0PR0202MB3297D74513C3E2C7F4C8B4459AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
 <1c40a7d1-fa36-b5d2-6072-eaa243cc9397@redhat.com>
Message-ID: <AM0PR0202MB3297869C05C5A950564C3C8C9AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>

Hi Andrew,

it doesn't run into the UB case, but log2_jint(x) returns 63 for negative x.
I'd treat this as bug. A jint has 32 bit and log2 should never return 63.

Best regards,
Martin


> -----Original Message-----
> From: Andrew Haley <aph at redhat.com>
> Sent: Montag, 9. M?rz 2020 16:42
> To: Doerr, Martin <martin.doerr at sap.com>; Stefan Karlsson
> <stefan.karlsson at oracle.com>; John Rose <john.r.rose at oracle.com>
> Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net
> Subject: Re: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
> inconsistent
> 
> On 3/9/20 2:46 PM, Doerr, Martin wrote:
> > Sorry, my comment about log2_int + log2_jint was not precise.
> >
> > The comment
> > "If x < 0, the function returns 31 on a 32-bit machine and 63 on a 64-bit
> machine."
> > belongs to log2_intptr,
> 
> Indeed.
> 
> > but it also applies to log2_int + log2_jint which is not desirable IMHO.
> > I guess using zero extend would be better for these 2 functions:
> > "If x < 0, the function returns 31 on all machines."
> 
> As far as I can see, log2_jint() does the right thing: it returns
> 31 for 0x8000_0000-0xffff_ffff, and it does this with no undefined
> behaviour.
> 
> --
> Andrew Haley  (he/him)
> Java Platform Lead Engineer
> Red Hat UK Ltd. <https://www.redhat.com>
> https://keybase.io/andrewhaley
> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From huangjia at loongson.cn  Tue Mar 10 01:30:29 2020
From: huangjia at loongson.cn (Jia Huang)
Date: Tue, 10 Mar 2020 09:30:29 +0800
Subject: RFR: 8240222: [TESTBUG] gtest/jfr/test_networkUtilization.cpp
 failed when the number of tests is greater than or equal to 2
In-Reply-To: <a5b687be-a9b4-4bc7-bace-b94f723cbdab@default>
References: <64960cc9-5021-d265-ab30-9cf9a91cc6e3@loongson.cn>
 <8f143c78-6bba-5f11-21ad-3096115287e1@oracle.com>
 <4f6c2613-de5a-cc28-f208-1a763270a5d0@loongson.cn>
 <44483657-df1f-f9cc-9e6a-f73b0e602f72@oracle.com>
 <931ecb2e-9898-a371-0aa9-bf463b9fdcf6@loongson.cn>
 <e9d5cca3-1f35-7708-1bdb-bb4b5e673d27@loongson.cn>
 <a5b687be-a9b4-4bc7-bace-b94f723cbdab@default>
Message-ID: <74960f03-542c-f7f5-9f46-994d355d62d3@loongson.cn>

Thanks Markus.

Could you give some comments or advice on this one:
Webrev:? http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/ ?

Best regards,
Jia

? 2020/3/9 19:41, Markus Gronlund ??:
> The interface_id assigns a unique primary key for each interface.
>
> To support GTEST=REPEAT, I think the easiest way is to add a corresponding counter in the gtest itself that mirror the production code:
>
> That is, as an example, instead of the "add_interface("eth0", 1), that hardcodes the traceid = 1, have it take a value from the corresponding counter, i.e. "add_interface("eth0", next_id++)";
>
> Thanks
> Markus
>
>
>
> -----Original Message-----
> From: Jia Huang <huangjia at loongson.cn>
> Sent: den 9 mars 2020 10:57
> To: hotspot-dev at openjdk.java.net; hotspot-jfr-dev at openjdk.java.net
> Subject: Re: RFR: 8240222: [TESTBUG] gtest/jfr/test_networkUtilization.cpp failed when the number of tests is greater than or equal to 2
>
> In addition, the static local variable "id" is assigned in the function of "new_entry", that is in share/jfr/periodic/jfrNetworkUtilization.cpp.
>
> ---------------------------------------------------------------
> share/jfr/periodic/jfrNetworkUtilization.cpp
>
> static InterfaceEntry& new_entry(const NetworkInterface* iface,
> GrowableArray<InterfaceEntry>* interfaces) {
>   ? assert(iface != NULL, "invariant");
>   ? assert(interfaces != NULL, "invariant");
>
>   ? // single threaded premise
>   ? static traceid interface_id = 0;????????????????? ? ?? // this is the static local variable
>
>   ? const char* name = iface->get_name();
>   ? assert(name != NULL, "invariant");
>
>   ? InterfaceEntry entry;
>   ? const size_t length = strlen(name);
>   ? entry.name = NEW_C_HEAP_ARRAY(char, length + 1, mtInternal);
>   ? strncpy(entry.name, name, length + 1);
>   ? entry.id = ++interface_id;
>   ? entry.bytes_in = iface->get_bytes_in();
>   ? entry.bytes_out = iface->get_bytes_out();
>   ? entry.written = false;
>   ? return _interfaces->at(_interfaces->append(entry));
> }
> ---------------------------------------------------------------
>
> ? 2020/3/9 15:29, Jia Huang ??:
>> Haha, I'm also trying to email hotspot-jfr-dev.
>> Thank you, David.
>>
>> ? 2020/3/9 13:03, David Holmes ??:
>>> Hi Jia,
>>>
>>> I see this is getting no response so have cc'd hotspot-jfr-dev as
>>> they should have a better idea of how this gtest can and cannot be used.
>>>
>>> Just because the GTest framework supports "REPEAT" it doesn't mean
>>> all tests can, or should, be repeatable.
>>>
>>> Cheers,
>>> David
>>>
>>> On 29/02/2020 1:53 pm, Jia Huang wrote:
>>>> Hi David,
>>>>
>>>> ? 2020/2/28 21:16, David Holmes ??:
>>>>> Hi Jia,
>>>>>
>>>>> On 28/02/2020 11:08 pm, Jia Huang wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8240222
>>>>>> Webrev: http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/
>>>>>>
>>>>>> gtest/jfr/test_networkUtilization.cpp failed when the number of
>>>>>> tests is greater than or equal to 2.
>>>>>> As shown in the following code, the test failed because "traceid
>>>>>> id" is a static local variable.
>>>>>> When the test runs a second time, "id" continues to grow,
>>>>>> resulting in the return value of "i" being "_interfaces.end()".
>>>>> Interesting! I'm not at all familiar with the details of gtest and
>>>>> how it actually runs things. But I would expect that we have many
>>>>> tests that rely on starting from a cleanly initialized VM state, or
>>>>> test state, and so can't simply be repeated (it would be like
>>>>> editing an arbitrary test and putting the body of main into a
>>>>> for-loop!).
>>>> I'm sorry that my last description may be a little incomplete.
>>>> JfrTestNetworkUtilization is a group of tests,it includes four
>>>> individual tests:
>>>>  ?? --JfrTestNetworkUtilization.RequestFunctionBasic_test_vm
>>>>  ?? --JfrTestNetworkUtilization.RequestFunctionMultiple_test_vm
>>>>  ?? --JfrTestNetworkUtilization.InterfaceRemoved_test_vm
>>>>  ?? --JfrTestNetworkUtilization.InterfaceReset_test_vm
>>>>
>>>> If you only run a individual test once, such as the following
>>>> Testing, it still fails.
>>>> The reason for the failure, as I said last time, is also because of
>>>> static local variables.
>>>> In the "RequestFunctionMultiple_test_vm" test, the sequence number
>>>> of "eth0" is 2.
>>>> However, the initial value of the static local variable "id" is 0.
>>>>
>>>> Testting:
>>>>  ??? --make run-test
>>>> TEST=gtest:JfrTestNetworkUtilization.RequestFunctionMultiple_test_vm
>>>> GTEST=REPEAT=1 CONF=release
>>>>  ??? --make run-test
>>>> TEST=gtest:JfrTestNetworkUtilization.InterfaceRemoved_test_vm
>>>> GTEST=REPEAT=1 CONF=release
>>>>  ??? --make run-test
>>>> TEST=gtest:JfrTestNetworkUtilization.InterfaceReset_test_vm
>>>> GTEST=REPEAT=1 CONF=release
>>>>
>>>>> So I'm not sure what the right way to fix this is:
>>>>> - don't use REPEAT?
>>>>> - declare the test as not repeatable somehow?
>>>>> - change the test to make it repeatable?
>>>> In the doc/testing.md or doc/testing.html, Gtest keywords inclue
>>>> "REPEAT", so I think this means that all GTEST tests should be able
>>>> to use the "REPEAT parameter.
>>>>
>>>>
>>>> Best regards,
>>>> Jia
>>>>
>>>>
>>>>> Cheers,
>>>>> David
>>>>>
>>>>>> ----------------------------------------------------------------------------
>>>>>>
>>>>>> test/hotspot/gtest/jfr/test_networkUtilization.cpp
>>>>>>
>>>>>> static const MockNetworkInterface& get_interface(traceid id) {
>>>>>>  ?? std::list<MockNetworkInterface>::const_iterator i =
>>>>>> _interfaces.begin();
>>>>>>  ?? for (; i != _interfaces.end(); ++i) {
>>>>>>  ???? if (i->id == id) {
>>>>>>  ?????? break;
>>>>>>  ???? }
>>>>>>  ?? }
>>>>>>  ?? return *i;
>>>>>> }
>>>>>> ----------------------------------------------------------------------------
>>>>>>
>>>>>>
>>>>>>
>>>>>> Testing:
>>>>>>  ???? - make run-test TEST=gtest:JfrTestNetworkUtilization
>>>>>> GTEST=REPEAT=2 CONF=server-release
>>>>>>  ???? - make run-test TEST=gtest:JfrTestNetworkUtilization
>>>>>> GTEST=REPEAT=2 CONF=server-fastdebug
>>>>>>
>>>>>> Thanks a lot
>>>>>>
>>>>>> Best regards,
>>>>>> Jia
>>>>>>


From xxinliu at amazon.com  Tue Mar 10 07:06:14 2020
From: xxinliu at amazon.com (Liu, Xin)
Date: Tue, 10 Mar 2020 07:06:14 +0000
Subject: RFR(L): JDK-8229517: Support for optional asynchronous/buffered
 logging
Message-ID: <49CE3CD3-37D9-458E-9B8A-B12F3A478EBF@amazon.com>

Hi, David, 

Thanks for commenting the RFR.

Indeed, LogAsyncFlusher needs to wait async loggers to release the RW mutex, but please note that LogAsyncFlusher releases the RW mutex before it traps into filesystem writing. It's not a blocker( refer to LogAsyncFlusher::task). 
Ie. The current implementation is blocking-free but not lock-free.  I mentioned it in early email.  I don't want to make thing complicated in early stage.  Lock-free FIFO requires extensive testing.  

For a FIFO queue, there's a broad design space for it.  If we only choose selective logging sites and offload them to the async buffer, it won't be contended situation.  If it turns out to be a contented lock, we could change it to spinlock, priority lock or even upgrade the FIFO to lock-free implementation.

LogAsyncFlusher is singleton. Its constructor is private.  Currently, I manually initialize and delete the only instance of it. It's very subtle because there's timing issue alongside of the initialization JVM. 
If we enables C++11 in the future, we can have Meyer's singleton idiom.  It will be more elegant because the instance can be lazy-initialized. 

Speaking of tagset, I think it's advantage of Unified logging framework. The framework separates frond-end and backend, interface and implementation. I love it and make use of it. 
A concrete class called 'LogTagSet' represents a set of tags.  They are all singleton and created by LogTagSetMapping. How to emit log messages is defined in LogTagSet.  They are even configurable in the runtime. 
<log.hpp> and <asynclog.hpp> are just two set of compatible interfaces. Under the hook, they shared the same LogTagset infrastructure. 

Yes, developers have to choose go conventional log or asynclog in build-time. I think developers have insights of their code which may cause long latency from logging. 
In order to have drop-in replacement api, I didn't go the direction that provide a new tag for async buffer.  So far, tags are all component names of hotspot. 

Thanks,
--lx


?On 3/8/20, 10:34 PM, "David Holmes" <david.holmes at oracle.com> wrote:

    CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
    
    
    
    Hi,
    
    On 5/03/2020 8:04 pm, Liu, Xin wrote:
    > Hi, hotspot reviewers,
    >
    > Could you take a look at my webrev?
    > Bug: https://bugs.openjdk.java.net/browse/JDK-8229517
    > Webrev: https://cr.openjdk.java.net/~xliu/8229517/01/webrev/
    >
    > Previously, we had discussion at runtime-dev. https://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2020-February/038211.html
    > I would like to continue it because I redo my implementation on the tip of jdk and have made it a general solution.
    
    Not a review as such but a couple of comments/queries.
    
    The async logging is done as a periodic task that will be executed the
    Watcher thread. This task requires a mutex that is also used by the
    other threads doing the log operations. So potentially the watcher
    thread could experience a significant delay waiting for the mutex.
    
    I'm not clear what the actual lifecycle of the LogAsyncFlusher instance
    is. I would have expected a single instance initialized at VM startup
    and cleaned up at VM exit. The code gives the appearance that this can
    actually be much more dynamic with multiple instances coming and going
    through the lifetime of the VM. Deletion of the instance is particularly
    problematic due to the Mutex, which has to be guaranteed not to be
    accessed when it gets deleted.
    
    > This webrev  provides <logging/asynclog.hpp> on apar with <log.hpp>. Even though it doesn?t touch any existing logging calls, it?s very possible to offload any callsite of logging to the asynchronous buffer.
    > My rationale is that we only need to redirect some time-critical logs to asynchronous buffer to dodge unexpected filesystem delay.  Eg. g1gc has some loggings at safepoint. We mitigate those long-tailed GC latencies rooted by filesystem writing to gclog.
    
    So this isn't an opt-in facility at runtime, but rather something that
    has to be decided a-priori by the developer.
    
    How does this interact with the synchronous logging that may be used for
    a given set of log tags?
    
    Thanks,
    David
    -----
    
    > I got feedback from my our internal users. They have a concern about my approach of dynamic memory. Indeed, I gave up ringbuffer design in previous version and turn to use a dynamic allocated linkedlist.  If writing to filesystem stalls infinitely, JVM may implode because of memory usage.  I plan to solve it by adding a memory quota to the FIFO of logAsyncFlusher.hpp. It will start dropping early messages of  the FIFO if it has consumed too many free memories. Another potential improvement is changing the FIFO from mutex protected to lock-free.  It make sense if we redirect high volume of logging messages to it and mutex becomes hot.
    >
    > Thanks,
    > --lx
    >
    >
    >
    


From sgehwolf at redhat.com  Tue Mar 10 09:10:19 2020
From: sgehwolf at redhat.com (Severin Gehwolf)
Date: Tue, 10 Mar 2020 10:10:19 +0100
Subject: PING? RFR(S): 8239785: Cgroups: Incorrect detection logic on
 old systems in hotspot
In-Reply-To: <C81DB191-91BF-4ED6-81E1-51605FAE6693@oracle.com>
References: <362678e8bc8231c205c6b3327d08dd5179154e23.camel@redhat.com>
 <2A934185-9B81-4371-B985-F0C3C39B3FD3@oracle.com>
 <215e52601b2da8600cbfd47ef73634720a7736d2.camel@redhat.com>
 <EF45A2BB-360E-4767-B542-B420BF76FD2E@oracle.com>
 <d1683592b67f99e7625713d2db2c71641d50a738.camel@redhat.com>
 <1B61ACE7-AFB9-458C-B6AB-465259BD31C7@oracle.com>
 <264b4edf94d60a3256a9ff8d5de6a078c3a0819b.camel@redhat.com>
 <A6EE0016-E33B-451D-BF0F-5F9C13A4DB32@oracle.com>
 <94c06270838cb3c090a227f2688a5c3f0aade64c.camel@redhat.com>
 <C81DB191-91BF-4ED6-81E1-51605FAE6693@oracle.com>
Message-ID: <07038b5c3cce3dab81aa812e985aad348e508c9b.camel@redhat.com>

I would still need a *R*eviewer for this one. Any takers? Bob has
already reviewed it.

http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/

Thanks,
Severin

On Thu, 2020-03-05 at 11:13 -0500, Bob Vandette wrote:
> > On Mar 5, 2020, at 11:00 AM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
> > 
> > Hi Bob,
> > 
> > On Thu, 2020-03-05 at 08:41 -0500, Bob Vandette wrote:
> > > > On Mar 5, 2020, at 4:36 AM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
> > > > 
> > > > Hi Bob,
> > > > 
> > > > On Wed, 2020-03-04 at 15:39 -0500, Bob Vandette wrote:
> > > > > > On Mar 4, 2020, at 1:59 PM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
> > > > > > 
> > > > > > Hi Bob,
> > > > > > 
> > > > > > Updated webrev:
> > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/
> > > > > > 
> > > > > > Testing: hotspot docker tests on Linux fastdebug/release cgroup v1 and
> > > > > > cgroup v2. New regression test. jdk submit. All pass.
> > > > > > 
> > > > > > On Fri, 2020-02-28 at 15:08 -0500, Bob Vandette wrote:
> > > > > > > It still looks like the _mount_path can be corrupted if both cgroupv1 and cgroupv2 are mounted.
> > > > > > > You?ll only initialize the _mount_path if cgroupv2 is detected in the cgroups file but if cgroupv1
> > > > > > > is also mounted, you?ll still clobber the _mount_path.
> > > > > > 
> > > > > > How so? When we start processing /proc/self/mountinfo on line 170, then
> > > > > > we have already looked at /proc/cgroups, thus is_cgroupsV2 is set to
> > > > > > true (A) or false (B).
> > > > > > 
> > > > > > Cases A:
> > > > > > 1. We are on an old system with *no* cgroup controllers (v1 or v2)
> > > > > >  mounted (this bug). We are fine to continue as no cgroups entry
> > > > > >  will be found in /proc/self/mountinfo. Thus, no clobbering.
> > > > > > 2. We are on a true unified hierarchy only system. In that case
> > > > > >  the sscanf for v1 would match (early match), but 'memory' et.al
> > > > > >  controllers would not be found. Thus, no clobbering.
> > > > > > 
> > > > > > Cases B:
> > > > > > 1. Hybrid hierarchy: controllers mounted via cgroup v1, but
> > > > > >  cgroup v2 mounted in addition for system use. is_cgroupV2 will
> > > > > >  prevent the cgroup v2 controller path lookup logic to be entered.
> > > > > >  No clobbering.
> > > > > > 2. cgroup v1 only hierarchy (legacy): Same logic applies. V2 setting
> > > > > >  of mount paths won't be entered since is_cgroupV2 == false.
> > > > > >  No clobbering.
> > > > > > 
> > > > > > What am I missing? FWIW, I've played around with the idea of keeping
> > > > > > cgroup v1 and cgroup v2 mount paths separate[1]. If that's preferred,
> > > > > > happy to go this route.
> > > > > 
> > > > > The case that I think is a problem is ?
> > > > > 
> > > > > cgroupv2 is detected in /proc/cgroups so is_cgroupv2 is true
> > > > > both cgroupv1 and cgroupv2 devices are mounted
> > > > 
> > > > That's not possible. is_cgroupv2 cannot be true in that case.
> > > > 
> > > > > At line cgroupSubsystem_linux.cpp:292 in your new webrev, you process the cgroupv1
> > > > > devices and clobber _mount_path.
> > > > > 
> > > > > What keeps this from occurring?
> > > > 
> > > > In that case 'is_cgroupv2 == false', because having any cgroups v1
> > > > controllers mounted will result in some non-zero hierarchy ids in
> > > > /proc/cgroups. That in turn will set is_cgroup2 == false.
> > > > 
> > > > From man 7 cgroups:
> > > > 
> > > > """"
> > > >  /proc files
> > > >      /proc/cgroups (since Linux 2.6.24)
> > > > 
> > > >             The fields in this file are, from left to right:
> > > > 
> > > >             1. The name of the controller.
> > > > 
> > > >             2. The unique ID of the cgroup hierarchy on which this controller is mounted.  If multiple cgroups v1 controllers are bound to the same hierarchy, then each will show the same hierarchy ID in
> > > >                this field.  The value in this field will be 0 if:
> > > > 
> > > >                  a) the controller is not mounted on a cgroups v1 hierarchy;
> > > > 
> > > >                  b) the controller is bound to the cgroups v2 single unified hierarchy; or
> > > > 
> > > >                  c) the controller is disabled (see below).
> > > > """
> > > > 
> > > > The only way hierarchy id == 0 for a controller on cgroups v1 is a).
> > > > It's never 0 when cgroup v1 and cgroup v2 are *both* mounted (hybrid).
> > > > One example of such a /proc/cgroups is on my F30 workstation (hybrid):
> > > > 
> > > > $ cat /proc/cgroups 
> > > > #subsys_name	hierarchy	num_cgroups	enabled
> > > > cpuset	5	1	1
> > > > cpu	3	1	1
> > > > cpuacct	3	1	1
> > > > blkio	11	1	1
> > > > memory	6	136	1
> > > > devices	9	75	1
> > > > freezer	7	1	1
> > > > net_cls	4	1	1
> > > > perf_event	2	1	1
> > > > net_prio	4	1	1
> > > > hugetlb	10	1	1
> > > > pids	8	81	1
> > > > 
> > > > And the corresponding mountinfo:
> > > > 
> > > > $ grep cgroup /proc/self/mountinfo 
> > > > 30 23 0:26 / /sys/fs/cgroup ro,nosuid,nodev,noexec shared:4 - tmpfs tmpfs ro,seclabel,mode=755
> > > > 31 30 0:27 / /sys/fs/cgroup/unified rw,nosuid,nodev,noexec,relatime shared:5 - cgroup2 cgroup2 rw,seclabel,nsdelegate
> > > > 32 30 0:28 / /sys/fs/cgroup/systemd rw,nosuid,nodev,noexec,relatime shared:6 - cgroup cgroup rw,seclabel,xattr,name=systemd
> > > > 35 30 0:31 / /sys/fs/cgroup/perf_event rw,nosuid,nodev,noexec,relatime shared:7 - cgroup cgroup rw,seclabel,perf_event
> > > > 36 30 0:32 / /sys/fs/cgroup/cpu,cpuacct rw,nosuid,nodev,noexec,relatime shared:8 - cgroup cgroup rw,seclabel,cpu,cpuacct
> > > > 37 30 0:33 / /sys/fs/cgroup/net_cls,net_prio rw,nosuid,nodev,noexec,relatime shared:9 - cgroup cgroup rw,seclabel,net_cls,net_prio
> > > > 38 30 0:34 / /sys/fs/cgroup/cpuset rw,nosuid,nodev,noexec,relatime shared:10 - cgroup cgroup rw,seclabel,cpuset
> > > > 39 30 0:35 / /sys/fs/cgroup/memory rw,nosuid,nodev,noexec,relatime shared:11 - cgroup cgroup rw,seclabel,memory
> > > > 40 30 0:36 / /sys/fs/cgroup/freezer rw,nosuid,nodev,noexec,relatime shared:12 - cgroup cgroup rw,seclabel,freezer
> > > > 41 30 0:37 / /sys/fs/cgroup/pids rw,nosuid,nodev,noexec,relatime shared:13 - cgroup cgroup rw,seclabel,pids
> > > > 42 30 0:38 / /sys/fs/cgroup/devices rw,nosuid,nodev,noexec,relatime shared:14 - cgroup cgroup rw,seclabel,devices
> > > > 43 30 0:39 / /sys/fs/cgroup/hugetlb rw,nosuid,nodev,noexec,relatime shared:15 - cgroup cgroup rw,seclabel,hugetlb
> > > > 44 30 0:40 / /sys/fs/cgroup/blkio rw,nosuid,nodev,noexec,relatime shared:16 - cgroup cgroup rw,seclabel,blkio
> > > > 
> > > > Does that make sense?
> > > 
> > > Ok, that makes sense.
> > > 
> > > Thanks for the clarification.
> > 
> > Anytime. Can I put you down as a reviewer (the non-census one)?
> 
> Yes, absolutely.
> 
> Bob.
> 
> > Thanks,
> > Severin
> > 
> > > Bob.
> > > 
> > > > Thanks,
> > > > Severin
> > > > 
> > > > > > While thinking about this issue here, I've decided to actually write a
> > > > > > test (using WhiteBox) so that all these cases are covered and will
> > > > > > continue to work. Running the test, CgroupSubsystemFactory, with
> > > > > > fastdebug proves that there is no clobbering if that matters. It's also
> > > > > > a way to reproduce the issue. So we have a regression test now too
> > > > > > (testCgroupv1NoMounts). Line 325-332 in [2] is the actual fix. Without
> > > > > > it, the test fails.
> > > > > > 
> > > > > > > Maybe you should just do the cgroupv2 processing if /proc/cgroups has
> > > > > > > cgroupv2 enabled and if that fails continue looking for cgroup v1
> > > > > > > mounts.
> > > > > > 
> > > > > > FWIW, I've amended the cgroup v1 logic to also check for the file
> > > > > > system type. Which needs to be 'cgroup' (not 'cgroup2') for cgroup v1.
> > > > > > This isn't strictly needed but more accurately reflects the intent IMO.
> > > > > 
> > > > > Sounds good.
> > > > > 
> > > > > > > Do you really need to do this?  The BSS section should be zeroed.
> > > > > > > 
> > > > > > > 55   for (int i = 0; i < CG_INFO_LENGTH; i++) {
> > > > > > > 56     cg_infos[i]._name = NULL;
> > > > > > > 57     cg_infos[i]._cgroup_path = NULL;
> > > > > > > 58     cg_infos[i]._root_mount_path = NULL;
> > > > > > > 59     cg_infos[i]._mount_path = NULL;
> > > > > > > 60   }
> > > > > > 
> > > > > > I believe so. CgroupInfo array is stack allocated and we should
> > > > > > initialize values properly. On the other hand, we should do that in the
> > > > > > constructor. Done now.
> > > > > 
> > > > > You are correct.  I was only looking at the diffs and not the full source.
> > > > > 
> > > > > > > You don?t need to check for NULL in cleanup, os::free can handle NULL
> > > > > > > pointers.
> > > > > > > 
> > > > > > > 339 void CgroupSubsystemFactory::cleanup(CgroupInfo* cg_infos) {
> > > > > > > 340   assert(cg_infos != NULL, "Invariant");
> > > > > > > 341   for (int i = 0; i < CG_INFO_LENGTH; i++) {
> > > > > > > 342     if (cg_infos[i]._name != NULL) {
> > > > > > > 343       os::free(cg_infos[i]._name);
> > > > > > > 344     }
> > > > > > > 345     if (cg_infos[i]._cgroup_path != NULL) {
> > > > > > > 346       os::free(cg_infos[i]._cgroup_path);
> > > > > > > 347     }
> > > > > > > 348     if (cg_infos[i]._root_mount_path != NULL) {
> > > > > > > 349       os::free(cg_infos[i]._root_mount_path);
> > > > > > > 350     }
> > > > > > > 351     if (cg_infos[i]._mount_path != NULL) {
> > > > > > > 352       os::free(cg_infos[i]._mount_path);
> > > > > > > 353     }
> > > > > > > 354   }
> > > > > > > 355 }
> > > > > > 
> > > > > > Fixed.
> > > > > > 
> > > > > > Let me know what you think about the updated webrev.
> > > > > > 
> > > > > > Thanks,
> > > > > > Severin
> > > > > > 
> > > > > > [1] http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/02/webrev/
> > > > > > [2] http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/src/hotspot/os/linux/cgroupSubsystem_linux.cpp.html
> > > > > > 
> > > > > > > Bob.
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > > On Feb 28, 2020, at 11:42 AM, Severin Gehwolf <
> > > > > > > > sgehwolf at redhat.com>
> > > > > > > > wrote:
> > > > > > > > 
> > > > > > > > Hi Bob,
> > > > > > > > 
> > > > > > > > Thanks for the review!
> > > > > > > > 
> > > > > > > > On Tue, 2020-02-25 at 15:08 -0500, Bob Vandette wrote:
> > > > > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/01/webrev/src/hotspot/os/linux/cgroupSubsystem_linux.cpp.sdiff.html
> > > > > > > > > 
> > > > > > > > > If both cgroupv1 and v2 controllers are mounted, you might
> > > > > > > > > have
> > > > > > > > > some problems.  You might want to only look for
> > > > > > > > > cgroup2 mounts if is_cgroupV2 is true otherwise only look for
> > > > > > > > > cgroup mounts.
> > > > > > > > 
> > > > > > > > OK.
> > > > > > > > 
> > > > > > > > > 194           cg_infos[i]._mount_path =
> > > > > > > > > os::strdup(tmp_mount_point);
> > > > > > > > > This will stomp the pointer:
> > > > > > > > > 213           cg_infos[memory_idx]._mount_path =
> > > > > > > > > os::strdup(tmpmount);
> > > > > > > > 
> > > > > > > > You are right. If we are on a hybrid hierarchy and for some
> > > > > > > > reason
> > > > > > > > the
> > > > > > > > cgroup v2 mount shows up *after* cgroup mounts in
> > > > > > > > /proc/self/mountinfo,
> > > > > > > > we'd get the mount path wrong. So ordering matters! The reason
> > > > > > > > this
> > > > > > > > worked for me on hybrid is that cgroup2 mounts come *before*
> > > > > > > > cgroup
> > > > > > > > mounts on the system I've tested.
> > > > > > > > 
> > > > > > > > > You should free all strdup?d pointers in cg_infos in places
> > > > > > > > > where
> > > > > > > > > you fail and return NULL.
> > > > > > > > > 
> > > > > > > > > 81       cg_infos[memory_idx]._name = os::strdup(name);
> > > > > > > > > 213           cg_infos[memory_idx]._mount_path =
> > > > > > > > > os::strdup(tmpmount);
> > > > > > > > > 214           cg_infos[memory_idx]._root_mount_path =
> > > > > > > > > os::strdup(tmproot);
> > > > > > > > 
> > > > > > > > Done now. More thoughts?
> > > > > > > > 
> > > > > > > > Updated webrev: 
> > > > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/03/webrev/
> > > > > > > > 
> > > > > > > > Thanks,
> > > > > > > > Severin
> > > > > > > > 
> > > > > > > > > > On Feb 25, 2020, at 2:22 PM, Severin Gehwolf <
> > > > > > > > > > sgehwolf at redhat.com> wrote:
> > > > > > > > > > 
> > > > > > > > > > Hi,
> > > > > > > > > > 
> > > > > > > > > > Could I please get reviews of this cgroup-related patch?
> > > > > > > > > > After
> > > > > > > > > > JDK-
> > > > > > > > > > 8230305 old systems with no mounted cgroup controllers
> > > > > > > > > > would
> > > > > > > > > > get
> > > > > > > > > > detected as cgroups v2 (wrong). Then, when initializing the
> > > > > > > > > > v2
> > > > > > > > > > subsystem, it would fail. The trace about cgroupv2 mount
> > > > > > > > > > point
> > > > > > > > > > not
> > > > > > > > > > found is misleading in this case. While the outcome is
> > > > > > > > > > similar
> > > > > > > > > > pre/post
> > > > > > > > > > patch (NULL cgroup subsystem), I'd like to be explicit
> > > > > > > > > > about
> > > > > > > > > > this case.
> > > > > > > > > > 
> > > > > > > > > > The suggested fix is to look at /proc/self/mountinfo in
> > > > > > > > > > order
> > > > > > > > > > to
> > > > > > > > > > correctly detect whether this is cgroup v2 or cgroup v1
> > > > > > > > > > with no
> > > > > > > > > > mounted
> > > > > > > > > > controllers. In the latter case we just stop initialization
> > > > > > > > > > as
> > > > > > > > > > we'd
> > > > > > > > > > fail later in cgroupv1 code anyway. This aligns hotspot
> > > > > > > > > > code
> > > > > > > > > > with core-
> > > > > > > > > > libs after JDK-8239559.
> > > > > > > > > > 
> > > > > > > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8239785
> > > > > > > > > > webrev: 
> > > > > > > > > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/01/webrev/
> > > > > > > > > > 
> > > > > > > > > > Testing: jdk-submit, hotspot docker tests on cgroup v1 and
> > > > > > > > > > cgroup v2. All pass.
> > > > > > > > > > 
> > > > > > > > > > Thoughts?
> > > > > > > > > > 
> > > > > > > > > > Thanks,
> > > > > > > > > > Severin
> > > > > > > > > > 


From aph at redhat.com  Tue Mar 10 09:44:54 2020
From: aph at redhat.com (Andrew Haley)
Date: Tue, 10 Mar 2020 09:44:54 +0000
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <AM0PR0202MB3297869C05C5A950564C3C8C9AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
 <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
 <e96653f2-3f5d-42cc-6d5e-8199a3caf8e8@oracle.com>
 <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com>
 <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com>
 <F9E331DA-B9FC-40E0-81FD-85CC25F45FD0@oracle.com>
 <a88c261a-799e-ba93-fbb8-79e488ee2a3b@redhat.com>
 <f500862b-80d9-a5eb-dd80-3cbbd7980e1a@oracle.com>
 <e0792f13-25b6-8d57-50f6-50239ef2150d@redhat.com>
 <AM0PR0202MB3297ED1D3059110035E679DF9AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
 <AM0PR0202MB3297D74513C3E2C7F4C8B4459AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
 <1c40a7d1-fa36-b5d2-6072-eaa243cc9397@redhat.com>
 <AM0PR0202MB3297869C05C5A950564C3C8C9AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
Message-ID: <57e26fa3-6c8f-a7db-f6fc-9d4b58cf4126@redhat.com>

On 3/9/20 3:56 PM, Doerr, Martin wrote:
> it doesn't run into the UB case, but log2_jint(x) returns 63 for negative x.
> I'd treat this as bug. A jint has 32 bit and log2 should never return 63.

OK.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From markus.gronlund at oracle.com  Tue Mar 10 10:15:15 2020
From: markus.gronlund at oracle.com (Markus Gronlund)
Date: Tue, 10 Mar 2020 03:15:15 -0700 (PDT)
Subject: RFR: 8240222: [TESTBUG] gtest/jfr/test_networkUtilization.cpp
 failed when the number of tests is greater than or equal to 2
In-Reply-To: <74960f03-542c-f7f5-9f46-994d355d62d3@loongson.cn>
References: <64960cc9-5021-d265-ab30-9cf9a91cc6e3@loongson.cn>
 <8f143c78-6bba-5f11-21ad-3096115287e1@oracle.com>
 <4f6c2613-de5a-cc28-f208-1a763270a5d0@loongson.cn>
 <44483657-df1f-f9cc-9e6a-f73b0e602f72@oracle.com>
 <931ecb2e-9898-a371-0aa9-bf463b9fdcf6@loongson.cn>
 <e9d5cca3-1f35-7708-1bdb-bb4b5e673d27@loongson.cn>
 <a5b687be-a9b4-4bc7-bace-b94f723cbdab@default>
 <74960f03-542c-f7f5-9f46-994d355d62d3@loongson.cn>
Message-ID: <50d5e790-ef5c-4c35-bc14-71577df50c9a@default>

Hi Jia,

Looks like it could work.

Thanks
Markus

-----Original Message-----
From: Jia Huang <huangjia at loongson.cn> 
Sent: den 10 mars 2020 02:30
To: Markus Gronlund <markus.gronlund at oracle.com>
Cc: hotspot-jfr-dev at openjdk.java.net; hotspot-dev at openjdk.java.net
Subject: Re: RFR: 8240222: [TESTBUG] gtest/jfr/test_networkUtilization.cpp failed when the number of tests is greater than or equal to 2

Thanks Markus.

Could you give some comments or advice on this one:
Webrev:? http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/ ?

Best regards,
Jia

? 2020/3/9 19:41, Markus Gronlund ??:
> The interface_id assigns a unique primary key for each interface.
>
> To support GTEST=REPEAT, I think the easiest way is to add a corresponding counter in the gtest itself that mirror the production code:
>
> That is, as an example, instead of the "add_interface("eth0", 1), that 
> hardcodes the traceid = 1, have it take a value from the corresponding 
> counter, i.e. "add_interface("eth0", next_id++)";
>
> Thanks
> Markus
>
>
>
> -----Original Message-----
> From: Jia Huang <huangjia at loongson.cn>
> Sent: den 9 mars 2020 10:57
> To: hotspot-dev at openjdk.java.net; hotspot-jfr-dev at openjdk.java.net
> Subject: Re: RFR: 8240222: [TESTBUG] 
> gtest/jfr/test_networkUtilization.cpp failed when the number of tests 
> is greater than or equal to 2
>
> In addition, the static local variable "id" is assigned in the function of "new_entry", that is in share/jfr/periodic/jfrNetworkUtilization.cpp.
>
> ---------------------------------------------------------------
> share/jfr/periodic/jfrNetworkUtilization.cpp
>
> static InterfaceEntry& new_entry(const NetworkInterface* iface,
> GrowableArray<InterfaceEntry>* interfaces) {
>   ? assert(iface != NULL, "invariant");
>   ? assert(interfaces != NULL, "invariant");
>
>   ? // single threaded premise
>   ? static traceid interface_id = 0;????????????????? ? ?? // this is 
> the static local variable
>
>   ? const char* name = iface->get_name();
>   ? assert(name != NULL, "invariant");
>
>   ? InterfaceEntry entry;
>   ? const size_t length = strlen(name);
>   ? entry.name = NEW_C_HEAP_ARRAY(char, length + 1, mtInternal);
>   ? strncpy(entry.name, name, length + 1);
>   ? entry.id = ++interface_id;
>   ? entry.bytes_in = iface->get_bytes_in();
>   ? entry.bytes_out = iface->get_bytes_out();
>   ? entry.written = false;
>   ? return _interfaces->at(_interfaces->append(entry));
> }
> ---------------------------------------------------------------
>
> ? 2020/3/9 15:29, Jia Huang ??:
>> Haha, I'm also trying to email hotspot-jfr-dev.
>> Thank you, David.
>>
>> ? 2020/3/9 13:03, David Holmes ??:
>>> Hi Jia,
>>>
>>> I see this is getting no response so have cc'd hotspot-jfr-dev as 
>>> they should have a better idea of how this gtest can and cannot be used.
>>>
>>> Just because the GTest framework supports "REPEAT" it doesn't mean 
>>> all tests can, or should, be repeatable.
>>>
>>> Cheers,
>>> David
>>>
>>> On 29/02/2020 1:53 pm, Jia Huang wrote:
>>>> Hi David,
>>>>
>>>> ? 2020/2/28 21:16, David Holmes ??:
>>>>> Hi Jia,
>>>>>
>>>>> On 28/02/2020 11:08 pm, Jia Huang wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8240222
>>>>>> Webrev: http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/
>>>>>>
>>>>>> gtest/jfr/test_networkUtilization.cpp failed when the number of 
>>>>>> tests is greater than or equal to 2.
>>>>>> As shown in the following code, the test failed because "traceid 
>>>>>> id" is a static local variable.
>>>>>> When the test runs a second time, "id" continues to grow, 
>>>>>> resulting in the return value of "i" being "_interfaces.end()".
>>>>> Interesting! I'm not at all familiar with the details of gtest and 
>>>>> how it actually runs things. But I would expect that we have many 
>>>>> tests that rely on starting from a cleanly initialized VM state, 
>>>>> or test state, and so can't simply be repeated (it would be like 
>>>>> editing an arbitrary test and putting the body of main into a 
>>>>> for-loop!).
>>>> I'm sorry that my last description may be a little incomplete.
>>>> JfrTestNetworkUtilization is a group of tests,it includes four 
>>>> individual tests:
>>>>  ?? --JfrTestNetworkUtilization.RequestFunctionBasic_test_vm
>>>>  ?? --JfrTestNetworkUtilization.RequestFunctionMultiple_test_vm
>>>>  ?? --JfrTestNetworkUtilization.InterfaceRemoved_test_vm
>>>>  ?? --JfrTestNetworkUtilization.InterfaceReset_test_vm
>>>>
>>>> If you only run a individual test once, such as the following 
>>>> Testing, it still fails.
>>>> The reason for the failure, as I said last time, is also because of 
>>>> static local variables.
>>>> In the "RequestFunctionMultiple_test_vm" test, the sequence number 
>>>> of "eth0" is 2.
>>>> However, the initial value of the static local variable "id" is 0.
>>>>
>>>> Testting:
>>>>  ??? --make run-test
>>>> TEST=gtest:JfrTestNetworkUtilization.RequestFunctionMultiple_test_v
>>>> m
>>>> GTEST=REPEAT=1 CONF=release
>>>>  ??? --make run-test
>>>> TEST=gtest:JfrTestNetworkUtilization.InterfaceRemoved_test_vm
>>>> GTEST=REPEAT=1 CONF=release
>>>>  ??? --make run-test
>>>> TEST=gtest:JfrTestNetworkUtilization.InterfaceReset_test_vm
>>>> GTEST=REPEAT=1 CONF=release
>>>>
>>>>> So I'm not sure what the right way to fix this is:
>>>>> - don't use REPEAT?
>>>>> - declare the test as not repeatable somehow?
>>>>> - change the test to make it repeatable?
>>>> In the doc/testing.md or doc/testing.html, Gtest keywords inclue 
>>>> "REPEAT", so I think this means that all GTEST tests should be able 
>>>> to use the "REPEAT parameter.
>>>>
>>>>
>>>> Best regards,
>>>> Jia
>>>>
>>>>
>>>>> Cheers,
>>>>> David
>>>>>
>>>>>> -----------------------------------------------------------------
>>>>>> -----------
>>>>>>
>>>>>> test/hotspot/gtest/jfr/test_networkUtilization.cpp
>>>>>>
>>>>>> static const MockNetworkInterface& get_interface(traceid id) {
>>>>>>  ?? std::list<MockNetworkInterface>::const_iterator i = 
>>>>>> _interfaces.begin();
>>>>>>  ?? for (; i != _interfaces.end(); ++i) {
>>>>>>  ???? if (i->id == id) {
>>>>>>  ?????? break;
>>>>>>  ???? }
>>>>>>  ?? }
>>>>>>  ?? return *i;
>>>>>> }
>>>>>> -----------------------------------------------------------------
>>>>>> -----------
>>>>>>
>>>>>>
>>>>>>
>>>>>> Testing:
>>>>>>  ???? - make run-test TEST=gtest:JfrTestNetworkUtilization
>>>>>> GTEST=REPEAT=2 CONF=server-release
>>>>>>  ???? - make run-test TEST=gtest:JfrTestNetworkUtilization
>>>>>> GTEST=REPEAT=2 CONF=server-fastdebug
>>>>>>
>>>>>> Thanks a lot
>>>>>>
>>>>>> Best regards,
>>>>>> Jia
>>>>>>


From huangjia at loongson.cn  Tue Mar 10 11:37:25 2020
From: huangjia at loongson.cn (Jia Huang)
Date: Tue, 10 Mar 2020 19:37:25 +0800
Subject: RFR: 8240222: [TESTBUG] gtest/jfr/test_networkUtilization.cpp
 failed when the number of tests is greater than or equal to 2
In-Reply-To: <50d5e790-ef5c-4c35-bc14-71577df50c9a@default>
References: <64960cc9-5021-d265-ab30-9cf9a91cc6e3@loongson.cn>
 <8f143c78-6bba-5f11-21ad-3096115287e1@oracle.com>
 <4f6c2613-de5a-cc28-f208-1a763270a5d0@loongson.cn>
 <44483657-df1f-f9cc-9e6a-f73b0e602f72@oracle.com>
 <931ecb2e-9898-a371-0aa9-bf463b9fdcf6@loongson.cn>
 <e9d5cca3-1f35-7708-1bdb-bb4b5e673d27@loongson.cn>
 <a5b687be-a9b4-4bc7-bace-b94f723cbdab@default>
 <74960f03-542c-f7f5-9f46-994d355d62d3@loongson.cn>
 <50d5e790-ef5c-4c35-bc14-71577df50c9a@default>
Message-ID: <d56604a5-5873-718d-eda0-86b95a7234b4@loongson.cn>

Thanks Markus.

Best regards,
Jia

? 2020/3/10 18:15, Markus Gronlund ??:
> Hi Jia,
>
> Looks like it could work.
>
> Thanks
> Markus
>
> -----Original Message-----
> From: Jia Huang <huangjia at loongson.cn>
> Sent: den 10 mars 2020 02:30
> To: Markus Gronlund <markus.gronlund at oracle.com>
> Cc: hotspot-jfr-dev at openjdk.java.net; hotspot-dev at openjdk.java.net
> Subject: Re: RFR: 8240222: [TESTBUG] gtest/jfr/test_networkUtilization.cpp failed when the number of tests is greater than or equal to 2
>
> Thanks Markus.
>
> Could you give some comments or advice on this one:
> Webrev:? http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/ ?
>
> Best regards,
> Jia
>
> ? 2020/3/9 19:41, Markus Gronlund ??:
>> The interface_id assigns a unique primary key for each interface.
>>
>> To support GTEST=REPEAT, I think the easiest way is to add a corresponding counter in the gtest itself that mirror the production code:
>>
>> That is, as an example, instead of the "add_interface("eth0", 1), that
>> hardcodes the traceid = 1, have it take a value from the corresponding
>> counter, i.e. "add_interface("eth0", next_id++)";
>>
>> Thanks
>> Markus
>>
>>
>>
>> -----Original Message-----
>> From: Jia Huang <huangjia at loongson.cn>
>> Sent: den 9 mars 2020 10:57
>> To: hotspot-dev at openjdk.java.net; hotspot-jfr-dev at openjdk.java.net
>> Subject: Re: RFR: 8240222: [TESTBUG]
>> gtest/jfr/test_networkUtilization.cpp failed when the number of tests
>> is greater than or equal to 2
>>
>> In addition, the static local variable "id" is assigned in the function of "new_entry", that is in share/jfr/periodic/jfrNetworkUtilization.cpp.
>>
>> ---------------------------------------------------------------
>> share/jfr/periodic/jfrNetworkUtilization.cpp
>>
>> static InterfaceEntry& new_entry(const NetworkInterface* iface,
>> GrowableArray<InterfaceEntry>* interfaces) {
>>    ? assert(iface != NULL, "invariant");
>>    ? assert(interfaces != NULL, "invariant");
>>
>>    ? // single threaded premise
>>    ? static traceid interface_id = 0;????????????????? ? ?? // this is
>> the static local variable
>>
>>    ? const char* name = iface->get_name();
>>    ? assert(name != NULL, "invariant");
>>
>>    ? InterfaceEntry entry;
>>    ? const size_t length = strlen(name);
>>    ? entry.name = NEW_C_HEAP_ARRAY(char, length + 1, mtInternal);
>>    ? strncpy(entry.name, name, length + 1);
>>    ? entry.id = ++interface_id;
>>    ? entry.bytes_in = iface->get_bytes_in();
>>    ? entry.bytes_out = iface->get_bytes_out();
>>    ? entry.written = false;
>>    ? return _interfaces->at(_interfaces->append(entry));
>> }
>> ---------------------------------------------------------------
>>
>> ? 2020/3/9 15:29, Jia Huang ??:
>>> Haha, I'm also trying to email hotspot-jfr-dev.
>>> Thank you, David.
>>>
>>> ? 2020/3/9 13:03, David Holmes ??:
>>>> Hi Jia,
>>>>
>>>> I see this is getting no response so have cc'd hotspot-jfr-dev as
>>>> they should have a better idea of how this gtest can and cannot be used.
>>>>
>>>> Just because the GTest framework supports "REPEAT" it doesn't mean
>>>> all tests can, or should, be repeatable.
>>>>
>>>> Cheers,
>>>> David
>>>>
>>>> On 29/02/2020 1:53 pm, Jia Huang wrote:
>>>>> Hi David,
>>>>>
>>>>> ? 2020/2/28 21:16, David Holmes ??:
>>>>>> Hi Jia,
>>>>>>
>>>>>> On 28/02/2020 11:08 pm, Jia Huang wrote:
>>>>>>> Hi all,
>>>>>>>
>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8240222
>>>>>>> Webrev: http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/
>>>>>>>
>>>>>>> gtest/jfr/test_networkUtilization.cpp failed when the number of
>>>>>>> tests is greater than or equal to 2.
>>>>>>> As shown in the following code, the test failed because "traceid
>>>>>>> id" is a static local variable.
>>>>>>> When the test runs a second time, "id" continues to grow,
>>>>>>> resulting in the return value of "i" being "_interfaces.end()".
>>>>>> Interesting! I'm not at all familiar with the details of gtest and
>>>>>> how it actually runs things. But I would expect that we have many
>>>>>> tests that rely on starting from a cleanly initialized VM state,
>>>>>> or test state, and so can't simply be repeated (it would be like
>>>>>> editing an arbitrary test and putting the body of main into a
>>>>>> for-loop!).
>>>>> I'm sorry that my last description may be a little incomplete.
>>>>> JfrTestNetworkUtilization is a group of tests,it includes four
>>>>> individual tests:
>>>>>   ?? --JfrTestNetworkUtilization.RequestFunctionBasic_test_vm
>>>>>   ?? --JfrTestNetworkUtilization.RequestFunctionMultiple_test_vm
>>>>>   ?? --JfrTestNetworkUtilization.InterfaceRemoved_test_vm
>>>>>   ?? --JfrTestNetworkUtilization.InterfaceReset_test_vm
>>>>>
>>>>> If you only run a individual test once, such as the following
>>>>> Testing, it still fails.
>>>>> The reason for the failure, as I said last time, is also because of
>>>>> static local variables.
>>>>> In the "RequestFunctionMultiple_test_vm" test, the sequence number
>>>>> of "eth0" is 2.
>>>>> However, the initial value of the static local variable "id" is 0.
>>>>>
>>>>> Testting:
>>>>>   ??? --make run-test
>>>>> TEST=gtest:JfrTestNetworkUtilization.RequestFunctionMultiple_test_v
>>>>> m
>>>>> GTEST=REPEAT=1 CONF=release
>>>>>   ??? --make run-test
>>>>> TEST=gtest:JfrTestNetworkUtilization.InterfaceRemoved_test_vm
>>>>> GTEST=REPEAT=1 CONF=release
>>>>>   ??? --make run-test
>>>>> TEST=gtest:JfrTestNetworkUtilization.InterfaceReset_test_vm
>>>>> GTEST=REPEAT=1 CONF=release
>>>>>
>>>>>> So I'm not sure what the right way to fix this is:
>>>>>> - don't use REPEAT?
>>>>>> - declare the test as not repeatable somehow?
>>>>>> - change the test to make it repeatable?
>>>>> In the doc/testing.md or doc/testing.html, Gtest keywords inclue
>>>>> "REPEAT", so I think this means that all GTEST tests should be able
>>>>> to use the "REPEAT parameter.
>>>>>
>>>>>
>>>>> Best regards,
>>>>> Jia
>>>>>
>>>>>
>>>>>> Cheers,
>>>>>> David
>>>>>>
>>>>>>> -----------------------------------------------------------------
>>>>>>> -----------
>>>>>>>
>>>>>>> test/hotspot/gtest/jfr/test_networkUtilization.cpp
>>>>>>>
>>>>>>> static const MockNetworkInterface& get_interface(traceid id) {
>>>>>>>   ?? std::list<MockNetworkInterface>::const_iterator i =
>>>>>>> _interfaces.begin();
>>>>>>>   ?? for (; i != _interfaces.end(); ++i) {
>>>>>>>   ???? if (i->id == id) {
>>>>>>>   ?????? break;
>>>>>>>   ???? }
>>>>>>>   ?? }
>>>>>>>   ?? return *i;
>>>>>>> }
>>>>>>> -----------------------------------------------------------------
>>>>>>> -----------
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Testing:
>>>>>>>   ???? - make run-test TEST=gtest:JfrTestNetworkUtilization
>>>>>>> GTEST=REPEAT=2 CONF=server-release
>>>>>>>   ???? - make run-test TEST=gtest:JfrTestNetworkUtilization
>>>>>>> GTEST=REPEAT=2 CONF=server-fastdebug
>>>>>>>
>>>>>>> Thanks a lot
>>>>>>>
>>>>>>> Best regards,
>>>>>>> Jia
>>>>>>>


From sgehwolf at redhat.com  Tue Mar 10 13:52:26 2020
From: sgehwolf at redhat.com (Severin Gehwolf)
Date: Tue, 10 Mar 2020 14:52:26 +0100
Subject: [8u] RFR: 8215355: Object monitor deadlock with no threads holding
 the monitor (using jemalloc 5.1)
Message-ID: <0ee88b2fa9e9fe03bfece97751e8ee48076852a6.camel@redhat.com>

Hi,

Could I please get a review of this OpenJDK 8u backport? The patch is
in Oracle JDK 8u and we should get it backported for parity. It doesn't
apply cleanly, but is rather trivial to be fixed up manually. The hunk
in thread.hpp didn't apply since JDK-8139864[1] is not in OpenJDK 8u.
Thus, changed '>=' to '>' manually in thread.hpp.

Bug: https://bugs.openjdk.java.net/browse/JDK-8215355
webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8215355/01/webrev/
Original change: https://hg.openjdk.java.net/jdk/jdk/rev/189f47d990b5

Testing: Reproducer in JDK-8215355 (fails prior/passes after), tier1 tests.

Thoughts?

Thanks,
Severin


From aph at redhat.com  Tue Mar 10 15:04:24 2020
From: aph at redhat.com (Andrew Haley)
Date: Tue, 10 Mar 2020 15:04:24 +0000
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <f500862b-80d9-a5eb-dd80-3cbbd7980e1a@oracle.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
 <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
 <e96653f2-3f5d-42cc-6d5e-8199a3caf8e8@oracle.com>
 <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com>
 <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com>
 <F9E331DA-B9FC-40E0-81FD-85CC25F45FD0@oracle.com>
 <a88c261a-799e-ba93-fbb8-79e488ee2a3b@redhat.com>
 <f500862b-80d9-a5eb-dd80-3cbbd7980e1a@oracle.com>
Message-ID: <d2f17edc-ceb0-0cea-ce49-030bec661c0e@redhat.com>

On 3/9/20 12:20 PM, Stefan Karlsson wrote:
> On 2020-03-09 11:54, Andrew Haley wrote:
>> So, my plan is to push http://cr.openjdk.java.net/~aph/8240615-1/ and
>> subsequently patch a few AArch64 cases plus the x86_64 immL_Pow2 and
>> immL_NotPow2 cases that we already identified.
>>
>> I do not intend to do bulk changes to any other clients of
>> is_power_of_2() because it's often unclear exactly what behaviour is
>> required, and I'd risk breaking something.
>>
>> OK?
> 
> I would prefer if the x86_64 and aarch64 patches were pushed first, or 
> folded into this patch. Otherwise, we knowingly introduce a regression 
> by pushing the proposed patch above. However, I'll leave it up to the 
> compiler devs/maintainers to decide if this is OK.

 http://cr.openjdk.java.net/~aph/8240615-2/

OK?

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From matthias.baesken at sap.com  Tue Mar 10 15:09:25 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 10 Mar 2020 15:09:25 +0000
Subject: RFR: 8240824: enhance print_full_memory_info on Linux by THP related
 information
Message-ID: <AM0PR02MB42272402734B0AD4B4E304CC93FF0@AM0PR02MB4227.eurprd02.prod.outlook.com>

Hello, please review this small enhancement .

On Linux, the change adds output of some files related to THP (transparent huge pages)   configuration to the hs_err file; the settings in those files configure how transparent huge pages are used on the system.

Example output in the hs_err file from a test machine :


/sys/kernel/mm/transparent_hugepage/enabled (transparent huge pages mode):
[always] madvise never

/sys/kernel/mm/transparent_hugepage/defrag (defrag/compaction efforts parameter):
[always] madvise never




Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8240824

http://cr.openjdk.java.net/~mbaesken/webrevs/8240824.0/


Thanks, Matthias


From stefan.karlsson at oracle.com  Tue Mar 10 15:12:46 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Tue, 10 Mar 2020 16:12:46 +0100
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <d2f17edc-ceb0-0cea-ce49-030bec661c0e@redhat.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
 <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
 <e96653f2-3f5d-42cc-6d5e-8199a3caf8e8@oracle.com>
 <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com>
 <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com>
 <F9E331DA-B9FC-40E0-81FD-85CC25F45FD0@oracle.com>
 <a88c261a-799e-ba93-fbb8-79e488ee2a3b@redhat.com>
 <f500862b-80d9-a5eb-dd80-3cbbd7980e1a@oracle.com>
 <d2f17edc-ceb0-0cea-ce49-030bec661c0e@redhat.com>
Message-ID: <51dfbbb3-f0cc-2210-a4f1-558a3a6a9ac7@oracle.com>

On 2020-03-10 16:04, Andrew Haley wrote:
> On 3/9/20 12:20 PM, Stefan Karlsson wrote:
>> On 2020-03-09 11:54, Andrew Haley wrote:
>>> So, my plan is to push http://cr.openjdk.java.net/~aph/8240615-1/ and
>>> subsequently patch a few AArch64 cases plus the x86_64 immL_Pow2 and
>>> immL_NotPow2 cases that we already identified.
>>>
>>> I do not intend to do bulk changes to any other clients of
>>> is_power_of_2() because it's often unclear exactly what behaviour is
>>> required, and I'd risk breaking something.
>>>
>>> OK?
>> I would prefer if the x86_64 and aarch64 patches were pushed first, or
>> folded into this patch. Otherwise, we knowingly introduce a regression
>> by pushing the proposed patch above. However, I'll leave it up to the
>> compiler devs/maintainers to decide if this is OK.
>   http://cr.openjdk.java.net/~aph/8240615-2/
>
> OK?

Looks good to me, but compiler devs need to review as well.

StefanK
>


From aph at redhat.com  Tue Mar 10 17:29:26 2020
From: aph at redhat.com (Andrew Haley)
Date: Tue, 10 Mar 2020 17:29:26 +0000
Subject: RFR: 8240829: Use a fast O(1) algorithm for exact_log2
Message-ID: <12280b6a-999f-4925-762f-6859cd330eaf@redhat.com>

exact_log2() is used a great deal in HotSpot, especially at
startup. exact_log2() uses a naive iterative algorithm. We should do
better, and in particular we should use an algorithm that compilers
commonly optimize well, so that constant expressions that use
exact_log2 are computed at compilation time.

GCC reliably does the constant propagation so that exact_log2(const)
calculations are always treated as constants, and even if the argument
is not a constant the runtime effort is no more than a multiplication,
a shift, and a load.

I also added a test. This is one of those few algorithms that can be
exhaustively tested, so I have very high confidence that it is correct.

OK?

http://cr.openjdk.java.net/~aph/8240829-1/

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From claes.redestad at oracle.com  Tue Mar 10 17:54:43 2020
From: claes.redestad at oracle.com (Claes Redestad)
Date: Tue, 10 Mar 2020 18:54:43 +0100
Subject: RFR: 8240829: Use a fast O(1) algorithm for exact_log2
In-Reply-To: <12280b6a-999f-4925-762f-6859cd330eaf@redhat.com>
References: <12280b6a-999f-4925-762f-6859cd330eaf@redhat.com>
Message-ID: <1782a730-b8e5-0b75-33aa-29b24c301b82@oracle.com>

Hi Andrew,

did you consider implementing these as [63,31] - 
count_leading_zeros(foo)? Or (sizeof(T) * BitsPerByte - 1 - lz),
as expressed in round_down_power_of_2. I've been meaning to try this
out now that exact_log2 has been freed from globalDefinitions.

count_leading_zeros use compiler intrinsics to use lzcnt and similar
instructions which are actually O(1) on many platforms - with a fallback
to a similar DeBruijn algorithm.

Thanks!

/Claes

On 2020-03-10 18:29, Andrew Haley wrote:
> exact_log2() is used a great deal in HotSpot, especially at
> startup. exact_log2() uses a naive iterative algorithm. We should do
> better, and in particular we should use an algorithm that compilers
> commonly optimize well, so that constant expressions that use
> exact_log2 are computed at compilation time.
> 
> GCC reliably does the constant propagation so that exact_log2(const)
> calculations are always treated as constants, and even if the argument
> is not a constant the runtime effort is no more than a multiplication,
> a shift, and a load.
> 
> I also added a test. This is one of those few algorithms that can be
> exhaustively tested, so I have very high confidence that it is correct.
> 
> OK?
> 
> http://cr.openjdk.java.net/~aph/8240829-1/
> 

From claes.redestad at oracle.com  Tue Mar 10 18:03:11 2020
From: claes.redestad at oracle.com (Claes Redestad)
Date: Tue, 10 Mar 2020 19:03:11 +0100
Subject: RFR: 8240829: Use a fast O(1) algorithm for exact_log2
In-Reply-To: <1782a730-b8e5-0b75-33aa-29b24c301b82@oracle.com>
References: <12280b6a-999f-4925-762f-6859cd330eaf@redhat.com>
 <1782a730-b8e5-0b75-33aa-29b24c301b82@oracle.com>
Message-ID: <8698a33e-dbe0-c414-e538-325bcb1bbc8e@oracle.com>

On 2020-03-10 18:54, Claes Redestad wrote:
> which are actually O(1)

Sorry, that came out wrong: DeBruijn-based LUT is technically constant
time, too. Just a costlier constant (several additions and at least
one multiplication).

/Claes

From aph at redhat.com  Tue Mar 10 18:05:33 2020
From: aph at redhat.com (Andrew Haley)
Date: Tue, 10 Mar 2020 18:05:33 +0000
Subject: RFR: 8240829: Use a fast O(1) algorithm for exact_log2
In-Reply-To: <1782a730-b8e5-0b75-33aa-29b24c301b82@oracle.com>
References: <12280b6a-999f-4925-762f-6859cd330eaf@redhat.com>
 <1782a730-b8e5-0b75-33aa-29b24c301b82@oracle.com>
Message-ID: <f0fc0553-66aa-3a0c-0afb-64fe1fc42c7c@redhat.com>

On 3/10/20 5:54 PM, Claes Redestad wrote:
> did you consider implementing these as [63,31] -
> count_leading_zeros(foo)? Or (sizeof(T) * BitsPerByte - 1 - lz),
> as expressed in round_down_power_of_2. I've been meaning to try this
> out now that exact_log2 has been freed from globalDefinitions.
>
> count_leading_zeros use compiler intrinsics to use lzcnt and similar
> instructions which are actually O(1) on many platforms - with a fallback
> to a similar DeBruijn algorithm.

Sure, I know. I should have mentioned that.

TL/DR: prefer portable code when possible.

I didn't do so because AFAIK count_leading_zeros() isn't part of
Standard C++ and I had no desire to try to find out what non-GCC
compilers do. If someone cares enough to get the last nanosecond out
of this, I guess that's fair, but it's such a small gain over this
algorithm that it wouldn't be noticeable.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From vladimir.kozlov at oracle.com  Tue Mar 10 18:06:40 2020
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Tue, 10 Mar 2020 11:06:40 -0700
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <d2f17edc-ceb0-0cea-ce49-030bec661c0e@redhat.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
 <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
 <e96653f2-3f5d-42cc-6d5e-8199a3caf8e8@oracle.com>
 <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com>
 <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com>
 <F9E331DA-B9FC-40E0-81FD-85CC25F45FD0@oracle.com>
 <a88c261a-799e-ba93-fbb8-79e488ee2a3b@redhat.com>
 <f500862b-80d9-a5eb-dd80-3cbbd7980e1a@oracle.com>
 <d2f17edc-ceb0-0cea-ce49-030bec661c0e@redhat.com>
Message-ID: <4d400304-d5cb-12ba-d772-3f594308355a@oracle.com>

Why you did not cast constant to (julong) in btrL_mem_imm() x86_64.ad?

I looked through C1 and C2 code usage of is_power_of_2(). And it is not clear to me if we handle MAX and MIN values 
correctly in some cases (mulnode.cpp, divnode.cpp). But it is a different issue.

I think your is_power_of_2() change should be fine for C2 and C1 code.

Thanks,
Vladimir

On 3/10/20 8:04 AM, Andrew Haley wrote:
> On 3/9/20 12:20 PM, Stefan Karlsson wrote:
>> On 2020-03-09 11:54, Andrew Haley wrote:
>>> So, my plan is to push http://cr.openjdk.java.net/~aph/8240615-1/ and
>>> subsequently patch a few AArch64 cases plus the x86_64 immL_Pow2 and
>>> immL_NotPow2 cases that we already identified.
>>>
>>> I do not intend to do bulk changes to any other clients of
>>> is_power_of_2() because it's often unclear exactly what behaviour is
>>> required, and I'd risk breaking something.
>>>
>>> OK?
>>
>> I would prefer if the x86_64 and aarch64 patches were pushed first, or
>> folded into this patch. Otherwise, we knowingly introduce a regression
>> by pushing the proposed patch above. However, I'll leave it up to the
>> compiler devs/maintainers to decide if this is OK.
> 
>   http://cr.openjdk.java.net/~aph/8240615-2/
> 
> OK?
> 

From aph at redhat.com  Tue Mar 10 18:14:39 2020
From: aph at redhat.com (Andrew Haley)
Date: Tue, 10 Mar 2020 18:14:39 +0000
Subject: RFR: 8240829: Use a fast O(1) algorithm for exact_log2
In-Reply-To: <f0fc0553-66aa-3a0c-0afb-64fe1fc42c7c@redhat.com>
References: <12280b6a-999f-4925-762f-6859cd330eaf@redhat.com>
 <1782a730-b8e5-0b75-33aa-29b24c301b82@oracle.com>
 <f0fc0553-66aa-3a0c-0afb-64fe1fc42c7c@redhat.com>
Message-ID: <2eab0686-fb26-3ba6-8f50-d321b3670f01@redhat.com>

On 3/10/20 6:05 PM, Andrew Haley wrote:
> TL/DR: prefer portable code when possible.

And -- of course -- I never saw the count_leading_zeros() !

Why on Earth did anyone do all of the count_leading_zeros() work but
*not use it* in one of the hottest parts of HotSpot? Incredible.

It never even occurred to me that there might be definitions for
count_leading_zeros that were not used for exact_log2(). I just
saw it and thought "that's horrible."

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From aph at redhat.com  Tue Mar 10 18:19:29 2020
From: aph at redhat.com (Andrew Haley)
Date: Tue, 10 Mar 2020 18:19:29 +0000
Subject: RFR: 8240829: Use a fast O(1) algorithm for exact_log2
In-Reply-To: <2eab0686-fb26-3ba6-8f50-d321b3670f01@redhat.com>
References: <12280b6a-999f-4925-762f-6859cd330eaf@redhat.com>
 <1782a730-b8e5-0b75-33aa-29b24c301b82@oracle.com>
 <f0fc0553-66aa-3a0c-0afb-64fe1fc42c7c@redhat.com>
 <2eab0686-fb26-3ba6-8f50-d321b3670f01@redhat.com>
Message-ID: <235cfb5c-b004-739d-5b57-0a05db2844cc@redhat.com>

On 3/10/20 6:14 PM, Andrew Haley wrote:
> On 3/10/20 6:05 PM, Andrew Haley wrote:
>> TL/DR: prefer portable code when possible.
> 
> And -- of course -- I never saw the count_leading_zeros() !

And finally: unless you have a burning desire to do it yourself, I'll make
another patch using count_leading_zeros(). OK?

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From aph at redhat.com  Tue Mar 10 18:25:10 2020
From: aph at redhat.com (Andrew Haley)
Date: Tue, 10 Mar 2020 18:25:10 +0000
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <4d400304-d5cb-12ba-d772-3f594308355a@oracle.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
 <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
 <e96653f2-3f5d-42cc-6d5e-8199a3caf8e8@oracle.com>
 <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com>
 <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com>
 <F9E331DA-B9FC-40E0-81FD-85CC25F45FD0@oracle.com>
 <a88c261a-799e-ba93-fbb8-79e488ee2a3b@redhat.com>
 <f500862b-80d9-a5eb-dd80-3cbbd7980e1a@oracle.com>
 <d2f17edc-ceb0-0cea-ce49-030bec661c0e@redhat.com>
 <4d400304-d5cb-12ba-d772-3f594308355a@oracle.com>
Message-ID: <af867ab9-1350-90dd-e17f-27b07132730f@redhat.com>

On 3/10/20 6:06 PM, Vladimir Kozlov wrote:
> Why you did not cast constant to (julong) in btrL_mem_imm() x86_64.ad?

I don't think that it's necessary. (~$con$$constant) returns a jlong
which gets passed to log2_long(julong x), so it gets converted to
from julong to julong at the point of the call.

> I looked through C1 and C2 code usage of is_power_of_2(). And it is not clear to me if we handle MAX and MIN values 
> correctly in some cases (mulnode.cpp, divnode.cpp). But it is a different issue.
> 
> I think your is_power_of_2() change should be fine for C2 and C1 code.

OK, thanks.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From claes.redestad at oracle.com  Tue Mar 10 18:31:46 2020
From: claes.redestad at oracle.com (Claes Redestad)
Date: Tue, 10 Mar 2020 18:31:46 +0000 (UTC)
Subject: RFR: 8240829: Use a fast O(1) algorithm for exact_log2
In-Reply-To: <235cfb5c-b004-739d-5b57-0a05db2844cc@redhat.com>
References: <12280b6a-999f-4925-762f-6859cd330eaf@redhat.com>
 <1782a730-b8e5-0b75-33aa-29b24c301b82@oracle.com>
 <f0fc0553-66aa-3a0c-0afb-64fe1fc42c7c@redhat.com>
 <2eab0686-fb26-3ba6-8f50-d321b3670f01@redhat.com>
 <235cfb5c-b004-739d-5b57-0a05db2844cc@redhat.com>
Message-ID: <9eefee97-2289-eee8-8775-56bf34f4f8d2@oracle.com>



On 2020-03-10 19:19, Andrew Haley wrote:
> On 3/10/20 6:14 PM, Andrew Haley wrote:
>> On 3/10/20 6:05 PM, Andrew Haley wrote:
>>> TL/DR: prefer portable code when possible.
>>
>> And -- of course -- I never saw the count_leading_zeros() !
> 
> And finally: unless you have a burning desire to do it yourself, I'll make
> another patch using count_leading_zeros(). OK?
> 

Feel free!

I added count_leading_zeros early last year to speed up some hot pieces
of code in C2. I did briefly look at re-implementing exact_log2 using
it, but since that was firmly embedded inside globalDefinitions I wasn't
sure I could pull off the code shuffle necessary to do so at the time
(and exact_log2 wasn't a bottleneck in the things I was looking at)

/Claes

From aph at redhat.com  Tue Mar 10 18:28:34 2020
From: aph at redhat.com (Andrew Haley)
Date: Tue, 10 Mar 2020 18:28:34 +0000
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <af867ab9-1350-90dd-e17f-27b07132730f@redhat.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
 <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
 <e96653f2-3f5d-42cc-6d5e-8199a3caf8e8@oracle.com>
 <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com>
 <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com>
 <F9E331DA-B9FC-40E0-81FD-85CC25F45FD0@oracle.com>
 <a88c261a-799e-ba93-fbb8-79e488ee2a3b@redhat.com>
 <f500862b-80d9-a5eb-dd80-3cbbd7980e1a@oracle.com>
 <d2f17edc-ceb0-0cea-ce49-030bec661c0e@redhat.com>
 <4d400304-d5cb-12ba-d772-3f594308355a@oracle.com>
 <af867ab9-1350-90dd-e17f-27b07132730f@redhat.com>
Message-ID: <0d9a51b2-f366-70e0-b0ac-aab185b6081a@redhat.com>

On 3/10/20 6:25 PM, Andrew Haley wrote:
> I don't think that it's necessary. (~$con$$constant) returns a jlong
> which gets passed to log2_long(julong x), so it gets converted to
> from julong to julong at the point of the call.

Uh, from jlong to julong.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From vladimir.kozlov at oracle.com  Tue Mar 10 18:57:57 2020
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Tue, 10 Mar 2020 11:57:57 -0700
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <0d9a51b2-f366-70e0-b0ac-aab185b6081a@redhat.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
 <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
 <e96653f2-3f5d-42cc-6d5e-8199a3caf8e8@oracle.com>
 <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com>
 <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com>
 <F9E331DA-B9FC-40E0-81FD-85CC25F45FD0@oracle.com>
 <a88c261a-799e-ba93-fbb8-79e488ee2a3b@redhat.com>
 <f500862b-80d9-a5eb-dd80-3cbbd7980e1a@oracle.com>
 <d2f17edc-ceb0-0cea-ce49-030bec661c0e@redhat.com>
 <4d400304-d5cb-12ba-d772-3f594308355a@oracle.com>
 <af867ab9-1350-90dd-e17f-27b07132730f@redhat.com>
 <0d9a51b2-f366-70e0-b0ac-aab185b6081a@redhat.com>
Message-ID: <f375a905-d796-d23f-fc87-8ddb1205a06a@oracle.com>

Okay.

Thanks,
Vladimir

On 3/10/20 11:28 AM, Andrew Haley wrote:
> On 3/10/20 6:25 PM, Andrew Haley wrote:
>> I don't think that it's necessary. (~$con$$constant) returns a jlong
>> which gets passed to log2_long(julong x), so it gets converted to
>> from julong to julong at the point of the call.
> 
> Uh, from jlong to julong.
> 

From vladimir.kozlov at oracle.com  Tue Mar 10 21:35:05 2020
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Tue, 10 Mar 2020 14:35:05 -0700
Subject: [15] RFR(T) [BACKOUT] 8240195: some jaotc failures of fastdebug build
 with specific flags
Message-ID: <606cb769-7a79-10a6-28de-e049c98bfcac@oracle.com>

https://bugs.openjdk.java.net/browse/JDK-8240830
http://cr.openjdk.java.net/~kvn/8240830/webrev.00/

8240195 caused timeouts and other failures in all testing tiers including tier1.

I ran fastdebug build through tier1 build and testing.

Thanks,
Vladimir

From daniel.daugherty at oracle.com  Tue Mar 10 21:37:21 2020
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Tue, 10 Mar 2020 17:37:21 -0400
Subject: [15] RFR(T) [BACKOUT] 8240195: some jaotc failures of fastdebug
 build with specific flags
In-Reply-To: <606cb769-7a79-10a6-28de-e049c98bfcac@oracle.com>
References: <606cb769-7a79-10a6-28de-e049c98bfcac@oracle.com>
Message-ID: <68d2a0ce-eab8-716a-8fde-57b5df71eb82@oracle.com>

Thumbs up! I've verified Vladimir's backout with my own and they match.
I verified my backout with manual diffs against the parent of 8240195.

Dan

On 3/10/20 5:35 PM, Vladimir Kozlov wrote:
> https://bugs.openjdk.java.net/browse/JDK-8240830
> http://cr.openjdk.java.net/~kvn/8240830/webrev.00/
>
> 8240195 caused timeouts and other failures in all testing tiers 
> including tier1.
>
> I ran fastdebug build through tier1 build and testing.
>
> Thanks,
> Vladimir


From vladimir.kozlov at oracle.com  Tue Mar 10 21:40:45 2020
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Tue, 10 Mar 2020 14:40:45 -0700
Subject: [15] RFR(T) [BACKOUT] 8240195: some jaotc failures of fastdebug
 build with specific flags
In-Reply-To: <68d2a0ce-eab8-716a-8fde-57b5df71eb82@oracle.com>
References: <606cb769-7a79-10a6-28de-e049c98bfcac@oracle.com>
 <68d2a0ce-eab8-716a-8fde-57b5df71eb82@oracle.com>
Message-ID: <8f7e7fe0-6629-c2d8-6557-b3cbe5e9f4f6@oracle.com>

Thank you, Dan

I pushed it.

Vladimir K

On 3/10/20 2:37 PM, Daniel D. Daugherty wrote:
> Thumbs up! I've verified Vladimir's backout with my own and they match.
> I verified my backout with manual diffs against the parent of 8240195.
> 
> Dan
> 
> On 3/10/20 5:35 PM, Vladimir Kozlov wrote:
>> https://bugs.openjdk.java.net/browse/JDK-8240830
>> http://cr.openjdk.java.net/~kvn/8240830/webrev.00/
>>
>> 8240195 caused timeouts and other failures in all testing tiers including tier1.
>>
>> I ran fastdebug build through tier1 build and testing.
>>
>> Thanks,
>> Vladimir
> 

From david.holmes at oracle.com  Tue Mar 10 23:05:14 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 11 Mar 2020 09:05:14 +1000
Subject: [8u] RFR: 8215355: Object monitor deadlock with no threads
 holding the monitor (using jemalloc 5.1)
In-Reply-To: <0ee88b2fa9e9fe03bfece97751e8ee48076852a6.camel@redhat.com>
References: <0ee88b2fa9e9fe03bfece97751e8ee48076852a6.camel@redhat.com>
Message-ID: <357e6864-5e97-8b67-6906-f2765064b322@oracle.com>

Hi Severin,

LGTM.

Thanks,
David

On 10/03/2020 11:52 pm, Severin Gehwolf wrote:
> Hi,
> 
> Could I please get a review of this OpenJDK 8u backport? The patch is
> in Oracle JDK 8u and we should get it backported for parity. It doesn't
> apply cleanly, but is rather trivial to be fixed up manually. The hunk
> in thread.hpp didn't apply since JDK-8139864[1] is not in OpenJDK 8u.
> Thus, changed '>=' to '>' manually in thread.hpp.
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8215355
> webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8215355/01/webrev/
> Original change: https://hg.openjdk.java.net/jdk/jdk/rev/189f47d990b5
> 
> Testing: Reproducer in JDK-8215355 (fails prior/passes after), tier1 tests.
> 
> Thoughts?
> 
> Thanks,
> Severin
> 

From john.r.rose at oracle.com  Tue Mar 10 23:24:25 2020
From: john.r.rose at oracle.com (John Rose)
Date: Tue, 10 Mar 2020 16:24:25 -0700
Subject: RFR: 8240829: Use a fast O(1) algorithm for exact_log2
In-Reply-To: <235cfb5c-b004-739d-5b57-0a05db2844cc@redhat.com>
References: <12280b6a-999f-4925-762f-6859cd330eaf@redhat.com>
 <1782a730-b8e5-0b75-33aa-29b24c301b82@oracle.com>
 <f0fc0553-66aa-3a0c-0afb-64fe1fc42c7c@redhat.com>
 <2eab0686-fb26-3ba6-8f50-d321b3670f01@redhat.com>
 <235cfb5c-b004-739d-5b57-0a05db2844cc@redhat.com>
Message-ID: <3AFF154B-F75B-44DB-B3F4-4162F19E2FC7@oracle.com>

On Mar 10, 2020, at 11:19 AM, Andrew Haley <aph at redhat.com> wrote:
> 
> On 3/10/20 6:14 PM, Andrew Haley wrote:
>> On 3/10/20 6:05 PM, Andrew Haley wrote:
>>> TL/DR: prefer portable code when possible.
>> 
>> And -- of course -- I never saw the count_leading_zeros() !
> 
> And finally: unless you have a burning desire to do it yourself, I'll make
> another patch using count_leading_zeros(). OK?

Yes, please build exact_log2 on top of count_leading_zeroes,
which (in turn) resorts to DeB multiply if necessary.

From sgehwolf at redhat.com  Wed Mar 11 08:33:14 2020
From: sgehwolf at redhat.com (Severin Gehwolf)
Date: Wed, 11 Mar 2020 09:33:14 +0100
Subject: [8u] RFR: 8215355: Object monitor deadlock with no threads
 holding the monitor (using jemalloc 5.1)
In-Reply-To: <357e6864-5e97-8b67-6906-f2765064b322@oracle.com>
References: <0ee88b2fa9e9fe03bfece97751e8ee48076852a6.camel@redhat.com>
 <357e6864-5e97-8b67-6906-f2765064b322@oracle.com>
Message-ID: <27206a62ea0421058241e653503b28f2a13fbd35.camel@redhat.com>

On Wed, 2020-03-11 at 09:05 +1000, David Holmes wrote:
> Hi Severin,
> 
> LGTM.

Thanks for the review, David!

Cheers,
Severin

> Thanks,
> David
> 
> On 10/03/2020 11:52 pm, Severin Gehwolf wrote:
> > Hi,
> > 
> > Could I please get a review of this OpenJDK 8u backport? The patch is
> > in Oracle JDK 8u and we should get it backported for parity. It doesn't
> > apply cleanly, but is rather trivial to be fixed up manually. The hunk
> > in thread.hpp didn't apply since JDK-8139864[1] is not in OpenJDK 8u.
> > Thus, changed '>=' to '>' manually in thread.hpp.
> > 
> > Bug: https://bugs.openjdk.java.net/browse/JDK-8215355
> > webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8215355/01/webrev/
> > Original change: https://hg.openjdk.java.net/jdk/jdk/rev/189f47d990b5
> > 
> > Testing: Reproducer in JDK-8215355 (fails prior/passes after), tier1 tests.
> > 
> > Thoughts?
> > 
> > Thanks,
> > Severin
> > 


From xxinliu at amazon.com  Wed Mar 11 08:50:03 2020
From: xxinliu at amazon.com (Liu, Xin)
Date: Wed, 11 Mar 2020 08:50:03 +0000
Subject: Propose to use clang-format to enforce hotspot codestyle
Message-ID: <84B163F2-5B97-438F-ACA4-CF84E6FB554C@amazon.com>

Hi, 

I just filed a JBS issue JDK-8240834 about hotspot codestyle(https://wiki.openjdk.java.net/display/HotSpot/StyleGuide). I found many  inconsistent code in hotspot so I propose to use clang-format[1] to enforce it. 
 
I know you?re all very experienced developers, but sometimes wrong indentations or tailing whitespaces creep in.  it's hard to catch them in webrev.  Even though we can, new contributors still need to contact their sponsors for formatting changes.   I think it's a good idea to have lint thing done at first place. 

Clang is powerful enough to parse modern C/C++ code and it's very handy to format code. clang-format should be available on all supporting platforms. It's still okay even if it is absent on some platforms because nightly build and submit repo will cover unsupported platforms.

I plan to have  a .clang-format and correcting styles in the 1st patch. We can bring auto clang-format check to hotspot's makefile in the second patch.  Does it sound like a plan? 
 
Reference: 
[1] https://clang.llvm.org/docs/ClangFormat.html
Thanks,
--lx
	


From glaubitz at physik.fu-berlin.de  Wed Mar 11 08:54:02 2020
From: glaubitz at physik.fu-berlin.de (John Paul Adrian Glaubitz)
Date: Wed, 11 Mar 2020 09:54:02 +0100
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <84B163F2-5B97-438F-ACA4-CF84E6FB554C@amazon.com>
References: <84B163F2-5B97-438F-ACA4-CF84E6FB554C@amazon.com>
Message-ID: <bff76dc3-8cc3-eeba-7a10-63691a69d42b@physik.fu-berlin.de>

On 3/11/20 9:50 AM, Liu, Xin wrote:
> I plan to have  a .clang-format and correcting styles in the 1st patch. We can bring auto clang-format check to hotspot's makefile in the second patch.  Does it sound like a plan? 

Please make the use of clang optional for the build as otherwise you make
OpenJDK unbuildable on a number of Zero architectures.

Clang isn't unfortunately as widely ported as GCC so any LLVM/Clang dependency
is also a risk of reducing portability.

Thanks,
Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz at debian.org
`. `'   Freie Universitaet Berlin - glaubitz at physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

From rwestrel at redhat.com  Wed Mar 11 10:25:08 2020
From: rwestrel at redhat.com (Roland Westrelin)
Date: Wed, 11 Mar 2020 11:25:08 +0100
Subject: [15] RFR(T) [BACKOUT] 8240195: some jaotc failures of fastdebug
 build with specific flags
In-Reply-To: <606cb769-7a79-10a6-28de-e049c98bfcac@oracle.com>
References: <606cb769-7a79-10a6-28de-e049c98bfcac@oracle.com>
Message-ID: <87sgifjiij.fsf@redhat.com>


Thanks for backing out the broken change.

Roland.


From volker.simonis at gmail.com  Wed Mar 11 12:06:16 2020
From: volker.simonis at gmail.com (Volker Simonis)
Date: Wed, 11 Mar 2020 13:06:16 +0100
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <84B163F2-5B97-438F-ACA4-CF84E6FB554C@amazon.com>
References: <84B163F2-5B97-438F-ACA4-CF84E6FB554C@amazon.com>
Message-ID: <CA+3eh11y5pEXV4dC-=u4P-JMpd8QT5qr+wY0aqdpRMkiVM3kug@mail.gmail.com>

Liu, Xin <xxinliu at amazon.com> schrieb am Mi., 11. M?rz 2020, 09:50:

> Hi,
>
> I just filed a JBS issue JDK-8240834 about hotspot codestyle(
> https://wiki.openjdk.java.net/display/HotSpot/StyleGuide). I found many
> inconsistent code in hotspot so I propose to use clang-format[1] to enforce
> it.
>
> I know you?re all very experienced developers, but sometimes wrong
> indentations or tailing whitespaces creep in.  it's hard to catch them in
> webrev.  Even though we can, new contributors still need to contact their
> sponsors for formatting changes.   I think it's a good idea to have lint
> thing done at first place.
>
> Clang is powerful enough to parse modern C/C++ code and it's very handy to
> format code. clang-format should be available on all supporting platforms.
> It's still okay even if it is absent on some platforms because nightly
> build and submit repo will cover unsupported platforms.
>
> I plan to have  a .clang-format and correcting styles in the 1st patch. We
> can bring auto clang-format check to hotspot's makefile in the second
> patch.  Does it sound like a plan?
>

Have you already checked how big this first change will be? I'm afraid it
will be huge because we have a lot of manually formatted code which doesn't
confirm to any specific style (e.g. macros). We also have consistent but
different code styles in various parts of HS. So changing all the sources
to conform to a single style will necessarily be quite invasive and I'm not
sure if we want that?


> Reference:
> [1] https://clang.llvm.org/docs/ClangFormat.html
> Thanks,
> --lx
>
>
>

From aph at redhat.com  Wed Mar 11 12:41:37 2020
From: aph at redhat.com (Andrew Haley)
Date: Wed, 11 Mar 2020 12:41:37 +0000
Subject: RFR: 8240829: Use a fast O(1) algorithm for exact_log2
In-Reply-To: <3AFF154B-F75B-44DB-B3F4-4162F19E2FC7@oracle.com>
References: <12280b6a-999f-4925-762f-6859cd330eaf@redhat.com>
 <1782a730-b8e5-0b75-33aa-29b24c301b82@oracle.com>
 <f0fc0553-66aa-3a0c-0afb-64fe1fc42c7c@redhat.com>
 <2eab0686-fb26-3ba6-8f50-d321b3670f01@redhat.com>
 <235cfb5c-b004-739d-5b57-0a05db2844cc@redhat.com>
 <3AFF154B-F75B-44DB-B3F4-4162F19E2FC7@oracle.com>
Message-ID: <47eabcb4-a965-6db0-3658-006af3bc3b53@redhat.com>

http://cr.openjdk.java.net/~aph/8240829-2/

Tested x86: 32- and 64-bit.

OK?

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From claes.redestad at oracle.com  Wed Mar 11 12:56:32 2020
From: claes.redestad at oracle.com (Claes Redestad)
Date: Wed, 11 Mar 2020 13:56:32 +0100
Subject: RFR: 8240829: Use a fast O(1) algorithm for exact_log2
In-Reply-To: <47eabcb4-a965-6db0-3658-006af3bc3b53@redhat.com>
References: <12280b6a-999f-4925-762f-6859cd330eaf@redhat.com>
 <1782a730-b8e5-0b75-33aa-29b24c301b82@oracle.com>
 <f0fc0553-66aa-3a0c-0afb-64fe1fc42c7c@redhat.com>
 <2eab0686-fb26-3ba6-8f50-d321b3670f01@redhat.com>
 <235cfb5c-b004-739d-5b57-0a05db2844cc@redhat.com>
 <3AFF154B-F75B-44DB-B3F4-4162F19E2FC7@oracle.com>
 <47eabcb4-a965-6db0-3658-006af3bc3b53@redhat.com>
Message-ID: <322e4ca8-7985-c744-c6b9-396d626478c0@oracle.com>

On 2020-03-11 13:41, Andrew Haley wrote:
> http://cr.openjdk.java.net/~aph/8240829-2/
> 
> Tested x86: 32- and 64-bit.
> 
> OK?
> 

OK!

/Claes

From aph at redhat.com  Wed Mar 11 13:03:47 2020
From: aph at redhat.com (Andrew Haley)
Date: Wed, 11 Mar 2020 13:03:47 +0000
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <84B163F2-5B97-438F-ACA4-CF84E6FB554C@amazon.com>
References: <84B163F2-5B97-438F-ACA4-CF84E6FB554C@amazon.com>
Message-ID: <cb86fbce-ccf5-f96c-2d74-fdc85eb58187@redhat.com>

On 3/11/20 8:50 AM, Liu, Xin wrote:

> I just filed a JBS issue JDK-8240834 about hotspot codestyle
> (https://wiki.openjdk.java.net/display/HotSpot/StyleGuide). I found
> many inconsistent code in hotspot so I propose to use
> clang-format[1] to enforce it.

[1] "Occasionally a guideline mentioned here may be just out of synch
with the actual Hotspot code base. That's why we're using a wiki to
document the guidelines."

Note that these are guidelines. Sometimes the style used is different,
and sometimes that's for a good reason.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From aph at redhat.com  Wed Mar 11 15:46:55 2020
From: aph at redhat.com (Andrew Haley)
Date: Wed, 11 Mar 2020 15:46:55 +0000
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <AM0PR0202MB3297869C05C5A950564C3C8C9AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
 <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
 <e96653f2-3f5d-42cc-6d5e-8199a3caf8e8@oracle.com>
 <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com>
 <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com>
 <F9E331DA-B9FC-40E0-81FD-85CC25F45FD0@oracle.com>
 <a88c261a-799e-ba93-fbb8-79e488ee2a3b@redhat.com>
 <f500862b-80d9-a5eb-dd80-3cbbd7980e1a@oracle.com>
 <e0792f13-25b6-8d57-50f6-50239ef2150d@redhat.com>
 <AM0PR0202MB3297ED1D3059110035E679DF9AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
 <AM0PR0202MB3297D74513C3E2C7F4C8B4459AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
 <1c40a7d1-fa36-b5d2-6072-eaa243cc9397@redhat.com>
 <AM0PR0202MB3297869C05C5A950564C3C8C9AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
Message-ID: <2343a628-d5d0-5042-649f-150ab649d1e3@redhat.com>

On 3/9/20 3:56 PM, Doerr, Martin wrote:
> it doesn't run into the UB case, but log2_jint(x) returns 63 for negative x.
> I'd treat this as bug. A jint has 32 bit and log2 should never return 63.

I guess this bug dates from the days when uintptr_t was 32 bits.

I think you're right, but this has never been a problem because all uses
are either guarded by is_power_of_2() or they occur in a context (such as,
say, os::active_processor_count()) where > 2**31 is extremely unlikely.

I'm happy to fix this, though, if anyone thinks I should. The fix is:

diff -r f9893c227e12 src/hotspot/share/utilities/globalDefinitions.hpp
--- a/src/hotspot/share/utilities/globalDefinitions.hpp	Wed Mar 11 15:02:09 2020 +0000
+++ b/src/hotspot/share/utilities/globalDefinitions.hpp	Wed Mar 11 15:45:26 2020 +0000
@@ -1022,12 +1022,12 @@

 inline int log2_int(int x) {
   STATIC_ASSERT(sizeof(int) <= sizeof(uintptr_t));
-  return log2_intptr((uintptr_t)x);
+  return log2_intptr((uintptr_t)(unsigned int)x);
 }

 inline int log2_jint(jint x) {
   STATIC_ASSERT(sizeof(jint) <= sizeof(uintptr_t));
-  return log2_intptr((uintptr_t)x);
+  return log2_intptr((uintptr_t)(juint)x);
 }

 inline int log2_uint(uint x) {

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From martin.doerr at sap.com  Wed Mar 11 16:27:24 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Wed, 11 Mar 2020 16:27:24 +0000
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <2343a628-d5d0-5042-649f-150ab649d1e3@redhat.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
 <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
 <e96653f2-3f5d-42cc-6d5e-8199a3caf8e8@oracle.com>
 <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com>
 <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com>
 <F9E331DA-B9FC-40E0-81FD-85CC25F45FD0@oracle.com>
 <a88c261a-799e-ba93-fbb8-79e488ee2a3b@redhat.com>
 <f500862b-80d9-a5eb-dd80-3cbbd7980e1a@oracle.com>
 <e0792f13-25b6-8d57-50f6-50239ef2150d@redhat.com>
 <AM0PR0202MB3297ED1D3059110035E679DF9AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
 <AM0PR0202MB3297D74513C3E2C7F4C8B4459AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
 <1c40a7d1-fa36-b5d2-6072-eaa243cc9397@redhat.com>
 <AM0PR0202MB3297869C05C5A950564C3C8C9AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
 <2343a628-d5d0-5042-649f-150ab649d1e3@redhat.com>
Message-ID: <AM0PR0202MB32970A25D2592BB5E70358F89AFC0@AM0PR0202MB3297.eurprd02.prod.outlook.com>

Hi Andrew,

I think it's worth fixing as you suggest.
Current usages don't have a problem, but who knows about future usages?

Thanks and best regards,
Martin


> -----Original Message-----
> From: Andrew Haley <aph at redhat.com>
> Sent: Mittwoch, 11. M?rz 2020 16:47
> To: Doerr, Martin <martin.doerr at sap.com>; Stefan Karlsson
> <stefan.karlsson at oracle.com>; John Rose <john.r.rose at oracle.com>
> Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net
> Subject: Re: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
> inconsistent
> 
> On 3/9/20 3:56 PM, Doerr, Martin wrote:
> > it doesn't run into the UB case, but log2_jint(x) returns 63 for negative x.
> > I'd treat this as bug. A jint has 32 bit and log2 should never return 63.
> 
> I guess this bug dates from the days when uintptr_t was 32 bits.
> 
> I think you're right, but this has never been a problem because all uses
> are either guarded by is_power_of_2() or they occur in a context (such as,
> say, os::active_processor_count()) where > 2**31 is extremely unlikely.
> 
> I'm happy to fix this, though, if anyone thinks I should. The fix is:
> 
> diff -r f9893c227e12 src/hotspot/share/utilities/globalDefinitions.hpp
> --- a/src/hotspot/share/utilities/globalDefinitions.hpp	Wed Mar 11
> 15:02:09 2020 +0000
> +++ b/src/hotspot/share/utilities/globalDefinitions.hpp	Wed Mar 11
> 15:45:26 2020 +0000
> @@ -1022,12 +1022,12 @@
> 
>  inline int log2_int(int x) {
>    STATIC_ASSERT(sizeof(int) <= sizeof(uintptr_t));
> -  return log2_intptr((uintptr_t)x);
> +  return log2_intptr((uintptr_t)(unsigned int)x);
>  }
> 
>  inline int log2_jint(jint x) {
>    STATIC_ASSERT(sizeof(jint) <= sizeof(uintptr_t));
> -  return log2_intptr((uintptr_t)x);
> +  return log2_intptr((uintptr_t)(juint)x);
>  }
> 
>  inline int log2_uint(uint x) {
> 
> --
> Andrew Haley  (he/him)
> Java Platform Lead Engineer
> Red Hat UK Ltd. <https://www.redhat.com>
> https://keybase.io/andrewhaley
> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From vladimir.kozlov at oracle.com  Wed Mar 11 18:04:46 2020
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Wed, 11 Mar 2020 11:04:46 -0700
Subject: [15] RFR(T) [BACKOUT] 8240195: some jaotc failures of fastdebug
 build with specific flags
In-Reply-To: <87sgifjiij.fsf@redhat.com>
References: <606cb769-7a79-10a6-28de-e049c98bfcac@oracle.com>
 <87sgifjiij.fsf@redhat.com>
Message-ID: <26f16a1a-4f20-3964-c163-6215addc4180@oracle.com>

NP. But we all should remember before push to run at least tier1 testing or submit repo or ask sponsor to do that.

Vladimir K

On 3/11/20 3:25 AM, Roland Westrelin wrote:
> 
> Thanks for backing out the broken change.
> 
> Roland.
> 

From jesper.wilhelmsson at oracle.com  Wed Mar 11 18:13:07 2020
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Wed, 11 Mar 2020 19:13:07 +0100
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <CA+3eh11y5pEXV4dC-=u4P-JMpd8QT5qr+wY0aqdpRMkiVM3kug@mail.gmail.com>
References: <84B163F2-5B97-438F-ACA4-CF84E6FB554C@amazon.com>
 <CA+3eh11y5pEXV4dC-=u4P-JMpd8QT5qr+wY0aqdpRMkiVM3kug@mail.gmail.com>
Message-ID: <314036DA-D654-4C54-94D7-51436F86DF9A@oracle.com>

> On 11 Mar 2020, at 13:06, Volker Simonis <volker.simonis at gmail.com> wrote:
> 
> Liu, Xin <xxinliu at amazon.com <mailto:xxinliu at amazon.com>> schrieb am Mi., 11. M?rz 2020, 09:50:
> 
>> Hi,
>> 
>> I just filed a JBS issue JDK-8240834 about hotspot codestyle(
>> https://wiki.openjdk.java.net/display/HotSpot/StyleGuide). I found many
>> inconsistent code in hotspot so I propose to use clang-format[1] to enforce
>> it.
>> 
>> I know you?re all very experienced developers, but sometimes wrong
>> indentations or tailing whitespaces creep in.  it's hard to catch them in
>> webrev.  Even though we can, new contributors still need to contact their
>> sponsors for formatting changes.   I think it's a good idea to have lint
>> thing done at first place.
>> 
>> Clang is powerful enough to parse modern C/C++ code and it's very handy to
>> format code. clang-format should be available on all supporting platforms.
>> It's still okay even if it is absent on some platforms because nightly
>> build and submit repo will cover unsupported platforms.
>> 
>> I plan to have  a .clang-format and correcting styles in the 1st patch. We
>> can bring auto clang-format check to hotspot's makefile in the second
>> patch.  Does it sound like a plan?
>> 
> 
> Have you already checked how big this first change will be? I'm afraid it
> will be huge because we have a lot of manually formatted code which doesn't
> confirm to any specific style (e.g. macros). We also have consistent but
> different code styles in various parts of HS. So changing all the sources
> to conform to a single style will necessarily be quite invasive and I'm not
> sure if we want that?
> 

In theory I support using a formatting tool to maintain a single code style throughout the hotspot code, but in practice this is extremely difficult. Volker brings up one point here, the situation today is bad - really bad - from a formatter's perspective. The size of the initial change will be huge. To review that requires people from all affected areas to look in detail on their code to make sure nothing is broken.  Andrew brings up a different point in his mail; parts of the code is manually formatted for a reason. No automatic formatter will be able to make sense of all code if it applies the same rules all over.

I have looked into this problem for quite some time and I believe we have to take a very slow approach to this. Not only to be able to actually review the changes, but also to keep our developers around. First, any changes we make to clean up formatting must be automatically verified from that point forward to stop people from pushing new code that violates the rules we have managed to agree on. This requires a tool that can enable individual checks one by one. Secondly, the tool can't go in and modify people's code, it can only verify that it is correct and give a warning if there is a violation. This means that what we are looking for is a validator, not a formatter. There must also be the option to disable validation for parts of the code.

I have not found any such tools out there, one that can accept the hotspot source code as is without any remarks if all checks are disabled, and then let you enable one check a time as we gradually clean up the code. The best I could find was uncrustify, but it still is a formatter, not a validator, and there are a few style rules that can't be disabled. I asked the developers of uncrustify if there was any chance of having the option for it to be a validator instead but that didn't seem to be in their interest.

As I didn't find one I started to implement my own validator a few years ago. I do have a tool that swallows most of the hotspot code as is, but there are a few issues with macro usages. You see, that is another issue we have to deal with. It's not enough to parse C++. If we want to read and format the code we need to treat our macros as part of the language as well. Hotspot has a few macros that make for real challenges for a parser :-)  I haven't yet included the option to disable verification for a block of code, coming to think of it, that could actually solve the most weird macro usages we have out there. I'll take another look at that at some point.

Until we have a tool of this kind I'm afraid it will be extremely difficult to enforce any style rules in practice. But I do agree that this is a goal worth pursuing.

Cheers,
/Jesper


> 
>> Reference:
>> [1] https://clang.llvm.org/docs/ClangFormat.html <https://clang.llvm.org/docs/ClangFormat.html>
>> Thanks,
>> --lx


From vladimir.kozlov at oracle.com  Wed Mar 11 18:37:02 2020
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Wed, 11 Mar 2020 11:37:02 -0700
Subject: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
 inconsistent
In-Reply-To: <AM0PR0202MB32970A25D2592BB5E70358F89AFC0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
References: <c3bb5a7b-4aa0-2341-37a5-172f85977328@redhat.com>
 <23f50345-3f31-73cc-c62d-40ed0fd53c10@oracle.com>
 <e96653f2-3f5d-42cc-6d5e-8199a3caf8e8@oracle.com>
 <978bd34e-4c5a-b656-b885-db134c6c8bcb@redhat.com>
 <094fd52f-a04b-c087-dfee-157705e323f4@redhat.com>
 <F9E331DA-B9FC-40E0-81FD-85CC25F45FD0@oracle.com>
 <a88c261a-799e-ba93-fbb8-79e488ee2a3b@redhat.com>
 <f500862b-80d9-a5eb-dd80-3cbbd7980e1a@oracle.com>
 <e0792f13-25b6-8d57-50f6-50239ef2150d@redhat.com>
 <AM0PR0202MB3297ED1D3059110035E679DF9AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
 <AM0PR0202MB3297D74513C3E2C7F4C8B4459AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
 <1c40a7d1-fa36-b5d2-6072-eaa243cc9397@redhat.com>
 <AM0PR0202MB3297869C05C5A950564C3C8C9AFE0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
 <2343a628-d5d0-5042-649f-150ab649d1e3@redhat.com>
 <AM0PR0202MB32970A25D2592BB5E70358F89AFC0@AM0PR0202MB3297.eurprd02.prod.outlook.com>
Message-ID: <6603cedc-fe50-7d9a-aba4-8bce876937cb@oracle.com>

+1

But please test it.

Thanks,
Vladimir

On 3/11/20 9:27 AM, Doerr, Martin wrote:
> Hi Andrew,
> 
> I think it's worth fixing as you suggest.
> Current usages don't have a problem, but who knows about future usages?
> 
> Thanks and best regards,
> Martin
> 
> 
>> -----Original Message-----
>> From: Andrew Haley <aph at redhat.com>
>> Sent: Mittwoch, 11. M?rz 2020 16:47
>> To: Doerr, Martin <martin.doerr at sap.com>; Stefan Karlsson
>> <stefan.karlsson at oracle.com>; John Rose <john.r.rose at oracle.com>
>> Cc: hotspot-dev at openjdk.java.net; hotspot-runtime-dev at openjdk.java.net
>> Subject: Re: RFR: 8240615: is_power_of_2() has Undefined Behaviour and is
>> inconsistent
>>
>> On 3/9/20 3:56 PM, Doerr, Martin wrote:
>>> it doesn't run into the UB case, but log2_jint(x) returns 63 for negative x.
>>> I'd treat this as bug. A jint has 32 bit and log2 should never return 63.
>>
>> I guess this bug dates from the days when uintptr_t was 32 bits.
>>
>> I think you're right, but this has never been a problem because all uses
>> are either guarded by is_power_of_2() or they occur in a context (such as,
>> say, os::active_processor_count()) where > 2**31 is extremely unlikely.
>>
>> I'm happy to fix this, though, if anyone thinks I should. The fix is:
>>
>> diff -r f9893c227e12 src/hotspot/share/utilities/globalDefinitions.hpp
>> --- a/src/hotspot/share/utilities/globalDefinitions.hpp	Wed Mar 11
>> 15:02:09 2020 +0000
>> +++ b/src/hotspot/share/utilities/globalDefinitions.hpp	Wed Mar 11
>> 15:45:26 2020 +0000
>> @@ -1022,12 +1022,12 @@
>>
>>   inline int log2_int(int x) {
>>     STATIC_ASSERT(sizeof(int) <= sizeof(uintptr_t));
>> -  return log2_intptr((uintptr_t)x);
>> +  return log2_intptr((uintptr_t)(unsigned int)x);
>>   }
>>
>>   inline int log2_jint(jint x) {
>>     STATIC_ASSERT(sizeof(jint) <= sizeof(uintptr_t));
>> -  return log2_intptr((uintptr_t)x);
>> +  return log2_intptr((uintptr_t)(juint)x);
>>   }
>>
>>   inline int log2_uint(uint x) {
>>
>> --
>> Andrew Haley  (he/him)
>> Java Platform Lead Engineer
>> Red Hat UK Ltd. <https://www.redhat.com>
>> https://keybase.io/andrewhaley
>> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671
> 

From volker.simonis at gmail.com  Wed Mar 11 23:19:59 2020
From: volker.simonis at gmail.com (Volker Simonis)
Date: Thu, 12 Mar 2020 00:19:59 +0100
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <314036DA-D654-4C54-94D7-51436F86DF9A@oracle.com>
References: <84B163F2-5B97-438F-ACA4-CF84E6FB554C@amazon.com>
 <CA+3eh11y5pEXV4dC-=u4P-JMpd8QT5qr+wY0aqdpRMkiVM3kug@mail.gmail.com>
 <314036DA-D654-4C54-94D7-51436F86DF9A@oracle.com>
Message-ID: <CA+3eh11or=pxZ7D752GJbNm7a7=OofxX74Gmza2HJSmgD_9ZNA@mail.gmail.com>

Jesper Wilhelmsson <jesper.wilhelmsson at oracle.com> schrieb am Mi., 11. M?rz
2020, 19:13:

> On 11 Mar 2020, at 13:06, Volker Simonis <volker.simonis at gmail.com> wrote:
>
> Liu, Xin <xxinliu at amazon.com> schrieb am Mi., 11. M?rz 2020, 09:50:
>
> Hi,
>
> I just filed a JBS issue JDK-8240834 about hotspot codestyle(
> https://wiki.openjdk.java.net/display/HotSpot/StyleGuide). I found many
> inconsistent code in hotspot so I propose to use clang-format[1] to enforce
> it.
>
> I know you?re all very experienced developers, but sometimes wrong
> indentations or tailing whitespaces creep in.  it's hard to catch them in
> webrev.  Even though we can, new contributors still need to contact their
> sponsors for formatting changes.   I think it's a good idea to have lint
> thing done at first place.
>
> Clang is powerful enough to parse modern C/C++ code and it's very handy to
> format code. clang-format should be available on all supporting platforms.
> It's still okay even if it is absent on some platforms because nightly
> build and submit repo will cover unsupported platforms.
>
> I plan to have  a .clang-format and correcting styles in the 1st patch. We
> can bring auto clang-format check to hotspot's makefile in the second
> patch.  Does it sound like a plan?
>
>
> Have you already checked how big this first change will be? I'm afraid it
> will be huge because we have a lot of manually formatted code which doesn't
> confirm to any specific style (e.g. macros). We also have consistent but
> different code styles in various parts of HS. So changing all the sources
> to conform to a single style will necessarily be quite invasive and I'm not
> sure if we want that?
>
>
> In theory I support using a formatting tool to maintain a single code
> style throughout the hotspot code, but in practice this is extremely
> difficult. Volker brings up one point here, the situation today is bad -
> really bad - from a formatter's perspective. The size of the initial change
> will be huge. To review that requires people from all affected areas to
> look in detail on their code to make sure nothing is broken.  Andrew brings
> up a different point in his mail; parts of the code is manually formatted
> for a reason. No automatic formatter will be able to make sense of all code
> if it applies the same rules all over.
>
> I have looked into this problem for quite some time and I believe we have
> to take a very slow approach to this. Not only to be able to actually
> review the changes, but also to keep our developers around. First, any
> changes we make to clean up formatting must be automatically verified from
> that point forward to stop people from pushing new code that violates the
> rules we have managed to agree on. This requires a tool that can enable
> individual checks one by one. Secondly, the tool can't go in and modify
> people's code, it can only verify that it is correct and give a warning if
> there is a violation. This means that what we are looking for is a
> validator, not a formatter. There must also be the option to disable
> validation for parts of the code.
>
> I have not found any such tools out there, one that can accept the hotspot
> source code as is without any remarks if all checks are disabled, and then
> let you enable one check a time as we gradually clean up the code. The best
> I could find was uncrustify, but it still is a formatter, not a validator,
> and there are a few style rules that can't be disabled. I asked the
> developers of uncrustify if there was any chance of having the option for
> it to be a validator instead but that didn't seem to be in their interest.
>
> As I didn't find one I started to implement my own validator a few years
> ago. I do have a tool that swallows most of the hotspot code as is, but
> there are a few issues with macro usages. You see, that is another issue we
> have to deal with. It's not enough to parse C++. If we want to read and
> format the code we need to treat our macros as part of the language as
> well. Hotspot has a few macros that make for real challenges for a parser
> :-)
>

The default Emacs on Ubuntu 18.04 will just hang forever consuming 100% cpu
when opening globals.hpp with syntax highlighting enabled. :)

 I haven't yet included the option to disable verification for a block of
> code, coming to think of it, that could actually solve the most weird macro
> usages we have out there. I'll take another look at that at some point.
>
> Until we have a tool of this kind I'm afraid it will be extremely
> difficult to enforce any style rules in practice. But I do agree that this
> is a goal worth pursuing.
>
> Cheers,
> /Jesper
>
>
>
> Reference:
> [1] https://clang.llvm.org/docs/ClangFormat.html
> Thanks,
> --lx
>
>
>

From ioi.lam at oracle.com  Thu Mar 12 05:17:55 2020
From: ioi.lam at oracle.com (Ioi Lam)
Date: Wed, 11 Mar 2020 22:17:55 -0700
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <CA+3eh11or=pxZ7D752GJbNm7a7=OofxX74Gmza2HJSmgD_9ZNA@mail.gmail.com>
References: <84B163F2-5B97-438F-ACA4-CF84E6FB554C@amazon.com>
 <CA+3eh11y5pEXV4dC-=u4P-JMpd8QT5qr+wY0aqdpRMkiVM3kug@mail.gmail.com>
 <314036DA-D654-4C54-94D7-51436F86DF9A@oracle.com>
 <CA+3eh11or=pxZ7D752GJbNm7a7=OofxX74Gmza2HJSmgD_9ZNA@mail.gmail.com>
Message-ID: <4a83db03-1295-52d5-7624-120396faa4ec@oracle.com>

On 3/11/20 4:19 PM, Volker Simonis wrote:
> Jesper Wilhelmsson <jesper.wilhelmsson at oracle.com> schrieb am Mi., 11. M?rz
> 2020, 19:13:
>
>> As I didn't find one I started to implement my own validator a few years
>> ago. I do have a tool that swallows most of the hotspot code as is, but
>> there are a few issues with macro usages. You see, that is another issue we
>> have to deal with. It's not enough to parse C++. If we want to read and
>> format the code we need to treat our macros as part of the language as
>> well. Hotspot has a few macros that make for real challenges for a parser
>> :-)
>>
> The default Emacs on Ubuntu 18.04 will just hang forever consuming 100% cpu
> when opening globals.hpp with syntax highlighting enabled. :)
>

Really off-topic, but I actually have a patch that overhauls the VM 
flags to get rid of this ridiculous macro. See

http://cr.openjdk.java.net/~iklam/jdk15/vm_flags_overhaul.006/

(Look at globals.hpp, globals.cpp and globals.flags.hpp).

I'll try to post an RFR for this soon ....

Thanks
- Ioi

From matthias.baesken at sap.com  Thu Mar 12 07:05:16 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Thu, 12 Mar 2020 07:05:16 +0000
Subject: PING? RFR(S): 8239785: Cgroups: Incorrect detection logic on old
 systems in hotspot
In-Reply-To: <07038b5c3cce3dab81aa812e985aad348e508c9b.camel@redhat.com>
References: <362678e8bc8231c205c6b3327d08dd5179154e23.camel@redhat.com>
 <2A934185-9B81-4371-B985-F0C3C39B3FD3@oracle.com>
 <215e52601b2da8600cbfd47ef73634720a7736d2.camel@redhat.com>
 <EF45A2BB-360E-4767-B542-B420BF76FD2E@oracle.com>
 <d1683592b67f99e7625713d2db2c71641d50a738.camel@redhat.com>
 <1B61ACE7-AFB9-458C-B6AB-465259BD31C7@oracle.com>
 <264b4edf94d60a3256a9ff8d5de6a078c3a0819b.camel@redhat.com>
 <A6EE0016-E33B-451D-BF0F-5F9C13A4DB32@oracle.com>
 <94c06270838cb3c090a227f2688a5c3f0aade64c.camel@redhat.com>
 <C81DB191-91BF-4ED6-81E1-51605FAE6693@oracle.com>
 <07038b5c3cce3dab81aa812e985aad348e508c9b.camel@redhat.com>
Message-ID: <AM0PR02MB4227BB0A5EF554C645FFADC993FD0@AM0PR02MB4227.eurprd02.prod.outlook.com>

Hello Severin, looks okay to me .
Some very minor issues (do not need  to see another webrev)  :

http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/src/hotspot/share/prims/whitebox.cpp.frames.html

CHECK_JNI_EXCEPTION_
might return early without calling  env->ReleaseStringUTFChars (these are in the normal cases called at the end of the method)


http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/src/hotspot/share/prims/whitebox.hpp.frames.html
please adjust Copyright year


http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/src/hotspot/os/linux/cgroupSubsystem_linux.hpp.frames.html

 260 class CgroupInfo : public StackObj {
  ....
 274     CgroupInfo() { // constructor    <---- remove comment



Best regards, Matthias



> 
> I would still need a *R*eviewer for this one. Any takers? Bob has
> already reviewed it.
> 
> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/
> 
> Thanks,
> Severin
> 


From sgehwolf at redhat.com  Thu Mar 12 08:20:58 2020
From: sgehwolf at redhat.com (Severin Gehwolf)
Date: Thu, 12 Mar 2020 09:20:58 +0100
Subject: PING? RFR(S): 8239785: Cgroups: Incorrect detection logic on
 old systems in hotspot
In-Reply-To: <AM0PR02MB4227BB0A5EF554C645FFADC993FD0@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <362678e8bc8231c205c6b3327d08dd5179154e23.camel@redhat.com>
 <2A934185-9B81-4371-B985-F0C3C39B3FD3@oracle.com>
 <215e52601b2da8600cbfd47ef73634720a7736d2.camel@redhat.com>
 <EF45A2BB-360E-4767-B542-B420BF76FD2E@oracle.com>
 <d1683592b67f99e7625713d2db2c71641d50a738.camel@redhat.com>
 <1B61ACE7-AFB9-458C-B6AB-465259BD31C7@oracle.com>
 <264b4edf94d60a3256a9ff8d5de6a078c3a0819b.camel@redhat.com>
 <A6EE0016-E33B-451D-BF0F-5F9C13A4DB32@oracle.com>
 <94c06270838cb3c090a227f2688a5c3f0aade64c.camel@redhat.com>
 <C81DB191-91BF-4ED6-81E1-51605FAE6693@oracle.com>
 <07038b5c3cce3dab81aa812e985aad348e508c9b.camel@redhat.com>
 <AM0PR02MB4227BB0A5EF554C645FFADC993FD0@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <e629ff3a3380f54759f9f6fd1d46615e65c75200.camel@redhat.com>

Hi Matthias,

On Thu, 2020-03-12 at 07:05 +0000, Baesken, Matthias wrote:
> Hello Severin, looks okay to me .

Thanks very much for the review!

> Some very minor issues (do not need  to see another webrev)  :
> 
> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/src/hotspot/share/prims/whitebox.cpp.frames.html
> 
> CHECK_JNI_EXCEPTION_
> might return early without calling  env->ReleaseStringUTFChars (these are in the normal cases called at the end of the method)
> 
> 
> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/src/hotspot/share/prims/whitebox.hpp.frames.html
> please adjust Copyright year
> 
> 
> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/src/hotspot/os/linux/cgroupSubsystem_linux.hpp.frames.html
> 
>  260 class CgroupInfo : public StackObj {
>   ....
>  274     CgroupInfo() { // constructor    <---- remove comment

OK. I'll make the suggested changes, retest and then push.

Thanks,
Severin

> 
> Best regards, Matthias
> 
> 
> 
> > I would still need a *R*eviewer for this one. Any takers? Bob has
> > already reviewed it.
> > 
> > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8239785/05/webrev/
> > 
> > Thanks,
> > Severin
> > 


From xxinliu at amazon.com  Thu Mar 12 09:30:08 2020
From: xxinliu at amazon.com (Liu, Xin)
Date: Thu, 12 Mar 2020 09:30:08 +0000
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <314036DA-D654-4C54-94D7-51436F86DF9A@oracle.com>
References: <84B163F2-5B97-438F-ACA4-CF84E6FB554C@amazon.com>
 <CA+3eh11y5pEXV4dC-=u4P-JMpd8QT5qr+wY0aqdpRMkiVM3kug@mail.gmail.com>
 <314036DA-D654-4C54-94D7-51436F86DF9A@oracle.com>
Message-ID: <F8403A86-3D2A-4400-83D7-36B0677AF13D@amazon.com>

Thanks for the feedbacks.

For Andrew?s concerns, I think I got answers.
Sometimes, hand-craft code is better indeed.  Clang-format provides a directive which can temporarily turn it off.
https://releases.llvm.org/6.0.1/tools/clang/docs/ClangFormatStyleOptions.html#disabling-formatting-on-a-piece-of-code
Developers can also define customized formats. clang-format takes precedence the local configure file over parent directory?s.  Assume that GC  guys would like to use an exotic code style, they can define it and place it in their own directory.

I expect to have a huge change, but it?s a one-time thing.  I don?t understand why we have to review it completely.  A formatter should never alter semantics of user?s code.  If hotspot can?t be built or spot any regression, there must be something wrong. We should be glad some bugs are caught by an automatic tool.

Yes, clang-format is a formatter,  but we can turn it into a validator very cheap by git/hg. If we run clang-format and generate any diff, we know the current code violates hotspot?s codestyle. In my proposal, I plan to do that in phase-2, eg. ?make check-style?.  Of course, it only works in a clean repo.

How about we just forget about validator. I think we need a .clang-format first. It helps developers to format code automatically. To proof my idea, I made a draft .clang-format and run it against compiler/*.cpp.  I think most of those changes make hotspot more readable.
https://cr.openjdk.java.net/~xliu/8240834/webrev/

I don?t realize that hotspot macros are special. It seems that clang-format has many macro-related styles too.  Let me try them on globals.hpp.

Thanks,
--lx


From: Jesper Wilhelmsson <jesper.wilhelmsson at oracle.com>
Date: Wednesday, March 11, 2020 at 11:18 AM
To: "Liu, Xin" <xxinliu at amazon.com>
Cc: Volker Simonis <volker.simonis at gmail.com>, HotSpot Open Source Developers <hotspot-dev at openjdk.java.net>
Subject: RE: [EXTERNAL]Propose to use clang-format to enforce hotspot codestyle

On 11 Mar 2020, at 13:06, Volker Simonis <volker.simonis at gmail.com<mailto:volker.simonis at gmail.com>> wrote:

Liu, Xin <xxinliu at amazon.com<mailto:xxinliu at amazon.com>> schrieb am Mi., 11. M?rz 2020, 09:50:


Hi,

I just filed a JBS issue JDK-8240834 about hotspot codestyle(
https://wiki.openjdk.java.net/display/HotSpot/StyleGuide). I found many
inconsistent code in hotspot so I propose to use clang-format[1] to enforce
it.

I know you?re all very experienced developers, but sometimes wrong
indentations or tailing whitespaces creep in.  it's hard to catch them in
webrev.  Even though we can, new contributors still need to contact their
sponsors for formatting changes.   I think it's a good idea to have lint
thing done at first place.

Clang is powerful enough to parse modern C/C++ code and it's very handy to
format code. clang-format should be available on all supporting platforms.
It's still okay even if it is absent on some platforms because nightly
build and submit repo will cover unsupported platforms.

I plan to have  a .clang-format and correcting styles in the 1st patch. We
can bring auto clang-format check to hotspot's makefile in the second
patch.  Does it sound like a plan?

Have you already checked how big this first change will be? I'm afraid it
will be huge because we have a lot of manually formatted code which doesn't
confirm to any specific style (e.g. macros). We also have consistent but
different code styles in various parts of HS. So changing all the sources
to conform to a single style will necessarily be quite invasive and I'm not
sure if we want that?


In theory I support using a formatting tool to maintain a single code style throughout the hotspot code, but in practice this is extremely difficult. Volker brings up one point here, the situation today is bad - really bad - from a formatter's perspective. The size of the initial change will be huge. To review that requires people from all affected areas to look in detail on their code to make sure nothing is broken.  Andrew brings up a different point in his mail; parts of the code is manually formatted for a reason. No automatic formatter will be able to make sense of all code if it applies the same rules all over.

I have looked into this problem for quite some time and I believe we have to take a very slow approach to this. Not only to be able to actually review the changes, but also to keep our developers around. First, any changes we make to clean up formatting must be automatically verified from that point forward to stop people from pushing new code that violates the rules we have managed to agree on. This requires a tool that can enable individual checks one by one. Secondly, the tool can't go in and modify people's code, it can only verify that it is correct and give a warning if there is a violation. This means that what we are looking for is a validator, not a formatter. There must also be the option to disable validation for parts of the code.

I have not found any such tools out there, one that can accept the hotspot source code as is without any remarks if all checks are disabled, and then let you enable one check a time as we gradually clean up the code. The best I could find was uncrustify, but it still is a formatter, not a validator, and there are a few style rules that can't be disabled. I asked the developers of uncrustify if there was any chance of having the option for it to be a validator instead but that didn't seem to be in their interest.

As I didn't find one I started to implement my own validator a few years ago. I do have a tool that swallows most of the hotspot code as is, but there are a few issues with macro usages. You see, that is another issue we have to deal with. It's not enough to parse C++. If we want to read and format the code we need to treat our macros as part of the language as well. Hotspot has a few macros that make for real challenges for a parser :-)  I haven't yet included the option to disable verification for a block of code, coming to think of it, that could actually solve the most weird macro usages we have out there. I'll take another look at that at some point.

Until we have a tool of this kind I'm afraid it will be extremely difficult to enforce any style rules in practice. But I do agree that this is a goal worth pursuing.

Cheers,
/Jesper





Reference:
[1] https://clang.llvm.org/docs/ClangFormat.html
Thanks,
--lx



From aph at redhat.com  Thu Mar 12 09:36:44 2020
From: aph at redhat.com (Andrew Haley)
Date: Thu, 12 Mar 2020 09:36:44 +0000
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <F8403A86-3D2A-4400-83D7-36B0677AF13D@amazon.com>
References: <84B163F2-5B97-438F-ACA4-CF84E6FB554C@amazon.com>
 <CA+3eh11y5pEXV4dC-=u4P-JMpd8QT5qr+wY0aqdpRMkiVM3kug@mail.gmail.com>
 <314036DA-D654-4C54-94D7-51436F86DF9A@oracle.com>
 <F8403A86-3D2A-4400-83D7-36B0677AF13D@amazon.com>
Message-ID: <c0da876e-117b-ce1e-88df-cfe3682cef92@redhat.com>

On 3/12/20 9:30 AM, Liu, Xin wrote:
> I expect to have a huge change, but it?s a one-time thing.

No, please do not do this.

It well mess up the diffs, and it will make auto-merging fail. It will
add considerable costs to the updates project for every backport. It
is in every way a bad idea.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From aph at redhat.com  Thu Mar 12 09:55:39 2020
From: aph at redhat.com (Andrew Haley)
Date: Thu, 12 Mar 2020 09:55:39 +0000
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <F8403A86-3D2A-4400-83D7-36B0677AF13D@amazon.com>
References: <84B163F2-5B97-438F-ACA4-CF84E6FB554C@amazon.com>
 <CA+3eh11y5pEXV4dC-=u4P-JMpd8QT5qr+wY0aqdpRMkiVM3kug@mail.gmail.com>
 <314036DA-D654-4C54-94D7-51436F86DF9A@oracle.com>
 <F8403A86-3D2A-4400-83D7-36B0677AF13D@amazon.com>
Message-ID: <ce34b3df-2d93-856d-3dfb-ba74d6e69d04@redhat.com>

On 3/12/20 9:30 AM, Liu, Xin wrote:
> For Andrew?s concerns, I think I got answers.

> Sometimes, hand-craft code is better indeed.  Clang-format provides
> a directive which can temporarily turn it off.

That's not really adequate. The programmer has a choice to format
their code in an informative of slightly nonstandard way, but only if
they clutter up the source with /* clang-format off */ markers. So
most of the time they won't do it, and something will have been lost.

Code layout adds information for the reader. That's why today we
mostly use free-format languages. Back in ancient times ( :-) it
wasn't like that, but people learned that a program is a text, to be
read by people not just computers, and layout can help with that.

Of course HotSpot's guidelines are important to ensure a consistent
style across many patches written by many different authors. I'm not
saying otherwise: without consistent layout it'd be harder for people
across many organizations to work together. But applying a fixed set
of rules regardless of context is too naive an approach.

Don't misunderstand me, I like auto-formatters. I use one all the
time, and my patches follow the guidelines. But occasionally any
auto-formatter gets it wrong and then I have to fix things up by hand.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From coleen.phillimore at oracle.com  Thu Mar 12 11:33:33 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Thu, 12 Mar 2020 04:33:33 -0700 (PDT)
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <c0da876e-117b-ce1e-88df-cfe3682cef92@redhat.com>
References: <84B163F2-5B97-438F-ACA4-CF84E6FB554C@amazon.com>
 <CA+3eh11y5pEXV4dC-=u4P-JMpd8QT5qr+wY0aqdpRMkiVM3kug@mail.gmail.com>
 <314036DA-D654-4C54-94D7-51436F86DF9A@oracle.com>
 <F8403A86-3D2A-4400-83D7-36B0677AF13D@amazon.com>
 <c0da876e-117b-ce1e-88df-cfe3682cef92@redhat.com>
Message-ID: <bf89d862-e8a7-4cd7-d60f-f8128d50cf3a@oracle.com>



On 3/12/20 5:36 AM, Andrew Haley wrote:
> On 3/12/20 9:30 AM, Liu, Xin wrote:
>> I expect to have a huge change, but it?s a one-time thing.
> No, please do not do this.
>
> It well mess up the diffs, and it will make auto-merging fail. It will
> add considerable costs to the updates project for every backport. It
> is in every way a bad idea.
>
Thank you.? Do not pretty print the source code.
Coleen

From thomas.stuefe at gmail.com  Thu Mar 12 13:33:22 2020
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Thu, 12 Mar 2020 14:33:22 +0100
Subject: RFR: 8240824: enhance print_full_memory_info on Linux by THP
 related information
In-Reply-To: <AM0PR02MB42272402734B0AD4B4E304CC93FF0@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB42272402734B0AD4B4E304CC93FF0@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <CAA-vtUzKF7qHzEiNjn8E_Gg6a5pGdheB0RA0tfzT86ywgJL3NQ@mail.gmail.com>

Hi Matthias,

too verbose for my taste:

+  st->print_cr("/sys/kernel/mm/transparent_hugepage/enabled
(transparent huge pages mode):");


I would cut that down to either
"/sys/kernel/mm/transparent_hugepage/enabled" or to "transparent huge pages
mode". Probably the former.

Otherwise looks good. I do not need a new webrev.

Cheers, Thomas



On Tue, Mar 10, 2020 at 4:09 PM Baesken, Matthias <matthias.baesken at sap.com>
wrote:

> Hello, please review this small enhancement .
>
> On Linux, the change adds output of some files related to THP (transparent
> huge pages)   configuration to the hs_err file; the settings in those files
> configure how transparent huge pages are used on the system.
>
> Example output in the hs_err file from a test machine :
>
>
> /sys/kernel/mm/transparent_hugepage/enabled (transparent huge pages mode):
> [always] madvise never
>
> /sys/kernel/mm/transparent_hugepage/defrag (defrag/compaction efforts
> parameter):
> [always] madvise never
>
>
>
>
> Bug/webrev :
>
> https://bugs.openjdk.java.net/browse/JDK-8240824
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8240824.0/
>
>
> Thanks, Matthias
>
>

From jesper.wilhelmsson at oracle.com  Thu Mar 12 14:26:58 2020
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Thu, 12 Mar 2020 15:26:58 +0100
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <F8403A86-3D2A-4400-83D7-36B0677AF13D@amazon.com>
References: <84B163F2-5B97-438F-ACA4-CF84E6FB554C@amazon.com>
 <CA+3eh11y5pEXV4dC-=u4P-JMpd8QT5qr+wY0aqdpRMkiVM3kug@mail.gmail.com>
 <314036DA-D654-4C54-94D7-51436F86DF9A@oracle.com>
 <F8403A86-3D2A-4400-83D7-36B0677AF13D@amazon.com>
Message-ID: <1A146F60-3FA6-4AAE-92E2-DDD446F64A4C@oracle.com>

> On 12 Mar 2020, at 10:30, Liu, Xin <xxinliu at amazon.com> wrote:
> 
> Thanks for the feedbacks.
> 
> For Andrew?s concerns, I think I got answers.
> Sometimes, hand-craft code is better indeed.  Clang-format provides a directive which can temporarily turn it off.
> https://releases.llvm.org/6.0.1/tools/clang/docs/ClangFormatStyleOptions.html#disabling-formatting-on-a-piece-of-code <https://urldefense.com/v3/__https://releases.llvm.org/6.0.1/tools/clang/docs/ClangFormatStyleOptions.html*disabling-formatting-on-a-piece-of-code__;Iw!!GqivPVa7Brio!OOfqbHl7BR-FAqgxbdaX1tTUPQasjG5INyltu654bQzTLbZuu6vEMxNnTJl0tGTIubLcgZ0$>
> Developers can also define customized formats. clang-format takes precedence the local configure file over parent directory?s.  Assume that GC  guys would like to use an exotic code style, they can define it and place it in their own directory.
> 
> I expect to have a huge change, but it?s a one-time thing.  I don?t understand why we have to review it completely.  A formatter should never alter semantics of user?s code.  If hotspot can?t be built or spot any regression, there must be something wrong. We should be glad some bugs are caught by an automatic tool.

If you get a huge patch committed into the JDK without proper review there is something wrong in the review process. There are so many aspects to this and I'm actually surprised that you would expect the community to accept a huge change like this. Even if you disregard all the practical problems with conflicts with every single project repo and clone out there and all the issues that a change like that will cause for backports for years to come, even if we only look at mainline and think forward, there are huge issues with a huge patch.

First of all, if we assume that the change would contain only non-semantic cleanup of indentation, spaces, and line breaks, the developers who work on this code on a daily basis would not accept the change without them looking through their areas first. We have already concluded that there is a need to opt out from formatting in parts of the code. These parts would have to be found and cleaned up in your patch before you push it. So already here we have a requirement from basically all teams out there working on the JVM to review the patch.

The patch is quite likely to contain other changes as well though. For instance it will likely add { } for all if/for/while etc that are missing it. This in it self is a good change, but without a proper review this automatic cleanup risk hiding bugs rather than finding them. Suppose you have code like this:

if (expression)
  foo();
  bar();
other_code();

It's pretty obvious that something is fishy here, but is it the indentation that is wrong, or are we missing {  }?  The formatter can't know but is likely to clean up like this:

if (expression) {
  foo();
}
bar();
other_code();

Now it's not at all obvious that something is fishy anymore. If the indentation was correct (we were missing { }) and this manifests as an intermittent bug we just made it a lot harder to find it. Actually this example shows the same point even without adding the { }. "Fixing" the indentation would also hide the bug.  There are many more examples of similar issues with common cleanups made by formatters.


Then there is the whole security aspect. Why should the community trust a huge patch where the author waves his hand and says "Nah, you don't have to look at this"? From a security perspective that would be insane. Anything could be hiding in there.

Just for the record since you brought it up, I'm not worried about the tool finding bugs in the hotspot code - that would be a good thing. I'm more worried about the hotspot code finding bugs in the tool.

> Yes, clang-format is a formatter,  but we can turn it into a validator very cheap by git/hg. If we run clang-format and generate any diff, we know the current code violates hotspot?s codestyle. In my proposal, I plan to do that in phase-2, eg. ?make check-style?.  Of course, it only works in a clean repo.

Yes, running the formatter on a copy of your modified file and diffing would tell if you have discrepancies, but that involves saving temporary files on disk, and it requires some additional formatting of the diff output to make it understandable what is wrong with your modified code. Doable, sure, but I'd prefer a tool that gives you clean warnings up front without an extra layer that must be kept up to date and free from bugs.

> How about we just forget about validator. I think we need a .clang-format first. It helps developers to format code automatically. To proof my idea, I made a draft .clang-format and run it against compiler/*.cpp.  I think most of those changes make hotspot more readable.
> https://cr.openjdk.java.net/~xliu/8240834/webrev/ <https://cr.openjdk.java.net/~xliu/8240834/webrev/>

Looking at your webrev I see that it hides a lot of changes since it doesn't show all whitespace-only changes as diffs. So already here there is a major flaw that requires an actual review to tell that your patch is even larger than what your webrev says.

This patch removes all aligned '=' in declarations. I assume this is configurable(?) It's not an improvement in my book.

default: ;    <---I think the removal of this space makes it harder to read.


How long lines does this formatter prefer? The change below creates a very long line that was broken into two for a reason.

-  int            filler_limit = align_instr() ? max_instr_size_in_bytes : ((instruction_size_in_bytes+abstract_instruction_bytes_per_block-1)/abstract_instruction_bytes_per_block)
-                                                                          *abstract_instruction_bytes_per_block;
+  int            filler_limit = align_instr() ? max_instr_size_in_bytes : ((instruction_size_in_bytes + abstract_instruction_bytes_per_block - 1) / abstract_instruction_bytes_per_block) * abstract_instruction_bytes_per_block;

I just looked at one file but already I can say that there is no way this can be pushed without the compiler team looking at it.


> I don?t realize that hotspot macros are special. It seems that clang-format has many macro-related styles too.  Let me try them on globals.hpp.

There are some really unfriendly macros out there...
/Jesper


> 
> Thanks,
> --lx
> 
> 
> From: Jesper Wilhelmsson <jesper.wilhelmsson at oracle.com <mailto:jesper.wilhelmsson at oracle.com>>
> Date: Wednesday, March 11, 2020 at 11:18 AM
> To: "Liu, Xin" <xxinliu at amazon.com <mailto:xxinliu at amazon.com>>
> Cc: Volker Simonis <volker.simonis at gmail.com <mailto:volker.simonis at gmail.com>>, HotSpot Open Source Developers <hotspot-dev at openjdk.java.net <mailto:hotspot-dev at openjdk.java.net>>
> Subject: RE: [EXTERNAL]Propose to use clang-format to enforce hotspot codestyle
> 
> On 11 Mar 2020, at 13:06, Volker Simonis <volker.simonis at gmail.com <mailto:volker.simonis at gmail.com>> wrote:
> 
> Liu, Xin <xxinliu at amazon.com <mailto:xxinliu at amazon.com>> schrieb am Mi., 11. M?rz 2020, 09:50:
> 
> 
> Hi,
> 
> I just filed a JBS issue JDK-8240834 about hotspot codestyle(
> https://wiki.openjdk.java.net/display/HotSpot/StyleGuide <https://wiki.openjdk.java.net/display/HotSpot/StyleGuide>). I found many
> inconsistent code in hotspot so I propose to use clang-format[1] to enforce
> it.
> 
> I know you?re all very experienced developers, but sometimes wrong
> indentations or tailing whitespaces creep in.  it's hard to catch them in
> webrev.  Even though we can, new contributors still need to contact their
> sponsors for formatting changes.   I think it's a good idea to have lint
> thing done at first place.
> 
> Clang is powerful enough to parse modern C/C++ code and it's very handy to
> format code. clang-format should be available on all supporting platforms.
> It's still okay even if it is absent on some platforms because nightly
> build and submit repo will cover unsupported platforms.
> 
> I plan to have  a .clang-format and correcting styles in the 1st patch. We
> can bring auto clang-format check to hotspot's makefile in the second
> patch.  Does it sound like a plan?
> 
> 
> Have you already checked how big this first change will be? I'm afraid it
> will be huge because we have a lot of manually formatted code which doesn't
> confirm to any specific style (e.g. macros). We also have consistent but
> different code styles in various parts of HS. So changing all the sources
> to conform to a single style will necessarily be quite invasive and I'm not
> sure if we want that?
> 
> 
> In theory I support using a formatting tool to maintain a single code style throughout the hotspot code, but in practice this is extremely difficult. Volker brings up one point here, the situation today is bad - really bad - from a formatter's perspective. The size of the initial change will be huge. To review that requires people from all affected areas to look in detail on their code to make sure nothing is broken.  Andrew brings up a different point in his mail; parts of the code is manually formatted for a reason. No automatic formatter will be able to make sense of all code if it applies the same rules all over.
> 
> I have looked into this problem for quite some time and I believe we have to take a very slow approach to this. Not only to be able to actually review the changes, but also to keep our developers around. First, any changes we make to clean up formatting must be automatically verified from that point forward to stop people from pushing new code that violates the rules we have managed to agree on. This requires a tool that can enable individual checks one by one. Secondly, the tool can't go in and modify people's code, it can only verify that it is correct and give a warning if there is a violation. This means that what we are looking for is a validator, not a formatter. There must also be the option to disable validation for parts of the code.
> 
> I have not found any such tools out there, one that can accept the hotspot source code as is without any remarks if all checks are disabled, and then let you enable one check a time as we gradually clean up the code. The best I could find was uncrustify, but it still is a formatter, not a validator, and there are a few style rules that can't be disabled. I asked the developers of uncrustify if there was any chance of having the option for it to be a validator instead but that didn't seem to be in their interest.
> 
> As I didn't find one I started to implement my own validator a few years ago. I do have a tool that swallows most of the hotspot code as is, but there are a few issues with macro usages. You see, that is another issue we have to deal with. It's not enough to parse C++. If we want to read and format the code we need to treat our macros as part of the language as well. Hotspot has a few macros that make for real challenges for a parser :-)  I haven't yet included the option to disable verification for a block of code, coming to think of it, that could actually solve the most weird macro usages we have out there. I'll take another look at that at some point.
> 
> Until we have a tool of this kind I'm afraid it will be extremely difficult to enforce any style rules in practice. But I do agree that this is a goal worth pursuing.
> 
> Cheers,
> /Jesper
> 
> 
> 
> 
> 
> Reference:
> [1] https://clang.llvm.org/docs/ClangFormat.html <https://urldefense.com/v3/__https://clang.llvm.org/docs/ClangFormat.html__;!!GqivPVa7Brio!OOfqbHl7BR-FAqgxbdaX1tTUPQasjG5INyltu654bQzTLbZuu6vEMxNnTJl0tGTIfZEEix4$>
> Thanks,
> --lx


From ioi.lam at oracle.com  Thu Mar 12 17:50:07 2020
From: ioi.lam at oracle.com (Ioi Lam)
Date: Thu, 12 Mar 2020 10:50:07 -0700
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <c0da876e-117b-ce1e-88df-cfe3682cef92@redhat.com>
References: <84B163F2-5B97-438F-ACA4-CF84E6FB554C@amazon.com>
 <CA+3eh11y5pEXV4dC-=u4P-JMpd8QT5qr+wY0aqdpRMkiVM3kug@mail.gmail.com>
 <314036DA-D654-4C54-94D7-51436F86DF9A@oracle.com>
 <F8403A86-3D2A-4400-83D7-36B0677AF13D@amazon.com>
 <c0da876e-117b-ce1e-88df-cfe3682cef92@redhat.com>
Message-ID: <200e1808-8097-5672-4365-3a1cbf0bef78@oracle.com>



On 3/12/20 2:36 AM, Andrew Haley wrote:
> On 3/12/20 9:30 AM, Liu, Xin wrote:
>> I expect to have a huge change, but it?s a one-time thing.
> No, please do not do this.
>
> It well mess up the diffs, and it will make auto-merging fail. It will
> add considerable costs to the updates project for every backport. It
> is in every way a bad idea.
>
Such a full-scale formatting change will also affect "hg annotate" (and 
the future GIT equivalent). E.g.,

http://hg.openjdk.java.net/jdk/jdk/annotate/f2a8072492df/src/hotspot/share/memory/universe.cpp

That's very valuable in tracking down why each line of source is written 
the way it is. Without this, it makes bug analysis more difficult.

Thanks
- Ioi

From john.r.rose at oracle.com  Fri Mar 13 02:12:25 2020
From: john.r.rose at oracle.com (John Rose)
Date: Thu, 12 Mar 2020 19:12:25 -0700
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <CA+3eh11or=pxZ7D752GJbNm7a7=OofxX74Gmza2HJSmgD_9ZNA@mail.gmail.com>
References: <CA+3eh11or=pxZ7D752GJbNm7a7=OofxX74Gmza2HJSmgD_9ZNA@mail.gmail.com>
Message-ID: <5B92DB3F-A436-4358-A8B4-56977A6EDE6E@oracle.com>

FWIW my workaround for that is to put that particular
file name (and two others) on auto-mode-alist in my emacs
config file, turning off C mode altogether:

(push '("/vmSymbols\\.hpp\\'" . fundamental-mode) auto-mode-alist)
(push '("/globals\\.hpp\\'" . fundamental-mode) auto-mode-alist)
(push '("/vmStructs\\.cpp\\'" . fundamental-mode) auto-mode-alist)

There are surely less blunt ways to do it, but it?s better than
the frustrating lock-up you describe, which I am also familiar
with (Aquamacs, many recent versions).

? John

> On Mar 11, 2020, at 4:20 PM, Volker Simonis <volker.simonis at gmail.com> wrote:
> 
> The default Emacs on Ubuntu 18.04 will just hang forever consuming 100% cpu
> when opening globals.hpp with syntax highlighting enabled. :)

From john.r.rose at oracle.com  Fri Mar 13 03:27:58 2020
From: john.r.rose at oracle.com (John Rose)
Date: Thu, 12 Mar 2020 20:27:58 -0700
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <c0da876e-117b-ce1e-88df-cfe3682cef92@redhat.com>
References: <84B163F2-5B97-438F-ACA4-CF84E6FB554C@amazon.com>
 <CA+3eh11y5pEXV4dC-=u4P-JMpd8QT5qr+wY0aqdpRMkiVM3kug@mail.gmail.com>
 <314036DA-D654-4C54-94D7-51436F86DF9A@oracle.com>
 <F8403A86-3D2A-4400-83D7-36B0677AF13D@amazon.com>
 <c0da876e-117b-ce1e-88df-cfe3682cef92@redhat.com>
Message-ID: <5293DB1A-AF6C-4B35-A530-9B434D335909@oracle.com>

On Mar 12, 2020, at 2:36 AM, Andrew Haley <aph at redhat.com> wrote:
> 
> No, please do not do this.
> 
> It well mess up the diffs, and it will make auto-merging fail. It will
> add considerable costs to the updates project for every backport. It
> is in every way a bad idea.

+100

I think one influence in discussions like this one is that they
turn on the ?prescriptive vs. descriptive? distinction discovered
by linguists, but applied (maybe surprisingly) to software notations.
It may take a while to realize it, but this distinction is relevant,
hard to resolve, and subtle.

Here?s a primer from the linguistic world:

https://www.ling.upenn.edu/courses/ling001/prescription.html

Short version:  Languages (and code), as used by humans, vary
over time and place in every possible way.  Variation must be
controlled to preserve mutual intelligibility. *Prescriptivists*
try to rescue the system by saying ?hey everybody just follow
my rules and we?ll be good; I promise all the good users of
this language have already been following them?.  Meanwhile,
people correct each others? pronunciation and grammar as
it occurs to them, and write dictionaries and grammar books
to make this easier.  These books are not usually ammo for
prescriptivists, but rather *descriptions* of how people
actually behave, at scale.  Prescription is brittle, since it doesn?t
take special cases into account, and requires a central ruling body,
an ?Academy? to define, evolve, and adjudicate.  Description
doesn?t need an Academy because it cedes authority to the
speakers as a whole to correct and evolve, and to put up or
tear down standards (dictionaries, etc.).

So when someone comes along and says, ?You forgot the
style rule book but luckily I have one for you?, you are
dealing with a prescriptivist.  If you say, ?wait, that doesn?t
describe what we are doing?, perhaps from a descriptivist
viewpoint, you will be talking at cross purposes for a while.
The prescriptivist will say, ?that?s OK, I?ll change my rules
for your practice?, but still assume that a central Academy
(of some sort) is the obvious right answer.  That?s what?s
seems to be going on here.  Liu clearly wants to be helpful
and to contribute something valuable to this project, but
in some of his proposals he seems to assume that a
mechanical solution will help, but this has well-known
drawbacks, deeply embedded.  I think style has to be managed
by humans not robots, and in a distributed manner not centralized.

You cannot easily change *any* rules in a large software project,
can you?  That goes for language choice, build tools, SCM, naming
conventions, and even whitespace use.  If we were to use a strong
prescriptive style guide, we would have to have made this decision
at the beginning of the project, or else make a decision to phase it
in very slowly, like we phase in tool chains, SCM (see how hard
that is right now?), language dialects (C++11 anyone?), and so on.
The various objections on this thread to reformatting the source
base are evidence of the sort of work we?d have to do, over months
and months, to guide the project into the purified style.  I think
the consensus will be, unless someone pulls an unprecedented
rabbit out of a hat, that it?s not worth the effort:  We can continue
to tolerate style noise, as long as we have some confidence that
we can catch the worst offenders at review time.  Prescriptivists
and descriptivists agree that there are folks out there whose
noise-to-signal ratio need to be lowered; they disagree about
what to do with it, between the extremes of ?have the Academy?s
robot enforcer refuse their PR?, or ?trust reviewers to apply
social pressure, backed by existing descriptions of known-good
behavior?.  There are other points on the spectrum, such as
hyper-P ?require certification before someone can talk? or
hyper-D ?If I don?t understand it it must be my fault?, and
a middle ground (for us) ?have a robot flag *possible* style
infractions in the PR, for reviewers to comment on?.  I think
that?s the most a robot can do (usefully), and only on future code,
not retroactively.

IMO, in the end, I?m a descriptivist (like most practitioners)
because I think we can get along better on balance without
an Academy, certainly in this matter.  (The Java and JVM specs.,
OTOH, are and have always been prescriptive.)  I?m not hyper-D
either, because, after all, I find style noise distracting and hope
folks will agree to reduce it? which is one reason I wrote the
HotSpot Style Guide.

? John

P.S. <memory-lane>

I was the original author of something that ended up
p0sted at https://wiki.openjdk.java.net/display/HotSpot/StyleGuide.
I took this task on because, at an early point in the project,
there were signs that styles would diverge greatly, making it
hard (yes, harder than today) to decide which example to
follow, to make one?s new code ?fit in? with old code.

(Disclaimer:  I?m talking HotSpot only, not JDK.  I never
had anything to do with Java style rules.  In that day
they were separate projects.)

At that moment, most of HotSpot had been developed by
a few engineers in a room, with one of them asking the
team to make coherent choices about naming and formatting.
Their choices are still with us today.  Also at that moment,
the team was quickly growing, and the personal influence
of the original team was waning.  Some of the new coders
were determined to code in their own special style and
the code base was getting noisy (in style-space).

Linguists note that languages diverge at all scales, and
in the setting of expanding (or fragmenting) social relations.
The smaller groups can exert more control over their
common conventions, while larger ones find it more difficult
to exert control, and usually opt for less control (more
diversity), instead of expensive and brittle central controls
like Academies.  This happened as our project grew.

I wrote the style guide neither because (a) I agreed that
the original style was perfect and wanted to be the new
Style Policeman, nor because (b) I aspired to a better style
that everyone else would adopt as soon as I explained it to
them.  I just wanted (c) to take practical steps to reduce
style-noise.  And it worked, I suppose:  There have been
times when a reviewer said, ?Hey that?s style-noisy, and
you can fix it by reading the online Style Guide.?

One of my sneakier goals in writing the style guide was
making it difficult to use by future would-be Style Police
to browbeat the rest of us into taking a step such as is
being proposed today, to reformat the code base in the
name of ?efficiency? or ?consistency? or whatever.
I?m also averse to confrontation, so it was easy to hope
that even the style-noisy folks in our group (/opto/) would
on balance contribute better if we let them set their own
rules in their own area.  I think that was the case, but as
a result we now have parts of the system which have their
special uglinesses.

But a Style Guide With Teeth or Style Enforcer Robot would
have been almost useless in attacking the root causes of
teams having trouble working together.  The different
whitespace styles are the tip of an iceberg of differences
between design rules, algorithm formulation, workflow,
and more.  A Style-Police Robot would affect, within its purview
of style correction, about 5% of the cross-team dynamics that
swirl across a multi-group team, dynamics which must be dealt
with via ad hoc social negotiations; robots are useless, and only
the simplest rules (like ?be kind?) are deeply useful.

Anyway, that?s why the HotSpot Style Guide says not so
much ?here?s what you must always do? but more ?here
are some good things to do?, and then ?follow local
conventions if they exist?.

One more influence:  I love that George Orwell said,
in his style guide ?Politics and the English Language?,
at the end of a short list of rules:  ?Break any of these
rules sooner than say anything outright barbarous.?
By ?barbarous? I think he means something that your
reader will have trouble understanding (root meaning
of term ?barbarous?), or that would somehow cause a
distraction.  I mentally tip my hat to him when I break
a small rule to make something look (as I think) more
intelligible.  I?m often wrong, and Coleen or David will
quickly correct me during review.  Meanwhile a robot
would just sit there counting whitespaces, oblivious.

</memory-lane>

From xxinliu at amazon.com  Fri Mar 13 08:30:25 2020
From: xxinliu at amazon.com (Liu, Xin)
Date: Fri, 13 Mar 2020 08:30:25 +0000
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <5293DB1A-AF6C-4B35-A530-9B434D335909@oracle.com>
References: <84B163F2-5B97-438F-ACA4-CF84E6FB554C@amazon.com>
 <CA+3eh11y5pEXV4dC-=u4P-JMpd8QT5qr+wY0aqdpRMkiVM3kug@mail.gmail.com>
 <314036DA-D654-4C54-94D7-51436F86DF9A@oracle.com>
 <F8403A86-3D2A-4400-83D7-36B0677AF13D@amazon.com>
 <c0da876e-117b-ce1e-88df-cfe3682cef92@redhat.com>
 <5293DB1A-AF6C-4B35-A530-9B434D335909@oracle.com>
Message-ID: <80F4B12D-1B17-42C1-A9E8-B977A8E52DA5@amazon.com>

Hi,

First of all, I have to admit that using a huge patch to correct all inconsistencies is a bad idea.  I take it back. It would devastate annotation information which is more important than styles.
Thanks John to share his rationale and a bit history behind the blog. I really fell the trap you left.  The reason Jesper criticized my very long code comes  from ?There is no hard line length limit? in the wiki.  I thought it means unlimited. Now we know it?s deliberately ambiguous and developers are at their discretion. It?s also very hard to determine AlignConsecutiveAlignments.

I will step back and try to use a .clang-format to format my new code locally.  I got it. It?s a distributed effort to maintain hotspot codestyle instead of authoritarian rules.  I really appreciate the inputs, which are very valuable for me.

Finally,  I would like share my discover about clang-format as a trick. I think you can deploy clang-format in emacs as well.
I was told to maintain include headers in alphabetical order. It?s mechanical. I found it?s pretty handy to get it done by IncludeCategories.
https://cr.openjdk.java.net/~xliu/8240834/webrev/src/hotspot/.clang-format.html

thanks,
--lx


From: John Rose <john.r.rose at oracle.com>
Date: Thursday, March 12, 2020 at 8:28 PM
To: Andrew Haley <aph at redhat.com>
Cc: "Liu, Xin" <xxinliu at amazon.com>, Jesper Wilhelmsson <jesper.wilhelmsson at oracle.com>, HotSpot Open Source Developers <hotspot-dev at openjdk.java.net>
Subject: RE: [EXTERNAL]Propose to use clang-format to enforce hotspot codestyle


CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.


On Mar 12, 2020, at 2:36 AM, Andrew Haley <aph at redhat.com<mailto:aph at redhat.com>> wrote:

No, please do not do this.

It well mess up the diffs, and it will make auto-merging fail. It will
add considerable costs to the updates project for every backport. It
is in every way a bad idea.

+100

I think one influence in discussions like this one is that they
turn on the ?prescriptive vs. descriptive? distinction discovered
by linguists, but applied (maybe surprisingly) to software notations.
It may take a while to realize it, but this distinction is relevant,
hard to resolve, and subtle.

Here?s a primer from the linguistic world:

https://www.ling.upenn.edu/courses/ling001/prescription.html

Short version:  Languages (and code), as used by humans, vary
over time and place in every possible way.  Variation must be
controlled to preserve mutual intelligibility. *Prescriptivists*
try to rescue the system by saying ?hey everybody just follow
my rules and we?ll be good; I promise all the good users of
this language have already been following them?.  Meanwhile,
people correct each others? pronunciation and grammar as
it occurs to them, and write dictionaries and grammar books
to make this easier.  These books are not usually ammo for
prescriptivists, but rather *descriptions* of how people
actually behave, at scale.  Prescription is brittle, since it doesn?t
take special cases into account, and requires a central ruling body,
an ?Academy? to define, evolve, and adjudicate.  Description
doesn?t need an Academy because it cedes authority to the
speakers as a whole to correct and evolve, and to put up or
tear down standards (dictionaries, etc.).

So when someone comes along and says, ?You forgot the
style rule book but luckily I have one for you?, you are
dealing with a prescriptivist.  If you say, ?wait, that doesn?t
describe what we are doing?, perhaps from a descriptivist
viewpoint, you will be talking at cross purposes for a while.
The prescriptivist will say, ?that?s OK, I?ll change my rules
for your practice?, but still assume that a central Academy
(of some sort) is the obvious right answer.  That?s what?s
seems to be going on here.  Liu clearly wants to be helpful
and to contribute something valuable to this project, but
in some of his proposals he seems to assume that a
mechanical solution will help, but this has well-known
drawbacks, deeply embedded.  I think style has to be managed
by humans not robots, and in a distributed manner not centralized.

You cannot easily change *any* rules in a large software project,
can you?  That goes for language choice, build tools, SCM, naming
conventions, and even whitespace use.  If we were to use a strong
prescriptive style guide, we would have to have made this decision
at the beginning of the project, or else make a decision to phase it
in very slowly, like we phase in tool chains, SCM (see how hard
that is right now?), language dialects (C++11 anyone?), and so on.
The various objections on this thread to reformatting the source
base are evidence of the sort of work we?d have to do, over months
and months, to guide the project into the purified style.  I think
the consensus will be, unless someone pulls an unprecedented
rabbit out of a hat, that it?s not worth the effort:  We can continue
to tolerate style noise, as long as we have some confidence that
we can catch the worst offenders at review time.  Prescriptivists
and descriptivists agree that there are folks out there whose
noise-to-signal ratio need to be lowered; they disagree about
what to do with it, between the extremes of ?have the Academy?s
robot enforcer refuse their PR?, or ?trust reviewers to apply
social pressure, backed by existing descriptions of known-good
behavior?.  There are other points on the spectrum, such as
hyper-P ?require certification before someone can talk? or
hyper-D ?If I don?t understand it it must be my fault?, and
a middle ground (for us) ?have a robot flag *possible* style
infractions in the PR, for reviewers to comment on?.  I think
that?s the most a robot can do (usefully), and only on future code,
not retroactively.

IMO, in the end, I?m a descriptivist (like most practitioners)
because I think we can get along better on balance without
an Academy, certainly in this matter.  (The Java and JVM specs.,
OTOH, are and have always been prescriptive.)  I?m not hyper-D
either, because, after all, I find style noise distracting and hope
folks will agree to reduce it? which is one reason I wrote the
HotSpot Style Guide.

? John

P.S. <memory-lane>

I was the original author of something that ended up
p0sted at https://wiki.openjdk.java.net/display/HotSpot/StyleGuide.
I took this task on because, at an early point in the project,
there were signs that styles would diverge greatly, making it
hard (yes, harder than today) to decide which example to
follow, to make one?s new code ?fit in? with old code.

(Disclaimer:  I?m talking HotSpot only, not JDK.  I never
had anything to do with Java style rules.  In that day
they were separate projects.)

At that moment, most of HotSpot had been developed by
a few engineers in a room, with one of them asking the
team to make coherent choices about naming and formatting.
Their choices are still with us today.  Also at that moment,
the team was quickly growing, and the personal influence
of the original team was waning.  Some of the new coders
were determined to code in their own special style and
the code base was getting noisy (in style-space).

Linguists note that languages diverge at all scales, and
in the setting of expanding (or fragmenting) social relations.
The smaller groups can exert more control over their
common conventions, while larger ones find it more difficult
to exert control, and usually opt for less control (more
diversity), instead of expensive and brittle central controls
like Academies.  This happened as our project grew.

I wrote the style guide neither because (a) I agreed that
the original style was perfect and wanted to be the new
Style Policeman, nor because (b) I aspired to a better style
that everyone else would adopt as soon as I explained it to
them.  I just wanted (c) to take practical steps to reduce
style-noise.  And it worked, I suppose:  There have been
times when a reviewer said, ?Hey that?s style-noisy, and
you can fix it by reading the online Style Guide.?

One of my sneakier goals in writing the style guide was
making it difficult to use by future would-be Style Police
to browbeat the rest of us into taking a step such as is
being proposed today, to reformat the code base in the
name of ?efficiency? or ?consistency? or whatever.
I?m also averse to confrontation, so it was easy to hope
that even the style-noisy folks in our group (/opto/) would
on balance contribute better if we let them set their own
rules in their own area.  I think that was the case, but as
a result we now have parts of the system which have their
special uglinesses.

But a Style Guide With Teeth or Style Enforcer Robot would
have been almost useless in attacking the root causes of
teams having trouble working together.  The different
whitespace styles are the tip of an iceberg of differences
between design rules, algorithm formulation, workflow,
and more.  A Style-Police Robot would affect, within its purview
of style correction, about 5% of the cross-team dynamics that
swirl across a multi-group team, dynamics which must be dealt
with via ad hoc social negotiations; robots are useless, and only
the simplest rules (like ?be kind?) are deeply useful.

Anyway, that?s why the HotSpot Style Guide says not so
much ?here?s what you must always do? but more ?here
are some good things to do?, and then ?follow local
conventions if they exist?.

One more influence:  I love that George Orwell said,
in his style guide ?Politics and the English Language?,
at the end of a short list of rules:  ?Break any of these
rules sooner than say anything outright barbarous.?
By ?barbarous? I think he means something that your
reader will have trouble understanding (root meaning
of term ?barbarous?), or that would somehow cause a
distraction.  I mentally tip my hat to him when I break
a small rule to make something look (as I think) more
intelligible.  I?m often wrong, and Coleen or David will
quickly correct me during review.  Meanwhile a robot
would just sit there counting whitespaces, oblivious.

</memory-lane>

From matthias.baesken at sap.com  Fri Mar 13 08:31:04 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 13 Mar 2020 08:31:04 +0000
Subject: RFR: 8240824: enhance print_full_memory_info on Linux by THP
 related information
In-Reply-To: <CAA-vtUzKF7qHzEiNjn8E_Gg6a5pGdheB0RA0tfzT86ywgJL3NQ@mail.gmail.com>
References: <AM0PR02MB42272402734B0AD4B4E304CC93FF0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <CAA-vtUzKF7qHzEiNjn8E_Gg6a5pGdheB0RA0tfzT86ywgJL3NQ@mail.gmail.com>
Message-ID: <AM0PR02MB42274C9523CE1D8ED33765B793FA0@AM0PR02MB4227.eurprd02.prod.outlook.com>

Thanks. May I have a second review ?

Best Regards, Matthias


From: Thomas St?fe <thomas.stuefe at gmail.com>
Sent: Donnerstag, 12. M?rz 2020 14:33
To: Baesken, Matthias <matthias.baesken at sap.com>
Cc: hotspot-dev at openjdk.java.net
Subject: Re: RFR: 8240824: enhance print_full_memory_info on Linux by THP related information

Hi Matthias,

too verbose for my taste:


+  st->print_cr("/sys/kernel/mm/transparent_hugepage/enabled (transparent huge pages mode):");

I would cut that down to either "/sys/kernel/mm/transparent_hugepage/enabled" or to "transparent huge pages mode". Probably the former.

Otherwise looks good. I do not need a new webrev.

Cheers, Thomas



On Tue, Mar 10, 2020 at 4:09 PM Baesken, Matthias <matthias.baesken at sap.com<mailto:matthias.baesken at sap.com>> wrote:
Hello, please review this small enhancement .

On Linux, the change adds output of some files related to THP (transparent huge pages)   configuration to the hs_err file; the settings in those files configure how transparent huge pages are used on the system.

Example output in the hs_err file from a test machine :


/sys/kernel/mm/transparent_hugepage/enabled (transparent huge pages mode):
[always] madvise never

/sys/kernel/mm/transparent_hugepage/defrag (defrag/compaction efforts parameter):
[always] madvise never




Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8240824

http://cr.openjdk.java.net/~mbaesken/webrevs/8240824.0/


Thanks, Matthias

From john.r.rose at oracle.com  Fri Mar 13 08:42:12 2020
From: john.r.rose at oracle.com (John Rose)
Date: Fri, 13 Mar 2020 01:42:12 -0700
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <80F4B12D-1B17-42C1-A9E8-B977A8E52DA5@amazon.com>
References: <84B163F2-5B97-438F-ACA4-CF84E6FB554C@amazon.com>
 <CA+3eh11y5pEXV4dC-=u4P-JMpd8QT5qr+wY0aqdpRMkiVM3kug@mail.gmail.com>
 <314036DA-D654-4C54-94D7-51436F86DF9A@oracle.com>
 <F8403A86-3D2A-4400-83D7-36B0677AF13D@amazon.com>
 <c0da876e-117b-ce1e-88df-cfe3682cef92@redhat.com>
 <5293DB1A-AF6C-4B35-A530-9B434D335909@oracle.com>
 <80F4B12D-1B17-42C1-A9E8-B977A8E52DA5@amazon.com>
Message-ID: <3FF7EAE3-7383-451D-975D-C8998C17CF8D@oracle.com>

On Mar 13, 2020, at 1:30 AM, Liu, Xin <xxinliu at amazon.com> wrote:
> 
> First of all, I have to admit that using a huge patch to correct all inconsistencies is a bad idea.  I take it back. It would devastate annotation information which is more important than styles.
> Thanks John to share his rationale and a bit history behind the blog. I really fell the trap you left.  The reason Jesper criticized my very long code comes  from ?There is no hard line length limit? in the wiki.  I thought it means unlimited. Now we know it?s deliberately ambiguous and developers are at their discretion. It?s also very hard to determine AlignConsecutiveAlignments.  
>  
> I will step back and try to use a .clang-format to format my new code locally.  I got it. It?s a distributed effort to maintain hotspot codestyle instead of authoritarian rules.  I really appreciate the inputs, which are very valuable for me.
>  
> Finally,  I would like share my discover about clang-format as a trick. I think you can deploy clang-format in emacs as well.   
> I was told to maintain include headers in alphabetical order. It?s mechanical. I found it?s pretty handy to get it done by IncludeCategories.  
> https://cr.openjdk.java.net/~xliu/8240834/webrev/src/hotspot/.clang-format.html

Thank you, Liu!  You probably noticed that I admitted it could possibly
be helpful to have some sort of automatic detection of style variations,
perhaps as reports on the source code base as a whole (kind of like
coverage data, but less useful?).  Or, more likely, automatic flagging
of style variations during the review of a pull request; we have a place
for robots like this, but they only do the most basic style checks,
and those are mandatory.  (Yes we have a tiny number of mandatory
format rules, under jcheck.)  But flagging a richer set of style variations,
when there is a chance to fix it, might help reviewers.  Or maybe not;
the overall problem with such robots is they interrupt the humans
when they are trying to talk.  But I would welcome something that
would occasionally flag a questionable indentation or identifier spelling,
as long as it avoided false positives, and as long is it didn?t prevent
the humans from deciding whether the rule should be broken or not.

? John

From aph at redhat.com  Fri Mar 13 09:08:57 2020
From: aph at redhat.com (Andrew Haley)
Date: Fri, 13 Mar 2020 09:08:57 +0000
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <5B92DB3F-A436-4358-A8B4-56977A6EDE6E@oracle.com>
References: <CA+3eh11or=pxZ7D752GJbNm7a7=OofxX74Gmza2HJSmgD_9ZNA@mail.gmail.com>
 <5B92DB3F-A436-4358-A8B4-56977A6EDE6E@oracle.com>
Message-ID: <7fe5e846-8157-0d53-69a7-f2c8cf878757@redhat.com>

On 3/13/20 2:12 AM, John Rose wrote:
> (push '("/vmSymbols\\.hpp\\'" . fundamental-mode) auto-mode-alist)
> (push '("/globals\\.hpp\\'" . fundamental-mode) auto-mode-alist)
> (push '("/vmStructs\\.cpp\\'" . fundamental-mode) auto-mode-alist)
>
> There are surely less blunt ways to do it, but it?s better than
> the frustrating lock-up you describe, which I am also familiar
> with (Aquamacs, many recent versions).

I could submit a patch to the offending files so that everyone editing
them with emacs will benefit. I'm not sure if that'll offend anyone's
sense of purity, though. It only needs this at the end of the files:

// Local Variables:
// mode: fundamental
// End:

WDYT?

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From adinn at redhat.com  Fri Mar 13 09:42:52 2020
From: adinn at redhat.com (Andrew Dinn)
Date: Fri, 13 Mar 2020 09:42:52 +0000
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <7fe5e846-8157-0d53-69a7-f2c8cf878757@redhat.com>
References: <CA+3eh11or=pxZ7D752GJbNm7a7=OofxX74Gmza2HJSmgD_9ZNA@mail.gmail.com>
 <5B92DB3F-A436-4358-A8B4-56977A6EDE6E@oracle.com>
 <7fe5e846-8157-0d53-69a7-f2c8cf878757@redhat.com>
Message-ID: <71a1abc6-cbd8-9a1e-2027-d5ac5ba9ab9b@redhat.com>

On 13/03/2020 09:08, Andrew Haley wrote:
> I could submit a patch to the offending files so that everyone editing
> them with emacs will benefit. I'm not sure if that'll offend anyone's
> sense of purity, though. It only needs this at the end of the files:
> 
> // Local Variables:
> // mode: fundamental
> // End:
> 
> WDYT?
All doubt re that question can be readily resolved by my handy,
/prescriptive/ emacs-as-one-true-OpenJDK-IDE-of-choice guide. Any takers?

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill


From aph at redhat.com  Fri Mar 13 09:52:00 2020
From: aph at redhat.com (Andrew Haley)
Date: Fri, 13 Mar 2020 09:52:00 +0000
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <71a1abc6-cbd8-9a1e-2027-d5ac5ba9ab9b@redhat.com>
References: <CA+3eh11or=pxZ7D752GJbNm7a7=OofxX74Gmza2HJSmgD_9ZNA@mail.gmail.com>
 <5B92DB3F-A436-4358-A8B4-56977A6EDE6E@oracle.com>
 <7fe5e846-8157-0d53-69a7-f2c8cf878757@redhat.com>
 <71a1abc6-cbd8-9a1e-2027-d5ac5ba9ab9b@redhat.com>
Message-ID: <569b8ee1-9142-5422-3b0d-91fce915b027@redhat.com>

On 3/13/20 9:42 AM, Andrew Dinn wrote:
> On 13/03/2020 09:08, Andrew Haley wrote:
>> I could submit a patch to the offending files so that everyone editing
>> them with emacs will benefit. I'm not sure if that'll offend anyone's
>> sense of purity, though. It only needs this at the end of the files:
>>
>> // Local Variables:
>> // mode: fundamental
>> // End:
>>
>> WDYT?
> All doubt re that question can be readily resolved by my handy,
> /prescriptive/ emacs-as-one-true-OpenJDK-IDE-of-choice guide. Any takers?

+1. All else is heresy.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From aph at redhat.com  Fri Mar 13 09:52:49 2020
From: aph at redhat.com (Andrew Haley)
Date: Fri, 13 Mar 2020 09:52:49 +0000
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <5293DB1A-AF6C-4B35-A530-9B434D335909@oracle.com>
References: <84B163F2-5B97-438F-ACA4-CF84E6FB554C@amazon.com>
 <CA+3eh11y5pEXV4dC-=u4P-JMpd8QT5qr+wY0aqdpRMkiVM3kug@mail.gmail.com>
 <314036DA-D654-4C54-94D7-51436F86DF9A@oracle.com>
 <F8403A86-3D2A-4400-83D7-36B0677AF13D@amazon.com>
 <c0da876e-117b-ce1e-88df-cfe3682cef92@redhat.com>
 <5293DB1A-AF6C-4B35-A530-9B434D335909@oracle.com>
Message-ID: <04ef50ed-9f78-3d9d-d1bf-bf2014fbd160@redhat.com>

On 3/13/20 3:27 AM, John Rose wrote:
> So when someone comes along and says, ?You forgot the
> style rule book but luckily I have one for you?, you are
> dealing with a prescriptivist.  If you say, ?wait, that doesn?t
> describe what we are doing?, perhaps from a descriptivist
> viewpoint, you will be talking at cross purposes for a while.
> The prescriptivist will say, ?that?s OK, I?ll change my rules
> for your practice?, but still assume that a central Academy
> (of some sort) is the obvious right answer.

I have been programming for a long while and I'm interested in
linguistics, but I have never seen the connection between programming
style enforcement and prescriptivist linguistics explained like this
before. Thank you! That was a very useful addition to my mental
furniture.

I'm reminded of David Crystal's (?) fantasy about linguists studying
whale song in which one, on finding a whale who sings differently from
the others, says with excitement "that whale made a mistake!"  :-)

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From adinn at redhat.com  Fri Mar 13 11:51:34 2020
From: adinn at redhat.com (Andrew Dinn)
Date: Fri, 13 Mar 2020 11:51:34 +0000
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <04ef50ed-9f78-3d9d-d1bf-bf2014fbd160@redhat.com>
References: <84B163F2-5B97-438F-ACA4-CF84E6FB554C@amazon.com>
 <CA+3eh11y5pEXV4dC-=u4P-JMpd8QT5qr+wY0aqdpRMkiVM3kug@mail.gmail.com>
 <314036DA-D654-4C54-94D7-51436F86DF9A@oracle.com>
 <F8403A86-3D2A-4400-83D7-36B0677AF13D@amazon.com>
 <c0da876e-117b-ce1e-88df-cfe3682cef92@redhat.com>
 <5293DB1A-AF6C-4B35-A530-9B434D335909@oracle.com>
 <04ef50ed-9f78-3d9d-d1bf-bf2014fbd160@redhat.com>
Message-ID: <551114a3-4859-4f2f-bd0b-fd73c18d72c3@redhat.com>

On 13/03/2020 09:52, Andrew Haley wrote:
> I'm reminded of David Crystal's (?) fantasy about linguists studying
> whale song in which one, on finding a whale who sings differently from
> the others, says with excitement "that whale made a mistake!"  :-)
I suspect that joke probably constitutes John's (excellent BTW) essay in
redux. John Rose the David Crystal of language virtual machines?

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill


From john.r.rose at oracle.com  Sat Mar 14 04:24:44 2020
From: john.r.rose at oracle.com (John Rose)
Date: Fri, 13 Mar 2020 21:24:44 -0700
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <04ef50ed-9f78-3d9d-d1bf-bf2014fbd160@redhat.com>
References: <84B163F2-5B97-438F-ACA4-CF84E6FB554C@amazon.com>
 <CA+3eh11y5pEXV4dC-=u4P-JMpd8QT5qr+wY0aqdpRMkiVM3kug@mail.gmail.com>
 <314036DA-D654-4C54-94D7-51436F86DF9A@oracle.com>
 <F8403A86-3D2A-4400-83D7-36B0677AF13D@amazon.com>
 <c0da876e-117b-ce1e-88df-cfe3682cef92@redhat.com>
 <5293DB1A-AF6C-4B35-A530-9B434D335909@oracle.com>
 <04ef50ed-9f78-3d9d-d1bf-bf2014fbd160@redhat.com>
Message-ID: <5AD7F908-C571-44A8-AABC-845E90EA3641@oracle.com>

On Mar 13, 2020, at 2:52 AM, Andrew Haley <aph at redhat.com> wrote:
> 
> I have been programming for a long while and I'm interested in
> linguistics, but I have never seen the connection between programming
> style enforcement and prescriptivist linguistics explained like this
> before. Thank you! That was a very useful addition to my mental
> furniture.

My hat is off to John McWhorter; his books and lectures (on teach12)
are the source of 90% of what I know about linguistics.  He describes
linguistic change as what a geek might call ergodic or thermodynamic
traversal of state space.  He doesn?t use physics terms for this, but
appeals to the way his cat inhabits every surface and cranny of its
environment.  ?If it can go somewhere, it eventually will.?  The
exploration is on a time scale that varies according to the number
of people who share the convention, with small unconnected
tribes and villages moving the fastest, and literate megabucks
moving the slowest.  I think a lot of conventional behaviors are
like this.

> I'm reminded of David Crystal's (?) fantasy about linguists studying
> whale song in which one, on finding a whale who sings differently from
> the others, says with excitement "that whale made a mistake!"  :-)

Excellent!  And maybe the whales are just waiting for the experts
to leave, so they can just kick back and sing natural.

? John

From john.r.rose at oracle.com  Sat Mar 14 04:31:58 2020
From: john.r.rose at oracle.com (John Rose)
Date: Fri, 13 Mar 2020 21:31:58 -0700
Subject: Propose to use clang-format to enforce hotspot codestyle
In-Reply-To: <5AD7F908-C571-44A8-AABC-845E90EA3641@oracle.com>
References: <84B163F2-5B97-438F-ACA4-CF84E6FB554C@amazon.com>
 <CA+3eh11y5pEXV4dC-=u4P-JMpd8QT5qr+wY0aqdpRMkiVM3kug@mail.gmail.com>
 <314036DA-D654-4C54-94D7-51436F86DF9A@oracle.com>
 <F8403A86-3D2A-4400-83D7-36B0677AF13D@amazon.com>
 <c0da876e-117b-ce1e-88df-cfe3682cef92@redhat.com>
 <5293DB1A-AF6C-4B35-A530-9B434D335909@oracle.com>
 <04ef50ed-9f78-3d9d-d1bf-bf2014fbd160@redhat.com>
 <5AD7F908-C571-44A8-AABC-845E90EA3641@oracle.com>
Message-ID: <E2054065-F83F-41CA-9161-52A886C8D7B3@oracle.com>

On Mar 13, 2020, at 9:24 PM, John Rose <john.r.rose at oracle.com> wrote:
> 
> megabucks

(Thanks autocorrect, but it?s mega-blocks I wanted.)

From huangjia at loongson.cn  Mon Mar 16 03:54:06 2020
From: huangjia at loongson.cn (Jia Huang)
Date: Mon, 16 Mar 2020 11:54:06 +0800
Subject: RFR: 8240222: [TESTBUG] gtest/jfr/test_networkUtilization.cpp
 failed when the number of tests is greater than or equal to 2
In-Reply-To: <d56604a5-5873-718d-eda0-86b95a7234b4@loongson.cn>
References: <64960cc9-5021-d265-ab30-9cf9a91cc6e3@loongson.cn>
 <8f143c78-6bba-5f11-21ad-3096115287e1@oracle.com>
 <4f6c2613-de5a-cc28-f208-1a763270a5d0@loongson.cn>
 <44483657-df1f-f9cc-9e6a-f73b0e602f72@oracle.com>
 <931ecb2e-9898-a371-0aa9-bf463b9fdcf6@loongson.cn>
 <e9d5cca3-1f35-7708-1bdb-bb4b5e673d27@loongson.cn>
 <a5b687be-a9b4-4bc7-bace-b94f723cbdab@default>
 <74960f03-542c-f7f5-9f46-994d355d62d3@loongson.cn>
 <50d5e790-ef5c-4c35-bc14-71577df50c9a@default>
 <d56604a5-5873-718d-eda0-86b95a7234b4@loongson.cn>
Message-ID: <9e526456-0547-b24a-5199-19d6dd465c88@loongson.cn>

Hi Markus,

Is it a trivial patch and OK to be pushed?
Or shall we wait for one more review?

Best regards,
Jia

? 2020/3/10 19:37, Jia Huang ??:
> Thanks Markus.
>
> Best regards,
> Jia
>
> ? 2020/3/10 18:15, Markus Gronlund ??:
>> Hi Jia,
>>
>> Looks like it could work.
>>
>> Thanks
>> Markus
>>
>> -----Original Message-----
>> From: Jia Huang <huangjia at loongson.cn>
>> Sent: den 10 mars 2020 02:30
>> To: Markus Gronlund <markus.gronlund at oracle.com>
>> Cc: hotspot-jfr-dev at openjdk.java.net; hotspot-dev at openjdk.java.net
>> Subject: Re: RFR: 8240222: [TESTBUG] 
>> gtest/jfr/test_networkUtilization.cpp failed when the number of tests 
>> is greater than or equal to 2
>>
>> Thanks Markus.
>>
>> Could you give some comments or advice on this one:
>> Webrev:? http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/ ?
>>
>> Best regards,
>> Jia
>>
>> ? 2020/3/9 19:41, Markus Gronlund ??:
>>> The interface_id assigns a unique primary key for each interface.
>>>
>>> To support GTEST=REPEAT, I think the easiest way is to add a 
>>> corresponding counter in the gtest itself that mirror the production 
>>> code:
>>>
>>> That is, as an example, instead of the "add_interface("eth0", 1), that
>>> hardcodes the traceid = 1, have it take a value from the corresponding
>>> counter, i.e. "add_interface("eth0", next_id++)";
>>>
>>> Thanks
>>> Markus
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: Jia Huang <huangjia at loongson.cn>
>>> Sent: den 9 mars 2020 10:57
>>> To: hotspot-dev at openjdk.java.net; hotspot-jfr-dev at openjdk.java.net
>>> Subject: Re: RFR: 8240222: [TESTBUG]
>>> gtest/jfr/test_networkUtilization.cpp failed when the number of tests
>>> is greater than or equal to 2
>>>
>>> In addition, the static local variable "id" is assigned in the 
>>> function of "new_entry", that is in 
>>> share/jfr/periodic/jfrNetworkUtilization.cpp.
>>>
>>> ---------------------------------------------------------------
>>> share/jfr/periodic/jfrNetworkUtilization.cpp
>>>
>>> static InterfaceEntry& new_entry(const NetworkInterface* iface,
>>> GrowableArray<InterfaceEntry>* interfaces) {
>>> ?? ? assert(iface != NULL, "invariant");
>>> ?? ? assert(interfaces != NULL, "invariant");
>>>
>>> ?? ? // single threaded premise
>>> ?? ? static traceid interface_id = 0;????????????????? ? ?? // this is
>>> the static local variable
>>>
>>> ?? ? const char* name = iface->get_name();
>>> ?? ? assert(name != NULL, "invariant");
>>>
>>> ?? ? InterfaceEntry entry;
>>> ?? ? const size_t length = strlen(name);
>>> ?? ? entry.name = NEW_C_HEAP_ARRAY(char, length + 1, mtInternal);
>>> ?? ? strncpy(entry.name, name, length + 1);
>>> ?? ? entry.id = ++interface_id;
>>> ?? ? entry.bytes_in = iface->get_bytes_in();
>>> ?? ? entry.bytes_out = iface->get_bytes_out();
>>> ?? ? entry.written = false;
>>> ?? ? return _interfaces->at(_interfaces->append(entry));
>>> }
>>> ---------------------------------------------------------------
>>>
>>> ? 2020/3/9 15:29, Jia Huang ??:
>>>> Haha, I'm also trying to email hotspot-jfr-dev.
>>>> Thank you, David.
>>>>
>>>> ? 2020/3/9 13:03, David Holmes ??:
>>>>> Hi Jia,
>>>>>
>>>>> I see this is getting no response so have cc'd hotspot-jfr-dev as
>>>>> they should have a better idea of how this gtest can and cannot be 
>>>>> used.
>>>>>
>>>>> Just because the GTest framework supports "REPEAT" it doesn't mean
>>>>> all tests can, or should, be repeatable.
>>>>>
>>>>> Cheers,
>>>>> David
>>>>>
>>>>> On 29/02/2020 1:53 pm, Jia Huang wrote:
>>>>>> Hi David,
>>>>>>
>>>>>> ? 2020/2/28 21:16, David Holmes ??:
>>>>>>> Hi Jia,
>>>>>>>
>>>>>>> On 28/02/2020 11:08 pm, Jia Huang wrote:
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8240222
>>>>>>>> Webrev: http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/
>>>>>>>>
>>>>>>>> gtest/jfr/test_networkUtilization.cpp failed when the number of
>>>>>>>> tests is greater than or equal to 2.
>>>>>>>> As shown in the following code, the test failed because "traceid
>>>>>>>> id" is a static local variable.
>>>>>>>> When the test runs a second time, "id" continues to grow,
>>>>>>>> resulting in the return value of "i" being "_interfaces.end()".
>>>>>>> Interesting! I'm not at all familiar with the details of gtest and
>>>>>>> how it actually runs things. But I would expect that we have many
>>>>>>> tests that rely on starting from a cleanly initialized VM state,
>>>>>>> or test state, and so can't simply be repeated (it would be like
>>>>>>> editing an arbitrary test and putting the body of main into a
>>>>>>> for-loop!).
>>>>>> I'm sorry that my last description may be a little incomplete.
>>>>>> JfrTestNetworkUtilization is a group of tests,it includes four
>>>>>> individual tests:
>>>>>> --JfrTestNetworkUtilization.RequestFunctionBasic_test_vm
>>>>>> --JfrTestNetworkUtilization.RequestFunctionMultiple_test_vm
>>>>>> --JfrTestNetworkUtilization.InterfaceRemoved_test_vm
>>>>>> ? ?? --JfrTestNetworkUtilization.InterfaceReset_test_vm
>>>>>>
>>>>>> If you only run a individual test once, such as the following
>>>>>> Testing, it still fails.
>>>>>> The reason for the failure, as I said last time, is also because of
>>>>>> static local variables.
>>>>>> In the "RequestFunctionMultiple_test_vm" test, the sequence number
>>>>>> of "eth0" is 2.
>>>>>> However, the initial value of the static local variable "id" is 0.
>>>>>>
>>>>>> Testting:
>>>>>> ? ??? --make run-test
>>>>>> TEST=gtest:JfrTestNetworkUtilization.RequestFunctionMultiple_test_v
>>>>>> m
>>>>>> GTEST=REPEAT=1 CONF=release
>>>>>> ? ??? --make run-test
>>>>>> TEST=gtest:JfrTestNetworkUtilization.InterfaceRemoved_test_vm
>>>>>> GTEST=REPEAT=1 CONF=release
>>>>>> ? ??? --make run-test
>>>>>> TEST=gtest:JfrTestNetworkUtilization.InterfaceReset_test_vm
>>>>>> GTEST=REPEAT=1 CONF=release
>>>>>>
>>>>>>> So I'm not sure what the right way to fix this is:
>>>>>>> - don't use REPEAT?
>>>>>>> - declare the test as not repeatable somehow?
>>>>>>> - change the test to make it repeatable?
>>>>>> In the doc/testing.md or doc/testing.html, Gtest keywords inclue
>>>>>> "REPEAT", so I think this means that all GTEST tests should be able
>>>>>> to use the "REPEAT parameter.
>>>>>>
>>>>>>
>>>>>> Best regards,
>>>>>> Jia
>>>>>>
>>>>>>
>>>>>>> Cheers,
>>>>>>> David
>>>>>>>
>>>>>>>> -----------------------------------------------------------------
>>>>>>>> -----------
>>>>>>>>
>>>>>>>> test/hotspot/gtest/jfr/test_networkUtilization.cpp
>>>>>>>>
>>>>>>>> static const MockNetworkInterface& get_interface(traceid id) {
>>>>>>>> std::list<MockNetworkInterface>::const_iterator i =
>>>>>>>> _interfaces.begin();
>>>>>>>> ? ?? for (; i != _interfaces.end(); ++i) {
>>>>>>>> ? ???? if (i->id == id) {
>>>>>>>> ? ?????? break;
>>>>>>>> ? ???? }
>>>>>>>> ? ?? }
>>>>>>>> ? ?? return *i;
>>>>>>>> }
>>>>>>>> -----------------------------------------------------------------
>>>>>>>> -----------
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Testing:
>>>>>>>> ? ???? - make run-test TEST=gtest:JfrTestNetworkUtilization
>>>>>>>> GTEST=REPEAT=2 CONF=server-release
>>>>>>>> ? ???? - make run-test TEST=gtest:JfrTestNetworkUtilization
>>>>>>>> GTEST=REPEAT=2 CONF=server-fastdebug
>>>>>>>>
>>>>>>>> Thanks a lot
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>> Jia
>>>>>>>>


From erik.gahlin at oracle.com  Mon Mar 16 04:00:14 2020
From: erik.gahlin at oracle.com (Erik Gahlin)
Date: Mon, 16 Mar 2020 05:00:14 +0100
Subject: RFR: 8240222: [TESTBUG] gtest/jfr/test_networkUtilization.cpp
 failed when the number of tests is greater than or equal to 2
In-Reply-To: <9e526456-0547-b24a-5199-19d6dd465c88@loongson.cn>
References: <64960cc9-5021-d265-ab30-9cf9a91cc6e3@loongson.cn>
 <8f143c78-6bba-5f11-21ad-3096115287e1@oracle.com>
 <4f6c2613-de5a-cc28-f208-1a763270a5d0@loongson.cn>
 <44483657-df1f-f9cc-9e6a-f73b0e602f72@oracle.com>
 <931ecb2e-9898-a371-0aa9-bf463b9fdcf6@loongson.cn>
 <e9d5cca3-1f35-7708-1bdb-bb4b5e673d27@loongson.cn>
 <a5b687be-a9b4-4bc7-bace-b94f723cbdab@default>
 <74960f03-542c-f7f5-9f46-994d355d62d3@loongson.cn>
 <50d5e790-ef5c-4c35-bc14-71577df50c9a@default>
 <d56604a5-5873-718d-eda0-86b95a7234b4@loongson.cn>
 <9e526456-0547-b24a-5199-19d6dd465c88@loongson.cn>
Message-ID: <672D6DA7-6FA1-4021-A9F4-FD951DE6D6CA@oracle.com>

Looks reasonable.

Erik

> On 16 Mar 2020, at 04:54, Jia Huang <huangjia at loongson.cn> wrote:
> 
> Hi Markus,
> 
> Is it a trivial patch and OK to be pushed?
> Or shall we wait for one more review?
> 
> Best regards,
> Jia
> 
> ? 2020/3/10 19:37, Jia Huang ??:
>> Thanks Markus.
>> 
>> Best regards,
>> Jia
>> 
>> ? 2020/3/10 18:15, Markus Gronlund ??:
>>> Hi Jia,
>>> 
>>> Looks like it could work.
>>> 
>>> Thanks
>>> Markus
>>> 
>>> -----Original Message-----
>>> From: Jia Huang <huangjia at loongson.cn>
>>> Sent: den 10 mars 2020 02:30
>>> To: Markus Gronlund <markus.gronlund at oracle.com>
>>> Cc: hotspot-jfr-dev at openjdk.java.net; hotspot-dev at openjdk.java.net
>>> Subject: Re: RFR: 8240222: [TESTBUG] gtest/jfr/test_networkUtilization.cpp failed when the number of tests is greater than or equal to 2
>>> 
>>> Thanks Markus.
>>> 
>>> Could you give some comments or advice on this one:
>>> Webrev:  http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/ ?
>>> 
>>> Best regards,
>>> Jia
>>> 
>>> ? 2020/3/9 19:41, Markus Gronlund ??:
>>>> The interface_id assigns a unique primary key for each interface.
>>>> 
>>>> To support GTEST=REPEAT, I think the easiest way is to add a corresponding counter in the gtest itself that mirror the production code:
>>>> 
>>>> That is, as an example, instead of the "add_interface("eth0", 1), that
>>>> hardcodes the traceid = 1, have it take a value from the corresponding
>>>> counter, i.e. "add_interface("eth0", next_id++)";
>>>> 
>>>> Thanks
>>>> Markus
>>>> 
>>>> 
>>>> 
>>>> -----Original Message-----
>>>> From: Jia Huang <huangjia at loongson.cn>
>>>> Sent: den 9 mars 2020 10:57
>>>> To: hotspot-dev at openjdk.java.net; hotspot-jfr-dev at openjdk.java.net
>>>> Subject: Re: RFR: 8240222: [TESTBUG]
>>>> gtest/jfr/test_networkUtilization.cpp failed when the number of tests
>>>> is greater than or equal to 2
>>>> 
>>>> In addition, the static local variable "id" is assigned in the function of "new_entry", that is in share/jfr/periodic/jfrNetworkUtilization.cpp.
>>>> 
>>>> ---------------------------------------------------------------
>>>> share/jfr/periodic/jfrNetworkUtilization.cpp
>>>> 
>>>> static InterfaceEntry& new_entry(const NetworkInterface* iface,
>>>> GrowableArray<InterfaceEntry>* interfaces) {
>>>>      assert(iface != NULL, "invariant");
>>>>      assert(interfaces != NULL, "invariant");
>>>> 
>>>>      // single threaded premise
>>>>      static traceid interface_id = 0;                       // this is
>>>> the static local variable
>>>> 
>>>>      const char* name = iface->get_name();
>>>>      assert(name != NULL, "invariant");
>>>> 
>>>>      InterfaceEntry entry;
>>>>      const size_t length = strlen(name);
>>>>      entry.name = NEW_C_HEAP_ARRAY(char, length + 1, mtInternal);
>>>>      strncpy(entry.name, name, length + 1);
>>>>      entry.id = ++interface_id;
>>>>      entry.bytes_in = iface->get_bytes_in();
>>>>      entry.bytes_out = iface->get_bytes_out();
>>>>      entry.written = false;
>>>>      return _interfaces->at(_interfaces->append(entry));
>>>> }
>>>> ---------------------------------------------------------------
>>>> 
>>>> ? 2020/3/9 15:29, Jia Huang ??:
>>>>> Haha, I'm also trying to email hotspot-jfr-dev.
>>>>> Thank you, David.
>>>>> 
>>>>> ? 2020/3/9 13:03, David Holmes ??:
>>>>>> Hi Jia,
>>>>>> 
>>>>>> I see this is getting no response so have cc'd hotspot-jfr-dev as
>>>>>> they should have a better idea of how this gtest can and cannot be used.
>>>>>> 
>>>>>> Just because the GTest framework supports "REPEAT" it doesn't mean
>>>>>> all tests can, or should, be repeatable.
>>>>>> 
>>>>>> Cheers,
>>>>>> David
>>>>>> 
>>>>>> On 29/02/2020 1:53 pm, Jia Huang wrote:
>>>>>>> Hi David,
>>>>>>> 
>>>>>>> ? 2020/2/28 21:16, David Holmes ??:
>>>>>>>> Hi Jia,
>>>>>>>> 
>>>>>>>> On 28/02/2020 11:08 pm, Jia Huang wrote:
>>>>>>>>> Hi all,
>>>>>>>>> 
>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8240222
>>>>>>>>> Webrev: http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/
>>>>>>>>> 
>>>>>>>>> gtest/jfr/test_networkUtilization.cpp failed when the number of
>>>>>>>>> tests is greater than or equal to 2.
>>>>>>>>> As shown in the following code, the test failed because "traceid
>>>>>>>>> id" is a static local variable.
>>>>>>>>> When the test runs a second time, "id" continues to grow,
>>>>>>>>> resulting in the return value of "i" being "_interfaces.end()".
>>>>>>>> Interesting! I'm not at all familiar with the details of gtest and
>>>>>>>> how it actually runs things. But I would expect that we have many
>>>>>>>> tests that rely on starting from a cleanly initialized VM state,
>>>>>>>> or test state, and so can't simply be repeated (it would be like
>>>>>>>> editing an arbitrary test and putting the body of main into a
>>>>>>>> for-loop!).
>>>>>>> I'm sorry that my last description may be a little incomplete.
>>>>>>> JfrTestNetworkUtilization is a group of tests,it includes four
>>>>>>> individual tests:
>>>>>>> --JfrTestNetworkUtilization.RequestFunctionBasic_test_vm
>>>>>>> --JfrTestNetworkUtilization.RequestFunctionMultiple_test_vm
>>>>>>> --JfrTestNetworkUtilization.InterfaceRemoved_test_vm
>>>>>>>      --JfrTestNetworkUtilization.InterfaceReset_test_vm
>>>>>>> 
>>>>>>> If you only run a individual test once, such as the following
>>>>>>> Testing, it still fails.
>>>>>>> The reason for the failure, as I said last time, is also because of
>>>>>>> static local variables.
>>>>>>> In the "RequestFunctionMultiple_test_vm" test, the sequence number
>>>>>>> of "eth0" is 2.
>>>>>>> However, the initial value of the static local variable "id" is 0.
>>>>>>> 
>>>>>>> Testting:
>>>>>>>       --make run-test
>>>>>>> TEST=gtest:JfrTestNetworkUtilization.RequestFunctionMultiple_test_v
>>>>>>> m
>>>>>>> GTEST=REPEAT=1 CONF=release
>>>>>>>       --make run-test
>>>>>>> TEST=gtest:JfrTestNetworkUtilization.InterfaceRemoved_test_vm
>>>>>>> GTEST=REPEAT=1 CONF=release
>>>>>>>       --make run-test
>>>>>>> TEST=gtest:JfrTestNetworkUtilization.InterfaceReset_test_vm
>>>>>>> GTEST=REPEAT=1 CONF=release
>>>>>>> 
>>>>>>>> So I'm not sure what the right way to fix this is:
>>>>>>>> - don't use REPEAT?
>>>>>>>> - declare the test as not repeatable somehow?
>>>>>>>> - change the test to make it repeatable?
>>>>>>> In the doc/testing.md or doc/testing.html, Gtest keywords inclue
>>>>>>> "REPEAT", so I think this means that all GTEST tests should be able
>>>>>>> to use the "REPEAT parameter.
>>>>>>> 
>>>>>>> 
>>>>>>> Best regards,
>>>>>>> Jia
>>>>>>> 
>>>>>>> 
>>>>>>>> Cheers,
>>>>>>>> David
>>>>>>>> 
>>>>>>>>> -----------------------------------------------------------------
>>>>>>>>> -----------
>>>>>>>>> 
>>>>>>>>> test/hotspot/gtest/jfr/test_networkUtilization.cpp
>>>>>>>>> 
>>>>>>>>> static const MockNetworkInterface& get_interface(traceid id) {
>>>>>>>>> std::list<MockNetworkInterface>::const_iterator i =
>>>>>>>>> _interfaces.begin();
>>>>>>>>>      for (; i != _interfaces.end(); ++i) {
>>>>>>>>>        if (i->id == id) {
>>>>>>>>>          break;
>>>>>>>>>        }
>>>>>>>>>      }
>>>>>>>>>      return *i;
>>>>>>>>> }
>>>>>>>>> -----------------------------------------------------------------
>>>>>>>>> -----------
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> Testing:
>>>>>>>>>        - make run-test TEST=gtest:JfrTestNetworkUtilization
>>>>>>>>> GTEST=REPEAT=2 CONF=server-release
>>>>>>>>>        - make run-test TEST=gtest:JfrTestNetworkUtilization
>>>>>>>>> GTEST=REPEAT=2 CONF=server-fastdebug
>>>>>>>>> 
>>>>>>>>> Thanks a lot
>>>>>>>>> 
>>>>>>>>> Best regards,
>>>>>>>>> Jia
>>>>>>>>> 
> 


From huangjia at loongson.cn  Mon Mar 16 07:38:53 2020
From: huangjia at loongson.cn (Jia Huang)
Date: Mon, 16 Mar 2020 15:38:53 +0800
Subject: RFR: 8240222: [TESTBUG] gtest/jfr/test_networkUtilization.cpp
 failed when the number of tests is greater than or equal to 2
In-Reply-To: <672D6DA7-6FA1-4021-A9F4-FD951DE6D6CA@oracle.com>
References: <64960cc9-5021-d265-ab30-9cf9a91cc6e3@loongson.cn>
 <8f143c78-6bba-5f11-21ad-3096115287e1@oracle.com>
 <4f6c2613-de5a-cc28-f208-1a763270a5d0@loongson.cn>
 <44483657-df1f-f9cc-9e6a-f73b0e602f72@oracle.com>
 <931ecb2e-9898-a371-0aa9-bf463b9fdcf6@loongson.cn>
 <e9d5cca3-1f35-7708-1bdb-bb4b5e673d27@loongson.cn>
 <a5b687be-a9b4-4bc7-bace-b94f723cbdab@default>
 <74960f03-542c-f7f5-9f46-994d355d62d3@loongson.cn>
 <50d5e790-ef5c-4c35-bc14-71577df50c9a@default>
 <d56604a5-5873-718d-eda0-86b95a7234b4@loongson.cn>
 <9e526456-0547-b24a-5199-19d6dd465c88@loongson.cn>
 <672D6DA7-6FA1-4021-A9F4-FD951DE6D6CA@oracle.com>
Message-ID: <b639372a-ca1f-936c-0b5f-7abfc93cce60@loongson.cn>

Hi Erik,

Thanks for your review.

Could you please sponsor it?

Thanks a lot.
Best regards,
Jia.

? 2020/3/16 12:00, Erik Gahlin ??:
> Looks reasonable.
>
> Erik
>
>> On 16 Mar 2020, at 04:54, Jia Huang <huangjia at loongson.cn> wrote:
>>
>> Hi Markus,
>>
>> Is it a trivial patch and OK to be pushed?
>> Or shall we wait for one more review?
>>
>> Best regards,
>> Jia
>>
>> ? 2020/3/10 19:37, Jia Huang ??:
>>> Thanks Markus.
>>>
>>> Best regards,
>>> Jia
>>>
>>> ? 2020/3/10 18:15, Markus Gronlund ??:
>>>> Hi Jia,
>>>>
>>>> Looks like it could work.
>>>>
>>>> Thanks
>>>> Markus
>>>>
>>>> -----Original Message-----
>>>> From: Jia Huang <huangjia at loongson.cn>
>>>> Sent: den 10 mars 2020 02:30
>>>> To: Markus Gronlund <markus.gronlund at oracle.com>
>>>> Cc: hotspot-jfr-dev at openjdk.java.net; hotspot-dev at openjdk.java.net
>>>> Subject: Re: RFR: 8240222: [TESTBUG] gtest/jfr/test_networkUtilization.cpp failed when the number of tests is greater than or equal to 2
>>>>
>>>> Thanks Markus.
>>>>
>>>> Could you give some comments or advice on this one:
>>>> Webrev:  http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/ ?
>>>>
>>>> Best regards,
>>>> Jia
>>>>
>>>> ? 2020/3/9 19:41, Markus Gronlund ??:
>>>>> The interface_id assigns a unique primary key for each interface.
>>>>>
>>>>> To support GTEST=REPEAT, I think the easiest way is to add a corresponding counter in the gtest itself that mirror the production code:
>>>>>
>>>>> That is, as an example, instead of the "add_interface("eth0", 1), that
>>>>> hardcodes the traceid = 1, have it take a value from the corresponding
>>>>> counter, i.e. "add_interface("eth0", next_id++)";
>>>>>
>>>>> Thanks
>>>>> Markus
>>>>>
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: Jia Huang <huangjia at loongson.cn>
>>>>> Sent: den 9 mars 2020 10:57
>>>>> To: hotspot-dev at openjdk.java.net; hotspot-jfr-dev at openjdk.java.net
>>>>> Subject: Re: RFR: 8240222: [TESTBUG]
>>>>> gtest/jfr/test_networkUtilization.cpp failed when the number of tests
>>>>> is greater than or equal to 2
>>>>>
>>>>> In addition, the static local variable "id" is assigned in the function of "new_entry", that is in share/jfr/periodic/jfrNetworkUtilization.cpp.
>>>>>
>>>>> ---------------------------------------------------------------
>>>>> share/jfr/periodic/jfrNetworkUtilization.cpp
>>>>>
>>>>> static InterfaceEntry& new_entry(const NetworkInterface* iface,
>>>>> GrowableArray<InterfaceEntry>* interfaces) {
>>>>>       assert(iface != NULL, "invariant");
>>>>>       assert(interfaces != NULL, "invariant");
>>>>>
>>>>>       // single threaded premise
>>>>>       static traceid interface_id = 0;                       // this is
>>>>> the static local variable
>>>>>
>>>>>       const char* name = iface->get_name();
>>>>>       assert(name != NULL, "invariant");
>>>>>
>>>>>       InterfaceEntry entry;
>>>>>       const size_t length = strlen(name);
>>>>>       entry.name = NEW_C_HEAP_ARRAY(char, length + 1, mtInternal);
>>>>>       strncpy(entry.name, name, length + 1);
>>>>>       entry.id = ++interface_id;
>>>>>       entry.bytes_in = iface->get_bytes_in();
>>>>>       entry.bytes_out = iface->get_bytes_out();
>>>>>       entry.written = false;
>>>>>       return _interfaces->at(_interfaces->append(entry));
>>>>> }
>>>>> ---------------------------------------------------------------
>>>>>
>>>>> ? 2020/3/9 15:29, Jia Huang ??:
>>>>>> Haha, I'm also trying to email hotspot-jfr-dev.
>>>>>> Thank you, David.
>>>>>>
>>>>>> ? 2020/3/9 13:03, David Holmes ??:
>>>>>>> Hi Jia,
>>>>>>>
>>>>>>> I see this is getting no response so have cc'd hotspot-jfr-dev as
>>>>>>> they should have a better idea of how this gtest can and cannot be used.
>>>>>>>
>>>>>>> Just because the GTest framework supports "REPEAT" it doesn't mean
>>>>>>> all tests can, or should, be repeatable.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> David
>>>>>>>
>>>>>>> On 29/02/2020 1:53 pm, Jia Huang wrote:
>>>>>>>> Hi David,
>>>>>>>>
>>>>>>>> ? 2020/2/28 21:16, David Holmes ??:
>>>>>>>>> Hi Jia,
>>>>>>>>>
>>>>>>>>> On 28/02/2020 11:08 pm, Jia Huang wrote:
>>>>>>>>>> Hi all,
>>>>>>>>>>
>>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8240222
>>>>>>>>>> Webrev: http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/
>>>>>>>>>>
>>>>>>>>>> gtest/jfr/test_networkUtilization.cpp failed when the number of
>>>>>>>>>> tests is greater than or equal to 2.
>>>>>>>>>> As shown in the following code, the test failed because "traceid
>>>>>>>>>> id" is a static local variable.
>>>>>>>>>> When the test runs a second time, "id" continues to grow,
>>>>>>>>>> resulting in the return value of "i" being "_interfaces.end()".
>>>>>>>>> Interesting! I'm not at all familiar with the details of gtest and
>>>>>>>>> how it actually runs things. But I would expect that we have many
>>>>>>>>> tests that rely on starting from a cleanly initialized VM state,
>>>>>>>>> or test state, and so can't simply be repeated (it would be like
>>>>>>>>> editing an arbitrary test and putting the body of main into a
>>>>>>>>> for-loop!).
>>>>>>>> I'm sorry that my last description may be a little incomplete.
>>>>>>>> JfrTestNetworkUtilization is a group of tests,it includes four
>>>>>>>> individual tests:
>>>>>>>> --JfrTestNetworkUtilization.RequestFunctionBasic_test_vm
>>>>>>>> --JfrTestNetworkUtilization.RequestFunctionMultiple_test_vm
>>>>>>>> --JfrTestNetworkUtilization.InterfaceRemoved_test_vm
>>>>>>>>       --JfrTestNetworkUtilization.InterfaceReset_test_vm
>>>>>>>>
>>>>>>>> If you only run a individual test once, such as the following
>>>>>>>> Testing, it still fails.
>>>>>>>> The reason for the failure, as I said last time, is also because of
>>>>>>>> static local variables.
>>>>>>>> In the "RequestFunctionMultiple_test_vm" test, the sequence number
>>>>>>>> of "eth0" is 2.
>>>>>>>> However, the initial value of the static local variable "id" is 0.
>>>>>>>>
>>>>>>>> Testting:
>>>>>>>>        --make run-test
>>>>>>>> TEST=gtest:JfrTestNetworkUtilization.RequestFunctionMultiple_test_v
>>>>>>>> m
>>>>>>>> GTEST=REPEAT=1 CONF=release
>>>>>>>>        --make run-test
>>>>>>>> TEST=gtest:JfrTestNetworkUtilization.InterfaceRemoved_test_vm
>>>>>>>> GTEST=REPEAT=1 CONF=release
>>>>>>>>        --make run-test
>>>>>>>> TEST=gtest:JfrTestNetworkUtilization.InterfaceReset_test_vm
>>>>>>>> GTEST=REPEAT=1 CONF=release
>>>>>>>>
>>>>>>>>> So I'm not sure what the right way to fix this is:
>>>>>>>>> - don't use REPEAT?
>>>>>>>>> - declare the test as not repeatable somehow?
>>>>>>>>> - change the test to make it repeatable?
>>>>>>>> In the doc/testing.md or doc/testing.html, Gtest keywords inclue
>>>>>>>> "REPEAT", so I think this means that all GTEST tests should be able
>>>>>>>> to use the "REPEAT parameter.
>>>>>>>>
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>> Jia
>>>>>>>>
>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> David
>>>>>>>>>
>>>>>>>>>> -----------------------------------------------------------------
>>>>>>>>>> -----------
>>>>>>>>>>
>>>>>>>>>> test/hotspot/gtest/jfr/test_networkUtilization.cpp
>>>>>>>>>>
>>>>>>>>>> static const MockNetworkInterface& get_interface(traceid id) {
>>>>>>>>>> std::list<MockNetworkInterface>::const_iterator i =
>>>>>>>>>> _interfaces.begin();
>>>>>>>>>>       for (; i != _interfaces.end(); ++i) {
>>>>>>>>>>         if (i->id == id) {
>>>>>>>>>>           break;
>>>>>>>>>>         }
>>>>>>>>>>       }
>>>>>>>>>>       return *i;
>>>>>>>>>> }
>>>>>>>>>> -----------------------------------------------------------------
>>>>>>>>>> -----------
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Testing:
>>>>>>>>>>         - make run-test TEST=gtest:JfrTestNetworkUtilization
>>>>>>>>>> GTEST=REPEAT=2 CONF=server-release
>>>>>>>>>>         - make run-test TEST=gtest:JfrTestNetworkUtilization
>>>>>>>>>> GTEST=REPEAT=2 CONF=server-fastdebug
>>>>>>>>>>
>>>>>>>>>> Thanks a lot
>>>>>>>>>>
>>>>>>>>>> Best regards,
>>>>>>>>>> Jia
>>>>>>>>>>


From matthias.baesken at sap.com  Mon Mar 16 13:14:01 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Mon, 16 Mar 2020 13:14:01 +0000
Subject: RFR: 8240824: enhance print_full_memory_info on Linux by THP
 related information
In-Reply-To: <AM0PR02MB42274C9523CE1D8ED33765B793FA0@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB42272402734B0AD4B4E304CC93FF0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <CAA-vtUzKF7qHzEiNjn8E_Gg6a5pGdheB0RA0tfzT86ywgJL3NQ@mail.gmail.com>
 <AM0PR02MB42274C9523CE1D8ED33765B793FA0@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <AM0PR02MB4227B05E293297C977ED08D193F90@AM0PR02MB4227.eurprd02.prod.outlook.com>

Ping ? may a have a second review please?



From aph at redhat.com  Mon Mar 16 17:14:26 2020
From: aph at redhat.com (Andrew Haley)
Date: Mon, 16 Mar 2020 17:14:26 +0000
Subject: RFR: 8241018: 32-bit integer log2 functions return the wrong value
 for negative arguments on 64-bit machines
Message-ID: <0c94b8ec-7a86-4354-c195-a79574e371ee@redhat.com>

https://bugs.openjdk.java.net/browse/JDK-8241018
http://cr.openjdk.java.net/~aph/8241018/

OK?

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From vladimir.kozlov at oracle.com  Mon Mar 16 20:07:15 2020
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Mon, 16 Mar 2020 13:07:15 -0700
Subject: RFR: 8241018: 32-bit integer log2 functions return the wrong
 value for negative arguments on 64-bit machines
In-Reply-To: <0c94b8ec-7a86-4354-c195-a79574e371ee@redhat.com>
References: <0c94b8ec-7a86-4354-c195-a79574e371ee@redhat.com>
Message-ID: <47fa4e8d-4085-e7c8-7588-3701157f9341@oracle.com>

LGTM

Thanks,
Vladimir

On 3/16/20 10:14 AM, Andrew Haley wrote:
> https://bugs.openjdk.java.net/browse/JDK-8241018
> http://cr.openjdk.java.net/~aph/8241018/
> 
> OK?
> 

From claes.redestad at oracle.com  Mon Mar 16 20:33:20 2020
From: claes.redestad at oracle.com (Claes Redestad)
Date: Mon, 16 Mar 2020 21:33:20 +0100
Subject: RFR: 8241018: 32-bit integer log2 functions return the wrong
 value for negative arguments on 64-bit machines
In-Reply-To: <0c94b8ec-7a86-4354-c195-a79574e371ee@redhat.com>
References: <0c94b8ec-7a86-4354-c195-a79574e371ee@redhat.com>
Message-ID: <e0dc3ce3-3275-e402-b831-bb0314839a3e@oracle.com>



On 2020-03-16 18:14, Andrew Haley wrote:
> https://bugs.openjdk.java.net/browse/JDK-8241018
> http://cr.openjdk.java.net/~aph/8241018/
> 
> OK?
> 

OK!

/Claes

From david.holmes at oracle.com  Tue Mar 17 02:05:57 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 17 Mar 2020 12:05:57 +1000
Subject: RFR: 8240824: enhance print_full_memory_info on Linux by THP
 related information
In-Reply-To: <AM0PR02MB42274C9523CE1D8ED33765B793FA0@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB42272402734B0AD4B4E304CC93FF0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <CAA-vtUzKF7qHzEiNjn8E_Gg6a5pGdheB0RA0tfzT86ywgJL3NQ@mail.gmail.com>
 <AM0PR02MB42274C9523CE1D8ED33765B793FA0@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <f88fc1ba-b2bd-9317-39db-d1755a01b74e@oracle.com>

Seems okay.

Thanks,
David

On 13/03/2020 6:31 pm, Baesken, Matthias wrote:
> Thanks. May I have a second review ?
> 
> Best Regards, Matthias
> 
> 
> From: Thomas St?fe <thomas.stuefe at gmail.com>
> Sent: Donnerstag, 12. M?rz 2020 14:33
> To: Baesken, Matthias <matthias.baesken at sap.com>
> Cc: hotspot-dev at openjdk.java.net
> Subject: Re: RFR: 8240824: enhance print_full_memory_info on Linux by THP related information
> 
> Hi Matthias,
> 
> too verbose for my taste:
> 
> 
> +  st->print_cr("/sys/kernel/mm/transparent_hugepage/enabled (transparent huge pages mode):");
> 
> I would cut that down to either "/sys/kernel/mm/transparent_hugepage/enabled" or to "transparent huge pages mode". Probably the former.
> 
> Otherwise looks good. I do not need a new webrev.
> 
> Cheers, Thomas
> 
> 
> 
> On Tue, Mar 10, 2020 at 4:09 PM Baesken, Matthias <matthias.baesken at sap.com<mailto:matthias.baesken at sap.com>> wrote:
> Hello, please review this small enhancement .
> 
> On Linux, the change adds output of some files related to THP (transparent huge pages)   configuration to the hs_err file; the settings in those files configure how transparent huge pages are used on the system.
> 
> Example output in the hs_err file from a test machine :
> 
> 
> /sys/kernel/mm/transparent_hugepage/enabled (transparent huge pages mode):
> [always] madvise never
> 
> /sys/kernel/mm/transparent_hugepage/defrag (defrag/compaction efforts parameter):
> [always] madvise never
> 
> 
> 
> 
> Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8240824
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8240824.0/
> 
> 
> Thanks, Matthias
> 

From matthias.baesken at sap.com  Tue Mar 17 07:28:44 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 17 Mar 2020 07:28:44 +0000
Subject: RFR [XS]: 8241064: JFR related tests
 TestMetaspaceAllocationFailure.java and TestEventInstrumentation.java miss
 requires tag
Message-ID: <AM0PR02MB4227CCF36A4A057D44904A4893F60@AM0PR02MB4227.eurprd02.prod.outlook.com>

Hello, please review this very small fix .
The JFR related tests
jdk/jfr/event/runtime/TestMetaspaceAllocationFailure.java
and
jdk/jfr/javaagent/TestEventInstrumentation.java

miss the vm.hasJFR tag, this leads to failures on platforms without JFR.

Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8241064

http://cr.openjdk.java.net/~mbaesken/webrevs/8241064.0/

Thanks, Matthias

From markus.gronlund at oracle.com  Tue Mar 17 10:58:40 2020
From: markus.gronlund at oracle.com (Markus Gronlund)
Date: Tue, 17 Mar 2020 03:58:40 -0700 (PDT)
Subject: RFR [XS]: 8241064: JFR related tests
 TestMetaspaceAllocationFailure.java and TestEventInstrumentation.java miss
 requires tag
In-Reply-To: <AM0PR02MB4227CCF36A4A057D44904A4893F60@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB4227CCF36A4A057D44904A4893F60@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <758e2818-205b-4938-8a9d-0894c6f0d612@default>

Hi Matthias,

Looks good and trivial.

Thank you for fixing this.
Markus

-----Original Message-----
From: Baesken, Matthias <matthias.baesken at sap.com> 
Sent: den 17 mars 2020 08:29
To: hotspot-dev at openjdk.java.net; hotspot-jfr-dev at openjdk.java.net
Subject: RFR [XS]: 8241064: JFR related tests TestMetaspaceAllocationFailure.java and TestEventInstrumentation.java miss requires tag

Hello, please review this very small fix .
The JFR related tests
jdk/jfr/event/runtime/TestMetaspaceAllocationFailure.java
and
jdk/jfr/javaagent/TestEventInstrumentation.java

miss the vm.hasJFR tag, this leads to failures on platforms without JFR.

Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8241064

http://cr.openjdk.java.net/~mbaesken/webrevs/8241064.0/

Thanks, Matthias

From matthias.baesken at sap.com  Tue Mar 17 11:18:33 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 17 Mar 2020 11:18:33 +0000
Subject: RFR [XS]: 8241064: JFR related tests
 TestMetaspaceAllocationFailure.java and TestEventInstrumentation.java miss
 requires tag
In-Reply-To: <758e2818-205b-4938-8a9d-0894c6f0d612@default>
References: <AM0PR02MB4227CCF36A4A057D44904A4893F60@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <758e2818-205b-4938-8a9d-0894c6f0d612@default>
Message-ID: <AM0PR02MB4227D4AD741FBF185984DF0A93F60@AM0PR02MB4227.eurprd02.prod.outlook.com>

Thanks for the review!

In case of no objections, I'll push it as trivial .

Best regards, Matthias

-----Original Message-----
From: Markus Gronlund <markus.gronlund at oracle.com> 
Sent: Dienstag, 17. M?rz 2020 11:59
To: Baesken, Matthias <matthias.baesken at sap.com>; hotspot-dev at openjdk.java.net; hotspot-jfr-dev at openjdk.java.net
Subject: RE: RFR [XS]: 8241064: JFR related tests TestMetaspaceAllocationFailure.java and TestEventInstrumentation.java miss requires tag

Hi Matthias,

Looks good and trivial.

Thank you for fixing this.
Markus

-----Original Message-----
From: Baesken, Matthias <matthias.baesken at sap.com> 
Sent: den 17 mars 2020 08:29
To: hotspot-dev at openjdk.java.net; hotspot-jfr-dev at openjdk.java.net
Subject: RFR [XS]: 8241064: JFR related tests TestMetaspaceAllocationFailure.java and TestEventInstrumentation.java miss requires tag

Hello, please review this very small fix .
The JFR related tests
jdk/jfr/event/runtime/TestMetaspaceAllocationFailure.java
and
jdk/jfr/javaagent/TestEventInstrumentation.java

miss the vm.hasJFR tag, this leads to failures on platforms without JFR.

Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8241064

http://cr.openjdk.java.net/~mbaesken/webrevs/8241064.0/

Thanks, Matthias

From mikhailo.seledtsov at oracle.com  Tue Mar 17 15:53:18 2020
From: mikhailo.seledtsov at oracle.com (mikhailo.seledtsov at oracle.com)
Date: Tue, 17 Mar 2020 08:53:18 -0700
Subject: RFR [XS]: 8241064: JFR related tests
 TestMetaspaceAllocationFailure.java and TestEventInstrumentation.java miss
 requires tag
In-Reply-To: <AM0PR02MB4227CCF36A4A057D44904A4893F60@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB4227CCF36A4A057D44904A4893F60@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <3bf235b8-72ab-3b29-f0bb-fbcabec504de@oracle.com>

Looks good to me,

Sorry I missed them, thanks for fixing,


Misha

On 3/17/20 12:28 AM, Baesken, Matthias wrote:
> Hello, please review this very small fix .
> The JFR related tests
> jdk/jfr/event/runtime/TestMetaspaceAllocationFailure.java
> and
> jdk/jfr/javaagent/TestEventInstrumentation.java
>
> miss the vm.hasJFR tag, this leads to failures on platforms without JFR.
>
> Bug/webrev :
>
> https://bugs.openjdk.java.net/browse/JDK-8241064
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8241064.0/
>
> Thanks, Matthias

From robbin.ehn at oracle.com  Thu Mar 19 12:44:29 2020
From: robbin.ehn at oracle.com (Robbin Ehn)
Date: Thu, 19 Mar 2020 13:44:29 +0100
Subject: RFR(L): Remove global safepoint code
Message-ID: <308290ab-0cf7-7a93-706f-94101597322f@oracle.com>

Hi all, please build, test and review.

We removed (obsoleted) the flag ThreadLocalHandshakes in JDK 14.
Any platform defining THREAD_LOCAL_POLL can only use per thread poll.
Since all platforms now define THREAD_LOCAL_POLL the global poll code is 
dead.
Let's remove it!

Issue: https://bugs.openjdk.java.net/browse/JDK-8220051
Changeset: http://cr.openjdk.java.net/~rehn/8220051/v1/webrev/index.html

I have built fastdebug:
linux-aarch64
linux-arm32
linux-ppc64le
linux-s390x
linux-x86_64-zero
linux-x86_64 (no PCH)
linux-x86_64 (slowdebug) (no PCH)
linux-x86
osx-x86_64
windows-x86_64
solaris-sparcv9

Tested t1-5:
linux-x86_64
osx-x86_64
windows-x86_64
solaris-sparcv9

Manually:
linux-aarch64
linux-x86_64-zero

Any extra help building and testing is appreciated, especially on arm32, 
ppc64le, s390x.

Thanks, Robbin

From robbin.ehn at oracle.com  Thu Mar 19 12:45:52 2020
From: robbin.ehn at oracle.com (Robbin Ehn)
Date: Thu, 19 Mar 2020 13:45:52 +0100
Subject: RFR(L): 8220051: Remove global safepoint code
In-Reply-To: <308290ab-0cf7-7a93-706f-94101597322f@oracle.com>
References: <308290ab-0cf7-7a93-706f-94101597322f@oracle.com>
Message-ID: <f467eac7-7fa4-f149-0c0f-cc1675351ae4@oracle.com>

Corrected mail-header.

/Robbin

On 2020-03-19 13:44, Robbin Ehn wrote:
> Hi all, please build, test and review.
> 
> We removed (obsoleted) the flag ThreadLocalHandshakes in JDK 14.
> Any platform defining THREAD_LOCAL_POLL can only use per thread poll.
> Since all platforms now define THREAD_LOCAL_POLL the global poll code is 
> dead.
> Let's remove it!
> 
> Issue: https://bugs.openjdk.java.net/browse/JDK-8220051
> Changeset: http://cr.openjdk.java.net/~rehn/8220051/v1/webrev/index.html
> 
> I have built fastdebug:
> linux-aarch64
> linux-arm32
> linux-ppc64le
> linux-s390x
> linux-x86_64-zero
> linux-x86_64 (no PCH)
> linux-x86_64 (slowdebug) (no PCH)
> linux-x86
> osx-x86_64
> windows-x86_64
> solaris-sparcv9
> 
> Tested t1-5:
> linux-x86_64
> osx-x86_64
> windows-x86_64
> solaris-sparcv9
> 
> Manually:
> linux-aarch64
> linux-x86_64-zero
> 
> Any extra help building and testing is appreciated, especially on arm32, 
> ppc64le, s390x.
> 
> Thanks, Robbin

From martin.doerr at sap.com  Thu Mar 19 13:41:45 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Thu, 19 Mar 2020 13:41:45 +0000
Subject: RFR(L): Remove global safepoint code
In-Reply-To: <308290ab-0cf7-7a93-706f-94101597322f@oracle.com>
References: <308290ab-0cf7-7a93-706f-94101597322f@oracle.com>
Message-ID: <AM0PR0202MB3297E5CCE50ED7A2E4E05F399AF40@AM0PR0202MB3297.eurprd02.prod.outlook.com>

Hi Robbin,

thanks for removing the dead code and for taking care of all platforms.

PPC64 and s390 code looks good and seems to work fine on linux.

Only AIX doesn't build. I suggest to fix it like this:

diff -r efebab2fd109 src/hotspot/os/aix/safepointMechanism_aix.cpp
--- a/src/hotspot/os/aix/safepointMechanism_aix.cpp     Thu Mar 19 13:30:39 2020 +0100
+++ b/src/hotspot/os/aix/safepointMechanism_aix.cpp     Thu Mar 19 14:33:09 2020 +0100
@@ -101,7 +101,9 @@
   MemTracker::record_virtual_memory_reserve_and_commit(map_address, map_size, CALLER_PC, mtSafepoint);

   // Use same page for thread local handshakes without SIGTRAP
-  os::protect_memory(_polling_page, page_size, os::MEM_PROT_NONE);
+  if (!os::guard_memory((char*)_polling_page, page_size)) {
+    fatal("Could not protect polling page");
+  }
   intptr_t bad_page_val  = reinterpret_cast<intptr_t>(map_address),
            good_page_val = bad_page_val + os::vm_page_size();
   _poll_armed_value    = reinterpret_cast<void*>(bad_page_val  + poll_bit());


Thanks and best regards,
Martin



> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
> Robbin Ehn
> Sent: Donnerstag, 19. M?rz 2020 13:44
> To: hotspot-dev at openjdk.java.net
> Subject: RFR(L): Remove global safepoint code
> 
> Hi all, please build, test and review.
> 
> We removed (obsoleted) the flag ThreadLocalHandshakes in JDK 14.
> Any platform defining THREAD_LOCAL_POLL can only use per thread poll.
> Since all platforms now define THREAD_LOCAL_POLL the global poll code is
> dead.
> Let's remove it!
> 
> Issue: https://bugs.openjdk.java.net/browse/JDK-8220051
> Changeset:
> http://cr.openjdk.java.net/~rehn/8220051/v1/webrev/index.html
> 
> I have built fastdebug:
> linux-aarch64
> linux-arm32
> linux-ppc64le
> linux-s390x
> linux-x86_64-zero
> linux-x86_64 (no PCH)
> linux-x86_64 (slowdebug) (no PCH)
> linux-x86
> osx-x86_64
> windows-x86_64
> solaris-sparcv9
> 
> Tested t1-5:
> linux-x86_64
> osx-x86_64
> windows-x86_64
> solaris-sparcv9
> 
> Manually:
> linux-aarch64
> linux-x86_64-zero
> 
> Any extra help building and testing is appreciated, especially on arm32,
> ppc64le, s390x.
> 
> Thanks, Robbin

From robbin.ehn at oracle.com  Thu Mar 19 14:11:26 2020
From: robbin.ehn at oracle.com (Robbin Ehn)
Date: Thu, 19 Mar 2020 15:11:26 +0100
Subject: RFR(L): Remove global safepoint code
In-Reply-To: <AM0PR0202MB3297E5CCE50ED7A2E4E05F399AF40@AM0PR0202MB3297.eurprd02.prod.outlook.com>
References: <308290ab-0cf7-7a93-706f-94101597322f@oracle.com>
 <AM0PR0202MB3297E5CCE50ED7A2E4E05F399AF40@AM0PR0202MB3297.eurprd02.prod.outlook.com>
Message-ID: <f2906411-a765-b5b7-d97e-8dff351fe39c@oracle.com>

Hi Martin,

On 2020-03-19 14:41, Doerr, Martin wrote:
> Hi Robbin,
> 
> thanks for removing the dead code and for taking care of all platforms.
> 
> PPC64 and s390 code looks good and seems to work fine on linux.

Great, thanks!

> 
> Only AIX doesn't build. I suggest to fix it like this:

Thank you, applied patch!

(waiting with incremental version until I get more feedback)

/Robbin

> 
> diff -r efebab2fd109 src/hotspot/os/aix/safepointMechanism_aix.cpp
> --- a/src/hotspot/os/aix/safepointMechanism_aix.cpp     Thu Mar 19 13:30:39 2020 +0100
> +++ b/src/hotspot/os/aix/safepointMechanism_aix.cpp     Thu Mar 19 14:33:09 2020 +0100
> @@ -101,7 +101,9 @@
>     MemTracker::record_virtual_memory_reserve_and_commit(map_address, map_size, CALLER_PC, mtSafepoint);
> 
>     // Use same page for thread local handshakes without SIGTRAP
> -  os::protect_memory(_polling_page, page_size, os::MEM_PROT_NONE);
> +  if (!os::guard_memory((char*)_polling_page, page_size)) {
> +    fatal("Could not protect polling page");
> +  }
>     intptr_t bad_page_val  = reinterpret_cast<intptr_t>(map_address),
>              good_page_val = bad_page_val + os::vm_page_size();
>     _poll_armed_value    = reinterpret_cast<void*>(bad_page_val  + poll_bit());
> 
> 
> Thanks and best regards,
> Martin
> 
> 
> 
>> -----Original Message-----
>> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
>> Robbin Ehn
>> Sent: Donnerstag, 19. M?rz 2020 13:44
>> To: hotspot-dev at openjdk.java.net
>> Subject: RFR(L): Remove global safepoint code
>>
>> Hi all, please build, test and review.
>>
>> We removed (obsoleted) the flag ThreadLocalHandshakes in JDK 14.
>> Any platform defining THREAD_LOCAL_POLL can only use per thread poll.
>> Since all platforms now define THREAD_LOCAL_POLL the global poll code is
>> dead.
>> Let's remove it!
>>
>> Issue: https://bugs.openjdk.java.net/browse/JDK-8220051
>> Changeset:
>> http://cr.openjdk.java.net/~rehn/8220051/v1/webrev/index.html
>>
>> I have built fastdebug:
>> linux-aarch64
>> linux-arm32
>> linux-ppc64le
>> linux-s390x
>> linux-x86_64-zero
>> linux-x86_64 (no PCH)
>> linux-x86_64 (slowdebug) (no PCH)
>> linux-x86
>> osx-x86_64
>> windows-x86_64
>> solaris-sparcv9
>>
>> Tested t1-5:
>> linux-x86_64
>> osx-x86_64
>> windows-x86_64
>> solaris-sparcv9
>>
>> Manually:
>> linux-aarch64
>> linux-x86_64-zero
>>
>> Any extra help building and testing is appreciated, especially on arm32,
>> ppc64le, s390x.
>>
>> Thanks, Robbin

From magnus.ihse.bursie at oracle.com  Thu Mar 19 17:20:41 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Thu, 19 Mar 2020 18:20:41 +0100
Subject: RFR: JDK-8241271 Make hotspot build reproducible
Message-ID: <b5aaadfc-99d5-9d65-313e-980efe071bf1@oracle.com>

Currently, it is not possible two make hotspot builds reproducible, 
since the current date and time is always inserted using the special 
macros __DATE__ and __TIME__. This patch makes it possible to override 
these values from the build system, paving the way for making re-builds 
of hotspot bit-by-bit identical.

On linux/gcc, we can now actually get bit-by-bit identical builds of 
libjvm.so (if not using precompiled headers), so I've updated the 
baseline-cmp profiles and compare script to reflect this.

Bug: https://bugs.openjdk.java.net/browse/JDK-8241271
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8241271-make-hotspot-reproducible/webrev.01

/Magnus

From erik.joelsson at oracle.com  Thu Mar 19 18:01:44 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Thu, 19 Mar 2020 11:01:44 -0700
Subject: RFR: JDK-8241271 Make hotspot build reproducible
In-Reply-To: <b5aaadfc-99d5-9d65-313e-980efe071bf1@oracle.com>
References: <b5aaadfc-99d5-9d65-313e-980efe071bf1@oracle.com>
Message-ID: <3e4ebab4-c31a-99b2-8062-6378443874cf@oracle.com>

On 2020-03-19 10:20, Magnus Ihse Bursie wrote:
> Currently, it is not possible two make hotspot builds reproducible, 
> since the current date and time is always inserted using the special 
> macros __DATE__ and __TIME__. This patch makes it possible to override 
> these values from the build system, paving the way for making 
> re-builds of hotspot bit-by-bit identical.
>
I like the ability to achieve this.
> On linux/gcc, we can now actually get bit-by-bit identical builds of 
> libjvm.so (if not using precompiled headers), so I've updated the 
> baseline-cmp profiles and compare script to reflect this.
>
I'm not sure this is a good idea though. It means I can't reasonably 
compare to validate build changes that affect the precompiled header. 
Similarly, the COMPARE_BUILD make variable argument will not work unless 
you use both of these configure arguments.

Since compare.sh is generated through configure, perhaps these 
exceptions can be made conditional on if these configure arguments are 
setup correctly for a reproducible libjvm.so?

/Erik

> Bug: https://bugs.openjdk.java.net/browse/JDK-8241271
> WebRev: 
> http://cr.openjdk.java.net/~ihse/JDK-8241271-make-hotspot-reproducible/webrev.01
>
> /Magnus

From paul.sandoz at oracle.com  Thu Mar 19 19:46:46 2020
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Thu, 19 Mar 2020 12:46:46 -0700
Subject: RFR JEP 338: Vector API (Incubator), JDK-8223347 Integration of
 Vector API (Incubator): Vector API 
Message-ID: <C39B16F3-476F-4489-8E67-562B9A576519@oracle.com>

Hi,

This email is intended to kick start the review of JEP 338: Vector API (Incubator).  

For ease of review we intend to split this up into separate parts for review of the Java API, Java implementation, Java unit tests, Java jmh tests, shared hotspot code, and platform specific hotspot code.  Emails will be sent for review hopefully all during this month (given global events though our productivity may be impacted as I might also anticipate for those reviewing).

First up for review is the Vector API itself.  JavaDoc and specdiff may be found here:

  http://cr.openjdk.java.net/~psandoz/panama/vector-api-review/ <http://cr.openjdk.java.net/~psandoz/panama/vector-api-review/>
  
  http://cr.openjdk.java.net/~psandoz/panama/vector-api-review/docs/api/jdk.incubator.vector/jdk/incubator/vector/package-summary.html <http://cr.openjdk.java.net/~psandoz/panama/vector-api-review/docs/api/jdk.incubator.vector/jdk/incubator/vector/package-summary.html>
  http://cr.openjdk.java.net/~psandoz/panama/vector-api-review/docs/api/jdk.incubator.vector/jdk/incubator/vector/Vector.html <http://cr.openjdk.java.net/~psandoz/panama/vector-api-review/docs/api/jdk.incubator.vector/jdk/incubator/vector/Vector.html>

The CSR issue (https://bugs.openjdk.java.net/browse/JDK-8223348 <https://bugs.openjdk.java.net/browse/JDK-8223348>) has also been moved to proposed.


For a holistic sense during this split review one can go directly to the Panama dev repository and checkout the (confusingly named) vector-unstable branch (it was just conveniently there separate from the vectorIntrinsics branch where ongoing work will happen).
  
  https://hg.openjdk.java.net/panama/dev/shortlog/vector-unstable <https://hg.openjdk.java.net/panama/dev/shortlog/vector-unstable>

Paul.

From daniel.daugherty at oracle.com  Thu Mar 19 20:39:46 2020
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Thu, 19 Mar 2020 16:39:46 -0400
Subject: RFR(L): 8220051: Remove global safepoint code
In-Reply-To: <f467eac7-7fa4-f149-0c0f-cc1675351ae4@oracle.com>
References: <308290ab-0cf7-7a93-706f-94101597322f@oracle.com>
 <f467eac7-7fa4-f149-0c0f-cc1675351ae4@oracle.com>
Message-ID: <9dd6fde0-f10e-ab3d-8db7-dcf919fd4b08@oracle.com>

Hi Robbin!

On 3/19/20 8:45 AM, Robbin Ehn wrote:
> On 2020-03-19 13:44, Robbin Ehn wrote:
>> Hi all, please build, test and review.
>>
>> We removed (obsoleted) the flag ThreadLocalHandshakes in JDK 14.
>> Any platform defining THREAD_LOCAL_POLL can only use per thread poll.
>> Since all platforms now define THREAD_LOCAL_POLL the global poll code 
>> is dead.
>> Let's remove it!
>>
>> Issue: https://bugs.openjdk.java.net/browse/JDK-8220051
>> Changeset: http://cr.openjdk.java.net/~rehn/8220051/v1/webrev/index.html

Lots of files in this one. I'm gonna try to review these files
in groups that make sense my warped mind... :-)


Thumbs up on the code review.

I did my usual crawl thru review, but this is a lot of files so
that boring detail is below...

I'm not entirely sure of the roll played by 
SafepointMechanism::_polling_page,
but that's probably because that didn't change except for the location of
the _polling_page field itself.

With this kind of a change, I think you also want to get some Tier[6-8]
test cycles in since that's where the stress tests really go to town...

Dan


src/hotspot/share/runtime/safepointMechanism.hpp
 ??? No comments.

src/hotspot/share/runtime/safepointMechanism.inline.hpp
 ??? No comments.

src/hotspot/share/runtime/safepointMechanism.cpp
 ??? old L73-81: Of course, now I'm wondering where the polling_page is 
setup...
 ??? Since it appears that we still have a global polling page for 
non-JavaThreads,
 ??? but I gotta read on...

 ??? I went to the bug to see if it would help my confusion, but the
 ??? bug has no description. That should be fixed.

 ??? Update: The _polling_page is setup here:

 ??? L54: ??? char* bad_page? = polling_page;
 ??? L57: ??? os::protect_memory(bad_page,? page_size, os::MEM_PROT_NONE);
 ??? L61: ??? _polling_page = (address)(bad_page);

 ??? And for some reason, os/aix/safepointMechanism_aix.cpp has its own
 ??? copy of this logic...

src/hotspot/share/runtime/safepoint.cpp
 ??? No comments.

src/hotspot/share/runtime/os.hpp
 ??? Okay... no more OS interface to the polling page... gotta read on...

src/hotspot/share/runtime/biasedLocking.cpp
 ??? No comments.

src/hotspot/share/runtime/handshake.cpp
 ??? No comments.

src/hotspot/share/runtime/sweeper.cpp
 ??? No comments.

src/hotspot/share/runtime/thread.cpp
 ??? No comments.

src/hotspot/share/utilities/vmError.cpp
 ??? No comments.


src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp
 ??? No comments.

src/hotspot/share/gc/z/zMark.cpp
 ??? No comments.

src/hotspot/share/opto/callnode.cpp
 ??? No comments.

src/hotspot/share/opto/machnode.hpp
 ??? No comments.

src/hotspot/share/opto/machnode.cpp
 ??? No comments.

src/hotspot/share/opto/parse1.cpp
 ??? No comments.

src/hotspot/share/aot/aotCodeHeap.cpp
 ??? No comments.

src/jdk.aot/share/classes/jdk.tools.jaotc.binformat/src/jdk/tools/jaotc/binformat/BinaryContainer.java
 ??? No comments.

src/jdk.aot/share/classes/jdk.tools.jaotc/src/jdk/tools/jaotc/MarkProcessor.java
 ??? No comments.


src/hotspot/os/bsd/os_bsd.cpp
 ??? No comments.

src/hotspot/os/linux/os_linux.cpp
 ??? No comments.

src/hotspot/os/solaris/os_solaris.cpp
 ??? No comments.

src/hotspot/os/windows/os_windows.cpp
 ??? No comments.


src/hotspot/os_cpu/bsd_x86/os_bsd_x86.cpp
 ??? No comments.

src/hotspot/os_cpu/linux_sparc/os_linux_sparc.cpp
 ??? No comments.

src/hotspot/os_cpu/linux_x86/os_linux_x86.cpp
 ??? No comments.

src/hotspot/os_cpu/solaris_sparc/os_solaris_sparc.cpp
 ??? No comments.

src/hotspot/os_cpu/solaris_x86/os_solaris_x86.cpp
 ??? No comments.


src/hotspot/cpu/sparc/c1_LIRAssembler_sparc.cpp
 ??? No comments.

src/hotspot/cpu/sparc/globalDefinitions_sparc.hpp
 ??? No comments.

src/hotspot/cpu/sparc/interp_masm_sparc.cpp
 ??? No comments.

src/hotspot/cpu/sparc/macroAssembler_sparc.cpp
 ??? No comments.

src/hotspot/cpu/sparc/sharedRuntime_sparc.cpp
 ??? No comments.

src/hotspot/cpu/sparc/sparc.ad
 ??? No comments.

src/hotspot/cpu/sparc/templateTable_sparc.cpp
 ??? No comments.


src/hotspot/cpu/x86/assembler_x86.cpp
 ??? No comments.

src/hotspot/cpu/x86/assembler_x86.hpp
 ??? No comments.

src/hotspot/cpu/x86/c1_LIRAssembler_x86.cpp
 ??? No comments.

src/hotspot/cpu/x86/c1_LIRGenerator_x86.cpp
 ??? No comments.

src/hotspot/cpu/x86/globalDefinitions_x86.hpp
 ??? No comments.

src/hotspot/cpu/x86/interp_masm_x86.cpp
 ??? No comments.

src/hotspot/cpu/x86/jvmciCodeInstaller_x86.cpp
 ??? No comments.

src/hotspot/cpu/x86/macroAssembler_x86.cpp
 ??? No comments.

src/hotspot/cpu/x86/nativeInst_x86.hpp
 ??? No comments.

src/hotspot/cpu/x86/relocInfo_x86.cpp
 ??? No comments.

src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp
 ??? No comments.

src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp
 ??? No comments.

src/hotspot/cpu/x86/templateTable_x86.cpp
 ??? No comments.

src/hotspot/cpu/x86/x86_32.ad
 ??? No comments.

src/hotspot/cpu/x86/x86_64.ad
 ??? No comments.


src/hotspot/os/aix/os_aix.cpp
 ??? No comments.

src/hotspot/os/aix/safepointMechanism_aix.cpp
 ??? No comments.

src/hotspot/os_cpu/aix_ppc/os_aix_ppc.cpp
 ??? No comments.

src/hotspot/os_cpu/linux_aarch64/os_linux_aarch64.cpp
 ??? No comments.

src/hotspot/os_cpu/linux_arm/os_linux_arm.cpp
 ??? No comments.

src/hotspot/os_cpu/linux_ppc/os_linux_ppc.cpp
 ??? No comments.

src/hotspot/os_cpu/linux_s390/os_linux_s390.cpp
 ??? No comments.


src/hotspot/cpu/aarch64/aarch64.ad
 ??? No comments.

src/hotspot/cpu/aarch64/c1_LIRAssembler_aarch64.cpp
 ??? No comments.

src/hotspot/cpu/aarch64/globalDefinitions_aarch64.hpp
 ??? No comments.

src/hotspot/cpu/aarch64/interp_masm_aarch64.cpp
 ??? No comments.

src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp
 ??? No comments.

src/hotspot/cpu/aarch64/macroAssembler_aarch64.hpp
 ??? No comments.

src/hotspot/cpu/aarch64/sharedRuntime_aarch64.cpp
 ??? No comments.

src/hotspot/cpu/arm/arm.ad
 ??? No comments.

src/hotspot/cpu/arm/globalDefinitions_arm.hpp
 ??? No comments.

src/hotspot/cpu/arm/interp_masm_arm.cpp
 ??? No comments.

src/hotspot/cpu/arm/macroAssembler_arm.cpp
 ??? No comments.

src/hotspot/cpu/arm/sharedRuntime_arm.cpp
 ??? No comments.


src/hotspot/cpu/ppc/c1_LIRAssembler_ppc.cpp
 ??? No comments.

src/hotspot/cpu/ppc/globalDefinitions_ppc.hpp
 ??? No comments.

src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp
 ??? No comments.

src/hotspot/cpu/ppc/macroAssembler_ppc.cpp
 ??? No comments.

src/hotspot/cpu/ppc/macroAssembler_ppc.inline.hpp
 ??? No comments.

src/hotspot/cpu/ppc/nativeInst_ppc.hpp
 ??? No comments.

src/hotspot/cpu/ppc/ppc.ad
 ??? No comments.

src/hotspot/cpu/ppc/sharedRuntime_ppc.cpp
 ??? No comments.

src/hotspot/cpu/ppc/templateTable_ppc_64.cpp
 ??? No comments.


src/hotspot/cpu/s390/c1_LIRAssembler_s390.cpp
 ??? No comments.

src/hotspot/cpu/s390/globalDefinitions_s390.hpp
 ??? No comments.

src/hotspot/cpu/s390/interp_masm_s390.cpp
 ??? No comments.

src/hotspot/cpu/s390/macroAssembler_s390.cpp
 ??? No comments.

src/hotspot/cpu/s390/s390.ad
 ??? No comments.

src/hotspot/cpu/s390/sharedRuntime_s390.cpp
 ??? No comments.

src/hotspot/cpu/s390/templateTable_s390.cpp
 ??? No comments.


src/hotspot/cpu/zero/globalDefinitions_zero.hpp
 ??? No comments.


>>
>> I have built fastdebug:
>> linux-aarch64
>> linux-arm32
>> linux-ppc64le
>> linux-s390x
>> linux-x86_64-zero
>> linux-x86_64 (no PCH)
>> linux-x86_64 (slowdebug) (no PCH)
>> linux-x86
>> osx-x86_64
>> windows-x86_64
>> solaris-sparcv9
>>
>> Tested t1-5:
>> linux-x86_64
>> osx-x86_64
>> windows-x86_64
>> solaris-sparcv9
>>
>> Manually:
>> linux-aarch64
>> linux-x86_64-zero
>>
>> Any extra help building and testing is appreciated, especially on 
>> arm32, ppc64le, s390x.
>>
>> Thanks, Robbin


From huangjia at loongson.cn  Fri Mar 20 03:58:45 2020
From: huangjia at loongson.cn (Jia Huang)
Date: Fri, 20 Mar 2020 11:58:45 +0800
Subject: RFR: 8240222: [TESTBUG] gtest/jfr/test_networkUtilization.cpp
 failed when the number of tests is greater than or equal to 2
In-Reply-To: <b639372a-ca1f-936c-0b5f-7abfc93cce60@loongson.cn>
References: <64960cc9-5021-d265-ab30-9cf9a91cc6e3@loongson.cn>
 <8f143c78-6bba-5f11-21ad-3096115287e1@oracle.com>
 <4f6c2613-de5a-cc28-f208-1a763270a5d0@loongson.cn>
 <44483657-df1f-f9cc-9e6a-f73b0e602f72@oracle.com>
 <931ecb2e-9898-a371-0aa9-bf463b9fdcf6@loongson.cn>
 <e9d5cca3-1f35-7708-1bdb-bb4b5e673d27@loongson.cn>
 <a5b687be-a9b4-4bc7-bace-b94f723cbdab@default>
 <74960f03-542c-f7f5-9f46-994d355d62d3@loongson.cn>
 <50d5e790-ef5c-4c35-bc14-71577df50c9a@default>
 <d56604a5-5873-718d-eda0-86b95a7234b4@loongson.cn>
 <9e526456-0547-b24a-5199-19d6dd465c88@loongson.cn>
 <672D6DA7-6FA1-4021-A9F4-FD951DE6D6CA@oracle.com>
 <b639372a-ca1f-936c-0b5f-7abfc93cce60@loongson.cn>
Message-ID: <1b6206f1-2e2f-d842-42ad-dc4c5196a298@loongson.cn>

Would someone please sponsor this patch?

Webrev:  http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/ ?

Best regards,
Jia.

>
> ? 2020/3/16 12:00, Erik Gahlin ??:
>> Looks reasonable.
>>
>> Erik
>>
>>> On 16 Mar 2020, at 04:54, Jia Huang <huangjia at loongson.cn> wrote:
>>>
>>> Hi Markus,
>>>
>>> Is it a trivial patch and OK to be pushed?
>>> Or shall we wait for one more review?
>>>
>>> Best regards,
>>> Jia
>>>
>>> ? 2020/3/10 19:37, Jia Huang ??:
>>>> Thanks Markus.
>>>>
>>>> Best regards,
>>>> Jia
>>>>
>>>> ? 2020/3/10 18:15, Markus Gronlund ??:
>>>>> Hi Jia,
>>>>>
>>>>> Looks like it could work.
>>>>>
>>>>> Thanks
>>>>> Markus
>>>>>
>>>>> -----Original Message-----
>>>>> From: Jia Huang <huangjia at loongson.cn>
>>>>> Sent: den 10 mars 2020 02:30
>>>>> To: Markus Gronlund <markus.gronlund at oracle.com>
>>>>> Cc: hotspot-jfr-dev at openjdk.java.net; hotspot-dev at openjdk.java.net
>>>>> Subject: Re: RFR: 8240222: [TESTBUG] 
>>>>> gtest/jfr/test_networkUtilization.cpp failed when the number of 
>>>>> tests is greater than or equal to 2
>>>>>
>>>>> Thanks Markus.
>>>>>
>>>>> Could you give some comments or advice on this one:
>>>>> Webrev: http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/ ?
>>>>>
>>>>> Best regards,
>>>>> Jia
>>>>>
>>>>>


From jiefu at tencent.com  Fri Mar 20 07:34:03 2020
From: jiefu at tencent.com (=?utf-8?B?amllZnUo5YKF5p2wKQ==?=)
Date: Fri, 20 Mar 2020 07:34:03 +0000
Subject: 8240222: [TESTBUG] gtest/jfr/test_networkUtilization.cpp failed
 when the number of tests is greater than or equal to 2(Internet mail)
Message-ID: <10B6AD8B-A6D2-4DB4-A073-2CAFAB2DADCB@tencent.com>

Hi Jia,

Please hg export your patch based on the latest jdk version.
I can push it for you.
Thanks.

Best regards,
Jie

?On 2020/3/20, 12:02 PM, "hotspot-dev on behalf of Jia Huang" <hotspot-dev-bounces at openjdk.java.net on behalf of huangjia at loongson.cn> wrote:

    Would someone please sponsor this patch?
    
    Webrev:  http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/ ?
    
    Best regards,
    Jia.
    
    >
    > ? 2020/3/16 12:00, Erik Gahlin ??:
    >> Looks reasonable.
    >>
    >> Erik
    >>
    >>> On 16 Mar 2020, at 04:54, Jia Huang <huangjia at loongson.cn> wrote:
    >>>
    >>> Hi Markus,
    >>>
    >>> Is it a trivial patch and OK to be pushed?
    >>> Or shall we wait for one more review?
    >>>
    >>> Best regards,
    >>> Jia
    >>>
    >>> ? 2020/3/10 19:37, Jia Huang ??:
    >>>> Thanks Markus.
    >>>>
    >>>> Best regards,
    >>>> Jia
    >>>>
    >>>> ? 2020/3/10 18:15, Markus Gronlund ??:
    >>>>> Hi Jia,
    >>>>>
    >>>>> Looks like it could work.
    >>>>>
    >>>>> Thanks
    >>>>> Markus
    >>>>>
    >>>>> -----Original Message-----
    >>>>> From: Jia Huang <huangjia at loongson.cn>
    >>>>> Sent: den 10 mars 2020 02:30
    >>>>> To: Markus Gronlund <markus.gronlund at oracle.com>
    >>>>> Cc: hotspot-jfr-dev at openjdk.java.net; hotspot-dev at openjdk.java.net
    >>>>> Subject: Re: RFR: 8240222: [TESTBUG] 
    >>>>> gtest/jfr/test_networkUtilization.cpp failed when the number of 
    >>>>> tests is greater than or equal to 2
    >>>>>
    >>>>> Thanks Markus.
    >>>>>
    >>>>> Could you give some comments or advice on this one:
    >>>>> Webrev: http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/ ?
    >>>>>
    >>>>> Best regards,
    >>>>> Jia
    >>>>>
    >>>>>
    
    
    


From huangjia at loongson.cn  Fri Mar 20 10:03:28 2020
From: huangjia at loongson.cn (Jia Huang)
Date: Fri, 20 Mar 2020 18:03:28 +0800
Subject: 8240222: [TESTBUG] gtest/jfr/test_networkUtilization.cpp failed
 when the number of tests is greater than or equal to 2(Internet mail)
In-Reply-To: <10B6AD8B-A6D2-4DB4-A073-2CAFAB2DADCB@tencent.com>
References: <10B6AD8B-A6D2-4DB4-A073-2CAFAB2DADCB@tencent.com>
Message-ID: <4941980d-048e-5a13-b9ce-201f22683b1a@loongson.cn>

Hi Jie,

Thanks a lot!

Webrev: http://cr.openjdk.java.net/~jiahuang/8240222/webrev.01/

Best regards,
Jia.

? 2020/3/20 15:34, jiefu(??) ??:
> Hi Jia,
>
> Please hg export your patch based on the latest jdk version.
> I can push it for you.
> Thanks.
>
> Best regards,
> Jie
>
> ?On 2020/3/20, 12:02 PM, "hotspot-dev on behalf of Jia Huang" <hotspot-dev-bounces at openjdk.java.net on behalf of huangjia at loongson.cn> wrote:
>
>      Would someone please sponsor this patch?
>      
>      Webrev:  http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/ ?
>      
>      Best regards,
>      Jia.
>      
>      >
>      > ? 2020/3/16 12:00, Erik Gahlin ??:
>      >> Looks reasonable.
>      >>
>      >> Erik
>      >>
>      >>> On 16 Mar 2020, at 04:54, Jia Huang <huangjia at loongson.cn> wrote:
>      >>>
>      >>> Hi Markus,
>      >>>
>      >>> Is it a trivial patch and OK to be pushed?
>      >>> Or shall we wait for one more review?
>      >>>
>      >>> Best regards,
>      >>> Jia
>      >>>
>      >>> ? 2020/3/10 19:37, Jia Huang ??:
>      >>>> Thanks Markus.
>      >>>>
>      >>>> Best regards,
>      >>>> Jia
>      >>>>
>      >>>> ? 2020/3/10 18:15, Markus Gronlund ??:
>      >>>>> Hi Jia,
>      >>>>>
>      >>>>> Looks like it could work.
>      >>>>>
>      >>>>> Thanks
>      >>>>> Markus
>      >>>>>
>      >>>>> -----Original Message-----
>      >>>>> From: Jia Huang <huangjia at loongson.cn>
>      >>>>> Sent: den 10 mars 2020 02:30
>      >>>>> To: Markus Gronlund <markus.gronlund at oracle.com>
>      >>>>> Cc: hotspot-jfr-dev at openjdk.java.net; hotspot-dev at openjdk.java.net
>      >>>>> Subject: Re: RFR: 8240222: [TESTBUG]
>      >>>>> gtest/jfr/test_networkUtilization.cpp failed when the number of
>      >>>>> tests is greater than or equal to 2
>      >>>>>
>      >>>>> Thanks Markus.
>      >>>>>
>      >>>>> Could you give some comments or advice on this one:
>      >>>>> Webrev: http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/ ?
>      >>>>>
>      >>>>> Best regards,
>      >>>>> Jia
>      >>>>>
>      >>>>>
>      
>      
>      
>


From jiefu at tencent.com  Fri Mar 20 10:55:45 2020
From: jiefu at tencent.com (=?utf-8?B?amllZnUo5YKF5p2wKQ==?=)
Date: Fri, 20 Mar 2020 10:55:45 +0000
Subject: (Internet mail)Re: 8240222: [TESTBUG]
 gtest/jfr/test_networkUtilization.cpp failed when the number of tests is
 greater than or equal to 2(Internet mail)
In-Reply-To: <4941980d-048e-5a13-b9ce-201f22683b1a@loongson.cn>
References: <10B6AD8B-A6D2-4DB4-A073-2CAFAB2DADCB@tencent.com>
 <4941980d-048e-5a13-b9ce-201f22683b1a@loongson.cn>
Message-ID: <5DC53BAA-0302-4D39-9882-563EBD1E8A52@tencent.com>

Pushed: http://hg.openjdk.java.net/jdk/jdk/rev/2c275730b19d

Cheers,
Jie

?On 2020/3/20, 6:03 PM, "Jia Huang" <huangjia at loongson.cn> wrote:

    Hi Jie,
    
    Thanks a lot!
    
    Webrev: http://cr.openjdk.java.net/~jiahuang/8240222/webrev.01/
    
    Best regards,
    Jia.
    
    ? 2020/3/20 15:34, jiefu(??) ??:
    > Hi Jia,
    >
    > Please hg export your patch based on the latest jdk version.
    > I can push it for you.
    > Thanks.
    >
    > Best regards,
    > Jie
    >
    > On 2020/3/20, 12:02 PM, "hotspot-dev on behalf of Jia Huang" <hotspot-dev-bounces at openjdk.java.net on behalf of huangjia at loongson.cn> wrote:
    >
    >      Would someone please sponsor this patch?
    >      
    >      Webrev:  http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/ ?
    >      
    >      Best regards,
    >      Jia.
    >      
    >      >
    >      > ? 2020/3/16 12:00, Erik Gahlin ??:
    >      >> Looks reasonable.
    >      >>
    >      >> Erik
    >      >>
    >      >>> On 16 Mar 2020, at 04:54, Jia Huang <huangjia at loongson.cn> wrote:
    >      >>>
    >      >>> Hi Markus,
    >      >>>
    >      >>> Is it a trivial patch and OK to be pushed?
    >      >>> Or shall we wait for one more review?
    >      >>>
    >      >>> Best regards,
    >      >>> Jia
    >      >>>
    >      >>> ? 2020/3/10 19:37, Jia Huang ??:
    >      >>>> Thanks Markus.
    >      >>>>
    >      >>>> Best regards,
    >      >>>> Jia
    >      >>>>
    >      >>>> ? 2020/3/10 18:15, Markus Gronlund ??:
    >      >>>>> Hi Jia,
    >      >>>>>
    >      >>>>> Looks like it could work.
    >      >>>>>
    >      >>>>> Thanks
    >      >>>>> Markus
    >      >>>>>
    >      >>>>> -----Original Message-----
    >      >>>>> From: Jia Huang <huangjia at loongson.cn>
    >      >>>>> Sent: den 10 mars 2020 02:30
    >      >>>>> To: Markus Gronlund <markus.gronlund at oracle.com>
    >      >>>>> Cc: hotspot-jfr-dev at openjdk.java.net; hotspot-dev at openjdk.java.net
    >      >>>>> Subject: Re: RFR: 8240222: [TESTBUG]
    >      >>>>> gtest/jfr/test_networkUtilization.cpp failed when the number of
    >      >>>>> tests is greater than or equal to 2
    >      >>>>>
    >      >>>>> Thanks Markus.
    >      >>>>>
    >      >>>>> Could you give some comments or advice on this one:
    >      >>>>> Webrev: http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/ ?
    >      >>>>>
    >      >>>>> Best regards,
    >      >>>>> Jia
    >      >>>>>
    >      >>>>>
    >      
    >      
    >      
    >
    
    
    


From magnus.ihse.bursie at oracle.com  Fri Mar 20 10:58:01 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Fri, 20 Mar 2020 11:58:01 +0100
Subject: RFR: JDK-8241271 Make hotspot build reproducible
In-Reply-To: <3e4ebab4-c31a-99b2-8062-6378443874cf@oracle.com>
References: <b5aaadfc-99d5-9d65-313e-980efe071bf1@oracle.com>
 <3e4ebab4-c31a-99b2-8062-6378443874cf@oracle.com>
Message-ID: <3d12f4e4-fab5-5dda-993c-a9fa94ca1288@oracle.com>

On 2020-03-19 19:01, Erik Joelsson wrote:
> On 2020-03-19 10:20, Magnus Ihse Bursie wrote:
>> Currently, it is not possible two make hotspot builds reproducible, 
>> since the current date and time is always inserted using the special 
>> macros __DATE__ and __TIME__. This patch makes it possible to 
>> override these values from the build system, paving the way for 
>> making re-builds of hotspot bit-by-bit identical.
>>
> I like the ability to achieve this.
>> On linux/gcc, we can now actually get bit-by-bit identical builds of 
>> libjvm.so (if not using precompiled headers), so I've updated the 
>> baseline-cmp profiles and compare script to reflect this.
>>
> I'm not sure this is a good idea though. It means I can't reasonably 
> compare to validate build changes that affect the precompiled header. 
> Similarly, the COMPARE_BUILD make variable argument will not work 
> unless you use both of these configure arguments.
>
> Since compare.sh is generated through configure, perhaps these 
> exceptions can be made conditional on if these configure arguments are 
> setup correctly for a reproducible libjvm.so?

Good thinking! I did not think of this possibility. And yes, I too did 
not like the current solution, but could think of no way around it. 
Here's an improved webrev.

WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8241271-make-hotspot-reproducible/webrev.02

Technically, you can make this fail by setting HOTSPOT_BUILD_TIME but to 
different values, but hey, there's no need to make this idiot-proof.

/Magnus
>
> /Erik
>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8241271
>> WebRev: 
>> http://cr.openjdk.java.net/~ihse/JDK-8241271-make-hotspot-reproducible/webrev.01
>>
>> /Magnus


From erik.osterlund at oracle.com  Fri Mar 20 10:58:37 2020
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Fri, 20 Mar 2020 11:58:37 +0100
Subject: RFR(L): Remove global safepoint code
In-Reply-To: <308290ab-0cf7-7a93-706f-94101597322f@oracle.com>
References: <308290ab-0cf7-7a93-706f-94101597322f@oracle.com>
Message-ID: <1150d7a9-d6e5-0b39-8693-2e60831f80b5@oracle.com>

Hi Robbin,

Was there any particular reason for changing the name of 
read_polling_page() on AArch64 to get_read_polling_page()?
It now sounds like tautology.

Otherwise this looks good, and I really like all the red lines. Dead 
code is the best code.

Thanks,
/Erik

On 2020-03-19 13:44, Robbin Ehn wrote:
> Hi all, please build, test and review.
>
> We removed (obsoleted) the flag ThreadLocalHandshakes in JDK 14.
> Any platform defining THREAD_LOCAL_POLL can only use per thread poll.
> Since all platforms now define THREAD_LOCAL_POLL the global poll code 
> is dead.
> Let's remove it!
>
> Issue: https://bugs.openjdk.java.net/browse/JDK-8220051
> Changeset: http://cr.openjdk.java.net/~rehn/8220051/v1/webrev/index.html
>
> I have built fastdebug:
> linux-aarch64
> linux-arm32
> linux-ppc64le
> linux-s390x
> linux-x86_64-zero
> linux-x86_64 (no PCH)
> linux-x86_64 (slowdebug) (no PCH)
> linux-x86
> osx-x86_64
> windows-x86_64
> solaris-sparcv9
>
> Tested t1-5:
> linux-x86_64
> osx-x86_64
> windows-x86_64
> solaris-sparcv9
>
> Manually:
> linux-aarch64
> linux-x86_64-zero
>
> Any extra help building and testing is appreciated, especially on 
> arm32, ppc64le, s390x.
>
> Thanks, Robbin


From robbin.ehn at oracle.com  Fri Mar 20 11:10:11 2020
From: robbin.ehn at oracle.com (Robbin Ehn)
Date: Fri, 20 Mar 2020 12:10:11 +0100
Subject: RFR(L): Remove global safepoint code
In-Reply-To: <1150d7a9-d6e5-0b39-8693-2e60831f80b5@oracle.com>
References: <308290ab-0cf7-7a93-706f-94101597322f@oracle.com>
 <1150d7a9-d6e5-0b39-8693-2e60831f80b5@oracle.com>
Message-ID: <10dc7515-4f61-62ae-9555-ede21032f914@oracle.com>

Hi Erik,

There was two read_polling_page:

address read_polling_page(Register r, address page, relocInfo::relocType 
rtype);
address read_polling_page(Register r, relocInfo::relocType rtype);

The first one took the global polling page as argument.
Since we no longer have that argument, to separate the methods I add 
get_ to the one that both gets and reads the thread polling page.
While the other one just reads the polling page from the address in the 
register is still called just read_.

Understandable?

On 2020-03-20 11:58, Erik ?sterlund wrote:
> Hi Robbin,
> 
> Was there any particular reason for changing the name of 
> read_polling_page() on AArch64 to get_read_polling_page()?
> It now sounds like tautology.
> 
> Otherwise this looks good, and I really like all the red lines. Dead 
> code is the best code.

Great, thanks!

/Robbin

> 
> Thanks,
> /Erik
> 
> On 2020-03-19 13:44, Robbin Ehn wrote:
>> Hi all, please build, test and review.
>>
>> We removed (obsoleted) the flag ThreadLocalHandshakes in JDK 14.
>> Any platform defining THREAD_LOCAL_POLL can only use per thread poll.
>> Since all platforms now define THREAD_LOCAL_POLL the global poll code 
>> is dead.
>> Let's remove it!
>>
>> Issue: https://bugs.openjdk.java.net/browse/JDK-8220051
>> Changeset: http://cr.openjdk.java.net/~rehn/8220051/v1/webrev/index.html
>>
>> I have built fastdebug:
>> linux-aarch64
>> linux-arm32
>> linux-ppc64le
>> linux-s390x
>> linux-x86_64-zero
>> linux-x86_64 (no PCH)
>> linux-x86_64 (slowdebug) (no PCH)
>> linux-x86
>> osx-x86_64
>> windows-x86_64
>> solaris-sparcv9
>>
>> Tested t1-5:
>> linux-x86_64
>> osx-x86_64
>> windows-x86_64
>> solaris-sparcv9
>>
>> Manually:
>> linux-aarch64
>> linux-x86_64-zero
>>
>> Any extra help building and testing is appreciated, especially on 
>> arm32, ppc64le, s390x.
>>
>> Thanks, Robbin
> 

From huangjia at loongson.cn  Fri Mar 20 11:10:22 2020
From: huangjia at loongson.cn (Jia Huang)
Date: Fri, 20 Mar 2020 19:10:22 +0800
Subject: (Internet mail)Re: 8240222: [TESTBUG]
 gtest/jfr/test_networkUtilization.cpp failed when the number of tests is
 greater than or equal to 2(Internet mail)
In-Reply-To: <5DC53BAA-0302-4D39-9882-563EBD1E8A52@tencent.com>
References: <10B6AD8B-A6D2-4DB4-A073-2CAFAB2DADCB@tencent.com>
 <4941980d-048e-5a13-b9ce-201f22683b1a@loongson.cn>
 <5DC53BAA-0302-4D39-9882-563EBD1E8A52@tencent.com>
Message-ID: <8a5abde9-af0d-ade0-8aff-db1a677a255f@loongson.cn>

Thanks a lot.

Best regards,
Jia.

? 2020/3/20 18:55, jiefu(??) ??:
> Pushed: http://hg.openjdk.java.net/jdk/jdk/rev/2c275730b19d
>
> Cheers,
> Jie
>
> ?On 2020/3/20, 6:03 PM, "Jia Huang" <huangjia at loongson.cn> wrote:
>
>      Hi Jie,
>      
>      Thanks a lot!
>      
>      Webrev: http://cr.openjdk.java.net/~jiahuang/8240222/webrev.01/
>      
>      Best regards,
>      Jia.
>      
>      ? 2020/3/20 15:34, jiefu(??) ??:
>      > Hi Jia,
>      >
>      > Please hg export your patch based on the latest jdk version.
>      > I can push it for you.
>      > Thanks.
>      >
>      > Best regards,
>      > Jie
>      >
>      > On 2020/3/20, 12:02 PM, "hotspot-dev on behalf of Jia Huang" <hotspot-dev-bounces at openjdk.java.net on behalf of huangjia at loongson.cn> wrote:
>      >
>      >      Would someone please sponsor this patch?
>      >
>      >      Webrev:  http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/ ?
>      >
>      >      Best regards,
>      >      Jia.
>      >
>      >      >
>      >      > ? 2020/3/16 12:00, Erik Gahlin ??:
>      >      >> Looks reasonable.
>      >      >>
>      >      >> Erik
>      >      >>
>      >      >>> On 16 Mar 2020, at 04:54, Jia Huang <huangjia at loongson.cn> wrote:
>      >      >>>
>      >      >>> Hi Markus,
>      >      >>>
>      >      >>> Is it a trivial patch and OK to be pushed?
>      >      >>> Or shall we wait for one more review?
>      >      >>>
>      >      >>> Best regards,
>      >      >>> Jia
>      >      >>>
>      >      >>> ? 2020/3/10 19:37, Jia Huang ??:
>      >      >>>> Thanks Markus.
>      >      >>>>
>      >      >>>> Best regards,
>      >      >>>> Jia
>      >      >>>>
>      >      >>>> ? 2020/3/10 18:15, Markus Gronlund ??:
>      >      >>>>> Hi Jia,
>      >      >>>>>
>      >      >>>>> Looks like it could work.
>      >      >>>>>
>      >      >>>>> Thanks
>      >      >>>>> Markus
>      >      >>>>>
>      >      >>>>> -----Original Message-----
>      >      >>>>> From: Jia Huang <huangjia at loongson.cn>
>      >      >>>>> Sent: den 10 mars 2020 02:30
>      >      >>>>> To: Markus Gronlund <markus.gronlund at oracle.com>
>      >      >>>>> Cc: hotspot-jfr-dev at openjdk.java.net; hotspot-dev at openjdk.java.net
>      >      >>>>> Subject: Re: RFR: 8240222: [TESTBUG]
>      >      >>>>> gtest/jfr/test_networkUtilization.cpp failed when the number of
>      >      >>>>> tests is greater than or equal to 2
>      >      >>>>>
>      >      >>>>> Thanks Markus.
>      >      >>>>>
>      >      >>>>> Could you give some comments or advice on this one:
>      >      >>>>> Webrev: http://cr.openjdk.java.net/~jiahuang/8240222/webrev.00/ ?
>      >      >>>>>
>      >      >>>>> Best regards,
>      >      >>>>> Jia
>      >      >>>>>
>      >      >>>>>
>      >
>      >
>      >
>      >
>      
>      
>      
>


From robbin.ehn at oracle.com  Fri Mar 20 11:11:13 2020
From: robbin.ehn at oracle.com (Robbin Ehn)
Date: Fri, 20 Mar 2020 04:11:13 -0700 (PDT)
Subject: RFR(L): 8220051: Remove global safepoint code
In-Reply-To: <9dd6fde0-f10e-ab3d-8db7-dcf919fd4b08@oracle.com>
References: <308290ab-0cf7-7a93-706f-94101597322f@oracle.com>
 <f467eac7-7fa4-f149-0c0f-cc1675351ae4@oracle.com>
 <9dd6fde0-f10e-ab3d-8db7-dcf919fd4b08@oracle.com>
Message-ID: <003ba651-a6f4-f4a9-0d46-72f9cf816024@oracle.com>

Hi Dan,

On 2020-03-19 21:39, Daniel D. Daugherty wrote:
> Hi Robbin!
> 
> On 3/19/20 8:45 AM, Robbin Ehn wrote:
>> On 2020-03-19 13:44, Robbin Ehn wrote:
>>> Hi all, please build, test and review.
>>>
>>> We removed (obsoleted) the flag ThreadLocalHandshakes in JDK 14.
>>> Any platform defining THREAD_LOCAL_POLL can only use per thread poll.
>>> Since all platforms now define THREAD_LOCAL_POLL the global poll code 
>>> is dead.
>>> Let's remove it!
>>>
>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8220051
>>> Changeset: http://cr.openjdk.java.net/~rehn/8220051/v1/webrev/index.html
> 
> Lots of files in this one. I'm gonna try to review these files
> in groups that make sense my warped mind... :-)
> 
> 
> Thumbs up on the code review.

Thanks alot!

> 
> I did my usual crawl thru review, but this is a lot of files so
> that boring detail is below...
> 
> I'm not entirely sure of the roll played by 
> SafepointMechanism::_polling_page,
> but that's probably because that didn't change except for the location of
> the _polling_page field itself.
> 
> With this kind of a change, I think you also want to get some Tier[6-8]
> test cycles in since that's where the stress tests really go to town...

Ok, testing!

Thanks, Robbin

> 
> Dan
> 
> 
> src/hotspot/share/runtime/safepointMechanism.hpp
>  ??? No comments.
> 
> src/hotspot/share/runtime/safepointMechanism.inline.hpp
>  ??? No comments.
> 
> src/hotspot/share/runtime/safepointMechanism.cpp
>  ??? old L73-81: Of course, now I'm wondering where the polling_page is 
> setup...
>  ??? Since it appears that we still have a global polling page for 
> non-JavaThreads,
>  ??? but I gotta read on...
> 
>  ??? I went to the bug to see if it would help my confusion, but the
>  ??? bug has no description. That should be fixed.
> 
>  ??? Update: The _polling_page is setup here:
> 
>  ??? L54: ??? char* bad_page? = polling_page;
>  ??? L57: ??? os::protect_memory(bad_page,? page_size, os::MEM_PROT_NONE);
>  ??? L61: ??? _polling_page = (address)(bad_page);
> 
>  ??? And for some reason, os/aix/safepointMechanism_aix.cpp has its own
>  ??? copy of this logic...
> 
> src/hotspot/share/runtime/safepoint.cpp
>  ??? No comments.
> 
> src/hotspot/share/runtime/os.hpp
>  ??? Okay... no more OS interface to the polling page... gotta read on...
> 
> src/hotspot/share/runtime/biasedLocking.cpp
>  ??? No comments.
> 
> src/hotspot/share/runtime/handshake.cpp
>  ??? No comments.
> 
> src/hotspot/share/runtime/sweeper.cpp
>  ??? No comments.
> 
> src/hotspot/share/runtime/thread.cpp
>  ??? No comments.
> 
> src/hotspot/share/utilities/vmError.cpp
>  ??? No comments.
> 
> 
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp
>  ??? No comments.
> 
> src/hotspot/share/gc/z/zMark.cpp
>  ??? No comments.
> 
> src/hotspot/share/opto/callnode.cpp
>  ??? No comments.
> 
> src/hotspot/share/opto/machnode.hpp
>  ??? No comments.
> 
> src/hotspot/share/opto/machnode.cpp
>  ??? No comments.
> 
> src/hotspot/share/opto/parse1.cpp
>  ??? No comments.
> 
> src/hotspot/share/aot/aotCodeHeap.cpp
>  ??? No comments.
> 
> src/jdk.aot/share/classes/jdk.tools.jaotc.binformat/src/jdk/tools/jaotc/binformat/BinaryContainer.java 
> 
>  ??? No comments.
> 
> src/jdk.aot/share/classes/jdk.tools.jaotc/src/jdk/tools/jaotc/MarkProcessor.java 
> 
>  ??? No comments.
> 
> 
> src/hotspot/os/bsd/os_bsd.cpp
>  ??? No comments.
> 
> src/hotspot/os/linux/os_linux.cpp
>  ??? No comments.
> 
> src/hotspot/os/solaris/os_solaris.cpp
>  ??? No comments.
> 
> src/hotspot/os/windows/os_windows.cpp
>  ??? No comments.
> 
> 
> src/hotspot/os_cpu/bsd_x86/os_bsd_x86.cpp
>  ??? No comments.
> 
> src/hotspot/os_cpu/linux_sparc/os_linux_sparc.cpp
>  ??? No comments.
> 
> src/hotspot/os_cpu/linux_x86/os_linux_x86.cpp
>  ??? No comments.
> 
> src/hotspot/os_cpu/solaris_sparc/os_solaris_sparc.cpp
>  ??? No comments.
> 
> src/hotspot/os_cpu/solaris_x86/os_solaris_x86.cpp
>  ??? No comments.
> 
> 
> src/hotspot/cpu/sparc/c1_LIRAssembler_sparc.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/sparc/globalDefinitions_sparc.hpp
>  ??? No comments.
> 
> src/hotspot/cpu/sparc/interp_masm_sparc.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/sparc/macroAssembler_sparc.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/sparc/sharedRuntime_sparc.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/sparc/sparc.ad
>  ??? No comments.
> 
> src/hotspot/cpu/sparc/templateTable_sparc.cpp
>  ??? No comments.
> 
> 
> src/hotspot/cpu/x86/assembler_x86.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/x86/assembler_x86.hpp
>  ??? No comments.
> 
> src/hotspot/cpu/x86/c1_LIRAssembler_x86.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/x86/c1_LIRGenerator_x86.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/x86/globalDefinitions_x86.hpp
>  ??? No comments.
> 
> src/hotspot/cpu/x86/interp_masm_x86.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/x86/jvmciCodeInstaller_x86.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/x86/macroAssembler_x86.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/x86/nativeInst_x86.hpp
>  ??? No comments.
> 
> src/hotspot/cpu/x86/relocInfo_x86.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/x86/sharedRuntime_x86_32.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/x86/sharedRuntime_x86_64.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/x86/templateTable_x86.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/x86/x86_32.ad
>  ??? No comments.
> 
> src/hotspot/cpu/x86/x86_64.ad
>  ??? No comments.
> 
> 
> src/hotspot/os/aix/os_aix.cpp
>  ??? No comments.
> 
> src/hotspot/os/aix/safepointMechanism_aix.cpp
>  ??? No comments.
> 
> src/hotspot/os_cpu/aix_ppc/os_aix_ppc.cpp
>  ??? No comments.
> 
> src/hotspot/os_cpu/linux_aarch64/os_linux_aarch64.cpp
>  ??? No comments.
> 
> src/hotspot/os_cpu/linux_arm/os_linux_arm.cpp
>  ??? No comments.
> 
> src/hotspot/os_cpu/linux_ppc/os_linux_ppc.cpp
>  ??? No comments.
> 
> src/hotspot/os_cpu/linux_s390/os_linux_s390.cpp
>  ??? No comments.
> 
> 
> src/hotspot/cpu/aarch64/aarch64.ad
>  ??? No comments.
> 
> src/hotspot/cpu/aarch64/c1_LIRAssembler_aarch64.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/aarch64/globalDefinitions_aarch64.hpp
>  ??? No comments.
> 
> src/hotspot/cpu/aarch64/interp_masm_aarch64.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/aarch64/macroAssembler_aarch64.hpp
>  ??? No comments.
> 
> src/hotspot/cpu/aarch64/sharedRuntime_aarch64.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/arm/arm.ad
>  ??? No comments.
> 
> src/hotspot/cpu/arm/globalDefinitions_arm.hpp
>  ??? No comments.
> 
> src/hotspot/cpu/arm/interp_masm_arm.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/arm/macroAssembler_arm.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/arm/sharedRuntime_arm.cpp
>  ??? No comments.
> 
> 
> src/hotspot/cpu/ppc/c1_LIRAssembler_ppc.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/ppc/globalDefinitions_ppc.hpp
>  ??? No comments.
> 
> src/hotspot/cpu/ppc/interp_masm_ppc_64.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/ppc/macroAssembler_ppc.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/ppc/macroAssembler_ppc.inline.hpp
>  ??? No comments.
> 
> src/hotspot/cpu/ppc/nativeInst_ppc.hpp
>  ??? No comments.
> 
> src/hotspot/cpu/ppc/ppc.ad
>  ??? No comments.
> 
> src/hotspot/cpu/ppc/sharedRuntime_ppc.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/ppc/templateTable_ppc_64.cpp
>  ??? No comments.
> 
> 
> src/hotspot/cpu/s390/c1_LIRAssembler_s390.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/s390/globalDefinitions_s390.hpp
>  ??? No comments.
> 
> src/hotspot/cpu/s390/interp_masm_s390.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/s390/macroAssembler_s390.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/s390/s390.ad
>  ??? No comments.
> 
> src/hotspot/cpu/s390/sharedRuntime_s390.cpp
>  ??? No comments.
> 
> src/hotspot/cpu/s390/templateTable_s390.cpp
>  ??? No comments.
> 
> 
> src/hotspot/cpu/zero/globalDefinitions_zero.hpp
>  ??? No comments.
> 
> 
>>>
>>> I have built fastdebug:
>>> linux-aarch64
>>> linux-arm32
>>> linux-ppc64le
>>> linux-s390x
>>> linux-x86_64-zero
>>> linux-x86_64 (no PCH)
>>> linux-x86_64 (slowdebug) (no PCH)
>>> linux-x86
>>> osx-x86_64
>>> windows-x86_64
>>> solaris-sparcv9
>>>
>>> Tested t1-5:
>>> linux-x86_64
>>> osx-x86_64
>>> windows-x86_64
>>> solaris-sparcv9
>>>
>>> Manually:
>>> linux-aarch64
>>> linux-x86_64-zero
>>>
>>> Any extra help building and testing is appreciated, especially on 
>>> arm32, ppc64le, s390x.
>>>
>>> Thanks, Robbin
> 

From erik.osterlund at oracle.com  Fri Mar 20 11:13:21 2020
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Fri, 20 Mar 2020 12:13:21 +0100
Subject: RFR(L): Remove global safepoint code
In-Reply-To: <10dc7515-4f61-62ae-9555-ede21032f914@oracle.com>
References: <308290ab-0cf7-7a93-706f-94101597322f@oracle.com>
 <1150d7a9-d6e5-0b39-8693-2e60831f80b5@oracle.com>
 <10dc7515-4f61-62ae-9555-ede21032f914@oracle.com>
Message-ID: <868d7338-e043-6d48-48a7-fa493124f130@oracle.com>

Hi Robbin,

I see. Maybe we can call it fetch_and_read_polling_page() instead.
Don't need another webrev for that if you agree; ship it!

Thanks,
/Erik

On 2020-03-20 12:10, Robbin Ehn wrote:
> Hi Erik,
>
> There was two read_polling_page:
>
> address read_polling_page(Register r, address page, 
> relocInfo::relocType rtype);
> address read_polling_page(Register r, relocInfo::relocType rtype);
>
> The first one took the global polling page as argument.
> Since we no longer have that argument, to separate the methods I add 
> get_ to the one that both gets and reads the thread polling page.
> While the other one just reads the polling page from the address in 
> the register is still called just read_.
>
> Understandable?
>
> On 2020-03-20 11:58, Erik ?sterlund wrote:
>> Hi Robbin,
>>
>> Was there any particular reason for changing the name of 
>> read_polling_page() on AArch64 to get_read_polling_page()?
>> It now sounds like tautology.
>>
>> Otherwise this looks good, and I really like all the red lines. Dead 
>> code is the best code.
>
> Great, thanks!
>
> /Robbin
>
>>
>> Thanks,
>> /Erik
>>
>> On 2020-03-19 13:44, Robbin Ehn wrote:
>>> Hi all, please build, test and review.
>>>
>>> We removed (obsoleted) the flag ThreadLocalHandshakes in JDK 14.
>>> Any platform defining THREAD_LOCAL_POLL can only use per thread poll.
>>> Since all platforms now define THREAD_LOCAL_POLL the global poll 
>>> code is dead.
>>> Let's remove it!
>>>
>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8220051
>>> Changeset: 
>>> http://cr.openjdk.java.net/~rehn/8220051/v1/webrev/index.html
>>>
>>> I have built fastdebug:
>>> linux-aarch64
>>> linux-arm32
>>> linux-ppc64le
>>> linux-s390x
>>> linux-x86_64-zero
>>> linux-x86_64 (no PCH)
>>> linux-x86_64 (slowdebug) (no PCH)
>>> linux-x86
>>> osx-x86_64
>>> windows-x86_64
>>> solaris-sparcv9
>>>
>>> Tested t1-5:
>>> linux-x86_64
>>> osx-x86_64
>>> windows-x86_64
>>> solaris-sparcv9
>>>
>>> Manually:
>>> linux-aarch64
>>> linux-x86_64-zero
>>>
>>> Any extra help building and testing is appreciated, especially on 
>>> arm32, ppc64le, s390x.
>>>
>>> Thanks, Robbin
>>


From robbin.ehn at oracle.com  Fri Mar 20 11:16:43 2020
From: robbin.ehn at oracle.com (Robbin Ehn)
Date: Fri, 20 Mar 2020 12:16:43 +0100
Subject: RFR(L): Remove global safepoint code
In-Reply-To: <868d7338-e043-6d48-48a7-fa493124f130@oracle.com>
References: <308290ab-0cf7-7a93-706f-94101597322f@oracle.com>
 <1150d7a9-d6e5-0b39-8693-2e60831f80b5@oracle.com>
 <10dc7515-4f61-62ae-9555-ede21032f914@oracle.com>
 <868d7338-e043-6d48-48a7-fa493124f130@oracle.com>
Message-ID: <d500c3df-aaea-d902-ba5d-54ba73ffb972@oracle.com>

Hi Erik,

On 2020-03-20 12:13, Erik ?sterlund wrote:
> Hi Robbin,
> 
> I see. Maybe we can call it fetch_and_read_polling_page() instead.
> Don't need another webrev for that if you agree; ship it!

I'll change!

Thanks, Robbin

> 
> Thanks,
> /Erik
> 
> On 2020-03-20 12:10, Robbin Ehn wrote:
>> Hi Erik,
>>
>> There was two read_polling_page:
>>
>> address read_polling_page(Register r, address page, 
>> relocInfo::relocType rtype);
>> address read_polling_page(Register r, relocInfo::relocType rtype);
>>
>> The first one took the global polling page as argument.
>> Since we no longer have that argument, to separate the methods I add 
>> get_ to the one that both gets and reads the thread polling page.
>> While the other one just reads the polling page from the address in 
>> the register is still called just read_.
>>
>> Understandable?
>>
>> On 2020-03-20 11:58, Erik ?sterlund wrote:
>>> Hi Robbin,
>>>
>>> Was there any particular reason for changing the name of 
>>> read_polling_page() on AArch64 to get_read_polling_page()?
>>> It now sounds like tautology.
>>>
>>> Otherwise this looks good, and I really like all the red lines. Dead 
>>> code is the best code.
>>
>> Great, thanks!
>>
>> /Robbin
>>
>>>
>>> Thanks,
>>> /Erik
>>>
>>> On 2020-03-19 13:44, Robbin Ehn wrote:
>>>> Hi all, please build, test and review.
>>>>
>>>> We removed (obsoleted) the flag ThreadLocalHandshakes in JDK 14.
>>>> Any platform defining THREAD_LOCAL_POLL can only use per thread poll.
>>>> Since all platforms now define THREAD_LOCAL_POLL the global poll 
>>>> code is dead.
>>>> Let's remove it!
>>>>
>>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8220051
>>>> Changeset: 
>>>> http://cr.openjdk.java.net/~rehn/8220051/v1/webrev/index.html
>>>>
>>>> I have built fastdebug:
>>>> linux-aarch64
>>>> linux-arm32
>>>> linux-ppc64le
>>>> linux-s390x
>>>> linux-x86_64-zero
>>>> linux-x86_64 (no PCH)
>>>> linux-x86_64 (slowdebug) (no PCH)
>>>> linux-x86
>>>> osx-x86_64
>>>> windows-x86_64
>>>> solaris-sparcv9
>>>>
>>>> Tested t1-5:
>>>> linux-x86_64
>>>> osx-x86_64
>>>> windows-x86_64
>>>> solaris-sparcv9
>>>>
>>>> Manually:
>>>> linux-aarch64
>>>> linux-x86_64-zero
>>>>
>>>> Any extra help building and testing is appreciated, especially on 
>>>> arm32, ppc64le, s390x.
>>>>
>>>> Thanks, Robbin
>>>
> 

From coleen.phillimore at oracle.com  Fri Mar 20 11:35:53 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Fri, 20 Mar 2020 07:35:53 -0400
Subject: RFR(L): Remove global safepoint code
In-Reply-To: <868d7338-e043-6d48-48a7-fa493124f130@oracle.com>
References: <308290ab-0cf7-7a93-706f-94101597322f@oracle.com>
 <1150d7a9-d6e5-0b39-8693-2e60831f80b5@oracle.com>
 <10dc7515-4f61-62ae-9555-ede21032f914@oracle.com>
 <868d7338-e043-6d48-48a7-fa493124f130@oracle.com>
Message-ID: <560608cf-7959-641e-fb8e-ff0381b93edd@oracle.com>

I reviewed this code and had the same comment :)
Everything else looked great.

Coleen

On 3/20/20 7:13 AM, Erik ?sterlund wrote:
> Hi Robbin,
>
> I see. Maybe we can call it fetch_and_read_polling_page() instead.
> Don't need another webrev for that if you agree; ship it!
>
> Thanks,
> /Erik
>
> On 2020-03-20 12:10, Robbin Ehn wrote:
>> Hi Erik,
>>
>> There was two read_polling_page:
>>
>> address read_polling_page(Register r, address page, 
>> relocInfo::relocType rtype);
>> address read_polling_page(Register r, relocInfo::relocType rtype);
>>
>> The first one took the global polling page as argument.
>> Since we no longer have that argument, to separate the methods I add 
>> get_ to the one that both gets and reads the thread polling page.
>> While the other one just reads the polling page from the address in 
>> the register is still called just read_.
>>
>> Understandable?
>>
>> On 2020-03-20 11:58, Erik ?sterlund wrote:
>>> Hi Robbin,
>>>
>>> Was there any particular reason for changing the name of 
>>> read_polling_page() on AArch64 to get_read_polling_page()?
>>> It now sounds like tautology.
>>>
>>> Otherwise this looks good, and I really like all the red lines. Dead 
>>> code is the best code.
>>
>> Great, thanks!
>>
>> /Robbin
>>
>>>
>>> Thanks,
>>> /Erik
>>>
>>> On 2020-03-19 13:44, Robbin Ehn wrote:
>>>> Hi all, please build, test and review.
>>>>
>>>> We removed (obsoleted) the flag ThreadLocalHandshakes in JDK 14.
>>>> Any platform defining THREAD_LOCAL_POLL can only use per thread poll.
>>>> Since all platforms now define THREAD_LOCAL_POLL the global poll 
>>>> code is dead.
>>>> Let's remove it!
>>>>
>>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8220051
>>>> Changeset: 
>>>> http://cr.openjdk.java.net/~rehn/8220051/v1/webrev/index.html
>>>>
>>>> I have built fastdebug:
>>>> linux-aarch64
>>>> linux-arm32
>>>> linux-ppc64le
>>>> linux-s390x
>>>> linux-x86_64-zero
>>>> linux-x86_64 (no PCH)
>>>> linux-x86_64 (slowdebug) (no PCH)
>>>> linux-x86
>>>> osx-x86_64
>>>> windows-x86_64
>>>> solaris-sparcv9
>>>>
>>>> Tested t1-5:
>>>> linux-x86_64
>>>> osx-x86_64
>>>> windows-x86_64
>>>> solaris-sparcv9
>>>>
>>>> Manually:
>>>> linux-aarch64
>>>> linux-x86_64-zero
>>>>
>>>> Any extra help building and testing is appreciated, especially on 
>>>> arm32, ppc64le, s390x.
>>>>
>>>> Thanks, Robbin
>>>
>


From erik.joelsson at oracle.com  Fri Mar 20 13:05:38 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Fri, 20 Mar 2020 06:05:38 -0700
Subject: RFR: JDK-8241271 Make hotspot build reproducible
In-Reply-To: <3d12f4e4-fab5-5dda-993c-a9fa94ca1288@oracle.com>
References: <b5aaadfc-99d5-9d65-313e-980efe071bf1@oracle.com>
 <3e4ebab4-c31a-99b2-8062-6378443874cf@oracle.com>
 <3d12f4e4-fab5-5dda-993c-a9fa94ca1288@oracle.com>
Message-ID: <0b9acd8e-bda8-d5bd-d1ef-0b46a17b6aa0@oracle.com>

Looks good!

/Erik

On 2020-03-20 03:58, Magnus Ihse Bursie wrote:
> On 2020-03-19 19:01, Erik Joelsson wrote:
>> On 2020-03-19 10:20, Magnus Ihse Bursie wrote:
>>> Currently, it is not possible two make hotspot builds reproducible, 
>>> since the current date and time is always inserted using the special 
>>> macros __DATE__ and __TIME__. This patch makes it possible to 
>>> override these values from the build system, paving the way for 
>>> making re-builds of hotspot bit-by-bit identical.
>>>
>> I like the ability to achieve this.
>>> On linux/gcc, we can now actually get bit-by-bit identical builds of 
>>> libjvm.so (if not using precompiled headers), so I've updated the 
>>> baseline-cmp profiles and compare script to reflect this.
>>>
>> I'm not sure this is a good idea though. It means I can't reasonably 
>> compare to validate build changes that affect the precompiled header. 
>> Similarly, the COMPARE_BUILD make variable argument will not work 
>> unless you use both of these configure arguments.
>>
>> Since compare.sh is generated through configure, perhaps these 
>> exceptions can be made conditional on if these configure arguments 
>> are setup correctly for a reproducible libjvm.so?
>
> Good thinking! I did not think of this possibility. And yes, I too did 
> not like the current solution, but could think of no way around it. 
> Here's an improved webrev.
>
> WebRev: 
> http://cr.openjdk.java.net/~ihse/JDK-8241271-make-hotspot-reproducible/webrev.02
>
> Technically, you can make this fail by setting HOTSPOT_BUILD_TIME but 
> to different values, but hey, there's no need to make this idiot-proof.
>
> /Magnus
>>
>> /Erik
>>
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8241271
>>> WebRev: 
>>> http://cr.openjdk.java.net/~ihse/JDK-8241271-make-hotspot-reproducible/webrev.01
>>>
>>> /Magnus
>

From magnus.ihse.bursie at oracle.com  Fri Mar 20 13:15:25 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Fri, 20 Mar 2020 14:15:25 +0100
Subject: RFR: JDK-8241271 Make hotspot build reproducible
In-Reply-To: <0b9acd8e-bda8-d5bd-d1ef-0b46a17b6aa0@oracle.com>
References: <b5aaadfc-99d5-9d65-313e-980efe071bf1@oracle.com>
 <3e4ebab4-c31a-99b2-8062-6378443874cf@oracle.com>
 <3d12f4e4-fab5-5dda-993c-a9fa94ca1288@oracle.com>
 <0b9acd8e-bda8-d5bd-d1ef-0b46a17b6aa0@oracle.com>
Message-ID: <663deddf-03d4-d982-08fe-427e974c111f@oracle.com>

Can I get some hotspot reviewers on this as well? And is this trivial, 
from the Hotspot PoV?

/Magnus

On 2020-03-20 14:05, Erik Joelsson wrote:
> Looks good!
>
> /Erik
>
> On 2020-03-20 03:58, Magnus Ihse Bursie wrote:
>> On 2020-03-19 19:01, Erik Joelsson wrote:
>>> On 2020-03-19 10:20, Magnus Ihse Bursie wrote:
>>>> Currently, it is not possible two make hotspot builds reproducible, 
>>>> since the current date and time is always inserted using the 
>>>> special macros __DATE__ and __TIME__. This patch makes it possible 
>>>> to override these values from the build system, paving the way for 
>>>> making re-builds of hotspot bit-by-bit identical.
>>>>
>>> I like the ability to achieve this.
>>>> On linux/gcc, we can now actually get bit-by-bit identical builds 
>>>> of libjvm.so (if not using precompiled headers), so I've updated 
>>>> the baseline-cmp profiles and compare script to reflect this.
>>>>
>>> I'm not sure this is a good idea though. It means I can't reasonably 
>>> compare to validate build changes that affect the precompiled 
>>> header. Similarly, the COMPARE_BUILD make variable argument will not 
>>> work unless you use both of these configure arguments.
>>>
>>> Since compare.sh is generated through configure, perhaps these 
>>> exceptions can be made conditional on if these configure arguments 
>>> are setup correctly for a reproducible libjvm.so?
>>
>> Good thinking! I did not think of this possibility. And yes, I too 
>> did not like the current solution, but could think of no way around 
>> it. Here's an improved webrev.
>>
>> WebRev: 
>> http://cr.openjdk.java.net/~ihse/JDK-8241271-make-hotspot-reproducible/webrev.02
>>
>> Technically, you can make this fail by setting HOTSPOT_BUILD_TIME but 
>> to different values, but hey, there's no need to make this idiot-proof.
>>
>> /Magnus
>>>
>>> /Erik
>>>
>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8241271
>>>> WebRev: 
>>>> http://cr.openjdk.java.net/~ihse/JDK-8241271-make-hotspot-reproducible/webrev.01
>>>>
>>>> /Magnus
>>


From stefan.karlsson at oracle.com  Fri Mar 20 13:19:16 2020
From: stefan.karlsson at oracle.com (Stefan Karlsson)
Date: Fri, 20 Mar 2020 06:19:16 -0700 (PDT)
Subject: RFR: JDK-8241271 Make hotspot build reproducible
In-Reply-To: <663deddf-03d4-d982-08fe-427e974c111f@oracle.com>
References: <b5aaadfc-99d5-9d65-313e-980efe071bf1@oracle.com>
 <3e4ebab4-c31a-99b2-8062-6378443874cf@oracle.com>
 <3d12f4e4-fab5-5dda-993c-a9fa94ca1288@oracle.com>
 <0b9acd8e-bda8-d5bd-d1ef-0b46a17b6aa0@oracle.com>
 <663deddf-03d4-d982-08fe-427e974c111f@oracle.com>
Message-ID: <cab4a050-3484-c2ef-8d34-2c43357d01e8@oracle.com>

HotSpot changes look good.

StefanK

On 2020-03-20 14:15, Magnus Ihse Bursie wrote:
> Can I get some hotspot reviewers on this as well? And is this trivial, 
> from the Hotspot PoV?
>
> /Magnus
>
> On 2020-03-20 14:05, Erik Joelsson wrote:
>> Looks good!
>>
>> /Erik
>>
>> On 2020-03-20 03:58, Magnus Ihse Bursie wrote:
>>> On 2020-03-19 19:01, Erik Joelsson wrote:
>>>> On 2020-03-19 10:20, Magnus Ihse Bursie wrote:
>>>>> Currently, it is not possible two make hotspot builds 
>>>>> reproducible, since the current date and time is always inserted 
>>>>> using the special macros __DATE__ and __TIME__. This patch makes 
>>>>> it possible to override these values from the build system, paving 
>>>>> the way for making re-builds of hotspot bit-by-bit identical.
>>>>>
>>>> I like the ability to achieve this.
>>>>> On linux/gcc, we can now actually get bit-by-bit identical builds 
>>>>> of libjvm.so (if not using precompiled headers), so I've updated 
>>>>> the baseline-cmp profiles and compare script to reflect this.
>>>>>
>>>> I'm not sure this is a good idea though. It means I can't 
>>>> reasonably compare to validate build changes that affect the 
>>>> precompiled header. Similarly, the COMPARE_BUILD make variable 
>>>> argument will not work unless you use both of these configure 
>>>> arguments.
>>>>
>>>> Since compare.sh is generated through configure, perhaps these 
>>>> exceptions can be made conditional on if these configure arguments 
>>>> are setup correctly for a reproducible libjvm.so?
>>>
>>> Good thinking! I did not think of this possibility. And yes, I too 
>>> did not like the current solution, but could think of no way around 
>>> it. Here's an improved webrev.
>>>
>>> WebRev: 
>>> http://cr.openjdk.java.net/~ihse/JDK-8241271-make-hotspot-reproducible/webrev.02
>>>
>>> Technically, you can make this fail by setting HOTSPOT_BUILD_TIME 
>>> but to different values, but hey, there's no need to make this 
>>> idiot-proof.
>>>
>>> /Magnus
>>>>
>>>> /Erik
>>>>
>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8241271
>>>>> WebRev: 
>>>>> http://cr.openjdk.java.net/~ihse/JDK-8241271-make-hotspot-reproducible/webrev.01
>>>>>
>>>>> /Magnus
>>>
>


From robbin.ehn at oracle.com  Fri Mar 20 13:35:00 2020
From: robbin.ehn at oracle.com (Robbin Ehn)
Date: Fri, 20 Mar 2020 14:35:00 +0100
Subject: RFR(L): Remove global safepoint code
In-Reply-To: <560608cf-7959-641e-fb8e-ff0381b93edd@oracle.com>
References: <308290ab-0cf7-7a93-706f-94101597322f@oracle.com>
 <1150d7a9-d6e5-0b39-8693-2e60831f80b5@oracle.com>
 <10dc7515-4f61-62ae-9555-ede21032f914@oracle.com>
 <868d7338-e043-6d48-48a7-fa493124f130@oracle.com>
 <560608cf-7959-641e-fb8e-ff0381b93edd@oracle.com>
Message-ID: <982b6691-48bd-c7a6-c987-8f058127ff4d@oracle.com>

Thanks Coleen!

/Robbin

On 2020-03-20 12:35, coleen.phillimore at oracle.com wrote:
> I reviewed this code and had the same comment :)
> Everything else looked great.
> 
> Coleen
> 
> On 3/20/20 7:13 AM, Erik ?sterlund wrote:
>> Hi Robbin,
>>
>> I see. Maybe we can call it fetch_and_read_polling_page() instead.
>> Don't need another webrev for that if you agree; ship it!
>>
>> Thanks,
>> /Erik
>>
>> On 2020-03-20 12:10, Robbin Ehn wrote:
>>> Hi Erik,
>>>
>>> There was two read_polling_page:
>>>
>>> address read_polling_page(Register r, address page, 
>>> relocInfo::relocType rtype);
>>> address read_polling_page(Register r, relocInfo::relocType rtype);
>>>
>>> The first one took the global polling page as argument.
>>> Since we no longer have that argument, to separate the methods I add 
>>> get_ to the one that both gets and reads the thread polling page.
>>> While the other one just reads the polling page from the address in 
>>> the register is still called just read_.
>>>
>>> Understandable?
>>>
>>> On 2020-03-20 11:58, Erik ?sterlund wrote:
>>>> Hi Robbin,
>>>>
>>>> Was there any particular reason for changing the name of 
>>>> read_polling_page() on AArch64 to get_read_polling_page()?
>>>> It now sounds like tautology.
>>>>
>>>> Otherwise this looks good, and I really like all the red lines. Dead 
>>>> code is the best code.
>>>
>>> Great, thanks!
>>>
>>> /Robbin
>>>
>>>>
>>>> Thanks,
>>>> /Erik
>>>>
>>>> On 2020-03-19 13:44, Robbin Ehn wrote:
>>>>> Hi all, please build, test and review.
>>>>>
>>>>> We removed (obsoleted) the flag ThreadLocalHandshakes in JDK 14.
>>>>> Any platform defining THREAD_LOCAL_POLL can only use per thread poll.
>>>>> Since all platforms now define THREAD_LOCAL_POLL the global poll 
>>>>> code is dead.
>>>>> Let's remove it!
>>>>>
>>>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8220051
>>>>> Changeset: 
>>>>> http://cr.openjdk.java.net/~rehn/8220051/v1/webrev/index.html
>>>>>
>>>>> I have built fastdebug:
>>>>> linux-aarch64
>>>>> linux-arm32
>>>>> linux-ppc64le
>>>>> linux-s390x
>>>>> linux-x86_64-zero
>>>>> linux-x86_64 (no PCH)
>>>>> linux-x86_64 (slowdebug) (no PCH)
>>>>> linux-x86
>>>>> osx-x86_64
>>>>> windows-x86_64
>>>>> solaris-sparcv9
>>>>>
>>>>> Tested t1-5:
>>>>> linux-x86_64
>>>>> osx-x86_64
>>>>> windows-x86_64
>>>>> solaris-sparcv9
>>>>>
>>>>> Manually:
>>>>> linux-aarch64
>>>>> linux-x86_64-zero
>>>>>
>>>>> Any extra help building and testing is appreciated, especially on 
>>>>> arm32, ppc64le, s390x.
>>>>>
>>>>> Thanks, Robbin
>>>>
>>
> 

From robbin.ehn at oracle.com  Mon Mar 23 08:12:10 2020
From: robbin.ehn at oracle.com (Robbin Ehn)
Date: Mon, 23 Mar 2020 09:12:10 +0100
Subject: RFR(L): 8220051: Remove global safepoint code
In-Reply-To: <f467eac7-7fa4-f149-0c0f-cc1675351ae4@oracle.com>
References: <308290ab-0cf7-7a93-706f-94101597322f@oracle.com>
 <f467eac7-7fa4-f149-0c0f-cc1675351ae4@oracle.com>
Message-ID: <ad465749-7378-58f9-e808-712ec0c8bc4c@oracle.com>

Hi,

Only arm32 changes are untested as of now (only built).
If you don't have time to test, can you please just acknowledge that you
have seen this before it goes in. If you need more time let me know, no 
rush.

Thanks, Robbin

On 2020-03-19 13:45, Robbin Ehn wrote:
> Corrected mail-header.
> 
> /Robbin
> 
> On 2020-03-19 13:44, Robbin Ehn wrote:
>> Hi all, please build, test and review.
>>
>> We removed (obsoleted) the flag ThreadLocalHandshakes in JDK 14.
>> Any platform defining THREAD_LOCAL_POLL can only use per thread poll.
>> Since all platforms now define THREAD_LOCAL_POLL the global poll code 
>> is dead.
>> Let's remove it!
>>
>> Issue: https://bugs.openjdk.java.net/browse/JDK-8220051
>> Changeset: http://cr.openjdk.java.net/~rehn/8220051/v1/webrev/index.html
>>
>> I have built fastdebug:
>> linux-aarch64
>> linux-arm32
>> linux-ppc64le
>> linux-s390x
>> linux-x86_64-zero
>> linux-x86_64 (no PCH)
>> linux-x86_64 (slowdebug) (no PCH)
>> linux-x86
>> osx-x86_64
>> windows-x86_64
>> solaris-sparcv9
>>
>> Tested t1-5:
>> linux-x86_64
>> osx-x86_64
>> windows-x86_64
>> solaris-sparcv9
>>
>> Manually:
>> linux-aarch64
>> linux-x86_64-zero
>>
>> Any extra help building and testing is appreciated, especially on 
>> arm32, ppc64le, s390x.
>>
>> Thanks, Robbin

From magnus.ihse.bursie at oracle.com  Mon Mar 23 08:45:20 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Mon, 23 Mar 2020 09:45:20 +0100
Subject: RFR: JDK-8241271 Make hotspot build reproducible
In-Reply-To: <cab4a050-3484-c2ef-8d34-2c43357d01e8@oracle.com>
References: <b5aaadfc-99d5-9d65-313e-980efe071bf1@oracle.com>
 <3e4ebab4-c31a-99b2-8062-6378443874cf@oracle.com>
 <3d12f4e4-fab5-5dda-993c-a9fa94ca1288@oracle.com>
 <0b9acd8e-bda8-d5bd-d1ef-0b46a17b6aa0@oracle.com>
 <663deddf-03d4-d982-08fe-427e974c111f@oracle.com>
 <cab4a050-3484-c2ef-8d34-2c43357d01e8@oracle.com>
Message-ID: <02aa2ca3-3421-44c8-8ae3-8ab17121dda3@oracle.com>

On 2020-03-20 14:19, Stefan Karlsson wrote:
> HotSpot changes look good.

Can I have a second hotspot reviewer on this, or a confirmation that 
it's considered trivial?

/Magnus
>
> StefanK
>
> On 2020-03-20 14:15, Magnus Ihse Bursie wrote:
>> Can I get some hotspot reviewers on this as well? And is this 
>> trivial, from the Hotspot PoV?
>>
>> /Magnus
>>
>> On 2020-03-20 14:05, Erik Joelsson wrote:
>>> Looks good!
>>>
>>> /Erik
>>>
>>> On 2020-03-20 03:58, Magnus Ihse Bursie wrote:
>>>> On 2020-03-19 19:01, Erik Joelsson wrote:
>>>>> On 2020-03-19 10:20, Magnus Ihse Bursie wrote:
>>>>>> Currently, it is not possible two make hotspot builds 
>>>>>> reproducible, since the current date and time is always inserted 
>>>>>> using the special macros __DATE__ and __TIME__. This patch makes 
>>>>>> it possible to override these values from the build system, 
>>>>>> paving the way for making re-builds of hotspot bit-by-bit identical.
>>>>>>
>>>>> I like the ability to achieve this.
>>>>>> On linux/gcc, we can now actually get bit-by-bit identical builds 
>>>>>> of libjvm.so (if not using precompiled headers), so I've updated 
>>>>>> the baseline-cmp profiles and compare script to reflect this.
>>>>>>
>>>>> I'm not sure this is a good idea though. It means I can't 
>>>>> reasonably compare to validate build changes that affect the 
>>>>> precompiled header. Similarly, the COMPARE_BUILD make variable 
>>>>> argument will not work unless you use both of these configure 
>>>>> arguments.
>>>>>
>>>>> Since compare.sh is generated through configure, perhaps these 
>>>>> exceptions can be made conditional on if these configure arguments 
>>>>> are setup correctly for a reproducible libjvm.so?
>>>>
>>>> Good thinking! I did not think of this possibility. And yes, I too 
>>>> did not like the current solution, but could think of no way around 
>>>> it. Here's an improved webrev.
>>>>
>>>> WebRev: 
>>>> http://cr.openjdk.java.net/~ihse/JDK-8241271-make-hotspot-reproducible/webrev.02
>>>>
>>>> Technically, you can make this fail by setting HOTSPOT_BUILD_TIME 
>>>> but to different values, but hey, there's no need to make this 
>>>> idiot-proof.
>>>>
>>>> /Magnus
>>>>>
>>>>> /Erik
>>>>>
>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8241271
>>>>>> WebRev: 
>>>>>> http://cr.openjdk.java.net/~ihse/JDK-8241271-make-hotspot-reproducible/webrev.01
>>>>>>
>>>>>> /Magnus
>>>>
>>
>


From jesper.wilhelmsson at oracle.com  Mon Mar 23 08:48:16 2020
From: jesper.wilhelmsson at oracle.com (Jesper Wilhelmsson)
Date: Mon, 23 Mar 2020 09:48:16 +0100
Subject: RFR: JDK-8241271 Make hotspot build reproducible
In-Reply-To: <02aa2ca3-3421-44c8-8ae3-8ab17121dda3@oracle.com>
References: <b5aaadfc-99d5-9d65-313e-980efe071bf1@oracle.com>
 <3e4ebab4-c31a-99b2-8062-6378443874cf@oracle.com>
 <3d12f4e4-fab5-5dda-993c-a9fa94ca1288@oracle.com>
 <0b9acd8e-bda8-d5bd-d1ef-0b46a17b6aa0@oracle.com>
 <663deddf-03d4-d982-08fe-427e974c111f@oracle.com>
 <cab4a050-3484-c2ef-8d34-2c43357d01e8@oracle.com>
 <02aa2ca3-3421-44c8-8ae3-8ab17121dda3@oracle.com>
Message-ID: <9883ED20-6868-46D8-AD62-22AB2D0BF63C@oracle.com>

Hotspot change looks good, and trivial.
/Jesper

> On 23 Mar 2020, at 09:45, Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com> wrote:
> 
> On 2020-03-20 14:19, Stefan Karlsson wrote:
>> HotSpot changes look good.
> 
> Can I have a second hotspot reviewer on this, or a confirmation that it's considered trivial?
> 
> /Magnus
>> 
>> StefanK
>> 
>> On 2020-03-20 14:15, Magnus Ihse Bursie wrote:
>>> Can I get some hotspot reviewers on this as well? And is this trivial, from the Hotspot PoV?
>>> 
>>> /Magnus
>>> 
>>> On 2020-03-20 14:05, Erik Joelsson wrote:
>>>> Looks good!
>>>> 
>>>> /Erik
>>>> 
>>>> On 2020-03-20 03:58, Magnus Ihse Bursie wrote:
>>>>> On 2020-03-19 19:01, Erik Joelsson wrote:
>>>>>> On 2020-03-19 10:20, Magnus Ihse Bursie wrote:
>>>>>>> Currently, it is not possible two make hotspot builds reproducible, since the current date and time is always inserted using the special macros __DATE__ and __TIME__. This patch makes it possible to override these values from the build system, paving the way for making re-builds of hotspot bit-by-bit identical.
>>>>>>> 
>>>>>> I like the ability to achieve this.
>>>>>>> On linux/gcc, we can now actually get bit-by-bit identical builds of libjvm.so (if not using precompiled headers), so I've updated the baseline-cmp profiles and compare script to reflect this.
>>>>>>> 
>>>>>> I'm not sure this is a good idea though. It means I can't reasonably compare to validate build changes that affect the precompiled header. Similarly, the COMPARE_BUILD make variable argument will not work unless you use both of these configure arguments.
>>>>>> 
>>>>>> Since compare.sh is generated through configure, perhaps these exceptions can be made conditional on if these configure arguments are setup correctly for a reproducible libjvm.so?
>>>>> 
>>>>> Good thinking! I did not think of this possibility. And yes, I too did not like the current solution, but could think of no way around it. Here's an improved webrev.
>>>>> 
>>>>> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8241271-make-hotspot-reproducible/webrev.02
>>>>> 
>>>>> Technically, you can make this fail by setting HOTSPOT_BUILD_TIME but to different values, but hey, there's no need to make this idiot-proof.
>>>>> 
>>>>> /Magnus
>>>>>> 
>>>>>> /Erik
>>>>>> 
>>>>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8241271
>>>>>>> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8241271-make-hotspot-reproducible/webrev.01
>>>>>>> 
>>>>>>> /Magnus
>>>>> 
>>> 
>> 
> 


From jiefu at tencent.com  Mon Mar 23 15:58:03 2020
From: jiefu at tencent.com (=?utf-8?B?amllZnUo5YKF5p2wKQ==?=)
Date: Mon, 23 Mar 2020 15:58:03 +0000
Subject: RFR: 8241423: NUMA APIs may fail to work in the docker due to
 operation not permitted
Message-ID: <8F3CA0ED-1051-4BF4-B61B-C91856878992@tencent.com>

Hi all,

JBS:    https://bugs.openjdk.java.net/browse/JDK-8241423
Webrev: http://cr.openjdk.java.net/~jiefu/8241423/webrev.00/

A VM fatal error may be observed if ZGC is used (see JDK-8241354).
The background is that some of our products run in the docker.
And for safety reasons, SYS_get_mempolicy is not allowed by default [1].

At first, we thought it just a zgc-only problem and filed JDK-8241354.
But Thomas had reminded me that other collectors are also affected [2].
So it would be better to fix them together.

After more investigation, we found that NUMA APIs are actually dependent on several syscalls, such as get_mempolicy, mbind and set_mempolicy.
When the required syscalls are unavailable, NUMA APIs fail to work as expected.

The fix is to check whether the required syscalls are available.
In theory, all NUMA-related syscalls should be checked.
But it seems hard to do so because some of them will cause unexpected side effect.
To fix our issue, checking get_mempolicy is enough.
And just as Per suggested that we can refine this later if it turns out to be a problem [3].

Please review it and give me some advice.

Thanks a lot.
Best regards,
Jie

[1] https://docs.docker.com/engine/security/seccomp/
[2] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028923.html
[3] https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2020-March/028933.html

From boris.ulasevich at bell-sw.com  Tue Mar 24 17:01:38 2020
From: boris.ulasevich at bell-sw.com (Boris Ulasevich)
Date: Tue, 24 Mar 2020 20:01:38 +0300
Subject: RFR(L): Remove global safepoint code
In-Reply-To: <308290ab-0cf7-7a93-706f-94101597322f@oracle.com>
References: <308290ab-0cf7-7a93-706f-94101597322f@oracle.com>
Message-ID: <CAOhyNwBKshOuGFRt+3A2V-bjQ8oSCZg-6cgKAGjgBzdvQ2B__w@mail.gmail.com>

Hi Robbin,

Sorry for the long delay.
ARM32 code is good and works Ok! :)

thanks,
Boris


On Thu, Mar 19, 2020 at 3:46 PM Robbin Ehn <robbin.ehn at oracle.com> wrote:

> Hi all, please build, test and review.
>
> We removed (obsoleted) the flag ThreadLocalHandshakes in JDK 14.
> Any platform defining THREAD_LOCAL_POLL can only use per thread poll.
> Since all platforms now define THREAD_LOCAL_POLL the global poll code is
> dead.
> Let's remove it!
>
> Issue: https://bugs.openjdk.java.net/browse/JDK-8220051
> Changeset: http://cr.openjdk.java.net/~rehn/8220051/v1/webrev/index.html
>
> I have built fastdebug:
> linux-aarch64
> linux-arm32
> linux-ppc64le
> linux-s390x
> linux-x86_64-zero
> linux-x86_64 (no PCH)
> linux-x86_64 (slowdebug) (no PCH)
> linux-x86
> osx-x86_64
> windows-x86_64
> solaris-sparcv9
>
> Tested t1-5:
> linux-x86_64
> osx-x86_64
> windows-x86_64
> solaris-sparcv9
>
> Manually:
> linux-aarch64
> linux-x86_64-zero
>
> Any extra help building and testing is appreciated, especially on arm32,
> ppc64le, s390x.
>
> Thanks, Robbin
>

From robbin.ehn at oracle.com  Wed Mar 25 07:50:46 2020
From: robbin.ehn at oracle.com (Robbin Ehn)
Date: Wed, 25 Mar 2020 08:50:46 +0100
Subject: RFR(L): Remove global safepoint code
In-Reply-To: <CAOhyNwBKshOuGFRt+3A2V-bjQ8oSCZg-6cgKAGjgBzdvQ2B__w@mail.gmail.com>
References: <308290ab-0cf7-7a93-706f-94101597322f@oracle.com>
 <CAOhyNwBKshOuGFRt+3A2V-bjQ8oSCZg-6cgKAGjgBzdvQ2B__w@mail.gmail.com>
Message-ID: <9e098736-757c-32e5-726c-50d020b87247@oracle.com>

Thanks Boris!

/Robbin

On 2020-03-24 18:01, Boris Ulasevich wrote:
> Hi Robbin,
> 
> Sorry for the long delay.
> ARM32 code is good and works Ok! :)
> 
> thanks,
> Boris
> 
> 
> On Thu, Mar 19, 2020 at 3:46 PM Robbin Ehn <robbin.ehn at oracle.com 
> <mailto:robbin.ehn at oracle.com>> wrote:
> 
>     Hi all, please build, test and review.
> 
>     We removed (obsoleted) the flag ThreadLocalHandshakes in JDK 14.
>     Any platform defining THREAD_LOCAL_POLL can only use per thread poll.
>     Since all platforms now define THREAD_LOCAL_POLL the global poll
>     code is
>     dead.
>     Let's remove it!
> 
>     Issue: https://bugs.openjdk.java.net/browse/JDK-8220051
>     Changeset: http://cr.openjdk.java.net/~rehn/8220051/v1/webrev/index.html
> 
>     I have built fastdebug:
>     linux-aarch64
>     linux-arm32
>     linux-ppc64le
>     linux-s390x
>     linux-x86_64-zero
>     linux-x86_64 (no PCH)
>     linux-x86_64 (slowdebug) (no PCH)
>     linux-x86
>     osx-x86_64
>     windows-x86_64
>     solaris-sparcv9
> 
>     Tested t1-5:
>     linux-x86_64
>     osx-x86_64
>     windows-x86_64
>     solaris-sparcv9
> 
>     Manually:
>     linux-aarch64
>     linux-x86_64-zero
> 
>     Any extra help building and testing is appreciated, especially on
>     arm32,
>     ppc64le, s390x.
> 
>     Thanks, Robbin
> 

From matthias.baesken at sap.com  Wed Mar 25 12:32:37 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 25 Mar 2020 12:32:37 +0000
Subject: RFR [XS]: 8241586:
 compiler/cpuflags/TestAESIntrinsicsOnUnsupportedConfig.java fails on aarch64
Message-ID: <AM0PR02MB422759E9E0D876A4424A580593CE0@AM0PR02MB4227.eurprd02.prod.outlook.com>

Hello, please review this small  aarch64 related fix.

Currently the hotspot jtreg test compiler/cpuflags/TestAESIntrinsicsOnUnsupportedConfig.java
fails on aarch64.
Reason is that in this test, we check the following  message (that is printed on other CPU platforms):

51 private static final String AES_NOT_AVAILABLE_MSG = "warning: AES "
52 + "instructions are not available on this CPU";

However the aarch64 message differs and prints instead :
warning: UseAES specified, but not supported on this CPU

Same with the 'warning: AES intrinsics are not available on this CPU' that is checked by the test as well.
(gives currently on aarch64 the warning: UseAESIntrinsics specified, but not supported on this CPU)

So I think the aarch64 messages might be adjusted.


Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8241586

http://cr.openjdk.java.net/~mbaesken/webrevs/8241586.0/

Thanks, Matthias


From vladimir.x.ivanov at oracle.com  Wed Mar 25 12:50:37 2020
From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov)
Date: Wed, 25 Mar 2020 15:50:37 +0300
Subject: [15] RFR (S): x86: Fix Assembler::emit_operand asserts for XMM
 registers
Message-ID: <aa20aba9-6990-16ad-4c41-251868496ec0@oracle.com>

http://cr.openjdk.java.net/~vlivanov/8241434/webrev.00/
https://bugs.openjdk.java.net/browse/JDK-8241434

Assembler::emit_operand() on x86 works with Registers, but is used to 
encode both GP registers (Register) and XMM registers (XMMRegisters).

It can cause problems since some asserts assume the arguments are 
Registers (e.g, assert(index != rsp)) and may fail when XMMRegister with 
the same encoding is passed.

The fix refactors operand encoding logic into 
Assembler::emit_operand_helper() which works with register encodings and 
it is called from overloaded Assembler::emit_operand() where proper 
asserts are put.

Also, as a cleanup, made Assembler::emit_operand32() and 
Assembler::emit_farith() available only on x86-32.

Testing: tier1-4

Thanks!

Best regards,
Vladimir Ivanov

From claes.redestad at oracle.com  Wed Mar 25 13:19:41 2020
From: claes.redestad at oracle.com (Claes Redestad)
Date: Wed, 25 Mar 2020 14:19:41 +0100
Subject: [15] RFR (S): x86: Fix Assembler::emit_operand asserts for XMM
 registers
In-Reply-To: <aa20aba9-6990-16ad-4c41-251868496ec0@oracle.com>
References: <aa20aba9-6990-16ad-4c41-251868496ec0@oracle.com>
Message-ID: <81376aa5-44f2-25f6-2eb8-25282a5b2b51@oracle.com>



On 2020-03-25 13:50, Vladimir Ivanov wrote:
> http://cr.openjdk.java.net/~vlivanov/8241434/webrev.00/

Good!

/Claes

From vladimir.x.ivanov at oracle.com  Wed Mar 25 14:00:13 2020
From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov)
Date: Wed, 25 Mar 2020 17:00:13 +0300
Subject: [15] RFR (S): x86: Remove MMX support
Message-ID: <117e5b79-f962-b2e7-82a8-184e5cd1456b@oracle.com>

http://cr.openjdk.java.net/~vlivanov/8241597/webrev.00/
https://bugs.openjdk.java.net/browse/JDK-8241597

MMX is not used in x86-64 code.

Instead of making relevant code 32-bit only, I propose to remove all 
usages from both x86-64 and x86-32 specific code. (The only piece left 
is Assembler::emms() which is used in MacroAssembler::empty_FPU_stack().)

Also, got rid of VM_Version::supports_mmx_ext() along the way.

Testing: hs-tier1,hs-tier2, successfully built x86-32 (just a build, no 
additional testing performed)

Thanks!

Best regards,
Vladimir Ivanov

From vladimir.x.ivanov at oracle.com  Wed Mar 25 14:11:25 2020
From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov)
Date: Wed, 25 Mar 2020 17:11:25 +0300
Subject: [15] RFR (S): 8241597: x86: Remove MMX support
In-Reply-To: <117e5b79-f962-b2e7-82a8-184e5cd1456b@oracle.com>
References: <117e5b79-f962-b2e7-82a8-184e5cd1456b@oracle.com>
Message-ID: <9400a7ca-c18c-75d2-bb1b-99edbd93569d@oracle.com>

Forgot to include bug id in the subject. Fixed.

Best regards,
Vladimir Ivanov

On 25.03.2020 17:00, Vladimir Ivanov wrote:
> http://cr.openjdk.java.net/~vlivanov/8241597/webrev.00/
> https://bugs.openjdk.java.net/browse/JDK-8241597
> 
> MMX is not used in x86-64 code.
> 
> Instead of making relevant code 32-bit only, I propose to remove all 
> usages from both x86-64 and x86-32 specific code. (The only piece left 
> is Assembler::emms() which is used in MacroAssembler::empty_FPU_stack().)
> 
> Also, got rid of VM_Version::supports_mmx_ext() along the way.
> 
> Testing: hs-tier1,hs-tier2, successfully built x86-32 (just a build, no 
> additional testing performed)
> 
> Thanks!
> 
> Best regards,
> Vladimir Ivanov

From vladimir.x.ivanov at oracle.com  Wed Mar 25 14:11:59 2020
From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov)
Date: Wed, 25 Mar 2020 17:11:59 +0300
Subject: [15] RFR (S): 8241434: x86: Fix Assembler::emit_operand asserts
 for XMM registers
In-Reply-To: <aa20aba9-6990-16ad-4c41-251868496ec0@oracle.com>
References: <aa20aba9-6990-16ad-4c41-251868496ec0@oracle.com>
Message-ID: <16b1a988-4113-d2e2-e3b2-46812dc3257f@oracle.com>

Forgot to include bug id in the subject. Fixed.

Best regards,
Vladimir Ivanov

On 25.03.2020 15:50, Vladimir Ivanov wrote:
> http://cr.openjdk.java.net/~vlivanov/8241434/webrev.00/
> https://bugs.openjdk.java.net/browse/JDK-8241434
> 
> Assembler::emit_operand() on x86 works with Registers, but is used to 
> encode both GP registers (Register) and XMM registers (XMMRegisters).
> 
> It can cause problems since some asserts assume the arguments are 
> Registers (e.g, assert(index != rsp)) and may fail when XMMRegister with 
> the same encoding is passed.
> 
> The fix refactors operand encoding logic into 
> Assembler::emit_operand_helper() which works with register encodings and 
> it is called from overloaded Assembler::emit_operand() where proper 
> asserts are put.
> 
> Also, as a cleanup, made Assembler::emit_operand32() and 
> Assembler::emit_farith() available only on x86-32.
> 
> Testing: tier1-4
> 
> Thanks!
> 
> Best regards,
> Vladimir Ivanov

From vladimir.x.ivanov at oracle.com  Wed Mar 25 14:12:24 2020
From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov)
Date: Wed, 25 Mar 2020 17:12:24 +0300
Subject: [15] RFR (S): x86: Fix Assembler::emit_operand asserts for XMM
 registers
In-Reply-To: <81376aa5-44f2-25f6-2eb8-25282a5b2b51@oracle.com>
References: <aa20aba9-6990-16ad-4c41-251868496ec0@oracle.com>
 <81376aa5-44f2-25f6-2eb8-25282a5b2b51@oracle.com>
Message-ID: <64ad5c71-3849-8364-76e3-4fc9e45e69f5@oracle.com>

Thanks, Claes.

Best regards,
Vladimir Ivanov

On 25.03.2020 16:19, Claes Redestad wrote:
> 
> 
> On 2020-03-25 13:50, Vladimir Ivanov wrote:
>> http://cr.openjdk.java.net/~vlivanov/8241434/webrev.00/
> 
> Good!
> 
> /Claes

From claes.redestad at oracle.com  Wed Mar 25 14:21:18 2020
From: claes.redestad at oracle.com (Claes Redestad)
Date: Wed, 25 Mar 2020 15:21:18 +0100
Subject: [15] RFR (S): 8241597: x86: Remove MMX support
In-Reply-To: <9400a7ca-c18c-75d2-bb1b-99edbd93569d@oracle.com>
References: <117e5b79-f962-b2e7-82a8-184e5cd1456b@oracle.com>
 <9400a7ca-c18c-75d2-bb1b-99edbd93569d@oracle.com>
Message-ID: <e7689500-ff79-9427-e996-3c882cb6852b@oracle.com>

Looks good!

/Claes

On 2020-03-25 15:11, Vladimir Ivanov wrote:
> Forgot to include bug id in the subject. Fixed.
> 
> Best regards,
> Vladimir Ivanov
> 
> On 25.03.2020 17:00, Vladimir Ivanov wrote:
>> http://cr.openjdk.java.net/~vlivanov/8241597/webrev.00/
>> https://bugs.openjdk.java.net/browse/JDK-8241597
>>
>> MMX is not used in x86-64 code.
>>
>> Instead of making relevant code 32-bit only, I propose to remove all 
>> usages from both x86-64 and x86-32 specific code. (The only piece left 
>> is Assembler::emms() which is used in MacroAssembler::empty_FPU_stack().)
>>
>> Also, got rid of VM_Version::supports_mmx_ext() along the way.
>>
>> Testing: hs-tier1,hs-tier2, successfully built x86-32 (just a build, 
>> no additional testing performed)
>>
>> Thanks!
>>
>> Best regards,
>> Vladimir Ivanov

From vladimir.kozlov at oracle.com  Wed Mar 25 17:15:38 2020
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Wed, 25 Mar 2020 10:15:38 -0700
Subject: [15] RFR (S): 8241434: x86: Fix Assembler::emit_operand asserts
 for XMM registers
In-Reply-To: <16b1a988-4113-d2e2-e3b2-46812dc3257f@oracle.com>
References: <aa20aba9-6990-16ad-4c41-251868496ec0@oracle.com>
 <16b1a988-4113-d2e2-e3b2-46812dc3257f@oracle.com>
Message-ID: <03f524c8-ed38-504a-4fd9-2eacbda46df9@oracle.com>

Looks good.

Thanks,
Vladimir

On 3/25/20 7:11 AM, Vladimir Ivanov wrote:
> Forgot to include bug id in the subject. Fixed.
> 
> Best regards,
> Vladimir Ivanov
> 
> On 25.03.2020 15:50, Vladimir Ivanov wrote:
>> http://cr.openjdk.java.net/~vlivanov/8241434/webrev.00/
>> https://bugs.openjdk.java.net/browse/JDK-8241434
>>
>> Assembler::emit_operand() on x86 works with Registers, but is used to encode both GP registers (Register) and XMM 
>> registers (XMMRegisters).
>>
>> It can cause problems since some asserts assume the arguments are Registers (e.g, assert(index != rsp)) and may fail 
>> when XMMRegister with the same encoding is passed.
>>
>> The fix refactors operand encoding logic into Assembler::emit_operand_helper() which works with register encodings and 
>> it is called from overloaded Assembler::emit_operand() where proper asserts are put.
>>
>> Also, as a cleanup, made Assembler::emit_operand32() and Assembler::emit_farith() available only on x86-32.
>>
>> Testing: tier1-4
>>
>> Thanks!
>>
>> Best regards,
>> Vladimir Ivanov

From vladimir.kozlov at oracle.com  Wed Mar 25 17:20:54 2020
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Wed, 25 Mar 2020 10:20:54 -0700
Subject: [15] RFR (S): 8241597: x86: Remove MMX support
In-Reply-To: <9400a7ca-c18c-75d2-bb1b-99edbd93569d@oracle.com>
References: <117e5b79-f962-b2e7-82a8-184e5cd1456b@oracle.com>
 <9400a7ca-c18c-75d2-bb1b-99edbd93569d@oracle.com>
Message-ID: <022cf603-980f-4f22-bd7d-bdff461aa92e@oracle.com>

Good.

Thanks,
Vladimir

On 3/25/20 7:11 AM, Vladimir Ivanov wrote:
> Forgot to include bug id in the subject. Fixed.
> 
> Best regards,
> Vladimir Ivanov
> 
> On 25.03.2020 17:00, Vladimir Ivanov wrote:
>> http://cr.openjdk.java.net/~vlivanov/8241597/webrev.00/
>> https://bugs.openjdk.java.net/browse/JDK-8241597
>>
>> MMX is not used in x86-64 code.
>>
>> Instead of making relevant code 32-bit only, I propose to remove all usages from both x86-64 and x86-32 specific code. 
>> (The only piece left is Assembler::emms() which is used in MacroAssembler::empty_FPU_stack().)
>>
>> Also, got rid of VM_Version::supports_mmx_ext() along the way.
>>
>> Testing: hs-tier1,hs-tier2, successfully built x86-32 (just a build, no additional testing performed)
>>
>> Thanks!
>>
>> Best regards,
>> Vladimir Ivanov

From matthias.baesken at sap.com  Thu Mar 26 09:56:28 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Thu, 26 Mar 2020 09:56:28 +0000
Subject: RFR [XXS]: 8241660: Add virtualization information output to hs_err
 file on macOS
Message-ID: <AM0PR02MB4227C4FB9E67C7F5EA96F0DB93CF0@AM0PR02MB4227.eurprd02.prod.outlook.com>

Hello , please review this very small enhancement  to the hs_err file output on macOS .

Currently we miss the output of virtualization information in the hs_err file on macOS, although we collect the info already in early JVM startup.

This patch adds the information.

Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8241660

http://cr.openjdk.java.net/~mbaesken/webrevs/8241660.0/

Thanks, Matthias


From vladimir.x.ivanov at oracle.com  Thu Mar 26 18:46:19 2020
From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov)
Date: Thu, 26 Mar 2020 21:46:19 +0300
Subject: [15] RFR (S): 8241434: x86: Fix Assembler::emit_operand asserts
 for XMM registers
In-Reply-To: <03f524c8-ed38-504a-4fd9-2eacbda46df9@oracle.com>
References: <aa20aba9-6990-16ad-4c41-251868496ec0@oracle.com>
 <16b1a988-4113-d2e2-e3b2-46812dc3257f@oracle.com>
 <03f524c8-ed38-504a-4fd9-2eacbda46df9@oracle.com>
Message-ID: <3a197787-c65d-d8f9-c421-9e69ed94e525@oracle.com>

Thanks for the review, Vladimir.

Best regards,
Vladimir Ivanov

On 25.03.2020 20:15, Vladimir Kozlov wrote:
> Looks good.
> 
> Thanks,
> Vladimir
> 
> On 3/25/20 7:11 AM, Vladimir Ivanov wrote:
>> Forgot to include bug id in the subject. Fixed.
>>
>> Best regards,
>> Vladimir Ivanov
>>
>> On 25.03.2020 15:50, Vladimir Ivanov wrote:
>>> http://cr.openjdk.java.net/~vlivanov/8241434/webrev.00/
>>> https://bugs.openjdk.java.net/browse/JDK-8241434
>>>
>>> Assembler::emit_operand() on x86 works with Registers, but is used to 
>>> encode both GP registers (Register) and XMM registers (XMMRegisters).
>>>
>>> It can cause problems since some asserts assume the arguments are 
>>> Registers (e.g, assert(index != rsp)) and may fail when XMMRegister 
>>> with the same encoding is passed.
>>>
>>> The fix refactors operand encoding logic into 
>>> Assembler::emit_operand_helper() which works with register encodings 
>>> and it is called from overloaded Assembler::emit_operand() where 
>>> proper asserts are put.
>>>
>>> Also, as a cleanup, made Assembler::emit_operand32() and 
>>> Assembler::emit_farith() available only on x86-32.
>>>
>>> Testing: tier1-4
>>>
>>> Thanks!
>>>
>>> Best regards,
>>> Vladimir Ivanov

From vladimir.x.ivanov at oracle.com  Thu Mar 26 18:47:18 2020
From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov)
Date: Thu, 26 Mar 2020 21:47:18 +0300
Subject: [15] RFR (S): 8241597: x86: Remove MMX support
In-Reply-To: <022cf603-980f-4f22-bd7d-bdff461aa92e@oracle.com>
References: <117e5b79-f962-b2e7-82a8-184e5cd1456b@oracle.com>
 <9400a7ca-c18c-75d2-bb1b-99edbd93569d@oracle.com>
 <022cf603-980f-4f22-bd7d-bdff461aa92e@oracle.com>
Message-ID: <0231e0f8-f009-091b-1640-e9d753707d93@oracle.com>

Thanks for the reviews, Vladimir & Claes.

Best regards,
Vladimir Ivanov

On 25.03.2020 20:20, Vladimir Kozlov wrote:
> Good.
> 
> Thanks,
> Vladimir
> 
> On 3/25/20 7:11 AM, Vladimir Ivanov wrote:
>> Forgot to include bug id in the subject. Fixed.
>>
>> Best regards,
>> Vladimir Ivanov
>>
>> On 25.03.2020 17:00, Vladimir Ivanov wrote:
>>> http://cr.openjdk.java.net/~vlivanov/8241597/webrev.00/
>>> https://bugs.openjdk.java.net/browse/JDK-8241597
>>>
>>> MMX is not used in x86-64 code.
>>>
>>> Instead of making relevant code 32-bit only, I propose to remove all 
>>> usages from both x86-64 and x86-32 specific code. (The only piece 
>>> left is Assembler::emms() which is used in 
>>> MacroAssembler::empty_FPU_stack().)
>>>
>>> Also, got rid of VM_Version::supports_mmx_ext() along the way.
>>>
>>> Testing: hs-tier1,hs-tier2, successfully built x86-32 (just a build, 
>>> no additional testing performed)
>>>
>>> Thanks!
>>>
>>> Best regards,
>>> Vladimir Ivanov

From mandy.chung at oracle.com  Thu Mar 26 23:57:39 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Thu, 26 Mar 2020 16:57:39 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
Message-ID: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>

Please review the implementation of JEP 371: Hidden Classes. The main 
changes are in core-libs and hotspot runtime area.? Small changes are 
made in javac, VM compiler (intrinsification of Class::isHiddenClass), 
JFR, JDI, and jcmd.? CSR [1]has been reviewed and is in the finalized 
state (see specdiff and javadoc below for reference).

Webrev:
http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03

Hidden class is created via `Lookup::defineHiddenClass`. From JVM's point
of view, a hidden class is a normal class except the following:

- A hidden class has no initiating class loader and is not registered in 
any dictionary.
- A hidden class has a name containing an illegal character 
`Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature` 
returns "Lp/Foo.0x1234;".
- A hidden class is not modifiable, i.e. cannot be redefined or 
retransformed. JVM TI IsModifableClass returns false on a hidden.
- Final fields in a hidden class is "final".? The value of final fields 
cannot be overriden via reflection.? setAccessible(true) can still be 
called on reflected objects representing final fields in a hidden class 
and its access check will be suppressed but only have read-access (i.e. 
can do Field::getXXX but not setXXX).

Brief summary of this patch:

1. A new Lookup::defineHiddenClass method is the API to create a hidden 
class.
2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG 
option that
 ?? can be specified when creating a hidden class.
3. A new Class::isHiddenClass method tests if a class is a hidden class.
4. Field::setXXX method will throw IAE on a final field of a hidden class
 ?? regardless of the value of the accessible flag.
5. JVM_LookupDefineClass is the new JVM entry point for Lookup::defineClass
 ?? and defineHiddenClass to create a class from the given bytes.
6. ClassLoaderData implementation is not changed.? There is one primary CLD
 ?? that holds the classes strongly referenced by its defining loader.? 
There
 ?? can be zero or more additional CLDs - one per weak class.
7. Nest host determination is updated per revised JVMS 5.4.4. Access control
 ?? check no longer throws LinkageError but instead it will throw IAE with
 ?? a clear message if a class fails to resolve/validate the nest host 
declared
 ?? in NestHost/NestMembers attribute.
8. JFR, jcmd, JDI are updated to support hidden classes.
9. update javac LambdaToMethod as lambda proxy starts using nestmates
 ?? and generate a bridge method to desuger a method reference to a 
protected
 ?? method in its supertype in a different package

This patch also updates StringConcatFactory, LambdaMetaFactory, and 
LambdaForms
to use hidden classes.? The webrev includes changes in nashorn to hidden 
class
and I will update the webrev if JEP 372 removes it any time soon.

We uncovered a bug in Lookup::defineClass spec throws LinkageError and 
intends
to have the newly created class linked.? However, the implementation in 14
does not link the class.? A separate CSR [2] proposes to update the
implementation to match the spec.? This patch fixes the implementation.

The spec update on JVM TI, JDI and Instrumentation will be done as
a separate RFE [3].? This patch includes new tests for JVM TI and
java.instrument that validates how the existing APIs work for hidden 
classes.

javadoc/specdiff
http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/

JVMS 5.4.4 change:
http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf

CSR:
https://bugs.openjdk.java.net/browse/JDK-8238359

Thanks
Mandy
[1] https://bugs.openjdk.java.net/browse/JDK-8238359
[2] https://bugs.openjdk.java.net/browse/JDK-8240338
[3] https://bugs.openjdk.java.net/browse/JDK-8230502

From thomas.stuefe at gmail.com  Fri Mar 27 07:34:12 2020
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Fri, 27 Mar 2020 08:34:12 +0100
Subject: RFR[T, urgent]: 8241723: Build error after 8241581
In-Reply-To: <CAA-vtUy7E3uMm+Q0Jrf-Z4fPoig_hH8HEMC+e5aXkrePZDVvYg@mail.gmail.com>
References: <CAA-vtUy7E3uMm+Q0Jrf-Z4fPoig_hH8HEMC+e5aXkrePZDVvYg@mail.gmail.com>
Message-ID: <CAA-vtUyuCW2QPUC18Hk8t1C14q=211g8XCp=r-z+9AQD82afRw@mail.gmail.com>

Crosspost to hs-dev.

On Fri, Mar 27, 2020 at 8:25 AM Thomas St?fe <thomas.stuefe at gmail.com>
wrote:

> Hi,
>
> not my proudest day.
>
> <https://bugs.openjdk.java.net/browse/JDK-8241723>
> Gtest for 8241581 got added accidentally after running the tests but
> before the push. Immediate result is an escaped build error in release:
>
> /shared/projects/openjdk/jdk-jdk/source/test/hotspot/gtest/utilities/test_bitMap_popcnt.cpp:
> In destructor 'ErrorDetailsPrinter::~ErrorDetailsPrinter()':
> /shared/projects/openjdk/jdk-jdk/source/test/hotspot/gtest/utilities/test_bitMap_popcnt.cpp:96:11:
> error: 'class BitMap' has no member named 'print_on'
>
> but since there may be other errors in this gtest as well, I'd like to
> remove the test, re-run all tests, then re-add it.
>
> The tested functionality is still unused, so we need not to roll-back the
> whole patch.
>
> https://bugs.openjdk.java.net/browse/JDK-8241723
>
> Fix consists of "hg
> remove test/hotspot/gtest/utilities/test_bitMap_popcnt.cpp", so there is no
> webrev.
>
> Thanks, Thomas
>
>

From christoph.langer at sap.com  Fri Mar 27 07:43:27 2020
From: christoph.langer at sap.com (Langer, Christoph)
Date: Fri, 27 Mar 2020 07:43:27 +0000
Subject: RFR[T, urgent]: 8241723: Build error after 8241581
In-Reply-To: <CAA-vtUyuCW2QPUC18Hk8t1C14q=211g8XCp=r-z+9AQD82afRw@mail.gmail.com>
References: <CAA-vtUy7E3uMm+Q0Jrf-Z4fPoig_hH8HEMC+e5aXkrePZDVvYg@mail.gmail.com>
 <CAA-vtUyuCW2QPUC18Hk8t1C14q=211g8XCp=r-z+9AQD82afRw@mail.gmail.com>
Message-ID: <AM0PR02MB57149B62524B6F82AC0ADF158ACC0@AM0PR02MB5714.eurprd02.prod.outlook.com>

Hi Thomas,

looks good to me.

Best regards
Christoph

> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
> Thomas St?fe
> Sent: Freitag, 27. M?rz 2020 08:34
> To: Hotspot dev runtime <hotspot-runtime-dev at openjdk.java.net>;
> HotSpot Open Source Developers <hotspot-dev at openjdk.java.net>
> Subject: Re: RFR[T, urgent]: 8241723: Build error after 8241581
> 
> Crosspost to hs-dev.
> 
> On Fri, Mar 27, 2020 at 8:25 AM Thomas St?fe <thomas.stuefe at gmail.com>
> wrote:
> 
> > Hi,
> >
> > not my proudest day.
> >
> > <https://bugs.openjdk.java.net/browse/JDK-8241723>
> > Gtest for 8241581 got added accidentally after running the tests but
> > before the push. Immediate result is an escaped build error in release:
> >
> > /shared/projects/openjdk/jdk-
> jdk/source/test/hotspot/gtest/utilities/test_bitMap_popcnt.cpp:
> > In destructor 'ErrorDetailsPrinter::~ErrorDetailsPrinter()':
> > /shared/projects/openjdk/jdk-
> jdk/source/test/hotspot/gtest/utilities/test_bitMap_popcnt.cpp:96:11:
> > error: 'class BitMap' has no member named 'print_on'
> >
> > but since there may be other errors in this gtest as well, I'd like to
> > remove the test, re-run all tests, then re-add it.
> >
> > The tested functionality is still unused, so we need not to roll-back the
> > whole patch.
> >
> > https://bugs.openjdk.java.net/browse/JDK-8241723
> >
> > Fix consists of "hg
> > remove test/hotspot/gtest/utilities/test_bitMap_popcnt.cpp", so there is
> no
> > webrev.
> >
> > Thanks, Thomas
> >
> >

From christoph.langer at sap.com  Fri Mar 27 09:47:20 2020
From: christoph.langer at sap.com (Langer, Christoph)
Date: Fri, 27 Mar 2020 09:47:20 +0000
Subject: RFR [XS]: 8241586:
 compiler/cpuflags/TestAESIntrinsicsOnUnsupportedConfig.java fails on aarch64
In-Reply-To: <AM0PR02MB422759E9E0D876A4424A580593CE0@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB422759E9E0D876A4424A580593CE0@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <AM0PR02MB5714662AF5D584B8EDE5A8728ACC0@AM0PR02MB5714.eurprd02.prod.outlook.com>

Hi Matthias,

this looks fine to me. The test error that we observe seems to have gone with your fix.

Cross-posting to aarch64-port-dev - maybe one of the folks involved in the aarch64 port can acknowledge (or object)?

Thanks
Christoph


From: Baesken, Matthias <matthias.baesken at sap.com>
Sent: Mittwoch, 25. M?rz 2020 13:33
To: 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
Cc: Langer, Christoph <christoph.langer at sap.com>
Subject: RFR [XS]: 8241586: compiler/cpuflags/TestAESIntrinsicsOnUnsupportedConfig.java fails on aarch64

Hello, please review this small  aarch64 related fix.

Currently the hotspot jtreg test compiler/cpuflags/TestAESIntrinsicsOnUnsupportedConfig.java
fails on aarch64.
Reason is that in this test, we check the following  message (that is printed on other CPU platforms):

51 private static final String AES_NOT_AVAILABLE_MSG = "warning: AES "
52 + "instructions are not available on this CPU";

However the aarch64 message differs and prints instead :
warning: UseAES specified, but not supported on this CPU

Same with the 'warning: AES intrinsics are not available on this CPU' that is checked by the test as well.
(gives currently on aarch64 the warning: UseAESIntrinsics specified, but not supported on this CPU)

So I think the aarch64 messages might be adjusted.


Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8241586

http://cr.openjdk.java.net/~mbaesken/webrevs/8241586.0/

Thanks, Matthias


From christoph.langer at sap.com  Fri Mar 27 09:51:24 2020
From: christoph.langer at sap.com (Langer, Christoph)
Date: Fri, 27 Mar 2020 09:51:24 +0000
Subject: RFR [XXS]: 8241660: Add virtualization information output to
 hs_err file on macOS
In-Reply-To: <AM0PR02MB4227C4FB9E67C7F5EA96F0DB93CF0@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB4227C4FB9E67C7F5EA96F0DB93CF0@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <AM0PR02MB571457FCEF4F2198C0FDF8018ACC0@AM0PR02MB5714.eurprd02.prod.outlook.com>

Hi Matthias,

this looks good to me.

Best regards
Christoph


> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
> Baesken, Matthias
> Sent: Donnerstag, 26. M?rz 2020 10:56
> To: 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: RFR [XXS]: 8241660: Add virtualization information output to hs_err
> file on macOS
> 
> Hello , please review this very small enhancement  to the hs_err file output
> on macOS .
> 
> Currently we miss the output of virtualization information in the hs_err file
> on macOS, although we collect the info already in early JVM startup.
> 
> This patch adds the information.
> 
> Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8241660
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8241660.0/
> 
> Thanks, Matthias


From matthias.baesken at sap.com  Fri Mar 27 10:01:43 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 27 Mar 2020 10:01:43 +0000
Subject: RFR [XS]: 8241586:
 compiler/cpuflags/TestAESIntrinsicsOnUnsupportedConfig.java fails on aarch64
In-Reply-To: <AM0PR02MB5714662AF5D584B8EDE5A8728ACC0@AM0PR02MB5714.eurprd02.prod.outlook.com>
References: <AM0PR02MB422759E9E0D876A4424A580593CE0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <AM0PR02MB5714662AF5D584B8EDE5A8728ACC0@AM0PR02MB5714.eurprd02.prod.outlook.com>
Message-ID: <AM0PR02MB4227D1F12C531AB2F31FB20293CC0@AM0PR02MB4227.eurprd02.prod.outlook.com>

Thanks for the review !

Best regards, Matthias

From: Langer, Christoph <christoph.langer at sap.com>
Sent: Freitag, 27. M?rz 2020 10:47
To: Baesken, Matthias <matthias.baesken at sap.com>; 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
Cc: aarch64-port-dev at openjdk.java.net
Subject: RE: RFR [XS]: 8241586: compiler/cpuflags/TestAESIntrinsicsOnUnsupportedConfig.java fails on aarch64

Hi Matthias,

this looks fine to me. The test error that we observe seems to have gone with your fix.

Cross-posting to aarch64-port-dev - maybe one of the folks involved in the aarch64 port can acknowledge (or object)?

Thanks
Christoph


From: Baesken, Matthias <matthias.baesken at sap.com<mailto:matthias.baesken at sap.com>>
Sent: Mittwoch, 25. M?rz 2020 13:33
To: 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net<mailto:hotspot-dev at openjdk.java.net>>
Cc: Langer, Christoph <christoph.langer at sap.com<mailto:christoph.langer at sap.com>>
Subject: RFR [XS]: 8241586: compiler/cpuflags/TestAESIntrinsicsOnUnsupportedConfig.java fails on aarch64

Hello, please review this small  aarch64 related fix.

Currently the hotspot jtreg test compiler/cpuflags/TestAESIntrinsicsOnUnsupportedConfig.java
fails on aarch64.
Reason is that in this test, we check the following  message (that is printed on other CPU platforms):

51 private static final String AES_NOT_AVAILABLE_MSG = "warning: AES "
52 + "instructions are not available on this CPU";

However the aarch64 message differs and prints instead :
warning: UseAES specified, but not supported on this CPU

Same with the 'warning: AES intrinsics are not available on this CPU' that is checked by the test as well.
(gives currently on aarch64 the warning: UseAESIntrinsics specified, but not supported on this CPU)

So I think the aarch64 messages might be adjusted.


Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8241586

http://cr.openjdk.java.net/~mbaesken/webrevs/8241586.0/

Thanks, Matthias


From matthias.baesken at sap.com  Fri Mar 27 10:54:25 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 27 Mar 2020 10:54:25 +0000
Subject: RFR [XXS]: 8241660: Add virtualization information output to
 hs_err file on macOS
In-Reply-To: <AM0PR02MB571457FCEF4F2198C0FDF8018ACC0@AM0PR02MB5714.eurprd02.prod.outlook.com>
References: <AM0PR02MB4227C4FB9E67C7F5EA96F0DB93CF0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <AM0PR02MB571457FCEF4F2198C0FDF8018ACC0@AM0PR02MB5714.eurprd02.prod.outlook.com>
Message-ID: <AM0PR02MB4227F474349B8AC8B8ACE00193CC0@AM0PR02MB4227.eurprd02.prod.outlook.com>

Thanks, may I get a second review please ?


-----Original Message-----
From: Langer, Christoph <christoph.langer at sap.com> 
Sent: Freitag, 27. M?rz 2020 10:51
To: Baesken, Matthias <matthias.baesken at sap.com>; 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
Subject: RE: RFR [XXS]: 8241660: Add virtualization information output to hs_err file on macOS

Hi Matthias,

this looks good to me.

Best regards
Christoph


> 
> Hello , please review this very small enhancement  to the hs_err file output
> on macOS .
> 
> Currently we miss the output of virtualization information in the hs_err file
> on macOS, although we collect the info already in early JVM startup.
> 
> This patch adds the information.
> 
> Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8241660
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8241660.0/
> 
> Thanks, Matthias


From martin.doerr at sap.com  Fri Mar 27 11:26:39 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Fri, 27 Mar 2020 11:26:39 +0000
Subject: RFR [XXS]: 8241660: Add virtualization information output to
 hs_err file on macOS
In-Reply-To: <AM0PR02MB4227F474349B8AC8B8ACE00193CC0@AM0PR02MB4227.eurprd02.prod.outlook.com>
References: <AM0PR02MB4227C4FB9E67C7F5EA96F0DB93CF0@AM0PR02MB4227.eurprd02.prod.outlook.com>
 <AM0PR02MB571457FCEF4F2198C0FDF8018ACC0@AM0PR02MB5714.eurprd02.prod.outlook.com>
 <AM0PR02MB4227F474349B8AC8B8ACE00193CC0@AM0PR02MB4227.eurprd02.prod.outlook.com>
Message-ID: <AM0PR0202MB3297CFAD09C8301BE94CCDB69ACC0@AM0PR0202MB3297.eurprd02.prod.outlook.com>

Hi Matthias,

looks good to me, too.

Best regards,
Martin


> -----Original Message-----
> From: Baesken, Matthias <matthias.baesken at sap.com>
> Sent: Freitag, 27. M?rz 2020 11:54
> To: Langer, Christoph <christoph.langer at sap.com>; 'hotspot-
> dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Cc: Doerr, Martin <martin.doerr at sap.com>
> Subject: RE: RFR [XXS]: 8241660: Add virtualization information output to
> hs_err file on macOS
> 
> Thanks, may I get a second review please ?
> 
> 
> -----Original Message-----
> From: Langer, Christoph <christoph.langer at sap.com>
> Sent: Freitag, 27. M?rz 2020 10:51
> To: Baesken, Matthias <matthias.baesken at sap.com>; 'hotspot-
> dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: RE: RFR [XXS]: 8241660: Add virtualization information output to
> hs_err file on macOS
> 
> Hi Matthias,
> 
> this looks good to me.
> 
> Best regards
> Christoph
> 
> 
> >
> > Hello , please review this very small enhancement  to the hs_err file output
> > on macOS .
> >
> > Currently we miss the output of virtualization information in the hs_err file
> > on macOS, although we collect the info already in early JVM startup.
> >
> > This patch adds the information.
> >
> > Bug/webrev :
> >
> > https://bugs.openjdk.java.net/browse/JDK-8241660
> >
> > http://cr.openjdk.java.net/~mbaesken/webrevs/8241660.0/
> >
> > Thanks, Matthias


From jan.lahoda at oracle.com  Fri Mar 27 11:31:33 2020
From: jan.lahoda at oracle.com (Jan Lahoda)
Date: Fri, 27 Mar 2020 12:31:33 +0100
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
Message-ID: <d3d2fa4f-4bf7-94c3-de4b-15dfe75b6370@oracle.com>

Hi Mandy,

Regarding the javac changes - should those be switched on/off depending 
the Target? Or, if one compiles with e.g. --release 14, will the newly 
generated output still work on JDK 14?

Jan

On 27. 03. 20 0:57, Mandy Chung wrote:
> Please review the implementation of JEP 371: Hidden Classes. The main 
> changes are in core-libs and hotspot runtime area.? Small changes are 
> made in javac, VM compiler (intrinsification of Class::isHiddenClass), 
> JFR, JDI, and jcmd.? CSR [1]has been reviewed and is in the finalized 
> state (see specdiff and javadoc below for reference).
> 
> Webrev:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
> 
> 
> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's point
> of view, a hidden class is a normal class except the following:
> 
> - A hidden class has no initiating class loader and is not registered in 
> any dictionary.
> - A hidden class has a name containing an illegal character 
> `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature` 
> returns "Lp/Foo.0x1234;".
> - A hidden class is not modifiable, i.e. cannot be redefined or 
> retransformed. JVM TI IsModifableClass returns false on a hidden.
> - Final fields in a hidden class is "final".? The value of final fields 
> cannot be overriden via reflection.? setAccessible(true) can still be 
> called on reflected objects representing final fields in a hidden class 
> and its access check will be suppressed but only have read-access (i.e. 
> can do Field::getXXX but not setXXX).
> 
> Brief summary of this patch:
> 
> 1. A new Lookup::defineHiddenClass method is the API to create a hidden 
> class.
> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG 
> option that
>  ?? can be specified when creating a hidden class.
> 3. A new Class::isHiddenClass method tests if a class is a hidden class.
> 4. Field::setXXX method will throw IAE on a final field of a hidden class
>  ?? regardless of the value of the accessible flag.
> 5. JVM_LookupDefineClass is the new JVM entry point for Lookup::defineClass
>  ?? and defineHiddenClass to create a class from the given bytes.
> 6. ClassLoaderData implementation is not changed.? There is one primary CLD
>  ?? that holds the classes strongly referenced by its defining loader. 
> There
>  ?? can be zero or more additional CLDs - one per weak class.
> 7. Nest host determination is updated per revised JVMS 5.4.4. Access 
> control
>  ?? check no longer throws LinkageError but instead it will throw IAE with
>  ?? a clear message if a class fails to resolve/validate the nest host 
> declared
>  ?? in NestHost/NestMembers attribute.
> 8. JFR, jcmd, JDI are updated to support hidden classes.
> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
>  ?? and generate a bridge method to desuger a method reference to a 
> protected
>  ?? method in its supertype in a different package
> 
> This patch also updates StringConcatFactory, LambdaMetaFactory, and 
> LambdaForms
> to use hidden classes.? The webrev includes changes in nashorn to hidden 
> class
> and I will update the webrev if JEP 372 removes it any time soon.
> 
> We uncovered a bug in Lookup::defineClass spec throws LinkageError and 
> intends
> to have the newly created class linked.? However, the implementation in 14
> does not link the class.? A separate CSR [2] proposes to update the
> implementation to match the spec.? This patch fixes the implementation.
> 
> The spec update on JVM TI, JDI and Instrumentation will be done as
> a separate RFE [3].? This patch includes new tests for JVM TI and
> java.instrument that validates how the existing APIs work for hidden 
> classes.
> 
> javadoc/specdiff
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
> 
> 
> JVMS 5.4.4 change:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
> 
> 
> CSR:
> https://bugs.openjdk.java.net/browse/JDK-8238359
> 
> Thanks
> Mandy
> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
> [3] https://bugs.openjdk.java.net/browse/JDK-8230502

From forax at univ-mlv.fr  Fri Mar 27 12:00:06 2020
From: forax at univ-mlv.fr (Remi Forax)
Date: Fri, 27 Mar 2020 13:00:06 +0100 (CET)
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
Message-ID: <1271059704.1214239.1585310406005.JavaMail.zimbra@u-pem.fr>

Hi Mandy,
in ReflectionFactory, why in the case of a constructor the check to the anonymous class is removed ?

in BytecodeGenerator, the comment "// bootstrapping issue if using condy"
can be promoted on top of clinit, because i ask myself the same question seeing a static block was generated

in AbstractValidatingLambdaMetafactory.java, the field caller is not used after all ?

regards,
R?mi

----- Mail original -----
> De: "mandy chung" <mandy.chung at oracle.com>
> ?: "valhalla-dev" <valhalla-dev at openjdk.java.net>, "core-libs-dev" <core-libs-dev at openjdk.java.net>,
> "serviceability-dev" <serviceability-dev at openjdk.java.net>, "hotspot-dev" <hotspot-dev at openjdk.java.net>
> Envoy?: Vendredi 27 Mars 2020 00:57:39
> Objet: Review Request: 8238358: Implementation of JEP 371: Hidden Classes

> Please review the implementation of JEP 371: Hidden Classes. The main
> changes are in core-libs and hotspot runtime area.? Small changes are
> made in javac, VM compiler (intrinsification of Class::isHiddenClass),
> JFR, JDI, and jcmd.? CSR [1]has been reviewed and is in the finalized
> state (see specdiff and javadoc below for reference).
> 
> Webrev:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03
> 
> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's point
> of view, a hidden class is a normal class except the following:
> 
> - A hidden class has no initiating class loader and is not registered in
> any dictionary.
> - A hidden class has a name containing an illegal character
> `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature`
> returns "Lp/Foo.0x1234;".
> - A hidden class is not modifiable, i.e. cannot be redefined or
> retransformed. JVM TI IsModifableClass returns false on a hidden.
> - Final fields in a hidden class is "final".? The value of final fields
> cannot be overriden via reflection.? setAccessible(true) can still be
> called on reflected objects representing final fields in a hidden class
> and its access check will be suppressed but only have read-access (i.e.
> can do Field::getXXX but not setXXX).
> 
> Brief summary of this patch:
> 
> 1. A new Lookup::defineHiddenClass method is the API to create a hidden
> class.
> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG
> option that
> ?? can be specified when creating a hidden class.
> 3. A new Class::isHiddenClass method tests if a class is a hidden class.
> 4. Field::setXXX method will throw IAE on a final field of a hidden class
> ?? regardless of the value of the accessible flag.
> 5. JVM_LookupDefineClass is the new JVM entry point for Lookup::defineClass
> ?? and defineHiddenClass to create a class from the given bytes.
> 6. ClassLoaderData implementation is not changed.? There is one primary CLD
> ?? that holds the classes strongly referenced by its defining loader.
> There
> ?? can be zero or more additional CLDs - one per weak class.
> 7. Nest host determination is updated per revised JVMS 5.4.4. Access control
> ?? check no longer throws LinkageError but instead it will throw IAE with
> ?? a clear message if a class fails to resolve/validate the nest host
> declared
> ?? in NestHost/NestMembers attribute.
> 8. JFR, jcmd, JDI are updated to support hidden classes.
> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
> ?? and generate a bridge method to desuger a method reference to a
> protected
> ?? method in its supertype in a different package
> 
> This patch also updates StringConcatFactory, LambdaMetaFactory, and
> LambdaForms
> to use hidden classes.? The webrev includes changes in nashorn to hidden
> class
> and I will update the webrev if JEP 372 removes it any time soon.
> 
> We uncovered a bug in Lookup::defineClass spec throws LinkageError and
> intends
> to have the newly created class linked.? However, the implementation in 14
> does not link the class.? A separate CSR [2] proposes to update the
> implementation to match the spec.? This patch fixes the implementation.
> 
> The spec update on JVM TI, JDI and Instrumentation will be done as
> a separate RFE [3].? This patch includes new tests for JVM TI and
> java.instrument that validates how the existing APIs work for hidden
> classes.
> 
> javadoc/specdiff
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/
> 
> JVMS 5.4.4 change:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf
> 
> CSR:
> https://bugs.openjdk.java.net/browse/JDK-8238359
> 
> Thanks
> Mandy
> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
> [3] https://bugs.openjdk.java.net/browse/JDK-8230502

From mandy.chung at oracle.com  Fri Mar 27 15:50:55 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Fri, 27 Mar 2020 08:50:55 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <1271059704.1214239.1585310406005.JavaMail.zimbra@u-pem.fr>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <1271059704.1214239.1585310406005.JavaMail.zimbra@u-pem.fr>
Message-ID: <b32743fe-6046-2d76-2edb-6fdaa8fb7f70@oracle.com>

On 3/27/20 5:00 AM, Remi Forax wrote:
> Hi Mandy,
> in ReflectionFactory, why in the case of a constructor the check to the anonymous class is removed ?

Good catch.? Fixed
>
> in BytecodeGenerator, the comment "// bootstrapping issue if using condy"
> can be promoted on top of clinit, because i ask myself the same question seeing a static block was generated

OK, that's clearer.
>
> in AbstractValidatingLambdaMetafactory.java, the field caller is not used after all ?

Thanks.? Removed.? It was left behind from an early prototype.

Below is the patch.? I will send out a new webrev and delta webrev in 
the next revision.

thanks
Mandy

diff --git 
a/src/java.base/share/classes/java/lang/invoke/AbstractValidatingLambdaMetafactory.java 
b/src/java.base/share/classes/java/lang/invoke/AbstractValidatingLambdaMetafactory.java
--- 
a/src/java.base/share/classes/java/lang/invoke/AbstractValidatingLambdaMetafactory.java
+++ 
b/src/java.base/share/classes/java/lang/invoke/AbstractValidatingLambdaMetafactory.java
@@ -51,7 +51,6 @@
 ????? *???????? System.out.printf(">>> %s\n", iii.foo(44));
 ????? * }}
 ????? */
-??? final MethodHandles.Lookup caller;
 ???? final Class<?> targetClass;?????????????? // The class calling the 
meta-factory via invokedynamic "class X"
 ???? final MethodType invokedType;???????????? // The type of the 
invoked method "(CC)II"
 ???? final Class<?> samBase;?????????????????? // The type of the 
returned instance "interface JJ"
@@ -121,7 +120,6 @@
 ???????????????????? "Invalid caller: %s",
 ???????????????????? caller.lookupClass().getName()));
 ???????? }
-??????? this.caller = caller;
 ???????? this.targetClass = caller.lookupClass();
 ???????? this.invokedType = invokedType;

diff --git 
a/src/java.base/share/classes/java/lang/invoke/InvokerBytecodeGenerator.java 
b/src/java.base/share/classes/java/lang/invoke/InvokerBytecodeGenerator.java
--- 
a/src/java.base/share/classes/java/lang/invoke/InvokerBytecodeGenerator.java
+++ 
b/src/java.base/share/classes/java/lang/invoke/InvokerBytecodeGenerator.java
@@ -363,6 +363,10 @@
 ???????? clinit(cw, className(), classData);
 ???? }

+??? /*
+???? * <clinit> to initialize the static final fields with the live 
class data
+???? * LambdaForms can't use condy due to bootstrapping issue.
+???? */
 ???? static void clinit(ClassWriter cw, String className, 
List<ClassData> classData) {
 ???????? if (classData.isEmpty())
 ???????????? return;
@@ -375,7 +379,6 @@

 ???????? MethodVisitor mv = cw.visitMethod(Opcodes.ACC_STATIC, 
"<clinit>", "()V", null, null);
 ???????? mv.visitCode();
-??????? // bootstrapping issue if using condy
 ???????? mv.visitLdcInsn(Type.getType("L" + className + ";"));
 ???????? mv.visitMethodInsn(Opcodes.INVOKESTATIC, 
"java/lang/invoke/MethodHandleNatives",
 ??????????????????????????? "classData", 
"(Ljava/lang/Class;)Ljava/lang/Object;", false);
diff --git 
a/src/java.base/share/classes/jdk/internal/reflect/ReflectionFactory.java 
b/src/java.base/share/classes/jdk/internal/reflect/ReflectionFactory.java
--- 
a/src/java.base/share/classes/jdk/internal/reflect/ReflectionFactory.java
+++ 
b/src/java.base/share/classes/jdk/internal/reflect/ReflectionFactory.java
@@ -245,7 +245,8 @@
 ???????????? return new BootstrapConstructorAccessorImpl(c);
 ???????? }

-??????? if (noInflation && !c.getDeclaringClass().isHiddenClass()) {
+??????? if (noInflation && !c.getDeclaringClass().isHiddenClass()
+??????????????? && 
!ReflectUtil.isVMAnonymousClass(c.getDeclaringClass())) {
 ???????????? return new MethodAccessorGenerator().
 ???????????????? generateConstructor(c.getDeclaringClass(),
 ???????????????????????????????????? c.getParameterTypes(),

From forax at univ-mlv.fr  Fri Mar 27 15:54:37 2020
From: forax at univ-mlv.fr (forax at univ-mlv.fr)
Date: Fri, 27 Mar 2020 16:54:37 +0100 (CET)
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <b32743fe-6046-2d76-2edb-6fdaa8fb7f70@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <1271059704.1214239.1585310406005.JavaMail.zimbra@u-pem.fr>
 <b32743fe-6046-2d76-2edb-6fdaa8fb7f70@oracle.com>
Message-ID: <387396409.1414184.1585324477993.JavaMail.zimbra@u-pem.fr>

> De: "mandy chung" <mandy.chung at oracle.com>
> ?: "Remi Forax" <forax at univ-mlv.fr>
> Cc: "valhalla-dev" <valhalla-dev at openjdk.java.net>, "core-libs-dev"
> <core-libs-dev at openjdk.java.net>, "serviceability-dev"
> <serviceability-dev at openjdk.java.net>, "hotspot-dev"
> <hotspot-dev at openjdk.java.net>
> Envoy?: Vendredi 27 Mars 2020 16:50:55
> Objet: Re: Review Request: 8238358: Implementation of JEP 371: Hidden Classes

> On 3/27/20 5:00 AM, Remi Forax wrote:

>> Hi Mandy,
>> in ReflectionFactory, why in the case of a constructor the check to the
>> anonymous class is removed ?

> Good catch. Fixed

>> in BytecodeGenerator, the comment "// bootstrapping issue if using condy"
>> can be promoted on top of clinit, because i ask myself the same question seeing
>> a static block was generated

> OK, that's clearer.

>> in AbstractValidatingLambdaMetafactory.java, the field caller is not used after
>> all ?

> Thanks. Removed. It was left behind from an early prototype.

> Below is the patch. I will send out a new webrev and delta webrev in the next
> revision.
Thanks Mandy, 
Looks good. 

R?mi 

> thanks
> Mandy

> diff --git
> a/src/java.base/share/classes/java/lang/invoke/AbstractValidatingLambdaMetafactory.java
> b/src/java.base/share/classes/java/lang/invoke/AbstractValidatingLambdaMetafactory.java
> ---
> a/src/java.base/share/classes/java/lang/invoke/AbstractValidatingLambdaMetafactory.java
> +++
> b/src/java.base/share/classes/java/lang/invoke/AbstractValidatingLambdaMetafactory.java
> @@ -51,7 +51,6 @@
> * System.out.printf(">>> %s\n", iii.foo(44));
> * }}
> */
> - final MethodHandles.Lookup caller;
> final Class<?> targetClass; // The class calling the meta-factory via
> invokedynamic "class X"
> final MethodType invokedType; // The type of the invoked method "(CC)II"
> final Class<?> samBase; // The type of the returned instance "interface JJ"
> @@ -121,7 +120,6 @@
> "Invalid caller: %s",
> caller.lookupClass().getName()));
> }
> - this.caller = caller;
> this.targetClass = caller.lookupClass();
> this.invokedType = invokedType;

> diff --git
> a/src/java.base/share/classes/java/lang/invoke/InvokerBytecodeGenerator.java
> b/src/java.base/share/classes/java/lang/invoke/InvokerBytecodeGenerator.java
> --- a/src/java.base/share/classes/java/lang/invoke/InvokerBytecodeGenerator.java
> +++ b/src/java.base/share/classes/java/lang/invoke/InvokerBytecodeGenerator.java
> @@ -363,6 +363,10 @@
> clinit(cw, className(), classData);
> }

> + /*
> + * <clinit> to initialize the static final fields with the live class data
> + * LambdaForms can't use condy due to bootstrapping issue.
> + */
> static void clinit(ClassWriter cw, String className, List<ClassData> classData)
> {
> if (classData.isEmpty())
> return;
> @@ -375,7 +379,6 @@

> MethodVisitor mv = cw.visitMethod(Opcodes.ACC_STATIC, "<clinit>", "()V", null,
> null);
> mv.visitCode();
> - // bootstrapping issue if using condy
> mv.visitLdcInsn(Type.getType("L" + className + ";"));
> mv.visitMethodInsn(Opcodes.INVOKESTATIC, "java/lang/invoke/MethodHandleNatives",
> "classData", "(Ljava/lang/Class;)Ljava/lang/Object;", false);
> diff --git
> a/src/java.base/share/classes/jdk/internal/reflect/ReflectionFactory.java
> b/src/java.base/share/classes/jdk/internal/reflect/ReflectionFactory.java
> --- a/src/java.base/share/classes/jdk/internal/reflect/ReflectionFactory.java
> +++ b/src/java.base/share/classes/jdk/internal/reflect/ReflectionFactory.java
> @@ -245,7 +245,8 @@
> return new BootstrapConstructorAccessorImpl(c);
> }

> - if (noInflation && !c.getDeclaringClass().isHiddenClass()) {
> + if (noInflation && !c.getDeclaringClass().isHiddenClass()
> + && !ReflectUtil.isVMAnonymousClass(c.getDeclaringClass())) {
> return new MethodAccessorGenerator().
> generateConstructor(c.getDeclaringClass(),
> c.getParameterTypes(),

From mandy.chung at oracle.com  Fri Mar 27 16:29:46 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Fri, 27 Mar 2020 09:29:46 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <d3d2fa4f-4bf7-94c3-de4b-15dfe75b6370@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <d3d2fa4f-4bf7-94c3-de4b-15dfe75b6370@oracle.com>
Message-ID: <08191054-8d0a-ae60-ac99-e2849f08ce85@oracle.com>

Hi Jan,

Good point.? The javac change only applies to JDK 15 and later and the 
lambda proxy class is not a nestmate when running on JDK 14 or earlier.

I probably need the help from langtools team to fix this.? I'll give it 
a try.

Mandy

On 3/27/20 4:31 AM, Jan Lahoda wrote:
> Hi Mandy,
>
> Regarding the javac changes - should those be switched on/off 
> depending the Target? Or, if one compiles with e.g. --release 14, will 
> the newly generated output still work on JDK 14?
>
> Jan
>
> On 27. 03. 20 0:57, Mandy Chung wrote:
>> Please review the implementation of JEP 371: Hidden Classes. The main 
>> changes are in core-libs and hotspot runtime area.? Small changes are 
>> made in javac, VM compiler (intrinsification of 
>> Class::isHiddenClass), JFR, JDI, and jcmd.? CSR [1]has been reviewed 
>> and is in the finalized state (see specdiff and javadoc below for 
>> reference).
>>
>> Webrev:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
>>
>>
>> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's 
>> point
>> of view, a hidden class is a normal class except the following:
>>
>> - A hidden class has no initiating class loader and is not registered 
>> in any dictionary.
>> - A hidden class has a name containing an illegal character 
>> `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature` 
>> returns "Lp/Foo.0x1234;".
>> - A hidden class is not modifiable, i.e. cannot be redefined or 
>> retransformed. JVM TI IsModifableClass returns false on a hidden.
>> - Final fields in a hidden class is "final".? The value of final 
>> fields cannot be overriden via reflection.? setAccessible(true) can 
>> still be called on reflected objects representing final fields in a 
>> hidden class and its access check will be suppressed but only have 
>> read-access (i.e. can do Field::getXXX but not setXXX).
>>
>> Brief summary of this patch:
>>
>> 1. A new Lookup::defineHiddenClass method is the API to create a 
>> hidden class.
>> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG 
>> option that
>> ??? can be specified when creating a hidden class.
>> 3. A new Class::isHiddenClass method tests if a class is a hidden class.
>> 4. Field::setXXX method will throw IAE on a final field of a hidden 
>> class
>> ??? regardless of the value of the accessible flag.
>> 5. JVM_LookupDefineClass is the new JVM entry point for 
>> Lookup::defineClass
>> ??? and defineHiddenClass to create a class from the given bytes.
>> 6. ClassLoaderData implementation is not changed.? There is one 
>> primary CLD
>> ??? that holds the classes strongly referenced by its defining 
>> loader. There
>> ??? can be zero or more additional CLDs - one per weak class.
>> 7. Nest host determination is updated per revised JVMS 5.4.4. Access 
>> control
>> ??? check no longer throws LinkageError but instead it will throw IAE 
>> with
>> ??? a clear message if a class fails to resolve/validate the nest 
>> host declared
>> ??? in NestHost/NestMembers attribute.
>> 8. JFR, jcmd, JDI are updated to support hidden classes.
>> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
>> ??? and generate a bridge method to desuger a method reference to a 
>> protected
>> ??? method in its supertype in a different package
>>
>> This patch also updates StringConcatFactory, LambdaMetaFactory, and 
>> LambdaForms
>> to use hidden classes.? The webrev includes changes in nashorn to 
>> hidden class
>> and I will update the webrev if JEP 372 removes it any time soon.
>>
>> We uncovered a bug in Lookup::defineClass spec throws LinkageError 
>> and intends
>> to have the newly created class linked.? However, the implementation 
>> in 14
>> does not link the class.? A separate CSR [2] proposes to update the
>> implementation to match the spec.? This patch fixes the implementation.
>>
>> The spec update on JVM TI, JDI and Instrumentation will be done as
>> a separate RFE [3].? This patch includes new tests for JVM TI and
>> java.instrument that validates how the existing APIs work for hidden 
>> classes.
>>
>> javadoc/specdiff
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
>>
>>
>> JVMS 5.4.4 change:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
>>
>>
>> CSR:
>> https://bugs.openjdk.java.net/browse/JDK-8238359
>>
>> Thanks
>> Mandy
>> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
>> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
>> [3] https://bugs.openjdk.java.net/browse/JDK-8230502


From paul.sandoz at oracle.com  Fri Mar 27 18:59:10 2020
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Fri, 27 Mar 2020 11:59:10 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
Message-ID: <72EC6AFD-412B-4238-B98E-90AF8C185D98@oracle.com>

Hi Mandy,

Very thorough, bravo!

Minor suggestions below.

Paul.

MethodHandleNatives.java
?

 142 
 143         /**
 144          * Flags for Lookup.ClassOptions
 145          */
 146         static final int
 147             NESTMATE_CLASS            = 0x00000001,
 148             HIDDEN_CLASS              = 0x00000002,
 149             STRONG_LOADER_LINK        = 0x00000004,
 150             ACCESS_VM_ANNOTATIONS     = 0x00000008;
 151     }
 

Suggest you add a comment to keep the values in sync with the VM component.


MethodHandles.java
?

1786          * (Given the {@code Lookup} object returned this method, its lookup class
1787          * is a {@code Class} object for which {@link Class#getName()} returns a string
1788          * that is not a binary name.)

?
(The {@code Lookup} object returned from this method has a lookup class that is 
a {@code Class} object whose {@link Class#getName()} returns a string
that is not a binary name.)
?


1902             Set<ClassOption> opts = options.length > 0 ? Set.of(options) : Set.of();

You can just do:

  Set<ClassOption> opts = Set.of(options)

And/or inline it into the subsequent method call.  The implementation of Set.of checks the array length.


2001         ClassDefiner makeHiddenClassDefiner(byte[] bytes,

I think you can telescope the methods for non-name and name accepting since IIUC the name is derived from the byte[].  Thereby you can remove some code duplication. i.e. pull ClassDefiner.className out from ClassDefiner and place the logic in the factory methods.  Alternative push the factory methods into ClassDefiner to keep all the logic together.



3797         public enum ClassOption {

Shuffle up to be closer to the defineHiddenClass


3798             /**
3799              * This class option specifies the hidden class be added to
3800              * {@linkplain Class#getNestHost nest} of a lookup class as
3801              * a nestmate.

Suggest:

"This class option specifies the hidden class ? -> ?Specifies that a hidden class 


3812              * This class option specifies the hidden class to have a <em>strong</em>

?Specifies that a hidden class have a ?"


3813              * relationship with the class loader marked as its defining loader,
3814              * as a normal class or interface has with its own defining loader.
3815              * This means that the hidden class may be unloaded if and only if
3816              * its defining loader is not reachable and thus may be reclaimed
3817              * by a garbage collector (JLS 12.7).


StringConcatFactory.java
?

 861             // use of @ForceInline no longer has any effect

?

 862             mv.visitAnnotation("Ljdk/internal/vm/annotation/ForceInline;", true);
 863             mv.visitCode();





> On Mar 26, 2020, at 4:57 PM, Mandy Chung <mandy.chung at oracle.com> wrote:
> 
> Please review the implementation of JEP 371: Hidden Classes. The main changes are in core-libs and hotspot runtime area.  Small changes are made in javac, VM compiler (intrinsification of Class::isHiddenClass), JFR, JDI, and jcmd.  CSR [1]has been reviewed and is in the finalized state (see specdiff and javadoc below for reference).
> 
> Webrev:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03
> 
> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's point
> of view, a hidden class is a normal class except the following:
> 
> - A hidden class has no initiating class loader and is not registered in any dictionary.
> - A hidden class has a name containing an illegal character `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature` returns "Lp/Foo.0x1234;".
> - A hidden class is not modifiable, i.e. cannot be redefined or retransformed. JVM TI IsModifableClass returns false on a hidden.
> - Final fields in a hidden class is "final".  The value of final fields cannot be overriden via reflection.  setAccessible(true) can still be called on reflected objects representing final fields in a hidden class and its access check will be suppressed but only have read-access (i.e. can do Field::getXXX but not setXXX).
> 
> Brief summary of this patch:
> 
> 1. A new Lookup::defineHiddenClass method is the API to create a hidden class.
> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG option that
>    can be specified when creating a hidden class.
> 3. A new Class::isHiddenClass method tests if a class is a hidden class.
> 4. Field::setXXX method will throw IAE on a final field of a hidden class
>    regardless of the value of the accessible flag.
> 5. JVM_LookupDefineClass is the new JVM entry point for Lookup::defineClass
>    and defineHiddenClass to create a class from the given bytes.
> 6. ClassLoaderData implementation is not changed.  There is one primary CLD
>    that holds the classes strongly referenced by its defining loader.  There
>    can be zero or more additional CLDs - one per weak class.
> 7. Nest host determination is updated per revised JVMS 5.4.4. Access control
>    check no longer throws LinkageError but instead it will throw IAE with
>    a clear message if a class fails to resolve/validate the nest host declared
>    in NestHost/NestMembers attribute.
> 8. JFR, jcmd, JDI are updated to support hidden classes.
> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
>    and generate a bridge method to desuger a method reference to a protected
>    method in its supertype in a different package
> 
> This patch also updates StringConcatFactory, LambdaMetaFactory, and LambdaForms
> to use hidden classes.  The webrev includes changes in nashorn to hidden class
> and I will update the webrev if JEP 372 removes it any time soon.
> 
> We uncovered a bug in Lookup::defineClass spec throws LinkageError and intends
> to have the newly created class linked.  However, the implementation in 14
> does not link the class.  A separate CSR [2] proposes to update the
> implementation to match the spec.  This patch fixes the implementation.
> 
> The spec update on JVM TI, JDI and Instrumentation will be done as
> a separate RFE [3].  This patch includes new tests for JVM TI and
> java.instrument that validates how the existing APIs work for hidden classes.
> 
> javadoc/specdiff
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/
> 
> JVMS 5.4.4 change:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf
> 
> CSR:
> https://bugs.openjdk.java.net/browse/JDK-8238359
> 
> Thanks
> Mandy
> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
> [3] https://bugs.openjdk.java.net/browse/JDK-8230502


From mandy.chung at oracle.com  Fri Mar 27 20:18:07 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Fri, 27 Mar 2020 13:18:07 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <72EC6AFD-412B-4238-B98E-90AF8C185D98@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <72EC6AFD-412B-4238-B98E-90AF8C185D98@oracle.com>
Message-ID: <c2ae90e1-5bf8-1d6b-a739-eb4aac542790@oracle.com>



On 3/27/20 11:59 AM, Paul Sandoz wrote:
> Hi Mandy,
>
> Very thorough, bravo!

Thanks.
> Minor suggestions below.
>
> Paul.
>
> MethodHandleNatives.java
> ?
>
>   142
>   143         /**
>   144          * Flags for Lookup.ClassOptions
>   145          */
>   146         static final int
>   147             NESTMATE_CLASS            = 0x00000001,
>   148             HIDDEN_CLASS              = 0x00000002,
>   149             STRONG_LOADER_LINK        = 0x00000004,
>   150             ACCESS_VM_ANNOTATIONS     = 0x00000008;
>   151     }
>   
>
> Suggest you add a comment to keep the values in sync with the VM component.

Already in the class spec of this Constants class.? The values of all 
constants defined in this Constants class are verified in sync with VM 
(see verifyConstants).

>
> MethodHandles.java
> ?
>
> 1786          * (Given the {@code Lookup} object returned this method, its lookup class
> 1787          * is a {@code Class} object for which {@link Class#getName()} returns a string
> 1788          * that is not a binary name.)
>
> ?
> (The {@code Lookup} object returned from this method has a lookup class that is
> a {@code Class} object whose {@link Class#getName()} returns a string
> that is not a binary name.)
> ?
>
>
> 1902             Set<ClassOption> opts = options.length > 0 ? Set.of(options) : Set.of();
>
> You can just do:
>
>    Set<ClassOption> opts = Set.of(options)
>
> And/or inline it into the subsequent method call.  The implementation of Set.of checks the array length.

Great to know.? Thanks.
>
> 2001         ClassDefiner makeHiddenClassDefiner(byte[] bytes,
>
> I think you can telescope the methods for non-name and name accepting since IIUC the name is derived from the byte[].  Thereby you can remove some code duplication. i.e. pull ClassDefiner.className out from ClassDefiner and place the logic in the factory methods.  Alternative push the factory methods into ClassDefiner to keep all the logic together.
>
Ok.? I will move the className out.
>
> 3797         public enum ClassOption {
>
> Shuffle up to be closer to the defineHiddenClass

Moved before defineHiddenClass.

>
> 3798             /**
> 3799              * This class option specifies the hidden class be added to
> 3800              * {@linkplain Class#getNestHost nest} of a lookup class as
> 3801              * a nestmate.
>
> Suggest:
>
> "This class option specifies the hidden class ? -> ?Specifies that a hidden class
>
> 3812              * This class option specifies the hidden class to have a <em>strong</em>
>
> ?Specifies that a hidden class have a ?"

Specifies that a hidden class has a...

>
> 3813              * relationship with the class loader marked as its defining loader,
> 3814              * as a normal class or interface has with its own defining loader.
> 3815              * This means that the hidden class may be unloaded if and only if
> 3816              * its defining loader is not reachable and thus may be reclaimed
> 3817              * by a garbage collector (JLS 12.7).
>
>
> StringConcatFactory.java
> ?
>
>   861             // use of @ForceInline no longer has any effect
>
> ?

Right, I should have explained this [1].

This @ForceInline is used by BytecodeStringBuilderStrategy that 
generates code to have the same StringBuilder chain javac would emit. It 
uses `@ForceInline` annotation which may probably be for performance.? 
It's believed people rarely uses this non-default strategy.? This patch 
changes StringConcatFactory to the standard defineHiddenClass method and 
hence `@ForceInline` has no effect in the generated class for this 
non-default strategy.? If it turns out to be an issue, then we will 
determine if it should enable the access to VM annotations (I doubt this 
is supported strategy).

[1] https://bugs.openjdk.java.net/browse/JDK-8241548

From vicente.romero at oracle.com  Fri Mar 27 21:15:29 2020
From: vicente.romero at oracle.com (Vicente Romero)
Date: Fri, 27 Mar 2020 17:15:29 -0400
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <08191054-8d0a-ae60-ac99-e2849f08ce85@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <d3d2fa4f-4bf7-94c3-de4b-15dfe75b6370@oracle.com>
 <08191054-8d0a-ae60-ac99-e2849f08ce85@oracle.com>
Message-ID: <e9ee79ca-e00a-c1d7-27a1-e110064a8fa3@oracle.com>

Hi Mandy,

The patch for nestmates [1] could be used as a reference. There a new 
method was added to class `com.sun.tools.javac.jvm.Target`, named: 
`hasNestmateAccess` which checks if a target is ready for nestmates or 
not. I think that you can follow a similar approach here.

Thanks,
Vicente

[1] http://hg.openjdk.java.net/jdk/jdk/rev/2f2af62dfac7

On 3/27/20 12:29 PM, Mandy Chung wrote:
> Hi Jan,
>
> Good point.? The javac change only applies to JDK 15 and later and the 
> lambda proxy class is not a nestmate when running on JDK 14 or earlier.
>
> I probably need the help from langtools team to fix this.? I'll give 
> it a try.
>
> Mandy
>
> On 3/27/20 4:31 AM, Jan Lahoda wrote:
>> Hi Mandy,
>>
>> Regarding the javac changes - should those be switched on/off 
>> depending the Target? Or, if one compiles with e.g. --release 14, 
>> will the newly generated output still work on JDK 14?
>>
>> Jan
>>
>> On 27. 03. 20 0:57, Mandy Chung wrote:
>>> Please review the implementation of JEP 371: Hidden Classes. The 
>>> main changes are in core-libs and hotspot runtime area.? Small 
>>> changes are made in javac, VM compiler (intrinsification of 
>>> Class::isHiddenClass), JFR, JDI, and jcmd.? CSR [1]has been reviewed 
>>> and is in the finalized state (see specdiff and javadoc below for 
>>> reference).
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
>>>
>>>
>>> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's 
>>> point
>>> of view, a hidden class is a normal class except the following:
>>>
>>> - A hidden class has no initiating class loader and is not 
>>> registered in any dictionary.
>>> - A hidden class has a name containing an illegal character 
>>> `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature` 
>>> returns "Lp/Foo.0x1234;".
>>> - A hidden class is not modifiable, i.e. cannot be redefined or 
>>> retransformed. JVM TI IsModifableClass returns false on a hidden.
>>> - Final fields in a hidden class is "final".? The value of final 
>>> fields cannot be overriden via reflection. setAccessible(true) can 
>>> still be called on reflected objects representing final fields in a 
>>> hidden class and its access check will be suppressed but only have 
>>> read-access (i.e. can do Field::getXXX but not setXXX).
>>>
>>> Brief summary of this patch:
>>>
>>> 1. A new Lookup::defineHiddenClass method is the API to create a 
>>> hidden class.
>>> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG 
>>> option that
>>> ??? can be specified when creating a hidden class.
>>> 3. A new Class::isHiddenClass method tests if a class is a hidden 
>>> class.
>>> 4. Field::setXXX method will throw IAE on a final field of a hidden 
>>> class
>>> ??? regardless of the value of the accessible flag.
>>> 5. JVM_LookupDefineClass is the new JVM entry point for 
>>> Lookup::defineClass
>>> ??? and defineHiddenClass to create a class from the given bytes.
>>> 6. ClassLoaderData implementation is not changed.? There is one 
>>> primary CLD
>>> ??? that holds the classes strongly referenced by its defining 
>>> loader. There
>>> ??? can be zero or more additional CLDs - one per weak class.
>>> 7. Nest host determination is updated per revised JVMS 5.4.4. Access 
>>> control
>>> ??? check no longer throws LinkageError but instead it will throw 
>>> IAE with
>>> ??? a clear message if a class fails to resolve/validate the nest 
>>> host declared
>>> ??? in NestHost/NestMembers attribute.
>>> 8. JFR, jcmd, JDI are updated to support hidden classes.
>>> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
>>> ??? and generate a bridge method to desuger a method reference to a 
>>> protected
>>> ??? method in its supertype in a different package
>>>
>>> This patch also updates StringConcatFactory, LambdaMetaFactory, and 
>>> LambdaForms
>>> to use hidden classes.? The webrev includes changes in nashorn to 
>>> hidden class
>>> and I will update the webrev if JEP 372 removes it any time soon.
>>>
>>> We uncovered a bug in Lookup::defineClass spec throws LinkageError 
>>> and intends
>>> to have the newly created class linked.? However, the implementation 
>>> in 14
>>> does not link the class.? A separate CSR [2] proposes to update the
>>> implementation to match the spec.? This patch fixes the implementation.
>>>
>>> The spec update on JVM TI, JDI and Instrumentation will be done as
>>> a separate RFE [3].? This patch includes new tests for JVM TI and
>>> java.instrument that validates how the existing APIs work for hidden 
>>> classes.
>>>
>>> javadoc/specdiff
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
>>>
>>>
>>> JVMS 5.4.4 change:
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
>>>
>>>
>>> CSR:
>>> https://bugs.openjdk.java.net/browse/JDK-8238359
>>>
>>> Thanks
>>> Mandy
>>> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
>>> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
>>> [3] https://bugs.openjdk.java.net/browse/JDK-8230502
>


From mandy.chung at oracle.com  Fri Mar 27 22:22:19 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Fri, 27 Mar 2020 15:22:19 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <c2ae90e1-5bf8-1d6b-a739-eb4aac542790@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <72EC6AFD-412B-4238-B98E-90AF8C185D98@oracle.com>
 <c2ae90e1-5bf8-1d6b-a739-eb4aac542790@oracle.com>
Message-ID: <41b99011-921a-9fc6-c738-98c47e9959c3@oracle.com>

Hi Paul,

This is the delta incorporating your comment:
http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03-delta-psandoz/

This patch also took Alex's comment to make it clear that the hidden 
class is the lookup class of the returned Lookup object and drops the 
sentence you commented on:

On 3/27/20 1:18 PM, Mandy Chung wrote:
>> MethodHandles.java
>> ?
>>
>> 1786????????? * (Given the {@code Lookup} object returned this 
>> method, its lookup class
>> 1787????????? * is a {@code Class} object for which {@link 
>> Class#getName()} returns a string
>> 1788????????? * that is not a binary name.)
>>
>> ?
>> (The {@code Lookup} object returned from this method has a lookup 
>> class that is
>> a {@code Class} object whose {@link Class#getName()} returns a string
>> that is not a binary name.)
>> ?


Mandy

From mandy.chung at oracle.com  Fri Mar 27 22:29:03 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Fri, 27 Mar 2020 15:29:03 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <e9ee79ca-e00a-c1d7-27a1-e110064a8fa3@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <d3d2fa4f-4bf7-94c3-de4b-15dfe75b6370@oracle.com>
 <08191054-8d0a-ae60-ac99-e2849f08ce85@oracle.com>
 <e9ee79ca-e00a-c1d7-27a1-e110064a8fa3@oracle.com>
Message-ID: <dbdeae81-7006-6922-9919-f2d253152006@oracle.com>

Hi Vicente,

hasNestmateAccess is about VM supports static nestmates on JDK release 
 >= 11.

However this is about javac --release 14 and the compiled classes may 
run on JDK 14 that lambda and string concat spin classes that are not 
nestmates. I have a patch with Jan's help:

http://cr.openjdk.java.net/~mchung/valhalla/webrevs/8171335/webrev-javac-target-release-14/index.html

(you can apply the above patch on valhalla repo "nestmates" branch)

About testing, I wanted to run BridgeMethodsForLambdaTest and 
TestLambdaBytecode test with --release 14 but it turns out not 
straight-forward.? Any help would be appreciated.

thanks
Mandy

On 3/27/20 2:15 PM, Vicente Romero wrote:
> Hi Mandy,
>
> The patch for nestmates [1] could be used as a reference. There a new 
> method was added to class `com.sun.tools.javac.jvm.Target`, named: 
> `hasNestmateAccess` which checks if a target is ready for nestmates or 
> not. I think that you can follow a similar approach here.
>
> Thanks,
> Vicente
>
> [1] http://hg.openjdk.java.net/jdk/jdk/rev/2f2af62dfac7
>
> On 3/27/20 12:29 PM, Mandy Chung wrote:
>> Hi Jan,
>>
>> Good point.? The javac change only applies to JDK 15 and later and 
>> the lambda proxy class is not a nestmate when running on JDK 14 or 
>> earlier.
>>
>> I probably need the help from langtools team to fix this.? I'll give 
>> it a try.
>>
>> Mandy
>>
>> On 3/27/20 4:31 AM, Jan Lahoda wrote:
>>> Hi Mandy,
>>>
>>> Regarding the javac changes - should those be switched on/off 
>>> depending the Target? Or, if one compiles with e.g. --release 14, 
>>> will the newly generated output still work on JDK 14?
>>>
>>> Jan
>>>
>>> On 27. 03. 20 0:57, Mandy Chung wrote:
>>>> Please review the implementation of JEP 371: Hidden Classes. The 
>>>> main changes are in core-libs and hotspot runtime area.? Small 
>>>> changes are made in javac, VM compiler (intrinsification of 
>>>> Class::isHiddenClass), JFR, JDI, and jcmd.? CSR [1]has been 
>>>> reviewed and is in the finalized state (see specdiff and javadoc 
>>>> below for reference).
>>>>
>>>> Webrev:
>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
>>>>
>>>>
>>>> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's 
>>>> point
>>>> of view, a hidden class is a normal class except the following:
>>>>
>>>> - A hidden class has no initiating class loader and is not 
>>>> registered in any dictionary.
>>>> - A hidden class has a name containing an illegal character 
>>>> `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature` 
>>>> returns "Lp/Foo.0x1234;".
>>>> - A hidden class is not modifiable, i.e. cannot be redefined or 
>>>> retransformed. JVM TI IsModifableClass returns false on a hidden.
>>>> - Final fields in a hidden class is "final".? The value of final 
>>>> fields cannot be overriden via reflection. setAccessible(true) can 
>>>> still be called on reflected objects representing final fields in a 
>>>> hidden class and its access check will be suppressed but only have 
>>>> read-access (i.e. can do Field::getXXX but not setXXX).
>>>>
>>>> Brief summary of this patch:
>>>>
>>>> 1. A new Lookup::defineHiddenClass method is the API to create a 
>>>> hidden class.
>>>> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG 
>>>> option that
>>>> ??? can be specified when creating a hidden class.
>>>> 3. A new Class::isHiddenClass method tests if a class is a hidden 
>>>> class.
>>>> 4. Field::setXXX method will throw IAE on a final field of a hidden 
>>>> class
>>>> ??? regardless of the value of the accessible flag.
>>>> 5. JVM_LookupDefineClass is the new JVM entry point for 
>>>> Lookup::defineClass
>>>> ??? and defineHiddenClass to create a class from the given bytes.
>>>> 6. ClassLoaderData implementation is not changed.? There is one 
>>>> primary CLD
>>>> ??? that holds the classes strongly referenced by its defining 
>>>> loader. There
>>>> ??? can be zero or more additional CLDs - one per weak class.
>>>> 7. Nest host determination is updated per revised JVMS 5.4.4. 
>>>> Access control
>>>> ??? check no longer throws LinkageError but instead it will throw 
>>>> IAE with
>>>> ??? a clear message if a class fails to resolve/validate the nest 
>>>> host declared
>>>> ??? in NestHost/NestMembers attribute.
>>>> 8. JFR, jcmd, JDI are updated to support hidden classes.
>>>> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
>>>> ??? and generate a bridge method to desuger a method reference to a 
>>>> protected
>>>> ??? method in its supertype in a different package
>>>>
>>>> This patch also updates StringConcatFactory, LambdaMetaFactory, and 
>>>> LambdaForms
>>>> to use hidden classes.? The webrev includes changes in nashorn to 
>>>> hidden class
>>>> and I will update the webrev if JEP 372 removes it any time soon.
>>>>
>>>> We uncovered a bug in Lookup::defineClass spec throws LinkageError 
>>>> and intends
>>>> to have the newly created class linked.? However, the 
>>>> implementation in 14
>>>> does not link the class.? A separate CSR [2] proposes to update the
>>>> implementation to match the spec.? This patch fixes the 
>>>> implementation.
>>>>
>>>> The spec update on JVM TI, JDI and Instrumentation will be done as
>>>> a separate RFE [3].? This patch includes new tests for JVM TI and
>>>> java.instrument that validates how the existing APIs work for 
>>>> hidden classes.
>>>>
>>>> javadoc/specdiff
>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/ 
>>>>
>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
>>>>
>>>>
>>>> JVMS 5.4.4 change:
>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
>>>>
>>>>
>>>> CSR:
>>>> https://bugs.openjdk.java.net/browse/JDK-8238359
>>>>
>>>> Thanks
>>>> Mandy
>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
>>>> [3] https://bugs.openjdk.java.net/browse/JDK-8230502
>>
>


From vicente.romero at oracle.com  Fri Mar 27 22:48:52 2020
From: vicente.romero at oracle.com (Vicente Romero)
Date: Fri, 27 Mar 2020 18:48:52 -0400
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <dbdeae81-7006-6922-9919-f2d253152006@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <d3d2fa4f-4bf7-94c3-de4b-15dfe75b6370@oracle.com>
 <08191054-8d0a-ae60-ac99-e2849f08ce85@oracle.com>
 <e9ee79ca-e00a-c1d7-27a1-e110064a8fa3@oracle.com>
 <dbdeae81-7006-6922-9919-f2d253152006@oracle.com>
Message-ID: <60348819-ace5-8e52-f1ff-5f9654c915e0@oracle.com>

Hi Mandy,

On 3/27/20 6:29 PM, Mandy Chung wrote:
> Hi Vicente,
>
> hasNestmateAccess is about VM supports static nestmates on JDK release 
> >= 11.

I was not suggesting the use of `hasNestmateAccess` but to follow the 
same approach which is adding a new method at class `Target` to check if 
the new goodies were in the given target
>
> However this is about javac --release 14 and the compiled classes may 
> run on JDK 14 that lambda and string concat spin classes that are not 
> nestmates. I have a patch with Jan's help:
>
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/8171335/webrev-javac-target-release-14/index.html

which is what the patch above is doing

>
> (you can apply the above patch on valhalla repo "nestmates" branch)
>
> About testing, I wanted to run BridgeMethodsForLambdaTest and 
> TestLambdaBytecode test with --release 14 but it turns out not 
> straight-forward.? Any help would be appreciated.
>
> thanks
> Mandy

Vicente
>
> On 3/27/20 2:15 PM, Vicente Romero wrote:
>> Hi Mandy,
>>
>> The patch for nestmates [1] could be used as a reference. There a new 
>> method was added to class `com.sun.tools.javac.jvm.Target`, named: 
>> `hasNestmateAccess` which checks if a target is ready for nestmates 
>> or not. I think that you can follow a similar approach here.
>>
>> Thanks,
>> Vicente
>>
>> [1] http://hg.openjdk.java.net/jdk/jdk/rev/2f2af62dfac7
>>
>> On 3/27/20 12:29 PM, Mandy Chung wrote:
>>> Hi Jan,
>>>
>>> Good point.? The javac change only applies to JDK 15 and later and 
>>> the lambda proxy class is not a nestmate when running on JDK 14 or 
>>> earlier.
>>>
>>> I probably need the help from langtools team to fix this. I'll give 
>>> it a try.
>>>
>>> Mandy
>>>
>>> On 3/27/20 4:31 AM, Jan Lahoda wrote:
>>>> Hi Mandy,
>>>>
>>>> Regarding the javac changes - should those be switched on/off 
>>>> depending the Target? Or, if one compiles with e.g. --release 14, 
>>>> will the newly generated output still work on JDK 14?
>>>>
>>>> Jan
>>>>
>>>> On 27. 03. 20 0:57, Mandy Chung wrote:
>>>>> Please review the implementation of JEP 371: Hidden Classes. The 
>>>>> main changes are in core-libs and hotspot runtime area.? Small 
>>>>> changes are made in javac, VM compiler (intrinsification of 
>>>>> Class::isHiddenClass), JFR, JDI, and jcmd.? CSR [1]has been 
>>>>> reviewed and is in the finalized state (see specdiff and javadoc 
>>>>> below for reference).
>>>>>
>>>>> Webrev:
>>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
>>>>>
>>>>>
>>>>> Hidden class is created via `Lookup::defineHiddenClass`. From 
>>>>> JVM's point
>>>>> of view, a hidden class is a normal class except the following:
>>>>>
>>>>> - A hidden class has no initiating class loader and is not 
>>>>> registered in any dictionary.
>>>>> - A hidden class has a name containing an illegal character 
>>>>> `Class::getName` returns `p.Foo/0x1234` whereas 
>>>>> `GetClassSignature` returns "Lp/Foo.0x1234;".
>>>>> - A hidden class is not modifiable, i.e. cannot be redefined or 
>>>>> retransformed. JVM TI IsModifableClass returns false on a hidden.
>>>>> - Final fields in a hidden class is "final".? The value of final 
>>>>> fields cannot be overriden via reflection. setAccessible(true) can 
>>>>> still be called on reflected objects representing final fields in 
>>>>> a hidden class and its access check will be suppressed but only 
>>>>> have read-access (i.e. can do Field::getXXX but not setXXX).
>>>>>
>>>>> Brief summary of this patch:
>>>>>
>>>>> 1. A new Lookup::defineHiddenClass method is the API to create a 
>>>>> hidden class.
>>>>> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG 
>>>>> option that
>>>>> ??? can be specified when creating a hidden class.
>>>>> 3. A new Class::isHiddenClass method tests if a class is a hidden 
>>>>> class.
>>>>> 4. Field::setXXX method will throw IAE on a final field of a 
>>>>> hidden class
>>>>> ??? regardless of the value of the accessible flag.
>>>>> 5. JVM_LookupDefineClass is the new JVM entry point for 
>>>>> Lookup::defineClass
>>>>> ??? and defineHiddenClass to create a class from the given bytes.
>>>>> 6. ClassLoaderData implementation is not changed.? There is one 
>>>>> primary CLD
>>>>> ??? that holds the classes strongly referenced by its defining 
>>>>> loader. There
>>>>> ??? can be zero or more additional CLDs - one per weak class.
>>>>> 7. Nest host determination is updated per revised JVMS 5.4.4. 
>>>>> Access control
>>>>> ??? check no longer throws LinkageError but instead it will throw 
>>>>> IAE with
>>>>> ??? a clear message if a class fails to resolve/validate the nest 
>>>>> host declared
>>>>> ??? in NestHost/NestMembers attribute.
>>>>> 8. JFR, jcmd, JDI are updated to support hidden classes.
>>>>> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
>>>>> ??? and generate a bridge method to desuger a method reference to 
>>>>> a protected
>>>>> ??? method in its supertype in a different package
>>>>>
>>>>> This patch also updates StringConcatFactory, LambdaMetaFactory, 
>>>>> and LambdaForms
>>>>> to use hidden classes.? The webrev includes changes in nashorn to 
>>>>> hidden class
>>>>> and I will update the webrev if JEP 372 removes it any time soon.
>>>>>
>>>>> We uncovered a bug in Lookup::defineClass spec throws LinkageError 
>>>>> and intends
>>>>> to have the newly created class linked.? However, the 
>>>>> implementation in 14
>>>>> does not link the class.? A separate CSR [2] proposes to update the
>>>>> implementation to match the spec.? This patch fixes the 
>>>>> implementation.
>>>>>
>>>>> The spec update on JVM TI, JDI and Instrumentation will be done as
>>>>> a separate RFE [3].? This patch includes new tests for JVM TI and
>>>>> java.instrument that validates how the existing APIs work for 
>>>>> hidden classes.
>>>>>
>>>>> javadoc/specdiff
>>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/ 
>>>>>
>>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
>>>>>
>>>>>
>>>>> JVMS 5.4.4 change:
>>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
>>>>>
>>>>>
>>>>> CSR:
>>>>> https://bugs.openjdk.java.net/browse/JDK-8238359
>>>>>
>>>>> Thanks
>>>>> Mandy
>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
>>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
>>>>> [3] https://bugs.openjdk.java.net/browse/JDK-8230502
>>>
>>
>


From david.holmes at oracle.com  Fri Mar 27 23:01:58 2020
From: david.holmes at oracle.com (David Holmes)
Date: Sat, 28 Mar 2020 09:01:58 +1000
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <dbdeae81-7006-6922-9919-f2d253152006@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <d3d2fa4f-4bf7-94c3-de4b-15dfe75b6370@oracle.com>
 <08191054-8d0a-ae60-ac99-e2849f08ce85@oracle.com>
 <e9ee79ca-e00a-c1d7-27a1-e110064a8fa3@oracle.com>
 <dbdeae81-7006-6922-9919-f2d253152006@oracle.com>
Message-ID: <51d0a4cf-2565-2d61-31bb-ff43bca6b0e4@oracle.com>

Hi Mandy,

On 28/03/2020 8:29 am, Mandy Chung wrote:
> Hi Vicente,
> 
> hasNestmateAccess is about VM supports static nestmates on JDK release 
>  >= 11.
> 
> However this is about javac --release 14 and the compiled classes may 
> run on JDK 14 that lambda and string concat spin classes that are not 
> nestmates. I have a patch with Jan's help:
> 
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/8171335/webrev-javac-target-release-14/index.html 

+             /**
+              * The VM does not support access across nested classes 
(8010319).
+              * Were that ever to change, this should be removed.
+              */
+             boolean isPrivateInOtherClass() {

I'm not at all sure what this means - access across different nests? 
(I'm not even sure what that means.)

Thanks,
David
-----

> 
> (you can apply the above patch on valhalla repo "nestmates" branch)
> 
> About testing, I wanted to run BridgeMethodsForLambdaTest and 
> TestLambdaBytecode test with --release 14 but it turns out not 
> straight-forward.? Any help would be appreciated.
> 
> thanks
> Mandy
> 
> On 3/27/20 2:15 PM, Vicente Romero wrote:
>> Hi Mandy,
>>
>> The patch for nestmates [1] could be used as a reference. There a new 
>> method was added to class `com.sun.tools.javac.jvm.Target`, named: 
>> `hasNestmateAccess` which checks if a target is ready for nestmates or 
>> not. I think that you can follow a similar approach here.
>>
>> Thanks,
>> Vicente
>>
>> [1] http://hg.openjdk.java.net/jdk/jdk/rev/2f2af62dfac7
>>
>> On 3/27/20 12:29 PM, Mandy Chung wrote:
>>> Hi Jan,
>>>
>>> Good point.? The javac change only applies to JDK 15 and later and 
>>> the lambda proxy class is not a nestmate when running on JDK 14 or 
>>> earlier.
>>>
>>> I probably need the help from langtools team to fix this.? I'll give 
>>> it a try.
>>>
>>> Mandy
>>>
>>> On 3/27/20 4:31 AM, Jan Lahoda wrote:
>>>> Hi Mandy,
>>>>
>>>> Regarding the javac changes - should those be switched on/off 
>>>> depending the Target? Or, if one compiles with e.g. --release 14, 
>>>> will the newly generated output still work on JDK 14?
>>>>
>>>> Jan
>>>>
>>>> On 27. 03. 20 0:57, Mandy Chung wrote:
>>>>> Please review the implementation of JEP 371: Hidden Classes. The 
>>>>> main changes are in core-libs and hotspot runtime area.? Small 
>>>>> changes are made in javac, VM compiler (intrinsification of 
>>>>> Class::isHiddenClass), JFR, JDI, and jcmd.? CSR [1]has been 
>>>>> reviewed and is in the finalized state (see specdiff and javadoc 
>>>>> below for reference).
>>>>>
>>>>> Webrev:
>>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
>>>>>
>>>>>
>>>>> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's 
>>>>> point
>>>>> of view, a hidden class is a normal class except the following:
>>>>>
>>>>> - A hidden class has no initiating class loader and is not 
>>>>> registered in any dictionary.
>>>>> - A hidden class has a name containing an illegal character 
>>>>> `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature` 
>>>>> returns "Lp/Foo.0x1234;".
>>>>> - A hidden class is not modifiable, i.e. cannot be redefined or 
>>>>> retransformed. JVM TI IsModifableClass returns false on a hidden.
>>>>> - Final fields in a hidden class is "final".? The value of final 
>>>>> fields cannot be overriden via reflection. setAccessible(true) can 
>>>>> still be called on reflected objects representing final fields in a 
>>>>> hidden class and its access check will be suppressed but only have 
>>>>> read-access (i.e. can do Field::getXXX but not setXXX).
>>>>>
>>>>> Brief summary of this patch:
>>>>>
>>>>> 1. A new Lookup::defineHiddenClass method is the API to create a 
>>>>> hidden class.
>>>>> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG 
>>>>> option that
>>>>> ??? can be specified when creating a hidden class.
>>>>> 3. A new Class::isHiddenClass method tests if a class is a hidden 
>>>>> class.
>>>>> 4. Field::setXXX method will throw IAE on a final field of a hidden 
>>>>> class
>>>>> ??? regardless of the value of the accessible flag.
>>>>> 5. JVM_LookupDefineClass is the new JVM entry point for 
>>>>> Lookup::defineClass
>>>>> ??? and defineHiddenClass to create a class from the given bytes.
>>>>> 6. ClassLoaderData implementation is not changed.? There is one 
>>>>> primary CLD
>>>>> ??? that holds the classes strongly referenced by its defining 
>>>>> loader. There
>>>>> ??? can be zero or more additional CLDs - one per weak class.
>>>>> 7. Nest host determination is updated per revised JVMS 5.4.4. 
>>>>> Access control
>>>>> ??? check no longer throws LinkageError but instead it will throw 
>>>>> IAE with
>>>>> ??? a clear message if a class fails to resolve/validate the nest 
>>>>> host declared
>>>>> ??? in NestHost/NestMembers attribute.
>>>>> 8. JFR, jcmd, JDI are updated to support hidden classes.
>>>>> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
>>>>> ??? and generate a bridge method to desuger a method reference to a 
>>>>> protected
>>>>> ??? method in its supertype in a different package
>>>>>
>>>>> This patch also updates StringConcatFactory, LambdaMetaFactory, and 
>>>>> LambdaForms
>>>>> to use hidden classes.? The webrev includes changes in nashorn to 
>>>>> hidden class
>>>>> and I will update the webrev if JEP 372 removes it any time soon.
>>>>>
>>>>> We uncovered a bug in Lookup::defineClass spec throws LinkageError 
>>>>> and intends
>>>>> to have the newly created class linked.? However, the 
>>>>> implementation in 14
>>>>> does not link the class.? A separate CSR [2] proposes to update the
>>>>> implementation to match the spec.? This patch fixes the 
>>>>> implementation.
>>>>>
>>>>> The spec update on JVM TI, JDI and Instrumentation will be done as
>>>>> a separate RFE [3].? This patch includes new tests for JVM TI and
>>>>> java.instrument that validates how the existing APIs work for 
>>>>> hidden classes.
>>>>>
>>>>> javadoc/specdiff
>>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/ 
>>>>>
>>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
>>>>>
>>>>>
>>>>> JVMS 5.4.4 change:
>>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
>>>>>
>>>>>
>>>>> CSR:
>>>>> https://bugs.openjdk.java.net/browse/JDK-8238359
>>>>>
>>>>> Thanks
>>>>> Mandy
>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
>>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
>>>>> [3] https://bugs.openjdk.java.net/browse/JDK-8230502
>>>
>>
> 

From forax at univ-mlv.fr  Fri Mar 27 23:40:59 2020
From: forax at univ-mlv.fr (Remi Forax)
Date: Sat, 28 Mar 2020 00:40:59 +0100 (CET)
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <51d0a4cf-2565-2d61-31bb-ff43bca6b0e4@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <d3d2fa4f-4bf7-94c3-de4b-15dfe75b6370@oracle.com>
 <08191054-8d0a-ae60-ac99-e2849f08ce85@oracle.com>
 <e9ee79ca-e00a-c1d7-27a1-e110064a8fa3@oracle.com>
 <dbdeae81-7006-6922-9919-f2d253152006@oracle.com>
 <51d0a4cf-2565-2d61-31bb-ff43bca6b0e4@oracle.com>
Message-ID: <405050984.1553152.1585352459094.JavaMail.zimbra@u-pem.fr>

----- Mail original -----
> De: "David Holmes" <david.holmes at oracle.com>
> ?: "mandy chung" <mandy.chung at oracle.com>, "Vicente Romero" <vicente.romero at oracle.com>, "jan lahoda"
> <jan.lahoda at oracle.com>
> Cc: "serviceability-dev" <serviceability-dev at openjdk.java.net>, "hotspot-dev" <hotspot-dev at openjdk.java.net>,
> "core-libs-dev" <core-libs-dev at openjdk.java.net>, "valhalla-dev" <valhalla-dev at openjdk.java.net>
> Envoy?: Samedi 28 Mars 2020 00:01:58
> Objet: Re: Review Request: 8238358: Implementation of JEP 371: Hidden Classes

> Hi Mandy,

Hi David,

> 
> On 28/03/2020 8:29 am, Mandy Chung wrote:
>> Hi Vicente,
>> 
>> hasNestmateAccess is about VM supports static nestmates on JDK release
>>  >= 11.
>> 
>> However this is about javac --release 14 and the compiled classes may
>> run on JDK 14 that lambda and string concat spin classes that are not
>> nestmates. I have a patch with Jan's help:
>> 
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/8171335/webrev-javac-target-release-14/index.html
> 
> +             /**
> +              * The VM does not support access across nested classes
> (8010319).
> +              * Were that ever to change, this should be removed.
> +              */
> +             boolean isPrivateInOtherClass() {
> 
> I'm not at all sure what this means - access across different nests?
> (I'm not even sure what that means.)

Access inside the same nest.
As you know, until now, a lambda proxy is a VM anonymous class that can only see the private fields of the class declaring the lambda (the host class) and not the private fields of a class of the nest (the enclosing classes in term of Java the language).

R?mi

> 
> Thanks,
> David
> -----
> 
>> 
>> (you can apply the above patch on valhalla repo "nestmates" branch)
>> 
>> About testing, I wanted to run BridgeMethodsForLambdaTest and
>> TestLambdaBytecode test with --release 14 but it turns out not
>> straight-forward.? Any help would be appreciated.
>> 
>> thanks
>> Mandy
>> 
>> On 3/27/20 2:15 PM, Vicente Romero wrote:
>>> Hi Mandy,
>>>
>>> The patch for nestmates [1] could be used as a reference. There a new
>>> method was added to class `com.sun.tools.javac.jvm.Target`, named:
>>> `hasNestmateAccess` which checks if a target is ready for nestmates or
>>> not. I think that you can follow a similar approach here.
>>>
>>> Thanks,
>>> Vicente
>>>
>>> [1] http://hg.openjdk.java.net/jdk/jdk/rev/2f2af62dfac7
>>>
>>> On 3/27/20 12:29 PM, Mandy Chung wrote:
>>>> Hi Jan,
>>>>
>>>> Good point.? The javac change only applies to JDK 15 and later and
>>>> the lambda proxy class is not a nestmate when running on JDK 14 or
>>>> earlier.
>>>>
>>>> I probably need the help from langtools team to fix this.? I'll give
>>>> it a try.
>>>>
>>>> Mandy
>>>>
>>>> On 3/27/20 4:31 AM, Jan Lahoda wrote:
>>>>> Hi Mandy,
>>>>>
>>>>> Regarding the javac changes - should those be switched on/off
>>>>> depending the Target? Or, if one compiles with e.g. --release 14,
>>>>> will the newly generated output still work on JDK 14?
>>>>>
>>>>> Jan
>>>>>
>>>>> On 27. 03. 20 0:57, Mandy Chung wrote:
>>>>>> Please review the implementation of JEP 371: Hidden Classes. The
>>>>>> main changes are in core-libs and hotspot runtime area.? Small
>>>>>> changes are made in javac, VM compiler (intrinsification of
>>>>>> Class::isHiddenClass), JFR, JDI, and jcmd.? CSR [1]has been
>>>>>> reviewed and is in the finalized state (see specdiff and javadoc
>>>>>> below for reference).
>>>>>>
>>>>>> Webrev:
>>>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03
>>>>>>
>>>>>>
>>>>>> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's
>>>>>> point
>>>>>> of view, a hidden class is a normal class except the following:
>>>>>>
>>>>>> - A hidden class has no initiating class loader and is not
>>>>>> registered in any dictionary.
>>>>>> - A hidden class has a name containing an illegal character
>>>>>> `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature`
>>>>>> returns "Lp/Foo.0x1234;".
>>>>>> - A hidden class is not modifiable, i.e. cannot be redefined or
>>>>>> retransformed. JVM TI IsModifableClass returns false on a hidden.
>>>>>> - Final fields in a hidden class is "final".? The value of final
>>>>>> fields cannot be overriden via reflection. setAccessible(true) can
>>>>>> still be called on reflected objects representing final fields in a
>>>>>> hidden class and its access check will be suppressed but only have
>>>>>> read-access (i.e. can do Field::getXXX but not setXXX).
>>>>>>
>>>>>> Brief summary of this patch:
>>>>>>
>>>>>> 1. A new Lookup::defineHiddenClass method is the API to create a
>>>>>> hidden class.
>>>>>> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG
>>>>>> option that
>>>>>> ??? can be specified when creating a hidden class.
>>>>>> 3. A new Class::isHiddenClass method tests if a class is a hidden
>>>>>> class.
>>>>>> 4. Field::setXXX method will throw IAE on a final field of a hidden
>>>>>> class
>>>>>> ??? regardless of the value of the accessible flag.
>>>>>> 5. JVM_LookupDefineClass is the new JVM entry point for
>>>>>> Lookup::defineClass
>>>>>> ??? and defineHiddenClass to create a class from the given bytes.
>>>>>> 6. ClassLoaderData implementation is not changed.? There is one
>>>>>> primary CLD
>>>>>> ??? that holds the classes strongly referenced by its defining
>>>>>> loader. There
>>>>>> ??? can be zero or more additional CLDs - one per weak class.
>>>>>> 7. Nest host determination is updated per revised JVMS 5.4.4.
>>>>>> Access control
>>>>>> ??? check no longer throws LinkageError but instead it will throw
>>>>>> IAE with
>>>>>> ??? a clear message if a class fails to resolve/validate the nest
>>>>>> host declared
>>>>>> ??? in NestHost/NestMembers attribute.
>>>>>> 8. JFR, jcmd, JDI are updated to support hidden classes.
>>>>>> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
>>>>>> ??? and generate a bridge method to desuger a method reference to a
>>>>>> protected
>>>>>> ??? method in its supertype in a different package
>>>>>>
>>>>>> This patch also updates StringConcatFactory, LambdaMetaFactory, and
>>>>>> LambdaForms
>>>>>> to use hidden classes.? The webrev includes changes in nashorn to
>>>>>> hidden class
>>>>>> and I will update the webrev if JEP 372 removes it any time soon.
>>>>>>
>>>>>> We uncovered a bug in Lookup::defineClass spec throws LinkageError
>>>>>> and intends
>>>>>> to have the newly created class linked.? However, the
>>>>>> implementation in 14
>>>>>> does not link the class.? A separate CSR [2] proposes to update the
>>>>>> implementation to match the spec.? This patch fixes the
>>>>>> implementation.
>>>>>>
>>>>>> The spec update on JVM TI, JDI and Instrumentation will be done as
>>>>>> a separate RFE [3].? This patch includes new tests for JVM TI and
>>>>>> java.instrument that validates how the existing APIs work for
>>>>>> hidden classes.
>>>>>>
>>>>>> javadoc/specdiff
>>>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
>>>>>>
>>>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/
>>>>>>
>>>>>>
>>>>>> JVMS 5.4.4 change:
>>>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf
>>>>>>
>>>>>>
>>>>>> CSR:
>>>>>> https://bugs.openjdk.java.net/browse/JDK-8238359
>>>>>>
>>>>>> Thanks
>>>>>> Mandy
>>>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
>>>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
>>>>>> [3] https://bugs.openjdk.java.net/browse/JDK-8230502
>>>>
>>>

From mandy.chung at oracle.com  Fri Mar 27 23:46:00 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Fri, 27 Mar 2020 16:46:00 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <51d0a4cf-2565-2d61-31bb-ff43bca6b0e4@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <d3d2fa4f-4bf7-94c3-de4b-15dfe75b6370@oracle.com>
 <08191054-8d0a-ae60-ac99-e2849f08ce85@oracle.com>
 <e9ee79ca-e00a-c1d7-27a1-e110064a8fa3@oracle.com>
 <dbdeae81-7006-6922-9919-f2d253152006@oracle.com>
 <51d0a4cf-2565-2d61-31bb-ff43bca6b0e4@oracle.com>
Message-ID: <09126b64-2aa5-9b0d-ec5a-62753a416ea9@oracle.com>



On 3/27/20 4:01 PM, David Holmes wrote:
> Hi Mandy,
>
> On 28/03/2020 8:29 am, Mandy Chung wrote:
>> Hi Vicente,
>>
>> hasNestmateAccess is about VM supports static nestmates on JDK 
>> release ?>= 11.
>>
>> However this is about javac --release 14 and the compiled classes may 
>> run on JDK 14 that lambda and string concat spin classes that are not 
>> nestmates. I have a patch with Jan's help:
>>
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/8171335/webrev-javac-target-release-14/index.html 
>
>
> +???????????? /**
> +????????????? * The VM does not support access across nested classes 
> (8010319).
> +????????????? * Were that ever to change, this should be removed.
> +????????????? */
> +???????????? boolean isPrivateInOtherClass() {
>
> I'm not at all sure what this means - access across different nests? 
> (I'm not even sure what that means.)

This just reverts? the old code that I removed.

What this method is trying to determine if it accesses a private in 
another class in the same nest (nested classes) that needs a synthetic 
bridge method to access.

Mandy

From david.holmes at oracle.com  Sat Mar 28 01:15:43 2020
From: david.holmes at oracle.com (David Holmes)
Date: Sat, 28 Mar 2020 11:15:43 +1000
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <09126b64-2aa5-9b0d-ec5a-62753a416ea9@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <d3d2fa4f-4bf7-94c3-de4b-15dfe75b6370@oracle.com>
 <08191054-8d0a-ae60-ac99-e2849f08ce85@oracle.com>
 <e9ee79ca-e00a-c1d7-27a1-e110064a8fa3@oracle.com>
 <dbdeae81-7006-6922-9919-f2d253152006@oracle.com>
 <51d0a4cf-2565-2d61-31bb-ff43bca6b0e4@oracle.com>
 <09126b64-2aa5-9b0d-ec5a-62753a416ea9@oracle.com>
Message-ID: <069c76b6-dd85-f8f5-c2dd-1ba994178084@oracle.com>

Hi Mandy,

On 28/03/2020 9:46 am, Mandy Chung wrote:
> 
> 
> On 3/27/20 4:01 PM, David Holmes wrote:
>> Hi Mandy,
>>
>> On 28/03/2020 8:29 am, Mandy Chung wrote:
>>> Hi Vicente,
>>>
>>> hasNestmateAccess is about VM supports static nestmates on JDK 
>>> release ?>= 11.
>>>
>>> However this is about javac --release 14 and the compiled classes may 
>>> run on JDK 14 that lambda and string concat spin classes that are not 
>>> nestmates. I have a patch with Jan's help:
>>>
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/8171335/webrev-javac-target-release-14/index.html 
>>
>>
>> +???????????? /**
>> +????????????? * The VM does not support access across nested classes 
>> (8010319).
>> +????????????? * Were that ever to change, this should be removed.
>> +????????????? */
>> +???????????? boolean isPrivateInOtherClass() {
>>
>> I'm not at all sure what this means - access across different nests? 
>> (I'm not even sure what that means.)
> 
> This just reverts? the old code that I removed.

Ah I see. This is ancient pre-nestmate code. Can we at least fix the 
comment as it really doesn't make any sense

> What this method is trying to determine if it accesses a private in 
> another class in the same nest (nested classes) that needs a synthetic 
> bridge method to access.

That would be a good comment to add. Something like:

If compiling for a release where the VM does not support access between
nested classes, this method indicates if a synthetic bridge method is
needed for access.

Thanks,
David

> Mandy

From paul.sandoz at oracle.com  Sat Mar 28 01:39:46 2020
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Fri, 27 Mar 2020 18:39:46 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <41b99011-921a-9fc6-c738-98c47e9959c3@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <72EC6AFD-412B-4238-B98E-90AF8C185D98@oracle.com>
 <c2ae90e1-5bf8-1d6b-a739-eb4aac542790@oracle.com>
 <41b99011-921a-9fc6-c738-98c47e9959c3@oracle.com>
Message-ID: <2667FFDE-44A9-4584-BF16-897B863D89F3@oracle.com>

+1
Paul.

> On Mar 27, 2020, at 3:22 PM, Mandy Chung <mandy.chung at oracle.com> wrote:
> 
> Hi Paul,
> 
> This is the delta incorporating your comment:
>   http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03-delta-psandoz/ <http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03-delta-psandoz/>
> 
> This patch also took Alex's comment to make it clear that the hidden class is the lookup class of the returned Lookup object and drops the sentence you commented on:
> 
> On 3/27/20 1:18 PM, Mandy Chung wrote:
>>> MethodHandles.java 
>>> ? 
>>> 
>>> 1786          * (Given the {@code Lookup} object returned this method, its lookup class 
>>> 1787          * is a {@code Class} object for which {@link Class#getName()} returns a string 
>>> 1788          * that is not a binary name.) 
>>> 
>>> ? 
>>> (The {@code Lookup} object returned from this method has a lookup class that is 
>>> a {@code Class} object whose {@link Class#getName()} returns a string 
>>> that is not a binary name.) 
>>> ? 
> 
> 
> Mandy


From dean.long at oracle.com  Sat Mar 28 02:25:36 2020
From: dean.long at oracle.com (Dean Long)
Date: Fri, 27 Mar 2020 19:25:36 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
Message-ID: <13d7ba73-e49f-a55d-7e80-bd10153152a4@oracle.com>

I looked at the AOT, C2, and JVMCI changes and I didn't find any issues.

dl

On 3/26/20 4:57 PM, Mandy Chung wrote:
> Please review the implementation of JEP 371: Hidden Classes. The main 
> changes are in core-libs and hotspot runtime area. Small changes are 
> made in javac, VM compiler (intrinsification of Class::isHiddenClass), 
> JFR, JDI, and jcmd.? CSR [1]has been reviewed and is in the finalized 
> state (see specdiff and javadoc below for reference).
>
> Webrev:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03
>
> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's point
> of view, a hidden class is a normal class except the following:
>
> - A hidden class has no initiating class loader and is not registered 
> in any dictionary.
> - A hidden class has a name containing an illegal character 
> `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature` 
> returns "Lp/Foo.0x1234;".
> - A hidden class is not modifiable, i.e. cannot be redefined or 
> retransformed. JVM TI IsModifableClass returns false on a hidden.
> - Final fields in a hidden class is "final".? The value of final 
> fields cannot be overriden via reflection.? setAccessible(true) can 
> still be called on reflected objects representing final fields in a 
> hidden class and its access check will be suppressed but only have 
> read-access (i.e. can do Field::getXXX but not setXXX).
>
> Brief summary of this patch:
>
> 1. A new Lookup::defineHiddenClass method is the API to create a 
> hidden class.
> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG 
> option that
> ?? can be specified when creating a hidden class.
> 3. A new Class::isHiddenClass method tests if a class is a hidden class.
> 4. Field::setXXX method will throw IAE on a final field of a hidden class
> ?? regardless of the value of the accessible flag.
> 5. JVM_LookupDefineClass is the new JVM entry point for 
> Lookup::defineClass
> ?? and defineHiddenClass to create a class from the given bytes.
> 6. ClassLoaderData implementation is not changed.? There is one 
> primary CLD
> ?? that holds the classes strongly referenced by its defining loader.? 
> There
> ?? can be zero or more additional CLDs - one per weak class.
> 7. Nest host determination is updated per revised JVMS 5.4.4. Access 
> control
> ?? check no longer throws LinkageError but instead it will throw IAE with
> ?? a clear message if a class fails to resolve/validate the nest host 
> declared
> ?? in NestHost/NestMembers attribute.
> 8. JFR, jcmd, JDI are updated to support hidden classes.
> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
> ?? and generate a bridge method to desuger a method reference to a 
> protected
> ?? method in its supertype in a different package
>
> This patch also updates StringConcatFactory, LambdaMetaFactory, and 
> LambdaForms
> to use hidden classes.? The webrev includes changes in nashorn to 
> hidden class
> and I will update the webrev if JEP 372 removes it any time soon.
>
> We uncovered a bug in Lookup::defineClass spec throws LinkageError and 
> intends
> to have the newly created class linked.? However, the implementation in 14
> does not link the class.? A separate CSR [2] proposes to update the
> implementation to match the spec.? This patch fixes the implementation.
>
> The spec update on JVM TI, JDI and Instrumentation will be done as
> a separate RFE [3].? This patch includes new tests for JVM TI and
> java.instrument that validates how the existing APIs work for hidden 
> classes.
>
> javadoc/specdiff
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/
>
> JVMS 5.4.4 change:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf
>
> CSR:
> https://bugs.openjdk.java.net/browse/JDK-8238359
>
> Thanks
> Mandy
> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
> [3] https://bugs.openjdk.java.net/browse/JDK-8230502


From chris.plummer at oracle.com  Sat Mar 28 03:51:37 2020
From: chris.plummer at oracle.com (Chris Plummer)
Date: Fri, 27 Mar 2020 20:51:37 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
Message-ID: <90f9276d-2777-88ba-b1ec-9901711fcf02@oracle.com>

Hi Mandy,

A couple of very minor nits in the jvmtiRedefineClasses.cpp comments:

 ?153???? // classes for primitives, arrays, hidden and vm unsafe 
anonymous classes
 ?154???? // cannot be redefined.? Check here so following code can 
assume these classes
 ?155???? // are InstanceKlass.
 ?156???? if (!is_modifiable_class(mirror)) {
 ?157?????? _res = JVMTI_ERROR_UNMODIFIABLE_CLASS;
 ?158?????? return false;
 ?159???? }

I think this code and comment predate anonymous classes. Probably before 
anonymous classes the check was not for !is_modifiable_class() but 
instead was just a check for primitive or array class types since they 
are not an InstanceKlass, and would cause issues when cast to one in the 
code that lies below this section. When anonymous classes were added, 
the code got changed to use !is_modifiable_class() and the comment was 
not correctly updated (anonymous classes are an InstanceKlass). Then 
with this webrev the mention of hidden classes was added, also 
incorrectly implying they are not an InstanceKlass. I think you should 
just leave off the last sentence of the comment.

There's some ambiguity in the application of adjectives in the following:

 ?297?? // Cannot redefine or retransform a hidden or an unsafe 
anonymous class.

I'd suggest:

 ?297?? // Cannot redefine or retransform a hidden class or an unsafe 
anonymous class.

There are some places in libjdwp that need to be fixed. I spoke to 
Serguei about those this afternoon. Basically the 
convertSignatureToClassname() function needs to be fixed to handle 
hidden classes. Without the fix classname filtering will have problems 
if the filter contains a pattern with a '/' to filter on hidden classes. 
Also CLASS_UNLOAD events will not properly convert hidden class names. 
We also need tests for these cases. I think these are all things that 
can be addressed later.

I still need to look over the JVMTI tests.

thanks,

Chris

On 3/26/20 4:57 PM, Mandy Chung wrote:
> Please review the implementation of JEP 371: Hidden Classes. The main 
> changes are in core-libs and hotspot runtime area. Small changes are 
> made in javac, VM compiler (intrinsification of Class::isHiddenClass), 
> JFR, JDI, and jcmd.? CSR [1]has been reviewed and is in the finalized 
> state (see specdiff and javadoc below for reference).
>
> Webrev:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03
>
> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's point
> of view, a hidden class is a normal class except the following:
>
> - A hidden class has no initiating class loader and is not registered 
> in any dictionary.
> - A hidden class has a name containing an illegal character 
> `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature` 
> returns "Lp/Foo.0x1234;".
> - A hidden class is not modifiable, i.e. cannot be redefined or 
> retransformed. JVM TI IsModifableClass returns false on a hidden.
> - Final fields in a hidden class is "final".? The value of final 
> fields cannot be overriden via reflection.? setAccessible(true) can 
> still be called on reflected objects representing final fields in a 
> hidden class and its access check will be suppressed but only have 
> read-access (i.e. can do Field::getXXX but not setXXX).
>
> Brief summary of this patch:
>
> 1. A new Lookup::defineHiddenClass method is the API to create a 
> hidden class.
> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG 
> option that
> ?? can be specified when creating a hidden class.
> 3. A new Class::isHiddenClass method tests if a class is a hidden class.
> 4. Field::setXXX method will throw IAE on a final field of a hidden class
> ?? regardless of the value of the accessible flag.
> 5. JVM_LookupDefineClass is the new JVM entry point for 
> Lookup::defineClass
> ?? and defineHiddenClass to create a class from the given bytes.
> 6. ClassLoaderData implementation is not changed.? There is one 
> primary CLD
> ?? that holds the classes strongly referenced by its defining loader.? 
> There
> ?? can be zero or more additional CLDs - one per weak class.
> 7. Nest host determination is updated per revised JVMS 5.4.4. Access 
> control
> ?? check no longer throws LinkageError but instead it will throw IAE with
> ?? a clear message if a class fails to resolve/validate the nest host 
> declared
> ?? in NestHost/NestMembers attribute.
> 8. JFR, jcmd, JDI are updated to support hidden classes.
> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
> ?? and generate a bridge method to desuger a method reference to a 
> protected
> ?? method in its supertype in a different package
>
> This patch also updates StringConcatFactory, LambdaMetaFactory, and 
> LambdaForms
> to use hidden classes.? The webrev includes changes in nashorn to 
> hidden class
> and I will update the webrev if JEP 372 removes it any time soon.
>
> We uncovered a bug in Lookup::defineClass spec throws LinkageError and 
> intends
> to have the newly created class linked.? However, the implementation in 14
> does not link the class.? A separate CSR [2] proposes to update the
> implementation to match the spec.? This patch fixes the implementation.
>
> The spec update on JVM TI, JDI and Instrumentation will be done as
> a separate RFE [3].? This patch includes new tests for JVM TI and
> java.instrument that validates how the existing APIs work for hidden 
> classes.
>
> javadoc/specdiff
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/
>
> JVMS 5.4.4 change:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf
>
> CSR:
> https://bugs.openjdk.java.net/browse/JDK-8238359
>
> Thanks
> Mandy
> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
> [3] https://bugs.openjdk.java.net/browse/JDK-8230502



From leo.korinth at oracle.com  Mon Mar 30 00:24:51 2020
From: leo.korinth at oracle.com (Leo Korinth)
Date: Mon, 30 Mar 2020 02:24:51 +0200
Subject: RFR: 8240110: Improve NULL
Message-ID: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>

Hi, could I have a review on this change to improve NULL?

I try to:
1) make NULL only convert to pointer types for better type safety, 
__null on gcc achieves some of this, but can be casted away.
2) define NULL in one place, in a platform independent way, no need to 
define it for every tool chain.
3) make NULL be of a special type (null_t) so that one can template 
specialise against it. This is for the future!

I fail with this, but continue to read...

I try to achieve this by defining NULL as a variable of type null_t in 
the header null.hpp using ODR. null_t converts to (almost) any pointer type.

Minor negative issues:
1) No conversion to method pointers, they are only used in three places 
though, I use a literal 0 instead.
2) gcc does unfortunately not know that our NULL is a safe format 
string, only needed to change one place.
3) No constexpr yet, so I needed to use 0 to initialise a thread variable.

IMO positive issues:
1) I have fixed many bad usages of NULL for integer types and instead 
used NULL_WORD or 0.
2) Too many implicit conversions in certain places makes the code not 
compile, the changed code does look better and more idiomatic IMO.
3) Possible benign bug found where fields are initialised to NULL (not 
in constructor), this should not be possible right?
4) NULL does not convert to array types, on Windows this results in 
compile time error as the windows compiler does not decay arrays when 
comparing to null_t, this did locate one bug, and possibly some 
unnecessary comparisons in xmacros that I kept by changing NULL to 0.
5) Found a reinterpret_cast on NULL that is not allowed by the standard.

I also made a new NULL_WORD in the same way as I did NULL, but instead 
converting to a intptr_t of value 0. I was about to release NULL_WORD as 
a separate patch, but due to the current practice of defining NULL_WORD 
as NULL on many platforms, it was easier to merge these two changes into 
one. Only small issues in JVMCI code for this change, and one fix in 
sparc assembler. I find NULL_WORD questionable, but it is widely used.

Something to notice is the strange comment in the solstudio header about 
intptr_t being wider than int on some systems that is to my 
understanding false (for systems we support). I have a static assert to 
check that intptr_t is indeed of (char) pointer size. I did remove an 
assert that checks the size of NULL.

Now, the sad thing is that my change does not work fully, I can not make 
NULL constexpr before c++ 11, and thus several initialisation order bugs 
appear on global variables set to NULL. I could set them to zero or not 
set them at all, and they would work, but it feels brittle and could 
cause future bugs when people expect NULLs on globals to be initialised 
directly. Thus instead, I propose that I revert somewhat and define NULL 
as 0 and change NULL_WORD to a constant 0 of type intptr_t.

When we get c++>=11 we can define NULL to nullptr of type nullptr_t, and 
hopefully that will be a simple fix after this change.

What do you think?

Below I have given a short description of the changes:
src/hotspot/cpu/aarch64/interpreterRT_aarch64.cpp
correct the usage of integer type from NULL to NULL_WORD

src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp
too many implicit conversions

src/hotspot/cpu/arm/interpreterRT_arm.cpp
correct the usage of integer type from NULL to NULL_WORD

src/hotspot/cpu/ppc/c1_LIRAssembler_ppc.cpp
too many implicit conversions

src/hotspot/cpu/sparc/c1_CodeStubs_sparc.cpp
correct the usage of integer type from NULL to NULL_WORD

src/hotspot/cpu/sparc/c1_LIRAssembler_sparc.cpp
correct the usage of integer type from NULL to NULL_WORD, and the
opposite way as well!

src/hotspot/cpu/sparc/interpreterRT_sparc.cpp
correct the usage of integer type from NULL to NULL_WORD

src/hotspot/cpu/x86/interp_masm_x86.cpp
correct the usage of integer type from NULL to NULL_WORD

src/hotspot/cpu/x86/interpreterRT_x86_64.cpp
correct the usage of integer type from NULL to NULL_WORD

src/hotspot/cpu/x86/stubGenerator_x86_64.cpp
correct the usage of integer type from NULL to NULL_WORD

src/hotspot/os/linux/cgroupV1Subsystem_linux.hpp
correct member initialisation to be in initialise list, gcc bug?

src/hotspot/os/linux/cgroupV2Subsystem_linux.hpp
correct member initialisation to be in initialise list, gcc bug?

src/hotspot/os/solaris/attachListener_solaris.cpp
correct the usage of integer type from NULL to 0

src/hotspot/os/windows/osThread_windows.cpp
correct the usage of integer type from NULL to 0

src/hotspot/os/windows/os_windows.cpp
correct the usage of integer type from NULL to 0

src/hotspot/os/windows/pdh_interface.cpp
new NULL value does unfortunately not convert to method pointers

src/hotspot/share/classfile/javaClasses.cpp
too many implicit conversions

src/hotspot/share/classfile/moduleEntry.hpp
too many implicit conversions

src/hotspot/share/classfile/modules.cpp
too many implicit conversions

src/hotspot/share/code/codeHeapState.cpp
new NULL does not compare with arrays witch is great, maybe (xmacros) 
the NULL check can be removed, this change is safe though

src/hotspot/share/compiler/directivesParser.cpp
new NULL value does unfortunately not convert to method pointers

src/hotspot/share/gc/g1/g1ConcurrentRefine.cpp
remove unnecessary system include, might redefine NULL

src/hotspot/share/gc/parallel/psAdaptiveSizePolicy.cpp
remove unnecessary system include, might redefine NULL

src/hotspot/share/gc/parallel/psParallelCompact.cpp
remove unnecessary system include, might redefine NULL

src/hotspot/share/gc/shared/memset_with_concurrent_readers.hpp
remove unnecessary system include, might redefine NULL

src/hotspot/share/gc/z/zErrno.cpp
remove unnecessary system include, might redefine NULL

src/hotspot/share/jfr/jni/jfrGetAllEventClasses.cpp
fix bad assert found due to new NULL

src/hotspot/share/jvmci/jvmciCompilerToVM.cpp
too many implicit conversions, and misc changes

src/hotspot/share/jvmci/jvmciCompilerToVMInit.cpp
new NULL does not compare with arrays witch is great, maybe (xmacros) 
the NULL check can be removed, this change is safe though

src/hotspot/share/jvmci/jvmciExceptions.hpp
too many implicit conversions

src/hotspot/share/oops/access.hpp 
reinterpret_cast is not safe on NULL, rewrite

src/hotspot/share/oops/constantPool.cpp
too many implicit conversions

src/hotspot/share/oops/cpCache.cpp
too many implicit conversions

src/hotspot/share/oops/klass.cpp
too many implicit conversions

src/hotspot/share/oops/oopsHierarchy.hpp
add null_t overload to oop class

src/hotspot/share/prims/jni.cpp
too many implicit conversions

src/hotspot/share/prims/jvm.cpp
too many implicit conversions

src/hotspot/share/prims/methodHandles.cpp
too many implicit conversions

src/hotspot/share/runtime/os.cpp
correct the usage of integer type from NULL to 0

src/hotspot/share/runtime/os.inline.hpp
#undef NULL set by system headers, use the new NULL instead

src/hotspot/share/runtime/sweeper.hpp
gcc does unfortunately not know that our NULL is a safe format string

src/hotspot/share/runtime/thread.cpp
our conversion can not be a constexpr before c++11, use 0 instead

src/hotspot/share/utilities/debug.hpp
stop using system headers for NULL

src/hotspot/share/utilities/globalDefinitions.cpp
change to static asserts, and remove one sizeof(NULL) assert.
I will change the assert on sizeof(uintptr_t) to sizeof(NULL_WORD) in 
the next webrev as it is both wrong and not clear (I defined NULL_WORD 
to be of type intptr_t and not uintptr_t)

src/hotspot/share/utilities/globalDefinitions.hpp
include the new definition of NULL *after* system includes, also do 
redefinition after include guard end

src/hotspot/share/utilities/globalDefinitions_gcc.hpp
remove old definition of NULL

src/hotspot/share/utilities/globalDefinitions_solstudio.hpp
remove old definition of NULL

src/hotspot/share/utilities/globalDefinitions_visCPP.hpp
remove old definition of NULL

src/hotspot/share/utilities/globalDefinitions_xlc.hpp
remove old definition of NULL

src/hotspot/share/utilities/linkedlist.hpp
new NULL value does unfortunately not convert to method pointers

src/hotspot/share/utilities/null.hpp
new NULL

test/hotspot/gtest/gc/z/test_zAddress.cpp
correct the usage of integer type from NULL to NULL_WORD

test/hotspot/gtest/unittest.hpp
do not use system NULL

I have a follow up patch that tries to (totally) remove usage of system 
header NULL, but that is orthogonal to this patch so I did not want to 
mix them if the approach is not liked.

Bug:
https://bugs.openjdk.java.net/browse/JDK-8240110

Webrev:
http://cr.openjdk.java.net/~lkorinth/8240110/part1 (NULL)
http://cr.openjdk.java.net/~lkorinth/8240110/part2 (NULL_WORD)
http://cr.openjdk.java.net/~lkorinth/8240110/part3 (reverse somewhat)
http://cr.openjdk.java.net/~lkorinth/8240110/0     (combined change that 
I will create incremental changes against)

Testing:
tier 1-3

Thanks,
Leo

From david.holmes at oracle.com  Mon Mar 30 01:05:21 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 30 Mar 2020 11:05:21 +1000
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
Message-ID: <98cd4998-58b8-60d1-b7a1-b4ecfbb89afc@oracle.com>

Hi Leo,

A couple of initial responses before looking at the code ...

On 30/03/2020 10:24 am, Leo Korinth wrote:
> Hi, could I have a review on this change to improve NULL?
> 
> I try to:
> 1) make NULL only convert to pointer types for better type safety, 
> __null on gcc achieves some of this, but can be casted away.
> 2) define NULL in one place, in a platform independent way, no need to 
> define it for every tool chain.

It will be good if that is the case. The reason it was defined for each 
tool chain was because differnet toolchains had different expectation 
and behaviours and quirks when it came to defining NULL. Of course that 
is mainly ancient history.

> 3) make NULL be of a special type (null_t) so that one can template 
> specialise against it. This is for the future!
> 
> I fail with this, but continue to read...
> 
> I try to achieve this by defining NULL as a variable of type null_t in 
> the header null.hpp using ODR. null_t converts to (almost) any pointer 
> type.
> 
> Minor negative issues:
> 1) No conversion to method pointers, they are only used in three places 
> though, I use a literal 0 instead.
> 2) gcc does unfortunately not know that our NULL is a safe format 
> string, only needed to change one place.
> 3) No constexpr yet, so I needed to use 0 to initialise a thread variable.
> 
> IMO positive issues:
> 1) I have fixed many bad usages of NULL for integer types and instead 
> used NULL_WORD or 0.
> 2) Too many implicit conversions in certain places makes the code not 
> compile, the changed code does look better and more idiomatic IMO.
> 3) Possible benign bug found where fields are initialised to NULL (not 
> in constructor), this should not be possible right?

We have some special classes that we do not allocate and construct using 
the C++ facilities. For example, hastable entry types like 
ResolutionErrorEntry. These do not have a constructor run and the fields 
are initialised directly following the allocation. ObjectMonitors are 
another special case where we block allocate and then run an init function.

> 4) NULL does not convert to array types, on Windows this results in 
> compile time error as the windows compiler does not decay arrays when 
> comparing to null_t, this did locate one bug, and possibly some 
> unnecessary comparisons in xmacros that I kept by changing NULL to 0.
> 5) Found a reinterpret_cast on NULL that is not allowed by the standard.
> 
> I also made a new NULL_WORD in the same way as I did NULL, but instead 
> converting to a intptr_t of value 0. I was about to release NULL_WORD as 
> a separate patch, but due to the current practice of defining NULL_WORD 
> as NULL on many platforms, it was easier to merge these two changes into 
> one. Only small issues in JVMCI code for this change, and one fix in 
> sparc assembler. I find NULL_WORD questionable, but it is widely used.
> 
> Something to notice is the strange comment in the solstudio header about 
> intptr_t being wider than int on some systems that is to my 
> understanding false (for systems we support). I have a static assert to 
> check that intptr_t is indeed of (char) pointer size. I did remove an 
> assert that checks the size of NULL.

This refers to 32-bit systems: int == 32bits; ptr == 64-bits; intptr_t 
== 64-bits

Cheers,
David
-----

> Now, the sad thing is that my change does not work fully, I can not make 
> NULL constexpr before c++ 11, and thus several initialisation order bugs 
> appear on global variables set to NULL. I could set them to zero or not 
> set them at all, and they would work, but it feels brittle and could 
> cause future bugs when people expect NULLs on globals to be initialised 
> directly. Thus instead, I propose that I revert somewhat and define NULL 
> as 0 and change NULL_WORD to a constant 0 of type intptr_t.
> 
> When we get c++>=11 we can define NULL to nullptr of type nullptr_t, and 
> hopefully that will be a simple fix after this change.
> 
> What do you think?
> 
> Below I have given a short description of the changes:
> src/hotspot/cpu/aarch64/interpreterRT_aarch64.cpp
> correct the usage of integer type from NULL to NULL_WORD
> 
> src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp
> too many implicit conversions
> 
> src/hotspot/cpu/arm/interpreterRT_arm.cpp
> correct the usage of integer type from NULL to NULL_WORD
> 
> src/hotspot/cpu/ppc/c1_LIRAssembler_ppc.cpp
> too many implicit conversions
> 
> src/hotspot/cpu/sparc/c1_CodeStubs_sparc.cpp
> correct the usage of integer type from NULL to NULL_WORD
> 
> src/hotspot/cpu/sparc/c1_LIRAssembler_sparc.cpp
> correct the usage of integer type from NULL to NULL_WORD, and the
> opposite way as well!
> 
> src/hotspot/cpu/sparc/interpreterRT_sparc.cpp
> correct the usage of integer type from NULL to NULL_WORD
> 
> src/hotspot/cpu/x86/interp_masm_x86.cpp
> correct the usage of integer type from NULL to NULL_WORD
> 
> src/hotspot/cpu/x86/interpreterRT_x86_64.cpp
> correct the usage of integer type from NULL to NULL_WORD
> 
> src/hotspot/cpu/x86/stubGenerator_x86_64.cpp
> correct the usage of integer type from NULL to NULL_WORD
> 
> src/hotspot/os/linux/cgroupV1Subsystem_linux.hpp
> correct member initialisation to be in initialise list, gcc bug?
> 
> src/hotspot/os/linux/cgroupV2Subsystem_linux.hpp
> correct member initialisation to be in initialise list, gcc bug?
> 
> src/hotspot/os/solaris/attachListener_solaris.cpp
> correct the usage of integer type from NULL to 0
> 
> src/hotspot/os/windows/osThread_windows.cpp
> correct the usage of integer type from NULL to 0
> 
> src/hotspot/os/windows/os_windows.cpp
> correct the usage of integer type from NULL to 0
> 
> src/hotspot/os/windows/pdh_interface.cpp
> new NULL value does unfortunately not convert to method pointers
> 
> src/hotspot/share/classfile/javaClasses.cpp
> too many implicit conversions
> 
> src/hotspot/share/classfile/moduleEntry.hpp
> too many implicit conversions
> 
> src/hotspot/share/classfile/modules.cpp
> too many implicit conversions
> 
> src/hotspot/share/code/codeHeapState.cpp
> new NULL does not compare with arrays witch is great, maybe (xmacros) 
> the NULL check can be removed, this change is safe though
> 
> src/hotspot/share/compiler/directivesParser.cpp
> new NULL value does unfortunately not convert to method pointers
> 
> src/hotspot/share/gc/g1/g1ConcurrentRefine.cpp
> remove unnecessary system include, might redefine NULL
> 
> src/hotspot/share/gc/parallel/psAdaptiveSizePolicy.cpp
> remove unnecessary system include, might redefine NULL
> 
> src/hotspot/share/gc/parallel/psParallelCompact.cpp
> remove unnecessary system include, might redefine NULL
> 
> src/hotspot/share/gc/shared/memset_with_concurrent_readers.hpp
> remove unnecessary system include, might redefine NULL
> 
> src/hotspot/share/gc/z/zErrno.cpp
> remove unnecessary system include, might redefine NULL
> 
> src/hotspot/share/jfr/jni/jfrGetAllEventClasses.cpp
> fix bad assert found due to new NULL
> 
> src/hotspot/share/jvmci/jvmciCompilerToVM.cpp
> too many implicit conversions, and misc changes
> 
> src/hotspot/share/jvmci/jvmciCompilerToVMInit.cpp
> new NULL does not compare with arrays witch is great, maybe (xmacros) 
> the NULL check can be removed, this change is safe though
> 
> src/hotspot/share/jvmci/jvmciExceptions.hpp
> too many implicit conversions
> 
> src/hotspot/share/oops/access.hpp reinterpret_cast is not safe on NULL, 
> rewrite
> 
> src/hotspot/share/oops/constantPool.cpp
> too many implicit conversions
> 
> src/hotspot/share/oops/cpCache.cpp
> too many implicit conversions
> 
> src/hotspot/share/oops/klass.cpp
> too many implicit conversions
> 
> src/hotspot/share/oops/oopsHierarchy.hpp
> add null_t overload to oop class
> 
> src/hotspot/share/prims/jni.cpp
> too many implicit conversions
> 
> src/hotspot/share/prims/jvm.cpp
> too many implicit conversions
> 
> src/hotspot/share/prims/methodHandles.cpp
> too many implicit conversions
> 
> src/hotspot/share/runtime/os.cpp
> correct the usage of integer type from NULL to 0
> 
> src/hotspot/share/runtime/os.inline.hpp
> #undef NULL set by system headers, use the new NULL instead
> 
> src/hotspot/share/runtime/sweeper.hpp
> gcc does unfortunately not know that our NULL is a safe format string
> 
> src/hotspot/share/runtime/thread.cpp
> our conversion can not be a constexpr before c++11, use 0 instead
> 
> src/hotspot/share/utilities/debug.hpp
> stop using system headers for NULL
> 
> src/hotspot/share/utilities/globalDefinitions.cpp
> change to static asserts, and remove one sizeof(NULL) assert.
> I will change the assert on sizeof(uintptr_t) to sizeof(NULL_WORD) in 
> the next webrev as it is both wrong and not clear (I defined NULL_WORD 
> to be of type intptr_t and not uintptr_t)
> 
> src/hotspot/share/utilities/globalDefinitions.hpp
> include the new definition of NULL *after* system includes, also do 
> redefinition after include guard end
> 
> src/hotspot/share/utilities/globalDefinitions_gcc.hpp
> remove old definition of NULL
> 
> src/hotspot/share/utilities/globalDefinitions_solstudio.hpp
> remove old definition of NULL
> 
> src/hotspot/share/utilities/globalDefinitions_visCPP.hpp
> remove old definition of NULL
> 
> src/hotspot/share/utilities/globalDefinitions_xlc.hpp
> remove old definition of NULL
> 
> src/hotspot/share/utilities/linkedlist.hpp
> new NULL value does unfortunately not convert to method pointers
> 
> src/hotspot/share/utilities/null.hpp
> new NULL
> 
> test/hotspot/gtest/gc/z/test_zAddress.cpp
> correct the usage of integer type from NULL to NULL_WORD
> 
> test/hotspot/gtest/unittest.hpp
> do not use system NULL
> 
> I have a follow up patch that tries to (totally) remove usage of system 
> header NULL, but that is orthogonal to this patch so I did not want to 
> mix them if the approach is not liked.
> 
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8240110
> 
> Webrev:
> http://cr.openjdk.java.net/~lkorinth/8240110/part1 (NULL)
> http://cr.openjdk.java.net/~lkorinth/8240110/part2 (NULL_WORD)
> http://cr.openjdk.java.net/~lkorinth/8240110/part3 (reverse somewhat)
> http://cr.openjdk.java.net/~lkorinth/8240110/0???? (combined change that 
> I will create incremental changes against)
> 
> Testing:
> tier 1-3
> 
> Thanks,
> Leo

From mandy.chung at oracle.com  Mon Mar 30 02:17:27 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Sun, 29 Mar 2020 19:17:27 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <90f9276d-2777-88ba-b1ec-9901711fcf02@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <90f9276d-2777-88ba-b1ec-9901711fcf02@oracle.com>
Message-ID: <c56eba10-13eb-aa86-11a3-f2dfd1a7b0b0@oracle.com>



On 3/27/20 8:51 PM, Chris Plummer wrote:
> Hi Mandy,
>
> A couple of very minor nits in the jvmtiRedefineClasses.cpp comments:
>
> ?153???? // classes for primitives, arrays, hidden and vm unsafe 
> anonymous classes
> ?154???? // cannot be redefined.? Check here so following code can 
> assume these classes
> ?155???? // are InstanceKlass.
> ?156???? if (!is_modifiable_class(mirror)) {
> ?157?????? _res = JVMTI_ERROR_UNMODIFIABLE_CLASS;
> ?158?????? return false;
> ?159???? }
>
> I think this code and comment predate anonymous classes. Probably 
> before anonymous classes the check was not for !is_modifiable_class() 
> but instead was just a check for primitive or array class types since 
> they are not an InstanceKlass, and would cause issues when cast to one 
> in the code that lies below this section. When anonymous classes were 
> added, the code got changed to use !is_modifiable_class() and the 
> comment was not correctly updated (anonymous classes are an 
> InstanceKlass). Then with this webrev the mention of hidden classes 
> was added, also incorrectly implying they are not an InstanceKlass. I 
> think you should just leave off the last sentence of the comment.
>

I agree with you that this comment needs update.?? Perhaps it should say 
"primitive, array types and hidden classes are non-modifiable. A 
modifiable class must be an InstanceKlass."

I leave it to Serguei who may have other opinion.

> There's some ambiguity in the application of adjectives in the following:
>
> ?297?? // Cannot redefine or retransform a hidden or an unsafe 
> anonymous class.
>
> I'd suggest:
>
> ?297?? // Cannot redefine or retransform a hidden class or an unsafe 
> anonymous class.
>

+1

> There are some places in libjdwp that need to be fixed. I spoke to 
> Serguei about those this afternoon. Basically the 
> convertSignatureToClassname() function needs to be fixed to handle 
> hidden classes. Without the fix classname filtering will have problems 
> if the filter contains a pattern with a '/' to filter on hidden 
> classes. Also CLASS_UNLOAD events will not properly convert hidden 
> class names. We also need tests for these cases. I think these are all 
> things that can be addressed later.
>

Good catch.? I have created a subtask under JDK-8230502:
 ?? https://bugs.openjdk.java.net/browse/JDK-8230502

> I still need to look over the JVMTI tests.
>

Thanks
Mandy
> thanks,
>
> Chris
>
> On 3/26/20 4:57 PM, Mandy Chung wrote:
>> Please review the implementation of JEP 371: Hidden Classes. The main 
>> changes are in core-libs and hotspot runtime area. Small changes are 
>> made in javac, VM compiler (intrinsification of 
>> Class::isHiddenClass), JFR, JDI, and jcmd.? CSR [1]has been reviewed 
>> and is in the finalized state (see specdiff and javadoc below for 
>> reference).
>>
>> Webrev:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
>>
>>
>> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's 
>> point
>> of view, a hidden class is a normal class except the following:
>>
>> - A hidden class has no initiating class loader and is not registered 
>> in any dictionary.
>> - A hidden class has a name containing an illegal character 
>> `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature` 
>> returns "Lp/Foo.0x1234;".
>> - A hidden class is not modifiable, i.e. cannot be redefined or 
>> retransformed. JVM TI IsModifableClass returns false on a hidden.
>> - Final fields in a hidden class is "final".? The value of final 
>> fields cannot be overriden via reflection.? setAccessible(true) can 
>> still be called on reflected objects representing final fields in a 
>> hidden class and its access check will be suppressed but only have 
>> read-access (i.e. can do Field::getXXX but not setXXX).
>>
>> Brief summary of this patch:
>>
>> 1. A new Lookup::defineHiddenClass method is the API to create a 
>> hidden class.
>> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG 
>> option that
>> ?? can be specified when creating a hidden class.
>> 3. A new Class::isHiddenClass method tests if a class is a hidden class.
>> 4. Field::setXXX method will throw IAE on a final field of a hidden 
>> class
>> ?? regardless of the value of the accessible flag.
>> 5. JVM_LookupDefineClass is the new JVM entry point for 
>> Lookup::defineClass
>> ?? and defineHiddenClass to create a class from the given bytes.
>> 6. ClassLoaderData implementation is not changed.? There is one 
>> primary CLD
>> ?? that holds the classes strongly referenced by its defining 
>> loader.? There
>> ?? can be zero or more additional CLDs - one per weak class.
>> 7. Nest host determination is updated per revised JVMS 5.4.4. Access 
>> control
>> ?? check no longer throws LinkageError but instead it will throw IAE 
>> with
>> ?? a clear message if a class fails to resolve/validate the nest host 
>> declared
>> ?? in NestHost/NestMembers attribute.
>> 8. JFR, jcmd, JDI are updated to support hidden classes.
>> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
>> ?? and generate a bridge method to desuger a method reference to a 
>> protected
>> ?? method in its supertype in a different package
>>
>> This patch also updates StringConcatFactory, LambdaMetaFactory, and 
>> LambdaForms
>> to use hidden classes.? The webrev includes changes in nashorn to 
>> hidden class
>> and I will update the webrev if JEP 372 removes it any time soon.
>>
>> We uncovered a bug in Lookup::defineClass spec throws LinkageError 
>> and intends
>> to have the newly created class linked.? However, the implementation 
>> in 14
>> does not link the class.? A separate CSR [2] proposes to update the
>> implementation to match the spec.? This patch fixes the implementation.
>>
>> The spec update on JVM TI, JDI and Instrumentation will be done as
>> a separate RFE [3].? This patch includes new tests for JVM TI and
>> java.instrument that validates how the existing APIs work for hidden 
>> classes.
>>
>> javadoc/specdiff
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
>>
>>
>> JVMS 5.4.4 change:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
>>
>>
>> CSR:
>> https://bugs.openjdk.java.net/browse/JDK-8238359
>>
>> Thanks
>> Mandy
>> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
>> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
>> [3] https://bugs.openjdk.java.net/browse/JDK-8230502
>
>


From serguei.spitsyn at oracle.com  Mon Mar 30 03:40:43 2020
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Sun, 29 Mar 2020 20:40:43 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <c56eba10-13eb-aa86-11a3-f2dfd1a7b0b0@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <90f9276d-2777-88ba-b1ec-9901711fcf02@oracle.com>
 <c56eba10-13eb-aa86-11a3-f2dfd1a7b0b0@oracle.com>
Message-ID: <7fe9fd65-8bee-beb3-03af-ab56120a4cc1@oracle.com>

Hi Mandy and Chris,


On 3/29/20 19:17, Mandy Chung wrote:
>
>
> On 3/27/20 8:51 PM, Chris Plummer wrote:
>> Hi Mandy,
>>
>> A couple of very minor nits in the jvmtiRedefineClasses.cpp comments:
>>
>> ?153???? // classes for primitives, arrays, hidden and vm unsafe 
>> anonymous classes
>> ?154???? // cannot be redefined.? Check here so following code can 
>> assume these classes
>> ?155???? // are InstanceKlass.
>> ?156???? if (!is_modifiable_class(mirror)) {
>> ?157?????? _res = JVMTI_ERROR_UNMODIFIABLE_CLASS;
>> ?158?????? return false;
>> ?159???? }
>>
>> I think this code and comment predate anonymous classes. Probably 
>> before anonymous classes the check was not for !is_modifiable_class() 
>> but instead was just a check for primitive or array class types since 
>> they are not an InstanceKlass, and would cause issues when cast to 
>> one in the code that lies below this section. When anonymous classes 
>> were added, the code got changed to use !is_modifiable_class() and 
>> the comment was not correctly updated (anonymous classes are an 
>> InstanceKlass). Then with this webrev the mention of hidden classes 
>> was added, also incorrectly implying they are not an InstanceKlass. I 
>> think you should just leave off the last sentence of the comment.
>>
>
> I agree with you that this comment needs update.?? Perhaps it should 
> say "primitive, array types and hidden classes are non-modifiable. A 
> modifiable class must be an InstanceKlass."
>
> I leave it to Serguei who may have other opinion.

We already had a chat with Chris about this.
This suggestion looks right.


>> There's some ambiguity in the application of adjectives in the 
>> following:
>>
>> ?297?? // Cannot redefine or retransform a hidden or an unsafe 
>> anonymous class.
>>
>> I'd suggest:
>>
>> ?297?? // Cannot redefine or retransform a hidden class or an unsafe 
>> anonymous class.
>>
>
> +1

+1

>> There are some places in libjdwp that need to be fixed. I spoke to 
>> Serguei about those this afternoon. Basically the 
>> convertSignatureToClassname() function needs to be fixed to handle 
>> hidden classes. Without the fix classname filtering will have 
>> problems if the filter contains a pattern with a '/' to filter on 
>> hidden classes. Also CLASS_UNLOAD events will not properly convert 
>> hidden class names. We also need tests for these cases. I think these 
>> are all things that can be addressed later.
>>
>
> Good catch.? I have created a subtask under JDK-8230502:
> ?? https://bugs.openjdk.java.net/browse/JDK-8230502

Yes, it is good catch. Thank you for filing the subtask.
We discussed this with Chris.
This was expected to be found with new test coverage and fixed in the 
JDI chunk of work which we have decided to separate from JEP 371.


Thanks,
Serguei

>> I still need to look over the JVMTI tests.
>>
>
> Thanks
> Mandy
>> thanks,
>>
>> Chris
>>
>> On 3/26/20 4:57 PM, Mandy Chung wrote:
>>> Please review the implementation of JEP 371: Hidden Classes. The 
>>> main changes are in core-libs and hotspot runtime area. Small 
>>> changes are made in javac, VM compiler (intrinsification of 
>>> Class::isHiddenClass), JFR, JDI, and jcmd.? CSR [1]has been reviewed 
>>> and is in the finalized state (see specdiff and javadoc below for 
>>> reference).
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
>>>
>>>
>>> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's 
>>> point
>>> of view, a hidden class is a normal class except the following:
>>>
>>> - A hidden class has no initiating class loader and is not 
>>> registered in any dictionary.
>>> - A hidden class has a name containing an illegal character 
>>> `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature` 
>>> returns "Lp/Foo.0x1234;".
>>> - A hidden class is not modifiable, i.e. cannot be redefined or 
>>> retransformed. JVM TI IsModifableClass returns false on a hidden.
>>> - Final fields in a hidden class is "final".? The value of final 
>>> fields cannot be overriden via reflection. setAccessible(true) can 
>>> still be called on reflected objects representing final fields in a 
>>> hidden class and its access check will be suppressed but only have 
>>> read-access (i.e. can do Field::getXXX but not setXXX).
>>>
>>> Brief summary of this patch:
>>>
>>> 1. A new Lookup::defineHiddenClass method is the API to create a 
>>> hidden class.
>>> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG 
>>> option that
>>> ?? can be specified when creating a hidden class.
>>> 3. A new Class::isHiddenClass method tests if a class is a hidden 
>>> class.
>>> 4. Field::setXXX method will throw IAE on a final field of a hidden 
>>> class
>>> ?? regardless of the value of the accessible flag.
>>> 5. JVM_LookupDefineClass is the new JVM entry point for 
>>> Lookup::defineClass
>>> ?? and defineHiddenClass to create a class from the given bytes.
>>> 6. ClassLoaderData implementation is not changed.? There is one 
>>> primary CLD
>>> ?? that holds the classes strongly referenced by its defining 
>>> loader.? There
>>> ?? can be zero or more additional CLDs - one per weak class.
>>> 7. Nest host determination is updated per revised JVMS 5.4.4. Access 
>>> control
>>> ?? check no longer throws LinkageError but instead it will throw IAE 
>>> with
>>> ?? a clear message if a class fails to resolve/validate the nest 
>>> host declared
>>> ?? in NestHost/NestMembers attribute.
>>> 8. JFR, jcmd, JDI are updated to support hidden classes.
>>> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
>>> ?? and generate a bridge method to desuger a method reference to a 
>>> protected
>>> ?? method in its supertype in a different package
>>>
>>> This patch also updates StringConcatFactory, LambdaMetaFactory, and 
>>> LambdaForms
>>> to use hidden classes.? The webrev includes changes in nashorn to 
>>> hidden class
>>> and I will update the webrev if JEP 372 removes it any time soon.
>>>
>>> We uncovered a bug in Lookup::defineClass spec throws LinkageError 
>>> and intends
>>> to have the newly created class linked.? However, the implementation 
>>> in 14
>>> does not link the class.? A separate CSR [2] proposes to update the
>>> implementation to match the spec.? This patch fixes the implementation.
>>>
>>> The spec update on JVM TI, JDI and Instrumentation will be done as
>>> a separate RFE [3].? This patch includes new tests for JVM TI and
>>> java.instrument that validates how the existing APIs work for hidden 
>>> classes.
>>>
>>> javadoc/specdiff
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
>>>
>>>
>>> JVMS 5.4.4 change:
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
>>>
>>>
>>> CSR:
>>> https://bugs.openjdk.java.net/browse/JDK-8238359
>>>
>>> Thanks
>>> Mandy
>>> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
>>> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
>>> [3] https://bugs.openjdk.java.net/browse/JDK-8230502
>>
>>
>


From serguei.spitsyn at oracle.com  Mon Mar 30 09:30:54 2020
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Mon, 30 Mar 2020 02:30:54 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
Message-ID: <ec8b6266-3eff-6d4a-987a-d46013c715e1@oracle.com>

Hi Mandy,

I have just one comment so far.

http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03/src/hotspot/share/classfile/classLoaderHierarchyDCmd.cpp.frames.html

 ?356?? void add_classes(LoadedClassInfo* first_class, int num_classes, 
bool has_class_mirror_holder) {
 ?357???? LoadedClassInfo** p_list_to_add_to;
 ?358???? bool is_hidden = first_class->_klass->is_hidden();
 ?359???? if (has_class_mirror_holder) {
 ?360?????? p_list_to_add_to = is_hidden ? &_hidden_weak_classes : 
&_anon_classes;
 ?361???? } else {
 ?362?????? p_list_to_add_to = &_classes;
 ?363???? }
 ?364???? // Search tail.
 ?365???? while ((*p_list_to_add_to) != NULL) {
 ?366?????? p_list_to_add_to = &(*p_list_to_add_to)->_next;
 ?367???? }
 ?368???? *p_list_to_add_to = first_class;
 ?369???? if (has_class_mirror_holder) {
 ?370?????? if (is_hidden) {
 ?371???????? _num_hidden_weak_classes += num_classes;
 ?372?????? } else {
 ?373???????? _num_anon_classes += num_classes;
 ?374?????? }
 ?375???? } else {
 ?376?????? _num_classes += num_classes;
 ?377???? }
 ?378?? }

 ?Q1: I'm just curious, what happens if a cld has arrays of hidden classes?
 ???? Is the bottom_klass always expected to be the first?


Thanks,
Serguei


On 3/26/20 16:57, Mandy Chung wrote:
> Please review the implementation of JEP 371: Hidden Classes. The main 
> changes are in core-libs and hotspot runtime area.? Small changes are 
> made in javac, VM compiler (intrinsification of Class::isHiddenClass), 
> JFR, JDI, and jcmd.? CSR [1]has been reviewed and is in the finalized 
> state (see specdiff and javadoc below for reference).
>
> Webrev:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
>
>
> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's point
> of view, a hidden class is a normal class except the following:
>
> - A hidden class has no initiating class loader and is not registered 
> in any dictionary.
> - A hidden class has a name containing an illegal character 
> `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature` 
> returns "Lp/Foo.0x1234;".
> - A hidden class is not modifiable, i.e. cannot be redefined or 
> retransformed. JVM TI IsModifableClass returns false on a hidden.
> - Final fields in a hidden class is "final".? The value of final 
> fields cannot be overriden via reflection.? setAccessible(true) can 
> still be called on reflected objects representing final fields in a 
> hidden class and its access check will be suppressed but only have 
> read-access (i.e. can do Field::getXXX but not setXXX).
>
> Brief summary of this patch:
>
> 1. A new Lookup::defineHiddenClass method is the API to create a 
> hidden class.
> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG 
> option that
> ?? can be specified when creating a hidden class.
> 3. A new Class::isHiddenClass method tests if a class is a hidden class.
> 4. Field::setXXX method will throw IAE on a final field of a hidden class
> ?? regardless of the value of the accessible flag.
> 5. JVM_LookupDefineClass is the new JVM entry point for 
> Lookup::defineClass
> ?? and defineHiddenClass to create a class from the given bytes.
> 6. ClassLoaderData implementation is not changed.? There is one 
> primary CLD
> ?? that holds the classes strongly referenced by its defining loader.? 
> There
> ?? can be zero or more additional CLDs - one per weak class.
> 7. Nest host determination is updated per revised JVMS 5.4.4. Access 
> control
> ?? check no longer throws LinkageError but instead it will throw IAE with
> ?? a clear message if a class fails to resolve/validate the nest host 
> declared
> ?? in NestHost/NestMembers attribute.
> 8. JFR, jcmd, JDI are updated to support hidden classes.
> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
> ?? and generate a bridge method to desuger a method reference to a 
> protected
> ?? method in its supertype in a different package
>
> This patch also updates StringConcatFactory, LambdaMetaFactory, and 
> LambdaForms
> to use hidden classes.? The webrev includes changes in nashorn to 
> hidden class
> and I will update the webrev if JEP 372 removes it any time soon.
>
> We uncovered a bug in Lookup::defineClass spec throws LinkageError and 
> intends
> to have the newly created class linked.? However, the implementation 
> in 14
> does not link the class.? A separate CSR [2] proposes to update the
> implementation to match the spec.? This patch fixes the implementation.
>
> The spec update on JVM TI, JDI and Instrumentation will be done as
> a separate RFE [3].? This patch includes new tests for JVM TI and
> java.instrument that validates how the existing APIs work for hidden 
> classes.
>
> javadoc/specdiff
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
>
>
> JVMS 5.4.4 change:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
>
>
> CSR:
> https://bugs.openjdk.java.net/browse/JDK-8238359
>
> Thanks
> Mandy
> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
> [3] https://bugs.openjdk.java.net/browse/JDK-8230502


From david.holmes at oracle.com  Mon Mar 30 09:54:51 2020
From: david.holmes at oracle.com (David Holmes)
Date: Mon, 30 Mar 2020 19:54:51 +1000
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <ec8b6266-3eff-6d4a-987a-d46013c715e1@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <ec8b6266-3eff-6d4a-987a-d46013c715e1@oracle.com>
Message-ID: <546cf8e4-00e4-1d22-6402-6620b8d7b2db@oracle.com>

Sorry to jump in on this but it caught my eye though I may be missing a 
larger context ...

On 30/03/2020 7:30 pm, serguei.spitsyn at oracle.com wrote:
> Hi Mandy,
> 
> I have just one comment so far.
> 
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03/src/hotspot/share/classfile/classLoaderHierarchyDCmd.cpp.frames.html 
> 
> 
>  ?356?? void add_classes(LoadedClassInfo* first_class, int num_classes, 
> bool has_class_mirror_holder) {
>  ?357???? LoadedClassInfo** p_list_to_add_to;
>  ?358???? bool is_hidden = first_class->_klass->is_hidden();
>  ?359???? if (has_class_mirror_holder) {
>  ?360?????? p_list_to_add_to = is_hidden ? &_hidden_weak_classes : 
> &_anon_classes;
>  ?361???? } else {
>  ?362?????? p_list_to_add_to = &_classes;
>  ?363???? }
>  ?364???? // Search tail.
>  ?365???? while ((*p_list_to_add_to) != NULL) {
>  ?366?????? p_list_to_add_to = &(*p_list_to_add_to)->_next;
>  ?367???? }
>  ?368???? *p_list_to_add_to = first_class;
>  ?369???? if (has_class_mirror_holder) {
>  ?370?????? if (is_hidden) {
>  ?371???????? _num_hidden_weak_classes += num_classes;

Why does hidden imply weak here?

David
-----

>  ?372?????? } else {
>  ?373???????? _num_anon_classes += num_classes;
>  ?374?????? }
>  ?375???? } else {
>  ?376?????? _num_classes += num_classes;
>  ?377???? }
>  ?378?? }
> 
>  ?Q1: I'm just curious, what happens if a cld has arrays of hidden classes?
>  ???? Is the bottom_klass always expected to be the first?
> 
> 
> Thanks,
> Serguei
> 
> 
> On 3/26/20 16:57, Mandy Chung wrote:
>> Please review the implementation of JEP 371: Hidden Classes. The main 
>> changes are in core-libs and hotspot runtime area.? Small changes are 
>> made in javac, VM compiler (intrinsification of Class::isHiddenClass), 
>> JFR, JDI, and jcmd.? CSR [1]has been reviewed and is in the finalized 
>> state (see specdiff and javadoc below for reference).
>>
>> Webrev:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
>>
>>
>> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's point
>> of view, a hidden class is a normal class except the following:
>>
>> - A hidden class has no initiating class loader and is not registered 
>> in any dictionary.
>> - A hidden class has a name containing an illegal character 
>> `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature` 
>> returns "Lp/Foo.0x1234;".
>> - A hidden class is not modifiable, i.e. cannot be redefined or 
>> retransformed. JVM TI IsModifableClass returns false on a hidden.
>> - Final fields in a hidden class is "final".? The value of final 
>> fields cannot be overriden via reflection.? setAccessible(true) can 
>> still be called on reflected objects representing final fields in a 
>> hidden class and its access check will be suppressed but only have 
>> read-access (i.e. can do Field::getXXX but not setXXX).
>>
>> Brief summary of this patch:
>>
>> 1. A new Lookup::defineHiddenClass method is the API to create a 
>> hidden class.
>> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG 
>> option that
>> ?? can be specified when creating a hidden class.
>> 3. A new Class::isHiddenClass method tests if a class is a hidden class.
>> 4. Field::setXXX method will throw IAE on a final field of a hidden class
>> ?? regardless of the value of the accessible flag.
>> 5. JVM_LookupDefineClass is the new JVM entry point for 
>> Lookup::defineClass
>> ?? and defineHiddenClass to create a class from the given bytes.
>> 6. ClassLoaderData implementation is not changed.? There is one 
>> primary CLD
>> ?? that holds the classes strongly referenced by its defining loader. 
>> There
>> ?? can be zero or more additional CLDs - one per weak class.
>> 7. Nest host determination is updated per revised JVMS 5.4.4. Access 
>> control
>> ?? check no longer throws LinkageError but instead it will throw IAE with
>> ?? a clear message if a class fails to resolve/validate the nest host 
>> declared
>> ?? in NestHost/NestMembers attribute.
>> 8. JFR, jcmd, JDI are updated to support hidden classes.
>> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
>> ?? and generate a bridge method to desuger a method reference to a 
>> protected
>> ?? method in its supertype in a different package
>>
>> This patch also updates StringConcatFactory, LambdaMetaFactory, and 
>> LambdaForms
>> to use hidden classes.? The webrev includes changes in nashorn to 
>> hidden class
>> and I will update the webrev if JEP 372 removes it any time soon.
>>
>> We uncovered a bug in Lookup::defineClass spec throws LinkageError and 
>> intends
>> to have the newly created class linked.? However, the implementation 
>> in 14
>> does not link the class.? A separate CSR [2] proposes to update the
>> implementation to match the spec.? This patch fixes the implementation.
>>
>> The spec update on JVM TI, JDI and Instrumentation will be done as
>> a separate RFE [3].? This patch includes new tests for JVM TI and
>> java.instrument that validates how the existing APIs work for hidden 
>> classes.
>>
>> javadoc/specdiff
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
>>
>>
>> JVMS 5.4.4 change:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
>>
>>
>> CSR:
>> https://bugs.openjdk.java.net/browse/JDK-8238359
>>
>> Thanks
>> Mandy
>> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
>> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
>> [3] https://bugs.openjdk.java.net/browse/JDK-8230502
> 

From harold.seigel at oracle.com  Mon Mar 30 12:31:13 2020
From: harold.seigel at oracle.com (Harold Seigel)
Date: Mon, 30 Mar 2020 08:31:13 -0400
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <546cf8e4-00e4-1d22-6402-6620b8d7b2db@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <ec8b6266-3eff-6d4a-987a-d46013c715e1@oracle.com>
 <546cf8e4-00e4-1d22-6402-6620b8d7b2db@oracle.com>
Message-ID: <90ca2946-05cb-dad5-15dc-ccdea1bea91b@oracle.com>

Hi David,

Hidden implies weak at lines 360 and 371 because only weak hidden 
classes can have ClassLoaderData's where has_class_mirror_holder is true.

Thanks, Harold

On 3/30/2020 5:54 AM, David Holmes wrote:
> Sorry to jump in on this but it caught my eye though I may be missing 
> a larger context ...
>
> On 30/03/2020 7:30 pm, serguei.spitsyn at oracle.com wrote:
>> Hi Mandy,
>>
>> I have just one comment so far.
>>
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03/src/hotspot/share/classfile/classLoaderHierarchyDCmd.cpp.frames.html 
>>
>>
>> ??356?? void add_classes(LoadedClassInfo* first_class, int 
>> num_classes, bool has_class_mirror_holder) {
>> ??357???? LoadedClassInfo** p_list_to_add_to;
>> ??358???? bool is_hidden = first_class->_klass->is_hidden();
>> ??359???? if (has_class_mirror_holder) {
>> ??360?????? p_list_to_add_to = is_hidden ? &_hidden_weak_classes : 
>> &_anon_classes;
>> ??361???? } else {
>> ??362?????? p_list_to_add_to = &_classes;
>> ??363???? }
>> ??364???? // Search tail.
>> ??365???? while ((*p_list_to_add_to) != NULL) {
>> ??366?????? p_list_to_add_to = &(*p_list_to_add_to)->_next;
>> ??367???? }
>> ??368???? *p_list_to_add_to = first_class;
>> ??369???? if (has_class_mirror_holder) {
>> ??370?????? if (is_hidden) {
>> ??371???????? _num_hidden_weak_classes += num_classes;
>
> Why does hidden imply weak here?
>
> David
> -----
>
>> ??372?????? } else {
>> ??373???????? _num_anon_classes += num_classes;
>> ??374?????? }
>> ??375???? } else {
>> ??376?????? _num_classes += num_classes;
>> ??377???? }
>> ??378?? }
>>
>> ??Q1: I'm just curious, what happens if a cld has arrays of hidden 
>> classes?
>> ????? Is the bottom_klass always expected to be the first?
>>
>>
>> Thanks,
>> Serguei
>>
>>
>> On 3/26/20 16:57, Mandy Chung wrote:
>>> Please review the implementation of JEP 371: Hidden Classes. The 
>>> main changes are in core-libs and hotspot runtime area.? Small 
>>> changes are made in javac, VM compiler (intrinsification of 
>>> Class::isHiddenClass), JFR, JDI, and jcmd.? CSR [1]has been reviewed 
>>> and is in the finalized state (see specdiff and javadoc below for 
>>> reference).
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
>>>
>>>
>>> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's 
>>> point
>>> of view, a hidden class is a normal class except the following:
>>>
>>> - A hidden class has no initiating class loader and is not 
>>> registered in any dictionary.
>>> - A hidden class has a name containing an illegal character 
>>> `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature` 
>>> returns "Lp/Foo.0x1234;".
>>> - A hidden class is not modifiable, i.e. cannot be redefined or 
>>> retransformed. JVM TI IsModifableClass returns false on a hidden.
>>> - Final fields in a hidden class is "final".? The value of final 
>>> fields cannot be overriden via reflection. setAccessible(true) can 
>>> still be called on reflected objects representing final fields in a 
>>> hidden class and its access check will be suppressed but only have 
>>> read-access (i.e. can do Field::getXXX but not setXXX).
>>>
>>> Brief summary of this patch:
>>>
>>> 1. A new Lookup::defineHiddenClass method is the API to create a 
>>> hidden class.
>>> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG 
>>> option that
>>> ?? can be specified when creating a hidden class.
>>> 3. A new Class::isHiddenClass method tests if a class is a hidden 
>>> class.
>>> 4. Field::setXXX method will throw IAE on a final field of a hidden 
>>> class
>>> ?? regardless of the value of the accessible flag.
>>> 5. JVM_LookupDefineClass is the new JVM entry point for 
>>> Lookup::defineClass
>>> ?? and defineHiddenClass to create a class from the given bytes.
>>> 6. ClassLoaderData implementation is not changed.? There is one 
>>> primary CLD
>>> ?? that holds the classes strongly referenced by its defining 
>>> loader. There
>>> ?? can be zero or more additional CLDs - one per weak class.
>>> 7. Nest host determination is updated per revised JVMS 5.4.4. Access 
>>> control
>>> ?? check no longer throws LinkageError but instead it will throw IAE 
>>> with
>>> ?? a clear message if a class fails to resolve/validate the nest 
>>> host declared
>>> ?? in NestHost/NestMembers attribute.
>>> 8. JFR, jcmd, JDI are updated to support hidden classes.
>>> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
>>> ?? and generate a bridge method to desuger a method reference to a 
>>> protected
>>> ?? method in its supertype in a different package
>>>
>>> This patch also updates StringConcatFactory, LambdaMetaFactory, and 
>>> LambdaForms
>>> to use hidden classes.? The webrev includes changes in nashorn to 
>>> hidden class
>>> and I will update the webrev if JEP 372 removes it any time soon.
>>>
>>> We uncovered a bug in Lookup::defineClass spec throws LinkageError 
>>> and intends
>>> to have the newly created class linked.? However, the implementation 
>>> in 14
>>> does not link the class.? A separate CSR [2] proposes to update the
>>> implementation to match the spec.? This patch fixes the implementation.
>>>
>>> The spec update on JVM TI, JDI and Instrumentation will be done as
>>> a separate RFE [3].? This patch includes new tests for JVM TI and
>>> java.instrument that validates how the existing APIs work for hidden 
>>> classes.
>>>
>>> javadoc/specdiff
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
>>>
>>>
>>> JVMS 5.4.4 change:
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
>>>
>>>
>>> CSR:
>>> https://bugs.openjdk.java.net/browse/JDK-8238359
>>>
>>> Thanks
>>> Mandy
>>> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
>>> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
>>> [3] https://bugs.openjdk.java.net/browse/JDK-8230502
>>

From coleen.phillimore at oracle.com  Mon Mar 30 14:16:06 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Mon, 30 Mar 2020 10:16:06 -0400
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <c56eba10-13eb-aa86-11a3-f2dfd1a7b0b0@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <90f9276d-2777-88ba-b1ec-9901711fcf02@oracle.com>
 <c56eba10-13eb-aa86-11a3-f2dfd1a7b0b0@oracle.com>
Message-ID: <ebe3de7f-0cba-99a6-8980-ad0b95ff93da@oracle.com>



On 3/29/20 10:17 PM, Mandy Chung wrote:
>
>
> On 3/27/20 8:51 PM, Chris Plummer wrote:
>> Hi Mandy,
>>
>> A couple of very minor nits in the jvmtiRedefineClasses.cpp comments:
>>
>> ?153???? // classes for primitives, arrays, hidden and vm unsafe 
>> anonymous classes
>> ?154???? // cannot be redefined.? Check here so following code can 
>> assume these classes
>> ?155???? // are InstanceKlass.
>> ?156???? if (!is_modifiable_class(mirror)) {
>> ?157?????? _res = JVMTI_ERROR_UNMODIFIABLE_CLASS;
>> ?158?????? return false;
>> ?159???? }
>>
>> I think this code and comment predate anonymous classes. Probably 
>> before anonymous classes the check was not for !is_modifiable_class() 
>> but instead was just a check for primitive or array class types since 
>> they are not an InstanceKlass, and would cause issues when cast to 
>> one in the code that lies below this section. When anonymous classes 
>> were added, the code got changed to use !is_modifiable_class() and 
>> the comment was not correctly updated (anonymous classes are an 
>> InstanceKlass). Then with this webrev the mention of hidden classes 
>> was added, also incorrectly implying they are not an InstanceKlass. I 
>> think you should just leave off the last sentence of the comment.
>>
>
> I agree with you that this comment needs update.?? Perhaps it should 
> say "primitive, array types and hidden classes are non-modifiable. A 
> modifiable class must be an InstanceKlass."

I may have written the last part of that comment (or remember it at 
least).? I think Chris's suggestion to remove the last sentence makes 
sense.? Anything further will just adds unnecessary confusion to the 
reader.? Anyone modifying this will get the InstanceKlass::cast() assert 
soon after if they mess up.

Coleen

>
> I leave it to Serguei who may have other opinion.
>
>> There's some ambiguity in the application of adjectives in the 
>> following:
>>
>> ?297?? // Cannot redefine or retransform a hidden or an unsafe 
>> anonymous class.
>>
>> I'd suggest:
>>
>> ?297?? // Cannot redefine or retransform a hidden class or an unsafe 
>> anonymous class.
>>
>
> +1
>
>> There are some places in libjdwp that need to be fixed. I spoke to 
>> Serguei about those this afternoon. Basically the 
>> convertSignatureToClassname() function needs to be fixed to handle 
>> hidden classes. Without the fix classname filtering will have 
>> problems if the filter contains a pattern with a '/' to filter on 
>> hidden classes. Also CLASS_UNLOAD events will not properly convert 
>> hidden class names. We also need tests for these cases. I think these 
>> are all things that can be addressed later.
>>
>
> Good catch.? I have created a subtask under JDK-8230502:
> ?? https://bugs.openjdk.java.net/browse/JDK-8230502
>
>> I still need to look over the JVMTI tests.
>>
>
> Thanks
> Mandy
>> thanks,
>>
>> Chris
>>
>> On 3/26/20 4:57 PM, Mandy Chung wrote:
>>> Please review the implementation of JEP 371: Hidden Classes. The 
>>> main changes are in core-libs and hotspot runtime area. Small 
>>> changes are made in javac, VM compiler (intrinsification of 
>>> Class::isHiddenClass), JFR, JDI, and jcmd.? CSR [1]has been reviewed 
>>> and is in the finalized state (see specdiff and javadoc below for 
>>> reference).
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
>>>
>>>
>>> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's 
>>> point
>>> of view, a hidden class is a normal class except the following:
>>>
>>> - A hidden class has no initiating class loader and is not 
>>> registered in any dictionary.
>>> - A hidden class has a name containing an illegal character 
>>> `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature` 
>>> returns "Lp/Foo.0x1234;".
>>> - A hidden class is not modifiable, i.e. cannot be redefined or 
>>> retransformed. JVM TI IsModifableClass returns false on a hidden.
>>> - Final fields in a hidden class is "final".? The value of final 
>>> fields cannot be overriden via reflection. setAccessible(true) can 
>>> still be called on reflected objects representing final fields in a 
>>> hidden class and its access check will be suppressed but only have 
>>> read-access (i.e. can do Field::getXXX but not setXXX).
>>>
>>> Brief summary of this patch:
>>>
>>> 1. A new Lookup::defineHiddenClass method is the API to create a 
>>> hidden class.
>>> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG 
>>> option that
>>> ?? can be specified when creating a hidden class.
>>> 3. A new Class::isHiddenClass method tests if a class is a hidden 
>>> class.
>>> 4. Field::setXXX method will throw IAE on a final field of a hidden 
>>> class
>>> ?? regardless of the value of the accessible flag.
>>> 5. JVM_LookupDefineClass is the new JVM entry point for 
>>> Lookup::defineClass
>>> ?? and defineHiddenClass to create a class from the given bytes.
>>> 6. ClassLoaderData implementation is not changed.? There is one 
>>> primary CLD
>>> ?? that holds the classes strongly referenced by its defining 
>>> loader.? There
>>> ?? can be zero or more additional CLDs - one per weak class.
>>> 7. Nest host determination is updated per revised JVMS 5.4.4. Access 
>>> control
>>> ?? check no longer throws LinkageError but instead it will throw IAE 
>>> with
>>> ?? a clear message if a class fails to resolve/validate the nest 
>>> host declared
>>> ?? in NestHost/NestMembers attribute.
>>> 8. JFR, jcmd, JDI are updated to support hidden classes.
>>> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
>>> ?? and generate a bridge method to desuger a method reference to a 
>>> protected
>>> ?? method in its supertype in a different package
>>>
>>> This patch also updates StringConcatFactory, LambdaMetaFactory, and 
>>> LambdaForms
>>> to use hidden classes.? The webrev includes changes in nashorn to 
>>> hidden class
>>> and I will update the webrev if JEP 372 removes it any time soon.
>>>
>>> We uncovered a bug in Lookup::defineClass spec throws LinkageError 
>>> and intends
>>> to have the newly created class linked.? However, the implementation 
>>> in 14
>>> does not link the class.? A separate CSR [2] proposes to update the
>>> implementation to match the spec.? This patch fixes the implementation.
>>>
>>> The spec update on JVM TI, JDI and Instrumentation will be done as
>>> a separate RFE [3].? This patch includes new tests for JVM TI and
>>> java.instrument that validates how the existing APIs work for hidden 
>>> classes.
>>>
>>> javadoc/specdiff
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
>>>
>>>
>>> JVMS 5.4.4 change:
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
>>>
>>>
>>> CSR:
>>> https://bugs.openjdk.java.net/browse/JDK-8238359
>>>
>>> Thanks
>>> Mandy
>>> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
>>> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
>>> [3] https://bugs.openjdk.java.net/browse/JDK-8230502
>>
>>
>


From coleen.phillimore at oracle.com  Mon Mar 30 14:18:58 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Mon, 30 Mar 2020 10:18:58 -0400
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <546cf8e4-00e4-1d22-6402-6620b8d7b2db@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <ec8b6266-3eff-6d4a-987a-d46013c715e1@oracle.com>
 <546cf8e4-00e4-1d22-6402-6620b8d7b2db@oracle.com>
Message-ID: <9cd71367-edc6-efc8-0a53-2e703ffbbfab@oracle.com>



On 3/30/20 5:54 AM, David Holmes wrote:
> Sorry to jump in on this but it caught my eye though I may be missing 
> a larger context ...
>
> On 30/03/2020 7:30 pm, serguei.spitsyn at oracle.com wrote:
>> Hi Mandy,
>>
>> I have just one comment so far.
>>
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03/src/hotspot/share/classfile/classLoaderHierarchyDCmd.cpp.frames.html 
>>
>>
>> ??356?? void add_classes(LoadedClassInfo* first_class, int 
>> num_classes, bool has_class_mirror_holder) {
>> ??357???? LoadedClassInfo** p_list_to_add_to;
>> ??358???? bool is_hidden = first_class->_klass->is_hidden();
>> ??359???? if (has_class_mirror_holder) {
>> ??360?????? p_list_to_add_to = is_hidden ? &_hidden_weak_classes : 
>> &_anon_classes;
>> ??361???? } else {
>> ??362?????? p_list_to_add_to = &_classes;
>> ??363???? }
>> ??364???? // Search tail.
>> ??365???? while ((*p_list_to_add_to) != NULL) {
>> ??366?????? p_list_to_add_to = &(*p_list_to_add_to)->_next;
>> ??367???? }
>> ??368???? *p_list_to_add_to = first_class;
>> ??369???? if (has_class_mirror_holder) {
>> ??370?????? if (is_hidden) {
>> ??371???????? _num_hidden_weak_classes += num_classes;
>
> Why does hidden imply weak here?

has_class_mirror_holder() implies weak.

Coleen
>
> David
> -----
>
>> ??372?????? } else {
>> ??373???????? _num_anon_classes += num_classes;
>> ??374?????? }
>> ??375???? } else {
>> ??376?????? _num_classes += num_classes;
>> ??377???? }
>> ??378?? }
>>
>> ??Q1: I'm just curious, what happens if a cld has arrays of hidden 
>> classes?
>> ????? Is the bottom_klass always expected to be the first?
>>
>>
>> Thanks,
>> Serguei
>>
>>
>> On 3/26/20 16:57, Mandy Chung wrote:
>>> Please review the implementation of JEP 371: Hidden Classes. The 
>>> main changes are in core-libs and hotspot runtime area.? Small 
>>> changes are made in javac, VM compiler (intrinsification of 
>>> Class::isHiddenClass), JFR, JDI, and jcmd.? CSR [1]has been reviewed 
>>> and is in the finalized state (see specdiff and javadoc below for 
>>> reference).
>>>
>>> Webrev:
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
>>>
>>>
>>> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's 
>>> point
>>> of view, a hidden class is a normal class except the following:
>>>
>>> - A hidden class has no initiating class loader and is not 
>>> registered in any dictionary.
>>> - A hidden class has a name containing an illegal character 
>>> `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature` 
>>> returns "Lp/Foo.0x1234;".
>>> - A hidden class is not modifiable, i.e. cannot be redefined or 
>>> retransformed. JVM TI IsModifableClass returns false on a hidden.
>>> - Final fields in a hidden class is "final".? The value of final 
>>> fields cannot be overriden via reflection. setAccessible(true) can 
>>> still be called on reflected objects representing final fields in a 
>>> hidden class and its access check will be suppressed but only have 
>>> read-access (i.e. can do Field::getXXX but not setXXX).
>>>
>>> Brief summary of this patch:
>>>
>>> 1. A new Lookup::defineHiddenClass method is the API to create a 
>>> hidden class.
>>> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG 
>>> option that
>>> ?? can be specified when creating a hidden class.
>>> 3. A new Class::isHiddenClass method tests if a class is a hidden 
>>> class.
>>> 4. Field::setXXX method will throw IAE on a final field of a hidden 
>>> class
>>> ?? regardless of the value of the accessible flag.
>>> 5. JVM_LookupDefineClass is the new JVM entry point for 
>>> Lookup::defineClass
>>> ?? and defineHiddenClass to create a class from the given bytes.
>>> 6. ClassLoaderData implementation is not changed.? There is one 
>>> primary CLD
>>> ?? that holds the classes strongly referenced by its defining 
>>> loader. There
>>> ?? can be zero or more additional CLDs - one per weak class.
>>> 7. Nest host determination is updated per revised JVMS 5.4.4. Access 
>>> control
>>> ?? check no longer throws LinkageError but instead it will throw IAE 
>>> with
>>> ?? a clear message if a class fails to resolve/validate the nest 
>>> host declared
>>> ?? in NestHost/NestMembers attribute.
>>> 8. JFR, jcmd, JDI are updated to support hidden classes.
>>> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
>>> ?? and generate a bridge method to desuger a method reference to a 
>>> protected
>>> ?? method in its supertype in a different package
>>>
>>> This patch also updates StringConcatFactory, LambdaMetaFactory, and 
>>> LambdaForms
>>> to use hidden classes.? The webrev includes changes in nashorn to 
>>> hidden class
>>> and I will update the webrev if JEP 372 removes it any time soon.
>>>
>>> We uncovered a bug in Lookup::defineClass spec throws LinkageError 
>>> and intends
>>> to have the newly created class linked.? However, the implementation 
>>> in 14
>>> does not link the class.? A separate CSR [2] proposes to update the
>>> implementation to match the spec.? This patch fixes the implementation.
>>>
>>> The spec update on JVM TI, JDI and Instrumentation will be done as
>>> a separate RFE [3].? This patch includes new tests for JVM TI and
>>> java.instrument that validates how the existing APIs work for hidden 
>>> classes.
>>>
>>> javadoc/specdiff
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
>>>
>>>
>>> JVMS 5.4.4 change:
>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
>>>
>>>
>>> CSR:
>>> https://bugs.openjdk.java.net/browse/JDK-8238359
>>>
>>> Thanks
>>> Mandy
>>> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
>>> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
>>> [3] https://bugs.openjdk.java.net/browse/JDK-8230502
>>


From coleen.phillimore at oracle.com  Mon Mar 30 14:20:01 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Mon, 30 Mar 2020 10:20:01 -0400
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <ebe3de7f-0cba-99a6-8980-ad0b95ff93da@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <90f9276d-2777-88ba-b1ec-9901711fcf02@oracle.com>
 <c56eba10-13eb-aa86-11a3-f2dfd1a7b0b0@oracle.com>
 <ebe3de7f-0cba-99a6-8980-ad0b95ff93da@oracle.com>
Message-ID: <ea6e4918-aaca-f64f-e089-d4013e02bf99@oracle.com>

Adding back serviceability-dev.? Sometimes reply (and myself) remembers 
it and sometimes it strips it off....

Coleen

On 3/30/20 10:16 AM, coleen.phillimore at oracle.com wrote:
>
>
> On 3/29/20 10:17 PM, Mandy Chung wrote:
>>
>>
>> On 3/27/20 8:51 PM, Chris Plummer wrote:
>>> Hi Mandy,
>>>
>>> A couple of very minor nits in the jvmtiRedefineClasses.cpp comments:
>>>
>>> ?153???? // classes for primitives, arrays, hidden and vm unsafe 
>>> anonymous classes
>>> ?154???? // cannot be redefined.? Check here so following code can 
>>> assume these classes
>>> ?155???? // are InstanceKlass.
>>> ?156???? if (!is_modifiable_class(mirror)) {
>>> ?157?????? _res = JVMTI_ERROR_UNMODIFIABLE_CLASS;
>>> ?158?????? return false;
>>> ?159???? }
>>>
>>> I think this code and comment predate anonymous classes. Probably 
>>> before anonymous classes the check was not for 
>>> !is_modifiable_class() but instead was just a check for primitive or 
>>> array class types since they are not an InstanceKlass, and would 
>>> cause issues when cast to one in the code that lies below this 
>>> section. When anonymous classes were added, the code got changed to 
>>> use !is_modifiable_class() and the comment was not correctly updated 
>>> (anonymous classes are an InstanceKlass). Then with this webrev the 
>>> mention of hidden classes was added, also incorrectly implying they 
>>> are not an InstanceKlass. I think you should just leave off the last 
>>> sentence of the comment.
>>>
>>
>> I agree with you that this comment needs update.?? Perhaps it should 
>> say "primitive, array types and hidden classes are non-modifiable. A 
>> modifiable class must be an InstanceKlass."
>
> I may have written the last part of that comment (or remember it at 
> least).? I think Chris's suggestion to remove the last sentence makes 
> sense.? Anything further will just adds unnecessary confusion to the 
> reader.? Anyone modifying this will get the InstanceKlass::cast() 
> assert soon after if they mess up.
>
> Coleen
>
>>
>> I leave it to Serguei who may have other opinion.
>>
>>> There's some ambiguity in the application of adjectives in the 
>>> following:
>>>
>>> ?297?? // Cannot redefine or retransform a hidden or an unsafe 
>>> anonymous class.
>>>
>>> I'd suggest:
>>>
>>> ?297?? // Cannot redefine or retransform a hidden class or an unsafe 
>>> anonymous class.
>>>
>>
>> +1
>>
>>> There are some places in libjdwp that need to be fixed. I spoke to 
>>> Serguei about those this afternoon. Basically the 
>>> convertSignatureToClassname() function needs to be fixed to handle 
>>> hidden classes. Without the fix classname filtering will have 
>>> problems if the filter contains a pattern with a '/' to filter on 
>>> hidden classes. Also CLASS_UNLOAD events will not properly convert 
>>> hidden class names. We also need tests for these cases. I think 
>>> these are all things that can be addressed later.
>>>
>>
>> Good catch.? I have created a subtask under JDK-8230502:
>> ?? https://bugs.openjdk.java.net/browse/JDK-8230502
>>
>>> I still need to look over the JVMTI tests.
>>>
>>
>> Thanks
>> Mandy
>>> thanks,
>>>
>>> Chris
>>>
>>> On 3/26/20 4:57 PM, Mandy Chung wrote:
>>>> Please review the implementation of JEP 371: Hidden Classes. The 
>>>> main changes are in core-libs and hotspot runtime area. Small 
>>>> changes are made in javac, VM compiler (intrinsification of 
>>>> Class::isHiddenClass), JFR, JDI, and jcmd.? CSR [1]has been 
>>>> reviewed and is in the finalized state (see specdiff and javadoc 
>>>> below for reference).
>>>>
>>>> Webrev:
>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
>>>>
>>>>
>>>> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's 
>>>> point
>>>> of view, a hidden class is a normal class except the following:
>>>>
>>>> - A hidden class has no initiating class loader and is not 
>>>> registered in any dictionary.
>>>> - A hidden class has a name containing an illegal character 
>>>> `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature` 
>>>> returns "Lp/Foo.0x1234;".
>>>> - A hidden class is not modifiable, i.e. cannot be redefined or 
>>>> retransformed. JVM TI IsModifableClass returns false on a hidden.
>>>> - Final fields in a hidden class is "final".? The value of final 
>>>> fields cannot be overriden via reflection. setAccessible(true) can 
>>>> still be called on reflected objects representing final fields in a 
>>>> hidden class and its access check will be suppressed but only have 
>>>> read-access (i.e. can do Field::getXXX but not setXXX).
>>>>
>>>> Brief summary of this patch:
>>>>
>>>> 1. A new Lookup::defineHiddenClass method is the API to create a 
>>>> hidden class.
>>>> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG 
>>>> option that
>>>> ?? can be specified when creating a hidden class.
>>>> 3. A new Class::isHiddenClass method tests if a class is a hidden 
>>>> class.
>>>> 4. Field::setXXX method will throw IAE on a final field of a hidden 
>>>> class
>>>> ?? regardless of the value of the accessible flag.
>>>> 5. JVM_LookupDefineClass is the new JVM entry point for 
>>>> Lookup::defineClass
>>>> ?? and defineHiddenClass to create a class from the given bytes.
>>>> 6. ClassLoaderData implementation is not changed.? There is one 
>>>> primary CLD
>>>> ?? that holds the classes strongly referenced by its defining 
>>>> loader.? There
>>>> ?? can be zero or more additional CLDs - one per weak class.
>>>> 7. Nest host determination is updated per revised JVMS 5.4.4. 
>>>> Access control
>>>> ?? check no longer throws LinkageError but instead it will throw 
>>>> IAE with
>>>> ?? a clear message if a class fails to resolve/validate the nest 
>>>> host declared
>>>> ?? in NestHost/NestMembers attribute.
>>>> 8. JFR, jcmd, JDI are updated to support hidden classes.
>>>> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
>>>> ?? and generate a bridge method to desuger a method reference to a 
>>>> protected
>>>> ?? method in its supertype in a different package
>>>>
>>>> This patch also updates StringConcatFactory, LambdaMetaFactory, and 
>>>> LambdaForms
>>>> to use hidden classes.? The webrev includes changes in nashorn to 
>>>> hidden class
>>>> and I will update the webrev if JEP 372 removes it any time soon.
>>>>
>>>> We uncovered a bug in Lookup::defineClass spec throws LinkageError 
>>>> and intends
>>>> to have the newly created class linked.? However, the 
>>>> implementation in 14
>>>> does not link the class.? A separate CSR [2] proposes to update the
>>>> implementation to match the spec.? This patch fixes the 
>>>> implementation.
>>>>
>>>> The spec update on JVM TI, JDI and Instrumentation will be done as
>>>> a separate RFE [3].? This patch includes new tests for JVM TI and
>>>> java.instrument that validates how the existing APIs work for 
>>>> hidden classes.
>>>>
>>>> javadoc/specdiff
>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/ 
>>>>
>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
>>>>
>>>>
>>>> JVMS 5.4.4 change:
>>>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
>>>>
>>>>
>>>> CSR:
>>>> https://bugs.openjdk.java.net/browse/JDK-8238359
>>>>
>>>> Thanks
>>>> Mandy
>>>> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
>>>> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
>>>> [3] https://bugs.openjdk.java.net/browse/JDK-8230502
>>>
>>>
>>
>


From coleen.phillimore at oracle.com  Mon Mar 30 15:23:19 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Mon, 30 Mar 2020 11:23:19 -0400
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <81f90a3e-dfda-0566-a9b2-ea0c4f17e7ac@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <81f90a3e-dfda-0566-a9b2-ea0c4f17e7ac@oracle.com>
Message-ID: <42af77ec-0f03-a4b0-164d-3b25c14c7f37@oracle.com>

Adding back hotspot-dev.

On 3/30/20 11:02 AM, coleen.phillimore at oracle.com wrote:
>
> Hi,? This is great work!? I did a prereview and all of my comments 
> were addressed.? These are a few minor things I noticed.
>
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03/src/hotspot/share/ci/ciInstanceKlass.hpp.udiff.html
>
> Nit. Can you add 'const' to the is_hidden accessor?
>
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03/src/hotspot/share/classfile/classFileParser.cpp.udiff.html
>
> + ID annotation_index(const ClassLoaderData* loader_data, const 
> Symbol* name, const bool can_access_vm_annotations);
>
> 'const' bool is weird and unnecessary.? Can you remove const here?
>
> + if (is_hidden()) { // Mark methods in hidden classes as 'hidden'.
> + m->set_hidden(true);
> + }
> +
> Could be:
>
> + // Mark methods in hidden classes as 'hidden'.
> + m->set_hidden(is_hidden());
> +
>
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03/src/hotspot/share/classfile/javaClasses.cpp.udiff.html
>
> + macro(_classData_offset, k, "classData", object_signature, false); \
>
> Probably should remove trailing backslash here.
>
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03/src/hotspot/share/classfile/systemDictionary.cpp.udiff.html
>
> I think in a future RFE, we should add a default parameter to 
> register_loader to make the code in the beginning of parse_stream() 
> cleaner and remove has_class_mirror_holder_cld().
>
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03/src/hotspot/share/prims/jvm.cpp.udiff.html
> + jboolean is_nestmate = (flags & NESTMATE) == NESTMATE;
> + jboolean is_hidden = (flags & HIDDEN_CLASS) == HIDDEN_CLASS;
> + jboolean is_strong = (flags & STRONG_LOADER_LINK) == STRONG_LOADER_LINK;
> + jboolean vm_annotations = (flags & ACCESS_VM_ANNOTATIONS) == 
> ACCESS_VM_ANNOTATION
>
> Instead of jboolean, please use C++ bool here.
>
> + oop loader = lookup_k->class_loader();
> + Handle class_loader (THREAD, loader);
> Can you rewrite as this to prevent potential unhandled oop for oop loader.
> + Handle class_loader (THREAD, lookup_k->class_loader());
>
> Here:
> + InstanceKlass::cast(defined_k)->class_loader_data()->dec_keep_alive();
>
> Don't have to cast defined_k to get class_loader_data(), but you 
> probably just want to move this up to remove the rest of the 
> InstanceKlass::cast().
>
> + InstanceKlass* ik = InstanceKlass::cast(defined_k);
>
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03/src/hotspot/share/runtime/vmStructs.cpp.udiff.html 
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/classfile/ClassLoaderData.java.udiff.html
>
> We agreed already that these changes aren't needed by the SA.? You can 
> revert these.
>
> These are minor changes.? I don't need to see another webrev.
>
> Thanks,
> Coleen
>
>
>
> On 3/26/20 7:57 PM, Mandy Chung wrote:
>> Please review the implementation of JEP 371: Hidden Classes.? The 
>> main changes are in core-libs and hotspot runtime area.? Small 
>> changes are made in javac, VM compiler (intrinsification of 
>> Class::isHiddenClass), JFR, JDI, and jcmd.? CSR [1]has been reviewed 
>> and is in the finalized state (see specdiff and javadoc below for 
>> reference).
>>
>> Webrev:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03
>>
>> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's point
>> of view, a hidden class is a normal class except the following:
>>
>> - A hidden class has no initiating class loader and is not registered 
>> in any dictionary.
>> - A hidden class has a name containing an illegal character 
>> `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature` 
>> returns "Lp/Foo.0x1234;".
>> - A hidden class is not modifiable, i.e. cannot be redefined or 
>> retransformed. JVM TI IsModifableClass returns false on a hidden.
>> - Final fields in a hidden class is "final".? The value of final 
>> fields cannot be overriden via reflection. setAccessible(true) can 
>> still be called on reflected objects representing final fields in a 
>> hidden class and its access check will be suppressed but only have 
>> read-access (i.e. can do Field::getXXX but not setXXX).
>>
>> Brief summary of this patch:
>>
>> 1. A new Lookup::defineHiddenClass method is the API to create a 
>> hidden class.
>> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG 
>> option that
>> ?? can be specified when creating a hidden class.
>> 3. A new Class::isHiddenClass method tests if a class is a hidden class.
>> 4. Field::setXXX method will throw IAE on a final field of a hidden class
>> ?? regardless of the value of the accessible flag.
>> 5. JVM_LookupDefineClass is the new JVM entry point for 
>> Lookup::defineClass
>> ?? and defineHiddenClass to create a class from the given bytes.
>> 6. ClassLoaderData implementation is not changed.? There is one 
>> primary CLD
>> ?? that holds the classes strongly referenced by its defining 
>> loader.? There
>> ?? can be zero or more additional CLDs - one per weak class.
>> 7. Nest host determination is updated per revised JVMS 5.4.4. Access 
>> control
>> ?? check no longer throws LinkageError but instead it will throw IAE with
>> ?? a clear message if a class fails to resolve/validate the nest host 
>> declared
>> ?? in NestHost/NestMembers attribute.
>> 8. JFR, jcmd, JDI are updated to support hidden classes.
>> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
>> ?? and generate a bridge method to desuger a method reference to a 
>> protected
>> ?? method in its supertype in a different package
>>
>> This patch also updates StringConcatFactory, LambdaMetaFactory, and 
>> LambdaForms
>> to use hidden classes.? The webrev includes changes in nashorn to 
>> hidden class
>> and I will update the webrev if JEP 372 removes it any time soon.
>>
>> We uncovered a bug in Lookup::defineClass spec throws LinkageError 
>> and intends
>> to have the newly created class linked.? However, the implementation 
>> in 14
>> does not link the class.? A separate CSR [2] proposes to update the
>> implementation to match the spec.? This patch fixes the implementation.
>>
>> The spec update on JVM TI, JDI and Instrumentation will be done as
>> a separate RFE [3].? This patch includes new tests for JVM TI and
>> java.instrument that validates how the existing APIs work for hidden 
>> classes.
>>
>> javadoc/specdiff
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/
>>
>> JVMS 5.4.4 change:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf
>>
>> CSR:
>> https://bugs.openjdk.java.net/browse/JDK-8238359
>>
>> Thanks
>> Mandy
>> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
>> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
>> [3] https://bugs.openjdk.java.net/browse/JDK-8230502
>


From mandy.chung at oracle.com  Mon Mar 30 16:18:48 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Mon, 30 Mar 2020 09:18:48 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <ebe3de7f-0cba-99a6-8980-ad0b95ff93da@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <90f9276d-2777-88ba-b1ec-9901711fcf02@oracle.com>
 <c56eba10-13eb-aa86-11a3-f2dfd1a7b0b0@oracle.com>
 <ebe3de7f-0cba-99a6-8980-ad0b95ff93da@oracle.com>
Message-ID: <0b585750-54f9-aaa0-19b3-752f723894d1@oracle.com>



On 3/30/20 7:16 AM, coleen.phillimore at oracle.com wrote:
>> I agree with you that this comment needs update.?? Perhaps it should 
>> say "primitive, array types and hidden classes are non-modifiable. A 
>> modifiable class must be an InstanceKlass."
>
> I may have written the last part of that comment (or remember it at 
> least).? I think Chris's suggestion to remove the last sentence makes 
> sense.? Anything further will just adds unnecessary confusion to the 
> reader.? Anyone modifying this will get the InstanceKlass::cast() 
> assert soon after if they mess up. 

OK.? That's fine too.

Mandy

From serguei.spitsyn at oracle.com  Mon Mar 30 16:19:19 2020
From: serguei.spitsyn at oracle.com (serguei.spitsyn at oracle.com)
Date: Mon, 30 Mar 2020 09:19:19 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <ec8b6266-3eff-6d4a-987a-d46013c715e1@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <ec8b6266-3eff-6d4a-987a-d46013c715e1@oracle.com>
Message-ID: <50b1658d-2195-53af-ea0b-e13842e00496@oracle.com>

On 3/30/20 02:30, serguei.spitsyn at oracle.com wrote:
> Hi Mandy,
>
> I have just one comment so far.
>
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03/src/hotspot/share/classfile/classLoaderHierarchyDCmd.cpp.frames.html 
>
>
> ?356?? void add_classes(LoadedClassInfo* first_class, int num_classes, 
> bool has_class_mirror_holder) {
> ?357???? LoadedClassInfo** p_list_to_add_to;
> ?358???? bool is_hidden = first_class->_klass->is_hidden();
> ?359???? if (has_class_mirror_holder) {
> ?360?????? p_list_to_add_to = is_hidden ? &_hidden_weak_classes : 
> &_anon_classes;
> ?361???? } else {
> ?362?????? p_list_to_add_to = &_classes;
> ?363???? }
> ?364???? // Search tail.
> ?365???? while ((*p_list_to_add_to) != NULL) {
> ?366?????? p_list_to_add_to = &(*p_list_to_add_to)->_next;
> ?367???? }
> ?368???? *p_list_to_add_to = first_class;
> ?369???? if (has_class_mirror_holder) {
> ?370?????? if (is_hidden) {
> ?371???????? _num_hidden_weak_classes += num_classes;
> ?372?????? } else {
> ?373???????? _num_anon_classes += num_classes;
> ?374?????? }
> ?375???? } else {
> ?376?????? _num_classes += num_classes;
> ?377???? }
> ?378?? }
>
> ?Q1: I'm just curious, what happens if a cld has arrays of hidden 
> classes?
> ???? Is the bottom_klass always expected to be the first?

Please, skip it. I've got the answer.
The array classes were not included into the LoadedClassInfo* by the 
classes_do.

Thanks,
Serguei

>
> Thanks,
> Serguei
>
>
> On 3/26/20 16:57, Mandy Chung wrote:
>> Please review the implementation of JEP 371: Hidden Classes. The main 
>> changes are in core-libs and hotspot runtime area.? Small changes are 
>> made in javac, VM compiler (intrinsification of 
>> Class::isHiddenClass), JFR, JDI, and jcmd.? CSR [1]has been reviewed 
>> and is in the finalized state (see specdiff and javadoc below for 
>> reference).
>>
>> Webrev:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
>>
>>
>> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's 
>> point
>> of view, a hidden class is a normal class except the following:
>>
>> - A hidden class has no initiating class loader and is not registered 
>> in any dictionary.
>> - A hidden class has a name containing an illegal character 
>> `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature` 
>> returns "Lp/Foo.0x1234;".
>> - A hidden class is not modifiable, i.e. cannot be redefined or 
>> retransformed. JVM TI IsModifableClass returns false on a hidden.
>> - Final fields in a hidden class is "final".? The value of final 
>> fields cannot be overriden via reflection.? setAccessible(true) can 
>> still be called on reflected objects representing final fields in a 
>> hidden class and its access check will be suppressed but only have 
>> read-access (i.e. can do Field::getXXX but not setXXX).
>>
>> Brief summary of this patch:
>>
>> 1. A new Lookup::defineHiddenClass method is the API to create a 
>> hidden class.
>> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG 
>> option that
>> ?? can be specified when creating a hidden class.
>> 3. A new Class::isHiddenClass method tests if a class is a hidden class.
>> 4. Field::setXXX method will throw IAE on a final field of a hidden 
>> class
>> ?? regardless of the value of the accessible flag.
>> 5. JVM_LookupDefineClass is the new JVM entry point for 
>> Lookup::defineClass
>> ?? and defineHiddenClass to create a class from the given bytes.
>> 6. ClassLoaderData implementation is not changed.? There is one 
>> primary CLD
>> ?? that holds the classes strongly referenced by its defining 
>> loader.? There
>> ?? can be zero or more additional CLDs - one per weak class.
>> 7. Nest host determination is updated per revised JVMS 5.4.4. Access 
>> control
>> ?? check no longer throws LinkageError but instead it will throw IAE 
>> with
>> ?? a clear message if a class fails to resolve/validate the nest host 
>> declared
>> ?? in NestHost/NestMembers attribute.
>> 8. JFR, jcmd, JDI are updated to support hidden classes.
>> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
>> ?? and generate a bridge method to desuger a method reference to a 
>> protected
>> ?? method in its supertype in a different package
>>
>> This patch also updates StringConcatFactory, LambdaMetaFactory, and 
>> LambdaForms
>> to use hidden classes.? The webrev includes changes in nashorn to 
>> hidden class
>> and I will update the webrev if JEP 372 removes it any time soon.
>>
>> We uncovered a bug in Lookup::defineClass spec throws LinkageError 
>> and intends
>> to have the newly created class linked.? However, the implementation 
>> in 14
>> does not link the class.? A separate CSR [2] proposes to update the
>> implementation to match the spec.? This patch fixes the implementation.
>>
>> The spec update on JVM TI, JDI and Instrumentation will be done as
>> a separate RFE [3].? This patch includes new tests for JVM TI and
>> java.instrument that validates how the existing APIs work for hidden 
>> classes.
>>
>> javadoc/specdiff
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
>>
>>
>> JVMS 5.4.4 change:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
>>
>>
>> CSR:
>> https://bugs.openjdk.java.net/browse/JDK-8238359
>>
>> Thanks
>> Mandy
>> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
>> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
>> [3] https://bugs.openjdk.java.net/browse/JDK-8230502
>


From mandy.chung at oracle.com  Mon Mar 30 19:13:57 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Mon, 30 Mar 2020 12:13:57 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <08191054-8d0a-ae60-ac99-e2849f08ce85@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <d3d2fa4f-4bf7-94c3-de4b-15dfe75b6370@oracle.com>
 <08191054-8d0a-ae60-ac99-e2849f08ce85@oracle.com>
Message-ID: <528c7933-be32-9863-6cc5-92223a75bbee@oracle.com>

This is the patch to keep the JDK 14 behavior if target release to 14 
(thanks to Jan for helping making change in javac to get the tests working)
http://cr.openjdk.java.net/~mchung/valhalla/webrevs/8171335/webrev-javac-target-release-14/

Mandy

On 3/27/20 9:29 AM, Mandy Chung wrote:
> Hi Jan,
>
> Good point.? The javac change only applies to JDK 15 and later and the 
> lambda proxy class is not a nestmate when running on JDK 14 or earlier.
>
> I probably need the help from langtools team to fix this.? I'll give 
> it a try.
>
> Mandy


From david.holmes at oracle.com  Tue Mar 31 10:04:46 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 31 Mar 2020 20:04:46 +1000
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
Message-ID: <135a6496-5b54-9117-5cbf-e3211ddc32dd@oracle.com>

Hi Leo,

I looked at the "combined change". Overall I like the cleanup, but I did 
find a few questionable items:

src/hotspot/cpu/arm/interpreterRT_arm.cpp

-      _toGP[_last_gp++] = (*(intptr_t*)from_addr == 0) ? NULL : from_addr;
+      _toGP[_last_gp++] = (*(intptr_t*)from_addr == 0) ? NULL_WORD : 
from_addr;
      } else {
-      *_to++ = (*(intptr_t*)from_addr == 0) ? NULL : from_addr;
+      *_to++ = (*(intptr_t*)from_addr == 0) ? NULL_WORD : from_addr;

If from_addr is being treated as intptr_t* then shouldn't it be compared 
against NULL_WORD rather than 0?

---

General query: are changes like:

Foo f = NULL;

to

Foo f(NULL);

just making the implicit constructor selection explicit? (I have to say 
it would nice if whenever you saw " x = NULL" that you knew x was a 
pointer type.)

---

src/hotspot/cpu/sparc/c1_CodeStubs_sparc.cpp

-    AddressLiteral addrlit(NULL, metadata_Relocation::spec(_index));
+    AddressLiteral addrlit(NULL_WORD, metadata_Relocation::spec(_index));

If the first constructor arg is an "address" which seems to be the 
pointer type "uchar*" then surely NULL is correct here, not NULL_WORD?

Same applies to other AddressLiteral changes.

---

src/hotspot/share/code/codeHeapState.cpp

This doesn't look right:

-    if (((_termString) != NULL) && (strlen(_termString) > 0)){\
+    if (((_termString) != 0) && (strlen(_termString) > 0)){   \

_termString should be a "const char*" so comparison against NULL is correct.

---

share/compiler/directivesParser.cpp

 > new NULL value does unfortunately not convert to method pointers

Is that a temporary limitation? I'm not sure what progress were are 
making if some pointers can use NULL and others must use 0.

---

src/hotspot/share/jfr/jni/jfrJavaCall.cpp

I don't understand why the NULL check has been removed:

-  assert(_storage != NULL, "invariant");
+  assert(_storage_index < (int) ARRAY_SIZE(_storage), "invariant");

---

src/hotspot/share/jvmci/jvmciCompilerToVMInit.cpp

-    if (value != NULL) { \
+    if ((value) != 0) { \

Unclear what type value is - in some cases char* in others I can't 
determine.

---

src/hotspot/share/jvmci/jvmciExceptions.hpp

-#define JVMCI_CHECK_0                                  JVMCI_CHECK_(0)
-#define JVMCI_CHECK_NULL                               JVMCI_CHECK_(NULL)
+#define JVMCI_CHECK_0 
JVMCI_CHECK_(NULL_WORD)
+#define JVMCI_CHECK_NULL                               JVMCI_CHECK_(0)

These just seem plain wrong. CHECK_0 is for returning a integral zero 
value (e.g. jint, jbtye etc), nothing to do with intptr_t and NULL_WORD.

Having CHECK_NULL become CHECK_(0) seems completely wrong when you 
consider that if you were expanding by hand in a function returning a 
pointer then you would write "return NULL". Further if this chasnge is 
necessary/desirable for JVMCI_CHECK_NULL then surely the same is true 
for the basic CHECK_NULL in exceptions.hpp ?

Aside:

./share/jvmci/jvmciCompilerToVM.cpp

C2V_VMENTRY_0(jboolean, isInternedString, (JNIEnv* env, jobject, jobject 
object))
   Handle str = JVMCIENV->asConstant(JVMCIENV->wrap(object), JVMCI_CHECK_0);
   if (!java_lang_String::is_instance(str())) {
     return false;

CHECK_0 should be CHECK_false. Ditto in isInstance, 
isCurrentThreadAttached,  attachCurrentThread.

---

src/hotspot/share/runtime/sweeper.hpp

 > gcc does unfortunately not know that our NULL is a safe format string

-  static void log_sweep(const char* msg, const char* format = NULL, 
...) ATTRIBUTE_PRINTF(2, 3);
+  static void log_sweep(const char* msg, const char* format = 0, ...) 
ATTRIBUTE_PRINTF(2, 3);

I don't follow. This isn't "our NULL" we're using the toolchain's NULL, 
and format is a pointer type so this should be fine. Is gcc broken?

---

src/hotspot/share/utilities/globalDefinitions_solstudio.hpp

It isn't clear that the issue of passing 0 as NULL in a varargs 
situation is now fixed in Sol Studio.

---

src/hotspot/share/utilities/linkedlist.hpp

-    return equal<E>(_data, t, NULL);
+    return equal<E>(_data, t, 0);

Again concerned about the method/function pointer situation.

----

That's all from me.

Thanks,
David
-----



On 30/03/2020 10:24 am, Leo Korinth wrote:
> Hi, could I have a review on this change to improve NULL?
> 
> I try to:
> 1) make NULL only convert to pointer types for better type safety, 
> __null on gcc achieves some of this, but can be casted away.
> 2) define NULL in one place, in a platform independent way, no need to 
> define it for every tool chain.
> 3) make NULL be of a special type (null_t) so that one can template 
> specialise against it. This is for the future!
> 
> I fail with this, but continue to read...
> 
> I try to achieve this by defining NULL as a variable of type null_t in 
> the header null.hpp using ODR. null_t converts to (almost) any pointer 
> type.
> 
> Minor negative issues:
> 1) No conversion to method pointers, they are only used in three places 
> though, I use a literal 0 instead.
> 2) gcc does unfortunately not know that our NULL is a safe format 
> string, only needed to change one place.
> 3) No constexpr yet, so I needed to use 0 to initialise a thread variable.
> 
> IMO positive issues:
> 1) I have fixed many bad usages of NULL for integer types and instead 
> used NULL_WORD or 0.
> 2) Too many implicit conversions in certain places makes the code not 
> compile, the changed code does look better and more idiomatic IMO.
> 3) Possible benign bug found where fields are initialised to NULL (not 
> in constructor), this should not be possible right?
> 4) NULL does not convert to array types, on Windows this results in 
> compile time error as the windows compiler does not decay arrays when 
> comparing to null_t, this did locate one bug, and possibly some 
> unnecessary comparisons in xmacros that I kept by changing NULL to 0.
> 5) Found a reinterpret_cast on NULL that is not allowed by the standard.
> 
> I also made a new NULL_WORD in the same way as I did NULL, but instead 
> converting to a intptr_t of value 0. I was about to release NULL_WORD as 
> a separate patch, but due to the current practice of defining NULL_WORD 
> as NULL on many platforms, it was easier to merge these two changes into 
> one. Only small issues in JVMCI code for this change, and one fix in 
> sparc assembler. I find NULL_WORD questionable, but it is widely used.
> 
> Something to notice is the strange comment in the solstudio header about 
> intptr_t being wider than int on some systems that is to my 
> understanding false (for systems we support). I have a static assert to 
> check that intptr_t is indeed of (char) pointer size. I did remove an 
> assert that checks the size of NULL.
> 
> Now, the sad thing is that my change does not work fully, I can not make 
> NULL constexpr before c++ 11, and thus several initialisation order bugs 
> appear on global variables set to NULL. I could set them to zero or not 
> set them at all, and they would work, but it feels brittle and could 
> cause future bugs when people expect NULLs on globals to be initialised 
> directly. Thus instead, I propose that I revert somewhat and define NULL 
> as 0 and change NULL_WORD to a constant 0 of type intptr_t.
> 
> When we get c++>=11 we can define NULL to nullptr of type nullptr_t, and 
> hopefully that will be a simple fix after this change.
> 
> What do you think?
> 
> Below I have given a short description of the changes:
> src/hotspot/cpu/aarch64/interpreterRT_aarch64.cpp
> correct the usage of integer type from NULL to NULL_WORD
> 
> src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp
> too many implicit conversions
> 
> src/hotspot/cpu/arm/interpreterRT_arm.cpp
> correct the usage of integer type from NULL to NULL_WORD
> 
> src/hotspot/cpu/ppc/c1_LIRAssembler_ppc.cpp
> too many implicit conversions
> 
> src/hotspot/cpu/sparc/c1_CodeStubs_sparc.cpp
> correct the usage of integer type from NULL to NULL_WORD
> 
> src/hotspot/cpu/sparc/c1_LIRAssembler_sparc.cpp
> correct the usage of integer type from NULL to NULL_WORD, and the
> opposite way as well!
> 
> src/hotspot/cpu/sparc/interpreterRT_sparc.cpp
> correct the usage of integer type from NULL to NULL_WORD
> 
> src/hotspot/cpu/x86/interp_masm_x86.cpp
> correct the usage of integer type from NULL to NULL_WORD
> 
> src/hotspot/cpu/x86/interpreterRT_x86_64.cpp
> correct the usage of integer type from NULL to NULL_WORD
> 
> src/hotspot/cpu/x86/stubGenerator_x86_64.cpp
> correct the usage of integer type from NULL to NULL_WORD
> 
> src/hotspot/os/linux/cgroupV1Subsystem_linux.hpp
> correct member initialisation to be in initialise list, gcc bug?
> 
> src/hotspot/os/linux/cgroupV2Subsystem_linux.hpp
> correct member initialisation to be in initialise list, gcc bug?
> 
> src/hotspot/os/solaris/attachListener_solaris.cpp
> correct the usage of integer type from NULL to 0
> 
> src/hotspot/os/windows/osThread_windows.cpp
> correct the usage of integer type from NULL to 0
> 
> src/hotspot/os/windows/os_windows.cpp
> correct the usage of integer type from NULL to 0
> 
> src/hotspot/os/windows/pdh_interface.cpp
> new NULL value does unfortunately not convert to method pointers
> 
> src/hotspot/share/classfile/javaClasses.cpp
> too many implicit conversions
> 
> src/hotspot/share/classfile/moduleEntry.hpp
> too many implicit conversions
> 
> src/hotspot/share/classfile/modules.cpp
> too many implicit conversions
> 
> src/hotspot/share/code/codeHeapState.cpp
> new NULL does not compare with arrays witch is great, maybe (xmacros) 
> the NULL check can be removed, this change is safe though
> 
> src/hotspot/share/compiler/directivesParser.cpp
> new NULL value does unfortunately not convert to method pointers
> 
> src/hotspot/share/gc/g1/g1ConcurrentRefine.cpp
> remove unnecessary system include, might redefine NULL
> 
> src/hotspot/share/gc/parallel/psAdaptiveSizePolicy.cpp
> remove unnecessary system include, might redefine NULL
> 
> src/hotspot/share/gc/parallel/psParallelCompact.cpp
> remove unnecessary system include, might redefine NULL
> 
> src/hotspot/share/gc/shared/memset_with_concurrent_readers.hpp
> remove unnecessary system include, might redefine NULL
> 
> src/hotspot/share/gc/z/zErrno.cpp
> remove unnecessary system include, might redefine NULL
> 
> src/hotspot/share/jfr/jni/jfrGetAllEventClasses.cpp
> fix bad assert found due to new NULL
> 
> src/hotspot/share/jvmci/jvmciCompilerToVM.cpp
> too many implicit conversions, and misc changes
> 
> src/hotspot/share/jvmci/jvmciCompilerToVMInit.cpp
> new NULL does not compare with arrays witch is great, maybe (xmacros) 
> the NULL check can be removed, this change is safe though
> 
> src/hotspot/share/jvmci/jvmciExceptions.hpp
> too many implicit conversions
> 
> src/hotspot/share/oops/access.hpp reinterpret_cast is not safe on NULL, 
> rewrite
> 
> src/hotspot/share/oops/constantPool.cpp
> too many implicit conversions
> 
> src/hotspot/share/oops/cpCache.cpp
> too many implicit conversions
> 
> src/hotspot/share/oops/klass.cpp
> too many implicit conversions
> 
> src/hotspot/share/oops/oopsHierarchy.hpp
> add null_t overload to oop class
> 
> src/hotspot/share/prims/jni.cpp
> too many implicit conversions
> 
> src/hotspot/share/prims/jvm.cpp
> too many implicit conversions
> 
> src/hotspot/share/prims/methodHandles.cpp
> too many implicit conversions
> 
> src/hotspot/share/runtime/os.cpp
> correct the usage of integer type from NULL to 0
> 
> src/hotspot/share/runtime/os.inline.hpp
> #undef NULL set by system headers, use the new NULL instead
> 
> src/hotspot/share/runtime/sweeper.hpp
> gcc does unfortunately not know that our NULL is a safe format string
> 
> src/hotspot/share/runtime/thread.cpp
> our conversion can not be a constexpr before c++11, use 0 instead
> 
> src/hotspot/share/utilities/debug.hpp
> stop using system headers for NULL
> 
> src/hotspot/share/utilities/globalDefinitions.cpp
> change to static asserts, and remove one sizeof(NULL) assert.
> I will change the assert on sizeof(uintptr_t) to sizeof(NULL_WORD) in 
> the next webrev as it is both wrong and not clear (I defined NULL_WORD 
> to be of type intptr_t and not uintptr_t)
> 
> src/hotspot/share/utilities/globalDefinitions.hpp
> include the new definition of NULL *after* system includes, also do 
> redefinition after include guard end
> 
> src/hotspot/share/utilities/globalDefinitions_gcc.hpp
> remove old definition of NULL
> 
> src/hotspot/share/utilities/globalDefinitions_solstudio.hpp
> remove old definition of NULL
> 
> src/hotspot/share/utilities/globalDefinitions_visCPP.hpp
> remove old definition of NULL
> 
> src/hotspot/share/utilities/globalDefinitions_xlc.hpp
> remove old definition of NULL
> 
> src/hotspot/share/utilities/linkedlist.hpp
> new NULL value does unfortunately not convert to method pointers
> 
> src/hotspot/share/utilities/null.hpp
> new NULL
> 
> test/hotspot/gtest/gc/z/test_zAddress.cpp
> correct the usage of integer type from NULL to NULL_WORD
> 
> test/hotspot/gtest/unittest.hpp
> do not use system NULL
> 
> I have a follow up patch that tries to (totally) remove usage of system 
> header NULL, but that is orthogonal to this patch so I did not want to 
> mix them if the approach is not liked.
> 
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8240110
> 
> Webrev:
> http://cr.openjdk.java.net/~lkorinth/8240110/part1 (NULL)
> http://cr.openjdk.java.net/~lkorinth/8240110/part2 (NULL_WORD)
> http://cr.openjdk.java.net/~lkorinth/8240110/part3 (reverse somewhat)
> http://cr.openjdk.java.net/~lkorinth/8240110/0???? (combined change that 
> I will create incremental changes against)
> 
> Testing:
> tier 1-3
> 
> Thanks,
> Leo

From kim.barrett at oracle.com  Tue Mar 31 12:06:22 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Tue, 31 Mar 2020 08:06:22 -0400
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
Message-ID: <EF20ABFB-CEBD-4DD4-9305-897287959C6B@oracle.com>

> On Mar 29, 2020, at 8:24 PM, Leo Korinth <leo.korinth at oracle.com> wrote:
> 
> Hi, could I have a review on this change to improve NULL?

I've not gone through all the changes since I disagree with a core
part of this change.  No point in looking at most of the rest until
that's been discussed.

------------------------------------------------------------------------------
src/hotspot/share/utilities/null.hpp

39 // This line is put after the include guard on purpose, this will make it more likely that NULL is defined NULL (NULL can be re-defined by system headers).
40 // I have not found a good way to make the system headers not define NULL. This line really helps to remove bad definitions of NULL, though not completely.
41 #undef NULL
42 #define NULL 0

What is a "bad definition of NULL"?

I don't think we should make such a change to NULL.

As you note, it's hard to ensure this definition actually gets used.
I think trying to provide our own definition of NULL that is different
from what's provided by the standard library is just asking for
trouble, probably even more so than the assert problem.

Also, this removes some beneficial type checking on some platforms
(notably on at least some gcc platforms, and probably clang too) where
NULL is a special non-integer value (__null) that prevents arithmetic
and is guaranteed to have the same size as a pointer.

And #define'ing NULL as 0 may be actively wrong for the reasons
discussed in the #ifdef SOLARIS definition in globalDefinitions_gcc.hpp.
The referenced bug says something like that code was needed for a
broken standard library on some unknown proprietary OS; it's unclear why
that code is conditionalized on SOLARIS, since the bug seems to suggest
that Solaris header actually do the right thing.  Note also that this
code is deprecated by JEP 362.

It would be interesting to find out if the AIX problem with dirent.h
unconditionally redefining NULL still exists.

I think the not-platform-specific definition of NULL_WORD is fine, but
could just be in globalDefinitions.hpp.  But there's no ODR issue so
long as there's only one definition per translation unit, since const
has translation-unit scope.  It does mean that pointers to NULL_WORD
from different translation units compare !=, but that's expected.  And
being more careful about the use of NULL vs NULL_WORD is good.  A
non-integral definition of NULL helps find some of those.  (Though not
ones like you found in interpreterRT_aarch64.cpp.)

I don't particularly like the changes from NULL to the literal 0 null
pointer constant; I don't think that improves readability at all.  I
wouldn't object to using nullptr in those places, but we either need
to wait for C++11 or provide a C++98 implementation until then.  I
think it's possible to get fairly close to a C++98 nullptr / nullptr_t
implementation, but there are a few features I'm not sure how to
implement in C++98.  (Having a constexpr constructor being one.)

So while I think some wrong code has been turned up and should be
fixed, I don't think the attempt to change NULL should proceed.

I have no particular opinion about the goodness of NULL_WORD as a
concept.  As noted, it is widely used.

------------------------------------------------------------------------------
src/hotspot/share/runtime/os.inline.hpp
34 #include OS_HEADER_INLINE(os)
35 #undef NULL
36 #include "utilities/null.hpp"

[added lines 35-36]

If that's the sort of thing needed by the proposed change to NULL,
that's another reason to not go down that path.

------------------------------------------------------------------------------ 
src/hotspot/share/runtime/os.cpp
1324   if (path == NULL || strlen(path) == 0 || file_name_length == (size_t)NULL) {
=>
1324   if (path == NULL || strlen(path) == 0 || file_name_length == 0) {

Wow!  Nice find!

------------------------------------------------------------------------------

Regarding David's comment: "(I have to say it would nice if whenever
you saw " x = NULL" that you knew x was a pointer type.)"
Unfortunately, there's no such guarantee.  NULL is a null pointer
constant, and a null pointer constant *is* an integral constant
expression, by definition (C++98 4.10/1).


From thomas.stuefe at gmail.com  Tue Mar 31 13:35:31 2020
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Tue, 31 Mar 2020 15:35:31 +0200
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <EF20ABFB-CEBD-4DD4-9305-897287959C6B@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
 <EF20ABFB-CEBD-4DD4-9305-897287959C6B@oracle.com>
Message-ID: <CAA-vtUwzsyQ-Oczrf7G6y9UzXe5oh9U0SRDD-9j5qxoOZTonEw@mail.gmail.com>

<snip>

And #define'ing NULL as 0 may be actively wrong for the reasons
> discussed in the #ifdef SOLARIS definition in globalDefinitions_gcc.hpp.
> The referenced bug says something like that code was needed for a
> broken standard library on some unknown proprietary OS; it's unclear why
> that code is conditionalized on SOLARIS, since the bug seems to suggest
> that Solaris header actually do the right thing.  Note also that this
> code is deprecated by JEP 362.
>
> It would be interesting to find out if the AIX problem with dirent.h
> unconditionally redefining NULL still exists.
>
>
As of AIX 7.2, it still exists.

Cheers, Thomas

From mandy.chung at oracle.com  Tue Mar 31 18:06:58 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Tue, 31 Mar 2020 11:06:58 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
Message-ID: <940c6907-612e-8744-376c-5362991d4a42@oracle.com>

This patch addresses Joe's feedback on the CSR [1]:

http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03-delta-jdarcy/

Specifically, it adds to the class specification of java.lang.Class to 
describe how the relevant methods behave for hidden classes.? In 
addition, use the new inline @jvms tag.

Thanks
Mandy
[1] https://bugs.openjdk.java.net/browse/JDK-8238359

On 3/26/20 4:57 PM, Mandy Chung wrote:
> Please review the implementation of JEP 371: Hidden Classes. The main 
> changes are in core-libs and hotspot runtime area.? Small changes are 
> made in javac, VM compiler (intrinsification of Class::isHiddenClass), 
> JFR, JDI, and jcmd.? CSR [1]has been reviewed and is in the finalized 
> state (see specdiff and javadoc below for reference).
>
> Webrev:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
>
>
> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's point
> of view, a hidden class is a normal class except the following:
>
> - A hidden class has no initiating class loader and is not registered 
> in any dictionary.
> - A hidden class has a name containing an illegal character 
> `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature` 
> returns "Lp/Foo.0x1234;".
> - A hidden class is not modifiable, i.e. cannot be redefined or 
> retransformed. JVM TI IsModifableClass returns false on a hidden.
> - Final fields in a hidden class is "final".? The value of final 
> fields cannot be overriden via reflection.? setAccessible(true) can 
> still be called on reflected objects representing final fields in a 
> hidden class and its access check will be suppressed but only have 
> read-access (i.e. can do Field::getXXX but not setXXX).
>
> Brief summary of this patch:
>
> 1. A new Lookup::defineHiddenClass method is the API to create a 
> hidden class.
> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG 
> option that
> ?? can be specified when creating a hidden class.
> 3. A new Class::isHiddenClass method tests if a class is a hidden class.
> 4. Field::setXXX method will throw IAE on a final field of a hidden class
> ?? regardless of the value of the accessible flag.
> 5. JVM_LookupDefineClass is the new JVM entry point for 
> Lookup::defineClass
> ?? and defineHiddenClass to create a class from the given bytes.
> 6. ClassLoaderData implementation is not changed.? There is one 
> primary CLD
> ?? that holds the classes strongly referenced by its defining loader.? 
> There
> ?? can be zero or more additional CLDs - one per weak class.
> 7. Nest host determination is updated per revised JVMS 5.4.4. Access 
> control
> ?? check no longer throws LinkageError but instead it will throw IAE with
> ?? a clear message if a class fails to resolve/validate the nest host 
> declared
> ?? in NestHost/NestMembers attribute.
> 8. JFR, jcmd, JDI are updated to support hidden classes.
> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
> ?? and generate a bridge method to desuger a method reference to a 
> protected
> ?? method in its supertype in a different package
>
> This patch also updates StringConcatFactory, LambdaMetaFactory, and 
> LambdaForms
> to use hidden classes.? The webrev includes changes in nashorn to 
> hidden class
> and I will update the webrev if JEP 372 removes it any time soon.
>
> We uncovered a bug in Lookup::defineClass spec throws LinkageError and 
> intends
> to have the newly created class linked.? However, the implementation 
> in 14
> does not link the class.? A separate CSR [2] proposes to update the
> implementation to match the spec.? This patch fixes the implementation.
>
> The spec update on JVM TI, JDI and Instrumentation will be done as
> a separate RFE [3].? This patch includes new tests for JVM TI and
> java.instrument that validates how the existing APIs work for hidden 
> classes.
>
> javadoc/specdiff
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
>
>
> JVMS 5.4.4 change:
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
>
>
> CSR:
> https://bugs.openjdk.java.net/browse/JDK-8238359
>
> Thanks
> Mandy
> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
> [3] https://bugs.openjdk.java.net/browse/JDK-8230502


From leo.korinth at oracle.com  Tue Mar 31 19:10:00 2020
From: leo.korinth at oracle.com (Leo Korinth)
Date: Tue, 31 Mar 2020 21:10:00 +0200
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <135a6496-5b54-9117-5cbf-e3211ddc32dd@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
 <135a6496-5b54-9117-5cbf-e3211ddc32dd@oracle.com>
Message-ID: <bbb2b5ef-62f2-6507-95f7-03712a382153@oracle.com>

Thanks for taking time to look through this!

On 31/03/2020 12:04, David Holmes wrote:
> Hi Leo,
> 
> I looked at the "combined change". Overall I like the cleanup, but I did 
> find a few questionable items:
> 
> src/hotspot/cpu/arm/interpreterRT_arm.cpp
> 
> -????? _toGP[_last_gp++] = (*(intptr_t*)from_addr == 0) ? NULL : from_addr;
> +????? _toGP[_last_gp++] = (*(intptr_t*)from_addr == 0) ? NULL_WORD : 
> from_addr;
>  ???? } else {
> -????? *_to++ = (*(intptr_t*)from_addr == 0) ? NULL : from_addr;
> +????? *_to++ = (*(intptr_t*)from_addr == 0) ? NULL_WORD : from_addr;
> 
> If from_addr is being treated as intptr_t* then shouldn't it be compared 
> against NULL_WORD rather than 0?

I agree that it could be compared against NULL_WORD, do you want me to 
change it? (I have no opinion on this issue)

> 
> ---
> 
> General query: are changes like:
> 
> Foo f = NULL;
> 
> to
> 
> Foo f(NULL);
> 
> just making the implicit constructor selection explicit? (I have to say 
> it would nice if whenever you saw " x = NULL" that you knew x was a 
> pointer type.)

Yes, with my original NULL two conversions would not compile. Assigning 
is adding one extra step from direct construction. The new style is IMO 
easier to read.

> ---
> 
> src/hotspot/cpu/sparc/c1_CodeStubs_sparc.cpp
> 
> -??? AddressLiteral addrlit(NULL, metadata_Relocation::spec(_index));
> +??? AddressLiteral addrlit(NULL_WORD, metadata_Relocation::spec(_index));
> 
> If the first constructor arg is an "address" which seems to be the 
> pointer type "uchar*" then surely NULL is correct here, not NULL_WORD
The first argument (to the constructor) can be address (u_char*), 
jobject (_jobject*), or intptr_t (not a pointer). Surely if we send a 
pointer (NULL), the compiler would have a hard time choosing between the 
first two pointer constructors, right? Therefore we send a non-pointer!

The reason this worked is because I believe integer /promotion/ has 
precedence over 0-to-pointer /implicit conversion/, but I am not too 
knowledgable about these things... Thus the old NULL prefer to be 
promoted to integral type! Do you agree or am I misunderstanding something?

This is /highly/ non-intuitive and one /strong/ argument to change NULL 
to be of the future c++11 nullptr_t type! (or my failed null_t type).

> 
> Same applies to other AddressLiteral changes.

After above reasoning, I should keep my changes right?

> ---
> 
> src/hotspot/share/code/codeHeapState.cpp
> 
> This doesn't look right:
> 
> -??? if (((_termString) != NULL) && (strlen(_termString) > 0)){\
> +??? if (((_termString) != 0) && (strlen(_termString) > 0)){?? \
> 
> _termString should be a "const char*" so comparison against NULL is 
> correct.

No, _termString is mostly a string literal and not a "const char*" and 
/can not/ in the instances of literal be NULL. Thus the (windows) 
compiler (rightly) will complain if you use a "proper" pointer-type 
NULL. The "technical" reason is that the array will not decay to a 
pointer and my original NULL will thus not convert to the array. Which 
makes sense.

The reason I did not remove the test is that I was afraid that I either 
missed some invocation of the macro where _termString really was of 
pointer type, or that someone in the future would use it as such. The 
real problem here is macros (and c strings).

It is a pity that neither gcc nor clang complains (I am not sure about 
the standard), I am bit unsure about nullptr behaviour in the future in 
this regard, but I hope it will complain as good as my failed NULL!

> ---
> 
> share/compiler/directivesParser.cpp
> 
>  > new NULL value does unfortunately not convert to method pointers
> 
> Is that a temporary limitation? I'm not sure what progress were are 
> making if some pointers can use NULL and others must use 0.

I believe this to be a temporary limitation, my understanding is that 
when we get c++11 nullptr will convert to method pointers as well, I 
could and think I should revert these changes as I did revert my 
implementation of NULL, would you like that? My null_t type could, to my 
knowledge, not auto convert to method pointers at least not without 
template varargs stuff that comes in later versions of the standard.

> 
> ---
> 
> src/hotspot/share/jfr/jni/jfrJavaCall.cpp
> 
> I don't understand why the NULL check has been removed:
> 
> -? assert(_storage != NULL, "invariant");
> +? assert(_storage_index < (int) ARRAY_SIZE(_storage), "invariant");

Because _storage can not be NULL (it is an array and not a pointer). I 
feel the new assert is strictly superior! Also an example of the power 
of static typing and more precise types.

> 
> ---
> 
> src/hotspot/share/jvmci/jvmciCompilerToVMInit.cpp
> 
> -??? if (value != NULL) { \
> +??? if ((value) != 0) { \
> 
> Unclear what type value is - in some cases char* in others I can't 
> determine.

This is the same issue as before with string literals and macros, my 
arguing is that I will not try to prove that value is (or in the future 
always will be) a string literal, instead I will make the code work for 
both string literals and pointers. I hope you do not mind the extra 
parentheses I added around "value".

> 
> ---
> 
> src/hotspot/share/jvmci/jvmciExceptions.hpp
> 
> -#define JVMCI_CHECK_0????????????????????????????????? JVMCI_CHECK_(0)
> -#define JVMCI_CHECK_NULL?????????????????????????????? JVMCI_CHECK_(NULL)
> +#define JVMCI_CHECK_0 JVMCI_CHECK_(NULL_WORD)
> +#define JVMCI_CHECK_NULL?????????????????????????????? JVMCI_CHECK_(0)
> 
> These just seem plain wrong. CHECK_0 is for returning a integral zero 
> value (e.g. jint, jbtye etc), nothing to do with intptr_t and NULL_WORD.

I feel NULL_WORD act as a more restricted 0 integral literal than the 
zero literal itself (it does not auto converts to pointers). The size of 
the zero constant is of less interest to me.

> Having CHECK_NULL become CHECK_(0) seems completely wrong when you 
> consider that if you were expanding by hand in a function returning a 
> pointer then you would write "return NULL".

I agree it is not as distinct as NULL. Now for example, JVMCIObjectArray 
is not a pointer (strictly at least, it is a class). With my first NULL 
it would need to go through two conversions to become a JVMCIObjectArray 
(NULL converted to any pointer and any pointer converted to 
JVMCIObjectArray), thus it would not compile.

There are several solutions to this including:
1) use 0 literal as I did.
2) change JVMCIObjectArray constructor to take a null(ptr)_t instead of 
a pointer (this is a nice example of how a specific null type can make 
that constructor /total/ and not require the assert(v == NULL, "must be 
NULL");). Although this is a tempting solution, I had to abandon my own 
NULL and this solution is no longer possible until c++11.
3) use NULL as before, and fix it when we get nullptr_t.
4) create another macro in some way?

> Further if this chasnge is 
> necessary/desirable for JVMCI_CHECK_NULL then surely the same is true 
> for the basic CHECK_NULL in exceptions.hpp ?

No, the same is not true for CHECK_NULL. I have mostly changed stuff 
that does not compile. This still compiles.

I could change CHECK_0 to use NULL_WORD though and if I do, I get 
compiler errors. CHECK_0 is used for both pointers and integral zeroes 
and types that implicitly converts from either. What the original 
thought was is unclear to me; is CHECK_0 to be used to return things 
convertible from pointers? If so, what is CHECK_NULL for?

> 
> Aside:
> 
> ./share/jvmci/jvmciCompilerToVM.cpp
> 
> C2V_VMENTRY_0(jboolean, isInternedString, (JNIEnv* env, jobject, jobject 
> object))
>  ? Handle str = JVMCIENV->asConstant(JVMCIENV->wrap(object), 
> JVMCI_CHECK_0);
>  ? if (!java_lang_String::is_instance(str())) {
>  ??? return false;
> 
> CHECK_0 should be CHECK_false. Ditto in isInstance, 
> isCurrentThreadAttached,? attachCurrentThread.
> 

Maybe all this should be cleaned up separately? Maybe this macro 
solution could be improved? And preferably not in this change?

> 
> src/hotspot/share/runtime/sweeper.hpp
> 
>  > gcc does unfortunately not know that our NULL is a safe format string
> 
> -? static void log_sweep(const char* msg, const char* format = NULL, 
> ...) ATTRIBUTE_PRINTF(2, 3);
> +? static void log_sweep(const char* msg, const char* format = 0, ...) 
> ATTRIBUTE_PRINTF(2, 3);
> 
> I don't follow. This isn't "our NULL" we're using the toolchain's NULL, 
> and format is a pointer type so this should be fine. Is gcc broken?

Sorry, my comment is unclear, this was from my original failed NULL 
implementation. I will revert this to NULL as I expect this to work with 
nullptr_t in the future.


> ---
> 
> src/hotspot/share/utilities/globalDefinitions_solstudio.hpp
> 
> It isn't clear that the issue of passing 0 as NULL in a varargs 
> situation is now fixed in Sol Studio.
> 

I guess tier1-3 would fail if not? Maybe that is a false assumption of mine?

> 
> src/hotspot/share/utilities/linkedlist.hpp
> 
> -??? return equal<E>(_data, t, NULL);
> +??? return equal<E>(_data, t, 0);
> 
> Again concerned about the method/function pointer situation.

I am not concerned :-) but I will revert this as it is not necessary now 
or in the future as I reverted my original implementation and c++11 will 
fix this.

> 
> ----
> 
> That's all from me.
> 
> Thanks,
> David
> -----

Before ending this mail, I also want to raise one question. Maybe I 
ought to #ifdef a #define NULL __null for the gcc tool chain to give 
better typing support until the real nullptr? I would hate to wait, but 
maybe changing to c++11 will take longer than I hope for.


Many thanks!
Leo
> 
> 
> 
> On 30/03/2020 10:24 am, Leo Korinth wrote:
>> Hi, could I have a review on this change to improve NULL?
>>
>> I try to:
>> 1) make NULL only convert to pointer types for better type safety, 
>> __null on gcc achieves some of this, but can be casted away.
>> 2) define NULL in one place, in a platform independent way, no need to 
>> define it for every tool chain.
>> 3) make NULL be of a special type (null_t) so that one can template 
>> specialise against it. This is for the future!
>>
>> I fail with this, but continue to read...
>>
>> I try to achieve this by defining NULL as a variable of type null_t in 
>> the header null.hpp using ODR. null_t converts to (almost) any pointer 
>> type.
>>
>> Minor negative issues:
>> 1) No conversion to method pointers, they are only used in three 
>> places though, I use a literal 0 instead.
>> 2) gcc does unfortunately not know that our NULL is a safe format 
>> string, only needed to change one place.
>> 3) No constexpr yet, so I needed to use 0 to initialise a thread 
>> variable.
>>
>> IMO positive issues:
>> 1) I have fixed many bad usages of NULL for integer types and instead 
>> used NULL_WORD or 0.
>> 2) Too many implicit conversions in certain places makes the code not 
>> compile, the changed code does look better and more idiomatic IMO.
>> 3) Possible benign bug found where fields are initialised to NULL (not 
>> in constructor), this should not be possible right?
>> 4) NULL does not convert to array types, on Windows this results in 
>> compile time error as the windows compiler does not decay arrays when 
>> comparing to null_t, this did locate one bug, and possibly some 
>> unnecessary comparisons in xmacros that I kept by changing NULL to 0.
>> 5) Found a reinterpret_cast on NULL that is not allowed by the standard.
>>
>> I also made a new NULL_WORD in the same way as I did NULL, but instead 
>> converting to a intptr_t of value 0. I was about to release NULL_WORD 
>> as a separate patch, but due to the current practice of defining 
>> NULL_WORD as NULL on many platforms, it was easier to merge these two 
>> changes into one. Only small issues in JVMCI code for this change, and 
>> one fix in sparc assembler. I find NULL_WORD questionable, but it is 
>> widely used.
>>
>> Something to notice is the strange comment in the solstudio header 
>> about intptr_t being wider than int on some systems that is to my 
>> understanding false (for systems we support). I have a static assert 
>> to check that intptr_t is indeed of (char) pointer size. I did remove 
>> an assert that checks the size of NULL.
>>
>> Now, the sad thing is that my change does not work fully, I can not 
>> make NULL constexpr before c++ 11, and thus several initialisation 
>> order bugs appear on global variables set to NULL. I could set them to 
>> zero or not set them at all, and they would work, but it feels brittle 
>> and could cause future bugs when people expect NULLs on globals to be 
>> initialised directly. Thus instead, I propose that I revert somewhat 
>> and define NULL as 0 and change NULL_WORD to a constant 0 of type 
>> intptr_t.
>>
>> When we get c++>=11 we can define NULL to nullptr of type nullptr_t, 
>> and hopefully that will be a simple fix after this change.
>>
>> What do you think?
>>
>> Below I have given a short description of the changes:
>> src/hotspot/cpu/aarch64/interpreterRT_aarch64.cpp
>> correct the usage of integer type from NULL to NULL_WORD
>>
>> src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp
>> too many implicit conversions
>>
>> src/hotspot/cpu/arm/interpreterRT_arm.cpp
>> correct the usage of integer type from NULL to NULL_WORD
>>
>> src/hotspot/cpu/ppc/c1_LIRAssembler_ppc.cpp
>> too many implicit conversions
>>
>> src/hotspot/cpu/sparc/c1_CodeStubs_sparc.cpp
>> correct the usage of integer type from NULL to NULL_WORD
>>
>> src/hotspot/cpu/sparc/c1_LIRAssembler_sparc.cpp
>> correct the usage of integer type from NULL to NULL_WORD, and the
>> opposite way as well!
>>
>> src/hotspot/cpu/sparc/interpreterRT_sparc.cpp
>> correct the usage of integer type from NULL to NULL_WORD
>>
>> src/hotspot/cpu/x86/interp_masm_x86.cpp
>> correct the usage of integer type from NULL to NULL_WORD
>>
>> src/hotspot/cpu/x86/interpreterRT_x86_64.cpp
>> correct the usage of integer type from NULL to NULL_WORD
>>
>> src/hotspot/cpu/x86/stubGenerator_x86_64.cpp
>> correct the usage of integer type from NULL to NULL_WORD
>>
>> src/hotspot/os/linux/cgroupV1Subsystem_linux.hpp
>> correct member initialisation to be in initialise list, gcc bug?
>>
>> src/hotspot/os/linux/cgroupV2Subsystem_linux.hpp
>> correct member initialisation to be in initialise list, gcc bug?
>>
>> src/hotspot/os/solaris/attachListener_solaris.cpp
>> correct the usage of integer type from NULL to 0
>>
>> src/hotspot/os/windows/osThread_windows.cpp
>> correct the usage of integer type from NULL to 0
>>
>> src/hotspot/os/windows/os_windows.cpp
>> correct the usage of integer type from NULL to 0
>>
>> src/hotspot/os/windows/pdh_interface.cpp
>> new NULL value does unfortunately not convert to method pointers
>>
>> src/hotspot/share/classfile/javaClasses.cpp
>> too many implicit conversions
>>
>> src/hotspot/share/classfile/moduleEntry.hpp
>> too many implicit conversions
>>
>> src/hotspot/share/classfile/modules.cpp
>> too many implicit conversions
>>
>> src/hotspot/share/code/codeHeapState.cpp
>> new NULL does not compare with arrays witch is great, maybe (xmacros) 
>> the NULL check can be removed, this change is safe though
>>
>> src/hotspot/share/compiler/directivesParser.cpp
>> new NULL value does unfortunately not convert to method pointers
>>
>> src/hotspot/share/gc/g1/g1ConcurrentRefine.cpp
>> remove unnecessary system include, might redefine NULL
>>
>> src/hotspot/share/gc/parallel/psAdaptiveSizePolicy.cpp
>> remove unnecessary system include, might redefine NULL
>>
>> src/hotspot/share/gc/parallel/psParallelCompact.cpp
>> remove unnecessary system include, might redefine NULL
>>
>> src/hotspot/share/gc/shared/memset_with_concurrent_readers.hpp
>> remove unnecessary system include, might redefine NULL
>>
>> src/hotspot/share/gc/z/zErrno.cpp
>> remove unnecessary system include, might redefine NULL
>>
>> src/hotspot/share/jfr/jni/jfrGetAllEventClasses.cpp
>> fix bad assert found due to new NULL
>>
>> src/hotspot/share/jvmci/jvmciCompilerToVM.cpp
>> too many implicit conversions, and misc changes
>>
>> src/hotspot/share/jvmci/jvmciCompilerToVMInit.cpp
>> new NULL does not compare with arrays witch is great, maybe (xmacros) 
>> the NULL check can be removed, this change is safe though
>>
>> src/hotspot/share/jvmci/jvmciExceptions.hpp
>> too many implicit conversions
>>
>> src/hotspot/share/oops/access.hpp reinterpret_cast is not safe on 
>> NULL, rewrite
>>
>> src/hotspot/share/oops/constantPool.cpp
>> too many implicit conversions
>>
>> src/hotspot/share/oops/cpCache.cpp
>> too many implicit conversions
>>
>> src/hotspot/share/oops/klass.cpp
>> too many implicit conversions
>>
>> src/hotspot/share/oops/oopsHierarchy.hpp
>> add null_t overload to oop class
>>
>> src/hotspot/share/prims/jni.cpp
>> too many implicit conversions
>>
>> src/hotspot/share/prims/jvm.cpp
>> too many implicit conversions
>>
>> src/hotspot/share/prims/methodHandles.cpp
>> too many implicit conversions
>>
>> src/hotspot/share/runtime/os.cpp
>> correct the usage of integer type from NULL to 0
>>
>> src/hotspot/share/runtime/os.inline.hpp
>> #undef NULL set by system headers, use the new NULL instead
>>
>> src/hotspot/share/runtime/sweeper.hpp
>> gcc does unfortunately not know that our NULL is a safe format string
>>
>> src/hotspot/share/runtime/thread.cpp
>> our conversion can not be a constexpr before c++11, use 0 instead
>>
>> src/hotspot/share/utilities/debug.hpp
>> stop using system headers for NULL
>>
>> src/hotspot/share/utilities/globalDefinitions.cpp
>> change to static asserts, and remove one sizeof(NULL) assert.
>> I will change the assert on sizeof(uintptr_t) to sizeof(NULL_WORD) in 
>> the next webrev as it is both wrong and not clear (I defined NULL_WORD 
>> to be of type intptr_t and not uintptr_t)
>>
>> src/hotspot/share/utilities/globalDefinitions.hpp
>> include the new definition of NULL *after* system includes, also do 
>> redefinition after include guard end
>>
>> src/hotspot/share/utilities/globalDefinitions_gcc.hpp
>> remove old definition of NULL
>>
>> src/hotspot/share/utilities/globalDefinitions_solstudio.hpp
>> remove old definition of NULL
>>
>> src/hotspot/share/utilities/globalDefinitions_visCPP.hpp
>> remove old definition of NULL
>>
>> src/hotspot/share/utilities/globalDefinitions_xlc.hpp
>> remove old definition of NULL
>>
>> src/hotspot/share/utilities/linkedlist.hpp
>> new NULL value does unfortunately not convert to method pointers
>>
>> src/hotspot/share/utilities/null.hpp
>> new NULL
>>
>> test/hotspot/gtest/gc/z/test_zAddress.cpp
>> correct the usage of integer type from NULL to NULL_WORD
>>
>> test/hotspot/gtest/unittest.hpp
>> do not use system NULL
>>
>> I have a follow up patch that tries to (totally) remove usage of 
>> system header NULL, but that is orthogonal to this patch so I did not 
>> want to mix them if the approach is not liked.
>>
>> Bug:
>> https://bugs.openjdk.java.net/browse/JDK-8240110
>>
>> Webrev:
>> http://cr.openjdk.java.net/~lkorinth/8240110/part1 (NULL)
>> http://cr.openjdk.java.net/~lkorinth/8240110/part2 (NULL_WORD)
>> http://cr.openjdk.java.net/~lkorinth/8240110/part3 (reverse somewhat)
>> http://cr.openjdk.java.net/~lkorinth/8240110/0???? (combined change 
>> that I will create incremental changes against)
>>
>> Testing:
>> tier 1-3
>>
>> Thanks,
>> Leo

From mandy.chung at oracle.com  Tue Mar 31 19:25:53 2020
From: mandy.chung at oracle.com (Mandy Chung)
Date: Tue, 31 Mar 2020 12:25:53 -0700
Subject: Review Request: 8238358: Implementation of JEP 371: Hidden Classes
In-Reply-To: <940c6907-612e-8744-376c-5362991d4a42@oracle.com>
References: <0d824617-3eaf-727c-6eb8-be2414111510@oracle.com>
 <940c6907-612e-8744-376c-5362991d4a42@oracle.com>
Message-ID: <cfe9c4ae-f70e-6408-9d8b-70f02158ff3a@oracle.com>

Alex's feedback:? rename isHiddenClass to isHidden as it can be a hidden 
class or interface.

`isLocalClass` and `sAnonymousClass` are correct because the Java 
language only has local classes and anon classes, not local interfaces 
or anon. interfaces.? `isHidden` is like `isSynthetic`, it could be a 
class or interface.

Although isHiddenClass seems clearer, I'm okay to rename it to `isHidden`.

Mandy

On 3/31/20 11:06 AM, Mandy Chung wrote:
> This patch addresses Joe's feedback on the CSR [1]:
>
> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03-delta-jdarcy/ 
>
>
> Specifically, it adds to the class specification of java.lang.Class to 
> describe how the relevant methods behave for hidden classes.? In 
> addition, use the new inline @jvms tag.
>
> Thanks
> Mandy
> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
>
> On 3/26/20 4:57 PM, Mandy Chung wrote:
>> Please review the implementation of JEP 371: Hidden Classes. The main 
>> changes are in core-libs and hotspot runtime area.? Small changes are 
>> made in javac, VM compiler (intrinsification of 
>> Class::isHiddenClass), JFR, JDI, and jcmd.? CSR [1]has been reviewed 
>> and is in the finalized state (see specdiff and javadoc below for 
>> reference).
>>
>> Webrev:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/webrev.03 
>>
>>
>> Hidden class is created via `Lookup::defineHiddenClass`. From JVM's 
>> point
>> of view, a hidden class is a normal class except the following:
>>
>> - A hidden class has no initiating class loader and is not registered 
>> in any dictionary.
>> - A hidden class has a name containing an illegal character 
>> `Class::getName` returns `p.Foo/0x1234` whereas `GetClassSignature` 
>> returns "Lp/Foo.0x1234;".
>> - A hidden class is not modifiable, i.e. cannot be redefined or 
>> retransformed. JVM TI IsModifableClass returns false on a hidden.
>> - Final fields in a hidden class is "final".? The value of final 
>> fields cannot be overriden via reflection.? setAccessible(true) can 
>> still be called on reflected objects representing final fields in a 
>> hidden class and its access check will be suppressed but only have 
>> read-access (i.e. can do Field::getXXX but not setXXX).
>>
>> Brief summary of this patch:
>>
>> 1. A new Lookup::defineHiddenClass method is the API to create a 
>> hidden class.
>> 2. A new Lookup.ClassOption enum class defines NESTMATE and STRONG 
>> option that
>> ?? can be specified when creating a hidden class.
>> 3. A new Class::isHiddenClass method tests if a class is a hidden class.
>> 4. Field::setXXX method will throw IAE on a final field of a hidden 
>> class
>> ?? regardless of the value of the accessible flag.
>> 5. JVM_LookupDefineClass is the new JVM entry point for 
>> Lookup::defineClass
>> ?? and defineHiddenClass to create a class from the given bytes.
>> 6. ClassLoaderData implementation is not changed.? There is one 
>> primary CLD
>> ?? that holds the classes strongly referenced by its defining 
>> loader.? There
>> ?? can be zero or more additional CLDs - one per weak class.
>> 7. Nest host determination is updated per revised JVMS 5.4.4. Access 
>> control
>> ?? check no longer throws LinkageError but instead it will throw IAE 
>> with
>> ?? a clear message if a class fails to resolve/validate the nest host 
>> declared
>> ?? in NestHost/NestMembers attribute.
>> 8. JFR, jcmd, JDI are updated to support hidden classes.
>> 9. update javac LambdaToMethod as lambda proxy starts using nestmates
>> ?? and generate a bridge method to desuger a method reference to a 
>> protected
>> ?? method in its supertype in a different package
>>
>> This patch also updates StringConcatFactory, LambdaMetaFactory, and 
>> LambdaForms
>> to use hidden classes.? The webrev includes changes in nashorn to 
>> hidden class
>> and I will update the webrev if JEP 372 removes it any time soon.
>>
>> We uncovered a bug in Lookup::defineClass spec throws LinkageError 
>> and intends
>> to have the newly created class linked.? However, the implementation 
>> in 14
>> does not link the class.? A separate CSR [2] proposes to update the
>> implementation to match the spec.? This patch fixes the implementation.
>>
>> The spec update on JVM TI, JDI and Instrumentation will be done as
>> a separate RFE [3].? This patch includes new tests for JVM TI and
>> java.instrument that validates how the existing APIs work for hidden 
>> classes.
>>
>> javadoc/specdiff
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/api/
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/specdiff/ 
>>
>>
>> JVMS 5.4.4 change:
>> http://cr.openjdk.java.net/~mchung/valhalla/webrevs/hidden-classes/Draft-JVMS-HiddenClasses.pdf 
>>
>>
>> CSR:
>> https://bugs.openjdk.java.net/browse/JDK-8238359
>>
>> Thanks
>> Mandy
>> [1] https://bugs.openjdk.java.net/browse/JDK-8238359
>> [2] https://bugs.openjdk.java.net/browse/JDK-8240338
>> [3] https://bugs.openjdk.java.net/browse/JDK-8230502
>


From leo.korinth at oracle.com  Tue Mar 31 21:11:20 2020
From: leo.korinth at oracle.com (Leo Korinth)
Date: Tue, 31 Mar 2020 23:11:20 +0200
Subject: RFR: 8240110: Improve NULL
In-Reply-To: <EF20ABFB-CEBD-4DD4-9305-897287959C6B@oracle.com>
References: <e6ecfd0c-372c-fe0c-1939-d41d1bfc8736@oracle.com>
 <EF20ABFB-CEBD-4DD4-9305-897287959C6B@oracle.com>
Message-ID: <c8b4e0ba-9f3e-123f-aa1c-540f14ea49b0@oracle.com>



On 31/03/2020 14:06, Kim Barrett wrote:
>> On Mar 29, 2020, at 8:24 PM, Leo Korinth <leo.korinth at oracle.com> wrote:
>>
>> Hi, could I have a review on this change to improve NULL?
> 
> I've not gone through all the changes since I disagree with a core
> part of this change.  No point in looking at most of the rest until
> that's been discussed.
> 
> ------------------------------------------------------------------------------
> src/hotspot/share/utilities/null.hpp
> 
> 39 // This line is put after the include guard on purpose, this will make it more likely that NULL is defined NULL (NULL can be re-defined by system headers).
> 40 // I have not found a good way to make the system headers not define NULL. This line really helps to remove bad definitions of NULL, though not completely.
> 41 #undef NULL
> 42 #define NULL 0
> 
> What is a "bad definition of NULL"?

One that we have not defined in null.hpp

> 
> I don't think we should make such a change to NULL.
> 
> As you note, it's hard to ensure this definition actually gets used.
> I think trying to provide our own definition of NULL that is different
> from what's provided by the standard library is just asking for
> trouble, probably even more so than the assert problem.

I would love to change all occurrences of NULL to nullptr, that would 
solve /all/ problems but the diff would be huge. I do not think 
reviewers will allow such a massive change although that would be my 
first choice. Thus the idea to define NULL to nullptr. At least after 
this change we only have one alternative definition of NULL.

> 
> Also, this removes some beneficial type checking on some platforms
> (notably on at least some gcc platforms, and probably clang too) where
> NULL is a special non-integer value (__null) that prevents arithmetic
> and is guaranteed to have the same size as a pointer.

Type checking will be vastly improved with c++11. You can see all kind 
of "errors" I found in the code already. I do agree with __null though 
and I will #ifdef it in null.hpp (as I said in the reply to David, sorry 
for not getting it the original webrev). But nullptr clearly is superior 
to __null.
> 
> And #define'ing NULL as 0 may be actively wrong for the reasons
> discussed in the #ifdef SOLARIS definition in globalDefinitions_gcc.hpp.
> The referenced bug says something like that code was needed for a
> broken standard library on some unknown proprietary OS; it's unclear why
> that code is conditionalized on SOLARIS, since the bug seems to suggest
> that Solaris header actually do the right thing.  Note also that this
> code is deprecated by JEP 362.

The change "seems" to work, and "should" work.

> 
> It would be interesting to find out if the AIX problem with dirent.h
> unconditionally redefining NULL still exists.

GNU headers also redefine NULL unconditionally, and as I said in the 
mail I do have a patch that #undef NULL after all system headers. 
However this is no problem /I/ have introduced or made worse (to my 
knowledge).
> 
> I think the not-platform-specific definition of NULL_WORD is fine, but
> could just be in globalDefinitions.hpp.  But there's no ODR issue so
> long as there's only one definition per translation unit, since const
> has translation-unit scope.  It does mean that pointers to NULL_WORD
> from different translation units compare !=, but that's expected.  And
> being more careful about the use of NULL vs NULL_WORD is good.  A
> non-integral definition of NULL helps find some of those.  (Though not
> ones like you found in interpreterRT_aarch64.cpp.)
> 
> I don't particularly like the changes from NULL to the literal 0 null
> pointer constant; I don't think that improves readability at all.  I
> wouldn't object to using nullptr in those places, but we either need
> to wait for C++11 or provide a C++98 implementation until then.  I
> think it's possible to get fairly close to a C++98 nullptr / nullptr_t
> implementation, but there are a few features I'm not sure how to
> implement in C++98.  (Having a constexpr constructor being one.)

Have you read null.hpp in part1 of the patch? I feel you miss a lot of 
context. I do /not/ want to define NULL to 0, I want to do it 
temporarily. I do not want to throw away my fixes, but instead prepare 
for your move to later version of c++.

> 
> So while I think some wrong code has been turned up and should be
> fixed, I don't think the attempt to change NULL should proceed.

Two questions:

1) Is my change in any way worse than what we have today (after I fix 
__null)?

2) Would it not be nice for you to just change the NULL #define to 
nullptr when you fix more modern c++?

> 
> I have no particular opinion about the goodness of NULL_WORD as a
> concept.  As noted, it is widely used.
> 
> ------------------------------------------------------------------------------
> src/hotspot/share/runtime/os.inline.hpp
> 34 #include OS_HEADER_INLINE(os)
> 35 #undef NULL
> 36 #include "utilities/null.hpp"
> 
> [added lines 35-36]
> 
> If that's the sort of thing needed by the proposed change to NULL,
> that's another reason to not go down that path.

I would like to have a better solution to system headers that 
unconditionally redefines NULL, unfortunately I have found none. Please 
help me find the better path!

> 
> ------------------------------------------------------------------------------
> src/hotspot/share/runtime/os.cpp
> 1324   if (path == NULL || strlen(path) == 0 || file_name_length == (size_t)NULL) {
> =>
> 1324   if (path == NULL || strlen(path) == 0 || file_name_length == 0) {
> 
> Wow!  Nice find!

__null can be casted to integral, my failed NULL could not.
> ------------------------------------------------------------------------------
> 
> Regarding David's comment: "(I have to say it would nice if whenever
> you saw " x = NULL" that you knew x was a pointer type.)"
> Unfortunately, there's no such guarantee.  NULL is a null pointer
> constant, and a null pointer constant *is* an integral constant
> expression, by definition (C++98 4.10/1).
> 

We can define NULL better than the standard (as nullptr).

Lastly, I do think moving NULL to null.hpp is highly beneficial to the 
current code base. If that can not be accepted by you I will try to at 
least push as many of the "fixes" as possible. But I still hope that you 
will like this and --- maybe --- I just failed to communicate the idea 
in a good way.

One alternative is to remove our NULL definitions altogether and replace 
NULL with nullptr. I would prefer that. Would you? Do you actually think 
we could get such a patch reviewed?

nullptr is so important for type safety and template specialization, I 
feel every step in that direction is important.

Thanks,
Leo