From shade at redhat.com  Wed Jul  1 13:59:04 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Wed, 1 Jul 2020 15:59:04 +0200
Subject: RFR (XS/T) 8248634: Shenandoah: incorrect include in
 shenandoahInitLogger.cpp
Message-ID: <177e1e58-82b8-e4af-d12b-b8f61dd26a4c@redhat.com>

Bug:
  https://bugs.openjdk.java.net/browse/JDK-8248634

diff -r f42f4771b5d1 src/hotspot/share/gc/shenandoah/shenandoahInitLogger.cpp
--- a/src/hotspot/share/gc/shenandoah/shenandoahInitLogger.cpp  Wed Jul 01 15:53:23 2020 +0200
+++ b/src/hotspot/share/gc/shenandoah/shenandoahInitLogger.cpp  Wed Jul 01 15:56:53 2020 +0200
@@ -24,5 +24,5 @@

 #include "precompiled.hpp"
-#include "gc/shenandoah/shenandoahHeap.hpp"
+#include "gc/shenandoah/shenandoahHeap.inline.hpp"
 #include "gc/shenandoah/shenandoahHeapRegion.hpp"
 #include "gc/shenandoah/shenandoahInitLogger.hpp"

Testing: builds

-- 
Thanks,
-Aleksey


From shade at redhat.com  Wed Jul  1 13:59:20 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Wed, 1 Jul 2020 15:59:20 +0200
Subject: RFR (XS) 8248632: Shenandoah: build fails without both JVMTI and JFR
Message-ID: <66cb5669-c1a7-af6c-065d-5640483bb30d@redhat.com>

Bug:
  https://bugs.openjdk.java.net/browse/JDK-8248632

We get dangling ":" when both JFR and JVMTI are disabled. The fix is to put the colon into the macro
calls themselves, which would select it properly:

diff -r 7ab23692b432 src/hotspot/share/gc/shenandoah/shenandoahRootProcessor.hpp
--- a/src/hotspot/share/gc/shenandoah/shenandoahRootProcessor.hpp       Wed Jul 01 15:09:28 2020 +0200
+++ b/src/hotspot/share/gc/shenandoah/shenandoahRootProcessor.hpp       Wed Jul 01 15:51:30 2020 +0200
@@ -91,13 +91,15 @@
 class ShenandoahSerialWeakRoots {
 private:
   JVMTI_ONLY(ShenandoahJVMTIWeakRoot _jvmti_weak_roots;)
   JFR_ONLY(ShenandoahJFRWeakRoot     _jfr_weak_roots;)
 public:
-  ShenandoahSerialWeakRoots(ShenandoahPhaseTimings::Phase phase) :
-  JVMTI_ONLY(_jvmti_weak_roots(phase))
-  JFR_ONLY(JVMTI_ONLY(COMMA)_jfr_weak_roots(phase)) {};
+  ShenandoahSerialWeakRoots(ShenandoahPhaseTimings::Phase phase)
+  JVMTI_ONLY(: _jvmti_weak_roots(phase))
+  JFR_ONLY(NOT_JVMTI(:) JVMTI_ONLY(COMMA) _jfr_weak_roots(phase))
+  {};
+
   void weak_oops_do(BoolObjectClosure* is_alive, OopClosure* keep_alive, uint worker_id);
   void weak_oops_do(OopClosure* cl, uint worker_id);
 };


Testing: builds

-- 
Thanks,
-Aleksey


From rkennke at redhat.com  Wed Jul  1 16:57:37 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Wed, 1 Jul 2020 18:57:37 +0200
Subject: RFR (XS/T) 8248634: Shenandoah: incorrect include in
 shenandoahInitLogger.cpp
In-Reply-To: <177e1e58-82b8-e4af-d12b-b8f61dd26a4c@redhat.com>
References: <177e1e58-82b8-e4af-d12b-b8f61dd26a4c@redhat.com>
Message-ID: <CAAN-KyjZ6rTD9Pfui4fBc6qM4Zr8As=jLxAFUk4TikmxQ3Og+Q@mail.gmail.com>

Ok!

Thanks, Roman


Aleksey Shipilev <shade at redhat.com> schrieb am Mi., 1. Juli 2020, 18:38:

> Bug:
>   https://bugs.openjdk.java.net/browse/JDK-8248634
>
> diff -r f42f4771b5d1
> src/hotspot/share/gc/shenandoah/shenandoahInitLogger.cpp
> --- a/src/hotspot/share/gc/shenandoah/shenandoahInitLogger.cpp  Wed Jul 01
> 15:53:23 2020 +0200
> +++ b/src/hotspot/share/gc/shenandoah/shenandoahInitLogger.cpp  Wed Jul 01
> 15:56:53 2020 +0200
> @@ -24,5 +24,5 @@
>
>  #include "precompiled.hpp"
> -#include "gc/shenandoah/shenandoahHeap.hpp"
> +#include "gc/shenandoah/shenandoahHeap.inline.hpp"
>  #include "gc/shenandoah/shenandoahHeapRegion.hpp"
>  #include "gc/shenandoah/shenandoahInitLogger.hpp"
>
> Testing: builds
>
> --
> Thanks,
> -Aleksey
>
>


From rkennke at redhat.com  Wed Jul  1 17:01:46 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Wed, 01 Jul 2020 19:01:46 +0200
Subject: RFR (XS) 8248632: Shenandoah: build fails without both JVMTI
 and JFR
In-Reply-To: <66cb5669-c1a7-af6c-065d-5640483bb30d@redhat.com>
References: <66cb5669-c1a7-af6c-065d-5640483bb30d@redhat.com>
Message-ID: <20c0000ffcf3778857f7dbd846637fe02481353a.camel@redhat.com>

Oh dear. Ok.

Thanks,
Roman

On Wed, 2020-07-01 at 15:59 +0200, Aleksey Shipilev wrote:
> 	Error verifying signature: Cannot verify message signature:
> Incorrect message format
> Bug:
>   https://bugs.openjdk.java.net/browse/JDK-8248632
> 
> We get dangling ":" when both JFR and JVMTI are disabled. The fix is
> to put the colon into the macro
> calls themselves, which would select it properly:
> 
> diff -r 7ab23692b432
> src/hotspot/share/gc/shenandoah/shenandoahRootProcessor.hpp
> ---
> a/src/hotspot/share/gc/shenandoah/shenandoahRootProcessor.hpp       W
> ed Jul 01 15:09:28 2020 +0200
> +++
> b/src/hotspot/share/gc/shenandoah/shenandoahRootProcessor.hpp       W
> ed Jul 01 15:51:30 2020 +0200
> @@ -91,13 +91,15 @@
>  class ShenandoahSerialWeakRoots {
>  private:
>    JVMTI_ONLY(ShenandoahJVMTIWeakRoot _jvmti_weak_roots;)
>    JFR_ONLY(ShenandoahJFRWeakRoot     _jfr_weak_roots;)
>  public:
> -  ShenandoahSerialWeakRoots(ShenandoahPhaseTimings::Phase phase) :
> -  JVMTI_ONLY(_jvmti_weak_roots(phase))
> -  JFR_ONLY(JVMTI_ONLY(COMMA)_jfr_weak_roots(phase)) {};
> +  ShenandoahSerialWeakRoots(ShenandoahPhaseTimings::Phase phase)
> +  JVMTI_ONLY(: _jvmti_weak_roots(phase))
> +  JFR_ONLY(NOT_JVMTI(:) JVMTI_ONLY(COMMA) _jfr_weak_roots(phase))
> +  {};
> +
>    void weak_oops_do(BoolObjectClosure* is_alive, OopClosure*
> keep_alive, uint worker_id);
>    void weak_oops_do(OopClosure* cl, uint worker_id);
>  };
> 
> 
> Testing: builds
> 



From zgu at redhat.com  Wed Jul  1 17:08:38 2020
From: zgu at redhat.com (Zhengyu Gu)
Date: Wed, 1 Jul 2020 13:08:38 -0400
Subject: RFR (XS/T) 8248634: Shenandoah: incorrect include in
 shenandoahInitLogger.cpp
In-Reply-To: <177e1e58-82b8-e4af-d12b-b8f61dd26a4c@redhat.com>
References: <177e1e58-82b8-e4af-d12b-b8f61dd26a4c@redhat.com>
Message-ID: <bd86acaf-a636-36ca-b4ea-1b7b82c50886@redhat.com>

Looks good.

-Zhengyu

On 7/1/20 9:59 AM, Aleksey Shipilev wrote:
> Bug:
>    https://bugs.openjdk.java.net/browse/JDK-8248634
> 
> diff -r f42f4771b5d1 src/hotspot/share/gc/shenandoah/shenandoahInitLogger.cpp
> --- a/src/hotspot/share/gc/shenandoah/shenandoahInitLogger.cpp  Wed Jul 01 15:53:23 2020 +0200
> +++ b/src/hotspot/share/gc/shenandoah/shenandoahInitLogger.cpp  Wed Jul 01 15:56:53 2020 +0200
> @@ -24,5 +24,5 @@
> 
>   #include "precompiled.hpp"
> -#include "gc/shenandoah/shenandoahHeap.hpp"
> +#include "gc/shenandoah/shenandoahHeap.inline.hpp"
>   #include "gc/shenandoah/shenandoahHeapRegion.hpp"
>   #include "gc/shenandoah/shenandoahInitLogger.hpp"
> 
> Testing: builds
> 



From zgu at redhat.com  Wed Jul  1 17:08:25 2020
From: zgu at redhat.com (Zhengyu Gu)
Date: Wed, 1 Jul 2020 13:08:25 -0400
Subject: RFR (XS) 8248632: Shenandoah: build fails without both JVMTI and
 JFR
In-Reply-To: <66cb5669-c1a7-af6c-065d-5640483bb30d@redhat.com>
References: <66cb5669-c1a7-af6c-065d-5640483bb30d@redhat.com>
Message-ID: <4c313084-39f6-3832-bb41-d04804cce752@redhat.com>

Okay.

-Zhengyu

On 7/1/20 9:59 AM, Aleksey Shipilev wrote:
> Bug:
>    https://bugs.openjdk.java.net/browse/JDK-8248632
> 
> We get dangling ":" when both JFR and JVMTI are disabled. The fix is to put the colon into the macro
> calls themselves, which would select it properly:
> 
> diff -r 7ab23692b432 src/hotspot/share/gc/shenandoah/shenandoahRootProcessor.hpp
> --- a/src/hotspot/share/gc/shenandoah/shenandoahRootProcessor.hpp       Wed Jul 01 15:09:28 2020 +0200
> +++ b/src/hotspot/share/gc/shenandoah/shenandoahRootProcessor.hpp       Wed Jul 01 15:51:30 2020 +0200
> @@ -91,13 +91,15 @@
>   class ShenandoahSerialWeakRoots {
>   private:
>     JVMTI_ONLY(ShenandoahJVMTIWeakRoot _jvmti_weak_roots;)
>     JFR_ONLY(ShenandoahJFRWeakRoot     _jfr_weak_roots;)
>   public:
> -  ShenandoahSerialWeakRoots(ShenandoahPhaseTimings::Phase phase) :
> -  JVMTI_ONLY(_jvmti_weak_roots(phase))
> -  JFR_ONLY(JVMTI_ONLY(COMMA)_jfr_weak_roots(phase)) {};
> +  ShenandoahSerialWeakRoots(ShenandoahPhaseTimings::Phase phase)
> +  JVMTI_ONLY(: _jvmti_weak_roots(phase))
> +  JFR_ONLY(NOT_JVMTI(:) JVMTI_ONLY(COMMA) _jfr_weak_roots(phase))
> +  {};
> +
>     void weak_oops_do(BoolObjectClosure* is_alive, OopClosure* keep_alive, uint worker_id);
>     void weak_oops_do(OopClosure* cl, uint worker_id);
>   };
> 
> 
> Testing: builds
> 



From shade at redhat.com  Wed Jul  1 17:56:51 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Wed, 1 Jul 2020 19:56:51 +0200
Subject: RFR (S) 8248652: Shenandoah: SATB buffer handling may assume no
 forwarded objects
Message-ID: <046b7175-361c-e823-06c5-90d053d4c47d@redhat.com>

RFE:
  https://bugs.openjdk.java.net/browse/JDK-8248652

Since CM-with-UR is gone, SATB may assume no forwarded objects are ever exposed through it. The only
way marking code can experience forwarded objects is due to Full GC marking. In that case, SATB
should be inactive.

Fix:
  https://cr.openjdk.java.net/~shade/8248652/webrev.01/

Testing: hotspot_gc_shenandoah

-- 
Thanks,
-Aleksey


From thomas.schatzl at oracle.com  Wed Jul  1 19:28:42 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Wed, 1 Jul 2020 19:28:42 +0000 (UTC)
Subject: RFR: 8248650: [BACKOUT] Backout JDK-8244603 because it generates too
 much noise in CI
Message-ID: <b8303ae1-deb2-429e-b3a0-3d10297ae2d0@default>

Hi all,

  can I have quick reviews for this backout of JDK-8244603?

Since the checking of these changes that make young gen sizing quite different than before, there are a lot of crashes in tier7/tier8 testing. Dan Daugherty traced this back to the push of JDK-8244603.

At this time I do not think the change is directly the cause for this issue but rather exposing a latent bug; however to reduce the noise I would like to backout the change, and later reapply after fixing this issue.

The backout applied without issues if the build fix that had to be applied to it (JDK-8248329) is considered, i.e. this change is actually backing out 8248329 and 8244603.

CR:
https://bugs.openjdk.java.net/browse/JDK-8248650
Webrev:
http://cr.openjdk.java.net/~tschatzl/8248650/webrev/
Original change:
https://hg.openjdk.java.net/jdk/jdk/rev/38ecd000c722
Testing:
tier1

Thanks,
  Thomas


From igor.ignatyev at oracle.com  Thu Jul  2 01:17:09 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Wed, 1 Jul 2020 18:17:09 -0700
Subject: RFR(S) [15] : 8208207 : Test nsk/stress/jni/gclocker/gcl001 fails
 after co-location
Message-ID: <9C92F73D-9320-4532-A819-9D162675712D@oracle.com>

?Hi Thomas,

please find the answers to your questions inlined below. 

I've made the cleanup you asked and some other changes to make the code, hopefully, nicer. I've also removed zeroing of hash in native part and updated java accordingly, so the test would fail if 1st native loop sees some other data.
 
 - incremental webrev: http://cr.openjdk.java.net/~iignatyev//8208207/webrev.0-1
 - full webrev: http://cr.openjdk.java.net/~iignatyev//8208207/webrev.0

Thanks,
-- Igor

>> On Jun 30, 2020, at 2:50 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
>> 
>> Hi,
>> 
>>> On 30.06.20 05:23, Igor Ignatyev wrote:
>>> http://cr.openjdk.java.net/~iignatyev//8208207/webrev.00
>>>> 38 lines changed: 1 ins; 4 del; 33 mod;
>>> Hi all,
>>> could you please review the small patch which fixes gcl001 test and returns it back to work?
>>> the issue reported in the bug (assert(!JavaThread::current()->in_critical()) failed: Would deadlock) was caused by calls to JNI functions other than (Get|Release).*Critical within a critical region. after this get fixed by moving Get.*Length calls outside of critical regions, the test started to fail w/ "Data validation failure" message. this was b/c of returning 0 and w/o sorting arrays in case isCopy was changed to TRUE by Get.*Critical. as there is no way to guarantee that we won't get a copy of array, I decided to remove checks of isCopy which, although might slightly change that this test check, makes the test more robust.
>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8208207
>>> webrev: http://cr.openjdk.java.net/~iignatyev//8208207/webrev.00
>>> testing:
>>>  - run the test against {linux,windows,macosx-x64}-{product,fastdebug} w/ default GC
>>>  - run the test against macosx-x64-slowdebug w/ Serial,Parallel,G1,ZGC
>> 
>> - I would prefer if the test removed the debug code, i.e. the native EnterCS/ReleaseCS methods and associated data structures. At least instead of commenting out code to disable it, add an ifdef.
> at 1st, I removed EnterCS/LeaveCS methods and two vars, but then restored them back just to keep the change as small as possible, a part of me was still disgusted by that, yet given the code around it was hard to pinpoint the reason for that :)  anyhow, I've cleaned that up.
> 
> - could the backslashes in the macro be lined up?
sure.
> Otherwise the code is even uglier than it already is :P
is it even possible ;) ?
> 
> - unless it is intentional, the code would be easier to read if the GetxxxCritical and ReleasexxxCritical were scoped better, i.e. now it is like:
> 
> GetPrimitiveArrayCritical
> GetStringCritical
> ReleaseStringCritical
> GetStringCritical
> ReleasePrimitiveArrayCritical <--- this one
> ReleaseStringCritical
> 
> and the question is why the ReleasePrimitiveArrayCritical is between the second get/releaseStringCritical and not the last call.
I'm not an original author of this test, but the current order does seem intentional, so I'd prefer not to change that.


> 
> Thanks,
>  Thomas



From kim.barrett at oracle.com  Thu Jul  2 06:26:39 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 2 Jul 2020 02:26:39 -0400
Subject: RFR: 8248650: [BACKOUT] Backout JDK-8244603 because it generates
 too much noise in CI
In-Reply-To: <b8303ae1-deb2-429e-b3a0-3d10297ae2d0@default>
References: <b8303ae1-deb2-429e-b3a0-3d10297ae2d0@default>
Message-ID: <0550A70C-D67C-4A4E-9B7F-0F306B869E2B@oracle.com>

> On Jul 1, 2020, at 3:28 PM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> 
> Hi all,
> 
>  can I have quick reviews for this backout of JDK-8244603?
> 
> Since the checking of these changes that make young gen sizing quite different than before, there are a lot of crashes in tier7/tier8 testing. Dan Daugherty traced this back to the push of JDK-8244603.
> 
> At this time I do not think the change is directly the cause for this issue but rather exposing a latent bug; however to reduce the noise I would like to backout the change, and later reapply after fixing this issue.
> 
> The backout applied without issues if the build fix that had to be applied to it (JDK-8248329) is considered, i.e. this change is actually backing out 8248329 and 8244603.
> 
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8248650
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8248650/webrev/
> Original change:
> https://hg.openjdk.java.net/jdk/jdk/rev/38ecd000c722
> Testing:
> tier1
> 
> Thanks,
>  Thomas

Looks good.



From thomas.schatzl at oracle.com  Thu Jul  2 07:42:17 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 2 Jul 2020 09:42:17 +0200
Subject: RFR: 8248650: [BACKOUT] Backout JDK-8244603 because it generates
 too much noise in CI
In-Reply-To: <0550A70C-D67C-4A4E-9B7F-0F306B869E2B@oracle.com>
References: <b8303ae1-deb2-429e-b3a0-3d10297ae2d0@default>
 <0550A70C-D67C-4A4E-9B7F-0F306B869E2B@oracle.com>
Message-ID: <5091789c-2860-ab37-417b-efdc480afb4d@oracle.com>

Hi Kim,

On 02.07.20 08:26, Kim Barrett wrote:
>> On Jul 1, 2020, at 3:28 PM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
>>
>> Hi all,
>>
>>   can I have quick reviews for this backout of JDK-8244603?
>>
[...]
> 
> Looks good.
> 

Thanks for your review. Pushed.

Thomas


From thomas.schatzl at oracle.com  Thu Jul  2 11:27:29 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 2 Jul 2020 13:27:29 +0200
Subject: RFR(S) [15] : 8208207 : Test nsk/stress/jni/gclocker/gcl001 fails
 after co-location
In-Reply-To: <9C92F73D-9320-4532-A819-9D162675712D@oracle.com>
References: <9C92F73D-9320-4532-A819-9D162675712D@oracle.com>
Message-ID: <5e70a76f-e425-2939-f988-45504411d97d@oracle.com>

Hi Igor,

On 02.07.20 03:17, Igor Ignatyev wrote:
> ?Hi Thomas,
> 
> please find the answers to your questions inlined below.
> 
> I've made the cleanup you asked and some other changes to make the code, 
> hopefully, nicer. I've also removed zeroing of hash in native part and 
> updated java accordingly, so the test would fail if 1st native loop sees 
> some other data.
>  ?- incremental webrev: 
> http://cr.openjdk.java.net/~iignatyev//8208207/webrev.0-1
>  ?- full webrev: http://cr.openjdk.java.net/~iignatyev//8208207/webrev.0
> 

Okay. Thanks for the cleanups. Ship it.

Thomas


From igor.ignatyev at oracle.com  Thu Jul  2 15:06:30 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Thu, 2 Jul 2020 08:06:30 -0700
Subject: RFR(S) [15] : 8208207 : Test nsk/stress/jni/gclocker/gcl001 fails
 after co-location
In-Reply-To: <5e70a76f-e425-2939-f988-45504411d97d@oracle.com>
References: <9C92F73D-9320-4532-A819-9D162675712D@oracle.com>
 <5e70a76f-e425-2939-f988-45504411d97d@oracle.com>
Message-ID: <F77B1DE5-9BFD-415A-9EFF-1C81EB99CA30@oracle.com>

Thanks Thomas, pushed to jdk/jdk15.

-- Igor

> On Jul 2, 2020, at 4:27 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> 
> Hi Igor,
> 
> On 02.07.20 03:17, Igor Ignatyev wrote:
>> ?Hi Thomas,
>> please find the answers to your questions inlined below.
>> I've made the cleanup you asked and some other changes to make the code, hopefully, nicer. I've also removed zeroing of hash in native part and updated java accordingly, so the test would fail if 1st native loop sees some other data.
>>  - incremental webrev: http://cr.openjdk.java.net/~iignatyev//8208207/webrev.0-1
>>  - full webrev: http://cr.openjdk.java.net/~iignatyev//8208207/webrev.0
> 
> Okay. Thanks for the cleanups. Ship it.
> 
> Thomas



From igor.ignatyev at oracle.com  Thu Jul  2 18:02:06 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Thu, 2 Jul 2020 11:02:06 -0700
Subject: RFR(S) [15] : 8208243 :
 vmTestbase/gc/lock/jni/jnilock002/TestDescription.java fails in jdk/hs
 nightly
Message-ID: <D20BCA8E-597A-43B8-864C-B0B9A47C6F99@oracle.com>

http://cr.openjdk.java.net/~iignatyev//8208243/webrev.00
> 325 lines changed: 18 ins; 0 del; 307 mod;

Hi all,

could you please review the patch for nsk.share.gc.lock.jni.*CriticalLocker.cpp files which:
 - uses difftime to compare current_time and start_time;
 - calls FatalError if Get*Critical returns NULL so the test would fail w/ a more obvious error than 'Native hash: 0 != Java hash:...' and won't hide this problem in cases when Java hash happens to be 0 (which is currently always true for all but float/double arrays);
 - resets initial hash value in each iteration of while-loop;
 - aligns whitespace w/ usual hotspot code style.

testing: vmTestbase/gc/lock/jni on {linux,windows,macosx}-x64-{product,fastedebug} (w/ jnilock002 being removed from the problem-list)
JBS: https://bugs.openjdk.java.net/browse/JDK-8208243
webrev: http://cr.openjdk.java.net/~iignatyev//8208243/webrev.00

Thanks,
-- Igor

From leo.korinth at oracle.com  Thu Jul  2 21:43:12 2020
From: leo.korinth at oracle.com (Leo Korinth)
Date: Thu, 2 Jul 2020 23:43:12 +0200
Subject: RFR (M): 8210462: Fix remaining mentions of initial mark
In-Reply-To: <bc347a79-63d4-e259-3e57-53ddb4a00be1@oracle.com>
References: <bc347a79-63d4-e259-3e57-53ddb4a00be1@oracle.com>
Message-ID: <6f950a83-6942-b4e9-fb49-5ee20635add6@oracle.com>



On 30/06/2020 16:21, Thomas Schatzl wrote:
> Hi all,
> 
>  ? can I have reviews for this change that removes remaining use of the 
> "initial mark pause" to the "newer" concurrent start in the code?
> 
> This is mostly a straightforward s/initial mark/concurrent start/ 
> replacement.

Changes looks mostly good to me.

in: src/hotspot/share/gc/g1/g1Policy.cpp
After initializing the variable "this_pause", maybe use it instead of young_gc_pause_kind() two statements below?

in: src/hotspot/share/gc/g1/g1Policy.hpp
#include "gc/g1/g1HeapRegionAttr.hpp"
#include "gc/g1/g1ConcurrentStartToMixedTimeTracker.hpp"
#include "gc/g1/g1MMUTracker.hpp"
(no longer sorted after renaming)

in: src/hotspot/share/gc/g1/g1InitialMarkToMixedTimeTracker.hpp
#include "utilities/globalDefinitions.hpp"
#include "utilities/debug.hpp"
(not sorted, pre-existing)

Some grepping (egrep -Ri "initial.?mark") shows more potential changes in:
open/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shared/GCCause.java:  _cms_initial_mark ("CMS Initial Mark"),
(oops, my fault, I should have removed it with CMS)

open/src/hotspot/share/gc/g1/g1CollectedHeap.inline.hpp:    res |= G1EvacuationFailureALotDuringInitialMark;
open/src/hotspot/share/gc/g1/g1_globals.hpp:  develop(bool, G1EvacuationFailureALotDuringInitialMark, true,             \
(maybe not rename the flag, unsure about the amount of work when it is a developer flag, just want to mention it)

open/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shared/G1YCType.java:  InitialMark ("Initial Mark"),
open/src/hotspot/share/gc/g1/g1Policy.cpp:    assert(!is_concurrent_start_pause(this_pause), "The young GC before mixed is not allowed to be an initial mark GC");
open/src/hotspot/share/gc/g1/g1CollectedHeap.cpp:    // Try to schedule an initial-mark evacuation pause that will
open/src/hotspot/share/gc/g1/g1CollectedHeap.cpp:      // collection an initial-mark won't be honored.  If we don't check for
open/src/hotspot/share/gc/g1/g1CollectedHeap.cpp:  // We do not allow initial-mark to be piggy-backed on a mixed GC.
open/test/hotspot/gtest/gc/shared/test_preservedMarks.cpp:  // Make sure initial marks are correct.
open/test/hotspot/jtreg/gc/g1/TestGCLogMessages.java:        new TestGCLogMessages().testWithInitialMark();
open/test/hotspot/jtreg/gc/g1/TestGCLogMessages.java:    private void testWithInitialMark() throws Exception {
open/test/hotspot/jtreg/gc/g1/TestGCLogMessages.java:                                                                  GCTestWithInitialMark.class.getName());
open/test/hotspot/jtreg/gc/g1/TestGCLogMessages.java:    static class GCTestWithInitialMark {
open/test/hotspot/jtreg/gc/g1/TestHumongousAllocInitialMark.java: * @test TestHumongousAllocInitialMark
open/test/hotspot/jtreg/gc/g1/TestHumongousAllocInitialMark.java: * @run driver gc.g1.TestHumongousAllocInitialMark
open/test/hotspot/jtreg/gc/g1/TestHumongousAllocInitialMark.java:public class TestHumongousAllocInitialMark {

Thanks,
Leo



> CR:
> https://bugs.openjdk.java.net/browse/JDK-8210462
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8210462/webrev/
> Testing:
> tier1-5
> 
> Thanks,
>  ? Thomas


From maoliang.ml at alibaba-inc.com  Fri Jul  3 08:35:02 2020
From: maoliang.ml at alibaba-inc.com (Liang Mao)
Date: Fri, 03 Jul 2020 16:35:02 +0800
Subject: =?UTF-8?B?UkZSIChTKSA4MjQwNTU2IGFuZCBSRlIgKFMpIDgyNDg3ODMgZm9yIEcxIGh1bW9uZ291cyBv?=
 =?UTF-8?B?YmplY3Rz?=
Message-ID: <84599c56-0ba3-4b88-8ffc-ee2cfa70a99f.maoliang.ml@alibaba-inc.com>

Hi Man and G1 team,

Previously we were discussing about abort of concurrent mark(JDK-8240556). I 
made a concurrent cleaning bitmap version instead of the original STW one.
 But there are still some other problems in real world. Recently I revisited
 the implementation and make more optimization and the result seems very good.

The most severe problems in our work loads are as below:
1) Humongous objects allocation will invade the space of young generation
and lead to long time STW because of to-space exhausted and even Full GC
2) Frequent concurrent marking will cost significant CPU resources (which
could be resolved by abort concurrent mark JDK-8240556)
3) Frequent GCs because the humongous allocation can reach IHOP very quickly
even no matter w or w/o abort concurrent mark.

Both 1) and 3) didn't have solutions yet. I made another change to share the 
young space for humongous allocation and eden allocation. Initial-mark will be
triggered if reserve space is invaded. My colleague helped to create a new bug
ID: https://bugs.openjdk.java.net/browse/JDK-8248783
With this change and abort concurrent mark our problematic work loads run
 very smoothly without any GC issues. I have tested the 2 changes in several
 different applications, like web services, database, etc.

Our real test is done with 8u and I have created JDK15 webrevs with the same
logic. The code completes the test of jtreg gc/g1 and jbb2015. Could you please
 take a look and run some tests?

Bug: https://bugs.openjdk.java.net/browse/JDK-8240556
Webrev: http://cr.openjdk.java.net/~ddong/liangmao/8240556/webrev.00/
(This may related to https://bugs.openjdk.java.net/browse/JDK-8247928)

Bug: https://bugs.openjdk.java.net/browse/JDK-8248783
Webrev: http://cr.openjdk.java.net/~ddong/liangmao/8248783/webrev.00/


Thanks,
Liang






------------------------------------------------------------------
From:MAO, Liang <maoliang.ml at alibaba-inc.com>
Send Time:2020 Mar. 6 (Fri.) 19:35
To:Stefan Johansson <stefan.johansson at oracle.com>; Thomas Schatzl <thomas.schatzl at oracle.com>; Man Cao <manc at google.com>; hotspot-gc-dev <hotspot-gc-dev at openjdk.java.net>
Subject:RFR (S): 8240556: Abort concurrent mark after effective eager reclamation of humongous objects

Hi,

Thanks for Man's accurate comments and I made the change
http://cr.openjdk.java.net/~luchsh/g1hum/humongous.webrev.1/

Stefan's concern is fairly reasonable since I have noticed if GC 
workers are not enough, the addition pause time caused by clearing 
could be considerable. concurrent_cycle_abort might not be easily to
 reuse because it still clears the bitmap in pause. I was thinking to let
 the concurrent mark thread continue and finish the last step of
 "_cm->cleanup_for_next_mark()" although it has chance to delay the 
next initial mark. Anyway I'm glad to make a try and you guys can compare
two approaches and provide comments.

Thanks,
Liang





------------------------------------------------------------------
From:Stefan Johansson <stefan.johansson at oracle.com>
Send Time:2020 Mar. 6 (Fri.) 18:59
To:"MAO, Liang" <maoliang.ml at alibaba-inc.com>; Thomas Schatzl <thomas.schatzl at oracle.com>; Man Cao <manc at google.com>; hotspot-gc-dev <hotspot-gc-dev at openjdk.java.net>
Subject:Re: RFR (S): 8240556: Abort concurrent mark after effective eager reclamation of humongous objects

Hi Liang,

Thanks for picking this up, really nice to see it progressing.

It would be nice if we could make the clearing concurrently to avoid 
prolonging the pause. An alternative to abort like you do now, would be 
to let the concurrent cycle start, but have it abort it self directly. 
This should be done by calling:
G1ConcurrentMark::concurrent_cycle_abort()

This would also reuse the abort mechanism already in place and if 
aborting needs updating in the future there is only one place to change. 
There might be some things that have to be altered to get this to work 
and I haven't explored this more than in theory. Would you consider 
trying this out?

I'm thinking this should look something like this in the log:
GC(1) Pause Young (Concurrent Start) (G1 Evacuation Pause) 
261M->262M(502M) 50.153ms
GC(2) Concurrent Cycle
GC(2) Concurrent Mark Abort
GC(2) Concurrent Cycle 12.345ms

We might want to call it something other than "Abort" in the logs to 
differ it from an abort by a Full GC, but we can discuss the details 
later on.

Thanks,
Stefan

On 2020-03-05 08:13, Liang Mao wrote:
> Hi All,
> 
> Now we have the bug id. I did more test to the patch. There's
> a little concern in the patch that when we decide to cancle
> the concurrent cycle in initial mark pause we need to clear
> the next bitmap which supposes to be cleared concurrently.
> In my test with -Xmx20g -Xms20g -XX:ParallelGCThreads=10,
> the time spent on clearing next bitmap was consistently less
> than 10ms. So I guess it could be acceptable.
> 
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8240556
> Webrev:
> http://cr.openjdk.java.net/~luchsh/g1hum/humongous.webrev/
> 
> Thanks,
> Liang
> 
> 
> 
> 
> 
> ------------------------------------------------------------------
> From:MAO, Liang <maoliang.ml at alibaba-inc.com>
> Send Time:2020 Mar. 3 (Tue.) 19:14
> To:Thomas Schatzl <thomas.schatzl at oracle.com>; Man Cao <manc at google.com>; hotspot-gc-dev <hotspot-gc-dev at openjdk.java.net>
> Subject:G1: Abort concurrent at initial mark pause
> 
> Hi All,
> 
> As previous discusion, there're several ideas to improve the humongous
> objects handling. We've made some experiments that canceling concurrent
>   mark at initial mark pause is proved to be effective in the senario that
> frequent temporary humongous objects allocation leads to frequent concurrent
>   mark and high CPU usage. The sub-test: scimark.fft.large in specjvm2008 is
> also the exact case but not GC sensative so there's little difference
> in score.
> 
> The patch is small and shall we have a bug id for it?
> http://cr.openjdk.java.net/~luchsh/g1hum/humongous.webrev/
> 
> Thanks,
> Liang
> 
> 
> 
> 
> 



From thomas.schatzl at oracle.com  Fri Jul  3 18:06:33 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 3 Jul 2020 20:06:33 +0200
Subject: RFR (M): 8210462: Fix remaining mentions of initial mark
In-Reply-To: <6f950a83-6942-b4e9-fb49-5ee20635add6@oracle.com>
References: <bc347a79-63d4-e259-3e57-53ddb4a00be1@oracle.com>
 <6f950a83-6942-b4e9-fb49-5ee20635add6@oracle.com>
Message-ID: <a5411b45-4a50-9a0d-ef7d-7dc45d603d22@oracle.com>

Hi Leo,

   thanks for your review.

On 02.07.20 23:43, Leo Korinth wrote:
> 
> in: src/hotspot/share/gc/g1/g1Policy.cpp
> After initializing the variable "this_pause", maybe use it instead of 
> young_gc_pause_kind() two statements below?
> 
> in: src/hotspot/share/gc/g1/g1Policy.hpp
> #include "gc/g1/g1HeapRegionAttr.hpp"
> #include "gc/g1/g1ConcurrentStartToMixedTimeTracker.hpp"
> #include "gc/g1/g1MMUTracker.hpp"
> (no longer sorted after renaming)
> 
> in: src/hotspot/share/gc/g1/g1InitialMarkToMixedTimeTracker.hpp
> #include "utilities/globalDefinitions.hpp"
> #include "utilities/debug.hpp"
> (not sorted, pre-existing)
> 

All fixed.

> Some grepping (egrep -Ri "initial.?mark") shows more potential changes in:

Your query string has obviously been better than mine...

> open/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shared/GCCause.java:  
> _cms_initial_mark ("CMS Initial Mark"),
> (oops, my fault, I should have removed it with CMS)
> 

Let's do this separately.

> open/src/hotspot/share/gc/g1/g1CollectedHeap.inline.hpp:??? res |= 
> G1EvacuationFailureALotDuringInitialMark;
> open/src/hotspot/share/gc/g1/g1_globals.hpp:? develop(bool, 
> G1EvacuationFailureALotDuringInitialMark, true,???????????? \
> (maybe not rename the flag, unsure about the amount of work when it is a 
> developer flag, just want to mention it)
> 

Renaming is easy because it is a development flag. Done.

> open/src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shared/G1YCType.java:  
> InitialMark ("Initial Mark"),
> open/src/hotspot/share/gc/g1/g1Policy.cpp: 
> assert(!is_concurrent_start_pause(this_pause), "The young GC before 
> mixed is not allowed to be an initial mark GC");
> open/src/hotspot/share/gc/g1/g1CollectedHeap.cpp:??? // Try to schedule 
> an initial-mark evacuation pause that will
> open/src/hotspot/share/gc/g1/g1CollectedHeap.cpp:????? // collection an 
> initial-mark won't be honored.? If we don't check for
> open/src/hotspot/share/gc/g1/g1CollectedHeap.cpp:? // We do not allow 
> initial-mark to be piggy-backed on a mixed GC.

Fixed.

> open/test/hotspot/gtest/gc/shared/test_preservedMarks.cpp:? // Make sure 
> initial marks are correct.

This is correct, and I kept it.

> open/test/hotspot/jtreg/gc/g1/TestGCLogMessages.java:??????? new 
> TestGCLogMessages().testWithInitialMark();
> open/test/hotspot/jtreg/gc/g1/TestGCLogMessages.java:??? private void 
> testWithInitialMark() throws Exception {
> open/test/hotspot/jtreg/gc/g1/TestGCLogMessages.java: 
> GCTestWithInitialMark.class.getName());
> open/test/hotspot/jtreg/gc/g1/TestGCLogMessages.java:??? static class 
> GCTestWithInitialMark {
> open/test/hotspot/jtreg/gc/g1/TestHumongousAllocInitialMark.java: * 
> @test TestHumongousAllocInitialMark
> open/test/hotspot/jtreg/gc/g1/TestHumongousAllocInitialMark.java: * @run 
> driver gc.g1.TestHumongousAllocInitialMark
> open/test/hotspot/jtreg/gc/g1/TestHumongousAllocInitialMark.java:public 
> class TestHumongousAllocInitialMark {

Done.

Webrevs:

http://cr.openjdk.java.net/~tschatzl/8210462/webrev.0_to_1/ (diff)
http://cr.openjdk.java.net/~tschatzl/8210462/webrev.1/ (full)

Thanks,
   Thomas


From kim.barrett at oracle.com  Fri Jul  3 20:29:45 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Fri, 3 Jul 2020 16:29:45 -0400
Subject: RFR (M): 8210462: Fix remaining mentions of initial mark
In-Reply-To: <a5411b45-4a50-9a0d-ef7d-7dc45d603d22@oracle.com>
References: <bc347a79-63d4-e259-3e57-53ddb4a00be1@oracle.com>
 <6f950a83-6942-b4e9-fb49-5ee20635add6@oracle.com>
 <a5411b45-4a50-9a0d-ef7d-7dc45d603d22@oracle.com>
Message-ID: <3D90704F-D7BC-4E24-9783-76993989602C@oracle.com>

> On Jul 3, 2020, at 2:06 PM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> 
> Webrevs:
> 
> http://cr.openjdk.java.net/~tschatzl/8210462/webrev.0_to_1/ (diff)
> http://cr.openjdk.java.net/~tschatzl/8210462/webrev.1/ (full)
> 
> Thanks,
>  Thomas

------------------------------------------------------------------------------
src/hotspot/share/gc/g1/g1CollectedHeap.hpp
 537   // otherwise reachable ensure that it is marked in the bitmap for concurrent marking

[pre-existing]
There should be a sentence break or a semi-colon or something between
"reachable" and "ensure"

------------------------------------------------------------------------------
src/hotspot/share/gc/g1/g1CollectorState.hpp
  53   volatile bool _in_concurrent_mark_gc;

Shouldn't this be _in_concurrent_start_gc?

------------------------------------------------------------------------------
src/hotspot/share/gc/g1/g1OopClosures.inline.hpp
 254     // closure during a concurrent mark pause then attempt to mark the object.
 
s/concurrent mark/concurrent start/ ?

------------------------------------------------------------------------------
src/hotspot/share/gc/g1/g1Policy.cpp 
 682   } else if (!is_young_only_pause(this_pause)) {
 683     // This is a mixed GC. Here we decide whether to continue doing more

Maybe there should be an assertion that this really is a mixed pause?

Or maybe this should be testing for a mixed pause, and the else clause
should assert is_young_only_pause.

------------------------------------------------------------------------------
src/hotspot/share/gc/g1/g1Policy.cpp
[removed]
 636   bool this_pause_was_young_only = collector_state()->in_young_only_phase();

Why was this variable removed and uses replaced with
is_young_only_only(this_pause), rather than keeping the variable but
updating the initialization?

------------------------------------------------------------------------------



From manavjeet18295 at iiitd.ac.in  Sat Jul  4 11:29:48 2020
From: manavjeet18295 at iiitd.ac.in (Manavjeet Singh)
Date: Sat, 4 Jul 2020 16:59:48 +0530
Subject: Understanding G1 Garbage collector
Message-ID: <CAO+=_QhaqNeJcUu214MCxq4=xn805rJWCO4vV5sV-1=AxTGdtQ@mail.gmail.com>

Greetings!
I am currently trying to understand the source code of the G1 garbage
collector in OpenJDK14. I learned the working of G1 from
https://www.oracle.com/technetwork/tutorials/tutorials-1876574.html. I have
also figured out the source code of G1 is present in
jdk14/src/hotspot/share/gc/g1/ folder.
Can someone please suggest where to dive into the code. I am trying to
locate the code related to young generation marking, collection and
evacuation phases.
Thanks and regards
Manavjeet Singh


From rs at jelastic.com  Sun Jul  5 16:21:49 2020
From: rs at jelastic.com (Ruslan Synytsky)
Date: Sun, 5 Jul 2020 18:21:49 +0200
Subject: RFC: Adaptively resize heap at any GC/SoftMaxHeapSize for G1
In-Reply-To: <mailman.582.1592907080.17886.hotspot-gc-dev@openjdk.java.net>
References: <mailman.582.1592907080.17886.hotspot-gc-dev@openjdk.java.net>
Message-ID: <CA++bR4NByktk1KK7qOQ3QyVO=wswX6AO8oU-6D3wOjjzt6H3fA@mail.gmail.com>

>
>
> > question regarding the naming: did we agree on how this parameter should
> be
> > called? What happens when heap usage goes higher than SoftMaxHeapSize -
> > OOMError or JVM gets a little bit more memory? If JVM throws OOMError I
> > believe the right naming should be HardMaxHeapSize. Sorry in advance if I
> > missed this point in the previous conversations.
>
> SoftMaxHeapSize is not what you describe here - SoftMaxHeapSize is only
> an internal goal for heap sizing without guarantees. Hence the name
> *Soft*MaxHeapSize.
>
> See https://bugs.openjdk.java.net/browse/JDK-8222145.
>
> There has been no progress on anything like Current/HardMaxHeapSize.
>
Thomas and Liang, is there a possibility to easily add an optional
manageable parameter that regulates behaviour of JVM when memory
consumption goes above SoftMaxHeapSize? For example, by default JVM
allocates more memory when it can't keep memory usage
below SoftMaxHeapSize, and if the optional parameter was specified then JVM
throws OutOfMemoryError. In this case we will cover two cases with the same
code base.


>
> >
> > Also, some news regarding analysis automation of memory usage efficiency
> > I'm working on in the background. We built a relatively small script that
> > collects memory usage metrics from many containers running inside the
> same
> > large host machine. After executing it in one of our dev environments
> with
> > about 150 containers we got interesting results - the used heap is very
> > close to the committed heap while Xmx is much higher compared to
> committed
> > value. Please note, almost all containers use JEP 346 improvement or
> > javaagent which triggers GC at idle state in the older JDK versions.
> >
> > [image: Screenshot 2020-06-18 at 13.20.19.jpg] >
> > Zoomed
> >
> > [image: Screenshot 2020-06-18 at 14.40.18.jpg]
>
> While the screenshots have been scrubbed by the mailing list it's very
> nice to hear that you have had success with these approaches.
>
I plan to share more details on this soon with a link to dynamic charts for
a more convenient analysis.


> > However, enabling JMX ManagementAgent via jcmd and connecting to JVM
> with a
> > JMX client is a quite complex operation for getting such a simple metric
> > about heap memory usage. Also, some java processes may already
> > start ManagementAgent on a custom port with auth protection, so we can't
> > collect statistics from such java processes without contacting the
> > application owner (you can see the gaps on the chart). Do you know any
> > other way for collecting accurate heap usage statistics from a running
> java
> > process? We plan to run this analysis tool on productions with a large
> > number of containers, so we can get a more realistic picture.
> >
>
> Jcmd with the GC.heap_info command provides some information, probably
> not enough (I filed JDK-8248136) though. More information can be
> retrieved with the "VM.info" command, the detailed per-region printout
> which might be too much information.
>
> There is also JFR with its event streaming API that could be an option,
> however it is JDK14 only (https://openjdk.java.net/jeps/349).
>
> Finally, there is jstat to gather some information.
>
Unfortunately GC.heap_info and VM.info do not provide information about
COMMITTED heap. And jstat documentation
<https://docs.oracle.com/en/java/javase/11/tools/jstat.html> does not
mention options for collecting committed heap as well. Analyzing used and
max without understanding committed heap is useless in this context as the
"lost memory" is located between used and committed.

Thanks
Ruslan


From thomas.schatzl at oracle.com  Mon Jul  6 08:44:19 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 6 Jul 2020 10:44:19 +0200
Subject: Understanding G1 Garbage collector
In-Reply-To: <CAO+=_QhaqNeJcUu214MCxq4=xn805rJWCO4vV5sV-1=AxTGdtQ@mail.gmail.com>
References: <CAO+=_QhaqNeJcUu214MCxq4=xn805rJWCO4vV5sV-1=AxTGdtQ@mail.gmail.com>
Message-ID: <c44fffc5-86be-6571-5170-62445c770de3@oracle.com>

Hi,

On 04.07.20 13:29, Manavjeet Singh wrote:
> Greetings!
> I am currently trying to understand the source code of the G1 garbage
> collector in OpenJDK14. I learned the working of G1 from
> https://www.oracle.com/technetwork/tutorials/tutorials-1876574.html. I have
> also figured out the source code of G1 is present in
> jdk14/src/hotspot/share/gc/g1/ folder.
> Can someone please suggest where to dive into the code. I am trying to
> locate the code related to young generation marking, collection and
> evacuation phases.

   it's probably easiest to set a breakpoint in 
G1CollectedHeap::do_collection_pause_at_safepoint() and step through the 
(minor) collection on an example program.

The entry point for full heap collection is in 
G1CollectedHeap::do_full_collection().

Hth,
   Thomas


From thomas.schatzl at oracle.com  Mon Jul  6 09:13:49 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 6 Jul 2020 02:13:49 -0700 (PDT)
Subject: How to run specific part of the SerialGC in a new thread?
In-Reply-To: <CAC9y=qCDKAdyKd05mV+2PCmUEF1coO49FLjEnoRODWCe1YguEQ@mail.gmail.com>
References: <CAC9y=qCXmGkeHZgWwuHwk-ypMEf8iHRBkQmDqpgdHGckL1QweQ@mail.gmail.com>
 <afabcff3-8741-4c32-e4f8-1a560f92f5cc@oracle.com>
 <CAC9y=qAGjBMMsNfZzB2c=kMOLmRKfSZdWEHQT_Tkya_hMaKoZw@mail.gmail.com>
 <2b1a3443-ccbd-602d-ae4c-4cb97284b43d@oracle.com>
 <CAC9y=qCDKAdyKd05mV+2PCmUEF1coO49FLjEnoRODWCe1YguEQ@mail.gmail.com>
Message-ID: <a01035c9-333b-0be8-ca27-bdaf9b70ee84@oracle.com>

Hi,

   sorry for the late answer, I started writing but then forgot to send 
the email. :(

On 30.06.20 10:27, Ofir Gordon wrote:
> I'll try to explain my end goal:
> I simply want to "take out" the part that runs the marking phase in a 
> full collection with serial GC to run on a dedicated cpu (on a simulator).
> So, I figured the simpler way to do that would be to create a new thread 
> which gets the "follow_stack" part as its task and bind it to the 
> dedicated cpu. Any aspect of performance?or synchronization issues is 
> currently not relevant (maybe later it will be). I'm just looking for 
> the simplest?way to split the "follow_stack" execution to a seperate 
> thread, while the main thread is waiting on it.
> That's why I thought that creating this AbstractGangClass task is too 
> complicated?for my purposes.
> If there is a way to create the Task class for running only 
> "follow_stack" in a simple way I would really?like to know.

WorkGang* w = new WorkGang("My Work Gang", 1, true, false);
w->initialize();

Store w in some static somewhere in MarkSweep.

[...]

void MarkSweep::new_follow_stack() {
   class MyGangTask : public AbstractGangTask {
   public:
     MyGangTask() : AbstractGangTask("My Gang Task") { }

     void work(uint worker_id) {
      // call original method, but may as well pass marking stack
      // (contents) to process to wherever you want, waiting for
      // simulator to finish etc.
       MarkSweep::follow_stack();
     }
   } cl;

   w->run_task(&cl);
}

Untested, but incidentally just today I wrote a hotspot gtest that very 
much looks like that. There are also ~70 example implementations of 
AbstractGangTask in the code, ranging from one-liners like that to more 
complicated ones for further reference.

I belive that manually setting up a pthread, implementing the logic to 
wait for work and synchronize again, tearing everything down, is a lot 
longer than that.

> 
> Regarding the compilation failure, when I'm trying to include <thread> 
> and use it the build fails because it doesn't recognize it. I tried to 
> add to the configuration: --with-extra-cxxflags="-std=c++0x -pthread" 
> but then the "make images" command fails with the message "use of global 
> operators new and delete is not allowed in Hotspot".
> Trying to use <pthread> didn't work out as well (different errors).

As mentioned, Hotspot uses the pthread library itself, so you can use it 
yourselves if you want. Please have a look at the platform dependent 
threading shim, e.g. os_linux.cpp for Linux and the os class to use them.

 From what I can see it does:# include <pthread.h>

# include <pthread.h>

and further down in that file using pthread_* methods.

Thanks,
   Thomas


From thomas.schatzl at oracle.com  Mon Jul  6 09:11:55 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 6 Jul 2020 11:11:55 +0200
Subject: RFC: Adaptively resize heap at any GC/SoftMaxHeapSize for G1
In-Reply-To: <CA++bR4NByktk1KK7qOQ3QyVO=wswX6AO8oU-6D3wOjjzt6H3fA@mail.gmail.com>
References: <mailman.582.1592907080.17886.hotspot-gc-dev@openjdk.java.net>
 <CA++bR4NByktk1KK7qOQ3QyVO=wswX6AO8oU-6D3wOjjzt6H3fA@mail.gmail.com>
Message-ID: <9a226b7d-cddb-45b8-47f7-7b7cd22a95aa@oracle.com>

Hi,

On 05.07.20 18:21, Ruslan Synytsky wrote:
>>
>>
>>> question regarding the naming: did we agree on how this parameter should
>> be
>>> called? What happens when heap usage goes higher than SoftMaxHeapSize -
>>> OOMError or JVM gets a little bit more memory? If JVM throws OOMError I
>>> believe the right naming should be HardMaxHeapSize. Sorry in advance if I
>>> missed this point in the previous conversations.
>>
>> SoftMaxHeapSize is not what you describe here - SoftMaxHeapSize is only
>> an internal goal for heap sizing without guarantees. Hence the name
>> *Soft*MaxHeapSize.
>>
>> See https://bugs.openjdk.java.net/browse/JDK-8222145.
>>
>> There has been no progress on anything like Current/HardMaxHeapSize.
>>
> Thomas and Liang, is there a possibility to easily add an optional
> manageable parameter that regulates behaviour of JVM when memory
> consumption goes above SoftMaxHeapSize? For example, by default JVM
> allocates more memory when it can't keep memory usage
> below SoftMaxHeapSize, and if the optional parameter was specified then JVM
> throws OutOfMemoryError. In this case we will cover two cases with the same
> code base.

I think there is still the patch from Rodrigo; from my understanding 
from last time there were some issues around when you are allowed to 
change that (as Current/HardMaxHeapSize is read at "arbitrary" locations 
you need to make sure the gc has a consistent view of it), and naming: I 
am not sure but CurrentMaxHeapSize has been the favorite or so.

>>> Also, some news regarding analysis automation of memory usage efficiency
>>> I'm working on in the background. We built a relatively small script that
>>> collects memory usage metrics from many containers running inside the
>> same
>>> large host machine. After executing it in one of our dev environments
>> with
>>> about 150 containers we got interesting results - the used heap is very
>>> close to the committed heap while Xmx is much higher compared to
>> committed
>>> value. Please note, almost all containers use JEP 346 improvement or
>>> javaagent which triggers GC at idle state in the older JDK versions.
>>>
>>> [image: Screenshot 2020-06-18 at 13.20.19.jpg] >
>>> Zoomed
>>>
>>> [image: Screenshot 2020-06-18 at 14.40.18.jpg]
>>
>> While the screenshots have been scrubbed by the mailing list it's very
>> nice to hear that you have had success with these approaches.
>>
> I plan to share more details on this soon with a link to dynamic charts for
> a more convenient analysis.

Looking forward to this. :)

>>> However, enabling JMX ManagementAgent via jcmd and connecting to JVM
>> with a
>>> JMX client is a quite complex operation for getting such a simple metric
>>> about heap memory usage. Also, some java processes may already
>>> start ManagementAgent on a custom port with auth protection, so we can't
>>> collect statistics from such java processes without contacting the
>>> application owner (you can see the gaps on the chart). Do you know any
>>> other way for collecting accurate heap usage statistics from a running
>> java
>>> process? We plan to run this analysis tool on productions with a large
>>> number of containers, so we can get a more realistic picture.
>>>
>>
>> Jcmd with the GC.heap_info command provides some information, probably
>> not enough (I filed JDK-8248136) though. More information can be
>> retrieved with the "VM.info" command, the detailed per-region printout
>> which might be too much information.
>>
>> There is also JFR with its event streaming API that could be an option,
>> however it is JDK14 only (https://openjdk.java.net/jeps/349).
>>
>> Finally, there is jstat to gather some information.
>>
> Unfortunately GC.heap_info and VM.info do not provide information about
> COMMITTED heap. And jstat documentation
> <https://docs.oracle.com/en/java/javase/11/tools/jstat.html> does not
> mention options for collecting committed heap as well. Analyzing used and
> max without understanding committed heap is useless in this context as the
> "lost memory" is located between used and committed.
> 

Sorry. However, VM.info prints the whole heap map with G1, i.e. for 
every region what type it is. For uncommitted regions, it does not print 
such a line in the "Heap regions" section...(*)(**) so you could count 
the lines and compare with what would be expected.

(*) the |   0|0x0000000600000000, 0x0000000600400000, 
0x0000000600400000|100%| O|  |TAMS 0x0000000600400000, 
0x0000000600000000| Untracked lines

(**) that is not completely correct, it prints "available" regions, i.e. 
regions that the topmost layer of g1 thinks are there, but in some 
cases, this is not entirely correct. I.e. if page size is > region size, 
a region on one page may be "unavailable", but another is, but obviously 
in that case the whole page, i.e. the space for all two regions, are 
committed.

But you can reconstruct the data as the indices in that list are fixed. 
E.g. if VM.info shows you, assuming 1m region and 2m page size. When 
using small (4k) pages, this situation can't happen because region size 
is always > page size at least on x86:

|  0| ...
|  1| ...
|  3| ...
|  4| ...
|  5| ...
|  8| ...
....

then the page 0 where region 0/1 are located is committed, page 1 where 
region 2/3 are located is committed (because g1 can't uncommit half 
pages obviously), page 2 where region 4/5 is committed, page 3 where 
regions 6/7 are NOT committed because both are missing, and so on.

I hope this makes sense to you.

While a somewhat cumbersome way to find this out, this works since 
jdk8u40 (implemented in JDK-8038423) iirc.

I recently filed JDK-8248136 for improving the heap info output for G1.

Thanks,
   Thomas


From thomas.schatzl at oracle.com  Mon Jul  6 09:44:11 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 6 Jul 2020 11:44:11 +0200
Subject: How to run specific part of the SerialGC in a new thread?
In-Reply-To: <a01035c9-333b-0be8-ca27-bdaf9b70ee84@oracle.com>
References: <CAC9y=qCXmGkeHZgWwuHwk-ypMEf8iHRBkQmDqpgdHGckL1QweQ@mail.gmail.com>
 <afabcff3-8741-4c32-e4f8-1a560f92f5cc@oracle.com>
 <CAC9y=qAGjBMMsNfZzB2c=kMOLmRKfSZdWEHQT_Tkya_hMaKoZw@mail.gmail.com>
 <2b1a3443-ccbd-602d-ae4c-4cb97284b43d@oracle.com>
 <CAC9y=qCDKAdyKd05mV+2PCmUEF1coO49FLjEnoRODWCe1YguEQ@mail.gmail.com>
 <a01035c9-333b-0be8-ca27-bdaf9b70ee84@oracle.com>
Message-ID: <3338c302-bcc4-2dc2-4e20-30d1aafeb48d@oracle.com>

Hi,

On 06.07.20 11:13, Thomas Schatzl wrote:
> Hi,
> 
>  ? sorry for the late answer, I started writing but then forgot to send 
> the email. :(
> 
> On 30.06.20 10:27, Ofir Gordon wrote:
>> I'll try to explain my end goal:

[...]

> WorkGang* w = new WorkGang("My Work Gang", 1, true, false);
> w->initialize();

A w->update_active_workers(1); may be missing here I think.

Thanks,
   Thomas


From hohensee at amazon.com  Mon Jul  6 21:57:10 2020
From: hohensee at amazon.com (Hohensee, Paul)
Date: Mon, 6 Jul 2020 21:57:10 +0000
Subject: RFR(L): 8215624: add parallel heap inspection support for jmap
 histo(G1)(Internet mail)
Message-ID: <02415944-2238-4430-A7C2-3625264A5505@amazon.com>

I'd like to see this feature added. :)

The CSR looks good, as does the basic parallel inspection algorithm. Stefan's done the GC part, so I'll stick to the non-GC part (fwiw, the GC part lgtm).

I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp.

heapInspection.hpp:

_shared_miss_count (s/b _missed_count, see below) isn't a size, so it should be a uint instead of a size_t. Same with the new parallel_thread_num argument to heap_inspection() and populate_table().

Comment copy-edit:
+// Parallel heap inspection task. Parallel inspection can fail due to
+// a native OOM when allocating memory for TL-KlassInfoTable.
+// _success will be set false on an OOM, and serial inspection tried.

_shared_miss_count should be _missed_count to match the missed_count() getter, or rename missed_count() to be shared_miss_count(). Whichever way you go, the field type should match the getter result type: uint is reasonable.

heapInspection.cpp:

You might use ResourceMark twice in populate_table, separately for the parallel attempt and the serial code. If the parallel attempt fails and available memory is low, it would be good to clean up the memory used by the parallel attempt before doing the serial code.

Style nit in KlassInfoTable::merge_entry(). I'd line up the definitions of k and elt, so "k" is even with "elt". And, because it's two lines shorter, I'd replace
+  } else {
+    return false;
+  }
with
+  return false;

KlassInfoTableMergeClosure.is_success() should be just success() (i.e., no "is_" prefix) because it's a getter.

I'd reorganize the code in populate_table() to make it more clear, vis (I changed _shared_missed_count to _missed_count)
+  if (cit.allocation_failed()) {
+    // fail to allocate memory, stop parallel mode
+    Atomic::store(&_success, false);
+    return;
+  }
+  RecordInstanceClosure ric(&cit, _filter);
+  _poi->object_iterate(&ric, worker_id);
+  missed_count = ric.missed_count();
+  {
+    MutexLocker x(&_mutex);
+    merge_success = _shared_cit->merge(&cit);
+  }
+  if (merge_success) {
+    Atomic::add(&_missed_count, missed_count);
+  else {
+    Atomic::store(&_success, false);
+  }

Thanks,
Paul

?On 6/29/20, 7:20 PM, "linzang(??)" <linzang at tencent.com> wrote:

    Dear All,
            Sorry to bother again, I just want to make sure that is this change worth to be continue to work on? If decision is made to not. I think I can drop this work and stop asking for help reviewing...
            Thanks for all your help about reviewing this previously.

    BRs,
    Lin

    On 2020/5/9, 3:47 PM, "linzang(??)" <linzang at tencent.com> wrote:

        Dear All,
               May I ask your help again for review the latest change?  Thanks!

        BRs,
        Lin

        On 2020/4/28, 1:54 PM, "linzang(??)" <linzang at tencent.com> wrote:

            Hi Stefan,
              >>  - Adding Atomic::load/store.
              >>  - Removing the time measurement in the run_task. I renamed G1's function
              >>  to run_task_timed. If we need this outside of G1, we can rethink the API
              >>  at that point.
               >>  - ZGC style cleanups
               Thanks for revising the patch,  they are all good to me, and I have made a tiny change based on it:
                   http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04/
                   http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04-delta/
              it reduce the scope of mutex in ParHeapInspectTask, and delete unnecessary comments.

            BRs,
            Lin

            On 2020/4/27, 4:34 PM, "Stefan Karlsson" <stefan.karlsson at oracle.com> wrote:

                Hi Lin,

                On 2020-04-26 05:10, linzang(??) wrote:
                > Hi Stefan and Paul?
                >      I have made a new patch based on your comments and Stefan's Poc code:
                >      Webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03/
                >      Delta(based on Stefan's change:) : http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03-delta/webrev_03-delta/

                Thanks for providing a delta patch. It makes it much easier to look at,
                and more likely for reviewers to continue reviewing.

                I'm going to continue focusing on the GC parts, and leave the rest to
                others to review.

                >
                >      And Here are main changed I made and want to discuss with you:
                >      1.  changed"parallelThreadNum=" to "parallel=" for jmap -histo options.
                >      2.  Add logic to test where parallelHeapInspection is fail, in heapInspection.cpp
                >            This is because the parHeapInspectTask create thread local KlassInfoTable in it's work() method, and this may fail because of native OOM, in this case, the parallel should fail and serial heap inspection can be tried.
                >            One more thing I want discuss with you is about the member "_success" of parHeapInspectTask, when native OOM happenes, it is set to false. And since this "set" operation can be conducted in multiple threads, should it be atomic ops?  IMO, this is not necessary because "_success" can only be set to false, and there is no way to change it from back to true after the ParHeapInspectTask instance is created, so it is save to be non-atomic, do you agree with that?

                In these situations you should be using the Atomic::load/store
                primitives. We're moving toward a later C++ standard were data races are
                considered undefined behavior.

                >     3. make CollectedHeap::run_task() be an abstract virtual func, so that every subclass of collectedHeap should support it, so later implementation of new collectedHeap will not miss the "parallel" features.
                >           The problem I want to discuss with you is about epsilonHeap and SerialHeap, as they may not need parallel heap iteration, so I only make task->work(0), in case the run_task() is invoked someway in future. Another way is to left run_task()  unimplemented, which one do you think is better?

                I don't have a strong opinion about this.

                  And also please help take a look at the zHeap, as there is a class
                zTask that wrap the abstractGangTask, and the collectedHeap::run_task()
                only accept  AbstraceGangTask* as argument, so I made a delegate class
                to adapt it , please see src/hotspot/share/gc/z/zHeap.cpp.
                >
                >        There maybe other better ways to sovle the above problems, welcome for any comments, Thanks!

                I've created a few cleanups and changes on top of your latest patch:

                https://cr.openjdk.java.net/~stefank/8215624/webrev.02.delta
                https://cr.openjdk.java.net/~stefank/8215624/webrev.02

                - Adding Atomic::load/store.
                - Removing the time measurement in the run_task. I renamed G1's function
                to run_task_timed. If we need this outside of G1, we can rethink the API
                at that point.
                - ZGC style cleanups

                Thanks,
                StefanK

                >
                > BRs,
                > Lin
                >
                > On 2020/4/23, 11:08 AM, "linzang(??)" <linzang at tencent.com> wrote:
                >
                >      Thanks Paul! I agree with using "parallel", will make the update in next patch, Thanks for help update the CSR.
                >
                >      BRs,
                >      Lin
                >
                >      On 2020/4/23, 4:42 AM, "Hohensee, Paul" <hohensee at amazon.com> wrote:
                >
                >          For the interface, I'd use "parallel" instead of "parallelThreadNum". All the other options are lower case, and it's a lot easier to type "parallel". I took the liberty of updating the CSR. If you're ok with it, you might want to change variable names and such, plus of course JMap.usage.
                >
                >          Thanks,
                >          Paul
                >
                >          On 4/22/20, 2:29 AM, "serviceability-dev on behalf of linzang(??)" <serviceability-dev-bounces at openjdk.java.net on behalf of linzang at tencent.com> wrote:
                >
                >              Dear Stefan,
                >
                >                      Thanks a lot! I agree with you to decouple the heap inspection code with GC's.
                >                      I will start  from your POC code, may discuss with you later.
                >
                >
                >              BRs,
                >              Lin
                >
                >              On 2020/4/22, 5:14 PM, "Stefan Karlsson" <stefan.karlsson at oracle.com> wrote:
                >
                >                  Hi Lin,
                >
                >                  I took a look at this earlier and saw that the heap inspection code is
                >                  strongly coupled with the CollectedHeap and G1CollectedHeap. I'd prefer
                >                  if we'd abstract this away, so that the GCs only provide a "parallel
                >                  object iteration" interface, and the heap inspection code is kept elsewhere.
                >
                >                  I started experimenting with doing that, but other higher-priority (to
                >                  me) tasks have had to take precedence.
                >
                >                  I've uploaded my work-in-progress / proof-of-concept:
                >                    https://cr.openjdk.java.net/~stefank/8215624/webrev.01.delta/
                >                    https://cr.openjdk.java.net/~stefank/8215624/webrev.01/
                >
                >                  The current code doesn't handle the lifecycle (deletion) of the
                >                  ParallelObjectIterators. There's also code left unimplemented in around
                >                  CollectedHeap::run_task. However, I think this could work as a basis to
                >                  pull out the heap inspection code out of the GCs.
                >
                >                  Thanks,
                >                  StefanK
                >
                >                  On 2020-04-22 02:21, linzang(??) wrote:
                >                  > Dear all,
                >                  >       May I ask you help to review? This RFR has been there for quite a while.
                >                  >       Thanks!
                >                  >
                >                  > BRs,
                >                  > Lin
                >                  >
                >                  > > On 2020/3/16, 5:18 PM, "linzang(??)" <linzang at tencent.com> wrote:>
                >                  >
                >                  >>    Just update a new path, my preliminary measure show about 3.5x speedup of jmap histo on a nearly full 4GB G1 heap (8-core platform with parallel thread number set to 4).
                >                  >>     webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_02/
                >                  >>     bug: https://bugs.openjdk.java.net/browse/JDK-8215624
                >                  >>     CSR: https://bugs.openjdk.java.net/browse/JDK-8239290
                >                  >>     BRs,
                >                  >>       Lin
                >                  >>       > On 2020/3/2, 9:56 PM, "linzang(??)" <linzang at tencent.com> wrote:
                >                  >>       >
                >                  >>       >    Dear all,
                >                  >>       >          Let me try to ease the reviewing work by some explanation :P
                >                  >>       >          The patch's target is to speed up jmap -histo for heap iteration, from my experience it is necessary for large heap investigation. E.g in bigData scenario I have tried to conduct jmap -histo against 180GB heap, it does take quite a while.
                >                  >>       >          And if my understanding is corrent, even the jmap -histo without "live" option does heap inspection with heap lock acquired. so it is very likely to block mutator thread in allocation-sensitive scenario. I would say the faster the heap inspection does, the shorter the mutator be blocked. This is parallel iteration for jmap is necessary.
                >                  >>       >          I think the parallel heap inspection should be applied to all kind of heap. However, consider the heap layout are different for  GCs, much time is required to understand all kinds of the heap layout to make the whole change. IMO, It is not wise to have a huge patch for the whole solution at once, and it is even harder to review it. So I plan to implement it incrementally, the first patch (this one) is going to confirm the implemention detail of how jmap accept the new option, passes it to attachListener of the jvm process and then how to make the parallel inspection closure be generic enough to make it easy to extend to different heap layout. And also how to implement the heap inspection in specific gc's heap. This patch use G1's heap as the begining.
                >                  >>       >          This patch actually do several things:
                >                  >>       >          1. Add an option "parallelThreadNum=<N>" to jmap -histo, the default behavior is to set N to 0, means let's JVM decide how many threads to use for heap inspection. Set this option to 1 will disable parallel heap inspection. (more details in CSR: https://bugs.openjdk.java.net/browse/JDK-8239290)
                >                  >>       >          2. Make a change in how Jmap passing arguments, changes in http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java.udiff.html, originally it pass options as separate arguments to attachListener, this patch change to that all options be compose to a single string. So the arg_count_max in attachListener.hpp do not need to be changed, and hence avoid the compatibility issue, as disscussed at https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-March/027334.html
                >                  >>       >         3. Add an abstract class ParHeapInspectTask in heapInspection.hpp / heapInspection.cpp, It's work(uint worker_id) method prepares the data structure (KlassInfoTable) need for every parallel worker thread, and then call do_object_iterate_parallel() which is heap specific implementation. I also added some machenism in KlassInfoTable to support parallel iteration, such as merge().
                >                  >>       >        4. In specific heap (G1 in this patch), create a subclass of ParHeapInspectTask, implement the do_object_iterate_parallel() for parallel heap inspection. For G1, it simply invoke g1CollectedHeap's object_iterate_parallel().
                >                  >>       >        5. Add related test.
                >                  >>       >        6. it may be easy to extend this patch for other kinds of heap by creating subclass of ParHeapInspectTask and implement the do_object_iterate_parallel().
                >                  >>       >
                >                  >>       >    Hope these info could help on code review and initate the discussion :-)
                >                  >>       >    Thanks!
                >                  >>       >
                >                  >>       >    BRs,
                >                  >>       >    Lin
                >                  >>       >    >On 2020/2/19, 9:40 AM, "linzang(??)" <linzang at tencent.com> wrote:.
                >                  >>       >    >
                >                  >>       >    >  Re-post this RFR with correct enhancement number to make it trackable.
                >                  >>       >    >  please ignore the previous wrong post. sorry for troubles.
                >                  >>       >    >
                >                  >>       >    >   webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/
                >                  >>       >    >    Hi bug: https://bugs.openjdk.java.net/browse/JDK-8215624
                >                  >>       >    >    CSR: https://bugs.openjdk.java.net/browse/JDK-8239290
                >                  >>       >    >    --------------
                >                  >>       >    >    Lin
                >                  >>       >    >    >Hi Lin,
                >                  >   >     >    >    >
                >                  >>       >    >    >Could you, please, re-post your RFR with the right enhancement number in
                >                  >>       >    >    >the message subject?
                >                  >>       >    >    >It will be more trackable this way.
                >                  >>       >    >    >
                >                  >>       >    >    >Thanks,
                >                  >>       >    >    >Serguei
                >                  >>       >    >    >
                >                  >>       >    >    >
                >                  >>       >    >    >On 2/17/20 10:29 PM, linzang(??) wrote:
                >                  >>       >    >    >> Dear David,
                >                  >>       >    >    >>        Thanks a lot!
                >                  >>       >    >    >>       I have updated the refined code to http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_01/.
                >                  >>       >    >    >>        IMHO the parallel heap inspection can be extended to all kinds of heap as long as the heap layout can support parallel iteration.
                >                  >>       >    >    >>        Maybe we can firstly use this webrev to discuss how to implement it, because I am not sure my current implementation is an appropriate way to communicate with collectedHeap, then we can extend the solution to other kinds of heap.
                >                  >>       >    >    >>
                >                  >>       >    >    >> Thanks,
                >                  >>       >    >    >> --------------
                >                  >>       >    >    >> Lin
                >                  >>       >    >    >>> Hi Lin,
                >                  >>       >    >    >>>
                >                  >>       >    >    >>> Adding in hotspot-gc-dev as they need to see how this interacts with GC
                >                  >>       >    >    >>> worker threads, and whether it needs to be extended beyond G1.
                >                  >>       >    >    >>>
                >                  >>       >    >   >>> I happened to spot one nit when browsing:
                >                  >>       >    >    >>>
                >                  >>       >    >    >>> src/hotspot/share/gc/shared/collectedHeap.hpp
                >                  >>       >    >    >>>
                >                  >>       >    >    >>> +   virtual bool run_par_heap_inspect_task(KlassInfoTable* cit,
                >                  >>       >    >    >>> +                                          BoolObjectClosure* filter,
                >                  >>       >    >    >>> +                                          size_t* missed_count,
                >                  >>       >    >    >>> +                                          size_t thread_num) {
                >                  >>       >    >    >>> +     return NULL;
                >                  >>       >    >    >>>
                >                  >>       >    >    >>> s/NULL/false/
                >                  >>       >    >    >>>
                >                  >>       >    >    >>> Cheers,
                >                  >>       >    >    >>> David
                >                  >   >     >    >    >>>
                >                  >>       >    >    >>> On 18/02/2020 2:15 pm, linzang(??) wrote:
                >                  >>       >    >    >>>> Dear All,
                >                  >>       >    >    >>>>         May I ask your help to review the follow changes:
                >                  >>       >    >    >>>>         webrev:
                >                  >>       >    >    >>>> http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_00/
                >                  >>       >    >    >>>>      bug: https://bugs.openjdk.java.net/browse/JDK-8215624
                >                  >>       >    >    >>>>      related CSR: https://bugs.openjdk.java.net/browse/JDK-8239290
                >                  >>       >    >    >>>>         This patch enable parallel heap inspection of G1 for jmap histo.
                >                  >>       >    >    >>>>         my simple test shown it can speed up 2x of jmap -histo with
                >                  >>       >    >    >>>> parallelThreadNum set to 2 for heap at ~500M on 4-core platform.
                >                  >>       >    >    >>>>
                >                  >>       >    >    >>>> ------------------------------------------------------------------------
                >                  >>       >    >    >>>> BRs,
                >                  >>       >    >    >>>> Lin
                >                  >>       >    >    >> >
                >                  >>       >    >    >
                >                  >
                >                  >
                >                  >
                >
                >
                >
                >
                >
                >






From igor.ignatyev at oracle.com  Tue Jul  7 02:57:52 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Mon, 6 Jul 2020 19:57:52 -0700
Subject: RFR(S) [15] : 8208243 :
 vmTestbase/gc/lock/jni/jnilock002/TestDescription.java fails in jdk/hs
 nightly
In-Reply-To: <D20BCA8E-597A-43B8-864C-B0B9A47C6F99@oracle.com>
References: <D20BCA8E-597A-43B8-864C-B0B9A47C6F99@oracle.com>
Message-ID: <7A067017-1FE4-4BEC-ACC7-3E309528E1F3@oracle.com>

ping?
-- Igor

> On Jul 2, 2020, at 11:02 AM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
> 
> http://cr.openjdk.java.net/~iignatyev//8208243/webrev.00
>> 325 lines changed: 18 ins; 0 del; 307 mod;
> 
> Hi all,
> 
> could you please review the patch for nsk.share.gc.lock.jni.*CriticalLocker.cpp files which:
> - uses difftime to compare current_time and start_time;
> - calls FatalError if Get*Critical returns NULL so the test would fail w/ a more obvious error than 'Native hash: 0 != Java hash:...' and won't hide this problem in cases when Java hash happens to be 0 (which is currently always true for all but float/double arrays);
> - resets initial hash value in each iteration of while-loop;
> - aligns whitespace w/ usual hotspot code style.
> 
> testing: vmTestbase/gc/lock/jni on {linux,windows,macosx}-x64-{product,fastedebug} (w/ jnilock002 being removed from the problem-list)
> JBS: https://bugs.openjdk.java.net/browse/JDK-8208243
> webrev: http://cr.openjdk.java.net/~iignatyev//8208243/webrev.00
> 
> Thanks,
> -- Igor



From felix.yang at huawei.com  Tue Jul  7 03:22:20 2020
From: felix.yang at huawei.com (Yangfei (Felix))
Date: Tue, 7 Jul 2020 03:22:20 +0000
Subject: RFR(S): 8248851: CMS: Missing memory fences between free chunk
 check and klass read
Message-ID: <DA41BE1DDCA941489001C7FBD7A8820EE7E63D86@dggeml527-mbx.china.huawei.com>

Hi,

We were witnessing random JVM crash that triggers in our production environment.
We were running an aarch64 jdk8u release build with -XX:+UseConcMarkSweepGC.
We see three different crash logs as reported on the issue.
Debugging show that this caused by missing memory fences in CMS for systems with weak memory model like aarch64.

The overall procedure in CMS promotion looks like:

ConcurrentMarkSweepGeneration::par_promote {

  HeapWord* obj_ptr = ps->lab.alloc(alloc_sz);
          |---> CFLS_LAB::alloc
                          |--->FreeChunk::markNotFree
	 
  oop obj = oop(obj_ptr);
  OrderAccess::storestore();
  
  obj->set_mark(m);
  OrderAccess::storestore();
  
  // Finally, install the klass pointer (this should be volatile).
  OrderAccess::storestore();
  obj->set_klass(old->klass());

  ......

void markNotFree() {
     // Set _prev (klass) to null before (if) clearing the mark word below
     _prev = NULL;
#ifdef _LP64
     if (UseCompressedOops) {
       OrderAccess::storestore();
       set_mark(markOopDesc::prototype());
     }
#endif
     assert(!is_free(), "Error");
}

From the first crash log on the issue, the crash site was in CompactibleFreeListSpace::block_size.
We found that it's possible on aarch64 (Reference [1]) that the klass load may be scheduled before the free chunk check in CompactibleFreeListSpace::block_size().
Then we may have an invalid non-null klass, which leads to the crash.  Same issue exists in CompactibleFreeListSpace::block_is_obj(), which leads to the other two crashes.

Webrev for jdk8u-dev: http://cr.openjdk.java.net/~fyang/8248851/webrev.00/ 
JTreg tested on x86_64-linux-gnu &aarch64-linux-gnu with jdk8u release builds.
Comments?

Thanks,
Felix

[1] Reference: armv8 architecture reference manual K11.6.1
This restriction applies only when the data value returned by a read is used as a data value to calculate the
address of a subsequent read or write. It does not apply if the data value returned by a read determines the
condition flags values, and the values of the flags are used for condition code evaluation to determine the
address of a subsequent read, either through conditional execution or the evaluation of a branch. This is called
a control dependency.

From thomas.schatzl at oracle.com  Tue Jul  7 08:39:27 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 7 Jul 2020 10:39:27 +0200
Subject: RFR (M): 8210462: Fix remaining mentions of initial mark
In-Reply-To: <3D90704F-D7BC-4E24-9783-76993989602C@oracle.com>
References: <bc347a79-63d4-e259-3e57-53ddb4a00be1@oracle.com>
 <6f950a83-6942-b4e9-fb49-5ee20635add6@oracle.com>
 <a5411b45-4a50-9a0d-ef7d-7dc45d603d22@oracle.com>
 <3D90704F-D7BC-4E24-9783-76993989602C@oracle.com>
Message-ID: <8edfe69d-822d-59c7-5130-5ab0b2fc2d1c@oracle.com>

Hi Kim,

   thanks for your review.

On 03.07.20 22:29, Kim Barrett wrote:
>> On Jul 3, 2020, at 2:06 PM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
>>
>> Webrevs:
>>
>> http://cr.openjdk.java.net/~tschatzl/8210462/webrev.0_to_1/ (diff)
>> http://cr.openjdk.java.net/~tschatzl/8210462/webrev.1/ (full)
>>
>> Thanks,
>>   Thomas
> 
> ------------------------------------------------------------------------------
> src/hotspot/share/gc/g1/g1CollectedHeap.hpp
>   537   // otherwise reachable ensure that it is marked in the bitmap for concurrent marking
> 
> [pre-existing]
> There should be a sentence break or a semi-colon or something between
> "reachable" and "ensure"
> 
> ------------------------------------------------------------------------------
> src/hotspot/share/gc/g1/g1CollectorState.hpp
>    53   volatile bool _in_concurrent_mark_gc;
> 
> Shouldn't this be _in_concurrent_start_gc?
> 
> ------------------------------------------------------------------------------
> src/hotspot/share/gc/g1/g1OopClosures.inline.hpp
>   254     // closure during a concurrent mark pause then attempt to mark the object.
>   
> s/concurrent mark/concurrent start/ ?
> 
> ------------------------------------------------------------------------------
> src/hotspot/share/gc/g1/g1Policy.cpp
>   682   } else if (!is_young_only_pause(this_pause)) {
>   683     // This is a mixed GC. Here we decide whether to continue doing more
> 
> Maybe there should be an assertion that this really is a mixed pause?
> 
> Or maybe this should be testing for a mixed pause, and the else clause
> should assert is_young_only_pause.
> 

Fixed. Ran through tier1 again because of that change.

> ------------------------------------------------------------------------------
> src/hotspot/share/gc/g1/g1Policy.cpp
> [removed]
>   636   bool this_pause_was_young_only = collector_state()->in_young_only_phase();
> 
> Why was this variable removed and uses replaced with
> is_young_only_only(this_pause), rather than keeping the variable but
> updating the initialization?
> 
> ------------------------------------------------------------------------------
> 

The reason is that this_pause_was_young_only is a local defined waaay up 
at the top of that method and I thought instead of referencing that one 
it is better to do the (very simple) function call.
I can change that again if you want.

Webrevs:
http://cr.openjdk.java.net/~tschatzl/8210462/webrev.1_to_2/ (diff)
http://cr.openjdk.java.net/~tschatzl/8210462/webrev.2/ (full)
Testing:
tier1, local gc/g1 jtreg run

Thanks,
   Thomas


From aph at redhat.com  Tue Jul  7 08:42:19 2020
From: aph at redhat.com (Andrew Haley)
Date: Tue, 7 Jul 2020 09:42:19 +0100
Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences
 between free chunk check and klass read
In-Reply-To: <DA41BE1DDCA941489001C7FBD7A8820EE7E63D86@dggeml527-mbx.china.huawei.com>
References: <DA41BE1DDCA941489001C7FBD7A8820EE7E63D86@dggeml527-mbx.china.huawei.com>
Message-ID: <d6590e74-7ce2-70b9-7432-96b969686ab7@redhat.com>

On 07/07/2020 04:22, Yangfei (Felix) wrote:

> Webrev for jdk8u-dev:
> http://cr.openjdk.java.net/~fyang/8248851/webrev.00/ JTreg tested on
> x86_64-linux-gnu &aarch64-linux-gnu with jdk8u release builds.
> Comments?

That looks right. I was thinking of arguing that you only need
loadload here, but other parts of the same function use acquire, so we
might as well be consistent.

I'd lose this line:

// Bugfix for systems with weak memory model (AARCH64).

It doesn't add anything, and it may mislead people into thinking that
it's AArch64 specific. Looking at the code, it seems like all the
casting to volatile was an earlier attempt to fix the same bug.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671



From thomas.schatzl at oracle.com  Tue Jul  7 09:13:10 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 7 Jul 2020 11:13:10 +0200
Subject: RFR(S) [15] : 8208243 :
 vmTestbase/gc/lock/jni/jnilock002/TestDescription.java fails in jdk/hs
 nightly
In-Reply-To: <7A067017-1FE4-4BEC-ACC7-3E309528E1F3@oracle.com>
References: <D20BCA8E-597A-43B8-864C-B0B9A47C6F99@oracle.com>
 <7A067017-1FE4-4BEC-ACC7-3E309528E1F3@oracle.com>
Message-ID: <dff910ff-3b4c-0bd6-24de-b1ef893a51d2@oracle.com>

Hi,

On 07.07.20 04:57, Igor Ignatyev wrote:
> ping?
> -- Igor
> 
>> On Jul 2, 2020, at 11:02 AM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
>>
>> http://cr.openjdk.java.net/~iignatyev//8208243/webrev.00
>>> 325 lines changed: 18 ins; 0 del; 307 mod;
>>
>> Hi all,
>>
>> could you please review the patch for nsk.share.gc.lock.jni.*CriticalLocker.cpp files which:
>> - uses difftime to compare current_time and start_time;
>> - calls FatalError if Get*Critical returns NULL so the test would fail w/ a more obvious error than 'Native hash: 0 != Java hash:...' and won't hide this problem in cases when Java hash happens to be 0 (which is currently always true for all but float/double arrays);
>> - resets initial hash value in each iteration of while-loop;
>> - aligns whitespace w/ usual hotspot code style.
>>
>> testing: vmTestbase/gc/lock/jni on {linux,windows,macosx}-x64-{product,fastedebug} (w/ jnilock002 being removed from the problem-list)
>> JBS: https://bugs.openjdk.java.net/browse/JDK-8208243
>> webrev: http://cr.openjdk.java.net/~iignatyev//8208243/webrev.00
>>
>> Thanks,
>> -- Igor
> 

- in the CR there is a concern about the objFieldId static which seems 
not an issue after all.

Looks good.

Thomas


From thomas.schatzl at oracle.com  Tue Jul  7 09:54:11 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Tue, 7 Jul 2020 11:54:11 +0200
Subject: RFC: Adaptively resize heap at any GC/SoftMaxHeapSize for G1
In-Reply-To: <9a226b7d-cddb-45b8-47f7-7b7cd22a95aa@oracle.com>
References: <mailman.582.1592907080.17886.hotspot-gc-dev@openjdk.java.net>
 <CA++bR4NByktk1KK7qOQ3QyVO=wswX6AO8oU-6D3wOjjzt6H3fA@mail.gmail.com>
 <9a226b7d-cddb-45b8-47f7-7b7cd22a95aa@oracle.com>
Message-ID: <35439959-9395-1c85-01b2-6be1206f18fc@oracle.com>

Hi Ruslan,

On 06.07.20 11:11, Thomas Schatzl wrote:
> Hi,
> 
> On 05.07.20 18:21, Ruslan Synytsky wrote:
>>>
>>>
>> Unfortunately GC.heap_info and VM.info do not provide information about
>> COMMITTED heap. And jstat documentation

That is actually not true :) While looking into JDK-8248136, G1 actually 
already prints committed heap with GC.heap_info.

E.g. on an application with -Xms64m -Xmx1024m the output is:

$ jcmd 30653 GC.heap_info
30653:
  garbage-first heap   total 519168K, used 315920K [0x00000000c0000000, 
0x0000000100000000)
   region size 1024K, 116 young (118784K), 23 survivors (23552K)
[...]

The "total" is "available" regions (i.e. ~committed) as explained in the 
previous post (I kept the relevant part below).

This is probably the case since introduction of uncommit within the heap 
in 8u40 (JDK-8038423).

Thanks,
   Thomas

> Sorry. However, VM.info prints the whole heap map with G1, i.e. for 
> every region what type it is. For uncommitted regions, it does not print 
> such a line in the "Heap regions" section...(*)(**) so you could count 
> the lines and compare with what would be expected.
> 
> (*) the |?? 0|0x0000000600000000, 0x0000000600400000, 
> 0x0000000600400000|100%| O|? |TAMS 0x0000000600400000, 
> 0x0000000600000000| Untracked lines
> 
> (**) that is not completely correct, it prints "available" regions, i.e. 
> regions that the topmost layer of g1 thinks are there, but in some 
> cases, this is not entirely correct. I.e. if page size is > region size, 
> a region on one page may be "unavailable", but another is, but obviously 
> in that case the whole page, i.e. the space for all two regions, are 
> committed.
> 
> But you can reconstruct the data as the indices in that list are fixed. 
> E.g. if VM.info shows you, assuming 1m region and 2m page size. When 
> using small (4k) pages, this situation can't happen because region size 
> is always > page size at least on x86:
> 
> |? 0| ...
> |? 1| ...
> |? 3| ...
> |? 4| ...
> |? 5| ...
> |? 8| ...
> ....
> 
> then the page 0 where region 0/1 are located is committed, page 1 where 
> region 2/3 are located is committed (because g1 can't uncommit half 
> pages obviously), page 2 where region 4/5 is committed, page 3 where 
> regions 6/7 are NOT committed because both are missing, and so on.
> 
> I hope this makes sense to you.
> 
> While a somewhat cumbersome way to find this out, this works since 
> jdk8u40 (implemented in JDK-8038423) iirc.
> 
> I recently filed JDK-8248136 for improving the heap info output for G1.
> 
> Thanks,
>  ? Thomas



From felix.yang at huawei.com  Tue Jul  7 11:48:00 2020
From: felix.yang at huawei.com (Yangfei (Felix))
Date: Tue, 7 Jul 2020 11:48:00 +0000
Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences
 between free chunk check and klass read
In-Reply-To: <d6590e74-7ce2-70b9-7432-96b969686ab7@redhat.com>
References: <DA41BE1DDCA941489001C7FBD7A8820EE7E63D86@dggeml527-mbx.china.huawei.com>
 <d6590e74-7ce2-70b9-7432-96b969686ab7@redhat.com>
Message-ID: <DA41BE1DDCA941489001C7FBD7A8820EE7E6466E@dggeml527-mbx.china.huawei.com>

Hi,

> -----Original Message-----
> From: Andrew Haley [mailto:aph at redhat.com]
> Sent: Tuesday, July 7, 2020 4:42 PM
> To: Yangfei (Felix) <felix.yang at huawei.com>; 'jdk8u-dev at openjdk.java.net'
> <jdk8u-dev at openjdk.java.net>
> Cc: hotspot-gc-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net
> Subject: Re: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory
> fences between free chunk check and klass read
> 
> On 07/07/2020 04:22, Yangfei (Felix) wrote:
> 
> > Webrev for jdk8u-dev:
> > http://cr.openjdk.java.net/~fyang/8248851/webrev.00/ JTreg tested on
> > x86_64-linux-gnu &aarch64-linux-gnu with jdk8u release builds.
> > Comments?
> 
> That looks right. I was thinking of arguing that you only need loadload here,
> but other parts of the same function use acquire, so we might as well be
> consistent.
> 
> I'd lose this line:
> 
> // Bugfix for systems with weak memory model (AARCH64).
> 
> It doesn't add anything, and it may mislead people into thinking that it's
> AArch64 specific. Looking at the code, it seems like all the casting to volatile
> was an earlier attempt to fix the same bug.

Good suggestion.
I have prepared a new webrev with the line removed: http://cr.openjdk.java.net/~fyang/8248851/webrev.01/ 
Does it look better?

Thanks,
Felix

From kim.barrett at oracle.com  Tue Jul  7 12:38:42 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Tue, 7 Jul 2020 08:38:42 -0400
Subject: RFR (M): 8210462: Fix remaining mentions of initial mark
In-Reply-To: <8edfe69d-822d-59c7-5130-5ab0b2fc2d1c@oracle.com>
References: <bc347a79-63d4-e259-3e57-53ddb4a00be1@oracle.com>
 <6f950a83-6942-b4e9-fb49-5ee20635add6@oracle.com>
 <a5411b45-4a50-9a0d-ef7d-7dc45d603d22@oracle.com>
 <3D90704F-D7BC-4E24-9783-76993989602C@oracle.com>
 <8edfe69d-822d-59c7-5130-5ab0b2fc2d1c@oracle.com>
Message-ID: <0AFEB8CC-039E-4402-838A-D88657B05D62@oracle.com>

> On Jul 7, 2020, at 4:39 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> 
> Hi Kim,
> 
>  thanks for your review.
> 
> On 03.07.20 22:29, Kim Barrett wrote:
>> src/hotspot/share/gc/g1/g1Policy.cpp
>> [removed]
>>  636   bool this_pause_was_young_only = collector_state()->in_young_only_phase();
>> Why was this variable removed and uses replaced with
>> is_young_only_only(this_pause), rather than keeping the variable but
>> updating the initialization?
>> ------------------------------------------------------------------------------
> 
> The reason is that this_pause_was_young_only is a local defined waaay up at the top of that method and I thought instead of referencing that one it is better to do the (very simple) function call.
> I can change that again if you want.

OK.  The ?far away? rationale makes sense.

> Webrevs:
> http://cr.openjdk.java.net/~tschatzl/8210462/webrev.1_to_2/ (diff)
> http://cr.openjdk.java.net/~tschatzl/8210462/webrev.2/ (full)
> Testing:
> tier1, local gc/g1 jtreg run

Looks good.



From kim.barrett at oracle.com  Tue Jul  7 13:28:02 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Tue, 7 Jul 2020 09:28:02 -0400
Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences
 between free chunk check and klass read
In-Reply-To: <DA41BE1DDCA941489001C7FBD7A8820EE7E6466E@dggeml527-mbx.china.huawei.com>
References: <DA41BE1DDCA941489001C7FBD7A8820EE7E63D86@dggeml527-mbx.china.huawei.com>
 <d6590e74-7ce2-70b9-7432-96b969686ab7@redhat.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E6466E@dggeml527-mbx.china.huawei.com>
Message-ID: <8957BB9E-617D-4E24-9CC4-31AF7D01D12E@oracle.com>

> On Jul 7, 2020, at 7:48 AM, Yangfei (Felix) <felix.yang at huawei.com> wrote:
> 
> Hi,
> 
>> -----Original Message-----
>> From: Andrew Haley [mailto:aph at redhat.com]
>> Sent: Tuesday, July 7, 2020 4:42 PM
>> To: Yangfei (Felix) <felix.yang at huawei.com>; 'jdk8u-dev at openjdk.java.net'
>> <jdk8u-dev at openjdk.java.net>
>> Cc: hotspot-gc-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net
>> Subject: Re: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory
>> fences between free chunk check and klass read
>> 
>> On 07/07/2020 04:22, Yangfei (Felix) wrote:
>> 
>>> Webrev for jdk8u-dev:
>>> http://cr.openjdk.java.net/~fyang/8248851/webrev.00/ JTreg tested on
>>> x86_64-linux-gnu &aarch64-linux-gnu with jdk8u release builds.
>>> Comments?
>> 
>> That looks right. I was thinking of arguing that you only need loadload here,
>> but other parts of the same function use acquire, so we might as well be
>> consistent.
>> 
>> I'd lose this line:
>> 
>> // Bugfix for systems with weak memory model (AARCH64).
>> 
>> It doesn't add anything, and it may mislead people into thinking that it's
>> AArch64 specific. Looking at the code, it seems like all the casting to volatile
>> was an earlier attempt to fix the same bug.
> 
> Good suggestion.
> I have prepared a new webrev with the line removed: http://cr.openjdk.java.net/~fyang/8248851/webrev.01/ 
> Does it look better?
> 
> Thanks,
> Felix

This change seems needed, but is it sufficient? Seems like there
should be a corresponding release/storestore/something for the
correspnding writes; is it there? (I haven't looked at CMS for quite a
while, so no longer sure where to look.)

This issue seems to persist all the way until CMS removal in JDK 14.

Also, there's the closely related JDK-8160369 (JDK 9), which doesn't
appear to have been backported to JDK 8.



From kim.barrett at oracle.com  Tue Jul  7 13:47:46 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Tue, 7 Jul 2020 09:47:46 -0400
Subject: [16] RFR: 8248391: Unify handling of all OopStorage instances in
 weak root processing
In-Reply-To: <eb24f5f4-fdcc-9f48-99d3-ff7a0ac9f86b@oracle.com>
References: <8e831a42-61b0-1605-ea71-09f97d82f328@oracle.com>
 <11EE1DA7-3265-4A40-BA37-95B66B129F50@oracle.com>
 <eb24f5f4-fdcc-9f48-99d3-ff7a0ac9f86b@oracle.com>
Message-ID: <3A6A85AC-50A7-41AF-AE7E-9967C5B43AF9@oracle.com>

> On Jun 30, 2020, at 12:35 PM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
> 
> I talked to kim about this point:
> 
> "The GC (in particular, OopStorage) shouldn't know about client
> subsystems in order to record the notification function"
> 
> The current OopStorageSet implementation already knows about the different client subsystems, therefore I though that adding the notification registration there actually made sense. I've seen the alternative where the notification registration happens in the client subsystem, and that patch didn't look that great either, IMHO.
> 
> So, I decided to create a PoC to (almost) completely decouple the client subsystems and the OopStorageSet. With the patch, the OopStorageSet doesn't know anything about the existing subsystems. It only knows about the number of weak and strong OopStorages in the JVM. The need to know the exact numbers arise from the wish to statically know that number during compile time. I'm not sure it's important to keep this static nature of the number of OopStorages, instead of using a more dynamic approach, but I've left this as is for now.
> 
> https://cr.openjdk.java.net/~stefank/8248391/webrev.02.delta/
> https://cr.openjdk.java.net/~stefank/8248391/webrev.02

I have some comments, but because of vacations I'm providing them as a
further patch on top of Stefan's webrev.02, plus the description below.
(I'm abandoning the earlier never-published prototype derived from Erik's
changes in favor of using Stefan's additional PoC plus further changes based
on it.)  If this all gets reviewed before folks are back, I'll deal with
pushing, with Erik and Stefan as contributors.

full: https://cr.openjdk.java.net/~kbarrett/8248391/open.03/
incr: https://cr.openjdk.java.net/~kbarrett/8248391/open.03.inc/

Testing:
mach5 tier1- TBD
Locally ran new StringTableCleaningTest.java with a duration of 1/2 hour for
each collector (including Shenandoah) except Epsilon.

(Note that the baseline for the incremental patch is a slight modification
of Stefan's webrev.02, to deal with a recent graal update that moved JVMCI's
interface to OopStorage from jvmci.cpp to jvmciRuntime.cpp.)

> One problem with the current state of this patch is that during G1's heap initialization, it wants to eagerly figure out all OopStorage names. The problem is that with my patch OopStorages are initialized together with its subsystem, and the StringTable and ResolvedMethodTable are initialized after the heap. It's a bit unfortunate, and I would like to find a good solution for that, but for this PoC I've changed the G1Policy::phase_times to lazily created phase times instance. This moves the allocations to the first GC. I'm open for suggestions on what do do about that part.

I haven't come up with anything that seems better than Stefan's lazy
allocation of the G1GCPhaseTimes object to deal with the ordering problem
with subsystem registration of the OopStorage objects.  So I'm leaving that
as is, at least for now.

I think the proposed terminology for the GC -> OopStorage owner callback was
too vague and generic; "notify" and "notification" are relatively
content-free names.  So I've renamed the functions and types.  Also, I think
"can_notify" should be "should_notify" (and then adjusted for the generic
"notify").

I've also changed (and documented) the semantics; the number of dead
reported for an iteration is the number of entries skipped because they were
already NULL plus the number cleared by the iteration.  (So the total dead,
not just the recently deceased.)  That's the meaning in the old reporting
mechanism.  Earlier versions of this change instead invoked the callback
with just the number of newly cleared entries, with it being up to the
receiving subsystem to accumulate the total.  But there are race conditions
between GC and concurrent subsystem cleanup (like the ServiceThread's
concurrent cleaning of the StringTable) that make such accumulation by the
client potentially wildly inaccurate, leading to excess re-cleanings.  (Even
with the callback receiving the total dead there are races, but I think the
potential for significant inaccuracies leading to time wasting are much
less.)

A possible further change that I've left for later is that the
WeakProcessorPhaseTimeTracker gets just the cleared (newly dead) count.
(That hasn't been touched by any version of this change.)  I think the total
dead would be a more useful statistic for the associated logging.

The (anonymous) enum of OopStorageSet iterator indices no longer serves any
purpose; I've changed them to be ordinary in-class constants.  With the
original code the enum was used to automatically assign values to some of
the enumerators, but with the removal of named indices for each storage,
there was only one autmatically assigned enumerator.

I've changed WeakProcessor::Task to also use the new
OopStorageSetWeakParState.  This involved adding support for client access
to the states, similar to what's in the strong parstate.  (I'm not sure the
weak parstate should provide an oops_do that counts and reports the dead,
but I've left that for now.)

Fixed a "bug" that WeakProcessor::oops_do(OopClosure*) counted and reported
dead entries, even though it never clears entries.  It is used for
iterations that shouldn't be reporting dead, such as for the adjust pointers
phase of mark-sweep-compact.

Besides adapting to the above changes, I moved StringTable's clearing of the
_has_work flag to the end of its do_concurrent_work.  This avoids queuing
a new cleanup request in the middle of a concurrent cleanup that might
eliminate the need for further cleaning.  We could try to re-evaluate at the
end of the cleanup, but instead leave that to the next GC to trigger a new
round of cleanup if needed.  Also removed some inconsistencies between
ResolvedMethodTable and StringTable.

I've also added a test that beats on StringTable allocation and cleaning.



From aph at redhat.com  Tue Jul  7 15:16:18 2020
From: aph at redhat.com (Andrew Haley)
Date: Tue, 7 Jul 2020 16:16:18 +0100
Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences
 between free chunk check and klass read
In-Reply-To: <DA41BE1DDCA941489001C7FBD7A8820EE7E6466E@dggeml527-mbx.china.huawei.com>
References: <DA41BE1DDCA941489001C7FBD7A8820EE7E63D86@dggeml527-mbx.china.huawei.com>
 <d6590e74-7ce2-70b9-7432-96b969686ab7@redhat.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E6466E@dggeml527-mbx.china.huawei.com>
Message-ID: <e8c2d4b0-bf97-0f20-d916-7d6ef3a5ff5b@redhat.com>

On 07/07/2020 12:48, Yangfei (Felix) wrote:
> I have prepared a new webrev with the line removed: http://cr.openjdk.java.net/~fyang/8248851/webrev.01/ 
> Does it look better?

Thanks, that's good.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671



From rs at jelastic.com  Tue Jul  7 16:23:32 2020
From: rs at jelastic.com (Ruslan Synytsky)
Date: Tue, 7 Jul 2020 18:23:32 +0200
Subject: RFC: Adaptively resize heap at any GC/SoftMaxHeapSize for G1
In-Reply-To: <mailman.52.1594122902.23224.hotspot-gc-dev@openjdk.java.net>
References: <mailman.52.1594122902.23224.hotspot-gc-dev@openjdk.java.net>
Message-ID: <CA++bR4OOoVv-PYJ-Ps3zkoNUwJC8-F6hD_X51C8YWsHV-TexpQ@mail.gmail.com>

> >> Unfortunately GC.heap_info and VM.info do not provide information about
> >> COMMITTED heap. And jstat documentation
>
> That is actually not true :) While looking into JDK-8248136, G1 actually
> already prints committed heap with GC.heap_info.
>
> E.g. on an application with -Xms64m -Xmx1024m the output is:
>
> $ jcmd 30653 GC.heap_info
> 30653:
>   garbage-first heap   total 519168K, used 315920K [0x00000000c0000000,
> 0x0000000100000000)
>    region size 1024K, 116 young (118784K), 23 survivors (23552K)
> [...]
>
> The "total" is "available" regions (i.e. ~committed) as explained in the
> previous post (I kept the relevant part below).
>
Thomas, thank you, it helps! I'm glad that I was wrong. The "total" naming
confused me, I was thinking it's the max (Xmx) heap.

As a follow up question - how to get the non heap usage? It can be useful
for understanding of the full picture. JMX provides the following option:




*MemoryMXBean mem = ManagementFactory.getPlatformMXBean(mbsc,
MemoryMXBean.class);MemoryUsage nonHeap =
mem.getNonHeapMemoryUsage();System.out.println(nonHeap.getInit() + "," +
nonHeap.getUsed() + "|, + nonHeap.getCommitted() + "," + nonHeap.getMax());*


> I recently filed JDK-8248136 for improving the heap info output for G1.

Improving the heap info sounds like a useful enhancement. At the moment
GC.heap_info is printed by different GCs with different naming.

*G1*




* garbage-first heap   total 204800K, used 1806K [0x0000000080000000,
0x0000000100000000)  region size 1024K, 0 young (0K), 0 survivors
(0K) Metaspace       used 1039K, capacity 4619K, committed 4864K, reserved
1056768K  class space    used 98K, capacity 431K, committed 512K, reserved
1048576K*
*Parallel*







* par new generation   total 27648K, used 4493K [0x000000068cc00000,
0x000000068ea00000, 0x000000068ea00000)  eden space 24576K,  18% used
[0x000000068cc00000, 0x000000068d063638, 0x000000068e400000)  from space
3072K,   0% used [0x000000068e400000, 0x000000068e400000,
0x000000068e700000)  to   space 3072K,   0% used [0x000000068e700000,
0x000000068e700000, 0x000000068ea00000) concurrent mark-sweep generation
total 2048K, used 0K [0x000000068ea00000, 0x000000068ec00000,
0x00000007c0000000) Metaspace       used 4439K, capacity 4602K, committed
4864K, reserved 1056768K  class space    used 423K, capacity 426K,
committed 512K, reserved 1048576K*
*ZGC*


* ZHeap           used 10M, capacity 48M, max capacity 3482M Metaspace
  used 7608K, capacity 7668K, committed 7680K, reserved 8192K*
*Shenandoah*











* Shenandoah Heap 4914M total, 32768K committed, 3345K used 2457 x 2048K
regionsStatus: not cancelledReserved region: - [0x000000068cc00000,
0x00000007c0000000) Collection set: - map (vanilla): 0x00007f9d98e52466 -
map (biased):  0x00007f9d98e4f000 Metaspace       used 4406K, capacity
4538K, committed 4864K, reserved 1056768K  class space    used 422K,
capacity 426K, committed 512K, reserved 1048576K*

Shenandoah provides the most distinct output including Xmx which is called
"total" while "total" means committed in G1. Also ZGC prints committed heap
as capacity while metaspace has both capacity and committed... Does it make
sense to harmonize the naming?

>> Thomas and Liang, is there a possibility to easily add an optional
>> manageable parameter that regulates behaviour of JVM when memory
>> consumption goes above SoftMaxHeapSize? For example, by default JVM
>> allocates more memory when it can't keep memory usage
>> below SoftMaxHeapSize, and if the optional parameter was specified then
JVM
>> throws OutOfMemoryError. In this case we will cover two cases with the
same
>> code base.

> I think there is still the patch from Rodrigo; from my understanding
> from last time there were some issues around when you are allowed to
> change that (as Current/HardMaxHeapSize is read at "arbitrary" locations
> you need to make sure the gc has a consistent view of it), and naming: I
> am not sure but CurrentMaxHeapSize has been the favorite or so.

Ok, I will talk to Rodrigo and Liang to clarify what we can do as the next
step.

Thank you
-- 
Ruslan


From igor.ignatyev at oracle.com  Tue Jul  7 17:14:43 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Tue, 7 Jul 2020 10:14:43 -0700
Subject: RFR(S) [15] : 8208243 :
 vmTestbase/gc/lock/jni/jnilock002/TestDescription.java fails in jdk/hs
 nightly
In-Reply-To: <dff910ff-3b4c-0bd6-24de-b1ef893a51d2@oracle.com>
References: <D20BCA8E-597A-43B8-864C-B0B9A47C6F99@oracle.com>
 <7A067017-1FE4-4BEC-ACC7-3E309528E1F3@oracle.com>
 <dff910ff-3b4c-0bd6-24de-b1ef893a51d2@oracle.com>
Message-ID: <38D405EE-F942-4942-8708-3EA6B4D9D64E@oracle.com>

thanks Thomas, pushed.

-- Igor

> On Jul 7, 2020, at 2:13 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> 
> Hi,
> 
> On 07.07.20 04:57, Igor Ignatyev wrote:
>> ping?
>> -- Igor
>>> On Jul 2, 2020, at 11:02 AM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
>>> 
>>> http://cr.openjdk.java.net/~iignatyev//8208243/webrev.00
>>>> 325 lines changed: 18 ins; 0 del; 307 mod;
>>> 
>>> Hi all,
>>> 
>>> could you please review the patch for nsk.share.gc.lock.jni.*CriticalLocker.cpp files which:
>>> - uses difftime to compare current_time and start_time;
>>> - calls FatalError if Get*Critical returns NULL so the test would fail w/ a more obvious error than 'Native hash: 0 != Java hash:...' and won't hide this problem in cases when Java hash happens to be 0 (which is currently always true for all but float/double arrays);
>>> - resets initial hash value in each iteration of while-loop;
>>> - aligns whitespace w/ usual hotspot code style.
>>> 
>>> testing: vmTestbase/gc/lock/jni on {linux,windows,macosx}-x64-{product,fastedebug} (w/ jnilock002 being removed from the problem-list)
>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8208243
>>> webrev: http://cr.openjdk.java.net/~iignatyev//8208243/webrev.00
>>> 
>>> Thanks,
>>> -- Igor
> 
> - in the CR there is a concern about the objFieldId static which seems not an issue after all.
> 
> Looks good.

> 
> Thomas



From igor.ignatyev at oracle.com  Wed Jul  8 00:38:18 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Tue, 7 Jul 2020 17:38:18 -0700
Subject: RFR(S) [15] : 8249000 : vm.gc.X should take selected JIT into account
Message-ID: <FFC49DC0-AA15-45B5-B285-E043D1AEAF97@oracle.com>

http://cr.openjdk.java.net/~iignatyev/8249000/webrev.00/
> 241 lines changed: 34 ins; 5 del; 202 mod;


Hi all,

could you please review the patch which modifies requires/VMProps to set vm.gc.X to false if Graal is selected and X GC isn't supported by Graal?

the patch also replaces @requires similar to `vm.gc.X & !vm.graal.enabled` w/ `vm.gc.X` where it's applicable.

from JBS:
> not all GCs are supported by Graal JIT, which leads to failures like JDK-8247527 and boilerplate fixes like replacing all `@requires vm.gc.Z` w/ `@requires vm.gc.Z & !vm.graal.enabled`. 
> 
> as vm.gc.X means that X GC can be selected, it would be more natural, less surprising, and much more clear to have it true if the selected JIT supports the said X GC.

webrev: http://cr.openjdk.java.net/~iignatyev/8249000/webrev.00/
JBS: https://bugs.openjdk.java.net/browse/JDK-8249000
testing: test/hotspot/jtreg/{gc,compiler,runtime,serviceability} on {linux,windows,macos}-x64 w/ and w/o Graal as JIT

Thanks,
-- Igor

From felix.yang at huawei.com  Wed Jul  8 02:49:17 2020
From: felix.yang at huawei.com (Yangfei (Felix))
Date: Wed, 8 Jul 2020 02:49:17 +0000
Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences
 between free chunk check and klass read
In-Reply-To: <8957BB9E-617D-4E24-9CC4-31AF7D01D12E@oracle.com>
References: <DA41BE1DDCA941489001C7FBD7A8820EE7E63D86@dggeml527-mbx.china.huawei.com>
 <d6590e74-7ce2-70b9-7432-96b969686ab7@redhat.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E6466E@dggeml527-mbx.china.huawei.com>
 <8957BB9E-617D-4E24-9CC4-31AF7D01D12E@oracle.com>
Message-ID: <DA41BE1DDCA941489001C7FBD7A8820EE7E64ECD@dggeml527-mbx.china.huawei.com>

Hi,

> -----Original Message-----
> From: Kim Barrett [mailto:kim.barrett at oracle.com]
> Sent: Tuesday, July 7, 2020 9:28 PM
> To: Yangfei (Felix) <felix.yang at huawei.com>
> Cc: Andrew Haley <aph at redhat.com>; jdk8u-dev at openjdk.java.net;
> hotspot-gc-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net
> Subject: Re: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory
> fences between free chunk check and klass read
> 

Cut...

> >
> > Good suggestion.
> > I have prepared a new webrev with the line removed:
> > http://cr.openjdk.java.net/~fyang/8248851/webrev.01/
> > Does it look better?
> >
> > Thanks,
> > Felix
> 
> This change seems needed, but is it sufficient? Seems like there should be a
> corresponding release/storestore/something for the correspnding writes; is
> it there? (I haven't looked at CMS for quite a while, so no longer sure where
> to look.)
 
Yes, I think it's there.  As mention in my first mail, the logic in markNotFree:

void markNotFree() {
     // Set _prev (klass) to null before (if) clearing the mark word below
     _prev = NULL;                                                        <========== Write klass (= NULL)
#ifdef _LP64
     if (UseCompressedOops) {
       OrderAccess::storestore();                             <========== StoreStore barrier
       set_mark(markOopDesc::prototype());    <========== Write marking not free
     }
#endif
     assert(!is_free(), "Error");
}

> This issue seems to persist all the way until CMS removal in JDK 14.

Since CMS is deprecated from JDK9, I am not sure if it's appropriate to fix this issue for those JDK9+ versions.

> Also, there's the closely related JDK-8160369 (JDK 9), which doesn't appear to
> have been backported to JDK 8.

Yeah, we also noticed that and was looking into it to see the possibility of a 8u backport.

Thanks,
Felix

From vladimir.kozlov at oracle.com  Wed Jul  8 03:00:12 2020
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Tue, 7 Jul 2020 20:00:12 -0700
Subject: RFR(S) [15] : 8249000 : vm.gc.X should take selected JIT into
 account
In-Reply-To: <FFC49DC0-AA15-45B5-B285-E043D1AEAF97@oracle.com>
References: <FFC49DC0-AA15-45B5-B285-E043D1AEAF97@oracle.com>
Message-ID: <6964ac32-e9ec-d700-0bdb-ea51f4610afe@oracle.com>

Nice clean up, Igor

test/hotspot/jtreg/gc/stress/TestReclaimStringsLeaksMemory.java

Do we even can have vm.gc=="null" based on code in VMProps.java? At least some GC should be selected ergonomically even 
if non is specified on command line.

- * @requires vm.gc=="null" & !vm.graal.enabled & !vm.debug
+ * @requires vm.gc == "null"
+ * @requires !vm.debug


test/hotspot/jtreg/runtime/cds/appcds/TestZGCWithCDS.java

Does next combination of @requires ever work? I thought such sequence means 'AND' operation on all such conditions.

   * @requires vm.gc.Z
   * @requires vm.gc.Serial
   * @requires vm.gc == null


Thanks,
Vladimir

On 7/7/20 5:38 PM, Igor Ignatyev wrote:
> http://cr.openjdk.java.net/~iignatyev/8249000/webrev.00/
>> 241 lines changed: 34 ins; 5 del; 202 mod;
> 
> 
> Hi all,
> 
> could you please review the patch which modifies requires/VMProps to set vm.gc.X to false if Graal is selected and X GC isn't supported by Graal?
> 
> the patch also replaces @requires similar to `vm.gc.X & !vm.graal.enabled` w/ `vm.gc.X` where it's applicable.
> 
> from JBS:
>> not all GCs are supported by Graal JIT, which leads to failures like JDK-8247527 and boilerplate fixes like replacing all `@requires vm.gc.Z` w/ `@requires vm.gc.Z & !vm.graal.enabled`.
>>
>> as vm.gc.X means that X GC can be selected, it would be more natural, less surprising, and much more clear to have it true if the selected JIT supports the said X GC.
> 
> webrev: http://cr.openjdk.java.net/~iignatyev/8249000/webrev.00/
> JBS: https://bugs.openjdk.java.net/browse/JDK-8249000
> testing: test/hotspot/jtreg/{gc,compiler,runtime,serviceability} on {linux,windows,macos}-x64 w/ and w/o Graal as JIT
> 
> Thanks,
> -- Igor
> 


From igor.ignatyev at oracle.com  Wed Jul  8 03:30:38 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Tue, 7 Jul 2020 20:30:38 -0700
Subject: RFR(S) [15] : 8249000 : vm.gc.X should take selected JIT into
 account
In-Reply-To: <6964ac32-e9ec-d700-0bdb-ea51f4610afe@oracle.com>
References: <FFC49DC0-AA15-45B5-B285-E043D1AEAF97@oracle.com>
 <6964ac32-e9ec-d700-0bdb-ea51f4610afe@oracle.com>
Message-ID: <7A1992A7-1493-4DF0-B621-195CE986D34F@oracle.com>

Hi Vladimir,

thanks for your review!

`vm.gc` and `vm.gc.X`-s are different beasts (and admittedly, they confuse people a lot), `vm.gc` is set to "X", by jtreg itself, only if there is UseXGC in vm flags, otherwise it's "null". `vm.gc.X` are set by VMProps class, and you can have more than one vm.gc.X == true, as vm.gc.X means that X gc is supported by JVM and it can be selected; so if there are no Use.*GC in vm flags, vm.gc.X will yield true for all GCs which JVM was built with; if one of UseXGC is provided, only corresponding vm.gc.X is true, and all others are false. so to answer your questions, yes `vm.gc` can be "null" (if there are no Use.*GC) , and yes `vm.gc.Z & vm.gc.Serial & vm.gc == null` can be true (if there are no Use.*GC and JVM supports both Z and Serial GCs).

Thanks,
-- Igor 


> On Jul 7, 2020, at 8:00 PM, Vladimir Kozlov <vladimir.kozlov at oracle.com> wrote:
> 
> Nice clean up, Igor
> 
> test/hotspot/jtreg/gc/stress/TestReclaimStringsLeaksMemory.java
> 
> Do we even can have vm.gc=="null" based on code in VMProps.java? At least some GC should be selected ergonomically even if non is specified on command line.
> 
> - * @requires vm.gc=="null" & !vm.graal.enabled & !vm.debug
> + * @requires vm.gc == "null"
> + * @requires !vm.debug

> 
> 
> test/hotspot/jtreg/runtime/cds/appcds/TestZGCWithCDS.java
> 
> Does next combination of @requires ever work? I thought such sequence means 'AND' operation on all such conditions.
> 
>  * @requires vm.gc.Z
>  * @requires vm.gc.Serial
>  * @requires vm.gc == null
> 
> 
> Thanks,
> Vladimir
> 
> On 7/7/20 5:38 PM, Igor Ignatyev wrote:
>> http://cr.openjdk.java.net/~iignatyev/8249000/webrev.00/
>>> 241 lines changed: 34 ins; 5 del; 202 mod;
>> Hi all,
>> could you please review the patch which modifies requires/VMProps to set vm.gc.X to false if Graal is selected and X GC isn't supported by Graal?
>> the patch also replaces @requires similar to `vm.gc.X & !vm.graal.enabled` w/ `vm.gc.X` where it's applicable.
>> from JBS:
>>> not all GCs are supported by Graal JIT, which leads to failures like JDK-8247527 and boilerplate fixes like replacing all `@requires vm.gc.Z` w/ `@requires vm.gc.Z & !vm.graal.enabled`.
>>> 
>>> as vm.gc.X means that X GC can be selected, it would be more natural, less surprising, and much more clear to have it true if the selected JIT supports the said X GC.
>> webrev: http://cr.openjdk.java.net/~iignatyev/8249000/webrev.00/
>> JBS: https://bugs.openjdk.java.net/browse/JDK-8249000
>> testing: test/hotspot/jtreg/{gc,compiler,runtime,serviceability} on {linux,windows,macos}-x64 w/ and w/o Graal as JIT
>> Thanks,
>> -- Igor



From kim.barrett at oracle.com  Wed Jul  8 05:02:09 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 8 Jul 2020 01:02:09 -0400
Subject: RFR (S): 8248322: G1: Refactor full collection sizing code
In-Reply-To: <1a156b7e-f5fc-9910-1248-8d60f3cb38a1@oracle.com>
References: <1a156b7e-f5fc-9910-1248-8d60f3cb38a1@oracle.com>
Message-ID: <6F553B34-90AD-406A-965B-978EC97DC88F@oracle.com>

> On Jun 26, 2020, at 10:30 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> 
> Hi all,
> 
>  can I have reviews for this small refactoring changes that moves some heap sizing policy related code into a the "right" place (into the heap sizing policy class), and refactors it a bit.
> 
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8248322
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8248322/webrev/
> Testing:
> tier1
> 
> Thanks,
>  Thomas

Looks good.



From kim.barrett at oracle.com  Wed Jul  8 08:41:03 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 8 Jul 2020 04:41:03 -0400
Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences
 between free chunk check and klass read
In-Reply-To: <DA41BE1DDCA941489001C7FBD7A8820EE7E64ECD@dggeml527-mbx.china.huawei.com>
References: <DA41BE1DDCA941489001C7FBD7A8820EE7E63D86@dggeml527-mbx.china.huawei.com>
 <d6590e74-7ce2-70b9-7432-96b969686ab7@redhat.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E6466E@dggeml527-mbx.china.huawei.com>
 <8957BB9E-617D-4E24-9CC4-31AF7D01D12E@oracle.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E64ECD@dggeml527-mbx.china.huawei.com>
Message-ID: <65012FED-2A93-4899-939E-887175AB07AC@oracle.com>

> On Jul 7, 2020, at 10:49 PM, Yangfei (Felix) <felix.yang at huawei.com> wrote:
>> 
>> This change seems needed, but is it sufficient? Seems like there should be a
>> corresponding release/storestore/something for the correspnding writes; is
>> it there? (I haven't looked at CMS for quite a while, so no longer sure where
>> to look.)
> 
> Yes, I think it's there.  As mention in my first mail, the logic in markNotFree:
> 
> void markNotFree() {
>     // Set _prev (klass) to null before (if) clearing the mark word below
>     _prev = NULL;                                                        <========== Write klass (= NULL)
> #ifdef _LP64
>     if (UseCompressedOops) {
>       OrderAccess::storestore();                             <========== StoreStore barrier
>       set_mark(markOopDesc::prototype());    <========== Write marking not free
>     }
> #endif
>     assert(!is_free(), "Error");
> }

OK, I think I see what?s going on here.

Change looks good.

>> This issue seems to persist all the way until CMS removal in JDK 14.
> 
> Since CMS is deprecated from JDK9, I am not sure if it's appropriate to fix this issue for those JDK9+ versions.

Deprecated != unsupported.



From ivan.walulya at oracle.com  Wed Jul  8 08:45:32 2020
From: ivan.walulya at oracle.com (Ivan Walulya)
Date: Wed, 8 Jul 2020 10:45:32 +0200
Subject: RFR (S): 8248322: G1: Refactor full collection sizing code
In-Reply-To: <1a156b7e-f5fc-9910-1248-8d60f3cb38a1@oracle.com>
References: <1a156b7e-f5fc-9910-1248-8d60f3cb38a1@oracle.com>
Message-ID: <B05819C1-E5A9-4D0E-9B72-C34BFA670BFF@oracle.com>

Looks good to me!

Minor comment.

+static size_t target_heap_capacity(size_t used_bytes, uintx free_ratio) {
+  const double free_percentage = (double) free_ratio / 100.0;
+  const double used_percentage = 1.0 - free_percentage;
Clearer if the above variable names are also prefixed with ?desired_?. 


> On 26 Jun 2020, at 16:30, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> 
> Hi all,
> 
>  can I have reviews for this small refactoring changes that moves some heap sizing policy related code into a the "right" place (into the heap sizing policy class), and refactors it a bit.
> 
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8248322
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8248322/webrev/
> Testing:
> tier1
> 
> Thanks,
>  Thomas
> 



From aph at redhat.com  Wed Jul  8 09:48:09 2020
From: aph at redhat.com (Andrew Haley)
Date: Wed, 8 Jul 2020 10:48:09 +0100
Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences
 between free chunk check and klass read
In-Reply-To: <65012FED-2A93-4899-939E-887175AB07AC@oracle.com>
References: <DA41BE1DDCA941489001C7FBD7A8820EE7E63D86@dggeml527-mbx.china.huawei.com>
 <d6590e74-7ce2-70b9-7432-96b969686ab7@redhat.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E6466E@dggeml527-mbx.china.huawei.com>
 <8957BB9E-617D-4E24-9CC4-31AF7D01D12E@oracle.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E64ECD@dggeml527-mbx.china.huawei.com>
 <65012FED-2A93-4899-939E-887175AB07AC@oracle.com>
Message-ID: <06646cf3-bc21-977f-96e2-1e9a974d1b00@redhat.com>

On 08/07/2020 09:41, Kim Barrett wrote:
>> Since CMS is deprecated from JDK9, I am not sure if it's appropriate to fix this issue for those JDK9+ versions.
> Deprecated != unsupported.
> 

Yes, it must be done. Thanks.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671



From kim.barrett at oracle.com  Wed Jul  8 15:31:06 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 8 Jul 2020 11:31:06 -0400
Subject: RFR (M): 8245721: Refactor the TaskTerminator
In-Reply-To: <bd530d10-a4bb-969f-b142-39ff259c91b8@oracle.com>
References: <bd530d10-a4bb-969f-b142-39ff259c91b8@oracle.com>
Message-ID: <931140A9-3FCF-41B4-905B-C53BAE4B8E40@oracle.com>

> On Jun 24, 2020, at 4:03 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> 
> CR:
> https://bugs.openjdk.java.net/browse/JDK-8245721
> Webrev:
> http://cr.openjdk.java.net/~tschatzl/8245721/webrev/
> Testing:
> tier1-5, many many perf rounds, many tier1-X rounds with other patches
> 
> Thanks,
>  Thomas

------------------------------------------------------------------------------
src/hotspot/share/gc/shared/taskTerminator.hpp
  54   struct SpinContext {
  55     uint yield_count;
...

While there's certainly variability on this, discussions I've had with
others and other reviews suggest the Style Guide's suggestion to use
leading underscores for data member names should apply even for public
members of structs.

------------------------------------------------------------------------------
src/hotspot/share/gc/shared/taskTerminator.hpp
 128       spin_context.hard_spin_limit = MIN2(2 * spin_context.hard_spin_limit,
 129                                           (uint) WorkStealingHardSpins);

In the new code, hard_spin_limit is always (re)initialized to
WorkStealingHardSpins, so this does nothing.

In the old code, the starting spin limit was
  WorkStealingHardSpins >> WorkStealingSpinToYieldRatio
and that value was captured in a variable for (hard_spin_start) for
use in reinitializing the limit after each yield iteration.

So this is a change, and I'm guessing not an intentional one.

------------------------------------------------------------------------------ 
src/hotspot/share/gc/shared/taskTerminator.cpp 
 154   SpinContext spin_context;

If this were moved inside the outer loop of offer_termination, there
would be no need for SpinContext::reset; just use the ctor-initializer
for initialization of the members.

------------------------------------------------------------------------------ 
src/hotspot/share/gc/shared/taskTerminator.cpp 

I *think* the inner loop of offer_termination would be made clearer by
hoisting the yield_count check out of do_spin_iteration.  I'm thinking
something like

instead of

 169       bool giveup_spin;
 170       do {
 171         size_t tasks;
 172         bool should_exit_termination;
 173         {
 174           MutexUnlocker y(_blocker, Mutex::_no_safepoint_check_flag);
 175           giveup_spin = do_spin_iteration(spin_context);
 176 
 177           // Dirty read of exit condition. 
 178           tasks = tasks_in_queue_set();
 179           should_exit_termination = exit_termination(tasks, terminator);
 180         }
 181 
 182         // Immediately check exit conditions after re-acquiring the lock using the
 183         // information gathered just recently.
 184         if (_offered_termination == _n_threads) {
 185           prepare_for_return(the_thread);
 186           assert_queue_set_empty();
 187           return true;
 188         } else if (should_exit_termination) {
 189           prepare_for_return(the_thread, tasks);
 190           _offered_termination--;
 191           return false;
 192         }
 193       } while (!giveup_spin);

something like

  while (spin_context.yield_count <= WorkStealingYieldsBeforeSleep) {
    // Each spin iteration is counted as a yield for purposes of
    // deciding when to sleep.
    ++spin_context.yield_count;
    size_t tasks;
    bool should_exit_termination;
    {
      MutexUnlocker y(_blocker ...);
      do_spin_iteration(spin_context);
      // Dirty read of exit condition.
      tasks = tasks_in_queue_set();
      should_exit_termination = exit_termination(tasks, terminator);
    }
    // Immediately check exit conditions after re-acquiring the lock.
    if (_offered_termination == _n_threads) {
      ... ;
      return true;
    } else if (should_exit_termination) {
      ... ;
    }
  }

This also makes some issues with naming a little more apparent.
do_spin_iteration might spin or might yield, yield_count is counting
both spins and yields, and spin_context is about more than spinning.
Maybe something a little more generic, using "delay" rather than
"spin", like do_delay_step, delay_count, and delay_context?

[pre-existing]
Side question: Should the yield_count test be <= or < ? It seems like
WorkStealingYieldsBeforeSleep == 0 should disable yields and just go
straight to the sleep case. And that's desirable for a uniprocessor,
though why we'd be using a GC using this on a uniprocessor system is
a question.  Exclusive test would also better match the doc string for
the option.  

------------------------------------------------------------------------------
src/hotspot/share/gc/shared/taskTerminator.cpp 
 177           // Dirty read of exit condition. 
 178           tasks = tasks_in_queue_set();

From comment in the bug, that's intentionally done while not holding
the lock, because doing otherwise makes the locked section too long.
There should be a comment about that so nobody is tempted to move it
(as I did in an earlier version of the above suggested move of the
yield_count checking).

------------------------------------------------------------------------------ 
src/hotspot/share/gc/shared/taskTerminator.cpp 

How bad would it be to not bother collecting the number of tasks and 
just have prepare_for_return unconditionally notify_all? 

Alternatively, if that's bad and the cost of tasks_in_queue_set() is
also a problem (which I can well believe), maybe the taskqueue
facility can have some specialized helper function added for use here.
All we really want to know in the current code is how many tasks are
available up to the number of workers, or more than the number of
workers and we don't need a precise answer.

Such a change can be dealt with later.

Another idea would be to record the task count in the terminator,
along with a last update epoch counter.  (Both are only touched under
the lock, so no lock-free synchronization to worry about.)  Use the
recorded count if it's epoch is more recent than the locally captured
epoch from the most recent local use.

Of course, this is being called after some kind of delay (spin, yield,
sleep), so it's not clear how important the performance is.

------------------------------------------------------------------------------
src/hotspot/share/gc/shared/taskTerminator.cpp 
 204     } else {
 205       size_t tasks = tasks_in_queue_set();
 206       if (exit_termination(tasks, terminator)) {
 207         prepare_for_return(the_thread, tasks);
 208         _offered_termination--;
 209         return false;
 210       }
 211     }

We get here only after our wait on the monitor ended.

If we were woken up by some other thread (so wait returned true), I
think we should skip the exit_termination test (just assume true) and
pass 0 to prepare_for_return. (So no need for task counting in this
case.) So insert something like this before the above

  } else if (!timedout) {
    // Don't bother waking up more tasks.
    prepare_for_return(the_thread, 0);
    _offered_termination--;
    return false;
  } else {
    ...

I think this also fixes a (pre-existing, I think) bug. Assume there
are enough tasks for exiting being called for, but fewer than the
number of threads waiting. Either the spin master or the first timeout
will wake up some, but not all, of the waiting threads. Each of those
threads (or the spin master) may wake up additional threads, making
the the limit on wakeups based on number of tasks pointless, and also
making the task counting of much more limited value; given that
exit_termination just checks tasks > 0, we could instead just check
whether the queue_set is empty.  So I think without this we just have
a slower version of always waking up all waiting threads.

(There is also a small improvement to TaskQueueSuper::is_empty() that
I didn't bother making because it didn't seem to matter.)

------------------------------------------------------------------------------
src/hotspot/share/gc/shared/taskTerminator.cpp
 156   MutexLocker x(_blocker, Mutex::_no_safepoint_check_flag);
...
 174           MutexUnlocker y(_blocker, Mutex::_no_safepoint_check_flag);
...
 197     _blocker->wait_without_safepoint_check(WorkStealingSleepMillis);

I agree with Zhengyu's suggestion of using MonitorLocker and waiting
on the locker, rather than using naked _blocker->wait_xxx.

You replied:
> I plan to drop the for-Java Monitor and use os::PlatformMonitor
> instead here. That removes all _no_safepoint_check_flag code.

I don't think that should be done. PlatformMutex/Monitor are intended
intended for use in the implementation of Monitor/Mutex, as an
abstraction over the actual platform facilities. The "usual"
facilities should not be casually bypassed. (I am aware there are a
few direct uses of the Platform layer. Some of those might be simple
bugs. Some may be doing somewhat weird things that have difficulties
with the usual facilities, particularly with rank nesting checks,
which might arguably be considered an overly convoluted code bug.)

> Also there is no MonitorUnlocker, so we'd still have to use
> MutexUnlocker. It will look a bit weird to not have matching
> locker/unlocker names.

Given that Mutex is a conceptual (and now for real) base class of
Monitor, I don't see anything wrong or strange with the unlocker
support being only at the Mutex layer. Literally duplicating it to
have a similar name at the Monitor layer would seem strange to me. If
it really makes you feel better, have a (nearby? or shared?) typedef
for MonitorLocker.

------------------------------------------------------------------------------



From vladimir.kozlov at oracle.com  Wed Jul  8 18:34:08 2020
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Wed, 8 Jul 2020 11:34:08 -0700
Subject: RFR(S) [15] : 8249000 : vm.gc.X should take selected JIT into
 account
In-Reply-To: <7A1992A7-1493-4DF0-B621-195CE986D34F@oracle.com>
References: <FFC49DC0-AA15-45B5-B285-E043D1AEAF97@oracle.com>
 <6964ac32-e9ec-d700-0bdb-ea51f4610afe@oracle.com>
 <7A1992A7-1493-4DF0-B621-195CE986D34F@oracle.com>
Message-ID: <2c92a9a5-77af-c100-fa9b-f765e9d23dce@oracle.com>

Thank you, Igor

I got the difference between `vm.gc` and `vm.gc.X`.

In this case TestReclaimStringsLeaksMemory.java should be put into ProblemList-graal.txt with 8207267 to enable it with 
libgraal.  Current usage of !vm.graal.enabled in test is to skip this test with Java Graal because its effect on Java heap.

On 7/7/20 8:30 PM, Igor Ignatyev wrote:
> Hi Vladimir,
> 
> thanks for your review!
> 
> `vm.gc` and `vm.gc.X`-s are different beasts (and admittedly, they confuse people a lot), `vm.gc` is set to "X", by jtreg itself, only if there is UseXGC in vm flags, otherwise it's "null". `vm.gc.X` are set by VMProps class, and you can have more than one vm.gc.X == true, as vm.gc.X means that X gc is supported by JVM and it can be selected; so if there are no Use.*GC in vm flags, vm.gc.X will yield true for all GCs which JVM was built with; if one of UseXGC is provided, only corresponding vm.gc.X is true, and all others are false. so to answer your questions, yes `vm.gc` can be "null" (if there are no Use.*GC) , and yes `vm.gc.Z & vm.gc.Serial & vm.gc == null` can be true (if there are no Use.*GC and JVM supports both Z and Serial GCs).

Interesting. I thought vmGC will list only one selected GC. That explains requires in TestZGCWithCDS.java.

You only need to add TestReclaimStringsLeaksMemory.java into ProblemList-graal.txt.

Thanks,
Vladimir

> 
> Thanks,
> -- Igor
> 
> 
>> On Jul 7, 2020, at 8:00 PM, Vladimir Kozlov <vladimir.kozlov at oracle.com> wrote:
>>
>> Nice clean up, Igor
>>
>> test/hotspot/jtreg/gc/stress/TestReclaimStringsLeaksMemory.java
>>
>> Do we even can have vm.gc=="null" based on code in VMProps.java? At least some GC should be selected ergonomically even if non is specified on command line.
>>
>> - * @requires vm.gc=="null" & !vm.graal.enabled & !vm.debug
>> + * @requires vm.gc == "null"
>> + * @requires !vm.debug
> 
>>
>>
>> test/hotspot/jtreg/runtime/cds/appcds/TestZGCWithCDS.java
>>
>> Does next combination of @requires ever work? I thought such sequence means 'AND' operation on all such conditions.
>>
>>   * @requires vm.gc.Z
>>   * @requires vm.gc.Serial
>>   * @requires vm.gc == null
>>
>>
>> Thanks,
>> Vladimir
>>
>> On 7/7/20 5:38 PM, Igor Ignatyev wrote:
>>> http://cr.openjdk.java.net/~iignatyev/8249000/webrev.00/
>>>> 241 lines changed: 34 ins; 5 del; 202 mod;
>>> Hi all,
>>> could you please review the patch which modifies requires/VMProps to set vm.gc.X to false if Graal is selected and X GC isn't supported by Graal?
>>> the patch also replaces @requires similar to `vm.gc.X & !vm.graal.enabled` w/ `vm.gc.X` where it's applicable.
>>> from JBS:
>>>> not all GCs are supported by Graal JIT, which leads to failures like JDK-8247527 and boilerplate fixes like replacing all `@requires vm.gc.Z` w/ `@requires vm.gc.Z & !vm.graal.enabled`.
>>>>
>>>> as vm.gc.X means that X GC can be selected, it would be more natural, less surprising, and much more clear to have it true if the selected JIT supports the said X GC.
>>> webrev: http://cr.openjdk.java.net/~iignatyev/8249000/webrev.00/
>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8249000
>>> testing: test/hotspot/jtreg/{gc,compiler,runtime,serviceability} on {linux,windows,macos}-x64 w/ and w/o Graal as JIT
>>> Thanks,
>>> -- Igor
> 


From igor.ignatyev at oracle.com  Wed Jul  8 18:40:14 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Wed, 8 Jul 2020 11:40:14 -0700
Subject: RFR(S) [15] : 8249000 : vm.gc.X should take selected JIT into
 account
In-Reply-To: <2c92a9a5-77af-c100-fa9b-f765e9d23dce@oracle.com>
References: <FFC49DC0-AA15-45B5-B285-E043D1AEAF97@oracle.com>
 <6964ac32-e9ec-d700-0bdb-ea51f4610afe@oracle.com>
 <7A1992A7-1493-4DF0-B621-195CE986D34F@oracle.com>
 <2c92a9a5-77af-c100-fa9b-f765e9d23dce@oracle.com>
Message-ID: <BA1B1E12-19E3-45E3-8FDD-0307DDADDC3E@oracle.com>

Thanks Vladimir. 

for the record, I've updated ProblemList-graal.txt w/ the following:

> diff -r 14ffd658a23a test/hotspot/jtreg/ProblemList-graal.txt
> --- a/test/hotspot/jtreg/ProblemList-graal.txt  Wed Jul 08 11:35:30 2020 -0700
> +++ b/test/hotspot/jtreg/ProblemList-graal.txt  Wed Jul 08 11:37:44 2020 -0700
> @@ -229,6 +229,7 @@
>  compiler/loopopts/TestOverunrolling.java                8207267   generic-all
>  compiler/jsr292/NonInlinedCall/InvokeTest.java          8207267   generic-all
>  compiler/codegen/TestTrichotomyExpressions.java         8207267   generic-all
> +gc/stress/TestReclaimStringsLeaksMemory.java            8207267   generic-all
>  
>  runtime/exceptionMsgs/AbstractMethodError/AbstractMethodErrorTest.java        8222582 generic-all

-- Igor


> On Jul 8, 2020, at 11:34 AM, Vladimir Kozlov <vladimir.kozlov at oracle.com> wrote:
> 
> Thank you, Igor
> 
> I got the difference between `vm.gc` and `vm.gc.X`.
> 
> In this case TestReclaimStringsLeaksMemory.java should be put into ProblemList-graal.txt with 8207267 to enable it with libgraal.  Current usage of !vm.graal.enabled in test is to skip this test with Java Graal because its effect on Java heap.
> 
> On 7/7/20 8:30 PM, Igor Ignatyev wrote:
>> Hi Vladimir,
>> thanks for your review!
>> `vm.gc` and `vm.gc.X`-s are different beasts (and admittedly, they confuse people a lot), `vm.gc` is set to "X", by jtreg itself, only if there is UseXGC in vm flags, otherwise it's "null". `vm.gc.X` are set by VMProps class, and you can have more than one vm.gc.X == true, as vm.gc.X means that X gc is supported by JVM and it can be selected; so if there are no Use.*GC in vm flags, vm.gc.X will yield true for all GCs which JVM was built with; if one of UseXGC is provided, only corresponding vm.gc.X is true, and all others are false. so to answer your questions, yes `vm.gc` can be "null" (if there are no Use.*GC) , and yes `vm.gc.Z & vm.gc.Serial & vm.gc == null` can be true (if there are no Use.*GC and JVM supports both Z and Serial GCs).
> 
> Interesting. I thought vmGC will list only one selected GC. That explains requires in TestZGCWithCDS.java.
> 
> You only need to add TestReclaimStringsLeaksMemory.java into ProblemList-graal.txt.
> 
> Thanks,
> Vladimir
> 
>> Thanks,
>> -- Igor
>>> On Jul 7, 2020, at 8:00 PM, Vladimir Kozlov <vladimir.kozlov at oracle.com> wrote:
>>> 
>>> Nice clean up, Igor
>>> 
>>> test/hotspot/jtreg/gc/stress/TestReclaimStringsLeaksMemory.java
>>> 
>>> Do we even can have vm.gc=="null" based on code in VMProps.java? At least some GC should be selected ergonomically even if non is specified on command line.
>>> 
>>> - * @requires vm.gc=="null" & !vm.graal.enabled & !vm.debug
>>> + * @requires vm.gc == "null"
>>> + * @requires !vm.debug
>>> 
>>> 
>>> test/hotspot/jtreg/runtime/cds/appcds/TestZGCWithCDS.java
>>> 
>>> Does next combination of @requires ever work? I thought such sequence means 'AND' operation on all such conditions.
>>> 
>>>  * @requires vm.gc.Z
>>>  * @requires vm.gc.Serial
>>>  * @requires vm.gc == null
>>> 
>>> 
>>> Thanks,
>>> Vladimir
>>> 
>>> On 7/7/20 5:38 PM, Igor Ignatyev wrote:
>>>> http://cr.openjdk.java.net/~iignatyev/8249000/webrev.00/
>>>>> 241 lines changed: 34 ins; 5 del; 202 mod;
>>>> Hi all,
>>>> could you please review the patch which modifies requires/VMProps to set vm.gc.X to false if Graal is selected and X GC isn't supported by Graal?
>>>> the patch also replaces @requires similar to `vm.gc.X & !vm.graal.enabled` w/ `vm.gc.X` where it's applicable.
>>>> from JBS:
>>>>> not all GCs are supported by Graal JIT, which leads to failures like JDK-8247527 and boilerplate fixes like replacing all `@requires vm.gc.Z` w/ `@requires vm.gc.Z & !vm.graal.enabled`.
>>>>> 
>>>>> as vm.gc.X means that X GC can be selected, it would be more natural, less surprising, and much more clear to have it true if the selected JIT supports the said X GC.
>>>> webrev: http://cr.openjdk.java.net/~iignatyev/8249000/webrev.00/
>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8249000
>>>> testing: test/hotspot/jtreg/{gc,compiler,runtime,serviceability} on {linux,windows,macos}-x64 w/ and w/o Graal as JIT
>>>> Thanks,
>>>> -- Igor



From vladimir.kozlov at oracle.com  Wed Jul  8 22:36:20 2020
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Wed, 8 Jul 2020 15:36:20 -0700
Subject: RFR(S) [15] : 8249000 : vm.gc.X should take selected JIT into
 account
In-Reply-To: <BA1B1E12-19E3-45E3-8FDD-0307DDADDC3E@oracle.com>
References: <FFC49DC0-AA15-45B5-B285-E043D1AEAF97@oracle.com>
 <6964ac32-e9ec-d700-0bdb-ea51f4610afe@oracle.com>
 <7A1992A7-1493-4DF0-B621-195CE986D34F@oracle.com>
 <2c92a9a5-77af-c100-fa9b-f765e9d23dce@oracle.com>
 <BA1B1E12-19E3-45E3-8FDD-0307DDADDC3E@oracle.com>
Message-ID: <dbf5a546-51e0-400f-94f7-5b528972f8fe@oracle.com>

Good.

Thanks,
Vladimir

On 7/8/20 11:40 AM, Igor Ignatyev wrote:
> Thanks Vladimir.
> 
> for the record, I've updated ProblemList-graal.txt w/ the following:
> 
>> diff -r 14ffd658a23a test/hotspot/jtreg/ProblemList-graal.txt
>> --- a/test/hotspot/jtreg/ProblemList-graal.txt  Wed Jul 08 11:35:30 2020 -0700
>> +++ b/test/hotspot/jtreg/ProblemList-graal.txt  Wed Jul 08 11:37:44 2020 -0700
>> @@ -229,6 +229,7 @@
>>   compiler/loopopts/TestOverunrolling.java                8207267   generic-all
>>   compiler/jsr292/NonInlinedCall/InvokeTest.java          8207267   generic-all
>>   compiler/codegen/TestTrichotomyExpressions.java         8207267   generic-all
>> +gc/stress/TestReclaimStringsLeaksMemory.java            8207267   generic-all
>>   
>>   runtime/exceptionMsgs/AbstractMethodError/AbstractMethodErrorTest.java        8222582 generic-all
> 
> -- Igor
> 
> 
>> On Jul 8, 2020, at 11:34 AM, Vladimir Kozlov <vladimir.kozlov at oracle.com> wrote:
>>
>> Thank you, Igor
>>
>> I got the difference between `vm.gc` and `vm.gc.X`.
>>
>> In this case TestReclaimStringsLeaksMemory.java should be put into ProblemList-graal.txt with 8207267 to enable it with libgraal.  Current usage of !vm.graal.enabled in test is to skip this test with Java Graal because its effect on Java heap.
>>
>> On 7/7/20 8:30 PM, Igor Ignatyev wrote:
>>> Hi Vladimir,
>>> thanks for your review!
>>> `vm.gc` and `vm.gc.X`-s are different beasts (and admittedly, they confuse people a lot), `vm.gc` is set to "X", by jtreg itself, only if there is UseXGC in vm flags, otherwise it's "null". `vm.gc.X` are set by VMProps class, and you can have more than one vm.gc.X == true, as vm.gc.X means that X gc is supported by JVM and it can be selected; so if there are no Use.*GC in vm flags, vm.gc.X will yield true for all GCs which JVM was built with; if one of UseXGC is provided, only corresponding vm.gc.X is true, and all others are false. so to answer your questions, yes `vm.gc` can be "null" (if there are no Use.*GC) , and yes `vm.gc.Z & vm.gc.Serial & vm.gc == null` can be true (if there are no Use.*GC and JVM supports both Z and Serial GCs).
>>
>> Interesting. I thought vmGC will list only one selected GC. That explains requires in TestZGCWithCDS.java.
>>
>> You only need to add TestReclaimStringsLeaksMemory.java into ProblemList-graal.txt.
>>
>> Thanks,
>> Vladimir
>>
>>> Thanks,
>>> -- Igor
>>>> On Jul 7, 2020, at 8:00 PM, Vladimir Kozlov <vladimir.kozlov at oracle.com> wrote:
>>>>
>>>> Nice clean up, Igor
>>>>
>>>> test/hotspot/jtreg/gc/stress/TestReclaimStringsLeaksMemory.java
>>>>
>>>> Do we even can have vm.gc=="null" based on code in VMProps.java? At least some GC should be selected ergonomically even if non is specified on command line.
>>>>
>>>> - * @requires vm.gc=="null" & !vm.graal.enabled & !vm.debug
>>>> + * @requires vm.gc == "null"
>>>> + * @requires !vm.debug
>>>>
>>>>
>>>> test/hotspot/jtreg/runtime/cds/appcds/TestZGCWithCDS.java
>>>>
>>>> Does next combination of @requires ever work? I thought such sequence means 'AND' operation on all such conditions.
>>>>
>>>>   * @requires vm.gc.Z
>>>>   * @requires vm.gc.Serial
>>>>   * @requires vm.gc == null
>>>>
>>>>
>>>> Thanks,
>>>> Vladimir
>>>>
>>>> On 7/7/20 5:38 PM, Igor Ignatyev wrote:
>>>>> http://cr.openjdk.java.net/~iignatyev/8249000/webrev.00/
>>>>>> 241 lines changed: 34 ins; 5 del; 202 mod;
>>>>> Hi all,
>>>>> could you please review the patch which modifies requires/VMProps to set vm.gc.X to false if Graal is selected and X GC isn't supported by Graal?
>>>>> the patch also replaces @requires similar to `vm.gc.X & !vm.graal.enabled` w/ `vm.gc.X` where it's applicable.
>>>>> from JBS:
>>>>>> not all GCs are supported by Graal JIT, which leads to failures like JDK-8247527 and boilerplate fixes like replacing all `@requires vm.gc.Z` w/ `@requires vm.gc.Z & !vm.graal.enabled`.
>>>>>>
>>>>>> as vm.gc.X means that X GC can be selected, it would be more natural, less surprising, and much more clear to have it true if the selected JIT supports the said X GC.
>>>>> webrev: http://cr.openjdk.java.net/~iignatyev/8249000/webrev.00/
>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8249000
>>>>> testing: test/hotspot/jtreg/{gc,compiler,runtime,serviceability} on {linux,windows,macos}-x64 w/ and w/o Graal as JIT
>>>>> Thanks,
>>>>> -- Igor
> 


From igor.ignatyev at oracle.com  Thu Jul  9 02:39:48 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Wed, 8 Jul 2020 19:39:48 -0700
Subject: RFR [15] : 8249037 : clean up FileInstaller $test.src $cwd in
 vmTestbase_vm_g1classunloading tests
Message-ID: <01BE191F-3AF8-4D7E-8285-0229FA56ED58@oracle.com>

http://cr.openjdk.java.net/~iignatyev//8249037/webrev.00
> 112 lines changed: 0 ins; 112 del; 0 mod;

Hi all,

could you please review the patch which removes `FileInstaller . .` jtreg action from :vmTestbase_vm_g1classunloading tests?
from the main issue(8204985):
> all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests.

the patch is as simple as `ag -l  '@run driver jdk.test.lib.FileInstaller . .' vmTestbase/gc/g1/unloading/tests | xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}`

testing: :vmTestbase_vm_g1classunloading on linux-x64
webrev: http://cr.openjdk.java.net/~iignatyev//8249037/webrev.00
JBS: https://bugs.openjdk.java.net/browse/JDK-8249037

Thanks,
-- Igor

From shade at redhat.com  Thu Jul  9 06:36:15 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Thu, 9 Jul 2020 08:36:15 +0200
Subject: RFR (S) 8249137: Remove CollectedHeap::obj_size
Message-ID: <a9bd4c4d-2d6b-733e-abbf-90e873c70951@redhat.com>

RFE:
  https://bugs.openjdk.java.net/browse/JDK-8249137

It was added by JDK-8211270 to support old-style Shenandoah that needed a separate fwdptr slot.
After JDK-8224584 it does not need this anymore. Additionally, CH::obj_size may disagree with other
code that pokes at layout helper directly, for example GraphKit::new_instance.

This also avoids a virtual call on some paths, although those paths are not very performance-sensitive.

The patch is a simple series of few-liners:

diff -r 9cc348ebdc82 src/hotspot/share/gc/shared/collectedHeap.cpp
--- a/src/hotspot/share/gc/shared/collectedHeap.cpp     Thu Jul 09 04:32:30 2020 +0200
+++ b/src/hotspot/share/gc/shared/collectedHeap.cpp     Thu Jul 09 08:05:46 2020 +0200
@@ -578,6 +578,2 @@

-size_t CollectedHeap::obj_size(oop obj) const {
-  return obj->size();
-}
-
 uint32_t CollectedHeap::hash_oop(oop obj) const {
diff -r 9cc348ebdc82 src/hotspot/share/gc/shared/collectedHeap.hpp
--- a/src/hotspot/share/gc/shared/collectedHeap.hpp     Thu Jul 09 04:32:30 2020 +0200
+++ b/src/hotspot/share/gc/shared/collectedHeap.hpp     Thu Jul 09 08:05:46 2020 +0200
@@ -495,4 +495,2 @@

-  virtual size_t obj_size(oop obj) const;
-
   // Non product verification and debugging.
diff -r 9cc348ebdc82 src/hotspot/share/prims/jvmtiEnv.cpp
--- a/src/hotspot/share/prims/jvmtiEnv.cpp      Thu Jul 09 04:32:30 2020 +0200
+++ b/src/hotspot/share/prims/jvmtiEnv.cpp      Thu Jul 09 08:05:46 2020 +0200
@@ -488,3 +488,3 @@
   NULL_CHECK(mirror, JVMTI_ERROR_INVALID_OBJECT);
-  *size_ptr = (jlong)Universe::heap()->obj_size(mirror) * wordSize;
+  *size_ptr = (jlong)mirror->size() * wordSize;
   return JVMTI_ERROR_NONE;
diff -r 9cc348ebdc82 src/hotspot/share/prims/jvmtiExport.cpp
--- a/src/hotspot/share/prims/jvmtiExport.cpp   Thu Jul 09 04:32:30 2020 +0200
+++ b/src/hotspot/share/prims/jvmtiExport.cpp   Thu Jul 09 08:05:46 2020 +0200
@@ -1067,3 +1067,3 @@
      _jobj = (jobject)to_jobject(obj);
-     _size = Universe::heap()->obj_size(obj) * wordSize;
+     _size = obj->size() * wordSize;
    };
diff -r 9cc348ebdc82 src/hotspot/share/prims/whitebox.cpp
--- a/src/hotspot/share/prims/whitebox.cpp      Thu Jul 09 04:32:30 2020 +0200
+++ b/src/hotspot/share/prims/whitebox.cpp      Thu Jul 09 08:05:46 2020 +0200
@@ -389,3 +389,3 @@
   oop p = JNIHandles::resolve(obj);
-  return Universe::heap()->obj_size(p) * HeapWordSize;
+  return p->size() * HeapWordSize;
 WB_END

Testing: tier{1,2}; jdk-submit (running)

-- 
Thanks,
-Aleksey


From linzang at tencent.com  Thu Jul  9 07:22:22 2020
From: linzang at tencent.com (=?utf-8?B?bGluemFuZyjoh6fnkLMp?=)
Date: Thu, 9 Jul 2020 07:22:22 +0000
Subject: RFR(L): 8215624: add parallel heap inspection support for jmap
 histo(G1)(Internet mail)
In-Reply-To: <02415944-2238-4430-A7C2-3625264A5505@amazon.com>
References: <02415944-2238-4430-A7C2-3625264A5505@amazon.com>
Message-ID: <C1CE05FA-3E6F-4ECC-8C72-9A4A2B923FB8@tencent.com>

Hi Paul, 
    Thanks for reviewing!
    >> 
    >>     I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp.
    >>
    The reason I made the change in Jmap.java that compose all arguments as 1 string , instead of passing 3 argments, is to avoid the compatibility issue, as we discussed in http://mail.openjdk.java.net/pipermail/serviceability-dev/2019-February/thread.html#27240.  The root cause of the compatibility issue is because max argument count in HotspotVirtualMachineImpl.java and attachlistener.cpp need to be enlarged (changes like http://hg.openjdk.java.net/jdk/jdk/rev/e7cf035682e3#l2.1) when jmap has more than 3 arguments. But if user use an old jcmd/jmap tool, it may stuck at socket read(), because the "max argument count" don't match.  
     I re-checked this change, the argument count of jmap histo is equal to 3 (live, file, parallel), so it can work normally even without the change of passing argument. But I think we have to face the problem if more arguments is added in jcmd alike tools later, not sure whether it should be sloved (or a workaround) in this changeset. 

    And here are the lastest webrev and delta:
    http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06/
    http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06-delta/

Cheers,
Lin

?On 2020/7/7, 5:57 AM, "Hohensee, Paul" <hohensee at amazon.com> wrote:

    I'd like to see this feature added. :)

    The CSR looks good, as does the basic parallel inspection algorithm. Stefan's done the GC part, so I'll stick to the non-GC part (fwiw, the GC part lgtm).

    I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp.

    heapInspection.hpp:

    _shared_miss_count (s/b _missed_count, see below) isn't a size, so it should be a uint instead of a size_t. Same with the new parallel_thread_num argument to heap_inspection() and populate_table().

    Comment copy-edit:
    +// Parallel heap inspection task. Parallel inspection can fail due to
    +// a native OOM when allocating memory for TL-KlassInfoTable.
    +// _success will be set false on an OOM, and serial inspection tried.

    _shared_miss_count should be _missed_count to match the missed_count() getter, or rename missed_count() to be shared_miss_count(). Whichever way you go, the field type should match the getter result type: uint is reasonable.

    heapInspection.cpp:

    You might use ResourceMark twice in populate_table, separately for the parallel attempt and the serial code. If the parallel attempt fails and available memory is low, it would be good to clean up the memory used by the parallel attempt before doing the serial code.

    Style nit in KlassInfoTable::merge_entry(). I'd line up the definitions of k and elt, so "k" is even with "elt". And, because it's two lines shorter, I'd replace
    +  } else {
    +    return false;
    +  }
    with
    +  return false;

    KlassInfoTableMergeClosure.is_success() should be just success() (i.e., no "is_" prefix) because it's a getter.

    I'd reorganize the code in populate_table() to make it more clear, vis (I changed _shared_missed_count to _missed_count)
    +  if (cit.allocation_failed()) {
    +    // fail to allocate memory, stop parallel mode
    +    Atomic::store(&_success, false);
    +    return;
    +  }
    +  RecordInstanceClosure ric(&cit, _filter);
    +  _poi->object_iterate(&ric, worker_id);
    +  missed_count = ric.missed_count();
    +  {
    +    MutexLocker x(&_mutex);
    +    merge_success = _shared_cit->merge(&cit);
    +  }
    +  if (merge_success) {
    +    Atomic::add(&_missed_count, missed_count);
    +  else {
    +    Atomic::store(&_success, false);
    +  }

    Thanks,
    Paul

    On 6/29/20, 7:20 PM, "linzang(??)" <linzang at tencent.com> wrote:

        Dear All,
                Sorry to bother again, I just want to make sure that is this change worth to be continue to work on? If decision is made to not. I think I can drop this work and stop asking for help reviewing...
                Thanks for all your help about reviewing this previously.

        BRs,
        Lin

        On 2020/5/9, 3:47 PM, "linzang(??)" <linzang at tencent.com> wrote:

            Dear All,
                   May I ask your help again for review the latest change?  Thanks!

            BRs,
            Lin

            On 2020/4/28, 1:54 PM, "linzang(??)" <linzang at tencent.com> wrote:

                Hi Stefan,
                  >>  - Adding Atomic::load/store.
                  >>  - Removing the time measurement in the run_task. I renamed G1's function
                  >>  to run_task_timed. If we need this outside of G1, we can rethink the API
                  >>  at that point.
                   >>  - ZGC style cleanups
                   Thanks for revising the patch,  they are all good to me, and I have made a tiny change based on it:
                       http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04/
                       http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04-delta/
                  it reduce the scope of mutex in ParHeapInspectTask, and delete unnecessary comments.

                BRs,
                Lin

                On 2020/4/27, 4:34 PM, "Stefan Karlsson" <stefan.karlsson at oracle.com> wrote:

                    Hi Lin,

                    On 2020-04-26 05:10, linzang(??) wrote:
                    > Hi Stefan and Paul?
                    >      I have made a new patch based on your comments and Stefan's Poc code:
                    >      Webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03/
                    >      Delta(based on Stefan's change:) : http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03-delta/webrev_03-delta/

                    Thanks for providing a delta patch. It makes it much easier to look at,
                    and more likely for reviewers to continue reviewing.

                    I'm going to continue focusing on the GC parts, and leave the rest to
                    others to review.

                    >
                    >      And Here are main changed I made and want to discuss with you:
                    >      1.  changed"parallelThreadNum=" to "parallel=" for jmap -histo options.
                    >      2.  Add logic to test where parallelHeapInspection is fail, in heapInspection.cpp
                    >            This is because the parHeapInspectTask create thread local KlassInfoTable in it's work() method, and this may fail because of native OOM, in this case, the parallel should fail and serial heap inspection can be tried.
                    >            One more thing I want discuss with you is about the member "_success" of parHeapInspectTask, when native OOM happenes, it is set to false. And since this "set" operation can be conducted in multiple threads, should it be atomic ops?  IMO, this is not necessary because "_success" can only be set to false, and there is no way to change it from back to true after the ParHeapInspectTask instance is created, so it is save to be non-atomic, do you agree with that?

                    In these situations you should be using the Atomic::load/store
                    primitives. We're moving toward a later C++ standard were data races are
                    considered undefined behavior.

                    >     3. make CollectedHeap::run_task() be an abstract virtual func, so that every subclass of collectedHeap should support it, so later implementation of new collectedHeap will not miss the "parallel" features.
                    >           The problem I want to discuss with you is about epsilonHeap and SerialHeap, as they may not need parallel heap iteration, so I only make task->work(0), in case the run_task() is invoked someway in future. Another way is to left run_task()  unimplemented, which one do you think is better?

                    I don't have a strong opinion about this.

                      And also please help take a look at the zHeap, as there is a class
                    zTask that wrap the abstractGangTask, and the collectedHeap::run_task()
                    only accept  AbstraceGangTask* as argument, so I made a delegate class
                    to adapt it , please see src/hotspot/share/gc/z/zHeap.cpp.
                    >
                    >        There maybe other better ways to sovle the above problems, welcome for any comments, Thanks!

                    I've created a few cleanups and changes on top of your latest patch:

                    https://cr.openjdk.java.net/~stefank/8215624/webrev.02.delta
                    https://cr.openjdk.java.net/~stefank/8215624/webrev.02

                    - Adding Atomic::load/store.
                    - Removing the time measurement in the run_task. I renamed G1's function
                    to run_task_timed. If we need this outside of G1, we can rethink the API
                    at that point.
                    - ZGC style cleanups

                    Thanks,
                    StefanK

                    >
                    > BRs,
                    > Lin
                    >
                    > On 2020/4/23, 11:08 AM, "linzang(??)" <linzang at tencent.com> wrote:
                    >
                    >      Thanks Paul! I agree with using "parallel", will make the update in next patch, Thanks for help update the CSR.
                    >
                    >      BRs,
                    >      Lin
                    >
                    >      On 2020/4/23, 4:42 AM, "Hohensee, Paul" <hohensee at amazon.com> wrote:
                    >
                    >          For the interface, I'd use "parallel" instead of "parallelThreadNum". All the other options are lower case, and it's a lot easier to type "parallel". I took the liberty of updating the CSR. If you're ok with it, you might want to change variable names and such, plus of course JMap.usage.
                    >
                    >          Thanks,
                    >          Paul
                    >
                    >          On 4/22/20, 2:29 AM, "serviceability-dev on behalf of linzang(??)" <serviceability-dev-bounces at openjdk.java.net on behalf of linzang at tencent.com> wrote:
                    >
                    >              Dear Stefan,
                    >
                    >                      Thanks a lot! I agree with you to decouple the heap inspection code with GC's.
                    >                      I will start  from your POC code, may discuss with you later.
                    >
                    >
                    >              BRs,
                    >              Lin
                    >
                    >              On 2020/4/22, 5:14 PM, "Stefan Karlsson" <stefan.karlsson at oracle.com> wrote:
                    >
                    >                  Hi Lin,
                    >
                    >                  I took a look at this earlier and saw that the heap inspection code is
                    >                  strongly coupled with the CollectedHeap and G1CollectedHeap. I'd prefer
                    >                  if we'd abstract this away, so that the GCs only provide a "parallel
                    >                  object iteration" interface, and the heap inspection code is kept elsewhere.
                    >
                    >                  I started experimenting with doing that, but other higher-priority (to
                    >                  me) tasks have had to take precedence.
                    >
                    >                  I've uploaded my work-in-progress / proof-of-concept:
                    >                    https://cr.openjdk.java.net/~stefank/8215624/webrev.01.delta/
                    >                    https://cr.openjdk.java.net/~stefank/8215624/webrev.01/
                    >
                    >                  The current code doesn't handle the lifecycle (deletion) of the
                    >                  ParallelObjectIterators. There's also code left unimplemented in around
                    >                  CollectedHeap::run_task. However, I think this could work as a basis to
                    >                  pull out the heap inspection code out of the GCs.
                    >
                    >                  Thanks,
                    >                  StefanK
                    >
                    >                  On 2020-04-22 02:21, linzang(??) wrote:
                    >                  > Dear all,
                    >                  >       May I ask you help to review? This RFR has been there for quite a while.
                    >                  >       Thanks!
                    >                  >
                    >                  > BRs,
                    >                  > Lin
                    >                  >
                    >                  > > On 2020/3/16, 5:18 PM, "linzang(??)" <linzang at tencent.com> wrote:>
                    >                  >
                    >                  >>    Just update a new path, my preliminary measure show about 3.5x speedup of jmap histo on a nearly full 4GB G1 heap (8-core platform with parallel thread number set to 4).
                    >                  >>     webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_02/
                    >                  >>     bug: https://bugs.openjdk.java.net/browse/JDK-8215624
                    >                  >>     CSR: https://bugs.openjdk.java.net/browse/JDK-8239290
                    >                  >>     BRs,
                    >                  >>       Lin
                    >                  >>       > On 2020/3/2, 9:56 PM, "linzang(??)" <linzang at tencent.com> wrote:
                    >                  >>       >
                    >                  >>       >    Dear all,
                    >                  >>       >          Let me try to ease the reviewing work by some explanation :P
                    >                  >>       >          The patch's target is to speed up jmap -histo for heap iteration, from my experience it is necessary for large heap investigation. E.g in bigData scenario I have tried to conduct jmap -histo against 180GB heap, it does take quite a while.
                    >                  >>       >          And if my understanding is corrent, even the jmap -histo without "live" option does heap inspection with heap lock acquired. so it is very likely to block mutator thread in allocation-sensitive scenario. I would say the faster the heap inspection does, the shorter the mutator be blocked. This is parallel iteration for jmap is necessary.
                    >                  >>       >          I think the parallel heap inspection should be applied to all kind of heap. However, consider the heap layout are different for  GCs, much time is required to understand all kinds of the heap layout to make the whole change. IMO, It is not wise to have a huge patch for the whole solution at once, and it is even harder to review it. So I plan to implement it incrementally, the first patch (this one) is going to confirm the implemention detail of how jmap accept the new option, passes it to attachListener of the jvm process and then how to make the parallel inspection closure be generic enough to make it easy to extend to different heap layout. And also how to implement the heap inspection in specific gc's heap. This patch use G1's heap as the begining.
                    >                  >>       >          This patch actually do several things:
                    >                  >>       >          1. Add an option "parallelThreadNum=<N>" to jmap -histo, the default behavior is to set N to 0, means let's JVM decide how many threads to use for heap inspection. Set this option to 1 will disable parallel heap inspection. (more details in CSR: https://bugs.openjdk.java.net/browse/JDK-8239290)
                    >                  >>       >          2. Make a change in how Jmap passing arguments, changes in http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java.udiff.html, originally it pass options as separate arguments to attachListener, this patch change to that all options be compose to a single string. So the arg_count_max in attachListener.hpp do not need to be changed, and hence avoid the compatibility issue, as disscussed at https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-March/027334.html
                    >                  >>       >         3. Add an abstract class ParHeapInspectTask in heapInspection.hpp / heapInspection.cpp, It's work(uint worker_id) method prepares the data structure (KlassInfoTable) need for every parallel worker thread, and then call do_object_iterate_parallel() which is heap specific implementation. I also added some machenism in KlassInfoTable to support parallel iteration, such as merge().
                    >                  >>       >        4. In specific heap (G1 in this patch), create a subclass of ParHeapInspectTask, implement the do_object_iterate_parallel() for parallel heap inspection. For G1, it simply invoke g1CollectedHeap's object_iterate_parallel().
                    >                  >>       >        5. Add related test.
                    >                  >>       >        6. it may be easy to extend this patch for other kinds of heap by creating subclass of ParHeapInspectTask and implement the do_object_iterate_parallel().
                    >                  >>       >
                    >                  >>       >    Hope these info could help on code review and initate the discussion :-)
                    >                  >>       >    Thanks!
                    >                  >>       >
                    >                  >>       >    BRs,
                    >                  >>       >    Lin
                    >                  >>       >    >On 2020/2/19, 9:40 AM, "linzang(??)" <linzang at tencent.com> wrote:.
                    >                  >>       >    >
                    >                  >>       >    >  Re-post this RFR with correct enhancement number to make it trackable.
                    >                  >>       >    >  please ignore the previous wrong post. sorry for troubles.
                    >                  >>       >    >
                    >                  >>       >    >   webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/
                    >                  >>       >    >    Hi bug: https://bugs.openjdk.java.net/browse/JDK-8215624
                    >                  >>       >    >    CSR: https://bugs.openjdk.java.net/browse/JDK-8239290
                    >                  >>       >    >    --------------
                    >                  >>       >    >    Lin
                    >                  >>       >    >    >Hi Lin,
                    >                  >   >     >    >    >
                    >                  >>       >    >    >Could you, please, re-post your RFR with the right enhancement number in
                    >                  >>       >    >    >the message subject?
                    >                  >>       >    >    >It will be more trackable this way.
                    >                  >>       >    >    >
                    >                  >>       >    >    >Thanks,
                    >                  >>       >    >    >Serguei
                    >                  >>       >    >    >
                    >                  >>       >    >    >
                    >                  >>       >    >    >On 2/17/20 10:29 PM, linzang(??) wrote:
                    >                  >>       >    >    >> Dear David,
                    >                  >>       >    >    >>        Thanks a lot!
                    >                  >>       >    >    >>       I have updated the refined code to http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_01/.
                    >                  >>       >    >    >>        IMHO the parallel heap inspection can be extended to all kinds of heap as long as the heap layout can support parallel iteration.
                    >                  >>       >    >    >>        Maybe we can firstly use this webrev to discuss how to implement it, because I am not sure my current implementation is an appropriate way to communicate with collectedHeap, then we can extend the solution to other kinds of heap.
                    >                  >>       >    >    >>
                    >                  >>       >    >    >> Thanks,
                    >                  >>       >    >    >> --------------
                    >                  >>       >    >    >> Lin
                    >                  >>       >    >    >>> Hi Lin,
                    >                  >>       >    >    >>>
                    >                  >>       >    >    >>> Adding in hotspot-gc-dev as they need to see how this interacts with GC
                    >                  >>       >    >    >>> worker threads, and whether it needs to be extended beyond G1.
                    >                  >>       >    >    >>>
                    >                  >>       >    >   >>> I happened to spot one nit when browsing:
                    >                  >>       >    >    >>>
                    >                  >>       >    >    >>> src/hotspot/share/gc/shared/collectedHeap.hpp
                    >                  >>       >    >    >>>
                    >                  >>       >    >    >>> +   virtual bool run_par_heap_inspect_task(KlassInfoTable* cit,
                    >                  >>       >    >    >>> +                                          BoolObjectClosure* filter,
                    >                  >>       >    >    >>> +                                          size_t* missed_count,
                    >                  >>       >    >    >>> +                                          size_t thread_num) {
                    >                  >>       >    >    >>> +     return NULL;
                    >                  >>       >    >    >>>
                    >                  >>       >    >    >>> s/NULL/false/
                    >                  >>       >    >    >>>
                    >                  >>       >    >    >>> Cheers,
                    >                  >>       >    >    >>> David
                    >                  >   >     >    >    >>>
                    >                  >>       >    >    >>> On 18/02/2020 2:15 pm, linzang(??) wrote:
                    >                  >>       >    >    >>>> Dear All,
                    >                  >>       >    >    >>>>         May I ask your help to review the follow changes:
                    >                  >>       >    >    >>>>         webrev:
                    >                  >>       >    >    >>>> http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_00/
                    >                  >>       >    >    >>>>      bug: https://bugs.openjdk.java.net/browse/JDK-8215624
                    >                  >>       >    >    >>>>      related CSR: https://bugs.openjdk.java.net/browse/JDK-8239290
                    >                  >>       >    >    >>>>         This patch enable parallel heap inspection of G1 for jmap histo.
                    >                  >>       >    >    >>>>         my simple test shown it can speed up 2x of jmap -histo with
                    >                  >>       >    >    >>>> parallelThreadNum set to 2 for heap at ~500M on 4-core platform.
                    >                  >>       >    >    >>>>
                    >                  >>       >    >    >>>> ------------------------------------------------------------------------
                    >                  >>       >    >    >>>> BRs,
                    >                  >>       >    >    >>>> Lin
                    >                  >>       >    >    >> >
                    >                  >>       >    >    >
                    >                  >
                    >                  >
                    >                  >
                    >
                    >
                    >
                    >
                    >
                    >







From linzang at tencent.com  Thu Jul  9 07:38:14 2020
From: linzang at tencent.com (=?utf-8?B?bGluemFuZyjoh6fnkLMp?=)
Date: Thu, 9 Jul 2020 07:38:14 +0000
Subject: RFR (S) 8249140: fail to build hotspot with gcc-4.8.5
Message-ID: <660DFF34-DF95-417B-A3CF-D8A6F0AC8B08@tencent.com>

Hi
                May I ask your help to review the fix of 8249140, which cause hotspot build fail with gcc-4.8.5
              Bug: https://bugs.openjdk.java.net/browse/JDK-8249140
                Webrev: http://cr.openjdk.java.net/~lzang/8249140/webrev/
                Thanks!

BRs,
Lin

From rkennke at redhat.com  Thu Jul  9 08:55:04 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Thu, 09 Jul 2020 10:55:04 +0200
Subject: RFR (S) 8249137: Remove CollectedHeap::obj_size
In-Reply-To: <a9bd4c4d-2d6b-733e-abbf-90e873c70951@redhat.com>
References: <a9bd4c4d-2d6b-733e-abbf-90e873c70951@redhat.com>
Message-ID: <b544098a92fbdcaba1273860bccc76f4269768f3.camel@redhat.com>

Looks good to me!

Thanks,
Roman

> > 
> RFE:
>   https://bugs.openjdk.java.net/browse/JDK-8249137
> 
> It was added by JDK-8211270 to support old-style Shenandoah that
> needed a separate fwdptr slot.
> After JDK-8224584 it does not need this anymore. Additionally,
> CH::obj_size may disagree with other
> code that pokes at layout helper directly, for example
> GraphKit::new_instance.
> 
> This also avoids a virtual call on some paths, although those paths
> are not very performance-sensitive.
> 
> The patch is a simple series of few-liners:
> 
> diff -r 9cc348ebdc82 src/hotspot/share/gc/shared/collectedHeap.cpp
> --- a/src/hotspot/share/gc/shared/collectedHeap.cpp     Thu Jul 09
> 04:32:30 2020 +0200
> +++ b/src/hotspot/share/gc/shared/collectedHeap.cpp     Thu Jul 09
> 08:05:46 2020 +0200
> @@ -578,6 +578,2 @@
> 
> -size_t CollectedHeap::obj_size(oop obj) const {
> -  return obj->size();
> -}
> -
>  uint32_t CollectedHeap::hash_oop(oop obj) const {
> diff -r 9cc348ebdc82 src/hotspot/share/gc/shared/collectedHeap.hpp
> --- a/src/hotspot/share/gc/shared/collectedHeap.hpp     Thu Jul 09
> 04:32:30 2020 +0200
> +++ b/src/hotspot/share/gc/shared/collectedHeap.hpp     Thu Jul 09
> 08:05:46 2020 +0200
> @@ -495,4 +495,2 @@
> 
> -  virtual size_t obj_size(oop obj) const;
> -
>    // Non product verification and debugging.
> diff -r 9cc348ebdc82 src/hotspot/share/prims/jvmtiEnv.cpp
> --- a/src/hotspot/share/prims/jvmtiEnv.cpp      Thu Jul 09 04:32:30
> 2020 +0200
> +++ b/src/hotspot/share/prims/jvmtiEnv.cpp      Thu Jul 09 08:05:46
> 2020 +0200
> @@ -488,3 +488,3 @@
>    NULL_CHECK(mirror, JVMTI_ERROR_INVALID_OBJECT);
> -  *size_ptr = (jlong)Universe::heap()->obj_size(mirror) * wordSize;
> +  *size_ptr = (jlong)mirror->size() * wordSize;
>    return JVMTI_ERROR_NONE;
> diff -r 9cc348ebdc82 src/hotspot/share/prims/jvmtiExport.cpp
> --- a/src/hotspot/share/prims/jvmtiExport.cpp   Thu Jul 09 04:32:30
> 2020 +0200
> +++ b/src/hotspot/share/prims/jvmtiExport.cpp   Thu Jul 09 08:05:46
> 2020 +0200
> @@ -1067,3 +1067,3 @@
>       _jobj = (jobject)to_jobject(obj);
> -     _size = Universe::heap()->obj_size(obj) * wordSize;
> +     _size = obj->size() * wordSize;
>     };
> diff -r 9cc348ebdc82 src/hotspot/share/prims/whitebox.cpp
> --- a/src/hotspot/share/prims/whitebox.cpp      Thu Jul 09 04:32:30
> 2020 +0200
> +++ b/src/hotspot/share/prims/whitebox.cpp      Thu Jul 09 08:05:46
> 2020 +0200
> @@ -389,3 +389,3 @@
>    oop p = JNIHandles::resolve(obj);
> -  return Universe::heap()->obj_size(p) * HeapWordSize;
> +  return p->size() * HeapWordSize;
>  WB_END
> 
> Testing: tier{1,2}; jdk-submit (running)
> 



From thomas.schatzl at oracle.com  Thu Jul  9 09:00:36 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 9 Jul 2020 11:00:36 +0200
Subject: RFR (S) 8249137: Remove CollectedHeap::obj_size
In-Reply-To: <a9bd4c4d-2d6b-733e-abbf-90e873c70951@redhat.com>
References: <a9bd4c4d-2d6b-733e-abbf-90e873c70951@redhat.com>
Message-ID: <1bc166b2-fce3-c9dc-5d3e-dfb9d09245e4@oracle.com>

Hi,

On 09.07.20 08:36, Aleksey Shipilev wrote:
> RFE:
>    https://bugs.openjdk.java.net/browse/JDK-8249137
> 
> It was added by JDK-8211270 to support old-style Shenandoah that needed a separate fwdptr slot.
> After JDK-8224584 it does not need this anymore. Additionally, CH::obj_size may disagree with other
> code that pokes at layout helper directly, for example GraphKit::new_instance.
> 
> This also avoids a virtual call on some paths, although those paths are not very performance-sensitive.
> 
> The patch is a simple series of few-liners:
> 

   looks good.

Thomas


From leo.korinth at oracle.com  Thu Jul  9 11:46:14 2020
From: leo.korinth at oracle.com (Leo Korinth)
Date: Thu, 9 Jul 2020 13:46:14 +0200
Subject: RFR (M): 8210462: Fix remaining mentions of initial mark
In-Reply-To: <0AFEB8CC-039E-4402-838A-D88657B05D62@oracle.com>
References: <bc347a79-63d4-e259-3e57-53ddb4a00be1@oracle.com>
 <6f950a83-6942-b4e9-fb49-5ee20635add6@oracle.com>
 <a5411b45-4a50-9a0d-ef7d-7dc45d603d22@oracle.com>
 <3D90704F-D7BC-4E24-9783-76993989602C@oracle.com>
 <8edfe69d-822d-59c7-5130-5ab0b2fc2d1c@oracle.com>
 <0AFEB8CC-039E-4402-838A-D88657B05D62@oracle.com>
Message-ID: <3046898b-39c2-000d-f1c6-c6fae69d58ee@oracle.com>



On 07/07/2020 14:38, Kim Barrett wrote:
>> On Jul 7, 2020, at 4:39 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
>>
>> Hi Kim,
>>
>>   thanks for your review.
>>
>> On 03.07.20 22:29, Kim Barrett wrote:
>>> src/hotspot/share/gc/g1/g1Policy.cpp
>>> [removed]
>>>   636   bool this_pause_was_young_only = collector_state()->in_young_only_phase();
>>> Why was this variable removed and uses replaced with
>>> is_young_only_only(this_pause), rather than keeping the variable but
>>> updating the initialization?
>>> ------------------------------------------------------------------------------
>>
>> The reason is that this_pause_was_young_only is a local defined waaay up at the top of that method and I thought instead of referencing that one it is better to do the (very simple) function call.
>> I can change that again if you want.
> 
> OK.  The ?far away? rationale makes sense.
> 
>> Webrevs:
>> http://cr.openjdk.java.net/~tschatzl/8210462/webrev.1_to_2/ (diff)
>> http://cr.openjdk.java.net/~tschatzl/8210462/webrev.2/ (full)
>> Testing:
>> tier1, local gc/g1 jtreg run
> 
> Looks good.
> 

Looks good to me too.

Thanks,
Leo


From thomas.schatzl at oracle.com  Thu Jul  9 12:26:43 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 9 Jul 2020 14:26:43 +0200
Subject: RFR (S): 8248322: G1: Refactor full collection sizing code
In-Reply-To: <B05819C1-E5A9-4D0E-9B72-C34BFA670BFF@oracle.com>
References: <1a156b7e-f5fc-9910-1248-8d60f3cb38a1@oracle.com>
 <B05819C1-E5A9-4D0E-9B72-C34BFA670BFF@oracle.com>
Message-ID: <2a381290-e722-c8cc-d257-8d5a4540b306@oracle.com>

Hi Kim, Ivan,

   thanks for your reviews. I will push with below suggested naming changes.

Thanks,
   Thomas

On 08.07.20 10:45, Ivan Walulya wrote:
> Looks good to me!
> 
> Minor comment.
> 
> +static size_t target_heap_capacity(size_t used_bytes, uintx free_ratio) {
> + const double free_percentage = (double) free_ratio / 100.0;
> + const double used_percentage = 1.0 - free_percentage;
> 
> Clearer if the above variable names are also prefixed with ?desired_?.
> 
> 
> 
>> On 26 Jun 2020, at 16:30, Thomas Schatzl <thomas.schatzl at oracle.com 
>> <mailto:thomas.schatzl at oracle.com>> wrote:
>>
>> Hi all,
>>
>> ?can I have reviews for this small refactoring changes that moves some 
>> heap sizing policy related code into a the "right" place (into the 
>> heap sizing policy class), and refactors it a bit.
>>
>> CR:
>> https://bugs.openjdk.java.net/browse/JDK-8248322
>> Webrev:
>> http://cr.openjdk.java.net/~tschatzl/8248322/webrev/
>> Testing:
>> tier1
>>
>> Thanks,
>> ?Thomas
>>
> 



From leo.korinth at oracle.com  Thu Jul  9 12:38:11 2020
From: leo.korinth at oracle.com (Leo Korinth)
Date: Thu, 9 Jul 2020 14:38:11 +0200
Subject: RFR (M): 8245721: Refactor the TaskTerminator
In-Reply-To: <bd530d10-a4bb-969f-b142-39ff259c91b8@oracle.com>
References: <bd530d10-a4bb-969f-b142-39ff259c91b8@oracle.com>
Message-ID: <d5dda78f-9767-7d38-81d0-f51a7d2bbd48@oracle.com>

I just have two small suggestions that have not been mentioned yet.

- TaskTerminator::yield() should be used or removed.
- Maybe _blocker could be "inlined" in the parent object (not being a pointer) and we would get rid of a delete as well as an assert and also do one allocation less.

Thanks,
Leo


From kim.barrett at oracle.com  Thu Jul  9 12:58:24 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 9 Jul 2020 08:58:24 -0400
Subject: RFR (S) 8249140: fail to build hotspot with gcc-4.8.5
In-Reply-To: <660DFF34-DF95-417B-A3CF-D8A6F0AC8B08@tencent.com>
References: <660DFF34-DF95-417B-A3CF-D8A6F0AC8B08@tencent.com>
Message-ID: <7172888B-C11F-48E9-B9CA-60F52E0208D7@oracle.com>

> On Jul 9, 2020, at 3:38 AM, linzang(??) <linzang at tencent.com> wrote:
> 
> Hi
>                May I ask your help to review the fix of 8249140, which cause hotspot build fail with gcc-4.8.5
>              Bug: https://bugs.openjdk.java.net/browse/JDK-8249140
>                Webrev: http://cr.openjdk.java.net/~lzang/8249140/webrev/
>                Thanks!
> 
> BRs,
> Lin

The warnings appear to be a bug in that (extremely old) compiler
version; the expressions involved clearly are constant expressions.
Also, that compiler version will very shortly (hopefully within a
couple of weeks) not work at all, because it lacks C++14 support.
And the change itself does not make the code better in any other way.

So I think this change should not be made.



From daniel.daugherty at oracle.com  Thu Jul  9 13:34:15 2020
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Thu, 9 Jul 2020 09:34:15 -0400
Subject: RFR (S) 8249137: Remove CollectedHeap::obj_size
In-Reply-To: <a9bd4c4d-2d6b-733e-abbf-90e873c70951@redhat.com>
References: <a9bd4c4d-2d6b-733e-abbf-90e873c70951@redhat.com>
Message-ID: <70984ebf-2717-c7b3-7076-12e2c8c7515c@oracle.com>

Adding serviceability-dev at ... since a couple of JVM/TI files are changed
in this RFR. Also, I moved the bug from hotspot/runtime -> hotspot/gc.

Dan


On 7/9/20 2:36 AM, Aleksey Shipilev wrote:
> RFE:
>    https://bugs.openjdk.java.net/browse/JDK-8249137
>
> It was added by JDK-8211270 to support old-style Shenandoah that needed a separate fwdptr slot.
> After JDK-8224584 it does not need this anymore. Additionally, CH::obj_size may disagree with other
> code that pokes at layout helper directly, for example GraphKit::new_instance.
>
> This also avoids a virtual call on some paths, although those paths are not very performance-sensitive.
>
> The patch is a simple series of few-liners:
>
> diff -r 9cc348ebdc82 src/hotspot/share/gc/shared/collectedHeap.cpp
> --- a/src/hotspot/share/gc/shared/collectedHeap.cpp     Thu Jul 09 04:32:30 2020 +0200
> +++ b/src/hotspot/share/gc/shared/collectedHeap.cpp     Thu Jul 09 08:05:46 2020 +0200
> @@ -578,6 +578,2 @@
>
> -size_t CollectedHeap::obj_size(oop obj) const {
> -  return obj->size();
> -}
> -
>   uint32_t CollectedHeap::hash_oop(oop obj) const {
> diff -r 9cc348ebdc82 src/hotspot/share/gc/shared/collectedHeap.hpp
> --- a/src/hotspot/share/gc/shared/collectedHeap.hpp     Thu Jul 09 04:32:30 2020 +0200
> +++ b/src/hotspot/share/gc/shared/collectedHeap.hpp     Thu Jul 09 08:05:46 2020 +0200
> @@ -495,4 +495,2 @@
>
> -  virtual size_t obj_size(oop obj) const;
> -
>     // Non product verification and debugging.
> diff -r 9cc348ebdc82 src/hotspot/share/prims/jvmtiEnv.cpp
> --- a/src/hotspot/share/prims/jvmtiEnv.cpp      Thu Jul 09 04:32:30 2020 +0200
> +++ b/src/hotspot/share/prims/jvmtiEnv.cpp      Thu Jul 09 08:05:46 2020 +0200
> @@ -488,3 +488,3 @@
>     NULL_CHECK(mirror, JVMTI_ERROR_INVALID_OBJECT);
> -  *size_ptr = (jlong)Universe::heap()->obj_size(mirror) * wordSize;
> +  *size_ptr = (jlong)mirror->size() * wordSize;
>     return JVMTI_ERROR_NONE;
> diff -r 9cc348ebdc82 src/hotspot/share/prims/jvmtiExport.cpp
> --- a/src/hotspot/share/prims/jvmtiExport.cpp   Thu Jul 09 04:32:30 2020 +0200
> +++ b/src/hotspot/share/prims/jvmtiExport.cpp   Thu Jul 09 08:05:46 2020 +0200
> @@ -1067,3 +1067,3 @@
>        _jobj = (jobject)to_jobject(obj);
> -     _size = Universe::heap()->obj_size(obj) * wordSize;
> +     _size = obj->size() * wordSize;
>      };
> diff -r 9cc348ebdc82 src/hotspot/share/prims/whitebox.cpp
> --- a/src/hotspot/share/prims/whitebox.cpp      Thu Jul 09 04:32:30 2020 +0200
> +++ b/src/hotspot/share/prims/whitebox.cpp      Thu Jul 09 08:05:46 2020 +0200
> @@ -389,3 +389,3 @@
>     oop p = JNIHandles::resolve(obj);
> -  return Universe::heap()->obj_size(p) * HeapWordSize;
> +  return p->size() * HeapWordSize;
>   WB_END
>
> Testing: tier{1,2}; jdk-submit (running)
>



From linzang at tencent.com  Thu Jul  9 13:48:05 2020
From: linzang at tencent.com (=?utf-8?B?bGluemFuZyjoh6fnkLMp?=)
Date: Thu, 9 Jul 2020 13:48:05 +0000
Subject: RFR (S) 8249140: fail to build hotspot with gcc-4.8.5(Internet
 mail)
In-Reply-To: <7172888B-C11F-48E9-B9CA-60F52E0208D7@oracle.com>
References: <660DFF34-DF95-417B-A3CF-D8A6F0AC8B08@tencent.com>
 <7172888B-C11F-48E9-B9CA-60F52E0208D7@oracle.com>
Message-ID: <42EF6318-B015-46F6-8DF8-314913A04C56@tencent.com>

Hi Kim,
     I am OK with not making this change if the gcc version is old and and very shotly and buggy. It happened to me that my linux box use this version of gcc.
     Just curious about what is the minimal acceptable version of gcc for building hotspot, maybe gcc version later than 6?  
     I will close the issue with not fix. Thanks!
 
BRs,
Lin

?On 2020/7/9, 8:59 PM, "Kim Barrett" <kim.barrett at oracle.com> wrote:

    > On Jul 9, 2020, at 3:38 AM, linzang(??) <linzang at tencent.com> wrote:
    > 
    > Hi
    >                May I ask your help to review the fix of 8249140, which cause hotspot build fail with gcc-4.8.5
    >              Bug: https://bugs.openjdk.java.net/browse/JDK-8249140
    >                Webrev: http://cr.openjdk.java.net/~lzang/8249140/webrev/
    >                Thanks!
    > 
    > BRs,
    > Lin

    The warnings appear to be a bug in that (extremely old) compiler
    version; the expressions involved clearly are constant expressions.
    Also, that compiler version will very shortly (hopefully within a
    couple of weeks) not work at all, because it lacks C++14 support.
    And the change itself does not make the code better in any other way.

    So I think this change should not be made.




From kim.barrett at oracle.com  Thu Jul  9 15:04:00 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 9 Jul 2020 11:04:00 -0400
Subject: RFR [15] : 8249037 : clean up FileInstaller $test.src $cwd in
 vmTestbase_vm_g1classunloading tests
In-Reply-To: <01BE191F-3AF8-4D7E-8285-0229FA56ED58@oracle.com>
References: <01BE191F-3AF8-4D7E-8285-0229FA56ED58@oracle.com>
Message-ID: <1BF68AC7-0B1A-4D18-A3E7-A169DA41B883@oracle.com>

> On Jul 8, 2020, at 10:39 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
> 
> http://cr.openjdk.java.net/~iignatyev//8249037/webrev.00
>> 112 lines changed: 0 ins; 112 del; 0 mod;
> 
> Hi all,
> 
> could you please review the patch which removes `FileInstaller . .` jtreg action from :vmTestbase_vm_g1classunloading tests?
> from the main issue(8204985):
>> all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests.
> 
> the patch is as simple as `ag -l  '@run driver jdk.test.lib.FileInstaller . .' vmTestbase/gc/g1/unloading/tests | xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}`
> 
> testing: :vmTestbase_vm_g1classunloading on linux-x64
> webrev: http://cr.openjdk.java.net/~iignatyev//8249037/webrev.00
> JBS: https://bugs.openjdk.java.net/browse/JDK-8249037
> 
> Thanks,
> -- Igor

Looks good.

I wonder though, why target this cleanup to JDK 15?



From kim.barrett at oracle.com  Thu Jul  9 15:34:18 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 9 Jul 2020 11:34:18 -0400
Subject: RFR (S) 8249140: fail to build hotspot with gcc-4.8.5(Internet
 mail)
In-Reply-To: <42EF6318-B015-46F6-8DF8-314913A04C56@tencent.com>
References: <660DFF34-DF95-417B-A3CF-D8A6F0AC8B08@tencent.com>
 <7172888B-C11F-48E9-B9CA-60F52E0208D7@oracle.com>
 <42EF6318-B015-46F6-8DF8-314913A04C56@tencent.com>
Message-ID: <4D7C0E01-76B7-4317-8C8B-74A7EE68AA1B@oracle.com>

> On Jul 9, 2020, at 9:48 AM, linzang(??) <linzang at tencent.com> wrote:
> 
> Hi Kim,
>     I am OK with not making this change if the gcc version is old and and very shotly and buggy. It happened to me that my linux box use this version of gcc.
>     Just curious about what is the minimal acceptable version of gcc for building hotspot, maybe gcc version later than 6?  

Sounds like your linux box could perhaps use an update.

Oracle currently builds and tests the development head with gcc 9.2. I
would not be surprised to see a switch to 10.x sometime relatively soon.

The minimum version supported by the JDK build system is gcc 5.0 (see
make/autoconf/toolchain.m4), though I don't know of anyone regularly
testing and reporting on anything that old (see
https://wiki.openjdk.java.net/display/Build/Supported+Build+Platforms).
(I think configure warns but tries to continue with older versions.)

JEP 347 changes the -std argument from -std=gnu++98 to -std=c++14,
which I think is not supported or accepted by anything older than gcc 5.0.

>     I will close the issue with not fix. Thanks!

Thanks.




From igor.ignatyev at oracle.com  Thu Jul  9 16:09:27 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Thu, 9 Jul 2020 09:09:27 -0700
Subject: RFR [15] : 8249037 : clean up FileInstaller $test.src $cwd in
 vmTestbase_vm_g1classunloading tests
In-Reply-To: <1BF68AC7-0B1A-4D18-A3E7-A169DA41B883@oracle.com>
References: <01BE191F-3AF8-4D7E-8285-0229FA56ED58@oracle.com>
 <1BF68AC7-0B1A-4D18-A3E7-A169DA41B883@oracle.com>
Message-ID: <CE111F0D-5179-4A1E-9AAB-B3CF3E558E5E@oracle.com>

> On Jul 9, 2020, at 8:04 AM, Kim Barrett <kim.barrett at oracle.com> wrote:
> 
>> On Jul 8, 2020, at 10:39 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
>> 
>> http://cr.openjdk.java.net/~iignatyev//8249037/webrev.00
>>> 112 lines changed: 0 ins; 112 del; 0 mod;
>> 
>> Hi all,
>> 
>> could you please review the patch which removes `FileInstaller . .` jtreg action from :vmTestbase_vm_g1classunloading tests?
>> from the main issue(8204985):
>>> all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests.
>> 
>> the patch is as simple as `ag -l  '@run driver jdk.test.lib.FileInstaller . .' vmTestbase/gc/g1/unloading/tests | xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}`
>> 
>> testing: :vmTestbase_vm_g1classunloading on linux-x64
>> webrev: http://cr.openjdk.java.net/~iignatyev//8249037/webrev.00
>> JBS: https://bugs.openjdk.java.net/browse/JDK-8249037
>> 
>> Thanks,
>> -- Igor
> 
> Looks good.
> 
> I wonder though, why target this cleanup to JDK 15?

Hi Kim,

thanks for your review, pushed.

this is targeted to jdk15 b/c it's a low risk test-only change which reduces test execution time and hence is particularly useful now when we have one release train to test in parallel.

Cheers,
-- Igor

From igor.ignatyev at oracle.com  Thu Jul  9 17:44:00 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Thu, 9 Jul 2020 10:44:00 -0700
Subject: RFR [15] : 8249038 : clean up FileInstaller $test.src $cwd in
 vmTestbase_vm_gc tests
Message-ID: <923B6A66-9578-4465-802D-19CF6AA0CE5F@oracle.com>

http://cr.openjdk.java.net/~iignatyev//8249038/webrev.00
> 267 lines changed: 1 ins; 264 del; 2 mod;

Hi all,

could you please review the patch which removes `FileInstaller . .` jtreg action from :vmTestbase_vm_gc tests?
from the main issue(8204985):
> all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests.

the biggest part of the patch is just `ag -l  '@run driver jdk.test.lib.FileInstaller . .' vmTestbase/gc/ vmTestbase/vm/gc  | xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}`. the only test which depended on FileInstaller was vmTestbase/gc/gctests/fileTest, this test has been updated to search for the specified file in Utils.TEST_SRC directory.

testing: 
 - :vmTestbase_vm_gc on linux-x64
 - vmTestbase/gc/gctests/fileTest on {linux,windows,macos}-x64
webrev: http://cr.openjdk.java.net/~iignatyev//8249038/webrev.00
JBS: https://bugs.openjdk.java.net/browse/JDK-8249038

Thanks,
-- Igor

From kim.barrett at oracle.com  Fri Jul 10 01:15:04 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Fri, 10 Jul 2020 01:15:04 +0000 (UTC)
Subject: RFR [15] : 8249038 : clean up FileInstaller $test.src $cwd in
 vmTestbase_vm_gc tests
In-Reply-To: <923B6A66-9578-4465-802D-19CF6AA0CE5F@oracle.com>
References: <923B6A66-9578-4465-802D-19CF6AA0CE5F@oracle.com>
Message-ID: <FABBC32F-A02A-4C21-B4E4-CF245F389B97@oracle.com>

> On Jul 9, 2020, at 1:44 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
> 
> http://cr.openjdk.java.net/~iignatyev//8249038/webrev.00
>> 267 lines changed: 1 ins; 264 del; 2 mod;
> 
> Hi all,
> 
> could you please review the patch which removes `FileInstaller . .` jtreg action from :vmTestbase_vm_gc tests?
> from the main issue(8204985):
>> all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests.
> 
> the biggest part of the patch is just `ag -l  '@run driver jdk.test.lib.FileInstaller . .' vmTestbase/gc/ vmTestbase/vm/gc  | xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}`. the only test which depended on FileInstaller was vmTestbase/gc/gctests/fileTest, this test has been updated to search for the specified file in Utils.TEST_SRC directory.
> 
> testing: 
> - :vmTestbase_vm_gc on linux-x64
> - vmTestbase/gc/gctests/fileTest on {linux,windows,macos}-x64
> webrev: http://cr.openjdk.java.net/~iignatyev//8249038/webrev.00
> JBS: https://bugs.openjdk.java.net/browse/JDK-8249038
> 
> Thanks,
> -- Igor

Looks good.



From shade at redhat.com  Fri Jul 10 06:41:43 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Fri, 10 Jul 2020 08:41:43 +0200
Subject: RFR (S) 8248652: Shenandoah: SATB buffer handling may assume no
 forwarded objects
In-Reply-To: <046b7175-361c-e823-06c5-90d053d4c47d@redhat.com>
References: <046b7175-361c-e823-06c5-90d053d4c47d@redhat.com>
Message-ID: <c5b3a3cb-f8b2-724a-4461-cd43dc2c4422@redhat.com>

On 7/1/20 7:56 PM, Aleksey Shipilev wrote:
> RFE:
>   https://bugs.openjdk.java.net/browse/JDK-8248652
> 
> Since CM-with-UR is gone, SATB may assume no forwarded objects are ever exposed through it. The only
> way marking code can experience forwarded objects is due to Full GC marking. In that case, SATB
> should be inactive.
> 
> Fix:
>   https://cr.openjdk.java.net/~shade/8248652/webrev.01/
> 
> Testing: hotspot_gc_shenandoah

Ping? :)

-- 
Thanks,
-Aleksey


From rkennke at redhat.com  Fri Jul 10 08:31:26 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Fri, 10 Jul 2020 10:31:26 +0200
Subject: RFR (S) 8248652: Shenandoah: SATB buffer handling may assume no
 forwarded objects
In-Reply-To: <c5b3a3cb-f8b2-724a-4461-cd43dc2c4422@redhat.com>
References: <046b7175-361c-e823-06c5-90d053d4c47d@redhat.com>
 <c5b3a3cb-f8b2-724a-4461-cd43dc2c4422@redhat.com>
Message-ID: <5d27e6809fc16574fbd7a6ae536b23ea91a55acf.camel@redhat.com>

On Fri, 2020-07-10 at 08:41 +0200, Aleksey Shipilev wrote:
> 	Error verifying signature: Cannot verify message signature:
> Incorrect message format
> On 7/1/20 7:56 PM, Aleksey Shipilev wrote:
> > RFE:
> >   https://bugs.openjdk.java.net/browse/JDK-8248652
> > 
> > Since CM-with-UR is gone, SATB may assume no forwarded objects are
> > ever exposed through it. The only
> > way marking code can experience forwarded objects is due to Full GC
> > marking. In that case, SATB
> > should be inactive.
> > 
> > Fix:
> >   https://cr.openjdk.java.net/~shade/8248652/webrev.01/
> > 
> > Testing: hotspot_gc_shenandoah
> 
> Ping? :)

Sorry, that one slipped through.

The patch looks good! Thank you!

Roman





From shade at redhat.com  Fri Jul 10 08:37:51 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Fri, 10 Jul 2020 10:37:51 +0200
Subject: RFR (S) 8249137: Remove CollectedHeap::obj_size
In-Reply-To: <70984ebf-2717-c7b3-7076-12e2c8c7515c@oracle.com>
References: <a9bd4c4d-2d6b-733e-abbf-90e873c70951@redhat.com>
 <70984ebf-2717-c7b3-7076-12e2c8c7515c@oracle.com>
Message-ID: <a2432ff6-79b2-aefc-0450-d1c2b3cabd22@redhat.com>

Okay, thanks.

I already have 2 reviewers (rkennke, tschatzl), do I need more specifically from serviceability-dev@?

-Aleksey

On 7/9/20 3:34 PM, Daniel D. Daugherty wrote:
> Adding serviceability-dev at ... since a couple of JVM/TI files are changed
> in this RFR. Also, I moved the bug from hotspot/runtime -> hotspot/gc.
> 
> Dan
> 
> 
> On 7/9/20 2:36 AM, Aleksey Shipilev wrote:
>> RFE:
>>    https://bugs.openjdk.java.net/browse/JDK-8249137
>>
>> It was added by JDK-8211270 to support old-style Shenandoah that needed a separate fwdptr slot.
>> After JDK-8224584 it does not need this anymore. Additionally, CH::obj_size may disagree with other
>> code that pokes at layout helper directly, for example GraphKit::new_instance.
>>
>> This also avoids a virtual call on some paths, although those paths are not very performance-sensitive.
>>
>> The patch is a simple series of few-liners:
>>
>> diff -r 9cc348ebdc82 src/hotspot/share/gc/shared/collectedHeap.cpp
>> --- a/src/hotspot/share/gc/shared/collectedHeap.cpp     Thu Jul 09 04:32:30 2020 +0200
>> +++ b/src/hotspot/share/gc/shared/collectedHeap.cpp     Thu Jul 09 08:05:46 2020 +0200
>> @@ -578,6 +578,2 @@
>>
>> -size_t CollectedHeap::obj_size(oop obj) const {
>> -  return obj->size();
>> -}
>> -
>>   uint32_t CollectedHeap::hash_oop(oop obj) const {
>> diff -r 9cc348ebdc82 src/hotspot/share/gc/shared/collectedHeap.hpp
>> --- a/src/hotspot/share/gc/shared/collectedHeap.hpp     Thu Jul 09 04:32:30 2020 +0200
>> +++ b/src/hotspot/share/gc/shared/collectedHeap.hpp     Thu Jul 09 08:05:46 2020 +0200
>> @@ -495,4 +495,2 @@
>>
>> -  virtual size_t obj_size(oop obj) const;
>> -
>>     // Non product verification and debugging.
>> diff -r 9cc348ebdc82 src/hotspot/share/prims/jvmtiEnv.cpp
>> --- a/src/hotspot/share/prims/jvmtiEnv.cpp      Thu Jul 09 04:32:30 2020 +0200
>> +++ b/src/hotspot/share/prims/jvmtiEnv.cpp      Thu Jul 09 08:05:46 2020 +0200
>> @@ -488,3 +488,3 @@
>>     NULL_CHECK(mirror, JVMTI_ERROR_INVALID_OBJECT);
>> -  *size_ptr = (jlong)Universe::heap()->obj_size(mirror) * wordSize;
>> +  *size_ptr = (jlong)mirror->size() * wordSize;
>>     return JVMTI_ERROR_NONE;
>> diff -r 9cc348ebdc82 src/hotspot/share/prims/jvmtiExport.cpp
>> --- a/src/hotspot/share/prims/jvmtiExport.cpp   Thu Jul 09 04:32:30 2020 +0200
>> +++ b/src/hotspot/share/prims/jvmtiExport.cpp   Thu Jul 09 08:05:46 2020 +0200
>> @@ -1067,3 +1067,3 @@
>>        _jobj = (jobject)to_jobject(obj);
>> -     _size = Universe::heap()->obj_size(obj) * wordSize;
>> +     _size = obj->size() * wordSize;
>>      };
>> diff -r 9cc348ebdc82 src/hotspot/share/prims/whitebox.cpp
>> --- a/src/hotspot/share/prims/whitebox.cpp      Thu Jul 09 04:32:30 2020 +0200
>> +++ b/src/hotspot/share/prims/whitebox.cpp      Thu Jul 09 08:05:46 2020 +0200
>> @@ -389,3 +389,3 @@
>>     oop p = JNIHandles::resolve(obj);
>> -  return Universe::heap()->obj_size(p) * HeapWordSize;
>> +  return p->size() * HeapWordSize;
>>   WB_END
>>
>> Testing: tier{1,2}; jdk-submit (running)
>>
> 


-- 
Thanks,
-Aleksey


From maoliang.ml at alibaba-inc.com  Fri Jul 10 09:47:20 2020
From: maoliang.ml at alibaba-inc.com (Liang Mao)
Date: Fri, 10 Jul 2020 17:47:20 +0800
Subject: RFR (S) 8240556 and RFR (S) 8248783 for G1 humongous objects
In-Reply-To: <006401d6569c$f43e0340$dcba09c0$@alibaba-inc.com>
References: <006401d6569c$f43e0340$dcba09c0$@alibaba-inc.com>
Message-ID: <006c01d6569f$1b130b30$51392190$@alibaba-inc.com>

Hi,

I just attached a test case to this bug to demonstrate the issue in our
workloads.
Both 8u and JDK15 could reproduce the same problem.
https://bugs.openjdk.java.net/browse/JDK-8248783

The high allocation rate of humongous objects will easily lead to evacuation
failures and Full GC while increasing region size can resolve the problem.
In
our workloads, we have too many humongous objects even with 32m region
size. So better heuristics to trigger GCs in such scenario is important to
us. I know 
it may be a big change since original heuristics exists for many years. Hope
you 
guys can make some tests and let me know if it works fine for the legacy
tests as
well.

Thanks,
Liang 

> -----Original Message-----
> Subject: RFR (S) 8240556 and RFR (S) 8248783 for G1 humongous objects
> 
> Hi Man and G1 team,
> 
> Previously we were discussing about abort of concurrent mark(JDK-8240556).
I
> 
> made a concurrent cleaning bitmap version instead of the original STW one.
> But there are still some other problems in real world. Recently I
revisited the
> implementation and make more optimization and the result seems very good.
> 
> The most severe problems in our work loads are as below:
> 1) Humongous objects allocation will invade the space of young generation
and
> lead to long time STW because of to-space exhausted and even Full GC
> 2) Frequent concurrent marking will cost significant CPU resources (which
could
> be resolved by abort concurrent mark JDK-8240556)
> 3) Frequent GCs because the humongous allocation can reach IHOP very
quickly
> even no matter w or w/o abort concurrent mark.
> 
> Both 1) and 3) didn't have solutions yet. I made another change to share
the
> 
> young space for humongous allocation and eden allocation. Initial-mark
will be
> triggered if reserve space is invaded. My colleague helped to create a new
bug
> ID: https://bugs.openjdk.java.net/browse/JDK-8248783
> With this change and abort concurrent mark our problematic work loads run
> very smoothly without any GC issues. I have tested the 2 changes in
several
> different applications, like web services, database, etc.
> 
> Our real test is done with 8u and I have created JDK15 webrevs with the
same
> logic. The code completes the test of jtreg gc/g1 and jbb2015. Could you
please
> take a look and run some tests?
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8240556
> Webrev: http://cr.openjdk.java.net/~ddong/liangmao/8240556/webrev.00/
> (This may related to https://bugs.openjdk.java.net/browse/JDK-8247928)
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8248783
> Webrev: http://cr.openjdk.java.net/~ddong/liangmao/8248783/webrev.00/
> 
> 
> Thanks,
> Liang



From felix.yang at huawei.com  Fri Jul 10 11:43:54 2020
From: felix.yang at huawei.com (Yangfei (Felix))
Date: Fri, 10 Jul 2020 11:43:54 +0000
Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences
 between free chunk check and klass read
In-Reply-To: <06646cf3-bc21-977f-96e2-1e9a974d1b00@redhat.com>
References: <DA41BE1DDCA941489001C7FBD7A8820EE7E63D86@dggeml527-mbx.china.huawei.com>
 <d6590e74-7ce2-70b9-7432-96b969686ab7@redhat.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E6466E@dggeml527-mbx.china.huawei.com>
 <8957BB9E-617D-4E24-9CC4-31AF7D01D12E@oracle.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E64ECD@dggeml527-mbx.china.huawei.com>
 <65012FED-2A93-4899-939E-887175AB07AC@oracle.com>
 <06646cf3-bc21-977f-96e2-1e9a974d1b00@redhat.com>
Message-ID: <DA41BE1DDCA941489001C7FBD7A8820EE7E688DE@dggeml527-mbx.china.huawei.com>

Hi,

> -----Original Message-----
> From: Andrew Haley [mailto:aph at redhat.com]
> Sent: Wednesday, July 8, 2020 5:48 PM
> To: Kim Barrett <kim.barrett at oracle.com>; Yangfei (Felix)
> <felix.yang at huawei.com>
> Cc: jdk8u-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net;
> aarch64-port-dev at openjdk.java.net
> Subject: Re: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory
> fences between free chunk check and klass read
> 
> On 08/07/2020 09:41, Kim Barrett wrote:
> >> Since CMS is deprecated from JDK9, I am not sure if it's appropriate to fix
> this issue for those JDK9+ versions.
> > Deprecated != unsupported.
> >
> 
> Yes, it must be done. Thanks.

CCing to jdk-updates-dev.

I have prepared another two webrevs for jdk11u and jdk13u:
  Jdk11u-dev:  http://cr.openjdk.java.net/~fyang/8248851-11u/ 
  Jdk13u-dev: http://cr.openjdk.java.net/~fyang/8248851-13u/ 

The only difference lies in copyright year update.  Both tiere1-3 tested on aarch64-linux-gnu.
I will put jdk11u-fix-request and jdk13u-fix-request label on the issue if they are good.

I have also prepared a new webrev for jdk8u incorporating copyright year update:
  Jdk8u-dev: http://cr.openjdk.java.net/~fyang/8248851-8u/ 

Thanks,
Felix



From daniel.daugherty at oracle.com  Fri Jul 10 15:07:01 2020
From: daniel.daugherty at oracle.com (Daniel D. Daugherty)
Date: Fri, 10 Jul 2020 11:07:01 -0400
Subject: RFR (S) 8249137: Remove CollectedHeap::obj_size
In-Reply-To: <a2432ff6-79b2-aefc-0450-d1c2b3cabd22@redhat.com>
References: <a9bd4c4d-2d6b-733e-abbf-90e873c70951@redhat.com>
 <70984ebf-2717-c7b3-7076-12e2c8c7515c@oracle.com>
 <a2432ff6-79b2-aefc-0450-d1c2b3cabd22@redhat.com>
Message-ID: <99b679ec-fcae-6f30-1186-1cf8de809c6f@oracle.com>

On 7/10/20 4:37 AM, Aleksey Shipilev wrote:
> Okay, thanks.
>
> I already have 2 reviewers (rkennke, tschatzl), do I need more specifically from serviceability-dev@?

Since you're touching the Serviceability team's code, it would be
polite to wait for a review...

Dan


>
> -Aleksey
>
> On 7/9/20 3:34 PM, Daniel D. Daugherty wrote:
>> Adding serviceability-dev at ... since a couple of JVM/TI files are changed
>> in this RFR. Also, I moved the bug from hotspot/runtime -> hotspot/gc.
>>
>> Dan
>>
>>
>> On 7/9/20 2:36 AM, Aleksey Shipilev wrote:
>>> RFE:
>>>     https://bugs.openjdk.java.net/browse/JDK-8249137
>>>
>>> It was added by JDK-8211270 to support old-style Shenandoah that needed a separate fwdptr slot.
>>> After JDK-8224584 it does not need this anymore. Additionally, CH::obj_size may disagree with other
>>> code that pokes at layout helper directly, for example GraphKit::new_instance.
>>>
>>> This also avoids a virtual call on some paths, although those paths are not very performance-sensitive.
>>>
>>> The patch is a simple series of few-liners:
>>>
>>> diff -r 9cc348ebdc82 src/hotspot/share/gc/shared/collectedHeap.cpp
>>> --- a/src/hotspot/share/gc/shared/collectedHeap.cpp     Thu Jul 09 04:32:30 2020 +0200
>>> +++ b/src/hotspot/share/gc/shared/collectedHeap.cpp     Thu Jul 09 08:05:46 2020 +0200
>>> @@ -578,6 +578,2 @@
>>>
>>> -size_t CollectedHeap::obj_size(oop obj) const {
>>> -  return obj->size();
>>> -}
>>> -
>>>    uint32_t CollectedHeap::hash_oop(oop obj) const {
>>> diff -r 9cc348ebdc82 src/hotspot/share/gc/shared/collectedHeap.hpp
>>> --- a/src/hotspot/share/gc/shared/collectedHeap.hpp     Thu Jul 09 04:32:30 2020 +0200
>>> +++ b/src/hotspot/share/gc/shared/collectedHeap.hpp     Thu Jul 09 08:05:46 2020 +0200
>>> @@ -495,4 +495,2 @@
>>>
>>> -  virtual size_t obj_size(oop obj) const;
>>> -
>>>      // Non product verification and debugging.
>>> diff -r 9cc348ebdc82 src/hotspot/share/prims/jvmtiEnv.cpp
>>> --- a/src/hotspot/share/prims/jvmtiEnv.cpp      Thu Jul 09 04:32:30 2020 +0200
>>> +++ b/src/hotspot/share/prims/jvmtiEnv.cpp      Thu Jul 09 08:05:46 2020 +0200
>>> @@ -488,3 +488,3 @@
>>>      NULL_CHECK(mirror, JVMTI_ERROR_INVALID_OBJECT);
>>> -  *size_ptr = (jlong)Universe::heap()->obj_size(mirror) * wordSize;
>>> +  *size_ptr = (jlong)mirror->size() * wordSize;
>>>      return JVMTI_ERROR_NONE;
>>> diff -r 9cc348ebdc82 src/hotspot/share/prims/jvmtiExport.cpp
>>> --- a/src/hotspot/share/prims/jvmtiExport.cpp   Thu Jul 09 04:32:30 2020 +0200
>>> +++ b/src/hotspot/share/prims/jvmtiExport.cpp   Thu Jul 09 08:05:46 2020 +0200
>>> @@ -1067,3 +1067,3 @@
>>>         _jobj = (jobject)to_jobject(obj);
>>> -     _size = Universe::heap()->obj_size(obj) * wordSize;
>>> +     _size = obj->size() * wordSize;
>>>       };
>>> diff -r 9cc348ebdc82 src/hotspot/share/prims/whitebox.cpp
>>> --- a/src/hotspot/share/prims/whitebox.cpp      Thu Jul 09 04:32:30 2020 +0200
>>> +++ b/src/hotspot/share/prims/whitebox.cpp      Thu Jul 09 08:05:46 2020 +0200
>>> @@ -389,3 +389,3 @@
>>>      oop p = JNIHandles::resolve(obj);
>>> -  return Universe::heap()->obj_size(p) * HeapWordSize;
>>> +  return p->size() * HeapWordSize;
>>>    WB_END
>>>
>>> Testing: tier{1,2}; jdk-submit (running)
>>>
>



From zgu at redhat.com  Fri Jul 10 18:04:16 2020
From: zgu at redhat.com (Zhengyu Gu)
Date: Fri, 10 Jul 2020 14:04:16 -0400
Subject: RFR 8247670: Shenandoah: deadlock during class unloading OOME
In-Reply-To: <a8f81fb6-718e-41c2-50eb-d04bab521df8@redhat.com>
References: <a8f81fb6-718e-41c2-50eb-d04bab521df8@redhat.com>
Message-ID: <271ad565-c60d-5741-eb91-ebd52213a6ca@redhat.com>

The deadlock is caused by JDK-8245288. After further investigation, 
JDK-8245288 change does not seem to provide improvement that reported in 
original bug. Let's just backout JDK-8245288.

Webrev: http://cr.openjdk.java.net/~zgu/JDK-8247670/webrev.01/

Test:
   hotspot_gc_shenandoah

Thanks,

-Zhengyu


On 6/16/20 3:48 PM, Zhengyu Gu wrote:
> The deadlock is caused by one thread holding per-nmethod lock, then 
> encountering evac-oom. At the same time, another thread entering 
> evac-oom scope, then acquiring the same per-nmethod lock.
> 
> The first thread expects the second thread to see evac-oom and exit the 
> scope, but the second thread is blocked on acquiring per-nmethod lock.
> 
> The solution is to introduce an abortable locker on per-nmethod lock. If 
> the second thread can not acquire the lock, but see evac-oom, it simply 
> aborts, so it can exit evac-oom scope.
> 
> The solution does come with penalties:
> 
> If the second thread is a Java thread (via nmethod entry barrier), the 
> nmethod will be deopt.
> 
> If the second thread is worker, it causes current code root processing 
> to abort, then restart.
> 
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8247670
> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8247670/webrev.00/
> 
> Test:
>  ? hotspot_gc_shenandoah (x86_64 and aarch64)
> 
> Thanks,
> 
> -Zhengyu



From rkennke at redhat.com  Fri Jul 10 18:14:08 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Fri, 10 Jul 2020 20:14:08 +0200
Subject: RFR 8247670: Shenandoah: deadlock during class unloading OOME
In-Reply-To: <271ad565-c60d-5741-eb91-ebd52213a6ca@redhat.com>
References: <a8f81fb6-718e-41c2-50eb-d04bab521df8@redhat.com>
 <271ad565-c60d-5741-eb91-ebd52213a6ca@redhat.com>
Message-ID: <a3511f642c89597fe99a03a7880a65dc588fb9f4.camel@redhat.com>


Ok, let's do that.

Thanks,
Roman


On Fri, 2020-07-10 at 14:04 -0400, Zhengyu Gu wrote:
> The deadlock is caused by JDK-8245288. After further investigation, 
> JDK-8245288 change does not seem to provide improvement that reported
> in 
> original bug. Let's just backout JDK-8245288.
> 
> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8247670/webrev.01/
> 
> Test:
>    hotspot_gc_shenandoah
> 
> Thanks,
> 
> -Zhengyu
> 
> 
> On 6/16/20 3:48 PM, Zhengyu Gu wrote:
> > The deadlock is caused by one thread holding per-nmethod lock,
> > then 
> > encountering evac-oom. At the same time, another thread entering 
> > evac-oom scope, then acquiring the same per-nmethod lock.
> > 
> > The first thread expects the second thread to see evac-oom and exit
> > the 
> > scope, but the second thread is blocked on acquiring per-nmethod
> > lock.
> > 
> > The solution is to introduce an abortable locker on per-nmethod
> > lock. If 
> > the second thread can not acquire the lock, but see evac-oom, it
> > simply 
> > aborts, so it can exit evac-oom scope.
> > 
> > The solution does come with penalties:
> > 
> > If the second thread is a Java thread (via nmethod entry barrier),
> > the 
> > nmethod will be deopt.
> > 
> > If the second thread is worker, it causes current code root
> > processing 
> > to abort, then restart.
> > 
> > 
> > Bug: https://bugs.openjdk.java.net/browse/JDK-8247670
> > Webrev: http://cr.openjdk.java.net/~zgu/JDK-8247670/webrev.00/
> > 
> > Test:
> >    hotspot_gc_shenandoah (x86_64 and aarch64)
> > 
> > Thanks,
> > 
> > -Zhengyu



From zgu at redhat.com  Fri Jul 10 18:17:39 2020
From: zgu at redhat.com (Zhengyu Gu)
Date: Fri, 10 Jul 2020 14:17:39 -0400
Subject: RFR 8247670: Shenandoah: deadlock during class unloading OOME
In-Reply-To: <a3511f642c89597fe99a03a7880a65dc588fb9f4.camel@redhat.com>
References: <a8f81fb6-718e-41c2-50eb-d04bab521df8@redhat.com>
 <271ad565-c60d-5741-eb91-ebd52213a6ca@redhat.com>
 <a3511f642c89597fe99a03a7880a65dc588fb9f4.camel@redhat.com>
Message-ID: <24d46c08-8f11-6ace-a216-f14eeefe18bd@redhat.com>

Thanks and pushed.

-Zhengyu

On 7/10/20 2:14 PM, Roman Kennke wrote:
> 
> Ok, let's do that.
> 
> Thanks,
> Roman
> 
> 
> On Fri, 2020-07-10 at 14:04 -0400, Zhengyu Gu wrote:
>> The deadlock is caused by JDK-8245288. After further investigation,
>> JDK-8245288 change does not seem to provide improvement that reported
>> in
>> original bug. Let's just backout JDK-8245288.
>>
>> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8247670/webrev.01/
>>
>> Test:
>>     hotspot_gc_shenandoah
>>
>> Thanks,
>>
>> -Zhengyu
>>
>>
>> On 6/16/20 3:48 PM, Zhengyu Gu wrote:
>>> The deadlock is caused by one thread holding per-nmethod lock,
>>> then
>>> encountering evac-oom. At the same time, another thread entering
>>> evac-oom scope, then acquiring the same per-nmethod lock.
>>>
>>> The first thread expects the second thread to see evac-oom and exit
>>> the
>>> scope, but the second thread is blocked on acquiring per-nmethod
>>> lock.
>>>
>>> The solution is to introduce an abortable locker on per-nmethod
>>> lock. If
>>> the second thread can not acquire the lock, but see evac-oom, it
>>> simply
>>> aborts, so it can exit evac-oom scope.
>>>
>>> The solution does come with penalties:
>>>
>>> If the second thread is a Java thread (via nmethod entry barrier),
>>> the
>>> nmethod will be deopt.
>>>
>>> If the second thread is worker, it causes current code root
>>> processing
>>> to abort, then restart.
>>>
>>>
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8247670
>>> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8247670/webrev.00/
>>>
>>> Test:
>>>     hotspot_gc_shenandoah (x86_64 and aarch64)
>>>
>>> Thanks,
>>>
>>> -Zhengyu
> 



From igor.ignatyev at oracle.com  Fri Jul 10 18:51:28 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Fri, 10 Jul 2020 11:51:28 -0700
Subject: RFR [15] : 8249038 : clean up FileInstaller $test.src $cwd in
 vmTestbase_vm_gc tests
In-Reply-To: <FABBC32F-A02A-4C21-B4E4-CF245F389B97@oracle.com>
References: <923B6A66-9578-4465-802D-19CF6AA0CE5F@oracle.com>
 <FABBC32F-A02A-4C21-B4E4-CF245F389B97@oracle.com>
Message-ID: <EC5F4EAF-9D2F-4E88-AA74-B8BBED0300EA@oracle.com>



> On Jul 9, 2020, at 6:15 PM, Kim Barrett <kim.barrett at oracle.com> wrote:
> 
>> On Jul 9, 2020, at 1:44 PM, Igor Ignatyev <igor.ignatyev at oracle.com> wrote:
>> 
>> http://cr.openjdk.java.net/~iignatyev//8249038/webrev.00
>>> 267 lines changed: 1 ins; 264 del; 2 mod;
>> 
>> Hi all,
>> 
>> could you please review the patch which removes `FileInstaller . .` jtreg action from :vmTestbase_vm_gc tests?
>> from the main issue(8204985):
>>> all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests.
>> 
>> the biggest part of the patch is just `ag -l  '@run driver jdk.test.lib.FileInstaller . .' vmTestbase/gc/ vmTestbase/vm/gc  | xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}`. the only test which depended on FileInstaller was vmTestbase/gc/gctests/fileTest, this test has been updated to search for the specified file in Utils.TEST_SRC directory.
>> 
>> testing: 
>> - :vmTestbase_vm_gc on linux-x64
>> - vmTestbase/gc/gctests/fileTest on {linux,windows,macos}-x64
>> webrev: http://cr.openjdk.java.net/~iignatyev//8249038/webrev.00
>> JBS: https://bugs.openjdk.java.net/browse/JDK-8249038
>> 
>> Thanks,
>> -- Igor
> 
> Looks good.
> 

Thanks Kim, pushed to jdk/jdk15.

-- Igor

From igor.ignatyev at oracle.com  Fri Jul 10 19:07:12 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Fri, 10 Jul 2020 12:07:12 -0700
Subject: RFR(S) [15] : 8249000 : vm.gc.X should take selected JIT into
 account
In-Reply-To: <dbf5a546-51e0-400f-94f7-5b528972f8fe@oracle.com>
References: <FFC49DC0-AA15-45B5-B285-E043D1AEAF97@oracle.com>
 <6964ac32-e9ec-d700-0bdb-ea51f4610afe@oracle.com>
 <7A1992A7-1493-4DF0-B621-195CE986D34F@oracle.com>
 <2c92a9a5-77af-c100-fa9b-f765e9d23dce@oracle.com>
 <BA1B1E12-19E3-45E3-8FDD-0307DDADDC3E@oracle.com>
 <dbf5a546-51e0-400f-94f7-5b528972f8fe@oracle.com>
Message-ID: <6409D2AD-173C-451A-814E-32C88860A5C5@oracle.com>

thanks Vladimir, pushed.
-- Igor

> On Jul 8, 2020, at 3:36 PM, Vladimir Kozlov <vladimir.kozlov at oracle.com> wrote:
> 
> Good.
> 
> Thanks,
> Vladimir
> 
> On 7/8/20 11:40 AM, Igor Ignatyev wrote:
>> Thanks Vladimir.
>> for the record, I've updated ProblemList-graal.txt w/ the following:
>>> diff -r 14ffd658a23a test/hotspot/jtreg/ProblemList-graal.txt
>>> --- a/test/hotspot/jtreg/ProblemList-graal.txt  Wed Jul 08 11:35:30 2020 -0700
>>> +++ b/test/hotspot/jtreg/ProblemList-graal.txt  Wed Jul 08 11:37:44 2020 -0700
>>> @@ -229,6 +229,7 @@
>>>  compiler/loopopts/TestOverunrolling.java                8207267   generic-all
>>>  compiler/jsr292/NonInlinedCall/InvokeTest.java          8207267   generic-all
>>>  compiler/codegen/TestTrichotomyExpressions.java         8207267   generic-all
>>> +gc/stress/TestReclaimStringsLeaksMemory.java            8207267   generic-all
>>>    runtime/exceptionMsgs/AbstractMethodError/AbstractMethodErrorTest.java        8222582 generic-all
>> -- Igor
>>> On Jul 8, 2020, at 11:34 AM, Vladimir Kozlov <vladimir.kozlov at oracle.com> wrote:
>>> 
>>> Thank you, Igor
>>> 
>>> I got the difference between `vm.gc` and `vm.gc.X`.
>>> 
>>> In this case TestReclaimStringsLeaksMemory.java should be put into ProblemList-graal.txt with 8207267 to enable it with libgraal.  Current usage of !vm.graal.enabled in test is to skip this test with Java Graal because its effect on Java heap.
>>> 
>>> On 7/7/20 8:30 PM, Igor Ignatyev wrote:
>>>> Hi Vladimir,
>>>> thanks for your review!
>>>> `vm.gc` and `vm.gc.X`-s are different beasts (and admittedly, they confuse people a lot), `vm.gc` is set to "X", by jtreg itself, only if there is UseXGC in vm flags, otherwise it's "null". `vm.gc.X` are set by VMProps class, and you can have more than one vm.gc.X == true, as vm.gc.X means that X gc is supported by JVM and it can be selected; so if there are no Use.*GC in vm flags, vm.gc.X will yield true for all GCs which JVM was built with; if one of UseXGC is provided, only corresponding vm.gc.X is true, and all others are false. so to answer your questions, yes `vm.gc` can be "null" (if there are no Use.*GC) , and yes `vm.gc.Z & vm.gc.Serial & vm.gc == null` can be true (if there are no Use.*GC and JVM supports both Z and Serial GCs).
>>> 
>>> Interesting. I thought vmGC will list only one selected GC. That explains requires in TestZGCWithCDS.java.
>>> 
>>> You only need to add TestReclaimStringsLeaksMemory.java into ProblemList-graal.txt.
>>> 
>>> Thanks,
>>> Vladimir
>>> 
>>>> Thanks,
>>>> -- Igor
>>>>> On Jul 7, 2020, at 8:00 PM, Vladimir Kozlov <vladimir.kozlov at oracle.com> wrote:
>>>>> 
>>>>> Nice clean up, Igor
>>>>> 
>>>>> test/hotspot/jtreg/gc/stress/TestReclaimStringsLeaksMemory.java
>>>>> 
>>>>> Do we even can have vm.gc=="null" based on code in VMProps.java? At least some GC should be selected ergonomically even if non is specified on command line.
>>>>> 
>>>>> - * @requires vm.gc=="null" & !vm.graal.enabled & !vm.debug
>>>>> + * @requires vm.gc == "null"
>>>>> + * @requires !vm.debug
>>>>> 
>>>>> 
>>>>> test/hotspot/jtreg/runtime/cds/appcds/TestZGCWithCDS.java
>>>>> 
>>>>> Does next combination of @requires ever work? I thought such sequence means 'AND' operation on all such conditions.
>>>>> 
>>>>>  * @requires vm.gc.Z
>>>>>  * @requires vm.gc.Serial
>>>>>  * @requires vm.gc == null
>>>>> 
>>>>> 
>>>>> Thanks,
>>>>> Vladimir
>>>>> 
>>>>> On 7/7/20 5:38 PM, Igor Ignatyev wrote:
>>>>>> http://cr.openjdk.java.net/~iignatyev/8249000/webrev.00/
>>>>>>> 241 lines changed: 34 ins; 5 del; 202 mod;
>>>>>> Hi all,
>>>>>> could you please review the patch which modifies requires/VMProps to set vm.gc.X to false if Graal is selected and X GC isn't supported by Graal?
>>>>>> the patch also replaces @requires similar to `vm.gc.X & !vm.graal.enabled` w/ `vm.gc.X` where it's applicable.
>>>>>> from JBS:
>>>>>>> not all GCs are supported by Graal JIT, which leads to failures like JDK-8247527 and boilerplate fixes like replacing all `@requires vm.gc.Z` w/ `@requires vm.gc.Z & !vm.graal.enabled`.
>>>>>>> 
>>>>>>> as vm.gc.X means that X GC can be selected, it would be more natural, less surprising, and much more clear to have it true if the selected JIT supports the said X GC.
>>>>>> webrev: http://cr.openjdk.java.net/~iignatyev/8249000/webrev.00/
>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8249000
>>>>>> testing: test/hotspot/jtreg/{gc,compiler,runtime,serviceability} on {linux,windows,macos}-x64 w/ and w/o Graal as JIT
>>>>>> Thanks,
>>>>>> -- Igor



From shade at redhat.com  Mon Jul 13 05:40:48 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Mon, 13 Jul 2020 07:40:48 +0200
Subject: RFR (S) 8249137: Remove CollectedHeap::obj_size
In-Reply-To: <f68ba7d4-2693-57c4-59ef-6b0c04b3260c@oracle.com>
References: <a9bd4c4d-2d6b-733e-abbf-90e873c70951@redhat.com>
 <70984ebf-2717-c7b3-7076-12e2c8c7515c@oracle.com>
 <a2432ff6-79b2-aefc-0450-d1c2b3cabd22@redhat.com>
 <99b679ec-fcae-6f30-1186-1cf8de809c6f@oracle.com>
 <f68ba7d4-2693-57c4-59ef-6b0c04b3260c@oracle.com>
Message-ID: <8fea1a3f-8d62-4408-8f07-b92af35b6632@redhat.com>

On 7/12/20 9:08 PM, Chris Plummer wrote:
> Looks good to me.
Thanks! Pushed.

-- 
-Aleksey


From shade at redhat.com  Mon Jul 13 09:09:46 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Mon, 13 Jul 2020 11:09:46 +0200
Subject: RFR (S) 8249230: Shenandoah: assertion failure with -XX:-ResizeTLAB
Message-ID: <2d76df7b-d88f-061b-5aad-c2dfad47c94c@redhat.com>

Bug:
  https://bugs.openjdk.java.net/browse/JDK-8249230

This is caused by recent refactoring, JDK-8247845. Fix:
  https://cr.openjdk.java.net/~shade/8249230/webrev.01/

Testing: hotspot_gc_shenandoah; new test; tier1 with Shenandoah

-- 
Thanks,
-Aleksey


From rkennke at redhat.com  Mon Jul 13 09:37:55 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Mon, 13 Jul 2020 11:37:55 +0200
Subject: RFR (S) 8249230: Shenandoah: assertion failure with
 -XX:-ResizeTLAB
In-Reply-To: <2d76df7b-d88f-061b-5aad-c2dfad47c94c@redhat.com>
References: <2d76df7b-d88f-061b-5aad-c2dfad47c94c@redhat.com>
Message-ID: <CAAN-Kyiwq61c+qjktrsK8E8fKJc=ewhLj6vTazBw3ZhF4=82DA@mail.gmail.com>

Very good! Thank you!

Roman


Aleksey Shipilev <shade at redhat.com> schrieb am Mo., 13. Juli 2020, 11:10:

> Bug:
>   https://bugs.openjdk.java.net/browse/JDK-8249230
>
> This is caused by recent refactoring, JDK-8247845. Fix:
>   https://cr.openjdk.java.net/~shade/8249230/webrev.01/
>
> Testing: hotspot_gc_shenandoah; new test; tier1 with Shenandoah
>
> --
> Thanks,
> -Aleksey
>
>


From shade at redhat.com  Mon Jul 13 09:56:03 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Mon, 13 Jul 2020 11:56:03 +0200
Subject: RFR (S) 8249230: Shenandoah: assertion failure with
 -XX:-ResizeTLAB
In-Reply-To: <CAAN-Kyiwq61c+qjktrsK8E8fKJc=ewhLj6vTazBw3ZhF4=82DA@mail.gmail.com>
References: <2d76df7b-d88f-061b-5aad-c2dfad47c94c@redhat.com>
 <CAAN-Kyiwq61c+qjktrsK8E8fKJc=ewhLj6vTazBw3ZhF4=82DA@mail.gmail.com>
Message-ID: <65732704-e3e2-1cee-a367-97019e814992@redhat.com>

On 7/13/20 11:37 AM, Roman Kennke wrote:
> Very good! Thank you!

No problem, I should have done that in the original change. Pushed.

-- 
Thanks,
-Aleksey


From igor.ignatyev at oracle.com  Mon Jul 13 17:32:19 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Mon, 13 Jul 2020 10:32:19 -0700
Subject: RFR [15] : 8249033 : clean up FileInstaller $test.src $cwd in
 vmTestbase_vm_metaspace tests
Message-ID: <BAEA719A-AE50-4D41-9202-AB40EA2370A7@oracle.com>

http://cr.openjdk.java.net/~iignatyev//8249033/webrev.00/
> 47 lines changed: 0 ins; 32 del; 15 mod; 

Hi all,

could you please review the patch which removes `FileInstaller . .` jtreg action from :vmTestbase_vm_metaspace tests?
from the main issue(8204985):
> all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests.

as none of these tests need FileInstaller, the patch is as simple as `ag -l  '@run driver jdk.test.lib.FileInstaller . .' vmTestbase/metaspace/ xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}`.

testing: :vmTestbase_vm_metaspace on linux-x64
webrev: http://cr.openjdk.java.net/~iignatyev//8249033/webrev.00
JBS: https://bugs.openjdk.java.net/browse/JDK-8249033

Thanks,
-- Igor

From david.holmes at oracle.com  Tue Jul 14 02:44:24 2020
From: david.holmes at oracle.com (David Holmes)
Date: Tue, 14 Jul 2020 12:44:24 +1000
Subject: RFR [15] : 8249033 : clean up FileInstaller $test.src $cwd in
 vmTestbase_vm_metaspace tests
In-Reply-To: <BAEA719A-AE50-4D41-9202-AB40EA2370A7@oracle.com>
References: <BAEA719A-AE50-4D41-9202-AB40EA2370A7@oracle.com>
Message-ID: <b38faba0-05a6-e60c-acd7-d7bb4869ab11@oracle.com>

Looks good!

Thanks,
David

On 14/07/2020 3:32 am, Igor Ignatyev wrote:
> http://cr.openjdk.java.net/~iignatyev//8249033/webrev.00/
>> 47 lines changed: 0 ins; 32 del; 15 mod;
> 
> Hi all,
> 
> could you please review the patch which removes `FileInstaller . .` jtreg action from :vmTestbase_vm_metaspace tests?
> from the main issue(8204985):
>> all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests.
> 
> as none of these tests need FileInstaller, the patch is as simple as `ag -l  '@run driver jdk.test.lib.FileInstaller . .' vmTestbase/metaspace/ xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}`.
> 
> testing: :vmTestbase_vm_metaspace on linux-x64
> webrev: http://cr.openjdk.java.net/~iignatyev//8249033/webrev.00
> JBS: https://bugs.openjdk.java.net/browse/JDK-8249033
> 
> Thanks,
> -- Igor
> 


From igor.ignatyev at oracle.com  Tue Jul 14 18:18:20 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Tue, 14 Jul 2020 11:18:20 -0700
Subject: RFR [15] : 8249033 : clean up FileInstaller $test.src $cwd in
 vmTestbase_vm_metaspace tests
In-Reply-To: <b38faba0-05a6-e60c-acd7-d7bb4869ab11@oracle.com>
References: <BAEA719A-AE50-4D41-9202-AB40EA2370A7@oracle.com>
 <b38faba0-05a6-e60c-acd7-d7bb4869ab11@oracle.com>
Message-ID: <B596EDF2-72DB-4C1B-9266-62CDAD2539FF@oracle.com>

Thanks David,

pushed to jdk15.

-- Igor

> On Jul 13, 2020, at 7:44 PM, David Holmes <david.holmes at oracle.com> wrote:
> 
> Looks good!
> 
> Thanks,
> David
> 
> On 14/07/2020 3:32 am, Igor Ignatyev wrote:
>> http://cr.openjdk.java.net/~iignatyev//8249033/webrev.00/
>>> 47 lines changed: 0 ins; 32 del; 15 mod;
>> Hi all,
>> could you please review the patch which removes `FileInstaller . .` jtreg action from :vmTestbase_vm_metaspace tests?
>> from the main issue(8204985):
>>> all vmTestbase tests have '@run driver jdk.test.lib.FileInstaller . .' to mimic old test harness behavior and copy all files from a test source directory to a current work directory. some tests depend on this step, so we need 1st identify such tests and then either rewrite them not to have this dependency or leave FileInstaller only in these tests.
>> as none of these tests need FileInstaller, the patch is as simple as `ag -l  '@run driver jdk.test.lib.FileInstaller . .' vmTestbase/metaspace/ xargs -I{} gsed -i '/@run driver jdk.test.lib.FileInstaller \. \./d' {}`.
>> testing: :vmTestbase_vm_metaspace on linux-x64
>> webrev: http://cr.openjdk.java.net/~iignatyev//8249033/webrev.00
>> JBS: https://bugs.openjdk.java.net/browse/JDK-8249033
>> Thanks,
>> -- Igor



From coleen.phillimore at oracle.com  Tue Jul 14 21:20:37 2020
From: coleen.phillimore at oracle.com (coleen.phillimore at oracle.com)
Date: Tue, 14 Jul 2020 17:20:37 -0400
Subject: [16] RFR: 8248391: Unify handling of all OopStorage instances in
 weak root processing
In-Reply-To: <3A6A85AC-50A7-41AF-AE7E-9967C5B43AF9@oracle.com>
References: <8e831a42-61b0-1605-ea71-09f97d82f328@oracle.com>
 <11EE1DA7-3265-4A40-BA37-95B66B129F50@oracle.com>
 <eb24f5f4-fdcc-9f48-99d3-ff7a0ac9f86b@oracle.com>
 <3A6A85AC-50A7-41AF-AE7E-9967C5B43AF9@oracle.com>
Message-ID: <af187f0b-eb64-6dd2-2e35-c12f1b752ee5@oracle.com>


I really like the runtime changes in this patch, and the addition of the 
StringTable clearing test.? Components that wish to have their own 
OopStorages will create and own them.? I didn't review the GC changes.

Coleen

On 7/7/20 9:47 AM, Kim Barrett wrote:
>> On Jun 30, 2020, at 12:35 PM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
>>
>> I talked to kim about this point:
>>
>> "The GC (in particular, OopStorage) shouldn't know about client
>> subsystems in order to record the notification function"
>>
>> The current OopStorageSet implementation already knows about the different client subsystems, therefore I though that adding the notification registration there actually made sense. I've seen the alternative where the notification registration happens in the client subsystem, and that patch didn't look that great either, IMHO.
>>
>> So, I decided to create a PoC to (almost) completely decouple the client subsystems and the OopStorageSet. With the patch, the OopStorageSet doesn't know anything about the existing subsystems. It only knows about the number of weak and strong OopStorages in the JVM. The need to know the exact numbers arise from the wish to statically know that number during compile time. I'm not sure it's important to keep this static nature of the number of OopStorages, instead of using a more dynamic approach, but I've left this as is for now.
>>
>> https://cr.openjdk.java.net/~stefank/8248391/webrev.02.delta/
>> https://cr.openjdk.java.net/~stefank/8248391/webrev.02
> I have some comments, but because of vacations I'm providing them as a
> further patch on top of Stefan's webrev.02, plus the description below.
> (I'm abandoning the earlier never-published prototype derived from Erik's
> changes in favor of using Stefan's additional PoC plus further changes based
> on it.)  If this all gets reviewed before folks are back, I'll deal with
> pushing, with Erik and Stefan as contributors.
>
> full: https://cr.openjdk.java.net/~kbarrett/8248391/open.03/
> incr: https://cr.openjdk.java.net/~kbarrett/8248391/open.03.inc/
>
> Testing:
> mach5 tier1- TBD
> Locally ran new StringTableCleaningTest.java with a duration of 1/2 hour for
> each collector (including Shenandoah) except Epsilon.
>
> (Note that the baseline for the incremental patch is a slight modification
> of Stefan's webrev.02, to deal with a recent graal update that moved JVMCI's
> interface to OopStorage from jvmci.cpp to jvmciRuntime.cpp.)
>
>> One problem with the current state of this patch is that during G1's heap initialization, it wants to eagerly figure out all OopStorage names. The problem is that with my patch OopStorages are initialized together with its subsystem, and the StringTable and ResolvedMethodTable are initialized after the heap. It's a bit unfortunate, and I would like to find a good solution for that, but for this PoC I've changed the G1Policy::phase_times to lazily created phase times instance. This moves the allocations to the first GC. I'm open for suggestions on what do do about that part.
> I haven't come up with anything that seems better than Stefan's lazy
> allocation of the G1GCPhaseTimes object to deal with the ordering problem
> with subsystem registration of the OopStorage objects.  So I'm leaving that
> as is, at least for now.
>
> I think the proposed terminology for the GC -> OopStorage owner callback was
> too vague and generic; "notify" and "notification" are relatively
> content-free names.  So I've renamed the functions and types.  Also, I think
> "can_notify" should be "should_notify" (and then adjusted for the generic
> "notify").
>
> I've also changed (and documented) the semantics; the number of dead
> reported for an iteration is the number of entries skipped because they were
> already NULL plus the number cleared by the iteration.  (So the total dead,
> not just the recently deceased.)  That's the meaning in the old reporting
> mechanism.  Earlier versions of this change instead invoked the callback
> with just the number of newly cleared entries, with it being up to the
> receiving subsystem to accumulate the total.  But there are race conditions
> between GC and concurrent subsystem cleanup (like the ServiceThread's
> concurrent cleaning of the StringTable) that make such accumulation by the
> client potentially wildly inaccurate, leading to excess re-cleanings.  (Even
> with the callback receiving the total dead there are races, but I think the
> potential for significant inaccuracies leading to time wasting are much
> less.)
>
> A possible further change that I've left for later is that the
> WeakProcessorPhaseTimeTracker gets just the cleared (newly dead) count.
> (That hasn't been touched by any version of this change.)  I think the total
> dead would be a more useful statistic for the associated logging.
>
> The (anonymous) enum of OopStorageSet iterator indices no longer serves any
> purpose; I've changed them to be ordinary in-class constants.  With the
> original code the enum was used to automatically assign values to some of
> the enumerators, but with the removal of named indices for each storage,
> there was only one autmatically assigned enumerator.
>
> I've changed WeakProcessor::Task to also use the new
> OopStorageSetWeakParState.  This involved adding support for client access
> to the states, similar to what's in the strong parstate.  (I'm not sure the
> weak parstate should provide an oops_do that counts and reports the dead,
> but I've left that for now.)
>
> Fixed a "bug" that WeakProcessor::oops_do(OopClosure*) counted and reported
> dead entries, even though it never clears entries.  It is used for
> iterations that shouldn't be reporting dead, such as for the adjust pointers
> phase of mark-sweep-compact.
>
> Besides adapting to the above changes, I moved StringTable's clearing of the
> _has_work flag to the end of its do_concurrent_work.  This avoids queuing
> a new cleanup request in the middle of a concurrent cleanup that might
> eliminate the need for further cleaning.  We could try to re-evaluate at the
> end of the cleanup, but instead leave that to the next GC to trigger a new
> round of cleanup if needed.  Also removed some inconsistencies between
> ResolvedMethodTable and StringTable.
>
> I've also added a test that beats on StringTable allocation and cleaning.
>



From kim.barrett at oracle.com  Tue Jul 14 22:49:45 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Tue, 14 Jul 2020 18:49:45 -0400
Subject: [16] RFR: 8248391: Unify handling of all OopStorage instances in
 weak root processing
In-Reply-To: <af187f0b-eb64-6dd2-2e35-c12f1b752ee5@oracle.com>
References: <8e831a42-61b0-1605-ea71-09f97d82f328@oracle.com>
 <11EE1DA7-3265-4A40-BA37-95B66B129F50@oracle.com>
 <eb24f5f4-fdcc-9f48-99d3-ff7a0ac9f86b@oracle.com>
 <3A6A85AC-50A7-41AF-AE7E-9967C5B43AF9@oracle.com>
 <af187f0b-eb64-6dd2-2e35-c12f1b752ee5@oracle.com>
Message-ID: <EB433FAF-73B9-4E0F-AA73-C8D3A1C4BE92@oracle.com>

> On Jul 14, 2020, at 5:20 PM, coleen.phillimore at oracle.com wrote:
> 
> 
> I really like the runtime changes in this patch, and the addition of the StringTable clearing test.  Components that wish to have their own OopStorages will create and own them.  I didn't review the GC changes.

Thanks.



From linzang at tencent.com  Wed Jul 15 09:12:31 2020
From: linzang at tencent.com (=?utf-8?B?bGluemFuZyjoh6fnkLMp?=)
Date: Wed, 15 Jul 2020 09:12:31 +0000
Subject: RFR(L): 8215624: add parallel heap inspection support for jmap
 histo(G1)(Internet mail)
In-Reply-To: <C1CE05FA-3E6F-4ECC-8C72-9A4A2B923FB8@tencent.com>
References: <02415944-2238-4430-A7C2-3625264A5505@amazon.com>
 <C1CE05FA-3E6F-4ECC-8C72-9A4A2B923FB8@tencent.com>
Message-ID: <CDF4486D-77BA-4F37-8596-5DAA35951801@tencent.com>

 Upload a new webrev at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07/
 It fix a potential issue that unexpected number of threads maybe calculated for "parallel" option of jmap -histo in container. 
As shown at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07-delta/src/hotspot/share/services/attachListener.cpp.udiff.html

############### attachListener.cpp ####################
@@ -252,11 +252,11 @@
 static jint heap_inspection(AttachOperation* op, outputStream* out) {
   bool live_objects_only = true;   // default is true to retain the behavior before this change is made
   outputStream* os = out;   // if path not specified or path is NULL, use out
   fileStream* fs = NULL;
   const char* arg0 = op->arg(0);
-  uint parallel_thread_num = MAX(1, os::processor_count() * 3 / 8); // default is less than half of processors.
+  uint parallel_thread_num = MAX(1, os::initial_active_processor_count() * 3 / 8); // default is less than half of processors.
   if (arg0 != NULL && (strlen(arg0) > 0)) {
     if (strcmp(arg0, "-all") != 0 && strcmp(arg0, "-live") != 0) {
       out->print_cr("Invalid argument to inspectheap operation: %s", arg0);
       return JNI_ERR;
     }
###################################################

Thanks.
 
BRs,
Lin

?On 2020/7/9, 3:22 PM, "linzang(??)" <linzang at tencent.com> wrote:

    Hi Paul, 
        Thanks for reviewing!
        >> 
        >>     I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp.
        >>
        The reason I made the change in Jmap.java that compose all arguments as 1 string , instead of passing 3 argments, is to avoid the compatibility issue, as we discussed in http://mail.openjdk.java.net/pipermail/serviceability-dev/2019-February/thread.html#27240.  The root cause of the compatibility issue is because max argument count in HotspotVirtualMachineImpl.java and attachlistener.cpp need to be enlarged (changes like http://hg.openjdk.java.net/jdk/jdk/rev/e7cf035682e3#l2.1) when jmap has more than 3 arguments. But if user use an old jcmd/jmap tool, it may stuck at socket read(), because the "max argument count" don't match.  
         I re-checked this change, the argument count of jmap histo is equal to 3 (live, file, parallel), so it can work normally even without the change of passing argument. But I think we have to face the problem if more arguments is added in jcmd alike tools later, not sure whether it should be sloved (or a workaround) in this changeset. 

        And here are the lastest webrev and delta:
        http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06/
        http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06-delta/

    Cheers,
    Lin

    On 2020/7/7, 5:57 AM, "Hohensee, Paul" <hohensee at amazon.com> wrote:

        I'd like to see this feature added. :)

        The CSR looks good, as does the basic parallel inspection algorithm. Stefan's done the GC part, so I'll stick to the non-GC part (fwiw, the GC part lgtm).

        I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp.

        heapInspection.hpp:

        _shared_miss_count (s/b _missed_count, see below) isn't a size, so it should be a uint instead of a size_t. Same with the new parallel_thread_num argument to heap_inspection() and populate_table().

        Comment copy-edit:
        +// Parallel heap inspection task. Parallel inspection can fail due to
        +// a native OOM when allocating memory for TL-KlassInfoTable.
        +// _success will be set false on an OOM, and serial inspection tried.

        _shared_miss_count should be _missed_count to match the missed_count() getter, or rename missed_count() to be shared_miss_count(). Whichever way you go, the field type should match the getter result type: uint is reasonable.

        heapInspection.cpp:

        You might use ResourceMark twice in populate_table, separately for the parallel attempt and the serial code. If the parallel attempt fails and available memory is low, it would be good to clean up the memory used by the parallel attempt before doing the serial code.

        Style nit in KlassInfoTable::merge_entry(). I'd line up the definitions of k and elt, so "k" is even with "elt". And, because it's two lines shorter, I'd replace
        +  } else {
        +    return false;
        +  }
        with
        +  return false;

        KlassInfoTableMergeClosure.is_success() should be just success() (i.e., no "is_" prefix) because it's a getter.

        I'd reorganize the code in populate_table() to make it more clear, vis (I changed _shared_missed_count to _missed_count)
        +  if (cit.allocation_failed()) {
        +    // fail to allocate memory, stop parallel mode
        +    Atomic::store(&_success, false);
        +    return;
        +  }
        +  RecordInstanceClosure ric(&cit, _filter);
        +  _poi->object_iterate(&ric, worker_id);
        +  missed_count = ric.missed_count();
        +  {
        +    MutexLocker x(&_mutex);
        +    merge_success = _shared_cit->merge(&cit);
        +  }
        +  if (merge_success) {
        +    Atomic::add(&_missed_count, missed_count);
        +  else {
        +    Atomic::store(&_success, false);
        +  }

        Thanks,
        Paul

        On 6/29/20, 7:20 PM, "linzang(??)" <linzang at tencent.com> wrote:

            Dear All,
                    Sorry to bother again, I just want to make sure that is this change worth to be continue to work on? If decision is made to not. I think I can drop this work and stop asking for help reviewing...
                    Thanks for all your help about reviewing this previously.

            BRs,
            Lin

            On 2020/5/9, 3:47 PM, "linzang(??)" <linzang at tencent.com> wrote:

                Dear All,
                       May I ask your help again for review the latest change?  Thanks!

                BRs,
                Lin

                On 2020/4/28, 1:54 PM, "linzang(??)" <linzang at tencent.com> wrote:

                    Hi Stefan,
                      >>  - Adding Atomic::load/store.
                      >>  - Removing the time measurement in the run_task. I renamed G1's function
                      >>  to run_task_timed. If we need this outside of G1, we can rethink the API
                      >>  at that point.
                       >>  - ZGC style cleanups
                       Thanks for revising the patch,  they are all good to me, and I have made a tiny change based on it:
                           http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04/
                           http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04-delta/
                      it reduce the scope of mutex in ParHeapInspectTask, and delete unnecessary comments.

                    BRs,
                    Lin

                    On 2020/4/27, 4:34 PM, "Stefan Karlsson" <stefan.karlsson at oracle.com> wrote:

                        Hi Lin,

                        On 2020-04-26 05:10, linzang(??) wrote:
                        > Hi Stefan and Paul?
                        >      I have made a new patch based on your comments and Stefan's Poc code:
                        >      Webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03/
                        >      Delta(based on Stefan's change:) : http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03-delta/webrev_03-delta/

                        Thanks for providing a delta patch. It makes it much easier to look at,
                        and more likely for reviewers to continue reviewing.

                        I'm going to continue focusing on the GC parts, and leave the rest to
                        others to review.

                        >
                        >      And Here are main changed I made and want to discuss with you:
                        >      1.  changed"parallelThreadNum=" to "parallel=" for jmap -histo options.
                        >      2.  Add logic to test where parallelHeapInspection is fail, in heapInspection.cpp
                        >            This is because the parHeapInspectTask create thread local KlassInfoTable in it's work() method, and this may fail because of native OOM, in this case, the parallel should fail and serial heap inspection can be tried.
                        >            One more thing I want discuss with you is about the member "_success" of parHeapInspectTask, when native OOM happenes, it is set to false. And since this "set" operation can be conducted in multiple threads, should it be atomic ops?  IMO, this is not necessary because "_success" can only be set to false, and there is no way to change it from back to true after the ParHeapInspectTask instance is created, so it is save to be non-atomic, do you agree with that?

                        In these situations you should be using the Atomic::load/store
                        primitives. We're moving toward a later C++ standard were data races are
                        considered undefined behavior.

                        >     3. make CollectedHeap::run_task() be an abstract virtual func, so that every subclass of collectedHeap should support it, so later implementation of new collectedHeap will not miss the "parallel" features.
                        >           The problem I want to discuss with you is about epsilonHeap and SerialHeap, as they may not need parallel heap iteration, so I only make task->work(0), in case the run_task() is invoked someway in future. Another way is to left run_task()  unimplemented, which one do you think is better?

                        I don't have a strong opinion about this.

                          And also please help take a look at the zHeap, as there is a class
                        zTask that wrap the abstractGangTask, and the collectedHeap::run_task()
                        only accept  AbstraceGangTask* as argument, so I made a delegate class
                        to adapt it , please see src/hotspot/share/gc/z/zHeap.cpp.
                        >
                        >        There maybe other better ways to sovle the above problems, welcome for any comments, Thanks!

                        I've created a few cleanups and changes on top of your latest patch:

                        https://cr.openjdk.java.net/~stefank/8215624/webrev.02.delta
                        https://cr.openjdk.java.net/~stefank/8215624/webrev.02

                        - Adding Atomic::load/store.
                        - Removing the time measurement in the run_task. I renamed G1's function
                        to run_task_timed. If we need this outside of G1, we can rethink the API
                        at that point.
                        - ZGC style cleanups

                        Thanks,
                        StefanK

                        >
                        > BRs,
                        > Lin
                        >
                        > On 2020/4/23, 11:08 AM, "linzang(??)" <linzang at tencent.com> wrote:
                        >
                        >      Thanks Paul! I agree with using "parallel", will make the update in next patch, Thanks for help update the CSR.
                        >
                        >      BRs,
                        >      Lin
                        >
                        >      On 2020/4/23, 4:42 AM, "Hohensee, Paul" <hohensee at amazon.com> wrote:
                        >
                        >          For the interface, I'd use "parallel" instead of "parallelThreadNum". All the other options are lower case, and it's a lot easier to type "parallel". I took the liberty of updating the CSR. If you're ok with it, you might want to change variable names and such, plus of course JMap.usage.
                        >
                        >          Thanks,
                        >          Paul
                        >
                        >          On 4/22/20, 2:29 AM, "serviceability-dev on behalf of linzang(??)" <serviceability-dev-bounces at openjdk.java.net on behalf of linzang at tencent.com> wrote:
                        >
                        >              Dear Stefan,
                        >
                        >                      Thanks a lot! I agree with you to decouple the heap inspection code with GC's.
                        >                      I will start  from your POC code, may discuss with you later.
                        >
                        >
                        >              BRs,
                        >              Lin
                        >
                        >              On 2020/4/22, 5:14 PM, "Stefan Karlsson" <stefan.karlsson at oracle.com> wrote:
                        >
                        >                  Hi Lin,
                        >
                        >                  I took a look at this earlier and saw that the heap inspection code is
                        >                  strongly coupled with the CollectedHeap and G1CollectedHeap. I'd prefer
                        >                  if we'd abstract this away, so that the GCs only provide a "parallel
                        >                  object iteration" interface, and the heap inspection code is kept elsewhere.
                        >
                        >                  I started experimenting with doing that, but other higher-priority (to
                        >                  me) tasks have had to take precedence.
                        >
                        >                  I've uploaded my work-in-progress / proof-of-concept:
                        >                    https://cr.openjdk.java.net/~stefank/8215624/webrev.01.delta/
                        >                    https://cr.openjdk.java.net/~stefank/8215624/webrev.01/
                        >
                        >                  The current code doesn't handle the lifecycle (deletion) of the
                        >                  ParallelObjectIterators. There's also code left unimplemented in around
                        >                  CollectedHeap::run_task. However, I think this could work as a basis to
                        >                  pull out the heap inspection code out of the GCs.
                        >
                        >                  Thanks,
                        >                  StefanK
                        >
                        >                  On 2020-04-22 02:21, linzang(??) wrote:
                        >                  > Dear all,
                        >                  >       May I ask you help to review? This RFR has been there for quite a while.
                        >                  >       Thanks!
                        >                  >
                        >                  > BRs,
                        >                  > Lin
                        >                  >
                        >                  > > On 2020/3/16, 5:18 PM, "linzang(??)" <linzang at tencent.com> wrote:>
                        >                  >
                        >                  >>    Just update a new path, my preliminary measure show about 3.5x speedup of jmap histo on a nearly full 4GB G1 heap (8-core platform with parallel thread number set to 4).
                        >                  >>     webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_02/
                        >                  >>     bug: https://bugs.openjdk.java.net/browse/JDK-8215624
                        >                  >>     CSR: https://bugs.openjdk.java.net/browse/JDK-8239290
                        >                  >>     BRs,
                        >                  >>       Lin
                        >                  >>       > On 2020/3/2, 9:56 PM, "linzang(??)" <linzang at tencent.com> wrote:
                        >                  >>       >
                        >                  >>       >    Dear all,
                        >                  >>       >          Let me try to ease the reviewing work by some explanation :P
                        >                  >>       >          The patch's target is to speed up jmap -histo for heap iteration, from my experience it is necessary for large heap investigation. E.g in bigData scenario I have tried to conduct jmap -histo against 180GB heap, it does take quite a while.
                        >                  >>       >          And if my understanding is corrent, even the jmap -histo without "live" option does heap inspection with heap lock acquired. so it is very likely to block mutator thread in allocation-sensitive scenario. I would say the faster the heap inspection does, the shorter the mutator be blocked. This is parallel iteration for jmap is necessary.
                        >                  >>       >          I think the parallel heap inspection should be applied to all kind of heap. However, consider the heap layout are different for  GCs, much time is required to understand all kinds of the heap layout to make the whole change. IMO, It is not wise to have a huge patch for the whole solution at once, and it is even harder to review it. So I plan to implement it incrementally, the first patch (this one) is going to confirm the implemention detail of how jmap accept the new option, passes it to attachListener of the jvm process and then how to make the parallel inspection closure be generic enough to make it easy to extend to different heap layout. And also how to implement the heap inspection in specific gc's heap. This patch use G1's heap as the begining.
                        >                  >>       >          This patch actually do several things:
                        >                  >>       >          1. Add an option "parallelThreadNum=<N>" to jmap -histo, the default behavior is to set N to 0, means let's JVM decide how many threads to use for heap inspection. Set this option to 1 will disable parallel heap inspection. (more details in CSR: https://bugs.openjdk.java.net/browse/JDK-8239290)
                        >                  >>       >          2. Make a change in how Jmap passing arguments, changes in http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java.udiff.html, originally it pass options as separate arguments to attachListener, this patch change to that all options be compose to a single string. So the arg_count_max in attachListener.hpp do not need to be changed, and hence avoid the compatibility issue, as disscussed at https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-March/027334.html
                        >                  >>       >         3. Add an abstract class ParHeapInspectTask in heapInspection.hpp / heapInspection.cpp, It's work(uint worker_id) method prepares the data structure (KlassInfoTable) need for every parallel worker thread, and then call do_object_iterate_parallel() which is heap specific implementation. I also added some machenism in KlassInfoTable to support parallel iteration, such as merge().
                        >                  >>       >        4. In specific heap (G1 in this patch), create a subclass of ParHeapInspectTask, implement the do_object_iterate_parallel() for parallel heap inspection. For G1, it simply invoke g1CollectedHeap's object_iterate_parallel().
                        >                  >>       >        5. Add related test.
                        >                  >>       >        6. it may be easy to extend this patch for other kinds of heap by creating subclass of ParHeapInspectTask and implement the do_object_iterate_parallel().
                        >                  >>       >
                        >                  >>       >    Hope these info could help on code review and initate the discussion :-)
                        >                  >>       >    Thanks!
                        >                  >>       >
                        >                  >>       >    BRs,
                        >                  >>       >    Lin
                        >                  >>       >    >On 2020/2/19, 9:40 AM, "linzang(??)" <linzang at tencent.com> wrote:.
                        >                  >>       >    >
                        >                  >>       >    >  Re-post this RFR with correct enhancement number to make it trackable.
                        >                  >>       >    >  please ignore the previous wrong post. sorry for troubles.
                        >                  >>       >    >
                        >                  >>       >    >   webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/
                        >                  >>       >    >    Hi bug: https://bugs.openjdk.java.net/browse/JDK-8215624
                        >                  >>       >    >    CSR: https://bugs.openjdk.java.net/browse/JDK-8239290
                        >                  >>       >    >    --------------
                        >                  >>       >    >    Lin
                        >                  >>       >    >    >Hi Lin,
                        >                  >   >     >    >    >
                        >                  >>       >    >    >Could you, please, re-post your RFR with the right enhancement number in
                        >                  >>       >    >    >the message subject?
                        >                  >>       >    >    >It will be more trackable this way.
                        >                  >>       >    >    >
                        >                  >>       >    >    >Thanks,
                        >                  >>       >    >    >Serguei
                        >                  >>       >    >    >
                        >                  >>       >    >    >
                        >                  >>       >    >    >On 2/17/20 10:29 PM, linzang(??) wrote:
                        >                  >>       >    >    >> Dear David,
                        >                  >>       >    >    >>        Thanks a lot!
                        >                  >>       >    >    >>       I have updated the refined code to http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_01/.
                        >                  >>       >    >    >>        IMHO the parallel heap inspection can be extended to all kinds of heap as long as the heap layout can support parallel iteration.
                        >                  >>       >    >    >>        Maybe we can firstly use this webrev to discuss how to implement it, because I am not sure my current implementation is an appropriate way to communicate with collectedHeap, then we can extend the solution to other kinds of heap.
                        >                  >>       >    >    >>
                        >                  >>       >    >    >> Thanks,
                        >                  >>       >    >    >> --------------
                        >                  >>       >    >    >> Lin
                        >                  >>       >    >    >>> Hi Lin,
                        >                  >>       >    >    >>>
                        >                  >>       >    >    >>> Adding in hotspot-gc-dev as they need to see how this interacts with GC
                        >                  >>       >    >    >>> worker threads, and whether it needs to be extended beyond G1.
                        >                  >>       >    >    >>>
                        >                  >>       >    >   >>> I happened to spot one nit when browsing:
                        >                  >>       >    >    >>>
                        >                  >>       >    >    >>> src/hotspot/share/gc/shared/collectedHeap.hpp
                        >                  >>       >    >    >>>
                        >                  >>       >    >    >>> +   virtual bool run_par_heap_inspect_task(KlassInfoTable* cit,
                        >                  >>       >    >    >>> +                                          BoolObjectClosure* filter,
                        >                  >>       >    >    >>> +                                          size_t* missed_count,
                        >                  >>       >    >    >>> +                                          size_t thread_num) {
                        >                  >>       >    >    >>> +     return NULL;
                        >                  >>       >    >    >>>
                        >                  >>       >    >    >>> s/NULL/false/
                        >                  >>       >    >    >>>
                        >                  >>       >    >    >>> Cheers,
                        >                  >>       >    >    >>> David
                        >                  >   >     >    >    >>>
                        >                  >>       >    >    >>> On 18/02/2020 2:15 pm, linzang(??) wrote:
                        >                  >>       >    >    >>>> Dear All,
                        >                  >>       >    >    >>>>         May I ask your help to review the follow changes:
                        >                  >>       >    >    >>>>         webrev:
                        >                  >>       >    >    >>>> http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_00/
                        >                  >>       >    >    >>>>      bug: https://bugs.openjdk.java.net/browse/JDK-8215624
                        >                  >>       >    >    >>>>      related CSR: https://bugs.openjdk.java.net/browse/JDK-8239290
                        >                  >>       >    >    >>>>         This patch enable parallel heap inspection of G1 for jmap histo.
                        >                  >>       >    >    >>>>         my simple test shown it can speed up 2x of jmap -histo with
                        >                  >>       >    >    >>>> parallelThreadNum set to 2 for heap at ~500M on 4-core platform.
                        >                  >>       >    >    >>>>
                        >                  >>       >    >    >>>> ------------------------------------------------------------------------
                        >                  >>       >    >    >>>> BRs,
                        >                  >>       >    >    >>>> Lin
                        >                  >>       >    >    >> >
                        >                  >>       >    >    >
                        >                  >
                        >                  >
                        >                  >
                        >
                        >
                        >
                        >
                        >
                        >








From albert.m.yang at oracle.com  Wed Jul 15 17:46:12 2020
From: albert.m.yang at oracle.com (Albert Yang)
Date: Wed, 15 Jul 2020 10:46:12 -0700 (PDT)
Subject: RFR (S): 8245030: CardTable::precleaned_card is unused
Message-ID: <64d8977e-2091-4023-bd80-fc47e1edf713@default>

Hi,

https://bugs.openjdk.java.net/browse/JDK-8245030

http://cr.openjdk.java.net/~lkorinth/albert/8245030/0/

Tested: hotspot_gc

/Albert


From rs at jelastic.com  Wed Jul 15 19:06:47 2020
From: rs at jelastic.com (Ruslan Synytsky)
Date: Wed, 15 Jul 2020 21:06:47 +0200
Subject: Analyzing Efficiency Of Java Memory Usage
Message-ID: <CA++bR4P9W+GYhVen_iwp98=yJLU0H6bG+-8b1Bci-R=xtYL34g@mail.gmail.com>

Hi all, I would like to share some preliminary results related to Java
elasticity analysis. We collected metrics from about 1000 containers. The
goal was to get a rough picture of the memory usage efficiency within our
user base.These containers are running different software stacks, java
versions, dev or production workloads, and they belong to different end
users. Each container may have one or multiple Java processes running
inside and even additional auxiliary non Java processes.

At the following link you can find a dynamic chart that reflects memory
usage at specific period of time (it's one time measurement)
https://cs.demo.jelastic.com/chart&appid=cc492725f550fcd637ab8a7c1f9810c9
where
- X axis represent Hosts, each host is a separated container
- Y Axis:
  * OSTotal: total RAM available for container
  * Xmx: max heap limit
  * NonHeapCommitted+: committed non heap stacked on top of committed heap
memory
  * HeapCommitted: committed heap memory
  * HeapUsed: used heap memory
  * OSBuffCache+: OS memory in buffer / cache stacked on top of OS used
memory
  * OSUsed: OS used memory
  * Count: number of java processes running inside the host
  ---
  * Java Version per each Java process
  * GC type per each Java process
  * Java Agent: n/count - how many processes are using jelastic GC agent
 (was used before JEP 346)
  * G1PeriodicGCInterval: n/count - how many processes are using this new
option

The chart is ordered from left to right by the following sequence
OSTotal, Xmx, NonHeapCommitted, HeapCommitted,
HeapUsed, OSBuffCache, OSUsed.

In general, the results look good as the vast majority of java processes
are running in a quite efficient mode (small difference between used heap
and committed heap). There are some java processes that have a big
difference between used heap and committed heap (for example host 91 or
host 109), but such processes do not use memory usage optimization (no
configured GC agent and no G1PeriodicGCInterval) or they use non-compacting
GC. Interesting findings that a) sometimes committed heap memory is
higher than real OS used memory, and b) in many cases non heap committed
memory is as big as heap committed.

Please note, the analysis does not represent all possible use cases and
rather represent web applications and services that are hosted at local
cloud hosting providers. Also, most likely our analysis does not reflect
the reality outside of Jelastic, because by default we optimize Java
runtimes for the maximum efficiency in terms of memory usage, so I expect
that the real world situation is worse.



We are planning to add more containers to the analysis for getting an even
bigger picture, but also I'm looking for other companies that can join this
analysis. It will help the community to get a better understanding of the
real (diverse and heterogeneous) Java world. Please let me know if you are
willing to participate and I will share a bash script that collects the
memory usage metrics (it will not collect sensitive or personal information
as well as it's safe to run on productions).

Looking forward to getting your feedback.
Thanks
-- 
Ruslan Synytsky
CEO @ Jelastic Multi-Cloud PaaS


From leo.korinth at oracle.com  Wed Jul 15 20:44:44 2020
From: leo.korinth at oracle.com (Leo Korinth)
Date: Wed, 15 Jul 2020 22:44:44 +0200
Subject: RFR (S): 8245030: CardTable::precleaned_card is unused
In-Reply-To: <64d8977e-2091-4023-bd80-fc47e1edf713@default>
References: <64d8977e-2091-4023-bd80-fc47e1edf713@default>
Message-ID: <3b3e6b03-6f05-565c-e47e-ee4ebc45b544@oracle.com>

On 15/07/2020 19:46, Albert Yang wrote:
> Hi,
> 
> https://bugs.openjdk.java.net/browse/JDK-8245030
> 
> http://cr.openjdk.java.net/~lkorinth/albert/8245030/0/
This looks good to me, thanks for fixing this. I will sponsor this change for you.

Thanks,
Leo
> 
> Tested: hotspot_gc
> 
> /Albert
> 


From rkennke at redhat.com  Wed Jul 15 21:40:03 2020
From: rkennke at redhat.com (rkennke at redhat.com)
Date: Wed, 15 Jul 2020 23:40:03 +0200
Subject: RFR: 8249560: Shenandoah: Fix racy GC request handling
Message-ID: <62c8518e702fd11d7be34cc591f3c48bb9ad9ea7.camel@redhat.com>

See discussion in bug:

https://bugs.openjdk.java.net/browse/JDK-8249560

Webrev:
http://cr.openjdk.java.net/~rkennke/JDK-8249560/webrev.00/

Testing: hotspot_gc_shenandoah, tier1&tier2 tests with Shenandoah

Ok?

Thanks,
Roman



From kim.barrett at oracle.com  Thu Jul 16 01:21:05 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 15 Jul 2020 21:21:05 -0400
Subject: RFR (S): 8245030: CardTable::precleaned_card is unused
In-Reply-To: <64d8977e-2091-4023-bd80-fc47e1edf713@default>
References: <64d8977e-2091-4023-bd80-fc47e1edf713@default>
Message-ID: <AF17BB31-F507-4FBD-B7DB-77FB641D5CA2@oracle.com>

> On Jul 15, 2020, at 1:46 PM, Albert Yang <albert.m.yang at oracle.com> wrote:
> 
> Hi,
> 
> https://bugs.openjdk.java.net/browse/JDK-8245030
> 
> http://cr.openjdk.java.net/~lkorinth/albert/8245030/0/
> 
> Tested: hotspot_gc
> 
> /Albert

Looks good.



From shade at redhat.com  Thu Jul 16 08:17:22 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Thu, 16 Jul 2020 10:17:22 +0200
Subject: RFR: 8249560: Shenandoah: Fix racy GC request handling
In-Reply-To: <62c8518e702fd11d7be34cc591f3c48bb9ad9ea7.camel@redhat.com>
References: <62c8518e702fd11d7be34cc591f3c48bb9ad9ea7.camel@redhat.com>
Message-ID: <28e3cd9f-fc34-99e0-2e16-3c905ae1c9a7@redhat.com>

On 7/15/20 11:40 PM, rkennke at redhat.com wrote:
> See discussion in bug:
> 
> https://bugs.openjdk.java.net/browse/JDK-8249560
> 
> Webrev:
> http://cr.openjdk.java.net/~rkennke/JDK-8249560/webrev.00/

OK, looks good.

-- 
Thanks,
-Aleksey


From felix.yang at huawei.com  Thu Jul 16 09:01:48 2020
From: felix.yang at huawei.com (Yangfei (Felix))
Date: Thu, 16 Jul 2020 09:01:48 +0000
Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences
 between free chunk check and klass read
References: <DA41BE1DDCA941489001C7FBD7A8820EE7E63D86@dggeml527-mbx.china.huawei.com>
 <d6590e74-7ce2-70b9-7432-96b969686ab7@redhat.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E6466E@dggeml527-mbx.china.huawei.com>
 <8957BB9E-617D-4E24-9CC4-31AF7D01D12E@oracle.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E64ECD@dggeml527-mbx.china.huawei.com>
 <65012FED-2A93-4899-939E-887175AB07AC@oracle.com>
 <06646cf3-bc21-977f-96e2-1e9a974d1b00@redhat.com> 
Message-ID: <DA41BE1DDCA941489001C7FBD7A8820EE7E74B25@dggeml507-mbx.china.huawei.com>

Ping ... 
Could someone please take another look? Especially the two webrevs for 11u & 13u.

Thanks,
Felix

> -----Original Message-----
> From: Yangfei (Felix)
> Sent: Friday, July 10, 2020 7:44 PM
> To: 'Andrew Haley' <aph at redhat.com>; Kim Barrett
> <kim.barrett at oracle.com>
> Cc: jdk8u-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net;
> aarch64-port-dev at openjdk.java.net; jdk-updates-dev at openjdk.java.net
> Subject: RE: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory
> fences between free chunk check and klass read
> 
> Hi,
> 
> > -----Original Message-----
> > From: Andrew Haley [mailto:aph at redhat.com]
> > Sent: Wednesday, July 8, 2020 5:48 PM
> > To: Kim Barrett <kim.barrett at oracle.com>; Yangfei (Felix)
> > <felix.yang at huawei.com>
> > Cc: jdk8u-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net;
> > aarch64-port-dev at openjdk.java.net
> > Subject: Re: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory
> > fences between free chunk check and klass read
> >
> > On 08/07/2020 09:41, Kim Barrett wrote:
> > >> Since CMS is deprecated from JDK9, I am not sure if it's
> > >> appropriate to fix
> > this issue for those JDK9+ versions.
> > > Deprecated != unsupported.
> > >
> >
> > Yes, it must be done. Thanks.
> 
> CCing to jdk-updates-dev.
> 
> I have prepared another two webrevs for jdk11u and jdk13u:
>   Jdk11u-dev:  http://cr.openjdk.java.net/~fyang/8248851-11u/
>   Jdk13u-dev: http://cr.openjdk.java.net/~fyang/8248851-13u/
> 
> The only difference lies in copyright year update.  Both tiere1-3 tested on
> aarch64-linux-gnu.
> I will put jdk11u-fix-request and jdk13u-fix-request label on the issue if they
> are good.
> 
> I have also prepared a new webrev for jdk8u incorporating copyright year
> update:
>   Jdk8u-dev: http://cr.openjdk.java.net/~fyang/8248851-8u/
> 
> Thanks,
> Felix
> 


From david.holmes at oracle.com  Thu Jul 16 09:16:04 2020
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 16 Jul 2020 19:16:04 +1000
Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences
 between free chunk check and klass read
In-Reply-To: <DA41BE1DDCA941489001C7FBD7A8820EE7E74B25@dggeml507-mbx.china.huawei.com>
References: <DA41BE1DDCA941489001C7FBD7A8820EE7E63D86@dggeml527-mbx.china.huawei.com>
 <d6590e74-7ce2-70b9-7432-96b969686ab7@redhat.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E6466E@dggeml527-mbx.china.huawei.com>
 <8957BB9E-617D-4E24-9CC4-31AF7D01D12E@oracle.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E64ECD@dggeml527-mbx.china.huawei.com>
 <65012FED-2A93-4899-939E-887175AB07AC@oracle.com>
 <06646cf3-bc21-977f-96e2-1e9a974d1b00@redhat.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E74B25@dggeml507-mbx.china.huawei.com>
Message-ID: <48d0abe9-29a1-a67f-b8f0-c823e15ce110@oracle.com>

Hi Felix,

Seems to me that you want OrderAccess::loadload() barriers to order the 
loads, not OrderAccess::acquire(). You should only use acquire semantics 
to pair with a corresponding release operation.

Cheers,
David

On 16/07/2020 7:01 pm, Yangfei (Felix) wrote:
> Ping ...
> Could someone please take another look? Especially the two webrevs for 11u & 13u.
> 
> Thanks,
> Felix
> 
>> -----Original Message-----
>> From: Yangfei (Felix)
>> Sent: Friday, July 10, 2020 7:44 PM
>> To: 'Andrew Haley' <aph at redhat.com>; Kim Barrett
>> <kim.barrett at oracle.com>
>> Cc: jdk8u-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net;
>> aarch64-port-dev at openjdk.java.net; jdk-updates-dev at openjdk.java.net
>> Subject: RE: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory
>> fences between free chunk check and klass read
>>
>> Hi,
>>
>>> -----Original Message-----
>>> From: Andrew Haley [mailto:aph at redhat.com]
>>> Sent: Wednesday, July 8, 2020 5:48 PM
>>> To: Kim Barrett <kim.barrett at oracle.com>; Yangfei (Felix)
>>> <felix.yang at huawei.com>
>>> Cc: jdk8u-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net;
>>> aarch64-port-dev at openjdk.java.net
>>> Subject: Re: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory
>>> fences between free chunk check and klass read
>>>
>>> On 08/07/2020 09:41, Kim Barrett wrote:
>>>>> Since CMS is deprecated from JDK9, I am not sure if it's
>>>>> appropriate to fix
>>> this issue for those JDK9+ versions.
>>>> Deprecated != unsupported.
>>>>
>>>
>>> Yes, it must be done. Thanks.
>>
>> CCing to jdk-updates-dev.
>>
>> I have prepared another two webrevs for jdk11u and jdk13u:
>>    Jdk11u-dev:  http://cr.openjdk.java.net/~fyang/8248851-11u/
>>    Jdk13u-dev: http://cr.openjdk.java.net/~fyang/8248851-13u/
>>
>> The only difference lies in copyright year update.  Both tiere1-3 tested on
>> aarch64-linux-gnu.
>> I will put jdk11u-fix-request and jdk13u-fix-request label on the issue if they
>> are good.
>>
>> I have also prepared a new webrev for jdk8u incorporating copyright year
>> update:
>>    Jdk8u-dev: http://cr.openjdk.java.net/~fyang/8248851-8u/
>>
>> Thanks,
>> Felix
>>
> 


From thomas.schatzl at oracle.com  Thu Jul 16 11:05:52 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 16 Jul 2020 13:05:52 +0200
Subject: [16] RFR: 8248391: Unify handling of all OopStorage instances in
 weak root processing
In-Reply-To: <3A6A85AC-50A7-41AF-AE7E-9967C5B43AF9@oracle.com>
References: <8e831a42-61b0-1605-ea71-09f97d82f328@oracle.com>
 <11EE1DA7-3265-4A40-BA37-95B66B129F50@oracle.com>
 <eb24f5f4-fdcc-9f48-99d3-ff7a0ac9f86b@oracle.com>
 <3A6A85AC-50A7-41AF-AE7E-9967C5B43AF9@oracle.com>
Message-ID: <722f234e-e905-c133-78e6-ce6bd0a13f44@oracle.com>

Hi,

On 07.07.20 15:47, Kim Barrett wrote:
>> On Jun 30, 2020, at 12:35 PM, Stefan Karlsson <stefan.karlsson at oracle.com> wrote:
>>
>> I talked to kim about this point:
>>
>> "The GC (in particular, OopStorage) shouldn't know about client
>> subsystems in order to record the notification function"
>>
>> The current OopStorageSet implementation already knows about the different client subsystems, therefore I though that adding the notification registration there actually made sense. I've seen the alternative where the notification registration happens in the client subsystem, and that patch didn't look that great either, IMHO.
>>
>> So, I decided to create a PoC to (almost) completely decouple the client subsystems and the OopStorageSet. With the patch, the OopStorageSet doesn't know anything about the existing subsystems. It only knows about the number of weak and strong OopStorages in the JVM. The need to know the exact numbers arise from the wish to statically know that number during compile time. I'm not sure it's important to keep this static nature of the number of OopStorages, instead of using a more dynamic approach, but I've left this as is for now.
>>
>> https://cr.openjdk.java.net/~stefank/8248391/webrev.02.delta/
>> https://cr.openjdk.java.net/~stefank/8248391/webrev.02
> 
> I have some comments, but because of vacations I'm providing them as a
> further patch on top of Stefan's webrev.02, plus the description below.
> (I'm abandoning the earlier never-published prototype derived from Erik's
> changes in favor of using Stefan's additional PoC plus further changes based
> on it.)  If this all gets reviewed before folks are back, I'll deal with
> pushing, with Erik and Stefan as contributors.
> 
> full: https://cr.openjdk.java.net/~kbarrett/8248391/open.03/
> incr: https://cr.openjdk.java.net/~kbarrett/8248391/open.03.inc/
> 
> Testing:
> mach5 tier1- TBD
> Locally ran new StringTableCleaningTest.java with a duration of 1/2 hour for
> each collector (including Shenandoah) except Epsilon.
> 
[...]

> 
> I haven't come up with anything that seems better than Stefan's lazy
> allocation of the G1GCPhaseTimes object to deal with the ordering problem
> with subsystem registration of the OopStorage objects.  So I'm leaving that
> as is, at least for now.
> 
> I think the proposed terminology for the GC -> OopStorage owner callback was
> too vague and generic; "notify" and "notification" are relatively
> content-free names.  So I've renamed the functions and types.  Also, I think
> "can_notify" should be "should_notify" (and then adjusted for the generic
> "notify").
> 
> I've also changed (and documented) the semantics; the number of dead
> reported for an iteration is the number of entries skipped because they were
> already NULL plus the number cleared by the iteration.  (So the total dead,
> not just the recently deceased.)  That's the meaning in the old reporting
> mechanism.  Earlier versions of this change instead invoked the callback
> with just the number of newly cleared entries, with it being up to the
> receiving subsystem to accumulate the total.  But there are race conditions
> between GC and concurrent subsystem cleanup (like the ServiceThread's
> concurrent cleaning of the StringTable) that make such accumulation by the
> client potentially wildly inaccurate, leading to excess re-cleanings.  (Even
> with the callback receiving the total dead there are races, but I think the
> potential for significant inaccuracies leading to time wasting are much
> less.)

I would have somewhat like this change to be separate if possible, but okay.

> 
> A possible further change that I've left for later is that the
> WeakProcessorPhaseTimeTracker gets just the cleared (newly dead) count.
> (That hasn't been touched by any version of this change.)  I think the total
> dead would be a more useful statistic for the associated logging.
> 
> The (anonymous) enum of OopStorageSet iterator indices no longer serves any
> purpose; I've changed them to be ordinary in-class constants.  With the
> original code the enum was used to automatically assign values to some of
> the enumerators, but with the removal of named indices for each storage,
> there was only one autmatically assigned enumerator.
> 
> I've changed WeakProcessor::Task to also use the new
> OopStorageSetWeakParState.  This involved adding support for client access
> to the states, similar to what's in the strong parstate.  (I'm not sure the
> weak parstate should provide an oops_do that counts and reports the dead,
> but I've left that for now.)
> 
> Fixed a "bug" that WeakProcessor::oops_do(OopClosure*) counted and reported
> dead entries, even though it never clears entries.  It is used for
> iterations that shouldn't be reporting dead, such as for the adjust pointers
> phase of mark-sweep-compact.
> 
> Besides adapting to the above changes, I moved StringTable's clearing of the
> _has_work flag to the end of its do_concurrent_work.  This avoids queuing
> a new cleanup request in the middle of a concurrent cleanup that might
> eliminate the need for further cleaning.  We could try to re-evaluate at the
> end of the cleanup, but instead leave that to the next GC to trigger a new
> round of cleanup if needed.  Also removed some inconsistencies between
> ResolvedMethodTable and StringTable.

All these improvements as well.

> 
> I've also added a test that beats on StringTable allocation and cleaning.
> 

Thanks! Looks good apart from one pre-existing typo: in 
oopStorage.hpp:164 there is "if any..", ie. one full stop too many.

Thomas


From aph at redhat.com  Thu Jul 16 13:43:15 2020
From: aph at redhat.com (Andrew Haley)
Date: Thu, 16 Jul 2020 14:43:15 +0100
Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences
 between free chunk check and klass read
In-Reply-To: <48d0abe9-29a1-a67f-b8f0-c823e15ce110@oracle.com>
References: <DA41BE1DDCA941489001C7FBD7A8820EE7E63D86@dggeml527-mbx.china.huawei.com>
 <d6590e74-7ce2-70b9-7432-96b969686ab7@redhat.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E6466E@dggeml527-mbx.china.huawei.com>
 <8957BB9E-617D-4E24-9CC4-31AF7D01D12E@oracle.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E64ECD@dggeml527-mbx.china.huawei.com>
 <65012FED-2A93-4899-939E-887175AB07AC@oracle.com>
 <06646cf3-bc21-977f-96e2-1e9a974d1b00@redhat.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E74B25@dggeml507-mbx.china.huawei.com>
 <48d0abe9-29a1-a67f-b8f0-c823e15ce110@oracle.com>
Message-ID: <561611d4-18aa-f9d8-7302-3d89aff4ad81@redhat.com>

On 16/07/2020 10:16, David Holmes wrote:
> Seems to me that you want OrderAccess::loadload() barriers to order the
> loads, not OrderAccess::acquire(). You should only use acquire semantics
> to pair with a corresponding release operation.

I agree, but it's unlikely to matter in practice.

Having said that, I'm strongly of the opinion that if you see a naked
StoreStore it may well be a bug, or at least you've got something very
hard to analyse. I know of a few cases (e.g. zeroing an object) where
this isn't true.

https://www.hboehm.info/c++mm/no_write_fences.html, etc.

But that's an argument for another day.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671



From kim.barrett at oracle.com  Thu Jul 16 17:04:12 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 16 Jul 2020 13:04:12 -0400
Subject: [16] RFR: 8248391: Unify handling of all OopStorage instances in
 weak root processing
In-Reply-To: <722f234e-e905-c133-78e6-ce6bd0a13f44@oracle.com>
References: <8e831a42-61b0-1605-ea71-09f97d82f328@oracle.com>
 <11EE1DA7-3265-4A40-BA37-95B66B129F50@oracle.com>
 <eb24f5f4-fdcc-9f48-99d3-ff7a0ac9f86b@oracle.com>
 <3A6A85AC-50A7-41AF-AE7E-9967C5B43AF9@oracle.com>
 <722f234e-e905-c133-78e6-ce6bd0a13f44@oracle.com>
Message-ID: <E4A8A5C0-F9CF-4DF1-9B2C-253694C1F181@oracle.com>

> On Jul 16, 2020, at 7:05 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:

Regarding some things that you would have preferred as separate changes, see below.

> On 07.07.20 15:47, Kim Barrett wrote:
>> I think the proposed terminology for the GC -> OopStorage owner callback was
>> too vague and generic; "notify" and "notification" are relatively
>> content-free names.  So I've renamed the functions and types.  Also, I think
>> "can_notify" should be "should_notify" (and then adjusted for the generic
>> "notify").
>> I've also changed (and documented) the semantics; [?]
> 
> I would have somewhat like this change to be separate if possible, but okay.

Where I said ?I?ve also changed? I meant with respect to Erik?s original proposal.
What I have proposed reverts Erik?s change in this area back to the current behavior.

>> The (anonymous) enum of OopStorageSet iterator indices no longer serves any
>> purpose; I've changed them to be ordinary in-class constants.  With the
>> original code the enum was used to automatically assign values to some of
>> the enumerators, but with the removal of named indices for each storage,
>> there was only one autmatically assigned enumerator.

It seemed strange to me to delete the reasons for using an enum but still use an enum.

>> I've changed WeakProcessor::Task to also use the new
>> OopStorageSetWeakParState.  This involved adding support for client access
>> to the states, similar to what's in the strong parstate.  (I'm not sure the
>> weak parstate should provide an oops_do that counts and reports the dead,
>> but I've left that for now.)

It seemed strange to me to introduce OopStorageSetWeakParState as part of this change,
but only use it in some places (ZGC and Shenandoah) but not others (WeakProcessor).

>> Fixed a "bug" that WeakProcessor::oops_do(OopClosure*) counted and reported
>> dead entries, even though it never clears entries.  It is used for
>> iterations that shouldn't be reporting dead, such as for the adjust pointers
>> phase of mark-sweep-compact.

Yeah, that could have been done separately.

>> Besides adapting to the above changes, I moved StringTable's clearing of the
>> _has_work flag to the end of its do_concurrent_work.  This avoids queuing
>> a new cleanup request in the middle of a concurrent cleanup that might
>> eliminate the need for further cleaning.

That's really part of reverting Erik?s change to the reporting behavior.  I suspected that
Erik?s reporting change was in part a result of the previous placement of that clearing;
Erik?s change reduces the likelihood of excess cleaning requests with the old placement.

> Thanks! Looks good apart from one pre-existing typo: in oopStorage.hpp:164 there is "if any..", ie. one full stop too many.

Fixed.

Thanks for reviewing.



From david.holmes at oracle.com  Thu Jul 16 22:30:20 2020
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 17 Jul 2020 08:30:20 +1000
Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences
 between free chunk check and klass read
In-Reply-To: <561611d4-18aa-f9d8-7302-3d89aff4ad81@redhat.com>
References: <DA41BE1DDCA941489001C7FBD7A8820EE7E63D86@dggeml527-mbx.china.huawei.com>
 <d6590e74-7ce2-70b9-7432-96b969686ab7@redhat.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E6466E@dggeml527-mbx.china.huawei.com>
 <8957BB9E-617D-4E24-9CC4-31AF7D01D12E@oracle.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E64ECD@dggeml527-mbx.china.huawei.com>
 <65012FED-2A93-4899-939E-887175AB07AC@oracle.com>
 <06646cf3-bc21-977f-96e2-1e9a974d1b00@redhat.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E74B25@dggeml507-mbx.china.huawei.com>
 <48d0abe9-29a1-a67f-b8f0-c823e15ce110@oracle.com>
 <561611d4-18aa-f9d8-7302-3d89aff4ad81@redhat.com>
Message-ID: <f1b98db1-63ee-4f60-7001-7952a9ea5065@oracle.com>

On 16/07/2020 11:43 pm, Andrew Haley wrote:
> On 16/07/2020 10:16, David Holmes wrote:
>> Seems to me that you want OrderAccess::loadload() barriers to order the
>> loads, not OrderAccess::acquire(). You should only use acquire semantics
>> to pair with a corresponding release operation.
> 
> I agree, but it's unlikely to matter in practice.

In terms of what underlying hardware barriers get used, no it won't 
(likely) matter in practice.

But from a code understandability perspective it matters very much IMHO. 
We have been actively trying to ensure that the right OrderAccess APIs 
are used, in the right way and only where actually needed. An acquire 
without a corresponding release shows a lack of understanding and leads 
to confusion for other developers. If you need ordered loads then use a 
loadload() barrier. If the loads need to be ordered you need to ensure 
there are not corresponding writes that also need to be ordered - which 
may show where release() is missing.

> Having said that, I'm strongly of the opinion that if you see a naked
> StoreStore it may well be a bug, or at least you've got something very
> hard to analyse. I know of a few cases (e.g. zeroing an object) where
> this isn't true.
> 
> https://www.hboehm.info/c++mm/no_write_fences.html, etc.
> 
> But that's an argument for another day.

Indeed :)

Cheers,
David



From felix.yang at huawei.com  Fri Jul 17 03:05:03 2020
From: felix.yang at huawei.com (Yangfei (Felix))
Date: Fri, 17 Jul 2020 03:05:03 +0000
Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences
 between free chunk check and klass read
In-Reply-To: <f1b98db1-63ee-4f60-7001-7952a9ea5065@oracle.com>
References: <DA41BE1DDCA941489001C7FBD7A8820EE7E63D86@dggeml527-mbx.china.huawei.com>
 <d6590e74-7ce2-70b9-7432-96b969686ab7@redhat.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E6466E@dggeml527-mbx.china.huawei.com>
 <8957BB9E-617D-4E24-9CC4-31AF7D01D12E@oracle.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E64ECD@dggeml527-mbx.china.huawei.com>
 <65012FED-2A93-4899-939E-887175AB07AC@oracle.com>
 <06646cf3-bc21-977f-96e2-1e9a974d1b00@redhat.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E74B25@dggeml507-mbx.china.huawei.com>
 <48d0abe9-29a1-a67f-b8f0-c823e15ce110@oracle.com>
 <561611d4-18aa-f9d8-7302-3d89aff4ad81@redhat.com>
 <f1b98db1-63ee-4f60-7001-7952a9ea5065@oracle.com>
Message-ID: <DA41BE1DDCA941489001C7FBD7A8820EE7E754F4@dggeml507-mbx.china.huawei.com>

Hi,

  Thanks for the suggestions.  It makes sense to me. 
  BTW: OrderAccess::loadload() and OrderAccess::acquire() both map to the same instruction for aarch64: dmb ishld. 
  Updated webrev: 
    http://cr.openjdk.java.net/~fyang/8248851-8u/webrev.01/ 
    http://cr.openjdk.java.net/~fyang/8248851-11u/webrev.01/ 
    http://cr.openjdk.java.net/~fyang/8248851-13u/webrev.01/ 

  Performed the same test as before, result looks good. 
  Does it look better? 

Felix

> -----Original Message-----
> From: David Holmes [mailto:david.holmes at oracle.com]
> Sent: Friday, July 17, 2020 6:30 AM
> To: Andrew Haley <aph at redhat.com>; Yangfei (Felix)
> <felix.yang at huawei.com>; Kim Barrett <kim.barrett at oracle.com>
> Cc: jdk8u-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net;
> aarch64-port-dev at openjdk.java.net; jdk-updates-dev at openjdk.java.net
> Subject: Re: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory
> fences between free chunk check and klass read
> 
> On 16/07/2020 11:43 pm, Andrew Haley wrote:
> > On 16/07/2020 10:16, David Holmes wrote:
> >> Seems to me that you want OrderAccess::loadload() barriers to order
> >> the loads, not OrderAccess::acquire(). You should only use acquire
> >> semantics to pair with a corresponding release operation.
> >
> > I agree, but it's unlikely to matter in practice.
> 
> In terms of what underlying hardware barriers get used, no it won't
> (likely) matter in practice.
> 
> But from a code understandability perspective it matters very much IMHO.
> We have been actively trying to ensure that the right OrderAccess APIs are
> used, in the right way and only where actually needed. An acquire without a
> corresponding release shows a lack of understanding and leads to confusion
> for other developers. If you need ordered loads then use a
> loadload() barrier. If the loads need to be ordered you need to ensure there
> are not corresponding writes that also need to be ordered - which may show
> where release() is missing.
> 
> > Having said that, I'm strongly of the opinion that if you see a naked
> > StoreStore it may well be a bug, or at least you've got something very
> > hard to analyse. I know of a few cases (e.g. zeroing an object) where
> > this isn't true.
> >
> > https://www.hboehm.info/c++mm/no_write_fences.html, etc.
> >
> > But that's an argument for another day.
> 
> Indeed :)
> 
> Cheers,
> David


From david.holmes at oracle.com  Fri Jul 17 04:51:35 2020
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 17 Jul 2020 14:51:35 +1000
Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences
 between free chunk check and klass read
In-Reply-To: <DA41BE1DDCA941489001C7FBD7A8820EE7E754F4@dggeml507-mbx.china.huawei.com>
References: <DA41BE1DDCA941489001C7FBD7A8820EE7E63D86@dggeml527-mbx.china.huawei.com>
 <d6590e74-7ce2-70b9-7432-96b969686ab7@redhat.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E6466E@dggeml527-mbx.china.huawei.com>
 <8957BB9E-617D-4E24-9CC4-31AF7D01D12E@oracle.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E64ECD@dggeml527-mbx.china.huawei.com>
 <65012FED-2A93-4899-939E-887175AB07AC@oracle.com>
 <06646cf3-bc21-977f-96e2-1e9a974d1b00@redhat.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E74B25@dggeml507-mbx.china.huawei.com>
 <48d0abe9-29a1-a67f-b8f0-c823e15ce110@oracle.com>
 <561611d4-18aa-f9d8-7302-3d89aff4ad81@redhat.com>
 <f1b98db1-63ee-4f60-7001-7952a9ea5065@oracle.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E754F4@dggeml507-mbx.china.huawei.com>
Message-ID: <cc714482-f925-7c68-e629-c200ec61bc91@oracle.com>

Hi Felix,

On 17/07/2020 1:05 pm, Yangfei (Felix) wrote:
> Hi,
> 
>    Thanks for the suggestions.  It makes sense to me.
>    BTW: OrderAccess::loadload() and OrderAccess::acquire() both map to the same instruction for aarch64: dmb ishld.
>    Updated webrev:
>      http://cr.openjdk.java.net/~fyang/8248851-8u/webrev.01/
>      http://cr.openjdk.java.net/~fyang/8248851-11u/webrev.01/
>      http://cr.openjdk.java.net/~fyang/8248851-13u/webrev.01/
> 
>    Performed the same test as before, result looks good.
>    Does it look better?

It certainly looks better to me. (This code still puzzles me somewhat 
but I'm not going to expend more time trying to understand it more broadly.)

Thanks,
David
-----

> Felix
> 
>> -----Original Message-----
>> From: David Holmes [mailto:david.holmes at oracle.com]
>> Sent: Friday, July 17, 2020 6:30 AM
>> To: Andrew Haley <aph at redhat.com>; Yangfei (Felix)
>> <felix.yang at huawei.com>; Kim Barrett <kim.barrett at oracle.com>
>> Cc: jdk8u-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net;
>> aarch64-port-dev at openjdk.java.net; jdk-updates-dev at openjdk.java.net
>> Subject: Re: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory
>> fences between free chunk check and klass read
>>
>> On 16/07/2020 11:43 pm, Andrew Haley wrote:
>>> On 16/07/2020 10:16, David Holmes wrote:
>>>> Seems to me that you want OrderAccess::loadload() barriers to order
>>>> the loads, not OrderAccess::acquire(). You should only use acquire
>>>> semantics to pair with a corresponding release operation.
>>>
>>> I agree, but it's unlikely to matter in practice.
>>
>> In terms of what underlying hardware barriers get used, no it won't
>> (likely) matter in practice.
>>
>> But from a code understandability perspective it matters very much IMHO.
>> We have been actively trying to ensure that the right OrderAccess APIs are
>> used, in the right way and only where actually needed. An acquire without a
>> corresponding release shows a lack of understanding and leads to confusion
>> for other developers. If you need ordered loads then use a
>> loadload() barrier. If the loads need to be ordered you need to ensure there
>> are not corresponding writes that also need to be ordered - which may show
>> where release() is missing.
>>
>>> Having said that, I'm strongly of the opinion that if you see a naked
>>> StoreStore it may well be a bug, or at least you've got something very
>>> hard to analyse. I know of a few cases (e.g. zeroing an object) where
>>> this isn't true.
>>>
>>> https://www.hboehm.info/c++mm/no_write_fences.html, etc.
>>>
>>> But that's an argument for another day.
>>
>> Indeed :)
>>
>> Cheers,
>> David
> 


From shade at redhat.com  Fri Jul 17 07:16:20 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Fri, 17 Jul 2020 09:16:20 +0200
Subject: RFR (S) 8249649: Shenandoah: provide per-cycle pacing stats
Message-ID: <db79774b-73fc-ec2d-f930-8425158dd29e@redhat.com>

RFE:
  https://bugs.openjdk.java.net/browse/JDK-8249649

Current pacer statistics is printing only at the end of the run. It would be convenient to have it
in per-cycle statistics too.

Webrev:
  https://cr.openjdk.java.net/~shade/8249649/webrev.01/

Testing: hotspot_gc_shenandoah; tier{1,2} with Shenandoah; eyeballing gc logs

-- 
Thanks,
-Aleksey


From felix.yang at huawei.com  Fri Jul 17 08:40:58 2020
From: felix.yang at huawei.com (Yangfei (Felix))
Date: Fri, 17 Jul 2020 08:40:58 +0000
Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences
 between free chunk check and klass read
In-Reply-To: <cc714482-f925-7c68-e629-c200ec61bc91@oracle.com>
References: <DA41BE1DDCA941489001C7FBD7A8820EE7E63D86@dggeml527-mbx.china.huawei.com>
 <d6590e74-7ce2-70b9-7432-96b969686ab7@redhat.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E6466E@dggeml527-mbx.china.huawei.com>
 <8957BB9E-617D-4E24-9CC4-31AF7D01D12E@oracle.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E64ECD@dggeml527-mbx.china.huawei.com>
 <65012FED-2A93-4899-939E-887175AB07AC@oracle.com>
 <06646cf3-bc21-977f-96e2-1e9a974d1b00@redhat.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E74B25@dggeml507-mbx.china.huawei.com>
 <48d0abe9-29a1-a67f-b8f0-c823e15ce110@oracle.com>
 <561611d4-18aa-f9d8-7302-3d89aff4ad81@redhat.com>
 <f1b98db1-63ee-4f60-7001-7952a9ea5065@oracle.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E754F4@dggeml507-mbx.china.huawei.com>
 <cc714482-f925-7c68-e629-c200ec61bc91@oracle.com>
Message-ID: <DA41BE1DDCA941489001C7FBD7A8820EE7E75892@dggeml507-mbx.china.huawei.com>

Hi,

    Thanks for reviewing this and the great comments.
    I have put fix-request labels and accompanying comments on this issue for these three repos.

Felix

> -----Original Message-----
> From: David Holmes [mailto:david.holmes at oracle.com]
> Sent: Friday, July 17, 2020 12:52 PM
> To: Yangfei (Felix) <felix.yang at huawei.com>; Andrew Haley
> <aph at redhat.com>; Kim Barrett <kim.barrett at oracle.com>
> Cc: jdk8u-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net;
> aarch64-port-dev at openjdk.java.net; jdk-updates-dev at openjdk.java.net
> Subject: Re: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory
> fences between free chunk check and klass read
> 
> Hi Felix,
> 
> On 17/07/2020 1:05 pm, Yangfei (Felix) wrote:
> > Hi,
> >
> >    Thanks for the suggestions.  It makes sense to me.
> >    BTW: OrderAccess::loadload() and OrderAccess::acquire() both map to
> the same instruction for aarch64: dmb ishld.
> >    Updated webrev:
> >      http://cr.openjdk.java.net/~fyang/8248851-8u/webrev.01/
> >      http://cr.openjdk.java.net/~fyang/8248851-11u/webrev.01/
> >      http://cr.openjdk.java.net/~fyang/8248851-13u/webrev.01/
> >
> >    Performed the same test as before, result looks good.
> >    Does it look better?
> 
> It certainly looks better to me. (This code still puzzles me somewhat but I'm
> not going to expend more time trying to understand it more broadly.)
> 
> Thanks,
> David
> -----
> 
> > Felix
> >
> >> -----Original Message-----
> >> From: David Holmes [mailto:david.holmes at oracle.com]
> >> Sent: Friday, July 17, 2020 6:30 AM
> >> To: Andrew Haley <aph at redhat.com>; Yangfei (Felix)
> >> <felix.yang at huawei.com>; Kim Barrett <kim.barrett at oracle.com>
> >> Cc: jdk8u-dev at openjdk.java.net; hotspot-gc-dev at openjdk.java.net;
> >> aarch64-port-dev at openjdk.java.net; jdk-updates-dev at openjdk.java.net
> >> Subject: Re: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory
> >> fences between free chunk check and klass read
> >>
> >> On 16/07/2020 11:43 pm, Andrew Haley wrote:
> >>> On 16/07/2020 10:16, David Holmes wrote:
> >>>> Seems to me that you want OrderAccess::loadload() barriers to order
> >>>> the loads, not OrderAccess::acquire(). You should only use acquire
> >>>> semantics to pair with a corresponding release operation.
> >>>
> >>> I agree, but it's unlikely to matter in practice.
> >>
> >> In terms of what underlying hardware barriers get used, no it won't
> >> (likely) matter in practice.
> >>
> >> But from a code understandability perspective it matters very much IMHO.
> >> We have been actively trying to ensure that the right OrderAccess
> >> APIs are used, in the right way and only where actually needed. An
> >> acquire without a corresponding release shows a lack of understanding
> >> and leads to confusion for other developers. If you need ordered
> >> loads then use a
> >> loadload() barrier. If the loads need to be ordered you need to
> >> ensure there are not corresponding writes that also need to be
> >> ordered - which may show where release() is missing.
> >>
> >>> Having said that, I'm strongly of the opinion that if you see a
> >>> naked StoreStore it may well be a bug, or at least you've got
> >>> something very hard to analyse. I know of a few cases (e.g. zeroing
> >>> an object) where this isn't true.
> >>>
> >>> https://www.hboehm.info/c++mm/no_write_fences.html, etc.
> >>>
> >>> But that's an argument for another day.
> >>
> >> Indeed :)
> >>
> >> Cheers,
> >> David
> >

From thomas.schatzl at oracle.com  Fri Jul 17 09:51:56 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 17 Jul 2020 11:51:56 +0200
Subject: RFR (M): 8210462: Fix remaining mentions of initial mark
In-Reply-To: <3046898b-39c2-000d-f1c6-c6fae69d58ee@oracle.com>
References: <bc347a79-63d4-e259-3e57-53ddb4a00be1@oracle.com>
 <6f950a83-6942-b4e9-fb49-5ee20635add6@oracle.com>
 <a5411b45-4a50-9a0d-ef7d-7dc45d603d22@oracle.com>
 <3D90704F-D7BC-4E24-9783-76993989602C@oracle.com>
 <8edfe69d-822d-59c7-5130-5ab0b2fc2d1c@oracle.com>
 <0AFEB8CC-039E-4402-838A-D88657B05D62@oracle.com>
 <3046898b-39c2-000d-f1c6-c6fae69d58ee@oracle.com>
Message-ID: <283cacb9-e956-af4d-16af-8323cd2f46d8@oracle.com>

Hi Kim, Leo,

   totally forgot to thank you for the reviews. Thanks!

Thomas

On 09.07.20 13:46, Leo Korinth wrote:
> 
> 
> On 07/07/2020 14:38, Kim Barrett wrote:
>>> On Jul 7, 2020, at 4:39 AM, Thomas Schatzl 
>>> <thomas.schatzl at oracle.com> wrote:
>>>
>>> Hi Kim,
>>>
>>> ? thanks for your review.
>>>
>>> On 03.07.20 22:29, Kim Barrett wrote:
>>>> src/hotspot/share/gc/g1/g1Policy.cpp
>>>> [removed]
>>>> ? 636?? bool this_pause_was_young_only = 
>>>> collector_state()->in_young_only_phase();
>>>> Why was this variable removed and uses replaced with
>>>> is_young_only_only(this_pause), rather than keeping the variable but
>>>> updating the initialization?
>>>> ------------------------------------------------------------------------------ 
>>>>
>>>
>>> The reason is that this_pause_was_young_only is a local defined waaay 
>>> up at the top of that method and I thought instead of referencing 
>>> that one it is better to do the (very simple) function call.
>>> I can change that again if you want.
>>
>> OK.? The ?far away? rationale makes sense.
>>
>>> Webrevs:
>>> http://cr.openjdk.java.net/~tschatzl/8210462/webrev.1_to_2/ (diff)
>>> http://cr.openjdk.java.net/~tschatzl/8210462/webrev.2/ (full)
>>> Testing:
>>> tier1, local gc/g1 jtreg run
>>
>> Looks good.
>>
> 
> Looks good to me too.
> 
> Thanks,
> Leo



From thomas.schatzl at oracle.com  Fri Jul 17 09:55:55 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 17 Jul 2020 11:55:55 +0200
Subject: RFC: Adaptively resize heap at any GC/SoftMaxHeapSize for G1
In-Reply-To: <CA++bR4OOoVv-PYJ-Ps3zkoNUwJC8-F6hD_X51C8YWsHV-TexpQ@mail.gmail.com>
References: <mailman.52.1594122902.23224.hotspot-gc-dev@openjdk.java.net>
 <CA++bR4OOoVv-PYJ-Ps3zkoNUwJC8-F6hD_X51C8YWsHV-TexpQ@mail.gmail.com>
Message-ID: <b41d670f-179a-ba62-d9f7-5038b579d7d9@oracle.com>

Hi,

On 07.07.20 18:23, Ruslan Synytsky wrote:
>>>> Unfortunately GC.heap_info and VM.info do not provide information about
>>>> COMMITTED heap. And jstat documentation
>>
>> That is actually not true :) While looking into JDK-8248136, G1 actually
>> already prints committed heap with GC.heap_info.
>>
>> E.g. on an application with -Xms64m -Xmx1024m the output is:
>>
>> $ jcmd 30653 GC.heap_info
>> 30653:
>>    garbage-first heap   total 519168K, used 315920K [0x00000000c0000000,
>> 0x0000000100000000)
>>     region size 1024K, 116 young (118784K), 23 survivors (23552K)
>> [...]
>>
>> The "total" is "available" regions (i.e. ~committed) as explained in the
>> previous post (I kept the relevant part below).
>>
> Thomas, thank you, it helps! I'm glad that I was wrong. The "total" naming
> confused me, I was thinking it's the max (Xmx) heap.
> 
> As a follow up question - how to get the non heap usage? It can be useful
> for understanding of the full picture. JMX provides the following option:
> 

The only thing I can think of at this time is using NMT output (which is 
probably much more accurate than what the MX-Beans provide), but this 
needs you to enable it at startup.

> *MemoryMXBean mem = ManagementFactory.getPlatformMXBean(mbsc,
> MemoryMXBean.class);MemoryUsage nonHeap =
> mem.getNonHeapMemoryUsage();System.out.println(nonHeap.getInit() + "," +
> nonHeap.getUsed() + "|, + nonHeap.getCommitted() + "," + nonHeap.getMax());*
> 
> 
>> I recently filed JDK-8248136 for improving the heap info output for G1.
> 
> Improving the heap info sounds like a useful enhancement. At the moment
> GC.heap_info is printed by different GCs with different naming.

[...]

> Shenandoah provides the most distinct output including Xmx which is called
> "total" while "total" means committed in G1. Also ZGC prints committed heap
> as capacity while metaspace has both capacity and committed... Does it make
> sense to harmonize the naming?

:) I noticed that as well when looking for that info. I will file an 
enhancement to see whether the output can be streamlined slightly. 
Thanks for the reminder.

Thanks,
   Thomas


From rkennke at redhat.com  Fri Jul 17 09:56:26 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Fri, 17 Jul 2020 11:56:26 +0200
Subject: RFR (S) 8249649: Shenandoah: provide per-cycle pacing stats
In-Reply-To: <db79774b-73fc-ec2d-f930-8425158dd29e@redhat.com>
References: <db79774b-73fc-ec2d-f930-8425158dd29e@redhat.com>
Message-ID: <c6d67db3957b65826c903ccc82022dcd8c149c8e.camel@redhat.com>

Looks good to me.

Thank you!
Roman


On Fri, 2020-07-17 at 09:16 +0200, Aleksey Shipilev wrote:
> 	Error verifying signature: Cannot verify message signature:
> Incorrect message format
> RFE:
>   https://bugs.openjdk.java.net/browse/JDK-8249649
> 
> Current pacer statistics is printing only at the end of the run. It
> would be convenient to have it
> in per-cycle statistics too.
> 
> Webrev:
>   https://cr.openjdk.java.net/~shade/8249649/webrev.01/
> 
> Testing: hotspot_gc_shenandoah; tier{1,2} with Shenandoah; eyeballing
> gc logs
> 



From volker.simonis at gmail.com  Fri Jul 17 14:43:55 2020
From: volker.simonis at gmail.com (Volker Simonis)
Date: Fri, 17 Jul 2020 16:43:55 +0200
Subject: Analyzing Efficiency Of Java Memory Usage
In-Reply-To: <CA++bR4P9W+GYhVen_iwp98=yJLU0H6bG+-8b1Bci-R=xtYL34g@mail.gmail.com>
References: <CA++bR4P9W+GYhVen_iwp98=yJLU0H6bG+-8b1Bci-R=xtYL34g@mail.gmail.com>
Message-ID: <CA+3eh11uzzD492s-nYd3e6nCmqLJx2i+1EPnn-PVGSTXjyE9jA@mail.gmail.com>

On Wed, Jul 15, 2020 at 9:07 PM Ruslan Synytsky <rs at jelastic.com> wrote:
>
> Hi all, I would like to share some preliminary results related to Java
> elasticity analysis. We collected metrics from about 1000 containers. The
> goal was to get a rough picture of the memory usage efficiency within our
> user base.These containers are running different software stacks, java
> versions, dev or production workloads, and they belong to different end
> users. Each container may have one or multiple Java processes running
> inside and even additional auxiliary non Java processes.
>
> At the following link you can find a dynamic chart that reflects memory
> usage at specific period of time (it's one time measurement)
> https://cs.demo.jelastic.com/chart&appid=cc492725f550fcd637ab8a7c1f9810c9
> where
> - X axis represent Hosts, each host is a separated container
> - Y Axis:
>   * OSTotal: total RAM available for container
>   * Xmx: max heap limit
>   * NonHeapCommitted+: committed non heap stacked on top of committed heap
> memory
>   * HeapCommitted: committed heap memory
>   * HeapUsed: used heap memory
>   * OSBuffCache+: OS memory in buffer / cache stacked on top of OS used
> memory
>   * OSUsed: OS used memory
>   * Count: number of java processes running inside the host
>   ---
>   * Java Version per each Java process
>   * GC type per each Java process
>   * Java Agent: n/count - how many processes are using jelastic GC agent
>  (was used before JEP 346)
>   * G1PeriodicGCInterval: n/count - how many processes are using this new
> option
>
> The chart is ordered from left to right by the following sequence
> OSTotal, Xmx, NonHeapCommitted, HeapCommitted,
> HeapUsed, OSBuffCache, OSUsed.
>
> In general, the results look good as the vast majority of java processes
> are running in a quite efficient mode (small difference between used heap
> and committed heap). There are some java processes that have a big
> difference between used heap and committed heap (for example host 91 or
> host 109), but such processes do not use memory usage optimization (no
> configured GC agent and no G1PeriodicGCInterval) or they use non-compacting
> GC. Interesting findings that a) sometimes committed heap memory is
> higher than real OS used memory

Yes, this is a long standing, annoying problem which is most probably
caused by the fact "Committed" as reported by the VM/NMT is not really
the same like RSS reported by system tools. I've opened "8249666:
Improve Native Memory Tracking to report the actual RSS usage" [0] to
track this. Here's an excerpt from the issue describing the problem:

Currently, NMT shows allocated memory as either "Reserved" or
"Committed". Reserved memory is actually just reserved, virtual
address space which was mmaped with MAP_NORESERVE, while Committed
memory is mapped without MAP_NORESERVE. In the output of top or pmap,
both Reserved and Committed show up as "Virtual" memory until they
will be used for the first time (i.e. touched). Only after a memory
page (usually 4k) has been written to for the first time, it will
consume physical memory and appear in the "resident set" (i.e. RSS) of
top's/pmap's output.

The difference between allocating memory with or without MAP_NORESERVE
depends on the Linux memory overcommit configuration [1]. By default,
overcommit is allowed and memory allocated with MAP_NORESERVE isn't
checked against the available physical memory (see man proc(5) [2]).
If the HotSpot VM tries to commit reserved memory (i.e. re-map a
memory region without MAP_NORESERVE which was previously mapped with
MAP_NORESERVE) and there's not enough free memory available an
OutOfMemoyError will be thrown.

But even committing a memory region doesn't mean that physical memory
pages will be allocated for that region (and accounted in the
processes RSS) until that memory will be written to for the first
time. So depending on the overcommit settings, an application might
still crash with a SIGBUS because it is running out of physical memory
when touching memory for the first time which was committed a long
time ago.

The main problem with the current NMT output is that it can not
distinguish between touched and untouched Committed memory. If a VM is
started with -Xms1g -Xmx1g the VM will commit the whole 1g heap and
NMT will report Reserved=Committed=1g. In contrast, system tools like
ps/top will only show the part of the heap as RSS which has really
been used (i.e. touched), usually just about 100m. This is at least
confusing.

But we can do better. We can use mincore() [3] to find the RSS part of
the amount of memory which is accounted as Committed in NMT's output
and report that instead (or in addition). Notice that this feature has
already been implemented for threads stacks with "JDK-8191369: NMT:
Enhance thread stack tracking" [4] and just needs to be extended to
all other kinds of memory, starting with the Java heap.

Alternatively, instead of using mincore() we could use the information
from /proc/<pid>/smaps (also accessible through the pmap [5] command
line utility) directly and merge it with the NMT data to get a
complete, annotated overview of the whole address space of a Java
process.

[0] https://bugs.openjdk.java.net/browse/JDK-8249666
[1] https://www.kernel.org/doc/Documentation/vm/overcommit-accounting
[2] https://man7.org/linux/man-pages/man5/proc.5.html
[3] https://man7.org/linux/man-pages/man2/mincore.2.html
[4] https://bugs.openjdk.java.net/browse/JDK-8191369
[5] https://man7.org/linux/man-pages/man1/pmap.1.html

> , and b) in many cases non heap committed
> memory is as big as heap committed.
>
> Please note, the analysis does not represent all possible use cases and
> rather represent web applications and services that are hosted at local
> cloud hosting providers. Also, most likely our analysis does not reflect
> the reality outside of Jelastic, because by default we optimize Java
> runtimes for the maximum efficiency in terms of memory usage, so I expect
> that the real world situation is worse.
>
>
>
> We are planning to add more containers to the analysis for getting an even
> bigger picture, but also I'm looking for other companies that can join this
> analysis. It will help the community to get a better understanding of the
> real (diverse and heterogeneous) Java world. Please let me know if you are
> willing to participate and I will share a bash script that collects the
> memory usage metrics (it will not collect sensitive or personal information
> as well as it's safe to run on productions).
>
> Looking forward to getting your feedback.
> Thanks
> --
> Ruslan Synytsky
> CEO @ Jelastic Multi-Cloud PaaS


From thomas.stuefe at gmail.com  Fri Jul 17 15:18:43 2020
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Fri, 17 Jul 2020 17:18:43 +0200
Subject: Analyzing Efficiency Of Java Memory Usage
In-Reply-To: <CA+3eh11uzzD492s-nYd3e6nCmqLJx2i+1EPnn-PVGSTXjyE9jA@mail.gmail.com>
References: <CA++bR4P9W+GYhVen_iwp98=yJLU0H6bG+-8b1Bci-R=xtYL34g@mail.gmail.com>
 <CA+3eh11uzzD492s-nYd3e6nCmqLJx2i+1EPnn-PVGSTXjyE9jA@mail.gmail.com>
Message-ID: <CAA-vtUxEVdxiZ47VBnJDuRpyZhPAN48n05qj9iboxYbwBM0i5g@mail.gmail.com>

Did Ruslan use NMT to get those numbers?

Small addition to your points, NMT only tracks hotspot memory, any memory
used by core libs is not tracked. We discussed extending NMT tracking to
the core libs (we have a similar solution at SAP), but so far have not done
anything about it.

Cheers Thomas


On Fri, Jul 17, 2020 at 4:46 PM Volker Simonis <volker.simonis at gmail.com>
wrote:

> On Wed, Jul 15, 2020 at 9:07 PM Ruslan Synytsky <rs at jelastic.com> wrote:
> >
> > Hi all, I would like to share some preliminary results related to Java
> > elasticity analysis. We collected metrics from about 1000 containers. The
> > goal was to get a rough picture of the memory usage efficiency within our
> > user base.These containers are running different software stacks, java
> > versions, dev or production workloads, and they belong to different end
> > users. Each container may have one or multiple Java processes running
> > inside and even additional auxiliary non Java processes.
> >
> > At the following link you can find a dynamic chart that reflects memory
> > usage at specific period of time (it's one time measurement)
> >
> https://cs.demo.jelastic.com/chart&appid=cc492725f550fcd637ab8a7c1f9810c9
> > where
> > - X axis represent Hosts, each host is a separated container
> > - Y Axis:
> >   * OSTotal: total RAM available for container
> >   * Xmx: max heap limit
> >   * NonHeapCommitted+: committed non heap stacked on top of committed
> heap
> > memory
> >   * HeapCommitted: committed heap memory
> >   * HeapUsed: used heap memory
> >   * OSBuffCache+: OS memory in buffer / cache stacked on top of OS used
> > memory
> >   * OSUsed: OS used memory
> >   * Count: number of java processes running inside the host
> >   ---
> >   * Java Version per each Java process
> >   * GC type per each Java process
> >   * Java Agent: n/count - how many processes are using jelastic GC agent
> >  (was used before JEP 346)
> >   * G1PeriodicGCInterval: n/count - how many processes are using this new
> > option
> >
> > The chart is ordered from left to right by the following sequence
> > OSTotal, Xmx, NonHeapCommitted, HeapCommitted,
> > HeapUsed, OSBuffCache, OSUsed.
> >
> > In general, the results look good as the vast majority of java processes
> > are running in a quite efficient mode (small difference between used heap
> > and committed heap). There are some java processes that have a big
> > difference between used heap and committed heap (for example host 91 or
> > host 109), but such processes do not use memory usage optimization (no
> > configured GC agent and no G1PeriodicGCInterval) or they use
> non-compacting
> > GC. Interesting findings that a) sometimes committed heap memory is
> > higher than real OS used memory
>
> Yes, this is a long standing, annoying problem which is most probably
> caused by the fact "Committed" as reported by the VM/NMT is not really
> the same like RSS reported by system tools. I've opened "8249666:
> Improve Native Memory Tracking to report the actual RSS usage" [0] to
> track this. Here's an excerpt from the issue describing the problem:
>
> Currently, NMT shows allocated memory as either "Reserved" or
> "Committed". Reserved memory is actually just reserved, virtual
> address space which was mmaped with MAP_NORESERVE, while Committed
> memory is mapped without MAP_NORESERVE. In the output of top or pmap,
> both Reserved and Committed show up as "Virtual" memory until they
> will be used for the first time (i.e. touched). Only after a memory
> page (usually 4k) has been written to for the first time, it will
> consume physical memory and appear in the "resident set" (i.e. RSS) of
> top's/pmap's output.
>
> The difference between allocating memory with or without MAP_NORESERVE
> depends on the Linux memory overcommit configuration [1]. By default,
> overcommit is allowed and memory allocated with MAP_NORESERVE isn't
> checked against the available physical memory (see man proc(5) [2]).
> If the HotSpot VM tries to commit reserved memory (i.e. re-map a
> memory region without MAP_NORESERVE which was previously mapped with
> MAP_NORESERVE) and there's not enough free memory available an
> OutOfMemoyError will be thrown.
>
> But even committing a memory region doesn't mean that physical memory
> pages will be allocated for that region (and accounted in the
> processes RSS) until that memory will be written to for the first
> time. So depending on the overcommit settings, an application might
> still crash with a SIGBUS because it is running out of physical memory
> when touching memory for the first time which was committed a long
> time ago.
>
> The main problem with the current NMT output is that it can not
> distinguish between touched and untouched Committed memory. If a VM is
> started with -Xms1g -Xmx1g the VM will commit the whole 1g heap and
> NMT will report Reserved=Committed=1g. In contrast, system tools like
> ps/top will only show the part of the heap as RSS which has really
> been used (i.e. touched), usually just about 100m. This is at least
> confusing.
>
> But we can do better. We can use mincore() [3] to find the RSS part of
> the amount of memory which is accounted as Committed in NMT's output
> and report that instead (or in addition). Notice that this feature has
> already been implemented for threads stacks with "JDK-8191369: NMT:
> Enhance thread stack tracking" [4] and just needs to be extended to
> all other kinds of memory, starting with the Java heap.
>
> Alternatively, instead of using mincore() we could use the information
> from /proc/<pid>/smaps (also accessible through the pmap [5] command
> line utility) directly and merge it with the NMT data to get a
> complete, annotated overview of the whole address space of a Java
> process.
>
> [0] https://bugs.openjdk.java.net/browse/JDK-8249666
> [1] https://www.kernel.org/doc/Documentation/vm/overcommit-accounting
> [2] https://man7.org/linux/man-pages/man5/proc.5.html
> [3] https://man7.org/linux/man-pages/man2/mincore.2.html
> [4] https://bugs.openjdk.java.net/browse/JDK-8191369
> [5] https://man7.org/linux/man-pages/man1/pmap.1.html
>
> > , and b) in many cases non heap committed
> > memory is as big as heap committed.
> >
> > Please note, the analysis does not represent all possible use cases and
> > rather represent web applications and services that are hosted at local
> > cloud hosting providers. Also, most likely our analysis does not reflect
> > the reality outside of Jelastic, because by default we optimize Java
> > runtimes for the maximum efficiency in terms of memory usage, so I expect
> > that the real world situation is worse.
> >
> >
> >
> > We are planning to add more containers to the analysis for getting an
> even
> > bigger picture, but also I'm looking for other companies that can join
> this
> > analysis. It will help the community to get a better understanding of the
> > real (diverse and heterogeneous) Java world. Please let me know if you
> are
> > willing to participate and I will share a bash script that collects the
> > memory usage metrics (it will not collect sensitive or personal
> information
> > as well as it's safe to run on productions).
> >
> > Looking forward to getting your feedback.
> > Thanks
> > --
> > Ruslan Synytsky
> > CEO @ Jelastic Multi-Cloud PaaS
>


From igor.ignatyev at oracle.com  Fri Jul 17 18:58:38 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Fri, 17 Jul 2020 11:58:38 -0700
Subject: [15] RFR(T) 8249681 :
 gc/stress/TestJNIBlockFullGC/TestJNIBlockFullGC.java fails w/
 UnsatisfiedLinkError
Message-ID: <C298A543-02A1-4520-ADC8-12FBFEC2AF92@oracle.com>

http://cr.openjdk.java.net/~iignatyev//8249681/webrev.00
> 2 lines changed: 0 ins; 0 del; 2 mod;


Hi all,

could you please review this trivial fix for TestJNIBlockFullGC test?

gc/stress/TestJNIBlockFullGC/TestJNIBlockFullGC.java fails w/ UnsatisfiedLinkError 'int gc.stress.TestJNIBlockFullGC.TestJNIBlockFullGC.TestCriticalArray0(int[])' b/c TestJNIBlockFullGC was moved to gc.stress.TestJNIBlockFullGC package by 8214799, but the JNI method hasn't been renamed.

testing: gc/stress/TestJNIBlockFullGC/TestJNIBlockFullGC.java on {windows,linux,macos}-x64
webrev: http://cr.openjdk.java.net/~iignatyev//8249681/webrev.00
JBS: https://bugs.openjdk.java.net/browse/JDK-8249681

PS the test fails w/ OOME and is still problem-listed due to 8192647

Thanks,
-- Igor

From volker.simonis at gmail.com  Fri Jul 17 19:59:20 2020
From: volker.simonis at gmail.com (Volker Simonis)
Date: Fri, 17 Jul 2020 21:59:20 +0200
Subject: Analyzing Efficiency Of Java Memory Usage
In-Reply-To: <CAA-vtUxEVdxiZ47VBnJDuRpyZhPAN48n05qj9iboxYbwBM0i5g@mail.gmail.com>
References: <CA++bR4P9W+GYhVen_iwp98=yJLU0H6bG+-8b1Bci-R=xtYL34g@mail.gmail.com>
 <CA+3eh11uzzD492s-nYd3e6nCmqLJx2i+1EPnn-PVGSTXjyE9jA@mail.gmail.com>
 <CAA-vtUxEVdxiZ47VBnJDuRpyZhPAN48n05qj9iboxYbwBM0i5g@mail.gmail.com>
Message-ID: <CA+3eh114C6V8C-La3_E+t2kwqhS+g7p+G+Y6gjZ1pCizz1943g@mail.gmail.com>

On Fri, Jul 17, 2020 at 5:18 PM Thomas St?fe <thomas.stuefe at gmail.com> wrote:
>
> Did Ruslan use NMT to get those numbers?

No, probably they used the JMX MemoryMXBean. But that reports both,
"committed" and "used", and "used" should be pretty much the same like
"touched" (or RSS). E.g.

MemoryUsage G1MonitoringSupport::memory_usage() {
  MutexLocker x(MonitoringSupport_lock, Mutex::_no_safepoint_check_flag);
  return MemoryUsage(InitialHeapSize, _overall_used,
_overall_committed, _g1h->max_capacity());
}

>
> Small addition to your points, NMT only tracks hotspot memory, any memory used by core libs is not tracked. We discussed extending NMT tracking to the core libs (we have a similar solution at SAP), but so far have not done anything about it.

I know that :)

>
> Cheers Thomas
>
>
> On Fri, Jul 17, 2020 at 4:46 PM Volker Simonis <volker.simonis at gmail.com> wrote:
>>
>> On Wed, Jul 15, 2020 at 9:07 PM Ruslan Synytsky <rs at jelastic.com> wrote:
>> >
>> > Hi all, I would like to share some preliminary results related to Java
>> > elasticity analysis. We collected metrics from about 1000 containers. The
>> > goal was to get a rough picture of the memory usage efficiency within our
>> > user base.These containers are running different software stacks, java
>> > versions, dev or production workloads, and they belong to different end
>> > users. Each container may have one or multiple Java processes running
>> > inside and even additional auxiliary non Java processes.
>> >
>> > At the following link you can find a dynamic chart that reflects memory
>> > usage at specific period of time (it's one time measurement)
>> > https://cs.demo.jelastic.com/chart&appid=cc492725f550fcd637ab8a7c1f9810c9
>> > where
>> > - X axis represent Hosts, each host is a separated container
>> > - Y Axis:
>> >   * OSTotal: total RAM available for container
>> >   * Xmx: max heap limit
>> >   * NonHeapCommitted+: committed non heap stacked on top of committed heap
>> > memory
>> >   * HeapCommitted: committed heap memory
>> >   * HeapUsed: used heap memory
>> >   * OSBuffCache+: OS memory in buffer / cache stacked on top of OS used
>> > memory
>> >   * OSUsed: OS used memory
>> >   * Count: number of java processes running inside the host
>> >   ---
>> >   * Java Version per each Java process
>> >   * GC type per each Java process
>> >   * Java Agent: n/count - how many processes are using jelastic GC agent
>> >  (was used before JEP 346)
>> >   * G1PeriodicGCInterval: n/count - how many processes are using this new
>> > option
>> >
>> > The chart is ordered from left to right by the following sequence
>> > OSTotal, Xmx, NonHeapCommitted, HeapCommitted,
>> > HeapUsed, OSBuffCache, OSUsed.
>> >
>> > In general, the results look good as the vast majority of java processes
>> > are running in a quite efficient mode (small difference between used heap
>> > and committed heap). There are some java processes that have a big
>> > difference between used heap and committed heap (for example host 91 or
>> > host 109), but such processes do not use memory usage optimization (no
>> > configured GC agent and no G1PeriodicGCInterval) or they use non-compacting
>> > GC. Interesting findings that a) sometimes committed heap memory is
>> > higher than real OS used memory
>>
>> Yes, this is a long standing, annoying problem which is most probably
>> caused by the fact "Committed" as reported by the VM/NMT is not really
>> the same like RSS reported by system tools. I've opened "8249666:
>> Improve Native Memory Tracking to report the actual RSS usage" [0] to
>> track this. Here's an excerpt from the issue describing the problem:
>>
>> Currently, NMT shows allocated memory as either "Reserved" or
>> "Committed". Reserved memory is actually just reserved, virtual
>> address space which was mmaped with MAP_NORESERVE, while Committed
>> memory is mapped without MAP_NORESERVE. In the output of top or pmap,
>> both Reserved and Committed show up as "Virtual" memory until they
>> will be used for the first time (i.e. touched). Only after a memory
>> page (usually 4k) has been written to for the first time, it will
>> consume physical memory and appear in the "resident set" (i.e. RSS) of
>> top's/pmap's output.
>>
>> The difference between allocating memory with or without MAP_NORESERVE
>> depends on the Linux memory overcommit configuration [1]. By default,
>> overcommit is allowed and memory allocated with MAP_NORESERVE isn't
>> checked against the available physical memory (see man proc(5) [2]).
>> If the HotSpot VM tries to commit reserved memory (i.e. re-map a
>> memory region without MAP_NORESERVE which was previously mapped with
>> MAP_NORESERVE) and there's not enough free memory available an
>> OutOfMemoyError will be thrown.
>>
>> But even committing a memory region doesn't mean that physical memory
>> pages will be allocated for that region (and accounted in the
>> processes RSS) until that memory will be written to for the first
>> time. So depending on the overcommit settings, an application might
>> still crash with a SIGBUS because it is running out of physical memory
>> when touching memory for the first time which was committed a long
>> time ago.
>>
>> The main problem with the current NMT output is that it can not
>> distinguish between touched and untouched Committed memory. If a VM is
>> started with -Xms1g -Xmx1g the VM will commit the whole 1g heap and
>> NMT will report Reserved=Committed=1g. In contrast, system tools like
>> ps/top will only show the part of the heap as RSS which has really
>> been used (i.e. touched), usually just about 100m. This is at least
>> confusing.
>>
>> But we can do better. We can use mincore() [3] to find the RSS part of
>> the amount of memory which is accounted as Committed in NMT's output
>> and report that instead (or in addition). Notice that this feature has
>> already been implemented for threads stacks with "JDK-8191369: NMT:
>> Enhance thread stack tracking" [4] and just needs to be extended to
>> all other kinds of memory, starting with the Java heap.
>>
>> Alternatively, instead of using mincore() we could use the information
>> from /proc/<pid>/smaps (also accessible through the pmap [5] command
>> line utility) directly and merge it with the NMT data to get a
>> complete, annotated overview of the whole address space of a Java
>> process.
>>
>> [0] https://bugs.openjdk.java.net/browse/JDK-8249666
>> [1] https://www.kernel.org/doc/Documentation/vm/overcommit-accounting
>> [2] https://man7.org/linux/man-pages/man5/proc.5.html
>> [3] https://man7.org/linux/man-pages/man2/mincore.2.html
>> [4] https://bugs.openjdk.java.net/browse/JDK-8191369
>> [5] https://man7.org/linux/man-pages/man1/pmap.1.html
>>
>> > , and b) in many cases non heap committed
>> > memory is as big as heap committed.
>> >
>> > Please note, the analysis does not represent all possible use cases and
>> > rather represent web applications and services that are hosted at local
>> > cloud hosting providers. Also, most likely our analysis does not reflect
>> > the reality outside of Jelastic, because by default we optimize Java
>> > runtimes for the maximum efficiency in terms of memory usage, so I expect
>> > that the real world situation is worse.
>> >
>> >
>> >
>> > We are planning to add more containers to the analysis for getting an even
>> > bigger picture, but also I'm looking for other companies that can join this
>> > analysis. It will help the community to get a better understanding of the
>> > real (diverse and heterogeneous) Java world. Please let me know if you are
>> > willing to participate and I will share a bash script that collects the
>> > memory usage metrics (it will not collect sensitive or personal information
>> > as well as it's safe to run on productions).
>> >
>> > Looking forward to getting your feedback.
>> > Thanks
>> > --
>> > Ruslan Synytsky
>> > CEO @ Jelastic Multi-Cloud PaaS


From thomas.schatzl at oracle.com  Fri Jul 17 20:49:28 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 17 Jul 2020 13:49:28 -0700 (PDT)
Subject: [15] RFR(T) 8249681 :
 gc/stress/TestJNIBlockFullGC/TestJNIBlockFullGC.java fails w/
 UnsatisfiedLinkError
Message-ID: <fea5afb1-3813-4e95-b5eb-5220b3e0553e@default>

Hi,

  looks good.

Thomas

----- Original Message -----
From: igor.ignatyev at oracle.com
To: hotspot-gc-dev at openjdk.java.net
Sent: Friday, July 17, 2020 8:58:48 PM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna
Subject: [15] RFR(T) 8249681 : gc/stress/TestJNIBlockFullGC/TestJNIBlockFullGC.java fails w/ UnsatisfiedLinkError

http://cr.openjdk.java.net/~iignatyev//8249681/webrev.00
> 2 lines changed: 0 ins; 0 del; 2 mod;


Hi all,

could you please review this trivial fix for TestJNIBlockFullGC test?

gc/stress/TestJNIBlockFullGC/TestJNIBlockFullGC.java fails w/ UnsatisfiedLinkError 'int gc.stress.TestJNIBlockFullGC.TestJNIBlockFullGC.TestCriticalArray0(int[])' b/c TestJNIBlockFullGC was moved to gc.stress.TestJNIBlockFullGC package by 8214799, but the JNI method hasn't been renamed.

testing: gc/stress/TestJNIBlockFullGC/TestJNIBlockFullGC.java on {windows,linux,macos}-x64
webrev: http://cr.openjdk.java.net/~iignatyev//8249681/webrev.00
JBS: https://bugs.openjdk.java.net/browse/JDK-8249681

PS the test fails w/ OOME and is still problem-listed due to 8192647

Thanks,
-- Igor


From thomas.schatzl at oracle.com  Fri Jul 17 20:50:50 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 17 Jul 2020 13:50:50 -0700 (PDT)
Subject: [15] RFR(T) 8249681 :
 gc/stress/TestJNIBlockFullGC/TestJNIBlockFullGC.java fails w/
 UnsatisfiedLinkError
Message-ID: <b133dd7e-a919-4050-bd18-5c9545c5c82f@default>

.. and indeed trivial.

Thomas

----- Original Message -----
From: thomas.schatzl at oracle.com
To: igor.ignatyev at oracle.com
Cc: hotspot-gc-dev at openjdk.java.net
Sent: Friday, July 17, 2020 10:49:42 PM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna
Subject: Re: [15] RFR(T) 8249681 : gc/stress/TestJNIBlockFullGC/TestJNIBlockFullGC.java fails w/ UnsatisfiedLinkError

Hi,

  looks good.

Thomas

----- Original Message -----
From: igor.ignatyev at oracle.com
To: hotspot-gc-dev at openjdk.java.net
Sent: Friday, July 17, 2020 8:58:48 PM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna
Subject: [15] RFR(T) 8249681 : gc/stress/TestJNIBlockFullGC/TestJNIBlockFullGC.java fails w/ UnsatisfiedLinkError

http://cr.openjdk.java.net/~iignatyev//8249681/webrev.00
> 2 lines changed: 0 ins; 0 del; 2 mod;


Hi all,

could you please review this trivial fix for TestJNIBlockFullGC test?

gc/stress/TestJNIBlockFullGC/TestJNIBlockFullGC.java fails w/ UnsatisfiedLinkError 'int gc.stress.TestJNIBlockFullGC.TestJNIBlockFullGC.TestCriticalArray0(int[])' b/c TestJNIBlockFullGC was moved to gc.stress.TestJNIBlockFullGC package by 8214799, but the JNI method hasn't been renamed.

testing: gc/stress/TestJNIBlockFullGC/TestJNIBlockFullGC.java on {windows,linux,macos}-x64
webrev: http://cr.openjdk.java.net/~iignatyev//8249681/webrev.00
JBS: https://bugs.openjdk.java.net/browse/JDK-8249681

PS the test fails w/ OOME and is still problem-listed due to 8192647

Thanks,
-- Igor


From luoziyi at amazon.com  Fri Jul 17 20:58:23 2020
From: luoziyi at amazon.com (Luo, Ziyi)
Date: Fri, 17 Jul 2020 20:58:23 +0000
Subject: RFR: 8245511: G1 adaptive IHOP does not account for reclamation of
 humongous objects by young GC
Message-ID: <9D0EA8F0-7C63-428D-87D8-8352B4314DF2@amazon.com>

Hi,

Please review this patch to adjust the old gen allocation rate for G1 Adaptive
IHOP to consider the humongous objects freed by young collector.

Webrev: http://cr.openjdk.java.net/~bmathiske/8245511/webrev.01/
CR: https://bugs.openjdk.java.net/browse/JDK-8245511

This revision addresses a comment from Thomas in webrev.00:
> one other improvement could be to put the old gen allocation tracking into
its own class...

This is added in JDK-8245511. This RFR refactors G1*IHOPControl to no longer
store any allocation data but read from the old gen allocation tracker
instance.

To answer the following questions from Thomas:
> Quick(!) testing on the reproducer with the suggested patch showed around
23% of young gcs were evacuation failures

The evacuation failures are "by design". The test case attached to the JBS bug
intends to create a low pressure on the survive space and then suddenly
increase the life cycle of objects in the Eden space to create a to-space
exhaustion (I checked GC log, all evacuation failures are to-space
exhaustions). This is the fastest approach I can come up with to get objects
promoted to the old gen directly to trigger enough concurrent cycles for
Adaptive IHOP to sample. Once Adaptive IHOP starts to predict, the to-space
exhaustion is no longer needed. The evacuation failures are also observed in
test run with Static IHOP. Why then are not observed without this fix? Because
GC happens every 50 ms. Even during the allocation spike, only one region is
filled up in Eden :)

> but in return the change imho needs to provide some measure of the spikyness
of the allocation between gcs. Otherwise G1 will run into to-space exhaustions
and potentially full gcs all the time with such loads.

I agree that the spikiness of the allocation between GCs should be considered
and I understand your concern here. My fix might bring the old gen allocation
rate too low that theoretically back-to-back full GC is possible. However, I
was not able to create a test case to produce it as those short-live humongous
objects are reclaimed so rapidly (my patch only accounts humongous objects
that are allocated and reclaimed in the same allocation cycle). Could you give
me some hint on how to produce the worst-case scenario?

Tests:
All hotspot tier-1 passed in fastdebug build.

Thanks,
Ziyi

From igor.ignatyev at oracle.com  Fri Jul 17 22:02:20 2020
From: igor.ignatyev at oracle.com (Igor Ignatyev)
Date: Fri, 17 Jul 2020 15:02:20 -0700
Subject: [15] RFR(T) 8249681 :
 gc/stress/TestJNIBlockFullGC/TestJNIBlockFullGC.java fails w/
 UnsatisfiedLinkError
In-Reply-To: <b133dd7e-a919-4050-bd18-5c9545c5c82f@default>
References: <b133dd7e-a919-4050-bd18-5c9545c5c82f@default>
Message-ID: <2057C53D-B1C2-40FB-A3F3-46C02585C71F@oracle.com>

thanks Thomas, pushed to jdk15.

-- Igor

> On Jul 17, 2020, at 1:50 PM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> 
> ... and indeed trivial.
> 
> Thomas
> 
> ----- Original Message -----
> From: thomas.schatzl at oracle.com
> To: igor.ignatyev at oracle.com
> Cc: hotspot-gc-dev at openjdk.java.net
> Sent: Friday, July 17, 2020 10:49:42 PM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna
> Subject: Re: [15] RFR(T) 8249681 : gc/stress/TestJNIBlockFullGC/TestJNIBlockFullGC.java fails w/ UnsatisfiedLinkError
> 
> Hi,
> 
>  looks good.
> 
> Thomas
> 
> ----- Original Message -----
> From: igor.ignatyev at oracle.com
> To: hotspot-gc-dev at openjdk.java.net
> Sent: Friday, July 17, 2020 8:58:48 PM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna
> Subject: [15] RFR(T) 8249681 : gc/stress/TestJNIBlockFullGC/TestJNIBlockFullGC.java fails w/ UnsatisfiedLinkError
> 
> http://cr.openjdk.java.net/~iignatyev//8249681/webrev.00
>> 2 lines changed: 0 ins; 0 del; 2 mod;
> 
> 
> Hi all,
> 
> could you please review this trivial fix for TestJNIBlockFullGC test?
> 
> gc/stress/TestJNIBlockFullGC/TestJNIBlockFullGC.java fails w/ UnsatisfiedLinkError 'int gc.stress.TestJNIBlockFullGC.TestJNIBlockFullGC.TestCriticalArray0(int[])' b/c TestJNIBlockFullGC was moved to gc.stress.TestJNIBlockFullGC package by 8214799, but the JNI method hasn't been renamed.
> 
> testing: gc/stress/TestJNIBlockFullGC/TestJNIBlockFullGC.java on {windows,linux,macos}-x64
> webrev: http://cr.openjdk.java.net/~iignatyev//8249681/webrev.00
> JBS: https://bugs.openjdk.java.net/browse/JDK-8249681
> 
> PS the test fails w/ OOME and is still problem-listed due to 8192647
> 
> Thanks,
> -- Igor



From rs at jelastic.com  Fri Jul 17 22:03:51 2020
From: rs at jelastic.com (Ruslan Synytsky)
Date: Sat, 18 Jul 2020 00:03:51 +0200
Subject: Analyzing Efficiency Of Java Memory Usage
In-Reply-To: <CA+3eh114C6V8C-La3_E+t2kwqhS+g7p+G+Y6gjZ1pCizz1943g@mail.gmail.com>
References: <CA++bR4P9W+GYhVen_iwp98=yJLU0H6bG+-8b1Bci-R=xtYL34g@mail.gmail.com>
 <CA+3eh11uzzD492s-nYd3e6nCmqLJx2i+1EPnn-PVGSTXjyE9jA@mail.gmail.com>
 <CAA-vtUxEVdxiZ47VBnJDuRpyZhPAN48n05qj9iboxYbwBM0i5g@mail.gmail.com>
 <CA+3eh114C6V8C-La3_E+t2kwqhS+g7p+G+Y6gjZ1pCizz1943g@mail.gmail.com>
Message-ID: <CA++bR4PMXDsOf++8infv3Hc=BB4O-UJm=cDeGNQRMdnoXfk-Kg@mail.gmail.com>

Hi Volker and Thomas, thank you for the feedback.

On Fri, 17 Jul 2020 at 21:59, Volker Simonis <volker.simonis at gmail.com>
wrote:

> On Fri, Jul 17, 2020 at 5:18 PM Thomas St?fe <thomas.stuefe at gmail.com>
> wrote:
> >
> > Did Ruslan use NMT to get those numbers?
>
> No, probably they used the JMX MemoryMXBean. But that reports both,
> "committed" and "used", and "used" should be pretty much the same like
> "touched" (or RSS). E.g.
>
> MemoryUsage G1MonitoringSupport::memory_usage() {
>   MutexLocker x(MonitoringSupport_lock, Mutex::_no_safepoint_check_flag);
>   return MemoryUsage(InitialHeapSize, _overall_used,
> _overall_committed, _g1h->max_capacity());
> }
>
Exactly, I used MemoryMXBean as I have not found another way to collect the
required metrics without interrupting the running java processes. Please
take a look at the source code for a better understanding of how the stats
were collected.

The main shell script
https://github.com/siruslan/java-memory-usage/blob/master/jmu.sh
The collector of java memory usage metrics
https://github.com/siruslan/java-memory-usage/blob/master/src/jmu/MemoryUsageCollector.java

Btw, none from the checked java processes has enabled native memory
tracking...

>>> Interesting findings that a) sometimes committed heap memory is
> >>> higher than real OS used memory

>> Yes, this is a long standing, annoying problem which is most probably
> >> caused by the fact "Committed" as reported by the VM/NMT is not really
> >> the same like RSS reported by system tools. I've opened "8249666:
> >> Improve Native Memory Tracking to report the actual RSS usage" [0] to
> >> track this. Here's an excerpt from the issue describing the problem:
> >>
> >> Currently, NMT shows allocated memory as either "Reserved" or
> >> "Committed". Reserved memory is actually just reserved, virtual
> >> address space which was mmaped with MAP_NORESERVE, while Committed
> >> memory is mapped without MAP_NORESERVE. In the output of top or pmap,
> >> both Reserved and Committed show up as "Virtual" memory until they
> >> will be used for the first time (i.e. touched). Only after a memory
> >> page (usually 4k) has been written to for the first time, it will
> >> consume physical memory and appear in the "resident set" (i.e. RSS) of
> >> top's/pmap's output.
> >>
> >> The difference between allocating memory with or without MAP_NORESERVE
> >> depends on the Linux memory overcommit configuration [1]. By default,
> >> overcommit is allowed and memory allocated with MAP_NORESERVE isn't
> >> checked against the available physical memory (see man proc(5) [2]).
> >> If the HotSpot VM tries to commit reserved memory (i.e. re-map a
> >> memory region without MAP_NORESERVE which was previously mapped with
> >> MAP_NORESERVE) and there's not enough free memory available an
> >> OutOfMemoyError will be thrown.
> >>
> >> But even committing a memory region doesn't mean that physical memory
> >> pages will be allocated for that region (and accounted in the
> >> processes RSS) until that memory will be written to for the first
> >> time. So depending on the overcommit settings, an application might
> >> still crash with a SIGBUS because it is running out of physical memory
> >> when touching memory for the first time which was committed a long
> >> time ago.
> >>
> >> The main problem with the current NMT output is that it can not
> >> distinguish between touched and untouched Committed memory. If a VM is
> >> started with -Xms1g -Xmx1g the VM will commit the whole 1g heap and
> >> NMT will report Reserved=Committed=1g. In contrast, system tools like
> >> ps/top will only show the part of the heap as RSS which has really
> >> been used (i.e. touched), usually just about 100m. This is at least
> >> confusing.
> >>
> >> But we can do better. We can use mincore() [3] to find the RSS part of
> >> the amount of memory which is accounted as Committed in NMT's output
> >> and report that instead (or in addition). Notice that this feature has
> >> already been implemented for threads stacks with "JDK-8191369: NMT:
> >> Enhance thread stack tracking" [4] and just needs to be extended to
> >> all other kinds of memory, starting with the Java heap.

>>
> >> Alternatively, instead of using mincore() we could use the information
> >> from /proc/<pid>/smaps (also accessible through the pmap [5] command
> >> line utility) directly and merge it with the NMT data to get a
> >> complete, annotated overview of the whole address space of a Java
> >> process.
>
This clarification helps. I confirm that in our case the committed memory
becomes counted as really used, and as a result billed by the cloud
platform, only after that memory is written for the first time. Then when
it's not needed anymore uncommitment should be triggered by JVM for
releasing it back to OS, so other processes or even containers can use it.


> >> > , and b) in many cases non heap committed
> >> > memory is as big as heap committed.
>

I just added to the graph a new metric NonHeapUsed+ (royal blue) which
equals the used non heap stacked on top of the used heap memory. In our
case, this metric is often relatively close to NonHeapCommitted+. It will
be very helpful to get similar stats from other companies, so we can get a
better understanding of the real world. The source code of the metrics
collector is available now for a security check. We will skip collection of
the hostname and java start options by default, so the data will be fully
anonymized. Please let me know if you can join the analysis for making the
Java world a more efficient place.

Thanks
-- 
Ruslan


From kim.barrett at oracle.com  Sat Jul 18 06:43:39 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Sat, 18 Jul 2020 02:43:39 -0400
Subject: [aarch64-port-dev ] RFR(S): 8248851: CMS: Missing memory fences
 between free chunk check and klass read
In-Reply-To: <DA41BE1DDCA941489001C7FBD7A8820EE7E754F4@dggeml507-mbx.china.huawei.com>
References: <DA41BE1DDCA941489001C7FBD7A8820EE7E63D86@dggeml527-mbx.china.huawei.com>
 <d6590e74-7ce2-70b9-7432-96b969686ab7@redhat.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E6466E@dggeml527-mbx.china.huawei.com>
 <8957BB9E-617D-4E24-9CC4-31AF7D01D12E@oracle.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E64ECD@dggeml527-mbx.china.huawei.com>
 <65012FED-2A93-4899-939E-887175AB07AC@oracle.com>
 <06646cf3-bc21-977f-96e2-1e9a974d1b00@redhat.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E74B25@dggeml507-mbx.china.huawei.com>
 <48d0abe9-29a1-a67f-b8f0-c823e15ce110@oracle.com>
 <561611d4-18aa-f9d8-7302-3d89aff4ad81@redhat.com>
 <f1b98db1-63ee-4f60-7001-7952a9ea5065@oracle.com>
 <DA41BE1DDCA941489001C7FBD7A8820EE7E754F4@dggeml507-mbx.china.huawei.com>
Message-ID: <ED261569-06C9-4C80-9861-9C8164229EC8@oracle.com>

> On Jul 16, 2020, at 11:05 PM, Yangfei (Felix) <felix.yang at huawei.com> wrote:
> 
> Hi,
> 
>  Thanks for the suggestions.  It makes sense to me. 
>  BTW: OrderAccess::loadload() and OrderAccess::acquire() both map to the same instruction for aarch64: dmb ishld. 
>  Updated webrev: 
>    http://cr.openjdk.java.net/~fyang/8248851-8u/webrev.01/ 
>    http://cr.openjdk.java.net/~fyang/8248851-11u/webrev.01/ 
>    http://cr.openjdk.java.net/~fyang/8248851-13u/webrev.01/ 
> 
>  Performed the same test as before, result looks good. 
>  Does it look better? 

Looks good.



From thomas.schatzl at oracle.com  Mon Jul 20 10:07:38 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Mon, 20 Jul 2020 12:07:38 +0200
Subject: [15?] RFR (S): 8249192: MonitorInfo stores raw oops across safepoints
Message-ID: <3f838c8a-6e70-cd94-1753-8abe269b7cdf@oracle.com>

Hi all,

   can I get some reviews to handle'ize some raw oops in the MonitorInfo 
class?

(Afaiu only) in LiveFrameStream::monitors_to_object_array() we try to 
allocate an objArray with raw oops held in the MonitorInfo class that 
are passed in a GrowableArray. This allocation can lead to a garbage 
collection, with the usual random crashes.

This change changes the raw oops in MonitorInfo to Handles, and adds a 
few HandleMarks along the way to make these handles go away asap.

This issue has been introduced in JDK-8140450: Implement Stack-Walking 
API in jdk9.

The CR has been triaged as P3, but I would like to ask whether it might 
be good to increase its priority to P2 and apply for inclusion in 15. My 
arguments are as follows:

- the original issue why I started looking at this were lots of 
seemingly random crashes (5 or 6 were reported and the change 
temporarily backed out for this reason) in tier8 with a g1 change that 
changed young gen sizing. These crashes including that young gen sizing 
change are all gone now with this bugfix.
I.e. this suggests that so far we seem to have not encountered this 
issue more frequently due to pure luck wrt to generation sizing.

- it affects all collectors (naturally).

- there are quite a few user reported random crashes with IntelliJ and 
variants, which due to the nature of IDEs tending to retrieve stack 
traces fairly frequently would be more affected than usual. So I suspect 
at least some of them to be caused by this issue, these are the only raw 
oops I am aware of.

My understanding of the cause and fix is fairly good, but I am no expert 
in this area, so I would like to defer to you about this suggestion. The 
change is imo important enough to be backported to 11 and 15 anyway, but 
the question is about the risk/reward tradeoff wrt to bringing it to 15 
and not 15.0.1.

CR:
https://bugs.openjdk.java.net/browse/JDK-8249192
Webrev:
http://cr.openjdk.java.net/~tschatzl/8249192/webrev/
Testing:
tier1-5,tier8 (with some unrelated changes), 1800+ runs of the reproducer

Thanks,
   Thomas



From Divino.Cesar at microsoft.com  Tue Jul 21 01:41:22 2020
From: Divino.Cesar at microsoft.com (Cesar Soares Lucas)
Date: Tue, 21 Jul 2020 01:41:22 +0000
Subject: RFR: 8249225 - Move definition of PADDING_ELEM_NUM
Message-ID: <SN6PR2101MB089344B2E6079D7E207EA5A19A780@SN6PR2101MB0893.namprd21.prod.outlook.com>

Bug: https://bugs.openjdk.java.net/browse/JDK-8249225

Webrev: http://cr.openjdk.java.net/~adityam/cesar/8249225/



Please review the above linked patch for the mentioned bug. It's a small change

that moves the definition of "PADDING_ELEM_NUM" to the only file that uses it.

I also took the chance of converting it from a "#define" directive into a "const"

declaration - I can revert it back to a #define if you think it's better.





Thank you,

Cesar





From rkennke at redhat.com  Tue Jul 21 14:38:19 2020
From: rkennke at redhat.com (rkennke at redhat.com)
Date: Tue, 21 Jul 2020 16:38:19 +0200
Subject: RFR: 8249801: Shenandoah: Clear soft-refs on requested GC cycle
Message-ID: <03a76972dec4222285cf7aff307714448b6029a3.camel@redhat.com>

Shenandoah is not clearing soft-refs on a requested (by System.gc() or
implicitely) GC. We should do that. It is breaking
java/util/logging/Logger/bundleLeak/BundleTest.java that exects soft-
refs to evevtually be cleared by System.gc().

Bug:
https://bugs.openjdk.java.net/browse/JDK-8249801

Webrev:
http://cr.openjdk.java.net/~rkennke/JDK-8249801/webrev.01/

Testing: hotspot_gc_shenandoah, tier1&2 with Shenandoah

Ok?

Thanks,
Roman



From shade at redhat.com  Tue Jul 21 14:44:48 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Tue, 21 Jul 2020 16:44:48 +0200
Subject: RFR: 8249801: Shenandoah: Clear soft-refs on requested GC cycle
In-Reply-To: <03a76972dec4222285cf7aff307714448b6029a3.camel@redhat.com>
References: <03a76972dec4222285cf7aff307714448b6029a3.camel@redhat.com>
Message-ID: <5115c525-afdf-b16a-ffa0-71ac227985ba@redhat.com>

On 7/21/20 4:38 PM, rkennke at redhat.com wrote:
> Webrev:
> http://cr.openjdk.java.net/~rkennke/JDK-8249801/webrev.01/

Looks fine.

Let's leave the _gc_requested flag alone (the intent to read it at the beginning), and instead poll
explicit/implicit gc requests, plus fix the comment:

diff -r e0788e02e0de src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp
--- a/src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp       Tue Jul 21 14:07:45 2020 +0200
+++ b/src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp       Tue Jul 21 16:43:46 2020 +0200
@@ -175,6 +175,6 @@

     // Blow all soft references on this cycle, if handling allocation failure,
-    // or we are requested to do so unconditionally.
-    if (alloc_failure_pending || ShenandoahAlwaysClearSoftRefs) {
+    // either implicit or explicit GC request,  or we are requested to do so unconditionally.
+    if (alloc_failure_pending || implicit_gc_requested || explicit_gc_requested ||
ShenandoahAlwaysClearSoftRefs) {
       heap->soft_ref_policy()->set_should_clear_all_soft_refs(true);
     }

-- 
Thanks,
-Aleksey


From kim.barrett at oracle.com  Wed Jul 22 08:04:57 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Wed, 22 Jul 2020 04:04:57 -0400
Subject: RFR: 8188055: (ref) Add Reference.refersTo predicate
In-Reply-To: <8f0baa70-0bc3-caa7-13eb-364c45a466d6@oracle.com>
References: <E125A0B6-7FC4-4007-A9C4-7A7DFB86EEE4@oracle.com>
 <8f0baa70-0bc3-caa7-13eb-364c45a466d6@oracle.com>
Message-ID: <F353471D-8C42-4A8F-A63A-70CA20FAD98F@oracle.com>

> On Apr 8, 2020, at 5:27 AM, David Holmes <david.holmes at oracle.com> wrote:
> 
> Hi Kim,

Apparently I lost track of these comments and forgot to respond.

I still won't be sending out a new webrev until some of the other
discussion gets settled.  There's been some internal discussion, but
I'm currently waiting on some other folks to have time to chime in.

Replies to your comments inline below.

> On 8/04/2020 10:25 am, Kim Barrett wrote:
>> [Note review on both core-libs and hotspot-gc-dev lists; try not to lose
>> either when replying.]
>> Please review a new function: java.lang.ref.Reference.refersTo.
>> This function is needed to test the referent of a Reference object
>> without artificially extending the lifetime of the referent object, as
>> may happen when calling Reference.get.  Some garbage collectors
>> require extending the lifetime of a weak referent when accessed, in
>> order to maintain collector invariants.  Lifetime extension may occur
>> with any collector when the Reference is a SoftReference, as calling
>> get indicates recent access.  This new function also allows testing
>> the referent of a PhantomReference, which can't be accessed by calling
>> get.
> 
> This causes a slight conflict with the documentation for get() which states:
> 
> "Because the referent of a phantom reference is always inaccessible ..."
> 
> when the new method obviously accesses it.

I take that "inaccessible" to mean "inaccessible to the application",
and refersTo doesn't make the referent accessible to the application.

>> The new function uses a native method whose implementation is in the
>> VM so it can use the Access API.  It is the intent that this function
>> will be intrinsified by optimizing compilers like C2 or graal, but
>> that hasn't been implemented yet.  Bear that in mind before rushing
>> off to change existing uses of Reference.get.
> 
> I assume such intrinsification is intended for JDK 15 as well though, as end users may well rush to change their code!

Looks like we missed the JDK 15 train.

>> CR:
>> https://bugs.openjdk.java.net/browse/JDK-8188055
>> https://bugs.openjdk.java.net/browse/JDK-8241029 (CSR)
> 
> In the specification:
> 
> * @param   obj is the object to compare with the referenced object, or
> *          {@code null}.
> 
> First delete "is", the commone style for @param is just to say "the xxx" or "a yyy?.

Done locally.

> Second I suggest:
> 
> s/the referenced object/this reference object's referent/

Done locally.

> In the apinote:
> 
> * collection cycle.  {@link #refersTo(Object) refersTo} can be used to
> 
> I suggest:
> 
> * collection cycle.  The {@link #refersTo(Object) refersTo} method can be used to
> 
> so we don't start a sentence with a lower-case code-font word.

Done locally.

> Also a query in the apinote:
> 
> * This method returns a strong reference to the referent. This may cause
> * the garbage collector to treat it as strongly reachable until some later
> 
> Surely if the method returns a strong reference then the GC _will_ treat it as strongly reachable, not "may? ?

Something like refersTo is needed because an access using get may
force some phases of some collectors to treat the referent as strongly
reachable for some additional period, even if the application
immediately drops all references to it.  Other collectors may not need
to do anything of the sort.  And even collectors that do sometimes
need to do so may not always need to do so.  It's all vaguely
weasel-worded because there is so much potential variation.

>> Webrev:
>> https://cr.openjdk.java.net/~kbarrett/8188055/open.04/
> 
> Code changes look good. No comment on the test - I'll leave it to others to analyse that.

Thanks.



From albert.m.yang at oracle.com  Wed Jul 22 09:51:57 2020
From: albert.m.yang at oracle.com (Albert Yang)
Date: Wed, 22 Jul 2020 02:51:57 -0700 (PDT)
Subject: RFR (S): 8242036: G1 HeapRegionRemSet::_n_coarse_entries could be a
 bool
Message-ID: <997c8aed-779f-4529-afcc-90fb60475870@default>

Hi,

Changed the type from `size_t` to `bool`, since we only care whether if it's zero or not.

https://bugs.openjdk.java.net/browse/JDK-8242036

http://cr.openjdk.java.net/~lkorinth/albert/8242036/0/

/Albert


From hohensee at amazon.com  Wed Jul 22 22:48:09 2020
From: hohensee at amazon.com (Hohensee, Paul)
Date: Wed, 22 Jul 2020 22:48:09 +0000
Subject: RFR(L): 8215624: add parallel heap inspection support for jmap
 histo(G1)(Internet mail)
Message-ID: <EF17E5D8-648B-46DA-82DE-0BF4CFDD42D6@amazon.com>

Just small things.



heapInspection.cpp:



In ParHeapInspectTask::work, remove the final return statement and fix the following ?}? indent. I.e., replace



+    Atomic::store(&_success, false);

+    return;

+   }



with



+    Atomic::store(&_success, false);

+  }



In HeapInspection::heap_inspection, missed_count should be a uint to match other missed_count declarations, and should be initialized to the result of populate_table() rather than separately to 0.



attachListener.cpp:



In heap_inspection, initial_processor_count returns an int, so cast its result to a uint.



Similarly, parse_uintx returns a uintx, so cast its result (num) to uint when assigning to parallel_thread_num.



BasicJMapTest.java:



I took the liberty of refactoring testHisto*/histoToFile/testDump*/dump to remove redundant interposition methods and make histoToFile and dump look as similar as possible.



Webrev with the above changes in



http://cr.openjdk.java.net/~phh/8214535/webrev.01/



Thanks,

Paul



?On 7/15/20, 2:13 AM, "linzang(??)" <linzang at tencent.com> wrote:



     Upload a new webrev at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07/

     It fix a potential issue that unexpected number of threads maybe calculated for "parallel" option of jmap -histo in container.

    As shown at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07-delta/src/hotspot/share/services/attachListener.cpp.udiff.html



    ############### attachListener.cpp ####################

    @@ -252,11 +252,11 @@

     static jint heap_inspection(AttachOperation* op, outputStream* out) {

       bool live_objects_only = true;   // default is true to retain the behavior before this change is made

       outputStream* os = out;   // if path not specified or path is NULL, use out

       fileStream* fs = NULL;

       const char* arg0 = op->arg(0);

    -  uint parallel_thread_num = MAX(1, os::processor_count() * 3 / 8); // default is less than half of processors.

    +  uint parallel_thread_num = MAX(1, os::initial_active_processor_count() * 3 / 8); // default is less than half of processors.

       if (arg0 != NULL && (strlen(arg0) > 0)) {

         if (strcmp(arg0, "-all") != 0 && strcmp(arg0, "-live") != 0) {

           out->print_cr("Invalid argument to inspectheap operation: %s", arg0);

           return JNI_ERR;

         }

    ###################################################



    Thanks.



    BRs,

   Lin



    On 2020/7/9, 3:22 PM, "linzang(??)" <linzang at tencent.com> wrote:



        Hi Paul,

            Thanks for reviewing!

            >>

            >>     I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp.

            >>

            The reason I made the change in Jmap.java that compose all arguments as 1 string , instead of passing 3 argments, is to avoid the compatibility issue, as we discussed in http://mail.openjdk.java.net/pipermail/serviceability-dev/2019-February/thread.html#27240.  The root cause of the compatibility issue is because max argument count in HotspotVirtualMachineImpl.java and attachlistener.cpp need to be enlarged (changes like http://hg.openjdk.java.net/jdk/jdk/rev/e7cf035682e3#l2.1) when jmap has more than 3 arguments. But if user use an old jcmd/jmap tool, it may stuck at socket read(), because the "max argument count" don't match.

             I re-checked this change, the argument count of jmap histo is equal to 3 (live, file, parallel), so it can work normally even without the change of passing argument. But I think we have to face the problem if more arguments is added in jcmd alike tools later, not sure whether it should be sloved (or a workaround) in this changeset.



            And here are the lastest webrev and delta:

            http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06/

            http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06-delta/



        Cheers,

        Lin



        On 2020/7/7, 5:57 AM, "Hohensee, Paul" <hohensee at amazon.com> wrote:



            I'd like to see this feature added. :)



            The CSR looks good, as does the basic parallel inspection algorithm. Stefan's done the GC part, so I'll stick to the non-GC part (fwiw, the GC part lgtm).



            I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp.



            heapInspection.hpp:



            _shared_miss_count (s/b _missed_count, see below) isn't a size, so it should be a uint instead of a size_t. Same with the new parallel_thread_num argument to heap_inspection() and populate_table().



            Comment copy-edit:

            +// Parallel heap inspection task. Parallel inspection can fail due to

            +// a native OOM when allocating memory for TL-KlassInfoTable.

            +// _success will be set false on an OOM, and serial inspection tried.



            _shared_miss_count should be _missed_count to match the missed_count() getter, or rename missed_count() to be shared_miss_count(). Whichever way you go, the field type should match the getter result type: uint is reasonable.



            heapInspection.cpp:



            You might use ResourceMark twice in populate_table, separately for the parallel attempt and the serial code. If the parallel attempt fails and available memory is low, it would be good to clean up the memory used by the parallel attempt before doing the serial code.



            Style nit in KlassInfoTable::merge_entry(). I'd line up the definitions of k and elt, so "k" is even with "elt". And, because it's two lines shorter, I'd replace

            +  } else {

            +    return false;

            +  }

            with

            +  return false;



            KlassInfoTableMergeClosure.is_success() should be just success() (i.e., no "is_" prefix) because it's a getter.



            I'd reorganize the code in populate_table() to make it more clear, vis (I changed _shared_missed_count to _missed_count)

            +  if (cit.allocation_failed()) {

            +    // fail to allocate memory, stop parallel mode

            +    Atomic::store(&_success, false);

            +    return;

            +  }

            +  RecordInstanceClosure ric(&cit, _filter);

            +  _poi->object_iterate(&ric, worker_id);

            +  missed_count = ric.missed_count();

            +  {

            +    MutexLocker x(&_mutex);

            +    merge_success = _shared_cit->merge(&cit);

            +  }

            +  if (merge_success) {

            +    Atomic::add(&_missed_count, missed_count);

            +  else {

            +    Atomic::store(&_success, false);

            +  }



            Thanks,

            Paul



            On 6/29/20, 7:20 PM, "linzang(??)" <linzang at tencent.com> wrote:



                Dear All,

                        Sorry to bother again, I just want to make sure that is this change worth to be continue to work on? If decision is made to not. I think I can drop this work and stop asking for help reviewing...

                        Thanks for all your help about reviewing this previously.



                BRs,

                Lin



                On 2020/5/9, 3:47 PM, "linzang(??)" <linzang at tencent.com> wrote:



                    Dear All,

                           May I ask your help again for review the latest change?  Thanks!



                    BRs,

                    Lin



                    On 2020/4/28, 1:54 PM, "linzang(??)" <linzang at tencent.com> wrote:



                        Hi Stefan,

                          >>  - Adding Atomic::load/store.

                          >>  - Removing the time measurement in the run_task. I renamed G1's function

                          >>  to run_task_timed. If we need this outside of G1, we can rethink the API

                          >>  at that point.

                           >>  - ZGC style cleanups

                           Thanks for revising the patch,  they are all good to me, and I have made a tiny change based on it:

                               http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04/

                               http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04-delta/

                          it reduce the scope of mutex in ParHeapInspectTask, and delete unnecessary comments.



                        BRs,

                        Lin



                        On 2020/4/27, 4:34 PM, "Stefan Karlsson" <stefan.karlsson at oracle.com> wrote:



                            Hi Lin,



                            On 2020-04-26 05:10, linzang(??) wrote:

                            > Hi Stefan and Paul?

                            >      I have made a new patch based on your comments and Stefan's Poc code:

                            >      Webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03/

                            >      Delta(based on Stefan's change:) : http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03-delta/webrev_03-delta/



                            Thanks for providing a delta patch. It makes it much easier to look at,

                            and more likely for reviewers to continue reviewing.



                            I'm going to continue focusing on the GC parts, and leave the rest to

                            others to review.



                            >

                            >      And Here are main changed I made and want to discuss with you:

                            >      1.  changed"parallelThreadNum=" to "parallel=" for jmap -histo options.

                            >      2.  Add logic to test where parallelHeapInspection is fail, in heapInspection.cpp

                            >            This is because the parHeapInspectTask create thread local KlassInfoTable in it's work() method, and this may fail because of native OOM, in this case, the parallel should fail and serial heap inspection can be tried.

                            >            One more thing I want discuss with you is about the member "_success" of parHeapInspectTask, when native OOM happenes, it is set to false. And since this "set" operation can be conducted in multiple threads, should it be atomic ops?  IMO, this is not necessary because "_success" can only be set to false, and there is no way to change it from back to true after the ParHeapInspectTask instance is created, so it is save to be non-atomic, do you agree with that?



                            In these situations you should be using the Atomic::load/store

                            primitives. We're moving toward a later C++ standard were data races are

                           considered undefined behavior.



                            >     3. make CollectedHeap::run_task() be an abstract virtual func, so that every subclass of collectedHeap should support it, so later implementation of new collectedHeap will not miss the "parallel" features.

                            >           The problem I want to discuss with you is about epsilonHeap and SerialHeap, as they may not need parallel heap iteration, so I only make task->work(0), in case the run_task() is invoked someway in future. Another way is to left run_task()  unimplemented, which one do you think is better?



                            I don't have a strong opinion about this.



                              And also please help take a look at the zHeap, as there is a class

                            zTask that wrap the abstractGangTask, and the collectedHeap::run_task()

                            only accept  AbstraceGangTask* as argument, so I made a delegate class

                            to adapt it , please see src/hotspot/share/gc/z/zHeap.cpp.

                            >

                            >        There maybe other better ways to sovle the above problems, welcome for any comments, Thanks!



                            I've created a few cleanups and changes on top of your latest patch:



                            https://cr.openjdk.java.net/~stefank/8215624/webrev.02.delta

                            https://cr.openjdk.java.net/~stefank/8215624/webrev.02



                            - Adding Atomic::load/store.

                            - Removing the time measurement in the run_task. I renamed G1's function

                            to run_task_timed. If we need this outside of G1, we can rethink the API

                            at that point.

                            - ZGC style cleanups



                            Thanks,

                            StefanK



                            >

                            > BRs,

                            > Lin

                            >

                            > On 2020/4/23, 11:08 AM, "linzang(??)" <linzang at tencent.com> wrote:

                            >

                            >      Thanks Paul! I agree with using "parallel", will make the update in next patch, Thanks for help update the CSR.

                            >

                            >      BRs,

                            >      Lin

                            >

                            >      On 2020/4/23, 4:42 AM, "Hohensee, Paul" <hohensee at amazon.com> wrote:

                            >

                            >          For the interface, I'd use "parallel" instead of "parallelThreadNum". All the other options are lower case, and it's a lot easier to type "parallel". I took the liberty of updating the CSR. If you're ok with it, you might want to change variable names and such, plus of course JMap.usage.

                            >

                            >          Thanks,

                            >          Paul

                            >

                            >          On 4/22/20, 2:29 AM, "serviceability-dev on behalf of linzang(??)" <serviceability-dev-bounces at openjdk.java.net on behalf of linzang at tencent.com> wrote:

                            >

                            >              Dear Stefan,

                            >

                            >                      Thanks a lot! I agree with you to decouple the heap inspection code with GC's.

                            >                      I will start  from your POC code, may discuss with you later.

                            >

                            >

                            >              BRs,

                            >              Lin

                            >

                            >              On 2020/4/22, 5:14 PM, "Stefan Karlsson" <stefan.karlsson at oracle.com> wrote:

                            >

                            >                  Hi Lin,

                            >

                            >                  I took a look at this earlier and saw that the heap inspection code is

                            >                  strongly coupled with the CollectedHeap and G1CollectedHeap. I'd prefer

                            >                  if we'd abstract this away, so that the GCs only provide a "parallel

                            >                  object iteration" interface, and the heap inspection code is kept elsewhere.

                            >

                            >                  I started experimenting with doing that, but other higher-priority (to

                            >                  me) tasks have had to take precedence.

                            >

                            >                  I've uploaded my work-in-progress / proof-of-concept:

                            >                    https://cr.openjdk.java.net/~stefank/8215624/webrev.01.delta/

                            >                    https://cr.openjdk.java.net/~stefank/8215624/webrev.01/

                            >

                            >                  The current code doesn't handle the lifecycle (deletion) of the

                            >                  ParallelObjectIterators. There's also code left unimplemented in around

                            >                  CollectedHeap::run_task. However, I think this could work as a basis to

                            >                  pull out the heap inspection code out of the GCs.

                            >

                            >                  Thanks,

                            >                  StefanK

                            >

                            >                  On 2020-04-22 02:21, linzang(??) wrote:

                            >                  > Dear all,

                            >                  >       May I ask you help to review? This RFR has been there for quite a while.

                            >                  >       Thanks!

                            >                  >

                            >                  > BRs,

                            >                  > Lin

                            >                  >

                            >                  > > On 2020/3/16, 5:18 PM, "linzang(??)" <linzang at tencent.com> wrote:>

                            >                  >

                            >                  >>    Just update a new path, my preliminary measure show about 3.5x speedup of jmap histo on a nearly full 4GB G1 heap (8-core platform with parallel thread number set to 4).

                            >                  >>     webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_02/

                            >                  >>     bug: https://bugs.openjdk.java.net/browse/JDK-8215624

                            >                  >>     CSR: https://bugs.openjdk.java.net/browse/JDK-8239290

                            >                  >>     BRs,

                            >                  >>       Lin

                            >                  >>       > On 2020/3/2, 9:56 PM, "linzang(??)" <linzang at tencent.com> wrote:

                            >                  >>       >

                            >                  >>       >    Dear all,

                            >                  >>       >          Let me try to ease the reviewing work by some explanation :P

                            >                  >>       >          The patch's target is to speed up jmap -histo for heap iteration, from my experience it is necessary for large heap investigation. E.g in bigData scenario I have tried to conduct jmap -histo against 180GB heap, it does take quite a while.

                            >                  >>       >          And if my understanding is corrent, even the jmap -histo without "live" option does heap inspection with heap lock acquired. so it is very likely to block mutator thread in allocation-sensitive scenario. I would say the faster the heap inspection does, the shorter the mutator be blocked. This is parallel iteration for jmap is necessary.

                            >                  >>       >          I think the parallel heap inspection should be applied to all kind of heap. However, consider the heap layout are different for  GCs, much time is required to understand all kinds of the heap layout to make the whole change. IMO, It is not wise to have a huge patch for the whole solution at once, and it is even harder to review it. So I plan to implement it incrementally, the first patch (this one) is going to confirm the implemention detail of how jmap accept the new option, passes it to attachListener of the jvm process and then how to make the parallel inspection closure be generic enough to make it easy to extend to different heap layout. And also how to implement the heap inspection in specific gc's heap. This patch use G1's heap as the begining.

                            >                  >>       >          This patch actually do several things:

                            >                  >>       >          1. Add an option "parallelThreadNum=<N>" to jmap -histo, the default behavior is to set N to 0, means let's JVM decide how many threads to use for heap inspection. Set this option to 1 will disable parallel heap inspection. (more details in CSR: https://bugs.openjdk.java.net/browse/JDK-8239290)

                            >                  >>       >          2. Make a change in how Jmap passing arguments, changes in http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java.udiff.html, originally it pass options as separate arguments to attachListener, this patch change to that all options be compose to a single string. So the arg_count_max in attachListener.hpp do not need to be changed, and hence avoid the compatibility issue, as disscussed at https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-March/027334.html

                            >                  >>       >         3. Add an abstract class ParHeapInspectTask in heapInspection.hpp / heapInspection.cpp, It's work(uint worker_id) method prepares the data structure (KlassInfoTable) need for every parallel worker thread, and then call do_object_iterate_parallel() which is heap specific implementation. I also added some machenism in KlassInfoTable to support parallel iteration, such as merge().

                            >                  >>       >        4. In specific heap (G1 in this patch), create a subclass of ParHeapInspectTask, implement the do_object_iterate_parallel() for parallel heap inspection. For G1, it simply invoke g1CollectedHeap's object_iterate_parallel().

                            >                  >>       >        5. Add related test.

                            >                  >>       >        6. it may be easy to extend this patch for other kinds of heap by creating subclass of ParHeapInspectTask and implement the do_object_iterate_parallel().

                            >                  >>       >

                            >                  >>       >    Hope these info could help on code review and initate the discussion :-)

                            >                  >>       >    Thanks!

                            >                  >>       >

                            >                  >>       >    BRs,

                            >                  >>       >    Lin

                            >                  >>       >    >On 2020/2/19, 9:40 AM, "linzang(??)" <linzang at tencent.com> wrote:.

                            >                  >>       >    >

                            >                  >>       >    >  Re-post this RFR with correct enhancement number to make it trackable.

                            >                  >>       >    >  please ignore the previous wrong post. sorry for troubles.

                            >                  >>       >    >

                            >                  >>       >    >   webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/

                            >                  >>       >    >    Hi bug: https://bugs.openjdk.java.net/browse/JDK-8215624

                            >                  >>       >    >    CSR: https://bugs.openjdk.java.net/browse/JDK-8239290

                            >                  >>       >    >    --------------

                            >                  >>       >    >    Lin

                            >                  >>       >    >    >Hi Lin,

                            >                  >   >     >    >    >

                            >                  >>       >    >    >Could you, please, re-post your RFR with the right enhancement number in

                            >                  >>       >    >    >the message subject?

                            >                  >>       >    >    >It will be more trackable this way.

                            >                  >>       >    >    >

                            >                  >>       >    >    >Thanks,

                            >                  >>       >    >    >Serguei

                            >                  >>       >    >    >

                            >                  >>       >    >    >

                            >                  >>       >    >    >On 2/17/20 10:29 PM, linzang(??) wrote:

                            >                  >>       >    >    >> Dear David,

                            >                  >>       >    >    >>        Thanks a lot!

                            >                  >>       >    >    >>       I have updated the refined code to http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_01/.

                            >                  >>       >    >    >>        IMHO the parallel heap inspection can be extended to all kinds of heap as long as the heap layout can support parallel iteration.

                            >                  >>       >    >    >>        Maybe we can firstly use this webrev to discuss how to implement it, because I am not sure my current implementation is an appropriate way to communicate with collectedHeap, then we can extend the solution to other kinds of heap.

                            >                  >>       >    >    >>

                            >                  >>       >    >    >> Thanks,

                            >                  >>       >    >    >> --------------

                            >                  >>       >    >    >> Lin

                            >                  >>       >    >    >>> Hi Lin,

                            >                  >>       >    >    >>>

                            >                  >>       >    >    >>> Adding in hotspot-gc-dev as they need to see how this interacts with GC

                            >                  >>       >    >    >>> worker threads, and whether it needs to be extended beyond G1.

                            >                  >>       >    >    >>>

                            >                  >>       >    >   >>> I happened to spot one nit when browsing:

                            >                  >>       >    >    >>>

                            >                  >>       >    >    >>> src/hotspot/share/gc/shared/collectedHeap.hpp

                            >                  >>       >    >    >>>

                            >                  >>       >    >    >>> +   virtual bool run_par_heap_inspect_task(KlassInfoTable* cit,

                            >                  >>       >    >    >>> +                                          BoolObjectClosure* filter,

                            >                  >>       >    >    >>> +                                          size_t* missed_count,

                            >                  >>       >    >    >>> +                                          size_t thread_num) {

                            >                  >>       >    >    >>> +     return NULL;

                            >                  >>       >    >    >>>

                            >                  >>       >    >    >>> s/NULL/false/

                            >                  >>       >    >    >>>

                            >                  >>       >    >    >>> Cheers,

                            >                  >>       >    >    >>> David

                            >                  >   >     >    >    >>>

                            >                  >>       >    >    >>> On 18/02/2020 2:15 pm, linzang(??) wrote:

                            >                  >>       >    >    >>>> Dear All,

                            >                  >>       >    >    >>>>         May I ask your help to review the follow changes:

                            >                  >>       >    >    >>>>         webrev:

                            >                  >>       >    >    >>>> http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_00/

                            >                  >>       >    >    >>>>      bug: https://bugs.openjdk.java.net/browse/JDK-8215624

                            >                  >>       >    >    >>>>      related CSR: https://bugs.openjdk.java.net/browse/JDK-8239290

                            >                  >>       >    >    >>>>         This patch enable parallel heap inspection of G1 for jmap histo.

                            >                  >>       >    >    >>>>         my simple test shown it can speed up 2x of jmap -histo with

                            >                  >>       >    >    >>>> parallelThreadNum set to 2 for heap at ~500M on 4-core platform.

                            >                  >>       >    >    >>>>

                            >                  >>       >    >    >>>> ------------------------------------------------------------------------

                            >                  >>       >    >    >>>> BRs,

                            >                  >>       >    >    >>>> Lin

                            >                  >>       >    >    >> >

                            >                  >>       >    >    >

                            >                  >

                            >                  >

                            >                  >

                            >

                            >

                            >

                            >

                            >

                            >















From linzang at tencent.com  Thu Jul 23 03:56:43 2020
From: linzang at tencent.com (=?utf-8?B?bGluemFuZyjoh6fnkLMp?=)
Date: Thu, 23 Jul 2020 03:56:43 +0000
Subject: RFR(L): 8215624: add parallel heap inspection support for jmap
 histo(G1)(Internet mail)
In-Reply-To: <EF17E5D8-648B-46DA-82DE-0BF4CFDD42D6@amazon.com>
References: <EF17E5D8-648B-46DA-82DE-0BF4CFDD42D6@amazon.com>
Message-ID: <B8E2CD92-2E27-4EBD-9FE1-9F51FB627CE0@tencent.com>

Hi Paul,
     Thanks for your help, that all looks good to me. 
     Just 2 minor changes:  
	? delete the final return in ParHeapInspectTask::work, you mentioned it but seems not include in the webrev :-)
	? delete a unnecessary blank line in heapInspect.cpp at merge_entry()

#########################################################################
--- old/src/hotspot/share/memory/heapInspection.cpp ? ? 2020-07-23 11:23:29.281666456 +0800
+++ new/src/hotspot/share/memory/heapInspection.cpp ? ? 2020-07-23 11:23:29.017666447 +0800
@@ -251,7 +251,6 @@
?? ? _size_of_instances_in_words += cie->words();
?? ? return true;
?? }? ?
-
?? return false;
?}
?
@@ -568,7 +567,6 @@
?? ? Atomic::add(&_missed_count, missed_count);
?? } else {
?? ? Atomic::store(&_success, false);
- ? return;
?? }? ?
?}
#########################################################################
  

Here is the webrev  http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_08/

BRs,
Lin
---------------------------------------------
From: "Hohensee, Paul" <hohensee at amazon.com>
Date: Thursday, July 23, 2020 at 6:48 AM
To: "linzang(??)" <linzang at tencent.com>, Stefan Karlsson <stefan.karlsson at oracle.com>, "serguei.spitsyn at oracle.com" <serguei.spitsyn at oracle.com>, David Holmes <david.holmes at oracle.com>, serviceability-dev <serviceability-dev at openjdk.java.net>, "hotspot-gc-dev at openjdk.java.net" <hotspot-gc-dev at openjdk.java.net>
Subject: RE: RFR(L): 8215624: add parallel heap inspection support for jmap histo(G1)(Internet mail)

Just small things.
?
heapInspection.cpp:
?
In ParHeapInspectTask::work, remove the final return statement and fix the following ?}? indent. I.e., replace
?
+??? Atomic::store(&_success, false);
+??? return;
+?? }
?
with
?
+??? Atomic::store(&_success, false);
+? }
?
In HeapInspection::heap_inspection, missed_count should be a uint to match other missed_count declarations, and should be initialized to the result of populate_table() rather than separately to 0.
?
attachListener.cpp:
?
In heap_inspection, initial_processor_count returns an int, so cast its result to a uint.
?
Similarly, parse_uintx returns a uintx, so cast its result (num) to uint when assigning to parallel_thread_num.
?
BasicJMapTest.java:
?
I took the liberty of refactoring testHisto*/histoToFile/testDump*/dump to remove redundant interposition methods and make histoToFile and dump look as similar as possible.
?
Webrev with the above changes in
?
http://cr.openjdk.java.net/~phh/8214535/webrev.01/
?
Thanks,
Paul
?
On 7/15/20, 2:13 AM, "linzang(??)" <linzang at tencent.com> wrote:
?
???? Upload a new webrev at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07/
???? It fix a potential issue that unexpected number of threads maybe calculated for "parallel" option of jmap -histo in container.
??? As shown at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07-delta/src/hotspot/share/services/attachListener.cpp.udiff.html
?
??? ############### attachListener.cpp ####################
??? @@ -252,11 +252,11 @@
???? static jint heap_inspection(AttachOperation* op, outputStream* out) {
?????? bool live_objects_only = true;?? // default is true to retain the behavior before this change is made
?????? outputStream* os = out;?? // if path not specified or path is NULL, use out
?????? fileStream* fs = NULL;
?????? const char* arg0 = op->arg(0);
??? -? uint parallel_thread_num = MAX(1, os::processor_count() * 3 / 8); // default is less than half of processors.
??? +? uint parallel_thread_num = MAX(1, os::initial_active_processor_count() * 3 / 8); // default is less than half of processors.
?????? if (arg0 != NULL && (strlen(arg0) > 0)) {
???????? if (strcmp(arg0, "-all") != 0 && strcmp(arg0, "-live") != 0) {
?????????? out->print_cr("Invalid argument to inspectheap operation: %s", arg0);
?????????? return JNI_ERR;
???????? }
??? ###################################################
?
??? Thanks.
?
??? BRs,
???Lin
?
??? On 2020/7/9, 3:22 PM, "linzang(??)" <linzang at tencent.com> wrote:
?
??????? Hi Paul,
??????????? Thanks for reviewing!
??????????? >>
??????????? >>???? I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp.
??????????? >>
??????????? The reason I made the change in Jmap.java that compose all arguments as 1 string , instead of passing 3 argments, is to avoid the compatibility issue, as we discussed in http://mail.openjdk.java.net/pipermail/serviceability-dev/2019-February/thread.html#27240.? The root cause of the compatibility issue is because max argument count in HotspotVirtualMachineImpl.java and attachlistener.cpp need to be enlarged (changes like http://hg.openjdk.java.net/jdk/jdk/rev/e7cf035682e3#l2.1) when jmap has more than 3 arguments. But if user use an old jcmd/jmap tool, it may stuck at socket read(), because the "max argument count" don't match.
???????????? I re-checked this change, the argument count of jmap histo is equal to 3 (live, file, parallel), so it can work normally even without the change of passing argument. But I think we have to face the problem if more arguments is added in jcmd alike tools later, not sure whether it should be sloved (or a workaround) in this changeset.
?
??????????? And here are the lastest webrev and delta:
??????????? http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06/
??????????? http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06-delta/
?
??????? Cheers,
??????? Lin
?
??????? On 2020/7/7, 5:57 AM, "Hohensee, Paul" <hohensee at amazon.com> wrote:
?
??????????? I'd like to see this feature added. :)
?
??????????? The CSR looks good, as does the basic parallel inspection algorithm. Stefan's done the GC part, so I'll stick to the non-GC part (fwiw, the GC part lgtm).
?
??????????? I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp.
?
??????????? heapInspection.hpp:
?
??????????? _shared_miss_count (s/b _missed_count, see below) isn't a size, so it should be a uint instead of a size_t. Same with the new parallel_thread_num argument to heap_inspection() and populate_table().
?
??????????? Comment copy-edit:
??????????? +// Parallel heap inspection task. Parallel inspection can fail due to
??????????? +// a native OOM when allocating memory for TL-KlassInfoTable.
??????????? +// _success will be set false on an OOM, and serial inspection tried.
?
??????????? _shared_miss_count should be _missed_count to match the missed_count() getter, or rename missed_count() to be shared_miss_count(). Whichever way you go, the field type should match the getter result type: uint is reasonable.
?
??????????? heapInspection.cpp:
?
??????????? You might use ResourceMark twice in populate_table, separately for the parallel attempt and the serial code. If the parallel attempt fails and available memory is low, it would be good to clean up the memory used by the parallel attempt before doing the serial code.
?
??????????? Style nit in KlassInfoTable::merge_entry(). I'd line up the definitions of k and elt, so "k" is even with "elt". And, because it's two lines shorter, I'd replace
??????????? +? } else {
??????????? +??? return false;
??????????? +? }
??????????? with
??????????? +? return false;
?
??????????? KlassInfoTableMergeClosure.is_success() should be just success() (i.e., no "is_" prefix) because it's a getter.
?
??????????? I'd reorganize the code in populate_table() to make it more clear, vis (I changed _shared_missed_count to _missed_count)
??????????? +? if (cit.allocation_failed()) {
??????????? +??? // fail to allocate memory, stop parallel mode
??????????? +??? Atomic::store(&_success, false);
??????????? +??? return;
??????????? +? }
??????????? +? RecordInstanceClosure ric(&cit, _filter);
??????????? +? _poi->object_iterate(&ric, worker_id);
??????????? +? missed_count = ric.missed_count();
??????????? +? {
??????????? +??? MutexLocker x(&_mutex);
??????????? +??? merge_success = _shared_cit->merge(&cit);
??????????? +? }
??????????? +? if (merge_success) {
??????????? +??? Atomic::add(&_missed_count, missed_count);
??????????? +? else {
??????????? +??? Atomic::store(&_success, false);
??????????? +? }
?
??????????? Thanks,
??????????? Paul
?
??????????? On 6/29/20, 7:20 PM, "linzang(??)" <linzang at tencent.com> wrote:
?
??????????????? Dear All,
??????????????????????? Sorry to bother again, I just want to make sure that is this change worth to be continue to work on? If decision is made to not. I think I can drop this work and stop asking for help reviewing...
??????????????????????? Thanks for all your help about reviewing this previously.
?
??????????????? BRs,
??????????????? Lin
?
??????????????? On 2020/5/9, 3:47 PM, "linzang(??)" <linzang at tencent.com> wrote:
?
??????????????????? Dear All,
?????????????????????????? May I ask your help again for review the latest change?? Thanks!
?
??????????????????? BRs,
??????????????????? Lin
?
??????????????????? On 2020/4/28, 1:54 PM, "linzang(??)" <linzang at tencent.com> wrote:
?
??????????????????????? Hi Stefan,
????????????????????????? >>? - Adding Atomic::load/store.
????????????????????????? >>? - Removing the time measurement in the run_task. I renamed G1's function
????????????????????????? >>? to run_task_timed. If we need this outside of G1, we can rethink the API
????????????????????????? >>? at that point.
?????????????????????????? >>? - ZGC style cleanups
?? ????????????????????????Thanks for revising the patch,? they are all good to me, and I have made a tiny change based on it:
?????????????????????????????? http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04/
?????????????????????????????? http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04-delta/
????????????????????????? it reduce the scope of mutex in ParHeapInspectTask, and delete unnecessary comments.
?
??????????????????????? BRs,
??????????????????????? Lin
?
????????????????? ??????On 2020/4/27, 4:34 PM, "Stefan Karlsson" <stefan.karlsson at oracle.com> wrote:
?
??????????????????????????? Hi Lin,
?
??????????????????????????? On 2020-04-26 05:10, linzang(??) wrote:
??????????????????????????? > Hi Stefan and Paul?
???????????????? ???????????>????? I have made a new patch based on your comments and Stefan's Poc code:
??????????????????????????? >????? Webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03/
??????????????????????????? >????? Delta(based on Stefan's change:) : http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03-delta/webrev_03-delta/
?
??????????????????????????? Thanks for providing a delta patch. It makes it much easier to look at,
??????????????????????????? and more likely for reviewers to continue reviewing.
?
??????????????????????????? I'm going to continue focusing on the GC parts, and leave the rest to
??????????????????????????? others to review.
?
??????????????????????????? >
??????????????????????????? >????? And Here are main changed I made and want to discuss with you:
??????????????????????????? >????? 1.? changed"parallelThreadNum=" to "parallel=" for jmap -histo options.
??????????????????????????? >????? 2.? Add logic to test where parallelHeapInspection is fail, in heapInspection.cpp
??????????????????????????? >??????????? This is because the parHeapInspectTask create thread local KlassInfoTable in it's work() method, and this may fail because of native OOM, in this case, the parallel should fail and serial heap inspection can be tried.
??????????????????????????? >??????????? One more thing I want discuss with you is about the member "_success" of parHeapInspectTask, when native OOM happenes, it is set to false. And since this "set" operation can be conducted in multiple threads, should it be atomic ops?? IMO, this is not necessary because "_success" can only be set to false, and there is no way to change it from back to true after the ParHeapInspectTask instance is created, so it is save to be non-atomic, do you agree with that?
?
??????????????????????????? In these situations you should be using the Atomic::load/store
??????????????????????????? primitives. We're moving toward a later C++ standard were data races are
???????????????????????????considered undefined behavior.
?
??????????????????????????? >???? 3. make CollectedHeap::run_task() be an abstract virtual func, so that every subclass of collectedHeap should support it, so later implementation of new collectedHeap will not miss the "parallel" features.
??????????????????????????? >?????????? The problem I want to discuss with you is about epsilonHeap and SerialHeap, as they may not need parallel heap iteration, so I only make task->work(0), in case the run_task() is invoked someway in future. Another way is to left run_task()? unimplemented, which one do you think is better?
?
??????????????????????????? I don't have a strong opinion about this.
?
????????????????????????????? And also please help take a look at the zHeap, as there is a class
??????????????????????????? zTask that wrap the abstractGangTask, and the collectedHeap::run_task()
??????????????????????????? only accept? AbstraceGangTask* as argument, so I made a delegate class
??????????????????????????? to adapt it , please see src/hotspot/share/gc/z/zHeap.cpp.
??????????????????????????? >
??????????????????????????? >??????? There maybe other better ways to sovle the above problems, welcome for any comments, Thanks!
?
??????????????????????????? I've created a few cleanups and changes on top of your latest patch:
?
??????????????????????????? https://cr.openjdk.java.net/~stefank/8215624/webrev.02.delta
??????????????????????????? https://cr.openjdk.java.net/~stefank/8215624/webrev.02
?
??????????????????????????? - Adding Atomic::load/store.
??????????????????????????? - Removing the time measurement in the run_task. I renamed G1's function
????????????? ??????????????to run_task_timed. If we need this outside of G1, we can rethink the API
??????????????????????????? at that point.
??????????????????????????? - ZGC style cleanups
?
??????????????????????????? Thanks,
??????????????????????????? StefanK
?
?? ?????????????????????????>
??????????????????????????? > BRs,
??????????????????????????? > Lin
??????????????????????????? >
??????????????????????????? > On 2020/4/23, 11:08 AM, "linzang(??)" <linzang at tencent.com> wrote:
??????????????????????????? >
?? ?????????????????????????>????? Thanks Paul! I agree with using "parallel", will make the update in next patch, Thanks for help update the CSR.
??????????????????????????? >
??????????????????????????? >????? BRs,
??????????????????????????? >????? Lin
?? ?????????????????????????>
??????????????????????????? >????? On 2020/4/23, 4:42 AM, "Hohensee, Paul" <hohensee at amazon.com> wrote:
??????????????????????????? >
??????????????????????????? >????????? For the interface, I'd use "parallel" instead of "parallelThreadNum". All the other options are lower case, and it's a lot easier to type "parallel". I took the liberty of updating the CSR. If you're ok with it, you might want to change variable names and such, plus of course JMap.usage.
?????????????????????? ?????>
??????????????????????????? >????????? Thanks,
??????????????????????????? >????????? Paul
??????????????????????????? >
??????????????????????????? >????????? On 4/22/20, 2:29 AM, "serviceability-dev on behalf of linzang(??)" <serviceability-dev-bounces at openjdk.java.net on behalf of linzang at tencent.com> wrote:
??????????????????????????? >
??????????????????????????? >???? ?????????Dear Stefan,
??????????????????????????? >
??????????????????????????? >????????????????????? Thanks a lot! I agree with you to decouple the heap inspection code with GC's.
??????????????????????????? >????????????????????? I will start? from your POC code, may discuss with you later.
??????????????????????????? >
??????????????????????????? >
??????????????????????????? >????????????? BRs,
??????????????????????????? >????????????? Lin
??????????????????????????? >
??????????????????????????? >?? ???????????On 2020/4/22, 5:14 PM, "Stefan Karlsson" <stefan.karlsson at oracle.com> wrote:
??????????????????????????? >
??????????????????????????? >????????????????? Hi Lin,
??????????????????????????? >
??????????????????????????? >????????????????? I took a look at this earlier and saw that the heap inspection code is
??????????????????????????? >????????????????? strongly coupled with the CollectedHeap and G1CollectedHeap. I'd prefer
??????????????????????????? >????????????????? if we'd abstract this away, so that the GCs only provide a "parallel
??????????????????????????? >????????????????? object iteration" interface, and the heap inspection code is kept elsewhere.
??????????????????????????? >
??????????????????????????? >????????????????? I started experimenting with doing that, but other higher-priority (to
??????????????????????????? >????????????????? me) tasks have had to take precedence.
??????????????????????????? >
??????????????????????????? >????????????????? I've uploaded my work-in-progress / proof-of-concept:
??????????????????????????? >??????????????????? https://cr.openjdk.java.net/~stefank/8215624/webrev.01.delta/
??????????????????????????? >??????????????????? https://cr.openjdk.java.net/~stefank/8215624/webrev.01/
?????????????????? ?????????>
??????????????????????????? >????????????????? The current code doesn't handle the lifecycle (deletion) of the
??????????????????????????? >????????????????? ParallelObjectIterators. There's also code left unimplemented in around
??????????????????????????? >????????????????? CollectedHeap::run_task. However, I think this could work as a basis to
????????????????? ??????????>????????????????? pull out the heap inspection code out of the GCs.
??????????????????????????? >
??????????????????????????? >????????????????? Thanks,
??????????????????????????? >????????????????? StefanK
??????????????????????????? >
?????? ?????????????????????>????????????????? On 2020-04-22 02:21, linzang(??) wrote:
??????????????????????????? >????????????????? > Dear all,
??????????????????????????? >????????????????? >?????? May I ask you help to review? This RFR has been there for quite a while.
??????????????????????????? >????????????????? >?????? Thanks!
??????????????????????????? >????????????????? >
??????????????????????????? >????????????????? > BRs,
??????????????????????????? >????????????????? > Lin
????????????????????????? ??>????????????????? >
??????????????????????????? >????????????????? > > On 2020/3/16, 5:18 PM, "linzang(??)" <linzang at tencent.com> wrote:>
??????????????????????????? >????????????????? >
??????????????????????????? >????????????????? >>??? Just update a new path, my preliminary measure show about 3.5x speedup of jmap histo on a nearly full 4GB G1 heap (8-core platform with parallel thread number set to 4).
??????????????????????????? >????????????????? >>???? webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_02/
??????????????????????????? >????????????????? >>???? bug: https://bugs.openjdk.java.net/browse/JDK-8215624
??????????????????????????? >????????????????? >>???? CSR: https://bugs.openjdk.java.net/browse/JDK-8239290
?????????? ?????????????????>????????????????? >>???? BRs,
??????????????????????????? >????????????????? >>?????? Lin
??????????????????????????? >????????????????? >>?????? > On 2020/3/2, 9:56 PM, "linzang(??)" <linzang at tencent.com> wrote:
???????????????????????? ???>????????????????? >>?????? >
??????????????????????????? >????????????????? >>?????? >??? Dear all,
??????????????????????????? >????????????????? >>?????? >????????? Let me try to ease the reviewing work by some explanation :P
??????????????????????? ????>????????????????? >>?????? >????????? The patch's target is to speed up jmap -histo for heap iteration, from my experience it is necessary for large heap investigation. E.g in bigData scenario I have tried to conduct jmap -histo against 180GB heap, it does take quite a while.
??????????????????????????? >????????????????? >>?????? >????????? And if my understanding is corrent, even the jmap -histo without "live" option does heap inspection with heap lock acquired. so it is very likely to block mutator thread in allocation-sensitive scenario. I would say the faster the heap inspection does, the shorter the mutator be blocked. This is parallel iteration for jmap is necessary.
??????????????????????????? >????????????????? >>?????? >????????? I think the parallel heap inspection should be applied to all kind of heap. However, consider the heap layout are different for? GCs, much time is required to understand all kinds of the heap layout to make the whole change. IMO, It is not wise to have a huge patch for the whole solution at once, and it is even harder to review it. So I plan to implement it incrementally, the first patch (this one) is going to confirm the implemention detail of how jmap accept the new option, passes it to attachListener of the jvm process and then how to make the parallel inspection closure be generic enough to make it easy to extend to different heap layout. And also how to implement the heap inspection in specific gc's heap. This patch use G1's heap as the begining.
??????????????????????????? >??????????????? ??>>?????? >????????? This patch actually do several things:
??????????????????????????? >????????????????? >>?????? >????????? 1. Add an option "parallelThreadNum=<N>" to jmap -histo, the default behavior is to set N to 0, means let's JVM decide how many threads to use for heap inspection. Set this option to 1 will disable parallel heap inspection. (more details in CSR: https://bugs.openjdk.java.net/browse/JDK-8239290)
??????????????????????????? >????????????????? >>?????? >????????? 2. Make a change in how Jmap passing arguments, changes in http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java.udiff.html, originally it pass options as separate arguments to attachListener, this patch change to that all options be compose to a single string. So the arg_count_max in attachListener.hpp do not need to be changed, and hence avoid the compatibility issue, as disscussed at https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-March/027334.html
??????????????????????????? >????????????????? >>?????? >???????? 3. Add an abstract class ParHeapInspectTask in heapInspection.hpp / heapInspection.cpp, It's work(uint worker_id) method prepares the data structure (KlassInfoTable) need for every parallel worker thread, and then call do_object_iterate_parallel() which is heap specific implementation. I also added some machenism in KlassInfoTable to support parallel iteration, such as merge().
??????????????????????????? >????????????????? >>?????? >????? ??4. In specific heap (G1 in this patch), create a subclass of ParHeapInspectTask, implement the do_object_iterate_parallel() for parallel heap inspection. For G1, it simply invoke g1CollectedHeap's object_iterate_parallel().
??????????????????????????? > ?????????????????>>?????? >??????? 5. Add related test.
??????????????????????????? >????????????????? >>?????? >??????? 6. it may be easy to extend this patch for other kinds of heap by creating subclass of ParHeapInspectTask and implement the do_object_iterate_parallel().
??????????????????????????? >????????????????? >>?????? >
??????????????????????????? >????????????????? >>?????? >??? Hope these info could help on code review and initate the discussion :-)
??????????????????????????? >??????????????? ??>>?????? >??? Thanks!
??????????????????????????? >????????????????? >>?????? >
??????????????????????????? >????????????????? >>?????? >??? BRs,
??????????????????????????? >????????????????? >>?????? >??? Lin
??????????????????????????? >????????????? ????>>?????? >??? >On 2020/2/19, 9:40 AM, "linzang(??)" <linzang at tencent.com> wrote:.
??????????????????????????? >????????????????? >>?????? >??? >
??????????????????????????? >????????????????? >>?????? >??? >? Re-post this RFR with correct enhancement number to make it trackable.
??????????????????????????? >????????????????? >>?????? >??? >? please ignore the previous wrong post. sorry for troubles.
??????????????????????????? >????????????????? >>?????? >??? >
??????????????????????????? >???????????? ?????>>?????? >??? >?? webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/
??????????????????????????? >????????????????? >>?????? >??? >??? Hi bug: https://bugs.openjdk.java.net/browse/JDK-8215624
??????????????????????????? >?????? ???????????>>?????? >??? >??? CSR: https://bugs.openjdk.java.net/browse/JDK-8239290
??????????????????????????? >????????????????? >>?????? >??? >??? --------------
??????????????????????????? >????????????????? >>?????? >??? >??? Lin
???????????????????? ???????>????????????????? >>?????? >??? >??? >Hi Lin,
??????????????????????????? >????????????????? >?? >???? >??? >??? >
??????????????????????????? >????????????????? >>?????? >??? >??? >Could you, please, re-post your RFR with the right enhancement number in
??????????????????????????? >????????????????? >>?????? >??? >??? >the message subject?
??????????????????????????? >????????????????? >>?????? >??? >??? >It will be more trackable this way.
??????????????????????????? >????????????????? >>?????? >??? >??? >
??????????????????????????? >????????????????? >>?????? >??? >??? >Thanks,
????????? ??????????????????>????????????????? >>?????? >??? >??? >Serguei
??????????????????????????? >????????????????? >>?????? >??? >??? >
??????????????????????????? >????????????????? >>?????? >??? >??? >
??????????????????????????? >????????????????? >>????? ?>??? >??? >On 2/17/20 10:29 PM, linzang(??) wrote:
??????????????????????????? >????????????????? >>?????? >??? >??? >> Dear David,
??????????????????????????? >????????????????? >>?????? >??? >??? >>??????? Thanks a lot!
??????????????????????????? >??? ??????????????>>?????? >??? >??? >>?????? I have updated the refined code to http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_01/.
??????????????????????????? >????????????????? >>?????? >??? >??? >>??????? IMHO the parallel heap inspection can be extended to all kinds of heap as long as the heap layout can support parallel iteration.
??????????????????????????? >????????????????? >>?????? >??? >??? >>??????? Maybe we can firstly use this webrev to discuss how to implement it, because I am not sure my current implementation is an appropriate way to communicate with collectedHeap, then we can extend the solution to other kinds of heap.
??????????????????????????? >????????????????? >>?????? >??? >??? >>
??????????????????????????? >????????????? ????>>?????? >??? >??? >> Thanks,
??????????????????????????? >????????????????? >>?????? >??? >??? >> --------------
??????????????????????????? >????????????????? >>?????? >??? >??? >> Lin
??????????????????????????? >????????????????? >>?????? >??? >?? ?>>> Hi Lin,
??????????????????????????? >????????????????? >>?????? >??? >??? >>>
??????????????????????????? >????????????????? >>?????? >??? >??? >>> Adding in hotspot-gc-dev as they need to see how this interacts with GC
??????????????????????????? >? ????????????????>>?????? >??? >??? >>> worker threads, and whether it needs to be extended beyond G1.
??????????????????????????? >????????????????? >>?????? >??? >??? >>>
??????????????????????????? >????????????????? >>?????? >??? >?? >>> I happened to spot one nit when browsing:
??????????????????????????? >????????????????? >>?????? >??? >??? >>>
??????????????????????????? >????????????????? >>?????? >??? >??? >>> src/hotspot/share/gc/shared/collectedHeap.hpp
??????????????????????????? >????????????? ????>>?????? >??? >??? >>>
??????????????????????????? >????????????????? >>?????? >??? >??? >>> +?? virtual bool run_par_heap_inspect_task(KlassInfoTable* cit,
??????????????????????????? >????????????????? >>?????? >??? >??? >>> +??????????????????????? ??????????????????BoolObjectClosure* filter,
??????????????????????????? >????????????????? >>?????? >??? >??? >>> +????????????????????????????????????????? size_t* missed_count,
??????????????????????????? >????????????????? >>?????? >??? >??? >>> +???? ?????????????????????????????????????size_t thread_num) {
??????????????????????????? >????????????????? >>?????? >??? >??? >>> +???? return NULL;
??????????????????????????? >????????????????? >>?????? >??? >??? >>>
??????????????????????????? >????????????????? >>?????? >??? >??? >>> s/NULL/false/
??????????????????????????? >????????????????? >>?????? >??? >??? >>>
??????????????????????????? >?? ???????????????>>?????? >??? >??? >>> Cheers,
??????????????????????????? >????????????????? >>?????? >??? >??? >>> David
??????????????????????????? >????????????????? >?? >???? >??? >??? >>>
??????????????????????????? >????????????????? >>?????? >??? > ???>>> On 18/02/2020 2:15 pm, linzang(??) wrote:
??????????????????????????? >????????????????? >>?????? >??? >??? >>>> Dear All,
??????????????????????????? >????????????????? >>?????? >??? >??? >>>>???????? May I ask your help to review the follow changes:
??????????????????????????? >????????????????? >>?????? >??? >??? >>>>???????? webrev:
??????????????????????????? >????????????????? >>?????? >??? >??? >>>> http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_00/
??????????????????????????? >????????????????? >>?????? >??? >??? >>>>????? bug: https://bugs.openjdk.java.net/browse/JDK-8215624
??????????????????????????? >????????????????? >>?????? >??? >??? >>>>????? related CSR: https://bugs.openjdk.java.net/browse/JDK-8239290
???????????????? ???????????>????????????????? >>?????? >??? >??? >>>>???????? This patch enable parallel heap inspection of G1 for jmap histo.
??????????????????????????? >????????????????? >>?????? >??? >??? >>>>???????? my simple test shown it can speed up 2x of jmap -histo with
??????????????????????????? >????????????????? >>?????? >??? >??? >>>> parallelThreadNum set to 2 for heap at ~500M on 4-core platform.
??????????????????????????? >????????????????? >>?????? >??? >??? >>>>
??????????????????????????? >????????? ????????>>?????? >??? >??? >>>> ------------------------------------------------------------------------
??????????????????????????? >????????????????? >>?????? >??? >??? >>>> BRs,
??????????????????????????? >????????????????? >>?????? >??? >??? >>>> Lin
??????????????????????????? >????????????????? >>?????? >??? >??? >> >
??????????????????????????? >????????????????? >>?????? >??? >??? >
??????????????????????????? >????????????????? >
??????????????????????????? >????????????????? >
?????????????????? ?????????>????????????????? >
??????????????????????????? >
??????????????????????????? >
??????????????????????????? >
??????????????????????????? >
??????????????????????????? >
??????????????????????????? >
?
?
?
?
?
?
?


From Divino.Cesar at microsoft.com  Thu Jul 23 04:13:54 2020
From: Divino.Cesar at microsoft.com (Cesar Soares Lucas)
Date: Thu, 23 Jul 2020 04:13:54 +0000
Subject: RFR: 8249225 - Move definition of PADDING_ELEM_NUM
In-Reply-To: <SN6PR2101MB089344B2E6079D7E207EA5A19A780@SN6PR2101MB0893.namprd21.prod.outlook.com>
References: <SN6PR2101MB089344B2E6079D7E207EA5A19A780@SN6PR2101MB0893.namprd21.prod.outlook.com>
Message-ID: <SN6PR2101MB0893314FBF674DC4BD9E0AE29A760@SN6PR2101MB0893.namprd21.prod.outlook.com>

Can someone please take a look? It's a simple change. It's my first contribution but I hope it'll be just one of many!


Cheers,
Cesar

-----Original Message-----
From: hotspot-gc-dev <hotspot-gc-dev-retn at openjdk.java.net> On Behalf Of Cesar Soares Lucas
Sent: Monday, July 20, 2020 6:41 PM
To: hotspot-gc-dev at openjdk.java.net
Cc: Aditya Mandaleeka <adityam at microsoft.com>; Brian Stafford <Brian.Stafford at microsoft.com>
Subject: RFR: 8249225 - Move definition of PADDING_ELEM_NUM

Bug: https://bugs.openjdk.java.net/browse/JDK-8249225
Webrev: http://cr.openjdk.java.net/~adityam/cesar/8249225/

Please review the above linked patch for the mentioned bug. It's a small change
that moves the definition of "PADDING_ELEM_NUM" to the only file that uses it.
I also took the chance of converting it from a "#define" directive into a "const"
declaration - I can revert it back to a #define if you think it's better.


Thank you,
Cesar





From albert.m.yang at oracle.com  Thu Jul 23 09:05:33 2020
From: albert.m.yang at oracle.com (Albert Yang)
Date: Thu, 23 Jul 2020 02:05:33 -0700 (PDT)
Subject: RFR (S): 8242036: G1 HeapRegionRemSet::_n_coarse_entries could be
 a bool
Message-ID: <0988550e-b96d-4cb1-9b54-339485b40908@default>

Additionally, in following the guideline for concurrent variables, it is marked `volatile` to signal potential concurrent access, and all access to it use explicit atomic operations now.

https://bugs.openjdk.java.net/browse/JDK-8242036

the complete patch:

http://cr.openjdk.java.net/~lkorinth/albert/8242036/1/

Tested: hotspot-gc

/Albert

----- Original Message -----
From: albert.m.yang at oracle.com
To: hotspot-gc-dev at openjdk.java.net
Sent: Wednesday, July 22, 2020 11:51:57 AM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna
Subject: RFR (S): 8242036: G1 HeapRegionRemSet::_n_coarse_entries could be a bool

Hi,

Changed the type from `size_t` to `bool`, since we only care whether if it's zero or not.

https://bugs.openjdk.java.net/browse/JDK-8242036

http://cr.openjdk.java.net/~lkorinth/albert/8242036/0/

/Albert


From kim.barrett at oracle.com  Thu Jul 23 10:20:11 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 23 Jul 2020 06:20:11 -0400
Subject: RFR (S): 8242036: G1 HeapRegionRemSet::_n_coarse_entries could be
 a bool
In-Reply-To: <0988550e-b96d-4cb1-9b54-339485b40908@default>
References: <0988550e-b96d-4cb1-9b54-339485b40908@default>
Message-ID: <112240E3-AF92-4519-B200-93264227F392@oracle.com>

> On Jul 23, 2020, at 5:05 AM, Albert Yang <albert.m.yang at oracle.com> wrote:
> 
> Additionally, in following the guideline for concurrent variables, it is marked `volatile` to signal potential concurrent access, and all access to it use explicit atomic operations now.
> 
> https://bugs.openjdk.java.net/browse/JDK-8242036
> 
> the complete patch:
> 
> http://cr.openjdk.java.net/~lkorinth/albert/8242036/1/
> 
> Tested: hotspot-gc

Looks good.



From shade at redhat.com  Thu Jul 23 10:12:38 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Thu, 23 Jul 2020 12:12:38 +0200
Subject: RFR (S) 8249953: Shenandoah: gc/shenandoah/mxbeans tests should
 account for corner cases
Message-ID: <ed761347-e7b1-3711-4fb9-acac0332941d@redhat.com>

Testbug:
  https://bugs.openjdk.java.net/browse/JDK-8249953

Fix:
  https://cr.openjdk.java.net/~shade/8249953/webrev.01/

Testing: hotspot_gc_shenandoah {fastdebug,release}

-- 
Thanks,
-Aleksey


From thomas.schatzl at oracle.com  Thu Jul 23 10:24:52 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 23 Jul 2020 12:24:52 +0200
Subject: RFR: 8249225 - Move definition of PADDING_ELEM_NUM
In-Reply-To: <SN6PR2101MB089344B2E6079D7E207EA5A19A780@SN6PR2101MB0893.namprd21.prod.outlook.com>
References: <SN6PR2101MB089344B2E6079D7E207EA5A19A780@SN6PR2101MB0893.namprd21.prod.outlook.com>
Message-ID: <a98d00f7-f6a7-3fb4-378e-0d6a00a4ac7e@oracle.com>

Hi,

On 21.07.20 03:41, Cesar Soares Lucas wrote:
> Bug: https://bugs.openjdk.java.net/browse/JDK-8249225
> 
> Webrev: http://cr.openjdk.java.net/~adityam/cesar/8249225/
> 
> Please review the above linked patch for the mentioned bug. It's a small change
> that moves the definition of "PADDING_ELEM_NUM" to the only file that uses it.
> I also took the chance of converting it from a "#define" directive into a "const"
> declaration - I can revert it back to a #define if you think it's better.
> 
> 

   could the change please re-add the newline after array_length? It is 
easier to read if generation of the constants is slightly offset from 
the other code.

Looks good otherwise. I do not need a re-review for this change.

Do you need a sponsor?

Thanks,
   Thomas


From rkennke at redhat.com  Thu Jul 23 10:25:07 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Thu, 23 Jul 2020 12:25:07 +0200
Subject: RFR (S) 8249953: Shenandoah: gc/shenandoah/mxbeans tests should
 account for corner cases
In-Reply-To: <ed761347-e7b1-3711-4fb9-acac0332941d@redhat.com>
References: <ed761347-e7b1-3711-4fb9-acac0332941d@redhat.com>
Message-ID: <410d99ffb769fb65c8e42e8f173a2a89bf38b0e4.camel@redhat.com>

Ok.

Thanks,
Roman


On Thu, 2020-07-23 at 12:12 +0200, Aleksey Shipilev wrote:
> 	Error verifying signature: Cannot verify message signature:
> Incorrect message format
> Testbug:
>   https://bugs.openjdk.java.net/browse/JDK-8249953
> 
> Fix:
>   https://cr.openjdk.java.net/~shade/8249953/webrev.01/
> 
> Testing: hotspot_gc_shenandoah {fastdebug,release}
> 



From erik.osterlund at oracle.com  Thu Jul 23 10:44:15 2020
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Thu, 23 Jul 2020 12:44:15 +0200
Subject: RFR: 8249787: Make TestGCLocker more resilient with concurrent GCs
Message-ID: <3cc4a166-40de-5aa3-09dc-f2a34ab3c371@oracle.com>

Hi,

(I am sending this RFR on Roman's behalf. For some reason, he is currently
unable to send emails to hotspot-gc-dev. This is his email:)

TestGCLocker seems to be made with the assumption that GCs are
triggered on allocation failure. It has a somewhat complicated
machinery to generate some GC pressure and especially to free up some
memory in its artificial MemoryUser. This leads to some weird
interactions with Shenandoah control machinery.

For example, it frees memory when heap usage is >75% AND a certain time
has passed since it last freed memory (500ms). In those 500ms, it will
keep on allocating chunks of memory, eventually running OOM because it
keeps holding on to those chunks, while the GC is running like mad
trying to free up memory, but can't because the stupid up doesn't let
go. However, it will still keep resetting timeSinceLastGC because of
some tiny objects getting freed-up since last time (not enough to
prevent OOM though).

Bug:
https://bugs.openjdk.java.net/browse/JDK-8249787

Proposed solution is an option to pass-in minFreeCriticalWaitMS. This
way we can let Shenandoah pass-in 0 and disable that check, which seems
pointless anyway with a concurrent GC that might be triggered earlier
than on allocation-failure.

Webrev:
http://cr.openjdk.java.net/~rkennke/JDK-8249787/webrev.00/

Testing: TestGCLockerWithShenandoah with various settings,
hotspot_gc_shenandoah

What do you think? Does it make sense?

Thanks,
Roman


From erik.osterlund at oracle.com  Thu Jul 23 10:46:11 2020
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Thu, 23 Jul 2020 12:46:11 +0200
Subject: RFR (S): 8242036: G1 HeapRegionRemSet::_n_coarse_entries could be
 a bool
In-Reply-To: <0988550e-b96d-4cb1-9b54-339485b40908@default>
References: <0988550e-b96d-4cb1-9b54-339485b40908@default>
Message-ID: <80e2cf91-edbd-2a59-86ce-e8bf51805577@oracle.com>

Hi Albert,

Looks good.

Thanks,
/Erik

On 2020-07-23 11:05, Albert Yang wrote:
> Additionally, in following the guideline for concurrent variables, it is marked `volatile` to signal potential concurrent access, and all access to it use explicit atomic operations now.
>
> https://bugs.openjdk.java.net/browse/JDK-8242036
>
> the complete patch:
>
> http://cr.openjdk.java.net/~lkorinth/albert/8242036/1/
>
> Tested: hotspot-gc
>
> /Albert
>
> ----- Original Message -----
> From: albert.m.yang at oracle.com
> To: hotspot-gc-dev at openjdk.java.net
> Sent: Wednesday, July 22, 2020 11:51:57 AM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna
> Subject: RFR (S): 8242036: G1 HeapRegionRemSet::_n_coarse_entries could be a bool
>
> Hi,
>
> Changed the type from `size_t` to `bool`, since we only care whether if it's zero or not.
>
> https://bugs.openjdk.java.net/browse/JDK-8242036
>
> http://cr.openjdk.java.net/~lkorinth/albert/8242036/0/
>
> /Albert



From kim.barrett at oracle.com  Thu Jul 23 10:51:08 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Thu, 23 Jul 2020 06:51:08 -0400
Subject: RFR: 8249225 - Move definition of PADDING_ELEM_NUM
In-Reply-To: <a98d00f7-f6a7-3fb4-378e-0d6a00a4ac7e@oracle.com>
References: <SN6PR2101MB089344B2E6079D7E207EA5A19A780@SN6PR2101MB0893.namprd21.prod.outlook.com>
 <a98d00f7-f6a7-3fb4-378e-0d6a00a4ac7e@oracle.com>
Message-ID: <10DC3859-3837-4CFD-8DEA-04117FE9BBAD@oracle.com>

> On Jul 23, 2020, at 6:24 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> 
> Hi,
> 
> On 21.07.20 03:41, Cesar Soares Lucas wrote:
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8249225
>> Webrev: http://cr.openjdk.java.net/~adityam/cesar/8249225/
>> Please review the above linked patch for the mentioned bug. It's a small change
>> that moves the definition of "PADDING_ELEM_NUM" to the only file that uses it.
>> I also took the chance of converting it from a "#define" directive into a "const"
>> declaration - I can revert it back to a #define if you think it's better.
> 
>  could the change please re-add the newline after array_length? It is easier to read if generation of the constants is slightly offset from the other code.
> 
> Looks good otherwise. I do not need a re-review for this change.

Looks good to me too, with that re-added newline.

> Do you need a sponsor?
> 
> Thanks,
>  Thomas




From thomas.schatzl at oracle.com  Thu Jul 23 10:54:26 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 23 Jul 2020 12:54:26 +0200
Subject: RFR(M): 8248401: Refactor/unify RMI gc support functionality
Message-ID: <3897e4f5-0ee4-8ba9-e438-2c55c378162c@oracle.com>

Hi all,

   can I have reviews for this change that unifies RMI gc support across 
collectors.

Based on an idea that came up by Kim in recent reviews, instead of every 
collector implementing that support by itself, the change moves the 
details into CollectedHeap and the collectors only need to call an 
appropriate method.

That method has formerly called CollectedHeap::millis_since_last_gc() 
which has a confusing name. I chose "last_whole_heap_examined_time" 
instead which I believe is while long, more appropriate. I did not find 
a good shorter name, I am open for issues.

For testing I used implemented originally for JDK-8248401 (at 
http://cr.openjdk.java.net/~tschatzl/8248401/webrev.test/). It still is 
very much dependent on not having a too big delay between a GC and the 
call to the RMI method. I do not know how to make it more stable either, 
and fearing false positives I did not add it to this CR. I could be 
convinced to do so if you think we should take the risk or have a better 
idea on how it could be improved.

CR:
https://bugs.openjdk.java.net/browse/JDK-8248401
Webrev:
http://cr.openjdk.java.net/~tschatzl/8248401/webrev
Testing:
tier1-5; plus mentioned test

Thanks,
   Thomas


From erik.osterlund at oracle.com  Thu Jul 23 11:13:32 2020
From: erik.osterlund at oracle.com (=?UTF-8?Q?Erik_=c3=96sterlund?=)
Date: Thu, 23 Jul 2020 13:13:32 +0200
Subject: RFR: 8249787: Make TestGCLocker more resilient with concurrent GCs
In-Reply-To: <3cc4a166-40de-5aa3-09dc-f2a34ab3c371@oracle.com>
References: <3cc4a166-40de-5aa3-09dc-f2a34ab3c371@oracle.com>
Message-ID: <e0318fb1-5ea8-d5b3-d798-1e8c672edd83@oracle.com>

Hi Roman,

Looks good.

Thanks,
/Erik

On 2020-07-23 12:44, Erik ?sterlund wrote:
> Hi,
>
> (I am sending this RFR on Roman's behalf. For some reason, he is 
> currently
> unable to send emails to hotspot-gc-dev. This is his email:)
>
> TestGCLocker seems to be made with the assumption that GCs are
> triggered on allocation failure. It has a somewhat complicated
> machinery to generate some GC pressure and especially to free up some
> memory in its artificial MemoryUser. This leads to some weird
> interactions with Shenandoah control machinery.
>
> For example, it frees memory when heap usage is >75% AND a certain time
> has passed since it last freed memory (500ms). In those 500ms, it will
> keep on allocating chunks of memory, eventually running OOM because it
> keeps holding on to those chunks, while the GC is running like mad
> trying to free up memory, but can't because the stupid up doesn't let
> go. However, it will still keep resetting timeSinceLastGC because of
> some tiny objects getting freed-up since last time (not enough to
> prevent OOM though).
>
> Bug:
> https://bugs.openjdk.java.net/browse/JDK-8249787
>
> Proposed solution is an option to pass-in minFreeCriticalWaitMS. This
> way we can let Shenandoah pass-in 0 and disable that check, which seems
> pointless anyway with a concurrent GC that might be triggered earlier
> than on allocation-failure.
>
> Webrev:
> http://cr.openjdk.java.net/~rkennke/JDK-8249787/webrev.00/
>
> Testing: TestGCLockerWithShenandoah with various settings,
> hotspot_gc_shenandoah
>
> What do you think? Does it make sense?
>
> Thanks,
> Roman



From thomas.schatzl at oracle.com  Thu Jul 23 11:56:33 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 23 Jul 2020 13:56:33 +0200
Subject: RFR: 8249787: Make TestGCLocker more resilient with concurrent GCs
In-Reply-To: <e0318fb1-5ea8-d5b3-d798-1e8c672edd83@oracle.com>
References: <3cc4a166-40de-5aa3-09dc-f2a34ab3c371@oracle.com>
 <e0318fb1-5ea8-d5b3-d798-1e8c672edd83@oracle.com>
Message-ID: <44edd881-2855-a29a-436c-e8b88393a600@oracle.com>

hi,

On 23.07.20 13:13, Erik ?sterlund wrote:
> Hi Roman,
> 
> Looks good.

   +1 (cc'ed Roman)

Thomas


From albert.m.yang at oracle.com  Thu Jul 23 12:29:34 2020
From: albert.m.yang at oracle.com (Albert Yang)
Date: Thu, 23 Jul 2020 05:29:34 -0700 (PDT)
Subject: RFR (S): 8242036: G1 HeapRegionRemSet::_n_coarse_entries could be
 a bool
Message-ID: <780c52d1-5f8b-4fe5-a5ba-4dc939b98a07@default>

I misunderstood the original request from the internal discussion.

The variable is marked `volatile`, and access to it uses `Atomic::load/store`, or `Atomic::load_acquire/release_store` only when memory ordering is needed.

https://bugs.openjdk.java.net/browse/JDK-8242036

the complete patch:

http://cr.openjdk.java.net/~lkorinth/albert/8242036/2/

Tested: hotspot-gc

PS: My apologies for the inconvenience. 

/Albert

----- Original Message -----
From: albert.m.yang at oracle.com
To: hotspot-gc-dev at openjdk.java.net
Sent: Thursday, July 23, 2020 11:05:33 AM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna
Subject: Re: RFR (S): 8242036: G1 HeapRegionRemSet::_n_coarse_entries could be a bool

Additionally, in following the guideline for concurrent variables, it is marked `volatile` to signal potential concurrent access, and all access to it use explicit atomic operations now.

https://bugs.openjdk.java.net/browse/JDK-8242036

the complete patch:

http://cr.openjdk.java.net/~lkorinth/albert/8242036/1/

Tested: hotspot-gc

/Albert

----- Original Message -----
From: albert.m.yang at oracle.com
To: hotspot-gc-dev at openjdk.java.net
Sent: Wednesday, July 22, 2020 11:51:57 AM GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna
Subject: RFR (S): 8242036: G1 HeapRegionRemSet::_n_coarse_entries could be a bool

Hi,

Changed the type from `size_t` to `bool`, since we only care whether if it's zero or not.

https://bugs.openjdk.java.net/browse/JDK-8242036

http://cr.openjdk.java.net/~lkorinth/albert/8242036/0/

/Albert


From thomas.schatzl at oracle.com  Thu Jul 23 12:42:16 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Thu, 23 Jul 2020 14:42:16 +0200
Subject: RFR (S): 8242036: G1 HeapRegionRemSet::_n_coarse_entries could be
 a bool
In-Reply-To: <780c52d1-5f8b-4fe5-a5ba-4dc939b98a07@default>
References: <780c52d1-5f8b-4fe5-a5ba-4dc939b98a07@default>
Message-ID: <606f19ec-f9c6-9a67-0866-23b1e6848000@oracle.com>

Hi Albert,

On 23.07.20 14:29, Albert Yang wrote:
> I misunderstood the original request from the internal discussion.
> 
> The variable is marked `volatile`, and access to it uses `Atomic::load/store`, or `Atomic::load_acquire/release_store` only when memory ordering is needed.
> 
> https://bugs.openjdk.java.net/browse/JDK-8242036
> 
> the complete patch:
> 
> http://cr.openjdk.java.net/~lkorinth/albert/8242036/2/

heapRegionRemSet.cpp:

273     assert(_coarse_map.at(max_hrm_index) == false, "No coarse entries");

Never compare to false/true directly but use ! here.

276     // concurrent reader will ever see _has_coarse_entries == true

s/==/is  (or "is set" or some similar text)

Looks good otherwise. I do not need to see a re-review for these changes.

Thanks,
   Thomas


From leo.korinth at oracle.com  Thu Jul 23 13:14:57 2020
From: leo.korinth at oracle.com (Leo Korinth)
Date: Thu, 23 Jul 2020 15:14:57 +0200
Subject: RFR (S): 8242036: G1 HeapRegionRemSet::_n_coarse_entries could be
 a bool
In-Reply-To: <606f19ec-f9c6-9a67-0866-23b1e6848000@oracle.com>
References: <780c52d1-5f8b-4fe5-a5ba-4dc939b98a07@default>
 <606f19ec-f9c6-9a67-0866-23b1e6848000@oracle.com>
Message-ID: <5a216db0-6ed2-ee0e-1ee8-350e4bb8f78c@oracle.com>



On 23/07/2020 14:42, Thomas Schatzl wrote:
> Hi Albert,
> 
> On 23.07.20 14:29, Albert Yang wrote:
>> I misunderstood the original request from the internal discussion.
>>
>> The variable is marked `volatile`, and access to it uses `Atomic::load/store`, or `Atomic::load_acquire/release_store` only when memory ordering is needed.
>>
>> https://bugs.openjdk.java.net/browse/JDK-8242036
>>
>> the complete patch:
>>
>> http://cr.openjdk.java.net/~lkorinth/albert/8242036/2/
> 
> heapRegionRemSet.cpp:
> 
> 273???? assert(_coarse_map.at(max_hrm_index) == false, "No coarse entries");
> 
> Never compare to false/true directly but use ! here.
> 
> 276???? // concurrent reader will ever see _has_coarse_entries == true
> 
> s/==/is? (or "is set" or some similar text)
> 
> Looks good otherwise. I do not need to see a re-review for these changes.
> 
> Thanks,
>  ? Thomas

Looks good to me as well, thanks for fixing this, I will sponsor this for you.

Thanks,
Leo


From Divino.Cesar at microsoft.com  Thu Jul 23 21:39:05 2020
From: Divino.Cesar at microsoft.com (Cesar Soares Lucas)
Date: Thu, 23 Jul 2020 21:39:05 +0000
Subject: RFR: 8249225 - Move definition of PADDING_ELEM_NUM
In-Reply-To: <10DC3859-3837-4CFD-8DEA-04117FE9BBAD@oracle.com>
References: <SN6PR2101MB089344B2E6079D7E207EA5A19A780@SN6PR2101MB0893.namprd21.prod.outlook.com>
 <a98d00f7-f6a7-3fb4-378e-0d6a00a4ac7e@oracle.com>,
 <10DC3859-3837-4CFD-8DEA-04117FE9BBAD@oracle.com>
Message-ID: <SN6PR2101MB089353F30284A160E205A14C9A760@SN6PR2101MB0893.namprd21.prod.outlook.com>

Hi Thomas and Kim, thanks for reviewing!

Here is the updated Webrev: https://cr.openjdk.java.net/~adityam/cesar/8249225_1/

Yes, I do need someone to commit this for me.


Cheers,
Cesar


From: hotspot-gc-dev <hotspot-gc-dev-retn at openjdk.java.net> on behalf of Kim Barrett <kim.barrett at oracle.com>
Sent: July 23, 2020 3:51 AM
To: Thomas Schatzl <thomas.schatzl at oracle.com>
Cc: hotspot-gc-dev at openjdk.java.net <hotspot-gc-dev at openjdk.java.net>
Subject: Re: RFR: 8249225 - Move definition of PADDING_ELEM_NUM 
?
> On Jul 23, 2020, at 6:24 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> 
> Hi,
> 
> On 21.07.20 03:41, Cesar Soares Lucas wrote:
>> Bug: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.openjdk.java.net%2Fbrowse%2FJDK-8249225&amp;data=02%7C01%7Cdivino.cesar%40microsoft.com%7Cbd9e6881086d46ee824408d82ef68fd3%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637310983797971949&amp;sdata=DaaUE212nQkJID064Hz3GlsAsANb5f1u0KkCYslbZfw%3D&amp;reserved=0
>> Webrev: https://nam06.safelinks.protection.outlook.com/?url=http:%2F%2Fcr.openjdk.java.net%2F~adityam%2Fcesar%2F8249225%2F&amp;data=02%7C01%7Cdivino.cesar%40microsoft.com%7Cbd9e6881086d46ee824408d82ef68fd3%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637310983797981947&amp;sdata=uQ0uwQwW7lEpbO8FZQNq%2BA3ijoB%2FzHhKxX5qytj%2FrTI%3D&amp;reserved=0
>> Please review the above linked patch for the mentioned bug. It's a small change
>> that moves the definition of "PADDING_ELEM_NUM" to the only file that uses it.
>> I also took the chance of converting it from a "#define" directive into a "const"
>> declaration - I can revert it back to a #define if you think it's better.
> 
>? could the change please re-add the newline after array_length? It is easier to read if generation of the constants is slightly offset from the other code.
> 
> Looks good otherwise. I do not need a re-review for this change.

Looks good to me too, with that re-added newline.

> Do you need a sponsor?
> 
> Thanks,
>? Thomas



From ningsheng.jian at arm.com  Fri Jul 24 01:02:34 2020
From: ningsheng.jian at arm.com (Ningsheng Jian)
Date: Fri, 24 Jul 2020 09:02:34 +0800
Subject: RFR(S): 8246373 AArch64: Refactor register spilling code in ZGC
 barriers
In-Reply-To: <dd523ae7-c6bf-9beb-2069-870aec192028@oracle.com>
References: <cbd923a3-4d32-ee45-c8c5-455983f7a616@arm.com>
 <80ad8cb8-fe7e-dce7-d04f-2ff81cd9d548@redhat.com>
 <1c08cd73-ef1e-43b4-8cd2-cf4801ce5c08@oracle.com>
 <2ddf7b16-ffe2-e0f4-2d8f-3ad7bf2257d2@arm.com>
 <dd523ae7-c6bf-9beb-2069-870aec192028@oracle.com>
Message-ID: <13b76c61-446d-6042-aac9-95aab626bcfe@arm.com>

Hi Stuart,

Looks good to me and pushed.

Thanks,
Ningsheng

On 7/23/20 12:37 AM, Erik ?sterlund wrote:
> Hi Stuart,
> 
> Looks good.
> 
> Thanks,
> /Erik
> 
> On 2020-07-22 17:47, Stuart Monteith wrote:
>> Thanks Eric,
>> ?????? I've got a new webrev that'll hopefully be satisfactory:
>> http://cr.openjdk.java.net/~smonteith/8246373/webrev.1/
>>
>>
>> Thanks,
>> ?????? Stuart
>>
>> On 21/07/2020 08:59, Erik ?sterlund wrote:
>>> Looks good, given Andrew's suggestion. Don't need another webrev for 
>>> that.
>>>
>>> Thanks,
>>> /Erik
>>>
>>> On 2020-07-17 09:42, Andrew Haley wrote:
>>>> Hi,
>>>>
>>>> On 16/07/2020 20:29, Stuart Monteith wrote:
>>>>> Hello,
>>>>> ???? There is an opportunity to spill/load fewer registers in ZGC 
>>>>> barriers. This is targetting JDK 16 jdk/jdk.
>>>>>
>>>>> JIRA:
>>>>> https://bugs.openjdk.java.net/browse/JDK-8246373
>>>>> Webrev:
>>>>> http://cr.openjdk.java.net/~smonteith/8246373/webrev.0/
>>>>>
>>>>> Tested with JTReg, SPECjbb2015, and other workloads.
>>>> Looks reasonable, but please keep push_call_clobbered_registers(). If
>>>> you need to do this, add push_call_clobbered_registers_except(RegSet).
>>>> Of course the former can call the latter.
>>>>
>>>
>>
>> IMPORTANT NOTICE: The contents of this email and any attachments are 
>> confidential and may also be privileged. If you are not the intended 
>> recipient, please notify the sender immediately and do not disclose 
>> the contents to any other person, use it for any purpose, or store or 
>> copy the information in any medium. Thank you.
> 



From david.holmes at oracle.com  Fri Jul 24 04:41:28 2020
From: david.holmes at oracle.com (David Holmes)
Date: Fri, 24 Jul 2020 14:41:28 +1000
Subject: RFR: 8188055: (ref) Add Reference.refersTo predicate
In-Reply-To: <F353471D-8C42-4A8F-A63A-70CA20FAD98F@oracle.com>
References: <E125A0B6-7FC4-4007-A9C4-7A7DFB86EEE4@oracle.com>
 <8f0baa70-0bc3-caa7-13eb-364c45a466d6@oracle.com>
 <F353471D-8C42-4A8F-A63A-70CA20FAD98F@oracle.com>
Message-ID: <3e54b6b9-d237-bcf8-ddcb-f8821d9a0016@oracle.com>

Hi Kim,

On 22/07/2020 6:04 pm, Kim Barrett wrote:
>> On Apr 8, 2020, at 5:27 AM, David Holmes <david.holmes at oracle.com> wrote:
>>
>> Hi Kim,
> 
> Apparently I lost track of these comments and forgot to respond.

Thanks for the follow up - I figured it was all "on hold".

Cheers,
David

> I still won't be sending out a new webrev until some of the other
> discussion gets settled.  There's been some internal discussion, but
> I'm currently waiting on some other folks to have time to chime in.
> 
> Replies to your comments inline below.
> 
>> On 8/04/2020 10:25 am, Kim Barrett wrote:
>>> [Note review on both core-libs and hotspot-gc-dev lists; try not to lose
>>> either when replying.]
>>> Please review a new function: java.lang.ref.Reference.refersTo.
>>> This function is needed to test the referent of a Reference object
>>> without artificially extending the lifetime of the referent object, as
>>> may happen when calling Reference.get.  Some garbage collectors
>>> require extending the lifetime of a weak referent when accessed, in
>>> order to maintain collector invariants.  Lifetime extension may occur
>>> with any collector when the Reference is a SoftReference, as calling
>>> get indicates recent access.  This new function also allows testing
>>> the referent of a PhantomReference, which can't be accessed by calling
>>> get.
>>
>> This causes a slight conflict with the documentation for get() which states:
>>
>> "Because the referent of a phantom reference is always inaccessible ..."
>>
>> when the new method obviously accesses it.
> 
> I take that "inaccessible" to mean "inaccessible to the application",
> and refersTo doesn't make the referent accessible to the application.
> 
>>> The new function uses a native method whose implementation is in the
>>> VM so it can use the Access API.  It is the intent that this function
>>> will be intrinsified by optimizing compilers like C2 or graal, but
>>> that hasn't been implemented yet.  Bear that in mind before rushing
>>> off to change existing uses of Reference.get.
>>
>> I assume such intrinsification is intended for JDK 15 as well though, as end users may well rush to change their code!
> 
> Looks like we missed the JDK 15 train.
> 
>>> CR:
>>> https://bugs.openjdk.java.net/browse/JDK-8188055
>>> https://bugs.openjdk.java.net/browse/JDK-8241029 (CSR)
>>
>> In the specification:
>>
>> * @param   obj is the object to compare with the referenced object, or
>> *          {@code null}.
>>
>> First delete "is", the commone style for @param is just to say "the xxx" or "a yyy?.
> 
> Done locally.
> 
>> Second I suggest:
>>
>> s/the referenced object/this reference object's referent/
> 
> Done locally.
> 
>> In the apinote:
>>
>> * collection cycle.  {@link #refersTo(Object) refersTo} can be used to
>>
>> I suggest:
>>
>> * collection cycle.  The {@link #refersTo(Object) refersTo} method can be used to
>>
>> so we don't start a sentence with a lower-case code-font word.
> 
> Done locally.
> 
>> Also a query in the apinote:
>>
>> * This method returns a strong reference to the referent. This may cause
>> * the garbage collector to treat it as strongly reachable until some later
>>
>> Surely if the method returns a strong reference then the GC _will_ treat it as strongly reachable, not "may? ?
> 
> Something like refersTo is needed because an access using get may
> force some phases of some collectors to treat the referent as strongly
> reachable for some additional period, even if the application
> immediately drops all references to it.  Other collectors may not need
> to do anything of the sort.  And even collectors that do sometimes
> need to do so may not always need to do so.  It's all vaguely
> weasel-worded because there is so much potential variation.
> 
>>> Webrev:
>>> https://cr.openjdk.java.net/~kbarrett/8188055/open.04/
>>
>> Code changes look good. No comment on the test - I'll leave it to others to analyse that.
> 
> Thanks.
> 


From thomas.schatzl at oracle.com  Fri Jul 24 08:11:45 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 24 Jul 2020 10:11:45 +0200
Subject: RFR: 8249225 - Move definition of PADDING_ELEM_NUM
In-Reply-To: <SN6PR2101MB089353F30284A160E205A14C9A760@SN6PR2101MB0893.namprd21.prod.outlook.com>
References: <SN6PR2101MB089344B2E6079D7E207EA5A19A780@SN6PR2101MB0893.namprd21.prod.outlook.com>
 <a98d00f7-f6a7-3fb4-378e-0d6a00a4ac7e@oracle.com>
 <10DC3859-3837-4CFD-8DEA-04117FE9BBAD@oracle.com>
 <SN6PR2101MB089353F30284A160E205A14C9A760@SN6PR2101MB0893.namprd21.prod.outlook.com>
Message-ID: <249a177f-ad17-b0a3-84cd-db0f7a13de67@oracle.com>

Hi,

On 23.07.20 23:39, Cesar Soares Lucas wrote:
> Hi Thomas and Kim, thanks for reviewing!
> 
> Here is the updated Webrev: https://cr.openjdk.java.net/~adityam/cesar/8249225_1/
> 
> Yes, I do need someone to commit this for me.

   I'll sponsor it.

Thanks,
   Thomas


From kim.barrett at oracle.com  Fri Jul 24 08:23:25 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Fri, 24 Jul 2020 04:23:25 -0400
Subject: RFR (S): 8242036: G1 HeapRegionRemSet::_n_coarse_entries could be
 a bool
In-Reply-To: <780c52d1-5f8b-4fe5-a5ba-4dc939b98a07@default>
References: <780c52d1-5f8b-4fe5-a5ba-4dc939b98a07@default>
Message-ID: <58FFC48F-2617-453B-A69B-23E816540692@oracle.com>

> On Jul 23, 2020, at 8:29 AM, Albert Yang <albert.m.yang at oracle.com> wrote:
> 
> I misunderstood the original request from the internal discussion.
> 
> The variable is marked `volatile`, and access to it uses `Atomic::load/store`, or `Atomic::load_acquire/release_store` only when memory ordering is needed.
> 
> https://bugs.openjdk.java.net/browse/JDK-8242036
> 
> the complete patch:
> 
> http://cr.openjdk.java.net/~lkorinth/albert/8242036/2/
> 
> Tested: hotspot-gc
> 
> PS: My apologies for the inconvenience. 
> 
> /Albert

src/hotspot/share/gc/g1/heapRegionRemSet.cpp  
267     if (!_coarse_map.at(max_hrm_index)) {
268       _coarse_map.at_put(max_hrm_index, true);
269     }

The old code conditionalized on !_course_map.at() to avoid
incrementing _n_coarse_entries if the entry was already set.  That's
no longer needed.  It seems like it might be clearer to just do the
at_put unconditionally.  If so, also change the comment on line 264 to
say something like "Ensure the corresponding coarse bit is set."

src/hotspot/share/gc/g1/heapRegionRemSet.cpp  
273     assert(_coarse_map.at(max_hrm_index) == false, "No coarse entries");

+1 to Thomas's suggestion to not compare to false.

Sorry I didn't notice either of these previously.



From thomas.schatzl at oracle.com  Fri Jul 24 10:29:55 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 24 Jul 2020 12:29:55 +0200
Subject: RFR (S?): 8249676: [REDO] G1 incorrectly limiting young gen size when
 using the reserve can result in repeated full gcs
Message-ID: <57f3433b-5e45-937c-ff89-9fea801445a6@oracle.com>

Hi all,

   I would like to ask for (re-)reviews to re-apply the changes for 
"JDK-8244603: G1 incorrectly limiting young gen size when using the 
reserve can result in repeated full gcs".

This has been backed out because it triggered the unrelated issue 
JDK-8249192: MonitorInfo stores raw oops across safepoints.

There are no changes to the webrev from the original.

CR:
https://bugs.openjdk.java.net/browse/JDK-8249676
Webrev:
http://cr.openjdk.java.net/~tschatzl/8249676/webrev/
Testing:
hs-tier1-8 with JDK-8249192

Original change:
http://cr.openjdk.java.net/~tschatzl/8244603/webrev.1/

Thanks,
   Thomas


From thomas.schatzl at oracle.com  Fri Jul 24 10:41:09 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Fri, 24 Jul 2020 12:41:09 +0200
Subject: RFR (M): 8245721: Refactor the TaskTerminator
In-Reply-To: <931140A9-3FCF-41B4-905B-C53BAE4B8E40@oracle.com>
References: <bd530d10-a4bb-969f-b142-39ff259c91b8@oracle.com>
 <931140A9-3FCF-41B4-905B-C53BAE4B8E40@oracle.com>
Message-ID: <d1cfdd15-1d56-a17b-297e-bd5c1c6a5df8@oracle.com>

Hi Kim and Leo,

   thanks for your reviews.

On 08.07.20 17:31, Kim Barrett wrote:
>> On Jun 24, 2020, at 4:03 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
>>
>> CR:
>> https://bugs.openjdk.java.net/browse/JDK-8245721
>> Webrev:
>> http://cr.openjdk.java.net/~tschatzl/8245721/webrev/
>> Testing:
>> tier1-5, many many perf rounds, many tier1-X rounds with other patches
>>
>> Thanks,
>>   Thomas
> 
> ------------------------------------------------------------------------------
> src/hotspot/share/gc/shared/taskTerminator.hpp
>    54   struct SpinContext {
>    55     uint yield_count;
> ...
> 
> While there's certainly variability on this, discussions I've had with
> others and other reviews suggest the Style Guide's suggestion to use
> leading underscores for data member names should apply even for public
> members of structs.
> 

Fixed.

> ------------------------------------------------------------------------------
> src/hotspot/share/gc/shared/taskTerminator.hpp
>   128       spin_context.hard_spin_limit = MIN2(2 * spin_context.hard_spin_limit,
>   129                                           (uint) WorkStealingHardSpins);
> 
> In the new code, hard_spin_limit is always (re)initialized to
> WorkStealingHardSpins, so this does nothing.
> 
> In the old code, the starting spin limit was
>    WorkStealingHardSpins >> WorkStealingSpinToYieldRatio
> and that value was captured in a variable for (hard_spin_start) for
> use in reinitializing the limit after each yield iteration.
> 
> So this is a change, and I'm guessing not an intentional one.

Fixed.

> 
> ------------------------------------------------------------------------------
> src/hotspot/share/gc/shared/taskTerminator.cpp
>   154   SpinContext spin_context;
> 
> If this were moved inside the outer loop of offer_termination, there
> would be no need for SpinContext::reset; just use the ctor-initializer
> for initialization of the members.
> 

Fixed.

> ------------------------------------------------------------------------------
> src/hotspot/share/gc/shared/taskTerminator.cpp
> 
> I *think* the inner loop of offer_termination would be made clearer by
> hoisting the yield_count check out of do_spin_iteration.  I'm thinking
> something like
> 
> instead of
> 
>   169       bool giveup_spin;
>   170       do {
>   171         size_t tasks;
>   172         bool should_exit_termination;
>   173         {
>   174           MutexUnlocker y(_blocker, Mutex::_no_safepoint_check_flag);
>   175           giveup_spin = do_spin_iteration(spin_context);
>   176
>   177           // Dirty read of exit condition.
>   178           tasks = tasks_in_queue_set();
>   179           should_exit_termination = exit_termination(tasks, terminator);
>   180         }
>   181
>   182         // Immediately check exit conditions after re-acquiring the lock using the
>   183         // information gathered just recently.
>   184         if (_offered_termination == _n_threads) {
>   185           prepare_for_return(the_thread);
>   186           assert_queue_set_empty();
>   187           return true;
>   188         } else if (should_exit_termination) {
>   189           prepare_for_return(the_thread, tasks);
>   190           _offered_termination--;
>   191           return false;
>   192         }
>   193       } while (!giveup_spin);
> 
> something like
> 
>    while (spin_context.yield_count <= WorkStealingYieldsBeforeSleep) {
>      // Each spin iteration is counted as a yield for purposes of
>      // deciding when to sleep.
>      ++spin_context.yield_count;
>      size_t tasks;
>      bool should_exit_termination;
>      {
>        MutexUnlocker y(_blocker ...);
>        do_spin_iteration(spin_context);
>        // Dirty read of exit condition.
>        tasks = tasks_in_queue_set();
>        should_exit_termination = exit_termination(tasks, terminator);
>      }
>      // Immediately check exit conditions after re-acquiring the lock.
>      if (_offered_termination == _n_threads) {
>        ... ;
>        return true;
>      } else if (should_exit_termination) {
>        ... ;
>      }
>    }
> 
> This also makes some issues with naming a little more apparent.
> do_spin_iteration might spin or might yield, yield_count is counting
> both spins and yields, and spin_context is about more than spinning.
> Maybe something a little more generic, using "delay" rather than
> "spin", like do_delay_step, delay_count, and delay_context?
> 
> [pre-existing]
> Side question: Should the yield_count test be <= or < ? It seems like
> WorkStealingYieldsBeforeSleep == 0 should disable yields and just go
> straight to the sleep case. And that's desirable for a uniprocessor,
> though why we'd be using a GC using this on a uniprocessor system is
> a question.  Exclusive test would also better match the doc string for
> the option.

Fixed.

> 
> ------------------------------------------------------------------------------
> src/hotspot/share/gc/shared/taskTerminator.cpp
>   177           // Dirty read of exit condition.
>   178           tasks = tasks_in_queue_set();
> 
>  From comment in the bug, that's intentionally done while not holding
> the lock, because doing otherwise makes the locked section too long.
> There should be a comment about that so nobody is tempted to move it
> (as I did in an earlier version of the above suggested move of the
> yield_count checking).
> 

Fixed.

> ------------------------------------------------------------------------------
> src/hotspot/share/gc/shared/taskTerminator.cpp
> 
> How bad would it be to not bother collecting the number of tasks and
> just have prepare_for_return unconditionally notify_all?

Fwiw, (really) unconditionally doing notify_all gives very very poor 
results, maybe because when work is completely done, all threads are 
hammering on it.

> 
> Alternatively, if that's bad and the cost of tasks_in_queue_set() is
> also a problem (which I can well believe), maybe the taskqueue
> facility can have some specialized helper function added for use here.
> All we really want to know in the current code is how many tasks are
> available up to the number of workers, or more than the number of
> workers and we don't need a precise answer.
>
> Such a change can be dealt with later.

I would like to defer that work (replacing with various degrees of using 
notify_all and improving gathering the number of tasks) for another change.

> Another idea would be to record the task count in the terminator,
> along with a last update epoch counter.  (Both are only touched under
> the lock, so no lock-free synchronization to worry about.)  Use the
> recorded count if it's epoch is more recent than the locally captured
> epoch from the most recent local use.
> 
> Of course, this is being called after some kind of delay (spin, yield,
> sleep), so it's not clear how important the performance is.
> 
> ------------------------------------------------------------------------------
> src/hotspot/share/gc/shared/taskTerminator.cpp
>   204     } else {
>   205       size_t tasks = tasks_in_queue_set();
>   206       if (exit_termination(tasks, terminator)) {
>   207         prepare_for_return(the_thread, tasks);
>   208         _offered_termination--;
>   209         return false;
>   210       }
>   211     }
> 
> We get here only after our wait on the monitor ended.
> 
> If we were woken up by some other thread (so wait returned true), I
> think we should skip the exit_termination test (just assume true) and
> pass 0 to prepare_for_return. (So no need for task counting in this
> case.) So insert something like this before the above
> 
>    } else if (!timedout) {
>      // Don't bother waking up more tasks.
>      prepare_for_return(the_thread, 0);
>      _offered_termination--;
>      return false;
>    } else {
>      ...
> 
> I think this also fixes a (pre-existing, I think) bug. Assume there
> are enough tasks for exiting being called for, but fewer than the
> number of threads waiting. Either the spin master or the first timeout
> will wake up some, but not all, of the waiting threads. Each of those
> threads (or the spin master) may wake up additional threads, making
> the the limit on wakeups based on number of tasks pointless, and also
> making the task counting of much more limited value; given that
> exit_termination just checks tasks > 0, we could instead just check
> whether the queue_set is empty.  So I think without this we just have
> a slower version of always waking up all waiting threads.
> 
> (There is also a small improvement to TaskQueueSuper::is_empty() that
> I didn't bother making because it didn't seem to matter.)

Done.

> 
> ------------------------------------------------------------------------------
> src/hotspot/share/gc/shared/taskTerminator.cpp
>   156   MutexLocker x(_blocker, Mutex::_no_safepoint_check_flag);
> ...
>   174           MutexUnlocker y(_blocker, Mutex::_no_safepoint_check_flag);
> ...
>   197     _blocker->wait_without_safepoint_check(WorkStealingSleepMillis);
> 
> I agree with Zhengyu's suggestion of using MonitorLocker and waiting
> on the locker, rather than using naked _blocker->wait_xxx.
> 

Fixed.

 From Leo:
On 09.07.20 14:38, Leo Korinth wrote:
 > I just have two small suggestions that have not been mentioned yet.
 >
 > - TaskTerminator::yield() should be used or removed.
 > - Maybe _blocker could be "inlined" in the parent object (not being a
 > pointer) and we would get rid of a delete as well as an assert and
 > also  do one allocation less.
 >

Fixed.

New webrev:
http://cr.openjdk.java.net/~tschatzl/8245721/webrev.1/ (full)
http://cr.openjdk.java.net/~tschatzl/8245721/webrev.0_to_1/ (diff)

Testing:
tier1-5, various benchmarks

Thomas


From albert.m.yang at oracle.com  Fri Jul 24 10:59:48 2020
From: albert.m.yang at oracle.com (Albert Yang)
Date: Fri, 24 Jul 2020 12:59:48 +0200
Subject: RFR (S): 8242036: G1 HeapRegionRemSet::_n_coarse_entries could be
 a bool
In-Reply-To: <58FFC48F-2617-453B-A69B-23E816540692@oracle.com>
References: <780c52d1-5f8b-4fe5-a5ba-4dc939b98a07@default>
 <58FFC48F-2617-453B-A69B-23E816540692@oracle.com>
Message-ID: <e54cbed4-5e22-3a49-36de-ecd5b6b8c4e1@oracle.com>

I don't think calling `at_put` unconditionally is clearer, since 
`_coarse_map.at_put` needs be after `_coarse_map.reinitialize`, which 
only happens when `_has_coarse_entries` is false.

Maybe I misunderstood what you meant.

On 2020-07-24 10:23, Kim Barrett wrote:
>> On Jul 23, 2020, at 8:29 AM, Albert Yang <albert.m.yang at oracle.com> wrote:
>>
>> I misunderstood the original request from the internal discussion.
>>
>> The variable is marked `volatile`, and access to it uses `Atomic::load/store`, or `Atomic::load_acquire/release_store` only when memory ordering is needed.
>>
>> https://bugs.openjdk.java.net/browse/JDK-8242036
>>
>> the complete patch:
>>
>> http://cr.openjdk.java.net/~lkorinth/albert/8242036/2/
>>
>> Tested: hotspot-gc
>>
>> PS: My apologies for the inconvenience.
>>
>> /Albert
> src/hotspot/share/gc/g1/heapRegionRemSet.cpp
> 267     if (!_coarse_map.at(max_hrm_index)) {
> 268       _coarse_map.at_put(max_hrm_index, true);
> 269     }
>
> The old code conditionalized on !_course_map.at() to avoid
> incrementing _n_coarse_entries if the entry was already set.  That's
> no longer needed.  It seems like it might be clearer to just do the
> at_put unconditionally.  If so, also change the comment on line 264 to
> say something like "Ensure the corresponding coarse bit is set."
>
> src/hotspot/share/gc/g1/heapRegionRemSet.cpp
> 273     assert(_coarse_map.at(max_hrm_index) == false, "No coarse entries");
>
> +1 to Thomas's suggestion to not compare to false.
>
> Sorry I didn't notice either of these previously.
>


From kim.barrett at oracle.com  Fri Jul 24 11:02:32 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Fri, 24 Jul 2020 07:02:32 -0400
Subject: RFR (S): 8242036: G1 HeapRegionRemSet::_n_coarse_entries could be
 a bool
In-Reply-To: <e54cbed4-5e22-3a49-36de-ecd5b6b8c4e1@oracle.com>
References: <780c52d1-5f8b-4fe5-a5ba-4dc939b98a07@default>
 <58FFC48F-2617-453B-A69B-23E816540692@oracle.com>
 <e54cbed4-5e22-3a49-36de-ecd5b6b8c4e1@oracle.com>
Message-ID: <5C4C4D95-F005-4504-8718-80E4ED56C1F4@oracle.com>

> On Jul 24, 2020, at 6:59 AM, Albert Yang <albert.m.yang at oracle.com> wrote:
> 
> I don't think calling `at_put` unconditionally is clearer, since `_coarse_map.at_put` needs be after `_coarse_map.reinitialize`, which only happens when `_has_coarse_entries` is false.
> 
> Maybe I misunderstood what you meant.

I meant drop the conditional on line 267 (see below):

> On 2020-07-24 10:23, Kim Barrett wrote:
>> src/hotspot/share/gc/g1/heapRegionRemSet.cpp
>> 267     if (!_coarse_map.at(max_hrm_index)) {
>> 268       _coarse_map.at_put(max_hrm_index, true);
>> 269     }
>> 
>> The old code conditionalized on !_course_map.at() to avoid
>> incrementing _n_coarse_entries if the entry was already set.  That's
>> no longer needed.  It seems like it might be clearer to just do the
>> at_put unconditionally.  If so, also change the comment on line 264 to
>> say something like "Ensure the corresponding coarse bit is set."



From albert.m.yang at oracle.com  Fri Jul 24 11:37:13 2020
From: albert.m.yang at oracle.com (Albert Yang)
Date: Fri, 24 Jul 2020 13:37:13 +0200
Subject: RFR (S): 8242036: G1 HeapRegionRemSet::_n_coarse_entries could be
 a bool
In-Reply-To: <5C4C4D95-F005-4504-8718-80E4ED56C1F4@oracle.com>
References: <780c52d1-5f8b-4fe5-a5ba-4dc939b98a07@default>
 <58FFC48F-2617-453B-A69B-23E816540692@oracle.com>
 <e54cbed4-5e22-3a49-36de-ecd5b6b8c4e1@oracle.com>
 <5C4C4D95-F005-4504-8718-80E4ED56C1F4@oracle.com>
Message-ID: <dcd62cc8-50fd-d7f8-cc83-3359a9435165@oracle.com>

I see.

Updated:

http://cr.openjdk.java.net/~lkorinth/albert/8242036/3/

I am not sure if this unconditional write will pollute the cacheline 
though, since this variable is accessed concurrently.

On 2020-07-24 13:02, Kim Barrett wrote:
>> On Jul 24, 2020, at 6:59 AM, Albert Yang <albert.m.yang at oracle.com> wrote:
>>
>> I don't think calling `at_put` unconditionally is clearer, since `_coarse_map.at_put` needs be after `_coarse_map.reinitialize`, which only happens when `_has_coarse_entries` is false.
>>
>> Maybe I misunderstood what you meant.
> I meant drop the conditional on line 267 (see below):
>
>> On 2020-07-24 10:23, Kim Barrett wrote:
>>> src/hotspot/share/gc/g1/heapRegionRemSet.cpp
>>> 267     if (!_coarse_map.at(max_hrm_index)) {
>>> 268       _coarse_map.at_put(max_hrm_index, true);
>>> 269     }
>>>
>>> The old code conditionalized on !_course_map.at() to avoid
>>> incrementing _n_coarse_entries if the entry was already set.  That's
>>> no longer needed.  It seems like it might be clearer to just do the
>>> at_put unconditionally.  If so, also change the comment on line 264 to
>>> say something like "Ensure the corresponding coarse bit is set."


From vladimir.kozlov at oracle.com  Fri Jul 24 16:02:23 2020
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Fri, 24 Jul 2020 09:02:23 -0700
Subject: RFR[XXS] 8248672: utilities: Introduce DEPRECATED macro for GCC
 and MSVC
In-Reply-To: <MWHPR21MB05113A054F2D8291BFAFB085B0770@MWHPR21MB0511.namprd21.prod.outlook.com>
References: <MWHPR21MB05113A054F2D8291BFAFB085B0770@MWHPR21MB0511.namprd21.prod.outlook.com>
Message-ID: <5e301790-8bfe-0ced-b5e2-8a9c76ae33de@oracle.com>

"And use it in the AArch64 sub system."

Can you explain more? Do you have RFE filed for DEPRECATED use?
It is small change which usually done together with usage. Why do this separately?

Thanks,
Vladimir K

On 7/24/20 8:53 AM, Ludovic Henry wrote:
> Hi,
> 
> Could I please have a review on the following. It simply adds the `DEPRECATED` macro to wrap `__attribute__ ((deprecated))` for GCC, and the equivalent for MSVC.
> 
> JBS: https://bugs.openjdk.java.net/browse/JDK-8248672
> Webrev: http://cr.openjdk.java.net/~burban/luhenry/8248672/webrev.00
> 
> Thank you,
> 
> --
> Ludovic
> 


From luhenry at microsoft.com  Fri Jul 24 16:41:52 2020
From: luhenry at microsoft.com (Ludovic Henry)
Date: Fri, 24 Jul 2020 16:41:52 +0000
Subject: RFR[XXS] 8248672: utilities: Introduce DEPRECATED macro for GCC
 and MSVC
In-Reply-To: <5e301790-8bfe-0ced-b5e2-8a9c76ae33de@oracle.com>
References: <MWHPR21MB05113A054F2D8291BFAFB085B0770@MWHPR21MB0511.namprd21.prod.outlook.com>,
 <5e301790-8bfe-0ced-b5e2-8a9c76ae33de@oracle.com>
Message-ID: <MWHPR21MB051146316FF56A63423DF514B0770@MWHPR21MB0511.namprd21.prod.outlook.com>

I'm not sure I understand your question. Are you asking whether I also replaced all uses of __attribute__((deprecated)) with DEPRECATED? If so, I did replace the only use of it [1] together with defining the macro.

Please let me know if I misunderstood your question.

Thank you.

--
Ludovic

[1] in src/hotspot/cpu/aarch64/macroAssembler_aarch64.hpp.

________________________________________
From: Vladimir Kozlov <vladimir.kozlov at oracle.com>
Sent: Friday, July 24, 2020 09:02
To: Ludovic Henry; hotspot-compiler-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net
Cc: openjdk-aarch64; hotspot-gc-dev at openjdk.java.net
Subject: Re: RFR[XXS] 8248672: utilities: Introduce DEPRECATED macro for GCC and MSVC

"And use it in the AArch64 sub system."

Can you explain more? Do you have RFE filed for DEPRECATED use?
It is small change which usually done together with usage. Why do this separately?

Thanks,
Vladimir K

On 7/24/20 8:53 AM, Ludovic Henry wrote:
> Hi,
>
> Could I please have a review on the following. It simply adds the `DEPRECATED` macro to wrap `__attribute__ ((deprecated))` for GCC, and the equivalent for MSVC.
>
> JBS: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.openjdk.java.net%2Fbrowse%2FJDK-8248672&amp;data=02%7C01%7Cluhenry%40microsoft.com%7Cf8e2909451c44ee9c9b108d82feafcd6%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637312033704245529&amp;sdata=p%2Bec5f3YYaazBblPt9vRWjQ2ZWa209lHGPLlsuMbpk8%3D&amp;reserved=0
> Webrev: https://nam06.safelinks.protection.outlook.com/?url=http:%2F%2Fcr.openjdk.java.net%2F~burban%2Fluhenry%2F8248672%2Fwebrev.00&amp;data=02%7C01%7Cluhenry%40microsoft.com%7Cf8e2909451c44ee9c9b108d82feafcd6%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637312033704245529&amp;sdata=aOC6Xq%2BJExcKQ8oeNew4aZMHZUD2idlTi3tRihRwpjs%3D&amp;reserved=0
>
> Thank you,
>
> --
> Ludovic
>


From vladimir.kozlov at oracle.com  Fri Jul 24 17:26:31 2020
From: vladimir.kozlov at oracle.com (Vladimir Kozlov)
Date: Fri, 24 Jul 2020 10:26:31 -0700
Subject: RFR[XXS] 8248672: utilities: Introduce DEPRECATED macro for GCC
 and MSVC
In-Reply-To: <MWHPR21MB051146316FF56A63423DF514B0770@MWHPR21MB0511.namprd21.prod.outlook.com>
References: <MWHPR21MB05113A054F2D8291BFAFB085B0770@MWHPR21MB0511.namprd21.prod.outlook.com>
 <5e301790-8bfe-0ced-b5e2-8a9c76ae33de@oracle.com>
 <MWHPR21MB051146316FF56A63423DF514B0770@MWHPR21MB0511.namprd21.prod.outlook.com>
Message-ID: <ded2765f-00e2-f875-4d99-cf0e97ad19f0@oracle.com>

It was my mistake - I missed that it is indeed used in macroAssembler_aarch64.hpp

And thank you, Monica, for RFE description change - it is more clear now.

Change is fine. You need someone from aarch64 to review this too to make sure it works with their GCC.

Regards,
Vladimir K

On 7/24/20 9:41 AM, Ludovic Henry wrote:
> I'm not sure I understand your question. Are you asking whether I also replaced all uses of __attribute__((deprecated)) with DEPRECATED? If so, I did replace the only use of it [1] together with defining the macro.
> 
> Please let me know if I misunderstood your question.
> 
> Thank you.
> 
> --
> Ludovic
> 
> [1] in src/hotspot/cpu/aarch64/macroAssembler_aarch64.hpp.
> 
> ________________________________________
> From: Vladimir Kozlov <vladimir.kozlov at oracle.com>
> Sent: Friday, July 24, 2020 09:02
> To: Ludovic Henry; hotspot-compiler-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net
> Cc: openjdk-aarch64; hotspot-gc-dev at openjdk.java.net
> Subject: Re: RFR[XXS] 8248672: utilities: Introduce DEPRECATED macro for GCC and MSVC
> 
> "And use it in the AArch64 sub system."
> 
> Can you explain more? Do you have RFE filed for DEPRECATED use?
> It is small change which usually done together with usage. Why do this separately?
> 
> Thanks,
> Vladimir K
> 
> On 7/24/20 8:53 AM, Ludovic Henry wrote:
>> Hi,
>>
>> Could I please have a review on the following. It simply adds the `DEPRECATED` macro to wrap `__attribute__ ((deprecated))` for GCC, and the equivalent for MSVC.
>>
>> JBS: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.openjdk.java.net%2Fbrowse%2FJDK-8248672&amp;data=02%7C01%7Cluhenry%40microsoft.com%7Cf8e2909451c44ee9c9b108d82feafcd6%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637312033704245529&amp;sdata=p%2Bec5f3YYaazBblPt9vRWjQ2ZWa209lHGPLlsuMbpk8%3D&amp;reserved=0
>> Webrev: https://nam06.safelinks.protection.outlook.com/?url=http:%2F%2Fcr.openjdk.java.net%2F~burban%2Fluhenry%2F8248672%2Fwebrev.00&amp;data=02%7C01%7Cluhenry%40microsoft.com%7Cf8e2909451c44ee9c9b108d82feafcd6%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637312033704245529&amp;sdata=aOC6Xq%2BJExcKQ8oeNew4aZMHZUD2idlTi3tRihRwpjs%3D&amp;reserved=0
>>
>> Thank you,
>>
>> --
>> Ludovic
>>


From kim.barrett at oracle.com  Fri Jul 24 23:34:20 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Fri, 24 Jul 2020 19:34:20 -0400
Subject: RFR (S): 8242036: G1 HeapRegionRemSet::_n_coarse_entries could be
 a bool
In-Reply-To: <dcd62cc8-50fd-d7f8-cc83-3359a9435165@oracle.com>
References: <780c52d1-5f8b-4fe5-a5ba-4dc939b98a07@default>
 <58FFC48F-2617-453B-A69B-23E816540692@oracle.com>
 <e54cbed4-5e22-3a49-36de-ecd5b6b8c4e1@oracle.com>
 <5C4C4D95-F005-4504-8718-80E4ED56C1F4@oracle.com>
 <dcd62cc8-50fd-d7f8-cc83-3359a9435165@oracle.com>
Message-ID: <33667E76-2717-4DF5-9283-33660F986716@oracle.com>

> On Jul 24, 2020, at 7:37 AM, Albert Yang <albert.m.yang at oracle.com> wrote:
> 
> I see.
> 
> Updated:
> 
> http://cr.openjdk.java.net/~lkorinth/albert/8242036/3/
> 
> I am not sure if this unconditional write will pollute the cacheline though, since this variable is accessed concurrently.

This code is on a rare path and holding the lock.

Change looks good.



From kim.barrett at oracle.com  Fri Jul 24 23:42:43 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Fri, 24 Jul 2020 19:42:43 -0400
Subject: RFR[XXS] 8248672: utilities: Introduce DEPRECATED macro for GCC
 and MSVC
In-Reply-To: <MWHPR21MB051146316FF56A63423DF514B0770@MWHPR21MB0511.namprd21.prod.outlook.com>
References: <MWHPR21MB05113A054F2D8291BFAFB085B0770@MWHPR21MB0511.namprd21.prod.outlook.com>
 <5e301790-8bfe-0ced-b5e2-8a9c76ae33de@oracle.com>
 <MWHPR21MB051146316FF56A63423DF514B0770@MWHPR21MB0511.namprd21.prod.outlook.com>
Message-ID: <F50C37BA-FE47-465C-A60C-10547DCF8F69@oracle.com>

> On Jul 24, 2020, at 12:41 PM, Ludovic Henry <luhenry at microsoft.com> wrote:
> 
> I'm not sure I understand your question. Are you asking whether I also replaced all uses of __attribute__((deprecated)) with DEPRECATED? If so, I did replace the only use of it [1] together with defining the macro.
> 
> Please let me know if I misunderstood your question.
> 
> Thank you.
> 
> --
> Ludovic
> 
> [1] in src/hotspot/cpu/aarch64/macroAssembler_aarch64.hpp.

Why are we deprecating something rather than just deleting it and
fixing any users?

If the point is to keep the overload but prevent it from being called,
there are better ways than a deprecation warning.

And if we *really* needed deprecation warnings, I suggest using the
C++14 [[deprecated]] attribute (after adding it to the approved new
feature list).  But I think we shouldn't be doing this at all.



> 
> ________________________________________
> From: Vladimir Kozlov <vladimir.kozlov at oracle.com>
> Sent: Friday, July 24, 2020 09:02
> To: Ludovic Henry; hotspot-compiler-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net
> Cc: openjdk-aarch64; hotspot-gc-dev at openjdk.java.net
> Subject: Re: RFR[XXS] 8248672: utilities: Introduce DEPRECATED macro for GCC and MSVC
> 
> "And use it in the AArch64 sub system."
> 
> Can you explain more? Do you have RFE filed for DEPRECATED use?
> It is small change which usually done together with usage. Why do this separately?
> 
> Thanks,
> Vladimir K
> 
> On 7/24/20 8:53 AM, Ludovic Henry wrote:
>> Hi,
>> 
>> Could I please have a review on the following. It simply adds the `DEPRECATED` macro to wrap `__attribute__ ((deprecated))` for GCC, and the equivalent for MSVC.
>> 
>> JBS: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.openjdk.java.net%2Fbrowse%2FJDK-8248672&amp;data=02%7C01%7Cluhenry%40microsoft.com%7Cf8e2909451c44ee9c9b108d82feafcd6%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637312033704245529&amp;sdata=p%2Bec5f3YYaazBblPt9vRWjQ2ZWa209lHGPLlsuMbpk8%3D&amp;reserved=0
>> Webrev: https://nam06.safelinks.protection.outlook.com/?url=http:%2F%2Fcr.openjdk.java.net%2F~burban%2Fluhenry%2F8248672%2Fwebrev.00&amp;data=02%7C01%7Cluhenry%40microsoft.com%7Cf8e2909451c44ee9c9b108d82feafcd6%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637312033704245529&amp;sdata=aOC6Xq%2BJExcKQ8oeNew4aZMHZUD2idlTi3tRihRwpjs%3D&amp;reserved=0
>> 
>> Thank you,
>> 
>> --
>> Ludovic




From thomas.schatzl at oracle.com  Sat Jul 25 08:36:23 2020
From: thomas.schatzl at oracle.com (Thomas Schatzl)
Date: Sat, 25 Jul 2020 10:36:23 +0200
Subject: RFR (S): 8242036: G1 HeapRegionRemSet::_n_coarse_entries could be
 a bool
In-Reply-To: <33667E76-2717-4DF5-9283-33660F986716@oracle.com>
References: <780c52d1-5f8b-4fe5-a5ba-4dc939b98a07@default>
 <58FFC48F-2617-453B-A69B-23E816540692@oracle.com>
 <e54cbed4-5e22-3a49-36de-ecd5b6b8c4e1@oracle.com>
 <5C4C4D95-F005-4504-8718-80E4ED56C1F4@oracle.com>
 <dcd62cc8-50fd-d7f8-cc83-3359a9435165@oracle.com>
 <33667E76-2717-4DF5-9283-33660F986716@oracle.com>
Message-ID: <c57d9114-4002-ec40-74ed-ca60f8a75648@oracle.com>

Hi,

On 25.07.20 01:34, Kim Barrett wrote:
>> On Jul 24, 2020, at 7:37 AM, Albert Yang <albert.m.yang at oracle.com> wrote:
>>
>> I see.
>>
>> Updated:
>>
>> http://cr.openjdk.java.net/~lkorinth/albert/8242036/3/
>>
>> I am not sure if this unconditional write will pollute the cacheline though, since this variable is accessed concurrently.
> 
> This code is on a rare path and holding the lock.
> 
> Change looks good.
> 

+1.

Thomas


From zgu at redhat.com  Sat Jul 25 23:13:51 2020
From: zgu at redhat.com (Zhengyu Gu)
Date: Sat, 25 Jul 2020 19:13:51 -0400
Subject: [15] RFR 8250588: Shenandoah: LRB needs to save/restore fp registers
 for runtime call
Message-ID: <6610a96b-1bc8-16b3-ed61-cef5488b7f84@redhat.com>

We recently experienced some test failures with 
VarHandleTestMethodHandleAccessDouble.java and 
VarHandleTestMethodHandleAccessfloat.java, where appeared to have 
corrupted fp registers.

It turns out that we have to save fp registers in LRB when making 
runtime calls.


Bug: https://bugs.openjdk.java.net/browse/JDK-8250588
Weberev: http://cr.openjdk.java.net/~zgu/JDK-8250588/webrev.00/

Test:
   hotspot_gc_shenandoah
   tier1 with Shenandoah


Thanks,

-Zhengyu



From rkennke at redhat.com  Sun Jul 26 08:27:23 2020
From: rkennke at redhat.com (Roman Kennke)
Date: Sun, 26 Jul 2020 10:27:23 +0200
Subject: [15] RFR 8250588: Shenandoah: LRB needs to save/restore fp
 registers for runtime call
In-Reply-To: <6610a96b-1bc8-16b3-ed61-cef5488b7f84@redhat.com>
References: <6610a96b-1bc8-16b3-ed61-cef5488b7f84@redhat.com>
Message-ID: <CAAN-KyjXhxaHU1_Ng-_V4KhXLVXqkN9yfwnB7JeoXqJgsOpksQ@mail.gmail.com>

Hi Zhengyu,

the patch looks good to me!

Thank you,
Roman


Zhengyu Gu <zgu at redhat.com> schrieb am So., 26. Juli 2020, 01:15:

> We recently experienced some test failures with
> VarHandleTestMethodHandleAccessDouble.java and
> VarHandleTestMethodHandleAccessfloat.java, where appeared to have
> corrupted fp registers.
>
> It turns out that we have to save fp registers in LRB when making
> runtime calls.
>
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8250588
> Weberev: http://cr.openjdk.java.net/~zgu/JDK-8250588/webrev.00/
>
> Test:
>    hotspot_gc_shenandoah
>    tier1 with Shenandoah
>
>
> Thanks,
>
> -Zhengyu
>
>


From aph at redhat.com  Sun Jul 26 09:56:31 2020
From: aph at redhat.com (Andrew Haley)
Date: Sun, 26 Jul 2020 10:56:31 +0100
Subject: [aarch64-port-dev ] RFR[XXS] 8248672: utilities: Introduce
 DEPRECATED macro for GCC and MSVC
In-Reply-To: <F50C37BA-FE47-465C-A60C-10547DCF8F69@oracle.com>
References: <MWHPR21MB05113A054F2D8291BFAFB085B0770@MWHPR21MB0511.namprd21.prod.outlook.com>
 <5e301790-8bfe-0ced-b5e2-8a9c76ae33de@oracle.com>
 <MWHPR21MB051146316FF56A63423DF514B0770@MWHPR21MB0511.namprd21.prod.outlook.com>
 <F50C37BA-FE47-465C-A60C-10547DCF8F69@oracle.com>
Message-ID: <1259c3fd-b69c-6d81-0427-cb769f00bca5@redhat.com>

On 25/07/2020 00:42, Kim Barrett wrote:
> Why are we deprecating something rather than just deleting it and
> fixing any users?

C++ overloading. AArch64 CMP (immediate) only has a limited range, so
we only have a byte-wide Assembler::cmp() definition. The deprecation
warning on the wider version makes sure that any maintenance
programmer is immediately warned if it is used. There are other things
we could do: by not providing a definition for the wider cmp() you get
a link error, but that wouldn't be as explicit as a deprecation
warning.

The root problem is that the immediate value to CMP isn't always known
when HotSpot is compiled, but may be calculated at runtime. We have
seen failures in production when an immediate offset overflowed.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671



From kim.barrett at oracle.com  Sun Jul 26 21:41:47 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Sun, 26 Jul 2020 17:41:47 -0400
Subject: [aarch64-port-dev ] RFR[XXS] 8248672: utilities: Introduce
 DEPRECATED macro for GCC and MSVC
In-Reply-To: <1259c3fd-b69c-6d81-0427-cb769f00bca5@redhat.com>
References: <MWHPR21MB05113A054F2D8291BFAFB085B0770@MWHPR21MB0511.namprd21.prod.outlook.com>
 <5e301790-8bfe-0ced-b5e2-8a9c76ae33de@oracle.com>
 <MWHPR21MB051146316FF56A63423DF514B0770@MWHPR21MB0511.namprd21.prod.outlook.com>
 <F50C37BA-FE47-465C-A60C-10547DCF8F69@oracle.com>
 <1259c3fd-b69c-6d81-0427-cb769f00bca5@redhat.com>
Message-ID: <CD1AE64E-8555-4074-97EB-831BF0A72C61@oracle.com>

> On Jul 26, 2020, at 5:56 AM, Andrew Haley <aph at redhat.com> wrote:
> 
> On 25/07/2020 00:42, Kim Barrett wrote:
>> Why are we deprecating something rather than just deleting it and
>> fixing any users?
> 
> C++ overloading. AArch64 CMP (immediate) only has a limited range, so
> we only have a byte-wide Assembler::cmp() definition. The deprecation
> warning on the wider version makes sure that any maintenance
> programmer is immediately warned if it is used. There are other things
> we could do: by not providing a definition for the wider cmp() you get
> a link error, but that wouldn't be as explicit as a deprecation
> warning.
> 
> The root problem is that the immediate value to CMP isn't always known
> when HotSpot is compiled, but may be calculated at runtime. We have
> seen failures in production when an immediate offset overflowed.

Yeah, I'd guessed that might be the point, and confirmed it later by
looking at the changeset that originally introduced the attribute.

As of early last week, a definition of "= delete;" is the way to
poison an overload.



From luhenry at microsoft.com  Sun Jul 26 23:10:56 2020
From: luhenry at microsoft.com (Ludovic Henry)
Date: Sun, 26 Jul 2020 23:10:56 +0000
Subject: [aarch64-port-dev ] RFR[XXS] 8248672: utilities: Introduce
 DEPRECATED macro for GCC and MSVC
In-Reply-To: <CD1AE64E-8555-4074-97EB-831BF0A72C61@oracle.com>
References: <MWHPR21MB05113A054F2D8291BFAFB085B0770@MWHPR21MB0511.namprd21.prod.outlook.com>
 <5e301790-8bfe-0ced-b5e2-8a9c76ae33de@oracle.com>
 <MWHPR21MB051146316FF56A63423DF514B0770@MWHPR21MB0511.namprd21.prod.outlook.com>
 <F50C37BA-FE47-465C-A60C-10547DCF8F69@oracle.com>
 <1259c3fd-b69c-6d81-0427-cb769f00bca5@redhat.com>,
 <CD1AE64E-8555-4074-97EB-831BF0A72C61@oracle.com>
Message-ID: <MWHPR21MB05117357CE8A5903943234C2B0750@MWHPR21MB0511.namprd21.prod.outlook.com>

> As of early last week, a definition of "= delete;" is the way to
> poison an overload.

Let me try that locally, compile on Windows-AArch64 and Linux-AArch64, and confirm whether it works for MSVC.

________________________________________
From: Kim Barrett <kim.barrett at oracle.com>
Sent: Sunday, July 26, 2020 14:41
To: Andrew Haley
Cc: Ludovic Henry; Vladimir Kozlov; hotspot-compiler-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; openjdk-aarch64; hotspot-gc-dev at openjdk.java.net
Subject: Re: [aarch64-port-dev ] RFR[XXS] 8248672: utilities: Introduce DEPRECATED macro for GCC and MSVC

> On Jul 26, 2020, at 5:56 AM, Andrew Haley <aph at redhat.com> wrote:
>
> On 25/07/2020 00:42, Kim Barrett wrote:
>> Why are we deprecating something rather than just deleting it and
>> fixing any users?
>
> C++ overloading. AArch64 CMP (immediate) only has a limited range, so
> we only have a byte-wide Assembler::cmp() definition. The deprecation
> warning on the wider version makes sure that any maintenance
> programmer is immediately warned if it is used. There are other things
> we could do: by not providing a definition for the wider cmp() you get
> a link error, but that wouldn't be as explicit as a deprecation
> warning.
>
> The root problem is that the immediate value to CMP isn't always known
> when HotSpot is compiled, but may be calculated at runtime. We have
> seen failures in production when an immediate offset overflowed.

Yeah, I'd guessed that might be the point, and confirmed it later by
looking at the changeset that originally introduced the attribute.

As of early last week, a definition of "= delete;" is the way to
poison an overload.



From linzang at tencent.com  Mon Jul 27 03:26:16 2020
From: linzang at tencent.com (=?utf-8?B?bGluemFuZyjoh6fnkLMp?=)
Date: Mon, 27 Jul 2020 03:26:16 +0000
Subject: RFR(L): 8215624: add parallel heap inspection support for jmap
 histo(G1)(Internet mail)
In-Reply-To: <B8E2CD92-2E27-4EBD-9FE1-9F51FB627CE0@tencent.com>
References: <EF17E5D8-648B-46DA-82DE-0BF4CFDD42D6@amazon.com>
 <B8E2CD92-2E27-4EBD-9FE1-9F51FB627CE0@tencent.com>
Message-ID: <2C4A6B09-F281-433E-95CE-9587FC3C769F@tencent.com>

I update a new change at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_09
It includes a tiny fix of build failure on windows:
####################################
In attachListener.cpp:
-  uint parallel_thread_num = MAX(1, (uint)os::initial_active_processor_count() * 3 / 8);
+  uint parallel_thread_num = MAX2<uint>(1, (uint)os::initial_active_processor_count() * 3 / 8);
####################################
 
BRs,
Lin

?On 2020/7/23, 11:56 AM, "linzang(??)" <linzang at tencent.com> wrote:

    Hi Paul,
         Thanks for your help, that all looks good to me. 
         Just 2 minor changes:  
    	? delete the final return in ParHeapInspectTask::work, you mentioned it but seems not include in the webrev :-)
    	? delete a unnecessary blank line in heapInspect.cpp at merge_entry()

    #########################################################################
    --- old/src/hotspot/share/memory/heapInspection.cpp     2020-07-23 11:23:29.281666456 +0800
    +++ new/src/hotspot/share/memory/heapInspection.cpp     2020-07-23 11:23:29.017666447 +0800
    @@ -251,7 +251,6 @@
         _size_of_instances_in_words += cie->words();
         return true;
       }   
    -
       return false;
     }

    @@ -568,7 +567,6 @@
         Atomic::add(&_missed_count, missed_count);
       } else {
         Atomic::store(&_success, false);
    -   return;
       }   
     }
    #########################################################################


    Here is the webrev  http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_08/

    BRs,
    Lin
    ---------------------------------------------
    From: "Hohensee, Paul" <hohensee at amazon.com>
    Date: Thursday, July 23, 2020 at 6:48 AM
    To: "linzang(??)" <linzang at tencent.com>, Stefan Karlsson <stefan.karlsson at oracle.com>, "serguei.spitsyn at oracle.com" <serguei.spitsyn at oracle.com>, David Holmes <david.holmes at oracle.com>, serviceability-dev <serviceability-dev at openjdk.java.net>, "hotspot-gc-dev at openjdk.java.net" <hotspot-gc-dev at openjdk.java.net>
    Subject: RE: RFR(L): 8215624: add parallel heap inspection support for jmap histo(G1)(Internet mail)

    Just small things.

    heapInspection.cpp:

    In ParHeapInspectTask::work, remove the final return statement and fix the following ?}? indent. I.e., replace

    +    Atomic::store(&_success, false);
    +    return;
    +   }

    with

    +    Atomic::store(&_success, false);
    +  }

    In HeapInspection::heap_inspection, missed_count should be a uint to match other missed_count declarations, and should be initialized to the result of populate_table() rather than separately to 0.

    attachListener.cpp:

    In heap_inspection, initial_processor_count returns an int, so cast its result to a uint.

    Similarly, parse_uintx returns a uintx, so cast its result (num) to uint when assigning to parallel_thread_num.

    BasicJMapTest.java:

    I took the liberty of refactoring testHisto*/histoToFile/testDump*/dump to remove redundant interposition methods and make histoToFile and dump look as similar as possible.

    Webrev with the above changes in

    http://cr.openjdk.java.net/~phh/8214535/webrev.01/

    Thanks,
    Paul

    On 7/15/20, 2:13 AM, "linzang(??)" <linzang at tencent.com> wrote:

         Upload a new webrev at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07/
         It fix a potential issue that unexpected number of threads maybe calculated for "parallel" option of jmap -histo in container.
        As shown at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07-delta/src/hotspot/share/services/attachListener.cpp.udiff.html

        ############### attachListener.cpp ####################
        @@ -252,11 +252,11 @@
         static jint heap_inspection(AttachOperation* op, outputStream* out) {
           bool live_objects_only = true;   // default is true to retain the behavior before this change is made
           outputStream* os = out;   // if path not specified or path is NULL, use out
           fileStream* fs = NULL;
           const char* arg0 = op->arg(0);
        -  uint parallel_thread_num = MAX(1, os::processor_count() * 3 / 8); // default is less than half of processors.
        +  uint parallel_thread_num = MAX(1, os::initial_active_processor_count() * 3 / 8); // default is less than half of processors.
           if (arg0 != NULL && (strlen(arg0) > 0)) {
             if (strcmp(arg0, "-all") != 0 && strcmp(arg0, "-live") != 0) {
               out->print_cr("Invalid argument to inspectheap operation: %s", arg0);
               return JNI_ERR;
             }
        ###################################################

        Thanks.

        BRs,
       Lin

        On 2020/7/9, 3:22 PM, "linzang(??)" <linzang at tencent.com> wrote:

            Hi Paul,
                Thanks for reviewing!
                >>
                >>     I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp.
                >>
                The reason I made the change in Jmap.java that compose all arguments as 1 string , instead of passing 3 argments, is to avoid the compatibility issue, as we discussed in http://mail.openjdk.java.net/pipermail/serviceability-dev/2019-February/thread.html#27240.  The root cause of the compatibility issue is because max argument count in HotspotVirtualMachineImpl.java and attachlistener.cpp need to be enlarged (changes like http://hg.openjdk.java.net/jdk/jdk/rev/e7cf035682e3#l2.1) when jmap has more than 3 arguments. But if user use an old jcmd/jmap tool, it may stuck at socket read(), because the "max argument count" don't match.
                 I re-checked this change, the argument count of jmap histo is equal to 3 (live, file, parallel), so it can work normally even without the change of passing argument. But I think we have to face the problem if more arguments is added in jcmd alike tools later, not sure whether it should be sloved (or a workaround) in this changeset.

                And here are the lastest webrev and delta:
                http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06/
                http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06-delta/

            Cheers,
            Lin

            On 2020/7/7, 5:57 AM, "Hohensee, Paul" <hohensee at amazon.com> wrote:

                I'd like to see this feature added. :)

                The CSR looks good, as does the basic parallel inspection algorithm. Stefan's done the GC part, so I'll stick to the non-GC part (fwiw, the GC part lgtm).

                I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp.

                heapInspection.hpp:

                _shared_miss_count (s/b _missed_count, see below) isn't a size, so it should be a uint instead of a size_t. Same with the new parallel_thread_num argument to heap_inspection() and populate_table().

                Comment copy-edit:
                +// Parallel heap inspection task. Parallel inspection can fail due to
                +// a native OOM when allocating memory for TL-KlassInfoTable.
                +// _success will be set false on an OOM, and serial inspection tried.

                _shared_miss_count should be _missed_count to match the missed_count() getter, or rename missed_count() to be shared_miss_count(). Whichever way you go, the field type should match the getter result type: uint is reasonable.

                heapInspection.cpp:

                You might use ResourceMark twice in populate_table, separately for the parallel attempt and the serial code. If the parallel attempt fails and available memory is low, it would be good to clean up the memory used by the parallel attempt before doing the serial code.

                Style nit in KlassInfoTable::merge_entry(). I'd line up the definitions of k and elt, so "k" is even with "elt". And, because it's two lines shorter, I'd replace
                +  } else {
                +    return false;
                +  }
                with
                +  return false;

                KlassInfoTableMergeClosure.is_success() should be just success() (i.e., no "is_" prefix) because it's a getter.

                I'd reorganize the code in populate_table() to make it more clear, vis (I changed _shared_missed_count to _missed_count)
                +  if (cit.allocation_failed()) {
                +    // fail to allocate memory, stop parallel mode
                +    Atomic::store(&_success, false);
                +    return;
                +  }
                +  RecordInstanceClosure ric(&cit, _filter);
                +  _poi->object_iterate(&ric, worker_id);
                +  missed_count = ric.missed_count();
                +  {
                +    MutexLocker x(&_mutex);
                +    merge_success = _shared_cit->merge(&cit);
                +  }
                +  if (merge_success) {
                +    Atomic::add(&_missed_count, missed_count);
                +  else {
                +    Atomic::store(&_success, false);
                +  }

                Thanks,
                Paul

                On 6/29/20, 7:20 PM, "linzang(??)" <linzang at tencent.com> wrote:

                    Dear All,
                            Sorry to bother again, I just want to make sure that is this change worth to be continue to work on? If decision is made to not. I think I can drop this work and stop asking for help reviewing...
                            Thanks for all your help about reviewing this previously.

                    BRs,
                    Lin

                    On 2020/5/9, 3:47 PM, "linzang(??)" <linzang at tencent.com> wrote:

                        Dear All,
                               May I ask your help again for review the latest change?  Thanks!

                        BRs,
                        Lin

                        On 2020/4/28, 1:54 PM, "linzang(??)" <linzang at tencent.com> wrote:

                            Hi Stefan,
                              >>  - Adding Atomic::load/store.
                              >>  - Removing the time measurement in the run_task. I renamed G1's function
                              >>  to run_task_timed. If we need this outside of G1, we can rethink the API
                              >>  at that point.
                               >>  - ZGC style cleanups
                               Thanks for revising the patch,  they are all good to me, and I have made a tiny change based on it:
                                   http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04/
                                   http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04-delta/
                              it reduce the scope of mutex in ParHeapInspectTask, and delete unnecessary comments.

                            BRs,
                            Lin

                            On 2020/4/27, 4:34 PM, "Stefan Karlsson" <stefan.karlsson at oracle.com> wrote:

                                Hi Lin,

                                On 2020-04-26 05:10, linzang(??) wrote:
                                > Hi Stefan and Paul?
                                >      I have made a new patch based on your comments and Stefan's Poc code:
                                >      Webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03/
                                >      Delta(based on Stefan's change:) : http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03-delta/webrev_03-delta/

                                Thanks for providing a delta patch. It makes it much easier to look at,
                                and more likely for reviewers to continue reviewing.

                                I'm going to continue focusing on the GC parts, and leave the rest to
                                others to review.

                                >
                                >      And Here are main changed I made and want to discuss with you:
                                >      1.  changed"parallelThreadNum=" to "parallel=" for jmap -histo options.
                                >      2.  Add logic to test where parallelHeapInspection is fail, in heapInspection.cpp
                                >            This is because the parHeapInspectTask create thread local KlassInfoTable in it's work() method, and this may fail because of native OOM, in this case, the parallel should fail and serial heap inspection can be tried.
                                >            One more thing I want discuss with you is about the member "_success" of parHeapInspectTask, when native OOM happenes, it is set to false. And since this "set" operation can be conducted in multiple threads, should it be atomic ops?  IMO, this is not necessary because "_success" can only be set to false, and there is no way to change it from back to true after the ParHeapInspectTask instance is created, so it is save to be non-atomic, do you agree with that?

                                In these situations you should be using the Atomic::load/store
                                primitives. We're moving toward a later C++ standard were data races are
                               considered undefined behavior.

                                >     3. make CollectedHeap::run_task() be an abstract virtual func, so that every subclass of collectedHeap should support it, so later implementation of new collectedHeap will not miss the "parallel" features.
                                >           The problem I want to discuss with you is about epsilonHeap and SerialHeap, as they may not need parallel heap iteration, so I only make task->work(0), in case the run_task() is invoked someway in future. Another way is to left run_task()  unimplemented, which one do you think is better?

                                I don't have a strong opinion about this.

                                  And also please help take a look at the zHeap, as there is a class
                                zTask that wrap the abstractGangTask, and the collectedHeap::run_task()
                                only accept  AbstraceGangTask* as argument, so I made a delegate class
                                to adapt it , please see src/hotspot/share/gc/z/zHeap.cpp.
                                >
                                >        There maybe other better ways to sovle the above problems, welcome for any comments, Thanks!

                                I've created a few cleanups and changes on top of your latest patch:

                                https://cr.openjdk.java.net/~stefank/8215624/webrev.02.delta
                                https://cr.openjdk.java.net/~stefank/8215624/webrev.02

                                - Adding Atomic::load/store.
                                - Removing the time measurement in the run_task. I renamed G1's function
                                to run_task_timed. If we need this outside of G1, we can rethink the API
                                at that point.
                                - ZGC style cleanups

                                Thanks,
                                StefanK

                                >
                                > BRs,
                                > Lin
                                >
                                > On 2020/4/23, 11:08 AM, "linzang(??)" <linzang at tencent.com> wrote:
                                >
                                >      Thanks Paul! I agree with using "parallel", will make the update in next patch, Thanks for help update the CSR.
                                >
                                >      BRs,
                                >      Lin
                                >
                                >      On 2020/4/23, 4:42 AM, "Hohensee, Paul" <hohensee at amazon.com> wrote:
                                >
                                >          For the interface, I'd use "parallel" instead of "parallelThreadNum". All the other options are lower case, and it's a lot easier to type "parallel". I took the liberty of updating the CSR. If you're ok with it, you might want to change variable names and such, plus of course JMap.usage.
                                >
                                >          Thanks,
                                >          Paul
                                >
                                >          On 4/22/20, 2:29 AM, "serviceability-dev on behalf of linzang(??)" <serviceability-dev-bounces at openjdk.java.net on behalf of linzang at tencent.com> wrote:
                                >
                                >              Dear Stefan,
                                >
                                >                      Thanks a lot! I agree with you to decouple the heap inspection code with GC's.
                                >                      I will start  from your POC code, may discuss with you later.
                                >
                                >
                                >              BRs,
                                >              Lin
                                >
                                >              On 2020/4/22, 5:14 PM, "Stefan Karlsson" <stefan.karlsson at oracle.com> wrote:
                                >
                                >                  Hi Lin,
                                >
                                >                  I took a look at this earlier and saw that the heap inspection code is
                                >                  strongly coupled with the CollectedHeap and G1CollectedHeap. I'd prefer
                                >                  if we'd abstract this away, so that the GCs only provide a "parallel
                                >                  object iteration" interface, and the heap inspection code is kept elsewhere.
                                >
                                >                  I started experimenting with doing that, but other higher-priority (to
                                >                  me) tasks have had to take precedence.
                                >
                                >                  I've uploaded my work-in-progress / proof-of-concept:
                                >                    https://cr.openjdk.java.net/~stefank/8215624/webrev.01.delta/
                                >                    https://cr.openjdk.java.net/~stefank/8215624/webrev.01/
                                >
                                >                  The current code doesn't handle the lifecycle (deletion) of the
                                >                  ParallelObjectIterators. There's also code left unimplemented in around
                                >                  CollectedHeap::run_task. However, I think this could work as a basis to
                                >                  pull out the heap inspection code out of the GCs.
                                >
                                >                  Thanks,
                                >                  StefanK
                                >
                                >                  On 2020-04-22 02:21, linzang(??) wrote:
                                >                  > Dear all,
                                >                  >       May I ask you help to review? This RFR has been there for quite a while.
                                >                  >       Thanks!
                                >                  >
                                >                  > BRs,
                                >                  > Lin
                                >                  >
                                >                  > > On 2020/3/16, 5:18 PM, "linzang(??)" <linzang at tencent.com> wrote:>
                                >                  >
                                >                  >>    Just update a new path, my preliminary measure show about 3.5x speedup of jmap histo on a nearly full 4GB G1 heap (8-core platform with parallel thread number set to 4).
                                >                  >>     webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_02/
                                >                  >>     bug: https://bugs.openjdk.java.net/browse/JDK-8215624
                                >                  >>     CSR: https://bugs.openjdk.java.net/browse/JDK-8239290
                                >                  >>     BRs,
                                >                  >>       Lin
                                >                  >>       > On 2020/3/2, 9:56 PM, "linzang(??)" <linzang at tencent.com> wrote:
                                >                  >>       >
                                >                  >>       >    Dear all,
                                >                  >>       >          Let me try to ease the reviewing work by some explanation :P
                                >                  >>       >          The patch's target is to speed up jmap -histo for heap iteration, from my experience it is necessary for large heap investigation. E.g in bigData scenario I have tried to conduct jmap -histo against 180GB heap, it does take quite a while.
                                >                  >>       >          And if my understanding is corrent, even the jmap -histo without "live" option does heap inspection with heap lock acquired. so it is very likely to block mutator thread in allocation-sensitive scenario. I would say the faster the heap inspection does, the shorter the mutator be blocked. This is parallel iteration for jmap is necessary.
                                >                  >>       >          I think the parallel heap inspection should be applied to all kind of heap. However, consider the heap layout are different for  GCs, much time is required to understand all kinds of the heap layout to make the whole change. IMO, It is not wise to have a huge patch for the whole solution at once, and it is even harder to review it. So I plan to implement it incrementally, the first patch (this one) is going to confirm the implemention detail of how jmap accept the new option, passes it to attachListener of the jvm process and then how to make the parallel inspection closure be generic enough to make it easy to extend to different heap layout. And also how to implement the heap inspection in specific gc's heap. This patch use G1's heap as the begining.
                                >                  >>       >          This patch actually do several things:
                                >                  >>       >          1. Add an option "parallelThreadNum=<N>" to jmap -histo, the default behavior is to set N to 0, means let's JVM decide how many threads to use for heap inspection. Set this option to 1 will disable parallel heap inspection. (more details in CSR: https://bugs.openjdk.java.net/browse/JDK-8239290)
                                >                  >>       >          2. Make a change in how Jmap passing arguments, changes in http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java.udiff.html, originally it pass options as separate arguments to attachListener, this patch change to that all options be compose to a single string. So the arg_count_max in attachListener.hpp do not need to be changed, and hence avoid the compatibility issue, as disscussed at https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-March/027334.html
                                >                  >>       >         3. Add an abstract class ParHeapInspectTask in heapInspection.hpp / heapInspection.cpp, It's work(uint worker_id) method prepares the data structure (KlassInfoTable) need for every parallel worker thread, and then call do_object_iterate_parallel() which is heap specific implementation. I also added some machenism in KlassInfoTable to support parallel iteration, such as merge().
                                >                  >>       >        4. In specific heap (G1 in this patch), create a subclass of ParHeapInspectTask, implement the do_object_iterate_parallel() for parallel heap inspection. For G1, it simply invoke g1CollectedHeap's object_iterate_parallel().
                                >                  >>       >        5. Add related test.
                                >                  >>       >        6. it may be easy to extend this patch for other kinds of heap by creating subclass of ParHeapInspectTask and implement the do_object_iterate_parallel().
                                >                  >>       >
                                >                  >>       >    Hope these info could help on code review and initate the discussion :-)
                                >                  >>       >    Thanks!
                                >                  >>       >
                                >                  >>       >    BRs,
                                >                  >>       >    Lin
                                >                  >>       >    >On 2020/2/19, 9:40 AM, "linzang(??)" <linzang at tencent.com> wrote:.
                                >                  >>       >    >
                                >                  >>       >    >  Re-post this RFR with correct enhancement number to make it trackable.
                                >                  >>       >    >  please ignore the previous wrong post. sorry for troubles.
                                >                  >>       >    >
                                >                  >>       >    >   webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/
                                >                  >>       >    >    Hi bug: https://bugs.openjdk.java.net/browse/JDK-8215624
                                >                  >>       >    >    CSR: https://bugs.openjdk.java.net/browse/JDK-8239290
                                >                  >>       >    >    --------------
                                >                  >>       >    >    Lin
                                >                  >>       >    >    >Hi Lin,
                                >                  >   >     >    >    >
                                >                  >>       >    >    >Could you, please, re-post your RFR with the right enhancement number in
                                >                  >>       >    >    >the message subject?
                                >                  >>       >    >    >It will be more trackable this way.
                                >                  >>       >    >    >
                                >                  >>       >    >    >Thanks,
                                >                  >>       >    >    >Serguei
                                >                  >>       >    >    >
                                >                  >>       >    >    >
                                >                  >>       >    >    >On 2/17/20 10:29 PM, linzang(??) wrote:
                                >                  >>       >    >    >> Dear David,
                                >                  >>       >    >    >>        Thanks a lot!
                                >                  >>       >    >    >>       I have updated the refined code to http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_01/.
                                >                  >>       >    >    >>        IMHO the parallel heap inspection can be extended to all kinds of heap as long as the heap layout can support parallel iteration.
                                >                  >>       >    >    >>        Maybe we can firstly use this webrev to discuss how to implement it, because I am not sure my current implementation is an appropriate way to communicate with collectedHeap, then we can extend the solution to other kinds of heap.
                                >                  >>       >    >    >>
                                >                  >>       >    >    >> Thanks,
                                >                  >>       >    >    >> --------------
                                >                  >>       >    >    >> Lin
                                >                  >>       >    >    >>> Hi Lin,
                                >                  >>       >    >    >>>
                                >                  >>       >    >    >>> Adding in hotspot-gc-dev as they need to see how this interacts with GC
                                >                  >>       >    >    >>> worker threads, and whether it needs to be extended beyond G1.
                                >                  >>       >    >    >>>
                                >                  >>       >    >   >>> I happened to spot one nit when browsing:
                                >                  >>       >    >    >>>
                                >                  >>       >    >    >>> src/hotspot/share/gc/shared/collectedHeap.hpp
                                >                  >>       >    >    >>>
                                >                  >>       >    >    >>> +   virtual bool run_par_heap_inspect_task(KlassInfoTable* cit,
                                >                  >>       >    >    >>> +                                          BoolObjectClosure* filter,
                                >                  >>       >    >    >>> +                                          size_t* missed_count,
                                >                  >>       >    >    >>> +                                          size_t thread_num) {
                                >                  >>       >    >    >>> +     return NULL;
                                >                  >>       >    >    >>>
                                >                  >>       >    >    >>> s/NULL/false/
                                >                  >>       >    >    >>>
                                >                  >>       >    >    >>> Cheers,
                                >                  >>       >    >    >>> David
                                >                  >   >     >    >    >>>
                                >                  >>       >    >    >>> On 18/02/2020 2:15 pm, linzang(??) wrote:
                                >                  >>       >    >    >>>> Dear All,
                                >                  >>       >    >    >>>>         May I ask your help to review the follow changes:
                                >                  >>       >    >    >>>>         webrev:
                                >                  >>       >    >    >>>> http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_00/
                                >                  >>       >    >    >>>>      bug: https://bugs.openjdk.java.net/browse/JDK-8215624
                                >                  >>       >    >    >>>>      related CSR: https://bugs.openjdk.java.net/browse/JDK-8239290
                                >                  >>       >    >    >>>>         This patch enable parallel heap inspection of G1 for jmap histo.
                                >                  >>       >    >    >>>>         my simple test shown it can speed up 2x of jmap -histo with
                                >                  >>       >    >    >>>> parallelThreadNum set to 2 for heap at ~500M on 4-core platform.
                                >                  >>       >    >    >>>>
                                >                  >>       >    >    >>>> ------------------------------------------------------------------------
                                >                  >>       >    >    >>>> BRs,
                                >                  >>       >    >    >>>> Lin
                                >                  >>       >    >    >> >
                                >                  >>       >    >    >
                                >                  >
                                >                  >
                                >                  >
                                >
                                >
                                >
                                >
                                >
                                >










From leo.korinth at oracle.com  Mon Jul 27 09:07:49 2020
From: leo.korinth at oracle.com (Leo Korinth)
Date: Mon, 27 Jul 2020 11:07:49 +0200
Subject: RFR (S): 8242036: G1 HeapRegionRemSet::_n_coarse_entries could be
 a bool
In-Reply-To: <c57d9114-4002-ec40-74ed-ca60f8a75648@oracle.com>
References: <780c52d1-5f8b-4fe5-a5ba-4dc939b98a07@default>
 <58FFC48F-2617-453B-A69B-23E816540692@oracle.com>
 <e54cbed4-5e22-3a49-36de-ecd5b6b8c4e1@oracle.com>
 <5C4C4D95-F005-4504-8718-80E4ED56C1F4@oracle.com>
 <dcd62cc8-50fd-d7f8-cc83-3359a9435165@oracle.com>
 <33667E76-2717-4DF5-9283-33660F986716@oracle.com>
 <c57d9114-4002-ec40-74ed-ca60f8a75648@oracle.com>
Message-ID: <ed1572b9-04d2-fdd9-b96b-dd38e077656a@oracle.com>

On 25/07/2020 10:36, Thomas Schatzl wrote:
> Hi,
> 
> On 25.07.20 01:34, Kim Barrett wrote:
>>> On Jul 24, 2020, at 7:37 AM, Albert Yang <albert.m.yang at oracle.com> wrote:
>>>
>>> I see.
>>>
>>> Updated:
>>>
>>> http://cr.openjdk.java.net/~lkorinth/albert/8242036/3/
>>>
>>> I am not sure if this unconditional write will pollute the cacheline though, since this variable is accessed concurrently.
>>
>> This code is on a rare path and holding the lock.
>>
>> Change looks good.
>>
> 
> +1.
> 
> Thomas

Looks good to me too.

Thanks,
Leo


From kim.barrett at oracle.com  Mon Jul 27 09:20:00 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Mon, 27 Jul 2020 05:20:00 -0400
Subject: RFR (M): 8245721: Refactor the TaskTerminator
In-Reply-To: <d1cfdd15-1d56-a17b-297e-bd5c1c6a5df8@oracle.com>
References: <bd530d10-a4bb-969f-b142-39ff259c91b8@oracle.com>
 <931140A9-3FCF-41B4-905B-C53BAE4B8E40@oracle.com>
 <d1cfdd15-1d56-a17b-297e-bd5c1c6a5df8@oracle.com>
Message-ID: <CED70EC7-5DFD-4D46-B685-D65F76E646B2@oracle.com>

> On Jul 24, 2020, at 6:41 AM, Thomas Schatzl <thomas.schatzl at oracle.com> wrote:
> On 08.07.20 17:31, Kim Barrett wrote:
>> src/hotspot/share/gc/shared/taskTerminator.hpp
>>  128       spin_context.hard_spin_limit = MIN2(2 * spin_context.hard_spin_limit,
>>  129                                           (uint) WorkStealingHardSpins);
>> In the new code, hard_spin_limit is always (re)initialized to
>> WorkStealingHardSpins, so this does nothing.
>> In the old code, the starting spin limit was
>>   WorkStealingHardSpins >> WorkStealingSpinToYieldRatio
>> and that value was captured in a variable for (hard_spin_start) for
>> use in reinitializing the limit after each yield iteration.
>> So this is a change, and I'm guessing not an intentional one.
> 
> Fixed.

I think this wasn?t fixed; see below.

> 
> New webrev:
> http://cr.openjdk.java.net/~tschatzl/8245721/webrev.1/ (full)
> http://cr.openjdk.java.net/~tschatzl/8245721/webrev.0_to_1/ (diff)
> 
> Testing:
> tier1-5, various benchmarks
> 
> Thomas

The webrev.0_to_1 doesn't seem to be webrev on the left, but rather some
intermediate stage between webrev and webrev.1. For example, webrev.0_to_1
has DelayContext (from webrev.1 change) rather than SpinContext (from webrev
change).

------------------------------------------------------------------------------
src/hotspot/share/gc/shared/taskTerminator.cpp
 119       delay_context._hard_spin_limit = MIN2(2 * delay_context._hard_spin_limit,
 120                                           (uint) WorkStealingHardSpins);
 121       for (uint j = 0; j < delay_context._hard_spin_limit; j++) {

[pre-existing]
Why is hard_spin_limit incremented before doing the spin?

Line 120 is mis-indented.

------------------------------------------------------------------------------
src/hotspot/share/gc/shared/taskTerminator.cpp
 115       delay_context._hard_spin_limit = WorkStealingHardSpins;

This is still resetting to WorkStealingHardSpins, rather than scaling that
down by WorkStealingSpinToYieldRatio, making further executions of line 119
do nothing.  So *not* fixed.

------------------------------------------------------------------------------
src/hotspot/share/gc/shared/taskTerminator.cpp
 109     // Periodically call yield() instead spinning

Calls os::naked_yield() rather than yield().  Maybe just say "Periodically
yield instead of spinning."

------------------------------------------------------------------------------
src/hotspot/share/gc/shared/taskTerminator.cpp
 104 bool TaskTerminator::do_delay_step(DelayContext& delay_context) {

Function is returning bool indicating yields before sleep limit has been
reached.  But call ignores the result

 167           do_delay_step(delay_context);

and there is a separate check in the the caller for yields before sleep
limit being reached.

Related, the _yield_count seems to be double incremented.
 108     delay_context._yield_count++;
 162         ++delay_context._yield_count;

------------------------------------------------------------------------------

I'm finding enough puzzling things to wonder if I'm actually looking at the
code you intended to have reviewed, so I'm stopping for now, pending getting
that cleared up.



From albert.m.yang at oracle.com  Tue Jul 28 13:07:01 2020
From: albert.m.yang at oracle.com (Albert Yang)
Date: Tue, 28 Jul 2020 15:07:01 +0200
Subject: RFR (S): 8242036: G1 HeapRegionRemSet::_n_coarse_entries could be
 a bool
In-Reply-To: <ed1572b9-04d2-fdd9-b96b-dd38e077656a@oracle.com>
References: <780c52d1-5f8b-4fe5-a5ba-4dc939b98a07@default>
 <58FFC48F-2617-453B-A69B-23E816540692@oracle.com>
 <e54cbed4-5e22-3a49-36de-ecd5b6b8c4e1@oracle.com>
 <5C4C4D95-F005-4504-8718-80E4ED56C1F4@oracle.com>
 <dcd62cc8-50fd-d7f8-cc83-3359a9435165@oracle.com>
 <33667E76-2717-4DF5-9283-33660F986716@oracle.com>
 <c57d9114-4002-ec40-74ed-ca60f8a75648@oracle.com>
 <ed1572b9-04d2-fdd9-b96b-dd38e077656a@oracle.com>
Message-ID: <775df53c-5145-d44b-de5b-b12e67023d44@oracle.com>

Thank you for your review.

On 2020-07-27 11:07, Leo Korinth wrote:
> On 25/07/2020 10:36, Thomas Schatzl wrote:
>> Hi,
>>
>> On 25.07.20 01:34, Kim Barrett wrote:
>>>> On Jul 24, 2020, at 7:37 AM, Albert Yang <albert.m.yang at oracle.com> 
>>>> wrote:
>>>>
>>>> I see.
>>>>
>>>> Updated:
>>>>
>>>> http://cr.openjdk.java.net/~lkorinth/albert/8242036/3/
>>>>
>>>> I am not sure if this unconditional write will pollute the 
>>>> cacheline though, since this variable is accessed concurrently.
>>>
>>> This code is on a rare path and holding the lock.
>>>
>>> Change looks good.
>>>
>>
>> +1.
>>
>> Thomas
>
> Looks good to me too.
>
> Thanks,
> Leo


From luhenry at microsoft.com  Tue Jul 28 16:22:52 2020
From: luhenry at microsoft.com (Ludovic Henry)
Date: Tue, 28 Jul 2020 16:22:52 +0000
Subject: [aarch64-port-dev ] RFR[XXS] 8248672: utilities: Introduce
 DEPRECATED macro for GCC and MSVC
In-Reply-To: <MWHPR21MB05117357CE8A5903943234C2B0750@MWHPR21MB0511.namprd21.prod.outlook.com>
References: <MWHPR21MB05113A054F2D8291BFAFB085B0770@MWHPR21MB0511.namprd21.prod.outlook.com>
 <5e301790-8bfe-0ced-b5e2-8a9c76ae33de@oracle.com>
 <MWHPR21MB051146316FF56A63423DF514B0770@MWHPR21MB0511.namprd21.prod.outlook.com>
 <F50C37BA-FE47-465C-A60C-10547DCF8F69@oracle.com>
 <1259c3fd-b69c-6d81-0427-cb769f00bca5@redhat.com>,
 <CD1AE64E-8555-4074-97EB-831BF0A72C61@oracle.com>,
 <MWHPR21MB05117357CE8A5903943234C2B0750@MWHPR21MB0511.namprd21.prod.outlook.com>
Message-ID: <MWHPR21MB0511A0713E147045CA02522CB0730@MWHPR21MB0511.namprd21.prod.outlook.com>

Hi,

I confirm that `= delete` works, and that we get a compile-time error if you try to use it.

Please find the updated webrev at http://cr.openjdk.java.net/~burban/luhenry/8248672/webrev.01

Thank you
Ludovic

________________________________________
From: Ludovic Henry <luhenry at microsoft.com>
Sent: Sunday, July 26, 2020 16:10
To: Kim Barrett; Andrew Haley
Cc: Vladimir Kozlov; hotspot-compiler-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; openjdk-aarch64; hotspot-gc-dev at openjdk.java.net
Subject: Re: [aarch64-port-dev ] RFR[XXS] 8248672: utilities: Introduce DEPRECATED macro for GCC and MSVC

> As of early last week, a definition of "= delete;" is the way to
> poison an overload.

Let me try that locally, compile on Windows-AArch64 and Linux-AArch64, and confirm whether it works for MSVC.

________________________________________
From: Kim Barrett <kim.barrett at oracle.com>
Sent: Sunday, July 26, 2020 14:41
To: Andrew Haley
Cc: Ludovic Henry; Vladimir Kozlov; hotspot-compiler-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; openjdk-aarch64; hotspot-gc-dev at openjdk.java.net
Subject: Re: [aarch64-port-dev ] RFR[XXS] 8248672: utilities: Introduce DEPRECATED macro for GCC and MSVC

> On Jul 26, 2020, at 5:56 AM, Andrew Haley <aph at redhat.com> wrote:
>
> On 25/07/2020 00:42, Kim Barrett wrote:
>> Why are we deprecating something rather than just deleting it and
>> fixing any users?
>
> C++ overloading. AArch64 CMP (immediate) only has a limited range, so
> we only have a byte-wide Assembler::cmp() definition. The deprecation
> warning on the wider version makes sure that any maintenance
> programmer is immediately warned if it is used. There are other things
> we could do: by not providing a definition for the wider cmp() you get
> a link error, but that wouldn't be as explicit as a deprecation
> warning.
>
> The root problem is that the immediate value to CMP isn't always known
> when HotSpot is compiled, but may be calculated at runtime. We have
> seen failures in production when an immediate offset overflowed.

Yeah, I'd guessed that might be the point, and confirmed it later by
looking at the changeset that originally introduced the attribute.

As of early last week, a definition of "= delete;" is the way to
poison an overload.



From aph at redhat.com  Tue Jul 28 18:21:18 2020
From: aph at redhat.com (Andrew Haley)
Date: Tue, 28 Jul 2020 19:21:18 +0100
Subject: [aarch64-port-dev ] RFR[XXS] 8248672: utilities: Introduce
 DEPRECATED macro for GCC and MSVC
In-Reply-To: <MWHPR21MB0511A0713E147045CA02522CB0730@MWHPR21MB0511.namprd21.prod.outlook.com>
References: <MWHPR21MB05113A054F2D8291BFAFB085B0770@MWHPR21MB0511.namprd21.prod.outlook.com>
 <5e301790-8bfe-0ced-b5e2-8a9c76ae33de@oracle.com>
 <MWHPR21MB051146316FF56A63423DF514B0770@MWHPR21MB0511.namprd21.prod.outlook.com>
 <F50C37BA-FE47-465C-A60C-10547DCF8F69@oracle.com>
 <1259c3fd-b69c-6d81-0427-cb769f00bca5@redhat.com>
 <CD1AE64E-8555-4074-97EB-831BF0A72C61@oracle.com>
 <MWHPR21MB05117357CE8A5903943234C2B0750@MWHPR21MB0511.namprd21.prod.outlook.com>
 <MWHPR21MB0511A0713E147045CA02522CB0730@MWHPR21MB0511.namprd21.prod.outlook.com>
Message-ID: <279cd44e-dfcc-00c2-4aee-1cca630bd5ec@redhat.com>

On 28/07/2020 17:22, Ludovic Henry wrote:
> I confirm that `= delete` works, and that we get a compile-time error if you try to use it.
> 
> Please find the updated webrev at http://cr.openjdk.java.net/~burban/luhenry/8248672/webrev.01

OK, thanks.

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671



From kim.barrett at oracle.com  Wed Jul 29 00:34:35 2020
From: kim.barrett at oracle.com (Kim Barrett)
Date: Tue, 28 Jul 2020 20:34:35 -0400
Subject: [aarch64-port-dev ] RFR[XXS] 8248672: utilities: Introduce
 DEPRECATED macro for GCC and MSVC
In-Reply-To: <MWHPR21MB0511A0713E147045CA02522CB0730@MWHPR21MB0511.namprd21.prod.outlook.com>
References: <MWHPR21MB05113A054F2D8291BFAFB085B0770@MWHPR21MB0511.namprd21.prod.outlook.com>
 <5e301790-8bfe-0ced-b5e2-8a9c76ae33de@oracle.com>
 <MWHPR21MB051146316FF56A63423DF514B0770@MWHPR21MB0511.namprd21.prod.outlook.com>
 <F50C37BA-FE47-465C-A60C-10547DCF8F69@oracle.com>
 <1259c3fd-b69c-6d81-0427-cb769f00bca5@redhat.com>
 <CD1AE64E-8555-4074-97EB-831BF0A72C61@oracle.com>
 <MWHPR21MB05117357CE8A5903943234C2B0750@MWHPR21MB0511.namprd21.prod.outlook.com>
 <MWHPR21MB0511A0713E147045CA02522CB0730@MWHPR21MB0511.namprd21.prod.outlook.com>
Message-ID: <116277BD-EA21-49AA-8DE1-DBC06ED43C43@oracle.com>

> On Jul 28, 2020, at 12:22 PM, Ludovic Henry <luhenry at microsoft.com> wrote:
> 
> Hi,
> 
> I confirm that `= delete` works, and that we get a compile-time error if you try to use it.
> 
> Please find the updated webrev at http://cr.openjdk.java.net/~burban/luhenry/8248672/webrev.01
> 
> Thank you
> Ludovic

Looks good.

Probably the bug title should be updated.



From linzang at tencent.com  Wed Jul 29 12:01:30 2020
From: linzang at tencent.com (=?utf-8?B?bGluemFuZyjoh6fnkLMp?=)
Date: Wed, 29 Jul 2020 12:01:30 +0000
Subject: RFR(L): 8215624: add parallel heap inspection support for jmap
 histo(G1)(Internet mail)
In-Reply-To: <2C4A6B09-F281-433E-95CE-9587FC3C769F@tencent.com>
References: <EF17E5D8-648B-46DA-82DE-0BF4CFDD42D6@amazon.com>
 <B8E2CD92-2E27-4EBD-9FE1-9F51FB627CE0@tencent.com>
 <2C4A6B09-F281-433E-95CE-9587FC3C769F@tencent.com>
Message-ID: <F24B78BB-6722-4A91-B1FF-35C69AA67196@tencent.com>

Upload a new change at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_10/
It fix an issue of windows fail :

####################################
In heapInspect.cpp
- size_t HeapInspection::populate_table(KlassInfoTable* cit, BoolObjectClosure *filter, uint parallel_thread_num) {
+ uint HeapInspection::populate_table(KlassInfoTable* cit, BoolObjectClosure *filter, uint parallel_thread_num) {
####################################
In heapInspect.hpp
- size_t populate_table(KlassInfoTable* cit, BoolObjectClosure* filter = NULL, uint parallel_thread_num = 1) NOT_SERVICES_RETURN_(0);
+  uint populate_table(KlassInfoTable* cit, BoolObjectClosure* filter = NULL, uint parallel_thread_num = 1) NOT_SERVICES_RETURN_(0);
####################################


BRs,
Lin

?On 2020/7/27, 11:26 AM, "linzang(??)" <linzang at tencent.com> wrote:

    I update a new change at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_09
    It includes a tiny fix of build failure on windows:
    ####################################
    In attachListener.cpp:
    -  uint parallel_thread_num = MAX(1, (uint)os::initial_active_processor_count() * 3 / 8);
    +  uint parallel_thread_num = MAX2<uint>(1, (uint)os::initial_active_processor_count() * 3 / 8);
    ####################################

    BRs,
    Lin

    On 2020/7/23, 11:56 AM, "linzang(??)" <linzang at tencent.com> wrote:

        Hi Paul,
             Thanks for your help, that all looks good to me. 
             Just 2 minor changes:  
        	? delete the final return in ParHeapInspectTask::work, you mentioned it but seems not include in the webrev :-)
        	? delete a unnecessary blank line in heapInspect.cpp at merge_entry()

        #########################################################################
        --- old/src/hotspot/share/memory/heapInspection.cpp     2020-07-23 11:23:29.281666456 +0800
        +++ new/src/hotspot/share/memory/heapInspection.cpp     2020-07-23 11:23:29.017666447 +0800
        @@ -251,7 +251,6 @@
             _size_of_instances_in_words += cie->words();
             return true;
           }   
        -
           return false;
         }

        @@ -568,7 +567,6 @@
             Atomic::add(&_missed_count, missed_count);
           } else {
             Atomic::store(&_success, false);
        -   return;
           }   
         }
        #########################################################################


        Here is the webrev  http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_08/

        BRs,
        Lin
        ---------------------------------------------
        From: "Hohensee, Paul" <hohensee at amazon.com>
        Date: Thursday, July 23, 2020 at 6:48 AM
        To: "linzang(??)" <linzang at tencent.com>, Stefan Karlsson <stefan.karlsson at oracle.com>, "serguei.spitsyn at oracle.com" <serguei.spitsyn at oracle.com>, David Holmes <david.holmes at oracle.com>, serviceability-dev <serviceability-dev at openjdk.java.net>, "hotspot-gc-dev at openjdk.java.net" <hotspot-gc-dev at openjdk.java.net>
        Subject: RE: RFR(L): 8215624: add parallel heap inspection support for jmap histo(G1)(Internet mail)

        Just small things.

        heapInspection.cpp:

        In ParHeapInspectTask::work, remove the final return statement and fix the following ?}? indent. I.e., replace

        +    Atomic::store(&_success, false);
        +    return;
        +   }

        with

        +    Atomic::store(&_success, false);
        +  }

        In HeapInspection::heap_inspection, missed_count should be a uint to match other missed_count declarations, and should be initialized to the result of populate_table() rather than separately to 0.

        attachListener.cpp:

        In heap_inspection, initial_processor_count returns an int, so cast its result to a uint.

        Similarly, parse_uintx returns a uintx, so cast its result (num) to uint when assigning to parallel_thread_num.

        BasicJMapTest.java:

        I took the liberty of refactoring testHisto*/histoToFile/testDump*/dump to remove redundant interposition methods and make histoToFile and dump look as similar as possible.

        Webrev with the above changes in

        http://cr.openjdk.java.net/~phh/8214535/webrev.01/

        Thanks,
        Paul

        On 7/15/20, 2:13 AM, "linzang(??)" <linzang at tencent.com> wrote:

             Upload a new webrev at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07/
             It fix a potential issue that unexpected number of threads maybe calculated for "parallel" option of jmap -histo in container.
            As shown at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07-delta/src/hotspot/share/services/attachListener.cpp.udiff.html

            ############### attachListener.cpp ####################
            @@ -252,11 +252,11 @@
             static jint heap_inspection(AttachOperation* op, outputStream* out) {
               bool live_objects_only = true;   // default is true to retain the behavior before this change is made
               outputStream* os = out;   // if path not specified or path is NULL, use out
               fileStream* fs = NULL;
               const char* arg0 = op->arg(0);
            -  uint parallel_thread_num = MAX(1, os::processor_count() * 3 / 8); // default is less than half of processors.
            +  uint parallel_thread_num = MAX(1, os::initial_active_processor_count() * 3 / 8); // default is less than half of processors.
               if (arg0 != NULL && (strlen(arg0) > 0)) {
                 if (strcmp(arg0, "-all") != 0 && strcmp(arg0, "-live") != 0) {
                   out->print_cr("Invalid argument to inspectheap operation: %s", arg0);
                   return JNI_ERR;
                 }
            ###################################################

            Thanks.

            BRs,
           Lin

            On 2020/7/9, 3:22 PM, "linzang(??)" <linzang at tencent.com> wrote:

                Hi Paul,
                    Thanks for reviewing!
                    >>
                    >>     I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp.
                    >>
                    The reason I made the change in Jmap.java that compose all arguments as 1 string , instead of passing 3 argments, is to avoid the compatibility issue, as we discussed in http://mail.openjdk.java.net/pipermail/serviceability-dev/2019-February/thread.html#27240.  The root cause of the compatibility issue is because max argument count in HotspotVirtualMachineImpl.java and attachlistener.cpp need to be enlarged (changes like http://hg.openjdk.java.net/jdk/jdk/rev/e7cf035682e3#l2.1) when jmap has more than 3 arguments. But if user use an old jcmd/jmap tool, it may stuck at socket read(), because the "max argument count" don't match.
                     I re-checked this change, the argument count of jmap histo is equal to 3 (live, file, parallel), so it can work normally even without the change of passing argument. But I think we have to face the problem if more arguments is added in jcmd alike tools later, not sure whether it should be sloved (or a workaround) in this changeset.

                    And here are the lastest webrev and delta:
                    http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06/
                    http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06-delta/

                Cheers,
                Lin

                On 2020/7/7, 5:57 AM, "Hohensee, Paul" <hohensee at amazon.com> wrote:

                    I'd like to see this feature added. :)

                    The CSR looks good, as does the basic parallel inspection algorithm. Stefan's done the GC part, so I'll stick to the non-GC part (fwiw, the GC part lgtm).

                    I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp.

                    heapInspection.hpp:

                    _shared_miss_count (s/b _missed_count, see below) isn't a size, so it should be a uint instead of a size_t. Same with the new parallel_thread_num argument to heap_inspection() and populate_table().

                    Comment copy-edit:
                    +// Parallel heap inspection task. Parallel inspection can fail due to
                    +// a native OOM when allocating memory for TL-KlassInfoTable.
                    +// _success will be set false on an OOM, and serial inspection tried.

                    _shared_miss_count should be _missed_count to match the missed_count() getter, or rename missed_count() to be shared_miss_count(). Whichever way you go, the field type should match the getter result type: uint is reasonable.

                    heapInspection.cpp:

                    You might use ResourceMark twice in populate_table, separately for the parallel attempt and the serial code. If the parallel attempt fails and available memory is low, it would be good to clean up the memory used by the parallel attempt before doing the serial code.

                    Style nit in KlassInfoTable::merge_entry(). I'd line up the definitions of k and elt, so "k" is even with "elt". And, because it's two lines shorter, I'd replace
                    +  } else {
                    +    return false;
                    +  }
                    with
                    +  return false;

                    KlassInfoTableMergeClosure.is_success() should be just success() (i.e., no "is_" prefix) because it's a getter.

                    I'd reorganize the code in populate_table() to make it more clear, vis (I changed _shared_missed_count to _missed_count)
                    +  if (cit.allocation_failed()) {
                    +    // fail to allocate memory, stop parallel mode
                    +    Atomic::store(&_success, false);
                    +    return;
                    +  }
                    +  RecordInstanceClosure ric(&cit, _filter);
                    +  _poi->object_iterate(&ric, worker_id);
                    +  missed_count = ric.missed_count();
                    +  {
                    +    MutexLocker x(&_mutex);
                    +    merge_success = _shared_cit->merge(&cit);
                    +  }
                    +  if (merge_success) {
                    +    Atomic::add(&_missed_count, missed_count);
                    +  else {
                    +    Atomic::store(&_success, false);
                    +  }

                    Thanks,
                    Paul

                    On 6/29/20, 7:20 PM, "linzang(??)" <linzang at tencent.com> wrote:

                        Dear All,
                                Sorry to bother again, I just want to make sure that is this change worth to be continue to work on? If decision is made to not. I think I can drop this work and stop asking for help reviewing...
                                Thanks for all your help about reviewing this previously.

                        BRs,
                        Lin

                        On 2020/5/9, 3:47 PM, "linzang(??)" <linzang at tencent.com> wrote:

                            Dear All,
                                   May I ask your help again for review the latest change?  Thanks!

                            BRs,
                            Lin

                            On 2020/4/28, 1:54 PM, "linzang(??)" <linzang at tencent.com> wrote:

                                Hi Stefan,
                                  >>  - Adding Atomic::load/store.
                                  >>  - Removing the time measurement in the run_task. I renamed G1's function
                                  >>  to run_task_timed. If we need this outside of G1, we can rethink the API
                                  >>  at that point.
                                   >>  - ZGC style cleanups
                                   Thanks for revising the patch,  they are all good to me, and I have made a tiny change based on it:
                                       http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04/
                                       http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04-delta/
                                  it reduce the scope of mutex in ParHeapInspectTask, and delete unnecessary comments.

                                BRs,
                                Lin

                                On 2020/4/27, 4:34 PM, "Stefan Karlsson" <stefan.karlsson at oracle.com> wrote:

                                    Hi Lin,

                                    On 2020-04-26 05:10, linzang(??) wrote:
                                    > Hi Stefan and Paul?
                                    >      I have made a new patch based on your comments and Stefan's Poc code:
                                    >      Webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03/
                                    >      Delta(based on Stefan's change:) : http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03-delta/webrev_03-delta/

                                    Thanks for providing a delta patch. It makes it much easier to look at,
                                    and more likely for reviewers to continue reviewing.

                                    I'm going to continue focusing on the GC parts, and leave the rest to
                                    others to review.

                                    >
                                    >      And Here are main changed I made and want to discuss with you:
                                    >      1.  changed"parallelThreadNum=" to "parallel=" for jmap -histo options.
                                    >      2.  Add logic to test where parallelHeapInspection is fail, in heapInspection.cpp
                                    >            This is because the parHeapInspectTask create thread local KlassInfoTable in it's work() method, and this may fail because of native OOM, in this case, the parallel should fail and serial heap inspection can be tried.
                                    >            One more thing I want discuss with you is about the member "_success" of parHeapInspectTask, when native OOM happenes, it is set to false. And since this "set" operation can be conducted in multiple threads, should it be atomic ops?  IMO, this is not necessary because "_success" can only be set to false, and there is no way to change it from back to true after the ParHeapInspectTask instance is created, so it is save to be non-atomic, do you agree with that?

                                    In these situations you should be using the Atomic::load/store
                                    primitives. We're moving toward a later C++ standard were data races are
                                   considered undefined behavior.

                                    >     3. make CollectedHeap::run_task() be an abstract virtual func, so that every subclass of collectedHeap should support it, so later implementation of new collectedHeap will not miss the "parallel" features.
                                    >           The problem I want to discuss with you is about epsilonHeap and SerialHeap, as they may not need parallel heap iteration, so I only make task->work(0), in case the run_task() is invoked someway in future. Another way is to left run_task()  unimplemented, which one do you think is better?

                                    I don't have a strong opinion about this.

                                      And also please help take a look at the zHeap, as there is a class
                                    zTask that wrap the abstractGangTask, and the collectedHeap::run_task()
                                    only accept  AbstraceGangTask* as argument, so I made a delegate class
                                    to adapt it , please see src/hotspot/share/gc/z/zHeap.cpp.
                                    >
                                    >        There maybe other better ways to sovle the above problems, welcome for any comments, Thanks!

                                    I've created a few cleanups and changes on top of your latest patch:

                                    https://cr.openjdk.java.net/~stefank/8215624/webrev.02.delta
                                    https://cr.openjdk.java.net/~stefank/8215624/webrev.02

                                    - Adding Atomic::load/store.
                                    - Removing the time measurement in the run_task. I renamed G1's function
                                    to run_task_timed. If we need this outside of G1, we can rethink the API
                                    at that point.
                                    - ZGC style cleanups

                                    Thanks,
                                    StefanK

                                    >
                                    > BRs,
                                    > Lin
                                    >
                                    > On 2020/4/23, 11:08 AM, "linzang(??)" <linzang at tencent.com> wrote:
                                    >
                                    >      Thanks Paul! I agree with using "parallel", will make the update in next patch, Thanks for help update the CSR.
                                    >
                                    >      BRs,
                                    >      Lin
                                    >
                                    >      On 2020/4/23, 4:42 AM, "Hohensee, Paul" <hohensee at amazon.com> wrote:
                                    >
                                    >          For the interface, I'd use "parallel" instead of "parallelThreadNum". All the other options are lower case, and it's a lot easier to type "parallel". I took the liberty of updating the CSR. If you're ok with it, you might want to change variable names and such, plus of course JMap.usage.
                                    >
                                    >          Thanks,
                                    >          Paul
                                    >
                                    >          On 4/22/20, 2:29 AM, "serviceability-dev on behalf of linzang(??)" <serviceability-dev-bounces at openjdk.java.net on behalf of linzang at tencent.com> wrote:
                                    >
                                    >              Dear Stefan,
                                    >
                                    >                      Thanks a lot! I agree with you to decouple the heap inspection code with GC's.
                                    >                      I will start  from your POC code, may discuss with you later.
                                    >
                                    >
                                    >              BRs,
                                    >              Lin
                                    >
                                    >              On 2020/4/22, 5:14 PM, "Stefan Karlsson" <stefan.karlsson at oracle.com> wrote:
                                    >
                                    >                  Hi Lin,
                                    >
                                    >                  I took a look at this earlier and saw that the heap inspection code is
                                    >                  strongly coupled with the CollectedHeap and G1CollectedHeap. I'd prefer
                                    >                  if we'd abstract this away, so that the GCs only provide a "parallel
                                    >                  object iteration" interface, and the heap inspection code is kept elsewhere.
                                    >
                                    >                  I started experimenting with doing that, but other higher-priority (to
                                    >                  me) tasks have had to take precedence.
                                    >
                                    >                  I've uploaded my work-in-progress / proof-of-concept:
                                    >                    https://cr.openjdk.java.net/~stefank/8215624/webrev.01.delta/
                                    >                    https://cr.openjdk.java.net/~stefank/8215624/webrev.01/
                                    >
                                    >                  The current code doesn't handle the lifecycle (deletion) of the
                                    >                  ParallelObjectIterators. There's also code left unimplemented in around
                                    >                  CollectedHeap::run_task. However, I think this could work as a basis to
                                    >                  pull out the heap inspection code out of the GCs.
                                    >
                                    >                  Thanks,
                                    >                  StefanK
                                    >
                                    >                  On 2020-04-22 02:21, linzang(??) wrote:
                                    >                  > Dear all,
                                    >                  >       May I ask you help to review? This RFR has been there for quite a while.
                                    >                  >       Thanks!
                                    >                  >
                                    >                  > BRs,
                                    >                  > Lin
                                    >                  >
                                    >                  > > On 2020/3/16, 5:18 PM, "linzang(??)" <linzang at tencent.com> wrote:>
                                    >                  >
                                    >                  >>    Just update a new path, my preliminary measure show about 3.5x speedup of jmap histo on a nearly full 4GB G1 heap (8-core platform with parallel thread number set to 4).
                                    >                  >>     webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_02/
                                    >                  >>     bug: https://bugs.openjdk.java.net/browse/JDK-8215624
                                    >                  >>     CSR: https://bugs.openjdk.java.net/browse/JDK-8239290
                                    >                  >>     BRs,
                                    >                  >>       Lin
                                    >                  >>       > On 2020/3/2, 9:56 PM, "linzang(??)" <linzang at tencent.com> wrote:
                                    >                  >>       >
                                    >                  >>       >    Dear all,
                                    >                  >>       >          Let me try to ease the reviewing work by some explanation :P
                                    >                  >>       >          The patch's target is to speed up jmap -histo for heap iteration, from my experience it is necessary for large heap investigation. E.g in bigData scenario I have tried to conduct jmap -histo against 180GB heap, it does take quite a while.
                                    >                  >>       >          And if my understanding is corrent, even the jmap -histo without "live" option does heap inspection with heap lock acquired. so it is very likely to block mutator thread in allocation-sensitive scenario. I would say the faster the heap inspection does, the shorter the mutator be blocked. This is parallel iteration for jmap is necessary.
                                    >                  >>       >          I think the parallel heap inspection should be applied to all kind of heap. However, consider the heap layout are different for  GCs, much time is required to understand all kinds of the heap layout to make the whole change. IMO, It is not wise to have a huge patch for the whole solution at once, and it is even harder to review it. So I plan to implement it incrementally, the first patch (this one) is going to confirm the implemention detail of how jmap accept the new option, passes it to attachListener of the jvm process and then how to make the parallel inspection closure be generic enough to make it easy to extend to different heap layout. And also how to implement the heap inspection in specific gc's heap. This patch use G1's heap as the begining.
                                    >                  >>       >          This patch actually do several things:
                                    >                  >>       >          1. Add an option "parallelThreadNum=<N>" to jmap -histo, the default behavior is to set N to 0, means let's JVM decide how many threads to use for heap inspection. Set this option to 1 will disable parallel heap inspection. (more details in CSR: https://bugs.openjdk.java.net/browse/JDK-8239290)
                                    >                  >>       >          2. Make a change in how Jmap passing arguments, changes in http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java.udiff.html, originally it pass options as separate arguments to attachListener, this patch change to that all options be compose to a single string. So the arg_count_max in attachListener.hpp do not need to be changed, and hence avoid the compatibility issue, as disscussed at https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-March/027334.html
                                    >                  >>       >         3. Add an abstract class ParHeapInspectTask in heapInspection.hpp / heapInspection.cpp, It's work(uint worker_id) method prepares the data structure (KlassInfoTable) need for every parallel worker thread, and then call do_object_iterate_parallel() which is heap specific implementation. I also added some machenism in KlassInfoTable to support parallel iteration, such as merge().
                                    >                  >>       >        4. In specific heap (G1 in this patch), create a subclass of ParHeapInspectTask, implement the do_object_iterate_parallel() for parallel heap inspection. For G1, it simply invoke g1CollectedHeap's object_iterate_parallel().
                                    >                  >>       >        5. Add related test.
                                    >                  >>       >        6. it may be easy to extend this patch for other kinds of heap by creating subclass of ParHeapInspectTask and implement the do_object_iterate_parallel().
                                    >                  >>       >
                                    >                  >>       >    Hope these info could help on code review and initate the discussion :-)
                                    >                  >>       >    Thanks!
                                    >                  >>       >
                                    >                  >>       >    BRs,
                                    >                  >>       >    Lin
                                    >                  >>       >    >On 2020/2/19, 9:40 AM, "linzang(??)" <linzang at tencent.com> wrote:.
                                    >                  >>       >    >
                                    >                  >>       >    >  Re-post this RFR with correct enhancement number to make it trackable.
                                    >                  >>       >    >  please ignore the previous wrong post. sorry for troubles.
                                    >                  >>       >    >
                                    >                  >>       >    >   webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/
                                    >                  >>       >    >    Hi bug: https://bugs.openjdk.java.net/browse/JDK-8215624
                                    >                  >>       >    >    CSR: https://bugs.openjdk.java.net/browse/JDK-8239290
                                    >                  >>       >    >    --------------
                                    >                  >>       >    >    Lin
                                    >                  >>       >    >    >Hi Lin,
                                    >                  >   >     >    >    >
                                    >                  >>       >    >    >Could you, please, re-post your RFR with the right enhancement number in
                                    >                  >>       >    >    >the message subject?
                                    >                  >>       >    >    >It will be more trackable this way.
                                    >                  >>       >    >    >
                                    >                  >>       >    >    >Thanks,
                                    >                  >>       >    >    >Serguei
                                    >                  >>       >    >    >
                                    >                  >>       >    >    >
                                    >                  >>       >    >    >On 2/17/20 10:29 PM, linzang(??) wrote:
                                    >                  >>       >    >    >> Dear David,
                                    >                  >>       >    >    >>        Thanks a lot!
                                    >                  >>       >    >    >>       I have updated the refined code to http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_01/.
                                    >                  >>       >    >    >>        IMHO the parallel heap inspection can be extended to all kinds of heap as long as the heap layout can support parallel iteration.
                                    >                  >>       >    >    >>        Maybe we can firstly use this webrev to discuss how to implement it, because I am not sure my current implementation is an appropriate way to communicate with collectedHeap, then we can extend the solution to other kinds of heap.
                                    >                  >>       >    >    >>
                                    >                  >>       >    >    >> Thanks,
                                    >                  >>       >    >    >> --------------
                                    >                  >>       >    >    >> Lin
                                    >                  >>       >    >    >>> Hi Lin,
                                    >                  >>       >    >    >>>
                                    >                  >>       >    >    >>> Adding in hotspot-gc-dev as they need to see how this interacts with GC
                                    >                  >>       >    >    >>> worker threads, and whether it needs to be extended beyond G1.
                                    >                  >>       >    >    >>>
                                    >                  >>       >    >   >>> I happened to spot one nit when browsing:
                                    >                  >>       >    >    >>>
                                    >                  >>       >    >    >>> src/hotspot/share/gc/shared/collectedHeap.hpp
                                    >                  >>       >    >    >>>
                                    >                  >>       >    >    >>> +   virtual bool run_par_heap_inspect_task(KlassInfoTable* cit,
                                    >                  >>       >    >    >>> +                                          BoolObjectClosure* filter,
                                    >                  >>       >    >    >>> +                                          size_t* missed_count,
                                    >                  >>       >    >    >>> +                                          size_t thread_num) {
                                    >                  >>       >    >    >>> +     return NULL;
                                    >                  >>       >    >    >>>
                                    >                  >>       >    >    >>> s/NULL/false/
                                    >                  >>       >    >    >>>
                                    >                  >>       >    >    >>> Cheers,
                                    >                  >>       >    >    >>> David
                                    >                  >   >     >    >    >>>
                                    >                  >>       >    >    >>> On 18/02/2020 2:15 pm, linzang(??) wrote:
                                    >                  >>       >    >    >>>> Dear All,
                                    >                  >>       >    >    >>>>         May I ask your help to review the follow changes:
                                    >                  >>       >    >    >>>>         webrev:
                                    >                  >>       >    >    >>>> http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_00/
                                    >                  >>       >    >    >>>>      bug: https://bugs.openjdk.java.net/browse/JDK-8215624
                                    >                  >>       >    >    >>>>      related CSR: https://bugs.openjdk.java.net/browse/JDK-8239290
                                    >                  >>       >    >    >>>>         This patch enable parallel heap inspection of G1 for jmap histo.
                                    >                  >>       >    >    >>>>         my simple test shown it can speed up 2x of jmap -histo with
                                    >                  >>       >    >    >>>> parallelThreadNum set to 2 for heap at ~500M on 4-core platform.
                                    >                  >>       >    >    >>>>
                                    >                  >>       >    >    >>>> ------------------------------------------------------------------------
                                    >                  >>       >    >    >>>> BRs,
                                    >                  >>       >    >    >>>> Lin
                                    >                  >>       >    >    >> >
                                    >                  >>       >    >    >
                                    >                  >
                                    >                  >
                                    >                  >
                                    >
                                    >
                                    >
                                    >
                                    >
                                    >











From albert.m.yang at oracle.com  Wed Jul 29 12:19:32 2020
From: albert.m.yang at oracle.com (Albert Yang)
Date: Wed, 29 Jul 2020 14:19:32 +0200
Subject: RFR (S): 8250628: ZGC: `fixup_partial_loads` was removed, but still
 are referenced
Message-ID: <4e64dfac-5405-7d21-5ed0-3ce1f7d27c1a@oracle.com>

Hi,

https://bugs.openjdk.java.net/browse/JDK-8250628

http://cr.openjdk.java.net/~lkorinth/albert/8250628/0/

Case 2&3 can be removed, since all roots will go through the load 
barrier before a safepoint. To be safe, I have tried setting them to 
true via `FLAG_SET_DEFAULT(_, true);`, and no failures are observed for 
hotspot-gc.

Tested: hotspot-gc

/Albert



From luhenry at microsoft.com  Wed Jul 29 13:59:47 2020
From: luhenry at microsoft.com (Ludovic Henry)
Date: Wed, 29 Jul 2020 13:59:47 +0000
Subject: [aarch64-port-dev ] RFR[XXS] 8248672: utilities: Introduce
 DEPRECATED macro for GCC and MSVC
In-Reply-To: <116277BD-EA21-49AA-8DE1-DBC06ED43C43@oracle.com>
References: <MWHPR21MB05113A054F2D8291BFAFB085B0770@MWHPR21MB0511.namprd21.prod.outlook.com>
 <5e301790-8bfe-0ced-b5e2-8a9c76ae33de@oracle.com>
 <MWHPR21MB051146316FF56A63423DF514B0770@MWHPR21MB0511.namprd21.prod.outlook.com>
 <F50C37BA-FE47-465C-A60C-10547DCF8F69@oracle.com>
 <1259c3fd-b69c-6d81-0427-cb769f00bca5@redhat.com>
 <CD1AE64E-8555-4074-97EB-831BF0A72C61@oracle.com>
 <MWHPR21MB05117357CE8A5903943234C2B0750@MWHPR21MB0511.namprd21.prod.outlook.com>
 <MWHPR21MB0511A0713E147045CA02522CB0730@MWHPR21MB0511.namprd21.prod.outlook.com>,
 <116277BD-EA21-49AA-8DE1-DBC06ED43C43@oracle.com>
Message-ID: <MWHPR21MB0511A595027B7328169F3816B0700@MWHPR21MB0511.namprd21.prod.outlook.com>

Hi Kim,

I just had it updated.

Thanks

________________________________________
From: Kim Barrett <kim.barrett at oracle.com>
Sent: Tuesday, July 28, 2020 17:34
To: Ludovic Henry
Cc: Andrew Haley; Vladimir Kozlov; hotspot-compiler-dev at openjdk.java.net; aarch64-port-dev at openjdk.java.net; openjdk-aarch64; hotspot-gc-dev at openjdk.java.net
Subject: Re: [aarch64-port-dev ] RFR[XXS] 8248672: utilities: Introduce DEPRECATED macro for GCC and MSVC

> On Jul 28, 2020, at 12:22 PM, Ludovic Henry <luhenry at microsoft.com> wrote:
>
> Hi,
>
> I confirm that `= delete` works, and that we get a compile-time error if you try to use it.
>
> Please find the updated webrev at https://nam06.safelinks.protection.outlook.com/?url=http:%2F%2Fcr.openjdk.java.net%2F~burban%2Fluhenry%2F8248672%2Fwebrev.01&amp;data=02%7C01%7Cluhenry%40microsoft.com%7C80a04191ac684f7ed64008d833577d4d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637315798145154680&amp;sdata=FLhVIXFkMZpRfpuV2jiVoZmqcm8dHKfg8SgNtQJSUrE%3D&amp;reserved=0
>
> Thank you
> Ludovic

Looks good.

Probably the bug title should be updated.



From erik.osterlund at oracle.com  Wed Jul 29 20:12:17 2020
From: erik.osterlund at oracle.com (=?utf-8?Q?Erik_=C3=96sterlund?=)
Date: Wed, 29 Jul 2020 22:12:17 +0200
Subject: RFR (S): 8250628: ZGC: `fixup_partial_loads` was removed,
 but still are referenced
In-Reply-To: <4e64dfac-5405-7d21-5ed0-3ce1f7d27c1a@oracle.com>
References: <4e64dfac-5405-7d21-5ed0-3ce1f7d27c1a@oracle.com>
Message-ID: <8CB025C4-FA9F-4F4C-BE85-EAE7FA059DD5@oracle.com>

Hi Albert,

Looks good.

Thanks,
/Erik

> On 29 Jul 2020, at 14:19, Albert Yang <albert.m.yang at oracle.com> wrote:
> 
> ?Hi,
> 
> https://bugs.openjdk.java.net/browse/JDK-8250628
> 
> http://cr.openjdk.java.net/~lkorinth/albert/8250628/0/
> 
> Case 2&3 can be removed, since all roots will go through the load barrier before a safepoint. To be safe, I have tried setting them to true via `FLAG_SET_DEFAULT(_, true);`, and no failures are observed for hotspot-gc.
> 
> Tested: hotspot-gc
> 
> /Albert
> 



From hohensee at amazon.com  Wed Jul 29 21:21:19 2020
From: hohensee at amazon.com (Hohensee, Paul)
Date: Wed, 29 Jul 2020 21:21:19 +0000
Subject: RFR(L): 8215624: add parallel heap inspection support for jmap
 histo(G1)(Internet mail)
Message-ID: <01181077-0DD7-4AEE-945B-5D6E21224A31@amazon.com>

A submit repo run with this succeeded, so afaic you're good to go. Stefan, you reviewed the GC part before, it'd be great if you could ok the final version.

Thanks,
Paul

?On 7/29/20, 5:02 AM, "linzang(??)" <linzang at tencent.com> wrote:

    Upload a new change at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_10/
    It fix an issue of windows fail :

    ####################################
    In heapInspect.cpp
    - size_t HeapInspection::populate_table(KlassInfoTable* cit, BoolObjectClosure *filter, uint parallel_thread_num) {
    + uint HeapInspection::populate_table(KlassInfoTable* cit, BoolObjectClosure *filter, uint parallel_thread_num) {
    ####################################
    In heapInspect.hpp
    - size_t populate_table(KlassInfoTable* cit, BoolObjectClosure* filter = NULL, uint parallel_thread_num = 1) NOT_SERVICES_RETURN_(0);
    +  uint populate_table(KlassInfoTable* cit, BoolObjectClosure* filter = NULL, uint parallel_thread_num = 1) NOT_SERVICES_RETURN_(0);
    ####################################


    BRs,
    Lin

    On 2020/7/27, 11:26 AM, "linzang(??)" <linzang at tencent.com> wrote:

        I update a new change at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_09
        It includes a tiny fix of build failure on windows:
        ####################################
        In attachListener.cpp:
        -  uint parallel_thread_num = MAX(1, (uint)os::initial_active_processor_count() * 3 / 8);
        +  uint parallel_thread_num = MAX2<uint>(1, (uint)os::initial_active_processor_count() * 3 / 8);
        ####################################

        BRs,
        Lin

        On 2020/7/23, 11:56 AM, "linzang(??)" <linzang at tencent.com> wrote:

            Hi Paul,
                 Thanks for your help, that all looks good to me.
                 Just 2 minor changes:
                    ? delete the final return in ParHeapInspectTask::work, you mentioned it but seems not include in the webrev :-)
                    ? delete a unnecessary blank line in heapInspect.cpp at merge_entry()

            #########################################################################
            --- old/src/hotspot/share/memory/heapInspection.cpp     2020-07-23 11:23:29.281666456 +0800
            +++ new/src/hotspot/share/memory/heapInspection.cpp     2020-07-23 11:23:29.017666447 +0800
            @@ -251,7 +251,6 @@
                 _size_of_instances_in_words += cie->words();
                 return true;
               }
            -
               return false;
             }

            @@ -568,7 +567,6 @@
                 Atomic::add(&_missed_count, missed_count);
               } else {
                 Atomic::store(&_success, false);
            -   return;
               }
             }
            #########################################################################


            Here is the webrev  http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_08/

            BRs,
            Lin
            ---------------------------------------------
            From: "Hohensee, Paul" <hohensee at amazon.com>
            Date: Thursday, July 23, 2020 at 6:48 AM
            To: "linzang(??)" <linzang at tencent.com>, Stefan Karlsson <stefan.karlsson at oracle.com>, "serguei.spitsyn at oracle.com" <serguei.spitsyn at oracle.com>, David Holmes <david.holmes at oracle.com>, serviceability-dev <serviceability-dev at openjdk.java.net>, "hotspot-gc-dev at openjdk.java.net" <hotspot-gc-dev at openjdk.java.net>
            Subject: RE: RFR(L): 8215624: add parallel heap inspection support for jmap histo(G1)(Internet mail)

            Just small things.

            heapInspection.cpp:

            In ParHeapInspectTask::work, remove the final return statement and fix the following ?}? indent. I.e., replace

            +    Atomic::store(&_success, false);
            +    return;
            +   }

            with

            +    Atomic::store(&_success, false);
            +  }

            In HeapInspection::heap_inspection, missed_count should be a uint to match other missed_count declarations, and should be initialized to the result of populate_table() rather than separately to 0.

            attachListener.cpp:

            In heap_inspection, initial_processor_count returns an int, so cast its result to a uint.

            Similarly, parse_uintx returns a uintx, so cast its result (num) to uint when assigning to parallel_thread_num.

            BasicJMapTest.java:

            I took the liberty of refactoring testHisto*/histoToFile/testDump*/dump to remove redundant interposition methods and make histoToFile and dump look as similar as possible.

            Webrev with the above changes in

            http://cr.openjdk.java.net/~phh/8214535/webrev.01/

            Thanks,
            Paul

            On 7/15/20, 2:13 AM, "linzang(??)" <linzang at tencent.com> wrote:

                 Upload a new webrev at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07/
                 It fix a potential issue that unexpected number of threads maybe calculated for "parallel" option of jmap -histo in container.
                As shown at http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_07-delta/src/hotspot/share/services/attachListener.cpp.udiff.html

                ############### attachListener.cpp ####################
                @@ -252,11 +252,11 @@
                 static jint heap_inspection(AttachOperation* op, outputStream* out) {
                   bool live_objects_only = true;   // default is true to retain the behavior before this change is made
                   outputStream* os = out;   // if path not specified or path is NULL, use out
                   fileStream* fs = NULL;
                   const char* arg0 = op->arg(0);
                -  uint parallel_thread_num = MAX(1, os::processor_count() * 3 / 8); // default is less than half of processors.
                +  uint parallel_thread_num = MAX(1, os::initial_active_processor_count() * 3 / 8); // default is less than half of processors.
                   if (arg0 != NULL && (strlen(arg0) > 0)) {
                     if (strcmp(arg0, "-all") != 0 && strcmp(arg0, "-live") != 0) {
                       out->print_cr("Invalid argument to inspectheap operation: %s", arg0);
                       return JNI_ERR;
                     }
                ###################################################

                Thanks.

                BRs,
               Lin

                On 2020/7/9, 3:22 PM, "linzang(??)" <linzang at tencent.com> wrote:

                    Hi Paul,
                        Thanks for reviewing!
                        >>
                        >>     I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp.
                        >>
                        The reason I made the change in Jmap.java that compose all arguments as 1 string , instead of passing 3 argments, is to avoid the compatibility issue, as we discussed in http://mail.openjdk.java.net/pipermail/serviceability-dev/2019-February/thread.html#27240.  The root cause of the compatibility issue is because max argument count in HotspotVirtualMachineImpl.java and attachlistener.cpp need to be enlarged (changes like http://hg.openjdk.java.net/jdk/jdk/rev/e7cf035682e3#l2.1) when jmap has more than 3 arguments. But if user use an old jcmd/jmap tool, it may stuck at socket read(), because the "max argument count" don't match.
                         I re-checked this change, the argument count of jmap histo is equal to 3 (live, file, parallel), so it can work normally even without the change of passing argument. But I think we have to face the problem if more arguments is added in jcmd alike tools later, not sure whether it should be sloved (or a workaround) in this changeset.

                        And here are the lastest webrev and delta:
                        http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06/
                        http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_06-delta/

                    Cheers,
                    Lin

                    On 2020/7/7, 5:57 AM, "Hohensee, Paul" <hohensee at amazon.com> wrote:

                        I'd like to see this feature added. :)

                        The CSR looks good, as does the basic parallel inspection algorithm. Stefan's done the GC part, so I'll stick to the non-GC part (fwiw, the GC part lgtm).

                        I'd move all the argument parsing code to JMap.java and just pass the results to Hotspot. Both histo() in JMap.java and code in attachListener.* parse the command line arguments, though the code in histo() doesn't parse the argument to "parallel". I'd upgrade the code in histo() to do a complete parse and pass the option values to executeCommandForPid as before: there would just be more of them now. That would allow you to eliminate all the parsing code in attachListener.cpp as well as the change to arguments.hpp.

                        heapInspection.hpp:

                        _shared_miss_count (s/b _missed_count, see below) isn't a size, so it should be a uint instead of a size_t. Same with the new parallel_thread_num argument to heap_inspection() and populate_table().

                        Comment copy-edit:
                        +// Parallel heap inspection task. Parallel inspection can fail due to
                        +// a native OOM when allocating memory for TL-KlassInfoTable.
                        +// _success will be set false on an OOM, and serial inspection tried.

                        _shared_miss_count should be _missed_count to match the missed_count() getter, or rename missed_count() to be shared_miss_count(). Whichever way you go, the field type should match the getter result type: uint is reasonable.

                        heapInspection.cpp:

                        You might use ResourceMark twice in populate_table, separately for the parallel attempt and the serial code. If the parallel attempt fails and available memory is low, it would be good to clean up the memory used by the parallel attempt before doing the serial code.

                        Style nit in KlassInfoTable::merge_entry(). I'd line up the definitions of k and elt, so "k" is even with "elt". And, because it's two lines shorter, I'd replace
                        +  } else {
                        +    return false;
                        +  }
                        with
                        +  return false;

                        KlassInfoTableMergeClosure.is_success() should be just success() (i.e., no "is_" prefix) because it's a getter.

                        I'd reorganize the code in populate_table() to make it more clear, vis (I changed _shared_missed_count to _missed_count)
                        +  if (cit.allocation_failed()) {
                        +    // fail to allocate memory, stop parallel mode
                        +    Atomic::store(&_success, false);
                        +    return;
                        +  }
                        +  RecordInstanceClosure ric(&cit, _filter);
                        +  _poi->object_iterate(&ric, worker_id);
                        +  missed_count = ric.missed_count();
                        +  {
                        +    MutexLocker x(&_mutex);
                        +    merge_success = _shared_cit->merge(&cit);
                        +  }
                        +  if (merge_success) {
                        +    Atomic::add(&_missed_count, missed_count);
                        +  else {
                        +    Atomic::store(&_success, false);
                        +  }

                        Thanks,
                        Paul

                        On 6/29/20, 7:20 PM, "linzang(??)" <linzang at tencent.com> wrote:

                            Dear All,
                                    Sorry to bother again, I just want to make sure that is this change worth to be continue to work on? If decision is made to not. I think I can drop this work and stop asking for help reviewing...
                                    Thanks for all your help about reviewing this previously.

                            BRs,
                            Lin

                            On 2020/5/9, 3:47 PM, "linzang(??)" <linzang at tencent.com> wrote:

                                Dear All,
                                       May I ask your help again for review the latest change?  Thanks!

                                BRs,
                                Lin

                                On 2020/4/28, 1:54 PM, "linzang(??)" <linzang at tencent.com> wrote:

                                    Hi Stefan,
                                      >>  - Adding Atomic::load/store.
                                      >>  - Removing the time measurement in the run_task. I renamed G1's function
                                      >>  to run_task_timed. If we need this outside of G1, we can rethink the API
                                      >>  at that point.
                                       >>  - ZGC style cleanups
                                       Thanks for revising the patch,  they are all good to me, and I have made a tiny change based on it:
                                           http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04/
                                           http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_04-delta/
                                      it reduce the scope of mutex in ParHeapInspectTask, and delete unnecessary comments.

                                    BRs,
                                    Lin

                                    On 2020/4/27, 4:34 PM, "Stefan Karlsson" <stefan.karlsson at oracle.com> wrote:

                                        Hi Lin,

                                        On 2020-04-26 05:10, linzang(??) wrote:
                                        > Hi Stefan and Paul?
                                        >      I have made a new patch based on your comments and Stefan's Poc code:
                                        >      Webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03/
                                        >      Delta(based on Stefan's change:) : http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_03-delta/webrev_03-delta/

                                        Thanks for providing a delta patch. It makes it much easier to look at,
                                        and more likely for reviewers to continue reviewing.

                                        I'm going to continue focusing on the GC parts, and leave the rest to
                                        others to review.

                                        >
                                        >      And Here are main changed I made and want to discuss with you:
                                        >      1.  changed"parallelThreadNum=" to "parallel=" for jmap -histo options.
                                        >      2.  Add logic to test where parallelHeapInspection is fail, in heapInspection.cpp
                                        >            This is because the parHeapInspectTask create thread local KlassInfoTable in it's work() method, and this may fail because of native OOM, in this case, the parallel should fail and serial heap inspection can be tried.
                                        >            One more thing I want discuss with you is about the member "_success" of parHeapInspectTask, when native OOM happenes, it is set to false. And since this "set" operation can be conducted in multiple threads, should it be atomic ops?  IMO, this is not necessary because "_success" can only be set to false, and there is no way to change it from back to true after the ParHeapInspectTask instance is created, so it is save to be non-atomic, do you agree with that?

                                        In these situations you should be using the Atomic::load/store
                                        primitives. We're moving toward a later C++ standard were data races are
                                       considered undefined behavior.

                                        >     3. make CollectedHeap::run_task() be an abstract virtual func, so that every subclass of collectedHeap should support it, so later implementation of new collectedHeap will not miss the "parallel" features.
                                        >           The problem I want to discuss with you is about epsilonHeap and SerialHeap, as they may not need parallel heap iteration, so I only make task->work(0), in case the run_task() is invoked someway in future. Another way is to left run_task()  unimplemented, which one do you think is better?

                                        I don't have a strong opinion about this.

                                          And also please help take a look at the zHeap, as there is a class
                                        zTask that wrap the abstractGangTask, and the collectedHeap::run_task()
                                        only accept  AbstraceGangTask* as argument, so I made a delegate class
                                        to adapt it , please see src/hotspot/share/gc/z/zHeap.cpp.
                                        >
                                        >        There maybe other better ways to sovle the above problems, welcome for any comments, Thanks!

                                        I've created a few cleanups and changes on top of your latest patch:

                                        https://cr.openjdk.java.net/~stefank/8215624/webrev.02.delta
                                        https://cr.openjdk.java.net/~stefank/8215624/webrev.02

                                        - Adding Atomic::load/store.
                                        - Removing the time measurement in the run_task. I renamed G1's function
                                        to run_task_timed. If we need this outside of G1, we can rethink the API
                                        at that point.
                                        - ZGC style cleanups

                                        Thanks,
                                        StefanK

                                        >
                                        > BRs,
                                        > Lin
                                        >
                                        > On 2020/4/23, 11:08 AM, "linzang(??)" <linzang at tencent.com> wrote:
                                        >
                                        >      Thanks Paul! I agree with using "parallel", will make the update in next patch, Thanks for help update the CSR.
                                        >
                                        >      BRs,
                                        >      Lin
                                        >
                                        >      On 2020/4/23, 4:42 AM, "Hohensee, Paul" <hohensee at amazon.com> wrote:
                                        >
                                        >          For the interface, I'd use "parallel" instead of "parallelThreadNum". All the other options are lower case, and it's a lot easier to type "parallel". I took the liberty of updating the CSR. If you're ok with it, you might want to change variable names and such, plus of course JMap.usage.
                                        >
                                        >          Thanks,
                                        >          Paul
                                        >
                                        >          On 4/22/20, 2:29 AM, "serviceability-dev on behalf of linzang(??)" <serviceability-dev-bounces at openjdk.java.net on behalf of linzang at tencent.com> wrote:
                                        >
                                        >              Dear Stefan,
                                        >
                                        >                      Thanks a lot! I agree with you to decouple the heap inspection code with GC's.
                                        >                      I will start  from your POC code, may discuss with you later.
                                        >
                                        >
                                        >              BRs,
                                        >              Lin
                                        >
                                        >              On 2020/4/22, 5:14 PM, "Stefan Karlsson" <stefan.karlsson at oracle.com> wrote:
                                        >
                                        >                  Hi Lin,
                                        >
                                        >                  I took a look at this earlier and saw that the heap inspection code is
                                        >                  strongly coupled with the CollectedHeap and G1CollectedHeap. I'd prefer
                                        >                  if we'd abstract this away, so that the GCs only provide a "parallel
                                        >                  object iteration" interface, and the heap inspection code is kept elsewhere.
                                        >
                                        >                  I started experimenting with doing that, but other higher-priority (to
                                        >                  me) tasks have had to take precedence.
                                        >
                                        >                  I've uploaded my work-in-progress / proof-of-concept:
                                        >                    https://cr.openjdk.java.net/~stefank/8215624/webrev.01.delta/
                                        >                    https://cr.openjdk.java.net/~stefank/8215624/webrev.01/
                                        >
                                        >                  The current code doesn't handle the lifecycle (deletion) of the
                                        >                  ParallelObjectIterators. There's also code left unimplemented in around
                                        >                  CollectedHeap::run_task. However, I think this could work as a basis to
                                        >                  pull out the heap inspection code out of the GCs.
                                        >
                                        >                  Thanks,
                                        >                  StefanK
                                        >
                                        >                  On 2020-04-22 02:21, linzang(??) wrote:
                                        >                  > Dear all,
                                        >                  >       May I ask you help to review? This RFR has been there for quite a while.
                                        >                  >       Thanks!
                                        >                  >
                                        >                  > BRs,
                                        >                  > Lin
                                        >                  >
                                        >                  > > On 2020/3/16, 5:18 PM, "linzang(??)" <linzang at tencent.com> wrote:>
                                        >                  >
                                        >                  >>    Just update a new path, my preliminary measure show about 3.5x speedup of jmap histo on a nearly full 4GB G1 heap (8-core platform with parallel thread number set to 4).
                                        >                  >>     webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_02/
                                        >                  >>     bug: https://bugs.openjdk.java.net/browse/JDK-8215624
                                        >                  >>     CSR: https://bugs.openjdk.java.net/browse/JDK-8239290
                                        >                  >>     BRs,
                                        >                  >>       Lin
                                        >                  >>       > On 2020/3/2, 9:56 PM, "linzang(??)" <linzang at tencent.com> wrote:
                                        >                  >>       >
                                        >                  >>       >    Dear all,
                                        >                  >>       >          Let me try to ease the reviewing work by some explanation :P
                                        >                  >>       >          The patch's target is to speed up jmap -histo for heap iteration, from my experience it is necessary for large heap investigation. E.g in bigData scenario I have tried to conduct jmap -histo against 180GB heap, it does take quite a while.
                                        >                  >>       >          And if my understanding is corrent, even the jmap -histo without "live" option does heap inspection with heap lock acquired. so it is very likely to block mutator thread in allocation-sensitive scenario. I would say the faster the heap inspection does, the shorter the mutator be blocked. This is parallel iteration for jmap is necessary.
                                        >                  >>       >          I think the parallel heap inspection should be applied to all kind of heap. However, consider the heap layout are different for  GCs, much time is required to understand all kinds of the heap layout to make the whole change. IMO, It is not wise to have a huge patch for the whole solution at once, and it is even harder to review it. So I plan to implement it incrementally, the first patch (this one) is going to confirm the implemention detail of how jmap accept the new option, passes it to attachListener of the jvm process and then how to make the parallel inspection closure be generic enough to make it easy to extend to different heap layout. And also how to implement the heap inspection in specific gc's heap. This patch use G1's heap as the begining.
                                        >                  >>       >          This patch actually do several things:
                                        >                  >>       >          1. Add an option "parallelThreadNum=<N>" to jmap -histo, the default behavior is to set N to 0, means let's JVM decide how many threads to use for heap inspection. Set this option to 1 will disable parallel heap inspection. (more details in CSR: https://bugs.openjdk.java.net/browse/JDK-8239290)
                                        >                  >>       >          2. Make a change in how Jmap passing arguments, changes in http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/src/jdk.jcmd/share/classes/sun/tools/jmap/JMap.java.udiff.html, originally it pass options as separate arguments to attachListener, this patch change to that all options be compose to a single string. So the arg_count_max in attachListener.hpp do not need to be changed, and hence avoid the compatibility issue, as disscussed at https://mail.openjdk.java.net/pipermail/serviceability-dev/2019-March/027334.html
                                        >                  >>       >         3. Add an abstract class ParHeapInspectTask in heapInspection.hpp / heapInspection.cpp, It's work(uint worker_id) method prepares the data structure (KlassInfoTable) need for every parallel worker thread, and then call do_object_iterate_parallel() which is heap specific implementation. I also added some machenism in KlassInfoTable to support parallel iteration, such as merge().
                                        >                  >>       >        4. In specific heap (G1 in this patch), create a subclass of ParHeapInspectTask, implement the do_object_iterate_parallel() for parallel heap inspection. For G1, it simply invoke g1CollectedHeap's object_iterate_parallel().
                                        >                  >>       >        5. Add related test.
                                        >                  >>       >        6. it may be easy to extend this patch for other kinds of heap by creating subclass of ParHeapInspectTask and implement the do_object_iterate_parallel().
                                        >                  >>       >
                                        >                  >>       >    Hope these info could help on code review and initate the discussion :-)
                                        >                  >>       >    Thanks!
                                        >                  >>       >
                                        >                  >>       >    BRs,
                                        >                  >>       >    Lin
                                        >                  >>       >    >On 2020/2/19, 9:40 AM, "linzang(??)" <linzang at tencent.com> wrote:.
                                        >                  >>       >    >
                                        >                  >>       >    >  Re-post this RFR with correct enhancement number to make it trackable.
                                        >                  >>       >    >  please ignore the previous wrong post. sorry for troubles.
                                        >                  >>       >    >
                                        >                  >>       >    >   webrev: http://cr.openjdk.java.net/~lzang/jmap-8214535/8215624/webrev_01/
                                        >                  >>       >    >    Hi bug: https://bugs.openjdk.java.net/browse/JDK-8215624
                                        >                  >>       >    >    CSR: https://bugs.openjdk.java.net/browse/JDK-8239290
                                        >                  >>       >    >    --------------
                                        >                  >>       >    >    Lin
                                        >                  >>       >    >    >Hi Lin,
                                        >                  >   >     >    >    >
                                        >                  >>       >    >    >Could you, please, re-post your RFR with the right enhancement number in
                                        >                  >>       >    >    >the message subject?
                                        >                  >>       >    >    >It will be more trackable this way.
                                        >                  >>       >    >    >
                                        >                  >>       >    >    >Thanks,
                                        >                  >>       >    >    >Serguei
                                        >                  >>       >    >    >
                                        >                  >>       >    >    >
                                        >                  >>       >    >    >On 2/17/20 10:29 PM, linzang(??) wrote:
                                        >                  >>       >    >    >> Dear David,
                                        >                  >>       >    >    >>        Thanks a lot!
                                        >                  >>       >    >    >>       I have updated the refined code to http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_01/.
                                        >                  >>       >    >    >>        IMHO the parallel heap inspection can be extended to all kinds of heap as long as the heap layout can support parallel iteration.
                                        >                  >>       >    >    >>        Maybe we can firstly use this webrev to discuss how to implement it, because I am not sure my current implementation is an appropriate way to communicate with collectedHeap, then we can extend the solution to other kinds of heap.
                                        >                  >>       >    >    >>
                                        >                  >>       >    >    >> Thanks,
                                        >                  >>       >    >    >> --------------
                                        >                  >>       >    >    >> Lin
                                        >                  >>       >    >    >>> Hi Lin,
                                        >                  >>       >    >    >>>
                                        >                  >>       >    >    >>> Adding in hotspot-gc-dev as they need to see how this interacts with GC
                                        >                  >>       >    >    >>> worker threads, and whether it needs to be extended beyond G1.
                                        >                  >>       >    >    >>>
                                        >                  >>       >    >   >>> I happened to spot one nit when browsing:
                                        >                  >>       >    >    >>>
                                        >                  >>       >    >    >>> src/hotspot/share/gc/shared/collectedHeap.hpp
                                        >                  >>       >    >    >>>
                                        >                  >>       >    >    >>> +   virtual bool run_par_heap_inspect_task(KlassInfoTable* cit,
                                        >                  >>       >    >    >>> +                                          BoolObjectClosure* filter,
                                        >                  >>       >    >    >>> +                                          size_t* missed_count,
                                        >                  >>       >    >    >>> +                                          size_t thread_num) {
                                        >                  >>       >    >    >>> +     return NULL;
                                        >                  >>       >    >    >>>
                                        >                  >>       >    >    >>> s/NULL/false/
                                        >                  >>       >    >    >>>
                                        >                  >>       >    >    >>> Cheers,
                                        >                  >>       >    >    >>> David
                                        >                  >   >     >    >    >>>
                                        >                  >>       >    >    >>> On 18/02/2020 2:15 pm, linzang(??) wrote:
                                        >                  >>       >    >    >>>> Dear All,
                                        >                  >>       >    >    >>>>         May I ask your help to review the follow changes:
                                        >                  >>       >    >    >>>>         webrev:
                                        >                  >>       >    >    >>>> http://cr.openjdk.java.net/~lzang/jmap-8214535/8215264/webrev_00/
                                        >                  >>       >    >    >>>>      bug: https://bugs.openjdk.java.net/browse/JDK-8215624
                                        >                  >>       >    >    >>>>      related CSR: https://bugs.openjdk.java.net/browse/JDK-8239290
                                        >                  >>       >    >    >>>>         This patch enable parallel heap inspection of G1 for jmap histo.
                                        >                  >>       >    >    >>>>         my simple test shown it can speed up 2x of jmap -histo with
                                        >                  >>       >    >    >>>> parallelThreadNum set to 2 for heap at ~500M on 4-core platform.
                                        >                  >>       >    >    >>>>
                                        >                  >>       >    >    >>>> ------------------------------------------------------------------------
                                        >                  >>       >    >    >>>> BRs,
                                        >                  >>       >    >    >>>> Lin
                                        >                  >>       >    >    >> >
                                        >                  >>       >    >    >
                                        >                  >
                                        >                  >
                                        >                  >
                                        >
                                        >
                                        >
                                        >
                                        >
                                        >












From ofirg6 at gmail.com  Thu Jul 30 07:07:35 2020
From: ofirg6 at gmail.com (Ofir Gordon)
Date: Thu, 30 Jul 2020 10:07:35 +0300
Subject: How to run specific part of the SerialGC in a new thread?
In-Reply-To: <3338c302-bcc4-2dc2-4e20-30d1aafeb48d@oracle.com>
References: <CAC9y=qCXmGkeHZgWwuHwk-ypMEf8iHRBkQmDqpgdHGckL1QweQ@mail.gmail.com>
 <afabcff3-8741-4c32-e4f8-1a560f92f5cc@oracle.com>
 <CAC9y=qAGjBMMsNfZzB2c=kMOLmRKfSZdWEHQT_Tkya_hMaKoZw@mail.gmail.com>
 <2b1a3443-ccbd-602d-ae4c-4cb97284b43d@oracle.com>
 <CAC9y=qCDKAdyKd05mV+2PCmUEF1coO49FLjEnoRODWCe1YguEQ@mail.gmail.com>
 <a01035c9-333b-0be8-ca27-bdaf9b70ee84@oracle.com>
 <3338c302-bcc4-2dc2-4e20-30d1aafeb48d@oracle.com>
Message-ID: <CAC9y=qAoeJR_=jLOPS4HpHuwYH4sPOKhnOOxgw0KzuEn8qRhOw@mail.gmail.com>

Hi,

It's a very late followup but I just got to work on it.
Thanks for the code example, it seems to work.
Is there a way to get the current thread's id, before activating the worker
and after activating it? Again, the normal cpp way using
std::this_thread::get_id() is not compiling (I can't seem to include
<thread> in the project..)
Also, is there a way to wait on a worker until it finishes? or does it
happen automatically since it initializes a "not concurrent" thread?

Thanks again,
Ofir

??????? ??? ??, 6 ????? 2020 ?-12:56 ??? ?Thomas Schatzl?? <?
thomas.schatzl at oracle.com??>:?

> Hi,
>
> On 06.07.20 11:13, Thomas Schatzl wrote:
> > Hi,
> >
> >    sorry for the late answer, I started writing but then forgot to send
> > the email. :(
> >
> > On 30.06.20 10:27, Ofir Gordon wrote:
> >> I'll try to explain my end goal:
>
> [...]
>
> > WorkGang* w = new WorkGang("My Work Gang", 1, true, false);
> > w->initialize();
>
> A w->update_active_workers(1); may be missing here I think.
>
> Thanks,
>    Thomas
>


From zgu at redhat.com  Thu Jul 30 17:55:01 2020
From: zgu at redhat.com (Zhengyu Gu)
Date: Thu, 30 Jul 2020 13:55:01 -0400
Subject: [15] RFR 8250841: Shenandoah: need to reset/finish dead counters for
 StringTable/ResolvedMethodTable during STW root processing
Message-ID: <762e60dc-3f1b-50f5-9e7a-0b70572f5007@redhat.com>

Please review this small patch that triggers 
StringTable/ResolvedMethodTable cleaning and resizing after STW processing.

Without this fix, dead entries may accumulate in the tables, result long 
pause time.


Bug: https://bugs.openjdk.java.net/browse/JDK-8250841
Webrev: http://cr.openjdk.java.net/~zgu/JDK-8250841/webrev.00/

Test:
   hotspot_gc_shenandoah

Thanks,

-Zhengyu



From shade at redhat.com  Thu Jul 30 18:15:05 2020
From: shade at redhat.com (Aleksey Shipilev)
Date: Thu, 30 Jul 2020 20:15:05 +0200
Subject: [15] RFR 8250841: Shenandoah: need to reset/finish dead counters
 for StringTable/ResolvedMethodTable during STW root processing
In-Reply-To: <762e60dc-3f1b-50f5-9e7a-0b70572f5007@redhat.com>
References: <762e60dc-3f1b-50f5-9e7a-0b70572f5007@redhat.com>
Message-ID: <44a74615-f044-d24e-8c30-568f4e29dd62@redhat.com>

On 7/30/20 7:55 PM, Zhengyu Gu wrote:
> Bug: https://bugs.openjdk.java.net/browse/JDK-8250841
> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8250841/webrev.00/

Looks fine to me.

-- 
Thanks,
-Aleksey


From zgu at redhat.com  Thu Jul 30 18:20:07 2020
From: zgu at redhat.com (Zhengyu Gu)
Date: Thu, 30 Jul 2020 14:20:07 -0400
Subject: [15] RFR 8250841: Shenandoah: need to reset/finish dead counters
 for StringTable/ResolvedMethodTable during STW root processing
In-Reply-To: <44a74615-f044-d24e-8c30-568f4e29dd62@redhat.com>
References: <762e60dc-3f1b-50f5-9e7a-0b70572f5007@redhat.com>
 <44a74615-f044-d24e-8c30-568f4e29dd62@redhat.com>
Message-ID: <f7e056a9-cc02-3871-8b64-b71cf059d01f@redhat.com>

Thanks, Aleksey.

-Zhengyu

On 7/30/20 2:15 PM, Aleksey Shipilev wrote:
> On 7/30/20 7:55 PM, Zhengyu Gu wrote:
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8250841
>> Webrev: http://cr.openjdk.java.net/~zgu/JDK-8250841/webrev.00/
> 
> Looks fine to me.
>