From rkennke at openjdk.java.net  Mon Mar  1 10:14:06 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 1 Mar 2021 10:14:06 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v3]
In-Reply-To: <4k7tZKJnbD_gHujh7Bn_aD0OxVachkbWih4TQBQ3EgM=.f68ea4fd-0126-47fc-9961-a07af8110725@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
 <ILaBVTmuFsrazdAgAs_hSFZDBv5e6Mtn8Qcxe7BgY30=.618ebfc6-1ece-4a5b-bff6-f958195d46eb@github.com>
 <l-VHrE-jVIxT7S4JuxOaXtP7diS3evQH-X-5R68vLFM=.850645a7-a032-4713-8c4c-48db3c07ff09@github.com>
 <pz_0hGoHbW9OuJ0JRPX5pas7J1pJIxk958LY2ip_3i4=.e329c755-8648-4f63-9b1e-1f531d7853e1@github.com>
 <qEqHsbku4_xUVeWpICqdJqRI4AumQdyD1XZ_IhRaaFs=.c304e31f-6fce-47f6-ba00-5596a8ee654c@github.com>
 <9Kxs_Fp-b2APxU4tU7zSHavQmCfbPMHOhGbNj26rvgQ=.2eb77974-6d40-4b0f-8254-73f67d78e18d@github.com>
 <jSpfLGAeyp28YbREwQXGmLCuejIoqstr34K1Mde9xTo=.3eb9e8e2-0518-4b4a-becf-74eec95a6dd3@github.com>
 <4k7tZKJnbD_gHujh7Bn_aD0OxVachkbWih4TQBQ3EgM=.f68ea4fd-0126-47fc-9961-a07af8110725@github.com>
Message-ID: <XjLsYtHD3rUpPXi3VYwo_sZ63GomOk_ln9dHlKbvQ7Y=.20ba2775-2e90-4aa8-9eda-e694f05975d3@github.com>

On Fri, 26 Feb 2021 20:16:57 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:

>> You should enable GitHub actions in your fork under Settings->Actions->Allow all actions. Then you'll get presubmit-tests which runs a whole bunch of stuff on various platforms. A great sanity check.
>
> Yes, it's looking much more stable now. Hmm, I have actions enabled and I see it running builds on different platforms. I've been abusing this branch/pull request to fix windows and aarch64 builds.

It seems that 'RegisterReferences' is a develop flag and does not exist in release build, which means it cannot be changed by the generational mode (it results in crash). If reference processing really is a concern, then the way out would be to introduce our own flag, e.g. ShenandoahReferenceProcessing, and use that (and set RegisterReferences accordingly in non-release builds). OR, maybe even better, put a method should_process_references() in ShenandoahHeurisitics and override that to return false. OTOH, I just tried a bunch of SPECjvm workloads, and they seem to run fine without disabling reference processing (until they attempt a full-GC which is not yet supported).

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19

From shade at openjdk.java.net  Mon Mar  1 11:01:50 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 1 Mar 2021 11:01:50 GMT
Subject: RFR: 8262122: [TESTBUG] Shenandoah-specific variant of
 TestReferenceRefersTo
In-Reply-To: <izReswRQOulMVo0iSU9-qthYE1Opi5K2L8jfo6Y-V5I=.d97f152a-b99f-4ef3-8095-9eb3a2afa68f@github.com>
References: <izReswRQOulMVo0iSU9-qthYE1Opi5K2L8jfo6Y-V5I=.d97f152a-b99f-4ef3-8095-9eb3a2afa68f@github.com>
Message-ID: <CjVERtgf4lvfASjJhYlLbw2zZY_6MWRSi5apcgJ6Mo4=.45f8ae55-2065-428e-8523-4abbebb1d5ff@github.com>

On Mon, 22 Feb 2021 11:52:25 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

> Before JDK-8262049, the test TestReferenceRefersTo.java has been failing with I-U mode, because it asserted that weak references would not be cleared when accessed during mark. JDK-8262049 split up the test into a generic part that removed the offending test, and a non-Shenandoah part that contains the test.
> I think it would be useful to add the full test with Shenandoah runners under gc/shenandoah to include it in hotspot_gc_shenandoah runs.
> 
> Test:
>  - [x] TestReferenceRefersToShenandoah.java
>  - [ ] hotspot_gc_shenandoah

Marked as reviewed by shade (Reviewer).

-------------

PR: https://git.openjdk.java.net/jdk/pull/2674

From shade at redhat.com  Mon Mar  1 11:42:35 2021
From: shade at redhat.com (Aleksey Shipilev)
Date: Mon, 1 Mar 2021 12:42:35 +0100
Subject: [8u] RFC: Pick up recent aarch64-port cleanups
Message-ID: <f79e70de-0765-b3da-aa93-2035f3ef5021@redhat.com>

There are a few code improvements in aarch64-port that we want to pick up and test before proposing 
the integration back. The pickup is clean, only affects hotspot repository, and gets merged 
automatically.

Testing: hotspot_gc_shenandoah {fastdebug,release}

-- 
Thanks,
-Aleksey


From jbachorik at openjdk.java.net  Mon Mar  1 13:15:49 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 1 Mar 2021 13:15:49 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
In-Reply-To: <sfCa2-VPnTrwD7wqUjbnqswtEQ-qY0VhmhntJVo62jo=.c1af3d13-92f7-4be9-a7ae-6c21b3feeeca@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <sfCa2-VPnTrwD7wqUjbnqswtEQ-qY0VhmhntJVo62jo=.c1af3d13-92f7-4be9-a7ae-6c21b3feeeca@github.com>
Message-ID: <H8JXgQJxlEzHqmmT5LUFXQGa7d2gLLw-tmiEXcSaR9k=.d1d92350-7ccb-4869-926e-9762cf1a8d38@github.com>

On Thu, 18 Feb 2021 10:18:03 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
>> 
>> ## Introducing new JFR event
>> 
>> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
>> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
>> 
>> ## Implementation
>> 
>> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
>> 
>> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
>> 
>> ### Epsilon GC
>> 
>> Trivial implementation - just return `used()` instead.
>> 
>> ### Serial GC
>> 
>> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
>> 
>> ### Parallel GC
>> 
>> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
>> 
>> ### G1 GC
>> 
>> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
>> 
>> ### Shenandoah
>> 
>> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
>> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
>> 
>> ### ZGC
>> 
>> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.
>
> src/hotspot/share/gc/shared/genCollectedHeap.cpp line 1144:
> 
>> 1142:   _old_gen->prepare_for_compaction(&cp);
>> 1143:   _young_gen->prepare_for_compaction(&cp);
>> 1144: 
> 
> Stray newline?

??

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar  1 13:15:48 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 1 Mar 2021 13:15:48 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
In-Reply-To: <rzB2tJzqS7HU8i2muvbqPNIcQ_qR7DF3ohoW2jQp6cc=.86af7cec-e896-4711-a68e-fc160531bacf@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <sfCa2-VPnTrwD7wqUjbnqswtEQ-qY0VhmhntJVo62jo=.c1af3d13-92f7-4be9-a7ae-6c21b3feeeca@github.com>
 <rzB2tJzqS7HU8i2muvbqPNIcQ_qR7DF3ohoW2jQp6cc=.86af7cec-e896-4711-a68e-fc160531bacf@github.com>
Message-ID: <30TKeg0tEzRXVOG5fcZp_Zsb9dTxU__Cn1VnpUUZJY4=.d7efe1f7-ebec-4b22-a5ac-9ba4b3d732eb@github.com>

On Mon, 22 Feb 2021 16:50:48 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

>> src/hotspot/share/gc/g1/g1CollectedHeap.cpp line 4578:
>> 
>>> 4576: 
>>> 4577: void G1CollectedHeap::set_live(size_t bytes) {
>>> 4578:   Atomic::release_store(&_live_size, bytes);
>> 
>> I don't think this requires `release_store`, regular `store` would be enough. G1 folks can say for sure.
>
> Not required.

??

>> src/hotspot/share/gc/shared/genCollectedHeap.hpp line 183:
>> 
>>> 181:     size_t live = _live_size;
>>> 182:     return live > 0 ? live : used();
>>> 183:   };
>> 
>> I think the implementation belongs to `genCollectedHeap.cpp`.
>
> +1. Does not seem to be performance sensitive.

??

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar  1 14:06:49 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 1 Mar 2021 14:06:49 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
In-Reply-To: <sfCa2-VPnTrwD7wqUjbnqswtEQ-qY0VhmhntJVo62jo=.c1af3d13-92f7-4be9-a7ae-6c21b3feeeca@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <sfCa2-VPnTrwD7wqUjbnqswtEQ-qY0VhmhntJVo62jo=.c1af3d13-92f7-4be9-a7ae-6c21b3feeeca@github.com>
Message-ID: <cCN32OP9x_FvbkSVKdkwHlvpeBojoqOKZ8146CmGTN8=.7b6b7d67-8711-4b5b-a1ec-e615b4925684@github.com>

On Thu, 18 Feb 2021 10:19:31 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
>> 
>> ## Introducing new JFR event
>> 
>> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
>> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
>> 
>> ## Implementation
>> 
>> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
>> 
>> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
>> 
>> ### Epsilon GC
>> 
>> Trivial implementation - just return `used()` instead.
>> 
>> ### Serial GC
>> 
>> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
>> 
>> ### Parallel GC
>> 
>> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
>> 
>> ### G1 GC
>> 
>> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
>> 
>> ### Shenandoah
>> 
>> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
>> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
>> 
>> ### ZGC
>> 
>> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.
>
> src/hotspot/share/gc/shared/generation.hpp line 140:
> 
>> 138:   virtual size_t used() const = 0;      // The number of used bytes in the gen.
>> 139:   virtual size_t free() const = 0;      // The number of free bytes in the gen.
>> 140:   virtual size_t live() const = 0;
> 
> Needs a comment to match the lines above? Say, `// The estimate of live bytes in the gen.`

??

> src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 579:
> 
>> 577:     event.set_heapLive(heap->live());
>> 578:     event.commit();
>> 579:   }
> 
> On the first sight, this belongs in `ShenandoahConcurrentMark::finish_mark()`. Placing the event here would fire the event when concurrent GC is cancelled, which is not what you want.

??

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From shade at openjdk.java.net  Mon Mar  1 14:21:40 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 1 Mar 2021 14:21:40 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
In-Reply-To: <V1O-9sR2sGh-d-pUwda_dOA-cTLQbCgrfrGrrgJhcPI=.cabebed8-6561-42c7-80ca-7d9fed043a81@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <V1O-9sR2sGh-d-pUwda_dOA-cTLQbCgrfrGrrgJhcPI=.cabebed8-6561-42c7-80ca-7d9fed043a81@github.com>
Message-ID: <vIclSQGpJVJWSb0jmbU-GREJoGJoUy_pNtYtBB5ry4Y=.5f4825b9-8319-4c5b-a9b5-4175d2338db3@github.com>

On Mon, 22 Feb 2021 17:20:49 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

>> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
>> 
>> ## Introducing new JFR event
>> 
>> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
>> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
>> 
>> ## Implementation
>> 
>> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
>> 
>> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
>> 
>> ### Epsilon GC
>> 
>> Trivial implementation - just return `used()` instead.
>> 
>> ### Serial GC
>> 
>> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
>> 
>> ### Parallel GC
>> 
>> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
>> 
>> ### G1 GC
>> 
>> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
>> 
>> ### Shenandoah
>> 
>> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
>> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
>> 
>> ### ZGC
>> 
>> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.
>
> The change also misses liveness update after G1 Full GC: it should at least reset the internal liveness counter to 0 so that `used()` is used.
> I think there is the same issue for Parallel Full GC. Serial seems to be handled.

Another general comment about Shenandoah. It would seem easier to piggyback liveness summarization on region iteration that heuristics does at the end of mark anyway. See `ShenandoahHeuristics::choose_collection_set`. I can do that when you are done with your changes, or try it yourself.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar  1 14:21:42 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 1 Mar 2021 14:21:42 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
In-Reply-To: <sfCa2-VPnTrwD7wqUjbnqswtEQ-qY0VhmhntJVo62jo=.c1af3d13-92f7-4be9-a7ae-6c21b3feeeca@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <sfCa2-VPnTrwD7wqUjbnqswtEQ-qY0VhmhntJVo62jo=.c1af3d13-92f7-4be9-a7ae-6c21b3feeeca@github.com>
Message-ID: <fW4pwshfwyWEzo6FkQnFFfokt-Qv8MTvBPv6OcgZKDQ=.e719c2f9-9407-483b-aaec-f7382ee9667c@github.com>

On Thu, 18 Feb 2021 10:22:58 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
>> 
>> ## Introducing new JFR event
>> 
>> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
>> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
>> 
>> ## Implementation
>> 
>> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
>> 
>> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
>> 
>> ### Epsilon GC
>> 
>> Trivial implementation - just return `used()` instead.
>> 
>> ### Serial GC
>> 
>> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
>> 
>> ### Parallel GC
>> 
>> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
>> 
>> ### G1 GC
>> 
>> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
>> 
>> ### Shenandoah
>> 
>> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
>> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
>> 
>> ### ZGC
>> 
>> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.
>
> src/hotspot/share/gc/shenandoah/shenandoahConcurrentMark.cpp line 265:
> 
>> 263:   ShenandoahHeap* const heap = ShenandoahHeap::heap();
>> 264:   heap->set_concurrent_mark_in_progress(false);
>> 265:   heap->mark_finished();
> 
> Let's not rename this method. Introduce a new method, `ShenandoahHeap::update_live`, and call it every time after `mark_complete_marking_context()` is called.

??

> src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 494:
> 
>> 492:   mark_complete_marking_context();
>> 493: 
>> 494:   class ShenandoahCollectLiveSizeClosure : public ShenandoahHeapRegionClosure {
> 
> We don't usually use the in-method declarations like these, pull it out of the method.

??

> src/hotspot/share/gc/epsilon/epsilonHeap.hpp line 80:
> 
>> 78:   virtual size_t capacity()     const { return _virtual_space.committed_size(); }
>> 79:   virtual size_t used()         const { return _space->used(); }
>> 80:   virtual size_t live()         const { return used(); }
> 
> I'd prefer to call `_space->used()` directly here. Minor optimization, I know.

??

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar  1 14:21:43 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 1 Mar 2021 14:21:43 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
In-Reply-To: <prkl3DH3V2chk4WeTTSnd0J_fSQ-9MD2w0biDMYLLHs=.1ed9f07d-5a98-4f35-bb30-30d22a3178b6@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <prkl3DH3V2chk4WeTTSnd0J_fSQ-9MD2w0biDMYLLHs=.1ed9f07d-5a98-4f35-bb30-30d22a3178b6@github.com>
Message-ID: <DvEihJiDWDlOdpYeT0k8mG7qJYfJCHSJbrV5gNoQJFs=.c3e4bca9-6afa-43c1-92a7-5bc662995675@github.com>

On Mon, 22 Feb 2021 08:44:25 GMT, Per Liden <pliden at openjdk.org> wrote:

>> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
>> 
>> ## Introducing new JFR event
>> 
>> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
>> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
>> 
>> ## Implementation
>> 
>> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
>> 
>> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
>> 
>> ### Epsilon GC
>> 
>> Trivial implementation - just return `used()` instead.
>> 
>> ### Serial GC
>> 
>> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
>> 
>> ### Parallel GC
>> 
>> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
>> 
>> ### G1 GC
>> 
>> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
>> 
>> ### Shenandoah
>> 
>> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
>> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
>> 
>> ### ZGC
>> 
>> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.
>
> src/hotspot/share/gc/z/zStat.hpp line 549:
> 
>> 547:   static size_t used_at_mark_start();
>> 548:   static size_t used_at_relocate_end();
>> 549:   static size_t live();
> 
> Please call this `live_at_mark_end()` to match the names of the neighboring functions.

??

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar  1 14:21:43 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 1 Mar 2021 14:21:43 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
In-Reply-To: <y_QIIisdV7CE1lKbYf8SxNwhEQ2Kcla8BtVI9M6pGtU=.d6ecb9b9-afc8-437c-9744-29bed3e89184@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <y_QIIisdV7CE1lKbYf8SxNwhEQ2Kcla8BtVI9M6pGtU=.d6ecb9b9-afc8-437c-9744-29bed3e89184@github.com>
Message-ID: <ilMNIbRMfydXhyE8GiBA9hbVYEJXrPDnDZ2m0nv5aow=.6b8ccb9e-435a-4fe4-a481-687468a6a007@github.com>

On Fri, 19 Feb 2021 08:21:36 GMT, Albert Mingkun Yang <ayang at openjdk.org> wrote:

>> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
>> 
>> ## Introducing new JFR event
>> 
>> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
>> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
>> 
>> ## Implementation
>> 
>> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
>> 
>> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
>> 
>> ### Epsilon GC
>> 
>> Trivial implementation - just return `used()` instead.
>> 
>> ### Serial GC
>> 
>> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
>> 
>> ### Parallel GC
>> 
>> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
>> 
>> ### G1 GC
>> 
>> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
>> 
>> ### Shenandoah
>> 
>> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
>> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
>> 
>> ### ZGC
>> 
>> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.
>
> src/hotspot/share/jfr/periodic/jfrPeriodic.cpp line 649:
> 
>> 647: TRACE_REQUEST_FUNC(HeapUsageSummary) {
>> 648:   EventHeapUsageSummary event;
>> 649:   if (event.should_commit()) {
> 
> I believe the `should_commit` check is not needed; the period check is handle by the caller.

??

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From rkennke at openjdk.java.net  Mon Mar  1 14:23:14 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 1 Mar 2021 14:23:14 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v14]
In-Reply-To: <7CiRCYgYXysI9lWIUUayxOqdXZAZ7alOWWtNXMMDDjQ=.6020f939-907e-4790-9c2b-93230c41dcb8@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
 <7CiRCYgYXysI9lWIUUayxOqdXZAZ7alOWWtNXMMDDjQ=.6020f939-907e-4790-9c2b-93230c41dcb8@github.com>
Message-ID: <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>

On Sat, 27 Feb 2021 01:42:18 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:

>> **This is a work in progress.**
>> 
>> ## Summary of changes
>> The goal of these changes is to allow young cycles to suspend marking in the old generation. When the young cycle is complete, the concurrent old marking will resume.
>> * Evaluation of heuristics was pulled into a new `ShenandoahRegulatorThread`. Taking this evaluation out of line from the control thread allows the regulator to suspend (using the cancellation mechanism) old generation marking and start a young generation cycle.
>> * Task queues for marking have been moved from `ShenandoahMarkingContext` into `ShenandoahGeneration`. This allows the marking state for the old generation to persist across young generation cycles. The associated `is_complete` state has also been moved to `ShenandoahGeneration`.
>> * Old generation marking is bootstrapped by a complete young generation cycle. In this scenario, the mark closures for a young cycle are given reference to the young generation mark queues _and_ the old generation mark queues. Rather than ignore old references as is done for a normal young cycle, they are enqueued in the old generation mark queues. When the young cycle completes, the old generation marking continues using the task queues primed by the preceding young cycle.
>> * There is a new flag: `ShenandoahAllowOldMarkingPreemption` (defaults to true). Disabling this option will cause the regulator to schedule  `young` or `global` collects (according to heuristics), but will _not_ schedule `old` collects.
>> * The `global` generation is used to support satb and incremental-update modes. The `global` generation is also used for degenerated and implicit/explicit gc requests. Degenerated cycles are not working on this branch and the root cause is understood.
>> * The `global` generation is also used for `FullGC`, but this is also broken. The `FullGC` doesn't update the remembered set during compaction. We reckon there is a non-trivial amount of work to fix this.
>> * The `MARKING` gc state has been split into `YOUNG_MARKING` and `OLD_MARKING`.
>> * Immediate garbage collection is broken in generational mode. The update references phase is used to repair the remembered set, so when this phase is skipped the remembered set scan runs into trouble. A fix for this is in progress.
>> * The remembered set is scanned on a safepoint during initial mark. Work to make this concurrent is in progress.
>
> earthling-amzn has updated the pull request incrementally with three additional commits since the last revision:
> 
>  - Encode active generation and old generation marking status
>  - Do not unset preemption request flag prematurely
>  - Disable reference processing generational mode

That is a huge change! Good work!
First-pass review follows.

(I need to remember to scan for 'HEY!' in addition to 'TODO' ;-) )

src/hotspot/cpu/aarch64/gc/shenandoah/c1/shenandoahBarrierSetC1_aarch64.cpp line 32:

> 30: #include "gc/shenandoah/shenandoahBarrierSetAssembler.hpp"
> 31: #include "gc/shenandoah/c1/shenandoahBarrierSetC1.hpp"
> 32: #include "gc/shenandoah/mode/shenandoahMode.hpp"

The include is unused, isn't it?

src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp line 64:

> 62:         __ tbz(rscratch1, ShenandoahHeap::HAS_FORWARDED_BITPOS, done);
> 63:       } else {
> 64:         __ mov(rscratch2, ShenandoahHeap::HAS_FORWARDED | ShenandoahHeap::OLD_MARKING);

In the x86 variant you preserve YOUNG_MARKING there.

src/hotspot/share/gc/shenandoah/mode/shenandoahGenerationalMode.cpp line 46:

> 44:   // HEY! Disabled while M7 work is in progress.
> 45:   FLAG_SET_ERGO(ShenandoahUnloadClassesFrequency, 0);
> 46:   FLAG_SET_ERGO(RegisterReferences, false);

This doesn't work in release build, unfortunately.

src/hotspot/share/gc/shenandoah/mode/shenandoahMode.hpp line 28:

> 26: #define SHARE_GC_SHENANDOAH_MODE_SHENANDOAHMODE_HPP
> 27: 
> 28: #include "runtime/java.hpp"

What is that include needed for?

src/hotspot/share/gc/shenandoah/shenandoahArguments.cpp line 159:

> 157:   }
> 158: 
> 159:   if (strcmp(ShenandoahGCMode, "generational") == 0) {

I guess this could be moved into ShenandoahGenerationalMode::initialize_flags()?

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 193:

> 191: }
> 192: #endif
> 193: 

This is probably better placed in a more general place? I can imagine it would be used in other places too? (Maybe in ShenandoahRegionAffiliation itself?)

src/hotspot/share/gc/shenandoah/shenandoahGeneration.cpp line 2:

> 1: /*
> 2:  * Copyright (c) 2001, 2018, Oracle and/or its affiliates. All rights reserved.

That copyright seems wrong.

src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 189:

> 187:   _cancelled_gc.set(CANCELLABLE);
> 188:   if (clear_oom_handler) {
> 189:     _oom_evac_handler.clear();

This change seems a bit suspicious to me. Why do we need to clear the OOM evac handler here?

src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 323:

> 321:     // we have to keep the fwdptr initialized and pointing to our (stale) copy.
> 322: 
> 323: 

Stray spacing changes here...

src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 342:

> 340:   if (_affiliation == FREE) {
> 341:     //assert(_live_data == 0, "Setting non-zero live data (%zu) on FREE region", s);
> 342:   }

Is this commented-out assert relevant? If not, maybe remove it?

src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 423:

> 421: }
> 422: 
> 423: void ShenandoahHeapRegion::oop_iterate_objects(OopIterateClosure* blk, bool fill_dead_objects, bool reregister_coalesced_objects) {

I wonder if this is better done as two separate methods: the normal/old oop_iterate_objects() and a new one that also fills dead objects?

src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 524:

> 522:   printf("SHR::recycle(), setting region (%llx, %llx, %llx) to FREE\n",
> 523:          (unsigned long long) bottom(), (unsigned long long) top(), (unsigned long long) end());
> 524:   fflush(stdout);

In Hotspot, it's simpler to use tty->print_cr("...") for this, or even better use logging e.g. log_trace(gc)("...") or so.

src/hotspot/share/gc/shenandoah/shenandoahMarkClosures.cpp line 1:

> 1: #include "precompiled.hpp"

It's lacking a copyright header here.

src/hotspot/share/gc/shenandoah/shenandoahMarkClosures.hpp line 2:

> 1: /*
> 2:  * Copyright (c) 2020, Red Hat, Inc. All rights reserved.

And this copyright header has the wrong year in it :-)

src/hotspot/share/gc/shenandoah/shenandoahMarkClosures.hpp line 35:

> 33: #include "gc/shenandoah/shenandoahMarkingContext.hpp"
> 34: #include "gc/shenandoah/shenandoahMarkClosures.hpp"
> 35: 

Instead of including everything, we can put forward declaration for any type that is only used as pointer in the header.

src/hotspot/share/gc/shenandoah/shenandoahOldGC.cpp line 1:

> 1: #include "precompiled.hpp"

Missing copyright header here.

src/hotspot/share/gc/shenandoah/shenandoahOldGC.hpp line 1:

> 1: #ifndef SHARE_GC_SHENANDOAH_SHENANDOAHOLDGC_HPP

And here too.

src/hotspot/share/gc/shenandoah/shenandoahOldGC.hpp line 7:

> 5: #include "gc/shenandoah/shenandoahConcurrentMark.hpp"
> 6: #include "gc/shenandoah/shenandoahConcurrentGC.hpp"
> 7: #include "gc/shenandoah/shenandoahHeap.hpp"

I see no use of ShenandoahHeap or ShenandoahConcurrentMark in this header. I believe those two includes could be removed.

src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 1:

> 1: #include "precompiled.hpp"

Missing header.

src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.hpp line 1:

> 1: 

Missing header

src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.hpp line 6:

> 4: 
> 5: #include "gc/shenandoah/shenandoahGeneration.hpp"
> 6: 

You should add a forward declaration of ShenandoahHeapRegion and ShenandoahHeapRegionClosure.

src/hotspot/share/gc/shenandoah/shenandoahPhaseTimings.hpp line 103:

> 101:                                                                                        \
> 102:   f(init_update_refs_gross,                         "Pause Init Update Refs (G)")      \
> 103:   f(init_update_refs,                               "Pause Init Update Refs (N)")      \

Looks like unrelated spacing changes

src/hotspot/share/gc/shenandoah/shenandoahRegulatorThread.cpp line 2:

> 1: /*
> 2:  * Copyright (c), Amazon.com, Inc. and/or its affiliates. All rights reserved.

You should add the copyright year (2021) here.

src/hotspot/share/gc/shenandoah/shenandoahScanRemembered.hpp line 28:

> 26: // During development of this new feature, we want the option to test
> 27: // with and without, and to compare performance before and after.
> 28: #define FAST_REMEMBERED_SET_SCANNING

You'll still want the normal #ifndef SHARE_GC_SHENANDOAH_SHENANDOAHSCANREMEMBERED_HPP protection, don't you?

-------------

Changes requested by rkennke (Reviewer).

PR: https://git.openjdk.java.net/shenandoah/pull/19

From jbachorik at openjdk.java.net  Mon Mar  1 14:27:41 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 1 Mar 2021 14:27:41 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
In-Reply-To: <V1O-9sR2sGh-d-pUwda_dOA-cTLQbCgrfrGrrgJhcPI=.cabebed8-6561-42c7-80ca-7d9fed043a81@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <V1O-9sR2sGh-d-pUwda_dOA-cTLQbCgrfrGrrgJhcPI=.cabebed8-6561-42c7-80ca-7d9fed043a81@github.com>
Message-ID: <6L-2nr_YRy2MikNW7fhxblbLswAZD3L05fliGk36WTM=.822e958d-3cc1-43cd-88fb-d4e6bbbed012@github.com>

On Mon, 22 Feb 2021 17:12:43 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

>> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
>> 
>> ## Introducing new JFR event
>> 
>> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
>> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
>> 
>> ## Implementation
>> 
>> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
>> 
>> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
>> 
>> ### Epsilon GC
>> 
>> Trivial implementation - just return `used()` instead.
>> 
>> ### Serial GC
>> 
>> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
>> 
>> ### Parallel GC
>> 
>> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
>> 
>> ### G1 GC
>> 
>> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
>> 
>> ### Shenandoah
>> 
>> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
>> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
>> 
>> ### ZGC
>> 
>> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.
>
> src/hotspot/share/gc/shared/genCollectedHeap.cpp line 683:
> 
>> 681:   }
>> 682:   // update the live size after last GC
>> 683:   _live_size = _young_gen->live() + _old_gen->live();
> 
> I would prefer if that code were placed into `gc_epilogue`.

??

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar  1 15:11:41 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 1 Mar 2021 15:11:41 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
In-Reply-To: <cCN32OP9x_FvbkSVKdkwHlvpeBojoqOKZ8146CmGTN8=.7b6b7d67-8711-4b5b-a1ec-e615b4925684@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <sfCa2-VPnTrwD7wqUjbnqswtEQ-qY0VhmhntJVo62jo=.c1af3d13-92f7-4be9-a7ae-6c21b3feeeca@github.com>
 <cCN32OP9x_FvbkSVKdkwHlvpeBojoqOKZ8146CmGTN8=.7b6b7d67-8711-4b5b-a1ec-e615b4925684@github.com>
Message-ID: <gzT3m7wNBC-SGDwyKlxjaYxFkgbj4q-TzdYrkE_LZKg=.a8b6e422-d940-4fe5-adc7-72b7cda0d296@github.com>

On Mon, 1 Mar 2021 14:03:37 GMT, Jaroslav Bachorik <jbachorik at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 579:
>> 
>>> 577:     event.set_heapLive(heap->live());
>>> 578:     event.commit();
>>> 579:   }
>> 
>> On the first sight, this belongs in `ShenandoahConcurrentMark::finish_mark()`. Placing the event here would fire the event when concurrent GC is cancelled, which is not what you want.
>
> ??

Actually, this shouldn't even be here. `EventGCHeapSummary` is emitted via `trace_heap*` calls which should already be hooked into Shenandoah. Let me remove this.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar  1 15:27:24 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 1 Mar 2021 15:27:24 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v2]
In-Reply-To: <vIclSQGpJVJWSb0jmbU-GREJoGJoUy_pNtYtBB5ry4Y=.5f4825b9-8319-4c5b-a9b5-4175d2338db3@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <V1O-9sR2sGh-d-pUwda_dOA-cTLQbCgrfrGrrgJhcPI=.cabebed8-6561-42c7-80ca-7d9fed043a81@github.com>
 <vIclSQGpJVJWSb0jmbU-GREJoGJoUy_pNtYtBB5ry4Y=.5f4825b9-8319-4c5b-a9b5-4175d2338db3@github.com>
Message-ID: <8Q_OEFTu-Npp_Pr74VPWQ6DrQUzcaDtVZy7nru2RCbU=.7e80a5b9-be94-4442-beaf-9593b067241d@github.com>

On Mon, 1 Mar 2021 14:17:17 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> The change also misses liveness update after G1 Full GC: it should at least reset the internal liveness counter to 0 so that `used()` is used.
>> I think there is the same issue for Parallel Full GC. Serial seems to be handled.
>
> Another general comment about Shenandoah. It would seem easier to piggyback liveness summarization on region iteration that heuristics does at the end of mark anyway. See `ShenandoahHeuristics::choose_collection_set`. I can do that when you are done with your changes, or try it yourself.

I have addressed comments with trivial fixes. 
Will take a look at the remainder of more complex ones next.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar  1 15:27:22 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 1 Mar 2021 15:27:22 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v2]
In-Reply-To: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
Message-ID: <j_hamDyKbq2F_TiHIjGcxHXZh6poHX5m9L3C8XWqTcw=.24e5535b-a251-4cc5-9c0b-a07a6957da15@github.com>

> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
> 
> ## Introducing new JFR event
> 
> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
> 
> ## Implementation
> 
> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
> 
> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
> 
> ### Epsilon GC
> 
> Trivial implementation - just return `used()` instead.
> 
> ### Serial GC
> 
> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
> 
> ### Parallel GC
> 
> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
> 
> ### G1 GC
> 
> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
> 
> ### Shenandoah
> 
> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
> 
> ### ZGC
> 
> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.

Jaroslav Bachorik has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains ten additional commits since the last revision:

 - Merge remote-tracking branch 'origin/master' into jb/live_set_1
 - Change dead space calculation
 - Common PR fixes
 - Minor G1 related PR fixes
 - Epsilon related PR fixes
 - Shenandoah related PR fixes
 - Rename ZStatHeap::live() to live_at_mark_end()
 - Update event definition and emission
 - 8258431: Provide a JFR event with live set size estimate

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2579/files
  - new: https://git.openjdk.java.net/jdk/pull/2579/files/ddc5b5c1..03a8617e

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=01
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=00-01

  Stats: 45701 lines in 1355 files changed: 27365 ins; 10881 del; 7455 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2579.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2579/head:pull/2579

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar  1 15:27:26 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 1 Mar 2021 15:27:26 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v2]
In-Reply-To: <IEdDnCnjEiK9xVhyJexbfReEu7wMMCXHVvxKOxOLJhY=.1e0e1a2f-ec45-411d-a62f-010a8e744586@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <sfCa2-VPnTrwD7wqUjbnqswtEQ-qY0VhmhntJVo62jo=.c1af3d13-92f7-4be9-a7ae-6c21b3feeeca@github.com>
 <IEdDnCnjEiK9xVhyJexbfReEu7wMMCXHVvxKOxOLJhY=.1e0e1a2f-ec45-411d-a62f-010a8e744586@github.com>
Message-ID: <QF_9P2DyufSKHBUoKD43tjJwDAFUXzHWqN3ZdfGJuc0=.806457d0-93b1-44b1-bda0-df90030c187b@github.com>

On Thu, 18 Feb 2021 10:27:21 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 627:
>> 
>>> 625: 
>>> 626: size_t ShenandoahHeap::live() const {
>>> 627:   size_t live = Atomic::load_acquire(&_live);
>> 
>> I understand you copy-pasted from the same file. We have removed `_acquire` with #2504. Do `Atomic::load` here.
>
> ...which also means you want to merge from master to get recent changes?

Yep. Done.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar  1 15:27:32 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 1 Mar 2021 15:27:32 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v2]
In-Reply-To: <sfCa2-VPnTrwD7wqUjbnqswtEQ-qY0VhmhntJVo62jo=.c1af3d13-92f7-4be9-a7ae-6c21b3feeeca@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <sfCa2-VPnTrwD7wqUjbnqswtEQ-qY0VhmhntJVo62jo=.c1af3d13-92f7-4be9-a7ae-6c21b3feeeca@github.com>
Message-ID: <WQxJvGk1L3BQWUn6FzdPkS2ys_uEY45qX6d3q0V3JS4=.d624da23-43f0-4d51-bf0e-8b604b289cf9@github.com>

On Thu, 18 Feb 2021 10:23:53 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Jaroslav Bachorik has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains ten additional commits since the last revision:
>> 
>>  - Merge remote-tracking branch 'origin/master' into jb/live_set_1
>>  - Change dead space calculation
>>  - Common PR fixes
>>  - Minor G1 related PR fixes
>>  - Epsilon related PR fixes
>>  - Shenandoah related PR fixes
>>  - Rename ZStatHeap::live() to live_at_mark_end()
>>  - Update event definition and emission
>>  - 8258431: Provide a JFR event with live set size estimate
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 655:
> 
>> 653: 
>> 654: void ShenandoahHeap::set_live(size_t bytes) {
>> 655:   Atomic::release_store_fence(&_live, bytes);
> 
> Same, do `Atomic::store` here.

??

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar  1 15:27:38 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 1 Mar 2021 15:27:38 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v2]
In-Reply-To: <V1O-9sR2sGh-d-pUwda_dOA-cTLQbCgrfrGrrgJhcPI=.cabebed8-6561-42c7-80ca-7d9fed043a81@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <V1O-9sR2sGh-d-pUwda_dOA-cTLQbCgrfrGrrgJhcPI=.cabebed8-6561-42c7-80ca-7d9fed043a81@github.com>
Message-ID: <wDJ5QGMQD3Gwx8-di2aNr-oF7iBHAKIJyO5ua7_bO2E=.2eca4bb6-e7d8-411e-a03d-479b23e9521b@github.com>

On Mon, 22 Feb 2021 17:16:46 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

>> Jaroslav Bachorik has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains ten additional commits since the last revision:
>> 
>>  - Merge remote-tracking branch 'origin/master' into jb/live_set_1
>>  - Change dead space calculation
>>  - Common PR fixes
>>  - Minor G1 related PR fixes
>>  - Epsilon related PR fixes
>>  - Shenandoah related PR fixes
>>  - Rename ZStatHeap::live() to live_at_mark_end()
>>  - Update event definition and emission
>>  - 8258431: Provide a JFR event with live set size estimate
>
> src/hotspot/share/gc/shared/space.inline.hpp line 189:
> 
>> 187:         oop obj = oop(cur_obj);
>> 188:         size_t obj_size = obj->size();
>> 189:         live_offset += obj_size;
> 
> It seems more natural to me to put this counting into the `DeadSpacer` as this is what this change does. Also, the actual dead space "used" can be calculated from the difference between the `_allowed_deadspace_words` and the maximum (calculated in the constructor of `DeadSpacer`) afaict at the end of evacuation. So there is no need to incur per-object costs during evacuation at all.

Something like https://github.com/openjdk/jdk/pull/2579/commits/b28e7a0959a4655173bf1599bd035ab668196af6 would be ok?

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar  1 15:30:02 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 1 Mar 2021 15:30:02 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v2]
In-Reply-To: <VTiZD6sSDBGRvSYdF6WbcqOJEFD4RqWhaGMIe8VlzWg=.d2694ffc-284e-4613-af52-805ce83c5030@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <VTiZD6sSDBGRvSYdF6WbcqOJEFD4RqWhaGMIe8VlzWg=.d2694ffc-284e-4613-af52-805ce83c5030@github.com>
Message-ID: <kO2Q210OY2-CYfrUkT8pdbvYeTm707qMVIpl-k8e3hA=.845ec300-9a48-473a-89cf-e840f1163002@github.com>

On Mon, 22 Feb 2021 19:37:37 GMT, Erik Gahlin <egahlin at openjdk.org> wrote:

>> Jaroslav Bachorik has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains ten additional commits since the last revision:
>> 
>>  - Merge remote-tracking branch 'origin/master' into jb/live_set_1
>>  - Change dead space calculation
>>  - Common PR fixes
>>  - Minor G1 related PR fixes
>>  - Epsilon related PR fixes
>>  - Shenandoah related PR fixes
>>  - Rename ZStatHeap::live() to live_at_mark_end()
>>  - Update event definition and emission
>>  - 8258431: Provide a JFR event with live set size estimate
>
> src/hotspot/share/jfr/metadata/metadata.xml line 205:
> 
>> 203:     <Field type="ulong" contentType="bytes" name="capacity" label="Heap Capacity" description="Maximum number of bytes to be allocated by objects in the heap" />
>> 204:     <Field type="ulong" contentType="bytes" name="used" label="Heap Used" description="Bytes allocated by objects in the heap" />
>> 205:     <Field type="ulong" contentType="bytes" name="live" label="Heap Live" description="Live bytes allocated by objects in the heap" />
> 
> I think it would be good to mention in the description that it is an estimate, i.e. "Estimate of live bytes ....".

??

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar  1 15:37:05 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 1 Mar 2021 15:37:05 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v3]
In-Reply-To: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
Message-ID: <pjo2CgWL0mBrmleduVNr5eKMO-MkZ5rdHz8B3BVZZkA=.b29fd4cd-9e71-4d6c-9062-f12458968338@github.com>

> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
> 
> ## Introducing new JFR event
> 
> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
> 
> ## Implementation
> 
> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
> 
> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
> 
> ### Epsilon GC
> 
> Trivial implementation - just return `used()` instead.
> 
> ### Serial GC
> 
> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
> 
> ### Parallel GC
> 
> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
> 
> ### G1 GC
> 
> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
> 
> ### Shenandoah
> 
> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
> 
> ### ZGC
> 
> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.

Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:

  Do not track young, eden and old live size separately

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2579/files
  - new: https://git.openjdk.java.net/jdk/pull/2579/files/03a8617e..01c22ce6

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=02
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=01-02

  Stats: 13 lines in 3 files changed: 1 ins; 7 del; 5 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2579.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2579/head:pull/2579

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar  1 15:37:06 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 1 Mar 2021 15:37:06 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v3]
In-Reply-To: <kiGjWIhdxe0kJCyimsFwZ3wQ48v3zp67FcOBd_0gpOQ=.b99f8641-ba77-4c60-a3fc-32cf63f94a49@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <y_QIIisdV7CE1lKbYf8SxNwhEQ2Kcla8BtVI9M6pGtU=.d6ecb9b9-afc8-437c-9744-29bed3e89184@github.com>
 <kiGjWIhdxe0kJCyimsFwZ3wQ48v3zp67FcOBd_0gpOQ=.b99f8641-ba77-4c60-a3fc-32cf63f94a49@github.com>
Message-ID: <s8sGp-Mm3UaVJ1qu8Sjk2ySdm93QGikQVZWGVlQ1kR0=.f35b6a49-5fb2-49f1-81c5-8ffb2b0f4957@github.com>

On Mon, 22 Feb 2021 17:08:08 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

>> src/hotspot/share/gc/parallel/parallelScavengeHeap.hpp line 79:
>> 
>>> 77:   size_t _young_live;
>>> 78:   size_t _eden_live;
>>> 79:   size_t _old_live;
>> 
>> It's only the sum that's ever exposed, right? I wonder if it makes sense to merge them into one var to only track the sum.
>
> I agree because they seem to be always read and written at the same time.

The original idea was that the sum might be computed from different areas depending on the GC phase. But, apparently, that's not the case so we can have just one common sum.

??

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar  1 15:41:11 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 1 Mar 2021 15:41:11 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v4]
In-Reply-To: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
Message-ID: <wytOu1cMogz1OVZxCwteyHYpYqhgXRKLouJ2EP-jOok=.6960fc86-5850-4dce-a342-0a94470152eb@github.com>

> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
> 
> ## Introducing new JFR event
> 
> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
> 
> ## Implementation
> 
> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
> 
> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
> 
> ### Epsilon GC
> 
> Trivial implementation - just return `used()` instead.
> 
> ### Serial GC
> 
> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
> 
> ### Parallel GC
> 
> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
> 
> ### G1 GC
> 
> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
> 
> ### Shenandoah
> 
> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
> 
> ### ZGC
> 
> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.

Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:

  Fix dangling space

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2579/files
  - new: https://git.openjdk.java.net/jdk/pull/2579/files/01c22ce6..6a1aa73e

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=03
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=02-03

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2579.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2579/head:pull/2579

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar  1 16:34:04 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 1 Mar 2021 16:34:04 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v5]
In-Reply-To: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
Message-ID: <BW14g-lCaMw3pk0rxH0m7i3Z-HEwjKNUwRSmVAWej-8=.f601d91d-bbf7-46db-afa2-1bf3d0c8d12d@github.com>

> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
> 
> ## Introducing new JFR event
> 
> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
> 
> ## Implementation
> 
> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
> 
> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
> 
> ### Epsilon GC
> 
> Trivial implementation - just return `used()` instead.
> 
> ### Serial GC
> 
> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
> 
> ### Parallel GC
> 
> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
> 
> ### G1 GC
> 
> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
> 
> ### Shenandoah
> 
> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
> 
> ### ZGC
> 
> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.

Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:

  Fix syntax error

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2579/files
  - new: https://git.openjdk.java.net/jdk/pull/2579/files/6a1aa73e..dd204d8c

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=04
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=03-04

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2579.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2579/head:pull/2579

PR: https://git.openjdk.java.net/jdk/pull/2579

From github.com+71722661+earthling-amzn at openjdk.java.net  Mon Mar  1 16:38:07 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Mon, 1 Mar 2021 16:38:07 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v14]
In-Reply-To: <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
 <7CiRCYgYXysI9lWIUUayxOqdXZAZ7alOWWtNXMMDDjQ=.6020f939-907e-4790-9c2b-93230c41dcb8@github.com>
 <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
Message-ID: <3U8Ikz2fHz0ieqwXscv6wcr8LVehS4O17xrTR21_FiQ=.0017df3f-3b06-4a61-91af-58592ef16be4@github.com>

On Mon, 1 Mar 2021 10:37:24 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> earthling-amzn has updated the pull request incrementally with three additional commits since the last revision:
>> 
>>  - Encode active generation and old generation marking status
>>  - Do not unset preemption request flag prematurely
>>  - Disable reference processing generational mode
>
> src/hotspot/share/gc/shenandoah/mode/shenandoahGenerationalMode.cpp line 46:
> 
>> 44:   // HEY! Disabled while M7 work is in progress.
>> 45:   FLAG_SET_ERGO(ShenandoahUnloadClassesFrequency, 0);
>> 46:   FLAG_SET_ERGO(RegisterReferences, false);
> 
> This doesn't work in release build, unfortunately.

I also tried some runs with reference processing enabled and they succeeded. As I think about it, the reference processing itself will probably be fine with these changes. My concern would be that finalizers in the old generation might not run. I disabled it out of caution, but we can try leaving it enabled.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19

From github.com+71722661+earthling-amzn at openjdk.java.net  Mon Mar  1 16:52:06 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Mon, 1 Mar 2021 16:52:06 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v14]
In-Reply-To: <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
 <7CiRCYgYXysI9lWIUUayxOqdXZAZ7alOWWtNXMMDDjQ=.6020f939-907e-4790-9c2b-93230c41dcb8@github.com>
 <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
Message-ID: <i9RKDFA2KhXyY3KfZ7JocGeYCG1IN-bY_XxKO0NZ-LA=.0a796303-cf70-459c-a862-1859f1a2c7ec@github.com>

On Mon, 1 Mar 2021 10:31:35 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> earthling-amzn has updated the pull request incrementally with three additional commits since the last revision:
>> 
>>  - Encode active generation and old generation marking status
>>  - Do not unset preemption request flag prematurely
>>  - Disable reference processing generational mode
>
> src/hotspot/cpu/aarch64/gc/shenandoah/c1/shenandoahBarrierSetC1_aarch64.cpp line 32:
> 
>> 30: #include "gc/shenandoah/shenandoahBarrierSetAssembler.hpp"
>> 31: #include "gc/shenandoah/c1/shenandoahBarrierSetC1.hpp"
>> 32: #include "gc/shenandoah/mode/shenandoahMode.hpp"
> 
> The include is unused, isn't it?

Yes, removed.

> src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp line 64:
> 
>> 62:         __ tbz(rscratch1, ShenandoahHeap::HAS_FORWARDED_BITPOS, done);
>> 63:       } else {
>> 64:         __ mov(rscratch2, ShenandoahHeap::HAS_FORWARDED | ShenandoahHeap::OLD_MARKING);
> 
> In the x86 variant you preserve YOUNG_MARKING there.

Good catch, was too focused on fixing the build.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19

From github.com+71722661+earthling-amzn at openjdk.java.net  Mon Mar  1 16:59:10 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Mon, 1 Mar 2021 16:59:10 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v14]
In-Reply-To: <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
 <7CiRCYgYXysI9lWIUUayxOqdXZAZ7alOWWtNXMMDDjQ=.6020f939-907e-4790-9c2b-93230c41dcb8@github.com>
 <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
Message-ID: <0ECnTIb5cygBacVRjorz2P__9AmQuk8Pv8ng4QRghDI=.36f1b30c-77f5-4674-962d-3667c9f5f9e3@github.com>

On Mon, 1 Mar 2021 10:38:23 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> earthling-amzn has updated the pull request incrementally with three additional commits since the last revision:
>> 
>>  - Encode active generation and old generation marking status
>>  - Do not unset preemption request flag prematurely
>>  - Disable reference processing generational mode
>
> src/hotspot/share/gc/shenandoah/mode/shenandoahMode.hpp line 28:
> 
>> 26: #define SHARE_GC_SHENANDOAH_MODE_SHENANDOAHMODE_HPP
>> 27: 
>> 28: #include "runtime/java.hpp"
> 
> What is that include needed for?

Not needed, removed.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19

From jbachorik at openjdk.java.net  Mon Mar  1 17:37:11 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 1 Mar 2021 17:37:11 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v6]
In-Reply-To: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
Message-ID: <Dueyqxpm2Dv1xV0HlG9nmFpbFkc565b-SwC_ezkhDr0=.3a1cde81-1bed-40f4-a567-74f4c564e4f4@github.com>

> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
> 
> ## Introducing new JFR event
> 
> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
> 
> ## Implementation
> 
> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
> 
> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
> 
> ### Epsilon GC
> 
> Trivial implementation - just return `used()` instead.
> 
> ### Serial GC
> 
> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
> 
> ### Parallel GC
> 
> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
> 
> ### G1 GC
> 
> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
> 
> ### Shenandoah
> 
> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
> 
> ### ZGC
> 
> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.

Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:

  Attempt to fix G1 live set size computation

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2579/files
  - new: https://git.openjdk.java.net/jdk/pull/2579/files/dd204d8c..08c715ab

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=05
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=04-05

  Stats: 19 lines in 1 file changed: 7 ins; 10 del; 2 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2579.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2579/head:pull/2579

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar  1 17:37:12 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 1 Mar 2021 17:37:12 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v6]
In-Reply-To: <V1O-9sR2sGh-d-pUwda_dOA-cTLQbCgrfrGrrgJhcPI=.cabebed8-6561-42c7-80ca-7d9fed043a81@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <V1O-9sR2sGh-d-pUwda_dOA-cTLQbCgrfrGrrgJhcPI=.cabebed8-6561-42c7-80ca-7d9fed043a81@github.com>
Message-ID: <xwLjWAQUaGjR8hJn5JqBPx_VouUfNxozaXAONOFfNak=.b7a18a46-c8d3-4d33-b118-1865c65ece12@github.com>

On Mon, 22 Feb 2021 17:00:19 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

>> Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Attempt to fix G1 live set size computation
>
> src/hotspot/share/gc/g1/g1ConcurrentMark.cpp line 1114:
> 
>> 1112: 
>> 1113:   _g1h->set_live(live_size * HeapWordSize);
>> 1114: 
> 
> This code is located in the wrong place. It will return only the live words for the areas that have been marked, not eden or objects allocated in old gen after the marking started.
> 
> Further it iterates over all regions, which can be large compared to actually active regions.
> 
> A better place is in `G1UpdateRemSetTrackingBeforeRebuild::do_heap_region()` after the last method call - at that point, `HeapRegion::live_bytes()` contains the per-region number of live data for all regions.
> 
> `G1UpdateRemSetTrackingBeforeRebuild` is instantiated and then called by multiple threads. It's probably best that that `HeapClosure` locally sums up the live byte estimates and then in the caller `G1UpdateRemSetTrackingBeforeRebuildTask::work()` sums up the per thread results like is done for `G1UpdateRemSetTrackingBeforeRebuildTask::_total_selected_for_rebuild`, which is then set in the caller of the `G1UpdateRemSetTrackingBeforeRebuildTask`.

Would something along the line of https://github.com/openjdk/jdk/pull/2579/commits/08c715abccddbd04ced58706b7a705670843b43a be ok?

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From github.com+71722661+earthling-amzn at openjdk.java.net  Mon Mar  1 17:55:59 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Mon, 1 Mar 2021 17:55:59 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v14]
In-Reply-To: <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
 <7CiRCYgYXysI9lWIUUayxOqdXZAZ7alOWWtNXMMDDjQ=.6020f939-907e-4790-9c2b-93230c41dcb8@github.com>
 <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
Message-ID: <xeeHKO3rNSxlAm6shqVnXgkj1anE5vNKLQ8kUTwCgRQ=.6cb7313d-f882-436f-a820-b27ef910a7fb@github.com>

On Mon, 1 Mar 2021 13:47:56 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> earthling-amzn has updated the pull request incrementally with three additional commits since the last revision:
>> 
>>  - Encode active generation and old generation marking status
>>  - Do not unset preemption request flag prematurely
>>  - Disable reference processing generational mode
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 189:
> 
>> 187:   _cancelled_gc.set(CANCELLABLE);
>> 188:   if (clear_oom_handler) {
>> 189:     _oom_evac_handler.clear();
> 
> This change seems a bit suspicious to me. Why do we need to clear the OOM evac handler here?

Prior to these changes, the gc cancellation flag was always set because of an allocation failure, so the flag was cleared on a safepoint (final update refs). Now we have a case where we've "cancelled" a gc, but not because we've run out of memory so we don't need to clear the oom evac handler.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19

From github.com+71722661+earthling-amzn at openjdk.java.net  Mon Mar  1 18:11:55 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Mon, 1 Mar 2021 18:11:55 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v14]
In-Reply-To: <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
 <7CiRCYgYXysI9lWIUUayxOqdXZAZ7alOWWtNXMMDDjQ=.6020f939-907e-4790-9c2b-93230c41dcb8@github.com>
 <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
Message-ID: <CpVZ8jzz5Td1zk-kmgv1x9unoXO6zvnhoEOfmCLvSiI=.ee723ca2-d6d9-426c-955f-e33a82f8ba75@github.com>

On Mon, 1 Mar 2021 13:56:28 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> earthling-amzn has updated the pull request incrementally with three additional commits since the last revision:
>> 
>>  - Encode active generation and old generation marking status
>>  - Do not unset preemption request flag prematurely
>>  - Disable reference processing generational mode
>
> src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 342:
> 
>> 340:   if (_affiliation == FREE) {
>> 341:     //assert(_live_data == 0, "Setting non-zero live data (%zu) on FREE region", s);
>> 342:   }
> 
> Is this commented-out assert relevant? If not, maybe remove it?

Just left over debugging, removed.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19

From github.com+71722661+earthling-amzn at openjdk.java.net  Mon Mar  1 18:17:06 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Mon, 1 Mar 2021 18:17:06 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v14]
In-Reply-To: <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
 <7CiRCYgYXysI9lWIUUayxOqdXZAZ7alOWWtNXMMDDjQ=.6020f939-907e-4790-9c2b-93230c41dcb8@github.com>
 <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
Message-ID: <1hwGqIq7i0QKC9wxgAVtPsoK7oKFvxfuxQ_C1ld9PDU=.3e76aeb5-51b5-445c-81f7-f03a35bde051@github.com>

On Mon, 1 Mar 2021 14:00:36 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> earthling-amzn has updated the pull request incrementally with three additional commits since the last revision:
>> 
>>  - Encode active generation and old generation marking status
>>  - Do not unset preemption request flag prematurely
>>  - Disable reference processing generational mode
>
> src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 524:
> 
>> 522:   printf("SHR::recycle(), setting region (%llx, %llx, %llx) to FREE\n",
>> 523:          (unsigned long long) bottom(), (unsigned long long) top(), (unsigned long long) end());
>> 524:   fflush(stdout);
> 
> In Hotspot, it's simpler to use tty->print_cr("...") for this, or even better use logging e.g. log_trace(gc)("...") or so.

I agree, I've taken out all the debug scaffolding.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19

From rkennke at openjdk.java.net  Mon Mar  1 18:17:07 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 1 Mar 2021 18:17:07 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v14]
In-Reply-To: <xeeHKO3rNSxlAm6shqVnXgkj1anE5vNKLQ8kUTwCgRQ=.6cb7313d-f882-436f-a820-b27ef910a7fb@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
 <7CiRCYgYXysI9lWIUUayxOqdXZAZ7alOWWtNXMMDDjQ=.6020f939-907e-4790-9c2b-93230c41dcb8@github.com>
 <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
 <xeeHKO3rNSxlAm6shqVnXgkj1anE5vNKLQ8kUTwCgRQ=.6cb7313d-f882-436f-a820-b27ef910a7fb@github.com>
Message-ID: <TQ9fSU6PQstl1VMKaSjVm0iV2v9XuIzDNpq626QeP4U=.77637dce-20b5-48f7-9a16-d4c53939a02e@github.com>

On Mon, 1 Mar 2021 17:53:37 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 189:
>> 
>>> 187:   _cancelled_gc.set(CANCELLABLE);
>>> 188:   if (clear_oom_handler) {
>>> 189:     _oom_evac_handler.clear();
>> 
>> This change seems a bit suspicious to me. Why do we need to clear the OOM evac handler here?
>
> Prior to these changes, the gc cancellation flag was always set because of an allocation failure, so the flag was cleared on a safepoint (final update refs). Now we have a case where we've "cancelled" a gc, but not because we've run out of memory so we don't need to clear the oom evac handler.

Ok. But I think the OOM handler still needs to be cleared at the safepoint.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19

From github.com+71722661+earthling-amzn at openjdk.java.net  Mon Mar  1 18:24:00 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Mon, 1 Mar 2021 18:24:00 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v14]
In-Reply-To: <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
 <7CiRCYgYXysI9lWIUUayxOqdXZAZ7alOWWtNXMMDDjQ=.6020f939-907e-4790-9c2b-93230c41dcb8@github.com>
 <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
Message-ID: <SOEypdPpNHdfa_HB9Msrk7J-Xvbq6FUVYodqoh-nX2Y=.b5bf7cd1-ace8-4c40-b017-8c65bbc3e5bb@github.com>

On Mon, 1 Mar 2021 14:08:52 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> earthling-amzn has updated the pull request incrementally with three additional commits since the last revision:
>> 
>>  - Encode active generation and old generation marking status
>>  - Do not unset preemption request flag prematurely
>>  - Disable reference processing generational mode
>
> src/hotspot/share/gc/shenandoah/shenandoahOldGC.cpp line 1:
> 
>> 1: #include "precompiled.hpp"
> 
> Missing copyright header here.

Fixed.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19

From github.com+71722661+earthling-amzn at openjdk.java.net  Mon Mar  1 18:30:06 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Mon, 1 Mar 2021 18:30:06 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v14]
In-Reply-To: <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
 <7CiRCYgYXysI9lWIUUayxOqdXZAZ7alOWWtNXMMDDjQ=.6020f939-907e-4790-9c2b-93230c41dcb8@github.com>
 <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
Message-ID: <lOBX2hLvxto627oPQxK6mFqlNyfT3UQ-8V6nwPpgC48=.0ebcc55a-73af-4a91-b86c-7ad2a5814149@github.com>

On Mon, 1 Mar 2021 14:10:37 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> earthling-amzn has updated the pull request incrementally with three additional commits since the last revision:
>> 
>>  - Encode active generation and old generation marking status
>>  - Do not unset preemption request flag prematurely
>>  - Disable reference processing generational mode
>
> src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 1:
> 
>> 1: #include "precompiled.hpp"
> 
> Missing header.

Wow, I sure did miss a lot of copyright headers. Can `git jcheck` be made to check for this?

> src/hotspot/share/gc/shenandoah/shenandoahPhaseTimings.hpp line 103:
> 
>> 101:                                                                                        \
>> 102:   f(init_update_refs_gross,                         "Pause Init Update Refs (G)")      \
>> 103:   f(init_update_refs,                               "Pause Init Update Refs (N)")      \
> 
> Looks like unrelated spacing changes

Do you want me to revert this and put it in a separate PR? Maybe when we add a phase for remembered set scanning?

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19

From github.com+71722661+earthling-amzn at openjdk.java.net  Mon Mar  1 18:35:55 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Mon, 1 Mar 2021 18:35:55 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v14]
In-Reply-To: <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
 <7CiRCYgYXysI9lWIUUayxOqdXZAZ7alOWWtNXMMDDjQ=.6020f939-907e-4790-9c2b-93230c41dcb8@github.com>
 <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
Message-ID: <84hL6xnnfOYxZ5a1oj3LDSs507qhSZOCNrS-W57ysUY=.2ca40f99-3a76-4682-82bf-d28df3df40a9@github.com>

On Mon, 1 Mar 2021 14:16:32 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> earthling-amzn has updated the pull request incrementally with three additional commits since the last revision:
>> 
>>  - Encode active generation and old generation marking status
>>  - Do not unset preemption request flag prematurely
>>  - Disable reference processing generational mode
>
> src/hotspot/share/gc/shenandoah/shenandoahScanRemembered.hpp line 28:
> 
>> 26: // During development of this new feature, we want the option to test
>> 27: // with and without, and to compare performance before and after.
>> 28: #define FAST_REMEMBERED_SET_SCANNING
> 
> You'll still want the normal #ifndef SHARE_GC_SHENANDOAH_SHENANDOAHSCANREMEMBERED_HPP protection, don't you?

Guards were further down in the file, I pulled them up closer to where you'd expect them.

> src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 423:
> 
>> 421: }
>> 422: 
>> 423: void ShenandoahHeapRegion::oop_iterate_objects(OopIterateClosure* blk, bool fill_dead_objects, bool reregister_coalesced_objects) {
> 
> I wonder if this is better done as two separate methods: the normal/old oop_iterate_objects() and a new one that also fills dead objects?

We have some work in progress (scheduled for completion these next two weeks) that will remove the need to fill dead objects. I'd rather clean this up as part of those changes.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19

From rkennke at openjdk.java.net  Mon Mar  1 19:24:00 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 1 Mar 2021 19:24:00 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v14]
In-Reply-To: <lOBX2hLvxto627oPQxK6mFqlNyfT3UQ-8V6nwPpgC48=.0ebcc55a-73af-4a91-b86c-7ad2a5814149@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
 <7CiRCYgYXysI9lWIUUayxOqdXZAZ7alOWWtNXMMDDjQ=.6020f939-907e-4790-9c2b-93230c41dcb8@github.com>
 <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
 <lOBX2hLvxto627oPQxK6mFqlNyfT3UQ-8V6nwPpgC48=.0ebcc55a-73af-4a91-b86c-7ad2a5814149@github.com>
Message-ID: <ieYm7ehPW8ATCcaPxQYSOaAzUw65-du5idAA8M582gk=.3fdb0a4f-c103-4e2d-a050-d602d022d18c@github.com>

On Mon, 1 Mar 2021 18:26:40 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahPhaseTimings.hpp line 103:
>> 
>>> 101:                                                                                        \
>>> 102:   f(init_update_refs_gross,                         "Pause Init Update Refs (G)")      \
>>> 103:   f(init_update_refs,                               "Pause Init Update Refs (N)")      \
>> 
>> Looks like unrelated spacing changes
>
> Do you want me to revert this and put it in a separate PR? Maybe when we add a phase for remembered set scanning?

You could propose it upstream. However, looking at it again, it may have been done on purpose, to line up "Init Update Refs" with "Final Update Refs". Not sure how reasonable that is, though. I'd revert that part of the change here, though.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19

From github.com+71722661+earthling-amzn at openjdk.java.net  Mon Mar  1 19:57:27 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Mon, 1 Mar 2021 19:57:27 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v15]
In-Reply-To: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
Message-ID: <NUEueEIptpsBQproYkkFaxXnj-5QmmLuKSqMMb7y7nM=.ab93dce1-24f4-4479-a89c-bf2a7a6b2510@github.com>

> **This is a work in progress.**
> 
> ## Summary of changes
> The goal of these changes is to allow young cycles to suspend marking in the old generation. When the young cycle is complete, the concurrent old marking will resume.
> * Evaluation of heuristics was pulled into a new `ShenandoahRegulatorThread`. Taking this evaluation out of line from the control thread allows the regulator to suspend (using the cancellation mechanism) old generation marking and start a young generation cycle.
> * Task queues for marking have been moved from `ShenandoahMarkingContext` into `ShenandoahGeneration`. This allows the marking state for the old generation to persist across young generation cycles. The associated `is_complete` state has also been moved to `ShenandoahGeneration`.
> * Old generation marking is bootstrapped by a complete young generation cycle. In this scenario, the mark closures for a young cycle are given reference to the young generation mark queues _and_ the old generation mark queues. Rather than ignore old references as is done for a normal young cycle, they are enqueued in the old generation mark queues. When the young cycle completes, the old generation marking continues using the task queues primed by the preceding young cycle.
> * There is a new flag: `ShenandoahAllowOldMarkingPreemption` (defaults to true). Disabling this option will cause the regulator to schedule  `young` or `global` collects (according to heuristics), but will _not_ schedule `old` collects.
> * The `global` generation is used to support satb and incremental-update modes. The `global` generation is also used for degenerated and implicit/explicit gc requests. Degenerated cycles are not working on this branch and the root cause is understood.
> * The `global` generation is also used for `FullGC`, but this is also broken. The `FullGC` doesn't update the remembered set during compaction. We reckon there is a non-trivial amount of work to fix this.
> * The `MARKING` gc state has been split into `YOUNG_MARKING` and `OLD_MARKING`.
> * Immediate garbage collection is broken in generational mode. The update references phase is used to repair the remembered set, so when this phase is skipped the remembered set scan runs into trouble. A fix for this is in progress.
> * The remembered set is scanned on a safepoint during initial mark. Work to make this concurrent is in progress.

earthling-amzn has updated the pull request incrementally with 10 additional commits since the last revision:

 - Setup generational flags in ShenandoahGenerationalMode
 - Move include guards closer to top of file
 - Add missing copyrights, clean up includes
 - Remove debug scaffolding
 - Remove stray whitespace
 - Move generation related enumerations into common header to simplify includes
 - Fix copyright notice
 - Remove unnecessary includes
 - Cannot disable RegisterReferences in release build
 - Check for young or old marking

-------------

Changes:
  - all: https://git.openjdk.java.net/shenandoah/pull/19/files
  - new: https://git.openjdk.java.net/shenandoah/pull/19/files/8db6bb11..fedd0aed

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=19&range=14
 - incr: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=19&range=13-14

  Stats: 292 lines in 24 files changed: 177 ins; 101 del; 14 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/19.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/19/head:pull/19

PR: https://git.openjdk.java.net/shenandoah/pull/19

From github.com+71722661+earthling-amzn at openjdk.java.net  Mon Mar  1 19:57:30 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Mon, 1 Mar 2021 19:57:30 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v14]
In-Reply-To: <lOBX2hLvxto627oPQxK6mFqlNyfT3UQ-8V6nwPpgC48=.0ebcc55a-73af-4a91-b86c-7ad2a5814149@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
 <7CiRCYgYXysI9lWIUUayxOqdXZAZ7alOWWtNXMMDDjQ=.6020f939-907e-4790-9c2b-93230c41dcb8@github.com>
 <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
 <lOBX2hLvxto627oPQxK6mFqlNyfT3UQ-8V6nwPpgC48=.0ebcc55a-73af-4a91-b86c-7ad2a5814149@github.com>
Message-ID: <QyvWgb4EskQH5X9LLd450czrEGwtq3-CxJsbmzxAOVQ=.b11fbf82-8b49-47c2-87fa-6eaa9aeb2038@github.com>

On Mon, 1 Mar 2021 18:24:48 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp line 1:
>> 
>>> 1: #include "precompiled.hpp"
>> 
>> Missing header.
>
> Wow, I sure did miss a lot of copyright headers. Can `git jcheck` be made to check for this?

`grep -r -L -i copyright src/hotspot/share/gc/shenandoah` also works.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19

From zgu at openjdk.java.net  Mon Mar  1 20:25:56 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 1 Mar 2021 20:25:56 GMT
Subject: RFR: 8262793: Shenandoah: Need to restore reference marking strength
 after mark through subgraph
Message-ID: <v-IOE56SJ9j9df2iB24eMSQTIMqsUQBfZ1KTHWorj0Q=.c4ac0b60-df5e-4f8b-8c45-56b8959609f8@github.com>

During marking phase, wavefront may switch to weak marking, but need to restore back to original wavefront after mark through the subgraph.

-------------

Commit messages:
 - JDK-8262793

Changes: https://git.openjdk.java.net/jdk/pull/2784/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2784&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8262793
  Stats: 22 lines in 3 files changed: 17 ins; 2 del; 3 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2784.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2784/head:pull/2784

PR: https://git.openjdk.java.net/jdk/pull/2784

From github.com+71722661+earthling-amzn at openjdk.java.net  Mon Mar  1 21:00:21 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Mon, 1 Mar 2021 21:00:21 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v16]
In-Reply-To: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
Message-ID: <B87068WDm8aD3iUIHtttuS3IOapS_b7mlaArE0PXerQ=.25c18e27-9c62-48f0-a183-0edc577d532d@github.com>

> **This is a work in progress.**
> 
> ## Summary of changes
> The goal of these changes is to allow young cycles to suspend marking in the old generation. When the young cycle is complete, the concurrent old marking will resume.
> * Evaluation of heuristics was pulled into a new `ShenandoahRegulatorThread`. Taking this evaluation out of line from the control thread allows the regulator to suspend (using the cancellation mechanism) old generation marking and start a young generation cycle.
> * Task queues for marking have been moved from `ShenandoahMarkingContext` into `ShenandoahGeneration`. This allows the marking state for the old generation to persist across young generation cycles. The associated `is_complete` state has also been moved to `ShenandoahGeneration`.
> * Old generation marking is bootstrapped by a complete young generation cycle. In this scenario, the mark closures for a young cycle are given reference to the young generation mark queues _and_ the old generation mark queues. Rather than ignore old references as is done for a normal young cycle, they are enqueued in the old generation mark queues. When the young cycle completes, the old generation marking continues using the task queues primed by the preceding young cycle.
> * There is a new flag: `ShenandoahAllowOldMarkingPreemption` (defaults to true). Disabling this option will cause the regulator to schedule  `young` or `global` collects (according to heuristics), but will _not_ schedule `old` collects.
> * The `global` generation is used to support satb and incremental-update modes. The `global` generation is also used for degenerated and implicit/explicit gc requests. Degenerated cycles are not working on this branch and the root cause is understood.
> * The `global` generation is also used for `FullGC`, but this is also broken. The `FullGC` doesn't update the remembered set during compaction. We reckon there is a non-trivial amount of work to fix this.
> * The `MARKING` gc state has been split into `YOUNG_MARKING` and `OLD_MARKING`.
> * Immediate garbage collection is broken in generational mode. The update references phase is used to repair the remembered set, so when this phase is skipped the remembered set scan runs into trouble. A fix for this is in progress.
> * The remembered set is scanned on a safepoint during initial mark. Work to make this concurrent is in progress.

earthling-amzn has updated the pull request incrementally with one additional commit since the last revision:

  Revert unrelated whitespace change

-------------

Changes:
  - all: https://git.openjdk.java.net/shenandoah/pull/19/files
  - new: https://git.openjdk.java.net/shenandoah/pull/19/files/fedd0aed..74b5bafa

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=19&range=15
 - incr: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=19&range=14-15

  Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/19.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/19/head:pull/19

PR: https://git.openjdk.java.net/shenandoah/pull/19

From zgu at openjdk.java.net  Mon Mar  1 21:00:53 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 1 Mar 2021 21:00:53 GMT
Subject: Withdrawn: 8262793: Shenandoah: Need to restore reference marking
 strength after mark through subgraph
In-Reply-To: <v-IOE56SJ9j9df2iB24eMSQTIMqsUQBfZ1KTHWorj0Q=.c4ac0b60-df5e-4f8b-8c45-56b8959609f8@github.com>
References: <v-IOE56SJ9j9df2iB24eMSQTIMqsUQBfZ1KTHWorj0Q=.c4ac0b60-df5e-4f8b-8c45-56b8959609f8@github.com>
Message-ID: <TIC_eiWjN_WP_IwjNFCIDUmchtdtoitx3WNHIo-nSt8=.4579558f-6693-4914-abdc-cd26cbcee633@github.com>

On Mon, 1 Mar 2021 20:21:31 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> During marking phase, wavefront may switch to weak marking, but need to restore back to original wavefront after mark through the subgraph.

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2784

From github.com+71722661+earthling-amzn at openjdk.java.net  Mon Mar  1 21:05:24 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Mon, 1 Mar 2021 21:05:24 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v17]
In-Reply-To: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
Message-ID: <_W4qt_nItus4F_NEI0wEBnY8xbpXeDLdLTNrI5jrsdo=.ae0339ec-b494-44ac-bd0e-201a31d15927@github.com>

> **This is a work in progress.**
> 
> ## Summary of changes
> The goal of these changes is to allow young cycles to suspend marking in the old generation. When the young cycle is complete, the concurrent old marking will resume.
> * Evaluation of heuristics was pulled into a new `ShenandoahRegulatorThread`. Taking this evaluation out of line from the control thread allows the regulator to suspend (using the cancellation mechanism) old generation marking and start a young generation cycle.
> * Task queues for marking have been moved from `ShenandoahMarkingContext` into `ShenandoahGeneration`. This allows the marking state for the old generation to persist across young generation cycles. The associated `is_complete` state has also been moved to `ShenandoahGeneration`.
> * Old generation marking is bootstrapped by a complete young generation cycle. In this scenario, the mark closures for a young cycle are given reference to the young generation mark queues _and_ the old generation mark queues. Rather than ignore old references as is done for a normal young cycle, they are enqueued in the old generation mark queues. When the young cycle completes, the old generation marking continues using the task queues primed by the preceding young cycle.
> * There is a new flag: `ShenandoahAllowOldMarkingPreemption` (defaults to true). Disabling this option will cause the regulator to schedule  `young` or `global` collects (according to heuristics), but will _not_ schedule `old` collects.
> * The `global` generation is used to support satb and incremental-update modes. The `global` generation is also used for degenerated and implicit/explicit gc requests. Degenerated cycles are not working on this branch and the root cause is understood.
> * The `global` generation is also used for `FullGC`, but this is also broken. The `FullGC` doesn't update the remembered set during compaction. We reckon there is a non-trivial amount of work to fix this.
> * The `MARKING` gc state has been split into `YOUNG_MARKING` and `OLD_MARKING`.
> * Immediate garbage collection is broken in generational mode. The update references phase is used to repair the remembered set, so when this phase is skipped the remembered set scan runs into trouble. A fix for this is in progress.
> * The remembered set is scanned on a safepoint during initial mark. Work to make this concurrent is in progress.

earthling-amzn has updated the pull request incrementally with one additional commit since the last revision:

  Clear oom handler when clearing cancelled GC after allocation failure

-------------

Changes:
  - all: https://git.openjdk.java.net/shenandoah/pull/19/files
  - new: https://git.openjdk.java.net/shenandoah/pull/19/files/74b5bafa..dd7fff8c

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=19&range=16
 - incr: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=19&range=15-16

  Stats: 3 lines in 3 files changed: 0 ins; 0 del; 3 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/19.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/19/head:pull/19

PR: https://git.openjdk.java.net/shenandoah/pull/19

From github.com+71722661+earthling-amzn at openjdk.java.net  Mon Mar  1 21:05:25 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Mon, 1 Mar 2021 21:05:25 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v14]
In-Reply-To: <TQ9fSU6PQstl1VMKaSjVm0iV2v9XuIzDNpq626QeP4U=.77637dce-20b5-48f7-9a16-d4c53939a02e@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
 <7CiRCYgYXysI9lWIUUayxOqdXZAZ7alOWWtNXMMDDjQ=.6020f939-907e-4790-9c2b-93230c41dcb8@github.com>
 <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
 <xeeHKO3rNSxlAm6shqVnXgkj1anE5vNKLQ8kUTwCgRQ=.6cb7313d-f882-436f-a820-b27ef910a7fb@github.com>
 <TQ9fSU6PQstl1VMKaSjVm0iV2v9XuIzDNpq626QeP4U=.77637dce-20b5-48f7-9a16-d4c53939a02e@github.com>
Message-ID: <wUvg6QA1L5giNn-acnhj_zDOtZguf5OYxlULa-mLUkA=.0a13c5ea-b112-4abc-a60c-cea5a5fec254@github.com>

On Mon, 1 Mar 2021 18:14:22 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Prior to these changes, the gc cancellation flag was always set because of an allocation failure, so the flag was cleared on a safepoint (final update refs). Now we have a case where we've "cancelled" a gc, but not because we've run out of memory so we don't need to clear the oom evac handler.
>
> Ok. But I think the OOM handler still needs to be cleared at the safepoint.

Ah yes, sorry. The default value for the boolean is true (to clear the oom handler). I've updated the other callers to pass the argument explicitly.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19

From github.com+71722661+earthling-amzn at openjdk.java.net  Mon Mar  1 22:58:32 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Mon, 1 Mar 2021 22:58:32 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v18]
In-Reply-To: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
Message-ID: <v56TR7vbwdNOT9pEoWIgHUCpL9VvGmtZUXOTEv924BU=.87815df7-b9ea-4a5f-a4b3-695dcf62e2bc@github.com>

> **This is a work in progress.**
> 
> ## Summary of changes
> The goal of these changes is to allow young cycles to suspend marking in the old generation. When the young cycle is complete, the concurrent old marking will resume.
> * Evaluation of heuristics was pulled into a new `ShenandoahRegulatorThread`. Taking this evaluation out of line from the control thread allows the regulator to suspend (using the cancellation mechanism) old generation marking and start a young generation cycle.
> * Task queues for marking have been moved from `ShenandoahMarkingContext` into `ShenandoahGeneration`. This allows the marking state for the old generation to persist across young generation cycles. The associated `is_complete` state has also been moved to `ShenandoahGeneration`.
> * Old generation marking is bootstrapped by a complete young generation cycle. In this scenario, the mark closures for a young cycle are given reference to the young generation mark queues _and_ the old generation mark queues. Rather than ignore old references as is done for a normal young cycle, they are enqueued in the old generation mark queues. When the young cycle completes, the old generation marking continues using the task queues primed by the preceding young cycle.
> * There is a new flag: `ShenandoahAllowOldMarkingPreemption` (defaults to true). Disabling this option will cause the regulator to schedule  `young` or `global` collects (according to heuristics), but will _not_ schedule `old` collects.
> * The `global` generation is used to support satb and incremental-update modes. The `global` generation is also used for degenerated and implicit/explicit gc requests. Degenerated cycles are not working on this branch and the root cause is understood.
> * The `global` generation is also used for `FullGC`, but this is also broken. The `FullGC` doesn't update the remembered set during compaction. We reckon there is a non-trivial amount of work to fix this.
> * The `MARKING` gc state has been split into `YOUNG_MARKING` and `OLD_MARKING`.
> * Immediate garbage collection is broken in generational mode. The update references phase is used to repair the remembered set, so when this phase is skipped the remembered set scan runs into trouble. A fix for this is in progress.
> * The remembered set is scanned on a safepoint during initial mark. Work to make this concurrent is in progress.

earthling-amzn has updated the pull request incrementally with one additional commit since the last revision:

  Assert out of FullGC lower in the stack for generational mode

-------------

Changes:
  - all: https://git.openjdk.java.net/shenandoah/pull/19/files
  - new: https://git.openjdk.java.net/shenandoah/pull/19/files/dd7fff8c..68318947

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=19&range=17
 - incr: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=19&range=16-17

  Stats: 7 lines in 2 files changed: 4 ins; 3 del; 0 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/19.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/19/head:pull/19

PR: https://git.openjdk.java.net/shenandoah/pull/19

From shade at redhat.com  Tue Mar  2 08:10:42 2021
From: shade at redhat.com (Aleksey Shipilev)
Date: Tue, 2 Mar 2021 09:10:42 +0100
Subject: [8u] RFC: Pick up recent aarch64-port cleanups
In-Reply-To: <f79e70de-0765-b3da-aa93-2035f3ef5021@redhat.com>
References: <f79e70de-0765-b3da-aa93-2035f3ef5021@redhat.com>
Message-ID: <5eb9f99b-37f2-00f5-92bd-e5581a06d8bc@redhat.com>

On 3/1/21 12:42 PM, Aleksey Shipilev wrote:
> There are a few code improvements in aarch64-port that we want to pick up and test before proposing
> the integration back. The pickup is clean, only affects hotspot repository, and gets merged
> automatically.
> 
> Testing: hotspot_gc_shenandoah {fastdebug,release}

Anyone?

-- 
Thanks,
-Aleksey


From jbachorik at openjdk.java.net  Tue Mar  2 09:07:55 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Tue, 2 Mar 2021 09:07:55 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v6]
In-Reply-To: <V1O-9sR2sGh-d-pUwda_dOA-cTLQbCgrfrGrrgJhcPI=.cabebed8-6561-42c7-80ca-7d9fed043a81@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <V1O-9sR2sGh-d-pUwda_dOA-cTLQbCgrfrGrrgJhcPI=.cabebed8-6561-42c7-80ca-7d9fed043a81@github.com>
Message-ID: <HgfCknsOs9IhBSdSiXKCdeQLrABGA9s6XcW67i6M9p0=.f1a7d78e-2b6b-4ef9-b8ce-10fbeb1e7860@github.com>

On Mon, 22 Feb 2021 17:10:28 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

>> Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Attempt to fix G1 live set size computation
>
> src/hotspot/share/gc/parallel/parallelScavengeHeap.inline.hpp line 49:
> 
>> 47:   _young_live = young_gen()->used_in_bytes();
>> 48:   _eden_live = young_gen()->eden_space()->used_in_bytes();
>> 49:   _old_live = old_gen()->used_in_bytes();
> 
> `_young_live` already seems to contain `_eden_live` looking at the implementation of `PSYoungGen::used_in_bytes()`:
> 
> I.e.
> 
> `size_t PSYoungGen::used_in_bytes() const {
>   return eden_space()->used_in_bytes()
>        + from_space()->used_in_bytes();      // to_space() is only used during scavenge
> }
> `
> 
> but maybe I'm wrong here.

This seems like a correct summary - I have updated the implementation.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Tue Mar  2 09:22:55 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Tue, 2 Mar 2021 09:22:55 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v6]
In-Reply-To: <8Q_OEFTu-Npp_Pr74VPWQ6DrQUzcaDtVZy7nru2RCbU=.7e80a5b9-be94-4442-beaf-9593b067241d@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <V1O-9sR2sGh-d-pUwda_dOA-cTLQbCgrfrGrrgJhcPI=.cabebed8-6561-42c7-80ca-7d9fed043a81@github.com>
 <vIclSQGpJVJWSb0jmbU-GREJoGJoUy_pNtYtBB5ry4Y=.5f4825b9-8319-4c5b-a9b5-4175d2338db3@github.com>
 <8Q_OEFTu-Npp_Pr74VPWQ6DrQUzcaDtVZy7nru2RCbU=.7e80a5b9-be94-4442-beaf-9593b067241d@github.com>
Message-ID: <TeUKTVHox65DRCKoANIl1OGRpf3xnDejtwHe4AqtTHo=.9fea4807-24f2-412e-9bdc-e6e82e8caa68@github.com>

On Mon, 1 Mar 2021 15:24:48 GMT, Jaroslav Bachorik <jbachorik at openjdk.org> wrote:

>> Another general comment about Shenandoah. It would seem easier to piggyback liveness summarization on region iteration that heuristics does at the end of mark anyway. See `ShenandoahHeuristics::choose_collection_set`. I can do that when you are done with your changes, or try it yourself.
>
> I have addressed comments with trivial fixes. 
> Will take a look at the remainder of more complex ones next.

Going over my initial changes I am starting to doubt the usefulness of defaulting to `used` value when `live` estimate is not available. It seems to be giving false information - perhaps it would be ok to return `0` as an invalid value to indicate that that particular information is not available and it would be up to the event consumer to fall back to using the `used` value or deal with the missing value by some other means.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From shade at openjdk.java.net  Tue Mar  2 09:54:40 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 2 Mar 2021 09:54:40 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v6]
In-Reply-To: <TeUKTVHox65DRCKoANIl1OGRpf3xnDejtwHe4AqtTHo=.9fea4807-24f2-412e-9bdc-e6e82e8caa68@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <V1O-9sR2sGh-d-pUwda_dOA-cTLQbCgrfrGrrgJhcPI=.cabebed8-6561-42c7-80ca-7d9fed043a81@github.com>
 <vIclSQGpJVJWSb0jmbU-GREJoGJoUy_pNtYtBB5ry4Y=.5f4825b9-8319-4c5b-a9b5-4175d2338db3@github.com>
 <8Q_OEFTu-Npp_Pr74VPWQ6DrQUzcaDtVZy7nru2RCbU=.7e80a5b9-be94-4442-beaf-9593b067241d@github.com>
 <TeUKTVHox65DRCKoANIl1OGRpf3xnDejtwHe4AqtTHo=.9fea4807-24f2-412e-9bdc-e6e82e8caa68@github.com>
Message-ID: <850jkqk912ORnuVtTM7JhVB4gP5BlxuggFktpkmJVvU=.f076b561-4b3a-4378-83b7-99b14b75e099@github.com>

On Tue, 2 Mar 2021 09:20:21 GMT, Jaroslav Bachorik <jbachorik at openjdk.org> wrote:

>> I have addressed comments with trivial fixes. 
>> Will take a look at the remainder of more complex ones next.
>
> Going over my initial changes I am starting to doubt the usefulness of defaulting to `used` value when `live` estimate is not available. It seems to be giving false information - perhaps it would be ok to return `0` as an invalid value to indicate that that particular information is not available and it would be up to the event consumer to fall back to using the `used` value or deal with the missing value by some other means.

Shenandoah parts are better like this (applies on top of your PR): https://cr.openjdk.java.net/~shade/8258431/shenandoah.patch

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Tue Mar  2 11:44:12 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Tue, 2 Mar 2021 11:44:12 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v7]
In-Reply-To: <850jkqk912ORnuVtTM7JhVB4gP5BlxuggFktpkmJVvU=.f076b561-4b3a-4378-83b7-99b14b75e099@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <V1O-9sR2sGh-d-pUwda_dOA-cTLQbCgrfrGrrgJhcPI=.cabebed8-6561-42c7-80ca-7d9fed043a81@github.com>
 <vIclSQGpJVJWSb0jmbU-GREJoGJoUy_pNtYtBB5ry4Y=.5f4825b9-8319-4c5b-a9b5-4175d2338db3@github.com>
 <8Q_OEFTu-Npp_Pr74VPWQ6DrQUzcaDtVZy7nru2RCbU=.7e80a5b9-be94-4442-beaf-9593b067241d@github.com>
 <TeUKTVHox65DRCKoANIl1OGRpf3xnDejtwHe4AqtTHo=.9fea4807-24f2-412e-9bdc-e6e82e8caa68@github.com>
 <850jkqk912ORnuVtTM7JhVB4gP5BlxuggFktpkmJVvU=.f076b561-4b3a-4378-83b7-99b14b75e099@github.com>
Message-ID: <gbzKaOv2QesiqqcDalN0N_i4xIp_d1uOqWPva42l2cA=.a5245b3f-7e81-49d3-86f9-2f8b1e578bba@github.com>

On Tue, 2 Mar 2021 09:50:20 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Going over my initial changes I am starting to doubt the usefulness of defaulting to `used` value when `live` estimate is not available. It seems to be giving false information - perhaps it would be ok to return `0` as an invalid value to indicate that that particular information is not available and it would be up to the event consumer to fall back to using the `used` value or deal with the missing value by some other means.
>
> Shenandoah parts are better like this (applies on top of your PR): https://cr.openjdk.java.net/~shade/8258431/shenandoah.patch

@shipilev Thanks! The patch has been applied.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Tue Mar  2 11:44:12 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Tue, 2 Mar 2021 11:44:12 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v7]
In-Reply-To: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
Message-ID: <5FpR9OM3pJ2JvMS0p9s7vHaft1oVb9wGdtmSqL7IlcM=.a5a51fde-ce07-4b9a-a4a1-5d9df52eb9df@github.com>

> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
> 
> ## Introducing new JFR event
> 
> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
> 
> ## Implementation
> 
> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
> 
> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
> 
> ### Epsilon GC
> 
> Trivial implementation - just return `used()` instead.
> 
> ### Serial GC
> 
> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
> 
> ### Parallel GC
> 
> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
> 
> ### G1 GC
> 
> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
> 
> ### Shenandoah
> 
> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
> 
> ### ZGC
> 
> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.

Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:

  Proper Shenandoah implementation of live size esitmate
  
  Co-authored-by: Aleksey Shipilev <shade at redhat.com>

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2579/files
  - new: https://git.openjdk.java.net/jdk/pull/2579/files/08c715ab..aa180d11

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=06
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=05-06

  Stats: 45 lines in 6 files changed: 16 ins; 27 del; 2 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2579.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2579/head:pull/2579

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Tue Mar  2 12:15:17 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Tue, 2 Mar 2021 12:15:17 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v8]
In-Reply-To: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
Message-ID: <i3MLmVEBZg_kYNYMCmKAwPq_HOI0Di-sbPCf-D8BtkM=.8881144a-e820-483c-b0ac-815429eb4d29@github.com>

> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
> 
> ## Introducing new JFR event
> 
> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
> 
> ## Implementation
> 
> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
> 
> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
> 
> ### Epsilon GC
> 
> Trivial implementation - just return `used()` instead.
> 
> ### Serial GC
> 
> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
> 
> ### Parallel GC
> 
> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
> 
> ### G1 GC
> 
> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
> 
> ### Shenandoah
> 
> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
> 
> ### ZGC
> 
> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.

Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:

  Use '0' to indicate unvailable live estimate

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2579/files
  - new: https://git.openjdk.java.net/jdk/pull/2579/files/aa180d11..6c6f8a8a

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=07
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=06-07

  Stats: 21 lines in 11 files changed: 1 ins; 4 del; 16 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2579.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2579/head:pull/2579

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Tue Mar  2 14:33:14 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Tue, 2 Mar 2021 14:33:14 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v9]
In-Reply-To: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
Message-ID: <K4Ac7D3fQw_lxHfpya748uuBMet058eaHA_PgVFadzc=.4f7d9350-f1a2-4e29-b3db-6e387a2f1369@github.com>

> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
> 
> ## Introducing new JFR event
> 
> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
> 
> ## Implementation
> 
> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
> 
> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
> 
> ### Epsilon GC
> 
> Trivial implementation - just return `used()` instead.
> 
> ### Serial GC
> 
> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
> 
> ### Parallel GC
> 
> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
> 
> ### G1 GC
> 
> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
> 
> ### Shenandoah
> 
> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
> 
> ### ZGC
> 
> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.

Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:

  Add tests for the heap usage summary event

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2579/files
  - new: https://git.openjdk.java.net/jdk/pull/2579/files/6c6f8a8a..f6954186

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=08
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=07-08

  Stats: 24 lines in 5 files changed: 24 ins; 0 del; 0 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2579.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2579/head:pull/2579

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Tue Mar  2 14:33:14 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Tue, 2 Mar 2021 14:33:14 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v9]
In-Reply-To: <gbzKaOv2QesiqqcDalN0N_i4xIp_d1uOqWPva42l2cA=.a5245b3f-7e81-49d3-86f9-2f8b1e578bba@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <V1O-9sR2sGh-d-pUwda_dOA-cTLQbCgrfrGrrgJhcPI=.cabebed8-6561-42c7-80ca-7d9fed043a81@github.com>
 <vIclSQGpJVJWSb0jmbU-GREJoGJoUy_pNtYtBB5ry4Y=.5f4825b9-8319-4c5b-a9b5-4175d2338db3@github.com>
 <8Q_OEFTu-Npp_Pr74VPWQ6DrQUzcaDtVZy7nru2RCbU=.7e80a5b9-be94-4442-beaf-9593b067241d@github.com>
 <TeUKTVHox65DRCKoANIl1OGRpf3xnDejtwHe4AqtTHo=.9fea4807-24f2-412e-9bdc-e6e82e8caa68@github.com>
 <850jkqk912ORnuVtTM7JhVB4gP5BlxuggFktpkmJVvU=.f076b561-4b3a-4378-83b7-99b14b75e099@github.com>
 <gbzKaOv2QesiqqcDalN0N_i4xIp_d1uOqWPva42l2cA=.a5245b3f-7e81-49d3-86f9-2f8b1e578bba@github.com>
Message-ID: <j2AztWFSLT4T9hzuCIpB2txRFzoVBQx6BO6jsqWhV1A=.04a9136b-d2f8-49ad-9155-0decef6f8e97@github.com>

On Tue, 2 Mar 2021 11:41:19 GMT, Jaroslav Bachorik <jbachorik at openjdk.org> wrote:

>> Shenandoah parts are better like this (applies on top of your PR): https://cr.openjdk.java.net/~shade/8258431/shenandoah.patch
>
> @shipilev Thanks! The patch has been applied.

@pliden @tschatzl @albertnetymk @egahlin 
I believe I addressed all review comments. Can you, please, take a second look?

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Tue Mar  2 14:33:15 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Tue, 2 Mar 2021 14:33:15 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v9]
In-Reply-To: <sfCa2-VPnTrwD7wqUjbnqswtEQ-qY0VhmhntJVo62jo=.c1af3d13-92f7-4be9-a7ae-6c21b3feeeca@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <sfCa2-VPnTrwD7wqUjbnqswtEQ-qY0VhmhntJVo62jo=.c1af3d13-92f7-4be9-a7ae-6c21b3feeeca@github.com>
Message-ID: <di148JAHZVt_UM3XXH63Cp0zRMm4VR6QdxiRWAxNMoc=.f2825b9e-3a72-4196-830d-38bdf78a8f90@github.com>

On Thu, 18 Feb 2021 10:25:34 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Add tests for the heap usage summary event
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 511:
> 
>> 509: 
>> 510:   ShenandoahCollectLiveSizeClosure cl;
>> 511:   heap_region_iterate(&cl);
> 
> I think you want `parallel_heap_region_iterate` on this path, and do `Atomic::add(&_live, r->get_live_data_bytes())` in the closure. We shall see if this makes sense to make fully concurrently...

Outdated by the patch provided by @shipilev

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From ayang at openjdk.java.net  Tue Mar  2 16:04:40 2021
From: ayang at openjdk.java.net (Albert Mingkun Yang)
Date: Tue, 2 Mar 2021 16:04:40 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v9]
In-Reply-To: <K4Ac7D3fQw_lxHfpya748uuBMet058eaHA_PgVFadzc=.4f7d9350-f1a2-4e29-b3db-6e387a2f1369@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <K4Ac7D3fQw_lxHfpya748uuBMet058eaHA_PgVFadzc=.4f7d9350-f1a2-4e29-b3db-6e387a2f1369@github.com>
Message-ID: <pZ0J4E0UvDUOactKP__ZEuZNgNFczMG1xX_vg_SYvks=.a7049f10-2822-47a1-8cbb-4771fe4edaf2@github.com>

On Tue, 2 Mar 2021 14:33:14 GMT, Jaroslav Bachorik <jbachorik at openjdk.org> wrote:

>> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
>> 
>> ## Introducing new JFR event
>> 
>> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
>> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
>> 
>> ## Implementation
>> 
>> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
>> 
>> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
>> 
>> ### Epsilon GC
>> 
>> Trivial implementation - just return `used()` instead.
>> 
>> ### Serial GC
>> 
>> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
>> 
>> ### Parallel GC
>> 
>> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
>> 
>> ### G1 GC
>> 
>> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
>> 
>> ### Shenandoah
>> 
>> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
>> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
>> 
>> ### ZGC
>> 
>> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.
>
> Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Add tests for the heap usage summary event

Marked as reviewed by ayang (Author).

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From shade at openjdk.java.net  Tue Mar  2 17:40:44 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 2 Mar 2021 17:40:44 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v9]
In-Reply-To: <K4Ac7D3fQw_lxHfpya748uuBMet058eaHA_PgVFadzc=.4f7d9350-f1a2-4e29-b3db-6e387a2f1369@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <K4Ac7D3fQw_lxHfpya748uuBMet058eaHA_PgVFadzc=.4f7d9350-f1a2-4e29-b3db-6e387a2f1369@github.com>
Message-ID: <yI-GJzGp2ZSTlYk-eoDVI6WCiJABdUW3PqKH-g51rnc=.019108bb-e8b6-4d82-8fed-7a141adf9774@github.com>

On Tue, 2 Mar 2021 14:33:14 GMT, Jaroslav Bachorik <jbachorik at openjdk.org> wrote:

>> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
>> 
>> ## Introducing new JFR event
>> 
>> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
>> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
>> 
>> ## Implementation
>> 
>> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
>> 
>> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
>> 
>> ### Epsilon GC
>> 
>> Trivial implementation - just return `used()` instead.
>> 
>> ### Serial GC
>> 
>> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
>> 
>> ### Parallel GC
>> 
>> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
>> 
>> ### G1 GC
>> 
>> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
>> 
>> ### Shenandoah
>> 
>> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
>> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
>> 
>> ### ZGC
>> 
>> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.
>
> Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Add tests for the heap usage summary event

Shenandoah parts look good. I have a few minor stylistic comments.

src/hotspot/share/gc/shared/space.inline.hpp line 190:

> 188:         oop obj = oop(cur_obj);
> 189:         size_t obj_size = obj->size();
> 190:         compact_top = cp->space->forward(obj, obj_size, cp, compact_top);

This change seems superfluous now. Inline `obj_size` back?

src/hotspot/share/gc/shared/space.hpp line 555:

> 553:   size_t live() const            {
> 554:     return used() - _dead_space;
> 555:   }

Move it a few lines down, so `capacity`, `used`, `live` line up?

src/hotspot/share/gc/shared/collectedHeap.hpp line 218:

> 216:   virtual size_t capacity() const = 0;
> 217:   virtual size_t used() const = 0;
> 218:   // Returns the estimate of live set size. Because live set changes over time,

I believe a blank line is in order here, look at other comments in the same header.

src/hotspot/share/gc/shared/space.inline.hpp line 90:

> 88: 
> 89: public:
> 90:   size_t _dead_space;

Should this really be "public"? Maybe `friend`-ing with the only user is better?

-------------

Marked as reviewed by shade (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2579

From zgu at openjdk.java.net  Tue Mar  2 20:04:04 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 2 Mar 2021 20:04:04 GMT
Subject: RFR: 8262876: Shenandoah: Fix comments regarding
 VM_ShenandoahOperation inheritances
Message-ID: <dZD2c_X0g1HeR8gtn4E17SubpVN1ngA6lgfF1g995QA=.49f7a5fd-8717-409a-9431-2c4a0664b912@github.com>

please review this comment only fix.

-------------

Commit messages:
 - JDK-8262876

Changes: https://git.openjdk.java.net/jdk/pull/2796/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2796&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8262876
  Stats: 7 lines in 1 file changed: 4 ins; 3 del; 0 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2796.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2796/head:pull/2796

PR: https://git.openjdk.java.net/jdk/pull/2796

From rkennke at openjdk.java.net  Tue Mar  2 20:11:39 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 2 Mar 2021 20:11:39 GMT
Subject: RFR: 8262876: Shenandoah: Fix comments regarding
 VM_ShenandoahOperation inheritances
In-Reply-To: <dZD2c_X0g1HeR8gtn4E17SubpVN1ngA6lgfF1g995QA=.49f7a5fd-8717-409a-9431-2c4a0664b912@github.com>
References: <dZD2c_X0g1HeR8gtn4E17SubpVN1ngA6lgfF1g995QA=.49f7a5fd-8717-409a-9431-2c4a0664b912@github.com>
Message-ID: <ZckwAz9sSKZpb0FH-MjA6n1srYPFaee37ieoUr4kG7I=.7e7fa51c-8896-4d37-bc9e-49fbda88a824@github.com>

On Tue, 2 Mar 2021 19:57:32 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> please review this comment only fix.

Looks good and trivial.

-------------

Marked as reviewed by rkennke (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2796

From zgu at openjdk.java.net  Tue Mar  2 21:32:39 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 2 Mar 2021 21:32:39 GMT
Subject: Integrated: 8262876: Shenandoah: Fix comments regarding
 VM_ShenandoahOperation inheritances
In-Reply-To: <dZD2c_X0g1HeR8gtn4E17SubpVN1ngA6lgfF1g995QA=.49f7a5fd-8717-409a-9431-2c4a0664b912@github.com>
References: <dZD2c_X0g1HeR8gtn4E17SubpVN1ngA6lgfF1g995QA=.49f7a5fd-8717-409a-9431-2c4a0664b912@github.com>
Message-ID: <-QuiYLxexYX2fKicz75xpj2WYwNX7cPPeYSIBIk__ig=.8859d324-0c7a-4b8e-a5e6-39c5463813ec@github.com>

On Tue, 2 Mar 2021 19:57:32 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> please review this comment only fix.

This pull request has now been integrated.

Changeset: c92f3bc3
Author:    Zhengyu Gu <zgu at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/c92f3bc3
Stats:     7 lines in 1 file changed: 4 ins; 3 del; 0 mod

8262876: Shenandoah: Fix comments regarding VM_ShenandoahOperation inheritances

Reviewed-by: rkennke

-------------

PR: https://git.openjdk.java.net/jdk/pull/2796

From zgu at openjdk.java.net  Tue Mar  2 21:46:55 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 2 Mar 2021 21:46:55 GMT
Subject: RFR: 8262885: Shenandoah: FullGC prologue does not need to
 save/restore heap has_forwarded_object flag
Message-ID: <gqSLxq2K_yS8WoWXfyq20OXyP93Vybb94Brg63mFpr0=.5b176a40-0f4b-4b9d-b939-184565afafac@github.com>

Another trivial cleanup. FullGC prologue code does not change heap's has_forwarded_objects flag, there is no point to save/restore it.

Test:
- [x] hotspot_gc_shenandoah

-------------

Commit messages:
 - JDK-8262885

Changes: https://git.openjdk.java.net/jdk/pull/2797/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2797&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8262885
  Stats: 8 lines in 1 file changed: 0 ins; 6 del; 2 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2797.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2797/head:pull/2797

PR: https://git.openjdk.java.net/jdk/pull/2797

From github.com+71722661+earthling-amzn at openjdk.java.net  Wed Mar  3 01:26:24 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Wed, 3 Mar 2021 01:26:24 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v19]
In-Reply-To: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
Message-ID: <NZfT7vMpjZXyQ5B_5P3u8esyrxtLvYmopXzMUyejr6E=.34e58910-e358-4caf-98ed-19e678d69e21@github.com>

> **This is a work in progress.**
> 
> ## Summary of changes
> The goal of these changes is to allow young cycles to suspend marking in the old generation. When the young cycle is complete, the concurrent old marking will resume.
> * Evaluation of heuristics was pulled into a new `ShenandoahRegulatorThread`. Taking this evaluation out of line from the control thread allows the regulator to suspend (using the cancellation mechanism) old generation marking and start a young generation cycle.
> * Task queues for marking have been moved from `ShenandoahMarkingContext` into `ShenandoahGeneration`. This allows the marking state for the old generation to persist across young generation cycles. The associated `is_complete` state has also been moved to `ShenandoahGeneration`.
> * Old generation marking is bootstrapped by a complete young generation cycle. In this scenario, the mark closures for a young cycle are given reference to the young generation mark queues _and_ the old generation mark queues. Rather than ignore old references as is done for a normal young cycle, they are enqueued in the old generation mark queues. When the young cycle completes, the old generation marking continues using the task queues primed by the preceding young cycle.
> * There is a new flag: `ShenandoahAllowOldMarkingPreemption` (defaults to true). Disabling this option will cause the regulator to schedule  `young` or `global` collects (according to heuristics), but will _not_ schedule `old` collects.
> * The `global` generation is used to support satb and incremental-update modes. The `global` generation is also used for degenerated and implicit/explicit gc requests. Degenerated cycles are not working on this branch and the root cause is understood.
> * The `global` generation is also used for `FullGC`, but this is also broken. The `FullGC` doesn't update the remembered set during compaction. We reckon there is a non-trivial amount of work to fix this.
> * The `MARKING` gc state has been split into `YOUNG_MARKING` and `OLD_MARKING`.
> * Immediate garbage collection is broken in generational mode. The update references phase is used to repair the remembered set, so when this phase is skipped the remembered set scan runs into trouble. A fix for this is in progress.
> * The remembered set is scanned on a safepoint during initial mark. Work to make this concurrent is in progress.

earthling-amzn has updated the pull request incrementally with three additional commits since the last revision:

 - Resume in-progress old generation marking, rather than starting a new one
 - Fix includes for zero build
 - Disable reference processing during old generation marking

-------------

Changes:
  - all: https://git.openjdk.java.net/shenandoah/pull/19/files
  - new: https://git.openjdk.java.net/shenandoah/pull/19/files/68318947..90ea8c22

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=19&range=18
 - incr: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=19&range=17-18

  Stats: 32 lines in 7 files changed: 29 ins; 0 del; 3 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/19.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/19/head:pull/19

PR: https://git.openjdk.java.net/shenandoah/pull/19

From gnu.andrew at redhat.com  Wed Mar  3 02:39:59 2021
From: gnu.andrew at redhat.com (Andrew Hughes)
Date: Wed, 3 Mar 2021 02:39:59 +0000
Subject: [RFR] [8u] 8u292-b02 Upstream Sync
Message-ID: <20210303023959.GA1412354@rincewind>

Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/

Merge changesets:
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/corba/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/jaxp/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/jaxws/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/jdk/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/hotspot/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/langtools/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/nashorn/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/root/merge.changeset

Changes in aarch64-shenandoah-jdk8u292-b02:
  - JDK-8078614: WindowsClassicLookAndFeel MetalComboBoxUI.getbaseLine fails with IllegalArgumentException
  - JDK-8198334: java/awt/FileDialog/8003399/bug8003399.java fails in headless mode
  - JDK-8249251: [dark_mode ubuntu 20.04] The selected menu is not highlighted in GTKLookAndFeel
  - JDK-8250582: Revert Principal Name type to NT-UNKNOWN when requesting TGS Kerberos tickets
  - JDK-8258833: Cancel multi-part cipher operations in SunPKCS11 after failures
  - Revert differences against upstream 8u [already upstream]

Main issues of note:
None, clean merge (no HotSpot changes).

diffstat for root
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for corba
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jaxp
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jaxws
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for langtools
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for nashorn
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jdk
 b/.hgtags                                                           |    1 
 b/src/share/classes/com/sun/java/swing/plaf/gtk/GTKPainter.java     |  100 +---
 b/src/share/classes/javax/swing/plaf/basic/BasicComboBoxUI.java     |    2 
 b/src/share/classes/sun/security/krb5/internal/CredentialsUtil.java |    2 
 b/src/share/classes/sun/security/pkcs11/P11AEADCipher.java          |   21 -
 b/src/share/classes/sun/security/pkcs11/P11Cipher.java              |   55 ++
 b/src/share/classes/sun/security/pkcs11/P11Mac.java                 |   15 
 b/src/share/classes/sun/security/pkcs11/P11PSSSignature.java        |   14 
 b/src/share/classes/sun/security/pkcs11/P11Signature.java           |   14 
 b/test/com/sun/java/swing/plaf/windows/Test8173145.java             |   57 +-
 b/test/javax/swing/JComboBox/6632953/bug6632953.java                |   36 +
 b/test/javax/swing/JMenu/JMenuSelectedColorTest.java                |  206 ++++++++++
 b/test/sun/security/pkcs11/Cipher/CancelMultipart.java              |  188 +++++++++
 13 files changed, 587 insertions(+), 124 deletions(-)

diffstat for hotspot
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

Successfully built on x86, x86_64, s390 (Zero), s390x (Zero), ppc (Zero),
ppc64, ppc64le, aarch32 (Zero) & aarch64.

Ok to push?

Thanks,
-- 
Andrew :)

Senior Free Java Software Engineer
OpenJDK Package Owner
Red Hat, Inc. (http://www.redhat.com)

PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04  C5A0 CFDA 0F9B 3596 4222

From shade at redhat.com  Wed Mar  3 06:51:37 2021
From: shade at redhat.com (Aleksey Shipilev)
Date: Wed, 3 Mar 2021 07:51:37 +0100
Subject: [RFR] [8u] 8u292-b02 Upstream Sync
In-Reply-To: <20210303023959.GA1412354@rincewind>
References: <20210303023959.GA1412354@rincewind>
Message-ID: <d04971ac-d12d-e4fa-4e97-71d226126073@redhat.com>

On 3/3/21 3:39 AM, Andrew Hughes wrote:
> Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/
> 
> Merge changesets:
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/corba/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/jaxp/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/jaxws/merge.changeset

Looks trivially fine.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/jdk/merge.changeset

Looks fine.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/hotspot/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/langtools/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/nashorn/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/root/merge.changeset

Look trivially fine.

> Ok to push?

Yes.

-- 
Thanks,
-Aleksey


From shade at openjdk.java.net  Wed Mar  3 06:56:56 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Wed, 3 Mar 2021 06:56:56 GMT
Subject: RFR: 8262885: Shenandoah: FullGC prologue does not need to
 save/restore heap has_forwarded_object flag
In-Reply-To: <gqSLxq2K_yS8WoWXfyq20OXyP93Vybb94Brg63mFpr0=.5b176a40-0f4b-4b9d-b939-184565afafac@github.com>
References: <gqSLxq2K_yS8WoWXfyq20OXyP93Vybb94Brg63mFpr0=.5b176a40-0f4b-4b9d-b939-184565afafac@github.com>
Message-ID: <5lM3ql1hRaeud67jpuUL6ADAuaotWZSxMWay3Ww1jYY=.22635b95-141d-43c3-b3b6-24d26069fbda@github.com>

On Tue, 2 Mar 2021 21:40:45 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Another trivial cleanup. FullGC prologue code does not change heap's has_forwarded_objects flag, there is no point to save/restore it.
> 
> Test:
> - [x] hotspot_gc_shenandoah

Yeah, but this code protects from some downstream code we are calling from the prologue to accidentally drop the flag. At least it should be asserted that `has_forwarded_objects` has the same value at the end of the prologue as it was at the beginning. Or better just leave this for safety?

-------------

PR: https://git.openjdk.java.net/jdk/pull/2797

From tschatzl at openjdk.java.net  Wed Mar  3 12:19:50 2021
From: tschatzl at openjdk.java.net (Thomas Schatzl)
Date: Wed, 3 Mar 2021 12:19:50 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v9]
In-Reply-To: <K4Ac7D3fQw_lxHfpya748uuBMet058eaHA_PgVFadzc=.4f7d9350-f1a2-4e29-b3db-6e387a2f1369@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <K4Ac7D3fQw_lxHfpya748uuBMet058eaHA_PgVFadzc=.4f7d9350-f1a2-4e29-b3db-6e387a2f1369@github.com>
Message-ID: <l0lr-q29vacGe0upkTwrhXwi7sFwH-Jvcc-PxIobLJM=.aa5b21aa-f0ec-461d-aa32-87c5a211c177@github.com>

On Tue, 2 Mar 2021 14:33:14 GMT, Jaroslav Bachorik <jbachorik at openjdk.org> wrote:

>> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
>> 
>> ## Introducing new JFR event
>> 
>> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
>> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
>> 
>> ## Implementation
>> 
>> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
>> 
>> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
>> 
>> ### Epsilon GC
>> 
>> Trivial implementation - just return `used()` instead.
>> 
>> ### Serial GC
>> 
>> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
>> 
>> ### Parallel GC
>> 
>> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
>> 
>> ### G1 GC
>> 
>> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
>> 
>> ### Shenandoah
>> 
>> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
>> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
>> 
>> ### ZGC
>> 
>> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.
>
> Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Add tests for the heap usage summary event

Fwiw, the change still does not capture G1 full gc `live_estimate()`.

src/hotspot/share/gc/g1/g1ConcurrentMark.cpp line 1070:

> 1068: 
> 1069:     uint num_selected_for_rebuild() const { return _num_regions_selected_for_rebuild; }
> 1070:     size_t live_estimate() const { return _live; }

Please sync the member name with the getter name. I.e. `_live` -> `_live_estimate`

src/hotspot/share/gc/parallel/psAdaptiveSizePolicy.hpp line 60:

> 58: class PSAdaptiveSizePolicy : public AdaptiveSizePolicy {
> 59:  friend class PSGCAdaptivePolicyCounters;
> 60:  friend class ParallelScavengeHeap;

Delete this apparently unneeded friend declaration (compiled successfully without here)

src/hotspot/share/gc/parallel/parallelScavengeHeap.hpp line 87:

> 85: 
> 86:   // in order to provide accurate estimate this method must be called only when the heap has just been collected and compacted
> 87:   inline void capture_live();

Sentences should start with upper case in the comment. Also I'd prefer to name the method `update_live_estimate()` instead.

src/hotspot/share/gc/g1/g1CollectedHeap.hpp line 182:

> 180:   G1BlockOffsetTable* _bot;
> 181: 
> 182:   volatile size_t _live;

I'm not happy with naming this `_live`, better use `_live_estimate`. The contents are not continuously updated and basically out of date after the first following allocation.
This includes the naming in all other instances too.

src/hotspot/share/gc/serial/serialHeap.hpp line 44:

> 42:   MemoryPool* _old_pool;
> 43: 
> 44:   size_t _live_size;

Please rename to `_live_estimate` like the others. Avoid having different names in different collectors for the same thing.

src/hotspot/share/gc/shared/space.inline.hpp line 128:

> 126:           p2i(dead_start), p2i(dead_end), dead_length * HeapWordSize);
> 127: 
> 128:       _dead_space += dead_length;

I do not think adding this to the counter here instead of the other method for every object makes a difference performance-wise.

As mentioned before, `_allowed_deadspace_words` counts *down* from `(space->capacity() * ratio / 100) / HeapWordSize;` to whatever end value.

So at the end of collection, `(space->capacity() * ratio / 100) / HeapWordSize - _allowed_deadspace_words` should be equal to what `_dead_space` is now.

Please add a getter to `DeadSpacer` that calculates this (factoring out the calculation of the maximum allowed deadspace).

src/hotspot/share/gc/shared/space.hpp line 553:

> 551:   size_t capacity() const        { return byte_size(bottom(), end()); }
> 552:   size_t used() const            { return byte_size(bottom(), top()); }
> 553:   size_t live() const            {

The code for serial gc, contrary to others, tries to give some resemblance of tracking actual liveness. I.e. calculating this anew every call to `SerialHeap::live()`.
However if calling an `update_live_estimate()` in parallel and G1 (and the other collectors) is fine at certain places, this should be as good for serial gc.
Doing so would reduce the footprint of this change quite a bit (for serial gc)

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From rkennke at openjdk.java.net  Wed Mar  3 12:43:03 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 3 Mar 2021 12:43:03 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v14]
In-Reply-To: <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
 <7CiRCYgYXysI9lWIUUayxOqdXZAZ7alOWWtNXMMDDjQ=.6020f939-907e-4790-9c2b-93230c41dcb8@github.com>
 <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
Message-ID: <YV2nQ2TwAJwMA2r4Hh0GyrRfIJmgoVH-6ytlfwIAhgk=.fd8c180a-73be-4c3a-9c9b-c39084e15f76@github.com>

On Mon, 1 Mar 2021 14:19:43 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> earthling-amzn has updated the pull request incrementally with three additional commits since the last revision:
>> 
>>  - Encode active generation and old generation marking status
>>  - Do not unset preemption request flag prematurely
>>  - Disable reference processing generational mode
>
> That is a huge change! Good work!
> First-pass review follows.
> 
> (I need to remember to scan for 'HEY!' in addition to 'TODO' ;-) )

Do you plan to enable reference processing again on old-gen marking? Because I think it's a bad idea to not do it. Ideally, we'd do reference processing on both young-gen and old-gen markings, if that is not possible, then we should at least do it on old-gen markings (I believe other GCs don't do it in young-gen).

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19

From zgu at openjdk.java.net  Wed Mar  3 13:33:09 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Wed, 3 Mar 2021 13:33:09 GMT
Subject: RFR: 8262885: Shenandoah: FullGC prologue does not need to
 save/restore heap has_forwarded_object flag [v2]
In-Reply-To: <gqSLxq2K_yS8WoWXfyq20OXyP93Vybb94Brg63mFpr0=.5b176a40-0f4b-4b9d-b939-184565afafac@github.com>
References: <gqSLxq2K_yS8WoWXfyq20OXyP93Vybb94Brg63mFpr0=.5b176a40-0f4b-4b9d-b939-184565afafac@github.com>
Message-ID: <htesFbek5dgrk0BBCKDsVYwTSZYxyzhXstWvA429AXk=.9274d61b-f9f0-4e41-a7d1-a0fb9942c2d5@github.com>

> Another trivial cleanup. FullGC prologue code does not change heap's has_forwarded_objects flag, there is no point to save/restore it.
> 
> Test:
> - [x] hotspot_gc_shenandoah

Zhengyu Gu has updated the pull request incrementally with three additional commits since the last revision:

 - Aleksey's comment
 - Removed whitespaces
 - Aleksey's comment

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2797/files
  - new: https://git.openjdk.java.net/jdk/pull/2797/files/26b8b6d3..72eb25c5

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2797&range=01
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2797&range=00-01

  Stats: 6 lines in 1 file changed: 5 ins; 0 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2797.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2797/head:pull/2797

PR: https://git.openjdk.java.net/jdk/pull/2797

From shade at openjdk.java.net  Wed Mar  3 13:33:10 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Wed, 3 Mar 2021 13:33:10 GMT
Subject: RFR: 8262885: Shenandoah: FullGC prologue does not need to
 save/restore heap has_forwarded_object flag [v2]
In-Reply-To: <htesFbek5dgrk0BBCKDsVYwTSZYxyzhXstWvA429AXk=.9274d61b-f9f0-4e41-a7d1-a0fb9942c2d5@github.com>
References: <gqSLxq2K_yS8WoWXfyq20OXyP93Vybb94Brg63mFpr0=.5b176a40-0f4b-4b9d-b939-184565afafac@github.com>
 <htesFbek5dgrk0BBCKDsVYwTSZYxyzhXstWvA429AXk=.9274d61b-f9f0-4e41-a7d1-a0fb9942c2d5@github.com>
Message-ID: <GXUAIGzkP-dq6HPh9sv6KwdZrqY0jm2kpQIwngcC9zA=.f111ae26-572c-46a2-88ba-e2b3aadce249@github.com>

On Wed, 3 Mar 2021 13:30:29 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> Another trivial cleanup. FullGC prologue code does not change heap's has_forwarded_objects flag, there is no point to save/restore it.
>> 
>> Test:
>> - [x] hotspot_gc_shenandoah
>
> Zhengyu Gu has updated the pull request incrementally with three additional commits since the last revision:
> 
>  - Aleksey's comment
>  - Removed whitespaces
>  - Aleksey's comment

Good!

src/hotspot/share/gc/shenandoah/shenandoahFullGC.cpp line 189:

> 187:     _preserved_marks->init(heap->workers()->active_workers());
> 188:     
> 189:     assert(heap->has_forwarded_objects() == has_forwarded_objects, "Not expect to change");

"This should not change"?

-------------

Marked as reviewed by shade (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2797

From zgu at openjdk.java.net  Wed Mar  3 13:33:10 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Wed, 3 Mar 2021 13:33:10 GMT
Subject: RFR: 8262885: Shenandoah: FullGC prologue does not need to
 save/restore heap has_forwarded_object flag [v2]
In-Reply-To: <GXUAIGzkP-dq6HPh9sv6KwdZrqY0jm2kpQIwngcC9zA=.f111ae26-572c-46a2-88ba-e2b3aadce249@github.com>
References: <gqSLxq2K_yS8WoWXfyq20OXyP93Vybb94Brg63mFpr0=.5b176a40-0f4b-4b9d-b939-184565afafac@github.com>
 <htesFbek5dgrk0BBCKDsVYwTSZYxyzhXstWvA429AXk=.9274d61b-f9f0-4e41-a7d1-a0fb9942c2d5@github.com>
 <GXUAIGzkP-dq6HPh9sv6KwdZrqY0jm2kpQIwngcC9zA=.f111ae26-572c-46a2-88ba-e2b3aadce249@github.com>
Message-ID: <ZiIDXwbcIBNi0u6WbSrXdO-Gl9RssRNLX_H3oHKzr50=.df1d0d75-da5c-4849-8696-59788c284f4b@github.com>

On Wed, 3 Mar 2021 13:27:12 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Zhengyu Gu has updated the pull request incrementally with three additional commits since the last revision:
>> 
>>  - Aleksey's comment
>>  - Removed whitespaces
>>  - Aleksey's comment
>
> Good!

> Yeah, but this code protects from some downstream code we are calling from the prologue to accidentally drop the flag. At least it should be asserted that `has_forwarded_objects` has the same value at the end of the prologue as it was at the beginning. Or better just leave this for safety?

Okay, I added assert. The restore code led me to search where it got overwritten, a bit confusing.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2797

From bmathiske at openjdk.java.net  Wed Mar  3 16:29:06 2021
From: bmathiske at openjdk.java.net (Bernd Mathiske)
Date: Wed, 3 Mar 2021 16:29:06 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v14]
In-Reply-To: <YV2nQ2TwAJwMA2r4Hh0GyrRfIJmgoVH-6ytlfwIAhgk=.fd8c180a-73be-4c3a-9c9b-c39084e15f76@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
 <7CiRCYgYXysI9lWIUUayxOqdXZAZ7alOWWtNXMMDDjQ=.6020f939-907e-4790-9c2b-93230c41dcb8@github.com>
 <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
 <YV2nQ2TwAJwMA2r4Hh0GyrRfIJmgoVH-6ytlfwIAhgk=.fd8c180a-73be-4c3a-9c9b-c39084e15f76@github.com>
Message-ID: <M-FFynDq_S1yB8vUyLyfwsDQWGBOkIREMWlPdj9k9jA=.07334715-7e3a-4882-b7fb-4d890a065be2@github.com>

On Wed, 3 Mar 2021 12:39:49 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> That is a huge change! Good work!
>> First-pass review follows.
>> 
>> (I need to remember to scan for 'HEY!' in addition to 'TODO' ;-) )
>
> Do you plan to enable reference processing again on old-gen marking? Because I think it's a bad idea to not do it. Ideally, we'd do reference processing on both young-gen and old-gen markings, if that is not possible, then we should at least do it on old-gen markings (I believe other GCs don't do it in young-gen).

Yes, we plan to make reference processing function again and at a minimum this will hold for non-young collections.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19

From zgu at openjdk.java.net  Wed Mar  3 17:56:41 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Wed, 3 Mar 2021 17:56:41 GMT
Subject: Integrated: 8262885: Shenandoah: FullGC prologue does not need to
 save/restore heap has_forwarded_object flag
In-Reply-To: <gqSLxq2K_yS8WoWXfyq20OXyP93Vybb94Brg63mFpr0=.5b176a40-0f4b-4b9d-b939-184565afafac@github.com>
References: <gqSLxq2K_yS8WoWXfyq20OXyP93Vybb94Brg63mFpr0=.5b176a40-0f4b-4b9d-b939-184565afafac@github.com>
Message-ID: <GD1wemIHVX-0RzTfwP_WTqK5ZRFA7APtrgoPxkxQMfU=.0305f6ff-6e11-4e0d-b2b0-9e7f93673f65@github.com>

On Tue, 2 Mar 2021 21:40:45 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Another trivial cleanup. FullGC prologue code does not change heap's has_forwarded_objects flag, there is no point to save/restore it.
> 
> Test:
> - [x] hotspot_gc_shenandoah

This pull request has now been integrated.

Changeset: 2d2ef08c
Author:    Zhengyu Gu <zgu at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/2d2ef08c
Stats:     6 lines in 1 file changed: 2 ins; 3 del; 1 mod

8262885: Shenandoah: FullGC prologue does not need to save/restore heap has_forwarded_object flag

Reviewed-by: shade

-------------

PR: https://git.openjdk.java.net/jdk/pull/2797

From github.com+71722661+earthling-amzn at openjdk.java.net  Thu Mar  4 00:21:26 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Thu, 4 Mar 2021 00:21:26 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v20]
In-Reply-To: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
Message-ID: <j5mHY3qOyCSoYTUCede585sx6aL8zl_ZHLeM6J-6QUc=.c55f04a3-a20e-4161-9230-0c1e8248fa07@github.com>

> **This is a work in progress.**
> 
> ## Summary of changes
> The goal of these changes is to allow young cycles to suspend marking in the old generation. When the young cycle is complete, the concurrent old marking will resume.
> * Evaluation of heuristics was pulled into a new `ShenandoahRegulatorThread`. Taking this evaluation out of line from the control thread allows the regulator to suspend (using the cancellation mechanism) old generation marking and start a young generation cycle.
> * Task queues for marking have been moved from `ShenandoahMarkingContext` into `ShenandoahGeneration`. This allows the marking state for the old generation to persist across young generation cycles. The associated `is_complete` state has also been moved to `ShenandoahGeneration`.
> * Old generation marking is bootstrapped by a complete young generation cycle. In this scenario, the mark closures for a young cycle are given reference to the young generation mark queues _and_ the old generation mark queues. Rather than ignore old references as is done for a normal young cycle, they are enqueued in the old generation mark queues. When the young cycle completes, the old generation marking continues using the task queues primed by the preceding young cycle.
> * There is a new flag: `ShenandoahAllowOldMarkingPreemption` (defaults to true). Disabling this option will cause the regulator to schedule  `young` or `global` collects (according to heuristics), but will _not_ schedule `old` collects.
> * The `global` generation is used to support satb and incremental-update modes. The `global` generation is also used for degenerated and implicit/explicit gc requests. Degenerated cycles are not working on this branch and the root cause is understood.
> * The `global` generation is also used for `FullGC`, but this is also broken. The `FullGC` doesn't update the remembered set during compaction. We reckon there is a non-trivial amount of work to fix this.
> * The `MARKING` gc state has been split into `YOUNG_MARKING` and `OLD_MARKING`.
> * Immediate garbage collection is broken in generational mode. The update references phase is used to repair the remembered set, so when this phase is skipped the remembered set scan runs into trouble. A fix for this is in progress.
> * The remembered set is scanned on a safepoint during initial mark. Work to make this concurrent is in progress.

earthling-amzn has updated the pull request incrementally with four additional commits since the last revision:

 - Renable reference processing during old generation marking
   
   This reverts commit 6af15804909715918acb709abb8e006fda988995.
 - Allow global cycles (explicit, implicit gc requests) to preempt old marking
 - Update ascii flow diagram describing transitions between generational collections
 - Purge old generation SATB buffers during final update references phase

-------------

Changes:
  - all: https://git.openjdk.java.net/shenandoah/pull/19/files
  - new: https://git.openjdk.java.net/shenandoah/pull/19/files/90ea8c22..c1e964d6

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=19&range=19
 - incr: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=19&range=18-19

  Stats: 176 lines in 9 files changed: 143 ins; 14 del; 19 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/19.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/19/head:pull/19

PR: https://git.openjdk.java.net/shenandoah/pull/19

From gnu.andrew at redhat.com  Thu Mar  4 03:06:48 2021
From: gnu.andrew at redhat.com (Andrew Hughes)
Date: Thu, 4 Mar 2021 03:06:48 +0000
Subject: [RFR] [8u] 8u292-b02 Upstream Sync
In-Reply-To: <d04971ac-d12d-e4fa-4e97-71d226126073@redhat.com>
References: <20210303023959.GA1412354@rincewind>
 <d04971ac-d12d-e4fa-4e97-71d226126073@redhat.com>
Message-ID: <20210304030648.GA1419804@rincewind>

On 07:51 Wed 03 Mar     , Aleksey Shipilev wrote:
> On 3/3/21 3:39 AM, Andrew Hughes wrote:
> > Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/
> > 
> > Merge changesets:
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/corba/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/jaxp/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/jaxws/merge.changeset
> 
> Looks trivially fine.
> 
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/jdk/merge.changeset
> 
> Looks fine.
> 
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/hotspot/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/langtools/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/nashorn/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b02/root/merge.changeset
> 
> Look trivially fine.
> 
> > Ok to push?
> 
> Yes.
> 
> -- 
> Thanks,
> -Aleksey
> 

Thanks. Pushed.
-- 
Andrew :)

Senior Free Java Software Engineer
OpenJDK Package Owner
Red Hat, Inc. (http://www.redhat.com)

PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04  C5A0 CFDA 0F9B 3596 4222

From rkennke at openjdk.java.net  Thu Mar  4 17:07:57 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Thu, 4 Mar 2021 17:07:57 GMT
Subject: Integrated: 8262122: [TESTBUG] Shenandoah-specific variant of
 TestReferenceRefersTo
In-Reply-To: <izReswRQOulMVo0iSU9-qthYE1Opi5K2L8jfo6Y-V5I=.d97f152a-b99f-4ef3-8095-9eb3a2afa68f@github.com>
References: <izReswRQOulMVo0iSU9-qthYE1Opi5K2L8jfo6Y-V5I=.d97f152a-b99f-4ef3-8095-9eb3a2afa68f@github.com>
Message-ID: <LUPFZjfMAyoaaAg0dFUNghg8_OpIzltVU2p1Im3z6Uc=.ce07e1c5-2918-4e2b-86d9-07983eaad55f@github.com>

On Mon, 22 Feb 2021 11:52:25 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

> Before JDK-8262049, the test TestReferenceRefersTo.java has been failing with I-U mode, because it asserted that weak references would not be cleared when accessed during mark. JDK-8262049 split up the test into a generic part that removed the offending test, and a non-Shenandoah part that contains the test.
> I think it would be useful to add the full test with Shenandoah runners under gc/shenandoah to include it in hotspot_gc_shenandoah runs.
> 
> Test:
>  - [x] TestReferenceRefersToShenandoah.java
>  - [ ] hotspot_gc_shenandoah

This pull request has now been integrated.

Changeset: 222a17ef
Author:    Roman Kennke <rkennke at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/222a17ef
Stats:     325 lines in 1 file changed: 325 ins; 0 del; 0 mod

8262122: [TESTBUG] Shenandoah-specific variant of TestReferenceRefersTo

Reviewed-by: shade

-------------

PR: https://git.openjdk.java.net/jdk/pull/2674

From gnu.andrew at redhat.com  Thu Mar  4 19:03:19 2021
From: gnu.andrew at redhat.com (Andrew Hughes)
Date: Thu, 4 Mar 2021 19:03:19 +0000
Subject: [RFR] [8u] 8u292-b03 Upstream Sync
Message-ID: <20210304190319.GA1506293@rincewind>

Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/

Merge changesets:
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/corba/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/jaxp/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/jaxws/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/jdk/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/hotspot/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/langtools/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/nashorn/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/root/merge.changeset

Changes in aarch64-shenandoah-jdk8u292-b03:
  - JDK-8145051: Wrong parameter name in synthetic lambda method leads to verifier error
  - JDK-8172404: Tools should warn if weak algorithms are used before restricting them
  - JDK-8209333: Socket reset issue for TLS 1.3 socket close
  - JDK-8219991: New fix of the deadlock in sun.security.ssl.SSLSocketImpl
  - JDK-8239091: Reversed arguments in call to strstr in freetype "debug" code.
  - JDK-8240827: Downport SSLSocketImpl.java from "8221882: Use fiber-friendly java.util.concurrent.locks in JSSE"
  - JDK-8255880: UI of Swing components is not redrawn after their internal state changed
  - JDK-8256682: JDK-8202343 is incomplete
  - JDK-8260930: AARCH64: Invalid value passed to critical JNI function

Main issues of note:
Only HotSpot change is JDK-8260930 and there were no conflicts.

diffstat for root
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for corba
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jaxp
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jaxws
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for langtools
 b/.hgtags                                                        |    1 
 b/src/share/classes/com/sun/tools/javac/comp/LambdaToMethod.java |    2 
 b/test/tools/javac/lambda/T8145051.java                          |   43 ++++++++++
 b/test/tools/javac/lambda/pkg/T8145051.java                      |   43 ++++++++++
 4 files changed, 88 insertions(+), 1 deletion(-)

diffstat for nashorn
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jdk
 b/.hgtags                                                                     |    1 
 b/src/share/classes/sun/security/ssl/SSLConfiguration.java                    |    2 
 b/src/share/classes/sun/security/ssl/SSLSocketImpl.java                       |  512 +++++++---
 b/src/share/classes/sun/security/ssl/SSLSocketInputRecord.java                |   13 
 b/src/share/classes/sun/security/tools/jarsigner/Main.java                    |  272 +++--
 b/src/share/classes/sun/security/tools/jarsigner/Resources.java               |   16 
 b/src/share/classes/sun/security/tools/keytool/Main.java                      |   60 -
 b/src/share/classes/sun/security/tools/keytool/Resources.java                 |   10 
 b/src/share/classes/sun/security/util/DisabledAlgorithmConstraints.java       |    5 
 b/src/share/lib/security/java.security-aix                                    |   20 
 b/src/share/lib/security/java.security-linux                                  |   20 
 b/src/share/lib/security/java.security-macosx                                 |   20 
 b/src/share/lib/security/java.security-solaris                                |   20 
 b/src/share/lib/security/java.security-windows                                |   20 
 b/src/share/native/sun/font/freetypeScaler.c                                  |    2 
 b/src/solaris/classes/sun/awt/X11/XFramePeer.java                             |    7 
 b/test/javax/swing/JFrame/8255880/RepaintOnFrameIconifiedStateChangeTest.java |  192 +++
 b/test/sun/security/ssl/SSLSocketImpl/SSLSocketBruceForceClose.java           |  147 ++
 b/test/sun/security/ssl/SSLSocketImpl/SSLSocketClose.java                     |  159 +++
 b/test/sun/security/tools/jarsigner/TimestampCheck.java                       |  219 +++-
 b/test/sun/security/tools/jarsigner/TsacertOptionTest.java                    |    4 
 b/test/sun/security/tools/jarsigner/Warning.java                              |    8 
 b/test/sun/security/tools/jarsigner/concise_jarsigner.sh                      |    6 
 b/test/sun/security/tools/jarsigner/ec.sh                                     |    4 
 b/test/sun/security/tools/jarsigner/nameclash.sh                              |    6 
 b/test/sun/security/tools/keytool/WeakAlg.java                                |  342 +++++-
 b/test/sun/security/util/HostnameMatcher/NullHostnameCheck.java               |   17 
 27 files changed, 1717 insertions(+), 387 deletions(-)

diffstat for hotspot
 b/.hgtags                                                         |    1 +
 b/src/cpu/aarch64/vm/sharedRuntime_aarch64.cpp                    |    2 +-
 b/src/cpu/aarch64/vm/vm_version_aarch64.cpp                       |    7 +++++++
 b/test/compiler/criticalnatives/argumentcorruption/Test8167409.sh |    6 ++++++
 4 files changed, 15 insertions(+), 1 deletion(-)

Successfully built on x86, x86_64, s390 (Zero), s390x (Zero), ppc (Zero),
ppc64, ppc64le, aarch32 (Zero) & aarch64.

Ok to push?

Thanks,
-- 
Andrew :)

Senior Free Java Software Engineer
OpenJDK Package Owner
Red Hat, Inc. (http://www.redhat.com)

PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04  C5A0 CFDA 0F9B 3596 4222

From shade at redhat.com  Thu Mar  4 19:12:32 2021
From: shade at redhat.com (Aleksey Shipilev)
Date: Thu, 4 Mar 2021 20:12:32 +0100
Subject: [RFR] [8u] 8u292-b03 Upstream Sync
In-Reply-To: <20210304190319.GA1506293@rincewind>
References: <20210304190319.GA1506293@rincewind>
Message-ID: <7b9b23b2-faba-17e7-f3c1-7e84529c8c86@redhat.com>

On 3/4/21 8:03 PM, Andrew Hughes wrote:
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/corba/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/jaxp/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/jaxws/merge.changeset

Look trivially good.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/jdk/merge.changeset

Looks fine.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/hotspot/merge.changeset

Looks fine.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/langtools/merge.changeset

Looks fine.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/nashorn/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/root/merge.changeset

Look trivially fine.

> Ok to push?

Yes.


-- 
Thanks,
-Aleksey


From zgu at openjdk.java.net  Thu Mar  4 19:28:53 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 4 Mar 2021 19:28:53 GMT
Subject: RFR: 8263041: Shenandoah: Cleanup C1 keep alive barrier check
Message-ID: <dX7PrXSkRS7mlobZd2Cj4zRYkOsnit2aPlOb_Fwrjz8=.2daba244-7ea1-43dd-8e0e-4226e18570eb@github.com>

Please review this cleanup. It appears that JDK-8233339 missed C1 change.

Test:
- [x] hotspot_gc_shenandoah

-------------

Commit messages:
 - JDK-8263041-c1-keepalive-check
 - update
 - JDK-8262852-varHandles-test

Changes: https://git.openjdk.java.net/jdk/pull/2834/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2834&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8263041
  Stats: 20 lines in 2 files changed: 1 ins; 5 del; 14 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2834.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2834/head:pull/2834

PR: https://git.openjdk.java.net/jdk/pull/2834

From github.com+71722661+earthling-amzn at openjdk.java.net  Fri Mar  5 01:49:21 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Fri, 5 Mar 2021 01:49:21 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v21]
In-Reply-To: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
Message-ID: <18fHB462_MTTqtUWPyjcTTPXQocf2IvjeqhqWNX9Cio=.e8ecbae9-9dbc-47bf-8b75-bb32d787a98b@github.com>

> **This is a work in progress.**
> 
> ## Summary of changes
> The goal of these changes is to allow young cycles to suspend marking in the old generation. When the young cycle is complete, the concurrent old marking will resume.
> * Evaluation of heuristics was pulled into a new `ShenandoahRegulatorThread`. Taking this evaluation out of line from the control thread allows the regulator to suspend (using the cancellation mechanism) old generation marking and start a young generation cycle.
> * Task queues for marking have been moved from `ShenandoahMarkingContext` into `ShenandoahGeneration`. This allows the marking state for the old generation to persist across young generation cycles. The associated `is_complete` state has also been moved to `ShenandoahGeneration`.
> * Old generation marking is bootstrapped by a complete young generation cycle. In this scenario, the mark closures for a young cycle are given reference to the young generation mark queues _and_ the old generation mark queues. Rather than ignore old references as is done for a normal young cycle, they are enqueued in the old generation mark queues. When the young cycle completes, the old generation marking continues using the task queues primed by the preceding young cycle.
> * There is a new flag: `ShenandoahAllowOldMarkingPreemption` (defaults to true). Disabling this option will cause the regulator to schedule  `young` or `global` collects (according to heuristics), but will _not_ schedule `old` collects.
> * The `global` generation is used to support satb and incremental-update modes. The `global` generation is also used for degenerated and implicit/explicit gc requests. Degenerated cycles are not working on this branch and the root cause is understood.
> * The `global` generation is also used for `FullGC`, but this is also broken. The `FullGC` doesn't update the remembered set during compaction. We reckon there is a non-trivial amount of work to fix this.
> * The `MARKING` gc state has been split into `YOUNG_MARKING` and `OLD_MARKING`.
> * Immediate garbage collection is broken in generational mode. The update references phase is used to repair the remembered set, so when this phase is skipped the remembered set scan runs into trouble. A fix for this is in progress.
> * The remembered set is scanned on a safepoint during initial mark. Work to make this concurrent is in progress.

earthling-amzn has updated the pull request incrementally with three additional commits since the last revision:

 - Refine protocol between regulator and control threads
   
   Stop using gc id to "hold down" regulator request. Control thread now has a "gc mode". Regulator requests are only made when control thread is in certain modes (idle, marking old).
 - Do not provide old mark queue when purging old satb buffers
   
   This may cause young pointers to be marked and queued, and then cleared from the queue resulting in an incomplete transitive closure.
 - Include generation name in heuristic trigger log message

-------------

Changes:
  - all: https://git.openjdk.java.net/shenandoah/pull/19/files
  - new: https://git.openjdk.java.net/shenandoah/pull/19/files/c1e964d6..ac246491

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=19&range=20
 - incr: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=19&range=19-20

  Stats: 180 lines in 9 files changed: 72 ins; 41 del; 67 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/19.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/19/head:pull/19

PR: https://git.openjdk.java.net/shenandoah/pull/19

From aph at redhat.com  Fri Mar  5 09:53:11 2021
From: aph at redhat.com (Andrew Haley)
Date: Fri, 5 Mar 2021 09:53:11 +0000
Subject: [8u] RFC: Pick up recent aarch64-port cleanups
In-Reply-To: <f79e70de-0765-b3da-aa93-2035f3ef5021@redhat.com>
References: <f79e70de-0765-b3da-aa93-2035f3ef5021@redhat.com>
Message-ID: <227d3a02-1b1f-472e-daef-fb470ec19ff2@redhat.com>

On 3/1/21 11:42 AM, Aleksey Shipilev wrote:
> There are a few code improvements in aarch64-port that we want to pick up and test before proposing 
> the integration back. The pickup is clean, only affects hotspot repository, and gets merged 
> automatically.
> 
> Testing: hotspot_gc_shenandoah {fastdebug,release}

There's nothing here!

-- 
Andrew Haley  (he/him)
Java Platform Lead Engineer
Red Hat UK Ltd. <https://www.redhat.com>
https://keybase.io/andrewhaley
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671


From shade at redhat.com  Fri Mar  5 09:58:38 2021
From: shade at redhat.com (Aleksey Shipilev)
Date: Fri, 5 Mar 2021 10:58:38 +0100
Subject: [8u] RFC: Pick up recent aarch64-port cleanups
In-Reply-To: <227d3a02-1b1f-472e-daef-fb470ec19ff2@redhat.com>
References: <f79e70de-0765-b3da-aa93-2035f3ef5021@redhat.com>
 <227d3a02-1b1f-472e-daef-fb470ec19ff2@redhat.com>
Message-ID: <986a0779-6e34-240c-3ad7-a0ec64102c63@redhat.com>

On 3/5/21 10:53 AM, Andrew Haley wrote:
> On 3/1/21 11:42 AM, Aleksey Shipilev wrote:
>> There are a few code improvements in aarch64-port that we want to pick up and test before proposing
>> the integration back. The pickup is clean, only affects hotspot repository, and gets merged
>> automatically.
>>
>> Testing: hotspot_gc_shenandoah {fastdebug,release}
> 
> There's nothing here!

Yes, we don't post useless RFRs for clean pickups. This is why this is "RFC" :)

This RFC is moot now, as Andrew Hughes is merging ...b05 to aarch64-port. We would need to pick that 
up first.

-- 
Thanks,
-Aleksey


From shade at openjdk.java.net  Fri Mar  5 16:02:06 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Fri, 5 Mar 2021 16:02:06 GMT
Subject: RFR: 8263041: Shenandoah: Cleanup C1 keep alive barrier check
In-Reply-To: <dX7PrXSkRS7mlobZd2Cj4zRYkOsnit2aPlOb_Fwrjz8=.2daba244-7ea1-43dd-8e0e-4226e18570eb@github.com>
References: <dX7PrXSkRS7mlobZd2Cj4zRYkOsnit2aPlOb_Fwrjz8=.2daba244-7ea1-43dd-8e0e-4226e18570eb@github.com>
Message-ID: <0HOS-bm4WG864dv-bWKCcMCoCS-evRBsIyQJU1YGO68=.4b924842-9736-4ae8-a8db-8c5e39332853@github.com>

On Thu, 4 Mar 2021 19:23:06 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Please review this cleanup. It appears that JDK-8233339 missed C1 change.
> 
> Test:
> - [x] hotspot_gc_shenandoah

Changes requested by shade (Reviewer).

src/hotspot/share/gc/shenandoah/shenandoahReferenceProcessor.cpp line 158:

> 156: ShenandoahRefProcThreadLocal::ShenandoahRefProcThreadLocal() :
> 157:   _discovered_list(NULL),
> 158:   _mark_closure(NULL),

The changes in this file do not look relevant to the issue at hand?

src/hotspot/share/gc/shenandoah/c1/shenandoahBarrierSetC1.cpp line 224:

> 222: 
> 223:   // 3: apply keep-alive barrier if ShenandoahSATBBarrier is set
> 224:   if (ShenandoahSATBBarrier && ShenandoahBarrierSet::need_keep_alive_barrier(decorators, type)) {

I wonder if it is actually valid to check for SATB barrier here. `ShenandoahBarrierSetC2::load_at_resolved` only does `need_keep_alive_barrier` check.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2834

From gnu.andrew at redhat.com  Fri Mar  5 17:16:42 2021
From: gnu.andrew at redhat.com (Andrew Hughes)
Date: Fri, 5 Mar 2021 17:16:42 +0000
Subject: [RFR] [8u] 8u292-b03 Upstream Sync
In-Reply-To: <7b9b23b2-faba-17e7-f3c1-7e84529c8c86@redhat.com>
References: <20210304190319.GA1506293@rincewind>
 <7b9b23b2-faba-17e7-f3c1-7e84529c8c86@redhat.com>
Message-ID: <20210305171642.GA1605474@rincewind>

On 20:12 Thu 04 Mar     , Aleksey Shipilev wrote:
> On 3/4/21 8:03 PM, Andrew Hughes wrote:
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/corba/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/jaxp/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/jaxws/merge.changeset
> 
> Look trivially good.
> 
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/jdk/merge.changeset
> 
> Looks fine.
> 
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/hotspot/merge.changeset
> 
> Looks fine.
> 
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/langtools/merge.changeset
> 
> Looks fine.
> 
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/nashorn/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b03/root/merge.changeset
> 
> Look trivially fine.
> 
> > Ok to push?
> 
> Yes.
> 
> 
> -- 
> Thanks,
> -Aleksey
> 

Thanks. Pushed.
-- 
Andrew :)

Senior Free Java Software Engineer
OpenJDK Package Owner
Red Hat, Inc. (http://www.redhat.com)

PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04  C5A0 CFDA 0F9B 3596 4222

From gnu.andrew at redhat.com  Fri Mar  5 17:59:47 2021
From: gnu.andrew at redhat.com (Andrew Hughes)
Date: Fri, 5 Mar 2021 17:59:47 +0000
Subject: [RFR] [8u] 8u292-b04 Upstream Sync
Message-ID: <20210305175947.GB1605474@rincewind>

Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/

Merge changesets:
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/corba/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/jaxp/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/jaxws/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/jdk/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/hotspot/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/langtools/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/nashorn/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/root/merge.changeset

Changes in aarch64-shenandoah-jdk8u292-b04:
  - JDK-8061777: (zipfs) IllegalArgumentException in ZipCoder.toString when using Shitft_JIS
  - JDK-8158525: Update a few java/net tests to use the loopback address instead of the host address
  - JDK-8171410: aarch64: long multiplyExact shifts by 31 instead of 63
  - JDK-8225435: Upgrade IANA Language Subtag Registry to the latest for JDK14
  - JDK-8235263: Revert TLS 1.3 change that wrapped IOExceptions
  - JDK-8235311: Tag mismatch may alert bad_record_mac
  - JDK-8236500: Windows ucrt.dll should be looked up in versioned WINSDK subdirectory
  - JDK-8260349: Cannot programmatically retrieve Metaspace max set via JAVA_TOOL_OPTIONS
  - JDK-8261766: [8u] hotspot needs to recognise cl.exe 19.16 to build with VS2017
  - JDK-8262075: sun/security/krb5/auto/UseCacheAndStoreKey.java timed out intermittently

Main issues of note:
None, clean merge (no conflics in HotSpot code).

diffstat for root
 b/.hgtags                                |    1 +
 b/common/autoconf/generated-configure.sh |   17 +++++++++++++----
 b/common/autoconf/toolchain_windows.m4   |   12 ++++++++++--
 3 files changed, 24 insertions(+), 6 deletions(-)

diffstat for corba
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jaxp
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jaxws
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for langtools
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for nashorn
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jdk
 b/.hgtags                                                           |    1 
 b/make/data/lsrdata/language-subtag-registry.txt                    |  203 ++++++++--
 b/src/share/classes/sun/security/krb5/Config.java                   |    6 
 b/src/share/classes/sun/security/ssl/SSLSocketImpl.java             |    8 
 b/src/share/classes/sun/security/ssl/SSLTransport.java              |    9 
 b/src/share/demo/nio/zipfs/src/com/sun/nio/zipfs/ZipFileSystem.java |    4 
 b/src/share/demo/nio/zipfs/src/com/sun/nio/zipfs/ZipPath.java       |   67 +++
 b/test/demo/zipfs/ZFSTests.java                                     |   31 +
 b/test/java/net/CookieHandler/CookieManagerTest.java                |   37 +
 b/test/java/net/HttpURLConnection/UnmodifiableMaps.java             |    8 
 b/test/java/net/URLConnection/HandleContentTypeWithAttrs.java       |    7 
 b/test/java/util/Locale/Bug8040211.java                             |  158 ++++---
 12 files changed, 406 insertions(+), 133 deletions(-)

diffstat for hotspot
 b/.hgtags                                                |    1 
 b/make/windows/makefiles/compile.make                    |    9 +
 b/src/cpu/aarch64/vm/aarch64.ad                          |    8 -
 b/src/share/vm/services/memoryPool.cpp                   |    6 
 b/test/runtime/Metaspace/MaxMetaspaceSizeEnvVarTest.java |  120 +++++++++++++++
 5 files changed, 137 insertions(+), 7 deletions(-)

Successfully built on x86, x86_64, s390 (Zero), s390x (Zero), ppc (Zero),
ppc64, ppc64le, aarch32 (Zero) & aarch64.

Ok to push?

Thanks,
-- 
Andrew :)

Senior Free Java Software Engineer
OpenJDK Package Owner
Red Hat, Inc. (http://www.redhat.com)

PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04  C5A0 CFDA 0F9B 3596 4222

From github.com+71722661+earthling-amzn at openjdk.java.net  Sat Mar  6 01:24:44 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Sat, 6 Mar 2021 01:24:44 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v22]
In-Reply-To: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
Message-ID: <Hwh0wMz-0SzrIxNGLPuiffYF3mPWB2Bs4ErYd12z1j0=.6b80e37e-f07c-49be-96c1-7fb30751cebd@github.com>

> **This is a work in progress.**
> 
> ## Summary of changes
> The goal of these changes is to allow young cycles to suspend marking in the old generation. When the young cycle is complete, the concurrent old marking will resume.
> * Evaluation of heuristics was pulled into a new `ShenandoahRegulatorThread`. Taking this evaluation out of line from the control thread allows the regulator to suspend (using the cancellation mechanism) old generation marking and start a young generation cycle.
> * Task queues for marking have been moved from `ShenandoahMarkingContext` into `ShenandoahGeneration`. This allows the marking state for the old generation to persist across young generation cycles. The associated `is_complete` state has also been moved to `ShenandoahGeneration`.
> * Old generation marking is bootstrapped by a complete young generation cycle. In this scenario, the mark closures for a young cycle are given reference to the young generation mark queues _and_ the old generation mark queues. Rather than ignore old references as is done for a normal young cycle, they are enqueued in the old generation mark queues. When the young cycle completes, the old generation marking continues using the task queues primed by the preceding young cycle.
> * There is a new flag: `ShenandoahAllowOldMarkingPreemption` (defaults to true). Disabling this option will cause the regulator to schedule  `young` or `global` collects (according to heuristics), but will _not_ schedule `old` collects.
> * The `global` generation is used to support satb and incremental-update modes. The `global` generation is also used for degenerated and implicit/explicit gc requests. Degenerated cycles are not working on this branch and the root cause is understood.
> * The `global` generation is also used for `FullGC`, but this is also broken. The `FullGC` doesn't update the remembered set during compaction. We reckon there is a non-trivial amount of work to fix this.
> * The `MARKING` gc state has been split into `YOUNG_MARKING` and `OLD_MARKING`.
> * Immediate garbage collection is broken in generational mode. The update references phase is used to repair the remembered set, so when this phase is skipped the remembered set scan runs into trouble. A fix for this is in progress.
> * The remembered set is scanned on a safepoint during initial mark. Work to make this concurrent is in progress.

earthling-amzn has updated the pull request incrementally with three additional commits since the last revision:

 - Don't let other threads change gc mode
 - Completely abandon old SATB buffers when beginning a global collection
 - Cancel old marking at start of STW collects

-------------

Changes:
  - all: https://git.openjdk.java.net/shenandoah/pull/19/files
  - new: https://git.openjdk.java.net/shenandoah/pull/19/files/ac246491..34375b77

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=19&range=21
 - incr: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=19&range=20-21

  Stats: 49 lines in 10 files changed: 13 ins; 22 del; 14 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/19.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/19/head:pull/19

PR: https://git.openjdk.java.net/shenandoah/pull/19

From shade at redhat.com  Sat Mar  6 10:58:07 2021
From: shade at redhat.com (Aleksey Shipilev)
Date: Sat, 6 Mar 2021 11:58:07 +0100
Subject: [RFR] [8u] 8u292-b04 Upstream Sync
In-Reply-To: <20210305175947.GB1605474@rincewind>
References: <20210305175947.GB1605474@rincewind>
Message-ID: <514c7a9f-f6fd-9380-dac4-9c2c2d971a4a@redhat.com>

On 3/5/21 6:59 PM, Andrew Hughes wrote:
> Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/
> 
> Merge changesets:
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/corba/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/jaxp/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/jaxws/merge.changeset

Look trivially good.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/jdk/merge.changeset

Looks good.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/hotspot/merge.changeset

Looks good.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/langtools/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/nashorn/merge.changeset

Look trivially good.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/root/merge.changeset

Looks good.

> Ok to push?

Yes.

-- 
Thanks,
-Aleksey


From zgu at openjdk.java.net  Mon Mar  8 15:04:28 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 8 Mar 2021 15:04:28 GMT
Subject: RFR: 8263041: Shenandoah: Cleanup C1 keep alive barrier check [v2]
In-Reply-To: <dX7PrXSkRS7mlobZd2Cj4zRYkOsnit2aPlOb_Fwrjz8=.2daba244-7ea1-43dd-8e0e-4226e18570eb@github.com>
References: <dX7PrXSkRS7mlobZd2Cj4zRYkOsnit2aPlOb_Fwrjz8=.2daba244-7ea1-43dd-8e0e-4226e18570eb@github.com>
Message-ID: <DODkPy9GEGACTDUxrOTRm2Ns0r2g0DDwpe5sJZjJFLw=.40086f90-7b92-4130-b793-06ef77d071ad@github.com>

> Please review this cleanup. It appears that JDK-8233339 missed C1 change.
> 
> Test:
> - [x] hotspot_gc_shenandoah

Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:

  Aleksey's comment

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2834/files
  - new: https://git.openjdk.java.net/jdk/pull/2834/files/e324e007..41c6c8d7

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2834&range=01
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2834&range=00-01

  Stats: 4 lines in 3 files changed: 0 ins; 1 del; 3 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2834.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2834/head:pull/2834

PR: https://git.openjdk.java.net/jdk/pull/2834

From zgu at openjdk.java.net  Mon Mar  8 15:32:25 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 8 Mar 2021 15:32:25 GMT
Subject: RFR: 8263041: Shenandoah: Cleanup C1 keep alive barrier check [v3]
In-Reply-To: <dX7PrXSkRS7mlobZd2Cj4zRYkOsnit2aPlOb_Fwrjz8=.2daba244-7ea1-43dd-8e0e-4226e18570eb@github.com>
References: <dX7PrXSkRS7mlobZd2Cj4zRYkOsnit2aPlOb_Fwrjz8=.2daba244-7ea1-43dd-8e0e-4226e18570eb@github.com>
Message-ID: <5b8yZ0g1mrBsfDvn9edDBwZ3rBIjtaWtEwT4Pf5OAtA=.3c739a31-2d1f-4ba7-8689-8bb4ab309962@github.com>

> Please review this cleanup. It appears that JDK-8233339 missed C1 change.
> 
> Test:
> - [x] hotspot_gc_shenandoah

Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:

  Removed unrelated change

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2834/files
  - new: https://git.openjdk.java.net/jdk/pull/2834/files/41c6c8d7..63589415

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2834&range=02
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2834&range=01-02

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2834.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2834/head:pull/2834

PR: https://git.openjdk.java.net/jdk/pull/2834

From jbachorik at openjdk.java.net  Mon Mar  8 15:34:23 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 8 Mar 2021 15:34:23 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v10]
In-Reply-To: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
Message-ID: <ggKKpH_AhC5b_Ecfbk8CQkCRUGl4IEDxinUmdR2OrAk=.2d32aa46-2b0e-4699-8d35-62995d5b226c@github.com>

> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
> 
> ## Introducing new JFR event
> 
> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
> 
> ## Implementation
> 
> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
> 
> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
> 
> ### Epsilon GC
> 
> Trivial implementation - just return `used()` instead.
> 
> ### Serial GC
> 
> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
> 
> ### Parallel GC
> 
> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
> 
> ### G1 GC
> 
> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
> 
> ### Shenandoah
> 
> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
> 
> ### ZGC
> 
> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.

Jaroslav Bachorik has updated the pull request incrementally with two additional commits since the last revision:

 - Adjust the deadspace calculation
 - Minor cleanup

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2579/files
  - new: https://git.openjdk.java.net/jdk/pull/2579/files/f6954186..f708023b

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=09
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=08-09

  Stats: 28 lines in 7 files changed: 11 ins; 7 del; 10 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2579.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2579/head:pull/2579

PR: https://git.openjdk.java.net/jdk/pull/2579

From shade at openjdk.java.net  Mon Mar  8 16:03:07 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 8 Mar 2021 16:03:07 GMT
Subject: RFR: 8263041: Shenandoah: Cleanup C1 keep alive barrier check [v3]
In-Reply-To: <5b8yZ0g1mrBsfDvn9edDBwZ3rBIjtaWtEwT4Pf5OAtA=.3c739a31-2d1f-4ba7-8689-8bb4ab309962@github.com>
References: <dX7PrXSkRS7mlobZd2Cj4zRYkOsnit2aPlOb_Fwrjz8=.2daba244-7ea1-43dd-8e0e-4226e18570eb@github.com>
 <5b8yZ0g1mrBsfDvn9edDBwZ3rBIjtaWtEwT4Pf5OAtA=.3c739a31-2d1f-4ba7-8689-8bb4ab309962@github.com>
Message-ID: <Sfdp5Z5zzXHBO7qOGzpdMZ5wBRO--T0-n1_Qiaq7kIU=.9fe43c4c-56bd-44d1-8b08-c1311173ef1f@github.com>

On Mon, 8 Mar 2021 15:32:25 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> Please review this cleanup. It appears that JDK-8233339 missed C1 change.
>> 
>> Test:
>> - [x] hotspot_gc_shenandoah
>
> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Removed unrelated change

Looks good, minor nit.

src/hotspot/share/gc/shenandoah/c2/shenandoahBarrierSetC2.cpp line 554:

> 552:   }
> 553: 
> 554:   // 3: apply keep-alive barrier for java.lang.ref.Reference if needed

"For java.lang.ref.Reference"?

I think BarrierSetC2 comment if pretty generic, so we might want to match it.

-------------

Marked as reviewed by shade (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2834

From rkennke at openjdk.java.net  Mon Mar  8 16:39:09 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 8 Mar 2021 16:39:09 GMT
Subject: RFR: 8263041: Shenandoah: Cleanup C1 keep alive barrier check [v3]
In-Reply-To: <5b8yZ0g1mrBsfDvn9edDBwZ3rBIjtaWtEwT4Pf5OAtA=.3c739a31-2d1f-4ba7-8689-8bb4ab309962@github.com>
References: <dX7PrXSkRS7mlobZd2Cj4zRYkOsnit2aPlOb_Fwrjz8=.2daba244-7ea1-43dd-8e0e-4226e18570eb@github.com>
 <5b8yZ0g1mrBsfDvn9edDBwZ3rBIjtaWtEwT4Pf5OAtA=.3c739a31-2d1f-4ba7-8689-8bb4ab309962@github.com>
Message-ID: <OPrY9aXfmsXN_3-S04eJP6gaTlNj7eWLvQ4YhRvREpg=.e75b7139-f33f-4803-afc7-2bb64afc994f@github.com>

On Mon, 8 Mar 2021 15:32:25 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> Please review this cleanup. It appears that JDK-8233339 missed C1 change.
>> 
>> Test:
>> - [x] hotspot_gc_shenandoah
>
> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Removed unrelated change

Looks good!

-------------

Marked as reviewed by rkennke (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2834

From jbachorik at openjdk.java.net  Mon Mar  8 17:08:27 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 8 Mar 2021 17:08:27 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v11]
In-Reply-To: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
Message-ID: <Tmo_cklfSVkpgc_oEmQmcskQKgLlqqnVNrNc34-Bbbc=.e811b536-c8cd-40e7-8eb8-59319b043064@github.com>

> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
> 
> ## Introducing new JFR event
> 
> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
> 
> ## Implementation
> 
> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
> 
> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
> 
> ### Epsilon GC
> 
> Trivial implementation - just return `used()` instead.
> 
> ### Serial GC
> 
> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
> 
> ### Parallel GC
> 
> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
> 
> ### G1 GC
> 
> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
> 
> ### Shenandoah
> 
> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
> 
> ### ZGC
> 
> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.

Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:

  Cache live size estimate for memory spaces

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2579/files
  - new: https://git.openjdk.java.net/jdk/pull/2579/files/f708023b..343e4809

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=10
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=09-10

  Stats: 17 lines in 3 files changed: 13 ins; 1 del; 3 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2579.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2579/head:pull/2579

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar  8 17:17:11 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 8 Mar 2021 17:17:11 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v9]
In-Reply-To: <l0lr-q29vacGe0upkTwrhXwi7sFwH-Jvcc-PxIobLJM=.aa5b21aa-f0ec-461d-aa32-87c5a211c177@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <K4Ac7D3fQw_lxHfpya748uuBMet058eaHA_PgVFadzc=.4f7d9350-f1a2-4e29-b3db-6e387a2f1369@github.com>
 <l0lr-q29vacGe0upkTwrhXwi7sFwH-Jvcc-PxIobLJM=.aa5b21aa-f0ec-461d-aa32-87c5a211c177@github.com>
Message-ID: <MaGVKajQYw1OgoX_Scp_ZKhd1CDLkPAsV48LKOBM84s=.372064e9-aa41-42aa-b35e-4287f53714c0@github.com>

On Wed, 3 Mar 2021 12:13:43 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

>> Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Add tests for the heap usage summary event
>
> src/hotspot/share/gc/shared/space.hpp line 553:
> 
>> 551:   size_t capacity() const        { return byte_size(bottom(), end()); }
>> 552:   size_t used() const            { return byte_size(bottom(), top()); }
>> 553:   size_t live() const            {
> 
> The code for serial gc, contrary to others, tries to give some resemblance of tracking actual liveness. I.e. calculating this anew every call to `SerialHeap::live()`.
> However if calling an `update_live_estimate()` in parallel and G1 (and the other collectors) is fine at certain places, this should be as good for serial gc.
> Doing so would reduce the footprint of this change quite a bit (for serial gc)

Ok. I am caching the live estimate per memory space now. 
Not sure how much it will change the footprint of this change, though, but it is good for consistency anyway.

> src/hotspot/share/gc/shared/space.inline.hpp line 128:
> 
>> 126:           p2i(dead_start), p2i(dead_end), dead_length * HeapWordSize);
>> 127: 
>> 128:       _dead_space += dead_length;
> 
> I do not think adding this to the counter here instead of the other method for every object makes a difference performance-wise.
> 
> As mentioned before, `_allowed_deadspace_words` counts *down* from `(space->capacity() * ratio / 100) / HeapWordSize;` to whatever end value.
> 
> So at the end of collection, `(space->capacity() * ratio / 100) / HeapWordSize - _allowed_deadspace_words` should be equal to what `_dead_space` is now.
> 
> Please add a getter to `DeadSpacer` that calculates this (factoring out the calculation of the maximum allowed deadspace).

Fixed in https://github.com/openjdk/jdk/pull/2579/commits/f708023b84e33aee34f92d183a0d03d2747646db

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar  8 17:29:27 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 8 Mar 2021 17:29:27 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
Message-ID: <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>

> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
> 
> ## Introducing new JFR event
> 
> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
> 
> ## Implementation
> 
> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
> 
> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
> 
> ### Epsilon GC
> 
> Trivial implementation - just return `used()` instead.
> 
> ### Serial GC
> 
> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
> 
> ### Parallel GC
> 
> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
> 
> ### G1 GC
> 
> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
> 
> ### Shenandoah
> 
> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
> 
> ### ZGC
> 
> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.

Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:

  Remove unused field

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2579/files
  - new: https://git.openjdk.java.net/jdk/pull/2579/files/343e4809..67d78940

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=11
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=10-11

  Stats: 2 lines in 1 file changed: 0 ins; 2 del; 0 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2579.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2579/head:pull/2579

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar  8 17:29:30 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 8 Mar 2021 17:29:30 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v9]
In-Reply-To: <l0lr-q29vacGe0upkTwrhXwi7sFwH-Jvcc-PxIobLJM=.aa5b21aa-f0ec-461d-aa32-87c5a211c177@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <K4Ac7D3fQw_lxHfpya748uuBMet058eaHA_PgVFadzc=.4f7d9350-f1a2-4e29-b3db-6e387a2f1369@github.com>
 <l0lr-q29vacGe0upkTwrhXwi7sFwH-Jvcc-PxIobLJM=.aa5b21aa-f0ec-461d-aa32-87c5a211c177@github.com>
Message-ID: <cCXgqgbD23CyvxtzXBN1YB1I0EwmxKE0U-2EI8kJdT8=.16961055-ab68-4563-9c71-9184e47b38bd@github.com>

On Wed, 3 Mar 2021 12:03:01 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

>> Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Add tests for the heap usage summary event
>
> src/hotspot/share/gc/g1/g1CollectedHeap.hpp line 182:
> 
>> 180:   G1BlockOffsetTable* _bot;
>> 181: 
>> 182:   volatile size_t _live;
> 
> I'm not happy with naming this `_live`, better use `_live_estimate`. The contents are not continuously updated and basically out of date after the first following allocation.
> This includes the naming in all other instances too.

I see your point - but that would probably lead to renaming `live()` method to `live_estimate()` (to keep the variable and the accessor method in sync) and that would break the nice symmetry we have now with `free()`, `used()` and `live()`. 
I have no strong feelings about this and if we can get quorum on this change I will do the renaming pass.

> src/hotspot/share/gc/parallel/parallelScavengeHeap.hpp line 87:
> 
>> 85: 
>> 86:   // in order to provide accurate estimate this method must be called only when the heap has just been collected and compacted
>> 87:   inline void capture_live();
> 
> Sentences should start with upper case in the comment. Also I'd prefer to name the method `update_live_estimate()` instead.

Done

> src/hotspot/share/gc/parallel/psAdaptiveSizePolicy.hpp line 60:
> 
>> 58: class PSAdaptiveSizePolicy : public AdaptiveSizePolicy {
>> 59:  friend class PSGCAdaptivePolicyCounters;
>> 60:  friend class ParallelScavengeHeap;
> 
> Delete this apparently unneeded friend declaration (compiled successfully without here)

Done

> src/hotspot/share/gc/g1/g1ConcurrentMark.cpp line 1070:
> 
>> 1068: 
>> 1069:     uint num_selected_for_rebuild() const { return _num_regions_selected_for_rebuild; }
>> 1070:     size_t live_estimate() const { return _live; }
> 
> Please sync the member name with the getter name. I.e. `_live` -> `_live_estimate`

Done

> src/hotspot/share/gc/serial/serialHeap.hpp line 44:
> 
>> 42:   MemoryPool* _old_pool;
>> 43: 
>> 44:   size_t _live_size;
> 
> Please rename to `_live_estimate` like the others. Avoid having different names in different collectors for the same thing.

This field is unused. Removed.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar  8 17:34:12 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 8 Mar 2021 17:34:12 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v9]
In-Reply-To: <yI-GJzGp2ZSTlYk-eoDVI6WCiJABdUW3PqKH-g51rnc=.019108bb-e8b6-4d82-8fed-7a141adf9774@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <K4Ac7D3fQw_lxHfpya748uuBMet058eaHA_PgVFadzc=.4f7d9350-f1a2-4e29-b3db-6e387a2f1369@github.com>
 <yI-GJzGp2ZSTlYk-eoDVI6WCiJABdUW3PqKH-g51rnc=.019108bb-e8b6-4d82-8fed-7a141adf9774@github.com>
Message-ID: <mET-7bCIP96IQwUJIvL94AI830fJtAX85gv3EMsHPKY=.ff19c474-979b-4d19-8c7b-84212984a9df@github.com>

On Tue, 2 Mar 2021 17:34:32 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Add tests for the heap usage summary event
>
> src/hotspot/share/gc/shared/space.inline.hpp line 190:
> 
>> 188:         oop obj = oop(cur_obj);
>> 189:         size_t obj_size = obj->size();
>> 190:         compact_top = cp->space->forward(obj, obj_size, cp, compact_top);
> 
> This change seems superfluous now. Inline `obj_size` back?

Done

> src/hotspot/share/gc/shared/space.hpp line 555:
> 
>> 553:   size_t live() const            {
>> 554:     return used() - _dead_space;
>> 555:   }
> 
> Move it a few lines down, so `capacity`, `used`, `live` line up?

Ok. I did my best to make the code look nice there :)

> src/hotspot/share/gc/shared/collectedHeap.hpp line 218:
> 
>> 216:   virtual size_t capacity() const = 0;
>> 217:   virtual size_t used() const = 0;
>> 218:   // Returns the estimate of live set size. Because live set changes over time,
> 
> I believe a blank line is in order here, look at other comments in the same header.

Done

> src/hotspot/share/gc/shared/space.inline.hpp line 90:
> 
>> 88: 
>> 89: public:
>> 90:   size_t _dead_space;
> 
> Should this really be "public"? Maybe `friend`-ing with the only user is better?

Yep. Befriended.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From zgu at openjdk.java.net  Mon Mar  8 18:11:07 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 8 Mar 2021 18:11:07 GMT
Subject: Integrated: 8263041: Shenandoah: Cleanup C1 keep alive barrier check
In-Reply-To: <dX7PrXSkRS7mlobZd2Cj4zRYkOsnit2aPlOb_Fwrjz8=.2daba244-7ea1-43dd-8e0e-4226e18570eb@github.com>
References: <dX7PrXSkRS7mlobZd2Cj4zRYkOsnit2aPlOb_Fwrjz8=.2daba244-7ea1-43dd-8e0e-4226e18570eb@github.com>
Message-ID: <-4OKgMnk5qx1KKb1a_JJRrTPYQSCLCQ1Ot4gDyZVWmk=.2dfeeced-047a-44a8-8fae-8f127a3f648d@github.com>

On Thu, 4 Mar 2021 19:23:06 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Please review this cleanup. It appears that JDK-8233339 missed C1 change.
> 
> Test:
> - [x] hotspot_gc_shenandoah

This pull request has now been integrated.

Changeset: a2b88581
Author:    Zhengyu Gu <zgu at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/a2b88581
Stats:     20 lines in 2 files changed: 0 ins; 5 del; 15 mod

8263041: Shenandoah: Cleanup C1 keep alive barrier check

Reviewed-by: shade, rkennke

-------------

PR: https://git.openjdk.java.net/jdk/pull/2834

From github.com+71722661+earthling-amzn at openjdk.java.net  Mon Mar  8 19:04:45 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Mon, 8 Mar 2021 19:04:45 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v23]
In-Reply-To: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
Message-ID: <0WV_jNaa5NlDA-aqTjngV_P-9_RJ7ABqjgIctYJP-Ls=.40145567-8821-4a46-89db-06cd05a02898@github.com>

> **This is a work in progress.**
> 
> ## Summary of changes
> The goal of these changes is to allow young cycles to suspend marking in the old generation. When the young cycle is complete, the concurrent old marking will resume.
> * Evaluation of heuristics was pulled into a new `ShenandoahRegulatorThread`. Taking this evaluation out of line from the control thread allows the regulator to suspend (using the cancellation mechanism) old generation marking and start a young generation cycle.
> * Task queues for marking have been moved from `ShenandoahMarkingContext` into `ShenandoahGeneration`. This allows the marking state for the old generation to persist across young generation cycles. The associated `is_complete` state has also been moved to `ShenandoahGeneration`.
> * Old generation marking is bootstrapped by a complete young generation cycle. In this scenario, the mark closures for a young cycle are given reference to the young generation mark queues _and_ the old generation mark queues. Rather than ignore old references as is done for a normal young cycle, they are enqueued in the old generation mark queues. When the young cycle completes, the old generation marking continues using the task queues primed by the preceding young cycle.
> * There is a new flag: `ShenandoahAllowOldMarkingPreemption` (defaults to true). Disabling this option will cause the regulator to schedule  `young` or `global` collects (according to heuristics), but will _not_ schedule `old` collects.
> * The `global` generation is used to support satb and incremental-update modes. The `global` generation is also used for degenerated and implicit/explicit gc requests. Degenerated cycles are not working on this branch and the root cause is understood.
> * The `global` generation is also used for `FullGC`, but this is also broken. The `FullGC` doesn't update the remembered set during compaction. We reckon there is a non-trivial amount of work to fix this.
> * The `MARKING` gc state has been split into `YOUNG_MARKING` and `OLD_MARKING`.
> * Immediate garbage collection is broken in generational mode. The update references phase is used to repair the remembered set, so when this phase is skipped the remembered set scan runs into trouble. A fix for this is in progress.
> * The remembered set is scanned on a safepoint during initial mark. Work to make this concurrent is in progress.

earthling-amzn has updated the pull request incrementally with three additional commits since the last revision:

 - Correct typo in assert message
 - Update generation memory usage during promotion
 - Report how long it takes to clear a cancel GC request

-------------

Changes:
  - all: https://git.openjdk.java.net/shenandoah/pull/19/files
  - new: https://git.openjdk.java.net/shenandoah/pull/19/files/34375b77..d483bd43

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=19&range=22
 - incr: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=19&range=21-22

  Stats: 19 lines in 7 files changed: 12 ins; 2 del; 5 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/19.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/19/head:pull/19

PR: https://git.openjdk.java.net/shenandoah/pull/19

From rkennke at openjdk.java.net  Mon Mar  8 19:56:29 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 8 Mar 2021 19:56:29 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v14]
In-Reply-To: <M-FFynDq_S1yB8vUyLyfwsDQWGBOkIREMWlPdj9k9jA=.07334715-7e3a-4882-b7fb-4d890a065be2@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
 <7CiRCYgYXysI9lWIUUayxOqdXZAZ7alOWWtNXMMDDjQ=.6020f939-907e-4790-9c2b-93230c41dcb8@github.com>
 <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
 <YV2nQ2TwAJwMA2r4Hh0GyrRfIJmgoVH-6ytlfwIAhgk=.fd8c180a-73be-4c3a-9c9b-c39084e15f76@github.com>
 <M-FFynDq_S1yB8vUyLyfwsDQWGBOkIREMWlPdj9k9jA=.07334715-7e3a-4882-b7fb-4d890a065be2@github.com>
Message-ID: <NyvZ6F-sfK8txz998onIC7avO6qyxkbLb7ziIM2QYJk=.3f5f5d3e-cc5a-4243-8f6a-e52692388bcd@github.com>

On Wed, 3 Mar 2021 16:26:06 GMT, Bernd Mathiske <bmathiske at openjdk.org> wrote:

>> Do you plan to enable reference processing again on old-gen marking? Because I think it's a bad idea to not do it. Ideally, we'd do reference processing on both young-gen and old-gen markings, if that is not possible, then we should at least do it on old-gen markings (I believe other GCs don't do it in young-gen).
>
> Yes, we plan to make reference processing function again and at a minimum this will hold for non-young collections.

Hey, you shout out when you think it's ready to go, yes? Because I think instead of piling more stuff into this PR, we should instead integrate it ASAP, and then progress with smaller PRs. Ok?

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19

From github.com+71722661+earthling-amzn at openjdk.java.net  Mon Mar  8 20:04:27 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Mon, 8 Mar 2021 20:04:27 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v14]
In-Reply-To: <NyvZ6F-sfK8txz998onIC7avO6qyxkbLb7ziIM2QYJk=.3f5f5d3e-cc5a-4243-8f6a-e52692388bcd@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
 <7CiRCYgYXysI9lWIUUayxOqdXZAZ7alOWWtNXMMDDjQ=.6020f939-907e-4790-9c2b-93230c41dcb8@github.com>
 <lht8GrKDjtkM_S3LG6AycLBTJivMtiUgxe6kypCWPe0=.e7f5967f-c1e0-47db-bc3a-c0c513b4f846@github.com>
 <YV2nQ2TwAJwMA2r4Hh0GyrRfIJmgoVH-6ytlfwIAhgk=.fd8c180a-73be-4c3a-9c9b-c39084e15f76@github.com>
 <M-FFynDq_S1yB8vUyLyfwsDQWGBOkIREMWlPdj9k9jA=.07334715-7e3a-4882-b7fb-4d890a065be2@github.com>
 <NyvZ6F-sfK8txz998onIC7avO6qyxkbLb7ziIM2QYJk=.3f5f5d3e-cc5a-4243-8f6a-e52692388bcd@github.com>
Message-ID: <HrONJPCkv_LgBtL5DvucJKX90EA_TOIcizFG4Y0XKAM=.9da1bfb1-5f83-42b7-ada4-ab770e5e9b6c@github.com>

On Mon, 8 Mar 2021 19:54:02 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Yes, we plan to make reference processing function again and at a minimum this will hold for non-young collections.
>
> Hey, you shout out when you think it's ready to go, yes? Because I think instead of piling more stuff into this PR, we should instead integrate it ASAP, and then progress with smaller PRs. Ok?

Ha - was just about to email you. Yes, I think it's good to go. It passes all 71 of the tier1 Shenandoah tests.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19

From jbachorik at openjdk.java.net  Tue Mar  9 12:16:17 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Tue, 9 Mar 2021 12:16:17 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate [v9]
In-Reply-To: <l0lr-q29vacGe0upkTwrhXwi7sFwH-Jvcc-PxIobLJM=.aa5b21aa-f0ec-461d-aa32-87c5a211c177@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <K4Ac7D3fQw_lxHfpya748uuBMet058eaHA_PgVFadzc=.4f7d9350-f1a2-4e29-b3db-6e387a2f1369@github.com>
 <l0lr-q29vacGe0upkTwrhXwi7sFwH-Jvcc-PxIobLJM=.aa5b21aa-f0ec-461d-aa32-87c5a211c177@github.com>
Message-ID: <1SitAwn_HPwglwUiTJdjoACmIhD4U9My1RVGzGJz5Ho=.f3f6556f-3fb1-44c7-b522-5c15de734b26@github.com>

On Wed, 3 Mar 2021 12:15:21 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

>> Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Add tests for the heap usage summary event
>
> Fwiw, the change still does not capture G1 full gc `live_estimate()`.

@tschatzl I think I have finished the changes you requested. Please, take a look once you have time. Thanks!

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From gnu.andrew at redhat.com  Wed Mar 10 01:34:43 2021
From: gnu.andrew at redhat.com (Andrew Hughes)
Date: Wed, 10 Mar 2021 01:34:43 +0000
Subject: [RFR] [8u] 8u292-b04 Upstream Sync
In-Reply-To: <514c7a9f-f6fd-9380-dac4-9c2c2d971a4a@redhat.com>
References: <20210305175947.GB1605474@rincewind>
 <514c7a9f-f6fd-9380-dac4-9c2c2d971a4a@redhat.com>
Message-ID: <20210310013443.GA2423505@rincewind>

On 11:58 Sat 06 Mar     , Aleksey Shipilev wrote:
> On 3/5/21 6:59 PM, Andrew Hughes wrote:
> > Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/
> > 
> > Merge changesets:
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/corba/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/jaxp/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/jaxws/merge.changeset
> 
> Look trivially good.
> 
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/jdk/merge.changeset
> 
> Looks good.
> 
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/hotspot/merge.changeset
> 
> Looks good.
> 
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/langtools/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/nashorn/merge.changeset
> 
> Look trivially good.
> 
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b04/root/merge.changeset
> 
> Looks good.
> 
> > Ok to push?
> 
> Yes.
> 
> -- 
> Thanks,
> -Aleksey
> 

Pushed. Thanks.
-- 
Andrew :)

Senior Free Java Software Engineer
OpenJDK Package Owner
Red Hat, Inc. (http://www.redhat.com)

PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04  C5A0 CFDA 0F9B 3596 4222

From gnu.andrew at redhat.com  Wed Mar 10 02:49:54 2021
From: gnu.andrew at redhat.com (Andrew Hughes)
Date: Wed, 10 Mar 2021 02:49:54 +0000
Subject: [RFR] [8u] 8u292-b05 Upstream Sync
Message-ID: <20210310024954.GB2423505@rincewind>

Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b05/

Merge changesets:
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b05/corba/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b05/jaxp/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b05/jaxws/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b05/jdk/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b05/hotspot/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b05/langtools/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b05/nashorn/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b05/root/merge.changeset

Changes in aarch64-shenandoah-jdk8u292-b05:
  - JDK-6345095: regression test EmptyClipRenderingTest fails
  - JDK-6896810: TEST_BUG: java/lang/ref/SoftReference/Pin.java fails with OOME during System.out.println
  - JDK-7107012: sun.jvm.hostspot.code.CompressedReadStream readDouble() conversion to long mishandled
  - JDK-7112454: TEST_BUG: java/awt/Choice/PopdownGeneratesMouseEvents/PopdownGeneratesMouseEvents.html failed
  - JDK-7131835: [TEST_BUG] Test does not consider that the rounded edges of the window in Mac OS 10.7
  - JDK-7185221: [macosx] Regtest should not throw exception if a suitable display mode found
  - JDK-8041464: [TEST_BUG] CustomClassLoaderTransferTest does not support OS X
  - JDK-8078024: javac, several incorporation steps are silently failing when an error should be reported
  - JDK-8129626: G1: set_in_progress() and clear_started() needs a barrier on non-TSO platforms
  - JDK-8211301: [macos] support full window content options
  - JDK-8240353: AArch64: missing support for -XX:+ExtendedDTraceProbes in C1
  - JDK-8248336: AArch64: C2: offset overflow in BoxLockNode::emit
  - JDK-8257746: Regression introduced with JDK-8250984 - memory might be null in some machines
  - JDK-8261231: Windows IME was disabled after DnD operation
  - JDK-8262073: assert(allocates2(pc)) failed: not in CodeBuffer memory

Main issues of note:
None, clean merge (no HotSpot conflicts).

diffstat for root
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for corba
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jaxp
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jaxws
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for langtools
 b/.hgtags                                                                                           |    1 
 b/src/share/classes/com/sun/tools/javac/comp/Infer.java                                             |   58 +++++++++-
 b/src/share/classes/com/sun/tools/javac/resources/compiler.properties                               |   23 +++
 b/test/tools/javac/Diagnostics/6722234/T6722234b.java                                               |    2 
 b/test/tools/javac/Diagnostics/6722234/T6722234b_1.out                                              |    2 
 b/test/tools/javac/Diagnostics/6722234/T6722234b_2.out                                              |    2 
 b/test/tools/javac/Diagnostics/6722234/T6722234d.java                                               |    2 
 b/test/tools/javac/Diagnostics/6722234/T6722234d_1.out                                              |    4 
 b/test/tools/javac/Diagnostics/6722234/T6722234d_2.out                                              |    4 
 b/test/tools/javac/Diagnostics/6799605/T6799605.java                                                |    2 
 b/test/tools/javac/Diagnostics/6799605/T6799605.out                                                 |    4 
 b/test/tools/javac/diags/examples.not-yet.txt                                                       |    2 
 b/test/tools/javac/diags/examples/WhereCaptured.java                                                |    4 
 b/test/tools/javac/diags/examples/WhereCaptured1.java                                               |    4 
 b/test/tools/javac/diags/examples/WhereIntersection.java                                            |   22 +--
 b/test/tools/javac/diags/examples/WhereIntersection2.java                                           |   46 +++++++
 b/test/tools/javac/diags/examples/WhereTypeVar2.java                                                |   42 +++++++
 b/test/tools/javac/generics/diamond/neg/Neg07.java                                                  |    2 
 b/test/tools/javac/generics/diamond/neg/Neg07.out                                                   |    2 
 b/test/tools/javac/generics/inference/4941882/T4941882.java                                         |    2 
 b/test/tools/javac/generics/inference/6315770/T6315770.java                                         |    2 
 b/test/tools/javac/generics/inference/6315770/T6315770.out                                          |    2 
 b/test/tools/javac/generics/inference/6611449/T6611449.java                                         |    2 
 b/test/tools/javac/generics/inference/6611449/T6611449.out                                          |    8 -
 b/test/tools/javac/generics/inference/6650759/T6650759m.java                                        |    2 
 b/test/tools/javac/generics/inference/6650759/T6650759m.out                                         |    2 
 b/test/tools/javac/generics/inference/EagerReturnTypeResolution/EagerReturnTypeResolutionTestb.java |    2 
 b/test/tools/javac/generics/inference/EagerReturnTypeResolution/EagerReturnTypeResolutionTestb.out  |   32 ++---
 b/test/tools/javac/generics/inference/EagerReturnTypeResolution/PrimitiveTypeBoxingTest.java        |    2 
 b/test/tools/javac/generics/inference/EagerReturnTypeResolution/PrimitiveTypeBoxingTest.out         |    4 
 b/test/tools/javac/generics/wildcards/6762569/T6762569b.java                                        |    2 
 b/test/tools/javac/lambda/8016177/T8016177g.java                                                    |    2 
 b/test/tools/javac/lambda/8016177/T8016177g.out                                                     |    2 
 b/test/tools/javac/lambda/MethodReference41.java                                                    |    2 
 b/test/tools/javac/lambda/MethodReference41.out                                                     |    4 
 b/test/tools/javac/lambda/MethodReference43.java                                                    |    2 
 b/test/tools/javac/lambda/MethodReference43.out                                                     |    6 -
 b/test/tools/javac/lambda/MethodReference46.java                                                    |    2 
 b/test/tools/javac/lambda/MethodReference46.out                                                     |    4 
 b/test/tools/javac/lambda/MethodReference58.java                                                    |    2 
 b/test/tools/javac/lambda/MethodReference58.out                                                     |    2 
 b/test/tools/javac/lambda/MethodReference68.java                                                    |    2 
 b/test/tools/javac/lambda/MethodReference68.out                                                     |    2 
 b/test/tools/javac/lambda/TargetType02.java                                                         |    2 
 b/test/tools/javac/lambda/TargetType02.out                                                          |    2 
 45 files changed, 241 insertions(+), 85 deletions(-)

diffstat for nashorn
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jdk
 b/.hgtags                                                                                     |    1 
 b/src/linux/classes/jdk/internal/platform/cgroupv1/Metrics.java                               |    8 
 b/src/macosx/classes/sun/lwawt/macosx/CPlatformWindow.java                                    |   36 +
 b/src/macosx/native/sun/awt/AWTWindow.m                                                       |   35 +
 b/src/windows/native/sun/windows/awt_Toolkit.cpp                                              |    6 
 b/test/java/awt/Choice/PopdownGeneratesMouseEvents/PopdownGeneratesMouseEvents.html           |   45 +
 b/test/java/awt/Choice/PopdownGeneratesMouseEvents/PopdownGeneratesMouseEvents.java           |  138 ++++
 b/test/java/awt/FullScreen/NonExistentDisplayModeTest/NonExistentDisplayModeTest.java         |  152 +++++
 b/test/java/awt/Window/FullWindowContentTest/FullWindowContentTest.java                       |  212 +++++++
 b/test/java/awt/datatransfer/CustomClassLoaderTransferTest/AnotherInterface.java              |    3 
 b/test/java/awt/datatransfer/CustomClassLoaderTransferTest/CustomClassLoaderTransferTest.java |   63 ++
 b/test/java/awt/datatransfer/CustomClassLoaderTransferTest/TransferableList.java              |   30 +
 b/test/java/awt/regtesthelpers/CopyClassFile.java                                             |   34 +
 b/test/java/lang/ref/SoftReference/Pin.java                                                   |    1 
 b/test/sun/java2d/OpenGL/CopyAreaOOB.java                                                     |  146 ++++
 b/test/sun/java2d/SunGraphics2D/EmptyClipRenderingTest.java                                   |  299 ++++++++++
 16 files changed, 1191 insertions(+), 18 deletions(-)

diffstat for hotspot
 b/.hgtags                                                                |    1 
 b/agent/src/share/classes/sun/jvm/hotspot/code/CompressedReadStream.java |    4 -
 b/src/cpu/aarch64/vm/aarch64.ad                                          |   16 +++--
 b/src/cpu/aarch64/vm/c1_LIRAssembler_aarch64.cpp                         |   11 +--
 b/src/cpu/aarch64/vm/c1_Runtime1_aarch64.cpp                             |   10 +++
 b/src/cpu/aarch64/vm/vtableStubs_aarch64.cpp                             |   26 +++++---
 b/src/share/vm/gc_implementation/g1/concurrentMarkThread.cpp             |    4 -
 b/src/share/vm/gc_implementation/g1/concurrentMarkThread.hpp             |   29 +++++-----
 b/src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp                  |    2 
 9 files changed, 61 insertions(+), 42 deletions(-)

Successfully built on x86, x86_64, s390 (Zero), s390x (Zero), ppc (Zero),
ppc64, ppc64le, aarch32 (Zero) & aarch64.

Ok to push?

Thanks,
-- 
Andrew :)

Senior Free Java Software Engineer
OpenJDK Package Owner
Red Hat, Inc. (http://www.redhat.com)

PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04  C5A0 CFDA 0F9B 3596 4222

From shade at redhat.com  Wed Mar 10 09:10:11 2021
From: shade at redhat.com (Aleksey Shipilev)
Date: Wed, 10 Mar 2021 10:10:11 +0100
Subject: [RFR] [8u] 8u292-b05 Upstream Sync
In-Reply-To: <20210310024954.GB2423505@rincewind>
References: <20210310024954.GB2423505@rincewind>
Message-ID: <0c7754d6-48f5-d6f6-a24d-9dd8b445cc45@redhat.com>

On 3/10/21 3:49 AM, Andrew Hughes wrote:
> Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b05/
> 
> Merge changesets:
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b05/corba/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b05/jaxp/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b05/jaxws/merge.changeset

Looks trivially fine.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b05/jdk/merge.changeset

Looks fine.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b05/hotspot/merge.changeset

Looks fine.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b05/langtools/merge.changeset

Looks fine.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b05/nashorn/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b05/root/merge.changeset

Look trivially fine.

> Ok to push?

Yes, I think so.

-- 
Thanks,
-Aleksey


From rkennke at openjdk.java.net  Wed Mar 10 12:33:29 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 10 Mar 2021 12:33:29 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v23]
In-Reply-To: <0WV_jNaa5NlDA-aqTjngV_P-9_RJ7ABqjgIctYJP-Ls=.40145567-8821-4a46-89db-06cd05a02898@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
 <0WV_jNaa5NlDA-aqTjngV_P-9_RJ7ABqjgIctYJP-Ls=.40145567-8821-4a46-89db-06cd05a02898@github.com>
Message-ID: <Elk5rtqMdRRQ9tisszxbPid94khMuiJKzUwN6U1GiNo=.49ced62c-c52d-4d39-9843-214555bc435b@github.com>

On Mon, 8 Mar 2021 19:04:45 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:

>> **This is a work in progress.**
>> 
>> ## Summary of changes
>> The goal of these changes is to allow young cycles to suspend marking in the old generation. When the young cycle is complete, the concurrent old marking will resume.
>> * Evaluation of heuristics was pulled into a new `ShenandoahRegulatorThread`. Taking this evaluation out of line from the control thread allows the regulator to suspend (using the cancellation mechanism) old generation marking and start a young generation cycle.
>> * Task queues for marking have been moved from `ShenandoahMarkingContext` into `ShenandoahGeneration`. This allows the marking state for the old generation to persist across young generation cycles. The associated `is_complete` state has also been moved to `ShenandoahGeneration`.
>> * Old generation marking is bootstrapped by a complete young generation cycle. In this scenario, the mark closures for a young cycle are given reference to the young generation mark queues _and_ the old generation mark queues. Rather than ignore old references as is done for a normal young cycle, they are enqueued in the old generation mark queues. When the young cycle completes, the old generation marking continues using the task queues primed by the preceding young cycle.
>> * There is a new flag: `ShenandoahAllowOldMarkingPreemption` (defaults to true). Disabling this option will cause the regulator to schedule  `young` or `global` collects (according to heuristics), but will _not_ schedule `old` collects.
>> * The `global` generation is used to support satb and incremental-update modes. The `global` generation is also used for degenerated and implicit/explicit gc requests. Degenerated cycles are not working on this branch and the root cause is understood.
>> * The `global` generation is also used for `FullGC`, but this is also broken. The `FullGC` doesn't update the remembered set during compaction. We reckon there is a non-trivial amount of work to fix this.
>> * The `MARKING` gc state has been split into `YOUNG_MARKING` and `OLD_MARKING`.
>> * Immediate garbage collection is broken in generational mode. The update references phase is used to repair the remembered set, so when this phase is skipped the remembered set scan runs into trouble. A fix for this is in progress.
>> * The remembered set is scanned on a safepoint during initial mark. Work to make this concurrent is in progress.
>
> earthling-amzn has updated the pull request incrementally with three additional commits since the last revision:
> 
>  - Correct typo in assert message
>  - Update generation memory usage during promotion
>  - Report how long it takes to clear a cancel GC request

Looks good!

-------------

Marked as reviewed by rkennke (Reviewer).

PR: https://git.openjdk.java.net/shenandoah/pull/19

From rkennke at openjdk.java.net  Wed Mar 10 19:10:29 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 10 Mar 2021 19:10:29 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v23]
In-Reply-To: <Elk5rtqMdRRQ9tisszxbPid94khMuiJKzUwN6U1GiNo=.49ced62c-c52d-4d39-9843-214555bc435b@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
 <0WV_jNaa5NlDA-aqTjngV_P-9_RJ7ABqjgIctYJP-Ls=.40145567-8821-4a46-89db-06cd05a02898@github.com>
 <Elk5rtqMdRRQ9tisszxbPid94khMuiJKzUwN6U1GiNo=.49ced62c-c52d-4d39-9843-214555bc435b@github.com>
Message-ID: <-du1pbsyM8urAPh4biHIqynNewVjNcTKBIoq11krv0U=.b6afcba1-1246-43ec-a78a-3f0e26a06bbe@github.com>

On Wed, 10 Mar 2021 12:30:32 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> earthling-amzn has updated the pull request incrementally with three additional commits since the last revision:
>> 
>>  - Correct typo in assert message
>>  - Update generation memory usage during promotion
>>  - Report how long it takes to clear a cancel GC request
>
> Looks good!

Can you please merge latest from shenandoah master? "This branch cannot be rebased due to conflicts" *sigh*

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19

From github.com+71722661+earthling-amzn at openjdk.java.net  Wed Mar 10 19:25:30 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Wed, 10 Mar 2021 19:25:30 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v23]
In-Reply-To: <-du1pbsyM8urAPh4biHIqynNewVjNcTKBIoq11krv0U=.b6afcba1-1246-43ec-a78a-3f0e26a06bbe@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
 <0WV_jNaa5NlDA-aqTjngV_P-9_RJ7ABqjgIctYJP-Ls=.40145567-8821-4a46-89db-06cd05a02898@github.com>
 <Elk5rtqMdRRQ9tisszxbPid94khMuiJKzUwN6U1GiNo=.49ced62c-c52d-4d39-9843-214555bc435b@github.com>
 <-du1pbsyM8urAPh4biHIqynNewVjNcTKBIoq11krv0U=.b6afcba1-1246-43ec-a78a-3f0e26a06bbe@github.com>
Message-ID: <q4nruutTBHAOu3rGkA9LpjuINPUamRSjLNkUB0xE28I=.fdf19b16-cb78-4e79-9dda-7c4a70bddcae@github.com>

On Wed, 10 Mar 2021 19:07:52 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Looks good!
>
> Can you please merge latest from shenandoah master? "This branch cannot be rebased due to conflicts" *sigh*

Hmm, shenandoah:master is up to date (179c167a184 Fix Zero builds) and merged. If I try to `merge --squash` locally into shenandoah:master it goes through without any problems.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19

From github.com+71722661+earthling-amzn at openjdk.java.net  Wed Mar 10 19:33:24 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Wed, 10 Mar 2021 19:33:24 GMT
Subject: RFR: Allow young collection to suspend marking in old generation
 [v23]
In-Reply-To: <q4nruutTBHAOu3rGkA9LpjuINPUamRSjLNkUB0xE28I=.fdf19b16-cb78-4e79-9dda-7c4a70bddcae@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
 <0WV_jNaa5NlDA-aqTjngV_P-9_RJ7ABqjgIctYJP-Ls=.40145567-8821-4a46-89db-06cd05a02898@github.com>
 <Elk5rtqMdRRQ9tisszxbPid94khMuiJKzUwN6U1GiNo=.49ced62c-c52d-4d39-9843-214555bc435b@github.com>
 <-du1pbsyM8urAPh4biHIqynNewVjNcTKBIoq11krv0U=.b6afcba1-1246-43ec-a78a-3f0e26a06bbe@github.com>
 <q4nruutTBHAOu3rGkA9LpjuINPUamRSjLNkUB0xE28I=.fdf19b16-cb78-4e79-9dda-7c4a70bddcae@github.com>
Message-ID: <yNZ7OAq2I3XbN0rIqzZXitjdiSzVXhcFl_Zfc_awlh4=.269bdd26-ba04-4404-9971-bd27f016238e@github.com>

On Wed, 10 Mar 2021 19:23:02 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:

>> Can you please merge latest from shenandoah master? "This branch cannot be rebased due to conflicts" *sigh*
>
> Hmm, shenandoah:master is up to date (179c167a184 Fix Zero builds) and merged. If I try to `merge --squash` locally into shenandoah:master it goes through without any problems.

I can rebase on the tip of shenandoah:master and (force) push that to this branch. Reckon that would help?

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19

From github.com+71722661+earthling-amzn at openjdk.java.net  Wed Mar 10 20:40:28 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Wed, 10 Mar 2021 20:40:28 GMT
Subject: Integrated: Allow young collection to suspend marking in old
 generation
In-Reply-To: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
References: <Q5UdF0_sTNVtlR46KLWYOw21mrnd2ziF1Y2Q2s52sDY=.fd88d230-5d57-410b-af44-79efd5fbdbcf@github.com>
Message-ID: <LbgGXyYJqoPur-q0ON85Bqg_ML_pK-Oji4NLNoido6g=.18e6f299-bdd3-43e8-a8d6-06ca78d16853@github.com>

On Fri, 19 Feb 2021 23:31:29 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:

> ## Summary of changes
> The goal of these changes is to allow young cycles to suspend marking in the old generation. When the young cycle is complete, the concurrent old marking will resume.
> * Evaluation of heuristics was pulled into a new `ShenandoahRegulatorThread`. Taking this evaluation out of line from the control thread allows the regulator to suspend (using the cancellation mechanism) old generation marking and start a young generation cycle.
> * Task queues for marking have been moved from `ShenandoahMarkingContext` into `ShenandoahGeneration`. This allows the marking state for the old generation to persist across young generation cycles. The associated `is_complete` state has also been moved to `ShenandoahGeneration`.
> * Old generation marking is bootstrapped by a complete young generation cycle. In this scenario, the mark closures for a young cycle are given reference to the young generation mark queues _and_ the old generation mark queues. Rather than ignore old references as is done for a normal young cycle, they are enqueued in the old generation mark queues. When the young cycle completes, the old generation marking continues using the task queues primed by the preceding young cycle.
> * There is a new flag: `ShenandoahAllowOldMarkingPreemption` (defaults to true). Disabling this option will cause the regulator to schedule  `young` or `global` collects (according to heuristics), but will _not_ schedule `old` collects.
> * The `global` generation is used to support satb and incremental-update modes. The `global` generation is also used for degenerated and implicit/explicit gc requests. Degenerated cycles are not working on this branch and the root cause is understood.
> * The `global` generation is also used for `FullGC`, but this is also broken. The `FullGC` doesn't update the remembered set during compaction. We reckon there is a non-trivial amount of work to fix this.
> * The `MARKING` gc state has been split into `YOUNG_MARKING` and `OLD_MARKING`.
> * Immediate garbage collection is broken in generational mode. The update references phase is used to repair the remembered set, so when this phase is skipped the remembered set scan runs into trouble. A fix for this is in progress.
> * The remembered set is scanned on a safepoint during initial mark. Work to make this concurrent is in progress.

This pull request has now been integrated.

Changeset: 4638e978
Author:    William Kemper <kemperw at amazon.com>
Committer: Roman Kennke <rkennke at openjdk.org>
URL:       https://git.openjdk.java.net/shenandoah/commit/4638e978
Stats:     4221 lines in 92 files changed: 3070 ins; 547 del; 604 mod

Allow young collection to suspend marking in old generation

Reviewed-by: rkennke

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/19

From github.com+71722661+earthling-amzn at openjdk.java.net  Wed Mar 10 22:54:35 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Wed, 10 Mar 2021 22:54:35 GMT
Subject: RFR: Account for usage of all regions in humongous object during
 promotion
Message-ID: <ZGBBMns1T4Gd5RMXB483ISNTsYzh6S8DCj-IzwX-EfI=.187e9e48-37ad-4985-b368-5868e5af87d1@github.com>

Fixes bug in how usage is accounted for in generations.

-------------

Commit messages:
 - Account for usage of all regions in humongous object during promotion

Changes: https://git.openjdk.java.net/shenandoah/pull/20/files
 Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=20&range=00
  Stats: 5 lines in 1 file changed: 2 ins; 3 del; 0 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/20.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/20/head:pull/20

PR: https://git.openjdk.java.net/shenandoah/pull/20

From shade at redhat.com  Thu Mar 11 09:47:19 2021
From: shade at redhat.com (Aleksey Shipilev)
Date: Thu, 11 Mar 2021 10:47:19 +0100
Subject: [8u] RFC: Pick up 8u292-b05 from aarch64-port
Message-ID: <9816d817-00db-2fad-e4c0-bd05e8b37976@redhat.com>

There are pre-8u292 merges in aarch64-port that we want to pick up and test before proposing
the integration back. The pickup is clean, only affects hotspot repository, and gets merged
automatically.

Testing: hotspot_gc_shenandoah {fastdebug,release}

-- 
Thanks,
-Aleksey


From rkennke at redhat.com  Thu Mar 11 09:59:22 2021
From: rkennke at redhat.com (Roman Kennke)
Date: Thu, 11 Mar 2021 10:59:22 +0100
Subject: [8u] RFC: Pick up 8u292-b05 from aarch64-port
In-Reply-To: <9816d817-00db-2fad-e4c0-bd05e8b37976@redhat.com>
References: <9816d817-00db-2fad-e4c0-bd05e8b37976@redhat.com>
Message-ID: <10f4e96e-13f3-6cd9-10a1-9e8a55623aae@redhat.com>

ok!

Roman

> There are pre-8u292 merges in aarch64-port that we want to pick up and 
> test before proposing
> the integration back. The pickup is clean, only affects hotspot 
> repository, and gets merged
> automatically.
> 
> Testing: hotspot_gc_shenandoah {fastdebug,release}
> 


From shade at redhat.com  Thu Mar 11 10:10:38 2021
From: shade at redhat.com (Aleksey Shipilev)
Date: Thu, 11 Mar 2021 11:10:38 +0100
Subject: [8u] RFC: Pick up 8u292-b05 from aarch64-port
In-Reply-To: <10f4e96e-13f3-6cd9-10a1-9e8a55623aae@redhat.com>
References: <9816d817-00db-2fad-e4c0-bd05e8b37976@redhat.com>
 <10f4e96e-13f3-6cd9-10a1-9e8a55623aae@redhat.com>
Message-ID: <9c9561ab-baea-a38e-f287-1177f4baa911@redhat.com>

On 3/11/21 10:59 AM, Roman Kennke wrote:
> ok!

Thanks, pushed.

-- 
-Aleksey


From shade at redhat.com  Thu Mar 11 10:09:16 2021
From: shade at redhat.com (shade at redhat.com)
Date: Thu, 11 Mar 2021 10:09:16 +0000
Subject: hg: shenandoah/jdk8/corba: 12 new changesets
Message-ID: <202103111009.12BA9G5m014636@aojmv0008.oracle.com>

Changeset: 764d6d5a5924
Author:    andrew
Date:      2021-02-08 06:17 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/corba/rev/764d6d5a5924

Added tag jdk8u292-b02 for changeset b5f450e091f9

! .hgtags

Changeset: 737d5e2de7dc
Author:    andrew
Date:      2021-03-02 06:13 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/corba/rev/737d5e2de7dc

Merge jdk8u292-b02

! .hgtags

Changeset: 22ea937c282e
Author:    andrew
Date:      2021-03-02 06:13 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/corba/rev/22ea937c282e

Added tag aarch64-shenandoah-jdk8u292-b02 for changeset 737d5e2de7dc

! .hgtags

Changeset: c178f5067fbf
Author:    andrew
Date:      2021-02-16 02:15 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/corba/rev/c178f5067fbf

Added tag jdk8u292-b03 for changeset 764d6d5a5924

! .hgtags

Changeset: a5196c76c54f
Author:    andrew
Date:      2021-03-03 02:16 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/corba/rev/a5196c76c54f

Merge jdk8u292-b03

! .hgtags

Changeset: c67d8d64b45b
Author:    andrew
Date:      2021-03-03 02:16 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/corba/rev/c67d8d64b45b

Added tag aarch64-shenandoah-jdk8u292-b03 for changeset a5196c76c54f

! .hgtags

Changeset: cae05a72a1f7
Author:    andrew
Date:      2021-02-22 06:57 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/corba/rev/cae05a72a1f7

Added tag jdk8u292-b04 for changeset c178f5067fbf

! .hgtags

Changeset: 9cb808651c8d
Author:    andrew
Date:      2021-03-05 02:42 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/corba/rev/9cb808651c8d

Merge jdk8u292-b04

! .hgtags

Changeset: 8fd5820220fa
Author:    andrew
Date:      2021-03-05 02:42 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/corba/rev/8fd5820220fa

Added tag aarch64-shenandoah-jdk8u292-b04 for changeset 9cb808651c8d

! .hgtags

Changeset: 78f287f3333a
Author:    andrew
Date:      2021-03-01 20:02 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/corba/rev/78f287f3333a

Added tag jdk8u292-b05 for changeset cae05a72a1f7

! .hgtags

Changeset: 955471863bd1
Author:    andrew
Date:      2021-03-05 19:28 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/corba/rev/955471863bd1

Merge jdk8u292-b05

! .hgtags

Changeset: 1fd2eb862203
Author:    andrew
Date:      2021-03-05 19:30 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/corba/rev/1fd2eb862203

Added tag aarch64-shenandoah-jdk8u292-b05 for changeset 955471863bd1

! .hgtags


From shade at redhat.com  Thu Mar 11 10:09:16 2021
From: shade at redhat.com (shade at redhat.com)
Date: Thu, 11 Mar 2021 10:09:16 +0000
Subject: hg: shenandoah/jdk8/jaxws: 12 new changesets
Message-ID: <202103111009.12BA9H9W014641@aojmv0008.oracle.com>

Changeset: 175be01763ea
Author:    andrew
Date:      2021-02-08 06:17 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/jaxws/rev/175be01763ea

Added tag jdk8u292-b02 for changeset d0fb8a4d90aa

! .hgtags

Changeset: 8fa3680894d6
Author:    andrew
Date:      2021-03-02 06:13 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/jaxws/rev/8fa3680894d6

Merge jdk8u292-b02

! .hgtags

Changeset: 8dcc1b5ad1fb
Author:    andrew
Date:      2021-03-02 06:13 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/jaxws/rev/8dcc1b5ad1fb

Added tag aarch64-shenandoah-jdk8u292-b02 for changeset 8fa3680894d6

! .hgtags

Changeset: 264b5b049b16
Author:    andrew
Date:      2021-02-16 02:15 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/jaxws/rev/264b5b049b16

Added tag jdk8u292-b03 for changeset 175be01763ea

! .hgtags

Changeset: d69333699637
Author:    andrew
Date:      2021-03-03 02:16 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/jaxws/rev/d69333699637

Merge jdk8u292-b03

! .hgtags

Changeset: 8c8c1f183ea3
Author:    andrew
Date:      2021-03-03 02:16 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/jaxws/rev/8c8c1f183ea3

Added tag aarch64-shenandoah-jdk8u292-b03 for changeset d69333699637

! .hgtags

Changeset: 53e36820cf90
Author:    andrew
Date:      2021-02-22 06:57 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/jaxws/rev/53e36820cf90

Added tag jdk8u292-b04 for changeset 264b5b049b16

! .hgtags

Changeset: de29f81e950e
Author:    andrew
Date:      2021-03-05 02:42 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/jaxws/rev/de29f81e950e

Merge jdk8u292-b04

! .hgtags

Changeset: 6655cc0ca55e
Author:    andrew
Date:      2021-03-05 02:42 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/jaxws/rev/6655cc0ca55e

Added tag aarch64-shenandoah-jdk8u292-b04 for changeset de29f81e950e

! .hgtags

Changeset: 0bbedbbfbd11
Author:    andrew
Date:      2021-03-01 20:02 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/jaxws/rev/0bbedbbfbd11

Added tag jdk8u292-b05 for changeset 53e36820cf90

! .hgtags

Changeset: 14952b25f832
Author:    andrew
Date:      2021-03-05 19:28 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/jaxws/rev/14952b25f832

Merge jdk8u292-b05

! .hgtags

Changeset: 69dd011f38fd
Author:    andrew
Date:      2021-03-05 19:30 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/jaxws/rev/69dd011f38fd

Added tag aarch64-shenandoah-jdk8u292-b05 for changeset 14952b25f832

! .hgtags


From shade at redhat.com  Thu Mar 11 10:10:35 2021
From: shade at redhat.com (shade at redhat.com)
Date: Thu, 11 Mar 2021 10:10:35 +0000
Subject: hg: shenandoah/jdk8/hotspot: 24 new changesets
Message-ID: <202103111010.12BAAa7U015089@aojmv0008.oracle.com>

Changeset: 4a7a0f935987
Author:    shade
Date:      2021-02-17 20:14 +0100
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/4a7a0f935987

Revert differences against upstream 8u

! agent/make/Makefile
! make/windows/makefiles/fastdebug.make
! src/cpu/aarch64/vm/c1_LIRGenerator_aarch64.cpp
! src/cpu/aarch64/vm/sharedRuntime_aarch64.cpp
! src/cpu/x86/vm/c1_LIRAssembler_x86.cpp
! src/cpu/x86/vm/macroAssembler_x86.hpp
! src/share/vm/asm/assembler.hpp
! src/share/vm/c1/c1_Runtime1.cpp
! src/share/vm/classfile/dictionary.cpp
! src/share/vm/classfile/javaClasses.hpp
! src/share/vm/classfile/systemDictionary.cpp
! src/share/vm/code/relocInfo.cpp
! src/share/vm/compiler/oopMap.cpp
! src/share/vm/interpreter/templateInterpreterGenerator.hpp
! src/share/vm/oops/method.hpp
! src/share/vm/oops/oop.hpp
! src/share/vm/oops/oop.inline.hpp
! src/share/vm/oops/typeArrayOop.hpp
! src/share/vm/opto/cfgnode.cpp
! src/share/vm/opto/parse2.cpp
! src/share/vm/opto/type.cpp
! src/share/vm/prims/jni.cpp
! src/share/vm/prims/jvm.cpp
! src/share/vm/runtime/jniHandles.cpp
! src/share/vm/runtime/jniHandles.hpp
! src/share/vm/runtime/sharedRuntime.cpp
! src/share/vm/runtime/thread.cpp
! src/share/vm/services/memoryPool.hpp

Changeset: 6ea5a8067d1f
Author:    andrew
Date:      2021-02-08 06:17 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/6ea5a8067d1f

Added tag jdk8u292-b02 for changeset 540c4d715391

! .hgtags

Changeset: 3befe35dd352
Author:    andrew
Date:      2021-03-02 06:13 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/3befe35dd352

Merge jdk8u292-b02

! .hgtags

Changeset: ea61743d9574
Author:    andrew
Date:      2021-03-02 06:13 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/ea61743d9574

Added tag aarch64-shenandoah-jdk8u292-b02 for changeset 3befe35dd352

! .hgtags

Changeset: 306a4643e4d2
Author:    dchuyko
Date:      2017-12-01 18:20 +0300
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/306a4643e4d2

8260930: AARCH64: Invalid value passed to critical JNI function
Reviewed-by: vlivanov, aph

! src/cpu/aarch64/vm/sharedRuntime_aarch64.cpp
! src/cpu/aarch64/vm/vm_version_aarch64.cpp
! test/compiler/criticalnatives/argumentcorruption/Test8167409.sh

Changeset: 3d2970e26c36
Author:    andrew
Date:      2021-02-16 02:16 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/3d2970e26c36

Added tag jdk8u292-b03 for changeset 306a4643e4d2

! .hgtags

Changeset: 3f29d8f9207e
Author:    andrew
Date:      2021-03-03 02:16 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/3f29d8f9207e

Merge jdk8u292-b03

! .hgtags
! src/cpu/aarch64/vm/sharedRuntime_aarch64.cpp
! src/cpu/aarch64/vm/vm_version_aarch64.cpp

Changeset: add92629a217
Author:    andrew
Date:      2021-03-03 02:16 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/add92629a217

Added tag aarch64-shenandoah-jdk8u292-b03 for changeset 3f29d8f9207e

! .hgtags

Changeset: 670f30def056
Author:    dholmes
Date:      2021-02-02 17:44 +0300
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/670f30def056

8260349: Cannot programmatically retrieve Metaspace max set via JAVA_TOOL_OPTIONS
Reviewed-by: shade, stuefe

! src/share/vm/services/memoryPool.cpp
+ test/runtime/Metaspace/MaxMetaspaceSizeEnvVarTest.java

Changeset: d644ac8583fd
Author:    dcherepanov
Date:      2021-02-17 12:44 +0300
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/d644ac8583fd

8261766: [8u] hotspot needs to recognise cl.exe 19.16 to build with VS2017
Reviewed-by: andrew

! make/windows/makefiles/compile.make

Changeset: 16bc2fd11c4c
Author:    enevill
Date:      2016-12-18 17:26 -0500
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/16bc2fd11c4c

8171410: aarch64: long multiplyExact shifts by 31 instead of 63
Reviewed-by: aph

! src/cpu/aarch64/vm/aarch64.ad

Changeset: 2048ae231a90
Author:    andrew
Date:      2021-02-22 06:58 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/2048ae231a90

Added tag jdk8u292-b04 for changeset 16bc2fd11c4c

! .hgtags

Changeset: 2d8c651f79bc
Author:    andrew
Date:      2021-03-05 02:42 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/2d8c651f79bc

Merge jdk8u292-b04

! .hgtags
! make/windows/makefiles/compile.make
! src/cpu/aarch64/vm/aarch64.ad
! src/share/vm/services/memoryPool.cpp

Changeset: 204c64acca92
Author:    andrew
Date:      2021-03-05 02:42 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/204c64acca92

Added tag aarch64-shenandoah-jdk8u292-b04 for changeset 2d8c651f79bc

! .hgtags

Changeset: 29720c52e25e
Author:    cjplummer
Date:      2020-06-29 14:22 -0700
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/29720c52e25e

7107012: sun.jvm.hostspot.code.CompressedReadStream readDouble() conversion to long mishandled
Reviewed-by: sspitsyn, dcubed

! agent/src/share/classes/sun/jvm/hotspot/code/CompressedReadStream.java

Changeset: bd1ccfcfc1c5
Author:    fyang
Date:      2021-02-22 21:10 +0800
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/bd1ccfcfc1c5

8262073: assert(allocates2(pc)) failed: not in CodeBuffer memory
Summary: increase size for aarch64 itable and vtable stubs
Reviewed-by: aph

! src/cpu/aarch64/vm/vtableStubs_aarch64.cpp

Changeset: 39e98bb4e79a
Author:    andrew
Date:      2021-02-25 00:46 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/39e98bb4e79a

Merge


Changeset: 631ce052d827
Author:    ngasson
Date:      2020-03-16 10:51 +0800
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/631ce052d827

8240353: AArch64: missing support for -XX:+ExtendedDTraceProbes in C1
Reviewed-by: aph

! src/cpu/aarch64/vm/c1_LIRAssembler_aarch64.cpp
! src/cpu/aarch64/vm/c1_Runtime1_aarch64.cpp

Changeset: c4390912be8a
Author:    aph
Date:      2020-06-25 12:24 -0400
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/c4390912be8a

8248336: AArch64: C2: offset overflow in BoxLockNode::emit
Reviewed-by: adinn

! src/cpu/aarch64/vm/aarch64.ad

Changeset: a435c913c8ce
Author:    brutisso
Date:      2015-06-25 08:15 +0200
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/a435c913c8ce

8129626: G1: set_in_progress() and clear_started() needs a barrier on non-TSO platforms
Summary: Also reviewed by vitalyd at gmail.com
Reviewed-by: pliden, bpittore, bdelsart

! src/share/vm/gc_implementation/g1/concurrentMarkThread.cpp
! src/share/vm/gc_implementation/g1/concurrentMarkThread.hpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp

Changeset: a5795acea814
Author:    andrew
Date:      2021-03-01 20:02 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/a5795acea814

Added tag jdk8u292-b05 for changeset a435c913c8ce

! .hgtags

Changeset: c2192892a952
Author:    andrew
Date:      2021-03-05 19:28 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/c2192892a952

Merge jdk8u292-b05

! .hgtags
! src/cpu/aarch64/vm/aarch64.ad
! src/cpu/aarch64/vm/c1_LIRAssembler_aarch64.cpp
! src/cpu/aarch64/vm/c1_Runtime1_aarch64.cpp
! src/cpu/aarch64/vm/vtableStubs_aarch64.cpp
! src/share/vm/gc_implementation/g1/g1CollectedHeap.cpp

Changeset: f571dc730a3a
Author:    andrew
Date:      2021-03-05 19:30 +0000
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/f571dc730a3a

Added tag aarch64-shenandoah-jdk8u292-b05 for changeset c2192892a952

! .hgtags

Changeset: ae72639266f4
Author:    shade
Date:      2021-03-11 09:40 +0100
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/hotspot/rev/ae72639266f4

Merge

! src/cpu/aarch64/vm/c1_LIRAssembler_aarch64.cpp
! src/cpu/aarch64/vm/c1_LIRGenerator_aarch64.cpp
! src/cpu/aarch64/vm/c1_Runtime1_aarch64.cpp
! src/cpu/x86/vm/c1_LIRAssembler_x86.cpp
! src/share/vm/c1/c1_Runtime1.cpp


From rkennke at openjdk.java.net  Thu Mar 11 11:13:27 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Thu, 11 Mar 2021 11:13:27 GMT
Subject: RFR: Account for usage of all regions in humongous object during
 promotion
In-Reply-To: <ZGBBMns1T4Gd5RMXB483ISNTsYzh6S8DCj-IzwX-EfI=.187e9e48-37ad-4985-b368-5868e5af87d1@github.com>
References: <ZGBBMns1T4Gd5RMXB483ISNTsYzh6S8DCj-IzwX-EfI=.187e9e48-37ad-4985-b368-5868e5af87d1@github.com>
Message-ID: <pDlMm-0oOlufAmiUlvzkCvJ_DPJBm9ULbYFrdG0oawU=.96d3585c-991e-482d-94d1-71d9e1daae0d@github.com>

On Wed, 10 Mar 2021 22:50:25 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:

> Fixes bug in how usage is accounted for in generations.

Marked as reviewed by rkennke (Reviewer).

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/20

From tschatzl at openjdk.java.net  Thu Mar 11 14:53:13 2021
From: tschatzl at openjdk.java.net (Thomas Schatzl)
Date: Thu, 11 Mar 2021 14:53:13 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
Message-ID: <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>

On Mon, 8 Mar 2021 17:29:27 GMT, Jaroslav Bachorik <jbachorik at openjdk.org> wrote:

>> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
>> 
>> ## Introducing new JFR event
>> 
>> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
>> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
>> 
>> ## Implementation
>> 
>> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
>> 
>> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
>> 
>> ### Epsilon GC
>> 
>> Trivial implementation - just return `used()` instead.
>> 
>> ### Serial GC
>> 
>> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
>> 
>> ### Parallel GC
>> 
>> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
>> 
>> ### G1 GC
>> 
>> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
>> 
>> ### Shenandoah
>> 
>> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
>> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
>> 
>> ### ZGC
>> 
>> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.
>
> Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Remove unused field

I am leaving this as "request changes" for now as the question I had earlier about that after G1 Full gc the value of `_live_estimate` still seems unanswered and there does not seem to be code in this change for this. Is this intentional? (Not even setting the live bytes to `used()` which at that point would be a good estimate)

There is another PR (#2760) that implements something like that although I haven't looked at it in detail.

Otherwise looks okay.

src/hotspot/share/gc/shared/space.inline.hpp line 140:

> 138:   size_t get_dead_space() {
> 139:     return (_max_deadspace_words - _allowed_deadspace_words) * HeapWordSize;
> 140:   }

Hotspot does not use a "get_" prefix for getters. Also not sure why this needs to be private (and the friend class), I would prefer this instead of the friending. Retrieving the actual amount of dead space from a class that calculates it does not  seem something that needs hiding.

-------------

Changes requested by tschatzl (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2579

From tschatzl at openjdk.java.net  Thu Mar 11 15:46:11 2021
From: tschatzl at openjdk.java.net (Thomas Schatzl)
Date: Thu, 11 Mar 2021 15:46:11 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
Message-ID: <nR7IvUeT4SQXZ3JLFAqeebGf_7jIbEgNlscTI2Wre3A=.b2bbdcdb-7e13-40c2-9154-a81c2f19feb0@github.com>

On Thu, 11 Mar 2021 14:50:10 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

>> Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Remove unused field
>
> I am leaving this as "request changes" for now as the question I had earlier about that after G1 Full gc the value of `_live_estimate` still seems unanswered and there does not seem to be code in this change for this. Is this intentional? (Not even setting the live bytes to `used()` which at that point would be a good estimate)
> 
> There is another PR (#2760) that implements something like that although I haven't looked at it in detail.
> 
> Otherwise looks okay.

Started reviewing PR #2760, and it implements liveness calculation for G1 full gc. I also suggested [there](https://github.com/openjdk/jdk/pull/2760#discussion_r592449837) to extract this functionality out into an extra CR. Maybe you can work together.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From shade at redhat.com  Thu Mar 11 16:19:44 2021
From: shade at redhat.com (Aleksey Shipilev)
Date: Thu, 11 Mar 2021 17:19:44 +0100
Subject: RFR/RFC: Shenandoah integration 2021-03-11
Message-ID: <600db610-01ff-c160-4a37-aa0192af9b0e@redhat.com>

Hi,

I would like to integrate the regular update of Shenandoah 8u to our integration repository, 
aarch64-port/jdk8u-shenandoah. It comes with a few large features:
  - Self-fixing load barriers
  - JFR events support
  - ShenadnoahSoftMaxHeapSize support
  - Other cleanups and bugfixes

Webrev:
   https://cr.openjdk.java.net/~shade/shenandoah/merges/aarch64-port-8u-20210311/webrev.01/

There is no merge changeset, as shenandoah/jdk8 forest is clean downstream of integration 
repository. I am tagging the whole thing as aarch64-shenandoah-jdk8u292-b05-shenandoah-merge-2021-03-11.

Testing: hotspot_gc_shenandoah; it also runs pretty stably in our CIs, and we have no user 
bugreports from our 8u nightlies

-- 
Thanks,
-Aleksey


From rkennke at redhat.com  Thu Mar 11 17:18:45 2021
From: rkennke at redhat.com (Roman Kennke)
Date: Thu, 11 Mar 2021 18:18:45 +0100
Subject: RFR/RFC: Shenandoah integration 2021-03-11
In-Reply-To: <600db610-01ff-c160-4a37-aa0192af9b0e@redhat.com>
References: <600db610-01ff-c160-4a37-aa0192af9b0e@redhat.com>
Message-ID: <196684b1-d4a3-292b-bd85-bcbbc9835764@redhat.com>

Looks good to me!

Thanks,
Roman

> Hi,
> 
> I would like to integrate the regular update of Shenandoah 8u to our 
> integration repository, aarch64-port/jdk8u-shenandoah. It comes with a 
> few large features:
>  ?- Self-fixing load barriers
>  ?- JFR events support
>  ?- ShenadnoahSoftMaxHeapSize support
>  ?- Other cleanups and bugfixes
> 
> Webrev:
>    
> https://cr.openjdk.java.net/~shade/shenandoah/merges/aarch64-port-8u-20210311/webrev.01/ 
> 
> 
> There is no merge changeset, as shenandoah/jdk8 forest is clean 
> downstream of integration repository. I am tagging the whole thing as 
> aarch64-shenandoah-jdk8u292-b05-shenandoah-merge-2021-03-11.
> 
> Testing: hotspot_gc_shenandoah; it also runs pretty stably in our CIs, 
> and we have no user bugreports from our 8u nightlies
> 


From gnu.andrew at redhat.com  Thu Mar 11 17:22:02 2021
From: gnu.andrew at redhat.com (Andrew Hughes)
Date: Thu, 11 Mar 2021 17:22:02 +0000
Subject: [aarch64-port-dev ] RFR/RFC: Shenandoah integration 2021-03-11
In-Reply-To: <600db610-01ff-c160-4a37-aa0192af9b0e@redhat.com>
References: <600db610-01ff-c160-4a37-aa0192af9b0e@redhat.com>
Message-ID: <20210311172202.GA7420@rincewind>

On 17:19 Thu 11 Mar     , Aleksey Shipilev wrote:
> Hi,
> 
> I would like to integrate the regular update of Shenandoah 8u to our
> integration repository, aarch64-port/jdk8u-shenandoah. It comes with a few
> large features:
>  - Self-fixing load barriers
>  - JFR events support
>  - ShenadnoahSoftMaxHeapSize support
>  - Other cleanups and bugfixes
> 
> Webrev:
>   https://cr.openjdk.java.net/~shade/shenandoah/merges/aarch64-port-8u-20210311/webrev.01/
> 
> There is no merge changeset, as shenandoah/jdk8 forest is clean downstream
> of integration repository. I am tagging the whole thing as
> aarch64-shenandoah-jdk8u292-b05-shenandoah-merge-2021-03-11.

Sounds good. The webrev does suggest there are merge revisions though:

https://cr.openjdk.java.net/~shade/shenandoah/merges/aarch64-port-8u-20210311/webrev.01/

e.g. "rev 12213 : Merge"

> 
> Testing: hotspot_gc_shenandoah; it also runs pretty stably in our CIs, and
> we have no user bugreports from our 8u nightlies

Was it tested without INCLUDE_ALL_GCS defined?

My only concern is the backporting of 8202976: "Add C1 lea patching support for x86"

It looks like the changes will have no effect when Shenandoah is disabled at runtime,
though the code changes differ between using INCLUDE_ALL_GCS as well in
c1_LIRAssembler_aarch64.cpp but not in c1_LIRAssembler_x86.cpp (difficult to do
so there with the structure of the code).

At the very least, this will produce some unused variable warnings, but that is true
of JDK-8202976 in 11u too.

> 
> -- 
> Thanks,
> -Aleksey
> 

Thanks,
-- 
Andrew :)

Senior Free Java Software Engineer
OpenJDK Package Owner
Red Hat, Inc. (http://www.redhat.com)

PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04  C5A0 CFDA 0F9B 3596 4222

From github.com+71722661+earthling-amzn at openjdk.java.net  Thu Mar 11 17:32:26 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Thu, 11 Mar 2021 17:32:26 GMT
Subject: Integrated: Account for usage of all regions in humongous object
 during promotion
In-Reply-To: <ZGBBMns1T4Gd5RMXB483ISNTsYzh6S8DCj-IzwX-EfI=.187e9e48-37ad-4985-b368-5868e5af87d1@github.com>
References: <ZGBBMns1T4Gd5RMXB483ISNTsYzh6S8DCj-IzwX-EfI=.187e9e48-37ad-4985-b368-5868e5af87d1@github.com>
Message-ID: <C7ZsGYfqLqV2su91msyL4cbTlt2k1KALLiKjKg_-w6M=.2775a8af-81d8-4d2c-8ba8-bb06288c123c@github.com>

On Wed, 10 Mar 2021 22:50:25 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:

> Fixes bug in how usage is accounted for in generations.

This pull request has now been integrated.

Changeset: f3cc4072
Author:    William Kemper <kemperw at amazon.com>
Committer: Roman Kennke <rkennke at openjdk.org>
URL:       https://git.openjdk.java.net/shenandoah/commit/f3cc4072
Stats:     5 lines in 1 file changed: 2 ins; 3 del; 0 mod

Account for usage of all regions in humongous object during promotion

Reviewed-by: rkennke

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/20

From shade at redhat.com  Thu Mar 11 17:33:38 2021
From: shade at redhat.com (Aleksey Shipilev)
Date: Thu, 11 Mar 2021 18:33:38 +0100
Subject: [aarch64-port-dev ] RFR/RFC: Shenandoah integration 2021-03-11
In-Reply-To: <20210311172202.GA7420@rincewind>
References: <600db610-01ff-c160-4a37-aa0192af9b0e@redhat.com>
 <20210311172202.GA7420@rincewind>
Message-ID: <37f5f62f-cc92-d5de-c747-78ee2df5d906@redhat.com>

On 3/11/21 6:22 PM, Andrew Hughes wrote:
>> There is no merge changeset, as shenandoah/jdk8 forest is clean downstream
>> of integration repository. I am tagging the whole thing as
>> aarch64-shenandoah-jdk8u292-b05-shenandoah-merge-2021-03-11.
> 
> Sounds good. The webrev does suggest there are merge revisions though:
> 
> https://cr.openjdk.java.net/~shade/shenandoah/merges/aarch64-port-8u-20210311/webrev.01/
> 
> e.g. "rev 12213 : Merge"

Yes, these are merges during the pickups _from_ aarch64-port/jdk8u-shenandoah to shenandoah/jdk8.

>> Testing: hotspot_gc_shenandoah; it also runs pretty stably in our CIs, and
>> we have no user bugreports from our 8u nightlies
> 
> Was it tested without INCLUDE_ALL_GCS defined?

Yes, our (well, my) CI implicitly tests this with when building Minimal VM. It does it for all 
repositories, including aarch64-port/jdk8u-shenandoah and shenandoah/jdk8, so neither side is broken 
at the moment :)

> My only concern is the backporting of 8202976: "Add C1 lea patching support for x86"

Yeah, alas we needed it for Shenandoah to do self-fixing barriers.

> It looks like the changes will have no effect when Shenandoah is disabled at runtime,
> though the code changes differ between using INCLUDE_ALL_GCS as well in
> c1_LIRAssembler_aarch64.cpp but not in c1_LIRAssembler_x86.cpp (difficult to do
> so there with the structure of the code).
> 
> At the very least, this will produce some unused variable warnings, but that is true
> of JDK-8202976 in 11u too.

It should not produce unused warnings, as UseShenandoahGC would always be available.

IIRC, Andrew Haley lifted the prior policy that every Shenandoah block should always be protected by 
INCLUDE_ALL_GCS. We now "only" use it to protect the code blocks that touch Shenandoah symbols, 
which might not be available in some configs. c1_LIRAssembler_x86.cpp is not such a block.

Anything else you want answered / done?

-- 
Thanks,
-Aleksey


From rkennke at openjdk.java.net  Thu Mar 11 18:43:24 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Thu, 11 Mar 2021 18:43:24 GMT
Subject: RFR: 8263427: Shenandoah: Trigger weak-LRB even when heap is stable
Message-ID: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>

We currently guard all LRBs, including weak-LRB, by a test for heap-stable and only enter the LRB when heap is unstable (e.g. evacuation or update-refs in progress). However, the weak LRB must also be entered when heap is stable and concurrent refs is in progress, otherwise we may accidentally resurrect otherwise unreachable weak referents. This can happen when we take the shortcut cycle and skip evac&update-refs.

I believe this might be the root cause for JDK-8262852. 

Testing:
 - [x] New testcase failed without change, passes now
 - [ ] hotspot_gc_shenandoah
 - [ ] tier1 (+Shenandoah)

-------------

Commit messages:
 - 8263427: Shenandoah: Trigger weak-LRB even when heap is stable

Changes: https://git.openjdk.java.net/jdk/pull/2945/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2945&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8263427
  Stats: 188 lines in 12 files changed: 161 ins; 12 del; 15 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2945.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2945/head:pull/2945

PR: https://git.openjdk.java.net/jdk/pull/2945

From rkennke at openjdk.java.net  Thu Mar 11 18:56:10 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Thu, 11 Mar 2021 18:56:10 GMT
Subject: RFR: 8263427: Shenandoah: Trigger weak-LRB even when heap is
 stable
In-Reply-To: <QTsvMHCeJnZDUL_jO5lIur8HTi4hgtleNpUZa9RSLN4=.ac19d45e-cd86-4946-a4d6-3a3c7fa5aacf@github.com>
References: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
 <QTsvMHCeJnZDUL_jO5lIur8HTi4hgtleNpUZa9RSLN4=.ac19d45e-cd86-4946-a4d6-3a3c7fa5aacf@github.com>
Message-ID: <FFtaA_Vr2a-zDQ44y2-z26c-x_HeARScGyvE900rJAw=.79751385-a145-4941-9960-a993689bbcbe@github.com>

On Thu, 11 Mar 2021 18:50:48 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> We currently guard all LRBs, including weak-LRB, by a test for heap-stable and only enter the LRB when heap is unstable (e.g. evacuation or update-refs in progress). However, the weak LRB must also be entered when heap is stable and concurrent refs is in progress, otherwise we may accidentally resurrect otherwise unreachable weak referents. This can happen when we take the shortcut cycle and skip evac&update-refs.
>> 
>> I believe this might be the root cause for JDK-8262852. 
>> 
>> Testing:
>>  - [x] New testcase failed without change, passes now
>>  - [ ] hotspot_gc_shenandoah
>>  - [ ] tier1 (+Shenandoah)
>
> src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp line 607:
> 
>> 605:   VerifyThreadGCState(const char* label, char expected) : _label(label), _expected(expected) {}
>> 606:   void do_thread(Thread* t) {
>> 607:     char actual = ShenandoahThreadLocalData::gc_state(t) & ~((char)ShenandoahHeap::WEAK_ROOTS);
> 
> Why these adjustments are needed? I think it shows we don't play well with GC-state lifecycle here...

The verifier would complain about the extra bit being set or maybe not set (e.g. concurrent cycles would set it, but degen cycles would not). We haven't verified conc-weakroots-in-progress before, so I figured it would be easiest to keep it that way and mask it out.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2945

From shade at openjdk.java.net  Thu Mar 11 18:56:07 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Thu, 11 Mar 2021 18:56:07 GMT
Subject: RFR: 8263427: Shenandoah: Trigger weak-LRB even when heap is
 stable
In-Reply-To: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
References: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
Message-ID: <QTsvMHCeJnZDUL_jO5lIur8HTi4hgtleNpUZa9RSLN4=.ac19d45e-cd86-4946-a4d6-3a3c7fa5aacf@github.com>

On Thu, 11 Mar 2021 18:38:26 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

> We currently guard all LRBs, including weak-LRB, by a test for heap-stable and only enter the LRB when heap is unstable (e.g. evacuation or update-refs in progress). However, the weak LRB must also be entered when heap is stable and concurrent refs is in progress, otherwise we may accidentally resurrect otherwise unreachable weak referents. This can happen when we take the shortcut cycle and skip evac&update-refs.
> 
> I believe this might be the root cause for JDK-8262852. 
> 
> Testing:
>  - [x] New testcase failed without change, passes now
>  - [ ] hotspot_gc_shenandoah
>  - [ ] tier1 (+Shenandoah)

I would need some time to understand this.

src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp line 607:

> 605:   VerifyThreadGCState(const char* label, char expected) : _label(label), _expected(expected) {}
> 606:   void do_thread(Thread* t) {
> 607:     char actual = ShenandoahThreadLocalData::gc_state(t) & ~((char)ShenandoahHeap::WEAK_ROOTS);

Why these adjustments are needed? I think it shows we don't play well with GC-state lifecycle here...

-------------

PR: https://git.openjdk.java.net/jdk/pull/2945

From zgu at openjdk.java.net  Thu Mar 11 19:16:18 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 11 Mar 2021 19:16:18 GMT
Subject: RFR: 8263433: Shenandoah: Don't expect forwarded objects in
 set_concurrent_mark_in_progress()
Message-ID: <jAQ2x2oNaXWxyRD0AxC86YSM1PYDoFMsshwT6fK3fh4=.ee68be39-1dca-411b-af1f-ae5ab9ca2ec7@github.com>

Please review this trivial cleanup.

After removing piggyback reference update cycle, we don't expect to see forwarded objects during concurrent mark phase. 
While it is possible during full gc mark, but this flag is irrelevant.

Test:
- [x] hotspot_gc_shenandoah

-------------

Commit messages:
 - JDK-8263433-fwd-mark

Changes: https://git.openjdk.java.net/jdk/pull/2941/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2941&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8263433
  Stats: 5 lines in 1 file changed: 0 ins; 3 del; 2 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2941.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2941/head:pull/2941

PR: https://git.openjdk.java.net/jdk/pull/2941

From shade at openjdk.java.net  Thu Mar 11 19:23:07 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Thu, 11 Mar 2021 19:23:07 GMT
Subject: RFR: 8263433: Shenandoah: Don't expect forwarded objects in
 set_concurrent_mark_in_progress()
In-Reply-To: <jAQ2x2oNaXWxyRD0AxC86YSM1PYDoFMsshwT6fK3fh4=.ee68be39-1dca-411b-af1f-ae5ab9ca2ec7@github.com>
References: <jAQ2x2oNaXWxyRD0AxC86YSM1PYDoFMsshwT6fK3fh4=.ee68be39-1dca-411b-af1f-ae5ab9ca2ec7@github.com>
Message-ID: <6wA9Fl-gJjGgXTeGjy2bR4kJ2-N0IRdfiNvT6pE31Ng=.2834a7eb-8ecd-4640-ace8-af8bc919bb84@github.com>

On Thu, 11 Mar 2021 14:57:43 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Please review this trivial cleanup.
> 
> After removing piggyback reference update cycle, we don't expect to see forwarded objects during concurrent mark phase. 
> While it is possible during full gc mark, but this flag is irrelevant.
> 
> Test:
> - [x] hotspot_gc_shenandoah

Marked as reviewed by shade (Reviewer).

-------------

PR: https://git.openjdk.java.net/jdk/pull/2941

From gnu.andrew at redhat.com  Thu Mar 11 19:26:06 2021
From: gnu.andrew at redhat.com (Andrew Hughes)
Date: Thu, 11 Mar 2021 19:26:06 +0000
Subject: [aarch64-port-dev ] RFR/RFC: Shenandoah integration 2021-03-11
In-Reply-To: <37f5f62f-cc92-d5de-c747-78ee2df5d906@redhat.com>
References: <600db610-01ff-c160-4a37-aa0192af9b0e@redhat.com>
 <20210311172202.GA7420@rincewind>
 <37f5f62f-cc92-d5de-c747-78ee2df5d906@redhat.com>
Message-ID: <20210311192606.GA145398@rincewind>

On 18:33 Thu 11 Mar     , Aleksey Shipilev wrote:
> On 3/11/21 6:22 PM, Andrew Hughes wrote:
> > > There is no merge changeset, as shenandoah/jdk8 forest is clean downstream
> > > of integration repository. I am tagging the whole thing as
> > > aarch64-shenandoah-jdk8u292-b05-shenandoah-merge-2021-03-11.
> > 
> > Sounds good. The webrev does suggest there are merge revisions though:
> > 
> > https://cr.openjdk.java.net/~shade/shenandoah/merges/aarch64-port-8u-20210311/webrev.01/
> > 
> > e.g. "rev 12213 : Merge"
> 
> Yes, these are merges during the pickups _from_ aarch64-port/jdk8u-shenandoah to shenandoah/jdk8.
>

Ok, thought so.

> > > Testing: hotspot_gc_shenandoah; it also runs pretty stably in our CIs, and
> > > we have no user bugreports from our 8u nightlies
> > 
> > Was it tested without INCLUDE_ALL_GCS defined?
> 
> Yes, our (well, my) CI implicitly tests this with when building Minimal VM.
> It does it for all repositories, including aarch64-port/jdk8u-shenandoah and
> shenandoah/jdk8, so neither side is broken at the moment :)
>

Good to hear :)

> > My only concern is the backporting of 8202976: "Add C1 lea patching support for x86"
> 
> Yeah, alas we needed it for Shenandoah to do self-fixing barriers.
> 
> > It looks like the changes will have no effect when Shenandoah is disabled at runtime,
> > though the code changes differ between using INCLUDE_ALL_GCS as well in
> > c1_LIRAssembler_aarch64.cpp but not in c1_LIRAssembler_x86.cpp (difficult to do
> > so there with the structure of the code).
> > 
> > At the very least, this will produce some unused variable warnings, but that is true
> > of JDK-8202976 in 11u too.
> 
> It should not produce unused warnings, as UseShenandoahGC would always be available.
>

I was thinking of AArch64 & sparc:

--- old/src/cpu/aarch64/vm/c1_LIRAssembler_aarch64.cpp	2021-03-11 15:54:23.162720736 +0100
+++ new/src/cpu/aarch64/vm/c1_LIRAssembler_aarch64.cpp	2021-03-11 15:54:23.086718776 +0100
@@ -2908,7 +2908,14 @@
 }
 
 
-void LIR_Assembler::leal(LIR_Opr addr, LIR_Opr dest) {
+void LIR_Assembler::leal(LIR_Opr addr, LIR_Opr dest, LIR_PatchCode patch_code, CodeEmitInfo* info) {
+#if INCLUDE_ALL_GCS
+  if (UseShenandoahGC && patch_code != lir_patch_none) {
+    deoptimize_trap(info);
+    return;
+  }
+#endif
+
   __ lea(dest->as_register_lo(), as_Address(addr->as_address_ptr()));
 }

If INCLUDE_ALL_GCS is not set, then nothing will use patch_code and info.

--- old/src/cpu/sparc/vm/c1_LIRAssembler_sparc.cpp	2021-03-11 15:54:26.550808044 +0100
+++ new/src/cpu/sparc/vm/c1_LIRAssembler_sparc.cpp	2021-03-11 15:54:26.478806188 +0100
@@ -3486,7 +3486,7 @@
 }
 
 
-void LIR_Assembler::leal(LIR_Opr addr_opr, LIR_Opr dest) {
+void LIR_Assembler::leal(LIR_Opr addr_opr, LIR_Opr dest, LIR_PatchCode patch_code, CodeEmitInfo* info) {

This is the same as 11u. Again, info & patch_code will never be used.

> IIRC, Andrew Haley lifted the prior policy that every Shenandoah block
> should always be protected by INCLUDE_ALL_GCS. We now "only" use it to
> protect the code blocks that touch Shenandoah symbols, which might not be
> available in some configs. c1_LIRAssembler_x86.cpp is not such a block.
>

Yeah, as I say, it would be difficult to do otherwise, without having to repeat
the same non-Shenandoah code for the !UseShenandoahGC + INCLUDE_ALL_GCS case and
the no INCLUDE_ALL_GCS case.

> Anything else you want answered / done?
>

I'm happy for this to go in.

> -- 
> Thanks,
> -Aleksey
> 

Thanks,
-- 
Andrew :)

Senior Free Java Software Engineer
OpenJDK Package Owner
Red Hat, Inc. (http://www.redhat.com)

PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04  C5A0 CFDA 0F9B 3596 4222

From zgu at openjdk.java.net  Thu Mar 11 19:27:06 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 11 Mar 2021 19:27:06 GMT
Subject: RFR: 8263427: Shenandoah: Trigger weak-LRB even when heap is
 stable
In-Reply-To: <FFtaA_Vr2a-zDQ44y2-z26c-x_HeARScGyvE900rJAw=.79751385-a145-4941-9960-a993689bbcbe@github.com>
References: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
 <QTsvMHCeJnZDUL_jO5lIur8HTi4hgtleNpUZa9RSLN4=.ac19d45e-cd86-4946-a4d6-3a3c7fa5aacf@github.com>
 <FFtaA_Vr2a-zDQ44y2-z26c-x_HeARScGyvE900rJAw=.79751385-a145-4941-9960-a993689bbcbe@github.com>
Message-ID: <9J6biNSo_KaeRY-EzdOAb7XCeVU6xpw_o9Tk7Tu8peM=.70097cc8-6a8e-4408-83cc-7a8012807447@github.com>

On Thu, 11 Mar 2021 18:52:54 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahVerifier.cpp line 607:
>> 
>>> 605:   VerifyThreadGCState(const char* label, char expected) : _label(label), _expected(expected) {}
>>> 606:   void do_thread(Thread* t) {
>>> 607:     char actual = ShenandoahThreadLocalData::gc_state(t) & ~((char)ShenandoahHeap::WEAK_ROOTS);
>> 
>> Why these adjustments are needed? I think it shows we don't play well with GC-state lifecycle here...
>
> The verifier would complain about the extra bit being set or maybe not set (e.g. concurrent cycles would set it, but degen cycles would not). We haven't verified conc-weakroots-in-progress before, so I figured it would be easiest to keep it that way and mask it out.

ShenandoahGCStateResetter used to disable the flag, so it makes sense.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2945

From zgu at openjdk.java.net  Thu Mar 11 19:32:08 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 11 Mar 2021 19:32:08 GMT
Subject: RFR: 8263427: Shenandoah: Trigger weak-LRB even when heap is
 stable
In-Reply-To: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
References: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
Message-ID: <yPP6l7zwhl7k5huKOPldBGIMs5HQ-wUvx73ALdlSjI8=.d03a5e5a-c4e1-4a30-ad21-a703c7276fc4@github.com>

On Thu, 11 Mar 2021 18:38:26 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

> We currently guard all LRBs, including weak-LRB, by a test for heap-stable and only enter the LRB when heap is unstable (e.g. evacuation or update-refs in progress). However, the weak LRB must also be entered when heap is stable and concurrent refs is in progress, otherwise we may accidentally resurrect otherwise unreachable weak referents. This can happen when we take the shortcut cycle and skip evac&update-refs.
> 
> I believe this might be the root cause for JDK-8262852. 
> 
> The way out of it is change conc-weakroots-in-progress flag to a bit in gc-state, and test for this in weak-LRB gc-state-check, and enter weak-LRB even when heap is stable, but conc-weakroots-in-progress.
> 
> There's one gotcha here: we used to change gc-state only at safepoints so that the flag can safely be propagated to all Java threads. But conc-weakroots-in-progress is turned-off concurrently. I deal with this by propagating the flag change to Java threads via the rendevouz (that we do anyway), and change the global flag only once all threads got the thread-local flag change.
> 
> This stuff makes the verifier unhappy, because it doesn't know about the new bit. And it'd be difficult to properly verify it, because sometimes it is set (conc-cycle) and sometimes it is not (degen-cycle), so instead of additing extra verification, I figured we could keep ignoring the flag (for now?)
> 
> Testing:
>  - [x] New testcase failed without change, passes now
>  - [x] hotspot_gc_shenandoah
>  - [ ] tier1 (+Shenandoah)

Changes requested by zgu (Reviewer).

test/hotspot/jtreg/gc/shenandoah/TestReferenceShortcutCycle.java line 86:

> 84:     private static void testConcurrentCollection() throws Exception {
> 85:         setup();
> 86:         WB.concurrentGCAcquireControl();

Need to ensure a concurrent GC between setup and test, otherwise, referent can be live.
I think calling WB.concurrentGCRunToIdle() before WB.concurrentGCRunTo(WB.AFTER_CONCURRENT_REFERENCE_PROCESSING_STARTED) will do the trick.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2945

From zgu at openjdk.java.net  Thu Mar 11 20:14:09 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 11 Mar 2021 20:14:09 GMT
Subject: Integrated: 8263433: Shenandoah: Don't expect forwarded objects in
 set_concurrent_mark_in_progress()
In-Reply-To: <jAQ2x2oNaXWxyRD0AxC86YSM1PYDoFMsshwT6fK3fh4=.ee68be39-1dca-411b-af1f-ae5ab9ca2ec7@github.com>
References: <jAQ2x2oNaXWxyRD0AxC86YSM1PYDoFMsshwT6fK3fh4=.ee68be39-1dca-411b-af1f-ae5ab9ca2ec7@github.com>
Message-ID: <80prCPiCUEUbrnhVz6boIcyFQAmssOGK2JFaXp_krSo=.8d4f9468-b369-4f27-bfa6-30fbfea5a131@github.com>

On Thu, 11 Mar 2021 14:57:43 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Please review this trivial cleanup.
> 
> After removing piggyback reference update cycle, we don't expect to see forwarded objects during concurrent mark phase. 
> While it is possible during full gc mark, but this flag is irrelevant.
> 
> Test:
> - [x] hotspot_gc_shenandoah

This pull request has now been integrated.

Changeset: b92abac2
Author:    Zhengyu Gu <zgu at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/b92abac2
Stats:     5 lines in 1 file changed: 0 ins; 3 del; 2 mod

8263433: Shenandoah: Don't expect forwarded objects in set_concurrent_mark_in_progress()

Reviewed-by: shade

-------------

PR: https://git.openjdk.java.net/jdk/pull/2941

From rkennke at openjdk.java.net  Thu Mar 11 20:17:28 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Thu, 11 Mar 2021 20:17:28 GMT
Subject: RFR: 8263427: Shenandoah: Trigger weak-LRB even when heap is
 stable [v2]
In-Reply-To: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
References: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
Message-ID: <RCQWcIV0qg_p-siCOTnDJ6beUlf6fX0iZtvWu2GzgXI=.cdc5bbb6-900c-4198-aac4-aecc140add87@github.com>

> We currently guard all LRBs, including weak-LRB, by a test for heap-stable and only enter the LRB when heap is unstable (e.g. evacuation or update-refs in progress). However, the weak LRB must also be entered when heap is stable and concurrent refs is in progress, otherwise we may accidentally resurrect otherwise unreachable weak referents. This can happen when we take the shortcut cycle and skip evac&update-refs.
> 
> I believe this might be the root cause for JDK-8262852. 
> 
> The way out of it is change conc-weakroots-in-progress flag to a bit in gc-state, and test for this in weak-LRB gc-state-check, and enter weak-LRB even when heap is stable, but conc-weakroots-in-progress.
> 
> There's one gotcha here: we used to change gc-state only at safepoints so that the flag can safely be propagated to all Java threads. But conc-weakroots-in-progress is turned-off concurrently. I deal with this by propagating the flag change to Java threads via the rendevouz (that we do anyway), and change the global flag only once all threads got the thread-local flag change.
> 
> This stuff makes the verifier unhappy, because it doesn't know about the new bit. And it'd be difficult to properly verify it, because sometimes it is set (conc-cycle) and sometimes it is not (degen-cycle), so instead of additing extra verification, I figured we could keep ignoring the flag (for now?)
> 
> Testing:
>  - [x] New testcase failed without change, passes now
>  - [x] hotspot_gc_shenandoah
>  - [ ] tier1 (+Shenandoah)

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Ensure test does a complete GC cycle before verification

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2945/files
  - new: https://git.openjdk.java.net/jdk/pull/2945/files/eec5e186..94575b41

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2945&range=01
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2945&range=00-01

  Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2945.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2945/head:pull/2945

PR: https://git.openjdk.java.net/jdk/pull/2945

From zgu at openjdk.java.net  Thu Mar 11 21:19:08 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 11 Mar 2021 21:19:08 GMT
Subject: RFR: 8263427: Shenandoah: Trigger weak-LRB even when heap is
 stable [v2]
In-Reply-To: <RCQWcIV0qg_p-siCOTnDJ6beUlf6fX0iZtvWu2GzgXI=.cdc5bbb6-900c-4198-aac4-aecc140add87@github.com>
References: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
 <RCQWcIV0qg_p-siCOTnDJ6beUlf6fX0iZtvWu2GzgXI=.cdc5bbb6-900c-4198-aac4-aecc140add87@github.com>
Message-ID: <d3St250sLCH4NqsUNP3bY-dwVhoCpciTkl-SAeKVGu0=.67fca762-a021-4634-a17b-6ee9906ec994@github.com>

On Thu, 11 Mar 2021 20:17:28 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> We currently guard all LRBs, including weak-LRB, by a test for heap-stable and only enter the LRB when heap is unstable (e.g. evacuation or update-refs in progress). However, the weak LRB must also be entered when heap is stable and concurrent refs is in progress, otherwise we may accidentally resurrect otherwise unreachable weak referents. This can happen when we take the shortcut cycle and skip evac&update-refs.
>> 
>> I believe this might be the root cause for JDK-8262852. 
>> 
>> The way out of it is change conc-weakroots-in-progress flag to a bit in gc-state, and test for this in weak-LRB gc-state-check, and enter weak-LRB even when heap is stable, but conc-weakroots-in-progress.
>> 
>> There's one gotcha here: we used to change gc-state only at safepoints so that the flag can safely be propagated to all Java threads. But conc-weakroots-in-progress is turned-off concurrently. I deal with this by propagating the flag change to Java threads via the rendevouz (that we do anyway), and change the global flag only once all threads got the thread-local flag change.
>> 
>> This stuff makes the verifier unhappy, because it doesn't know about the new bit. And it'd be difficult to properly verify it, because sometimes it is set (conc-cycle) and sometimes it is not (degen-cycle), so instead of additing extra verification, I figured we could keep ignoring the flag (for now?)
>> 
>> Testing:
>>  - [x] New testcase failed without change, passes now
>>  - [x] hotspot_gc_shenandoah
>>  - [ ] tier1 (+Shenandoah)
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Ensure test does a complete GC cycle before verification

Looks good

-------------

Marked as reviewed by zgu (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2945

From github.com+71722661+earthling-amzn at openjdk.java.net  Thu Mar 11 23:57:37 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Thu, 11 Mar 2021 23:57:37 GMT
Subject: RFR: Update periodic gc test for changed heuristic log message
Message-ID: <2VCkfRyfAYiBzdnyaHJHiKoR_Gc05h2XFMLu1ZhbHew=.4b099ff1-9096-439e-bf2e-f10b458fe300@github.com>

This also includes a change to hold down regulator requests when the GC is already running in non-generational modes.

-------------

Commit messages:
 - Hold down regulator requests when GC is already running
 - Update test for heuristic trigger message that includes generation name

Changes: https://git.openjdk.java.net/shenandoah/pull/21/files
 Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=21&range=00
  Stats: 12 lines in 2 files changed: 4 ins; 0 del; 8 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/21.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/21/head:pull/21

PR: https://git.openjdk.java.net/shenandoah/pull/21

From mli at openjdk.java.net  Fri Mar 12 05:25:15 2021
From: mli at openjdk.java.net (Hamlin Li)
Date: Fri, 12 Mar 2021 05:25:15 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <nR7IvUeT4SQXZ3JLFAqeebGf_7jIbEgNlscTI2Wre3A=.b2bbdcdb-7e13-40c2-9154-a81c2f19feb0@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
 <nR7IvUeT4SQXZ3JLFAqeebGf_7jIbEgNlscTI2Wre3A=.b2bbdcdb-7e13-40c2-9154-a81c2f19feb0@github.com>
Message-ID: <BhZlIEK0OPNi_wPRQtZx8Gi09BaZkMfb9PG9kkMIm2s=.9e5e142c-44c1-4598-ac59-200b6452ffe1@github.com>

On Thu, 11 Mar 2021 15:42:51 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

> Started reviewing PR #2760, and it implements liveness calculation for G1 full gc. I also suggested [there](https://github.com/openjdk/jdk/pull/2760#discussion_r592449837) to extract this functionality out into an extra CR. Maybe you can work together.

Hi Thomas, Jaroslav,
How about we track jfr liveness event in g1 full gc in a separate bug, so this PR #2579 can go ahead without blocking?
If you don't mind I can work on new bug later after liveness collection in g1 full gc is finished in another separate bug.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From shade at redhat.com  Fri Mar 12 07:21:40 2021
From: shade at redhat.com (Aleksey Shipilev)
Date: Fri, 12 Mar 2021 08:21:40 +0100
Subject: [aarch64-port-dev ] RFR/RFC: Shenandoah integration 2021-03-11
In-Reply-To: <20210311192606.GA145398@rincewind>
References: <600db610-01ff-c160-4a37-aa0192af9b0e@redhat.com>
 <20210311172202.GA7420@rincewind>
 <37f5f62f-cc92-d5de-c747-78ee2df5d906@redhat.com>
 <20210311192606.GA145398@rincewind>
Message-ID: <46ba330f-f85a-9b9f-3e6c-56e1eda89f88@redhat.com>

On 3/11/21 8:26 PM, Andrew Hughes wrote:
>> Anything else you want answered / done?
> 
> I'm happy for this to go in.

Integrated.

-- 
Thanks,
-Aleksey


From shade at openjdk.java.net  Fri Mar 12 10:58:07 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Fri, 12 Mar 2021 10:58:07 GMT
Subject: RFR: 8263427: Shenandoah: Trigger weak-LRB even when heap is
 stable [v2]
In-Reply-To: <RCQWcIV0qg_p-siCOTnDJ6beUlf6fX0iZtvWu2GzgXI=.cdc5bbb6-900c-4198-aac4-aecc140add87@github.com>
References: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
 <RCQWcIV0qg_p-siCOTnDJ6beUlf6fX0iZtvWu2GzgXI=.cdc5bbb6-900c-4198-aac4-aecc140add87@github.com>
Message-ID: <U8Np09PjLq21iBWOUAtC9LC9OO0gK4VsezoOAS73R-k=.b2f207ba-b072-46be-9968-3c5cfb5050ef@github.com>

On Thu, 11 Mar 2021 20:17:28 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> We currently guard all LRBs, including weak-LRB, by a test for heap-stable and only enter the LRB when heap is unstable (e.g. evacuation or update-refs in progress). However, the weak LRB must also be entered when heap is stable and concurrent refs is in progress, otherwise we may accidentally resurrect otherwise unreachable weak referents. This can happen when we take the shortcut cycle and skip evac&update-refs.
>> 
>> I believe this might be the root cause for JDK-8262852. 
>> 
>> The way out of it is change conc-weakroots-in-progress flag to a bit in gc-state, and test for this in weak-LRB gc-state-check, and enter weak-LRB even when heap is stable, but conc-weakroots-in-progress.
>> 
>> There's one gotcha here: we used to change gc-state only at safepoints so that the flag can safely be propagated to all Java threads. But conc-weakroots-in-progress is turned-off concurrently. I deal with this by propagating the flag change to Java threads via the rendevouz (that we do anyway), and change the global flag only once all threads got the thread-local flag change.
>> 
>> This stuff makes the verifier unhappy, because it doesn't know about the new bit. And it'd be difficult to properly verify it, because sometimes it is set (conc-cycle) and sometimes it is not (degen-cycle), so instead of additing extra verification, I figured we could keep ignoring the flag (for now?)
>> 
>> Testing:
>>  - [x] New testcase failed without change, passes now
>>  - [x] hotspot_gc_shenandoah
>>  - [ ] tier1 (+Shenandoah)
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Ensure test does a complete GC cycle before verification

Still don't like the Verifier mess. Here is my attempt to un-confuse Verifier about this (applies on top of this PR): https://cr.openjdk.java.net/~shade/8263427/verifier-1.patch

I think it catches a few threads having "weak roots enabled" state while starting the mark in `gc/stress/systemgc/TestSystemGCWithShenandoah.java`. Please see if this points to lifecycle bugs in current flag handling?

-------------

Changes requested by shade (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2945

From rkennke at openjdk.java.net  Fri Mar 12 14:06:27 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Fri, 12 Mar 2021 14:06:27 GMT
Subject: RFR: 8263427: Shenandoah: Trigger weak-LRB even when heap is
 stable [v3]
In-Reply-To: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
References: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
Message-ID: <mzOOoG1w09sH_f_m9-bntO3lmWDCnUEYpoXO6NruE8Q=.85fdb891-d17f-4e5b-bb4c-4c70fbb9d643@github.com>

> We currently guard all LRBs, including weak-LRB, by a test for heap-stable and only enter the LRB when heap is unstable (e.g. evacuation or update-refs in progress). However, the weak LRB must also be entered when heap is stable and concurrent refs is in progress, otherwise we may accidentally resurrect otherwise unreachable weak referents. This can happen when we take the shortcut cycle and skip evac&update-refs.
> 
> I believe this might be the root cause for JDK-8262852. 
> 
> The way out of it is change conc-weakroots-in-progress flag to a bit in gc-state, and test for this in weak-LRB gc-state-check, and enter weak-LRB even when heap is stable, but conc-weakroots-in-progress.
> 
> There's one gotcha here: we used to change gc-state only at safepoints so that the flag can safely be propagated to all Java threads. But conc-weakroots-in-progress is turned-off concurrently. I deal with this by propagating the flag change to Java threads via the rendevouz (that we do anyway), and change the global flag only once all threads got the thread-local flag change.
> 
> This stuff makes the verifier unhappy, because it doesn't know about the new bit. And it'd be difficult to properly verify it, because sometimes it is set (conc-cycle) and sometimes it is not (degen-cycle), so instead of additing extra verification, I figured we could keep ignoring the flag (for now?)
> 
> Testing:
>  - [x] New testcase failed without change, passes now
>  - [x] hotspot_gc_shenandoah
>  - [ ] tier1 (+Shenandoah)

Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:

 - Correct order of rendezvous, global- and local-flag updates; cleanup rendezvous
 - Verify correct weakroots-in-progress state (by Aleksey)

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2945/files
  - new: https://git.openjdk.java.net/jdk/pull/2945/files/94575b41..739c9b62

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2945&range=02
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2945&range=01-02

  Stats: 55 lines in 5 files changed: 32 ins; 17 del; 6 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2945.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2945/head:pull/2945

PR: https://git.openjdk.java.net/jdk/pull/2945

From github.com+71722661+earthling-amzn at openjdk.java.net  Fri Mar 12 19:36:25 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Fri, 12 Mar 2021 19:36:25 GMT
Subject: RFR: Update periodic gc test for changed heuristic log message
In-Reply-To: <p1_rJxOdByS5tTjMxsOIHbrS-pJuQt-1w_OwSDjdsqY=.22a4b07b-6c60-45fd-9873-35d6cfef1227@github.com>
References: <2VCkfRyfAYiBzdnyaHJHiKoR_Gc05h2XFMLu1ZhbHew=.4b099ff1-9096-439e-bf2e-f10b458fe300@github.com>
 <p1_rJxOdByS5tTjMxsOIHbrS-pJuQt-1w_OwSDjdsqY=.22a4b07b-6c60-45fd-9873-35d6cfef1227@github.com>
Message-ID: <I5L4GfiQlQ3hoFPtfCcJK6lF8jImjATJ2igjObko99M=.29206e27-62e3-4226-8d7b-bb6cbab7683d@github.com>

On Fri, 12 Mar 2021 19:22:32 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> This also includes a change to hold down regulator requests when the GC is already running in non-generational modes.
>
> Looks ok.
> I was just thinking that GCMode kinda clashes with the ShenandoahGCMode (satb, iu, generational) distinction, and is thus a little confusing. It *did* confuse me for a bit :-)

What about `GCPhase` or `GCStage`?

Another enumeration that warrants some discussion: `GenerationMode` - which is confusingly similar to the class name `ShenandoahGenerationalMode` (and the class `ShenandoahGeneration`). I think at the least it should have the Shenandoah prefix, or perhaps declare it inside `ShenandoahGenerationalMode` as just `Generation`?

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/21

From rkennke at openjdk.java.net  Fri Mar 12 19:25:24 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Fri, 12 Mar 2021 19:25:24 GMT
Subject: RFR: Update periodic gc test for changed heuristic log message
In-Reply-To: <2VCkfRyfAYiBzdnyaHJHiKoR_Gc05h2XFMLu1ZhbHew=.4b099ff1-9096-439e-bf2e-f10b458fe300@github.com>
References: <2VCkfRyfAYiBzdnyaHJHiKoR_Gc05h2XFMLu1ZhbHew=.4b099ff1-9096-439e-bf2e-f10b458fe300@github.com>
Message-ID: <p1_rJxOdByS5tTjMxsOIHbrS-pJuQt-1w_OwSDjdsqY=.22a4b07b-6c60-45fd-9873-35d6cfef1227@github.com>

On Thu, 11 Mar 2021 23:53:58 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:

> This also includes a change to hold down regulator requests when the GC is already running in non-generational modes.

Looks ok.
I was just thinking that GCMode kinda clashes with the ShenandoahGCMode (satb, iu, generational) distinction, and is thus a little confusing. It *did* confuse me for a bit :-)

-------------

Marked as reviewed by rkennke (Reviewer).

PR: https://git.openjdk.java.net/shenandoah/pull/21

From iignatyev at openjdk.java.net  Sat Mar 13 04:38:48 2021
From: iignatyev at openjdk.java.net (Igor Ignatyev)
Date: Sat, 13 Mar 2021 04:38:48 GMT
Subject: RFR: 8263549:  8263412 can cause jtreg testlibrary split
Message-ID: <68VznhnTGY9ALWqvXzAulGxWtvI5-z2ljGj8zy07SKc=.1b9ef93b-f288-4e96-8ea7-b7080c93fa4f@github.com>

Hi all,

could you please review this dull patch that replaces `ClassFileInstaller` w/ `jdk.test.lib.helpers.ClassFileInstaller` in all jtreg test descriptions to ensure we won't get split testlibrary, and removes `jdk/test/lib/ClassFileInstaller.java` (so it won't be accidentally used).

from JBS:
> after JDK-8263412, we might (again) encounter NCDFE b/c parts of testlibraries aren't on the classpath. this happens when jtreg builds `jdk.test.lib.helpers.ClassFileInstaller` as a part of test-specific code, but `ClassFileInstaller` as part of shared testibrary directory in one test, when in the following test, jtreg sees `ClassFileInstaller` in the shared directory, hence javac won't recompile it/its dependencies, but in runtime `jdk.test.lib.helpers.ClassFileInstaller` is nowhere to be found, hence we get NCDFE. 

testing:
- [x]  `grep ' ClassFileInstaller[^.]`
- [ ] tier1-3

Thanks,
-- Igor

-------------

Commit messages:
 - fixup
 - update copyright year
 - rm test/lib/ClassFileInstaller.java
 - 's/ ClassFileInstaller / jdk.test.lib.helpers.ClassFileInstaller /g'

Changes: https://git.openjdk.java.net/jdk/pull/2985/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2985&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8263549
  Stats: 1736 lines in 867 files changed: 0 ins; 67 del; 1669 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2985.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2985/head:pull/2985

PR: https://git.openjdk.java.net/jdk/pull/2985

From iignatyev at openjdk.java.net  Sat Mar 13 04:42:08 2021
From: iignatyev at openjdk.java.net (Igor Ignatyev)
Date: Sat, 13 Mar 2021 04:42:08 GMT
Subject: RFR: 8263549:  8263412 can cause jtreg testlibrary split
In-Reply-To: <68VznhnTGY9ALWqvXzAulGxWtvI5-z2ljGj8zy07SKc=.1b9ef93b-f288-4e96-8ea7-b7080c93fa4f@github.com>
References: <68VznhnTGY9ALWqvXzAulGxWtvI5-z2ljGj8zy07SKc=.1b9ef93b-f288-4e96-8ea7-b7080c93fa4f@github.com>
Message-ID: <dEaJYzwXo6IiFslhETMamL51NDQcfBQ7BCfiE_ccsvk=.f5062de3-b0bb-459b-9391-e3bb44a73b33@github.com>

On Sat, 13 Mar 2021 04:31:31 GMT, Igor Ignatyev <iignatyev at openjdk.org> wrote:

> Hi all,
> 
> could you please review this dull patch that replaces `ClassFileInstaller` w/ `jdk.test.lib.helpers.ClassFileInstaller` in all jtreg test descriptions to ensure we won't get split testlibrary, and removes `jdk/test/lib/ClassFileInstaller.java` (so it won't be accidentally used).
> 
> from JBS:
>> after JDK-8263412, we might (again) encounter NCDFE b/c parts of testlibraries aren't on the classpath. this happens when jtreg builds `jdk.test.lib.helpers.ClassFileInstaller` as a part of test-specific code, but `ClassFileInstaller` as part of shared testibrary directory in one test, when in the following test, jtreg sees `ClassFileInstaller` in the shared directory, hence javac won't recompile it/its dependencies, but in runtime `jdk.test.lib.helpers.ClassFileInstaller` is nowhere to be found, hence we get NCDFE. 
> 
> testing:
> - [x]  `grep ' ClassFileInstaller[^.]`
> - [ ] tier1-3
> 
> Thanks,
> -- Igor

note for reviewers: the big part of the patch is just `sed -i  's/ ClassFileInstaller / jdk.test.lib.helpers.ClassFileInstaller /g'`

-------------

PR: https://git.openjdk.java.net/jdk/pull/2985

From iklam at openjdk.java.net  Sat Mar 13 06:19:06 2021
From: iklam at openjdk.java.net (Ioi Lam)
Date: Sat, 13 Mar 2021 06:19:06 GMT
Subject: RFR: 8263549:  8263412 can cause jtreg testlibrary split
In-Reply-To: <68VznhnTGY9ALWqvXzAulGxWtvI5-z2ljGj8zy07SKc=.1b9ef93b-f288-4e96-8ea7-b7080c93fa4f@github.com>
References: <68VznhnTGY9ALWqvXzAulGxWtvI5-z2ljGj8zy07SKc=.1b9ef93b-f288-4e96-8ea7-b7080c93fa4f@github.com>
Message-ID: <2ydx9TUT868fiCQNxF6IaEsq9toXBiDJjJK3GqWRREE=.77fc9c8d-efed-47e8-8f53-02255da6e97e@github.com>

On Sat, 13 Mar 2021 04:31:31 GMT, Igor Ignatyev <iignatyev at openjdk.org> wrote:

> Hi all,
> 
> could you please review this dull patch that replaces `ClassFileInstaller` w/ `jdk.test.lib.helpers.ClassFileInstaller` in all jtreg test descriptions to ensure we won't get split testlibrary, and removes `jdk/test/lib/ClassFileInstaller.java` (so it won't be accidentally used).
> 
> from JBS:
>> after JDK-8263412, we might (again) encounter NCDFE b/c parts of testlibraries aren't on the classpath. this happens when jtreg builds `jdk.test.lib.helpers.ClassFileInstaller` as a part of test-specific code, but `ClassFileInstaller` as part of shared testibrary directory in one test, when in the following test, jtreg sees `ClassFileInstaller` in the shared directory, hence javac won't recompile it/its dependencies, but in runtime `jdk.test.lib.helpers.ClassFileInstaller` is nowhere to be found, hence we get NCDFE. 
> 
> testing:
> - [x]  `grep ' ClassFileInstaller[^.]`
> - [ ] tier1-3
> 
> Thanks,
> -- Igor

I did this and scanned the differences (with the diff file from the webrev) and it looks reasonable to me.

grep '^[+-]' diff.txt | grep -v Copyright | grep -v '^.[+-]' | less

It looks like most of the changes are mechanical. There were only a few cases where manual changes were made. I trusted that you have tested those cases individually.

But I don't understand why this error can happen. It seems like jtreg would allow two test cases to interfere with each other.

-------------

Marked as reviewed by iklam (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2985

From iignatyev at openjdk.java.net  Sat Mar 13 06:30:00 2021
From: iignatyev at openjdk.java.net (Igor Ignatyev)
Date: Sat, 13 Mar 2021 06:30:00 GMT
Subject: RFR: 8263549:  8263412 can cause jtreg testlibrary split [v2]
In-Reply-To: <68VznhnTGY9ALWqvXzAulGxWtvI5-z2ljGj8zy07SKc=.1b9ef93b-f288-4e96-8ea7-b7080c93fa4f@github.com>
References: <68VznhnTGY9ALWqvXzAulGxWtvI5-z2ljGj8zy07SKc=.1b9ef93b-f288-4e96-8ea7-b7080c93fa4f@github.com>
Message-ID: <At8rU6AnwStuhNrALmVmVvxaqOvNe-PqnnddIqLkOB4=.66b0d17b-6f4f-4cf8-b6bf-a5592f9a46e3@github.com>

> Hi all,
> 
> could you please review this dull patch that replaces `ClassFileInstaller` w/ `jdk.test.lib.helpers.ClassFileInstaller` in all jtreg test descriptions to ensure we won't get split testlibrary, and removes `jdk/test/lib/ClassFileInstaller.java` (so it won't be accidentally used).
> 
> from JBS:
>> after JDK-8263412, we might (again) encounter NCDFE b/c parts of testlibraries aren't on the classpath. this happens when jtreg builds `jdk.test.lib.helpers.ClassFileInstaller` as a part of test-specific code, but `ClassFileInstaller` as part of shared testibrary directory in one test, when in the following test, jtreg sees `ClassFileInstaller` in the shared directory, hence javac won't recompile it/its dependencies, but in runtime `jdk.test.lib.helpers.ClassFileInstaller` is nowhere to be found, hence we get NCDFE. 
> 
> testing:
> - [x]  `grep ' ClassFileInstaller[^.]`
> - [ ] tier1-3
> 
> Thanks,
> -- Igor

Igor Ignatyev has updated the pull request incrementally with one additional commit since the last revision:

  fix compilation error in IncorrectAOTLibraryTest test

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2985/files
  - new: https://git.openjdk.java.net/jdk/pull/2985/files/6e53ad97..ff6d4f91

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2985&range=01
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2985&range=00-01

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2985.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2985/head:pull/2985

PR: https://git.openjdk.java.net/jdk/pull/2985

From iignatyev at openjdk.java.net  Sat Mar 13 06:44:12 2021
From: iignatyev at openjdk.java.net (Igor Ignatyev)
Date: Sat, 13 Mar 2021 06:44:12 GMT
Subject: RFR: 8263549:  8263412 can cause jtreg testlibrary split [v3]
In-Reply-To: <68VznhnTGY9ALWqvXzAulGxWtvI5-z2ljGj8zy07SKc=.1b9ef93b-f288-4e96-8ea7-b7080c93fa4f@github.com>
References: <68VznhnTGY9ALWqvXzAulGxWtvI5-z2ljGj8zy07SKc=.1b9ef93b-f288-4e96-8ea7-b7080c93fa4f@github.com>
Message-ID: <c3UIWUcXOnH5Hd_Th0_YIiKCTklnxTtWFyPOC6QfCpQ=.dae1c7f3-0f63-4047-9df9-79ab0e2306f8@github.com>

> Hi all,
> 
> could you please review this dull patch that replaces `ClassFileInstaller` w/ `jdk.test.lib.helpers.ClassFileInstaller` in all jtreg test descriptions to ensure we won't get split testlibrary, and removes `jdk/test/lib/ClassFileInstaller.java` (so it won't be accidentally used).
> 
> from JBS:
>> after JDK-8263412, we might (again) encounter NCDFE b/c parts of testlibraries aren't on the classpath. this happens when jtreg builds `jdk.test.lib.helpers.ClassFileInstaller` as a part of test-specific code, but `ClassFileInstaller` as part of shared testibrary directory in one test, when in the following test, jtreg sees `ClassFileInstaller` in the shared directory, hence javac won't recompile it/its dependencies, but in runtime `jdk.test.lib.helpers.ClassFileInstaller` is nowhere to be found, hence we get NCDFE. 
> 
> testing:
> - [x]  `grep ' ClassFileInstaller[^.]`
> - [ ] tier1-3
> 
> Thanks,
> -- Igor

Igor Ignatyev has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision:

  fix compilation error in IncorrectAOTLibraryTest test

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2985/files
  - new: https://git.openjdk.java.net/jdk/pull/2985/files/ff6d4f91..3a3b7a84

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2985&range=02
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2985&range=01-02

  Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2985.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2985/head:pull/2985

PR: https://git.openjdk.java.net/jdk/pull/2985

From iignatyev at openjdk.java.net  Sat Mar 13 06:44:13 2021
From: iignatyev at openjdk.java.net (Igor Ignatyev)
Date: Sat, 13 Mar 2021 06:44:13 GMT
Subject: RFR: 8263549:  8263412 can cause jtreg testlibrary split [v3]
In-Reply-To: <2ydx9TUT868fiCQNxF6IaEsq9toXBiDJjJK3GqWRREE=.77fc9c8d-efed-47e8-8f53-02255da6e97e@github.com>
References: <68VznhnTGY9ALWqvXzAulGxWtvI5-z2ljGj8zy07SKc=.1b9ef93b-f288-4e96-8ea7-b7080c93fa4f@github.com>
 <2ydx9TUT868fiCQNxF6IaEsq9toXBiDJjJK3GqWRREE=.77fc9c8d-efed-47e8-8f53-02255da6e97e@github.com>
Message-ID: <FmxUanOC2tsK13CBu-bdeJ6gziFttKoyMFipgjTdAEM=.c1bc20fa-7173-44ae-8ea6-60bbc2ac5b6d@github.com>

On Sat, 13 Mar 2021 06:16:37 GMT, Ioi Lam <iklam at openjdk.org> wrote:

>> Igor Ignatyev has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR. The pull request contains one new commit since the last revision:
>> 
>>   fix compilation error in IncorrectAOTLibraryTest test
>
> I did this and scanned the differences (with the diff file from the webrev) and it looks reasonable to me.
> 
> grep '^[+-]' diff.txt | grep -v Copyright | grep -v '^.[+-]' | less
> 
> It looks like most of the changes are mechanical. There were only a few cases where manual changes were made. I trusted that you have tested those cases individually.
> 
> But I don't understand why this error can happen. It seems like jtreg would allow two test cases to interfere with each other.

Hi Ioi,

thanks for review this, I ran the whole tier1-3 jobs which should provide enough coverage. as oracle builds don't have AOT feature enabled, I missed a compilation error in `IncorrectAOTLibraryTest` test. the test failed in GitHub action and should be fixed by [3a3b7a8](https://github.com/openjdk/jdk/pull/2985/commits/3a3b7a846289181b466b3c1eb478a0a571d9468b).

-- Igor

-------------

PR: https://git.openjdk.java.net/jdk/pull/2985

From iklam at openjdk.java.net  Sat Mar 13 07:14:09 2021
From: iklam at openjdk.java.net (Ioi Lam)
Date: Sat, 13 Mar 2021 07:14:09 GMT
Subject: RFR: 8263549:  8263412 can cause jtreg testlibrary split [v3]
In-Reply-To: <2ydx9TUT868fiCQNxF6IaEsq9toXBiDJjJK3GqWRREE=.77fc9c8d-efed-47e8-8f53-02255da6e97e@github.com>
References: <68VznhnTGY9ALWqvXzAulGxWtvI5-z2ljGj8zy07SKc=.1b9ef93b-f288-4e96-8ea7-b7080c93fa4f@github.com>
 <2ydx9TUT868fiCQNxF6IaEsq9toXBiDJjJK3GqWRREE=.77fc9c8d-efed-47e8-8f53-02255da6e97e@github.com>
Message-ID: <uz8GwSfLLDJa8FwDhUBymXo4YkgbnT59TEjSYdHv15Y=.7cc79134-b687-477b-8e72-7dc65cfa8ef7@github.com>

On Sat, 13 Mar 2021 06:16:37 GMT, Ioi Lam <iklam at openjdk.org> wrote:

> But I don't understand why this error can happen. It seems like jtreg would allow two test cases to interfere with each other.

The root cause seems to be https://bugs.openjdk.java.net/browse/CODETOOLS-7902847

-------------

PR: https://git.openjdk.java.net/jdk/pull/2985

From dcubed at openjdk.java.net  Sat Mar 13 14:23:08 2021
From: dcubed at openjdk.java.net (Daniel D.Daugherty)
Date: Sat, 13 Mar 2021 14:23:08 GMT
Subject: RFR: 8263549:  8263412 can cause jtreg testlibrary split [v3]
In-Reply-To: <c3UIWUcXOnH5Hd_Th0_YIiKCTklnxTtWFyPOC6QfCpQ=.dae1c7f3-0f63-4047-9df9-79ab0e2306f8@github.com>
References: <68VznhnTGY9ALWqvXzAulGxWtvI5-z2ljGj8zy07SKc=.1b9ef93b-f288-4e96-8ea7-b7080c93fa4f@github.com>
 <c3UIWUcXOnH5Hd_Th0_YIiKCTklnxTtWFyPOC6QfCpQ=.dae1c7f3-0f63-4047-9df9-79ab0e2306f8@github.com>
Message-ID: <HflI5BUCqXAwVWrhyNcQFIwzvsdBUcUX7AB6Xp5RxQU=.5439c807-612d-4c01-b502-6e075ca70ab2@github.com>

On Sat, 13 Mar 2021 06:44:12 GMT, Igor Ignatyev <iignatyev at openjdk.org> wrote:

>> Hi all,
>> 
>> could you please review this dull patch that replaces `ClassFileInstaller` w/ `jdk.test.lib.helpers.ClassFileInstaller` in all jtreg test descriptions to ensure we won't get split testlibrary, and removes `jdk/test/lib/ClassFileInstaller.java` (so it won't be accidentally used).
>> 
>> from JBS:
>>> after JDK-8263412, we might (again) encounter NCDFE b/c parts of testlibraries aren't on the classpath. this happens when jtreg builds `jdk.test.lib.helpers.ClassFileInstaller` as a part of test-specific code, but `ClassFileInstaller` as part of shared testibrary directory in one test, when in the following test, jtreg sees `ClassFileInstaller` in the shared directory, hence javac won't recompile it/its dependencies, but in runtime `jdk.test.lib.helpers.ClassFileInstaller` is nowhere to be found, hence we get NCDFE. 
>> 
>> testing:
>> - [x]  `grep ' ClassFileInstaller[^.]`
>> - [ ] tier1-3
>> 
>> Thanks,
>> -- Igor
>
> Igor Ignatyev has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR.

I downloaded the patch and used Ioi's cmd pattern to scroll through
the changes. I can't honestly say that I looked at every line since 867
changed files would overwhelm anyone's brain...

I did notice a couple of `@run main` instead of `@run driver` calls
to the ClassFileInstaller, but those are pre-existing.

Thumbs up.

-------------

Marked as reviewed by dcubed (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2985

From iignatyev at openjdk.java.net  Sat Mar 13 14:56:07 2021
From: iignatyev at openjdk.java.net (Igor Ignatyev)
Date: Sat, 13 Mar 2021 14:56:07 GMT
Subject: Integrated: 8263549:  8263412 can cause jtreg testlibrary split
In-Reply-To: <68VznhnTGY9ALWqvXzAulGxWtvI5-z2ljGj8zy07SKc=.1b9ef93b-f288-4e96-8ea7-b7080c93fa4f@github.com>
References: <68VznhnTGY9ALWqvXzAulGxWtvI5-z2ljGj8zy07SKc=.1b9ef93b-f288-4e96-8ea7-b7080c93fa4f@github.com>
Message-ID: <_mpUEh9QVZbxsshQP1hNQlstVSmOwqImgOwhZV4lvIQ=.f49472ae-9d8e-4058-beb8-77e0f7b4fd3a@github.com>

On Sat, 13 Mar 2021 04:31:31 GMT, Igor Ignatyev <iignatyev at openjdk.org> wrote:

> Hi all,
> 
> could you please review this dull patch that replaces `ClassFileInstaller` w/ `jdk.test.lib.helpers.ClassFileInstaller` in all jtreg test descriptions to ensure we won't get split testlibrary, and removes `jdk/test/lib/ClassFileInstaller.java` (so it won't be accidentally used).
> 
> from JBS:
>> after JDK-8263412, we might (again) encounter NCDFE b/c parts of testlibraries aren't on the classpath. this happens when jtreg builds `jdk.test.lib.helpers.ClassFileInstaller` as a part of test-specific code, but `ClassFileInstaller` as part of shared testibrary directory in one test, when in the following test, jtreg sees `ClassFileInstaller` in the shared directory, hence javac won't recompile it/its dependencies, but in runtime `jdk.test.lib.helpers.ClassFileInstaller` is nowhere to be found, hence we get NCDFE. 
> 
> testing:
> - [x]  `grep ' ClassFileInstaller[^.]`
> - [ ] tier1-3
> 
> Thanks,
> -- Igor

This pull request has now been integrated.

Changeset: a7aba2b6
Author:    Igor Ignatyev <iignatyev at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/a7aba2b6
Stats:     1738 lines in 867 files changed: 2 ins; 67 del; 1669 mod

8263549: 8263412 can cause jtreg testlibrary split

Reviewed-by: iklam, dcubed

-------------

PR: https://git.openjdk.java.net/jdk/pull/2985

From iignatyev at openjdk.java.net  Sat Mar 13 14:56:06 2021
From: iignatyev at openjdk.java.net (Igor Ignatyev)
Date: Sat, 13 Mar 2021 14:56:06 GMT
Subject: RFR: 8263549:  8263412 can cause jtreg testlibrary split [v3]
In-Reply-To: <HflI5BUCqXAwVWrhyNcQFIwzvsdBUcUX7AB6Xp5RxQU=.5439c807-612d-4c01-b502-6e075ca70ab2@github.com>
References: <68VznhnTGY9ALWqvXzAulGxWtvI5-z2ljGj8zy07SKc=.1b9ef93b-f288-4e96-8ea7-b7080c93fa4f@github.com>
 <c3UIWUcXOnH5Hd_Th0_YIiKCTklnxTtWFyPOC6QfCpQ=.dae1c7f3-0f63-4047-9df9-79ab0e2306f8@github.com>
 <HflI5BUCqXAwVWrhyNcQFIwzvsdBUcUX7AB6Xp5RxQU=.5439c807-612d-4c01-b502-6e075ca70ab2@github.com>
Message-ID: <cWG4uOCEDr5X77Jm1uCxyw-Z1jjgtU4JUsieDCX1py0=.c6cea055-3a06-4a73-aa82-53fe44e0fd77@github.com>

On Sat, 13 Mar 2021 14:20:20 GMT, Daniel D. Daugherty <dcubed at openjdk.org> wrote:

>> Igor Ignatyev has refreshed the contents of this pull request, and previous commits have been removed. The incremental views will show differences compared to the previous content of the PR.
>
> I downloaded the patch and used Ioi's cmd pattern to scroll through
> the changes. I can't honestly say that I looked at every line since 867
> changed files would overwhelm anyone's brain...
> 
> I did notice a couple of `@run main` instead of `@run driver` calls
> to the ClassFileInstaller, but those are pre-existing.
> 
> Thumbs up.

Hi Dan,

Thanks for your review!

> I did notice a couple of @run main instead of @run driver calls to the ClassFileInstaller, but those are pre-existing.

I noticed this too, planning to fix that with a separate RFE.

-- Igor

-------------

PR: https://git.openjdk.java.net/jdk/pull/2985

From iignatyev at openjdk.java.net  Sat Mar 13 20:18:12 2021
From: iignatyev at openjdk.java.net (Igor Ignatyev)
Date: Sat, 13 Mar 2021 20:18:12 GMT
Subject: RFR: 8263555: use driver-mode to run ClassFileInstaller
Message-ID: <gMTIAjhWZSR8I_9fY-1nIe12mPuNPEPnBu0z92_1hrQ=.05960c05-82d1-406f-9974-7c7d24f63007@github.com>

Hi all,

could you please review this small and trivial clean-up patch that replaces `@run main j.t.l.h.ClassFileInstaller` w/ `@run driver j.t.l.h.ClassFileInstaller`?

from JBS:
> there is no point in running ClassFileInstaller class w/ external flags, so it should be run w/ `@run driver`.  

Thanks,
-- Igor

-------------

Commit messages:
 - 8263555: use driver-mode to run ClassFileInstaller

Changes: https://git.openjdk.java.net/jdk/pull/2989/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2989&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8263555
  Stats: 6 lines in 5 files changed: 0 ins; 0 del; 6 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2989.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2989/head:pull/2989

PR: https://git.openjdk.java.net/jdk/pull/2989

From iklam at openjdk.java.net  Sun Mar 14 05:02:09 2021
From: iklam at openjdk.java.net (Ioi Lam)
Date: Sun, 14 Mar 2021 05:02:09 GMT
Subject: RFR: 8263555: use driver-mode to run ClassFileInstaller
In-Reply-To: <gMTIAjhWZSR8I_9fY-1nIe12mPuNPEPnBu0z92_1hrQ=.05960c05-82d1-406f-9974-7c7d24f63007@github.com>
References: <gMTIAjhWZSR8I_9fY-1nIe12mPuNPEPnBu0z92_1hrQ=.05960c05-82d1-406f-9974-7c7d24f63007@github.com>
Message-ID: <eAUKhODslxhGiN6u9K7VlqkgXKZU7BjDYfMSTUPRr8g=.cd4d2a8d-ee5b-4f31-bdc1-8125da8cc652@github.com>

On Sat, 13 Mar 2021 20:12:32 GMT, Igor Ignatyev <iignatyev at openjdk.org> wrote:

> Hi all,
> 
> could you please review this small and trivial clean-up patch that replaces `@run main j.t.l.h.ClassFileInstaller` w/ `@run driver j.t.l.h.ClassFileInstaller`?
> 
> from JBS:
>> there is no point in running ClassFileInstaller class w/ external flags, so it should be run w/ `@run driver`.  
> 
> Thanks,
> -- Igor

Looks good and trivial.

-------------

Marked as reviewed by iklam (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2989

From iignatyev at openjdk.java.net  Sun Mar 14 05:25:09 2021
From: iignatyev at openjdk.java.net (Igor Ignatyev)
Date: Sun, 14 Mar 2021 05:25:09 GMT
Subject: Integrated: 8263555: use driver-mode to run ClassFileInstaller
In-Reply-To: <gMTIAjhWZSR8I_9fY-1nIe12mPuNPEPnBu0z92_1hrQ=.05960c05-82d1-406f-9974-7c7d24f63007@github.com>
References: <gMTIAjhWZSR8I_9fY-1nIe12mPuNPEPnBu0z92_1hrQ=.05960c05-82d1-406f-9974-7c7d24f63007@github.com>
Message-ID: <lBguSCUF4njzTX-9gbNEIFRsTLYa5b8qvu7ZTVKAPbE=.50a4fa5d-5891-4684-915e-673e255342ba@github.com>

On Sat, 13 Mar 2021 20:12:32 GMT, Igor Ignatyev <iignatyev at openjdk.org> wrote:

> Hi all,
> 
> could you please review this small and trivial clean-up patch that replaces `@run main j.t.l.h.ClassFileInstaller` w/ `@run driver j.t.l.h.ClassFileInstaller`?
> 
> from JBS:
>> there is no point in running ClassFileInstaller class w/ external flags, so it should be run w/ `@run driver`.  
> 
> Thanks,
> -- Igor

This pull request has now been integrated.

Changeset: 9c84899d
Author:    Igor Ignatyev <iignatyev at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/9c84899d
Stats:     6 lines in 5 files changed: 0 ins; 0 del; 6 mod

8263555: use driver-mode to run ClassFileInstaller

Reviewed-by: iklam

-------------

PR: https://git.openjdk.java.net/jdk/pull/2989

From iignatyev at openjdk.java.net  Sun Mar 14 05:25:08 2021
From: iignatyev at openjdk.java.net (Igor Ignatyev)
Date: Sun, 14 Mar 2021 05:25:08 GMT
Subject: RFR: 8263555: use driver-mode to run ClassFileInstaller
In-Reply-To: <eAUKhODslxhGiN6u9K7VlqkgXKZU7BjDYfMSTUPRr8g=.cd4d2a8d-ee5b-4f31-bdc1-8125da8cc652@github.com>
References: <gMTIAjhWZSR8I_9fY-1nIe12mPuNPEPnBu0z92_1hrQ=.05960c05-82d1-406f-9974-7c7d24f63007@github.com>
 <eAUKhODslxhGiN6u9K7VlqkgXKZU7BjDYfMSTUPRr8g=.cd4d2a8d-ee5b-4f31-bdc1-8125da8cc652@github.com>
Message-ID: <pZnU9mkFYh_BbZuxhlRxQszQlRIz1SleSFP7hg41wW0=.a7282bd9-0152-43c6-a45e-51624dfa3c65@github.com>

On Sun, 14 Mar 2021 04:58:56 GMT, Ioi Lam <iklam at openjdk.org> wrote:

>> Hi all,
>> 
>> could you please review this small and trivial clean-up patch that replaces `@run main j.t.l.h.ClassFileInstaller` w/ `@run driver j.t.l.h.ClassFileInstaller`?
>> 
>> from JBS:
>>> there is no point in running ClassFileInstaller class w/ external flags, so it should be run w/ `@run driver`.  
>> 
>> Thanks,
>> -- Igor
>
> Looks good and trivial.

thanks, Ioi!

-------------

PR: https://git.openjdk.java.net/jdk/pull/2989

From zgu at openjdk.java.net  Mon Mar 15 00:29:08 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 15 Mar 2021 00:29:08 GMT
Subject: RFR: 8263427: Shenandoah: Trigger weak-LRB even when heap is
 stable [v3]
In-Reply-To: <mzOOoG1w09sH_f_m9-bntO3lmWDCnUEYpoXO6NruE8Q=.85fdb891-d17f-4e5b-bb4c-4c70fbb9d643@github.com>
References: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
 <mzOOoG1w09sH_f_m9-bntO3lmWDCnUEYpoXO6NruE8Q=.85fdb891-d17f-4e5b-bb4c-4c70fbb9d643@github.com>
Message-ID: <4qDJ9Cdv-DWx84RIVLL-CYL7peOUye65AY8HUXFeTbo=.a6113f50-5362-475a-92bf-fd754b3ec066@github.com>

On Fri, 12 Mar 2021 14:06:27 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> We currently guard all LRBs, including weak-LRB, by a test for heap-stable and only enter the LRB when heap is unstable (e.g. evacuation or update-refs in progress). However, the weak LRB must also be entered when heap is stable and concurrent refs is in progress, otherwise we may accidentally resurrect otherwise unreachable weak referents. This can happen when we take the shortcut cycle and skip evac&update-refs.
>> 
>> I believe this might be the root cause for JDK-8262852. 
>> 
>> The way out of it is change conc-weakroots-in-progress flag to a bit in gc-state, and test for this in weak-LRB gc-state-check, and enter weak-LRB even when heap is stable, but conc-weakroots-in-progress.
>> 
>> There's one gotcha here: we used to change gc-state only at safepoints so that the flag can safely be propagated to all Java threads. But conc-weakroots-in-progress is turned-off concurrently. I deal with this by propagating the flag change to Java threads via the rendevouz (that we do anyway), and change the global flag only once all threads got the thread-local flag change.
>> 
>> This stuff makes the verifier unhappy, because it doesn't know about the new bit. And it'd be difficult to properly verify it, because sometimes it is set (conc-cycle) and sometimes it is not (degen-cycle), so instead of additing extra verification, I figured we could keep ignoring the flag (for now?)
>> 
>> Testing:
>>  - [x] New testcase failed without change, passes now
>>  - [x] hotspot_gc_shenandoah
>>  - [ ] tier1 (+Shenandoah)
>
> Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Correct order of rendezvous, global- and local-flag updates; cleanup rendezvous
>  - Verify correct weakroots-in-progress state (by Aleksey)

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1737:

> 1735:   // We need a rendezvous here to avoid the following race:
> 1736:   // 1. Java thread reads referent, sees non-null but unreachable oop
> 1737:   // 2. GC thread clears the referent

How is this possible? GC threads (workers) are not running when rendezvous roots. What do I miss?

-------------

PR: https://git.openjdk.java.net/jdk/pull/2945

From iignatyev at openjdk.java.net  Mon Mar 15 05:03:20 2021
From: iignatyev at openjdk.java.net (Igor Ignatyev)
Date: Mon, 15 Mar 2021 05:03:20 GMT
Subject: RFR: 8263556: remove `@modules java.base` from tests
Message-ID: <mhVD9GRUgDYhNnaeM4_mNSg3jUjqrk9HV48wBI0cO2E=.ce4baf13-aed3-4a44-8482-9b2930bc94d8@github.com>

Hi all,

could you please review this trivial cleanup?
from JBS:

> jtreg `@modules X` directive does two things: 
>  - exclude a test from execution if JDK under test doesn't have module X 
>  - if JDK under test has module X, make sure it's resolved 
> 
> both these things have no sense for `java.base` module as it's always available and is always resolved.


Thanks,
-- Igor

-------------

Commit messages:
 - update copyright year
 - 8263556: remove `@modules java.base` from tests

Changes: https://git.openjdk.java.net/jdk/pull/2990/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2990&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8263556
  Stats: 21 lines in 13 files changed: 0 ins; 13 del; 8 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2990.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2990/head:pull/2990

PR: https://git.openjdk.java.net/jdk/pull/2990

From jbachorik at openjdk.java.net  Mon Mar 15 09:25:12 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 15 Mar 2021 09:25:12 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <qL39He-Ha5kAtNvPFRFatfEG4Qz_Qr6Qw-I_gRNkkVc=.cb1a6d64-faac-46bb-bc2c-60885ddc56ed@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
 <qL39He-Ha5kAtNvPFRFatfEG4Qz_Qr6Qw-I_gRNkkVc=.cb1a6d64-faac-46bb-bc2c-60885ddc56ed@github.com>
Message-ID: <yQWl1ySFwQV0McTx-gzYap7kQPlxvfgiKygYzW9qRI8=.da8ac1bd-d345-44e8-9be5-d0d1e9705f1d@github.com>

On Mon, 15 Mar 2021 09:18:52 GMT, Jaroslav Bachorik <jbachorik at openjdk.org> wrote:

>> src/hotspot/share/gc/shared/space.inline.hpp line 140:
>> 
>>> 138:   size_t get_dead_space() {
>>> 139:     return (_max_deadspace_words - _allowed_deadspace_words) * HeapWordSize;
>>> 140:   }
>> 
>> Hotspot does not use a "get_" prefix for getters. Also not sure why this needs to be private (and the friend class), I would prefer this instead of the friending. Retrieving the actual amount of dead space from a class that calculates it does not  seem something that needs hiding.
>
> The visibility for `_dead_space` was changed based on this comment: https://github.com/openjdk/jdk/pull/2579/files/f69541864e093bc5b250bf625ec75983764ba2bf#r585771280 by @shipilev

Also, I see a bunch of methods named `get_*` in GC code alone. I have no problem renaming it to eg. `dead_space()` but it does not seem that this naming pattern is not used in hotspot.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar 15 09:25:12 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 15 Mar 2021 09:25:12 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
Message-ID: <qL39He-Ha5kAtNvPFRFatfEG4Qz_Qr6Qw-I_gRNkkVc=.cb1a6d64-faac-46bb-bc2c-60885ddc56ed@github.com>

On Thu, 11 Mar 2021 14:44:10 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

>> Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Remove unused field
>
> src/hotspot/share/gc/shared/space.inline.hpp line 140:
> 
>> 138:   size_t get_dead_space() {
>> 139:     return (_max_deadspace_words - _allowed_deadspace_words) * HeapWordSize;
>> 140:   }
> 
> Hotspot does not use a "get_" prefix for getters. Also not sure why this needs to be private (and the friend class), I would prefer this instead of the friending. Retrieving the actual amount of dead space from a class that calculates it does not  seem something that needs hiding.

The visibility for `_dead_space` was changed based on this comment: https://github.com/openjdk/jdk/pull/2579/files/f69541864e093bc5b250bf625ec75983764ba2bf#r585771280 by @shipilev

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From rkennke at openjdk.java.net  Mon Mar 15 09:30:12 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 15 Mar 2021 09:30:12 GMT
Subject: RFR: 8263427: Shenandoah: Trigger weak-LRB even when heap is
 stable [v3]
In-Reply-To: <4qDJ9Cdv-DWx84RIVLL-CYL7peOUye65AY8HUXFeTbo=.a6113f50-5362-475a-92bf-fd754b3ec066@github.com>
References: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
 <mzOOoG1w09sH_f_m9-bntO3lmWDCnUEYpoXO6NruE8Q=.85fdb891-d17f-4e5b-bb4c-4c70fbb9d643@github.com>
 <4qDJ9Cdv-DWx84RIVLL-CYL7peOUye65AY8HUXFeTbo=.a6113f50-5362-475a-92bf-fd754b3ec066@github.com>
Message-ID: <jBvuRNYUp51aeP1xoyEsKBbCwdduiNLkQ9295CCY-XA=.cde4c6d7-f9f8-4bf3-8336-69a043066e9c@github.com>

On Mon, 15 Mar 2021 00:25:54 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
>> 
>>  - Correct order of rendezvous, global- and local-flag updates; cleanup rendezvous
>>  - Verify correct weakroots-in-progress state (by Aleksey)
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1737:
> 
>> 1735:   // We need a rendezvous here to avoid the following race:
>> 1736:   // 1. Java thread reads referent, sees non-null but unreachable oop
>> 1737:   // 2. GC thread clears the referent
> 
> How is this possible? GC threads (workers) are not running when rendezvous roots. What do I miss?

But they are running *before* rendezvous roots.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2945

From jbachorik at openjdk.java.net  Mon Mar 15 09:31:10 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 15 Mar 2021 09:31:10 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <nR7IvUeT4SQXZ3JLFAqeebGf_7jIbEgNlscTI2Wre3A=.b2bbdcdb-7e13-40c2-9154-a81c2f19feb0@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
 <nR7IvUeT4SQXZ3JLFAqeebGf_7jIbEgNlscTI2Wre3A=.b2bbdcdb-7e13-40c2-9154-a81c2f19feb0@github.com>
Message-ID: <P9p-koXqhcZktsnAcU0NGu7Bfsd1PjLOB0QOIzdEo7Y=.423674c1-d518-4776-9dc9-dbcbe7f95337@github.com>

On Thu, 11 Mar 2021 15:42:51 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

>> I am leaving this as "request changes" for now as the question I had earlier about that after G1 Full gc the value of `_live_estimate` still seems unanswered and there does not seem to be code in this change for this. Is this intentional? (Not even setting the live bytes to `used()` which at that point would be a good estimate)
>> 
>> There is another PR (#2760) that implements something like that although I haven't looked at it in detail.
>> 
>> Otherwise looks okay.
>
> Started reviewing PR #2760, and it implements liveness calculation for G1 full gc. I also suggested [there](https://github.com/openjdk/jdk/pull/2760#discussion_r592449837) to extract this functionality out into an extra CR. Maybe you can work together.

@tschatzl @Hamlin-Li 
Would it be ok to set the live estimate to the `used()` value at the end of `G1FullCollector::phase4_do_compaction()` method to have something suboptimal but working and refine in https://github.com/openjdk/jdk/pull/2760 (or a subsequent ticket/PR once both parts are ready)?

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From roland at openjdk.java.net  Mon Mar 15 10:12:19 2021
From: roland at openjdk.java.net (Roland Westrelin)
Date: Mon, 15 Mar 2021 10:12:19 GMT
Subject: RFR: 8263376: CTW (Shenandoah): assert(mems <= 1) failed: No node
 right after call if multiple mem projections
Message-ID: <d9ZCPTRo-b7lYXvGIBBdZ7TE4dl-rI076HyZZeK4gDA=.dba7433d-033b-4e16-b984-9bde627bcfe4@github.com>

This is another case of anti-dependence analysis being too conservative.

In TestBadRawMemoryAfterCall.test2(), test

if (i == 42) {

is split thru the Phi that merges values from the fallthru and
exception paths. As a consequence, control flow at the call is
roughly:
a.m() call
|         \       
fallthru  exception
|             |
if (i == 42)  |
|      \      |
|       Region1
\        /
  Region2
When anti-dependence analysis runs for the load after the call, it
starts from the memory state out of the call on the fallthru path. One
use is a memory Phi at Region2 (say Phi2). Another path leads to
Region1 at, say, Phi1.

When anti-dependence analysis then goes over Phis, it processes Phi2,
goes over the its inputs and finds that 2 are reachable: the one that
has a direct edge to the memory state on the fallthru path and the one
from Phi1. As a consequence, control for the load is set to be right
after the call, which is too conservative.

When following the memory edges, the code stops at Phi1. It doesn't
process uses of Phi1. Any anti-dependence that's needed between the
load and Phi1 is then taken into account when Phis are processed. When
inputs to Phi2 are processed, considering the Phi2->Phi1 is too
conservative. As mentioned above, anti-dependences for Phi1 are taken
into account separately. I think it's true for all Phi->Phi edges that
they can be safely ignored. That's what I propose as a fix.

-------------

Commit messages:
 - test & fix

Changes: https://git.openjdk.java.net/jdk/pull/3006/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3006&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8263376
  Stats: 29 lines in 2 files changed: 24 ins; 0 del; 5 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3006.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3006/head:pull/3006

PR: https://git.openjdk.java.net/jdk/pull/3006

From shade at redhat.com  Mon Mar 15 10:34:37 2021
From: shade at redhat.com (shade at redhat.com)
Date: Mon, 15 Mar 2021 10:34:37 +0000
Subject: hg: shenandoah/jdk8/jdk: Added tag
 aarch64-shenandoah-jdk8u292-b05-shenandoah-merge-2021-03-11 for
 changeset a4c01cb47bd2
Message-ID: <202103151034.12FAYbCv021454@aojmv0008.oracle.com>

Changeset: 0dc765586e7b
Author:    shade
Date:      2021-03-11 16:02 +0100
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/jdk/rev/0dc765586e7b

Added tag aarch64-shenandoah-jdk8u292-b05-shenandoah-merge-2021-03-11 for changeset a4c01cb47bd2

! .hgtags


From shade at redhat.com  Mon Mar 15 10:34:40 2021
From: shade at redhat.com (shade at redhat.com)
Date: Mon, 15 Mar 2021 10:34:40 +0000
Subject: hg: shenandoah/jdk8/jaxws: Added tag
 aarch64-shenandoah-jdk8u292-b05-shenandoah-merge-2021-03-11 for
 changeset 69dd011f38fd
Message-ID: <202103151034.12FAYeJ2021475@aojmv0008.oracle.com>

Changeset: c30e21d25c12
Author:    shade
Date:      2021-03-11 16:02 +0100
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/jaxws/rev/c30e21d25c12

Added tag aarch64-shenandoah-jdk8u292-b05-shenandoah-merge-2021-03-11 for changeset 69dd011f38fd

! .hgtags


From shade at redhat.com  Mon Mar 15 10:34:37 2021
From: shade at redhat.com (shade at redhat.com)
Date: Mon, 15 Mar 2021 10:34:37 +0000
Subject: hg: shenandoah/jdk8: Added tag
 aarch64-shenandoah-jdk8u292-b05-shenandoah-merge-2021-03-11 for
 changeset c0b2bd6ee6d2
Message-ID: <202103151034.12FAYcn6021458@aojmv0008.oracle.com>

Changeset: 359c0dadb84e
Author:    shade
Date:      2021-03-11 16:02 +0100
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/rev/359c0dadb84e

Added tag aarch64-shenandoah-jdk8u292-b05-shenandoah-merge-2021-03-11 for changeset c0b2bd6ee6d2

! .hgtags


From shade at redhat.com  Mon Mar 15 10:34:39 2021
From: shade at redhat.com (shade at redhat.com)
Date: Mon, 15 Mar 2021 10:34:39 +0000
Subject: hg: shenandoah/jdk8/nashorn: Added tag
 aarch64-shenandoah-jdk8u292-b05-shenandoah-merge-2021-03-11 for
 changeset 5c5f63062b6c
Message-ID: <202103151034.12FAYdpA021468@aojmv0008.oracle.com>

Changeset: 9069b365c2ec
Author:    shade
Date:      2021-03-11 16:02 +0100
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/nashorn/rev/9069b365c2ec

Added tag aarch64-shenandoah-jdk8u292-b05-shenandoah-merge-2021-03-11 for changeset 5c5f63062b6c

! .hgtags


From shade at redhat.com  Mon Mar 15 10:34:38 2021
From: shade at redhat.com (shade at redhat.com)
Date: Mon, 15 Mar 2021 10:34:38 +0000
Subject: hg: shenandoah/jdk8/corba: Added tag
 aarch64-shenandoah-jdk8u292-b05-shenandoah-merge-2021-03-11 for
 changeset 1fd2eb862203
Message-ID: <202103151034.12FAYcaa021462@aojmv0008.oracle.com>

Changeset: 5ee715554881
Author:    shade
Date:      2021-03-11 16:02 +0100
URL:       https://hg.openjdk.java.net/shenandoah/jdk8/corba/rev/5ee715554881

Added tag aarch64-shenandoah-jdk8u292-b05-shenandoah-merge-2021-03-11 for changeset 1fd2eb862203

! .hgtags


From sjohanss at openjdk.java.net  Mon Mar 15 12:27:17 2021
From: sjohanss at openjdk.java.net (Stefan Johansson)
Date: Mon, 15 Mar 2021 12:27:17 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <P9p-koXqhcZktsnAcU0NGu7Bfsd1PjLOB0QOIzdEo7Y=.423674c1-d518-4776-9dc9-dbcbe7f95337@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
 <nR7IvUeT4SQXZ3JLFAqeebGf_7jIbEgNlscTI2Wre3A=.b2bbdcdb-7e13-40c2-9154-a81c2f19feb0@github.com>
 <P9p-koXqhcZktsnAcU0NGu7Bfsd1PjLOB0QOIzdEo7Y=.423674c1-d518-4776-9dc9-dbcbe7f95337@github.com>
Message-ID: <jisIvoUcZWmMRMvBSUlmPn3LECqPpbQ3--tcst7iSzg=.f961458b-5e71-4344-b442-42dbff80da83@github.com>

On Mon, 15 Mar 2021 09:26:32 GMT, Jaroslav Bachorik <jbachorik at openjdk.org> wrote:

>> Started reviewing PR #2760, and it implements liveness calculation for G1 full gc. I also suggested [there](https://github.com/openjdk/jdk/pull/2760#discussion_r592449837) to extract this functionality out into an extra CR. Maybe you can work together.
>
> @tschatzl @Hamlin-Li 
> Would it be ok to set the live estimate to the `used()` value at the end of `G1FullCollector::phase4_do_compaction()` method to have something suboptimal but working and refine in https://github.com/openjdk/jdk/pull/2760 (or a subsequent ticket/PR once both parts are ready)?

Sorry for being a bit late to the party. Looking at the suggested implementation for G1 I see a problem with only updating this after concurrent mark (and the Full GC). Say for example you have a concurrent mark cycle before the heap has expanded a lot and you get a low value stored in `G1CollectedHeap::_live`. Then the heap expands and your application get to a steady state that doesn't require any more marking cycles. In this case the same value will be reported for the entire run.

For this to work the _live value would have to be updated at every GC, but this is a bit costly. Maybe the first version could just use `used()` for G1. Have you done any tests to see how off this would be compared to the other GCs?

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From dcubed at openjdk.java.net  Mon Mar 15 13:33:09 2021
From: dcubed at openjdk.java.net (Daniel D.Daugherty)
Date: Mon, 15 Mar 2021 13:33:09 GMT
Subject: RFR: 8263556: remove `@modules java.base` from tests
In-Reply-To: <mhVD9GRUgDYhNnaeM4_mNSg3jUjqrk9HV48wBI0cO2E=.ce4baf13-aed3-4a44-8482-9b2930bc94d8@github.com>
References: <mhVD9GRUgDYhNnaeM4_mNSg3jUjqrk9HV48wBI0cO2E=.ce4baf13-aed3-4a44-8482-9b2930bc94d8@github.com>
Message-ID: <vt_ig8proE47pyW_GO65iNSBJze1J1lth7funqC5hLY=.767f73f0-c662-40de-961d-8f6e542abdff@github.com>

On Sat, 13 Mar 2021 20:26:42 GMT, Igor Ignatyev <iignatyev at openjdk.org> wrote:

> Hi all,
> 
> could you please review this trivial cleanup?
> from JBS:
> 
>> jtreg `@modules X` directive does two things: 
>>  - exclude a test from execution if JDK under test doesn't have module X 
>>  - if JDK under test has module X, make sure it's resolved 
>> 
>> both these things have no sense for `java.base` module as it's always available and is always resolved.
> 
> 
> Thanks,
> -- Igor

Thumbs up.

I agree that this is a trivial change.

-------------

Marked as reviewed by dcubed (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2990

From zgu at openjdk.java.net  Mon Mar 15 14:08:09 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 15 Mar 2021 14:08:09 GMT
Subject: RFR: 8263427: Shenandoah: Trigger weak-LRB even when heap is
 stable [v3]
In-Reply-To: <jBvuRNYUp51aeP1xoyEsKBbCwdduiNLkQ9295CCY-XA=.cde4c6d7-f9f8-4bf3-8336-69a043066e9c@github.com>
References: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
 <mzOOoG1w09sH_f_m9-bntO3lmWDCnUEYpoXO6NruE8Q=.85fdb891-d17f-4e5b-bb4c-4c70fbb9d643@github.com>
 <4qDJ9Cdv-DWx84RIVLL-CYL7peOUye65AY8HUXFeTbo=.a6113f50-5362-475a-92bf-fd754b3ec066@github.com>
 <jBvuRNYUp51aeP1xoyEsKBbCwdduiNLkQ9295CCY-XA=.cde4c6d7-f9f8-4bf3-8336-69a043066e9c@github.com>
Message-ID: <91kRqt6lWB9itbiGgBo0ZoHIbVjeumUFHoFOY_Pc2wY=.f4d5f185-f689-4cd8-a41c-85482aec676a@github.com>

On Mon, 15 Mar 2021 09:27:01 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1737:
>> 
>>> 1735:   // We need a rendezvous here to avoid the following race:
>>> 1736:   // 1. Java thread reads referent, sees non-null but unreachable oop
>>> 1737:   // 2. GC thread clears the referent
>> 
>> How is this possible? GC threads (workers) are not running when rendezvous roots. What do I miss?
>
> But they are running *before* rendezvous roots.

Okay, looks good.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2945

From jbachorik at openjdk.java.net  Mon Mar 15 15:24:12 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 15 Mar 2021 15:24:12 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <jisIvoUcZWmMRMvBSUlmPn3LECqPpbQ3--tcst7iSzg=.f961458b-5e71-4344-b442-42dbff80da83@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
 <nR7IvUeT4SQXZ3JLFAqeebGf_7jIbEgNlscTI2Wre3A=.b2bbdcdb-7e13-40c2-9154-a81c2f19feb0@github.com>
 <P9p-koXqhcZktsnAcU0NGu7Bfsd1PjLOB0QOIzdEo7Y=.423674c1-d518-4776-9dc9-dbcbe7f95337@github.com>
 <jisIvoUcZWmMRMvBSUlmPn3LECqPpbQ3--tcst7iSzg=.f961458b-5e71-4344-b442-42dbff80da83@github.com>
Message-ID: <_1nv16MS2e-Ed1xHZyA8hewbCCCSCCkq0OafP4rRJq8=.75ee1e9d-17b3-46c3-87d1-c03ef2765cf3@github.com>

On Mon, 15 Mar 2021 12:24:02 GMT, Stefan Johansson <sjohanss at openjdk.org> wrote:

>> @tschatzl @Hamlin-Li 
>> Would it be ok to set the live estimate to the `used()` value at the end of `G1FullCollector::phase4_do_compaction()` method to have something suboptimal but working and refine in https://github.com/openjdk/jdk/pull/2760 (or a subsequent ticket/PR once both parts are ready)?
>
> Sorry for being a bit late to the party. Looking at the suggested implementation for G1 I see a problem with only updating this after concurrent mark (and the Full GC). Say for example you have a concurrent mark cycle before the heap has expanded a lot and you get a low value stored in `G1CollectedHeap::_live`. Then the heap expands and your application get to a steady state that doesn't require any more marking cycles. In this case the same value will be reported for the entire run.
> 
> For this to work the _live value would have to be updated at every GC, but this is a bit costly. Maybe the first version could just use `used()` for G1. Have you done any tests to see how off this would be compared to the other GCs?

@kstefanj 
> Then the heap expands and your application get to a steady state that doesn't require any more marking cycles. 

Is there a way to get the liveness info when the heap expands? If not that would mean we had no way to figure out the new live set size and would assume, conservatively, the last known value.

As I mentioned in the PR description the live size value will be a 'best effort estimate' depending on what can each particular GC provide.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From tschatzl at openjdk.java.net  Mon Mar 15 15:41:11 2021
From: tschatzl at openjdk.java.net (Thomas Schatzl)
Date: Mon, 15 Mar 2021 15:41:11 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <yQWl1ySFwQV0McTx-gzYap7kQPlxvfgiKygYzW9qRI8=.da8ac1bd-d345-44e8-9be5-d0d1e9705f1d@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
 <qL39He-Ha5kAtNvPFRFatfEG4Qz_Qr6Qw-I_gRNkkVc=.cb1a6d64-faac-46bb-bc2c-60885ddc56ed@github.com>
 <yQWl1ySFwQV0McTx-gzYap7kQPlxvfgiKygYzW9qRI8=.da8ac1bd-d345-44e8-9be5-d0d1e9705f1d@github.com>
Message-ID: <ajtmJacq8eysFDYEipZ53Vye1-L_Nt2LCw-Hsmn6fMo=.5bf36fae-0c07-41f8-81d7-da49975b5ffe@github.com>

On Mon, 15 Mar 2021 09:21:50 GMT, Jaroslav Bachorik <jbachorik at openjdk.org> wrote:

>> The visibility for `_dead_space` was changed based on this comment: https://github.com/openjdk/jdk/pull/2579/files/f69541864e093bc5b250bf625ec75983764ba2bf#r585771280 by @shipilev
>
> Also, I see a bunch of methods named `get_*` in GC code alone. I have no problem renaming it to eg. `dead_space()` but it does not seem that this naming pattern is not used in hotspot.

https://wiki.openjdk.java.net/display/HotSpot/StyleGuide

> Nearly all of the guidelines mentioned below have many counter-examples in the Hotspot code base. Finding a counterexample is not sufficient justification for new code to follow the counterexample as a precedent, since readers of your code will rightfully expect your code to follow the greater bulk of precedents documented here. For more on counterexamples, see the section at the bottom of this page.
> 
>    When changing pre-existing code, it is reasonable to adjust it to match these conventions. Exception: If the pre-existing code clearly conforms locally to its own peculiar conventions, it is not worth reformatting the whole thing.

and

>  Getter accessor names are noun phrases, with no "get_" noise word. Boolean getters can also begin with "is_" or "has_".

Unless there is a good reason, please keep to few rules the official style guide for new code.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar 15 16:03:12 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 15 Mar 2021 16:03:12 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <ajtmJacq8eysFDYEipZ53Vye1-L_Nt2LCw-Hsmn6fMo=.5bf36fae-0c07-41f8-81d7-da49975b5ffe@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
 <qL39He-Ha5kAtNvPFRFatfEG4Qz_Qr6Qw-I_gRNkkVc=.cb1a6d64-faac-46bb-bc2c-60885ddc56ed@github.com>
 <yQWl1ySFwQV0McTx-gzYap7kQPlxvfgiKygYzW9qRI8=.da8ac1bd-d345-44e8-9be5-d0d1e9705f1d@github.com>
 <ajtmJacq8eysFDYEipZ53Vye1-L_Nt2LCw-Hsmn6fMo=.5bf36fae-0c07-41f8-81d7-da49975b5ffe@github.com>
Message-ID: <hkkmrXe5akOPWzzGnc8hbk_WC0B99u_-Udxf1A5prr0=.78d05dbe-012f-4457-92e9-27884460566d@github.com>

On Mon, 15 Mar 2021 15:38:48 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

>> Also, I see a bunch of methods named `get_*` in GC code alone. I have no problem renaming it to eg. `dead_space()` but it does not seem that this naming pattern is not used in hotspot.
>
> https://wiki.openjdk.java.net/display/HotSpot/StyleGuide
> 
>> Nearly all of the guidelines mentioned below have many counter-examples in the Hotspot code base. Finding a counterexample is not sufficient justification for new code to follow the counterexample as a precedent, since readers of your code will rightfully expect your code to follow the greater bulk of precedents documented here. For more on counterexamples, see the section at the bottom of this page.
>> 
>>    When changing pre-existing code, it is reasonable to adjust it to match these conventions. Exception: If the pre-existing code clearly conforms locally to its own peculiar conventions, it is not worth reformatting the whole thing.
> 
> and
> 
>>  Getter accessor names are noun phrases, with no "get_" noise word. Boolean getters can also begin with "is_" or "has_".
> 
> Unless there is a good reason, please keep to few rules the official style guide for new code.

??

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Mon Mar 15 16:17:23 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 15 Mar 2021 16:17:23 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v13]
In-Reply-To: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
Message-ID: <SzaXFZcKjIcK-bibCKGqGjpQ49O0Rut45CEzIkNFzXs=.d1a8d23b-096e-4c7f-a6f5-ab0444aa1fc4@github.com>

> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
> 
> ## Introducing new JFR event
> 
> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
> 
> ## Implementation
> 
> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
> 
> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
> 
> ### Epsilon GC
> 
> Trivial implementation - just return `used()` instead.
> 
> ### Serial GC
> 
> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
> 
> ### Parallel GC
> 
> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
> 
> ### G1 GC
> 
> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
> 
> ### Shenandoah
> 
> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
> 
> ### ZGC
> 
> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.

Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:

  Change get_dead_space() to dead_space()

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2579/files
  - new: https://git.openjdk.java.net/jdk/pull/2579/files/67d78940..056f5fd7

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=12
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=11-12

  Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2579.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2579/head:pull/2579

PR: https://git.openjdk.java.net/jdk/pull/2579

From naoto at openjdk.java.net  Mon Mar 15 16:21:08 2021
From: naoto at openjdk.java.net (Naoto Sato)
Date: Mon, 15 Mar 2021 16:21:08 GMT
Subject: RFR: 8263556: remove `@modules java.base` from tests
In-Reply-To: <mhVD9GRUgDYhNnaeM4_mNSg3jUjqrk9HV48wBI0cO2E=.ce4baf13-aed3-4a44-8482-9b2930bc94d8@github.com>
References: <mhVD9GRUgDYhNnaeM4_mNSg3jUjqrk9HV48wBI0cO2E=.ce4baf13-aed3-4a44-8482-9b2930bc94d8@github.com>
Message-ID: <HyuCdO-YEWWAPBA0aMaYStTr9sbkPq4CLtDhDDKFOqo=.d48cb16a-9c44-43bc-9982-8caaa5cb77ea@github.com>

On Sat, 13 Mar 2021 20:26:42 GMT, Igor Ignatyev <iignatyev at openjdk.org> wrote:

> Hi all,
> 
> could you please review this trivial cleanup?
> from JBS:
> 
>> jtreg `@modules X` directive does two things: 
>>  - exclude a test from execution if JDK under test doesn't have module X 
>>  - if JDK under test has module X, make sure it's resolved 
>> 
>> both these things have no sense for `java.base` module as it's always available and is always resolved.
> 
> 
> Thanks,
> -- Igor

Marked as reviewed by naoto (Reviewer).

-------------

PR: https://git.openjdk.java.net/jdk/pull/2990

From iris at openjdk.java.net  Mon Mar 15 16:28:08 2021
From: iris at openjdk.java.net (Iris Clark)
Date: Mon, 15 Mar 2021 16:28:08 GMT
Subject: RFR: 8263556: remove `@modules java.base` from tests
In-Reply-To: <mhVD9GRUgDYhNnaeM4_mNSg3jUjqrk9HV48wBI0cO2E=.ce4baf13-aed3-4a44-8482-9b2930bc94d8@github.com>
References: <mhVD9GRUgDYhNnaeM4_mNSg3jUjqrk9HV48wBI0cO2E=.ce4baf13-aed3-4a44-8482-9b2930bc94d8@github.com>
Message-ID: <THNM-XilKIXLEJ5ppuFH13GRabVhgx82vU7VYnNygJA=.70eb2d24-6ab2-4e0c-ac7f-2f1776513d97@github.com>

On Sat, 13 Mar 2021 20:26:42 GMT, Igor Ignatyev <iignatyev at openjdk.org> wrote:

> Hi all,
> 
> could you please review this trivial cleanup?
> from JBS:
> 
>> jtreg `@modules X` directive does two things: 
>>  - exclude a test from execution if JDK under test doesn't have module X 
>>  - if JDK under test has module X, make sure it's resolved 
>> 
>> both these things have no sense for `java.base` module as it's always available and is always resolved.
> 
> 
> Thanks,
> -- Igor

Marked as reviewed by iris (Reviewer).

-------------

PR: https://git.openjdk.java.net/jdk/pull/2990

From jbachorik at openjdk.java.net  Mon Mar 15 16:32:31 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Mon, 15 Mar 2021 16:32:31 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v14]
In-Reply-To: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
Message-ID: <ZEmBg0iptiiFA1JMhTNs-SGingf4JGr2uEKWKgX6puU=.a9a38b12-dc14-46d7-ac32-e3ec2d7b5c4f@github.com>

> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
> 
> ## Introducing new JFR event
> 
> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
> 
> ## Implementation
> 
> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
> 
> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
> 
> ### Epsilon GC
> 
> Trivial implementation - just return `used()` instead.
> 
> ### Serial GC
> 
> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
> 
> ### Parallel GC
> 
> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
> 
> ### G1 GC
> 
> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
> 
> ### Shenandoah
> 
> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
> 
> ### ZGC
> 
> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.

Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:

  Capture live estimate for G1 full cycle

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2579/files
  - new: https://git.openjdk.java.net/jdk/pull/2579/files/056f5fd7..81250d1c

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=13
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=12-13

  Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2579.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2579/head:pull/2579

PR: https://git.openjdk.java.net/jdk/pull/2579

From iignatyev at openjdk.java.net  Mon Mar 15 17:08:09 2021
From: iignatyev at openjdk.java.net (Igor Ignatyev)
Date: Mon, 15 Mar 2021 17:08:09 GMT
Subject: RFR: 8263556: remove `@modules java.base` from tests
In-Reply-To: <THNM-XilKIXLEJ5ppuFH13GRabVhgx82vU7VYnNygJA=.70eb2d24-6ab2-4e0c-ac7f-2f1776513d97@github.com>
References: <mhVD9GRUgDYhNnaeM4_mNSg3jUjqrk9HV48wBI0cO2E=.ce4baf13-aed3-4a44-8482-9b2930bc94d8@github.com>
 <THNM-XilKIXLEJ5ppuFH13GRabVhgx82vU7VYnNygJA=.70eb2d24-6ab2-4e0c-ac7f-2f1776513d97@github.com>
Message-ID: <W4rj4p-NDWVUTm1bihvZuOXxeH550Oks3UZbs7G5Am0=.db3e6087-440d-4da4-8b21-a55780dfe190@github.com>

On Mon, 15 Mar 2021 16:25:48 GMT, Iris Clark <iris at openjdk.org> wrote:

>> Hi all,
>> 
>> could you please review this trivial cleanup?
>> from JBS:
>> 
>>> jtreg `@modules X` directive does two things: 
>>>  - exclude a test from execution if JDK under test doesn't have module X 
>>>  - if JDK under test has module X, make sure it's resolved 
>>> 
>>> both these things have no sense for `java.base` module as it's always available and is always resolved.
>> 
>> 
>> Thanks,
>> -- Igor
>
> Marked as reviewed by iris (Reviewer).

Iris, Naoto, Dan, thank you for your reviews!

-- Igor

-------------

PR: https://git.openjdk.java.net/jdk/pull/2990

From iignatyev at openjdk.java.net  Mon Mar 15 17:08:10 2021
From: iignatyev at openjdk.java.net (Igor Ignatyev)
Date: Mon, 15 Mar 2021 17:08:10 GMT
Subject: Integrated: 8263556: remove `@modules java.base` from tests
In-Reply-To: <mhVD9GRUgDYhNnaeM4_mNSg3jUjqrk9HV48wBI0cO2E=.ce4baf13-aed3-4a44-8482-9b2930bc94d8@github.com>
References: <mhVD9GRUgDYhNnaeM4_mNSg3jUjqrk9HV48wBI0cO2E=.ce4baf13-aed3-4a44-8482-9b2930bc94d8@github.com>
Message-ID: <47c19e0cAuCEtsxE3PtvwXJdmbdvqPSj7oZglqg5c_E=.6d397d21-b62d-4c77-a6fb-7d682b983870@github.com>

On Sat, 13 Mar 2021 20:26:42 GMT, Igor Ignatyev <iignatyev at openjdk.org> wrote:

> Hi all,
> 
> could you please review this trivial cleanup?
> from JBS:
> 
>> jtreg `@modules X` directive does two things: 
>>  - exclude a test from execution if JDK under test doesn't have module X 
>>  - if JDK under test has module X, make sure it's resolved 
>> 
>> both these things have no sense for `java.base` module as it's always available and is always resolved.
> 
> 
> Thanks,
> -- Igor

This pull request has now been integrated.

Changeset: d825198e
Author:    Igor Ignatyev <iignatyev at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/d825198e
Stats:     21 lines in 13 files changed: 0 ins; 13 del; 8 mod

8263556: remove `@modules java.base` from tests

Reviewed-by: dcubed, naoto, iris

-------------

PR: https://git.openjdk.java.net/jdk/pull/2990

From rkennke at openjdk.java.net  Mon Mar 15 18:46:36 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 15 Mar 2021 18:46:36 GMT
Subject: RFR: 8263427: Shenandoah: Trigger weak-LRB even when heap is
 stable [v4]
In-Reply-To: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
References: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
Message-ID: <RNUPE87Zghk7bVG8Wx9D8Zavj50gVAOMPu607jHi8fY=.3df1663a-fe2f-4787-82e3-45d55b64f7d8@github.com>

> We currently guard all LRBs, including weak-LRB, by a test for heap-stable and only enter the LRB when heap is unstable (e.g. evacuation or update-refs in progress). However, the weak LRB must also be entered when heap is stable and concurrent refs is in progress, otherwise we may accidentally resurrect otherwise unreachable weak referents. This can happen when we take the shortcut cycle and skip evac&update-refs.
> 
> I believe this might be the root cause for JDK-8262852. 
> 
> The way out of it is change conc-weakroots-in-progress flag to a bit in gc-state, and test for this in weak-LRB gc-state-check, and enter weak-LRB even when heap is stable, but conc-weakroots-in-progress.
> 
> There's one gotcha here: we used to change gc-state only at safepoints so that the flag can safely be propagated to all Java threads. But conc-weakroots-in-progress is turned-off concurrently. I deal with this by propagating the flag change to Java threads via the rendevouz (that we do anyway), and change the global flag only once all threads got the thread-local flag change.
> 
> This stuff makes the verifier unhappy, because it doesn't know about the new bit. And it'd be difficult to properly verify it, because sometimes it is set (conc-cycle) and sometimes it is not (degen-cycle), so instead of additing extra verification, I figured we could keep ignoring the flag (for now?)
> 
> Testing:
>  - [x] New testcase failed without change, passes now
>  - [x] hotspot_gc_shenandoah
>  - [ ] tier1 (+Shenandoah)

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Disable weakroots together with evacuation at safepoint, and in a separate vmop in shortcut cycles

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2945/files
  - new: https://git.openjdk.java.net/jdk/pull/2945/files/739c9b62..8c103236

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2945&range=03
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2945&range=02-03

  Stats: 70 lines in 7 files changed: 52 ins; 18 del; 0 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2945.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2945/head:pull/2945

PR: https://git.openjdk.java.net/jdk/pull/2945

From shade at openjdk.java.net  Mon Mar 15 18:51:15 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 15 Mar 2021 18:51:15 GMT
Subject: RFR: 8263427: Shenandoah: Trigger weak-LRB even when heap is
 stable [v4]
In-Reply-To: <RNUPE87Zghk7bVG8Wx9D8Zavj50gVAOMPu607jHi8fY=.3df1663a-fe2f-4787-82e3-45d55b64f7d8@github.com>
References: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
 <RNUPE87Zghk7bVG8Wx9D8Zavj50gVAOMPu607jHi8fY=.3df1663a-fe2f-4787-82e3-45d55b64f7d8@github.com>
Message-ID: <iZDivZXeSGOpcjRes9-D_0nZFy1ip4d-0HgzzrPmuWM=.125b1911-71a4-454f-8756-3ec9ce69e0d6@github.com>

On Mon, 15 Mar 2021 18:46:36 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> We currently guard all LRBs, including weak-LRB, by a test for heap-stable and only enter the LRB when heap is unstable (e.g. evacuation or update-refs in progress). However, the weak LRB must also be entered when heap is stable and concurrent refs is in progress, otherwise we may accidentally resurrect otherwise unreachable weak referents. This can happen when we take the shortcut cycle and skip evac&update-refs.
>> 
>> I believe this might be the root cause for JDK-8262852. 
>> 
>> The way out of it is change conc-weakroots-in-progress flag to a bit in gc-state, and test for this in weak-LRB gc-state-check, and enter weak-LRB even when heap is stable, but conc-weakroots-in-progress.
>> 
>> There's one gotcha here: we used to change gc-state only at safepoints so that the flag can safely be propagated to all Java threads. But conc-weakroots-in-progress is turned-off concurrently. I deal with this by propagating the flag change to Java threads via the rendevouz (that we do anyway), and change the global flag only once all threads got the thread-local flag change.
>> 
>> This stuff makes the verifier unhappy, because it doesn't know about the new bit. And it'd be difficult to properly verify it, because sometimes it is set (conc-cycle) and sometimes it is not (degen-cycle), so instead of additing extra verification, I figured we could keep ignoring the flag (for now?)
>> 
>> Testing:
>>  - [x] New testcase failed without change, passes now
>>  - [x] hotspot_gc_shenandoah
>>  - [ ] tier1 (+Shenandoah)
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Disable weakroots together with evacuation at safepoint, and in a separate vmop in shortcut cycles

src/hotspot/share/gc/shenandoah/shenandoahPhaseTimings.hpp line 102:

> 100:                                                                                        \
> 101:   f(disable_weakroots_gross,                        "Disable Weak Roots (G)")          \
> 102:   f(disable_weakroots,                              "Disable Weak Roots (N)")          \

Should start with "Pause". And should probably be something generic, like "Pause Final Roots"? This probably percolates to the names of everything else...

src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 226:

> 224: }
> 225: 
> 226: 

Double new line.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2945

From zgu at openjdk.java.net  Mon Mar 15 19:08:12 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 15 Mar 2021 19:08:12 GMT
Subject: RFR: 8263427: Shenandoah: Trigger weak-LRB even when heap is
 stable [v4]
In-Reply-To: <RNUPE87Zghk7bVG8Wx9D8Zavj50gVAOMPu607jHi8fY=.3df1663a-fe2f-4787-82e3-45d55b64f7d8@github.com>
References: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
 <RNUPE87Zghk7bVG8Wx9D8Zavj50gVAOMPu607jHi8fY=.3df1663a-fe2f-4787-82e3-45d55b64f7d8@github.com>
Message-ID: <ITBHqZqZDY8qRwrdFbYPCiHBekGw5Rog-SB4hJmKX2g=.979a871d-c825-4f09-a1f2-b275eb75c432@github.com>

On Mon, 15 Mar 2021 18:46:36 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> We currently guard all LRBs, including weak-LRB, by a test for heap-stable and only enter the LRB when heap is unstable (e.g. evacuation or update-refs in progress). However, the weak LRB must also be entered when heap is stable and concurrent refs is in progress, otherwise we may accidentally resurrect otherwise unreachable weak referents. This can happen when we take the shortcut cycle and skip evac&update-refs.
>> 
>> I believe this might be the root cause for JDK-8262852. 
>> 
>> The way out of it is change conc-weakroots-in-progress flag to a bit in gc-state, and test for this in weak-LRB gc-state-check, and enter weak-LRB even when heap is stable, but conc-weakroots-in-progress.
>> 
>> There's one gotcha here: we used to change gc-state only at safepoints so that the flag can safely be propagated to all Java threads. But conc-weakroots-in-progress is turned-off concurrently. I deal with this by propagating the flag change to Java threads via the rendevouz (that we do anyway), and change the global flag only once all threads got the thread-local flag change.
>> 
>> This stuff makes the verifier unhappy, because it doesn't know about the new bit. And it'd be difficult to properly verify it, because sometimes it is set (conc-cycle) and sometimes it is not (degen-cycle), so instead of additing extra verification, I figured we could keep ignoring the flag (for now?)
>> 
>> Testing:
>>  - [x] New testcase failed without change, passes now
>>  - [x] hotspot_gc_shenandoah
>>  - [ ] tier1 (+Shenandoah)
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Disable weakroots together with evacuation at safepoint, and in a separate vmop in shortcut cycles

Changes requested by zgu (Reviewer).

src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 909:

> 907: }
> 908: 
> 909: void ShenandoahConcurrentGC::op_rendezvous_roots() {

This method is no longer needed, so as disable_concurrent_weak_root_in_progress_concurrently()

-------------

PR: https://git.openjdk.java.net/jdk/pull/2945

From rkennke at openjdk.java.net  Mon Mar 15 19:26:34 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Mon, 15 Mar 2021 19:26:34 GMT
Subject: RFR: 8263427: Shenandoah: Trigger weak-LRB even when heap is
 stable [v5]
In-Reply-To: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
References: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
Message-ID: <fBxZj4bxapnOJjAginzcFUso8ChdpG-ZSeLs7acwT10=.3daaffac-bd0b-41b1-af87-8511c9a1374a@github.com>

> We currently guard all LRBs, including weak-LRB, by a test for heap-stable and only enter the LRB when heap is unstable (e.g. evacuation or update-refs in progress). However, the weak LRB must also be entered when heap is stable and concurrent refs is in progress, otherwise we may accidentally resurrect otherwise unreachable weak referents. This can happen when we take the shortcut cycle and skip evac&update-refs.
> 
> I believe this might be the root cause for JDK-8262852. 
> 
> The way out of it is change conc-weakroots-in-progress flag to a bit in gc-state, and test for this in weak-LRB gc-state-check, and enter weak-LRB even when heap is stable, but conc-weakroots-in-progress.
> 
> There's one gotcha here: we used to change gc-state only at safepoints so that the flag can safely be propagated to all Java threads. But conc-weakroots-in-progress is turned-off concurrently. I deal with this by propagating the flag change to Java threads via the rendevouz (that we do anyway), and change the global flag only once all threads got the thread-local flag change.
> 
> This stuff makes the verifier unhappy, because it doesn't know about the new bit. And it'd be difficult to properly verify it, because sometimes it is set (conc-cycle) and sometimes it is not (degen-cycle), so instead of additing extra verification, I figured we could keep ignoring the flag (for now?)
> 
> Testing:
>  - [x] New testcase failed without change, passes now
>  - [x] hotspot_gc_shenandoah
>  - [ ] tier1 (+Shenandoah)

Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:

 - Remove unused code. Move ShenandoahRendezvousClosure close to where it's used.
 - Rename 'disable weak roots' -> 'final roots' everywhere

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2945/files
  - new: https://git.openjdk.java.net/jdk/pull/2945/files/8c103236..5abeece7

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2945&range=04
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2945&range=03-04

  Stats: 87 lines in 13 files changed: 7 ins; 57 del; 23 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2945.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2945/head:pull/2945

PR: https://git.openjdk.java.net/jdk/pull/2945

From sjohanss at openjdk.java.net  Mon Mar 15 21:25:10 2021
From: sjohanss at openjdk.java.net (Stefan Johansson)
Date: Mon, 15 Mar 2021 21:25:10 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <jisIvoUcZWmMRMvBSUlmPn3LECqPpbQ3--tcst7iSzg=.f961458b-5e71-4344-b442-42dbff80da83@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
 <nR7IvUeT4SQXZ3JLFAqeebGf_7jIbEgNlscTI2Wre3A=.b2bbdcdb-7e13-40c2-9154-a81c2f19feb0@github.com>
 <P9p-koXqhcZktsnAcU0NGu7Bfsd1PjLOB0QOIzdEo7Y=.423674c1-d518-4776-9dc9-dbcbe7f95337@github.com>
 <jisIvoUcZWmMRMvBSUlmPn3LECqPpbQ3--tcst7iSzg=.f961458b-5e71-4344-b442-42dbff80da83@github.com>
Message-ID: <VhlwpLHpAXcTddxJvnUApHVsubFAbhL7rBl6hUB5PaI=.03b98760-804a-4efd-bcaa-b59e29fb4d09@github.com>

On Mon, 15 Mar 2021 12:24:02 GMT, Stefan Johansson <sjohanss at openjdk.org> wrote:

>> @tschatzl @Hamlin-Li 
>> Would it be ok to set the live estimate to the `used()` value at the end of `G1FullCollector::phase4_do_compaction()` method to have something suboptimal but working and refine in https://github.com/openjdk/jdk/pull/2760 (or a subsequent ticket/PR once both parts are ready)?
>
> Sorry for being a bit late to the party. Looking at the suggested implementation for G1 I see a problem with only updating this after concurrent mark (and the Full GC). Say for example you have a concurrent mark cycle before the heap has expanded a lot and you get a low value stored in `G1CollectedHeap::_live`. Then the heap expands and your application get to a steady state that doesn't require any more marking cycles. In this case the same value will be reported for the entire run.
> 
> For this to work the _live value would have to be updated at every GC, but this is a bit costly. Maybe the first version could just use `used()` for G1. Have you done any tests to see how off this would be compared to the other GCs?

> @kstefanj
> 
> > Then the heap expands and your application get to a steady state that doesn't require any more marking cycles.
> 
> Is there a way to get the liveness info when the heap expands? If not that would mean we had no way to figure out the new live set size and would assume, conservatively, the last known value.
> 
> As I mentioned in the PR description the live size value will be a 'best effort estimate' depending on what can each particular GC provide.

Sure, and this is fair, my concern is just that this 'best effort estimate' for G1 will often be worse than just using `used()`. This is not only a problem for when the heap expands, that was just an example, the live value will become more and more stale the longer an application run without triggering a new concurrent cycle. 

For the liveness value to be useful it would have to be updated at each GC, and we need to investigate further to see how we can do that in a "cheap" way. I would prefer if G1 did just return `used()` in `live()` as a start and we can create a follow-up task to investigate how to best add a better estimate. Do you see any problem with this?

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Tue Mar 16 09:40:11 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Tue, 16 Mar 2021 09:40:11 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <VhlwpLHpAXcTddxJvnUApHVsubFAbhL7rBl6hUB5PaI=.03b98760-804a-4efd-bcaa-b59e29fb4d09@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
 <nR7IvUeT4SQXZ3JLFAqeebGf_7jIbEgNlscTI2Wre3A=.b2bbdcdb-7e13-40c2-9154-a81c2f19feb0@github.com>
 <P9p-koXqhcZktsnAcU0NGu7Bfsd1PjLOB0QOIzdEo7Y=.423674c1-d518-4776-9dc9-dbcbe7f95337@github.com>
 <jisIvoUcZWmMRMvBSUlmPn3LECqPpbQ3--tcst7iSzg=.f961458b-5e71-4344-b442-42dbff80da83@github.com>
 <VhlwpLHpAXcTddxJvnUApHVsubFAbhL7rBl6hUB5PaI=.03b98760-804a-4efd-bcaa-b59e29fb4d09@github.com>
Message-ID: <YW40etRX5UG_or6xj8tebs_N3DsEw6ibXuP-11d-t9U=.4281df01-d0ac-41a5-9c17-f1dc0ff849f9@github.com>

On Mon, 15 Mar 2021 21:22:44 GMT, Stefan Johansson <sjohanss at openjdk.org> wrote:

> For the liveness value to be useful it would have to be updated at each GC, and we need to investigate further to see how we can do that in a "cheap" way. I would prefer if G1 did just return used() in live() as a start and we can create a follow-up task to investigate how to best add a better estimate. Do you see any problem with this?

Actually yes - this event was meant to be a cheap way to see whether the *known* live size is growing (has an upwards trend) which would using the `used()` value make more difficult and unreliable. 

I would prefer keeping the implementation to return the lower bound of the live size as is the case for other GCs as well. May add explanatory comments to the code and the event definition to make this clear.

Also, the `used()` value is already captured in the event so we would have it duplicated.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From shade at openjdk.java.net  Tue Mar 16 09:53:14 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 16 Mar 2021 09:53:14 GMT
Subject: RFR: 8263427: Shenandoah: Trigger weak-LRB even when heap is
 stable [v5]
In-Reply-To: <fBxZj4bxapnOJjAginzcFUso8ChdpG-ZSeLs7acwT10=.3daaffac-bd0b-41b1-af87-8511c9a1374a@github.com>
References: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
 <fBxZj4bxapnOJjAginzcFUso8ChdpG-ZSeLs7acwT10=.3daaffac-bd0b-41b1-af87-8511c9a1374a@github.com>
Message-ID: <-EcS4FJW792kabI0eFg2odUsAs23suq4qfIAWFdmaYQ=.c0574380-200a-4065-8bd2-3e2ffa2f13eb@github.com>

On Mon, 15 Mar 2021 19:26:34 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> We currently guard all LRBs, including weak-LRB, by a test for heap-stable and only enter the LRB when heap is unstable (e.g. evacuation or update-refs in progress). However, the weak LRB must also be entered when heap is stable and concurrent refs is in progress, otherwise we may accidentally resurrect otherwise unreachable weak referents. This can happen when we take the shortcut cycle and skip evac&update-refs.
>> 
>> I believe this might be the root cause for JDK-8262852. 
>> 
>> The way out of it is change conc-weakroots-in-progress flag to a bit in gc-state, and test for this in weak-LRB gc-state-check, and enter weak-LRB even when heap is stable, but conc-weakroots-in-progress.
>> 
>> There's one gotcha here: we used to change gc-state only at safepoints so that the flag can safely be propagated to all Java threads. But conc-weakroots-in-progress is turned-off concurrently. I deal with this by propagating the flag change to Java threads via the rendevouz (that we do anyway), and change the global flag only once all threads got the thread-local flag change.
>> 
>> This stuff makes the verifier unhappy, because it doesn't know about the new bit. And it'd be difficult to properly verify it, because sometimes it is set (conc-cycle) and sometimes it is not (degen-cycle), so instead of additing extra verification, I figured we could keep ignoring the flag (for now?)
>> 
>> Testing:
>>  - [x] New testcase failed without change, passes now
>>  - [x] hotspot_gc_shenandoah
>>  - [ ] tier1 (+Shenandoah)
>
> Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Remove unused code. Move ShenandoahRendezvousClosure close to where it's used.
>  - Rename 'disable weak roots' -> 'final roots' everywhere

This looks good to me, with a minor nit.

src/hotspot/share/gc/shenandoah/shenandoahNMethod.inline.hpp line 31:

> 29: #include "gc/shared/barrierSetNMethod.hpp"
> 30: #include "gc/shenandoah/shenandoahNMethod.hpp"
> 31: #include "gc/shenandoah/shenandoahClosures.inline.hpp"

Is this include necessary? I cannot see why.

-------------

Marked as reviewed by shade (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2945

From shade at openjdk.java.net  Tue Mar 16 09:53:14 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 16 Mar 2021 09:53:14 GMT
Subject: RFR: 8263427: Shenandoah: Trigger weak-LRB even when heap is
 stable [v5]
In-Reply-To: <-EcS4FJW792kabI0eFg2odUsAs23suq4qfIAWFdmaYQ=.c0574380-200a-4065-8bd2-3e2ffa2f13eb@github.com>
References: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
 <fBxZj4bxapnOJjAginzcFUso8ChdpG-ZSeLs7acwT10=.3daaffac-bd0b-41b1-af87-8511c9a1374a@github.com>
 <-EcS4FJW792kabI0eFg2odUsAs23suq4qfIAWFdmaYQ=.c0574380-200a-4065-8bd2-3e2ffa2f13eb@github.com>
Message-ID: <8XxRz-fmzcV1d4HKJYKMBMsw2Cgh0foiGY3E05hfZNA=.eeea1051-976c-4ba4-afbd-f1067b8cd777@github.com>

On Tue, 16 Mar 2021 09:50:32 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
>> 
>>  - Remove unused code. Move ShenandoahRendezvousClosure close to where it's used.
>>  - Rename 'disable weak roots' -> 'final roots' everywhere
>
> This looks good to me, with a minor nit.

Also, pull from master to get Windows builds fixed.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2945

From rkennke at openjdk.java.net  Tue Mar 16 10:57:39 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 16 Mar 2021 10:57:39 GMT
Subject: RFR: 8263427: Shenandoah: Trigger weak-LRB even when heap is
 stable [v6]
In-Reply-To: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
References: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
Message-ID: <6SJB82pxF38mpJu9xMMgZNE00rlpkRE0NeflRjb7aic=.90f927f9-e6b2-40b1-8472-13995c765eb9@github.com>

> We currently guard all LRBs, including weak-LRB, by a test for heap-stable and only enter the LRB when heap is unstable (e.g. evacuation or update-refs in progress). However, the weak LRB must also be entered when heap is stable and concurrent refs is in progress, otherwise we may accidentally resurrect otherwise unreachable weak referents. This can happen when we take the shortcut cycle and skip evac&update-refs.
> 
> I believe this might be the root cause for JDK-8262852. 
> 
> The way out of it is change conc-weakroots-in-progress flag to a bit in gc-state, and test for this in weak-LRB gc-state-check, and enter weak-LRB even when heap is stable, but conc-weakroots-in-progress.
> 
> There's one gotcha here: we used to change gc-state only at safepoints so that the flag can safely be propagated to all Java threads. But conc-weakroots-in-progress is turned-off concurrently. I deal with this by propagating the flag change to Java threads via the rendevouz (that we do anyway), and change the global flag only once all threads got the thread-local flag change.
> 
> This stuff makes the verifier unhappy, because it doesn't know about the new bit. And it'd be difficult to properly verify it, because sometimes it is set (conc-cycle) and sometimes it is not (degen-cycle), so instead of additing extra verification, I figured we could keep ignoring the flag (for now?)
> 
> Testing:
>  - [x] New testcase failed without change, passes now
>  - [x] hotspot_gc_shenandoah
>  - [ ] tier1 (+Shenandoah)

Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision:

 - Merge branch 'master' into JDK-8263427
 - Remove unused code. Move ShenandoahRendezvousClosure close to where it's used.
 - Rename 'disable weak roots' -> 'final roots' everywhere
 - Disable weakroots together with evacuation at safepoint, and in a separate vmop in shortcut cycles
 - Correct order of rendezvous, global- and local-flag updates; cleanup rendezvous
 - Verify correct weakroots-in-progress state (by Aleksey)
 - Ensure test does a complete GC cycle before verification
 - 8263427: Shenandoah: Trigger weak-LRB even when heap is stable

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2945/files
  - new: https://git.openjdk.java.net/jdk/pull/2945/files/5abeece7..bb76b16e

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2945&range=05
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2945&range=04-05

  Stats: 26071 lines in 1258 files changed: 21363 ins; 2179 del; 2529 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2945.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2945/head:pull/2945

PR: https://git.openjdk.java.net/jdk/pull/2945

From shade at openjdk.java.net  Tue Mar 16 10:57:47 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 16 Mar 2021 10:57:47 GMT
Subject: RFR: 8263427: Shenandoah: Trigger weak-LRB even when heap is
 stable [v6]
In-Reply-To: <6SJB82pxF38mpJu9xMMgZNE00rlpkRE0NeflRjb7aic=.90f927f9-e6b2-40b1-8472-13995c765eb9@github.com>
References: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
 <6SJB82pxF38mpJu9xMMgZNE00rlpkRE0NeflRjb7aic=.90f927f9-e6b2-40b1-8472-13995c765eb9@github.com>
Message-ID: <BuB5YW9Ji44nsWnYaQ8lcfeAUVxn1eQbyc4pLYnPAqM=.d8895bae-398d-4125-abce-5a5575ef3aae@github.com>

On Tue, 16 Mar 2021 10:54:49 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> We currently guard all LRBs, including weak-LRB, by a test for heap-stable and only enter the LRB when heap is unstable (e.g. evacuation or update-refs in progress). However, the weak LRB must also be entered when heap is stable and concurrent refs is in progress, otherwise we may accidentally resurrect otherwise unreachable weak referents. This can happen when we take the shortcut cycle and skip evac&update-refs.
>> 
>> I believe this might be the root cause for JDK-8262852. 
>> 
>> The way out of it is change conc-weakroots-in-progress flag to a bit in gc-state, and test for this in weak-LRB gc-state-check, and enter weak-LRB even when heap is stable, but conc-weakroots-in-progress.
>> 
>> There's one gotcha here: we used to change gc-state only at safepoints so that the flag can safely be propagated to all Java threads. But conc-weakroots-in-progress is turned-off concurrently. I deal with this by propagating the flag change to Java threads via the rendevouz (that we do anyway), and change the global flag only once all threads got the thread-local flag change.
>> 
>> This stuff makes the verifier unhappy, because it doesn't know about the new bit. And it'd be difficult to properly verify it, because sometimes it is set (conc-cycle) and sometimes it is not (degen-cycle), so instead of additing extra verification, I figured we could keep ignoring the flag (for now?)
>> 
>> Testing:
>>  - [x] New testcase failed without change, passes now
>>  - [x] hotspot_gc_shenandoah
>>  - [ ] tier1 (+Shenandoah)
>
> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision:
> 
>  - Merge branch 'master' into JDK-8263427
>  - Remove unused code. Move ShenandoahRendezvousClosure close to where it's used.
>  - Rename 'disable weak roots' -> 'final roots' everywhere
>  - Disable weakroots together with evacuation at safepoint, and in a separate vmop in shortcut cycles
>  - Correct order of rendezvous, global- and local-flag updates; cleanup rendezvous
>  - Verify correct weakroots-in-progress state (by Aleksey)
>  - Ensure test does a complete GC cycle before verification
>  - 8263427: Shenandoah: Trigger weak-LRB even when heap is stable

Marked as reviewed by shade (Reviewer).

-------------

PR: https://git.openjdk.java.net/jdk/pull/2945

From shade at openjdk.java.net  Tue Mar 16 10:57:49 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 16 Mar 2021 10:57:49 GMT
Subject: RFR: 8263427: Shenandoah: Trigger weak-LRB even when heap is
 stable [v5]
In-Reply-To: <WABBNPKIMLZZcjzQ_wYpmuXvTdVeEVgz-5gzcFlqFFU=.0f6e48b8-a552-46b7-bc45-edd0a70826eb@github.com>
References: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
 <fBxZj4bxapnOJjAginzcFUso8ChdpG-ZSeLs7acwT10=.3daaffac-bd0b-41b1-af87-8511c9a1374a@github.com>
 <-EcS4FJW792kabI0eFg2odUsAs23suq4qfIAWFdmaYQ=.c0574380-200a-4065-8bd2-3e2ffa2f13eb@github.com>
 <WABBNPKIMLZZcjzQ_wYpmuXvTdVeEVgz-5gzcFlqFFU=.0f6e48b8-a552-46b7-bc45-edd0a70826eb@github.com>
Message-ID: <mTtOPu5YMW5_Skjrc5AX6Uue2GMNKlEH7un6HPWVytA=.2c4f0913-45c8-4b65-b687-1ef17ad8cb41@github.com>

On Tue, 16 Mar 2021 10:52:56 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahNMethod.inline.hpp line 31:
>> 
>>> 29: #include "gc/shared/barrierSetNMethod.hpp"
>>> 30: #include "gc/shenandoah/shenandoahNMethod.hpp"
>>> 31: #include "gc/shenandoah/shenandoahClosures.inline.hpp"
>> 
>> Is this include necessary? I cannot see why.
>
> I removed the same include in shenandoahUnload.cpp because it is no longer needed there. But it's needed here because of ShenandoahEvacuateUpdateMetadataClosure. It was only reachable through the include in shenandoahUnload.cpp before.

Ah. Makes sense then.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2945

From rkennke at openjdk.java.net  Tue Mar 16 10:57:47 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 16 Mar 2021 10:57:47 GMT
Subject: RFR: 8263427: Shenandoah: Trigger weak-LRB even when heap is
 stable [v5]
In-Reply-To: <-EcS4FJW792kabI0eFg2odUsAs23suq4qfIAWFdmaYQ=.c0574380-200a-4065-8bd2-3e2ffa2f13eb@github.com>
References: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
 <fBxZj4bxapnOJjAginzcFUso8ChdpG-ZSeLs7acwT10=.3daaffac-bd0b-41b1-af87-8511c9a1374a@github.com>
 <-EcS4FJW792kabI0eFg2odUsAs23suq4qfIAWFdmaYQ=.c0574380-200a-4065-8bd2-3e2ffa2f13eb@github.com>
Message-ID: <WABBNPKIMLZZcjzQ_wYpmuXvTdVeEVgz-5gzcFlqFFU=.0f6e48b8-a552-46b7-bc45-edd0a70826eb@github.com>

On Tue, 16 Mar 2021 09:50:16 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
>> 
>>  - Remove unused code. Move ShenandoahRendezvousClosure close to where it's used.
>>  - Rename 'disable weak roots' -> 'final roots' everywhere
>
> src/hotspot/share/gc/shenandoah/shenandoahNMethod.inline.hpp line 31:
> 
>> 29: #include "gc/shared/barrierSetNMethod.hpp"
>> 30: #include "gc/shenandoah/shenandoahNMethod.hpp"
>> 31: #include "gc/shenandoah/shenandoahClosures.inline.hpp"
> 
> Is this include necessary? I cannot see why.

I removed the same include in shenandoahUnload.cpp because it is no longer needed there. But it's needed here because of ShenandoahEvacuateUpdateMetadataClosure. It was only reachable through the include in shenandoahUnload.cpp before.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2945

From sjohanss at openjdk.java.net  Tue Mar 16 10:50:09 2021
From: sjohanss at openjdk.java.net (Stefan Johansson)
Date: Tue, 16 Mar 2021 10:50:09 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <YW40etRX5UG_or6xj8tebs_N3DsEw6ibXuP-11d-t9U=.4281df01-d0ac-41a5-9c17-f1dc0ff849f9@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
 <nR7IvUeT4SQXZ3JLFAqeebGf_7jIbEgNlscTI2Wre3A=.b2bbdcdb-7e13-40c2-9154-a81c2f19feb0@github.com>
 <P9p-koXqhcZktsnAcU0NGu7Bfsd1PjLOB0QOIzdEo7Y=.423674c1-d518-4776-9dc9-dbcbe7f95337@github.com>
 <jisIvoUcZWmMRMvBSUlmPn3LECqPpbQ3--tcst7iSzg=.f961458b-5e71-4344-b442-42dbff80da83@github.com>
 <VhlwpLHpAXcTddxJvnUApHVsubFAbhL7rBl6hUB5PaI=.03b98760-804a-4efd-bcaa-b59e29fb4d09@github.com>
 <YW40etRX5UG_or6xj8tebs_N3DsEw6ibXuP-11d-t9U=.4281df01-d0ac-41a5-9c17-f1dc0ff849f9@github.com>
Message-ID: <ACzsS56fFDL2OU-KQzLr0aCA5iFkjt97aQFtX2hBrR4=.9285865f-b780-4a0d-bfcd-50205ffbb7a3@github.com>

On Tue, 16 Mar 2021 09:37:08 GMT, Jaroslav Bachorik <jbachorik at openjdk.org> wrote:

>>> @kstefanj
>>> 
>>> > Then the heap expands and your application get to a steady state that doesn't require any more marking cycles.
>>> 
>>> Is there a way to get the liveness info when the heap expands? If not that would mean we had no way to figure out the new live set size and would assume, conservatively, the last known value.
>>> 
>>> As I mentioned in the PR description the live size value will be a 'best effort estimate' depending on what can each particular GC provide.
>> 
>> Sure, and this is fair, my concern is just that this 'best effort estimate' for G1 will often be worse than just using `used()`. This is not only a problem for when the heap expands, that was just an example, the live value will become more and more stale the longer an application run without triggering a new concurrent cycle. 
>> 
>> For the liveness value to be useful it would have to be updated at each GC, and we need to investigate further to see how we can do that in a "cheap" way. I would prefer if G1 did just return `used()` in `live()` as a start and we can create a follow-up task to investigate how to best add a better estimate. Do you see any problem with this?
>
>> For the liveness value to be useful it would have to be updated at each GC, and we need to investigate further to see how we can do that in a "cheap" way. I would prefer if G1 did just return used() in live() as a start and we can create a follow-up task to investigate how to best add a better estimate. Do you see any problem with this?
> 
> Actually yes - this event was meant to be a cheap way to see whether the *known* live size is growing (has an upwards trend) which would using the `used()` value make more difficult and unreliable. 
> 
> I would prefer keeping the implementation to return the lower bound of the live size as is the case for other GCs as well. May add explanatory comments to the code and the event definition to make this clear.
> 
> Also, the `used()` value is already captured in the event so we would have it duplicated.

> > For the liveness value to be useful it would have to be updated at each GC, and we need to investigate further to see how we can do that in a "cheap" way. I would prefer if G1 did just return used() in live() as a start and we can create a follow-up task to investigate how to best add a better estimate. Do you see any problem with this?
> 
> Actually yes - this event was meant to be a cheap way to see whether the _known_ live size is growing (has an upwards trend) which would using the `used()` value make more difficult and unreliable.
> 
> I would prefer keeping the implementation to return the lower bound of the live size as is the case for other GCs as well. May add explanatory comments to the code and the event definition to make this clear.
> 
> Also, the `used()` value is already captured in the event so we would have it duplicated.

Sure, but for the event to be useful we want the reported value to be as close to the reality as possible. I don't understand why you want the lower bound, can you explain why? I would go for the upper bound, which in that case would be `used()` at the end of the GC. I know `used()` is not perfect, but for G1 this is the best "cheap" value we have for liveness at the end of any GC. 

So as a middle road I would suggest to update `G1CollectedHeap::gc_epilogue(bool full)` to include:
set_live(used());

With this you don't need the changes for the `G1FullCollector`. The liveness calculated at Remark would be used until the next young collection and I think here is where some improvements could be made. During the mixed phase a better solution would make use of the liveness information we have for the old regions together with what is newly allocated, but this needs further investigation.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Tue Mar 16 11:09:09 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Tue, 16 Mar 2021 11:09:09 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <ACzsS56fFDL2OU-KQzLr0aCA5iFkjt97aQFtX2hBrR4=.9285865f-b780-4a0d-bfcd-50205ffbb7a3@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
 <nR7IvUeT4SQXZ3JLFAqeebGf_7jIbEgNlscTI2Wre3A=.b2bbdcdb-7e13-40c2-9154-a81c2f19feb0@github.com>
 <P9p-koXqhcZktsnAcU0NGu7Bfsd1PjLOB0QOIzdEo7Y=.423674c1-d518-4776-9dc9-dbcbe7f95337@github.com>
 <jisIvoUcZWmMRMvBSUlmPn3LECqPpbQ3--tcst7iSzg=.f961458b-5e71-4344-b442-42dbff80da83@github.com>
 <VhlwpLHpAXcTddxJvnUApHVsubFAbhL7rBl6hUB5PaI=.03b98760-804a-4efd-bcaa-b59e29fb4d09@github.com>
 <YW40etRX5UG_or6xj8tebs_N3DsEw6ibXuP-11d-t9U=.4281df01-d0ac-41a5-9c17-f1dc0ff849f9@github.com>
 <ACzsS56fFDL2OU-KQzLr0aCA5iFkjt97aQFtX2hBrR4=.9285865f-b780-4a0d-bfcd-50205ffbb7a3@github.com>
Message-ID: <PfdQDu2_De1mezV67LsCvlNZlyTQAK7iEp06DSpON5c=.e6b3d0d5-0fd6-4c51-89d7-008129cfadb3@github.com>

On Tue, 16 Mar 2021 10:47:22 GMT, Stefan Johansson <sjohanss at openjdk.org> wrote:

> Sure, but for the event to be useful we want the reported value to be as close to the reality as possible. I don't understand why you want the lower bound, can you explain why? I would go for the upper bound, which in that case would be used() at the end of the GC. I know used() is not perfect, but for G1 this is the best "cheap" value we have for liveness at the end of any GC.

Mostly because `used()` will report all live instances and potential garbage and will make it inconsistent with what the other GCs would report. 

> So as a middle road I would suggest to update G1CollectedHeap::gc_epilogue(bool full) to include:
>
> set_live(used());
With this you don't need the changes for the G1FullCollector. The liveness calculated at Remark would be used until the next young collection and I think here is where some improvements could be made. During the mixed phase a better solution would make use of the liveness information we have for the old regions together with what is newly allocated, but this needs further investigation.

This sounds interesting. Let me try this out.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From sjohanss at openjdk.java.net  Tue Mar 16 11:32:14 2021
From: sjohanss at openjdk.java.net (Stefan Johansson)
Date: Tue, 16 Mar 2021 11:32:14 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <PfdQDu2_De1mezV67LsCvlNZlyTQAK7iEp06DSpON5c=.e6b3d0d5-0fd6-4c51-89d7-008129cfadb3@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
 <nR7IvUeT4SQXZ3JLFAqeebGf_7jIbEgNlscTI2Wre3A=.b2bbdcdb-7e13-40c2-9154-a81c2f19feb0@github.com>
 <P9p-koXqhcZktsnAcU0NGu7Bfsd1PjLOB0QOIzdEo7Y=.423674c1-d518-4776-9dc9-dbcbe7f95337@github.com>
 <jisIvoUcZWmMRMvBSUlmPn3LECqPpbQ3--tcst7iSzg=.f961458b-5e71-4344-b442-42dbff80da83@github.com>
 <VhlwpLHpAXcTddxJvnUApHVsubFAbhL7rBl6hUB5PaI=.03b98760-804a-4efd-bcaa-b59e29fb4d09@github.com>
 <YW40etRX5UG_or6xj8tebs_N3DsEw6ibXuP-11d-t9U=.4281df01-d0ac-41a5-9c17-f1dc0ff849f9@github.com>
 <ACzsS56fFDL2OU-KQzLr0aCA5iFkjt97aQFtX2hBrR4=.9285865f-b780-4a0d-bfcd-50205ffbb7a3@github.com>
 <PfdQDu2_De1mezV67LsCvlNZlyTQAK7iEp06DSpON5c=.e6b3d0d5-0fd6-4c51-89d7-008129cfadb3@github.com>
Message-ID: <8kTv1ENK1FbxIt7NVkNJNgT55bLw5ao4lWi8uq0nktQ=.a969127a-351a-4393-a932-934fbe9b6924@github.com>

On Tue, 16 Mar 2021 11:06:22 GMT, Jaroslav Bachorik <jbachorik at openjdk.org> wrote:

> > Sure, but for the event to be useful we want the reported value to be as close to the reality as possible. I don't understand why you want the lower bound, can you explain why? I would go for the upper bound, which in that case would be used() at the end of the GC. I know used() is not perfect, but for G1 this is the best "cheap" value we have for liveness at the end of any GC.
> 
> Mostly because `used()` will report all live instances and potential garbage and will make it inconsistent with what the other GCs would report.
> 
The other STW GCs do report the same, right? 

> > So as a middle road I would suggest to update G1CollectedHeap::gc_epilogue(bool full) to include:
> > set_live(used());
> > With this you don't need the changes for the G1FullCollector. The liveness calculated at Remark would be used until the next young collection and I think here is where some improvements could be made. During the mixed phase a better solution would make use of the liveness information we have for the old regions together with what is newly allocated, but this needs further investigation.
> 
> This sounds interesting. Let me try this out.

Glad you like the idea :) I did a quick test locally and it shows the trend ok, even if it is an over estimate of live: 
  live = 1.1 GB
  live = 1.2 GB
  live = 1.5 GB
  live = 1.7 GB
  live = 2.1 GB
  name = "G1Old"
  live = 1.4 GB
  live = 1.6 GB
  live = 1.8 GB
  live = 2.0 GB
  live = 2.3 GB
  live = 2.5 GB
  live = 2.8 GB
  live = 3.1 GB
  live = 3.3 GB
  live = 3.7 GB
  live = 4.0 GB
  live = 4.3 GB
  name = "G1Old"
  live = 1.2 GB
G1Old is from concurrent mark events.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Tue Mar 16 11:41:12 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Tue, 16 Mar 2021 11:41:12 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <8kTv1ENK1FbxIt7NVkNJNgT55bLw5ao4lWi8uq0nktQ=.a969127a-351a-4393-a932-934fbe9b6924@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
 <nR7IvUeT4SQXZ3JLFAqeebGf_7jIbEgNlscTI2Wre3A=.b2bbdcdb-7e13-40c2-9154-a81c2f19feb0@github.com>
 <P9p-koXqhcZktsnAcU0NGu7Bfsd1PjLOB0QOIzdEo7Y=.423674c1-d518-4776-9dc9-dbcbe7f95337@github.com>
 <jisIvoUcZWmMRMvBSUlmPn3LECqPpbQ3--tcst7iSzg=.f961458b-5e71-4344-b442-42dbff80da83@github.com>
 <VhlwpLHpAXcTddxJvnUApHVsubFAbhL7rBl6hUB5PaI=.03b98760-804a-4efd-bcaa-b59e29fb4d09@github.com>
 <YW40etRX5UG_or6xj8tebs_N3DsEw6ibXuP-11d-t9U=.4281df01-d0ac-41a5-9c17-f1dc0ff849f9@github.com>
 <ACzsS56fFDL2OU-KQzLr0aCA5iFkjt97aQFtX2hBrR4=.9285865f-b780-4a0d-bfcd-50205ffbb7a3@github.com>
 <PfdQDu2_De1mezV67LsCvlNZlyTQAK7iEp06DSpON5c=.e6b3d0d5-0fd6-4c51-89d7-008129cfadb3@github.com>
 <8kTv1ENK1FbxIt7NVkNJNgT55bLw5ao4lWi
 8uq0nktQ=.a969127a-351a-4393-a932-934fbe9b6924@github.com>
Message-ID: <ueWRKfAX82wsJ4R9UyRoAOBD-H2Eoho-YSDAan6eCRw=.3d9d9d32-4675-43db-8d01-eb25f0b8c770@github.com>

On Tue, 16 Mar 2021 11:28:50 GMT, Stefan Johansson <sjohanss at openjdk.org> wrote:

> The other STW GCs do report the same, right?

They report the lower bound - basically the real live size right at the end of a GC cycle (either gathered during marking or the used size after compaction). Of course, a few moments later it might (and probably will) not be 100% correct but that's why it is just an estimate.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From tschatzl at openjdk.java.net  Tue Mar 16 11:54:09 2021
From: tschatzl at openjdk.java.net (Thomas Schatzl)
Date: Tue, 16 Mar 2021 11:54:09 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <ACzsS56fFDL2OU-KQzLr0aCA5iFkjt97aQFtX2hBrR4=.9285865f-b780-4a0d-bfcd-50205ffbb7a3@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
 <nR7IvUeT4SQXZ3JLFAqeebGf_7jIbEgNlscTI2Wre3A=.b2bbdcdb-7e13-40c2-9154-a81c2f19feb0@github.com>
 <P9p-koXqhcZktsnAcU0NGu7Bfsd1PjLOB0QOIzdEo7Y=.423674c1-d518-4776-9dc9-dbcbe7f95337@github.com>
 <jisIvoUcZWmMRMvBSUlmPn3LECqPpbQ3--tcst7iSzg=.f961458b-5e71-4344-b442-42dbff80da83@github.com>
 <VhlwpLHpAXcTddxJvnUApHVsubFAbhL7rBl6hUB5PaI=.03b98760-804a-4efd-bcaa-b59e29fb4d09@github.com>
 <YW40etRX5UG_or6xj8tebs_N3DsEw6ibXuP-11d-t9U=.4281df01-d0ac-41a5-9c17-f1dc0ff849f9@github.com>
 <ACzsS56fFDL2OU-KQzLr0aCA5iFkjt97aQFtX2hBrR4=.9285865f-b780-4a0d-bfcd-50205ffbb7a3@github.com>
Message-ID: <_XwpqPW-sKwuvZG26bSgdW6RKtQaIJz-0fpRR5wGa0c=.d6c5394b-7fea-4f0e-ab5f-9092286d3db1@github.com>

On Tue, 16 Mar 2021 10:47:22 GMT, Stefan Johansson <sjohanss at openjdk.org> wrote:

> > For the liveness value to be useful it would have to be updated at each GC, and we need to investigate further to see how we can do that in a "cheap" way. I would prefer if G1 did just return used() in live() as a start and we can create a follow-up task to investigate how to best add a better estimate. Do you see any problem with this?
> 
> Actually yes - this event was meant to be a cheap way to see whether the _known_ live size is growing (has an upwards trend) which would using the `used()` value make more difficult and unreliable.

So one of the actual purposes seems to be some kind of leak detection: there is this JFR leak detector (I only know the feature name, not completely how it works and what its overhead is) for this purpose, wouldn't that work?

Also, for this purpose, why would used() not be a good substitute for liveness? If e.g. used() average grows over time you can deduce the same I would assume (particularly used() after mixed gc phase in g1).

Do you have any numbers on what the impact of using used() vs. this live() would be in such a use case?

What I'm afraid of is that mixing values taken at different times - used and capacity are taken at the time of the event, and the liveness estimated updated at other, irregular intervals may cause significiant amount of confusion in interpreting this value. It might be obvious to you, but there will be other users.

One option could be detaching the liveness estimate from used()/capacity() (I see a value in having some heap usage summary at regular intervals) and send the liveness estimate event just when they are generated? Then the various collectors could send this liveness value at times when they think they are fairly accurate, not when the collectors must and particularly not in conjunction with samples taken at completely different times.

Independent of whether used/capacity and liveness are sent, the receiver needs to do statistics (trend lines) on those anyway.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From github.com+71722661+earthling-amzn at openjdk.java.net  Tue Mar 16 12:06:28 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Tue, 16 Mar 2021 12:06:28 GMT
Subject: Integrated: Update periodic gc test for changed heuristic log message
In-Reply-To: <2VCkfRyfAYiBzdnyaHJHiKoR_Gc05h2XFMLu1ZhbHew=.4b099ff1-9096-439e-bf2e-f10b458fe300@github.com>
References: <2VCkfRyfAYiBzdnyaHJHiKoR_Gc05h2XFMLu1ZhbHew=.4b099ff1-9096-439e-bf2e-f10b458fe300@github.com>
Message-ID: <X_Gziz6GjKhzxyf8Sz6UWPUFtgTronZC1U283LMCVjU=.1c11bad6-77fa-4d9e-b1bf-6a438af539e6@github.com>

On Thu, 11 Mar 2021 23:53:58 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:

> This also includes a change to hold down regulator requests when the GC is already running in non-generational modes.

This pull request has now been integrated.

Changeset: b5575d8e
Author:    William Kemper <kemperw at amazon.com>
Committer: Roman Kennke <rkennke at openjdk.org>
URL:       https://git.openjdk.java.net/shenandoah/commit/b5575d8e
Stats:     12 lines in 2 files changed: 4 ins; 0 del; 8 mod

Update periodic gc test for changed heuristic log message

Reviewed-by: rkennke

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/21

From bmathiske at openjdk.java.net  Tue Mar 16 12:07:31 2021
From: bmathiske at openjdk.java.net (Bernd Mathiske)
Date: Tue, 16 Mar 2021 12:07:31 GMT
Subject: RFR: Only scan dirty cards, not entire regions,
 in update refs remset scan.
Message-ID: <N9hypfdi6_PMANGXV0duSNnaauuRp4wDrXvfEzS1j60=.a466c8a0-6e04-4477-8cca-0f3a60ba6518@github.com>

So far in update refs, when iterating over regions to process the remset, we scan an entire region for references into young gen if we find any dirty card in it. With this update, we only process dirty cards in each region and leave out clean ones. To do so, we refactor the scan code from the marking phase into a method that is then reused for update refs, passing down a different closure. The JVM switch `ShenandoahUseSimpleCardScanning` provides a means to fall back on the previous code.

(In the future, we may want to replace region stealing with card stealing.)

-------------

Commit messages:
 - Only scan dirty cards, not entire regions, in remembered set scanning in update refs.

Changes: https://git.openjdk.java.net/shenandoah/pull/22/files
 Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=22&range=00
  Stats: 78 lines in 4 files changed: 30 ins; 26 del; 22 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/22.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/22/head:pull/22

PR: https://git.openjdk.java.net/shenandoah/pull/22

From rkennke at openjdk.java.net  Tue Mar 16 12:07:31 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 16 Mar 2021 12:07:31 GMT
Subject: RFR: Only scan dirty cards, not entire regions,
 in update refs remset scan.
In-Reply-To: <N9hypfdi6_PMANGXV0duSNnaauuRp4wDrXvfEzS1j60=.a466c8a0-6e04-4477-8cca-0f3a60ba6518@github.com>
References: <N9hypfdi6_PMANGXV0duSNnaauuRp4wDrXvfEzS1j60=.a466c8a0-6e04-4477-8cca-0f3a60ba6518@github.com>
Message-ID: <oU0Ab7cwB0OOKsGxtefrwnH-rKFwHqWyZeDWUfvvuEs=.b14cec10-255e-4ca9-9333-a23fca49191a@github.com>

On Fri, 12 Mar 2021 21:11:45 GMT, Bernd Mathiske <bmathiske at openjdk.org> wrote:

> So far in update refs, when iterating over regions to process the remset, we scan an entire region for references into young gen if we find any dirty card in it. With this update, we only process dirty cards in each region and leave out clean ones. To do so, we refactor the scan code from the marking phase into a method that is then reused for update refs, passing down a different closure. The JVM switch `ShenandoahUseSimpleCardScanning` provides a means to fall back on the previous code.
> 
> (In the future, we may want to replace region stealing with card stealing.)

Looks good! Thanks!

-------------

Marked as reviewed by rkennke (Reviewer).

PR: https://git.openjdk.java.net/shenandoah/pull/22

From zgu at openjdk.java.net  Tue Mar 16 12:23:17 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 16 Mar 2021 12:23:17 GMT
Subject: RFR: 8263427: Shenandoah: Trigger weak-LRB even when heap is
 stable [v6]
In-Reply-To: <6SJB82pxF38mpJu9xMMgZNE00rlpkRE0NeflRjb7aic=.90f927f9-e6b2-40b1-8472-13995c765eb9@github.com>
References: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
 <6SJB82pxF38mpJu9xMMgZNE00rlpkRE0NeflRjb7aic=.90f927f9-e6b2-40b1-8472-13995c765eb9@github.com>
Message-ID: <YunpcOEUYr6Sx6w7Pmhhk5tibYbqND_4LdTM2pmCpKU=.53926c67-13b5-4c66-aa27-bfd68141d30e@github.com>

On Tue, 16 Mar 2021 10:57:39 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> We currently guard all LRBs, including weak-LRB, by a test for heap-stable and only enter the LRB when heap is unstable (e.g. evacuation or update-refs in progress). However, the weak LRB must also be entered when heap is stable and concurrent refs is in progress, otherwise we may accidentally resurrect otherwise unreachable weak referents. This can happen when we take the shortcut cycle and skip evac&update-refs.
>> 
>> I believe this might be the root cause for JDK-8262852. 
>> 
>> The way out of it is change conc-weakroots-in-progress flag to a bit in gc-state, and test for this in weak-LRB gc-state-check, and enter weak-LRB even when heap is stable, but conc-weakroots-in-progress.
>> 
>> There's one gotcha here: we used to change gc-state only at safepoints so that the flag can safely be propagated to all Java threads. But conc-weakroots-in-progress is turned-off concurrently. I deal with this by propagating the flag change to Java threads via the rendevouz (that we do anyway), and change the global flag only once all threads got the thread-local flag change.
>> 
>> This stuff makes the verifier unhappy, because it doesn't know about the new bit. And it'd be difficult to properly verify it, because sometimes it is set (conc-cycle) and sometimes it is not (degen-cycle), so instead of additing extra verification, I figured we could keep ignoring the flag (for now?)
>> 
>> Testing:
>>  - [x] New testcase failed without change, passes now
>>  - [x] hotspot_gc_shenandoah
>>  - [ ] tier1 (+Shenandoah)
>
> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision:
> 
>  - Merge branch 'master' into JDK-8263427
>  - Remove unused code. Move ShenandoahRendezvousClosure close to where it's used.
>  - Rename 'disable weak roots' -> 'final roots' everywhere
>  - Disable weakroots together with evacuation at safepoint, and in a separate vmop in shortcut cycles
>  - Correct order of rendezvous, global- and local-flag updates; cleanup rendezvous
>  - Verify correct weakroots-in-progress state (by Aleksey)
>  - Ensure test does a complete GC cycle before verification
>  - 8263427: Shenandoah: Trigger weak-LRB even when heap is stable

Looks good.

-------------

Marked as reviewed by zgu (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2945

From jbachorik at openjdk.java.net  Tue Mar 16 12:26:14 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Tue, 16 Mar 2021 12:26:14 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <_XwpqPW-sKwuvZG26bSgdW6RKtQaIJz-0fpRR5wGa0c=.d6c5394b-7fea-4f0e-ab5f-9092286d3db1@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
 <nR7IvUeT4SQXZ3JLFAqeebGf_7jIbEgNlscTI2Wre3A=.b2bbdcdb-7e13-40c2-9154-a81c2f19feb0@github.com>
 <P9p-koXqhcZktsnAcU0NGu7Bfsd1PjLOB0QOIzdEo7Y=.423674c1-d518-4776-9dc9-dbcbe7f95337@github.com>
 <jisIvoUcZWmMRMvBSUlmPn3LECqPpbQ3--tcst7iSzg=.f961458b-5e71-4344-b442-42dbff80da83@github.com>
 <VhlwpLHpAXcTddxJvnUApHVsubFAbhL7rBl6hUB5PaI=.03b98760-804a-4efd-bcaa-b59e29fb4d09@github.com>
 <YW40etRX5UG_or6xj8tebs_N3DsEw6ibXuP-11d-t9U=.4281df01-d0ac-41a5-9c17-f1dc0ff849f9@github.com>
 <ACzsS56fFDL2OU-KQzLr0aCA5iFkjt97aQFtX2hBrR4=.9285865f-b780-4a0d-bfcd-50205ffbb7a3@github.com>
 <_XwpqPW-sKwuvZG26bSgdW6RKtQaIJz-0fpRR5wGa0c=.d6c5394b-7fea-4f0e-ab5f-9092286d3db1@github.com>
Message-ID: <XhTy_KX3x9O6YjWfh38Z7tlsGXKvoATf_abq4tzBmAE=.f3a957d1-6931-45c1-8390-8d5333dc5921@github.com>

On Tue, 16 Mar 2021 11:51:45 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

> So one of the actual purposes seems to be some kind of leak detection: there is this JFR leak detector (I only know the feature name, not completely how it works and what its overhead is) for this purpose, wouldn't that work?

Yes. But enabling that comes with an extra price so it is more of a focused tool than something you could use in continuous monitoring/signal evaluation.

> Also, for this purpose, why would used() not be a good substitute for liveness? If e.g. used() average grows over time you can deduce the same I would assume (particularly used() after mixed gc phase in g1).

The major problem is that eg. for g1 given large enough heap the used value can keep on growing for quite long time, possibly generating wrong signal about potential memory leak. 

If the live estimate is set to `used()` after mixed gc phase in g1 I think it still will be a good estimate. 
The only thing I am opposing is having `live()` call return the current `used()` value which, IMO, might become rather confusing.

> Do you have any numbers on what the impact of using used() vs. this live() would be in such a use case?

Nope. Do you mean perf impact?

> What I'm afraid of is that mixing values taken at different times - used and capacity are taken at the time of the event, and the liveness estimated updated at other, irregular intervals may cause significiant amount of confusion in interpreting this value. It might be obvious to you, but there will be other users.

IDK. If the event field would explicitly mention that this is the **last known live size estimate** it should set the expectations right.

>
>One option could be detaching the liveness estimate from used()/capacity() (I see a value in having some heap usage summary at regular intervals) and send the liveness estimate event just when they are generated? Then the various collectors could send this liveness value at times when they think they are fairly accurate, not when the collectors must and particularly not in conjunction with samples taken at completely different times.

The problem is the irregularity - when the live size is reported only when it is calculated there might be long periods in the recording missing the live size data at all. In order for this information to be useful it should be reported at least at the beginning and end of a JFR chunk.

> Independent of whether used/capacity and liveness are sent, the receiver needs to do statistics (trend lines) on those anyway.

Yes. It's just that with the live size estimate one wouldn't be getting the false positives one would get with used heap trend.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Tue Mar 16 12:32:12 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Tue, 16 Mar 2021 12:32:12 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <8kTv1ENK1FbxIt7NVkNJNgT55bLw5ao4lWi8uq0nktQ=.a969127a-351a-4393-a932-934fbe9b6924@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
 <nR7IvUeT4SQXZ3JLFAqeebGf_7jIbEgNlscTI2Wre3A=.b2bbdcdb-7e13-40c2-9154-a81c2f19feb0@github.com>
 <P9p-koXqhcZktsnAcU0NGu7Bfsd1PjLOB0QOIzdEo7Y=.423674c1-d518-4776-9dc9-dbcbe7f95337@github.com>
 <jisIvoUcZWmMRMvBSUlmPn3LECqPpbQ3--tcst7iSzg=.f961458b-5e71-4344-b442-42dbff80da83@github.com>
 <VhlwpLHpAXcTddxJvnUApHVsubFAbhL7rBl6hUB5PaI=.03b98760-804a-4efd-bcaa-b59e29fb4d09@github.com>
 <YW40etRX5UG_or6xj8tebs_N3DsEw6ibXuP-11d-t9U=.4281df01-d0ac-41a5-9c17-f1dc0ff849f9@github.com>
 <ACzsS56fFDL2OU-KQzLr0aCA5iFkjt97aQFtX2hBrR4=.9285865f-b780-4a0d-bfcd-50205ffbb7a3@github.com>
 <PfdQDu2_De1mezV67LsCvlNZlyTQAK7iEp06DSpON5c=.e6b3d0d5-0fd6-4c51-89d7-008129cfadb3@github.com>
 <8kTv1ENK1FbxIt7NVkNJNgT55bLw5ao4lWi
 8uq0nktQ=.a969127a-351a-4393-a932-934fbe9b6924@github.com>
Message-ID: <HDvYpjJDgYo4sSzo8HlCgxpIQT0QBAkesgStDYDiDqs=.bdf6dbe3-57e8-49f1-813b-6e76a56e7136@github.com>

On Tue, 16 Mar 2021 11:28:50 GMT, Stefan Johansson <sjohanss at openjdk.org> wrote:

>>> Sure, but for the event to be useful we want the reported value to be as close to the reality as possible. I don't understand why you want the lower bound, can you explain why? I would go for the upper bound, which in that case would be used() at the end of the GC. I know used() is not perfect, but for G1 this is the best "cheap" value we have for liveness at the end of any GC.
>> 
>> Mostly because `used()` will report all live instances and potential garbage and will make it inconsistent with what the other GCs would report. 
>> 
>>> So as a middle road I would suggest to update G1CollectedHeap::gc_epilogue(bool full) to include:
>>>
>>> set_live(used());
>> With this you don't need the changes for the G1FullCollector. The liveness calculated at Remark would be used until the next young collection and I think here is where some improvements could be made. During the mixed phase a better solution would make use of the liveness information we have for the old regions together with what is newly allocated, but this needs further investigation.
>> 
>> This sounds interesting. Let me try this out.
>
>> > Sure, but for the event to be useful we want the reported value to be as close to the reality as possible. I don't understand why you want the lower bound, can you explain why? I would go for the upper bound, which in that case would be used() at the end of the GC. I know used() is not perfect, but for G1 this is the best "cheap" value we have for liveness at the end of any GC.
>> 
>> Mostly because `used()` will report all live instances and potential garbage and will make it inconsistent with what the other GCs would report.
>> 
> The other STW GCs do report the same, right? 
> 
>> > So as a middle road I would suggest to update G1CollectedHeap::gc_epilogue(bool full) to include:
>> > set_live(used());
>> > With this you don't need the changes for the G1FullCollector. The liveness calculated at Remark would be used until the next young collection and I think here is where some improvements could be made. During the mixed phase a better solution would make use of the liveness information we have for the old regions together with what is newly allocated, but this needs further investigation.
>> 
>> This sounds interesting. Let me try this out.
> 
> Glad you like the idea :) I did a quick test locally and it shows the trend ok, even if it is an over estimate of live: 
>   live = 1.1 GB
>   live = 1.2 GB
>   live = 1.5 GB
>   live = 1.7 GB
>   live = 2.1 GB
>   name = "G1Old"
>   live = 1.4 GB
>   live = 1.6 GB
>   live = 1.8 GB
>   live = 2.0 GB
>   live = 2.3 GB
>   live = 2.5 GB
>   live = 2.8 GB
>   live = 3.1 GB
>   live = 3.3 GB
>   live = 3.7 GB
>   live = 4.0 GB
>   live = 4.3 GB
>   name = "G1Old"
>   live = 1.2 GB
> G1Old is from concurrent mark events.

@kstefanj 
Just to make sure - `set_live(used())` should be the last call in `G1CollectedHeap::gc_prologue(bool full)` ?

I am getting some funny numbers with this change - basically, last known live size is getting bigger than the current used size ?? 

![Screen Shot 2021-03-16 at 1 28 27 PM](https://user-images.githubusercontent.com/738413/111308939-97b06d80-865b-11eb-9e2f-d595b49d6401.png)

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From zgu at openjdk.java.net  Tue Mar 16 14:13:34 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 16 Mar 2021 14:13:34 GMT
Subject: RFR: 8262398: Shenandoah: Disable nmethod barrier and stack
 watermark when running with passive mode [v2]
In-Reply-To: <kh_ptVV0GTO1UFhFX3DvdIQ8UzMGMwKINN_-hsOLEoM=.be1e940b-0806-4736-a33a-9c733c3ff6c0@github.com>
References: <kh_ptVV0GTO1UFhFX3DvdIQ8UzMGMwKINN_-hsOLEoM=.be1e940b-0806-4736-a33a-9c733c3ff6c0@github.com>
Message-ID: <-z3FwIn6LV32kasfJlcpDrVSBmaC5ISxWEbeMZNg4VY=.e42ef919-fd89-4a2a-a70b-ab61f530aa96@github.com>

> nmethod barrier and stack watermark allow GC not to process nmethods at GC pauses, and aim to reduce GC latency, they do not benefit STW GCs, who process nmethods at pauses anyway.
> 
> Test:
> 
> - [x] hotspot_gc_shenandoah
> - [x] tier1 with -XX:+UseShenandoahGC
> - [x] tier1 with -XX:+UseShenandoahGC -XX:ShenandoahGCMode=passive
> - [x]  tier1 with -XX:+UseShenandoahGC -XX:ShenandoahGCMode=passive -XX:-ShenandoahDegeneratedGC

Zhengyu Gu has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision:

 - Uses flags instead of mode
 - Merge branch 'master' into JDK-8262398-disable-barriers-stw-gc
 - JDK-8262398
 - init

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2727/files
  - new: https://git.openjdk.java.net/jdk/pull/2727/files/045c7af1..317e7e48

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2727&range=01
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2727&range=00-01

  Stats: 49768 lines in 2079 files changed: 34223 ins; 8768 del; 6777 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2727.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2727/head:pull/2727

PR: https://git.openjdk.java.net/jdk/pull/2727

From sjohanss at openjdk.java.net  Tue Mar 16 14:15:33 2021
From: sjohanss at openjdk.java.net (Stefan Johansson)
Date: Tue, 16 Mar 2021 14:15:33 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <8kTv1ENK1FbxIt7NVkNJNgT55bLw5ao4lWi8uq0nktQ=.a969127a-351a-4393-a932-934fbe9b6924@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
 <nR7IvUeT4SQXZ3JLFAqeebGf_7jIbEgNlscTI2Wre3A=.b2bbdcdb-7e13-40c2-9154-a81c2f19feb0@github.com>
 <P9p-koXqhcZktsnAcU0NGu7Bfsd1PjLOB0QOIzdEo7Y=.423674c1-d518-4776-9dc9-dbcbe7f95337@github.com>
 <jisIvoUcZWmMRMvBSUlmPn3LECqPpbQ3--tcst7iSzg=.f961458b-5e71-4344-b442-42dbff80da83@github.com>
 <VhlwpLHpAXcTddxJvnUApHVsubFAbhL7rBl6hUB5PaI=.03b98760-804a-4efd-bcaa-b59e29fb4d09@github.com>
 <YW40etRX5UG_or6xj8tebs_N3DsEw6ibXuP-11d-t9U=.4281df01-d0ac-41a5-9c17-f1dc0ff849f9@github.com>
 <ACzsS56fFDL2OU-KQzLr0aCA5iFkjt97aQFtX2hBrR4=.9285865f-b780-4a0d-bfcd-50205ffbb7a3@github.com>
 <PfdQDu2_De1mezV67LsCvlNZlyTQAK7iEp06DSpON5c=.e6b3d0d5-0fd6-4c51-89d7-008129cfadb3@github.com>
 <8kTv1ENK1FbxIt7NVkNJNgT55bLw5ao4lWi
 8uq0nktQ=.a969127a-351a-4393-a932-934fbe9b6924@github.com>
Message-ID: <Csa0_vN-mZ_MEYTnvRym3IniVvd64vG6GgHVzj2MkK4=.e82f74a8-3c23-4b5a-9420-06beaea00d13@github.com>

On Tue, 16 Mar 2021 11:28:50 GMT, Stefan Johansson <sjohanss at openjdk.org> wrote:

>>> Sure, but for the event to be useful we want the reported value to be as close to the reality as possible. I don't understand why you want the lower bound, can you explain why? I would go for the upper bound, which in that case would be used() at the end of the GC. I know used() is not perfect, but for G1 this is the best "cheap" value we have for liveness at the end of any GC.
>> 
>> Mostly because `used()` will report all live instances and potential garbage and will make it inconsistent with what the other GCs would report. 
>> 
>>> So as a middle road I would suggest to update G1CollectedHeap::gc_epilogue(bool full) to include:
>>>
>>> set_live(used());
>> With this you don't need the changes for the G1FullCollector. The liveness calculated at Remark would be used until the next young collection and I think here is where some improvements could be made. During the mixed phase a better solution would make use of the liveness information we have for the old regions together with what is newly allocated, but this needs further investigation.
>> 
>> This sounds interesting. Let me try this out.
>
>> > Sure, but for the event to be useful we want the reported value to be as close to the reality as possible. I don't understand why you want the lower bound, can you explain why? I would go for the upper bound, which in that case would be used() at the end of the GC. I know used() is not perfect, but for G1 this is the best "cheap" value we have for liveness at the end of any GC.
>> 
>> Mostly because `used()` will report all live instances and potential garbage and will make it inconsistent with what the other GCs would report.
>> 
> The other STW GCs do report the same, right? 
> 
>> > So as a middle road I would suggest to update G1CollectedHeap::gc_epilogue(bool full) to include:
>> > set_live(used());
>> > With this you don't need the changes for the G1FullCollector. The liveness calculated at Remark would be used until the next young collection and I think here is where some improvements could be made. During the mixed phase a better solution would make use of the liveness information we have for the old regions together with what is newly allocated, but this needs further investigation.
>> 
>> This sounds interesting. Let me try this out.
> 
> Glad you like the idea :) I did a quick test locally and it shows the trend ok, even if it is an over estimate of live: 
>   live = 1.1 GB
>   live = 1.2 GB
>   live = 1.5 GB
>   live = 1.7 GB
>   live = 2.1 GB
>   name = "G1Old"
>   live = 1.4 GB
>   live = 1.6 GB
>   live = 1.8 GB
>   live = 2.0 GB
>   live = 2.3 GB
>   live = 2.5 GB
>   live = 2.8 GB
>   live = 3.1 GB
>   live = 3.3 GB
>   live = 3.7 GB
>   live = 4.0 GB
>   live = 4.3 GB
>   name = "G1Old"
>   live = 1.2 GB
> G1Old is from concurrent mark events.

> @kstefanj
> Just to make sure - `set_live(used())` should be the last call in `G1CollectedHeap::gc_prologue(bool full)` ?
> 
Anywhere in there should be fine, we should look if there is anything related in there it could be grouped with.

> I am getting some funny numbers with this change - basically, last known live size is getting bigger than the current used size ??
> 
> ![Screen Shot 2021-03-16 at 1 28 27 PM](https://user-images.githubusercontent.com/738413/111308939-97b06d80-865b-11eb-9e2f-d595b49d6401.png)

This is strange, what kind of GCs are happening around those samples?

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Tue Mar 16 14:15:32 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Tue, 16 Mar 2021 14:15:32 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v15]
In-Reply-To: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
Message-ID: <2GhOBO4qaEDTMS1tOwAj3u_-_3__n-M_HVFLE9cSJ-s=.9d7830c1-1f65-46a3-8b16-ce6b77367559@github.com>

> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
> 
> ## Introducing new JFR event
> 
> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
> 
> ## Implementation
> 
> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
> 
> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
> 
> ### Epsilon GC
> 
> Trivial implementation - just return `used()` instead.
> 
> ### Serial GC
> 
> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
> 
> ### Parallel GC
> 
> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
> 
> ### G1 GC
> 
> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
> 
> ### Shenandoah
> 
> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
> 
> ### ZGC
> 
> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.

Jaroslav Bachorik has updated the pull request incrementally with one additional commit since the last revision:

  Update liveness for G1 mixed GC

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2579/files
  - new: https://git.openjdk.java.net/jdk/pull/2579/files/81250d1c..f767f257

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=14
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2579&range=13-14

  Stats: 5 lines in 2 files changed: 3 ins; 2 del; 0 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2579.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2579/head:pull/2579

PR: https://git.openjdk.java.net/jdk/pull/2579

From sjohanss at openjdk.java.net  Tue Mar 16 14:15:33 2021
From: sjohanss at openjdk.java.net (Stefan Johansson)
Date: Tue, 16 Mar 2021 14:15:33 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <Csa0_vN-mZ_MEYTnvRym3IniVvd64vG6GgHVzj2MkK4=.e82f74a8-3c23-4b5a-9420-06beaea00d13@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
 <nR7IvUeT4SQXZ3JLFAqeebGf_7jIbEgNlscTI2Wre3A=.b2bbdcdb-7e13-40c2-9154-a81c2f19feb0@github.com>
 <P9p-koXqhcZktsnAcU0NGu7Bfsd1PjLOB0QOIzdEo7Y=.423674c1-d518-4776-9dc9-dbcbe7f95337@github.com>
 <jisIvoUcZWmMRMvBSUlmPn3LECqPpbQ3--tcst7iSzg=.f961458b-5e71-4344-b442-42dbff80da83@github.com>
 <VhlwpLHpAXcTddxJvnUApHVsubFAbhL7rBl6hUB5PaI=.03b98760-804a-4efd-bcaa-b59e29fb4d09@github.com>
 <YW40etRX5UG_or6xj8tebs_N3DsEw6ibXuP-11d-t9U=.4281df01-d0ac-41a5-9c17-f1dc0ff849f9@github.com>
 <ACzsS56fFDL2OU-KQzLr0aCA5iFkjt97aQFtX2hBrR4=.9285865f-b780-4a0d-bfcd-50205ffbb7a3@github.com>
 <PfdQDu2_De1mezV67LsCvlNZlyTQAK7iEp06DSpON5c=.e6b3d0d5-0fd6-4c51-89d7-008129cfadb3@github.com>
 <8kTv1ENK1FbxIt7NVkNJNgT55bLw5ao4lWi
 8uq0nktQ=.a969127a-351a-4393-a932-934fbe9b6924@github.com>
 <Csa0_vN-mZ_MEYTnvRym3IniVvd64vG6GgHVzj2MkK4=.e82f74a8-3c23-4b5a-9420-06beaea00d13@github.com>
Message-ID: <eq10rlXXObqNaRShzrBB1D9sI1J0DW2NdCYU8ntjZqA=.414cea7e-6d4f-4b78-a382-8b50ecb234d1@github.com>

On Tue, 16 Mar 2021 13:24:38 GMT, Stefan Johansson <sjohanss at openjdk.org> wrote:

>>> > Sure, but for the event to be useful we want the reported value to be as close to the reality as possible. I don't understand why you want the lower bound, can you explain why? I would go for the upper bound, which in that case would be used() at the end of the GC. I know used() is not perfect, but for G1 this is the best "cheap" value we have for liveness at the end of any GC.
>>> 
>>> Mostly because `used()` will report all live instances and potential garbage and will make it inconsistent with what the other GCs would report.
>>> 
>> The other STW GCs do report the same, right? 
>> 
>>> > So as a middle road I would suggest to update G1CollectedHeap::gc_epilogue(bool full) to include:
>>> > set_live(used());
>>> > With this you don't need the changes for the G1FullCollector. The liveness calculated at Remark would be used until the next young collection and I think here is where some improvements could be made. During the mixed phase a better solution would make use of the liveness information we have for the old regions together with what is newly allocated, but this needs further investigation.
>>> 
>>> This sounds interesting. Let me try this out.
>> 
>> Glad you like the idea :) I did a quick test locally and it shows the trend ok, even if it is an over estimate of live: 
>>   live = 1.1 GB
>>   live = 1.2 GB
>>   live = 1.5 GB
>>   live = 1.7 GB
>>   live = 2.1 GB
>>   name = "G1Old"
>>   live = 1.4 GB
>>   live = 1.6 GB
>>   live = 1.8 GB
>>   live = 2.0 GB
>>   live = 2.3 GB
>>   live = 2.5 GB
>>   live = 2.8 GB
>>   live = 3.1 GB
>>   live = 3.3 GB
>>   live = 3.7 GB
>>   live = 4.0 GB
>>   live = 4.3 GB
>>   name = "G1Old"
>>   live = 1.2 GB
>> G1Old is from concurrent mark events.
>
>> @kstefanj
>> Just to make sure - `set_live(used())` should be the last call in `G1CollectedHeap::gc_prologue(bool full)` ?
>> 
> Anywhere in there should be fine, we should look if there is anything related in there it could be grouped with.
> 
>> I am getting some funny numbers with this change - basically, last known live size is getting bigger than the current used size ??
>> 
>> ![Screen Shot 2021-03-16 at 1 28 27 PM](https://user-images.githubusercontent.com/738413/111308939-97b06d80-865b-11eb-9e2f-d595b49d6401.png)
> 
> This is strange, what kind of GCs are happening around those samples?

Oh, sorry! I messed up, you should put the code in `gc_epilogue()` ??

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Tue Mar 16 14:15:33 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Tue, 16 Mar 2021 14:15:33 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <eq10rlXXObqNaRShzrBB1D9sI1J0DW2NdCYU8ntjZqA=.414cea7e-6d4f-4b78-a382-8b50ecb234d1@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
 <nR7IvUeT4SQXZ3JLFAqeebGf_7jIbEgNlscTI2Wre3A=.b2bbdcdb-7e13-40c2-9154-a81c2f19feb0@github.com>
 <P9p-koXqhcZktsnAcU0NGu7Bfsd1PjLOB0QOIzdEo7Y=.423674c1-d518-4776-9dc9-dbcbe7f95337@github.com>
 <jisIvoUcZWmMRMvBSUlmPn3LECqPpbQ3--tcst7iSzg=.f961458b-5e71-4344-b442-42dbff80da83@github.com>
 <VhlwpLHpAXcTddxJvnUApHVsubFAbhL7rBl6hUB5PaI=.03b98760-804a-4efd-bcaa-b59e29fb4d09@github.com>
 <YW40etRX5UG_or6xj8tebs_N3DsEw6ibXuP-11d-t9U=.4281df01-d0ac-41a5-9c17-f1dc0ff849f9@github.com>
 <ACzsS56fFDL2OU-KQzLr0aCA5iFkjt97aQFtX2hBrR4=.9285865f-b780-4a0d-bfcd-50205ffbb7a3@github.com>
 <PfdQDu2_De1mezV67LsCvlNZlyTQAK7iEp06DSpON5c=.e6b3d0d5-0fd6-4c51-89d7-008129cfadb3@github.com>
 <8kTv1ENK1FbxIt7NVkNJNgT55bLw5ao4lWi
 8uq0nktQ=.a969127a-351a-4393-a932-934fbe9b6924@github.com>
 <Csa0_vN-mZ_MEYTnvRym3IniVvd64vG6GgHVzj2MkK4=.e82f74a8-3c23-4b5a-9420-06beaea00d13@github.com>
 <eq10rlXXObqNaRShzrBB1D9sI1J0DW2NdCYU8ntjZqA=.414cea7e-6d4f-4b78-a382-8b50ecb234d1@github.com>
Message-ID: <6T_A5WVMZCvYpceI82Ho8ltG-fLddego0zquojRykOI=.36314a7c-1e28-4e0c-8425-ab583d78c8d8@github.com>

On Tue, 16 Mar 2021 13:25:35 GMT, Stefan Johansson <sjohanss at openjdk.org> wrote:

> Oh, sorry! I messed up, you should put the code in gc_epilogue() ??

Np! Cool, this version works as expected.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From rkennke at openjdk.java.net  Tue Mar 16 14:16:11 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Tue, 16 Mar 2021 14:16:11 GMT
Subject: Integrated: 8263427: Shenandoah: Trigger weak-LRB even when heap is
 stable
In-Reply-To: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
References: <JSCIvthaALcRgskxNDbrN8L1dzJS5dSkkG_zm-dcZoM=.e39f1c1b-f75e-4a86-aaf3-ca5f2726b529@github.com>
Message-ID: <8heQXgfbyTP4HrJoFnWQh0hleVOEMgpfCsqyhxuYYTE=.e902fffd-aac2-491b-a326-71a7d99f230c@github.com>

On Thu, 11 Mar 2021 18:38:26 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

> We currently guard all LRBs, including weak-LRB, by a test for heap-stable and only enter the LRB when heap is unstable (e.g. evacuation or update-refs in progress). However, the weak LRB must also be entered when heap is stable and concurrent refs is in progress, otherwise we may accidentally resurrect otherwise unreachable weak referents. This can happen when we take the shortcut cycle and skip evac&update-refs.
> 
> I believe this might be the root cause for JDK-8262852. 
> 
> The way out of it is change conc-weakroots-in-progress flag to a bit in gc-state, and test for this in weak-LRB gc-state-check, and enter weak-LRB even when heap is stable, but conc-weakroots-in-progress.
> 
> There's one gotcha here: we used to change gc-state only at safepoints so that the flag can safely be propagated to all Java threads. But conc-weakroots-in-progress is turned-off concurrently. I deal with this by propagating the flag change to Java threads via the rendevouz (that we do anyway), and change the global flag only once all threads got the thread-local flag change.
> 
> This stuff makes the verifier unhappy, because it doesn't know about the new bit. And it'd be difficult to properly verify it, because sometimes it is set (conc-cycle) and sometimes it is not (degen-cycle), so instead of additing extra verification, I figured we could keep ignoring the flag (for now?)
> 
> Testing:
>  - [x] New testcase failed without change, passes now
>  - [x] hotspot_gc_shenandoah
>  - [ ] tier1 (+Shenandoah)

This pull request has now been integrated.

Changeset: 75ef6f58
Author:    Roman Kennke <rkennke at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/75ef6f58
Stats:     292 lines in 23 files changed: 215 ins; 66 del; 11 mod

8263427: Shenandoah: Trigger weak-LRB even when heap is stable

Reviewed-by: shade, zgu

-------------

PR: https://git.openjdk.java.net/jdk/pull/2945

From shade at openjdk.java.net  Tue Mar 16 15:17:13 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 16 Mar 2021 15:17:13 GMT
Subject: RFR: 8262398: Shenandoah: Disable nmethod barrier and stack
 watermark when running with passive mode [v2]
In-Reply-To: <-z3FwIn6LV32kasfJlcpDrVSBmaC5ISxWEbeMZNg4VY=.e42ef919-fd89-4a2a-a70b-ab61f530aa96@github.com>
References: <kh_ptVV0GTO1UFhFX3DvdIQ8UzMGMwKINN_-hsOLEoM=.be1e940b-0806-4736-a33a-9c733c3ff6c0@github.com>
 <-z3FwIn6LV32kasfJlcpDrVSBmaC5ISxWEbeMZNg4VY=.e42ef919-fd89-4a2a-a70b-ab61f530aa96@github.com>
Message-ID: <y3Ix9wpOF9I3iC0jGir7YzgQAl-YqrGBT6dlPpBuDys=.b18aa700-d843-4c80-8373-6dc3050f4add@github.com>

On Tue, 16 Mar 2021 14:13:34 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> nmethod barrier and stack watermark allow GC not to process nmethods at GC pauses, and aim to reduce GC latency, they do not benefit STW GCs, who process nmethods at pauses anyway.
>> 
>> Test:
>> 
>> - [x] hotspot_gc_shenandoah
>> - [x] tier1 with -XX:+UseShenandoahGC
>> - [x] tier1 with -XX:+UseShenandoahGC -XX:ShenandoahGCMode=passive
>> - [x]  tier1 with -XX:+UseShenandoahGC -XX:ShenandoahGCMode=passive -XX:-ShenandoahDegeneratedGC
>
> Zhengyu Gu has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision:
> 
>  - Uses flags instead of mode
>  - Merge branch 'master' into JDK-8262398-disable-barriers-stw-gc
>  - JDK-8262398
>  - init

Yes, thank you. A few minor nits below.

src/hotspot/share/gc/shenandoah/mode/shenandoahPassiveMode.cpp line 46:

> 44:   if (FLAG_IS_DEFAULT(ShenandoahStackWatermarkBarrier)) {
> 45:     FLAG_SET_DEFAULT(ShenandoahStackWatermarkBarrier, false);
> 46:   }

This should just go to "// Disable known barriers by default" block below.

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 405:

> 403: }
> 404: 
> 405: void ShenandoahHeap::initialize_gc_mode() {

There is `initialize_heuristics`, which means this should be `initialize_mode`?

src/hotspot/share/gc/shenandoah/shenandoahRootProcessor.inline.hpp line 179:

> 177:   CodeBlobToOopClosure update_blobs(keep_alive, CodeBlobToOopClosure::FixRelocations);
> 178:   ShenandoahCodeBlobAndDisarmClosure blobs_and_disarm_Cl(keep_alive);
> 179:   CodeBlobToOopClosure* codes_cl = ClassUnloading && ShenandoahNMethodBarrier ?

The hunk above has this in parentheses. Pick a style and stick to it, whether parentheses or not.

src/hotspot/share/gc/shenandoah/shenandoahCodeRoots.cpp line 182:

> 180: 
> 181: void ShenandoahCodeRoots::disarm_nmethods() {
> 182:   ShenandoahHeap* const heap = ShenandoahHeap::heap();

This `heap` getter can be moved into the block below, and maybe even inlined?

-------------

Marked as reviewed by shade (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2727

From shade at openjdk.java.net  Tue Mar 16 15:20:12 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 16 Mar 2021 15:20:12 GMT
Subject: RFR: 8262398: Shenandoah: Disable nmethod barrier and stack
 watermark when running with passive mode [v2]
In-Reply-To: <y3Ix9wpOF9I3iC0jGir7YzgQAl-YqrGBT6dlPpBuDys=.b18aa700-d843-4c80-8373-6dc3050f4add@github.com>
References: <kh_ptVV0GTO1UFhFX3DvdIQ8UzMGMwKINN_-hsOLEoM=.be1e940b-0806-4736-a33a-9c733c3ff6c0@github.com>
 <-z3FwIn6LV32kasfJlcpDrVSBmaC5ISxWEbeMZNg4VY=.e42ef919-fd89-4a2a-a70b-ab61f530aa96@github.com>
 <y3Ix9wpOF9I3iC0jGir7YzgQAl-YqrGBT6dlPpBuDys=.b18aa700-d843-4c80-8373-6dc3050f4add@github.com>
Message-ID: <S7PARuGKKY4G3tGghO8UOE9W54uMbTEvrJ3-Fq6NC3U=.863d47f4-bb15-49f7-936e-4751fa1f04d4@github.com>

On Tue, 16 Mar 2021 15:14:05 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Zhengyu Gu has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revision:
>> 
>>  - Uses flags instead of mode
>>  - Merge branch 'master' into JDK-8262398-disable-barriers-stw-gc
>>  - JDK-8262398
>>  - init
>
> Yes, thank you. A few minor nits below.

Forgot one thing. These tests might also need modification to test that we never disable this for true heuristics, etc.

./test/hotspot/jtreg/gc/shenandoah/options/TestSelectiveBarrierFlags.java
./test/hotspot/jtreg/gc/shenandoah/options/TestWrongBarrierEnable.java
./test/hotspot/jtreg/gc/shenandoah/options/TestWrongBarrierDisable.java

-------------

PR: https://git.openjdk.java.net/jdk/pull/2727

From zgu at openjdk.java.net  Tue Mar 16 16:58:23 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 16 Mar 2021 16:58:23 GMT
Subject: RFR: 8262398: Shenandoah: Disable nmethod barrier and stack
 watermark when running with passive mode [v3]
In-Reply-To: <kh_ptVV0GTO1UFhFX3DvdIQ8UzMGMwKINN_-hsOLEoM=.be1e940b-0806-4736-a33a-9c733c3ff6c0@github.com>
References: <kh_ptVV0GTO1UFhFX3DvdIQ8UzMGMwKINN_-hsOLEoM=.be1e940b-0806-4736-a33a-9c733c3ff6c0@github.com>
Message-ID: <FwYuhhJMlXuy0jtvjg8rA343wL7kKe7bB9rR172WGkY=.bc589682-395f-4ca1-b2f2-4780bc91788d@github.com>

> nmethod barrier and stack watermark allow GC not to process nmethods at GC pauses, and aim to reduce GC latency, they do not benefit STW GCs, who process nmethods at pauses anyway.
> 
> Test:
> 
> - [x] hotspot_gc_shenandoah
> - [x] tier1 with -XX:+UseShenandoahGC
> - [x] tier1 with -XX:+UseShenandoahGC -XX:ShenandoahGCMode=passive
> - [x]  tier1 with -XX:+UseShenandoahGC -XX:ShenandoahGCMode=passive -XX:-ShenandoahDegeneratedGC

Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:

  Aleksey's comments

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2727/files
  - new: https://git.openjdk.java.net/jdk/pull/2727/files/317e7e48..2a45cb02

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2727&range=02
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2727&range=01-02

  Stats: 26 lines in 9 files changed: 12 ins; 9 del; 5 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2727.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2727/head:pull/2727

PR: https://git.openjdk.java.net/jdk/pull/2727

From shade at openjdk.java.net  Tue Mar 16 17:35:13 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 16 Mar 2021 17:35:13 GMT
Subject: RFR: 8262398: Shenandoah: Disable nmethod barrier and stack
 watermark when running with passive mode [v3]
In-Reply-To: <FwYuhhJMlXuy0jtvjg8rA343wL7kKe7bB9rR172WGkY=.bc589682-395f-4ca1-b2f2-4780bc91788d@github.com>
References: <kh_ptVV0GTO1UFhFX3DvdIQ8UzMGMwKINN_-hsOLEoM=.be1e940b-0806-4736-a33a-9c733c3ff6c0@github.com>
 <FwYuhhJMlXuy0jtvjg8rA343wL7kKe7bB9rR172WGkY=.bc589682-395f-4ca1-b2f2-4780bc91788d@github.com>
Message-ID: <zvTfs17aBRiPCtH2z8gchz1XaOigPwo4_ggjEBcs_x8=.6c9aadc0-e47e-4db8-9bdc-27e630570602@github.com>

On Tue, 16 Mar 2021 16:58:23 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> nmethod barrier and stack watermark allow GC not to process nmethods at GC pauses, and aim to reduce GC latency, they do not benefit STW GCs, who process nmethods at pauses anyway.
>> 
>> Test:
>> 
>> - [x] hotspot_gc_shenandoah
>> - [x] tier1 with -XX:+UseShenandoahGC
>> - [x] tier1 with -XX:+UseShenandoahGC -XX:ShenandoahGCMode=passive
>> - [x]  tier1 with -XX:+UseShenandoahGC -XX:ShenandoahGCMode=passive -XX:-ShenandoahDegeneratedGC
>
> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Aleksey's comments

Marked as reviewed by shade (Reviewer).

test/hotspot/jtreg/gc/shenandoah/options/TestSelectiveBarrierFlags.java line 50:

> 48:                 new String[] { "ShenandoahCloneBarrier" },
> 49:                 new String[] { "ShenandoahNMethodBarrier"},
> 50:                 new String[] { "ShenandoahStackWatermarkBarrier"}

Note there is a space between `"` and `}`. Minor nit, fix before push.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2727

From zgu at openjdk.java.net  Tue Mar 16 17:52:26 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 16 Mar 2021 17:52:26 GMT
Subject: RFR: 8262398: Shenandoah: Disable nmethod barrier and stack
 watermark when running with passive mode [v4]
In-Reply-To: <kh_ptVV0GTO1UFhFX3DvdIQ8UzMGMwKINN_-hsOLEoM=.be1e940b-0806-4736-a33a-9c733c3ff6c0@github.com>
References: <kh_ptVV0GTO1UFhFX3DvdIQ8UzMGMwKINN_-hsOLEoM=.be1e940b-0806-4736-a33a-9c733c3ff6c0@github.com>
Message-ID: <0WuHVrE-yvdj2WtDK1lXZ1lNGuL3je_JyktVLoPLgaU=.7eda2999-f503-4e4a-8e67-f442aa81dd51@github.com>

> nmethod barrier and stack watermark allow GC not to process nmethods at GC pauses, and aim to reduce GC latency, they do not benefit STW GCs, who process nmethods at pauses anyway.
> 
> Test:
> 
> - [x] hotspot_gc_shenandoah
> - [x] tier1 with -XX:+UseShenandoahGC
> - [x] tier1 with -XX:+UseShenandoahGC -XX:ShenandoahGCMode=passive
> - [x]  tier1 with -XX:+UseShenandoahGC -XX:ShenandoahGCMode=passive -XX:-ShenandoahDegeneratedGC

Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:

  Fix spaces

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/2727/files
  - new: https://git.openjdk.java.net/jdk/pull/2727/files/2a45cb02..f9358209

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=2727&range=03
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=2727&range=02-03

  Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
  Patch: https://git.openjdk.java.net/jdk/pull/2727.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/2727/head:pull/2727

PR: https://git.openjdk.java.net/jdk/pull/2727

From rkennke at openjdk.java.net  Wed Mar 17 09:58:32 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 17 Mar 2021 09:58:32 GMT
Subject: Withdrawn: Merge shenandoah:genshen
In-Reply-To: <fQQ2r0L9mjoPTIc3W9hQ_sfFfuSV8tZeiXE7RGBx65o=.f8085fed-40ac-4632-b88c-b8b49715158c@github.com>
References: <fQQ2r0L9mjoPTIc3W9hQ_sfFfuSV8tZeiXE7RGBx65o=.f8085fed-40ac-4632-b88c-b8b49715158c@github.com>
Message-ID: <VPEBNBFGYoQiPuIIvtqziNOh71AQ58Smk8b-2a4oMnI=.ef2e31d3-6b4c-4b23-ac99-1801a96ca840@github.com>

On Fri, 12 Feb 2021 17:44:24 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:

> Brought mainline (master) commits in a series of smaller merges to make the conflicts more manageable. The refactoring of concurrent mark and the control thread required some non-trivial conflict resolution. I've tested `satb` mode and `generational` mode on the dacapo suite.

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/14

From rkennke at openjdk.java.net  Wed Mar 17 14:46:49 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Wed, 17 Mar 2021 14:46:49 GMT
Subject: RFR: 8262398: Shenandoah: Disable nmethod barrier and stack
 watermark when running with passive mode [v4]
In-Reply-To: <0WuHVrE-yvdj2WtDK1lXZ1lNGuL3je_JyktVLoPLgaU=.7eda2999-f503-4e4a-8e67-f442aa81dd51@github.com>
References: <kh_ptVV0GTO1UFhFX3DvdIQ8UzMGMwKINN_-hsOLEoM=.be1e940b-0806-4736-a33a-9c733c3ff6c0@github.com>
 <0WuHVrE-yvdj2WtDK1lXZ1lNGuL3je_JyktVLoPLgaU=.7eda2999-f503-4e4a-8e67-f442aa81dd51@github.com>
Message-ID: <gTyUBieB871_gTsdV7GMWQ3VZCrRotS2xsrxo6UGQBI=.606b672d-b528-42ae-992b-b5da7741d471@github.com>

On Tue, 16 Mar 2021 17:52:26 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> nmethod barrier and stack watermark allow GC not to process nmethods at GC pauses, and aim to reduce GC latency, they do not benefit STW GCs, who process nmethods at pauses anyway.
>> 
>> Test:
>> 
>> - [x] hotspot_gc_shenandoah
>> - [x] tier1 with -XX:+UseShenandoahGC
>> - [x] tier1 with -XX:+UseShenandoahGC -XX:ShenandoahGCMode=passive
>> - [x]  tier1 with -XX:+UseShenandoahGC -XX:ShenandoahGCMode=passive -XX:-ShenandoahDegeneratedGC
>
> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix spaces

Looks good! Thanks!

-------------

Marked as reviewed by rkennke (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2727

From zgu at openjdk.java.net  Wed Mar 17 16:23:50 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Wed, 17 Mar 2021 16:23:50 GMT
Subject: Integrated: 8262398: Shenandoah: Disable nmethod barrier and stack
 watermark when running with passive mode
In-Reply-To: <kh_ptVV0GTO1UFhFX3DvdIQ8UzMGMwKINN_-hsOLEoM=.be1e940b-0806-4736-a33a-9c733c3ff6c0@github.com>
References: <kh_ptVV0GTO1UFhFX3DvdIQ8UzMGMwKINN_-hsOLEoM=.be1e940b-0806-4736-a33a-9c733c3ff6c0@github.com>
Message-ID: <J0ANg7VH0ixEByMmnKYlBdISb70eDW3Tp011zo5cEew=.9af60851-f819-439c-97a3-d42d3ee50ad8@github.com>

On Thu, 25 Feb 2021 20:06:40 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> nmethod barrier and stack watermark allow GC not to process nmethods at GC pauses, and aim to reduce GC latency, they do not benefit STW GCs, who process nmethods at pauses anyway.
> 
> Test:
> 
> - [x] hotspot_gc_shenandoah
> - [x] tier1 with -XX:+UseShenandoahGC
> - [x] tier1 with -XX:+UseShenandoahGC -XX:ShenandoahGCMode=passive
> - [x]  tier1 with -XX:+UseShenandoahGC -XX:ShenandoahGCMode=passive -XX:-ShenandoahDegeneratedGC

This pull request has now been integrated.

Changeset: 7674da43
Author:    Zhengyu Gu <zgu at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/7674da43
Stats:     49 lines in 13 files changed: 31 ins; 0 del; 18 mod

8262398: Shenandoah: Disable nmethod barrier and stack watermark when running with passive mode

Reviewed-by: rkennke, shade

-------------

PR: https://git.openjdk.java.net/jdk/pull/2727

From tschatzl at openjdk.java.net  Wed Mar 17 17:21:51 2021
From: tschatzl at openjdk.java.net (Thomas Schatzl)
Date: Wed, 17 Mar 2021 17:21:51 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <XhTy_KX3x9O6YjWfh38Z7tlsGXKvoATf_abq4tzBmAE=.f3a957d1-6931-45c1-8390-8d5333dc5921@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
 <nR7IvUeT4SQXZ3JLFAqeebGf_7jIbEgNlscTI2Wre3A=.b2bbdcdb-7e13-40c2-9154-a81c2f19feb0@github.com>
 <P9p-koXqhcZktsnAcU0NGu7Bfsd1PjLOB0QOIzdEo7Y=.423674c1-d518-4776-9dc9-dbcbe7f95337@github.com>
 <jisIvoUcZWmMRMvBSUlmPn3LECqPpbQ3--tcst7iSzg=.f961458b-5e71-4344-b442-42dbff80da83@github.com>
 <VhlwpLHpAXcTddxJvnUApHVsubFAbhL7rBl6hUB5PaI=.03b98760-804a-4efd-bcaa-b59e29fb4d09@github.com>
 <YW40etRX5UG_or6xj8tebs_N3DsEw6ibXuP-11d-t9U=.4281df01-d0ac-41a5-9c17-f1dc0ff849f9@github.com>
 <ACzsS56fFDL2OU-KQzLr0aCA5iFkjt97aQFtX2hBrR4=.9285865f-b780-4a0d-bfcd-50205ffbb7a3@github.com>
 <_XwpqPW-sKwuvZG26bSgdW6RKtQaIJz-0fpRR5wGa0c=.d6c5394b-7fea-4f0e-ab5f-9092286d3db1@github.com>
 <XhTy_KX3x9O6YjWfh38Z7tlsGXKvoATf_ab
 q4tzBmAE=.f3a957d1-6931-45c1-8390-8d5333dc5921@github.com>
Message-ID: <3FYRI-abe_Q_yAVKNJad9uB-TOTFD9p22wO7VW5ojH8=.6f73c0f0-aa0a-44fe-b38c-a8758c9cb46d@github.com>

On Tue, 16 Mar 2021 12:23:52 GMT, Jaroslav Bachorik <jbachorik at openjdk.org> wrote:

> > Also, for this purpose, why would used() not be a good substitute for liveness? If e.g. used() average grows over time you can deduce the same I would assume (particularly used() after mixed gc phase in g1).
> 
> The major problem is that eg. for g1 given large enough heap the used value can keep on growing for quite long time, possibly generating wrong signal about potential memory leak.

So can this liveness estimate - just filter used() a bit more.

Actually I strongly believe that due to containing more details, used() could be more appropriate for this: first, it updates more often, second, information like distance between valleys and peaks of the sawtooth pattern are indicative of memory running away.
If you really want to limit yourselves to something that is similar in update frequency to this liveness estimate, one could use just one value in a "tooth". However I think just strong enough low-pass filtering enough is as fine.

> 
> If the live estimate is set to `used()` after mixed gc phase in g1 I think it still will be a good estimate.
> The only thing I am opposing is having `live()` call return the current `used()` value which, IMO, might become rather confusing.
> 
> > Do you have any numbers on what the impact of using used() vs. this live() would be in such a use case?
> 
> Nope. Do you mean perf impact?

Impact on false positives/negatives.

> 
> > What I'm afraid of is that mixing values taken at different times - used and capacity are taken at the time of the event, and the liveness estimated updated at other, irregular intervals may cause significiant amount of confusion in interpreting this value. It might be obvious to you, but there will be other users.
> 
> IDK. If the event field would explicitly mention that this is the **last known live size estimate** it should set the expectations right.
> 
> > One option could be detaching the liveness estimate from used()/capacity() (I see a value in having some heap usage summary at regular intervals) and send the liveness estimate event just when they are generated? Then the various collectors could send this liveness value at times when they think they are fairly accurate, not when the collectors must and particularly not in conjunction with samples taken at completely different times.
> 
> The problem is the irregularity - when the live size is reported only when it is calculated there might be long periods in the recording missing the live size data at all. In order for this information to be useful it should be reported at least at the beginning and end of a JFR chunk.
> 
> > Independent of whether used/capacity and liveness are sent, the receiver needs to do statistics (trend lines) on those anyway.
> 
> Yes. It's just that with the live size estimate one wouldn't be getting the false positives one would get with used heap trend.

We've now discussed this issue within the gc team a bit and came to the following conclusions.

Before going into that, our guidelines for adding new tracing code: generally we avoid providing functionality in the VM/GC that can be procured easily otherwise or there is a good substitute, particularly if their actual content is unclear. The VM is also generally not the place to store or accumulate data that is only used in external applications for their convenience, particularly where the general public usefulness is questionable or do not drive forward GC algorithms in some way.

In the past in the cases we have done this a few times, and this has caused lots of maintenance burden (adding it, keeping it up to date, and finally removing it because nobody used it after all).

We are open to providing raw data for events that fits this in the most painless way for the VM if they are well specified.

The whole periodic HeapSummary event and its contents are questionable in this light:

- used() and capacity() are provided "regularly", and it could either be retrieved at any time by other means (e.g. MXBeans), or even forced to be provided (cause a gc if you are really desperate). Particularly in cases of continuous monitoring, there should be no problem getting them even with existing JFR events.
- it can be argued that it is *very* painless for GC to provide the periodic used/capacity, their values are well defined for all collectors. Still I believe particularly if you do continuous monitoring, this is kind of unnecessary and should be polled instead of pushed if required at higher frequency as provided now.

- the suggested "liveness estimate" however goes against all of these guidelines:
  - the value is ill-defined if at all. The quality of the implementations is all over the place:
    - Epsilon the used() value
    - Serial the used() value and sometimes some attempt to actually return the live data using the dead-wood heuristic
    - Parallel returns used() always
    - G1 returns the amount of bytes marked plus the bytes allocated while marking, used() in other cases (although that may change)
    - Z returns the amount of bytes marked without the bytes allocated while marking (this is unclear to me actually)
    - Shenandoah seems to be fairly close to G1, I have no idea what the results are on the various additional modes it uses.
  - for those collectors that can not give you a good value, the application could as well easily generate it - just use used(). (That deadwood optimization does not change the situation a lot as the difference would be at most 5% or so difference, well within "estimate" range).
  - it forces gcs that do not use or need that value at all first calculate it and then keep it around for just this event
  - this liveness estimate, which is outdated a few instructions after the application runs, is coupled with current used()/capacity() values
  - there is no indication if that estimate in that event can actually used for the suggested purpose: It could have been calculated at any point in time, so its use for trend lines is limited (e.g.. regression etc). For such a regression you typically need multiple values anyway, and even more for some output with a significant amount of considence, so that single value without timestamp does not seem to help. If you track continuously, you would get all values anyway.

So overall I believe the current suggestion to have the VM provide all these values and the event is just introducing complexity in the VM for convenience of the application.

Still I think there might be need for the raw data if available, and if it's easily obtainable, then fine, do send some duplicate data. So my suggestion and what we in the gc team can support is to

a) provide that HeapSummary event with capacity() and used() (but as mentioned, on a change they are sent out already so I do not see the exact situation in particular with continuous tracking...).

b) provide some Liveness (or "Marked bytes" or similar) event when the value is generated (if they are generated) as a one-off event in places it can be derived without significant costs. I.e. some "send_marked_bytes()/live_bytes()" method to be called in Serial, Parallel, G1, Z and Shenandoah when generated and available. No additional storage of that value in the and repeated emission of that event by the VM.
I'm intentionally using "marked bytes" here because this is a value that can actually be defined and verified by reviewers that it's actually returned. Some best effort estimate is just misleading, and is a pain to maintain and argue whether the goal has been met (and it's even worse to (dis-)prove that a change introduced a bug). Maybe it could be extended to "marked bytes plus allocated during marking, sent when marking finished" for all collectors that do marking - I do not know (particularly for Z), maybe it can.
This approach also allows anyone to easily incrementally add new occurrences of that event as more code to support it has been written (e.g. with PR #2966 for g1 full gc), and allows leaving out collectors that do not support it, or places where this has not been gathered.

We think this may be a useful value, that can be explained to others, will cause minimal misunderstanding, can be verified at least in the reviews (and tests where sending of that event is verified for the various situations it should be sent), and maintained.

Returning "liveness" in a good way (both in accuracy and overhead) is a completely different issue, and probably worth a few PhDs. Just dodging the issue with appending "estimate" to the name is not the fix given alternatives.

I would further ask you to at least create two different CRs for adding the two events (more for later additions of the event) for easier and faster review. You can provide a link to a "all-in" diff to let the reviewers see where you want to go with this. However having non-trivial changes across (at this point) 34 files for different collectors for different reasons is nontrivial and very exhausting to review and re-review (10 times so far at least for me).

Thanks,
  Thomas

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From maoliang.ml at alibaba-inc.com  Thu Mar 18 03:51:40 2021
From: maoliang.ml at alibaba-inc.com (Liang Mao)
Date: Thu, 18 Mar 2021 11:51:40 +0800
Subject: =?UTF-8?B?SkVQIDQwNDogR2VuZXJhdGlvbmFsIFNoZW5hbmRvYWg=?=
Message-ID: <b10c84c7-972d-4573-bd4d-398f6153efed.maoliang.ml@alibaba-inc.com>

Hi Shenandoah developers,

Excited to hear the great news of JEP of Generational Shenandoah GC!
Is there a plan or target release to merge it as an experimental feature? Is JDK17 an option?

Thanks,
Liang


From tschatzl at openjdk.java.net  Thu Mar 18 08:36:50 2021
From: tschatzl at openjdk.java.net (Thomas Schatzl)
Date: Thu, 18 Mar 2021 08:36:50 GMT
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <3FYRI-abe_Q_yAVKNJad9uB-TOTFD9p22wO7VW5ojH8=.6f73c0f0-aa0a-44fe-b38c-a8758c9cb46d@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
 <nR7IvUeT4SQXZ3JLFAqeebGf_7jIbEgNlscTI2Wre3A=.b2bbdcdb-7e13-40c2-9154-a81c2f19feb0@github.com>
 <P9p-koXqhcZktsnAcU0NGu7Bfsd1PjLOB0QOIzdEo7Y=.423674c1-d518-4776-9dc9-dbcbe7f95337@github.com>
 <jisIvoUcZWmMRMvBSUlmPn3LECqPpbQ3--tcst7iSzg=.f961458b-5e71-4344-b442-42dbff80da83@github.com>
 <VhlwpLHpAXcTddxJvnUApHVsubFAbhL7rBl6hUB5PaI=.03b98760-804a-4efd-bcaa-b59e29fb4d09@github.com>
 <YW40etRX5UG_or6xj8tebs_N3DsEw6ibXuP-11d-t9U=.4281df01-d0ac-41a5-9c17-f1dc0ff849f9@github.com>
 <ACzsS56fFDL2OU-KQzLr0aCA5iFkjt97aQFtX2hBrR4=.9285865f-b780-4a0d-bfcd-50205ffbb7a3@github.com>
 <_XwpqPW-sKwuvZG26bSgdW6RKtQaIJz-0fpRR5wGa0c=.d6c5394b-7fea-4f0e-ab5f-9092286d3db1@github.com>
 <XhTy_KX3x9O6YjWfh38Z7tlsGXKvoATf_ab
 q4tzBmAE=.f3a957d1-6931-45c1-8390-8d5333dc5921@github.com>
 <3FYRI-abe_Q_yAVKNJad9uB-TOTFD9p22wO7VW5ojH8=.6f73c0f0-aa0a-44fe-b38c-a8758c9cb46d@github.com>
Message-ID: <H4KDalgWkXD4FGwJKWRYcRJKQlP4vraWALeO3RjZfpk=.f8891723-f90b-4671-b08b-11a43b0c4ced@github.com>

On Wed, 17 Mar 2021 17:19:06 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

> Still I think there might be need for the raw data if available, and if it's easily obtainable, then fine, do send some duplicate data. So my suggestion and what we in the gc team can support is to
>
> a) provide that HeapSummary event with capacity() and used() (but as mentioned, on a change they are sent out already so I do not see the exact situation in particular with continuous tracking...).

There has been some question in the latter part of this statement: the "but as mentioned...." part refers to the situation that if you are already continuously monitoring, you will get all of the `used()` events the VM currently sends anyway (if subscribed). So this periodic event does not give you more information.

There may be need for sending `used()` in particular more often as it is done now (and I am open to somebody improving this), not for convenience but because something interesting happens in the Java heap. I am not sure that a "JFR chunk" is the right periodicity though, because it can potentially mean sending (assuming that a "chunk" is some fixed amount of events) every ms to every hour or day. This "random" sampling may send events both too often and too infrequent and not when it matters.

Thomas

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jbachorik at openjdk.java.net  Thu Mar 18 09:32:52 2021
From: jbachorik at openjdk.java.net (Jaroslav Bachorik)
Date: Thu, 18 Mar 2021 09:32:52 GMT
Subject: Withdrawn: 8258431: Provide a JFR event with live set size estimate
In-Reply-To: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
Message-ID: <gz-YW1JONlcOU6kf1DUdUITTOnTl_vWli9bH3xt9dpM=.ea756930-d19f-4ec3-9281-fa03f8d55fda@github.com>

On Mon, 15 Feb 2021 17:23:44 GMT, Jaroslav Bachorik <jbachorik at openjdk.org> wrote:

> The purpose of this change is to  expose a 'cheap' estimate of the current live set size (the meaning of 'current' is dependent on each particular GC implementation but in worst case 'at last full GC') in form of a periodically emitted JFR event.
> 
> ## Introducing new JFR event
> 
> While there is already 'GC Heap Summary' JFR event it does not fit the requirements as it is closely tied to GC cycle so eg. for ZGC or Shenandoah it may not happen for quite a long time, increasing the risk of not having the heap summary events being present in the JFR recording at all. 
> Because of this I am proposing to add a new 'Heap Usage Summary' event which will be emitted periodically, by default on each JFR chunk, and will contain the information abut the heap capacity, the used and live bytes. This information is available from all GC implementations and can be provided at literally any time.
> 
> ## Implementation
> 
> The implementation differs from GC to GC because each GC algorithm/implementation provides a slightly different way to track the liveness. The common part is `size_t live() const` method added to `CollectedHeap` superclass and the use of a cached 'liveness' value computed after the last GC cycle. If `liveness` hasn't been calculated yet the implementation will default to returning 'used' value.
> 
> The implementations are based on my (rather shallow) knowledge of inner working of the respective GC engines and I am open to suggestions to make them better/correct.
> 
> ### Epsilon GC
> 
> Trivial implementation - just return `used()` instead.
> 
> ### Serial GC
> 
> Here we utilize the fact that mark-copy phase is naturally compacting so the number of bytes after copy is 'live' and that the mark-sweep implementation keeps an internal info about objects being 'dead' but excluded from the compaction effort and we can these numbers to derive the old-gen live set size (used bytes minus the cumulative size of the 'un-dead' objects).
> 
> ### Parallel GC
> 
> For Parallel GC the liveness is calculated as the sum of used bytes in all regions after the last GC cycle. This seems to be a safe bet because this collector is always compacting (AFAIK).
> 
> ### G1 GC
> 
> Using `G1ConcurrentMark::remark()` method the live set size is computed as a sum of `_live_words` from the associated `G1RegionMarkStats` objects. Here I am not 100% sure this approach covers all eventualities and it would be great to have someone skilled in G1 implementation to chime in so I can fix it. However, the numbers I am getting for G1 are comparable to other GCs for the same application.
> 
> ### Shenandoah
> 
> In Shenandoah, the regions are keeping the liveness info. However, the VM op that is used for iterating regions is a safe-pointing one so it would be great to run it in an already safe-pointed context.
> This leads to hooking into `ShenandoahConcurrentMark::finish_mark()` and `ShenandoahSTWMark::mark()` where at the end of the marking process the liveness info is summarized and set to `ShenandoahHeap::_live` volatile field - which is later read by the event emitting code.
> 
> ### ZGC
> 
> `ZStatHeap` is already holding the liveness info - so this implementation is just making it accessible via `ZCollectedHeap::live()` method.

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.java.net/jdk/pull/2579

From jaroslav.bachorik at datadoghq.com  Thu Mar 18 09:26:53 2021
From: jaroslav.bachorik at datadoghq.com (=?UTF-8?Q?Jaroslav_Bachor=C3=ADk?=)
Date: Thu, 18 Mar 2021 10:26:53 +0100
Subject: RFR: 8258431: Provide a JFR event with live set size estimate
 [v12]
In-Reply-To: <3FYRI-abe_Q_yAVKNJad9uB-TOTFD9p22wO7VW5ojH8=.6f73c0f0-aa0a-44fe-b38c-a8758c9cb46d@github.com>
References: <7HVs4jngEbNIQIPQByuE6IRYAxdijfa82uhEFWHld5U=.a7784482-d7e1-4d59-88ee-455d8691631e@github.com>
 <z77ggJByTT9jFZKVToGlXtnt3iEup1he3gq_fCxzEuI=.583e8923-d66b-4f43-b52a-39da29471ce4@github.com>
 <CI04lZprlq75d9vtwaEqckMTplf_1y0kT8-8pHvOlcs=.395bd20d-ab16-4592-aa2c-db91eb627fda@github.com>
 <nR7IvUeT4SQXZ3JLFAqeebGf_7jIbEgNlscTI2Wre3A=.b2bbdcdb-7e13-40c2-9154-a81c2f19feb0@github.com>
 <P9p-koXqhcZktsnAcU0NGu7Bfsd1PjLOB0QOIzdEo7Y=.423674c1-d518-4776-9dc9-dbcbe7f95337@github.com>
 <jisIvoUcZWmMRMvBSUlmPn3LECqPpbQ3--tcst7iSzg=.f961458b-5e71-4344-b442-42dbff80da83@github.com>
 <VhlwpLHpAXcTddxJvnUApHVsubFAbhL7rBl6hUB5PaI=.03b98760-804a-4efd-bcaa-b59e29fb4d09@github.com>
 <YW40etRX5UG_or6xj8tebs_N3DsEw6ibXuP-11d-t9U=.4281df01-d0ac-41a5-9c17-f1dc0ff849f9@github.com>
 <ACzsS56fFDL2OU-KQzLr0aCA5iFkjt97aQFtX2hBrR4=.9285865f-b780-4a0d-bfcd-50205ffbb7a3@github.com>
 <_XwpqPW-sKwuvZG26bSgdW6RKtQaIJz-0fpRR5wGa0c=.d6c5394b-7fea-4f0e-ab5f-9092286d3db1@github.com>
 <XhTy_KX3x9O6YjWfh38Z7tlsGXKvoATf_abq4tzBmAE=.f3a957d1-6931-45c1-8390-8d5333dc5921@github.com>
 <3FYRI-abe_Q_yAVKNJad9uB-TOTFD9p22wO7VW5ojH8=.6f73c0f0-aa0a-44fe-b38c-a8758c9cb46d@github.com>
Message-ID: <CAPb8aJ6tJADOJrtFdpuWwaaxb9F_UPpzG3GU_YVBfgSduj2XqQ@mail.gmail.com>

First of all, it is a pity my [attempt to open a discussion regarding
this topic](https://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2021-February/033705.html)
didn't get more attention.

On Wed, Mar 17, 2021 at 6:22 PM Thomas Schatzl
<tschatzl at openjdk.java.net> wrote:
>
> On Tue, 16 Mar 2021 12:23:52 GMT, Jaroslav Bachorik <jbachorik at openjdk.org> wrote:
>
> > > Also, for this purpose, why would used() not be a good substitute for liveness? If e.g. used() average grows over time you can deduce the same I would assume (particularly used() after mixed gc phase in g1).
> >
> > The major problem is that eg. for g1 given large enough heap the used value can keep on growing for quite long time, possibly generating wrong signal about potential memory leak.
>
> So can this liveness estimate - just filter used() a bit more.
>
> Actually I strongly believe that due to containing more details, used() could be more appropriate for this: first, it updates more often, second, information like distance between valleys and peaks of the sawtooth pattern are indicative of memory running away.
> If you really want to limit yourselves to something that is similar in update frequency to this liveness estimate, one could use just one value in a "tooth". However I think just strong enough low-pass filtering enough is as fine.
>
> >
> > If the live estimate is set to `used()` after mixed gc phase in g1 I think it still will be a good estimate.
> > The only thing I am opposing is having `live()` call return the current `used()` value which, IMO, might become rather confusing.
> >
> > > Do you have any numbers on what the impact of using used() vs. this live() would be in such a use case?
> >
> > Nope. Do you mean perf impact?
>
> Impact on false positives/negatives.
>
> >
> > > What I'm afraid of is that mixing values taken at different times - used and capacity are taken at the time of the event, and the liveness estimated updated at other, irregular intervals may cause significiant amount of confusion in interpreting this value. It might be obvious to you, but there will be other users.
> >
> > IDK. If the event field would explicitly mention that this is the **last known live size estimate** it should set the expectations right.
> >
> > > One option could be detaching the liveness estimate from used()/capacity() (I see a value in having some heap usage summary at regular intervals) and send the liveness estimate event just when they are generated? Then the various collectors could send this liveness value at times when they think they are fairly accurate, not when the collectors must and particularly not in conjunction with samples taken at completely different times.
> >
> > The problem is the irregularity - when the live size is reported only when it is calculated there might be long periods in the recording missing the live size data at all. In order for this information to be useful it should be reported at least at the beginning and end of a JFR chunk.
> >
> > > Independent of whether used/capacity and liveness are sent, the receiver needs to do statistics (trend lines) on those anyway.
> >
> > Yes. It's just that with the live size estimate one wouldn't be getting the false positives one would get with used heap trend.
>
> We've now discussed this issue within the gc team a bit and came to the following conclusions.
>
> Before going into that, our guidelines for adding new tracing code: generally we avoid providing functionality in the VM/GC that can be procured easily otherwise or there is a good substitute, particularly if their actual content is unclear. The VM is also generally not the place to store or accumulate data that is only used in external applications for their convenience, particularly where the general public usefulness is questionable or do not drive forward GC algorithms in some way.
>
> In the past in the cases we have done this a few times, and this has caused lots of maintenance burden (adding it, keeping it up to date, and finally removing it because nobody used it after all).
>
> We are open to providing raw data for events that fits this in the most painless way for the VM if they are well specified.
>
> The whole periodic HeapSummary event and its contents are questionable in this light:
>
> - used() and capacity() are provided "regularly", and it could either be retrieved at any time by other means (e.g. MXBeans), or even forced to be provided (cause a gc if you are really desperate). Particularly in cases of continuous monitoring, there should be no problem getting them even with existing JFR events.

These values might be quite imprecise - it is not uncommon for G1 to
see heap usage increasing for long time which could falsely trigger a
leak alarm.
Also, triggering GC does not seem to be that great idea considering
that it will cause a rather lengthy safepoint.

> - it can be argued that it is *very* painless for GC to provide the periodic used/capacity, their values are well defined for all collectors. Still I believe particularly if you do continuous monitoring, this is kind of unnecessary and should be polled instead of pushed if required at higher frequency as provided now.

Well, we are doing continuous monitoring and it is not unnecessary ..

>
> - the suggested "liveness estimate" however goes against all of these guidelines:
>   - the value is ill-defined if at all. The quality of the implementations is all over the place:
>     - Epsilon the used() value
There is literally no other information for epsilon if I am not
totally mistaken.

>     - Serial the used() value and sometimes some attempt to actually return the live data using the dead-wood heuristic
>     - Parallel returns used() always
>     - G1 returns the amount of bytes marked plus the bytes allocated while marking, used() in other cases (although that may change)
>     - Z returns the amount of bytes marked without the bytes allocated while marking (this is unclear to me actually)
>     - Shenandoah seems to be fairly close to G1, I have no idea what the results are on the various additional modes it uses.
Yes. And this was my initial question in the email thread I started
before even attempting this PR. Whatever limited response I got seemed
to confirm this way of getting the liveness estimate.

>   - for those collectors that can not give you a good value, the application could as well easily generate it - just use used(). (That deadwood optimization does not change the situation a lot as the difference would be at most 5% or so difference, well within "estimate" range).
>   - it forces gcs that do not use or need that value at all first calculate it and then keep it around for just this event
>   - this liveness estimate, which is outdated a few instructions after the application runs, is coupled with current used()/capacity() values
>   - there is no indication if that estimate in that event can actually used for the suggested purpose: It could have been calculated at any point in time, so its use for trend lines is limited (e.g.. regression etc). For such a regression you typically need multiple values anyway, and even more for some output with a significant amount of considence, so that single value without timestamp does not seem to help. If you track continuously, you would get all values anyway.
>
> So overall I believe the current suggestion to have the VM provide all these values and the event is just introducing complexity in the VM for convenience of the application.
>
> Still I think there might be need for the raw data if available, and if it's easily obtainable, then fine, do send some duplicate data. So my suggestion and what we in the gc team can support is to
>
> a) provide that HeapSummary event with capacity() and used() (but as mentioned, on a change they are sent out already so I do not see the exact situation in particular with continuous tracking...).
>
> b) provide some Liveness (or "Marked bytes" or similar) event when the value is generated (if they are generated) as a one-off event in places it can be derived without significant costs. I.e. some "send_marked_bytes()/live_bytes()" method to be called in Serial, Parallel, G1, Z and Shenandoah when generated and available. No additional storage of that value in the and repeated emission of that event by the VM.
> I'm intentionally using "marked bytes" here because this is a value that can actually be defined and verified by reviewers that it's actually returned. Some best effort estimate is just misleading, and is a pain to maintain and argue whether the goal has been met (and it's even worse to (dis-)prove that a change introduced a bug). Maybe it could be extended to "marked bytes plus allocated during marking, sent when marking finished" for all collectors that do marking - I do not know (particularly for Z), maybe it can.
> This approach also allows anyone to easily incrementally add new occurrences of that event as more code to support it has been written (e.g. with PR #2966 for g1 full gc), and allows leaving out collectors that do not support it, or places where this has not been gathered.
>
> We think this may be a useful value, that can be explained to others, will cause minimal misunderstanding, can be verified at least in the reviews (and tests where sending of that event is verified for the various situations it should be sent), and maintained.
>
> Returning "liveness" in a good way (both in accuracy and overhead) is a completely different issue, and probably worth a few PhDs. Just dodging the issue with appending "estimate" to the name is not the fix given alternatives.
>
> I would further ask you to at least create two different CRs for adding the two events (more for later additions of the event) for easier and faster review. You can provide a link to a "all-in" diff to let the reviewers see where you want to go with this. However having non-trivial changes across (at this point) 34 files for different collectors for different reasons is nontrivial and very exhausting to review and re-review (10 times so far at least for me).

I am sorry for wasting yours and other reviewers time. I really tried
getting some consensus before coming with a half-baked PR but as you
can see the responses start flowing only when the PR is 'almost done'.

I am sensing quite negative attitude toward this change and the whole
idea of capturing the liveness estimate. At the same time the
solutions you propose give me nothing over just doing custom JFR event
generation using data obtained via JMX - so I really don't see a point
in doing this change in hotspot.

I am going to close this PR and I apologize again for wasting time of
all involved engineers.

Regards,

-JB-

>
> Thanks,
>   Thomas
>
> -------------
>
> PR: https://git.openjdk.java.net/jdk/pull/2579

From bmathiske at openjdk.java.net  Thu Mar 18 10:59:58 2021
From: bmathiske at openjdk.java.net (Bernd Mathiske)
Date: Thu, 18 Mar 2021 10:59:58 GMT
Subject: Integrated: Only scan dirty cards, not entire regions,
 in update refs remset scan.
In-Reply-To: <N9hypfdi6_PMANGXV0duSNnaauuRp4wDrXvfEzS1j60=.a466c8a0-6e04-4477-8cca-0f3a60ba6518@github.com>
References: <N9hypfdi6_PMANGXV0duSNnaauuRp4wDrXvfEzS1j60=.a466c8a0-6e04-4477-8cca-0f3a60ba6518@github.com>
Message-ID: <hhV0_O6Ol0UtnJXPn7r9be5MUEtcEqvyWeMP6fmyGPg=.e0eb0daa-e770-4c53-b0c6-60c3ce3d3f60@github.com>

On Fri, 12 Mar 2021 21:11:45 GMT, Bernd Mathiske <bmathiske at openjdk.org> wrote:

> So far in update refs, when iterating over regions to process the remset, we scan an entire region for references into young gen if we find any dirty card in it. With this update, we only process dirty cards in each region and leave out clean ones. To do so, we refactor the scan code from the marking phase into a method that is then reused for update refs, passing down a different closure. The JVM switch `ShenandoahUseSimpleCardScanning` provides a means to fall back on the previous code.
> 
> (In the future, we may want to replace region stealing with card stealing.)

This pull request has now been integrated.

Changeset: 1c4b4ef9
Author:    Bernd Mathiske <bmathiske at openjdk.org>
Committer: Roman Kennke <rkennke at openjdk.org>
URL:       https://git.openjdk.java.net/shenandoah/commit/1c4b4ef9
Stats:     78 lines in 4 files changed: 30 ins; 26 del; 22 mod

Only scan dirty cards, not entire regions, in update refs remset scan.

Reviewed-by: rkennke

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/22

From rkennke at redhat.com  Thu Mar 18 12:19:27 2021
From: rkennke at redhat.com (Roman Kennke)
Date: Thu, 18 Mar 2021 13:19:27 +0100
Subject: CFV: New Shenandoah Committer: William Kemper
Message-ID: <493d0ab6-0ce5-63e2-64e9-2d1b4d45cd81@redhat.com>

I hereby nominate William Kemper to Shenandoah Committer.

William is an engineer in Amazon's Corretto team and is working on 
Generational Shenandoah. He has already contributed several very 
significant changes to Shenandoah as part of that work. See here for a 
full list:

https://github.com/openjdk/shenandoah/commits?author=earthling-amzn


Votes are due by April 2nd.

Only current Shenandoah Committers [1] are eligible to vote
on this nomination.  Votes must be cast in the open by replying
to this mailing list.

For Lazy Consensus voting instructions, see [2].

Roman Kennke

[1] https://openjdk.java.net/census
[2] https://openjdk.java.net/projects/#committer-vote


From zgu at redhat.com  Thu Mar 18 12:25:56 2021
From: zgu at redhat.com (Zhengyu Gu)
Date: Thu, 18 Mar 2021 08:25:56 -0400
Subject: CFV: New Shenandoah Committer: William Kemper
In-Reply-To: <493d0ab6-0ce5-63e2-64e9-2d1b4d45cd81@redhat.com>
References: <493d0ab6-0ce5-63e2-64e9-2d1b4d45cd81@redhat.com>
Message-ID: <db06de00-2045-eab0-8be5-46c97ce271a9@redhat.com>

Vote: yes

-Zhengyu

On 3/18/21 8:19 AM, Roman Kennke wrote:
> I hereby nominate William Kemper to Shenandoah Committer.
> 
> William is an engineer in Amazon's Corretto team and is working on 
> Generational Shenandoah. He has already contributed several very 
> significant changes to Shenandoah as part of that work. See here for a 
> full list:
> 
> https://github.com/openjdk/shenandoah/commits?author=earthling-amzn
> 
> 
> Votes are due by April 2nd.
> 
> Only current Shenandoah Committers [1] are eligible to vote
> on this nomination.? Votes must be cast in the open by replying
> to this mailing list.
> 
> For Lazy Consensus voting instructions, see [2].
> 
> Roman Kennke
> 
> [1] https://openjdk.java.net/census
> [2] https://openjdk.java.net/projects/#committer-vote
> 


From shade at redhat.com  Thu Mar 18 12:28:39 2021
From: shade at redhat.com (Aleksey Shipilev)
Date: Thu, 18 Mar 2021 13:28:39 +0100
Subject: CFV: New Shenandoah Committer: William Kemper
In-Reply-To: <493d0ab6-0ce5-63e2-64e9-2d1b4d45cd81@redhat.com>
References: <493d0ab6-0ce5-63e2-64e9-2d1b4d45cd81@redhat.com>
Message-ID: <6dfd09f9-1651-830e-bb6c-1cf3a21e9348@redhat.com>

Vote: yes

On 3/18/21 1:19 PM, Roman Kennke wrote:
> I hereby nominate William Kemper to Shenandoah Committer.

-- 
Thanks,
-Aleksey


From zgu at openjdk.java.net  Thu Mar 18 14:59:10 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 18 Mar 2021 14:59:10 GMT
Subject: RFR: Fix ShenandoahPurgeSATBTask
Message-ID: <LkZKPqvHoDRCLjxTuZAKlqU3gfWTTlH55u2Pfhym7RU=.429f629d-e977-4b85-896e-6e5af2774ab8@github.com>

ShenandoahPurgeSATBTask looks wrong.

1) Thread's token should be claimed by task, not by closure
2) Threads::threads_do() is single thread version.

Note: It can be improved by only iterating JavaThread with ShenandoahJavaThreadsIterator, but it needs phase time ...

-------------

Commit messages:
 - Fix ShenandoahPurgeSATBTask

Changes: https://git.openjdk.java.net/shenandoah/pull/23/files
 Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=23&range=00
  Stats: 17 lines in 1 file changed: 6 ins; 4 del; 7 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/23.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/23/head:pull/23

PR: https://git.openjdk.java.net/shenandoah/pull/23

From github.com+71722661+earthling-amzn at openjdk.java.net  Thu Mar 18 15:42:09 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Thu, 18 Mar 2021 15:42:09 GMT
Subject: RFR: Restore ShenandoahVerify functionality for generational mode
Message-ID: <hTYw7u97JwI0kXCpVjbuI8pE49AXY6uNunoi0CDz9gQ=.e8d2b318-0ac9-44ea-88f6-f376c3d64413@github.com>

### Summary
1. Add remembered set to verifier roots for young generation collections
2. Don't follow pointers outside the generation during verify traversals
3. Verify generation's `used` value
4. Correct errors in computing a generation's `used` values

### Testing
1. Dacapo suite completes without verification errors
2. hotspot_gc_shenandoah completes without verification errors

-------------

Commit messages:
 - Fix young generation usage when humongous objects are promoted
 - Fix copy paste error accounting for humongous region usage
 - Count actual size of humongous objects towards generation usage
 - Only include rset in roots for young generation
 - Verify regions and oops in the active generation
 - Include remembered set in verification roots
 - Verify generation usage and region states

Changes: https://git.openjdk.java.net/shenandoah/pull/24/files
 Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=24&range=00
  Stats: 176 lines in 15 files changed: 154 ins; 6 del; 16 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/24.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/24/head:pull/24

PR: https://git.openjdk.java.net/shenandoah/pull/24

From github.com+71722661+earthling-amzn at openjdk.java.net  Thu Mar 18 15:45:57 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Thu, 18 Mar 2021 15:45:57 GMT
Subject: RFR: Fix ShenandoahPurgeSATBTask
In-Reply-To: <LkZKPqvHoDRCLjxTuZAKlqU3gfWTTlH55u2Pfhym7RU=.429f629d-e977-4b85-896e-6e5af2774ab8@github.com>
References: <LkZKPqvHoDRCLjxTuZAKlqU3gfWTTlH55u2Pfhym7RU=.429f629d-e977-4b85-896e-6e5af2774ab8@github.com>
Message-ID: <xQ7ocsCd4j-xtvXFzuGqBbmyez3cQBVl8mVg1xfNktE=.641a45bd-d399-40b9-807c-7a47604429d5@github.com>

On Thu, 18 Mar 2021 14:54:02 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> ShenandoahPurgeSATBTask looks wrong.
> 
> 1) Thread's token should be claimed by task, not by closure
> 2) Threads::threads_do() is single thread version.
> 
> Note: It can be improved by only iterating JavaThread with ShenandoahJavaThreadsIterator, but it needs phase time ...

Thanks for catching this! For what it's worth, I followed the pattern from `ShenandoahSATBAndRemarkThreadsClosure` in `shenandoahConcurrentMark.cpp`, do you think the code there needs a similar change?

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/23

From zgu at openjdk.java.net  Thu Mar 18 15:51:19 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 18 Mar 2021 15:51:19 GMT
Subject: RFR: Fix ShenandoahPurgeSATBTask [v2]
In-Reply-To: <LkZKPqvHoDRCLjxTuZAKlqU3gfWTTlH55u2Pfhym7RU=.429f629d-e977-4b85-896e-6e5af2774ab8@github.com>
References: <LkZKPqvHoDRCLjxTuZAKlqU3gfWTTlH55u2Pfhym7RU=.429f629d-e977-4b85-896e-6e5af2774ab8@github.com>
Message-ID: <3nfoKGkMFuYQJRjXpzruHPKurgDCoeHVUA78qn2_-10=.a3608bcf-6a65-47a1-ab62-7dfb05a302bd@github.com>

> ShenandoahPurgeSATBTask looks wrong.
> 
> 1) Thread's token should be claimed by task, not by closure
> 2) Threads::threads_do() is single thread version.
> 
> Note: It can be improved by only iterating JavaThread with ShenandoahJavaThreadsIterator, but it needs phase time ...

Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:

  Remove bad include

-------------

Changes:
  - all: https://git.openjdk.java.net/shenandoah/pull/23/files
  - new: https://git.openjdk.java.net/shenandoah/pull/23/files/7a0d62fd..b9c27ee8

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=23&range=01
 - incr: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=23&range=00-01

  Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/23.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/23/head:pull/23

PR: https://git.openjdk.java.net/shenandoah/pull/23

From rkennke at openjdk.java.net  Thu Mar 18 15:57:00 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Thu, 18 Mar 2021 15:57:00 GMT
Subject: RFR: Fix ShenandoahPurgeSATBTask [v2]
In-Reply-To: <3nfoKGkMFuYQJRjXpzruHPKurgDCoeHVUA78qn2_-10=.a3608bcf-6a65-47a1-ab62-7dfb05a302bd@github.com>
References: <LkZKPqvHoDRCLjxTuZAKlqU3gfWTTlH55u2Pfhym7RU=.429f629d-e977-4b85-896e-6e5af2774ab8@github.com>
 <3nfoKGkMFuYQJRjXpzruHPKurgDCoeHVUA78qn2_-10=.a3608bcf-6a65-47a1-ab62-7dfb05a302bd@github.com>
Message-ID: <TMe06K7SYBJF9VnENzl2TsI6-jjOkrIevniFHAywF2E=.9258934b-03b2-4f86-b8cc-c1fc83f2f3f3@github.com>

On Thu, 18 Mar 2021 15:51:19 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> ShenandoahPurgeSATBTask looks wrong.
>> 
>> 1) Thread's token should be claimed by task, not by closure
>> 2) Threads::threads_do() is single thread version.
>> 
>> Note: It can be improved by only iterating JavaThread with ShenandoahJavaThreadsIterator, but it needs phase time ...
>
> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Remove bad include

Ugh. Right. Looks good!

-------------

Marked as reviewed by rkennke (Reviewer).

PR: https://git.openjdk.java.net/shenandoah/pull/23

From zgu at openjdk.java.net  Thu Mar 18 16:22:57 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 18 Mar 2021 16:22:57 GMT
Subject: RFR: Fix ShenandoahPurgeSATBTask [v2]
In-Reply-To: <TMe06K7SYBJF9VnENzl2TsI6-jjOkrIevniFHAywF2E=.9258934b-03b2-4f86-b8cc-c1fc83f2f3f3@github.com>
References: <LkZKPqvHoDRCLjxTuZAKlqU3gfWTTlH55u2Pfhym7RU=.429f629d-e977-4b85-896e-6e5af2774ab8@github.com>
 <3nfoKGkMFuYQJRjXpzruHPKurgDCoeHVUA78qn2_-10=.a3608bcf-6a65-47a1-ab62-7dfb05a302bd@github.com>
 <TMe06K7SYBJF9VnENzl2TsI6-jjOkrIevniFHAywF2E=.9258934b-03b2-4f86-b8cc-c1fc83f2f3f3@github.com>
Message-ID: <nroYk4pUF8-3ovBCxEEvSQwFrME2UvoWE4uS0_KSioQ=.36a7c13f-3725-414a-a76e-06b8d09c1d0d@github.com>

On Thu, 18 Mar 2021 15:54:22 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Remove bad include
>
> Ugh. Right. Looks good!

> shenandoahConcurrentMark

Thanks for pointing out. I will fix it in jdk/jdk

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/23

From rkennke at openjdk.java.net  Thu Mar 18 16:41:56 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Thu, 18 Mar 2021 16:41:56 GMT
Subject: RFR: Restore ShenandoahVerify functionality for generational mode
In-Reply-To: <hTYw7u97JwI0kXCpVjbuI8pE49AXY6uNunoi0CDz9gQ=.e8d2b318-0ac9-44ea-88f6-f376c3d64413@github.com>
References: <hTYw7u97JwI0kXCpVjbuI8pE49AXY6uNunoi0CDz9gQ=.e8d2b318-0ac9-44ea-88f6-f376c3d64413@github.com>
Message-ID: <CgZNJh2uNZLM4kPWJxXS_i47IeEFRzq1CI66hZcHl40=.555e1081-7998-415f-afc2-00475bd2b3c9@github.com>

On Thu, 18 Mar 2021 15:36:58 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:

> ### Summary
> 1. Add remembered set to verifier roots for young generation collections
> 2. Don't follow pointers outside the generation during verify traversals
> 3. Verify generation's `used` value
> 4. Correct errors in computing a generation's `used` values
> 
> ### Testing
> 1. Dacapo suite completes without verification errors
> 2. hotspot_gc_shenandoah completes without verification errors

Very nice! Looks good except a few stylistic nits.
How far away is it from running all of hotspot_gc_shenandoah with added generational runners?

src/hotspot/share/gc/shenandoah/shenandoahHeapRegion.cpp line 828:

> 826: 
> 827:   UpdateCardValuesClosure update_card_values;
> 828:   ShenandoahGeneration *old_generation = heap->old_generation();

We usually use Type* var instead of Type *var notation.

src/hotspot/share/gc/shenandoah/shenandoahRootVerifier.cpp line 113:

> 111:   }
> 112: 
> 113:   ShenandoahHeap *heap = ShenandoahHeap::heap();

Same as above. Use ShenandoahHeap* heap. Occurs a few times in this file.

src/hotspot/share/gc/shenandoah/shenandoahScanRemembered.inline.hpp line 596:

> 594:   ShenandoahHeap* heap = ShenandoahHeap::heap();
> 595:   for (size_t i = 0, n = heap->num_regions(); i < n; ++i) {
> 596:     ShenandoahHeapRegion* region = heap->get_region(i);

And in this file too. Please check the whole change for this pattern ;-)

-------------

Marked as reviewed by rkennke (Reviewer).

PR: https://git.openjdk.java.net/shenandoah/pull/24

From github.com+71722661+earthling-amzn at openjdk.java.net  Thu Mar 18 17:24:15 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Thu, 18 Mar 2021 17:24:15 GMT
Subject: RFR: Restore ShenandoahVerify functionality for generational mode
 [v2]
In-Reply-To: <hTYw7u97JwI0kXCpVjbuI8pE49AXY6uNunoi0CDz9gQ=.e8d2b318-0ac9-44ea-88f6-f376c3d64413@github.com>
References: <hTYw7u97JwI0kXCpVjbuI8pE49AXY6uNunoi0CDz9gQ=.e8d2b318-0ac9-44ea-88f6-f376c3d64413@github.com>
Message-ID: <7JCUNVWITflRY7qjS3GFgP5fj1a5KMe6hSXmUio5yQY=.924a0401-67c7-4098-a09c-d0f6e4f2432b@github.com>

> ### Summary
> 1. Add remembered set to verifier roots for young generation collections
> 2. Don't follow pointers outside the generation during verify traversals
> 3. Verify generation's `used` value
> 4. Correct errors in computing a generation's `used` values
> 
> ### Testing
> 1. Dacapo suite completes without verification errors
> 2. hotspot_gc_shenandoah completes without verification errors

earthling-amzn has updated the pull request incrementally with one additional commit since the last revision:

  Place '*' next to type, not variable
  
  Also remove unapproved 'override' keyword

-------------

Changes:
  - all: https://git.openjdk.java.net/shenandoah/pull/24/files
  - new: https://git.openjdk.java.net/shenandoah/pull/24/files/b4d1ea78..c05fe2e9

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=24&range=01
 - incr: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=24&range=00-01

  Stats: 27 lines in 11 files changed: 0 ins; 0 del; 27 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/24.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/24/head:pull/24

PR: https://git.openjdk.java.net/shenandoah/pull/24

From github.com+71722661+earthling-amzn at openjdk.java.net  Thu Mar 18 17:24:16 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Thu, 18 Mar 2021 17:24:16 GMT
Subject: RFR: Restore ShenandoahVerify functionality for generational mode
 [v2]
In-Reply-To: <CgZNJh2uNZLM4kPWJxXS_i47IeEFRzq1CI66hZcHl40=.555e1081-7998-415f-afc2-00475bd2b3c9@github.com>
References: <hTYw7u97JwI0kXCpVjbuI8pE49AXY6uNunoi0CDz9gQ=.e8d2b318-0ac9-44ea-88f6-f376c3d64413@github.com>
 <CgZNJh2uNZLM4kPWJxXS_i47IeEFRzq1CI66hZcHl40=.555e1081-7998-415f-afc2-00475bd2b3c9@github.com>
Message-ID: <5nuOyRe3DzVg19g0vmHsdurXyWw5jHwnXOZDEa7KWCs=.c2e051c3-9d97-4289-a607-c91b65dd0aa0@github.com>

On Thu, 18 Mar 2021 16:38:53 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> earthling-amzn has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Place '*' next to type, not variable
>>   
>>   Also remove unapproved 'override' keyword
>
> Very nice! Looks good except a few stylistic nits.
> How far away is it from running all of hotspot_gc_shenandoah with added generational runners?

@rkennke - to answer your question about testing. The generational mode has failures if the test causes a FullGC to run (this is expected right now). It also looks like there may be an issue with `UseStringDeduplication`, but we haven't investigated this yet. Apart from those failures we pass ~200 out of 234 hotspot_gc_shenandoah tests.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/24

From zgu at openjdk.java.net  Thu Mar 18 17:31:07 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 18 Mar 2021 17:31:07 GMT
Subject: RFR: Fix ShenandoahPurgeSATBTask [v3]
In-Reply-To: <LkZKPqvHoDRCLjxTuZAKlqU3gfWTTlH55u2Pfhym7RU=.429f629d-e977-4b85-896e-6e5af2774ab8@github.com>
References: <LkZKPqvHoDRCLjxTuZAKlqU3gfWTTlH55u2Pfhym7RU=.429f629d-e977-4b85-896e-6e5af2774ab8@github.com>
Message-ID: <scU0xT_VFje6PFB3PswNVFrkefPQQGNDflkSBVDyR-U=.63baf2c6-682c-456c-aed9-bc257f5c5b1d@github.com>

> ShenandoahPurgeSATBTask looks wrong.
> 
> 1) Thread's token should be claimed by task, not by closure
> 2) Threads::threads_do() is single thread version.
> 
> Note: It can be improved by only iterating JavaThread with ShenandoahJavaThreadsIterator, but it needs phase time ...

Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:

  Removed unused _claim_token

-------------

Changes:
  - all: https://git.openjdk.java.net/shenandoah/pull/23/files
  - new: https://git.openjdk.java.net/shenandoah/pull/23/files/b9c27ee8..12e380b7

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=23&range=02
 - incr: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=23&range=01-02

  Stats: 2 lines in 1 file changed: 0 ins; 2 del; 0 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/23.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/23/head:pull/23

PR: https://git.openjdk.java.net/shenandoah/pull/23

From zgu at openjdk.java.net  Thu Mar 18 20:03:23 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 18 Mar 2021 20:03:23 GMT
Subject: git: openjdk/shenandoah: master: Fix ShenandoahPurgeSATBTask
Message-ID: <03df4d32-88b0-4473-ab64-46f521897074@openjdk.org>

Changeset: 41ba54b6
Author:    Zhengyu Gu <zgu at openjdk.org>
Date:      2021-03-18 20:03:03 +0000
URL:       https://git.openjdk.java.net/shenandoah/commit/41ba54b6

Fix ShenandoahPurgeSATBTask

Reviewed-by: rkennke

! src/hotspot/share/gc/shenandoah/shenandoahOldGeneration.cpp


From zgu at openjdk.java.net  Thu Mar 18 20:05:59 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 18 Mar 2021 20:05:59 GMT
Subject: Integrated: Fix ShenandoahPurgeSATBTask
In-Reply-To: <LkZKPqvHoDRCLjxTuZAKlqU3gfWTTlH55u2Pfhym7RU=.429f629d-e977-4b85-896e-6e5af2774ab8@github.com>
References: <LkZKPqvHoDRCLjxTuZAKlqU3gfWTTlH55u2Pfhym7RU=.429f629d-e977-4b85-896e-6e5af2774ab8@github.com>
Message-ID: <ZLtqGYbXjbYDFfFLVq25s7SIX6_iJsxDadRi13NziWE=.5f33a59a-b722-43ca-85d1-c1c61fcdf155@github.com>

On Thu, 18 Mar 2021 14:54:02 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> ShenandoahPurgeSATBTask looks wrong.
> 
> 1) Thread's token should be claimed by task, not by closure
> 2) Threads::threads_do() is single thread version.
> 
> Note: It can be improved by only iterating JavaThread with ShenandoahJavaThreadsIterator, but it needs phase time ...

This pull request has now been integrated.

Changeset: 41ba54b6
Author:    Zhengyu Gu <zgu at openjdk.org>
URL:       https://git.openjdk.java.net/shenandoah/commit/41ba54b6
Stats:     14 lines in 1 file changed: 3 ins; 4 del; 7 mod

Fix ShenandoahPurgeSATBTask

Reviewed-by: rkennke

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/23

From zgu at openjdk.java.net  Thu Mar 18 20:06:52 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 18 Mar 2021 20:06:52 GMT
Subject: RFR: 8263832: Shenandoah: Fixing parallel thread iteration in final
 mark task
Message-ID: <I4AZoBoSHNsIvpDXiKwO1kI_3B1nJlwb9HLM26OT9HU=.c828095d-30cc-403b-8cdd-aff518b423fb@github.com>

Please review this trivial patch that parallelizes thread iteration in ShenandoahFinalMarkingTask

Test:
- [x] hotspot_gc_shenandoah

-------------

Commit messages:
 - JDK-8263832-satb-remark

Changes: https://git.openjdk.java.net/jdk/pull/3080/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3080&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8263832
  Stats: 12 lines in 1 file changed: 0 ins; 4 del; 8 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3080.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3080/head:pull/3080

PR: https://git.openjdk.java.net/jdk/pull/3080

From rkennke at openjdk.java.net  Thu Mar 18 20:16:39 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Thu, 18 Mar 2021 20:16:39 GMT
Subject: RFR: 8263832: Shenandoah: Fixing parallel thread iteration in
 final mark task
In-Reply-To: <I4AZoBoSHNsIvpDXiKwO1kI_3B1nJlwb9HLM26OT9HU=.c828095d-30cc-403b-8cdd-aff518b423fb@github.com>
References: <I4AZoBoSHNsIvpDXiKwO1kI_3B1nJlwb9HLM26OT9HU=.c828095d-30cc-403b-8cdd-aff518b423fb@github.com>
Message-ID: <_QoFSdCBwvq-LtsqvwkXkmZ1FdaDzRnLi9mWq5NFXbs=.a67b7980-c2ef-4c67-9722-69b86ff6a472@github.com>

On Thu, 18 Mar 2021 20:01:10 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Please review this trivial patch that parallelizes thread iteration in ShenandoahFinalMarkingTask
> 
> Test:
> - [x] hotspot_gc_shenandoah

It's not trivial. Maybe I'm missing something, but where are the threads claimed now? I believe you should leave the claiming code where it is, right? And only change threads_do() to possibly_parallel_threads_do().

-------------

PR: https://git.openjdk.java.net/jdk/pull/3080

From rkennke at openjdk.java.net  Thu Mar 18 20:18:59 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Thu, 18 Mar 2021 20:18:59 GMT
Subject: RFR: Fix ShenandoahPurgeSATBTask [v2]
In-Reply-To: <nroYk4pUF8-3ovBCxEEvSQwFrME2UvoWE4uS0_KSioQ=.36a7c13f-3725-414a-a76e-06b8d09c1d0d@github.com>
References: <LkZKPqvHoDRCLjxTuZAKlqU3gfWTTlH55u2Pfhym7RU=.429f629d-e977-4b85-896e-6e5af2774ab8@github.com>
 <3nfoKGkMFuYQJRjXpzruHPKurgDCoeHVUA78qn2_-10=.a3608bcf-6a65-47a1-ab62-7dfb05a302bd@github.com>
 <TMe06K7SYBJF9VnENzl2TsI6-jjOkrIevniFHAywF2E=.9258934b-03b2-4f86-b8cc-c1fc83f2f3f3@github.com>
 <nroYk4pUF8-3ovBCxEEvSQwFrME2UvoWE4uS0_KSioQ=.36a7c13f-3725-414a-a76e-06b8d09c1d0d@github.com>
Message-ID: <ih82T1EskDAqJy-b1LJ4pGgzF5BJMiDCAAzJBauEamY=.c111316a-1c2f-4497-b359-15d334309b5d@github.com>

On Thu, 18 Mar 2021 16:19:46 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> Ugh. Right. Looks good!
>
>> shenandoahConcurrentMark
> 
> Thanks for pointing out. I will fix it in jdk/jdk

Actually, now that I think about it a little more, I believe you should leave the claiming code where it was. It was one closure per task, and that should correctly claim the threads. Now you let all tasks see all the threads, which is wrong. Or I'm missing something.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/23

From zgu at openjdk.java.net  Thu Mar 18 20:46:59 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 18 Mar 2021 20:46:59 GMT
Subject: RFR: Fix ShenandoahPurgeSATBTask [v2]
In-Reply-To: <ih82T1EskDAqJy-b1LJ4pGgzF5BJMiDCAAzJBauEamY=.c111316a-1c2f-4497-b359-15d334309b5d@github.com>
References: <LkZKPqvHoDRCLjxTuZAKlqU3gfWTTlH55u2Pfhym7RU=.429f629d-e977-4b85-896e-6e5af2774ab8@github.com>
 <3nfoKGkMFuYQJRjXpzruHPKurgDCoeHVUA78qn2_-10=.a3608bcf-6a65-47a1-ab62-7dfb05a302bd@github.com>
 <TMe06K7SYBJF9VnENzl2TsI6-jjOkrIevniFHAywF2E=.9258934b-03b2-4f86-b8cc-c1fc83f2f3f3@github.com>
 <nroYk4pUF8-3ovBCxEEvSQwFrME2UvoWE4uS0_KSioQ=.36a7c13f-3725-414a-a76e-06b8d09c1d0d@github.com>
 <ih82T1EskDAqJy-b1LJ4pGgzF5BJMiDCAAzJBauEamY=.c111316a-1c2f-4497-b359-15d334309b5d@github.com>
Message-ID: <ABBDB3LLE80MxRzl-RjbhXbA8CE6Xqrve9WBVEUks_c=.abae1d04-5a30-453f-8944-db6fdc5310bc@github.com>

On Thu, 18 Mar 2021 20:15:52 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

> Actually, now that I think about it a little more, I believe you should leave the claiming code where it was. It was one closure per task, and that should correctly claim the threads. Now you let all tasks see all the threads, which is wrong. Or I'm missing something.

The threads are claimed inside Threads::possibly_parallel_threads_do()

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/23

From zgu at openjdk.java.net  Thu Mar 18 20:47:38 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Thu, 18 Mar 2021 20:47:38 GMT
Subject: RFR: 8263832: Shenandoah: Fixing parallel thread iteration in
 final mark task
In-Reply-To: <_QoFSdCBwvq-LtsqvwkXkmZ1FdaDzRnLi9mWq5NFXbs=.a67b7980-c2ef-4c67-9722-69b86ff6a472@github.com>
References: <I4AZoBoSHNsIvpDXiKwO1kI_3B1nJlwb9HLM26OT9HU=.c828095d-30cc-403b-8cdd-aff518b423fb@github.com>
 <_QoFSdCBwvq-LtsqvwkXkmZ1FdaDzRnLi9mWq5NFXbs=.a67b7980-c2ef-4c67-9722-69b86ff6a472@github.com>
Message-ID: <TwlJ30XcC-Jn9yr47t9LCgua4ntnBrswu1_mhDzmVpE=.ec023c45-9b30-4975-ad40-fc476ce9e516@github.com>

On Thu, 18 Mar 2021 20:13:32 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

> It's not trivial. Maybe I'm missing something, but where are the threads claimed now? I believe you should leave the claiming code where it is, right? And only change threads_do() to possibly_parallel_threads_do().

Threads are claimed inside Threads::possibly_parallel_threads_do() (https://github.com/openjdk/jdk/blob/master/src/hotspot/share/runtime/thread.cpp#L2866)

-------------

PR: https://git.openjdk.java.net/jdk/pull/3080

From rkennke at openjdk.java.net  Thu Mar 18 20:47:37 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Thu, 18 Mar 2021 20:47:37 GMT
Subject: RFR: 8263832: Shenandoah: Fixing parallel thread iteration in
 final mark task
In-Reply-To: <I4AZoBoSHNsIvpDXiKwO1kI_3B1nJlwb9HLM26OT9HU=.c828095d-30cc-403b-8cdd-aff518b423fb@github.com>
References: <I4AZoBoSHNsIvpDXiKwO1kI_3B1nJlwb9HLM26OT9HU=.c828095d-30cc-403b-8cdd-aff518b423fb@github.com>
Message-ID: <vryBV_1ysmJTyTnY2r9NHdIaMIicCx-Ppv5YlGJiDiU=.64fd9ff4-212d-4180-9af9-19a145ef8155@github.com>

On Thu, 18 Mar 2021 20:01:10 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Please review this trivial patch that parallelizes thread iteration in ShenandoahFinalMarkingTask
> 
> Test:
> - [x] hotspot_gc_shenandoah

Ok, thanks!

-------------

Marked as reviewed by rkennke (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/3080

From rkennke at openjdk.java.net  Thu Mar 18 20:47:39 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Thu, 18 Mar 2021 20:47:39 GMT
Subject: RFR: 8263832: Shenandoah: Fixing parallel thread iteration in
 final mark task
In-Reply-To: <TwlJ30XcC-Jn9yr47t9LCgua4ntnBrswu1_mhDzmVpE=.ec023c45-9b30-4975-ad40-fc476ce9e516@github.com>
References: <I4AZoBoSHNsIvpDXiKwO1kI_3B1nJlwb9HLM26OT9HU=.c828095d-30cc-403b-8cdd-aff518b423fb@github.com>
 <_QoFSdCBwvq-LtsqvwkXkmZ1FdaDzRnLi9mWq5NFXbs=.a67b7980-c2ef-4c67-9722-69b86ff6a472@github.com>
 <TwlJ30XcC-Jn9yr47t9LCgua4ntnBrswu1_mhDzmVpE=.ec023c45-9b30-4975-ad40-fc476ce9e516@github.com>
Message-ID: <__BROZOFaUDnKJ-dG_uvr5l9nqlH_i88inT_i_Q_C6E=.dba33dc1-28d9-4906-9b61-368009ef820a@github.com>

On Thu, 18 Mar 2021 20:42:15 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> It's not trivial. Maybe I'm missing something, but where are the threads claimed now? I believe you should leave the claiming code where it is, right? And only change threads_do() to possibly_parallel_threads_do().
>
>> It's not trivial. Maybe I'm missing something, but where are the threads claimed now? I believe you should leave the claiming code where it is, right? And only change threads_do() to possibly_parallel_threads_do().
> 
> Threads are claimed inside Threads::possibly_parallel_threads_do() (https://github.com/openjdk/jdk/blob/master/src/hotspot/share/runtime/thread.cpp#L2866)

Ah

> > It's not trivial. Maybe I'm missing something, but where are the threads claimed now? I believe you should leave the claiming code where it is, right? And only change threads_do() to possibly_parallel_threads_do().
> 
> Threads are claimed inside Threads::possibly_parallel_threads_do() (https://github.com/openjdk/jdk/blob/master/src/hotspot/share/runtime/thread.cpp#L2866)

Ah ok, alright. It's clearly too late for work! Sorry for the noise.

-------------

PR: https://git.openjdk.java.net/jdk/pull/3080

From zgu at openjdk.java.net  Fri Mar 19 00:23:39 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Fri, 19 Mar 2021 00:23:39 GMT
Subject: Integrated: 8263832: Shenandoah: Fixing parallel thread iteration in
 final mark task
In-Reply-To: <I4AZoBoSHNsIvpDXiKwO1kI_3B1nJlwb9HLM26OT9HU=.c828095d-30cc-403b-8cdd-aff518b423fb@github.com>
References: <I4AZoBoSHNsIvpDXiKwO1kI_3B1nJlwb9HLM26OT9HU=.c828095d-30cc-403b-8cdd-aff518b423fb@github.com>
Message-ID: <EGGGqiuqzD1bK1IKmoPH1bY555fZL855SR5wPe1jEcI=.843796ed-ba3c-4f16-aca5-30b76635c7a5@github.com>

On Thu, 18 Mar 2021 20:01:10 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Please review this trivial patch that parallelizes thread iteration in ShenandoahFinalMarkingTask
> 
> Test:
> - [x] hotspot_gc_shenandoah

This pull request has now been integrated.

Changeset: d185655c
Author:    Zhengyu Gu <zgu at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/d185655c
Stats:     12 lines in 1 file changed: 0 ins; 4 del; 8 mod

8263832: Shenandoah: Fixing parallel thread iteration in final mark task

Reviewed-by: rkennke

-------------

PR: https://git.openjdk.java.net/jdk/pull/3080

From rkennke at openjdk.java.net  Fri Mar 19 11:28:58 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Fri, 19 Mar 2021 11:28:58 GMT
Subject: RFR: Restore ShenandoahVerify functionality for generational mode
 [v2]
In-Reply-To: <CgZNJh2uNZLM4kPWJxXS_i47IeEFRzq1CI66hZcHl40=.555e1081-7998-415f-afc2-00475bd2b3c9@github.com>
References: <hTYw7u97JwI0kXCpVjbuI8pE49AXY6uNunoi0CDz9gQ=.e8d2b318-0ac9-44ea-88f6-f376c3d64413@github.com>
 <CgZNJh2uNZLM4kPWJxXS_i47IeEFRzq1CI66hZcHl40=.555e1081-7998-415f-afc2-00475bd2b3c9@github.com>
Message-ID: <SugW9p_0uJXxN5au3FWN6G-UIUtjfKFRdkJad0lOiuc=.289bfa6c-3e2c-4b31-bbef-ac6100dabc2d@github.com>

On Thu, 18 Mar 2021 16:38:53 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> earthling-amzn has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Place '*' next to type, not variable
>>   
>>   Also remove unapproved 'override' keyword
>
> Very nice! Looks good except a few stylistic nits.
> How far away is it from running all of hotspot_gc_shenandoah with added generational runners?

> @rkennke - to answer your question about testing. The generational mode has failures if the test causes a FullGC to run (this is expected right now). It also looks like there may be an issue with `UseStringDeduplication`, but we haven't investigated this yet. Apart from those failures we pass ~200 out of 234 hotspot_gc_shenandoah tests.

Very nice! Maybe it would be worth to add run configurations to the tests that are expected to pass under hotspot_gc_shenandoah_generational?

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/24

From github.com+71722661+earthling-amzn at openjdk.java.net  Fri Mar 19 11:28:59 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Fri, 19 Mar 2021 11:28:59 GMT
Subject: Integrated: Restore ShenandoahVerify functionality for generational
 mode
In-Reply-To: <hTYw7u97JwI0kXCpVjbuI8pE49AXY6uNunoi0CDz9gQ=.e8d2b318-0ac9-44ea-88f6-f376c3d64413@github.com>
References: <hTYw7u97JwI0kXCpVjbuI8pE49AXY6uNunoi0CDz9gQ=.e8d2b318-0ac9-44ea-88f6-f376c3d64413@github.com>
Message-ID: <45GazlUYIYiv_c4_6jemViJx_QlwWNIsJIyrHmOb52E=.e382d044-000c-4160-ad6f-723de038e8ab@github.com>

On Thu, 18 Mar 2021 15:36:58 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:

> ### Summary
> 1. Add remembered set to verifier roots for young generation collections
> 2. Don't follow pointers outside the generation during verify traversals
> 3. Verify generation's `used` value
> 4. Correct errors in computing a generation's `used` values
> 
> ### Testing
> 1. Dacapo suite completes without verification errors
> 2. hotspot_gc_shenandoah completes without verification errors

This pull request has now been integrated.

Changeset: bcde2f61
Author:    William Kemper <kemperw at amazon.com>
Committer: Roman Kennke <rkennke at openjdk.org>
URL:       https://git.openjdk.java.net/shenandoah/commit/bcde2f61
Stats:     185 lines in 16 files changed: 154 ins; 6 del; 25 mod

Restore ShenandoahVerify functionality for generational mode

Reviewed-by: rkennke

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/24

From rkennke at openjdk.java.net  Fri Mar 19 11:35:43 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Fri, 19 Mar 2021 11:35:43 GMT
Subject: RFR: 8261492: Shenandoah: reconsider forwardee accesses memory
 ordering [v3]
In-Reply-To: <OeK5Ff0cZaQ7qV2HFv8IsffVTbME9-ozEXSSu-gWsO8=.72593252-988a-478d-9360-51282530b8a6@github.com>
References: <eP3bMKbgFS1MJ1MO2DUcTom0eNJT8zszs4A-PJXIgsI=.3cead947-d093-466a-9cfd-0c312eec592a@github.com>
 <OeK5Ff0cZaQ7qV2HFv8IsffVTbME9-ozEXSSu-gWsO8=.72593252-988a-478d-9360-51282530b8a6@github.com>
Message-ID: <6m_sTEtwhTj1zH7SK6fdyzjavt9cykNU48ktFB96CmM=.a32e4a69-80b7-42f6-90ca-984a88209211@github.com>

On Tue, 16 Feb 2021 10:26:06 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Shenandoah carries forwardee information in object's mark word. Installing the new mark word is effectively "releasing" the object copy, and reading from the new mark word is "acquiring" that object copy.
>> 
>> For the forwardee update side, Hotspot's default for atomic operations is memory_order_conservative, which emits two-way memory fences around the CASes at least on AArch64 and PPC64. This seems to be excessive for Shenandoah forwardee updates, and "release" is enough.
>> 
>> For the forwardee load side, we need to guarantee "acquire". We do not do it now, reading the markword without memory semantics. It does not seem to pose a practical problem today, because GC does not access the object contents in the new copy, and mutators get this from the JRT-called stub that separates the fwdptr access and object contents access by a lot. It still should be cleaner to "acquire" the mark on load to avoid surprises.
>> 
>> Additional testing:
>>  - [x] Linux x86_64 `hotspot_gc_shenandoah`
>>  - [x] Linux AArch64 `hotspot_gc_shenandoah`
>>  - [x] Linux AArch64 `tier1` with Shenandoah
>
> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
> 
>  - A few minor touchups
>  - Add a blurb to x86 code as well
>  - Use implicit "consume" in AArch64, add more notes.
>  - Merge branch 'master' into JDK-8261492-shenandoah-forwardee-memord
>  - Make sure to access fwdptr with acquire semantics in assembler code
>  - 8261492: Shenandoah: reconsider forwardee accesses memory ordering

Looks good to me.

-------------

Marked as reviewed by rkennke (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2496

From rkennke at openjdk.java.net  Fri Mar 19 11:41:44 2021
From: rkennke at openjdk.java.net (Roman Kennke)
Date: Fri, 19 Mar 2021 11:41:44 GMT
Subject: RFR: 8261495: Shenandoah: reconsider update references memory
 ordering [v4]
In-Reply-To: <mmDtPov974Ez620ICbTzeSGQ6DJpv9CMAzfRW0Ai3ro=.a3b6d2c2-7b2e-4902-a5dc-c7241b7e6ada@github.com>
References: <4RLKvcdaWu0Cu6owC3yGoVY1KVEsYjBZEFJhfdwnhWg=.65fbeae1-58f6-48d3-a2ed-981858ef7da9@github.com>
 <mmDtPov974Ez620ICbTzeSGQ6DJpv9CMAzfRW0Ai3ro=.a3b6d2c2-7b2e-4902-a5dc-c7241b7e6ada@github.com>
Message-ID: <XSAvTe3aCY8BKKTOa0pnjpKgb5cCO-jS5-mBVZS1bWI=.4c3007c0-16ba-4139-84e5-a13fb9f9f1b9@github.com>

On Tue, 16 Feb 2021 13:21:00 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Shenandoah update heap references code uses default Atomic::cmpxchg to avoid races with mutator updates. Hotspot's default for atomic operations is memory_order_conservative, which emits two-way memory fences around the CASes at least on AArch64 and PPC64.
>> 
>> This seems to be excessive for Shenandoah update references code, and "release" is enough. We do not seem to piggyback on update-references memory effects anywhere (in fact, if not for mutator, we would not even need a CAS). But, there is an interplay with concurrent evacuation and updates from self-healing.
>> 
>> Average time goes down, the number of GC cycles go up, since the cycles are shorter.
>> 
>> Additional testing:
>>  - [x] Linux x86_64 hotspot_gc_shenandoah
>>  - [x] Linux AArch64 hotspot_gc_shenandoah
>>  - [x] Linux AArch64 tier1 with Shenandoah
>
> Aleksey Shipilev has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 10 additional commits since the last revision:
> 
>  - Comment touchup
>  - Specialize out witness-checking methods, drop acquire again
>  - Even more explanation
>  - Move the comment
>  - Also handle clearing the oops
>  - Minor touchups to the comment
>  - Merge branch 'master' into JDK-8261495-shenandoah-updaterefs-memord
>  - Use release only
>  - Do acq_rel instead
>  - 8261495: Shenandoah: reconsider update references memory ordering

Looks good to me.

-------------

Marked as reviewed by rkennke (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/2498

From zgu at openjdk.java.net  Fri Mar 19 14:04:53 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Fri, 19 Mar 2021 14:04:53 GMT
Subject: RFR: 8263861: Shenandoah: Remove unused member in
 ShenandoahGCStateResetter
Message-ID: <1AUC4wIT7iGxSv7EKZ5xf7hK9KYmVHLlvRY8RTwbGmg=.892cdb6f-42fd-4bc0-bd9e-1ac4d8a35d40@github.com>

Please review this trivial patch that removed unused member of ShenandoahGCStateResetter.

Test:
- [x] hotspot_gc_shenandoah with -XX:+ShenandoahVerify

-------------

Commit messages:
 - JDK-8263861

Changes: https://git.openjdk.java.net/jdk/pull/3088/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3088&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8263861
  Stats: 4 lines in 2 files changed: 0 ins; 2 del; 2 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3088.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3088/head:pull/3088

PR: https://git.openjdk.java.net/jdk/pull/3088

From shade at openjdk.java.net  Mon Mar 22 13:19:39 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Mon, 22 Mar 2021 13:19:39 GMT
Subject: RFR: 8263861: Shenandoah: Remove unused member in
 ShenandoahGCStateResetter
In-Reply-To: <1AUC4wIT7iGxSv7EKZ5xf7hK9KYmVHLlvRY8RTwbGmg=.892cdb6f-42fd-4bc0-bd9e-1ac4d8a35d40@github.com>
References: <1AUC4wIT7iGxSv7EKZ5xf7hK9KYmVHLlvRY8RTwbGmg=.892cdb6f-42fd-4bc0-bd9e-1ac4d8a35d40@github.com>
Message-ID: <ZIqo0ltZETdmkp_WXUqwi_xFDawLNc6DRATgL6ehHkE=.3c87dff2-3815-4021-9b6f-c4d38c5bf4ec@github.com>

On Fri, 19 Mar 2021 13:57:55 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Please review this trivial patch that removed unused member of ShenandoahGCStateResetter.
> 
> Test:
> - [x] hotspot_gc_shenandoah with -XX:+ShenandoahVerify

Looks fine to me.

-------------

Marked as reviewed by shade (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/3088

From zgu at openjdk.java.net  Mon Mar 22 13:55:58 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 22 Mar 2021 13:55:58 GMT
Subject: Integrated: 8263861: Shenandoah: Remove unused member in
 ShenandoahGCStateResetter
In-Reply-To: <1AUC4wIT7iGxSv7EKZ5xf7hK9KYmVHLlvRY8RTwbGmg=.892cdb6f-42fd-4bc0-bd9e-1ac4d8a35d40@github.com>
References: <1AUC4wIT7iGxSv7EKZ5xf7hK9KYmVHLlvRY8RTwbGmg=.892cdb6f-42fd-4bc0-bd9e-1ac4d8a35d40@github.com>
Message-ID: <XYrTOpC5B3SBUn0ymUlyGxrsuB1C8V7woxL4KmWlAhM=.9034f5de-824b-4cf6-96f3-b4b094bfbebe@github.com>

On Fri, 19 Mar 2021 13:57:55 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Please review this trivial patch that removed unused member of ShenandoahGCStateResetter.
> 
> Test:
> - [x] hotspot_gc_shenandoah with -XX:+ShenandoahVerify

This pull request has now been integrated.

Changeset: 6c2220e6
Author:    Zhengyu Gu <zgu at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/6c2220e6
Stats:     4 lines in 2 files changed: 0 ins; 2 del; 2 mod

8263861: Shenandoah: Remove unused member in ShenandoahGCStateResetter

Reviewed-by: shade

-------------

PR: https://git.openjdk.java.net/jdk/pull/3088

From zgu at openjdk.java.net  Tue Mar 23 17:06:58 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 23 Mar 2021 17:06:58 GMT
Subject: RFR: 8264052: Shenandoah: Backout 8263832
Message-ID: <J6apiMJTZ_nIg6cqCtIEiU6wxaCPUHBNw04r3sKQITA=.6c87871b-24c1-4632-8f8d-0ccfd79c0708@github.com>

TestStringDedupStress started to fail after JDK-8263832. 

JDK-8263832 is incorrect, because Threads::possibly_parallel_threads_do() only iterates over Java and VM threads, it should be backed out. The original works correctly.

-------------

Commit messages:
 - JDK-8264052

Changes: https://git.openjdk.java.net/jdk/pull/3158/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3158&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8264052
  Stats: 12 lines in 1 file changed: 4 ins; 0 del; 8 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3158.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3158/head:pull/3158

PR: https://git.openjdk.java.net/jdk/pull/3158

From shade at openjdk.java.net  Tue Mar 23 17:14:40 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 23 Mar 2021 17:14:40 GMT
Subject: RFR: 8264052: Shenandoah: Backout 8263832
In-Reply-To: <J6apiMJTZ_nIg6cqCtIEiU6wxaCPUHBNw04r3sKQITA=.6c87871b-24c1-4632-8f8d-0ccfd79c0708@github.com>
References: <J6apiMJTZ_nIg6cqCtIEiU6wxaCPUHBNw04r3sKQITA=.6c87871b-24c1-4632-8f8d-0ccfd79c0708@github.com>
Message-ID: <l5CKoE5AYPNRbjxCUvTxFQzBVmfzuRWJoOPnQAIxohw=.b33ad378-508e-49a1-bab1-5092424a8955@github.com>

On Tue, 23 Mar 2021 17:01:49 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> TestStringDedupStress started to fail after JDK-8263832. 
> 
> JDK-8263832 is incorrect, because Threads::possibly_parallel_threads_do() only iterates over Java and VM threads, it should be backed out. The original works correctly.

Looks fine.

-------------

Marked as reviewed by shade (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/3158

From zgu at openjdk.java.net  Tue Mar 23 17:50:15 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 23 Mar 2021 17:50:15 GMT
Subject: RFR: Refix ShenandoahPurgeSATBTask
Message-ID: <CfwhM_ZK93mjjWuZNRdpOsBCCmahsemo05XRsWwa9AM=.69ec4bdc-d449-489b-a725-569bc002a525@github.com>

My early fix was incomplete and incorrect.

1) ShenandoahPurgeSATBTask needs to change thread claim token for iterating threads in parallel, this was missing in early change.
2) Threads::threads_do() -> Threads::possibly_parallel_threads_do() was incorrect, the later version only iterates Java and VM threads. This part of changes are reverted.
3) A minor change: declaring _trashed_oops as volatile as it is a shared variable.

-------------

Commit messages:
 - Refix ShenandoahPurgeSATBTask

Changes: https://git.openjdk.java.net/shenandoah/pull/25/files
 Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=25&range=00
  Stats: 12 lines in 1 file changed: 6 ins; 0 del; 6 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/25.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/25/head:pull/25

PR: https://git.openjdk.java.net/shenandoah/pull/25

From github.com+71722661+earthling-amzn at openjdk.java.net  Tue Mar 23 18:23:03 2021
From: github.com+71722661+earthling-amzn at openjdk.java.net (earthling-amzn)
Date: Tue, 23 Mar 2021 18:23:03 GMT
Subject: RFR: Refix ShenandoahPurgeSATBTask
In-Reply-To: <CfwhM_ZK93mjjWuZNRdpOsBCCmahsemo05XRsWwa9AM=.69ec4bdc-d449-489b-a725-569bc002a525@github.com>
References: <CfwhM_ZK93mjjWuZNRdpOsBCCmahsemo05XRsWwa9AM=.69ec4bdc-d449-489b-a725-569bc002a525@github.com>
Message-ID: <wDsmCUqHdidwuVQZ7dkeHhBZNFkgAvAJ7f9NrenSMHE=.5135de25-0acc-46e3-8cd6-81b9c80203eb@github.com>

On Tue, 23 Mar 2021 17:45:17 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> My early fix was incomplete and incorrect.
> 
> 1) ShenandoahPurgeSATBTask needs to change thread claim token for iterating threads in parallel, this was missing in early change.
> 2) Threads::threads_do() -> Threads::possibly_parallel_threads_do() was incorrect, the later version only iterates Java and VM threads. This part of changes are reverted.
> 3) A minor change: declaring _trashed_oops as volatile as it is a shared variable.

Thanks again for setting this straight.

-------------

Marked as reviewed by earthling-amzn at github.com (no known OpenJDK username).

PR: https://git.openjdk.java.net/shenandoah/pull/25

From zgu at openjdk.java.net  Tue Mar 23 19:20:59 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 23 Mar 2021 19:20:59 GMT
Subject: RFR: Refix ShenandoahPurgeSATBTask
In-Reply-To: <wDsmCUqHdidwuVQZ7dkeHhBZNFkgAvAJ7f9NrenSMHE=.5135de25-0acc-46e3-8cd6-81b9c80203eb@github.com>
References: <CfwhM_ZK93mjjWuZNRdpOsBCCmahsemo05XRsWwa9AM=.69ec4bdc-d449-489b-a725-569bc002a525@github.com>
 <wDsmCUqHdidwuVQZ7dkeHhBZNFkgAvAJ7f9NrenSMHE=.5135de25-0acc-46e3-8cd6-81b9c80203eb@github.com>
Message-ID: <IO94NU252dcTJhXUKhYHr2MoHV5i82mQ2kuOSDWi5Dk=.46701d26-75b6-4c12-8976-271b8ea5e482@github.com>

On Tue, 23 Mar 2021 18:20:16 GMT, earthling-amzn <github.com+71722661+earthling-amzn at openjdk.org> wrote:

> Thanks again for setting this straight.

@earthling-amzn Thanks for reviewing and sorry for the noise.

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/25

From gnu.andrew at redhat.com  Wed Mar 24 00:54:46 2021
From: gnu.andrew at redhat.com (Andrew Hughes)
Date: Wed, 24 Mar 2021 00:54:46 +0000
Subject: [RFR] [8u] 8u292-b06 Upstream Sync
Message-ID: <20210324005446.GA1204687@rincewind>

Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/

Merge changesets:
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/corba/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/jaxp/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/jaxws/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/jdk/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/hotspot/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/langtools/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/nashorn/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/root/merge.changeset

Changes in aarch64-shenandoah-jdk8u292-b06:
  - JDK-8259048: (tz) Upgrade time-zone data to tzdata2020f

Main issues of note:
None, clean merge (no HotSpot changes).

diffstat for root
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for corba
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jaxp
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jaxws
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for langtools
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for nashorn
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jdk
 b/.hgtags                                       |    1 
 b/make/data/tzdata/VERSION                      |    2 
 b/make/data/tzdata/africa                       |  236 +++++++++++++++++---
 b/make/data/tzdata/asia                         |  274 ++++++++++++++++++------
 b/make/data/tzdata/australasia                  |  106 +++++++--
 b/make/data/tzdata/backward                     |    1 
 b/make/data/tzdata/etcetera                     |   11 
 b/make/data/tzdata/europe                       |   16 +
 b/make/data/tzdata/leapseconds                  |    4 
 b/make/data/tzdata/northamerica                 |  202 ++++++++++++++++-
 b/make/data/tzdata/zone.tab                     |    9 
 b/test/sun/util/calendar/zi/tzdata/VERSION      |    2 
 b/test/sun/util/calendar/zi/tzdata/africa       |  236 +++++++++++++++++---
 b/test/sun/util/calendar/zi/tzdata/asia         |  274 ++++++++++++++++++------
 b/test/sun/util/calendar/zi/tzdata/australasia  |  106 +++++++--
 b/test/sun/util/calendar/zi/tzdata/backward     |    1 
 b/test/sun/util/calendar/zi/tzdata/etcetera     |   11 
 b/test/sun/util/calendar/zi/tzdata/europe       |   16 +
 b/test/sun/util/calendar/zi/tzdata/leapseconds  |    4 
 b/test/sun/util/calendar/zi/tzdata/northamerica |  202 ++++++++++++++++-
 b/test/sun/util/calendar/zi/tzdata/zone.tab     |    9 
 21 files changed, 1421 insertions(+), 302 deletions(-)

diffstat for hotspot
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

Successfully built on x86, x86_64, s390 (Zero), s390x (Zero), ppc (Zero),
ppc64, ppc64le, aarch32 (Zero) & aarch64.

Ok to push?

Thanks,
-- 
Andrew :)

Senior Free Java Software Engineer
OpenJDK Package Owner
Red Hat, Inc. (http://www.redhat.com)

PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04  C5A0 CFDA 0F9B 3596 4222

From shade at redhat.com  Wed Mar 24 07:17:27 2021
From: shade at redhat.com (Aleksey Shipilev)
Date: Wed, 24 Mar 2021 08:17:27 +0100
Subject: [RFR] [8u] 8u292-b06 Upstream Sync
In-Reply-To: <20210324005446.GA1204687@rincewind>
References: <20210324005446.GA1204687@rincewind>
Message-ID: <396e09d4-9e4d-92e4-7c9c-ed72c9c78ff1@redhat.com>

On 3/24/21 1:54 AM, Andrew Hughes wrote:
> Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/
> 
> Merge changesets:
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/corba/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/jaxp/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/jaxws/merge.changeset

Look trivially good.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/jdk/merge.changeset

Looks good.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/hotspot/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/langtools/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/nashorn/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/root/merge.changeset

Look trivially good.

> Ok to push?

Yes.

-- 
Thanks,
-Aleksey


From zgu at openjdk.java.net  Wed Mar 24 12:18:40 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Wed, 24 Mar 2021 12:18:40 GMT
Subject: Integrated: 8264052: Shenandoah: Backout 8263832
In-Reply-To: <J6apiMJTZ_nIg6cqCtIEiU6wxaCPUHBNw04r3sKQITA=.6c87871b-24c1-4632-8f8d-0ccfd79c0708@github.com>
References: <J6apiMJTZ_nIg6cqCtIEiU6wxaCPUHBNw04r3sKQITA=.6c87871b-24c1-4632-8f8d-0ccfd79c0708@github.com>
Message-ID: <s4tTm7GbICwUhIX_e6oYlo7vMy9EpsJNsjJUaTNM21o=.c7465fd4-3ef9-44f3-a58f-93ba63dbb596@github.com>

On Tue, 23 Mar 2021 17:01:49 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> TestStringDedupStress started to fail after JDK-8263832. 
> 
> JDK-8263832 is incorrect, because Threads::possibly_parallel_threads_do() only iterates over Java and VM threads, it should be backed out. The original works correctly.

This pull request has now been integrated.

Changeset: 9ee0b9a1
Author:    Zhengyu Gu <zgu at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/9ee0b9a1
Stats:     12 lines in 1 file changed: 4 ins; 0 del; 8 mod

8264052: Shenandoah: Backout 8263832

Reviewed-by: shade

-------------

PR: https://git.openjdk.java.net/jdk/pull/3158

From gnu.andrew at redhat.com  Wed Mar 24 15:09:34 2021
From: gnu.andrew at redhat.com (Andrew Hughes)
Date: Wed, 24 Mar 2021 15:09:34 +0000
Subject: [RFR] [8u] 8u292-b06 Upstream Sync
In-Reply-To: <396e09d4-9e4d-92e4-7c9c-ed72c9c78ff1@redhat.com>
References: <20210324005446.GA1204687@rincewind>
 <396e09d4-9e4d-92e4-7c9c-ed72c9c78ff1@redhat.com>
Message-ID: <20210324150934.GA38165@rincewind>

On 08:17 Wed 24 Mar     , Aleksey Shipilev wrote:
> On 3/24/21 1:54 AM, Andrew Hughes wrote:
> > Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/
> > 
> > Merge changesets:
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/corba/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/jaxp/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/jaxws/merge.changeset
> 
> Look trivially good.
> 
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/jdk/merge.changeset
> 
> Looks good.
> 
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/hotspot/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/langtools/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/nashorn/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b06/root/merge.changeset
> 
> Look trivially good.
> 
> > Ok to push?
> 
> Yes.
> 
> -- 
> Thanks,
> -Aleksey
> 

Pushed.

Thanks,
-- 
Andrew :)

Senior Free Java Software Engineer
OpenJDK Package Owner
Red Hat, Inc. (http://www.redhat.com)

PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04  C5A0 CFDA 0F9B 3596 4222

From roland at openjdk.java.net  Thu Mar 25 13:01:41 2021
From: roland at openjdk.java.net (Roland Westrelin)
Date: Thu, 25 Mar 2021 13:01:41 GMT
Subject: RFR: 8263376: CTW (Shenandoah): assert(mems <= 1) failed: No node
 right after call if multiple mem projections
In-Reply-To: <d9ZCPTRo-b7lYXvGIBBdZ7TE4dl-rI076HyZZeK4gDA=.dba7433d-033b-4e16-b984-9bde627bcfe4@github.com>
References: <d9ZCPTRo-b7lYXvGIBBdZ7TE4dl-rI076HyZZeK4gDA=.dba7433d-033b-4e16-b984-9bde627bcfe4@github.com>
Message-ID: <8_3-RdfOS6GFY_ntMqMgj7rI3BJmNFJ9ibafoTYx1ck=.d4cc97c2-1788-4e84-a140-f441735e9873@github.com>

On Mon, 15 Mar 2021 10:06:20 GMT, Roland Westrelin <roland at openjdk.org> wrote:

> This is another case of anti-dependence analysis being too conservative.
> 
> In TestBadRawMemoryAfterCall.test2(), test
> 
> if (i == 42) {
> 
> is split thru the Phi that merges values from the fallthru and
> exception paths. As a consequence, control flow at the call is
> roughly:
> a.m() call
> |         \       
> fallthru  exception
> |             |
> if (i == 42)  |
> |      \      |
> |       Region1
> \        /
>   Region2
> When anti-dependence analysis runs for the load after the call, it
> starts from the memory state out of the call on the fallthru path. One
> use is a memory Phi at Region2 (say Phi2). Another path leads to
> Region1 at, say, Phi1.
> 
> When anti-dependence analysis then goes over Phis, it processes Phi2,
> goes over the its inputs and finds that 2 are reachable: the one that
> has a direct edge to the memory state on the fallthru path and the one
> from Phi1. As a consequence, control for the load is set to be right
> after the call, which is too conservative.
> 
> When following the memory edges, the code stops at Phi1. It doesn't
> process uses of Phi1. Any anti-dependence that's needed between the
> load and Phi1 is then taken into account when Phis are processed. When
> inputs to Phi2 are processed, considering the Phi2->Phi1 is too
> conservative. As mentioned above, anti-dependences for Phi1 are taken
> into account separately. I think it's true for all Phi->Phi edges that
> they can be safely ignored. That's what I propose as a fix.

Anyone for a review of this relatively simple change? Change is in shared code.

-------------

PR: https://git.openjdk.java.net/jdk/pull/3006

From kvn at openjdk.java.net  Thu Mar 25 17:18:27 2021
From: kvn at openjdk.java.net (Vladimir Kozlov)
Date: Thu, 25 Mar 2021 17:18:27 GMT
Subject: RFR: 8263376: CTW (Shenandoah): assert(mems <= 1) failed: No node
 right after call if multiple mem projections
In-Reply-To: <d9ZCPTRo-b7lYXvGIBBdZ7TE4dl-rI076HyZZeK4gDA=.dba7433d-033b-4e16-b984-9bde627bcfe4@github.com>
References: <d9ZCPTRo-b7lYXvGIBBdZ7TE4dl-rI076HyZZeK4gDA=.dba7433d-033b-4e16-b984-9bde627bcfe4@github.com>
Message-ID: <2NEotCAN_F4Dk9kjnim9pmP37zcD_Y3iZ7PcZ5WvtZ4=.5034e774-2ea4-49eb-bd4d-3bb298460d09@github.com>

On Mon, 15 Mar 2021 10:06:20 GMT, Roland Westrelin <roland at openjdk.org> wrote:

> This is another case of anti-dependence analysis being too conservative.
> 
> In TestBadRawMemoryAfterCall.test2(), test
> 
> if (i == 42) {
> 
> is split thru the Phi that merges values from the fallthru and
> exception paths. As a consequence, control flow at the call is
> roughly:
> a.m() call
> |         \       
> fallthru  exception
> |             |
> if (i == 42)  |
> |      \      |
> |       Region1
> \        /
>   Region2
> When anti-dependence analysis runs for the load after the call, it
> starts from the memory state out of the call on the fallthru path. One
> use is a memory Phi at Region2 (say Phi2). Another path leads to
> Region1 at, say, Phi1.
> 
> When anti-dependence analysis then goes over Phis, it processes Phi2,
> goes over the its inputs and finds that 2 are reachable: the one that
> has a direct edge to the memory state on the fallthru path and the one
> from Phi1. As a consequence, control for the load is set to be right
> after the call, which is too conservative.
> 
> When following the memory edges, the code stops at Phi1. It doesn't
> process uses of Phi1. Any anti-dependence that's needed between the
> load and Phi1 is then taken into account when Phis are processed. When
> inputs to Phi2 are processed, considering the Phi2->Phi1 is too
> conservative. As mentioned above, anti-dependences for Phi1 are taken
> into account separately. I think it's true for all Phi->Phi edges that
> they can be safely ignored. That's what I propose as a fix.

Okay.

-------------

Marked as reviewed by kvn (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/3006

From chagedorn at openjdk.java.net  Fri Mar 26 10:10:26 2021
From: chagedorn at openjdk.java.net (Christian Hagedorn)
Date: Fri, 26 Mar 2021 10:10:26 GMT
Subject: RFR: 8263376: CTW (Shenandoah): assert(mems <= 1) failed: No node
 right after call if multiple mem projections
In-Reply-To: <d9ZCPTRo-b7lYXvGIBBdZ7TE4dl-rI076HyZZeK4gDA=.dba7433d-033b-4e16-b984-9bde627bcfe4@github.com>
References: <d9ZCPTRo-b7lYXvGIBBdZ7TE4dl-rI076HyZZeK4gDA=.dba7433d-033b-4e16-b984-9bde627bcfe4@github.com>
Message-ID: <7aqHNhZ88aUOC1L-MOx3l5mMRjhoNDgLqX7XuEc6z1k=.b382af57-4f5d-44d5-9372-8bd2c987a907@github.com>

On Mon, 15 Mar 2021 10:06:20 GMT, Roland Westrelin <roland at openjdk.org> wrote:

> This is another case of anti-dependence analysis being too conservative.
> 
> In TestBadRawMemoryAfterCall.test2(), test
> 
> if (i == 42) {
> 
> is split thru the Phi that merges values from the fallthru and
> exception paths. As a consequence, control flow at the call is
> roughly:
> a.m() call
> |         \       
> fallthru  exception
> |             |
> if (i == 42)  |
> |      \      |
> |       Region1
> \        /
>   Region2
> When anti-dependence analysis runs for the load after the call, it
> starts from the memory state out of the call on the fallthru path. One
> use is a memory Phi at Region2 (say Phi2). Another path leads to
> Region1 at, say, Phi1.
> 
> When anti-dependence analysis then goes over Phis, it processes Phi2,
> goes over the its inputs and finds that 2 are reachable: the one that
> has a direct edge to the memory state on the fallthru path and the one
> from Phi1. As a consequence, control for the load is set to be right
> after the call, which is too conservative.
> 
> When following the memory edges, the code stops at Phi1. It doesn't
> process uses of Phi1. Any anti-dependence that's needed between the
> load and Phi1 is then taken into account when Phis are processed. When
> inputs to Phi2 are processed, considering the Phi2->Phi1 is too
> conservative. As mentioned above, anti-dependences for Phi1 are taken
> into account separately. I think it's true for all Phi->Phi edges that
> they can be safely ignored. That's what I propose as a fix.

You should also add the bug number to the test.

Otherwise, looks good to me.

-------------

Marked as reviewed by chagedorn (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/3006

From roland at openjdk.java.net  Fri Mar 26 16:52:48 2021
From: roland at openjdk.java.net (Roland Westrelin)
Date: Fri, 26 Mar 2021 16:52:48 GMT
Subject: RFR: 8263376: CTW (Shenandoah): assert(mems <= 1) failed: No node
 right after call if multiple mem projections [v2]
In-Reply-To: <d9ZCPTRo-b7lYXvGIBBdZ7TE4dl-rI076HyZZeK4gDA=.dba7433d-033b-4e16-b984-9bde627bcfe4@github.com>
References: <d9ZCPTRo-b7lYXvGIBBdZ7TE4dl-rI076HyZZeK4gDA=.dba7433d-033b-4e16-b984-9bde627bcfe4@github.com>
Message-ID: <vN1ooc4WzB4soI1ievuCulPNyRX4TqMlnlzHqf-XL84=.e109d418-1f04-40fc-887c-d6445661129b@github.com>

> This is another case of anti-dependence analysis being too conservative.
> 
> In TestBadRawMemoryAfterCall.test2(), test
> 
> if (i == 42) {
> 
> is split thru the Phi that merges values from the fallthru and
> exception paths. As a consequence, control flow at the call is
> roughly:
> a.m() call
> |         \       
> fallthru  exception
> |             |
> if (i == 42)  |
> |      \      |
> |       Region1
> \        /
>   Region2
> When anti-dependence analysis runs for the load after the call, it
> starts from the memory state out of the call on the fallthru path. One
> use is a memory Phi at Region2 (say Phi2). Another path leads to
> Region1 at, say, Phi1.
> 
> When anti-dependence analysis then goes over Phis, it processes Phi2,
> goes over the its inputs and finds that 2 are reachable: the one that
> has a direct edge to the memory state on the fallthru path and the one
> from Phi1. As a consequence, control for the load is set to be right
> after the call, which is too conservative.
> 
> When following the memory edges, the code stops at Phi1. It doesn't
> process uses of Phi1. Any anti-dependence that's needed between the
> load and Phi1 is then taken into account when Phis are processed. When
> inputs to Phi2 are processed, considering the Phi2->Phi1 is too
> conservative. As mentioned above, anti-dependences for Phi1 are taken
> into account separately. I think it's true for all Phi->Phi edges that
> they can be safely ignored. That's what I propose as a fix.

Roland Westrelin has updated the pull request incrementally with one additional commit since the last revision:

  bug number in test

-------------

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/3006/files
  - new: https://git.openjdk.java.net/jdk/pull/3006/files/a47fa7b7..9529fa73

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=3006&range=01
 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=3006&range=00-01

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3006.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3006/head:pull/3006

PR: https://git.openjdk.java.net/jdk/pull/3006

From roland at openjdk.java.net  Fri Mar 26 16:52:48 2021
From: roland at openjdk.java.net (Roland Westrelin)
Date: Fri, 26 Mar 2021 16:52:48 GMT
Subject: RFR: 8263376: CTW (Shenandoah): assert(mems <= 1) failed: No node
 right after call if multiple mem projections [v2]
In-Reply-To: <7aqHNhZ88aUOC1L-MOx3l5mMRjhoNDgLqX7XuEc6z1k=.b382af57-4f5d-44d5-9372-8bd2c987a907@github.com>
References: <d9ZCPTRo-b7lYXvGIBBdZ7TE4dl-rI076HyZZeK4gDA=.dba7433d-033b-4e16-b984-9bde627bcfe4@github.com>
 <7aqHNhZ88aUOC1L-MOx3l5mMRjhoNDgLqX7XuEc6z1k=.b382af57-4f5d-44d5-9372-8bd2c987a907@github.com>
Message-ID: <avaKrALa3u7w1DmSNaxmTCejqWP1Q-LyaAc1_8B2BNY=.5cbc4ba2-c439-4f7e-9979-4faff6d1a49f@github.com>

On Fri, 26 Mar 2021 10:07:43 GMT, Christian Hagedorn <chagedorn at openjdk.org> wrote:

> You should also add the bug number to the test.
 
done.

> Otherwise, looks good to me.

Thanks for the review.

-------------

PR: https://git.openjdk.java.net/jdk/pull/3006

From roland at openjdk.java.net  Fri Mar 26 16:52:50 2021
From: roland at openjdk.java.net (Roland Westrelin)
Date: Fri, 26 Mar 2021 16:52:50 GMT
Subject: Integrated: 8263376: CTW (Shenandoah): assert(mems <= 1) failed: No
 node right after call if multiple mem projections
In-Reply-To: <d9ZCPTRo-b7lYXvGIBBdZ7TE4dl-rI076HyZZeK4gDA=.dba7433d-033b-4e16-b984-9bde627bcfe4@github.com>
References: <d9ZCPTRo-b7lYXvGIBBdZ7TE4dl-rI076HyZZeK4gDA=.dba7433d-033b-4e16-b984-9bde627bcfe4@github.com>
Message-ID: <MO0WReHhM0MOa7K_-qpLzkL9giVHGKjA3580mOl0avk=.95d8a589-6dd7-4bdb-a6bb-aaa29224a291@github.com>

On Mon, 15 Mar 2021 10:06:20 GMT, Roland Westrelin <roland at openjdk.org> wrote:

> This is another case of anti-dependence analysis being too conservative.
> 
> In TestBadRawMemoryAfterCall.test2(), test
> 
> if (i == 42) {
> 
> is split thru the Phi that merges values from the fallthru and
> exception paths. As a consequence, control flow at the call is
> roughly:
> a.m() call
> |         \       
> fallthru  exception
> |             |
> if (i == 42)  |
> |      \      |
> |       Region1
> \        /
>   Region2
> When anti-dependence analysis runs for the load after the call, it
> starts from the memory state out of the call on the fallthru path. One
> use is a memory Phi at Region2 (say Phi2). Another path leads to
> Region1 at, say, Phi1.
> 
> When anti-dependence analysis then goes over Phis, it processes Phi2,
> goes over the its inputs and finds that 2 are reachable: the one that
> has a direct edge to the memory state on the fallthru path and the one
> from Phi1. As a consequence, control for the load is set to be right
> after the call, which is too conservative.
> 
> When following the memory edges, the code stops at Phi1. It doesn't
> process uses of Phi1. Any anti-dependence that's needed between the
> load and Phi1 is then taken into account when Phis are processed. When
> inputs to Phi2 are processed, considering the Phi2->Phi1 is too
> conservative. As mentioned above, anti-dependences for Phi1 are taken
> into account separately. I think it's true for all Phi->Phi edges that
> they can be safely ignored. That's what I propose as a fix.

This pull request has now been integrated.

Changeset: 33c94ffc
Author:    Roland Westrelin <roland at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/33c94ffc
Stats:     30 lines in 2 files changed: 24 ins; 0 del; 6 mod

8263376: CTW (Shenandoah): assert(mems <= 1) failed: No node right after call if multiple mem projections

Reviewed-by: kvn, chagedorn

-------------

PR: https://git.openjdk.java.net/jdk/pull/3006

From roland at openjdk.java.net  Fri Mar 26 16:52:49 2021
From: roland at openjdk.java.net (Roland Westrelin)
Date: Fri, 26 Mar 2021 16:52:49 GMT
Subject: RFR: 8263376: CTW (Shenandoah): assert(mems <= 1) failed: No node
 right after call if multiple mem projections [v2]
In-Reply-To: <2NEotCAN_F4Dk9kjnim9pmP37zcD_Y3iZ7PcZ5WvtZ4=.5034e774-2ea4-49eb-bd4d-3bb298460d09@github.com>
References: <d9ZCPTRo-b7lYXvGIBBdZ7TE4dl-rI076HyZZeK4gDA=.dba7433d-033b-4e16-b984-9bde627bcfe4@github.com>
 <2NEotCAN_F4Dk9kjnim9pmP37zcD_Y3iZ7PcZ5WvtZ4=.5034e774-2ea4-49eb-bd4d-3bb298460d09@github.com>
Message-ID: <B3KH1OVjarW_y8TKurRcgHijKeP8ScvXVfESNt5KPFg=.512e8f92-c4e3-4ef8-b5f8-ce1f07cafd57@github.com>

On Thu, 25 Mar 2021 17:15:13 GMT, Vladimir Kozlov <kvn at openjdk.org> wrote:

> Okay.

thanks for the review

-------------

PR: https://git.openjdk.java.net/jdk/pull/3006

From gnu.andrew at redhat.com  Mon Mar 29 03:28:21 2021
From: gnu.andrew at redhat.com (Andrew Hughes)
Date: Mon, 29 Mar 2021 04:28:21 +0100
Subject: [RFR] [8u] 8u292-b07 Upstream Sync
Message-ID: <20210329032821.GB169175@rincewind>

Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/

Merge changesets:
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/corba/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/jaxp/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/jaxws/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/jdk/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/hotspot/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/langtools/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/nashorn/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/root/merge.changeset

Changes in aarch64-shenandoah-jdk8u292-b07:
  - JDK-8263008: AARCH64: Add debug info for libsaproc.so

Main issues of note:
None, clean merge.

diffstat for root
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for corba
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jaxp
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jaxws
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for langtools
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for nashorn
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jdk
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for hotspot
 b/.hgtags                        |    1 +
 b/make/linux/makefiles/defs.make |   12 +++++++-----
 2 files changed, 8 insertions(+), 5 deletions(-)

Successfully built on x86, x86_64, s390 (Zero), s390x (Zero), ppc (Zero),
ppc64, ppc64le, aarch32 (Zero) & aarch64.

Ok to push?

Thanks,
-- 
Andrew :)

Senior Free Java Software Engineer
OpenJDK Package Owner
Red Hat, Inc. (http://www.redhat.com)

PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04  C5A0 CFDA 0F9B 3596 4222

From shade at redhat.com  Mon Mar 29 08:03:13 2021
From: shade at redhat.com (Aleksey Shipilev)
Date: Mon, 29 Mar 2021 10:03:13 +0200
Subject: [RFR] [8u] 8u292-b07 Upstream Sync
In-Reply-To: <20210329032821.GB169175@rincewind>
References: <20210329032821.GB169175@rincewind>
Message-ID: <dbb07e51-87d4-2a66-fb4e-1670fcbd266e@redhat.com>

On 3/29/21 5:28 AM, Andrew Hughes wrote:
> Merge changesets:
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/corba/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/jaxp/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/jaxws/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/jdk/merge.changeset

Look trivially fine.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/hotspot/merge.changeset

Looks fine.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/langtools/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/nashorn/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/root/merge.changeset

Look trivially fine.

> Ok to push?

Yes.


-- 
Thanks,
-Aleksey


From gnu.andrew at redhat.com  Mon Mar 29 12:27:25 2021
From: gnu.andrew at redhat.com (Andrew Hughes)
Date: Mon, 29 Mar 2021 13:27:25 +0100
Subject: [RFR] [8u] 8u292-b07 Upstream Sync
In-Reply-To: <dbb07e51-87d4-2a66-fb4e-1670fcbd266e@redhat.com>
References: <20210329032821.GB169175@rincewind>
 <dbb07e51-87d4-2a66-fb4e-1670fcbd266e@redhat.com>
Message-ID: <20210329122725.GA175114@rincewind>

On 10:03 Mon 29 Mar     , Aleksey Shipilev wrote:
> On 3/29/21 5:28 AM, Andrew Hughes wrote:
> > Merge changesets:
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/corba/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/jaxp/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/jaxws/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/jdk/merge.changeset
> 
> Look trivially fine.
> 
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/hotspot/merge.changeset
> 
> Looks fine.
> 
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/langtools/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/nashorn/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b07/root/merge.changeset
> 
> Look trivially fine.
> 
> > Ok to push?
> 
> Yes.
> 
> 
> -- 
> Thanks,
> -Aleksey
> 

Thanks. Pushed.
-- 
Andrew :)

Senior Free Java Software Engineer
OpenJDK Package Owner
Red Hat, Inc. (http://www.redhat.com)

PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04  C5A0 CFDA 0F9B 3596 4222

From gnu.andrew at redhat.com  Mon Mar 29 14:32:01 2021
From: gnu.andrew at redhat.com (Andrew Hughes)
Date: Mon, 29 Mar 2021 15:32:01 +0100
Subject: [RFR] [8u] 8u292-b08 Upstream Sync
Message-ID: <20210329143201.GB175114@rincewind>

Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/

Merge changesets:
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/corba/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/jaxp/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/jaxws/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/jdk/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/hotspot/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/langtools/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/nashorn/merge.changeset
http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/root/merge.changeset

Changes in aarch64-shenandoah-jdk8u292-b08:
  - JDK-8191915: JCK tests produce incorrect results with C2
  - JDK-8256421: Add 2 HARICA roots to cacerts truststore
  - JDK-8260356: (tz) Upgrade time-zone data to tzdata2021a

Main issues of note:
None, clean merge.

diffstat for root
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for corba
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jaxp
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jaxws
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for langtools
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for nashorn
 b/.hgtags |    1 +
 1 file changed, 1 insertion(+)

diffstat for jdk
 b/.hgtags                                                                              |    1 
 b/make/data/cacerts/haricaeccrootca2015                                                |   24 
 b/make/data/cacerts/haricarootca2015                                                   |   42 +
 b/make/data/tzdata/VERSION                                                             |    2 
 b/make/data/tzdata/africa                                                              |    8 
 b/make/data/tzdata/leapseconds                                                         |    8 
 b/test/security/infra/java/security/cert/CertPathValidator/certification/HaricaCA.java |  320 ++++++++++
 b/test/sun/security/lib/cacerts/VerifyCACerts.java                                     |   10 
 b/test/sun/util/calendar/zi/tzdata/VERSION                                             |    2 
 b/test/sun/util/calendar/zi/tzdata/africa                                              |    8 
 b/test/sun/util/calendar/zi/tzdata/leapseconds                                         |    8 
 11 files changed, 418 insertions(+), 15 deletions(-)

diffstat for hotspot
 b/.hgtags                                                     |    1 
 b/src/share/vm/opto/mathexactnode.cpp                         |   40 ++++--
 b/src/share/vm/opto/mathexactnode.hpp                         |    6 -
 b/test/compiler/intrinsics/mathexact/LongMulOverflowTest.java |   59 ++++++++++
 4 files changed, 89 insertions(+), 17 deletions(-)

Successfully built on x86, x86_64, s390 (Zero), s390x (Zero), ppc (Zero),
ppc64, ppc64le, aarch32 (Zero) & aarch64.

Ok to push?

Thanks,
-- 
Andrew :)

Senior Free Java Software Engineer
OpenJDK Package Owner
Red Hat, Inc. (http://www.redhat.com)

PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04  C5A0 CFDA 0F9B 3596 4222

From shade at redhat.com  Mon Mar 29 16:40:12 2021
From: shade at redhat.com (Aleksey Shipilev)
Date: Mon, 29 Mar 2021 18:40:12 +0200
Subject: [RFR] [8u] 8u292-b08 Upstream Sync
In-Reply-To: <20210329143201.GB175114@rincewind>
References: <20210329143201.GB175114@rincewind>
Message-ID: <c981089a-7f0d-ddbf-8da3-11b2060df6ce@redhat.com>

On 3/29/21 4:32 PM, Andrew Hughes wrote:
> Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/
> 
> Merge changesets:
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/corba/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/jaxp/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/jaxws/merge.changeset

Look trivially good.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/jdk/merge.changeset

Looks good.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/hotspot/merge.changeset

Looks good.

> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/langtools/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/nashorn/merge.changeset
> http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/root/merge.changeset

Look trivially good.

> Ok to push?

Yes.


-- 
Thanks,
-Aleksey


From zgu at openjdk.java.net  Mon Mar 29 17:35:50 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 29 Mar 2021 17:35:50 GMT
Subject: RFR: 8264374: Shenandoah: Remove leftover parallel reference
 processing argument
Message-ID: <ZeyXmsf1l_ced8wAmDf69JonN7fWdvRmKjqLxNsgKys=.bf19b1e8-f8e8-4175-9b51-2b43b706be22@github.com>

Shenandoah no longer uses parallel reference processor,  no reason to enable it.

Test:
- [x] hotspot_gc_shenandoah

-------------

Commit messages:
 - JDK-8264374

Changes: https://git.openjdk.java.net/jdk/pull/3246/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3246&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8264374
  Stats: 4 lines in 1 file changed: 0 ins; 4 del; 0 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3246.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3246/head:pull/3246

PR: https://git.openjdk.java.net/jdk/pull/3246

From zgu at openjdk.java.net  Mon Mar 29 18:10:49 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Mon, 29 Mar 2021 18:10:49 GMT
Subject: RFR: 8264279: Shenandoah: Missing handshake after JDK-8263427
Message-ID: <cnNWvw0HbfRLv7yCk1JYFOwsqGFaRt4_9_2_A1jDSZk=.8eae745e-eac4-4231-9b77-9e50bc003ab2@github.com>

JDK-8263427 mistakenly removed handshake after concurrent weak roots, which is needed to flush out stale oops before we can safely recycle trash regions. Otherwise, those stale oops are suddenly resurrected, because they are above TAMS after regions are recycled.

Test:
- [x] hotspot_gc_shenandoah

-------------

Commit messages:
 - JDK-8264279

Changes: https://git.openjdk.java.net/jdk/pull/3248/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=3248&range=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8264279
  Stats: 29 lines in 5 files changed: 20 ins; 8 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3248.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3248/head:pull/3248

PR: https://git.openjdk.java.net/jdk/pull/3248

From shade at openjdk.java.net  Tue Mar 30 07:03:36 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 30 Mar 2021 07:03:36 GMT
Subject: RFR: 8264374: Shenandoah: Remove leftover parallel reference
 processing argument
In-Reply-To: <ZeyXmsf1l_ced8wAmDf69JonN7fWdvRmKjqLxNsgKys=.bf19b1e8-f8e8-4175-9b51-2b43b706be22@github.com>
References: <ZeyXmsf1l_ced8wAmDf69JonN7fWdvRmKjqLxNsgKys=.bf19b1e8-f8e8-4175-9b51-2b43b706be22@github.com>
Message-ID: <vuMzFm6VgOMxvsDTwdcrRDFWmSTgJB9weo9q7bG_X5o=.5211d80b-47fc-4b53-8005-65c90ca44497@github.com>

On Mon, 29 Mar 2021 17:27:36 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Shenandoah no longer uses parallel reference processor,  no reason to enable it.
> 
> Test:
> - [x] hotspot_gc_shenandoah

Looks fine and trivial.

-------------

Marked as reviewed by shade (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/3246

From shade at openjdk.java.net  Tue Mar 30 07:08:45 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Tue, 30 Mar 2021 07:08:45 GMT
Subject: RFR: 8264279: Shenandoah: Missing handshake after JDK-8263427
In-Reply-To: <cnNWvw0HbfRLv7yCk1JYFOwsqGFaRt4_9_2_A1jDSZk=.8eae745e-eac4-4231-9b77-9e50bc003ab2@github.com>
References: <cnNWvw0HbfRLv7yCk1JYFOwsqGFaRt4_9_2_A1jDSZk=.8eae745e-eac4-4231-9b77-9e50bc003ab2@github.com>
Message-ID: <t6rHvfBAu2lOu-3rbSdd3jJOsGJfhs2SJql_5gbNSlU=.9186f763-8e71-43bf-9d36-3ab0fc75d51c@github.com>

On Mon, 29 Mar 2021 18:03:37 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> JDK-8263427 mistakenly removed handshake after concurrent weak roots, which is needed to flush out stale oops before we can safely recycle trash regions. Otherwise, those stale oops are suddenly resurrected, because they are above TAMS after regions are recycled.
> 
> Test:
> - [x] hotspot_gc_shenandoah

D'oh. Looks good.

-------------

Marked as reviewed by shade (Reviewer).

PR: https://git.openjdk.java.net/jdk/pull/3248

From zgu at openjdk.java.net  Tue Mar 30 11:46:35 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 30 Mar 2021 11:46:35 GMT
Subject: Integrated: 8264279: Shenandoah: Missing handshake after JDK-8263427
In-Reply-To: <cnNWvw0HbfRLv7yCk1JYFOwsqGFaRt4_9_2_A1jDSZk=.8eae745e-eac4-4231-9b77-9e50bc003ab2@github.com>
References: <cnNWvw0HbfRLv7yCk1JYFOwsqGFaRt4_9_2_A1jDSZk=.8eae745e-eac4-4231-9b77-9e50bc003ab2@github.com>
Message-ID: <5HFsf0Z6UmFEI13c-Sem6uF_lPzDMt3peZm7ODQLnqY=.689db82f-9fec-4cf2-bafc-ea0e481ce1ee@github.com>

On Mon, 29 Mar 2021 18:03:37 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> JDK-8263427 mistakenly removed handshake after concurrent weak roots, which is needed to flush out stale oops before we can safely recycle trash regions. Otherwise, those stale oops are suddenly resurrected, because they are above TAMS after regions are recycled.
> 
> Test:
> - [x] hotspot_gc_shenandoah

This pull request has now been integrated.

Changeset: ee5e00b0
Author:    Zhengyu Gu <zgu at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/ee5e00b0
Stats:     29 lines in 5 files changed: 20 ins; 8 del; 1 mod

8264279: Shenandoah: Missing handshake after JDK-8263427

Reviewed-by: shade

-------------

PR: https://git.openjdk.java.net/jdk/pull/3248

From zgu at openjdk.java.net  Tue Mar 30 11:46:36 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Tue, 30 Mar 2021 11:46:36 GMT
Subject: Integrated: 8264374: Shenandoah: Remove leftover parallel reference
 processing argument
In-Reply-To: <ZeyXmsf1l_ced8wAmDf69JonN7fWdvRmKjqLxNsgKys=.bf19b1e8-f8e8-4175-9b51-2b43b706be22@github.com>
References: <ZeyXmsf1l_ced8wAmDf69JonN7fWdvRmKjqLxNsgKys=.bf19b1e8-f8e8-4175-9b51-2b43b706be22@github.com>
Message-ID: <Dv_dYswd68RgU_ia2NsaHaIaxVh-Yx5L2KaDu_mcAmw=.bb702a71-badd-4692-8802-44557bf2854c@github.com>

On Mon, 29 Mar 2021 17:27:36 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Shenandoah no longer uses parallel reference processor,  no reason to enable it.
> 
> Test:
> - [x] hotspot_gc_shenandoah

This pull request has now been integrated.

Changeset: ac604a18
Author:    Zhengyu Gu <zgu at openjdk.org>
URL:       https://git.openjdk.java.net/jdk/commit/ac604a18
Stats:     4 lines in 1 file changed: 0 ins; 4 del; 0 mod

8264374: Shenandoah: Remove leftover parallel reference processing argument

Reviewed-by: shade

-------------

PR: https://git.openjdk.java.net/jdk/pull/3246

From shade at openjdk.java.net  Wed Mar 31 13:40:56 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Wed, 31 Mar 2021 13:40:56 GMT
Subject: RFR: Clean up CondCardMark code: remove excess barriers,
 enable it by default
Message-ID: <eChq52l3NYgnnGYGx1tQ4r5WZEoGj0Dbo7E_WIXS77I=.624ef79a-2cb2-489d-a975-f689dc48156c@github.com>

The `StoreLoad`, `StoreStore` barriers in that code are only needed for CMS, and does not relate to Shenandoah. Also, `UseCondCardMark` is a recommended option for most machines today, and therefore we better just enable it ergonomically.

-------------

Commit messages:
 - Clean up CondCardMark code: remove excess barriers, enable it by default

Changes: https://git.openjdk.java.net/shenandoah/pull/26/files
 Webrev: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=26&range=00
  Stats: 20 lines in 4 files changed: 3 ins; 16 del; 1 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/26.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/26/head:pull/26

PR: https://git.openjdk.java.net/shenandoah/pull/26

From shade at openjdk.java.net  Wed Mar 31 14:24:02 2021
From: shade at openjdk.java.net (Aleksey Shipilev)
Date: Wed, 31 Mar 2021 14:24:02 GMT
Subject: RFR: Clean up CondCardMark code: remove excess barriers,
 enable it by default [v2]
In-Reply-To: <eChq52l3NYgnnGYGx1tQ4r5WZEoGj0Dbo7E_WIXS77I=.624ef79a-2cb2-489d-a975-f689dc48156c@github.com>
References: <eChq52l3NYgnnGYGx1tQ4r5WZEoGj0Dbo7E_WIXS77I=.624ef79a-2cb2-489d-a975-f689dc48156c@github.com>
Message-ID: <XLIwLVwkU036BbbFC9DGHc1-CcLK_KlNRWM_AjibF-A=.679d45db-c758-4bff-90d2-d9efa700c25c@github.com>

> The `StoreLoad`, `StoreStore` barriers in that code are only needed for CMS, and does not relate to Shenandoah. Also, `UseCondCardMark` is a recommended option for most machines today, and therefore we better just enable it ergonomically.
> 
> Additional testing:
>  - [x] hotspot_gc_shenandoah, x86_64, {fastdebug,release}

Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:

  More cleanups of scanned_concurrently

-------------

Changes:
  - all: https://git.openjdk.java.net/shenandoah/pull/26/files
  - new: https://git.openjdk.java.net/shenandoah/pull/26/files/6714df04..2668e01f

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=26&range=01
 - incr: https://webrevs.openjdk.java.net/?repo=shenandoah&pr=26&range=00-01

  Stats: 15 lines in 3 files changed: 0 ins; 14 del; 1 mod
  Patch: https://git.openjdk.java.net/shenandoah/pull/26.diff
  Fetch: git fetch https://git.openjdk.java.net/shenandoah pull/26/head:pull/26

PR: https://git.openjdk.java.net/shenandoah/pull/26

From gnu.andrew at redhat.com  Wed Mar 31 21:40:59 2021
From: gnu.andrew at redhat.com (Andrew Hughes)
Date: Wed, 31 Mar 2021 22:40:59 +0100
Subject: [RFR] [8u] 8u292-b08 Upstream Sync
In-Reply-To: <c981089a-7f0d-ddbf-8da3-11b2060df6ce@redhat.com>
References: <20210329143201.GB175114@rincewind>
 <c981089a-7f0d-ddbf-8da3-11b2060df6ce@redhat.com>
Message-ID: <20210331214059.GA353026@rincewind>

On 18:40 Mon 29 Mar     , Aleksey Shipilev wrote:
> On 3/29/21 4:32 PM, Andrew Hughes wrote:
> > Webrevs: https://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/
> > 
> > Merge changesets:
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/corba/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/jaxp/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/jaxws/merge.changeset
> 
> Look trivially good.
> 
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/jdk/merge.changeset
> 
> Looks good.
> 
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/hotspot/merge.changeset
> 
> Looks good.
> 
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/langtools/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/nashorn/merge.changeset
> > http://cr.openjdk.java.net/~andrew/shenandoah-8/u292-b08/root/merge.changeset
> 
> Look trivially good.
> 
> > Ok to push?
> 
> Yes.
> 
> 
> -- 
> Thanks,
> -Aleksey
> 

Thanks. Pushed.
-- 
Andrew :)

Senior Free Java Software Engineer
OpenJDK Package Owner
Red Hat, Inc. (http://www.redhat.com)

PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04  C5A0 CFDA 0F9B 3596 4222

From zgu at openjdk.java.net  Wed Mar 31 23:57:35 2021
From: zgu at openjdk.java.net (Zhengyu Gu)
Date: Wed, 31 Mar 2021 23:57:35 GMT
Subject: RFR: Clean up CondCardMark code: remove excess barriers,
 enable it by default [v2]
In-Reply-To: <XLIwLVwkU036BbbFC9DGHc1-CcLK_KlNRWM_AjibF-A=.679d45db-c758-4bff-90d2-d9efa700c25c@github.com>
References: <eChq52l3NYgnnGYGx1tQ4r5WZEoGj0Dbo7E_WIXS77I=.624ef79a-2cb2-489d-a975-f689dc48156c@github.com>
 <XLIwLVwkU036BbbFC9DGHc1-CcLK_KlNRWM_AjibF-A=.679d45db-c758-4bff-90d2-d9efa700c25c@github.com>
Message-ID: <IdDC0Ik6dNZwCLpJKsRk7G_4O7lez1_rX16ytHPYlvo=.9268c4ba-ac45-49d0-be61-86efbaec7300@github.com>

On Wed, 31 Mar 2021 14:24:02 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> The `StoreLoad`, `StoreStore` barriers in that code are only needed for CMS, and does not relate to Shenandoah. Also, `UseCondCardMark` is a recommended option for most machines today, and therefore we better just enable it ergonomically.
>> 
>> Additional testing:
>>  - [x] hotspot_gc_shenandoah, x86_64, {fastdebug,release}
>
> Aleksey Shipilev has updated the pull request incrementally with one additional commit since the last revision:
> 
>   More cleanups of scanned_concurrently

src/hotspot/cpu/aarch64/gc/shenandoah/shenandoahBarrierSetAssembler_aarch64.cpp line 393:

> 391:   if (UseCondCardMark) {
> 392:     Label L_already_dirty;
> 393:     __ membar(Assembler::StoreLoad);

I believe this StoreStore barrier is for preventing race against concurrent refinement. Question is that, generational Shenandoah going to implement concurrent refinement in the future?

-------------

PR: https://git.openjdk.java.net/shenandoah/pull/26