From shade at openjdk.org  Mon Oct  2 08:41:35 2023
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 2 Oct 2023 08:41:35 GMT
Subject: RFR: 8316929: Shenandoah: Shenandoah degenerated GC and full GC
 need to cleanup old OopMapCache entries
In-Reply-To: <tdLT6kB8pABZSprrUEwUmVsObCWR-dDTzNepgSzxmDI=.56840299-8a27-4d94-b0bf-c655e7e54c4b@github.com>
References: <tdLT6kB8pABZSprrUEwUmVsObCWR-dDTzNepgSzxmDI=.56840299-8a27-4d94-b0bf-c655e7e54c4b@github.com>
Message-ID: <RpHHD9CjRLZ90vruWJgL_q6yyWd53ei1REKKRm_IMa0=.a4562f3e-d16f-488c-a5b4-4ae4a3d75d68@github.com>

On Tue, 26 Sep 2023 12:48:12 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> During STW root scan, interpreted frame's oop map may be cached. But due to limited cache size (32 entries per instance class), entries may be evicted to old entries list due to collision, should be cleanup in VM_Operation's doit_epilogue(), or risk leaking memory.
> 
> Test:
>   hotspot_gc_shenandoah (fastdebug and release on MacOSX)

This looks okay, but I think you can "just" add `OMC::cleanup_old_entries` in `VM_ShenandoahReferenceOperation` destructor, without introducing the intermediary?

-------------

PR Review: https://git.openjdk.org/jdk/pull/15921#pullrequestreview-1652325437

From zgu at openjdk.org  Mon Oct  2 13:37:05 2023
From: zgu at openjdk.org (Zhengyu Gu)
Date: Mon, 2 Oct 2023 13:37:05 GMT
Subject: RFR: 8316929: Shenandoah: Shenandoah degenerated GC and full GC
 need to cleanup old OopMapCache entries [v2]
In-Reply-To: <tdLT6kB8pABZSprrUEwUmVsObCWR-dDTzNepgSzxmDI=.56840299-8a27-4d94-b0bf-c655e7e54c4b@github.com>
References: <tdLT6kB8pABZSprrUEwUmVsObCWR-dDTzNepgSzxmDI=.56840299-8a27-4d94-b0bf-c655e7e54c4b@github.com>
Message-ID: <fDywmz6oVHdng6bkwBQTwi7_E-lBeRu0UkwiZNzy0xE=.4f21c383-757e-46af-b624-085178741618@github.com>

> During STW root scan, interpreted frame's oop map may be cached. But due to limited cache size (32 entries per instance class), entries may be evicted to old entries list due to collision, should be cleanup in VM_Operation's doit_epilogue(), or risk leaking memory.
> 
> Test:
>   hotspot_gc_shenandoah (fastdebug and release on MacOSX)

Zhengyu Gu has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains three additional commits since the last revision:

 - Merge branch 'master' into JDK-8316929
 - cleanup
 - 8316929: Shenandoah: Shenandoah degenerated GC and full GC need to cleanup old OopMapCache entries

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/15921/files
  - new: https://git.openjdk.org/jdk/pull/15921/files/92dc4f4e..b84d2cad

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=15921&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15921&range=00-01

  Stats: 7940 lines in 283 files changed: 5780 ins; 1041 del; 1119 mod
  Patch: https://git.openjdk.org/jdk/pull/15921.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15921/head:pull/15921

PR: https://git.openjdk.org/jdk/pull/15921

From zgu at openjdk.org  Mon Oct  2 13:41:56 2023
From: zgu at openjdk.org (Zhengyu Gu)
Date: Mon, 2 Oct 2023 13:41:56 GMT
Subject: RFR: 8316929: Shenandoah: Shenandoah degenerated GC and full GC
 need to cleanup old OopMapCache entries [v3]
In-Reply-To: <tdLT6kB8pABZSprrUEwUmVsObCWR-dDTzNepgSzxmDI=.56840299-8a27-4d94-b0bf-c655e7e54c4b@github.com>
References: <tdLT6kB8pABZSprrUEwUmVsObCWR-dDTzNepgSzxmDI=.56840299-8a27-4d94-b0bf-c655e7e54c4b@github.com>
Message-ID: <y5p8fpYw_03lJ_Y4qoL6VZL-ND867Ux4_XoBlBN0Rcg=.b427ba40-86d9-47fe-9b8e-5c438ebe4e11@github.com>

> During STW root scan, interpreted frame's oop map may be cached. But due to limited cache size (32 entries per instance class), entries may be evicted to old entries list due to collision, should be cleanup in VM_Operation's doit_epilogue(), or risk leaking memory.
> 
> Test:
>   hotspot_gc_shenandoah (fastdebug and release on MacOSX)

Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:

  Fix order

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/15921/files
  - new: https://git.openjdk.org/jdk/pull/15921/files/b84d2cad..c1e9cf27

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=15921&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15921&range=01-02

  Stats: 2 lines in 1 file changed: 1 ins; 1 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/15921.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15921/head:pull/15921

PR: https://git.openjdk.org/jdk/pull/15921

From zgu at openjdk.org  Mon Oct  2 13:48:58 2023
From: zgu at openjdk.org (Zhengyu Gu)
Date: Mon, 2 Oct 2023 13:48:58 GMT
Subject: RFR: 8316929: Shenandoah: Shenandoah degenerated GC and full GC
 need to cleanup old OopMapCache entries [v4]
In-Reply-To: <tdLT6kB8pABZSprrUEwUmVsObCWR-dDTzNepgSzxmDI=.56840299-8a27-4d94-b0bf-c655e7e54c4b@github.com>
References: <tdLT6kB8pABZSprrUEwUmVsObCWR-dDTzNepgSzxmDI=.56840299-8a27-4d94-b0bf-c655e7e54c4b@github.com>
Message-ID: <S7eocikkCDKJa6EXWroVL9xXP7YxmYTyoSqh003yCj8=.d8d15d49-8377-41dd-9acc-b06e2f09edcc@github.com>

> During STW root scan, interpreted frame's oop map may be cached. But due to limited cache size (32 entries per instance class), entries may be evicted to old entries list due to collision, should be cleanup in VM_Operation's doit_epilogue(), or risk leaking memory.
> 
> Test:
>   hotspot_gc_shenandoah (fastdebug and release on MacOSX)

Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:

  More cleanup

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/15921/files
  - new: https://git.openjdk.org/jdk/pull/15921/files/c1e9cf27..4f41dc14

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=15921&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15921&range=02-03

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/15921.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15921/head:pull/15921

PR: https://git.openjdk.org/jdk/pull/15921

From zgu at openjdk.org  Mon Oct  2 13:51:13 2023
From: zgu at openjdk.org (Zhengyu Gu)
Date: Mon, 2 Oct 2023 13:51:13 GMT
Subject: RFR: 8316929: Shenandoah: Shenandoah degenerated GC and full GC
 need to cleanup old OopMapCache entries [v3]
In-Reply-To: <y5p8fpYw_03lJ_Y4qoL6VZL-ND867Ux4_XoBlBN0Rcg=.b427ba40-86d9-47fe-9b8e-5c438ebe4e11@github.com>
References: <tdLT6kB8pABZSprrUEwUmVsObCWR-dDTzNepgSzxmDI=.56840299-8a27-4d94-b0bf-c655e7e54c4b@github.com>
 <y5p8fpYw_03lJ_Y4qoL6VZL-ND867Ux4_XoBlBN0Rcg=.b427ba40-86d9-47fe-9b8e-5c438ebe4e11@github.com>
Message-ID: <Vt_IifdARDs7gGaOVh-oAMB_tx8XUxTnZmXvwreFlAE=.c66434d7-4561-4bf9-81b2-c0122f587f75@github.com>

On Mon, 2 Oct 2023 13:41:56 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> During STW root scan, interpreted frame's oop map may be cached. But due to limited cache size (32 entries per instance class), entries may be evicted to old entries list due to collision, should be cleanup in VM_Operation's doit_epilogue(), or risk leaking memory.
>> 
>> Test:
>>   hotspot_gc_shenandoah (fastdebug and release on MacOSX)
>
> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix order

> This looks okay, but I think you can "just" add `OMC::cleanup_old_entries` in `VM_ShenandoahReferenceOperation` destructor, without introducing the intermediary?



> This looks okay, but I think you can "just" add `OMC::cleanup_old_entries` in `VM_ShenandoahReferenceOperation` destructor, without introducing the intermediary?

Place the call in `doit_epilogue()` for the symmetry with other [GCs](https://github.com/openjdk/jdk/blob/master/src/hotspot/share/gc/shared/gcVMOperations.cpp#L135). But you are right, we don't need another  VM operation.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15921#issuecomment-1743052006

From shade at openjdk.org  Mon Oct  2 14:32:14 2023
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 2 Oct 2023 14:32:14 GMT
Subject: RFR: 8316929: Shenandoah: Shenandoah degenerated GC and full GC
 need to cleanup old OopMapCache entries [v4]
In-Reply-To: <S7eocikkCDKJa6EXWroVL9xXP7YxmYTyoSqh003yCj8=.d8d15d49-8377-41dd-9acc-b06e2f09edcc@github.com>
References: <tdLT6kB8pABZSprrUEwUmVsObCWR-dDTzNepgSzxmDI=.56840299-8a27-4d94-b0bf-c655e7e54c4b@github.com>
 <S7eocikkCDKJa6EXWroVL9xXP7YxmYTyoSqh003yCj8=.d8d15d49-8377-41dd-9acc-b06e2f09edcc@github.com>
Message-ID: <TNLlk5rrdHnzcScG1YNXf4de--WuySrAci12yWyU_jQ=.5100771b-3f8d-45b2-b0c8-d8b37c337721@github.com>

On Mon, 2 Oct 2023 13:48:58 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> During STW root scan, interpreted frame's oop map may be cached. But due to limited cache size (32 entries per instance class), entries may be evicted to old entries list due to collision, should be cleanup in VM_Operation's doit_epilogue(), or risk leaking memory.
>> 
>> Test:
>>   hotspot_gc_shenandoah (fastdebug and release on MacOSX)
>
> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
> 
>   More cleanup

Looks fine, thanks!

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/15921#pullrequestreview-1652875148

From wkemper at openjdk.org  Mon Oct  2 17:35:18 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 2 Oct 2023 17:35:18 GMT
Subject: RFR: 8312117: GenShen: Preempt OLD marking more quickly when YOUNG
 triggers arise
Message-ID: <uUgnLUhEWa2zzkKPvzoRmMlF_gDABd-soaTBsLsD1w4=.a636fb35-8af1-4cb0-83f5-055ad23cefd0@github.com>

Have regulator thread wait until expected condition is met

-------------

Commit messages:
 - Add timeout for extra paranoia
 - Change log level to debug
 - Have regulator thread wait until expected condition is met

Changes: https://git.openjdk.org/shenandoah/pull/333/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=333&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8312117
  Stats: 44 lines in 3 files changed: 37 ins; 0 del; 7 mod
  Patch: https://git.openjdk.org/shenandoah/pull/333.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/333/head:pull/333

PR: https://git.openjdk.org/shenandoah/pull/333

From kdnilsen at openjdk.org  Mon Oct  2 17:41:30 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 2 Oct 2023 17:41:30 GMT
Subject: RFR: 8312117: GenShen: Preempt OLD marking more quickly when YOUNG
 triggers arise
In-Reply-To: <uUgnLUhEWa2zzkKPvzoRmMlF_gDABd-soaTBsLsD1w4=.a636fb35-8af1-4cb0-83f5-055ad23cefd0@github.com>
References: <uUgnLUhEWa2zzkKPvzoRmMlF_gDABd-soaTBsLsD1w4=.a636fb35-8af1-4cb0-83f5-055ad23cefd0@github.com>
Message-ID: <gQKPZTPr_4kgS0VvxZQSXi8_pyF_Qw3SvI2utR0SNag=.510588da-140f-46d0-ab57-3df0bc456bac@github.com>

On Mon, 2 Oct 2023 17:15:32 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Have regulator thread wait until expected condition is met

Thanks for sorting this out.

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/333#pullrequestreview-1653201910

From wkemper at openjdk.org  Mon Oct  2 18:05:03 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 2 Oct 2023 18:05:03 GMT
Subject: RFR: 8312117: GenShen: Preempt OLD marking more quickly when YOUNG
 triggers arise [v2]
In-Reply-To: <uUgnLUhEWa2zzkKPvzoRmMlF_gDABd-soaTBsLsD1w4=.a636fb35-8af1-4cb0-83f5-055ad23cefd0@github.com>
References: <uUgnLUhEWa2zzkKPvzoRmMlF_gDABd-soaTBsLsD1w4=.a636fb35-8af1-4cb0-83f5-055ad23cefd0@github.com>
Message-ID: <pk7FuJ643D_ikxDbx32ySfg1PYMkJ1r0BowL-LKn8fE=.569fd3fe-222d-43b5-a882-4a1db4187ae8@github.com>

> Have regulator thread wait until expected condition is met

William Kemper has updated the pull request incrementally with two additional commits since the last revision:

 - Use same precision for time checks as log messages
 - Regulator lock should protect the write to mode

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/333/files
  - new: https://git.openjdk.org/shenandoah/pull/333/files/fbc5de1f..7319707c

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=333&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=333&range=00-01

  Stats: 4 lines in 2 files changed: 1 ins; 1 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/333.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/333/head:pull/333

PR: https://git.openjdk.org/shenandoah/pull/333

From ysr at openjdk.org  Mon Oct  2 19:06:47 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Mon, 2 Oct 2023 19:06:47 GMT
Subject: RFR: 8312117: GenShen: Preempt OLD marking more quickly when YOUNG
 triggers arise [v2]
In-Reply-To: <vOeGYR-BoJNbsRUtZIUFlK9cA5uF_YlNlvhNq7edhq8=.0c19eda6-1258-4920-a63b-9730cca4a06b@github.com>
References: <uUgnLUhEWa2zzkKPvzoRmMlF_gDABd-soaTBsLsD1w4=.a636fb35-8af1-4cb0-83f5-055ad23cefd0@github.com>
 <pk7FuJ643D_ikxDbx32ySfg1PYMkJ1r0BowL-LKn8fE=.569fd3fe-222d-43b5-a882-4a1db4187ae8@github.com>
 <vOeGYR-BoJNbsRUtZIUFlK9cA5uF_YlNlvhNq7edhq8=.0c19eda6-1258-4920-a63b-9730cca4a06b@github.com>
Message-ID: <kCgIvrnnLLI-jJ6Wk27XaT3NzeE7HjZrfUOe0WEmNs4=.d4baf7ea-dbfc-4b4d-863a-f585ac90fae8@github.com>

On Mon, 2 Oct 2023 18:26:27 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> William Kemper has updated the pull request incrementally with two additional commits since the last revision:
>> 
>>  - Use same precision for time checks as log messages
>>  - Regulator lock should protect the write to mode
>
> src/hotspot/share/gc/shenandoah/shenandoahRegulatorThread.cpp line 150:
> 
>> 148: 
>> 149: bool ShenandoahRegulatorThread::start_old_cycle() {
>> 150:   // TODO: These first two checks might be vestigial
> 
> It would seem that not starting a fresh old marking cycle while we have copying work left to do from a previous one does however seem to be reasonable.
> 
> Why do you believe this is vestigial? Have we already checked these conditions elsewhere recently? (I realize we can't assert those two conditions since the state may have changed concurrently because of concurrent collection activity.)

Also wondered if the presence of only pinned old regions at this check might confound the second conjunct in the check?

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/333#discussion_r1343023242

From ysr at openjdk.org  Mon Oct  2 19:06:47 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Mon, 2 Oct 2023 19:06:47 GMT
Subject: RFR: 8312117: GenShen: Preempt OLD marking more quickly when YOUNG
 triggers arise [v2]
In-Reply-To: <pk7FuJ643D_ikxDbx32ySfg1PYMkJ1r0BowL-LKn8fE=.569fd3fe-222d-43b5-a882-4a1db4187ae8@github.com>
References: <uUgnLUhEWa2zzkKPvzoRmMlF_gDABd-soaTBsLsD1w4=.a636fb35-8af1-4cb0-83f5-055ad23cefd0@github.com>
 <pk7FuJ643D_ikxDbx32ySfg1PYMkJ1r0BowL-LKn8fE=.569fd3fe-222d-43b5-a882-4a1db4187ae8@github.com>
Message-ID: <vOeGYR-BoJNbsRUtZIUFlK9cA5uF_YlNlvhNq7edhq8=.0c19eda6-1258-4920-a63b-9730cca4a06b@github.com>

On Mon, 2 Oct 2023 18:05:03 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Have regulator thread wait until expected condition is met
>
> William Kemper has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Use same precision for time checks as log messages
>  - Regulator lock should protect the write to mode

Looks good to me, although I have not yet fully digested the request-response synchronization skeleton fully yet.

Just left a few minor comments.

src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp line 911:

> 909: 
> 910:     MonitorLocker ml(&_regulator_lock, Mutex::_no_safepoint_check_flag);
> 911:     while (_mode == none) {

`gc_mode()` instead of `_mode`.

src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp line 922:

> 920:   if (preempt_old_marking(generation)) {
> 921:     log_info(gc)("Preempting old generation mark to allow %s GC", shenandoah_generation_name(generation));
> 922:     assert(_mode == servicing_old, "Cannot preempt old if old cycle isn't running.");

Is this a stable condition at the point of this check? (i.e. can its value be changed by concurrent activity between the test at line 920 and the assert at line 922?)

Also, for programming hygiene, I'd use the accessor method `gc_mode()` everywhere, instead of `_mode` when accessing its value. This allows for easy changes for insertion of assertions, tests, etc. in the accessor method if needed (as well as making debugging a tad easier). Avoiding naked accesses to fields is almost always a good idea.

src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp line 930:

> 928: 
> 929:     MonitorLocker ml(&_regulator_lock, Mutex::_no_safepoint_check_flag);
> 930:     while (_mode == servicing_old) {

`gc_mode()` instead of `_mode`.

src/hotspot/share/gc/shenandoah/shenandoahRegulatorThread.cpp line 150:

> 148: 
> 149: bool ShenandoahRegulatorThread::start_old_cycle() {
> 150:   // TODO: These first two checks might be vestigial

It would seem that not starting a fresh old marking cycle while we have copying work left to do from a previous one does however seem to be reasonable.

Why do you believe this is vestigial? Have we already checked these conditions elsewhere recently? (I realize we can't assert those two conditions since the state may have changed concurrently because of concurrent collection activity.)

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/333#pullrequestreview-1653270193
PR Review Comment: https://git.openjdk.org/shenandoah/pull/333#discussion_r1343049765
PR Review Comment: https://git.openjdk.org/shenandoah/pull/333#discussion_r1343042620
PR Review Comment: https://git.openjdk.org/shenandoah/pull/333#discussion_r1343050571
PR Review Comment: https://git.openjdk.org/shenandoah/pull/333#discussion_r1343017858

From ysr at openjdk.org  Mon Oct  2 19:14:43 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Mon, 2 Oct 2023 19:14:43 GMT
Subject: RFR: 8312117: GenShen: Preempt OLD marking more quickly when YOUNG
 triggers arise [v2]
In-Reply-To: <pk7FuJ643D_ikxDbx32ySfg1PYMkJ1r0BowL-LKn8fE=.569fd3fe-222d-43b5-a882-4a1db4187ae8@github.com>
References: <uUgnLUhEWa2zzkKPvzoRmMlF_gDABd-soaTBsLsD1w4=.a636fb35-8af1-4cb0-83f5-055ad23cefd0@github.com>
 <pk7FuJ643D_ikxDbx32ySfg1PYMkJ1r0BowL-LKn8fE=.569fd3fe-222d-43b5-a882-4a1db4187ae8@github.com>
Message-ID: <L8WFNlTQFjdT5ZfXHQl1oYgtYvxhb9Y5HG-bohwIoBY=.f05a36a4-ab8f-4d97-9bec-0c423eb537d6@github.com>

On Mon, 2 Oct 2023 18:05:03 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Have regulator thread wait until expected condition is met
>
> William Kemper has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Use same precision for time checks as log messages
>  - Regulator lock should protect the write to mode

src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp line 912:

> 910:     MonitorLocker ml(&_regulator_lock, Mutex::_no_safepoint_check_flag);
> 911:     while (_mode == none) {
> 912:       bool timeout = ml.wait(5);

Should this be a globals value? Are there other such "took too long to respond" loops governed by a timeout variable defined in shenandoah_globals.hpp that might be open to overloaded use here perhaps?

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/333#discussion_r1343056111

From wkemper at openjdk.org  Mon Oct  2 19:14:43 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 2 Oct 2023 19:14:43 GMT
Subject: RFR: 8312117: GenShen: Preempt OLD marking more quickly when YOUNG
 triggers arise [v2]
In-Reply-To: <L8WFNlTQFjdT5ZfXHQl1oYgtYvxhb9Y5HG-bohwIoBY=.f05a36a4-ab8f-4d97-9bec-0c423eb537d6@github.com>
References: <uUgnLUhEWa2zzkKPvzoRmMlF_gDABd-soaTBsLsD1w4=.a636fb35-8af1-4cb0-83f5-055ad23cefd0@github.com>
 <pk7FuJ643D_ikxDbx32ySfg1PYMkJ1r0BowL-LKn8fE=.569fd3fe-222d-43b5-a882-4a1db4187ae8@github.com>
 <L8WFNlTQFjdT5ZfXHQl1oYgtYvxhb9Y5HG-bohwIoBY=.f05a36a4-ab8f-4d97-9bec-0c423eb537d6@github.com>
Message-ID: <aubOGPp2qv_SI2MU6KBRKdF-3jcE_reI_OKmXG5hbGk=.568752eb-b8f9-4713-9f61-54fed6b6f797@github.com>

On Mon, 2 Oct 2023 19:06:53 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> William Kemper has updated the pull request incrementally with two additional commits since the last revision:
>> 
>>  - Use same precision for time checks as log messages
>>  - Regulator lock should protect the write to mode
>
> src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp line 912:
> 
>> 910:     MonitorLocker ml(&_regulator_lock, Mutex::_no_safepoint_check_flag);
>> 911:     while (_mode == none) {
>> 912:       bool timeout = ml.wait(5);
> 
> Should this be a globals value? Are there other such "took too long to respond" loops governed by a timeout variable defined in shenandoah_globals.hpp that might be open to overloaded use here perhaps?

I can remove the timeout, I was just being paranoid. With the `regulator_lock` being used correctly, it shouldn't be necessary to have a timeout.

> src/hotspot/share/gc/shenandoah/shenandoahControlThread.cpp line 922:
> 
>> 920:   if (preempt_old_marking(generation)) {
>> 921:     log_info(gc)("Preempting old generation mark to allow %s GC", shenandoah_generation_name(generation));
>> 922:     assert(_mode == servicing_old, "Cannot preempt old if old cycle isn't running.");
> 
> Is this a stable condition at the point of this check? (i.e. can its value be changed by concurrent activity between the test at line 920 and the assert at line 922?)
> 
> Also, for programming hygiene, I'd use the accessor method `gc_mode()` everywhere, instead of `_mode` when accessing its value. This allows for easy changes for insertion of assertions, tests, etc. in the accessor method if needed (as well as making debugging a tad easier). Avoiding naked accesses to fields is almost always a good idea.

Yes, the `preempt_old_marking` condition can only happen during certain phases of an old cycle. If we are able to preempt old marking, then the controller thread _must_ be `servicing_old`.

I'll make changes to use the accessor method.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/333#discussion_r1343059607
PR Review Comment: https://git.openjdk.org/shenandoah/pull/333#discussion_r1343058643

From wkemper at openjdk.org  Mon Oct  2 19:14:44 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 2 Oct 2023 19:14:44 GMT
Subject: RFR: 8312117: GenShen: Preempt OLD marking more quickly when YOUNG
 triggers arise [v2]
In-Reply-To: <kCgIvrnnLLI-jJ6Wk27XaT3NzeE7HjZrfUOe0WEmNs4=.d4baf7ea-dbfc-4b4d-863a-f585ac90fae8@github.com>
References: <uUgnLUhEWa2zzkKPvzoRmMlF_gDABd-soaTBsLsD1w4=.a636fb35-8af1-4cb0-83f5-055ad23cefd0@github.com>
 <pk7FuJ643D_ikxDbx32ySfg1PYMkJ1r0BowL-LKn8fE=.569fd3fe-222d-43b5-a882-4a1db4187ae8@github.com>
 <vOeGYR-BoJNbsRUtZIUFlK9cA5uF_YlNlvhNq7edhq8=.0c19eda6-1258-4920-a63b-9730cca4a06b@github.com>
 <kCgIvrnnLLI-jJ6Wk27XaT3NzeE7HjZrfUOe0WEmNs4=.d4baf7ea-dbfc-4b4d-863a-f585ac90fae8@github.com>
Message-ID: <WJVaT532A_NNOHyh7NN8IWP6vCVUxOMLdc1QOzLi_WQ=.9e542825-f779-4506-b6c1-daffb7764e0c@github.com>

On Mon, 2 Oct 2023 18:33:12 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahRegulatorThread.cpp line 150:
>> 
>>> 148: 
>>> 149: bool ShenandoahRegulatorThread::start_old_cycle() {
>>> 150:   // TODO: These first two checks might be vestigial
>> 
>> It would seem that not starting a fresh old marking cycle while we have copying work left to do from a previous one does however seem to be reasonable.
>> 
>> Why do you believe this is vestigial? Have we already checked these conditions elsewhere recently? (I realize we can't assert those two conditions since the state may have changed concurrently because of concurrent collection activity.)
>
> Also wondered if the presence of only pinned old regions at this check might confound the second conjunct in the check?

These conditions are "re-checked" in `ShenandoahOldHeuristic::should_start_gc`, though with a different API.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/333#discussion_r1343057028

From wkemper at openjdk.org  Mon Oct  2 19:38:39 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 2 Oct 2023 19:38:39 GMT
Subject: RFR: 8312117: GenShen: Preempt OLD marking more quickly when YOUNG
 triggers arise [v3]
In-Reply-To: <uUgnLUhEWa2zzkKPvzoRmMlF_gDABd-soaTBsLsD1w4=.a636fb35-8af1-4cb0-83f5-055ad23cefd0@github.com>
References: <uUgnLUhEWa2zzkKPvzoRmMlF_gDABd-soaTBsLsD1w4=.a636fb35-8af1-4cb0-83f5-055ad23cefd0@github.com>
Message-ID: <yipnJqnkLPf-JJIrQoFgI9IngXpUWwQenj6ofJTCPwE=.b46a4820-81bc-45ad-8bcd-36afe3c30a99@github.com>

> Have regulator thread wait until expected condition is met

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Remove arbitrary timeout for wait, use accessors for _mode

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/333/files
  - new: https://git.openjdk.org/shenandoah/pull/333/files/7319707c..639ced2a

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=333&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=333&range=01-02

  Stats: 14 lines in 1 file changed: 0 ins; 6 del; 8 mod
  Patch: https://git.openjdk.org/shenandoah/pull/333.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/333/head:pull/333

PR: https://git.openjdk.org/shenandoah/pull/333

From wkemper at openjdk.org  Mon Oct  2 20:18:31 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 2 Oct 2023 20:18:31 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
 [v3]
In-Reply-To: <2bXcUWi2pXLKmpiPTNIT2_xu2fGBwPHB5YUMInN-OFo=.92656fc9-7549-41fa-bb9d-445cee3db5f8@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
 <mFCZNYew3b5-eFhumy8hnEX-C-9uw7X1bihJeVzZ7Gc=.2a63e9ab-79a6-4ae6-b2fd-fc25fc201c54@github.com>
 <2bXcUWi2pXLKmpiPTNIT2_xu2fGBwPHB5YUMInN-OFo=.92656fc9-7549-41fa-bb9d-445cee3db5f8@github.com>
Message-ID: <zGeIJi0gRUKtYIAO99hnXKKYcOmmj4wfUtHFugCoz8E=.5735fed9-0654-4938-96cf-15b9b2672336@github.com>

On Fri, 29 Sep 2023 16:28:30 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision:
>> 
>>  - Merge remote-tracking branch 'openjdk/master' into shenandoah-oome-redux
>>  - Extend exemption for EATests that rely on timely OOME to Shenandoah
>>  - Improve comment, increase default for no progress threshold
>>  - Allocator should not reset bad progress count
>>  - Allocator should not reset bad progress count
>>  - Fix 32-bit build error
>>  - Do not use atomics in header
>>  - Signal gc threshold exceeded when appropriate
>
> test/hotspot/jtreg/TEST.groups line 612:
> 
>> 610:   gtest/NMTGtests.java
>> 611: 
>> 612: hotspot_oome = \
> 
> Adding new test groups require additional attention. I think this should be removed.
> 
> Note that during debugging/testing, you can run the same by:
> 
> 
> $ make test TEST="runtime/reflect/ReflectOutOfMemoryError.java gc/InfiniteList.java runtime/ClassInitErrors/TestOutOfMemoryDuringInit.java"

I'll revert this.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15852#discussion_r1343104448

From zgu at openjdk.org  Mon Oct  2 20:56:38 2023
From: zgu at openjdk.org (Zhengyu Gu)
Date: Mon, 2 Oct 2023 20:56:38 GMT
Subject: Integrated: 8316929: Shenandoah: Shenandoah degenerated GC and full GC
 need to cleanup old OopMapCache entries
In-Reply-To: <tdLT6kB8pABZSprrUEwUmVsObCWR-dDTzNepgSzxmDI=.56840299-8a27-4d94-b0bf-c655e7e54c4b@github.com>
References: <tdLT6kB8pABZSprrUEwUmVsObCWR-dDTzNepgSzxmDI=.56840299-8a27-4d94-b0bf-c655e7e54c4b@github.com>
Message-ID: <c_PjpnvM67kIKBOpAqjYvlfQtW2dCCMRWJupz6g6i80=.a313a359-674a-4042-9000-10a222d5ccd8@github.com>

On Tue, 26 Sep 2023 12:48:12 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> During STW root scan, interpreted frame's oop map may be cached. But due to limited cache size (32 entries per instance class), entries may be evicted to old entries list due to collision, should be cleanup in VM_Operation's doit_epilogue(), or risk leaking memory.
> 
> Test:
>   hotspot_gc_shenandoah (fastdebug and release on MacOSX)

This pull request has now been integrated.

Changeset: e25121d1
Author:    Zhengyu Gu <zgu at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/e25121d1d908bd74e7a5914d85284ab322bed1a3
Stats:     2 lines in 1 file changed: 2 ins; 0 del; 0 mod

8316929: Shenandoah: Shenandoah degenerated GC and full GC need to cleanup old OopMapCache entries

Reviewed-by: shade

-------------

PR: https://git.openjdk.org/jdk/pull/15921

From zgu at openjdk.org  Mon Oct  2 20:56:33 2023
From: zgu at openjdk.org (Zhengyu Gu)
Date: Mon, 2 Oct 2023 20:56:33 GMT
Subject: RFR: 8316929: Shenandoah: Shenandoah degenerated GC and full GC
 need to cleanup old OopMapCache entries [v4]
In-Reply-To: <TNLlk5rrdHnzcScG1YNXf4de--WuySrAci12yWyU_jQ=.5100771b-3f8d-45b2-b0c8-d8b37c337721@github.com>
References: <tdLT6kB8pABZSprrUEwUmVsObCWR-dDTzNepgSzxmDI=.56840299-8a27-4d94-b0bf-c655e7e54c4b@github.com>
 <S7eocikkCDKJa6EXWroVL9xXP7YxmYTyoSqh003yCj8=.d8d15d49-8377-41dd-9acc-b06e2f09edcc@github.com>
 <TNLlk5rrdHnzcScG1YNXf4de--WuySrAci12yWyU_jQ=.5100771b-3f8d-45b2-b0c8-d8b37c337721@github.com>
Message-ID: <f59BI3m-fr1W7yOlRM-qsDQt0Cp7tQbDOQAfV9wug40=.5110b415-e923-4e7d-a9a4-4beaf97cc476@github.com>

On Mon, 2 Oct 2023 14:29:11 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   More cleanup
>
> Looks fine, thanks!

Thanks, @shipilev

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15921#issuecomment-1743693867

From wkemper at openjdk.org  Mon Oct  2 21:31:13 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 2 Oct 2023 21:31:13 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
 [v4]
In-Reply-To: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
Message-ID: <b1TOHHtcREYY327DX1za8LbbzqQw5IjuqMPxjWLxKo0=.e211440b-622d-41a3-bf68-652c4849cfd5@github.com>

> Shenandoah will run back-to-back full GCs and _almost_ grind mutator progress to a halt before eventually exhausting memory. This change will have Shenandoah raise a gc threshold exceeded exception if the collector fails to make progress after `ShenandoahNoProgressThreshold` full GC cycles (default is 3).

William Kemper has updated the pull request incrementally with two additional commits since the last revision:

 - Merge check for no-progress into retry allocation block
 - Revert change to TEST.groups

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/15852/files
  - new: https://git.openjdk.org/jdk/pull/15852/files/cd4989e7..1971467f

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=15852&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15852&range=02-03

  Stats: 33 lines in 2 files changed: 10 ins; 14 del; 9 mod
  Patch: https://git.openjdk.org/jdk/pull/15852.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15852/head:pull/15852

PR: https://git.openjdk.org/jdk/pull/15852

From wkemper at openjdk.org  Mon Oct  2 21:31:19 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 2 Oct 2023 21:31:19 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
 [v3]
In-Reply-To: <2bXcUWi2pXLKmpiPTNIT2_xu2fGBwPHB5YUMInN-OFo=.92656fc9-7549-41fa-bb9d-445cee3db5f8@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
 <mFCZNYew3b5-eFhumy8hnEX-C-9uw7X1bihJeVzZ7Gc=.2a63e9ab-79a6-4ae6-b2fd-fc25fc201c54@github.com>
 <2bXcUWi2pXLKmpiPTNIT2_xu2fGBwPHB5YUMInN-OFo=.92656fc9-7549-41fa-bb9d-445cee3db5f8@github.com>
Message-ID: <PTMqLDZglzX895tnsdZiCTw8vQoX2AfE7QwvD-lXI2g=.5ec6edc2-4f20-40a1-8523-8e0eca47e032@github.com>

On Fri, 29 Sep 2023 16:40:22 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:

>> William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains eight additional commits since the last revision:
>> 
>>  - Merge remote-tracking branch 'openjdk/master' into shenandoah-oome-redux
>>  - Extend exemption for EATests that rely on timely OOME to Shenandoah
>>  - Improve comment, increase default for no progress threshold
>>  - Allocator should not reset bad progress count
>>  - Allocator should not reset bad progress count
>>  - Fix 32-bit build error
>>  - Do not use atomics in header
>>  - Signal gc threshold exceeded when appropriate
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 877:
> 
>> 875:     }
>> 876: 
>> 877:     if (result == nullptr && !req.is_lab_alloc() && get_gc_no_progress_count() > ShenandoahNoProgressThreshold) {
> 
> Can this be moved to the block that already does allocation-after-gc retries? That block already exits with `nullptr` (implies delivering OOME), and it already calls `handle_alloc_failure` (thus triggering GC). We "only" need it to specialize for `is_lab_alloc` and `ShenandoahNoProgressThreshold`?
> 
> This PR changes that block anyway...

Yes, I've made this change as you suggest, but I'm not sure it improves readability. Note that when the `ShenandoahNoProgressThreshold` is exceeded, we do _not_ retry the allocation or wait for another gc cycle to complete.

> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 902:
> 
>> 900:       ResourceMark rm;
>> 901:       log_debug(gc, alloc)("Thread: %s, Result: " PTR_FORMAT ", Shared: %s, Size: " SIZE_FORMAT ", Original: " SIZE_FORMAT ", Latest: " SIZE_FORMAT,
>> 902:                            Thread::current()->name(), p2i(result), BOOL_TO_STR(!req.is_lab_alloc()), req.size(), original_count, get_gc_no_progress_count());
> 
> There is `type_string()` that can be used instead of `is_lab_alloc()` here.

TIL - thank you.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15852#discussion_r1343174921
PR Review Comment: https://git.openjdk.org/jdk/pull/15852#discussion_r1343175156

From wkemper at openjdk.org  Mon Oct  2 22:25:25 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 2 Oct 2023 22:25:25 GMT
Subject: Integrated: 8312117: GenShen: Preempt OLD marking more quickly when
 YOUNG triggers arise
In-Reply-To: <uUgnLUhEWa2zzkKPvzoRmMlF_gDABd-soaTBsLsD1w4=.a636fb35-8af1-4cb0-83f5-055ad23cefd0@github.com>
References: <uUgnLUhEWa2zzkKPvzoRmMlF_gDABd-soaTBsLsD1w4=.a636fb35-8af1-4cb0-83f5-055ad23cefd0@github.com>
Message-ID: <ZwK2kb142_2ag7rXvwt6UerEaLeGYoxSJlHTB7yy-bQ=.54da4f64-649b-4e26-9150-14411977f3af@github.com>

On Mon, 2 Oct 2023 17:15:32 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Have regulator thread wait until expected condition is met

This pull request has now been integrated.

Changeset: 8232bb77
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/8232bb7792b620b73aa154e4977ee02fa7378733
Stats:     43 lines in 3 files changed: 32 ins; 1 del; 10 mod

8312117: GenShen: Preempt OLD marking more quickly when YOUNG triggers arise

Reviewed-by: kdnilsen, ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/333

From kdnilsen at openjdk.org  Mon Oct  2 22:58:17 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 2 Oct 2023 22:58:17 GMT
Subject: Withdrawn: 8317049: GenShen: Very rare loss of function for
 ShenandoahControlThread::_regulator_lock
In-Reply-To: <yullOkPUQLqsUHeDukVzLER0HOya70A_6uZYh72G9TE=.c9656218-7fb4-482f-b125-e7228041b41d@github.com>
References: <yullOkPUQLqsUHeDukVzLER0HOya70A_6uZYh72G9TE=.c9656218-7fb4-482f-b125-e7228041b41d@github.com>
Message-ID: <3CjJuZR9DPpcqP4VrXC25_-fjNlFz-mamdJQxgkJRZI=.ce970dbf-e7d0-42f2-bfeb-64402058c66d@github.com>

On Thu, 28 Sep 2023 17:06:04 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> We have observed that we very rarely miss _regulator_lock notifications.  We have identified the root cause to be a race, which occurs when the ShenandoahControlThread notifies the _regulator_lock before the ShenandoahRegulatorThread has begun to wait for the notification.
> 
> This change forces the ShenandoahControlThread to wait until the ShenandoahRegulatorThread is inside its wait() invocation.  It does so by acquiring the Mutator lock for _regulator_thread before notifying the control thread.  The control thread must acquire this lock before it can notify.  The control thread will not be able to acquire the lock until the regulator thread releases the lock by invoking ml.wait().

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/shenandoah/pull/332

From tschatzl at openjdk.org  Tue Oct  3 13:04:03 2023
From: tschatzl at openjdk.org (Thomas Schatzl)
Date: Tue, 3 Oct 2023 13:04:03 GMT
Subject: RFR: 8317350: Move code cache purging out of CodeCache::UnloadingScope
Message-ID: <BvcvMniLilmj4oHWHZRAratUJar05TJ1XptU_XoPHm0=.c1c84329-b5e0-4493-8133-aa528dbfb081@github.com>

Hi all,

  please review this refactoring that moves actual code cache flushing/purging out of `CodeCache::UnloadingScope`. Reasons:

* I prefer that a destructor does not do anything substantial - in some cases, 90% of time is spent in the destructor in that extracted method (due to https://bugs.openjdk.org/browse/JDK-8316959)
* imho it does not fit the class which does nothing but sets/resets some code cache unloading behavior (probably should be renamed to `UnloadingBehaviorScope` too in a separate CR).
* other existing methods at that level are placed out of that (or any other) scope object too - which is already the case for when doing concurrent unloading.
* putting it there makes future logging of the various phases a little bit easier, not having `GCTraceTimer` et al. in various places.

Testing: gha

Thanks,
  Thomas

-------------

Commit messages:
 - 8317350 move codecache purging out of unloading scope

Changes: https://git.openjdk.org/jdk/pull/16011/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16011&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8317350
  Stats: 60 lines in 7 files changed: 26 ins; 6 del; 28 mod
  Patch: https://git.openjdk.org/jdk/pull/16011.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/16011/head:pull/16011

PR: https://git.openjdk.org/jdk/pull/16011

From wkemper at openjdk.org  Thu Oct  5 14:25:35 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 5 Oct 2023 14:25:35 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <VUN4vebhNwhFEcDzkoXzTHkm9Qx-iPwyBorbqFfdnIk=.32f2bcf1-31fc-40f9-a107-cc6fbcb51b60@github.com>

Merges tag jdk-22+18

-------------

Commit messages:
 - 8316146: Open some swing tests 4
 - 8267509: Improve IllegalAccessException message to include the cause of the exception
 - 8305765: CompressedClassPointers.java is unreliable due to ASLR
 - 8317354: Serial: Move DirtyCardToOopClosure to gc/serial folder
 - 8316414: C2: large byte array clone triggers "failed: malformed control flow" assertion failure on linux-x86
 - 8316396: Endless loop in C2 compilation triggered by AddNode::IdealIL
 - 8280120: [IR Framework] Add attribute to @IR to enable/disable IR matching based on the architecture
 - 8317452: [JVMCI] Export symbols used by lightweight locking to JVMCI compilers.
 - 8316971: Add Lint warning for restricted method calls
 - 8317294: Classloading throws exceptions over already pending exceptions
 - ... and 81 more: https://git.openjdk.org/shenandoah/compare/edcc559f...3105538d

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah/pull/334/files
  Stats: 11361 lines in 390 files changed: 5944 ins; 1943 del; 3474 mod
  Patch: https://git.openjdk.org/shenandoah/pull/334.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/334/head:pull/334

PR: https://git.openjdk.org/shenandoah/pull/334

From wkemper at openjdk.org  Thu Oct  5 17:12:57 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 5 Oct 2023 17:12:57 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <VUN4vebhNwhFEcDzkoXzTHkm9Qx-iPwyBorbqFfdnIk=.32f2bcf1-31fc-40f9-a107-cc6fbcb51b60@github.com>
References: <VUN4vebhNwhFEcDzkoXzTHkm9Qx-iPwyBorbqFfdnIk=.32f2bcf1-31fc-40f9-a107-cc6fbcb51b60@github.com>
Message-ID: <VpT7SEUeHryHgRAir1kmH7K9Kbkr-KQVdy8mJK344AY=.5c1c37b1-c602-403a-97b6-2baa4b42befb@github.com>

> Merges tag jdk-22+18

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/334/files
  - new: https://git.openjdk.org/shenandoah/pull/334/files/3105538d..3105538d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=334&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=334&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/334.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/334/head:pull/334

PR: https://git.openjdk.org/shenandoah/pull/334

From wkemper at openjdk.org  Thu Oct  5 17:13:00 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 5 Oct 2023 17:13:00 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <VUN4vebhNwhFEcDzkoXzTHkm9Qx-iPwyBorbqFfdnIk=.32f2bcf1-31fc-40f9-a107-cc6fbcb51b60@github.com>
References: <VUN4vebhNwhFEcDzkoXzTHkm9Qx-iPwyBorbqFfdnIk=.32f2bcf1-31fc-40f9-a107-cc6fbcb51b60@github.com>
Message-ID: <6fKEZBsGnEMiJaCfK3TZohlCifUCcUVChQU6YbEsoUQ=.a8b2ab21-863e-4e0e-a900-a3b80ed233d9@github.com>

On Thu, 5 Oct 2023 14:16:35 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-22+18

This pull request has now been integrated.

Changeset: ddd34c5a
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/ddd34c5adb204a8dc59b629978613944c8449809
Stats:     11361 lines in 390 files changed: 5944 ins; 1943 del; 3474 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah/pull/334

From wkemper at openjdk.org  Thu Oct  5 17:15:17 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 5 Oct 2023 17:15:17 GMT
Subject: RFR: 8317534: GenShen: Remove unused code
Message-ID: <fMAkDU2SIDMJhKP7WrHYSHfaHIQ-Ykpi4VPYO74vVCU=.05d0508c-dcb1-4da4-a3d7-8097aa252c53@github.com>

Remove unused code, fix typos, remove empty statements

-------------

Commit messages:
 - remove unused variable
 - Remove unused code

Changes: https://git.openjdk.org/shenandoah/pull/335/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=335&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8317534
  Stats: 286 lines in 27 files changed: 0 ins; 267 del; 19 mod
  Patch: https://git.openjdk.org/shenandoah/pull/335.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/335/head:pull/335

PR: https://git.openjdk.org/shenandoah/pull/335

From ysr at openjdk.org  Thu Oct  5 19:01:41 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 5 Oct 2023 19:01:41 GMT
Subject: RFR: 8317534: GenShen: Remove unused code
In-Reply-To: <fMAkDU2SIDMJhKP7WrHYSHfaHIQ-Ykpi4VPYO74vVCU=.05d0508c-dcb1-4da4-a3d7-8097aa252c53@github.com>
References: <fMAkDU2SIDMJhKP7WrHYSHfaHIQ-Ykpi4VPYO74vVCU=.05d0508c-dcb1-4da4-a3d7-8097aa252c53@github.com>
Message-ID: <m5PalbxKOE3IrgOjcyV81RNtm9qnAyY2bePH3j5ivzM=.a9504e01-71b9-44ab-b6b1-9e19091e2941@github.com>

On Thu, 5 Oct 2023 17:08:52 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Remove unused code, fix typos, remove empty statements

LGTM; thanks for the clean-up.

A few somewhat minor comments.

src/hotspot/share/gc/shenandoah/shenandoahAgeCensus.hpp line 113:

> 111:   // This uses the data in the ShenandoahAgeCensus object's _global_age_table and the
> 112:   // current _epoch to compute a new tenuring threshold, which will be remembered
> 113:   // until the next invocation of compute_tenuring_threshold.

Could you please move this documentation to `compute_tenuring_threshold()` further below. Thanks!

src/hotspot/share/gc/shenandoah/shenandoahGeneration.hpp line 139:

> 137: 
> 138:   void set_soft_max_capacity(size_t soft_max_capacity) {
> 139:     _soft_max_capacity = soft_max_capacity;

Not only the accessor, but it seems like the `_soft_max_capacity` field in `ShenandoahGeneration` is vestigial and can be deleted?

A cursory look seemed to indicate that a field with a similar name (similar accessor, but confusingly different field name) is used for the purpose for that purpose for the whole heap, and the generation's field is obsolete/vestigial. I may have missed some uses in my inspection, but you can check if deleting the field is ok (it's only used in logging where it seems not to have much value?).

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 1960:

> 1958:   }
> 1959: }
> 1960: 

Nice!

src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 859:

> 857:   void clear_cards_for(ShenandoahHeapRegion* region);
> 858:   void dirty_cards(HeapWord* start, HeapWord* end);
> 859:   void clear_cards(HeapWord* start, HeapWord* end);

These aren't used, but in looking through the code for marking and clearing cards in `ShenandoahDirectCardMarkRememberedSet::mark_range_as_...` I realize we might want to see if we want to use vector instructions or use `memset`, when the range is sufficiently large (future work).

src/hotspot/share/gc/shenandoah/shenandoahScanRemembered.inline.hpp line 1026:

> 1024:   }
> 1025: }
> 1026: 

Very curious. I could have sworn I had used this in some assertions/verification code in card scanning or card verification, but may be that code went away. Does seem to be unused now, so best to get rid of it.

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/335#pullrequestreview-1660471452
PR Review Comment: https://git.openjdk.org/shenandoah/pull/335#discussion_r1347821180
PR Review Comment: https://git.openjdk.org/shenandoah/pull/335#discussion_r1347834125
PR Review Comment: https://git.openjdk.org/shenandoah/pull/335#discussion_r1347837192
PR Review Comment: https://git.openjdk.org/shenandoah/pull/335#discussion_r1347849963
PR Review Comment: https://git.openjdk.org/shenandoah/pull/335#discussion_r1347843253

From kdnilsen at openjdk.org  Thu Oct  5 23:05:57 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 5 Oct 2023 23:05:57 GMT
Subject: RFR: 8317534: GenShen: Remove unused code
In-Reply-To: <fMAkDU2SIDMJhKP7WrHYSHfaHIQ-Ykpi4VPYO74vVCU=.05d0508c-dcb1-4da4-a3d7-8097aa252c53@github.com>
References: <fMAkDU2SIDMJhKP7WrHYSHfaHIQ-Ykpi4VPYO74vVCU=.05d0508c-dcb1-4da4-a3d7-8097aa252c53@github.com>
Message-ID: <ijl51r0qb-sDOnQS6odlhdRwR4rsqqMTobcuBby-e0U=.da4efc69-7a7d-4582-8641-38952b2dad00@github.com>

On Thu, 5 Oct 2023 17:08:52 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Remove unused code, fix typos, remove empty statements

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/335#pullrequestreview-1660892155

From zgu at openjdk.org  Fri Oct  6 13:37:00 2023
From: zgu at openjdk.org (Zhengyu Gu)
Date: Fri, 6 Oct 2023 13:37:00 GMT
Subject: RFR: 8317466: Enable interpreter oopMapCache for concurrent GCs
Message-ID: <dS_92JFN-F9Jizw_KWx1T7EoHQeo6n01CzNV19Xu_iQ=.81a01ca0-0308-4951-9bbf-7bfdd565bdbd@github.com>

Interpreter oop maps are computed lazily during GC root scan and they are expensive to compute.

GCs uses a small hash table per instance class to cache computed oop maps during STW root scan, but not for concurrent root scan.

This patch is intended to enable `OopMapCache` for concurrent GCs.

Test:
  tier1 and tier2 fastdebug and release on MacOSX, Linux 86_84 and Linux 86_32.

-------------

Commit messages:
 - Fix merge conflicts
 - Merge
 - cleanup
 - Merge branch 'master' into JDK-8317466
 - Cleanup
 - Merge branch 'master' into oopmapcache_for_concurrent_root_scan
 - v2
 - v1
 - v0
 - 8317240: Promptly free OopMapEntry after fail to insert the entry to OopMapCache

Changes: https://git.openjdk.org/jdk/pull/16074/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16074&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8317466
  Stats: 56 lines in 10 files changed: 20 ins; 15 del; 21 mod
  Patch: https://git.openjdk.org/jdk/pull/16074.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/16074/head:pull/16074

PR: https://git.openjdk.org/jdk/pull/16074

From wkemper at openjdk.org  Mon Oct  9 16:55:37 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 9 Oct 2023 16:55:37 GMT
Subject: RFR: 8317535 Shenandoah: Remove unused code
Message-ID: <USH2vwBkLIKGvh8KRyDfaCbjXdqrEaOijSoemc7sUxE=.03cf83ac-3215-475a-8558-70d1e2f15319@github.com>

Tested with `hotspot_gc_shenandoah`, `specjbb`, `specjvm`, `dacapo`, `extremem` and `heapothesys`.

-------------

Commit messages:
 - Merge upstream
 - Remove unused code and other minor fixes

Changes: https://git.openjdk.org/jdk/pull/16104/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16104&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8317535
  Stats: 185 lines in 20 files changed: 0 ins; 179 del; 6 mod
  Patch: https://git.openjdk.org/jdk/pull/16104.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/16104/head:pull/16104

PR: https://git.openjdk.org/jdk/pull/16104

From kdnilsen at openjdk.org  Mon Oct  9 17:43:01 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 9 Oct 2023 17:43:01 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous allocation
Message-ID: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>

Several improvements:
1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
2. Reduce likelihood of humongous allocation errors by:
    a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
    b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.

-------------

Commit messages:
 - Fix whitespace
 - Fix accounting of mixed evac regions added to defragment old-gen for logging
 - Cleanup code for upstream
 - Fix mismatched brace
 - Misc improvements to make humongous allocation support more performant
 - Fix signed iteration termination bug
 - Add instrumentation to help debug
 - Improvements to support humongous allocation

Changes: https://git.openjdk.org/shenandoah/pull/336/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8317371
  Stats: 264 lines in 11 files changed: 213 ins; 6 del; 45 mod
  Patch: https://git.openjdk.org/shenandoah/pull/336.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/336/head:pull/336

PR: https://git.openjdk.org/shenandoah/pull/336

From wkemper at openjdk.org  Tue Oct 10 00:12:28 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 10 Oct 2023 00:12:28 GMT
Subject: RFR: 8317534: GenShen: Remove unused code [v2]
In-Reply-To: <fMAkDU2SIDMJhKP7WrHYSHfaHIQ-Ykpi4VPYO74vVCU=.05d0508c-dcb1-4da4-a3d7-8097aa252c53@github.com>
References: <fMAkDU2SIDMJhKP7WrHYSHfaHIQ-Ykpi4VPYO74vVCU=.05d0508c-dcb1-4da4-a3d7-8097aa252c53@github.com>
Message-ID: <T6f8QAAI1UN-bGljP_VqSbtI-L0WmjzZelvpqR-JfEQ=.49d382c5-ed16-4f23-ad84-7b5c9d709769@github.com>

> Remove unused code, fix typos, remove empty statements

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Restore deleted comment on correct method

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/335/files
  - new: https://git.openjdk.org/shenandoah/pull/335/files/6918e929..ec8ad460

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=335&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=335&range=00-01

  Stats: 4 lines in 1 file changed: 4 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/335.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/335/head:pull/335

PR: https://git.openjdk.org/shenandoah/pull/335

From wkemper at openjdk.org  Tue Oct 10 00:12:29 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 10 Oct 2023 00:12:29 GMT
Subject: RFR: 8317534: GenShen: Remove unused code [v2]
In-Reply-To: <m5PalbxKOE3IrgOjcyV81RNtm9qnAyY2bePH3j5ivzM=.a9504e01-71b9-44ab-b6b1-9e19091e2941@github.com>
References: <fMAkDU2SIDMJhKP7WrHYSHfaHIQ-Ykpi4VPYO74vVCU=.05d0508c-dcb1-4da4-a3d7-8097aa252c53@github.com>
 <m5PalbxKOE3IrgOjcyV81RNtm9qnAyY2bePH3j5ivzM=.a9504e01-71b9-44ab-b6b1-9e19091e2941@github.com>
Message-ID: <sHk4Wprm_6aWncIqaTpqxPQEAIMz-XZUKy2IPtD0NXg=.d8ecfc5d-4347-4929-b9c4-3b1cf6fa026a@github.com>

On Thu, 5 Oct 2023 18:27:23 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> William Kemper has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Restore deleted comment on correct method
>
> src/hotspot/share/gc/shenandoah/shenandoahAgeCensus.hpp line 113:
> 
>> 111:   // This uses the data in the ShenandoahAgeCensus object's _global_age_table and the
>> 112:   // current _epoch to compute a new tenuring threshold, which will be remembered
>> 113:   // until the next invocation of compute_tenuring_threshold.
> 
> Could you please move this documentation to `compute_tenuring_threshold()` further below. Thanks!

Done.

> src/hotspot/share/gc/shenandoah/shenandoahGeneration.hpp line 139:
> 
>> 137: 
>> 138:   void set_soft_max_capacity(size_t soft_max_capacity) {
>> 139:     _soft_max_capacity = soft_max_capacity;
> 
> Not only the accessor, but it seems like the `_soft_max_capacity` field in `ShenandoahGeneration` is vestigial and can be deleted?
> 
> A cursory look seemed to indicate that a field with a similar name (similar accessor, but confusingly different field name) is used for the purpose for that purpose for the whole heap, and the generation's field is obsolete/vestigial. I may have missed some uses in my inspection, but you can check if deleting the field is ok (it's only used in logging where it seems not to have much value?).

Hmm, this field is used by the heuristics. This looks like a regression to me. The soft max heap size can be given as a command line argument, but it is also supposed to be a 'managed' option that can be changed with jmx. I will open a separate ticket for this.

> src/hotspot/share/gc/shenandoah/shenandoahHeap.hpp line 859:
> 
>> 857:   void clear_cards_for(ShenandoahHeapRegion* region);
>> 858:   void dirty_cards(HeapWord* start, HeapWord* end);
>> 859:   void clear_cards(HeapWord* start, HeapWord* end);
> 
> These aren't used, but in looking through the code for marking and clearing cards in `ShenandoahDirectCardMarkRememberedSet::mark_range_as_...` I realize we might want to see if we want to use vector instructions or use `memset`, when the range is sufficiently large (future work).

That's a good idea!

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/335#discussion_r1351010105
PR Review Comment: https://git.openjdk.org/shenandoah/pull/335#discussion_r1351009661
PR Review Comment: https://git.openjdk.org/shenandoah/pull/335#discussion_r1351011714

From wkemper at openjdk.org  Tue Oct 10 00:15:57 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 10 Oct 2023 00:15:57 GMT
Subject: Integrated: 8317534: GenShen: Remove unused code
In-Reply-To: <fMAkDU2SIDMJhKP7WrHYSHfaHIQ-Ykpi4VPYO74vVCU=.05d0508c-dcb1-4da4-a3d7-8097aa252c53@github.com>
References: <fMAkDU2SIDMJhKP7WrHYSHfaHIQ-Ykpi4VPYO74vVCU=.05d0508c-dcb1-4da4-a3d7-8097aa252c53@github.com>
Message-ID: <fxBxK3PWj_sUUoBJ8lhKR6KAz6KnKyUK4YLqbhT-qaw=.47cc8683-8d12-4421-a02c-091190a73907@github.com>

On Thu, 5 Oct 2023 17:08:52 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Remove unused code, fix typos, remove empty statements

This pull request has now been integrated.

Changeset: f3c9edaf
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/f3c9edafa36deb2cf66f1e71ab9a7caeb527d3f8
Stats:     290 lines in 27 files changed: 4 ins; 267 del; 19 mod

8317534: GenShen: Remove unused code

Reviewed-by: ysr, kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/335

From kdnilsen at openjdk.org  Wed Oct 11 14:47:31 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 11 Oct 2023 14:47:31 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v2]
In-Reply-To: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
Message-ID: <17fGD32_U3n-ck34-esftA2LoPZHyvvSzuP6yf0myrI=.6d64d8bb-cf08-4d51-b5da-ec9cf6c35c17@github.com>

> Several improvements:
> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
> 2. Reduce likelihood of humongous allocation errors by:
>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Fix assertion to deal with promote-in-place-only evacuations

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/336/files
  - new: https://git.openjdk.org/shenandoah/pull/336/files/6f75a522..9535b511

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=00-01

  Stats: 3 lines in 1 file changed: 2 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/336.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/336/head:pull/336

PR: https://git.openjdk.org/shenandoah/pull/336

From zgu at openjdk.org  Wed Oct 11 18:50:14 2023
From: zgu at openjdk.org (Zhengyu Gu)
Date: Wed, 11 Oct 2023 18:50:14 GMT
Subject: RFR: 8317466: Enable interpreter oopMapCache for concurrent GCs
 [v2]
In-Reply-To: <dS_92JFN-F9Jizw_KWx1T7EoHQeo6n01CzNV19Xu_iQ=.81a01ca0-0308-4951-9bbf-7bfdd565bdbd@github.com>
References: <dS_92JFN-F9Jizw_KWx1T7EoHQeo6n01CzNV19Xu_iQ=.81a01ca0-0308-4951-9bbf-7bfdd565bdbd@github.com>
Message-ID: <EAurOs1oZa5lA26Cp2GxnQXB0wamCkfNVuwGTKBXE3I=.9fffd66c-c755-4d1b-9c22-2b23828dc5de@github.com>

> Interpreter oop maps are computed lazily during GC root scan and they are expensive to compute.
> 
> GCs uses a small hash table per instance class to cache computed oop maps during STW root scan, but not for concurrent root scan.
> 
> This patch is intended to enable `OopMapCache` for concurrent GCs.
> 
> Test:
>   tier1 and tier2 fastdebug and release on MacOSX, Linux 86_84 and Linux 86_32.

Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:

  Cleanup old oop map cache entry after class redefinition

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/16074/files
  - new: https://git.openjdk.org/jdk/pull/16074/files/11936030..015d4fb3

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=16074&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16074&range=00-01

  Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/16074.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/16074/head:pull/16074

PR: https://git.openjdk.org/jdk/pull/16074

From zgu at openjdk.org  Wed Oct 11 18:50:18 2023
From: zgu at openjdk.org (Zhengyu Gu)
Date: Wed, 11 Oct 2023 18:50:18 GMT
Subject: RFR: 8317466: Enable interpreter oopMapCache for concurrent GCs
 [v2]
In-Reply-To: <xp3iXMithuNCkDW6y7VDr-2ceYPUsD0iJsvOxBULIKs=.2c70ef59-a7fa-4700-ba7d-d6f876819af8@github.com>
References: <dS_92JFN-F9Jizw_KWx1T7EoHQeo6n01CzNV19Xu_iQ=.81a01ca0-0308-4951-9bbf-7bfdd565bdbd@github.com>
 <EAurOs1oZa5lA26Cp2GxnQXB0wamCkfNVuwGTKBXE3I=.9fffd66c-c755-4d1b-9c22-2b23828dc5de@github.com>
 <xp3iXMithuNCkDW6y7VDr-2ceYPUsD0iJsvOxBULIKs=.2c70ef59-a7fa-4700-ba7d-d6f876819af8@github.com>
Message-ID: <0YGIJj0crvW3iZ_fd5MlNJ8UxWPECo9XgVppni70cjQ=.391f488a-077a-47ee-9cc4-9e47bceeec65@github.com>

On Wed, 11 Oct 2023 16:40:57 GMT, Leela Mohan Venati <duke at openjdk.org> wrote:

>> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Cleanup old oop map cache entry after class redefinition
>
> src/hotspot/share/interpreter/oopMapCache.cpp line 506:
> 
>> 504: 
>> 505:       if (Atomic::cmpxchg(&_array[i], entry, (OopMapCacheEntry*)nullptr, memory_order_relaxed) == entry) {
>> 506:         enqueue_for_cleanup(entry);
> 
> OopMapCache::flush_obsolete_entries() is called from VM_RedefineClasses::doit(). Enqueuing  OopMapCacheEntries for cleanup means that we would need call OopMapCache::cleanup_old_entries() in VM_RedefineClasses::doit_epilogue()

Good catch. Fixed.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/16074#discussion_r1355594222

From duke at openjdk.org  Wed Oct 11 18:50:17 2023
From: duke at openjdk.org (Leela Mohan Venati)
Date: Wed, 11 Oct 2023 18:50:17 GMT
Subject: RFR: 8317466: Enable interpreter oopMapCache for concurrent GCs
 [v2]
In-Reply-To: <EAurOs1oZa5lA26Cp2GxnQXB0wamCkfNVuwGTKBXE3I=.9fffd66c-c755-4d1b-9c22-2b23828dc5de@github.com>
References: <dS_92JFN-F9Jizw_KWx1T7EoHQeo6n01CzNV19Xu_iQ=.81a01ca0-0308-4951-9bbf-7bfdd565bdbd@github.com>
 <EAurOs1oZa5lA26Cp2GxnQXB0wamCkfNVuwGTKBXE3I=.9fffd66c-c755-4d1b-9c22-2b23828dc5de@github.com>
Message-ID: <xp3iXMithuNCkDW6y7VDr-2ceYPUsD0iJsvOxBULIKs=.2c70ef59-a7fa-4700-ba7d-d6f876819af8@github.com>

On Wed, 11 Oct 2023 18:44:04 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> Interpreter oop maps are computed lazily during GC root scan and they are expensive to compute.
>> 
>> GCs uses a small hash table per instance class to cache computed oop maps during STW root scan, but not for concurrent root scan.
>> 
>> This patch is intended to enable `OopMapCache` for concurrent GCs.
>> 
>> Test:
>>   tier1 and tier2 fastdebug and release on MacOSX, Linux 86_84 and Linux 86_32.
>
> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Cleanup old oop map cache entry after class redefinition

src/hotspot/share/interpreter/oopMapCache.cpp line 506:

> 504: 
> 505:       if (Atomic::cmpxchg(&_array[i], entry, (OopMapCacheEntry*)nullptr, memory_order_relaxed) == entry) {
> 506:         enqueue_for_cleanup(entry);

OopMapCache::flush_obsolete_entries() is called from VM_RedefineClasses::doit(). Enqueuing  OopMapCacheEntries for cleanup means that we would need call OopMapCache::cleanup_old_entries() in VM_RedefineClasses::doit_epilogue()

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/16074#discussion_r1355360462

From kdnilsen at openjdk.org  Wed Oct 11 19:09:52 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 11 Oct 2023 19:09:52 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v3]
In-Reply-To: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
Message-ID: <3-Diw93ErtU2_lT8G1fnDe-yY6dJw-EAoBHr31BAKZw=.4eb9f20a-bc99-47fd-a0a6-b4684eab0ecd@github.com>

> Several improvements:
> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
> 2. Reduce likelihood of humongous allocation errors by:
>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Cosmetic improvements

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/336/files
  - new: https://git.openjdk.org/shenandoah/pull/336/files/9535b511..72ce60eb

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=01-02

  Stats: 4 lines in 1 file changed: 0 ins; 0 del; 4 mod
  Patch: https://git.openjdk.org/shenandoah/pull/336.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/336/head:pull/336

PR: https://git.openjdk.org/shenandoah/pull/336

From wkemper at openjdk.org  Thu Oct 12 14:28:11 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 12 Oct 2023 14:28:11 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <TpRP0bY1Puw4QI-rZgC-QFsvN7KB0qmYRP_WeP8ODgI=.5c6c2ee5-79a9-44ed-81d2-cec5236486c4@github.com>

Merges tag jdk-22+19

-------------

Commit messages:
 - 8317730: Change byte_size to return size_t
 - 8261894: Remove support for UseSHM
 - 8315505: CompileTask timestamp printed can overflow
 - 8317706: Exclude java/awt/Graphics2D/DrawString/RotTransText.java on linux
 - 8317711: Exclude gtest/GTestWrapper.java on AIX
 - 8317705: ProblemList sun/tools/jstat/jstatLineCountsX.sh on linux-ppc64le and aix due to JDK-8248691
 - 8314199: Initial size PBEKeyFactory#validTypes is not up-to-date
 - 8316233: VirtualThreadStart events should not be thread-filtered
 - 8316447: 8 sun/management/jmxremote tests ignore VM flags
 - 8316446: 4 sun/management/jdp tests ignore VM flags
 - ... and 33 more: https://git.openjdk.org/shenandoah/compare/3105538d...fb4098ff

The merge commit only contains trivial merges, so no merge-specific webrevs have been generated.

Changes: https://git.openjdk.org/shenandoah/pull/338/files
  Stats: 6567 lines in 136 files changed: 3827 ins; 1892 del; 848 mod
  Patch: https://git.openjdk.org/shenandoah/pull/338.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/338/head:pull/338

PR: https://git.openjdk.org/shenandoah/pull/338

From kdnilsen at openjdk.org  Thu Oct 12 14:35:40 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 12 Oct 2023 14:35:40 GMT
Subject: RFR: 8318019: GenShen: Fix assertion to allow empty evacuation
 cycles
In-Reply-To: <FsBK0AGTCuelUNrYU3I-Nc22gQhNoPf7QlqplIdMMGA=.85219dd9-023e-40b0-be25-724a2d3ba7d3@github.com>
References: <glX1l7PTRkck36Mxt0vB88laebthOBzpbYyd_D2hyGg=.24ea8525-dc1c-4f86-bfc8-ebf9c9fd9540@github.com>
 <FsBK0AGTCuelUNrYU3I-Nc22gQhNoPf7QlqplIdMMGA=.85219dd9-023e-40b0-be25-724a2d3ba7d3@github.com>
Message-ID: <4ILuSU_tp-lUo0UMPIkzQqBrSV9bf7YIrnWJN566SJk=.da970f0b-c7f5-4828-93de-e388d3f89aab@github.com>

On Wed, 11 Oct 2023 17:26:56 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> Assertion needs to allow for gc phase EVACUATION even if no forwarding objects because we are only promoting in place.
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2428:
> 
>> 2426:   bool has_forwarded = has_forwarded_objects()? 1: 0;
>> 2427:   bool updating_or_evacuating = _gc_state.is_set(UPDATEREFS | EVACUATION)? 1: 0;
>> 2428:   bool evacuating = gc_state.is_set(EVACUATION)? 1: 0;
> 
> suggestion: 
> `
> bool evacuating = gc_state.is_set(EVACUATION);
> `
> pre-exist:
> `bool has_forwarded = has_forwarded_objects()? 1: 0;` to `bool has_forwarded = has_forwarded_objects();`

Thanks.  I'll fix those, and my misspelling of gc_state.

> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2429:
> 
>> 2427:   bool updating_or_evacuating = _gc_state.is_set(UPDATEREFS | EVACUATION);
>> 2428:   bool evacuating = _gc_state.is_set(EVACUATION);
>> 2429:   assert ((has_forwarded == updating_or_evacuating) || (evacuating && !has_forwarded && collection_set()->is_empty()),
> 
> Sorry, I did not realize this is generational Shenandoah. I am a bit puzzled why setting `EVACUATION` flag if collection set is empty?

It's a trick we use for a form of "abbreviated" cycle.  Sometimes, we have nothing to evacuate, but we have a bunch of regions that reside in young, which are old enough to be promoted.  If they are populated with a sufficient amount of live data, we promote them "in place" by simply relabeling the affiliation of the region rather than evacuating it.  But there's a non-trivial amount of processing required on the content of the region when it becomes old, so that the region can now maintain a remembered set of pointers from old to young.  We normally do this processing as part of the evacuation cycle.  In very rare scenarios, we'll end up with an evacuation cycle that only deals with promote-in-place regions, and nothing else...  That's the special scenario we're talking about here.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/337#discussion_r1355449317
PR Review Comment: https://git.openjdk.org/shenandoah/pull/337#discussion_r1355848454

From kdnilsen at openjdk.org  Thu Oct 12 14:35:37 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 12 Oct 2023 14:35:37 GMT
Subject: RFR: 8318019: GenShen: Fix assertion to allow empty evacuation cycles
Message-ID: <glX1l7PTRkck36Mxt0vB88laebthOBzpbYyd_D2hyGg=.24ea8525-dc1c-4f86-bfc8-ebf9c9fd9540@github.com>

Assertion needs to allow for gc phase EVACUATION even if no forwarding objects because we are only promoting in place.

-------------

Commit messages:
 - Fix typo and style issues
 - Relax forwarding assertion to account for promote-in-place evacuations

Changes: https://git.openjdk.org/shenandoah/pull/337/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=337&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8318019
  Stats: 5 lines in 1 file changed: 2 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/shenandoah/pull/337.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/337/head:pull/337

PR: https://git.openjdk.org/shenandoah/pull/337

From kdnilsen at openjdk.org  Thu Oct 12 14:35:39 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 12 Oct 2023 14:35:39 GMT
Subject: RFR: 8318019: GenShen: Fix assertion to allow empty evacuation
 cycles
In-Reply-To: <glX1l7PTRkck36Mxt0vB88laebthOBzpbYyd_D2hyGg=.24ea8525-dc1c-4f86-bfc8-ebf9c9fd9540@github.com>
References: <glX1l7PTRkck36Mxt0vB88laebthOBzpbYyd_D2hyGg=.24ea8525-dc1c-4f86-bfc8-ebf9c9fd9540@github.com>
Message-ID: <iqKLv5sSBeblRcusds2wpdGavNzHsKixitY1_ijhU0Q=.74c5e150-0237-48b7-9a30-b811bda1cf32@github.com>

On Wed, 11 Oct 2023 16:27:18 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Assertion needs to allow for gc phase EVACUATION even if no forwarding objects because we are only promoting in place.

After testing, I'll match this to a JBS issue.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/337#issuecomment-1758066648

From zgu at openjdk.org  Thu Oct 12 14:35:41 2023
From: zgu at openjdk.org (Zhengyu Gu)
Date: Thu, 12 Oct 2023 14:35:41 GMT
Subject: RFR: 8318019: GenShen: Fix assertion to allow empty evacuation
 cycles
In-Reply-To: <4ILuSU_tp-lUo0UMPIkzQqBrSV9bf7YIrnWJN566SJk=.da970f0b-c7f5-4828-93de-e388d3f89aab@github.com>
References: <glX1l7PTRkck36Mxt0vB88laebthOBzpbYyd_D2hyGg=.24ea8525-dc1c-4f86-bfc8-ebf9c9fd9540@github.com>
 <FsBK0AGTCuelUNrYU3I-Nc22gQhNoPf7QlqplIdMMGA=.85219dd9-023e-40b0-be25-724a2d3ba7d3@github.com>
 <4ILuSU_tp-lUo0UMPIkzQqBrSV9bf7YIrnWJN566SJk=.da970f0b-c7f5-4828-93de-e388d3f89aab@github.com>
Message-ID: <_WZm0dP8wAeWGqBZDdgH2-YLnBfIxVoK6PlVo_V7CKY=.e83853de-ecb0-431b-b5a1-4bb664687213@github.com>

On Wed, 11 Oct 2023 22:35:13 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2429:
>> 
>>> 2427:   bool updating_or_evacuating = _gc_state.is_set(UPDATEREFS | EVACUATION);
>>> 2428:   bool evacuating = _gc_state.is_set(EVACUATION);
>>> 2429:   assert ((has_forwarded == updating_or_evacuating) || (evacuating && !has_forwarded && collection_set()->is_empty()),
>> 
>> Sorry, I did not realize this is generational Shenandoah. I am a bit puzzled why setting `EVACUATION` flag if collection set is empty?
>
> It's a trick we use for a form of "abbreviated" cycle.  Sometimes, we have nothing to evacuate, but we have a bunch of regions that reside in young, which are old enough to be promoted.  If they are populated with a sufficient amount of live data, we promote them "in place" by simply relabeling the affiliation of the region rather than evacuating it.  But there's a non-trivial amount of processing required on the content of the region when it becomes old, so that the region can now maintain a remembered set of pointers from old to young.  We normally do this processing as part of the evacuation cycle.  In very rare scenarios, we'll end up with an evacuation cycle that only deals with promote-in-place regions, and nothing else...  That's the special scenario we're talking about here.

Thanks for the explanation!

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/337#discussion_r1355866169

From zgu at openjdk.org  Thu Oct 12 14:35:38 2023
From: zgu at openjdk.org (Zhengyu Gu)
Date: Thu, 12 Oct 2023 14:35:38 GMT
Subject: RFR: 8318019: GenShen: Fix assertion to allow empty evacuation
 cycles
In-Reply-To: <glX1l7PTRkck36Mxt0vB88laebthOBzpbYyd_D2hyGg=.24ea8525-dc1c-4f86-bfc8-ebf9c9fd9540@github.com>
References: <glX1l7PTRkck36Mxt0vB88laebthOBzpbYyd_D2hyGg=.24ea8525-dc1c-4f86-bfc8-ebf9c9fd9540@github.com>
Message-ID: <FsBK0AGTCuelUNrYU3I-Nc22gQhNoPf7QlqplIdMMGA=.85219dd9-023e-40b0-be25-724a2d3ba7d3@github.com>

On Wed, 11 Oct 2023 16:27:18 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Assertion needs to allow for gc phase EVACUATION even if no forwarding objects because we are only promoting in place.

Changes requested by zgu (Committer).

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2428:

> 2426:   bool has_forwarded = has_forwarded_objects()? 1: 0;
> 2427:   bool updating_or_evacuating = _gc_state.is_set(UPDATEREFS | EVACUATION)? 1: 0;
> 2428:   bool evacuating = gc_state.is_set(EVACUATION)? 1: 0;

suggestion: 
`
bool evacuating = gc_state.is_set(EVACUATION);
`
pre-exist:
`bool has_forwarded = has_forwarded_objects()? 1: 0;` to `bool has_forwarded = has_forwarded_objects();`

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 2429:

> 2427:   bool updating_or_evacuating = _gc_state.is_set(UPDATEREFS | EVACUATION);
> 2428:   bool evacuating = _gc_state.is_set(EVACUATION);
> 2429:   assert ((has_forwarded == updating_or_evacuating) || (evacuating && !has_forwarded && collection_set()->is_empty()),

Sorry, I did not realize this is generational Shenandoah. I am a bit puzzled why setting `EVACUATION` flag if collection set is empty?

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/337#pullrequestreview-1672023292
PR Review Comment: https://git.openjdk.org/shenandoah/pull/337#discussion_r1355435553
PR Review Comment: https://git.openjdk.org/shenandoah/pull/337#discussion_r1355609862

From wkemper at openjdk.org  Thu Oct 12 17:13:04 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 12 Oct 2023 17:13:04 GMT
Subject: RFR: 8318019: GenShen: Fix assertion to allow empty evacuation
 cycles
In-Reply-To: <glX1l7PTRkck36Mxt0vB88laebthOBzpbYyd_D2hyGg=.24ea8525-dc1c-4f86-bfc8-ebf9c9fd9540@github.com>
References: <glX1l7PTRkck36Mxt0vB88laebthOBzpbYyd_D2hyGg=.24ea8525-dc1c-4f86-bfc8-ebf9c9fd9540@github.com>
Message-ID: <XKE7ljsOuQfvztpII4NlJoibXQfAALYnR6xv7UKVtPo=.083866e0-6566-4e1d-9157-aa5d3ff11256@github.com>

On Wed, 11 Oct 2023 16:27:18 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Assertion needs to allow for gc phase EVACUATION even if no forwarding objects because we are only promoting in place.

Marked as reviewed by wkemper (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/337#pullrequestreview-1674731832

From kdnilsen at openjdk.org  Thu Oct 12 17:49:36 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 12 Oct 2023 17:49:36 GMT
Subject: Integrated: 8318019: GenShen: Fix assertion to allow empty evacuation
 cycles
In-Reply-To: <glX1l7PTRkck36Mxt0vB88laebthOBzpbYyd_D2hyGg=.24ea8525-dc1c-4f86-bfc8-ebf9c9fd9540@github.com>
References: <glX1l7PTRkck36Mxt0vB88laebthOBzpbYyd_D2hyGg=.24ea8525-dc1c-4f86-bfc8-ebf9c9fd9540@github.com>
Message-ID: <VvVuapKs20LDzGUR-zSwoXkXZYFyEZxFxxDkzfBhU9w=.bd10555a-3004-466e-92aa-f8c0f829adc3@github.com>

On Wed, 11 Oct 2023 16:27:18 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Assertion needs to allow for gc phase EVACUATION even if no forwarding objects because we are only promoting in place.

This pull request has now been integrated.

Changeset: 401acb05
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/401acb05ecfd6e57e1a81678f3374e7809a5a97e
Stats:     5 lines in 1 file changed: 2 ins; 0 del; 3 mod

8318019: GenShen: Fix assertion to allow empty evacuation cycles

Reviewed-by: wkemper

-------------

PR: https://git.openjdk.org/shenandoah/pull/337

From wkemper at openjdk.org  Thu Oct 12 18:05:31 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 12 Oct 2023 18:05:31 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v3]
In-Reply-To: <3-Diw93ErtU2_lT8G1fnDe-yY6dJw-EAoBHr31BAKZw=.4eb9f20a-bc99-47fd-a0a6-b4684eab0ecd@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
 <3-Diw93ErtU2_lT8G1fnDe-yY6dJw-EAoBHr31BAKZw=.4eb9f20a-bc99-47fd-a0a6-b4684eab0ecd@github.com>
Message-ID: <VvO_JspOaG9Z-7dQ9o6QMZXArtZudOxey8vRqh8rlic=.4eefa9d5-e922-4a92-9538-86bc6a454978@github.com>

On Wed, 11 Oct 2023 19:09:52 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Several improvements:
>> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
>> 2. Reduce likelihood of humongous allocation errors by:
>>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Cosmetic improvements

Changes requested by wkemper (Committer).

src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 405:

> 403:   if (cand_idx > _last_old_collection_candidate) {
> 404:     // Sort the regions that were initially rejected from the collection set in order of index.
> 405:     QuickSort::sort<RegionData>(candidates + _last_old_collection_candidate, cand_idx - _last_old_collection_candidate,

What is the purpose of sorting these regions by index? How does this aid defragmentation?

src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 562:

> 560:     size_t first_active_region = heap->free_set()->first_old_region();
> 561:     size_t last_active_region = heap->free_set()->last_old_region();
> 562:     size_t span_of_active_regions = (last_active_region > first_active_region)? last_active_region + 1 - first_active_region: 0;

Why did we compute `span_of_active_regions`? Doesn't seem used. If we don't need `span_of_active_regions`, do we still need the new methods: `first_old_region` and `last_old_region`?

src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 566:

> 564:     size_t first_old_region, last_old_region;
> 565:     double density;
> 566:     fragmentation_trigger_reason(density, first_old_region, last_old_region);

Just use the member fields here?

src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 572:

> 570: 
> 571:     // New active regions may have came into play following the trigger.
> 572:     size_t first_region = MIN2(first_active_region, first_old_region);

I don't understand this. The active region counts are not updated after the trigger?

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 3115:

> 3113:   size_t young_cset_regions, old_cset_regions;
> 3114:   size_t first_old_region, last_old_region, old_region_count;
> 3115:   _free_set->prepare_to_rebuild(young_cset_regions, old_cset_regions, first_old_region, last_old_region, old_region_count);

Should we have an assert here like:
```C++
assert(first_old_region == _free_set->first_old_region() && last_old_region == _free_set->last_old_region());

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 3163:

> 3161: 
> 3162:     uint eighths = 8;
> 3163:     bool triggered = false;

`triggered` looks unused?

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/336#pullrequestreview-1674779498
PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1357200341
PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1357190839
PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1357198127
PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1357195186
PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1357188019
PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1357172088

From wkemper at openjdk.org  Thu Oct 12 18:10:12 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 12 Oct 2023 18:10:12 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v3]
In-Reply-To: <3-Diw93ErtU2_lT8G1fnDe-yY6dJw-EAoBHr31BAKZw=.4eb9f20a-bc99-47fd-a0a6-b4684eab0ecd@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
 <3-Diw93ErtU2_lT8G1fnDe-yY6dJw-EAoBHr31BAKZw=.4eb9f20a-bc99-47fd-a0a6-b4684eab0ecd@github.com>
Message-ID: <koVOzZ_MJktVDkCUGjDZMQB3ILMk0-QPFHCI6eD6zQw=.09093d4d-8164-422b-833c-cda6d5fe40a2@github.com>

On Wed, 11 Oct 2023 19:09:52 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Several improvements:
>> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
>> 2. Reduce likelihood of humongous allocation errors by:
>>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Cosmetic improvements

src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 570:

> 568:     case ShenandoahAllocRequest::_alloc_shared: {
> 569:       // Try to allocate in the mutator view
> 570:       // Allocate within mutator free from high memory to low so as to preserve low memory for humongous allocations

> a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory

Don't we usually take the collector reserve from the top end of the heap? Would it make sense to assume humongous objects tend to be long lived and allocate them from the back of the heap?

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1357209856

From kdnilsen at openjdk.org  Thu Oct 12 19:37:54 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 12 Oct 2023 19:37:54 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v3]
In-Reply-To: <koVOzZ_MJktVDkCUGjDZMQB3ILMk0-QPFHCI6eD6zQw=.09093d4d-8164-422b-833c-cda6d5fe40a2@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
 <3-Diw93ErtU2_lT8G1fnDe-yY6dJw-EAoBHr31BAKZw=.4eb9f20a-bc99-47fd-a0a6-b4684eab0ecd@github.com>
 <koVOzZ_MJktVDkCUGjDZMQB3ILMk0-QPFHCI6eD6zQw=.09093d4d-8164-422b-833c-cda6d5fe40a2@github.com>
Message-ID: <cAEwpFjbUuhSfNLVVqWcNdscoAgppTkfL8Usc2qXnDg=.72670385-f09f-4314-9e89-7d5e7713f22e@github.com>

On Thu, 12 Oct 2023 18:07:32 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Cosmetic improvements
>
> src/hotspot/share/gc/shenandoah/shenandoahFreeSet.cpp line 570:
> 
>> 568:     case ShenandoahAllocRequest::_alloc_shared: {
>> 569:       // Try to allocate in the mutator view
>> 570:       // Allocate within mutator free from high memory to low so as to preserve low memory for humongous allocations
> 
>> a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
> 
> Don't we usually take the collector reserve from the top end of the heap? Would it make sense to assume humongous objects tend to be long lived and allocate them from the back of the heap?

You are correct that the top of heap has Old Collector Reserve, then Collector reserve.  So there is an established practice of putting longer-lived objects at the top of the heap.  In the existing implementation, we start the search for a humongous heap region at the bottom of the heap.

I think it will not matter much whether we allocate humongous from top or bottom of heap.  The important thing is that we do not allocate long-lived object in the "middle" of the heap, as this hinders allocation of humongous objects.  It feels a little bit simpler to me to keep the humongous objects in a different memory than the rest of the old collector reserve (and retired old-gen regions).  The top of memory experiences churn as the result of mixed evacuations to "compact" old-gen memory, whereas the bottom of memory should be largely empty most of the time.

The problem occurs when the mutator has to allocate all the way to the bottom of the heap before GC finishes and replenishes the mutator free set.  If some of the bottom-of-memory mutator allocations are subsequently promoted in place (and this has been observed in practice), we end up with fragmentation of the low-memory address range that we want to reserve for humongous allocations.  This is when we trigger old_is_fragmented(), so these bottom-of-memory promoted-in-place regions can be evacuated into top-of-memory regions, and humongous allocation space can be restored.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1357311562

From wkemper at openjdk.org  Thu Oct 12 23:50:31 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 12 Oct 2023 23:50:31 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <TpRP0bY1Puw4QI-rZgC-QFsvN7KB0qmYRP_WeP8ODgI=.5c6c2ee5-79a9-44ed-81d2-cec5236486c4@github.com>
References: <TpRP0bY1Puw4QI-rZgC-QFsvN7KB0qmYRP_WeP8ODgI=.5c6c2ee5-79a9-44ed-81d2-cec5236486c4@github.com>
Message-ID: <vJyQqKY6hZqDKP-AIEskCzURrcxqZKbBDatmi99KGWc=.2152c139-5e6b-4d82-a22e-013af7c69374@github.com>

> Merges tag jdk-22+19

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/338/files
  - new: https://git.openjdk.org/shenandoah/pull/338/files/fb4098ff..fb4098ff

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=338&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=338&range=00-01

  Stats: 0 lines in 0 files changed: 0 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/338.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/338/head:pull/338

PR: https://git.openjdk.org/shenandoah/pull/338

From wkemper at openjdk.org  Thu Oct 12 23:50:34 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 12 Oct 2023 23:50:34 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <TpRP0bY1Puw4QI-rZgC-QFsvN7KB0qmYRP_WeP8ODgI=.5c6c2ee5-79a9-44ed-81d2-cec5236486c4@github.com>
References: <TpRP0bY1Puw4QI-rZgC-QFsvN7KB0qmYRP_WeP8ODgI=.5c6c2ee5-79a9-44ed-81d2-cec5236486c4@github.com>
Message-ID: <6LeLrPMX2EFEDTVIA4JWdVEBCySMKx7ctyqxqOCycgI=.19ca2791-07e8-4453-9caa-15b1043c8fd2@github.com>

On Thu, 12 Oct 2023 14:16:56 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-22+19

This pull request has now been integrated.

Changeset: 206e5b82
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/206e5b82f6a4eddc2513dd45da3ce3b657b4a8bc
Stats:     6567 lines in 136 files changed: 3827 ins; 1892 del; 848 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah/pull/338

From wkemper at openjdk.org  Thu Oct 12 23:59:06 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 12 Oct 2023 23:59:06 GMT
Subject: RFR: 8318053: GenShen: Control thread may ignore requests to start
 concurrent GC
Message-ID: <bRPndrpxRK_afp-KzJbLu25_hJVqA61YsESrDL5TVfQ=.cbf6ce15-0886-4a44-a745-ac7b633e8243@github.com>

If a request to interrupt an old cycle is accepted, but there is an allocation failure before the requested cycle begins, then the control thread may become stuck in a state where it believes it should ignore further concurrent cycle requests.

-------------

Commit messages:
 - Clear preemption requested flag when old gc is cancelled for allocation failure
 - Log when control thread does _not_ start a cycle

Changes: https://git.openjdk.org/shenandoah/pull/339/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=339&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8318053
  Stats: 8 lines in 1 file changed: 8 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/339.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/339/head:pull/339

PR: https://git.openjdk.org/shenandoah/pull/339

From kdnilsen at openjdk.org  Thu Oct 12 23:59:07 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 12 Oct 2023 23:59:07 GMT
Subject: RFR: 8318053: GenShen: Control thread may ignore requests to start
 concurrent GC
In-Reply-To: <bRPndrpxRK_afp-KzJbLu25_hJVqA61YsESrDL5TVfQ=.cbf6ce15-0886-4a44-a745-ac7b633e8243@github.com>
References: <bRPndrpxRK_afp-KzJbLu25_hJVqA61YsESrDL5TVfQ=.cbf6ce15-0886-4a44-a745-ac7b633e8243@github.com>
Message-ID: <e-k5gOPJhOe4ihvRCh4VxECZ8Z6VTMLY57hP82kUui8=.3d319ddd-9fd1-49b0-a4cb-71ffdf6301e1@github.com>

On Thu, 12 Oct 2023 23:42:43 GMT, William Kemper <wkemper at openjdk.org> wrote:

> If a request to interrupt an old cycle is accepted, but there is an allocation failure before the requested cycle begins, then the control thread may become stuck in a state where it believes it should ignore further concurrent cycle requests.

Thanks for figuring this out.

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/339#pullrequestreview-1675419262

From ysr at openjdk.org  Thu Oct 12 23:59:08 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 12 Oct 2023 23:59:08 GMT
Subject: RFR: 8318053: GenShen: Control thread may ignore requests to start
 concurrent GC
In-Reply-To: <bRPndrpxRK_afp-KzJbLu25_hJVqA61YsESrDL5TVfQ=.cbf6ce15-0886-4a44-a745-ac7b633e8243@github.com>
References: <bRPndrpxRK_afp-KzJbLu25_hJVqA61YsESrDL5TVfQ=.cbf6ce15-0886-4a44-a745-ac7b633e8243@github.com>
Message-ID: <vvWv7J5RDHU_A0vhdx2Vx_EwYJvinlaunDeOf8GsRH8=.67eddbef-4606-435a-bec0-753bdc8f075b@github.com>

On Thu, 12 Oct 2023 23:42:43 GMT, William Kemper <wkemper at openjdk.org> wrote:

> If a request to interrupt an old cycle is accepted, but there is an allocation failure before the requested cycle begins, then the control thread may become stuck in a state where it believes it should ignore further concurrent cycle requests.

Thanks for the extra logging!

Changes look good to me.

Can you add into the ticket the command-line that was used for inducing the errant behavior?

Also can you add into this PR how you tested to confirm that it no longer occurs after the fix?

Thanks!

-------------

Marked as reviewed by ysr (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/339#pullrequestreview-1675423288

From kdnilsen at openjdk.org  Thu Oct 12 23:59:55 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 12 Oct 2023 23:59:55 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v3]
In-Reply-To: <VvO_JspOaG9Z-7dQ9o6QMZXArtZudOxey8vRqh8rlic=.4eefa9d5-e922-4a92-9538-86bc6a454978@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
 <3-Diw93ErtU2_lT8G1fnDe-yY6dJw-EAoBHr31BAKZw=.4eb9f20a-bc99-47fd-a0a6-b4684eab0ecd@github.com>
 <VvO_JspOaG9Z-7dQ9o6QMZXArtZudOxey8vRqh8rlic=.4eefa9d5-e922-4a92-9538-86bc6a454978@github.com>
Message-ID: <LaYm273iW8XvHCcDWgJaG9wIgFLYBi1RAJFIreV0XUs=.8e79bc82-3e68-498d-a1e6-3ea5da29a39f@github.com>

On Thu, 12 Oct 2023 17:37:59 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Cosmetic improvements
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 3163:
> 
>> 3161: 
>> 3162:     uint eighths = 8;
>> 3163:     bool triggered = false;
> 
> `triggered` looks unused?

Thanks. Removing.  That was helping me with some diagnostic logging which I removed after debugging was "complete".

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1357573037

From wkemper at openjdk.org  Fri Oct 13 00:07:59 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 13 Oct 2023 00:07:59 GMT
Subject: RFR: 8318053: GenShen: Control thread may ignore requests to start
 concurrent GC
In-Reply-To: <e-k5gOPJhOe4ihvRCh4VxECZ8Z6VTMLY57hP82kUui8=.3d319ddd-9fd1-49b0-a4cb-71ffdf6301e1@github.com>
References: <bRPndrpxRK_afp-KzJbLu25_hJVqA61YsESrDL5TVfQ=.cbf6ce15-0886-4a44-a745-ac7b633e8243@github.com>
 <e-k5gOPJhOe4ihvRCh4VxECZ8Z6VTMLY57hP82kUui8=.3d319ddd-9fd1-49b0-a4cb-71ffdf6301e1@github.com>
Message-ID: <t9csGGGO4O4E2McREjbqIfpfpsTDR5xKaENlNixr-40=.95e16cab-f2b2-42ae-8c40-3d739fcdc810@github.com>

On Thu, 12 Oct 2023 23:49:41 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> If a request to interrupt an old cycle is accepted, but there is an allocation failure before the requested cycle begins, then the control thread may become stuck in a state where it believes it should ignore further concurrent cycle requests.
>
> Thanks for figuring this out.

I updated the ticket with the command we used to reproduce it. I've seen the issue on the first run and @kdnilsen has seen it usually before the 10th run. If I can get 50 runs without the issue, I'll consider it fixed. 

Also, the additional logging showed the `_preeption_requested` flag was stuck on, when it shouldn't have been.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/339#issuecomment-1760557556

From kdnilsen at openjdk.org  Fri Oct 13 00:16:04 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 13 Oct 2023 00:16:04 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v3]
In-Reply-To: <VvO_JspOaG9Z-7dQ9o6QMZXArtZudOxey8vRqh8rlic=.4eefa9d5-e922-4a92-9538-86bc6a454978@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
 <3-Diw93ErtU2_lT8G1fnDe-yY6dJw-EAoBHr31BAKZw=.4eb9f20a-bc99-47fd-a0a6-b4684eab0ecd@github.com>
 <VvO_JspOaG9Z-7dQ9o6QMZXArtZudOxey8vRqh8rlic=.4eefa9d5-e922-4a92-9538-86bc6a454978@github.com>
Message-ID: <sGO3zxp8qtRvmSZ2ly8Q2zcmruaWjIyh_6bDQC8si3M=.4b538cef-f659-4640-88fb-298c041bac38@github.com>

On Thu, 12 Oct 2023 17:51:23 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Cosmetic improvements
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 562:
> 
>> 560:     size_t first_active_region = heap->free_set()->first_old_region();
>> 561:     size_t last_active_region = heap->free_set()->last_old_region();
>> 562:     size_t span_of_active_regions = (last_active_region > first_active_region)? last_active_region + 1 - first_active_region: 0;
> 
> Why did we compute `span_of_active_regions`? Doesn't seem used. If we don't need `span_of_active_regions`, do we still need the new methods: `first_old_region` and `last_old_region`?

Thanks for this catch also.  I went through a few iterations of how to implement this, and forgot to clean up all of my experiments.  Am removing this code and am removing first_old_region() and last_old_region().  We only "need" the first and last old region info immediately after we rebuild the Shenandoah Free Set.  Since the rebuild process already iterates over all regions, it's best to compute the information during that pass.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1357584893

From kdnilsen at openjdk.org  Fri Oct 13 00:20:54 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 13 Oct 2023 00:20:54 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v3]
In-Reply-To: <VvO_JspOaG9Z-7dQ9o6QMZXArtZudOxey8vRqh8rlic=.4eefa9d5-e922-4a92-9538-86bc6a454978@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
 <3-Diw93ErtU2_lT8G1fnDe-yY6dJw-EAoBHr31BAKZw=.4eb9f20a-bc99-47fd-a0a6-b4684eab0ecd@github.com>
 <VvO_JspOaG9Z-7dQ9o6QMZXArtZudOxey8vRqh8rlic=.4eefa9d5-e922-4a92-9538-86bc6a454978@github.com>
Message-ID: <eaHvgjIoeZMs5ndtzcvYyQFHYRnrOZkPqBa23UShuW4=.e6ac9209-5f1f-46ca-864f-c52f4e209ddf@github.com>

On Thu, 12 Oct 2023 17:48:42 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Cosmetic improvements
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 3115:
> 
>> 3113:   size_t young_cset_regions, old_cset_regions;
>> 3114:   size_t first_old_region, last_old_region, old_region_count;
>> 3115:   _free_set->prepare_to_rebuild(young_cset_regions, old_cset_regions, first_old_region, last_old_region, old_region_count);
> 
> Should we have an assert here like:
> ```C++
> assert(first_old_region == _free_set->first_old_region() && last_old_region == _free_set->last_old_region());

Since we don't really need the first_old_region() and last_old_region() methods of ShenandoahFreeSet, they are removed, so we cannot write this assertion.  I'll add a "comparable" assertion to assure sanity.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1357587633

From kdnilsen at openjdk.org  Fri Oct 13 00:36:06 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 13 Oct 2023 00:36:06 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v3]
In-Reply-To: <VvO_JspOaG9Z-7dQ9o6QMZXArtZudOxey8vRqh8rlic=.4eefa9d5-e922-4a92-9538-86bc6a454978@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
 <3-Diw93ErtU2_lT8G1fnDe-yY6dJw-EAoBHr31BAKZw=.4eb9f20a-bc99-47fd-a0a6-b4684eab0ecd@github.com>
 <VvO_JspOaG9Z-7dQ9o6QMZXArtZudOxey8vRqh8rlic=.4eefa9d5-e922-4a92-9538-86bc6a454978@github.com>
Message-ID: <Newg6ulj4v3-5tfyLnx5CO34Xsmx24K01Oq6cIY62yU=.bb94e05d-6684-438c-849b-e1b5f6521a62@github.com>

On Thu, 12 Oct 2023 17:55:31 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Cosmetic improvements
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 572:
> 
>> 570: 
>> 571:     // New active regions may have came into play following the trigger.
>> 572:     size_t first_region = MIN2(first_active_region, first_old_region);
> 
> I don't understand this. The active region counts are not updated after the trigger?

I agree this is confusing.  I'm going to simplify this code and have the log message report the data that drove the trigger.  The reality is that between the moment in time that we raised the trigger (from inside ShenandoahHeap::rebuild_free_set(), and the moment that the regulator polls the old heuristic's state to discover that we should_start_gc(), the state may change so this log message may be slightly out of date.  This would be extremely unlikely.  The trigger happens at final-update-refs.  The old-gen state will not change until we begin a subsequent evacuation phase.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1357601310

From kdnilsen at openjdk.org  Fri Oct 13 00:41:06 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 13 Oct 2023 00:41:06 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v3]
In-Reply-To: <VvO_JspOaG9Z-7dQ9o6QMZXArtZudOxey8vRqh8rlic=.4eefa9d5-e922-4a92-9538-86bc6a454978@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
 <3-Diw93ErtU2_lT8G1fnDe-yY6dJw-EAoBHr31BAKZw=.4eb9f20a-bc99-47fd-a0a6-b4684eab0ecd@github.com>
 <VvO_JspOaG9Z-7dQ9o6QMZXArtZudOxey8vRqh8rlic=.4eefa9d5-e922-4a92-9538-86bc6a454978@github.com>
Message-ID: <3vXpepGcrIdcbx_a5v5YJKOsOlpXiKh0_Bjtm5p8-nc=.1b5c31b3-5afb-4c0d-8c44-2555f158b189@github.com>

On Thu, 12 Oct 2023 18:00:21 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Cosmetic improvements
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 405:
> 
>> 403:   if (cand_idx > _last_old_collection_candidate) {
>> 404:     // Sort the regions that were initially rejected from the collection set in order of index.
>> 405:     QuickSort::sort<RegionData>(candidates + _last_old_collection_candidate, cand_idx - _last_old_collection_candidate,
> 
> What is the purpose of sorting these regions by index? How does this aid defragmentation?

I will add a comment to clarify.

> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 566:
> 
>> 564:     size_t first_old_region, last_old_region;
>> 565:     double density;
>> 566:     fragmentation_trigger_reason(density, first_old_region, last_old_region);
> 
> Just use the member fields here?

I'm not sure which member fields you are recommending here.

ShenandoahFreeSet remembers first and last indexes of the OldCollector free set, but the range we are interested in here includes all retired old-gen regions in addition to the free regions.

If I'm misunderstanding your suggestion, please clarify.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1357605590
PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1357604320

From kdnilsen at openjdk.org  Fri Oct 13 00:41:08 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 13 Oct 2023 00:41:08 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v3]
In-Reply-To: <3vXpepGcrIdcbx_a5v5YJKOsOlpXiKh0_Bjtm5p8-nc=.1b5c31b3-5afb-4c0d-8c44-2555f158b189@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
 <3-Diw93ErtU2_lT8G1fnDe-yY6dJw-EAoBHr31BAKZw=.4eb9f20a-bc99-47fd-a0a6-b4684eab0ecd@github.com>
 <VvO_JspOaG9Z-7dQ9o6QMZXArtZudOxey8vRqh8rlic=.4eefa9d5-e922-4a92-9538-86bc6a454978@github.com>
 <3vXpepGcrIdcbx_a5v5YJKOsOlpXiKh0_Bjtm5p8-nc=.1b5c31b3-5afb-4c0d-8c44-2555f158b189@github.com>
Message-ID: <nrh0PYfMucy4CQONGgn9c_y5zo7HDPKl663IbqJFquQ=.cea00125-5812-45f6-b482-69600f88cc50@github.com>

On Fri, 13 Oct 2023 00:37:06 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 566:
>> 
>>> 564:     size_t first_old_region, last_old_region;
>>> 565:     double density;
>>> 566:     fragmentation_trigger_reason(density, first_old_region, last_old_region);
>> 
>> Just use the member fields here?
>
> I'm not sure which member fields you are recommending here.
> 
> ShenandoahFreeSet remembers first and last indexes of the OldCollector free set, but the range we are interested in here includes all retired old-gen regions in addition to the free regions.
> 
> If I'm misunderstanding your suggestion, please clarify.

Not sure which member fields.  Please clarify.

>> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 572:
>> 
>>> 570: 
>>> 571:     // New active regions may have came into play following the trigger.
>>> 572:     size_t first_region = MIN2(first_active_region, first_old_region);
>> 
>> I don't understand this. The active region counts are not updated after the trigger?
>
> I agree this is confusing.  I'm going to simplify this code and have the log message report the data that drove the trigger.  The reality is that between the moment in time that we raised the trigger (from inside ShenandoahHeap::rebuild_free_set(), and the moment that the regulator polls the old heuristic's state to discover that we should_start_gc(), the state may change so this log message may be slightly out of date.  This would be extremely unlikely.  The trigger happens at final-update-refs.  The old-gen state will not change until we begin a subsequent evacuation phase.

Will simplify this code to make it more clear.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1357605401
PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1357604738

From kdnilsen at openjdk.org  Fri Oct 13 01:24:44 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 13 Oct 2023 01:24:44 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v4]
In-Reply-To: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
Message-ID: <Z97ZfMjgTg60bMrSMrm_zqN16Dtzyy6cTjKzWmGVCYM=.63a7de55-89c3-4a7b-b065-2a0e3c488aac@github.com>

> Several improvements:
> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
> 2. Reduce likelihood of humongous allocation errors by:
>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.

Kelvin Nilsen has updated the pull request incrementally with two additional commits since the last revision:

 - Respond to reviewer feedback
 - Experiment with 0 for default humongous reserve

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/336/files
  - new: https://git.openjdk.org/shenandoah/pull/336/files/72ce60eb..0c471ab4

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=03
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=02-03

  Stats: 57 lines in 5 files changed: 12 ins; 40 del; 5 mod
  Patch: https://git.openjdk.org/shenandoah/pull/336.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/336/head:pull/336

PR: https://git.openjdk.org/shenandoah/pull/336

From kdnilsen at openjdk.org  Fri Oct 13 01:29:46 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 13 Oct 2023 01:29:46 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v5]
In-Reply-To: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
Message-ID: <2Et1bXaWkWrLB9J2QdVYkdVNcFQ5EOV0j5cgARG-CbE=.084072c4-8d98-49b6-a398-8446ad168746@github.com>

> Several improvements:
> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
> 2. Reduce likelihood of humongous allocation errors by:
>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.

Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 13 commits:

 - Merge master
 - Respond to reviewer feedback
 - Experiment with 0 for default humongous reserve
 - Cosmetic improvements
 - Fix assertion to deal with promote-in-place-only evacuations
 - Fix whitespace
 - Fix accounting of mixed evac regions added to defragment old-gen for logging
 - Cleanup code for upstream
 - Fix mismatched brace
 - Misc improvements to make humongous allocation support more performant
 - ... and 3 more: https://git.openjdk.org/shenandoah/compare/206e5b82...041abc3f

-------------

Changes: https://git.openjdk.org/shenandoah/pull/336/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=04
  Stats: 237 lines in 11 files changed: 185 ins; 6 del; 46 mod
  Patch: https://git.openjdk.org/shenandoah/pull/336.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/336/head:pull/336

PR: https://git.openjdk.org/shenandoah/pull/336

From kdnilsen at openjdk.org  Fri Oct 13 17:25:02 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 13 Oct 2023 17:25:02 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v6]
In-Reply-To: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
Message-ID: <XBiwcRX2PWgQzBO1bSuB046m38okcM-ELYULJW-JmbM=.19183498-e0a4-4885-9654-de961b241ab3@github.com>

> Several improvements:
> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
> 2. Reduce likelihood of humongous allocation errors by:
>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Restore _humongous_alloc_failure_gc flag removed in merge conflict

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/336/files
  - new: https://git.openjdk.org/shenandoah/pull/336/files/041abc3f..9dd5dc7c

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=05
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=04-05

  Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/336.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/336/head:pull/336

PR: https://git.openjdk.org/shenandoah/pull/336

From wkemper at openjdk.org  Fri Oct 13 17:43:47 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 13 Oct 2023 17:43:47 GMT
Subject: Integrated: 8318053: GenShen: Control thread may ignore requests to
 start concurrent GC
In-Reply-To: <bRPndrpxRK_afp-KzJbLu25_hJVqA61YsESrDL5TVfQ=.cbf6ce15-0886-4a44-a745-ac7b633e8243@github.com>
References: <bRPndrpxRK_afp-KzJbLu25_hJVqA61YsESrDL5TVfQ=.cbf6ce15-0886-4a44-a745-ac7b633e8243@github.com>
Message-ID: <I5cdUa252FerKqUqL7pebm3KjeUAcMehlDjKr8vTb7A=.e76e379e-e4f6-4221-91f4-a2952a473c7e@github.com>

On Thu, 12 Oct 2023 23:42:43 GMT, William Kemper <wkemper at openjdk.org> wrote:

> If a request to interrupt an old cycle is accepted, but there is an allocation failure before the requested cycle begins, then the control thread may become stuck in a state where it believes it should ignore further concurrent cycle requests.

This pull request has now been integrated.

Changeset: ccbb0d85
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/ccbb0d85c2ec2fb94deaa60d84719f84f050d967
Stats:     8 lines in 1 file changed: 8 ins; 0 del; 0 mod

8318053: GenShen: Control thread may ignore requests to start concurrent GC

Reviewed-by: kdnilsen, ysr

-------------

PR: https://git.openjdk.org/shenandoah/pull/339

From wkemper at openjdk.org  Fri Oct 13 18:05:56 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 13 Oct 2023 18:05:56 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v3]
In-Reply-To: <nrh0PYfMucy4CQONGgn9c_y5zo7HDPKl663IbqJFquQ=.cea00125-5812-45f6-b482-69600f88cc50@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
 <3-Diw93ErtU2_lT8G1fnDe-yY6dJw-EAoBHr31BAKZw=.4eb9f20a-bc99-47fd-a0a6-b4684eab0ecd@github.com>
 <VvO_JspOaG9Z-7dQ9o6QMZXArtZudOxey8vRqh8rlic=.4eefa9d5-e922-4a92-9538-86bc6a454978@github.com>
 <3vXpepGcrIdcbx_a5v5YJKOsOlpXiKh0_Bjtm5p8-nc=.1b5c31b3-5afb-4c0d-8c44-2555f158b189@github.com>
 <nrh0PYfMucy4CQONGgn9c_y5zo7HDPKl663IbqJFquQ=.cea00125-5812-45f6-b482-69600f88cc50@github.com>
Message-ID: <0YwQrRd2UH7R772tyG2wPvEr__rypL_71AmUAfDL8K4=.ea2751b2-7a1e-4ae8-a2c7-276cda8d8969@github.com>

On Fri, 13 Oct 2023 00:38:09 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> I'm not sure which member fields you are recommending here.
>> 
>> ShenandoahFreeSet remembers first and last indexes of the OldCollector free set, but the range we are interested in here includes all retired old-gen regions in addition to the free regions.
>> 
>> If I'm misunderstanding your suggestion, please clarify.
>
> Not sure which member fields.  Please clarify.

It looks like `fragmentation_trigger_reason` just copies the values of the member fields into its arguments.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1358640725

From wkemper at openjdk.org  Fri Oct 13 19:12:03 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 13 Oct 2023 19:12:03 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v6]
In-Reply-To: <XBiwcRX2PWgQzBO1bSuB046m38okcM-ELYULJW-JmbM=.19183498-e0a4-4885-9654-de961b241ab3@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
 <XBiwcRX2PWgQzBO1bSuB046m38okcM-ELYULJW-JmbM=.19183498-e0a4-4885-9654-de961b241ab3@github.com>
Message-ID: <htpPDjwhWnvOQwIDXHDbu2a4_jh_Cm17dorT0k_Merg=.ec158efe-d339-4407-95dd-e55a314ae760@github.com>

On Fri, 13 Oct 2023 17:25:02 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Several improvements:
>> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
>> 2. Reduce likelihood of humongous allocation errors by:
>>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Restore _humongous_alloc_failure_gc flag removed in merge conflict

> If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first

Can we set `ShenandoahFullGCThreshold` back to the upstream default of 3 now? It should have been done when we merged https://github.com/openjdk/jdk/pull/15500 .

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/336#issuecomment-1762055716

From kdnilsen at openjdk.org  Fri Oct 13 19:26:40 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 13 Oct 2023 19:26:40 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v3]
In-Reply-To: <0YwQrRd2UH7R772tyG2wPvEr__rypL_71AmUAfDL8K4=.ea2751b2-7a1e-4ae8-a2c7-276cda8d8969@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
 <3-Diw93ErtU2_lT8G1fnDe-yY6dJw-EAoBHr31BAKZw=.4eb9f20a-bc99-47fd-a0a6-b4684eab0ecd@github.com>
 <VvO_JspOaG9Z-7dQ9o6QMZXArtZudOxey8vRqh8rlic=.4eefa9d5-e922-4a92-9538-86bc6a454978@github.com>
 <3vXpepGcrIdcbx_a5v5YJKOsOlpXiKh0_Bjtm5p8-nc=.1b5c31b3-5afb-4c0d-8c44-2555f158b189@github.com>
 <nrh0PYfMucy4CQONGgn9c_y5zo7HDPKl663IbqJFquQ=.cea00125-5812-45f6-b482-69600f88cc50@github.com>
 <0YwQrRd2UH7R772tyG2wPvEr__rypL_71AmUAfDL8K4=.ea2751b2-7a1e-4ae8-a2c7-276cda8d8969@github.com>
Message-ID: <XnKW598w0w5ngjNXNyVp78jaKKhtWZLUwHIEBaNJreQ=.47ac9996-afc3-42e1-a061-5dedbc78070b@github.com>

On Fri, 13 Oct 2023 18:03:33 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Not sure which member fields.  Please clarify.
>
> It looks like `fragmentation_trigger_reason` just copies the values of the member fields into its arguments.

Thanks for clarifying.  I kind of like the encapsulating the update as is, because it makes more clear to the code maintainer what these fields represent, and how they got their values.  For performance, I'll add inline to the function definition.  Is that ok?

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1358724079

From rkennke at openjdk.org  Fri Oct 13 19:35:05 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Fri, 13 Oct 2023 19:35:05 GMT
Subject: RFR: 8317535 Shenandoah: Remove unused code
In-Reply-To: <USH2vwBkLIKGvh8KRyDfaCbjXdqrEaOijSoemc7sUxE=.03cf83ac-3215-475a-8558-70d1e2f15319@github.com>
References: <USH2vwBkLIKGvh8KRyDfaCbjXdqrEaOijSoemc7sUxE=.03cf83ac-3215-475a-8558-70d1e2f15319@github.com>
Message-ID: <IyppC079asO5-0-FaujjlfGSTHHWrLZA-14idlQj6Zc=.a98f3f46-7631-4780-bc8d-9939eab8a595@github.com>

On Mon, 9 Oct 2023 16:47:32 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Tested with `hotspot_gc_shenandoah`, `specjbb`, `specjvm`, `dacapo`, `extremem` and `heapothesys`.

Nice cleanup! Thank you!
I'm really quite baffled by some of it, but OTOH, in all cases it looks obviously ok if compiler doesn't complain. *shrugs*

-------------

Marked as reviewed by rkennke (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/16104#pullrequestreview-1677215067

From kdnilsen at openjdk.org  Fri Oct 13 19:46:31 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 13 Oct 2023 19:46:31 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v7]
In-Reply-To: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
Message-ID: <GWUpbKeq5q2eILFxng7wnHc3PYpva5ucGHf5YvipKFg=.758708ca-6d91-4461-8e44-f18ed0c7b9d4@github.com>

> Several improvements:
> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
> 2. Reduce likelihood of humongous allocation errors by:
>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  In-line two small single-use functions

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/336/files
  - new: https://git.openjdk.org/shenandoah/pull/336/files/9dd5dc7c..15a89810

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=06
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=05-06

  Stats: 3 lines in 1 file changed: 1 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/336.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/336/head:pull/336

PR: https://git.openjdk.org/shenandoah/pull/336

From ysr at openjdk.org  Fri Oct 13 21:40:41 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Fri, 13 Oct 2023 21:40:41 GMT
Subject: RFR: 8317535 Shenandoah: Remove unused code
In-Reply-To: <USH2vwBkLIKGvh8KRyDfaCbjXdqrEaOijSoemc7sUxE=.03cf83ac-3215-475a-8558-70d1e2f15319@github.com>
References: <USH2vwBkLIKGvh8KRyDfaCbjXdqrEaOijSoemc7sUxE=.03cf83ac-3215-475a-8558-70d1e2f15319@github.com>
Message-ID: <kYBB6EozBDHAfdgVPY63g0DoI-6cZ2g5D--HyMON98E=.7bda0c16-dd09-4781-8dd2-ea5eb016d552@github.com>

On Mon, 9 Oct 2023 16:47:32 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Tested with `hotspot_gc_shenandoah`, `specjbb`, `specjvm`, `dacapo`, `extremem` and `heapothesys`.

Nice clean up indeed; thank you!

I am in awe of how you found these (discarded) needles in the proverbial haystack.

-------------

Marked as reviewed by ysr (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/16104#pullrequestreview-1677399323

From wkemper at openjdk.org  Fri Oct 13 21:54:54 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 13 Oct 2023 21:54:54 GMT
Subject: RFR: 8317535 Shenandoah: Remove unused code
In-Reply-To: <USH2vwBkLIKGvh8KRyDfaCbjXdqrEaOijSoemc7sUxE=.03cf83ac-3215-475a-8558-70d1e2f15319@github.com>
References: <USH2vwBkLIKGvh8KRyDfaCbjXdqrEaOijSoemc7sUxE=.03cf83ac-3215-475a-8558-70d1e2f15319@github.com>
Message-ID: <ELJXGSQtyFoO31AeHgMsaKDFzDpR89rOg2v98vCneBs=.2cfd5663-6bc3-4033-90b4-297b53da5b1d@github.com>

On Mon, 9 Oct 2023 16:47:32 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Tested with `hotspot_gc_shenandoah`, `specjbb`, `specjvm`, `dacapo`, `extremem` and `heapothesys`.

The power of the IDE!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16104#issuecomment-1762282997

From wkemper at openjdk.org  Fri Oct 13 21:59:06 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 13 Oct 2023 21:59:06 GMT
Subject: Integrated: 8317535 Shenandoah: Remove unused code
In-Reply-To: <USH2vwBkLIKGvh8KRyDfaCbjXdqrEaOijSoemc7sUxE=.03cf83ac-3215-475a-8558-70d1e2f15319@github.com>
References: <USH2vwBkLIKGvh8KRyDfaCbjXdqrEaOijSoemc7sUxE=.03cf83ac-3215-475a-8558-70d1e2f15319@github.com>
Message-ID: <qt5pq7NA2lApCCQCzQirlHEctwrTPah-8hb5O8hVWTU=.a3aa7072-51d8-4eee-95dd-e072d8c1266d@github.com>

On Mon, 9 Oct 2023 16:47:32 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Tested with `hotspot_gc_shenandoah`, `specjbb`, `specjvm`, `dacapo`, `extremem` and `heapothesys`.

This pull request has now been integrated.

Changeset: e942f368
Author:    William Kemper <wkemper at openjdk.org>
Committer: Y. Srinivas Ramakrishna <ysr at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/e942f368c370e059c654e33408940a987013a5c7
Stats:     185 lines in 20 files changed: 0 ins; 179 del; 6 mod

8317535: Shenandoah: Remove unused code

Reviewed-by: rkennke, ysr

-------------

PR: https://git.openjdk.org/jdk/pull/16104

From ayang at openjdk.org  Mon Oct 16 13:31:41 2023
From: ayang at openjdk.org (Albert Mingkun Yang)
Date: Mon, 16 Oct 2023 13:31:41 GMT
Subject: RFR: 8317350: Move code cache purging out of
 CodeCache::UnloadingScope
In-Reply-To: <BvcvMniLilmj4oHWHZRAratUJar05TJ1XptU_XoPHm0=.c1c84329-b5e0-4493-8133-aa528dbfb081@github.com>
References: <BvcvMniLilmj4oHWHZRAratUJar05TJ1XptU_XoPHm0=.c1c84329-b5e0-4493-8133-aa528dbfb081@github.com>
Message-ID: <dLVgIGPlDVJ4dhVBVLocMmVB_6Qxdmkv_P_BulJCcRA=.d620ce8e-ad55-4849-81ab-ecb5c0dbae7c@github.com>

On Mon, 2 Oct 2023 12:56:27 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

> Hi all,
> 
>   please review this refactoring that moves actual code cache flushing/purging out of `CodeCache::UnloadingScope`. Reasons:
> 
> * I prefer that a destructor does not do anything substantial - in some cases, 90% of time is spent in the destructor in that extracted method (due to https://bugs.openjdk.org/browse/JDK-8316959)
> * imho it does not fit the class which does nothing but sets/resets some code cache unloading behavior (probably should be renamed to `UnloadingBehaviorScope` too in a separate CR).
> * other existing methods at that level are placed out of that (or any other) scope object too - which is already the case for when doing concurrent unloading.
> * putting it there makes future logging of the various phases a little bit easier, not having `GCTraceTimer` et al. in various places.
> 
> Testing: gha
> 
> Thanks,
>   Thomas

src/hotspot/share/gc/parallel/psParallelCompact.cpp line 2068:

> 2066: 
> 2067:     // Release unloaded nmethods's memory.
> 2068:     CodeCache::flush_unlinked_nmethods();

The fact that recycling resources for nmethods involves two steps, unlink and free-unlinked, seems an insignificant impl detail in this caller context. Can that be hidden away from the public API, e.g. `do_unloading` perform these two steps directly?

Is there a reason why `flush_unlinked_nmethods` is outside the scope?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/16011#discussion_r1360667446

From tschatzl at openjdk.org  Mon Oct 16 14:40:49 2023
From: tschatzl at openjdk.org (Thomas Schatzl)
Date: Mon, 16 Oct 2023 14:40:49 GMT
Subject: RFR: 8317350: Move code cache purging out of
 CodeCache::UnloadingScope
In-Reply-To: <dLVgIGPlDVJ4dhVBVLocMmVB_6Qxdmkv_P_BulJCcRA=.d620ce8e-ad55-4849-81ab-ecb5c0dbae7c@github.com>
References: <BvcvMniLilmj4oHWHZRAratUJar05TJ1XptU_XoPHm0=.c1c84329-b5e0-4493-8133-aa528dbfb081@github.com>
 <dLVgIGPlDVJ4dhVBVLocMmVB_6Qxdmkv_P_BulJCcRA=.d620ce8e-ad55-4849-81ab-ecb5c0dbae7c@github.com>
Message-ID: <I5PkYOgKATueEt99THsckfm6rkZMd2ar1KrTHJtLkO4=.e1de91e1-18ea-4836-878d-d06d1a35c2a0@github.com>

On Mon, 16 Oct 2023 13:28:03 GMT, Albert Mingkun Yang <ayang at openjdk.org> wrote:

>> Hi all,
>> 
>>   please review this refactoring that moves actual code cache flushing/purging out of `CodeCache::UnloadingScope`. Reasons:
>> 
>> * I prefer that a destructor does not do anything substantial - in some cases, 90% of time is spent in the destructor in that extracted method (due to https://bugs.openjdk.org/browse/JDK-8316959)
>> * imho it does not fit the class which does nothing but sets/resets some code cache unloading behavior (probably should be renamed to `UnloadingBehaviorScope` too in a separate CR).
>> * other existing methods at that level are placed out of that (or any other) scope object too - which is already the case for when doing concurrent unloading.
>> * putting it there makes future logging of the various phases a little bit easier, not having `GCTraceTimer` et al. in various places.
>> 
>> Testing: gha
>> 
>> Thanks,
>>   Thomas
>
> src/hotspot/share/gc/parallel/psParallelCompact.cpp line 2068:
> 
>> 2066: 
>> 2067:     // Release unloaded nmethods's memory.
>> 2068:     CodeCache::flush_unlinked_nmethods();
> 
> The fact that recycling resources for nmethods involves two steps, unlink and free-unlinked, seems an insignificant impl detail in this caller context. Can that be hidden away from the public API, e.g. `do_unloading` perform these two steps directly?
> 
> Is there a reason why `flush_unlinked_nmethods` is outside the scope?

It is maybe an insignificant detail in the context of stw collectors, but for gcs doing concurrent class/code unloading there needs to be a handshake/memory synchronization between unlinking and freeing, so these two operations need to be split.

I.e. what they do at a high level is:

  unlink classes
  unlink code cache
  unlink other stuff

  handshake

  free unlinked classes
  free unlinked code
  free other stuff

Similarly I would like to split G1 class unloading into a STW part (unlinking stuff) and make all the freeing concurrent to avoid the additional (mostly) implementation overhead. Or at least start with that and then see if it is worth making everything concurrent. (The unlinking can be trimmed down further afaics).

Back to this CR: as the description states I think the current code wrongly hides this free-unlinked procedure in that `UnloadingScope` (there is a reason it is only used for the STW collectors) unnecessarily making the observed behavior (of one of the most time-consuming parts of class/code unloading) surprising.

Putting it on this level also allows more straightforward logging.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/16011#discussion_r1360764085

From shade at openjdk.org  Mon Oct 16 17:45:31 2023
From: shade at openjdk.org (Aleksey Shipilev)
Date: Mon, 16 Oct 2023 17:45:31 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
 [v4]
In-Reply-To: <b1TOHHtcREYY327DX1za8LbbzqQw5IjuqMPxjWLxKo0=.e211440b-622d-41a3-bf68-652c4849cfd5@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
 <b1TOHHtcREYY327DX1za8LbbzqQw5IjuqMPxjWLxKo0=.e211440b-622d-41a3-bf68-652c4849cfd5@github.com>
Message-ID: <dpVdNvCePi5UyqYkUmvHz8ALa4VjtzmBT5a74QYxUZo=.e51d0cef-54d8-47b1-9d3a-17b50ee8ec8a@github.com>

On Mon, 2 Oct 2023 21:31:13 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Shenandoah will run back-to-back full GCs and _almost_ grind mutator progress to a halt before eventually exhausting memory. This change will have Shenandoah raise a gc threshold exceeded exception if the collector fails to make progress after `ShenandoahNoProgressThreshold` full GC cycles (default is 3).
>
> William Kemper has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Merge check for no-progress into retry allocation block
>  - Revert change to TEST.groups

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 887:

> 885:       size_t original_count = get_gc_no_progress_count() + 1;
> 886:       while (result == nullptr && original_count > get_gc_no_progress_count()) {
> 887:         if (!req.is_lab_alloc() && get_gc_no_progress_count() > ShenandoahNoProgressThreshold) {

I am confused by this logic. 

So we come in here. Let's assume we come in with good conditions, `no_progress_count` (`npc`) is `0`. So `original_count` is `1`. We proceed to the loop. `original_count > npc` is true, so we enter. Suppose we finish the cycle without progress. `npc` is now `1`. We circle back to the loop. `original_count > npc` now fails, and we exit. At which point the `npc > ShenandoahNoProgressThreshold` comes into picture?

I think the new code got `gc_count` confused with `no_progress_count`, really?

test/jdk/com/sun/jdi/EATests.java line 274:

> 272:         public final boolean DeoptimizeObjectsALot;
> 273:         public final boolean DoEscapeAnalysis;
> 274:         public final boolean ConcurrentGCIsSelected;

Let's do explicit `ZGCIsSelected` and `ShenandoahIsSelected`. Unfortunately, "Concurrent GC" is fairly ambiguous.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15852#discussion_r1361045071
PR Review Comment: https://git.openjdk.org/jdk/pull/15852#discussion_r1361023475

From duke at openjdk.org  Mon Oct 16 20:12:25 2023
From: duke at openjdk.org (Leela Mohan Venati)
Date: Mon, 16 Oct 2023 20:12:25 GMT
Subject: RFR: 8317466: Enable interpreter oopMapCache for concurrent GCs
 [v2]
In-Reply-To: <EAurOs1oZa5lA26Cp2GxnQXB0wamCkfNVuwGTKBXE3I=.9fffd66c-c755-4d1b-9c22-2b23828dc5de@github.com>
References: <dS_92JFN-F9Jizw_KWx1T7EoHQeo6n01CzNV19Xu_iQ=.81a01ca0-0308-4951-9bbf-7bfdd565bdbd@github.com>
 <EAurOs1oZa5lA26Cp2GxnQXB0wamCkfNVuwGTKBXE3I=.9fffd66c-c755-4d1b-9c22-2b23828dc5de@github.com>
Message-ID: <39f-0nlOdQBABHr1cQOq7jITuRLzM9yLDUEUm1--0N8=.f3aeed11-7151-4885-8376-7d91ca84e8a7@github.com>

On Wed, 11 Oct 2023 18:50:14 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> Interpreter oop maps are computed lazily during GC root scan and they are expensive to compute.
>> 
>> GCs uses a small hash table per instance class to cache computed oop maps during STW root scan, but not for concurrent root scan.
>> 
>> This patch is intended to enable `OopMapCache` for concurrent GCs.
>> 
>> Test:
>>   tier1 and tier2 fastdebug and release on MacOSX, Linux 86_84 and Linux 86_32.
>
> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Cleanup old oop map cache entry after class redefinition

Changes requested by leelamv at github.com (no known OpenJDK username).

src/hotspot/share/gc/shenandoah/shenandoahVMOperations.cpp line 64:

> 62:   OopMapCache::cleanup_old_entries();
> 63: }
> 64: 

Do you think, VM_ShenandoahFinalMarkStartEvac walks the stack roots.  If yes, i recommend adding OopMapCache::cleanup_old_entries() in  VM_ShenandoahOperation::doit_epilogue().  And this would make the change simple and also revert the change in this [PR](https://github.com/openjdk/jdk/pull/15921)

src/hotspot/share/oops/method.cpp line 311:

> 309: void Method::mask_for(int bci, InterpreterOopMap* mask) {
> 310:   methodHandle h_this(Thread::current(), this);
> 311:   method_holder()->mask_for(h_this, bci, mask);

Removing this condition allows all the threads including java threads to use/mutate oopMapCache. 

For ex: Java threads calls [JVM_CallStackWalk](https://github.com/openjdk/jdk/blob/741ae06c55de65dcdfe38e328022bd8dde4fa007/src/hotspot/share/prims/jvm.cpp#L586) which walks the stack and calls locals() and expressions [here](https://github.com/openjdk/jdk/blob/741ae06c55de65dcdfe38e328022bd8dde4fa007/src/hotspot/share/prims/stackwalk.cpp#L345)  which access oopMapCache.

-------------

PR Review: https://git.openjdk.org/jdk/pull/16074#pullrequestreview-1680858067
PR Review Comment: https://git.openjdk.org/jdk/pull/16074#discussion_r1361210493
PR Review Comment: https://git.openjdk.org/jdk/pull/16074#discussion_r1361202988

From kdnilsen at openjdk.org  Mon Oct 16 20:35:17 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 16 Oct 2023 20:35:17 GMT
Subject: RFR: 8318190: GenShen: Be less aggressive with triggers for old-gen
 has expanded
Message-ID: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>

This patch reduces the frequency of old-has-grown triggers in the case that old usage is smaller than a particular threshold percentage (default value 5%) of the total heap size.  In this case, old is triggered for growth no more frequently than once every N cycles, where N defaults to 50.

-------------

Commit messages:
 - Restrict old triggers by size and frequency

Changes: https://git.openjdk.org/shenandoah/pull/341/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=341&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8318190
  Stats: 52 lines in 5 files changed: 48 ins; 2 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/341.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/341/head:pull/341

PR: https://git.openjdk.org/shenandoah/pull/341

From kdnilsen at openjdk.org  Mon Oct 16 20:35:18 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 16 Oct 2023 20:35:18 GMT
Subject: RFR: 8318190: GenShen: Be less aggressive with triggers for
 old-gen has expanded
In-Reply-To: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
References: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
Message-ID: <Ci2QT2gYGPsHD6cd1-sfa2gQTQuJ8eZg-7OkhQczPu8=.bbd9d9e1-abea-4553-833a-1929004641ee@github.com>

On Fri, 13 Oct 2023 19:16:39 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> This patch reduces the frequency of old-has-grown triggers in the case that old usage is smaller than a particular threshold percentage (default value 5%) of the total heap size.  In this case, old is triggered for growth no more frequently than once every N cycles, where N defaults to 50.

I'll leave this in draft mode until I pass tests.  I will link to a JBS issue when I mark PR as ready for review.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/341#issuecomment-1762064461

From kdnilsen at openjdk.org  Mon Oct 16 22:19:47 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 16 Oct 2023 22:19:47 GMT
Subject: RFR: 8318190: GenShen: Be less aggressive with triggers for
 old-gen has expanded [v2]
In-Reply-To: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
References: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
Message-ID: <Fo5CbEca3x8GkJa3-8TvsOg-QkMrtmnN9GwZhIbE550=.288e9bd2-c7c8-40fd-8648-a54f86040904@github.com>

> This patch reduces the frequency of old-has-grown triggers in the case that old usage is smaller than a particular threshold percentage (default value 5%) of the total heap size.  In this case, old is triggered for growth no more frequently than once every N cycles, where N defaults to 50.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Make minimum old-gen size growth configurable

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/341/files
  - new: https://git.openjdk.org/shenandoah/pull/341/files/955f0faa..22070f03

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=341&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=341&range=00-01

  Stats: 22 lines in 3 files changed: 15 ins; 1 del; 6 mod
  Patch: https://git.openjdk.org/shenandoah/pull/341.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/341/head:pull/341

PR: https://git.openjdk.org/shenandoah/pull/341

From kdnilsen at openjdk.org  Mon Oct 16 23:17:34 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 16 Oct 2023 23:17:34 GMT
Subject: RFR: 8318190: GenShen: Be less aggressive with triggers for
 old-gen has expanded [v3]
In-Reply-To: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
References: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
Message-ID: <ZnUOPZdxgn_oUAWUwl3M86YQIv1WRd5KqW3Pq4jrUa8=.ffbc2b01-da31-463f-b28b-6468a13d44cf@github.com>

> This patch reduces the frequency of old-has-grown triggers in the case that old usage is smaller than a particular threshold percentage (default value 5%) of the total heap size.  In this case, old is triggered for growth no more frequently than once every N cycles, where N defaults to 50.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Fix whitespace

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/341/files
  - new: https://git.openjdk.org/shenandoah/pull/341/files/22070f03..e7747100

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=341&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=341&range=01-02

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/341.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/341/head:pull/341

PR: https://git.openjdk.org/shenandoah/pull/341

From kdnilsen at openjdk.org  Mon Oct 16 23:20:39 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 16 Oct 2023 23:20:39 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v8]
In-Reply-To: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
Message-ID: <TCdnJwkftlGD4YxAurKRsHGFqNw4qbd2hf8rbok7P0Y=.4987c3be-e4d6-4939-a568-232a68204d1f@github.com>

> Several improvements:
> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
> 2. Reduce likelihood of humongous allocation errors by:
>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Restore ShenandoahFullGCThreshold to 3

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/336/files
  - new: https://git.openjdk.org/shenandoah/pull/336/files/15a89810..d87d50f4

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=07
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=06-07

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/336.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/336/head:pull/336

PR: https://git.openjdk.org/shenandoah/pull/336

From kdnilsen at openjdk.org  Mon Oct 16 23:31:20 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 16 Oct 2023 23:31:20 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v9]
In-Reply-To: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
Message-ID: <Q45l3ME3hRTT6s7W7gwqsNkWpXTP0EjDir9m1ldXu-A=.2391b2c7-c4ed-43e8-8d8a-53710fdec390@github.com>

> Several improvements:
> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
> 2. Reduce likelihood of humongous allocation errors by:
>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Fix assertion to deal with humongous accounting

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/336/files
  - new: https://git.openjdk.org/shenandoah/pull/336/files/d87d50f4..baaba77c

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=08
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=07-08

  Stats: 2 lines in 1 file changed: 1 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/336.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/336/head:pull/336

PR: https://git.openjdk.org/shenandoah/pull/336

From kdnilsen at openjdk.org  Mon Oct 16 23:46:27 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 16 Oct 2023 23:46:27 GMT
Subject: Integrated: 8318201: GenShen: Remove preparatory young GC before
 bootstrap
Message-ID: <lwHw22c6fbsFQ2r7qfwpoRly9EBvkHQDlgm_1L0sa8Y=.2368f6a9-81ec-460f-9048-57fedd270dd1@github.com>

We currently precede every old-mark bootstrap gc with a young gc.  The intent was to replenish the young allocation pool before beginning the "long" coalesce-and-fill effort which is part of the bootstrap gc.  This current design is mis-guided because the coalesce-and-fill effort will be preempted if a young gc is triggered while we are working on coalesce-and-fill.

This PR has the potential of reducing the number of concurrent young gcs by the number of times we initiate an old bootstrap gcs.

-------------

Commit messages:
 - Remove preparatory young GC before boostrap cycle

Changes: https://git.openjdk.org/shenandoah/pull/340/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=340&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8318201
  Stats: 23 lines in 1 file changed: 0 ins; 19 del; 4 mod
  Patch: https://git.openjdk.org/shenandoah/pull/340.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/340/head:pull/340

PR: https://git.openjdk.org/shenandoah/pull/340

From wkemper at openjdk.org  Mon Oct 16 23:46:27 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 16 Oct 2023 23:46:27 GMT
Subject: Integrated: 8318201: GenShen: Remove preparatory young GC before
 bootstrap
In-Reply-To: <lwHw22c6fbsFQ2r7qfwpoRly9EBvkHQDlgm_1L0sa8Y=.2368f6a9-81ec-460f-9048-57fedd270dd1@github.com>
References: <lwHw22c6fbsFQ2r7qfwpoRly9EBvkHQDlgm_1L0sa8Y=.2368f6a9-81ec-460f-9048-57fedd270dd1@github.com>
Message-ID: <PTW4dUQjz_BhvXYvprobW3YGnqXjVfTxOiXhDTWX0xI=.e2d998cb-c6bc-43ee-ac50-abfdbfb0d315@github.com>

On Fri, 13 Oct 2023 15:14:15 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> We currently precede every old-mark bootstrap gc with a young gc.  The intent was to replenish the young allocation pool before beginning the "long" coalesce-and-fill effort which is part of the bootstrap gc.  This current design is mis-guided because the coalesce-and-fill effort will be preempted if a young gc is triggered while we are working on coalesce-and-fill.
> 
> This PR has the potential of reducing the number of concurrent young gcs by the number of times we initiate an old bootstrap gcs.

LGTM!

-------------

Marked as reviewed by wkemper (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/340#pullrequestreview-1680707627

From kdnilsen at openjdk.org  Mon Oct 16 23:46:28 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 16 Oct 2023 23:46:28 GMT
Subject: Integrated: 8318201: GenShen: Remove preparatory young GC before
 bootstrap
In-Reply-To: <lwHw22c6fbsFQ2r7qfwpoRly9EBvkHQDlgm_1L0sa8Y=.2368f6a9-81ec-460f-9048-57fedd270dd1@github.com>
References: <lwHw22c6fbsFQ2r7qfwpoRly9EBvkHQDlgm_1L0sa8Y=.2368f6a9-81ec-460f-9048-57fedd270dd1@github.com>
Message-ID: <BZVyad7qPvI8LrEAXXDLHOODXlRvkL9Ht_mcVPqxvQU=.c185f127-0c68-4e68-95a1-830eac28179a@github.com>

On Fri, 13 Oct 2023 15:14:15 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> We currently precede every old-mark bootstrap gc with a young gc.  The intent was to replenish the young allocation pool before beginning the "long" coalesce-and-fill effort which is part of the bootstrap gc.  This current design is mis-guided because the coalesce-and-fill effort will be preempted if a young gc is triggered while we are working on coalesce-and-fill.
> 
> This PR has the potential of reducing the number of concurrent young gcs by the number of times we initiate an old bootstrap gcs.

I'll leave this in draft mode until we pass tests.  I'll associate with a JBS ticket when I mark read to review.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/340#issuecomment-1761686181

From kdnilsen at openjdk.org  Mon Oct 16 23:46:29 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 16 Oct 2023 23:46:29 GMT
Subject: Integrated: 8318201: GenShen: Remove preparatory young GC before
 bootstrap
In-Reply-To: <lwHw22c6fbsFQ2r7qfwpoRly9EBvkHQDlgm_1L0sa8Y=.2368f6a9-81ec-460f-9048-57fedd270dd1@github.com>
References: <lwHw22c6fbsFQ2r7qfwpoRly9EBvkHQDlgm_1L0sa8Y=.2368f6a9-81ec-460f-9048-57fedd270dd1@github.com>
Message-ID: <zrx15m1yrIfIlv23lfZj5lT0FYV6xaNnKOwF9VdD5RA=.00796646-f989-4f59-9196-72bf96179845@github.com>

On Fri, 13 Oct 2023 15:14:15 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> We currently precede every old-mark bootstrap gc with a young gc.  The intent was to replenish the young allocation pool before beginning the "long" coalesce-and-fill effort which is part of the bootstrap gc.  This current design is mis-guided because the coalesce-and-fill effort will be preempted if a young gc is triggered while we are working on coalesce-and-fill.
> 
> This PR has the potential of reducing the number of concurrent young gcs by the number of times we initiate an old bootstrap gcs.

This pull request has now been integrated.

Changeset: e2821e44
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/e2821e44d3543774f4fbac3591ef7fae44eca362
Stats:     23 lines in 1 file changed: 0 ins; 19 del; 4 mod

8318201: GenShen: Remove preparatory young GC before bootstrap

Reviewed-by: wkemper

-------------

PR: https://git.openjdk.org/shenandoah/pull/340

From kdnilsen at openjdk.org  Mon Oct 16 23:59:33 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 16 Oct 2023 23:59:33 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v10]
In-Reply-To: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
Message-ID: <v_V_gsZCBIlAE7AwdM2WkR2m-TdS8KnCyfiYENPw6wg=.8eb33bb6-36a1-473b-82c7-c66151167b6d@github.com>

> Several improvements:
> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
> 2. Reduce likelihood of humongous allocation errors by:
>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Add a comment

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/336/files
  - new: https://git.openjdk.org/shenandoah/pull/336/files/baaba77c..037293e2

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=09
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=08-09

  Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/shenandoah/pull/336.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/336/head:pull/336

PR: https://git.openjdk.org/shenandoah/pull/336

From kdnilsen at openjdk.org  Tue Oct 17 04:22:28 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 17 Oct 2023 04:22:28 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v11]
In-Reply-To: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
Message-ID: <fDqGgvatvszCfhIMT0ctuBu85AH05LPQe-sKktzzyBk=.a8cef051-28fe-4946-ade2-26fa0163cafa@github.com>

> Several improvements:
> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
> 2. Reduce likelihood of humongous allocation errors by:
>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Fix assert to deal with humongous old

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/336/files
  - new: https://git.openjdk.org/shenandoah/pull/336/files/037293e2..8aeeb113

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=10
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=09-10

  Stats: 2 lines in 1 file changed: 0 ins; 1 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/336.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/336/head:pull/336

PR: https://git.openjdk.org/shenandoah/pull/336

From kdnilsen at openjdk.org  Tue Oct 17 04:37:28 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 17 Oct 2023 04:37:28 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v12]
In-Reply-To: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
Message-ID: <-_DHTo0Z44VnLKcI_qO8cdvxP4V5LMCejjaQQ3xjBrw=.94f002a6-b843-48fd-bc03-2d168bed8851@github.com>

> Several improvements:
> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
> 2. Reduce likelihood of humongous allocation errors by:
>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  One more try at this assertion

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/336/files
  - new: https://git.openjdk.org/shenandoah/pull/336/files/8aeeb113..344832b8

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=11
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=10-11

  Stats: 4 lines in 1 file changed: 3 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/336.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/336/head:pull/336

PR: https://git.openjdk.org/shenandoah/pull/336

From kdnilsen at openjdk.org  Tue Oct 17 05:14:20 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 17 Oct 2023 05:14:20 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v13]
In-Reply-To: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
Message-ID: <KV_MsLWD9s9mgB0WrF9KiIq_LuUeH4o2b3tbstBlq4w=.bac28e9a-3104-4169-bf52-7328be6f4fbb@github.com>

> Several improvements:
> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
> 2. Reduce likelihood of humongous allocation errors by:
>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Allow for no old regions in assertion

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/336/files
  - new: https://git.openjdk.org/shenandoah/pull/336/files/344832b8..065363de

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=12
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=11-12

  Stats: 5 lines in 1 file changed: 2 ins; 0 del; 3 mod
  Patch: https://git.openjdk.org/shenandoah/pull/336.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/336/head:pull/336

PR: https://git.openjdk.org/shenandoah/pull/336

From ayang at openjdk.org  Tue Oct 17 07:34:22 2023
From: ayang at openjdk.org (Albert Mingkun Yang)
Date: Tue, 17 Oct 2023 07:34:22 GMT
Subject: RFR: 8317350: Move code cache purging out of
 CodeCache::UnloadingScope
In-Reply-To: <I5PkYOgKATueEt99THsckfm6rkZMd2ar1KrTHJtLkO4=.e1de91e1-18ea-4836-878d-d06d1a35c2a0@github.com>
References: <BvcvMniLilmj4oHWHZRAratUJar05TJ1XptU_XoPHm0=.c1c84329-b5e0-4493-8133-aa528dbfb081@github.com>
 <dLVgIGPlDVJ4dhVBVLocMmVB_6Qxdmkv_P_BulJCcRA=.d620ce8e-ad55-4849-81ab-ecb5c0dbae7c@github.com>
 <I5PkYOgKATueEt99THsckfm6rkZMd2ar1KrTHJtLkO4=.e1de91e1-18ea-4836-878d-d06d1a35c2a0@github.com>
Message-ID: <aeUa7RC8Gb2CIpIoGGJ3sgIFGIFHG5sCEQYmxTRp2-I=.1f1e7f3f-9e1a-42e1-9345-fc44ce485120@github.com>

On Mon, 16 Oct 2023 14:37:04 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

> It is maybe an insignificant detail in the context of stw collectors

Then, could Serial and Parallel use APIs that don't expose these details? For instance, move `flush_unlinked_nmethods` inside `CodeCache::do_unloading`, as it is used only by those collectors.

Why is `flush_unlinked_nmethods` outside of `UnloadingScope`? This newly-introduced scope in the caller context seems extremely out of place, IMO.

> Putting it on this level also allows more straightforward logging.

I don't get this. Can't see any log-print logic inside `flush_unlinked_nmethods`.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/16011#discussion_r1361632099

From tschatzl at openjdk.org  Tue Oct 17 09:21:22 2023
From: tschatzl at openjdk.org (Thomas Schatzl)
Date: Tue, 17 Oct 2023 09:21:22 GMT
Subject: RFR: 8317350: Move code cache purging out of
 CodeCache::UnloadingScope
In-Reply-To: <aeUa7RC8Gb2CIpIoGGJ3sgIFGIFHG5sCEQYmxTRp2-I=.1f1e7f3f-9e1a-42e1-9345-fc44ce485120@github.com>
References: <BvcvMniLilmj4oHWHZRAratUJar05TJ1XptU_XoPHm0=.c1c84329-b5e0-4493-8133-aa528dbfb081@github.com>
 <dLVgIGPlDVJ4dhVBVLocMmVB_6Qxdmkv_P_BulJCcRA=.d620ce8e-ad55-4849-81ab-ecb5c0dbae7c@github.com>
 <I5PkYOgKATueEt99THsckfm6rkZMd2ar1KrTHJtLkO4=.e1de91e1-18ea-4836-878d-d06d1a35c2a0@github.com>
 <aeUa7RC8Gb2CIpIoGGJ3sgIFGIFHG5sCEQYmxTRp2-I=.1f1e7f3f-9e1a-42e1-9345-fc44ce485120@github.com>
Message-ID: <h2L-UkUGpIB1nuaFmgSuSamXiWgFGd3l-GRJzuhe1Lo=.17b20427-87af-4bcc-8e6e-2f22e09980d4@github.com>

On Tue, 17 Oct 2023 07:31:51 GMT, Albert Mingkun Yang <ayang at openjdk.org> wrote:

>Then, could Serial and Parallel use APIs that don't expose these details?

There is no such API. This change does not intend to expose such a new API too. Just moving the various phases of code/class unloading to the same level in the source code as apparent to me to keep surprises low (and simplify logging to be able to _see_ problems in the first place. Then we can fix them in subsequent PRs).

Also I would like to keep the existing structure of class/code unloading for all collectors for uniformity (e.g. separate unlinking from free-unlinking - maybe call it "purging" in the future?) - so that they will have the same structure and print the same logging messages in the same order in the future (at least for the STW collectors including G1).

Note that other collectors have exactly the same phases, and unlinking is separate from purging everywhere else too. They just re-implement some phases using their own code (doing some renames in the process).

Having both the same structure/phases and same (more comprehensive) logging output for more collectors will make troubleshooting much easier.

There is no renaming in this change, e.g. `CodeCache::do_unloading()` (="unlinking") as this method does not do the complete unloading either indicated by having this external `flush_unlinked_nmethods` (="purging"). This is not the change to do this imo. 
(Iirc `CodeCache::do_unloading()` still has does some purging in it anyway, but let's not digress too much here).

> For instance, move flush_unlinked_nmethods inside CodeCache::do_unloading, as it is used only by those collectors.

It could, but then for (future) logging you would have to pass `GCTraceTimer` as the common way to do timing in gc code into `do_unloading()` which is some compiler code. Not sure if this is what we want as it is imo awkward. It is imho better, cleaner and sufficient to have timing outside at least initially. I.e. I would at this point prefer the style of


{ // Phase x
  GCTraceTimer x("do phase X");
  do_phase_x();
}


as how to time can be heavily collector dependent too. I would at the end of all that refactoring see if there is some useful coalescing of the methods into helpers that can be done.
Maybe these scopes should be put into separate helper methods, I haven't decided yet what's best.

>Why is flush_unlinked_nmethods outside of UnloadingScope? This newly-introduced scope in the caller context seems extremely out of place, IMO.

I believe most of your questions stem from the naming. Please, do not be too hung up on names. The description of this PR already mentioned that `UnloadingScope` is a misnomer (and misnamed code is common in Hotspot code, or responsibilities changed over the course of years without fixing up the naming). So in addition to this class, be aware that there are lots of misnamed and not uniformly named methods in class/code unloading code too. `UnloadingScope` controls *unlinking* behavior by setting some globals and does not control the whole unloading process (ie. unlinking + purging).

Additionally `UnloadingScope` actually did _not_ really contain `flush_unlinked_nmethods` earlier either. It has been the last call in the destructor of the `UnloadingScope`, _outside_, after the unlinking behavior has been reset. This is even more strange if you think about it.

So from my understanding of the code this scope object ought to enclose only the unlinking part of the unloading (i.e. the decision of what to unlink, that is `CodeCache::do_unloading()`) before. This change only at most exposes the existing (as you say ugly - I agree) structure.

Which isn't a bad thing to me to have exposed. It's not in scope of this change to fix this imo because that would mix it with other unrelated changes.

(`UnloadingScope` should be called something different, maybe after this discussion something like `UnlinkingScope` or similar? Idk.)

>>  Putting it on this level also allows more straightforward logging.
>I don't get this. Can't see any log-print logic inside flush_unlinked_nmethods.

... in the future. (https://bugs.openjdk.org/browse/JDK-8315504)[https://bugs.openjdk.org/browse/JDK-8315504] intends to add more timing/logging for every phase.

There is currently no plan for comprehensive logging inside the various phases of the class/code unloading (at least not to the level of selected methods I did in recent investigations).

What I intend to do is revisiting the phases in the future and move around work to better reflect the unlinking/purging split, and also link them together with a real `UnloadingScope` covering the whole unloading process (not to be mixed up with the current `UnloadingScope`) to allow gc-specific replacements/optimizations of the phases.

Obviously there can be helper methods that simplify things for various gcs. This PR isn't this change though.

Hth,
  Thomas

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/16011#discussion_r1361795034

From ayang at openjdk.org  Tue Oct 17 11:54:53 2023
From: ayang at openjdk.org (Albert Mingkun Yang)
Date: Tue, 17 Oct 2023 11:54:53 GMT
Subject: RFR: 8317350: Move code cache purging out of
 CodeCache::UnloadingScope
In-Reply-To: <BvcvMniLilmj4oHWHZRAratUJar05TJ1XptU_XoPHm0=.c1c84329-b5e0-4493-8133-aa528dbfb081@github.com>
References: <BvcvMniLilmj4oHWHZRAratUJar05TJ1XptU_XoPHm0=.c1c84329-b5e0-4493-8133-aa528dbfb081@github.com>
Message-ID: <ChzvGFD9iAORq5ZSHmqaq5AXCniIF39SE3QPADA3YBM=.8762b835-9ba8-48a9-bc5a-4b068bdd979c@github.com>

On Mon, 2 Oct 2023 12:56:27 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

> Hi all,
> 
>   please review this refactoring that moves actual code cache flushing/purging out of `CodeCache::UnloadingScope`. Reasons:
> 
> * I prefer that a destructor does not do anything substantial - in some cases, 90% of time is spent in the destructor in that extracted method (due to https://bugs.openjdk.org/browse/JDK-8316959)
> * imho it does not fit the class which does nothing but sets/resets some code cache unloading behavior (probably should be renamed to `UnloadingBehaviorScope` too in a separate CR).
> * other existing methods at that level are placed out of that (or any other) scope object too - which is already the case for when doing concurrent unloading.
> * putting it there makes future logging of the various phases a little bit easier, not having `GCTraceTimer` et al. in various places.
> 
> Testing: gha
> 
> Thanks,
>   Thomas

Marked as reviewed by ayang (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/16011#pullrequestreview-1682122861

From ayang at openjdk.org  Tue Oct 17 11:54:55 2023
From: ayang at openjdk.org (Albert Mingkun Yang)
Date: Tue, 17 Oct 2023 11:54:55 GMT
Subject: RFR: 8317350: Move code cache purging out of
 CodeCache::UnloadingScope
In-Reply-To: <h2L-UkUGpIB1nuaFmgSuSamXiWgFGd3l-GRJzuhe1Lo=.17b20427-87af-4bcc-8e6e-2f22e09980d4@github.com>
References: <BvcvMniLilmj4oHWHZRAratUJar05TJ1XptU_XoPHm0=.c1c84329-b5e0-4493-8133-aa528dbfb081@github.com>
 <dLVgIGPlDVJ4dhVBVLocMmVB_6Qxdmkv_P_BulJCcRA=.d620ce8e-ad55-4849-81ab-ecb5c0dbae7c@github.com>
 <I5PkYOgKATueEt99THsckfm6rkZMd2ar1KrTHJtLkO4=.e1de91e1-18ea-4836-878d-d06d1a35c2a0@github.com>
 <aeUa7RC8Gb2CIpIoGGJ3sgIFGIFHG5sCEQYmxTRp2-I=.1f1e7f3f-9e1a-42e1-9345-fc44ce485120@github.com>
 <h2L-UkUGpIB1nuaFmgSuSamXiWgFGd3l-GRJzuhe1Lo=.17b20427-87af-4bcc-8e6e-2f22e09980d4@github.com>
Message-ID: <a0vw6ZEONTpP67eQnRbACpectU0-kg0dcYW0VqfjkXU=.203c89e4-1d2d-4005-88b5-6f709d939f79@github.com>

On Tue, 17 Oct 2023 09:19:05 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

> UnloadingScope should be called something different, maybe after this discussion something like UnlinkingScope or similar?

OK, the new-scope starts to make sense if it's named `UnlinkingScope`.

In my mind, the concept of `UnloadingScope` includes unlinking + purging; the implementation supports my understanding, as I believe the destructor still belongs to the obj-on-stack. However, I now realize that this interpretation could be subjective.

> This change only at most exposes the existing (as you say ugly - I agree) structure.

OK, that's all I wanna convey. Maybe this is an inevitable transient state.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/16011#discussion_r1361991674

From lkorinth at openjdk.org  Tue Oct 17 12:29:46 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Tue, 17 Oct 2023 12:29:46 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v4]
In-Reply-To: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
Message-ID: <4pRda3ZAZzVzGiVrDv6LN9Pw__DhrmTm4qZjTHzaq80=.a009bb29-4869-4047-8b62-80fbe7bef692@github.com>

> Rename createJavaProcessBuilder so that it is not used by mistake instead of createTestJvm.
> 
> I have used the following sed script: `find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createJavaProcessBuilderIgnoreTestJavaOpts(/g"`
> 
> Then I have manually modified ProcessTools.java. In that file I have moved one version of createJavaProcessBuilder so that it is close to the other version. Then I have added a javadoc comment in bold telling:
> 
>    /**
>      * Create ProcessBuilder using the java launcher from the jdk to
>      * be tested.
>      *
>      * <p><b> Please observe that you likely should use
>      * createTestJvm() instead of this method because createTestJvm()
>      * will add JVM options from "test.vm.opts" and "test.java.opts"
>      * </b> and this method will not do that.
>      *
>      * @param command Arguments to pass to the java command.
>      * @return The ProcessBuilder instance representing the java command.
>      */
> 
> 
> I have used the name createJavaProcessBuilderIgnoreTestJavaOpts because of the name of Utils.prependTestJavaOpts that adds those VM flags. If you have a better name I could do a rename of the method. I kind of like that it is long and clumsy, that makes it harder to use...
> 
> I have run tier 1 testing, and I have started more exhaustive testing.

Leo Korinth has updated the pull request incrementally with three additional commits since the last revision:

 - Revert "8315097: Rename createJavaProcessBuilder"
   
   This reverts commit 4b2d171133c40c5c48114602bfd0d4da75531317.
 - Revert "copyright"
   
   This reverts commit f3418c80cc0d4cbb722ee5e368f1a001e898b43e.
 - Revert "fix static import"
   
   This reverts commit 27da71508aec9a4bec1c0ad07031887286580171.

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/15452/files
  - new: https://git.openjdk.org/jdk/pull/15452/files/27da7150..44af07b9

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=15452&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15452&range=02-03

  Stats: 1102 lines in 462 files changed: 11 ins; 22 del; 1069 mod
  Patch: https://git.openjdk.org/jdk/pull/15452.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15452/head:pull/15452

PR: https://git.openjdk.org/jdk/pull/15452

From kdnilsen at openjdk.org  Tue Oct 17 15:43:18 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 17 Oct 2023 15:43:18 GMT
Subject: RFR: 8318190: GenShen: Be less aggressive with triggers for
 old-gen has expanded [v3]
In-Reply-To: <ZnUOPZdxgn_oUAWUwl3M86YQIv1WRd5KqW3Pq4jrUa8=.ffbc2b01-da31-463f-b28b-6468a13d44cf@github.com>
References: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
 <ZnUOPZdxgn_oUAWUwl3M86YQIv1WRd5KqW3Pq4jrUa8=.ffbc2b01-da31-463f-b28b-6468a13d44cf@github.com>
Message-ID: <yzU4lSxLUFK6zlYMcCi4hgSTj7xPNRavcar_NiOA1vY=.1e0673d7-6a73-4db2-b3ed-6a65f2b1a574@github.com>

On Mon, 16 Oct 2023 23:17:34 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> This patch reduces the frequency of old-has-grown triggers in the case that old usage is smaller than a particular threshold percentage (default value 5%) of the total heap size.  In this case, old is triggered for growth no more frequently than once every N cycles, where N defaults to 50.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix whitespace

src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp line 62:

> 60:           "is below ShenandoahGenerationalIgnoreOldGrowthBelowPercentage, " \
> 61:           "trigger an old-generation mark if old has grown and this "       \
> 62:           "many young-gen consecutive young-gen collections have been "     \

will remove redundant young-gen from this sentence

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/341#discussion_r1362347837

From wkemper at openjdk.org  Tue Oct 17 16:25:21 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 17 Oct 2023 16:25:21 GMT
Subject: RFR: 8318190: GenShen: Be less aggressive with triggers for
 old-gen has expanded [v3]
In-Reply-To: <ZnUOPZdxgn_oUAWUwl3M86YQIv1WRd5KqW3Pq4jrUa8=.ffbc2b01-da31-463f-b28b-6468a13d44cf@github.com>
References: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
 <ZnUOPZdxgn_oUAWUwl3M86YQIv1WRd5KqW3Pq4jrUa8=.ffbc2b01-da31-463f-b28b-6468a13d44cf@github.com>
Message-ID: <-ZAZv0hi5Fy_rWoZ8WSNMDh07ByOt2U5V345bAxZpJk=.0b5433b1-1cd7-4bcd-b7f4-21a7511fcdac@github.com>

On Mon, 16 Oct 2023 23:17:34 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> This patch reduces the frequency of old-has-grown triggers in the case that old usage is smaller than a particular threshold percentage (default value 5%) of the total heap size.  In this case, old is triggered for growth no more frequently than once every N cycles, where N defaults to 50.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix whitespace

Some minor suggestions.

src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 519:

> 517:   }
> 518: 
> 519:   if (_growth_trigger) {

Could the trigger just evaluate the condition directly? It's not clear why we double check `_growth_trigger` condition instead of just checking once when trigger is evaluated?

src/hotspot/share/gc/shenandoah/shenandoahControlThread.hpp line 106:

> 104:   volatile GCMode _mode;
> 105:   shenandoah_padding(3);
> 106:   volatile size_t _consecutive_young;

This member would fit in better with `ShenandoahCollectorPolicy`, which gathers similar statistics via its `record` methods (`record_success_concurrent`, `record_success_old`, e.g.). We wouldn't need changes in `ShenandoahControlThread` in that approach.

src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp line 38:

> 36:                             constraint)                                     \
> 37:                                                                             \
> 38:   product(double, ShenandoahGenerationalMinOldGenGrowthPercent,             \

I'd rather not include `Generational` in these property names - I think it just makes them too long and I don't think users will confuse them with non-generational settings.

-------------

Changes requested by wkemper (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/341#pullrequestreview-1682699388
PR Review Comment: https://git.openjdk.org/shenandoah/pull/341#discussion_r1362349454
PR Review Comment: https://git.openjdk.org/shenandoah/pull/341#discussion_r1362337986
PR Review Comment: https://git.openjdk.org/shenandoah/pull/341#discussion_r1362341400

From kdnilsen at openjdk.org  Tue Oct 17 17:05:18 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 17 Oct 2023 17:05:18 GMT
Subject: RFR: 8318190: GenShen: Be less aggressive with triggers for
 old-gen has expanded [v4]
In-Reply-To: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
References: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
Message-ID: <AR5XzPAzp4_3x8c3UnkTPsKIA_rZldzQGGoVN6eoe1M=.ff881126-cd3a-4c22-887a-d39e969e7623@github.com>

> This patch reduces the frequency of old-has-grown triggers in the case that old usage is smaller than a particular threshold percentage (default value 5%) of the total heap size.  In this case, old is triggered for growth no more frequently than once every N cycles, where N defaults to 50.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Fix user instructions

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/341/files
  - new: https://git.openjdk.org/shenandoah/pull/341/files/e7747100..b4071011

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=341&range=03
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=341&range=02-03

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/341.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/341/head:pull/341

PR: https://git.openjdk.org/shenandoah/pull/341

From wkemper at openjdk.org  Tue Oct 17 18:19:08 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 17 Oct 2023 18:19:08 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
 [v5]
In-Reply-To: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
Message-ID: <Q_a9dZFAbRxESoAtsOBomIGlkwtk5uEXbiNXNmqNZf8=.ced8e029-13d0-43dd-9dc3-c68248f373ee@github.com>

> Shenandoah will run back-to-back full GCs and _almost_ grind mutator progress to a halt before eventually exhausting memory. This change will have Shenandoah raise a gc threshold exceeded exception if the collector fails to make progress after `ShenandoahNoProgressThreshold` full GC cycles (default is 3).

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 13 additional commits since the last revision:

 - Make explicit field for Shenandoah
 - Restore original retry logic, pull gc overhead check back over retry
 - Merge branch 'openjdk-master' into shenandoah-oome-redux
 - Merge check for no-progress into retry allocation block
 - Revert change to TEST.groups
 - Merge remote-tracking branch 'openjdk/master' into shenandoah-oome-redux
 - Extend exemption for EATests that rely on timely OOME to Shenandoah
 - Improve comment, increase default for no progress threshold
 - Allocator should not reset bad progress count
 - Allocator should not reset bad progress count
 - ... and 3 more: https://git.openjdk.org/jdk/compare/b9836bee...868af376

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/15852/files
  - new: https://git.openjdk.org/jdk/pull/15852/files/1971467f..868af376

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=15852&range=04
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15852&range=03-04

  Stats: 32231 lines in 1017 files changed: 18757 ins; 6567 del; 6907 mod
  Patch: https://git.openjdk.org/jdk/pull/15852.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15852/head:pull/15852

PR: https://git.openjdk.org/jdk/pull/15852

From wkemper at openjdk.org  Tue Oct 17 18:36:41 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 17 Oct 2023 18:36:41 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
 [v6]
In-Reply-To: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
Message-ID: <f6NKxGmkarmFYQZR6LQ4uxmjz5Qec8hKTc0ql68JGQQ=.023bb406-4a01-4770-a4e2-ecb83bc19f25@github.com>

> Shenandoah will run back-to-back full GCs and _almost_ grind mutator progress to a halt before eventually exhausting memory. This change will have Shenandoah raise a gc threshold exceeded exception if the collector fails to make progress after `ShenandoahNoProgressThreshold` full GC cycles (default is 3).

William Kemper has updated the pull request incrementally with two additional commits since the last revision:

 - Remove unnecessary nesting and other differences
 - Update formatting

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/15852/files
  - new: https://git.openjdk.org/jdk/pull/15852/files/868af376..4154370c

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=15852&range=05
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15852&range=04-05

  Stats: 56 lines in 2 files changed: 17 ins; 16 del; 23 mod
  Patch: https://git.openjdk.org/jdk/pull/15852.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15852/head:pull/15852

PR: https://git.openjdk.org/jdk/pull/15852

From shade at openjdk.org  Tue Oct 17 19:26:45 2023
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 17 Oct 2023 19:26:45 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
 [v6]
In-Reply-To: <f6NKxGmkarmFYQZR6LQ4uxmjz5Qec8hKTc0ql68JGQQ=.023bb406-4a01-4770-a4e2-ecb83bc19f25@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
 <f6NKxGmkarmFYQZR6LQ4uxmjz5Qec8hKTc0ql68JGQQ=.023bb406-4a01-4770-a4e2-ecb83bc19f25@github.com>
Message-ID: <nqwRiObw4uzrCsOBNQTfxwaYG_LDpxCtaFKYyRPBGQ8=.bae10352-c95e-4acf-aa90-e2099921450c@github.com>

On Tue, 17 Oct 2023 18:36:41 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Shenandoah will run back-to-back full GCs and _almost_ grind mutator progress to a halt before eventually exhausting memory. This change will have Shenandoah raise a gc threshold exceeded exception if the collector fails to make progress after `ShenandoahNoProgressThreshold` full GC cycles (default is 3).
>
> William Kemper has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Remove unnecessary nesting and other differences
>  - Update formatting

All right, this looks fine.

-------------

Marked as reviewed by shade (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/15852#pullrequestreview-1683198268

From kdnilsen at openjdk.org  Tue Oct 17 21:35:01 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 17 Oct 2023 21:35:01 GMT
Subject: RFR: 8318190: GenShen: Be less aggressive with triggers for
 old-gen has expanded [v3]
In-Reply-To: <-ZAZv0hi5Fy_rWoZ8WSNMDh07ByOt2U5V345bAxZpJk=.0b5433b1-1cd7-4bcd-b7f4-21a7511fcdac@github.com>
References: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
 <ZnUOPZdxgn_oUAWUwl3M86YQIv1WRd5KqW3Pq4jrUa8=.ffbc2b01-da31-463f-b28b-6468a13d44cf@github.com>
 <-ZAZv0hi5Fy_rWoZ8WSNMDh07ByOt2U5V345bAxZpJk=.0b5433b1-1cd7-4bcd-b7f4-21a7511fcdac@github.com>
Message-ID: <WNWiyMISt5S6BFuXGaYOkX_iaa3woecHmUfMTpfO0Qo=.b17ca19f-6aab-4a55-8ac5-1d12820832d6@github.com>

On Tue, 17 Oct 2023 15:37:36 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix whitespace
>
> src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp line 38:
> 
>> 36:                             constraint)                                     \
>> 37:                                                                             \
>> 38:   product(double, ShenandoahGenerationalMinOldGenGrowthPercent,             \
> 
> I'd rather not include `Generational` in these property names - I think it just makes them too long and I don't think users will confuse them with non-generational settings.

I'll make that change.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/341#discussion_r1362811391

From kdnilsen at openjdk.org  Tue Oct 17 21:35:11 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 17 Oct 2023 21:35:11 GMT
Subject: RFR: 8318190: GenShen: Be less aggressive with triggers for
 old-gen has expanded [v4]
In-Reply-To: <AR5XzPAzp4_3x8c3UnkTPsKIA_rZldzQGGoVN6eoe1M=.ff881126-cd3a-4c22-887a-d39e969e7623@github.com>
References: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
 <AR5XzPAzp4_3x8c3UnkTPsKIA_rZldzQGGoVN6eoe1M=.ff881126-cd3a-4c22-887a-d39e969e7623@github.com>
Message-ID: <uVi9HgWwvE4y9SpiKc15uA-qVROzwRcJewYL5Svac3c=.7c731577-3a4b-483b-bc99-7655148739fe@github.com>

On Tue, 17 Oct 2023 17:05:18 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> This patch reduces the frequency of old-has-grown triggers in the case that old usage is smaller than a particular threshold percentage (default value 5%) of the total heap size.  In this case, old is triggered for growth no more frequently than once every N cycles, where N defaults to 50.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix user instructions

src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp line 47:

> 45:           range(0.0,100.0)                                                  \
> 46:                                                                             \
> 47:   product(uintx, ShenandoahGenerationalIgnoreOldGrowthBelowPercentage,      \

Do we want to experiment with different workloads to decide the ideal default?  10 might work even better as the ideal value for best out-of-box experience.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/341#discussion_r1362800334

From kdnilsen at openjdk.org  Tue Oct 17 23:27:22 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 17 Oct 2023 23:27:22 GMT
Subject: RFR: 8318190: GenShen: Be less aggressive with triggers for
 old-gen has expanded [v3]
In-Reply-To: <-ZAZv0hi5Fy_rWoZ8WSNMDh07ByOt2U5V345bAxZpJk=.0b5433b1-1cd7-4bcd-b7f4-21a7511fcdac@github.com>
References: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
 <ZnUOPZdxgn_oUAWUwl3M86YQIv1WRd5KqW3Pq4jrUa8=.ffbc2b01-da31-463f-b28b-6468a13d44cf@github.com>
 <-ZAZv0hi5Fy_rWoZ8WSNMDh07ByOt2U5V345bAxZpJk=.0b5433b1-1cd7-4bcd-b7f4-21a7511fcdac@github.com>
Message-ID: <Cd2Lonmdbp72HHT5OFEkCU31PgXIAh2clG6V0GjjNsI=.2f59fbd8-88b6-46da-b026-826a2ceb70e6@github.com>

On Tue, 17 Oct 2023 15:35:41 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix whitespace
>
> src/hotspot/share/gc/shenandoah/shenandoahControlThread.hpp line 106:
> 
>> 104:   volatile GCMode _mode;
>> 105:   shenandoah_padding(3);
>> 106:   volatile size_t _consecutive_young;
> 
> This member would fit in better with `ShenandoahCollectorPolicy`, which gathers similar statistics via its `record` methods (`record_success_concurrent`, `record_success_old`, e.g.). We wouldn't need changes in `ShenandoahControlThread` in that approach.

Thanks.  Good suggestion.  I'm moving this into ShenandoahCollectorPolicy.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/341#discussion_r1362921259

From kdnilsen at openjdk.org  Tue Oct 17 23:35:30 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 17 Oct 2023 23:35:30 GMT
Subject: RFR: 8318190: GenShen: Be less aggressive with triggers for
 old-gen has expanded [v3]
In-Reply-To: <-ZAZv0hi5Fy_rWoZ8WSNMDh07ByOt2U5V345bAxZpJk=.0b5433b1-1cd7-4bcd-b7f4-21a7511fcdac@github.com>
References: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
 <ZnUOPZdxgn_oUAWUwl3M86YQIv1WRd5KqW3Pq4jrUa8=.ffbc2b01-da31-463f-b28b-6468a13d44cf@github.com>
 <-ZAZv0hi5Fy_rWoZ8WSNMDh07ByOt2U5V345bAxZpJk=.0b5433b1-1cd7-4bcd-b7f4-21a7511fcdac@github.com>
Message-ID: <UapduVTZB0XDZwvS6OFo1d5WpZ2ixiot1vWgTmNxoHM=.366b77ca-169d-4671-a5f2-428ce8b3f657@github.com>

On Tue, 17 Oct 2023 15:41:30 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix whitespace
>
> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 519:
> 
>> 517:   }
>> 518: 
>> 519:   if (_growth_trigger) {
> 
> Could the trigger just evaluate the condition directly? It's not clear why we double check `_growth_trigger` condition instead of just checking once when trigger is evaluated?

Just to be sure: are you suggesting that before we raise the _growth_trigger, we should confirm that we are not doing mixed evacuations?  And then we don't have to recheck whether the trigger condition is still valid when we get to here?

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/341#discussion_r1362925498

From kdnilsen at openjdk.org  Tue Oct 17 23:46:06 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 17 Oct 2023 23:46:06 GMT
Subject: RFR: 8318190: GenShen: Be less aggressive with triggers for
 old-gen has expanded [v5]
In-Reply-To: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
References: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
Message-ID: <gB3jL9Q0B3nMhOCG0TME2wpC1DPOasC-kH_6R4vK_qE=.bcce6ca1-c61c-4f7b-9767-329e73066daf@github.com>

> This patch reduces the frequency of old-has-grown triggers in the case that old usage is smaller than a particular threshold percentage (default value 5%) of the total heap size.  In this case, old is triggered for growth no more frequently than once every N cycles, where N defaults to 50.

Kelvin Nilsen has updated the pull request incrementally with two additional commits since the last revision:

 - Move consecutive young accounting into ShenandoahCollectorPolicy
   
   This change was requested by PR reviewer.
 - Simplify naming of of configuration parameters
   
   Remove Generational substring from the parameter names, as requested by
   reviewer feedback.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/341/files
  - new: https://git.openjdk.org/shenandoah/pull/341/files/b4071011..8f434fb9

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=341&range=04
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=341&range=03-04

  Stats: 57 lines in 9 files changed: 20 ins; 19 del; 18 mod
  Patch: https://git.openjdk.org/shenandoah/pull/341.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/341/head:pull/341

PR: https://git.openjdk.org/shenandoah/pull/341

From wkemper at openjdk.org  Tue Oct 17 23:57:14 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 17 Oct 2023 23:57:14 GMT
Subject: RFR: 8318190: GenShen: Be less aggressive with triggers for
 old-gen has expanded [v3]
In-Reply-To: <UapduVTZB0XDZwvS6OFo1d5WpZ2ixiot1vWgTmNxoHM=.366b77ca-169d-4671-a5f2-428ce8b3f657@github.com>
References: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
 <ZnUOPZdxgn_oUAWUwl3M86YQIv1WRd5KqW3Pq4jrUa8=.ffbc2b01-da31-463f-b28b-6468a13d44cf@github.com>
 <-ZAZv0hi5Fy_rWoZ8WSNMDh07ByOt2U5V345bAxZpJk=.0b5433b1-1cd7-4bcd-b7f4-21a7511fcdac@github.com>
 <UapduVTZB0XDZwvS6OFo1d5WpZ2ixiot1vWgTmNxoHM=.366b77ca-169d-4671-a5f2-428ce8b3f657@github.com>
Message-ID: <EbxkgSoKoDz2e6iBdtdsTPR4HP60NbxWvyU__7-bpD0=.a4deeb2a-2bb9-4fda-8112-dae1d37756dc@github.com>

On Tue, 17 Oct 2023 23:32:43 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/heuristics/shenandoahOldHeuristics.cpp line 519:
>> 
>>> 517:   }
>>> 518: 
>>> 519:   if (_growth_trigger) {
>> 
>> Could the trigger just evaluate the condition directly? It's not clear why we double check `_growth_trigger` condition instead of just checking once when trigger is evaluated?
>
> Just to be sure: are you suggesting that before we raise the _growth_trigger, we should confirm that we are not doing mixed evacuations?  And then we don't have to recheck whether the trigger condition is still valid when we get to here?

I was suggesting we move [the logic that raises these triggers](https://github.com/openjdk/shenandoah/blob/master/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp#L3100-L3123) into the heuristic itself. This would reduce changes in shenandoahHeap.cpp and better encapsulate the logic for the heuristic.

If we want to align evaluation of these triggers with rebuilding the freeset (so that they aren't being evaluated at the same frequency as the young generation), we could add a `heap_has_changed` flag (or something). I'm also working on a change to only evaluate old generation triggers when the young generation trigger has fired.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/341#discussion_r1362938186

From wkemper at openjdk.org  Wed Oct 18 00:01:28 2023
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 18 Oct 2023 00:01:28 GMT
Subject: RFR: 8318190: GenShen: Be less aggressive with triggers for
 old-gen has expanded [v4]
In-Reply-To: <uVi9HgWwvE4y9SpiKc15uA-qVROzwRcJewYL5Svac3c=.7c731577-3a4b-483b-bc99-7655148739fe@github.com>
References: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
 <AR5XzPAzp4_3x8c3UnkTPsKIA_rZldzQGGoVN6eoe1M=.ff881126-cd3a-4c22-887a-d39e969e7623@github.com>
 <uVi9HgWwvE4y9SpiKc15uA-qVROzwRcJewYL5Svac3c=.7c731577-3a4b-483b-bc99-7655148739fe@github.com>
Message-ID: <TUzzP5vhzs3epNssm5TGmTZOJtjZeiOARi0ndIHoQV8=.e160b512-abcd-45de-9242-fdb256fbf69f@github.com>

On Tue, 17 Oct 2023 21:23:13 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix user instructions
>
> src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp line 47:
> 
>> 45:           range(0.0,100.0)                                                  \
>> 46:                                                                             \
>> 47:   product(uintx, ShenandoahGenerationalIgnoreOldGrowthBelowPercentage,      \
> 
> Do we want to experiment with different workloads to decide the ideal default?  10 might work even better as the ideal value for best out-of-box experience.

Yes, 10 seems sensible. I think many people have been conditioned to hand over more than 1/2 the heap to the old generation without blinking an eye.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/341#discussion_r1362940411

From ysr at openjdk.org  Wed Oct 18 06:06:01 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Wed, 18 Oct 2023 06:06:01 GMT
Subject: RFR: 8318057: GenShen: investigate small refactor of remset swap code
Message-ID: <qEO5M57wWdi78hROoJlrNua7hys3sobUN_pXRVxeL_8=.a2ae0dba-7391-449e-b02d-2c8d25a5f6be@github.com>

Some closures were being unnecessarily applied to free regions rather than just to allocated old and young regions. Restrict the closure application to old and young regions as appropriate, eliminating extra generation affiliation checks in some closures. Also fixed a slightly incorrect documentation comment.

**Testing:**
- jtreg and stress tests in progress in test pipeline helper
- locally tested with jtreg hotsot_gc tests and with specjbb with and without heap verification
- performance differences, if any, to be measured

-------------

Commit messages:
 - Merge branch 'master' into swap_rs
 - fix for comp error (wrong variable name).
 -     Avoid including free young regions in young gen heap region
 - Avoid including free old regions in old gen heap region

Changes: https://git.openjdk.org/shenandoah/pull/342/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=342&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8318057
  Stats: 16 lines in 2 files changed: 2 ins; 3 del; 11 mod
  Patch: https://git.openjdk.org/shenandoah/pull/342.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/342/head:pull/342

PR: https://git.openjdk.org/shenandoah/pull/342

From wkemper at openjdk.org  Wed Oct 18 18:14:13 2023
From: wkemper at openjdk.org (William Kemper)
Date: Wed, 18 Oct 2023 18:14:13 GMT
Subject: RFR: 8318190: GenShen: Be less aggressive with triggers for
 old-gen has expanded [v5]
In-Reply-To: <gB3jL9Q0B3nMhOCG0TME2wpC1DPOasC-kH_6R4vK_qE=.bcce6ca1-c61c-4f7b-9767-329e73066daf@github.com>
References: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
 <gB3jL9Q0B3nMhOCG0TME2wpC1DPOasC-kH_6R4vK_qE=.bcce6ca1-c61c-4f7b-9767-329e73066daf@github.com>
Message-ID: <g3jvxd7NWoK_kCh8BmJJ2Ckxbv8_JBXHLIX1BmYAORA=.67ad2cf2-c0bb-49b5-a3dd-675d9fb81c2f@github.com>

On Tue, 17 Oct 2023 23:46:06 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> This patch reduces the frequency of old-has-grown triggers in the case that old usage is smaller than a particular threshold percentage (default value 5%) of the total heap size.  In this case, old is triggered for growth no more frequently than once every N cycles, where N defaults to 50.
>
> Kelvin Nilsen has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Move consecutive young accounting into ShenandoahCollectorPolicy
>    
>    This change was requested by PR reviewer.
>  - Simplify naming of of configuration parameters
>    
>    Remove Generational substring from the parameter names, as requested by
>    reviewer feedback.

Thank you for making the changes.

-------------

Marked as reviewed by wkemper (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/341#pullrequestreview-1685798625

From kdnilsen at openjdk.org  Wed Oct 18 20:42:27 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 18 Oct 2023 20:42:27 GMT
Subject: RFR: 8318057: GenShen: investigate small refactor of remset swap
 code
In-Reply-To: <qEO5M57wWdi78hROoJlrNua7hys3sobUN_pXRVxeL_8=.a2ae0dba-7391-449e-b02d-2c8d25a5f6be@github.com>
References: <qEO5M57wWdi78hROoJlrNua7hys3sobUN_pXRVxeL_8=.a2ae0dba-7391-449e-b02d-2c8d25a5f6be@github.com>
Message-ID: <TM-VGF87U47MXuPOBrRmKgEOU2rp-HzRFIyeoVGLayc=.cc663de8-8fa0-432c-8fcc-f3fc7cf442f7@github.com>

On Tue, 17 Oct 2023 23:26:31 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

> Some closures were being unnecessarily applied to free regions rather than just to allocated old and young regions. Restrict the closure application to old and young regions as appropriate, eliminating extra generation affiliation checks in some closures. Also fixed a slightly incorrect documentation comment.
> 
> **Testing:**
> - jtreg and stress tests in progress in test pipeline helper
> - locally tested with jtreg hotsot_gc tests and with specjbb with and without heap verification
> - performance differences, if any, to be measured

Do we see any performance improvement from these changes?   Has this passed the internal pipeline tests?

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/342#issuecomment-1769287133

From kdnilsen at openjdk.org  Wed Oct 18 20:43:20 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 18 Oct 2023 20:43:20 GMT
Subject: RFR: 8318190: GenShen: Be less aggressive with triggers for
 old-gen has expanded [v3]
In-Reply-To: <yzU4lSxLUFK6zlYMcCi4hgSTj7xPNRavcar_NiOA1vY=.1e0673d7-6a73-4db2-b3ed-6a65f2b1a574@github.com>
References: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
 <ZnUOPZdxgn_oUAWUwl3M86YQIv1WRd5KqW3Pq4jrUa8=.ffbc2b01-da31-463f-b28b-6468a13d44cf@github.com>
 <yzU4lSxLUFK6zlYMcCi4hgSTj7xPNRavcar_NiOA1vY=.1e0673d7-6a73-4db2-b3ed-6a65f2b1a574@github.com>
Message-ID: <niuquJ-C1BZ4pg5v3-aIYJ6-hsnM1FhJWmRI1sDmJf8=.7fe89dc3-28c4-40e5-90c7-6fbd668e2c9d@github.com>

On Tue, 17 Oct 2023 15:40:55 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix whitespace
>
> src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp line 62:
> 
>> 60:           "is below ShenandoahGenerationalIgnoreOldGrowthBelowPercentage, " \
>> 61:           "trigger an old-generation mark if old has grown and this "       \
>> 62:           "many young-gen consecutive young-gen collections have been "     \
> 
> will remove redundant young-gen from this sentence

done.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/341#discussion_r1364512093

From kdnilsen at openjdk.org  Wed Oct 18 20:47:29 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 18 Oct 2023 20:47:29 GMT
Subject: RFR: 8318190: GenShen: Be less aggressive with triggers for
 old-gen has expanded [v3]
In-Reply-To: <EbxkgSoKoDz2e6iBdtdsTPR4HP60NbxWvyU__7-bpD0=.a4deeb2a-2bb9-4fda-8112-dae1d37756dc@github.com>
References: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
 <ZnUOPZdxgn_oUAWUwl3M86YQIv1WRd5KqW3Pq4jrUa8=.ffbc2b01-da31-463f-b28b-6468a13d44cf@github.com>
 <-ZAZv0hi5Fy_rWoZ8WSNMDh07ByOt2U5V345bAxZpJk=.0b5433b1-1cd7-4bcd-b7f4-21a7511fcdac@github.com>
 <UapduVTZB0XDZwvS6OFo1d5WpZ2ixiot1vWgTmNxoHM=.366b77ca-169d-4671-a5f2-428ce8b3f657@github.com>
 <EbxkgSoKoDz2e6iBdtdsTPR4HP60NbxWvyU__7-bpD0=.a4deeb2a-2bb9-4fda-8112-dae1d37756dc@github.com>
Message-ID: <d_i53Bdt7Ls9oC3HQ3DIIh-_VwrN0QTFMCsUd3u1FXE=.d6e11b3e-3cfa-498f-8414-05f58de65980@github.com>

On Tue, 17 Oct 2023 23:54:27 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Just to be sure: are you suggesting that before we raise the _growth_trigger, we should confirm that we are not doing mixed evacuations?  And then we don't have to recheck whether the trigger condition is still valid when we get to here?
>
> I was suggesting we move [the logic that raises these triggers](https://github.com/openjdk/shenandoah/blob/master/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp#L3100-L3123) into the heuristic itself. This would reduce changes in shenandoahHeap.cpp and better encapsulate the logic for the heuristic.
> 
> If we want to align evaluation of these triggers with rebuilding the freeset (so that they aren't being evaluated at the same frequency as the young generation), we could add a `heap_has_changed` flag (or something). I'm also working on a change to only evaluate old generation triggers when the young generation trigger has fired.

Thanks for clarifying.  I "guess" the main tradeoff is how often do we do the work of evaluating the trigger condition.  As currently implemented, we evaluate this condition at the end of each GC cycle, which is relatively rare.  If we move it to should_start_gc(), it could be revaluated every ms or so.  On the other hand, after we integrate your change, the question of should_start_gc(OLD) will only be asked at the start of a new young gc, which is also relatively rare.

I'll move it now, in anticipation of your improvement, if you concur.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/341#discussion_r1364515072

From kdnilsen at openjdk.org  Wed Oct 18 20:55:45 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 18 Oct 2023 20:55:45 GMT
Subject: RFR: 8318190: GenShen: Be less aggressive with triggers for
 old-gen has expanded [v3]
In-Reply-To: <d_i53Bdt7Ls9oC3HQ3DIIh-_VwrN0QTFMCsUd3u1FXE=.d6e11b3e-3cfa-498f-8414-05f58de65980@github.com>
References: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
 <ZnUOPZdxgn_oUAWUwl3M86YQIv1WRd5KqW3Pq4jrUa8=.ffbc2b01-da31-463f-b28b-6468a13d44cf@github.com>
 <-ZAZv0hi5Fy_rWoZ8WSNMDh07ByOt2U5V345bAxZpJk=.0b5433b1-1cd7-4bcd-b7f4-21a7511fcdac@github.com>
 <UapduVTZB0XDZwvS6OFo1d5WpZ2ixiot1vWgTmNxoHM=.366b77ca-169d-4671-a5f2-428ce8b3f657@github.com>
 <EbxkgSoKoDz2e6iBdtdsTPR4HP60NbxWvyU__7-bpD0=.a4deeb2a-2bb9-4fda-8112-dae1d37756dc@github.com>
 <d_i53Bdt7Ls9oC3HQ3DIIh-_VwrN0QTFMCsUd3u1FXE=.d6e11b3e-3cfa-498f-8414-05f58de65980@github.com>
Message-ID: <Cd876iVFfvmmmnZoizyDGX9CgS0A5LouBwsIVDi7kP4=.2f2bc697-679d-495b-b072-95cc9d1a9b3c@github.com>

On Wed, 18 Oct 2023 20:44:30 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> I was suggesting we move [the logic that raises these triggers](https://github.com/openjdk/shenandoah/blob/master/src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp#L3100-L3123) into the heuristic itself. This would reduce changes in shenandoahHeap.cpp and better encapsulate the logic for the heuristic.
>> 
>> If we want to align evaluation of these triggers with rebuilding the freeset (so that they aren't being evaluated at the same frequency as the young generation), we could add a `heap_has_changed` flag (or something). I'm also working on a change to only evaluate old generation triggers when the young generation trigger has fired.
>
> Thanks for clarifying.  I "guess" the main tradeoff is how often do we do the work of evaluating the trigger condition.  As currently implemented, we evaluate this condition at the end of each GC cycle, which is relatively rare.  If we move it to should_start_gc(), it could be revaluated every ms or so.  On the other hand, after we integrate your change, the question of should_start_gc(OLD) will only be asked at the start of a new young gc, which is also relatively rare.
> 
> I'll move it now, in anticipation of your improvement, if you concur.

Or should we integrate as is, and let you move the test when as part of your improvement?

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/341#discussion_r1364522634

From kdnilsen at openjdk.org  Wed Oct 18 23:37:18 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 18 Oct 2023 23:37:18 GMT
Subject: RFR: 8318057: GenShen: investigate small refactor of remset swap
 code
In-Reply-To: <qEO5M57wWdi78hROoJlrNua7hys3sobUN_pXRVxeL_8=.a2ae0dba-7391-449e-b02d-2c8d25a5f6be@github.com>
References: <qEO5M57wWdi78hROoJlrNua7hys3sobUN_pXRVxeL_8=.a2ae0dba-7391-449e-b02d-2c8d25a5f6be@github.com>
Message-ID: <OsVD7gLjTMIbzVO_aeJN2xj3ep27ECErJ-4nI7rHz6g=.246fca7a-571d-422f-b093-73a7f8d7b65d@github.com>

On Tue, 17 Oct 2023 23:26:31 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

> Some closures were being unnecessarily applied to free regions rather than just to allocated old and young regions. Restrict the closure application to old and young regions as appropriate, eliminating extra generation affiliation checks in some closures. Also fixed a slightly incorrect documentation comment.
> 
> **Testing:**
> - jtreg and stress tests in progress in test pipeline helper
> - locally tested with jtreg hotsot_gc tests and with specjbb with and without heap verification
> - performance differences, if any, to be measured

Thanks for quick turnaround

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/342#pullrequestreview-1686377737

From kdnilsen at openjdk.org  Wed Oct 18 23:38:17 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 18 Oct 2023 23:38:17 GMT
Subject: RFR: 8318190: GenShen: Be less aggressive with triggers for
 old-gen has expanded [v3]
In-Reply-To: <Cd876iVFfvmmmnZoizyDGX9CgS0A5LouBwsIVDi7kP4=.2f2bc697-679d-495b-b072-95cc9d1a9b3c@github.com>
References: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
 <ZnUOPZdxgn_oUAWUwl3M86YQIv1WRd5KqW3Pq4jrUa8=.ffbc2b01-da31-463f-b28b-6468a13d44cf@github.com>
 <-ZAZv0hi5Fy_rWoZ8WSNMDh07ByOt2U5V345bAxZpJk=.0b5433b1-1cd7-4bcd-b7f4-21a7511fcdac@github.com>
 <UapduVTZB0XDZwvS6OFo1d5WpZ2ixiot1vWgTmNxoHM=.366b77ca-169d-4671-a5f2-428ce8b3f657@github.com>
 <EbxkgSoKoDz2e6iBdtdsTPR4HP60NbxWvyU__7-bpD0=.a4deeb2a-2bb9-4fda-8112-dae1d37756dc@github.com>
 <d_i53Bdt7Ls9oC3HQ3DIIh-_VwrN0QTFMCsUd3u1FXE=.d6e11b3e-3cfa-498f-8414-05f58de65980@github.com>
 <Cd876iVFfvmmmnZoizyDGX9CgS0A5LouBwsIVDi7kP4=.2f2bc697-679d-495b-b072-95cc9d1a9b3c@github.com>
Message-ID: <dsBfPWbaE8bFp4HzPAoKCVixzbGqSdNIkXgMsiGMe2Y=.379e20c7-a402-4d17-ba74-f7c90c174e69@github.com>

On Wed, 18 Oct 2023 20:52:50 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Thanks for clarifying.  I "guess" the main tradeoff is how often do we do the work of evaluating the trigger condition.  As currently implemented, we evaluate this condition at the end of each GC cycle, which is relatively rare.  If we move it to should_start_gc(), it could be revaluated every ms or so.  On the other hand, after we integrate your change, the question of should_start_gc(OLD) will only be asked at the start of a new young gc, which is also relatively rare.
>> 
>> I'll move it now, in anticipation of your improvement, if you concur.
>
> Or should we integrate as is, and let you move the test when as part of your improvement?

I'm going to leave code as is for now.  It makes sense to move the trigger evaluation as part of the fix that you (Wiliam Kemper) have under development.  Thanks.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/341#discussion_r1364692588

From kdnilsen at openjdk.org  Wed Oct 18 23:48:41 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 18 Oct 2023 23:48:41 GMT
Subject: RFR: 8318190: GenShen: Be less aggressive with triggers for
 old-gen has expanded [v6]
In-Reply-To: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
References: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
Message-ID: <sOYp1J-DgSRSIxhP8yu6IBEyLZjuRSJ5-HUJg0i7MPA=.01e7dda6-d047-4ba3-b5f3-1a24ee03c5e1@github.com>

> This patch reduces the frequency of old-has-grown triggers in the case that old usage is smaller than a particular threshold percentage (default value 5%) of the total heap size.  In this case, old is triggered for growth no more frequently than once every N cycles, where N defaults to 50.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Change default and fix comment

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/341/files
  - new: https://git.openjdk.org/shenandoah/pull/341/files/8f434fb9..5d0b3e10

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=341&range=05
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=341&range=04-05

  Stats: 4 lines in 2 files changed: 0 ins; 0 del; 4 mod
  Patch: https://git.openjdk.org/shenandoah/pull/341.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/341/head:pull/341

PR: https://git.openjdk.org/shenandoah/pull/341

From kdnilsen at openjdk.org  Wed Oct 18 23:55:32 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 18 Oct 2023 23:55:32 GMT
Subject: Integrated: 8318190: GenShen: Be less aggressive with triggers for
 old-gen has expanded
In-Reply-To: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
References: <kB5yAN8iFEjYAkSYeM3ZcAf_2ejfDfnoBjQabOQ37N8=.12c893db-4663-481e-a86e-2bf038b58831@github.com>
Message-ID: <aMh6CQYeA2-lRn9HUyxi0omoJ-fWYMEcdyk7J7rTC2Q=.0b8094c5-febe-43fa-81bb-588ade701cc3@github.com>

On Fri, 13 Oct 2023 19:16:39 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> This patch reduces the frequency of old-has-grown triggers in the case that old usage is smaller than a particular threshold percentage (default value 5%) of the total heap size.  In this case, old is triggered for growth no more frequently than once every N cycles, where N defaults to 50.

This pull request has now been integrated.

Changeset: ac30d485
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/ac30d485ef494efbdb1def1c97fd0fc1fe982321
Stats:     85 lines in 9 files changed: 64 ins; 3 del; 18 mod

8318190: GenShen: Be less aggressive with triggers for old-gen has expanded

Reviewed-by: wkemper

-------------

PR: https://git.openjdk.org/shenandoah/pull/341

From rkennke at openjdk.org  Thu Oct 19 11:17:15 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 19 Oct 2023 11:17:15 GMT
Subject: RFR: 8318462: [GenShen] Prevent unsafe access to displaced mark-word
Message-ID: <uOeuEpzRFvBx1AFgOvvTNfxJIyI4o84yT9D5A_q6BjU=.92103c05-893a-4b0b-a165-54289729f74f@github.com>

We must be very careful when we access (load or update) the object age concurrently because the age is stored in the object's header and that header can be 'displaced'. This means that the header is overloaded with a pointer (stack or ObjectMonitor*) and the original header is stored at that location. The header can also be in INFLATING state, which means that a stack-lock is currently in the process to be inflated to an ObjectMonitor. Let's consider the cases separately:

Loading object age:
- INFLATING: we can not access the header, and thus not the age.
- Stack-locked: the thread which locks the object can unlock at any time, concurrently. It is not safe to access the header, and thus the age.
- Monitor-locked: monitors don't go away without coordinating with Java threads and possibly GC threads. This coordination is done by handshaking the thread - a Java thread would be brought to its next safepoint, and GC threads, which are typically participating in handshakes by joining the SuspendibleThreadSet, can opt to handshake at a safe point by calling SuspendibleThreadSet::yield(). If 'our' thread participates in handshakes in one way or the other, it is safe to access an ObjectMonitor once we got a valid ObjectMonitor* from an object's header, but only until that thread handshakes. Long story short: it is typically safe to access ObjectMonitor :-)

Updating object age:
Updating object age only happens during evacuation, and only on a new copy of an object, before that copy has been published. Access to the object header is therefore fully thread-local and not problematic. What *is* problematic is when we have to deal with displaced headers, because displaced headers are *not* thread local, and must be considered a shared resource. A competing thread may succeed to evacuate the object and publish its copy before 'our' thread can do so. If 'our' thread would update the displaced header, it may over-write whatever the other thread has done.
- INFLATING: We cannot access the header at all. However, that should not happen: a thread can only inflate once it has successfully evacuated an object, and 
- Stack-locked: the other thread may succeed evacuation and continue to unlock the object, at which point the stack-pointer would be 'dangling' and we may over-write random information on the foreign thread stack.
- Monitor-locked: even though monitor deflation is coordinated (see above), updating the displaced header in a monitor might over-write whatever a competing thread has done there.

For these reasons, we must not update displaced headers, and must be careful when loading displaced headers - in particular, only ever load from ObjectMonitor.

-------------

Commit messages:
 - Assert that no inflation happens
 - 8318462: [Shenandoah] Prevent unsafe access to displaced mark-word

Changes: https://git.openjdk.org/shenandoah/pull/343/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=343&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8318462
  Stats: 36 lines in 3 files changed: 14 ins; 12 del; 10 mod
  Patch: https://git.openjdk.org/shenandoah/pull/343.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/343/head:pull/343

PR: https://git.openjdk.org/shenandoah/pull/343

From ysr at openjdk.org  Thu Oct 19 11:17:15 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 19 Oct 2023 11:17:15 GMT
Subject: RFR: 8318462: [GenShen] Prevent unsafe access to displaced
 mark-word
In-Reply-To: <uOeuEpzRFvBx1AFgOvvTNfxJIyI4o84yT9D5A_q6BjU=.92103c05-893a-4b0b-a165-54289729f74f@github.com>
References: <uOeuEpzRFvBx1AFgOvvTNfxJIyI4o84yT9D5A_q6BjU=.92103c05-893a-4b0b-a165-54289729f74f@github.com>
Message-ID: <GfzQwdshCZcWzSbdO9KFEDuT7v3k3t4qIc0Id6-DZNI=.f33cb4ab-2664-419e-af79-41c3fe6d70e9@github.com>

On Wed, 18 Oct 2023 17:23:15 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

> We must be very careful when we access (load or update) the object age concurrently because the age is stored in the object's header and that header can be 'displaced'. This means that the header is overloaded with a pointer (stack or ObjectMonitor*) and the original header is stored at that location. The header can also be in INFLATING state, which means that a stack-lock is currently in the process to be inflated to an ObjectMonitor. Let's consider the cases separately:
> 
> Loading object age:
> - INFLATING: we can not access the header, and thus not the age.
> - Stack-locked: the thread which locks the object can unlock at any time, concurrently. It is not safe to access the header, and thus the age.
> - Monitor-locked: monitors don't go away without coordinating with Java threads and possibly GC threads. This coordination is done by handshaking the thread - a Java thread would be brought to its next safepoint, and GC threads, which are typically participating in handshakes by joining the SuspendibleThreadSet, can opt to handshake at a safe point by calling SuspendibleThreadSet::yield(). If 'our' thread participates in handshakes in one way or the other, it is safe to access an ObjectMonitor once we got a valid ObjectMonitor* from an object's header, but only until that thread handshakes. Long story short: it is typically safe to access ObjectMonitor :-)
> 
> Updating object age:
> Updating object age only happens during evacuation, and only on a new copy of an object, before that copy has been published. Access to the object header is therefore fully thread-local and not problematic. What *is* problematic is when we have to deal with displaced headers, because displaced headers are *not* thread local, and must be considered a shared resource. A competing thread may succeed to evacuate the object and publish its copy before 'our' thread can do so. If 'our' thread would update the displaced header, it may over-write whatever the other thread has done.
> - INFLATING: We cannot access the header at all. However, that should not happen: a thread can only inflate once it has successfully evacuated an object, and 
> - Stack-locked: the other thread may succeed evacuation and continue to unlock the object, at which point the stack-pointer would be 'dangling' and we may over-write random information on the foreign thread stack.
> - Monitor-locked: even though monitor deflation is coordinated (see above), updating the displaced header in a monitor might over-write whatever a competi...

src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 569:

> 567:   markWord w = obj->mark();
> 568:   if (w.has_monitor()) {
> 569:     w = w.monitor()->header();

Isn't this prone to potential error because the monitor might be recycled/reused between line 567 and line 569?

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/343#discussion_r1364578506

From rkennke at openjdk.org  Thu Oct 19 11:17:15 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 19 Oct 2023 11:17:15 GMT
Subject: RFR: 8318462: [GenShen] Prevent unsafe access to displaced
 mark-word
In-Reply-To: <GfzQwdshCZcWzSbdO9KFEDuT7v3k3t4qIc0Id6-DZNI=.f33cb4ab-2664-419e-af79-41c3fe6d70e9@github.com>
References: <uOeuEpzRFvBx1AFgOvvTNfxJIyI4o84yT9D5A_q6BjU=.92103c05-893a-4b0b-a165-54289729f74f@github.com>
 <GfzQwdshCZcWzSbdO9KFEDuT7v3k3t4qIc0Id6-DZNI=.f33cb4ab-2664-419e-af79-41c3fe6d70e9@github.com>
Message-ID: <-T54h1iLgwsZ3iecZbQtOc9xee_y12ToquoRorY22fE=.2bc8d199-2de0-4dff-a3e2-62166799371a@github.com>

On Wed, 18 Oct 2023 21:53:09 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> We must be very careful when we access (load or update) the object age concurrently because the age is stored in the object's header and that header can be 'displaced'. This means that the header is overloaded with a pointer (stack or ObjectMonitor*) and the original header is stored at that location. The header can also be in INFLATING state, which means that a stack-lock is currently in the process to be inflated to an ObjectMonitor. Let's consider the cases separately:
>> 
>> Loading object age:
>> - INFLATING: we can not access the header, and thus not the age.
>> - Stack-locked: the thread which locks the object can unlock at any time, concurrently. It is not safe to access the header, and thus the age.
>> - Monitor-locked: monitors don't go away without coordinating with Java threads and possibly GC threads. This coordination is done by handshaking the thread - a Java thread would be brought to its next safepoint, and GC threads, which are typically participating in handshakes by joining the SuspendibleThreadSet, can opt to handshake at a safe point by calling SuspendibleThreadSet::yield(). If 'our' thread participates in handshakes in one way or the other, it is safe to access an ObjectMonitor once we got a valid ObjectMonitor* from an object's header, but only until that thread handshakes. Long story short: it is typically safe to access ObjectMonitor :-)
>> 
>> Updating object age:
>> Updating object age only happens during evacuation, and only on a new copy of an object, before that copy has been published. Access to the object header is therefore fully thread-local and not problematic. What *is* problematic is when we have to deal with displaced headers, because displaced headers are *not* thread local, and must be considered a shared resource. A competing thread may succeed to evacuate the object and publish its copy before 'our' thread can do so. If 'our' thread would update the displaced header, it may over-write whatever the other thread has done.
>> - INFLATING: We cannot access the header at all. However, that should not happen: a thread can only inflate once it has successfully evacuated an object, and 
>> - Stack-locked: the other thread may succeed evacuation and continue to unlock the object, at which point the stack-pointer would be 'dangling' and we may over-write random information on the foreign thread stack.
>> - Monitor-locked: even though monitor deflation is coordinated (see above), updating the displaced header in a monitor might over-...
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 569:
> 
>> 567:   markWord w = obj->mark();
>> 568:   if (w.has_monitor()) {
>> 569:     w = w.monitor()->header();
> 
> Isn't this prone to potential error because the monitor might be recycled/reused between line 567 and line 569?

Monitor deflation is coordinated with Java threads by safepointing (or more precisely handshaking) the Java threads. The same works with GC threads if they participate in safepointing/handshaking. This is usually achieved by joining SuspendibleThreadSet. The effect of is that all such threads can safely access monitors without fearing that they go away. When we got a valid monitor pointer, then that monitor will not disappear before the next safepoint, and safepoints cannot happen at random points in execution. For example, a thread that has joined STS would only be able to safepoint when it calls STS::yield(). We might want to verify that all GC threads which can evacuate objects are properly joining (and yielding) STS.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/343#discussion_r1365182526

From rkennke at openjdk.org  Thu Oct 19 12:32:10 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 19 Oct 2023 12:32:10 GMT
Subject: RFR: 8318462: [GenShen] Prevent unsafe access to displaced
 mark-word [v2]
In-Reply-To: <uOeuEpzRFvBx1AFgOvvTNfxJIyI4o84yT9D5A_q6BjU=.92103c05-893a-4b0b-a165-54289729f74f@github.com>
References: <uOeuEpzRFvBx1AFgOvvTNfxJIyI4o84yT9D5A_q6BjU=.92103c05-893a-4b0b-a165-54289729f74f@github.com>
Message-ID: <2lwvFiHSE-N-kXfzhKYx4HhQASkwI01cjVO_tPJ_KZI=.5c873318-cb3f-44e6-8745-d6f5cbe3e52d@github.com>

> We must be very careful when we access (load or update) the object age concurrently because the age is stored in the object's header and that header can be 'displaced'. This means that the header is overloaded with a pointer (stack or ObjectMonitor*) and the original header is stored at that location. The header can also be in INFLATING state, which means that a stack-lock is currently in the process to be inflated to an ObjectMonitor. Let's consider the cases separately:
> 
> Loading object age:
> - INFLATING: we can not access the header, and thus not the age.
> - Stack-locked: the thread which locks the object can unlock at any time, concurrently. It is not safe to access the header, and thus the age.
> - Monitor-locked: monitors don't go away without coordinating with Java threads and possibly GC threads. This coordination is done by handshaking the thread - a Java thread would be brought to its next safepoint, and GC threads, which are typically participating in handshakes by joining the SuspendibleThreadSet, can opt to handshake at a safe point by calling SuspendibleThreadSet::yield(). If 'our' thread participates in handshakes in one way or the other, it is safe to access an ObjectMonitor once we got a valid ObjectMonitor* from an object's header, but only until that thread handshakes. Long story short: it is typically safe to access ObjectMonitor :-)
> 
> Updating object age:
> Updating object age only happens during evacuation, and only on a new copy of an object, before that copy has been published. Access to the object header is therefore fully thread-local and not problematic. What *is* problematic is when we have to deal with displaced headers, because displaced headers are *not* thread local, and must be considered a shared resource. A competing thread may succeed to evacuate the object and publish its copy before 'our' thread can do so. If 'our' thread would update the displaced header, it may over-write whatever the other thread has done.
> - INFLATING: We cannot access the header at all. However, that should not happen: a thread can only inflate once it has successfully evacuated an object, and 
> - Stack-locked: the other thread may succeed evacuation and continue to unlock the object, at which point the stack-pointer would be 'dangling' and we may over-write random information on the foreign thread stack.
> - Monitor-locked: even though monitor deflation is coordinated (see above), updating the displaced header in a monitor might over-write whatever a competi...

Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:

  Handle forwarded headers

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/343/files
  - new: https://git.openjdk.org/shenandoah/pull/343/files/3869c012..dd6329a3

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=343&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=343&range=00-01

  Stats: 10 lines in 1 file changed: 9 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/343.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/343/head:pull/343

PR: https://git.openjdk.org/shenandoah/pull/343

From zgu at openjdk.org  Thu Oct 19 13:07:23 2023
From: zgu at openjdk.org (Zhengyu Gu)
Date: Thu, 19 Oct 2023 13:07:23 GMT
Subject: RFR: 8318462: [GenShen] Prevent unsafe access to displaced
 mark-word [v2]
In-Reply-To: <2lwvFiHSE-N-kXfzhKYx4HhQASkwI01cjVO_tPJ_KZI=.5c873318-cb3f-44e6-8745-d6f5cbe3e52d@github.com>
References: <uOeuEpzRFvBx1AFgOvvTNfxJIyI4o84yT9D5A_q6BjU=.92103c05-893a-4b0b-a165-54289729f74f@github.com>
 <2lwvFiHSE-N-kXfzhKYx4HhQASkwI01cjVO_tPJ_KZI=.5c873318-cb3f-44e6-8745-d6f5cbe3e52d@github.com>
Message-ID: <feghfM7FVv2wf1ImOidYnLI0eSGm27pDetlSsk4lmIU=.2a68f506-fc4a-4aea-bf27-f6812d859e8d@github.com>

On Thu, 19 Oct 2023 12:32:10 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> We must be very careful when we access (load or update) the object age concurrently because the age is stored in the object's header and that header can be 'displaced'. This means that the header is overloaded with a pointer (stack or ObjectMonitor*) and the original header is stored at that location. The header can also be in INFLATING state, which means that a stack-lock is currently in the process to be inflated to an ObjectMonitor. Let's consider the cases separately:
>> 
>> Loading object age:
>> - INFLATING: we can not access the header, and thus not the age.
>> - Stack-locked: the thread which locks the object can unlock at any time, concurrently. It is not safe to access the header, and thus the age.
>> - Monitor-locked: monitors don't go away without coordinating with Java threads and possibly GC threads. This coordination is done by handshaking the thread - a Java thread would be brought to its next safepoint, and GC threads, which are typically participating in handshakes by joining the SuspendibleThreadSet, can opt to handshake at a safe point by calling SuspendibleThreadSet::yield(). If 'our' thread participates in handshakes in one way or the other, it is safe to access an ObjectMonitor once we got a valid ObjectMonitor* from an object's header, but only until that thread handshakes. Long story short: it is typically safe to access ObjectMonitor :-)
>> 
>> Updating object age:
>> Updating object age only happens during evacuation, and only on a new copy of an object, before that copy has been published. Access to the object header is therefore fully thread-local and not problematic. What *is* problematic is when we have to deal with displaced headers, because displaced headers are *not* thread local, and must be considered a shared resource. A competing thread may succeed to evacuate the object and publish its copy before 'our' thread can do so. If 'our' thread would update the displaced header, it may over-write whatever the other thread has done.
>> - INFLATING: We cannot access the header at all. However, that should not happen: a thread can only inflate once it has successfully evacuated an object, and 
>> - Stack-locked: the other thread may succeed evacuation and continue to unlock the object, at which point the stack-pointer would be 'dangling' and we may over-write random information on the foreign thread stack.
>> - Monitor-locked: even though monitor deflation is coordinated (see above), updating the displaced header in a monitor might over-...
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Handle forwarded headers

Just FYI: Here is another place to update object age, probably you want to consolidate.

https://github.com/openjdk/shenandoah/blob/master/src/hotspot/share/gc/shenandoah/shenandoahStringDedup.inline.hpp#L55

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/343#issuecomment-1770949532

From lkorinth at openjdk.org  Thu Oct 19 15:16:13 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Thu, 19 Oct 2023 15:16:13 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v5]
In-Reply-To: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
Message-ID: <uC0TE90D-q3e-Zj64YAyow9DkIridnaRbzlXi0zmr2Q=.b32622c2-fdc3-4fd9-862c-cc9f934dbfe2@github.com>

> Rename createJavaProcessBuilder so that it is not used by mistake instead of createTestJvm.
> 
> I have used the following sed script: `find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createJavaProcessBuilderIgnoreTestJavaOpts(/g"`
> 
> Then I have manually modified ProcessTools.java. In that file I have moved one version of createJavaProcessBuilder so that it is close to the other version. Then I have added a javadoc comment in bold telling:
> 
>    /**
>      * Create ProcessBuilder using the java launcher from the jdk to
>      * be tested.
>      *
>      * <p><b> Please observe that you likely should use
>      * createTestJvm() instead of this method because createTestJvm()
>      * will add JVM options from "test.vm.opts" and "test.java.opts"
>      * </b> and this method will not do that.
>      *
>      * @param command Arguments to pass to the java command.
>      * @return The ProcessBuilder instance representing the java command.
>      */
> 
> 
> I have used the name createJavaProcessBuilderIgnoreTestJavaOpts because of the name of Utils.prependTestJavaOpts that adds those VM flags. If you have a better name I could do a rename of the method. I kind of like that it is long and clumsy, that makes it harder to use...
> 
> I have run tier 1 testing, and I have started more exhaustive testing.

Leo Korinth has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains ten commits:

 - Batch update using sed
   
   find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createLimitedJavaTestProcessBuilder(/g"
   find -name "*.java" | xargs -n 1 sed -i -e "s/createTestJvm(/createJavaTestProcessBuilder(/g"
   find -name "*.java" | xargs -n 1 sed -i -e "s/import static jdk.test.lib.process.ProcessTools.createJavaProcessBuilder/import static jdk.test.lib.process.ProcessTools.createLimitedJavaTestProcessBuilder/g"
 - Merge branch '_master_jdk' into _8315097
 - explain usage
 - Revert "8315097: Rename createJavaProcessBuilder"
   
   This reverts commit 4b2d171133c40c5c48114602bfd0d4da75531317.
 - Revert "copyright"
   
   This reverts commit f3418c80cc0d4cbb722ee5e368f1a001e898b43e.
 - Revert "fix static import"
   
   This reverts commit 27da71508aec9a4bec1c0ad07031887286580171.
 - fix static import
 - copyright
 - 8315097: Rename createJavaProcessBuilder

-------------

Changes: https://git.openjdk.org/jdk/pull/15452/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=15452&range=04
  Stats: 894 lines in 560 files changed: 34 ins; 10 del; 850 mod
  Patch: https://git.openjdk.org/jdk/pull/15452.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15452/head:pull/15452

PR: https://git.openjdk.org/jdk/pull/15452

From lkorinth at openjdk.org  Thu Oct 19 15:16:42 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Thu, 19 Oct 2023 15:16:42 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v4]
In-Reply-To: <4pRda3ZAZzVzGiVrDv6LN9Pw__DhrmTm4qZjTHzaq80=.a009bb29-4869-4047-8b62-80fbe7bef692@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <4pRda3ZAZzVzGiVrDv6LN9Pw__DhrmTm4qZjTHzaq80=.a009bb29-4869-4047-8b62-80fbe7bef692@github.com>
Message-ID: <3XTw5IAFj_YpaMrqyLdO9mNbWDENFMVkwk8JBmKHDcE=.5e4b1c70-0b36-4ff5-9cce-ee4f25a8e3bb@github.com>

On Tue, 17 Oct 2023 12:29:46 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> Rename createJavaProcessBuilder so that it is not used by mistake instead of createTestJvm.
>> 
>> I have used the following sed script: `find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createJavaProcessBuilderIgnoreTestJavaOpts(/g"`
>> 
>> Then I have manually modified ProcessTools.java. In that file I have moved one version of createJavaProcessBuilder so that it is close to the other version. Then I have added a javadoc comment in bold telling:
>> 
>>    /**
>>      * Create ProcessBuilder using the java launcher from the jdk to
>>      * be tested.
>>      *
>>      * <p><b> Please observe that you likely should use
>>      * createTestJvm() instead of this method because createTestJvm()
>>      * will add JVM options from "test.vm.opts" and "test.java.opts"
>>      * </b> and this method will not do that.
>>      *
>>      * @param command Arguments to pass to the java command.
>>      * @return The ProcessBuilder instance representing the java command.
>>      */
>> 
>> 
>> I have used the name createJavaProcessBuilderIgnoreTestJavaOpts because of the name of Utils.prependTestJavaOpts that adds those VM flags. If you have a better name I could do a rename of the method. I kind of like that it is long and clumsy, that makes it harder to use...
>> 
>> I have run tier 1 testing, and I have started more exhaustive testing.
>
> Leo Korinth has updated the pull request incrementally with three additional commits since the last revision:
> 
>  - Revert "8315097: Rename createJavaProcessBuilder"
>    
>    This reverts commit 4b2d171133c40c5c48114602bfd0d4da75531317.
>  - Revert "copyright"
>    
>    This reverts commit f3418c80cc0d4cbb722ee5e368f1a001e898b43e.
>  - Revert "fix static import"
>    
>    This reverts commit 27da71508aec9a4bec1c0ad07031887286580171.

If this looks roughly acceptable, I will manually add indentation spaces. I am now running tests.

The changes can be verified by running the following commands:

git switch -c _reproduce 15acf4b8d7cffcd0d74bf1b9c43cde9acaf31ea9 

find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createLimitedJavaTestProcessBuilder(/g"
find -name "*.java" | xargs -n 1 sed -i -e "s/createTestJvm(/createJavaTestProcessBuilder(/g"
find -name "*.java" | xargs -n 1 sed -i -e "s/import static jdk.test.lib.process.ProcessTools.createJavaProcessBuilder/import static jdk.test.lib.process.ProcessTools.createLimitedJavaTestProcessBuilder/g"

git diff HEAD f80dda8d7109c2ef6bc1f685d0b611704dec645e


Only the documentation changes should be visible.  When I have manually indented everything it should be easy to that verify that change as a whitespace-only change. But that is for tomorrow (at best).

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15452#issuecomment-1771194189

From duke at openjdk.org  Thu Oct 19 16:14:20 2023
From: duke at openjdk.org (Leela Mohan Venati)
Date: Thu, 19 Oct 2023 16:14:20 GMT
Subject: RFR: 8317466: Enable interpreter oopMapCache for concurrent GCs
 [v2]
In-Reply-To: <39f-0nlOdQBABHr1cQOq7jITuRLzM9yLDUEUm1--0N8=.f3aeed11-7151-4885-8376-7d91ca84e8a7@github.com>
References: <dS_92JFN-F9Jizw_KWx1T7EoHQeo6n01CzNV19Xu_iQ=.81a01ca0-0308-4951-9bbf-7bfdd565bdbd@github.com>
 <EAurOs1oZa5lA26Cp2GxnQXB0wamCkfNVuwGTKBXE3I=.9fffd66c-c755-4d1b-9c22-2b23828dc5de@github.com>
 <39f-0nlOdQBABHr1cQOq7jITuRLzM9yLDUEUm1--0N8=.f3aeed11-7151-4885-8376-7d91ca84e8a7@github.com>
Message-ID: <K1kYWYUIopbwlKkxHlQW-hiFgBUFToAsKmgDiNujSKk=.350b2d4d-a57b-467c-af6d-521cafa1aa89@github.com>

On Mon, 16 Oct 2023 20:03:44 GMT, Leela Mohan Venati <duke at openjdk.org> wrote:

>> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Cleanup old oop map cache entry after class redefinition
>
> src/hotspot/share/gc/shenandoah/shenandoahVMOperations.cpp line 64:
> 
>> 62:   OopMapCache::cleanup_old_entries();
>> 63: }
>> 64: 
> 
> Do you think, VM_ShenandoahFinalMarkStartEvac walks the stack roots.  If yes, i recommend adding OopMapCache::cleanup_old_entries() in  VM_ShenandoahOperation::doit_epilogue().  And this would make the change simple and also revert the change in this [PR](https://github.com/openjdk/jdk/pull/15921)

I stand corrected. 

My question is still relevant 
>> Do you think, VM_ShenandoahFinalMarkStartEvac walks the stack roots.

My recommendation is incorrect.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/16074#discussion_r1365804097

From ysr at openjdk.org  Thu Oct 19 18:00:20 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Thu, 19 Oct 2023 18:00:20 GMT
Subject: RFR: 8318057: GenShen: investigate small refactor of remset swap
 code
In-Reply-To: <TM-VGF87U47MXuPOBrRmKgEOU2rp-HzRFIyeoVGLayc=.cc663de8-8fa0-432c-8fcc-f3fc7cf442f7@github.com>
References: <qEO5M57wWdi78hROoJlrNua7hys3sobUN_pXRVxeL_8=.a2ae0dba-7391-449e-b02d-2c8d25a5f6be@github.com>
 <TM-VGF87U47MXuPOBrRmKgEOU2rp-HzRFIyeoVGLayc=.cc663de8-8fa0-432c-8fcc-f3fc7cf442f7@github.com>
Message-ID: <zz3NWOIvSMqrDpOqrKBW5I_2goMPG3A_rP-7qEpPvZU=.1aae23ac-f6f7-4af8-afc7-0ecc1bee6d1b@github.com>

On Wed, 18 Oct 2023 20:40:03 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Do we see any performance improvement from these changes?

No, none. I left some comments & data in the JBS ticket. I am looking at gathering some more data as well a further small improvement which also is unlikely to improve performance by much.

> Has this passed the internal pipeline tests?

Yes. It's failure equivalent to the reference/baseline. (Each exhibits one failure in a particular stress test w/dacapo that is under active investigation in a separate ticket by William, I believe.)

> Thanks for quick turnaround

Thanks for the prompt review, Kelvin. I've updated the description and am making this a draft again to try a couple more improvements, and to gather some performance data along the lines described in the ticket.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/342#issuecomment-1771364655
PR Comment: https://git.openjdk.org/shenandoah/pull/342#issuecomment-1771375212

From iwalulya at openjdk.org  Thu Oct 19 18:07:01 2023
From: iwalulya at openjdk.org (Ivan Walulya)
Date: Thu, 19 Oct 2023 18:07:01 GMT
Subject: RFR: 8317350: Move code cache purging out of
 CodeCache::UnloadingScope
In-Reply-To: <BvcvMniLilmj4oHWHZRAratUJar05TJ1XptU_XoPHm0=.c1c84329-b5e0-4493-8133-aa528dbfb081@github.com>
References: <BvcvMniLilmj4oHWHZRAratUJar05TJ1XptU_XoPHm0=.c1c84329-b5e0-4493-8133-aa528dbfb081@github.com>
Message-ID: <ghCX-zQBQlVTj2-fS0LHDXpRJUf8uBhn78sKLP6Uo_s=.71dccb1a-27ad-4284-be2f-84cb8c6cbd38@github.com>

On Mon, 2 Oct 2023 12:56:27 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

> Hi all,
> 
>   please review this refactoring that moves actual code cache flushing/purging out of `CodeCache::UnloadingScope`. Reasons:
> 
> * I prefer that a destructor does not do anything substantial - in some cases, 90% of time is spent in the destructor in that extracted method (due to https://bugs.openjdk.org/browse/JDK-8316959)
> * imho it does not fit the class which does nothing but sets/resets some code cache unloading behavior (probably should be renamed to `UnloadingBehaviorScope` too in a separate CR).
> * other existing methods at that level are placed out of that (or any other) scope object too - which is already the case for when doing concurrent unloading.
> * putting it there makes future logging of the various phases a little bit easier, not having `GCTraceTimer` et al. in various places.
> 
> Testing: gha
> 
> Thanks,
>   Thomas

Marked as reviewed by iwalulya (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/16011#pullrequestreview-1688249138

From jsjolen at openjdk.org  Thu Oct 19 20:20:21 2023
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Thu, 19 Oct 2023 20:20:21 GMT
Subject: RFR: 8318447: Move NMT source code to own subdirectory
Message-ID: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>

I think that NMT is deserving of its own subdirectory. Can we do a review of the changes before I fix the merge conflicts?

1. Moved all the nmt source code from services/ to nmt/
2. Renamed all the include statements and sorted them
3. Fixed the include guards

-------------

Commit messages:
 - Move NMT to its own subdirectory

Changes: https://git.openjdk.org/jdk/pull/16276/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16276&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8318447
  Stats: 485 lines in 100 files changed: 204 ins; 206 del; 75 mod
  Patch: https://git.openjdk.org/jdk/pull/16276.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/16276/head:pull/16276

PR: https://git.openjdk.org/jdk/pull/16276

From wkemper at openjdk.org  Thu Oct 19 22:03:07 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 19 Oct 2023 22:03:07 GMT
Subject: RFR: 8318570: GenShen: Only start old cycles if a young cycle is ready
 to run
Message-ID: <3NViNLyZrEMg3xrnSzCTgZhfoN598ccSj-Xlpy1gHJQ=.484d5c47-d057-4bef-be5b-4547fe5a16a2@github.com>

Each old cycle in the generational mode is preceded by a "bootstrap" young cycle. When old cycles are triggered independently of young cycles, the bootstrap cycle may be unproductive (especially if it follows a recent young cycle). With this change, the bootstrap cycle effectively replaces the ordinary young cycle.

-------------

Commit messages:
 - Merge remote-tracking branch 'shenandoah/master' into subordinate-old-collections
 - Print heap region details at debug level
 - Transition to waiting-for-fill when mixed collections candidates are empty
 - Merge remote-tracking branch 'shenandoah/master' into subordinate-old-collections
 - Only start old cycles if it is time for a young cycle
 - Do not start a cycle when cause is `no_gc`
 - Fix misleading logging and double cancellation of old collections

Changes: https://git.openjdk.org/shenandoah/pull/344/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=344&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8318570
  Stats: 38 lines in 6 files changed: 13 ins; 0 del; 25 mod
  Patch: https://git.openjdk.org/shenandoah/pull/344.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/344/head:pull/344

PR: https://git.openjdk.org/shenandoah/pull/344

From kdnilsen at openjdk.org  Thu Oct 19 22:11:42 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 19 Oct 2023 22:11:42 GMT
Subject: RFR: 8318570: GenShen: Only start old cycles if a young cycle is
 ready to run
In-Reply-To: <3NViNLyZrEMg3xrnSzCTgZhfoN598ccSj-Xlpy1gHJQ=.484d5c47-d057-4bef-be5b-4547fe5a16a2@github.com>
References: <3NViNLyZrEMg3xrnSzCTgZhfoN598ccSj-Xlpy1gHJQ=.484d5c47-d057-4bef-be5b-4547fe5a16a2@github.com>
Message-ID: <vKSDHhmj8bSTHRUnEJhawXOxOFWmsYAcfaPLcHj3gH4=.ce82255c-a1fe-4f26-80b1-f83e56c1f22c@github.com>

On Thu, 19 Oct 2023 21:56:21 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Each old cycle in the generational mode is preceded by a "bootstrap" young cycle. When old cycles are triggered independently of young cycles, the bootstrap cycle may be unproductive (especially if it follows a recent young cycle). With this change, the bootstrap cycle effectively replaces the ordinary young cycle.

Thanks.

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/344#pullrequestreview-1688760559

From wkemper at openjdk.org  Thu Oct 19 22:23:49 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 19 Oct 2023 22:23:49 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <sNtakAFQZaJykc9t8PSoYuscumjUbDUPENz6htHhFqk=.f7831246-671e-4790-8701-27be6b2773dc@github.com>

Merges tag jdk-22+20

-------------

Commit messages:
 - 8318363: Foreign benchmarks fail to build on some platforms
 - 8318183: C2: VM may crash after hitting node limit
 - 8315974: Make fields final in 'com.sun.crypto.provider' package
 - 8317886: Add @sealedGraph to ByteBuffer
 - 8318365: Test runtime/cds/appcds/sharedStrings/InternSharedString.java fails after JDK-8311538
 - 8309966: Enhanced TLS connections
 - 8286503: Enhance security classes
 - 8297856: Improve handling of Bidi characters
 - 8296581: Better system proxy support
 - 8316967: Correct the scope of vmtimer in UnregisteredClasses::load_class
 - ... and 117 more: https://git.openjdk.org/shenandoah/compare/fb4098ff...6fc35142

The webrev contains the conflicts with master:
 - merge conflicts: https://webrevs.openjdk.org/?repo=shenandoah&pr=346&range=00.conflicts

Changes: https://git.openjdk.org/shenandoah/pull/346/files
  Stats: 16913 lines in 704 files changed: 10828 ins; 3072 del; 3013 mod
  Patch: https://git.openjdk.org/shenandoah/pull/346.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/346/head:pull/346

PR: https://git.openjdk.org/shenandoah/pull/346

From kdnilsen at openjdk.org  Thu Oct 19 22:53:09 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 19 Oct 2023 22:53:09 GMT
Subject: RFR: 8318572: GenShen: Fix error in log message for ingored
 old-has-grown triggers
Message-ID: <qlXWsnd0a9V_WX0YNb4ydCpM_Y4cWopzNFCyFHc0KYo=.289ce22d-8ebb-4a27-bba3-23f52852fa5f@github.com>

Fix messages in log and user guidance.

-------------

Commit messages:
 - Change log_info to log_debug for ignored old-has-grown triggers
 - Fix log message and user guidance

Changes: https://git.openjdk.org/shenandoah/pull/345/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=345&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8318572
  Stats: 11 lines in 2 files changed: 1 ins; 1 del; 9 mod
  Patch: https://git.openjdk.org/shenandoah/pull/345.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/345/head:pull/345

PR: https://git.openjdk.org/shenandoah/pull/345

From stefank at openjdk.org  Fri Oct 20 06:32:44 2023
From: stefank at openjdk.org (Stefan Karlsson)
Date: Fri, 20 Oct 2023 06:32:44 GMT
Subject: RFR: 8318447: Move NMT source code to own subdirectory
In-Reply-To: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
References: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
Message-ID: <y9-5k0oFUWDlFwy2RVEcQJoieSnI-1ISpTQt5BpE-hI=.9edefe93-79cb-40f0-92da-42ce844d8ca0@github.com>

On Thu, 19 Oct 2023 20:06:50 GMT, Johan Sj?len <jsjolen at openjdk.org> wrote:

> I think that NMT is deserving of its own subdirectory. Can we do a review of the changes before I fix the merge conflicts?
> 
> 1. Moved all the nmt source code from services/ to nmt/
> 2. Renamed all the include statements and sorted them
> 3. Fixed the include guards

Changes requested by stefank (Reviewer).

Changes requested by stefank (Reviewer).

src/hotspot/share/gc/parallel/psParallelCompact.cpp line 33:

> 31: #include "code/codeCache.hpp"
> 32: #include "compiler/oopMap.hpp"
> 33: #include "gc/parallel/parMarkBitMap.inline.hpp"

This uses a case-sensitive sort, whereas I think that when we add includes without using a sorting tool we would do so in a case-insensitive manner. I'm not sure we should make this change here. (The same goes for similar cases in the rest of the patch)

src/hotspot/share/gc/shenandoah/shenandoahHeap.cpp line 89:

> 87: #include "runtime/safepointMechanism.hpp"
> 88: #include "runtime/vmThread.hpp"
> 89: #include "nmt/memTracker.hpp"

This included end up at the wrong place.

src/hotspot/share/nmt/memBaseline.cpp line 32:

> 30: #include "runtime/safepoint.hpp"
> 31: #include "nmt/memBaseline.hpp"
> 32: #include "nmt/memTracker.hpp"

Sort order

src/hotspot/share/nmt/nmtPreInit.hpp line 33:

> 31: #include "runtime/atomic.hpp"
> 32: #endif
> 33: #include "nmt/memTracker.hpp"

The ASSERT Section should be moved down to after the main include block, or even better just include it without the guard. Might be nice to fix this while you are moving around the includes.

src/hotspot/share/nmt/threadStackTracker.cpp line 31:

> 29: #include "nmt/memTracker.hpp"
> 30: #include "nmt/threadStackTracker.hpp"
> 31: #include "nmt/virtualMemoryTracker.hpp"

Sort order

src/hotspot/share/nmt/virtualMemoryTracker.hpp line 33:

> 31: #include "nmt/allocationSite.hpp"
> 32: #include "nmt/nmtCommon.hpp"
> 33: #include "utilities/linkedlist.hpp"

This file didn't get an update to the include guard

src/hotspot/share/services/mallocTracker.inline.hpp line 30:

> 28: 
> 29: #include "services/mallocLimit.hpp"
> 30: #include "services/mallocTracker.hpp"

Missing include guard rename

src/hotspot/share/services/nmtPreInit.hpp line 33:

> 31: #include "runtime/atomic.hpp"
> 32: #endif
> 33: #include "services/memTracker.hpp"

Missing include guard rename

test/hotspot/gtest/nmt/test_nmt_buffer_overflow_detection.cpp line 31:

> 29: #include "utilities/debug.hpp"
> 30: #include "utilities/ostream.hpp"
> 31: 

You make this change in some files but not all. Maybe revert this unrelated change?

test/hotspot/gtest/nmt/test_nmt_cornercases.cpp line 26:

> 24: 
> 25: #include "precompiled.hpp"
> 26: 

You make this change in some files but not all. Maybe revert this unrelated change?

-------------

PR Review: https://git.openjdk.org/jdk/pull/16276#pullrequestreview-1689312655
PR Review: https://git.openjdk.org/jdk/pull/16276#pullrequestreview-1689327479
PR Review Comment: https://git.openjdk.org/jdk/pull/16276#discussion_r1366529454
PR Review Comment: https://git.openjdk.org/jdk/pull/16276#discussion_r1366521538
PR Review Comment: https://git.openjdk.org/jdk/pull/16276#discussion_r1366522881
PR Review Comment: https://git.openjdk.org/jdk/pull/16276#discussion_r1366524596
PR Review Comment: https://git.openjdk.org/jdk/pull/16276#discussion_r1366524886
PR Review Comment: https://git.openjdk.org/jdk/pull/16276#discussion_r1366525477
PR Review Comment: https://git.openjdk.org/jdk/pull/16276#discussion_r1366530660
PR Review Comment: https://git.openjdk.org/jdk/pull/16276#discussion_r1366530318
PR Review Comment: https://git.openjdk.org/jdk/pull/16276#discussion_r1366526935
PR Review Comment: https://git.openjdk.org/jdk/pull/16276#discussion_r1366527038

From dholmes at openjdk.org  Fri Oct 20 06:37:33 2023
From: dholmes at openjdk.org (David Holmes)
Date: Fri, 20 Oct 2023 06:37:33 GMT
Subject: RFR: 8318447: Move NMT source code to own subdirectory
In-Reply-To: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
References: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
Message-ID: <yaENONibWyggRm6meBk5cRfWsqx-AesoAyeHTH9imvc=.5e00059e-477c-4893-b7cc-73314e83c101@github.com>

On Thu, 19 Oct 2023 20:06:50 GMT, Johan Sj?len <jsjolen at openjdk.org> wrote:

> I think that NMT is deserving of its own subdirectory. Can we do a review of the changes before I fix the merge conflicts?
> 
> 1. Moved all the nmt source code from services/ to nmt/
> 2. Renamed all the include statements and sorted them
> 3. Fixed the include guards

Looks okay for moving the NMT files.

The unrelated include file ordering changes just made it harder to review though.

Thanks.

src/hotspot/os/linux/os_linux.cpp line 65:

> 63: #include "runtime/threadCritical.hpp"
> 64: #include "runtime/threadSMR.hpp"
> 65: #include "runtime/threads.hpp"

This was correct originally - lowercase sorts before upper.

-------------

Marked as reviewed by dholmes (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/16276#pullrequestreview-1689327994
PR Review Comment: https://git.openjdk.org/jdk/pull/16276#discussion_r1366530642

From stuefe at openjdk.org  Fri Oct 20 07:05:40 2023
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Fri, 20 Oct 2023 07:05:40 GMT
Subject: RFR: 8318447: Move NMT source code to own subdirectory
In-Reply-To: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
References: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
Message-ID: <TFcfU7eKse5AU2UslgnHzOoyXv4p9l6OF86_SB-KujQ=.f0c44bcf-8f22-44a3-be3e-524468851161@github.com>

On Thu, 19 Oct 2023 20:06:50 GMT, Johan Sj?len <jsjolen at openjdk.org> wrote:

> I think that NMT is deserving of its own subdirectory. Can we do a review of the changes before I fix the merge conflicts?
> 
> 1. Moved all the nmt source code from services/ to nmt/
> 2. Renamed all the include statements and sorted them
> 3. Fixed the include guards

Mostly okay. Was overdue.
Don't need another look if you fix Davids and Stefan's remarks.

test/hotspot/gtest/nmt/test_nmt_locationprinting.cpp line 30:

> 28: #include "nmt/mallocHeader.inline.hpp"
> 29: #include "runtime/os.hpp"
> 30: #include "nmt/memTracker.hpp"

order

-------------

Marked as reviewed by stuefe (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/16276#pullrequestreview-1689369969
PR Review Comment: https://git.openjdk.org/jdk/pull/16276#discussion_r1366556167

From tschatzl at openjdk.org  Fri Oct 20 07:33:43 2023
From: tschatzl at openjdk.org (Thomas Schatzl)
Date: Fri, 20 Oct 2023 07:33:43 GMT
Subject: RFR: 8317350: Move code cache purging out of
 CodeCache::UnloadingScope
In-Reply-To: <ghCX-zQBQlVTj2-fS0LHDXpRJUf8uBhn78sKLP6Uo_s=.71dccb1a-27ad-4284-be2f-84cb8c6cbd38@github.com>
References: <BvcvMniLilmj4oHWHZRAratUJar05TJ1XptU_XoPHm0=.c1c84329-b5e0-4493-8133-aa528dbfb081@github.com>
 <ghCX-zQBQlVTj2-fS0LHDXpRJUf8uBhn78sKLP6Uo_s=.71dccb1a-27ad-4284-be2f-84cb8c6cbd38@github.com>
Message-ID: <24PQu5F8UyuLPpat6tVbKzzpIXquqsZqUhJKeiNObTY=.3a9dd5fe-0284-4e39-8621-a17371d4cb59@github.com>

On Thu, 19 Oct 2023 16:59:39 GMT, Ivan Walulya <iwalulya at openjdk.org> wrote:

>> Hi all,
>> 
>>   please review this refactoring that moves actual code cache flushing/purging out of `CodeCache::UnloadingScope`. Reasons:
>> 
>> * I prefer that a destructor does not do anything substantial - in some cases, 90% of time is spent in the destructor in that extracted method (due to https://bugs.openjdk.org/browse/JDK-8316959)
>> * imho it does not fit the class which does nothing but sets/resets some code cache unloading behavior (probably should be renamed to `UnloadingBehaviorScope` too in a separate CR).
>> * other existing methods at that level are placed out of that (or any other) scope object too - which is already the case for when doing concurrent unloading.
>> * putting it there makes future logging of the various phases a little bit easier, not having `GCTraceTimer` et al. in various places.
>> 
>> Testing: gha
>> 
>> Thanks,
>>   Thomas
>
> Marked as reviewed by iwalulya (Reviewer).

Thanks @walulyai @albertnetymk for your reviews

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16011#issuecomment-1772224539

From tschatzl at openjdk.org  Fri Oct 20 07:33:45 2023
From: tschatzl at openjdk.org (Thomas Schatzl)
Date: Fri, 20 Oct 2023 07:33:45 GMT
Subject: Integrated: 8317350: Move code cache purging out of
 CodeCache::UnloadingScope
In-Reply-To: <BvcvMniLilmj4oHWHZRAratUJar05TJ1XptU_XoPHm0=.c1c84329-b5e0-4493-8133-aa528dbfb081@github.com>
References: <BvcvMniLilmj4oHWHZRAratUJar05TJ1XptU_XoPHm0=.c1c84329-b5e0-4493-8133-aa528dbfb081@github.com>
Message-ID: <4Y8Eh6q0_GqZyQwNc3JqCBY6yjXw-GfeMfZJ0wAuvHY=.57cdd6f9-61c2-49ee-a0b9-eab9bf96fbf5@github.com>

On Mon, 2 Oct 2023 12:56:27 GMT, Thomas Schatzl <tschatzl at openjdk.org> wrote:

> Hi all,
> 
>   please review this refactoring that moves actual code cache flushing/purging out of `CodeCache::UnloadingScope`. Reasons:
> 
> * I prefer that a destructor does not do anything substantial - in some cases, 90% of time is spent in the destructor in that extracted method (due to https://bugs.openjdk.org/browse/JDK-8316959)
> * imho it does not fit the class which does nothing but sets/resets some code cache unloading behavior (probably should be renamed to `UnloadingBehaviorScope` too in a separate CR).
> * other existing methods at that level are placed out of that (or any other) scope object too - which is already the case for when doing concurrent unloading.
> * putting it there makes future logging of the various phases a little bit easier, not having `GCTraceTimer` et al. in various places.
> 
> Testing: gha
> 
> Thanks,
>   Thomas

This pull request has now been integrated.

Changeset: bd3bc2c6
Author:    Thomas Schatzl <tschatzl at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/bd3bc2c6181668b5856732666dc251136b7fbb99
Stats:     60 lines in 7 files changed: 26 ins; 6 del; 28 mod

8317350: Move code cache purging out of CodeCache::UnloadingScope

Reviewed-by: ayang, iwalulya

-------------

PR: https://git.openjdk.org/jdk/pull/16011

From lkorinth at openjdk.org  Fri Oct 20 08:34:35 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Fri, 20 Oct 2023 08:34:35 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v5]
In-Reply-To: <uC0TE90D-q3e-Zj64YAyow9DkIridnaRbzlXi0zmr2Q=.b32622c2-fdc3-4fd9-862c-cc9f934dbfe2@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <uC0TE90D-q3e-Zj64YAyow9DkIridnaRbzlXi0zmr2Q=.b32622c2-fdc3-4fd9-862c-cc9f934dbfe2@github.com>
Message-ID: <CQLu4BqqXQpv1m-Z4mKvfK890bjfytSkMMBunchwlmc=.95e9f73e-0275-4dcf-bdbb-707ce50c4902@github.com>

On Thu, 19 Oct 2023 15:16:13 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> Rename createJavaProcessBuilder so that it is not used by mistake instead of createTestJvm.
>> 
>> I have used the following sed script: `find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createJavaProcessBuilderIgnoreTestJavaOpts(/g"`
>> 
>> Then I have manually modified ProcessTools.java. In that file I have moved one version of createJavaProcessBuilder so that it is close to the other version. Then I have added a javadoc comment in bold telling:
>> 
>>    /**
>>      * Create ProcessBuilder using the java launcher from the jdk to
>>      * be tested.
>>      *
>>      * <p><b> Please observe that you likely should use
>>      * createTestJvm() instead of this method because createTestJvm()
>>      * will add JVM options from "test.vm.opts" and "test.java.opts"
>>      * </b> and this method will not do that.
>>      *
>>      * @param command Arguments to pass to the java command.
>>      * @return The ProcessBuilder instance representing the java command.
>>      */
>> 
>> 
>> I have used the name createJavaProcessBuilderIgnoreTestJavaOpts because of the name of Utils.prependTestJavaOpts that adds those VM flags. If you have a better name I could do a rename of the method. I kind of like that it is long and clumsy, that makes it harder to use...
>> 
>> I have run tier 1 testing, and I have started more exhaustive testing.
>
> Leo Korinth has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains ten commits:
> 
>  - Batch update using sed
>    
>    find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createLimitedJavaTestProcessBuilder(/g"
>    find -name "*.java" | xargs -n 1 sed -i -e "s/createTestJvm(/createJavaTestProcessBuilder(/g"
>    find -name "*.java" | xargs -n 1 sed -i -e "s/import static jdk.test.lib.process.ProcessTools.createJavaProcessBuilder/import static jdk.test.lib.process.ProcessTools.createLimitedJavaTestProcessBuilder/g"
>  - Merge branch '_master_jdk' into _8315097
>  - explain usage
>  - Revert "8315097: Rename createJavaProcessBuilder"
>    
>    This reverts commit 4b2d171133c40c5c48114602bfd0d4da75531317.
>  - Revert "copyright"
>    
>    This reverts commit f3418c80cc0d4cbb722ee5e368f1a001e898b43e.
>  - Revert "fix static import"
>    
>    This reverts commit 27da71508aec9a4bec1c0ad07031887286580171.
>  - fix static import
>  - copyright
>  - 8315097: Rename createJavaProcessBuilder

Just ignore what I just pushed, I will have a new version out

sorry...

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15452#issuecomment-1772309579
PR Comment: https://git.openjdk.org/jdk/pull/15452#issuecomment-1772309992

From jsjolen at openjdk.org  Fri Oct 20 11:34:50 2023
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Fri, 20 Oct 2023 11:34:50 GMT
Subject: RFR: 8318447: Move NMT source code to own subdirectory
In-Reply-To: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
References: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
Message-ID: <jtqr_lZBWW0gy8fHQFT0fMRyN_UTmLPEa8zeEKLYwBc=.cc345e4f-4eeb-4951-8edf-ba3b245cc087@github.com>

On Thu, 19 Oct 2023 20:06:50 GMT, Johan Sj?len <jsjolen at openjdk.org> wrote:

> I think that NMT is deserving of its own subdirectory. Can we do a review of the changes before I fix the merge conflicts?
> 
> 1. Moved all the nmt source code from services/ to nmt/
> 2. Renamed all the include statements and sorted them
> 3. Fixed the include guards

Hi,

Thank you for looking through these changes. I applied your comments and also did a run through to look for incorrectly ordered includes. For the gtest source files I separated the includes in a consistent manner, they all look like this pattern now:

```c++
#include "precompiled.hpp"
#include "memory/allocation.hpp"
#include "nmt/mallocHeader.inline.hpp"
#include "nmt/memTracker.hpp"
#include "runtime/os.hpp"

#include "testutils.hpp"
#include "unittest.hpp"

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16276#issuecomment-1772570292

From jsjolen at openjdk.org  Fri Oct 20 11:34:49 2023
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Fri, 20 Oct 2023 11:34:49 GMT
Subject: RFR: 8318447: Move NMT source code to own subdirectory [v2]
In-Reply-To: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
References: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
Message-ID: <n012paZtMaOScD7qvUkJQAS2tlacYkuwf5hcHSxtGrU=.d7b8dd09-c489-44a2-852a-2fe0c398acc8@github.com>

> I think that NMT is deserving of its own subdirectory. Can we do a review of the changes before I fix the merge conflicts?
> 
> 1. Moved all the nmt source code from services/ to nmt/
> 2. Renamed all the include statements and sorted them
> 3. Fixed the include guards

Johan Sj?len has updated the pull request incrementally with one additional commit since the last revision:

  Fixed reviewed changes

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/16276/files
  - new: https://git.openjdk.org/jdk/pull/16276/files/2fd7c355..08e6f4bf

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=16276&range=01
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16276&range=00-01

  Stats: 43 lines in 15 files changed: 19 ins; 15 del; 9 mod
  Patch: https://git.openjdk.org/jdk/pull/16276.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/16276/head:pull/16276

PR: https://git.openjdk.org/jdk/pull/16276

From jsjolen at openjdk.org  Fri Oct 20 12:14:52 2023
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Fri, 20 Oct 2023 12:14:52 GMT
Subject: RFR: 8318447: Move NMT source code to own subdirectory [v3]
In-Reply-To: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
References: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
Message-ID: <a_9o8nuS6P9ix0WPPf5IYAoFMcJWzPdd22Q0cgq4Vv4=.6615354b-d686-4644-9a0b-1376acbd1698@github.com>

> I think that NMT is deserving of its own subdirectory. Can we do a review of the changes before I fix the merge conflicts?
> 
> 1. Moved all the nmt source code from services/ to nmt/
> 2. Renamed all the include statements and sorted them
> 3. Fixed the include guards

Johan Sj?len has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains three commits:

 - Merge remote-tracking branch 'origin/master' into move-nmt
 - Fixed reviewed changes
 - Move NMT to its own subdirectory

-------------

Changes: https://git.openjdk.org/jdk/pull/16276/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16276&range=02
  Stats: 502 lines in 100 files changed: 211 ins; 210 del; 81 mod
  Patch: https://git.openjdk.org/jdk/pull/16276.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/16276/head:pull/16276

PR: https://git.openjdk.org/jdk/pull/16276

From jsjolen at openjdk.org  Fri Oct 20 12:33:50 2023
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Fri, 20 Oct 2023 12:33:50 GMT
Subject: RFR: 8318447: Move NMT source code to own subdirectory [v4]
In-Reply-To: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
References: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
Message-ID: <RrYEp441ITS0bnxR_UHMjEiJOKADkhZnqHZJAhgPKM0=.12d5cc5e-084a-461f-9c6c-8aa0cb7d06ef@github.com>

> I think that NMT is deserving of its own subdirectory. Can we do a review of the changes before I fix the merge conflicts?
> 
> 1. Moved all the nmt source code from services/ to nmt/
> 2. Renamed all the include statements and sorted them
> 3. Fixed the include guards

Johan Sj?len has updated the pull request incrementally with one additional commit since the last revision:

  Missed this include

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/16276/files
  - new: https://git.openjdk.org/jdk/pull/16276/files/c2b14b34..1647b41e

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=16276&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16276&range=02-03

  Stats: 2 lines in 1 file changed: 1 ins; 1 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/16276.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/16276/head:pull/16276

PR: https://git.openjdk.org/jdk/pull/16276

From jsjolen at openjdk.org  Fri Oct 20 12:49:46 2023
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Fri, 20 Oct 2023 12:49:46 GMT
Subject: RFR: 8318447: Move NMT source code to own subdirectory [v5]
In-Reply-To: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
References: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
Message-ID: <Ym7czX6VGn9CV6Y5Bu-XdyQO1NeJOLICgVW6o1ykSYk=.d44c1adc-03ec-48d7-91dc-61b96366d26d@github.com>

> I think that NMT is deserving of its own subdirectory. Can we do a review of the changes before I fix the merge conflicts?
> 
> 1. Moved all the nmt source code from services/ to nmt/
> 2. Renamed all the include statements and sorted them
> 3. Fixed the include guards

Johan Sj?len has updated the pull request incrementally with one additional commit since the last revision:

  Fix messed up include

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/16276/files
  - new: https://git.openjdk.org/jdk/pull/16276/files/1647b41e..4dfb027e

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=16276&range=04
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16276&range=03-04

  Stats: 4 lines in 1 file changed: 2 ins; 2 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/16276.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/16276/head:pull/16276

PR: https://git.openjdk.org/jdk/pull/16276

From wkemper at openjdk.org  Fri Oct 20 15:47:44 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 20 Oct 2023 15:47:44 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <sNtakAFQZaJykc9t8PSoYuscumjUbDUPENz6htHhFqk=.f7831246-671e-4790-8701-27be6b2773dc@github.com>
References: <sNtakAFQZaJykc9t8PSoYuscumjUbDUPENz6htHhFqk=.f7831246-671e-4790-8701-27be6b2773dc@github.com>
Message-ID: <EeGf2W-QhnzG0LiDC_e_fVlf8xwLbUCj4C5d0dRXqXA=.b9c16e1d-6129-4589-9f38-d8747282c255@github.com>

> Merges tag jdk-22+20

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 128 commits:

 - Merge remote-tracking branch 'shenandoah/master' into merge-jdk-22+20
 - 8318363: Foreign benchmarks fail to build on some platforms
   
   Reviewed-by: jvernee, mcimadamore
 - 8318183: C2: VM may crash after hitting node limit
   
   Reviewed-by: kvn, thartmann
 - 8315974: Make fields final in 'com.sun.crypto.provider' package
   
   Reviewed-by: valeriep
 - 8317886: Add @sealedGraph to ByteBuffer
   
   Reviewed-by: rriggs, iris
 - 8318365: Test runtime/cds/appcds/sharedStrings/InternSharedString.java fails after JDK-8311538
   
   Reviewed-by: cjplummer, lmesnik
 - 8309966: Enhanced TLS connections
   
   Reviewed-by: mschoene, hchao, rhalade, jnimeh
 - 8286503: Enhance security classes
   
   Reviewed-by: rhalade, mullan, skoivu, weijun
 - 8297856: Improve handling of Bidi characters
   
   Reviewed-by: rhalade, mschoene, rriggs
 - 8296581: Better system proxy support
   
   Reviewed-by: rhalade, dfuchs, michaelm, alanb
 - ... and 118 more: https://git.openjdk.org/shenandoah/compare/ac30d485...de29536b

-------------

Changes: https://git.openjdk.org/shenandoah/pull/346/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=346&range=01
  Stats: 16896 lines in 699 files changed: 10828 ins; 3057 del; 3011 mod
  Patch: https://git.openjdk.org/shenandoah/pull/346.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/346/head:pull/346

PR: https://git.openjdk.org/shenandoah/pull/346

From wkemper at openjdk.org  Fri Oct 20 17:00:21 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 20 Oct 2023 17:00:21 GMT
Subject: RFR: 8318572: GenShen: Fix error in log message for ingored
 old-has-grown triggers
In-Reply-To: <qlXWsnd0a9V_WX0YNb4ydCpM_Y4cWopzNFCyFHc0KYo=.289ce22d-8ebb-4a27-bba3-23f52852fa5f@github.com>
References: <qlXWsnd0a9V_WX0YNb4ydCpM_Y4cWopzNFCyFHc0KYo=.289ce22d-8ebb-4a27-bba3-23f52852fa5f@github.com>
Message-ID: <6zWb-8DKTMsOIG501zajCRaiVqCh_g3UyXqI7KFEpLs=.fae829d4-a208-4d82-a0bf-6c1655fc6183@github.com>

On Thu, 19 Oct 2023 22:15:09 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Fix messages in log and user guidance.

Marked as reviewed by wkemper (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/345#pullrequestreview-1690551964

From kdnilsen at openjdk.org  Fri Oct 20 17:06:05 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 20 Oct 2023 17:06:05 GMT
Subject: Integrated: 8318572: GenShen: Fix error in log message for ingored
 old-has-grown triggers
In-Reply-To: <qlXWsnd0a9V_WX0YNb4ydCpM_Y4cWopzNFCyFHc0KYo=.289ce22d-8ebb-4a27-bba3-23f52852fa5f@github.com>
References: <qlXWsnd0a9V_WX0YNb4ydCpM_Y4cWopzNFCyFHc0KYo=.289ce22d-8ebb-4a27-bba3-23f52852fa5f@github.com>
Message-ID: <YicoObkXCUtao1CvUYVidqbXQBIKEERdqhEhOrkc8mU=.06a7df13-5509-4db4-9911-68a000c9e51b@github.com>

On Thu, 19 Oct 2023 22:15:09 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Fix messages in log and user guidance.

This pull request has now been integrated.

Changeset: e6c788e0
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/e6c788e0afd2323d43792064d266d9da0fb25f4c
Stats:     11 lines in 2 files changed: 1 ins; 1 del; 9 mod

8318572: GenShen: Fix error in log message for ingored old-has-grown triggers

Reviewed-by: wkemper

-------------

PR: https://git.openjdk.org/shenandoah/pull/345

From kdnilsen at openjdk.org  Fri Oct 20 17:43:13 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Fri, 20 Oct 2023 17:43:13 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v14]
In-Reply-To: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
Message-ID: <J3FLyHBxo-U7YgNPQ-jgr_KM_iEz1QXwNgvbkyvNjUE=.1a657508-b0ed-4306-83f6-a5b3ba351251@github.com>

> Several improvements:
> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
> 2. Reduce likelihood of humongous allocation errors by:
>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.

Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 22 commits:

 - Merge remote-tracking branch 'origin/master' into improve-humongous-allocations
 - Allow for no old regions in assertion
 - One more try at this assertion
 - Fix assert to deal with humongous old
 - Add a comment
 - Fix assertion to deal with humongous accounting
 - Restore ShenandoahFullGCThreshold to 3
 - In-line two small single-use functions
 - Restore _humongous_alloc_failure_gc flag removed in merge conflict
 - Merge master
 - ... and 12 more: https://git.openjdk.org/shenandoah/compare/e6c788e0...07f7f1f9

-------------

Changes: https://git.openjdk.org/shenandoah/pull/336/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=13
  Stats: 247 lines in 12 files changed: 191 ins; 8 del; 48 mod
  Patch: https://git.openjdk.org/shenandoah/pull/336.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/336/head:pull/336

PR: https://git.openjdk.org/shenandoah/pull/336

From wkemper at openjdk.org  Fri Oct 20 18:13:08 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 20 Oct 2023 18:13:08 GMT
Subject: Integrated: 8318570: GenShen: Only start old cycles if a young cycle
 is ready to run
In-Reply-To: <3NViNLyZrEMg3xrnSzCTgZhfoN598ccSj-Xlpy1gHJQ=.484d5c47-d057-4bef-be5b-4547fe5a16a2@github.com>
References: <3NViNLyZrEMg3xrnSzCTgZhfoN598ccSj-Xlpy1gHJQ=.484d5c47-d057-4bef-be5b-4547fe5a16a2@github.com>
Message-ID: <_6K__xTttKVKKgQ5xc3R6ShJO3mvuVOxWyWENS4bnNE=.562b1d38-4bb0-41d6-a36e-ee8160434f44@github.com>

On Thu, 19 Oct 2023 21:56:21 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Each old cycle in the generational mode is preceded by a "bootstrap" young cycle. When old cycles are triggered independently of young cycles, the bootstrap cycle may be unproductive (especially if it follows a recent young cycle). With this change, the bootstrap cycle effectively replaces the ordinary young cycle.

This pull request has now been integrated.

Changeset: 5e15c2b1
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/5e15c2b150b8ed1e5331cf02d37a0821c85a5746
Stats:     38 lines in 6 files changed: 13 ins; 0 del; 25 mod

8318570: GenShen: Only start old cycles if a young cycle is ready to run

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/344

From wkemper at openjdk.org  Fri Oct 20 20:34:07 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 20 Oct 2023 20:34:07 GMT
Subject: RFR: 8318619: GenShen: Add configurable threshold for young triggers
 to expedite old evacuations
Message-ID: <-xp6ofpP2uCkVcHW-RNl-JvuR5XlKGTf1dzT1PYUYOg=.ff3703fe-6500-459f-b137-42147736fd89@github.com>

When Shenandoah expects to free up space by promotions into young, or evacuations in old it may trigger a young collection. This change sets configurable thresholds for these conditions.

-------------

Commit messages:
 - Can't used checked cast here
 - Create configurable thresholds for expedited triggers

Changes: https://git.openjdk.org/shenandoah/pull/347/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=347&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8318619
  Stats: 16 lines in 2 files changed: 13 ins; 1 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/347.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/347/head:pull/347

PR: https://git.openjdk.org/shenandoah/pull/347

From dholmes at openjdk.org  Mon Oct 23 06:14:39 2023
From: dholmes at openjdk.org (David Holmes)
Date: Mon, 23 Oct 2023 06:14:39 GMT
Subject: RFR: 8318447: Move NMT source code to own subdirectory [v5]
In-Reply-To: <Ym7czX6VGn9CV6Y5Bu-XdyQO1NeJOLICgVW6o1ykSYk=.d44c1adc-03ec-48d7-91dc-61b96366d26d@github.com>
References: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
 <Ym7czX6VGn9CV6Y5Bu-XdyQO1NeJOLICgVW6o1ykSYk=.d44c1adc-03ec-48d7-91dc-61b96366d26d@github.com>
Message-ID: <FR6P8zCrJg8BaUa0OZWXGzUUBZImEp7xq-rxTvZS6DY=.3936237a-d1b7-46f2-a38a-84fb6bccddee@github.com>

On Fri, 20 Oct 2023 12:49:46 GMT, Johan Sj?len <jsjolen at openjdk.org> wrote:

>> I think that NMT is deserving of its own subdirectory. Can we do a review of the changes before I fix the merge conflicts?
>> 
>> 1. Moved all the nmt source code from services/ to nmt/
>> 2. Renamed all the include statements and sorted them
>> 3. Fixed the include guards
>
> Johan Sj?len has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix messed up include

Still good.

(Your merge commit seems to have confused GitHub for some reason.)

-------------

Marked as reviewed by dholmes (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/16276#pullrequestreview-1691770241

From jsjolen at openjdk.org  Mon Oct 23 08:34:59 2023
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Mon, 23 Oct 2023 08:34:59 GMT
Subject: RFR: 8318447: Move NMT source code to own subdirectory [v6]
In-Reply-To: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
References: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
Message-ID: <yL6X_wpeZKjxsJJ2puMPpMp8gslTnhg7UNxFC1fvgV8=.7b1d5d12-0cac-4f8a-b739-e23a1fb64f97@github.com>

> I think that NMT is deserving of its own subdirectory. Can we do a review of the changes before I fix the merge conflicts?
> 
> 1. Moved all the nmt source code from services/ to nmt/
> 2. Renamed all the include statements and sorted them
> 3. Fixed the include guards

Johan Sj?len has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits:

 - Merge remote-tracking branch 'origin/master' into move-nmt
 - Fix messed up include
 - Missed this include
 - Merge remote-tracking branch 'origin/master' into move-nmt
 - Fixed reviewed changes
 - Move NMT to its own subdirectory

-------------

Changes: https://git.openjdk.org/jdk/pull/16276/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16276&range=05
  Stats: 507 lines in 102 files changed: 214 ins; 212 del; 81 mod
  Patch: https://git.openjdk.org/jdk/pull/16276.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/16276/head:pull/16276

PR: https://git.openjdk.org/jdk/pull/16276

From stefank at openjdk.org  Mon Oct 23 08:41:39 2023
From: stefank at openjdk.org (Stefan Karlsson)
Date: Mon, 23 Oct 2023 08:41:39 GMT
Subject: RFR: 8318447: Move NMT source code to own subdirectory [v5]
In-Reply-To: <Ym7czX6VGn9CV6Y5Bu-XdyQO1NeJOLICgVW6o1ykSYk=.d44c1adc-03ec-48d7-91dc-61b96366d26d@github.com>
References: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
 <Ym7czX6VGn9CV6Y5Bu-XdyQO1NeJOLICgVW6o1ykSYk=.d44c1adc-03ec-48d7-91dc-61b96366d26d@github.com>
Message-ID: <se-1yZM5MPloQgTUFP811cnvbP_AYMcIzyZR9F1JgSw=.248905ff-92b9-4e97-bd18-d487d385f51a@github.com>

On Fri, 20 Oct 2023 12:49:46 GMT, Johan Sj?len <jsjolen at openjdk.org> wrote:

>> I think that NMT is deserving of its own subdirectory. Can we do a review of the changes before I fix the merge conflicts?
>> 
>> 1. Moved all the nmt source code from services/ to nmt/
>> 2. Renamed all the include statements and sorted them
>> 3. Fixed the include guards
>
> Johan Sj?len has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Fix messed up include

Changes requested by stefank (Reviewer).

src/hotspot/share/nmt/memTracker.inline.hpp line 30:

> 28: 
> 29: #include "nmt/mallocTracker.inline.hpp"
> 30: #include "nmt/memTracker.hpp"

Bonus points if you fix the include for this .inline.hpp file.
Suggestion:

#include "nmt/memTracker.hpp"

#include "nmt/mallocTracker.inline.hpp"

src/hotspot/share/nmt/nmtPreInit.hpp line 35:

> 33: #include "utilities/macros.hpp"
> 34: 
> 35: #ifdef ASSERT

The blank line at 34 is not following the style for our conditional includes. Remove it, or better yet skip conventionalize the include of runtime/atomic.hpp since it just adds to noise to the file.

-------------

PR Review: https://git.openjdk.org/jdk/pull/16276#pullrequestreview-1691967144
PR Review Comment: https://git.openjdk.org/jdk/pull/16276#discussion_r1368296670
PR Review Comment: https://git.openjdk.org/jdk/pull/16276#discussion_r1368299406

From stefank at openjdk.org  Mon Oct 23 08:44:43 2023
From: stefank at openjdk.org (Stefan Karlsson)
Date: Mon, 23 Oct 2023 08:44:43 GMT
Subject: RFR: 8318447: Move NMT source code to own subdirectory
In-Reply-To: <jtqr_lZBWW0gy8fHQFT0fMRyN_UTmLPEa8zeEKLYwBc=.cc345e4f-4eeb-4951-8edf-ba3b245cc087@github.com>
References: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
 <jtqr_lZBWW0gy8fHQFT0fMRyN_UTmLPEa8zeEKLYwBc=.cc345e4f-4eeb-4951-8edf-ba3b245cc087@github.com>
Message-ID: <mknsPgZrSLBWBnaE4BYz9VzrMWGynmVmfi7OmM48kaY=.1a47a4ba-aca4-4ee8-9318-819936c6a758@github.com>

On Fri, 20 Oct 2023 11:31:11 GMT, Johan Sj?len <jsjolen at openjdk.org> wrote:

> For the gtest source files I separated the includes in a consistent manner, they all look like this pattern now:

That's not what I see in the latest patch. Could you revert that separation and then we can consider that style change in a separate RFE?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16276#issuecomment-1774703929

From stefank at openjdk.org  Mon Oct 23 08:44:42 2023
From: stefank at openjdk.org (Stefan Karlsson)
Date: Mon, 23 Oct 2023 08:44:42 GMT
Subject: RFR: 8318447: Move NMT source code to own subdirectory [v6]
In-Reply-To: <yL6X_wpeZKjxsJJ2puMPpMp8gslTnhg7UNxFC1fvgV8=.7b1d5d12-0cac-4f8a-b739-e23a1fb64f97@github.com>
References: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
 <yL6X_wpeZKjxsJJ2puMPpMp8gslTnhg7UNxFC1fvgV8=.7b1d5d12-0cac-4f8a-b739-e23a1fb64f97@github.com>
Message-ID: <ZbZ1ty9PilecM83cb1oBj0N-n5uhWrk9EC3JrjqW_Is=.1c010381-dbe5-499e-8905-e71293591d5b@github.com>

On Mon, 23 Oct 2023 08:34:59 GMT, Johan Sj?len <jsjolen at openjdk.org> wrote:

>> I think that NMT is deserving of its own subdirectory. Can we do a review of the changes before I fix the merge conflicts?
>> 
>> 1. Moved all the nmt source code from services/ to nmt/
>> 2. Renamed all the include statements and sorted them
>> 3. Fixed the include guards
>
> Johan Sj?len has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains six commits:
> 
>  - Merge remote-tracking branch 'origin/master' into move-nmt
>  - Fix messed up include
>  - Missed this include
>  - Merge remote-tracking branch 'origin/master' into move-nmt
>  - Fixed reviewed changes
>  - Move NMT to its own subdirectory

Changes requested by stefank (Reviewer).

src/hotspot/share/services/mallocLimit.cpp line 32:

> 30: #include "runtime/globals.hpp"
> 31: #include "services/mallocLimit.hpp"
> 32: #include "nmt/nmtCommon.hpp"

Sort order

test/hotspot/gtest/nmt/test_nmt_cornercases.cpp line 30:

> 28: #include "nmt/mallocTracker.hpp"
> 29: #include "runtime/os.hpp"
> 30: #include "nmt/memTracker.hpp"

Sort order

-------------

PR Review: https://git.openjdk.org/jdk/pull/16276#pullrequestreview-1692002562
PR Review Comment: https://git.openjdk.org/jdk/pull/16276#discussion_r1368319264
PR Review Comment: https://git.openjdk.org/jdk/pull/16276#discussion_r1368318877

From wkemper at openjdk.org  Mon Oct 23 17:18:15 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 23 Oct 2023 17:18:15 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v14]
In-Reply-To: <J3FLyHBxo-U7YgNPQ-jgr_KM_iEz1QXwNgvbkyvNjUE=.1a657508-b0ed-4306-83f6-a5b3ba351251@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
 <J3FLyHBxo-U7YgNPQ-jgr_KM_iEz1QXwNgvbkyvNjUE=.1a657508-b0ed-4306-83f6-a5b3ba351251@github.com>
Message-ID: <93YpzE3aBBNlQYrItOGMhHBm6jcP9LbcTHIqlOWiWJs=.75952f7a-0814-485f-ab34-7285db1878f0@github.com>

On Fri, 20 Oct 2023 17:43:13 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Several improvements:
>> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
>> 2. Reduce likelihood of humongous allocation errors by:
>>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.
>
> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 22 commits:
> 
>  - Merge remote-tracking branch 'origin/master' into improve-humongous-allocations
>  - Allow for no old regions in assertion
>  - One more try at this assertion
>  - Fix assert to deal with humongous old
>  - Add a comment
>  - Fix assertion to deal with humongous accounting
>  - Restore ShenandoahFullGCThreshold to 3
>  - In-line two small single-use functions
>  - Restore _humongous_alloc_failure_gc flag removed in merge conflict
>  - Merge master
>  - ... and 12 more: https://git.openjdk.org/shenandoah/compare/e6c788e0...07f7f1f9

Typo in log fragmentation trigger's log message

src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 174:

> 172: 
> 173:   {
> 174:     // TODO: Not sure there is value in logging free-set status right here

Doesn't early cleanup rebuild the freeset?

-------------

Changes requested by wkemper (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/336#pullrequestreview-1693090361
PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1369003428

From wkemper at openjdk.org  Mon Oct 23 17:18:15 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 23 Oct 2023 17:18:15 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v3]
In-Reply-To: <XnKW598w0w5ngjNXNyVp78jaKKhtWZLUwHIEBaNJreQ=.47ac9996-afc3-42e1-a061-5dedbc78070b@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
 <3-Diw93ErtU2_lT8G1fnDe-yY6dJw-EAoBHr31BAKZw=.4eb9f20a-bc99-47fd-a0a6-b4684eab0ecd@github.com>
 <VvO_JspOaG9Z-7dQ9o6QMZXArtZudOxey8vRqh8rlic=.4eefa9d5-e922-4a92-9538-86bc6a454978@github.com>
 <3vXpepGcrIdcbx_a5v5YJKOsOlpXiKh0_Bjtm5p8-nc=.1b5c31b3-5afb-4c0d-8c44-2555f158b189@github.com>
 <nrh0PYfMucy4CQONGgn9c_y5zo7HDPKl663IbqJFquQ=.cea00125-5812-45f6-b482-69600f88cc50@github.com>
 <0YwQrRd2UH7R772tyG2wPvEr__rypL_71AmUAfDL8K4=.ea2751b2-7a1e-4ae8-a2c7-276cda8d8969@github.com>
 <XnKW598w0w5ngjNXNyVp78jaKKhtWZLUwHIEBaNJreQ=.47ac9996-afc3-42e1-a061-5dedbc78070b@github.com>
Message-ID: <BzqKmpJAQSLJElpRTS_DeNbk7jCKTpgt1dtvRYJ3HTQ=.49f38ae4-a219-418d-9c3c-542eb78e9954@github.com>

On Fri, 13 Oct 2023 19:23:57 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> It looks like `fragmentation_trigger_reason` just copies the values of the member fields into its arguments.
>
> Thanks for clarifying.  I kind of like the encapsulating the update as is, because it makes more clear to the code maintainer what these fields represent, and how they got their values.  For performance, I'll add inline to the function definition.  Is that ok?

Okay. Looks like a typo in the log message: `densityr`. A comment describing what `density` is would also be welcome.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1369000495

From wkemper at openjdk.org  Mon Oct 23 17:18:15 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 23 Oct 2023 17:18:15 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v3]
In-Reply-To: <BzqKmpJAQSLJElpRTS_DeNbk7jCKTpgt1dtvRYJ3HTQ=.49f38ae4-a219-418d-9c3c-542eb78e9954@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
 <3-Diw93ErtU2_lT8G1fnDe-yY6dJw-EAoBHr31BAKZw=.4eb9f20a-bc99-47fd-a0a6-b4684eab0ecd@github.com>
 <VvO_JspOaG9Z-7dQ9o6QMZXArtZudOxey8vRqh8rlic=.4eefa9d5-e922-4a92-9538-86bc6a454978@github.com>
 <3vXpepGcrIdcbx_a5v5YJKOsOlpXiKh0_Bjtm5p8-nc=.1b5c31b3-5afb-4c0d-8c44-2555f158b189@github.com>
 <nrh0PYfMucy4CQONGgn9c_y5zo7HDPKl663IbqJFquQ=.cea00125-5812-45f6-b482-69600f88cc50@github.com>
 <0YwQrRd2UH7R772tyG2wPvEr__rypL_71AmUAfDL8K4=.ea2751b2-7a1e-4ae8-a2c7-276cda8d8969@github.com>
 <XnKW598w0w5ngjNXNyVp78jaKKhtWZLUwHIEBaNJreQ=.47ac9996-afc3-42e1-a061-5dedbc78070b@github.com>
 <BzqKmpJAQSLJElpRTS_DeNbk7jCKTpgt1dtvRYJ3HTQ=.49f38ae4-a219-418d-9c3c-542eb78e9954@github.com>
Message-ID: <nNd3xDAESShI6IoRkTZFnNV8G90t6T0Yl_dvE3izSZY=.e87a3ff0-d445-4a57-90dc-cbd38f3beccc@github.com>

On Mon, 23 Oct 2023 17:10:17 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Thanks for clarifying.  I kind of like the encapsulating the update as is, because it makes more clear to the code maintainer what these fields represent, and how they got their values.  For performance, I'll add inline to the function definition.  Is that ok?
>
> Okay. Looks like a typo in the log message: `densityr`. A comment describing what `density` is would also be welcome.

Maybe a method `log_fragmentation_trigger_reason`?

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1369002496

From kdnilsen at openjdk.org  Mon Oct 23 18:15:14 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 23 Oct 2023 18:15:14 GMT
Subject: RFR: 8318619: GenShen: Add configurable threshold for young
 triggers to expedite old evacuations
In-Reply-To: <-xp6ofpP2uCkVcHW-RNl-JvuR5XlKGTf1dzT1PYUYOg=.ff3703fe-6500-459f-b137-42147736fd89@github.com>
References: <-xp6ofpP2uCkVcHW-RNl-JvuR5XlKGTf1dzT1PYUYOg=.ff3703fe-6500-459f-b137-42147736fd89@github.com>
Message-ID: <EaiGl8ndJNaDonXsNP0-PofMGcAhvnRiyGxImG4JxDU=.88899dc4-dbd4-4994-81f6-8d675803f6fc@github.com>

On Fri, 20 Oct 2023 20:29:48 GMT, William Kemper <wkemper at openjdk.org> wrote:

> When Shenandoah expects to free up space by promotions into young, or evacuations in old it may trigger a young collection. This change sets configurable thresholds for these conditions.

Thanks.

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/347#pullrequestreview-1693203804

From zgu at openjdk.org  Mon Oct 23 19:30:27 2023
From: zgu at openjdk.org (Zhengyu Gu)
Date: Mon, 23 Oct 2023 19:30:27 GMT
Subject: RFR: 8317466: Enable interpreter oopMapCache for concurrent GCs
 [v2]
In-Reply-To: <K1kYWYUIopbwlKkxHlQW-hiFgBUFToAsKmgDiNujSKk=.350b2d4d-a57b-467c-af6d-521cafa1aa89@github.com>
References: <dS_92JFN-F9Jizw_KWx1T7EoHQeo6n01CzNV19Xu_iQ=.81a01ca0-0308-4951-9bbf-7bfdd565bdbd@github.com>
 <EAurOs1oZa5lA26Cp2GxnQXB0wamCkfNVuwGTKBXE3I=.9fffd66c-c755-4d1b-9c22-2b23828dc5de@github.com>
 <39f-0nlOdQBABHr1cQOq7jITuRLzM9yLDUEUm1--0N8=.f3aeed11-7151-4885-8376-7d91ca84e8a7@github.com>
 <K1kYWYUIopbwlKkxHlQW-hiFgBUFToAsKmgDiNujSKk=.350b2d4d-a57b-467c-af6d-521cafa1aa89@github.com>
Message-ID: <DeJklrchJFZogoBsceyZkZ_KNS9VXWyynt9bviILi7U=.63f3e220-5e7b-4628-825b-e3e2751d36b2@github.com>

On Thu, 19 Oct 2023 16:11:56 GMT, Leela Mohan Venati <duke at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahVMOperations.cpp line 64:
>> 
>>> 62:   OopMapCache::cleanup_old_entries();
>>> 63: }
>>> 64: 
>> 
>> Do you think, VM_ShenandoahFinalMarkStartEvac walks the stack roots.  If yes, i recommend adding OopMapCache::cleanup_old_entries() in  VM_ShenandoahOperation::doit_epilogue().  And this would make the change simple and also revert the change in this [PR](https://github.com/openjdk/jdk/pull/15921)
>
> I stand corrected. 
> 
> My question is still relevant 
>>> Do you think, VM_ShenandoahFinalMarkStartEvac walks the stack roots.
> 
> My recommendation is incorrect.

No, `VM_ShenandoahFinalMarkStartEvac ` does not walk the stack roots, it signals the end of mark phase.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/16074#discussion_r1369175629

From zgu at openjdk.org  Mon Oct 23 19:30:30 2023
From: zgu at openjdk.org (Zhengyu Gu)
Date: Mon, 23 Oct 2023 19:30:30 GMT
Subject: RFR: 8317466: Enable interpreter oopMapCache for concurrent GCs
 [v2]
In-Reply-To: <39f-0nlOdQBABHr1cQOq7jITuRLzM9yLDUEUm1--0N8=.f3aeed11-7151-4885-8376-7d91ca84e8a7@github.com>
References: <dS_92JFN-F9Jizw_KWx1T7EoHQeo6n01CzNV19Xu_iQ=.81a01ca0-0308-4951-9bbf-7bfdd565bdbd@github.com>
 <EAurOs1oZa5lA26Cp2GxnQXB0wamCkfNVuwGTKBXE3I=.9fffd66c-c755-4d1b-9c22-2b23828dc5de@github.com>
 <39f-0nlOdQBABHr1cQOq7jITuRLzM9yLDUEUm1--0N8=.f3aeed11-7151-4885-8376-7d91ca84e8a7@github.com>
Message-ID: <cYQw1rshBQ-qrgOQQKtBRTik3VprcsTciazZDCF4KYA=.48a2b193-1eb4-4bdd-aa99-df12f73f467b@github.com>

On Mon, 16 Oct 2023 19:56:52 GMT, Leela Mohan Venati <duke at openjdk.org> wrote:

>> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Cleanup old oop map cache entry after class redefinition
>
> src/hotspot/share/oops/method.cpp line 311:
> 
>> 309: void Method::mask_for(int bci, InterpreterOopMap* mask) {
>> 310:   methodHandle h_this(Thread::current(), this);
>> 311:   method_holder()->mask_for(h_this, bci, mask);
> 
> Removing this condition allows all the threads including java threads to use/mutate oopMapCache. 
> 
> For ex: Java threads calls [JVM_CallStackWalk](https://github.com/openjdk/jdk/blob/741ae06c55de65dcdfe38e328022bd8dde4fa007/src/hotspot/share/prims/jvm.cpp#L586) which walks the stack and calls locals() and expressions [here](https://github.com/openjdk/jdk/blob/741ae06c55de65dcdfe38e328022bd8dde4fa007/src/hotspot/share/prims/stackwalk.cpp#L345)  which access oopMapCache.

The `oopMapCache` now is fully concurrent, it can be used/modified by Java threads.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/16074#discussion_r1369176823

From kdnilsen at openjdk.org  Mon Oct 23 20:04:09 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 23 Oct 2023 20:04:09 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v14]
In-Reply-To: <93YpzE3aBBNlQYrItOGMhHBm6jcP9LbcTHIqlOWiWJs=.75952f7a-0814-485f-ab34-7285db1878f0@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
 <J3FLyHBxo-U7YgNPQ-jgr_KM_iEz1QXwNgvbkyvNjUE=.1a657508-b0ed-4306-83f6-a5b3ba351251@github.com>
 <93YpzE3aBBNlQYrItOGMhHBm6jcP9LbcTHIqlOWiWJs=.75952f7a-0814-485f-ab34-7285db1878f0@github.com>
Message-ID: <_noZ98yRsbsIjUUfXXZnb-uX4NBDQoHtXL-HQe4P8mM=.37ae0901-c19d-4a8d-9ce9-83fc8f6fd2b1@github.com>

On Mon, 23 Oct 2023 17:15:42 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Typo in log fragmentation trigger's log message

Thanks.  Fixing.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/336#issuecomment-1775930764

From kdnilsen at openjdk.org  Mon Oct 23 20:40:32 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 23 Oct 2023 20:40:32 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v14]
In-Reply-To: <93YpzE3aBBNlQYrItOGMhHBm6jcP9LbcTHIqlOWiWJs=.75952f7a-0814-485f-ab34-7285db1878f0@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
 <J3FLyHBxo-U7YgNPQ-jgr_KM_iEz1QXwNgvbkyvNjUE=.1a657508-b0ed-4306-83f6-a5b3ba351251@github.com>
 <93YpzE3aBBNlQYrItOGMhHBm6jcP9LbcTHIqlOWiWJs=.75952f7a-0814-485f-ab34-7285db1878f0@github.com>
Message-ID: <xbp9xjUfGxfGy__pEPO8N3bPJ4CkehTJg4pnNbR2nmo=.c5d89ada-d387-4a2a-b204-3967144ad5e0@github.com>

On Mon, 23 Oct 2023 17:13:18 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 22 commits:
>> 
>>  - Merge remote-tracking branch 'origin/master' into improve-humongous-allocations
>>  - Allow for no old regions in assertion
>>  - One more try at this assertion
>>  - Fix assert to deal with humongous old
>>  - Add a comment
>>  - Fix assertion to deal with humongous accounting
>>  - Restore ShenandoahFullGCThreshold to 3
>>  - In-line two small single-use functions
>>  - Restore _humongous_alloc_failure_gc flag removed in merge conflict
>>  - Merge master
>>  - ... and 12 more: https://git.openjdk.org/shenandoah/compare/e6c788e0...07f7f1f9
>
> src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 174:
> 
>> 172: 
>> 173:   {
>> 174:     // TODO: Not sure there is value in logging free-set status right here
> 
> Doesn't early cleanup rebuild the freeset?

I'm not sure.  It does recycle immediate trash.  But it appears to me that the rebuild of freeset following the recycling of trash happens somewhere else.  Of note, the last thing done at the end of each free set rebuild is to log the status, so even if we do rebuild inside entry_cleanup_early(), there would be no need to log the status again right here.

Of course, one reason I left this as a TODO comment instead of just yanking out the code is because it is a bit difficult to understand the control flows, so I thought this could deserve some additional attention at a later time.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1369240000

From kdnilsen at openjdk.org  Mon Oct 23 20:46:37 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 23 Oct 2023 20:46:37 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v15]
In-Reply-To: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
Message-ID: <2EgXO4a_hyoOM1ayGbUIPSmdWMUBxzF_PhU4q57-s0U=.5eeb18cd-9400-4c03-a6aa-55b1c543525f@github.com>

> Several improvements:
> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
> 2. Reduce likelihood of humongous allocation errors by:
>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Fix up comments and log message
  
  As prompted by reviewer feedback.

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/336/files
  - new: https://git.openjdk.org/shenandoah/pull/336/files/07f7f1f9..60ba8dae

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=14
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=13-14

  Stats: 3 lines in 2 files changed: 1 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/336.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/336/head:pull/336

PR: https://git.openjdk.org/shenandoah/pull/336

From duke at openjdk.org  Mon Oct 23 20:56:36 2023
From: duke at openjdk.org (Leela Mohan Venati)
Date: Mon, 23 Oct 2023 20:56:36 GMT
Subject: RFR: 8317466: Enable interpreter oopMapCache for concurrent GCs
 [v2]
In-Reply-To: <EAurOs1oZa5lA26Cp2GxnQXB0wamCkfNVuwGTKBXE3I=.9fffd66c-c755-4d1b-9c22-2b23828dc5de@github.com>
References: <dS_92JFN-F9Jizw_KWx1T7EoHQeo6n01CzNV19Xu_iQ=.81a01ca0-0308-4951-9bbf-7bfdd565bdbd@github.com>
 <EAurOs1oZa5lA26Cp2GxnQXB0wamCkfNVuwGTKBXE3I=.9fffd66c-c755-4d1b-9c22-2b23828dc5de@github.com>
Message-ID: <yG8iCpUYWIZ25iFaPq_ADkHRnyTSE6jAJUFUTBpfV8I=.1a7dbb95-94c8-49bd-9a28-8e3699465dd7@github.com>

On Wed, 11 Oct 2023 18:50:14 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> Interpreter oop maps are computed lazily during GC root scan and they are expensive to compute.
>> 
>> GCs uses a small hash table per instance class to cache computed oop maps during STW root scan, but not for concurrent root scan.
>> 
>> This patch is intended to enable `OopMapCache` for concurrent GCs.
>> 
>> Test:
>>   tier1 and tier2 fastdebug and release on MacOSX, Linux 86_84 and Linux 86_32.
>
> Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Cleanup old oop map cache entry after class redefinition

Before of this change,  VM_Operations which enabled "is_gc_active" and walks stack to collect roots had a call to OopMapCache::cleanup_old_entries() in its corresponding doit_epilogue().  This is to free the old entries which are collected during safepoint. 


Now scope of the change isn't clear.

We seem to extend them to concurrent GCs during their concurrent phases (Not just safepoints of concurrent GCs).  Calling OopMapCache::cleanup_old_entries() in doit_epilogue() would effectively cleanup old entries accumulated during concurrent phase of the GC and also during safepoint.  

But change also allows java threads to accumulate old entries. When/Who calls cleanup_old_entries() in this case ? These needs to wait until future GC which does cleanup in doit_epilogue(). However, at least theoretically, we can have large time windows without GCs. Old entries accumulated by java threads can be seen as a leak (until next GC)


A separate thought  to make change simpler. Can cleanup_old_entries become a [list of Cleanup tasks](https://github.com/openjdk/jdk/blob/8d9a4b43f4fff30fd217dab2c224e641cb913c18/src/hotspot/share/runtime/safepoint.hpp#L72) VM thread does during cleanup phase.

-------------

PR Review: https://git.openjdk.org/jdk/pull/16074#pullrequestreview-1693431578

From duke at openjdk.org  Mon Oct 23 20:56:38 2023
From: duke at openjdk.org (Leela Mohan Venati)
Date: Mon, 23 Oct 2023 20:56:38 GMT
Subject: RFR: 8317466: Enable interpreter oopMapCache for concurrent GCs
 [v2]
In-Reply-To: <DeJklrchJFZogoBsceyZkZ_KNS9VXWyynt9bviILi7U=.63f3e220-5e7b-4628-825b-e3e2751d36b2@github.com>
References: <dS_92JFN-F9Jizw_KWx1T7EoHQeo6n01CzNV19Xu_iQ=.81a01ca0-0308-4951-9bbf-7bfdd565bdbd@github.com>
 <EAurOs1oZa5lA26Cp2GxnQXB0wamCkfNVuwGTKBXE3I=.9fffd66c-c755-4d1b-9c22-2b23828dc5de@github.com>
 <39f-0nlOdQBABHr1cQOq7jITuRLzM9yLDUEUm1--0N8=.f3aeed11-7151-4885-8376-7d91ca84e8a7@github.com>
 <K1kYWYUIopbwlKkxHlQW-hiFgBUFToAsKmgDiNujSKk=.350b2d4d-a57b-467c-af6d-521cafa1aa89@github.com>
 <DeJklrchJFZogoBsceyZkZ_KNS9VXWyynt9bviILi7U=.63f3e220-5e7b-4628-825b-e3e2751d36b2@github.com>
Message-ID: <5-zr90Y1fr5aTEKhVxgYJOgqiwGYtil5jW66h3mgE1w=.ae65f301-52ac-4ce1-93c2-a39b31d8d56f@github.com>

On Mon, 23 Oct 2023 19:26:03 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> I stand corrected. 
>> 
>> My question is still relevant 
>>>> Do you think, VM_ShenandoahFinalMarkStartEvac walks the stack roots.
>> 
>> My recommendation is incorrect.
>
> No, `VM_ShenandoahFinalMarkStartEvac ` does not walk the stack roots, it signals the end of mark phase.

Note we don't need to call   OopMapCache::cleanup_old_entries() after this [PR](https://github.com/openjdk/jdk/pull/15921).  VM_ShenandoahFinalMarkStartEvac is derived from VM_ShenandoahReferenceOperation. VM_ShenandoahReferenceOperation::doit_epilogue calls OopMapCache::cleanup_old_entries();

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/16074#discussion_r1369207915

From duke at openjdk.org  Mon Oct 23 20:56:41 2023
From: duke at openjdk.org (Leela Mohan Venati)
Date: Mon, 23 Oct 2023 20:56:41 GMT
Subject: RFR: 8317466: Enable interpreter oopMapCache for concurrent GCs
 [v2]
In-Reply-To: <cYQw1rshBQ-qrgOQQKtBRTik3VprcsTciazZDCF4KYA=.48a2b193-1eb4-4bdd-aa99-df12f73f467b@github.com>
References: <dS_92JFN-F9Jizw_KWx1T7EoHQeo6n01CzNV19Xu_iQ=.81a01ca0-0308-4951-9bbf-7bfdd565bdbd@github.com>
 <EAurOs1oZa5lA26Cp2GxnQXB0wamCkfNVuwGTKBXE3I=.9fffd66c-c755-4d1b-9c22-2b23828dc5de@github.com>
 <39f-0nlOdQBABHr1cQOq7jITuRLzM9yLDUEUm1--0N8=.f3aeed11-7151-4885-8376-7d91ca84e8a7@github.com>
 <cYQw1rshBQ-qrgOQQKtBRTik3VprcsTciazZDCF4KYA=.48a2b193-1eb4-4bdd-aa99-df12f73f467b@github.com>
Message-ID: <fymYc2-NuJcEHDNLtDsMd3eh9npGndUYvQv0aArm6JM=.a55b008c-9d13-4691-bf68-5c7da2c4cb6e@github.com>

On Mon, 23 Oct 2023 19:27:26 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

>> src/hotspot/share/oops/method.cpp line 311:
>> 
>>> 309: void Method::mask_for(int bci, InterpreterOopMap* mask) {
>>> 310:   methodHandle h_this(Thread::current(), this);
>>> 311:   method_holder()->mask_for(h_this, bci, mask);
>> 
>> Removing this condition allows all the threads including java threads to use/mutate oopMapCache. 
>> 
>> For ex: Java threads calls [JVM_CallStackWalk](https://github.com/openjdk/jdk/blob/741ae06c55de65dcdfe38e328022bd8dde4fa007/src/hotspot/share/prims/jvm.cpp#L586) which walks the stack and calls locals() and expressions [here](https://github.com/openjdk/jdk/blob/741ae06c55de65dcdfe38e328022bd8dde4fa007/src/hotspot/share/prims/stackwalk.cpp#L345)  which access oopMapCache.
>
> The `oopMapCache` now is fully concurrent, it can be used/modified by Java threads.

Which thread takes up responsibility of cleaning up old entries accumulated by java threads.  Allowing for java threads definitely increases the scope (than it is mentioned in the bug ).

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/16074#discussion_r1369212468

From kdnilsen at openjdk.org  Mon Oct 23 21:29:17 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 23 Oct 2023 21:29:17 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v16]
In-Reply-To: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
Message-ID: <UczmIy84I4lamuThS3irZ6YbInItq75Z6JqXRgmmlE8=.d4ec2366-7deb-424e-96b5-0e3f6e9cd82a@github.com>

> Several improvements:
> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
> 2. Reduce likelihood of humongous allocation errors by:
>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.

Kelvin Nilsen has updated the pull request incrementally with two additional commits since the last revision:

 - Fix merge conflict
 - Remove redundant option to request dump of memory logs

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/336/files
  - new: https://git.openjdk.org/shenandoah/pull/336/files/60ba8dae..baa1e183

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=15
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=14-15

  Stats: 12 lines in 2 files changed: 1 ins; 10 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/336.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/336/head:pull/336

PR: https://git.openjdk.org/shenandoah/pull/336

From kdnilsen at openjdk.org  Mon Oct 23 22:27:22 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 23 Oct 2023 22:27:22 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v17]
In-Reply-To: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
Message-ID: <YiELdg82uB6Kq938XE9GKwRF70ZDN4X3yd6kQkzPqco=.b734408b-e592-4c82-993c-1ef26f5f267c@github.com>

> Several improvements:
> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
> 2. Reduce likelihood of humongous allocation errors by:
>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.

Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 26 commits:

 - Merge remote-tracking branch 'origin/master' into improve-humongous-allocations
 - Fix merge conflict
 - Remove redundant option to request dump of memory logs
 - Fix up comments and log message
   
   As prompted by reviewer feedback.
 - Merge remote-tracking branch 'origin/master' into improve-humongous-allocations
 - Allow for no old regions in assertion
 - One more try at this assertion
 - Fix assert to deal with humongous old
 - Add a comment
 - Fix assertion to deal with humongous accounting
 - ... and 16 more: https://git.openjdk.org/shenandoah/compare/5e15c2b1...18ea9c0d

-------------

Changes: https://git.openjdk.org/shenandoah/pull/336/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=16
  Stats: 236 lines in 12 files changed: 183 ins; 7 del; 46 mod
  Patch: https://git.openjdk.org/shenandoah/pull/336.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/336/head:pull/336

PR: https://git.openjdk.org/shenandoah/pull/336

From kdnilsen at openjdk.org  Mon Oct 23 22:27:27 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 23 Oct 2023 22:27:27 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v16]
In-Reply-To: <UczmIy84I4lamuThS3irZ6YbInItq75Z6JqXRgmmlE8=.d4ec2366-7deb-424e-96b5-0e3f6e9cd82a@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
 <UczmIy84I4lamuThS3irZ6YbInItq75Z6JqXRgmmlE8=.d4ec2366-7deb-424e-96b5-0e3f6e9cd82a@github.com>
Message-ID: <ad2JXPWTIx4_96GABJCHaH_luOuAr_wKTRZaLnwKtqk=.2bf5a3a4-77b2-4343-8deb-1f27dafb236e@github.com>

On Mon, 23 Oct 2023 21:29:17 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Several improvements:
>> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
>> 2. Reduce likelihood of humongous allocation errors by:
>>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.
>
> Kelvin Nilsen has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Fix merge conflict
>  - Remove redundant option to request dump of memory logs

> > If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
> 
> Can we set `ShenandoahFullGCThreshold` back to the upstream default of 3 now? It should have been done when we merged [openjdk/jdk#15500](https://github.com/openjdk/jdk/pull/15500) .

This has been reverted to 3.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/336#issuecomment-1776117404

From kdnilsen at openjdk.org  Mon Oct 23 22:31:03 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 23 Oct 2023 22:31:03 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v3]
In-Reply-To: <nNd3xDAESShI6IoRkTZFnNV8G90t6T0Yl_dvE3izSZY=.e87a3ff0-d445-4a57-90dc-cbd38f3beccc@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
 <3-Diw93ErtU2_lT8G1fnDe-yY6dJw-EAoBHr31BAKZw=.4eb9f20a-bc99-47fd-a0a6-b4684eab0ecd@github.com>
 <VvO_JspOaG9Z-7dQ9o6QMZXArtZudOxey8vRqh8rlic=.4eefa9d5-e922-4a92-9538-86bc6a454978@github.com>
 <3vXpepGcrIdcbx_a5v5YJKOsOlpXiKh0_Bjtm5p8-nc=.1b5c31b3-5afb-4c0d-8c44-2555f158b189@github.com>
 <nrh0PYfMucy4CQONGgn9c_y5zo7HDPKl663IbqJFquQ=.cea00125-5812-45f6-b482-69600f88cc50@github.com>
 <0YwQrRd2UH7R772tyG2wPvEr__rypL_71AmUAfDL8K4=.ea2751b2-7a1e-4ae8-a2c7-276cda8d8969@github.com>
 <XnKW598w0w5ngjNXNyVp78jaKKhtWZLUwHIEBaNJreQ=.47ac9996-afc3-42e1-a061-5dedbc78070b@github.com>
 <BzqKmpJAQSLJElpRTS_DeNbk7jCKTpgt1dtvRYJ3HTQ=.49f38ae4-a219-418d-9c3c-542eb78e9954@github.com>
 <nNd3xDAESShI6IoRkTZFnNV8G90t6T0Yl_dvE3izSZY=.e87a3ff0-d445-4a57-90dc-cbd38f3beccc@github.com>
Message-ID: <A5SjgKYDrGd0x6TQGbH9jcgu6NuguCOKWmU4RonsmE4=.2809fcfd-2a26-48e2-9d13-1f9ccfdb5450@github.com>

On Mon, 23 Oct 2023 17:12:19 GMT, William Kemper <wkemper at openjdk.org> wrote:

>> Okay. Looks like a typo in the log message: `densityr`. A comment describing what `density` is would also be welcome.
>
> Maybe a method `log_fragmentation_trigger_reason`?

Thanks. Fixing the log message, and fixing this method name.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1369325036

From wkemper at openjdk.org  Mon Oct 23 22:39:03 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 23 Oct 2023 22:39:03 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v14]
In-Reply-To: <xbp9xjUfGxfGy__pEPO8N3bPJ4CkehTJg4pnNbR2nmo=.c5d89ada-d387-4a2a-b204-3967144ad5e0@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
 <J3FLyHBxo-U7YgNPQ-jgr_KM_iEz1QXwNgvbkyvNjUE=.1a657508-b0ed-4306-83f6-a5b3ba351251@github.com>
 <93YpzE3aBBNlQYrItOGMhHBm6jcP9LbcTHIqlOWiWJs=.75952f7a-0814-485f-ab34-7285db1878f0@github.com>
 <xbp9xjUfGxfGy__pEPO8N3bPJ4CkehTJg4pnNbR2nmo=.c5d89ada-d387-4a2a-b204-3967144ad5e0@github.com>
Message-ID: <J4AuFEdcMHoLvAF79GABMe36-7NJrpMzRPrC3_3ri-I=.f2e94cda-c272-48cd-930f-bc09d3e5c7b2@github.com>

On Mon, 23 Oct 2023 20:37:14 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahConcurrentGC.cpp line 174:
>> 
>>> 172: 
>>> 173:   {
>>> 174:     // TODO: Not sure there is value in logging free-set status right here
>> 
>> Doesn't early cleanup rebuild the freeset?
>
> I'm not sure.  It does recycle immediate trash.  But it appears to me that the rebuild of freeset following the recycling of trash happens somewhere else.  Of note, the last thing done at the end of each free set rebuild is to log the status, so even if we do rebuild inside entry_cleanup_early(), there would be no need to log the status again right here.
> 
> Of course, one reason I left this as a TODO comment instead of just yanking out the code is because it is a bit difficult to understand the control flows, so I thought this could deserve some additional attention at a later time.

Okay.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/336#discussion_r1369329555

From kdnilsen at openjdk.org  Mon Oct 23 22:47:11 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 23 Oct 2023 22:47:11 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v18]
In-Reply-To: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
Message-ID: <6i3338jYqhKwolQtGI-1Q1vEU-au2WssXhYLqa6bNcc=.8c2e8ee4-8f5d-4423-b1be-8632448116d9@github.com>

> Several improvements:
> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
> 2. Reduce likelihood of humongous allocation errors by:
>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Improve name of fragmentation_trigger_reason method

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/336/files
  - new: https://git.openjdk.org/shenandoah/pull/336/files/18ea9c0d..b0e0999f

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=17
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=16-17

  Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/336.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/336/head:pull/336

PR: https://git.openjdk.org/shenandoah/pull/336

From wkemper at openjdk.org  Mon Oct 23 23:40:10 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 23 Oct 2023 23:40:10 GMT
Subject: Integrated: 8318619: GenShen: Add configurable threshold for young
 triggers to expedite old evacuations
In-Reply-To: <-xp6ofpP2uCkVcHW-RNl-JvuR5XlKGTf1dzT1PYUYOg=.ff3703fe-6500-459f-b137-42147736fd89@github.com>
References: <-xp6ofpP2uCkVcHW-RNl-JvuR5XlKGTf1dzT1PYUYOg=.ff3703fe-6500-459f-b137-42147736fd89@github.com>
Message-ID: <4TBQN4JwxLTK0GinlXEvVMfZZlj43eILZHkZxhqROck=.9181e047-65bc-4b0d-acfc-863393bd943c@github.com>

On Fri, 20 Oct 2023 20:29:48 GMT, William Kemper <wkemper at openjdk.org> wrote:

> When Shenandoah expects to free up space by promotions into young, or evacuations in old it may trigger a young collection. This change sets configurable thresholds for these conditions.

This pull request has now been integrated.

Changeset: 0c863645
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/0c8636452cd286d06c32ace4b922ac23afd6c41e
Stats:     16 lines in 2 files changed: 13 ins; 1 del; 2 mod

8318619: GenShen: Add configurable threshold for young triggers to expedite old evacuations

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/347

From wkemper at openjdk.org  Mon Oct 23 23:47:22 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 23 Oct 2023 23:47:22 GMT
Subject: RFR: 8315560: GenShen: assert failed: Object klass pointer must go to
 metaspace
Message-ID: <rdud6-488rNRQNUgtbKE7luTVfFgAhDKUThdDpjP4K0=.fa259ffd-de6a-4fbe-b9b6-26b781ef5711@github.com>

We have seen crashes caused when the interned string table is cleaned concurrently by mutator actions. Strings which have become unreachable may be inserted in the SATB buffer during concurrent mark of old. If these pointers reside in young regions which are later promoted in place to old regions, the referenced object (which is young and unmarked) will be filled.  When the old marking tries to follow through the pointer, it may crash.

-------------

Commit messages:
 - Do not promote regions in place during marking of the old generation

Changes: https://git.openjdk.org/shenandoah/pull/348/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=348&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8315560
  Stats: 3 lines in 2 files changed: 1 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/348.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/348/head:pull/348

PR: https://git.openjdk.org/shenandoah/pull/348

From zgu at openjdk.org  Mon Oct 23 23:48:34 2023
From: zgu at openjdk.org (Zhengyu Gu)
Date: Mon, 23 Oct 2023 23:48:34 GMT
Subject: RFR: 8317466: Enable interpreter oopMapCache for concurrent GCs
 [v2]
In-Reply-To: <yG8iCpUYWIZ25iFaPq_ADkHRnyTSE6jAJUFUTBpfV8I=.1a7dbb95-94c8-49bd-9a28-8e3699465dd7@github.com>
References: <dS_92JFN-F9Jizw_KWx1T7EoHQeo6n01CzNV19Xu_iQ=.81a01ca0-0308-4951-9bbf-7bfdd565bdbd@github.com>
 <EAurOs1oZa5lA26Cp2GxnQXB0wamCkfNVuwGTKBXE3I=.9fffd66c-c755-4d1b-9c22-2b23828dc5de@github.com>
 <yG8iCpUYWIZ25iFaPq_ADkHRnyTSE6jAJUFUTBpfV8I=.1a7dbb95-94c8-49bd-9a28-8e3699465dd7@github.com>
Message-ID: <1u5Usl2mr-Q2jvGdBW0V0qVeC4u-Aqh0MdZS03lIUJk=.45e08b12-60f0-41b0-a76f-5ad2a0aff4ea@github.com>

On Mon, 23 Oct 2023 20:53:39 GMT, Leela Mohan Venati <duke at openjdk.org> wrote:

> Before of this change, VM_Operations which enabled "is_gc_active" and walks stack to collect roots had a call to OopMapCache::cleanup_old_entries() in its corresponding doit_epilogue(). This is to free the old entries which are collected during safepoint.
> 
> Now scope of the change isn't clear.
> 
> We seem to extend them to concurrent GCs during their concurrent phases (Not just safepoints of concurrent GCs). Calling OopMapCache::cleanup_old_entries() in doit_epilogue() would effectively cleanup old entries accumulated during concurrent phase of the GC and also during safepoint.
> 
> But change also allows java threads to accumulate old entries. When/Who calls cleanup_old_entries() in this case ? These needs to wait until future GC which does cleanup in doit_epilogue(). However, at least theoretically, we can have large time windows without GCs. Old entries accumulated by java threads can be seen as a leak (until next GC)
> 
Yes, it could accumulate a few. But from what I saw, there are not many. They are cleanup by:
- Serial, Parallel and G1: any `VM_GC_Operation`s
- ZGC: any `VM_ZOperation`s
- Shenandoah: I should have added `OopMapCache::cleanup_old_entries()` in `VM_ShenandoahOperation`, where should help jdk11u backport.

> A separate thought to make change simpler. Can cleanup_old_entries become a [list of Cleanup tasks](https://github.com/openjdk/jdk/blob/8d9a4b43f4fff30fd217dab2c224e641cb913c18/src/hotspot/share/runtime/safepoint.hpp#L72) VM thread does during cleanup phase.

No. There is no point to add safepoint latency when it can not done outside of safepoint.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16074#issuecomment-1776227534

From wkemper at openjdk.org  Mon Oct 23 23:49:59 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 23 Oct 2023 23:49:59 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v18]
In-Reply-To: <6i3338jYqhKwolQtGI-1Q1vEU-au2WssXhYLqa6bNcc=.8c2e8ee4-8f5d-4423-b1be-8632448116d9@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
 <6i3338jYqhKwolQtGI-1Q1vEU-au2WssXhYLqa6bNcc=.8c2e8ee4-8f5d-4423-b1be-8632448116d9@github.com>
Message-ID: <ZFcD4eBf1o_UZ9LMWZSQwN_2kLweqabsjl7so5g9hpg=.45597b90-3674-4311-95f0-2034c3d3b01d@github.com>

On Mon, 23 Oct 2023 22:47:11 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Several improvements:
>> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
>> 2. Reduce likelihood of humongous allocation errors by:
>>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.
>
> Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Improve name of fragmentation_trigger_reason method

Marked as reviewed by wkemper (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/336#pullrequestreview-1693719953

From kdnilsen at openjdk.org  Mon Oct 23 23:57:59 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 23 Oct 2023 23:57:59 GMT
Subject: RFR: 8315560: GenShen: assert failed: Object klass pointer must go
 to metaspace
In-Reply-To: <rdud6-488rNRQNUgtbKE7luTVfFgAhDKUThdDpjP4K0=.fa259ffd-de6a-4fbe-b9b6-26b781ef5711@github.com>
References: <rdud6-488rNRQNUgtbKE7luTVfFgAhDKUThdDpjP4K0=.fa259ffd-de6a-4fbe-b9b6-26b781ef5711@github.com>
Message-ID: <Cnvish5uvAy9xsvG6JsaHll1KG0A0AwV86R85VnKVJg=.5e5c3d8c-ba61-4c14-8406-8d2cf979a1cd@github.com>

On Mon, 23 Oct 2023 23:41:48 GMT, William Kemper <wkemper at openjdk.org> wrote:

> We have seen crashes caused when the interned string table is cleaned concurrently by mutator actions. Strings which have become unreachable may be inserted in the SATB buffer during concurrent mark of old. If these pointers reside in young regions which are later promoted in place to old regions, the referenced object (which is young and unmarked) will be filled.  When the old marking tries to follow through the pointer, it may crash.

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/348#pullrequestreview-1693725500

From lkorinth at openjdk.org  Tue Oct 24 07:49:30 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Tue, 24 Oct 2023 07:49:30 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v6]
In-Reply-To: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
Message-ID: <CxPRMRrp5YUzS45-ci5i1hRr3pLNRE_y1QsPH-_NnSM=.0bd7e9ea-f56c-4f52-8bb4-a8aa06501881@github.com>

> Rename createJavaProcessBuilder so that it is not used by mistake instead of createTestJvm.
> 
> I have used the following sed script: `find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createJavaProcessBuilderIgnoreTestJavaOpts(/g"`
> 
> Then I have manually modified ProcessTools.java. In that file I have moved one version of createJavaProcessBuilder so that it is close to the other version. Then I have added a javadoc comment in bold telling:
> 
>    /**
>      * Create ProcessBuilder using the java launcher from the jdk to
>      * be tested.
>      *
>      * <p><b> Please observe that you likely should use
>      * createTestJvm() instead of this method because createTestJvm()
>      * will add JVM options from "test.vm.opts" and "test.java.opts"
>      * </b> and this method will not do that.
>      *
>      * @param command Arguments to pass to the java command.
>      * @return The ProcessBuilder instance representing the java command.
>      */
> 
> 
> I have used the name createJavaProcessBuilderIgnoreTestJavaOpts because of the name of Utils.prependTestJavaOpts that adds those VM flags. If you have a better name I could do a rename of the method. I kind of like that it is long and clumsy, that makes it harder to use...
> 
> I have run tier 1 testing, and I have started more exhaustive testing.

Leo Korinth has updated the pull request incrementally with six additional commits since the last revision:

 - static import
 - copyright
 - whitespace
 - whitespace
 - sed
 - fix test/lib/jdk/test/lib/process/ProcessTools.java

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/15452/files
  - new: https://git.openjdk.org/jdk/pull/15452/files/f80dda8d..2f57a32d

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=15452&range=05
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15452&range=04-05

  Stats: 1580 lines in 560 files changed: 44 ins; 34 del; 1502 mod
  Patch: https://git.openjdk.org/jdk/pull/15452.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15452/head:pull/15452

PR: https://git.openjdk.org/jdk/pull/15452

From lkorinth at openjdk.org  Tue Oct 24 09:10:43 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Tue, 24 Oct 2023 09:10:43 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v6]
In-Reply-To: <CxPRMRrp5YUzS45-ci5i1hRr3pLNRE_y1QsPH-_NnSM=.0bd7e9ea-f56c-4f52-8bb4-a8aa06501881@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <CxPRMRrp5YUzS45-ci5i1hRr3pLNRE_y1QsPH-_NnSM=.0bd7e9ea-f56c-4f52-8bb4-a8aa06501881@github.com>
Message-ID: <U_NEwp1RAk4Dq_2OXmgi4TOR6vYQF9IVj-PU6X7-h-M=.8ee4eda5-7d48-44d7-b896-3237fae1620d@github.com>

On Tue, 24 Oct 2023 07:49:30 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> Rename createJavaProcessBuilder so that it is not used by mistake instead of createTestJvm.
>> 
>> I have used the following sed script: `find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createJavaProcessBuilderIgnoreTestJavaOpts(/g"`
>> 
>> Then I have manually modified ProcessTools.java. In that file I have moved one version of createJavaProcessBuilder so that it is close to the other version. Then I have added a javadoc comment in bold telling:
>> 
>>    /**
>>      * Create ProcessBuilder using the java launcher from the jdk to
>>      * be tested.
>>      *
>>      * <p><b> Please observe that you likely should use
>>      * createTestJvm() instead of this method because createTestJvm()
>>      * will add JVM options from "test.vm.opts" and "test.java.opts"
>>      * </b> and this method will not do that.
>>      *
>>      * @param command Arguments to pass to the java command.
>>      * @return The ProcessBuilder instance representing the java command.
>>      */
>> 
>> 
>> I have used the name createJavaProcessBuilderIgnoreTestJavaOpts because of the name of Utils.prependTestJavaOpts that adds those VM flags. If you have a better name I could do a rename of the method. I kind of like that it is long and clumsy, that makes it harder to use...
>> 
>> I have run tier 1 testing, and I have started more exhaustive testing.
>
> Leo Korinth has updated the pull request incrementally with six additional commits since the last revision:
> 
>  - static import
>  - copyright
>  - whitespace
>  - whitespace
>  - sed
>  - fix test/lib/jdk/test/lib/process/ProcessTools.java

Hi, if you want to see what I have modified manually, you can do my sed commands and compare to this pull request:

git switch -c _reproduce 15acf4b8d7cffcd0d74bf1b9c43cde9acaf31ea9

find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createLimitedTestJavaProcessBuilder(/g"
find -name "*.java" | xargs -n 1 sed -i -e "s/createTestJvm(/createTestJavaProcessBuilder(/g"
find -name "*.java" | xargs -n 1 sed -i -e "s/import static jdk.test.lib.process.ProcessTools.createJavaProcessBuilder/import static jdk.test.lib.process.ProcessTools.createLimitedTestJavaProcessBuilder/g"
find -name "*.java" | xargs -n 1 sed -i -e "s/import static jdk.test.lib.process.ProcessTools.createTestJvm/import static jdk.test.lib.process.ProcessTools.createTestJavaProcessBuilder/g"
git add -u; git commit -m sed

git diff-tree --no-commit-id --name-only -r 15acf4b8d7cffcd0d74bf1b9c43cde9acaf31ea9..HEAD | xargs sed -i -e "s%^( * Copyright (c) ....)[^[:alpha:]]*(Oracle.*)%\1, 2023, \2%"
git ls-files -m | xargs sed -i -e "s%(Copyright (c) 2023), 2023, (Oracle.*)%\1, \2%"
git add -u; git commit -m copyright

git diff HEAD 2f57a32df8d17da51a04177563327ca2a75e8061


It will give you an easier way to review.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15452#issuecomment-1776817287

From stefank at openjdk.org  Tue Oct 24 09:51:39 2023
From: stefank at openjdk.org (Stefan Karlsson)
Date: Tue, 24 Oct 2023 09:51:39 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v6]
In-Reply-To: <CxPRMRrp5YUzS45-ci5i1hRr3pLNRE_y1QsPH-_NnSM=.0bd7e9ea-f56c-4f52-8bb4-a8aa06501881@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <CxPRMRrp5YUzS45-ci5i1hRr3pLNRE_y1QsPH-_NnSM=.0bd7e9ea-f56c-4f52-8bb4-a8aa06501881@github.com>
Message-ID: <gQyz0Ymz5D09Wuwv4hsUjuWMSHG32Fkp_vhzQtrMAhI=.806fba1c-cc72-452e-b3d2-316b25600e3c@github.com>

On Tue, 24 Oct 2023 07:49:30 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> Rename createJavaProcessBuilder so that it is not used by mistake instead of createTestJvm.
>> 
>> I have used the following sed script: `find -name "*.java" | xargs -n 1 sed -i -e "s/createJavaProcessBuilder(/createJavaProcessBuilderIgnoreTestJavaOpts(/g"`
>> 
>> Then I have manually modified ProcessTools.java. In that file I have moved one version of createJavaProcessBuilder so that it is close to the other version. Then I have added a javadoc comment in bold telling:
>> 
>>    /**
>>      * Create ProcessBuilder using the java launcher from the jdk to
>>      * be tested.
>>      *
>>      * <p><b> Please observe that you likely should use
>>      * createTestJvm() instead of this method because createTestJvm()
>>      * will add JVM options from "test.vm.opts" and "test.java.opts"
>>      * </b> and this method will not do that.
>>      *
>>      * @param command Arguments to pass to the java command.
>>      * @return The ProcessBuilder instance representing the java command.
>>      */
>> 
>> 
>> I have used the name createJavaProcessBuilderIgnoreTestJavaOpts because of the name of Utils.prependTestJavaOpts that adds those VM flags. If you have a better name I could do a rename of the method. I kind of like that it is long and clumsy, that makes it harder to use...
>> 
>> I have run tier 1 testing, and I have started more exhaustive testing.
>
> Leo Korinth has updated the pull request incrementally with six additional commits since the last revision:
> 
>  - static import
>  - copyright
>  - whitespace
>  - whitespace
>  - sed
>  - fix test/lib/jdk/test/lib/process/ProcessTools.java

Marked as reviewed by stefank (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/15452#pullrequestreview-1694437335

From jsjolen at openjdk.org  Tue Oct 24 10:13:33 2023
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Tue, 24 Oct 2023 10:13:33 GMT
Subject: RFR: 8318447: Move NMT source code to own subdirectory [v5]
In-Reply-To: <se-1yZM5MPloQgTUFP811cnvbP_AYMcIzyZR9F1JgSw=.248905ff-92b9-4e97-bd18-d487d385f51a@github.com>
References: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
 <Ym7czX6VGn9CV6Y5Bu-XdyQO1NeJOLICgVW6o1ykSYk=.d44c1adc-03ec-48d7-91dc-61b96366d26d@github.com>
 <se-1yZM5MPloQgTUFP811cnvbP_AYMcIzyZR9F1JgSw=.248905ff-92b9-4e97-bd18-d487d385f51a@github.com>
Message-ID: <EopStJjyldIByWmI0Asrnoa6xxjNR0fwqG-isG5yQJw=.06c91206-940e-4797-8e2c-80df7617e331@github.com>

On Mon, 23 Oct 2023 08:23:44 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Johan Sj?len has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Fix messed up include
>
> src/hotspot/share/nmt/nmtPreInit.hpp line 35:
> 
>> 33: #include "utilities/macros.hpp"
>> 34: 
>> 35: #ifdef ASSERT
> 
> The blank line at 34 is not following the style for our conditional includes. Remove it, or better yet skip conventionalize the include of runtime/atomic.hpp since it just adds to noise to the file.

Removed the blank line, I think the noise is meaningful in this case.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/16276#discussion_r1369924184

From jsjolen at openjdk.org  Tue Oct 24 11:51:45 2023
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Tue, 24 Oct 2023 11:51:45 GMT
Subject: RFR: 8318447: Move NMT source code to own subdirectory [v7]
In-Reply-To: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
References: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
Message-ID: <81yK2Yxh7AVOSjVoAzZwIlriUwHRfN5s5LLowgA-34o=.1ed62ff1-d3d6-4fc1-8e3e-6ca945d86468@github.com>

> I think that NMT is deserving of its own subdirectory. Can we do a review of the changes before I fix the merge conflicts?
> 
> 1. Moved all the nmt source code from services/ to nmt/
> 2. Renamed all the include statements and sorted them
> 3. Fixed the include guards

Johan Sj?len has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits:

 - Merge remote-tracking branch 'upstream/master' into move-nmt
 - Fix stefank suggestions
 - Merge remote-tracking branch 'origin/master' into move-nmt
 - Fix messed up include
 - Missed this include
 - Merge remote-tracking branch 'origin/master' into move-nmt
 - Fixed reviewed changes
 - Move NMT to its own subdirectory

-------------

Changes: https://git.openjdk.org/jdk/pull/16276/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=16276&range=06
  Stats: 508 lines in 102 files changed: 214 ins; 219 del; 75 mod
  Patch: https://git.openjdk.org/jdk/pull/16276.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/16276/head:pull/16276

PR: https://git.openjdk.org/jdk/pull/16276

From jsjolen at openjdk.org  Tue Oct 24 11:51:46 2023
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Tue, 24 Oct 2023 11:51:46 GMT
Subject: RFR: 8318447: Move NMT source code to own subdirectory
In-Reply-To: <mknsPgZrSLBWBnaE4BYz9VzrMWGynmVmfi7OmM48kaY=.1a47a4ba-aca4-4ee8-9318-819936c6a758@github.com>
References: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
 <jtqr_lZBWW0gy8fHQFT0fMRyN_UTmLPEa8zeEKLYwBc=.cc345e4f-4eeb-4951-8edf-ba3b245cc087@github.com>
 <mknsPgZrSLBWBnaE4BYz9VzrMWGynmVmfi7OmM48kaY=.1a47a4ba-aca4-4ee8-9318-819936c6a758@github.com>
Message-ID: <BqzrR7N6clwnwtC4BdA8EiZ_rILNZErhBGdAp3qcPSA=.4235b9e4-d064-4a89-972f-fff430f81f05@github.com>

On Mon, 23 Oct 2023 08:41:56 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

> > For the gtest source files I separated the includes in a consistent manner, they all look like this pattern now:
> 
> That's not what I see in the latest patch. Could you revert that separation and then we can consider that style change in a separate RFE?

Sure, this could be done for all tests (not only NMT).

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16276#issuecomment-1777053794

From wkemper at openjdk.org  Tue Oct 24 15:52:13 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 24 Oct 2023 15:52:13 GMT
Subject: Integrated: 8315560: GenShen: assert failed: Object klass pointer must
 go to metaspace
In-Reply-To: <rdud6-488rNRQNUgtbKE7luTVfFgAhDKUThdDpjP4K0=.fa259ffd-de6a-4fbe-b9b6-26b781ef5711@github.com>
References: <rdud6-488rNRQNUgtbKE7luTVfFgAhDKUThdDpjP4K0=.fa259ffd-de6a-4fbe-b9b6-26b781ef5711@github.com>
Message-ID: <HXEha1vqqXHw0tVeaZcMgM5-11TPdaA9Z_jMJkRD1dU=.7eef080a-9f31-460a-ac0d-43e3ba123130@github.com>

On Mon, 23 Oct 2023 23:41:48 GMT, William Kemper <wkemper at openjdk.org> wrote:

> We have seen crashes caused when the interned string table is cleaned concurrently by mutator actions. Strings which have become unreachable may be inserted in the SATB buffer during concurrent mark of old. If these pointers reside in young regions which are later promoted in place to old regions, the referenced object (which is young and unmarked) will be filled.  When the old marking tries to follow through the pointer, it may crash.

This pull request has now been integrated.

Changeset: 31e5124e
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/31e5124edbceabf69d0999f3eca8a670fccb138d
Stats:     3 lines in 2 files changed: 1 ins; 0 del; 2 mod

8315560: GenShen: assert failed: Object klass pointer must go to metaspace

Reviewed-by: kdnilsen

-------------

PR: https://git.openjdk.org/shenandoah/pull/348

From duke at openjdk.org  Tue Oct 24 16:26:40 2023
From: duke at openjdk.org (duke)
Date: Tue, 24 Oct 2023 16:26:40 GMT
Subject: Withdrawn: JDK-8310111: Shenandoah wastes memory when running with
 very large page sizes
In-Reply-To: <g2vZQuNNbJJoSTc2tE8nZhcLf0muaoZLnkuKZEzHoZQ=.88748998-31db-44bb-a45f-d75e46a4247e@github.com>
References: <g2vZQuNNbJJoSTc2tE8nZhcLf0muaoZLnkuKZEzHoZQ=.88748998-31db-44bb-a45f-d75e46a4247e@github.com>
Message-ID: <6u_CRw9ULZ1L6bh854iwWjfi3aZO1USonxA8JZlLk2Q=.59a6e485-3fc1-47cf-811a-b6bc60049baf@github.com>

On Tue, 20 Jun 2023 12:55:26 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:

> This proposal changes the reservation of bitmaps and region storage to reduce the wastage associated with running with very large page sizes (e.g. 1GB on x64, 512M on arm64) for both non-THP- and THP-mode.
> 
> This patch does:
> - introducing the notion of "allowed overdraft factor" - allocations for a given page size are rejected if they would cause more wastage than the factor allows
> - if it makes sense, it places mark- and aux-bitmap into a contiguous region to let them share a ginourmous page. E.g. for a heap of 16G, both bitmaps would now share a single GB page.
> 
> Examples:
> 
> Note: annoyingly, huge page usage does not show up in RSS. I therefore use a script that parses /proc/pid/smaps and counts hugepage usage to count cost for the following examples:
> 
> Example 1:
> 
> A machine configured with 1G pages (and nothing else). Heap is allocated with 1G pages, the bitmaps fall back to 4K pages because JVM figures 1GB would be wasted:
> 
> 
> thomas at starfish$ ./images/jdk/bin/java -Xmx4600m -Xlog:pagesize -XX:+UseShenandoahGC -XX:+UseLargePages
> ...
> [0.028s][info][pagesize] Mark Bitmap: req_size=160M req_page_size=4K base=0x00007f8149fff000 size=160M page_size=4K
> [0.028s][info][pagesize] Aux Bitmap: req_size=160M req_page_size=4K base=0x00007f813fffe000 size=160M page_size=4K
> [0.028s][info][pagesize] Region Storage: req_size=320K req_page_size=4K base=0x00007f817c06f000 size=320K page_size=4K
> 
> 
> Cost before: 8GB. Cost now: 5GB + (2*160M)
> 
> Example 2: JVM with 14GB heap: mark and aux bitmap together are large enough to justify another 1G page, so they share it. Notice how we also place the region storage on this page:
> 
> 
> thomas at starfish:/shared/projects/openjdk/jdk-jdk/output-release$ ./images/jdk/bin/java -Xmx14g -Xlog:pagesize 
> -XX:+UseShenandoahGC -XX:+UseLargePages -cp $REPROS_JAR de.stuefe.repros.Simple
> [0.003s][info][pagesize] Heap: req_size=14G req_page_size=1G base=0x0000000480000000 size=14G page_size=1G
> [0.003s][info][pagesize] Mark+Aux Bitmap: req_size=896M req_page_size=1G base=0x00007fee00000000 size=1G page_size=1G
> [0.003s][info][pagesize] Region Storage: piggy-backing on Mark Bitmap: base=0x00007fee38000000 size=1792
> <press key>
> 
> 
> Cost before: 17GB. Cost now: 15GB.
> 
> From a bang-for-hugepages-buck multiples of 16GB are a sweet spot here since (on x64 with 1GB pages) since this allows us to put both 512m bitmaps onto a single huge page.
> 
> -----------
> 
> No test yet, since I wanted to see if people reject this proposal. It does add a bit of complexity. If this is well-r...

This pull request has been closed without being integrated.

-------------

PR: https://git.openjdk.org/jdk/pull/14559

From lmesnik at openjdk.org  Tue Oct 24 17:04:41 2023
From: lmesnik at openjdk.org (Leonid Mesnik)
Date: Tue, 24 Oct 2023 17:04:41 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v6]
In-Reply-To: <CxPRMRrp5YUzS45-ci5i1hRr3pLNRE_y1QsPH-_NnSM=.0bd7e9ea-f56c-4f52-8bb4-a8aa06501881@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <CxPRMRrp5YUzS45-ci5i1hRr3pLNRE_y1QsPH-_NnSM=.0bd7e9ea-f56c-4f52-8bb4-a8aa06501881@github.com>
Message-ID: <G2CSM_N-_TwMdAlZX8c-AJtweGPX39TTxuu9HXtGrbc=.1d47edf7-9d08-46ba-9e3b-3549574ebbea@github.com>

On Tue, 24 Oct 2023 07:49:30 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> This pull request renames `createJavaProcessBuilder` to `createLimitedTestJavaProcessBuilder` and renames `createTestJvm` to `createTestJavaProcessBuilder`. Both are implemented through a private `createJavaProcessBuilder`. It also updates the java doc.
>> 
>> This is so that it should be harder to by mistake use `createLimitedTestJavaProcessBuilder` that is problematic because it will not forward JVM flags to the tested JVM.
>
> Leo Korinth has updated the pull request incrementally with six additional commits since the last revision:
> 
>  - static import
>  - copyright
>  - whitespace
>  - whitespace
>  - sed
>  - fix test/lib/jdk/test/lib/process/ProcessTools.java

Marked as reviewed by lmesnik (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/15452#pullrequestreview-1695447658

From rriggs at openjdk.org  Tue Oct 24 19:42:43 2023
From: rriggs at openjdk.org (Roger Riggs)
Date: Tue, 24 Oct 2023 19:42:43 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v6]
In-Reply-To: <CxPRMRrp5YUzS45-ci5i1hRr3pLNRE_y1QsPH-_NnSM=.0bd7e9ea-f56c-4f52-8bb4-a8aa06501881@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <CxPRMRrp5YUzS45-ci5i1hRr3pLNRE_y1QsPH-_NnSM=.0bd7e9ea-f56c-4f52-8bb4-a8aa06501881@github.com>
Message-ID: <6qyfRp98A2DA7Q7YhscmdmokkIvVVn9HxB_XjRdM47g=.615d7a65-e109-41db-b955-31c0a934debd@github.com>

On Tue, 24 Oct 2023 07:49:30 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> This pull request renames `createJavaProcessBuilder` to `createLimitedTestJavaProcessBuilder` and renames `createTestJvm` to `createTestJavaProcessBuilder`. Both are implemented through a private `createJavaProcessBuilder`. It also updates the java doc.
>> 
>> This is so that it should be harder to by mistake use `createLimitedTestJavaProcessBuilder` that is problematic because it will not forward JVM flags to the tested JVM.
>
> Leo Korinth has updated the pull request incrementally with six additional commits since the last revision:
> 
>  - static import
>  - copyright
>  - whitespace
>  - whitespace
>  - sed
>  - fix test/lib/jdk/test/lib/process/ProcessTools.java

test/lib/jdk/test/lib/process/ProcessTools.java line 506:

> 504:      */
> 505:     public static ProcessBuilder createTestJavaProcessBuilder(List<String> command) {
> 506:         return createTestJavaProcessBuilder(command.toArray(String[]::new));

The javadoc shoul d describe all of the options being added to the ProcessBuilder.
They were inadequated described previously and still are.
The other options (seem to be from the code), test.noclasspath, java.class.path, and test.thread.factory.
The description of test.thread.factory and addTestThreadFactoryArgs method seems inadequately described.

test/lib/jdk/test/lib/process/ProcessTools.java line 527:

> 525:      * Create ProcessBuilder using the java launcher from the jdk to
> 526:      * be tested.
> 527:      *

As above, should described the limited options that are added to the ProcessBuilder, the same as for `reateTestJavaProcessBuilder(...)`

test/lib/jdk/test/lib/process/ProcessTools.java line 549:

> 547:      * Create ProcessBuilder using the java launcher from the jdk to
> 548:      * be tested.
> 549:      *

As above, should described the limited options that are added to the ProcessBuilder, the same as for reateTestJavaProcessBuilder(...)

test/lib/jdk/test/lib/process/ProcessTools.java line 599:

> 597:      */
> 598:     public static OutputAnalyzer executeTestJvm(String... cmds) throws Exception {
> 599:         ProcessBuilder pb = createTestJavaProcessBuilder(cmds);

This should also describe *all* of the options being set in the ProcessBuilder before executing the process.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/15452#discussion_r1370728371
PR Review Comment: https://git.openjdk.org/jdk/pull/15452#discussion_r1370729609
PR Review Comment: https://git.openjdk.org/jdk/pull/15452#discussion_r1370729925
PR Review Comment: https://git.openjdk.org/jdk/pull/15452#discussion_r1370730637

From ysr at openjdk.org  Tue Oct 24 20:52:10 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 24 Oct 2023 20:52:10 GMT
Subject: RFR: 8318619: GenShen: Add configurable threshold for young
 triggers to expedite old evacuations
In-Reply-To: <-xp6ofpP2uCkVcHW-RNl-JvuR5XlKGTf1dzT1PYUYOg=.ff3703fe-6500-459f-b137-42147736fd89@github.com>
References: <-xp6ofpP2uCkVcHW-RNl-JvuR5XlKGTf1dzT1PYUYOg=.ff3703fe-6500-459f-b137-42147736fd89@github.com>
Message-ID: <waT9Ck8jflwiujwt9wb0CCmCFkPhLR4rjSDDI510IxY=.cc3ee63e-36a4-4c91-bb26-12647dc3e049@github.com>

On Fri, 20 Oct 2023 20:29:48 GMT, William Kemper <wkemper at openjdk.org> wrote:

> When Shenandoah expects to free up space by promotions into young, or evacuations in old it may trigger a young collection. This change sets configurable thresholds for these conditions.

LGTM, but left some pretty minor comments.

src/hotspot/share/gc/shenandoah/heuristics/shenandoahYoungHeuristics.cpp line 152:

> 150:   ShenandoahOldHeuristics* old_heuristics = heap->old_heuristics();
> 151:   size_t mixed_candidates = old_heuristics->unprocessed_old_collection_candidates();
> 152:   if (mixed_candidates > ShenandoahExpediteMixedThreshold && !heap->is_concurrent_weak_root_in_progress()) {

Might be a good idea to move the counting & test of threshold inside the test for collection state.

src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp line 147:

> 145:           "free up more memory.")                                           \
> 146:                                                                             \
> 147:   product(uintx, ShenandoahExpeditePromotionsThreshold, 5, EXPERIMENTAL,    \

It's a good idea to add a Percent suffix to this.

src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp line 153:

> 151:           range(0,100)                                                      \
> 152:                                                                             \
> 153:   product(uintx, ShenandoahExpediteMixedThreshold, 10, EXPERIMENTAL,        \

Similar to above, may be a good idea to add a Regions suffix to this.

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/347#pullrequestreview-1695848963
PR Review Comment: https://git.openjdk.org/shenandoah/pull/347#discussion_r1370795292
PR Review Comment: https://git.openjdk.org/shenandoah/pull/347#discussion_r1370792577
PR Review Comment: https://git.openjdk.org/shenandoah/pull/347#discussion_r1370793220

From kdnilsen at openjdk.org  Tue Oct 24 23:22:14 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 24 Oct 2023 23:22:14 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v19]
In-Reply-To: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
Message-ID: <XPOBw7sS-l8y4eYa7yUNRUzu8_D_mjq5xqYjHUmOknc=.921b523d-2dfa-4550-916d-2bc05f864cb9@github.com>

> Several improvements:
> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
> 2. Reduce likelihood of humongous allocation errors by:
>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.

Kelvin Nilsen has updated the pull request incrementally with one additional commit since the last revision:

  Prohibit humongous regions from being added to mixed-evac candidates

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/336/files
  - new: https://git.openjdk.org/shenandoah/pull/336/files/b0e0999f..081de8da

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=18
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=17-18

  Stats: 4 lines in 1 file changed: 3 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/336.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/336/head:pull/336

PR: https://git.openjdk.org/shenandoah/pull/336

From kdnilsen at openjdk.org  Tue Oct 24 23:30:41 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 24 Oct 2023 23:30:41 GMT
Subject: RFR: 8317371: GenShen: Needs improved support for humongous
 allocation [v20]
In-Reply-To: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
Message-ID: <LgdslRgmZ3OTOzOJXFvh8hs0l3ZGsmIyBMF2gV4J9JA=.f8acf188-71f0-4014-870f-0d5752df4b52@github.com>

> Several improvements:
> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
> 2. Reduce likelihood of humongous allocation errors by:
>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.

Kelvin Nilsen has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 29 commits:

 - Merge remote-tracking branch 'origin/master' into improve-humongous-allocations
 - Prohibit humongous regions from being added to mixed-evac candidates
 - Improve name of fragmentation_trigger_reason method
 - Merge remote-tracking branch 'origin/master' into improve-humongous-allocations
 - Fix merge conflict
 - Remove redundant option to request dump of memory logs
 - Fix up comments and log message
   
   As prompted by reviewer feedback.
 - Merge remote-tracking branch 'origin/master' into improve-humongous-allocations
 - Allow for no old regions in assertion
 - One more try at this assertion
 - ... and 19 more: https://git.openjdk.org/shenandoah/compare/31e5124e...ac94ed53

-------------

Changes: https://git.openjdk.org/shenandoah/pull/336/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=336&range=19
  Stats: 240 lines in 12 files changed: 186 ins; 7 del; 47 mod
  Patch: https://git.openjdk.org/shenandoah/pull/336.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/336/head:pull/336

PR: https://git.openjdk.org/shenandoah/pull/336

From stefank at openjdk.org  Wed Oct 25 07:27:32 2023
From: stefank at openjdk.org (Stefan Karlsson)
Date: Wed, 25 Oct 2023 07:27:32 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v3]
In-Reply-To: <CZcrmy99xK8GvWF8BD1TmmM_Q8F0po6d3zExg2zNtSY=.9d126ffd-f3ad-4257-a0e4-0a83e3e17a47@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <yXSuvUGhjCND4oyxnP9Aa0LP-t0XFaMebLzPqEZEGpA=.1658ad96-5848-4a9f-ae75-776cb4076705@github.com>
 <5o5B7LbCQN_C9xzd1EvrvTp04-6Atr0gih5WH69LeK4=.3a977034-8fe9-4da8-a167-f5dad3a97d75@github.com>
 <CZcrmy99xK8GvWF8BD1TmmM_Q8F0po6d3zExg2zNtSY=.9d126ffd-f3ad-4257-a0e4-0a83e3e17a47@github.com>
Message-ID: <BYTPUH_b0INyCGVwslaQVyQkTYKNOQvJfnLhKv6mlLM=.bbd8a311-3af8-4288-9491-55c8098af7c8@github.com>

On Tue, 5 Sep 2023 18:05:34 GMT, Roger Riggs <rriggs at openjdk.org> wrote:

>> I have created an alternative that uses enums to force the user to make a decision: https://github.com/openjdk/jdk/compare/master...lkorinth:jdk:+process_tools . Another alternative is to do the same but instead using an enum (I think it is not as good). A third alternative is to use the current pull request with a better name.
>> 
>> What do you prefer? Do you have a better alternative? Do someone still think the current code is good? I think what we have today is inferior to all these improvements, and I would like to make it harder to develop bad test cases.
>
>> What do you prefer? Do you have a better alternative? Do someone still think the current code is good? I think what we have today is inferior to all these improvements, and I would like to make it harder to develop bad test ca
> 
> The current API (name) is fine and fit for purpose; it does not promise or hide extra functionality under a simple name.
> 
> There needs to be an explicit intention in the test(s) to support after the fact that arbitrary flags can be added.
> @AlanBateman's proposal for naming [above](https://github.com/openjdk/jdk/pull/15452#issuecomment-1700459277) (or similar) would capture more clearly that test options are propagated to the child process.
> Every test writer should be aware that additional command line options may be mixed in.
> 
> There are many cases in which the ProcessTools APIs are not used to create child processes and do not need to be used in writing tests. They provide some convenience but also add a dependency and another API layer to work through in the case of failures.
> 
> As far as I'm aware, there is no general guidance or design pattern outside of hotspot tests to propagate flags or use ProcessTools. Adding that as a requirement will need a different level of communication and change.

@RogerRiggs You seem to know what you want w.r.t. the extra java doc comments. Could you help write those? Could we also do that as a separate RFE? I think that would make it easier to get this PR and the javadoc update through the door.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15452#issuecomment-1778669353

From lkorinth at openjdk.org  Wed Oct 25 08:44:29 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Wed, 25 Oct 2023 08:44:29 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v7]
In-Reply-To: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
Message-ID: <fUZIxEzy3YZUqfx87qCu8fVP3jSaRzQKKNMIHjmu9F0=.ae9a3e9c-70a8-42cf-bf81-8e7193c72003@github.com>

> This pull request renames `createJavaProcessBuilder` to `createLimitedTestJavaProcessBuilder` and renames `createTestJvm` to `createTestJavaProcessBuilder`. Both are implemented through a private `createJavaProcessBuilder`. It also updates the java doc.
> 
> This is so that it should be harder to by mistake use `createLimitedTestJavaProcessBuilder` that is problematic because it will not forward JVM flags to the tested JVM.

Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:

  fix copyright year and indentation

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/15452/files
  - new: https://git.openjdk.org/jdk/pull/15452/files/2f57a32d..4cc3865a

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=15452&range=06
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15452&range=05-06

  Stats: 23 lines in 1 file changed: 0 ins; 0 del; 23 mod
  Patch: https://git.openjdk.org/jdk/pull/15452.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15452/head:pull/15452

PR: https://git.openjdk.org/jdk/pull/15452

From kdnilsen at openjdk.org  Wed Oct 25 12:23:08 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Wed, 25 Oct 2023 12:23:08 GMT
Subject: Integrated: 8317371: GenShen: Needs improved support for humongous
 allocation
In-Reply-To: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
References: <7pVbrRyXGcxh8O7nsIxyv4Fbiwv_azUvR0n5d4cWq3M=.d5b57f85-b7d9-4d2d-9156-6328129d3de8@github.com>
Message-ID: <CHmdgRzif7hCVz6qdWNByULjGH-Z-DGUbYik2ISklek=.4ce7e0cf-cac3-49c1-82a8-b90263bc43e4@github.com>

On Sat, 7 Oct 2023 14:11:28 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

> Several improvements:
> 1. If there is a humongous allocation failure, go immediately to Full GC rather than trying 64 degenerated GC first
> 2. Reduce likelihood of humongous allocation errors by:
>     a. Allocating regular objects from top end of memory while allocating humongous from bottom end of memory
>     b. (Especially) in the case that regular regions that are promoted in place reside within the bottom end of memory, detect that old-gen memory has become fragmented and trigger old-gen GC, which will make a special effort to move these out-of-place old-gen regions to higher addresses.

This pull request has now been integrated.

Changeset: c584b481
Author:    Kelvin Nilsen <kdnilsen at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/c584b481e9cd14258f1297cb835815c6a9023ca3
Stats:     240 lines in 12 files changed: 186 ins; 7 del; 47 mod

8317371: GenShen: Needs improved support for humongous allocation

Reviewed-by: wkemper

-------------

PR: https://git.openjdk.org/shenandoah/pull/336

From zgu at openjdk.org  Wed Oct 25 12:51:58 2023
From: zgu at openjdk.org (Zhengyu Gu)
Date: Wed, 25 Oct 2023 12:51:58 GMT
Subject: RFR: 8317466: Enable interpreter oopMapCache for concurrent GCs
 [v3]
In-Reply-To: <dS_92JFN-F9Jizw_KWx1T7EoHQeo6n01CzNV19Xu_iQ=.81a01ca0-0308-4951-9bbf-7bfdd565bdbd@github.com>
References: <dS_92JFN-F9Jizw_KWx1T7EoHQeo6n01CzNV19Xu_iQ=.81a01ca0-0308-4951-9bbf-7bfdd565bdbd@github.com>
Message-ID: <3oEH8zFRDSOC9kcENyJXxj0Gb2rbk2Dimnzr__4d4jE=.ccfde890-e51b-4b1c-83c8-34516307c5cb@github.com>

> Interpreter oop maps are computed lazily during GC root scan and they are expensive to compute.
> 
> GCs uses a small hash table per instance class to cache computed oop maps during STW root scan, but not for concurrent root scan.
> 
> This patch is intended to enable `OopMapCache` for concurrent GCs.
> 
> Test:
>   tier1 and tier2 fastdebug and release on MacOSX, Linux 86_84 and Linux 86_32.

Zhengyu Gu has updated the pull request incrementally with one additional commit since the last revision:

  Refactor ShenandoahOperation to be symmetric to other GCs

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/16074/files
  - new: https://git.openjdk.org/jdk/pull/16074/files/015d4fb3..b63a105e

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=16074&range=02
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16074&range=01-02

  Stats: 6 lines in 2 files changed: 5 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/16074.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/16074/head:pull/16074

PR: https://git.openjdk.org/jdk/pull/16074

From jsjolen at openjdk.org  Wed Oct 25 13:21:40 2023
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Wed, 25 Oct 2023 13:21:40 GMT
Subject: RFR: 8318447: Move NMT source code to own subdirectory
In-Reply-To: <mknsPgZrSLBWBnaE4BYz9VzrMWGynmVmfi7OmM48kaY=.1a47a4ba-aca4-4ee8-9318-819936c6a758@github.com>
References: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
 <jtqr_lZBWW0gy8fHQFT0fMRyN_UTmLPEa8zeEKLYwBc=.cc345e4f-4eeb-4951-8edf-ba3b245cc087@github.com>
 <mknsPgZrSLBWBnaE4BYz9VzrMWGynmVmfi7OmM48kaY=.1a47a4ba-aca4-4ee8-9318-819936c6a758@github.com>
Message-ID: <m_aauUt2VHDnCGVUixVU2CiJJZqdaRTENl0tkL9AGvc=.106b1ac3-0c53-46e6-ba5d-eb7e384c928b@github.com>

On Mon, 23 Oct 2023 08:41:56 GMT, Stefan Karlsson <stefank at openjdk.org> wrote:

>> Hi,
>> 
>> Thank you for looking through these changes. I applied your comments and also did a run through to look for incorrectly ordered includes. For the gtest source files I separated the includes in a consistent manner, they all look like this pattern now:
>> 
>> ```c++
>> #include "precompiled.hpp"
>> #include "memory/allocation.hpp"
>> #include "nmt/mallocHeader.inline.hpp"
>> #include "nmt/memTracker.hpp"
>> #include "runtime/os.hpp"
>> 
>> #include "testutils.hpp"
>> #include "unittest.hpp"
>
>> For the gtest source files I separated the includes in a consistent manner, they all look like this pattern now:
> 
> That's not what I see in the latest patch. Could you revert that separation and then we can consider that style change in a separate RFE?

@stefank, I believe that I've applied your suggestions, would you mind having another look? Thanks

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16276#issuecomment-1779260132

From stefank at openjdk.org  Wed Oct 25 16:58:42 2023
From: stefank at openjdk.org (Stefan Karlsson)
Date: Wed, 25 Oct 2023 16:58:42 GMT
Subject: RFR: 8318447: Move NMT source code to own subdirectory [v7]
In-Reply-To: <81yK2Yxh7AVOSjVoAzZwIlriUwHRfN5s5LLowgA-34o=.1ed62ff1-d3d6-4fc1-8e3e-6ca945d86468@github.com>
References: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
 <81yK2Yxh7AVOSjVoAzZwIlriUwHRfN5s5LLowgA-34o=.1ed62ff1-d3d6-4fc1-8e3e-6ca945d86468@github.com>
Message-ID: <xVJWCObhNZGe9irLr03Xn6zfDu052eRVviWEm_RoVNk=.623cc145-cadd-412a-a0d8-a68005b46e47@github.com>

On Tue, 24 Oct 2023 11:51:45 GMT, Johan Sj?len <jsjolen at openjdk.org> wrote:

>> I think that NMT is deserving of its own subdirectory. Can we do a review of the changes before I fix the merge conflicts?
>> 
>> 1. Moved all the nmt source code from services/ to nmt/
>> 2. Renamed all the include statements and sorted them
>> 3. Fixed the include guards
>
> Johan Sj?len has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits:
> 
>  - Merge remote-tracking branch 'upstream/master' into move-nmt
>  - Fix stefank suggestions
>  - Merge remote-tracking branch 'origin/master' into move-nmt
>  - Fix messed up include
>  - Missed this include
>  - Merge remote-tracking branch 'origin/master' into move-nmt
>  - Fixed reviewed changes
>  - Move NMT to its own subdirectory

I've approved the patch, but I think you should revert the two gtest changes I mention below. I have given suggestions for two cleanups that you might want to do.

src/hotspot/share/memory/resourceArea.inline.hpp line 30:

> 28: #include "memory/resourceArea.hpp"
> 29: 
> 30: #include "nmt/memTracker.hpp"

Another new line that should not be here. Bonus points if it gets removed.

src/hotspot/share/nmt/mallocHeader.cpp line 29:

> 27: #include "nmt/mallocHeader.inline.hpp"
> 28: 
> 29: #include "nmt/mallocSiteTable.hpp"

There should be no newline between these two includes. If you make another round of changes I think it would be good to get rid of it.

test/hotspot/gtest/nmt/test_nmt_buffer_overflow_detection.cpp line 33:

> 31: #include "testutils.hpp"
> 32: #include "unittest.hpp"
> 33: 

It is inclear to me if there is an ordering requirement between testutils and unittest. I'd prefer if you didn't change those unit test header files in this patch and make these cleanups separately.

test/hotspot/gtest/nmt/test_nmt_locationprinting.cpp line 35:

> 33: // Uncomment to get test output
> 34: //#define LOG_PLEASE
> 35: 

There seems to be an ordering requirement between LOG_PLEASE and the inclusion of LOG_PLEASE. I think you break it with this change.

-------------

Marked as reviewed by stefank (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/16276#pullrequestreview-1697824750
PR Review Comment: https://git.openjdk.org/jdk/pull/16276#discussion_r1372058081
PR Review Comment: https://git.openjdk.org/jdk/pull/16276#discussion_r1372057335
PR Review Comment: https://git.openjdk.org/jdk/pull/16276#discussion_r1372053129
PR Review Comment: https://git.openjdk.org/jdk/pull/16276#discussion_r1372051716

From rriggs at openjdk.org  Wed Oct 25 22:00:31 2023
From: rriggs at openjdk.org (Roger Riggs)
Date: Wed, 25 Oct 2023 22:00:31 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v7]
In-Reply-To: <fUZIxEzy3YZUqfx87qCu8fVP3jSaRzQKKNMIHjmu9F0=.ae9a3e9c-70a8-42cf-bf81-8e7193c72003@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <fUZIxEzy3YZUqfx87qCu8fVP3jSaRzQKKNMIHjmu9F0=.ae9a3e9c-70a8-42cf-bf81-8e7193c72003@github.com>
Message-ID: <qyv3_2FCcacwbDL17B7JewcsR5XazNJjio3Z_WNTq_U=.88ec5e8f-7929-443c-a85f-617260b584e4@github.com>

On Wed, 25 Oct 2023 08:44:29 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> This pull request renames `createJavaProcessBuilder` to `createLimitedTestJavaProcessBuilder` and renames `createTestJvm` to `createTestJavaProcessBuilder`. Both are implemented through a private `createJavaProcessBuilder`. It also updates the java doc.
>> 
>> This is so that it should be harder to by mistake use `createLimitedTestJavaProcessBuilder` that is problematic because it will not forward JVM flags to the tested JVM.
>
> Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix copyright year and indentation

Suggestions to complete the descriptions of the createXXXJavaProcessBuilder methods.

test/lib/jdk/test/lib/process/ProcessTools.java line 505:

> 503:      * @return The ProcessBuilder instance representing the java command.
> 504:      */
> 505:     public static ProcessBuilder createTestJavaProcessBuilder(List<String> command) {

Include the same description of other properties that are included in creating the ProcessBuilder...
```     * Unless the "test.noclasspath" property is "true"
     * the classpath property "java.class.path" is appended to the command line and
     * the environment of the ProcessBuilder is modified to remove "CLASSPATH".
     * If the property "test.thread.factory" is provided the command args are
     * updated and appended to invoke ProcessTools main() and provide the 
     * name of the thread factory.
     * The "-Dtest.thread.factory" is appended to the arguments with the thread factory value.
     * The remaining command args are scanned for unsupported options and 
     * are appended to the ProcessBuilder.

test/lib/jdk/test/lib/process/ProcessTools.java line 520:

> 518:      * @return The ProcessBuilder instance representing the java command.
> 519:      */
> 520:     public static ProcessBuilder createTestJavaProcessBuilder(String... command) {

Include the same description of other properties that are included in creating the ProcessBuilder...

     * Unless the "test.noclasspath" property is "true"
     * the classpath property "java.class.path" is appended to the command line and
     * the environment of the ProcessBuilder is modified to remove "CLASSPATH".
     * If the property "test.thread.factory" is provided the command args are
     * updated and appended to invoke ProcessTools main() and provide the 
     * name of the thread factory.
     * The "-Dtest.thread.factory" is appended to the arguments with the thread factory value.
     * The remaining command args are scanned for unsupported options and 
     * are appended to the ProcessBuilder.

test/lib/jdk/test/lib/process/ProcessTools.java line 538:

> 536:      * it in combination with <b>@requires vm.flagless</b> JTREG
> 537:      * anotation as to not waste energy and test resources.
> 538:      *

Consider adding this description of what this method does.
Suggestion:

     * Unless the "test.noclasspath" property is "true"
     * the classpath property "java.class.path" is appended to the command line and
     * the environment of the ProcessBuilder is modified to remove "CLASSPATH".
     * If the property "test.thread.factory" is provided the command args are
     * updated and appended to invoke ProcessTools main() and provide the 
     * name of the thread factory.
     * The "-Dtest.thread.factory" is appended to the arguments with the thread factory value.
     * The remaining command args are scanned for unsupported options and 
     * are appended to the ProcessBuilder.

test/lib/jdk/test/lib/process/ProcessTools.java line 560:

> 558:      * it in combination with <b>@requires vm.flagless</b> JTREG
> 559:      * anotation as to not waste energy and test resources.
> 560:      *

Suggestion:

     * Unless the "test.noclasspath" property is "true"
     * the classpath property "java.class.path" is appended to the command line and
     * the environment of the ProcessBuilder is modified to remove "CLASSPATH".
     * If the property "test.thread.factory" is provided the command args are
     * updated and appended to invoke ProcessTools main() and provide the 
     * name of the thread factory.
     * The "-Dtest.thread.factory" is appended to the arguments with the thread factory value.
     * The remaining command args are scanned for unsupported options and 
     * are appended to the ProcessBuilder.

-------------

PR Review: https://git.openjdk.org/jdk/pull/15452#pullrequestreview-1698308785
PR Review Comment: https://git.openjdk.org/jdk/pull/15452#discussion_r1372364800
PR Review Comment: https://git.openjdk.org/jdk/pull/15452#discussion_r1372364171
PR Review Comment: https://git.openjdk.org/jdk/pull/15452#discussion_r1372361862
PR Review Comment: https://git.openjdk.org/jdk/pull/15452#discussion_r1372362333

From jsjolen at openjdk.org  Thu Oct 26 09:51:16 2023
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Thu, 26 Oct 2023 09:51:16 GMT
Subject: RFR: 8318447: Move NMT source code to own subdirectory [v7]
In-Reply-To: <81yK2Yxh7AVOSjVoAzZwIlriUwHRfN5s5LLowgA-34o=.1ed62ff1-d3d6-4fc1-8e3e-6ca945d86468@github.com>
References: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
 <81yK2Yxh7AVOSjVoAzZwIlriUwHRfN5s5LLowgA-34o=.1ed62ff1-d3d6-4fc1-8e3e-6ca945d86468@github.com>
Message-ID: <NxT0imZp1ECoJ0nqzPGeu-QuxDmPVs5_NKjwnj7Xr64=.8ed32fe7-fdbc-427e-b32f-d4739bc4aec7@github.com>

On Tue, 24 Oct 2023 11:51:45 GMT, Johan Sj?len <jsjolen at openjdk.org> wrote:

>> I think that NMT is deserving of its own subdirectory. Can we do a review of the changes before I fix the merge conflicts?
>> 
>> 1. Moved all the nmt source code from services/ to nmt/
>> 2. Renamed all the include statements and sorted them
>> 3. Fixed the include guards
>
> Johan Sj?len has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains eight commits:
> 
>  - Merge remote-tracking branch 'upstream/master' into move-nmt
>  - Fix stefank suggestions
>  - Merge remote-tracking branch 'origin/master' into move-nmt
>  - Fix messed up include
>  - Missed this include
>  - Merge remote-tracking branch 'origin/master' into move-nmt
>  - Fixed reviewed changes
>  - Move NMT to its own subdirectory

Thanks Stefan, good catch on the ordering requirements. I'll integrate this ASAP.

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16276#issuecomment-1780775333

From jsjolen at openjdk.org  Thu Oct 26 09:51:15 2023
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Thu, 26 Oct 2023 09:51:15 GMT
Subject: RFR: 8318447: Move NMT source code to own subdirectory [v8]
In-Reply-To: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
References: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
Message-ID: <PiFHnbl6QEzTZV5nWpc-UAXOQ_SJs_If_iIyWwpnnps=.e07411fa-4836-424d-b345-5e34f06be086@github.com>

> I think that NMT is deserving of its own subdirectory. Can we do a review of the changes before I fix the merge conflicts?
> 
> 1. Moved all the nmt source code from services/ to nmt/
> 2. Renamed all the include statements and sorted them
> 3. Fixed the include guards

Johan Sj?len has updated the pull request incrementally with one additional commit since the last revision:

  Stefan's suggestions

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/16276/files
  - new: https://git.openjdk.org/jdk/pull/16276/files/70b39e41..bb72984b

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=16276&range=07
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=16276&range=06-07

  Stats: 6 lines in 4 files changed: 2 ins; 4 del; 0 mod
  Patch: https://git.openjdk.org/jdk/pull/16276.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/16276/head:pull/16276

PR: https://git.openjdk.org/jdk/pull/16276

From stefank at openjdk.org  Thu Oct 26 10:03:34 2023
From: stefank at openjdk.org (Stefan Karlsson)
Date: Thu, 26 Oct 2023 10:03:34 GMT
Subject: RFR: 8318447: Move NMT source code to own subdirectory [v8]
In-Reply-To: <PiFHnbl6QEzTZV5nWpc-UAXOQ_SJs_If_iIyWwpnnps=.e07411fa-4836-424d-b345-5e34f06be086@github.com>
References: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
 <PiFHnbl6QEzTZV5nWpc-UAXOQ_SJs_If_iIyWwpnnps=.e07411fa-4836-424d-b345-5e34f06be086@github.com>
Message-ID: <g-Qq0q2UkTGrsBZcg2SmlAMCyA9yTpO3k2CQZhwrhug=.e316112f-d111-4b6f-b385-ed51df51b590@github.com>

On Thu, 26 Oct 2023 09:51:15 GMT, Johan Sj?len <jsjolen at openjdk.org> wrote:

>> I think that NMT is deserving of its own subdirectory. Can we do a review of the changes before I fix the merge conflicts?
>> 
>> 1. Moved all the nmt source code from services/ to nmt/
>> 2. Renamed all the include statements and sorted them
>> 3. Fixed the include guards
>
> Johan Sj?len has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Stefan's suggestions

Marked as reviewed by stefank (Reviewer).

-------------

PR Review: https://git.openjdk.org/jdk/pull/16276#pullrequestreview-1699205834

From rkennke at openjdk.org  Thu Oct 26 10:36:30 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 26 Oct 2023 10:36:30 GMT
Subject: RFR: 8318462: [GenShen] Prevent unsafe access to displaced
 mark-word [v3]
In-Reply-To: <uOeuEpzRFvBx1AFgOvvTNfxJIyI4o84yT9D5A_q6BjU=.92103c05-893a-4b0b-a165-54289729f74f@github.com>
References: <uOeuEpzRFvBx1AFgOvvTNfxJIyI4o84yT9D5A_q6BjU=.92103c05-893a-4b0b-a165-54289729f74f@github.com>
Message-ID: <G4dtAHgyXda6zZiq16qHiyQWWCapLgkoW0dASfjLIF8=.14abe76d-7980-4227-9698-35ded44fc141@github.com>

> We must be very careful when we access (load or update) the object age concurrently because the age is stored in the object's header and that header can be 'displaced'. This means that the header is overloaded with a pointer (stack or ObjectMonitor*) and the original header is stored at that location. The header can also be in INFLATING state, which means that a stack-lock is currently in the process to be inflated to an ObjectMonitor. Let's consider the cases separately:
> 
> Loading object age:
> - INFLATING: we can not access the header, and thus not the age.
> - Stack-locked: the thread which locks the object can unlock at any time, concurrently. It is not safe to access the header, and thus the age.
> - Monitor-locked: monitors don't go away without coordinating with Java threads and possibly GC threads. This coordination is done by handshaking the thread - a Java thread would be brought to its next safepoint, and GC threads, which are typically participating in handshakes by joining the SuspendibleThreadSet, can opt to handshake at a safe point by calling SuspendibleThreadSet::yield(). If 'our' thread participates in handshakes in one way or the other, it is safe to access an ObjectMonitor once we got a valid ObjectMonitor* from an object's header, but only until that thread handshakes. Long story short: it is typically safe to access ObjectMonitor :-)
> 
> Updating object age:
> Updating object age only happens during evacuation, and only on a new copy of an object, before that copy has been published. Access to the object header is therefore fully thread-local and not problematic. What *is* problematic is when we have to deal with displaced headers, because displaced headers are *not* thread local, and must be considered a shared resource. A competing thread may succeed to evacuate the object and publish its copy before 'our' thread can do so. If 'our' thread would update the displaced header, it may over-write whatever the other thread has done.
> - INFLATING: We cannot access the header at all. However, that should not happen: a thread can only inflate once it has successfully evacuated an object, and 
> - Stack-locked: the other thread may succeed evacuation and continue to unlock the object, at which point the stack-pointer would be 'dangling' and we may over-write random information on the foreign thread stack.
> - Monitor-locked: even though monitor deflation is coordinated (see above), updating the displaced header in a monitor might over-write whatever a competi...

Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:

 - Fix inflating check with LW locking
 - Merge branch 'master' into JDK-8318462
 - Accept all-zero header with LW locking
 - Handle forwarded headers
 - Assert that no inflation happens
 - 8318462: [Shenandoah] Prevent unsafe access to displaced mark-word

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/343/files
  - new: https://git.openjdk.org/shenandoah/pull/343/files/dd6329a3..31ef2d9c

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=343&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=343&range=01-02

  Stats: 384 lines in 22 files changed: 276 ins; 10 del; 98 mod
  Patch: https://git.openjdk.org/shenandoah/pull/343.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/343/head:pull/343

PR: https://git.openjdk.org/shenandoah/pull/343

From jsjolen at openjdk.org  Thu Oct 26 14:06:48 2023
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Thu, 26 Oct 2023 14:06:48 GMT
Subject: RFR: 8318447: Move NMT source code to own subdirectory [v8]
In-Reply-To: <PiFHnbl6QEzTZV5nWpc-UAXOQ_SJs_If_iIyWwpnnps=.e07411fa-4836-424d-b345-5e34f06be086@github.com>
References: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
 <PiFHnbl6QEzTZV5nWpc-UAXOQ_SJs_If_iIyWwpnnps=.e07411fa-4836-424d-b345-5e34f06be086@github.com>
Message-ID: <7AjGEXTYBkbRhxV8OiYMhbo-hCALbZI_Oa2ee9Pkk34=.035bcce9-eaec-4a65-a520-758b3723a598@github.com>

On Thu, 26 Oct 2023 09:51:15 GMT, Johan Sj?len <jsjolen at openjdk.org> wrote:

>> I think that NMT is deserving of its own subdirectory. Can we do a review of the changes before I fix the merge conflicts?
>> 
>> 1. Moved all the nmt source code from services/ to nmt/
>> 2. Renamed all the include statements and sorted them
>> 3. Fixed the include guards
>
> Johan Sj?len has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Stefan's suggestions

Thank you for the reviews!

-------------

PR Comment: https://git.openjdk.org/jdk/pull/16276#issuecomment-1781194359

From jsjolen at openjdk.org  Thu Oct 26 14:06:49 2023
From: jsjolen at openjdk.org (Johan =?UTF-8?B?U2rDtmxlbg==?=)
Date: Thu, 26 Oct 2023 14:06:49 GMT
Subject: Integrated: 8318447: Move NMT source code to own subdirectory
In-Reply-To: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
References: <VYNZ-6UA9nV6FKnjAT_5Z36_EGaZH9CJuKfQM42aJM4=.43a2ad07-1a6e-4d3b-bb51-0f80c16a0776@github.com>
Message-ID: <1VfmMoDMZl4JslQojSgYubYs8wLQJpEhiJlmaJTTzvA=.ef7284a5-49e3-40fb-b2e1-0e3fa8ae5a98@github.com>

On Thu, 19 Oct 2023 20:06:50 GMT, Johan Sj?len <jsjolen at openjdk.org> wrote:

> I think that NMT is deserving of its own subdirectory. Can we do a review of the changes before I fix the merge conflicts?
> 
> 1. Moved all the nmt source code from services/ to nmt/
> 2. Renamed all the include statements and sorted them
> 3. Fixed the include guards

This pull request has now been integrated.

Changeset: 9864951d
Author:    Johan Sj?len <jsjolen at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/9864951dceb0ddc4479ced04b6d5a2363f1e307d
Stats:     506 lines in 102 files changed: 212 ins; 219 del; 75 mod

8318447: Move NMT source code to own subdirectory

Reviewed-by: stefank, dholmes, stuefe

-------------

PR: https://git.openjdk.org/jdk/pull/16276

From wkemper at openjdk.org  Thu Oct 26 14:17:27 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 26 Oct 2023 14:17:27 GMT
Subject: RFR: Merge openjdk/jdk:master
Message-ID: <wdsao1bTO2smDL_gzJaICcwf4IJ-uqae_i3tC8pUwfM=.3ce36561-358a-468f-aed6-ba07b237a648@github.com>

Merges tag jdk-22+21

-------------

Commit messages:
 - 8317510: Change Windows debug symbol files naming to avoid losing info when an executable and a library share the same name
 - 8318613: ChoiceFormat patterns are not well tested
 - 8318186: ChoiceFormat inconsistency between applyPattern() and setChoices()
 - 8318487: Specification of the ListFormat.equals() method can be improved
 - 8317360: Missing null checks in JfrCheckpointManager and JfrStringPool initialization routines
 - 8318735: RISC-V: Enable related hotspot tests run on riscv
 - 8318727: Enable parallelism in vmTestbase/vm/gc/concurrent tests
 - 8318607: Enable parallelism in vmTestbase/nsk/stress/jni tests
 - 8318608: Enable parallelism in vmTestbase/nsk/stress/threads tests
 - 8312980: C2: "malformed control flow" created during incremental inlining
 - ... and 241 more: https://git.openjdk.org/shenandoah/compare/fb4098ff...d96f38b8

The webrev contains the conflicts with master:
 - merge conflicts: https://webrevs.openjdk.org/?repo=shenandoah&pr=349&range=00.conflicts

Changes: https://git.openjdk.org/shenandoah/pull/349/files
  Stats: 31003 lines in 1082 files changed: 19698 ins; 6224 del; 5081 mod
  Patch: https://git.openjdk.org/shenandoah/pull/349.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/349/head:pull/349

PR: https://git.openjdk.org/shenandoah/pull/349

From lkorinth at openjdk.org  Thu Oct 26 15:33:38 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Thu, 26 Oct 2023 15:33:38 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v7]
In-Reply-To: <fUZIxEzy3YZUqfx87qCu8fVP3jSaRzQKKNMIHjmu9F0=.ae9a3e9c-70a8-42cf-bf81-8e7193c72003@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <fUZIxEzy3YZUqfx87qCu8fVP3jSaRzQKKNMIHjmu9F0=.ae9a3e9c-70a8-42cf-bf81-8e7193c72003@github.com>
Message-ID: <CZyt2tGATgYnxwwXcRUx-f2W4UTwCRouztmSTtinrTw=.6dc482ad-77ad-4d75-94ab-0ae273d0a2a8@github.com>

On Wed, 25 Oct 2023 08:44:29 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> This pull request renames `createJavaProcessBuilder` to `createLimitedTestJavaProcessBuilder` and renames `createTestJvm` to `createTestJavaProcessBuilder`. Both are implemented through a private `createJavaProcessBuilder`. It also updates the java doc.
>> 
>> This is so that it should be harder to by mistake use `createLimitedTestJavaProcessBuilder` that is problematic because it will not forward JVM flags to the tested JVM.
>
> Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix copyright year and indentation

Would it be okay if we handle the new method documentation in a separate pull request? 

After applying your changes, I also noted that the existing description `The command line will be like: {test.jdk}/bin/java {test.vm.opts} {test.java.opts} cmds` is not only incorrect (or at least incomplete), but now also clashes with the added description. I then removed the sentence, but after I did that I also found out that similar wording exist in `executeTestJvm` and I fear that if I continue to pull strings, I will create more and more changes that you will have opinions on.

Is it all right if we push what we have now, and that I create a new pull requests with these improvements in documentation that are actually not related to the changes in this pull request?

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15452#issuecomment-1781359450

From kdnilsen at openjdk.org  Thu Oct 26 16:28:35 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 26 Oct 2023 16:28:35 GMT
Subject: RFR: 8318462: [GenShen] Prevent unsafe access to displaced
 mark-word [v3]
In-Reply-To: <G4dtAHgyXda6zZiq16qHiyQWWCapLgkoW0dASfjLIF8=.14abe76d-7980-4227-9698-35ded44fc141@github.com>
References: <uOeuEpzRFvBx1AFgOvvTNfxJIyI4o84yT9D5A_q6BjU=.92103c05-893a-4b0b-a165-54289729f74f@github.com>
 <G4dtAHgyXda6zZiq16qHiyQWWCapLgkoW0dASfjLIF8=.14abe76d-7980-4227-9698-35ded44fc141@github.com>
Message-ID: <S7Jx2poxtFfB7voH5TUPgzc40hy4jIPQGhHF6kW8VEM=.40dd23a1-78a4-4534-8a86-aa1d99ef7347@github.com>

On Thu, 26 Oct 2023 10:36:30 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> We must be very careful when we access (load or update) the object age concurrently because the age is stored in the object's header and that header can be 'displaced'. This means that the header is overloaded with a pointer (stack or ObjectMonitor*) and the original header is stored at that location. The header can also be in INFLATING state, which means that a stack-lock is currently in the process to be inflated to an ObjectMonitor. Let's consider the cases separately:
>> 
>> Loading object age:
>> - INFLATING: we can not access the header, and thus not the age.
>> - Stack-locked: the thread which locks the object can unlock at any time, concurrently. It is not safe to access the header, and thus the age.
>> - Monitor-locked: monitors don't go away without coordinating with Java threads and possibly GC threads. This coordination is done by handshaking the thread - a Java thread would be brought to its next safepoint, and GC threads, which are typically participating in handshakes by joining the SuspendibleThreadSet, can opt to handshake at a safe point by calling SuspendibleThreadSet::yield(). If 'our' thread participates in handshakes in one way or the other, it is safe to access an ObjectMonitor once we got a valid ObjectMonitor* from an object's header, but only until that thread handshakes. Long story short: it is typically safe to access ObjectMonitor :-)
>> 
>> Updating object age:
>> Updating object age only happens during evacuation, and only on a new copy of an object, before that copy has been published. Access to the object header is therefore fully thread-local and not problematic. What *is* problematic is when we have to deal with displaced headers, because displaced headers are *not* thread local, and must be considered a shared resource. A competing thread may succeed to evacuate the object and publish its copy before 'our' thread can do so. If 'our' thread would update the displaced header, it may over-write whatever the other thread has done.
>> - INFLATING: We cannot access the header at all. However, that should not happen: a thread can only inflate once it has successfully evacuated an object, and 
>> - Stack-locked: the other thread may succeed evacuation and continue to unlock the object, at which point the stack-pointer would be 'dangling' and we may over-write random information on the foreign thread stack.
>> - Monitor-locked: even though monitor deflation is coordinated (see above), updating the displaced header in a monitor might over-...
>
> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
> 
>  - Fix inflating check with LW locking
>  - Merge branch 'master' into JDK-8318462
>  - Accept all-zero header with LW locking
>  - Handle forwarded headers
>  - Assert that no inflation happens
>  - 8318462: [Shenandoah] Prevent unsafe access to displaced mark-word

src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 544:

> 542:   // is thread-local and therefore safe to access. However, when the mark is
> 543:   // displaced (i.e. stack-locked or monitor-locked), then it must be considered
> 544:   //a shared memory location. It can/ be accessed by other threads.

Minor formatting issues in this line of comment.

src/hotspot/share/gc/shenandoah/shenandoahMark.inline.hpp line 123:

> 121:       assert(region->is_young(), "Only for young objects");
> 122:       uint age = ShenandoahHeap::get_object_age(obj);
> 123:       CENSUS_NOISE(heap->age_census()->add(age, region->age(), region->youth(), size, worker_id);)

I'm thinking if get_object_age() returns max_age+1, the CENSUS_NOISE macros might end up with invalid data.  Don't we need a check here?

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/343#discussion_r1373409115
PR Review Comment: https://git.openjdk.org/shenandoah/pull/343#discussion_r1373435503

From kdnilsen at openjdk.org  Thu Oct 26 16:28:35 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Thu, 26 Oct 2023 16:28:35 GMT
Subject: RFR: 8318462: [GenShen] Prevent unsafe access to displaced
 mark-word [v3]
In-Reply-To: <S7Jx2poxtFfB7voH5TUPgzc40hy4jIPQGhHF6kW8VEM=.40dd23a1-78a4-4534-8a86-aa1d99ef7347@github.com>
References: <uOeuEpzRFvBx1AFgOvvTNfxJIyI4o84yT9D5A_q6BjU=.92103c05-893a-4b0b-a165-54289729f74f@github.com>
 <G4dtAHgyXda6zZiq16qHiyQWWCapLgkoW0dASfjLIF8=.14abe76d-7980-4227-9698-35ded44fc141@github.com>
 <S7Jx2poxtFfB7voH5TUPgzc40hy4jIPQGhHF6kW8VEM=.40dd23a1-78a4-4534-8a86-aa1d99ef7347@github.com>
Message-ID: <62iAP1xppfg6WDXm-ZdoT8cugt1L-2athpTtFc145M8=.9e763172-f272-4675-93a7-3733d63b335b@github.com>

On Thu, 26 Oct 2023 16:03:12 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> Roman Kennke has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision:
>> 
>>  - Fix inflating check with LW locking
>>  - Merge branch 'master' into JDK-8318462
>>  - Accept all-zero header with LW locking
>>  - Handle forwarded headers
>>  - Assert that no inflation happens
>>  - 8318462: [Shenandoah] Prevent unsafe access to displaced mark-word
>
> src/hotspot/share/gc/shenandoah/shenandoahHeap.inline.hpp line 544:
> 
>> 542:   // is thread-local and therefore safe to access. However, when the mark is
>> 543:   // displaced (i.e. stack-locked or monitor-locked), then it must be considered
>> 544:   //a shared memory location. It can/ be accessed by other threads.
> 
> Minor formatting issues in this line of comment.

and below.

> src/hotspot/share/gc/shenandoah/shenandoahMark.inline.hpp line 123:
> 
>> 121:       assert(region->is_young(), "Only for young objects");
>> 122:       uint age = ShenandoahHeap::get_object_age(obj);
>> 123:       CENSUS_NOISE(heap->age_census()->add(age, region->age(), region->youth(), size, worker_id);)
> 
> I'm thinking if get_object_age() returns max_age+1, the CENSUS_NOISE macros might end up with invalid data.  Don't we need a check here?

I think the safe thing to do is treat max_age+1 as if it were 0, insofar as promotion semantics is concerned.  We may delay promotion that way.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/343#discussion_r1373431845
PR Review Comment: https://git.openjdk.org/shenandoah/pull/343#discussion_r1373436928

From rriggs at openjdk.org  Thu Oct 26 16:29:40 2023
From: rriggs at openjdk.org (Roger Riggs)
Date: Thu, 26 Oct 2023 16:29:40 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v7]
In-Reply-To: <fUZIxEzy3YZUqfx87qCu8fVP3jSaRzQKKNMIHjmu9F0=.ae9a3e9c-70a8-42cf-bf81-8e7193c72003@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <fUZIxEzy3YZUqfx87qCu8fVP3jSaRzQKKNMIHjmu9F0=.ae9a3e9c-70a8-42cf-bf81-8e7193c72003@github.com>
Message-ID: <Yx-2Zuy3fBrCV9jiHIiGJSLOBBy7Hv9BVRBeB1ytvLk=.e3026827-3141-4ee4-b8ea-2d459d42eabf@github.com>

On Wed, 25 Oct 2023 08:44:29 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> This pull request renames `createJavaProcessBuilder` to `createLimitedTestJavaProcessBuilder` and renames `createTestJvm` to `createTestJavaProcessBuilder`. Both are implemented through a private `createJavaProcessBuilder`. It also updates the java doc.
>> 
>> This is so that it should be harder to by mistake use `createLimitedTestJavaProcessBuilder` that is problematic because it will not forward JVM flags to the tested JVM.
>
> Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix copyright year and indentation

ok, to correct the javadoc in a subsequent PR.

-------------

Marked as reviewed by rriggs (Reviewer).

PR Review: https://git.openjdk.org/jdk/pull/15452#pullrequestreview-1700061972

From wkemper at openjdk.org  Thu Oct 26 18:28:17 2023
From: wkemper at openjdk.org (William Kemper)
Date: Thu, 26 Oct 2023 18:28:17 GMT
Subject: RFR: 8316632: Shenandoah: Raise OOME when gc threshold is exceeded
 [v7]
In-Reply-To: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
Message-ID: <_VKFq1r7AzgsP99bp0MtA2IbF1RmggmqLE1LlknyDBY=.e6886387-1dc5-4907-aa3b-9c1210ea85d6@github.com>

> Shenandoah will run back-to-back full GCs and _almost_ grind mutator progress to a halt before eventually exhausting memory. This change will have Shenandoah raise a gc threshold exceeded exception if the collector fails to make progress after `ShenandoahNoProgressThreshold` full GC cycles (default is 3).

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 16 additional commits since the last revision:

 - Merge remote-tracking branch 'openjdk/master' into shenandoah-oome-redux
 - Remove unnecessary nesting and other differences
 - Update formatting
 - Make explicit field for Shenandoah
 - Restore original retry logic, pull gc overhead check back over retry
 - Merge branch 'openjdk-master' into shenandoah-oome-redux
 - Merge check for no-progress into retry allocation block
 - Revert change to TEST.groups
 - Merge remote-tracking branch 'openjdk/master' into shenandoah-oome-redux
 - Extend exemption for EATests that rely on timely OOME to Shenandoah
 - ... and 6 more: https://git.openjdk.org/jdk/compare/1787b19e...95f3379a

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/15852/files
  - new: https://git.openjdk.org/jdk/pull/15852/files/4154370c..95f3379a

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=15852&range=06
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=15852&range=05-06

  Stats: 17285 lines in 730 files changed: 10879 ins; 3673 del; 2733 mod
  Patch: https://git.openjdk.org/jdk/pull/15852.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/15852/head:pull/15852

PR: https://git.openjdk.org/jdk/pull/15852

From wkemper at openjdk.org  Fri Oct 27 08:21:42 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 27 Oct 2023 08:21:42 GMT
Subject: Integrated: 8316632: Shenandoah: Raise OOME when gc threshold is
 exceeded
In-Reply-To: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
References: <mGnKoXbZEMneVrvKd0tP5vdkuAwhB2DHGEFTLT15dHQ=.af99f4e9-a677-49b2-8c26-5e2be71df96f@github.com>
Message-ID: <odHwihzpMsgmcCQYG309bMMsNJGW5dYQYc2CELRROjA=.6e93b435-4a23-4818-bb56-f0f178290ae8@github.com>

On Wed, 20 Sep 2023 22:41:52 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Shenandoah will run back-to-back full GCs and _almost_ grind mutator progress to a halt before eventually exhausting memory. This change will have Shenandoah raise a gc threshold exceeded exception if the collector fails to make progress after `ShenandoahNoProgressThreshold` full GC cycles (default is 3).

This pull request has now been integrated.

Changeset: 5b5fd369
Author:    William Kemper <wkemper at openjdk.org>
Committer: Aleksey Shipilev <shade at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/5b5fd3694ac6ef224af311a7ab62547dac976da4
Stats:     73 lines in 7 files changed: 54 ins; 2 del; 17 mod

8316632: Shenandoah: Raise OOME when gc threshold is exceeded

Reviewed-by: kdnilsen, ysr, shade

-------------

PR: https://git.openjdk.org/jdk/pull/15852

From lkorinth at openjdk.org  Fri Oct 27 08:50:47 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Fri, 27 Oct 2023 08:50:47 GMT
Subject: Integrated: 8315097: Rename createJavaProcessBuilder
In-Reply-To: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
Message-ID: <n5Oky40OR8ZXdkJbSQiivcZCzZ6XP12wnJEWzcY4xYw=.d997c70c-cae2-4bf3-9d83-2f24d1cbcb22@github.com>

On Mon, 28 Aug 2023 15:54:08 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

> This pull request renames `createJavaProcessBuilder` to `createLimitedTestJavaProcessBuilder` and renames `createTestJvm` to `createTestJavaProcessBuilder`. Both are implemented through a private `createJavaProcessBuilder`. It also updates the java doc.
> 
> This is so that it should be harder to by mistake use `createLimitedTestJavaProcessBuilder` that is problematic because it will not forward JVM flags to the tested JVM.

This pull request has now been integrated.

Changeset: d52a995f
Author:    Leo Korinth <lkorinth at openjdk.org>
URL:       https://git.openjdk.org/jdk/commit/d52a995f35de26c2cc4074297a75141e4a363e1b
Stats:     1574 lines in 560 files changed: 44 ins; 10 del; 1520 mod

8315097: Rename createJavaProcessBuilder

Reviewed-by: lmesnik, dholmes, rriggs, stefank

-------------

PR: https://git.openjdk.org/jdk/pull/15452

From lkorinth at openjdk.org  Fri Oct 27 09:00:48 2023
From: lkorinth at openjdk.org (Leo Korinth)
Date: Fri, 27 Oct 2023 09:00:48 GMT
Subject: RFR: 8315097: Rename createJavaProcessBuilder [v7]
In-Reply-To: <fUZIxEzy3YZUqfx87qCu8fVP3jSaRzQKKNMIHjmu9F0=.ae9a3e9c-70a8-42cf-bf81-8e7193c72003@github.com>
References: <uiRsPN6Y-xlpEwV-mnXYs8JbFseCnxbtGrxfgUAk2XE=.c9be5d27-e11f-401b-9797-c2943e7a83db@github.com>
 <fUZIxEzy3YZUqfx87qCu8fVP3jSaRzQKKNMIHjmu9F0=.ae9a3e9c-70a8-42cf-bf81-8e7193c72003@github.com>
Message-ID: <b2XfkO_Ez_LuJQZffSPp63wciX1NImzAD5l3Xy1elOU=.48bdc440-238c-4a86-b8b8-c2f37e2277a7@github.com>

On Wed, 25 Oct 2023 08:44:29 GMT, Leo Korinth <lkorinth at openjdk.org> wrote:

>> This pull request renames `createJavaProcessBuilder` to `createLimitedTestJavaProcessBuilder` and renames `createTestJvm` to `createTestJavaProcessBuilder`. Both are implemented through a private `createJavaProcessBuilder`. It also updates the java doc.
>> 
>> This is so that it should be harder to by mistake use `createLimitedTestJavaProcessBuilder` that is problematic because it will not forward JVM flags to the tested JVM.
>
> Leo Korinth has updated the pull request incrementally with one additional commit since the last revision:
> 
>   fix copyright year and indentation

Thanks, see:
https://bugs.openjdk.org/browse/JDK-8318962

-------------

PR Comment: https://git.openjdk.org/jdk/pull/15452#issuecomment-1782552641

From wkemper at openjdk.org  Fri Oct 27 17:08:41 2023
From: wkemper at openjdk.org (William Kemper)
Date: Fri, 27 Oct 2023 17:08:41 GMT
Subject: RFR: Merge openjdk/jdk:master [v2]
In-Reply-To: <wdsao1bTO2smDL_gzJaICcwf4IJ-uqae_i3tC8pUwfM=.3ce36561-358a-468f-aed6-ba07b237a648@github.com>
References: <wdsao1bTO2smDL_gzJaICcwf4IJ-uqae_i3tC8pUwfM=.3ce36561-358a-468f-aed6-ba07b237a648@github.com>
Message-ID: <lUVWs0ZKUEBMUcdR2TO_HCal4jH8x6VO98I1OLlIULI=.a7290ba4-6857-4d5a-b7bb-612ecdcf13c1@github.com>

> Merges tag jdk-22+21

William Kemper has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains 252 commits:

 - Merge tag 'jdk-22+21' into merge-jdk-22+21
   
   Added tag jdk-22+21 for changeset d96f38b8
 - 8317510: Change Windows debug symbol files naming to avoid losing info when an executable and a library share the same name
   
   Reviewed-by: ihse, erikj
 - 8318613: ChoiceFormat patterns are not well tested
   
   Reviewed-by: naoto
 - 8318186: ChoiceFormat inconsistency between applyPattern() and setChoices()
   
   Reviewed-by: naoto
 - 8318487: Specification of the ListFormat.equals() method can be improved
   
   Reviewed-by: joehw, rriggs, lancea, iris
 - 8317360: Missing null checks in JfrCheckpointManager and JfrStringPool initialization routines
   
   Reviewed-by: shade, egahlin
 - 8318735: RISC-V: Enable related hotspot tests run on riscv
   
   Reviewed-by: fyang, luhenry
 - 8318727: Enable parallelism in vmTestbase/vm/gc/concurrent tests
   
   Reviewed-by: shade, lmesnik
 - 8318607: Enable parallelism in vmTestbase/nsk/stress/jni tests
   
   Reviewed-by: lmesnik, shade
 - 8318608: Enable parallelism in vmTestbase/nsk/stress/threads tests
   
   Reviewed-by: lmesnik, shade
 - ... and 242 more: https://git.openjdk.org/shenandoah/compare/c584b481...12124e43

-------------

Changes: https://git.openjdk.org/shenandoah/pull/349/files
 Webrev: https://webrevs.openjdk.org/?repo=shenandoah&pr=349&range=01
  Stats: 30986 lines in 1077 files changed: 19698 ins; 6209 del; 5079 mod
  Patch: https://git.openjdk.org/shenandoah/pull/349.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/349/head:pull/349

PR: https://git.openjdk.org/shenandoah/pull/349

From ysr at openjdk.org  Mon Oct 30 20:56:59 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Mon, 30 Oct 2023 20:56:59 GMT
Subject: RFR: 8318057: GenShen: investigate small refactor of remset swap
 code [v2]
In-Reply-To: <qEO5M57wWdi78hROoJlrNua7hys3sobUN_pXRVxeL_8=.a2ae0dba-7391-449e-b02d-2c8d25a5f6be@github.com>
References: <qEO5M57wWdi78hROoJlrNua7hys3sobUN_pXRVxeL_8=.a2ae0dba-7391-449e-b02d-2c8d25a5f6be@github.com>
Message-ID: <Z7QIBW_Vo8wzRsvMxeCHal9q02TEnenkHG44MlX0QT0=.40d4b875-d85c-4c12-b494-6069c722f705@github.com>

> Some closures were being unnecessarily applied to free regions rather than just to allocated old and young regions. Restrict the closure application to old and young regions as appropriate, eliminating extra generation affiliation checks in some closures. Also fixed a slightly incorrect documentation comment.
> 
> **Update:** I am converting this back to a draft until some further improvements are attempted, and more performance data gathered. Will reopen when done.
> 
> **Testing:**
> 
> - [x] jtreg and stress tests in test pipeline helper
> - [x] locally tested with jtreg hotsot_gc tests and with specjbb with and without heap verification
> - [x] no performance improvements noted; gathering more data, see ticket for latest updates

Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 11 additional commits since the last revision:

 - Slight improvement on code in last commit
 - A paranoid, but useful assert on sizes of types.
 - Fix Windows build warning.
 - Fix off by one violating card table API precondition.
 - Move implementations into .cpp from .hpp.
   Fix an indexing error.
 - Merge branch 'master' into swap_rs
 - Avoid division in card table merge/clear.
 - Merge branch 'master' into swap_rs
 - fix for comp error (wrong variable name).
 -     Avoid including free young regions in young gen heap region
       iteration.
 - ... and 1 more: https://git.openjdk.org/shenandoah/compare/300d4969...6f241a48

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/342/files
  - new: https://git.openjdk.org/shenandoah/pull/342/files/7d465163..6f241a48

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=342&range=01
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=342&range=00-01

  Stats: 469 lines in 22 files changed: 335 ins; 32 del; 102 mod
  Patch: https://git.openjdk.org/shenandoah/pull/342.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/342/head:pull/342

PR: https://git.openjdk.org/shenandoah/pull/342

From wkemper at openjdk.org  Mon Oct 30 20:58:08 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 30 Oct 2023 20:58:08 GMT
Subject: RFR: Merge openjdk/jdk:master [v3]
In-Reply-To: <wdsao1bTO2smDL_gzJaICcwf4IJ-uqae_i3tC8pUwfM=.3ce36561-358a-468f-aed6-ba07b237a648@github.com>
References: <wdsao1bTO2smDL_gzJaICcwf4IJ-uqae_i3tC8pUwfM=.3ce36561-358a-468f-aed6-ba07b237a648@github.com>
Message-ID: <0dUk3aPDYkwFBfahUQVfIoPaJSw1JxsNIv9md6MBx0M=.92b0a847-775a-4b73-8590-f92758a1d4e6@github.com>

> Merges tag jdk-22+21

William Kemper has updated the pull request incrementally with one additional commit since the last revision:

  Increase maximum number of log tag selections

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/349/files
  - new: https://git.openjdk.org/shenandoah/pull/349/files/12124e43..03ea0668

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=349&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=349&range=01-02

  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/shenandoah/pull/349.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/349/head:pull/349

PR: https://git.openjdk.org/shenandoah/pull/349

From kdnilsen at openjdk.org  Mon Oct 30 21:14:06 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Mon, 30 Oct 2023 21:14:06 GMT
Subject: RFR: 8318057: GenShen: investigate small refactor of remset swap
 code [v2]
In-Reply-To: <Z7QIBW_Vo8wzRsvMxeCHal9q02TEnenkHG44MlX0QT0=.40d4b875-d85c-4c12-b494-6069c722f705@github.com>
References: <qEO5M57wWdi78hROoJlrNua7hys3sobUN_pXRVxeL_8=.a2ae0dba-7391-449e-b02d-2c8d25a5f6be@github.com>
 <Z7QIBW_Vo8wzRsvMxeCHal9q02TEnenkHG44MlX0QT0=.40d4b875-d85c-4c12-b494-6069c722f705@github.com>
Message-ID: <rVDs_-wHbbqGPngIN6AUfTv6K6VJtQ1_sfgYlEBQVbQ=.e9430de4-f03b-4bc6-93aa-8969a63ec577@github.com>

On Mon, 30 Oct 2023 20:56:59 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Some closures were being unnecessarily applied to free regions rather than just to allocated old and young regions. Restrict the closure application to old and young regions as appropriate, eliminating extra generation affiliation checks in some closures. Also fixed a slightly incorrect documentation comment.
>> 
>> **Update:** I am converting this back to a draft until some further improvements are attempted, and more performance data gathered. Will reopen when done.
>> 
>> **Testing:**
>> 
>> - [x] jtreg and stress tests in test pipeline helper
>> - [x] locally tested with jtreg hotsot_gc tests and with specjbb with and without heap verification
>> - [x] no performance improvements noted; gathering more data, see ticket for latest updates
>
> Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 11 additional commits since the last revision:
> 
>  - Slight improvement on code in last commit
>  - A paranoid, but useful assert on sizes of types.
>  - Fix Windows build warning.
>  - Fix off by one violating card table API precondition.
>  - Move implementations into .cpp from .hpp.
>    Fix an indexing error.
>  - Merge branch 'master' into swap_rs
>  - Avoid division in card table merge/clear.
>  - Merge branch 'master' into swap_rs
>  - fix for comp error (wrong variable name).
>  -     Avoid including free young regions in young gen heap region
>        iteration.
>  - ... and 1 more: https://git.openjdk.org/shenandoah/compare/b88b8f40...6f241a48

Thanks.

-------------

Marked as reviewed by kdnilsen (Committer).

PR Review: https://git.openjdk.org/shenandoah/pull/342#pullrequestreview-1705081554

From wkemper at openjdk.org  Mon Oct 30 22:07:12 2023
From: wkemper at openjdk.org (William Kemper)
Date: Mon, 30 Oct 2023 22:07:12 GMT
Subject: RFR: 8318057: GenShen: investigate small refactor of remset swap
 code [v2]
In-Reply-To: <Z7QIBW_Vo8wzRsvMxeCHal9q02TEnenkHG44MlX0QT0=.40d4b875-d85c-4c12-b494-6069c722f705@github.com>
References: <qEO5M57wWdi78hROoJlrNua7hys3sobUN_pXRVxeL_8=.a2ae0dba-7391-449e-b02d-2c8d25a5f6be@github.com>
 <Z7QIBW_Vo8wzRsvMxeCHal9q02TEnenkHG44MlX0QT0=.40d4b875-d85c-4c12-b494-6069c722f705@github.com>
Message-ID: <DR_feIMPIAN33fzQfGcCPEZg24YXgYCHnd9eWR7zfb8=.b0ecc376-98b8-4800-ba7f-b2a5a3491eed@github.com>

On Mon, 30 Oct 2023 20:56:59 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Some closures were being unnecessarily applied to free regions rather than just to allocated old and young regions. Restrict the closure application to old and young regions as appropriate, eliminating extra generation affiliation checks in some closures. Also fixed a slightly incorrect documentation comment.
>> 
>> **Update:** I am converting this back to a draft until some further improvements are attempted, and more performance data gathered. Will reopen when done.
>> 
>> **Testing:**
>> 
>> - [x] jtreg and stress tests in test pipeline helper
>> - [x] locally tested with jtreg hotsot_gc tests and with specjbb with and without heap verification
>> - [x] no performance improvements noted; gathering more data, see ticket for latest updates
>
> Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 11 additional commits since the last revision:
> 
>  - Slight improvement on code in last commit
>  - A paranoid, but useful assert on sizes of types.
>  - Fix Windows build warning.
>  - Fix off by one violating card table API precondition.
>  - Move implementations into .cpp from .hpp.
>    Fix an indexing error.
>  - Merge branch 'master' into swap_rs
>  - Avoid division in card table merge/clear.
>  - Merge branch 'master' into swap_rs
>  - fix for comp error (wrong variable name).
>  -     Avoid including free young regions in young gen heap region
>        iteration.
>  - ... and 1 more: https://git.openjdk.org/shenandoah/compare/32774144...6f241a48

Marked as reviewed by wkemper (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/342#pullrequestreview-1705146074

From wkemper at openjdk.org  Tue Oct 31 00:36:08 2023
From: wkemper at openjdk.org (William Kemper)
Date: Tue, 31 Oct 2023 00:36:08 GMT
Subject: Integrated: Merge openjdk/jdk:master
In-Reply-To: <wdsao1bTO2smDL_gzJaICcwf4IJ-uqae_i3tC8pUwfM=.3ce36561-358a-468f-aed6-ba07b237a648@github.com>
References: <wdsao1bTO2smDL_gzJaICcwf4IJ-uqae_i3tC8pUwfM=.3ce36561-358a-468f-aed6-ba07b237a648@github.com>
Message-ID: <5y7okl5wf961LMS0IiBp6e76fUAcfwsAWY5yqh5zMfo=.b4f5b831-0296-40c5-a166-0c5ef7675956@github.com>

On Thu, 26 Oct 2023 14:10:04 GMT, William Kemper <wkemper at openjdk.org> wrote:

> Merges tag jdk-22+21

This pull request has now been integrated.

Changeset: 92bf8879
Author:    William Kemper <wkemper at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/92bf8879112b679ec7c9dce8d9d40e02edcd377a
Stats:     30987 lines in 1078 files changed: 19698 ins; 6209 del; 5080 mod

Merge

-------------

PR: https://git.openjdk.org/shenandoah/pull/349

From rkennke at openjdk.org  Tue Oct 31 09:42:19 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 31 Oct 2023 09:42:19 GMT
Subject: RFR: 8318462: [GenShen] Prevent unsafe access to displaced
 mark-word [v2]
In-Reply-To: <feghfM7FVv2wf1ImOidYnLI0eSGm27pDetlSsk4lmIU=.2a68f506-fc4a-4aea-bf27-f6812d859e8d@github.com>
References: <uOeuEpzRFvBx1AFgOvvTNfxJIyI4o84yT9D5A_q6BjU=.92103c05-893a-4b0b-a165-54289729f74f@github.com>
 <2lwvFiHSE-N-kXfzhKYx4HhQASkwI01cjVO_tPJ_KZI=.5c873318-cb3f-44e6-8745-d6f5cbe3e52d@github.com>
 <feghfM7FVv2wf1ImOidYnLI0eSGm27pDetlSsk4lmIU=.2a68f506-fc4a-4aea-bf27-f6812d859e8d@github.com>
Message-ID: <AXtEGeCXjkjSAQ2WvwkGeSI-fEDm9MRKujSaPXcPTWc=.728e373c-a3b7-437e-8694-e29748acd4fa@github.com>

On Thu, 19 Oct 2023 13:04:35 GMT, Zhengyu Gu <zgu at openjdk.org> wrote:

> Just FYI: Here is another place to update object age, probably you want to consolidate.
> 
> https://github.com/openjdk/shenandoah/blob/master/src/hotspot/share/gc/shenandoah/shenandoahStringDedup.inline.hpp#L55

Right. The problem there is that this block of code does a sort of concurrent/atomic update of the object age that would require re-working and/or adding age helper methods. I'd rather not do that now.

-------------

PR Comment: https://git.openjdk.org/shenandoah/pull/343#issuecomment-1786851940

From rkennke at openjdk.org  Tue Oct 31 09:51:29 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 31 Oct 2023 09:51:29 GMT
Subject: RFR: 8318462: [GenShen] Prevent unsafe access to displaced
 mark-word [v4]
In-Reply-To: <uOeuEpzRFvBx1AFgOvvTNfxJIyI4o84yT9D5A_q6BjU=.92103c05-893a-4b0b-a165-54289729f74f@github.com>
References: <uOeuEpzRFvBx1AFgOvvTNfxJIyI4o84yT9D5A_q6BjU=.92103c05-893a-4b0b-a165-54289729f74f@github.com>
Message-ID: <7g3I1quU1AP4udb0EdB_s_8dK1coaiFR8wGTEUZ2ASw=.4102020d-3d17-4901-87da-7a47bb4106fd@github.com>

> We must be very careful when we access (load or update) the object age concurrently because the age is stored in the object's header and that header can be 'displaced'. This means that the header is overloaded with a pointer (stack or ObjectMonitor*) and the original header is stored at that location. The header can also be in INFLATING state, which means that a stack-lock is currently in the process to be inflated to an ObjectMonitor. Let's consider the cases separately:
> 
> Loading object age:
> - INFLATING: we can not access the header, and thus not the age.
> - Stack-locked: the thread which locks the object can unlock at any time, concurrently. It is not safe to access the header, and thus the age.
> - Monitor-locked: monitors don't go away without coordinating with Java threads and possibly GC threads. This coordination is done by handshaking the thread - a Java thread would be brought to its next safepoint, and GC threads, which are typically participating in handshakes by joining the SuspendibleThreadSet, can opt to handshake at a safe point by calling SuspendibleThreadSet::yield(). If 'our' thread participates in handshakes in one way or the other, it is safe to access an ObjectMonitor once we got a valid ObjectMonitor* from an object's header, but only until that thread handshakes. Long story short: it is typically safe to access ObjectMonitor :-)
> 
> Updating object age:
> Updating object age only happens during evacuation, and only on a new copy of an object, before that copy has been published. Access to the object header is therefore fully thread-local and not problematic. What *is* problematic is when we have to deal with displaced headers, because displaced headers are *not* thread local, and must be considered a shared resource. A competing thread may succeed to evacuate the object and publish its copy before 'our' thread can do so. If 'our' thread would update the displaced header, it may over-write whatever the other thread has done.
> - INFLATING: We cannot access the header at all. However, that should not happen: a thread can only inflate once it has successfully evacuated an object, and 
> - Stack-locked: the other thread may succeed evacuation and continue to unlock the object, at which point the stack-pointer would be 'dangling' and we may over-write random information on the foreign thread stack.
> - Monitor-locked: even though monitor deflation is coordinated (see above), updating the displaced header in a monitor might over-write whatever a competi...

Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:

 - Treat age sentinel correctly
 - Fix comment formatting

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/343/files
  - new: https://git.openjdk.org/shenandoah/pull/343/files/31ef2d9c..113a3d2b

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=343&range=03
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=343&range=02-03

  Stats: 3 lines in 2 files changed: 1 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/shenandoah/pull/343.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/343/head:pull/343

PR: https://git.openjdk.org/shenandoah/pull/343

From kdnilsen at openjdk.org  Tue Oct 31 16:20:24 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 31 Oct 2023 16:20:24 GMT
Subject: RFR: 8318462: [GenShen] Prevent unsafe access to displaced
 mark-word [v4]
In-Reply-To: <7g3I1quU1AP4udb0EdB_s_8dK1coaiFR8wGTEUZ2ASw=.4102020d-3d17-4901-87da-7a47bb4106fd@github.com>
References: <uOeuEpzRFvBx1AFgOvvTNfxJIyI4o84yT9D5A_q6BjU=.92103c05-893a-4b0b-a165-54289729f74f@github.com>
 <7g3I1quU1AP4udb0EdB_s_8dK1coaiFR8wGTEUZ2ASw=.4102020d-3d17-4901-87da-7a47bb4106fd@github.com>
Message-ID: <42gYblh4e4IcCy5prnyxsUsJjvWRZj1S5WGg7kXU8Io=.a0b8dd2e-9d82-4469-9142-48e00b769c2f@github.com>

On Tue, 31 Oct 2023 09:51:29 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> We must be very careful when we access (load or update) the object age concurrently because the age is stored in the object's header and that header can be 'displaced'. This means that the header is overloaded with a pointer (stack or ObjectMonitor*) and the original header is stored at that location. The header can also be in INFLATING state, which means that a stack-lock is currently in the process to be inflated to an ObjectMonitor. Let's consider the cases separately:
>> 
>> Loading object age:
>> - INFLATING: we can not access the header, and thus not the age.
>> - Stack-locked: the thread which locks the object can unlock at any time, concurrently. It is not safe to access the header, and thus the age.
>> - Monitor-locked: monitors don't go away without coordinating with Java threads and possibly GC threads. This coordination is done by handshaking the thread - a Java thread would be brought to its next safepoint, and GC threads, which are typically participating in handshakes by joining the SuspendibleThreadSet, can opt to handshake at a safe point by calling SuspendibleThreadSet::yield(). If 'our' thread participates in handshakes in one way or the other, it is safe to access an ObjectMonitor once we got a valid ObjectMonitor* from an object's header, but only until that thread handshakes. Long story short: it is typically safe to access ObjectMonitor :-)
>> 
>> Updating object age:
>> Updating object age only happens during evacuation, and only on a new copy of an object, before that copy has been published. Access to the object header is therefore fully thread-local and not problematic. What *is* problematic is when we have to deal with displaced headers, because displaced headers are *not* thread local, and must be considered a shared resource. A competing thread may succeed to evacuate the object and publish its copy before 'our' thread can do so. If 'our' thread would update the displaced header, it may over-write whatever the other thread has done.
>> - INFLATING: We cannot access the header at all. However, that should not happen: a thread can only inflate once it has successfully evacuated an object, and 
>> - Stack-locked: the other thread may succeed evacuation and continue to unlock the object, at which point the stack-pointer would be 'dangling' and we may over-write random information on the foreign thread stack.
>> - Monitor-locked: even though monitor deflation is coordinated (see above), updating the displaced header in a monitor might over-...
>
> Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
> 
>  - Treat age sentinel correctly
>  - Fix comment formatting

Marked as reviewed by kdnilsen (Committer).

-------------

PR Review: https://git.openjdk.org/shenandoah/pull/343#pullrequestreview-1706735249

From kdnilsen at openjdk.org  Tue Oct 31 16:20:28 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 31 Oct 2023 16:20:28 GMT
Subject: RFR: 8318462: [GenShen] Prevent unsafe access to displaced
 mark-word [v3]
In-Reply-To: <62iAP1xppfg6WDXm-ZdoT8cugt1L-2athpTtFc145M8=.9e763172-f272-4675-93a7-3733d63b335b@github.com>
References: <uOeuEpzRFvBx1AFgOvvTNfxJIyI4o84yT9D5A_q6BjU=.92103c05-893a-4b0b-a165-54289729f74f@github.com>
 <G4dtAHgyXda6zZiq16qHiyQWWCapLgkoW0dASfjLIF8=.14abe76d-7980-4227-9698-35ded44fc141@github.com>
 <S7Jx2poxtFfB7voH5TUPgzc40hy4jIPQGhHF6kW8VEM=.40dd23a1-78a4-4534-8a86-aa1d99ef7347@github.com>
 <62iAP1xppfg6WDXm-ZdoT8cugt1L-2athpTtFc145M8=.9e763172-f272-4675-93a7-3733d63b335b@github.com>
Message-ID: <KsJtEqDREZmNhyzaiGDx04yXm_SW-1q-dsrryuS65qc=.2084b551-eade-48e2-ad49-316a7a69f25b@github.com>

On Thu, 26 Oct 2023 16:25:05 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> src/hotspot/share/gc/shenandoah/shenandoahMark.inline.hpp line 123:
>> 
>>> 121:       assert(region->is_young(), "Only for young objects");
>>> 122:       uint age = ShenandoahHeap::get_object_age(obj);
>>> 123:       CENSUS_NOISE(heap->age_census()->add(age, region->age(), region->youth(), size, worker_id);)
>> 
>> I'm thinking if get_object_age() returns max_age+1, the CENSUS_NOISE macros might end up with invalid data.  Don't we need a check here?
>
> I think the safe thing to do is treat max_age+1 as if it were 0, insofar as promotion semantics is concerned.  We may delay promotion that way.

Thanks for change.  Code might read easier with parens around (age == markWord::max_age + 1), but ok as is.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/343#discussion_r1377851974

From ysr at openjdk.org  Tue Oct 31 17:31:33 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 31 Oct 2023 17:31:33 GMT
Subject: RFR: 8318057: GenShen: investigate small refactor of remset swap
 code [v3]
In-Reply-To: <qEO5M57wWdi78hROoJlrNua7hys3sobUN_pXRVxeL_8=.a2ae0dba-7391-449e-b02d-2c8d25a5f6be@github.com>
References: <qEO5M57wWdi78hROoJlrNua7hys3sobUN_pXRVxeL_8=.a2ae0dba-7391-449e-b02d-2c8d25a5f6be@github.com>
Message-ID: <Uy06w8knbNPIkSrIlCBxBLzqxpGurI2jO0mQyekh4Bc=.ced5c9af-e301-4669-9225-f3e64f9440a0@github.com>

> Some closures were being unnecessarily applied to free regions rather than just to allocated old and young regions. Restrict the closure application to old and young regions as appropriate, eliminating extra generation affiliation checks in some closures. Also fixed a slightly incorrect documentation comment.
> 
> Replaced use of integer division with right shifts. Added a few useful assertions associated with these operations.
> 
> However, neither of these made any measurable difference to the performance of the bulk operations on the card table.
> 
> **Testing:**
> 
> - [x] jtreg and stress tests in test pipeline helper
> - [x] locally tested with jtreg hotsot_gc tests and with specjbb with and without heap verification
> - [x] no performance improvements noted; gathering more data, see ticket for latest updates

Y. Srinivas Ramakrishna has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains 12 additional commits since the last revision:

 - Merge branch 'master' into swap_rs
 - Slight improvement on code in last commit
 - A paranoid, but useful assert on sizes of types.
 - Fix Windows build warning.
 - Fix off by one violating card table API precondition.
 - Move implementations into .cpp from .hpp.
   Fix an indexing error.
 - Merge branch 'master' into swap_rs
 - Avoid division in card table merge/clear.
 - Merge branch 'master' into swap_rs
 - fix for comp error (wrong variable name).
 - ... and 2 more: https://git.openjdk.org/shenandoah/compare/c1e87514...3903b3ec

-------------

Changes:
  - all: https://git.openjdk.org/shenandoah/pull/342/files
  - new: https://git.openjdk.org/shenandoah/pull/342/files/6f241a48..3903b3ec

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=shenandoah&pr=342&range=02
 - incr: https://webrevs.openjdk.org/?repo=shenandoah&pr=342&range=01-02

  Stats: 30987 lines in 1078 files changed: 19698 ins; 6209 del; 5080 mod
  Patch: https://git.openjdk.org/shenandoah/pull/342.diff
  Fetch: git fetch https://git.openjdk.org/shenandoah.git pull/342/head:pull/342

PR: https://git.openjdk.org/shenandoah/pull/342

From ysr at openjdk.org  Tue Oct 31 17:38:14 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 31 Oct 2023 17:38:14 GMT
Subject: Integrated: 8318057: GenShen: investigate small refactor of remset
 swap code
In-Reply-To: <qEO5M57wWdi78hROoJlrNua7hys3sobUN_pXRVxeL_8=.a2ae0dba-7391-449e-b02d-2c8d25a5f6be@github.com>
References: <qEO5M57wWdi78hROoJlrNua7hys3sobUN_pXRVxeL_8=.a2ae0dba-7391-449e-b02d-2c8d25a5f6be@github.com>
Message-ID: <hcYTs7G4Y1U0IxkCkigpbVZEwO-Gk-XdCx4BOezgbTc=.99b7d763-4c9f-4643-bb1d-884805f40c01@github.com>

On Tue, 17 Oct 2023 23:26:31 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

> Some closures were being unnecessarily applied to free regions rather than just to allocated old and young regions. Restrict the closure application to old and young regions as appropriate, eliminating extra generation affiliation checks in some closures. Also fixed a slightly incorrect documentation comment.
> 
> Replaced use of integer division with right shifts. Added a few useful assertions associated with these operations.
> 
> However, neither of these made any measurable difference to the performance of the bulk operations on the card table.
> 
> **Testing:**
> 
> - [x] jtreg and stress tests in test pipeline helper
> - [x] locally tested with jtreg hotsot_gc tests and with specjbb with and without heap verification
> - [x] no performance improvements noted; see ticket

This pull request has now been integrated.

Changeset: 05b7a3c5
Author:    Y. Srinivas Ramakrishna <ysr at openjdk.org>
URL:       https://git.openjdk.org/shenandoah/commit/05b7a3c5d59173c9f7c9db4ee7a7bca642c3d4a9
Stats:     103 lines in 4 files changed: 61 ins; 25 del; 17 mod

8318057: GenShen: investigate small refactor of remset swap code

Reviewed-by: kdnilsen, wkemper

-------------

PR: https://git.openjdk.org/shenandoah/pull/342

From ysr at openjdk.org  Tue Oct 31 22:35:45 2023
From: ysr at openjdk.org (Y. Srinivas Ramakrishna)
Date: Tue, 31 Oct 2023 22:35:45 GMT
Subject: RFR: 8318462: [GenShen] Prevent unsafe access to displaced
 mark-word [v3]
In-Reply-To: <KsJtEqDREZmNhyzaiGDx04yXm_SW-1q-dsrryuS65qc=.2084b551-eade-48e2-ad49-316a7a69f25b@github.com>
References: <uOeuEpzRFvBx1AFgOvvTNfxJIyI4o84yT9D5A_q6BjU=.92103c05-893a-4b0b-a165-54289729f74f@github.com>
 <G4dtAHgyXda6zZiq16qHiyQWWCapLgkoW0dASfjLIF8=.14abe76d-7980-4227-9698-35ded44fc141@github.com>
 <S7Jx2poxtFfB7voH5TUPgzc40hy4jIPQGhHF6kW8VEM=.40dd23a1-78a4-4534-8a86-aa1d99ef7347@github.com>
 <62iAP1xppfg6WDXm-ZdoT8cugt1L-2athpTtFc145M8=.9e763172-f272-4675-93a7-3733d63b335b@github.com>
 <KsJtEqDREZmNhyzaiGDx04yXm_SW-1q-dsrryuS65qc=.2084b551-eade-48e2-ad49-316a7a69f25b@github.com>
Message-ID: <3ZaUyCn3Fxysz9hN4CK8EHQmDiGW3VEK8eIiBhFM70k=.e9b27e34-c63c-419a-b6b8-4af376c28480@github.com>

On Tue, 31 Oct 2023 16:17:01 GMT, Kelvin Nilsen <kdnilsen at openjdk.org> wrote:

>> I think the safe thing to do is treat max_age+1 as if it were 0, insofar as promotion semantics is concerned.  We may delay promotion that way.
>
> Thanks for change.  Code might read easier with parens around (age == markWord::max_age + 1), but ok as is.

The add method of ShenandoahAgeCensus explicitly recognizes and uses the sentinel value, and explicitly skips sentinels that are used to signal that the age could not be read:


CENSUS_NOISE(void ShenandoahAgeCensus::add(uint obj_age, uint region_age, uint region_youth, size_t size, uint worker_id) {)
NO_CENSUS_NOISE(void ShenandoahAgeCensus::add(uint obj_age, uint region_age, size_t size, uint worker_id) {)
  if (obj_age <= markWord::max_age) {
    assert(obj_age < MAX_COHORTS && region_age < MAX_COHORTS, "Should have been tenured");
#ifdef SHENANDOAH_CENSUS_NOISE
    // Region ageing is stochastic and non-monotonic; this vitiates mortality
    // demographics in ways that might defeat our algorithms. Marking may be a
    // time when we might be able to correct this, but we currently do not do
    // this. Like skipped statistics further below, we want to track the
    // impact of this noise to see if this may be worthwhile. JDK-<TBD>.
    uint age = obj_age;
    if (region_age > 0) {
      add_aged(size, worker_id);   // this tracking is coarse for now
      age += region_age;
      if (age >= MAX_COHORTS) {
        age = (uint)(MAX_COHORTS - 1);  // clamp
        add_clamped(size, worker_id);
      }
    }
    if (region_youth > 0) {   // track object volume with retrograde age
      add_young(size, worker_id);
    }
#else   // SHENANDOAH_CENSUS_NOISE
    uint age = MIN2(obj_age + region_age, (uint)(MAX_COHORTS - 1));  // clamp
#endif  // SHENANDOAH_CENSUS_NOISE
    get_local_age_table(worker_id)->add(age, size);
  } else {
    // update skipped statistics
    CENSUS_NOISE(add_skipped(size, worker_id);)
  }
}


I think we should leave in place the original code, where all of the special handling of the sentinel values is relegated to the ShenandoahAgeCensus's add method, with no special handling needed at the caller.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/343#discussion_r1378223807

From kdnilsen at openjdk.org  Tue Oct 31 23:24:33 2023
From: kdnilsen at openjdk.org (Kelvin Nilsen)
Date: Tue, 31 Oct 2023 23:24:33 GMT
Subject: RFR: 8318462: [GenShen] Prevent unsafe access to displaced
 mark-word [v3]
In-Reply-To: <3ZaUyCn3Fxysz9hN4CK8EHQmDiGW3VEK8eIiBhFM70k=.e9b27e34-c63c-419a-b6b8-4af376c28480@github.com>
References: <uOeuEpzRFvBx1AFgOvvTNfxJIyI4o84yT9D5A_q6BjU=.92103c05-893a-4b0b-a165-54289729f74f@github.com>
 <G4dtAHgyXda6zZiq16qHiyQWWCapLgkoW0dASfjLIF8=.14abe76d-7980-4227-9698-35ded44fc141@github.com>
 <S7Jx2poxtFfB7voH5TUPgzc40hy4jIPQGhHF6kW8VEM=.40dd23a1-78a4-4534-8a86-aa1d99ef7347@github.com>
 <62iAP1xppfg6WDXm-ZdoT8cugt1L-2athpTtFc145M8=.9e763172-f272-4675-93a7-3733d63b335b@github.com>
 <KsJtEqDREZmNhyzaiGDx04yXm_SW-1q-dsrryuS65qc=.2084b551-eade-48e2-ad49-316a7a69f25b@github.com>
 <3ZaUyCn3Fxysz9hN4CK8EHQmDiGW3VEK8eIiBhFM70k=.e9b27e34-c63c-419a-b6b8-4af376c28480@github.com>
Message-ID: <7WgrlAQun762-rhgugi6qSKkqbrJqAskJDBEDMJ8O6o=.9516f2e1-314a-4173-9b0d-d4e334884c2e@github.com>

On Tue, 31 Oct 2023 22:33:16 GMT, Y. Srinivas Ramakrishna <ysr at openjdk.org> wrote:

>> Thanks for change.  Code might read easier with parens around (age == markWord::max_age + 1), but ok as is.
>
> The add method of ShenandoahAgeCensus explicitly recognizes and uses the sentinel value, and explicitly skips sentinels that are used to signal that the age could not be read:
> 
> 
> CENSUS_NOISE(void ShenandoahAgeCensus::add(uint obj_age, uint region_age, uint region_youth, size_t size, uint worker_id) {)
> NO_CENSUS_NOISE(void ShenandoahAgeCensus::add(uint obj_age, uint region_age, size_t size, uint worker_id) {)
>   if (obj_age <= markWord::max_age) {
>     assert(obj_age < MAX_COHORTS && region_age < MAX_COHORTS, "Should have been tenured");
> #ifdef SHENANDOAH_CENSUS_NOISE
>     // Region ageing is stochastic and non-monotonic; this vitiates mortality
>     // demographics in ways that might defeat our algorithms. Marking may be a
>     // time when we might be able to correct this, but we currently do not do
>     // this. Like skipped statistics further below, we want to track the
>     // impact of this noise to see if this may be worthwhile. JDK-<TBD>.
>     uint age = obj_age;
>     if (region_age > 0) {
>       add_aged(size, worker_id);   // this tracking is coarse for now
>       age += region_age;
>       if (age >= MAX_COHORTS) {
>         age = (uint)(MAX_COHORTS - 1);  // clamp
>         add_clamped(size, worker_id);
>       }
>     }
>     if (region_youth > 0) {   // track object volume with retrograde age
>       add_young(size, worker_id);
>     }
> #else   // SHENANDOAH_CENSUS_NOISE
>     uint age = MIN2(obj_age + region_age, (uint)(MAX_COHORTS - 1));  // clamp
> #endif  // SHENANDOAH_CENSUS_NOISE
>     get_local_age_table(worker_id)->add(age, size);
>   } else {
>     // update skipped statistics
>     CENSUS_NOISE(add_skipped(size, worker_id);)
>   }
> }
> 
> 
> I think we should leave in place the original code, where all of the special handling of the sentinel values is relegated to the ShenandoahAgeCensus's add method, with no special handling needed at the caller.

Thanks.  That looks good.  I concur.

-------------

PR Review Comment: https://git.openjdk.org/shenandoah/pull/343#discussion_r1378252443