From rkennke at openjdk.org Mon Apr 3 19:03:30 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 3 Apr 2023 19:03:30 GMT
Subject: [lilliput-jdk17u:lilliput] Integrated: 8304329: [Lilliput/JDK17] Fix
DiagnoseSyncOnValueBasedClasses
In-Reply-To: <YJsGB_SUMYthFr_AOziUlUi5ck26VlbQN4cRF5xI2kw=.4c0b50f2-478d-41d4-89e1-956e5edb5787@github.com>
References: <YJsGB_SUMYthFr_AOziUlUi5ck26VlbQN4cRF5xI2kw=.4c0b50f2-478d-41d4-89e1-956e5edb5787@github.com>
Message-ID: <S427GVpJOrNI5zyL5AgxCXVytT2eZWozJUbycVwHqhI=.6dcf60ea-e0d6-4ec7-9e62-8e46411fd911@github.com>
On Thu, 16 Mar 2023 09:15:02 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> Some deeper testing shows that the following test is currently failing in Lilliput/JDK17:
>
> java/net/httpclient/LineBodyHandlerTest.java
>
> That is caused by that test using DiagnoseSyncOnValueBasedClasses, and that seems currently broken with Lilliput/JDK17 because it sets a plain prototype header on a class, where it should also set the narrow Klass* in the prototype.
>
> Testing:
> - [x] java/net/httpclient/LineBodyHandlerTest.java
> - [x] tier1
This pull request has now been integrated.
Changeset: 71052d21
Author: Roman Kennke <rkennke at openjdk.org>
URL: https://git.openjdk.org/lilliput-jdk17u/commit/71052d21867ce7c90588886c443daf195b5cceda
Stats: 5 lines in 1 file changed: 4 ins; 0 del; 1 mod
8304329: [Lilliput/JDK17] Fix DiagnoseSyncOnValueBasedClasses
Reviewed-by: shade
-------------
PR: https://git.openjdk.org/lilliput-jdk17u/pull/11
From rkennke at openjdk.org Mon Apr 3 19:13:14 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 3 Apr 2023 19:13:14 GMT
Subject: [lilliput-jdk17u:lilliput] RFR: 8304331: [Lilliput/JDK17] Fix RTM
locking [v3]
In-Reply-To: <TjXqrV0uCxQqxboV1uoeZrYGfkIXMlxnJOgCEqATQrA=.1525308a-f64c-4fe5-a41a-3e9a9bfe645a@github.com>
References: <TjXqrV0uCxQqxboV1uoeZrYGfkIXMlxnJOgCEqATQrA=.1525308a-f64c-4fe5-a41a-3e9a9bfe645a@github.com>
Message-ID: <V1f8EsG5IKPFcHZeF2ZWaN6VqnXC4isbEZrYCe4Tnwo=.4d5d3bab-fcee-454f-b946-79442ff3f447@github.com>
> Some RTM locking tests are currently failing:
>
> compiler/rtm/cli/TestUseRTMLockingOptionWithBiasedLocking.java
> compiler/rtm/locking/TestRTMRetryCount.java
> compiler/rtm/locking/TestRTMTotalCountIncrRate.java
> compiler/rtm/locking/TestUseRTMAfterLockInflation.java
> compiler/rtm/locking/TestUseRTMForInflatedLocks.java
> compiler/rtm/locking/TestUseRTMForStackLocks.java
> compiler/rtm/method_options/TestUseRTMLockElidingOption.java
>
> Most are because there's a jmpb in UseRTMLocking parts that fails because of longer locking code with Lilliput. One test is failing because it checks a warning message when somebody tries +UseBiasedLocking together with RTM locking, but Lilliput forces biased locking to be off, so the warning never appears. I problem-listed that test.
>
> Testing:
> - [x] all tests listed above
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
Don't use this bug's bug-ID in problem-list
-------------
Changes:
- all: https://git.openjdk.org/lilliput-jdk17u/pull/12/files
- new: https://git.openjdk.org/lilliput-jdk17u/pull/12/files/45a345b8..a8b00af6
Webrevs:
- full: https://webrevs.openjdk.org/?repo=lilliput-jdk17u&pr=12&range=02
- incr: https://webrevs.openjdk.org/?repo=lilliput-jdk17u&pr=12&range=01-02
Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
Patch: https://git.openjdk.org/lilliput-jdk17u/pull/12.diff
Fetch: git fetch https://git.openjdk.org/lilliput-jdk17u.git pull/12/head:pull/12
PR: https://git.openjdk.org/lilliput-jdk17u/pull/12
From rkennke at openjdk.org Mon Apr 3 19:13:15 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 3 Apr 2023 19:13:15 GMT
Subject: [lilliput-jdk17u:lilliput] Integrated: 8304331: [Lilliput/JDK17] Fix
RTM locking
In-Reply-To: <TjXqrV0uCxQqxboV1uoeZrYGfkIXMlxnJOgCEqATQrA=.1525308a-f64c-4fe5-a41a-3e9a9bfe645a@github.com>
References: <TjXqrV0uCxQqxboV1uoeZrYGfkIXMlxnJOgCEqATQrA=.1525308a-f64c-4fe5-a41a-3e9a9bfe645a@github.com>
Message-ID: <5S1c_J6UArwhLrHRJEFr5K_H-Zx8XOuXjAIXNA49hxM=.9eaa035b-939b-48cc-9d62-f2dd4426efb6@github.com>
On Thu, 16 Mar 2023 10:01:42 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> Some RTM locking tests are currently failing:
>
> compiler/rtm/cli/TestUseRTMLockingOptionWithBiasedLocking.java
> compiler/rtm/locking/TestRTMRetryCount.java
> compiler/rtm/locking/TestRTMTotalCountIncrRate.java
> compiler/rtm/locking/TestUseRTMAfterLockInflation.java
> compiler/rtm/locking/TestUseRTMForInflatedLocks.java
> compiler/rtm/locking/TestUseRTMForStackLocks.java
> compiler/rtm/method_options/TestUseRTMLockElidingOption.java
>
> Most are because there's a jmpb in UseRTMLocking parts that fails because of longer locking code with Lilliput. One test is failing because it checks a warning message when somebody tries +UseBiasedLocking together with RTM locking, but Lilliput forces biased locking to be off, so the warning never appears. I problem-listed that test.
>
> Testing:
> - [x] all tests listed above
This pull request has now been integrated.
Changeset: 1a70eb69
Author: Roman Kennke <rkennke at openjdk.org>
URL: https://git.openjdk.org/lilliput-jdk17u/commit/1a70eb696f48ecc2242becc47f4b9e86ceadcdb0
Stats: 4 lines in 2 files changed: 2 ins; 0 del; 2 mod
8304331: [Lilliput/JDK17] Fix RTM locking
Reviewed-by: shade
-------------
PR: https://git.openjdk.org/lilliput-jdk17u/pull/12
From rkennke at openjdk.org Wed Apr 12 18:19:12 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Wed, 12 Apr 2023 18:19:12 GMT
Subject: [master] RFR: 8305926: [Lilliput] Revert some ZGC changes
Message-ID: <kLUATsa9kgz7GGq_GCUuj1_wlGl2pBeLFRq1-tN1F2s=.1fb87286-b5ec-4ef8-b905-266de267e3bd@github.com>
I'm cleaning up some stuff and found these changes in ZGC that we don't need anymore.
Testing:
- [x] hotspot_gc
- [x] tier1
- [x] tier2
-------------
Commit messages:
- 8305926: [Lilliput] Revert some ZGC changes
Changes: https://git.openjdk.org/lilliput/pull/84/files
Webrev: https://webrevs.openjdk.org/?repo=lilliput&pr=84&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8305926
Stats: 5 lines in 1 file changed: 4 ins; 0 del; 1 mod
Patch: https://git.openjdk.org/lilliput/pull/84.diff
Fetch: git fetch https://git.openjdk.org/lilliput.git pull/84/head:pull/84
PR: https://git.openjdk.org/lilliput/pull/84
From shade at openjdk.org Wed Apr 12 18:26:05 2023
From: shade at openjdk.org (Aleksey Shipilev)
Date: Wed, 12 Apr 2023 18:26:05 GMT
Subject: [master] RFR: 8305926: [Lilliput] Revert some ZGC changes
In-Reply-To: <kLUATsa9kgz7GGq_GCUuj1_wlGl2pBeLFRq1-tN1F2s=.1fb87286-b5ec-4ef8-b905-266de267e3bd@github.com>
References: <kLUATsa9kgz7GGq_GCUuj1_wlGl2pBeLFRq1-tN1F2s=.1fb87286-b5ec-4ef8-b905-266de267e3bd@github.com>
Message-ID: <Mqoxgwtur3-m29XCpYAo6o4TvEeZiy7Zz3watKBIlpQ=.4f5306d8-0d27-4ba2-a73e-e45af0774003@github.com>
On Wed, 12 Apr 2023 18:12:26 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> I'm cleaning up some stuff and found these changes in ZGC that we don't need anymore.
>
> Testing:
> - [x] hotspot_gc
> - [x] tier1
> - [x] tier2
This looks fine.
-------------
Marked as reviewed by shade (Committer).
PR Review: https://git.openjdk.org/lilliput/pull/84#pullrequestreview-1381875827
From rkennke at openjdk.org Wed Apr 12 19:01:01 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Wed, 12 Apr 2023 19:01:01 GMT
Subject: [master] RFR: 8304710: [Lilliput] Use forwarding table for sliding GCs
Message-ID: <e0ORe1LYxeyGmJed6AI02mAHy48K-0HlSDXwJRFOhrg=.6f327c58-5897-4cb9-82af-433f04ce0fd9@github.com>
This changes the full-GC forwarding of Serial, Parallel, G1 and Shenandoah to use a forwarding table instead of the fancy SlidingForwarding that we had until now. SlidingForwarding has been problematic because it did not work universally, in particular we needed to disable the G1 serial full GC (the last last ditch GC, after parallel full GC failed). Also, sliding forwarding would not work when we start working towards 32bit headers.
The implementation of the forwarding table is a dense array of forwardings, with sorted entries and binary search. For details, have a look at the comments at the top of class ForwardingTable.
The change also greatly reduces the upstream diff.
Testing:
- [x] tier1
- [x] tier2
-------------
Commit messages:
- Add/fix comments
- Add GTest
- Various fixes and improvements
- Support G1 serial compaction
- Prototype to use region-based binary-search forwarding table
- 8304710: [Lilliput] Use forwarding table for sliding GCs
Changes: https://git.openjdk.org/lilliput/pull/83/files
Webrev: https://webrevs.openjdk.org/?repo=lilliput&pr=83&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8304710
Stats: 1212 lines in 44 files changed: 678 ins; 390 del; 144 mod
Patch: https://git.openjdk.org/lilliput/pull/83.diff
Fetch: git fetch https://git.openjdk.org/lilliput.git pull/83/head:pull/83
PR: https://git.openjdk.org/lilliput/pull/83
From rkennke at openjdk.org Wed Apr 12 19:01:57 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Wed, 12 Apr 2023 19:01:57 GMT
Subject: [master] Integrated: 8305926: [Lilliput] Revert some ZGC changes
In-Reply-To: <kLUATsa9kgz7GGq_GCUuj1_wlGl2pBeLFRq1-tN1F2s=.1fb87286-b5ec-4ef8-b905-266de267e3bd@github.com>
References: <kLUATsa9kgz7GGq_GCUuj1_wlGl2pBeLFRq1-tN1F2s=.1fb87286-b5ec-4ef8-b905-266de267e3bd@github.com>
Message-ID: <Gwq3dwG0oeoH00XIRKDKK_z2acixB_4rXJSl4uA_DbE=.63fd1355-636c-4986-ba54-5b0094ce73a7@github.com>
On Wed, 12 Apr 2023 18:12:26 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> I'm cleaning up some stuff and found these changes in ZGC that we don't need anymore.
>
> Testing:
> - [x] hotspot_gc
> - [x] tier1
> - [x] tier2
This pull request has now been integrated.
Changeset: 40a549b3
Author: Roman Kennke <rkennke at openjdk.org>
URL: https://git.openjdk.org/lilliput/commit/40a549b3eea81172aeac56725c6335a5450bd5ba
Stats: 5 lines in 1 file changed: 4 ins; 0 del; 1 mod
8305926: [Lilliput] Revert some ZGC changes
Reviewed-by: shade
-------------
PR: https://git.openjdk.org/lilliput/pull/84
From rkennke at openjdk.org Thu Apr 13 08:40:10 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 13 Apr 2023 08:40:10 GMT
Subject: [master] RFR: 8304710: [Lilliput] Use forwarding table for sliding
GCs [v2]
In-Reply-To: <e0ORe1LYxeyGmJed6AI02mAHy48K-0HlSDXwJRFOhrg=.6f327c58-5897-4cb9-82af-433f04ce0fd9@github.com>
References: <e0ORe1LYxeyGmJed6AI02mAHy48K-0HlSDXwJRFOhrg=.6f327c58-5897-4cb9-82af-433f04ce0fd9@github.com>
Message-ID: <H9mDgHQz8VX9z-s_byiobp9QCezJxYRiYOH5jSrkykQ=.e8929182-85ed-4855-a98f-85c01f1e223b@github.com>
> This changes the full-GC forwarding of Serial, Parallel, G1 and Shenandoah to use a forwarding table instead of the fancy SlidingForwarding that we had until now. SlidingForwarding has been problematic because it did not work universally, in particular we needed to disable the G1 serial full GC (the last last ditch GC, after parallel full GC failed). Also, sliding forwarding would not work when we start working towards 32bit headers.
>
> The implementation of the forwarding table is a dense array of forwardings, with sorted entries and binary search. For details, have a look at the comments at the top of class ForwardingTable.
>
> The change also greatly reduces the upstream diff.
>
> Testing:
> - [x] tier1
> - [x] tier2
Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
- Track old and young separately in serial GC
- Fix test
-------------
Changes:
- all: https://git.openjdk.org/lilliput/pull/83/files
- new: https://git.openjdk.org/lilliput/pull/83/files/f1b694f4..1c811a65
Webrevs:
- full: https://webrevs.openjdk.org/?repo=lilliput&pr=83&range=01
- incr: https://webrevs.openjdk.org/?repo=lilliput&pr=83&range=00-01
Stats: 70 lines in 6 files changed: 13 ins; 0 del; 57 mod
Patch: https://git.openjdk.org/lilliput/pull/83.diff
Fetch: git fetch https://git.openjdk.org/lilliput.git pull/83/head:pull/83
PR: https://git.openjdk.org/lilliput/pull/83
From rkennke at openjdk.org Thu Apr 13 12:03:39 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Thu, 13 Apr 2023 12:03:39 GMT
Subject: [master] RFR: 8304710: [Lilliput] Use forwarding table for sliding
GCs [v3]
In-Reply-To: <e0ORe1LYxeyGmJed6AI02mAHy48K-0HlSDXwJRFOhrg=.6f327c58-5897-4cb9-82af-433f04ce0fd9@github.com>
References: <e0ORe1LYxeyGmJed6AI02mAHy48K-0HlSDXwJRFOhrg=.6f327c58-5897-4cb9-82af-433f04ce0fd9@github.com>
Message-ID: <QTocAxV2xndfLrFAJrN00oTyXk_czzUxMJJCRhsg75E=.30b2429a-4113-47cc-b5df-5bd28fe198b0@github.com>
> This changes the full-GC forwarding of Serial, Parallel, G1 and Shenandoah to use a forwarding table instead of the fancy SlidingForwarding that we had until now. SlidingForwarding has been problematic because it did not work universally, in particular we needed to disable the G1 serial full GC (the last last ditch GC, after parallel full GC failed). Also, sliding forwarding would not work when we start working towards 32bit headers.
>
> The implementation of the forwarding table is a dense array of forwardings, with sorted entries and binary search. For details, have a look at the comments at the top of class ForwardingTable.
>
> The change also greatly reduces the upstream diff.
>
> In G1 I added a little code to count the marked objects in each region by counting bits in the mark-bitmap.
>
> In Serial GC I likewise added some code to count new vs old objects during full-GC marking. Here there would be the alternative to treat the heap as a single region, and count marked objects without distinguishing old vs young. The problem is that old is allocated in higher memory than young, and the full-GC compacts old-gen first, and then young-gen, which breaks the assumption in ForwardingTable, which leads to very slow forwarding insertions. I also tried to simply swap the old and young reserved memory, and that is possible, but triggers all sorts of related changes where the memory order is implicitely used for stuff like is_in_young() etc.
>
> Testing:
> - [x] tier1
> - [x] tier2
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
Ensure compaction order in serial GC
-------------
Changes:
- all: https://git.openjdk.org/lilliput/pull/83/files
- new: https://git.openjdk.org/lilliput/pull/83/files/1c811a65..40d889bc
Webrevs:
- full: https://webrevs.openjdk.org/?repo=lilliput&pr=83&range=02
- incr: https://webrevs.openjdk.org/?repo=lilliput&pr=83&range=01-02
Stats: 11 lines in 2 files changed: 9 ins; 0 del; 2 mod
Patch: https://git.openjdk.org/lilliput/pull/83.diff
Fetch: git fetch https://git.openjdk.org/lilliput.git pull/83/head:pull/83
PR: https://git.openjdk.org/lilliput/pull/83
From shade at openjdk.org Thu Apr 13 19:47:58 2023
From: shade at openjdk.org (Aleksey Shipilev)
Date: Thu, 13 Apr 2023 19:47:58 GMT
Subject: [master] RFR: 8304710: [Lilliput] Use forwarding table for sliding
GCs [v3]
In-Reply-To: <QTocAxV2xndfLrFAJrN00oTyXk_czzUxMJJCRhsg75E=.30b2429a-4113-47cc-b5df-5bd28fe198b0@github.com>
References: <e0ORe1LYxeyGmJed6AI02mAHy48K-0HlSDXwJRFOhrg=.6f327c58-5897-4cb9-82af-433f04ce0fd9@github.com>
<QTocAxV2xndfLrFAJrN00oTyXk_czzUxMJJCRhsg75E=.30b2429a-4113-47cc-b5df-5bd28fe198b0@github.com>
Message-ID: <F3X1vMN42-5dxxwxTYBajTXnVzHOajoyuWhY3unqcgk=.d5485cc2-08cd-4fc6-be57-0b78fa9fad56@github.com>
On Thu, 13 Apr 2023 12:03:39 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> This changes the full-GC forwarding of Serial, Parallel, G1 and Shenandoah to use a forwarding table instead of the fancy SlidingForwarding that we had until now. SlidingForwarding has been problematic because it did not work universally, in particular we needed to disable the G1 serial full GC (the last last ditch GC, after parallel full GC failed). Also, sliding forwarding would not work when we start working towards 32bit headers.
>>
>> The implementation of the forwarding table is a dense array of forwardings, with sorted entries and binary search. For details, have a look at the comments at the top of class ForwardingTable.
>>
>> The change also greatly reduces the upstream diff.
>>
>> In G1 I added a little code to count the marked objects in each region by counting bits in the mark-bitmap.
>>
>> In Serial GC I likewise added some code to count new vs old objects during full-GC marking. Here there would be the alternative to treat the heap as a single region, and count marked objects without distinguishing old vs young. The problem is that old is allocated in higher memory than young, and the full-GC compacts old-gen first, and then young-gen, which breaks the assumption in ForwardingTable, which leads to very slow forwarding insertions. I also tried to simply swap the old and young reserved memory, and that is possible, but triggers all sorts of related changes where the memory order is implicitely used for stuff like is_in_young() etc.
>>
>> Testing:
>> - [x] tier1
>> - [x] tier2
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>
> Ensure compaction order in serial GC
Cursory review. I have not got into the thick of how the forwarding tables are working, I'll do it in the second round.
src/hotspot/share/gc/serial/markSweep.cpp line 211:
> 209: } else {
> 210: _old_marked_objects++;
> 211: }
Is this supposed to be thread-safe? Or we don't care, because it is Serial?
src/hotspot/share/gc/shared/forwardingTable.cpp line 1:
> 1:
Critical bug: excess newline.
src/hotspot/share/gc/shared/forwardingTable.cpp line 57:
> 55: }
> 56:
> 57: void ForwardingTable::begin_region(size_t idx, size_t num_forwardings) {
Should assert `idx < _max_regions` here?
src/hotspot/share/gc/shared/forwardingTable.hpp line 34:
> 32: // TODO: Address range and num-regions permitting, we could switch to
> 33: // a more compact encoding:
> 34: // - N bits to encode number of words from start of from region
Suggestion:
// - N bits to encode number of words from start of from-region
src/hotspot/share/gc/shared/forwardingTable.hpp line 37:
> 35: // - M bits to encode number of words from start of to-region
> 36: // - X bits to encode to-region index
> 37: // from-region index is known implicitely.
Suggestion:
// from-region index is known implicitly.
src/hotspot/share/gc/shared/forwardingTable.hpp line 107:
> 105: * begin_region(). This initializes the corresponding per-region table.
> 106: * - Insert and look-up forwardings.
> 107: * - When forwaring is finished, call end(). This will dispose all internal data structures.
Suggestion:
* - When forwarding is finished, call end(). This will dispose all internal data structures.
src/hotspot/share/gc/shared/forwardingTable.hpp line 127:
> 125: };
> 126:
> 127: #endif // SHARE_GC_SHARED_FORWARDINGABLE_HPP
Suggestion:
#endif // SHARE_GC_SHARED_FORWARDINGTABLE_HPP
src/hotspot/share/gc/shared/forwardingTable.inline.hpp line 37:
> 35: }
> 36:
> 37: // When found, returns theindex into _table
Suggestion:
// When found, returns the index into _table
src/hotspot/share/gc/shared/gcForwarding.hpp line 40:
> 38: public:
> 39:
> 40: static void initialize(AddrToIdxFn addr_to_idx, size_t max_regions);
Suggestion:
private:
static ForwardingTable* _forwarding_table;
public:
static void initialize(AddrToIdxFn addr_to_idx, size_t max_regions);
src/hotspot/share/gc/shared/gcForwarding.hpp line 45:
> 43: static void end();
> 44:
> 45: static inline bool is_forwarded(oop obj);
I'd suggest additional `is_not_forwarded`, so that we don't have to invert `!` and `!=` on some paths.
src/hotspot/share/gc/shared/gcForwarding.inline.hpp line 54:
> 52: assert(_forwarding_table != nullptr, "expect forwarding table initialized");
> 53: _forwarding_table->forward_to(obj, fwd);
> 54: assert(is_forwarded(obj), "must be forwarded");
This assert looks redundant, since the next one would test it implicitly.
src/hotspot/share/gc/shared/genCollectedHeap.cpp line 108:
> 106: auto addr_to_idx = [](const void* addr) {
> 107: if (GenCollectedHeap::heap()->is_in_young(addr)) return (size_t)0;
> 108: else return (size_t)1;
Suggestion:
if (GenCollectedHeap::heap()->is_in_young(addr)) {
return (size_t)0;
} else {
return (size_t)1;
}
test/hotspot/gtest/gc/shared/test_preservedMarks.cpp line 59:
> 57:
> 58: bool old_compact_headers = UseCompactObjectHeaders;
> 59: UseCompactObjectHeaders = false;
FlagSetting fs(UseCompactObjectHeaders, false);
?
-------------
PR Review: https://git.openjdk.org/lilliput/pull/83#pullrequestreview-1383531256
PR Review Comment: https://git.openjdk.org/lilliput/pull/83#discussion_r1165599869
PR Review Comment: https://git.openjdk.org/lilliput/pull/83#discussion_r1165603072
PR Review Comment: https://git.openjdk.org/lilliput/pull/83#discussion_r1165666592
PR Review Comment: https://git.openjdk.org/lilliput/pull/83#discussion_r1165640939
PR Review Comment: https://git.openjdk.org/lilliput/pull/83#discussion_r1165641371
PR Review Comment: https://git.openjdk.org/lilliput/pull/83#discussion_r1165643213
PR Review Comment: https://git.openjdk.org/lilliput/pull/83#discussion_r1165672770
PR Review Comment: https://git.openjdk.org/lilliput/pull/83#discussion_r1165643947
PR Review Comment: https://git.openjdk.org/lilliput/pull/83#discussion_r1165658239
PR Review Comment: https://git.openjdk.org/lilliput/pull/83#discussion_r1165659916
PR Review Comment: https://git.openjdk.org/lilliput/pull/83#discussion_r1165661823
PR Review Comment: https://git.openjdk.org/lilliput/pull/83#discussion_r1165662664
PR Review Comment: https://git.openjdk.org/lilliput/pull/83#discussion_r1165782919
From rkennke at openjdk.org Fri Apr 14 09:38:04 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Fri, 14 Apr 2023 09:38:04 GMT
Subject: [master] RFR: 8304710: [Lilliput] Use forwarding table for sliding
GCs [v3]
In-Reply-To: <F3X1vMN42-5dxxwxTYBajTXnVzHOajoyuWhY3unqcgk=.d5485cc2-08cd-4fc6-be57-0b78fa9fad56@github.com>
References: <e0ORe1LYxeyGmJed6AI02mAHy48K-0HlSDXwJRFOhrg=.6f327c58-5897-4cb9-82af-433f04ce0fd9@github.com>
<QTocAxV2xndfLrFAJrN00oTyXk_czzUxMJJCRhsg75E=.30b2429a-4113-47cc-b5df-5bd28fe198b0@github.com>
<F3X1vMN42-5dxxwxTYBajTXnVzHOajoyuWhY3unqcgk=.d5485cc2-08cd-4fc6-be57-0b78fa9fad56@github.com>
Message-ID: <-VAUPqXmc5f1ATOz-p_dxzCPhW2QU1wk62_LCjQREQE=.e3cce818-4317-4128-a33c-5fee4250689f@github.com>
On Thu, 13 Apr 2023 14:18:56 GMT, Aleksey Shipilev <shade at openjdk.org> wrote:
>> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>>
>> Ensure compaction order in serial GC
>
> src/hotspot/share/gc/serial/markSweep.cpp line 211:
>
>> 209: } else {
>> 210: _old_marked_objects++;
>> 211: }
>
> Is this supposed to be thread-safe? Or we don't care, because it is Serial?
Marking is single-threaded in Serial Full GC. We should be good there.
-------------
PR Review Comment: https://git.openjdk.org/lilliput/pull/83#discussion_r1166593890
From rkennke at openjdk.org Fri Apr 14 10:59:10 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Fri, 14 Apr 2023 10:59:10 GMT
Subject: [master] RFR: 8304710: [Lilliput] Use forwarding table for sliding
GCs [v4]
In-Reply-To: <e0ORe1LYxeyGmJed6AI02mAHy48K-0HlSDXwJRFOhrg=.6f327c58-5897-4cb9-82af-433f04ce0fd9@github.com>
References: <e0ORe1LYxeyGmJed6AI02mAHy48K-0HlSDXwJRFOhrg=.6f327c58-5897-4cb9-82af-433f04ce0fd9@github.com>
Message-ID: <lSo4eOgyuX_1Pvv1JBaVaGamD2UviyOIwxp1Z-mxkcI=.7ba0f75e-96af-4552-9daa-9ea5b53adfa4@github.com>
> This changes the full-GC forwarding of Serial, Parallel, G1 and Shenandoah to use a forwarding table instead of the fancy SlidingForwarding that we had until now. SlidingForwarding has been problematic because it did not work universally, in particular we needed to disable the G1 serial full GC (the last last ditch GC, after parallel full GC failed). Also, sliding forwarding would not work when we start working towards 32bit headers.
>
> The implementation of the forwarding table is a dense array of forwardings, with sorted entries and binary search. For details, have a look at the comments at the top of class ForwardingTable.
>
> The change also greatly reduces the upstream diff.
>
> In G1 I added a little code to count the marked objects in each region by counting bits in the mark-bitmap.
>
> In Serial GC I likewise added some code to count new vs old objects during full-GC marking. Here there would be the alternative to treat the heap as a single region, and count marked objects without distinguishing old vs young. The problem is that old is allocated in higher memory than young, and the full-GC compacts old-gen first, and then young-gen, which breaks the assumption in ForwardingTable, which leads to very slow forwarding insertions. I also tried to simply swap the old and young reserved memory, and that is possible, but triggers all sorts of related changes where the memory order is implicitely used for stuff like is_in_young() etc.
>
> Testing:
> - [x] tier1
> - [x] tier2
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
Aleksey's review
-------------
Changes:
- all: https://git.openjdk.org/lilliput/pull/83/files
- new: https://git.openjdk.org/lilliput/pull/83/files/40d889bc..74d6483a
Webrevs:
- full: https://webrevs.openjdk.org/?repo=lilliput&pr=83&range=03
- incr: https://webrevs.openjdk.org/?repo=lilliput&pr=83&range=02-03
Stats: 33 lines in 10 files changed: 15 ins; 6 del; 12 mod
Patch: https://git.openjdk.org/lilliput/pull/83.diff
Fetch: git fetch https://git.openjdk.org/lilliput.git pull/83/head:pull/83
PR: https://git.openjdk.org/lilliput/pull/83
From rkennke at openjdk.org Mon Apr 17 18:50:12 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 17 Apr 2023 18:50:12 GMT
Subject: [master] RFR: 8304710: [Lilliput] Use forwarding table for sliding
GCs [v5]
In-Reply-To: <e0ORe1LYxeyGmJed6AI02mAHy48K-0HlSDXwJRFOhrg=.6f327c58-5897-4cb9-82af-433f04ce0fd9@github.com>
References: <e0ORe1LYxeyGmJed6AI02mAHy48K-0HlSDXwJRFOhrg=.6f327c58-5897-4cb9-82af-433f04ce0fd9@github.com>
Message-ID: <DLMMBDL7QOjQm3WMUYidSdusAnASlMe347ujQ8MPotM=.4042cd1b-fd1b-4d49-b830-dab67be1487d@github.com>
> This changes the full-GC forwarding of Serial, Parallel, G1 and Shenandoah to use a forwarding table instead of the fancy SlidingForwarding that we had until now. SlidingForwarding has been problematic because it did not work universally, in particular we needed to disable the G1 serial full GC (the last last ditch GC, after parallel full GC failed). Also, sliding forwarding would not work when we start working towards 32bit headers.
>
> The implementation of the forwarding table is a dense array of forwardings, with sorted entries and binary search. For details, have a look at the comments at the top of class ForwardingTable.
>
> The change also greatly reduces the upstream diff.
>
> In G1 I added a little code to count the marked objects in each region by counting bits in the mark-bitmap.
>
> In Serial GC I likewise added some code to count new vs old objects during full-GC marking. Here there would be the alternative to treat the heap as a single region, and count marked objects without distinguishing old vs young. The problem is that old is allocated in higher memory than young, and the full-GC compacts old-gen first, and then young-gen, which breaks the assumption in ForwardingTable, which leads to very slow forwarding insertions. I also tried to simply swap the old and young reserved memory, and that is possible, but triggers all sorts of related changes where the memory order is implicitely used for stuff like is_in_young() etc.
>
> Testing:
> - [x] tier1
> - [x] tier2
Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
- Hashtable with Robin Hood hashing
- Use hashtable
-------------
Changes:
- all: https://git.openjdk.org/lilliput/pull/83/files
- new: https://git.openjdk.org/lilliput/pull/83/files/74d6483a..b65e589f
Webrevs:
- full: https://webrevs.openjdk.org/?repo=lilliput&pr=83&range=04
- incr: https://webrevs.openjdk.org/?repo=lilliput&pr=83&range=03-04
Stats: 188 lines in 4 files changed: 118 ins; 41 del; 29 mod
Patch: https://git.openjdk.org/lilliput/pull/83.diff
Fetch: git fetch https://git.openjdk.org/lilliput.git pull/83/head:pull/83
PR: https://git.openjdk.org/lilliput/pull/83
From rkennke at openjdk.org Mon Apr 17 19:05:52 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Mon, 17 Apr 2023 19:05:52 GMT
Subject: [master] RFR: 8306122: [Lilliput] Refactor full GC forwarding
Message-ID: <mXMEQRnJXkccb3cunMGWPhOsfZ7oJF8vW12AnvWxmgI=.607d942e-b29f-43b4-b22f-53c1f8db6f0f@github.com>
Current implementation of full-GC/sliding-GC forwarding in Lilliput is problematic:
- It's scattered and hardwired all over GC implementations
- It's not switched by UseCompactObjectMonitors
- The serial compaction in G1 is commented-out and not used, not even under -XX:-UseCompactObjectMonitors
This change aims to refactor the forwarding code to address these issues.
It should be functionally equivalent but provides cleaner separation of Lilliput changes. It also cleans up and greatly reduces the upstream diff. The lack of G1 serial compaction with Lilliput enabled will be addressed in a follow-up change.
Testing:
- [x] hotspot_gc
- [ ] tier1
- [ ] tier2
-------------
Commit messages:
- Remove forwardingTable* stuff
- 8306122: [Lilliput] Refactor full GC forwarding
- Hashtable with Robin Hood hashing
- Use hashtable
- Aleksey's review
- Ensure compaction order in serial GC
- Track old and young separately in serial GC
- Fix test
- Add/fix comments
- Add GTest
- ... and 4 more: https://git.openjdk.org/lilliput/compare/88b34412...d2c6bed7
Changes: https://git.openjdk.org/lilliput/pull/85/files
Webrev: https://webrevs.openjdk.org/?repo=lilliput&pr=85&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8306122
Stats: 500 lines in 38 files changed: 259 ins; 96 del; 145 mod
Patch: https://git.openjdk.org/lilliput/pull/85.diff
Fetch: git fetch https://git.openjdk.org/lilliput.git pull/85/head:pull/85
PR: https://git.openjdk.org/lilliput/pull/85
From shade at openjdk.org Tue Apr 18 10:28:15 2023
From: shade at openjdk.org (Aleksey Shipilev)
Date: Tue, 18 Apr 2023 10:28:15 GMT
Subject: [master] RFR: 8306122: [Lilliput] Refactor full GC forwarding
In-Reply-To: <mXMEQRnJXkccb3cunMGWPhOsfZ7oJF8vW12AnvWxmgI=.607d942e-b29f-43b4-b22f-53c1f8db6f0f@github.com>
References: <mXMEQRnJXkccb3cunMGWPhOsfZ7oJF8vW12AnvWxmgI=.607d942e-b29f-43b4-b22f-53c1f8db6f0f@github.com>
Message-ID: <MYoXlkBqMk8hY1zQeIFzHO89Ucm-0Z9If0U7qA8_sIQ=.a713a266-fa39-4399-8c2c-7980468b0e2f@github.com>
On Mon, 17 Apr 2023 18:50:16 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> Current implementation of full-GC/sliding-GC forwarding in Lilliput is problematic:
> - It's scattered and hardwired all over GC implementations
> - It's not switched by UseCompactObjectMonitors
> - The serial compaction in G1 is commented-out and not used, not even under -XX:-UseCompactObjectMonitors
>
> This change aims to refactor the forwarding code to address these issues.
>
> It should be functionally equivalent but provides cleaner separation of Lilliput changes. It also cleans up and greatly reduces the upstream diff. The lack of G1 serial compaction with Lilliput enabled will be addressed in a follow-up change.
>
> Testing:
> - [x] hotspot_gc
> - [x] tier1
> - [ ] tier2
Looks fine for Lilliput
-------------
Marked as reviewed by shade (Committer).
PR Review: https://git.openjdk.org/lilliput/pull/85#pullrequestreview-1389794687
From rkennke at openjdk.org Tue Apr 18 13:45:16 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 18 Apr 2023 13:45:16 GMT
Subject: [master] RFR: 8306122: [Lilliput] Refactor full GC forwarding [v2]
In-Reply-To: <mXMEQRnJXkccb3cunMGWPhOsfZ7oJF8vW12AnvWxmgI=.607d942e-b29f-43b4-b22f-53c1f8db6f0f@github.com>
References: <mXMEQRnJXkccb3cunMGWPhOsfZ7oJF8vW12AnvWxmgI=.607d942e-b29f-43b4-b22f-53c1f8db6f0f@github.com>
Message-ID: <LS0jbFuG_DkrQopUYZRY0N2WtN1hivgaBdOxl1S29V8=.b396088e-c172-41cd-8428-59ed9d16a356@github.com>
> Current implementation of full-GC/sliding-GC forwarding in Lilliput is problematic:
> - It's scattered and hardwired all over GC implementations
> - It's not switched by UseCompactObjectMonitors
> - The serial compaction in G1 is commented-out and not used, not even under -XX:-UseCompactObjectMonitors
>
> This change aims to refactor the forwarding code to address these issues.
>
> It should be functionally equivalent but provides cleaner separation of Lilliput changes. It also cleans up and greatly reduces the upstream diff. The lack of G1 serial compaction with Lilliput enabled will be addressed in a follow-up change.
>
> Testing:
> - [x] hotspot_gc
> - [x] tier1
> - [ ] tier2
Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
32 bit build fix
-------------
Changes:
- all: https://git.openjdk.org/lilliput/pull/85/files
- new: https://git.openjdk.org/lilliput/pull/85/files/d2c6bed7..e4a81227
Webrevs:
- full: https://webrevs.openjdk.org/?repo=lilliput&pr=85&range=01
- incr: https://webrevs.openjdk.org/?repo=lilliput&pr=85&range=00-01
Stats: 2 lines in 1 file changed: 2 ins; 0 del; 0 mod
Patch: https://git.openjdk.org/lilliput/pull/85.diff
Fetch: git fetch https://git.openjdk.org/lilliput.git pull/85/head:pull/85
PR: https://git.openjdk.org/lilliput/pull/85
From rkennke at openjdk.org Tue Apr 18 16:58:21 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 18 Apr 2023 16:58:21 GMT
Subject: [master] RFR: 8306122: [Lilliput] Refactor full GC forwarding [v2]
In-Reply-To: <LS0jbFuG_DkrQopUYZRY0N2WtN1hivgaBdOxl1S29V8=.b396088e-c172-41cd-8428-59ed9d16a356@github.com>
References: <mXMEQRnJXkccb3cunMGWPhOsfZ7oJF8vW12AnvWxmgI=.607d942e-b29f-43b4-b22f-53c1f8db6f0f@github.com>
<LS0jbFuG_DkrQopUYZRY0N2WtN1hivgaBdOxl1S29V8=.b396088e-c172-41cd-8428-59ed9d16a356@github.com>
Message-ID: <M5g0VSFNjO6fTc-oKuB-ePezKFwNkeWCnbX_1MVcXNs=.c5c6c61c-05a5-4d3a-8496-675ec87330a3@github.com>
On Tue, 18 Apr 2023 13:45:16 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> Current implementation of full-GC/sliding-GC forwarding in Lilliput is problematic:
>> - It's scattered and hardwired all over GC implementations
>> - It's not switched by UseCompactObjectMonitors
>> - The serial compaction in G1 is commented-out and not used, not even under -XX:-UseCompactObjectMonitors
>>
>> This change aims to refactor the forwarding code to address these issues.
>>
>> It should be functionally equivalent but provides cleaner separation of Lilliput changes. It also cleans up and greatly reduces the upstream diff. The lack of G1 serial compaction with Lilliput enabled will be addressed in a follow-up change.
>>
>> Testing:
>> - [x] hotspot_gc
>> - [x] tier1
>> - [x] tier2
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
>
> 32 bit build fix
Thanks!
-------------
PR Comment: https://git.openjdk.org/lilliput/pull/85#issuecomment-1513500313
From rkennke at openjdk.org Tue Apr 18 17:01:14 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Tue, 18 Apr 2023 17:01:14 GMT
Subject: [master] Integrated: 8306122: [Lilliput] Refactor full GC forwarding
In-Reply-To: <mXMEQRnJXkccb3cunMGWPhOsfZ7oJF8vW12AnvWxmgI=.607d942e-b29f-43b4-b22f-53c1f8db6f0f@github.com>
References: <mXMEQRnJXkccb3cunMGWPhOsfZ7oJF8vW12AnvWxmgI=.607d942e-b29f-43b4-b22f-53c1f8db6f0f@github.com>
Message-ID: <QYgWR6IgOFw4hAVzxFrK3dBsuZtCatXBldQ-_qdn6qQ=.c89bba31-b38b-4840-b1d4-5e537f426bdd@github.com>
On Mon, 17 Apr 2023 18:50:16 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
> Current implementation of full-GC/sliding-GC forwarding in Lilliput is problematic:
> - It's scattered and hardwired all over GC implementations
> - It's not switched by UseCompactObjectMonitors
> - The serial compaction in G1 is commented-out and not used, not even under -XX:-UseCompactObjectMonitors
>
> This change aims to refactor the forwarding code to address these issues.
>
> It should be functionally equivalent but provides cleaner separation of Lilliput changes. It also cleans up and greatly reduces the upstream diff. The lack of G1 serial compaction with Lilliput enabled will be addressed in a follow-up change.
>
> Testing:
> - [x] hotspot_gc
> - [x] tier1
> - [x] tier2
This pull request has now been integrated.
Changeset: e12ed8a2
Author: Roman Kennke <rkennke at openjdk.org>
URL: https://git.openjdk.org/lilliput/commit/e12ed8a27d06df123f3653a33a08cccc838a98d7
Stats: 502 lines in 38 files changed: 261 ins; 96 del; 145 mod
8306122: [Lilliput] Refactor full GC forwarding
Reviewed-by: shade
-------------
PR: https://git.openjdk.org/lilliput/pull/85
From stuefe at openjdk.org Wed Apr 19 15:04:38 2023
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Wed, 19 Apr 2023 15:04:38 GMT
Subject: [master] RFR: 8304710: [Lilliput] Use forwarding table for sliding
GCs [v5]
In-Reply-To: <DLMMBDL7QOjQm3WMUYidSdusAnASlMe347ujQ8MPotM=.4042cd1b-fd1b-4d49-b830-dab67be1487d@github.com>
References: <e0ORe1LYxeyGmJed6AI02mAHy48K-0HlSDXwJRFOhrg=.6f327c58-5897-4cb9-82af-433f04ce0fd9@github.com>
<DLMMBDL7QOjQm3WMUYidSdusAnASlMe347ujQ8MPotM=.4042cd1b-fd1b-4d49-b830-dab67be1487d@github.com>
Message-ID: <-LEHbeZkNGjp4WTXn71zuokDKIlCuXr7glcaf5w_IDM=.1e911f2e-912c-454a-a102-9866eeb461cc@github.com>
On Mon, 17 Apr 2023 18:50:12 GMT, Roman Kennke <rkennke at openjdk.org> wrote:
>> This changes the full-GC forwarding of Serial, Parallel, G1 and Shenandoah to use a forwarding table instead of the fancy SlidingForwarding that we had until now. SlidingForwarding has been problematic because it did not work universally, in particular we needed to disable the G1 serial full GC (the last last ditch GC, after parallel full GC failed). Also, sliding forwarding would not work when we start working towards 32bit headers.
>>
>> The implementation of the forwarding table is a dense array of forwardings, with sorted entries and binary search. For details, have a look at the comments at the top of class ForwardingTable.
>>
>> The change also greatly reduces the upstream diff.
>>
>> In G1 I added a little code to count the marked objects in each region by counting bits in the mark-bitmap.
>>
>> In Serial GC I likewise added some code to count new vs old objects during full-GC marking. Here there would be the alternative to treat the heap as a single region, and count marked objects without distinguishing old vs young. The problem is that old is allocated in higher memory than young, and the full-GC compacts old-gen first, and then young-gen, which breaks the assumption in ForwardingTable, which leads to very slow forwarding insertions. I also tried to simply swap the old and young reserved memory, and that is possible, but triggers all sorts of related changes where the memory order is implicitely used for stuff like is_in_young() etc.
>>
>> Testing:
>> - [x] tier1
>> - [x] tier2
>
> Roman Kennke has updated the pull request incrementally with two additional commits since the last revision:
>
> - Hashtable with Robin Hood hashing
> - Use hashtable
What would be the worst case native memory consumption?
-------------
PR Comment: https://git.openjdk.org/lilliput/pull/83#issuecomment-1514895369
From jrose at openjdk.org Wed Apr 19 17:02:45 2023
From: jrose at openjdk.org (John R Rose)
Date: Wed, 19 Apr 2023 17:02:45 GMT
Subject: [master] RFR: Smaller class pointers [v3]
In-Reply-To: <BKx_pu6sIdA2HLiG9PXN9UXyE50veuptMfAlJkQ30lk=.868e21be-7525-4dc8-8de8-70163bac1e2f@github.com>
References: <_9UGqIAQh1r1nnIjB3-jqX0qeg8ZshsjT65qoHrxZY8=.b47c567d-21f0-4a7c-960a-4dff79779c64@github.com>
<r51rqKPWeRBQgD63tPyLoHac9AcFRqXe9ADyrtYZea8=.202d91b5-2646-40f5-b381-75574e773460@github.com>
<BKx_pu6sIdA2HLiG9PXN9UXyE50veuptMfAlJkQ30lk=.868e21be-7525-4dc8-8de8-70163bac1e2f@github.com>
Message-ID: <ynaXK-gqGr0UohqTzZWXaRJOtE1NVwG8hlfpLPh64JI=.604304e9-0fe4-44fe-9b95-7e230c32d57a@github.com>
On Wed, 8 Dec 2021 18:47:47 GMT, John R Rose <jrose at openjdk.org> wrote:
>> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:
>>
>> feedback Ioi, fix CDS archive size estimate
>
> Also, FTR, there are other ways to keep cache density without the extra add/xor step of smearing M-shifted bits. It would require deeper cuts, but it is worth thinking again of the thing you already mentioned, where a Klass block is split into a fixed-sized "near Klass" and an indirectly addressed "far Klass". The near Klass would be sized as a small number of cache blocks, to keep Klass metadata dense in the data cache. There is a sensitive performance tradeoff, though, since you want parts of the Klass which are accessed frequently (dynamic type checks, vtables, GC metadata) to be in the near Klass. Since v-tables are variable in size, this means some v-table entries (probably) get favored over others. Tricky business. I think Coleen looked into this many years ago and may have further insights.
>
> Independently, a possible future benefit of a near-Klass/far-Klass split would be that we could consider building in a one-to-many relation across the linking indirection. That could support a tightly integrated "species" concept, where an object's header can point to a near-Klass which represents an individuated species of its class. In theory that could help represent specialized types like ArrayList<int> as distinct from the erased class ArrayList.
> Since I did not hear back from @rose00, @rkennke did ask me to speed this up and I have vacation looming, I decided to push. There is nothing we could not possibly revert later, but it gives us 22 bit class pointers for now and we can work with that.
FTR (16 months later!) my comments were future-oriented and I am happy with the progress this push made. Some of those comments may help reduce risk of class ID overflow later on.
I personally think it would be interesting to experiment with a version of this change which makes `MaxNarrowKlassPointerBits` a command-line configurable value, and then (as a follow-on) undertakes to deal with overflow by adding a second injected field (with a full 32 bits) to impacted layouts, but not to layouts of the ?favored? classes, which will usually be the majority.
The C++ access code would be slightly slower. And the extra injected field requires a usually-not-taken branch to a not-so-fast path that picks up the injected overflow bits.
Such an experiment would help make stress-tests (with very small MNKPB values). It could facilitate experiments with different header layouts, even 32-bit layouts, or maybe hoisting (some or all) klass bits into 64-bit oops.
-------------
PR Comment: https://git.openjdk.org/lilliput/pull/13#issuecomment-1515071487
From stuefe at openjdk.org Thu Apr 27 15:45:22 2023
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Thu, 27 Apr 2023 15:45:22 GMT
Subject: [master] RFR: JDK-8307002: [Lilliput] [Metaspace] Make use of
Klass alignment gaps
In-Reply-To: <5sLl4o4TcgXbG37aibf5ZDq0nJqQkRuyd3UXPXehLPQ=.a031f83d-f009-423b-8943-e2a2b7e0d2db@github.com>
References: <5sLl4o4TcgXbG37aibf5ZDq0nJqQkRuyd3UXPXehLPQ=.a031f83d-f009-423b-8943-e2a2b7e0d2db@github.com>
Message-ID: <f7B9urwZUGpqHRwAgyZJw-yhPSA1YJeuFReOYXXtg8Y=.69aa8901-75f4-4756-8424-7b43f568d3bf@github.com>
On Wed, 26 Apr 2023 15:59:34 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:
> https://github.com/openjdk/lilliput/pull/13 changed nKlass encoding by increasing the Klass alignment gap to 512. Works fine and gives us a nKlass-value-point-to-class ratio of about 1.5:1, which means we get close to the goal of representing each class with a single value of nKlass without changing Klass to a fixed-sized Layout. It has two disadvantages, though. The most obvious one - the memory waste caused by alignment shadows - is addressed by this issue.
>
> (The second disadvantage that I don't address in this issue is what John Rose called "naive overalignment", but it has effects on the patch, see below)
>
> That waste matters little for normal programs: the average waste is 256 bytes per class, and if Lilliput saves 4 bytes per object, we break even if we create more than 64 objects per class on average. We usually do that.
>
> But class-generation scenarios are a different matter. There, we often only have a single or very few objects per class, and many many more classes, so the numbers don't add up anymore. So we need a way to eliminate the gaps.
>
> The simple solution is to save non-class metadata from the same class loader into these gaps. This works fine since much of our non-class metadata is very fine granular. Like sand into cracks, they settle into the gaps and fill them nicely. And it is not even complicated since we already have the machinery (the descendant of Coleen's original dark-matter-prevention-structure that we use to manage prematurely deallocated metaspace).
>
> ----
>
> Numbers:
>
> Memory usage is almost back to baseline now. Volkers insane lambda generation test from JDK-8302154, loading 1 million lambdas, -Xshare:off, shows a reduction of 1.74GB->1.32GB, very close to the original 1.31GB baseline:
>
> (interesting because scenarios like these will be the worst case from a metaspace perspective - metaspace usage of normal apps is unexciting with or without Lilliput):
>
>
> Lilliput unpatched:
> Virtual space:
> Non-class space: 832,00 MB reserved, 807,12 MB ( 97%) committed, 13 nodes.
> Class space: 1,00 GB reserved, 977,31 MB ( 95%) committed, 1 nodes.
> Both: 1,81 GB reserved, 1,74 GB ( 96%) committed.
>
> Lilliput patched:
> Virtual space:
> Non-class space: 384,00 MB reserved, 370,75 MB ( 97%) committed, 6 nodes.
> Class space: 1,00 GB reserved, 977,31 MB ( 95%) committed, 1 nodes.
> Both: 1,38 GB reserved, 1,32 GB ( 96%) committed.
>
> Stock VM:
> Virtual space:
> Non-class space: 832,00 MB reserved, 822,44 MB ( 99%) committed, 13 nodes.
> Class space: 1,00 GB reserved, 521,31 MB ( 51%) committed, 1 nodes.
> Both: 1,81 GB reserved, 1,31 GB ( 72%) committed.
>
>
> ----
>
> Patch:
>
> https://github.com/openjdk/lilliput/pull/13 introduced the notion that every MetaspaceArena has an alignment - instead, as in the stock VM - working with one global alignment. This patch reverts much of this code (makes it easier to merge too). Instead, it treats alignment as an allocation option. An arena can satisfy allocations with different alignment requirements and will do its best to salvage the alignment waste.
>
> That is the core of the patch. The rest is fluff and tests.
>
> ---
>
> Notes:
>
> Numbers are somewhat fluid: at the moment, we use a Klass alignment of 512 and a nKlass size of 21 bits. We will probably change nKlass width to 24 or 26 bits for Lilliputs first milestone.
>
> And once this patch is in, the disadvantage of memory waste is eliminated, so we could increase the Klass alignment. Both 512 and 1K are good values: on average a Klass is between 400b...1K. A 1K alignment would spread out the nKlass value points more. Shift the nKlass-value-point-to-class ratio closer to 1:1.
>
> The still unsolved problem is naive overalignment. We may change the encoding scheme of Klass*->nKlass to take that into account. But not in this patch. But this uncertainty is the reason why I opted for a new method `MetaspaceArena::allocate_for_klass` instead of just giving the existing `Metaspace::allocate` method an alignment parameter.
src/hotspot/share/memory/metaspace/freeBlocks.cpp line 56:
> 54: if (p == nullptr) {
> 55: p = _tree.remove_block(requested_word_size, &real_size);
> 56: }
Reviewer info: this is a functional change, arguably a bug upstream:
`FreeBlocks` manages small blocks in a linear bin list (`_small_blacks`), larger blocks in a binary tree (`_tree`). Before, when trying to allocate a small block from `FreeBlocks`, we only would look into the bin list. But it makes sense to look into the large block tree too if that fails. Later down, we chop the block in two and save off the remainder anyway.
src/hotspot/share/memory/metaspace/metaspaceAlignment.hpp line 61:
> 59: word_size = align_up(word_size, metaspace::MetaspaceMinAlignmentWords);
> 60:
> 61: return word_size;
Reviewer note: both versions of `get_raw_word_size_for_requested_word_size` are equivalent save for the fact that the old one took alignment as a second input parameter.
test/hotspot/gtest/metaspace/test_metaspacearena_stress.cpp line 34:
> 32: #include "memory/metaspace/metaspaceSettings.hpp"
> 33: #include "memory/metaspace/metaspaceStatistics.hpp"
> 34: #include "runtime/mutexLocker.hpp"
Reviewer note: Since I removed the "alignment" property from MetaspaceArena, both `test_metaspacearena.cpp` and `test_metaspacearena_stress.cpp` had been reverted back to its upstream state; hence the seemingly random changes in those files.
-------------
PR Review Comment: https://git.openjdk.org/lilliput/pull/86#discussion_r1179172320
PR Review Comment: https://git.openjdk.org/lilliput/pull/86#discussion_r1179177259
PR Review Comment: https://git.openjdk.org/lilliput/pull/86#discussion_r1179343749
From stuefe at openjdk.org Thu Apr 27 15:42:53 2023
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Thu, 27 Apr 2023 15:42:53 GMT
Subject: [master] RFR: JDK-8307002: [Lilliput] [Metaspace] Make use of Klass
alignment gaps
Message-ID: <5sLl4o4TcgXbG37aibf5ZDq0nJqQkRuyd3UXPXehLPQ=.a031f83d-f009-423b-8943-e2a2b7e0d2db@github.com>
https://github.com/openjdk/lilliput/pull/13 changed nKlass encoding by increasing the Klass alignment gap to 512. Works fine and gives us a nKlass-value-point-to-class ratio of about 1.5:1, which means we get close to the goal of representing each class with a single value of nKlass without changing Klass to a fixed-sized Layout. It has two disadvantages, though. The most obvious one - the memory waste caused by alignment shadows - is addressed by this issue.
(The second disadvantage that I don't address in this issue is what John Rose called "naive overalignment", but it has effects on the patch, see below)
That waste matters little for normal programs: the average waste is 256 bytes per class, and if Lilliput saves 4 bytes per object, we break even if we create more than 64 objects per class on average. We usually do that.
But class-generation scenarios are a different matter. There, we often only have a single or very few objects per class, and many many more classes, so the numbers don't add up anymore. So we need a way to eliminate the gaps.
The simple solution is to save non-class metadata from the same class loader into these gaps. This works fine since much of our non-class metadata is very fine granular. Like sand into cracks, they settle into the gaps and fill them nicely. And it is not even complicated since we already have the machinery (the descendant of Coleen's original dark-matter-prevention-structure that we use to manage prematurely deallocated metaspace).
----
Numbers:
Memory usage is almost back to baseline now. Volkers insane lambda generation test from JDK-8302154, loading 1 million lambdas, -Xshare:off, shows a reduction of 1.74GB->1.32GB, very close to the original 1.31GB baseline:
(interesting because scenarios like these will be the worst case from a metaspace perspective - metaspace usage of normal apps is unexciting with or without Lilliput):
Lilliput unpatched:
Virtual space:
Non-class space: 832,00 MB reserved, 807,12 MB ( 97%) committed, 13 nodes.
Class space: 1,00 GB reserved, 977,31 MB ( 95%) committed, 1 nodes.
Both: 1,81 GB reserved, 1,74 GB ( 96%) committed.
Lilliput patched:
Virtual space:
Non-class space: 384,00 MB reserved, 370,75 MB ( 97%) committed, 6 nodes.
Class space: 1,00 GB reserved, 977,31 MB ( 95%) committed, 1 nodes.
Both: 1,38 GB reserved, 1,32 GB ( 96%) committed.
Stock VM:
Virtual space:
Non-class space: 832,00 MB reserved, 822,44 MB ( 99%) committed, 13 nodes.
Class space: 1,00 GB reserved, 521,31 MB ( 51%) committed, 1 nodes.
Both: 1,81 GB reserved, 1,31 GB ( 72%) committed.
----
Patch:
https://github.com/openjdk/lilliput/pull/13 introduced the notion that every MetaspaceArena has an alignment - instead, as in the stock VM - working with one global alignment. This patch reverts much of this code (makes it easier to merge too). Instead, it treats alignment as an allocation option. An arena can satisfy allocations with different alignment requirements and will do its best to salvage the alignment waste.
That is the core of the patch. The rest is fluff and tests.
---
Notes:
Numbers are somewhat fluid: at the moment, we use a Klass alignment of 512 and a nKlass size of 21 bits. We will probably change nKlass width to 24 or 26 bits for Lilliputs first milestone.
And once this patch is in, the disadvantage of memory waste is eliminated, so we could increase the Klass alignment. Both 512 and 1K are good values: on average a Klass is between 400b...1K. A 1K alignment would spread out the nKlass value points more. Shift the nKlass-value-point-to-class ratio closer to 1:1.
The still unsolved problem is naive overalignment. We may change the encoding scheme of Klass*->nKlass to take that into account. But not in this patch. But this uncertainty is the reason why I opted for a new method `MetaspaceArena::allocate_for_klass` instead of just giving the existing `Metaspace::allocate` method an alignment parameter.
-------------
Commit messages:
- reuse-class-space-alignment-gaps
Changes: https://git.openjdk.org/lilliput/pull/86/files
Webrev: https://webrevs.openjdk.org/?repo=lilliput&pr=86&range=00
Issue: https://bugs.openjdk.org/browse/JDK-8307002
Stats: 422 lines in 18 files changed: 321 ins; 40 del; 61 mod
Patch: https://git.openjdk.org/lilliput/pull/86.diff
Fetch: git fetch https://git.openjdk.org/lilliput.git pull/86/head:pull/86
PR: https://git.openjdk.org/lilliput/pull/86
From stuefe at openjdk.org Fri Apr 28 07:47:26 2023
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Fri, 28 Apr 2023 07:47:26 GMT
Subject: [master] RFR: Merge jdk:jdk-21+20
Message-ID: <8NjUikY2GD94qTUM43aad4Q2fkQHda8_Ok7R_BoC0Q0=.da5cd8ee-b283-4715-9c6a-05a69f848aed@github.com>
Merge jdk21+20
-------------
Commit messages:
- Fix Minimal
- Take over newest changes from upstream 8305896: Alternative full GC forwarding
- Move stub generation for check_lock_stack to initial stubs
- Merge jdk-21+20
- 8305771: SA ClassWriter.java fails to skip overpass methods
- 8306951: [BACKOUT] JDK-8305252 make_method_handle_intrinsic may call java code under a lock
- 8306409: Open source AWT KeyBoardFocusManger, LightWeightComponent related tests
- 8306705: com/sun/jdi/PopAndInvokeTest.java fails with NativeMethodException
- 8302182: Update Public Suffix List to 88467c9
- 8305853: java/text/Format/DateFormat/DateFormatRegression.java fails with "Uncaught exception thrown in test method Test4089106"
- ... and 728 more: https://git.openjdk.org/lilliput/compare/e12ed8a2...e42fae8f
The webrevs contain the adjustments done while merging with regards to each parent branch:
- master: https://webrevs.openjdk.org/?repo=lilliput&pr=87&range=00.0
- jdk:jdk-21+20: https://webrevs.openjdk.org/?repo=lilliput&pr=87&range=00.1
Changes: https://git.openjdk.org/lilliput/pull/87/files
Stats: 502200 lines in 5003 files changed: 393483 ins; 71533 del; 37184 mod
Patch: https://git.openjdk.org/lilliput/pull/87.diff
Fetch: git fetch https://git.openjdk.org/lilliput.git pull/87/head:pull/87
PR: https://git.openjdk.org/lilliput/pull/87
From stuefe at openjdk.org Fri Apr 28 11:50:23 2023
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Fri, 28 Apr 2023 11:50:23 GMT
Subject: [master] RFR: Merge jdk:jdk-21+20 [v2]
In-Reply-To: <8NjUikY2GD94qTUM43aad4Q2fkQHda8_Ok7R_BoC0Q0=.da5cd8ee-b283-4715-9c6a-05a69f848aed@github.com>
References: <8NjUikY2GD94qTUM43aad4Q2fkQHda8_Ok7R_BoC0Q0=.da5cd8ee-b283-4715-9c6a-05a69f848aed@github.com>
Message-ID: <K4IsEtQRiHidXrgIbZ4ddRgOzUU2GIke1A_KyTeidos=.bad11626-0cdd-4c68-8209-efa249d78928@github.com>
> Merge jdk21+20
Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:
shenandoah-fixes
-------------
Changes:
- all: https://git.openjdk.org/lilliput/pull/87/files
- new: https://git.openjdk.org/lilliput/pull/87/files/e42fae8f..8dd88c38
Webrevs:
- full: https://webrevs.openjdk.org/?repo=lilliput&pr=87&range=01
- incr: https://webrevs.openjdk.org/?repo=lilliput&pr=87&range=00-01
Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
Patch: https://git.openjdk.org/lilliput/pull/87.diff
Fetch: git fetch https://git.openjdk.org/lilliput.git pull/87/head:pull/87
PR: https://git.openjdk.org/lilliput/pull/87
From stuefe at openjdk.org Fri Apr 28 15:30:23 2023
From: stuefe at openjdk.org (Thomas Stuefe)
Date: Fri, 28 Apr 2023 15:30:23 GMT
Subject: [master] RFR: Merge jdk:jdk-21+20 [v3]
In-Reply-To: <8NjUikY2GD94qTUM43aad4Q2fkQHda8_Ok7R_BoC0Q0=.da5cd8ee-b283-4715-9c6a-05a69f848aed@github.com>
References: <8NjUikY2GD94qTUM43aad4Q2fkQHda8_Ok7R_BoC0Q0=.da5cd8ee-b283-4715-9c6a-05a69f848aed@github.com>
Message-ID: <ZPgmnpfUffW367hXl5PmmzRqpkCOWFl71Lqe48QI8tA=.ebfa8678-8ea3-4aab-9ebc-9fc3874c413e@github.com>
> Merge jdk21+20
Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:
Fix jtreg tests
-------------
Changes:
- all: https://git.openjdk.org/lilliput/pull/87/files
- new: https://git.openjdk.org/lilliput/pull/87/files/8dd88c38..a27f147b
Webrevs:
- full: https://webrevs.openjdk.org/?repo=lilliput&pr=87&range=02
- incr: https://webrevs.openjdk.org/?repo=lilliput&pr=87&range=01-02
Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
Patch: https://git.openjdk.org/lilliput/pull/87.diff
Fetch: git fetch https://git.openjdk.org/lilliput.git pull/87/head:pull/87
PR: https://git.openjdk.org/lilliput/pull/87
From rkennke at openjdk.org Fri Apr 28 15:43:56 2023
From: rkennke at openjdk.org (Roman Kennke)
Date: Fri, 28 Apr 2023 15:43:56 GMT
Subject: [master] RFR: Merge jdk:jdk-21+20 [v3]
In-Reply-To: <ZPgmnpfUffW367hXl5PmmzRqpkCOWFl71Lqe48QI8tA=.ebfa8678-8ea3-4aab-9ebc-9fc3874c413e@github.com>
References: <8NjUikY2GD94qTUM43aad4Q2fkQHda8_Ok7R_BoC0Q0=.da5cd8ee-b283-4715-9c6a-05a69f848aed@github.com>
<ZPgmnpfUffW367hXl5PmmzRqpkCOWFl71Lqe48QI8tA=.ebfa8678-8ea3-4aab-9ebc-9fc3874c413e@github.com>
Message-ID: <fsVIuY_4drN6j3Dh0ikmWY97vz_JcC969QeQogsV2oE=.0f2afdc4-4fb2-4d21-aea8-67a2d91d564a@github.com>
On Fri, 28 Apr 2023 15:30:23 GMT, Thomas Stuefe <stuefe at openjdk.org> wrote:
>> Merge jdk21+20
>
> Thomas Stuefe has updated the pull request incrementally with one additional commit since the last revision:
>
> Fix jtreg tests
Let's wait until GHA are green, and then push it!
Thanks for doing this!
-------------
Marked as reviewed by rkennke (Lead).
PR Review: https://git.openjdk.org/lilliput/pull/87#pullrequestreview-1406191175