[lilliput-jdk17u:lilliput] RFR: 8303027: [Lilliput/JDK17] Correctly resolve forwarded objects in G1 heap iteration [v9]

[Roman Kennke]

On Thu, 23 Feb 2023 16:07:33 GMT, Roman Kennke <rkennke at openjdk.org> wrote:

>> A user provided a (pretty simple) test case that makes the VM crash with Lilliput, when it should exit (somewhat more) gracefully with an OOME. The reason for the crash is that in G1, object_iterate() (or rather, the block_size() method that it calls) does not correctly resolve possibly forwarded objects, and then crashes because it accesses the Klass*, which is overridden by the forwarding pointer.
>> 
>> The trouble is that, we don't need (or even can't) resolve forwarded objects during full-GC, because there we are using sliding-forwarding which is designed precisely to preserve the object's Klass* while being forwarded. Outside of full-GC we don't need to preserve the Klass* because the Klass* is preserved in the forwarded copy.
>> 
>> This bug only exists in Lilliput/JDK17, the code path that leads to object_iterate() (restoring self-forwarded objects at evac failure) doesn't exist anymore in later versions - it uses a better way to iterate over the relevant self-forwarded objects.
>> 
>> The proposed fix is to do two different loops, one that resolves the forwardees when needed, and the other one that doesn't. The alternative would have to check for the condition in_full_gc() on every object being scanned, which would affect performance. I included the test case as jtreg test.
>> 
>> Testing:
>>  - [x] runtime/oom/TestOOM.java
>>  - [x] tier1
>>  - [ ] tier2
>
> Roman Kennke has updated the pull request incrementally with one additional commit since the last revision:
> 
>   Separate TestOOM tests for 32 bits and 64 bits

It finally passes all GHA except the x-compiles, which is expected. Let's

-------------

PR: https://git.openjdk.org/lilliput-jdk17u/pull/6