RFR: 8301988: VerifyLiveClosure::verify_liveness asserts on bad pointers outside heap [v2]

[Thomas Schatzl]

On Wed, 8 Feb 2023 02:26:26 GMT, David Holmes <dholmes at openjdk.org> wrote:

>> Thomas Schatzl has updated the pull request incrementally with one additional commit since the last revision:
>> 
>>   Missing changes
>
> src/hotspot/share/gc/g1/g1CollectedHeap.inline.hpp line 217:
> 
>> 215: 
>> 216: inline bool G1CollectedHeap::is_obj_filler(const oop obj) {
>> 217:   Klass* k = obj->klass_raw();
> 
> Not clear how you can get here from ` HeapRegion::is_obj_dead` with a bad oop, such that you need the raw variant. ??

The object is in the heap, but the occupying memory has already been zapped (in debug mode); i.e. the call in `heapRegion.cpp:518` could read `badHeapWordVal` as (compressed) klass value in the header.

In that case the current code asserts in this call because in `oopDesc::klass()`, the call to `CompressedKlassPointers::decode_not_null` will assert in `compressedOops.inline.hpp:135` due to the `check_alignment` condition not satisfied.

This makes this verification code assert before printing out any useful information to diagnose the problem quickly (in my case this has been a change that wrongly managed remembered sets).

If I had had the remembered set verification printout, I would have found the problem immediately in this case (because the message would have told me that there is a problem with remembered sets). So it took a while to diagnose the issue, having to go into the debugger to painfully find the exact same information.

I.e. this makes the verification code more robust.

Imo the suggested solution to just continue execution is fine, because `is_obj_filler` will always return false (i.e. object is dead) for garbage objects and do the right thing here.
There is the concern that now other non-verification code might not immediately trigger now, but most of it just fails the VM anyway if it finds a bad reference (after printing some information about it), for all other cases this is the right choice.

-------------

PR: https://git.openjdk.org/jdk/pull/12456